Skip to contentSkip to conversation

KVM: VMX: Rip out "deferred nested VM-Exit updates"

PATCH Applied Sean Christopherson 1 participant 10 messages Jan 9 – Jan 15

Subsystems (4)
lore
Share & tools
lore

Sean Christopherson

author maintainer
#0 lore raw

v4 cover KVM: VMX: Rip out "deferred nested VM-Exit updates"

v4 of what was "KVM: VMX: Fix APICv activation bugs", but the two bug fixes were already merged and so that name doesn't fit.

Rip out the "defer updates until nested VM-Exit" that contributed to the APICv bugs and made them harder to fix.

v4:
- Collect reviews. [Chao]
- Use the existing vmcall() in the selftest. [Chao]
- Use GUEST_FAIL() instead of TEST_FAIL() in the guest. [Chao]
- Correct the number of args passed to the guest in the test. [Chao]
- Remove the redundant hwapic_isr_update() call in kvm_lapic_reset()
- Use the local "vmx" variable in vmx_set_virtual_apic_mode(). [Chao]
- Relocate the "update SVI" comment from __kvm_vcpu_update_apicv() to kvm_apic_update_apicv().

v3:
- https://lore.kernel.org/all/[email protected]
- Add a selftest.
- Rip out the deferred updates stuff.
- Collect Chao's review.
- Add Dongli's fix for bug #2. [Chao]

v2:
- https://lore.kernel.org/all/[email protected]
- Add support for guest mode (suggested by Chao Gao).
- Add comments in the code (suggested by Chao Gao).
- Remove WARN_ON_ONCE from vmx_hwapic_isr_update().
- Edit commit message "AMD SVM APICv" to "AMD SVM AVIC" (suggested by Alejandro Jimenez).

Sean Christopherson (8):
KVM: selftests: Add a test to verify APICv updates (while L2 is
active)
KVM: nVMX: Switch to vmcs01 to update PML controls on-demand if L2 is
active
KVM: nVMX: Switch to vmcs01 to update TPR threshold on-demand if L2 is
active
KVM: nVMX: Switch to vmcs01 to update SVI on-demand if L2 is active
KVM: nVMX: Switch to vmcs01 to refresh APICv controls on-demand if L2
is active
KVM: nVMX: Switch to vmcs01 to update APIC page on-demand if L2 is
active
KVM: nVMX: Switch to vmcs01 to set virtual APICv mode on-demand if L2
is active
KVM: x86: Update APICv ISR (a.k.a. SVI) as part of
kvm_apic_update_apicv()

 arch/x86/kvm/lapic.c                          |  31 ++--
 arch/x86/kvm/lapic.h                          |   1 -
 arch/x86/kvm/vmx/nested.c                     |  29 ----
 arch/x86/kvm/vmx/vmx.c                        |  95 +++++++----
 arch/x86/kvm/vmx/vmx.h                        |   9 -
 arch/x86/kvm/x86.c                            |   7 -
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../testing/selftests/kvm/include/x86/apic.h  |   4 +
 .../kvm/x86/vmx_apicv_updates_test.c          | 155 ++++++++++++++++++
 9 files changed, 232 insertions(+), 100 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/x86/vmx_apicv_updates_test.c

base-commit: 9448598b22c50c8a5bb77a9103e2d49f134c9578

Signature · 1 line
2.52.0.457.g6b5491de43-goog
author maintainer
#1 lore raw

PATCH v4 1/8 KVM: selftests: Add a test to verify APICv updates (while L2 is active)

Add a test to verify KVM correctly handles a variety of edge cases related to APICv updates, and in particular updates that are triggered while L2 is actively running.

---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../testing/selftests/kvm/include/x86/apic.h  |   4 +
 .../kvm/x86/vmx_apicv_updates_test.c          | 155 ++++++++++++++++++
 3 files changed, 160 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/x86/vmx_apicv_updates_test.c
tools/testing/selftests/kvm/Makefile.kvm +1 -0
index ba5c2b643efa..6f00bd8271c2 100644
@@ -115,6 +115,7 @@ TEST_GEN_PROGS_x86 += x86/ucna_injection_test
 TEST_GEN_PROGS_x86 += x86/userspace_io_test
 TEST_GEN_PROGS_x86 += x86/userspace_msr_exit_test
 TEST_GEN_PROGS_x86 += x86/vmx_apic_access_test
+TEST_GEN_PROGS_x86 += x86/vmx_apicv_updates_test
 TEST_GEN_PROGS_x86 += x86/vmx_dirty_log_test
 TEST_GEN_PROGS_x86 += x86/vmx_exception_with_invalid_guest_state
 TEST_GEN_PROGS_x86 += x86/vmx_msrs_test
tools/testing/selftests/kvm/include/x86/apic.h +4 -0
index 80fe9f69b38d..d42a0998d868 100644
@@ -32,6 +32,7 @@
 #define	APIC_SPIV	0xF0
 #define		APIC_SPIV_FOCUS_DISABLED	(1 << 9)
 #define		APIC_SPIV_APIC_ENABLED		(1 << 8)
+#define	APIC_ISR	0x100
 #define APIC_IRR	0x200
 #define	APIC_ICR	0x300
 #define	APIC_LVTCMCI	0x2f0
@@ -68,6 +69,9 @@
 #define	APIC_TMCCT	0x390
 #define	APIC_TDCR	0x3E0
 
+#define APIC_VECTOR_TO_BIT_NUMBER(v) ((unsigned int)(v) % 32)
+#define APIC_VECTOR_TO_REG_OFFSET(v) ((unsigned int)(v) / 32 * 0x10)
+
 void apic_disable(void);
 void xapic_enable(void);
 void x2apic_enable(void);
tools/testing/selftests/kvm/x86/vmx_apicv_updates_test.c new file +155 -0

155 lines not on this page · raw message

Signature · 1 line
2.52.0.457.g6b5491de43-goog
author maintainer
#2 lore raw

PATCH v4 2/8 KVM: nVMX: Switch to vmcs01 to update PML controls on-demand if L2 is active

In reply to Sean Christopherson

If KVM toggles "CPU dirty logging", a.k.a. Page-Modification Logging (PML), while L2 is active, temporarily load vmcs01 and immediately update the relevant controls instead of deferring the update until the next nested VM-Exit. For PML, deferring the update is relatively straightforward, but for several APICv related updates, deferring updates creates ordering and state consistency problems, e.g. KVM at-large thinks APICv is enabled, but vmcs01 is still running with stale (and effectively unknown) state.

Convert PML first precisely because it's the simplest case to handle: if something is broken with the vmcs01 <=> vmcs02 dance, then hopefully bugs will bisect here.

---
 arch/x86/kvm/vmx/nested.c |  5 -----
 arch/x86/kvm/vmx/vmx.c    | 40 +++++++++++++++++++++++++++++++++++----
 arch/x86/kvm/vmx/vmx.h    |  1 -
 3 files changed, 36 insertions(+), 10 deletions(-)
arch/x86/kvm/vmx/nested.c +0 -5
index 6137e5307d0f..920a925bb46f 100644
@@ -5152,11 +5152,6 @@ void __nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,
 		vmx_set_virtual_apic_mode(vcpu);
 	}
 
-	if (vmx->nested.update_vmcs01_cpu_dirty_logging) {
-		vmx->nested.update_vmcs01_cpu_dirty_logging = false;
-		vmx_update_cpu_dirty_logging(vcpu);
-	}
-
 	nested_put_vmcs12_pages(vcpu);
 
 	if (vmx->nested.reload_vmcs01_apic_access_page) {
arch/x86/kvm/vmx/vmx.c +36 -4
index 6b96f7aea20b..1420665fbb66 100644
@@ -1594,6 +1594,41 @@ void vmx_vcpu_put(struct kvm_vcpu *vcpu)
 	vmx_prepare_switch_to_host(to_vmx(vcpu));
 }
 
+static void vmx_switch_loaded_vmcs(struct kvm_vcpu *vcpu,
+				   struct loaded_vmcs *vmcs)
+{
+	struct vcpu_vmx *vmx = to_vmx(vcpu);
+	int cpu;
+
+	cpu = get_cpu();
+	vmx->loaded_vmcs = vmcs;
+	vmx_vcpu_load_vmcs(vcpu, cpu);
+	put_cpu();
+}
+
+static void vmx_load_vmcs01(struct kvm_vcpu *vcpu)
+{
+	struct vcpu_vmx *vmx = to_vmx(vcpu);
+
+	if (!is_guest_mode(vcpu)) {
+		WARN_ON_ONCE(vmx->loaded_vmcs != &vmx->vmcs01);
+		return;
+	}
+
+	WARN_ON_ONCE(vmx->loaded_vmcs != &vmx->nested.vmcs02);
+	vmx_switch_loaded_vmcs(vcpu, &vmx->vmcs01);
+}
+
+static void vmx_put_vmcs01(struct kvm_vcpu *vcpu)
+{
+	if (!is_guest_mode(vcpu))
+		return;
+
+	vmx_switch_loaded_vmcs(vcpu, &to_vmx(vcpu)->nested.vmcs02);
+}
+DEFINE_GUARD(vmx_vmcs01, struct kvm_vcpu *,
+	     vmx_load_vmcs01(_T), vmx_put_vmcs01(_T))
+
 bool vmx_emulation_required(struct kvm_vcpu *vcpu)
 {
 	return emulate_invalid_guest_state && !vmx_guest_state_valid(vcpu);
@@ -8267,10 +8302,7 @@ void vmx_update_cpu_dirty_logging(struct kvm_vcpu *vcpu)
 	if (WARN_ON_ONCE(!enable_pml))
 		return;
 
-	if (is_guest_mode(vcpu)) {
-		vmx->nested.update_vmcs01_cpu_dirty_logging = true;
-		return;
-	}
+	guard(vmx_vmcs01)(vcpu);
 
 	/*
 	 * Note, nr_memslots_dirty_logging can be changed concurrent with this
arch/x86/kvm/vmx/vmx.h +0 -1
index bc3ed3145d7e..b44eda6225f4 100644
@@ -133,7 +133,6 @@ struct nested_vmx {
 
 	bool change_vmcs01_virtual_apic_mode;
 	bool reload_vmcs01_apic_access_page;
-	bool update_vmcs01_cpu_dirty_logging;
 	bool update_vmcs01_apicv_status;
 	bool update_vmcs01_hwapic_isr;
 
Signature · 1 line
2.52.0.457.g6b5491de43-goog
author maintainer
#3 lore raw

PATCH v4 3/8 KVM: nVMX: Switch to vmcs01 to update TPR threshold on-demand if L2 is active

In reply to Sean Christopherson

If KVM updates L1's TPR Threshold while L2 is active, temporarily load vmcs01 and immediately update TPR_THRESHOLD instead of deferring the update until the next nested VM-Exit. Deferring the TPR Threshold update is relatively straightforward, but for several APICv related updates, deferring updates creates ordering and state consistency problems, e.g. KVM at-large thinks APICv is enabled, but vmcs01 is still running with stale (and effectively unknown) state.

---
 arch/x86/kvm/vmx/nested.c | 4 ----
 arch/x86/kvm/vmx/vmx.c    | 7 +++----
 arch/x86/kvm/vmx/vmx.h    | 3 ---
 3 files changed, 3 insertions(+), 11 deletions(-)
arch/x86/kvm/vmx/nested.c +0 -4
index 920a925bb46f..8efab1cf833f 100644
@@ -2402,7 +2402,6 @@ static void prepare_vmcs02_early(struct vcpu_vmx *vmx, struct loaded_vmcs *vmcs0
 	exec_control &= ~CPU_BASED_TPR_SHADOW;
 	exec_control |= vmcs12->cpu_based_vm_exec_control;
 
-	vmx->nested.l1_tpr_threshold = -1;
 	if (exec_control & CPU_BASED_TPR_SHADOW)
 		vmcs_write32(TPR_THRESHOLD, vmcs12->tpr_threshold);
 #ifdef CONFIG_X86_64
@@ -5144,9 +5143,6 @@ void __nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,
 	if (kvm_caps.has_tsc_control)
 		vmcs_write64(TSC_MULTIPLIER, vcpu->arch.tsc_scaling_ratio);
 
-	if (vmx->nested.l1_tpr_threshold != -1)
-		vmcs_write32(TPR_THRESHOLD, vmx->nested.l1_tpr_threshold);
-
 	if (vmx->nested.change_vmcs01_virtual_apic_mode) {
 		vmx->nested.change_vmcs01_virtual_apic_mode = false;
 		vmx_set_virtual_apic_mode(vcpu);
arch/x86/kvm/vmx/vmx.c +3 -4
index 1420665fbb66..3ee86665d8de 100644
@@ -6827,11 +6827,10 @@ void vmx_update_cr8_intercept(struct kvm_vcpu *vcpu, int tpr, int irr)
 		nested_cpu_has(vmcs12, CPU_BASED_TPR_SHADOW))
 		return;
 
+	guard(vmx_vmcs01)(vcpu);
+
 	tpr_threshold = (irr == -1 || tpr < irr) ? 0 : irr;
-	if (is_guest_mode(vcpu))
-		to_vmx(vcpu)->nested.l1_tpr_threshold = tpr_threshold;
-	else
-		vmcs_write32(TPR_THRESHOLD, tpr_threshold);
+	vmcs_write32(TPR_THRESHOLD, tpr_threshold);
 }
 
 void vmx_set_virtual_apic_mode(struct kvm_vcpu *vcpu)
arch/x86/kvm/vmx/vmx.h +0 -3
index b44eda6225f4..36f48c4b39c0 100644
@@ -184,9 +184,6 @@ struct nested_vmx {
 	u64 pre_vmenter_ssp;
 	u64 pre_vmenter_ssp_tbl;
 
-	/* to migrate it to L1 if L2 writes to L1's CR8 directly */
-	int l1_tpr_threshold;
-
 	u16 vpid02;
 	u16 last_vpid;
 
Signature · 1 line
2.52.0.457.g6b5491de43-goog
author maintainer
#4 lore raw

PATCH v4 4/8 KVM: nVMX: Switch to vmcs01 to update SVI on-demand if L2 is active

In reply to Sean Christopherson

If APICv is activated while L2 is running and triggers an SVI update, temporarily load vmcs01 and immediately update SVI instead of deferring the update until the next nested VM-Exit. This will eventually allow killing off kvm_apic_update_hwapic_isr(), and all of nVMX's deferred APICv updates.

---
 arch/x86/kvm/vmx/nested.c |  5 -----
 arch/x86/kvm/vmx/vmx.c    | 19 +++++++------------
 arch/x86/kvm/vmx/vmx.h    |  1 -
 3 files changed, 7 insertions(+), 18 deletions(-)
arch/x86/kvm/vmx/nested.c +0 -5
index 8efab1cf833f..c2c96e4fe20e 100644
@@ -5160,11 +5160,6 @@ void __nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,
 		vmx_refresh_apicv_exec_ctrl(vcpu);
 	}
 
-	if (vmx->nested.update_vmcs01_hwapic_isr) {
-		vmx->nested.update_vmcs01_hwapic_isr = false;
-		kvm_apic_update_hwapic_isr(vcpu);
-	}
-
 	if ((vm_exit_reason != -1) &&
 	    (enable_shadow_vmcs || nested_vmx_is_evmptr12_valid(vmx)))
 		vmx->nested.need_vmcs12_to_shadow_sync = true;
arch/x86/kvm/vmx/vmx.c +7 -12
index 3ee86665d8de..74a815cddd37 100644
@@ -6963,21 +6963,16 @@ void vmx_hwapic_isr_update(struct kvm_vcpu *vcpu, int max_isr)
 	u16 status;
 	u8 old;
 
-	/*
-	 * If L2 is active, defer the SVI update until vmcs01 is loaded, as SVI
-	 * is only relevant for if and only if Virtual Interrupt Delivery is
-	 * enabled in vmcs12, and if VID is enabled then L2 EOIs affect L2's
-	 * vAPIC, not L1's vAPIC.  KVM must update vmcs01 on the next nested
-	 * VM-Exit, otherwise L1 with run with a stale SVI.
-	 */
-	if (is_guest_mode(vcpu)) {
-		to_vmx(vcpu)->nested.update_vmcs01_hwapic_isr = true;
-		return;
-	}
-
 	if (max_isr == -1)
 		max_isr = 0;
 
+	/*
+	 * Always update SVI in vmcs01, as SVI is only relevant for L2 if and
+	 * only if Virtual Interrupt Delivery is enabled in vmcs12, and if VID
+	 * is enabled then L2 EOIs affect L2's vAPIC, not L1's vAPIC.
+	 */
+	guard(vmx_vmcs01)(vcpu);
+
 	status = vmcs_read16(GUEST_INTR_STATUS);
 	old = status >> 8;
 	if (max_isr != old) {
arch/x86/kvm/vmx/vmx.h +0 -1
index 36f48c4b39c0..53969e49d9d1 100644
@@ -134,7 +134,6 @@ struct nested_vmx {
 	bool change_vmcs01_virtual_apic_mode;
 	bool reload_vmcs01_apic_access_page;
 	bool update_vmcs01_apicv_status;
-	bool update_vmcs01_hwapic_isr;
 
 	/*
 	 * Enlightened VMCS has been enabled. It does not mean that L1 has to
Signature · 1 line
2.52.0.457.g6b5491de43-goog
author maintainer
#5 lore raw

PATCH v4 5/8 KVM: nVMX: Switch to vmcs01 to refresh APICv controls on-demand if L2 is active

In reply to Sean Christopherson

If APICv is (un)inhibited while L2 is running, temporarily load vmcs01 and immediately refresh the APICv controls in vmcs01 instead of deferring the update until the next nested VM-Exit. This all but eliminates potential ordering issues due to vmcs01 not being synchronized with kvm_lapic.apicv_active, e.g. where KVM _thinks_ it refreshed APICv, but vmcs01 still contains stale state.

---
 arch/x86/kvm/vmx/nested.c | 5 -----
 arch/x86/kvm/vmx/vmx.c    | 5 +----
 arch/x86/kvm/vmx/vmx.h    | 1 -
 3 files changed, 1 insertion(+), 10 deletions(-)
arch/x86/kvm/vmx/nested.c +0 -5
index c2c96e4fe20e..2b0702349aa1 100644
@@ -5155,11 +5155,6 @@ void __nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,
 		kvm_make_request(KVM_REQ_APIC_PAGE_RELOAD, vcpu);
 	}
 
-	if (vmx->nested.update_vmcs01_apicv_status) {
-		vmx->nested.update_vmcs01_apicv_status = false;
-		vmx_refresh_apicv_exec_ctrl(vcpu);
-	}
-
 	if ((vm_exit_reason != -1) &&
 	    (enable_shadow_vmcs || nested_vmx_is_evmptr12_valid(vmx)))
 		vmx->nested.need_vmcs12_to_shadow_sync = true;
arch/x86/kvm/vmx/vmx.c +1 -4
index 74a815cddd37..90e167f296d0 100644
@@ -4578,10 +4578,7 @@ void vmx_refresh_apicv_exec_ctrl(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
 
-	if (is_guest_mode(vcpu)) {
-		vmx->nested.update_vmcs01_apicv_status = true;
-		return;
-	}
+	guard(vmx_vmcs01)(vcpu);
 
 	pin_controls_set(vmx, vmx_pin_based_exec_ctrl(vmx));
 
arch/x86/kvm/vmx/vmx.h +0 -1
index 53969e49d9d1..dfc9766a7fa3 100644
@@ -133,7 +133,6 @@ struct nested_vmx {
 
 	bool change_vmcs01_virtual_apic_mode;
 	bool reload_vmcs01_apic_access_page;
-	bool update_vmcs01_apicv_status;
 
 	/*
 	 * Enlightened VMCS has been enabled. It does not mean that L1 has to
Signature · 1 line
2.52.0.457.g6b5491de43-goog
author maintainer
#6 lore raw

PATCH v4 6/8 KVM: nVMX: Switch to vmcs01 to update APIC page on-demand if L2 is active

In reply to Sean Christopherson

If the KVM-owned APIC-access page is migrated while L2 is running, temporarily load vmcs01 and immediately update APIC_ACCESS_ADDR instead of deferring the update until the next nested VM-Exit. Once changing the virtual APIC mode is converted to always do on-demand updates, all of the "defer until vmcs01 is active" logic will be gone.

---
 arch/x86/kvm/vmx/nested.c | 5 -----
 arch/x86/kvm/vmx/vmx.c    | 7 ++-----
 arch/x86/kvm/vmx/vmx.h    | 1 -
 3 files changed, 2 insertions(+), 11 deletions(-)
arch/x86/kvm/vmx/nested.c +0 -5
index 2b0702349aa1..8196a1ac22e1 100644
@@ -5150,11 +5150,6 @@ void __nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,
 
 	nested_put_vmcs12_pages(vcpu);
 
-	if (vmx->nested.reload_vmcs01_apic_access_page) {
-		vmx->nested.reload_vmcs01_apic_access_page = false;
-		kvm_make_request(KVM_REQ_APIC_PAGE_RELOAD, vcpu);
-	}
-
 	if ((vm_exit_reason != -1) &&
 	    (enable_shadow_vmcs || nested_vmx_is_evmptr12_valid(vmx)))
 		vmx->nested.need_vmcs12_to_shadow_sync = true;
arch/x86/kvm/vmx/vmx.c +2 -5
index 90e167f296d0..af8ec72e8ebf 100644
@@ -6895,11 +6895,8 @@ void vmx_set_apic_access_page_addr(struct kvm_vcpu *vcpu)
 	kvm_pfn_t pfn;
 	bool writable;
 
-	/* Defer reload until vmcs01 is the current VMCS. */
-	if (is_guest_mode(vcpu)) {
-		to_vmx(vcpu)->nested.reload_vmcs01_apic_access_page = true;
-		return;
-	}
+	/* Note, the VIRTUALIZE_APIC_ACCESSES check needs to query vmcs01. */
+	guard(vmx_vmcs01)(vcpu);
 
 	if (!(secondary_exec_controls_get(to_vmx(vcpu)) &
 	    SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES))
arch/x86/kvm/vmx/vmx.h +0 -1
index dfc9766a7fa3..078bc6fef7e6 100644
@@ -132,7 +132,6 @@ struct nested_vmx {
 	bool vmcs02_initialized;
 
 	bool change_vmcs01_virtual_apic_mode;
-	bool reload_vmcs01_apic_access_page;
 
 	/*
 	 * Enlightened VMCS has been enabled. It does not mean that L1 has to
Signature · 1 line
2.52.0.457.g6b5491de43-goog
author maintainer
#7 lore raw

PATCH v4 7/8 KVM: nVMX: Switch to vmcs01 to set virtual APICv mode on-demand if L2 is active

In reply to Sean Christopherson

If L1's virtual APIC mode changes while L2 is active, e.g. because L1 doesn't intercept writes to the APIC_BASE MSR and L2 changes the mode, temporarily load vmcs01 and do all of the necessary actions instead of deferring the update until the next nested VM-Exit.

This will help in fixing yet more issues related to updates while L2 is active, e.g. KVM neglects to update vmcs02 MSR intercepts if vmcs01's MSR intercepts are modified while L2 is active. Not updating x2APIC MSRs is benign because vmcs01's settings are not factored into vmcs02's bitmap, but deferring the x2APIC MSR updates would create a weird, inconsistent state.

---
 arch/x86/kvm/vmx/nested.c |  5 -----
 arch/x86/kvm/vmx/vmx.c    | 17 +++++++++++------
 arch/x86/kvm/vmx/vmx.h    |  2 --
 3 files changed, 11 insertions(+), 13 deletions(-)
arch/x86/kvm/vmx/nested.c +0 -5
index 8196a1ac22e1..b99e3c80d43e 100644
@@ -5143,11 +5143,6 @@ void __nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,
 	if (kvm_caps.has_tsc_control)
 		vmcs_write64(TSC_MULTIPLIER, vcpu->arch.tsc_scaling_ratio);
 
-	if (vmx->nested.change_vmcs01_virtual_apic_mode) {
-		vmx->nested.change_vmcs01_virtual_apic_mode = false;
-		vmx_set_virtual_apic_mode(vcpu);
-	}
-
 	nested_put_vmcs12_pages(vcpu);
 
 	if ((vm_exit_reason != -1) &&
arch/x86/kvm/vmx/vmx.c +11 -6
index af8ec72e8ebf..54701bb815eb 100644
@@ -6842,11 +6842,7 @@ void vmx_set_virtual_apic_mode(struct kvm_vcpu *vcpu)
 	    !cpu_has_vmx_virtualize_x2apic_mode())
 		return;
 
-	/* Postpone execution until vmcs01 is the current VMCS. */
-	if (is_guest_mode(vcpu)) {
-		vmx->nested.change_vmcs01_virtual_apic_mode = true;
-		return;
-	}
+	guard(vmx_vmcs01)(vcpu);
 
 	sec_exec_control = secondary_exec_controls_get(vmx);
 	sec_exec_control &= ~(SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES |
@@ -6869,8 +6865,17 @@ void vmx_set_virtual_apic_mode(struct kvm_vcpu *vcpu)
 			 * only do so if its physical address has changed, but
 			 * the guest may have inserted a non-APIC mapping into
 			 * the TLB while the APIC access page was disabled.
+			 *
+			 * If L2 is active, immediately flush L1's TLB instead
+			 * of requesting a flush of the current TLB, because
+			 * the current TLB context is L2's.
 			 */
-			kvm_make_request(KVM_REQ_TLB_FLUSH_CURRENT, vcpu);
+			if (!is_guest_mode(vcpu))
+				kvm_make_request(KVM_REQ_TLB_FLUSH_CURRENT, vcpu);
+			else if (!enable_ept)
+				vpid_sync_context(vmx->vpid);
+			else if (VALID_PAGE(vcpu->arch.root_mmu.root.hpa))
+				vmx_flush_tlb_ept_root(vcpu->arch.root_mmu.root.hpa);
 		}
 		break;
 	case LAPIC_MODE_X2APIC:
arch/x86/kvm/vmx/vmx.h +0 -2
index 078bc6fef7e6..a926ce43ad40 100644
@@ -131,8 +131,6 @@ struct nested_vmx {
 	 */
 	bool vmcs02_initialized;
 
-	bool change_vmcs01_virtual_apic_mode;
-
 	/*
 	 * Enlightened VMCS has been enabled. It does not mean that L1 has to
 	 * use it. However, VMX features available to L1 will be limited based
Signature · 1 line
2.52.0.457.g6b5491de43-goog
author maintainer
#8 lore raw

PATCH v4 8/8 KVM: x86: Update APICv ISR (a.k.a. SVI) as part of kvm_apic_update_apicv()

In reply to Sean Christopherson

Fold the calls to .hwapic_isr_update() in kvm_apic_set_state(), kvm_lapic_reset(), and __kvm_vcpu_update_apicv() into kvm_apic_update_apicv(), as updating SVI is directly related to updating KVM's own cache of ISR information, e.g. SVI is more or less the APICv equivalent of highest_isr_cache.

Note, calling .hwapic_isr_update() during kvm_apic_update_apicv() has benign side effects, as doing so changes the orders of the calls in kvm_lapic_reset() and kvm_apic_set_state(), specifically with respect to to the order between .hwapic_isr_update() and .apicv_post_state_restore(). However, the changes in ordering are glorified nops as the former hook is VMX-only and the latter is SVM-only.

---
 arch/x86/kvm/lapic.c | 31 ++++++++++++-------------------
 arch/x86/kvm/lapic.h |  1 -
 arch/x86/kvm/x86.c   |  7 -------
 3 files changed, 12 insertions(+), 27 deletions(-)
arch/x86/kvm/lapic.c +12 -19
index 1597dd0b0cc6..9b791e728ec1 100644
@@ -770,17 +770,6 @@ static inline void apic_clear_isr(int vec, struct kvm_lapic *apic)
 	}
 }
 
-void kvm_apic_update_hwapic_isr(struct kvm_vcpu *vcpu)
-{
-	struct kvm_lapic *apic = vcpu->arch.apic;
-
-	if (WARN_ON_ONCE(!lapic_in_kernel(vcpu)) || !apic->apicv_active)
-		return;
-
-	kvm_x86_call(hwapic_isr_update)(vcpu, apic_find_highest_isr(apic));
-}
-EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_apic_update_hwapic_isr);
-
 int kvm_lapic_find_highest_irr(struct kvm_vcpu *vcpu)
 {
 	/* This may race with setting of irr in __apic_accept_irq() and
@@ -2785,10 +2774,18 @@ void kvm_apic_update_apicv(struct kvm_vcpu *vcpu)
 	 */
 	apic->irr_pending = true;
 
-	if (apic->apicv_active)
+	/*
+	 * Update SVI when APICv gets enabled, otherwise SVI won't reflect the
+	 * highest bit in vISR and the next accelerated EOI in the guest won't
+	 * be virtualized correctly (the CPU uses SVI to determine which vISR
+	 * vector to clear).
+	 */
+	if (apic->apicv_active) {
 		apic->isr_count = 1;
-	else
+		kvm_x86_call(hwapic_isr_update)(vcpu, apic_find_highest_isr(apic));
+	} else {
 		apic->isr_count = count_vectors(apic->regs + APIC_ISR);
+	}
 
 	apic->highest_isr_cache = -1;
 }
@@ -2916,10 +2913,8 @@ void kvm_lapic_reset(struct kvm_vcpu *vcpu, bool init_event)
 
 	vcpu->arch.pv_eoi.msr_val = 0;
 	apic_update_ppr(apic);
-	if (apic->apicv_active) {
+	if (apic->apicv_active)
 		kvm_x86_call(apicv_post_state_restore)(vcpu);
-		kvm_x86_call(hwapic_isr_update)(vcpu, -1);
-	}
 
 	vcpu->arch.apic_arb_prio = 0;
 	vcpu->arch.apic_attention = 0;
@@ -3232,10 +3227,8 @@ int kvm_apic_set_state(struct kvm_vcpu *vcpu, struct kvm_lapic_state *s)
 	__start_apic_timer(apic, APIC_TMCCT);
 	kvm_lapic_set_reg(apic, APIC_TMCCT, 0);
 	kvm_apic_update_apicv(vcpu);
-	if (apic->apicv_active) {
+	if (apic->apicv_active)
 		kvm_x86_call(apicv_post_state_restore)(vcpu);
-		kvm_x86_call(hwapic_isr_update)(vcpu, apic_find_highest_isr(apic));
-	}
 	kvm_make_request(KVM_REQ_EVENT, vcpu);
 
 #ifdef CONFIG_KVM_IOAPIC
arch/x86/kvm/lapic.h +0 -1
index 282b9b7da98c..aa0a9b55dbb7 100644
@@ -126,7 +126,6 @@ void kvm_apic_send_ipi(struct kvm_lapic *apic, u32 icr_low, u32 icr_high);
 int kvm_apic_set_base(struct kvm_vcpu *vcpu, u64 value, bool host_initiated);
 int kvm_apic_get_state(struct kvm_vcpu *vcpu, struct kvm_lapic_state *s);
 int kvm_apic_set_state(struct kvm_vcpu *vcpu, struct kvm_lapic_state *s);
-void kvm_apic_update_hwapic_isr(struct kvm_vcpu *vcpu);
 int kvm_lapic_find_highest_irr(struct kvm_vcpu *vcpu);
 
 u64 kvm_get_lapic_tscdeadline_msr(struct kvm_vcpu *vcpu);
arch/x86/kvm/x86.c +0 -7
index ff8812f3a129..0c6d899d53dd 100644
@@ -10886,16 +10886,9 @@ void __kvm_vcpu_update_apicv(struct kvm_vcpu *vcpu)
 	 * pending. At the same time, KVM_REQ_EVENT may not be set as APICv was
 	 * still active when the interrupt got accepted. Make sure
 	 * kvm_check_and_inject_events() is called to check for that.
-	 *
-	 * Update SVI when APICv gets enabled, otherwise SVI won't reflect the
-	 * highest bit in vISR and the next accelerated EOI in the guest won't
-	 * be virtualized correctly (the CPU uses SVI to determine which vISR
-	 * vector to clear).
 	 */
 	if (!apic->apicv_active)
 		kvm_make_request(KVM_REQ_EVENT, vcpu);
-	else
-		kvm_apic_update_hwapic_isr(vcpu);
 
 out:
 	preempt_enable();
Signature · 1 line
2.52.0.457.g6b5491de43-goog
author maintainer Applied
#9 lore raw
In reply to Sean Christopherson
Quoted 16 lines · Sean Christopherson

v4 of what was "KVM: VMX: Fix APICv activation bugs", but the two bug fixes were already merged and so that name doesn't fit.

Rip out the "defer updates until nested VM-Exit" that contributed to the APICv bugs and made them harder to fix.

v4:
- Collect reviews. [Chao]
- Use the existing vmcall() in the selftest. [Chao]
- Use GUEST_FAIL() instead of TEST_FAIL() in the guest. [Chao]
- Correct the number of args passed to the guest in the test. [Chao]
- Remove the redundant hwapic_isr_update() call in kvm_lapic_reset()
- Use the local "vmx" variable in vmx_set_virtual_apic_mode(). [Chao]
- Relocate the "update SVI" comment from __kvm_vcpu_update_apicv() to kvm_apic_update_apicv().

Quoted 1 line

[...]

Applied to kvm-x86 apic, thanks!

[1/8] KVM: selftests: Add a test to verify APICv updates (while L2 is active)

      https://github.com/kvm-x86/linux/commit/c3a9a27c79e4
[2/8] KVM: nVMX: Switch to vmcs01 to update PML controls on-demand if L2 is active
      https://github.com/kvm-x86/linux/commit/3e013d0a7099
[3/8] KVM: nVMX: Switch to vmcs01 to update TPR threshold on-demand if L2 is active
      https://github.com/kvm-x86/linux/commit/51ca2746078e
[4/8] KVM: nVMX: Switch to vmcs01 to update SVI on-demand if L2 is active
      https://github.com/kvm-x86/linux/commit/f0044429b257
[5/8] KVM: nVMX: Switch to vmcs01 to refresh APICv controls on-demand if L2 is active
      https://github.com/kvm-x86/linux/commit/2bf889a68fba
[6/8] KVM: nVMX: Switch to vmcs01 to update APIC page on-demand if L2 is active
      https://github.com/kvm-x86/linux/commit/51c821d6d0ba
[7/8] KVM: nVMX: Switch to vmcs01 to set virtual APICv mode on-demand if L2 is active
      https://github.com/kvm-x86/linux/commit/249cc1ab4b9a
[8/8] KVM: x86: Update APICv ISR (a.k.a. SVI) as part of kvm_apic_update_apicv()
      https://github.com/kvm-x86/linux/commit/000d75b0b186

--
https://github.com/kvm-x86/linux/tree/next