Skip to contentSkip to conversation

KVM: x86: AVIC and local APIC fixes+cleanups

PATCH Superseded Sean Christopherson 3 participants 43 messages Sep 20, 2022 – Sep 28, 2022

Subsystems (4)

v3 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Sean Christopherson

author maintainer
#0 lore raw

v3 cover KVM: x86: AVIC and local APIC fixes+cleanups

TL;DR: KVM's AVIC and optimized APIC map code doesn't correctly handle various edge cases that are architecturally legal(ish), but are unlikely to occur in most real world scenarios.

I have tested this heavily with KUT, but I haven't booted Windows and don't have access to x2AVIC, so additional testing would be much appreciated.

v3:
- Collect reviews. [Paolo]
- Drop "partial" x2APIC inhibit and instead delete the memslot. [Maxim, Suravee]
- Skip logical mode updates for x2APIC, which just reuses the phys_map with some clever logic. [Suravee]
- Add a fix for "nodecode write" traps. [Alejandro]

v2:
- https://lore.kernel.org/all/[email protected]
- Collect reviews. [Li, Maxim]
- Disable only MMIO access when x2APIC is enabled (instead of disabling all of AVIC). [Maxim]
- Inhibit AVIC when logical IDs are aliased. [Maxim]
- Tweak name of set_virtual_apic_mode() hook. [Maxim]
- Straight up revert logical ID fastpath mess. [Maxim]
- Reword changelog about skipping vCPU during logical setup. [Maxim]
- Fix LDR updates on AVIC. [Maxim?]
- Fix a nasty ISR caching bug.
- Flush TLB when activating AVIC.

v1: https://lore.kernel.org/all/[email protected]

Sean Christopherson (27):
KVM: x86: Blindly get current x2APIC reg value on "nodecode write"
traps
KVM: x86: Purge "highest ISR" cache when updating APICv state
KVM: SVM: Flush the "current" TLB when activating AVIC
KVM: SVM: Process ICR on AVIC IPI delivery failure due to invalid
target
KVM: x86: Don't inhibit APICv/AVIC if xAPIC ID mismatch is due to

    32-bit ID

KVM: x86: Move APIC access page helper to common x86 code
KVM: x86: Inhibit APIC memslot if x2APIC and AVIC are enabled
KVM: SVM: Don't put/load AVIC when setting virtual APIC mode
KVM: SVM: Replace "avic_mode" enum with "x2avic_enabled" boolean
KVM: SVM: Compute dest based on sender's x2APIC status for AVIC kick Revert "KVM: SVM: Use target APIC ID to complete x2AVIC IRQs when
possible"
KVM: SVM: Document that vCPU ID == APIC ID in AVIC kick fastpatch
KVM: SVM: Add helper to perform final AVIC "kick" of single vCPU
KVM: x86: Explicitly skip optimized logical map setup if vCPU's LDR==0
KVM: x86: Explicitly track all possibilities for APIC map's logical
modes
KVM: x86: Skip redundant x2APIC logical mode optimized cluster setup
KVM: x86: Disable APIC logical map if logical ID covers multiple MDAs
KVM: x86: Disable APIC logical map if vCPUs are aliased in logical
mode
KVM: x86: Honor architectural behavior for aliased 8-bit APIC IDs
KVM: x86: Inhibit APICv/AVIC if the optimized physical map is disabled
KVM: SVM: Inhibit AVIC if vCPUs are aliased in logical mode
KVM: SVM: Always update local APIC on writes to logical dest register
KVM: SVM: Update svm->ldr_reg cache even if LDR is "bad"
KVM: SVM: Require logical ID to be power-of-2 for AVIC entry
KVM: SVM: Handle multiple logical targets in AVIC kick fastpath
KVM: SVM: Ignore writes to Remote Read Data on AVIC write traps Revert "KVM: SVM: Do not throw warning when calling avic_vcpu_load on
a running vcpu"

Suravee Suthikulpanit (1):
KVM: SVM: Fix x2APIC Logical ID calculation for
avic_kick_target_vcpus_fast

 Documentation/virt/kvm/x86/errata.rst |  11 +
 arch/x86/include/asm/kvm_host.h       |  51 +++-
 arch/x86/kvm/lapic.c                  | 224 +++++++++++++---
 arch/x86/kvm/lapic.h                  |   2 +
 arch/x86/kvm/svm/avic.c               | 363 ++++++++++++--------------
 arch/x86/kvm/svm/svm.c                |   4 +-
 arch/x86/kvm/svm/svm.h                |  11 +-
 arch/x86/kvm/vmx/vmx.c                |  36 +--
 arch/x86/kvm/x86.c                    |   7 +
 9 files changed, 426 insertions(+), 283 deletions(-)

base-commit: 372d07084593dc7a399bf9bee815711b1fb1bcf2

Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#1 lore raw

PATCH v3 1/28 KVM: x86: Blindly get current x2APIC reg value on "nodecode write" traps

When emulating a x2APIC write in response to an APICv/AVIC trap, get the the written value from the vAPIC page without checking that reads are allowed for the target register. AVIC can generate trap-like VM-Exits on writes to EOI, and so KVM needs to get the written value from the backing page without running afoul of EOI's write-only behavior.

Alternatively, EOI could be special cased to always write '0', e.g. so that the sanity check could be preserved, but x2APIC on AMD is actually supposed to disallow non-zero writes (not emulated by KVM), and the sanity check was a byproduct of how the KVM code was written, i.e. wasn't added to guard against anything in particular.

Fixes 70c8327c11c6 ("KVM: x86: Bug the VM if an accelerated x2APIC trap occurs on a "bad" reg")
Fixes 1bd9dfec9fd4 ("KVM: x86: Do not block APIC write for non ICR registers")
---
 arch/x86/kvm/lapic.c | 9 ++-------
 1 file changed, 2 insertions(+), 7 deletions(-)
arch/x86/kvm/lapic.c +2 -7
index 9dda989a1cf0..8004c4d0a8e5 100644
@@ -2284,23 +2284,18 @@ void kvm_apic_write_nodecode(struct kvm_vcpu *vcpu, u32 offset)
 	struct kvm_lapic *apic = vcpu->arch.apic;
 	u64 val;
 
-	if (apic_x2apic_mode(apic)) {
-		if (KVM_BUG_ON(kvm_lapic_msr_read(apic, offset, &val), vcpu->kvm))
-			return;
-	} else {
-		val = kvm_lapic_get_reg(apic, offset);
-	}
-
 	/*
 	 * ICR is a single 64-bit register when x2APIC is enabled.  For legacy
 	 * xAPIC, ICR writes need to go down the common (slightly slower) path
 	 * to get the upper half from ICR2.
 	 */
 	if (apic_x2apic_mode(apic) && offset == APIC_ICR) {
+		val = kvm_lapic_get_reg64(apic, APIC_ICR);
 		kvm_apic_send_ipi(apic, (u32)val, (u32)(val >> 32));
 		trace_kvm_apic_write(APIC_ICR, val);
 	} else {
 		/* TODO: optimize to just emulate side effect w/o one more write */
+		val = kvm_lapic_get_reg(apic, offset);
 		kvm_lapic_reg_write(apic, offset, (u32)val);
 	}
 }
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#2 lore raw

PATCH v3 2/28 KVM: x86: Purge "highest ISR" cache when updating APICv state

In reply to Sean Christopherson

Purge the "highest ISR" cache when updating APICv state on a vCPU. The cache must not be used when APICv is active as hardware may emulate EOIs (and other operations) without exiting to KVM.

This fixes a bug where KVM will effectively block IRQs in perpetuity due to the "highest ISR" never getting reset if APICv is activated on a vCPU while an IRQ is in-service. Hardware emulates the EOI and KVM never gets a chance to update its cache.

Fixes b26a695a1d78 ("kvm: lapic: Introduce APICv update helper function")
Cc Suravee Suthikulpanit <[email protected]>
Cc Maxim Levitsky <[email protected]>
---
 arch/x86/kvm/lapic.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
arch/x86/kvm/lapic.c +1 -2
index 8004c4d0a8e5..adac6ca9b7dc 100644
@@ -2424,6 +2424,7 @@ void kvm_apic_update_apicv(struct kvm_vcpu *vcpu)
 		 */
 		apic->isr_count = count_vectors(apic->regs + APIC_ISR);
 	}
+	apic->highest_isr_cache = -1;
 }
 EXPORT_SYMBOL_GPL(kvm_apic_update_apicv);
 
@@ -2480,7 +2481,6 @@ void kvm_lapic_reset(struct kvm_vcpu *vcpu, bool init_event)
 		kvm_lapic_set_reg(apic, APIC_TMR + 0x10 * i, 0);
 	}
 	kvm_apic_update_apicv(vcpu);
-	apic->highest_isr_cache = -1;
 	update_divide_count(apic);
 	atomic_set(&apic->lapic_timer.pending, 0);
 
@@ -2767,7 +2767,6 @@ int kvm_apic_set_state(struct kvm_vcpu *vcpu, struct kvm_lapic_state *s)
 	__start_apic_timer(apic, APIC_TMCCT);
 	kvm_lapic_set_reg(apic, APIC_TMCCT, 0);
 	kvm_apic_update_apicv(vcpu);
-	apic->highest_isr_cache = -1;
 	if (apic->apicv_active) {
 		static_call_cond(kvm_x86_apicv_post_state_restore)(vcpu);
 		static_call_cond(kvm_x86_hwapic_irr_update)(vcpu, apic_find_highest_irr(apic));
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#3 lore raw

PATCH v3 3/28 KVM: SVM: Flush the "current" TLB when activating AVIC

In reply to Sean Christopherson

Flush the TLB when activating AVIC as the CPU can insert into the TLB while AVIC is "locally" disabled. KVM doesn't treat "APIC hardware disabled" as VM-wide AVIC inhibition, and so when a vCPU has its APIC hardware disabled, AVIC is not guaranteed to be inhibited. As a result, KVM may create a valid NPT mapping for the APIC base, which the CPU can cache as a non-AVIC translation.

Note, Intel handles this in vmx_set_virtual_apic_mode().

---
 arch/x86/kvm/svm/avic.c | 6 ++++++
 1 file changed, 6 insertions(+)
arch/x86/kvm/svm/avic.c +6 -0
index 6919dee69f18..712330b80891 100644
@@ -86,6 +86,12 @@ static void avic_activate_vmcb(struct vcpu_svm *svm)
 		/* Disabling MSR intercept for x2APIC registers */
 		svm_set_x2apic_msr_interception(svm, false);
 	} else {
+		/*
+		 * Flush the TLB, the guest may have inserted a non-APIC
+		 * mapping into the TLB while AVIC was disabled.
+		 */
+		kvm_make_request(KVM_REQ_TLB_FLUSH_CURRENT, &svm->vcpu);
+
 		/* For xAVIC and hybrid-xAVIC modes */
 		vmcb->control.avic_physical_id |= AVIC_MAX_PHYSICAL_ID;
 		/* Enabling MSR intercept for x2APIC registers */
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#4 lore raw

PATCH v3 4/28 KVM: SVM: Process ICR on AVIC IPI delivery failure due to invalid target

In reply to Sean Christopherson

Emulate ICR writes on AVIC IPI failures due to invalid targets using the same logic as failures due to invalid types. AVIC acceleration fails if _any_ of the targets are invalid, and crucially VM-Exits before sending IPIs to targets that _are_ valid. In logical mode, the destination is a bitmap, i.e. a single IPI can target multiple logical IDs. Doing nothing causes KVM to drop IPIs if at least one target is valid and at least one target is invalid.

Fixes 18f40c53e10f ("svm: Add VMEXIT handlers for AVIC")
---
 arch/x86/kvm/svm/avic.c | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)
arch/x86/kvm/svm/avic.c +9 -7
index 712330b80891..3b2c88b168ba 100644
@@ -502,14 +502,18 @@ int avic_incomplete_ipi_interception(struct kvm_vcpu *vcpu)
 	trace_kvm_avic_incomplete_ipi(vcpu->vcpu_id, icrh, icrl, id, index);
 
 	switch (id) {
+	case AVIC_IPI_FAILURE_INVALID_TARGET:
 	case AVIC_IPI_FAILURE_INVALID_INT_TYPE:
 		/*
 		 * Emulate IPIs that are not handled by AVIC hardware, which
-		 * only virtualizes Fixed, Edge-Triggered INTRs.  The exit is
-		 * a trap, e.g. ICR holds the correct value and RIP has been
-		 * advanced, KVM is responsible only for emulating the IPI.
-		 * Sadly, hardware may sometimes leave the BUSY flag set, in
-		 * which case KVM needs to emulate the ICR write as well in
+		 * only virtualizes Fixed, Edge-Triggered INTRs, and falls over
+		 * if _any_ targets are invalid, e.g. if the logical mode mask
+		 * is a superset of running vCPUs.
+		 *
+		 * The exit is a trap, e.g. ICR holds the correct value and RIP
+		 * has been advanced, KVM is responsible only for emulating the
+		 * IPI.  Sadly, hardware may sometimes leave the BUSY flag set,
+		 * in which case KVM needs to emulate the ICR write as well in
 		 * order to clear the BUSY flag.
 		 */
 		if (icrl & APIC_ICR_BUSY)
@@ -525,8 +529,6 @@ int avic_incomplete_ipi_interception(struct kvm_vcpu *vcpu)
 		 */
 		avic_kick_target_vcpus(vcpu->kvm, apic, icrl, icrh, index);
 		break;
-	case AVIC_IPI_FAILURE_INVALID_TARGET:
-		break;
 	case AVIC_IPI_FAILURE_INVALID_BACKING_PAGE:
 		WARN_ONCE(1, "Invalid backing page\n");
 		break;
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#5 lore raw

PATCH v3 5/28 KVM: x86: Don't inhibit APICv/AVIC if xAPIC ID mismatch is due to 32-bit ID

In reply to Sean Christopherson

Truncate the vcpu_id, a.k.a. x2APIC ID, to an 8-bit value when comparing it against the xAPIC ID to avoid false positives (sort of) on systems with >255 CPUs, i.e. with IDs that don't fit into a u8. The intent of APIC_ID_MODIFIED is to inhibit APICv/AVIC when the xAPIC is changed from it's original value,

The mismatch isn't technically a false positive, as architecturally the xAPIC IDs do end up being aliased in this scenario, and neither APICv nor AVIC correctly handles IPI virtualization when there is aliasing. However, KVM already deliberately does not honor the aliasing behavior that results when an x2APIC ID gets truncated to an xAPIC ID. I.e. the resulting APICv/AVIC behavior is aligned with KVM's existing behavior when KVM's x2APIC hotplug hack is effectively enabled.

If/when KVM provides a way to disable the hotplug hack, APICv/AVIC can piggyback whatever logic disables the optimized APIC map (which is what provides the hotplug hack), i.e. so that KVM's optimized map and APIC virtualization yield the same behavior.

For now, fix the immediate problem of APIC virtualization being disabled for large VMs, which is a much more pressing issue than ensuring KVM honors architectural behavior for APIC ID aliasing.

Fixes 3743c2f02517 ("KVM: x86: inhibit APICv/AVIC on changes to APIC ID or APIC base")
Cc Maxim Levitsky <[email protected]>
---
 arch/x86/kvm/lapic.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
arch/x86/kvm/lapic.c +6 -1
index adac6ca9b7dc..a02defa3f7b5 100644
@@ -2075,7 +2075,12 @@ static void kvm_lapic_xapic_id_updated(struct kvm_lapic *apic)
 	if (KVM_BUG_ON(apic_x2apic_mode(apic), kvm))
 		return;
 
-	if (kvm_xapic_id(apic) == apic->vcpu->vcpu_id)
+	/*
+	 * Deliberately truncate the vCPU ID when detecting a modified APIC ID
+	 * to avoid false positives if the vCPU ID, i.e. x2APIC ID, is a 32-bit
+	 * value.
+	 */
+	if (kvm_xapic_id(apic) == (u8)apic->vcpu->vcpu_id)
 		return;
 
 	kvm_set_apicv_inhibit(apic->vcpu->kvm, APICV_INHIBIT_REASON_APIC_ID_MODIFIED);
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
#6 lore raw

Re: [PATCH v3 05/28] KVM: x86: Don't inhibit APICv/AVIC if xAPIC ID mismatch is due to 32-bit ID

Quoted 14 lines · Sean Christopherson

Truncate the vcpu_id, a.k.a. x2APIC ID, to an 8-bit value when comparing it against the xAPIC ID to avoid false positives (sort of) on systems with >255 CPUs, i.e. with IDs that don't fit into a u8. The intent of APIC_ID_MODIFIED is to inhibit APICv/AVIC when the xAPIC is changed from it's original value,

The mismatch isn't technically a false positive, as architecturally the xAPIC IDs do end up being aliased in this scenario, and neither APICv nor AVIC correctly handles IPI virtualization when there is aliasing. However, KVM already deliberately does not honor the aliasing behavior that results when an x2APIC ID gets truncated to an xAPIC ID. I.e. the resulting APICv/AVIC behavior is aligned with KVM's existing behavior when KVM's x2APIC hotplug hack is effectively enabled.

Quoted 8 lines

If/when KVM provides a way to disable the hotplug hack, APICv/AVIC can piggyback whatever logic disables the optimized APIC map (which is what provides the hotplug hack), i.e. so that KVM's optimized map and APIC virtualization yield the same behavior.

For now, fix the immediate problem of APIC virtualization being disabled for large VMs, which is a much more pressing issue than ensuring KVM honors architectural behavior for APIC ID aliasing.

I built a host kernel with this entire series on top of mainline v6.0-rc6, and booting a guest with AVIC enabled works as expected on the initial boot. The issue is that during the first reboot AVIC is inhibited due to APICV_INHIBIT_REASON_APIC_ID_MODIFIED, and I see constant inhibition events due to APICV_INHIBIT_REASON_IRQWIN as seen in the traces:

qemu-system-x86-10147   [222] .....  1116.519052: kvm_apicv_inhibit_changed: set reason=8, inhibits=0x120
qemu-system-x86-10147   [222] .....  1116.519063: kvm_apicv_inhibit_changed: cleared reason=8, inhibits=0x20
qemu-system-x86-10147   [222] .....  1117.934222: kvm_apicv_inhibit_changed: set reason=8, inhibits=0x120
qemu-system-x86-10147   [222] .....  1117.934233: kvm_apicv_inhibit_changed: cleared reason=8, inhibits=0x20

It happens regardless of vCPU count (tested with 2, 32, 255, 380, and 512 vCPUs). This state persists for all subsequent reboots, until the VM is terminated. For vCPU counts < 256, when x2apic is disabled the problem does not occur, and AVIC continues to work properly after reboots.

I did not see this issue when testing a similar host kernel that did not include this current patchset, but instead applied the earlier:
https://lore.kernel.org/lkml/[email protected]/
which inspired this [05/23] patch and the follow up [22/28] in this series.

I am using QEMU built from v7.1.0 upstream tag, plus the patch at:
https://lore.kernel.org/qemu-devel/[email protected]/

Please feel free to request any other data points that might be relevant and I'll try to collect them.

Alejandro

Quoted 27 lines
Fixes 3743c2f02517 ("KVM: x86: inhibit APICv/AVIC on changes to APIC ID or APIC base")
Cc Maxim Levitsky <[email protected]>
---
  arch/x86/kvm/lapic.c | 7 ++++++-
  1 file changed, 6 insertions(+), 1 deletion(-)
arch/x86/kvm/lapic.c +6 -1
index adac6ca9b7dc..a02defa3f7b5 100644
@@ -2075,7 +2075,12 @@ static void kvm_lapic_xapic_id_updated(struct kvm_lapic *apic)
  	if (KVM_BUG_ON(apic_x2apic_mode(apic), kvm))
  		return;
 
-	if (kvm_xapic_id(apic) == apic->vcpu->vcpu_id)
+	/*
+	 * Deliberately truncate the vCPU ID when detecting a modified APIC ID
+	 * to avoid false positives if the vCPU ID, i.e. x2APIC ID, is a 32-bit
+	 * value.
+	 */
+	if (kvm_xapic_id(apic) == (u8)apic->vcpu->vcpu_id)
  		return;
 
  	kvm_set_apicv_inhibit(apic->vcpu->kvm, APICV_INHIBIT_REASON_APIC_ID_MODIFIED);
#7 lore raw

Re: [PATCH v3 05/28] KVM: x86: Don't inhibit APICv/AVIC if xAPIC ID mismatch is due to 32-bit ID

Quoted 25 lines · Alejandro Jimenez
Quoted 14 lines · Sean Christopherson

Truncate the vcpu_id, a.k.a. x2APIC ID, to an 8-bit value when comparing it against the xAPIC ID to avoid false positives (sort of) on systems with >255 CPUs, i.e. with IDs that don't fit into a u8. The intent of APIC_ID_MODIFIED is to inhibit APICv/AVIC when the xAPIC is changed from it's original value,

The mismatch isn't technically a false positive, as architecturally the xAPIC IDs do end up being aliased in this scenario, and neither APICv nor AVIC correctly handles IPI virtualization when there is aliasing. However, KVM already deliberately does not honor the aliasing behavior that results when an x2APIC ID gets truncated to an xAPIC ID. I.e. the resulting APICv/AVIC behavior is aligned with KVM's existing behavior when KVM's x2APIC hotplug hack is effectively enabled.

Quoted 8 lines

If/when KVM provides a way to disable the hotplug hack, APICv/AVIC can piggyback whatever logic disables the optimized APIC map (which is what provides the hotplug hack), i.e. so that KVM's optimized map and APIC virtualization yield the same behavior.

For now, fix the immediate problem of APIC virtualization being disabled for large VMs, which is a much more pressing issue than ensuring KVM honors architectural behavior for APIC ID aliasing.

Quoted 5 lines

I built a host kernel with this entire series on top of mainline v6.0-rc6, and booting a guest with AVIC enabled works as expected on the initial boot. The issue is that during the first reboot AVIC is inhibited due to APICV_INHIBIT_REASON_APIC_ID_MODIFIED, and I see constant inhibition events due to APICV_INHIBIT_REASON_IRQWIN as seen in

APICV_INHIBIT_REASON_IRQWIN is OK, because that happens about every time the good old PIT timer fires which happens on reboot.

APICV_INHIBIT_REASON_APIC_ID_MODIFIED should not happen as you noted, this needs investigation.

Best regards,
Maxim Levitsky

Quoted 54 lines

the traces:

qemu-system-x86-10147 [222] ..... 1116.519052:
kvm_apicv_inhibit_changed: set reason=8, inhibits=0x120
qemu-system-x86-10147 [222] ..... 1116.519063:
kvm_apicv_inhibit_changed: cleared reason=8, inhibits=0x20
qemu-system-x86-10147 [222] ..... 1117.934222:
kvm_apicv_inhibit_changed: set reason=8, inhibits=0x120
qemu-system-x86-10147 [222] ..... 1117.934233:
kvm_apicv_inhibit_changed: cleared reason=8, inhibits=0x20

It happens regardless of vCPU count (tested with 2, 32, 255, 380, and 512 vCPUs). This state persists for all subsequent reboots, until the VM is terminated. For vCPU counts < 256, when x2apic is disabled the problem does not occur, and AVIC continues to work properly after reboots.

I did not see this issue when testing a similar host kernel that did not include this current patchset, but instead applied the earlier:
https://lore.kernel.org/lkml/[email protected]/
which inspired this [05/23] patch and the follow up [22/28] in this series.

I am using QEMU built from v7.1.0 upstream tag, plus the patch at:
https://lore.kernel.org/qemu-devel/[email protected]/

Please feel free to request any other data points that might be relevant and I'll try to collect them.

Alejandro

Quoted 26 lines
Fixes 3743c2f02517 ("KVM: x86: inhibit APICv/AVIC on changes to APIC ID or APIC base")
Cc Maxim Levitsky <[email protected]>
---
  arch/x86/kvm/lapic.c | 7 ++++++-
  1 file changed, 6 insertions(+), 1 deletion(-)
arch/x86/kvm/lapic.c +6 -1
index adac6ca9b7dc..a02defa3f7b5 100644
@@ -2075,7 +2075,12 @@ static void kvm_lapic_xapic_id_updated(struct kvm_lapic *apic)
  	if (KVM_BUG_ON(apic_x2apic_mode(apic), kvm))
  		return;
 
-	if (kvm_xapic_id(apic) == apic->vcpu->vcpu_id)
+	/*
+	 * Deliberately truncate the vCPU ID when detecting a modified APIC ID
+	 * to avoid false positives if the vCPU ID, i.e. x2APIC ID, is a 32-bit
+	 * value.
+	 */
+	if (kvm_xapic_id(apic) == (u8)apic->vcpu->vcpu_id)
  		return;
 
  	kvm_set_apicv_inhibit(apic->vcpu->kvm, APICV_INHIBIT_REASON_APIC_ID_MODIFIED);
author maintainer
#8 lore raw

Re: [PATCH v3 05/28] KVM: x86: Don't inhibit APICv/AVIC if xAPIC ID mismatch is due to 32-bit ID

Quoted 33 lines · Maxim Levitsky
Quoted 25 lines · Alejandro Jimenez
Quoted 14 lines · Sean Christopherson

Truncate the vcpu_id, a.k.a. x2APIC ID, to an 8-bit value when comparing it against the xAPIC ID to avoid false positives (sort of) on systems with >255 CPUs, i.e. with IDs that don't fit into a u8. The intent of APIC_ID_MODIFIED is to inhibit APICv/AVIC when the xAPIC is changed from it's original value,

The mismatch isn't technically a false positive, as architecturally the xAPIC IDs do end up being aliased in this scenario, and neither APICv nor AVIC correctly handles IPI virtualization when there is aliasing. However, KVM already deliberately does not honor the aliasing behavior that results when an x2APIC ID gets truncated to an xAPIC ID. I.e. the resulting APICv/AVIC behavior is aligned with KVM's existing behavior when KVM's x2APIC hotplug hack is effectively enabled.

Quoted 8 lines

If/when KVM provides a way to disable the hotplug hack, APICv/AVIC can piggyback whatever logic disables the optimized APIC map (which is what provides the hotplug hack), i.e. so that KVM's optimized map and APIC virtualization yield the same behavior.

For now, fix the immediate problem of APIC virtualization being disabled for large VMs, which is a much more pressing issue than ensuring KVM honors architectural behavior for APIC ID aliasing.

Quoted 5 lines

I built a host kernel with this entire series on top of mainline v6.0-rc6, and booting a guest with AVIC enabled works as expected on the initial boot. The issue is that during the first reboot AVIC is inhibited due to APICV_INHIBIT_REASON_APIC_ID_MODIFIED, and I see constant inhibition events due to APICV_INHIBIT_REASON_IRQWIN as seen in

Quoted 5 lines

APICV_INHIBIT_REASON_IRQWIN is OK, because that happens about every time the good old PIT timer fires which happens on reboot.

APICV_INHIBIT_REASON_APIC_ID_MODIFIED should not happen as you noted, this needs investigation.

Ya, I'll take a look.

Quoted 4 lines
Quoted 4 lines

It happens regardless of vCPU count (tested with 2, 32, 255, 380, and 512 vCPUs). This state persists for all subsequent reboots, until the VM is terminated. For vCPU counts < 256, when x2apic is disabled the problem does not occur, and AVIC continues to work properly after reboots.

Bit of a shot in the dark, but does the below fix the issue? There are two issues with calling kvm_lapic_xapic_id_updated() from kvm_apic_state_fixup():

1. The xAPIC ID should only be refreshed on "set".

2. The refresh needs to be noted after memcpy(vcpu->arch.apic->regs, s->regs, sizeof(*s));

and a third bug in the helper itself, as changes to the ID should be ignored if the APIC is hardward disabled since the ID is reset to the vcpu_id when the APIC is hardware enabled (architecturally behavior).

---
 arch/x86/kvm/lapic.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
arch/x86/kvm/lapic.c +6 -2
index 804d529d9bfb..b8b2faf5abc7 100644
@@ -2159,6 +2159,9 @@ static void kvm_lapic_xapic_id_updated(struct kvm_lapic *apic)
 {
 	struct kvm *kvm = apic->vcpu->kvm;
 
+	if (!kvm_apic_hw_enabled(apic))
+		return;
+
 	if (KVM_BUG_ON(apic_x2apic_mode(apic), kvm))
 		return;
 
@@ -2875,8 +2878,6 @@ static int kvm_apic_state_fixup(struct kvm_vcpu *vcpu,
 			icr = __kvm_lapic_get_reg64(s->regs, APIC_ICR);
 			__kvm_lapic_set_reg(s->regs, APIC_ICR2, icr >> 32);
 		}
-	} else {
-		kvm_lapic_xapic_id_updated(vcpu->arch.apic);
 	}
 
 	return 0;
@@ -2912,6 +2913,9 @@ int kvm_apic_set_state(struct kvm_vcpu *vcpu, struct kvm_lapic_state *s)
 	}
 	memcpy(vcpu->arch.apic->regs, s->regs, sizeof(*s));
 
+	if (!apic_x2apic_mode(vcpu->arch.apic))
+		kvm_lapic_xapic_id_updated(vcpu->arch.apic);
+
 	atomic_set_release(&apic->vcpu->kvm->arch.apic_map_dirty, DIRTY);
 	kvm_recalculate_apic_map(vcpu->kvm);
 	kvm_apic_set_version(vcpu);

base-commit: 0b502152c0b8523f399bdb53096e2d620c5795b5

#9 lore raw

Re: [PATCH v3 05/28] KVM: x86: Don't inhibit APICv/AVIC if xAPIC ID mismatch is due to 32-bit ID

Quoted 45 lines · Sean Christopherson
Quoted 31 lines · Maxim Levitsky
Quoted 24 lines · Alejandro Jimenez
Quoted 14 lines · Sean Christopherson

Truncate the vcpu_id, a.k.a. x2APIC ID, to an 8-bit value when comparing it against the xAPIC ID to avoid false positives (sort of) on systems with >255 CPUs, i.e. with IDs that don't fit into a u8. The intent of APIC_ID_MODIFIED is to inhibit APICv/AVIC when the xAPIC is changed from it's original value,

The mismatch isn't technically a false positive, as architecturally the xAPIC IDs do end up being aliased in this scenario, and neither APICv nor AVIC correctly handles IPI virtualization when there is aliasing. However, KVM already deliberately does not honor the aliasing behavior that results when an x2APIC ID gets truncated to an xAPIC ID. I.e. the resulting APICv/AVIC behavior is aligned with KVM's existing behavior when KVM's x2APIC hotplug hack is effectively enabled.

Quoted 8 lines

If/when KVM provides a way to disable the hotplug hack, APICv/AVIC can piggyback whatever logic disables the optimized APIC map (which is what provides the hotplug hack), i.e. so that KVM's optimized map and APIC virtualization yield the same behavior.

For now, fix the immediate problem of APIC virtualization being disabled for large VMs, which is a much more pressing issue than ensuring KVM honors architectural behavior for APIC ID aliasing.

Quoted 5 lines

I built a host kernel with this entire series on top of mainline v6.0-rc6, and booting a guest with AVIC enabled works as expected on the initial boot. The issue is that during the first reboot AVIC is inhibited due to APICV_INHIBIT_REASON_APIC_ID_MODIFIED, and I see constant inhibition events due to APICV_INHIBIT_REASON_IRQWIN as seen in

Quoted 5 lines

APICV_INHIBIT_REASON_IRQWIN is OK, because that happens about every time the good old PIT timer fires which happens on reboot.

APICV_INHIBIT_REASON_APIC_ID_MODIFIED should not happen as you noted, this needs investigation.

Ya, I'll take a look.

Quoted 4 lines
Quoted 4 lines

It happens regardless of vCPU count (tested with 2, 32, 255, 380, and 512 vCPUs). This state persists for all subsequent reboots, until the VM is terminated. For vCPU counts < 256, when x2apic is disabled the problem does not occur, and AVIC continues to work properly after reboots.

Quoted 4 lines

Bit of a shot in the dark, but does the below fix the issue? There are two issues with calling kvm_lapic_xapic_id_updated() from kvm_apic_state_fixup():

1. The xAPIC ID should only be refreshed on "set".

True - I didn't bother to fix it yet because it shouldn't cause harm, but sure this needs to be fixed.

Quoted 2 lines

2. The refresh needs to be noted after memcpy(vcpu->arch.apic->regs, s->regs, sizeof(*s));

Are you sure? The check is first because if it fails, then error is returned to userspace and the KVM's state left unchanged.

I assume you are talking about

        ....
	r = kvm_apic_state_fixup(vcpu, s, true);
	if (r) {
		kvm_recalculate_apic_map(vcpu->kvm);
		return r;
	}
	memcpy(vcpu->arch.apic->regs, s->regs, sizeof(*s));
        ....
Quoted 4 lines

and a third bug in the helper itself, as changes to the ID should be ignored if the APIC is hardward disabled since the ID is reset to the vcpu_id when the APIC is hardware enabled (architecturally behavior).

That is true, and something I haven't thought about.

Best regards,
Maxim Levitsky

Quoted 40 lines
---
 arch/x86/kvm/lapic.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
arch/x86/kvm/lapic.c +6 -2
index 804d529d9bfb..b8b2faf5abc7 100644
@@ -2159,6 +2159,9 @@ static void kvm_lapic_xapic_id_updated(struct kvm_lapic *apic)
 {
 	struct kvm *kvm = apic->vcpu->kvm;
 
+	if (!kvm_apic_hw_enabled(apic))
+		return;
+
 	if (KVM_BUG_ON(apic_x2apic_mode(apic), kvm))
 		return;
 
@@ -2875,8 +2878,6 @@ static int kvm_apic_state_fixup(struct kvm_vcpu *vcpu,
 			icr = __kvm_lapic_get_reg64(s->regs, APIC_ICR);
 			__kvm_lapic_set_reg(s->regs, APIC_ICR2, icr >> 32);
 		}
-	} else {
-		kvm_lapic_xapic_id_updated(vcpu->arch.apic);
 	}
 
 	return 0;
@@ -2912,6 +2913,9 @@ int kvm_apic_set_state(struct kvm_vcpu *vcpu, struct kvm_lapic_state *s)
 	}
 	memcpy(vcpu->arch.apic->regs, s->regs, sizeof(*s));
 
+	if (!apic_x2apic_mode(vcpu->arch.apic))
+		kvm_lapic_xapic_id_updated(vcpu->arch.apic);
+
 	atomic_set_release(&apic->vcpu->kvm->arch.apic_map_dirty, DIRTY);
 	kvm_recalculate_apic_map(vcpu->kvm);
 	kvm_apic_set_version(vcpu);

base-commit: 0b502152c0b8523f399bdb53096e2d620c5795b5

author maintainer
#10 lore raw

Re: [PATCH v3 05/28] KVM: x86: Don't inhibit APICv/AVIC if xAPIC ID mismatch is due to 32-bit ID

Quoted 6 lines · Maxim Levitsky
Quoted 5 lines · Sean Christopherson
Quoted 4 lines
Quoted 4 lines

It happens regardless of vCPU count (tested with 2, 32, 255, 380, and 512 vCPUs). This state persists for all subsequent reboots, until the VM is terminated. For vCPU counts < 256, when x2apic is disabled the problem does not occur, and AVIC continues to work properly after reboots.

Quoted 6 lines
Quoted 4 lines

Bit of a shot in the dark, but does the below fix the issue? There are two issues with calling kvm_lapic_xapic_id_updated() from kvm_apic_state_fixup():

1. The xAPIC ID should only be refreshed on "set".

True - I didn't bother to fix it yet because it shouldn't cause harm, but sure this needs to be fixed.

It's probably benign on its own, but with the missing "hardware enabled" check, it could be problematic if userspace does KVM_GET_LAPIC while the APIC is hardware disabled, after the APIC was previously in x2APIC mode. I'm guessing QEMU does KVM_GET_LAPIC state when emulating reboot, hence the potential for being involved in the bug Alejandro is seeing.

Quoted 6 lines
Quoted 1 line

2. The refresh needs to be noted after memcpy(vcpu->arch.apic->regs, s->regs, sizeof(*s));

Are you sure? The check is first because if it fails, then error is returned to userspace and the KVM's state left unchanged.

I assume you are talking about

Quoted 7 lines
        ....
	r = kvm_apic_state_fixup(vcpu, s, true);
	if (r) {
		kvm_recalculate_apic_map(vcpu->kvm);
		return r;
	}
	memcpy(vcpu->arch.apic->regs, s->regs, sizeof(*s));

This isn't a failure path though, it's purely a "take note of the update", and KVM needs to do that processing _after_ the actual update. Specifically, kvm_lapic_xapic_id_updated() consumes the internal APIC state:

	if (kvm_xapic_id(apic) == apic->vcpu->vcpu_id)
		return;

Calling that before the internal state has been set with the incoming state from userspace is simply wrong.

The check that the x2APIC ID is "correct" stays where it is, this is purely the "is the xAPIC ID different" path.

#11 lore raw

Re: [PATCH v3 05/28] KVM: x86: Don't inhibit APICv/AVIC if xAPIC ID mismatch is due to 32-bit ID

Quoted 34 lines · Sean Christopherson
Quoted 6 lines · Maxim Levitsky
Quoted 5 lines · Sean Christopherson
Quoted 4 lines
Quoted 4 lines

It happens regardless of vCPU count (tested with 2, 32, 255, 380, and 512 vCPUs). This state persists for all subsequent reboots, until the VM is terminated. For vCPU counts < 256, when x2apic is disabled the problem does not occur, and AVIC continues to work properly after reboots.

Quoted 6 lines
Quoted 4 lines

Bit of a shot in the dark, but does the below fix the issue? There are two issues with calling kvm_lapic_xapic_id_updated() from kvm_apic_state_fixup():

1. The xAPIC ID should only be refreshed on "set".

True - I didn't bother to fix it yet because it shouldn't cause harm, but sure this needs to be fixed.

It's probably benign on its own, but with the missing "hardware enabled" check, it could be problematic if userspace does KVM_GET_LAPIC while the APIC is hardware disabled, after the APIC was previously in x2APIC mode. I'm guessing QEMU does KVM_GET_LAPIC state when emulating reboot, hence the potential for being involved in the bug Alejandro is seeing.

Quoted 6 lines
Quoted 1 line

2. The refresh needs to be noted after memcpy(vcpu->arch.apic->regs, s->regs, sizeof(*s));

Are you sure? The check is first because if it fails, then error is returned to userspace and the KVM's state left unchanged.

I assume you are talking about

Quoted 7 lines
        ....
	r = kvm_apic_state_fixup(vcpu, s, true);
	if (r) {
		kvm_recalculate_apic_map(vcpu->kvm);
		return r;
	}
	memcpy(vcpu->arch.apic->regs, s->regs, sizeof(*s));
Quoted 3 lines

This isn't a failure path though, it's purely a "take note of the update", and KVM needs to do that processing _after_ the actual update. Specifically, kvm_lapic_xapic_id_updated() consumes the internal APIC state:

Yes, I somehow blindly assumed that kvm_apic_state_fixup actually checks the new state and not the existing state.

Probably because my original code did that, I think it just checked the 'id' variable.. Oh well.
Thanks for catching this bug!

Best regards,
Maxim Levitsky

Quoted 10 lines
	if (kvm_xapic_id(apic) == apic->vcpu->vcpu_id)
		return;

Calling that before the internal state has been set with the incoming state from userspace is simply wrong.

The check that the x2APIC ID is "correct" stays where it is, this is purely the "is the xAPIC ID different" path.

author maintainer
#12 lore raw

Re: [PATCH v3 05/28] KVM: x86: Don't inhibit APICv/AVIC if xAPIC ID mismatch is due to 32-bit ID

Quoted 39 lines · Maxim Levitsky
Quoted 34 lines · Sean Christopherson
Quoted 6 lines · Maxim Levitsky
Quoted 5 lines · Sean Christopherson
Quoted 4 lines
Quoted 4 lines

It happens regardless of vCPU count (tested with 2, 32, 255, 380, and 512 vCPUs). This state persists for all subsequent reboots, until the VM is terminated. For vCPU counts < 256, when x2apic is disabled the problem does not occur, and AVIC continues to work properly after reboots.

Quoted 6 lines
Quoted 4 lines

Bit of a shot in the dark, but does the below fix the issue? There are two issues with calling kvm_lapic_xapic_id_updated() from kvm_apic_state_fixup():

1. The xAPIC ID should only be refreshed on "set".

True - I didn't bother to fix it yet because it shouldn't cause harm, but sure this needs to be fixed.

It's probably benign on its own, but with the missing "hardware enabled" check, it could be problematic if userspace does KVM_GET_LAPIC while the APIC is hardware disabled, after the APIC was previously in x2APIC mode. I'm guessing QEMU does KVM_GET_LAPIC state when emulating reboot, hence the potential for being involved in the bug Alejandro is seeing.

Quoted 6 lines
Quoted 1 line

2. The refresh needs to be noted after memcpy(vcpu->arch.apic->regs, s->regs, sizeof(*s));

Are you sure? The check is first because if it fails, then error is returned to userspace and the KVM's state left unchanged.

I assume you are talking about

Quoted 7 lines
        ....
	r = kvm_apic_state_fixup(vcpu, s, true);
	if (r) {
		kvm_recalculate_apic_map(vcpu->kvm);
		return r;
	}
	memcpy(vcpu->arch.apic->regs, s->regs, sizeof(*s));
Quoted 3 lines

This isn't a failure path though, it's purely a "take note of the update", and KVM needs to do that processing _after_ the actual update. Specifically, kvm_lapic_xapic_id_updated() consumes the internal APIC state:

Quoted 6 lines

Yes, I somehow blindly assumed that kvm_apic_state_fixup actually checks the new state and not the existing state.

Probably because my original code did that, I think it just checked the 'id' variable.. Oh well.
Thanks for catching this bug!

This was indeed the bug Alejandro hit. QEMU does KVM_SET_LAPIC and KVM checks the stale vAPIC state. Because the vCPU was in x2APIC mode, the 32-bit ID is in bits 31:0, not bit 31:24, and so kvm_xapic_id() returned '0' instead of the correct ID.

#13 lore raw

Re: [PATCH v3 05/28] KVM: x86: Don't inhibit APICv/AVIC if xAPIC ID mismatch is due to 32-bit ID

In reply to Sean Christopherson
Quoted 40 lines · Sean Christopherson
Quoted 33 lines · Maxim Levitsky
Quoted 25 lines · Alejandro Jimenez
Quoted 14 lines · Sean Christopherson

Truncate the vcpu_id, a.k.a. x2APIC ID, to an 8-bit value when comparing it against the xAPIC ID to avoid false positives (sort of) on systems with >255 CPUs, i.e. with IDs that don't fit into a u8. The intent of APIC_ID_MODIFIED is to inhibit APICv/AVIC when the xAPIC is changed from it's original value,

The mismatch isn't technically a false positive, as architecturally the xAPIC IDs do end up being aliased in this scenario, and neither APICv nor AVIC correctly handles IPI virtualization when there is aliasing. However, KVM already deliberately does not honor the aliasing behavior that results when an x2APIC ID gets truncated to an xAPIC ID. I.e. the resulting APICv/AVIC behavior is aligned with KVM's existing behavior when KVM's x2APIC hotplug hack is effectively enabled.

Quoted 8 lines

If/when KVM provides a way to disable the hotplug hack, APICv/AVIC can piggyback whatever logic disables the optimized APIC map (which is what provides the hotplug hack), i.e. so that KVM's optimized map and APIC virtualization yield the same behavior.

For now, fix the immediate problem of APIC virtualization being disabled for large VMs, which is a much more pressing issue than ensuring KVM honors architectural behavior for APIC ID aliasing.

Quoted 5 lines

I built a host kernel with this entire series on top of mainline v6.0-rc6, and booting a guest with AVIC enabled works as expected on the initial boot. The issue is that during the first reboot AVIC is inhibited due to APICV_INHIBIT_REASON_APIC_ID_MODIFIED, and I see constant inhibition events due to APICV_INHIBIT_REASON_IRQWIN as seen in

Quoted 5 lines

APICV_INHIBIT_REASON_IRQWIN is OK, because that happens about every time the good old PIT timer fires which happens on reboot.

APICV_INHIBIT_REASON_APIC_ID_MODIFIED should not happen as you noted, this needs investigation.

Quoted 8 lines

Ya, I'll take a look.

Quoted 4 lines
Quoted 4 lines

It happens regardless of vCPU count (tested with 2, 32, 255, 380, and 512 vCPUs). This state persists for all subsequent reboots, until the VM is terminated. For vCPU counts < 256, when x2apic is disabled the problem does not occur, and AVIC continues to work properly after reboots.

Bit of a shot in the dark, but does the below fix the issue?

The patch below fixes the problems for all the scenarios I have tested so far.

Thank you,
Alejandro

There are two

Quoted 49 lines

issues with calling kvm_lapic_xapic_id_updated() from kvm_apic_state_fixup():

1. The xAPIC ID should only be refreshed on "set".

2. The refresh needs to be noted after memcpy(vcpu->arch.apic->regs, s->regs, sizeof(*s));

and a third bug in the helper itself, as changes to the ID should be ignored if the APIC is hardward disabled since the ID is reset to the vcpu_id when the APIC is hardware enabled (architecturally behavior).

---
  arch/x86/kvm/lapic.c | 8 ++++++--
  1 file changed, 6 insertions(+), 2 deletions(-)
arch/x86/kvm/lapic.c +6 -2
index 804d529d9bfb..b8b2faf5abc7 100644
@@ -2159,6 +2159,9 @@ static void kvm_lapic_xapic_id_updated(struct kvm_lapic *apic)
  {
  	struct kvm *kvm = apic->vcpu->kvm;
 
+	if (!kvm_apic_hw_enabled(apic))
+		return;
+
  	if (KVM_BUG_ON(apic_x2apic_mode(apic), kvm))
  		return;
 
@@ -2875,8 +2878,6 @@ static int kvm_apic_state_fixup(struct kvm_vcpu *vcpu,
  			icr = __kvm_lapic_get_reg64(s->regs, APIC_ICR);
  			__kvm_lapic_set_reg(s->regs, APIC_ICR2, icr >> 32);
  		}
-	} else {
-		kvm_lapic_xapic_id_updated(vcpu->arch.apic);
  	}
 
  	return 0;
@@ -2912,6 +2913,9 @@ int kvm_apic_set_state(struct kvm_vcpu *vcpu, struct kvm_lapic_state *s)
  	}
  	memcpy(vcpu->arch.apic->regs, s->regs, sizeof(*s));
 
+	if (!apic_x2apic_mode(vcpu->arch.apic))
+		kvm_lapic_xapic_id_updated(vcpu->arch.apic);
+
  	atomic_set_release(&apic->vcpu->kvm->arch.apic_map_dirty, DIRTY);
  	kvm_recalculate_apic_map(vcpu->kvm);
  	kvm_apic_set_version(vcpu);

base-commit: 0b502152c0b8523f399bdb53096e2d620c5795b5

author maintainer
#14 lore raw

PATCH v3 6/28 KVM: x86: Move APIC access page helper to common x86 code

In reply to Sean Christopherson

Move the APIC access page allocation helper function to common x86 code, the allocation routine is virtually identical between APICv (VMX) and AVIC (SVM). Keep APICv's gfn_to_page() + put_page() sequence, which verifies that a backing page can be allocated, i.e. that the system isn't under heavy memory pressure. Forcing the backing page to be populated isn't strictly necessary, but skipping the effective prefetch only delays the inevitable.

---
 arch/x86/kvm/lapic.c    | 35 +++++++++++++++++++++++++++++++++++
 arch/x86/kvm/lapic.h    |  1 +
 arch/x86/kvm/svm/avic.c | 41 +++++++----------------------------------
 arch/x86/kvm/vmx/vmx.c  | 35 +----------------------------------
 4 files changed, 44 insertions(+), 68 deletions(-)
arch/x86/kvm/lapic.c +35 -0
index a02defa3f7b5..99994d2470a2 100644
@@ -2433,6 +2433,41 @@ void kvm_apic_update_apicv(struct kvm_vcpu *vcpu)
 }
 EXPORT_SYMBOL_GPL(kvm_apic_update_apicv);
 
+int kvm_alloc_apic_access_page(struct kvm *kvm)
+{
+	struct page *page;
+	void __user *hva;
+	int ret = 0;
+
+	mutex_lock(&kvm->slots_lock);
+	if (kvm->arch.apic_access_memslot_enabled)
+		goto out;
+
+	hva = __x86_set_memory_region(kvm, APIC_ACCESS_PAGE_PRIVATE_MEMSLOT,
+				      APIC_DEFAULT_PHYS_BASE, PAGE_SIZE);
+	if (IS_ERR(hva)) {
+		ret = PTR_ERR(hva);
+		goto out;
+	}
+
+	page = gfn_to_page(kvm, APIC_DEFAULT_PHYS_BASE >> PAGE_SHIFT);
+	if (is_error_page(page)) {
+		ret = -EFAULT;
+		goto out;
+	}
+
+	/*
+	 * Do not pin the page in memory, so that memory hot-unplug
+	 * is able to migrate it.
+	 */
+	put_page(page);
+	kvm->arch.apic_access_memslot_enabled = true;
+out:
+	mutex_unlock(&kvm->slots_lock);
+	return ret;
+}
+EXPORT_SYMBOL_GPL(kvm_alloc_apic_access_page);
+
 void kvm_lapic_reset(struct kvm_vcpu *vcpu, bool init_event)
 {
 	struct kvm_lapic *apic = vcpu->arch.apic;
arch/x86/kvm/lapic.h +1 -0
index 117a46df5cc1..6d06397683d0 100644
@@ -111,6 +111,7 @@ int kvm_apic_set_irq(struct kvm_vcpu *vcpu, struct kvm_lapic_irq *irq,
 		     struct dest_map *dest_map);
 int kvm_apic_local_deliver(struct kvm_lapic *apic, int lvt_type);
 void kvm_apic_update_apicv(struct kvm_vcpu *vcpu);
+int kvm_alloc_apic_access_page(struct kvm *kvm);
 
 bool kvm_irq_delivery_to_apic_fast(struct kvm *kvm, struct kvm_lapic *src,
 		struct kvm_lapic_irq *irq, int *r, struct dest_map *dest_map);
arch/x86/kvm/svm/avic.c +7 -34

53 lines not on this page · raw message

arch/x86/kvm/vmx/vmx.c +1 -34
index c9b49a09e6b5..b39095ef9bd7 100644
@@ -3818,39 +3818,6 @@ static void seg_setup(int seg)
 	vmcs_write32(sf->ar_bytes, ar);
 }
 
-static int alloc_apic_access_page(struct kvm *kvm)
-{
-	struct page *page;
-	void __user *hva;
-	int ret = 0;
-
-	mutex_lock(&kvm->slots_lock);
-	if (kvm->arch.apic_access_memslot_enabled)
-		goto out;
-	hva = __x86_set_memory_region(kvm, APIC_ACCESS_PAGE_PRIVATE_MEMSLOT,
-				      APIC_DEFAULT_PHYS_BASE, PAGE_SIZE);
-	if (IS_ERR(hva)) {
-		ret = PTR_ERR(hva);
-		goto out;
-	}
-
-	page = gfn_to_page(kvm, APIC_DEFAULT_PHYS_BASE >> PAGE_SHIFT);
-	if (is_error_page(page)) {
-		ret = -EFAULT;
-		goto out;
-	}
-
-	/*
-	 * Do not pin the page in memory, so that memory hot-unplug
-	 * is able to migrate it.
-	 */
-	put_page(page);
-	kvm->arch.apic_access_memslot_enabled = true;
-out:
-	mutex_unlock(&kvm->slots_lock);
-	return ret;
-}
-
 int allocate_vpid(void)
 {
 	int vpid;
@@ -7356,7 +7323,7 @@ static int vmx_vcpu_create(struct kvm_vcpu *vcpu)
 	vmx->loaded_vmcs = &vmx->vmcs01;
 
 	if (cpu_need_virtualize_apic_accesses(vcpu)) {
-		err = alloc_apic_access_page(vcpu->kvm);
+		err = kvm_alloc_apic_access_page(vcpu->kvm);
 		if (err)
 			goto free_vmcs;
 	}
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#15 lore raw

PATCH v3 7/28 KVM: x86: Inhibit APIC memslot if x2APIC and AVIC are enabled

In reply to Sean Christopherson

Free the APIC access page memslot if any vCPU enables x2APIC and SVM's AVIC is enabled to prevent accesses to the virtual APIC on vCPUs with
x2APIC enabled. On AMD, due to its "hybrid" mode where AVIC is enabled
when x2APIC is enabled even without x2AVIC support, keeping the APIC access page memslot results in the guest being able to access the virtual APIC page as x2APIC is fully emulated by KVM. I.e. hardware isn't aware that the guest is operating in x2APIC mode.

Intel doesn't suffer from the same issue as APICv has fully independent VMCS controls for xAPIC vs. x2APIC virtualization. Technically, KVM should provide bus error semantics and not memory semantics for the APIC page when x2APIC is enabled, but KVM already provides memory semantics in other scenarios, e.g. if APICv/AVIC is enabled and the APIC is hardware disabled (via APIC_BASE MSR).

Reserve an inhibit bit so that common code can detect whether or not the "x2APIC inhibit" applies, but use a dedicated flag to track the inhibit so that it doesn't need to be stripped from apicv_inhibit_reasons (since it's not a "full" inhibit).

Note, setting apic_access_memslot_inhibited without taking locks relies on it being sticky, and also relies on apic_access_memslot_enabled being set during vCPU creation (before kvm_vcpu_reset()). vCPUs can race to set the inhibit and delete the memslot, i.e. can get false positives, but can't false negatives as apic_access_memslot_enabled can't be toggle "on" once any vCPU reaches kvm_lapic_set_base().

Opportunistically drop the "can" while updating avic_activate_vmcb()'s comment, i.e. to state that KVM _does_ support the hybrid mode. Move the "Note:" down a line to conform to preferred kernel/KVM multi-line comment style.

Opportunistically update the apicv_update_lock comment, as it isn't actually used to protect apic_access_memslot_enabled (it's protected by slots_lock).

Fixes 0e311d33bfbe ("KVM: SVM: Introduce hybrid-AVIC mode")
---
 arch/x86/include/asm/kvm_host.h | 18 ++++++++++++++---
 arch/x86/kvm/lapic.c            | 34 +++++++++++++++++++++++++++++++--
 arch/x86/kvm/lapic.h            |  1 +
 arch/x86/kvm/svm/avic.c         | 15 ++++++++-------
 arch/x86/kvm/x86.c              |  7 +++++++
 5 files changed, 63 insertions(+), 12 deletions(-)
arch/x86/include/asm/kvm_host.h +15 -3
index 2c96c43c313a..6475c882b359 100644
@@ -1132,6 +1132,17 @@ enum kvm_apicv_inhibit {
 	 * AVIC is disabled because SEV doesn't support it.
 	 */
 	APICV_INHIBIT_REASON_SEV,
+
+	/*
+	 * Due to sharing page tables across vCPUs, the xAPIC memslot must be
+	 * deleted if any vCPU has x2APIC enabled as SVM doesn't provide fully
+	 * independent controls for AVIC vs. x2AVIC, and also because SVM
+	 * supports a "hybrid" AVIC mode for CPUs that support AVIC but not
+	 * x2AVIC.  Note, this isn't a "full" inhibit and is tracked separately.
+	 * AVIC can still be activated, but KVM must not create SPTEs for the
+	 * APIC base.  For simplicity, this is sticky.
+	 */
+	APICV_INHIBIT_REASON_X2APIC,
 };
 
 struct kvm_arch {
@@ -1169,10 +1180,11 @@ struct kvm_arch {
 	struct kvm_apic_map __rcu *apic_map;
 	atomic_t apic_map_dirty;
 
-	/* Protects apic_access_memslot_enabled and apicv_inhibit_reasons */
-	struct rw_semaphore apicv_update_lock;
-
 	bool apic_access_memslot_enabled;
+	bool apic_access_memslot_inhibited;
+
+	/* Protects apicv_inhibit_reasons */
+	struct rw_semaphore apicv_update_lock;
 	unsigned long apicv_inhibit_reasons;
 
 	gpa_t wall_clock;
arch/x86/kvm/lapic.c +32 -2
index 99994d2470a2..70f00eda75b2 100644
@@ -2394,9 +2394,26 @@ void kvm_lapic_set_base(struct kvm_vcpu *vcpu, u64 value)
 		}
 	}
 
-	if (((old_value ^ value) & X2APIC_ENABLE) && (value & X2APIC_ENABLE))
+	if (((old_value ^ value) & X2APIC_ENABLE) && (value & X2APIC_ENABLE)) {
 		kvm_apic_set_x2apic_id(apic, vcpu->vcpu_id);
 
+		/*
+		 * Mark the APIC memslot as inhibited if x2APIC is enabled and
+		 * the x2APIC inhibit is required.  The actual deletion of the
+		 * memslot is handled by vcpu_run() as SRCU may or may not be
+		 * held at this time, i.e. updating memslots isn't safe.  Don't
+		 * check apic_access_memslot_inhibited, this vCPU needs to
+		 * ensure the memslot is deleted before re-entering the guest,
+		 * i.e. needs to make the request even if the inhibit flag was
+		 * already set by a different vCPU.
+		 */
+		if (vcpu->kvm->arch.apic_access_memslot_enabled &&
+		    static_call(kvm_x86_check_apicv_inhibit_reasons)(APICV_INHIBIT_REASON_X2APIC)) {
+			vcpu->kvm->arch.apic_access_memslot_inhibited = true;
+			kvm_make_request(KVM_REQ_UNBLOCK, vcpu);
+		}
+	}
+
 	if ((old_value ^ value) & (MSR_IA32_APICBASE_ENABLE | X2APIC_ENABLE)) {
 		kvm_vcpu_update_apicv(vcpu);
 		static_call_cond(kvm_x86_set_virtual_apic_mode)(vcpu);
@@ -2440,7 +2457,8 @@ int kvm_alloc_apic_access_page(struct kvm *kvm)
 	int ret = 0;
 
 	mutex_lock(&kvm->slots_lock);
-	if (kvm->arch.apic_access_memslot_enabled)
+	if (kvm->arch.apic_access_memslot_enabled ||
+	    kvm->arch.apic_access_memslot_inhibited)
 		goto out;
 
 	hva = __x86_set_memory_region(kvm, APIC_ACCESS_PAGE_PRIVATE_MEMSLOT,
@@ -2468,6 +2486,18 @@ int kvm_alloc_apic_access_page(struct kvm *kvm)
 }
 EXPORT_SYMBOL_GPL(kvm_alloc_apic_access_page);
 
+void kvm_free_apic_access_page(struct kvm *kvm)
+{
+	mutex_lock(&kvm->slots_lock);
+
+	if (kvm->arch.apic_access_memslot_enabled) {
+		__x86_set_memory_region(kvm, APIC_ACCESS_PAGE_PRIVATE_MEMSLOT, 0, 0);
+		kvm->arch.apic_access_memslot_enabled = false;
+	}
+
+	mutex_unlock(&kvm->slots_lock);
+}
+
 void kvm_lapic_reset(struct kvm_vcpu *vcpu, bool init_event)
 {
 	struct kvm_lapic *apic = vcpu->arch.apic;
arch/x86/kvm/lapic.h +1 -0
index 6d06397683d0..e2271ffa7ac0 100644
@@ -112,6 +112,7 @@ int kvm_apic_set_irq(struct kvm_vcpu *vcpu, struct kvm_lapic_irq *irq,
 int kvm_apic_local_deliver(struct kvm_lapic *apic, int lvt_type);
 void kvm_apic_update_apicv(struct kvm_vcpu *vcpu);
 int kvm_alloc_apic_access_page(struct kvm *kvm);
+void kvm_free_apic_access_page(struct kvm *kvm);
 
 bool kvm_irq_delivery_to_apic_fast(struct kvm *kvm, struct kvm_lapic *src,
 		struct kvm_lapic_irq *irq, int *r, struct dest_map *dest_map);
arch/x86/kvm/svm/avic.c +8 -7
index 0424a5e664bb..8f9426f21bbf 100644
@@ -72,12 +72,12 @@ static void avic_activate_vmcb(struct vcpu_svm *svm)
 
 	vmcb->control.int_ctl |= AVIC_ENABLE_MASK;
 
-	/* Note:
-	 * KVM can support hybrid-AVIC mode, where KVM emulates x2APIC
-	 * MSR accesses, while interrupt injection to a running vCPU
-	 * can be achieved using AVIC doorbell. The AVIC hardware still
-	 * accelerate MMIO accesses, but this does not cause any harm
-	 * as the guest is not supposed to access xAPIC mmio when uses x2APIC.
+	/*
+	 * Note: KVM supports hybrid-AVIC mode, where KVM emulates x2APIC MSR
+	 * accesses, while interrupt injection to a running vCPU can be
+	 * achieved using AVIC doorbell.  KVM disables the APIC access page
+	 * (deletes the memslot) if any vCPU has x2APIC enabled, thus enabling
+	 * AVIC in hybrid mode activates only the doorbell mechanism.
 	 */
 	if (apic_x2apic_mode(svm->vcpu.arch.apic) &&
 	    avic_mode == AVIC_MODE_X2) {
@@ -987,7 +987,8 @@ bool avic_check_apicv_inhibit_reasons(enum kvm_apicv_inhibit reason)
 			  BIT(APICV_INHIBIT_REASON_BLOCKIRQ) |
 			  BIT(APICV_INHIBIT_REASON_SEV)      |
 			  BIT(APICV_INHIBIT_REASON_APIC_ID_MODIFIED) |
-			  BIT(APICV_INHIBIT_REASON_APIC_BASE_MODIFIED);
+			  BIT(APICV_INHIBIT_REASON_APIC_BASE_MODIFIED) |
+			  BIT(APICV_INHIBIT_REASON_X2APIC);
 
 	return supported & BIT(reason);
 }
arch/x86/kvm/x86.c +7 -0
index d7374d768296..aa5ab0c620de 100644
@@ -10705,6 +10705,13 @@ static int vcpu_run(struct kvm_vcpu *vcpu)
 			break;
 		}
 
+		if (vcpu->kvm->arch.apic_access_memslot_inhibited &&
+		    vcpu->kvm->arch.apic_access_memslot_enabled) {
+			kvm_vcpu_srcu_read_unlock(vcpu);
+			kvm_free_apic_access_page(vcpu->kvm);
+			kvm_vcpu_srcu_read_lock(vcpu);
+		}
+
 		if (__xfer_to_guest_mode_work_pending()) {
 			kvm_vcpu_srcu_read_unlock(vcpu);
 			r = xfer_to_guest_mode_handle_work(vcpu);
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
#16 lore raw

Re: [PATCH v3 07/28] KVM: x86: Inhibit APIC memslot if x2APIC and AVIC are enabled

Quoted 56 lines · Sean Christopherson

Free the APIC access page memslot if any vCPU enables x2APIC and SVM's AVIC is enabled to prevent accesses to the virtual APIC on vCPUs with
x2APIC enabled. On AMD, due to its "hybrid" mode where AVIC is enabled
when x2APIC is enabled even without x2AVIC support, keeping the APIC access page memslot results in the guest being able to access the virtual APIC page as x2APIC is fully emulated by KVM. I.e. hardware isn't aware that the guest is operating in x2APIC mode.

Intel doesn't suffer from the same issue as APICv has fully independent VMCS controls for xAPIC vs. x2APIC virtualization. Technically, KVM should provide bus error semantics and not memory semantics for the APIC page when x2APIC is enabled, but KVM already provides memory semantics in other scenarios, e.g. if APICv/AVIC is enabled and the APIC is hardware disabled (via APIC_BASE MSR).

Reserve an inhibit bit so that common code can detect whether or not the "x2APIC inhibit" applies, but use a dedicated flag to track the inhibit so that it doesn't need to be stripped from apicv_inhibit_reasons (since it's not a "full" inhibit).

Note, setting apic_access_memslot_inhibited without taking locks relies on it being sticky, and also relies on apic_access_memslot_enabled being set during vCPU creation (before kvm_vcpu_reset()). vCPUs can race to set the inhibit and delete the memslot, i.e. can get false positives, but can't false negatives as apic_access_memslot_enabled can't be toggle "on" once any vCPU reaches kvm_lapic_set_base().

Opportunistically drop the "can" while updating avic_activate_vmcb()'s comment, i.e. to state that KVM _does_ support the hybrid mode. Move the "Note:" down a line to conform to preferred kernel/KVM multi-line comment style.

Opportunistically update the apicv_update_lock comment, as it isn't actually used to protect apic_access_memslot_enabled (it's protected by slots_lock).

Fixes 0e311d33bfbe ("KVM: SVM: Introduce hybrid-AVIC mode")
---
 arch/x86/include/asm/kvm_host.h | 18 ++++++++++++++---
 arch/x86/kvm/lapic.c            | 34 +++++++++++++++++++++++++++++++--
 arch/x86/kvm/lapic.h            |  1 +
 arch/x86/kvm/svm/avic.c         | 15 ++++++++-------
 arch/x86/kvm/x86.c              |  7 +++++++
 5 files changed, 63 insertions(+), 12 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
index 2c96c43c313a..6475c882b359 100644
@@ -1132,6 +1132,17 @@ enum kvm_apicv_inhibit {
 	 * AVIC is disabled because SEV doesn't support it.
 	 */
 	APICV_INHIBIT_REASON_SEV,
+

arch/x86/include/asm/kvm_host.h enum kvm_apicv_inhibit { · line 1145

Quoted 10 lines
arch/x86/include/asm/kvm_host.h +10 -0
+	/*
+	 * Due to sharing page tables across vCPUs, the xAPIC memslot must be
+	 * deleted if any vCPU has x2APIC enabled as SVM doesn't provide fully
+	 * independent controls for AVIC vs. x2AVIC, and also because SVM
+	 * supports a "hybrid" AVIC mode for CPUs that support AVIC but not
+	 * x2AVIC.  Note, this isn't a "full" inhibit and is tracked separately.
+	 * AVIC can still be activated, but KVM must not create SPTEs for the
+	 * APIC base.  For simplicity, this is sticky.
+	 */
+	APICV_INHIBIT_REASON_X2APIC,

Hi Sean!

So assuming that I won't object to making it SVM specific (I still think that VMX should also inhibit this memslot because this is closer to x86 spec, but if you really want it this way, I won't fight over it):

I somewhat don't like this inhibit, because now it is used just to say 'I am AVIC'.

What do you think if you just move the code that removes the memslot to SVM, to avic_set_virtual_apic_mode?

arch/x86/include/asm/kvm_host.h struct kvm_arch { · line 1184

Quoted 12 lines
arch/x86/include/asm/kvm_host.h +1 -3
 };
 
 struct kvm_arch {
@@ -1169,10 +1180,11 @@ struct kvm_arch {
 	struct kvm_apic_map __rcu *apic_map;
 	atomic_t apic_map_dirty;
 
-	/* Protects apic_access_memslot_enabled and apicv_inhibit_reasons */
-	struct rw_semaphore apicv_update_lock;
-
 	bool apic_access_memslot_enabled;
+	bool apic_access_memslot_inhibited;

So the apic_access_memslot_enabled currently tracks if the memslot is enabled. As I see later in the patch when you free the memslot, you set it to false, which means that if a vCPU is created after that (it can happen in theory), the memslot will be created again :(

I say we need 'enabled', and 'allocated' booleans instead. Inhibit will set enabled to false, and then on next vcpu run, that will free the memslot.

when enabled == false, the code needs to be changed to not allocate it again.

Quoted 22 lines
arch/x86/include/asm/kvm_host.h +3 -0
+
+	/* Protects apicv_inhibit_reasons */
+	struct rw_semaphore apicv_update_lock;
 	unsigned long apicv_inhibit_reasons;
 
 	gpa_t wall_clock;
arch/x86/kvm/lapic.c +5 -1
index 99994d2470a2..70f00eda75b2 100644
@@ -2394,9 +2394,26 @@ void kvm_lapic_set_base(struct kvm_vcpu *vcpu, u64 value)
 		}
 	}
 
-	if (((old_value ^ value) & X2APIC_ENABLE) && (value & X2APIC_ENABLE))
+	if (((old_value ^ value) & X2APIC_ENABLE) && (value & X2APIC_ENABLE)) {
 		kvm_apic_set_x2apic_id(apic, vcpu->vcpu_id);
 
+		/*
+		 * Mark the APIC memslot as inhibited if x2APIC is enabled and
+		 * the x2APIC inhibit is required.  The actual deletion of the
+		 * memslot is handled by vcpu_run() as SRCU may or may not be

arch/x86/kvm/lapic.c void kvm_lapic_set_base(struct kvm_vcpu *vcpu, u64 value) · line 2413

Quoted 10 lines
arch/x86/kvm/lapic.c +10 -0
+		 * held at this time, i.e. updating memslots isn't safe.  Don't
+		 * check apic_access_memslot_inhibited, this vCPU needs to
+		 * ensure the memslot is deleted before re-entering the guest,
+		 * i.e. needs to make the request even if the inhibit flag was
+		 * already set by a different vCPU.
+		 */
+		if (vcpu->kvm->arch.apic_access_memslot_enabled &&
+		    static_call(kvm_x86_check_apicv_inhibit_reasons)(APICV_INHIBIT_REASON_X2APIC)) {
+			vcpu->kvm->arch.apic_access_memslot_inhibited = true;
+			kvm_make_request(KVM_REQ_UNBLOCK, vcpu);

You are about to remove the KVM_REQ_UNBLOCK in other patch series.

How about just raising KVM_REQ_APICV_UPDATE on current vCPU and having a special case in kvm_vcpu_update_apicv of

if (apic_access_memslot_enabled == false && apic_access_memslot_allocaed == true) {
drop srcu lock
free the memslot
take srcu lock
}

That wasn't possible to do with regular AVIC inhibit as I tried, because it has to be done before any vCPU re-enters the guest, so the KVM_REQ_APICV_UPDATE has to be raised on all vCPUs, and then a 'winning' vCPU would toggle the memslot, but that would cause a lot of impossible to solve races, thus I abandoned that approach back then.

Here the accuracy is not that critical so we can raise the request on current vCPU, as you do in your patch anyway, so it should work.

Quoted 88 lines
arch/x86/kvm/lapic.c +17 -1
+		}
+	}
+
 	if ((old_value ^ value) & (MSR_IA32_APICBASE_ENABLE | X2APIC_ENABLE)) {
 		kvm_vcpu_update_apicv(vcpu);
 		static_call_cond(kvm_x86_set_virtual_apic_mode)(vcpu);
@@ -2440,7 +2457,8 @@ int kvm_alloc_apic_access_page(struct kvm *kvm)
 	int ret = 0;
 
 	mutex_lock(&kvm->slots_lock);
-	if (kvm->arch.apic_access_memslot_enabled)
+	if (kvm->arch.apic_access_memslot_enabled ||
+	    kvm->arch.apic_access_memslot_inhibited)
 		goto out;
 
 	hva = __x86_set_memory_region(kvm, APIC_ACCESS_PAGE_PRIVATE_MEMSLOT,
@@ -2468,6 +2486,18 @@ int kvm_alloc_apic_access_page(struct kvm *kvm)
 }
 EXPORT_SYMBOL_GPL(kvm_alloc_apic_access_page);
 
+void kvm_free_apic_access_page(struct kvm *kvm)
+{
+	mutex_lock(&kvm->slots_lock);
+
+	if (kvm->arch.apic_access_memslot_enabled) {
+		__x86_set_memory_region(kvm, APIC_ACCESS_PAGE_PRIVATE_MEMSLOT, 0, 0);
+		kvm->arch.apic_access_memslot_enabled = false;
+	}
+
+	mutex_unlock(&kvm->slots_lock);
+}
+
 void kvm_lapic_reset(struct kvm_vcpu *vcpu, bool init_event)
 {
 	struct kvm_lapic *apic = vcpu->arch.apic;
arch/x86/kvm/lapic.h +1 -0
index 6d06397683d0..e2271ffa7ac0 100644
@@ -112,6 +112,7 @@ int kvm_apic_set_irq(struct kvm_vcpu *vcpu, struct kvm_lapic_irq *irq,
 int kvm_apic_local_deliver(struct kvm_lapic *apic, int lvt_type);
 void kvm_apic_update_apicv(struct kvm_vcpu *vcpu);
 int kvm_alloc_apic_access_page(struct kvm *kvm);
+void kvm_free_apic_access_page(struct kvm *kvm);
 
 bool kvm_irq_delivery_to_apic_fast(struct kvm *kvm, struct kvm_lapic *src,
 		struct kvm_lapic_irq *irq, int *r, struct dest_map *dest_map);
arch/x86/kvm/svm/avic.c +8 -7
index 0424a5e664bb..8f9426f21bbf 100644
@@ -72,12 +72,12 @@ static void avic_activate_vmcb(struct vcpu_svm *svm)
 
 	vmcb->control.int_ctl |= AVIC_ENABLE_MASK;
 
-	/* Note:
-	 * KVM can support hybrid-AVIC mode, where KVM emulates x2APIC
-	 * MSR accesses, while interrupt injection to a running vCPU
-	 * can be achieved using AVIC doorbell. The AVIC hardware still
-	 * accelerate MMIO accesses, but this does not cause any harm
-	 * as the guest is not supposed to access xAPIC mmio when uses x2APIC.
+	/*
+	 * Note: KVM supports hybrid-AVIC mode, where KVM emulates x2APIC MSR
+	 * accesses, while interrupt injection to a running vCPU can be
+	 * achieved using AVIC doorbell.  KVM disables the APIC access page
+	 * (deletes the memslot) if any vCPU has x2APIC enabled, thus enabling
+	 * AVIC in hybrid mode activates only the doorbell mechanism.
 	 */
 	if (apic_x2apic_mode(svm->vcpu.arch.apic) &&
 	    avic_mode == AVIC_MODE_X2) {
@@ -987,7 +987,8 @@ bool avic_check_apicv_inhibit_reasons(enum kvm_apicv_inhibit reason)
 			  BIT(APICV_INHIBIT_REASON_BLOCKIRQ) |
 			  BIT(APICV_INHIBIT_REASON_SEV)      |
 			  BIT(APICV_INHIBIT_REASON_APIC_ID_MODIFIED) |
-			  BIT(APICV_INHIBIT_REASON_APIC_BASE_MODIFIED);
+			  BIT(APICV_INHIBIT_REASON_APIC_BASE_MODIFIED) |
+			  BIT(APICV_INHIBIT_REASON_X2APIC);
 
 	return supported & BIT(reason);
 }
arch/x86/kvm/x86.c +0 -0
index d7374d768296..aa5ab0c620de 100644
@@ -10705,6 +10705,13 @@ static int vcpu_run(struct kvm_vcpu *vcpu)
 			break;
 		}
 

arch/x86/kvm/x86.c static int vcpu_run(struct kvm_vcpu *vcpu) · line 10717

Quoted 10 lines
arch/x86/kvm/x86.c +7 -0
+		if (vcpu->kvm->arch.apic_access_memslot_inhibited &&
+		    vcpu->kvm->arch.apic_access_memslot_enabled) {
+			kvm_vcpu_srcu_read_unlock(vcpu);
+			kvm_free_apic_access_page(vcpu->kvm);
+			kvm_vcpu_srcu_read_lock(vcpu);
+		}
+
 		if (__xfer_to_guest_mode_work_pending()) {
 			kvm_vcpu_srcu_read_unlock(vcpu);
 			r = xfer_to_guest_mode_handle_work(vcpu);

I will review the rest of the patch series on next Wednesday (after our holidays).
From a quick glance they look good to me.

Thanks,
Best regards,
Maxim Levitsky

author maintainer
#17 lore raw

Re: [PATCH v3 07/28] KVM: x86: Inhibit APIC memslot if x2APIC and AVIC are enabled

Quoted 23 lines · Maxim Levitsky
Quoted 9 lines · Sean Christopherson
arch/x86/include/asm/kvm_host.h +1 -0
index 2c96c43c313a..6475c882b359 100644
@@ -1132,6 +1132,17 @@ enum kvm_apicv_inhibit {
 	 * AVIC is disabled because SEV doesn't support it.
 	 */
 	APICV_INHIBIT_REASON_SEV,
+

arch/x86/include/asm/kvm_host.h enum kvm_apicv_inhibit { · line 1145

Quoted 10 lines
arch/x86/include/asm/kvm_host.h +10 -0
+	/*
+	 * Due to sharing page tables across vCPUs, the xAPIC memslot must be
+	 * deleted if any vCPU has x2APIC enabled as SVM doesn't provide fully
+	 * independent controls for AVIC vs. x2AVIC, and also because SVM
+	 * supports a "hybrid" AVIC mode for CPUs that support AVIC but not
+	 * x2AVIC.  Note, this isn't a "full" inhibit and is tracked separately.
+	 * AVIC can still be activated, but KVM must not create SPTEs for the
+	 * APIC base.  For simplicity, this is sticky.
+	 */
+	APICV_INHIBIT_REASON_X2APIC,

Hi Sean!

Quoted 3 lines

So assuming that I won't object to making it SVM specific (I still think that VMX should also inhibit this memslot because this is closer to x86 spec, but if you really want it this way, I won't fight over it):

Heh, I don't necessarily "want" it this way, it's more that I don't see a compelling reason to change KVM's behavior and risk silently causing a performance regression. If KVM didn't already have the "APIC base may have RAM semantics" quirk, and/or if this were the initial APICv implementation and thus no possible users, then I would probably also vote to give APICv the same treatment.

Quoted 5 lines

I somewhat don't like this inhibit, because now it is used just to say 'I am AVIC'.

What do you think if you just move the code that removes the memslot to SVM, to avic_set_virtual_apic_mode?

Suffers the same SRCU issue (see below) :-/

Given the SRCU problem, I'd prefer to keep the management of the memslot in common code, even though I agree it's a bit silly. And KVM_REQ_UNBLOCK is a perfect fit for dealing with the SRCU issue, i.e. handling this in AVIC code would require another hook on top of spreading the memslot management across x86 and SVM code.

Quoted 15 lines

arch/x86/include/asm/kvm_host.h struct kvm_arch { · line 1184

Quoted 9 lines
arch/x86/include/asm/kvm_host.h +1 -3
@@ -1169,10 +1180,11 @@ struct kvm_arch {
 	struct kvm_apic_map __rcu *apic_map;
 	atomic_t apic_map_dirty;
 
-	/* Protects apic_access_memslot_enabled and apicv_inhibit_reasons */
-	struct rw_semaphore apicv_update_lock;
-
 	bool apic_access_memslot_enabled;
+	bool apic_access_memslot_inhibited;

So the apic_access_memslot_enabled currently tracks if the memslot is enabled. As I see later in the patch when you free the memslot, you set it to false, which means that if a vCPU is created after that (it can happen in theory), the memslot will be created again :(

Quoted 4 lines

I say we need 'enabled', and 'allocated' booleans instead. Inhibit will set enabled to false, and then on next vcpu run, that will free the memslot.

when enabled == false, the code needs to be changed to not allocate it again.

This should be handled already. apic_access_memslot_enabled is toggled from true=>false if and only if apic_access_memslot_inhibited is set, and the "enabled" flag is protected by slots_lock. Thus, newly created vCPUs are guaranteed to either see apic_access_memslot_enabled==true or apic_access_memslot_inhibited==true.

int kvm_alloc_apic_access_page(struct kvm *kvm)

  {
	struct page *page;
	void __user *hva;
	int ret = 0;

	mutex_lock(&kvm->slots_lock);
	if (kvm->arch.apic_access_memslot_enabled ||
	    kvm->arch.apic_access_memslot_inhibited)  <=== prevents reallocation
		goto out;

out:

	mutex_unlock(&kvm->slots_lock);
	return ret;

}

That could be made more obvious by adding a WARN in kvm_free_apic_access_page(), i.e.

void kvm_free_apic_access_page(struct kvm *kvm)

  {
	WARN_ON_ONCE(!kvm->arch.apic_access_memslot_inhibited);

	mutex_lock(&kvm->slots_lock);

	if (kvm->arch.apic_access_memslot_enabled) {
		__x86_set_memory_region(kvm, APIC_ACCESS_PAGE_PRIVATE_MEMSLOT, 0, 0);
		kvm->arch.apic_access_memslot_enabled = false;
	}

	mutex_unlock(&kvm->slots_lock);

}

Quoted 18 lines
Quoted 18 lines
arch/x86/include/asm/kvm_host.h +3 -0
+
+	/* Protects apicv_inhibit_reasons */
+	struct rw_semaphore apicv_update_lock;
 	unsigned long apicv_inhibit_reasons;
 
 	gpa_t wall_clock;
arch/x86/kvm/lapic.c +1 -1
index 99994d2470a2..70f00eda75b2 100644
@@ -2394,9 +2394,26 @@ void kvm_lapic_set_base(struct kvm_vcpu *vcpu, u64 value)
 		}
 	}
 
-	if (((old_value ^ value) & X2APIC_ENABLE) && (value & X2APIC_ENABLE))
+	if (((old_value ^ value) & X2APIC_ENABLE) && (value & X2APIC_ENABLE)) {
 		kvm_apic_set_x2apic_id(apic, vcpu->vcpu_id);
 
Quoted 16 lines
Quoted 4 lines
arch/x86/kvm/lapic.c +4 -0
+		/*
+		 * Mark the APIC memslot as inhibited if x2APIC is enabled and
+		 * the x2APIC inhibit is required.  The actual deletion of the
+		 * memslot is handled by vcpu_run() as SRCU may or may not be

arch/x86/kvm/lapic.c void kvm_lapic_set_base(struct kvm_vcpu *vcpu, u64 value) · line 2413

Quoted 10 lines
arch/x86/kvm/lapic.c +10 -0
+		 * held at this time, i.e. updating memslots isn't safe.  Don't
+		 * check apic_access_memslot_inhibited, this vCPU needs to
+		 * ensure the memslot is deleted before re-entering the guest,
+		 * i.e. needs to make the request even if the inhibit flag was
+		 * already set by a different vCPU.
+		 */
+		if (vcpu->kvm->arch.apic_access_memslot_enabled &&
+		    static_call(kvm_x86_check_apicv_inhibit_reasons)(APICV_INHIBIT_REASON_X2APIC)) {
+			vcpu->kvm->arch.apic_access_memslot_inhibited = true;
+			kvm_make_request(KVM_REQ_UNBLOCK, vcpu);

You are about to remove the KVM_REQ_UNBLOCK in other patch series.

No, KVM_REQ_UNHALT is being removed. KVM_REQ_UNBLOCK needs to stay, although it has a rather weird name, e.g. KVM_REQ_WORK would probably be better.

Quoted 5 lines

How about just raising KVM_REQ_APICV_UPDATE on current vCPU and having a special case in kvm_vcpu_update_apicv of

if (apic_access_memslot_enabled == false && apic_access_memslot_allocaed == true) {
drop srcu lock

This was my initial thought as well, but the issue is that SRCU may or may not be held, and so the unlock+lock would need to be conditional. That's technically a solvable problem, as it's possible to detect if SRCU is held, but I really don't want to rely on kvm_vcpu.srcu_depth for anything other than proving that KVM doesn't screw up SRCU.

Quoted 3 lines

free the memslot
take srcu lock
}

#18 lore raw

Re: [PATCH v3 07/28] KVM: x86: Inhibit APIC memslot if x2APIC and AVIC are enabled

Quoted 40 lines · Sean Christopherson
Quoted 23 lines · Maxim Levitsky
Quoted 9 lines · Sean Christopherson
arch/x86/include/asm/kvm_host.h +1 -0
index 2c96c43c313a..6475c882b359 100644
@@ -1132,6 +1132,17 @@ enum kvm_apicv_inhibit {
 	 * AVIC is disabled because SEV doesn't support it.
 	 */
 	APICV_INHIBIT_REASON_SEV,
+

arch/x86/include/asm/kvm_host.h enum kvm_apicv_inhibit { · line 1145

Quoted 10 lines
arch/x86/include/asm/kvm_host.h +10 -0
+	/*
+	 * Due to sharing page tables across vCPUs, the xAPIC memslot must be
+	 * deleted if any vCPU has x2APIC enabled as SVM doesn't provide fully
+	 * independent controls for AVIC vs. x2AVIC, and also because SVM
+	 * supports a "hybrid" AVIC mode for CPUs that support AVIC but not
+	 * x2AVIC.  Note, this isn't a "full" inhibit and is tracked separately.
+	 * AVIC can still be activated, but KVM must not create SPTEs for the
+	 * APIC base.  For simplicity, this is sticky.
+	 */
+	APICV_INHIBIT_REASON_X2APIC,

Hi Sean!

Quoted 3 lines

So assuming that I won't object to making it SVM specific (I still think that VMX should also inhibit this memslot because this is closer to x86 spec, but if you really want it this way, I won't fight over it):

Heh, I don't necessarily "want" it this way, it's more that I don't see a compelling reason to change KVM's behavior and risk silently causing a performance regression. If KVM didn't already have the "APIC base may have RAM semantics" quirk, and/or if this were the initial APICv implementation and thus no possible users, then I would probably also vote to give APICv the same treatment.

Quoted 5 lines

I somewhat don't like this inhibit, because now it is used just to say 'I am AVIC'.

What do you think if you just move the code that removes the memslot to SVM, to avic_set_virtual_apic_mode?

Quoted 6 lines

Suffers the same SRCU issue (see below) :-/

Given the SRCU problem, I'd prefer to keep the management of the memslot in common code, even though I agree it's a bit silly. And KVM_REQ_UNBLOCK is a perfect fit for dealing with the SRCU issue, i.e. handling this in AVIC code would require another hook on top of spreading the memslot management across x86 and SVM code.

OK, I am not going to argue about this. But what about at least not using an inhibit bit for that but something else like a boolean, or maybe really add 'I am AVIC bit' or rather something like vcpu->arch.apicv_type enum?

Or we can make SVM code just call a common function - just put these in a function and call it from avic_set_virtual_apic_mode?

void kvm_disable_apicv_memslot(struct kvm_vcpu *vcpu)

{
	if (!vcpu->kvm->arch.apic_access_memslot_inhibited) {
            vcpu->kvm->arch.apic_access_memslot_inhibited = true;
            kvm_make_request(KVM_REQ_UNBLOCK, vcpu);
	}
}
Quoted 37 lines
Quoted 16 lines

arch/x86/include/asm/kvm_host.h struct kvm_arch { · line 1184

Quoted 10 lines
arch/x86/include/asm/kvm_host.h +1 -3
@@ -1169,10 +1180,11 @@ struct kvm_arch {
 	struct kvm_apic_map __rcu *apic_map;
 	atomic_t apic_map_dirty;
 
-	/* Protects apic_access_memslot_enabled and apicv_inhibit_reasons */
-	struct rw_semaphore apicv_update_lock;
-
 	bool apic_access_memslot_enabled;
+	bool apic_access_memslot_inhibited;

So the apic_access_memslot_enabled currently tracks if the memslot is enabled. As I see later in the patch when you free the memslot, you set it to false, which means that if a vCPU is created after that (it can happen in theory), the memslot will be created again :(

Quoted 4 lines

I say we need 'enabled', and 'allocated' booleans instead. Inhibit will set enabled to false, and then on next vcpu run, that will free the memslot.

when enabled == false, the code needs to be changed to not allocate it again.

This should be handled already. apic_access_memslot_enabled is toggled from true=>false if and only if apic_access_memslot_inhibited is set, and the "enabled" flag is protected by slots_lock. Thus, newly created vCPUs are guaranteed to either see apic_access_memslot_enabled==true or apic_access_memslot_inhibited==true.

int kvm_alloc_apic_access_page(struct kvm *kvm)

  {
	struct page *page;
	void __user *hva;
	int ret = 0;

	mutex_lock(&kvm->slots_lock);
	if (kvm->arch.apic_access_memslot_enabled ||
	    kvm->arch.apic_access_memslot_inhibited)  <=== prevents reallocation
		goto out;
Quoted 4 lines

out:

	mutex_unlock(&kvm->slots_lock);
	return ret;

}

Ah, you added this in previous patch which I didn't see, makes sense.

Quoted 2 lines

That could be made more obvious by adding a WARN in kvm_free_apic_access_page(), i.e.

Yep, a WARN_ON_ONCE unless in hot path, is almost always a good idea, so lets add it.

Quoted 50 lines

void kvm_free_apic_access_page(struct kvm *kvm)

  {
	WARN_ON_ONCE(!kvm->arch.apic_access_memslot_inhibited);

	mutex_lock(&kvm->slots_lock);

	if (kvm->arch.apic_access_memslot_enabled) {
		__x86_set_memory_region(kvm, APIC_ACCESS_PAGE_PRIVATE_MEMSLOT, 0, 0);
		kvm->arch.apic_access_memslot_enabled = false;
	}

	mutex_unlock(&kvm->slots_lock);

}

Quoted 18 lines
Quoted 18 lines
arch/x86/include/asm/kvm_host.h +3 -0
+
+	/* Protects apicv_inhibit_reasons */
+	struct rw_semaphore apicv_update_lock;
 	unsigned long apicv_inhibit_reasons;
 
 	gpa_t wall_clock;
arch/x86/kvm/lapic.c +1 -1
index 99994d2470a2..70f00eda75b2 100644
@@ -2394,9 +2394,26 @@ void kvm_lapic_set_base(struct kvm_vcpu *vcpu, u64 value)
 		}
 	}
 
-	if (((old_value ^ value) & X2APIC_ENABLE) && (value & X2APIC_ENABLE))
+	if (((old_value ^ value) & X2APIC_ENABLE) && (value & X2APIC_ENABLE)) {
 		kvm_apic_set_x2apic_id(apic, vcpu->vcpu_id);
 
Quoted 16 lines
Quoted 4 lines
arch/x86/kvm/lapic.c +4 -0
+		/*
+		 * Mark the APIC memslot as inhibited if x2APIC is enabled and
+		 * the x2APIC inhibit is required.  The actual deletion of the
+		 * memslot is handled by vcpu_run() as SRCU may or may not be

arch/x86/kvm/lapic.c void kvm_lapic_set_base(struct kvm_vcpu *vcpu, u64 value) · line 2413

Quoted 10 lines
arch/x86/kvm/lapic.c +10 -0
+		 * held at this time, i.e. updating memslots isn't safe.  Don't
+		 * check apic_access_memslot_inhibited, this vCPU needs to
+		 * ensure the memslot is deleted before re-entering the guest,
+		 * i.e. needs to make the request even if the inhibit flag was
+		 * already set by a different vCPU.
+		 */
+		if (vcpu->kvm->arch.apic_access_memslot_enabled &&
+		    static_call(kvm_x86_check_apicv_inhibit_reasons)(APICV_INHIBIT_REASON_X2APIC)) {
+			vcpu->kvm->arch.apic_access_memslot_inhibited = true;
+			kvm_make_request(KVM_REQ_UNBLOCK, vcpu);

You are about to remove the KVM_REQ_UNBLOCK in other patch series.

Quoted 2 lines

No, KVM_REQ_UNHALT is being removed. KVM_REQ_UNBLOCK needs to stay, although it has a rather weird name, e.g. KVM_REQ_WORK would probably be better.

Roger that!
And I guess lets rename it while we are at it.

Quoted 4 lines
Quoted 4 lines

How about just raising KVM_REQ_APICV_UPDATE on current vCPU and having a special case in kvm_vcpu_update_apicv of

Quoted 8 lines
Quoted 2 lines

if (apic_access_memslot_enabled == false && apic_access_memslot_allocaed == true) {
drop srcu lock

This was my initial thought as well, but the issue is that SRCU may or may not be held, and so the unlock+lock would need to be conditional. That's technically a solvable problem, as it's possible to detect if SRCU is held, but I really don't want to rely on kvm_vcpu.srcu_depth for anything other than proving that KVM doesn't screw up SRCU.

Why though? the KVM_REQ_APICV_UPDATE is only handled AFAIK in vcpu_enter_guest which drops the srcu lock few lines afterwards, and therefore the kvm_vcpu_update_apicv is always called with the lock held and it means that it can drop it for the duration of slot update.

The original issue we had was that we tried to drop the srcu lock in 'kvm_set_apicv_inhibit' which indeed is called from various places, with, or without the lock held.

Moving the memslot disable code to kvm_vcpu_update_apicv would actually solve that, but it was not possible because kvm_vcpu_update_apicv is called simultaneously on all vCPUs, and created various races, including toggling the memslot twice.

So if possible please take another look at using KVM_REQ_APICV_UPDATE instead of KVM_REQ_UNBLOCK.

Best regards,
Maxim Levitsky

Quoted 4 lines
Quoted 4 lines

free the memslot
take srcu lock
}

author maintainer
#19 lore raw

Re: [PATCH v3 07/28] KVM: x86: Inhibit APIC memslot if x2APIC and AVIC are enabled

Quoted 6 lines · Maxim Levitsky
Quoted 4 lines · Sean Christopherson

Given the SRCU problem, I'd prefer to keep the management of the memslot in common code, even though I agree it's a bit silly. And KVM_REQ_UNBLOCK is a perfect fit for dealing with the SRCU issue, i.e. handling this in AVIC code would require another hook on top of spreading the memslot management across x86 and SVM code.

Quoted 6 lines

OK, I am not going to argue about this. But what about at least not using an inhibit bit for that but something else like a boolean, or maybe really add 'I am AVIC bit' or rather something like vcpu->arch.apicv_type enum?

Or we can make SVM code just call a common function - just put these in a function and call it from avic_set_virtual_apic_mode?

The issue is that kvm_vcpu_update_apicv() is called from kvm_lapic_set_base(), which is the one that may or may not hold SRCU.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

You are about to remove the KVM_REQ_UNBLOCK in other patch series.

No, KVM_REQ_UNHALT is being removed. KVM_REQ_UNBLOCK needs to stay, although it has a rather weird name, e.g. KVM_REQ_WORK would probably be better.

Roger that!
And I guess lets rename it while we are at it.

I'll prep a patch.

Quoted 17 lines
Quoted 11 lines
Quoted 5 lines

How about just raising KVM_REQ_APICV_UPDATE on current vCPU and having a special case in kvm_vcpu_update_apicv of

if (apic_access_memslot_enabled == false && apic_access_memslot_allocaed == true) {
drop srcu lock

This was my initial thought as well, but the issue is that SRCU may or may not be held, and so the unlock+lock would need to be conditional. That's technically a solvable problem, as it's possible to detect if SRCU is held, but I really don't want to rely on kvm_vcpu.srcu_depth for anything other than proving that KVM doesn't screw up SRCU.

Why though? the KVM_REQ_APICV_UPDATE is only handled AFAIK in vcpu_enter_guest which drops the srcu lock few lines afterwards, and therefore the kvm_vcpu_update_apicv is always called with the lock held and it means that it can drop it for the duration of slot update.

Quoted 8 lines

The original issue we had was that we tried to drop the srcu lock in 'kvm_set_apicv_inhibit' which indeed is called from various places, with, or without the lock held.

Moving the memslot disable code to kvm_vcpu_update_apicv would actually solve that, but it was not possible because kvm_vcpu_update_apicv is called simultaneously on all vCPUs, and created various races, including toggling the memslot twice.

As above, kvm_vcpu_update_apicv() can be called without SRCU held. Oh, but that was a recent addition, commit 8fc9c7a3079e ("KVM: x86: Deactivate APICv on vCPU with APIC disabled"). I was wary of using KVM_REQ_APICV_UPDATE in kvm_lapic_set_base(), e.g. in case there was some dependency on updating _immediately, but since that's such a new addition I have no objection to switching to the request.

Similarly, is there a good reason for having nested_svm_vmexit() invoke kvm_vcpu_update_apicv() directly? I'm confused by the "so that other vCPUs can start to benefit from it right away". The nested inhibit is per-vCPU and so should only affect the current vCPU, no? I.e. for all intents and purposes, using a request should be functionally equivalent.

	/*
	 * Un-inhibit the AVIC right away, so that other vCPUs can start
	 * to benefit from it right away.
	 */
	if (kvm_apicv_activated(vcpu->kvm))
		kvm_vcpu_update_apicv(vcpu);
#20 lore raw

Re: [PATCH v3 07/28] KVM: x86: Inhibit APIC memslot if x2APIC and AVIC are enabled

Quoted 11 lines · Sean Christopherson
Quoted 10 lines · Maxim Levitsky
Quoted 4 lines · Sean Christopherson

Given the SRCU problem, I'd prefer to keep the management of the memslot in common code, even though I agree it's a bit silly. And KVM_REQ_UNBLOCK is a perfect fit for dealing with the SRCU issue, i.e. handling this in AVIC code would require another hook on top of spreading the memslot management across x86 and SVM code.

OK, I am not going to argue about this. But what about at least not using an inhibit bit for that but something else like a boolean, or maybe really add 'I am AVIC bit' or rather something like vcpu->arch.apicv_type enum?

Quoted 5 lines
Quoted 2 lines

Or we can make SVM code just call a common function - just put these in a function and call it from avic_set_virtual_apic_mode?

The issue is that kvm_vcpu_update_apicv() is called from kvm_lapic_set_base(), which is the one that may or may not hold SRCU.

Makes sense now.

Quoted 43 lines
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

You are about to remove the KVM_REQ_UNBLOCK in other patch series.

No, KVM_REQ_UNHALT is being removed. KVM_REQ_UNBLOCK needs to stay, although it has a rather weird name, e.g. KVM_REQ_WORK would probably be better.

Roger that!
And I guess lets rename it while we are at it.

I'll prep a patch.

Quoted 17 lines
Quoted 11 lines
Quoted 5 lines

How about just raising KVM_REQ_APICV_UPDATE on current vCPU and having a special case in kvm_vcpu_update_apicv of

if (apic_access_memslot_enabled == false && apic_access_memslot_allocaed == true) {
drop srcu lock

This was my initial thought as well, but the issue is that SRCU may or may not be held, and so the unlock+lock would need to be conditional. That's technically a solvable problem, as it's possible to detect if SRCU is held, but I really don't want to rely on kvm_vcpu.srcu_depth for anything other than proving that KVM doesn't screw up SRCU.

Why though? the KVM_REQ_APICV_UPDATE is only handled AFAIK in vcpu_enter_guest which drops the srcu lock few lines afterwards, and therefore the kvm_vcpu_update_apicv is always called with the lock held and it means that it can drop it for the duration of slot update.

Quoted 8 lines

The original issue we had was that we tried to drop the srcu lock in 'kvm_set_apicv_inhibit' which indeed is called from various places, with, or without the lock held.

Moving the memslot disable code to kvm_vcpu_update_apicv would actually solve that, but it was not possible because kvm_vcpu_update_apicv is called simultaneously on all vCPUs, and created various races, including toggling the memslot twice.

As above, kvm_vcpu_update_apicv() can be called without SRCU held. Oh, but that was a recent addition, commit 8fc9c7a3079e ("KVM: x86: Deactivate APICv on vCPU with APIC disabled"). I was wary of using KVM_REQ_APICV_UPDATE in kvm_lapic_set_base(), e.g. in case there was some dependency on updating _immediately, but since that's such a new addition I have no objection to switching to the request.

Quoted 5 lines

Similarly, is there a good reason for having nested_svm_vmexit() invoke kvm_vcpu_update_apicv() directly? I'm confused by the "so that other vCPUs can start to benefit from it right away". The nested inhibit is per-vCPU and so should only affect the current vCPU, no? I.e. for all intents and purposes, using a request should be functionally equivalent.

It is kind of the other way around:

The mere fact of switching to vmcb02 *inhibits* the AVIC on the current vCPU, but the AVIC inhibit is there only to set the is_running bits in the physid table and in IOMMU to prevent its *peers* to try and send interrupts to it via AVIC.

It is the reason why APICv doesn't need it - the posted interrupts still work just fine when a vCPU doens't use APICv, or uses a different posted interrupt vector when it uses the nested APICv.

So it makes sense to remove that inhibit as soon as possible that the peers could stop getting 'unaccellerated IPI' vmexits for nothing.

However back to the discussion, I don't think this is a problem.

We can just call both the kvm_vcpu_update_apicv() and a new function that does the memslot disable from KVM_REQ_APICV_UPDATE, then plain kvm_vcpu_update_apicv() won't need to drop the srcu lock.

It is pretty much the same that you proposed, just instead of piggybacking on KVM_REQ_UNBLOCK, I proposed to piggyback on KVM_REQ_APICV_UPDATE.

Best regards,
Maxim Levitsky

Quoted 8 lines
	/*
	 * Un-inhibit the AVIC right away, so that other vCPUs can start
	 * to benefit from it right away.
	 */
	if (kvm_apicv_activated(vcpu->kvm))
		kvm_vcpu_update_apicv(vcpu);
author maintainer
#21 lore raw

Re: [PATCH v3 07/28] KVM: x86: Inhibit APIC memslot if x2APIC and AVIC are enabled

Quoted 9 lines · Maxim Levitsky
Quoted 5 lines · Sean Christopherson

Similarly, is there a good reason for having nested_svm_vmexit() invoke kvm_vcpu_update_apicv() directly? I'm confused by the "so that other vCPUs can start to benefit from it right away". The nested inhibit is per-vCPU and so should only affect the current vCPU, no? I.e. for all intents and purposes, using a request should be functionally equivalent.

It is kind of the other way around:

Quoted 7 lines

The mere fact of switching to vmcb02 *inhibits* the AVIC on the current vCPU, but the AVIC inhibit is there only to set the is_running bits in the physid table and in IOMMU to prevent its *peers* to try and send interrupts to it via AVIC.

It is the reason why APICv doesn't need it - the posted interrupts still work just fine when a vCPU doens't use APICv, or uses a different posted interrupt vector when it uses the nested APICv.

Gotcha, the "other vCPUs" part is where I got confused.

Quoted 2 lines

So it makes sense to remove that inhibit as soon as possible that the peers could stop getting 'unaccellerated IPI' vmexits for nothing.

But practically speaking, the delay between the nested VM-Exit and servicing the request is minimal. Might be a moot point if nested AVIC is supported, i.e. an inline update may be "required" at that point.

Not a sticking point by any means, but if possible, it would be nice to have a single call site for the per-vCPU APICv update.

Quoted 8 lines

However back to the discussion, I don't think this is a problem.

We can just call both the kvm_vcpu_update_apicv() and a new function that does the memslot disable from KVM_REQ_APICV_UPDATE, then plain kvm_vcpu_update_apicv() won't need to drop the srcu lock.

It is pretty much the same that you proposed, just instead of piggybacking on KVM_REQ_UNBLOCK, I proposed to piggyback on KVM_REQ_APICV_UPDATE.

Yep, easy to do after converting the x2APIC toggling to use a request.

Thanks!

author maintainer
#22 lore raw

PATCH v3 8/28 KVM: SVM: Don't put/load AVIC when setting virtual APIC mode

In reply to Sean Christopherson

Move the VMCB updates from avic_refresh_apicv_exec_ctrl() into avic_set_virtual_apic_mode() and invert the dependency being said functions to avoid calling avic_vcpu_{load,put}() and avic_set_pi_irte_mode() when "only" setting the virtual APIC mode.

avic_set_virtual_apic_mode() is invoked from common x86 with preemption enabled, which makes avic_vcpu_{load,put}() unhappy. Luckily, calling those and updating IRTE stuff is unnecessary as the only reason avic_set_virtual_apic_mode() is called is to handle transitions between xAPIC and x2APIC that don't also toggle APICv activation. And if activation doesn't change, there's no need to fiddle with the physical APIC ID table or update IRTE.

The "full" refresh is guaranteed to be called if activation changes in this case as the only call to the "set" path is:

	kvm_vcpu_update_apicv(vcpu);
	static_call_cond(kvm_x86_set_virtual_apic_mode)(vcpu);

and kvm_vcpu_update_apicv() invokes the refresh if activation changes:

	if (apic->apicv_active == activate)
		goto out;

	apic->apicv_active = activate;
	kvm_apic_update_apicv(vcpu);
	static_call(kvm_x86_refresh_apicv_exec_ctrl)(vcpu);

Rename the helper to reflect that it is also called during "refresh".

  WARNING: CPU: 183 PID: 49186 at arch/x86/kvm/svm/avic.c:1081 avic_vcpu_put+0xde/0xf0 [kvm_amd]
  CPU: 183 PID: 49186 Comm: stable Tainted: G           O       6.0.0-smp--fcddbca45f0a-sink #34
  Hardware name: Google, Inc. Arcadia_IT_80/Arcadia_IT_80, BIOS 10.48.0 01/27/2022
  RIP: 0010:avic_vcpu_put+0xde/0xf0 [kvm_amd]
   avic_refresh_apicv_exec_ctrl+0x142/0x1c0 [kvm_amd]
   avic_set_virtual_apic_mode+0x5a/0x70 [kvm_amd]
   kvm_lapic_set_base+0x149/0x1a0 [kvm]
   kvm_set_apic_base+0x8f/0xd0 [kvm]
   kvm_set_msr_common+0xa3a/0xdc0 [kvm]
   svm_set_msr+0x364/0x6b0 [kvm_amd]
   __kvm_set_msr+0xb8/0x1c0 [kvm]
   kvm_emulate_wrmsr+0x58/0x1d0 [kvm]
   msr_interception+0x1c/0x30 [kvm_amd]
   svm_invoke_exit_handler+0x31/0x100 [kvm_amd]
   svm_handle_exit+0xfc/0x160 [kvm_amd]
   vcpu_enter_guest+0x21bb/0x23e0 [kvm]
   vcpu_run+0x92/0x450 [kvm]
   kvm_arch_vcpu_ioctl_run+0x43e/0x6e0 [kvm]
   kvm_vcpu_ioctl+0x559/0x620 [kvm]
Fixes 05c4fe8c1bd9 ("KVM: SVM: Refresh AVIC configuration when changing APIC mode")
Cc Suravee Suthikulpanit <[email protected]>
Cc Maxim Levitsky <[email protected]>
---
 arch/x86/kvm/svm/avic.c | 31 +++++++++++++++----------------
 arch/x86/kvm/svm/svm.c  |  2 +-
 arch/x86/kvm/svm/svm.h  |  2 +-
 3 files changed, 17 insertions(+), 18 deletions(-)
arch/x86/kvm/svm/avic.c +15 -16
index 8f9426f21bbf..535e35edce1d 100644
@@ -720,18 +720,6 @@ void avic_apicv_post_state_restore(struct kvm_vcpu *vcpu)
 	avic_handle_ldr_update(vcpu);
 }
 
-void avic_set_virtual_apic_mode(struct kvm_vcpu *vcpu)
-{
-	if (!lapic_in_kernel(vcpu) || avic_mode == AVIC_MODE_NONE)
-		return;
-
-	if (kvm_get_apic_mode(vcpu) == LAPIC_MODE_INVALID) {
-		WARN_ONCE(true, "Invalid local APIC state (vcpu_id=%d)", vcpu->vcpu_id);
-		return;
-	}
-	avic_refresh_apicv_exec_ctrl(vcpu);
-}
-
 static int avic_set_pi_irte_mode(struct kvm_vcpu *vcpu, bool activate)
 {
 	int ret = 0;
@@ -1074,17 +1062,18 @@ void avic_vcpu_put(struct kvm_vcpu *vcpu)
 	WRITE_ONCE(*(svm->avic_physical_id_cache), entry);
 }
 
-
-void avic_refresh_apicv_exec_ctrl(struct kvm_vcpu *vcpu)
+void avic_refresh_virtual_apic_mode(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
 	struct vmcb *vmcb = svm->vmcb01.ptr;
-	bool activated = kvm_vcpu_apicv_active(vcpu);
+
+	if (!lapic_in_kernel(vcpu) || avic_mode == AVIC_MODE_NONE)
+		return;
 
 	if (!enable_apicv)
 		return;
 
-	if (activated) {
+	if (kvm_vcpu_apicv_active(vcpu)) {
 		/**
 		 * During AVIC temporary deactivation, guest could update
 		 * APIC ID, DFR and LDR registers, which would not be trapped
@@ -1098,6 +1087,16 @@ void avic_refresh_apicv_exec_ctrl(struct kvm_vcpu *vcpu)
 		avic_deactivate_vmcb(svm);
 	}
 	vmcb_mark_dirty(vmcb, VMCB_AVIC);
+}
+
+void avic_refresh_apicv_exec_ctrl(struct kvm_vcpu *vcpu)
+{
+	bool activated = kvm_vcpu_apicv_active(vcpu);
+
+	if (!enable_apicv)
+		return;
+
+	avic_refresh_virtual_apic_mode(vcpu);
 
 	if (activated)
 		avic_vcpu_load(vcpu, vcpu->cpu);
arch/x86/kvm/svm/svm.c +1 -1
index f3813dbacb9f..2aa5069bafb2 100644
@@ -4807,7 +4807,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = {
 	.enable_nmi_window = svm_enable_nmi_window,
 	.enable_irq_window = svm_enable_irq_window,
 	.update_cr8_intercept = svm_update_cr8_intercept,
-	.set_virtual_apic_mode = avic_set_virtual_apic_mode,
+	.set_virtual_apic_mode = avic_refresh_virtual_apic_mode,
 	.refresh_apicv_exec_ctrl = avic_refresh_apicv_exec_ctrl,
 	.check_apicv_inhibit_reasons = avic_check_apicv_inhibit_reasons,
 	.apicv_post_state_restore = avic_apicv_post_state_restore,
arch/x86/kvm/svm/svm.h +1 -1
index 6a7686bf6900..7a95f50e80e7 100644
@@ -646,7 +646,7 @@ void avic_vcpu_blocking(struct kvm_vcpu *vcpu);
 void avic_vcpu_unblocking(struct kvm_vcpu *vcpu);
 void avic_ring_doorbell(struct kvm_vcpu *vcpu);
 unsigned long avic_vcpu_get_apicv_inhibit_reasons(struct kvm_vcpu *vcpu);
-void avic_set_virtual_apic_mode(struct kvm_vcpu *vcpu);
+void avic_refresh_virtual_apic_mode(struct kvm_vcpu *vcpu);
 
 
 /* sev.c */
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#23 lore raw

PATCH v3 9/28 KVM: SVM: Replace "avic_mode" enum with "x2avic_enabled" boolean

In reply to Sean Christopherson

Replace the "avic_mode" enum with a single bool to track whether or not x2AVIC is enabled. KVM already has "apicv_enabled" that tracks if any flavor of AVIC is enabled, i.e. AVIC_MODE_NONE and AVIC_MODE_X1 are redundant and unnecessary noise.

No functional change intended.

---
 arch/x86/kvm/svm/avic.c | 46 +++++++++++++++++++----------------------
 arch/x86/kvm/svm/svm.c  |  2 +-
 arch/x86/kvm/svm/svm.h  |  9 +-------
 3 files changed, 23 insertions(+), 34 deletions(-)
arch/x86/kvm/svm/avic.c +21 -25
index 535e35edce1d..84beef0edae3 100644
@@ -53,7 +53,7 @@ static DEFINE_HASHTABLE(svm_vm_data_hash, SVM_VM_DATA_HASH_BITS);
 static u32 next_vm_id = 0;
 static bool next_vm_id_wrapped = 0;
 static DEFINE_SPINLOCK(svm_vm_data_hash_lock);
-enum avic_modes avic_mode;
+bool x2avic_enabled;
 
 /*
  * This is a wrapper of struct amd_iommu_ir_data.
@@ -79,8 +79,7 @@ static void avic_activate_vmcb(struct vcpu_svm *svm)
 	 * (deletes the memslot) if any vCPU has x2APIC enabled, thus enabling
 	 * AVIC in hybrid mode activates only the doorbell mechanism.
 	 */
-	if (apic_x2apic_mode(svm->vcpu.arch.apic) &&
-	    avic_mode == AVIC_MODE_X2) {
+	if (x2avic_enabled && apic_x2apic_mode(svm->vcpu.arch.apic)) {
 		vmcb->control.int_ctl |= X2APIC_MODE_MASK;
 		vmcb->control.avic_physical_id |= X2AVIC_MAX_PHYSICAL_ID;
 		/* Disabling MSR intercept for x2APIC registers */
@@ -247,8 +246,8 @@ static u64 *avic_get_physical_id_entry(struct kvm_vcpu *vcpu,
 	u64 *avic_physical_id_table;
 	struct kvm_svm *kvm_svm = to_kvm_svm(vcpu->kvm);
 
-	if ((avic_mode == AVIC_MODE_X1 && index > AVIC_MAX_PHYSICAL_ID) ||
-	    (avic_mode == AVIC_MODE_X2 && index > X2AVIC_MAX_PHYSICAL_ID))
+	if ((!x2avic_enabled && index > AVIC_MAX_PHYSICAL_ID) ||
+	    (index > X2AVIC_MAX_PHYSICAL_ID))
 		return NULL;
 
 	avic_physical_id_table = page_address(kvm_svm->avic_physical_id_table_page);
@@ -262,8 +261,8 @@ static int avic_init_backing_page(struct kvm_vcpu *vcpu)
 	int id = vcpu->vcpu_id;
 	struct vcpu_svm *svm = to_svm(vcpu);
 
-	if ((avic_mode == AVIC_MODE_X1 && id > AVIC_MAX_PHYSICAL_ID) ||
-	    (avic_mode == AVIC_MODE_X2 && id > X2AVIC_MAX_PHYSICAL_ID))
+	if ((!x2avic_enabled && id > AVIC_MAX_PHYSICAL_ID) ||
+	    (id > X2AVIC_MAX_PHYSICAL_ID))
 		return -EINVAL;
 
 	if (!vcpu->arch.apic->regs)
@@ -1067,10 +1066,7 @@ void avic_refresh_virtual_apic_mode(struct kvm_vcpu *vcpu)
 	struct vcpu_svm *svm = to_svm(vcpu);
 	struct vmcb *vmcb = svm->vmcb01.ptr;
 
-	if (!lapic_in_kernel(vcpu) || avic_mode == AVIC_MODE_NONE)
-		return;
-
-	if (!enable_apicv)
+	if (!lapic_in_kernel(vcpu) || !enable_apicv)
 		return;
 
 	if (kvm_vcpu_apicv_active(vcpu)) {
@@ -1146,32 +1142,32 @@ bool avic_hardware_setup(struct kvm_x86_ops *x86_ops)
 	if (!npt_enabled)
 		return false;
 
+	/* AVIC is a prerequisite for x2AVIC. */
+	if (!boot_cpu_has(X86_FEATURE_AVIC) && !force_avic) {
+		if (boot_cpu_has(X86_FEATURE_X2AVIC)) {
+			pr_warn(FW_BUG "Cannot support x2AVIC due to AVIC is disabled");
+			pr_warn(FW_BUG "Try enable AVIC using force_avic option");
+		}
+		return false;
+	}
+
 	if (boot_cpu_has(X86_FEATURE_AVIC)) {
-		avic_mode = AVIC_MODE_X1;
 		pr_info("AVIC enabled\n");
 	} else if (force_avic) {
 		/*
 		 * Some older systems does not advertise AVIC support.
 		 * See Revision Guide for specific AMD processor for more detail.
 		 */
-		avic_mode = AVIC_MODE_X1;
 		pr_warn("AVIC is not supported in CPUID but force enabled");
 		pr_warn("Your system might crash and burn");
 	}
 
 	/* AVIC is a prerequisite for x2AVIC. */
-	if (boot_cpu_has(X86_FEATURE_X2AVIC)) {
-		if (avic_mode == AVIC_MODE_X1) {
-			avic_mode = AVIC_MODE_X2;
-			pr_info("x2AVIC enabled\n");
-		} else {
-			pr_warn(FW_BUG "Cannot support x2AVIC due to AVIC is disabled");
-			pr_warn(FW_BUG "Try enable AVIC using force_avic option");
-		}
-	}
+	x2avic_enabled = boot_cpu_has(X86_FEATURE_X2AVIC);
+	if (x2avic_enabled)
+		pr_info("x2AVIC enabled\n");
 
-	if (avic_mode != AVIC_MODE_NONE)
-		amd_iommu_register_ga_log_notifier(&avic_ga_log_notifier);
+	amd_iommu_register_ga_log_notifier(&avic_ga_log_notifier);
 
-	return !!avic_mode;
+	return true;
 }
arch/x86/kvm/svm/svm.c +1 -1
index 2aa5069bafb2..709f0b3e7a48 100644
@@ -821,7 +821,7 @@ void svm_set_x2apic_msr_interception(struct vcpu_svm *svm, bool intercept)
 	if (intercept == svm->x2avic_msrs_intercepted)
 		return;
 
-	if (avic_mode != AVIC_MODE_X2 ||
+	if (!x2avic_enabled ||
 	    !apic_x2apic_mode(svm->vcpu.arch.apic))
 		return;
 
arch/x86/kvm/svm/svm.h +1 -8
index 7a95f50e80e7..29c334a932c3 100644
@@ -35,14 +35,7 @@ extern u32 msrpm_offsets[MSRPM_OFFSETS] __read_mostly;
 extern bool npt_enabled;
 extern int vgif;
 extern bool intercept_smi;
-
-enum avic_modes {
-	AVIC_MODE_NONE = 0,
-	AVIC_MODE_X1,
-	AVIC_MODE_X2,
-};
-
-extern enum avic_modes avic_mode;
+extern bool x2avic_enabled;
 
 /*
  * Clean bits in VMCB.
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#24 lore raw

PATCH v3 10/28 KVM: SVM: Compute dest based on sender's x2APIC status for AVIC kick

In reply to Sean Christopherson

Compute the destination from ICRH using the sender's x2APIC status, not each (potential) target's x2APIC status.

Fixes c514d3a348ac ("KVM: SVM: Update avic_kick_target_vcpus to support 32-bit APIC ID")
Cc Li RongQing <[email protected]>
---
 arch/x86/kvm/svm/avic.c | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)
arch/x86/kvm/svm/avic.c +1 -7
index 84beef0edae3..e9aab8ecce83 100644
@@ -429,6 +429,7 @@ static int avic_kick_target_vcpus_fast(struct kvm *kvm, struct kvm_lapic *source
 static void avic_kick_target_vcpus(struct kvm *kvm, struct kvm_lapic *source,
 				   u32 icrl, u32 icrh, u32 index)
 {
+	u32 dest = apic_x2apic_mode(source) ? icrh : GET_XAPIC_DEST_FIELD(icrh);
 	unsigned long i;
 	struct kvm_vcpu *vcpu;
 
@@ -444,13 +445,6 @@ static void avic_kick_target_vcpus(struct kvm *kvm, struct kvm_lapic *source,
 	 * since entered the guest will have processed pending IRQs at VMRUN.
 	 */
 	kvm_for_each_vcpu(i, vcpu, kvm) {
-		u32 dest;
-
-		if (apic_x2apic_mode(vcpu->arch.apic))
-			dest = icrh;
-		else
-			dest = GET_XAPIC_DEST_FIELD(icrh);
-
 		if (kvm_apic_match_dest(vcpu, source, icrl & APIC_SHORT_MASK,
 					dest, icrl & APIC_DEST_MASK)) {
 			vcpu->arch.apic->irr_pending = true;
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#25 lore raw

PATCH v3 11/28 KVM: SVM: Fix x2APIC Logical ID calculation for avic_kick_target_vcpus_fast

In reply to Sean Christopherson

From: Suravee Suthikulpanit <[email protected]>

For X2APIC ID in cluster mode, the logical ID is bit [15:0].

Fixes 603ccef42ce9 ("KVM: x86: SVM: fix avic_kick_target_vcpus_fast")
Cc Maxim Levitsky <[email protected]>
---
 arch/x86/kvm/svm/avic.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/svm/avic.c +1 -1
index e9aab8ecce83..e35e9363e7ff 100644
@@ -356,7 +356,7 @@ static int avic_kick_target_vcpus_fast(struct kvm *kvm, struct kvm_lapic *source
 
 		if (apic_x2apic_mode(source)) {
 			/* 16 bit dest mask, 16 bit cluster id */
-			bitmap = dest & 0xFFFF0000;
+			bitmap = dest & 0xFFFF;
 			cluster = (dest >> 16) << 4;
 		} else if (kvm_lapic_get_reg(source, APIC_DFR) == APIC_DFR_FLAT) {
 			/* 8 bit dest mask*/
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#26 lore raw

PATCH v3 12/28 Revert "KVM: SVM: Use target APIC ID to complete x2AVIC IRQs when possible"

In reply to Sean Christopherson

Due to a likely mismerge of patches, KVM ended up with a superfluous commit to "enable" AVIC's fast path for x2AVIC mode. Even worse, the superfluous commit has several bugs and creates a nasty local shadow variable.

Rather than fix the bugs piece-by-piece[*] to achieve the same end result, revert the patch wholesale.

Opportunistically add a comment documenting the x2AVIC dependencies.

This reverts commit 8c9e639da435874fb845c4d296ce55664071ea7a.

[*] https://lore.kernel.org/all/[email protected]

Fixes 8c9e639da435 ("KVM: SVM: Use target APIC ID to complete x2AVIC IRQs when possible")
---
 arch/x86/kvm/svm/avic.c | 29 +++++++++++------------------
 1 file changed, 11 insertions(+), 18 deletions(-)
arch/x86/kvm/svm/avic.c +11 -18
index e35e9363e7ff..605c36569ddf 100644
@@ -378,7 +378,17 @@ static int avic_kick_target_vcpus_fast(struct kvm *kvm, struct kvm_lapic *source
 
 		logid_index = cluster + __ffs(bitmap);
 
-		if (!apic_x2apic_mode(source)) {
+		if (apic_x2apic_mode(source)) {
+			/*
+			 * For x2APIC, the logical APIC ID is a read-only value
+			 * that is derived from the x2APIC ID, thus the x2APIC
+			 * ID can be found by reversing the calculation (done
+			 * above).  Note, bits 31:20 of the x2APIC ID are not
+			 * propagated to the logical ID, but KVM limits the
+			 * x2APIC ID limited to KVM_MAX_VCPU_IDS.
+			 */
+			l1_physical_id = logid_index;
+		} else {
 			u32 *avic_logical_id_table =
 				page_address(kvm_svm->avic_logical_id_table_page);
 
@@ -393,23 +403,6 @@ static int avic_kick_target_vcpus_fast(struct kvm *kvm, struct kvm_lapic *source
 
 			l1_physical_id = logid_entry &
 					 AVIC_LOGICAL_ID_ENTRY_GUEST_PHYSICAL_ID_MASK;
-		} else {
-			/*
-			 * For x2APIC logical mode, cannot leverage the index.
-			 * Instead, calculate physical ID from logical ID in ICRH.
-			 */
-			int cluster = (icrh & 0xffff0000) >> 16;
-			int apic = ffs(icrh & 0xffff) - 1;
-
-			/*
-			 * If the x2APIC logical ID sub-field (i.e. icrh[15:0])
-			 * contains anything but a single bit, we cannot use the
-			 * fast path, because it is limited to a single vCPU.
-			 */
-			if (apic < 0 || icrh != (1 << apic))
-				return -EINVAL;
-
-			l1_physical_id = (cluster << 4) + apic;
 		}
 	}
 
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#27 lore raw

PATCH v3 13/28 KVM: SVM: Document that vCPU ID == APIC ID in AVIC kick fastpatch

In reply to Sean Christopherson

Document that AVIC is inhibited if any vCPU's APIC ID diverges from its vCPU ID, i.e. that there's no need to check for a destination match in the AVIC kick fast path.

Opportunistically tweak comments to remove "guest bug", as that suggests KVM is punting on error handling, which is not the case. Targeting a non-existent vCPU or no vCPUs _may_ be a guest software bug, but whether or not it's a guest bug is irrelevant. Such behavior is architecturally legal and thus needs to faithfully emulated by KVM (and it is).

---
 arch/x86/kvm/svm/avic.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
arch/x86/kvm/svm/avic.c +7 -3
index 605c36569ddf..40a1ea21074d 100644
@@ -368,8 +368,8 @@ static int avic_kick_target_vcpus_fast(struct kvm *kvm, struct kvm_lapic *source
 			cluster = (dest >> 4) << 2;
 		}
 
+		/* Nothing to do if there are no destinations in the cluster. */
 		if (unlikely(!bitmap))
-			/* guest bug: nobody to send the logical interrupt to */
 			return 0;
 
 		if (!is_power_of_2(bitmap))
@@ -397,7 +397,7 @@ static int avic_kick_target_vcpus_fast(struct kvm *kvm, struct kvm_lapic *source
 			if (WARN_ON_ONCE(index != logid_index))
 				return -EINVAL;
 
-			/* guest bug: non existing/reserved logical destination */
+			/* Nothing to do if the logical destination is invalid. */
 			if (unlikely(!(logid_entry & AVIC_LOGICAL_ID_ENTRY_VALID_MASK)))
 				return 0;
 
@@ -406,9 +406,13 @@ static int avic_kick_target_vcpus_fast(struct kvm *kvm, struct kvm_lapic *source
 		}
 	}
 
+	/*
+	 * KVM inhibits AVIC if any vCPU ID diverges from the vCPUs APIC ID,
+	 * i.e. APIC ID == vCPU ID.  Once again, nothing to do if the target
+	 * vCPU doesn't exist.
+	 */
 	target_vcpu = kvm_get_vcpu_by_id(kvm, l1_physical_id);
 	if (unlikely(!target_vcpu))
-		/* guest bug: non existing vCPU is a target of this IPI*/
 		return 0;
 
 	target_vcpu->arch.apic->irr_pending = true;
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#28 lore raw

PATCH v3 14/28 KVM: SVM: Add helper to perform final AVIC "kick" of single vCPU

In reply to Sean Christopherson

Add a helper to perform the final kick, two instances of the ICR decoding is one too many.

No functional change intended.

---
 arch/x86/kvm/svm/avic.c | 25 +++++++++++++------------
 1 file changed, 13 insertions(+), 12 deletions(-)
arch/x86/kvm/svm/avic.c +13 -12
index 40a1ea21074d..dd0e41d454a7 100644
@@ -317,6 +317,16 @@ void avic_ring_doorbell(struct kvm_vcpu *vcpu)
 	put_cpu();
 }
 
+
+static void avic_kick_vcpu(struct kvm_vcpu *vcpu, u32 icrl)
+{
+	vcpu->arch.apic->irr_pending = true;
+	svm_complete_interrupt_delivery(vcpu,
+					icrl & APIC_MODE_MASK,
+					icrl & APIC_INT_LEVELTRIG,
+					icrl & APIC_VECTOR_MASK);
+}
+
 /*
  * A fast-path version of avic_kick_target_vcpus(), which attempts to match
  * destination APIC ID to vCPU without looping through all vCPUs.
@@ -415,11 +425,7 @@ static int avic_kick_target_vcpus_fast(struct kvm *kvm, struct kvm_lapic *source
 	if (unlikely(!target_vcpu))
 		return 0;
 
-	target_vcpu->arch.apic->irr_pending = true;
-	svm_complete_interrupt_delivery(target_vcpu,
-					icrl & APIC_MODE_MASK,
-					icrl & APIC_INT_LEVELTRIG,
-					icrl & APIC_VECTOR_MASK);
+	avic_kick_vcpu(target_vcpu, icrl);
 	return 0;
 }
 
@@ -443,13 +449,8 @@ static void avic_kick_target_vcpus(struct kvm *kvm, struct kvm_lapic *source,
 	 */
 	kvm_for_each_vcpu(i, vcpu, kvm) {
 		if (kvm_apic_match_dest(vcpu, source, icrl & APIC_SHORT_MASK,
-					dest, icrl & APIC_DEST_MASK)) {
-			vcpu->arch.apic->irr_pending = true;
-			svm_complete_interrupt_delivery(vcpu,
-							icrl & APIC_MODE_MASK,
-							icrl & APIC_INT_LEVELTRIG,
-							icrl & APIC_VECTOR_MASK);
-		}
+					dest, icrl & APIC_DEST_MASK))
+			avic_kick_vcpu(vcpu, icrl);
 	}
 }
 
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#29 lore raw

PATCH v3 15/28 KVM: x86: Explicitly skip optimized logical map setup if vCPU's LDR==0

In reply to Sean Christopherson

Explicitly skip the optimized map setup if the vCPU's LDR is '0', i.e. if the vCPU will never respond to logical mode interrupts. KVM already skips setup in this case, but relies on kvm_apic_map_get_logical_dest() to generate mask==0. KVM still needs the mask=0 check as a non-zero LDR can yield mask==0 depending on the mode, but explicitly handling the LDR will make it simpler to clean up the logical mode tracking in the future.

No functional change intended.

---
 arch/x86/kvm/lapic.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
arch/x86/kvm/lapic.c +3 -1
index 70f00eda75b2..bf647af50031 100644
@@ -286,10 +286,12 @@ void kvm_recalculate_apic_map(struct kvm *kvm)
 			continue;
 
 		ldr = kvm_lapic_get_reg(apic, APIC_LDR);
+		if (!ldr)
+			continue;
 
 		if (apic_x2apic_mode(apic)) {
 			new->mode |= KVM_APIC_MODE_X2APIC;
-		} else if (ldr) {
+		} else {
 			ldr = GET_APIC_LOGICAL_ID(ldr);
 			if (kvm_lapic_get_reg(apic, APIC_DFR) == APIC_DFR_FLAT)
 				new->mode |= KVM_APIC_MODE_XAPIC_FLAT;
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#30 lore raw

PATCH v3 16/28 KVM: x86: Explicitly track all possibilities for APIC map's logical modes

In reply to Sean Christopherson

Track all possibilities for the optimized APIC map's logical modes instead of overloading the pseudo-bitmap and treating any "unknown" value as "invalid".

As documented by the now-stale comment above the mode values, the values did have meaning when the optimized map was originally added. That dependent logical was removed by commit e45115b62f9a ("KVM: x86: use physical LAPIC array for logical x2APIC"), but the obfuscated behavior and its comment were left behind.

Opportunistically rename "mode" to "logical_mode", partly to make it clear that the "disabled" case applies only to the logical map, but also to prove that there is no lurking code that expects "mode" to be a bitmap.

Functionally, this is a glorified nop.

---
 arch/x86/include/asm/kvm_host.h | 21 +++++++++++---------
 arch/x86/kvm/lapic.c            | 35 +++++++++++++++++++++++++--------
 2 files changed, 39 insertions(+), 17 deletions(-)
arch/x86/include/asm/kvm_host.h +12 -9
index 6475c882b359..6570b5d728ef 100644
@@ -955,19 +955,22 @@ struct kvm_arch_memory_slot {
 };
 
 /*
- * We use as the mode the number of bits allocated in the LDR for the
- * logical processor ID.  It happens that these are all powers of two.
- * This makes it is very easy to detect cases where the APICs are
- * configured for multiple modes; in that case, we cannot use the map and
- * hence cannot use kvm_irq_delivery_to_apic_fast either.
+ * Track the mode of the optimized logical map, as the rules for decoding the
+ * destination vary per mode.  Enabling the optimized logical map requires all
+ * software-enabled local APIs to be in the same mode, each addressable APIC to
+ * be mapped to only one MDA, and each MDA to map to at most one APIC.
  */
-#define KVM_APIC_MODE_XAPIC_CLUSTER          4
-#define KVM_APIC_MODE_XAPIC_FLAT             8
-#define KVM_APIC_MODE_X2APIC                16
+enum kvm_apic_logical_mode {
+	KVM_APIC_MODE_SW_DISABLED,
+	KVM_APIC_MODE_XAPIC_CLUSTER,
+	KVM_APIC_MODE_XAPIC_FLAT,
+	KVM_APIC_MODE_X2APIC,
+	KVM_APIC_MODE_MAP_DISABLED,
+};
 
 struct kvm_apic_map {
 	struct rcu_head rcu;
-	u8 mode;
+	enum kvm_apic_logical_mode logical_mode;
 	u32 max_apic_id;
 	union {
 		struct kvm_lapic *xapic_flat_map[8];
arch/x86/kvm/lapic.c +27 -8
index bf647af50031..84b7a1c1834d 100644
@@ -168,7 +168,12 @@ static bool kvm_use_posted_timer_interrupt(struct kvm_vcpu *vcpu)
 
 static inline bool kvm_apic_map_get_logical_dest(struct kvm_apic_map *map,
 		u32 dest_id, struct kvm_lapic ***cluster, u16 *mask) {
-	switch (map->mode) {
+	switch (map->logical_mode) {
+	case KVM_APIC_MODE_SW_DISABLED:
+		/* Arbitrarily use the flat map so that @cluster isn't NULL. */
+		*cluster = map->xapic_flat_map;
+		*mask = 0;
+		return true;
 	case KVM_APIC_MODE_X2APIC: {
 		u32 offset = (dest_id >> 16) * 16;
 		u32 max_apic_id = map->max_apic_id;
@@ -193,8 +198,10 @@ static inline bool kvm_apic_map_get_logical_dest(struct kvm_apic_map *map,
 		*cluster = map->xapic_cluster_map[(dest_id >> 4) & 0xf];
 		*mask = dest_id & 0xf;
 		return true;
+	case KVM_APIC_MODE_MAP_DISABLED:
+		return false;
 	default:
-		/* Not optimized. */
+		WARN_ON_ONCE(1);
 		return false;
 	}
 }
@@ -256,10 +263,12 @@ void kvm_recalculate_apic_map(struct kvm *kvm)
 		goto out;
 
 	new->max_apic_id = max_id;
+	new->logical_mode = KVM_APIC_MODE_SW_DISABLED;
 
 	kvm_for_each_vcpu(i, vcpu, kvm) {
 		struct kvm_lapic *apic = vcpu->arch.apic;
 		struct kvm_lapic **cluster;
+		enum kvm_apic_logical_mode logical_mode;
 		u16 mask;
 		u32 ldr;
 		u8 xapic_id;
@@ -282,7 +291,8 @@ void kvm_recalculate_apic_map(struct kvm *kvm)
 		if (!apic_x2apic_mode(apic) && !new->phys_map[xapic_id])
 			new->phys_map[xapic_id] = apic;
 
-		if (!kvm_apic_sw_enabled(apic))
+		if (new->logical_mode == KVM_APIC_MODE_MAP_DISABLED ||
+		    !kvm_apic_sw_enabled(apic))
 			continue;
 
 		ldr = kvm_lapic_get_reg(apic, APIC_LDR);
@@ -290,17 +300,26 @@ void kvm_recalculate_apic_map(struct kvm *kvm)
 			continue;
 
 		if (apic_x2apic_mode(apic)) {
-			new->mode |= KVM_APIC_MODE_X2APIC;
+			logical_mode = KVM_APIC_MODE_X2APIC;
 		} else {
 			ldr = GET_APIC_LOGICAL_ID(ldr);
 			if (kvm_lapic_get_reg(apic, APIC_DFR) == APIC_DFR_FLAT)
-				new->mode |= KVM_APIC_MODE_XAPIC_FLAT;
+				logical_mode = KVM_APIC_MODE_XAPIC_FLAT;
 			else
-				new->mode |= KVM_APIC_MODE_XAPIC_CLUSTER;
+				logical_mode = KVM_APIC_MODE_XAPIC_CLUSTER;
 		}
+		if (new->logical_mode != KVM_APIC_MODE_SW_DISABLED &&
+		    new->logical_mode != logical_mode) {
+			new->logical_mode = KVM_APIC_MODE_MAP_DISABLED;
+			continue;
+		}
+		new->logical_mode = logical_mode;
 
-		if (!kvm_apic_map_get_logical_dest(new, ldr, &cluster, &mask))
+		if (WARN_ON_ONCE(!kvm_apic_map_get_logical_dest(new, ldr,
+								&cluster, &mask))) {
+			new->logical_mode = KVM_APIC_MODE_MAP_DISABLED;
 			continue;
+		}
 
 		if (mask)
 			cluster[ffs(mask) - 1] = apic;
@@ -953,7 +972,7 @@ static bool kvm_apic_is_broadcast_dest(struct kvm *kvm, struct kvm_lapic **src,
 {
 	if (kvm->arch.x2apic_broadcast_quirk_disabled) {
 		if ((irq->dest_id == APIC_BROADCAST &&
-				map->mode != KVM_APIC_MODE_X2APIC))
+		     map->logical_mode != KVM_APIC_MODE_X2APIC))
 			return true;
 		if (irq->dest_id == X2APIC_BROADCAST)
 			return true;
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#31 lore raw

PATCH v3 17/28 KVM: x86: Skip redundant x2APIC logical mode optimized cluster setup

In reply to Sean Christopherson

Skip the optimized cluster[] setup for x2APIC logical mode, as KVM reuses the optimized map's phys_map[] and doesn't actually need to insert the target apic into the cluster[]. The LDR is derived from the x2APIC ID, and both are read-only in KVM, thus the vCPU's cluster[ldr] is guaranteed to be the same entry as the vCPU's phys_map[x2apic_id] entry.

Skipping the unnecessary setup will allow a future fix for aliased xAPIC logical IDs to simply require that cluster[ldr] is non-NULL, i.e. won't have to special case x2APIC.

Alternatively, the future check could allow "cluster[ldr] == apic", but that ends up being terribly confusing because cluster[ldr] is only set at the very end, i.e. it's only possible due to x2APIC's shenanigans.

Another alternative would be to send x2APIC down a separate path _after_ the calculation and then assert that all of the above, but the resulting code is rather messy, and it's arguably unnecessary since asserting that the actual LDR matches the expected LDR means that simply testing that interrupts are delivered correctly provides the same guarantees.

---
 arch/x86/kvm/lapic.c | 22 +++++++++++++++++-----
 1 file changed, 17 insertions(+), 5 deletions(-)
arch/x86/kvm/lapic.c +17 -5
index 84b7a1c1834d..7a39d7be4cc9 100644
@@ -166,6 +166,11 @@ static bool kvm_use_posted_timer_interrupt(struct kvm_vcpu *vcpu)
 	return kvm_can_post_timer_interrupt(vcpu) && vcpu->mode == IN_GUEST_MODE;
 }
 
+static inline u32 kvm_apic_calc_x2apic_ldr(u32 id)
+{
+	return ((id >> 4) << 16) | (1 << (id & 0xf));
+}
+
 static inline bool kvm_apic_map_get_logical_dest(struct kvm_apic_map *map,
 		u32 dest_id, struct kvm_lapic ***cluster, u16 *mask) {
 	switch (map->logical_mode) {
@@ -315,6 +320,18 @@ void kvm_recalculate_apic_map(struct kvm *kvm)
 		}
 		new->logical_mode = logical_mode;
 
+		/*
+		 * In x2APIC mode, the LDR is read-only and derived directly
+		 * from the x2APIC ID, thus is guaranteed to be addressable.
+		 * KVM reuses kvm_apic_map.phys_map to optimize logical mode
+		 * x2APIC interrupts by reversing the LDR calculation to get
+		 * cluster of APICs, i.e. no additional work is required.
+		 */
+		if (apic_x2apic_mode(apic)) {
+			WARN_ON_ONCE(ldr != kvm_apic_calc_x2apic_ldr(x2apic_id));
+			continue;
+		}
+
 		if (WARN_ON_ONCE(!kvm_apic_map_get_logical_dest(new, ldr,
 								&cluster, &mask))) {
 			new->logical_mode = KVM_APIC_MODE_MAP_DISABLED;
@@ -381,11 +398,6 @@ static inline void kvm_apic_set_dfr(struct kvm_lapic *apic, u32 val)
 	atomic_set_release(&apic->vcpu->kvm->arch.apic_map_dirty, DIRTY);
 }
 
-static inline u32 kvm_apic_calc_x2apic_ldr(u32 id)
-{
-	return ((id >> 4) << 16) | (1 << (id & 0xf));
-}
-
 static inline void kvm_apic_set_x2apic_id(struct kvm_lapic *apic, u32 id)
 {
 	u32 ldr = kvm_apic_calc_x2apic_ldr(id);
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#32 lore raw

PATCH v3 18/28 KVM: x86: Disable APIC logical map if logical ID covers multiple MDAs

In reply to Sean Christopherson

Disable the optimized APIC logical map if a logical ID covers multiple MDAs, i.e. if a vCPU has multiple bits set in its ID. In logical mode, events match if "ID & MDA != 0", i.e. creating an entry for only the first bit can cause interrupts to be missed.

Note, creating an entry for every bit is also wrong as KVM would generate IPIs for every matching bit. It would be possible to teach KVM to play nice with this edge case, but it is very much an edge case and probably not used in any real world OS, i.e. it's not worth optimizing.

Fixes 1e08ec4a130e ("KVM: optimize apic interrupt delivery")
---
 arch/x86/kvm/lapic.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
arch/x86/kvm/lapic.c +8 -2
index 7a39d7be4cc9..a12360fd4df6 100644
@@ -338,8 +338,14 @@ void kvm_recalculate_apic_map(struct kvm *kvm)
 			continue;
 		}
 
-		if (mask)
-			cluster[ffs(mask) - 1] = apic;
+		if (!mask)
+			continue;
+
+		if (!is_power_of_2(mask)) {
+			new->logical_mode = KVM_APIC_MODE_MAP_DISABLED;
+			continue;
+		}
+		cluster[ffs(mask) - 1] = apic;
 	}
 out:
 	old = rcu_dereference_protected(kvm->arch.apic_map,
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#33 lore raw

PATCH v3 19/28 KVM: x86: Disable APIC logical map if vCPUs are aliased in logical mode

In reply to Sean Christopherson

Disable the optimized APIC logical map if multiple vCPUs are aliased to the same logical ID. Architecturally, all CPUs whose logical ID matches the MDA are supposed to receive the interrupt; overwriting existing map entries can result in missed IPIs.

Fixes 1e08ec4a130e ("KVM: optimize apic interrupt delivery")
---
 arch/x86/kvm/lapic.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
arch/x86/kvm/lapic.c +3 -2
index a12360fd4df6..e447278d1986 100644
@@ -341,11 +341,12 @@ void kvm_recalculate_apic_map(struct kvm *kvm)
 		if (!mask)
 			continue;
 
-		if (!is_power_of_2(mask)) {
+		ldr = ffs(mask) - 1;
+		if (!is_power_of_2(mask) || cluster[ldr]) {
 			new->logical_mode = KVM_APIC_MODE_MAP_DISABLED;
 			continue;
 		}
-		cluster[ffs(mask) - 1] = apic;
+		cluster[ldr] = apic;
 	}
 out:
 	old = rcu_dereference_protected(kvm->arch.apic_map,
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#34 lore raw

PATCH v3 20/28 KVM: x86: Honor architectural behavior for aliased 8-bit APIC IDs

In reply to Sean Christopherson

Apply KVM's hotplug hack if and only if userspace has enabled 32-bit IDs for x2APIC. If 32-bit IDs are not enabled, disable the optimized map to honor x86 architectural behavior if multiple vCPUs shared a physical APIC ID. As called out in the changelog that added the hack, all CPUs whose (possibly truncated) APIC ID matches the target are supposed to receive the IPI.

KVM intentionally differs from real hardware, because real hardware (Knights Landing) does just "x2apic_id & 0xff" to decide whether to accept the interrupt in xAPIC mode and it can deliver one interrupt to more than one physical destination, e.g. 0x123 to 0x123 and 0x23.

Applying the hack even when x2APIC is not fully enabled means KVM doesn't correctly handle scenarios where the guest has aliased xAPIC IDs across multiple vCPUs, as only the vCPU with the lowest vCPU ID will receive any interrupts. It's extremely unlikely any real world guest aliase APIC IDs, or even modifies APIC IDs, but KVM's behavior is arbitrary, e.g. the lowest vCPU ID "wins" regardless of which vCPU is "aliasing" and which vCPU is "normal".

Furthermore, the hack is _not_ guaranteed to work! The hack works if and only if the optimized APIC map is successfully allocated. If the map allocation fails (unlikely), KVM will fall back to its unoptimized behavior, which _does_ honor the architectural behavior.

Pivot on 32-bit x2APIC IDs being enabled as that is required to take advantage of the hotplug hack (see kvm_apic_state_fixup()), i.e. won't break existing setups unless they are way, way off in the weeds.

And an entry in KVM's errata to document the hack. Alternatively, KVM could provide an actual x2APIC quirk and document the hack that way, but there's unlikely to ever be a use case for disabling the quirk. Go the errata route to avoid having to validate a quirk no one cares about.

Fixes 5bd5db385b3e ("KVM: x86: allow hotplug of VCPU with APIC ID over 0xff")
---
 Documentation/virt/kvm/x86/errata.rst | 11 ++++++
 arch/x86/kvm/lapic.c                  | 50 ++++++++++++++++++++++-----
 2 files changed, 52 insertions(+), 9 deletions(-)
Documentation/virt/kvm/x86/errata.rst +11 -0
index 410e0aa63493..49a05f24747b 100644
@@ -37,3 +37,14 @@ Nested virtualization features
 ------------------------------
 
 TBD
+
+x2APIC
+------
+When KVM_X2APIC_API_USE_32BIT_IDS is enabled, KVM activates a hack/quirk that
+allows sending events to a single vCPU using its x2APIC ID even if the target
+vCPU has legacy xAPIC enabled, e.g. to bring up hotplugged vCPUs via INIT-SIPI
+on VMs with > 255 vCPUs.  A side effect of the quirk is that, if multiple vCPUs
+have the same physical APIC ID, KVM will deliver events targeting that APIC ID
+only to the vCPU with the lowest vCPU ID.  If KVM_X2APIC_API_USE_32BIT_IDS is
+not enabled, KVM follows x86 architecture when processing interrupts (all vCPUs
+matching the target APIC ID receive the interrupt).
arch/x86/kvm/lapic.c +41 -9

67 lines not on this page · raw message

Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#35 lore raw

PATCH v3 21/28 KVM: x86: Inhibit APICv/AVIC if the optimized physical map is disabled

In reply to Sean Christopherson

Inhibit APICv/AVIC if the optimized physical map is disabled so that KVM KVM provides consistent APIC behavior if xAPIC IDs are aliased due to vcpu_id being truncated and the x2APIC hotplug hack isn't enabled. If the hotplug hack is disabled, events that are emulated by KVM will follow architectural behavior (all matching vCPUs receive events, even if the "match" is due to truncation), whereas APICv and AVIC will deliver events only to the first matching vCPU, i.e. the vCPU that matches without truncation.

Note, the "extra" inhibit is needed because KVM deliberately ignores mismatches due to truncation when applying the APIC_ID_MODIFIED inhibit so that large VMs (>255 vCPUs) can run with APICv/AVIC.

---
 arch/x86/include/asm/kvm_host.h |  6 ++++++
 arch/x86/kvm/lapic.c            | 13 ++++++++++++-
 arch/x86/kvm/svm/avic.c         |  1 +
 arch/x86/kvm/vmx/vmx.c          |  1 +
 4 files changed, 20 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +6 -0
index 6570b5d728ef..594674eefe59 100644
@@ -1097,6 +1097,12 @@ enum kvm_apicv_inhibit {
 	 */
 	APICV_INHIBIT_REASON_BLOCKIRQ,
 
+	/*
+	 * APICv is disabled because not all vCPUs have a 1:1 mapping between
+	 * APIC ID and vCPU, _and_ KVM is not applying its x2APIC hotplug hack.
+	 */
+	APICV_INHIBIT_REASON_PHYSICAL_ID_ALIASED,
+
 	/*
 	 * For simplicity, the APIC acceleration is inhibited
 	 * first time either APIC ID or APIC base are changed by the guest
arch/x86/kvm/lapic.c +12 -1
index b344ab52556e..4db162b1f0b1 100644
@@ -381,6 +381,16 @@ void kvm_recalculate_apic_map(struct kvm *kvm)
 		cluster[ldr] = apic;
 	}
 out:
+	/*
+	 * The optimized map is effectively KVM's internal version of APICv,
+	 * and all unwanted aliasing that results in disabling the optimized
+	 * map also applies to APICv.
+	 */
+	if (!new)
+		kvm_set_apicv_inhibit(kvm, APICV_INHIBIT_REASON_PHYSICAL_ID_ALIASED);
+	else
+		kvm_clear_apicv_inhibit(kvm, APICV_INHIBIT_REASON_PHYSICAL_ID_ALIASED);
+
 	old = rcu_dereference_protected(kvm->arch.apic_map,
 			lockdep_is_held(&kvm->arch.apic_map_lock));
 	rcu_assign_pointer(kvm->arch.apic_map, new);
@@ -2150,7 +2160,8 @@ static void kvm_lapic_xapic_id_updated(struct kvm_lapic *apic)
 	/*
 	 * Deliberately truncate the vCPU ID when detecting a modified APIC ID
 	 * to avoid false positives if the vCPU ID, i.e. x2APIC ID, is a 32-bit
-	 * value.
+	 * value.  If the wrap/truncation results in unwatned aliasing, APICv
+	 * will be inhibited as part of updating KVM's optimized APIC maps.
 	 */
 	if (kvm_xapic_id(apic) == (u8)apic->vcpu->vcpu_id)
 		return;
arch/x86/kvm/svm/avic.c +1 -0
index dd0e41d454a7..2908adc79ea6 100644
@@ -965,6 +965,7 @@ bool avic_check_apicv_inhibit_reasons(enum kvm_apicv_inhibit reason)
 			  BIT(APICV_INHIBIT_REASON_PIT_REINJ) |
 			  BIT(APICV_INHIBIT_REASON_BLOCKIRQ) |
 			  BIT(APICV_INHIBIT_REASON_SEV)      |
+			  BIT(APICV_INHIBIT_REASON_PHYSICAL_ID_ALIASED) |
 			  BIT(APICV_INHIBIT_REASON_APIC_ID_MODIFIED) |
 			  BIT(APICV_INHIBIT_REASON_APIC_BASE_MODIFIED) |
 			  BIT(APICV_INHIBIT_REASON_X2APIC);
arch/x86/kvm/vmx/vmx.c +1 -0
index b39095ef9bd7..0f9f8ae59f85 100644
@@ -7963,6 +7963,7 @@ static bool vmx_check_apicv_inhibit_reasons(enum kvm_apicv_inhibit reason)
 			  BIT(APICV_INHIBIT_REASON_ABSENT) |
 			  BIT(APICV_INHIBIT_REASON_HYPERV) |
 			  BIT(APICV_INHIBIT_REASON_BLOCKIRQ) |
+			  BIT(APICV_INHIBIT_REASON_PHYSICAL_ID_ALIASED) |
 			  BIT(APICV_INHIBIT_REASON_APIC_ID_MODIFIED) |
 			  BIT(APICV_INHIBIT_REASON_APIC_BASE_MODIFIED);
 
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#36 lore raw

PATCH v3 22/28 KVM: SVM: Inhibit AVIC if vCPUs are aliased in logical mode

In reply to Sean Christopherson

Inhibit SVM's AVIC if multiple vCPUs are aliased to the same logical ID. Architecturally, all CPUs whose logical ID matches the MDA are supposed to receive the interrupt; overwriting existing entries in AVIC's logical=>physical map can result in missed IPIs.

Fixes 18f40c53e10f ("svm: Add VMEXIT handlers for AVIC")
---
 arch/x86/include/asm/kvm_host.h | 6 ++++++
 arch/x86/kvm/lapic.c            | 5 +++++
 arch/x86/kvm/svm/avic.c         | 3 ++-
 3 files changed, 13 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +6 -0
index 594674eefe59..32c0bca052e3 100644
@@ -1152,6 +1152,12 @@ enum kvm_apicv_inhibit {
 	 * APIC base.  For simplicity, this is sticky.
 	 */
 	APICV_INHIBIT_REASON_X2APIC,
+
+	/*
+	 * AVIC is disabled because not all vCPUs with a valid LDR have a 1:1
+	 * mapping between logical ID and vCPU.
+	 */
+	APICV_INHIBIT_REASON_LOGICAL_ID_ALIASED,
 };
 
 struct kvm_arch {
arch/x86/kvm/lapic.c +5 -0
index 4db162b1f0b1..804d529d9bfb 100644
@@ -391,6 +391,11 @@ void kvm_recalculate_apic_map(struct kvm *kvm)
 	else
 		kvm_clear_apicv_inhibit(kvm, APICV_INHIBIT_REASON_PHYSICAL_ID_ALIASED);
 
+	if (!new || new->logical_mode == KVM_APIC_MODE_MAP_DISABLED)
+		kvm_set_apicv_inhibit(kvm, APICV_INHIBIT_REASON_LOGICAL_ID_ALIASED);
+	else
+		kvm_clear_apicv_inhibit(kvm, APICV_INHIBIT_REASON_LOGICAL_ID_ALIASED);
+
 	old = rcu_dereference_protected(kvm->arch.apic_map,
 			lockdep_is_held(&kvm->arch.apic_map_lock));
 	rcu_assign_pointer(kvm->arch.apic_map, new);
arch/x86/kvm/svm/avic.c +2 -1
index 2908adc79ea6..27d5abc15a91 100644
@@ -968,7 +968,8 @@ bool avic_check_apicv_inhibit_reasons(enum kvm_apicv_inhibit reason)
 			  BIT(APICV_INHIBIT_REASON_PHYSICAL_ID_ALIASED) |
 			  BIT(APICV_INHIBIT_REASON_APIC_ID_MODIFIED) |
 			  BIT(APICV_INHIBIT_REASON_APIC_BASE_MODIFIED) |
-			  BIT(APICV_INHIBIT_REASON_X2APIC);
+			  BIT(APICV_INHIBIT_REASON_X2APIC) |
+			  BIT(APICV_INHIBIT_REASON_LOGICAL_ID_ALIASED);
 
 	return supported & BIT(reason);
 }
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#37 lore raw

PATCH v3 23/28 KVM: SVM: Always update local APIC on writes to logical dest register

In reply to Sean Christopherson

Update the vCPU's local (virtual) APIC on LDR writes even if the write "fails". The APIC needs to recalc the optimized logical map even if the LDR is invalid or zero, e.g. if the guest clears its LDR, the optimized map will be left as is and the vCPU will receive interrupts using its old LDR.

Fixes 18f40c53e10f ("svm: Add VMEXIT handlers for AVIC")
---
 arch/x86/kvm/svm/avic.c | 11 ++++-------
 1 file changed, 4 insertions(+), 7 deletions(-)
arch/x86/kvm/svm/avic.c +4 -7
index 27d5abc15a91..2b640c73f447 100644
@@ -573,7 +573,7 @@ static void avic_invalidate_logical_id_entry(struct kvm_vcpu *vcpu)
 		clear_bit(AVIC_LOGICAL_ID_ENTRY_VALID_BIT, (unsigned long *)entry);
 }
 
-static int avic_handle_ldr_update(struct kvm_vcpu *vcpu)
+static void avic_handle_ldr_update(struct kvm_vcpu *vcpu)
 {
 	int ret = 0;
 	struct vcpu_svm *svm = to_svm(vcpu);
@@ -582,10 +582,10 @@ static int avic_handle_ldr_update(struct kvm_vcpu *vcpu)
 
 	/* AVIC does not support LDR update for x2APIC */
 	if (apic_x2apic_mode(vcpu->arch.apic))
-		return 0;
+		return;
 
 	if (ldr == svm->ldr_reg)
-		return 0;
+		return;
 
 	avic_invalidate_logical_id_entry(vcpu);
 
@@ -594,8 +594,6 @@ static int avic_handle_ldr_update(struct kvm_vcpu *vcpu)
 
 	if (!ret)
 		svm->ldr_reg = ldr;
-
-	return ret;
 }
 
 static void avic_handle_dfr_update(struct kvm_vcpu *vcpu)
@@ -617,8 +615,7 @@ static int avic_unaccel_trap_write(struct kvm_vcpu *vcpu)
 
 	switch (offset) {
 	case APIC_LDR:
-		if (avic_handle_ldr_update(vcpu))
-			return 0;
+		avic_handle_ldr_update(vcpu);
 		break;
 	case APIC_DFR:
 		avic_handle_dfr_update(vcpu);
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#38 lore raw

PATCH v3 24/28 KVM: SVM: Update svm->ldr_reg cache even if LDR is "bad"

In reply to Sean Christopherson

Update SVM's cache of the LDR even if the new value is "bad". Leaving stale information in the cache can result in KVM missing updates and/or invalidating the wrong entry, e.g. if avic_invalidate_logical_id_entry() is triggered after a different vCPU has "claimed" the old LDR.

Fixes 18f40c53e10f ("svm: Add VMEXIT handlers for AVIC")
---
 arch/x86/kvm/svm/avic.c | 17 +++++++----------
 1 file changed, 7 insertions(+), 10 deletions(-)
arch/x86/kvm/svm/avic.c +7 -10
index 2b640c73f447..4b6fc9d64f4d 100644
@@ -539,23 +539,24 @@ static u32 *avic_get_logical_id_entry(struct kvm_vcpu *vcpu, u32 ldr, bool flat)
 	return &logical_apic_id_table[index];
 }
 
-static int avic_ldr_write(struct kvm_vcpu *vcpu, u8 g_physical_id, u32 ldr)
+static void avic_ldr_write(struct kvm_vcpu *vcpu, u8 g_physical_id, u32 ldr)
 {
 	bool flat;
 	u32 *entry, new_entry;
 
+	if (!ldr)
+		return;
+
 	flat = kvm_lapic_get_reg(vcpu->arch.apic, APIC_DFR) == APIC_DFR_FLAT;
 	entry = avic_get_logical_id_entry(vcpu, ldr, flat);
 	if (!entry)
-		return -EINVAL;
+		return;
 
 	new_entry = READ_ONCE(*entry);
 	new_entry &= ~AVIC_LOGICAL_ID_ENTRY_GUEST_PHYSICAL_ID_MASK;
 	new_entry |= (g_physical_id & AVIC_LOGICAL_ID_ENTRY_GUEST_PHYSICAL_ID_MASK);
 	new_entry |= AVIC_LOGICAL_ID_ENTRY_VALID_MASK;
 	WRITE_ONCE(*entry, new_entry);
-
-	return 0;
 }
 
 static void avic_invalidate_logical_id_entry(struct kvm_vcpu *vcpu)
@@ -575,7 +576,6 @@ static void avic_invalidate_logical_id_entry(struct kvm_vcpu *vcpu)
 
 static void avic_handle_ldr_update(struct kvm_vcpu *vcpu)
 {
-	int ret = 0;
 	struct vcpu_svm *svm = to_svm(vcpu);
 	u32 ldr = kvm_lapic_get_reg(vcpu->arch.apic, APIC_LDR);
 	u32 id = kvm_xapic_id(vcpu->arch.apic);
@@ -589,11 +589,8 @@ static void avic_handle_ldr_update(struct kvm_vcpu *vcpu)
 
 	avic_invalidate_logical_id_entry(vcpu);
 
-	if (ldr)
-		ret = avic_ldr_write(vcpu, id, ldr);
-
-	if (!ret)
-		svm->ldr_reg = ldr;
+	svm->ldr_reg = ldr;
+	avic_ldr_write(vcpu, id, ldr);
 }
 
 static void avic_handle_dfr_update(struct kvm_vcpu *vcpu)
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#39 lore raw

PATCH v3 25/28 KVM: SVM: Require logical ID to be power-of-2 for AVIC entry

In reply to Sean Christopherson

Do not modify AVIC's logical ID table if the logical ID portion of the LDR is not a power-of-2, i.e. if the LDR has multiple bits set. Taking only the first bit means that KVM will fail to match MDAs that intersect with "higher" bits in the "ID"

The "ID" acts as a bitmap, but is referred to as an ID because theres an implicit, unenforced "requirement" that software only set one bit. This edge case is arguably out-of-spec behavior, but KVM cleanly handles it in all other cases, e.g. the optimized logical map (and AVIC!) is also disabled in this scenario.

Refactor the code to consolidate the checks, and so that the code looks more like avic_kick_target_vcpus_fast().

Fixes 18f40c53e10f ("svm: Add VMEXIT handlers for AVIC")
Cc Suravee Suthikulpanit <[email protected]>
Cc Maxim Levitsky <[email protected]>
---
 arch/x86/kvm/svm/avic.c | 30 +++++++++++++++---------------
 1 file changed, 15 insertions(+), 15 deletions(-)
arch/x86/kvm/svm/avic.c +15 -15
index 4b6fc9d64f4d..a9e4e09f83fc 100644
@@ -513,26 +513,26 @@ unsigned long avic_vcpu_get_apicv_inhibit_reasons(struct kvm_vcpu *vcpu)
 static u32 *avic_get_logical_id_entry(struct kvm_vcpu *vcpu, u32 ldr, bool flat)
 {
 	struct kvm_svm *kvm_svm = to_kvm_svm(vcpu->kvm);
-	int index;
 	u32 *logical_apic_id_table;
-	int dlid = GET_APIC_LOGICAL_ID(ldr);
+	u32 cluster, index;
 
-	if (!dlid)
-		return NULL;
+	ldr = GET_APIC_LOGICAL_ID(ldr);
 
-	if (flat) { /* flat */
-		index = ffs(dlid) - 1;
-		if (index > 7)
+	if (flat) {
+		cluster = 0;
+	} else {
+		cluster = (ldr >> 4) << 2;
+		if (cluster >= 0xf)
 			return NULL;
-	} else { /* cluster */
-		int cluster = (dlid & 0xf0) >> 4;
-		int apic = ffs(dlid & 0x0f) - 1;
-
-		if ((apic < 0) || (apic > 7) ||
-		    (cluster >= 0xf))
-			return NULL;
-		index = (cluster << 2) + apic;
+		ldr &= 0xf;
 	}
+	if (!ldr || !is_power_of_2(ldr))
+		return NULL;
+
+	index = __ffs(ldr);
+	if (WARN_ON_ONCE(index > 7))
+		return NULL;
+	index += (cluster << 2);
 
 	logical_apic_id_table = (u32 *) page_address(kvm_svm->avic_logical_id_table_page);
 
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#40 lore raw

PATCH v3 26/28 KVM: SVM: Handle multiple logical targets in AVIC kick fastpath

In reply to Sean Christopherson

Iterate over all target logical IDs in the AVIC kick fastpath instead of bailing if there is more than one target. Now that KVM inhibits AVIC if vCPUs aren't mapped 1:1 with logical IDs, each bit in the destination is guaranteed to match to at most one vCPU, i.e. iterating over the bitmap is guaranteed to kick each valid target exactly once.

---
 arch/x86/kvm/svm/avic.c | 112 ++++++++++++++++++++++------------------
 1 file changed, 63 insertions(+), 49 deletions(-)
arch/x86/kvm/svm/avic.c +63 -49
index a9e4e09f83fc..17e64b056e4e 100644
@@ -327,6 +327,50 @@ static void avic_kick_vcpu(struct kvm_vcpu *vcpu, u32 icrl)
 					icrl & APIC_VECTOR_MASK);
 }
 
+static void avic_kick_vcpu_by_physical_id(struct kvm *kvm, u32 physical_id,
+					  u32 icrl)
+{
+	/*
+	 * KVM inhibits AVIC if any vCPU ID diverges from the vCPUs APIC ID,
+	 * i.e. APIC ID == vCPU ID.
+	 */
+	struct kvm_vcpu *target_vcpu = kvm_get_vcpu_by_id(kvm, physical_id);
+
+	/* Once again, nothing to do if the target vCPU doesn't exist. */
+	if (unlikely(!target_vcpu))
+		return;
+
+	avic_kick_vcpu(target_vcpu, icrl);
+}
+
+static void avic_kick_vcpu_by_logical_id(struct kvm *kvm, u32 *avic_logical_id_table,
+					 u32 logid_index, u32 icrl)
+{
+	u32 physical_id;
+
+	if (avic_logical_id_table) {
+		u32 logid_entry = avic_logical_id_table[logid_index];
+
+		/* Nothing to do if the logical destination is invalid. */
+		if (unlikely(!(logid_entry & AVIC_LOGICAL_ID_ENTRY_VALID_MASK)))
+			return;
+
+		physical_id = logid_entry &
+			      AVIC_LOGICAL_ID_ENTRY_GUEST_PHYSICAL_ID_MASK;
+	} else {
+		/*
+		 * For x2APIC, the logical APIC ID is a read-only value that is
+		 * derived from the x2APIC ID, thus the x2APIC ID can be found
+		 * by reversing the calculation (stored in logid_index).  Note,
+		 * bits 31:20 of the x2APIC ID aren't propagated to the logical
+		 * ID, but KVM limits the x2APIC ID limited to KVM_MAX_VCPU_IDS.
+		 */
+		physical_id = logid_index;
+	}
+
+	avic_kick_vcpu_by_physical_id(kvm, physical_id, icrl);
+}
+
 /*
  * A fast-path version of avic_kick_target_vcpus(), which attempts to match
  * destination APIC ID to vCPU without looping through all vCPUs.
@@ -334,11 +378,10 @@ static void avic_kick_vcpu(struct kvm_vcpu *vcpu, u32 icrl)
 static int avic_kick_target_vcpus_fast(struct kvm *kvm, struct kvm_lapic *source,
 				       u32 icrl, u32 icrh, u32 index)
 {
-	u32 l1_physical_id, dest;
-	struct kvm_vcpu *target_vcpu;
 	int dest_mode = icrl & APIC_DEST_MASK;
 	int shorthand = icrl & APIC_SHORT_MASK;
 	struct kvm_svm *kvm_svm = to_kvm_svm(kvm);
+	u32 dest;
 
 	if (shorthand != APIC_DEST_NOSHORT)
 		return -EINVAL;
@@ -355,14 +398,14 @@ static int avic_kick_target_vcpus_fast(struct kvm *kvm, struct kvm_lapic *source
 		if (!apic_x2apic_mode(source) && dest == APIC_BROADCAST)
 			return -EINVAL;
 
-		l1_physical_id = dest;
-
-		if (WARN_ON_ONCE(l1_physical_id != index))
+		if (WARN_ON_ONCE(dest != index))
 			return -EINVAL;
 
+		avic_kick_vcpu_by_physical_id(kvm, dest, icrl);
 	} else {
-		u32 bitmap, cluster;
-		int logid_index;
+		u32 *avic_logical_id_table;
+		unsigned long bitmap, i;
+		u32 cluster;
 
 		if (apic_x2apic_mode(source)) {
 			/* 16 bit dest mask, 16 bit cluster id */
@@ -382,50 +425,21 @@ static int avic_kick_target_vcpus_fast(struct kvm *kvm, struct kvm_lapic *source
 		if (unlikely(!bitmap))
 			return 0;
 
-		if (!is_power_of_2(bitmap))
-			/* multiple logical destinations, use slow path */
-			return -EINVAL;
-
-		logid_index = cluster + __ffs(bitmap);
-
-		if (apic_x2apic_mode(source)) {
-			/*
-			 * For x2APIC, the logical APIC ID is a read-only value
-			 * that is derived from the x2APIC ID, thus the x2APIC
-			 * ID can be found by reversing the calculation (done
-			 * above).  Note, bits 31:20 of the x2APIC ID are not
-			 * propagated to the logical ID, but KVM limits the
-			 * x2APIC ID limited to KVM_MAX_VCPU_IDS.
-			 */
-			l1_physical_id = logid_index;
-		} else {
-			u32 *avic_logical_id_table =
-				page_address(kvm_svm->avic_logical_id_table_page);
-
-			u32 logid_entry = avic_logical_id_table[logid_index];
-
-			if (WARN_ON_ONCE(index != logid_index))
-				return -EINVAL;
-
-			/* Nothing to do if the logical destination is invalid. */
-			if (unlikely(!(logid_entry & AVIC_LOGICAL_ID_ENTRY_VALID_MASK)))
-				return 0;
-
-			l1_physical_id = logid_entry &
-					 AVIC_LOGICAL_ID_ENTRY_GUEST_PHYSICAL_ID_MASK;
-		}
+		if (apic_x2apic_mode(source))
+			avic_logical_id_table = NULL;
+		else
+			avic_logical_id_table = page_address(kvm_svm->avic_logical_id_table_page);
+
+		/*
+		 * AVIC is inhibited if vCPUs aren't mapped 1:1 with logical
+		 * IDs, thus each bit in the destination is guaranteed to map
+		 * to at most one vCPU.
+		 */
+		for_each_set_bit(i, &bitmap, 16)
+			avic_kick_vcpu_by_logical_id(kvm, avic_logical_id_table,
+						     cluster + i, icrl);
 	}
 
-	/*
-	 * KVM inhibits AVIC if any vCPU ID diverges from the vCPUs APIC ID,
-	 * i.e. APIC ID == vCPU ID.  Once again, nothing to do if the target
-	 * vCPU doesn't exist.
-	 */
-	target_vcpu = kvm_get_vcpu_by_id(kvm, l1_physical_id);
-	if (unlikely(!target_vcpu))
-		return 0;
-
-	avic_kick_vcpu(target_vcpu, icrl);
 	return 0;
 }
 
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#41 lore raw

PATCH v3 27/28 KVM: SVM: Ignore writes to Remote Read Data on AVIC write traps

In reply to Sean Christopherson

Drop writes to APIC_RRR, a.k.a. Remote Read Data Register, on AVIC unaccelerated write traps. The register is read-only and isn't emulated by KVM. Sending the register through kvm_apic_write_nodecode() will result in screaming when x2APIC is enabled due to the unexpected failure to retrieve the MSR (KVM expects that only "legal" accesses will trap).

Fixes 4d1d7942e36a ("KVM: SVM: Introduce logic to (de)activate x2AVIC mode")
---
 arch/x86/kvm/svm/avic.c | 3 +++
 1 file changed, 3 insertions(+)
arch/x86/kvm/svm/avic.c +3 -0
index 17e64b056e4e..953b1fd14b6d 100644
@@ -631,6 +631,9 @@ static int avic_unaccel_trap_write(struct kvm_vcpu *vcpu)
 	case APIC_DFR:
 		avic_handle_dfr_update(vcpu);
 		break;
+	case APIC_RRR:
+		/* Ignore writes to Read Remote Data, it's read-only. */
+		return 1;
 	default:
 		break;
 	}
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog
author maintainer
#42 lore raw

PATCH v3 28/28 Revert "KVM: SVM: Do not throw warning when calling avic_vcpu_load on a running vcpu"

In reply to Sean Christopherson

Turns out that some warnings exist for good reasons. Restore the warning in avic_vcpu_load() that guards against calling avic_vcpu_load() on a running vCPU now that KVM avoids doing so when switching between x2APIC and xAPIC. The entire point of the WARN is to highlight that KVM should not be reloading an AVIC.

Opportunistically convert the WARN_ON() to WARN_ON_ONCE() to avoid spamming the kernel if it does fire.

This reverts commit c0caeee65af3944b7b8abbf566e7cc1fae15c775.

---
 arch/x86/kvm/svm/avic.c | 1 +
 1 file changed, 1 insertion(+)
arch/x86/kvm/svm/avic.c +1 -0
index 953b1fd14b6d..35b0ef877e53 100644
@@ -1038,6 +1038,7 @@ void avic_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 		return;
 
 	entry = READ_ONCE(*(svm->avic_physical_id_cache));
+	WARN_ON_ONCE(entry & AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK);
 
 	entry &= ~AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK;
 	entry |= (h_physical_id & AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK);
Signature · 1 line
2.37.3.968.ga6b4b080e4-goog