Skip to contentSkip to conversation

KVM: Only use posted interrupts for Fixes/LowPrio MSIs

PATCH Superseded Alexander Graf 2 participants 6 messages Sep 3, 2019 – Sep 4, 2019

Subsystems (3)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

cover KVM: Only use posted interrupts for Fixes/LowPrio MSIs

The MSI-X descriptor has a "delivery mode" field which can be set to various different targets, such as "Fixed" (default), SMI, NMI or INIT.

Usually when we pass devices into guests, we only ever see this MSI-X descriptor configured as Fixed, so nobody realized that the other modes were broken when using posted interrupts.

With posted interrupts, we end up configuring these special modes just the same as a Fixed interrupt. That means instead of generating an SMI, we inject a normal GSI into the guest.

Of course, that if completely broken. These two patches attempt to fix the situation for x86 systems. If anyone has a great idea how to generalize the filtering though, I'm all ears.

Alex

Alexander Graf (2):
KVM: VMX: Disable posted interrupts for odd IRQs
KVM: SVM: Disable posted interrupts for odd IRQs

 arch/x86/kvm/svm.c     | 16 ++++++++++++++++
 arch/x86/kvm/vmx/vmx.c | 22 ++++++++++++++++++++++
 2 files changed, 38 insertions(+)
Signature · 12 lines
2.17.1




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879
#1 lore raw

PATCH 1/2 KVM: VMX: Disable posted interrupts for odd IRQs

We can easily route hardware interrupts directly into VM context when they target the "Fixed" or "LowPriority" delivery modes.

However, on modes such as "SMI" or "Init", we need to go via KVM code to actually put the vCPU into a different mode of operation, so we can not post the interrupt

Add code in the VMX PI logic to explicitly refuse to establish posted mappings for advanced IRQ deliver modes.

This fixes a bug I have with code which configures real hardware to inject virtual SMIs into my guest.

---
 arch/x86/kvm/vmx/vmx.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
arch/x86/kvm/vmx/vmx.c +22 -0
index 570a233e272b..d16c4ae8f685 100644
@@ -7401,6 +7401,28 @@ static int vmx_update_pi_irte(struct kvm *kvm, unsigned int host_irq,
 			continue;
 		}
 
+		switch (irq.delivery_mode) {
+		case dest_Fixed:
+		case dest_LowestPrio:
+			break;
+		default:
+			/*
+			 * For non-trivial interrupt events, we need to go
+			 * through the full KVM IRQ code, so refuse to take
+			 * any direct PI assignments here.
+			 */
+
+			ret = irq_set_vcpu_affinity(host_irq, NULL);
+			if (ret < 0) {
+				printk(KERN_INFO
+				   "failed to back to remapped mode, irq: %u\n",
+				   host_irq);
+				goto out;
+			}
+
+			continue;
+		}
+
 		vcpu_info.pi_desc_addr = __pa(vcpu_to_pi_desc(vcpu));
 		vcpu_info.vector = irq.vector;
 
Signature · 12 lines
2.17.1




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879
Reviewed-by on v1 1/2
#2 lore raw

Re: [PATCH 1/2] KVM: VMX: Disable posted interrupts for odd IRQs

Quoted 10 lines · Alexander Graf

We can easily route hardware interrupts directly into VM context when they target the "Fixed" or "LowPriority" delivery modes.

However, on modes such as "SMI" or "Init", we need to go via KVM code to actually put the vCPU into a different mode of operation, so we can not post the interrupt

I would also mention in commit message that one can see this is also true in KVM’s vLAPIC code. i.e. __apic_accept_irq() can call kvm_x86_ops->deliver_posted_interrupt() only in case deliver-mode is either “Fixed” or “LowPriority”.

Quoted 8 lines

Add code in the VMX PI logic to explicitly refuse to establish posted mappings for advanced IRQ deliver modes.

This fixes a bug I have with code which configures real hardware to inject virtual SMIs into my guest.

With some small improvements I written inline below:

Quoted 19 lines

---
arch/x86/kvm/vmx/vmx.c | 22 ++++++++++++++++++++++

1 file changed, 22 insertions(+)
arch/x86/kvm/vmx/vmx.c +7 -0
index 570a233e272b..d16c4ae8f685 100644
@@ -7401,6 +7401,28 @@ static int vmx_update_pi_irte(struct kvm *kvm, unsigned int host_irq,
 			continue;
 		}
 
+		switch (irq.delivery_mode) {
+		case dest_Fixed:
+		case dest_LowestPrio:
+			break;
+		default:
+			/*
+			 * For non-trivial interrupt events, we need to go

arch/x86/kvm/vmx/vmx.c static int vmx_update_pi_irte(struct kvm *kvm, unsigned int host_irq, · line 7420

Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +10 -0
+			 * through the full KVM IRQ code, so refuse to take
+			 * any direct PI assignments here.
+			 */
+
+			ret = irq_set_vcpu_affinity(host_irq, NULL);
+			if (ret < 0) {
+				printk(KERN_INFO
+				   "failed to back to remapped mode, irq: %u\n",
+				   host_irq);
+				goto out;

I recommend we will chose to print here a string that is different than the !kvm_intr_is_single_vcpu() case to make it easier to diagnose which case exactly failed.

-Liran

Quoted 24 lines
arch/x86/kvm/vmx/vmx.c +5 -0
+			}
+
+			continue;
+		}
+
 		vcpu_info.pi_desc_addr = __pa(vcpu_to_pi_desc(vcpu));
 		vcpu_info.vector = irq.vector;
 
Signature · 12 lines
2.17.1




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879
#3 lore raw

PATCH 2/2 KVM: SVM: Disable posted interrupts for odd IRQs

In reply to Alexander Graf

We can easily route hardware interrupts directly into VM context when they target the "Fixed" or "LowPriority" delivery modes.

However, on modes such as "SMI" or "Init", we need to go via KVM code to actually put the vCPU into a different mode of operation, so we can not post the interrupt

Add code in the SVM PI logic to explicitly refuse to establish posted mappings for advanced IRQ deliver modes.

This fixes a bug I have with code which configures real hardware to inject virtual SMIs into my guest.

---
 arch/x86/kvm/svm.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
arch/x86/kvm/svm.c +16 -0
index 1f220a85514f..9a6ea78c3239 100644
@@ -5266,6 +5266,21 @@ get_pi_vcpu_info(struct kvm *kvm, struct kvm_kernel_irq_routing_entry *e,
 		return -1;
 	}
 
+	switch (irq.delivery_mode) {
+	case dest_Fixed:
+	case dest_LowestPrio:
+		break;
+	default:
+		/*
+		 * For non-trivial interrupt events, we need to go
+		 * through the full KVM IRQ code, so refuse to take
+		 * any direct PI assignments here.
+		 */
+		pr_debug("SVM: %s: use legacy intr remap mode for irq %u\n",
+			 __func__, irq.vector);
+		return -1;
+	}
+
 	pr_debug("SVM: %s: use GA mode for irq %u\n", __func__,
 		 irq.vector);
 	*svm = to_svm(vcpu);
@@ -5314,6 +5329,7 @@ static int svm_update_pi_irte(struct kvm *kvm, unsigned int host_irq,
 		 * 1. When cannot target interrupt to a specific vcpu.
 		 * 2. Unsetting posted interrupt.
 		 * 3. APIC virtialization is disabled for the vcpu.
+		 * 4. IRQ has extended delivery mode (SMI, INIT, etc)
 		 */
 		if (!get_pi_vcpu_info(kvm, e, &vcpu_info, &svm) && set &&
 		    kvm_vcpu_apicv_active(&svm->vcpu)) {
Signature · 12 lines
2.17.1




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879
Reviewed-by on v1 2/2
#4 lore raw

Re: [PATCH 2/2] KVM: SVM: Disable posted interrupts for odd IRQs

Quoted 11 lines · Alexander Graf

We can easily route hardware interrupts directly into VM context when they target the "Fixed" or "LowPriority" delivery modes.

However, on modes such as "SMI" or "Init", we need to go via KVM code to actually put the vCPU into a different mode of operation, so we can not post the interrupt

Quoted 7 lines

Add code in the SVM PI logic to explicitly refuse to establish posted mappings for advanced IRQ deliver modes.

This fixes a bug I have with code which configures real hardware to inject virtual SMIs into my guest.

Nit: I prefer to squash both commits into one that change both VMX & SVM. As it’s exactly the same change.

Quoted 17 lines

---
arch/x86/kvm/svm.c | 16 ++++++++++++++++

1 file changed, 16 insertions(+)
arch/x86/kvm/svm.c +5 -0
index 1f220a85514f..9a6ea78c3239 100644
@@ -5266,6 +5266,21 @@ get_pi_vcpu_info(struct kvm *kvm, struct kvm_kernel_irq_routing_entry *e,
 		return -1;
 	}
 
+	switch (irq.delivery_mode) {
+	case dest_Fixed:
+	case dest_LowestPrio:
+		break;
+	default:

arch/x86/kvm/svm.c get_pi_vcpu_info(struct kvm *kvm, struct kvm_kernel_irq_routing_entry *e, · line 5283

Quoted 10 lines
arch/x86/kvm/svm.c +10 -0
+		/*
+		 * For non-trivial interrupt events, we need to go
+		 * through the full KVM IRQ code, so refuse to take
+		 * any direct PI assignments here.
+		 */
+		pr_debug("SVM: %s: use legacy intr remap mode for irq %u\n",
+			 __func__, irq.vector);
+		return -1;
+	}
+

Prefer changing printed string to something different than the !kvm_intr_is_single_vcpu() case. To assist debugging.

Having said that,

-Liran

Quoted 27 lines
arch/x86/kvm/svm.c +1 -0
 	pr_debug("SVM: %s: use GA mode for irq %u\n", __func__,
 		 irq.vector);
 	*svm = to_svm(vcpu);
@@ -5314,6 +5329,7 @@ static int svm_update_pi_irte(struct kvm *kvm, unsigned int host_irq,
 		 * 1. When cannot target interrupt to a specific vcpu.
 		 * 2. Unsetting posted interrupt.
 		 * 3. APIC virtialization is disabled for the vcpu.
+		 * 4. IRQ has extended delivery mode (SMI, INIT, etc)
 		 */
 		if (!get_pi_vcpu_info(kvm, e, &vcpu_info, &svm) && set &&
 		    kvm_vcpu_apicv_active(&svm->vcpu)) {
Signature · 12 lines
2.17.1




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879
#5 lore raw

Re: [PATCH 2/2] KVM: SVM: Disable posted interrupts for odd IRQs

Quoted 14 lines · Liran Alon
Quoted 14 lines · Alexander Graf

We can easily route hardware interrupts directly into VM context when they target the "Fixed" or "LowPriority" delivery modes.

However, on modes such as "SMI" or "Init", we need to go via KVM code to actually put the vCPU into a different mode of operation, so we can not post the interrupt

Add code in the SVM PI logic to explicitly refuse to establish posted mappings for advanced IRQ deliver modes.

Quoted 7 lines
Quoted 4 lines

This fixes a bug I have with code which configures real hardware to inject virtual SMIs into my guest.

Nit: I prefer to squash both commits into one that change both VMX & SVM. As it’s exactly the same change.

It's the same change (hence the same patch set), but they touch different files and so for bisectability it's still convenient to have them as different commits. I'd really prefer to have them stay separate.

Thanks a lot for the review! :)

Alex

Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879