KVM: Only use posted interrupts for Fixes/LowPrio MSIs
cover KVM: Only use posted interrupts for Fixes/LowPrio MSIs
The MSI-X descriptor has a "delivery mode" field which can be set to various different targets, such as "Fixed" (default), SMI, NMI or INIT.
Usually when we pass devices into guests, we only ever see this MSI-X descriptor configured as Fixed, so nobody realized that the other modes were broken when using posted interrupts.
With posted interrupts, we end up configuring these special modes just the same as a Fixed interrupt. That means instead of generating an SMI, we inject a normal GSI into the guest.
Of course, that if completely broken. These two patches attempt to fix the situation for x86 systems. If anyone has a great idea how to generalize the filtering though, I'm all ears.
Alex
Alexander Graf (2):
KVM: VMX: Disable posted interrupts for odd IRQs
KVM: SVM: Disable posted interrupts for odd IRQs
arch/x86/kvm/svm.c | 16 ++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 22 ++++++++++++++++++++++ 2 files changed, 38 insertions(+)
Signature · 12 lines
2.17.1 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879
PATCH 1/2 KVM: VMX: Disable posted interrupts for odd IRQs
We can easily route hardware interrupts directly into VM context when they target the "Fixed" or "LowPriority" delivery modes.
However, on modes such as "SMI" or "Init", we need to go via KVM code to actually put the vCPU into a different mode of operation, so we can not post the interrupt
Add code in the VMX PI logic to explicitly refuse to establish posted mappings for advanced IRQ deliver modes.
This fixes a bug I have with code which configures real hardware to inject virtual SMIs into my guest.
--- arch/x86/kvm/vmx/vmx.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)
arch/x86/kvm/vmx/vmx.c +22 -0
@@ -7401,6 +7401,28 @@ static int vmx_update_pi_irte(struct kvm *kvm, unsigned int host_irq,continue ; } +switch (irq.delivery_mode) { +case dest_Fixed: +case dest_LowestPrio: +break ; +default : +/* +* For non-trivial interrupt events, we need to go +* through the full KVM IRQ code, so refuse to take +* any direct PI assignments here. +*/ + + ret =irq_set_vcpu_affinity (host_irq,NULL ); +if (ret <0 ) { +printk (KERN_INFO +"failed to back to remapped mode, irq: %u\n" , + host_irq); + goto out; + } + +continue ; + } + vcpu_info.pi_desc_addr =__pa (vcpu_to_pi_desc (vcpu)); vcpu_info.vector = irq.vector;
Signature · 12 lines
2.17.1 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879
Re: [PATCH 1/2] KVM: VMX: Disable posted interrupts for odd IRQs
Quoted 10 lines · Alexander Graf
We can easily route hardware interrupts directly into VM context when they target the "Fixed" or "LowPriority" delivery modes.
However, on modes such as "SMI" or "Init", we need to go via KVM code to actually put the vCPU into a different mode of operation, so we can not post the interrupt
I would also mention in commit message that one can see this is also true in KVM’s vLAPIC code. i.e. __apic_accept_irq() can call kvm_x86_ops->deliver_posted_interrupt() only in case deliver-mode is either “Fixed” or “LowPriority”.
Quoted 8 lines
Add code in the VMX PI logic to explicitly refuse to establish posted mappings for advanced IRQ deliver modes.
This fixes a bug I have with code which configures real hardware to inject virtual SMIs into my guest.
Signed-off-by Alexander Graf <[email protected]>
With some small improvements I written inline below:
Quoted 19 lines
---
arch/x86/kvm/vmx/vmx.c| 22 ++++++++++++++++++++++1 file changed, 22 insertions(+)arch/x86/kvm/vmx/vmx.c +7
-0@@ -7401,6 +7401,28 @@ static int vmx_update_pi_irte(struct kvm *kvm, unsigned int host_irq,continue ; } +switch (irq.delivery_mode) { +case dest_Fixed: +case dest_LowestPrio: +break ; +default : +/* +* For non-trivial interrupt events, we need to go
arch/x86/kvm/vmx/vmx.c static int vmx_update_pi_irte(struct kvm *kvm, unsigned int host_irq, · line 7420
Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +10
-0+* through the full KVM IRQ code, so refuse to take +* any direct PI assignments here. +*/ + + ret =irq_set_vcpu_affinity (host_irq,NULL ); +if (ret <0 ) { +printk (KERN_INFO +"failed to back to remapped mode, irq: %u\n" , + host_irq); + goto out;
I recommend we will chose to print here a string that is different than the !kvm_intr_is_single_vcpu() case to make it easier to diagnose which case exactly failed.
-Liran
Quoted 24 lines
arch/x86/kvm/vmx/vmx.c +5
-0+ } + +continue ; + } + vcpu_info.pi_desc_addr =__pa (vcpu_to_pi_desc (vcpu)); vcpu_info.vector = irq.vector;Signature · 12 lines
2.17.1 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879
PATCH 2/2 KVM: SVM: Disable posted interrupts for odd IRQs
In reply to Alexander GrafWe can easily route hardware interrupts directly into VM context when they target the "Fixed" or "LowPriority" delivery modes.
However, on modes such as "SMI" or "Init", we need to go via KVM code to actually put the vCPU into a different mode of operation, so we can not post the interrupt
Add code in the SVM PI logic to explicitly refuse to establish posted mappings for advanced IRQ deliver modes.
This fixes a bug I have with code which configures real hardware to inject virtual SMIs into my guest.
--- arch/x86/kvm/svm.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
arch/x86/kvm/svm.c +16 -0
@@ -5266,6 +5266,21 @@ get_pi_vcpu_info(struct kvm *kvm, struct kvm_kernel_irq_routing_entry *e,return -1 ; } +switch (irq.delivery_mode) { +case dest_Fixed: +case dest_LowestPrio: +break ; +default : +/* +* For non-trivial interrupt events, we need to go +* through the full KVM IRQ code, so refuse to take +* any direct PI assignments here. +*/ +pr_debug ("SVM: %s: use legacy intr remap mode for irq %u\n" , + __func__, irq.vector); +return -1 ; + } +pr_debug ("SVM: %s: use GA mode for irq %u\n" , __func__, irq.vector); *svm =to_svm (vcpu);
@@ -5314,6 +5329,7 @@ static int svm_update_pi_irte(struct kvm *kvm, unsigned int host_irq,* 1. When cannot target interrupt to a specific vcpu. * 2. Unsetting posted interrupt. * 3. APIC virtialization is disabled for the vcpu. +* 4. IRQ has extended delivery mode (SMI, INIT, etc) */ if (!get_pi_vcpu_info (kvm, e, &vcpu_info, &svm) && set &&kvm_vcpu_apicv_active (&svm->vcpu)) {
Signature · 12 lines
2.17.1 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879
Re: [PATCH 2/2] KVM: SVM: Disable posted interrupts for odd IRQs
Quoted 11 lines · Alexander Graf
We can easily route hardware interrupts directly into VM context when they target the "Fixed" or "LowPriority" delivery modes.
However, on modes such as "SMI" or "Init", we need to go via KVM code to actually put the vCPU into a different mode of operation, so we can not post the interrupt
Quoted 7 lines
Add code in the SVM PI logic to explicitly refuse to establish posted mappings for advanced IRQ deliver modes.
This fixes a bug I have with code which configures real hardware to inject virtual SMIs into my guest.
Signed-off-by Alexander Graf <[email protected]>
Nit: I prefer to squash both commits into one that change both VMX & SVM. As it’s exactly the same change.
Quoted 17 lines
---
arch/x86/kvm/svm.c| 16 ++++++++++++++++1 file changed, 16 insertions(+)arch/x86/kvm/svm.c +5
-0@@ -5266,6 +5266,21 @@ get_pi_vcpu_info(struct kvm *kvm, struct kvm_kernel_irq_routing_entry *e,return -1 ; } +switch (irq.delivery_mode) { +case dest_Fixed: +case dest_LowestPrio: +break ; +default :
arch/x86/kvm/svm.c get_pi_vcpu_info(struct kvm *kvm, struct kvm_kernel_irq_routing_entry *e, · line 5283
Quoted 10 lines
arch/x86/kvm/svm.c +10
-0+/* +* For non-trivial interrupt events, we need to go +* through the full KVM IRQ code, so refuse to take +* any direct PI assignments here. +*/ +pr_debug ("SVM: %s: use legacy intr remap mode for irq %u\n" , + __func__, irq.vector); +return -1 ; + } +
Prefer changing printed string to something different than the !kvm_intr_is_single_vcpu() case. To assist debugging.
Having said that,
-Liran
Quoted 27 lines
arch/x86/kvm/svm.c +1
-0pr_debug ("SVM: %s: use GA mode for irq %u\n" , __func__, irq.vector); *svm =to_svm (vcpu);@@ -5314,6 +5329,7 @@ static int svm_update_pi_irte(struct kvm *kvm, unsigned int host_irq,* 1. When cannot target interrupt to a specific vcpu. * 2. Unsetting posted interrupt. * 3. APIC virtialization is disabled for the vcpu. +* 4. IRQ has extended delivery mode (SMI, INIT, etc) */ if (!get_pi_vcpu_info (kvm, e, &vcpu_info, &svm) && set &&kvm_vcpu_apicv_active (&svm->vcpu)) {Signature · 12 lines
2.17.1 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879
Re: [PATCH 2/2] KVM: SVM: Disable posted interrupts for odd IRQs
Quoted 14 lines · Liran Alon
Quoted 14 lines · Alexander Graf
We can easily route hardware interrupts directly into VM context when they target the "Fixed" or "LowPriority" delivery modes.
However, on modes such as "SMI" or "Init", we need to go via KVM code to actually put the vCPU into a different mode of operation, so we can not post the interrupt
Add code in the SVM PI logic to explicitly refuse to establish posted mappings for advanced IRQ deliver modes.
Quoted 7 lines
Quoted 4 lines
This fixes a bug I have with code which configures real hardware to inject virtual SMIs into my guest.
Signed-off-by Alexander Graf <[email protected]>Nit: I prefer to squash both commits into one that change both VMX & SVM. As it’s exactly the same change.
It's the same change (hence the same patch set), but they touch different files and so for bisectability it's still convenient to have them as different commits. I'd really prefer to have them stay separate.
Thanks a lot for the review! :)
Alex
Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879