Skip to contentSkip to conversation

virtio/pci: Serialize legacy ISR and INTx updates

PATCH Applied Xie Bo 2 participants 2 messages Jul 29 – Aug 1 KVM

lore
Share & tools
lore

Xie Bo

author
#0 lore raw

PATCH 1/1 virtio/pci: Serialize legacy ISR and INTx updates

The legacy virtio-pci interrupt state is represented by two pieces of
state: vpci->isr and the legacy INTx line level. Backend threads set ISR bits and assert INTx when queue or configuration changes occur, while vCPU threads read and clear the ISR and deassert INTx.

Those operations currently race. One possible interleaving is:

vCPU thread backend thread
----------- --------------
read old ISR value
set ISR queue bit assert INTx
clear ISR
deassert INTx

The guest can then miss the completion because no ISR bit remains and the line is no longer asserted. A guest waiting for an interrupt-driven virtqueue can sleep indefinitely despite completed host-side work.

Fix this by adding a per-virtio-pci device mutex and using it for all legacy ISR/INTx state transitions: queue notification, configuration notification, legacy ISR read-clear, and modern ISR read-clear. MSI-X/MSI delivery continues to bypass the legacy ISR path.

Fixes 2108c86d0623 ("virtio/pci: Signal INTx interrupts as level instead of edge")
---
 include/kvm/virtio-pci.h | 3 +++
 virtio/pci-legacy.c      | 4 +++-
 virtio/pci-modern.c      | 4 +++-
 virtio/pci.c             | 5 +++++
 4 files changed, 14 insertions(+), 2 deletions(-)
include/kvm/virtio-pci.h +3 -0
index 143028b..1fb931e 100644
@@ -2,6 +2,7 @@
 #define KVM__VIRTIO_PCI_H
 
 #include "kvm/devices.h"
+#include "kvm/mutex.h"
 #include "kvm/pci.h"
 #include "kvm/virtio.h"
 
@@ -37,6 +38,8 @@ struct virtio_pci {
 	bool			signal_msi;
 	u8			status;
 	u8			isr;
+	/* Serializes legacy ISR bits and INTx line state. */
+	struct mutex		isr_lock;
 	u32			device_features_sel;
 	u32			driver_features_sel;
 
virtio/pci-legacy.c +3 -1
index 02a8f8c..3038a1e 100644
@@ -59,9 +59,11 @@ static bool virtio_pci__data_in(struct kvm_cpu *vcpu, struct virtio_device *vdev
 		ioport__write8(data, vpci->status);
 		break;
 	case VIRTIO_PCI_ISR:
+		mutex_lock(&vpci->isr_lock);
 		ioport__write8(data, vpci->isr);
-		kvm__irq_line(kvm, vpci->legacy_irq_line, VIRTIO_IRQ_LOW);
 		vpci->isr = 0;
+		kvm__irq_line(kvm, vpci->legacy_irq_line, VIRTIO_IRQ_LOW);
+		mutex_unlock(&vpci->isr_lock);
 		break;
 	default:
 		ret = virtio_pci__specific_data_in(kvm, vdev, data, size, offset);
virtio/pci-modern.c +3 -1
index 888afa5..2b234a7 100644
@@ -246,9 +246,11 @@ static bool virtio_pci__isr_read(struct virtio_device *vdev,
 	if (WARN_ON(offset - VPCI_CFG_ISR_START != 0))
 		return false;
 
+	mutex_lock(&vpci->isr_lock);
 	ioport__write8(data, vpci->isr);
-	kvm__irq_line(vpci->kvm, vpci->legacy_irq_line, VIRTIO_IRQ_LOW);
 	vpci->isr = 0;
+	kvm__irq_line(vpci->kvm, vpci->legacy_irq_line, VIRTIO_IRQ_LOW);
+	mutex_unlock(&vpci->isr_lock);
 
 	return 0;
 }
virtio/pci.c +5 -0
index 8a34cec..4852ce3 100644
@@ -242,8 +242,10 @@ int virtio_pci__signal_vq(struct kvm *kvm, struct virtio_device *vdev, u32 vq)
 		else
 			kvm__irq_trigger(kvm, vpci->gsis[vq]);
 	} else {
+		mutex_lock(&vpci->isr_lock);
 		vpci->isr |= VIRTIO_PCI_ISR_QUEUE;
 		kvm__irq_line(kvm, vpci->legacy_irq_line, VIRTIO_IRQ_HIGH);
+		mutex_unlock(&vpci->isr_lock);
 	}
 	return 0;
 }
@@ -266,8 +268,10 @@ int virtio_pci__signal_config(struct kvm *kvm, struct virtio_device *vdev)
 		else
 			kvm__irq_trigger(kvm, vpci->config_gsi);
 	} else {
+		mutex_lock(&vpci->isr_lock);
 		vpci->isr |= VIRTIO_PCI_ISR_CONFIG;
 		kvm__irq_line(kvm, vpci->legacy_irq_line, VIRTIO_IRQ_HIGH);
+		mutex_unlock(&vpci->isr_lock);
 	}
 
 	return 0;
@@ -346,6 +350,7 @@ int virtio_pci__init(struct kvm *kvm, void *dev, struct virtio_device *vdev,
 
 	vpci->kvm = kvm;
 	vpci->dev = dev;
+	mutex_init(&vpci->isr_lock);
 
 	BUILD_BUG_ON(!is_power_of_two(PCI_IO_SIZE));
 
Signature · 1 line
2.17.1
Applied
#1 lore raw
Quoted 8 lines · Xie Bo

The legacy virtio-pci interrupt state is represented by two pieces of
state: vpci->isr and the legacy INTx line level. Backend threads set ISR bits and assert INTx when queue or configuration changes occur, while vCPU threads read and clear the ISR and deassert INTx.

Those operations currently race. One possible interleaving is:

[...]

Applied to kvmtool (master), thanks!

[1/1] virtio/pci: Serialize legacy ISR and INTx updates

      https://git.kernel.org/will/kvmtool/c/51f527bde8a2

Cheers,

Signature · 5 lines
Will

https://fixes.arm64.dev
https://next.arm64.dev
https://will.arm64.dev