Skip to contentSkip to conversation

Intel-IOMMU: source-id checking for interrupt remapping

PATCH In review Weidong Han 1 participant 4 messages May 22, 2009 – Jun 1, 2009

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v3 cover Intel-IOMMU: source-id checking for interrupt remapping

Support source-id checking for interrupt remapping, and then isolates interrupts for guests/VMs with assigned devices.

Eric raised pci rebalance issue with VT-d. Yes, it's an issue now. Linux needs to handle pci rebalance changes to DRHD scopes. It's tricky to support it. This patch just supports source-id for interrupt remapping, won't touch that.

The patchset can be applied on linux-2.6-tip tree.

v2 -> v3 changelog:
As Ingo suggested, restructured some code and fixed some code
style issues.

v1 -> v2 changelog:
Access PCI directly (read_pci_config_byte) to parse IOAPIC,
instead of PCI related discovery, because PCI subsystem is not initialized at that time.

Weidong Han (2):
Intel-IOMMU, intr-remap: set the whole 128bits of irte when
modify/free it
Intel-IOMMU, intr-remap: source-id checking

 arch/x86/kernel/apic/io_apic.c |    6 ++
 drivers/pci/intr_remapping.c   |  160 +++++++++++++++++++++++++++++++++++----
 drivers/pci/intr_remapping.h   |    2 +
 include/linux/dmar.h           |   11 +++
 4 files changed, 162 insertions(+), 17 deletions(-)
#1 lore raw

PATCH v3 1/2 Intel-IOMMU, intr-remap: set the whole 128bits of irte when modify/free it

Interrupt remapping table entry is 128bits. Currently, it only sets low 64bits of irte in modify_irte and free_irte. This ignores high 64bits setting of irte, that means source-id setting will be ignored. This patch sets the whole 128bits of irte when modify/free it. Following source-id checking patch depends on this.

---
 drivers/pci/intr_remapping.c |   40 ++++++++++++++++++++++++++--------------
 1 files changed, 26 insertions(+), 14 deletions(-)
drivers/pci/intr_remapping.c +26 -14
index 3a0cb0b..a2ccb62 100644
@@ -314,7 +314,8 @@ int modify_irte(int irq, struct irte *irte_modified)
 	index = irq_iommu->irte_index + irq_iommu->sub_handle;
 	irte = &iommu->ir_table->base[index];
 
-	set_64bit((unsigned long *)irte, irte_modified->low);
+	set_64bit((unsigned long *)&irte->low, irte_modified->low);
+	set_64bit((unsigned long *)&irte->high, irte_modified->high);
 	__iommu_flush_cache(iommu, irte, sizeof(*irte));
 
 	rc = qi_flush_iec(iommu, index, 0);
@@ -369,12 +370,32 @@ struct intel_iommu *map_dev_to_ir(struct pci_dev *dev)
 	return drhd->iommu;
 }
 
+static int clear_entries(struct irq_2_iommu *irq_iommu)
+{
+	struct irte *start, *entry, *end;
+	struct intel_iommu *iommu;
+	int index;
+
+	if (irq_iommu->sub_handle)
+		return 0;
+
+	iommu = irq_iommu->iommu;
+	index = irq_iommu->irte_index + irq_iommu->sub_handle;
+
+	start = iommu->ir_table->base + index;
+	end = start + (1 << irq_iommu->irte_mask);
+
+	for (entry = start; entry < end; entry++) {
+		set_64bit((unsigned long *)&entry->low, 0);
+		set_64bit((unsigned long *)&entry->high, 0);
+	}
+
+	return qi_flush_iec(iommu, index, irq_iommu->irte_mask);
+}
+
 int free_irte(int irq)
 {
 	int rc = 0;
-	int index, i;
-	struct irte *irte;
-	struct intel_iommu *iommu;
 	struct irq_2_iommu *irq_iommu;
 	unsigned long flags;
 
@@ -385,16 +406,7 @@ int free_irte(int irq)
 		return -1;
 	}
 
-	iommu = irq_iommu->iommu;
-
-	index = irq_iommu->irte_index + irq_iommu->sub_handle;
-	irte = &iommu->ir_table->base[index];
-
-	if (!irq_iommu->sub_handle) {
-		for (i = 0; i < (1 << irq_iommu->irte_mask); i++)
-			set_64bit((unsigned long *)(irte + i), 0);
-		rc = qi_flush_iec(iommu, index, irq_iommu->irte_mask);
-	}
+	rc = clear_entries(irq_iommu);
 
 	irq_iommu->iommu = NULL;
 	irq_iommu->irte_index = 0;
Signature · 1 line
1.6.0.4
#2 lore raw

PATCH v3 2/2 Intel-IOMMU, intr-remap: source-id checking

In reply to Weidong Han

To support domain-isolation usages, the platform hardware must be capable of uniquely identifying the requestor (source-id) for each interrupt message. Without source-id checking for interrupt remapping , a rouge guest/VM with assigned devices can launch interrupt attacks to bring down anothe guest/VM or the VMM itself.

This patch adds source-id checking for interrupt remapping, and then really isolates interrupts for guests/VMs with assigned devices.

Because PCI subsystem is not initialized yet when set up IOAPIC entries, use read_pci_config_byte to access PCI config space directly.

---
 arch/x86/kernel/apic/io_apic.c |    6 ++
 drivers/pci/intr_remapping.c   |  120 +++++++++++++++++++++++++++++++++++++++-
 drivers/pci/intr_remapping.h   |    2 +
 include/linux/dmar.h           |   11 ++++
 4 files changed, 136 insertions(+), 3 deletions(-)
arch/x86/kernel/apic/io_apic.c +6 -0
index ed6a91c..f33e39c 100644
@@ -1411,6 +1411,9 @@ int setup_ioapic_entry(int apic_id, int irq,
 		irte.vector = vector;
 		irte.dest_id = IRTE_DEST(destination);
 
+		/* Set source-id of interrupt request */
+		set_ioapic_sid(&irte, apic_id);
+
 		modify_irte(irq, &irte);
 
 		ir_entry->index2 = (index >> 15) & 0x1;
@@ -3285,6 +3288,9 @@ static int msi_compose_msg(struct pci_dev *pdev, unsigned int irq, struct msi_ms
 		irte.vector = cfg->vector;
 		irte.dest_id = IRTE_DEST(dest);
 
+		/* Set source-id of interrupt request */
+		set_msi_sid(&irte, pdev);
+
 		modify_irte(irq, &irte);
 
 		msg->address_hi = MSI_ADDR_BASE_HI;
drivers/pci/intr_remapping.c +117 -3

144 lines not on this page · raw message

drivers/pci/intr_remapping.h +2 -0
index ca48f0d..63a263c 100644
@@ -3,6 +3,8 @@
 struct ioapic_scope {
 	struct intel_iommu *iommu;
 	unsigned int id;
+	unsigned int bus;	/* PCI bus number */
+	unsigned int devfn;	/* PCI devfn number */
 };
 
 #define IR_X2APIC_MODE(mode) (mode ? (1 << 11) : 0)
include/linux/dmar.h +11 -0
index cd83d2a..0acc73c 100644
@@ -126,6 +126,8 @@ extern int free_irte(int irq);
 extern int irq_remapped(int irq);
 extern struct intel_iommu *map_dev_to_ir(struct pci_dev *dev);
 extern struct intel_iommu *map_ioapic_to_ir(int apic);
+extern int set_ioapic_sid(struct irte *irte, int apic);
+extern int set_msi_sid(struct irte *irte, struct pci_dev *dev);
 #else
 static inline int alloc_irte(struct intel_iommu *iommu, int irq, u16 count)
 {
@@ -156,6 +158,15 @@ static inline struct intel_iommu *map_ioapic_to_ir(int apic)
 {
 	return NULL;
 }
+static inline int set_ioapic_sid(struct irte *irte, int apic)
+{
+	return 0;
+}
+static inline int set_msi_sid(struct irte *irte, struct pci_dev *dev)
+{
+	return 0;
+}
+
 #define irq_remapped(irq)		(0)
 #define enable_intr_remapping(mode)	(-1)
 #define disable_intr_remapping()	(0)
Signature · 1 line
1.6.0.4
#3 lore raw
In reply to Weidong Han

Any comments on the patchset? Thanks.

Regards,
Weidong

Quoted 30 lines · Weidong

Support source-id checking for interrupt remapping, and then isolates interrupts for guests/VMs with assigned devices.

Eric raised pci rebalance issue with VT-d. Yes, it's an issue now. Linux needs to handle pci rebalance changes to DRHD scopes. It's tricky to support it. This patch just supports source-id for interrupt remapping, won't touch that.

The patchset can be applied on linux-2.6-tip tree.

v2 -> v3 changelog:
As Ingo suggested, restructured some code and fixed some code
style issues.

v1 -> v2 changelog:
Access PCI directly (read_pci_config_byte) to parse IOAPIC,
instead of PCI related discovery, because PCI subsystem is not initialized at that time.

Weidong Han (2):
Intel-IOMMU, intr-remap: set the whole 128bits of irte when
modify/free it
Intel-IOMMU, intr-remap: source-id checking

 arch/x86/kernel/apic/io_apic.c |    6 ++
 drivers/pci/intr_remapping.c   |  160
 +++++++++++++++++++++++++++++++++++---- drivers/pci/intr_remapping.h
 |    2 + include/linux/dmar.h           |   11 +++
 4 files changed, 162 insertions(+), 17 deletions(-)