Skip to contentSkip to conversation

Intel IOMMU support, take #2

PATCH NAK Keshavamurthy, Anil S 10 participants 65 messages Jun 19, 2007 – Jul 4, 2007

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover Intel IOMMU support, take #2

Hi All,
This patch supports the upcomming Intel IOMMU hardware
a.k.a. Intel(R) Virtualization Technology for Directed I/O Architecture and the hardware spec for the same can be found here
http://www.intel.com/technology/virtualization/index.htm

This version of the patches incorporates several
feedback obtained from previous postings.

Some of the major changes are
1) Removed resource pool (a.k.a. pre-allocate pool) patch
2) For memory allocation in the DMA map api calls we now use kmem_cache_alloc() and get_zeroedpage() functions to allocate memory for internal data structures and for page table setup memory.
3) The allocation of memory in the DMA map api calls is very critical and to avoid failures during memory allocation in the DMA map api calls we evaluated several technique
a) mempool - We found that mempool is pretty much useless if we try to allocate memory with GFP_ATOMIC which is our case. Also we found that it is difficult to judge how much to reserver during the creation of mempool.
b) PF_MEMALLOC - When a task flags (current->flags) are set with PF_MEMALLOC then watermark checks are avoided during the memory allocation.
We choose to use the latter (option b) and make this as a separate patch which can be debated further. Please see patch 6/10.

Other minor changes are mostly coding style fixes and making sure that checkpatch.pl passes the patches.

Please include this set of patches for next MM release.

Thanks and regards,
-Anil S Keshavamurthy
E-mail: [email protected]

#1 lore raw

PATCH 1/10 DMAR detection and parsing logic

This patch adds support for early detection and parsing of DMAR's (DMA Remapping ) reported to OS via ACPI tables.

DMA remapping(DMAR) devices support enables independent address translations for Direct Memory Access(DMA) from Devices. These DMA remapping devices are reported via ACPI tables and includes pci device scope covered by these DMA remapping device.

For detailed info on the specification of "Intel(R) Virtualization Technology for Directed I/O Architecture" please see
http://www.intel.com/technology/virtualization/index.htm

---
 arch/x86_64/Kconfig   |   11 +
 drivers/pci/Makefile  |    3 
 drivers/pci/dmar.c    |  327 ++++++++++++++++++++++++++++++++++++++++++++++++++
 include/acpi/actbl1.h |   27 +++-
 include/linux/dmar.h  |   52 +++++++
 5 files changed, 413 insertions(+), 7 deletions(-)
arch/x86_64/Kconfig +11 -0
Index: linux-2.6.22-rc4-mm2/arch/x86_64/Kconfig
===================================================================
@@ -730,6 +730,17 @@
 	bool "Support mmconfig PCI config space access"
 	depends on PCI && ACPI
 
+config DMAR
+	bool "Support for DMA Remapping Devices (EXPERIMENTAL)"
+	depends on PCI_MSI && ACPI && EXPERIMENTAL
+	default y
+	help
+	  DMA remapping(DMAR) devices support enables independent address
+	  translations for Direct Memory Access(DMA) from Devices.
+	  These DMA remapping devices are reported via ACPI tables
+	  and includes pci device scope covered by these DMA
+	  remapping device.
+
 source "drivers/pci/pcie/Kconfig"
 
 source "drivers/pci/Kconfig"
drivers/pci/Makefile +3 -0
Index: linux-2.6.22-rc4-mm2/drivers/pci/Makefile
===================================================================
@@ -20,6 +20,9 @@
 # Build the Hypertransport interrupt support
 obj-$(CONFIG_HT_IRQ) += htirq.o
 
+# Build Intel IOMMU support
+obj-$(CONFIG_DMAR) += dmar.o
+
 #
 # Some architectures use the generic PCI setup functions
 #
drivers/pci/dmar.c new file +327 -0

327 lines not on this page · raw message

include/acpi/actbl1.h +20 -7
Index: linux-2.6.22-rc4-mm2/include/acpi/actbl1.h
===================================================================
@@ -257,7 +257,8 @@
 struct acpi_table_dmar {
 	struct acpi_table_header header;	/* Common ACPI table header */
 	u8 width;		/* Host Address Width */
-	u8 reserved[11];
+	u8 flags;
+	u8 reserved[10];
 };
 
 /* DMAR subtable header */
@@ -265,8 +266,6 @@
 struct acpi_dmar_header {
 	u16 type;
 	u16 length;
-	u8 flags;
-	u8 reserved[3];
 };
 
 /* Values for subtable type in struct acpi_dmar_header */
@@ -274,13 +273,15 @@
 enum acpi_dmar_type {
 	ACPI_DMAR_TYPE_HARDWARE_UNIT = 0,
 	ACPI_DMAR_TYPE_RESERVED_MEMORY = 1,
-	ACPI_DMAR_TYPE_RESERVED = 2	/* 2 and greater are reserved */
+	ACPI_DMAR_TYPE_ATSR = 2,
+	ACPI_DMAR_TYPE_RESERVED = 3	/* 3 and greater are reserved */
 };
 
 struct acpi_dmar_device_scope {
 	u8 entry_type;
 	u8 length;
-	u8 segment;
+	u16 reserved;
+	u8 enumeration_id;
 	u8 bus;
 };
 
@@ -290,7 +291,14 @@
 	ACPI_DMAR_SCOPE_TYPE_NOT_USED = 0,
 	ACPI_DMAR_SCOPE_TYPE_ENDPOINT = 1,
 	ACPI_DMAR_SCOPE_TYPE_BRIDGE = 2,
-	ACPI_DMAR_SCOPE_TYPE_RESERVED = 3	/* 3 and greater are reserved */
+	ACPI_DMAR_SCOPE_TYPE_IOAPIC = 3,
+	ACPI_DMAR_SCOPE_TYPE_HPET = 4,
+	ACPI_DMAR_SCOPE_TYPE_RESERVED = 5	/* 5 and greater are reserved */
+};
+
+struct acpi_dmar_pci_path {
+	u8 dev;
+	u8 fn;
 };
 
 /*
@@ -301,6 +309,9 @@
 
 struct acpi_dmar_hardware_unit {
 	struct acpi_dmar_header header;
+	u8 flags;
+	u8 reserved;
+	u16 segment;
 	u64 address;		/* Register Base Address */
 };
 
@@ -312,7 +323,9 @@
 
 struct acpi_dmar_reserved_memory {
 	struct acpi_dmar_header header;
-	u64 address;		/* 4_k aligned base address */
+	u16 reserved;
+	u16 segment;
+	u64 base_address;		/* 4_k aligned base address */
 	u64 end_address;	/* 4_k aligned limit address */
 };
 
include/linux/dmar.h new file +52 -0

52 lines not on this page · raw message

#2 lore raw

Re: [Intel IOMMU 01/10] DMAR detection and parsing logic

Quoted 5 lines · Anil S

plain text document attachment (dmar_detection.patch)

drivers/pci/dmar.c new file +3 -0
+/**
+ * parse_dmar_table - parses the DMA reporting table
+ */

drivers/pci/dmar.c line 264

Quoted 10 lines
drivers/pci/dmar.c new file +10 -0
+static int __init
+parse_dmar_table(void)
+{
+	struct acpi_table_dmar *dmar;
+	struct acpi_dmar_header *entry_header;
+	int ret = 0;
+
+	dmar = (struct acpi_table_dmar *)dmar_tbl;
+
+	if (!dmar->width) {
          ^^^^^^^^^^^^^^^^^^^

That goes *splat* on my opteron box.

Quoted 33 lines
drivers/pci/dmar.c new file +33 -0
+		printk (KERN_WARNING PREFIX "Zero: Invalid DMAR haw\n");
+		return -EINVAL;
+	}
+
+	printk (KERN_INFO PREFIX "Host address width %d\n",
+		dmar->width + 1);
+
+	entry_header = (struct acpi_dmar_header *)(dmar + 1);
+	while (((unsigned long)entry_header) <
+			(((unsigned long)dmar) + dmar_tbl->length)) {
+		dmar_table_print_dmar_entry(entry_header);
+
+		switch (entry_header->type) {
+		case ACPI_DMAR_TYPE_HARDWARE_UNIT:
+			ret = dmar_parse_one_drhd(entry_header);
+			break;
+		case ACPI_DMAR_TYPE_RESERVED_MEMORY:
+			ret = dmar_parse_one_rmrr(entry_header);
+			break;
+		default:
+			printk(KERN_WARNING PREFIX
+				"Unknown DMAR structure type\n");
+			ret = 0; /* for forward compatibility */
+			break;
+		}
+		if (ret)
+			break;
+
+		entry_header = ((void *)entry_header + entry_header->length);
+	}
+	return ret;
+}
+
#3 lore raw

Re: [Intel IOMMU 01/10] DMAR detection and parsing logic

Quoted 18 lines · On Wed, 04 Jul 2007 11:18:56 +0200 Peter Zijlstra <[email protected]> wrote:
Quoted 5 lines · Anil S

plain text document attachment (dmar_detection.patch)

(quoted diff) +3 -0
+/**
+ * parse_dmar_table - parses the DMA reporting table
+ */

Review

Quoted 10 lines
(quoted diff) +10 -0
+static int __init
+parse_dmar_table(void)
+{
+	struct acpi_table_dmar *dmar;
+	struct acpi_dmar_header *entry_header;
+	int ret = 0;
+
+	dmar = (struct acpi_table_dmar *)dmar_tbl;
+
+	if (!dmar->width) {
          ^^^^^^^^^^^^^^^^^^^
Quoted 1 line

That goes *splat* on my opteron box.

This?

From: "Keshavamurthy, Anil S" <[email protected]>

Check for dmar_tbl pointer as this can be NULL on systems with no Intel VT-d support.

---

 drivers/pci/dmar.c |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/pci/dmar.c renamed from drivers/pci/dmar.c~intel-iommu-dmar-detection-and-parsing-logic-fix-intel-dmar-crash-on-amd-x86_64 +3 -1
diff -puN drivers/pci/dmar.c~intel-iommu-dmar-detection-and-parsing-logic-fix-intel-dmar-crash-on-amd-x86_64 drivers/pci/dmar.c
@@ -260,6 +260,8 @@ parse_dmar_table(void)
 	int ret = 0;
 
 	dmar = (struct acpi_table_dmar *)dmar_tbl;
+	if (!dmar)
+		return -ENODEV;
 
 	if (!dmar->width) {
 		printk (KERN_WARNING PREFIX "Zero: Invalid DMAR haw\n");
@@ -301,7 +303,7 @@ int __init dmar_table_init(void)
 
 	parse_dmar_table();
 	if (list_empty(&dmar_drhd_units)) {
-		printk(KERN_ERR PREFIX "No DMAR devices found\n");
+		printk(KERN_INFO PREFIX "No DMAR devices found\n");
 		return -ENODEV;
 	}
 	return 0;

_

#4 lore raw

Re: [Intel IOMMU 01/10] DMAR detection and parsing logic

Quoted 24 lines · Andrew Morton
Quoted 18 lines · On Wed, 04 Jul 2007 11:18:56 +0200 Peter Zijlstra <[email protected]> wrote:
Quoted 5 lines · Anil S

plain text document attachment (dmar_detection.patch)

(quoted diff) +3 -0
+/**
+ * parse_dmar_table - parses the DMA reporting table
+ */

Review

Quoted 10 lines
(quoted diff) +10 -0
+static int __init
+parse_dmar_table(void)
+{
+	struct acpi_table_dmar *dmar;
+	struct acpi_dmar_header *entry_header;
+	int ret = 0;
+
+	dmar = (struct acpi_table_dmar *)dmar_tbl;
+
+	if (!dmar->width) {
          ^^^^^^^^^^^^^^^^^^^
Quoted 1 line

That goes *splat* on my opteron box.

This?

Quoted 7 lines

From: "Keshavamurthy, Anil S" <[email protected]>

Check for dmar_tbl pointer as this can be NULL on systems with no Intel VT-d support.

Ah, that does look sane, I'll test it whenever the next -mm comes around.

#5 lore raw

PATCH 2/10 PCI generic helper function

In reply to Keshavamurthy, Anil S

When devices are under a p2p bridge, upstream
transactions get replaced by the device id of the bridge as it owns the PCIE transaction. Hence its necessary to setup translations on behalf of the bridge as well. Due to this limitation all devices under a p2p share the same domain in a DMAR.

We just cache the type of device, if its a native PCIe device or not for later use.

changes from previous posting
1) fixed mostly coding style issues nothing major

---
 drivers/pci/pci.h    |    1 +
 drivers/pci/probe.c  |   14 ++++++++++++++
 drivers/pci/search.c |   30 ++++++++++++++++++++++++++++++
 include/linux/pci.h  |    2 ++
 4 files changed, 47 insertions(+)
drivers/pci/pci.h +1 -0
Index: linux-2.6.22-rc4-mm2/drivers/pci/pci.h
===================================================================
@@ -92,3 +92,4 @@
 	return NULL;
 }
 
+struct pci_dev *pci_find_upstream_pcie_bridge(struct pci_dev *pdev);
drivers/pci/probe.c +14 -0
Index: linux-2.6.22-rc4-mm2/drivers/pci/probe.c
===================================================================
@@ -851,6 +851,19 @@
 	kfree(pci_dev);
 }
 
+static void set_pcie_port_type(struct pci_dev *pdev)
+{
+	int pos;
+	u16 reg16;
+
+	pos = pci_find_capability(pdev, PCI_CAP_ID_EXP);
+	if (!pos)
+		return;
+	pdev->is_pcie = 1;
+	pci_read_config_word(pdev, pos + PCI_EXP_FLAGS, &reg16);
+	pdev->pcie_type = (reg16 & PCI_EXP_FLAGS_TYPE) >> 4;
+}
+
 /**
  * pci_cfg_space_size - get the configuration space size of the PCI device.
  * @dev: PCI device
@@ -965,6 +978,7 @@
 	dev->device = (l >> 16) & 0xffff;
 	dev->cfg_size = pci_cfg_space_size(dev);
 	dev->error_state = pci_channel_io_normal;
+	set_pcie_port_type(dev);
 
 	/* Assume 32-bit PCI; let 64-bit PCI cards (which are far rarer)
 	   set this higher, assuming the system even supports it.  */
drivers/pci/search.c +30 -0
Index: linux-2.6.22-rc4-mm2/drivers/pci/search.c
===================================================================
@@ -14,6 +14,36 @@
 #include "pci.h"
 
 DECLARE_RWSEM(pci_bus_sem);
+/*
+ * find the upstream PCIE-to-PCI bridge of a PCI device
+ * if the device is PCIE, return NULL
+ * if the device isn't connected to a PCIE bridge (that is its parent is a
+ * legacy PCI bridge and the bridge is directly connected to bus 0), return its
+ * parent
+ */
+struct pci_dev *
+pci_find_upstream_pcie_bridge(struct pci_dev *pdev)
+{
+	struct pci_dev *tmp = NULL;
+
+	if (pdev->is_pcie)
+		return NULL;
+	while (1) {
+		if (!pdev->bus->self)
+			break;
+		pdev = pdev->bus->self;
+		/* a p2p bridge */
+		if (!pdev->is_pcie) {
+			tmp = pdev;
+			continue;
+		}
+		/* PCI device should connect to a PCIE bridge */
+		BUG_ON(pdev->pcie_type != PCI_EXP_TYPE_PCI_BRIDGE);
+		return pdev;
+	}
+
+	return tmp;
+}
 
 static struct pci_bus *pci_do_find_bus(struct pci_bus *bus, unsigned char busnr)
 {
include/linux/pci.h +2 -0
Index: linux-2.6.22-rc4-mm2/include/linux/pci.h
===================================================================
@@ -140,6 +140,7 @@
 	unsigned short	subsystem_device;
 	unsigned int	class;		/* 3 bytes: (base,sub,prog-if) */
 	u8		hdr_type;	/* PCI header type (`multi' flag masked out) */
+	u8		pcie_type;	/* PCI-E device/port type */
 	u8		rom_base_reg;	/* which config register controls the ROM */
 	u8		pin;  		/* which interrupt pin this device uses */
 
@@ -182,6 +183,7 @@
 	unsigned int 	msi_enabled:1;
 	unsigned int	msix_enabled:1;
 	unsigned int	is_managed:1;
+	unsigned int	is_pcie:1;
 	atomic_t	enable_cnt;	/* pci_enable_device has been called */
 
 	u32		saved_config_space[16]; /* config space saved at suspend time */
#6 lore raw

Re: [Intel IOMMU 02/10] PCI generic helper function

Review

Quoted 2 lines · Anil S
(quoted diff) +2 -0
+struct pci_dev *
+pci_find_upstream_pcie_bridge(struct pci_dev *pdev)

You didn't need a newline there, but that's what the rest of that file does. Hu hum.

Quoted 6 lines
(quoted diff) +6 -0
+{
+	struct pci_dev *tmp = NULL;
+
+	if (pdev->is_pcie)
+		return NULL;
+	while (1) {

Review

Quoted 10 lines
(quoted diff) +10 -0
+		if (!pdev->bus->self)
+			break;
+		pdev = pdev->bus->self;
+		/* a p2p bridge */
+		if (!pdev->is_pcie) {
+			tmp = pdev;
+			continue;
+		}
+		/* PCI device should connect to a PCIE bridge */
+		BUG_ON(pdev->pcie_type != PCI_EXP_TYPE_PCI_BRIDGE);

I assume that if this bug triggers, we've found some broken hardware?

Going BUG seems like a pretty rude reaction to this, especially when it would be so easy to drop a warning and then recover.

How's about this?

drivers/pci/search.c renamed from drivers/pci/search.c~intel-iommu-pci-generic-helper-function-fix +5 -1
@@ -38,7 +38,11 @@ pci_find_upstream_pcie_bridge(struct pci
 			continue;
 		}
 		/* PCI device should connect to a PCIE bridge */
-		BUG_ON(pdev->pcie_type != PCI_EXP_TYPE_PCI_BRIDGE);
+		if (pdev->pcie_type != PCI_EXP_TYPE_PCI_BRIDGE) {
+			/* Busted hardware? */
+			WARN_ON_ONCE(1);
+			return NULL;
+		}
 		return pdev;
 	}
 
#7 lore raw

Re: [Intel IOMMU 02/10] PCI generic helper function

Quoted 27 lines · Andrew Morton

Review

Quoted 2 lines · Anil S
(quoted diff) +2 -0
+struct pci_dev *
+pci_find_upstream_pcie_bridge(struct pci_dev *pdev)

You didn't need a newline there, but that's what the rest of that file does. Hu hum.

Quoted 6 lines
(quoted diff) +6 -0
+{
+	struct pci_dev *tmp = NULL;
+
+	if (pdev->is_pcie)
+		return NULL;
+	while (1) {

Review

Quoted 10 lines
(quoted diff) +10 -0
+		if (!pdev->bus->self)
+			break;
+		pdev = pdev->bus->self;
+		/* a p2p bridge */
+		if (!pdev->is_pcie) {
+			tmp = pdev;
+			continue;
+		}
+		/* PCI device should connect to a PCIE bridge */
+		BUG_ON(pdev->pcie_type != PCI_EXP_TYPE_PCI_BRIDGE);

I assume that if this bug triggers, we've found some broken hardware?

Quoted 5 lines

Going BUG seems like a pretty rude reaction to this, especially when it would be so easy to drop a warning and then recover.

How's about this?

Looks good, thanks.

Quoted 16 lines
drivers/pci/search.c renamed from drivers/pci/search.c~intel-iommu-pci-generic-helper-function-fix +5 -1
@@ -38,7 +38,11 @@ pci_find_upstream_pcie_bridge(struct pci
 			continue;
 		}
 		/* PCI device should connect to a PCIE bridge */
-		BUG_ON(pdev->pcie_type != PCI_EXP_TYPE_PCI_BRIDGE);
+		if (pdev->pcie_type != PCI_EXP_TYPE_PCI_BRIDGE) {
+			/* Busted hardware? */
+			WARN_ON_ONCE(1);
+			return NULL;
+		}
 		return pdev;
 	}
 
#8 lore raw

PATCH 3/10 clflush_cache_range now takes size param

In reply to Keshavamurthy, Anil S

Introduce the size param for clflush_cache_range().

---
 arch/x86_64/mm/pageattr.c       |    6 +++---
 include/asm-x86_64/cacheflush.h |    1 +
 2 files changed, 4 insertions(+), 3 deletions(-)
arch/x86_64/mm/pageattr.c +3 -3
Index: linux-2.6.22-rc4-mm2/arch/x86_64/mm/pageattr.c
===================================================================
@@ -61,10 +61,10 @@
 	return base;
 } 
 
-static void cache_flush_page(void *adr)
+void clflush_cache_range(void *adr, int size)
 {
 	int i;
-	for (i = 0; i < PAGE_SIZE; i += boot_cpu_data.x86_clflush_size)
+	for (i = 0; i < size; i += boot_cpu_data.x86_clflush_size)
 		asm volatile("clflush (%0)" :: "r" (adr + i));
 }
 
@@ -80,7 +80,7 @@
 	list_for_each_entry(pg, l, lru) {
 		void *adr = page_address(pg);
 		if (cpu_has_clflush)
-			cache_flush_page(adr);
+			clflush_cache_range(adr, PAGE_SIZE);
 	}
 	__flush_tlb_all();
 }
include/asm-x86_64/cacheflush.h +1 -0
Index: linux-2.6.22-rc4-mm2/include/asm-x86_64/cacheflush.h
===================================================================
@@ -27,6 +27,7 @@
 void global_flush_tlb(void); 
 int change_page_attr(struct page *page, int numpages, pgprot_t prot);
 int change_page_attr_addr(unsigned long addr, int numpages, pgprot_t prot);
+void clflush_cache_range(void *addr, int size);
 
 #ifdef CONFIG_DEBUG_RODATA
 void mark_rodata_ro(void);
#9 lore raw

PATCH 4/10 IOVA allocation and management routines

In reply to Keshavamurthy, Anil S

This code implements a generic IOVA allocation and
management. As per Dave's suggestion we are now allocating IO virtual address from Higher DMA limit address rather than lower end address and this eliminated the need to preserve the IO virtual address for multiple devices sharing the same domain virtual address.

Also this code uses red black trees to store the allocated and reserved iova nodes. This showed a good performance improvements over previous linear linked list.

Changes from previous posting:
1) Fixed mostly coding style issues

---
 drivers/pci/iova.c |  351 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 drivers/pci/iova.h |   62 +++++++++
 2 files changed, 413 insertions(+)
drivers/pci/iova.c new file +351 -0
Index: linux-2.6.22-rc4-mm2/drivers/pci/iova.c
===================================================================
@@ -0,0 +1,351 @@
+/*
+ * Copyright (c) 2006, Intel Corporation.
+ *
+ * This file is released under the GPLv2.
+ *
+ * Copyright (C) 2006 Anil S Keshavamurthy <[email protected]>
+ */
+
+#include "iova.h"
+
+void
+init_iova_domain(struct iova_domain *iovad)
+{
+	spin_lock_init(&iovad->iova_alloc_lock);
+	spin_lock_init(&iovad->iova_rbtree_lock);
+	iovad->rbroot = RB_ROOT;
+	iovad->cached32_node = NULL;
+
+}
+
+static struct rb_node *
+__get_cached_rbnode(struct iova_domain *iovad, unsigned long *limit_pfn)
+{
+	if ((*limit_pfn != DMA_32BIT_PFN) ||
+		(iovad->cached32_node == NULL))
+		return rb_last(&iovad->rbroot);
+	else {
+		struct rb_node *prev_node = rb_prev(iovad->cached32_node);
+		struct iova *curr_iova =
+			container_of(iovad->cached32_node, struct iova, node);
+		*limit_pfn = curr_iova->pfn_lo - 1;
+		return prev_node;
+	}
+}
+
+static inline void
+__cached_rbnode_insert_update(struct iova_domain *iovad,
+	unsigned long limit_pfn, struct iova *new)
+{
+	if (limit_pfn != DMA_32BIT_PFN)
+		return;
+	iovad->cached32_node = &new->node;
+}
+
+static inline void
+__cached_rbnode_delete_update(struct iova_domain *iovad, struct iova *free)
+{
+	struct iova *cached_iova;
+	struct rb_node *curr;
+
+	if (!iovad->cached32_node)
+		return;
+	curr = iovad->cached32_node;
+	cached_iova = container_of(curr, struct iova, node);
+
+	if (free->pfn_lo >= cached_iova->pfn_lo)
+		iovad->cached32_node = rb_next(&free->node);
+}
+
+static inline int __alloc_iova_range(struct iova_domain *iovad,
+	unsigned long size, unsigned long limit_pfn, struct iova *new)
+{
+	struct rb_node *curr = NULL;
+	unsigned long flags;
+	unsigned long saved_pfn;
+
+	/* Walk the tree backwards */
+	spin_lock_irqsave(&iovad->iova_rbtree_lock, flags);
+	saved_pfn = limit_pfn;
+	curr = __get_cached_rbnode(iovad, &limit_pfn);
+	while (curr) {
+		struct iova *curr_iova = container_of(curr, struct iova, node);
+		if (limit_pfn < curr_iova->pfn_lo)
+			goto move_left;
+		if (limit_pfn < curr_iova->pfn_hi)
+			goto adjust_limit_pfn;
+		if ((curr_iova->pfn_hi + size) <= limit_pfn)
+			break;	/* found a free slot */
+adjust_limit_pfn:
+		limit_pfn = curr_iova->pfn_lo - 1;
+move_left:
+		curr = rb_prev(curr);
+	}
+
+	if ((!curr) && !(IOVA_START_PFN + size <= limit_pfn)) {
+		spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags);
+		return -ENOMEM;
+	}
+	new->pfn_hi = limit_pfn;
+	new->pfn_lo = limit_pfn - size + 1;
+
+	spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags);
+	return 0;
+}
+
+static void
+iova_insert_rbtree(struct rb_root *root, struct iova *iova)
+{
+	struct rb_node **new = &(root->rb_node), *parent = NULL;
+	/* Figure out where to put new node */
+	while (*new) {
+		struct iova *this = container_of(*new, struct iova, node);
+		parent = *new;
+
+		if (iova->pfn_lo < this->pfn_lo)
+			new = &((*new)->rb_left);
+		else if (iova->pfn_lo > this->pfn_lo)
+			new = &((*new)->rb_right);
+		else
+			BUG(); /* this should not happen */
+	}
+	/* Add new node and rebalance tree. */
+	rb_link_node(&iova->node, parent, new);
+	rb_insert_color(&iova->node, root);
+}
+
+/**
+ * alloc_iova - allocates an iova
+ * @iovad - iova domain in question
+ * @size - size of page frames to allocate
+ * @limit_pfn - max limit address
+ * This function allocates an iova in the range limit_pfn to IOVA_START_PFN
+ * looking from limit_pfn instead from IOVA_START_PFN.
+ */
+struct iova *
+alloc_iova(struct iova_domain *iovad, unsigned long size,
+	unsigned long limit_pfn)
+{
+	unsigned long flags;
+	struct iova *new_iova;
+	int ret;
+
+	new_iova = alloc_iova_mem();
+	if (!new_iova)
+		return NULL;
+
+	spin_lock_irqsave(&iovad->iova_alloc_lock, flags);
+	ret = __alloc_iova_range(iovad, size, limit_pfn, new_iova);
+
+	if (ret) {
+		spin_unlock_irqrestore(&iovad->iova_alloc_lock, flags);
+		free_iova_mem(new_iova);
+		return NULL;
+	}
+
+	/* Insert the new_iova into domain rbtree by holding writer lock */
+	spin_lock(&iovad->iova_rbtree_lock);
+	iova_insert_rbtree(&iovad->rbroot, new_iova);
+	__cached_rbnode_insert_update(iovad, limit_pfn, new_iova);
+	spin_unlock(&iovad->iova_rbtree_lock);
+
+	spin_unlock_irqrestore(&iovad->iova_alloc_lock, flags);
+
+	return new_iova;
+}
+
+/**
+ * find_iova - find's an iova for a given pfn
+ * @iovad - iova domain in question.
+ * pfn - page frame number
+ * This function finds and returns an iova belonging to the
+ * given doamin which matches the given pfn.
+ */
+struct iova *find_iova(struct iova_domain *iovad, unsigned long pfn)
+{
+	unsigned long flags;
+	struct rb_node *node;
+
+	spin_lock_irqsave(&iovad->iova_rbtree_lock, flags);
+	node = iovad->rbroot.rb_node;
+	while (node) {
+		struct iova *iova = container_of(node, struct iova, node);
+
+		/* If pfn falls within iova's range, return iova */
+		if ((pfn >= iova->pfn_lo) && (pfn <= iova->pfn_hi)) {
+			spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags);
+			return iova;
+		}
+
+		if (pfn < iova->pfn_lo)
+			node = node->rb_left;
+		else if (pfn > iova->pfn_lo)
+			node = node->rb_right;
+	}
+
+	spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags);
+	return NULL;
+}
+
+/**
+ * __free_iova - frees the given iova
+ * @iovad: iova domain in question.
+ * @iova: iova in question.
+ * Frees the given iova belonging to the giving domain
+ */
+void
+__free_iova(struct iova_domain *iovad, struct iova *iova)
+{
+	unsigned long flags;
+
+	if (iova) {
+		spin_lock_irqsave(&iovad->iova_rbtree_lock, flags);
+		__cached_rbnode_delete_update(iovad, iova);
+		rb_erase(&iova->node, &iovad->rbroot);
+		spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags);
+		free_iova_mem(iova);
+	}
+}
+
+/**
+ * free_iova - finds and frees the iova for a given pfn
+ * @iovad: - iova domain in question.
+ * @pfn: - pfn that is allocated previously
+ * This functions finds an iova for a given pfn and then
+ * frees the iova from that domain.
+ */
+void
+free_iova(struct iova_domain *iovad, unsigned long pfn)
+{
+	struct iova *iova = find_iova(iovad, pfn);
+	__free_iova(iovad, iova);
+
+}
+
+/**
+ * put_iova_domain - destroys the iova doamin
+ * @iovad: - iova domain in question.
+ * All the iova's in that domain are destroyed.
+ */
+void put_iova_domain(struct iova_domain *iovad)
+{
+	struct rb_node *node;
+	unsigned long flags;
+
+	spin_lock_irqsave(&iovad->iova_rbtree_lock, flags);
+	node = rb_first(&iovad->rbroot);
+	while (node) {
+		struct iova *iova = container_of(node, struct iova, node);
+		rb_erase(node, &iovad->rbroot);
+		free_iova_mem(iova);
+		node = rb_first(&iovad->rbroot);
+	}
+	spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags);
+}
+
+static inline int
+__is_range_overlap(struct rb_node *node,
+	unsigned long pfn_lo, unsigned long pfn_hi)
+{
+	struct iova * iova = container_of(node, struct iova, node);
+
+	if ((pfn_lo <= iova->pfn_hi) && (pfn_hi >= iova->pfn_lo))
+		return 1;
+	return 0;
+}
+
+static inline struct iova *
+__insert_new_range(struct iova_domain *iovad,
+	unsigned long pfn_lo, unsigned long pfn_hi)
+{
+	struct iova *iova;
+
+	iova = alloc_iova_mem();
+	if (!iova)
+		return iova;
+
+	iova->pfn_hi = pfn_hi;
+	iova->pfn_lo = pfn_lo;
+	iova_insert_rbtree(&iovad->rbroot, iova);
+	return iova;
+}
+
+static inline void
+__adjust_overlap_range(struct iova *iova,
+	unsigned long *pfn_lo, unsigned long *pfn_hi)
+{
+	if (*pfn_lo < iova->pfn_lo)
+		iova->pfn_lo = *pfn_lo;
+	if (*pfn_hi > iova->pfn_hi)
+		*pfn_lo = iova->pfn_hi + 1;
+}
+
+/**
+ * reserve_iova - reserves an iova in the given range
+ * @iovad: - iova domain pointer
+ * @pfn_lo: - lower page frame address
+ * @pfn_hi:- higher pfn adderss
+ * This function allocates reserves the address range from pfn_lo to pfn_hi so
+ * that this address is not dished out as part of alloc_iova.
+ */
+struct iova *
+reserve_iova(struct iova_domain *iovad,
+	unsigned long pfn_lo, unsigned long pfn_hi)
+{
+	struct rb_node *node;
+	unsigned long flags;
+	struct iova *iova;
+	unsigned int overlap = 0;
+
+	spin_lock_irqsave(&iovad->iova_alloc_lock, flags);
+	spin_lock(&iovad->iova_rbtree_lock);
+	for (node = rb_first(&iovad->rbroot); node; node = rb_next(node)) {
+		if (__is_range_overlap(node, pfn_lo, pfn_hi)) {
+			iova = container_of(node, struct iova, node);
+			__adjust_overlap_range(iova, &pfn_lo, &pfn_hi);
+			if ((pfn_lo >= iova->pfn_lo) &&
+				(pfn_hi <= iova->pfn_hi))
+				goto finish;
+			overlap = 1;
+
+		} else if (overlap)
+				break;
+	}
+
+	/* We are here either becasue this is the first reserver node
+	 * or need to insert remaining non overlap addr range
+	 */
+	iova = __insert_new_range(iovad, pfn_lo, pfn_hi);
+finish:
+
+	spin_unlock(&iovad->iova_rbtree_lock);
+	spin_unlock_irqrestore(&iovad->iova_alloc_lock, flags);
+	return iova;
+}
+
+/**
+ * copy_reserved_iova - copies the reserved between domains
+ * @from: - source doamin from where to copy
+ * @to: - destination domin where to copy
+ * This function copies reserved iova's from one doamin to
+ * other.
+ */
+void
+copy_reserved_iova(struct iova_domain *from, struct iova_domain *to)
+{
+	unsigned long flags;
+	struct rb_node *node;
+
+	spin_lock_irqsave(&from->iova_alloc_lock, flags);
+	spin_lock(&from->iova_rbtree_lock);
+	for (node = rb_first(&from->rbroot); node; node = rb_next(node)) {
+		struct iova *iova = container_of(node, struct iova, node);
+		struct iova *new_iova;
+		new_iova = reserve_iova(to, iova->pfn_lo, iova->pfn_hi);
+		if (!new_iova)
+			printk(KERN_ERR "Reserve iova range %lx@%lx failed\n",
+				iova->pfn_lo, iova->pfn_lo);
+	}
+	spin_unlock(&from->iova_rbtree_lock);
+	spin_unlock_irqrestore(&from->iova_alloc_lock, flags);
+}
drivers/pci/iova.h new file +62 -0

62 lines not on this page · raw message

#10 lore raw

Re: [Intel IOMMU 04/10] IOVA allocation and management routines

Quoted 7 lines · Anil S

This code implements a generic IOVA allocation and
management. As per Dave's suggestion we are now allocating IO virtual address from Higher DMA limit address rather than lower end address and this eliminated the need to preserve the IO virtual address for multiple devices sharing the same domain virtual address.

Quoted 7 lines

Also this code uses red black trees to store the allocated and reserved iova nodes. This showed a good performance improvements over previous linear linked list.

Changes from previous posting:
1) Fixed mostly coding style issues

All the inlines in this code are pretty pointless: all those functions have a single callsite so the compiler inlines them anyway. If we later add more callsites for these functions, they're too big to be inlined.

inline is usually wrong: don't do it!

Quoted 23 lines
drivers/pci/iova.c new file +23 -0
+
+/**
+ * find_iova - find's an iova for a given pfn
+ * @iovad - iova domain in question.
+ * pfn - page frame number
+ * This function finds and returns an iova belonging to the
+ * given doamin which matches the given pfn.
+ */
+struct iova *find_iova(struct iova_domain *iovad, unsigned long pfn)
+{
+	unsigned long flags;
+	struct rb_node *node;
+
+	spin_lock_irqsave(&iovad->iova_rbtree_lock, flags);
+	node = iovad->rbroot.rb_node;
+	while (node) {
+		struct iova *iova = container_of(node, struct iova, node);
+
+		/* If pfn falls within iova's range, return iova */
+		if ((pfn >= iova->pfn_lo) && (pfn <= iova->pfn_hi)) {
+			spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags);
+			return iova;
+		}

drivers/pci/iova.c line 188

Quoted 10 lines
drivers/pci/iova.c new file +10 -0
+
+		if (pfn < iova->pfn_lo)
+			node = node->rb_left;
+		else if (pfn > iova->pfn_lo)
+			node = node->rb_right;
+	}
+
+	spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags);
+	return NULL;
+}

So we take the lock, look up an item, then drop the lock then return the item we just found. We took no refcount on it and we didn't do anything to keep this object alive.

Is that a bug, or does the (afacit undocumented) lifecycle management of these things take care of it in some manner? If yes, please reply via an add-a-comment patch.

Quoted 9 lines
drivers/pci/iova.c new file +9 -0
+/**
+ * __free_iova - frees the given iova
+ * @iovad: iova domain in question.
+ * @iova: iova in question.
+ * Frees the given iova belonging to the giving domain
+ */
+void
+__free_iova(struct iova_domain *iovad, struct iova *iova)
+{

drivers/pci/iova.c line 208

Quoted 10 lines
drivers/pci/iova.c new file +10 -0
+	unsigned long flags;
+
+	if (iova) {
+		spin_lock_irqsave(&iovad->iova_rbtree_lock, flags);
+		__cached_rbnode_delete_update(iovad, iova);
+		rb_erase(&iova->node, &iovad->rbroot);
+		spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags);
+		free_iova_mem(iova);
+	}
+}

Can this really be called with NULL? If so, under what circumstances? (This reader couldn't work it out from a brief look at the code, so perhaps others will not be able to either. Perhaps a comment is needed)

Quoted 25 lines
drivers/pci/iova.c new file +25 -0
+/**
+ * free_iova - finds and frees the iova for a given pfn
+ * @iovad: - iova domain in question.
+ * @pfn: - pfn that is allocated previously
+ * This functions finds an iova for a given pfn and then
+ * frees the iova from that domain.
+ */
+void
+free_iova(struct iova_domain *iovad, unsigned long pfn)
+{
+	struct iova *iova = find_iova(iovad, pfn);
+	__free_iova(iovad, iova);
+
+}
+
+/**
+ * put_iova_domain - destroys the iova doamin
+ * @iovad: - iova domain in question.
+ * All the iova's in that domain are destroyed.
+ */
+void put_iova_domain(struct iova_domain *iovad)
+{
+	struct rb_node *node;
+	unsigned long flags;
+

drivers/pci/iova.c line 244

Quoted 10 lines
drivers/pci/iova.c new file +10 -0
+	spin_lock_irqsave(&iovad->iova_rbtree_lock, flags);
+	node = rb_first(&iovad->rbroot);
+	while (node) {
+		struct iova *iova = container_of(node, struct iova, node);
+		rb_erase(node, &iovad->rbroot);
+		free_iova_mem(iova);
+		node = rb_first(&iovad->rbroot);
+	}
+	spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags);
+}

Right, so I suspect what's happening here is that all iova's remain valid until their entire domain is destroyed, yes?

What is the upper bound to the memory consumpotion here, and what provides it?

Again, some code comments about these design issues are appropriate.

drivers/pci/iova.h line 25

Quoted 8 lines
drivers/pci/iova.h new file +8 -0
+/*
+ * We need a fixed PAGE_SIZE of 4K irrespective of
+ * arch PAGE_SIZE for IOMMU page tables.
+ */
+#define PAGE_SHIFT_4K		(12)
+#define PAGE_SIZE_4K		(1UL << PAGE_SHIFT_4K)
+#define PAGE_MASK_4K		(((u64)-1) << PAGE_SHIFT_4K)
+#define PAGE_ALIGN_4K(addr)	(((addr) + PAGE_SIZE_4K - 1) & PAGE_MASK_4K)

Am still wondering why we cannot use PAGE_SIZE, PAGE_SHIFT, etc here.

Quoted 1 line

+#define IOVA_START_ADDR (0x1000)

What determined that address? (Needs comment)

drivers/pci/iova.h line 30

Quoted 3 lines
drivers/pci/iova.h new file +3 -0
+#define IOVA_START_PFN		(IOVA_START_ADDR >> PAGE_SHIFT_4K)
+
+#define IOVA_PFN(addr)		((addr) >> PAGE_SHIFT_4K)

So I'm looking at this and wondering "what type does addr have"?

If it's unsigned long then perhaps we have a problem on x86_32 PAE. Maybe we don't support x86_32 PAE, but still, I'd have thought that the appropriate type here is dma_addr_t.

But alas, it was needlessly implemented as a macro, so the reader cannot tell.

Quoted 33 lines
drivers/pci/iova.h new file +32 -0
+#define DMA_32BIT_PFN	IOVA_PFN(DMA_32BIT_MASK)
+#define DMA_64BIT_PFN	IOVA_PFN(DMA_64BIT_MASK)
+
+/* iova structure */
+struct iova {
+	struct rb_node	node;
+	unsigned long	pfn_hi; /* IOMMU dish out addr hi */
+	unsigned long	pfn_lo; /* IOMMU dish out addr lo */
+};
+
+/* holds all the iova translations for a domain */
+struct iova_domain {
+	spinlock_t	iova_alloc_lock;/* Lock to protect iova  allocation */
+	spinlock_t	iova_rbtree_lock; /* Lock to protect update of rbtree */
+	struct rb_root	rbroot;		/* iova domain rbtree root */
+	struct rb_node	*cached32_node; /* Save last alloced node */
+};
+
+struct iova *alloc_iova_mem(void);
+void free_iova_mem(struct iova *iova);
+void free_iova(struct iova_domain *iovad, unsigned long pfn);
+void __free_iova(struct iova_domain *iovad, struct iova *iova);
+struct iova * alloc_iova(struct iova_domain *iovad, unsigned long size,
+	unsigned long limit_pfn);
+struct iova * reserve_iova(struct iova_domain *iovad, unsigned long pfn_lo,
+	unsigned long pfn_hi);
+void copy_reserved_iova(struct iova_domain *from, struct iova_domain *to);
+void init_iova_domain(struct iova_domain *iovad);
+struct iova * find_iova(struct iova_domain *iovad, unsigned long pfn);
+void put_iova_domain(struct iova_domain *iovad);
+
+#endif
#11 lore raw

Re: [Intel IOMMU 04/10] IOVA allocation and management routines

Quoted 7 lines · Andrew Morton

On Tue, 19 Jun 2007 14:37:05 -0700 "Keshavamurthy, Anil S" <[email protected]> wrote:

All the inlines in this code are pretty pointless: all those functions have a single callsite so the compiler inlines them anyway. If we later add more callsites for these functions, they're too big to be inlined.

inline is usually wrong: don't do it!

Yup, I agree and will follow in future.

Quoted 38 lines
Quoted 23 lines
(quoted diff) +23 -0
+
+/**
+ * find_iova - find's an iova for a given pfn
+ * @iovad - iova domain in question.
+ * pfn - page frame number
+ * This function finds and returns an iova belonging to the
+ * given doamin which matches the given pfn.
+ */
+struct iova *find_iova(struct iova_domain *iovad, unsigned long pfn)
+{
+	unsigned long flags;
+	struct rb_node *node;
+
+	spin_lock_irqsave(&iovad->iova_rbtree_lock, flags);
+	node = iovad->rbroot.rb_node;
+	while (node) {
+		struct iova *iova = container_of(node, struct iova, node);
+
+		/* If pfn falls within iova's range, return iova */
+		if ((pfn >= iova->pfn_lo) && (pfn <= iova->pfn_hi)) {
+			spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags);
+			return iova;
+		}

Review

Quoted 10 lines
(quoted diff) +10 -0
+
+		if (pfn < iova->pfn_lo)
+			node = node->rb_left;
+		else if (pfn > iova->pfn_lo)
+			node = node->rb_right;
+	}
+
+	spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags);
+	return NULL;
+}

So we take the lock, look up an item, then drop the lock then return the item we just found. We took no refcount on it and we didn't do anything to keep this object alive.

Quoted 3 lines

Is that a bug, or does the (afacit undocumented) lifecycle management of these things take care of it in some manner? If yes, please reply via an add-a-comment patch.

Nope, this is not a bug. Adding a comment patch which explains the same.

Quoted 18 lines
Quoted 18 lines
(quoted diff) +16 -0
+/**
+ * __free_iova - frees the given iova
+ * @iovad: iova domain in question.
+ * @iova: iova in question.
+ * Frees the given iova belonging to the giving domain
+ */
+void
+__free_iova(struct iova_domain *iovad, struct iova *iova)
+{
+	unsigned long flags;
+
+	if (iova) {
+		spin_lock_irqsave(&iovad->iova_rbtree_lock, flags);
+		__cached_rbnode_delete_update(iovad, iova);
+		rb_erase(&iova->node, &iovad->rbroot);
+		spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags);
Quoted 8 lines

Review

Quoted 3 lines
(quoted diff) +3 -0
+		free_iova_mem(iova);
+	}
+}

Can this really be called with NULL? If so, under what circumstances? (This reader couldn't work it out from a brief look at the code, so perhaps others will not be able to either. Perhaps a comment is needed)

It was getting called from only one place free_iova(). Below patch address your concern.

Quoted 31 lines
Quoted 31 lines
(quoted diff) +30 -0
+/**
+ * free_iova - finds and frees the iova for a given pfn
+ * @iovad: - iova domain in question.
+ * @pfn: - pfn that is allocated previously
+ * This functions finds an iova for a given pfn and then
+ * frees the iova from that domain.
+ */
+void
+free_iova(struct iova_domain *iovad, unsigned long pfn)
+{
+	struct iova *iova = find_iova(iovad, pfn);
+	__free_iova(iovad, iova);
+
+}
+
+/**
+ * put_iova_domain - destroys the iova doamin
+ * @iovad: - iova domain in question.
+ * All the iova's in that domain are destroyed.
+ */
+void put_iova_domain(struct iova_domain *iovad)
+{
+	struct rb_node *node;
+	unsigned long flags;
+
+	spin_lock_irqsave(&iovad->iova_rbtree_lock, flags);
+	node = rb_first(&iovad->rbroot);
+	while (node) {
+		struct iova *iova = container_of(node, struct iova, node);
+		rb_erase(node, &iovad->rbroot);
Quoted 8 lines

Review

Quoted 5 lines
(quoted diff) +5 -0
+		free_iova_mem(iova);
+		node = rb_first(&iovad->rbroot);
+	}
+	spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags);
+}

Right, so I suspect what's happening here is that all iova's remain valid until their entire domain is destroyed, yes?

Nope. IOVA are valid only for the duration of DMA MAP and DMA UNMAP calls. In case of Intel-iommu driver, the iova's are valid only for the duration of __intel_map_singl() and __intel_unmap_single() calls.

Quoted 3 lines

What is the upper bound to the memory consumpotion here, and what provides it?

As explained above, iova are freed and reused again during the DMA map calls.

Quoted 13 lines

Again, some code comments about these design issues are appropriate.

Review

Quoted 8 lines
(quoted diff) +8 -0
+/*
+ * We need a fixed PAGE_SIZE of 4K irrespective of
+ * arch PAGE_SIZE for IOMMU page tables.
+ */
+#define PAGE_SHIFT_4K		(12)
+#define PAGE_SIZE_4K		(1UL << PAGE_SHIFT_4K)
+#define PAGE_MASK_4K		(((u64)-1) << PAGE_SHIFT_4K)
+#define PAGE_ALIGN_4K(addr)	(((addr) + PAGE_SIZE_4K - 1) & PAGE_MASK_4K)

Am still wondering why we cannot use PAGE_SIZE, PAGE_SHIFT, etc here.

VT-d hardware(a.k.a Intel IOMMU hardware) page table size is always 4K irrespective of the OS PAGE_SIZE. We want to use the same code for IA64 too where the OS PAGE_SIZE may not be 4K size and hence had to define here for IOMMU.

Quoted 4 lines
Quoted 2 lines

+#define IOVA_START_ADDR (0x1000)

What determined that address? (Needs comment)

Fixed. Please see bleow patch.

Quoted 10 lines

Review

Quoted 4 lines
(quoted diff) +3 -0
+#define IOVA_START_PFN		(IOVA_START_ADDR >> PAGE_SHIFT_4K)
+
+#define IOVA_PFN(addr)		((addr) >> PAGE_SHIFT_4K)

So I'm looking at this and wondering "what type does addr have"?

If it's unsigned long then perhaps we have a problem on x86_32 PAE. Maybe we don't support x86_32 PAE, but still, I'd have thought that the appropriate type here is dma_addr_t.

Yup that is correct. We don;t support x86_32.

Quoted 3 lines

But alas, it was needlessly implemented as a macro, so the reader cannot tell.

I am in the process of making the same code base to work for IA64 architecure too and in this process I will do more cleanup.

Thanks.

Please apply the below patch as a fix the existing patch.

---
 drivers/pci/iova.c |   22 ++++++++++++++--------
 drivers/pci/iova.h |    4 ++--
 2 files changed, 16 insertions(+), 10 deletions(-)
drivers/pci/iova.c +14 -8
Index: linux-2.6.22-rc4-mm2/drivers/pci/iova.c
===================================================================
@@ -166,6 +166,7 @@
 	unsigned long flags;
 	struct rb_node *node;
 
+	/* Take the lock so that no other thread is manipulating the rbtree */
 	spin_lock_irqsave(&iovad->iova_rbtree_lock, flags);
 	node = iovad->rbroot.rb_node;
 	while (node) {
@@ -174,6 +175,12 @@
 		/* If pfn falls within iova's range, return iova */
 		if ((pfn >= iova->pfn_lo) && (pfn <= iova->pfn_hi)) {
 			spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags);
+			/* We are not holding the lock while this iova
+			 * is referenced by the caller as the same thread
+			 * which called this function also calls __free_iova()
+			 * and it is by desing that only one thread can possibly
+			 * reference a particular iova and hence no conflict.
+			 */
 			return iova;
 		}
 
@@ -198,13 +205,11 @@
 {
 	unsigned long flags;
 
-	if (iova) {
-		spin_lock_irqsave(&iovad->iova_rbtree_lock, flags);
-		__cached_rbnode_delete_update(iovad, iova);
-		rb_erase(&iova->node, &iovad->rbroot);
-		spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags);
-		free_iova_mem(iova);
-	}
+	spin_lock_irqsave(&iovad->iova_rbtree_lock, flags);
+	__cached_rbnode_delete_update(iovad, iova);
+	rb_erase(&iova->node, &iovad->rbroot);
+	spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags);
+	free_iova_mem(iova);
 }
 
 /**
@@ -218,7 +223,8 @@
 free_iova(struct iova_domain *iovad, unsigned long pfn)
 {
 	struct iova *iova = find_iova(iovad, pfn);
-	__free_iova(iovad, iova);
+	if (iova)
+		__free_iova(iovad, iova);
 
 }
 
drivers/pci/iova.h +2 -2
Index: linux-2.6.22-rc4-mm2/drivers/pci/iova.h
===================================================================
@@ -24,8 +24,8 @@
 #define PAGE_MASK_4K		(((u64)-1) << PAGE_SHIFT_4K)
 #define PAGE_ALIGN_4K(addr)	(((addr) + PAGE_SIZE_4K - 1) & PAGE_MASK_4K)
 
-#define IOVA_START_ADDR		(0x1000)
-#define IOVA_START_PFN		(IOVA_START_ADDR >> PAGE_SHIFT_4K)
+/* IO virtual address start page frame number */
+#define IOVA_START_PFN		(1)
 
 #define IOVA_PFN(addr)		((addr) >> PAGE_SHIFT_4K)
 #define DMA_32BIT_PFN	IOVA_PFN(DMA_32BIT_MASK)
Quoted 1 line
#12 lore raw

PATCH 5/10 Intel IOMMU driver

In reply to Keshavamurthy, Anil S

Actual intel IOMMU driver. Hardware spec can be found at:
http://www.intel.com/technology/virtualization

This driver sets X86_64 'dma_ops', so hook into standard DMA APIs. In this way, PCI driver will get virtual DMA address. This change is transparent to PCI drivers.

Changes from previous postings:
1) Fixed all the coding style errors - checkpatches.pl passes this patch
2) Addressed all Andrew's comments
3) Removed resource pool ( a.k.a pre-allocate pool)
4) Now uses the standard kmem_cache_alloc functions to allocate memory during dma map api calls.

---
 Documentation/Intel-IOMMU.txt       |   93 +
 Documentation/kernel-parameters.txt |   10 
 arch/x86_64/kernel/pci-dma.c        |    5 
 drivers/pci/Makefile                |    5 
 drivers/pci/intel-iommu.c           | 1956 ++++++++++++++++++++++++++++++++++++
 drivers/pci/intel-iommu.h           |  318 +++++
 include/linux/dmar.h                |   22 
 7 files changed, 2408 insertions(+), 1 deletion(-)
Documentation/Intel-IOMMU.txt new file +93 -0
Index: linux-2.6.22-rc4-mm2/Documentation/Intel-IOMMU.txt
===================================================================
@@ -0,0 +1,93 @@
+Linux IOMMU Support
+===================
+
+The architecture spec can be obtained from the below location.
+
+http://www.intel.com/technology/virtualization/
+
+This guide gives a quick cheat sheet for some basic understanding.
+
+Some Keywords
+
+DMAR - DMA remapping
+DRHD - DMA Engine Reporting Structure
+RMRR - Reserved memory Region Reporting Structure
+ZLR  - Zero length reads from PCI devices
+IOVA - IO Virtual address.
+
+Basic stuff
+-----------
+
+ACPI enumerates and lists the different DMA engines in the platform, and
+device scope relationships between PCI devices and which DMA engine  controls
+them.
+
+What is RMRR?
+-------------
+
+There are some devices the BIOS controls, for e.g USB devices to perform
+PS2 emulation. The regions of memory used for these devices are marked
+reserved in the e820 map. When we turn on DMA translation, DMA to those
+regions will fail. Hence BIOS uses RMRR to specify these regions along with
+devices that need to access these regions. OS is expected to setup
+unity mappings for these regions for these devices to access these regions.
+
+How is IOVA generated?
+---------------------
+
+Well behaved drivers call pci_map_*() calls before sending command to device
+that needs to perform DMA. Once DMA is completed and mapping is no longer
+required, device performs a pci_unmap_*() calls to unmap the region.
+
+The Intel IOMMU driver allocates a virtual address per domain. Each PCIE
+device has its own domain (hence protection). Devices under p2p bridges
+share the virtual address with all devices under the p2p bridge due to
+transaction id aliasing for p2p bridges.
+
+IOVA generation is pretty generic. We used the same technique as vmalloc()
+but these are not global address spaces, but separate for each domain.
+Different DMA engines may support different number of domains.
+
+We also allocate gaurd pages with each mapping, so we can attempt to catch
+any overflow that might happen.
+
+
+Graphics Problems?
+------------------
+If you encounter issues with graphics devices, you can try adding
+option intel_iommu=igfx_off to turn off the integrated graphics engine.
+
+Some exceptions to IOVA
+-----------------------
+Interrupt ranges are not address translated, (0xfee00000 - 0xfeefffff).
+The same is true for peer to peer transactions. Hence we reserve the
+address from PCI MMIO ranges so they are not allocated for IOVA addresses.
+
+Boot Message Sample
+-------------------
+
+Something like this gets printed indicating presence of DMAR tables
+in ACPI.
+
+ACPI: DMAR (v001 A M I  OEMDMAR  0x00000001 MSFT 0x00000097) @ 0x000000007f5b5ef0
+
+When DMAR is being processed and initialized by ACPI, prints DMAR locations
+and any RMRR's processed.
+
+ACPI DMAR:Host address width 36
+ACPI DMAR:DRHD (flags: 0x00000000)base: 0x00000000fed90000
+ACPI DMAR:DRHD (flags: 0x00000000)base: 0x00000000fed91000
+ACPI DMAR:DRHD (flags: 0x00000001)base: 0x00000000fed93000
+ACPI DMAR:RMRR base: 0x00000000000ed000 end: 0x00000000000effff
+ACPI DMAR:RMRR base: 0x000000007f600000 end: 0x000000007fffffff
+
+When DMAR is enabled for use, you will notice..
+
+PCI-DMA: Using DMAR IOMMU
+
+TBD
+----
+
+- For compatibility testing, could use unity map domain for all devices, just
+  provide a 1-1 for all useful memory under a single domain for all devices.
+- API for paravirt ops for abstracting functionlity for VMM folks.
Documentation/kernel-parameters.txt +10 -0
Index: linux-2.6.22-rc4-mm2/Documentation/kernel-parameters.txt
===================================================================
@@ -779,6 +779,16 @@
 
 	inttest=	[IA64]
 
+	intel_iommu=	[DMAR] Intel IOMMU driver (DMAR) option
+		off
+			Disable intel iommu driver.
+		igfx_off [Default Off]
+			By default, gfx is mapped as normal device. If a gfx
+			device has a dedicated DMAR unit, the DMAR unit is
+			bypassed by not enabling DMAR with this option. In
+			this case, gfx device will use physical address for
+			DMA.
+
 	io7=		[HW] IO7 for Marvel based alpha systems
 			See comment before marvel_specify_io7 in
 			arch/alpha/kernel/core_marvel.c.
arch/x86_64/kernel/pci-dma.c +5 -0
Index: linux-2.6.22-rc4-mm2/arch/x86_64/kernel/pci-dma.c
===================================================================
@@ -7,6 +7,7 @@
 #include <linux/string.h>
 #include <linux/pci.h>
 #include <linux/module.h>
+#include <linux/dmar.h>
 #include <asm/io.h>
 #include <asm/proto.h>
 #include <asm/calgary.h>
@@ -302,6 +303,8 @@
 	detect_calgary();
 #endif
 
+	detect_intel_iommu();
+
 #ifdef CONFIG_SWIOTLB
 	pci_swiotlb_init();
 #endif
@@ -313,6 +316,8 @@
 	calgary_iommu_init();
 #endif
 
+	intel_iommu_init();
+
 #ifdef CONFIG_IOMMU
 	gart_iommu_init();
 #endif
drivers/pci/Makefile +4 -1
Index: linux-2.6.22-rc4-mm2/drivers/pci/Makefile
===================================================================
@@ -21,7 +21,10 @@
 obj-$(CONFIG_HT_IRQ) += htirq.o
 
 # Build Intel IOMMU support
-obj-$(CONFIG_DMAR) += dmar.o
+obj-$(CONFIG_DMAR) += dmar.o iova.o intel-iommu.o
+
+#Build Intel-IOMMU support
+obj-$(CONFIG_DMAR) += iova.o dmar.o intel-iommu.o
 
 #
 # Some architectures use the generic PCI setup functions
drivers/pci/intel-iommu.c new file +1956 -0

1,956 lines not on this page · raw message

drivers/pci/intel-iommu.h new file +318 -0

318 lines not on this page · raw message

include/linux/dmar.h +22 -0
Index: linux-2.6.22-rc4-mm2/include/linux/dmar.h
===================================================================
@@ -23,7 +23,14 @@
 
 #include <linux/acpi.h>
 #include <linux/types.h>
+#include <linux/msi.h>
 
+#ifdef CONFIG_DMAR
+struct intel_iommu;
+
+/* Intel IOMMU detection and initialization functions */
+extern void detect_intel_iommu(void);
+extern int intel_iommu_init(void);
 
 extern int dmar_table_init(void);
 extern int early_dmar_detect(void);
@@ -49,4 +56,19 @@
 	int	devices_cnt;		/* target device count */
 };
 
+#define for_each_drhd_unit(drhd) \
+	list_for_each_entry(drhd, &dmar_drhd_units, list)
+#define for_each_rmrr_units(rmrr) \
+	list_for_each_entry(rmrr, &dmar_rmrr_units, list)
+#else
+static inline void detect_intel_iommu(void)
+{
+	return;
+}
+static inline int intel_iommu_init(void)
+{
+	return -ENODEV;
+}
+
+#endif /* !CONFIG_DMAR */
 #endif /* __DMAR_H__ */
#13 lore raw

Re: [Intel IOMMU 05/10] Intel IOMMU driver

drivers/pci/intel-iommu.c line 90

Quoted 4 lines · Anil S
drivers/pci/intel-iommu.c new file +4 -0
+static inline void *alloc_pgtable_page(void)
+{
+	return (void *)get_zeroed_page(GFP_ATOMIC);
+}

Need to pass gfp_t parameter. Repeates a couple of times.

Quoted 4 lines
drivers/pci/intel-iommu.c new file +4 -0
+	addr &= (((u64)1) << addr_width) - 1;
+	parent = domain->pgd;
+
+	spin_lock_irqsave(&domain->mapping_lock, flags);

drivers/pci/intel-iommu.c line 290

Quoted 10 lines
drivers/pci/intel-iommu.c new file +10 -0
+	while (level > 0) {
+		void *tmp_page;
+
+		offset = address_level_offset(addr, level);
+		pte = &parent[offset];
+		if (level == 1)
+			break;
+
+		if (!dma_pte_present(*pte)) {
+			tmp_page = alloc_pgtable_page();

Is it not possible here to drop the lock and do the alloc with GFP_KERNEL and deal with the resulting race? That is done in other parts of the kernel.

drivers/pci/intel-iommu.c line 417

Quoted 7 lines
drivers/pci/intel-iommu.c new file +7 -0
+/* iommu handling */
+static int iommu_alloc_root_entry(struct intel_iommu *iommu)
+{
+	struct root_entry *root;
+	unsigned long flags;
+
+	root = (struct root_entry *)alloc_pgtable_page();

This may be able to become a GFP_KERNEL alloc since interrupts are enabled at this point?

Quoted 8 lines
drivers/pci/intel-iommu.c new file +8 -0
+int __init init_dmars(void)
+{
+	struct dmar_drhd_unit *drhd;
+	struct dmar_rmrr_unit *rmrr;
+	struct pci_dev *pdev;
+	struct intel_iommu *iommu;
+	int ret, unit = 0;
+

drivers/pci/intel-iommu.c line 1404

Quoted 10 lines
drivers/pci/intel-iommu.c new file +10 -0
+	/*
+	 * for each drhd
+	 *    allocate root
+	 *    initialize and program root entry to not present
+	 * endfor
+	 */
+	for_each_drhd_unit(drhd) {
+		if (drhd->ignored)
+			continue;
+		iommu = alloc_iommu(drhd);

GFP_KERNEL alloc possible?

drivers/pci/intel-iommu.c line 1819

Quoted 8 lines
drivers/pci/intel-iommu.c new file +8 -0
+{
+	int ret = 0;
+
+	iommu_devinfo_cache = kmem_cache_create("iommu_devinfo",
+					 sizeof(struct device_domain_info),
+					 0,
+					 SLAB_HWCACHE_ALIGN,
+					 NULL,

Replace by

iommu_devinfo_cache = KMEM_CACHE(iommu_devinfo, SLAB_HWCACHE_ALIGN)

drivers/pci/intel-iommu.c line 1838

Quoted 10 lines
drivers/pci/intel-iommu.c new file +10 -0
+static inline int iommu_iova_cache_init(void)
+{
+	int ret = 0;
+
+	iommu_iova_cache = kmem_cache_create("iommu_iova",
+					 sizeof(struct iova),
+					 0,
+					 SLAB_HWCACHE_ALIGN,
+					 NULL,
+					 NULL);

Ditto

#14 lore raw

Re: [Intel IOMMU 05/10] Intel IOMMU driver

Quoted 28 lines · Christoph Lameter

drivers/pci/intel-iommu.c line 90

Quoted 4 lines · Anil S
drivers/pci/intel-iommu.c new file +4 -0
+static inline void *alloc_pgtable_page(void)
+{
+	return (void *)get_zeroed_page(GFP_ATOMIC);
+}

Need to pass gfp_t parameter. Repeates a couple of times.

Quoted 4 lines
drivers/pci/intel-iommu.c new file +4 -0
+	addr &= (((u64)1) << addr_width) - 1;
+	parent = domain->pgd;
+
+	spin_lock_irqsave(&domain->mapping_lock, flags);

drivers/pci/intel-iommu.c line 290

Quoted 10 lines
drivers/pci/intel-iommu.c new file +10 -0
+	while (level > 0) {
+		void *tmp_page;
+
+		offset = address_level_offset(addr, level);
+		pte = &parent[offset];
+		if (level == 1)
+			break;
+
+		if (!dma_pte_present(*pte)) {
+			tmp_page = alloc_pgtable_page();

Is it not possible here to drop the lock and do the alloc with GFP_KERNEL and deal with the resulting race? That is done in other parts of the kernel.

Quoted 10 lines

drivers/pci/intel-iommu.c line 417

Quoted 7 lines
drivers/pci/intel-iommu.c new file +7 -0
+/* iommu handling */
+static int iommu_alloc_root_entry(struct intel_iommu *iommu)
+{
+	struct root_entry *root;
+	unsigned long flags;
+
+	root = (struct root_entry *)alloc_pgtable_page();

This may be able to become a GFP_KERNEL alloc since interrupts are enabled at this point?

Memory allocated during driver init is very less and not much benefit with the suggested changes I think. Please correct me If I am wrong.

The biggest benifit will be when we can figure out GPF_XXXX flags during runtime when DMA map api's are called.

#15 lore raw

Re: [Intel IOMMU 05/10] Intel IOMMU driver

Quoted 2 lines · Anil S

Memory allocated during driver init is very less and not much benefit with the suggested changes I think. Please correct me If I am wrong.

If its just a small amount of memory then the benefit will not be large. You are likely right.

Quoted 2 lines

The biggest benifit will be when we can figure out GPF_XXXX flags during runtime when DMA map api's are called.

Right.

#16 lore raw

Re: [Intel IOMMU 05/10] Intel IOMMU driver

In reply to Christoph Lameter
Quoted 18 lines · On Tue, 19 Jun 2007 16:32:23 -0700 (PDT) Christoph Lameter <[email protected]>…

drivers/pci/intel-iommu.c line 90

Quoted 4 lines · Anil S
drivers/pci/intel-iommu.c new file +4 -0
+static inline void *alloc_pgtable_page(void)
+{
+	return (void *)get_zeroed_page(GFP_ATOMIC);
+}

Need to pass gfp_t parameter. Repeates a couple of times.
...
Is it not possible here to drop the lock and do the alloc with GFP_KERNEL and deal with the resulting race? That is done in other parts of the kernel.
...
This may be able to become a GFP_KERNEL alloc since interrupts are enabled at this point?
...
GFP_KERNEL alloc possible?

Yeah, if there are any callsites at all at which we know that we can perform a sleeping allocation, Christoph's suggestions should be adopted. Because even a bare GFP_NOIO is heaps more robust than GFP_ATOMIC, and it will also reload the free-pages reserves, making subsequent GFP_ATOMIC allocations more likely to succeed.

#17 lore raw

Re: [Intel IOMMU 05/10] Intel IOMMU driver

Quoted 21 lines · Andrew Morton
Quoted 18 lines · On Tue, 19 Jun 2007 16:32:23 -0700 (PDT) Christoph Lameter <[email protected]>…

drivers/pci/intel-iommu.c line 90

Quoted 4 lines · Anil S
drivers/pci/intel-iommu.c new file +4 -0
+static inline void *alloc_pgtable_page(void)
+{
+	return (void *)get_zeroed_page(GFP_ATOMIC);
+}

Need to pass gfp_t parameter. Repeates a couple of times.
...
Is it not possible here to drop the lock and do the alloc with GFP_KERNEL and deal with the resulting race? That is done in other parts of the kernel.
...
This may be able to become a GFP_KERNEL alloc since interrupts are enabled at this point?
...
GFP_KERNEL alloc possible?

Quoted 5 lines

Yeah, if there are any callsites at all at which we know that we can perform a sleeping allocation, Christoph's suggestions should be adopted. Because even a bare GFP_NOIO is heaps more robust than GFP_ATOMIC, and it will also reload the free-pages reserves, making subsequent GFP_ATOMIC allocations more likely to succeed.

Yup, will do as part of making this code work for IA64, which is my next item in my todo list.

-Anil

#18 lore raw

Re: [Intel IOMMU 05/10] Intel IOMMU driver

In reply to Keshavamurthy, Anil S
Quoted 26 lines · Anil S
drivers/pci/intel-iommu.h new file +26 -0
+/*
+ * Decoding Capability Register
+ */
+#define cap_read_drain(c)	(((c) >> 55) & 1)
+#define cap_write_drain(c)	(((c) >> 54) & 1)
+#define cap_max_amask_val(c)	(((c) >> 48) & 0x3f)
+#define cap_num_fault_regs(c)	((((c) >> 40) & 0xff) + 1)
+#define cap_pgsel_inv(c)	(((c) >> 39) & 1)
+
+#define cap_super_page_val(c)	(((c) >> 34) & 0xf)
+#define cap_super_offset(c)	(((find_first_bit(&cap_super_page_val(c), 4)) \
+					* OFFSET_STRIDE) + 21)
+
+#define cap_fault_reg_offset(c)	((((c) >> 24) & 0x3ff) * 16)
+#define cap_max_fault_reg_offset(c) \
+	(cap_fault_reg_offset(c) + cap_num_fault_regs(c) * 16)
+
+#define cap_zlr(c)		(((c) >> 22) & 1)
+#define cap_isoch(c)		(((c) >> 23) & 1)
+#define cap_mgaw(c)		((((c) >> 16) & 0x3f) + 1)
+#define cap_sagaw(c)		(((c) >> 8) & 0x1f)
+#define cap_caching_mode(c)	(((c) >> 7) & 1)
+#define cap_phmr(c)		(((c) >> 6) & 1)
+#define cap_plmr(c)		(((c) >> 5) & 1)
+#define cap_rwbf(c)		(((c) >> 4) & 1)
+#define cap_afl(c)		(((c) >> 3) & 1)

drivers/pci/intel-iommu.h line 113

Quoted 10 lines
drivers/pci/intel-iommu.h new file +10 -0
+#define cap_ndoms(c)		(((unsigned long)1) << (4 + 2 * ((c) & 0x7)))
+/*
+ * Extended Capability Register
+ */
+
+#define ecap_niotlb_iunits(e)	((((e) >> 24) & 0xff) + 1)
+#define ecap_iotlb_offset(e) 	((((e) >> 8) & 0x3ff) * 16)
+#define ecap_max_iotlb_offset(e) \
+	(ecap_iotlb_offset(e) + ecap_niotlb_iunits(e) * 16)
+#define ecap_coherent(e)	((e) & 0x1)

None of these actually _need_ to be macros and it would be better to implement them in C. That way things are more self-documenting, more pleasant to read, more likely to get commented and you'll fix the two bugs wherein the argument to a macro is evaluated more than once.

#19 lore raw

Re: [Intel IOMMU 05/10] Intel IOMMU driver

Quoted 7 lines · Andrew Morton

On Tue, 19 Jun 2007 14:37:06 -0700 "Keshavamurthy, Anil S" <[email protected]> wrote:

None of these actually _need_ to be macros and it would be better to implement them in C. That way things are more self-documenting, more pleasant to read, more likely to get commented and you'll fix the two bugs wherein the argument to a macro is evaluated more than once.

Agree. Will send a patch when I get back from OLS as any changes to this needs thorugh testing.

-Anil

#20 lore raw

Re: [Intel IOMMU 05/10] Intel IOMMU driver

In reply to Keshavamurthy, Anil S
Quoted 16 lines · Anil S

Actual intel IOMMU driver. Hardware spec can be found at:
http://www.intel.com/technology/virtualization

This driver sets X86_64 'dma_ops', so hook into standard DMA APIs. In this way, PCI driver will get virtual DMA address. This change is transparent to PCI drivers.

Changes from previous postings:
1) Fixed all the coding style errors - checkpatches.pl passes this patch
2) Addressed all Andrew's comments
3) Removed resource pool ( a.k.a pre-allocate pool)
4) Now uses the standard kmem_cache_alloc functions to allocate memory during dma map api calls.

...

drivers/pci/intel-iommu.h line 234

Quoted 5 lines
drivers/pci/intel-iommu.h new file +5 -0
+#define context_set_translation_type(c, val) \
+	do { \
+		(c).lo &= (((u64)-1) << 4) | 3; \
+		(c).lo |= ((val) & 3) << 2; \
+	} while (0)

That evaluates `c' twice. It's a little handgrenade waiting to go off.

Quoted 1 line

+#define context_clear_entry(c) do {(c).lo = 0; (c).hi = 0;} while (0)

Ditto

#21 lore raw

PATCH 6/10 Avoid memory allocation failures in dma map api calls

In reply to Keshavamurthy, Anil S

Intel IOMMU driver needs memory during DMA map calls to setup its internal page tables and for other data structures. As we all know that these DMA map calls are mostly called in the interrupt context or with the spinlock held by the upper level drivers(network/storage drivers), so in order to avoid any memory allocation failure due to low memory issues, this patch makes memory allocation by temporarily setting PF_MEMALLOC flags for the current task before making memory allocation calls.

We evaluated mempools as a backup when kmem_cache_alloc() fails and found that mempools are really not useful here because
1) We don;t know for sure how much to reserve in advance
2) And mempools are not useful for GFP_ATOMIC case (as we call memory alloc functions with GFP_ATOMIC)

With PF_MEMALLOC flag set in the current->flags, the VM subsystem avoids any watermark checks before allocating memory thus guarantee'ing the memory till the last free page. Further, looking at the code in mm/page_alloc.c in __alloc_pages() function, looks like this flag is useful only in the non-interrupt context.

If we are in the interrupt context and memory allocation in IOMMU driver fails for some reason, then the DMA map api's will return failure and it is up to the higher level drivers to retry. Suppose, if upper level driver programs the controller with the buggy DMA virtual address, the IOMMU will block that DMA transaction when that happens thus preventing any corruption to main memory.

So far in our test scenario, we were unable to create any memory allocation failure inside dma map api calls.

---
 drivers/pci/intel-iommu.c |   30 ++++++++++++++++++++++++++----
 1 file changed, 26 insertions(+), 4 deletions(-)
drivers/pci/intel-iommu.c +26 -4
Index: linux-2.6.22-rc4-mm2/drivers/pci/intel-iommu.c
===================================================================
@@ -84,9 +84,31 @@
 static struct kmem_cache *iommu_devinfo_cache;
 static struct kmem_cache *iommu_iova_cache;
 
+static inline void *iommu_kmem_cache_alloc(struct kmem_cache *cachep)
+{
+	unsigned int flags;
+	void *vaddr;
+
+	/* trying to avoid low memory issues */
+	flags = current->flags & PF_MEMALLOC;
+	current->flags |= PF_MEMALLOC;
+	vaddr = kmem_cache_alloc(cachep, GFP_ATOMIC);
+	current->flags &= (~PF_MEMALLOC | flags);
+	return vaddr;
+}
+
+
 static inline void *alloc_pgtable_page(void)
 {
-	return (void *)get_zeroed_page(GFP_ATOMIC);
+	unsigned int flags;
+	void *vaddr;
+
+	/* trying to avoid low memory issues */
+	flags = current->flags & PF_MEMALLOC;
+	current->flags |= PF_MEMALLOC;
+	vaddr = (void *)get_zeroed_page(GFP_ATOMIC);
+	current->flags &= (~PF_MEMALLOC | flags);
+	return vaddr;
 }
 
 static inline void free_pgtable_page(void *vaddr)
@@ -96,7 +118,7 @@
 
 static inline void *alloc_domain_mem(void)
 {
-	return kmem_cache_alloc(iommu_domain_cache, GFP_ATOMIC);
+	return iommu_kmem_cache_alloc(iommu_domain_cache);
 }
 
 static inline void free_domain_mem(void *vaddr)
@@ -106,7 +128,7 @@
 
 static inline void * alloc_devinfo_mem(void)
 {
-	return kmem_cache_alloc(iommu_devinfo_cache, GFP_ATOMIC);
+	return iommu_kmem_cache_alloc(iommu_devinfo_cache);
 }
 
 static inline void free_devinfo_mem(void *vaddr)
@@ -116,7 +138,7 @@
 
 struct iova *alloc_iova_mem(void)
 {
-	return kmem_cache_alloc(iommu_iova_cache, GFP_ATOMIC);
+	return iommu_kmem_cache_alloc(iommu_iova_cache);
 }
 
 void free_iova_mem(struct iova *iova)
#22 lore raw

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

Quoted 2 lines · Anil S

So far in our test scenario, we were unable to create any memory allocation failure inside dma map api calls.

All these functions should have gfp_t flags passed to them. Otherwise you are locked into the use of GFP_ATOMIC. If this is a parameter then you may be able to use GFP_KERNEL in various places and you may develop the code to have less GFP_ATOMIC allocs in the future.

#23 lore raw

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

Quoted 7 lines · Christoph Lameter
Quoted 2 lines · Anil S

So far in our test scenario, we were unable to create any memory allocation failure inside dma map api calls.

All these functions should have gfp_t flags passed to them.

why?

Quoted 1 line

Otherwise you are locked into the use of GFP_ATOMIC.

all callers pretty much are either in irq context or with spinlocks held. Good luck..... it's also called primarily from the PCI DMA API which doesn't take a gfp_t argument in the first place...

so I'm not seeing the point.

#24 lore raw

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

Quoted 7 lines · Arjan van de Ven
Quoted 1 line

Otherwise you are locked into the use of GFP_ATOMIC.

all callers pretty much are either in irq context or with spinlocks held. Good luck..... it's also called primarily from the PCI DMA API which doesn't take a gfp_t argument in the first place...

so I'm not seeing the point.

Hmmm... From my superficial look at things it seems that one could avoid GFP_ATOMIC at times. I do not know too much about the driver though but it seems a bit restrictive to always do GFP_ATOMIC allocs.

#25 lore raw

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

Quoted 4 lines · Christoph Lameter
Quoted 2 lines · Arjan van de Ven
Quoted 1 line

Otherwise you are locked into the use of GFP_ATOMIC.

Quoted 8 lines
Quoted 5 lines

all callers pretty much are either in irq context or with spinlocks held. Good luck..... it's also called primarily from the PCI DMA API which doesn't take a gfp_t argument in the first place...

so I'm not seeing the point.

Hmmm... From my superficial look at things it seems that one could avoid GFP_ATOMIC at times.

by changing ALL drivers. And then you realize the common scenario is that it's taken in irq context ;)

Quoted 2 lines

I do not know too much about the driver though but it seems a bit restrictive to always do GFP_ATOMIC allocs.

the only alternative to GFP_ATOMIC is GFP_NOIO which is... barely better. And then add that it can be used only sporadic... feel free to first change all the drivers.. once the callers of these functions have a gfp_t then I'm sure Anil will be happy to take one of those as well ;-)

#26 lore raw

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

In reply to Keshavamurthy, Anil S
Quoted 9 lines · Anil S

plain text document attachment (intel_iommu_pf_memalloc.patch) Intel IOMMU driver needs memory during DMA map calls to setup its internal page tables and for other data structures. As we all know that these DMA map calls are mostly called in the interrupt context or with the spinlock held by the upper level drivers(network/storage drivers), so in order to avoid any memory allocation failure due to low memory issues, this patch makes memory allocation by temporarily setting PF_MEMALLOC flags for the current task before making memory allocation calls.

Quoted 3 lines

We evaluated mempools as a backup when kmem_cache_alloc() fails and found that mempools are really not useful here because
1) We don;t know for sure how much to reserve in advance

So you just unleashed an unbounded allocation context on PF_MEMALLOC? seems like a really really bad idea.

Quoted 2 lines

2) And mempools are not useful for GFP_ATOMIC case (as we call memory alloc functions with GFP_ATOMIC)

Mempools work as intended with GFP_ATOMIC, it gets filled up to the specified number of elements using GFP_KERNEL (at creation time). This gives GFP_ATOMIC allocations nr_elements extra items once it would start failing.

Quoted 3 lines

With PF_MEMALLOC flag set in the current->flags, the VM subsystem avoids any watermark checks before allocating memory thus guarantee'ing the memory till the last free page.

PF_MEMALLOC as is, is meant to salvage the VM from the typical VM deadlock. Using it as you do now is not something a driver should ever do, and I'm afraid I will have to strongly oppose this patch.

You really really want to calculate an upper bound on your memory consumption and reserve this.

So, I'm afraid I'll have to..

NACK!

#27 lore raw

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

Quoted 4 lines · Peter Zijlstra

PF_MEMALLOC as is, is meant to salvage the VM from the typical VM deadlock.

.. and this IS the typical VM deadlock.. it is your storage driver trying to write out a piece of memory on behalf of the VM, and calls the iommu to map it, which then needs a bit of memory....

#28 lore raw

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

Quoted 9 lines · Arjan van de Ven
Quoted 4 lines · Peter Zijlstra

PF_MEMALLOC as is, is meant to salvage the VM from the typical VM deadlock.

.. and this IS the typical VM deadlock.. it is your storage driver trying to write out a piece of memory on behalf of the VM, and calls the iommu to map it, which then needs a bit of memory....

Today PF_MEMALLOC doesn't do much in interrupt context. If PF_MEMALLOC is the right usage model for this, then we need to fix the behavior of PF_MEMALLOC in the interrupt context(for our usage model, we do most of the allocations in interrupt context).

I am not very familiar with PF_MEMALLOC. So experts please comment.

thanks,
suresh

#29 lore raw

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

Quoted 7 lines · Suresh B
Quoted 6 lines · Arjan van de Ven
Quoted 4 lines · Peter Zijlstra

PF_MEMALLOC as is, is meant to salvage the VM from the typical VM deadlock.

Quoted 8 lines
Quoted 3 lines

.. and this IS the typical VM deadlock.. it is your storage driver trying to write out a piece of memory on behalf of the VM, and calls the iommu to map it, which then needs a bit of memory....

Today PF_MEMALLOC doesn't do much in interrupt context. If PF_MEMALLOC is the right usage model for this, then we need to fix the behavior of PF_MEMALLOC in the interrupt context(for our usage model, we do most of the allocations in interrupt context).

Right, I have patches that add GFP_EMERGENCY to do basically that.

Quoted 1 line

I am not very familiar with PF_MEMALLOC. So experts please comment.

PF_MEMALLOC is meant to avoid the VM deadlock - that is we need memory to free memory. The one constraint is that its use be bounded. (which is currently violated in that there is no bound on the number of direct reclaim contexts - which is on my to-fix list)

So a reclaim context (kswapd and direct reclaim) set PF_MEMALLOC to ensure they themselves will not block on a memory allocation. And it is understood that these code paths have a bounded memory footprint.

Now, this code seems to be running from interrupt context, which makes it impossible to tell if the work is being done on behalf of a reclaim task. Is it possible to setup the needed data for the IRQ handler from process context?

Blindly adding GFP_EMERGENCY to do this, has the distinct disadvantage that there is no inherent bound on the amount of memory consumed. In my patch set I add an emergency reserve (below the current watermarks, because ALLOC_HIGH and ALLOC_HARDER modify the threshold in a relative way, and thus cannot provide a guaranteed limit). I then accurately account all allocations made from this reserve to ensure I never cross the set limit.

Like has been said before, if possible move to blocking allocs (GFP_NOIO), if that is not possible use mempools (for kmem_cache, or page alloc), if that is not possible use ALLOC_NO_WATERMARKS (PF_MEMALLOC, GFP_EMERGENCY) but put in a reserve and account its usage.

The last option basically boils down to reserved based allocation, something which I hope to introduce some-day...

That is, failure is a OK, unless you're from a reclaim context, those should make progress.

One thing I'm confused about, in earlier discussions it was said that mempools are not sufficient because they deplete the GFP_ATOMIC reserve and only then use the mempool. This would not work because some downstream allocation would then go splat --- using PF_MEMALLOC/GFP_EMERGENCY has exactly the same problem!

#30 lore raw

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

Quoted 3 lines · Peter Zijlstra

So a reclaim context (kswapd and direct reclaim) set PF_MEMALLOC to ensure they themselves will not block on a memory allocation. And it is understood that these code paths have a bounded memory footprint.

that's a too simplistic view though; what happens is that kswapd will queue the IO, but the irq context will then take the IO from the queue and do the DMA mapping... which needs the memory.....

#31 lore raw

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

Quoted 9 lines · Arjan van de Ven
Quoted 3 lines · Peter Zijlstra

So a reclaim context (kswapd and direct reclaim) set PF_MEMALLOC to ensure they themselves will not block on a memory allocation. And it is understood that these code paths have a bounded memory footprint.

that's a too simplistic view though; what happens is that kswapd will queue the IO, but the irq context will then take the IO from the queue and do the DMA mapping... which needs the memory.....

Right, but who stops some unrelated interrupt handler from completely depleting memory?

What I'm saying is that there should be some coupling between the reclaim context and the irq context doing work on its behalf.

For instance, you know how many pages are in the queue, and which queue. So you could preallocate enough memory to handle that many pages from irq context and couple that reserve to the queue object. Then irq context can use that memory to do the work.

#32 lore raw

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

Quoted 10 lines · Peter Zijlstra
Quoted 9 lines · Arjan van de Ven
Quoted 3 lines · Peter Zijlstra

So a reclaim context (kswapd and direct reclaim) set PF_MEMALLOC to ensure they themselves will not block on a memory allocation. And it is understood that these code paths have a bounded memory footprint.

that's a too simplistic view though; what happens is that kswapd will queue the IO, but the irq context will then take the IO from the queue and do the DMA mapping... which needs the memory.....

As Arjan is saying, that a reclaim context sets PF_MEMALLOC flag and submits the IO, but the controller driver decides to queue the IO, and later in the interrupt context it de-queues and calls the IOMMU driver for mapping the DMA physical address and in this DMA map api call we may need the memory to satisfy the DMA map api call. Hence PF_MEMALLOC set by the reclaim context should work from interrupt context too, if it is not then that needs to be fixed.

Quoted 3 lines

Right, but who stops some unrelated interrupt handler from completely depleting memory?

The DMA map API's exposed by IOMMU are called by the storage/network controller driver and it is in this IOMMU driver we are talking about allocating memory. So we have no clue how much I/O the controller above us is capable of submitting. And all we do is the mapping of DMA phyical address and provide the caller with the virtual DMA address and it is in this process we may need some memory.

Quoted 3 lines

What I'm saying is that there should be some coupling between the reclaim context and the irq context doing work on its behalf.

Nope this info is not available when upper level drivers calls the standard DMA map api's.

Quoted 5 lines

For instance, you know how many pages are in the queue, and which queue. So you could preallocate enough memory to handle that many pages from irq context and couple that reserve to the queue object. Then irq context can use that memory to do the work.

As I have said earlier, we are not the storage or network controller driver and we have no idea how much the IO the controller is capable of.

Again this patch is the best effort for not to fail the DMA map api, if it fails due to lack of memory, we have no choice but to return failure to upper level driver. But today, upper level drivers are not capable of handling failures from DMA map api's hence this best effort for not to fail the DMA map api call.

thanks,
-Anil

#33 lore raw

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

Quoted 12 lines · Anil S
Quoted 10 lines · Peter Zijlstra
Quoted 9 lines · Arjan van de Ven
Quoted 3 lines · Peter Zijlstra

So a reclaim context (kswapd and direct reclaim) set PF_MEMALLOC to ensure they themselves will not block on a memory allocation. And it is understood that these code paths have a bounded memory footprint.

that's a too simplistic view though; what happens is that kswapd will queue the IO, but the irq context will then take the IO from the queue and do the DMA mapping... which needs the memory.....

Quoted 7 lines

As Arjan is saying, that a reclaim context sets PF_MEMALLOC flag and submits the IO, but the controller driver decides to queue the IO, and later in the interrupt context it de-queues and calls the IOMMU driver for mapping the DMA physical address and in this DMA map api call we may need the memory to satisfy the DMA map api call. Hence PF_MEMALLOC set by the reclaim context should work from interrupt context too, if it is not then that needs to be fixed.

PF_MEMALLOC cannot work from interrupt context, nor should it. But there are other options.

What I'm saying is that if you do use the reserves, you should ensure the use is bounded. I'm not seeing anything like that.

This is a generic API, who is to ensure some other non-swap device will not deplete memory and deadlock the reclaim process?

Also, please do explain why mempools are not usable? those will have exactly the same semantics as you are now getting (if PF_MEMALLOC would work from interrupt context).

Also, explain to me how an IOMMU is different from bounce buffers? They both do the same thing, no? They both need memory in order to complete DMA.

Is it just a broken API you're working against? If so, isn't the Linux way to fix these things, that is why we have the source code after all.

#34 lore raw

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

Quoted 2 lines · Peter Zijlstra

What I'm saying is that if you do use the reserves, you should ensure the use is bounded. I'm not seeing anything like that.

each mapping takes at most 3 pages

Quoted 4 lines

This is a generic API, who is to ensure some other non-swap device will not deplete memory and deadlock the reclaim process?

that information is not available at this level ;(

Quoted 5 lines

Also, explain to me how an IOMMU is different from bounce buffers? They both do the same thing, no? They both need memory in order to complete DMA.

bounce buffers happen in a place where you can sleep.... that makes a lot of difference.

Quoted 3 lines

Is it just a broken API you're working against? If so, isn't the Linux way to fix these things, that is why we have the source code after all.

well yes and no... the other iommu's snuck in as well... it's not entirely fair to hold this one back until a 2 year, 1400 driver project is completed ;(

#35 lore raw

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

Quoted 5 lines · Arjan van de Ven
Quoted 2 lines · Peter Zijlstra

What I'm saying is that if you do use the reserves, you should ensure the use is bounded. I'm not seeing anything like that.

each mapping takes at most 3 pages

That is a start, but the thing I'm worried most about is non-reclaim related devices using the thing when in dire straights.

Quoted 4 lines
Quoted 2 lines

This is a generic API, who is to ensure some other non-swap device will not deplete memory and deadlock the reclaim process?

that information is not available at this level ;(

Can we bring it there?

Quoted 6 lines
Quoted 3 lines

Also, explain to me how an IOMMU is different from bounce buffers? They both do the same thing, no? They both need memory in order to complete DMA.

bounce buffers happen in a place where you can sleep.... that makes a lot of difference.

Right, can't you stick part of this work there?

Quoted 7 lines
Quoted 3 lines

Is it just a broken API you're working against? If so, isn't the Linux way to fix these things, that is why we have the source code after all.

well yes and no... the other iommu's snuck in as well... it's not entirely fair to hold this one back until a 2 year, 1400 driver project is completed ;(

I understand, but at some point we should stop; we cannot keep taking crap in deference of such things.

Also, the other iommu code you pointed me to, was happy to fail, it did not attempt to use the emergency reserves.

But you left out the mempools question again. I have read the earlier threads, and it was said mempools are no good because they first deplete GFP_ATOMIC reserves and then down-stream allocs could go splat. PF_MEMALLOC/GFP_EMERGENCY has exactly the same problem...

So why no mempools?

#36 lore raw

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

Quoted 4 lines · Peter Zijlstra
Quoted 1 line · Arjan van de Ven

Peter Zijlstra wrote:

Also, the other iommu code you pointed me to, was happy to fail, it did not attempt to use the emergency reserves.

Is the same behavior acceptable here?

-Anil

#37 lore raw

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

Quoted 7 lines · Anil S
Quoted 4 lines · Peter Zijlstra
Quoted 1 line · Arjan van de Ven

Peter Zijlstra wrote:

Also, the other iommu code you pointed me to, was happy to fail, it did not attempt to use the emergency reserves.

Is the same behavior acceptable here?

I would say it is. Failure is a part of life.

If you have a (small) mempool with 16 pages or so, that should give you plenty megabytes of io-space to get out of a tight spot. That is, you can queue many pages with that. If it is depleted you know you have at least that many pages outstanding. So failing will just delay the next pages.

Throughput is not a key issue when that low on memory, a guarantee of progress is.

#38 lore raw

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

Quoted 11 lines · Peter Zijlstra
Quoted 7 lines · Anil S
Quoted 4 lines · Peter Zijlstra
Quoted 1 line · Arjan van de Ven

Peter Zijlstra wrote:

Also, the other iommu code you pointed me to, was happy to fail, it did not attempt to use the emergency reserves.

Is the same behavior acceptable here?

I would say it is. Failure is a part of life.

Quoted 8 lines

If you have a (small) mempool with 16 pages or so, that should give you plenty megabytes of io-space to get out of a tight spot. That is, you can queue many pages with that. If it is depleted you know you have at least that many pages outstanding. So failing will just delay the next pages.

Throughput is not a key issue when that low on memory, a guarantee of progress is.

Andrew,
Can you please queue all the other patches except this one for your
next MM release? (Yes, We can safely drop this patch without any issues in applying rest of the patches).

-thanks,
Anil Keshavamurthy

#39 lore raw

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

In reply to Arjan van de Ven
Quoted 5 lines · Arjan van de Ven
Quoted 2 lines · Peter Zijlstra

What I'm saying is that if you do use the reserves, you should ensure the use is bounded. I'm not seeing anything like that.

each mapping takes at most 3 pages

With 3 pages(3 level page table), IOMMU can map at most 2MB and each additional last level page helps map another 2MB. Again, the IOMMU driver re-uses the virtual address instead of giving contiguous virtual address which helps to keep the page table growth and helps reuse the same page table entries, in that sense we are bounded, again we are not sure how much IO will be in flight on a perticular system so as to reserve that many pages for IOMMU page table setup.

-Anil

#40 lore raw

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

In reply to Peter Zijlstra
Quoted 17 lines · On Wed, 20 Jun 2007 10:06:39 +0200 Peter Zijlstra <[email protected]> wrote:
Quoted 9 lines · Anil S

plain text document attachment (intel_iommu_pf_memalloc.patch) Intel IOMMU driver needs memory during DMA map calls to setup its internal page tables and for other data structures. As we all know that these DMA map calls are mostly called in the interrupt context or with the spinlock held by the upper level drivers(network/storage drivers), so in order to avoid any memory allocation failure due to low memory issues, this patch makes memory allocation by temporarily setting PF_MEMALLOC flags for the current task before making memory allocation calls.

Quoted 3 lines

We evaluated mempools as a backup when kmem_cache_alloc() fails and found that mempools are really not useful here because
1) We don;t know for sure how much to reserve in advance

So you just unleashed an unbounded allocation context on PF_MEMALLOC? seems like a really really bad idea.

Quoted 7 lines
Quoted 2 lines

2) And mempools are not useful for GFP_ATOMIC case (as we call memory alloc functions with GFP_ATOMIC)

Mempools work as intended with GFP_ATOMIC, it gets filled up to the specified number of elements using GFP_KERNEL (at creation time). This gives GFP_ATOMIC allocations nr_elements extra items once it would start failing.

Yup. Changelog is buggy.

Quoted 8 lines
Quoted 3 lines

With PF_MEMALLOC flag set in the current->flags, the VM subsystem avoids any watermark checks before allocating memory thus guarantee'ing the memory till the last free page.

PF_MEMALLOC as is, is meant to salvage the VM from the typical VM deadlock. Using it as you do now is not something a driver should ever do, and I'm afraid I will have to strongly oppose this patch.

Quoted 6 lines

You really really want to calculate an upper bound on your memory consumption and reserve this.

So, I'm afraid I'll have to..

NACK!

err, PF_MEMALLOC doesn't actually do anything if in_interrupt(), so your reason-for-nacking isn't legitimate. And neither is Anil's patch ;)

So I'm thinking that if this patch passed all his testing, a patch which didn't play these PF_MEMALLOC games would pass the same tests.

#41 lore raw

PATCH 7/10 Intel iommu cmdline option - forcedac

In reply to Keshavamurthy, Anil S

Introduce intel_iommu=forcedac commandline option.
This option is helpful to verify the pci device capability of handling physical dma'able address greater than 4G.

---
 Documentation/kernel-parameters.txt |    7 +++++++
 drivers/pci/intel-iommu.c           |    7 ++++++-
 2 files changed, 13 insertions(+), 1 deletion(-)
Documentation/kernel-parameters.txt +7 -0
Index: linux-2.6.22-rc4-mm2/Documentation/kernel-parameters.txt
===================================================================
@@ -788,6 +788,13 @@
 			bypassed by not enabling DMAR with this option. In
 			this case, gfx device will use physical address for
 			DMA.
+		forcedac [x86_64]
+			With this option iommu will not optimize to look
+			for io virtual address below 32 bit forcing dual
+			address cycle on pci bus for cards supporting greater
+			than 32 bit addressing. The default is to look
+			for translation below 32 bit and if not available
+			then look in the higher range.
 
 	io7=		[HW] IO7 for Marvel based alpha systems
 			See comment before marvel_specify_io7 in
drivers/pci/intel-iommu.c +6 -1
Index: linux-2.6.22-rc4-mm2/drivers/pci/intel-iommu.c
===================================================================
@@ -53,6 +53,7 @@
 
 static int dmar_disabled;
 static int __initdata dmar_map_gfx = 1;
+static int dmar_forcedac;
 
 #define DUMMY_DEVICE_DOMAIN_INFO ((struct device_domain_info *)(-1))
 static DEFINE_SPINLOCK(device_domain_lock);
@@ -70,6 +71,10 @@
 			dmar_map_gfx = 0;
 			printk(KERN_INFO
 				"Intel-IOMMU: disable GFX device mapping\n");
+		} else if (!strncmp(str, "forcedac", 8)) {
+			printk (KERN_INFO
+				"Intel-IOMMU: Forcing DAC for PCI devices\n");
+			dmar_forcedac = 1;
 		}
 
 		str += strcspn(str, ",");
@@ -1557,7 +1562,7 @@
 
 	start_addr = IOVA_START_ADDR;
 
-	if (pdev->dma_mask <= DMA_32BIT_MASK) {
+	if ((pdev->dma_mask <= DMA_32BIT_MASK) || (dmar_forcedac)) {
 		iova = iommu_alloc_iova(domain, addr, size, start_addr,
 			pdev->dma_mask);
 	} else  {
#42 lore raw

PATCH 8/10 DMAR fault handling support

In reply to Keshavamurthy, Anil S

MSI interrupt handler registrations and fault handling support
for Intel-IOMMU hadrware.

This patch enables the MSI interrupts for the DMA remapping units and in the interrupt handler read the fault cause and outputs the same on to the console.

---
 Documentation/Intel-IOMMU.txt |   17 +++
 arch/x86_64/kernel/io_apic.c  |   59 ++++++++++++
 drivers/pci/intel-iommu.c     |  194 ++++++++++++++++++++++++++++++++++++++++++
 include/linux/dmar.h          |   12 ++
 4 files changed, 281 insertions(+), 1 deletion(-)
Documentation/Intel-IOMMU.txt +17 -0
Index: linux-2.6.22-rc4-mm2/Documentation/Intel-IOMMU.txt
===================================================================
@@ -63,6 +63,15 @@
 The same is true for peer to peer transactions. Hence we reserve the
 address from PCI MMIO ranges so they are not allocated for IOVA addresses.
 
+
+Fault reporting
+---------------
+When errors are reported, the DMA engine signals via an interrupt. The fault
+reason and device that caused it with fault reason is printed on console.
+
+See below for sample.
+
+
 Boot Message Sample
 -------------------
 
@@ -85,6 +94,14 @@
 
 PCI-DMA: Using DMAR IOMMU
 
+Fault reporting
+---------------
+
+DMAR:[DMA Write] Request device [00:02.0] fault addr 6df084000
+DMAR:[fault reason 05] PTE Write access is not set
+DMAR:[DMA Write] Request device [00:02.0] fault addr 6df084000
+DMAR:[fault reason 05] PTE Write access is not set
+
 TBD
 ----
 
arch/x86_64/kernel/io_apic.c +58 -1

72 lines not on this page · raw message

drivers/pci/intel-iommu.c +194 -0

206 lines not on this page · raw message

include/linux/dmar.h +12 -0
Index: linux-2.6.22-rc4-mm2/include/linux/dmar.h
===================================================================
@@ -28,6 +28,18 @@
 #ifdef CONFIG_DMAR
 struct intel_iommu;
 
+extern char *dmar_get_fault_reason(u8 fault_reason);
+
+/* Can't use the common MSI interrupt functions
+ * since DMAR is not a pci device
+ */
+extern void dmar_msi_unmask(unsigned int irq);
+extern void dmar_msi_mask(unsigned int irq);
+extern void dmar_msi_read(int irq, struct msi_msg *msg);
+extern void dmar_msi_write(int irq, struct msi_msg *msg);
+extern int dmar_set_interrupt(struct intel_iommu *iommu);
+extern int arch_setup_dmar_msi(unsigned int irq);
+
 /* Intel IOMMU detection and initialization functions */
 extern void detect_intel_iommu(void);
 extern int intel_iommu_init(void);
#43 lore raw

PATCH 9/10 Iommu Gfx workaround

In reply to Keshavamurthy, Anil S

When we fix all the opensource gfx drivers to use the DMA api's, at that time we can yank this config options out.

---
 Documentation/Intel-IOMMU.txt |    5 +++++
 arch/x86_64/Kconfig           |   11 +++++++++++
 arch/x86_64/kernel/e820.c     |   19 +++++++++++++++++++
 drivers/pci/intel-iommu.c     |   33 +++++++++++++++++++++++++++++++++
 drivers/pci/intel-iommu.h     |    7 +++++++
 5 files changed, 75 insertions(+)
Documentation/Intel-IOMMU.txt +5 -0
Index: linux-2.6.22-rc4-mm2/Documentation/Intel-IOMMU.txt
===================================================================
@@ -57,6 +57,11 @@
 If you encounter issues with graphics devices, you can try adding
 option intel_iommu=igfx_off to turn off the integrated graphics engine.
 
+If it happens to be a PCI device included in the INCLUDE_ALL Engine,
+then try enabling CONFIG_DMAR_GFX_WA to setup a 1-1 map. We hear
+graphics drivers may be in process of using DMA api's in the near
+future and at that time this option can be yanked out.
+
 Some exceptions to IOVA
 -----------------------
 Interrupt ranges are not address translated, (0xfee00000 - 0xfeefffff).
arch/x86_64/Kconfig +11 -0
Index: linux-2.6.22-rc4-mm2/arch/x86_64/Kconfig
===================================================================
@@ -741,6 +741,17 @@
 	  and includes pci device scope covered by these DMA
 	  remapping device.
 
+config DMAR_GFX_WA
+	bool "Support for Graphics workaround"
+	depends on DMAR
+	default y
+	help
+	 Current Graphics drivers tend to use physical address
+	 for DMA and avoid using DMA api's. Setting this config
+	 option permits the IOMMU driver to set a unity map for
+	 all the OS visible memory. Hence the driver can continue
+	 to use physical addresses for DMA.
+
 source "drivers/pci/pcie/Kconfig"
 
 source "drivers/pci/Kconfig"
arch/x86_64/kernel/e820.c +19 -0
Index: linux-2.6.22-rc4-mm2/arch/x86_64/kernel/e820.c
===================================================================
@@ -723,3 +723,22 @@
 	printk(KERN_INFO "Allocating PCI resources starting at %lx (gap: %lx:%lx)\n",
 		pci_mem_start, gapstart, gapsize);
 }
+
+int __init arch_get_ram_range(int slot, u64 *addr, u64 *size)
+{
+	int i;
+
+	if (slot < 0 || slot >= e820.nr_map)
+		return -1;
+	for (i = slot; i < e820.nr_map; i++) {
+		if (e820.map[i].type != E820_RAM)
+			continue;
+		break;
+	}
+	if (i == e820.nr_map || e820.map[i].addr > (max_pfn << PAGE_SHIFT))
+		return -1;
+	*addr = e820.map[i].addr;
+	*size = min_t(u64, e820.map[i].size + e820.map[i].addr,
+		max_pfn << PAGE_SHIFT) - *addr;
+	return i + 1;
+}
drivers/pci/intel-iommu.c +33 -0
Index: linux-2.6.22-rc4-mm2/drivers/pci/intel-iommu.c
===================================================================
@@ -1601,6 +1601,36 @@
 		rmrr->end_address + 1);
 }
 
+#ifdef CONFIG_DMAR_GFX_WA
+extern int arch_get_ram_range(int slot, u64 *addr, u64 *size);
+static void __init iommu_prepare_gfx_mapping(void)
+{
+	struct pci_dev *pdev = NULL;
+	u64 base, size;
+	int slot;
+	int ret;
+
+	for_each_pci_dev(pdev) {
+		if (pdev->sysdata == DUMMY_DEVICE_DOMAIN_INFO ||
+				!IS_GFX_DEVICE(pdev))
+			continue;
+		printk(KERN_INFO "IOMMU: gfx device %s 1-1 mapping\n",
+			pci_name(pdev));
+		slot = arch_get_ram_range(0, &base, &size);
+		while (slot >= 0) {
+			ret = iommu_prepare_identity_map(pdev,
+					base, base + size);
+			if (ret)
+				goto error;
+			slot = arch_get_ram_range(slot, &base, &size);
+		}
+		continue;
+error:
+		printk(KERN_ERR "IOMMU: mapping reserved region failed\n");
+	}
+}
+#endif
+
 int __init init_dmars(void)
 {
 	struct dmar_drhd_unit *drhd;
@@ -1664,6 +1694,8 @@
 		}
 	}
 
+	iommu_prepare_gfx_mapping();
+
 	/*
 	 * for each drhd
 	 *   enable fault log
@@ -2175,3 +2207,4 @@
 	dma_ops = &intel_dma_ops;
 	return 0;
 }
+
drivers/pci/intel-iommu.h +7 -0
Index: linux-2.6.22-rc4-mm2/drivers/pci/intel-iommu.h
===================================================================
@@ -315,4 +315,11 @@
 	struct sys_device sysdev;
 };
 
+#ifndef CONFIG_DMAR_GFX_WA
+static inline void iommu_prepare_gfx_mapping(void)
+{
+	return;
+}
+#endif /* !CONFIG_DMAR_GFX_WA */
+
 #endif
#44 lore raw

PATCH 10/10 Iommu floppy workaround

In reply to Keshavamurthy, Anil S

This config option (DMAR_FLPY_WA) sets up 1:1 mapping for the
floppy device so that the floppy device which does not use DMA api's will continue to work.

Once the floppy driver starts using DMA api's this config option can be turn off or this patch can be yanked out of kernel at that time.

---
 arch/x86_64/Kconfig       |   10 ++++++++++
 drivers/pci/intel-iommu.c |   22 ++++++++++++++++++++++
 drivers/pci/intel-iommu.h |    7 +++++++
 3 files changed, 39 insertions(+)
arch/x86_64/Kconfig +10 -0
Index: linux-2.6.22-rc4-mm2/arch/x86_64/Kconfig
===================================================================
@@ -752,6 +752,16 @@
 	 all the OS visible memory. Hence the driver can continue
 	 to use physical addresses for DMA.
 
+config DMAR_FLPY_WA
+	bool "Support for Floppy disk workaround"
+	depends on DMAR
+	default y
+	help
+	 Floppy disk drivers are know to by pass dma api calls
+	 their by failing to work when IOMMU is enabled. This
+	 work around will setup a 1 to 1 mappings for the first
+	 16M to make floppy(isa device) work.
+
 source "drivers/pci/pcie/Kconfig"
 
 source "drivers/pci/Kconfig"
drivers/pci/intel-iommu.c +22 -0
Index: linux-2.6.22-rc4-mm2/drivers/pci/intel-iommu.c
===================================================================
@@ -1631,6 +1631,26 @@
 }
 #endif
 
+#ifdef CONFIG_DMAR_FLPY_WA
+static inline void iommu_prepare_isa(void)
+{
+	struct pci_dev *pdev = NULL;
+	int ret;
+
+	pdev = pci_get_class (PCI_CLASS_BRIDGE_ISA << 8, NULL);
+	if (!pdev)
+		return;
+
+	printk (KERN_INFO "IOMMU: Prepare 0-16M unity mapping for LPC\n");
+	ret = iommu_prepare_identity_map(pdev, 0, 16*1024*1024);
+
+	if (ret)
+		printk ("IOMMU: Failed to create 0-64M identity map, \
+			Floppy might not work\n");
+
+}
+#endif
+
 int __init init_dmars(void)
 {
 	struct dmar_drhd_unit *drhd;
@@ -1696,6 +1716,8 @@
 
 	iommu_prepare_gfx_mapping();
 
+	iommu_prepare_isa();
+
 	/*
 	 * for each drhd
 	 *   enable fault log
drivers/pci/intel-iommu.h +7 -0
Index: linux-2.6.22-rc4-mm2/drivers/pci/intel-iommu.h
===================================================================
@@ -322,4 +322,11 @@
 }
 #endif /* !CONFIG_DMAR_GFX_WA */
 
+#ifndef CONFIG_DMAR_FLPY_WA
+static inline void iommu_prepare_isa(void)
+{
+	return;
+}
+#endif /* !CONFIG_DMAR_FLPY_WA */
+
 #endif
#45 lore raw

Re: [Intel IOMMU 10/10] Iommu floppy workaround

Quoted 21 lines · Anil S

This config option (DMAR_FLPY_WA) sets up 1:1 mapping for the
floppy device so that the floppy device which does not use DMA api's will continue to work.

Once the floppy driver starts using DMA api's this config option can be turn off or this patch can be yanked out of kernel at that time.

---
 arch/x86_64/Kconfig       |   10 ++++++++++
 drivers/pci/intel-iommu.c |   22 ++++++++++++++++++++++
 drivers/pci/intel-iommu.h |    7 +++++++
 3 files changed, 39 insertions(+)

Index: linux-2.6.22-rc4-mm2/arch/x86_64/Kconfig
===================================================================

arch/x86_64/Kconfig line 755

Quoted 4 lines
arch/x86_64/Kconfig +1 -0
@@ -752,6 +752,16 @@
 	 all the OS visible memory. Hence the driver can continue
 	 to use physical addresses for DMA.
 
+config DMAR_FLPY_WA

FLOPPY is spelled "FLOPPY"!

Quoted 44 lines

===================================================================

drivers/pci/intel-iommu.c +22 -0
@@ -1631,6 +1631,26 @@
 }
 #endif
 
+#ifdef CONFIG_DMAR_FLPY_WA
+static inline void iommu_prepare_isa(void)
+{
+	struct pci_dev *pdev = NULL;
+	int ret;
+
+	pdev = pci_get_class (PCI_CLASS_BRIDGE_ISA << 8, NULL);
+	if (!pdev)
+		return;
+
+	printk (KERN_INFO "IOMMU: Prepare 0-16M unity mapping for LPC\n");
+	ret = iommu_prepare_identity_map(pdev, 0, 16*1024*1024);
+
+	if (ret)
+		printk ("IOMMU: Failed to create 0-64M identity map, \
+			Floppy might not work\n");
+
+}
+#endif
+
 int __init init_dmars(void)
 {
 	struct dmar_drhd_unit *drhd;
@@ -1696,6 +1716,8 @@
 
 	iommu_prepare_gfx_mapping();
 
+	iommu_prepare_isa();
+
 	/*
 	 * for each drhd
 	 *   enable fault log

Index: linux-2.6.22-rc4-mm2/drivers/pci/intel-iommu.h
===================================================================

drivers/pci/intel-iommu.h line 330

Quoted 9 lines
drivers/pci/intel-iommu.h +6 -0
@@ -322,4 +322,11 @@
 }
 #endif /* !CONFIG_DMAR_GFX_WA */
 
+#ifndef CONFIG_DMAR_FLPY_WA
+static inline void iommu_prepare_isa(void)
+{
+	return;
+}
+#endif /* !CONFIG_DMAR_FLPY_WA */

Bit weird that this was implemented in the header like that.

How about this? (Also contains rather a lot of obvious style fixes)

 arch/x86_64/Kconfig       |    2 +-
 drivers/pci/intel-iommu.c |   19 ++++++++++++-------
 drivers/pci/intel-iommu.h |    7 -------
 3 files changed, 13 insertions(+), 15 deletions(-)
arch/x86_64/Kconfig renamed from arch/x86_64/Kconfig~intel-iommu-iommu-floppy-workaround-fix +1 -1
diff -puN arch/x86_64/Kconfig~intel-iommu-iommu-floppy-workaround-fix arch/x86_64/Kconfig
@@ -770,7 +770,7 @@ config DMAR_GFX_WA
 	 all the OS visible memory. Hence the driver can continue
 	 to use physical addresses for DMA.
 
-config DMAR_FLPY_WA
+config DMAR_FLOPPY_WA
 	bool "Support for Floppy disk workaround"
 	depends on DMAR
 	default y
drivers/pci/intel-iommu.c renamed from drivers/pci/intel-iommu.c~intel-iommu-iommu-floppy-workaround-fix +12 -7
diff -puN drivers/pci/intel-iommu.c~intel-iommu-iommu-floppy-workaround-fix drivers/pci/intel-iommu.c
@@ -1631,25 +1631,30 @@ error:
 }
 #endif
 
-#ifdef CONFIG_DMAR_FLPY_WA
+#ifdef CONFIG_DMAR_FLOPPY_WA
 static inline void iommu_prepare_isa(void)
 {
-	struct pci_dev *pdev = NULL;
+	struct pci_dev *pdev;
 	int ret;
 
-	pdev = pci_get_class (PCI_CLASS_BRIDGE_ISA << 8, NULL);
+	pdev = pci_get_class(PCI_CLASS_BRIDGE_ISA << 8, NULL);
 	if (!pdev)
 		return;
 
-	printk (KERN_INFO "IOMMU: Prepare 0-16M unity mapping for LPC\n");
+	printk(KERN_INFO "IOMMU: Prepare 0-16M unity mapping for LPC\n");
 	ret = iommu_prepare_identity_map(pdev, 0, 16*1024*1024);
 
 	if (ret)
-		printk ("IOMMU: Failed to create 0-64M identity map, \
-			Floppy might not work\n");
+		printk("IOMMU: Failed to create 0-64M identity map, "
+			"floppy might not work\n");
 
 }
-#endif
+#else
+static inline void iommu_prepare_isa(void)
+{
+	return;
+}
+#endif /* !CONFIG_DMAR_FLPY_WA */
 
 int __init init_dmars(void)
 {
drivers/pci/intel-iommu.h renamed from drivers/pci/intel-iommu.h~intel-iommu-iommu-floppy-workaround-fix +0 -7
diff -puN drivers/pci/intel-iommu.h~intel-iommu-iommu-floppy-workaround-fix drivers/pci/intel-iommu.h
@@ -322,11 +322,4 @@ static inline void iommu_prepare_gfx_map
 }
 #endif /* !CONFIG_DMAR_GFX_WA */
 
-#ifndef CONFIG_DMAR_FLPY_WA
-static inline void iommu_prepare_isa(void)
-{
-	return;
-}
-#endif /* !CONFIG_DMAR_FLPY_WA */
-
 #endif

_

#46 lore raw

Re: [Intel IOMMU 10/10] Iommu floppy workaround

Quoted 9 lines
Quoted 9 lines

Index: linux-2.6.22-rc4-mm2/arch/x86_64/Kconfig
===================================================================

arch/x86_64/Kconfig +0 -0
@@ -752,6 +752,16 @@
 	 all the OS visible memory. Hence the driver can continue
 	 to use physical addresses for DMA.
 
Quoted 3 lines

arch/x86_64/Kconfig line 755

Quoted 1 line
arch/x86_64/Kconfig +1 -0
+config DMAR_FLPY_WA

FLOPPY is spelled "FLOPPY"!

Also this shouldn't be a user visible config. The floppy driver should just do this transparently when loaded and undo when unloaded.

Otherwise it would be a CONFIG_MAKE_MY_FLOPPY_WORK which is just not very nice.

-Andi

#47 lore raw

Re: [Intel IOMMU 10/10] Iommu floppy workaround

Quoted 9 lines · Andi Kleen
Quoted 9 lines
Quoted 9 lines

Index: linux-2.6.22-rc4-mm2/arch/x86_64/Kconfig
===================================================================

arch/x86_64/Kconfig +0 -0
@@ -752,6 +752,16 @@
 	 all the OS visible memory. Hence the driver can continue
 	 to use physical addresses for DMA.
 
Quoted 6 lines
Quoted 3 lines

arch/x86_64/Kconfig line 755

Quoted 1 line
arch/x86_64/Kconfig +1 -0
+config DMAR_FLPY_WA

FLOPPY is spelled "FLOPPY"!

Also this shouldn't be a user visible config. The floppy driver should just do this transparently when loaded and undo when unloaded.

Yup, I agree.

Here goes the patch to make it user invisible config option.

---
 arch/x86_64/Kconfig |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86_64/Kconfig +1 -1
Index: linux-2.6.22-rc4-mm2/arch/x86_64/Kconfig
===================================================================
@@ -753,7 +753,7 @@
 	 to use physical addresses for DMA.
 
 config DMAR_FLOPPY_WA
-	bool "Support for Floppy disk workaround"
+	bool
 	depends on DMAR
 	default y
 	help
#48 lore raw

Re: [Intel IOMMU 10/10] Iommu floppy workaround

In reply to Andrew Morton
Quoted 3 lines · Andrew Morton

On Tue, 19 Jun 2007 14:37:11 -0700 "Keshavamurthy, Anil S" <[email protected]> wrote:

Bit weird that this was implemented in the header like that.

Sorry, it is my mistake as I understood thus from your previous code review comment.

Quoted 2 lines

How about this? (Also contains rather a lot of obvious style fixes)

Yup, looks good.

-Anil

#49 lore raw
In reply to Keshavamurthy, Anil S
Quoted 3 lines · Anil S

This patch supports the upcomming Intel IOMMU hardware
a.k.a. Intel(R) Virtualization Technology for Directed I/O Architecture

So... what's all this code for?

I assume that the intent here is to speed things up under Xen, etc? Do we have any benchmark results to help us to decide whether a merge would be justified?

Does it slow anything down?

#50 lore raw
Quoted 9 lines · On Tuesday 26 June 2007 08:45:50 Andrew Morton wrote:
Quoted 3 lines · Anil S

This patch supports the upcomming Intel IOMMU hardware
a.k.a. Intel(R) Virtualization Technology for Directed I/O Architecture

So... what's all this code for?

I assume that the intent here is to speed things up under Xen, etc?

Yes in some cases, but not this code. That would be the Xen version of this code that could potentially assign whole devices to guests. I expect this to be only useful in some special cases though because most hardware is not virtualizable and you typically want an own instance for each guest.

Ok at some point KVM might implement this too; i likely would use this code for this.

Quoted 3 lines

Do we
have any benchmark results to help us to decide whether a merge would be justified?

The main advantage for doing it in the normal kernel is not performance, but more safety. Broken devices won't be able to corrupt memory by doing random DMA.

Unfortunately that doesn't work for graphics yet, for that need user space interfaces for the X server are needed.

There are some potential performance benefits too:
- When you have a device that cannot address the complete address range an IOMMU can remap its memory instead of bounce buffering. Remapping is likely cheaper than copying.
- The IOMMU can merge sg lists into a single virtual block. This could potentially speed up SG IO when the device is slow walking SG lists. [I long ago benchmarked 5% on some block benchmark with an old MPT Fusion; but it probably depends a lot on the HBA]

And you get better driver debugging because unexpected memory accesses from the devices will cause an trapable event.

Quoted 2 lines

Does it slow anything down?

It adds more overhead to each IO so yes.

-Andi

#51 lore raw
Quoted 4 lines · Andi Kleen

There are some potential performance benefits too:
- When you have a device that cannot address the complete address range an IOMMU can remap its memory instead of bounce buffering. Remapping is likely cheaper than copying.

But those devices aren't likely to be found on modern systems.

Quoted 4 lines

- The IOMMU can merge sg lists into a single virtual block. This could potentially speed up SG IO when the device is slow walking SG lists. [I long ago benchmarked 5% on some block benchmark with an old MPT Fusion; but it probably depends a lot on the HBA]

But most devices are SG-capable.

Quoted 2 lines

And you get better driver debugging because unexpected memory accesses from the devices will cause an trapable event.

That and direct-access for KVM the big ones, IMHO, and definitely justify merging.

Quoted 3 lines
Quoted 1 line

Does it slow anything down?

It adds more overhead to each IO so yes.

How much? we have numbers (to be presented at OLS later this week) that show that on bare-metal an IOMMU can cost as much as 15%-30% more CPU utilization for an IO intensive workload (netperf). It will be interesting to see comparable numbers for VT-d.

Cheers,
Muli

#52 lore raw
Quoted 8 lines · Muli Ben-Yehuda
Quoted 4 lines · Andi Kleen

There are some potential performance benefits too:
- When you have a device that cannot address the complete address range an IOMMU can remap its memory instead of bounce buffering. Remapping is likely cheaper than copying.

But those devices aren't likely to be found on modern systems.

Not true. I don't see anybody designing DAC capable USB or firewire or sound or TV cards. And there are plenty of non AHCI SATA interfaces too (often the BIOS defaults are this way because XP doesn't deal well with AHCI). And video cards generally don't support it (although they don't like IOMMUs either). Just these devices all might not be performance relevant (except for the video cards)

Quoted 6 lines
Quoted 4 lines

- The IOMMU can merge sg lists into a single virtual block. This could potentially speed up SG IO when the device is slow walking SG lists. [I long ago benchmarked 5% on some block benchmark with an old MPT Fusion; but it probably depends a lot on the HBA]

But most devices are SG-capable.

Your point being? It depends on if the SG hardware is slow enough that it makes a difference. I found one case where that was true, but it's unknown how common that is.

Only benchmarks can tell.

Also my results were on a pretty slow IOMMU implementation so with a fast one it might be different too.

Quoted 4 lines

How much? we have numbers (to be presented at OLS later this week) that show that on bare-metal an IOMMU can cost as much as 15%-30% more CPU utilization for an IO intensive workload (netperf). It will be interesting to see comparable numbers for VT-d.

That is something that needs more work.

We should probably have a switch to use the IOMMU only for specific devices (e.g. for the KVM case) r only when remapping is needed. Only boot options for this is probably not good enough. But that is something that can be worked on once everything is in tree.

Also the user interface for X server case needs more work.

-Andi

#53 lore raw
Quoted 8 lines · Andi Kleen
Quoted 6 lines
Quoted 4 lines

- The IOMMU can merge sg lists into a single virtual block. This could potentially speed up SG IO when the device is slow walking SG lists. [I long ago benchmarked 5% on some block benchmark with an old MPT Fusion; but it probably depends a lot on the HBA]

But most devices are SG-capable.

Your point being?

That the fact that an IOMMU can do SG for non-SG-capble cards is not interesting from a "reason for inclusion" POV.

Quoted 6 lines
Quoted 4 lines

How much? we have numbers (to be presented at OLS later this week) that show that on bare-metal an IOMMU can cost as much as 15%-30% more CPU utilization for an IO intensive workload (netperf). It will be interesting to see comparable numbers for VT-d.

That is something that needs more work.

Yup. I'm working on it (mostly in the context of Calgary) but also looking at improvements to the DMA-API interface and usage.

Quoted 3 lines

We should probably have a switch to use the IOMMU only for specific devices (e.g. for the KVM case) r only when remapping is needed.

Calgary already does this internally (via calgary=disable=<BUSNUM>) but that's pretty ugly. It would be better to do it in a generic fashion when deciding which dma_ops to call (i.e., a dma_ops per bus or device).

Quoted 1 line

Also the user interface for X server case needs more work.

Is anyone working on it?

Cheers,
Muli

#54 lore raw
Quoted 7 lines · Muli Ben-Yehuda
Quoted 5 lines · Andi Kleen
Quoted 5 lines
Quoted 4 lines

- The IOMMU can merge sg lists into a single virtual block. This could potentially speed up SG IO when the device is slow walking SG lists. [I long ago benchmarked 5% on some block benchmark with an old MPT Fusion; but it probably depends a lot on the HBA]

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

But most devices are SG-capable.

Your point being?

That the fact that an IOMMU can do SG for non-SG-capble cards is not interesting from a "reason for inclusion" POV.

You misunderstood me; my point was that some SG capable devices can go faster if they get shorter SG lists.

But yes for non SG capable devices it is also interesting. I expect it will obsolete most users of that ugly external patch to allocate large memory areas for IOs. That's a point I didn't mention earlier.

Quoted 3 lines
Quoted 1 line

Also the user interface for X server case needs more work.

Is anyone working on it?

It's somewhere on the todo list.

-Andi

#55 lore raw
In reply to Andi Kleen
Quoted 3 lines

Also the user interface for X server case needs more work.

actually with the mode setting of X moving into the kernel... X won't use /dev/mem anymore at all (and I think it mostly already doesn't even without that)

#56 lore raw
Quoted 6 lines · Arjan van de Ven
Quoted 3 lines

Also the user interface for X server case needs more work.

actually with the mode setting of X moving into the kernel... X won't use /dev/mem anymore at all

We'll see if that happens. It has been talked about forever, but results are sparse.

Quoted 1 line

(and I think it mostly already doesn't even without that)

It uses /sys/bus/pci/* which is not any better as seen from the IOMMU.

Any interface will need to be explicit because user space needs to know which DMA addresses to put into the hardware. It's not enough to just transparently translate the mappings.

-Andi

#57 lore raw
Quoted 7 lines · Andi Kleen
Quoted 3 lines · Arjan van de Ven
Quoted 2 lines

Also the user interface for X server case needs more work.

actually with the mode setting of X moving into the kernel... X won't use /dev/mem anymore at all

We'll see if that happens. It has been talked about forever, but results are sparse.

jbarnes posted the code a few weeks ago.

Quoted 8 lines
Quoted 2 lines

(and I think it mostly already doesn't even without that)

It uses /sys/bus/pci/* which is not any better as seen from the IOMMU.

Any interface will need to be explicit because user space needs to know which DMA addresses to put into the hardware. It's not enough to just transparently translate the mappings.

that's what DRM is used for nowadays...

#58 lore raw
Quoted 11 lines
Quoted 9 lines
Quoted 1 line

(and I think it mostly already doesn't even without that)

It uses /sys/bus/pci/* which is not any better as seen from the IOMMU.

Any interface will need to be explicit because user space needs to know which
DMA addresses to put into the hardware. It's not enough to just transparently
translate the mappings.

that's what DRM is used for nowadays...

But DRM does support much less hardware than the X server?

Perhaps we just need an ioctl where an X server can switch this.

-Andi

#59 lore raw
Quoted 11 lines · On Tuesday, June 26, 2007 10:31:57 Andi Kleen wrote:
Quoted 11 lines
Quoted 10 lines
Quoted 1 line

(and I think it mostly already doesn't even without that)

It uses /sys/bus/pci/* which is not any better as seen from the IOMMU.

Any interface will need to be explicit because user space needs to know which
DMA addresses to put into the hardware. It's not enough to just transparently
translate the mappings.

Quoted 3 lines
Quoted 1 line

that's what DRM is used for nowadays...

But DRM does support much less hardware than the X server?

Yeah, the number of DRM drivers is relatively small compared to X or fbdev, but for simple DMA they're fairly easy to write.

Quoted 1 line

Perhaps we just need an ioctl where an X server can switch this.

Switch what? Turn on or off transparent translation?

Jesse

#60 lore raw
Quoted 4 lines
Quoted 2 lines

Perhaps we just need an ioctl where an X server can switch this.

Switch what? Turn on or off transparent translation?

Turn on/off bypass for its device.

-Andi

#61 lore raw
In reply to Muli Ben-Yehuda
Quoted 4 lines · Muli Ben-Yehuda

How much? we have numbers (to be presented at OLS later this week) that show that on bare-metal an IOMMU can cost as much as 15%-30% more CPU utilization for an IO intensive workload (netperf). It will be interesting to see comparable numbers for VT-d.

for VT-d it is a LOT less. I'll let anil give you his data :)

#62 lore raw
Quoted 7 lines · Arjan van de Ven
Quoted 4 lines · Muli Ben-Yehuda

How much? we have numbers (to be presented at OLS later this week) that show that on bare-metal an IOMMU can cost as much as 15%-30% more CPU utilization for an IO intensive workload (netperf). It will be interesting to see comparable numbers for VT-d.

for VT-d it is a LOT less. I'll let anil give you his data :)

Looking forward to it. Note that this is on a large SMP machine with Gigabit ethernet, with netperf TCP stream. Comparing numbers for other benchmarks on other machines is ... less than useful, but the numbers themeselves are interesting.

Cheers,
Muli

#63 lore raw
Quoted 7 lines · Muli Ben-Yehuda
Quoted 6 lines · Arjan van de Ven
Quoted 4 lines · Muli Ben-Yehuda

How much? we have numbers (to be presented at OLS later this week) that show that on bare-metal an IOMMU can cost as much as 15%-30% more CPU utilization for an IO intensive workload (netperf). It will be interesting to see comparable numbers for VT-d.

Quoted 6 lines
Quoted 1 line

for VT-d it is a LOT less. I'll let anil give you his data :)

Looking forward to it. Note that this is on a large SMP machine with Gigabit ethernet, with netperf TCP stream. Comparing numbers for other benchmarks on other machines is ... less than useful, but the numbers themeselves are interesting.

Our initial benchmark results showed we had around 3% extra CPU utilization overhead when compared to native(i.e without IOMMU). Again, our benchmark was on small SMP machine and we used iperf and a 1G ethernet cards.

Going forward we will do more benchmark tests and will share the results.

-Anil

#64 lore raw
Quoted 4 lines · Anil S

Our initial benchmark results showed we had around 3% extra CPU utilization overhead when compared to native(i.e without IOMMU). Again, our benchmark was on small SMP machine and we used iperf and a 1G ethernet cards.

Please try netperf and a bigger machine for a meaningful comparison :-) I assume this is with e1000?

Quoted 2 lines

Going forward we will do more benchmark tests and will share the results.

Looking forward to it.

Cheers,
Muli