KVM device passthrough support with AMD IOMMU

PATCHSETS #2 New Joerg Roedel 4 participants 45 messages Dec 9, 2008 – Dec 15, 2008

Subsystems (6)

x86 KVM PCI Driver core +2 more subsystems

lore

Share & tools

lore

Joerg Roedel

author

2008-12-09 14:12 UTC #0 lore raw

Hi,

the two patchsets posted as reply to this email implement KVM device passthrough support for AMD IOMMU hardware. The changes to the previous posts are descibed below

The first patchset is version 4 of the generic iommu api patchset which generalizes the VT-d functions exported to KVM into a common api where the AMD IOMMU code can plug into. In this version the patchset was rebased to the latest post of Han Weidong's patches.

The second patchset finally implements the KVM device passthrough support in the AMD IOMMU code. Together with KVM-79 I successfully passed an 10GBit network card into an KVM guest. In this version the patchset was changed to remove any device before a protection domain is freed instead of printing a BUG. Also the patchset was rebased to the updated IOMMU-API patches.

These two patchsets apply in order in top of the latest post of Han Weidong's Multiple device assignement support patches. Anybody who wants to try this out can pull the whole stuff from

git://git.kernel.org/pub/scm/linux/kernel/git/joro/linux-2.6-iommu.git kvm-amd-iommu

Please give these patches a good review.

Thanks,

Joerg

Signature · 6 lines

           |           AMD Saxony Limited Liability Company & Co. KG
 Operating |         Wilschdorfer Landstr. 101, 01109 Dresden, Germany
 System    |                  Register Court Dresden: HRA 4896
 Research  |              General Partner authorized to represent:
 Center    |             AMD Saxony LLC (Wilmington, Delaware, US)
           | General Manager of AMD Saxony LLC: Dr. Hans-R. Deppe, Thomas McCoy

Joerg Roedel

author

2008-12-09 14:16 UTC #1 lore raw

cover Factor VT-d KVM functions into a generic API

This patch series makes the current KVM device passthrough code generic enough so that other IOMMU implementation can also plug into this code. It works by factoring the functions Vt-d code exports to KVM into a generic interface which allows different backends.

This is the fourth version of the patchset. I rebased these patches onto the 17-patches post of Han Weidongs multiple device assignment work and included changes to the commit-messages according the comments I got from the review.

This a basic implementation of a generic interface. It can and should be improved later to support more types of hardware IOMMUs then VT-d and AMD IOMMU.

Since I have no VT-d hardware available these patches are only compile tested on x86 and ia64 for now.

Please review, comment and test these patches.

Thanks,

Joerg

diffstat:

 arch/ia64/Kconfig                |    3 +
 arch/ia64/include/asm/kvm_host.h |    2 +-
 arch/ia64/kvm/Makefile           |    4 +-
 arch/ia64/kvm/kvm-ia64.c         |    3 +-
 arch/x86/Kconfig                 |    3 +
 arch/x86/include/asm/kvm_host.h  |    2 +-
 arch/x86/kvm/Makefile            |    4 +-
 arch/x86/kvm/x86.c               |    3 +-
 drivers/base/Makefile            |    1 +
 drivers/base/iommu.c             |  100 +++++++++++++++++++++++++++++++
 drivers/pci/intel-iommu.c        |  121 ++++++++++++++++++++++---------------
 include/linux/intel-iommu.h      |   21 -------
 include/linux/iommu.h            |  112 +++++++++++++++++++++++++++++++++++
 include/linux/kvm_host.h         |    6 +-
 virt/kvm/{vtd.c => iommu.c}      |   45 +++++++--------
 virt/kvm/kvm_main.c              |    2 +-
 16 files changed, 326 insertions(+), 106 deletions(-)

Joerg Roedel

author

2008-12-09 14:16 UTC #2 lore raw

PATCH 1/11 KVM: rename vtd.c to iommu.c

Impact: file renamed

The code in the vtd.c file can be reused for other IOMMUs as well. So rename it to make it clear that it handle more than VT-d.

Signed-off-by Joerg Roedel <[email protected]>

---
 arch/ia64/kvm/Makefile      |    2 +-
 arch/x86/kvm/Makefile       |    2 +-
 virt/kvm/{vtd.c => iommu.c} |    0 
 3 files changed, 2 insertions(+), 2 deletions(-)
 rename virt/kvm/{vtd.c => iommu.c} (100%)

arch/ia64/kvm/Makefile +1 -1

index 76464dc..cb69dfc 100644
@@ -52,7 +52,7 @@ common-objs = $(addprefix ../../../virt/kvm/, kvm_main.o ioapic.o \
 		coalesced_mmio.o irq_comm.o)
 
 ifeq ($(CONFIG_DMAR),y)
-common-objs += $(addprefix ../../../virt/kvm/, vtd.o)
+common-objs += $(addprefix ../../../virt/kvm/, iommu.o)
 endif
 
 kvm-objs := $(common-objs) kvm-ia64.o kvm_fw.o

arch/x86/kvm/Makefile +1 -1

index c023435..00f46c2 100644
@@ -8,7 +8,7 @@ ifeq ($(CONFIG_KVM_TRACE),y)
 common-objs += $(addprefix ../../../virt/kvm/, kvm_trace.o)
 endif
 ifeq ($(CONFIG_DMAR),y)
-common-objs += $(addprefix ../../../virt/kvm/, vtd.o)
+common-objs += $(addprefix ../../../virt/kvm/, iommu.o)
 endif
 
 EXTRA_CFLAGS += -Ivirt/kvm -Iarch/x86/kvm

virt/kvm/iommu.c renamed from virt/kvm/vtd.c +0 -0

similarity index 100%
rename from virt/kvm/vtd.c
rename to virt/kvm/iommu.c

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #3 lore raw

PATCH 2/11 introcude linux/iommu.h for an iommu api

This patch introduces the API to abstract the exported VT-d functions for KVM into a generic API. This way the AMD IOMMU implementation can plug into this API later.

Signed-off-by Joerg Roedel <[email protected]>

---
 include/linux/iommu.h |  112 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 112 insertions(+), 0 deletions(-)
 create mode 100644 include/linux/iommu.h

include/linux/iommu.h new file +112 -0

new file mode 100644
index 0000000..8a7bfb1
@@ -0,0 +1,112 @@
+/*
+ * Copyright (C) 2007-2008 Advanced Micro Devices, Inc.
+ * Author: Joerg Roedel <[email protected]>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published
+ * by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307 USA
+ */
+
+#ifndef __LINUX_IOMMU_H
+#define __LINUX_IOMMU_H
+
+#define IOMMU_READ	(1)
+#define IOMMU_WRITE	(2)
+
+struct device;
+
+struct iommu_domain {
+	void *priv;
+};
+
+struct iommu_ops {
+	int (*domain_init)(struct iommu_domain *domain);
+	void (*domain_destroy)(struct iommu_domain *domain);
+	int (*attach_dev)(struct iommu_domain *domain, struct device *dev);
+	void (*detach_dev)(struct iommu_domain *domain, struct device *dev);
+	int (*map)(struct iommu_domain *domain, unsigned long iova,
+		   phys_addr_t paddr, size_t size, int prot);
+	void (*unmap)(struct iommu_domain *domain, unsigned long iova,
+		      size_t size);
+	phys_addr_t (*iova_to_phys)(struct iommu_domain *domain,
+				    unsigned long iova);
+};
+
+#ifdef CONFIG_IOMMU_API
+
+extern void register_iommu(struct iommu_ops *ops);
+extern bool iommu_found(void);
+extern struct iommu_domain *iommu_domain_alloc(void);
+extern void iommu_domain_free(struct iommu_domain *domain);
+extern int iommu_attach_device(struct iommu_domain *domain,
+			       struct device *dev);
+extern void iommu_detach_device(struct iommu_domain *domain,
+				struct device *dev);
+extern int iommu_map_range(struct iommu_domain *domain, unsigned long iova,
+			   phys_addr_t paddr, size_t size, int prot);
+extern void iommu_unmap_range(struct iommu_domain *domain, unsigned long iova,
+			      size_t size);
+extern phys_addr_t iommu_iova_to_phys(struct iommu_domain *domain,
+				      unsigned long iova);
+
+#else /* CONFIG_IOMMU_API */
+
+static inline void register_iommu(struct iommu_ops *ops)
+{
+}
+
+static inline bool iommu_found(void)
+{
+	return false;
+}
+
+static inline struct iommu_domain *iommu_domain_alloc(void)
+{
+	return NULL;
+}
+
+static inline void iommu_domain_free(struct iommu_domain *domain)
+{
+}
+
+static inline int iommu_attach_device(struct iommu_domain *domain,
+				      struct device *dev)
+{
+	return -ENODEV;
+}
+
+static inline void iommu_detach_device(struct iommu_domain *domain,
+				       struct device *dev)
+{
+}
+
+static inline int iommu_map_range(struct iommu_domain *domain,
+				  unsigned long iova, phys_addr_t paddr,
+				  size_t size, int prot)
+{
+	return -ENODEV;
+}
+
+static inline void iommu_unmap_range(struct iommu_domain *domain,
+				     unsigned long iova, size_t size)
+{
+}
+
+static inline phys_addr_t iommu_iova_to_phys(struct iommu_domain *domain,
+					     unsigned long iova)
+{
+	return 0;
+}
+
+#endif /* CONFIG_IOMMU_API */
+
+#endif /* __LINUX_IOMMU_H */

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #4 lore raw

PATCH 3/11 add frontend implementation for the IOMMU API

This API can be used by KVM for accessing different types of IOMMUs to do device passthrough to guests. Beside that this API can also be used by device drivers to map non-linear host memory into dma-linear addresses to prevent sgather-gather DMA. UIO may be another user for this API.

Signed-off-by Joerg Roedel <[email protected]>

Acked-by Greg Kroah-Hartman <[email protected]>

---
 drivers/base/iommu.c |  100 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 100 insertions(+), 0 deletions(-)
 create mode 100644 drivers/base/iommu.c

drivers/base/iommu.c new file +100 -0

new file mode 100644
index 0000000..5e039d4
@@ -0,0 +1,100 @@
+/*
+ * Copyright (C) 2007-2008 Advanced Micro Devices, Inc.
+ * Author: Joerg Roedel <[email protected]>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published
+ * by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307 USA
+ */
+
+#include <linux/bug.h>
+#include <linux/types.h>
+#include <linux/errno.h>
+#include <linux/iommu.h>
+
+static struct iommu_ops *iommu_ops;
+
+void register_iommu(struct iommu_ops *ops)
+{
+	if (iommu_ops)
+		BUG();
+
+	iommu_ops = ops;
+}
+
+bool iommu_found()
+{
+	return iommu_ops != NULL;
+}
+EXPORT_SYMBOL_GPL(iommu_found);
+
+struct iommu_domain *iommu_domain_alloc(void)
+{
+	struct iommu_domain *domain;
+	int ret;
+
+	domain = kmalloc(sizeof(*domain), GFP_KERNEL);
+	if (!domain)
+		return NULL;
+
+	ret = iommu_ops->domain_init(domain);
+	if (ret)
+		goto out_free;
+
+	return domain;
+
+out_free:
+	kfree(domain);
+
+	return NULL;
+}
+EXPORT_SYMBOL_GPL(iommu_domain_alloc);
+
+void iommu_domain_free(struct iommu_domain *domain)
+{
+	iommu_ops->domain_destroy(domain);
+	kfree(domain);
+}
+EXPORT_SYMBOL_GPL(iommu_domain_free);
+
+int iommu_attach_device(struct iommu_domain *domain, struct device *dev)
+{
+	return iommu_ops->attach_dev(domain, dev);
+}
+EXPORT_SYMBOL_GPL(iommu_attach_device);
+
+void iommu_detach_device(struct iommu_domain *domain, struct device *dev)
+{
+	iommu_ops->detach_dev(domain, dev);
+}
+EXPORT_SYMBOL_GPL(iommu_detach_device);
+
+int iommu_map_range(struct iommu_domain *domain, unsigned long iova,
+		    phys_addr_t paddr, size_t size, int prot)
+{
+	return iommu_ops->map(domain, iova, paddr, size, prot);
+}
+EXPORT_SYMBOL_GPL(iommu_map_range);
+
+void iommu_unmap_range(struct iommu_domain *domain, unsigned long iova,
+		      size_t size)
+{
+	iommu_ops->unmap(domain, iova, size);
+}
+EXPORT_SYMBOL_GPL(iommu_unmap_range);
+
+phys_addr_t iommu_iova_to_phys(struct iommu_domain *domain,
+			       unsigned long iova)
+{
+	return iommu_ops->iova_to_phys(domain, iova);
+}
+EXPORT_SYMBOL_GPL(iommu_iova_to_phys);

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #5 lore raw

PATCH 4/11 select IOMMU_API when DMAR and/or AMD_IOMMU is selected

These two IOMMUs can implement the current version of this API. So select the API if one or both of these IOMMU drivers is selected.

Signed-off-by Joerg Roedel <[email protected]>

---
 arch/ia64/Kconfig     |    3 +++
 arch/x86/Kconfig      |    3 +++
 drivers/base/Makefile |    1 +
 3 files changed, 7 insertions(+), 0 deletions(-)

arch/ia64/Kconfig +3 -0

index 6bd91ed..6a7b0c9 100644
@@ -687,3 +687,6 @@ config IRQ_PER_CPU
 
 config IOMMU_HELPER
 	def_bool (IA64_HP_ZX1 || IA64_HP_ZX1_SWIOTLB || IA64_GENERIC || SWIOTLB)
+
+config IOMMU_API
+	def_bool (DMAR)

arch/x86/Kconfig +3 -0

index ac22bb7..b9f7187 100644
@@ -580,6 +580,9 @@ config SWIOTLB
 config IOMMU_HELPER
 	def_bool (CALGARY_IOMMU || GART_IOMMU || SWIOTLB || AMD_IOMMU)
 
+config IOMMU_API
+	def_bool (AMD_IOMMU || DMAR)
+
 config MAXSMP
 	bool "Configure Maximum number of SMP Processors and NUMA Nodes"
 	depends on X86_64 && SMP && BROKEN

drivers/base/Makefile +1 -0

index c666373..b5b8ba5 100644
@@ -11,6 +11,7 @@ obj-$(CONFIG_FW_LOADER)	+= firmware_class.o
 obj-$(CONFIG_NUMA)	+= node.o
 obj-$(CONFIG_MEMORY_HOTPLUG_SPARSE) += memory.o
 obj-$(CONFIG_SMP)	+= topology.o
+obj-$(CONFIG_IOMMU_API) += iommu.o
 ifeq ($(CONFIG_SYSFS),y)
 obj-$(CONFIG_MODULES)	+= module.o
 endif

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #6 lore raw

PATCH 5/11 KVM: change KVM to use IOMMU API

Signed-off-by Joerg Roedel <[email protected]>

---
 arch/ia64/include/asm/kvm_host.h |    2 +-
 arch/ia64/kvm/Makefile           |    2 +-
 arch/ia64/kvm/kvm-ia64.c         |    3 +-
 arch/x86/include/asm/kvm_host.h  |    2 +-
 arch/x86/kvm/Makefile            |    2 +-
 arch/x86/kvm/x86.c               |    3 +-
 include/linux/kvm_host.h         |    6 ++--
 virt/kvm/iommu.c                 |   45 +++++++++++++++++--------------------
 virt/kvm/kvm_main.c              |    2 +-
 9 files changed, 33 insertions(+), 34 deletions(-)

arch/ia64/include/asm/kvm_host.h +1 -1

index 0560f3f..3486636 100644
@@ -467,7 +467,7 @@ struct kvm_arch {
 	struct kvm_sal_data rdv_sal_data;
 
 	struct list_head assigned_dev_head;
-	struct dmar_domain *intel_iommu_domain;
+	struct iommu_domain *iommu_domain;
 	struct hlist_head irq_ack_notifier_list;
 
 	unsigned long irq_sources_bitmap;

arch/ia64/kvm/Makefile +1 -1

index cb69dfc..0bb99b7 100644
@@ -51,7 +51,7 @@ EXTRA_AFLAGS += -Ivirt/kvm -Iarch/ia64/kvm/
 common-objs = $(addprefix ../../../virt/kvm/, kvm_main.o ioapic.o \
 		coalesced_mmio.o irq_comm.o)
 
-ifeq ($(CONFIG_DMAR),y)
+ifeq ($(CONFIG_IOMMU_API),y)
 common-objs += $(addprefix ../../../virt/kvm/, iommu.o)
 endif

arch/ia64/kvm/kvm-ia64.c +2 -1

index b4d24e2..483a15b 100644
@@ -31,6 +31,7 @@
 #include <linux/bitops.h>
 #include <linux/hrtimer.h>
 #include <linux/uaccess.h>
+#include <linux/iommu.h>
 #include <linux/intel-iommu.h>
 
 #include <asm/pgtable.h>

@@ -189,7 +190,7 @@ int kvm_dev_ioctl_check_extension(long ext)
 		r = KVM_COALESCED_MMIO_PAGE_OFFSET;
 		break;
 	case KVM_CAP_IOMMU:
-		r = intel_iommu_found();
+		r = iommu_found();
 		break;
 	default:
 		r = 0;

arch/x86/include/asm/kvm_host.h +1 -1

index 97215a4..730843d 100644
@@ -360,7 +360,7 @@ struct kvm_arch{
 	struct list_head active_mmu_pages;
 	struct list_head assigned_dev_head;
 	struct list_head oos_global_pages;
-	struct dmar_domain *intel_iommu_domain;
+	struct iommu_domain *iommu_domain;
 	struct kvm_pic *vpic;
 	struct kvm_ioapic *vioapic;
 	struct kvm_pit *vpit;

arch/x86/kvm/Makefile +1 -1

index 00f46c2..d3ec292 100644
@@ -7,7 +7,7 @@ common-objs = $(addprefix ../../../virt/kvm/, kvm_main.o ioapic.o \
 ifeq ($(CONFIG_KVM_TRACE),y)
 common-objs += $(addprefix ../../../virt/kvm/, kvm_trace.o)
 endif
-ifeq ($(CONFIG_DMAR),y)
+ifeq ($(CONFIG_IOMMU_API),y)
 common-objs += $(addprefix ../../../virt/kvm/, iommu.o)
 endif

arch/x86/kvm/x86.c +2 -1

index ba10287..074a1ee 100644
@@ -34,6 +34,7 @@
 #include <linux/module.h>
 #include <linux/mman.h>
 #include <linux/highmem.h>
+#include <linux/iommu.h>
 #include <linux/intel-iommu.h>
 
 #include <asm/uaccess.h>

@@ -990,7 +991,7 @@ int kvm_dev_ioctl_check_extension(long ext)
 		r = !tdp_enabled;
 		break;
 	case KVM_CAP_IOMMU:
-		r = intel_iommu_found();
+		r = iommu_found();
 		break;
 	default:
 		r = 0;

include/linux/kvm_host.h +3 -3

index 673fabd..c428f2a 100644
@@ -327,7 +327,7 @@ void kvm_unregister_irq_ack_notifier(struct kvm_irq_ack_notifier *kian);
 int kvm_request_irq_source_id(struct kvm *kvm);
 void kvm_free_irq_source_id(struct kvm *kvm, int irq_source_id);
 
-#ifdef CONFIG_DMAR
+#ifdef CONFIG_IOMMU_API
 int kvm_iommu_map_pages(struct kvm *kvm, gfn_t base_gfn,
 			unsigned long npages);
 int kvm_iommu_map_guest(struct kvm *kvm);

@@ -336,7 +336,7 @@ int kvm_assign_device(struct kvm *kvm,
 		      struct kvm_assigned_dev_kernel *assigned_dev);
 int kvm_deassign_device(struct kvm *kvm,
 			struct kvm_assigned_dev_kernel *assigned_dev);
-#else /* CONFIG_DMAR */
+#else /* CONFIG_IOMMU_API */
 static inline int kvm_iommu_map_pages(struct kvm *kvm,
 				      gfn_t base_gfn,
 				      unsigned long npages)

@@ -365,7 +365,7 @@ static inline int kvm_deassign_device(struct kvm *kvm,
 {
 	return 0;
 }
-#endif /* CONFIG_DMAR */
+#endif /* CONFIG_IOMMU_API */
 
 static inline void kvm_guest_enter(void)
 {

virt/kvm/iommu.c +21 ~~-24~~

127 lines not on this page · raw message

virt/kvm/kvm_main.c +1 -1

index 4973ded..93a54fc 100644
@@ -494,7 +494,7 @@ static int kvm_vm_ioctl_assign_device(struct kvm *kvm,
 	list_add(&match->list, &kvm->arch.assigned_dev_head);
 
 	if (assigned_dev->flags & KVM_DEV_ASSIGN_ENABLE_IOMMU) {
-		if (!kvm->arch.intel_iommu_domain) {
+		if (!kvm->arch.iommu_domain) {
 			r = kvm_iommu_map_guest(kvm);
 			if (r)
 				goto out_list_del;

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #7 lore raw

PATCH 6/11 VT-d: adapt domain init and destroy functions for IOMMU API

Signed-off-by Joerg Roedel <[email protected]>

---
 drivers/pci/intel-iommu.c   |   33 ++++++++++++++++++---------------
 include/linux/intel-iommu.h |    2 --
 2 files changed, 18 insertions(+), 17 deletions(-)

drivers/pci/intel-iommu.c +18 ~~-15~~

index 772fb22..5c95a5a 100644
@@ -34,6 +34,7 @@
 #include <linux/mempool.h>
 #include <linux/timer.h>
 #include <linux/iova.h>
+#include <linux/iommu.h>
 #include <linux/intel-iommu.h>
 #include <asm/cacheflush.h>
 #include <asm/iommu.h>

@@ -2962,32 +2963,34 @@ static void vm_domain_exit(struct dmar_domain *domain)
 	free_domain_mem(domain);
 }
 
-struct dmar_domain *intel_iommu_alloc_domain(void)
+static int intel_iommu_domain_init(struct iommu_domain *domain)
 {
-	struct dmar_domain *domain;
+	struct dmar_domain *dmar_domain;
 
-	domain = iommu_alloc_vm_domain();
-	if (!domain) {
+	dmar_domain = iommu_alloc_vm_domain();
+	if (!dmar_domain) {
 		printk(KERN_ERR
-			"intel_iommu_domain_alloc: domain == NULL\n");
-		return NULL;
+			"intel_iommu_domain_init: dmar_domain == NULL\n");
+		return -ENOMEM;
 	}
-	if (vm_domain_init(domain, DEFAULT_DOMAIN_ADDRESS_WIDTH)) {
+	if (vm_domain_init(dmar_domain, DEFAULT_DOMAIN_ADDRESS_WIDTH)) {
 		printk(KERN_ERR
-			"intel_iommu_domain_alloc: domain_init() failed\n");
-		vm_domain_exit(domain);
-		return NULL;
+			"intel_iommu_domain_init() failed\n");
+		vm_domain_exit(dmar_domain);
+		return -ENOMEM;
 	}
+	domain->priv = dmar_domain;
 
-	return domain;
+	return 0;
 }
-EXPORT_SYMBOL_GPL(intel_iommu_alloc_domain);
 
-void intel_iommu_free_domain(struct dmar_domain *domain)
+static void intel_iommu_domain_destroy(struct iommu_domain *domain)
 {
-	vm_domain_exit(domain);
+	struct dmar_domain *dmar_domain = domain->priv;
+
+	domain->priv = NULL;
+	vm_domain_exit(dmar_domain);
 }
-EXPORT_SYMBOL_GPL(intel_iommu_free_domain);
 
 int intel_iommu_attach_device(struct dmar_domain *domain,
 			      struct pci_dev *pdev)

include/linux/intel-iommu.h +0 -2

index 07973c4..0a7ba0c 100644
@@ -330,8 +330,6 @@ extern int qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr,
 
 extern void qi_submit_sync(struct qi_desc *desc, struct intel_iommu *iommu);
 
-struct dmar_domain *intel_iommu_alloc_domain(void);
-void intel_iommu_free_domain(struct dmar_domain *domain);
 int intel_iommu_attach_device(struct dmar_domain *domain,
 			      struct pci_dev *pdev);
 void intel_iommu_detach_device(struct dmar_domain *domain,

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #8 lore raw

PATCH 7/11 VT-d: adapt device attach and detach functions for IOMMU API

Signed-off-by Joerg Roedel <[email protected]>

---
 drivers/pci/intel-iommu.c   |   27 +++++++++++++++------------
 include/linux/intel-iommu.h |    4 ----
 2 files changed, 15 insertions(+), 16 deletions(-)

drivers/pci/intel-iommu.c +15 ~~-12~~

index 5c95a5a..db9a26c 100644
@@ -2992,9 +2992,11 @@ static void intel_iommu_domain_destroy(struct iommu_domain *domain)
 	vm_domain_exit(dmar_domain);
 }
 
-int intel_iommu_attach_device(struct dmar_domain *domain,
-			      struct pci_dev *pdev)
+static int intel_iommu_attach_device(struct iommu_domain *domain,
+				     struct device *dev)
 {
+	struct dmar_domain *dmar_domain = domain->priv;
+	struct pci_dev *pdev = to_pci_dev(dev);
 	struct intel_iommu *iommu;
 	int addr_width;
 	u64 end;

@@ -3006,7 +3008,7 @@ int intel_iommu_attach_device(struct dmar_domain *domain,
 
 		old_domain = find_domain(pdev);
 		if (old_domain) {
-			if (domain->flags & DOMAIN_FLAG_VIRTUAL_MACHINE)
+			if (dmar_domain->flags & DOMAIN_FLAG_VIRTUAL_MACHINE)
 				vm_domain_remove_one_dev_info(old_domain, pdev);
 			else
 				domain_remove_dev_info(old_domain);

@@ -3021,28 +3023,29 @@ int intel_iommu_attach_device(struct dmar_domain *domain,
 	addr_width = agaw_to_width(iommu->agaw);
 	end = DOMAIN_MAX_ADDR(addr_width);
 	end = end & VTD_PAGE_MASK;
-	if (end < domain->max_addr) {
+	if (end < dmar_domain->max_addr) {
 		printk(KERN_ERR "%s: iommu agaw (%d) is not "
 		       "sufficient for the mapped address (%llx)\n",
-		       __func__, iommu->agaw, domain->max_addr);
+		       __func__, iommu->agaw, dmar_domain->max_addr);
 		return -EFAULT;
 	}
 
-	ret = domain_context_mapping(domain, pdev);
+	ret = domain_context_mapping(dmar_domain, pdev);
 	if (ret)
 		return ret;
 
-	ret = vm_domain_add_dev_info(domain, pdev);
+	ret = vm_domain_add_dev_info(dmar_domain, pdev);
 	return ret;
 }
-EXPORT_SYMBOL_GPL(intel_iommu_attach_device);
 
-void intel_iommu_detach_device(struct dmar_domain *domain,
-			       struct pci_dev *pdev)
+static void intel_iommu_detach_device(struct iommu_domain *domain,
+				      struct device *dev)
 {
-	vm_domain_remove_one_dev_info(domain, pdev);
+	struct dmar_domain *dmar_domain = domain->priv;
+	struct pci_dev *pdev = to_pci_dev(dev);
+
+	vm_domain_remove_one_dev_info(dmar_domain, pdev);
 }
-EXPORT_SYMBOL_GPL(intel_iommu_detach_device);
 
 int intel_iommu_map_address(struct dmar_domain *domain, dma_addr_t iova,
 			    u64 hpa, size_t size, int prot)

include/linux/intel-iommu.h +0 -4

index 0a7ba0c..9909c5a 100644
@@ -330,10 +330,6 @@ extern int qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr,
 
 extern void qi_submit_sync(struct qi_desc *desc, struct intel_iommu *iommu);
 
-int intel_iommu_attach_device(struct dmar_domain *domain,
-			      struct pci_dev *pdev);
-void intel_iommu_detach_device(struct dmar_domain *domain,
-			       struct pci_dev *pdev);
 int intel_iommu_map_address(struct dmar_domain *domain, dma_addr_t iova,
 			    u64 hpa, size_t size, int prot);
 void intel_iommu_unmap_address(struct dmar_domain *domain,

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #9 lore raw

PATCH 8/11 VT-d: adapt domain map and unmap functions for IOMMU API

Signed-off-by Joerg Roedel <[email protected]>

---
 drivers/pci/intel-iommu.c   |   33 ++++++++++++++++++++-------------
 include/linux/intel-iommu.h |    4 ----
 2 files changed, 20 insertions(+), 17 deletions(-)

drivers/pci/intel-iommu.c +20 ~~-13~~

index db9a26c..8af6c96 100644
@@ -3047,20 +3047,28 @@ static void intel_iommu_detach_device(struct iommu_domain *domain,
 	vm_domain_remove_one_dev_info(dmar_domain, pdev);
 }
 
-int intel_iommu_map_address(struct dmar_domain *domain, dma_addr_t iova,
-			    u64 hpa, size_t size, int prot)
+static int intel_iommu_map_range(struct iommu_domain *domain,
+				 unsigned long iova, phys_addr_t hpa,
+				 size_t size, int iommu_prot)
 {
+	struct dmar_domain *dmar_domain = domain->priv;
 	u64 max_addr;
 	int addr_width;
+	int prot = 0;
 	int ret;
 
+	if (iommu_prot & IOMMU_READ)
+		prot |= DMA_PTE_READ;
+	if (iommu_prot & IOMMU_WRITE)
+		prot |= DMA_PTE_WRITE;
+
 	max_addr = (iova & VTD_PAGE_MASK) + VTD_PAGE_ALIGN(size);
-	if (domain->max_addr < max_addr) {
+	if (dmar_domain->max_addr < max_addr) {
 		int min_agaw;
 		u64 end;
 
 		/* check if minimum agaw is sufficient for mapped address */
-		min_agaw = vm_domain_min_agaw(domain);
+		min_agaw = vm_domain_min_agaw(dmar_domain);
 		addr_width = agaw_to_width(min_agaw);
 		end = DOMAIN_MAX_ADDR(addr_width);
 		end = end & VTD_PAGE_MASK;

@@ -3070,28 +3078,27 @@ int intel_iommu_map_address(struct dmar_domain *domain, dma_addr_t iova,
 			       __func__, min_agaw, max_addr);
 			return -EFAULT;
 		}
-		domain->max_addr = max_addr;
+		dmar_domain->max_addr = max_addr;
 	}
 
-	ret = domain_page_mapping(domain, iova, hpa, size, prot);
+	ret = domain_page_mapping(dmar_domain, iova, hpa, size, prot);
 	return ret;
 }
-EXPORT_SYMBOL_GPL(intel_iommu_map_address);
 
-void intel_iommu_unmap_address(struct dmar_domain *domain,
-			       dma_addr_t iova, size_t size)
+static void intel_iommu_unmap_range(struct iommu_domain *domain,
+				    unsigned long iova, size_t size)
 {
+	struct dmar_domain *dmar_domain = domain->priv;
 	dma_addr_t base;
 
 	/* The address might not be aligned */
 	base = iova & VTD_PAGE_MASK;
 	size = VTD_PAGE_ALIGN(size);
-	dma_pte_clear_range(domain, base, base + size);
+	dma_pte_clear_range(dmar_domain, base, base + size);
 
-	if (domain->max_addr == base + size)
-		domain->max_addr = base;
+	if (dmar_domain->max_addr == base + size)
+		dmar_domain->max_addr = base;
 }
-EXPORT_SYMBOL_GPL(intel_iommu_unmap_address);
 
 int intel_iommu_found(void)
 {

include/linux/intel-iommu.h +0 -4

index 9909c5a..6bc26e0 100644
@@ -330,10 +330,6 @@ extern int qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr,
 
 extern void qi_submit_sync(struct qi_desc *desc, struct intel_iommu *iommu);
 
-int intel_iommu_map_address(struct dmar_domain *domain, dma_addr_t iova,
-			    u64 hpa, size_t size, int prot);
-void intel_iommu_unmap_address(struct dmar_domain *domain,
-			       dma_addr_t iova, size_t size);
 u64 intel_iommu_iova_to_phys(struct dmar_domain *domain, u64 iova);
 
 #ifdef CONFIG_DMAR

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #10 lore raw

PATCH 9/11 VT-d: adapt domain iova_to_phys function for IOMMU API

Signed-off-by Joerg Roedel <[email protected]>

---
 drivers/pci/intel-iommu.c   |    7 ++++---
 include/linux/intel-iommu.h |    2 --
 2 files changed, 4 insertions(+), 5 deletions(-)

drivers/pci/intel-iommu.c +4 -3

index 8af6c96..7128105 100644
@@ -3106,15 +3106,16 @@ int intel_iommu_found(void)
 }
 EXPORT_SYMBOL_GPL(intel_iommu_found);
 
-u64 intel_iommu_iova_to_phys(struct dmar_domain *domain, u64 iova)
+static phys_addr_t intel_iommu_iova_to_phys(struct iommu_domain *domain,
+					    unsigned long iova)
 {
+	struct dmar_domain *dmar_domain = domain->priv;
 	struct dma_pte *pte;
 	u64 phys = 0;
 
-	pte = addr_to_dma_pte(domain, iova);
+	pte = addr_to_dma_pte(dmar_domain, iova);
 	if (pte)
 		phys = dma_pte_addr(pte);
 
 	return phys;
 }
-EXPORT_SYMBOL_GPL(intel_iommu_iova_to_phys);

include/linux/intel-iommu.h +0 -2

index 6bc26e0..26ccc02 100644
@@ -330,8 +330,6 @@ extern int qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr,
 
 extern void qi_submit_sync(struct qi_desc *desc, struct intel_iommu *iommu);
 
-u64 intel_iommu_iova_to_phys(struct dmar_domain *domain, u64 iova);
-
 #ifdef CONFIG_DMAR
 int intel_iommu_found(void);
 #else /* CONFIG_DMAR */

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #11 lore raw

PATCH 10/11 VT-d: register functions for the IOMMU API

Signed-off-by Joerg Roedel <[email protected]>

---
 drivers/pci/intel-iommu.c |   15 +++++++++++++++
 1 files changed, 15 insertions(+), 0 deletions(-)

drivers/pci/intel-iommu.c +15 -0

index 7128105..81e04ec 100644
@@ -277,6 +277,8 @@ static int intel_iommu_strict;
 static DEFINE_SPINLOCK(device_domain_lock);
 static LIST_HEAD(device_domain_list);
 
+static struct iommu_ops intel_iommu_ops;
+
 static int __init intel_iommu_setup(char *str)
 {
 	if (!str)

@@ -2729,6 +2731,9 @@ int __init intel_iommu_init(void)
 	init_timer(&unmap_timer);
 	force_iommu = 1;
 	dma_ops = &intel_dma_ops;
+
+	register_iommu(&intel_iommu_ops);
+
 	return 0;
 }

@@ -3119,3 +3124,13 @@ static phys_addr_t intel_iommu_iova_to_phys(struct iommu_domain *domain,
 
 	return phys;
 }
+
+static struct iommu_ops intel_iommu_ops = {
+	.domain_init	= intel_iommu_domain_init,
+	.domain_destroy = intel_iommu_domain_destroy,
+	.attach_dev	= intel_iommu_attach_device,
+	.detach_dev	= intel_iommu_detach_device,
+	.map		= intel_iommu_map_range,
+	.unmap		= intel_iommu_unmap_range,
+	.iova_to_phys	= intel_iommu_iova_to_phys,
+};

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #12 lore raw

PATCH 11/11 VT-d: remove now unused intel_iommu_found function

Signed-off-by Joerg Roedel <[email protected]>

---
 drivers/pci/intel-iommu.c   |    6 ------
 include/linux/intel-iommu.h |    9 ---------
 2 files changed, 0 insertions(+), 15 deletions(-)

drivers/pci/intel-iommu.c +0 -6

index 81e04ec..ecb5fd3 100644
@@ -3105,12 +3105,6 @@ static void intel_iommu_unmap_range(struct iommu_domain *domain,
 		dmar_domain->max_addr = base;
 }
 
-int intel_iommu_found(void)
-{
-	return g_num_of_iommus;
-}
-EXPORT_SYMBOL_GPL(intel_iommu_found);
-
 static phys_addr_t intel_iommu_iova_to_phys(struct iommu_domain *domain,
 					    unsigned long iova)
 {

include/linux/intel-iommu.h +0 -9

index 26ccc02..c4f6c10 100644
@@ -330,15 +330,6 @@ extern int qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr,
 
 extern void qi_submit_sync(struct qi_desc *desc, struct intel_iommu *iommu);
 
-#ifdef CONFIG_DMAR
-int intel_iommu_found(void);
-#else /* CONFIG_DMAR */
-static inline int intel_iommu_found(void)
-{
-	return 0;
-}
-#endif /* CONFIG_DMAR */
-
 extern void *intel_alloc_coherent(struct device *, size_t, dma_addr_t *, gfp_t);
 extern void intel_free_coherent(struct device *, size_t, void *, dma_addr_t);
 extern dma_addr_t intel_map_single(struct device *, phys_addr_t, size_t, int);

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #13 lore raw

cover AMD IOMMU support for KVM device assignment

In reply to Joerg Roedel

This patchset implements KVM device assignment support in the AMD IOMMU driver. It uses the generic interface from the iommu-api patchset and was successfully tested using an 10GBit network card passed through to an KVM guest.

diffstat:

 arch/x86/include/asm/amd_iommu_types.h |   15 +-
 arch/x86/kernel/amd_iommu.c            |  448 +++++++++++++++++++++++++++++---
 2 files changed, 416 insertions(+), 47 deletions(-)

Joerg Roedel

author

2008-12-09 14:16 UTC #14 lore raw

PATCH 1/21 AMD IOMMU: rename iommu_map to iommu_map_page

Impact: function rename

The iommu_map function maps only one page. Make this clear in the function name.

Signed-off-by Joerg Roedel <[email protected]>

---
 arch/x86/kernel/amd_iommu.c |   10 +++++-----
 1 files changed, 5 insertions(+), 5 deletions(-)

arch/x86/kernel/amd_iommu.c +5 -5

index 5662e22..4db4542 100644
@@ -336,10 +336,10 @@ static void iommu_flush_tlb(struct amd_iommu *iommu, u16 domid)
  * supporting all features of AMD IOMMU page tables like level skipping
  * and full 64 bit address spaces.
  */
-static int iommu_map(struct protection_domain *dom,
-		     unsigned long bus_addr,
-		     unsigned long phys_addr,
-		     int prot)
+static int iommu_map_page(struct protection_domain *dom,
+			  unsigned long bus_addr,
+			  unsigned long phys_addr,
+			  int prot)
 {
 	u64 __pte, *pte, *page;

@@ -438,7 +438,7 @@ static int dma_ops_unity_map(struct dma_ops_domain *dma_dom,
 
 	for (addr = e->address_start; addr < e->address_end;
 	     addr += PAGE_SIZE) {
-		ret = iommu_map(&dma_dom->domain, addr, addr, e->prot);
+		ret = iommu_map_page(&dma_dom->domain, addr, addr, e->prot);
 		if (ret)
 			return ret;
 		/*

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:17 UTC #15 lore raw

PATCH 2/21 AMD IOMMU: fix iommu_map_page function

Impact: bugfix in iommu_map_page function

Signed-off-by Joerg Roedel <[email protected]>

---
 arch/x86/kernel/amd_iommu.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

arch/x86/kernel/amd_iommu.c +1 -1

index 4db4542..153db6e 100644
@@ -344,7 +344,7 @@ static int iommu_map_page(struct protection_domain *dom,
 	u64 __pte, *pte, *page;
 
 	bus_addr  = PAGE_ALIGN(bus_addr);
-	phys_addr = PAGE_ALIGN(bus_addr);
+	phys_addr = PAGE_ALIGN(phys_addr);
 
 	/* only support 512GB address spaces for now */
 	if (bus_addr > IOMMU_MAP_SIZE_L3 || !(prot & IOMMU_PROT_MASK))

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #16 lore raw

PATCH 3/21 AMD IOMMU: fix loop counter in free_pagetable function

Impact: bugfix

Signed-off-by Joerg Roedel <[email protected]>

---
 arch/x86/kernel/amd_iommu.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

arch/x86/kernel/amd_iommu.c +1 -1

index 153db6e..e96a2f3 100644
@@ -600,7 +600,7 @@ static void dma_ops_free_pagetable(struct dma_ops_domain *dma_dom)
 			continue;
 
 		p2 = IOMMU_PTE_PAGE(p1[i]);
-		for (j = 0; j < 512; ++i) {
+		for (j = 0; j < 512; ++j) {
 			if (!IOMMU_PTE_PRESENT(p2[j]))
 				continue;
 			p3 = IOMMU_PTE_PAGE(p2[j]);

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #17 lore raw

PATCH 4/21 AMD IOMMU: make dma_ops_free_pagetable generic

Impact: change code to free pagetables from protection domains

The dma_ops_free_pagetable function can only free pagetables from dma_ops domains. Change that to free pagetables of pure protection domains.

Signed-off-by Joerg Roedel <[email protected]>

---
 arch/x86/kernel/amd_iommu.c |    8 +++++---
 1 files changed, 5 insertions(+), 3 deletions(-)

arch/x86/kernel/amd_iommu.c +5 -3

index e96a2f3..85a62e9 100644
@@ -585,12 +585,12 @@ static void dma_ops_reserve_addresses(struct dma_ops_domain *dom,
 	iommu_area_reserve(dom->bitmap, start_page, pages);
 }
 
-static void dma_ops_free_pagetable(struct dma_ops_domain *dma_dom)
+static void free_pagetable(struct protection_domain *domain)
 {
 	int i, j;
 	u64 *p1, *p2, *p3;
 
-	p1 = dma_dom->domain.pt_root;
+	p1 = domain->pt_root;
 
 	if (!p1)
 		return;

@@ -611,6 +611,8 @@ static void dma_ops_free_pagetable(struct dma_ops_domain *dma_dom)
 	}
 
 	free_page((unsigned long)p1);
+
+	domain->pt_root = NULL;
 }
 
 /*

@@ -622,7 +624,7 @@ static void dma_ops_domain_free(struct dma_ops_domain *dom)
 	if (!dom)
 		return;
 
-	dma_ops_free_pagetable(dom);
+	free_pagetable(&dom->domain);
 
 	kfree(dom->pte_pages);

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #18 lore raw

PATCH 5/21 AMD IOMMU: add domain id free function

Impact: add code to release a domain id

Signed-off-by Joerg Roedel <[email protected]>

---
 arch/x86/kernel/amd_iommu.c |   10 ++++++++++
 1 files changed, 10 insertions(+), 0 deletions(-)

arch/x86/kernel/amd_iommu.c +10 -0

index 85a62e9..11de0f1 100644
@@ -569,6 +569,16 @@ static u16 domain_id_alloc(void)
 	return id;
 }
 
+static void domain_id_free(int id)
+{
+	unsigned long flags;
+
+	write_lock_irqsave(&amd_iommu_devtable_lock, flags);
+	if (id > 0 && id < MAX_DOMAIN_ID)
+		__clear_bit(id, amd_iommu_pd_alloc_bitmap);
+	write_unlock_irqrestore(&amd_iommu_devtable_lock, flags);
+}
+
 /*
  * Used to reserve address ranges in the aperture (e.g. for exclusion
  * ranges.

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #19 lore raw

PATCH 6/21 AMD IOMMU: refactor completion wait handling into separate functions

Impact: split one function into three

The separate functions are required synchronize commands across all hardware IOMMUs in the system.

Signed-off-by Joerg Roedel <[email protected]>

---
 arch/x86/kernel/amd_iommu.c |   65 +++++++++++++++++++++++++++++--------------
 1 files changed, 44 insertions(+), 21 deletions(-)

arch/x86/kernel/amd_iommu.c +44 ~~-21~~

index 11de0f1..db89001 100644
@@ -195,6 +195,46 @@ static int iommu_queue_command(struct amd_iommu *iommu, struct iommu_cmd *cmd)
 }
 
 /*
+ * This function waits until an IOMMU has completed a completion
+ * wait command
+ */
+static void __iommu_wait_for_completion(struct amd_iommu *iommu)
+{
+	int ready = 0;
+	unsigned status = 0;
+	unsigned long i = 0;
+
+	while (!ready && (i < EXIT_LOOP_COUNT)) {
+		++i;
+		/* wait for the bit to become one */
+		status = readl(iommu->mmio_base + MMIO_STATUS_OFFSET);
+		ready = status & MMIO_STATUS_COM_WAIT_INT_MASK;
+	}
+
+	/* set bit back to zero */
+	status &= ~MMIO_STATUS_COM_WAIT_INT_MASK;
+	writel(status, iommu->mmio_base + MMIO_STATUS_OFFSET);
+
+	if (unlikely((i == EXIT_LOOP_COUNT) && printk_ratelimit()))
+		printk(KERN_WARNING "AMD IOMMU: Completion wait loop failed\n");
+}
+
+/*
+ * This function queues a completion wait command into the command
+ * buffer of an IOMMU
+ */
+static int __iommu_completion_wait(struct amd_iommu *iommu)
+{
+	struct iommu_cmd cmd;
+
+	 memset(&cmd, 0, sizeof(cmd));
+	 cmd.data[0] = CMD_COMPL_WAIT_INT_MASK;
+	 CMD_SET_TYPE(&cmd, CMD_COMPL_WAIT);
+
+	 return __iommu_queue_command(iommu, &cmd);
+}
+
+/*
  * This function is called whenever we need to ensure that the IOMMU has
  * completed execution of all commands we sent. It sends a
  * COMPLETION_WAIT command and waits for it to finish. The IOMMU informs

@@ -203,14 +243,8 @@ static int iommu_queue_command(struct amd_iommu *iommu, struct iommu_cmd *cmd)
  */
 static int iommu_completion_wait(struct amd_iommu *iommu)
 {
-	int ret = 0, ready = 0;
-	unsigned status = 0;
-	struct iommu_cmd cmd;
-	unsigned long flags, i = 0;
-
-	memset(&cmd, 0, sizeof(cmd));
-	cmd.data[0] = CMD_COMPL_WAIT_INT_MASK;
-	CMD_SET_TYPE(&cmd, CMD_COMPL_WAIT);
+	int ret = 0;
+	unsigned long flags;
 
 	spin_lock_irqsave(&iommu->lock, flags);

@@ -219,24 +253,13 @@ static int iommu_completion_wait(struct amd_iommu *iommu)
 
 	iommu->need_sync = 0;
 
-	ret = __iommu_queue_command(iommu, &cmd);
+	ret = __iommu_completion_wait(iommu);
 
 	if (ret)
 		goto out;
 
-	while (!ready && (i < EXIT_LOOP_COUNT)) {
-		++i;
-		/* wait for the bit to become one */
-		status = readl(iommu->mmio_base + MMIO_STATUS_OFFSET);
-		ready = status & MMIO_STATUS_COM_WAIT_INT_MASK;
-	}
-
-	/* set bit back to zero */
-	status &= ~MMIO_STATUS_COM_WAIT_INT_MASK;
-	writel(status, iommu->mmio_base + MMIO_STATUS_OFFSET);
+	__iommu_wait_for_completion(iommu);
 
-	if (unlikely((i == EXIT_LOOP_COUNT) && printk_ratelimit()))
-		printk(KERN_WARNING "AMD IOMMU: Completion wait loop failed\n");
 out:
 	spin_unlock_irqrestore(&iommu->lock, flags);

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #20 lore raw

PATCH 7/21 AMD IOMMU: move invalidation command building to a separate function

Impact: refactoring of iommu_queue_inv_iommu_pages

Signed-off-by Joerg Roedel <[email protected]>

---
 arch/x86/kernel/amd_iommu.c |   26 ++++++++++++++++----------
 1 files changed, 16 insertions(+), 10 deletions(-)

arch/x86/kernel/amd_iommu.c +16 ~~-10~~

index db89001..80758cc 100644
@@ -285,6 +285,21 @@ static int iommu_queue_inv_dev_entry(struct amd_iommu *iommu, u16 devid)
 	return ret;
 }
 
+static void __iommu_build_inv_iommu_pages(struct iommu_cmd *cmd, u64 address,
+					  u16 domid, int pde, int s)
+{
+	memset(cmd, 0, sizeof(*cmd));
+	address &= PAGE_MASK;
+	CMD_SET_TYPE(cmd, CMD_INV_IOMMU_PAGES);
+	cmd->data[1] |= domid;
+	cmd->data[2] = lower_32_bits(address);
+	cmd->data[3] = upper_32_bits(address);
+	if (s) /* size bit - we flush more than one 4kb page */
+		cmd->data[2] |= CMD_INV_IOMMU_PAGES_SIZE_MASK;
+	if (pde) /* PDE bit - we wan't flush everything not only the PTEs */
+		cmd->data[2] |= CMD_INV_IOMMU_PAGES_PDE_MASK;
+}
+
 /*
  * Generic command send function for invalidaing TLB entries
  */

@@ -294,16 +309,7 @@ static int iommu_queue_inv_iommu_pages(struct amd_iommu *iommu,
 	struct iommu_cmd cmd;
 	int ret;
 
-	memset(&cmd, 0, sizeof(cmd));
-	address &= PAGE_MASK;
-	CMD_SET_TYPE(&cmd, CMD_INV_IOMMU_PAGES);
-	cmd.data[1] |= domid;
-	cmd.data[2] = lower_32_bits(address);
-	cmd.data[3] = upper_32_bits(address);
-	if (s) /* size bit - we flush more than one 4kb page */
-		cmd.data[2] |= CMD_INV_IOMMU_PAGES_SIZE_MASK;
-	if (pde) /* PDE bit - we wan't flush everything not only the PTEs */
-		cmd.data[2] |= CMD_INV_IOMMU_PAGES_PDE_MASK;
+	__iommu_build_inv_iommu_pages(&cmd, address, domid, pde, s);
 
 	ret = iommu_queue_command(iommu, &cmd);

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #21 lore raw

PATCH 8/21 AMD IOMMU: add iommu_flush_domain function

Impact: add a function to flush a domain id on every IOMMU

Signed-off-by Joerg Roedel <[email protected]>

---
 arch/x86/kernel/amd_iommu.c |   22 ++++++++++++++++++++++
 1 files changed, 22 insertions(+), 0 deletions(-)

arch/x86/kernel/amd_iommu.c +22 -0

index 80758cc..7355da0 100644
@@ -351,6 +351,28 @@ static void iommu_flush_tlb(struct amd_iommu *iommu, u16 domid)
 	iommu_queue_inv_iommu_pages(iommu, address, domid, 0, 1);
 }
 
+/*
+ * This function is used to flush the IO/TLB for a given protection domain
+ * on every IOMMU in the system
+ */
+static void iommu_flush_domain(u16 domid)
+{
+	unsigned long flags;
+	struct amd_iommu *iommu;
+	struct iommu_cmd cmd;
+
+	__iommu_build_inv_iommu_pages(&cmd, CMD_INV_IOMMU_ALL_PAGES_ADDRESS,
+				      domid, 1, 1);
+
+	list_for_each_entry(iommu, &amd_iommu_list, list) {
+		spin_lock_irqsave(&iommu->lock, flags);
+		__iommu_queue_command(iommu, &cmd);
+		__iommu_completion_wait(iommu);
+		__iommu_wait_for_completion(iommu);
+		spin_unlock_irqrestore(&iommu->lock, flags);
+	}
+}
+
 /****************************************************************************
  *
  * The functions below are used the create the page table mappings for

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #22 lore raw

PATCH 9/21 AMD IOMMU: add protection domain flags

Imapct: add a new struct member to 'struct protection_domain'

When using protection domains for dma_ops and KVM its better to know for which subsystem it was allocated. Add a flags member to struct protection domain for that purpose.

Signed-off-by Joerg Roedel <[email protected]>

---
 arch/x86/include/asm/amd_iommu_types.h |   14 +++++++++-----
 arch/x86/kernel/amd_iommu.c            |    1 +
 2 files changed, 10 insertions(+), 5 deletions(-)

arch/x86/include/asm/amd_iommu_types.h +9 -5

index ac302a2..4862a5b 100644
@@ -190,16 +190,20 @@
 /* FIXME: move this macro to <linux/pci.h> */
 #define PCI_BUS(x) (((x) >> 8) & 0xff)
 
+/* Protection domain flags */
+#define PD_DMA_OPS_MASK		(1UL << 0) /* domain used for dma_ops */
+
 /*
  * This structure contains generic data for  IOMMU protection domains
  * independent of their use.
  */
 struct protection_domain {
-	spinlock_t lock; /* mostly used to lock the page table*/
-	u16 id;		 /* the domain id written to the device table */
-	int mode;	 /* paging mode (0-6 levels) */
-	u64 *pt_root;	 /* page table root pointer */
-	void *priv;	 /* private data */
+	spinlock_t lock;	/* mostly used to lock the page table*/
+	u16 id;			/* the domain id written to the device table */
+	int mode;		/* paging mode (0-6 levels) */
+	u64 *pt_root;		/* page table root pointer */
+	unsigned long flags;	/* flags to find out type of domain */
+	void *priv;		/* private data */
 };
 
 /*

arch/x86/kernel/amd_iommu.c +1 -0

index 7355da0..6d26544 100644
@@ -724,6 +724,7 @@ static struct dma_ops_domain *dma_ops_domain_alloc(struct amd_iommu *iommu,
 		goto free_dma_dom;
 	dma_dom->domain.mode = PAGE_MODE_3_LEVEL;
 	dma_dom->domain.pt_root = (void *)get_zeroed_page(GFP_KERNEL);
+	dma_dom->domain.flags = PD_DMA_OPS_MASK;
 	dma_dom->domain.priv = dma_dom;
 	if (!dma_dom->domain.pt_root)
 		goto free_dma_dom;

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #23 lore raw

PATCH 10/21 AMD IOMMU: add checks for dma_ops domain to dma_ops functions

Impact: detect when a driver uses a device assigned otherwise

Signed-off-by Joerg Roedel <[email protected]>

---
 arch/x86/kernel/amd_iommu.c |   21 +++++++++++++++++++++
 1 files changed, 21 insertions(+), 0 deletions(-)

arch/x86/kernel/amd_iommu.c +21 -0

index 6d26544..2173283 100644
@@ -787,6 +787,15 @@ free_dma_dom:
 }
 
 /*
+ * little helper function to check whether a given protection domain is a
+ * dma_ops domain
+ */
+static bool dma_ops_domain(struct protection_domain *domain)
+{
+	return domain->flags & PD_DMA_OPS_MASK;
+}
+
+/*
  * Find out the protection domain structure for a given PCI device. This
  * will give us the pointer to the page table root for example.
  */

@@ -1091,6 +1100,8 @@ static dma_addr_t map_single(struct device *dev, phys_addr_t paddr,
 		/* device not handled by any AMD IOMMU */
 		return (dma_addr_t)paddr;
 
+	BUG_ON(!dma_ops_domain(domain));
+
 	spin_lock_irqsave(&domain->lock, flags);
 	addr = __map_single(dev, iommu, domain->priv, paddr, size, dir, false,
 			    dma_mask);

@@ -1121,6 +1132,8 @@ static void unmap_single(struct device *dev, dma_addr_t dma_addr,
 		/* device not handled by any AMD IOMMU */
 		return;
 
+	BUG_ON(!dma_ops_domain(domain));
+
 	spin_lock_irqsave(&domain->lock, flags);
 
 	__unmap_single(iommu, domain->priv, dma_addr, size, dir);

@@ -1175,6 +1188,8 @@ static int map_sg(struct device *dev, struct scatterlist *sglist,
 	if (!iommu || !domain)
 		return map_sg_no_iommu(dev, sglist, nelems, dir);
 
+	BUG_ON(!dma_ops_domain(domain));
+
 	spin_lock_irqsave(&domain->lock, flags);
 
 	for_each_sg(sglist, s, nelems, i) {

@@ -1228,6 +1243,8 @@ static void unmap_sg(struct device *dev, struct scatterlist *sglist,
 	    !get_device_resources(dev, &iommu, &domain, &devid))
 		return;
 
+	BUG_ON(!dma_ops_domain(domain));
+
 	spin_lock_irqsave(&domain->lock, flags);
 
 	for_each_sg(sglist, s, nelems, i) {

@@ -1273,6 +1290,8 @@ static void *alloc_coherent(struct device *dev, size_t size,
 		return virt_addr;
 	}
 
+	BUG_ON(!dma_ops_domain(domain));
+
 	if (!dma_mask)
 		dma_mask = *dev->dma_mask;

@@ -1314,6 +1333,8 @@ static void free_coherent(struct device *dev, size_t size,
 	if (!iommu || !domain)
 		goto free_mem;
 
+	BUG_ON(!dma_ops_domain(domain));
+
 	spin_lock_irqsave(&domain->lock, flags);
 
 	__unmap_single(iommu, domain->priv, dma_addr, size, DMA_BIDIRECTIONAL);

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #24 lore raw

PATCH 11/21 AMD IOMMU: add device reference counting for protection domains

Impact: know how many devices are assigned to a domain

Signed-off-by Joerg Roedel <[email protected]>

---
 arch/x86/include/asm/amd_iommu_types.h |    1 +
 arch/x86/kernel/amd_iommu.c            |    3 ++-
 2 files changed, 3 insertions(+), 1 deletions(-)

arch/x86/include/asm/amd_iommu_types.h +1 -0

index 4862a5b..1c769f4 100644
@@ -203,6 +203,7 @@ struct protection_domain {
 	int mode;		/* paging mode (0-6 levels) */
 	u64 *pt_root;		/* page table root pointer */
 	unsigned long flags;	/* flags to find out type of domain */
+	unsigned dev_cnt;	/* devices assigned to this domain */
 	void *priv;		/* private data */
 };

arch/x86/kernel/amd_iommu.c +2 -1

index 2173283..61584d5 100644
@@ -820,9 +820,10 @@ static void set_device_domain(struct amd_iommu *iommu,
 			      u16 devid)
 {
 	unsigned long flags;
-
 	u64 pte_root = virt_to_phys(domain->pt_root);
 
+	domain->dev_cnt += 1;
+
 	pte_root |= (domain->mode & DEV_ENTRY_MODE_MASK)
 		    << DEV_ENTRY_MODE_SHIFT;
 	pte_root |= IOMMU_PTE_IR | IOMMU_PTE_IW | IOMMU_PTE_P | IOMMU_PTE_TV;

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #25 lore raw

PATCH 12/21 AMD IOMMU: add device detach helper functions

Impact: add helper functions to detach a device from a domain

Signed-off-by Joerg Roedel <[email protected]>

---
 arch/x86/kernel/amd_iommu.c |   41 +++++++++++++++++++++++++++++++++++++++++
 1 files changed, 41 insertions(+), 0 deletions(-)

arch/x86/kernel/amd_iommu.c +41 -0

index 61584d5..47158e3 100644
@@ -1469,3 +1469,44 @@ free_domains:
 
 	return ret;
 }
+
+/*****************************************************************************
+ *
+ * The following functions belong to the exported interface of AMD IOMMU
+ *
+ * This interface allows access to lower level functions of the IOMMU
+ * like protection domain handling and assignement of devices to domains
+ * which is not possible with the dma_ops interface.
+ *
+ *****************************************************************************/
+
+static void __detach_device(struct protection_domain *domain, u16 devid)
+{
+
+	/* lock domain */
+	spin_lock(&domain->lock);
+
+	/* remove domain from the lookup table */
+	amd_iommu_pd_table[devid] = NULL;
+
+	/* remove entry from the device table seen by the hardware */
+	amd_iommu_dev_table[devid].data[0] = IOMMU_PTE_P | IOMMU_PTE_TV;
+	amd_iommu_dev_table[devid].data[1] = 0;
+	amd_iommu_dev_table[devid].data[2] = 0;
+
+	/* decrease reference counter */
+	domain->dev_cnt -= 1;
+
+	/* ready */
+	spin_unlock(&domain->lock);
+}
+
+static void detach_device(struct protection_domain *domain, u16 devid)
+{
+	unsigned long flags;
+
+	/* lock device table */
+	write_lock_irqsave(&amd_iommu_devtable_lock, flags);
+	__detach_device(domain, devid);
+	write_unlock_irqrestore(&amd_iommu_devtable_lock, flags);
+}

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #26 lore raw

PATCH 13/21 AMD IOMMU: add domain cleanup helper function

Impact: add a function to remove all devices from a domain

Signed-off-by Joerg Roedel <[email protected]>

---
 arch/x86/kernel/amd_iommu.c |   14 ++++++++++++++
 1 files changed, 14 insertions(+), 0 deletions(-)

arch/x86/kernel/amd_iommu.c +14 -0

index 47158e3..26b929b 100644
@@ -1510,3 +1510,17 @@ static void detach_device(struct protection_domain *domain, u16 devid)
 	__detach_device(domain, devid);
 	write_unlock_irqrestore(&amd_iommu_devtable_lock, flags);
 }
+
+static void cleanup_domain(struct protection_domain *domain)
+{
+	unsigned long flags;
+	u16 devid;
+
+	write_lock_irqsave(&amd_iommu_devtable_lock, flags);
+
+	for (devid = 0; devid <= amd_iommu_last_bdf; ++devid)
+		if (amd_iommu_pd_table[devid] == domain)
+			__detach_device(domain, devid);
+
+	write_unlock_irqrestore(&amd_iommu_devtable_lock, flags);
+}

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #27 lore raw

PATCH 14/21 AMD IOMMU: add domain init function for IOMMU API

Impact: add a generic function for allocation protection domains

Signed-off-by Joerg Roedel <[email protected]>

---
 arch/x86/kernel/amd_iommu.c |   28 ++++++++++++++++++++++++++++
 1 files changed, 28 insertions(+), 0 deletions(-)

arch/x86/kernel/amd_iommu.c +28 -0

index 26b929b..cb7db66 100644
@@ -22,6 +22,7 @@
 #include <linux/bitops.h>
 #include <linux/scatterlist.h>
 #include <linux/iommu-helper.h>
+#include <linux/iommu.h>
 #include <asm/proto.h>
 #include <asm/iommu.h>
 #include <asm/amd_iommu_types.h>

@@ -1524,3 +1525,30 @@ static void cleanup_domain(struct protection_domain *domain)
 
 	write_unlock_irqrestore(&amd_iommu_devtable_lock, flags);
 }
+
+static int amd_iommu_domain_init(struct iommu_domain *dom)
+{
+	struct protection_domain *domain;
+
+	domain = kzalloc(sizeof(*domain), GFP_KERNEL);
+	if (!domain)
+		return -ENOMEM;
+
+	spin_lock_init(&domain->lock);
+	domain->mode = PAGE_MODE_3_LEVEL;
+	domain->id = domain_id_alloc();
+	if (!domain->id)
+		goto out_free;
+	domain->pt_root = (void *)get_zeroed_page(GFP_KERNEL);
+	if (!domain->pt_root)
+		goto out_free;
+
+	dom->priv = domain;
+
+	return 0;
+
+out_free:
+	kfree(domain);
+
+	return -ENOMEM;
+}

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #28 lore raw

PATCH 15/21 AMD IOMMU: add domain destroy function for IOMMU API

Impact: add a generic function for releasing protection domains

Signed-off-by Joerg Roedel <[email protected]>

---
 arch/x86/kernel/amd_iommu.c |   21 +++++++++++++++++++++
 1 files changed, 21 insertions(+), 0 deletions(-)

arch/x86/kernel/amd_iommu.c +21 -0

index cb7db66..d5581bb 100644
@@ -1552,3 +1552,24 @@ out_free:
 
 	return -ENOMEM;
 }
+
+static void amd_iommu_domain_destroy(struct iommu_domain *dom)
+{
+	struct protection_domain *domain = dom->priv;
+
+	if (!domain)
+		return;
+
+	if (domain->dev_cnt > 0)
+		cleanup_domain(domain);
+
+	BUG_ON(domain->dev_cnt != 0);
+
+	free_pagetable(domain);
+
+	domain_id_free(domain->id);
+
+	kfree(domain);
+
+	dom->priv = NULL;
+}

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #29 lore raw

PATCH 16/21 AMD IOMMU: add device detach function for IOMMU API

Impact: add a generic function to detach devices from protection domains

Signed-off-by Joerg Roedel <[email protected]>

---
 arch/x86/kernel/amd_iommu.c |   27 +++++++++++++++++++++++++++
 1 files changed, 27 insertions(+), 0 deletions(-)

arch/x86/kernel/amd_iommu.c +27 -0

index d5581bb..60e268a 100644
@@ -1573,3 +1573,30 @@ static void amd_iommu_domain_destroy(struct iommu_domain *dom)
 
 	dom->priv = NULL;
 }
+
+static void amd_iommu_detach_device(struct iommu_domain *dom,
+				    struct device *dev)
+{
+	struct protection_domain *domain = dom->priv;
+	struct amd_iommu *iommu;
+	struct pci_dev *pdev;
+	u16 devid;
+
+	if (dev->bus != &pci_bus_type)
+		return;
+
+	pdev = to_pci_dev(dev);
+
+	devid = calc_devid(pdev->bus->number, pdev->devfn);
+
+	if (devid > 0)
+		detach_device(domain, devid);
+
+	iommu = amd_iommu_rlookup_table[devid];
+	if (!iommu)
+		return;
+
+	iommu_queue_inv_dev_entry(iommu, devid);
+	iommu_completion_wait(iommu);
+}
+

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #30 lore raw

PATCH 17/21 AMD IOMMU: add device attach function for IOMMU API

Impact: add a generic function to attach devices to protection domains

Signed-off-by Joerg Roedel <[email protected]>

---
 arch/x86/kernel/amd_iommu.c |   35 +++++++++++++++++++++++++++++++++++
 1 files changed, 35 insertions(+), 0 deletions(-)

arch/x86/kernel/amd_iommu.c +35 -0

index 60e268a..97eb60d 100644
@@ -1600,3 +1600,38 @@ static void amd_iommu_detach_device(struct iommu_domain *dom,
 	iommu_completion_wait(iommu);
 }
 
+static int amd_iommu_attach_device(struct iommu_domain *dom,
+				   struct device *dev)
+{
+	struct protection_domain *domain = dom->priv;
+	struct protection_domain *old_domain;
+	struct amd_iommu *iommu;
+	struct pci_dev *pdev;
+	u16 devid;
+
+	if (dev->bus != &pci_bus_type)
+		return -EINVAL;
+
+	pdev = to_pci_dev(dev);
+
+	devid = calc_devid(pdev->bus->number, pdev->devfn);
+
+	if (devid >= amd_iommu_last_bdf ||
+			devid != amd_iommu_alias_table[devid])
+		return -EINVAL;
+
+	iommu = amd_iommu_rlookup_table[devid];
+	if (!iommu)
+		return -EINVAL;
+
+	old_domain = domain_for_device(devid);
+	if (old_domain)
+		__detach_device(old_domain, devid);
+
+	set_device_domain(iommu, domain, devid);
+
+	iommu_completion_wait(iommu);
+
+	return 0;
+}
+

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #31 lore raw

PATCH 18/21 AMD IOMMU: add domain map function for IOMMU API

Impact: add a generic function to map pages into protection domains

Signed-off-by Joerg Roedel <[email protected]>

---
 arch/x86/kernel/amd_iommu.c |   28 ++++++++++++++++++++++++++++
 1 files changed, 28 insertions(+), 0 deletions(-)

arch/x86/kernel/amd_iommu.c +28 -0

index 97eb60d..511f782 100644
@@ -1635,3 +1635,31 @@ static int amd_iommu_attach_device(struct iommu_domain *dom,
 	return 0;
 }
 
+static int amd_iommu_map_range(struct iommu_domain *dom,
+			       unsigned long iova, phys_addr_t paddr,
+			       size_t size, int iommu_prot)
+{
+	struct protection_domain *domain = dom->priv;
+	unsigned long i,  npages = iommu_num_pages(paddr, size, PAGE_SIZE);
+	int prot = 0;
+	int ret;
+
+	if (iommu_prot & IOMMU_READ)
+		prot |= IOMMU_PROT_IR;
+	if (iommu_prot & IOMMU_WRITE)
+		prot |= IOMMU_PROT_IW;
+
+	iova  &= PAGE_MASK;
+	paddr &= PAGE_MASK;
+
+	for (i = 0; i < npages; ++i) {
+		ret = iommu_map_page(domain, iova, paddr, prot);
+		if (ret)
+			return ret;
+
+		iova  += PAGE_SIZE;
+		paddr += PAGE_SIZE;
+	}
+
+	return 0;
+}

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #32 lore raw

PATCH 19/21 AMD IOMMU: add domain unmap function for IOMMU API

Impact: add a generic function to unmap pages into protection domains

Signed-off-by Joerg Roedel <[email protected]>

---
 arch/x86/kernel/amd_iommu.c |   39 +++++++++++++++++++++++++++++++++++++++
 1 files changed, 39 insertions(+), 0 deletions(-)

arch/x86/kernel/amd_iommu.c +39 -0

index 511f782..6ee9377 100644
@@ -438,6 +438,28 @@ static int iommu_map_page(struct protection_domain *dom,
 	return 0;
 }
 
+static void iommu_unmap_page(struct protection_domain *dom,
+			     unsigned long bus_addr)
+{
+	u64 *pte;
+
+	pte = &dom->pt_root[IOMMU_PTE_L2_INDEX(bus_addr)];
+
+	if (!IOMMU_PTE_PRESENT(*pte))
+		return;
+
+	pte = IOMMU_PTE_PAGE(*pte);
+	pte = &pte[IOMMU_PTE_L1_INDEX(bus_addr)];
+
+	if (!IOMMU_PTE_PRESENT(*pte))
+		return;
+
+	pte = IOMMU_PTE_PAGE(*pte);
+	pte = &pte[IOMMU_PTE_L1_INDEX(bus_addr)];
+
+	*pte = 0;
+}
+
 /*
  * This function checks if a specific unity mapping entry is needed for
  * this specific IOMMU.

@@ -1663,3 +1685,20 @@ static int amd_iommu_map_range(struct iommu_domain *dom,
 
 	return 0;
 }
+
+static void amd_iommu_unmap_range(struct iommu_domain *dom,
+				  unsigned long iova, size_t size)
+{
+
+	struct protection_domain *domain = dom->priv;
+	unsigned long i,  npages = iommu_num_pages(iova, size, PAGE_SIZE);
+
+	iova  &= PAGE_MASK;
+
+	for (i = 0; i < npages; ++i) {
+		iommu_unmap_page(domain, iova);
+		iova  += PAGE_SIZE;
+	}
+
+	iommu_flush_domain(domain->id);
+}

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #33 lore raw

PATCH 20/21 AMD IOMMU: add domain address lookup function for IOMMU API

Impact: add a generic function to lockup addresses in protection domains

Signed-off-by Joerg Roedel <[email protected]>

---
 arch/x86/kernel/amd_iommu.c |   31 +++++++++++++++++++++++++++++++
 1 files changed, 31 insertions(+), 0 deletions(-)

arch/x86/kernel/amd_iommu.c +31 -0

index 6ee9377..b42578b 100644
@@ -1702,3 +1702,34 @@ static void amd_iommu_unmap_range(struct iommu_domain *dom,
 
 	iommu_flush_domain(domain->id);
 }
+
+static phys_addr_t amd_iommu_iova_to_phys(struct iommu_domain *dom,
+					  unsigned long iova)
+{
+	struct protection_domain *domain = dom->priv;
+	unsigned long offset = iova & ~PAGE_MASK;
+	phys_addr_t paddr;
+	u64 *pte;
+
+	pte = &domain->pt_root[IOMMU_PTE_L2_INDEX(iova)];
+
+	if (!IOMMU_PTE_PRESENT(*pte))
+		return 0;
+
+	pte = IOMMU_PTE_PAGE(*pte);
+	pte = &pte[IOMMU_PTE_L1_INDEX(iova)];
+
+	if (!IOMMU_PTE_PRESENT(*pte))
+		return 0;
+
+	pte = IOMMU_PTE_PAGE(*pte);
+	pte = &pte[IOMMU_PTE_L0_INDEX(iova)];
+
+	if (!IOMMU_PTE_PRESENT(*pte))
+		return 0;
+
+	paddr  = *pte & IOMMU_PAGE_MASK;
+	paddr |= offset;
+
+	return paddr;
+}

Signature · 1 line

1.5.6.4

Joerg Roedel

author

2008-12-09 14:16 UTC #34 lore raw

PATCH 21/21 AMD IOMMU: register functions for the IOMMU API

Signed-off-by Joerg Roedel <[email protected]>

---
 arch/x86/kernel/amd_iommu.c |   14 ++++++++++++++
 1 files changed, 14 insertions(+), 0 deletions(-)

arch/x86/kernel/amd_iommu.c +14 -0

index b42578b..6670aae 100644
@@ -38,6 +38,8 @@ static DEFINE_RWLOCK(amd_iommu_devtable_lock);
 static LIST_HEAD(iommu_pd_list);
 static DEFINE_SPINLOCK(iommu_pd_list_lock);
 
+static struct iommu_ops amd_iommu_ops;
+
 /*
  * general struct to manage commands send to an IOMMU
  */

@@ -1481,6 +1483,8 @@ int __init amd_iommu_init_dma_ops(void)
 	/* Make the driver finally visible to the drivers */
 	dma_ops = &amd_iommu_dma_ops;
 
+	register_iommu(&amd_iommu_ops);
+
 	return 0;
 
 free_domains:

@@ -1733,3 +1737,13 @@ static phys_addr_t amd_iommu_iova_to_phys(struct iommu_domain *dom,
 
 	return paddr;
 }
+
+static struct iommu_ops amd_iommu_ops = {
+	.domain_init = amd_iommu_domain_init,
+	.domain_destroy = amd_iommu_domain_destroy,
+	.attach_dev = amd_iommu_attach_device,
+	.detach_dev = amd_iommu_detach_device,
+	.map = amd_iommu_map_range,
+	.unmap = amd_iommu_unmap_range,
+	.iova_to_phys = amd_iommu_iova_to_phys,
+};

Signature · 1 line

1.5.6.4

Avi Kivity

2008-12-10 09:36 UTC #35 lore raw

In reply to Joerg Roedel

Quoted 18 lines · Joerg Roedel

Hi,
the two patchsets posted as reply to this email implement KVM device passthrough support for AMD IOMMU hardware. The changes to the previous posts are descibed below
The first patchset is version 4 of the generic iommu api patchset which generalizes the VT-d functions exported to KVM into a common api where the AMD IOMMU code can plug into. In this version the patchset was rebased to the latest post of Han Weidong's patches.
The second patchset finally implements the KVM device passthrough support in the AMD IOMMU code. Together with KVM-79 I successfully passed an 10GBit network card into an KVM guest. In this version the patchset was changed to remove any device before a protection domain is freed instead of printing a BUG. Also the patchset was rebased to the updated IOMMU-API patches.

Quoted 9 lines

These two patchsets apply in order in top of the latest post of Han Weidong's Multiple device assignement support patches. Anybody who wants to try this out can pull the whole stuff from
git://git.kernel.org/pub/scm/linux/kernel/git/joro/linux-2.6-iommu.git kvm-amd-iommu
Please give these patches a good review.

Ack for the kvm bits.

Signature · 1 line

error compiling committee.c: too many arguments to function

Joerg Roedel

author

2008-12-10 14:11 UTC #36 lore raw

Quoted 23 lines · Avi Kivity

Quoted 22 lines · Joerg Roedel
Hi,
the two patchsets posted as reply to this email implement KVM device passthrough support for AMD IOMMU hardware. The changes to the previous posts are descibed below
The first patchset is version 4 of the generic iommu api patchset which generalizes the VT-d functions exported to KVM into a common api where the AMD IOMMU code can plug into. In this version the patchset was rebased to the latest post of Han Weidong's patches.
The second patchset finally implements the KVM device passthrough support in the AMD IOMMU code. Together with KVM-79 I successfully passed an 10GBit network card into an KVM guest. In this version the patchset was changed to remove any device before a protection domain is freed instead of printing a BUG. Also the patchset was rebased to the updated IOMMU-API patches.
These two patchsets apply in order in top of the latest post of Han Weidong's Multiple device assignement support patches. Anybody who wants to try this out can pull the whole stuff from

Quoted 7 lines

Quoted 5 lines
git://git.kernel.org/pub/scm/linux/kernel/git/joro/linux-2.6-iommu.git kvm-amd-iommu
Please give these patches a good review.
Ack for the kvm bits.

Thanks Avi.

So now its open how this will be merged alltogether. We can merge it in three steps (first from Dave's tree, then Avi and at last my IOMMU updates which has to happen in that order).
The other option is to merge this all with one pull-request I send to Linus. It should not conflict with your updates Avi because all these changes apply also cleanly to current linus/master.
Is this acceptable for everyone?

David, its important to hear your opinion on that because this single pull-request would include all VT-d updates you have queued in your tree? Is this ok for you?

Joerg

Signature · 6 lines

           |           AMD Saxony Limited Liability Company & Co. KG
 Operating |         Wilschdorfer Landstr. 101, 01109 Dresden, Germany
 System    |                  Register Court Dresden: HRA 4896
 Research  |              General Partner authorized to represent:
 Center    |             AMD Saxony LLC (Wilmington, Delaware, US)
           | General Manager of AMD Saxony LLC: Dr. Hans-R. Deppe, Thomas McCoy

David Woodhouse

2008-12-10 14:22 UTC #37 lore raw

Quoted 11 lines · Joerg Roedel

So now its open how this will be merged alltogether. We can merge it in three steps (first from Dave's tree, then Avi and at last my IOMMU updates which has to happen in that order).
The other option is to merge this all with one pull-request I send to Linus. It should not conflict with your updates Avi because all these changes apply also cleanly to current linus/master.
Is this acceptable for everyone?
David, its important to hear your opinion on that because this single pull-request would include all VT-d updates you have queued in your tree? Is this ok for you?

Yes, that's fine by me. Should I apply patches 1-14 of Weidong's patch set from Monday first? Or just let you do that?

Signature · 1 line

dwmw2

Joerg Roedel

author

2008-12-10 14:25 UTC #38 lore raw

Quoted 9 lines · David Woodhouse

Quoted 8 lines · Joerg Roedel
So now its open how this will be merged alltogether. We can merge it in three steps (first from Dave's tree, then Avi and at last my IOMMU updates which has to happen in that order).
The other option is to merge this all with one pull-request I send to Linus. It should not conflict with your updates Avi because all these changes apply also cleanly to current linus/master.
Is this acceptable for everyone?

Quoted 6 lines

Quoted 3 lines
David, its important to hear your opinion on that because this single pull-request would include all VT-d updates you have queued in your tree? Is this ok for you?
Yes, that's fine by me. Should I apply patches 1-14 of Weidong's patch set from Monday first? Or just let you do that?

I already did it on-top of your tree because Han Weidong's patches 1-17 were rebased to your tree and my IOMMU-API patches apply on-top of his patches.

Joerg

Signature · 6 lines

           |           AMD Saxony Limited Liability Company & Co. KG
 Operating |         Wilschdorfer Landstr. 101, 01109 Dresden, Germany
 System    |                  Register Court Dresden: HRA 4896
 Research  |              General Partner authorized to represent:
 Center    |             AMD Saxony LLC (Wilmington, Delaware, US)
           | General Manager of AMD Saxony LLC: Dr. Hans-R. Deppe, Thomas McCoy

David Woodhouse

2008-12-10 18:25 UTC #39 lore raw

Quoted 3 lines · Joerg Roedel

I already did it on-top of your tree because Han Weidong's patches 1-17 were rebased to your tree and my IOMMU-API patches apply on-top of his patches.

OK, thanks. Looks like you still need this though, as Mike Day pointed out:

drivers/pci/intel-iommu.c +1 -1

index 97df47c..a5f6e28 100644
@@ -204,7 +204,7 @@ static inline bool dma_pte_present(struct dma_pte *pte)
 }
 
 /* devices under the same p2p bridge are owned in one domain */
-#define DOMAIN_FLAG_P2P_MULTIPLE_DEVICES (1 < 0)
+#define DOMAIN_FLAG_P2P_MULTIPLE_DEVICES (1 << 0)
 
 /* domain represents a virtual machine, more than one devices
  * across iommus may be owned in one domain, e.g. kvm guest.

Signature · 2 lines

David Woodhouse                            Open Source Technology Centre
[email protected]                              Intel Corporation

Joerg Roedel

author

2008-12-10 18:42 UTC #40 lore raw

Quoted 13 lines · David Woodhouse

Quoted 3 lines · Joerg Roedel
I already did it on-top of your tree because Han Weidong's patches 1-17 were rebased to your tree and my IOMMU-API patches apply on-top of his patches.
OK, thanks. Looks like you still need this though, as Mike Day pointed out:
drivers/pci/intel-iommu.c +0 -0
index 97df47c..a5f6e28 100644

drivers/pci/intel-iommu.c static inline bool dma_pte_present(struct dma_pte *pte) · line 210

Quoted 8 lines

drivers/pci/intel-iommu.c +1 -1

@@ -204,7 +204,7 @@ static inline bool dma_pte_present(struct dma_pte *pte)
 }
 
 /* devices under the same p2p bridge are owned in one domain */
-#define DOMAIN_FLAG_P2P_MULTIPLE_DEVICES (1 < 0)
+#define DOMAIN_FLAG_P2P_MULTIPLE_DEVICES (1 << 0)
 
 /* domain represents a virtual machine, more than one devices
  * across iommus may be owned in one domain, e.g. kvm guest.

Ok, I add it, thanks. Who is the author, Mike or you?

Signature · 6 lines

           |           AMD Saxony Limited Liability Company & Co. KG
 Operating |         Wilschdorfer Landstr. 101, 01109 Dresden, Germany
 System    |                  Register Court Dresden: HRA 4896
 Research  |              General Partner authorized to represent:
 Center    |             AMD Saxony LLC (Wilmington, Delaware, US)
           | General Manager of AMD Saxony LLC: Dr. Hans-R. Deppe, Thomas McCoy

David Woodhouse

2008-12-10 18:48 UTC #41 lore raw

Quoted 1 line · Joerg Roedel

Ok, I add it, thanks. Who is the author, Mike or you?

Might as well attribute it to Mike; he spotted it.

Signature · 1 line

dwmw2

Joerg Roedel

author

2008-12-15 11:29 UTC #42 lore raw

Quoted 4 lines · David Woodhouse

Quoted 1 line · Joerg Roedel
Ok, I add it, thanks. Who is the author, Mike or you?
Might as well attribute it to Mike; he spotted it.

Ok, here is what I applied, fyi.

Signature · 6 lines

           |           AMD Saxony Limited Liability Company & Co. KG
 Operating |         Wilschdorfer Landstr. 101, 01109 Dresden, Germany
 System    |                  Register Court Dresden: HRA 4896
 Research  |              General Partner authorized to represent:
 Center    |             AMD Saxony LLC (Wilmington, Delaware, US)
           | General Manager of AMD Saxony LLC: Dr. Hans-R. Deppe, Thomas McCoy

commit 8876830866c8c9e4471a578a4e474eeef80e98dd

Author: Mike Day <[email protected]>
Date:   Fri Dec 12 17:16:30 2008 +0100

intel-iommu: fix bit shift at DOMAIN_FLAG_P2P_MULTIPLE_DEVICES

Signed-off-by Mike Day <[email protected]>

Signed-off-by Joerg Roedel <[email protected]>

drivers/pci/intel-iommu.c +1 -1

index 772fb22..f39af75 100644
@@ -204,7 +204,7 @@ static inline bool dma_pte_present(struct dma_pte *pte)
 }
 
 /* devices under the same p2p bridge are owned in one domain */
-#define DOMAIN_FLAG_P2P_MULTIPLE_DEVICES (1 < 0)
+#define DOMAIN_FLAG_P2P_MULTIPLE_DEVICES (1 << 0)
 
 /* domain represents a virtual machine, more than one devices
  * across iommus may be owned in one domain, e.g. kvm guest.

Han, Weidong

2008-12-11 01:11 UTC #43 lore raw

In reply to Joerg Roedel

Quoted 14 lines · Joerg Roedel

Quoted 13 lines · David Woodhouse
Quoted 11 lines · Joerg Roedel
So now its open how this will be merged alltogether. We can merge it in three steps (first from Dave's tree, then Avi and at last my IOMMU updates which has to happen in that order).
The other option is to merge this all with one pull-request I send to Linus. It should not conflict with your updates Avi because all these changes apply also cleanly to current linus/master. Is this acceptable for everyone?
David, its important to hear your opinion on that because this single pull-request would include all VT-d updates you have queued in your tree? Is this ok for you?

Quoted 8 lines

Quoted 2 lines
Yes, that's fine by me. Should I apply patches 1-14 of Weidong's patch set from Monday first? Or just let you do that?
I already did it on-top of your tree because Han Weidong's patches 1-17 were rebased to your tree and my IOMMU-API patches apply on-top of his patches.
Joerg

It's good. Thanks Dave and Joerg.

Regards,
Weidong

Avi Kivity

2008-12-10 14:24 UTC #44 lore raw

In reply to Joerg Roedel

Quoted 8 lines · Joerg Roedel

So now its open how this will be merged alltogether. We can merge it in three steps (first from Dave's tree, then Avi and at last my IOMMU updates which has to happen in that order).
The other option is to merge this all with one pull-request I send to Linus. It should not conflict with your updates Avi because all these changes apply also cleanly to current linus/master.
Is this acceptable for everyone?

Okay for me.

Signature · 1 line

error compiling committee.c: too many arguments to function