Xen-SWIOTLB v0.8.6 for 2.6.36

Hey Linus,

Please pull the following git tree:

git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git stable/xen-swiotlb-0.8.6

in your new 2.6.36 branch.

They are based on top of this git commit:e40152ee1e1c7a63f4777791863215e3faa37a86
Author: Linus Torvalds <[email protected]>

    Linus 2.6.34

They have been reviewed, acked, discussed, tested and retested and been brewing in linux-next.

This feature adds an IOMMU that can be used by Xen PV guests when a PCI pass-through device is utilized. The necessity of adding another IOMMU is that PV guests under Xen are running in an non-contiguous memory architecture. More details on this are at the end of this e-mail.

One patch touched common code:
x86: Detect whether we should use Xen SWIOTLB.

which was contentious and the comments raised were "inserting a sub-system call in the generic code is not good." and " we should remove all sub-system detect calls (Calgary, AMD, Intel, AGP) altogether from pci-dma.c and depend more on x86_init.iommu structure (perhaps expend it?)"

We [hpa, Fujita, me] decided that the patch that sparked this discussion would be Ack-ed for 2.6.36 and I would start working on a new framework to address the concerns: http://lkml.org/lkml/2010/8/2/282

The rest are Xen specific patches:

Alex Nixon (3):
xen: Allow unprivileged Xen domains to create iomap pages
xen: Rename the balloon lock
xen: Add xen_create_contiguous_region

Jeremy Fitzhardinge (3):
xen: use _PAGE_IOMAP in ioremap to do machine mappings
vmap: add flag to allow lazy unmap to be disabled at runtime xen/mmu: inhibit vmap aliases rather than trying to clear them out

Konrad Rzeszutek Wilk (3):
swiotlb-xen: SWIOTLB library for Xen PV guest with PCI passthrough.
pci-swiotlb-xen: Add glue code to setup dma_ops utilizing xen_swiotlb_* functions.
x86: Detect whether we should use Xen SWIOTLB.

 arch/x86/include/asm/xen/page.h        |    8 +-
 arch/x86/include/asm/xen/swiotlb-xen.h |   14 +
 arch/x86/kernel/pci-dma.c              |    7 +-
 arch/x86/xen/Makefile                  |    1 +
 arch/x86/xen/enlighten.c               |    4 +
 arch/x86/xen/mmu.c                     |  293 ++++++++++++++++++-
 arch/x86/xen/pci-swiotlb-xen.c         |   58 ++++
 drivers/xen/Kconfig                    |    4 +
 drivers/xen/Makefile                   |    3 +-
 drivers/xen/balloon.c                  |   15 +-
 drivers/xen/swiotlb-xen.c              |  515 ++++++++++++++++++++++++++++++++
 include/linux/vmalloc.h                |    2 +
 include/xen/interface/memory.h         |   50 +++
 include/xen/swiotlb-xen.h              |   65 ++++
 include/xen/xen-ops.h                  |    6 +
 mm/vmalloc.c                           |    4 +
 16 files changed, 1024 insertions(+), 25 deletions(-)

And the lengthy explanation:

These nine patches lay the groundwork for Xen Paravirtualized (PV) domains to access PCI pass-through devices (with and without an hardware IOMMU) These patches utilize the SWIOTLB library modifications
(http://lkml.org/lkml/2010/6/4/272).

The end user of this are:
- The Xen PCI frontend and Xen PCI [1] which require a DMA API "backend" that understands Xen's MMU. This allows the PV domains to use PCI devices. The use case is for machines with and without hardware IOMMU. Without an hardware IOMMU you have a potential security hole wherein a guest domain can use the hardware to map pages outside its memory range and slurp pages up. As such, this is more restricted to a Privileged PV domain, aka - device driver domain (similar to Qubes but a poor-man mechanism [2]).
- Xen PV domain 0 support. Without this domain 0 is incapable of using any PCI devices.

This patch-set is split in two groups. The first alter the Xen components, while the second introduces the SWIOTLB-Xen.

The Xen components patches consist of:

xen: Allow unprivileged Xen domains to create iomap pages
xen: Rename the balloon lock
xen: Add xen_create_contiguous_region
xen: use _PAGE_IOMAP in ioremap to do machine mappings
vmap: add flag to allow lazy unmap to be disabled at runtime xen/mmu: inhibit vmap aliases rather than trying to clear them out

which alter the Xen MMU, which by default utilizes a layer of indirection wherein the PFN is translated to the Machine Frame Number (MFN) and vice-versa. This is required to "fool" the guest in thinking its memory starts at PFN 0 and goes up to the available amount. While in the background, PFN 0 might as well be MFN 1048576 (4GB).

For PCI/DMA API calls (ioremap, pci_map_page, etc) having a PFN != MFN is not too good, so two new mechanisms are introduced:
a). For PTEs which manipulate the PCI/DMA pages the PFN == MFN. This is done
via utilizing the _PAGE_IOMAP flag to distinguish the PTE as an IO type.
b). Allow a mechanism to "swizzle" the MFN's under a PFN to be under a certain
bus width (say 32). This allows us to provide a mechanism for the SWIOTLB Xen to allocate memory for its pool that will be guaranteed to be under the 4GB mark.

We also introduce a mechanism to inhibit vmap aliases as we need to clear the vmap's but might be doing this in atomic state.

The SWIOTLB-Xen adds a library that is only used if the machine is detected to be running under Xen. It utilizes the SWIOTLB library bookkeeping functions (swiotlb_tbl_*) and only deals with the virtual to [physical, bus] (and vice-versa) address translations.

Here is the full diff:

index 018a0a4..bf5f7d3 100644
@@ -112,13 +112,9 @@ static inline xpaddr_t machine_to_phys(xmaddr_t machine)
  */
 static inline unsigned long mfn_to_local_pfn(unsigned long mfn)
 {
-	extern unsigned long max_mapnr;
 	unsigned long pfn = mfn_to_pfn(mfn);
-	if ((pfn < max_mapnr)
-	    && !xen_feature(XENFEAT_auto_translated_physmap)
-	    && (get_phys_to_machine(pfn) != mfn))
-		return max_mapnr; /* force !pfn_valid() */
-	/* XXX fixme; not true with sparsemem */
+	if (get_phys_to_machine(pfn) != mfn)
+		return -1; /* force !pfn_valid() */
 	return pfn;
 }
 

new file mode 100644
index 0000000..1be1ab7
@@ -0,0 +1,14 @@
+#ifndef _ASM_X86_SWIOTLB_XEN_H
+#define _ASM_X86_SWIOTLB_XEN_H
+
+#ifdef CONFIG_SWIOTLB_XEN
+extern int xen_swiotlb;
+extern int __init pci_xen_swiotlb_detect(void);
+extern void __init pci_xen_swiotlb_init(void);
+#else
+#define xen_swiotlb (0)
+static inline int __init pci_xen_swiotlb_detect(void) { return 0; }
+static inline void __init pci_xen_swiotlb_init(void) { }
+#endif
+
+#endif /* _ASM_X86_SWIOTLB_XEN_H */

index 4b7e3d8..9f07cfc 100644
@@ -13,6 +13,7 @@
 #include <asm/calgary.h>
 #include <asm/amd_iommu.h>
 #include <asm/x86_init.h>
+#include <asm/xen/swiotlb-xen.h>
 
 static int forbid_dac __read_mostly;
 

@@ -132,7 +133,7 @@ void __init pci_iommu_alloc(void)
 	/* free the range so iommu could get some range less than 4G */
 	dma32_free_bootmem();
 
-	if (pci_swiotlb_detect())
+	if (pci_xen_swiotlb_detect() || pci_swiotlb_detect())
 		goto out;
 
 	gart_iommu_hole_init();

@@ -144,6 +145,8 @@ void __init pci_iommu_alloc(void)
 	/* needs to be called after gart_iommu_hole_init */
 	amd_iommu_detect();
 out:
+	pci_xen_swiotlb_init();
+
 	pci_swiotlb_init();
 }
 

@@ -296,7 +299,7 @@ static int __init pci_iommu_init(void)
 #endif
 	x86_init.iommu.iommu_init();
 
-	if (swiotlb) {
+	if (swiotlb || xen_swiotlb) {
 		printk(KERN_INFO "PCI-DMA: "
 		       "Using software bounce buffering for IO (SWIOTLB)\n");
 		swiotlb_print_info();

index 3bb4fc2..32af238 100644
@@ -18,3 +18,4 @@ obj-$(CONFIG_SMP)		+= smp.o
 obj-$(CONFIG_PARAVIRT_SPINLOCKS)+= spinlock.o
 obj-$(CONFIG_XEN_DEBUG_FS)	+= debugfs.o
 
+obj-$(CONFIG_SWIOTLB_XEN)	+= pci-swiotlb-xen.o

index 65d8d79..3254e8b 100644
@@ -1145,6 +1145,10 @@ asmlinkage void __init xen_start_kernel(void)
 
 	pgd = (pgd_t *)xen_start_info->pt_base;
 
+	if (!xen_initial_domain())
+		__supported_pte_mask &= ~(_PAGE_PWT | _PAGE_PCD);
+
+	__supported_pte_mask |= _PAGE_IOMAP;
 	/* Don't do the full vcpu_info placement stuff until we have a
 	   possible map and a non-dummy shared_info. */
 	per_cpu(xen_vcpu, 0) = &HYPERVISOR_shared_info->vcpu_info[0];

index fad3df2..97199c2 100644
@@ -62,4 +62,8 @@ config XEN_SYS_HYPERVISOR
 	 virtual environment, /sys/hypervisor will still be present,
 	 but will have no xen contents.
 
+config SWIOTLB_XEN
+	def_bool y
+	depends on SWIOTLB
+
 endmenu

index 7c28434..85f84cf 100644
@@ -9,4 +9,5 @@ obj-$(CONFIG_XEN_XENCOMM)	+= xencomm.o
 obj-$(CONFIG_XEN_BALLOON)	+= balloon.o
 obj-$(CONFIG_XEN_DEV_EVTCHN)	+= evtchn.o
 obj-$(CONFIG_XENFS)		+= xenfs/
-obj-$(CONFIG_XEN_SYS_HYPERVISOR)	+= sys-hypervisor.o
\ No newline at end of file
+obj-$(CONFIG_XEN_SYS_HYPERVISOR)	+= sys-hypervisor.o
+obj-$(CONFIG_SWIOTLB_XEN)	+= swiotlb-xen.o

index 1a0d8c2..500290b 100644
@@ -85,13 +85,6 @@ static struct sys_device balloon_sysdev;
 
 static int register_balloon(struct sys_device *sysdev);
 
-/*
- * Protects atomic reservation decrease/increase against concurrent increases.
- * Also protects non-atomic updates of current_pages and driver_pages, and
- * balloon lists.
- */
-static DEFINE_SPINLOCK(balloon_lock);
-
 static struct balloon_stats balloon_stats;
 
 /* We increase/decrease in batches which fit in a page */

@@ -210,7 +203,7 @@ static int increase_reservation(unsigned long nr_pages)
 	if (nr_pages > ARRAY_SIZE(frame_list))
 		nr_pages = ARRAY_SIZE(frame_list);
 
-	spin_lock_irqsave(&balloon_lock, flags);
+	spin_lock_irqsave(&xen_reservation_lock, flags);
 
 	page = balloon_first_page();
 	for (i = 0; i < nr_pages; i++) {

@@ -254,7 +247,7 @@ static int increase_reservation(unsigned long nr_pages)
 	balloon_stats.current_pages += rc;
 
  out:
-	spin_unlock_irqrestore(&balloon_lock, flags);
+	spin_unlock_irqrestore(&xen_reservation_lock, flags);
 
 	return rc < 0 ? rc : rc != nr_pages;
 }

@@ -299,7 +292,7 @@ static int decrease_reservation(unsigned long nr_pages)
 	kmap_flush_unused();
 	flush_tlb_all();
 
-	spin_lock_irqsave(&balloon_lock, flags);
+	spin_lock_irqsave(&xen_reservation_lock, flags);
 
 	/* No more mappings: invalidate P2M and add to balloon. */
 	for (i = 0; i < nr_pages; i++) {

@@ -315,7 +308,7 @@ static int decrease_reservation(unsigned long nr_pages)
 
 	balloon_stats.current_pages -= nr_pages;
 
-	spin_unlock_irqrestore(&balloon_lock, flags);
+	spin_unlock_irqrestore(&xen_reservation_lock, flags);
 
 	return need_sleep;
 }

index 227c2a5..b840fda 100644
@@ -7,6 +7,8 @@
 
 struct vm_area_struct;		/* vma defining user mapping in mm_types.h */
 
+extern bool vmap_lazy_unmap;
+
 /* bits in flags of vmalloc's vm_struct below */
 #define VM_IOREMAP	0x00000001	/* ioremap() and friends */
 #define VM_ALLOC	0x00000002	/* vmalloc() */

index 883a21b..d789c93 100644
@@ -14,4 +14,10 @@ void xen_mm_unpin_all(void);
 void xen_timer_resume(void);
 void xen_arch_resume(void);
 
+extern unsigned long *xen_contiguous_bitmap;
+int xen_create_contiguous_region(unsigned long vstart, unsigned int order,
+				unsigned int address_bits);
+
+void xen_destroy_contiguous_region(unsigned long vstart, unsigned int order);
+
 #endif /* INCLUDE_XEN_OPS_H */

index ae00746..7f35fe2 100644
@@ -31,6 +31,7 @@
 #include <asm/tlbflush.h>
 #include <asm/shmparam.h>
 
+bool vmap_lazy_unmap __read_mostly = true;
 
 /*** Page table manipulation functions ***/
 

@@ -502,6 +503,9 @@ static unsigned long lazy_max_pages(void)
 {
 	unsigned int log;
 
+	if (!vmap_lazy_unmap)
+		return 0;
+
 	log = fls(num_online_cpus());
 
 	return log * (32UL * 1024 * 1024 / PAGE_SIZE);