Skip to contentSkip to conversation

swiotlb maintenance and x86_64 dma_sync_single_range_for_{cpu,device}

PATCH In review Luck, Tony 4 participants 12 messages Sep 26, 2005 – Sep 29, 2005 Library code

lore
Share & tools
lore

Luck, Tony

author
#0 lore raw
Quoted 5 lines

Well, now, this is a quandry isn't it... Actually, I'm inclined to agree with you.

Tony, et al., care to restate your reasoning for moving it under drivers/pci?

Historically swiotlb.c was written with just PCI in mind (hence all the comments ("... implement the PCI DMA API", "The PCI address to use is returned", "teardown the PCI dma mapping") and a few error messages ("PCI-DMA: Out of SW-IOMMU space ...", "PCI-DMA: Memory would be corrupted", "PCI-DMA: Random memory would be DMAed"). Perhaps back then the only options were PCI and ISA????

Matthew is probably technically right in that this is a more generic interface ... but is it actually being used for anything other than PCI? Will it ever be so used?

Ignoring the comments and error messages, the attached patch removed the <linux/pci.h> and <asm/pci.h> includes (and adds a couple of others that are then needed, and converts a few PCI_DMA_* defines to DMA_* ones that sound like they mean the same thing). Compiles, but not tested.

-Tony

arch/ia64/lib/swiotlb.c +4 -4
@@ -15,17 +15,17 @@
  */
 
 #include <linux/cache.h>
+#include <linux/dma-mapping.h>
 #include <linux/mm.h>
 #include <linux/module.h>
-#include <linux/pci.h>
 #include <linux/spinlock.h>
 #include <linux/string.h>
 #include <linux/types.h>
 #include <linux/ctype.h>
 
 #include <asm/io.h>
-#include <asm/pci.h>
 #include <asm/dma.h>
+#include <asm/scatterlist.h>
 
 #include <linux/init.h>
 #include <linux/bootmem.h>
@@ -391,9 +391,9 @@
 	       "device %s\n", size, dev ? dev->bus_id : "?");
 
 	if (size > io_tlb_overflow && do_panic) {
-		if (dir == PCI_DMA_FROMDEVICE || dir == PCI_DMA_BIDIRECTIONAL)
+		if (dir == DMA_FROM_DEVICE || dir == DMA_BIDIRECTIONAL)
 			panic("PCI-DMA: Memory would be corrupted\n");
-		if (dir == PCI_DMA_TODEVICE || dir == PCI_DMA_BIDIRECTIONAL)
+		if (dir == DMA_TO_DEVICE || dir == DMA_BIDIRECTIONAL)
 			panic("PCI-DMA: Random memory would be DMAed\n");
 	}
 }
#1 lore raw
Quoted 6 lines · Tony

Historically swiotlb.c was written with just PCI in mind (hence all the comments ("... implement the PCI DMA API", "The PCI address to use is returned", "teardown the PCI dma mapping") and a few error messages ("PCI-DMA: Out of SW-IOMMU space ...", "PCI-DMA: Memory would be corrupted", "PCI-DMA: Random memory would be DMAed"). Perhaps back then the only options were PCI and ISA????

Yes. The DMA interface davem/et al introduce in linux-2.4 only supported "PCI-Like" busses. Ie the API required struct pci_dev.

Quoted 3 lines

Matthew is probably technically right in that this is a more generic interface ... but is it actually being used for anything other than PCI? Will it ever be so used?

Besides 32-bit PCI devices, I expect legacy 24-bit E/ISA DMA will need it. Is ISA ~= PCI? I never got a clear answer on that. I'm inclined to say it's not.

But since swiotlb complies with DMA-API interface and is not related to any particular type of bus, I'd rather it go into lib/ instead of drivers/pci.

hth,
grant

#2 lore raw
Quoted 3 lines · Grant Grundler

But since swiotlb complies with DMA-API interface and is not related to any particular type of bus, I'd rather it go into lib/ instead of drivers/pci.

OK...I read Tony's patch post to imply that he is consenting to lib/ as the location as well. Tony, please speak-up if that is not the case.

If everyone agrees, I'll re-collect the patches moving everything to lib/ (plus the one I forgot to repost) and Tony's patch and repost.

So, who is the proper committer for this? Tony? Or should I send them directly to Andrew?

Thanks,

John

Signature · 2 lines
John W. Linville
[email protected]
#3 lore raw
Quoted 2 lines

OK...I read Tony's patch post to imply that he is consenting to lib/ as the location as well. Tony, please speak-up if that is not the case.

lib/ is certainly better than drivers/pci ... so unless anyone has a better location, we'll go with that.

Quoted 2 lines

If everyone agrees, I'll re-collect the patches moving everything to lib/ (plus the one I forgot to repost) and Tony's patch and repost.

My patch only removed the include for linux/pci.h and asm/pci.h (as per Mathhew Wilcox's suggestion) and then cleaned up the resulting compilation fluff. There are a few more PCI references in comments and printk messages that should be cleaned up too. Oh, I only compile tested for ia64 ... so it needs compile-testing for x86-64, and boot testing for both archs.

Quoted 2 lines

So, who is the proper committer for this? Tony? Or should I send them directly to Andrew?

I have one other change to swiotlb.c (from Alex Williamson) sitting in my "test" tree, which needs to not get lost. I'm happy to drive this (target 2.6.15 ... this isn't 2.6.14 bug fix material). So long as I don't get burned to a crisp for offloading this Intel specific[1] stuff into the generic part of the tree.

-Tony

[1] Only ia64 and em64t use this right now.

#4 lore raw

cover swiotlb maintenance and x86_64 dma_sync_single_range_for_{cpu,device}

Conduct some maintenance of the swiotlb code:

-- Move the code from arch/ia64/lib/ to lib/

-- Cleanup some cruft (code duplication)

-- Add support for syncing sub-ranges of mappings

-- Add support for syncing DMA_BIDIRECTIONAL mappings

-- Comment fixup & change record

Also, tack-on an x86_64 implementation of dma_sync_single_range_for_cpu and dma_sync_single_range_for _device. This makes use of the new swiotlb sync sub-range support.

In this (third) round, the new location for swiotlb is (again) lib/swiotlb.c.

This set does NOT include the PCI excision that Tony Luck started after the last round of discussion.

Patches to follow...

#5 lore raw

PATCH 1/6 swiotlb: move from arch/ia64/lib/ to lib/

The swiotlb implementation is shared by both IA-64 and EM64T. However, the source itself lives under arch/ia64. This patch moves swiotlb.c from arch/ia64/lib to lib/ and fixes-up the appropriate Makefile and Kconfig files. No actual changes are made to swiotlb.c.

---

 arch/ia64/Kconfig           |    4 
 arch/ia64/lib/Makefile      |    2 
 arch/ia64/lib/swiotlb.c     |  657 --------------------------------------------
 arch/x86_64/kernel/Makefile |    2 
 lib/Makefile                |    2 
 lib/swiotlb.c               |  657 ++++++++++++++++++++++++++++++++++++++++++++
 6 files changed, 664 insertions(+), 660 deletions(-)
arch/ia64/Kconfig +4 -0
@@ -26,6 +26,10 @@ config MMU
 	bool
 	default y
 
+config SWIOTLB
+       bool
+       default y
+
 config RWSEM_XCHGADD_ALGORITHM
 	bool
 	default y
arch/ia64/lib/Makefile +1 -1
@@ -9,7 +9,7 @@ lib-y := __divsi3.o __udivsi3.o __modsi3
 	bitop.o checksum.o clear_page.o csum_partial_copy.o		\
 	clear_user.o strncpy_from_user.o strlen_user.o strnlen_user.o	\
 	flush.o ip_fast_csum.o do_csum.o				\
-	memset.o strlen.o swiotlb.o
+	memset.o strlen.o
 
 lib-$(CONFIG_ITANIUM)	+= copy_page.o copy_user.o memcpy.o
 lib-$(CONFIG_MCKINLEY)	+= copy_page_mck.o memcpy_mck.o
arch/ia64/lib/swiotlb.c +0 -657

657 lines not on this page · raw message

arch/x86_64/kernel/Makefile +0 -2
@@ -27,7 +27,6 @@ obj-$(CONFIG_CPU_FREQ)		+= cpufreq/
 obj-$(CONFIG_EARLY_PRINTK)	+= early_printk.o
 obj-$(CONFIG_GART_IOMMU)	+= pci-gart.o aperture.o
 obj-$(CONFIG_DUMMY_IOMMU)	+= pci-nommu.o pci-dma.o
-obj-$(CONFIG_SWIOTLB)		+= swiotlb.o
 obj-$(CONFIG_KPROBES)		+= kprobes.o
 obj-$(CONFIG_X86_PM_TIMER)	+= pmtimer.o
 
@@ -41,7 +40,6 @@ CFLAGS_vsyscall.o		:= $(PROFILING) -g0
 bootflag-y			+= ../../i386/kernel/bootflag.o
 cpuid-$(subst m,y,$(CONFIG_X86_CPUID))  += ../../i386/kernel/cpuid.o
 topology-y                     += ../../i386/mach-default/topology.o
-swiotlb-$(CONFIG_SWIOTLB)      += ../../ia64/lib/swiotlb.o
 microcode-$(subst m,y,$(CONFIG_MICROCODE))  += ../../i386/kernel/microcode.o
 intel_cacheinfo-y		+= ../../i386/kernel/cpu/intel_cacheinfo.o
 quirks-y			+= ../../i386/kernel/quirks.o
lib/Makefile +2 -0
@@ -44,6 +44,8 @@ obj-$(CONFIG_TEXTSEARCH_KMP) += ts_kmp.o
 obj-$(CONFIG_TEXTSEARCH_BM) += ts_bm.o
 obj-$(CONFIG_TEXTSEARCH_FSM) += ts_fsm.o
 
+obj-$(CONFIG_SWIOTLB) += swiotlb.o
+
 hostprogs-y	:= gen_crc32table
 clean-files	:= crc32table.h
 
lib/swiotlb.c +657 -0

657 lines not on this page · raw message

#6 lore raw

PATCH 2/6 swiotlb: cleanup some code duplication cruft

The implementations of swiotlb_sync_single_for_{cpu,device} are identical. Likewise for swiotlb_syng_sg_for_{cpu,device}. This patch move the guts of those functions to two new inline functions, and calls the appropriate one from the bodies of those functions.

---

 lib/swiotlb.c |   45 ++++++++++++++++++++++-----------------------
 1 files changed, 22 insertions(+), 23 deletions(-)
lib/swiotlb.c +22 -23
@@ -492,9 +492,9 @@ swiotlb_unmap_single(struct device *hwde
  * address back to the card, you must first perform a
  * swiotlb_dma_sync_for_device, and then the device again owns the buffer
  */
-void
-swiotlb_sync_single_for_cpu(struct device *hwdev, dma_addr_t dev_addr,
-			    size_t size, int dir)
+static inline void
+swiotlb_sync_single(struct device *hwdev, dma_addr_t dev_addr,
+		    size_t size, int dir)
 {
 	char *dma_addr = phys_to_virt(dev_addr);
 
@@ -507,17 +507,17 @@ swiotlb_sync_single_for_cpu(struct devic
 }
 
 void
+swiotlb_sync_single_for_cpu(struct device *hwdev, dma_addr_t dev_addr,
+			    size_t size, int dir)
+{
+	swiotlb_sync_single(hwdev, dev_addr, size, dir);
+}
+
+void
 swiotlb_sync_single_for_device(struct device *hwdev, dma_addr_t dev_addr,
 			       size_t size, int dir)
 {
-	char *dma_addr = phys_to_virt(dev_addr);
-
-	if (dir = DMA_NONE)
-		BUG();
-	if (dma_addr >= io_tlb_start && dma_addr < io_tlb_end)
-		sync_single(hwdev, dma_addr, size, dir);
-	else if (dir = DMA_FROM_DEVICE)
-		mark_clean(dma_addr, size);
+	swiotlb_sync_single(hwdev, dev_addr, size, dir);
 }
 
 /*
@@ -594,9 +594,9 @@ swiotlb_unmap_sg(struct device *hwdev, s
  * The same as swiotlb_sync_single_* but for a scatter-gather list, same rules
  * and usage.
  */
-void
-swiotlb_sync_sg_for_cpu(struct device *hwdev, struct scatterlist *sg,
-			int nelems, int dir)
+static inline void
+swiotlb_sync_sg(struct device *hwdev, struct scatterlist *sg,
+		int nelems, int dir)
 {
 	int i;
 
@@ -610,18 +610,17 @@ swiotlb_sync_sg_for_cpu(struct device *h
 }
 
 void
+swiotlb_sync_sg_for_cpu(struct device *hwdev, struct scatterlist *sg,
+			int nelems, int dir)
+{
+	swiotlb_sync_sg(hwdev, sg, nelems, dir);
+}
+
+void
 swiotlb_sync_sg_for_device(struct device *hwdev, struct scatterlist *sg,
 			   int nelems, int dir)
 {
-	int i;
-
-	if (dir = DMA_NONE)
-		BUG();
-
-	for (i = 0; i < nelems; i++, sg++)
-		if (sg->dma_address != SG_ENT_PHYS_ADDRESS(sg))
-			sync_single(hwdev, (void *) sg->dma_address,
-				    sg->dma_length, dir);
+	swiotlb_sync_sg(hwdev, sg, nelems, dir);
 }
 
 int
#7 lore raw

PATCH 3/6 swiotlb: support syncing sub-ranges of mappings

This patch implements swiotlb_sync_single_range_for_{cpu,device}. This is intended to support an x86_64 implementation of dma_sync_single_range_for_{cpu,device}.

---

 include/asm-x86_64/swiotlb.h |    8 ++++++++
 lib/swiotlb.c                |   33 +++++++++++++++++++++++++++++++++
 2 files changed, 41 insertions(+)
include/asm-x86_64/swiotlb.h +8 -0
@@ -15,6 +15,14 @@ extern void swiotlb_sync_single_for_cpu(
 extern void swiotlb_sync_single_for_device(struct device *hwdev,
 					    dma_addr_t dev_addr,
 					    size_t size, int dir);
+extern void swiotlb_sync_single_range_for_cpu(struct device *hwdev,
+					      dma_addr_t dev_addr,
+					      unsigned long offset,
+					      size_t size, int dir);
+extern void swiotlb_sync_single_range_for_device(struct device *hwdev,
+						 dma_addr_t dev_addr,
+						 unsigned long offset,
+						 size_t size, int dir);
 extern void swiotlb_sync_sg_for_cpu(struct device *hwdev,
 				     struct scatterlist *sg, int nelems,
 				     int dir);
lib/swiotlb.c +33 -0
@@ -521,6 +521,37 @@ swiotlb_sync_single_for_device(struct de
 }
 
 /*
+ * Same as above, but for a sub-range of the mapping.
+ */
+static inline void
+swiotlb_sync_single_range(struct device *hwdev, dma_addr_t dev_addr,
+			  unsigned long offset, size_t size, int dir)
+{
+	char *dma_addr = phys_to_virt(dev_addr) + offset;
+
+	if (dir = DMA_NONE)
+		BUG();
+	if (dma_addr >= io_tlb_start && dma_addr < io_tlb_end)
+		sync_single(hwdev, dma_addr, size, dir);
+	else if (dir = DMA_FROM_DEVICE)
+		mark_clean(dma_addr, size);
+}
+
+void
+swiotlb_sync_single_range_for_cpu(struct device *hwdev, dma_addr_t dev_addr,
+				  unsigned long offset, size_t size, int dir)
+{
+	swiotlb_sync_single_range(hwdev, dev_addr, offset, size, dir);
+}
+
+void
+swiotlb_sync_single_range_for_device(struct device *hwdev, dma_addr_t dev_addr,
+				     unsigned long offset, size_t size, int dir)
+{
+	swiotlb_sync_single_range(hwdev, dev_addr, offset, size, dir);
+}
+
+/*
  * Map a set of buffers described by scatterlist in streaming mode for DMA.
  * This is the scatter-gather version of the above swiotlb_map_single
  * interface.  Here the scatter gather list elements are each tagged with the
@@ -648,6 +679,8 @@ EXPORT_SYMBOL(swiotlb_map_sg);
 EXPORT_SYMBOL(swiotlb_unmap_sg);
 EXPORT_SYMBOL(swiotlb_sync_single_for_cpu);
 EXPORT_SYMBOL(swiotlb_sync_single_for_device);
+EXPORT_SYMBOL_GPL(swiotlb_sync_single_range_for_cpu);
+EXPORT_SYMBOL_GPL(swiotlb_sync_single_range_for_device);
 EXPORT_SYMBOL(swiotlb_sync_sg_for_cpu);
 EXPORT_SYMBOL(swiotlb_sync_sg_for_device);
 EXPORT_SYMBOL(swiotlb_dma_mapping_error);
#8 lore raw

PATCH 4/6 swiotlb: support syncing DMA_BIDIRECTIONAL mappings

The current implementation of sync_single in swiotlb.c chokes on DMA_BIDIRECTIONAL mappings. This patch adds the capability to sync those mappings, and optimizes other syncs by accounting for the sync target (i.e. cpu or device) in addition to the DMA direction of the mapping.

---

 lib/swiotlb.c |   62 +++++++++++++++++++++++++++++++++++++---------------------
 1 files changed, 40 insertions(+), 22 deletions(-)
lib/swiotlb.c +40 -22
@@ -49,6 +49,14 @@
  */
 #define IO_TLB_SHIFT 11
 
+/*
+ * Enumeration for sync targets
+ */
+enum dma_sync_target {
+	SYNC_FOR_CPU = 0,
+	SYNC_FOR_DEVICE = 1,
+};
+
 int swiotlb_force;
 
 /*
@@ -295,21 +303,28 @@ unmap_single(struct device *hwdev, char 
 }
 
 static void
-sync_single(struct device *hwdev, char *dma_addr, size_t size, int dir)
+sync_single(struct device *hwdev, char *dma_addr, size_t size,
+	    int dir, int target)
 {
 	int index = (dma_addr - io_tlb_start) >> IO_TLB_SHIFT;
 	char *buffer = io_tlb_orig_addr[index];
 
-	/*
-	 * bounce... copy the data back into/from the original buffer
-	 * XXX How do you handle DMA_BIDIRECTIONAL here ?
-	 */
-	if (dir = DMA_FROM_DEVICE)
-		memcpy(buffer, dma_addr, size);
-	else if (dir = DMA_TO_DEVICE)
-		memcpy(dma_addr, buffer, size);
-	else
+	switch (target) {
+	case SYNC_FOR_CPU:
+		if (likely(dir = DMA_FROM_DEVICE || dir = DMA_BIDIRECTIONAL))
+			memcpy(buffer, dma_addr, size);
+		else if (dir != DMA_TO_DEVICE)
+			BUG();
+		break;
+	case SYNC_FOR_DEVICE:
+		if (likely(dir = DMA_TO_DEVICE || dir = DMA_BIDIRECTIONAL))
+			memcpy(dma_addr, buffer, size);
+		else if (dir != DMA_FROM_DEVICE)
+			BUG();
+		break;
+	default:
 		BUG();
+	}
 }
 
 void *
@@ -494,14 +509,14 @@ swiotlb_unmap_single(struct device *hwde
  */
 static inline void
 swiotlb_sync_single(struct device *hwdev, dma_addr_t dev_addr,
-		    size_t size, int dir)
+		    size_t size, int dir, int target)
 {
 	char *dma_addr = phys_to_virt(dev_addr);
 
 	if (dir = DMA_NONE)
 		BUG();
 	if (dma_addr >= io_tlb_start && dma_addr < io_tlb_end)
-		sync_single(hwdev, dma_addr, size, dir);
+		sync_single(hwdev, dma_addr, size, dir, target);
 	else if (dir = DMA_FROM_DEVICE)
 		mark_clean(dma_addr, size);
 }
@@ -510,14 +525,14 @@ void
 swiotlb_sync_single_for_cpu(struct device *hwdev, dma_addr_t dev_addr,
 			    size_t size, int dir)
 {
-	swiotlb_sync_single(hwdev, dev_addr, size, dir);
+	swiotlb_sync_single(hwdev, dev_addr, size, dir, SYNC_FOR_CPU);
 }
 
 void
 swiotlb_sync_single_for_device(struct device *hwdev, dma_addr_t dev_addr,
 			       size_t size, int dir)
 {
-	swiotlb_sync_single(hwdev, dev_addr, size, dir);
+	swiotlb_sync_single(hwdev, dev_addr, size, dir, SYNC_FOR_DEVICE);
 }
 
 /*
@@ -525,14 +540,15 @@ swiotlb_sync_single_for_device(struct de
  */
 static inline void
 swiotlb_sync_single_range(struct device *hwdev, dma_addr_t dev_addr,
-			  unsigned long offset, size_t size, int dir)
+			  unsigned long offset, size_t size,
+			  int dir, int target)
 {
 	char *dma_addr = phys_to_virt(dev_addr) + offset;
 
 	if (dir = DMA_NONE)
 		BUG();
 	if (dma_addr >= io_tlb_start && dma_addr < io_tlb_end)
-		sync_single(hwdev, dma_addr, size, dir);
+		sync_single(hwdev, dma_addr, size, dir, target);
 	else if (dir = DMA_FROM_DEVICE)
 		mark_clean(dma_addr, size);
 }
@@ -541,14 +557,16 @@ void
 swiotlb_sync_single_range_for_cpu(struct device *hwdev, dma_addr_t dev_addr,
 				  unsigned long offset, size_t size, int dir)
 {
-	swiotlb_sync_single_range(hwdev, dev_addr, offset, size, dir);
+	swiotlb_sync_single_range(hwdev, dev_addr, offset, size, dir,
+				  SYNC_FOR_CPU);
 }
 
 void
 swiotlb_sync_single_range_for_device(struct device *hwdev, dma_addr_t dev_addr,
 				     unsigned long offset, size_t size, int dir)
 {
-	swiotlb_sync_single_range(hwdev, dev_addr, offset, size, dir);
+	swiotlb_sync_single_range(hwdev, dev_addr, offset, size, dir,
+				  SYNC_FOR_DEVICE);
 }
 
 /*
@@ -627,7 +645,7 @@ swiotlb_unmap_sg(struct device *hwdev, s
  */
 static inline void
 swiotlb_sync_sg(struct device *hwdev, struct scatterlist *sg,
-		int nelems, int dir)
+		int nelems, int dir, int target)
 {
 	int i;
 
@@ -637,21 +655,21 @@ swiotlb_sync_sg(struct device *hwdev, st
 	for (i = 0; i < nelems; i++, sg++)
 		if (sg->dma_address != SG_ENT_PHYS_ADDRESS(sg))
 			sync_single(hwdev, (void *) sg->dma_address,
-				    sg->dma_length, dir);
+				    sg->dma_length, dir, target);
 }
 
 void
 swiotlb_sync_sg_for_cpu(struct device *hwdev, struct scatterlist *sg,
 			int nelems, int dir)
 {
-	swiotlb_sync_sg(hwdev, sg, nelems, dir);
+	swiotlb_sync_sg(hwdev, sg, nelems, dir, SYNC_FOR_CPU);
 }
 
 void
 swiotlb_sync_sg_for_device(struct device *hwdev, struct scatterlist *sg,
 			   int nelems, int dir)
 {
-	swiotlb_sync_sg(hwdev, sg, nelems, dir);
+	swiotlb_sync_sg(hwdev, sg, nelems, dir, SYNC_FOR_DEVICE);
 }
 
 int
#9 lore raw

PATCH 5/6 swiotlb: file header comments

Change comment at top of swiotlb.c to reflect that the code is shared with EM64T (i.e. Intel x86_64). Also add an entry for myself so that if I "broke it", everyone knows who "bought it"... :-)

---

 lib/swiotlb.c |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)
lib/swiotlb.c +4 -2
@@ -1,7 +1,7 @@
 /*
  * Dynamic DMA mapping support.
  *
- * This implementation is for IA-64 platforms that do not support
+ * This implementation is for IA-64 and EM64T platforms that do not support
  * I/O TLBs (aka DMA address translation hardware).
  * Copyright (C) 2000 Asit Mallick <[email protected]>
  * Copyright (C) 2000 Goutham Rao <[email protected]>
@@ -11,7 +11,9 @@
  * 03/05/07 davidm	Switch from PCI-DMA to generic device DMA API.
  * 00/12/13 davidm	Rename to swiotlb.c and add mark_clean() to avoid
  *			unnecessary i-cache flushing.
- * 04/07/.. ak          Better overflow handling. Assorted fixes.
+ * 04/07/.. ak		Better overflow handling. Assorted fixes.
+ * 05/09/10 linville	Add support for syncing ranges, support syncing for
+ *			DMA_BIDIRECTIONAL mappings, miscellaneous cleanup.
  */
 
 #include <linux/cache.h>
#10 lore raw

PATCH 6/6 x86_64: implement dma_sync_single_range_for_{cpu,device}

Re-implement dma_sync_single_range_for_{cpu,device} for x86_64 using swiotlb_sync_single_range_for_{cpu,device}.

---

 include/asm-x86_64/dma-mapping.h |   31 +++++++++++++++++++++++++++----
 1 files changed, 27 insertions(+), 4 deletions(-)
include/asm-x86_64/dma-mapping.h +27 -4
@@ -85,10 +85,33 @@ static inline void dma_sync_single_for_d
 	flush_write_buffers();
 }
 
-#define dma_sync_single_range_for_cpu(dev, dma_handle, offset, size, dir)       \
-        dma_sync_single_for_cpu(dev, dma_handle, size, dir)
-#define dma_sync_single_range_for_device(dev, dma_handle, offset, size, dir)    \
-        dma_sync_single_for_device(dev, dma_handle, size, dir)
+static inline void dma_sync_single_range_for_cpu(struct device *hwdev,
+						 dma_addr_t dma_handle,
+						 unsigned long offset,
+						 size_t size, int direction)
+{
+	if (direction == DMA_NONE)
+		out_of_line_bug();
+
+	if (swiotlb)
+		return swiotlb_sync_single_range_for_cpu(hwdev,dma_handle,offset,size,direction);
+
+	flush_write_buffers();
+}
+
+static inline void dma_sync_single_range_for_device(struct device *hwdev,
+						    dma_addr_t dma_handle,
+						    unsigned long offset,
+						    size_t size, int direction)
+{
+        if (direction == DMA_NONE)
+		out_of_line_bug();
+
+	if (swiotlb)
+		return swiotlb_sync_single_range_for_device(hwdev,dma_handle,offset,size,direction);
+
+	flush_write_buffers();
+}
 
 static inline void dma_sync_sg_for_cpu(struct device *hwdev,
 				       struct scatterlist *sg,
author
#11 lore raw
In reply to John W. Linville
Quoted 2 lines · John W. Linville

This set does NOT include the PCI excision that Tony Luck started after the last round of discussion.

Here's my attempt at part 7of6 to complete the removal of references to PCI. This time I fixed all the comments and printk() messages. I even checked that it compiles on x86_64/em64t.

There will be some small amount of fuzz in this patch as I started applying from a base that included a patch from Alex Williamson.

Andi: earlier you had made positive noises about moving swiotlb.c up to lib/ ... now we have a specific implementation can you confirm that you are still OK with this. If you are, then I'll put this whole series into my test tree so it will get into the next -mm release.

---

lib/swiotlb.c +14 -14
@@ -17,17 +17,17 @@
  */
 
 #include <linux/cache.h>
+#include <linux/dma-mapping.h>
 #include <linux/mm.h>
 #include <linux/module.h>
-#include <linux/pci.h>
 #include <linux/spinlock.h>
 #include <linux/string.h>
 #include <linux/types.h>
 #include <linux/ctype.h>
 
 #include <asm/io.h>
-#include <asm/pci.h>
 #include <asm/dma.h>
+#include <asm/scatterlist.h>
 
 #include <linux/init.h>
 #include <linux/bootmem.h>
@@ -127,7 +127,7 @@ __setup("swiotlb=", setup_io_tlb_npages)
 
 /*
  * Statically reserve bounce buffer space and initialize bounce buffer data
- * structures for the software IO TLB used to implement the PCI DMA API.
+ * structures for the software IO TLB used to implement the DMA API.
  */
 void
 swiotlb_init_with_default_size (size_t default_size)
@@ -502,24 +502,24 @@ swiotlb_full(struct device *dev, size_t 
 	/*
 	 * Ran out of IOMMU space for this operation. This is very bad.
 	 * Unfortunately the drivers cannot handle this operation properly.
-	 * unless they check for pci_dma_mapping_error (most don't)
+	 * unless they check for dma_mapping_error (most don't)
 	 * When the mapping is small enough return a static buffer to limit
 	 * the damage, or panic when the transfer is too big.
 	 */
-	printk(KERN_ERR "PCI-DMA: Out of SW-IOMMU space for %lu bytes at "
+	printk(KERN_ERR "DMA: Out of SW-IOMMU space for %lu bytes at "
 	       "device %s\n", size, dev ? dev->bus_id : "?");
 
 	if (size > io_tlb_overflow && do_panic) {
-		if (dir = PCI_DMA_FROMDEVICE || dir = PCI_DMA_BIDIRECTIONAL)
-			panic("PCI-DMA: Memory would be corrupted\n");
-		if (dir = PCI_DMA_TODEVICE || dir = PCI_DMA_BIDIRECTIONAL)
-			panic("PCI-DMA: Random memory would be DMAed\n");
+		if (dir = DMA_FROM_DEVICE || dir = DMA_BIDIRECTIONAL)
+			panic("DMA: Memory would be corrupted\n");
+		if (dir = DMA_TO_DEVICE || dir = DMA_BIDIRECTIONAL)
+			panic("DMA: Random memory would be DMAed\n");
 	}
 }
 
 /*
  * Map a single buffer of the indicated size for DMA in streaming mode.  The
- * PCI address to use is returned.
+ * physical address to use is returned.
  *
  * Once the device is given the dma address, the device owns this memory until
  * either swiotlb_unmap_single or swiotlb_dma_sync_single is performed.
@@ -606,8 +606,8 @@ swiotlb_unmap_single(struct device *hwde
  * after a transfer.
  *
  * If you perform a swiotlb_map_single() but wish to interrogate the buffer
- * using the cpu, yet do not wish to teardown the PCI dma mapping, you must
- * call this function before doing so.  At the next point you give the PCI dma
+ * using the cpu, yet do not wish to teardown the dma mapping, you must
+ * call this function before doing so.  At the next point you give the dma
  * address back to the card, you must first perform a
  * swiotlb_dma_sync_for_device, and then the device again owns the buffer
  */
@@ -783,9 +783,9 @@ swiotlb_dma_mapping_error(dma_addr_t dma
 }
 
 /*
- * Return whether the given PCI device DMA address mask can be supported
+ * Return whether the given device DMA address mask can be supported
  * properly.  For example, if your device can only drive the low 24-bits
- * during PCI bus mastering, then you would pass 0x00ffffff as the mask to
+ * during bus mastering, then you would pass 0x00ffffff as the mask to
  * this function.
  */
 int