Skip to contentSkip to conversation

arc dma mapping cleanups/fixes and generic noncohernt dma code

PATCH New Christoph Hellwig 4 participants 10 messages May 18, 2018 – May 19, 2018 2 messages not in the archive

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

Hi all,

this series continues consolidating the dma-mapping code, with a focus on architectures that do not (always) provide cache coherence for DMA. Three architectures (arm, mips and powerpc) are still left to be converted later due to complexity of their dma ops selection.

The dma-noncoherent ops calls the dma-direct ops for the actual translation of streaming mappins and allow the architecture to provide any cache flushing required for cpu to device and/or device to cpu ownership transfers. The dma coherent allocator is for now still left entirely to architecture supplied implementations due the amount of variations. Hopefully we can do some consolidation for them later on as well.

Because this series sits on top of two previously submitted series a git tree might be useful to actually test it. It is provided here:

git://git.infradead.org/users/hch/misc.git arc-dma

Gitweb:

http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/arc-dma

Changes since previous bigger series:
- take the offset into account in noncoherent_dma_map_page (Alexey Brodkin)
- fix dma directions in arc (Alexey Brodkin)
- split arc changes into smaller patches

#1 lore raw

PATCH 1/6 dma-mapping: simplify Kconfig dependencies

ARCH_DMA_ADDR_T_64BIT is always true for 64-bit architectures now, so we can skip the clause requiring it. 'n' is the default default, so no need to explicitly state it.

---
 lib/Kconfig | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)
lib/Kconfig +2 -4
index 1d84e61cccfe..6c4e9d0ce5d1 100644
@@ -443,13 +443,11 @@ config IOMMU_HELPER
 
 config DMA_DIRECT_OPS
 	bool
-	depends on HAS_DMA && (!64BIT || ARCH_DMA_ADDR_T_64BIT)
-	default n
+	depends on HAS_DMA
 
 config DMA_VIRT_OPS
 	bool
-	depends on HAS_DMA && (!64BIT || ARCH_DMA_ADDR_T_64BIT)
-	default n
+	depends on HAS_DMA
 
 config SWIOTLB
 	bool
Signature · 1 line
2.17.0
#2 lore raw

PATCH 2/6 dma-mapping: provide a generic dma-noncoherent implementation

In reply to Christoph Hellwig

Add a new dma_map_ops implementation that uses dma-direct for the address mapping of streaming mappings, and which requires arch-specific implemenations of coherent allocate/free.

Architectures have to provide flushing helpers to ownership trasnfers to the device and/or CPU, and can provide optional implementations of the coherent mmap functionality, and the cache_flush routines for non-coherent long term allocations.

---
 MAINTAINERS                       |   2 +
 include/asm-generic/dma-mapping.h |   9 +++
 include/linux/dma-direct.h        |   7 +-
 include/linux/dma-mapping.h       |   1 +
 include/linux/dma-noncoherent.h   |  47 ++++++++++++++
 lib/Kconfig                       |  20 ++++++
 lib/Makefile                      |   1 +
 lib/dma-direct.c                  |   8 +--
 lib/dma-noncoherent.c             | 102 ++++++++++++++++++++++++++++++
 9 files changed, 192 insertions(+), 5 deletions(-)
 create mode 100644 include/linux/dma-noncoherent.h
 create mode 100644 lib/dma-noncoherent.c
MAINTAINERS +2 -0
index 79bb02ff812f..08d0d15d4958 100644
@@ -4334,12 +4334,14 @@ W:	http://git.infradead.org/users/hch/dma-mapping.git
 S:	Supported
 F:	lib/dma-debug.c
 F:	lib/dma-direct.c
+F:	lib/dma-noncoherent.c
 F:	lib/dma-virt.c
 F:	drivers/base/dma-mapping.c
 F:	drivers/base/dma-coherent.c
 F:	include/asm-generic/dma-mapping.h
 F:	include/linux/dma-direct.h
 F:	include/linux/dma-mapping.h
+F:	include/linux/dma-noncoherent.h
 
 DME1737 HARDWARE MONITOR DRIVER
 M:	Juerg Haefliger <[email protected]>
include/asm-generic/dma-mapping.h +9 -0
index 880a292d792f..ad2868263867 100644
@@ -4,7 +4,16 @@
 
 static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
 {
+	/*
+	 * Use the non-coherent ops if available.  If an architecture wants a
+	 * more fine-grained selection of operations it will have to implement
+	 * get_arch_dma_ops itself or use the per-device dma_ops.
+	 */
+#ifdef CONFIG_DMA_NONCOHERENT_OPS
+	return &dma_noncoherent_ops;
+#else
 	return &dma_direct_ops;
+#endif
 }
 
 #endif /* _ASM_GENERIC_DMA_MAPPING_H */
include/linux/dma-direct.h +6 -1
index 53ad6a47f513..8d9f33febde5 100644
@@ -59,6 +59,11 @@ void *dma_direct_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle,
 		gfp_t gfp, unsigned long attrs);
 void dma_direct_free(struct device *dev, size_t size, void *cpu_addr,
 		dma_addr_t dma_addr, unsigned long attrs);
+dma_addr_t dma_direct_map_page(struct device *dev, struct page *page,
+		unsigned long offset, size_t size, enum dma_data_direction dir,
+		unsigned long attrs);
+int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl, int nents,
+		enum dma_data_direction dir, unsigned long attrs);
 int dma_direct_supported(struct device *dev, u64 mask);
-
+int dma_direct_mapping_error(struct device *dev, dma_addr_t dma_addr);
 #endif /* _LINUX_DMA_DIRECT_H */
include/linux/dma-mapping.h +1 -0
index 25a9a2b04f78..4be070df5fc5 100644
@@ -136,6 +136,7 @@ struct dma_map_ops {
 };
 
 extern const struct dma_map_ops dma_direct_ops;
+extern const struct dma_map_ops dma_noncoherent_ops;
 extern const struct dma_map_ops dma_virt_ops;
 
 #define DMA_BIT_MASK(n)	(((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
include/linux/dma-noncoherent.h new file +47 -0

47 lines not on this page · raw message

lib/Kconfig +20 -0
index 6c4e9d0ce5d1..7a913937888b 100644
@@ -441,10 +441,30 @@ config ARCH_DMA_ADDR_T_64BIT
 config IOMMU_HELPER
 	bool
 
+config ARCH_HAS_SYNC_DMA_FOR_DEVICE
+	bool
+
+config ARCH_HAS_SYNC_DMA_FOR_CPU
+	bool
+	select NEED_DMA_MAP_STATE
+
 config DMA_DIRECT_OPS
 	bool
 	depends on HAS_DMA
 
+config DMA_NONCOHERENT_OPS
+	bool
+	depends on HAS_DMA
+	select DMA_DIRECT_OPS
+
+config DMA_NONCOHERENT_MMAP
+	bool
+	depends on DMA_NONCOHERENT_OPS
+
+config DMA_NONCOHERENT_CACHE_SYNC
+	bool
+	depends on DMA_NONCOHERENT_OPS
+
 config DMA_VIRT_OPS
 	bool
 	depends on HAS_DMA
lib/Makefile +1 -0
index 94203b5eecd4..9f18c8152281 100644
@@ -30,6 +30,7 @@ lib-$(CONFIG_PRINTK) += dump_stack.o
 lib-$(CONFIG_MMU) += ioremap.o
 lib-$(CONFIG_SMP) += cpumask.o
 lib-$(CONFIG_DMA_DIRECT_OPS) += dma-direct.o
+lib-$(CONFIG_DMA_NONCOHERENT_OPS) += dma-noncoherent.o
 lib-$(CONFIG_DMA_VIRT_OPS) += dma-virt.o
 
 lib-y	+= kobject.o klist.o
lib/dma-direct.c +4 -4
index df9e726e0712..b824eb218782 100644
@@ -128,7 +128,7 @@ void dma_direct_free(struct device *dev, size_t size, void *cpu_addr,
 		free_pages((unsigned long)cpu_addr, page_order);
 }
 
-static dma_addr_t dma_direct_map_page(struct device *dev, struct page *page,
+dma_addr_t dma_direct_map_page(struct device *dev, struct page *page,
 		unsigned long offset, size_t size, enum dma_data_direction dir,
 		unsigned long attrs)
 {
@@ -139,8 +139,8 @@ static dma_addr_t dma_direct_map_page(struct device *dev, struct page *page,
 	return dma_addr;
 }
 
-static int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl,
-		int nents, enum dma_data_direction dir, unsigned long attrs)
+int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl, int nents,
+		enum dma_data_direction dir, unsigned long attrs)
 {
 	int i;
 	struct scatterlist *sg;
@@ -175,7 +175,7 @@ int dma_direct_supported(struct device *dev, u64 mask)
 	return 1;
 }
 
-static int dma_direct_mapping_error(struct device *dev, dma_addr_t dma_addr)
+int dma_direct_mapping_error(struct device *dev, dma_addr_t dma_addr)
 {
 	return dma_addr == DIRECT_MAPPING_ERROR;
 }
lib/dma-noncoherent.c new file +102 -0

102 lines not on this page · raw message

Signature · 1 line
2.17.0
#3 lore raw

PATCH 3/6 arc: simplify arc_dma_sync_single_for_{cpu,device}

In reply to Christoph Hellwig

Remove the indirection through _dma_cache_sync. Also move the functions up a bit in the source file as we'll need them in more places soon.

---
 arch/arc/mm/dma.c | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)
arch/arc/mm/dma.c +12 -12
index 1dcc404b5aec..98e21ce526be 100644
@@ -153,6 +153,18 @@ static void _dma_cache_sync(phys_addr_t paddr, size_t size,
 	}
 }
 
+static void arc_dma_sync_single_for_device(struct device *dev,
+		dma_addr_t dma_handle, size_t size, enum dma_data_direction dir)
+{
+	dma_cache_wback(dma_handle, size);
+}
+
+static void arc_dma_sync_single_for_cpu(struct device *dev,
+		dma_addr_t dma_handle, size_t size, enum dma_data_direction dir)
+{
+	dma_cache_inv(dma_handle, size);
+}
+
 /*
  * arc_dma_map_page - map a portion of a page for streaming DMA
  *
@@ -221,18 +233,6 @@ static void arc_dma_unmap_sg(struct device *dev, struct scatterlist *sg,
 				   attrs);
 }
 
-static void arc_dma_sync_single_for_cpu(struct device *dev,
-		dma_addr_t dma_handle, size_t size, enum dma_data_direction dir)
-{
-	_dma_cache_sync(dma_handle, size, DMA_FROM_DEVICE);
-}
-
-static void arc_dma_sync_single_for_device(struct device *dev,
-		dma_addr_t dma_handle, size_t size, enum dma_data_direction dir)
-{
-	_dma_cache_sync(dma_handle, size, DMA_TO_DEVICE);
-}
-
 static void arc_dma_sync_sg_for_cpu(struct device *dev,
 		struct scatterlist *sglist, int nelems,
 		enum dma_data_direction dir)
Signature · 1 line
2.17.0
#4 lore raw

PATCH 4/6 arc: fix arc_dma_sync_sg_for_{cpu,device}

In reply to Christoph Hellwig

These functions should perform the same functionality as calling arc_dma_sync_single_for_{cpu,device} on each S/G list element. Ensure they actually do that by calling arc_dma_sync_single_for_{cpu,device}. Otherwise we could be passing a different dir argument.

---
 arch/arc/mm/dma.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
arch/arc/mm/dma.c +3 -2
index 98e21ce526be..eafdbd2ad20a 100644
@@ -241,7 +241,7 @@ static void arc_dma_sync_sg_for_cpu(struct device *dev,
 	struct scatterlist *sg;
 
 	for_each_sg(sglist, sg, nelems, i)
-		_dma_cache_sync(sg_phys(sg), sg->length, dir);
+		arc_dma_sync_single_for_cpu(dev, sg_phys(sg), sg->length, dir);
 }
 
 static void arc_dma_sync_sg_for_device(struct device *dev,
@@ -252,7 +252,8 @@ static void arc_dma_sync_sg_for_device(struct device *dev,
 	struct scatterlist *sg;
 
 	for_each_sg(sglist, sg, nelems, i)
-		_dma_cache_sync(sg_phys(sg), sg->length, dir);
+		arc_dma_sync_single_for_device(dev, sg_phys(sg), sg->length,
+				dir);
 }
 
 static int arc_dma_supported(struct device *dev, u64 dma_mask)
Signature · 1 line
2.17.0
#5 lore raw

PATCH 5/6 arc: fix arc_dma_{map,unmap}_page

In reply to Christoph Hellwig

These functions should perform the same cache synchronoization as calling arc_dma_sync_single_for_{cpu,device} in addition to doing any required address translation or mapping [1]. Ensure they actually do that by calling arc_dma_sync_single_for_{cpu,device} instead of passing the dir argument along to _dma_cache_sync.

The now unused _dma_cache_sync function is removed as well.

[1] in fact various drivers rely on that by passing DMA_ATTR_SKIP_CPU_SYNC to the map/unmap routines and doing the cache synchronization manually.

---
 arch/arc/mm/dma.c | 27 ++-------------------------
 1 file changed, 2 insertions(+), 25 deletions(-)
arch/arc/mm/dma.c +2 -25
index eafdbd2ad20a..08d91c13ac52 100644
@@ -130,29 +130,6 @@ static int arc_dma_mmap(struct device *dev, struct vm_area_struct *vma,
 	return ret;
 }
 
-/*
- * streaming DMA Mapping API...
- * CPU accesses page via normal paddr, thus needs to explicitly made
- * consistent before each use
- */
-static void _dma_cache_sync(phys_addr_t paddr, size_t size,
-		enum dma_data_direction dir)
-{
-	switch (dir) {
-	case DMA_FROM_DEVICE:
-		dma_cache_inv(paddr, size);
-		break;
-	case DMA_TO_DEVICE:
-		dma_cache_wback(paddr, size);
-		break;
-	case DMA_BIDIRECTIONAL:
-		dma_cache_wback_inv(paddr, size);
-		break;
-	default:
-		pr_err("Invalid DMA dir [%d] for OP @ %pa[p]\n", dir, &paddr);
-	}
-}
-
 static void arc_dma_sync_single_for_device(struct device *dev,
 		dma_addr_t dma_handle, size_t size, enum dma_data_direction dir)
 {
@@ -185,7 +162,7 @@ static dma_addr_t arc_dma_map_page(struct device *dev, struct page *page,
 	phys_addr_t paddr = page_to_phys(page) + offset;
 
 	if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
-		_dma_cache_sync(paddr, size, dir);
+		arc_dma_sync_single_for_device(dev, paddr, size, dir);
 
 	return paddr;
 }
@@ -205,7 +182,7 @@ static void arc_dma_unmap_page(struct device *dev, dma_addr_t handle,
 	phys_addr_t paddr = handle;
 
 	if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
-		_dma_cache_sync(paddr, size, dir);
+		arc_dma_sync_single_for_cpu(dev, paddr, size, dir);
 }
 
 static int arc_dma_map_sg(struct device *dev, struct scatterlist *sg,
Signature · 1 line
2.17.0
#6 lore raw

PATCH 6/6 arc: use generic dma_noncoherent_ops

In reply to Christoph Hellwig

Switch to the generic noncoherent direct mapping implementation.

---
 arch/arc/Kconfig                   |   4 +
 arch/arc/include/asm/Kbuild        |   1 +
 arch/arc/include/asm/dma-mapping.h |  21 -----
 arch/arc/mm/dma.c                  | 140 +++--------------------------
 4 files changed, 18 insertions(+), 148 deletions(-)
 delete mode 100644 arch/arc/include/asm/dma-mapping.h
arch/arc/Kconfig +4 -0
index 7498aca4b887..89d47eac18b2 100644
@@ -9,11 +9,15 @@
 config ARC
 	def_bool y
 	select ARC_TIMERS
+	select ARCH_HAS_SYNC_DMA_FOR_CPU
+	select ARCH_HAS_SYNC_DMA_FOR_DEVICE
 	select ARCH_HAS_SG_CHAIN
 	select ARCH_SUPPORTS_ATOMIC_RMW if ARC_HAS_LLSC
 	select BUILDTIME_EXTABLE_SORT
 	select CLONE_BACKWARDS
 	select COMMON_CLK
+	select DMA_NONCOHERENT_OPS
+	select DMA_NONCOHERENT_MMAP
 	select GENERIC_ATOMIC64 if !ISA_ARCV2 || !(ARC_HAS_LL64 && ARC_HAS_LLSC)
 	select GENERIC_CLOCKEVENTS
 	select GENERIC_FIND_FIRST_BIT
arch/arc/include/asm/Kbuild +1 -0
index 4bd5d4369e05..bbdcb955e18f 100644
@@ -2,6 +2,7 @@
 generic-y += bugs.h
 generic-y += device.h
 generic-y += div64.h
+generic-y += dma-mapping.h
 generic-y += emergency-restart.h
 generic-y += extable.h
 generic-y += fb.h
arch/arc/include/asm/dma-mapping.h deleted +0 -21
deleted file mode 100644
index 7a16824bfe98..000000000000
@@ -1,21 +0,0 @@
-/*
- * DMA Mapping glue for ARC
- *
- * Copyright (C) 2004, 2007-2010, 2011-2012 Synopsys, Inc. (www.synopsys.com)
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-
-#ifndef ASM_ARC_DMA_MAPPING_H
-#define ASM_ARC_DMA_MAPPING_H
-
-extern const struct dma_map_ops arc_dma_ops;
-
-static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
-{
-	return &arc_dma_ops;
-}
-
-#endif
arch/arc/mm/dma.c +13 -127

169 lines not on this page · raw message

Signature · 1 line
2.17.0

A message that is not in the archive (sent off-list or not yet received): [email protected]

Tested-by
#8 lore raw

Hi Christoph,

Quoted 22 lines · Christoph Hellwig

Hi all,

this series continues consolidating the dma-mapping code, with a focus on architectures that do not (always) provide cache coherence for DMA. Three architectures (arm, mips and powerpc) are still left to be converted later due to complexity of their dma ops selection.

The dma-noncoherent ops calls the dma-direct ops for the actual translation of streaming mappins and allow the architecture to provide any cache flushing required for cpu to device and/or device to cpu ownership transfers. The dma coherent allocator is for now still left entirely to architecture supplied implementations due the amount of variations. Hopefully we can do some consolidation for them later on as well.

Because this series sits on top of two previously submitted series a git tree might be useful to actually test it. It is provided here:

git://git.infradead.org/users/hch/misc.git arc-dma

Gitweb:

Quoted 8 lines

https://urldefense.proofpoint.com/v2/url?u=http-3A__git.infradead.org_users_hch_misc.git_shortlog_refs_heads_arc-2Ddma&d=DwIBAg&c=DPL6_X_6JkXFx7
AXWqB0tg&r=lqdeeSSEes0GFDDl656eViXO7breS55ytWkhpk5R81I&m=QNC0zOG4bRotSxdjQfFXl0dAQJCtnxtF9435wAuJlu0&s=Wad1ptsFpknoSn0ebr3_QMJn9G_D3eIS-LYMwGfOJ- I&e=

Changes since previous bigger series:
- take the offset into account in noncoherent_dma_map_page (Alexey Brodkin)
- fix dma directions in arc (Alexey Brodkin)
- split arc changes into smaller patches

I really like this series especially how it trimmed down arch/arc/mm/dma.c:
---------------------------->8----------------------------
git diff --stat HEAD~4

 arch/arc/Kconfig                   |   4 +++
 arch/arc/include/asm/Kbuild        |   1 +
 arch/arc/include/asm/dma-mapping.h |  21 ---------------
 arch/arc/mm/cache.c                |   2 +-
 arch/arc/mm/dma.c                  | 162 ++++++++++------------------------------------------------------------------------------------------------

---------
---------------------------->8----------------------------

So if we set aside my complaints about direction in arch_sync_dma_for_{device|cpu}()...

Let's still wait for Vineet's ack as he's the chief maintainer for ARC :)

-Alexey

A message that is not in the archive (sent off-list or not yet received): dfa0f4a194553f9bdd7bf4d56ac75d91cedc2609.camel-HKixBCOQz3hWk0Htik3J/[email protected]

#10 lore raw
Quoted 1 line · Alexey Brodkin

So if we set aside my complaints about direction in arch_sync_dma_for_{device|cpu}()...

Many other architectures use the argument. Various of those look bogus, but for now I want to be able to do straight forward conversions. If we end up not needing the argument in the end we can still remove it eventually.

Acked-by
#11 lore raw
In reply to a message not in the archive
Quoted 20 lines · Christoph Hellwig

Hi all,

this series continues consolidating the dma-mapping code, with a focus on architectures that do not (always) provide cache coherence for DMA. Three architectures (arm, mips and powerpc) are still left to be converted later due to complexity of their dma ops selection.

The dma-noncoherent ops calls the dma-direct ops for the actual translation of streaming mappins and allow the architecture to provide any cache flushing required for cpu to device and/or device to cpu ownership transfers. The dma coherent allocator is for now still left entirely to architecture supplied implementations due the amount of variations. Hopefully we can do some consolidation for them later on as well.

Because this series sits on top of two previously submitted series a git tree might be useful to actually test it. It is provided here:

git://git.infradead.org/users/hch/misc.git arc-dma

Quoted 9 lines

Gitweb:

https://urldefense.proofpoint.com/v2/url?u=http-3A__git.infradead.org_users_hch_misc.git_shortlog_refs_heads_arc-2Ddma&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=7FgpX6o3vAhwMrMhLh-4ZJey5kjdNUwOL2CWsFwR4T8&m=9mEjcnTLzOrHy9MqruvxfxgCZQ1c0Bs4EQ8F7G-F4M8&s=TV7HWlNcSsk79JReNvLa5u1vsIkJxrk_s6RsL5CMNW4&e=

Changes since previous bigger series:
- take the offset into account in noncoherent_dma_map_page (Alexey Brodkin)
- fix dma directions in arc (Alexey Brodkin)
- split arc changes into smaller patches

LGTM !

Thx,
-Vineet