arc dma mapping cleanups/fixes and generic noncohernt dma code
Subsystems (4)
Hi all,
this series continues consolidating the dma-mapping code, with a focus on architectures that do not (always) provide cache coherence for DMA. Three architectures (arm, mips and powerpc) are still left to be converted later due to complexity of their dma ops selection.
The dma-noncoherent ops calls the dma-direct ops for the actual translation of streaming mappins and allow the architecture to provide any cache flushing required for cpu to device and/or device to cpu ownership transfers. The dma coherent allocator is for now still left entirely to architecture supplied implementations due the amount of variations. Hopefully we can do some consolidation for them later on as well.
Because this series sits on top of two previously submitted series a git tree might be useful to actually test it. It is provided here:
git://git.infradead.org/users/hch/misc.git arc-dma
Gitweb:
http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/arc-dma
Changes since previous bigger series:
- take the offset into account in noncoherent_dma_map_page (Alexey Brodkin)
- fix dma directions in arc (Alexey Brodkin)
- split arc changes into smaller patches
PATCH 1/6 dma-mapping: simplify Kconfig dependencies
ARCH_DMA_ADDR_T_64BIT is always true for 64-bit architectures now, so we can skip the clause requiring it. 'n' is the default default, so no need to explicitly state it.
--- lib/Kconfig | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
lib/Kconfig +2 -4
@@ -443,13 +443,11 @@ config IOMMU_HELPER config DMA_DIRECT_OPS bool - depends on HAS_DMA && (!64BIT || ARCH_DMA_ADDR_T_64BIT) - default n + depends on HAS_DMA config DMA_VIRT_OPS bool - depends on HAS_DMA && (!64BIT || ARCH_DMA_ADDR_T_64BIT) - default n + depends on HAS_DMA config SWIOTLB bool
Signature · 1 line
2.17.0
PATCH 2/6 dma-mapping: provide a generic dma-noncoherent implementation
In reply to Christoph HellwigAdd a new dma_map_ops implementation that uses dma-direct for the address mapping of streaming mappings, and which requires arch-specific implemenations of coherent allocate/free.
Architectures have to provide flushing helpers to ownership trasnfers to the device and/or CPU, and can provide optional implementations of the coherent mmap functionality, and the cache_flush routines for non-coherent long term allocations.
--- MAINTAINERS | 2 + include/asm-generic/dma-mapping.h | 9 +++ include/linux/dma-direct.h | 7 +- include/linux/dma-mapping.h | 1 + include/linux/dma-noncoherent.h | 47 ++++++++++++++ lib/Kconfig | 20 ++++++ lib/Makefile | 1 + lib/dma-direct.c | 8 +-- lib/dma-noncoherent.c | 102 ++++++++++++++++++++++++++++++ 9 files changed, 192 insertions(+), 5 deletions(-) create mode 100644 include/linux/dma-noncoherent.h create mode 100644 lib/dma-noncoherent.c
MAINTAINERS +2 -0
@@ -4334,12 +4334,14 @@ W: http://git.infradead.org/users/hch/dma-mapping.git S: Supported F: lib/dma-debug.c F: lib/dma-direct.c +F: lib/dma-noncoherent.c F: lib/dma-virt.c F: drivers/base/dma-mapping.c F: drivers/base/dma-coherent.c F: include/asm-generic/dma-mapping.h F: include/linux/dma-direct.h F: include/linux/dma-mapping.h +F: include/linux/dma-noncoherent.h DME1737 HARDWARE MONITOR DRIVER M: Juerg Haefliger <[email protected]>
include/asm-generic/dma-mapping.h +9 -0
@@ -4,7 +4,16 @@static inline const struct dma_map_ops *get_arch_dma_ops (struct bus_type *bus) { +/* +* Use the non-coherent ops if available. If an architecture wants a +* more fine-grained selection of operations it will have to implement +* get_arch_dma_ops itself or use the per-device dma_ops. +*/ +#ifdef CONFIG_DMA_NONCOHERENT_OPS +return &dma_noncoherent_ops; +#else return &dma_direct_ops; +#endif }#endif /* _ASM_GENERIC_DMA_MAPPING_H */
include/linux/dma-direct.h +6 -1
@@ -59,6 +59,11 @@ void *dma_direct_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle, gfp_t gfp, unsigned long attrs);void dma_direct_free (struct device *dev,size_t size,void *cpu_addr,dma_addr_t dma_addr,unsigned long attrs); +dma_addr_t dma_direct_map_page (struct device *dev,struct page *page, +unsigned long offset,size_t size,enum dma_data_direction dir, +unsigned long attrs); +int dma_direct_map_sg (struct device *dev,struct scatterlist *sgl,int nents, +enum dma_data_direction dir,unsigned long attrs);int dma_direct_supported (struct device *dev,u64 mask); - +int dma_direct_mapping_error (struct device *dev,dma_addr_t dma_addr);#endif /* _LINUX_DMA_DIRECT_H */
include/linux/dma-mapping.h +1 -0
@@ -136,6 +136,7 @@ struct dma_map_ops { };extern const struct dma_map_ops dma_direct_ops; +extern const struct dma_map_ops dma_noncoherent_ops;extern const struct dma_map_ops dma_virt_ops;#define DMA_BIT_MASK (n) (((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
include/linux/dma-noncoherent.h new file +47 -0
47 lines not on this page · raw message
lib/Kconfig +20 -0
@@ -441,10 +441,30 @@ config ARCH_DMA_ADDR_T_64BIT config IOMMU_HELPER bool +config ARCH_HAS_SYNC_DMA_FOR_DEVICE + bool + +config ARCH_HAS_SYNC_DMA_FOR_CPU + bool + select NEED_DMA_MAP_STATE + config DMA_DIRECT_OPS bool depends on HAS_DMA +config DMA_NONCOHERENT_OPS + bool + depends on HAS_DMA + select DMA_DIRECT_OPS + +config DMA_NONCOHERENT_MMAP + bool + depends on DMA_NONCOHERENT_OPS + +config DMA_NONCOHERENT_CACHE_SYNC + bool + depends on DMA_NONCOHERENT_OPS + config DMA_VIRT_OPS bool depends on HAS_DMA
lib/Makefile +1 -0
@@ -30,6 +30,7 @@ lib-$(CONFIG_PRINTK) += dump_stack.o lib-$(CONFIG_MMU) += ioremap.o lib-$(CONFIG_SMP) += cpumask.o lib-$(CONFIG_DMA_DIRECT_OPS) += dma-direct.o +lib-$(CONFIG_DMA_NONCOHERENT_OPS) += dma-noncoherent.o lib-$(CONFIG_DMA_VIRT_OPS) += dma-virt.o lib-y += kobject.o klist.o
lib/dma-direct.c +4 -4
@@ -128,7 +128,7 @@ void dma_direct_free(struct device *dev, size_t size, void *cpu_addr,free_pages ((unsigned long )cpu_addr, page_order); } -static dma_addr_t dma_direct_map_page (struct device *dev,struct page *page, +dma_addr_t dma_direct_map_page (struct device *dev,struct page *page,unsigned long offset,size_t size,enum dma_data_direction dir,unsigned long attrs) {
@@ -139,8 +139,8 @@ static dma_addr_t dma_direct_map_page(struct device *dev, struct page *page,return dma_addr; } -static int dma_direct_map_sg (struct device *dev,struct scatterlist *sgl, -int nents,enum dma_data_direction dir,unsigned long attrs) +int dma_direct_map_sg (struct device *dev,struct scatterlist *sgl,int nents, +enum dma_data_direction dir,unsigned long attrs) {int i;struct scatterlist *sg;
@@ -175,7 +175,7 @@ int dma_direct_supported(struct device *dev, u64 mask)return 1 ; } -static int dma_direct_mapping_error (struct device *dev,dma_addr_t dma_addr) +int dma_direct_mapping_error (struct device *dev,dma_addr_t dma_addr) {return dma_addr ==DIRECT_MAPPING_ERROR ; }
lib/dma-noncoherent.c new file +102 -0
102 lines not on this page · raw message
Signature · 1 line
2.17.0
PATCH 3/6 arc: simplify arc_dma_sync_single_for_{cpu,device}
In reply to Christoph HellwigRemove the indirection through _dma_cache_sync. Also move the functions up a bit in the source file as we'll need them in more places soon.
--- arch/arc/mm/dma.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-)
arch/arc/mm/dma.c +12 -12
@@ -153,6 +153,18 @@ static void _dma_cache_sync(phys_addr_t paddr, size_t size, } } +static void arc_dma_sync_single_for_device (struct device *dev, +dma_addr_t dma_handle,size_t size,enum dma_data_direction dir) +{ +dma_cache_wback (dma_handle, size); +} + +static void arc_dma_sync_single_for_cpu (struct device *dev, +dma_addr_t dma_handle,size_t size,enum dma_data_direction dir) +{ +dma_cache_inv (dma_handle, size); +} +/* * arc_dma_map_page - map a portion of a page for streaming DMA *
@@ -221,18 +233,6 @@ static void arc_dma_unmap_sg(struct device *dev, struct scatterlist *sg, attrs); } -static void arc_dma_sync_single_for_cpu (struct device *dev, -dma_addr_t dma_handle,size_t size,enum dma_data_direction dir) -{ -_dma_cache_sync (dma_handle, size,DMA_FROM_DEVICE ); -} - -static void arc_dma_sync_single_for_device (struct device *dev, -dma_addr_t dma_handle,size_t size,enum dma_data_direction dir) -{ -_dma_cache_sync (dma_handle, size,DMA_TO_DEVICE ); -} -static void arc_dma_sync_sg_for_cpu (struct device *dev,struct scatterlist *sglist,int nelems,enum dma_data_direction dir)
Signature · 1 line
2.17.0
PATCH 4/6 arc: fix arc_dma_sync_sg_for_{cpu,device}
In reply to Christoph HellwigThese functions should perform the same functionality as calling arc_dma_sync_single_for_{cpu,device} on each S/G list element. Ensure they actually do that by calling arc_dma_sync_single_for_{cpu,device}. Otherwise we could be passing a different dir argument.
--- arch/arc/mm/dma.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
arch/arc/mm/dma.c +3 -2
@@ -241,7 +241,7 @@ static void arc_dma_sync_sg_for_cpu(struct device *dev,struct scatterlist *sg;for_each_sg (sglist, sg, nelems, i) -_dma_cache_sync (sg_phys (sg), sg->length, dir); +arc_dma_sync_single_for_cpu (dev,sg_phys (sg), sg->length, dir); } static void arc_dma_sync_sg_for_device(struct device *dev,
@@ -252,7 +252,8 @@ static void arc_dma_sync_sg_for_device(struct device *dev,struct scatterlist *sg;for_each_sg (sglist, sg, nelems, i) -_dma_cache_sync (sg_phys (sg), sg->length, dir); +arc_dma_sync_single_for_device (dev,sg_phys (sg), sg->length, + dir); }static int arc_dma_supported (struct device *dev,u64 dma_mask)
Signature · 1 line
2.17.0
PATCH 5/6 arc: fix arc_dma_{map,unmap}_page
In reply to Christoph HellwigThese functions should perform the same cache synchronoization as calling arc_dma_sync_single_for_{cpu,device} in addition to doing any required address translation or mapping [1]. Ensure they actually do that by calling arc_dma_sync_single_for_{cpu,device} instead of passing the dir argument along to _dma_cache_sync.
The now unused _dma_cache_sync function is removed as well.
[1] in fact various drivers rely on that by passing DMA_ATTR_SKIP_CPU_SYNC to the map/unmap routines and doing the cache synchronization manually.
--- arch/arc/mm/dma.c | 27 ++------------------------- 1 file changed, 2 insertions(+), 25 deletions(-)
arch/arc/mm/dma.c +2 -25
@@ -130,29 +130,6 @@ static int arc_dma_mmap(struct device *dev, struct vm_area_struct *vma,return ret; } -/* -* streaming DMA Mapping API... -* CPU accesses page via normal paddr, thus needs to explicitly made -* consistent before each use -*/ -static void _dma_cache_sync (phys_addr_t paddr,size_t size, -enum dma_data_direction dir) -{ -switch (dir) { -case DMA_FROM_DEVICE : -dma_cache_inv (paddr, size); -break ; -case DMA_TO_DEVICE : -dma_cache_wback (paddr, size); -break ; -case DMA_BIDIRECTIONAL : -dma_cache_wback_inv (paddr, size); -break ; -default : -pr_err ("Invalid DMA dir [%d] for OP @ %pa[p]\n" , dir, &paddr); - } -} -static void arc_dma_sync_single_for_device (struct device *dev,dma_addr_t dma_handle,size_t size,enum dma_data_direction dir) {
@@ -185,7 +162,7 @@ static dma_addr_t arc_dma_map_page(struct device *dev, struct page *page,phys_addr_t paddr =page_to_phys (page) + offset;if (!(attrs &DMA_ATTR_SKIP_CPU_SYNC )) -_dma_cache_sync (paddr, size, dir); +arc_dma_sync_single_for_device (dev, paddr, size, dir);return paddr; }
@@ -205,7 +182,7 @@ static void arc_dma_unmap_page(struct device *dev, dma_addr_t handle,phys_addr_t paddr = handle;if (!(attrs &DMA_ATTR_SKIP_CPU_SYNC )) -_dma_cache_sync (paddr, size, dir); +arc_dma_sync_single_for_cpu (dev, paddr, size, dir); } static int arc_dma_map_sg(struct device *dev, struct scatterlist *sg,
Signature · 1 line
2.17.0
PATCH 6/6 arc: use generic dma_noncoherent_ops
In reply to Christoph HellwigSwitch to the generic noncoherent direct mapping implementation.
--- arch/arc/Kconfig | 4 + arch/arc/include/asm/Kbuild | 1 + arch/arc/include/asm/dma-mapping.h | 21 ----- arch/arc/mm/dma.c | 140 +++-------------------------- 4 files changed, 18 insertions(+), 148 deletions(-) delete mode 100644 arch/arc/include/asm/dma-mapping.h
arch/arc/Kconfig +4 -0
@@ -9,11 +9,15 @@ config ARC def_bool y select ARC_TIMERS + select ARCH_HAS_SYNC_DMA_FOR_CPU + select ARCH_HAS_SYNC_DMA_FOR_DEVICE select ARCH_HAS_SG_CHAIN select ARCH_SUPPORTS_ATOMIC_RMW if ARC_HAS_LLSC select BUILDTIME_EXTABLE_SORT select CLONE_BACKWARDS select COMMON_CLK + select DMA_NONCOHERENT_OPS + select DMA_NONCOHERENT_MMAP select GENERIC_ATOMIC64 if !ISA_ARCV2 || !(ARC_HAS_LL64 && ARC_HAS_LLSC) select GENERIC_CLOCKEVENTS select GENERIC_FIND_FIRST_BIT
arch/arc/include/asm/Kbuild +1 -0
@@ -2,6 +2,7 @@ generic-y += bugs.h generic-y += device.h generic-y += div64.h +generic-y += dma-mapping.h generic-y += emergency-restart.h generic-y += extable.h generic-y += fb.h
arch/arc/include/asm/dma-mapping.h deleted +0 -21
@@ -1,21 +0,0 @@ -/* -* DMA Mapping glue for ARC -* -* Copyright (C) 2004, 2007-2010, 2011-2012 Synopsys, Inc. (www.synopsys.com) -* -* This program is free software; you can redistribute it and/or modify -* it under the terms of the GNU General Public License version 2 as -* published by the Free Software Foundation. -*/ - -#ifndef ASM_ARC_DMA_MAPPING_H -#define ASM_ARC_DMA_MAPPING_H - -extern const struct dma_map_ops arc_dma_ops; - -static inline const struct dma_map_ops *get_arch_dma_ops (struct bus_type *bus) -{ -return &arc_dma_ops; -} - -#endif
arch/arc/mm/dma.c +13 -127
169 lines not on this page · raw message
Signature · 1 line
2.17.0
A message that is not in the archive (sent off-list or not yet received): [email protected]
Alexey Brodkin
Tested-byHi Christoph,
Quoted 22 lines · Christoph Hellwig
Hi all,
this series continues consolidating the dma-mapping code, with a focus on architectures that do not (always) provide cache coherence for DMA. Three architectures (arm, mips and powerpc) are still left to be converted later due to complexity of their dma ops selection.
The dma-noncoherent ops calls the dma-direct ops for the actual translation of streaming mappins and allow the architecture to provide any cache flushing required for cpu to device and/or device to cpu ownership transfers. The dma coherent allocator is for now still left entirely to architecture supplied implementations due the amount of variations. Hopefully we can do some consolidation for them later on as well.
Because this series sits on top of two previously submitted series a git tree might be useful to actually test it. It is provided here:
git://git.infradead.org/users/hch/misc.gitarc-dmaGitweb:
Quoted 8 lines
https://urldefense.proofpoint.com/v2/url?u=http-3A__git.infradead.org_users_hch_misc.git_shortlog_refs_heads_arc-2Ddma&d=DwIBAg&c=DPL6_X_6JkXFx7
AXWqB0tg&r=lqdeeSSEes0GFDDl656eViXO7breS55ytWkhpk5R81I&m=QNC0zOG4bRotSxdjQfFXl0dAQJCtnxtF9435wAuJlu0&s=Wad1ptsFpknoSn0ebr3_QMJn9G_D3eIS-LYMwGfOJ- I&e=Changes since previous bigger series:
- take the offset into account innoncoherent_dma_map_page(Alexey Brodkin)
- fix dma directions in arc (Alexey Brodkin)
- split arc changes into smaller patches
I really like this series especially how it trimmed down arch/arc/mm/dma.c:
---------------------------->8----------------------------
git diff --stat HEAD~4
arch/arc/Kconfig | 4 +++ arch/arc/include/asm/Kbuild | 1 + arch/arc/include/asm/dma-mapping.h | 21 --------------- arch/arc/mm/cache.c | 2 +- arch/arc/mm/dma.c | 162 ++++++++++------------------------------------------------------------------------------------------------
---------
---------------------------->8----------------------------
So if we set aside my complaints about direction in arch_sync_dma_for_{device|cpu}()...
Let's still wait for Vineet's ack as he's the chief maintainer for ARC :)
-Alexey
A message that is not in the archive (sent off-list or not yet received): dfa0f4a194553f9bdd7bf4d56ac75d91cedc2609.camel-HKixBCOQz3hWk0Htik3J/[email protected]
Quoted 1 line · Alexey Brodkin
So if we set aside my complaints about direction in arch_sync_dma_for_{device|cpu}()...
Many other architectures use the argument. Various of those look bogus, but for now I want to be able to do straight forward conversions. If we end up not needing the argument in the end we can still remove it eventually.
Vineet Gupta
Acked-byQuoted 20 lines · Christoph Hellwig
Hi all,
this series continues consolidating the dma-mapping code, with a focus on architectures that do not (always) provide cache coherence for DMA. Three architectures (arm, mips and powerpc) are still left to be converted later due to complexity of their dma ops selection.
The dma-noncoherent ops calls the dma-direct ops for the actual translation of streaming mappins and allow the architecture to provide any cache flushing required for cpu to device and/or device to cpu ownership transfers. The dma coherent allocator is for now still left entirely to architecture supplied implementations due the amount of variations. Hopefully we can do some consolidation for them later on as well.
Because this series sits on top of two previously submitted series a git tree might be useful to actually test it. It is provided here:
git://git.infradead.org/users/hch/misc.gitarc-dma
Quoted 9 lines
Gitweb:
Changes since previous bigger series:
- take the offset into account innoncoherent_dma_map_page(Alexey Brodkin)
- fix dma directions in arc (Alexey Brodkin)
- split arc changes into smaller patches
LGTM !
Thx,
-Vineet