implement generic dma_map_ops for IOMMUs v5
Subsystems (3)
Hi Robin and Joerg,
I think we are finally ready for the generic dma-iommu series. I have various DMA API changes pending, and Tom has patches ready to convert the AMD and Intel iommu drivers over to it. I'd love to have this in a stable branch shared between the dma-mapping and iommu trees the day after rc2 is released. I volunteer to create the branch, but I'm fine with it living in the iommu tree as well. Before that Will has already said he wants to send the first patch in the series to Linus for this merge window.
A git tree is also available at:
git://git.infradead.org/users/hch/misc.git dma-iommu-ops.5
Gitweb:
http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/dma-iommu-ops.5
Changes since v4:
- rebased to 5.2-rc1
Changes since v3:
- fold the separate patch to refactor mmap bounds checking
- don't warn on not finding a vm_area
- improve a commit log
- refactor __dma_iommu_free a little differently
- remove a minor MSI map cleanup to avoid a conflict with the "Split iommu_dma_map_msi_msg" series
Changes since v2:
- address various review comments and include patches from Robin
Changes since v1:
- only include other headers in dma-iommu.h if CONFIG_DMA_IOMMU is enabled
- keep using a scatterlist in iommu_dma_alloc
- split out mmap/sgtable fixes and move them early in the series
- updated a few commit logs
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 1/24 arm64/iommu: handle non-remapped addresses in ->mmap and ->get_sgtable
DMA allocations that can't sleep may return non-remapped addresses, but we do not properly handle them in the mmap and get_sgtable methods. Resolve non-vmalloc addresses using virt_to_page to handle this corner case.
--- arch/arm64/mm/dma-mapping.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
arch/arm64/mm/dma-mapping.c +10 -0
@@ -249,6 +249,11 @@ static int __iommu_mmap_attrs(struct device *dev, struct vm_area_struct *vma,if (dma_mmap_from_dev_coherent (dev, vma, cpu_addr, size, &ret))return ret; +if (!is_vmalloc_addr (cpu_addr)) { +unsigned long pfn =page_to_pfn (virt_to_page (cpu_addr)); +return __swiotlb_mmap_pfn (vma, pfn, size); + } +if (attrs &DMA_ATTR_FORCE_CONTIGUOUS ) {/* * DMA_ATTR_FORCE_CONTIGUOUS allocations are always remapped,
@@ -272,6 +277,11 @@ static int __iommu_get_sgtable(struct device *dev, struct sg_table *sgt,unsigned int count =PAGE_ALIGN (size) >>PAGE_SHIFT ;struct vm_struct *area =find_vm_area (cpu_addr); +if (!is_vmalloc_addr (cpu_addr)) { +struct page *page =virt_to_page (cpu_addr); +return __swiotlb_get_sgtable_page (sgt, page, size); + } +if (attrs &DMA_ATTR_FORCE_CONTIGUOUS ) {/* * DMA_ATTR_FORCE_CONTIGUOUS allocations are always remapped,
Signature · 1 line
2.20.1
Re: [PATCH 01/24] arm64/iommu: handle non-remapped addresses in ->mmap and ->get_sgtable
Quoted 11 lines · Christoph Hellwig
DMA allocations that can't sleep may return non-remapped addresses, but we do not properly handle them in the mmap and
get_sgtablemethods. Resolve non-vmalloc addresses usingvirt_to_pageto handle this corner case.Signed-off-by Christoph Hellwig <[email protected]>Reviewed-by Robin Murphy <[email protected]>Acked-by Catalin Marinas <[email protected]>--- arch/arm64/mm/dma-mapping.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
I'll send this one to Linus this week as part of the arm64 fixes.
Will
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 2/24 iommu/dma: Cleanup dma-iommu.h
In reply to Christoph HellwigNo need for a __KERNEL__ guard outside uapi and add a missing comment describing the #else cpp statement. Last but not least include <linux/errno.h> instead of the asm version, which is frowned upon.
--- include/linux/dma-iommu.h | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
include/linux/dma-iommu.h +2 -4
@@ -16,9 +16,8 @@#ifndef __DMA_IOMMU_H#define __DMA_IOMMU_H -#ifdef __KERNEL__ +#include <linux/errno.h> #include <linux/types.h> -#include <asm/errno.h> #ifdef CONFIG_IOMMU_DMA #include <linux/dma-mapping.h>
@@ -86,7 +85,7 @@ void iommu_dma_compose_msi_msg(struct msi_desc *desc,void iommu_dma_get_resv_regions (struct device *dev,struct list_head *list); -#else +#else /* CONFIG_IOMMU_DMA */ struct iommu_domain ;struct msi_desc ;
@@ -128,5 +127,4 @@ static inline void iommu_dma_get_resv_regions(struct device *dev, struct list_he }#endif /* CONFIG_IOMMU_DMA */ -#endif /* __KERNEL__ */ #endif /* __DMA_IOMMU_H */
Signature · 1 line
2.20.1
PATCH 3/24 iommu/dma: Remove the flush_page callback
In reply to Christoph HellwigWe now have a arch_dma_prep_coherent architecture hook that is used for the generic DMA remap allocator, and we should use the same interface for the dma-iommu code.
--- arch/arm64/mm/dma-mapping.c | 8 +------- drivers/iommu/dma-iommu.c | 8 +++----- include/linux/dma-iommu.h | 3 +-- 3 files changed, 5 insertions(+), 14 deletions(-)
arch/arm64/mm/dma-mapping.c +1 -7
@@ -104,12 +104,6 @@ arch_initcall(arm64_dma_init);#include <linux/platform_device.h> #include <linux/amba/bus.h> -/* Thankfully, all cache ops are by VA so we can ignore phys here */ -static void flush_page (struct device *dev,const void *virt,phys_addr_t phys) -{ -__dma_flush_area (virt,PAGE_SIZE ); -} -static void *__iommu_alloc_attrs (struct device *dev,size_t size,dma_addr_t *handle,gfp_t gfp,unsigned long attrs)
@@ -186,7 +180,7 @@ static void *__iommu_alloc_attrs(struct device *dev, size_t size,struct page **pages; pages =iommu_dma_alloc (dev, iosize, gfp, attrs, ioprot, - handle, flush_page); + handle);if (!pages)return NULL ;
drivers/iommu/dma-iommu.c +3 -5
@@ -22,6 +22,7 @@#include <linux/acpi_iort.h> #include <linux/device.h> #include <linux/dma-iommu.h> +#include <linux/dma-noncoherent.h> #include <linux/gfp.h> #include <linux/huge_mm.h> #include <linux/iommu.h>
@@ -560,8 +561,6 @@ void iommu_dma_free(struct device *dev, struct page **pages, size_t size,* @attrs: DMA attributes for this allocation * @prot: IOMMU mapping flags * @handle: Out argument for allocated DMA handle -* @flush_page: Arch callback which must ensure PAGE_SIZE bytes from the -* given VA/PA are visible to the given non-coherent device. * * If @size is less than PAGE_SIZE, then a full CPU page will be allocated, * but an IOMMU which supports smaller pages might not map the whole thing.
@@ -570,8 +569,7 @@ void iommu_dma_free(struct device *dev, struct page **pages, size_t size,* or NULL on failure. */ struct page **iommu_dma_alloc (struct device *dev,size_t size,gfp_t gfp, -unsigned long attrs,int prot,dma_addr_t *handle, -void (*flush_page)(struct device *,const void *,phys_addr_t )) +unsigned long attrs,int prot,dma_addr_t *handle) {struct iommu_domain *domain =iommu_get_dma_domain (dev);struct iommu_dma_cookie *cookie = domain->iova_cookie;
@@ -615,7 +613,7 @@ struct page **iommu_dma_alloc(struct device *dev, size_t size, gfp_t gfp,*/ sg_miter_start (&miter, sgt.sgl, sgt.orig_nents,SG_MITER_FROM_SG );while (sg_miter_next (&miter)) -flush_page (dev, miter.addr,page_to_phys (miter.page)); +arch_dma_prep_coherent (miter.page,PAGE_SIZE );sg_miter_stop (&miter); }
include/linux/dma-iommu.h +1 -2
@@ -44,8 +44,7 @@ int dma_info_to_prot(enum dma_data_direction dir, bool coherent,* the arch code to take care of attributes and cache maintenance */ struct page **iommu_dma_alloc (struct device *dev,size_t size,gfp_t gfp, -unsigned long attrs,int prot,dma_addr_t *handle, -void (*flush_page)(struct device *,const void *,phys_addr_t )); +unsigned long attrs,int prot,dma_addr_t *handle);void iommu_dma_free (struct device *dev,struct page **pages,size_t size,dma_addr_t *handle);
Signature · 1 line
2.20.1
PATCH 4/24 iommu/dma: Use for_each_sg in iommu_dma_alloc
In reply to Christoph Hellwigarch_dma_prep_coherent can handle physically contiguous ranges larger than PAGE_SIZE just fine, which means we don't need a page-based iterator.
--- drivers/iommu/dma-iommu.c | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-)
drivers/iommu/dma-iommu.c +5 -9
@@ -606,15 +606,11 @@ struct page **iommu_dma_alloc(struct device *dev, size_t size, gfp_t gfp, goto out_free_iova;if (!(prot &IOMMU_CACHE )) { -struct sg_mapping_iter miter; -/* -* The CPU-centric flushing implied by SG_MITER_TO_SG isn't -* sufficient here, so skip it by using the "wrong" direction. -*/ -sg_miter_start (&miter, sgt.sgl, sgt.orig_nents,SG_MITER_FROM_SG ); -while (sg_miter_next (&miter)) -arch_dma_prep_coherent (miter.page,PAGE_SIZE ); -sg_miter_stop (&miter); +struct scatterlist *sg; +int i; + +for_each_sg (sgt.sgl, sg, sgt.orig_nents, i) +arch_dma_prep_coherent (sg_page (sg), sg->length); } if (iommu_map_sg(domain, iova, sgt.sgl, sgt.orig_nents, prot)
Signature · 1 line
2.20.1
PATCH 5/24 iommu/dma: move the arm64 wrappers to common code
In reply to Christoph HellwigThere is nothing really arm64 specific in the iommu_dma_ops implementation, so move it to dma-iommu.c and keep a lot of symbols self-contained. Note the implementation does depend on the DMA_DIRECT_REMAP infrastructure for now, so we'll have to make the DMA_IOMMU support depend on it, but this will be relaxed soon.
--- arch/arm64/mm/dma-mapping.c | 394 +---------------------------------- drivers/iommu/Kconfig | 1 + drivers/iommu/dma-iommu.c | 398 +++++++++++++++++++++++++++++++++--- include/linux/dma-iommu.h | 42 +--- 4 files changed, 378 insertions(+), 457 deletions(-)
arch/arm64/mm/dma-mapping.c +4 -390
@@ -27,6 +27,7 @@#include <linux/dma-direct.h> #include <linux/dma-noncoherent.h> #include <linux/dma-contiguous.h> +#include <linux/dma-iommu.h> #include <linux/vmalloc.h> #include <linux/swiotlb.h> #include <linux/pci.h>
@@ -58,37 +59,6 @@ void arch_dma_prep_coherent(struct page *page, size_t size)__dma_flush_area (page_address (page), size); } -#ifdef CONFIG_IOMMU_DMA -static int __swiotlb_get_sgtable_page (struct sg_table *sgt, -struct page *page,size_t size) -{ -int ret =sg_alloc_table (sgt,1 ,GFP_KERNEL ); - -if (!ret) -sg_set_page (sgt->sgl, page,PAGE_ALIGN (size),0 ); - -return ret; -} - -static int __swiotlb_mmap_pfn (struct vm_area_struct *vma, -unsigned long pfn,size_t size) -{ -int ret = -ENXIO ; -unsigned long nr_vma_pages =vma_pages (vma); -unsigned long nr_pages =PAGE_ALIGN (size) >>PAGE_SHIFT ; -unsigned long off = vma->vm_pgoff; - -if (off < nr_pages && nr_vma_pages <= (nr_pages - off)) { - ret =remap_pfn_range (vma, vma->vm_start, - pfn + off, - vma->vm_end - vma->vm_start, - vma->vm_page_prot); - } - -return ret; -} -#endif /* CONFIG_IOMMU_DMA */ -static int __init arm64_dma_init (void ) { WARN_TAINT(ARCH_DMA_MINALIGN < cache_line_size(),
@@ -100,374 +70,18 @@ static int __init arm64_dma_init(void)arch_initcall (arm64_dma_init);#ifdef CONFIG_IOMMU_DMA -#include <linux/dma-iommu.h> -#include <linux/platform_device.h> -#include <linux/amba/bus.h> - -static void *__iommu_alloc_attrs (struct device *dev,size_t size, -dma_addr_t *handle,gfp_t gfp, -unsigned long attrs) -{ -bool coherent =dev_is_dma_coherent (dev); -int ioprot =dma_info_to_prot (DMA_BIDIRECTIONAL , coherent, attrs); -size_t iosize = size; -void *addr; - -if (WARN (!dev,"cannot create IOMMU mapping for unknown device\n" )) -return NULL ; - - size =PAGE_ALIGN (size); - -/* -* Some drivers rely on this, and we probably don't want the -* possibility of stale kernel data being read by devices anyway. -*/ - gfp |= __GFP_ZERO; - -if (!gfpflags_allow_blocking (gfp)) { -struct page *page; -/* -* In atomic context we can't remap anything, so we'll only -* get the virtually contiguous buffer we need by way of a -* physically contiguous allocation. -*/ -if (coherent) { - page =alloc_pages (gfp,get_order (size)); - addr = page ?page_address (page) :NULL ; - }else { - addr =dma_alloc_from_pool (size, &page, gfp); - } -if (!addr) -return NULL ; - - *handle =iommu_dma_map_page (dev, page,0 , iosize, ioprot); -if (*handle ==DMA_MAPPING_ERROR ) { -if (coherent) -__free_pages (page,get_order (size)); -else -dma_free_from_pool (addr, size); - addr =NULL ; - } - }else if (attrs &DMA_ATTR_FORCE_CONTIGUOUS ) { -pgprot_t prot =arch_dma_mmap_pgprot (dev,PAGE_KERNEL , attrs); -struct page *page; - - page =dma_alloc_from_contiguous (dev, size >>PAGE_SHIFT , -get_order (size), gfp & __GFP_NOWARN); -if (!page) -return NULL ; - - *handle =iommu_dma_map_page (dev, page,0 , iosize, ioprot); -if (*handle ==DMA_MAPPING_ERROR ) { -dma_release_from_contiguous (dev, page, - size >>PAGE_SHIFT ); -return NULL ; - } - addr =dma_common_contiguous_remap (page, size,VM_USERMAP , - prot, -__builtin_return_address (0 )); -if (addr) { -if (!coherent) -__dma_flush_area (page_to_virt (page), iosize); -memset (addr,0 , size); - }else { -iommu_dma_unmap_page (dev, *handle, iosize,0 , attrs); -dma_release_from_contiguous (dev, page, - size >>PAGE_SHIFT ); - } - }else { -pgprot_t prot =arch_dma_mmap_pgprot (dev,PAGE_KERNEL , attrs); -struct page **pages; - - pages =iommu_dma_alloc (dev, iosize, gfp, attrs, ioprot, - handle); -if (!pages) -return NULL ; - - addr =dma_common_pages_remap (pages, size,VM_USERMAP , prot, -__builtin_return_address (0 )); -if (!addr) -iommu_dma_free (dev, pages, iosize, handle); - } -return addr; -} - -static void __iommu_free_attrs (struct device *dev,size_t size,void *cpu_addr, -dma_addr_t handle,unsigned long attrs) -{ -size_t iosize = size; - - size =PAGE_ALIGN (size); -/* -* @cpu_addr will be one of 4 things depending on how it was allocated: -* - A remapped array of pages for contiguous allocations. -* - A remapped array of pages from iommu_dma_alloc(), for all -* non-atomic allocations. -* - A non-cacheable alias from the atomic pool, for atomic -* allocations by non-coherent devices. -* - A normal lowmem address, for atomic allocations by -* coherent devices. -* Hence how dodgy the below logic looks... -*/ -if (dma_in_atomic_pool (cpu_addr, size)) { -iommu_dma_unmap_page (dev, handle, iosize,0 ,0 ); -dma_free_from_pool (cpu_addr, size); - }else if (attrs &DMA_ATTR_FORCE_CONTIGUOUS ) { -struct page *page =vmalloc_to_page (cpu_addr); - -iommu_dma_unmap_page (dev, handle, iosize,0 , attrs); -dma_release_from_contiguous (dev, page, size >>PAGE_SHIFT ); -dma_common_free_remap (cpu_addr, size,VM_USERMAP ); - }else if (is_vmalloc_addr (cpu_addr)){ -struct vm_struct *area =find_vm_area (cpu_addr); - -if (WARN_ON (!area || !area->pages)) -return ; -iommu_dma_free (dev, area->pages, iosize, &handle); -dma_common_free_remap (cpu_addr, size,VM_USERMAP ); - }else { -iommu_dma_unmap_page (dev, handle, iosize,0 ,0 ); -__free_pages (virt_to_page (cpu_addr),get_order (size)); - } -} - -static int __iommu_mmap_attrs (struct device *dev,struct vm_area_struct *vma, -void *cpu_addr,dma_addr_t dma_addr,size_t size, -unsigned long attrs) -{ -struct vm_struct *area; -int ret; - - vma->vm_page_prot =arch_dma_mmap_pgprot (dev, vma->vm_page_prot, attrs); - -if (dma_mmap_from_dev_coherent (dev, vma, cpu_addr, size, &ret)) -return ret; - -if (!is_vmalloc_addr (cpu_addr)) { -unsigned long pfn =page_to_pfn (virt_to_page (cpu_addr)); -return __swiotlb_mmap_pfn (vma, pfn, size); - } - -if (attrs &DMA_ATTR_FORCE_CONTIGUOUS ) { -/* -* DMA_ATTR_FORCE_CONTIGUOUS allocations are always remapped, -* hence in the vmalloc space. -*/ -unsigned long pfn =vmalloc_to_pfn (cpu_addr); -return __swiotlb_mmap_pfn (vma, pfn, size); - } - - area =find_vm_area (cpu_addr); -if (WARN_ON (!area || !area->pages)) -return -ENXIO ; - -return iommu_dma_mmap (area->pages, size, vma); -} - -static int __iommu_get_sgtable (struct device *dev,struct sg_table *sgt, -void *cpu_addr,dma_addr_t dma_addr, -size_t size,unsigned long attrs) -{ -unsigned int count =PAGE_ALIGN (size) >>PAGE_SHIFT ; -struct vm_struct *area =find_vm_area (cpu_addr); - -if (!is_vmalloc_addr (cpu_addr)) { -struct page *page =virt_to_page (cpu_addr); -return __swiotlb_get_sgtable_page (sgt, page, size); - } - -if (attrs &DMA_ATTR_FORCE_CONTIGUOUS ) { -/* -* DMA_ATTR_FORCE_CONTIGUOUS allocations are always remapped, -* hence in the vmalloc space. -*/ -struct page *page =vmalloc_to_page (cpu_addr); -return __swiotlb_get_sgtable_page (sgt, page, size); - } - -if (WARN_ON (!area || !area->pages)) -return -ENXIO ; - -return sg_alloc_table_from_pages (sgt, area->pages, count,0 , size, -GFP_KERNEL ); -} - -static void __iommu_sync_single_for_cpu (struct device *dev, -dma_addr_t dev_addr,size_t size, -enum dma_data_direction dir) -{ -phys_addr_t phys; - -if (dev_is_dma_coherent (dev)) -return ; - - phys =iommu_iova_to_phys (iommu_get_dma_domain (dev), dev_addr); -arch_sync_dma_for_cpu (dev, phys, size, dir); -} - -static void __iommu_sync_single_for_device (struct device *dev, -dma_addr_t dev_addr,size_t size, -enum dma_data_direction dir) -{ -phys_addr_t phys; - -if (dev_is_dma_coherent (dev)) -return ; - - phys =iommu_iova_to_phys (iommu_get_dma_domain (dev), dev_addr); -arch_sync_dma_for_device (dev, phys, size, dir); -} - -static dma_addr_t __iommu_map_page (struct device *dev,struct page *page, -unsigned long offset,size_t size, -enum dma_data_direction dir, -unsigned long attrs) -{ -bool coherent =dev_is_dma_coherent (dev); -int prot =dma_info_to_prot (dir, coherent, attrs); -dma_addr_t dev_addr =iommu_dma_map_page (dev, page, offset, size, prot); - -if (!coherent && !(attrs &DMA_ATTR_SKIP_CPU_SYNC ) && - dev_addr !=DMA_MAPPING_ERROR ) -__dma_map_area (page_address (page) + offset, size, dir); - -return dev_addr; -} - -static void __iommu_unmap_page (struct device *dev,dma_addr_t dev_addr, -size_t size,enum dma_data_direction dir, -unsigned long attrs) -{ -if ((attrs &DMA_ATTR_SKIP_CPU_SYNC ) ==0 ) -__iommu_sync_single_for_cpu (dev, dev_addr, size, dir); - -iommu_dma_unmap_page (dev, dev_addr, size, dir, attrs); -} - -static void __iommu_sync_sg_for_cpu (struct device *dev, -struct scatterlist *sgl,int nelems, -enum dma_data_direction dir) -{ -struct scatterlist *sg; -int i; - -if (dev_is_dma_coherent (dev)) -return ; - -for_each_sg (sgl, sg, nelems, i) -arch_sync_dma_for_cpu (dev,sg_phys (sg), sg->length, dir); -} - -static void __iommu_sync_sg_for_device (struct device *dev, -struct scatterlist *sgl,int nelems, -enum dma_data_direction dir) -{ -struct scatterlist *sg; -int i; - -if (dev_is_dma_coherent (dev)) -return ; - -for_each_sg (sgl, sg, nelems, i) -arch_sync_dma_for_device (dev,sg_phys (sg), sg->length, dir); -} - -static int __iommu_map_sg_attrs (struct device *dev,struct scatterlist *sgl, -int nelems,enum dma_data_direction dir, -unsigned long attrs) -{ -bool coherent =dev_is_dma_coherent (dev); - -if ((attrs &DMA_ATTR_SKIP_CPU_SYNC ) ==0 ) -__iommu_sync_sg_for_device (dev, sgl, nelems, dir); - -return iommu_dma_map_sg (dev, sgl, nelems, -dma_info_to_prot (dir, coherent, attrs)); -} - -static void __iommu_unmap_sg_attrs (struct device *dev, -struct scatterlist *sgl,int nelems, -enum dma_data_direction dir, -unsigned long attrs) -{ -if ((attrs &DMA_ATTR_SKIP_CPU_SYNC ) ==0 ) -__iommu_sync_sg_for_cpu (dev, sgl, nelems, dir); - -iommu_dma_unmap_sg (dev, sgl, nelems, dir, attrs); -} - -static const struct dma_map_ops iommu_dma_ops = { - .alloc = __iommu_alloc_attrs, - .free = __iommu_free_attrs, - .mmap = __iommu_mmap_attrs, - .get_sgtable = __iommu_get_sgtable, - .map_page = __iommu_map_page, - .unmap_page = __iommu_unmap_page, - .map_sg = __iommu_map_sg_attrs, - .unmap_sg = __iommu_unmap_sg_attrs, - .sync_single_for_cpu = __iommu_sync_single_for_cpu, - .sync_single_for_device = __iommu_sync_single_for_device, - .sync_sg_for_cpu = __iommu_sync_sg_for_cpu, - .sync_sg_for_device = __iommu_sync_sg_for_device, - .map_resource = iommu_dma_map_resource, - .unmap_resource = iommu_dma_unmap_resource, -}; - -static int __init __iommu_dma_init (void ) -{ -return iommu_dma_init (); -} -arch_initcall (__iommu_dma_init); - -static void __iommu_setup_dma_ops (struct device *dev,u64 dma_base,u64 size, -const struct iommu_ops *ops) -{ -struct iommu_domain *domain; - -if (!ops) -return ; - -/* -* The IOMMU core code allocates the default DMA domain, which the -* underlying IOMMU driver needs to support via the dma-iommu layer. -*/ - domain =iommu_get_domain_for_dev (dev); - -if (!domain) - goto out_err; - -if (domain->type ==IOMMU_DOMAIN_DMA ) { -if (iommu_dma_init_domain (domain, dma_base, size, dev)) - goto out_err; - - dev->dma_ops = &iommu_dma_ops; - } - -return ; - -out_err: -pr_warn ("Failed to set up IOMMU for device %s; retaining platform DMA ops\n" , -dev_name (dev)); -} -void arch_teardown_dma_ops (struct device *dev) { dev->dma_ops =NULL ; } - -#else - -static void __iommu_setup_dma_ops (struct device *dev,u64 dma_base,u64 size, -const struct iommu_ops *iommu) -{ } - -#endif /* CONFIG_IOMMU_DMA */ +#endif void arch_setup_dma_ops (struct device *dev,u64 dma_base,u64 size,const struct iommu_ops *iommu,bool coherent) { dev->dma_coherent = coherent; -__iommu_setup_dma_ops (dev, dma_base, size, iommu); +if (iommu) +iommu_setup_dma_ops (dev, dma_base, size);#ifdef CONFIG_XEN if (xen_initial_domain ())
drivers/iommu/Kconfig +1 -0
@@ -96,6 +96,7 @@ config IOMMU_DMA select IOMMU_IOVA select IRQ_MSI_IOMMU select NEED_SG_DMA_LENGTH + depends on DMA_DIRECT_REMAP config FSL_PAMU bool "Freescale IOMMU support"
drivers/iommu/dma-iommu.c +370 -28
531 lines not on this page · raw message
include/linux/dma-iommu.h +3 -39
61 lines not on this page · raw message
Signature · 6 lines
2.20.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 6/24 iommu/dma: Move __iommu_dma_map
In reply to Christoph HellwigMoving this function up to its unmap counterpart helps to keep related code together for the following changes.
--- drivers/iommu/dma-iommu.c | 46 +++++++++++++++++++-------------------- 1 file changed, 23 insertions(+), 23 deletions(-)
drivers/iommu/dma-iommu.c +23 -23
@@ -464,6 +464,29 @@ static void __iommu_dma_unmap(struct iommu_domain *domain, dma_addr_t dma_addr,iommu_dma_free_iova (cookie, dma_addr, size); } +static dma_addr_t __iommu_dma_map (struct device *dev,phys_addr_t phys, +size_t size,int prot,struct iommu_domain *domain) +{ +struct iommu_dma_cookie *cookie = domain->iova_cookie; +size_t iova_off =0 ; +dma_addr_t iova; + +if (cookie->type ==IOMMU_DMA_IOVA_COOKIE ) { + iova_off =iova_offset (&cookie->iovad, phys); + size =iova_align (&cookie->iovad, size + iova_off); + } + + iova =iommu_dma_alloc_iova (domain, size,dma_get_mask (dev), dev); +if (!iova) +return DMA_MAPPING_ERROR ; + +if (iommu_map (domain, iova, phys - iova_off, size, prot)) { +iommu_dma_free_iova (cookie, iova, size); +return DMA_MAPPING_ERROR ; + } +return iova + iova_off; +} +static void __iommu_dma_free_pages (struct page **pages,int count) {while (count--)
@@ -692,29 +715,6 @@ static void iommu_dma_sync_sg_for_device(struct device *dev,arch_sync_dma_for_device (dev,sg_phys (sg), sg->length, dir); } -static dma_addr_t __iommu_dma_map (struct device *dev,phys_addr_t phys, -size_t size,int prot,struct iommu_domain *domain) -{ -struct iommu_dma_cookie *cookie = domain->iova_cookie; -size_t iova_off =0 ; -dma_addr_t iova; - -if (cookie->type ==IOMMU_DMA_IOVA_COOKIE ) { - iova_off =iova_offset (&cookie->iovad, phys); - size =iova_align (&cookie->iovad, size + iova_off); - } - - iova =iommu_dma_alloc_iova (domain, size,dma_get_mask (dev), dev); -if (!iova) -return DMA_MAPPING_ERROR ; - -if (iommu_map (domain, iova, phys - iova_off, size, prot)) { -iommu_dma_free_iova (cookie, iova, size); -return DMA_MAPPING_ERROR ; - } -return iova + iova_off; -} -static dma_addr_t __iommu_dma_map_page (struct device *dev,struct page *page,unsigned long offset,size_t size,int prot) {
Signature · 1 line
2.20.1
PATCH 7/24 iommu/dma: Move domain lookup into __iommu_dma_{map,unmap}
In reply to Christoph HellwigFrom: Robin Murphy <[email protected]>
Most of the callers don't care, and the couple that do already have the domain to hand for other reasons are in slow paths where the (trivial) overhead of a repeated lookup will be utterly immaterial.
[hch: dropped the hunk touching iommu_dma_get_msi_page to avoid a
conflict with another series]
--- drivers/iommu/dma-iommu.c | 29 ++++++++++++++--------------- 1 file changed, 14 insertions(+), 15 deletions(-)
drivers/iommu/dma-iommu.c +14 -15
@@ -448,9 +448,10 @@ static void iommu_dma_free_iova(struct iommu_dma_cookie *cookie, size >> iova_shift(iovad)); } -static void __iommu_dma_unmap (struct iommu_domain *domain,dma_addr_t dma_addr, +static void __iommu_dma_unmap (struct device *dev,dma_addr_t dma_addr,size_t size) { +struct iommu_domain *domain =iommu_get_dma_domain (dev);struct iommu_dma_cookie *cookie = domain->iova_cookie;struct iova_domain *iovad = &cookie->iovad;size_t iova_off =iova_offset (iovad, dma_addr);
@@ -465,8 +466,9 @@ static void __iommu_dma_unmap(struct iommu_domain *domain, dma_addr_t dma_addr, }static dma_addr_t __iommu_dma_map (struct device *dev,phys_addr_t phys, -size_t size,int prot,struct iommu_domain *domain) +size_t size,int prot) { +struct iommu_domain *domain =iommu_get_dma_domain (dev);struct iommu_dma_cookie *cookie = domain->iova_cookie;size_t iova_off =0 ;dma_addr_t iova;
@@ -565,7 +567,7 @@ static struct page **__iommu_dma_alloc_pages(struct device *dev,static void __iommu_dma_free (struct device *dev,struct page **pages,size_t size,dma_addr_t *handle) { -__iommu_dma_unmap (iommu_get_dma_domain (dev), *handle, size); +__iommu_dma_unmap (dev, *handle, size);__iommu_dma_free_pages (pages,PAGE_ALIGN (size) >>PAGE_SHIFT ); *handle =DMA_MAPPING_ERROR ; }
@@ -718,14 +720,13 @@ static void iommu_dma_sync_sg_for_device(struct device *dev,static dma_addr_t __iommu_dma_map_page (struct device *dev,struct page *page,unsigned long offset,size_t size,int prot) { -return __iommu_dma_map (dev,page_to_phys (page) + offset, size, prot, -iommu_get_dma_domain (dev)); +return __iommu_dma_map (dev,page_to_phys (page) + offset, size, prot); }static void __iommu_dma_unmap_page (struct device *dev,dma_addr_t handle,size_t size,enum dma_data_direction dir,unsigned long attrs) { -__iommu_dma_unmap (iommu_get_dma_domain (dev), handle, size); +__iommu_dma_unmap (dev, handle, size); } static dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page,
@@ -734,11 +735,10 @@ static dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page, {phys_addr_t phys =page_to_phys (page) + offset;bool coherent =dev_is_dma_coherent (dev); +int prot =dma_info_to_prot (dir, coherent, attrs);dma_addr_t dma_handle; - dma_handle =__iommu_dma_map (dev, phys, size, -dma_info_to_prot (dir, coherent, attrs), -iommu_get_dma_domain (dev)); + dma_handle =__iommu_dma_map (dev, phys, size, prot);if (!coherent && !(attrs &DMA_ATTR_SKIP_CPU_SYNC ) && dma_handle !=DMA_MAPPING_ERROR )arch_sync_dma_for_device (dev, phys, size, dir);
@@ -750,7 +750,7 @@ static void iommu_dma_unmap_page(struct device *dev, dma_addr_t dma_handle, {if (!(attrs &DMA_ATTR_SKIP_CPU_SYNC ))iommu_dma_sync_single_for_cpu (dev, dma_handle, size, dir); -__iommu_dma_unmap (iommu_get_dma_domain (dev), dma_handle, size); +__iommu_dma_unmap (dev, dma_handle, size); }/*
@@ -931,21 +931,20 @@ static void iommu_dma_unmap_sg(struct device *dev, struct scatterlist *sg, sg = tmp; } end =sg_dma_address (sg) +sg_dma_len (sg); -__iommu_dma_unmap (iommu_get_dma_domain (dev), start, end - start); +__iommu_dma_unmap (dev, start, end - start); }static dma_addr_t iommu_dma_map_resource (struct device *dev,phys_addr_t phys,size_t size,enum dma_data_direction dir,unsigned long attrs) {return __iommu_dma_map (dev, phys, size, -dma_info_to_prot (dir, false, attrs) |IOMMU_MMIO , -iommu_get_dma_domain (dev)); +dma_info_to_prot (dir, false, attrs) |IOMMU_MMIO ); }static void iommu_dma_unmap_resource (struct device *dev,dma_addr_t handle,size_t size,enum dma_data_direction dir,unsigned long attrs) { -__iommu_dma_unmap (iommu_get_dma_domain (dev), handle, size); +__iommu_dma_unmap (dev, handle, size); } static void *iommu_dma_alloc(struct device *dev, size_t size,
@@ -1222,7 +1221,7 @@ static struct iommu_dma_msi_page *iommu_dma_get_msi_page(struct device *dev,if (!msi_page)return NULL ; - iova =__iommu_dma_map (dev, msi_addr, size, prot, domain); + iova =__iommu_dma_map (dev, msi_addr, size, prot);if (iova ==DMA_MAPPING_ERROR ) goto out_free_page;
Signature · 1 line
2.20.1
Robin Murphy
maintainerRe: [PATCH 07/24] iommu/dma: Move domain lookup into __iommu_dma_{map, unmap}
Quoted 9 lines · Christoph Hellwig
From: Robin Murphy <[email protected]>
Most of the callers don't care, and the couple that do already have the domain to hand for other reasons are in slow paths where the (trivial) overhead of a repeated lookup will be utterly immaterial.
Signed-off-by Robin Murphy <[email protected]>[hch: dropped the hunk touching
iommu_dma_get_msi_pageto avoid a
conflict with another series]
Since the MSI changes made it into 5.2, do you want to resurrect that hunk here, or shall I spin it up as a follow-on patch?
Robin.
Quoted 116 lines
Signed-off-by Christoph Hellwig <[email protected]>--- drivers/iommu/dma-iommu.c | 29 ++++++++++++++--------------- 1 file changed, 14 insertions(+), 15 deletions(-)drivers/iommu/dma-iommu.c +14
-15@@ -448,9 +448,10 @@ static void iommu_dma_free_iova(struct iommu_dma_cookie *cookie, size >> iova_shift(iovad)); } -static void __iommu_dma_unmap (struct iommu_domain *domain,dma_addr_t dma_addr, +static void __iommu_dma_unmap (struct device *dev,dma_addr_t dma_addr,size_t size) { +struct iommu_domain *domain =iommu_get_dma_domain (dev);struct iommu_dma_cookie *cookie = domain->iova_cookie;struct iova_domain *iovad = &cookie->iovad;size_t iova_off =iova_offset (iovad, dma_addr);@@ -465,8 +466,9 @@ static void __iommu_dma_unmap(struct iommu_domain *domain, dma_addr_t dma_addr, }static dma_addr_t __iommu_dma_map (struct device *dev,phys_addr_t phys, -size_t size,int prot,struct iommu_domain *domain) +size_t size,int prot) { +struct iommu_domain *domain =iommu_get_dma_domain (dev);struct iommu_dma_cookie *cookie = domain->iova_cookie;size_t iova_off =0 ;dma_addr_t iova;@@ -565,7 +567,7 @@ static struct page **__iommu_dma_alloc_pages(struct device *dev,static void __iommu_dma_free (struct device *dev,struct page **pages,size_t size,dma_addr_t *handle) { -__iommu_dma_unmap (iommu_get_dma_domain (dev), *handle, size); +__iommu_dma_unmap (dev, *handle, size);__iommu_dma_free_pages (pages,PAGE_ALIGN (size) >>PAGE_SHIFT ); *handle =DMA_MAPPING_ERROR ; }@@ -718,14 +720,13 @@ static void iommu_dma_sync_sg_for_device(struct device *dev,static dma_addr_t __iommu_dma_map_page (struct device *dev,struct page *page,unsigned long offset,size_t size,int prot) { -return __iommu_dma_map (dev,page_to_phys (page) + offset, size, prot, -iommu_get_dma_domain (dev)); +return __iommu_dma_map (dev,page_to_phys (page) + offset, size, prot); }static void __iommu_dma_unmap_page (struct device *dev,dma_addr_t handle,size_t size,enum dma_data_direction dir,unsigned long attrs) { -__iommu_dma_unmap (iommu_get_dma_domain (dev), handle, size); +__iommu_dma_unmap (dev, handle, size); } static dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page,@@ -734,11 +735,10 @@ static dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page, {phys_addr_t phys =page_to_phys (page) + offset;bool coherent =dev_is_dma_coherent (dev); +int prot =dma_info_to_prot (dir, coherent, attrs);dma_addr_t dma_handle; - dma_handle =__iommu_dma_map (dev, phys, size, -dma_info_to_prot (dir, coherent, attrs), -iommu_get_dma_domain (dev)); + dma_handle =__iommu_dma_map (dev, phys, size, prot);if (!coherent && !(attrs &DMA_ATTR_SKIP_CPU_SYNC ) && dma_handle !=DMA_MAPPING_ERROR )arch_sync_dma_for_device (dev, phys, size, dir);@@ -750,7 +750,7 @@ static void iommu_dma_unmap_page(struct device *dev, dma_addr_t dma_handle, {if (!(attrs &DMA_ATTR_SKIP_CPU_SYNC ))iommu_dma_sync_single_for_cpu (dev, dma_handle, size, dir); -__iommu_dma_unmap (iommu_get_dma_domain (dev), dma_handle, size); +__iommu_dma_unmap (dev, dma_handle, size); }/* @@ -931,21 +931,20 @@ static void iommu_dma_unmap_sg(struct device *dev, struct scatterlist *sg, sg = tmp; } end =sg_dma_address (sg) +sg_dma_len (sg); -__iommu_dma_unmap (iommu_get_dma_domain (dev), start, end - start); +__iommu_dma_unmap (dev, start, end - start); }static dma_addr_t iommu_dma_map_resource (struct device *dev,phys_addr_t phys,size_t size,enum dma_data_direction dir,unsigned long attrs) {return __iommu_dma_map (dev, phys, size, -dma_info_to_prot (dir, false, attrs) |IOMMU_MMIO , -iommu_get_dma_domain (dev)); +dma_info_to_prot (dir, false, attrs) |IOMMU_MMIO ); }static void iommu_dma_unmap_resource (struct device *dev,dma_addr_t handle,size_t size,enum dma_data_direction dir,unsigned long attrs) { -__iommu_dma_unmap (iommu_get_dma_domain (dev), handle, size); +__iommu_dma_unmap (dev, handle, size); } static void *iommu_dma_alloc(struct device *dev, size_t size,@@ -1222,7 +1221,7 @@ static struct iommu_dma_msi_page *iommu_dma_get_msi_page(struct device *dev,if (!msi_page)return NULL ; - iova =__iommu_dma_map (dev, msi_addr, size, prot, domain); + iova =__iommu_dma_map (dev, msi_addr, size, prot);if (iova ==DMA_MAPPING_ERROR ) goto out_free_page;
Re: [PATCH 07/24] iommu/dma: Move domain lookup into __iommu_dma_{map, unmap}
Quoted 7 lines · Robin Murphy
Quoted 6 lines · Christoph Hellwig
From: Robin Murphy <[email protected]>
Most of the callers don't care, and the couple that do already have the domain to hand for other reasons are in slow paths where the (trivial) overhead of a repeated lookup will be utterly immaterial.
Quoted 6 lines
Quoted 3 lines
Signed-off-by Robin Murphy <[email protected]>[hch: dropped the hunk touching
iommu_dma_get_msi_pageto avoid a
conflict with another series]Since the MSI changes made it into 5.2, do you want to resurrect that hunk here, or shall I spin it up as a follow-on patch?
Either way is fine with me.
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
RE: [PATCH 07/24] iommu/dma: Move domain lookup into __iommu_dma_{map, unmap}
In reply to Christoph HellwigHi Robin,
Quoted 13 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Christoph Hellwig
Sent: 20 May 2019 08:30
To: Robin Murphy <[email protected]>
Cc: Tom Murphy <[email protected]>; Catalin Marinas <[email protected]>; Will Deacon <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: [PATCH 07/24] iommu/dma: Move domain lookup into __iommu_dma_{map, unmap}
Quoted 5 lines
From: Robin Murphy <[email protected]>
Most of the callers don't care, and the couple that do already have the domain to hand for other reasons are in slow paths where the (trivial) overhead of a repeated lookup will be utterly immaterial.
On a Hisilicon ARM64 platform with 5.3-rc1, a F_TRANSALTION error from smmuv3 is reported when an attempt is made to assign a ixgbe vf dev to a Guest.
[ 196.747107] arm-smmu-v3 arm-smmu-v3.0.auto: event 0x10 received: [ 196.747109] arm-smmu-v3 arm-smmu-v3.0.auto: 0x00000180 00000010 [ 196.747110] arm-smmu-v3 arm-smmu-v3.0.auto: 0x0000020100000000 [ 196.747111] arm-smmu-v3 arm-smmu-v3.0.auto: 0x00000000ffffe040 [ 196.747113] arm-smmu-v3 arm-smmu-v3.0.auto: 0x00000000ffffe000
Git bisect points to this patch.
Please see below.
Quoted 124 lines
Signed-off-by Robin Murphy <[email protected]>[hch: dropped the hunk touching
iommu_dma_get_msi_pageto avoid a
conflict with another series]Signed-off-by Christoph Hellwig <[email protected]>--- drivers/iommu/dma-iommu.c | 29 ++++++++++++++--------------- 1 file changed, 14 insertions(+), 15 deletions(-)drivers/iommu/dma-iommu.c +2
-1@@ -448,9 +448,10 @@ static void iommu_dma_free_iova(struct iommu_dma_cookie *cookie, size >> iova_shift(iovad)); } -static void __iommu_dma_unmap (struct iommu_domain *domain,dma_addr_t dma_addr, +static void __iommu_dma_unmap(struct device *dev, dma_addr_t dma_addr, size_t size) { + struct iommu_domain *domain = iommu_get_dma_domain(dev);struct iommu_dma_cookie *cookie = domain->iova_cookie;drivers/iommu/dma-iommu.c +2
-1struct iova_domain *iovad = &cookie->iovad;size_t iova_off =iova_offset (iovad, dma_addr);@@ -465,8 +466,9 @@ static void __iommu_dma_unmap(struct iommu_domain *domain, dma_addr_t dma_addr, }static dma_addr_t __iommu_dma_map (struct device *dev,phys_addr_t phys, -size_t size,int prot,struct iommu_domain *domain) +size_t size,int prot) { +struct iommu_domain *domain =iommu_get_dma_domain (dev);struct iommu_dma_cookie *cookie = domain->iova_cookie;size_t iova_off =0 ;drivers/iommu/dma-iommu.c +1
-1dma_addr_t iova;@@ -565,7 +567,7 @@ static struct page **__iommu_dma_alloc_pages(struct device *dev, static void __iommu_dma_free(struct device *dev, struct page **pages, size_t size, dma_addr_t *handle) { - __iommu_dma_unmap(iommu_get_dma_domain(dev), *handle, size); + __iommu_dma_unmap(dev, *handle, size); __iommu_dma_free_pages(pages, PAGE_ALIGN(size) >> PAGE_SHIFT); *handle = DMA_MAPPING_ERROR;}
drivers/iommu/dma-iommu.c +2
-3@@ -718,14 +720,13 @@ static void iommu_dma_sync_sg_for_device(structdevice *dev, static dma_addr_t __iommu_dma_map_page(struct device *dev, struct page *page, unsigned long offset, size_t size, int prot) { - return __iommu_dma_map(dev, page_to_phys(page) + offset, size, prot, -iommu_get_dma_domain (dev )); + return __iommu_dma_map(dev, page_to_phys(page) + offset, size, prot); }static void __iommu_dma_unmap_page (struct device *dev,dma_addr_t handle,size_t size,enum dma_data_direction dir,unsigned long attrs) { -__iommu_dma_unmap (iommu_get_dma_domain (dev), handle, size); +__iommu_dma_unmap (dev, handle, size);}
static
dma_addr_tiommu_dma_map_page(struct device *dev, struct page
*page,drivers/iommu/dma-iommu.c +2
-3@@ -734,11 +735,10 @@ static dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page, { phys_addr_t phys = page_to_phys(page) + offset;bool coherent =dev_is_dma_coherent (dev); +int prot =dma_info_to_prot (dir, coherent, attrs);dma_addr_t dma_handle; - dma_handle =__iommu_dma_map (dev, phys, size, -dma_info_to_prot (dir, coherent, attrs), -iommu_get_dma_domain (dev)); + dma_handle =__iommu_dma_map (dev, phys, size, prot);if (!coherent && !(attrs &DMA_ATTR_SKIP_CPU_SYNC ) && dma_handle !=DMA_MAPPING_ERROR )drivers/iommu/dma-iommu.c +1
-1arch_sync_dma_for_device (dev, phys, size, dir);@@ -750,7 +750,7 @@ static void iommu_dma_unmap_page(struct device *dev, dma_addr_t dma_handle, {if (!(attrs &DMA_ATTR_SKIP_CPU_SYNC ))iommu_dma_sync_single_for_cpu (dev, dma_handle, size, dir); -__iommu_dma_unmap (iommu_get_dma_domain (dev), dma_handle, size); +__iommu_dma_unmap (dev, dma_handle, size); }/*
drivers/iommu/dma-iommu.c +3
-4@@ -931,21 +931,20 @@ static void iommu_dma_unmap_sg(struct device *dev, struct scatterlist *sg, sg = tmp; } end =sg_dma_address (sg) +sg_dma_len (sg); -__iommu_dma_unmap (iommu_get_dma_domain (dev), start, end - start); +__iommu_dma_unmap (dev, start, end - start); }static dma_addr_t iommu_dma_map_resource (struct device *dev,phys_addr_t phys,size_t size,enum dma_data_direction dir,unsigned long attrs) {return __iommu_dma_map (dev, phys, size, -dma_info_to_prot (dir, false, attrs) |IOMMU_MMIO , -iommu_get_dma_domain (dev)); +dma_info_to_prot (dir, false, attrs) |IOMMU_MMIO ); }static void iommu_dma_unmap_resource (struct device *dev,dma_addr_t handle,size_t size,enum dma_data_direction dir,unsigned long attrs) { -__iommu_dma_unmap (iommu_get_dma_domain (dev), handle, size); +__iommu_dma_unmap (dev, handle, size);}
static void *iommu_dma_alloc(struct device *dev,
size_tsize,
drivers/iommu/dma-iommu.c static struct iommu_dma_msi_page · line 1225
Quoted 6 lines
drivers/iommu/dma-iommu.c +1
-1@@ -1222,7 +1221,7 @@ static struct iommu_dma_msi_page *iommu_dma_get_msi_page(struct device *dev, if (!msi_page) return NULL; - iova = __iommu_dma_map(dev, msi_addr, size, prot, domain); + iova = __iommu_dma_map(dev, msi_addr, size, prot);
I think the domain here is retrieved using iommu_get_domain_for_dev() which may not be the default domain returned by iommu_get_dma_domain().
Please check and let me know.
Thanks,
Shameer
Quoted 9 lines
drivers/iommu/dma-iommu.c +0
-1if (iova ==DMA_MAPPING_ERROR ) goto out_free_page; --2.20.1
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Robin Murphy
maintainerRe: [PATCH 07/24] iommu/dma: Move domain lookup into __iommu_dma_{map, unmap}
Hi Shameer,
Quoted 34 lines · Shameerali Kolothum Thodi
Hi Robin,
Quoted 13 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Christoph Hellwig
Sent: 20 May 2019 08:30
To: Robin Murphy <[email protected]>
Cc: Tom Murphy <[email protected]>; Catalin Marinas <[email protected]>; Will Deacon <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: [PATCH 07/24] iommu/dma: Move domain lookup into __iommu_dma_{map, unmap}Quoted 5 lines
From: Robin Murphy <[email protected]>
Most of the callers don't care, and the couple that do already have the domain to hand for other reasons are in slow paths where the (trivial) overhead of a repeated lookup will be utterly immaterial.
On a Hisilicon ARM64 platform with 5.3-rc1, a
F_TRANSALTIONerror from smmuv3 is reported when an attempt is made to assign a ixgbe vf dev to a Guest.[ 196.747107] arm-smmu-v3 arm-smmu-v3.0.auto: event 0x10 received: [ 196.747109] arm-smmu-v3 arm-smmu-v3.0.auto: 0x00000180 00000010 [ 196.747110] arm-smmu-v3 arm-smmu-v3.0.auto: 0x0000020100000000 [ 196.747111] arm-smmu-v3 arm-smmu-v3.0.auto: 0x00000000ffffe040 [ 196.747113] arm-smmu-v3 arm-smmu-v3.0.auto: 0x00000000ffffe000Git bisect points to this patch.
Please see below.
[...]
Quoted 9 lines
drivers/iommu/dma-iommu.c static struct iommu_dma_msi_page *iommu_dma_get_msi_page(struct device *dev, · line 1224
Quoted 6 lines
*iommu_dma_get_msi_page(struct device *dev,
if (!msi_page)
return NULL;drivers/iommu/dma-iommu.c +1
-1- iova =__iommu_dma_map (dev, msi_addr, size, prot, domain); + iova =__iommu_dma_map (dev, msi_addr, size, prot);I think the domain here is retrieved using
iommu_get_domain_for_dev()which may not be the default domain returned byiommu_get_dma_domain().
Urgh, yes, how did I manage to miss that? :(
I doubt this patch reverts cleanly, but I don't think it needs to be completely undone anyway; give me a moment...
Robin.
Quoted 15 lines
Please check and let me know.
Thanks,
ShameerQuoted 9 lines
if (iova ==
DMA_MAPPING_ERROR)
gotoout_free_page;
--
2.20.1_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 8/24 iommu/dma: Squash __iommu_dma_{map,unmap}_page helpers
In reply to Christoph HellwigFrom: Robin Murphy <[email protected]>
The remaining internal callsites don't care about having prototypes compatible with the relevant dma_map_ops callbacks, so the extra level of indirection just wastes space and complictaes things.
--- drivers/iommu/dma-iommu.c | 25 +++++++------------------ 1 file changed, 7 insertions(+), 18 deletions(-)
drivers/iommu/dma-iommu.c +7 -18
@@ -717,18 +717,6 @@ static void iommu_dma_sync_sg_for_device(struct device *dev,arch_sync_dma_for_device (dev,sg_phys (sg), sg->length, dir); } -static dma_addr_t __iommu_dma_map_page (struct device *dev,struct page *page, -unsigned long offset,size_t size,int prot) -{ -return __iommu_dma_map (dev,page_to_phys (page) + offset, size, prot); -} - -static void __iommu_dma_unmap_page (struct device *dev,dma_addr_t handle, -size_t size,enum dma_data_direction dir,unsigned long attrs) -{ -__iommu_dma_unmap (dev, handle, size); -} -static dma_addr_t iommu_dma_map_page (struct device *dev,struct page *page,unsigned long offset,size_t size,enum dma_data_direction dir,unsigned long attrs)
@@ -974,7 +962,8 @@ static void *iommu_dma_alloc(struct device *dev, size_t size,if (!addr)return NULL ; - *handle =__iommu_dma_map_page (dev, page,0 , iosize, ioprot); + *handle =__iommu_dma_map (dev,page_to_phys (page), iosize, + ioprot);if (*handle ==DMA_MAPPING_ERROR ) {if (coherent)__free_pages (page,get_order (size));
@@ -991,7 +980,7 @@ static void *iommu_dma_alloc(struct device *dev, size_t size,if (!page)return NULL ; - *handle =__iommu_dma_map_page (dev, page,0 , iosize, ioprot); + *handle =__iommu_dma_map (dev,page_to_phys (page), iosize, ioprot);if (*handle ==DMA_MAPPING_ERROR ) {dma_release_from_contiguous (dev, page, size >>PAGE_SHIFT );
@@ -1005,7 +994,7 @@ static void *iommu_dma_alloc(struct device *dev, size_t size,arch_dma_prep_coherent (page, iosize);memset (addr,0 , size); }else { -__iommu_dma_unmap_page (dev, *handle, iosize,0 , attrs); +__iommu_dma_unmap (dev, *handle, iosize);dma_release_from_contiguous (dev, page, size >>PAGE_SHIFT ); }
@@ -1044,12 +1033,12 @@ static void iommu_dma_free(struct device *dev, size_t size, void *cpu_addr,* Hence how dodgy the below logic looks... */ if (dma_in_atomic_pool (cpu_addr, size)) { -__iommu_dma_unmap_page (dev, handle, iosize,0 ,0 ); +__iommu_dma_unmap (dev, handle, iosize);dma_free_from_pool (cpu_addr, size); }else if (attrs &DMA_ATTR_FORCE_CONTIGUOUS ) {struct page *page =vmalloc_to_page (cpu_addr); -__iommu_dma_unmap_page (dev, handle, iosize,0 , attrs); +__iommu_dma_unmap (dev, handle, iosize);dma_release_from_contiguous (dev, page, size >>PAGE_SHIFT );dma_common_free_remap (cpu_addr, size,VM_USERMAP ); }else if (is_vmalloc_addr (cpu_addr)){
@@ -1060,7 +1049,7 @@ static void iommu_dma_free(struct device *dev, size_t size, void *cpu_addr,__iommu_dma_free (dev, area->pages, iosize, &handle);dma_common_free_remap (cpu_addr, size,VM_USERMAP ); }else { -__iommu_dma_unmap_page (dev, handle, iosize,0 ,0 ); +__iommu_dma_unmap (dev, handle, iosize);__free_pages (virt_to_page (cpu_addr),get_order (size)); } }
Signature · 6 lines
2.20.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 9/24 iommu/dma: Factor out remapped pages lookup
In reply to Christoph HellwigFrom: Robin Murphy <[email protected]>
Since we duplicate the find_vm_area() logic a few times in places where we only care aboute the pages, factor out a helper to abstract it.
--- drivers/iommu/dma-iommu.c | 32 ++++++++++++++++++++------------ 1 file changed, 20 insertions(+), 12 deletions(-)
drivers/iommu/dma-iommu.c +20 -12
@@ -554,6 +554,15 @@ static struct page **__iommu_dma_alloc_pages(struct device *dev,return pages; } +static struct page **__iommu_dma_get_pages (void *cpu_addr) +{ +struct vm_struct *area =find_vm_area (cpu_addr); + +if (!area || !area->pages) +return NULL ; +return area->pages; +} +/** * iommu_dma_free - Free a buffer allocated by __iommu_dma_alloc() * @dev: Device which owns this buffer
@@ -1042,11 +1051,11 @@ static void iommu_dma_free(struct device *dev, size_t size, void *cpu_addr,dma_release_from_contiguous (dev, page, size >>PAGE_SHIFT );dma_common_free_remap (cpu_addr, size,VM_USERMAP ); }else if (is_vmalloc_addr (cpu_addr)){ -struct vm_struct *area =find_vm_area (cpu_addr); +struct page **pages =__iommu_dma_get_pages (cpu_addr); -if (WARN_ON (!area || !area->pages)) +if (!pages)return ; -__iommu_dma_free (dev, area->pages, iosize, &handle); +__iommu_dma_free (dev, pages, iosize, &handle);dma_common_free_remap (cpu_addr, size,VM_USERMAP ); }else {__iommu_dma_unmap (dev, handle, iosize);
@@ -1078,7 +1087,7 @@ static int iommu_dma_mmap(struct device *dev, struct vm_area_struct *vma, {unsigned long nr_pages =PAGE_ALIGN (size) >>PAGE_SHIFT ;unsigned long off = vma->vm_pgoff; -struct vm_struct *area; +struct page **pages;int ret; vma->vm_page_prot =arch_dma_mmap_pgprot (dev, vma->vm_page_prot, attrs);
@@ -1103,11 +1112,10 @@ static int iommu_dma_mmap(struct device *dev, struct vm_area_struct *vma,return __iommu_dma_mmap_pfn (vma, pfn, size); } - area =find_vm_area (cpu_addr); -if (WARN_ON (!area || !area->pages)) + pages =__iommu_dma_get_pages (cpu_addr); +if (!pages)return -ENXIO ; - -return __iommu_dma_mmap (area->pages, size, vma); +return __iommu_dma_mmap (pages, size, vma); } static int __iommu_dma_get_sgtable_page(struct sg_table *sgt, struct page *page,
@@ -1125,7 +1133,7 @@ static int iommu_dma_get_sgtable(struct device *dev, struct sg_table *sgt, unsigned long attrs) {unsigned int count =PAGE_ALIGN (size) >>PAGE_SHIFT ; -struct vm_struct *area =find_vm_area (cpu_addr); +struct page **pages;if (!is_vmalloc_addr (cpu_addr)) {struct page *page =virt_to_page (cpu_addr);
@@ -1141,10 +1149,10 @@ static int iommu_dma_get_sgtable(struct device *dev, struct sg_table *sgt,return __iommu_dma_get_sgtable_page (sgt, page, size); } -if (WARN_ON (!area || !area->pages)) + pages =__iommu_dma_get_pages (cpu_addr); +if (!pages)return -ENXIO ; - -return sg_alloc_table_from_pages (sgt, area->pages, count,0 , size, +return sg_alloc_table_from_pages (sgt, pages, count,0 , size,GFP_KERNEL ); }
Signature · 1 line
2.20.1
PATCH 10/24 iommu/dma: Refactor the page array remapping allocator
In reply to Christoph HellwigMove the call to dma_common_pages_remap into __iommu_dma_alloc and rename it to iommu_dma_alloc_remap. This creates a self-contained helper for remapped pages allocation and mapping.
--- drivers/iommu/dma-iommu.c | 54 +++++++++++++++++++-------------------- 1 file changed, 26 insertions(+), 28 deletions(-)
drivers/iommu/dma-iommu.c +26 -28
@@ -564,9 +564,9 @@ static struct page **__iommu_dma_get_pages(void *cpu_addr) }/** -* iommu_dma_free - Free a buffer allocated by __iommu_dma_alloc() +* iommu_dma_free - Free a buffer allocated by iommu_dma_alloc_remap() * @dev: Device which owns this buffer -* @pages: Array of buffer pages as returned by __iommu_dma_alloc() +* @pages: Array of buffer pages as returned by __iommu_dma_alloc_remap() * @size: Size of buffer in bytes * @handle: DMA address of buffer *
@@ -582,33 +582,35 @@ static void __iommu_dma_free(struct device *dev, struct page **pages, }/** -* __iommu_dma_alloc - Allocate and map a buffer contiguous in IOVA space +* iommu_dma_alloc_remap - Allocate and map a buffer contiguous in IOVA space * @dev: Device to allocate memory for. Must be a real device * attached to an iommu_dma_domain * @size: Size of buffer in bytes +* @dma_handle: Out argument for allocated DMA handle * @gfp: Allocation flags * @attrs: DMA attributes for this allocation -* @prot: IOMMU mapping flags -* @handle: Out argument for allocated DMA handle * * If @size is less than PAGE_SIZE, then a full CPU page will be allocated, * but an IOMMU which supports smaller pages might not map the whole thing. * -* Return: Array of struct page pointers describing the buffer, -* or NULL on failure. +* Return: Mapped virtual address, or NULL on failure. */ -static struct page **__iommu_dma_alloc (struct device *dev,size_t size, -gfp_t gfp,unsigned long attrs,int prot,dma_addr_t *handle) +static void *iommu_dma_alloc_remap (struct device *dev,size_t size, +dma_addr_t *dma_handle,gfp_t gfp,unsigned long attrs) {struct iommu_domain *domain =iommu_get_dma_domain (dev);struct iommu_dma_cookie *cookie = domain->iova_cookie;struct iova_domain *iovad = &cookie->iovad; +bool coherent =dev_is_dma_coherent (dev); +int ioprot =dma_info_to_prot (DMA_BIDIRECTIONAL , coherent, attrs); +pgprot_t prot =arch_dma_mmap_pgprot (dev,PAGE_KERNEL , attrs); +unsigned int count, min_size, alloc_sizes = domain->pgsize_bitmap;struct page **pages;struct sg_table sgt;dma_addr_t iova; -unsigned int count, min_size, alloc_sizes = domain->pgsize_bitmap; +void *vaddr; - *handle =DMA_MAPPING_ERROR ; + *dma_handle =DMA_MAPPING_ERROR ; min_size = alloc_sizes & -alloc_sizes;if (min_size <PAGE_SIZE ) {
@@ -634,7 +636,7 @@ static struct page **__iommu_dma_alloc(struct device *dev, size_t size,if (sg_alloc_table_from_pages (&sgt, pages, count,0 , size,GFP_KERNEL )) goto out_free_iova; -if (!(prot &IOMMU_CACHE )) { +if (!(ioprot &IOMMU_CACHE )) {struct scatterlist *sg;int i;
@@ -642,14 +644,21 @@ static struct page **__iommu_dma_alloc(struct device *dev, size_t size,arch_dma_prep_coherent (sg_page (sg), sg->length); } -if (iommu_map_sg (domain, iova, sgt.sgl, sgt.orig_nents, prot) +if (iommu_map_sg (domain, iova, sgt.sgl, sgt.orig_nents, ioprot) < size) goto out_free_sg; - *handle = iova; + vaddr =dma_common_pages_remap (pages, size,VM_USERMAP , prot, +__builtin_return_address (0 )); +if (!vaddr) + goto out_unmap; + + *dma_handle = iova;sg_free_table (&sgt); -return pages; +return vaddr; +out_unmap: +__iommu_dma_unmap (dev, iova, size); out_free_sg:sg_free_table (&sgt); out_free_iova:
@@ -1008,18 +1017,7 @@ static void *iommu_dma_alloc(struct device *dev, size_t size, size >> PAGE_SHIFT); } }else { -pgprot_t prot =arch_dma_mmap_pgprot (dev,PAGE_KERNEL , attrs); -struct page **pages; - - pages =__iommu_dma_alloc (dev, iosize, gfp, attrs, ioprot, - handle); -if (!pages) -return NULL ; - - addr =dma_common_pages_remap (pages, size,VM_USERMAP , prot, -__builtin_return_address (0 )); -if (!addr) -__iommu_dma_free (dev, pages, iosize, handle); + addr =iommu_dma_alloc_remap (dev, iosize, handle, gfp, attrs); }return addr; }
@@ -1033,7 +1031,7 @@ static void iommu_dma_free(struct device *dev, size_t size, void *cpu_addr,/* * @cpu_addr will be one of 4 things depending on how it was allocated: * - A remapped array of pages for contiguous allocations. -* - A remapped array of pages from __iommu_dma_alloc(), for all +* - A remapped array of pages from iommu_dma_alloc_remap(), for all * non-atomic allocations. * - A non-cacheable alias from the atomic pool, for atomic * allocations by non-coherent devices.
Signature · 1 line
2.20.1
PATCH 11/24 iommu/dma: Remove __iommu_dma_free
In reply to Christoph HellwigWe only have a single caller of this function left, so open code it there.
--- drivers/iommu/dma-iommu.c | 21 ++------------------- 1 file changed, 2 insertions(+), 19 deletions(-)
drivers/iommu/dma-iommu.c +2 -19
@@ -563,24 +563,6 @@ static struct page **__iommu_dma_get_pages(void *cpu_addr)return area->pages; } -/** -* iommu_dma_free - Free a buffer allocated by iommu_dma_alloc_remap() -* @dev: Device which owns this buffer -* @pages: Array of buffer pages as returned by __iommu_dma_alloc_remap() -* @size: Size of buffer in bytes -* @handle: DMA address of buffer -* -* Frees both the pages associated with the buffer, and the array -* describing them -*/ -static void __iommu_dma_free (struct device *dev,struct page **pages, -size_t size,dma_addr_t *handle) -{ -__iommu_dma_unmap (dev, *handle, size); -__iommu_dma_free_pages (pages,PAGE_ALIGN (size) >>PAGE_SHIFT ); - *handle =DMA_MAPPING_ERROR ; -} -/** * iommu_dma_alloc_remap - Allocate and map a buffer contiguous in IOVA space * @dev: Device to allocate memory for. Must be a real device
@@ -1053,7 +1035,8 @@ static void iommu_dma_free(struct device *dev, size_t size, void *cpu_addr,if (!pages)return ; -__iommu_dma_free (dev, pages, iosize, &handle); +__iommu_dma_unmap (dev, handle, iosize); +__iommu_dma_free_pages (pages, size >>PAGE_SHIFT );dma_common_free_remap (cpu_addr, size,VM_USERMAP ); }else {__iommu_dma_unmap (dev, handle, iosize);
Signature · 1 line
2.20.1
PATCH 12/24 iommu/dma: Refactor iommu_dma_free
In reply to Christoph HellwigFrom: Robin Murphy <[email protected]>
The freeing logic was made particularly horrible by part of it being opaque to the arch wrapper, which led to a lot of convoluted repetition to ensure each path did everything in the right order. Now that it's all private, we can pick apart and consolidate the logically-distinct steps of freeing the IOMMU mapping, the underlying pages, and the CPU remap (if necessary) into something much more manageable.
--- drivers/iommu/dma-iommu.c | 73 ++++++++++++++++++--------------------- 1 file changed, 33 insertions(+), 40 deletions(-)
drivers/iommu/dma-iommu.c +33 -40
@@ -935,6 +935,39 @@ static void iommu_dma_unmap_resource(struct device *dev, dma_addr_t handle,__iommu_dma_unmap (dev, handle, size); } +static void iommu_dma_free (struct device *dev,size_t size,void *cpu_addr, +dma_addr_t handle,unsigned long attrs) +{ +size_t alloc_size =PAGE_ALIGN (size); +int count = alloc_size >>PAGE_SHIFT ; +struct page *page =NULL , **pages =NULL ; + +__iommu_dma_unmap (dev, handle, size); + +/* Non-coherent atomic allocation? Easy */ +if (dma_free_from_pool (cpu_addr, alloc_size)) +return ; + +if (is_vmalloc_addr (cpu_addr)) { +/* +* If it the address is remapped, then it's either non-coherent +* or highmem CMA, or an iommu_dma_alloc_remap() construction. +*/ + pages =__iommu_dma_get_pages (cpu_addr); +if (!pages) + page =vmalloc_to_page (cpu_addr); +dma_common_free_remap (cpu_addr, alloc_size,VM_USERMAP ); + }else { +/* Lowmem means a coherent atomic or CMA allocation */ + page =virt_to_page (cpu_addr); + } + +if (pages) +__iommu_dma_free_pages (pages, count); +if (page && !dma_release_from_contiguous (dev, page, count)) +__free_pages (page,get_order (alloc_size)); +} +static void *iommu_dma_alloc (struct device *dev,size_t size,dma_addr_t *handle,gfp_t gfp,unsigned long attrs) {
@@ -1004,46 +1037,6 @@ static void *iommu_dma_alloc(struct device *dev, size_t size,return addr; } -static void iommu_dma_free (struct device *dev,size_t size,void *cpu_addr, -dma_addr_t handle,unsigned long attrs) -{ -size_t iosize = size; - - size =PAGE_ALIGN (size); -/* -* @cpu_addr will be one of 4 things depending on how it was allocated: -* - A remapped array of pages for contiguous allocations. -* - A remapped array of pages from iommu_dma_alloc_remap(), for all -* non-atomic allocations. -* - A non-cacheable alias from the atomic pool, for atomic -* allocations by non-coherent devices. -* - A normal lowmem address, for atomic allocations by -* coherent devices. -* Hence how dodgy the below logic looks... -*/ -if (dma_in_atomic_pool (cpu_addr, size)) { -__iommu_dma_unmap (dev, handle, iosize); -dma_free_from_pool (cpu_addr, size); - }else if (attrs &DMA_ATTR_FORCE_CONTIGUOUS ) { -struct page *page =vmalloc_to_page (cpu_addr); - -__iommu_dma_unmap (dev, handle, iosize); -dma_release_from_contiguous (dev, page, size >>PAGE_SHIFT ); -dma_common_free_remap (cpu_addr, size,VM_USERMAP ); - }else if (is_vmalloc_addr (cpu_addr)){ -struct page **pages =__iommu_dma_get_pages (cpu_addr); - -if (!pages) -return ; -__iommu_dma_unmap (dev, handle, iosize); -__iommu_dma_free_pages (pages, size >>PAGE_SHIFT ); -dma_common_free_remap (cpu_addr, size,VM_USERMAP ); - }else { -__iommu_dma_unmap (dev, handle, iosize); -__free_pages (virt_to_page (cpu_addr),get_order (size)); - } -} -static int __iommu_dma_mmap_pfn (struct vm_area_struct *vma,unsigned long pfn,size_t size) {
Signature · 1 line
2.20.1
PATCH 13/24 iommu/dma: Refactor iommu_dma_alloc
In reply to Christoph HellwigFrom: Robin Murphy <[email protected]>
Shuffle around the self-contained atomic and non-contiguous cases to return early and get out of the way of the CMA case that we're about to work on next.
--- drivers/iommu/dma-iommu.c | 60 +++++++++++++++++++-------------------- 1 file changed, 30 insertions(+), 30 deletions(-)
drivers/iommu/dma-iommu.c +30 -30
@@ -973,14 +973,19 @@ static void *iommu_dma_alloc(struct device *dev, size_t size, {bool coherent =dev_is_dma_coherent (dev);int ioprot =dma_info_to_prot (DMA_BIDIRECTIONAL , coherent, attrs); +pgprot_t prot =arch_dma_mmap_pgprot (dev,PAGE_KERNEL , attrs);size_t iosize = size; +struct page *page;void *addr; size =PAGE_ALIGN (size); gfp |= __GFP_ZERO; +if (gfpflags_allow_blocking (gfp) && + !(attrs &DMA_ATTR_FORCE_CONTIGUOUS )) +return iommu_dma_alloc_remap (dev, iosize, handle, gfp, attrs); +if (!gfpflags_allow_blocking (gfp)) { -struct page *page;/* * In atomic context we can't remap anything, so we'll only * get the virtually contiguous buffer we need by way of a
@@ -1002,39 +1007,34 @@ static void *iommu_dma_alloc(struct device *dev, size_t size,__free_pages (page,get_order (size)); elsedma_free_from_pool (addr ,size ); - addr =NULL ; - } - }else if (attrs &DMA_ATTR_FORCE_CONTIGUOUS ) { -pgprot_t prot =arch_dma_mmap_pgprot (dev,PAGE_KERNEL , attrs); -struct page *page; - - page =dma_alloc_from_contiguous (dev, size >>PAGE_SHIFT , -get_order (size), gfp & __GFP_NOWARN); -if (!page)return NULL ; - - *handle =__iommu_dma_map (dev,page_to_phys (page), iosize, ioprot); -if (*handle ==DMA_MAPPING_ERROR ) { -dma_release_from_contiguous (dev, page, - size >>PAGE_SHIFT ); -return NULL ; - } - addr =dma_common_contiguous_remap (page, size,VM_USERMAP , - prot, -__builtin_return_address (0 )); -if (addr) { -if (!coherent) -arch_dma_prep_coherent (page, iosize); -memset (addr,0 , size); - }else { -__iommu_dma_unmap (dev, *handle, iosize); -dma_release_from_contiguous (dev, page, - size >>PAGE_SHIFT ); } - }else { - addr =iommu_dma_alloc_remap (dev, iosize, handle, gfp, attrs); +return addr; } + + page =dma_alloc_from_contiguous (dev, size >>PAGE_SHIFT , +get_order (size), gfp & __GFP_NOWARN); +if (!page) +return NULL ; + + *handle =__iommu_dma_map (dev,page_to_phys (page), iosize, ioprot); +if (*handle ==DMA_MAPPING_ERROR ) + goto out_free_pages; + + addr =dma_common_contiguous_remap (page, size,VM_USERMAP , prot, +__builtin_return_address (0 )); +if (!addr) + goto out_unmap; + +if (!coherent) +arch_dma_prep_coherent (page, iosize); +memset (addr,0 , size);return addr; +out_unmap: +__iommu_dma_unmap (dev, *handle, iosize); +out_free_pages: +dma_release_from_contiguous (dev, page, size >>PAGE_SHIFT ); +return NULL ; } static int __iommu_dma_mmap_pfn(struct vm_area_struct *vma,
Signature · 1 line
2.20.1
PATCH 14/24 iommu/dma: Don't remap CMA unnecessarily
In reply to Christoph HellwigFrom: Robin Murphy <[email protected]>
Always remapping CMA allocations was largely a bodge to keep the freeing logic manageable when it was split between here and an arch wrapper. Now that it's all together and streamlined, we can relax that limitation.
--- drivers/iommu/dma-iommu.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-)
drivers/iommu/dma-iommu.c +12 -7
@@ -973,7 +973,6 @@ static void *iommu_dma_alloc(struct device *dev, size_t size, {bool coherent =dev_is_dma_coherent (dev);int ioprot =dma_info_to_prot (DMA_BIDIRECTIONAL , coherent, attrs); -pgprot_t prot =arch_dma_mmap_pgprot (dev,PAGE_KERNEL , attrs);size_t iosize = size;struct page *page;void *addr;
@@ -1021,13 +1020,19 @@ static void *iommu_dma_alloc(struct device *dev, size_t size,if (*handle ==DMA_MAPPING_ERROR ) goto out_free_pages; - addr =dma_common_contiguous_remap (page, size,VM_USERMAP , prot, -__builtin_return_address (0 )); -if (!addr) - goto out_unmap; +if (!coherent ||PageHighMem (page)) { +pgprot_t prot =arch_dma_mmap_pgprot (dev,PAGE_KERNEL , attrs); -if (!coherent) -arch_dma_prep_coherent (page, iosize); + addr =dma_common_contiguous_remap (page, size,VM_USERMAP , prot, +__builtin_return_address (0 )); +if (!addr) + goto out_unmap; + +if (!coherent) +arch_dma_prep_coherent (page, iosize); + }else { + addr =page_address (page); + }memset (addr,0 , size);return addr; out_unmap:
Signature · 1 line
2.20.1
PATCH 15/24 iommu/dma: Merge the CMA and alloc_pages allocation paths
In reply to Christoph HellwigInstead of having a separate code path for the non-blocking alloc_pages and CMA allocations paths merge them into one. There is a slight behavior change here in that we try the page allocator if CMA fails. This matches what dma-direct and other iommu drivers do and will be needed to use the dma-iommu code on architectures without DMA remapping later on.
--- drivers/iommu/dma-iommu.c | 32 ++++++++++++-------------------- 1 file changed, 12 insertions(+), 20 deletions(-)
drivers/iommu/dma-iommu.c +12 -20
@@ -974,7 +974,7 @@ static void *iommu_dma_alloc(struct device *dev, size_t size,bool coherent =dev_is_dma_coherent (dev);int ioprot =dma_info_to_prot (DMA_BIDIRECTIONAL , coherent, attrs);size_t iosize = size; -struct page *page; +struct page *page =NULL ;void *addr; size =PAGE_ALIGN (size);
@@ -984,35 +984,26 @@ static void *iommu_dma_alloc(struct device *dev, size_t size, !(attrs & DMA_ATTR_FORCE_CONTIGUOUS))return iommu_dma_alloc_remap (dev, iosize, handle, gfp, attrs); -if (!gfpflags_allow_blocking (gfp)) { -/* -* In atomic context we can't remap anything, so we'll only -* get the virtually contiguous buffer we need by way of a -* physically contiguous allocation. -*/ -if (coherent) { - page =alloc_pages (gfp,get_order (size)); - addr = page ?page_address (page) :NULL ; - }else { - addr =dma_alloc_from_pool (size, &page, gfp); - } +if (!gfpflags_allow_blocking (gfp) && !coherent) { + addr =dma_alloc_from_pool (size, &page, gfp);if (!addr)return NULL ; *handle =__iommu_dma_map (dev,page_to_phys (page), iosize, ioprot);if (*handle ==DMA_MAPPING_ERROR ) { -if (coherent) -__free_pages (page,get_order (size)); -else -dma_free_from_pool (addr, size); +dma_free_from_pool (addr, size);return NULL ; }return addr; } - page =dma_alloc_from_contiguous (dev, size >>PAGE_SHIFT , -get_order (size), gfp & __GFP_NOWARN); +if (gfpflags_allow_blocking (gfp)) + page =dma_alloc_from_contiguous (dev, size >>PAGE_SHIFT , +get_order (size), + gfp & __GFP_NOWARN); +if (!page) + page =alloc_pages (gfp,get_order (size));if (!page)return NULL ;
@@ -1038,7 +1029,8 @@ static void *iommu_dma_alloc(struct device *dev, size_t size, out_unmap:__iommu_dma_unmap (dev, *handle, iosize); out_free_pages: -dma_release_from_contiguous (dev, page, size >>PAGE_SHIFT ); +if (!dma_release_from_contiguous (dev, page, size >>PAGE_SHIFT )) +__free_pages (page,get_order (size));return NULL ; }
Signature · 1 line
2.20.1
PATCH 16/24 iommu/dma: Split iommu_dma_free
In reply to Christoph HellwigFrom: Robin Murphy <[email protected]>
Most of it can double up to serve the failure cleanup path for iommu_dma_alloc().
--- drivers/iommu/dma-iommu.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-)
drivers/iommu/dma-iommu.c +8 -4
@@ -935,15 +935,12 @@ static void iommu_dma_unmap_resource(struct device *dev, dma_addr_t handle,__iommu_dma_unmap (dev, handle, size); } -static void iommu_dma_free (struct device *dev,size_t size,void *cpu_addr, -dma_addr_t handle,unsigned long attrs) +static void __iommu_dma_free (struct device *dev,size_t size,void *cpu_addr) {size_t alloc_size =PAGE_ALIGN (size);int count = alloc_size >>PAGE_SHIFT ;struct page *page =NULL , **pages =NULL ; -__iommu_dma_unmap (dev, handle, size); -/* Non-coherent atomic allocation? Easy */ if (dma_free_from_pool (cpu_addr, alloc_size))return ;
@@ -968,6 +965,13 @@ static void iommu_dma_free(struct device *dev, size_t size, void *cpu_addr,__free_pages (page,get_order (alloc_size)); } +static void iommu_dma_free (struct device *dev,size_t size,void *cpu_addr, +dma_addr_t handle,unsigned long attrs) +{ +__iommu_dma_unmap (dev, handle, size); +__iommu_dma_free (dev, size, cpu_addr); +} +static void *iommu_dma_alloc (struct device *dev,size_t size,dma_addr_t *handle,gfp_t gfp,unsigned long attrs) {
Signature · 1 line
2.20.1
PATCH 17/24 iommu/dma: Cleanup variable naming in iommu_dma_alloc
In reply to Christoph HellwigFrom: Robin Murphy <[email protected]>
Most importantly clear up the size / iosize confusion. Also rename addr to cpu_addr to match the surrounding code and make the intention a little more clear.
--- drivers/iommu/dma-iommu.c | 45 +++++++++++++++++++-------------------- 1 file changed, 22 insertions(+), 23 deletions(-)
drivers/iommu/dma-iommu.c +22 -23
@@ -977,64 +977,63 @@ static void *iommu_dma_alloc(struct device *dev, size_t size, {bool coherent =dev_is_dma_coherent (dev);int ioprot =dma_info_to_prot (DMA_BIDIRECTIONAL , coherent, attrs); -size_t iosize = size; +size_t alloc_size =PAGE_ALIGN (size);struct page *page =NULL ; -void *addr; +void *cpu_addr; - size =PAGE_ALIGN (size); gfp |= __GFP_ZERO;if (gfpflags_allow_blocking (gfp) && !(attrs &DMA_ATTR_FORCE_CONTIGUOUS )) -return iommu_dma_alloc_remap (dev, iosize, handle, gfp, attrs); +return iommu_dma_alloc_remap (dev, size, handle, gfp, attrs);if (!gfpflags_allow_blocking (gfp) && !coherent) { - addr =dma_alloc_from_pool (size, &page, gfp); -if (!addr) + cpu_addr =dma_alloc_from_pool (alloc_size, &page, gfp); +if (!cpu_addr)return NULL ; - *handle =__iommu_dma_map (dev,page_to_phys (page), iosize, + *handle =__iommu_dma_map (dev,page_to_phys (page), size, ioprot);if (*handle ==DMA_MAPPING_ERROR ) { -dma_free_from_pool (addr, size); +dma_free_from_pool (cpu_addr, alloc_size);return NULL ; } -return addr; +return cpu_addr; }if (gfpflags_allow_blocking (gfp)) - page =dma_alloc_from_contiguous (dev, size >>PAGE_SHIFT , -get_order (size), + page =dma_alloc_from_contiguous (dev, alloc_size >>PAGE_SHIFT , +get_order (alloc_size), gfp & __GFP_NOWARN);if (!page) - page =alloc_pages (gfp,get_order (size)); + page =alloc_pages (gfp,get_order (alloc_size));if (!page)return NULL ; - *handle =__iommu_dma_map (dev,page_to_phys (page), iosize, ioprot); + *handle =__iommu_dma_map (dev,page_to_phys (page), size, ioprot);if (*handle ==DMA_MAPPING_ERROR ) goto out_free_pages;if (!coherent ||PageHighMem (page)) {pgprot_t prot =arch_dma_mmap_pgprot (dev,PAGE_KERNEL , attrs); - addr =dma_common_contiguous_remap (page, size,VM_USERMAP , prot, -__builtin_return_address (0 )); -if (!addr) + cpu_addr =dma_common_contiguous_remap (page, alloc_size, +VM_USERMAP , prot,__builtin_return_address (0 )); +if (!cpu_addr) goto out_unmap;if (!coherent) -arch_dma_prep_coherent (page, iosize); +arch_dma_prep_coherent (page, size); }else { - addr =page_address (page); + cpu_addr =page_address (page); } -memset (addr,0 , size); -return addr; +memset (cpu_addr,0 , alloc_size); +return cpu_addr; out_unmap: -__iommu_dma_unmap (dev, *handle, iosize); +__iommu_dma_unmap (dev, *handle, size); out_free_pages: -if (!dma_release_from_contiguous (dev, page, size >>PAGE_SHIFT )) -__free_pages (page,get_order (size)); +if (!dma_release_from_contiguous (dev, page, alloc_size >>PAGE_SHIFT )) +__free_pages (page,get_order (alloc_size));return NULL ; }
Signature · 1 line
2.20.1
PATCH 18/24 iommu/dma: Refactor iommu_dma_alloc, part 2
In reply to Christoph HellwigAll the logic in iommu_dma_alloc that deals with page allocation from the CMA or page allocators can be split into a self-contained helper, and we can than map the result of that or the atomic pool allocation with the iommu later. This also allows reusing __iommu_dma_free to tear down the allocations and MMU mappings when the IOMMU mapping fails.
Based on a patch from Robin Murphy.
--- drivers/iommu/dma-iommu.c | 65 +++++++++++++++++++++------------------ 1 file changed, 35 insertions(+), 30 deletions(-)
drivers/iommu/dma-iommu.c +35 -30
@@ -972,35 +972,14 @@ static void iommu_dma_free(struct device *dev, size_t size, void *cpu_addr,__iommu_dma_free (dev, size, cpu_addr); } -static void *iommu_dma_alloc (struct device *dev,size_t size, -dma_addr_t *handle,gfp_t gfp,unsigned long attrs) +static void *iommu_dma_alloc_pages (struct device *dev,size_t size, +struct page **pagep,gfp_t gfp,unsigned long attrs) {bool coherent =dev_is_dma_coherent (dev); -int ioprot =dma_info_to_prot (DMA_BIDIRECTIONAL , coherent, attrs);size_t alloc_size =PAGE_ALIGN (size);struct page *page =NULL ;void *cpu_addr; - gfp |= __GFP_ZERO; - -if (gfpflags_allow_blocking (gfp) && - !(attrs &DMA_ATTR_FORCE_CONTIGUOUS )) -return iommu_dma_alloc_remap (dev, size, handle, gfp, attrs); - -if (!gfpflags_allow_blocking (gfp) && !coherent) { - cpu_addr =dma_alloc_from_pool (alloc_size, &page, gfp); -if (!cpu_addr) -return NULL ; - - *handle =__iommu_dma_map (dev,page_to_phys (page), size, - ioprot); -if (*handle ==DMA_MAPPING_ERROR ) { -dma_free_from_pool (cpu_addr, alloc_size); -return NULL ; - } -return cpu_addr; - } -if (gfpflags_allow_blocking (gfp)) page = dma_alloc_from_contiguous(dev, alloc_size >> PAGE_SHIFT, get_order(alloc_size),
@@ -1010,33 +989,59 @@ static void *iommu_dma_alloc(struct device *dev, size_t size,if (!page)return NULL ; - *handle =__iommu_dma_map (dev,page_to_phys (page), size, ioprot); -if (*handle ==DMA_MAPPING_ERROR ) - goto out_free_pages; -if (!coherent ||PageHighMem (page)) {pgprot_t prot =arch_dma_mmap_pgprot (dev,PAGE_KERNEL , attrs); cpu_addr =dma_common_contiguous_remap (page, alloc_size,VM_USERMAP , prot,__builtin_return_address (0 ));if (!cpu_addr) - goto out_unmap; + goto out_free_pages;if (!coherent)arch_dma_prep_coherent (page, size); }else { cpu_addr =page_address (page); } + + *pagep = page;memset (cpu_addr,0 , alloc_size);return cpu_addr; -out_unmap: -__iommu_dma_unmap (dev, *handle, size); out_free_pages:if (!dma_release_from_contiguous (dev, page, alloc_size >>PAGE_SHIFT ))__free_pages (page,get_order (alloc_size));return NULL ; } +static void *iommu_dma_alloc (struct device *dev,size_t size, +dma_addr_t *handle,gfp_t gfp,unsigned long attrs) +{ +bool coherent =dev_is_dma_coherent (dev); +int ioprot =dma_info_to_prot (DMA_BIDIRECTIONAL , coherent, attrs); +struct page *page =NULL ; +void *cpu_addr; + + gfp |= __GFP_ZERO; + +if (gfpflags_allow_blocking (gfp) && + !(attrs &DMA_ATTR_FORCE_CONTIGUOUS )) +return iommu_dma_alloc_remap (dev, size, handle, gfp, attrs); + +if (!gfpflags_allow_blocking (gfp) && !coherent) + cpu_addr =dma_alloc_from_pool (PAGE_ALIGN (size), &page, gfp); +else + cpu_addr =iommu_dma_alloc_pages (dev, size, &page, gfp, attrs); +if (!cpu_addr) +return NULL ; + + *handle =__iommu_dma_map (dev,page_to_phys (page), size, ioprot); +if (*handle ==DMA_MAPPING_ERROR ) { +__iommu_dma_free (dev, size, cpu_addr); +return NULL ; + } + +return cpu_addr; +} +static int __iommu_dma_mmap_pfn (struct vm_area_struct *vma,unsigned long pfn,size_t size) {
Signature · 1 line
2.20.1
PATCH 19/24 iommu/dma: Refactor iommu_dma_get_sgtable
In reply to Christoph HellwigInline __iommu_dma_get_sgtable_page into the main function, and use the fact that __iommu_dma_get_pages return NULL for remapped contigous allocations to simplify the code flow a bit.
--- drivers/iommu/dma-iommu.c | 45 +++++++++++++++------------------------ 1 file changed, 17 insertions(+), 28 deletions(-)
drivers/iommu/dma-iommu.c +17 -28
@@ -1097,42 +1097,31 @@ static int iommu_dma_mmap(struct device *dev, struct vm_area_struct *vma,return __iommu_dma_mmap (pages, size, vma); } -static int __iommu_dma_get_sgtable_page (struct sg_table *sgt,struct page *page, -size_t size) -{ -int ret =sg_alloc_table (sgt,1 ,GFP_KERNEL ); - -if (!ret) -sg_set_page (sgt->sgl, page,PAGE_ALIGN (size),0 ); -return ret; -} -static int iommu_dma_get_sgtable (struct device *dev,struct sg_table *sgt,void *cpu_addr,dma_addr_t dma_addr,size_t size,unsigned long attrs) { -unsigned int count =PAGE_ALIGN (size) >>PAGE_SHIFT ; -struct page **pages; +struct page *page; +int ret; -if (!is_vmalloc_addr (cpu_addr)) { -struct page *page =virt_to_page (cpu_addr); -return __iommu_dma_get_sgtable_page (sgt, page, size); - } +if (is_vmalloc_addr (cpu_addr)) { +struct page **pages =__iommu_dma_get_pages (cpu_addr); -if (attrs &DMA_ATTR_FORCE_CONTIGUOUS ) { -/* -* DMA_ATTR_FORCE_CONTIGUOUS allocations are always remapped, -* hence in the vmalloc space. -*/ -struct page *page =vmalloc_to_page (cpu_addr); -return __iommu_dma_get_sgtable_page (sgt, page, size); +if (pages) { +return sg_alloc_table_from_pages (sgt, pages, +PAGE_ALIGN (size) >>PAGE_SHIFT , +0 , size,GFP_KERNEL ); + } + + page =vmalloc_to_page (cpu_addr); + }else { + page =virt_to_page (cpu_addr); } - pages =__iommu_dma_get_pages (cpu_addr); -if (!pages) -return -ENXIO ; -return sg_alloc_table_from_pages (sgt, pages, count,0 , size, -GFP_KERNEL ); + ret =sg_alloc_table (sgt,1 ,GFP_KERNEL ); +if (!ret) +sg_set_page (sgt->sgl, page,PAGE_ALIGN (size),0 ); +return ret; }static const struct dma_map_ops iommu_dma_ops = {
Signature · 1 line
2.20.1
PATCH 20/24 iommu/dma: Refactor iommu_dma_mmap
In reply to Christoph HellwigInline __iommu_dma_mmap_pfn into the main function, and use the fact that __iommu_dma_get_pages return NULL for remapped contigous allocations to simplify the code flow a bit.
--- drivers/iommu/dma-iommu.c | 46 ++++++++++----------------------------- 1 file changed, 11 insertions(+), 35 deletions(-)
drivers/iommu/dma-iommu.c +11 -35
@@ -1042,31 +1042,12 @@ static void *iommu_dma_alloc(struct device *dev, size_t size,return cpu_addr; } -static int __iommu_dma_mmap_pfn (struct vm_area_struct *vma, -unsigned long pfn,size_t size) -{ -int ret = -ENXIO ; -unsigned long nr_vma_pages =vma_pages (vma); -unsigned long nr_pages =PAGE_ALIGN (size) >>PAGE_SHIFT ; -unsigned long off = vma->vm_pgoff; - -if (off < nr_pages && nr_vma_pages <= (nr_pages - off)) { - ret =remap_pfn_range (vma, vma->vm_start, - pfn + off, - vma->vm_end - vma->vm_start, - vma->vm_page_prot); - } - -return ret; -} -static int iommu_dma_mmap (struct device *dev,struct vm_area_struct *vma,void *cpu_addr,dma_addr_t dma_addr,size_t size,unsigned long attrs) {unsigned long nr_pages =PAGE_ALIGN (size) >>PAGE_SHIFT ; -unsigned long off = vma->vm_pgoff; -struct page **pages; +unsigned long pfn, off = vma->vm_pgoff;int ret; vma->vm_page_prot =arch_dma_mmap_pgprot (dev, vma->vm_page_prot, attrs);
@@ -1077,24 +1058,19 @@ static int iommu_dma_mmap(struct device *dev, struct vm_area_struct *vma,if (off >= nr_pages ||vma_pages (vma) > nr_pages - off)return -ENXIO ; -if (!is_vmalloc_addr (cpu_addr)) { -unsigned long pfn =page_to_pfn (virt_to_page (cpu_addr)); -return __iommu_dma_mmap_pfn (vma, pfn, size); - } +if (is_vmalloc_addr (cpu_addr)) { +struct page **pages =__iommu_dma_get_pages (cpu_addr); -if (attrs &DMA_ATTR_FORCE_CONTIGUOUS ) { -/* -* DMA_ATTR_FORCE_CONTIGUOUS allocations are always remapped, -* hence in the vmalloc space. -*/ -unsigned long pfn =vmalloc_to_pfn (cpu_addr); -return __iommu_dma_mmap_pfn (vma, pfn, size); +if (pages) +return __iommu_dma_mmap (pages, size, vma); + pfn =vmalloc_to_pfn (cpu_addr); + }else { + pfn =page_to_pfn (virt_to_page (cpu_addr)); } - pages =__iommu_dma_get_pages (cpu_addr); -if (!pages) -return -ENXIO ; -return __iommu_dma_mmap (pages, size, vma); +return remap_pfn_range (vma, vma->vm_start, pfn + off, + vma->vm_end - vma->vm_start, + vma->vm_page_prot); } static int iommu_dma_get_sgtable(struct device *dev, struct sg_table *sgt,
Signature · 1 line
2.20.1
Robin Murphy
maintainerRe: [PATCH 20/24] iommu/dma: Refactor iommu_dma_mmap
Quoted 3 lines · Christoph Hellwig
Inline
__iommu_dma_mmap_pfninto the main function, and use the fact that__iommu_dma_get_pagesreturn NULL for remapped contigous allocations to simplify the code flow a bit.
This would be a good point to get rid of __iommu_dma_mmap() now too.
Robin.
Quoted 79 lines
Signed-off-by Christoph Hellwig <[email protected]>Reviewed-by Robin Murphy <[email protected]>--- drivers/iommu/dma-iommu.c | 46 ++++++++++----------------------------- 1 file changed, 11 insertions(+), 35 deletions(-)drivers/iommu/dma-iommu.c +11
-35@@ -1042,31 +1042,12 @@ static void *iommu_dma_alloc(struct device *dev, size_t size,return cpu_addr; } -static int __iommu_dma_mmap_pfn (struct vm_area_struct *vma, -unsigned long pfn,size_t size) -{ -int ret = -ENXIO ; -unsigned long nr_vma_pages =vma_pages (vma); -unsigned long nr_pages =PAGE_ALIGN (size) >>PAGE_SHIFT ; -unsigned long off = vma->vm_pgoff; - -if (off < nr_pages && nr_vma_pages <= (nr_pages - off)) { - ret =remap_pfn_range (vma, vma->vm_start, - pfn + off, - vma->vm_end - vma->vm_start, - vma->vm_page_prot); - } - -return ret; -} -static int iommu_dma_mmap (struct device *dev,struct vm_area_struct *vma,void *cpu_addr,dma_addr_t dma_addr,size_t size,unsigned long attrs) {unsigned long nr_pages =PAGE_ALIGN (size) >>PAGE_SHIFT ; -unsigned long off = vma->vm_pgoff; -struct page **pages; +unsigned long pfn, off = vma->vm_pgoff;int ret; vma->vm_page_prot =arch_dma_mmap_pgprot (dev, vma->vm_page_prot, attrs);@@ -1077,24 +1058,19 @@ static int iommu_dma_mmap(struct device *dev, struct vm_area_struct *vma,if (off >= nr_pages ||vma_pages (vma) > nr_pages - off)return -ENXIO ; -if (!is_vmalloc_addr (cpu_addr)) { -unsigned long pfn =page_to_pfn (virt_to_page (cpu_addr)); -return __iommu_dma_mmap_pfn (vma, pfn, size); - } +if (is_vmalloc_addr (cpu_addr)) { +struct page **pages =__iommu_dma_get_pages (cpu_addr); -if (attrs &DMA_ATTR_FORCE_CONTIGUOUS ) { -/* -* DMA_ATTR_FORCE_CONTIGUOUS allocations are always remapped, -* hence in the vmalloc space. -*/ -unsigned long pfn =vmalloc_to_pfn (cpu_addr); -return __iommu_dma_mmap_pfn (vma, pfn, size); +if (pages) +return __iommu_dma_mmap (pages, size, vma); + pfn =vmalloc_to_pfn (cpu_addr); + }else { + pfn =page_to_pfn (virt_to_page (cpu_addr)); } - pages =__iommu_dma_get_pages (cpu_addr); -if (!pages) -return -ENXIO ; -return __iommu_dma_mmap (pages, size, vma); +return remap_pfn_range (vma, vma->vm_start, pfn + off, + vma->vm_end - vma->vm_start, + vma->vm_page_prot); } static int iommu_dma_get_sgtable(struct device *dev, struct sg_table *sgt,
PATCH 21/24 iommu/dma: Don't depend on CONFIG_DMA_DIRECT_REMAP
In reply to Christoph HellwigFor entirely dma coherent architectures there is no requirement to ever remap dma coherent allocation. Move all the remap and pool code under IS_ENABLED() checks and drop the Kconfig dependency.
--- drivers/iommu/Kconfig | 1 - drivers/iommu/dma-iommu.c | 16 +++++++++------- 2 files changed, 9 insertions(+), 8 deletions(-)
drivers/iommu/Kconfig +0 -1
@@ -96,7 +96,6 @@ config IOMMU_DMA select IOMMU_IOVA select IRQ_MSI_IOMMU select NEED_SG_DMA_LENGTH - depends on DMA_DIRECT_REMAP config FSL_PAMU bool "Freescale IOMMU support"
drivers/iommu/dma-iommu.c +9 -7
@@ -942,10 +942,11 @@ static void __iommu_dma_free(struct device *dev, size_t size, void *cpu_addr)struct page *page =NULL , **pages =NULL ;/* Non-coherent atomic allocation? Easy */ -if (dma_free_from_pool (cpu_addr, alloc_size)) +if (IS_ENABLED (CONFIG_DMA_DIRECT_REMAP ) && +dma_free_from_pool (cpu_addr, alloc_size))return ; -if (is_vmalloc_addr (cpu_addr)) { +if (IS_ENABLED (CONFIG_DMA_REMAP ) &&is_vmalloc_addr (cpu_addr)) {/* * If it the address is remapped, then it's either non-coherent * or highmem CMA, or an iommu_dma_alloc_remap() construction.
@@ -989,7 +990,7 @@ static void *iommu_dma_alloc_pages(struct device *dev, size_t size,if (!page)return NULL ; -if (!coherent ||PageHighMem (page)) { +if (IS_ENABLED (CONFIG_DMA_REMAP ) && (!coherent ||PageHighMem (page))) {pgprot_t prot =arch_dma_mmap_pgprot (dev,PAGE_KERNEL , attrs); cpu_addr = dma_common_contiguous_remap(page, alloc_size,
@@ -1022,11 +1023,12 @@ static void *iommu_dma_alloc(struct device *dev, size_t size, gfp |= __GFP_ZERO; -if (gfpflags_allow_blocking (gfp) && +if (IS_ENABLED (CONFIG_DMA_REMAP ) &&gfpflags_allow_blocking (gfp) && !(attrs &DMA_ATTR_FORCE_CONTIGUOUS ))return iommu_dma_alloc_remap (dev, size, handle, gfp, attrs); -if (!gfpflags_allow_blocking (gfp) && !coherent) +if (IS_ENABLED (CONFIG_DMA_DIRECT_REMAP ) && + !gfpflags_allow_blocking (gfp) && !coherent) cpu_addr =dma_alloc_from_pool (PAGE_ALIGN (size), &page, gfp);else cpu_addr =iommu_dma_alloc_pages (dev, size, &page, gfp, attrs);
@@ -1058,7 +1060,7 @@ static int iommu_dma_mmap(struct device *dev, struct vm_area_struct *vma,if (off >= nr_pages ||vma_pages (vma) > nr_pages - off)return -ENXIO ; -if (is_vmalloc_addr (cpu_addr)) { +if (IS_ENABLED (CONFIG_DMA_REMAP ) &&is_vmalloc_addr (cpu_addr)) {struct page **pages =__iommu_dma_get_pages (cpu_addr);if (pages)
@@ -1080,7 +1082,7 @@ static int iommu_dma_get_sgtable(struct device *dev, struct sg_table *sgt,struct page *page;int ret; -if (is_vmalloc_addr (cpu_addr)) { +if (IS_ENABLED (CONFIG_DMA_REMAP ) &&is_vmalloc_addr (cpu_addr)) {struct page **pages =__iommu_dma_get_pages (cpu_addr);if (pages) {
Signature · 1 line
2.20.1
PATCH 22/24 iommu/dma: Switch copyright boilerplace to SPDX
In reply to Christoph Hellwig--- drivers/iommu/dma-iommu.c | 13 +------------ include/linux/dma-iommu.h | 13 +------------ 2 files changed, 2 insertions(+), 24 deletions(-)
drivers/iommu/dma-iommu.c +1 -12
@@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * A fairly generic DMA-API to IOMMU-API glue layer. *
@@ -5,18 +6,6 @@* * based in part on arch/arm/mm/dma-mapping.c: * Copyright (C) 2000-2004 Russell King -* -* This program is free software; you can redistribute it and/or modify -* it under the terms of the GNU General Public License version 2 as -* published by the Free Software Foundation. -* -* This program is distributed in the hope that it will be useful, -* but WITHOUT ANY WARRANTY; without even the implied warranty of -* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -* GNU General Public License for more details. -* -* You should have received a copy of the GNU General Public License -* along with this program. If not, see <http://www.gnu.org/licenses/>. */ #include <linux/acpi_iort.h>
include/linux/dma-iommu.h +1 -12
@@ -1,17 +1,6 @@ +/* SPDX-License-Identifier: GPL-2.0 */ /* * Copyright (C) 2014-2015 ARM Ltd. -* -* This program is free software; you can redistribute it and/or modify -* it under the terms of the GNU General Public License version 2 as -* published by the Free Software Foundation. -* -* This program is distributed in the hope that it will be useful, -* but WITHOUT ANY WARRANTY; without even the implied warranty of -* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -* GNU General Public License for more details. -* -* You should have received a copy of the GNU General Public License -* along with this program. If not, see <http://www.gnu.org/licenses/>. */ #ifndef __DMA_IOMMU_H#define __DMA_IOMMU_H
Signature · 1 line
2.20.1
PATCH 23/24 arm64: switch copyright boilerplace to SPDX in dma-mapping.c
In reply to Christoph Hellwig--- arch/arm64/mm/dma-mapping.c | 15 +-------------- 1 file changed, 1 insertion(+), 14 deletions(-)
arch/arm64/mm/dma-mapping.c +1 -14
@@ -1,20 +1,7 @@ +// SPDX-License-Identifier: GPL-2.0 /* -* SWIOTLB-based DMA API implementation -* * Copyright (C) 2012 ARM Ltd. * Author: Catalin Marinas <[email protected]> -* -* This program is free software; you can redistribute it and/or modify -* it under the terms of the GNU General Public License version 2 as -* published by the Free Software Foundation. -* -* This program is distributed in the hope that it will be useful, -* but WITHOUT ANY WARRANTY; without even the implied warranty of -* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -* GNU General Public License for more details. -* -* You should have received a copy of the GNU General Public License -* along with this program. If not, see <http://www.gnu.org/licenses/>. */ #include <linux/gfp.h>
Signature · 1 line
2.20.1
PATCH 24/24 arm64: trim includes in dma-mapping.c
In reply to Christoph HellwigWith most of the previous functionality now elsewhere a lot of the headers included in this file are not needed.
--- arch/arm64/mm/dma-mapping.c | 10 ---------- 1 file changed, 10 deletions(-)
arch/arm64/mm/dma-mapping.c +0 -10
@@ -5,19 +5,9 @@*/ #include <linux/gfp.h> -#include <linux/acpi.h> -#include <linux/memblock.h> #include <linux/cache.h> -#include <linux/export.h> -#include <linux/slab.h> -#include <linux/genalloc.h> -#include <linux/dma-direct.h> #include <linux/dma-noncoherent.h> -#include <linux/dma-contiguous.h> #include <linux/dma-iommu.h> -#include <linux/vmalloc.h> -#include <linux/swiotlb.h> -#include <linux/pci.h> #include <asm/cacheflush.h>
Signature · 1 line
2.20.1
Joerg Roedel
maintainerHi Christoph, Hi Robin,
Quoted 8 lines · Christoph Hellwig
I think we are finally ready for the generic dma-iommu series. I have various DMA API changes pending, and Tom has patches ready to convert the AMD and Intel iommu drivers over to it. I'd love to have this in a stable branch shared between the dma-mapping and iommu trees the day after rc2 is released. I volunteer to create the branch, but I'm fine with it living in the iommu tree as well. Before that Will has already said he wants to send the first patch in the series to Linus for this merge window.
First a big "THANK YOU" for working on this and getting it ready. It is an important step towards generic dma-ops for all iommu drivers, something I wished to have for years and planned to work on myself, but didn't find the time.
I applied this series to a new generic-dma-ops branch in the iommu tree and plan to send it upstream in the next merge window.
Thanks again,
Joerg
Quoted 2 lines · Joerg Roedel
I applied this series to a new generic-dma-ops branch in the iommu tree and plan to send it upstream in the next merge window.
Thanks! Can you push the branch out ASAP, as I'll need to pull it into the dma-mapping tree to avoid creating nasty conflicts for the pending rework of the DMA CMA and contigous allocators?
Joerg Roedel
maintainerQuoted 7 lines · Christoph Hellwig
Quoted 2 lines · Joerg Roedel
I applied this series to a new generic-dma-ops branch in the iommu tree and plan to send it upstream in the next merge window.
Thanks! Can you push the branch out ASAP, as I'll need to pull it into the dma-mapping tree to avoid creating nasty conflicts for the pending rework of the DMA CMA and contigous allocators?
Done, pushed this one branch. Have a lot of fun!
Joerg