Skip to contentSkip to conversation

fix a layering violation in videobuf2 and improve dma_map_resource

PATCH In review Christoph Hellwig 4 participants 16 messages Jan 11, 2019 – Jan 31, 2019

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

Hi all,

this series fixes a rather gross layering violation in videobuf2, which pokes into arm DMA mapping internals to get a DMA address for memory that does not have a page structure, and to do so fixes up the dma_map_resource implementation to be practically useful.

#1 lore raw

PATCH 1/3 dma-mapping: remove the default map_resource implementation

Just returning the physical address when not map_resource method is present is highly dangerous as it doesn't take any offset in the direct mapping into account and does the completely wrong thing for IOMMUs. Instead provide a proper implementation in the direct mapping code, and also wire it up for arm and powerpc.

---
 arch/arm/mm/dma-mapping.c         |  2 ++
 arch/powerpc/kernel/dma-swiotlb.c |  1 +
 arch/powerpc/kernel/dma.c         |  1 +
 include/linux/dma-mapping.h       | 12 +++++++-----
 kernel/dma/direct.c               | 14 ++++++++++++++
 5 files changed, 25 insertions(+), 5 deletions(-)
arch/arm/mm/dma-mapping.c +2 -0
index f1e2922e447c..3c8534904209 100644
@@ -188,6 +188,7 @@ const struct dma_map_ops arm_dma_ops = {
 	.unmap_page		= arm_dma_unmap_page,
 	.map_sg			= arm_dma_map_sg,
 	.unmap_sg		= arm_dma_unmap_sg,
+	.map_resource		= dma_direct_map_resource,
 	.sync_single_for_cpu	= arm_dma_sync_single_for_cpu,
 	.sync_single_for_device	= arm_dma_sync_single_for_device,
 	.sync_sg_for_cpu	= arm_dma_sync_sg_for_cpu,
@@ -211,6 +212,7 @@ const struct dma_map_ops arm_coherent_dma_ops = {
 	.get_sgtable		= arm_dma_get_sgtable,
 	.map_page		= arm_coherent_dma_map_page,
 	.map_sg			= arm_dma_map_sg,
+	.map_resource		= dma_direct_map_resource,
 	.dma_supported		= arm_dma_supported,
 };
 EXPORT_SYMBOL(arm_coherent_dma_ops);
arch/powerpc/kernel/dma-swiotlb.c +1 -0
index 7d5fc9751622..fbb2506a414e 100644
@@ -55,6 +55,7 @@ const struct dma_map_ops powerpc_swiotlb_dma_ops = {
 	.dma_supported = swiotlb_dma_supported,
 	.map_page = dma_direct_map_page,
 	.unmap_page = dma_direct_unmap_page,
+	.map_resource = dma_direct_map_resource,
 	.sync_single_for_cpu = dma_direct_sync_single_for_cpu,
 	.sync_single_for_device = dma_direct_sync_single_for_device,
 	.sync_sg_for_cpu = dma_direct_sync_sg_for_cpu,
arch/powerpc/kernel/dma.c +1 -0
index b1903ebb2e9c..258b9e8ebb99 100644
@@ -273,6 +273,7 @@ const struct dma_map_ops dma_nommu_ops = {
 	.dma_supported			= dma_nommu_dma_supported,
 	.map_page			= dma_nommu_map_page,
 	.unmap_page			= dma_nommu_unmap_page,
+	.map_resource			= dma_direct_map_resource,
 	.get_required_mask		= dma_nommu_get_required_mask,
 #ifdef CONFIG_NOT_COHERENT_CACHE
 	.sync_single_for_cpu 		= dma_nommu_sync_single,
include/linux/dma-mapping.h +7 -5
index cef2127e1d70..d3087829a6df 100644
@@ -208,6 +208,8 @@ dma_addr_t dma_direct_map_page(struct device *dev, struct page *page,
 		unsigned long attrs);
 int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl, int nents,
 		enum dma_data_direction dir, unsigned long attrs);
+dma_addr_t dma_direct_map_resource(struct device *dev, phys_addr_t paddr,
+		size_t size, enum dma_data_direction dir, unsigned long attrs);
 
 #if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
     defined(CONFIG_SWIOTLB)
@@ -346,19 +348,19 @@ static inline dma_addr_t dma_map_resource(struct device *dev,
 					  unsigned long attrs)
 {
 	const struct dma_map_ops *ops = get_dma_ops(dev);
-	dma_addr_t addr;
+	dma_addr_t addr = DMA_MAPPING_ERROR;
 
 	BUG_ON(!valid_dma_direction(dir));
 
 	/* Don't allow RAM to be mapped */
 	BUG_ON(pfn_valid(PHYS_PFN(phys_addr)));
 
-	addr = phys_addr;
-	if (ops && ops->map_resource)
+	if (dma_is_direct(ops))
+		addr = dma_direct_map_resource(dev, phys_addr, size, dir, attrs);
+	else if (ops->map_resource)
 		addr = ops->map_resource(dev, phys_addr, size, dir, attrs);
 
 	debug_dma_map_resource(dev, phys_addr, size, dir, addr);
-
 	return addr;
 }
 
@@ -369,7 +371,7 @@ static inline void dma_unmap_resource(struct device *dev, dma_addr_t addr,
 	const struct dma_map_ops *ops = get_dma_ops(dev);
 
 	BUG_ON(!valid_dma_direction(dir));
-	if (ops && ops->unmap_resource)
+	if (!dma_is_direct(ops) && ops->unmap_resource)
 		ops->unmap_resource(dev, addr, size, dir, attrs);
 	debug_dma_unmap_resource(dev, addr, size, dir);
 }
kernel/dma/direct.c +14 -0
index 355d16acee6d..8e0359b04957 100644
@@ -356,6 +356,20 @@ int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl, int nents,
 }
 EXPORT_SYMBOL(dma_direct_map_sg);
 
+dma_addr_t dma_direct_map_resource(struct device *dev, phys_addr_t paddr,
+		size_t size, enum dma_data_direction dir, unsigned long attrs)
+{
+	dma_addr_t dma_addr = phys_to_dma(dev, paddr);
+
+	if (unlikely(!dma_direct_possible(dev, dma_addr, size))) {
+		report_addr(dev, dma_addr, size);
+		return DMA_MAPPING_ERROR;
+	}
+
+	return dma_addr;
+}
+EXPORT_SYMBOL(dma_direct_map_resource);
+
 /*
  * Because 32-bit DMA masks are so common we expect every architecture to be
  * able to satisfy them - either by not supporting more physical memory, or by
Signature · 1 line
2.20.1
reviewer
#2 lore raw

Re: [PATCH 1/3] dma-mapping: remove the default map_resource implementation

Quoted 5 lines · Christoph Hellwig

Just returning the physical address when not map_resource method is present is highly dangerous as it doesn't take any offset in the direct mapping into account and does the completely wrong thing for IOMMUs. Instead provide a proper implementation in the direct mapping code, and also wire it up for arm and powerpc.

Ignoring the offset was kind of intentional there, because at the time I was primarily thinking about it in terms of the Keystone 2 platform where the peripherals are all in the same place (0-2GB) in both the bus and CPU physical address maps, and only the view of RAM differs between the two (2-4GB vs. 32-34GB). However, on something like BCM283x, the peripherals region is also offset from its bus address in the CPU view, but at a *different* offset relative to that of RAM.

Fortunately, I'm not aware of any platform which has a DMA engine behind an IOMMU (and thus *needs* to use dma_map_resource() to avoid said IOMMU blocking the slave device register reads/writes) and also has any nonzero offsets, and AFAIK the IOMMU-less platforms above aren't using dma_map_resource() at all, so this change shouldn't actually break anything, but I guess we have a bit of a problem making it truly generic and robust :(

Is this perhaps another shove in the direction of overhauling dma_pfn_offset into an arbitrary "DMA ranges" lookup table?

Quoted 74 lines
---
  arch/arm/mm/dma-mapping.c         |  2 ++
  arch/powerpc/kernel/dma-swiotlb.c |  1 +
  arch/powerpc/kernel/dma.c         |  1 +
  include/linux/dma-mapping.h       | 12 +++++++-----
  kernel/dma/direct.c               | 14 ++++++++++++++
  5 files changed, 25 insertions(+), 5 deletions(-)
arch/arm/mm/dma-mapping.c +2 -0
index f1e2922e447c..3c8534904209 100644
@@ -188,6 +188,7 @@ const struct dma_map_ops arm_dma_ops = {
  	.unmap_page		= arm_dma_unmap_page,
  	.map_sg			= arm_dma_map_sg,
  	.unmap_sg		= arm_dma_unmap_sg,
+	.map_resource		= dma_direct_map_resource,
  	.sync_single_for_cpu	= arm_dma_sync_single_for_cpu,
  	.sync_single_for_device	= arm_dma_sync_single_for_device,
  	.sync_sg_for_cpu	= arm_dma_sync_sg_for_cpu,
@@ -211,6 +212,7 @@ const struct dma_map_ops arm_coherent_dma_ops = {
  	.get_sgtable		= arm_dma_get_sgtable,
  	.map_page		= arm_coherent_dma_map_page,
  	.map_sg			= arm_dma_map_sg,
+	.map_resource		= dma_direct_map_resource,
  	.dma_supported		= arm_dma_supported,
  };
  EXPORT_SYMBOL(arm_coherent_dma_ops);
arch/powerpc/kernel/dma-swiotlb.c +1 -0
index 7d5fc9751622..fbb2506a414e 100644
@@ -55,6 +55,7 @@ const struct dma_map_ops powerpc_swiotlb_dma_ops = {
  	.dma_supported = swiotlb_dma_supported,
  	.map_page = dma_direct_map_page,
  	.unmap_page = dma_direct_unmap_page,
+	.map_resource = dma_direct_map_resource,
  	.sync_single_for_cpu = dma_direct_sync_single_for_cpu,
  	.sync_single_for_device = dma_direct_sync_single_for_device,
  	.sync_sg_for_cpu = dma_direct_sync_sg_for_cpu,
arch/powerpc/kernel/dma.c +1 -0
index b1903ebb2e9c..258b9e8ebb99 100644
@@ -273,6 +273,7 @@ const struct dma_map_ops dma_nommu_ops = {
  	.dma_supported			= dma_nommu_dma_supported,
  	.map_page			= dma_nommu_map_page,
  	.unmap_page			= dma_nommu_unmap_page,
+	.map_resource			= dma_direct_map_resource,
  	.get_required_mask		= dma_nommu_get_required_mask,
  #ifdef CONFIG_NOT_COHERENT_CACHE
  	.sync_single_for_cpu 		= dma_nommu_sync_single,
include/linux/dma-mapping.h +3 -1
index cef2127e1d70..d3087829a6df 100644
@@ -208,6 +208,8 @@ dma_addr_t dma_direct_map_page(struct device *dev, struct page *page,
  		unsigned long attrs);
  int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl, int nents,
  		enum dma_data_direction dir, unsigned long attrs);
+dma_addr_t dma_direct_map_resource(struct device *dev, phys_addr_t paddr,
+		size_t size, enum dma_data_direction dir, unsigned long attrs);
 
  #if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
      defined(CONFIG_SWIOTLB)
@@ -346,19 +348,19 @@ static inline dma_addr_t dma_map_resource(struct device *dev,
  					  unsigned long attrs)
  {
  	const struct dma_map_ops *ops = get_dma_ops(dev);
-	dma_addr_t addr;
+	dma_addr_t addr = DMA_MAPPING_ERROR;
 
  	BUG_ON(!valid_dma_direction(dir));

include/linux/dma-mapping.h static inline dma_addr_t dma_map_resource(struct device *dev, · line 361

Quoted 10 lines
include/linux/dma-mapping.h +3 -2
 
  	/* Don't allow RAM to be mapped */
  	BUG_ON(pfn_valid(PHYS_PFN(phys_addr)));
 
-	addr = phys_addr;
-	if (ops && ops->map_resource)
+	if (dma_is_direct(ops))
+		addr = dma_direct_map_resource(dev, phys_addr, size, dir, attrs);
+	else if (ops->map_resource)
  		addr = ops->map_resource(dev, phys_addr, size, dir, attrs);

Might it be reasonable to do:

	if (!dma_is_direct(ops) && ops->map_resource)
		addr = ops->map_resource(...);
	else
		addr = dma_direct_map_resource(...);

and avoid having to explicitly wire up the dma_direct callback elsewhere?

Robin.

Quoted 41 lines
include/linux/dma-mapping.h +1 -2
  	debug_dma_map_resource(dev, phys_addr, size, dir, addr);
-
  	return addr;
  }
 
@@ -369,7 +371,7 @@ static inline void dma_unmap_resource(struct device *dev, dma_addr_t addr,
  	const struct dma_map_ops *ops = get_dma_ops(dev);
 
  	BUG_ON(!valid_dma_direction(dir));
-	if (ops && ops->unmap_resource)
+	if (!dma_is_direct(ops) && ops->unmap_resource)
  		ops->unmap_resource(dev, addr, size, dir, attrs);
  	debug_dma_unmap_resource(dev, addr, size, dir);
  }
kernel/dma/direct.c +14 -0
index 355d16acee6d..8e0359b04957 100644
@@ -356,6 +356,20 @@ int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl, int nents,
  }
  EXPORT_SYMBOL(dma_direct_map_sg);
 
+dma_addr_t dma_direct_map_resource(struct device *dev, phys_addr_t paddr,
+		size_t size, enum dma_data_direction dir, unsigned long attrs)
+{
+	dma_addr_t dma_addr = phys_to_dma(dev, paddr);
+
+	if (unlikely(!dma_direct_possible(dev, dma_addr, size))) {
+		report_addr(dev, dma_addr, size);
+		return DMA_MAPPING_ERROR;
+	}
+
+	return dma_addr;
+}
+EXPORT_SYMBOL(dma_direct_map_resource);
+
  /*
   * Because 32-bit DMA masks are so common we expect every architecture to be
   * able to satisfy them - either by not supporting more physical memory, or by
#3 lore raw

Re: [PATCH 1/3] dma-mapping: remove the default map_resource implementation

Quoted 7 lines · Robin Murphy

Ignoring the offset was kind of intentional there, because at the time I was primarily thinking about it in terms of the Keystone 2 platform where the peripherals are all in the same place (0-2GB) in both the bus and CPU physical address maps, and only the view of RAM differs between the two (2-4GB vs. 32-34GB). However, on something like BCM283x, the peripherals region is also offset from its bus address in the CPU view, but at a *different* offset relative to that of RAM.

I was more thinking of the PCIe P2P case, where we need to apply a consistent offset to translate between the CPU and the bus view.

But this isn't really used for PCIe P2P, so I guess keeping the original sematics might be a better idea. That being said the videobuf code seems to rely on these offsets, so we might be between a rock and a hard place.

Quoted 7 lines

Fortunately, I'm not aware of any platform which has a DMA engine behind an IOMMU (and thus *needs* to use dma_map_resource() to avoid said IOMMU blocking the slave device register reads/writes) and also has any nonzero offsets, and AFAIK the IOMMU-less platforms above aren't using dma_map_resource() at all, so this change shouldn't actually break anything, but I guess we have a bit of a problem making it truly generic and robust :(

Note that we don't actually use the code in this patch for ARM/ARM64 platforms with IOMMUs, as both the ARM and the ARM64 iommu code have their own implementations of ->map_resource that actually program the iommu (which at least for the PCIe P2P case would be wrong).

Quoted 2 lines

Is this perhaps another shove in the direction of overhauling dma_pfn_offset into an arbitrary "DMA ranges" lookup table?

It is long overdue anyway.

Quoted 10 lines
Quoted 1 line
  		addr = ops->map_resource(dev, phys_addr, size, dir, attrs);

Might it be reasonable to do:

	if (!dma_is_direct(ops) && ops->map_resource)
		addr = ops->map_resource(...);
	else
		addr = dma_direct_map_resource(...);

and avoid having to explicitly wire up the dma_direct callback elsewhere?

No, I absolutely _want_ the callback explicitly wired up. That is the only way to ensure we actually intentionally support it and don't just get a default that often won't work. Same issue for ->mmap and ->get_sgtable.

#4 lore raw

PATCH 2/3 dma-mapping: don't BUG when calling dma_map_resource on RAM

In reply to Christoph Hellwig

Use WARN_ON_ONCE to print a stack trace and return a proper error code instead.

---
 include/linux/dma-mapping.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
include/linux/dma-mapping.h +2 -1
index d3087829a6df..91add0751aa5 100644
@@ -353,7 +353,8 @@ static inline dma_addr_t dma_map_resource(struct device *dev,
 	BUG_ON(!valid_dma_direction(dir));
 
 	/* Don't allow RAM to be mapped */
-	BUG_ON(pfn_valid(PHYS_PFN(phys_addr)));
+	if (WARN_ON_ONCE(pfn_valid(PHYS_PFN(phys_addr))))
+		return DMA_MAPPING_ERROR;
 
 	if (dma_is_direct(ops))
 		addr = dma_direct_map_resource(dev, phys_addr, size, dir, attrs);
Signature · 1 line
2.20.1
reviewer Reviewed-by on v1 2/3
#5 lore raw

Re: [PATCH 2/3] dma-mapping: don't BUG when calling dma_map_resource on RAM

Quoted 2 lines · Christoph Hellwig

Use WARN_ON_ONCE to print a stack trace and return a proper error code instead.

I was racking my brain to remember the reasoning behind BUG_ON() being the only viable way to prevent errors getting through unhandled, but of course that was before we had a standardised DMA_MAPPING_ERROR that would work across all implementations.

include/linux/dma-mapping.h static inline dma_addr_t dma_map_resource(struct device *dev, · line 355

Quoted 13 lines
---
  include/linux/dma-mapping.h | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)
include/linux/dma-mapping.h +0 -0
index d3087829a6df..91add0751aa5 100644
@@ -353,7 +353,8 @@ static inline dma_addr_t dma_map_resource(struct device *dev,
  	BUG_ON(!valid_dma_direction(dir));
 
  	/* Don't allow RAM to be mapped */

Ugh, I'm pretty sure that that "pfn_valid means RAM" misunderstanding originally came from me - it might be nice to have a less-misleading comment here, but off-hand I can't think of a succinct way to say "only for 'DMA' access to MMIO registers/SRAMs/etc. and not for anything the kernel knows as actual system/device memory" to better explain the WARN...

Either way, though,

Quoted 7 lines
include/linux/dma-mapping.h +2 -1
-	BUG_ON(pfn_valid(PHYS_PFN(phys_addr)));
+	if (WARN_ON_ONCE(pfn_valid(PHYS_PFN(phys_addr))))
+		return DMA_MAPPING_ERROR;
 
  	if (dma_is_direct(ops))
  		addr = dma_direct_map_resource(dev, phys_addr, size, dir, attrs);
#6 lore raw

PATCH 3/3 videobuf2: replace a layering violation with dma_map_resource

In reply to Christoph Hellwig

vb2_dc_get_userptr pokes into arm direct mapping details to get the resemblance of a dma address for a a physical address that does is not backed by a page struct. Not only is this not portable to other architectures with dma direct mapping offsets, but also not to uses of IOMMUs of any kind. Switch to the proper dma_map_resource / dma_unmap_resource interface instead.

---
 .../common/videobuf2/videobuf2-dma-contig.c   | 41 ++++---------------
 1 file changed, 9 insertions(+), 32 deletions(-)
drivers/media/common/videobuf2/videobuf2-dma-contig.c +9 -32
index aff0ab7bf83d..82389aead6ed 100644
@@ -439,42 +439,14 @@ static void vb2_dc_put_userptr(void *buf_priv)
 				set_page_dirty_lock(pages[i]);
 		sg_free_table(sgt);
 		kfree(sgt);
+	} else {
+		dma_unmap_resource(buf->dev, buf->dma_addr, buf->size,
+				   buf->dma_dir, 0);
 	}
 	vb2_destroy_framevec(buf->vec);
 	kfree(buf);
 }
 
-/*
- * For some kind of reserved memory there might be no struct page available,
- * so all that can be done to support such 'pages' is to try to convert
- * pfn to dma address or at the last resort just assume that
- * dma address == physical address (like it has been assumed in earlier version
- * of videobuf2-dma-contig
- */
-
-#ifdef __arch_pfn_to_dma
-static inline dma_addr_t vb2_dc_pfn_to_dma(struct device *dev, unsigned long pfn)
-{
-	return (dma_addr_t)__arch_pfn_to_dma(dev, pfn);
-}
-#elif defined(__pfn_to_bus)
-static inline dma_addr_t vb2_dc_pfn_to_dma(struct device *dev, unsigned long pfn)
-{
-	return (dma_addr_t)__pfn_to_bus(pfn);
-}
-#elif defined(__pfn_to_phys)
-static inline dma_addr_t vb2_dc_pfn_to_dma(struct device *dev, unsigned long pfn)
-{
-	return (dma_addr_t)__pfn_to_phys(pfn);
-}
-#else
-static inline dma_addr_t vb2_dc_pfn_to_dma(struct device *dev, unsigned long pfn)
-{
-	/* really, we cannot do anything better at this point */
-	return (dma_addr_t)(pfn) << PAGE_SHIFT;
-}
-#endif
-
 static void *vb2_dc_get_userptr(struct device *dev, unsigned long vaddr,
 	unsigned long size, enum dma_data_direction dma_dir)
 {
@@ -528,7 +500,12 @@ static void *vb2_dc_get_userptr(struct device *dev, unsigned long vaddr,
 		for (i = 1; i < n_pages; i++)
 			if (nums[i-1] + 1 != nums[i])
 				goto fail_pfnvec;
-		buf->dma_addr = vb2_dc_pfn_to_dma(buf->dev, nums[0]);
+		buf->dma_addr = dma_map_resource(buf->dev,
+				__pfn_to_phys(nums[0]), size, buf->dma_dir, 0);
+		if (dma_mapping_error(buf->dev, buf->dma_addr)) {
+			ret = -ENOMEM;
+			goto fail_pfnvec;
+		}
 		goto out;
 	}
 
Signature · 1 line
2.20.1
Acked-by on v1 3/3
#7 lore raw

Re: [PATCH 3/3] videobuf2: replace a layering violation with dma_map_resource

Em Fri, 11 Jan 2019 19:17:31 +0100
Christoph Hellwig <[email protected]> escreveu:

Quoted 6 lines

vb2_dc_get_userptr pokes into arm direct mapping details to get the resemblance of a dma address for a a physical address that does is not backed by a page struct. Not only is this not portable to other architectures with dma direct mapping offsets, but also not to uses of IOMMUs of any kind. Switch to the proper dma_map_resource / dma_unmap_resource interface instead.

Makes sense to me. I'm assuming that you'll be pushing it together with other mm patches, so:

Quoted 60 lines
---
 .../common/videobuf2/videobuf2-dma-contig.c   | 41 ++++---------------
 1 file changed, 9 insertions(+), 32 deletions(-)
drivers/media/common/videobuf2/videobuf2-dma-contig.c +3 -31
index aff0ab7bf83d..82389aead6ed 100644
@@ -439,42 +439,14 @@ static void vb2_dc_put_userptr(void *buf_priv)
 				set_page_dirty_lock(pages[i]);
 		sg_free_table(sgt);
 		kfree(sgt);
+	} else {
+		dma_unmap_resource(buf->dev, buf->dma_addr, buf->size,
+				   buf->dma_dir, 0);
 	}
 	vb2_destroy_framevec(buf->vec);
 	kfree(buf);
 }
 
-/*
- * For some kind of reserved memory there might be no struct page available,
- * so all that can be done to support such 'pages' is to try to convert
- * pfn to dma address or at the last resort just assume that
- * dma address == physical address (like it has been assumed in earlier version
- * of videobuf2-dma-contig
- */
-
-#ifdef __arch_pfn_to_dma
-static inline dma_addr_t vb2_dc_pfn_to_dma(struct device *dev, unsigned long pfn)
-{
-	return (dma_addr_t)__arch_pfn_to_dma(dev, pfn);
-}
-#elif defined(__pfn_to_bus)
-static inline dma_addr_t vb2_dc_pfn_to_dma(struct device *dev, unsigned long pfn)
-{
-	return (dma_addr_t)__pfn_to_bus(pfn);
-}
-#elif defined(__pfn_to_phys)
-static inline dma_addr_t vb2_dc_pfn_to_dma(struct device *dev, unsigned long pfn)
-{
-	return (dma_addr_t)__pfn_to_phys(pfn);
-}
-#else
-static inline dma_addr_t vb2_dc_pfn_to_dma(struct device *dev, unsigned long pfn)
-{
-	/* really, we cannot do anything better at this point */
-	return (dma_addr_t)(pfn) << PAGE_SHIFT;
-}
-#endif
-
 static void *vb2_dc_get_userptr(struct device *dev, unsigned long vaddr,
 	unsigned long size, enum dma_data_direction dma_dir)
 {
@@ -528,7 +500,12 @@ static void *vb2_dc_get_userptr(struct device *dev, unsigned long vaddr,
 		for (i = 1; i < n_pages; i++)
 			if (nums[i-1] + 1 != nums[i])
 				goto fail_pfnvec;

drivers/media/common/videobuf2/videobuf2-dma-contig.c static void *vb2_dc_get_userptr(struct device *dev, unsigned long vaddr, · line 511

Quoted 10 lines
drivers/media/common/videobuf2/videobuf2-dma-contig.c +6 -1
-		buf->dma_addr = vb2_dc_pfn_to_dma(buf->dev, nums[0]);
+		buf->dma_addr = dma_map_resource(buf->dev,
+				__pfn_to_phys(nums[0]), size, buf->dma_dir, 0);
+		if (dma_mapping_error(buf->dev, buf->dma_addr)) {
+			ret = -ENOMEM;
+			goto fail_pfnvec;
+		}
 		goto out;
 	}
 

Thanks,
Mauro

#8 lore raw

Re: [PATCH 3/3] videobuf2: replace a layering violation with dma_map_resource

Quoted 10 lines · Mauro Carvalho Chehab

Em Fri, 11 Jan 2019 19:17:31 +0100
Christoph Hellwig <[email protected]> escreveu:

Quoted 6 lines

vb2_dc_get_userptr pokes into arm direct mapping details to get the resemblance of a dma address for a a physical address that does is not backed by a page struct. Not only is this not portable to other architectures with dma direct mapping offsets, but also not to uses of IOMMUs of any kind. Switch to the proper dma_map_resource / dma_unmap_resource interface instead.

Quoted 2 lines

Makes sense to me. I'm assuming that you'll be pushing it together with other mm patches, so:

Not really mm, but rather DMA mapping, but yes, I'd love to take it all together.

Thanks!

#9 lore raw

Re: [PATCH 3/3] videobuf2: replace a layering violation with dma_map_resource

Em Mon, 14 Jan 2019 11:31:39 +0100
Christoph Hellwig <[email protected]> escreveu:

Quoted 11 lines · Mauro Carvalho Chehab
Quoted 10 lines

Em Fri, 11 Jan 2019 19:17:31 +0100
Christoph Hellwig <[email protected]> escreveu:

Quoted 6 lines

vb2_dc_get_userptr pokes into arm direct mapping details to get the resemblance of a dma address for a a physical address that does is not backed by a page struct. Not only is this not portable to other architectures with dma direct mapping offsets, but also not to uses of IOMMUs of any kind. Switch to the proper dma_map_resource / dma_unmap_resource interface instead.

Quoted 5 lines
Quoted 2 lines

Makes sense to me. I'm assuming that you'll be pushing it together with other mm patches, so:

Not really mm, but rather DMA mapping, but yes, I'd love to take it all together.

Ah, OK! Anyway, feel free to place it altogether.

It would be good if you could later send us a stable branch where you merged, in order to allow us to run some tests with the new DMA mapping patchset and be sure that it won't cause regressions to videobuf2.

Thank you!

Mauro

#10 lore raw

Re: [PATCH 3/3] videobuf2: replace a layering violation with dma_map_resource

Quoted 4 lines · Mauro Carvalho Chehab

It would be good if you could later send us a stable branch where you merged, in order to allow us to run some tests with the new DMA mapping patchset and be sure that it won't cause regressions to videobuf2.

I can do that, but the series as sent should be testable, results welcome!

maintainer Tested-by on v1 3/3
#11 lore raw

Re: [PATCH 3/3] videobuf2: replace a layering violation with dma_map_resource

In reply to Christoph Hellwig

Hi Christoph,

Quoted 8 lines · Christoph Hellwig

vb2_dc_get_userptr pokes into arm direct mapping details to get the resemblance of a dma address for a a physical address that does is not backed by a page struct. Not only is this not portable to other architectures with dma direct mapping offsets, but also not to uses of IOMMUs of any kind. Switch to the proper dma_map_resource / dma_unmap_resource interface instead.

There are checks for IOMMU presence in other places in vb2-dma-contig, so it was used only when no IOMMU is available, but I agree that the hacky code should be replaced by a generic code if possible.

V4L2 pipeline works fine on older Exynos-based boards with CMA and IOMMU disabled.

Quoted 58 lines
---
 .../common/videobuf2/videobuf2-dma-contig.c   | 41 ++++---------------
 1 file changed, 9 insertions(+), 32 deletions(-)
drivers/media/common/videobuf2/videobuf2-dma-contig.c +3 -31
index aff0ab7bf83d..82389aead6ed 100644
@@ -439,42 +439,14 @@ static void vb2_dc_put_userptr(void *buf_priv)
 				set_page_dirty_lock(pages[i]);
 		sg_free_table(sgt);
 		kfree(sgt);
+	} else {
+		dma_unmap_resource(buf->dev, buf->dma_addr, buf->size,
+				   buf->dma_dir, 0);
 	}
 	vb2_destroy_framevec(buf->vec);
 	kfree(buf);
 }
 
-/*
- * For some kind of reserved memory there might be no struct page available,
- * so all that can be done to support such 'pages' is to try to convert
- * pfn to dma address or at the last resort just assume that
- * dma address == physical address (like it has been assumed in earlier version
- * of videobuf2-dma-contig
- */
-
-#ifdef __arch_pfn_to_dma
-static inline dma_addr_t vb2_dc_pfn_to_dma(struct device *dev, unsigned long pfn)
-{
-	return (dma_addr_t)__arch_pfn_to_dma(dev, pfn);
-}
-#elif defined(__pfn_to_bus)
-static inline dma_addr_t vb2_dc_pfn_to_dma(struct device *dev, unsigned long pfn)
-{
-	return (dma_addr_t)__pfn_to_bus(pfn);
-}
-#elif defined(__pfn_to_phys)
-static inline dma_addr_t vb2_dc_pfn_to_dma(struct device *dev, unsigned long pfn)
-{
-	return (dma_addr_t)__pfn_to_phys(pfn);
-}
-#else
-static inline dma_addr_t vb2_dc_pfn_to_dma(struct device *dev, unsigned long pfn)
-{
-	/* really, we cannot do anything better at this point */
-	return (dma_addr_t)(pfn) << PAGE_SHIFT;
-}
-#endif
-
 static void *vb2_dc_get_userptr(struct device *dev, unsigned long vaddr,
 	unsigned long size, enum dma_data_direction dma_dir)
 {
@@ -528,7 +500,12 @@ static void *vb2_dc_get_userptr(struct device *dev, unsigned long vaddr,
 		for (i = 1; i < n_pages; i++)
 			if (nums[i-1] + 1 != nums[i])
 				goto fail_pfnvec;

drivers/media/common/videobuf2/videobuf2-dma-contig.c static void *vb2_dc_get_userptr(struct device *dev, unsigned long vaddr, · line 511

Quoted 10 lines
drivers/media/common/videobuf2/videobuf2-dma-contig.c +6 -1
-		buf->dma_addr = vb2_dc_pfn_to_dma(buf->dev, nums[0]);
+		buf->dma_addr = dma_map_resource(buf->dev,
+				__pfn_to_phys(nums[0]), size, buf->dma_dir, 0);
+		if (dma_mapping_error(buf->dev, buf->dma_addr)) {
+			ret = -ENOMEM;
+			goto fail_pfnvec;
+		}
 		goto out;
 	}
 

Best regards

Signature · 2 lines
Marek Szyprowski, PhD
Samsung R&D Institute Poland
#12 lore raw

Re: [PATCH 3/3] videobuf2: replace a layering violation with dma_map_resource

Quoted 12 lines · Marek Szyprowski

Hi Christoph,

Quoted 8 lines · Christoph Hellwig

vb2_dc_get_userptr pokes into arm direct mapping details to get the resemblance of a dma address for a a physical address that does is not backed by a page struct. Not only is this not portable to other architectures with dma direct mapping offsets, but also not to uses of IOMMUs of any kind. Switch to the proper dma_map_resource / dma_unmap_resource interface instead.

Quoted 8 lines

There are checks for IOMMU presence in other places in vb2-dma-contig, so it was used only when no IOMMU is available, but I agree that the hacky code should be replaced by a generic code if possible.

V4L2 pipeline works fine on older Exynos-based boards with CMA and IOMMU disabled.

Do you know if these rely on the offsets? E.g. would they still work with the patch below applied on top. That would keep the map_resource semantics as-is as solve the issue pointed out by Robin for now.

If not I can only think of a flag to bypass the offseting for now, but that would be pretty ugly. Or go for the long-term solution of discovering the relationship between the two devices, as done by the PCIe P2P code..

kernel/dma/direct.c +1 -1
index 8e0359b04957..25bd19974223 100644
@@ -359,7 +359,7 @@ EXPORT_SYMBOL(dma_direct_map_sg);
 dma_addr_t dma_direct_map_resource(struct device *dev, phys_addr_t paddr,
 		size_t size, enum dma_data_direction dir, unsigned long attrs)
 {
-	dma_addr_t dma_addr = phys_to_dma(dev, paddr);
+	dma_addr_t dma_addr = paddr;
 
 	if (unlikely(!dma_direct_possible(dev, dma_addr, size))) {
 		report_addr(dev, dma_addr, size);
maintainer
#13 lore raw

Re: [PATCH 3/3] videobuf2: replace a layering violation with dma_map_resource

Hi Christoph,

Quoted 14 lines · Christoph Hellwig
Quoted 13 lines · Marek Szyprowski
Quoted 8 lines · Christoph Hellwig

vb2_dc_get_userptr pokes into arm direct mapping details to get the resemblance of a dma address for a a physical address that does is not backed by a page struct. Not only is this not portable to other architectures with dma direct mapping offsets, but also not to uses of IOMMUs of any kind. Switch to the proper dma_map_resource / dma_unmap_resource interface instead.

There are checks for IOMMU presence in other places in vb2-dma-contig, so it was used only when no IOMMU is available, but I agree that the hacky code should be replaced by a generic code if possible.

Quoted 7 lines
Quoted 4 lines

V4L2 pipeline works fine on older Exynos-based boards with CMA and IOMMU disabled.

Do you know if these rely on the offsets? E.g. would they still work with the patch below applied on top. That would keep the map_resource semantics as-is as solve the issue pointed out by Robin for now.

AFAIK that code was only used for sharing buffers between hardware modules that are a part of the same SoC, usually implemented as platform devices. AFAIR this never worked for devices on different buses. So far I wasn't aware on ANY which would require an offset for the DMA access.

The first version of videobuf2-dc code even incorrectly used paddr instead of dma_addr as a buffer 'address' returned to the client drivers, because in case of those SoC this was exactly the same (see commits 472af2b05bdefcaee7e754e22cbf131110017ad6 and ba7fcb0c954921534707f08ebc4d8beeb2eb17e7).

Quoted 13 lines

If not I can only think of a flag to bypass the offseting for now, but that would be pretty ugly. Or go for the long-term solution of discovering the relationship between the two devices, as done by the PCIe P2P code..

kernel/dma/direct.c +0 -0
index 8e0359b04957..25bd19974223 100644

kernel/dma/direct.c EXPORT_SYMBOL(dma_direct_map_sg); · line 365

Quoted 8 lines
kernel/dma/direct.c +1 -1
@@ -359,7 +359,7 @@ EXPORT_SYMBOL(dma_direct_map_sg);
 dma_addr_t dma_direct_map_resource(struct device *dev, phys_addr_t paddr,
 		size_t size, enum dma_data_direction dir, unsigned long attrs)
 {
-	dma_addr_t dma_addr = phys_to_dma(dev, paddr);
+	dma_addr_t dma_addr = paddr;
 
 	if (unlikely(!dma_direct_possible(dev, dma_addr, size))) {
 		report_addr(dev, dma_addr, size);

Best regards

Signature · 2 lines
Marek Szyprowski, PhD
Samsung R&D Institute Poland
#14 lore raw

Re: [PATCH 3/3] videobuf2: replace a layering violation with dma_map_resource

Hi Marek,

can chance you could retest the v2 version?

maintainer Reviewed-by
#15 lore raw
In reply to Christoph Hellwig

Hi Christoph,

Quoted 6 lines · Christoph Hellwig

Hi all,

this series fixes a rather gross layering violation in videobuf2, which pokes into arm DMA mapping internals to get a DMA address for memory that does not have a page structure, and to do so fixes up the dma_map_resource implementation to be practically useful.

Thanks for rewriting this 'temporary code'! It predates dma_map_resource() and that time this was the only way to get it working somehow. Good that now it is possible to implement in it a clean way without any unwritten assumptions about the DMA mapping internals. Feel free to add my:

Best regards

Signature · 2 lines
Marek Szyprowski, PhD
Samsung R&D Institute Poland