Skip to contentSkip to conversation

drm/panfrost: DMA map all pages shared with the GPU

PATCH In review Steven Price 2 participants 4 messages Oct 14, 2019

Subsystems (3)
lore
Share & tools
lore

Steven Price

author maintainer
#0 lore raw

PATCH 1/1 drm/panfrost: DMA map all pages shared with the GPU

Pages shared with the GPU are (often) not cache coherent with the CPU so cache maintenance is required to flush the CPU's caches. This was already done when mapping pages on fault, but wasn't previously done when mapping a freshly allocated page.

Fix this by moving the call to dma_map_sg() into mmu_map_sg() ensuring that it is always called when pages are mapped onto the GPU. Since mmu_map_sg() can now fail the code also now has to handle an error return.

Not performing this cache maintenance can cause errors in the GPU output (CPU caches are later flushed and overwrite the GPU output). In theory it also allows the GPU (and by extension user space) to observe the memory contents prior to sanitization.

Fixes f3ba91228e8e ("drm/panfrost: Add initial panfrost driver")
---
 drivers/gpu/drm/panfrost/panfrost_mmu.c | 20 ++++++++++++--------
 1 file changed, 12 insertions(+), 8 deletions(-)
drivers/gpu/drm/panfrost/panfrost_mmu.c +12 -8
index bdd990568476..0495e48c238d 100644
@@ -248,6 +248,9 @@ static int mmu_map_sg(struct panfrost_device *pfdev, struct panfrost_mmu *mmu,
 	struct io_pgtable_ops *ops = mmu->pgtbl_ops;
 	u64 start_iova = iova;
 
+	if (!dma_map_sg(pfdev->dev, sgt->sgl, sgt->nents, DMA_BIDIRECTIONAL))
+		return -EINVAL;
+
 	for_each_sg(sgt->sgl, sgl, sgt->nents, count) {
 		unsigned long paddr = sg_dma_address(sgl);
 		size_t len = sg_dma_len(sgl);
@@ -275,6 +278,7 @@ int panfrost_mmu_map(struct panfrost_gem_object *bo)
 	struct panfrost_device *pfdev = to_panfrost_device(obj->dev);
 	struct sg_table *sgt;
 	int prot = IOMMU_READ | IOMMU_WRITE;
+	int ret;
 
 	if (WARN_ON(bo->is_mapped))
 		return 0;
@@ -286,10 +290,12 @@ int panfrost_mmu_map(struct panfrost_gem_object *bo)
 	if (WARN_ON(IS_ERR(sgt)))
 		return PTR_ERR(sgt);
 
-	mmu_map_sg(pfdev, bo->mmu, bo->node.start << PAGE_SHIFT, prot, sgt);
-	bo->is_mapped = true;
+	ret = mmu_map_sg(pfdev, bo->mmu, bo->node.start << PAGE_SHIFT,
+			 prot, sgt);
+	if (ret == 0)
+		bo->is_mapped = true;
 
-	return 0;
+	return ret;
 }
 
 void panfrost_mmu_unmap(struct panfrost_gem_object *bo)
@@ -503,12 +509,10 @@ int panfrost_mmu_map_fault_addr(struct panfrost_device *pfdev, int as, u64 addr)
 	if (ret)
 		goto err_pages;
 
-	if (!dma_map_sg(pfdev->dev, sgt->sgl, sgt->nents, DMA_BIDIRECTIONAL)) {
-		ret = -EINVAL;
+	ret = mmu_map_sg(pfdev, bo->mmu, addr,
+			 IOMMU_WRITE | IOMMU_READ | IOMMU_NOEXEC, sgt);
+	if (ret)
 		goto err_map;
-	}
-
-	mmu_map_sg(pfdev, bo->mmu, addr, IOMMU_WRITE | IOMMU_READ | IOMMU_NOEXEC, sgt);
 
 	bo->is_mapped = true;
 
Signature · 1 line
2.20.1
#1 lore raw
Quoted 10 lines · Steven Price

Pages shared with the GPU are (often) not cache coherent with the CPU so cache maintenance is required to flush the CPU's caches. This was already done when mapping pages on fault, but wasn't previously done when mapping a freshly allocated page.

Fix this by moving the call to dma_map_sg() into mmu_map_sg() ensuring that it is always called when pages are mapped onto the GPU. Since mmu_map_sg() can now fail the code also now has to handle an error return.

Quoted 4 lines

Not performing this cache maintenance can cause errors in the GPU output (CPU caches are later flushed and overwrite the GPU output). In theory it also allows the GPU (and by extension user space) to observe the memory contents prior to sanitization.

For the non-heap case, aren't the pages already supposed to be mapped by drm_gem_shmem_get_pages_sgt()?

(Hmm, maybe I should try hooking up the GPU SMMU on my Juno to serve as a cheeky DMA-API-mishap detector...)

Robin.

Quoted 61 lines
Fixes f3ba91228e8e ("drm/panfrost: Add initial panfrost driver")
---
  drivers/gpu/drm/panfrost/panfrost_mmu.c | 20 ++++++++++++--------
  1 file changed, 12 insertions(+), 8 deletions(-)
drivers/gpu/drm/panfrost/panfrost_mmu.c +12 -8
index bdd990568476..0495e48c238d 100644
@@ -248,6 +248,9 @@ static int mmu_map_sg(struct panfrost_device *pfdev, struct panfrost_mmu *mmu,
  	struct io_pgtable_ops *ops = mmu->pgtbl_ops;
  	u64 start_iova = iova;
 
+	if (!dma_map_sg(pfdev->dev, sgt->sgl, sgt->nents, DMA_BIDIRECTIONAL))
+		return -EINVAL;
+
  	for_each_sg(sgt->sgl, sgl, sgt->nents, count) {
  		unsigned long paddr = sg_dma_address(sgl);
  		size_t len = sg_dma_len(sgl);
@@ -275,6 +278,7 @@ int panfrost_mmu_map(struct panfrost_gem_object *bo)
  	struct panfrost_device *pfdev = to_panfrost_device(obj->dev);
  	struct sg_table *sgt;
  	int prot = IOMMU_READ | IOMMU_WRITE;
+	int ret;
 
  	if (WARN_ON(bo->is_mapped))
  		return 0;
@@ -286,10 +290,12 @@ int panfrost_mmu_map(struct panfrost_gem_object *bo)
  	if (WARN_ON(IS_ERR(sgt)))
  		return PTR_ERR(sgt);
 
-	mmu_map_sg(pfdev, bo->mmu, bo->node.start << PAGE_SHIFT, prot, sgt);
-	bo->is_mapped = true;
+	ret = mmu_map_sg(pfdev, bo->mmu, bo->node.start << PAGE_SHIFT,
+			 prot, sgt);
+	if (ret == 0)
+		bo->is_mapped = true;
 
-	return 0;
+	return ret;
  }
 
  void panfrost_mmu_unmap(struct panfrost_gem_object *bo)
@@ -503,12 +509,10 @@ int panfrost_mmu_map_fault_addr(struct panfrost_device *pfdev, int as, u64 addr)
  	if (ret)
  		goto err_pages;
 
-	if (!dma_map_sg(pfdev->dev, sgt->sgl, sgt->nents, DMA_BIDIRECTIONAL)) {
-		ret = -EINVAL;
+	ret = mmu_map_sg(pfdev, bo->mmu, addr,
+			 IOMMU_WRITE | IOMMU_READ | IOMMU_NOEXEC, sgt);
+	if (ret)
  		goto err_map;
-	}
-
-	mmu_map_sg(pfdev, bo->mmu, addr, IOMMU_WRITE | IOMMU_READ | IOMMU_NOEXEC, sgt);
 
  	bo->is_mapped = true;
 
author maintainer
#2 lore raw
Quoted 11 lines · Robin Murphy
Quoted 10 lines · Steven Price

Pages shared with the GPU are (often) not cache coherent with the CPU so cache maintenance is required to flush the CPU's caches. This was already done when mapping pages on fault, but wasn't previously done when mapping a freshly allocated page.

Fix this by moving the call to dma_map_sg() into mmu_map_sg() ensuring that it is always called when pages are mapped onto the GPU. Since mmu_map_sg() can now fail the code also now has to handle an error return.

Quoted 7 lines
Quoted 4 lines

Not performing this cache maintenance can cause errors in the GPU output (CPU caches are later flushed and overwrite the GPU output). In theory it also allows the GPU (and by extension user space) to observe the memory contents prior to sanitization.

For the non-heap case, aren't the pages already supposed to be mapped by drm_gem_shmem_get_pages_sgt()?

Hmm, well yes - it looks like it *should* work - but I was definitely seeing cache artefacts until I did this change... let me do some more testing. It's possible that this is actually only affecting buffers imported from another driver. Perhaps it's drm_gem_shmem_prime_import_sg_table() that needs fixing.

Quoted 2 lines

(Hmm, maybe I should try hooking up the GPU SMMU on my Juno to serve as a cheeky DMA-API-mishap detector...)

Yes I was wondering about that - it would certainly give some confidence there's no missing cases.

Steve

Quoted 67 lines

Robin.

Quoted 59 lines
Fixes f3ba91228e8e ("drm/panfrost: Add initial panfrost driver")

---
  drivers/gpu/drm/panfrost/panfrost_mmu.c | 20 ++++++++++++--------

  1 file changed, 12 insertions(+), 8 deletions(-)
drivers/gpu/drm/panfrost/panfrost_mmu.c +2 -0
index bdd990568476..0495e48c238d 100644
@@ -248,6 +248,9 @@ static int mmu_map_sg(struct panfrost_device
 *pfdev, struct panfrost_mmu *mmu,
       struct io_pgtable_ops *ops = mmu->pgtbl_ops;
       u64 start_iova = iova;
   +    if (!dma_map_sg(pfdev->dev, sgt->sgl, sgt->nents,
 DMA_BIDIRECTIONAL))
+        return -EINVAL;
+
       for_each_sg(sgt->sgl, sgl, sgt->nents, count) {

          unsigned long paddr = sg_dma_address(sgl); size_t len = sg_dma_len(sgl);

drivers/gpu/drm/panfrost/panfrost_mmu.c +6 -1
@@ -275,6 +278,7 @@ int panfrost_mmu_map(struct panfrost_gem_object *bo)
       struct panfrost_device *pfdev = to_panfrost_device(obj->dev);
       struct sg_table *sgt;
       int prot = IOMMU_READ | IOMMU_WRITE;
+    int ret;
         if (WARN_ON(bo->is_mapped))
           return 0;
@@ -286,10 +290,12 @@ int panfrost_mmu_map(struct panfrost_gem_object
 *bo)
       if (WARN_ON(IS_ERR(sgt)))
           return PTR_ERR(sgt);
   -    mmu_map_sg(pfdev, bo->mmu, bo->node.start << PAGE_SHIFT, prot,
 sgt);
-    bo->is_mapped = true;
+    ret = mmu_map_sg(pfdev, bo->mmu, bo->node.start << PAGE_SHIFT,
+             prot, sgt);
+    if (ret == 0)
+        bo->is_mapped = true;
   -    return 0;
+    return ret;
   }

    void panfrost_mmu_unmap(struct panfrost_gem_object *bo)

drivers/gpu/drm/panfrost/panfrost_mmu.c +3 -4
@@ -503,12 +509,10 @@ int panfrost_mmu_map_fault_addr(struct
 panfrost_device *pfdev, int as, u64 addr)
       if (ret)
           goto err_pages;
   -    if (!dma_map_sg(pfdev->dev, sgt->sgl, sgt->nents,
 DMA_BIDIRECTIONAL)) {
-        ret = -EINVAL;
+    ret = mmu_map_sg(pfdev, bo->mmu, addr,
+             IOMMU_WRITE | IOMMU_READ | IOMMU_NOEXEC, sgt);
+    if (ret)
           goto err_map;
-    }
-
-    mmu_map_sg(pfdev, bo->mmu, addr, IOMMU_WRITE | IOMMU_READ |
 IOMMU_NOEXEC, sgt);
Quoted 2 lines

        bo->is_mapped = true;

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

author maintainer
#3 lore raw
Quoted 17 lines · Steven Price
Quoted 16 lines · Robin Murphy
Quoted 14 lines · Steven Price

Pages shared with the GPU are (often) not cache coherent with the CPU so cache maintenance is required to flush the CPU's caches. This was already done when mapping pages on fault, but wasn't previously done when mapping a freshly allocated page.

Fix this by moving the call to dma_map_sg() into mmu_map_sg() ensuring that it is always called when pages are mapped onto the GPU. Since mmu_map_sg() can now fail the code also now has to handle an error return.

Not performing this cache maintenance can cause errors in the GPU output (CPU caches are later flushed and overwrite the GPU output). In theory it also allows the GPU (and by extension user space) to observe the memory contents prior to sanitization.

Quoted 8 lines
Quoted 2 lines

For the non-heap case, aren't the pages already supposed to be mapped by drm_gem_shmem_get_pages_sgt()?

Hmm, well yes - it looks like it *should* work - but I was definitely seeing cache artefacts until I did this change... let me do some more testing. It's possible that this is actually only affecting buffers imported from another driver. Perhaps it's drm_gem_shmem_prime_import_sg_table() that needs fixing.

Yes this does appear to only affect imported buffers from what I can tell. Looking through the code there is something suspicious in drm_gem_map_dma_buf(). The following "fixes" the problem. But I'm not sure the reasoning behind DMA_ATTR_SKIP_CPU_SYNC being specified - presumably someone though it was a good idea! I'm not sure which driver's responsibility it is to ensure the caches are flushed.

---8<----

drivers/gpu/drm/drm_prime.c +1 -1
index 0a2316e0e812..1f7353abd255 100644
@@ -625,7 +625,7 @@ struct sg_table *drm_gem_map_dma_buf(struct dma_buf_attachment *attach,
 		sgt = obj->dev->driver->gem_prime_get_sg_table(obj);
 
 	if (!dma_map_sg_attrs(attach->dev, sgt->sgl, sgt->nents, dir,
-			      DMA_ATTR_SKIP_CPU_SYNC)) {
+			      0)) {
 		sg_free_table(sgt);
 		kfree(sgt);
 		sgt = ERR_PTR(-ENOMEM);

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel