Skip to contentSkip to conversation

iommu/dma: free the entire IOVA reservation in dma_iova_destroy()

PATCH In review Honglei Huang 4 participants 7 messages Jul 1 – Jul 3

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 iommu/dma: free the entire IOVA reservation in dma_iova_destroy()

dma_iova_try_alloc() reserves IOVA for the whole requested size and records it in state->__size, but callers may subsequently link only a part of that reservation, for example the drm_gpusvm mixed range case, where a device page range is linked incrementally.

The doc for dma_iova_destroy() is:

"Unlink the IOVA range up to @mapped_len and free the entire IOVA
space."

However __iommu_dma_iova_unlink() computed the amount of IOVA to free from @mapped_len rather than from the full reservation. When the reservation is larger than the linked length, the tail [mapped_len, reserved size] is never returned to the allocator and is leaked, contrary to the documented contract.

Free the whole reservation using dma_iova_size(), mirroring dma_iova_free(). The unmap step still operates on @mapped_len only, and the same iotlb_gather is reused so a single IOTLB flush is performed.

Fixes 433a76207dcf ("dma-mapping: Implement link/unlink ranges API")
---
 drivers/iommu/dma-iommu.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/iommu/dma-iommu.c +4 -1
index 9abaec0703e..bb29c82d1c8 100644
@@ -2096,8 +2096,11 @@ static void __iommu_dma_iova_unlink(struct device *dev,
 
 	if (!iotlb_gather.queued)
 		iommu_iotlb_sync(domain, &iotlb_gather);
-	if (free_iova)
+	if (free_iova) {
+		/* Free the whole reservation, not just the linked @size. */
+		size = iova_align(iovad, dma_iova_size(state) + iova_start_pad);
 		iommu_dma_free_iova(domain, addr, size, &iotlb_gather);
+	}
 }
 
 /**

base-commit: dc59e4fea9d83f03bad6bddf3fa2e52491777482

Signature · 1 line
2.34.1
maintainer
#1 lore raw
Quoted 7 lines · Honglei Huang

dma_iova_try_alloc() reserves IOVA for the whole requested size and records it in state->__size, but callers may subsequently link only a part of that reservation, for example the drm_gpusvm mixed range case, where a device page range is linked incrementally.

The doc for dma_iova_destroy() is:

Quoted 8 lines

"Unlink the IOVA range up to @mapped_len and free the entire IOVA

    space."

However __iommu_dma_iova_unlink() computed the amount of IOVA to free from @mapped_len rather than from the full reservation. When the reservation is larger than the linked length, the tail [mapped_len, reserved size] is never returned to the allocator and is leaked, contrary to the documented contract.

That's not what really happens in practice though - note that free_iova() doesn't even take a size, only a pfn with which to look up the corresponding rbtree entry. At worst, for sizes small enough for the rcaches, a larger IOVA may be put in a cache for a smaller size, which although wasteful, is otherwise pretty much benign.

This isn't to say that the allocator behaviour might not eventually change in future, but for now I can only assume that dma_iova_destroy() doing this is intentional, because I pointed it out at least 3 times over the course of the original review from RFC to eventual merge, and Leon made a point of refusing to do anything about it :/

Thanks,
Robin.

Quoted 30 lines

Free the whole reservation using dma_iova_size(), mirroring dma_iova_free(). The unmap step still operates on @mapped_len only, and the same iotlb_gather is reused so a single IOTLB flush is performed.

Fixes 433a76207dcf ("dma-mapping: Implement link/unlink ranges API")
---
  drivers/iommu/dma-iommu.c | 5 ++++-
  1 file changed, 4 insertions(+), 1 deletion(-)
drivers/iommu/dma-iommu.c +4 -1
index 9abaec0703e..bb29c82d1c8 100644
@@ -2096,8 +2096,11 @@ static void __iommu_dma_iova_unlink(struct device *dev,
 
  	if (!iotlb_gather.queued)
  		iommu_iotlb_sync(domain, &iotlb_gather);
-	if (free_iova)
+	if (free_iova) {
+		/* Free the whole reservation, not just the linked @size. */
+		size = iova_align(iovad, dma_iova_size(state) + iova_start_pad);
  		iommu_dma_free_iova(domain, addr, size, &iotlb_gather);
+	}
  }
 
  /**

base-commit: dc59e4fea9d83f03bad6bddf3fa2e52491777482

#2 lore raw
Quoted 23 lines · Robin Murphy
Quoted 7 lines · Honglei Huang

dma_iova_try_alloc() reserves IOVA for the whole requested size and records it in state->__size, but callers may subsequently link only a part of that reservation, for example the drm_gpusvm mixed range case, where a device page range is linked incrementally.

The doc for dma_iova_destroy() is:

Quoted 8 lines

"Unlink the IOVA range up to @mapped_len and free the entire IOVA

    space."

However __iommu_dma_iova_unlink() computed the amount of IOVA to free from @mapped_len rather than from the full reservation. When the reservation is larger than the linked length, the tail [mapped_len, reserved size] is never returned to the allocator and is leaked, contrary to the documented contract.

That's not what really happens in practice though - note that free_iova() doesn't even take a size, only a pfn with which to look up the corresponding rbtree entry. At worst, for sizes small enough for the rcaches, a larger IOVA may be put in a cache for a smaller size, which although wasteful, is otherwise pretty much benign.

Quoted 5 lines

This isn't to say that the allocator behaviour might not eventually change in future, but for now I can only assume that dma_iova_destroy() doing this is intentional, because I pointed it out at least 3 times over the course of the original review from RFC to eventual merge, and Leon made a point of refusing to do anything about it :/

I found only one sentence that was somewhat unclear in this context. Are you referring to it? """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
https://lore.kernel.org/all/?q=dma_iova_destroy+robin+romanovsky
https://lore.kernel.org/all/[email protected]/

Review

Quoted 2 lines
(quoted diff) +2 -0
+	if (free_iova)
+		iommu_dma_free_iova(cookie, addr, size, &iotlb_gather);

Case in point, can you spot the bug here if dma_iova_destroy() is used as intended? At least it's the relatively benign direction of this bug, not the really fun pagetable corruption one. """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""

Regarding the fix, I think it is the correct one. I intended to free all space previously allocated by iommu_dma_alloc_iova(), not just "mapped_len", as I wrote at the end.

Thanks

#3 lore raw
In reply to Robin Murphy
Quoted 17 lines · Robin Murphy
Quoted 7 lines · Honglei Huang

dma_iova_try_alloc() reserves IOVA for the whole requested size and records it in state->__size, but callers may subsequently link only a part of that reservation, for example the drm_gpusvm mixed range case, where a device page range is linked incrementally.

The doc for dma_iova_destroy() is:

Quoted 8 lines

   "Unlink the IOVA range up to @mapped_len and free the entire IOVA space."

However __iommu_dma_iova_unlink() computed the amount of IOVA to free from @mapped_len rather than from the full reservation. When the reservation is larger than the linked length, the tail [mapped_len, reserved size] is never returned to the allocator and is leaked, contrary to the documented contract.

Quoted 4 lines

That's not what really happens in practice though - note that free_iova() doesn't even take a size, only a pfn with which to look up the corresponding rbtree entry. At worst, for sizes small enough for the rcaches, a larger IOVA may be put in a cache for a smaller size, which although wasteful, is otherwise pretty much benign.

This isn't to say that the allocator behaviour might not eventually change in future, but for now I can only assume that dma_iova_destroy() doing this is intentional, because I pointed it out at least 3 times over the course of the original review from RFC to eventual merge, and Leon made a point of refusing to do anything about it :/

Got it, I checked the code, the current code behavior does not cause a leak. Leon proposed a new modification, will send V2. And I believe is a cleanup rather than a fix.

Regards,
Honglei

Quoted 33 lines

Thanks,
Robin.

Quoted 29 lines

Free the whole reservation using dma_iova_size(), mirroring dma_iova_free(). The unmap step still operates on @mapped_len only, and the same iotlb_gather is reused so a single IOTLB flush is performed.

Fixes 433a76207dcf ("dma-mapping: Implement link/unlink ranges API")

---
  drivers/iommu/dma-iommu.c | 5 ++++-

  1 file changed, 4 insertions(+), 1 deletion(-)
drivers/iommu/dma-iommu.c +4 -1
index 9abaec0703e..bb29c82d1c8 100644
@@ -2096,8 +2096,11 @@ static void __iommu_dma_iova_unlink(struct 
 device *dev,
       if (!iotlb_gather.queued)
           iommu_iotlb_sync(domain, &iotlb_gather);
-    if (free_iova)
+    if (free_iova) {
+        /* Free the whole reservation, not just the linked @size. */
+        size = iova_align(iovad, dma_iova_size(state) + iova_start_pad);
           iommu_dma_free_iova(domain, addr, size, &iotlb_gather);
+    }
   }
   /**
 

base-commit: dc59e4fea9d83f03bad6bddf3fa2e52491777482

Reviewed-by on v1 1/1
#4 lore raw
In reply to Honglei Huang
Quoted 20 lines · Honglei Huang

dma_iova_try_alloc() reserves IOVA for the whole requested size and records it in state->__size, but callers may subsequently link only a part of that reservation, for example the drm_gpusvm mixed range case, where a device page range is linked incrementally.

The doc for dma_iova_destroy() is:

"Unlink the IOVA range up to @mapped_len and free the entire IOVA
space."

However __iommu_dma_iova_unlink() computed the amount of IOVA to free from @mapped_len rather than from the full reservation. When the reservation is larger than the linked length, the tail [mapped_len, reserved size] is never returned to the allocator and is leaked, contrary to the documented contract.

Free the whole reservation using dma_iova_size(), mirroring dma_iova_free(). The unmap step still operates on @mapped_len only, and the same iotlb_gather is reused so a single IOTLB flush is performed.

Quoted 7 lines
Fixes 433a76207dcf ("dma-mapping: Implement link/unlink ranges API")
---
 drivers/iommu/dma-iommu.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

Thanks,

#5 lore raw
In reply to Honglei Huang
Quoted 32 lines · Honglei Huang

dma_iova_try_alloc() reserves IOVA for the whole requested size and records it in state->__size, but callers may subsequently link only a part of that reservation, for example the drm_gpusvm mixed range case, where a device page range is linked incrementally.

The doc for dma_iova_destroy() is:

"Unlink the IOVA range up to @mapped_len and free the entire IOVA
space."

However __iommu_dma_iova_unlink() computed the amount of IOVA to free from @mapped_len rather than from the full reservation. When the reservation is larger than the linked length, the tail [mapped_len, reserved size] is never returned to the allocator and is leaked, contrary to the documented contract.

Free the whole reservation using dma_iova_size(), mirroring dma_iova_free(). The unmap step still operates on @mapped_len only, and the same iotlb_gather is reused so a single IOTLB flush is performed.

Fixes 433a76207dcf ("dma-mapping: Implement link/unlink ranges API")
---
 drivers/iommu/dma-iommu.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/iommu/dma-iommu.c +0 -0
index 9abaec0703e..bb29c82d1c8 100644

drivers/iommu/dma-iommu.c static void __iommu_dma_iova_unlink(struct device *dev, · line 2103

Quoted 9 lines
drivers/iommu/dma-iommu.c +4 -1
@@ -2096,8 +2096,11 @@ static void __iommu_dma_iova_unlink(struct device *dev,
 
 	if (!iotlb_gather.queued)
 		iommu_iotlb_sync(domain, &iotlb_gather);
-	if (free_iova)
+	if (free_iova) {
+		/* Free the whole reservation, not just the linked @size. */
+		size = iova_align(iovad, dma_iova_size(state) + iova_start_pad);
 		iommu_dma_free_iova(domain, addr, size, &iotlb_gather);
+	}

Probably the best change will be something like this:

drivers/iommu/dma-iommu.c +21 -32
index 9abaec0703ef..56173e24c8cc 100644
@@ -2068,10 +2068,20 @@ static void iommu_dma_iova_unlink_range_slow(struct device *dev,
 		arch_sync_dma_flush();
 }
 
-static void __iommu_dma_iova_unlink(struct device *dev,
-		struct dma_iova_state *state, size_t offset, size_t size,
-		enum dma_data_direction dir, unsigned long attrs,
-		bool free_iova)
+/**
+ * dma_iova_unlink - Unlink a range of IOVA space
+ * @dev: DMA device
+ * @state: IOVA state
+ * @offset: offset into the IOVA state to unlink
+ * @size: size of the buffer
+ * @dir: DMA direction
+ * @attrs: attributes of mapping properties
+ *
+ * Unlink a range of IOVA space for the given IOVA state.
+ */
+void dma_iova_unlink(struct device *dev, struct dma_iova_state *state,
+		size_t offset, size_t size, enum dma_data_direction dir,
+		unsigned long attrs)
 {
 	struct iommu_domain *domain = iommu_get_dma_domain(dev);
 	struct iommu_dma_cookie *cookie = domain->iova_cookie;
@@ -2096,26 +2106,6 @@ static void __iommu_dma_iova_unlink(struct device *dev,
 
 	if (!iotlb_gather.queued)
 		iommu_iotlb_sync(domain, &iotlb_gather);
-	if (free_iova)
-		iommu_dma_free_iova(domain, addr, size, &iotlb_gather);
-}
-
-/**
- * dma_iova_unlink - Unlink a range of IOVA space
- * @dev: DMA device
- * @state: IOVA state
- * @offset: offset into the IOVA state to unlink
- * @size: size of the buffer
- * @dir: DMA direction
- * @attrs: attributes of mapping properties
- *
- * Unlink a range of IOVA space for the given IOVA state.
- */
-void dma_iova_unlink(struct device *dev, struct dma_iova_state *state,
-		size_t offset, size_t size, enum dma_data_direction dir,
-		unsigned long attrs)
-{
-	 __iommu_dma_iova_unlink(dev, state, offset, size, dir, attrs, false);
 }
 EXPORT_SYMBOL_GPL(dma_iova_unlink);
 
@@ -2136,14 +2126,13 @@ void dma_iova_destroy(struct device *dev, struct dma_iova_state *state,
 		unsigned long attrs)
 {
 	if (mapped_len)
-		__iommu_dma_iova_unlink(dev, state, 0, mapped_len, dir, attrs,
-				true);
-	else
-		/*
-		 * We can be here if first call to dma_iova_link() failed and
-		 * there is nothing to unlink, so let's be more clear.
-		 */
-		dma_iova_free(dev, state);
+		dma_iova_unlink(dev, state, 0, mapped_len, dir, attrs);
+
+	/*
+	 * We can be here if first call to dma_iova_link() failed and
+	 * there is nothing to unlink, so let's be more clear.
+	 */
+	dma_iova_free(dev, state);
 }
 EXPORT_SYMBOL_GPL(dma_iova_destroy);
 
Quoted 8 lines
drivers/iommu/dma-iommu.c +0 -0
 }
 
 /**

base-commit: dc59e4fea9d83f03bad6bddf3fa2e52491777482

Signature · 1 line
2.34.1
#6 lore raw
Quoted 34 lines · Leon Romanovsky
Quoted 33 lines · Honglei Huang

dma_iova_try_alloc() reserves IOVA for the whole requested size and records it in state->__size, but callers may subsequently link only a part of that reservation, for example the drm_gpusvm mixed range case, where a device page range is linked incrementally.

The doc for dma_iova_destroy() is:

"Unlink the IOVA range up to @mapped_len and free the entire IOVA

    space."

However __iommu_dma_iova_unlink() computed the amount of IOVA to free from @mapped_len rather than from the full reservation. When the reservation is larger than the linked length, the tail [mapped_len, reserved size] is never returned to the allocator and is leaked, contrary to the documented contract.

Free the whole reservation using dma_iova_size(), mirroring dma_iova_free(). The unmap step still operates on @mapped_len only, and the same iotlb_gather is reused so a single IOTLB flush is performed.

Fixes 433a76207dcf ("dma-mapping: Implement link/unlink ranges API")
---
  drivers/iommu/dma-iommu.c | 5 ++++-
  1 file changed, 4 insertions(+), 1 deletion(-)
drivers/iommu/dma-iommu.c +0 -0
index 9abaec0703e..bb29c82d1c8 100644
@@ -2096,8 +2096,11 @@ static void __iommu_dma_iova_unlink(struct device *dev,
 
Quoted 10 lines

drivers/iommu/dma-iommu.c static void __iommu_dma_iova_unlink(struct device *dev, · line 2103

Quoted 8 lines
drivers/iommu/dma-iommu.c +4 -1
  	if (!iotlb_gather.queued)
  		iommu_iotlb_sync(domain, &iotlb_gather);
-	if (free_iova)
+	if (free_iova) {
+		/* Free the whole reservation, not just the linked @size. */
+		size = iova_align(iovad, dma_iova_size(state) + iova_start_pad);
  		iommu_dma_free_iova(domain, addr, size, &iotlb_gather);
+	}

Probably the best change will be something like this:

Thanks for the review and for the shown rework, that's much cleaner than my original approach.

Based on the your code, I have made some modifications to the compilation issues and will send V2.

And the v2 is essentially your diff, I wasn't sure which attribution you'd prefer:

- Suggested-by, what I used in v2
or
- Co-developed-by: + your Signed-off-by.

Just let me know, will adjust in V3.

Regards,
Honglei

Quoted 88 lines
drivers/iommu/dma-iommu.c +21 -32
index 9abaec0703ef..56173e24c8cc 100644
@@ -2068,10 +2068,20 @@ static void iommu_dma_iova_unlink_range_slow(struct device *dev,
  		arch_sync_dma_flush();
  }
 
-static void __iommu_dma_iova_unlink(struct device *dev,
-		struct dma_iova_state *state, size_t offset, size_t size,
-		enum dma_data_direction dir, unsigned long attrs,
-		bool free_iova)
+/**
+ * dma_iova_unlink - Unlink a range of IOVA space
+ * @dev: DMA device
+ * @state: IOVA state
+ * @offset: offset into the IOVA state to unlink
+ * @size: size of the buffer
+ * @dir: DMA direction
+ * @attrs: attributes of mapping properties
+ *
+ * Unlink a range of IOVA space for the given IOVA state.
+ */
+void dma_iova_unlink(struct device *dev, struct dma_iova_state *state,
+		size_t offset, size_t size, enum dma_data_direction dir,
+		unsigned long attrs)
  {
  	struct iommu_domain *domain = iommu_get_dma_domain(dev);
  	struct iommu_dma_cookie *cookie = domain->iova_cookie;
@@ -2096,26 +2106,6 @@ static void __iommu_dma_iova_unlink(struct device *dev,
 
  	if (!iotlb_gather.queued)
  		iommu_iotlb_sync(domain, &iotlb_gather);
-	if (free_iova)
-		iommu_dma_free_iova(domain, addr, size, &iotlb_gather);
-}
-
-/**
- * dma_iova_unlink - Unlink a range of IOVA space
- * @dev: DMA device
- * @state: IOVA state
- * @offset: offset into the IOVA state to unlink
- * @size: size of the buffer
- * @dir: DMA direction
- * @attrs: attributes of mapping properties
- *
- * Unlink a range of IOVA space for the given IOVA state.
- */
-void dma_iova_unlink(struct device *dev, struct dma_iova_state *state,
-		size_t offset, size_t size, enum dma_data_direction dir,
-		unsigned long attrs)
-{
-	 __iommu_dma_iova_unlink(dev, state, offset, size, dir, attrs, false);
  }
  EXPORT_SYMBOL_GPL(dma_iova_unlink);
 
@@ -2136,14 +2126,13 @@ void dma_iova_destroy(struct device *dev, struct dma_iova_state *state,
  		unsigned long attrs)
  {
  	if (mapped_len)
-		__iommu_dma_iova_unlink(dev, state, 0, mapped_len, dir, attrs,
-				true);
-	else
-		/*
-		 * We can be here if first call to dma_iova_link() failed and
-		 * there is nothing to unlink, so let's be more clear.
-		 */
-		dma_iova_free(dev, state);
+		dma_iova_unlink(dev, state, 0, mapped_len, dir, attrs);
+
+	/*
+	 * We can be here if first call to dma_iova_link() failed and
+	 * there is nothing to unlink, so let's be more clear.
+	 */
+	dma_iova_free(dev, state);
  }
  EXPORT_SYMBOL_GPL(dma_iova_destroy);
 
Quoted 9 lines
drivers/iommu/dma-iommu.c +0 -0
  }
 
  /**

base-commit: dc59e4fea9d83f03bad6bddf3fa2e52491777482

Signature · 1 line
2.34.1