iommu/dma: free the entire IOVA reservation in dma_iova_destroy()
Subsystems (2)
PATCH 1/1 iommu/dma: free the entire IOVA reservation in dma_iova_destroy()
dma_iova_try_alloc() reserves IOVA for the whole requested size and records it in state->__size, but callers may subsequently link only a part of that reservation, for example the drm_gpusvm mixed range case, where a device page range is linked incrementally.
The doc for dma_iova_destroy() is:
"Unlink the IOVA range up to @mapped_len and free the entire IOVA
space."
However __iommu_dma_iova_unlink() computed the amount of IOVA to free from @mapped_len rather than from the full reservation. When the reservation is larger than the linked length, the tail [mapped_len, reserved size] is never returned to the allocator and is leaked, contrary to the documented contract.
Free the whole reservation using dma_iova_size(), mirroring dma_iova_free(). The unmap step still operates on @mapped_len only, and the same iotlb_gather is reused so a single IOTLB flush is performed.
--- drivers/iommu/dma-iommu.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/iommu/dma-iommu.c +4 -1
@@ -2096,8 +2096,11 @@ static void __iommu_dma_iova_unlink(struct device *dev,if (!iotlb_gather.queued)iommu_iotlb_sync (domain, &iotlb_gather); -if (free_iova) +if (free_iova) { +/* Free the whole reservation, not just the linked @size. */ + size =iova_align (iovad,dma_iova_size (state) + iova_start_pad);iommu_dma_free_iova (domain, addr, size, &iotlb_gather); + } }/**
base-commit: dc59e4fea9d83f03bad6bddf3fa2e52491777482
Signature · 1 line
2.34.1
Robin Murphy
maintainerQuoted 7 lines · Honglei Huang
dma_iova_try_alloc()reserves IOVA for the whole requested size and records it instate->__size, but callers may subsequently link only a part of that reservation, for example thedrm_gpusvmmixed range case, where a device page range is linked incrementally.The doc for
dma_iova_destroy()is:
Quoted 8 lines
"Unlink the IOVA range up to @mapped_len and free the entire IOVA
space."However
__iommu_dma_iova_unlink()computed the amount of IOVA to free from @mapped_len rather than from the full reservation. When the reservation is larger than the linked length, the tail [mapped_len, reserved size] is never returned to the allocator and is leaked, contrary to the documented contract.
That's not what really happens in practice though - note that free_iova() doesn't even take a size, only a pfn with which to look up the corresponding rbtree entry. At worst, for sizes small enough for the rcaches, a larger IOVA may be put in a cache for a smaller size, which although wasteful, is otherwise pretty much benign.
This isn't to say that the allocator behaviour might not eventually change in future, but for now I can only assume that dma_iova_destroy() doing this is intentional, because I pointed it out at least 3 times over the course of the original review from RFC to eventual merge, and Leon made a point of refusing to do anything about it :/
Thanks,
Robin.
Quoted 30 lines
Free the whole reservation using
dma_iova_size(), mirroringdma_iova_free(). The unmap step still operates on @mapped_len only, and the sameiotlb_gatheris reused so a single IOTLB flush is performed.Fixes 433a76207dcf ("dma-mapping: Implement link/unlink ranges API")Signed-off-by Honglei Huang <[email protected]>--- drivers/iommu/dma-iommu.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)drivers/iommu/dma-iommu.c +4
-1@@ -2096,8 +2096,11 @@ static void __iommu_dma_iova_unlink(struct device *dev,if (!iotlb_gather.queued)iommu_iotlb_sync (domain, &iotlb_gather); -if (free_iova) +if (free_iova) { +/* Free the whole reservation, not just the linked @size. */ + size =iova_align (iovad,dma_iova_size (state) + iova_start_pad);iommu_dma_free_iova (domain, addr, size, &iotlb_gather); + } }/** base-commit:
dc59e4fea9d83f03bad6bddf3fa2e52491777482
Quoted 23 lines · Robin Murphy
Quoted 7 lines · Honglei Huang
dma_iova_try_alloc()reserves IOVA for the whole requested size and records it instate->__size, but callers may subsequently link only a part of that reservation, for example thedrm_gpusvmmixed range case, where a device page range is linked incrementally.The doc for
dma_iova_destroy()is:Quoted 8 lines
"Unlink the IOVA range up to @mapped_len and free the entire IOVA
space."However
__iommu_dma_iova_unlink()computed the amount of IOVA to free from @mapped_len rather than from the full reservation. When the reservation is larger than the linked length, the tail [mapped_len, reserved size] is never returned to the allocator and is leaked, contrary to the documented contract.That's not what really happens in practice though - note that
free_iova()doesn't even take a size, only a pfn with which to look up the corresponding rbtree entry. At worst, for sizes small enough for the rcaches, a larger IOVA may be put in a cache for a smaller size, which although wasteful, is otherwise pretty much benign.
Quoted 5 lines
This isn't to say that the allocator behaviour might not eventually change in future, but for now I can only assume that
dma_iova_destroy()doing this is intentional, because I pointed it out at least 3 times over the course of the original review from RFC to eventual merge, and Leon made a point of refusing to do anything about it :/
I found only one sentence that was somewhat unclear in this context. Are you referring to it? """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
https://lore.kernel.org/all/?q=dma_iova_destroy+robin+romanovsky
https://lore.kernel.org/all/[email protected]/
Review
Quoted 2 lines
(quoted diff) +2
-0+ if (free_iova) + iommu_dma_free_iova(cookie, addr, size, &iotlb_gather);
Case in point, can you spot the bug here if dma_iova_destroy() is used as intended? At least it's the relatively benign direction of this bug, not the really fun pagetable corruption one. """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Regarding the fix, I think it is the correct one. I intended to free all space previously allocated by iommu_dma_alloc_iova(), not just "mapped_len", as I wrote at the end.
Thanks
Quoted 17 lines · Robin Murphy
Quoted 7 lines · Honglei Huang
dma_iova_try_alloc()reserves IOVA for the whole requested size and records it instate->__size, but callers may subsequently link only a part of that reservation, for example thedrm_gpusvmmixed range case, where a device page range is linked incrementally.The doc for
dma_iova_destroy()is:Quoted 8 lines
"Unlink the IOVA range up to @mapped_len and free the entire IOVA space."
However
__iommu_dma_iova_unlink()computed the amount of IOVA to free from @mapped_len rather than from the full reservation. When the reservation is larger than the linked length, the tail [mapped_len, reserved size] is never returned to the allocator and is leaked, contrary to the documented contract.
Quoted 4 lines
That's not what really happens in practice though - note that
free_iova()doesn't even take a size, only a pfn with which to look up the corresponding rbtree entry. At worst, for sizes small enough for the rcaches, a larger IOVA may be put in a cache for a smaller size, which although wasteful, is otherwise pretty much benign.This isn't to say that the allocator behaviour might not eventually change in future, but for now I can only assume that
dma_iova_destroy()doing this is intentional, because I pointed it out at least 3 times over the course of the original review from RFC to eventual merge, and Leon made a point of refusing to do anything about it :/
Got it, I checked the code, the current code behavior does not cause a leak. Leon proposed a new modification, will send V2. And I believe is a cleanup rather than a fix.
Regards,
Honglei
Quoted 33 lines
Thanks,
Robin.Quoted 29 lines
Free the whole reservation using
dma_iova_size(), mirroringdma_iova_free(). The unmap step still operates on @mapped_len only, and the sameiotlb_gatheris reused so a single IOTLB flush is performed.Fixes 433a76207dcf ("dma-mapping: Implement link/unlink ranges API")Signed-off-by Honglei Huang <[email protected]>---
drivers/iommu/dma-iommu.c| 5 ++++-1 file changed, 4 insertions(+), 1 deletion(-)drivers/iommu/dma-iommu.c +4
-1@@ -2096,8 +2096,11 @@ static void __iommu_dma_iova_unlink(struct device *dev, if (!iotlb_gather.queued) iommu_iotlb_sync(domain, &iotlb_gather); - if (free_iova) + if (free_iova) { + /* Free the whole reservation, not just the linked @size. */ + size = iova_align(iovad, dma_iova_size(state) + iova_start_pad); iommu_dma_free_iova(domain, addr, size, &iotlb_gather); + } } /**base-commit:
dc59e4fea9d83f03bad6bddf3fa2e52491777482
Quoted 20 lines · Honglei Huang
dma_iova_try_alloc()reserves IOVA for the whole requested size and records it instate->__size, but callers may subsequently link only a part of that reservation, for example thedrm_gpusvmmixed range case, where a device page range is linked incrementally.The doc for
dma_iova_destroy()is:"Unlink the IOVA range up to @mapped_len and free the entire IOVA
space."However
__iommu_dma_iova_unlink()computed the amount of IOVA to free from @mapped_len rather than from the full reservation. When the reservation is larger than the linked length, the tail [mapped_len, reserved size] is never returned to the allocator and is leaked, contrary to the documented contract.Free the whole reservation using
dma_iova_size(), mirroringdma_iova_free(). The unmap step still operates on @mapped_len only, and the sameiotlb_gatheris reused so a single IOTLB flush is performed.
Quoted 7 lines
Fixes 433a76207dcf ("dma-mapping: Implement link/unlink ranges API")Signed-off-by Honglei Huang <[email protected]>--- drivers/iommu/dma-iommu.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
Thanks,
Quoted 32 lines · Honglei Huang
dma_iova_try_alloc()reserves IOVA for the whole requested size and records it instate->__size, but callers may subsequently link only a part of that reservation, for example thedrm_gpusvmmixed range case, where a device page range is linked incrementally.The doc for
dma_iova_destroy()is:"Unlink the IOVA range up to @mapped_len and free the entire IOVA
space."However
__iommu_dma_iova_unlink()computed the amount of IOVA to free from @mapped_len rather than from the full reservation. When the reservation is larger than the linked length, the tail [mapped_len, reserved size] is never returned to the allocator and is leaked, contrary to the documented contract.Free the whole reservation using
dma_iova_size(), mirroringdma_iova_free(). The unmap step still operates on @mapped_len only, and the sameiotlb_gatheris reused so a single IOTLB flush is performed.Fixes 433a76207dcf ("dma-mapping: Implement link/unlink ranges API")Signed-off-by Honglei Huang <[email protected]>--- drivers/iommu/dma-iommu.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)drivers/iommu/dma-iommu.c +0
-0
drivers/iommu/dma-iommu.c static void __iommu_dma_iova_unlink(struct device *dev, · line 2103
Quoted 9 lines
drivers/iommu/dma-iommu.c +4
-1@@ -2096,8 +2096,11 @@ static void __iommu_dma_iova_unlink(struct device *dev,if (!iotlb_gather.queued)iommu_iotlb_sync (domain, &iotlb_gather); -if (free_iova) +if (free_iova) { +/* Free the whole reservation, not just the linked @size. */ + size =iova_align (iovad,dma_iova_size (state) + iova_start_pad);iommu_dma_free_iova (domain, addr, size, &iotlb_gather); + }
Probably the best change will be something like this:
drivers/iommu/dma-iommu.c +21 -32
@@ -2068,10 +2068,20 @@ static void iommu_dma_iova_unlink_range_slow(struct device *dev,arch_sync_dma_flush (); } -static void __iommu_dma_iova_unlink (struct device *dev, -struct dma_iova_state *state,size_t offset,size_t size, -enum dma_data_direction dir,unsigned long attrs, -bool free_iova) +/** +* dma_iova_unlink - Unlink a range of IOVA space +* @dev: DMA device +* @state: IOVA state +* @offset: offset into the IOVA state to unlink +* @size: size of the buffer +* @dir: DMA direction +* @attrs: attributes of mapping properties +* +* Unlink a range of IOVA space for the given IOVA state. +*/ +void dma_iova_unlink (struct device *dev,struct dma_iova_state *state, +size_t offset,size_t size,enum dma_data_direction dir, +unsigned long attrs) {struct iommu_domain *domain =iommu_get_dma_domain (dev);struct iommu_dma_cookie *cookie = domain->iova_cookie;
@@ -2096,26 +2106,6 @@ static void __iommu_dma_iova_unlink(struct device *dev,if (!iotlb_gather.queued)iommu_iotlb_sync (domain, &iotlb_gather); -if (free_iova) -iommu_dma_free_iova (domain, addr, size, &iotlb_gather); -} - -/** -* dma_iova_unlink - Unlink a range of IOVA space -* @dev: DMA device -* @state: IOVA state -* @offset: offset into the IOVA state to unlink -* @size: size of the buffer -* @dir: DMA direction -* @attrs: attributes of mapping properties -* -* Unlink a range of IOVA space for the given IOVA state. -*/ -void dma_iova_unlink (struct device *dev,struct dma_iova_state *state, -size_t offset,size_t size,enum dma_data_direction dir, -unsigned long attrs) -{ -__iommu_dma_iova_unlink (dev, state, offset, size, dir, attrs, false); }EXPORT_SYMBOL_GPL (dma_iova_unlink);
@@ -2136,14 +2126,13 @@ void dma_iova_destroy(struct device *dev, struct dma_iova_state *state, unsigned long attrs) {if (mapped_len) -__iommu_dma_iova_unlink (dev, state,0 , mapped_len, dir, attrs, - true); -else -/* -* We can be here if first call to dma_iova_link() failed and -* there is nothing to unlink, so let's be more clear. -*/ -dma_iova_free (dev, state); +dma_iova_unlink (dev, state,0 , mapped_len, dir, attrs); + +/* +* We can be here if first call to dma_iova_link() failed and +* there is nothing to unlink, so let's be more clear. +*/ +dma_iova_free (dev, state); }EXPORT_SYMBOL_GPL (dma_iova_destroy);
Quoted 8 lines
drivers/iommu/dma-iommu.c +0
-0}/** base-commit:
dc59e4fea9d83f03bad6bddf3fa2e52491777482Signature · 1 line
2.34.1
Quoted 34 lines · Leon Romanovsky
Quoted 33 lines · Honglei Huang
dma_iova_try_alloc()reserves IOVA for the whole requested size and records it instate->__size, but callers may subsequently link only a part of that reservation, for example thedrm_gpusvmmixed range case, where a device page range is linked incrementally.The doc for
dma_iova_destroy()is:"Unlink the IOVA range up to @mapped_len and free the entire IOVA
space."However
__iommu_dma_iova_unlink()computed the amount of IOVA to free from @mapped_len rather than from the full reservation. When the reservation is larger than the linked length, the tail [mapped_len, reserved size] is never returned to the allocator and is leaked, contrary to the documented contract.Free the whole reservation using
dma_iova_size(), mirroringdma_iova_free(). The unmap step still operates on @mapped_len only, and the sameiotlb_gatheris reused so a single IOTLB flush is performed.Fixes 433a76207dcf ("dma-mapping: Implement link/unlink ranges API")Signed-off-by Honglei Huang <[email protected]>--- drivers/iommu/dma-iommu.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)drivers/iommu/dma-iommu.c +0
-0@@ -2096,8 +2096,11 @@ static void __iommu_dma_iova_unlink(struct device *dev,
Quoted 10 lines
drivers/iommu/dma-iommu.c static void __iommu_dma_iova_unlink(struct device *dev, · line 2103
Quoted 8 lines
drivers/iommu/dma-iommu.c +4
-1if (!iotlb_gather.queued)iommu_iotlb_sync (domain, &iotlb_gather); -if (free_iova) +if (free_iova) { +/* Free the whole reservation, not just the linked @size. */ + size =iova_align (iovad,dma_iova_size (state) + iova_start_pad);iommu_dma_free_iova (domain, addr, size, &iotlb_gather); + }Probably the best change will be something like this:
Thanks for the review and for the shown rework, that's much cleaner than my original approach.
Based on the your code, I have made some modifications to the compilation issues and will send V2.
And the v2 is essentially your diff, I wasn't sure which attribution you'd prefer:
- Suggested-by, what I used in v2
or
- Co-developed-by: + your Signed-off-by.
Just let me know, will adjust in V3.
Regards,
Honglei
Quoted 88 lines
drivers/iommu/dma-iommu.c +21
-32@@ -2068,10 +2068,20 @@ static void iommu_dma_iova_unlink_range_slow(struct device *dev,arch_sync_dma_flush (); } -static void __iommu_dma_iova_unlink (struct device *dev, -struct dma_iova_state *state,size_t offset,size_t size, -enum dma_data_direction dir,unsigned long attrs, -bool free_iova) +/** +* dma_iova_unlink - Unlink a range of IOVA space +* @dev: DMA device +* @state: IOVA state +* @offset: offset into the IOVA state to unlink +* @size: size of the buffer +* @dir: DMA direction +* @attrs: attributes of mapping properties +* +* Unlink a range of IOVA space for the given IOVA state. +*/ +void dma_iova_unlink (struct device *dev,struct dma_iova_state *state, +size_t offset,size_t size,enum dma_data_direction dir, +unsigned long attrs) {struct iommu_domain *domain =iommu_get_dma_domain (dev);struct iommu_dma_cookie *cookie = domain->iova_cookie;@@ -2096,26 +2106,6 @@ static void __iommu_dma_iova_unlink(struct device *dev,if (!iotlb_gather.queued)iommu_iotlb_sync (domain, &iotlb_gather); -if (free_iova) -iommu_dma_free_iova (domain, addr, size, &iotlb_gather); -} - -/** -* dma_iova_unlink - Unlink a range of IOVA space -* @dev: DMA device -* @state: IOVA state -* @offset: offset into the IOVA state to unlink -* @size: size of the buffer -* @dir: DMA direction -* @attrs: attributes of mapping properties -* -* Unlink a range of IOVA space for the given IOVA state. -*/ -void dma_iova_unlink (struct device *dev,struct dma_iova_state *state, -size_t offset,size_t size,enum dma_data_direction dir, -unsigned long attrs) -{ -__iommu_dma_iova_unlink (dev, state, offset, size, dir, attrs, false); }EXPORT_SYMBOL_GPL (dma_iova_unlink);@@ -2136,14 +2126,13 @@ void dma_iova_destroy(struct device *dev, struct dma_iova_state *state, unsigned long attrs) {if (mapped_len) -__iommu_dma_iova_unlink (dev, state,0 , mapped_len, dir, attrs, - true); -else -/* -* We can be here if first call to dma_iova_link() failed and -* there is nothing to unlink, so let's be more clear. -*/ -dma_iova_free (dev, state); +dma_iova_unlink (dev, state,0 , mapped_len, dir, attrs); + +/* +* We can be here if first call to dma_iova_link() failed and +* there is nothing to unlink, so let's be more clear. +*/ +dma_iova_free (dev, state); }EXPORT_SYMBOL_GPL (dma_iova_destroy);Quoted 9 lines
drivers/iommu/dma-iommu.c +0
-0}/** base-commit:
dc59e4fea9d83f03bad6bddf3fa2e52491777482Signature · 1 line
2.34.1