x86 swiotlb: Verify we can perform the remapping requested.
PATCH 1/2 x86 swiotlb: Verify we can perform the remapping requested.
Recently I had a driver try with a peculiar 2G dma memory limit. It failed in weird and strange ways because my bounce buffers were being allocated above 2G where the driver could not reach, and no error was reported when the mappings were setup.
Use the swiotlb_dma_supported to avoid silent problems like this in the future.
--- arch/x86/kernel/pci-swiotlb.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
arch/x86/kernel/pci-swiotlb.c +1 -1
@@ -38,7 +38,7 @@ static struct dma_map_ops swiotlb_dma_ops = { .unmap_sg = swiotlb_unmap_sg_attrs, .map_page = swiotlb_map_page, .unmap_page = swiotlb_unmap_page, - .dma_supported = NULL, + .dma_supported = swiotlb_dma_supported, };/*
Signature · 1 line
1.7.2.5
PATCH 2/2 x86 amd_gart_64: Verify we can perform the remapping requested
Recently I had a driver try with a peculiar 2G dma memory limit. The driver failed in weird and strange ways because the GART remapping apperture had been allocated above 2G where the driver cound not reach, and no error was reported when the mappings were setup.
Implement gart_dma_supported to test for this problem case and to return and error if we can not support the remapping.
--- arch/x86/kernel/amd_gart_64.c | 9 +++++++++ 1 files changed, 9 insertions(+), 0 deletions(-)
arch/x86/kernel/amd_gart_64.c +9 -0
@@ -519,6 +519,14 @@ static int gart_mapping_error(struct device *dev, dma_addr_t dma_addr)return (dma_addr == bad_dma_addr); } +static int gart_dma_supported (struct device *dev,u64 mask) +{ +unsigned long iommu_max_addr = iommu_bus_base + iommu_size -1 ; + +/* Fail if the gart window is too high to fit in the devices dma mask */ +return iommu_max_addr <= mask; +} +static int no_agp;static __init unsigned long check_iommu_size (unsigned long aper,u64 aper_size)
@@ -703,6 +711,7 @@ static struct dma_map_ops gart_dma_ops = { .alloc_coherent = gart_alloc_coherent, .free_coherent = gart_free_coherent, .mapping_error = gart_mapping_error, + .dma_supported = gart_dma_supported, };static void gart_iommu_shutdown (void )
Signature · 1 line
1.7.2.5
Re: [PATCH 2/2] x86 amd_gart_64: Verify we can perform the remapping requested
Quoted 8 lines · Eric W. Biederman
Recently I had a driver try with a peculiar 2G dma memory limit. The driver failed in weird and strange ways because the GART remapping apperture had been allocated above 2G where the driver cound not reach, and no error was reported when the mappings were setup.
Implement
gart_dma_supportedto test for this problem case and to return and error if we can not support the remapping.
You do basically the same as for swiotlb, so it must be good :)
Quoted 1 line
Signed-off-by Eric W. Biederman <[email protected]>
For both patches:
Quoted 39 lines
--- arch/x86/kernel/amd_gart_64.c | 9 +++++++++ 1 files changed, 9 insertions(+), 0 deletions(-)arch/x86/kernel/amd_gart_64.c +9
-0@@ -519,6 +519,14 @@ static int gart_mapping_error(struct device *dev, dma_addr_t dma_addr)return (dma_addr == bad_dma_addr); } +static int gart_dma_supported (struct device *dev,u64 mask) +{ +unsigned long iommu_max_addr = iommu_bus_base + iommu_size -1 ; + +/* Fail if the gart window is too high to fit in the devices dma mask */ +return iommu_max_addr <= mask; +} +static int no_agp;static __init unsigned long check_iommu_size (unsigned long aper,u64 aper_size)@@ -703,6 +711,7 @@ static struct dma_map_ops gart_dma_ops = { .alloc_coherent = gart_alloc_coherent, .free_coherent = gart_free_coherent, .mapping_error = gart_mapping_error, + .dma_supported = gart_dma_supported, };static void gart_iommu_shutdown (void )Signature · 7 lines
1.7.2.5 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Signature · 1 line
Nothing great was ever achieved without enthusiasm.
Re: [PATCH 2/2] x86 amd_gart_64: Verify we can perform the remapping requested
Quoted 11 lines · Joerg Roedel
Quoted 8 lines · Eric W. Biederman
Recently I had a driver try with a peculiar 2G dma memory limit. The driver failed in weird and strange ways because the GART remapping apperture had been allocated above 2G where the driver cound not reach, and no error was reported when the mappings were setup.
Implement
gart_dma_supportedto test for this problem case and to return and error if we can not support the remapping.You do basically the same as for swiotlb, so it must be good :)
Okay, thinking more about this, I am not so sure anymore it is a really good idea to fix it this way. It basically signals the driver that DMA is not possible with the device if the aperture does not fit into the dma_mask. But DMA within the ZONE_DMA might still work, no?
So I think the right way to fix it is to return a dma-mapping error if and only if the address needs to be remapped by the aperture (of swiotlb or gart).
This still allows the driver to use GFP_DMA allocations.
Joerg
Re: [PATCH 2/2] x86 amd_gart_64: Verify we can perform the remapping requested
Quoted 13 lines · Joerg Roedel
Quoted 11 lines · Joerg Roedel
Quoted 8 lines · Eric W. Biederman
Recently I had a driver try with a peculiar 2G dma memory limit. The driver failed in weird and strange ways because the GART remapping apperture had been allocated above 2G where the driver cound not reach, and no error was reported when the mappings were setup.
Implement
gart_dma_supportedto test for this problem case and to return and error if we can not support the remapping.You do basically the same as for swiotlb, so it must be good :)
Quoted 8 lines
Okay, thinking more about this, I am not so sure anymore it is a really good idea to fix it this way. It basically signals the driver that DMA is not possible with the device if the aperture does not fit into the
dma_mask. But DMA within theZONE_DMAmight still work, no?So I think the right way to fix it is to return a dma-mapping error if and only if the address needs to be remapped by the aperture (of swiotlb or gart).
Perhaps I am mistaken but since the method is iommu type specific if we don't actually use the iommu we should not call the method so I think this is patch is already implementing what you want.
Quoted 1 line
This still allows the driver to use
GFP_DMAallocations.
I definitely agree on that.
Eric
Re: [PATCH 2/2] x86 amd_gart_64: Verify we can perform the remapping requested
Quoted 6 lines · Eric W. Biederman
Quoted 5 lines · Joerg Roedel
Okay, thinking more about this, I am not so sure anymore it is a really good idea to fix it this way. It basically signals the driver that DMA is not possible with the device if the aperture does not fit into the
dma_mask. But DMA within theZONE_DMAmight still work, no?
Quoted 7 lines
Quoted 3 lines
So I think the right way to fix it is to return a dma-mapping error if and only if the address needs to be remapped by the aperture (of swiotlb or gart).
Perhaps I am mistaken but since the method is iommu type specific if we don't actually use the iommu we should not call the method so I think this is patch is already implementing what you want.
No it does not. With GART or SWIOTLB all DMA-mapping functions call directly into the IOMMU driver. The driver then checks whether it needs to remap or not (see dma_capable() in SWIOTLB or need_iommu() in GART).
I think the check you want to add needs to happen after these checks in each of the IOMMU drivers.
Joerg
Quoted 5 lines · Eric W. Biederman
Recently I had a driver try with a peculiar 2G dma memory limit. It failed in weird and strange ways because my bounce buffers were being allocated above 2G where the driver could not reach, and no error was reported when the mappings were setup.
OK, so the overflow buffer was used instead.. which presumarily also was allocated above the 2G? That seems to point that alloc_bootmem_low_pages is not doing its job?
Quoted 3 lines
Use the
swiotlb_dma_supportedto avoid silent problems like this in the future.
Which driver was it that had this limit?
Quoted 2 lines
Signed-off-by Eric W. Biederman <[email protected]>
I also CC-ed Fujita on this as he is the swiotlb maintainer.
Quoted 25 lines
--- arch/x86/kernel/pci-swiotlb.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)arch/x86/kernel/pci-swiotlb.c +1
-1@@ -38,7 +38,7 @@ static struct dma_map_ops swiotlb_dma_ops = { .unmap_sg = swiotlb_unmap_sg_attrs, .map_page = swiotlb_map_page, .unmap_page = swiotlb_unmap_page, - .dma_supported = NULL, + .dma_supported = swiotlb_dma_supported, };/* Signature · 7 lines
1.7.2.5 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Quoted 10 lines · Konrad Rzeszutek Wilk
Quoted 5 lines · Eric W. Biederman
Recently I had a driver try with a peculiar 2G dma memory limit. It failed in weird and strange ways because my bounce buffers were being allocated above 2G where the driver could not reach, and no error was reported when the mappings were setup.
OK, so the overflow buffer was used instead.. which presumarily also was allocated above the 2G? That seems to point that
alloc_bootmem_low_pagesis not doing its job?
I just looked alloc_bootmem_low_pages allocates memory below ARCH_ADDRESS_LIMIT which for everything except s390 is 4G.
I know I was mostly using the amd gart driver. So I may be mistaken that the swiotlb driver had the same issue. However my only solution at the time was to boot with mem=2G. So I believe the swiotlb did have this issue.
Mostly the patch was. Hmm. That looks stupid not wiring up the swiotlb address space limit check when someone has already written it.
Quoted 4 lines
Quoted 2 lines
Use the
swiotlb_dma_supportedto avoid silent problems like this in the future.Which driver was it that had this limit?
An unmerged driver. The driver has since been fixed to use it's hardware in a way that doesn't hit this limit.
At the time that I looked drivers in the kernel with a dma limit above 16MB and blow 4GB only existed on exotic architectures in very special cases.
Unfortunately I no longer have access to this hardware to test with.
Eric
Quoted 16 lines · On Mon, 24 Oct 2011 08:07:31 -0700 [email protected] (Eric W. Biederman)…
Quoted 10 lines · Konrad Rzeszutek Wilk
Quoted 5 lines · Eric W. Biederman
Recently I had a driver try with a peculiar 2G dma memory limit. It failed in weird and strange ways because my bounce buffers were being allocated above 2G where the driver could not reach, and no error was reported when the mappings were setup.
OK, so the overflow buffer was used instead.. which presumarily also was allocated above the 2G? That seems to point that
alloc_bootmem_low_pagesis not doing its job?I just looked
alloc_bootmem_low_pagesallocates memory belowARCH_ADDRESS_LIMITwhich for everything except s390 is 4G.
Quoted 7 lines
I know I was mostly using the amd gart driver. So I may be mistaken that the swiotlb driver had the same issue. However my only solution at the time was to boot with mem=2G. So I believe the swiotlb did have this issue.
Mostly the patch was. Hmm. That looks stupid not wiring up the swiotlb address space limit check when someone has already written it.
http://www.gossamer-threads.com/lists/linux/kernel/1333685?do=post_view_threaded#1333685
swiotlb isn't designed to handle hardware having odd dma mask (e.g. 2G or something). Such device needs the own bouncing mechanism.
Quoted 24 lines · FUJITA Tomonori
Quoted 21 lines · On Mon, 24 Oct 2011 08:07:31 -0700 [email protected] (Eric W. Biederman)…
Quoted 10 lines · Konrad Rzeszutek Wilk
Quoted 5 lines · Eric W. Biederman
Recently I had a driver try with a peculiar 2G dma memory limit. It failed in weird and strange ways because my bounce buffers were being allocated above 2G where the driver could not reach, and no error was reported when the mappings were setup.
OK, so the overflow buffer was used instead.. which presumarily also was allocated above the 2G? That seems to point that
alloc_bootmem_low_pagesis not doing its job?I just looked
alloc_bootmem_low_pagesallocates memory belowARCH_ADDRESS_LIMITwhich for everything except s390 is 4G.I know I was mostly using the amd gart driver. So I may be mistaken that the swiotlb driver had the same issue. However my only solution at the time was to boot with mem=2G. So I believe the swiotlb did have this issue.
Quoted 7 lines
Quoted 2 lines
Mostly the patch was. Hmm. That looks stupid not wiring up the swiotlb address space limit check when someone has already written it.
http://www.gossamer-threads.com/lists/linux/kernel/1333685?do=post_view_threaded#1333685
swiotlb isn't designed to handle hardware having odd dma mask (e.g. 2G or something). Such device needs the own bouncing mechanism.
I have no problem with that with not supporting weird dma masks.
There is a perfectly fine method to report an unsupported dma mask programmatically that just needs a trivial code change to be wired up. I just want the method to be wired up on x86 so that we fail gracefully, instead of mysteriously.
Eric