swiotlb: Fix unexpected swiotlb_alloc_coherent() failures
PATCH 1/1 swiotlb: Fix unexpected swiotlb_alloc_coherent() failures
The code refactoring by commit 0176adb00406 ("swiotlb: refactor coherent buffer allocation") made swiotlb_alloc_buffer() almost always failing due to a thinko: namely, the function evaluates the dma_coherent_ok() call incorrectly and dealing as if it's invalid. This ends up with weird errors like iwlwifi probe failure or amdgpu screen flickering.
This patch corrects the logic error.
Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1088658
Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1088902
--- lib/swiotlb.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/swiotlb.c +1 -1
@@ -719,7 +719,7 @@ swiotlb_alloc_buffer(struct device *dev, size_t size, dma_addr_t *dma_handle, goto out_warn; *dma_handle =__phys_to_dma (dev, phys_addr); -if (dma_coherent_ok (dev, *dma_handle, size)) +if (!dma_coherent_ok (dev, *dma_handle, size)) goto out_unmap;memset (phys_to_virt (phys_addr),0 , size);
Signature · 1 line
2.16.3
Quoted 8 lines · Takashi Iwai
The code refactoring by commit
0176adb00406("swiotlb: refactor coherent buffer allocation") madeswiotlb_alloc_buffer()almost always failing due to a thinko: namely, the function evaluates thedma_coherent_ok()call incorrectly and dealing as if it's invalid. This ends up with weird errors like iwlwifi probe failure or amdgpu screen flickering.This patch corrects the logic error.
This looks ok, although even hitting this code is a bad sign. Do you know why the previous dma_direct_alloc didn't succeed?
Quoted 25 lines
Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1088658
Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1088902Fixes 0176adb00406 ("swiotlb: refactor coherent buffer allocation")Cc <[email protected]> # v4.16+Signed-off-by Takashi Iwai <[email protected]>--- lib/swiotlb.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/swiotlb.c +1
-1@@ -719,7 +719,7 @@ swiotlb_alloc_buffer(struct device *dev, size_t size, dma_addr_t *dma_handle, goto out_warn; *dma_handle =__phys_to_dma (dev, phys_addr); -if (dma_coherent_ok (dev, *dma_handle, size)) +if (!dma_coherent_ok (dev, *dma_handle, size)) goto out_unmap;memset (phys_to_virt (phys_addr),0 , size);Signature · 1 line
2.16.3
---end quoted text---
Quoted 9 lines · Christoph Hellwig
Quoted 7 lines · Takashi Iwai
The code refactoring by commit
0176adb00406("swiotlb: refactor coherent buffer allocation") madeswiotlb_alloc_buffer()almost always failing due to a thinko: namely, the function evaluates thedma_coherent_ok()call incorrectly and dealing as if it's invalid. This ends up with weird errors like iwlwifi probe failure or amdgpu screen flickering.
Quoted 4 lines
Quoted 1 line
This patch corrects the logic error.
This looks ok, although even hitting this code is a bad sign. Do you know why the previous
dma_direct_allocdidn't succeed?
That I have no idea, sorry. I just figured out the regression just from the dmesg output of 4.16 kernel in bugzilla reports. Could you take a look at the bugzilla reports there?
thanks,
Takashi
Quoted 26 lines
Quoted 24 lines
Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1088658
Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1088902Fixes 0176adb00406 ("swiotlb: refactor coherent buffer allocation")Cc <[email protected]> # v4.16+Signed-off-by Takashi Iwai <[email protected]>--- lib/swiotlb.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/swiotlb.c +1
-1@@ -719,7 +719,7 @@ swiotlb_alloc_buffer(struct device *dev, size_t size, dma_addr_t *dma_handle, goto out_warn; *dma_handle =__phys_to_dma (dev, phys_addr); -if (dma_coherent_ok (dev, *dma_handle, size)) +if (!dma_coherent_ok (dev, *dma_handle, size)) goto out_unmap;memset (phys_to_virt (phys_addr),0 , size);Signature · 1 line
2.16.3---end quoted text---
Quoted 11 lines · Takashi Iwai
Quoted 9 lines · Christoph Hellwig
Quoted 7 lines · Takashi Iwai
The code refactoring by commit
0176adb00406("swiotlb: refactor coherent buffer allocation") madeswiotlb_alloc_buffer()almost always failing due to a thinko: namely, the function evaluates thedma_coherent_ok()call incorrectly and dealing as if it's invalid. This ends up with weird errors like iwlwifi probe failure or amdgpu screen flickering.
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
This patch corrects the logic error.
This looks ok, although even hitting this code is a bad sign. Do you know why the previous
dma_direct_allocdidn't succeed?That I have no idea, sorry. I just figured out the regression just from the dmesg output of 4.16 kernel in bugzilla reports. Could you take a look at the bugzilla reports there?
In the first one, the machine appears to have enough RAM that most of it is beyond the device's 36-bit DMA mask, thus it's fairly likely for the initial direct alloc to come back with something unsuitable.
Robin.
Quoted 37 lines
thanks,
Takashi
Quoted 21 lines
Quoted 21 lines
Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1088658
Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1088902Fixes 0176adb00406 ("swiotlb: refactor coherent buffer allocation")Cc <[email protected]> # v4.16+Signed-off-by Takashi Iwai <[email protected]>--- lib/swiotlb.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/swiotlb.c +1
-1@@ -719,7 +719,7 @@ swiotlb_alloc_buffer(struct device *dev, size_t size, dma_addr_t *dma_handle, goto out_warn; *dma_handle =__phys_to_dma (dev, phys_addr); -if (dma_coherent_ok (dev, *dma_handle, size)) +if (!dma_coherent_ok (dev, *dma_handle, size)) goto out_unmap;Quoted 5 lines
Quoted 3 lines
lib/swiotlb.c +0
-0memset (phys_to_virt (phys_addr),0 , size);Signature · 1 line
2.16.3---end quoted text---
_______________________________________________
iommu mailing list iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Quoted 3 lines · Robin Murphy
In the first one, the machine appears to have enough RAM that most of it is beyond the device's 36-bit DMA mask, thus it's fairly likely for the initial direct alloc to come back with something unsuitable.
But we should try a GFP_DMA32 allocation first, so this is a bit surprising.
Quoted 8 lines · Christoph Hellwig
Quoted 3 lines · Robin Murphy
In the first one, the machine appears to have enough RAM that most of it is beyond the device's 36-bit DMA mask, thus it's fairly likely for the initial direct alloc to come back with something unsuitable.
But we should try a
GFP_DMA32allocation first, so this is a bit surprising.
Hm, do we really try that?
Through a quick glance, dma_alloc_coherent_gfp_flags() gives GFP_DMA32 only when coherent mask <= DMA_BIT_MASK(32); in the case of iwlwifi, it's 36bit, so GFP_DMA isn't set.
We had a fallback allocation with GFP_DMA32 in the past, but this seems gone long time ago along with cleanups (commit c647c3bb2d16).
But I haven't followed about this topic for long time, so I might have missed obviously...
thanks,
Takashi
Quoted 7 lines · Takashi Iwai
Quoted 2 lines
But we should try a
GFP_DMA32allocation first, so this is a bit surprising.Hm, do we really try that?
Through a quick glance,dma_alloc_coherent_gfp_flags()givesGFP_DMA32only when coherent mask <=DMA_BIT_MASK(32); in the case of iwlwifi, it's 36bit, soGFP_DMAisn't set.
Oh, yes - it is using an odd dma mask, and amdgpu seems to use an just as odd 40-bit dma mask.
Quoted 5 lines
We had a fallback allocation with
GFP_DMA32in the past, but this seems gone long time ago along with cleanups (commitc647c3bb2d16).But I haven't followed about this topic for long time, so I might have missed obviously...
I think a fallback would be much better here rather than relying on the limited swiotlb buffer bool. dma_direct_alloc (which in 4.17 is also used for x86) already has a GFP_DMA fallback, so extending this for GFP_DMA32 as well would seem reasonable.
Any volunteers?
Quoted 19 lines · Christoph Hellwig
Quoted 7 lines · Takashi Iwai
Quoted 2 lines
But we should try a
GFP_DMA32allocation first, so this is a bit surprising.Hm, do we really try that?
Through a quick glance,dma_alloc_coherent_gfp_flags()givesGFP_DMA32only when coherent mask <=DMA_BIT_MASK(32); in the case of iwlwifi, it's 36bit, soGFP_DMAisn't set.Oh, yes - it is using an odd dma mask, and amdgpu seems to use an just as odd 40-bit dma mask.
Quoted 5 lines
We had a fallback allocation with
GFP_DMA32in the past, but this seems gone long time ago along with cleanups (commitc647c3bb2d16).But I haven't followed about this topic for long time, so I might have missed obviously...
Quoted 6 lines
I think a fallback would be much better here rather than relying on the limited swiotlb buffer bool.
dma_direct_alloc(which in 4.17 is also used for x86) already has aGFP_DMAfallback, so extending this forGFP_DMA32as well would seem reasonable.Any volunteers?
Below is a quick attempt, totally untested. Actually the retry with GFP_DMA is superfluous for archs without it, so the first patch corrects it. The second patch adds the retry with GFP_DMA32.
I'll resubmit properly if these are OK (and better if anyone could test them :)
thanks,
Takashi
Quoted 1 line
From
490d11cd440d169c8e1a234c1baee3022c2f3464Mon Sep 17 00:00:00 2001
From: Takashi Iwai <[email protected]>
Date: Thu, 12 Apr 2018 09:46:12 +0200
Subject: [PATCH 1/2] dma-direct: Don't repeat allocation for no-op GFP_DMA
When an allocation with lower dma_coherent mask fails, dma_direct_alloc() retries the allocation with GFP_DMA. But, it's useless for architectures that has no GFP_DMA, obviously.
This patch fixes it by adding the check of GFP_DMA before retrying the allocation.
--- lib/dma-direct.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/dma-direct.c +1 -1
@@ -84,7 +84,7 @@ void *dma_direct_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle,__free_pages (page, page_order); page =NULL ; -if (dev->coherent_dma_mask <DMA_BIT_MASK (32 ) && +if (GFP_DMA && dev->coherent_dma_mask <DMA_BIT_MASK (32 ) && !(gfp &GFP_DMA )) { gfp = (gfp & ~GFP_DMA32 ) |GFP_DMA ; goto again;
Signature · 1 line
2.16.3
Quoted 1 line
From
48ce0bb0a6b916f5615e24861cd9ba78c98ee33aMon Sep 17 00:00:00 2001
From: Takashi Iwai <[email protected]>
Date: Thu, 12 Apr 2018 09:53:48 +0200
Subject: [PATCH 2/2] dma-direct: Try reallocation with GFP_DMA32 if possible
This patch adds a similar fallback reallocation with GFP_DMA32 as we've done with GFP_DMA. The condition is that the coherent mask is smaller than 64bit (i.e. some address limitation), and neither GFP_DMA nor GFP_DMA32 is set beforehand.
--- lib/dma-direct.c | 6 ++++++ 1 file changed, 6 insertions(+)
lib/dma-direct.c +6 -0
@@ -84,6 +84,12 @@ void *dma_direct_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle,__free_pages (page, page_order); page =NULL ; +if (GFP_DMA32 && dev->coherent_dma_mask <DMA_BIT_MASK (64 ) && + !(gfp & (GFP_DMA32 |GFP_DMA ))) { + gfp |=GFP_DMA32 ; + goto again; + } +if (GFP_DMA && dev->coherent_dma_mask <DMA_BIT_MASK (32 ) && !(gfp &GFP_DMA )) { gfp = (gfp & ~GFP_DMA32 ) |GFP_DMA ;
Signature · 1 line
2.16.3
Quoted 27 lines · Takashi Iwai
Quoted 24 lines · Christoph Hellwig
Quoted 7 lines · Takashi Iwai
Quoted 2 lines
But we should try a
GFP_DMA32allocation first, so this is a bit surprising.Hm, do we really try that?
Through a quick glance,dma_alloc_coherent_gfp_flags()givesGFP_DMA32only when coherent mask <=DMA_BIT_MASK(32); in the case of iwlwifi, it's 36bit, soGFP_DMAisn't set.Oh, yes - it is using an odd dma mask, and amdgpu seems to use an just as odd 40-bit dma mask.
Quoted 5 lines
We had a fallback allocation with
GFP_DMA32in the past, but this seems gone long time ago along with cleanups (commitc647c3bb2d16).But I haven't followed about this topic for long time, so I might have missed obviously...
I think a fallback would be much better here rather than relying on the limited swiotlb buffer bool.
dma_direct_alloc(which in 4.17 is also used for x86) already has aGFP_DMAfallback, so extending this forGFP_DMA32as well would seem reasonable.
Quoted 5 lines
Quoted 1 line
Any volunteers?
Below is a quick attempt, totally untested. Actually the retry with
GFP_DMAis superfluous for archs without it, so the first patch corrects it.
Gah, scratch this, it doesn't work. A different check is needed...
Takashi
Quoted 30 lines · Takashi Iwai
Quoted 27 lines · Takashi Iwai
Quoted 24 lines · Christoph Hellwig
Quoted 7 lines · Takashi Iwai
Quoted 2 lines
But we should try a
GFP_DMA32allocation first, so this is a bit surprising.Hm, do we really try that?
Through a quick glance,dma_alloc_coherent_gfp_flags()givesGFP_DMA32only when coherent mask <=DMA_BIT_MASK(32); in the case of iwlwifi, it's 36bit, soGFP_DMAisn't set.Oh, yes - it is using an odd dma mask, and amdgpu seems to use an just as odd 40-bit dma mask.
Quoted 5 lines
We had a fallback allocation with
GFP_DMA32in the past, but this seems gone long time ago along with cleanups (commitc647c3bb2d16).But I haven't followed about this topic for long time, so I might have missed obviously...
I think a fallback would be much better here rather than relying on the limited swiotlb buffer bool.
dma_direct_alloc(which in 4.17 is also used for x86) already has aGFP_DMAfallback, so extending this forGFP_DMA32as well would seem reasonable.
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
Any volunteers?
Below is a quick attempt, totally untested. Actually the retry with
GFP_DMAis superfluous for archs without it, so the first patch corrects it.Gah, scratch this, it doesn't work. A different check is needed...
The v2 patches are below, replaced with IS_ENABLED(CONFIG_ZONE_DMA).
Takashi
Quoted 1 line
From
8d320bed20265ea98087a2b9c4abf2b4bf9cb9ecMon Sep 17 00:00:00 2001
From: Takashi Iwai <[email protected]>
Date: Thu, 12 Apr 2018 09:46:12 +0200
Subject: [PATCH v2 1/2] dma-direct: Don't repeat allocation for no-op GFP_DMA
When an allocation with lower dma_coherent mask fails, dma_direct_alloc() retries the allocation with GFP_DMA. But, it's useless for architectures that has no ZONE_DMA, obviously.
This patch fixes it by adding the check of CONFIG_ZONE_DMA before retrying the allocation.
--- lib/dma-direct.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
lib/dma-direct.c +2 -1
@@ -84,7 +84,8 @@ void *dma_direct_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle,__free_pages (page, page_order); page =NULL ; -if (dev->coherent_dma_mask <DMA_BIT_MASK (32 ) && +if (IS_ENABLED (CONFIG_ZONE_DMA ) && + dev->coherent_dma_mask <DMA_BIT_MASK (32 ) && !(gfp &GFP_DMA )) { gfp = (gfp & ~GFP_DMA32 ) |GFP_DMA ; goto again;
Signature · 1 line
2.16.3
Quoted 1 line
From
0d3781b8c0c397768c29c0351390f0fa04c9eaa8Mon Sep 17 00:00:00 2001
From: Takashi Iwai <[email protected]>
Date: Thu, 12 Apr 2018 09:53:48 +0200
Subject: [PATCH v2 2/2] dma-direct: Try reallocation with GFP_DMA32 if
possible
This patch adds a similar fallback reallocation with GFP_DMA32 as we've done with GFP_DMA. The condition is that the coherent mask is smaller than 64bit (i.e. some address limitation), and neither GFP_DMA nor GFP_DMA32 is set beforehand.
--- lib/dma-direct.c | 7 +++++++ 1 file changed, 7 insertions(+)
lib/dma-direct.c +7 -0
@@ -84,6 +84,13 @@ void *dma_direct_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle,__free_pages (page, page_order); page =NULL ; +if (IS_ENABLED (CONFIG_ZONE_DMA32 ) && + dev->coherent_dma_mask <DMA_BIT_MASK (64 ) && + !(gfp & (GFP_DMA32 |GFP_DMA ))) { + gfp |=GFP_DMA32 ; + goto again; + } +if (IS_ENABLED (CONFIG_ZONE_DMA ) && dev->coherent_dma_mask <DMA_BIT_MASK (32 ) && !(gfp &GFP_DMA )) {
Signature · 1 line
2.16.3
Quoted 34 lines · Takashi Iwai
Quoted 32 lines · Takashi Iwai
Quoted 29 lines · Takashi Iwai
Quoted 25 lines · Christoph Hellwig
Quoted 7 lines · Takashi Iwai
Quoted 2 lines
But we should try a
GFP_DMA32allocation first, so this is a bit surprising.Hm, do we really try that?
Through a quick glance,dma_alloc_coherent_gfp_flags()givesGFP_DMA32only when coherent mask <=DMA_BIT_MASK(32); in the case of iwlwifi, it's 36bit, soGFP_DMAisn't set.Oh, yes - it is using an odd dma mask, and amdgpu seems to use an just as odd 40-bit dma mask.
Quoted 5 lines
We had a fallback allocation with
GFP_DMA32in the past, but this seems gone long time ago along with cleanups (commitc647c3bb2d16).But I haven't followed about this topic for long time, so I might have missed obviously...
I think a fallback would be much better here rather than relying on the limited swiotlb buffer bool.
dma_direct_alloc(which in 4.17 is also used for x86) already has aGFP_DMAfallback, so extending this forGFP_DMA32as well would seem reasonable.Any volunteers?
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Below is a quick attempt, totally untested. Actually the retry with
GFP_DMAis superfluous for archs without it, so the first patch corrects it.Gah, scratch this, it doesn't work. A different check is needed...
The v2 patches are below, replaced with
IS_ENABLED(CONFIG_ZONE_DMA).
That looks pretty reasonable to me - I'd be tempted to try a factoring out a helper function to avoid the "goto again" logic, but that's hardly crucial.
What I'd really love to see is something like the alloc_pages_mask() proposal from years ago to come back such that GFP_DMA32 could die entirely, but I don't know anywhere near enough about the mm layer to consider taking that on myself :(
Robin.
Quoted 40 lines · Robin Murphy
Quoted 38 lines · Takashi Iwai
Quoted 36 lines · Takashi Iwai
Quoted 32 lines · Takashi Iwai
Quoted 19 lines · Christoph Hellwig
Quoted 7 lines · Takashi Iwai
Quoted 2 lines
But we should try a
GFP_DMA32allocation first, so this is a bit surprising.Hm, do we really try that?
Through a quick glance,dma_alloc_coherent_gfp_flags()givesGFP_DMA32only when coherent mask <=DMA_BIT_MASK(32); in the case of iwlwifi, it's 36bit, soGFP_DMAisn't set.Oh, yes - it is using an odd dma mask, and amdgpu seems to use an just as odd 40-bit dma mask.
Quoted 5 lines
We had a fallback allocation with
GFP_DMA32in the past, but this seems gone long time ago along with cleanups (commitc647c3bb2d16).But I haven't followed about this topic for long time, so I might have missed obviously...
Quoted 6 lines
I think a fallback would be much better here rather than relying on the limited swiotlb buffer bool.
dma_direct_alloc(which in 4.17 is also used for x86) already has aGFP_DMAfallback, so extending this forGFP_DMA32as well would seem reasonable.Any volunteers?
Below is a quick attempt, totally untested. Actually the retry with
GFP_DMAis superfluous for archs without it, so the first patch corrects it.
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
Gah, scratch this, it doesn't work. A different check is needed...
The v2 patches are below, replaced with
IS_ENABLED(CONFIG_ZONE_DMA).That looks pretty reasonable to me - I'd be tempted to try a factoring out a helper function to avoid the "goto again" logic, but that's hardly crucial.
Right, this change won't make the code flow more complex than before, so I guess it's OK as a first step.
I'll submit this second patch as RFC while the first one as a safe fix (in addition to another regression fix I stumbled on).
Quoted 4 lines
What I'd really love to see is something like the
alloc_pages_mask()proposal from years ago to come back such thatGFP_DMA32could die entirely, but I don't know anywhere near enough about the mm layer to consider taking that on myself :(
Yeah, it'd be lovely, but I'm afraid it's hard to fly high...
thanks,
Takashi