Skip to contentSkip to conversation

swiotlb: Fix unexpected swiotlb_alloc_coherent() failures

PATCH In review Takashi Iwai 3 participants 12 messages Apr 10, 2018 – Apr 15, 2018 Library code

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 swiotlb: Fix unexpected swiotlb_alloc_coherent() failures

The code refactoring by commit 0176adb00406 ("swiotlb: refactor coherent buffer allocation") made swiotlb_alloc_buffer() almost always failing due to a thinko: namely, the function evaluates the dma_coherent_ok() call incorrectly and dealing as if it's invalid. This ends up with weird errors like iwlwifi probe failure or amdgpu screen flickering.

This patch corrects the logic error.

Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1088658
Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1088902

Fixes 0176adb00406 ("swiotlb: refactor coherent buffer allocation")
Cc <[email protected]> # v4.16+
---
 lib/swiotlb.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/swiotlb.c +1 -1
index 47aeb04c1997..de7cc540450f 100644
@@ -719,7 +719,7 @@ swiotlb_alloc_buffer(struct device *dev, size_t size, dma_addr_t *dma_handle,
 		goto out_warn;
 
 	*dma_handle = __phys_to_dma(dev, phys_addr);
-	if (dma_coherent_ok(dev, *dma_handle, size))
+	if (!dma_coherent_ok(dev, *dma_handle, size))
 		goto out_unmap;
 
 	memset(phys_to_virt(phys_addr), 0, size);
Signature · 1 line
2.16.3
#1 lore raw
Quoted 8 lines · Takashi Iwai

The code refactoring by commit 0176adb00406 ("swiotlb: refactor coherent buffer allocation") made swiotlb_alloc_buffer() almost always failing due to a thinko: namely, the function evaluates the dma_coherent_ok() call incorrectly and dealing as if it's invalid. This ends up with weird errors like iwlwifi probe failure or amdgpu screen flickering.

This patch corrects the logic error.

This looks ok, although even hitting this code is a bad sign. Do you know why the previous dma_direct_alloc didn't succeed?

Quoted 25 lines

Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1088658
Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1088902

Fixes 0176adb00406 ("swiotlb: refactor coherent buffer allocation")
Cc <[email protected]> # v4.16+
---
 lib/swiotlb.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/swiotlb.c +1 -1
index 47aeb04c1997..de7cc540450f 100644
@@ -719,7 +719,7 @@ swiotlb_alloc_buffer(struct device *dev, size_t size, dma_addr_t *dma_handle,
 		goto out_warn;
 
 	*dma_handle = __phys_to_dma(dev, phys_addr);
-	if (dma_coherent_ok(dev, *dma_handle, size))
+	if (!dma_coherent_ok(dev, *dma_handle, size))
 		goto out_unmap;
 
 	memset(phys_to_virt(phys_addr), 0, size);
Signature · 1 line
2.16.3

---end quoted text---

#2 lore raw
Quoted 9 lines · Christoph Hellwig
Quoted 7 lines · Takashi Iwai

The code refactoring by commit 0176adb00406 ("swiotlb: refactor coherent buffer allocation") made swiotlb_alloc_buffer() almost always failing due to a thinko: namely, the function evaluates the dma_coherent_ok() call incorrectly and dealing as if it's invalid. This ends up with weird errors like iwlwifi probe failure or amdgpu screen flickering.

Quoted 4 lines
Quoted 1 line

This patch corrects the logic error.

This looks ok, although even hitting this code is a bad sign. Do you know why the previous dma_direct_alloc didn't succeed?

That I have no idea, sorry. I just figured out the regression just from the dmesg output of 4.16 kernel in bugzilla reports. Could you take a look at the bugzilla reports there?

thanks,

Takashi

Quoted 26 lines
Quoted 24 lines

Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1088658
Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1088902

Fixes 0176adb00406 ("swiotlb: refactor coherent buffer allocation")
Cc <[email protected]> # v4.16+
---
 lib/swiotlb.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/swiotlb.c +1 -1
index 47aeb04c1997..de7cc540450f 100644
@@ -719,7 +719,7 @@ swiotlb_alloc_buffer(struct device *dev, size_t size, dma_addr_t *dma_handle,
 		goto out_warn;
 
 	*dma_handle = __phys_to_dma(dev, phys_addr);
-	if (dma_coherent_ok(dev, *dma_handle, size))
+	if (!dma_coherent_ok(dev, *dma_handle, size))
 		goto out_unmap;
 
 	memset(phys_to_virt(phys_addr), 0, size);
Signature · 1 line
2.16.3

---end quoted text---

#3 lore raw
Quoted 11 lines · Takashi Iwai
Quoted 9 lines · Christoph Hellwig
Quoted 7 lines · Takashi Iwai

The code refactoring by commit 0176adb00406 ("swiotlb: refactor coherent buffer allocation") made swiotlb_alloc_buffer() almost always failing due to a thinko: namely, the function evaluates the dma_coherent_ok() call incorrectly and dealing as if it's invalid. This ends up with weird errors like iwlwifi probe failure or amdgpu screen flickering.

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

This patch corrects the logic error.

This looks ok, although even hitting this code is a bad sign. Do you know why the previous dma_direct_alloc didn't succeed?

That I have no idea, sorry. I just figured out the regression just from the dmesg output of 4.16 kernel in bugzilla reports. Could you take a look at the bugzilla reports there?

In the first one, the machine appears to have enough RAM that most of it is beyond the device's 36-bit DMA mask, thus it's fairly likely for the initial direct alloc to come back with something unsuitable.

Robin.

Quoted 37 lines

thanks,

Takashi

Quoted 21 lines
Quoted 21 lines

Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1088658
Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1088902

Fixes 0176adb00406 ("swiotlb: refactor coherent buffer allocation")
Cc <[email protected]> # v4.16+
---
  lib/swiotlb.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
lib/swiotlb.c +1 -1
index 47aeb04c1997..de7cc540450f 100644
@@ -719,7 +719,7 @@ swiotlb_alloc_buffer(struct device *dev, size_t size, dma_addr_t *dma_handle,
  		goto out_warn;
 
  	*dma_handle = __phys_to_dma(dev, phys_addr);
-	if (dma_coherent_ok(dev, *dma_handle, size))
+	if (!dma_coherent_ok(dev, *dma_handle, size))
  		goto out_unmap;
 
Quoted 5 lines
Quoted 3 lines
lib/swiotlb.c +0 -0
  	memset(phys_to_virt(phys_addr), 0, size);
Signature · 1 line
2.16.3

---end quoted text---

_______________________________________________
iommu mailing list iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#4 lore raw
Quoted 3 lines · Robin Murphy

In the first one, the machine appears to have enough RAM that most of it is beyond the device's 36-bit DMA mask, thus it's fairly likely for the initial direct alloc to come back with something unsuitable.

But we should try a GFP_DMA32 allocation first, so this is a bit surprising.

#5 lore raw
Quoted 8 lines · Christoph Hellwig
Quoted 3 lines · Robin Murphy

In the first one, the machine appears to have enough RAM that most of it is beyond the device's 36-bit DMA mask, thus it's fairly likely for the initial direct alloc to come back with something unsuitable.

But we should try a GFP_DMA32 allocation first, so this is a bit surprising.

Hm, do we really try that?
Through a quick glance, dma_alloc_coherent_gfp_flags() gives GFP_DMA32 only when coherent mask <= DMA_BIT_MASK(32); in the case of iwlwifi, it's 36bit, so GFP_DMA isn't set.

We had a fallback allocation with GFP_DMA32 in the past, but this seems gone long time ago along with cleanups (commit c647c3bb2d16).

But I haven't followed about this topic for long time, so I might have missed obviously...

thanks,

Takashi

#6 lore raw
Quoted 7 lines · Takashi Iwai
Quoted 2 lines

But we should try a GFP_DMA32 allocation first, so this is a bit surprising.

Hm, do we really try that?
Through a quick glance, dma_alloc_coherent_gfp_flags() gives GFP_DMA32 only when coherent mask <= DMA_BIT_MASK(32); in the case of iwlwifi, it's 36bit, so GFP_DMA isn't set.

Oh, yes - it is using an odd dma mask, and amdgpu seems to use an just as odd 40-bit dma mask.

Quoted 5 lines

We had a fallback allocation with GFP_DMA32 in the past, but this seems gone long time ago along with cleanups (commit c647c3bb2d16).

But I haven't followed about this topic for long time, so I might have missed obviously...

I think a fallback would be much better here rather than relying on the limited swiotlb buffer bool. dma_direct_alloc (which in 4.17 is also used for x86) already has a GFP_DMA fallback, so extending this for GFP_DMA32 as well would seem reasonable.

Any volunteers?

#7 lore raw
Quoted 19 lines · Christoph Hellwig
Quoted 7 lines · Takashi Iwai
Quoted 2 lines

But we should try a GFP_DMA32 allocation first, so this is a bit surprising.

Hm, do we really try that?
Through a quick glance, dma_alloc_coherent_gfp_flags() gives GFP_DMA32 only when coherent mask <= DMA_BIT_MASK(32); in the case of iwlwifi, it's 36bit, so GFP_DMA isn't set.

Oh, yes - it is using an odd dma mask, and amdgpu seems to use an just as odd 40-bit dma mask.

Quoted 5 lines

We had a fallback allocation with GFP_DMA32 in the past, but this seems gone long time ago along with cleanups (commit c647c3bb2d16).

But I haven't followed about this topic for long time, so I might have missed obviously...

Quoted 6 lines

I think a fallback would be much better here rather than relying on the limited swiotlb buffer bool. dma_direct_alloc (which in 4.17 is also used for x86) already has a GFP_DMA fallback, so extending this for GFP_DMA32 as well would seem reasonable.

Any volunteers?

Below is a quick attempt, totally untested. Actually the retry with GFP_DMA is superfluous for archs without it, so the first patch corrects it. The second patch adds the retry with GFP_DMA32.

I'll resubmit properly if these are OK (and better if anyone could test them :)

thanks,

Takashi

Quoted 1 line

From 490d11cd440d169c8e1a234c1baee3022c2f3464 Mon Sep 17 00:00:00 2001

From: Takashi Iwai <[email protected]>
Date: Thu, 12 Apr 2018 09:46:12 +0200
Subject: [PATCH 1/2] dma-direct: Don't repeat allocation for no-op GFP_DMA

When an allocation with lower dma_coherent mask fails, dma_direct_alloc() retries the allocation with GFP_DMA. But, it's useless for architectures that has no GFP_DMA, obviously.

This patch fixes it by adding the check of GFP_DMA before retrying the allocation.

Fixes 95f183916d4b ("dma-direct: retry allocations using GFP_DMA for small masks")
---
 lib/dma-direct.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/dma-direct.c +1 -1
index c0bba30fef0a..191f96453419 100644
@@ -84,7 +84,7 @@ void *dma_direct_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle,
 		__free_pages(page, page_order);
 		page = NULL;
 
-		if (dev->coherent_dma_mask < DMA_BIT_MASK(32) &&
+		if (GFP_DMA && dev->coherent_dma_mask < DMA_BIT_MASK(32) &&
 		    !(gfp & GFP_DMA)) {
 			gfp = (gfp & ~GFP_DMA32) | GFP_DMA;
 			goto again;
Signature · 1 line
2.16.3
Quoted 1 line

From 48ce0bb0a6b916f5615e24861cd9ba78c98ee33a Mon Sep 17 00:00:00 2001

From: Takashi Iwai <[email protected]>
Date: Thu, 12 Apr 2018 09:53:48 +0200
Subject: [PATCH 2/2] dma-direct: Try reallocation with GFP_DMA32 if possible

This patch adds a similar fallback reallocation with GFP_DMA32 as we've done with GFP_DMA. The condition is that the coherent mask is smaller than 64bit (i.e. some address limitation), and neither GFP_DMA nor GFP_DMA32 is set beforehand.

---
 lib/dma-direct.c | 6 ++++++
 1 file changed, 6 insertions(+)
lib/dma-direct.c +6 -0
index 191f96453419..b3cabf06ce90 100644
@@ -84,6 +84,12 @@ void *dma_direct_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle,
 		__free_pages(page, page_order);
 		page = NULL;
 
+		if (GFP_DMA32 && dev->coherent_dma_mask < DMA_BIT_MASK(64) &&
+		    !(gfp & (GFP_DMA32 | GFP_DMA))) {
+			gfp |= GFP_DMA32;
+			goto again;
+		}
+
 		if (GFP_DMA && dev->coherent_dma_mask < DMA_BIT_MASK(32) &&
 		    !(gfp & GFP_DMA)) {
 			gfp = (gfp & ~GFP_DMA32) | GFP_DMA;
Signature · 1 line
2.16.3
#8 lore raw
Quoted 27 lines · Takashi Iwai
Quoted 24 lines · Christoph Hellwig
Quoted 7 lines · Takashi Iwai
Quoted 2 lines

But we should try a GFP_DMA32 allocation first, so this is a bit surprising.

Hm, do we really try that?
Through a quick glance, dma_alloc_coherent_gfp_flags() gives GFP_DMA32 only when coherent mask <= DMA_BIT_MASK(32); in the case of iwlwifi, it's 36bit, so GFP_DMA isn't set.

Oh, yes - it is using an odd dma mask, and amdgpu seems to use an just as odd 40-bit dma mask.

Quoted 5 lines

We had a fallback allocation with GFP_DMA32 in the past, but this seems gone long time ago along with cleanups (commit c647c3bb2d16).

But I haven't followed about this topic for long time, so I might have missed obviously...

I think a fallback would be much better here rather than relying on the limited swiotlb buffer bool. dma_direct_alloc (which in 4.17 is also used for x86) already has a GFP_DMA fallback, so extending this for GFP_DMA32 as well would seem reasonable.

Quoted 5 lines
Quoted 1 line

Any volunteers?

Below is a quick attempt, totally untested. Actually the retry with GFP_DMA is superfluous for archs without it, so the first patch corrects it.

Gah, scratch this, it doesn't work. A different check is needed...

Takashi

#9 lore raw
Quoted 30 lines · Takashi Iwai
Quoted 27 lines · Takashi Iwai
Quoted 24 lines · Christoph Hellwig
Quoted 7 lines · Takashi Iwai
Quoted 2 lines

But we should try a GFP_DMA32 allocation first, so this is a bit surprising.

Hm, do we really try that?
Through a quick glance, dma_alloc_coherent_gfp_flags() gives GFP_DMA32 only when coherent mask <= DMA_BIT_MASK(32); in the case of iwlwifi, it's 36bit, so GFP_DMA isn't set.

Oh, yes - it is using an odd dma mask, and amdgpu seems to use an just as odd 40-bit dma mask.

Quoted 5 lines

We had a fallback allocation with GFP_DMA32 in the past, but this seems gone long time ago along with cleanups (commit c647c3bb2d16).

But I haven't followed about this topic for long time, so I might have missed obviously...

I think a fallback would be much better here rather than relying on the limited swiotlb buffer bool. dma_direct_alloc (which in 4.17 is also used for x86) already has a GFP_DMA fallback, so extending this for GFP_DMA32 as well would seem reasonable.

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

Any volunteers?

Below is a quick attempt, totally untested. Actually the retry with GFP_DMA is superfluous for archs without it, so the first patch corrects it.

Gah, scratch this, it doesn't work. A different check is needed...

The v2 patches are below, replaced with IS_ENABLED(CONFIG_ZONE_DMA).

Takashi

Quoted 1 line

From 8d320bed20265ea98087a2b9c4abf2b4bf9cb9ec Mon Sep 17 00:00:00 2001

From: Takashi Iwai <[email protected]>
Date: Thu, 12 Apr 2018 09:46:12 +0200
Subject: [PATCH v2 1/2] dma-direct: Don't repeat allocation for no-op GFP_DMA

When an allocation with lower dma_coherent mask fails, dma_direct_alloc() retries the allocation with GFP_DMA. But, it's useless for architectures that has no ZONE_DMA, obviously.

This patch fixes it by adding the check of CONFIG_ZONE_DMA before retrying the allocation.

Fixes 95f183916d4b ("dma-direct: retry allocations using GFP_DMA for small masks")
---
 lib/dma-direct.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
lib/dma-direct.c +2 -1
index c0bba30fef0a..bbfb229aa067 100644
@@ -84,7 +84,8 @@ void *dma_direct_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle,
 		__free_pages(page, page_order);
 		page = NULL;
 
-		if (dev->coherent_dma_mask < DMA_BIT_MASK(32) &&
+		if (IS_ENABLED(CONFIG_ZONE_DMA) &&
+		    dev->coherent_dma_mask < DMA_BIT_MASK(32) &&
 		    !(gfp & GFP_DMA)) {
 			gfp = (gfp & ~GFP_DMA32) | GFP_DMA;
 			goto again;
Signature · 1 line
2.16.3
Quoted 1 line

From 0d3781b8c0c397768c29c0351390f0fa04c9eaa8 Mon Sep 17 00:00:00 2001

From: Takashi Iwai <[email protected]>
Date: Thu, 12 Apr 2018 09:53:48 +0200
Subject: [PATCH v2 2/2] dma-direct: Try reallocation with GFP_DMA32 if
possible

This patch adds a similar fallback reallocation with GFP_DMA32 as we've done with GFP_DMA. The condition is that the coherent mask is smaller than 64bit (i.e. some address limitation), and neither GFP_DMA nor GFP_DMA32 is set beforehand.

---
 lib/dma-direct.c | 7 +++++++
 1 file changed, 7 insertions(+)
lib/dma-direct.c +7 -0
index bbfb229aa067..970d39155618 100644
@@ -84,6 +84,13 @@ void *dma_direct_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle,
 		__free_pages(page, page_order);
 		page = NULL;
 
+		if (IS_ENABLED(CONFIG_ZONE_DMA32) &&
+		    dev->coherent_dma_mask < DMA_BIT_MASK(64) &&
+		    !(gfp & (GFP_DMA32 | GFP_DMA))) {
+			gfp |= GFP_DMA32;
+			goto again;
+		}
+
 		if (IS_ENABLED(CONFIG_ZONE_DMA) &&
 		    dev->coherent_dma_mask < DMA_BIT_MASK(32) &&
 		    !(gfp & GFP_DMA)) {
Signature · 1 line
2.16.3
#10 lore raw
Quoted 34 lines · Takashi Iwai
Quoted 32 lines · Takashi Iwai
Quoted 29 lines · Takashi Iwai
Quoted 25 lines · Christoph Hellwig
Quoted 7 lines · Takashi Iwai
Quoted 2 lines

But we should try a GFP_DMA32 allocation first, so this is a bit surprising.

Hm, do we really try that?
Through a quick glance, dma_alloc_coherent_gfp_flags() gives GFP_DMA32 only when coherent mask <= DMA_BIT_MASK(32); in the case of iwlwifi, it's 36bit, so GFP_DMA isn't set.

Oh, yes - it is using an odd dma mask, and amdgpu seems to use an just as odd 40-bit dma mask.

Quoted 5 lines

We had a fallback allocation with GFP_DMA32 in the past, but this seems gone long time ago along with cleanups (commit c647c3bb2d16).

But I haven't followed about this topic for long time, so I might have missed obviously...

I think a fallback would be much better here rather than relying on the limited swiotlb buffer bool. dma_direct_alloc (which in 4.17 is also used for x86) already has a GFP_DMA fallback, so extending this for GFP_DMA32 as well would seem reasonable.

Any volunteers?

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

Below is a quick attempt, totally untested. Actually the retry with GFP_DMA is superfluous for archs without it, so the first patch corrects it.

Gah, scratch this, it doesn't work. A different check is needed...

The v2 patches are below, replaced with IS_ENABLED(CONFIG_ZONE_DMA).

That looks pretty reasonable to me - I'd be tempted to try a factoring out a helper function to avoid the "goto again" logic, but that's hardly crucial.

What I'd really love to see is something like the alloc_pages_mask() proposal from years ago to come back such that GFP_DMA32 could die entirely, but I don't know anywhere near enough about the mm layer to consider taking that on myself :(

Robin.

#11 lore raw
Quoted 40 lines · Robin Murphy
Quoted 38 lines · Takashi Iwai
Quoted 36 lines · Takashi Iwai
Quoted 32 lines · Takashi Iwai
Quoted 19 lines · Christoph Hellwig
Quoted 7 lines · Takashi Iwai
Quoted 2 lines

But we should try a GFP_DMA32 allocation first, so this is a bit surprising.

Hm, do we really try that?
Through a quick glance, dma_alloc_coherent_gfp_flags() gives GFP_DMA32 only when coherent mask <= DMA_BIT_MASK(32); in the case of iwlwifi, it's 36bit, so GFP_DMA isn't set.

Oh, yes - it is using an odd dma mask, and amdgpu seems to use an just as odd 40-bit dma mask.

Quoted 5 lines

We had a fallback allocation with GFP_DMA32 in the past, but this seems gone long time ago along with cleanups (commit c647c3bb2d16).

But I haven't followed about this topic for long time, so I might have missed obviously...

Quoted 6 lines

I think a fallback would be much better here rather than relying on the limited swiotlb buffer bool. dma_direct_alloc (which in 4.17 is also used for x86) already has a GFP_DMA fallback, so extending this for GFP_DMA32 as well would seem reasonable.

Any volunteers?

Below is a quick attempt, totally untested. Actually the retry with GFP_DMA is superfluous for archs without it, so the first patch corrects it.

Quoted 7 lines
Quoted 3 lines
Quoted 1 line

Gah, scratch this, it doesn't work. A different check is needed...

The v2 patches are below, replaced with IS_ENABLED(CONFIG_ZONE_DMA).

That looks pretty reasonable to me - I'd be tempted to try a factoring out a helper function to avoid the "goto again" logic, but that's hardly crucial.

Right, this change won't make the code flow more complex than before, so I guess it's OK as a first step.

I'll submit this second patch as RFC while the first one as a safe fix (in addition to another regression fix I stumbled on).

Quoted 4 lines

What I'd really love to see is something like the alloc_pages_mask() proposal from years ago to come back such that GFP_DMA32 could die entirely, but I don't know anywhere near enough about the mm layer to consider taking that on myself :(

Yeah, it'd be lovely, but I'm afraid it's hard to fly high...

thanks,

Takashi