dma: Fix max PFN arithmetic overflow on 32 bit systems
PATCH 1/1 dma: Fix max PFN arithmetic overflow on 32 bit systems
For ARCH=x86 (32 bit) when you set CONFIG_IOMMU_INTEL since c5a5dc4cbbf4 ("iommu/vt-d: Don't switch off swiotlb if bounce page is used") there's a dependency on CONFIG_SWIOTLB, which was not necessarily active before.
The init code for swiotlb in 'pci_swiotlb_detect_4gb()' compares something against MAX_DMA32_PFN to decide if it should be active. However that define suffers from an arithmetic overflow since 1b7e03ef7570 ("x86, NUMA: Enable emulation on 32bit too") when it was first made visible to x86_32.
The effect is at boot time 64 MiB (default size) were allocated for bounce buffers now, which is a noticeable amount of memory on small systems. We noticed this effect on the fli4l Linux distribution when migrating from kernel v4.19 (LTS) to v5.4 (LTS) on boards like pcengines ALIX 2D3 with 256 MiB memory for example:
Linux version 5.4.22 (buildroot@buildroot) (gcc version 7.3.0 (Buildroot 2018.02.8)) #1 SMP Mon Nov 26 23:40:00 CET 2018
…
Memory: 183484K/261756K available (4594K kernel code, 393K rwdata, 1660K rodata, 536K init, 456K bss , 78272K reserved, 0K cma-reserved, 0K highmem) …
PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
software IO TLB: mapped [mem 0x0bb78000-0x0fb78000] (64MB)
The initial analysis and the suggested fix was done by user 'sourcejedi' at stackoverflow and explicitly marked as GPLv2 for inclusion in the Linux kernel:
https://unix.stackexchange.com/a/520525/50007
---
We tested this in qemu and on real hardware with fli4l on top of v5.4, v5.5, and v5.6-rc kernels, but only as far as the reserved memory goes. The patch itself is based on v5.6-rc3 (IIRC).
A quick grep over the kernel code showed me this define MAX_DMA32_PFN is used in other places as well. I would appreciate feedback on this, because I can not oversee all side effects this might have?!
Thanks again to Alan who proposed the fix, and for his permission to send it upstream.
Greets
Alex
--- arch/x86/include/asm/dma.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/include/asm/dma.h +1 -1
@@ -74,7 +74,7 @@#define MAX_DMA_PFN ((16UL * 1024 * 1024) >> PAGE_SHIFT)/* 4GB broken PCI/AGP hardware bus master zone */ -#define MAX_DMA32_PFN ((4UL * 1024 * 1024 * 1024) >> PAGE_SHIFT) +#define MAX_DMA32_PFN (4UL * ((1024 * 1024 * 1024) >> PAGE_SHIFT))#ifdef CONFIG_X86_32 /* The maximum address that we can perform a DMA transfer to on this platform */
Signature · 1 line
2.20.1
Hei hei,
gentle ping on this patch from two weeks ago. Did anyone have time to look into it? Did I miss someone in Cc or sent it to the wrong lists maybe?
Quoted 29 lines · Alexander Dahl
For ARCH=x86 (32 bit) when you set
CONFIG_IOMMU_INTELsincec5a5dc4cbbf4("iommu/vt-d: Don't switch off swiotlb if bounce page is used") there's a dependency onCONFIG_SWIOTLB, which was not necessarily active before.The init code for swiotlb in '
pci_swiotlb_detect_4gb()' compares something againstMAX_DMA32_PFNto decide if it should be active. However that define suffers from an arithmetic overflow since1b7e03ef7570("x86, NUMA: Enable emulation on 32bit too") when it was first made visible tox86_32.The effect is at boot time 64 MiB (default size) were allocated for bounce buffers now, which is a noticeable amount of memory on small systems. We noticed this effect on the fli4l Linux distribution when migrating from kernel v4.19 (LTS) to v5.4 (LTS) on boards like pcengines ALIX 2D3 with 256 MiB memory for example:
Linux version 5.4.22 (buildroot@buildroot) (gcc version 7.3.0 (Buildroot 2018.02.8)) #1 SMP Mon Nov 26 23:40:00 CET 2018
…
Memory: 183484K/261756K available (4594K kernel code, 393K rwdata, 1660K rodata, 536K init, 456K bss , 78272K reserved, 0K cma-reserved, 0K highmem) …
PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
software IO TLB: mapped [mem 0x0bb78000-0x0fb78000] (64MB)The initial analysis and the suggested fix was done by user 'sourcejedi' at stackoverflow and explicitly marked as GPLv2 for inclusion in the Linux kernel:
Quoted 8 lines
Suggested-by Alan Jenkins <[email protected]>Signed-off-by Alexander Dahl <[email protected]>---
We tested this in qemu and on real hardware with fli4l on top of v5.4, v5.5, and v5.6-rc kernels, but only as far as the reserved memory goes. The patch itself is based on v5.6-rc3 (IIRC).
We had no complaints of our fli4l users, since we applied this patch to our distribution kernels.
Thanks & greets
Alex
Quoted 29 lines
A quick grep over the kernel code showed me this define
MAX_DMA32_PFNis used in other places as well. I would appreciate feedback on this, because I can not oversee all side effects this might have?!Thanks again to Alan who proposed the fix, and for his permission to send it upstream.
Greets
Alex--- arch/x86/include/asm/dma.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/x86/include/asm/dma.h +1
-1@@ -74,7 +74,7 @@#define MAX_DMA_PFN ((16UL * 1024 * 1024) >> PAGE_SHIFT)/* 4GB broken PCI/AGP hardware bus master zone */ -#define MAX_DMA32_PFN ((4UL * 1024 * 1024 * 1024) >> PAGE_SHIFT) +#define MAX_DMA32_PFN (4UL * ((1024 * 1024 * 1024) >> PAGE_SHIFT))#ifdef CONFIG_X86_32 /* The maximum address that we can perform a DMA transfer to on this platform */ Signature · 1 line
2.20.1
Signature · 4 lines
/"\ ASCII RIBBON | »With the first link, the chain is forged. The first \ / CAMPAIGN | speech censured, the first thought forbidden, the X AGAINST | first freedom denied, chains us all irrevocably.« / \ HTML MAIL | (Jean-Luc Picard, quoting Judge Aaron Satie)
Quoted 56 lines · Alexander Dahl
For ARCH=x86 (32 bit) when you set
CONFIG_IOMMU_INTELsincec5a5dc4cbbf4("iommu/vt-d: Don't switch off swiotlb if bounce page is used") there's a dependency onCONFIG_SWIOTLB, which was not necessarily active before.The init code for swiotlb in '
pci_swiotlb_detect_4gb()' compares something againstMAX_DMA32_PFNto decide if it should be active. However that define suffers from an arithmetic overflow since1b7e03ef7570("x86, NUMA: Enable emulation on 32bit too") when it was first made visible tox86_32.The effect is at boot time 64 MiB (default size) were allocated for bounce buffers now, which is a noticeable amount of memory on small systems. We noticed this effect on the fli4l Linux distribution when migrating from kernel v4.19 (LTS) to v5.4 (LTS) on boards like pcengines ALIX 2D3 with 256 MiB memory for example:
Linux version 5.4.22 (buildroot@buildroot) (gcc version 7.3.0 (Buildroot 2018.02.8)) #1 SMP Mon Nov 26 23:40:00 CET 2018 … Memory: 183484K/261756K available (4594K kernel code, 393K rwdata, 1660K rodata, 536K init, 456K bss , 78272K reserved, 0K cma-reserved, 0K highmem) … PCI-DMA: Using software bounce buffering for IO (SWIOTLB) software IO TLB: mapped [mem 0x0bb78000-0x0fb78000] (64MB)The initial analysis and the suggested fix was done by user 'sourcejedi' at stackoverflow and explicitly marked as GPLv2 for inclusion in the Linux kernel:
https://unix.stackexchange.com/a/520525/50007Suggested-by Alan Jenkins <[email protected]>Signed-off-by Alexander Dahl <[email protected]>---
We tested this in qemu and on real hardware with fli4l on top of v5.4, v5.5, and v5.6-rc kernels, but only as far as the reserved memory goes. The patch itself is based on v5.6-rc3 (IIRC).A quick grep over the kernel code showed me this define
MAX_DMA32_PFNis used in other places as well. I would appreciate feedback on this, because I can not oversee all side effects this might have?!Thanks again to Alan who proposed the fix, and for his permission to send it upstream.
Greets
Alex--- arch/x86/include/asm/dma.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/x86/include/asm/dma.h +0
-0
arch/x86/include/asm/dma.h line 77
Quoted 5 lines
arch/x86/include/asm/dma.h +1
-1@@ -74,7 +74,7 @@#define MAX_DMA_PFN ((16UL * 1024 * 1024) >> PAGE_SHIFT)/* 4GB broken PCI/AGP hardware bus master zone */ -#define MAX_DMA32_PFN ((4UL * 1024 * 1024 * 1024) >> PAGE_SHIFT) +#define MAX_DMA32_PFN (4UL * ((1024 * 1024 * 1024) >> PAGE_SHIFT))
FWIW, wouldn't s/UL/ULL/ in the original expression suffice? Failing that, rather than awkward parenthesis trickery it might be clearer to just copy the one from arch/mips/include/asm/dma.h.
Robin.
Quoted 4 lines
arch/x86/include/asm/dma.h +0
-0#ifdef CONFIG_X86_32 /* The maximum address that we can perform a DMA transfer to on this platform */
Hello Robin,
Quoted 5 lines · Robin Murphy
Quoted 4 lines · Alexander Dahl
--- arch/x86/include/asm/dma.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 13 lines
arch/x86/include/asm/dma.h line 76
Quoted 9 lines
arch/x86/include/asm/dma.h +1
-1@@ -74,7 +74,7 @@#define MAX_DMA_PFN ((16UL * 1024 * 1024) >> PAGE_SHIFT)/* 4GB broken PCI/AGP hardware bus master zone */ -#define MAX_DMA32_PFN ((4UL * 1024 * 1024 * 1024) >> PAGE_SHIFT) +#define MAX_DMA32_PFN (4UL * ((1024 * 1024 * 1024) >> PAGE_SHIFT))FWIW, wouldn't s/UL/ULL/ in the original expression suffice? Failing that, rather than awkward parenthesis trickery it might be clearer to just copy the one from
arch/mips/include/asm/dma.h.
Both of your suggestions yield the correct result, and at least for me both look easier to understand than what Alan proposed in the first place.
I would opt for the variant which is already in arch/mips, because it avoids 64 bit calculation, is most obvious in intent in my eyes, and we have the same calculation twice then instead of two variants.
Thanks for your review, I'll send a v2. :-)
Greets
Alex
Signature · 4 lines
/"\ ASCII RIBBON | »With the first link, the chain is forged. The first \ / CAMPAIGN | speech censured, the first thought forbidden, the X AGAINST | first freedom denied, chains us all irrevocably.« / \ HTML MAIL | (Jean-Luc Picard, quoting Judge Aaron Satie)
PATCH v2 1/1 dma: Fix max PFN arithmetic overflow on 32 bit systems
For ARCH=x86 (32 bit) when you set CONFIG_IOMMU_INTEL since c5a5dc4cbbf4 ("iommu/vt-d: Don't switch off swiotlb if bounce page is used") there's a dependency on CONFIG_SWIOTLB, which was not necessarily active before.
The init code for swiotlb in 'pci_swiotlb_detect_4gb()' compares something against MAX_DMA32_PFN to decide if it should be active. However that define suffers from an arithmetic overflow since 1b7e03ef7570 ("x86, NUMA: Enable emulation on 32bit too") when it was first made visible to x86_32.
The effect is at boot time 64 MiB (default size) were allocated for bounce buffers now, which is a noticeable amount of memory on small systems. We noticed this effect on the fli4l Linux distribution when migrating from kernel v4.19 (LTS) to v5.4 (LTS) on boards like pcengines ALIX 2D3 with 256 MiB memory for example:
Linux version 5.4.22 (buildroot@buildroot) (gcc version 7.3.0 (Buildroot 2018.02.8)) #1 SMP Mon Nov 26 23:40:00 CET 2018
…
Memory: 183484K/261756K available (4594K kernel code, 393K rwdata, 1660K rodata, 536K init, 456K bss , 78272K reserved, 0K cma-reserved, 0K highmem) …
PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
software IO TLB: mapped [mem 0x0bb78000-0x0fb78000] (64MB)
The initial analysis and the suggested fix was done by user 'sourcejedi' at stackoverflow and explicitly marked as GPLv2 for inclusion in the Linux kernel:
https://unix.stackexchange.com/a/520525/50007
The actual calculation however is the same as for arch/mips now as suggested by Robin Murphy.
---
Notes:
v1 -> v2:
- use the same calculation as with arch/mips (Robin Murphy)
arch/x86/include/asm/dma.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/include/asm/dma.h +1 -1
@@ -74,7 +74,7 @@#define MAX_DMA_PFN ((16UL * 1024 * 1024) >> PAGE_SHIFT)/* 4GB broken PCI/AGP hardware bus master zone */ -#define MAX_DMA32_PFN ((4UL * 1024 * 1024 * 1024) >> PAGE_SHIFT) +#define MAX_DMA32_PFN (1UL << (32 - PAGE_SHIFT))#ifdef CONFIG_X86_32 /* The maximum address that we can perform a DMA transfer to on this platform */
Signature · 1 line
2.20.1
Hello,
now after v5.7-rc1 is out, I would kindly ask, if anyone had time to review this one line patch? Is anything wrong with that fix?
(I added the current fli4l kernel package maintainer Florian to Cc to let him know I'm still having an eye on this.)
Greets
Alex
Quoted 66 lines · Alexander Dahl
For ARCH=x86 (32 bit) when you set
CONFIG_IOMMU_INTELsincec5a5dc4cbbf4("iommu/vt-d: Don't switch off swiotlb if bounce page is used") there's a dependency onCONFIG_SWIOTLB, which was not necessarily active before.The init code for swiotlb in '
pci_swiotlb_detect_4gb()' compares something againstMAX_DMA32_PFNto decide if it should be active. However that define suffers from an arithmetic overflow since1b7e03ef7570("x86, NUMA: Enable emulation on 32bit too") when it was first made visible tox86_32.The effect is at boot time 64 MiB (default size) were allocated for bounce buffers now, which is a noticeable amount of memory on small systems. We noticed this effect on the fli4l Linux distribution when migrating from kernel v4.19 (LTS) to v5.4 (LTS) on boards like pcengines ALIX 2D3 with 256 MiB memory for example:
Linux version 5.4.22 (buildroot@buildroot) (gcc version 7.3.0 (Buildroot 2018.02.8)) #1 SMP Mon Nov 26 23:40:00 CET 2018
…
Memory: 183484K/261756K available (4594K kernel code, 393K rwdata, 1660K rodata, 536K init, 456K bss , 78272K reserved, 0K cma-reserved, 0K highmem) …
PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
software IO TLB: mapped [mem 0x0bb78000-0x0fb78000] (64MB)The initial analysis and the suggested fix was done by user 'sourcejedi' at stackoverflow and explicitly marked as GPLv2 for inclusion in the Linux kernel:
https://unix.stackexchange.com/a/520525/50007
The actual calculation however is the same as for
arch/mipsnow as suggested by Robin Murphy.Reported-by Alan Jenkins <[email protected]>Suggested-by Robin Murphy <[email protected]>Signed-off-by Alexander Dahl <[email protected]>---
Notes:
v1 -> v2:
- use the same calculation as witharch/mips(Robin Murphy)arch/x86/include/asm/dma.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/x86/include/asm/dma.h +1
-1@@ -74,7 +74,7 @@#define MAX_DMA_PFN ((16UL * 1024 * 1024) >> PAGE_SHIFT)/* 4GB broken PCI/AGP hardware bus master zone */ -#define MAX_DMA32_PFN ((4UL * 1024 * 1024 * 1024) >> PAGE_SHIFT) +#define MAX_DMA32_PFN (1UL << (32 - PAGE_SHIFT))#ifdef CONFIG_X86_32 /* The maximum address that we can perform a DMA transfer to on this platform */ Signature · 6 lines
2.20.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Signature · 4 lines
/"\ ASCII RIBBON | »With the first link, the chain is forged. The first \ / CAMPAIGN | speech censured, the first thought forbidden, the X AGAINST | first freedom denied, chains us all irrevocably.« / \ HTML MAIL | (Jean-Luc Picard, quoting Judge Aaron Satie)
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Hei hei,
I would like to kindly ask about the status of this patch.
Quoted 2 lines · Alexander Dahl
now after v5.7-rc1 is out, I would kindly ask, if anyone had time to review this one line patch? Is anything wrong with that fix?
Did it maybe not reach the right maintainers? I used scripts/get_maintainer.pl to get my recipient list.
Quoted 60 lines
(I added the current fli4l kernel package maintainer Florian to Cc to let him know I'm still having an eye on this.)
Greets
AlexQuoted 53 lines · Alexander Dahl
For ARCH=x86 (32 bit) when you set
CONFIG_IOMMU_INTELsincec5a5dc4cbbf4("iommu/vt-d: Don't switch off swiotlb if bounce page is used") there's a dependency onCONFIG_SWIOTLB, which was not necessarily active before.The init code for swiotlb in '
pci_swiotlb_detect_4gb()' compares something againstMAX_DMA32_PFNto decide if it should be active. However that define suffers from an arithmetic overflow since1b7e03ef7570("x86, NUMA: Enable emulation on 32bit too") when it was first made visible tox86_32.The effect is at boot time 64 MiB (default size) were allocated for bounce buffers now, which is a noticeable amount of memory on small systems. We noticed this effect on the fli4l Linux distribution when migrating from kernel v4.19 (LTS) to v5.4 (LTS) on boards like pcengines ALIX 2D3 with 256 MiB memory for example:
Linux version 5.4.22 (buildroot@buildroot) (gcc version 7.3.0 (Buildroot 2018.02.8)) #1 SMP Mon Nov 26 23:40:00 CET 2018
…
Memory: 183484K/261756K available (4594K kernel code, 393K rwdata, 1660K rodata, 536K init, 456K bss , 78272K reserved, 0K cma-reserved, 0K highmem) …
PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
software IO TLB: mapped [mem 0x0bb78000-0x0fb78000] (64MB)The initial analysis and the suggested fix was done by user 'sourcejedi' at stackoverflow and explicitly marked as GPLv2 for inclusion in the Linux kernel:
https://unix.stackexchange.com/a/520525/50007
The actual calculation however is the same as for
arch/mipsnow as suggested by Robin Murphy.Reported-by Alan Jenkins <[email protected]>Suggested-by Robin Murphy <[email protected]>Signed-off-by Alexander Dahl <[email protected]>---
Notes:
v1 -> v2:
- use the same calculation as witharch/mips(Robin Murphy)arch/x86/include/asm/dma.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/x86/include/asm/dma.h +0
-0@@ -74,7 +74,7 @@#define MAX_DMA_PFN ((16UL * 1024 * 1024) >> PAGE_SHIFT)
Quoted 8 lines
Quoted 8 lines
arch/x86/include/asm/dma.h +1
-1/* 4GB broken PCI/AGP hardware bus master zone */ -#define MAX_DMA32_PFN ((4UL * 1024 * 1024 * 1024) >> PAGE_SHIFT) +#define MAX_DMA32_PFN (1UL << (32 - PAGE_SHIFT))#ifdef CONFIG_X86_32 /* The maximum address that we can perform a DMA transfer to on this platform */ Signature · 1 line
2.20.1
Greets
Alex
Signature · 4 lines
/"\ ASCII RIBBON | »With the first link, the chain is forged. The first \ / CAMPAIGN | speech censured, the first thought forbidden, the X AGAINST | first freedom denied, chains us all irrevocably.« / \ HTML MAIL | (Jean-Luc Picard, quoting Judge Aaron Satie)