x86: enable swiotlb for > 4GiG ram on 32-bit kernels
PATCH 1/1 x86: enable swiotlb for > 4GiG ram on 32-bit kernels
We already build the swiotlb code for 32b-t kernels with PAE support, but the code to actually use swiotlb has only been enabled for 64-bit kernel for an unknown reason.
Before Linux 4.18 we papers over this fact because the networking code, the scsi layer and some random block drivers implenented their own bounce buffering scheme.
--- arch/x86/kernel/pci-swiotlb.c | 2 -- 1 file changed, 2 deletions(-)
arch/x86/kernel/pci-swiotlb.c +0 -2
@@ -42,10 +42,8 @@ IOMMU_INIT_FINISH(pci_swiotlb_detect_override,int __init pci_swiotlb_detect_4gb (void ) {/* don't initialize swiotlb if iommu=off (no_iommu=1) */ -#ifdef CONFIG_X86_64 if (!no_iommu && max_possible_pfn >MAX_DMA32_PFN ) swiotlb =1 ; -#endif /* * If SME is active then swiotlb will be set to 1 so that bounce
Signature · 1 line
2.19.1
Quoted 4 lines · Christoph Hellwig
We already build the swiotlb code for 32b-t kernels with PAE support, but the code to actually use swiotlb has only been enabled for 64-bit kernel for an unknown reason.
Quoted 8 lines
Before Linux 4.18 we papers over this fact because the networking code, the scsi layer and some random block drivers implenented their own bounce buffering scheme.
Fixes 21e07dba ("scsi: reduce use of block bounce buffers")Fixes ab74cfeb ("net: remove the PCI_DMA_BUS_IS_PHYS check in illegal_highdma")Reported-by tedheadster <[email protected]>Tested-by tedheadster <[email protected]>
I'll add your SOB when picking this up :)
Quoted 23 lines
--- arch/x86/kernel/pci-swiotlb.c | 2 -- 1 file changed, 2 deletions(-)arch/x86/kernel/pci-swiotlb.c +0
-2@@ -42,10 +42,8 @@ IOMMU_INIT_FINISH(pci_swiotlb_detect_override,int __init pci_swiotlb_detect_4gb (void ) {/* don't initialize swiotlb if iommu=off (no_iommu=1) */ -#ifdef CONFIG_X86_64 if (!no_iommu && max_possible_pfn >MAX_DMA32_PFN ) swiotlb =1 ; -#endif /* * If SME is active then swiotlb will be set to 1 so that bounce Signature · 1 line
2.19.1
Quoted 10 lines · Thomas Gleixner
Quoted 8 lines · Christoph Hellwig
We already build the swiotlb code for 32b-t kernels with PAE support, but the code to actually use swiotlb has only been enabled for 64-bit kernel for an unknown reason.
Before Linux 4.18 we papers over this fact because the networking code, the scsi layer and some random block drivers implenented their own bounce buffering scheme.
Quoted 6 lines
Quoted 4 lines
Fixes 21e07dba ("scsi: reduce use of block bounce buffers")Fixes ab74cfeb ("net: remove the PCI_DMA_BUS_IS_PHYS check in illegal_highdma")Reported-by tedheadster <[email protected]>Tested-by tedheadster <[email protected]>I'll add your SOB when picking this up :)
Thanks. Here it is in writing:
Quoted 7 lines · Christoph Hellwig
Quoted 6 lines · Thomas Gleixner
Quoted 4 lines · Christoph Hellwig
We already build the swiotlb code for 32b-t kernels with PAE support, but the code to actually use swiotlb has only been enabled for 64-bit kernel for an unknown reason.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Before Linux 4.18 we papers over this fact because the networking code, the scsi layer and some random block drivers implenented their own bounce buffering scheme.
Fixes 21e07dba ("scsi: reduce use of block bounce buffers")
Please use the first 12 characters of the commit SHA for fixes tags in the future, as documented. No need to resend, I fixed it up for you and added a
Cc: stable as well
Thanks,
tglx
Quoted 13 lines · Thomas Gleixner
Quoted 11 lines · Christoph Hellwig
Quoted 10 lines · Thomas Gleixner
Quoted 8 lines · Christoph Hellwig
We already build the swiotlb code for 32b-t kernels with PAE support, but the code to actually use swiotlb has only been enabled for 64-bit kernel for an unknown reason.
Before Linux 4.18 we papers over this fact because the networking code, the scsi layer and some random block drivers implenented their own bounce buffering scheme.
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Fixes 21e07dba ("scsi: reduce use of block bounce buffers")Please use the first 12 characters of the commit SHA for fixes tags in the future, as documented. No need to resend, I fixed it up for you and added a
Cc: stable as well
For those who have their ~/.gitconfig's from ancient Git history, this can be done via:
git config --global core.abbrev 12
Thanks,
Ingo
Please change:
to
- Matthew
tip-bot for Christoph Hellwig
merge bot[tip:x86/urgent] x86/swiotlb: Enable swiotlb for > 4GiG ram on 32-bit kernels
In reply to Christoph Hellwigx86/swiotlb: Enable swiotlb for > 4GiG ram on 32-bit kernels
Commit-ID: ab555321e4ddc7f6f17f6c80dfaad228883a8e7c Gitweb: https://git.kernel.org/tip/ab555321e4ddc7f6f17f6c80dfaad228883a8e7c Author: Christoph Hellwig <[email protected]> AuthorDate: Sun, 14 Oct 2018 09:52:08 +0200 Committer: Thomas Gleixner <[email protected]> CommitDate: Sun, 14 Oct 2018 11:11:23 +0200
x86/swiotlb: Enable swiotlb for > 4GiG ram on 32-bit kernels
We already build the swiotlb code for 32b-t kernels with PAE support, but the code to actually use swiotlb has only been enabled for 64-bit kernel for an unknown reason.
Before Linux 4.18 we paper over this fact because the networking code, the scsi layer and some random block drivers implemented their own bounce buffering scheme.
--- arch/x86/kernel/pci-swiotlb.c | 2 -- 1 file changed, 2 deletions(-)
arch/x86/kernel/pci-swiotlb.c +0 -2
@@ -42,10 +42,8 @@ IOMMU_INIT_FINISH(pci_swiotlb_detect_override,int __init pci_swiotlb_detect_4gb (void ) {/* don't initialize swiotlb if iommu=off (no_iommu=1) */ -#ifdef CONFIG_X86_64 if (!no_iommu && max_possible_pfn >MAX_DMA32_PFN ) swiotlb =1 ; -#endif /* * If SME is active then swiotlb will be set to 1 so that bounce
tip-bot for Christoph Hellwig
merge bot[tip:x86/urgent] x86/swiotlb: Enable swiotlb for > 4GiG ram on 32-bit kernels
In reply to Christoph Hellwigx86/swiotlb: Enable swiotlb for > 4GiG ram on 32-bit kernels
Commit-ID: 6f3bc8028570e4c326030e8795dbcd57c561b723 Gitweb: https://git.kernel.org/tip/6f3bc8028570e4c326030e8795dbcd57c561b723 Author: Christoph Hellwig <[email protected]> AuthorDate: Sun, 14 Oct 2018 09:52:08 +0200 Committer: Thomas Gleixner <[email protected]> CommitDate: Mon, 15 Oct 2018 10:55:03 +0200
x86/swiotlb: Enable swiotlb for > 4GiG ram on 32-bit kernels
We already build the swiotlb code for 32b-t kernels with PAE support, but the code to actually use swiotlb has only been enabled for 64-bit kernel for an unknown reason.
Before Linux 4.18 we paper over this fact because the networking code, the scsi layer and some random block drivers implemented their own bounce buffering scheme.
--- arch/x86/kernel/pci-swiotlb.c | 2 -- 1 file changed, 2 deletions(-)
arch/x86/kernel/pci-swiotlb.c +0 -2
@@ -42,10 +42,8 @@ IOMMU_INIT_FINISH(pci_swiotlb_detect_override,int __init pci_swiotlb_detect_4gb (void ) {/* don't initialize swiotlb if iommu=off (no_iommu=1) */ -#ifdef CONFIG_X86_64 if (!no_iommu && max_possible_pfn >MAX_DMA32_PFN ) swiotlb =1 ; -#endif /* * If SME is active then swiotlb will be set to 1 so that bounce
Quoted 4 lines · Christoph Hellwig
We already build the swiotlb code for 32b-t kernels with PAE support, but the code to actually use swiotlb has only been enabled for 64-bit kernel for an unknown reason.
Quoted 8 lines
Before Linux 4.18 we papers over this fact because the networking code, the scsi layer and some random block drivers implenented their own bounce buffering scheme.
Fixes 21e07dba ("scsi: reduce use of block bounce buffers")Fixes ab74cfeb ("net: remove the PCI_DMA_BUS_IS_PHYS check in illegal_highdma")Reported-by tedheadster <[email protected]>Tested-by tedheadster <[email protected]>
But since it is already in 'tip' feel free to ignore my tag.
Thanks!
Quoted 20 lines
--- arch/x86/kernel/pci-swiotlb.c | 2 -- 1 file changed, 2 deletions(-)arch/x86/kernel/pci-swiotlb.c +0
-2@@ -42,10 +42,8 @@ IOMMU_INIT_FINISH(pci_swiotlb_detect_override,int __init pci_swiotlb_detect_4gb (void ) {/* don't initialize swiotlb if iommu=off (no_iommu=1) */ -#ifdef CONFIG_X86_64 if (!no_iommu && max_possible_pfn >MAX_DMA32_PFN ) swiotlb =1 ; -#endif /* * If SME is active then swiotlb will be set to 1 so that bounce
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 9 lines · Christoph Hellwig
We already build the swiotlb code for 32b-t kernels with PAE support, but the code to actually use swiotlb has only been enabled for 64-bit kernel for an unknown reason.
Before Linux 4.18 we papers over this fact because the networking code, the scsi layer and some random block drivers implenented their own bounce buffering scheme.
Quoted 6 lines
Fixes 21e07dba ("scsi: reduce use of block bounce buffers")Fixes ab74cfeb ("net: remove the PCI_DMA_BUS_IS_PHYS check in illegal_highdma")Reported-by tedheadster <[email protected]>Tested-by tedheadster <[email protected]>---
Christoph,
this fix has causes performance to decrease dramatically. Kernel
builds that used to take 10-15 minutes are now taking 45-60 minutes on the same machine.
- Matthew Whitehead
A message that is not in the archive (sent off-list or not yet received): CAP8WD_Zj5aMcTCVtV-hot3zLNfNuMzHyeogm0Lqb=+XZ1LdkKw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Quoted 17 lines · tedheadster
Quoted 9 lines · Christoph Hellwig
We already build the swiotlb code for 32b-t kernels with PAE support, but the code to actually use swiotlb has only been enabled for 64-bit kernel for an unknown reason.
Before Linux 4.18 we papers over this fact because the networking code, the scsi layer and some random block drivers implenented their own bounce buffering scheme.
Quoted 6 lines
Fixes 21e07dba ("scsi: reduce use of block bounce buffers")Fixes ab74cfeb ("net: remove the PCI_DMA_BUS_IS_PHYS check in illegal_highdma")Reported-by tedheadster <[email protected]>Tested-by tedheadster <[email protected]>---
Quoted 4 lines
Christoph,
this fix has causes performance to decrease dramatically. Kernel
builds that used to take 10-15 minutes are now taking 45-60 minutes on the same machine.
Christoph, can you have a look please?
Thanks,
tglx
Quoted 4 lines · tedheadster
Christoph,
this fix has causes performance to decrease dramatically. Kernel
builds that used to take 10-15 minutes are now taking 45-60 minutes on the same machine.
What is the comparism? Is this on a non-aic7xxx kernel with and without the fix?
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 2 lines
What is the comparism? Is this on a non-aic7xxx kernel with and without the fix?
Christoph,
I will try and do some strict before-and-after benchmarking of your
patch. It will take a few days because I am on travel. This is on an aic7xxx (it does have that controller) kernel with and without the fix.
- Matthew Whitehead
A message that is not in the archive (sent off-list or not yet received): CAP8WD_boL-DxEXxa-ci382Ff8yTdMS9T+sy2Y0-LEX-WPc8L_g-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Quoted 8 lines · tedheadster
Quoted 2 lines
What is the comparism? Is this on a non-aic7xxx kernel with and without the fix?
Christoph,
I will try and do some strict before-and-after benchmarking of your
patch. It will take a few days because I am on travel. This is on an aic7xxx (it does have that controller) kernel with and without the fix.
But you said without the fix it doesn't work at all? Or is this the same box, just with the aic7xxx controller disabled?
In general the patch should only have two effects:
- set a small amount of memory aside for bounce buffering
- switch the default dma_ops from dma_direct_ops to swiotlb_ops
I can't really see how either could have such a huge effect, even with swiotlb having a couple more wired up ops for which we'd enable spectre mitigations.
So a strict before and after would be very interesting, if it is really just this one change that causes such a huge drop we have hidden dragons somewhere..
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 8 lines
But you said without the fix it doesn't work at all? Or is this the same box, just with the aic7xxx controller disabled?
In general the patch should only have two effects:
- set a small amount of memory aside for bounce buffering
- switch the defaultdma_opsfromdma_direct_opstoswiotlb_ops
Quoted 7 lines
I can't really see how either could have such a huge effect, even with swiotlb having a couple more wired up ops for which we'd enable spectre mitigations.
So a strict before and after would be very interesting, if it is really just this one change that causes such a huge drop we have hidden dragons somewhere..
Christoph,
I did a very controlled before-and-after and got very sensible
results. All compiles were close in time with patched and un-patched kernels.
I must have screwed something up with my last round of testing.
Ingo: I am confident this patch should be accepted.
- Matthew Whitehead
A message that is not in the archive (sent off-list or not yet received): CAP8WD_Z2XvLsqimSnODwr5gj6GRuzh4SMKN56n6Y0JzufG=E8A-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Quoted 16 lines · tedheadster
Quoted 8 lines
But you said without the fix it doesn't work at all? Or is this the same box, just with the aic7xxx controller disabled?
In general the patch should only have two effects:
- set a small amount of memory aside for bounce buffering
- switch the defaultdma_opsfromdma_direct_opstoswiotlb_opsQuoted 7 lines
I can't really see how either could have such a huge effect, even with swiotlb having a couple more wired up ops for which we'd enable spectre mitigations.
So a strict before and after would be very interesting, if it is really just this one change that causes such a huge drop we have hidden dragons somewhere..
Quoted 8 lines
Christoph,
I did a very controlled before-and-after and got very sensible
results. All compiles were close in time with patched and un-patched kernels.I must have screwed something up with my last round of testing.
Ingo: I am confident this patch should be accepted.
Thanks for the update, I've re-applied this to tip:x86/urgent.
Ingo
Ingo Molnar
Tested-by Matthew WhiteheadQuoted 17 lines · tedheadster
Quoted 9 lines · Christoph Hellwig
We already build the swiotlb code for 32b-t kernels with PAE support, but the code to actually use swiotlb has only been enabled for 64-bit kernel for an unknown reason.
Before Linux 4.18 we papers over this fact because the networking code, the scsi layer and some random block drivers implenented their own bounce buffering scheme.
Quoted 6 lines
Fixes 21e07dba ("scsi: reduce use of block bounce buffers")Fixes ab74cfeb ("net: remove the PCI_DMA_BUS_IS_PHYS check in illegal_highdma")Reported-by tedheadster <[email protected]>Tested-by tedheadster <[email protected]>---
Quoted 4 lines
Christoph,
this fix has causes performance to decrease dramatically. Kernel
builds that used to take 10-15 minutes are now taking 45-60 minutes on the same machine.
Ok, this is way too severe regression, and because the two offending commits:
... are from half a year ago and are in v4.18 already. Fixes should not cause new regressions in any case.
So I've removed this patch from tip:x86/urgent for now, could you please re-apply it when you do your testing? I've attached it below.
Thanks,
Ingo
===================>
Quoted 1 line
From
6f3bc8028570e4c326030e8795dbcd57c561b723Mon Sep 17 00:00:00 2001
From: Christoph Hellwig <[email protected]>
Date: Sun, 14 Oct 2018 09:52:08 +0200
Subject: [PATCH] x86/swiotlb: Enable swiotlb for > 4GiG ram on 32-bit kernels
We already build the swiotlb code for 32b-t kernels with PAE support, but the code to actually use swiotlb has only been enabled for 64-bit kernel for an unknown reason.
Before Linux 4.18 we paper over this fact because the networking code, the scsi layer and some random block drivers implemented their own bounce buffering scheme.
--- arch/x86/kernel/pci-swiotlb.c | 2 -- 1 file changed, 2 deletions(-)
arch/x86/kernel/pci-swiotlb.c +0 -2
@@ -42,10 +42,8 @@ IOMMU_INIT_FINISH(pci_swiotlb_detect_override,int __init pci_swiotlb_detect_4gb (void ) {/* don't initialize swiotlb if iommu=off (no_iommu=1) */ -#ifdef CONFIG_X86_64 if (!no_iommu && max_possible_pfn >MAX_DMA32_PFN ) swiotlb =1 ; -#endif /* * If SME is active then swiotlb will be set to 1 so that bounce
tip-bot for Christoph Hellwig
merge bot[tip:x86/urgent] x86/swiotlb: Enable swiotlb for > 4GiG RAM on 32-bit kernels
In reply to Christoph Hellwigx86/swiotlb: Enable swiotlb for > 4GiG RAM on 32-bit kernels
Commit-ID: 485734f3fc77c1eb77ffe138c027b9a4bf0178f3 Gitweb: https://git.kernel.org/tip/485734f3fc77c1eb77ffe138c027b9a4bf0178f3 Author: Christoph Hellwig <[email protected]> AuthorDate: Sun, 14 Oct 2018 09:52:08 +0200 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 19 Oct 2018 07:49:32 +0200
x86/swiotlb: Enable swiotlb for > 4GiG RAM on 32-bit kernels
We already build the swiotlb code for 32-bit kernels with PAE support, but the code to actually use swiotlb has only been enabled for 64-bit kernels for an unknown reason.
Before Linux v4.18 we paper over this fact because the networking code, the SCSI layer and some random block drivers implemented their own bounce buffering scheme.
[ mingo: Changelog fixes. ]
--- arch/x86/kernel/pci-swiotlb.c | 2 -- 1 file changed, 2 deletions(-)
arch/x86/kernel/pci-swiotlb.c +0 -2
@@ -42,10 +42,8 @@ IOMMU_INIT_FINISH(pci_swiotlb_detect_override,int __init pci_swiotlb_detect_4gb (void ) {/* don't initialize swiotlb if iommu=off (no_iommu=1) */ -#ifdef CONFIG_X86_64 if (!no_iommu && max_possible_pfn >MAX_DMA32_PFN ) swiotlb =1 ; -#endif /* * If SME is active then swiotlb will be set to 1 so that bounce