iommu/vt-d: add NUMA awareness to intel_alloc_coherent()
Subsystems (3)
PATCH 1/1 iommu/vt-d: add NUMA awareness to intel_alloc_coherent()
From: Eric Dumazet <[email protected]>
Some devices (like mlx4) try hard to allocate memory on selected NUMA node, but it turns out intel_alloc_coherent() is not NUMA aware yet.
Note that dma_generic_alloc_coherent() in arch/x86/kernel/pci-dma.c gets this right.
---
drivers/iommu/intel-iommu.c | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
drivers/iommu/intel-iommu.c +5 -2
@@ -3734,8 +3734,11 @@ static void *intel_alloc_coherent(struct device *dev, size_t size, } } -if (!page) - page =alloc_pages (flags, order); +if (!page) { + page =alloc_pages_node (dev_to_node (dev), flags, order); +if (!page) + page =alloc_pages (flags, order); + }if (!page)return NULL ;memset (page_address (page),0 , size);
Quoted 22 lines · Eric Dumazet
From: Eric Dumazet <[email protected]>
Some devices (like mlx4) try hard to allocate memory on selected NUMA node, but it turns out
intel_alloc_coherent()is not NUMA aware yet.Note that
dma_generic_alloc_coherent()inarch/x86/kernel/pci-dma.cgets this right.Signed-off-by Eric Dumazet <[email protected]>Cc Benjamin Serebrin <[email protected]>Cc David Woodhouse <[email protected]>Cc Joerg Roedel <[email protected]>---
drivers/iommu/intel-iommu.c | 7 +++++--1 file changed, 5 insertions(+), 2 deletions(-)drivers/iommu/intel-iommu.c +0
-0
drivers/iommu/intel-iommu.c static void *intel_alloc_coherent(struct device *dev, size_t size, · line 3740
Quoted 9 lines
drivers/iommu/intel-iommu.c +4
-2@@ -3734,8 +3734,11 @@ static void *intel_alloc_coherent(struct device *dev, size_t size, } } -if (!page) - page =alloc_pages (flags, order); +if (!page) { + page =alloc_pages_node (dev_to_node (dev), flags, order); +if (!page) + page =alloc_pages (flags, order);
I'll send a V2 without the fallback to alloc_pages()
This seems not necessary at all.
Quoted 4 lines
drivers/iommu/intel-iommu.c +1
-0+ }if (!page)return NULL ;memset (page_address (page),0 , size);
PATCH v2 1/1 iommu/vt-d: add NUMA awareness to intel_alloc_coherent()
In reply to Eric DumazetFrom: Eric Dumazet <[email protected]>
Some devices (like mlx4) try hard to allocate memory on selected NUMA node, but it turns out intel_alloc_coherent() is not NUMA aware yet.
Note that dma_generic_alloc_coherent() in arch/x86/kernel/pci-dma.c gets this right.
---
v2: no fallback to alloc_pages(), this is not needed and might even hurt in OOM cases.
drivers/iommu/intel-iommu.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iommu/intel-iommu.c +1 -1
@@ -3735,7 +3735,7 @@ static void *intel_alloc_coherent(struct device *dev, size_t size, }if (!page) - page =alloc_pages (flags, order); + page =alloc_pages_node (dev_to_node (dev), flags, order);if (!page)return NULL ;memset (page_address (page),0 , size);
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
I've got patches pending to replace all that code with dma_direct_alloc, which will do the right thing. They were submitted for 4.16, and I will resend them after -rc1.
Quoted 3 lines · Christoph Hellwig
I've got patches pending to replace all that code with
dma_direct_alloc, which will do the right thing. They were submitted for 4.16, and I will resend them after -rc1.
I see, thanks Christoph !
Quoted 7 lines · Eric Dumazet
Quoted 3 lines · Christoph Hellwig
I've got patches pending to replace all that code with
dma_direct_alloc, which will do the right thing. They were submitted for 4.16, and I will resend them after -rc1.I see, thanks Christoph !
Hi Christoph
It seems 4.16 has shipped ( :) ) , and intel_alloc_coherent() still has no NUMA awareness.
Should I respin https://lore.kernel.org/patchwork/patch/884326/
Thanks !
Quoted 8 lines · Eric Dumazet
Quoted 5 lines · Eric Dumazet
Quoted 3 lines · Christoph Hellwig
I've got patches pending to replace all that code with
dma_direct_alloc, which will do the right thing. They were submitted for 4.16, and I will resend them after -rc1.
Quoted 6 lines
Quoted 2 lines
I see, thanks Christoph !
Hi Christoph
It seems 4.16 has shipped ( :) ) , and
intel_alloc_coherent()still has no NUMA awareness.
Actually, that code went in and then got reverted again..
Quoted 1 line
Should I respin https://lore.kernel.org/patchwork/patch/884326/
Maybe. We are still hoping to convert intel-iommu to the dma-iommu framework, but I'm not sure how long that is going to take, so maybe just respin it for now.