Skip to contentSkip to conversation

x86/pci-dma: add a SWIOTLB_ANY flag to lift the low mem limitation

PATCH In review Jun Miao 3 participants 4 messages Jun 25 – Jul 23 x86

lore
Share & tools
lore

Jun Miao

author
#0 lore raw

PATCH v4 1/1 x86/pci-dma: add a SWIOTLB_ANY flag to lift the low mem limitation

When high-speed NICs or multi-GPU setups are passed through into confidential VMs, the SWIOTLB bounce buffer becomes the critical path between private and shared memory. Restricting it to low memory limits throughput and fails to scale for larger workloads.

AMD SEV-SNP and Intel TDX guests run in a TEE where the hypervisor is untrusted. DMA-capable devices require bounce buffers to mediate between encrypted private memory and unencrypted shared memory. Confining these buffers to low memory (<4GB) unnecessarily caps their size and degrades performance.

Power SVM already supports this; x86 does not. See commit 8ba2ed1be9 ("swiotlb: add a SWIOTLB_ANY flag to lift the low memory restriction").

[ aakarsh: completely trim down/rewrite changelog ]

---

v1 -> v2:
- Updated commit message and description.
- Add Reviewed and Tested.
V1 Latest Feedback : https://lists.openwall.net/linux-kernel/2026/02/11/483

v2 -> v3:
- We can alloc 4GB with the dynamic swiotlb, rather than 1GB. 1G is not correct. So change the commit log.

v3 -> v4:
- Not only TDX-specific but all encrypted guests include SEV.
- SEV-SNP guest passed the test with the help of Aakarsh.

- Add "Acked-by: Marek Szyprowski"
- Explain the usage case in the commit log following Boris`s suggestion.

---
 arch/x86/kernel/pci-dma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kernel/pci-dma.c +1 -1
index 6267363e0189..73b9320c4a7d 100644
@@ -61,7 +61,7 @@ static void __init pci_swiotlb_detect(void)
 	 */
 	if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) {
 		x86_swiotlb_enable = true;
-		x86_swiotlb_flags |= SWIOTLB_FORCE;
+		x86_swiotlb_flags |= SWIOTLB_ANY | SWIOTLB_FORCE;
 	}
 }
 #else
Signature · 1 line
2.47.1
Reviewed-by Tested-by on v4 1/1
#1 lore raw

From: Jun Miao <[email protected]> Sent: Wednesday, June 24, 2026 6:26 PM

Quoted 14 lines

When high-speed NICs or multi-GPU setups are passed through into confidential VMs, the SWIOTLB bounce buffer becomes the critical path between private and shared memory. Restricting it to low memory limits throughput and fails to scale for larger workloads.

AMD SEV-SNP and Intel TDX guests run in a TEE where the hypervisor is untrusted. DMA-capable devices require bounce buffers to mediate between encrypted private memory and unencrypted shared memory. Confining these buffers to low memory (<4GB) unnecessarily caps their size and degrades performance.

Power SVM already supports this; x86 does not. See commit 8ba2ed1be9 ("swiotlb: add a SWIOTLB_ANY flag to lift the low memory restriction").

Quoted 7 lines

[ aakarsh: completely trim down/rewrite changelog ]

I tested this change in Intel TDX and AMD SEV-SNP VMs in the Azure cloud, which are running on Hyper-V and with a paravisor. The SEV-SNP VM operates in vTOM mode.

I was able to allocate a 4 GiB swiotlb, whereas prior to this patch specifying 4 GiB resulted in a memory allocation failure and a downgrade to 2 GiB. All bounce buffered DMA operations from the Hyper-V synthetic SCSI and NIC controllers worked correctly, as did DMA operations from a PCI pass-thru NVMe controller in the TDX VM.

Quoted 39 lines

---

v1 -> v2:
- Updated commit message and description.
- Add Reviewed and Tested.
V1 Latest Feedback : https://lists.openwall.net/linux-kernel/2026/02/11/483

v2 -> v3:
- We can alloc 4GB with the dynamic swiotlb, rather than 1GB. 1G is not correct. So change the commit log.

v3 -> v4:
- Not only TDX-specific but all encrypted guests include SEV.
- SEV-SNP guest passed the test with the help of Aakarsh.

- Add "Acked-by: Marek Szyprowski"
- Explain the usage case in the commit log following Boris`s suggestion.

---
 arch/x86/kernel/pci-dma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kernel/pci-dma.c +1 -2
index 6267363e0189..73b9320c4a7d 100644
@@ -61,7 +61,7 @@ static void __init pci_swiotlb_detect(void)
 	 */
 	if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) {
 		x86_swiotlb_enable = true;
-		x86_swiotlb_flags |= SWIOTLB_FORCE;
+		x86_swiotlb_flags |= SWIOTLB_ANY | SWIOTLB_FORCE;
 	}
 }
 #else
--

2.47.1

maintainer
#2 lore raw
In reply to Jun Miao
Quoted 13 lines · Jun Miao

When high-speed NICs or multi-GPU setups are passed through into confidential VMs, the SWIOTLB bounce buffer becomes the critical path between private and shared memory. Restricting it to low memory limits throughput and fails to scale for larger workloads.

AMD SEV-SNP and Intel TDX guests run in a TEE where the hypervisor is untrusted. DMA-capable devices require bounce buffers to mediate between encrypted private memory and unencrypted shared memory. Confining these buffers to low memory (<4GB) unnecessarily caps their size and degrades performance.

Power SVM already supports this; x86 does not. See commit 8ba2ed1be9 ("swiotlb: add a SWIOTLB_ANY flag to lift the low memory restriction").

Quoted 8 lines

[ aakarsh: completely trim down/rewrite changelog ]

---

Sashiko has a question:

https://lore.kernel.org/r/[email protected]

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
author
#3 lore raw
Quoted 26 lines · Jun Miao
Quoted 6 lines

When high-speed NICs or multi-GPU setups are passed through into confidential VMs, the SWIOTLB bounce buffer becomes the critical path between private and shared memory. Restricting it to low memory limits throughput and fails to scale for larger workloads.

AMD SEV-SNP and Intel TDX guests run in a TEE where the hypervisor is

untrusted.

Quoted 3 lines

DMA-capable devices require bounce buffers to mediate between encrypted private memory and unencrypted shared memory. Confining these buffers to low memory (<4GB) unnecessarily caps their size and

degrades performance.

Quoted 4 lines

Power SVM already supports this; x86 does not. See commit 8ba2ed1be9 ("swiotlb: add a SWIOTLB_ANY flag to lift the low memory restriction").

Quoted 8 lines

[ aakarsh: completely trim down/rewrite changelog ]

---

Quoted 5 lines

Sashiko has a question:

https://lore.kernel.org/r/[email protected]

Sashiko said “Does x86 lack DMA-mask filtering during pool search?”:

Conclusion: yes, it does (at least in the current common SWIOTLB search path).
Therefore, after enabling SWIOTLB_ANY, dma-direct with 32-bit DMA-mask devices is indeed more likely to hit the case where a high-address bounce buffer is allocated first and then rejected by dma_capable().

I plan to modify SWIOTLB directly: in the direct DMA path, I’ll filter candidate pool/slot addresses by dma_capable() during the search phase, so we avoid the case where a high address is allocated first and then fails without retrying a low-memory buffer. At the same time, I’ll keep the IOMMU/Xen paths unchanged.

Warm regards
Jun Miao

Quoted 5 lines

--
Regards/Gruss,
Boris.

https://people.kernel.org/tglx/notes-about-netiquette