Bounce buffer for untrusted devices
Subsystems (3)
cover Bounce buffer for untrusted devices
An external PCI device is a PCI peripheral device connected to the system through an external bus, such as Thunderbolt. What makes it different is that it can't be trusted to the same degree as the devices build into the system. Generally, a trusted PCIe device will DMA into the designated buffers and not overrun or otherwise write outside the specified bounds. But it's different for an external device. The minimum IOMMU mapping granularity is one page (4k), so for DMA transfers smaller than that a malicious PCIe device can access the whole page of memory even if it does not belong to the driver in question. This opens a possibility for DMA attack. For more information about DMA attacks imposed by an untrusted PCI/PCIe device, please refer to [2].
This implements bounce buffer for the untrusted external devices. The transfers should be limited in isolated pages so the IOMMU window does not cover memory outside of what the driver expects. Full pages within a buffer could be directly mapped in IOMMU page table, but for partial pages we use bounce page instead.
In addition, the IOMMU mappings cached in the IOTLB for untrusted devices should be invalidated immediately after the unmap operation. Otherwise, the IOMMU window is left open to attacks.
The implementation of bounce buffers for untrusted devices will cause a little performance overhead, but we didn't see any user experience problems. The users could use the kernel parameter of "intel_iommu=nobounce" to remove the performance overhead if they trust their devices enough.
The Thunderbolt vulnerabiltiies is public and has a nice name as Thunderclap nowadays. Please refer to [1] [3] for more information. This patch series aims to mitigate the concerns.
The bounce buffer idea:
The patch series has been tested by:
[1] https://thunderclap.io/
[2] https://thunderclap.io/thunderclap-paper-ndss2019.pdf
[3] https://christian.kellner.me/2019/02/27/thunderclap-and-linux/
Lu Baolu (9):
iommu/vt-d: Add trace events for domain map/unmap iommu/vt-d: Add helpers for domain mapping/unmapping iommu/vt-d: Add address walk helper iommu/vt-d: Add bounce buffer API for domain map/unmap iommu/vt-d: Add bounce buffer API for dma sync iommu/vt-d: Check whether device requires bounce buffer iommu/vt-d: Add dma sync ops for untrusted devices iommu/vt-d: Flush IOTLB for untrusted device in time iommu/vt-d: Use bounce buffer for untrusted devices
.../admin-guide/kernel-parameters.txt | 5 + drivers/iommu/Makefile | 1 + drivers/iommu/intel-iommu.c | 360 ++++++++++-- drivers/iommu/intel-pgtable.c | 518 ++++++++++++++++++ drivers/iommu/intel-trace.c | 14 + include/linux/intel-iommu.h | 24 + include/trace/events/intel_iommu.h | 132 +++++ 7 files changed, 1010 insertions(+), 44 deletions(-) create mode 100644 drivers/iommu/intel-pgtable.c create mode 100644 drivers/iommu/intel-trace.c create mode 100644 include/trace/events/intel_iommu.h
Signature · 1 line
2.17.1
PATCH 1/9 iommu/vt-d: Add trace events for domain map/unmap
This adds trace support for the Intel IOMMU driver. It also declares some events which could be used to trace the events when an IOVA is being mapped or unmapped in a domain.
--- drivers/iommu/Makefile | 1 + drivers/iommu/intel-trace.c | 14 +++ include/trace/events/intel_iommu.h | 132 +++++++++++++++++++++++++++++ 3 files changed, 147 insertions(+) create mode 100644 drivers/iommu/intel-trace.c create mode 100644 include/trace/events/intel_iommu.h
drivers/iommu/Makefile +1 -0
@@ -17,6 +17,7 @@ obj-$(CONFIG_ARM_SMMU) += arm-smmu.o obj-$(CONFIG_ARM_SMMU_V3) += arm-smmu-v3.o obj-$(CONFIG_DMAR_TABLE) += dmar.o obj-$(CONFIG_INTEL_IOMMU) += intel-iommu.o intel-pasid.o +obj-$(CONFIG_INTEL_IOMMU) += intel-trace.o obj-$(CONFIG_INTEL_IOMMU_DEBUGFS) += intel-iommu-debugfs.o obj-$(CONFIG_INTEL_IOMMU_SVM) += intel-svm.o obj-$(CONFIG_IPMMU_VMSA) += ipmmu-vmsa.o
drivers/iommu/intel-trace.c new file +14 -0
@@ -0,0 +1,14 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Intel IOMMU trace support +* +* Copyright (C) 2019 Intel Corporation +* +* Author: Lu Baolu <[email protected]> +*/ + +#include <linux/string.h> +#include <linux/types.h> + +#define CREATE_TRACE_POINTS +#include <trace/events/intel_iommu.h>
include/trace/events/intel_iommu.h new file +132 -0
132 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH 2/9 iommu/vt-d: Add helpers for domain mapping/unmapping
In reply to Lu BaoluThis adds two helpers to map or unmap a physically contiguous memory region in the page table of an iommu domain.
--- drivers/iommu/intel-iommu.c | 35 +++++++++++++++++++++++++++++++++++ include/linux/intel-iommu.h | 5 +++++ 2 files changed, 40 insertions(+)
drivers/iommu/intel-iommu.c +35 -0
@@ -5847,3 +5847,38 @@ static void __init check_tylersburg_isoch(void)pr_warn ("Recommended TLB entries for ISOCH unit is 16; your BIOS set %d\n" , vtisochctrl); } + +/* +* Map a physically contiguous memory region in the dma page table +* of an iommu domain. +*/ +int +domain_iomap_range (struct dmar_domain *domain,unsigned long addr, +phys_addr_t paddr,size_t size,int prot) +{ +return domain_mapping (domain, +mm_to_dma_pfn (IOVA_PFN (addr)), +NULL , +mm_to_dma_pfn (IOVA_PFN (paddr)), +aligned_nrpages (addr, size), + prot); +} + +/* +* Unmap a physically contiguous memory region in the dma page table +* of an iommu domain. Return the free page list that should be freed +* after iotlb flush. We can't free these pages here since it possibly +* is still used by iotlb cache for a DMA translation. +*/ +struct page * +domain_iounmap_range (struct dmar_domain *domain,unsigned long addr, +size_t size) +{ +unsigned long nrpages, start_pfn, last_pfn; + + nrpages =aligned_nrpages (addr, size); + start_pfn =mm_to_dma_pfn (IOVA_PFN (addr)); + last_pfn = start_pfn + nrpages -1 ; + +return domain_unmap (domain, start_pfn, last_pfn); +}
include/linux/intel-iommu.h +5 -0
@@ -663,6 +663,11 @@ int for_each_device_domain(int (*fn)(struct device_domain_info *info,void iommu_flush_write_buffer (struct intel_iommu *iommu);int intel_iommu_enable_pasid (struct intel_iommu *iommu,struct device *dev); +int domain_iomap_range (struct dmar_domain *domain,unsigned long addr, +phys_addr_t paddr,size_t size,int prot); +struct page *domain_iounmap_range (struct dmar_domain *domain, +unsigned long addr,size_t size); +#ifdef CONFIG_INTEL_IOMMU_SVM int intel_svm_init (struct intel_iommu *iommu);extern int intel_svm_enable_prq (struct intel_iommu *iommu);
Signature · 1 line
2.17.1
PATCH 3/9 iommu/vt-d: Add address walk helper
In reply to Lu BaoluThis adds a helper to walk a contiguous dma address and divide the address space into possiblely three
parts: a start partial page, middle full pages and an end partial page, and call the callback for each part of the address.
--- drivers/iommu/Makefile | 2 +- drivers/iommu/intel-pgtable.c | 109 ++++++++++++++++++++++++++++++++++ include/linux/intel-iommu.h | 6 ++ 3 files changed, 116 insertions(+), 1 deletion(-) create mode 100644 drivers/iommu/intel-pgtable.c
drivers/iommu/Makefile +1 -1
@@ -17,7 +17,7 @@ obj-$(CONFIG_ARM_SMMU) += arm-smmu.o obj-$(CONFIG_ARM_SMMU_V3) += arm-smmu-v3.o obj-$(CONFIG_DMAR_TABLE) += dmar.o obj-$(CONFIG_INTEL_IOMMU) += intel-iommu.o intel-pasid.o -obj-$(CONFIG_INTEL_IOMMU) += intel-trace.o +obj-$(CONFIG_INTEL_IOMMU) += intel-trace.o intel-pgtable.o obj-$(CONFIG_INTEL_IOMMU_DEBUGFS) += intel-iommu-debugfs.o obj-$(CONFIG_INTEL_IOMMU_SVM) += intel-svm.o obj-$(CONFIG_IPMMU_VMSA) += ipmmu-vmsa.o
drivers/iommu/intel-pgtable.c new file +109 -0
109 lines not on this page · raw message
include/linux/intel-iommu.h +6 -0
@@ -668,6 +668,12 @@ int domain_iomap_range(struct dmar_domain *domain, unsigned long addr,struct page *domain_iounmap_range (struct dmar_domain *domain,unsigned long addr,size_t size); +struct bounce_param { +int prot; +enum dma_data_direction dir; +struct page **freelist; +}; +#ifdef CONFIG_INTEL_IOMMU_SVM int intel_svm_init (struct intel_iommu *iommu);extern int intel_svm_enable_prq (struct intel_iommu *iommu);
Signature · 1 line
2.17.1
PATCH 4/9 iommu/vt-d: Add bounce buffer API for domain map/unmap
In reply to Lu BaoluThis adds the APIs for bounce buffer specified domain map() and unmap(). The start and end partial pages will be mapped with bounce buffered pages instead. This will enhance the security of DMA buffer by isolating the DMA attacks from malicious devices.
--- drivers/iommu/intel-iommu.c | 3 + drivers/iommu/intel-pgtable.c | 305 +++++++++++++++++++++++++++++++++- include/linux/intel-iommu.h | 7 + 3 files changed, 311 insertions(+), 4 deletions(-)
drivers/iommu/intel-iommu.c +3 -0
@@ -1724,6 +1724,7 @@ static struct dmar_domain *alloc_domain(int flags) domain->flags = flags; domain->has_iotlb_device = false;INIT_LIST_HEAD (&domain->devices); +idr_init (&domain->bounce_idr);return domain; }
@@ -1919,6 +1920,8 @@ static void domain_exit(struct dmar_domain *domain)dma_free_pagelist (freelist); +idr_destroy (&domain->bounce_idr); +free_domain_mem (domain); }
drivers/iommu/intel-pgtable.c +301 -4
332 lines not on this page · raw message
include/linux/intel-iommu.h +7 -0
@@ -498,6 +498,7 @@ struct dmar_domain {struct dma_pte *pgd;/* virtual address */ int gaw;/* max guest address width */ +struct idr bounce_idr;/* IDR for iova_pfn to bounce page */ /* adjusted guest address width, 0 is level 2 30-bit */ int agaw;
@@ -674,6 +675,12 @@ struct bounce_param {struct page **freelist; }; +int domain_bounce_map (struct dmar_domain *domain,dma_addr_t addr, +phys_addr_t paddr,size_t size, +struct bounce_param *param); +int domain_bounce_unmap (struct dmar_domain *domain,dma_addr_t addr, +phys_addr_t paddr,size_t size, +struct bounce_param *param);#ifdef CONFIG_INTEL_IOMMU_SVM int intel_svm_init (struct intel_iommu *iommu);extern int intel_svm_enable_prq (struct intel_iommu *iommu);
Signature · 1 line
2.17.1
Re: [PATCH v1 4/9] iommu/vt-d: Add bounce buffer API for domain map/unmap
Quoted 5 lines · Lu Baolu
This adds the APIs for bounce buffer specified domain
map()andunmap(). The start and end partial pages will be mapped with bounce buffered pages instead. This will enhance the security of DMA buffer by isolating the DMA attacks from malicious devices.
Please reuse the swiotlb code instead of reinventing it.
Re: [PATCH v1 4/9] iommu/vt-d: Add bounce buffer API for domain map/unmap
Hi,
Quoted 9 lines · Christoph Hellwig
Quoted 5 lines · Lu Baolu
This adds the APIs for bounce buffer specified domain
map()andunmap(). The start and end partial pages will be mapped with bounce buffered pages instead. This will enhance the security of DMA buffer by isolating the DMA attacks from malicious devices.Please reuse the swiotlb code instead of reinventing it.
I don't think we are doing the same thing as swiotlb here. But it's always a good thing to reuse code if possible. I considered this when writing this code, but I found it's hard to do. Do you mind pointing me to the code which I could reuse?
Best regards,
Lu Baolu
Re: [PATCH v1 4/9] iommu/vt-d: Add bounce buffer API for domain map/unmap
Hi again,
Quoted 12 lines · Lu Baolu
Hi,
Quoted 9 lines · Christoph Hellwig
Quoted 5 lines · Lu Baolu
This adds the APIs for bounce buffer specified domain
map()andunmap(). The start and end partial pages will be mapped with bounce buffered pages instead. This will enhance the security of DMA buffer by isolating the DMA attacks from malicious devices.Please reuse the swiotlb code instead of reinventing it.
Just looked into the code again. At least we could reuse below functions:
swiotlb_tbl_map_single()swiotlb_tbl_unmap_single()swiotlb_tbl_sync_single()
Anything else?
Best regards,
Lu Baolu
Quoted 9 lines
I don't think we are doing the same thing as swiotlb here. But it's always a good thing to reuse code if possible. I considered this when writing this code, but I found it's hard to do. Do you mind pointing me to the code which I could reuse?
Best regards,
Lu Baolu
Re: [PATCH v1 4/9] iommu/vt-d: Add bounce buffer API for domain map/unmap
Quoted 16 lines · Lu Baolu
Hi again,
Quoted 12 lines · Lu Baolu
Hi,
Quoted 9 lines · Christoph Hellwig
Quoted 5 lines · Lu Baolu
This adds the APIs for bounce buffer specified domain
map()andunmap(). The start and end partial pages will be mapped with bounce buffered pages instead. This will enhance the security of DMA buffer by isolating the DMA attacks from malicious devices.Please reuse the swiotlb code instead of reinventing it.
Quoted 8 lines
Just looked into the code again. At least we could reuse below functions:
swiotlb_tbl_map_single()swiotlb_tbl_unmap_single()swiotlb_tbl_sync_single()Anything else?
Yes, that is probably about the level you want to reuse, given that the next higher layer already has hooks into the direct mapping code.
Re: [PATCH v1 4/9] iommu/vt-d: Add bounce buffer API for domain map/unmap
Hi,
Quoted 20 lines · Christoph Hellwig
Quoted 19 lines · Lu Baolu
Hi again,
Quoted 12 lines · Lu Baolu
Hi,
Quoted 9 lines · Christoph Hellwig
Quoted 5 lines · Lu Baolu
This adds the APIs for bounce buffer specified domain
map()andunmap(). The start and end partial pages will be mapped with bounce buffered pages instead. This will enhance the security of DMA buffer by isolating the DMA attacks from malicious devices.Please reuse the swiotlb code instead of reinventing it.
Just looked into the code again. At least we could reuse below functions:
Quoted 9 lines
Quoted 5 lines
swiotlb_tbl_map_single()swiotlb_tbl_unmap_single()swiotlb_tbl_sync_single()Anything else?
Yes, that is probably about the level you want to reuse, given that the next higher layer already has hooks into the direct mapping code.
Okay. Thank you!
I will try to make this happen in v2.
Best regards,
Lu Baolu
Re: [PATCH v1 4/9] iommu/vt-d: Add bounce buffer API for domain map/unmap
In reply to Christoph HellwigHi Christoph,
Quoted 20 lines · Christoph Hellwig
Quoted 19 lines · Lu Baolu
Hi again,
Quoted 12 lines · Lu Baolu
Hi,
Quoted 9 lines · Christoph Hellwig
Quoted 5 lines · Lu Baolu
This adds the APIs for bounce buffer specified domain
map()andunmap(). The start and end partial pages will be mapped with bounce buffered pages instead. This will enhance the security of DMA buffer by isolating the DMA attacks from malicious devices.Please reuse the swiotlb code instead of reinventing it.
Just looked into the code again. At least we could reuse below functions:
Quoted 9 lines
Quoted 5 lines
swiotlb_tbl_map_single()swiotlb_tbl_unmap_single()swiotlb_tbl_sync_single()Anything else?
Yes, that is probably about the level you want to reuse, given that the next higher layer already has hooks into the direct mapping code.
I am trying to change my code to reuse swiotlb. But I found that swiotlb might not be suitable for my case.
Below is what I got with swiotlb_map():
phy_addr size tlb_addr
--------------------------------
0x167eec330 0x8 0x85dc6000 0x167eef5c0 0x40 0x85dc6800 0x167eec330 0x8 0x85dc7000 0x167eef5c0 0x40 0x85dc7800
But what I expected to get is:
phy_addr size tlb_addr
--------------------------------
0x167eec330 0x8 0xAAAAA330 0x167eef5c0 0x40 0xBBBBB5c0 0x167eec330 0x8 0xCCCCC330 0x167eef5c0 0x40 0xDDDDD5c0
, where 0xXXXXXX000 is the physical address of a bounced page.
Basically, I want a bounce page to replace a leaf page in the vt-d page table, which maps a buffer with size less than a PAGE_SIZE.
Best regards,
Lu Baolu
Robin Murphy
reviewerRe: [PATCH v1 4/9] iommu/vt-d: Add bounce buffer API for domain map/unmap
Quoted 54 lines · Lu Baolu
Hi Christoph,
Quoted 20 lines · Christoph Hellwig
Quoted 19 lines · Lu Baolu
Hi again,
Quoted 12 lines · Lu Baolu
Hi,
Quoted 9 lines · Christoph Hellwig
Quoted 5 lines · Lu Baolu
This adds the APIs for bounce buffer specified domain
map()andunmap(). The start and end partial pages will be mapped with bounce buffered pages instead. This will enhance the security of DMA buffer by isolating the DMA attacks from malicious devices.Please reuse the swiotlb code instead of reinventing it.
Just looked into the code again. At least we could reuse below functions:
Quoted 9 lines
Quoted 5 lines
swiotlb_tbl_map_single()swiotlb_tbl_unmap_single()swiotlb_tbl_sync_single()Anything else?
Yes, that is probably about the level you want to reuse, given that the next higher layer already has hooks into the direct mapping code.
I am trying to change my code to reuse swiotlb. But I found that swiotlb might not be suitable for my case.
Below is what I got with
swiotlb_map():phy_addr size
tlb_addr
--------------------------------
0x167eec330 0x80x85dc6000
0x167eef5c0 0x400x85dc6800
0x167eec330 0x80x85dc7000
0x167eef5c0 0x400x85dc7800But what I expected to get is:
phy_addr size
tlb_addr
--------------------------------
0x167eec330 0x80xAAAAA330
0x167eef5c0 0x400xBBBBB5c0
0x167eec330 0x80xCCCCC330
0x167eef5c0 0x400xDDDDD5c0
Quoted 4 lines
, where 0xXXXXXX000 is the physical address of a bounced page.
Basically, I want a bounce page to replace a leaf page in the vt-d page table, which maps a buffer with size less than a
PAGE_SIZE.
I'd imagine the thing to do would be to factor out the slot allocation in swiotlb_tbl_map_single() so that an IOMMU page pool/allocator can be hooked in as an alternative.
However we implement it, though, this should absolutely be a common IOMMU thing that all relevant DMA backends can opt into, and not specific to VT-d. I mean, it's already more or less the same concept as the PowerPC secure VM thing.
Robin.
PATCH 5/9 iommu/vt-d: Add bounce buffer API for dma sync
In reply to Lu BaoluThis adds the APIs for bounce buffer specified dma sync ops.
--- drivers/iommu/intel-pgtable.c | 112 ++++++++++++++++++++++++++++++++++ include/linux/intel-iommu.h | 6 ++ 2 files changed, 118 insertions(+)
drivers/iommu/intel-pgtable.c +112 -0
@@ -331,6 +331,100 @@ static const struct addr_walk walk_bounce_unmap = { .high = bounce_unmap_high, }; +static int +bounce_sync_iova_pfn (struct dmar_domain *domain,dma_addr_t addr, +size_t size,struct bounce_param *param, +enum dma_data_direction dir) +{ +struct bounce_cookie *cookie; +unsigned long flags; + +spin_lock_irqsave (&bounce_lock, flags); + cookie =idr_find (&domain->bounce_idr, addr >>PAGE_SHIFT ); +spin_unlock_irqrestore (&bounce_lock, flags); +if (!cookie) +return 0 ; + +return bounce_sync (cookie->original_phys, cookie->bounce_phys, + size, dir); +} + +static int +bounce_sync_for_device_low (struct dmar_domain *domain,dma_addr_t addr, +phys_addr_t paddr,size_t size, +struct bounce_param *param) +{ +if (param->dir ==DMA_BIDIRECTIONAL || param->dir ==DMA_TO_DEVICE ) +return bounce_sync_iova_pfn (domain, addr, size, + param,DMA_TO_DEVICE ); + +return 0 ; +} + +static int +bounce_sync_for_device_middle (struct dmar_domain *domain,dma_addr_t addr, +phys_addr_t paddr,size_t size, +struct bounce_param *param) +{ +return 0 ; +} + +static int +bounce_sync_for_device_high (struct dmar_domain *domain,dma_addr_t addr, +phys_addr_t paddr,size_t size, +struct bounce_param *param) +{ +if (param->dir ==DMA_BIDIRECTIONAL || param->dir ==DMA_TO_DEVICE ) +return bounce_sync_iova_pfn (domain, addr, size, + param,DMA_TO_DEVICE ); + +return 0 ; +} + +const struct addr_walk walk_bounce_sync_for_device = { + .low = bounce_sync_for_device_low, + .middle = bounce_sync_for_device_middle, + .high = bounce_sync_for_device_high, +}; + +static int +bounce_sync_for_cpu_low (struct dmar_domain *domain,dma_addr_t addr, +phys_addr_t paddr,size_t size, +struct bounce_param *param) +{ +if (param->dir ==DMA_BIDIRECTIONAL || param->dir ==DMA_FROM_DEVICE ) +return bounce_sync_iova_pfn (domain, addr, size, + param,DMA_FROM_DEVICE ); + +return 0 ; +} + +static int +bounce_sync_for_cpu_middle (struct dmar_domain *domain,dma_addr_t addr, +phys_addr_t paddr,size_t size, +struct bounce_param *param) +{ +return 0 ; +} + +static int +bounce_sync_for_cpu_high (struct dmar_domain *domain,dma_addr_t addr, +phys_addr_t paddr,size_t size, +struct bounce_param *param) +{ +if (param->dir ==DMA_BIDIRECTIONAL || param->dir ==DMA_FROM_DEVICE ) +return bounce_sync_iova_pfn (domain, addr, size, + param,DMA_FROM_DEVICE ); + +return 0 ; +} + +const struct addr_walk walk_bounce_sync_for_cpu = { + .low = bounce_sync_for_cpu_low, + .middle = bounce_sync_for_cpu_middle, + .high = bounce_sync_for_cpu_high, +}; + static int domain_walk_addr_range(const struct addr_walk *walk, struct dmar_domain *domain,
@@ -404,3 +498,21 @@ domain_bounce_unmap(struct dmar_domain *domain, dma_addr_t addr,return domain_walk_addr_range (&walk_bounce_unmap, domain, addr, paddr, size, param); } + +int +domain_bounce_sync_for_device (struct dmar_domain *domain,dma_addr_t addr, +phys_addr_t paddr,size_t size, +struct bounce_param *param) +{ +return domain_walk_addr_range (&walk_bounce_sync_for_device, domain, + addr, paddr, size, param); +} + +int +domain_bounce_sync_for_cpu (struct dmar_domain *domain,dma_addr_t addr, +phys_addr_t paddr,size_t size, +struct bounce_param *param) +{ +return domain_walk_addr_range (&walk_bounce_sync_for_cpu, domain, + addr, paddr, size, param); +}
include/linux/intel-iommu.h +6 -0
@@ -681,6 +681,12 @@ int domain_bounce_map(struct dmar_domain *domain, dma_addr_t addr,int domain_bounce_unmap (struct dmar_domain *domain,dma_addr_t addr,phys_addr_t paddr,size_t size,struct bounce_param *param); +int domain_bounce_sync_for_device (struct dmar_domain *domain,dma_addr_t addr, +phys_addr_t paddr,size_t size, +struct bounce_param *param); +int domain_bounce_sync_for_cpu (struct dmar_domain *domain,dma_addr_t addr, +phys_addr_t paddr,size_t size, +struct bounce_param *param);#ifdef CONFIG_INTEL_IOMMU_SVM int intel_svm_init (struct intel_iommu *iommu);extern int intel_svm_enable_prq (struct intel_iommu *iommu);
Signature · 1 line
2.17.1
PATCH 6/9 iommu/vt-d: Check whether device requires bounce buffer
In reply to Lu BaoluThis adds a helper to check whether a device needs to use bounce buffer. It also provides a boot time option to disable the bounce buffer. Users can use this to prevent the iommu driver from using the bounce buffer for performance gain.
--- Documentation/admin-guide/kernel-parameters.txt | 5 +++++ drivers/iommu/intel-iommu.c | 17 +++++++++++++++++ 2 files changed, 22 insertions(+)
Documentation/admin-guide/kernel-parameters.txt +5 -0
@@ -1726,6 +1726,11 @@ Note that using this option lowers the security provided by tboot because it makes the system vulnerable to DMA attacks. + nobounce [Default off] + Do not use the bounce buffer for untrusted devices like + the Thunderbolt devices. This will treat the untrusted + devices as the trusted ones, hence might expose security + risks of DMA attacks. intel_idle.max_cstate= [KNL,HW,ACPI,X86] 0 disables intel_idle and fall back on acpi_idle.
drivers/iommu/intel-iommu.c +17 -0
@@ -366,6 +366,7 @@ static int intel_iommu_strict;static int intel_iommu_superpage =1 ;static int intel_iommu_sm;static int iommu_identity_mapping; +static int intel_no_bounce;#define IDENTMAP_ALL 1#define IDENTMAP_GFX 2
@@ -382,6 +383,19 @@ EXPORT_SYMBOL_GPL(intel_iommu_gfx_mapped);static DEFINE_SPINLOCK (device_domain_lock);static LIST_HEAD (device_domain_list); +static inline bool device_needs_bounce (struct device *dev) +{ +struct pci_dev *pdev =NULL ; + +if (intel_no_bounce) +return false; + +if (dev_is_pci (dev)) + pdev =to_pci_dev (dev); + +return pdev ? pdev->untrusted : false; +} +/* * Iterate over elements in device_domain_list and call the specified * callback @fn against each element.
@@ -464,6 +478,9 @@ static int __init intel_iommu_setup(char *str)printk (KERN_INFO "Intel-IOMMU: not forcing on after tboot. This could expose security risk for tboot\n" ); intel_iommu_tboot_noforce =1 ; + }else if (!strncmp (str,"nobounce" ,8 )) { +pr_info ("Intel-IOMMU: No bounce buffer. This could expose security risks of DMA attacks\n" ); + intel_no_bounce =1 ; } str +=strcspn (str,"," );
Signature · 1 line
2.17.1
PATCH 7/9 iommu/vt-d: Add dma sync ops for untrusted devices
In reply to Lu BaoluThis adds the dma sync ops for dma buffers used by any untrusted device. We need to sync such buffers because they might have been mapped with bounce pages.
--- drivers/iommu/intel-iommu.c | 154 +++++++++++++++++++++++++++++++++--- 1 file changed, 145 insertions(+), 9 deletions(-)
drivers/iommu/intel-iommu.c +145 -9
@@ -3940,16 +3940,152 @@ static int intel_map_sg(struct device *dev, struct scatterlist *sglist, int nelereturn nelems; } +static void +sync_dma_for_device (struct device *dev,dma_addr_t dev_addr,size_t size, +enum dma_data_direction dir) +{ +struct dmar_domain *domain; +struct bounce_param param; + + domain =find_domain (dev); +if (WARN_ON (!domain)) +return ; + +memset (¶m,0 ,sizeof (param)); + param.dir = dir; +if (dir ==DMA_BIDIRECTIONAL || dir ==DMA_TO_DEVICE ) +domain_bounce_sync_for_device (domain, dev_addr, +0 , size, ¶m); +} + +static void +sync_dma_for_cpu (struct device *dev,dma_addr_t dev_addr,size_t size, +enum dma_data_direction dir) +{ +struct dmar_domain *domain; +struct bounce_param param; + + domain =find_domain (dev); +if (WARN_ON (!domain)) +return ; + +memset (¶m,0 ,sizeof (param)); + param.dir = dir; +if (dir ==DMA_BIDIRECTIONAL || dir ==DMA_FROM_DEVICE ) +domain_bounce_sync_for_cpu (domain, dev_addr, +0 , size, ¶m); +} + +static void +intel_sync_single_for_cpu (struct device *dev,dma_addr_t addr, +size_t size,enum dma_data_direction dir) +{ +struct dmar_domain *domain; + +if (WARN_ON (dir ==DMA_NONE )) +return ; + +if (!device_needs_bounce (dev)) +return ; + +if (iommu_no_mapping (dev)) +return ; + + domain =get_valid_domain_for_dev (dev); +if (!domain) +return ; + +sync_dma_for_cpu (dev, addr, size, dir); +} + +static void +intel_sync_single_for_device (struct device *dev,dma_addr_t addr, +size_t size,enum dma_data_direction dir) +{ +struct dmar_domain *domain; + +if (WARN_ON (dir ==DMA_NONE )) +return ; + +if (!device_needs_bounce (dev)) +return ; + +if (iommu_no_mapping (dev)) +return ; + + domain =get_valid_domain_for_dev (dev); +if (!domain) +return ; + +sync_dma_for_device (dev, addr, size, dir); +} + +static void +intel_sync_sg_for_cpu (struct device *dev,struct scatterlist *sglist, +int nelems,enum dma_data_direction dir) +{ +struct dmar_domain *domain; +struct scatterlist *sg; +int i; + +if (WARN_ON (dir ==DMA_NONE )) +return ; + +if (!device_needs_bounce (dev)) +return ; + +if (iommu_no_mapping (dev)) +return ; + + domain =get_valid_domain_for_dev (dev); +if (!domain) +return ; + +for_each_sg (sglist, sg, nelems, i) +sync_dma_for_cpu (dev,sg_dma_address (sg), +sg_dma_len (sg), dir); +} + +static void +intel_sync_sg_for_device (struct device *dev,struct scatterlist *sglist, +int nelems,enum dma_data_direction dir) +{ +struct dmar_domain *domain; +struct scatterlist *sg; +int i; + +if (WARN_ON (dir ==DMA_NONE )) +return ; + +if (!device_needs_bounce (dev)) +return ; + +if (iommu_no_mapping (dev)) +return ; + + domain =get_valid_domain_for_dev (dev); +if (!domain) +return ; + +for_each_sg (sglist, sg, nelems, i) +sync_dma_for_device (dev,sg_dma_address (sg), +sg_dma_len (sg), dir); +} +static const struct dma_map_ops intel_dma_ops = { - .alloc = intel_alloc_coherent, - .free = intel_free_coherent, - .map_sg = intel_map_sg, - .unmap_sg = intel_unmap_sg, - .map_page = intel_map_page, - .unmap_page = intel_unmap_page, - .map_resource = intel_map_resource, - .unmap_resource = intel_unmap_page, - .dma_supported = dma_direct_supported, + .alloc = intel_alloc_coherent, + .free = intel_free_coherent, + .map_sg = intel_map_sg, + .unmap_sg = intel_unmap_sg, + .map_page = intel_map_page, + .unmap_page = intel_unmap_page, + .sync_single_for_cpu = intel_sync_single_for_cpu, + .sync_single_for_device = intel_sync_single_for_device, + .sync_sg_for_cpu = intel_sync_sg_for_cpu, + .sync_sg_for_device = intel_sync_sg_for_device, + .map_resource = intel_map_resource, + .unmap_resource = intel_unmap_page, + .dma_supported = dma_direct_supported, };static inline int iommu_domain_cache_init (void )
Signature · 1 line
2.17.1
PATCH 8/9 iommu/vt-d: Flush IOTLB for untrusted device in time
In reply to Lu BaoluBy default, for performance consideration, Intel IOMMU driver won't flush IOTLB immediately after a buffer is unmapped. It schedules a thread and flushes IOTLB in a batched mode. This isn't suitable for untrusted device since it still can access the memory even if it isn't supposed to do so.
--- drivers/iommu/intel-iommu.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
drivers/iommu/intel-iommu.c +5 -1
@@ -3749,10 +3749,14 @@ static void intel_unmap(struct device *dev, dma_addr_t dev_addr, size_t size)unsigned long iova_pfn;struct intel_iommu *iommu;struct page *freelist; +struct pci_dev *pdev =NULL ;if (iommu_no_mapping (dev))return ; +if (dev_is_pci (dev)) + pdev =to_pci_dev (dev); + domain =find_domain (dev);BUG_ON (!domain);
@@ -3768,7 +3772,7 @@ static void intel_unmap(struct device *dev, dma_addr_t dev_addr, size_t size) freelist =domain_unmap (domain, start_pfn, last_pfn); -if (intel_iommu_strict) { +if (intel_iommu_strict || (pdev && pdev->untrusted)) {iommu_flush_iotlb_psi (iommu, domain, start_pfn, nrpages, !freelist,0 );/* free iova */
Signature · 1 line
2.17.1
PATCH 9/9 iommu/vt-d: Use bounce buffer for untrusted devices
In reply to Lu BaoluThe Intel VT-d hardware uses paging for DMA remapping. The minimum mapped window is a page size. The device drivers may map buffers not filling the whole IOMMU window. This allows the device to access to possibly unrelated memory and a malicious device could exploit this to perform DMA attacks. To address this, the Intel IOMMU driver will use bounce pages for those buffers which don't fill a whole IOMMU page.
--- drivers/iommu/intel-iommu.c | 151 +++++++++++++++++++++++++++--------- 1 file changed, 114 insertions(+), 37 deletions(-)
drivers/iommu/intel-iommu.c +114 -37
@@ -52,6 +52,7 @@#include <asm/irq_remapping.h> #include <asm/cacheflush.h> #include <asm/iommu.h> +#include <trace/events/intel_iommu.h> #include "irq_remapping.h" #include "intel-pasid.h"
@@ -3670,12 +3671,14 @@ static dma_addr_t __intel_map_single(struct device *dev, phys_addr_t paddr, size_t size, int dir, u64 dma_mask) {struct dmar_domain *domain; -phys_addr_t start_paddr; +dma_addr_t start_dma;unsigned long iova_pfn;int prot =0 ;int ret;struct intel_iommu *iommu;unsigned long paddr_pfn = paddr >>PAGE_SHIFT ; +unsigned long nrpages; +struct bounce_param param;BUG_ON (dir ==DMA_NONE );
@@ -3687,9 +3690,10 @@ static dma_addr_t __intel_map_single(struct device *dev, phys_addr_t paddr,return DMA_MAPPING_ERROR ; iommu =domain_get_iommu (domain); - size =aligned_nrpages (paddr, size); + nrpages =aligned_nrpages (paddr, size); - iova_pfn =intel_alloc_iova (dev, domain,dma_to_mm_pfn (size), dma_mask); + iova_pfn =intel_alloc_iova (dev, domain, +dma_to_mm_pfn (nrpages), dma_mask);if (!iova_pfn) goto error;
@@ -3702,24 +3706,36 @@ static dma_addr_t __intel_map_single(struct device *dev, phys_addr_t paddr, prot |=DMA_PTE_READ ;if (dir ==DMA_FROM_DEVICE || dir ==DMA_BIDIRECTIONAL ) prot |=DMA_PTE_WRITE ; + + start_dma = (dma_addr_t )iova_pfn <<PAGE_SHIFT ; + start_dma +=offset_in_page (paddr); +/* * paddr - (paddr + size) might be partial page, we should map the whole * page. Note: if two part of one page are separately mapped, we * might have two guest_addr mapping to the same host paddr, but this * is not a big problem */ - ret =domain_pfn_mapping (domain,mm_to_dma_pfn (iova_pfn), -mm_to_dma_pfn (paddr_pfn), size, prot); +memset (¶m,0 ,sizeof (param)); + param.prot = prot; + param.dir = dir; +if (device_needs_bounce (dev)) { + ret =domain_bounce_map (domain, start_dma, paddr, size, ¶m); +if (!ret) +trace_bounce_map_single (dev, start_dma, paddr, size); + }else { + ret =domain_pfn_mapping (domain,mm_to_dma_pfn (iova_pfn), +mm_to_dma_pfn (paddr_pfn), + nrpages, prot); + }if (ret) goto error; - start_paddr = (phys_addr_t )iova_pfn <<PAGE_SHIFT ; - start_paddr += paddr & ~PAGE_MASK ; -return start_paddr; - +return start_dma; error:if (iova_pfn) -free_iova_fast (&domain->iovad, iova_pfn,dma_to_mm_pfn (size)); +free_iova_fast (&domain->iovad, iova_pfn, +dma_to_mm_pfn (nrpages));dev_err (dev,"Device request: %zx@%llx dir %d --- failed\n" , size, (unsigned long long )paddr, dir);return DMA_MAPPING_ERROR ;
@@ -3741,36 +3757,81 @@ static dma_addr_t intel_map_resource(struct device *dev, phys_addr_t phys_addr,return __intel_map_single (dev, phys_addr, size, dir, *dev->dma_mask); } -static void intel_unmap (struct device *dev,dma_addr_t dev_addr,size_t size) +static void +intel_unmap (struct device *dev,dma_addr_t dev_addr,size_t size, +struct scatterlist *sglist,int nelems, +enum dma_data_direction dir,unsigned long attrs) {struct dmar_domain *domain;unsigned long start_pfn, last_pfn; -unsigned long nrpages; +unsigned long nrpages =0 ;unsigned long iova_pfn;struct intel_iommu *iommu; -struct page *freelist; +struct page *freelist =NULL ;struct pci_dev *pdev =NULL ; - -if (iommu_no_mapping (dev)) -return ; +struct bounce_param param;if (dev_is_pci (dev)) pdev =to_pci_dev (dev); +if (iommu_no_mapping (dev)) +return ; + domain =find_domain (dev);BUG_ON (!domain); iommu =domain_get_iommu (domain); - iova_pfn =IOVA_PFN (dev_addr); - - nrpages =aligned_nrpages (dev_addr, size); - start_pfn =mm_to_dma_pfn (iova_pfn); - last_pfn = start_pfn + nrpages -1 ; - -dev_dbg (dev,"Device unmapping: pfn %lx-%lx\n" , start_pfn, last_pfn); +if (sglist) { +struct scatterlist *sg; +int i; - freelist =domain_unmap (domain, start_pfn, last_pfn); + dev_addr =sg_dma_address (sglist) &PAGE_MASK ; + iova_pfn =IOVA_PFN (dev_addr); +for_each_sg (sglist, sg, nelems, i) { + nrpages +=aligned_nrpages (sg_dma_address (sg), +sg_dma_len (sg)); + } + start_pfn =mm_to_dma_pfn (iova_pfn); + last_pfn = start_pfn + nrpages -1 ; + +if (device_needs_bounce (dev)) +for_each_sg (sglist, sg, nelems, i) { +struct page *tmp; + + tmp =NULL ; +memset (¶m,0 ,sizeof (param)); + param.freelist = &tmp; + param.dir = dir; +domain_bounce_unmap (domain,sg_dma_address (sg), +sg_phys (sg), sg->length, + ¶m); +if (tmp) { + tmp->freelist = freelist; + freelist = tmp; + } +trace_bounce_unmap_sg (dev, i, nelems, +sg_dma_address (sg), +sg_phys (sg), sg->length); + } +else + freelist =domain_unmap (domain, start_pfn, last_pfn); + }else { + iova_pfn =IOVA_PFN (dev_addr); + nrpages =aligned_nrpages (dev_addr, size); + start_pfn =mm_to_dma_pfn (iova_pfn); + last_pfn = start_pfn + nrpages -1 ; + +if (device_needs_bounce (dev)) { +memset (¶m,0 ,sizeof (param)); + param.freelist = &freelist; + param.dir = dir; +domain_bounce_unmap (domain, dev_addr,0 , size, ¶m); +trace_bounce_unmap_single (dev, dev_addr, size); + }else { + freelist =domain_unmap (domain, start_pfn, last_pfn); + } + }if (intel_iommu_strict || (pdev && pdev->untrusted)) { iommu_flush_iotlb_psi(iommu, domain, start_pfn,
@@ -3792,7 +3853,7 @@ static void intel_unmap_page(struct device *dev, dma_addr_t dev_addr, size_t size, enum dma_data_direction dir, unsigned long attrs) { -intel_unmap (dev ,dev_addr ,size ); + intel_unmap(dev, dev_addr, size, NULL, 0, dir, attrs); } static void *intel_alloc_coherent(struct device *dev, size_t size,
@@ -3852,7 +3913,7 @@ static void intel_free_coherent(struct device *dev, size_t size, void *vaddr, size =PAGE_ALIGN (size); order =get_order (size); -intel_unmap (dev, dma_handle, size); +intel_unmap (dev, dma_handle, size,NULL ,0 ,0 , attrs);if (!dma_release_from_contiguous (dev, page, size >>PAGE_SHIFT ))__free_pages (page, order); }
@@ -3861,16 +3922,7 @@ static void intel_unmap_sg(struct device *dev, struct scatterlist *sglist, int nelems, enum dma_data_direction dir, unsigned long attrs) { - dma_addr_t startaddr = sg_dma_address(sglist) & PAGE_MASK; -unsigned long nrpages =0 ; -struct scatterlist *sg; -int i; - -for_each_sg (sglist, sg, nelems, i) { - nrpages +=aligned_nrpages (sg_dma_address (sg),sg_dma_len (sg)); - } - -intel_unmap (dev, startaddr, nrpages <<VTD_PAGE_SHIFT ); + intel_unmap(dev, 0, 0, sglist, nelems, dir, attrs); } static int intel_nontranslate_map_sg(struct device *hddev,
@@ -3932,7 +3984,32 @@ static int intel_map_sg(struct device *dev, struct scatterlist *sglist, int nele start_vpfn =mm_to_dma_pfn (iova_pfn); - ret =domain_sg_mapping (domain, start_vpfn, sglist, size, prot); +if (device_needs_bounce (dev)) { +for_each_sg (sglist, sg, nelems, i) { +unsigned int pgoff =offset_in_page (sg->offset); +struct bounce_param param; +dma_addr_t addr; + + addr = ((dma_addr_t )iova_pfn <<PAGE_SHIFT ) + pgoff; +memset (¶m,0 ,sizeof (param)); + param.prot = prot; + param.dir = dir; + ret =domain_bounce_map (domain, addr,sg_phys (sg), + sg->length, ¶m); +if (ret) +break ; + +trace_bounce_map_sg (dev, i, nelems, addr, +sg_phys (sg), sg->length); + + sg->dma_address = addr; + sg->dma_length = sg->length; + iova_pfn +=aligned_nrpages (sg->offset, sg->length); + } + }else { + ret =domain_sg_mapping (domain, start_vpfn, sglist, size, prot); + } +if (unlikely (ret)) { dma_pte_free_pagetable(domain, start_vpfn, start_vpfn + size - 1,
Signature · 1 line
2.17.1
Should be titled as "iommu/vt-d: Bounce buffer for untrusted devices". Sorry for the inconvenience.
Quoted 75 lines · Lu Baolu
An external PCI device is a PCI peripheral device connected to the system through an external bus, such as Thunderbolt. What makes it different is that it can't be trusted to the same degree as the devices build into the system. Generally, a trusted PCIe device will DMA into the designated buffers and not overrun or otherwise write outside the specified bounds. But it's different for an external device. The minimum IOMMU mapping granularity is one page (4k), so for DMA transfers smaller than that a malicious PCIe device can access the whole page of memory even if it does not belong to the driver in question. This opens a possibility for DMA attack. For more information about DMA attacks imposed by an untrusted PCI/PCIe device, please refer to [2].
This implements bounce buffer for the untrusted external devices. The transfers should be limited in isolated pages so the IOMMU window does not cover memory outside of what the driver expects. Full pages within a buffer could be directly mapped in IOMMU page table, but for partial pages we use bounce page instead.
In addition, the IOMMU mappings cached in the IOTLB for untrusted devices should be invalidated immediately after the unmap operation. Otherwise, the IOMMU window is left open to attacks.
The implementation of bounce buffers for untrusted devices will cause a little performance overhead, but we didn't see any user experience problems. The users could use the kernel parameter of "intel_iommu=nobounce" to remove the performance overhead if they trust their devices enough.
The Thunderbolt vulnerabiltiies is public and has a nice name as Thunderclap nowadays. Please refer to [1] [3] for more information. This patch series aims to mitigate the concerns.
The bounce buffer idea:
Based-on-idea-by Mika Westerberg <[email protected]>Based-on-idea-by Ashok Raj <[email protected]>Based-on-idea-by Alan Cox <[email protected]>The patch series has been tested by:
Tested-by Xu Pengfei <[email protected]>Tested-by Mika Westerberg <[email protected]>[1] https://thunderclap.io/
[2] https://thunderclap.io/thunderclap-paper-ndss2019.pdf
[3] https://christian.kellner.me/2019/02/27/thunderclap-and-linux/Lu Baolu (9):
iommu/vt-d: Add trace events for domain map/unmap iommu/vt-d: Add helpers for domain mapping/unmapping iommu/vt-d: Add address walk helper iommu/vt-d: Add bounce buffer API for domain map/unmap iommu/vt-d: Add bounce buffer API for dma sync iommu/vt-d: Check whether device requires bounce buffer iommu/vt-d: Add dma sync ops for untrusted devices iommu/vt-d: Flush IOTLB for untrusted device in time iommu/vt-d: Use bounce buffer for untrusted devices.../admin-guide/kernel-parameters.txt | 5 + drivers/iommu/Makefile | 1 + drivers/iommu/intel-iommu.c | 360 ++++++++++-- drivers/iommu/intel-pgtable.c | 518 ++++++++++++++++++ drivers/iommu/intel-trace.c | 14 + include/linux/intel-iommu.h | 24 + include/trace/events/intel_iommu.h | 132 +++++ 7 files changed, 1010 insertions(+), 44 deletions(-)create mode 100644
drivers/iommu/intel-pgtable.ccreate mode 100644drivers/iommu/intel-trace.ccreate mode 100644include/trace/events/intel_iommu.h