Restricted DMA
Subsystems (12)
v8 cover Restricted DMA
This series implements mitigations for lack of DMA access control on systems without an IOMMU, which could result in the DMA accessing the system memory at unexpected times and/or unexpected addresses, possibly leading to data leakage or corruption.
For example, we plan to use the PCI-e bus for Wi-Fi and that PCI-e bus is not behind an IOMMU. As PCI-e, by design, gives the device full access to system memory, a vulnerability in the Wi-Fi firmware could easily escalate to a full system exploit (remote wifi exploits: [1a], [1b] that shows a full chain of exploits; [2], [3]).
To mitigate the security concerns, we introduce restricted DMA. Restricted DMA utilizes the existing swiotlb to bounce streaming DMA in and out of a specially allocated region and does memory allocation from the same region. The feature on its own provides a basic level of protection against the DMA overwriting buffer contents at unexpected times. However, to protect against general data leakage and system memory corruption, the system needs to provide a way to restrict the DMA to a predefined memory region (this is usually done at firmware level, e.g. MPU in ATF on some ARM platforms [4]).
[1a] https://googleprojectzero.blogspot.com/2017/04/over-air-exploiting-broadcoms-wi-fi_4.html
[1b] https://googleprojectzero.blogspot.com/2017/04/over-air-exploiting-broadcoms-wi-fi_11.html
[2] https://blade.tencent.com/en/advisories/qualpwn/
[3] https://www.bleepingcomputer.com/news/security/vulnerabilities-found-in-highly-popular-firmware-for-wifi-chips/
[4] https://github.com/ARM-software/arm-trusted-firmware/blob/master/plat/mediatek/mt8183/drivers/emi_mpu/emi_mpu.c#L132
v8:
- Fix reserved-memory.txt and add the reg property in example.
- Fix sizeof for of_property_count_elems_of_size in drivers/of/address.c#of_dma_set_restricted_buffer.
- Apply Will's suggestion to try the OF node having DMA configuration in drivers/of/address.c#of_dma_set_restricted_buffer.
- Fix typo in the comment of drivers/of/address.c#of_dma_set_restricted_buffer.
- Add error message for PageHighMem in kernel/dma/swiotlb.c#rmem_swiotlb_device_init and move it to rmem_swiotlb_setup.
- Fix the message string in rmem_swiotlb_setup.
v7:
Fix debugfs, PageHighMem and comment style in rmem_swiotlb_device_init
https://lore.kernel.org/patchwork/cover/1431031/
v6:
Address the comments in v5
https://lore.kernel.org/patchwork/cover/1423201/
v5:
Rebase on latest linux-next
https://lore.kernel.org/patchwork/cover/1416899/
v4:
- Fix spinlock bad magic
- Use rmem->name for debugfs entry
- Address the comments in v3
https://lore.kernel.org/patchwork/cover/1378113/
v3:
Using only one reserved memory region for both streaming DMA and memory allocation.
https://lore.kernel.org/patchwork/cover/1360992/
v2:
Building on top of swiotlb.
https://lore.kernel.org/patchwork/cover/1280705/
v1:
Using dma_map_ops.
https://lore.kernel.org/patchwork/cover/1271660/
Claire Chang (15):
swiotlb: Refactor swiotlb init functions
swiotlb: Refactor swiotlb_create_debugfs
swiotlb: Add DMA_RESTRICTED_POOL
swiotlb: Add restricted DMA pool initialization
swiotlb: Add a new get_io_tlb_mem getter
swiotlb: Update is_swiotlb_buffer to add a struct device argument
swiotlb: Update is_swiotlb_active to add a struct device argument
swiotlb: Bounce data from/to restricted DMA pool if available
swiotlb: Move alloc_size to find_slots
swiotlb: Refactor swiotlb_tbl_unmap_single
dma-direct: Add a new wrapper __dma_direct_free_pages()
swiotlb: Add restricted DMA alloc/free support.
dma-direct: Allocate memory from restricted DMA pool if available
dt-bindings: of: Add restricted DMA pool
of: Add plumbing for restricted DMA pool
.../reserved-memory/reserved-memory.txt | 36 ++- drivers/gpu/drm/i915/gem/i915_gem_internal.c | 2 +- drivers/gpu/drm/nouveau/nouveau_ttm.c | 2 +- drivers/iommu/dma-iommu.c | 12 +- drivers/of/address.c | 25 ++ drivers/of/device.c | 3 + drivers/of/of_private.h | 5 + drivers/pci/xen-pcifront.c | 2 +- drivers/xen/swiotlb-xen.c | 2 +- include/linux/device.h | 4 + include/linux/swiotlb.h | 41 ++- kernel/dma/Kconfig | 14 + kernel/dma/direct.c | 63 +++-- kernel/dma/direct.h | 9 +- kernel/dma/swiotlb.c | 242 +++++++++++++----- 15 files changed, 362 insertions(+), 100 deletions(-)
Signature · 1 line
2.31.1.818.g46aad6cb9e-goog
PATCH v8 1/15 swiotlb: Refactor swiotlb init functions
Add a new function, swiotlb_init_io_tlb_mem, for the io_tlb_mem struct initialization to make the code reusable.
Note that we now also call set_memory_decrypted in swiotlb_init_with_tbl.
--- kernel/dma/swiotlb.c | 51 ++++++++++++++++++++++---------------------- 1 file changed, 25 insertions(+), 26 deletions(-)
kernel/dma/swiotlb.c +25 -26
@@ -168,9 +168,30 @@ void __init swiotlb_update_mem_attributes(void)memset (vaddr,0 , bytes); } -int __init swiotlb_init_with_tbl (char *tlb,unsigned long nslabs,int verbose) +static void swiotlb_init_io_tlb_mem (struct io_tlb_mem *mem,phys_addr_t start, +unsigned long nslabs,bool late_alloc) { +void *vaddr =phys_to_virt (start);unsigned long bytes = nslabs <<IO_TLB_SHIFT , i; + + mem->nslabs = nslabs; + mem->start = start; + mem->end = mem->start + bytes; + mem->index =0 ; + mem->late_alloc = late_alloc; +spin_lock_init (&mem->lock); +for (i =0 ; i < mem->nslabs; i++) { + mem->slots[i].list =IO_TLB_SEGSIZE -io_tlb_offset (i); + mem->slots[i].orig_addr =INVALID_PHYS_ADDR ; + mem->slots[i].alloc_size =0 ; + } + +set_memory_decrypted ((unsigned long )vaddr, bytes >>PAGE_SHIFT ); +memset (vaddr,0 , bytes); +} + +int __init swiotlb_init_with_tbl (char *tlb,unsigned long nslabs,int verbose) +{struct io_tlb_mem *mem;size_t alloc_size;
@@ -186,16 +207,8 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)if (!mem)panic ("%s: Failed to allocate %zu bytes align=0x%lx\n" , __func__, alloc_size,PAGE_SIZE ); - mem->nslabs = nslabs; - mem->start =__pa (tlb); - mem->end = mem->start + bytes; - mem->index =0 ; -spin_lock_init (&mem->lock); -for (i =0 ; i < mem->nslabs; i++) { - mem->slots[i].list =IO_TLB_SEGSIZE -io_tlb_offset (i); - mem->slots[i].orig_addr =INVALID_PHYS_ADDR ; - mem->slots[i].alloc_size =0 ; - } + +swiotlb_init_io_tlb_mem (mem,__pa (tlb), nslabs, false); io_tlb_default_mem = mem;if (verbose)
@@ -282,7 +295,6 @@ swiotlb_late_init_with_default_size(size_t default_size)int swiotlb_late_init_with_tbl (char *tlb,unsigned long nslabs) { -unsigned long bytes = nslabs <<IO_TLB_SHIFT , i;struct io_tlb_mem *mem;if (swiotlb_force ==SWIOTLB_NO_FORCE )
@@ -297,20 +309,7 @@ swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)if (!mem)return -ENOMEM ; - mem->nslabs = nslabs; - mem->start =virt_to_phys (tlb); - mem->end = mem->start + bytes; - mem->index =0 ; - mem->late_alloc =1 ; -spin_lock_init (&mem->lock); -for (i =0 ; i < mem->nslabs; i++) { - mem->slots[i].list =IO_TLB_SEGSIZE -io_tlb_offset (i); - mem->slots[i].orig_addr =INVALID_PHYS_ADDR ; - mem->slots[i].alloc_size =0 ; - } - -set_memory_decrypted ((unsigned long )tlb, bytes >>PAGE_SHIFT ); -memset (tlb,0 , bytes); +swiotlb_init_io_tlb_mem (mem,virt_to_phys (tlb), nslabs, true); io_tlb_default_mem = mem;swiotlb_print_info ();
Signature · 1 line
2.31.1.818.g46aad6cb9e-goog
PATCH v8 2/15 swiotlb: Refactor swiotlb_create_debugfs
In reply to Claire ChangSplit the debugfs creation to make the code reusable for supporting different bounce buffer pools, e.g. restricted DMA pool.
--- kernel/dma/swiotlb.c | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-)
kernel/dma/swiotlb.c +19 -6
@@ -64,6 +64,7 @@enum swiotlb_force swiotlb_force;struct io_tlb_mem *io_tlb_default_mem; +static struct dentry *debugfs_dir;/* * Max segment that we can provide which (if pages are contingous) will
@@ -662,18 +663,30 @@ EXPORT_SYMBOL_GPL(is_swiotlb_active);#ifdef CONFIG_DEBUG_FS -static int __init swiotlb_create_debugfs (void ) +static void swiotlb_create_debugfs (struct io_tlb_mem *mem,const char *name) { -struct io_tlb_mem *mem = io_tlb_default_mem; -if (!mem) -return 0 ; - mem->debugfs =debugfs_create_dir ("swiotlb" ,NULL ); +return ; + + mem->debugfs =debugfs_create_dir (name, debugfs_dir);debugfs_create_ulong ("io_tlb_nslabs" ,0400 , mem->debugfs, &mem->nslabs);debugfs_create_ulong ("io_tlb_used" ,0400 , mem->debugfs, &mem->used); +} + +static int __init swiotlb_create_default_debugfs (void ) +{ +struct io_tlb_mem *mem = io_tlb_default_mem; + +if (mem) { +swiotlb_create_debugfs (mem,"swiotlb" ); + debugfs_dir = mem->debugfs; + }else { + debugfs_dir =debugfs_create_dir ("swiotlb" ,NULL ); + } +return 0 ; } -late_initcall (swiotlb_create_debugfs); +late_initcall (swiotlb_create_default_debugfs);#endif
Signature · 1 line
2.31.1.818.g46aad6cb9e-goog
PATCH v8 3/15 swiotlb: Add DMA_RESTRICTED_POOL
In reply to Claire ChangAdd a new kconfig symbol, DMA_RESTRICTED_POOL, for restricted DMA pool.
--- kernel/dma/Kconfig | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
kernel/dma/Kconfig +14 -0
@@ -80,6 +80,20 @@ config SWIOTLB bool select NEED_DMA_MAP_STATE +config DMA_RESTRICTED_POOL + bool "DMA Restricted Pool" + depends on OF && OF_RESERVED_MEM + select SWIOTLB + help + This enables support for restricted DMA pools which provide a level of + DMA memory protection on systems with limited hardware protection + capabilities, such as those lacking an IOMMU. + + For more information see + <Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt> + and <kernel/dma/swiotlb.c>. + If unsure, say "n". + # # Should be selected if we can mmap non-coherent mappings to userspace. # The only thing that is really required is a way to set an uncached bit
Signature · 1 line
2.31.1.818.g46aad6cb9e-goog
PATCH v8 4/15 swiotlb: Add restricted DMA pool initialization
In reply to Claire ChangAdd the initialization function to create restricted DMA pools from matching reserved-memory nodes.
--- include/linux/device.h | 4 +++ include/linux/swiotlb.h | 3 +- kernel/dma/swiotlb.c | 76 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 82 insertions(+), 1 deletion(-)
include/linux/device.h +4 -0
@@ -416,6 +416,7 @@ struct dev_links_info {* @dma_pools: Dma pools (if dma'ble device). * @dma_mem: Internal for coherent mem override. * @cma_area: Contiguous memory area for dma allocations +* @dma_io_tlb_mem: Internal for swiotlb io_tlb_mem override. * @archdata: For arch-specific additions. * @of_node: Associated device tree node. * @fwnode: Associated device node supplied by platform firmware.
@@ -521,6 +522,9 @@ struct device {#ifdef CONFIG_DMA_CMA struct cma *cma_area;/* contiguous memory area for dma allocations */ +#endif +#ifdef CONFIG_DMA_RESTRICTED_POOL +struct io_tlb_mem *dma_io_tlb_mem;#endif /* arch specific additions */ struct dev_archdata archdata;
include/linux/swiotlb.h +2 -1
@@ -72,7 +72,8 @@ extern enum swiotlb_force swiotlb_force;* range check to see if the memory was in fact allocated by this * API. * @nslabs: The number of IO TLB blocks (in groups of 64) between @start and -* @end. This is command line adjustable via setup_io_tlb_npages. +* @end. For default swiotlb, this is command line adjustable via +* setup_io_tlb_npages. * @used: The number of used IO TLB block. * @list: The free list describing the number of free entries available * from each index.
kernel/dma/swiotlb.c +76 -0
85 lines not on this page · raw message
Signature · 6 lines
2.31.1.818.g46aad6cb9e-goog _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v8 5/15 swiotlb: Add a new get_io_tlb_mem getter
In reply to Claire ChangAdd a new getter, get_io_tlb_mem, to help select the io_tlb_mem struct. The restricted DMA pool is preferred if available.
The reason it was done this way instead of assigning the active pool to dev->dma_io_tlb_mem was because directly using dev->dma_io_tlb_mem might cause memory allocation issues for existing devices. The pool can't support atomic coherent allocation so swiotlb_alloc needs to distinguish it from the default swiotlb pool.
--- include/linux/swiotlb.h | 11 +++++++++++ 1 file changed, 11 insertions(+)
include/linux/swiotlb.h +11 -0
@@ -2,6 +2,7 @@#ifndef __LINUX_SWIOTLB_H#define __LINUX_SWIOTLB_H +#include <linux/device.h> #include <linux/dma-direction.h> #include <linux/init.h> #include <linux/types.h>
@@ -102,6 +103,16 @@ struct io_tlb_mem { };extern struct io_tlb_mem *io_tlb_default_mem; +static inline struct io_tlb_mem *get_io_tlb_mem (struct device *dev) +{ +#ifdef CONFIG_DMA_RESTRICTED_POOL +if (dev && dev->dma_io_tlb_mem) +return dev->dma_io_tlb_mem; +#endif /* CONFIG_DMA_RESTRICTED_POOL */ + +return io_tlb_default_mem; +} +static inline bool is_swiotlb_buffer (phys_addr_t paddr) {struct io_tlb_mem *mem = io_tlb_default_mem;
Signature · 1 line
2.31.1.818.g46aad6cb9e-goog
PATCH v8 6/15 swiotlb: Update is_swiotlb_buffer to add a struct device argument
In reply to Claire ChangUpdate is_swiotlb_buffer to add a struct device argument. This will be useful later to allow for restricted DMA pool.
--- drivers/iommu/dma-iommu.c | 12 ++++++------ drivers/xen/swiotlb-xen.c | 2 +- include/linux/swiotlb.h | 6 +++--- kernel/dma/direct.c | 6 +++--- kernel/dma/direct.h | 6 +++--- 5 files changed, 16 insertions(+), 16 deletions(-)
drivers/iommu/dma-iommu.c +6 -6
@@ -504,7 +504,7 @@ static void __iommu_dma_unmap_swiotlb(struct device *dev, dma_addr_t dma_addr,__iommu_dma_unmap (dev, dma_addr, size); -if (unlikely (is_swiotlb_buffer (phys))) +if (unlikely (is_swiotlb_buffer (dev, phys)))swiotlb_tbl_unmap_single (dev, phys, size, dir, attrs); }
@@ -575,7 +575,7 @@ static dma_addr_t __iommu_dma_map_swiotlb(struct device *dev, phys_addr_t phys, } iova =__iommu_dma_map (dev, phys, aligned_size, prot, dma_mask); -if (iova ==DMA_MAPPING_ERROR &&is_swiotlb_buffer (phys)) +if (iova ==DMA_MAPPING_ERROR &&is_swiotlb_buffer (dev, phys))swiotlb_tbl_unmap_single (dev, phys, org_size, dir, attrs);return iova; }
@@ -781,7 +781,7 @@ static void iommu_dma_sync_single_for_cpu(struct device *dev,if (!dev_is_dma_coherent (dev))arch_sync_dma_for_cpu (phys, size, dir); -if (is_swiotlb_buffer (phys)) +if (is_swiotlb_buffer (dev, phys))swiotlb_sync_single_for_cpu (dev, phys, size, dir); }
@@ -794,7 +794,7 @@ static void iommu_dma_sync_single_for_device(struct device *dev,return ; phys =iommu_iova_to_phys (iommu_get_dma_domain (dev), dma_handle); -if (is_swiotlb_buffer (phys)) +if (is_swiotlb_buffer (dev, phys))swiotlb_sync_single_for_device (dev, phys, size, dir);if (!dev_is_dma_coherent (dev))
@@ -815,7 +815,7 @@ static void iommu_dma_sync_sg_for_cpu(struct device *dev,if (!dev_is_dma_coherent (dev))arch_sync_dma_for_cpu (sg_phys (sg), sg->length, dir); -if (is_swiotlb_buffer (sg_phys (sg))) +if (is_swiotlb_buffer (dev,sg_phys (sg)))swiotlb_sync_single_for_cpu (dev,sg_phys (sg), sg->length, dir); }
@@ -832,7 +832,7 @@ static void iommu_dma_sync_sg_for_device(struct device *dev,return ;for_each_sg (sgl, sg, nelems, i) { -if (is_swiotlb_buffer (sg_phys (sg))) +if (is_swiotlb_buffer (dev,sg_phys (sg)))swiotlb_sync_single_for_device (dev,sg_phys (sg), sg->length, dir);
drivers/xen/swiotlb-xen.c +1 -1
@@ -100,7 +100,7 @@ static int is_xen_swiotlb_buffer(struct device *dev, dma_addr_t dma_addr)* in our domain. Therefore _only_ check address within our domain. */ if (pfn_valid (PFN_DOWN (paddr))) -return is_swiotlb_buffer (paddr); +return is_swiotlb_buffer (dev, paddr);return 0 ; }
include/linux/swiotlb.h +3 -3
@@ -113,9 +113,9 @@ static inline struct io_tlb_mem *get_io_tlb_mem(struct device *dev)return io_tlb_default_mem; } -static inline bool is_swiotlb_buffer (phys_addr_t paddr) +static inline bool is_swiotlb_buffer (struct device *dev,phys_addr_t paddr) { -struct io_tlb_mem *mem = io_tlb_default_mem; +struct io_tlb_mem *mem =get_io_tlb_mem (dev);return mem && paddr >= mem->start && paddr < mem->end; }
@@ -127,7 +127,7 @@ bool is_swiotlb_active(void);void __init swiotlb_adjust_size (unsigned long size);#else #define swiotlb_force SWIOTLB_NO_FORCE -static inline bool is_swiotlb_buffer (phys_addr_t paddr) +static inline bool is_swiotlb_buffer (struct device *dev,phys_addr_t paddr) {return false; }
kernel/dma/direct.c +3 -3
@@ -343,7 +343,7 @@ void dma_direct_sync_sg_for_device(struct device *dev,for_each_sg (sgl, sg, nents, i) {phys_addr_t paddr =dma_to_phys (dev,sg_dma_address (sg)); -if (unlikely (is_swiotlb_buffer (paddr))) +if (unlikely (is_swiotlb_buffer (dev, paddr)))swiotlb_sync_single_for_device (dev, paddr, sg->length, dir);
@@ -369,7 +369,7 @@ void dma_direct_sync_sg_for_cpu(struct device *dev,if (!dev_is_dma_coherent (dev))arch_sync_dma_for_cpu (paddr, sg->length, dir); -if (unlikely (is_swiotlb_buffer (paddr))) +if (unlikely (is_swiotlb_buffer (dev, paddr)))swiotlb_sync_single_for_cpu (dev, paddr, sg->length, dir);
@@ -504,7 +504,7 @@ size_t dma_direct_max_mapping_size(struct device *dev)bool dma_direct_need_sync (struct device *dev,dma_addr_t dma_addr) {return !dev_is_dma_coherent (dev) || -is_swiotlb_buffer (dma_to_phys (dev, dma_addr)); +is_swiotlb_buffer (dev,dma_to_phys (dev, dma_addr)); }/**
kernel/dma/direct.h +3 -3
@@ -56,7 +56,7 @@ static inline void dma_direct_sync_single_for_device(struct device *dev, {phys_addr_t paddr =dma_to_phys (dev, addr); -if (unlikely (is_swiotlb_buffer (paddr))) +if (unlikely (is_swiotlb_buffer (dev, paddr)))swiotlb_sync_single_for_device (dev, paddr, size, dir);if (!dev_is_dma_coherent (dev))
@@ -73,7 +73,7 @@ static inline void dma_direct_sync_single_for_cpu(struct device *dev,arch_sync_dma_for_cpu_all (); } -if (unlikely (is_swiotlb_buffer (paddr))) +if (unlikely (is_swiotlb_buffer (dev, paddr)))swiotlb_sync_single_for_cpu (dev, paddr, size, dir);if (dir ==DMA_FROM_DEVICE )
@@ -113,7 +113,7 @@ static inline void dma_direct_unmap_page(struct device *dev, dma_addr_t addr,if (!(attrs &DMA_ATTR_SKIP_CPU_SYNC ))dma_direct_sync_single_for_cpu (dev, addr, size, dir); -if (unlikely (is_swiotlb_buffer (phys))) +if (unlikely (is_swiotlb_buffer (dev, phys))) swiotlb_tbl_unmap_single(dev, phys, size, dir, attrs); }#endif /* _KERNEL_DMA_DIRECT_H */
Signature · 6 lines
2.31.1.818.g46aad6cb9e-goog _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v8 7/15 swiotlb: Update is_swiotlb_active to add a struct device argument
In reply to Claire ChangUpdate is_swiotlb_active to add a struct device argument. This will be useful later to allow for restricted DMA pool.
--- drivers/gpu/drm/i915/gem/i915_gem_internal.c | 2 +- drivers/gpu/drm/nouveau/nouveau_ttm.c | 2 +- drivers/pci/xen-pcifront.c | 2 +- include/linux/swiotlb.h | 4 ++-- kernel/dma/direct.c | 2 +- kernel/dma/swiotlb.c | 4 ++-- 6 files changed, 8 insertions(+), 8 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_internal.c +1 -1
@@ -42,7 +42,7 @@ static int i915_gem_object_get_pages_internal(struct drm_i915_gem_object *obj) max_order =MAX_ORDER ;#ifdef CONFIG_SWIOTLB -if (is_swiotlb_active ()) { +if (is_swiotlb_active (NULL )) {unsigned int max_segment; max_segment =swiotlb_max_segment ();
drivers/gpu/drm/nouveau/nouveau_ttm.c +1 -1
@@ -270,7 +270,7 @@ nouveau_ttm_init(struct nouveau_drm *drm) }#if IS_ENABLED (CONFIG_SWIOTLB ) &&IS_ENABLED (CONFIG_X86 ) - need_swiotlb =is_swiotlb_active (); + need_swiotlb =is_swiotlb_active (NULL );#endif ret = ttm_device_init(&drm->ttm.bdev, &nouveau_bo_driver, drm->dev->dev,
drivers/pci/xen-pcifront.c +1 -1
@@ -693,7 +693,7 @@ static int pcifront_connect_and_init_dma(struct pcifront_device *pdev)spin_unlock (&pcifront_dev_lock); -if (!err && !is_swiotlb_active ()) { +if (!err && !is_swiotlb_active (NULL )) { err =pci_xen_swiotlb_init_late ();if (err)dev_err (&pdev->xdev->dev,"Could not setup SWIOTLB!\n" );
include/linux/swiotlb.h +2 -2
@@ -123,7 +123,7 @@ static inline bool is_swiotlb_buffer(struct device *dev, phys_addr_t paddr)void __init swiotlb_exit (void );unsigned int swiotlb_max_segment (void );size_t swiotlb_max_mapping_size (struct device *dev); -bool is_swiotlb_active (void ); +bool is_swiotlb_active (struct device *dev);void __init swiotlb_adjust_size (unsigned long size);#else #define swiotlb_force SWIOTLB_NO_FORCE
@@ -143,7 +143,7 @@ static inline size_t swiotlb_max_mapping_size(struct device *dev)return SIZE_MAX ; } -static inline bool is_swiotlb_active (void ) +static inline bool is_swiotlb_active (struct device *dev) {return false; }
kernel/dma/direct.c +1 -1
@@ -495,7 +495,7 @@ int dma_direct_supported(struct device *dev, u64 mask)size_t dma_direct_max_mapping_size (struct device *dev) {/* If SWIOTLB is active, use its maximum mapping size */ -if (is_swiotlb_active () && +if (is_swiotlb_active (dev) && (dma_addressing_limited (dev) || swiotlb_force ==SWIOTLB_FORCE ))return swiotlb_max_mapping_size (dev);return SIZE_MAX ;
kernel/dma/swiotlb.c +2 -2
@@ -662,9 +662,9 @@ size_t swiotlb_max_mapping_size(struct device *dev)return ((size_t )IO_TLB_SIZE ) *IO_TLB_SEGSIZE ; } -bool is_swiotlb_active (void ) +bool is_swiotlb_active (struct device *dev) { -return io_tlb_default_mem !=NULL ; +return get_io_tlb_mem (dev) !=NULL ; }EXPORT_SYMBOL_GPL (is_swiotlb_active);
Signature · 1 line
2.31.1.818.g46aad6cb9e-goog
PATCH v8 8/15 swiotlb: Bounce data from/to restricted DMA pool if available
In reply to Claire ChangRegardless of swiotlb setting, the restricted DMA pool is preferred if available.
The restricted DMA pools provide a basic level of protection against the DMA overwriting buffer contents at unexpected times. However, to protect against general data leakage and system memory corruption, the system needs to provide a way to lock down the memory access, e.g., MPU.
Note that is_dev_swiotlb_force doesn't check if swiotlb_force == SWIOTLB_FORCE. Otherwise the memory allocation behavior with default swiotlb will be changed by the following patche ("dma-direct: Allocate memory from restricted DMA pool if available").
--- include/linux/swiotlb.h | 13 +++++++++++++ kernel/dma/direct.c | 3 ++- kernel/dma/direct.h | 3 ++- kernel/dma/swiotlb.c | 8 ++++---- 4 files changed, 21 insertions(+), 6 deletions(-)
include/linux/swiotlb.h +13 -0
@@ -120,6 +120,15 @@ static inline bool is_swiotlb_buffer(struct device *dev, phys_addr_t paddr)return mem && paddr >= mem->start && paddr < mem->end; } +static inline bool is_dev_swiotlb_force (struct device *dev) +{ +#ifdef CONFIG_DMA_RESTRICTED_POOL +if (dev->dma_io_tlb_mem) +return true; +#endif /* CONFIG_DMA_RESTRICTED_POOL */ +return false; +} +void __init swiotlb_exit (void );unsigned int swiotlb_max_segment (void );size_t swiotlb_max_mapping_size (struct device *dev);
@@ -131,6 +140,10 @@ static inline bool is_swiotlb_buffer(struct device *dev, phys_addr_t paddr) {return false; } +static inline bool is_dev_swiotlb_force (struct device *dev) +{ +return false; +}static inline void swiotlb_exit (void ) { }
kernel/dma/direct.c +2 -1
@@ -496,7 +496,8 @@ size_t dma_direct_max_mapping_size(struct device *dev) {/* If SWIOTLB is active, use its maximum mapping size */ if (is_swiotlb_active (dev) && - (dma_addressing_limited (dev) || swiotlb_force ==SWIOTLB_FORCE )) + (dma_addressing_limited (dev) || swiotlb_force ==SWIOTLB_FORCE || +is_dev_swiotlb_force (dev)))return swiotlb_max_mapping_size (dev);return SIZE_MAX ; }
kernel/dma/direct.h +2 -1
@@ -87,7 +87,8 @@ static inline dma_addr_t dma_direct_map_page(struct device *dev,phys_addr_t phys =page_to_phys (page) + offset;dma_addr_t dma_addr =phys_to_dma (dev, phys); -if (unlikely (swiotlb_force ==SWIOTLB_FORCE )) +if (unlikely (swiotlb_force ==SWIOTLB_FORCE ) || +is_dev_swiotlb_force (dev))return swiotlb_map (dev, phys, size, dir, attrs);if (unlikely (!dma_capable (dev, dma_addr, size, true))) {
kernel/dma/swiotlb.c +4 -4
@@ -347,7 +347,7 @@ void __init swiotlb_exit(void)static void swiotlb_bounce (struct device *dev,phys_addr_t tlb_addr,size_t size,enum dma_data_direction dir) { -struct io_tlb_mem *mem = io_tlb_default_mem; +struct io_tlb_mem *mem =get_io_tlb_mem (dev);int index = (tlb_addr - mem->start) >>IO_TLB_SHIFT ;phys_addr_t orig_addr = mem->slots[index].orig_addr;size_t alloc_size = mem->slots[index].alloc_size;
@@ -429,7 +429,7 @@ static unsigned int wrap_index(struct io_tlb_mem *mem, unsigned int index)static int find_slots (struct device *dev,phys_addr_t orig_addr,size_t alloc_size) { -struct io_tlb_mem *mem = io_tlb_default_mem; +struct io_tlb_mem *mem =get_io_tlb_mem (dev);unsigned long boundary_mask =dma_get_seg_boundary (dev);dma_addr_t tbl_dma_addr =phys_to_dma_unencrypted (dev, mem->start) & boundary_mask;
@@ -506,7 +506,7 @@ phys_addr_t swiotlb_tbl_map_single(struct device *dev, phys_addr_t orig_addr, size_t mapping_size, size_t alloc_size, enum dma_data_direction dir, unsigned long attrs) { - struct io_tlb_mem *mem = io_tlb_default_mem; + struct io_tlb_mem *mem = get_io_tlb_mem(dev);unsigned int offset =swiotlb_align_offset (dev, orig_addr);unsigned int i;int index;
@@ -557,7 +557,7 @@ void swiotlb_tbl_unmap_single(struct device *hwdev, phys_addr_t tlb_addr, size_t mapping_size, enum dma_data_direction dir, unsigned long attrs) { - struct io_tlb_mem *mem = io_tlb_default_mem; + struct io_tlb_mem *mem = get_io_tlb_mem(hwdev);unsigned long flags;unsigned int offset =swiotlb_align_offset (hwdev, tlb_addr);int index = (tlb_addr - offset - mem->start) >>IO_TLB_SHIFT ;
Signature · 6 lines
2.31.1.818.g46aad6cb9e-goog _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v8 9/15 swiotlb: Move alloc_size to find_slots
In reply to Claire ChangMove the maintenance of alloc_size to find_slots for better code reusability later.
--- kernel/dma/swiotlb.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
kernel/dma/swiotlb.c +5 -5
@@ -482,8 +482,11 @@ static int find_slots(struct device *dev, phys_addr_t orig_addr,return -1 ; found: -for (i = index; i < index + nslots; i++) +for (i = index; i < index + nslots; i++) { mem->slots[i].list =0 ; + mem->slots[i].alloc_size = + alloc_size - ((i - index) <<IO_TLB_SHIFT ); + }for (i = index -1 ;io_tlb_offset (i) !=IO_TLB_SEGSIZE -1 && mem->slots[i].list; i--)
@@ -538,11 +541,8 @@ phys_addr_t swiotlb_tbl_map_single(struct device *dev, phys_addr_t orig_addr,* This is needed when we sync the memory. Then we sync the buffer if * needed. */ -for (i =0 ; i <nr_slots (alloc_size + offset); i++) { +for (i =0 ; i <nr_slots (alloc_size + offset); i++) mem->slots[index + i].orig_addr =slot_addr (orig_addr, i); - mem->slots[index + i].alloc_size = - alloc_size - (i <<IO_TLB_SHIFT ); - } tlb_addr =slot_addr (mem->start, index) + offset;if (!(attrs &DMA_ATTR_SKIP_CPU_SYNC ) && (dir ==DMA_TO_DEVICE || dir ==DMA_BIDIRECTIONAL ))
Signature · 1 line
2.31.1.818.g46aad6cb9e-goog
PATCH v8 10/15 swiotlb: Refactor swiotlb_tbl_unmap_single
In reply to Claire ChangAdd a new function, release_slots, to make the code reusable for supporting different bounce buffer pools, e.g. restricted DMA pool.
--- kernel/dma/swiotlb.c | 35 ++++++++++++++++++++--------------- 1 file changed, 20 insertions(+), 15 deletions(-)
kernel/dma/swiotlb.c +20 -15
@@ -550,27 +550,15 @@ phys_addr_t swiotlb_tbl_map_single(struct device *dev, phys_addr_t orig_addr,return tlb_addr; } -/* -* tlb_addr is the physical address of the bounce buffer to unmap. -*/ -void swiotlb_tbl_unmap_single (struct device *hwdev,phys_addr_t tlb_addr, -size_t mapping_size,enum dma_data_direction dir, -unsigned long attrs) +static void release_slots (struct device *dev,phys_addr_t tlb_addr) { -struct io_tlb_mem *mem =get_io_tlb_mem (hwdev); +struct io_tlb_mem *mem =get_io_tlb_mem (dev);unsigned long flags; -unsigned int offset =swiotlb_align_offset (hwdev, tlb_addr); +unsigned int offset =swiotlb_align_offset (dev, tlb_addr);int index = (tlb_addr - offset - mem->start) >>IO_TLB_SHIFT ;int nslots =nr_slots (mem->slots[index].alloc_size + offset);int count, i; -/* -* First, sync the memory before unmapping the entry -*/ -if (!(attrs &DMA_ATTR_SKIP_CPU_SYNC ) && - (dir ==DMA_FROM_DEVICE || dir ==DMA_BIDIRECTIONAL )) -swiotlb_bounce (hwdev, tlb_addr, mapping_size,DMA_FROM_DEVICE ); -/* * Return the buffer to the free list by setting the corresponding * entries to indicate the number of contiguous entries available.
@@ -605,6 +593,23 @@ void swiotlb_tbl_unmap_single(struct device *hwdev, phys_addr_t tlb_addr,spin_unlock_irqrestore (&mem->lock, flags); } +/* +* tlb_addr is the physical address of the bounce buffer to unmap. +*/ +void swiotlb_tbl_unmap_single (struct device *dev,phys_addr_t tlb_addr, +size_t mapping_size,enum dma_data_direction dir, +unsigned long attrs) +{ +/* +* First, sync the memory before unmapping the entry +*/ +if (!(attrs &DMA_ATTR_SKIP_CPU_SYNC ) && + (dir ==DMA_FROM_DEVICE || dir ==DMA_BIDIRECTIONAL )) +swiotlb_bounce (dev, tlb_addr, mapping_size,DMA_FROM_DEVICE ); + +release_slots (dev, tlb_addr); +} +void swiotlb_sync_single_for_device (struct device *dev,phys_addr_t tlb_addr,size_t size,enum dma_data_direction dir) {
Signature · 6 lines
2.31.1.818.g46aad6cb9e-goog _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
I just finished reviewing v7, sorry. Let me find some time to see what difference this version makes.
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Hi Claire,
Quoted 26 lines · Claire Chang
This series implements mitigations for lack of DMA access control on systems without an IOMMU, which could result in the DMA accessing the system memory at unexpected times and/or unexpected addresses, possibly leading to data leakage or corruption.
For example, we plan to use the PCI-e bus for Wi-Fi and that PCI-e bus is not behind an IOMMU. As PCI-e, by design, gives the device full access to system memory, a vulnerability in the Wi-Fi firmware could easily escalate to a full system exploit (remote wifi exploits: [1a], [1b] that shows a full chain of exploits; [2], [3]).
To mitigate the security concerns, we introduce restricted DMA. Restricted DMA utilizes the existing swiotlb to bounce streaming DMA in and out of a specially allocated region and does memory allocation from the same region. The feature on its own provides a basic level of protection against the DMA overwriting buffer contents at unexpected times. However, to protect against general data leakage and system memory corruption, the system needs to provide a way to restrict the DMA to a predefined memory region (this is usually done at firmware level, e.g. MPU in ATF on some ARM platforms [4]).
[1a] https://googleprojectzero.blogspot.com/2017/04/over-air-exploiting-broadcoms-wi-fi_4.html
[1b] https://googleprojectzero.blogspot.com/2017/04/over-air-exploiting-broadcoms-wi-fi_11.html
[2] https://blade.tencent.com/en/advisories/qualpwn/
[3] https://www.bleepingcomputer.com/news/security/vulnerabilities-found-in-highly-popular-firmware-for-wifi-chips/
[4] https://github.com/ARM-software/arm-trusted-firmware/blob/master/plat/mediatek/mt8183/drivers/emi_mpu/emi_mpu.c#L132
Quoted 11 lines
v8:
- Fix reserved-memory.txt and add the reg property in example.
- Fix sizeof forof_property_count_elems_of_sizein drivers/of/address.c#of_dma_set_restricted_buffer.
- Apply Will's suggestion to try the OF node having DMA configuration in drivers/of/address.c#of_dma_set_restricted_buffer.
- Fix typo in the comment of drivers/of/address.c#of_dma_set_restricted_buffer.
- Add error message for PageHighMem in kernel/dma/swiotlb.c#rmem_swiotlb_device_init and move it tormem_swiotlb_setup.
- Fix the message string inrmem_swiotlb_setup.
Thanks for the v8. It works for me out of the box on arm64 under KVM, so:
Note that something seems to have gone wrong with the mail threading, so the last 5 patches ended up as a separate thread for me. Probably worth posting again with all the patches in one place, if you can.
Cheers,
Will
Quoted 42 lines · Will Deacon
Hi Claire,
Quoted 26 lines · Claire Chang
This series implements mitigations for lack of DMA access control on systems without an IOMMU, which could result in the DMA accessing the system memory at unexpected times and/or unexpected addresses, possibly leading to data leakage or corruption.
For example, we plan to use the PCI-e bus for Wi-Fi and that PCI-e bus is not behind an IOMMU. As PCI-e, by design, gives the device full access to system memory, a vulnerability in the Wi-Fi firmware could easily escalate to a full system exploit (remote wifi exploits: [1a], [1b] that shows a full chain of exploits; [2], [3]).
To mitigate the security concerns, we introduce restricted DMA. Restricted DMA utilizes the existing swiotlb to bounce streaming DMA in and out of a specially allocated region and does memory allocation from the same region. The feature on its own provides a basic level of protection against the DMA overwriting buffer contents at unexpected times. However, to protect against general data leakage and system memory corruption, the system needs to provide a way to restrict the DMA to a predefined memory region (this is usually done at firmware level, e.g. MPU in ATF on some ARM platforms [4]).
[1a] https://googleprojectzero.blogspot.com/2017/04/over-air-exploiting-broadcoms-wi-fi_4.html
[1b] https://googleprojectzero.blogspot.com/2017/04/over-air-exploiting-broadcoms-wi-fi_11.html
[2] https://blade.tencent.com/en/advisories/qualpwn/
[3] https://www.bleepingcomputer.com/news/security/vulnerabilities-found-in-highly-popular-firmware-for-wifi-chips/
[4] https://github.com/ARM-software/arm-trusted-firmware/blob/master/plat/mediatek/mt8183/drivers/emi_mpu/emi_mpu.c#L132Quoted 11 lines
v8:
- Fix reserved-memory.txt and add the reg property in example.
- Fix sizeof forof_property_count_elems_of_sizein drivers/of/address.c#of_dma_set_restricted_buffer.
- Apply Will's suggestion to try the OF node having DMA configuration in drivers/of/address.c#of_dma_set_restricted_buffer.
- Fix typo in the comment of drivers/of/address.c#of_dma_set_restricted_buffer.
- Add error message for PageHighMem in kernel/dma/swiotlb.c#rmem_swiotlb_device_init and move it tormem_swiotlb_setup.
- Fix the message string inrmem_swiotlb_setup.
Quoted 7 lines
Thanks for the v8. It works for me out of the box on arm64 under KVM, so:
Tested-by Will Deacon <[email protected]>Note that something seems to have gone wrong with the mail threading, so the last 5 patches ended up as a separate thread for me. Probably worth posting again with all the patches in one place, if you can.
Thanks for testing.
Christoph also added some comments in v7, so I'll prepare v9.
Quoted 4 lines
Cheers,
Will
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
v9 here: https://lore.kernel.org/patchwork/cover/1445081/
Quoted 48 lines · Claire Chang
Quoted 46 lines · Will Deacon
Hi Claire,
Quoted 26 lines · Claire Chang
This series implements mitigations for lack of DMA access control on systems without an IOMMU, which could result in the DMA accessing the system memory at unexpected times and/or unexpected addresses, possibly leading to data leakage or corruption.
For example, we plan to use the PCI-e bus for Wi-Fi and that PCI-e bus is not behind an IOMMU. As PCI-e, by design, gives the device full access to system memory, a vulnerability in the Wi-Fi firmware could easily escalate to a full system exploit (remote wifi exploits: [1a], [1b] that shows a full chain of exploits; [2], [3]).
To mitigate the security concerns, we introduce restricted DMA. Restricted DMA utilizes the existing swiotlb to bounce streaming DMA in and out of a specially allocated region and does memory allocation from the same region. The feature on its own provides a basic level of protection against the DMA overwriting buffer contents at unexpected times. However, to protect against general data leakage and system memory corruption, the system needs to provide a way to restrict the DMA to a predefined memory region (this is usually done at firmware level, e.g. MPU in ATF on some ARM platforms [4]).
[1a] https://googleprojectzero.blogspot.com/2017/04/over-air-exploiting-broadcoms-wi-fi_4.html
[1b] https://googleprojectzero.blogspot.com/2017/04/over-air-exploiting-broadcoms-wi-fi_11.html
[2] https://blade.tencent.com/en/advisories/qualpwn/
[3] https://www.bleepingcomputer.com/news/security/vulnerabilities-found-in-highly-popular-firmware-for-wifi-chips/
[4] https://github.com/ARM-software/arm-trusted-firmware/blob/master/plat/mediatek/mt8183/drivers/emi_mpu/emi_mpu.c#L132Quoted 11 lines
v8:
- Fix reserved-memory.txt and add the reg property in example.
- Fix sizeof forof_property_count_elems_of_sizein drivers/of/address.c#of_dma_set_restricted_buffer.
- Apply Will's suggestion to try the OF node having DMA configuration in drivers/of/address.c#of_dma_set_restricted_buffer.
- Fix typo in the comment of drivers/of/address.c#of_dma_set_restricted_buffer.
- Add error message for PageHighMem in kernel/dma/swiotlb.c#rmem_swiotlb_device_init and move it tormem_swiotlb_setup.
- Fix the message string inrmem_swiotlb_setup.Thanks for the v8. It works for me out of the box on arm64 under KVM, so:
Tested-by Will Deacon <[email protected]>
Quoted 12 lines
Quoted 3 lines
Note that something seems to have gone wrong with the mail threading, so the last 5 patches ended up as a separate thread for me. Probably worth posting again with all the patches in one place, if you can.
Thanks for testing.
Christoph also added some comments in v7, so I'll prepare v9.
Quoted 4 lines
Cheers,
Will
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx