iommu/io-pgtable-arm: Don't use dma_to_phys()
Subsystems (2)
Robin Murphy
reviewerPATCH 1/1 iommu/io-pgtable-arm: Don't use dma_to_phys()
In checking whether DMA addresses differ from physical addresses, using dma_to_phys() is actually the wrong thing to do, since it may hide any DMA offset, which is precisely one of the things we are checking for. Simply casting between the two address types, whilst ugly, is in fact the appropriate course of action.
We can also reject any device with a fixed DMA offset up-front at page table creation, leaving the allocation-time check for the more subtle cases like bounce buffering due to an incorrect DMA mask.
Furthermore, we can then fix the hackish KConfig dependency so that architectures without a dma_to_phys() implementation may still COMPILE_TEST (or even use!) the code. The true dependency is on the DMA API, so use the appropriate symbol for that.
--- drivers/iommu/Kconfig | 3 +-- drivers/iommu/io-pgtable-arm.c | 16 ++++++++-------- 2 files changed, 9 insertions(+), 10 deletions(-)
drivers/iommu/Kconfig +1 -2
@@ -23,8 +23,7 @@ config IOMMU_IO_PGTABLE config IOMMU_IO_PGTABLE_LPAE bool "ARMv7/v8 Long Descriptor Format" select IOMMU_IO_PGTABLE - # SWIOTLB guarantees a dma_to_phys() implementation - depends on ARM || ARM64 || (COMPILE_TEST && SWIOTLB) + depends on HAS_DMA && (ARM || ARM64 || COMPILE_TEST) help Enable support for the ARM long descriptor pagetable format. This allocator supports 4K/2M/1G, 16K/32M and 64K/512M page
drivers/iommu/io-pgtable-arm.c +8 -8
@@ -202,11 +202,6 @@ typedef u64 arm_lpae_iopte;static bool selftest_running = false; -static dma_addr_t __arm_lpae_dma_addr (struct device *dev,void *pages) -{ -return phys_to_dma (dev,virt_to_phys (pages)); -} -static void *__arm_lpae_alloc_pages (size_t size,gfp_t gfp,struct io_pgtable_cfg *cfg) {
@@ -226,7 +221,7 @@ static void *__arm_lpae_alloc_pages(size_t size, gfp_t gfp,* address directly, so if the DMA layer suggests it can't by * giving us back some translation, that bodes very badly... */ -if (dma !=__arm_lpae_dma_addr (dev, pages)) +if ((u64 )dma != (u64 )virt_to_phys (pages)) goto out_unmap; }
@@ -246,7 +241,7 @@ static void __arm_lpae_free_pages(void *pages, size_t size,struct device *dev = cfg->iommu_dev;if (!selftest_running) -dma_unmap_single (dev,__arm_lpae_dma_addr (dev, pages), +dma_unmap_single (dev, (dma_addr_t )virt_to_phys (pages), size,DMA_TO_DEVICE );free_pages_exact (pages, size); }
@@ -259,7 +254,7 @@ static void __arm_lpae_set_pte(arm_lpae_iopte *ptep, arm_lpae_iopte pte, *ptep = pte;if (!selftest_running) -dma_sync_single_for_device (dev,__arm_lpae_dma_addr (dev, ptep), +dma_sync_single_for_device (dev, (dma_addr_t )virt_to_phys (ptep),sizeof (pte),DMA_TO_DEVICE ); }
@@ -629,6 +624,11 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg)if (cfg->oas >ARM_LPAE_MAX_ADDR_BITS )return NULL ; +if (cfg->iommu_dev->dma_pfn_offset) { +dev_err (cfg->iommu_dev,"Cannot accommodate DMA offset for IOMMU page tables\n" ); +return NULL ; + } + data =kmalloc (sizeof (*data),GFP_KERNEL );if (!data)return NULL ;
Signature · 1 line
1.9.1
PATCH 1/1 iommu/io-pgtable-arm: Don't use dma_to_phys()
Quoted 5 lines · Robin Murphy
In checking whether DMA addresses differ from physical addresses, using
dma_to_phys()is actually the wrong thing to do, since it may hide any DMA offset, which is precisely one of the things we are checking for. Simply casting between the two address types, whilst ugly, is in fact the appropriate course of action.
Urgh... yes.
Quoted 34 lines
We can also reject any device with a fixed DMA offset up-front at page table creation, leaving the allocation-time check for the more subtle cases like bounce buffering due to an incorrect DMA mask.
Furthermore, we can then fix the hackish KConfig dependency so that architectures without a
dma_to_phys()implementation may stillCOMPILE_TEST(or even use!) the code. The true dependency is on the DMA API, so use the appropriate symbol for that.Signed-off-by Robin Murphy <[email protected]>--- drivers/iommu/Kconfig | 3 +-- drivers/iommu/io-pgtable-arm.c | 16 ++++++++-------- 2 files changed, 9 insertions(+), 10 deletions(-)drivers/iommu/Kconfig +1
-2@@ -23,8 +23,7 @@ config IOMMU_IO_PGTABLE config IOMMU_IO_PGTABLE_LPAE bool "ARMv7/v8 Long Descriptor Format" select IOMMU_IO_PGTABLE - # SWIOTLB guarantees a dma_to_phys() implementation - depends on ARM || ARM64 || (COMPILE_TEST && SWIOTLB) + depends on HAS_DMA && (ARM || ARM64 || COMPILE_TEST) help Enable support for the ARM long descriptor pagetable format. This allocator supports 4K/2M/1G, 16K/32M and 64K/512M pagedrivers/iommu/io-pgtable-arm.c +0
-0
drivers/iommu/io-pgtable-arm.c typedef u64 arm_lpae_iopte; · line 208
Quoted 7 lines
drivers/iommu/io-pgtable-arm.c +0
-4@@ -202,11 +202,6 @@ typedef u64 arm_lpae_iopte;static bool selftest_running = false; -static dma_addr_t __arm_lpae_dma_addr (struct device *dev,void *pages) -{ -return phys_to_dma (dev,virt_to_phys (pages)); -}
Can we keep this helper kicking around, at least to contain the ugliness of the virt_to_phys + cast?
Will
Robin Murphy
reviewerPATCH 1/1 iommu/io-pgtable-arm: Don't use dma_to_phys()
On 17/09/15 15:52, Will Deacon wrote:
[...]
Quoted 7 lines
drivers/iommu/io-pgtable-arm.c typedef u64 arm_lpae_iopte; · line 208
Quoted 4 lines
drivers/iommu/io-pgtable-arm.c +0
-4-static dma_addr_t __arm_lpae_dma_addr (struct device *dev,void *pages) -{ -return phys_to_dma (dev,virt_to_phys (pages)); -}Can we keep this helper kicking around, at least to contain the ugliness of the
virt_to_phys+ cast?
Can do for the sync and unmap cases - the alloc_pages one needs to avoid truncation in the comparison if dma_addr_t and phys_addr_t are different sizes, so can't safely cast one to the other either way. I can at least get rid of the u64 casts there though, as they seem superfluous after a bit of experimentation and another read of the good old "Usual arithmetic conversions". I'll fix up and repost.
Robin.
Quoted 3 lines
Will
Robin Murphy
reviewerPATCH v2 1/1 iommu/io-pgtable-arm: Don't use dma_to_phys()
In reply to Robin MurphyIn checking whether DMA addresses differ from physical addresses, using dma_to_phys() is actually the wrong thing to do, since it may hide any DMA offset, which is precisely one of the things we are checking for. Simply casting between the two address types, whilst ugly, is in fact the appropriate course of action. Further care (and ugliness) is also necessary in the comparison to avoid truncation if phys_addr_t and dma_addr_t differ in size.
We can also reject any device with a fixed DMA offset up-front at page table creation, leaving the allocation-time check for the more subtle cases like bounce buffering due to an incorrect DMA mask.
Furthermore, we can then fix the hackish KConfig dependency so that architectures without a dma_to_phys() implementation may still COMPILE_TEST (or even use!) the code. The true dependency is on the DMA API, so use the appropriate symbol for that.
--- drivers/iommu/Kconfig | 3 +-- drivers/iommu/io-pgtable-arm.c | 24 +++++++++++++----------- 2 files changed, 14 insertions(+), 13 deletions(-)
drivers/iommu/Kconfig +1 -2
@@ -23,8 +23,7 @@ config IOMMU_IO_PGTABLE config IOMMU_IO_PGTABLE_LPAE bool "ARMv7/v8 Long Descriptor Format" select IOMMU_IO_PGTABLE - # SWIOTLB guarantees a dma_to_phys() implementation - depends on ARM || ARM64 || (COMPILE_TEST && SWIOTLB) + depends on HAS_DMA && (ARM || ARM64 || COMPILE_TEST) help Enable support for the ARM long descriptor pagetable format. This allocator supports 4K/2M/1G, 16K/32M and 64K/512M page
drivers/iommu/io-pgtable-arm.c +13 -11
@@ -202,9 +202,9 @@ typedef u64 arm_lpae_iopte;static bool selftest_running = false; -static dma_addr_t __arm_lpae_dma_addr (struct device *dev,void *pages) +static dma_addr_t __arm_lpae_dma_addr (void *pages) { -return phys_to_dma (dev,virt_to_phys (pages)); +return (dma_addr_t )virt_to_phys (pages); } static void *__arm_lpae_alloc_pages(size_t size, gfp_t gfp,
@@ -223,10 +223,10 @@ static void *__arm_lpae_alloc_pages(size_t size, gfp_t gfp, goto out_free;/* * We depend on the IOMMU being able to work with any physical -* address directly, so if the DMA layer suggests it can't by -* giving us back some translation, that bodes very badly... +* address directly, so if the DMA layer suggests otherwise by +* translating or truncating them, that bodes very badly... */ -if (dma !=__arm_lpae_dma_addr (dev, pages)) +if (dma !=virt_to_phys (pages)) goto out_unmap; }
@@ -243,10 +243,8 @@ out_free:static void __arm_lpae_free_pages (void *pages,size_t size,struct io_pgtable_cfg *cfg) { -struct device *dev = cfg->iommu_dev; -if (!selftest_running) -dma_unmap_single (dev,__arm_lpae_dma_addr (dev, pages), +dma_unmap_single (cfg->iommu_dev,__arm_lpae_dma_addr (pages), size,DMA_TO_DEVICE );free_pages_exact (pages, size); }
@@ -254,12 +252,11 @@ static void __arm_lpae_free_pages(void *pages, size_t size,static void __arm_lpae_set_pte (arm_lpae_iopte *ptep,arm_lpae_iopte pte,struct io_pgtable_cfg *cfg) { -struct device *dev = cfg->iommu_dev; - *ptep = pte;if (!selftest_running) -dma_sync_single_for_device (dev,__arm_lpae_dma_addr (dev, ptep), +dma_sync_single_for_device (cfg->iommu_dev, +__arm_lpae_dma_addr (ptep),sizeof (pte),DMA_TO_DEVICE ); }
@@ -629,6 +626,11 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg)if (cfg->oas >ARM_LPAE_MAX_ADDR_BITS )return NULL ; +if (cfg->iommu_dev->dma_pfn_offset) { +dev_err (cfg->iommu_dev,"Cannot accommodate DMA offset for IOMMU page tables\n" ); +return NULL ; + } + data =kmalloc (sizeof (*data),GFP_KERNEL );if (!data)return NULL ;
Signature · 1 line
1.9.1
PATCH v2 1/1 iommu/io-pgtable-arm: Don't use dma_to_phys()
Quoted 19 lines · Robin Murphy
In checking whether DMA addresses differ from physical addresses, using
dma_to_phys()is actually the wrong thing to do, since it may hide any DMA offset, which is precisely one of the things we are checking for. Simply casting between the two address types, whilst ugly, is in fact the appropriate course of action. Further care (and ugliness) is also necessary in the comparison to avoid truncation ifphys_addr_tanddma_addr_tdiffer in size.We can also reject any device with a fixed DMA offset up-front at page table creation, leaving the allocation-time check for the more subtle cases like bounce buffering due to an incorrect DMA mask.
Furthermore, we can then fix the hackish KConfig dependency so that architectures without a
dma_to_phys()implementation may stillCOMPILE_TEST(or even use!) the code. The true dependency is on the DMA API, so use the appropriate symbol for that.Signed-off-by Robin Murphy <[email protected]>---
[...]
Quoted 12 lines
static bool
selftest_running= false;drivers/iommu/io-pgtable-arm.c +2
-2-static dma_addr_t __arm_lpae_dma_addr (struct device *dev,void *pages) +static dma_addr_t __arm_lpae_dma_addr (void *pages) { -return phys_to_dma (dev,virt_to_phys (pages)); +return (dma_addr_t )virt_to_phys (pages); } static void *__arm_lpae_alloc_pages(size_t size, gfp_t gfp,
drivers/iommu/io-pgtable-arm.c static void *__arm_lpae_alloc_pages(size_t size, gfp_t gfp, · line 229
Quoted 10 lines
drivers/iommu/io-pgtable-arm.c +3
-3@@ -223,10 +223,10 @@ static void *__arm_lpae_alloc_pages(size_t size, gfp_t gfp, goto out_free;/* * We depend on the IOMMU being able to work with any physical -* address directly, so if the DMA layer suggests it can't by -* giving us back some translation, that bodes very badly... +* address directly, so if the DMA layer suggests otherwise by +* translating or truncating them, that bodes very badly... */ -if (dma !=__arm_lpae_dma_addr (dev, pages)) +if (dma !=virt_to_phys (pages))
Could I ask why not use __arm_lpae_dma_addr(pages) here? dma is dma_addr_t.
Quoted 42 lines
drivers/iommu/io-pgtable-arm.c +8
-6goto out_unmap; }@@ -243,10 +243,8 @@ out_free:static void __arm_lpae_free_pages (void *pages,size_t size,struct io_pgtable_cfg *cfg) { -struct device *dev = cfg->iommu_dev; -if (!selftest_running) -dma_unmap_single (dev,__arm_lpae_dma_addr (dev, pages), +dma_unmap_single (cfg->iommu_dev,__arm_lpae_dma_addr (pages), size,DMA_TO_DEVICE );free_pages_exact (pages, size); }@@ -254,12 +252,11 @@ static void __arm_lpae_free_pages(void *pages, size_t size,static void __arm_lpae_set_pte (arm_lpae_iopte *ptep,arm_lpae_iopte pte,struct io_pgtable_cfg *cfg) { -struct device *dev = cfg->iommu_dev; - *ptep = pte;if (!selftest_running) -dma_sync_single_for_device (dev,__arm_lpae_dma_addr (dev, ptep), +dma_sync_single_for_device (cfg->iommu_dev, +__arm_lpae_dma_addr (ptep),sizeof (pte),DMA_TO_DEVICE ); }@@ -629,6 +626,11 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg)if (cfg->oas >ARM_LPAE_MAX_ADDR_BITS )return NULL ; +if (cfg->iommu_dev->dma_pfn_offset) { +dev_err (cfg->iommu_dev,"Cannot accommodate DMA offset for IOMMU page tables\n" ); +return NULL ; + } + data =kmalloc (sizeof (*data),GFP_KERNEL );if (!data)return NULL ;
Robin Murphy
reviewerPATCH v2 1/1 iommu/io-pgtable-arm: Don't use dma_to_phys()
Quoted 1 line · Yong Wu
On Thu, 2015-09-17 at 17:42 +0100, Robin Murphy wrote:
[...]
Quoted 3 lines
Quoted 3 lines
the appropriate course of action. Further care (and ugliness) is also necessary in the comparison to avoid truncation if
phys_addr_tanddma_addr_tdiffer in size.
[...]
Quoted 4 lines
Quoted 4 lines
/*
* We depend on the IOMMU being able to work with any physicaldrivers/iommu/io-pgtable-arm.c +0
-2-* address directly, so if the DMA layer suggests it can't by -* giving us back some translation, that bodes very badly...
Quoted 8 lines
drivers/iommu/io-pgtable-arm.c static void *__arm_lpae_alloc_pages(size_t size, gfp_t gfp, · line 229
Quoted 5 lines
drivers/iommu/io-pgtable-arm.c +3
-1+* address directly, so if the DMA layer suggests otherwise by +* translating or truncating them, that bodes very badly... */ -if (dma !=__arm_lpae_dma_addr (dev, pages)) +if (dma !=virt_to_phys (pages))Could I ask why not use
__arm_lpae_dma_addr(pages)here? dma isdma_addr_t.
Specifically, the problem case for that is when phys_addr_t is 64-bit but dma_addr_t is 32-bit. The cast in __arm_lpae_dma_addr is necessary to avoid a truncation warning when we make the DMA API calls, but we actually need the opposite in the comparison here - comparing the different types directly allows integer promotion to kick in appropriately so we don't lose the top half of the larger address. Otherwise, you'd never spot the difference between, say, your original page at 0x88c0000000 and a bounce-buffered copy that happened to end up mapped to 0xc0000000.
Robin.
PATCH v2 1/1 iommu/io-pgtable-arm: Don't use dma_to_phys()
Quoted 8 lines · Robin Murphy
Specifically, the problem case for that is when
phys_addr_tis 64-bit butdma_addr_tis 32-bit. The cast in__arm_lpae_dma_addris necessary to avoid a truncation warning when we make the DMA API calls, but we actually need the opposite in the comparison here - comparing the different types directly allows integer promotion to kick in appropriately so we don't lose the top half of the larger address. Otherwise, you'd never spot the difference between, say, your original page at0x88c0000000and a bounce-buffered copy that happened to end up mapped to0xc0000000.
Hmm. Thinking about this, I think we ought to add to arch/arm/mm/Kconfig:
config ARCH_PHYS_ADDR_T_64BIT
def_bool ARM_LPAE
config ARCH_DMA_ADDR_T_64BIT
bool
+ select ARCH_PHYS_ADDR_T_64BIT
I seem to remember that you're quite right that dma_addr_t <= phys_addr_t but dma_addr_t must never be bigger than phys_addr_t.
Signature · 2 lines
FTTC broadband for 0.8mile line: [email protected] down 400kbps up according to speedtest.net.
PATCH v2 1/1 iommu/io-pgtable-arm: Don't use dma_to_phys()
In reply to Robin MurphyQuoted 20 lines · Robin Murphy
Quoted 1 line · Yong Wu
On Thu, 2015-09-17 at 17:42 +0100, Robin Murphy wrote:
[...]
Quoted 3 lines
Quoted 3 lines
the appropriate course of action. Further care (and ugliness) is also necessary in the comparison to avoid truncation if
phys_addr_tanddma_addr_tdiffer in size.[...]
Quoted 4 lines
Quoted 4 lines
/*
* We depend on the IOMMU being able to work with any physicaldrivers/iommu/io-pgtable-arm.c +0
-2-* address directly, so if the DMA layer suggests it can't by -* giving us back some translation, that bodes very badly... Quoted 8 lines
drivers/iommu/io-pgtable-arm.c static void *__arm_lpae_alloc_pages(size_t size, gfp_t gfp, · line 229
Quoted 5 lines
drivers/iommu/io-pgtable-arm.c +3
-1+* address directly, so if the DMA layer suggests otherwise by +* translating or truncating them, that bodes very badly... */ -if (dma !=__arm_lpae_dma_addr (dev, pages)) +if (dma !=virt_to_phys (pages))Could I ask why not use
__arm_lpae_dma_addr(pages)here? dma isdma_addr_t.
Quoted 9 lines
Specifically, the problem case for that is when
phys_addr_tis 64-bit butdma_addr_tis 32-bit. The cast in__arm_lpae_dma_addris necessary to avoid a truncation warning when we make the DMA API calls, but we actually need the opposite in the comparison here - comparing the different types directly allows integer promotion to kick in appropriately so we don't lose the top half of the larger address. Otherwise, you'd never spot the difference between, say, your original page at0x88c0000000and a bounce-buffered copy that happened to end up mapped to0xc0000000.
Thanks.
About here:
Review arm_lpae_alloc_pgtable(struct io_pgtable_cfg
Quoted 1 line
(quoted diff) +0
-0@@ -629,6 +626,11 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg
*cfg)
drivers/iommu/io-pgtable-arm.c arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg) · line 629
Quoted 4 lines
drivers/iommu/io-pgtable-arm.c +1
-0if (cfg->oas >ARM_LPAE_MAX_ADDR_BITS )return NULL ; +if (cfg->iommu_dev->dma_pfn_offset) {
Do we need change to :
if (!selftest_running && cfg->iommu_dev->dma_pfn_offset) {cfg->iommu_dev will be null while self test.
Review arm_lpae_alloc_pgtable(struct io_pgtable_cfg · line 630
Quoted 1 line
(quoted diff) +1
-0+ dev_err(cfg->iommu_dev, "Cannot accommodate DMA offset
for IOMMU page tables\n");
Quoted 10 lines
drivers/iommu/io-pgtable-arm.c +3
-0+return NULL ; + } + data =kmalloc (sizeof (*data),GFP_KERNEL );if (!data)return NULL ;Robin.
Robin Murphy
reviewerPATCH v2 1/1 iommu/io-pgtable-arm: Don't use dma_to_phys()
On 22/09/15 14:25, Yong Wu wrote:
[...]
Quoted 6 lines
About here:
Review arm_lpae_alloc_pgtable(struct io_pgtable_cfg
Quoted 1 line
(quoted diff) +0
-0@@ -629,6 +626,11 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg*cfg)
Quoted 3 lines
(quoted diff) +0
-0if (cfg->oas > ARM_LPAE_MAX_ADDR_BITS) return NULL;
Quoted 6 lines
Review arm_lpae_alloc_pgtable(struct io_pgtable_cfg · line 629
Quoted 1 line
(quoted diff) +1
-0+ if (cfg->iommu_dev->dma_pfn_offset) {Do we need change to :
if (!selftest_running && cfg->iommu_dev->dma_pfn_offset) {
cfg->iommu_devwill be null while self test.
Urgh, you're absolutely right. Must have been one of those days when I had loads of noisy debug in there and turned the self-tests off :(
Will; since the branch hasn't gone anywhere yet, are you OK to take the below?
Robin.
----->8-----
From: Robin Murphy <[email protected]>
Date: Tue, 22 Sep 2015 14:52:27 +0100
Subject: [PATCH] fixup! iommu/io-pgtable-arm: Don't use dma_to_phys()
--- drivers/iommu/io-pgtable-arm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iommu/io-pgtable-arm.c +1 -1
@@ -628,7 +628,7 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg)if (cfg->oas >ARM_LPAE_MAX_ADDR_BITS )return NULL ; -if (cfg->iommu_dev->dma_pfn_offset) { +if (!selftest_running && cfg->iommu_dev->dma_pfn_offset) {dev_err (cfg->iommu_dev,"Cannot accommodate DMA offset for IOMMU page tables\n" );return NULL ; }
PATCH v2 1/1 iommu/io-pgtable-arm: Don't use dma_to_phys()
Quoted 13 lines · Robin Murphy
On 22/09/15 14:25, Yong Wu wrote:
[...]Quoted 11 lines
About here:
Review arm_lpae_alloc_pgtable(struct io_pgtable_cfg
Quoted 1 line
(quoted diff) +0
-0@@ -629,6 +626,11 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg*cfg)
drivers/iommu/io-pgtable-arm.c arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg)
Quoted 4 lines
drivers/iommu/io-pgtable-arm.c +1
-0if (cfg->oas >ARM_LPAE_MAX_ADDR_BITS )return NULL ; +if (cfg->iommu_dev->dma_pfn_offset) {Do we need change to :
if (!selftest_running && cfg->iommu_dev->dma_pfn_offset) {
Quoted 6 lines
Quoted 1 line
cfg->iommu_devwill be null while self test.Urgh, you're absolutely right. Must have been one of those days when I had loads of noisy debug in there and turned the self-tests off :(
Will; since the branch hasn't gone anywhere yet, are you OK to take the below?
Yeah, no problem. I'll fold it in when I get a chance.
Will