iommu/dma: Fix race condition during iova_domain initialization
Subsystems (3)
PATCH 1/1 iommu/dma: Fix race condition during iova_domain initialization
From: Yunfei Wang <[email protected]>
When many devices share the same iova domain, iommu_dma_init_domain() may be called at the same time. The checking of iovad->start_pfn will all get false in iommu_dma_init_domain() and both enter init_iova_domain() to do iovad initialization.
Fix this by protecting init_iova_domain() with iommu_dma_cookie->mutex.
Exception backtrace:rb_insert_color(param1=0xFFFFFF80CD2BDB40, param3=1) + 64 init_iova_domain() + 180iommu_setup_dma_ops() + 260arch_setup_dma_ops() + 132of_dma_configure_id() + 468 platform_dma_configure() + 32really_probe() + 1168driver_probe_device() + 268 __device_attach_driver() + 524__device_attach() + 524bus_probe_device() + 64 deferred_probe_work_func() + 260 process_one_work() + 580worker_thread() + 1076kthread() + 332ret_from_fork() + 16
--- drivers/iommu/dma-iommu.c | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-)
drivers/iommu/dma-iommu.c +13 -4
@@ -63,6 +63,7 @@ struct iommu_dma_cookie {/* Domain for flush queue callback; NULL if flush queue not in use */ struct iommu_domain *fq_domain; +struct mutex mutex; };static DEFINE_STATIC_KEY_FALSE (iommu_deferred_attach_enabled);
@@ -309,6 +310,7 @@ int iommu_get_dma_cookie(struct iommu_domain *domain)if (!domain->iova_cookie)return -ENOMEM ; +mutex_init (&domain->iova_cookie->mutex);return 0 ; }
@@ -549,26 +551,33 @@ static int iommu_dma_init_domain(struct iommu_domain *domain, dma_addr_t base, }/* start_pfn is always nonzero for an already-initialised domain */ +mutex_lock (&cookie->mutex);if (iovad->start_pfn) {if (1UL << order != iovad->granule || base_pfn != iovad->start_pfn) {pr_warn ("Incompatible range for DMA domain\n" ); -return -EFAULT ; + ret = -EFAULT ; + goto done_unlock; } -return 0 ; + ret =0 ; + goto done_unlock; }init_iova_domain (iovad,1UL << order, base_pfn); ret =iova_domain_init_rcaches (iovad);if (ret) -return ret; + goto done_unlock;/* If the FQ fails we can simply fall back to strict mode */ if (domain->type ==IOMMU_DOMAIN_DMA_FQ &&iommu_dma_init_fq (domain)) domain->type =IOMMU_DOMAIN_DMA ; -return iova_reserve_iommu_regions (dev, domain); + ret =iova_reserve_iommu_regions (dev, domain); + +done_unlock: +mutex_unlock (&cookie->mutex); +return ret; }/**
Signature · 6 lines
2.18.0 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Hi YF,
Quoted 4 lines
When many devices share the same iova domain,
iommu_dma_init_domain()may be called at the same time. The checking ofiovad->start_pfnwill all get false iniommu_dma_init_domain()and both enterinit_iova_domain()to do iovad initialization.
After reading this patch.
It means that we use iovad->start_pfn as a key to tell if an iovad is already initialized, but we do not protect iovad->start_pfn from concurrent access.
So what's happening is like:
cpu0 cpu1of_dma_configure_id() of_dma_configure_id()
arch_setup_dma_ops() arch_setup_dma_ops()
iommu_setup_dma_ops() iommu_setup_dma_ops()
init_iova_domain() init_iova_domain()
if (iovad->start_pfn) { if (iovad->start_pfn) {
} }
init_iova_domain() init_iova_domain()init_iova_domain() is called at the same time.
Quoted 1 line
Fix this by protecting
init_iova_domain()withiommu_dma_cookie->mutex.
Quoted 92 lines
Exception backtrace:
rb_insert_color(param1=0xFFFFFF80CD2BDB40, param3=1)+ 64init_iova_domain()+ 180iommu_setup_dma_ops()+ 260arch_setup_dma_ops()+ 132of_dma_configure_id()+ 468platform_dma_configure()+ 32really_probe()+ 1168driver_probe_device()+ 268__device_attach_driver()+ 524__device_attach()+ 524bus_probe_device()+ 64deferred_probe_work_func()+ 260process_one_work()+ 580worker_thread()+ 1076kthread()+ 332ret_from_fork()+ 16Signed-off-by Ning Li <[email protected]>Signed-off-by Yunfei Wang <[email protected]>---
drivers/iommu/dma-iommu.c| 17 +++++++++++++----1 file changed, 13 insertions(+), 4 deletions(-)drivers/iommu/dma-iommu.c +13
-4@@ -63,6 +63,7 @@ struct iommu_dma_cookie {/* Domain for flush queue callback; NULL if flush queue not in use */ struct iommu_domain *fq_domain; +struct mutex mutex; };static DEFINE_STATIC_KEY_FALSE (iommu_deferred_attach_enabled);@@ -309,6 +310,7 @@ int iommu_get_dma_cookie(struct iommu_domain *domain)if (!domain->iova_cookie)return -ENOMEM ; +mutex_init (&domain->iova_cookie->mutex);return 0 ; }@@ -549,26 +551,33 @@ static int iommu_dma_init_domain(struct iommu_domain *domain, dma_addr_t base, }/* start_pfn is always nonzero for an already-initialised domain */ +mutex_lock (&cookie->mutex);if (iovad->start_pfn) {if (1UL << order != iovad->granule || base_pfn != iovad->start_pfn) {pr_warn ("Incompatible range for DMA domain\n" ); -return -EFAULT ; + ret = -EFAULT ; + goto done_unlock; } -return 0 ; + ret =0 ; + goto done_unlock; }init_iova_domain (iovad,1UL << order, base_pfn); ret =iova_domain_init_rcaches (iovad);if (ret) -return ret; + goto done_unlock;/* If the FQ fails we can simply fall back to strict mode */ if (domain->type ==IOMMU_DOMAIN_DMA_FQ &&iommu_dma_init_fq (domain)) domain->type =IOMMU_DOMAIN_DMA ; -return iova_reserve_iommu_regions (dev, domain); + ret =iova_reserve_iommu_regions (dev, domain); + +done_unlock: +mutex_unlock (&cookie->mutex); +return ret; }/**
Signature · 7 lines
2.18.0 _______________________________________________ Linux-mediatek mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-mediatek
Joerg Roedel
maintainerPlease re-send with
Robin Murphy <[email protected]>
in Cc.
Quoted 93 lines · On Mon, May 30, 2022 at 08:07:45PM +0800, [email protected] wrote:
From: Yunfei Wang <[email protected]>
When many devices share the same iova domain,
iommu_dma_init_domain()may be called at the same time. The checking ofiovad->start_pfnwill all get false iniommu_dma_init_domain()and both enterinit_iova_domain()to do iovad initialization.Fix this by protecting
init_iova_domain()withiommu_dma_cookie->mutex.Exception backtrace:
rb_insert_color(param1=0xFFFFFF80CD2BDB40, param3=1)+ 64init_iova_domain()+ 180iommu_setup_dma_ops()+ 260arch_setup_dma_ops()+ 132of_dma_configure_id()+ 468platform_dma_configure()+ 32really_probe()+ 1168driver_probe_device()+ 268__device_attach_driver()+ 524__device_attach()+ 524bus_probe_device()+ 64deferred_probe_work_func()+ 260process_one_work()+ 580worker_thread()+ 1076kthread()+ 332ret_from_fork()+ 16Signed-off-by Ning Li <[email protected]>Signed-off-by Yunfei Wang <[email protected]>--- drivers/iommu/dma-iommu.c | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-)drivers/iommu/dma-iommu.c +13
-4@@ -63,6 +63,7 @@ struct iommu_dma_cookie {/* Domain for flush queue callback; NULL if flush queue not in use */ struct iommu_domain *fq_domain; +struct mutex mutex; };static DEFINE_STATIC_KEY_FALSE (iommu_deferred_attach_enabled);@@ -309,6 +310,7 @@ int iommu_get_dma_cookie(struct iommu_domain *domain)if (!domain->iova_cookie)return -ENOMEM ; +mutex_init (&domain->iova_cookie->mutex);return 0 ; }@@ -549,26 +551,33 @@ static int iommu_dma_init_domain(struct iommu_domain *domain, dma_addr_t base, }/* start_pfn is always nonzero for an already-initialised domain */ +mutex_lock (&cookie->mutex);if (iovad->start_pfn) {if (1UL << order != iovad->granule || base_pfn != iovad->start_pfn) {pr_warn ("Incompatible range for DMA domain\n" ); -return -EFAULT ; + ret = -EFAULT ; + goto done_unlock; } -return 0 ; + ret =0 ; + goto done_unlock; }init_iova_domain (iovad,1UL << order, base_pfn); ret =iova_domain_init_rcaches (iovad);if (ret) -return ret; + goto done_unlock;/* If the FQ fails we can simply fall back to strict mode */ if (domain->type ==IOMMU_DOMAIN_DMA_FQ &&iommu_dma_init_fq (domain)) domain->type =IOMMU_DOMAIN_DMA ; -return iova_reserve_iommu_regions (dev, domain); + ret =iova_reserve_iommu_regions (dev, domain); + +done_unlock: +mutex_unlock (&cookie->mutex); +return ret; }/** Signature · 1 line
2.18.0
Quoted 5 lines · Joerg Roedel
Please re-send with
Robin Murphy <[email protected]>
in Cc.
Apologies, I did spot this before, I've just been tied up with other things and dropping everything non-critical on the floor, so didn't get round to replying before it slipped my mind again.
In summary, I hate it, but mostly because the whole situation of calling iommu_probe_device off the back of driver probe is fundamentally broken. I'm still a few steps away from fixing that properly, at which point I can just as well rip all these little bodges out again. If it really does need mitigating in the meantime (i.e. this is real-world async probe, not just some contrived testcase), then I can't easily think of any cleaner hack, so,
(somewhat reluctantly)
Cheers,
Robin.
Quoted 91 lines · On Mon, May 30, 2022 at 08:07:45PM +0800, [email protected] wrote:
Quoted 90 lines
From: Yunfei Wang <[email protected]>
When many devices share the same iova domain,
iommu_dma_init_domain()may be called at the same time. The checking ofiovad->start_pfnwill all get false iniommu_dma_init_domain()and both enterinit_iova_domain()to do iovad initialization.Fix this by protecting
init_iova_domain()withiommu_dma_cookie->mutex.Exception backtrace:
rb_insert_color(param1=0xFFFFFF80CD2BDB40, param3=1)+ 64init_iova_domain()+ 180iommu_setup_dma_ops()+ 260arch_setup_dma_ops()+ 132of_dma_configure_id()+ 468platform_dma_configure()+ 32really_probe()+ 1168driver_probe_device()+ 268__device_attach_driver()+ 524__device_attach()+ 524bus_probe_device()+ 64deferred_probe_work_func()+ 260process_one_work()+ 580worker_thread()+ 1076kthread()+ 332ret_from_fork()+ 16Signed-off-by Ning Li <[email protected]>Signed-off-by Yunfei Wang <[email protected]>--- drivers/iommu/dma-iommu.c | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-)drivers/iommu/dma-iommu.c +13
-4@@ -63,6 +63,7 @@ struct iommu_dma_cookie {/* Domain for flush queue callback; NULL if flush queue not in use */ struct iommu_domain *fq_domain; +struct mutex mutex; };static DEFINE_STATIC_KEY_FALSE (iommu_deferred_attach_enabled);@@ -309,6 +310,7 @@ int iommu_get_dma_cookie(struct iommu_domain *domain)if (!domain->iova_cookie)return -ENOMEM ; +mutex_init (&domain->iova_cookie->mutex);return 0 ; }@@ -549,26 +551,33 @@ static int iommu_dma_init_domain(struct iommu_domain *domain, dma_addr_t base, }/* start_pfn is always nonzero for an already-initialised domain */ +mutex_lock (&cookie->mutex);if (iovad->start_pfn) {if (1UL << order != iovad->granule || base_pfn != iovad->start_pfn) {pr_warn ("Incompatible range for DMA domain\n" ); -return -EFAULT ; + ret = -EFAULT ; + goto done_unlock; } -return 0 ; + ret =0 ; + goto done_unlock; }init_iova_domain (iovad,1UL << order, base_pfn); ret =iova_domain_init_rcaches (iovad);if (ret) -return ret; + goto done_unlock;/* If the FQ fails we can simply fall back to strict mode */ if (domain->type ==IOMMU_DOMAIN_DMA_FQ &&iommu_dma_init_fq (domain)) domain->type =IOMMU_DOMAIN_DMA ; -return iova_reserve_iommu_regions (dev, domain); + ret =iova_reserve_iommu_regions (dev, domain); + +done_unlock: +mutex_unlock (&cookie->mutex); +return ret; }
Quoted 7 lines
Quoted 3 lines
drivers/iommu/dma-iommu.c +0
-0/** Signature · 1 line
2.18.0_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Joerg Roedel
maintainerQuoted 4 lines · Robin Murphy
Apologies, I did spot this before, I've just been tied up with other things and dropping everything non-critical on the floor, so didn't get round to replying before it slipped my mind again.
Quoted 8 lines
In summary, I hate it, but mostly because the whole situation of calling
iommu_probe_deviceoff the back of driver probe is fundamentally broken. I'm still a few steps away from fixing that properly, at which point I can just as well rip all these little bodges out again. If it really does need mitigating in the meantime (i.e. this is real-world async probe, not just some contrived testcase), then I can't easily think of any cleaner hack, so,Acked-by Robin Murphy <[email protected]>
Alright, applied this now.
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu