intel-iommu: Fix domain_ids exhaustion
v2 cover intel-iommu: Fix domain_ids exhaustion
David,
Please comment or apply. Thanks,
Alex
When we unbind a device from a driver, we don't properly unlink the domain from the iommu, so we never free the domain id it was using. We're typically limited to something like 256 domain ids, so a loop of unbinding and rebinding a device can exhaust this pretty quickly. If we're assigning the device to a KVM guest, libvirt does exactly this each time the device is removed from the host driver or added back. When we do run out, we oops the kernel. Fix these.
v2:
We only want to call domain_exit() for domains automatically created via the dma ops path. VM and SI domains have their own life cycle and should not be destroyed here. With v1, if a device was unbound from pci-stub while assigned to a VM, the kernel would oops on the next call into iommu ops.
---
Alex Williamson (2):
intel-iommu: Fix get_domain_for_dev() error path
intel-iommu: Unlink domain from iommu
drivers/pci/intel-iommu.c | 15 +++++++++++++-- 1 files changed, 13 insertions(+), 2 deletions(-)
PATCH RESEND v2 1/2 intel-iommu: Unlink domain from iommu
When we remove a device, we unlink the iommu from the domain, but we never do the reverse unlinking of the domain from the iommu. This means that we never clear iommu->domain_ids, eventually leading to resource exhaustion if we repeatedly bind and unbind a device to a driver. Also free empty domains to avoid a resource leak.
---
drivers/pci/intel-iommu.c | 13 ++++++++++++- 1 files changed, 12 insertions(+), 1 deletions(-)
drivers/pci/intel-iommu.c +12 -1
@@ -3260,9 +3260,15 @@ static int device_notifier(struct notifier_block *nb,if (!domain)return 0 ; -if (action ==BUS_NOTIFY_UNBOUND_DRIVER && !iommu_pass_through) +if (action ==BUS_NOTIFY_UNBOUND_DRIVER && !iommu_pass_through) {domain_remove_one_dev_info (domain, pdev); +if (!(domain->flags &DOMAIN_FLAG_VIRTUAL_MACHINE ) && + !(domain->flags &DOMAIN_FLAG_STATIC_IDENTITY ) && +list_empty (&domain->devices)) +domain_exit (domain); + } +return 0 ; }
@@ -3411,6 +3417,11 @@ static void domain_remove_one_dev_info(struct dmar_domain *domain, domain->iommu_count--;domain_update_iommu_cap (domain);spin_unlock_irqrestore (&domain->iommu_lock, tmp_flags); + +spin_lock_irqsave (&iommu->lock, tmp_flags); +clear_bit (domain->id, iommu->domain_ids); + iommu->domains[domain->id] =NULL ; +spin_unlock_irqrestore (&iommu->lock, tmp_flags); }spin_unlock_irqrestore (&device_domain_lock, flags);
PATCH RESEND v2 2/2 intel-iommu: Fix get_domain_for_dev() error path
In reply to Alex WilliamsonIf we run out of domain_ids and fail iommu_attach_domain(), we fall into domain_exit() without having setup enough of the domain structure for this to do anything useful. In fact, it typically runs off into the weeds walking the bogus domain->devices list. Just free the domain.
---
drivers/pci/intel-iommu.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
drivers/pci/intel-iommu.c +1 -1
@@ -1835,7 +1835,7 @@ static struct dmar_domain *get_domain_for_dev(struct pci_dev *pdev, int gaw) ret =iommu_attach_domain (domain, iommu);if (ret) { -domain_exit (domain); +free_domain_mem (domain); goto error; }
Re: [RESEND PATCH v2 2/2] intel-iommu: Fix get_domain_for_dev() error path
Quoted 19 lines · Alex Williamson
If we run out of
domain_idsand failiommu_attach_domain(), we fall intodomain_exit()without having setup enough of the domain structure for this to do anything useful. In fact, it typically runs off into the weeds walking the bogusdomain->deviceslist. Just free the domain.Signed-off-by Alex Williamson <[email protected]>Acked-by Donald Dutile <[email protected]>---
drivers/pci/intel-iommu.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)drivers/pci/intel-iommu.c +0
-0
drivers/pci/intel-iommu.c static struct dmar_domain *get_domain_for_dev(struct pci_dev *pdev, int gaw) · line 1841
Quoted 8 lines
drivers/pci/intel-iommu.c +1
-1@@ -1835,7 +1835,7 @@ static struct dmar_domain *get_domain_for_dev(struct pci_dev *pdev, int gaw) ret = iommu_attach_domain(domain, iommu); if (ret) { - domain_exit(domain); + free_domain_mem(domain); goto error; }
can you fix another path like David pointed out last year?
https://patchwork.kernel.org/patch/91732/
Thanks
Yinghai