Skip to contentSkip to conversation

iommu: amd: Fix IOMMU page flush when detach device from a domain

PATCH Applied Suthikulpanit, Suravee 2 participants 4 messages Jan 24, 2019 IOMMU

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 iommu: amd: Fix IOMMU page flush when detach device from a domain

From: Suravee Suthikulpanit <[email protected]>

When a VM is terminated, the VFIO driver detaches all pass-through devices from VFIO domain by clearing domain id and page table root pointer from each device table entry (DTE), and then invalidates the DTE. Then, the VFIO driver unmap pages and invalidate IOMMU pages.

Currently, the IOMMU driver keeps track of which IOMMU and how many devices are attached to the domain. When invalidate IOMMU pages, the driver checks if the IOMMU is still attached to the domain before issuing the invalidate page command.

However, since VFIO has already detached all devices from the domain, the subsequent INVALIDATE_IOMMU_PAGES commands are being skipped as there is no IOMMU attached to the domain. This results in data corruption and could cause the PCI device to end up in indeterministic state.

Fix this by invalidate IOMMU pages when detach a device, and before decrementing the per-domain device reference counts.

---
 drivers/iommu/amd_iommu.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)
drivers/iommu/amd_iommu.c +11 -4
index 87ba23a75b38..6cd4a00036c1 100644
@@ -1991,16 +1991,13 @@ static void do_attach(struct iommu_dev_data *dev_data,
 
 static void do_detach(struct iommu_dev_data *dev_data)
 {
+	struct protection_domain *domain = dev_data->domain;
 	struct amd_iommu *iommu;
 	u16 alias;
 
 	iommu = amd_iommu_rlookup_table[dev_data->devid];
 	alias = dev_data->alias;
 
-	/* decrease reference counters */
-	dev_data->domain->dev_iommu[iommu->index] -= 1;
-	dev_data->domain->dev_cnt                 -= 1;
-
 	/* Update data structures */
 	dev_data->domain = NULL;
 	list_del(&dev_data->list);
@@ -2010,6 +2007,16 @@ static void do_detach(struct iommu_dev_data *dev_data)
 
 	/* Flush the DTE entry */
 	device_flush_dte(dev_data);
+
+	/* Flush IOTLB */
+	domain_flush_tlb_pde(domain);
+
+	/* Wait for the flushes to finish */
+	domain_flush_complete(domain);
+
+	/* decrease reference counters - needs to happen after the flushes */
+	domain->dev_iommu[iommu->index] -= 1;
+	domain->dev_cnt                 -= 1;
 }
 
 /*
Signature · 1 line
2.17.1
maintainer Applied
#1 lore raw
Quoted 2 lines · Suravee
 drivers/iommu/amd_iommu.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

Applied, thanks Suravee.

#2 lore raw

Joerg,

Quoted 6 lines · On 1/24/19 9:11 PM, [email protected] wrote:
Quoted 2 lines · Suravee
  drivers/iommu/amd_iommu.c | 15 +++++++++++----
  1 file changed, 11 insertions(+), 4 deletions(-)

Applied, thanks Suravee.

Thanks. Also, should this also back-ported to stable tree as well?

Suravee

#3 lore raw
Quoted 9 lines · Suravee
Quoted 6 lines · On 1/24/19 9:11 PM, [email protected] wrote:
Quoted 2 lines · Suravee
  drivers/iommu/amd_iommu.c | 15 +++++++++++----
  1 file changed, 11 insertions(+), 4 deletions(-)

Applied, thanks Suravee.

Thanks. Also, should this also back-ported to stable tree as well?

I added a Fixes tag, so stable should pick it up.

Joerg