Skip to contentSkip to conversation

iommu/hyperv: Don't free a paging domain the hypervisor did not delete

PATCH In review Yuho Choi 4 participants 7 messages Nov 12 – Nov 16

Subsystems (3)
lore
Share & tools
lore

Yuho Choi

author
#0 lore raw

v2 cover iommu/hyperv: Don't free a paging domain the hypervisor did not delete

The Hyper-V IOMMU driver frees a paging domain's page tables even when HVCALL_DELETE_DEVICE_DOMAIN fails and the hypervisor still has the domain. Patch 1 fixes this when a domain is freed, and patch 2 when the allocation of a new domain fails after the domain has been created.

Changes since v1:
- Patch 1: WARN when HVCALL_DELETE_DEVICE_DOMAIN fails (Jason). Kept the Reviewed-by tags from Easwar and Jason. Rebased on current mainline (067c5c24cef2).
- Patch 2: new. Handle the same failure in the error path of hv_iommu_domain_alloc_paging() (Jacob).

Both are compile-tested only (x86_64 with HYPERV and HYPERV_PVIOMMU, W=1). I have no Hyper-V guest with the para-virtualized IOMMU to test them on.

v1: https://lore.kernel.org/all/[email protected]/

Yuho Choi (2):
iommu/hyperv: Don't free a paging domain the hypervisor did not delete iommu/hyperv: Keep a new domain's page tables if it cannot be deleted

 drivers/iommu/hyperv/hv-iommu-guest.c | 29 +++++++++++++++++++++------
 1 file changed, 23 insertions(+), 6 deletions(-)

base-commit: 067c5c24cef23d913b67eaaf71ae67e7169a6620

Signature · 1 line
2.43.0
author
#1 lore raw

PATCH v2 1/2 iommu/hyperv: Don't free a paging domain the hypervisor did not delete

hv_iommu_paging_domain_free() frees all page-table pages with pt_iommu_deinit() and only then asks the hypervisor to delete the device domain that points at them. If HVCALL_DELETE_DEVICE_DOMAIN fails, hv_delete_device_domain() only logs the error, returns the domain ID to the IDA, and the domain is freed anyway.

The hypervisor then keeps a device domain whose page tables have been freed and can be reused for anything. The ID is also handed out again by the next paging domain allocation, while the hypervisor still has a domain with that ID.

Delete the domain before freeing its page tables, and keep the ID, the page tables and the domain if the hypervisor fails to delete it. WARN when the delete fails, as the domain is then leaked.

Fixes 2690d4156b51 ("iommu/hyperv: Add para-virtualized IOMMU support for Hyper-V guest")
---
 drivers/iommu/hyperv/hv-iommu-guest.c | 19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)
drivers/iommu/hyperv/hv-iommu-guest.c +14 -5
index 9f4fa4315091..aa1db9db1e97 100644
@@ -107,7 +107,7 @@ static int hv_create_device_domain(struct hv_iommu_domain *hv_domain)
 	return hv_result_to_errno(status);
 }
 
-static void hv_delete_device_domain(struct hv_iommu_domain *hv_domain)
+static int hv_delete_device_domain(struct hv_iommu_domain *hv_domain)
 {
 	u64 status;
 	unsigned long flags;
@@ -122,11 +122,15 @@ static void hv_delete_device_domain(struct hv_iommu_domain *hv_domain)
 
 	local_irq_restore(flags);
 
-	if (!hv_result_success(status))
-		hv_status_err(status, "HVCALL_DELETE_DEVICE_DOMAIN failed\n");
+	/* If the domain still exists, its ID cannot be reused */
+	if (WARN(!hv_result_success(status),
+		 "HVCALL_DELETE_DEVICE_DOMAIN failed: %#llx (%s)\n",
+		 status, hv_result_to_string(status)))
+		return hv_result_to_errno(status);
 
 	ida_free(&hv_domain->hv_iommu->domain_ids,
 		 hv_domain->device_domain.domain_id.id);
+	return 0;
 }
 
 static int
@@ -466,11 +470,16 @@ static void hv_iommu_paging_domain_free(struct iommu_domain *domain)
 {
 	struct hv_iommu_domain *hv_domain = to_hv_iommu_domain(domain);
 
+	/*
+	 * If the hypervisor still has the domain, it may still use its page
+	 * tables, so leak them.
+	 */
+	if (hv_delete_device_domain(hv_domain))
+		return;
+
 	/* Free all remaining mappings */
 	pt_iommu_deinit(&hv_domain->pt_iommu);
 
-	hv_delete_device_domain(hv_domain);
-
 	kfree(hv_domain);
 }
 
Signature · 1 line
2.43.0
Reviewed-by on v2 1/2
#2 lore raw
Quoted 76 lines · Yuho Choi

hv_iommu_paging_domain_free() frees all page-table pages with pt_iommu_deinit() and only then asks the hypervisor to delete the device domain that points at them. If HVCALL_DELETE_DEVICE_DOMAIN fails, hv_delete_device_domain() only logs the error, returns the domain ID to the IDA, and the domain is freed anyway.

The hypervisor then keeps a device domain whose page tables have been freed and can be reused for anything. The ID is also handed out again by the next paging domain allocation, while the hypervisor still has a domain with that ID.

Delete the domain before freeing its page tables, and keep the ID, the page tables and the domain if the hypervisor fails to delete it. WARN when the delete fails, as the domain is then leaked.

Fixes 2690d4156b51 ("iommu/hyperv: Add para-virtualized IOMMU support for Hyper-V guest")
---
 drivers/iommu/hyperv/hv-iommu-guest.c | 19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)
drivers/iommu/hyperv/hv-iommu-guest.c +14 -5
index 9f4fa4315091..aa1db9db1e97 100644
@@ -107,7 +107,7 @@ static int hv_create_device_domain(struct hv_iommu_domain *hv_domain)
 	return hv_result_to_errno(status);
 }
 
-static void hv_delete_device_domain(struct hv_iommu_domain *hv_domain)
+static int hv_delete_device_domain(struct hv_iommu_domain *hv_domain)
 {
 	u64 status;
 	unsigned long flags;
@@ -122,11 +122,15 @@ static void hv_delete_device_domain(struct hv_iommu_domain *hv_domain)
 
 	local_irq_restore(flags);
 
-	if (!hv_result_success(status))
-		hv_status_err(status, "HVCALL_DELETE_DEVICE_DOMAIN failed\n");
+	/* If the domain still exists, its ID cannot be reused */
+	if (WARN(!hv_result_success(status),
+		 "HVCALL_DELETE_DEVICE_DOMAIN failed: %#llx (%s)\n",
+		 status, hv_result_to_string(status)))
+		return hv_result_to_errno(status);
 
 	ida_free(&hv_domain->hv_iommu->domain_ids,
 		 hv_domain->device_domain.domain_id.id);
+	return 0;
 }
 
 static int
@@ -466,11 +470,16 @@ static void hv_iommu_paging_domain_free(struct iommu_domain *domain)
 {
 	struct hv_iommu_domain *hv_domain = to_hv_iommu_domain(domain);
 
+	/*
+	 * If the hypervisor still has the domain, it may still use its page
+	 * tables, so leak them.
+	 */
+	if (hv_delete_device_domain(hv_domain))
+		return;
+
 	/* Free all remaining mappings */
 	pt_iommu_deinit(&hv_domain->pt_iommu);
 
-	hv_delete_device_domain(hv_domain);
-
 	kfree(hv_domain);
 }
 
Signature · 1 line
2.43.0

B.R.
Yu

author
#3 lore raw

PATCH v2 2/2 iommu/hyperv: Keep a new domain's page tables if it cannot be deleted

In reply to Yuho Choi

When hv_iommu_domain_alloc_paging() fails to configure a new device domain, hv_create_configure_device_domain() deletes the domain again, and the caller then frees the page tables with pt_iommu_deinit(). If the delete fails as well, the hypervisor still has the domain, which may already point at the page tables, and they are freed anyway.

Create and configure the domain in hv_iommu_domain_alloc_paging() itself, and leak the page tables and the domain when the delete fails, as hv_iommu_paging_domain_free() does.

Fixes 2690d4156b51 ("iommu/hyperv: Add para-virtualized IOMMU support for Hyper-V guest")
---
 drivers/iommu/hyperv/hv-iommu-guest.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
drivers/iommu/hyperv/hv-iommu-guest.c +9 -1
index aa1db9db1e97..ef486ffa0be4 100644
@@ -540,10 +540,18 @@ static struct iommu_domain *hv_iommu_domain_alloc_paging(struct device *dev)
 	settings.page_table_root = pt_info.gcr3_pt;
 	settings.flags.first_stage_paging_mode = pt_info.levels == 5;
 
-	ret = hv_create_configure_device_domain(hv_domain, &settings);
+	ret = hv_create_device_domain(hv_domain);
 	if (ret)
 		goto err_pt_deinit;
 
+	ret = hv_configure_device_domain(hv_domain, &settings);
+	if (ret) {
+		/* See hv_iommu_paging_domain_free() */
+		if (hv_delete_device_domain(hv_domain))
+			return ERR_PTR(ret);
+		goto err_pt_deinit;
+	}
+
 	return &hv_domain->domain;
 
 err_pt_deinit:
Signature · 1 line
2.43.0
Reviewed-by on v2 2/2
#4 lore raw

Re: [PATCH v2 2/2] iommu/hyperv: Keep a new domain's page tables if it cannot be deleted

Quoted 10 lines · Yuho Choi

When hv_iommu_domain_alloc_paging() fails to configure a new device domain, hv_create_configure_device_domain() deletes the domain again, and the caller then frees the page tables with pt_iommu_deinit(). If the delete fails as well, the hypervisor still has the domain, which may already point at the page tables, and they are freed anyway.

Create and configure the domain in hv_iommu_domain_alloc_paging() itself, and leak the page tables and the domain when the delete fails, as hv_iommu_paging_domain_free() does.

Quoted 7 lines
Fixes 2690d4156b51 ("iommu/hyperv: Add para-virtualized IOMMU support for Hyper-V guest")
---
 drivers/iommu/hyperv/hv-iommu-guest.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

Jason

#5 lore raw

Re: [PATCH v2 2/2] iommu/hyperv: Keep a new domain's page tables if it cannot be deleted

In reply to Yuho Choi
Quoted 10 lines · Yuho Choi

When hv_iommu_domain_alloc_paging() fails to configure a new device domain, hv_create_configure_device_domain() deletes the domain again, and the caller then frees the page tables with pt_iommu_deinit(). If the delete fails as well, the hypervisor still has the domain, which may already point at the page tables, and they are freed anyway.

Create and configure the domain in hv_iommu_domain_alloc_paging() itself, and leak the page tables and the domain when the delete fails, as hv_iommu_paging_domain_free() does.

My understanding is that, for the newly created domain, current Hyper-V implementation does not adopt the supplied stage-1 page table root if HVCALL_CONFIGURE_DEVICE_DOMAIN fails. Therefore, a failed configuration does not leave these page tables referenced. It is different than the hv_iommu_paging_domain_free() case.

But I'm OK with keeping this as defensive error handling. Could you update the commit message(the comment above hv_delete_device_domain() as well) to distinguish that motivation from the behavior of the current implementation?

B.R.
Yu

Quoted 35 lines
Fixes 2690d4156b51 ("iommu/hyperv: Add para-virtualized IOMMU support for Hyper-V guest")
---
 drivers/iommu/hyperv/hv-iommu-guest.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
drivers/iommu/hyperv/hv-iommu-guest.c +9 -1
index aa1db9db1e97..ef486ffa0be4 100644
@@ -540,10 +540,18 @@ static struct iommu_domain *hv_iommu_domain_alloc_paging(struct device *dev)
 	settings.page_table_root = pt_info.gcr3_pt;
 	settings.flags.first_stage_paging_mode = pt_info.levels == 5;
 
-	ret = hv_create_configure_device_domain(hv_domain, &settings);
+	ret = hv_create_device_domain(hv_domain);
 	if (ret)
 		goto err_pt_deinit;
 
+	ret = hv_configure_device_domain(hv_domain, &settings);
+	if (ret) {
+		/* See hv_iommu_paging_domain_free() */
+		if (hv_delete_device_domain(hv_domain))
+			return ERR_PTR(ret);
+		goto err_pt_deinit;
+	}
+
 	return &hv_domain->domain;
 
 err_pt_deinit:
Signature · 1 line
2.43.0
Reviewed-by on v2 2/2
#6 lore raw

Re: [PATCH v2 2/2] iommu/hyperv: Keep a new domain's page tables if it cannot be deleted

In reply to Yuho Choi
Quoted 32 lines · Yuho Choi

When hv_iommu_domain_alloc_paging() fails to configure a new device domain, hv_create_configure_device_domain() deletes the domain again, and the caller then frees the page tables with pt_iommu_deinit(). If the delete fails as well, the hypervisor still has the domain, which may already point at the page tables, and they are freed anyway.

Create and configure the domain in hv_iommu_domain_alloc_paging() itself, and leak the page tables and the domain when the delete fails, as hv_iommu_paging_domain_free() does.

Fixes 2690d4156b51 ("iommu/hyperv: Add para-virtualized IOMMU support for Hyper-V guest")
---
 drivers/iommu/hyperv/hv-iommu-guest.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
drivers/iommu/hyperv/hv-iommu-guest.c +2 -1
index aa1db9db1e97..ef486ffa0be4 100644
@@ -540,10 +540,18 @@ static struct iommu_domain *hv_iommu_domain_alloc_paging(struct device *dev)
 	settings.page_table_root = pt_info.gcr3_pt;
 	settings.flags.first_stage_paging_mode = pt_info.levels == 5;
 
-	ret = hv_create_configure_device_domain(hv_domain, &settings);
+	ret = hv_create_device_domain(hv_domain);
 	if (ret)
 		goto err_pt_deinit;
 
+	ret = hv_configure_device_domain(hv_domain, &settings);

drivers/iommu/hyperv/hv-iommu-guest.c static struct iommu_domain *hv_iommu_domain_alloc_paging(struct device *dev) · line 557

Quoted 10 lines
drivers/iommu/hyperv/hv-iommu-guest.c +7 -0
+	if (ret) {
+		/* See hv_iommu_paging_domain_free() */
+		if (hv_delete_device_domain(hv_domain))
+			return ERR_PTR(ret);
+		goto err_pt_deinit;
+	}
+
 	return &hv_domain->domain;
 
 err_pt_deinit:

With the commit message and comment fixes as requested by Yu: