Skip to contentSkip to conversation

kvm/vfio: ensure kvg instance stays around in kvm_vfio_group_add()

PATCH Superseded Dmitry Torokhov 4 participants 6 messages Jul 10, 2023 – Jul 31, 2023 KVM

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 kvm/vfio: ensure kvg instance stays around in kvm_vfio_group_add()

kvm_vfio_group_add() creates kvg instance, links it to kv->group_list, and calls kvm_vfio_file_set_kvm() with kvg->file as an argument after dropping kv->lock. If we race group addition and deletion calls, kvg instance may get freed by the time we get around to calling kvm_vfio_file_set_kvm().

Fix this by moving call to kvm_vfio_file_set_kvm() under the protection of kv->lock. We already call it while holding the same lock when vfio group is being deleted, so it should be safe here as well.

Fixes ba70a89f3c2a ("vfio: Change vfio_group_set_kvm() to vfio_file_set_kvm()")
---
 virt/kvm/vfio.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
virt/kvm/vfio.c +1 -1
index 9584eb57e0ed..cd46d7ef98d6 100644
@@ -179,10 +179,10 @@ static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd)
 	list_add_tail(&kvg->node, &kv->group_list);
 
 	kvm_arch_start_assignment(dev->kvm);
+	kvm_vfio_file_set_kvm(kvg->file, dev->kvm);
 
 	mutex_unlock(&kv->lock);
 
-	kvm_vfio_file_set_kvm(kvg->file, dev->kvm);
 	kvm_vfio_update_coherency(dev);
 
 	return 0;
Signature · 5 lines
2.41.0.255.g8b1d071c50-goog


-- 
Dmitry
#1 lore raw
Quoted 23 lines · On Mon, 10 Jul 2023 15:20:31 -0700 Dmitry Torokhov <[email protected]>…

kvm_vfio_group_add() creates kvg instance, links it to kv->group_list, and calls kvm_vfio_file_set_kvm() with kvg->file as an argument after dropping kv->lock. If we race group addition and deletion calls, kvg instance may get freed by the time we get around to calling kvm_vfio_file_set_kvm().

Fix this by moving call to kvm_vfio_file_set_kvm() under the protection of kv->lock. We already call it while holding the same lock when vfio group is being deleted, so it should be safe here as well.

Fixes ba70a89f3c2a ("vfio: Change vfio_group_set_kvm() to vfio_file_set_kvm()")
---
 virt/kvm/vfio.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
virt/kvm/vfio.c +0 -0
index 9584eb57e0ed..cd46d7ef98d6 100644
@@ -179,10 +179,10 @@ static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd)
 	list_add_tail(&kvg->node, &kv->group_list);

virt/kvm/vfio.c static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd) · line 188

Quoted 10 lines
virt/kvm/vfio.c +1 -1
 
 	kvm_arch_start_assignment(dev->kvm);
+	kvm_vfio_file_set_kvm(kvg->file, dev->kvm);
 
 	mutex_unlock(&kv->lock);
 
-	kvm_vfio_file_set_kvm(kvg->file, dev->kvm);
 	kvm_vfio_update_coherency(dev);
 
 	return 0;

I'm not sure this hasn't been an issue since it was originally introduced in 2fc1bec15883 ("kvm: set/clear kvm to/from vfio_group when group add/delete").

The change added by the blamed ba70a89f3c2a in this respect is simply that we get the file pointer from the mutex protected object, but that mutex protected object is also what maintains that the file pointer is valid. The vfio_group implementation suffered the same issue, the delete path could put the group reference, which could theoretically cause a use after free of the vfio_group.

We could effectively restore the pre-ba70a89f3c2a behavior by replacing kvg->file with filp here, but that would still leave us vulnerable to the original issue.

Note also that kvm_vfio_update_coherency() takes the same mutex separately, I wonder if it wouldn't make more sense if it were moved under the caller's mutex to avoid bouncing the lock and unnecessarily taking it in the release path. Thanks,

Alex

#2 lore raw

Hi Alex,

Quoted 42 lines · Alex Williamson
Quoted 23 lines · On Mon, 10 Jul 2023 15:20:31 -0700 Dmitry Torokhov <[email protected]>…

kvm_vfio_group_add() creates kvg instance, links it to kv->group_list, and calls kvm_vfio_file_set_kvm() with kvg->file as an argument after dropping kv->lock. If we race group addition and deletion calls, kvg instance may get freed by the time we get around to calling kvm_vfio_file_set_kvm().

Fix this by moving call to kvm_vfio_file_set_kvm() under the protection of kv->lock. We already call it while holding the same lock when vfio group is being deleted, so it should be safe here as well.

Fixes ba70a89f3c2a ("vfio: Change vfio_group_set_kvm() to vfio_file_set_kvm()")
---
 virt/kvm/vfio.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
virt/kvm/vfio.c +0 -0
index 9584eb57e0ed..cd46d7ef98d6 100644
@@ -179,10 +179,10 @@ static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd)
 	list_add_tail(&kvg->node, &kv->group_list);

virt/kvm/vfio.c static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd) · line 188

Quoted 10 lines
virt/kvm/vfio.c +1 -1
 
 	kvm_arch_start_assignment(dev->kvm);
+	kvm_vfio_file_set_kvm(kvg->file, dev->kvm);
 
 	mutex_unlock(&kv->lock);
 
-	kvm_vfio_file_set_kvm(kvg->file, dev->kvm);
 	kvm_vfio_update_coherency(dev);
 
 	return 0;

I'm not sure this hasn't been an issue since it was originally introduced in 2fc1bec15883 ("kvm: set/clear kvm to/from vfio_group when group add/delete").

Quoted 6 lines

The change added by the blamed ba70a89f3c2a in this respect is simply that we get the file pointer from the mutex protected object, but that mutex protected object is also what maintains that the file pointer is valid. The vfio_group implementation suffered the same issue, the delete path could put the group reference, which could theoretically cause a use after free of the vfio_group.

Yes, you are right, I'll update the patch with the correct "Fixes".

Quoted 9 lines

We could effectively restore the pre-ba70a89f3c2a behavior by replacing kvg->file with filp here, but that would still leave us vulnerable to the original issue.

Note also that kvm_vfio_update_coherency() takes the same mutex separately, I wonder if it wouldn't make more sense if it were moved under the caller's mutex to avoid bouncing the lock and unnecessarily taking it in the release path. Thanks,

I think I will make it a separate patch.

Thanks.

Signature · 1 line
Dmitry
Reviewed-by on v1 1/1
#3 lore raw
In reply to Dmitry Torokhov
Quoted 10 lines · Dmitry Torokhov

kvm_vfio_group_add() creates kvg instance, links it to kv->group_list, and calls kvm_vfio_file_set_kvm() with kvg->file as an argument after dropping kv->lock. If we race group addition and deletion calls, kvg instance may get freed by the time we get around to calling kvm_vfio_file_set_kvm().

Fix this by moving call to kvm_vfio_file_set_kvm() under the protection of kv->lock. We already call it while holding the same lock when vfio group is being deleted, so it should be safe here as well.

Quoted 6 lines
Fixes ba70a89f3c2a ("vfio: Change vfio_group_set_kvm() to vfio_file_set_kvm()")
---
 virt/kvm/vfio.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

This looks correct, I don't know of any lock cylces that could form with kv->lock at least

Jason

#4 lore raw
In reply to Dmitry Torokhov
Quoted 35 lines · Dmitry Torokhov

kvm_vfio_group_add() creates kvg instance, links it to kv->group_list, and calls kvm_vfio_file_set_kvm() with kvg->file as an argument after dropping kv->lock. If we race group addition and deletion calls, kvg instance may get freed by the time we get around to calling kvm_vfio_file_set_kvm().

Fix this by moving call to kvm_vfio_file_set_kvm() under the protection of kv->lock. We already call it while holding the same lock when vfio group is being deleted, so it should be safe here as well.

Fixes ba70a89f3c2a ("vfio: Change vfio_group_set_kvm() to vfio_file_set_kvm()")
---
 virt/kvm/vfio.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
virt/kvm/vfio.c +1 -1
index 9584eb57e0ed..cd46d7ef98d6 100644
@@ -179,10 +179,10 @@ static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd)
 	list_add_tail(&kvg->node, &kv->group_list);
 
 	kvm_arch_start_assignment(dev->kvm);
+	kvm_vfio_file_set_kvm(kvg->file, dev->kvm);
 
 	mutex_unlock(&kv->lock);
 
-	kvm_vfio_file_set_kvm(kvg->file, dev->kvm);
 	kvm_vfio_update_coherency(dev);
 
 	return 0;
Signature · 1 line
2.41.0.255.g8b1d071c50-goog

What ever happened to this change? Did it end up in a KVM tree somewhere?

thanks,

greg k-h

#5 lore raw
Quoted 33 lines · Greg KH
Quoted 32 lines · Dmitry Torokhov

kvm_vfio_group_add() creates kvg instance, links it to kv->group_list, and calls kvm_vfio_file_set_kvm() with kvg->file as an argument after dropping kv->lock. If we race group addition and deletion calls, kvg instance may get freed by the time we get around to calling kvm_vfio_file_set_kvm().

Fix this by moving call to kvm_vfio_file_set_kvm() under the protection of kv->lock. We already call it while holding the same lock when vfio group is being deleted, so it should be safe here as well.

Fixes ba70a89f3c2a ("vfio: Change vfio_group_set_kvm() to vfio_file_set_kvm()")
---
 virt/kvm/vfio.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
virt/kvm/vfio.c +1 -1
index 9584eb57e0ed..cd46d7ef98d6 100644
@@ -179,10 +179,10 @@ static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd)
 	list_add_tail(&kvg->node, &kv->group_list);
 
 	kvm_arch_start_assignment(dev->kvm);
+	kvm_vfio_file_set_kvm(kvg->file, dev->kvm);
 
 	mutex_unlock(&kv->lock);
 
-	kvm_vfio_file_set_kvm(kvg->file, dev->kvm);
 	kvm_vfio_update_coherency(dev);
 
Quoted 6 lines
Quoted 3 lines
virt/kvm/vfio.c +0 -0
 	return 0;
Signature · 1 line
2.41.0.255.g8b1d071c50-goog

What ever happened to this change? Did it end up in a KVM tree somewhere?

It was posted as:

https://lore.kernel.org/all/[email protected]/

and I believe Alex Williamson is planning to take it through VFIO tree.

Thanks.

Signature · 1 line
Dmitry