Skip to contentSkip to conversation

vhost-vdpa: log warning message if vhost_vdpa_remove gets blocked

PATCH In review Gautam Dawar 5 participants 5 messages Jun 6, 2021 – Jul 3, 2021

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 vhost-vdpa: log warning message if vhost_vdpa_remove gets blocked

From: Gautam Dawar <[email protected]>

If some module invokes vdpa_device_unregister (usually in the module unload function) when the userspace app (eg. QEMU) which had opened the vhost-vdpa character device is still running, vhost_vdpa_remove() function will block indefinitely in call to wait_for_completion().

This causes the vdpa_device_unregister caller to hang and with a usual side-effect of rmmod command not returning when this call is in the module_exit function.

This patch converts the wait_for_completion call to its timeout based counterpart (wait_for_completion_timeout) and also adds a warning message to alert the user/administrator about this hang situation.

To eventually fix this problem, a mechanism will be required to let vhost-vdpa module inform the userspace of this situation and userspace will close the descriptor of vhost-vdpa char device. This will enable vhost-vdpa to continue with graceful clean-up.

---
 drivers/vhost/vdpa.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
drivers/vhost/vdpa.c +5 -1
index bfa4c6ef554e..572b64d09b06 100644
@@ -1091,7 +1091,11 @@ static void vhost_vdpa_remove(struct vdpa_device *vdpa)
 		opened = atomic_cmpxchg(&v->opened, 0, 1);
 		if (!opened)
 			break;
-		wait_for_completion(&v->completion);
+		wait_for_completion_timeout(&v->completion,
+					    msecs_to_jiffies(1000));
+		dev_warn_ratelimited(&v->dev,
+				     "%s waiting for /dev/%s to be closed\n",
+				     __func__, dev_name(&v->dev));
 	} while (1);
 
 	put_device(&v->dev);
Signature · 6 lines
2.30.1

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Acked-by on v1 1/1
#1 lore raw
Quoted 32 lines · Gautam Dawar

From: Gautam Dawar <[email protected]>

If some module invokes vdpa_device_unregister (usually in the module unload function) when the userspace app (eg. QEMU) which had opened the vhost-vdpa character device is still running, vhost_vdpa_remove() function will block indefinitely in call to wait_for_completion().

This causes the vdpa_device_unregister caller to hang and with a usual side-effect of rmmod command not returning when this call is in the module_exit function.

This patch converts the wait_for_completion call to its timeout based counterpart (wait_for_completion_timeout) and also adds a warning message to alert the user/administrator about this hang situation.

To eventually fix this problem, a mechanism will be required to let vhost-vdpa module inform the userspace of this situation and userspace will close the descriptor of vhost-vdpa char device. This will enable vhost-vdpa to continue with graceful clean-up.

---
  drivers/vhost/vdpa.c | 6 +++++-
  1 file changed, 5 insertions(+), 1 deletion(-)
drivers/vhost/vdpa.c +0 -0
index bfa4c6ef554e..572b64d09b06 100644
@@ -1091,7 +1091,11 @@ static void vhost_vdpa_remove(struct vdpa_device *vdpa)
  		opened = atomic_cmpxchg(&v->opened, 0, 1);
  		if (!opened)

drivers/vhost/vdpa.c static void vhost_vdpa_remove(struct vdpa_device *vdpa) · line 1101

Quoted 10 lines
drivers/vhost/vdpa.c +5 -1
  			break;
-		wait_for_completion(&v->completion);
+		wait_for_completion_timeout(&v->completion,
+					    msecs_to_jiffies(1000));
+		dev_warn_ratelimited(&v->dev,
+				     "%s waiting for /dev/%s to be closed\n",
+				     __func__, dev_name(&v->dev));
  	} while (1);
 
  	put_device(&v->dev);
#2 lore raw
Quoted 27 lines · Jason Wang
Quoted 25 lines · Gautam Dawar

From: Gautam Dawar <[email protected]>

If some module invokes vdpa_device_unregister (usually in the module unload function) when the userspace app (eg. QEMU) which had opened the vhost-vdpa character device is still running, vhost_vdpa_remove() function will block indefinitely in call to wait_for_completion().

This causes the vdpa_device_unregister caller to hang and with a usual side-effect of rmmod command not returning when this call is in the module_exit function.

This patch converts the wait_for_completion call to its timeout based counterpart (wait_for_completion_timeout) and also adds a warning message to alert the user/administrator about this hang situation.

To eventually fix this problem, a mechanism will be required to let vhost-vdpa module inform the userspace of this situation and userspace will close the descriptor of vhost-vdpa char device. This will enable vhost-vdpa to continue with graceful clean-up.

---
  drivers/vhost/vdpa.c | 6 +++++-
  1 file changed, 5 insertions(+), 1 deletion(-)
Quoted 14 lines
Quoted 5 lines
drivers/vhost/vdpa.c +0 -0
index bfa4c6ef554e..572b64d09b06 100644

drivers/vhost/vdpa.c static void vhost_vdpa_remove(struct vdpa_device *vdpa) · line 1098

Quoted 9 lines
drivers/vhost/vdpa.c +5 -1
@@ -1091,7 +1091,11 @@ static void vhost_vdpa_remove(struct vdpa_device *vdpa)
  		opened = atomic_cmpxchg(&v->opened, 0, 1);
  		if (!opened)
  			break;
-		wait_for_completion(&v->completion);
+		wait_for_completion_timeout(&v->completion,
+					    msecs_to_jiffies(1000));
+		dev_warn_ratelimited(&v->dev,
+				     "%s waiting for /dev/%s to be closed\n",
+				     __func__, dev_name(&v->dev));

Can fill up the kernel log in this case ... dev_warn_once seems more appropriate.

Quoted 6 lines

drivers/vhost/vdpa.c static void vhost_vdpa_remove(struct vdpa_device *vdpa) · line 1101

Quoted 2 lines
drivers/vhost/vdpa.c +0 -0
  	} while (1);
  	put_device(&v->dev);

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

#3 lore raw

Pls see inline [GD>>]

-----Original Message-----
From: Michael S. Tsirkin <[email protected]>
Sent: Saturday, July 3, 2021 1:42 PM
To: Jason Wang <[email protected]>
Cc: Gautam Dawar <[email protected]>; Martin Petrus Hubertus Habets <[email protected]>; Harpreet Singh Anand <[email protected]>; Gautam Dawar <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH] vhost-vdpa: log warning message if vhost_vdpa_remove gets blocked

Quoted 27 lines · Jason Wang
Quoted 25 lines · Gautam Dawar

From: Gautam Dawar <[email protected]>

If some module invokes vdpa_device_unregister (usually in the module unload function) when the userspace app (eg. QEMU) which had opened
the vhost-vdpa character device is still running,
vhost_vdpa_remove() function will block indefinitely in call to wait_for_completion().

This causes the vdpa_device_unregister caller to hang and with a usual side-effect of rmmod command not returning when this call is in the module_exit function.

This patch converts the wait_for_completion call to its timeout based counterpart (wait_for_completion_timeout) and also adds a warning message to alert the user/administrator about this hang situation.

To eventually fix this problem, a mechanism will be required to let vhost-vdpa module inform the userspace of this situation and userspace will close the descriptor of vhost-vdpa char device. This will enable vhost-vdpa to continue with graceful clean-up.

---
  drivers/vhost/vdpa.c | 6 +++++-
  1 file changed, 5 insertions(+), 1 deletion(-)
Quoted 14 lines
Quoted 5 lines
drivers/vhost/vdpa.c index renamed from drivers/vhost/vdpa.c +0 -0

bfa4c6ef554e..572b64d09b06 100644

drivers/vhost/vdpa.c static void vhost_vdpa_remove(struct vdpa_device *vdpa) · line 1098

Quoted 9 lines
drivers/vhost/vdpa.c +5 -1
@@ -1091,7 +1091,11 @@ static void vhost_vdpa_remove(struct vdpa_device *vdpa)
  		opened = atomic_cmpxchg(&v->opened, 0, 1);
  		if (!opened)
  			break;
-		wait_for_completion(&v->completion);
+		wait_for_completion_timeout(&v->completion,
+					    msecs_to_jiffies(1000));
+		dev_warn_ratelimited(&v->dev,
+				     "%s waiting for /dev/%s to be closed\n",
+				     __func__, dev_name(&v->dev));

Can fill up the kernel log in this case ... dev_warn_once seems more appropriate. [GD>>] Submitted the patch with suggested modification.

Quoted 6 lines

drivers/vhost/vdpa.c static void vhost_vdpa_remove(struct vdpa_device *vdpa) · line 1101

Quoted 2 lines
drivers/vhost/vdpa.c +0 -0
  	} while (1);
  	put_device(&v->dev);
#4 lore raw

PATCH 1/1 vhost-vdpa: log warning message if vhost_vdpa_remove gets blocked

In reply to Gautam Dawar

From: Gautam Dawar <[email protected]>

If some module invokes vdpa_device_unregister (usually in the module unload function) when the userspace app (eg. QEMU) which had opened the vhost-vdpa character device is still running, vhost_vdpa_remove() function will block indefinitely in call to wait_for_completion().

This causes the vdpa_device_unregister caller to hang and with a usual side-effect of rmmod command not returning when this call is in the module_exit function.

This patch converts the wait_for_completion call to its timeout based counterpart (wait_for_completion_timeout) and also adds a warning message to alert the user/administrator about this hang situation.

To eventually fix this problem, a mechanism will be required to let vhost-vdpa module inform the userspace of this situation and userspace will close the descriptor of vhost-vdpa char device. This will enable vhost-vdpa to continue with graceful clean-up.

---
 drivers/vhost/vdpa.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
drivers/vhost/vdpa.c +5 -1
index bfa4c6ef554e..e4b7d26649d8 100644
@@ -1091,7 +1091,11 @@ static void vhost_vdpa_remove(struct vdpa_device *vdpa)
 		opened = atomic_cmpxchg(&v->opened, 0, 1);
 		if (!opened)
 			break;
-		wait_for_completion(&v->completion);
+		wait_for_completion_timeout(&v->completion,
+					    msecs_to_jiffies(1000));
+		dev_warn_once(&v->dev,
+			      "%s waiting for /dev/%s to be closed\n",
+			      __func__, dev_name(&v->dev));
 	} while (1);
 
 	put_device(&v->dev);
Signature · 1 line
2.30.1