Skip to contentSkip to conversation

hw/virtio: Fix the de-initialization of vhost-user devices

PATCH In review Thomas Huth 3 participants 5 messages Jun 18, 2024 – Jul 1, 2024

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 hw/virtio: Fix the de-initialization of vhost-user devices

The unrealize functions of the various vhost-user devices are calling the corresponding vhost_*_set_status() functions with a status of 0 to shut down the device correctly.

Now these vhost_*_set_status() functions all follow this scheme:

    bool should_start = virtio_device_should_start(vdev, status);

    if (vhost_dev_is_started(&vvc->vhost_dev) == should_start) {
        return;
    }

    if (should_start) {
        /* ... do the initialization stuff ... */
    } else {
        /* ... do the cleanup stuff ... */
    }

The problem here is virtio_device_should_start(vdev, 0) currently always returns "true" since it internally only looks at vdev->started instead of looking at the "status" parameter. Thus once the device got started once, virtio_device_should_start() always returns true and thus the vhost_*_set_status() functions return early, without ever doing any clean-up when being called with status == 0. This causes e.g. problems when trying to hot-plug and hot-unplug a vhost user devices multiple times since the de-initialization step is completely skipped during the unplug operation.

This bug has been introduced in commit 9f6bcfd99f ("hw/virtio: move vm_running check to virtio_device_started") which replaced

should_start = status & VIRTIO_CONFIG_S_DRIVER_OK;

with

should_start = virtio_device_started(vdev, status);

which later got replaced by virtio_device_should_start(). This blocked the possibility to set should_start to false in case the status flag VIRTIO_CONFIG_S_DRIVER_OK was not set.

Fix it by adjusting the virtio_device_should_start() function to only consider the status flag instead of vdev->started. Since this function is only used in the various vhost_*_set_status() functions for exactly the same purpose, it should be fine to fix it in this central place there without any risk to change the behavior of other code.

Fixes 9f6bcfd99f ("hw/virtio: move vm_running check to virtio_device_started")
---
 include/hw/virtio/virtio.h | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
include/hw/virtio/virtio.h +4 -4
index 7d5ffdc145..2eafad17b8 100644
@@ -470,9 +470,9 @@ static inline bool virtio_device_started(VirtIODevice *vdev, uint8_t status)
  * @vdev - the VirtIO device
  * @status - the devices status bits
  *
- * This is similar to virtio_device_started() but also encapsulates a
- * check on the VM status which would prevent a device starting
- * anyway.
+ * This is similar to virtio_device_started() but ignores vdev->started
+ * and also encapsulates a check on the VM status which would prevent a
+ * device from starting anyway.
  */
 static inline bool virtio_device_should_start(VirtIODevice *vdev, uint8_t status)
 {
@@ -480,7 +480,7 @@ static inline bool virtio_device_should_start(VirtIODevice *vdev, uint8_t status
         return false;
     }
 
-    return virtio_device_started(vdev, status);
+    return status & VIRTIO_CONFIG_S_DRIVER_OK;
 }
 
 static inline void virtio_set_started(VirtIODevice *vdev, bool started)
Signature · 1 line
2.45.2
Reviewed-by on v1 1/1
#1 lore raw
Quoted 18 lines · Thomas Huth

The unrealize functions of the various vhost-user devices are calling the corresponding vhost_*_set_status() functions with a status of 0 to shut down the device correctly.

Now these vhost_*_set_status() functions all follow this scheme:

   bool should_start = virtio_device_should_start(vdev, status);

   if (vhost_dev_is_started(&vvc->vhost_dev) == should_start) {
       return;
   }

   if (should_start) {
       /* ... do the initialization stuff ... */
   } else {
       /* ... do the cleanup stuff ... */
   }
Quoted 4 lines

The problem here is virtio_device_should_start(vdev, 0) currently always returns "true" since it internally only looks at vdev->started instead of looking at the "status" parameter. Thus once the device got started once, virtio_device_should_start() always returns true

virtio_device_should_start() returning true if it's already started and running looks like a code smell to me... it intuitively feels like a ternary state instead of boolean: not startable, startable, already started.

Quoted 26 lines

and thus the vhost_*_set_status() functions return early, without ever doing any clean-up when being called with status == 0. This causes e.g. problems when trying to hot-plug and hot-unplug a vhost user devices multiple times since the de-initialization step is completely skipped during the unplug operation.

This bug has been introduced in commit 9f6bcfd99f ("hw/virtio: move vm_running check to virtio_device_started") which replaced

should_start = status & VIRTIO_CONFIG_S_DRIVER_OK;

with

should_start = virtio_device_started(vdev, status);

which later got replaced by virtio_device_should_start(). This blocked the possibility to set should_start to false in case the status flag VIRTIO_CONFIG_S_DRIVER_OK was not set.

Fix it by adjusting the virtio_device_should_start() function to only consider the status flag instead of vdev->started. Since this function is only used in the various vhost_*_set_status() functions for exactly the same purpose, it should be fine to fix it in this central place there without any risk to change the behavior of other code.

Quoted 4 lines
Fixes 9f6bcfd99f ("hw/virtio: move vm_running check to virtio_device_started")

---

#2 lore raw
In reply to Thomas Huth
Quoted 73 lines · Thomas Huth

The unrealize functions of the various vhost-user devices are calling the corresponding vhost_*_set_status() functions with a status of 0 to shut down the device correctly.

Now these vhost_*_set_status() functions all follow this scheme:

     bool should_start = virtio_device_should_start(vdev, status);

     if (vhost_dev_is_started(&vvc->vhost_dev) == should_start) {
         return;
     }

     if (should_start) {
         /* ... do the initialization stuff ... */
     } else {
         /* ... do the cleanup stuff ... */
     }

The problem here is virtio_device_should_start(vdev, 0) currently always returns "true" since it internally only looks at vdev->started instead of looking at the "status" parameter. Thus once the device got started once, virtio_device_should_start() always returns true and thus the vhost_*_set_status() functions return early, without ever doing any clean-up when being called with status == 0. This causes e.g. problems when trying to hot-plug and hot-unplug a vhost user devices multiple times since the de-initialization step is completely skipped during the unplug operation.

This bug has been introduced in commit 9f6bcfd99f ("hw/virtio: move vm_running check to virtio_device_started") which replaced

  should_start = status & VIRTIO_CONFIG_S_DRIVER_OK;

with

  should_start = virtio_device_started(vdev, status);

which later got replaced by virtio_device_should_start(). This blocked the possibility to set should_start to false in case the status flag VIRTIO_CONFIG_S_DRIVER_OK was not set.

Fix it by adjusting the virtio_device_should_start() function to only consider the status flag instead of vdev->started. Since this function is only used in the various vhost_*_set_status() functions for exactly the same purpose, it should be fine to fix it in this central place there without any risk to change the behavior of other code.

Fixes 9f6bcfd99f ("hw/virtio: move vm_running check to virtio_device_started")
---
  include/hw/virtio/virtio.h | 8 ++++----
  1 file changed, 4 insertions(+), 4 deletions(-)
include/hw/virtio/virtio.h +3 -3
index 7d5ffdc145..2eafad17b8 100644
@@ -470,9 +470,9 @@ static inline bool virtio_device_started(VirtIODevice *vdev, uint8_t status)
   * @vdev - the VirtIO device
   * @status - the devices status bits
   *
- * This is similar to virtio_device_started() but also encapsulates a
- * check on the VM status which would prevent a device starting
- * anyway.
+ * This is similar to virtio_device_started() but ignores vdev->started
+ * and also encapsulates a check on the VM status which would prevent a
+ * device from starting anyway.
   */
  static inline bool virtio_device_should_start(VirtIODevice *vdev, uint8_t status)
  {

include/hw/virtio/virtio.h static inline bool virtio_device_should_start(VirtIODevice *vdev, uint8_t status · line 484

Quoted 6 lines
include/hw/virtio/virtio.h +1 -1
@@ -480,7 +480,7 @@ static inline bool virtio_device_should_start(VirtIODevice *vdev, uint8_t status
          return false;
      }
 
-    return virtio_device_started(vdev, status);
+    return status & VIRTIO_CONFIG_S_DRIVER_OK;
  }

Michael, any concerns or comments about this patch?

If not, I could also take it via my s390x tree since this fixes vhost-ccw devices on s390x.

Thomas

#3 lore raw
Quoted 82 lines · Thomas Huth
Quoted 73 lines · Thomas Huth

The unrealize functions of the various vhost-user devices are calling the corresponding vhost_*_set_status() functions with a status of 0 to shut down the device correctly.

Now these vhost_*_set_status() functions all follow this scheme:

     bool should_start = virtio_device_should_start(vdev, status);

     if (vhost_dev_is_started(&vvc->vhost_dev) == should_start) {
         return;
     }

     if (should_start) {
         /* ... do the initialization stuff ... */
     } else {
         /* ... do the cleanup stuff ... */
     }

The problem here is virtio_device_should_start(vdev, 0) currently always returns "true" since it internally only looks at vdev->started instead of looking at the "status" parameter. Thus once the device got started once, virtio_device_should_start() always returns true and thus the vhost_*_set_status() functions return early, without ever doing any clean-up when being called with status == 0. This causes e.g. problems when trying to hot-plug and hot-unplug a vhost user devices multiple times since the de-initialization step is completely skipped during the unplug operation.

This bug has been introduced in commit 9f6bcfd99f ("hw/virtio: move vm_running check to virtio_device_started") which replaced

  should_start = status & VIRTIO_CONFIG_S_DRIVER_OK;

with

  should_start = virtio_device_started(vdev, status);

which later got replaced by virtio_device_should_start(). This blocked the possibility to set should_start to false in case the status flag VIRTIO_CONFIG_S_DRIVER_OK was not set.

Fix it by adjusting the virtio_device_should_start() function to only consider the status flag instead of vdev->started. Since this function is only used in the various vhost_*_set_status() functions for exactly the same purpose, it should be fine to fix it in this central place there without any risk to change the behavior of other code.

Fixes 9f6bcfd99f ("hw/virtio: move vm_running check to virtio_device_started")
---
  include/hw/virtio/virtio.h | 8 ++++----
  1 file changed, 4 insertions(+), 4 deletions(-)
include/hw/virtio/virtio.h +3 -3
index 7d5ffdc145..2eafad17b8 100644
@@ -470,9 +470,9 @@ static inline bool virtio_device_started(VirtIODevice *vdev, uint8_t status)
   * @vdev - the VirtIO device
   * @status - the devices status bits
   *
- * This is similar to virtio_device_started() but also encapsulates a
- * check on the VM status which would prevent a device starting
- * anyway.
+ * This is similar to virtio_device_started() but ignores vdev->started
+ * and also encapsulates a check on the VM status which would prevent a
+ * device from starting anyway.
   */
  static inline bool virtio_device_should_start(VirtIODevice *vdev, uint8_t status)
  {

include/hw/virtio/virtio.h static inline bool virtio_device_should_start(VirtIODevice *vdev, uint8_t status · line 483

Quoted 5 lines
include/hw/virtio/virtio.h +1 -1
@@ -480,7 +480,7 @@ static inline bool virtio_device_should_start(VirtIODevice *vdev, uint8_t status
          return false;
      }
-    return virtio_device_started(vdev, status);
+    return status & VIRTIO_CONFIG_S_DRIVER_OK;
  }

Michael, any concerns or comments about this patch?

Quoted 4 lines

If not, I could also take it via my s390x tree since this fixes vhost-ccw devices on s390x.

Thomas

I'm working on a pull request with this today.
I can drop it if you prefer ...

#4 lore raw
Quoted 83 lines · Michael S. Tsirkin
Quoted 82 lines · Thomas Huth
Quoted 73 lines · Thomas Huth

The unrealize functions of the various vhost-user devices are calling the corresponding vhost_*_set_status() functions with a status of 0 to shut down the device correctly.

Now these vhost_*_set_status() functions all follow this scheme:

      bool should_start = virtio_device_should_start(vdev, status);

      if (vhost_dev_is_started(&vvc->vhost_dev) == should_start) {
          return;
      }

      if (should_start) {
          /* ... do the initialization stuff ... */
      } else {
          /* ... do the cleanup stuff ... */
      }

The problem here is virtio_device_should_start(vdev, 0) currently always returns "true" since it internally only looks at vdev->started instead of looking at the "status" parameter. Thus once the device got started once, virtio_device_should_start() always returns true and thus the vhost_*_set_status() functions return early, without ever doing any clean-up when being called with status == 0. This causes e.g. problems when trying to hot-plug and hot-unplug a vhost user devices multiple times since the de-initialization step is completely skipped during the unplug operation.

This bug has been introduced in commit 9f6bcfd99f ("hw/virtio: move vm_running check to virtio_device_started") which replaced

   should_start = status & VIRTIO_CONFIG_S_DRIVER_OK;

with

   should_start = virtio_device_started(vdev, status);

which later got replaced by virtio_device_should_start(). This blocked the possibility to set should_start to false in case the status flag VIRTIO_CONFIG_S_DRIVER_OK was not set.

Fix it by adjusting the virtio_device_should_start() function to only consider the status flag instead of vdev->started. Since this function is only used in the various vhost_*_set_status() functions for exactly the same purpose, it should be fine to fix it in this central place there without any risk to change the behavior of other code.

Fixes 9f6bcfd99f ("hw/virtio: move vm_running check to virtio_device_started")
---
   include/hw/virtio/virtio.h | 8 ++++----
   1 file changed, 4 insertions(+), 4 deletions(-)
include/hw/virtio/virtio.h +3 -3
index 7d5ffdc145..2eafad17b8 100644
@@ -470,9 +470,9 @@ static inline bool virtio_device_started(VirtIODevice *vdev, uint8_t status)
    * @vdev - the VirtIO device
    * @status - the devices status bits
    *
- * This is similar to virtio_device_started() but also encapsulates a
- * check on the VM status which would prevent a device starting
- * anyway.
+ * This is similar to virtio_device_started() but ignores vdev->started
+ * and also encapsulates a check on the VM status which would prevent a
+ * device from starting anyway.
    */
   static inline bool virtio_device_should_start(VirtIODevice *vdev, uint8_t status)
   {

include/hw/virtio/virtio.h static inline bool virtio_device_should_start(VirtIODevice *vdev, uint8_t status · line 483

Quoted 5 lines
include/hw/virtio/virtio.h +1 -1
@@ -480,7 +480,7 @@ static inline bool virtio_device_should_start(VirtIODevice *vdev, uint8_t status
           return false;
       }
-    return virtio_device_started(vdev, status);
+    return status & VIRTIO_CONFIG_S_DRIVER_OK;
   }

Michael, any concerns or comments about this patch?

Quoted 7 lines
Quoted 4 lines

If not, I could also take it via my s390x tree since this fixes vhost-ccw devices on s390x.

Thomas

I'm working on a pull request with this today.
I can drop it if you prefer ...

Ah, perfect, please include it in your PR then!

Thanks,
Thomas