Skip to contentSkip to conversation

virtio-pci: store virtqueue size directly to a device

PATCH In review Denis Plotnikov 2 participants 3 messages Dec 23, 2019 – Dec 24, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 virtio-pci: store virtqueue size directly to a device

Currenly, the virtqueue size is saved to the proxy on pci writing and is read from the device pci reading.
The virtqueue size is propagated later on form the proxy to the device on virqueue enabling stage.

This could be a problem, if a guest, on the virtqueue configuration, sets the size and then re-read it immediatly before the queue enabling in order to check if the desiged size has been set.

This happens in seabios: (sebios snippet)

vp_find_vq()

{
    ...
    /* check if the queue is available */
    if (vp->use_modern) {
        num = vp_read(&vp->common, virtio_pci_common_cfg, queue_size);
        if (num > MAX_QUEUE_NUM) {
            vp_write(&vp->common, virtio_pci_common_cfg, queue_size,
                     MAX_QUEUE_NUM);
            num = vp_read(&vp->common, virtio_pci_common_cfg, queue_size);
        }
    } else {
        num = vp_read(&vp->legacy, virtio_pci_legacy, queue_num);
    }
    if (!num) {
        dprintf(1, "ERROR: queue size is 0\n");
        goto fail;
    }
    if (num > MAX_QUEUE_NUM) {
        dprintf(1, "ERROR: queue size %d > %d\n", num, MAX_QUEUE_NUM);
        goto fail;
    }
    ...
}

If the device queue num is greater then the max queue size supported by seabios, seabios tries to reduce the queue size, then re-read it again, I suppose to check if the setting actually happens, and then checks the virtqueue size again, to deside whether it is satisfied with the vaule.
In this case, if device's virtqueue size is 512 and seabios max supported queue size is 256, seabios tries to set 256 but than read 512 again and can't proceed with that vaule, preventing the guest from successful booting.
The root case was investigated by Roman Kagan <[email protected]>

The patch fixes the problem, by propagating the queue size to the device right away, so the written value could be read on the next step, if the value was ok for the device.

---
 hw/virtio/virtio-pci.c | 2 ++
 1 file changed, 2 insertions(+)
hw/virtio/virtio-pci.c +2 -0
index c6b47a9c73..e5c759e19e 100644
@@ -1256,6 +1256,8 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,
         break;
     case VIRTIO_PCI_COMMON_Q_SIZE:
         proxy->vqs[vdev->queue_sel].num = val;
+        virtio_queue_set_num(vdev, vdev->queue_sel,
+                             proxy->vqs[vdev->queue_sel].num);
         break;
     case VIRTIO_PCI_COMMON_Q_MSIX:
         msix_vector_unuse(&proxy->pci_dev,
Signature · 1 line
2.17.0
#1 lore raw
Quoted 45 lines · Denis Plotnikov

Currenly, the virtqueue size is saved to the proxy on pci writing and is read from the device pci reading.
The virtqueue size is propagated later on form the proxy to the device on virqueue enabling stage.

This could be a problem, if a guest, on the virtqueue configuration, sets the size and then re-read it immediatly before the queue enabling in order to check if the desiged size has been set.

This happens in seabios: (sebios snippet)

vp_find_vq()

{
    ...
    /* check if the queue is available */
    if (vp->use_modern) {
        num = vp_read(&vp->common, virtio_pci_common_cfg, queue_size);
        if (num > MAX_QUEUE_NUM) {
            vp_write(&vp->common, virtio_pci_common_cfg, queue_size,
                     MAX_QUEUE_NUM);
            num = vp_read(&vp->common, virtio_pci_common_cfg, queue_size);
        }
    } else {
        num = vp_read(&vp->legacy, virtio_pci_legacy, queue_num);
    }
    if (!num) {
        dprintf(1, "ERROR: queue size is 0\n");
        goto fail;
    }
    if (num > MAX_QUEUE_NUM) {
        dprintf(1, "ERROR: queue size %d > %d\n", num, MAX_QUEUE_NUM);
        goto fail;
    }
    ...
}

If the device queue num is greater then the max queue size supported by seabios, seabios tries to reduce the queue size, then re-read it again, I suppose to check if the setting actually happens, and then checks the virtqueue size again, to deside whether it is satisfied with the vaule.
In this case, if device's virtqueue size is 512 and seabios max supported queue size is 256, seabios tries to set 256 but than read 512 again and can't proceed with that vaule, preventing the guest from successful booting.
The root case was investigated by Roman Kagan <[email protected]>

Quoted 7 lines

The patch fixes the problem, by propagating the queue size to the device right away, so the written value could be read on the next step, if the value was ok for the device.

Thanks, I already have this queued as:

commit 8aabbbd9d04f95d5581d2275362996ecb5516dd9

Author: Michael S. Tsirkin <[email protected]>
Date:   Fri Dec 13 09:22:48 2019 -0500

virtio: update queue size on guest write

Some guests read back queue size after writing it. Update the size immediatly upon write otherwise they get confused.

I would appreciate checking other transports, they likely need the same fix.

Quoted 19 lines
---
 hw/virtio/virtio-pci.c | 2 ++
 1 file changed, 2 insertions(+)
hw/virtio/virtio-pci.c +2 -0
index c6b47a9c73..e5c759e19e 100644
@@ -1256,6 +1256,8 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,
         break;
     case VIRTIO_PCI_COMMON_Q_SIZE:
         proxy->vqs[vdev->queue_sel].num = val;
+        virtio_queue_set_num(vdev, vdev->queue_sel,
+                             proxy->vqs[vdev->queue_sel].num);
         break;
     case VIRTIO_PCI_COMMON_Q_MSIX:
         msix_vector_unuse(&proxy->pci_dev,
Signature · 1 line
2.17.0
#2 lore raw
Quoted 61 lines · Michael S. Tsirkin
Quoted 45 lines · Denis Plotnikov

Currenly, the virtqueue size is saved to the proxy on pci writing and is read from the device pci reading.
The virtqueue size is propagated later on form the proxy to the device on virqueue enabling stage.

This could be a problem, if a guest, on the virtqueue configuration, sets the size and then re-read it immediatly before the queue enabling in order to check if the desiged size has been set.

This happens in seabios: (sebios snippet)

vp_find_vq()

{
     ...
     /* check if the queue is available */
     if (vp->use_modern) {
         num = vp_read(&vp->common, virtio_pci_common_cfg, queue_size);
         if (num > MAX_QUEUE_NUM) {
             vp_write(&vp->common, virtio_pci_common_cfg, queue_size,
                      MAX_QUEUE_NUM);
             num = vp_read(&vp->common, virtio_pci_common_cfg, queue_size);
         }
     } else {
         num = vp_read(&vp->legacy, virtio_pci_legacy, queue_num);
     }
     if (!num) {
         dprintf(1, "ERROR: queue size is 0\n");
         goto fail;
     }
     if (num > MAX_QUEUE_NUM) {
         dprintf(1, "ERROR: queue size %d > %d\n", num, MAX_QUEUE_NUM);
         goto fail;
     }
     ...
}

If the device queue num is greater then the max queue size supported by seabios, seabios tries to reduce the queue size, then re-read it again, I suppose to check if the setting actually happens, and then checks the virtqueue size again, to deside whether it is satisfied with the vaule.
In this case, if device's virtqueue size is 512 and seabios max supported queue size is 256, seabios tries to set 256 but than read 512 again and can't proceed with that vaule, preventing the guest from successful booting.
The root case was investigated by Roman Kagan <[email protected]>

Quoted 7 lines

The patch fixes the problem, by propagating the queue size to the device right away, so the written value could be read on the next step, if the value was ok for the device.

Thanks, I already have this queued as:

commit 8aabbbd9d04f95d5581d2275362996ecb5516dd9

Author: Michael S. Tsirkin <[email protected]>
Date:   Fri Dec 13 09:22:48 2019 -0500

virtio: update queue size on guest write

Quoted 8 lines

Some guests read back queue size after writing it. Update the size immediatly upon write otherwise they get confused.

I would appreciate checking other transports, they likely need the same fix.

ok, I'll send the patch shortly

Quoted 21 lines
Quoted 21 lines

---
  hw/virtio/virtio-pci.c | 2 ++
  1 file changed, 2 insertions(+)
hw/virtio/virtio-pci.c +2 -0
index c6b47a9c73..e5c759e19e 100644
@@ -1256,6 +1256,8 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,
          break;
      case VIRTIO_PCI_COMMON_Q_SIZE:
          proxy->vqs[vdev->queue_sel].num = val;
+        virtio_queue_set_num(vdev, vdev->queue_sel,
+                             proxy->vqs[vdev->queue_sel].num);
          break;
      case VIRTIO_PCI_COMMON_Q_MSIX:
          msix_vector_unuse(&proxy->pci_dev,
Signature · 1 line
2.17.0