Shadow VirtQueue event index support
cover Shadow VirtQueue event index support
Event idx helps to reduce the number of notifications between the device and the driver. It allows them to specify an index on the circular descriptors rings where to issue the notification, instead of a single binary indicator.
Adding support for SVQ.
These patches are sent on top of [1] series, so trivial conflicts could arise if it is applied directly on master. Future versions can be not based on it is more convenient.
[1] https://lists.nongnu.org/archive/html/qemu-devel/2022-10/msg03280.html
Eugenio Pérez (4):
vhost: allocate event_idx fields on vring
vhost: toggle device callbacks using used event idx
vhost: use avail event idx on vhost_svq_kick
vhost: Accept event idx flag
hw/virtio/vhost-shadow-virtqueue.c | 39 ++++++++++++++++++++++++------ 1 file changed, 31 insertions(+), 8 deletions(-)
Signature · 1 line
2.31.1
PATCH 1/4 vhost: allocate event_idx fields on vring
There was not enough room to accomodate them.
--- hw/virtio/vhost-shadow-virtqueue.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
hw/virtio/vhost-shadow-virtqueue.c +4 -4
@@ -570,16 +570,16 @@ void vhost_svq_get_vring_addr(const VhostShadowVirtqueue *svq,size_t vhost_svq_driver_area_size (const VhostShadowVirtqueue *svq) {size_t desc_size =sizeof (vring_desc_t) * svq->vring.num; -size_t avail_size = offsetof(vring_avail_t , ring) + -sizeof (uint16_t ) * svq->vring.num; + size_t avail_size = offsetof(vring_avail_t, ring[svq->vring.num]) + +sizeof (uint16_t );return ROUND_UP (desc_size + avail_size,qemu_real_host_page_size ()); }size_t vhost_svq_device_area_size (const VhostShadowVirtqueue *svq) { -size_t used_size = offsetof(vring_used_t , ring) + -sizeof (vring_used_elem_t) * svq->vring.num; + size_t used_size = offsetof(vring_used_t, ring[svq->vring.num]) + +sizeof (uint16_t );return ROUND_UP (used_size,qemu_real_host_page_size ()); }
Signature · 1 line
2.31.1
PATCH 2/4 vhost: toggle device callbacks using used event idx
In reply to Eugenio PérezActually use the new field of the used ring and tell the device if SVQ wants to be notified.
The code is not reachable at the moment.
--- hw/virtio/vhost-shadow-virtqueue.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)
hw/virtio/vhost-shadow-virtqueue.c +15 -3
@@ -369,15 +369,27 @@ static bool vhost_svq_more_used(VhostShadowVirtqueue *svq)*/ static bool vhost_svq_enable_notification (VhostShadowVirtqueue *svq) { - svq->vring.avail->flags &= ~cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); -/* Make sure the flag is written before the read of used_idx */ +if (virtio_vdev_has_feature (svq->vdev,VIRTIO_RING_F_EVENT_IDX )) { +uint16_t *used_event = (uint16_t *)&svq->vring.avail->ring[svq->vring.num]; + *used_event = svq->shadow_used_idx; + }else { + svq->vring.avail->flags &= ~cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); + } + +/* Make sure the event is enabled before the read of used_idx */ smp_mb ();return !vhost_svq_more_used (svq); }static void vhost_svq_disable_notification (VhostShadowVirtqueue *svq) { - svq->vring.avail->flags |=cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); +/* +* No need to disable notification in the event idx case, since used event +* index is already an index too far away. +*/ +if (!virtio_vdev_has_feature (svq->vdev,VIRTIO_RING_F_EVENT_IDX )) { + svq->vring.avail->flags |=cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); + } } static uint16_t vhost_svq_last_desc_of_chain(const VhostShadowVirtqueue *svq,
Signature · 1 line
2.31.1
Re: [PATCH 2/4] vhost: toggle device callbacks using used event idx
Quoted 16 lines · Eugenio Pérez
Actually use the new field of the used ring and tell the device if SVQ wants to be notified.
The code is not reachable at the moment.
Signed-off-by Eugenio Pérez <[email protected]>--- hw/virtio/vhost-shadow-virtqueue.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)hw/virtio/vhost-shadow-virtqueue.c +0
-0
hw/virtio/vhost-shadow-virtqueue.c static bool vhost_svq_more_used(VhostShadowVirtqueue *svq) · line 374
Quoted 8 lines
hw/virtio/vhost-shadow-virtqueue.c +3
-2@@ -369,15 +369,27 @@ static bool vhost_svq_more_used(VhostShadowVirtqueue *svq)*/ static bool vhost_svq_enable_notification (VhostShadowVirtqueue *svq) { - svq->vring.avail->flags &= ~cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); -/* Make sure the flag is written before the read of used_idx */ +if (virtio_vdev_has_feature (svq->vdev,VIRTIO_RING_F_EVENT_IDX )) { +uint16_t *used_event = (uint16_t *)&svq->vring.avail->ring[svq->vring.num]; + *used_event = svq->shadow_used_idx;
Do we need to care about the endian here?
E.g vduse has:
*((uint16_t *)&vq->vring.used->ring[vq->vring.num]) = htole16(val);
Thanks
Quoted 25 lines
hw/virtio/vhost-shadow-virtqueue.c +12
-2+ }else { + svq->vring.avail->flags &= ~cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); + } + +/* Make sure the event is enabled before the read of used_idx */ smp_mb ();return !vhost_svq_more_used (svq); }static void vhost_svq_disable_notification (VhostShadowVirtqueue *svq) { - svq->vring.avail->flags |=cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); +/* +* No need to disable notification in the event idx case, since used event +* index is already an index too far away. +*/ +if (!virtio_vdev_has_feature (svq->vdev,VIRTIO_RING_F_EVENT_IDX )) { + svq->vring.avail->flags |=cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); + } } static uint16_t vhost_svq_last_desc_of_chain(const VhostShadowVirtqueue *svq, --2.31.1
Re: [PATCH 2/4] vhost: toggle device callbacks using used event idx
Quoted 29 lines · Jason Wang
Quoted 16 lines · Eugenio Pérez
Actually use the new field of the used ring and tell the device if SVQ wants to be notified.
The code is not reachable at the moment.
Signed-off-by Eugenio Pérez <[email protected]>--- hw/virtio/vhost-shadow-virtqueue.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)hw/virtio/vhost-shadow-virtqueue.c +0
-0hw/virtio/vhost-shadow-virtqueue.c static bool vhost_svq_more_used(VhostShadowVirtqueue *svq) · line 374
Quoted 8 lines
hw/virtio/vhost-shadow-virtqueue.c +3
-2@@ -369,15 +369,27 @@ static bool vhost_svq_more_used(VhostShadowVirtqueue *svq)*/ static bool vhost_svq_enable_notification (VhostShadowVirtqueue *svq) { - svq->vring.avail->flags &= ~cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); -/* Make sure the flag is written before the read of used_idx */ +if (virtio_vdev_has_feature (svq->vdev,VIRTIO_RING_F_EVENT_IDX )) { +uint16_t *used_event = (uint16_t *)&svq->vring.avail->ring[svq->vring.num]; + *used_event = svq->shadow_used_idx;Do we need to care about the endian here?
Quoted 4 lines
E.g vduse has:
*((uint16_t *)&vq->vring.used->ring[vq->vring.num]) = htole16(val);
Good catch, I forgot about endianness.
I'll review the series, thanks!
Quoted 28 lines
Thanks
Quoted 25 lines
hw/virtio/vhost-shadow-virtqueue.c +12
-2+ }else { + svq->vring.avail->flags &= ~cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); + } + +/* Make sure the event is enabled before the read of used_idx */ smp_mb ();return !vhost_svq_more_used (svq); }static void vhost_svq_disable_notification (VhostShadowVirtqueue *svq) { - svq->vring.avail->flags |=cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); +/* +* No need to disable notification in the event idx case, since used event +* index is already an index too far away. +*/ +if (!virtio_vdev_has_feature (svq->vdev,VIRTIO_RING_F_EVENT_IDX )) { + svq->vring.avail->flags |=cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); + } } static uint16_t vhost_svq_last_desc_of_chain(const VhostShadowVirtqueue *svq, --2.31.1
Re: [PATCH 2/4] vhost: toggle device callbacks using used event idx
Quoted 30 lines · Eugenio Perez Martin
Quoted 29 lines · Jason Wang
Quoted 16 lines · Eugenio Pérez
Actually use the new field of the used ring and tell the device if SVQ wants to be notified.
The code is not reachable at the moment.
Signed-off-by Eugenio Pérez <[email protected]>--- hw/virtio/vhost-shadow-virtqueue.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)hw/virtio/vhost-shadow-virtqueue.c +0
-0hw/virtio/vhost-shadow-virtqueue.c static bool vhost_svq_more_used(VhostShadowVirtqueue *svq) · line 374
Quoted 8 lines
hw/virtio/vhost-shadow-virtqueue.c +3
-2@@ -369,15 +369,27 @@ static bool vhost_svq_more_used(VhostShadowVirtqueue *svq)*/ static bool vhost_svq_enable_notification (VhostShadowVirtqueue *svq) { - svq->vring.avail->flags &= ~cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); -/* Make sure the flag is written before the read of used_idx */ +if (virtio_vdev_has_feature (svq->vdev,VIRTIO_RING_F_EVENT_IDX )) { +uint16_t *used_event = (uint16_t *)&svq->vring.avail->ring[svq->vring.num]; + *used_event = svq->shadow_used_idx;Do we need to care about the endian here?
Quoted 8 lines
Quoted 4 lines
E.g vduse has:
*((uint16_t *)&vq->vring.used->ring[vq->vring.num]) = htole16(val);Good catch, I forgot about endianness.
I'll review the series, thanks!
It's generally a waste that we don't use endian-ness annotations the way linux does.
Quoted 28 lines
Quoted 28 lines
Thanks
Quoted 25 lines
hw/virtio/vhost-shadow-virtqueue.c +12
-2+ }else { + svq->vring.avail->flags &= ~cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); + } + +/* Make sure the event is enabled before the read of used_idx */ smp_mb ();return !vhost_svq_more_used (svq); }static void vhost_svq_disable_notification (VhostShadowVirtqueue *svq) { - svq->vring.avail->flags |=cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); +/* +* No need to disable notification in the event idx case, since used event +* index is already an index too far away. +*/ +if (!virtio_vdev_has_feature (svq->vdev,VIRTIO_RING_F_EVENT_IDX )) { + svq->vring.avail->flags |=cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); + } } static uint16_t vhost_svq_last_desc_of_chain(const VhostShadowVirtqueue *svq, --2.31.1
Re: [PATCH 2/4] vhost: toggle device callbacks using used event idx
Quoted 37 lines · Michael S. Tsirkin
Quoted 35 lines · Eugenio Perez Martin
Quoted 33 lines · Jason Wang
Quoted 16 lines · Eugenio Pérez
Actually use the new field of the used ring and tell the device if SVQ wants to be notified.
The code is not reachable at the moment.
Signed-off-by Eugenio Pérez <[email protected]>--- hw/virtio/vhost-shadow-virtqueue.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)hw/virtio/vhost-shadow-virtqueue.c +0
-0hw/virtio/vhost-shadow-virtqueue.c static bool vhost_svq_more_used(VhostShadowVirtqueue *svq) · line 374
Quoted 8 lines
hw/virtio/vhost-shadow-virtqueue.c +3
-2@@ -369,15 +369,27 @@ static bool vhost_svq_more_used(VhostShadowVirtqueue *svq)*/ static bool vhost_svq_enable_notification (VhostShadowVirtqueue *svq) { - svq->vring.avail->flags &= ~cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); -/* Make sure the flag is written before the read of used_idx */ +if (virtio_vdev_has_feature (svq->vdev,VIRTIO_RING_F_EVENT_IDX )) { +uint16_t *used_event = (uint16_t *)&svq->vring.avail->ring[svq->vring.num]; + *used_event = svq->shadow_used_idx;Do we need to care about the endian here?
E.g vduse has:
*((uint16_t *)&vq->vring.used->ring[vq->vring.num]) = htole16(val);
Quoted 6 lines
Quoted 3 lines
Good catch, I forgot about endianness.
I'll review the series, thanks!
It's generally a waste that we don't use endian-ness annotations the way linux does.
Yes, it's worth doing something similar sometime.
Thanks
Quoted 31 lines
Quoted 30 lines
Quoted 30 lines
Thanks
Quoted 25 lines
hw/virtio/vhost-shadow-virtqueue.c +12
-2+ }else { + svq->vring.avail->flags &= ~cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); + } + +/* Make sure the event is enabled before the read of used_idx */ smp_mb ();return !vhost_svq_more_used (svq); }static void vhost_svq_disable_notification (VhostShadowVirtqueue *svq) { - svq->vring.avail->flags |=cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); +/* +* No need to disable notification in the event idx case, since used event +* index is already an index too far away. +*/ +if (!virtio_vdev_has_feature (svq->vdev,VIRTIO_RING_F_EVENT_IDX )) { + svq->vring.avail->flags |=cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); + } } static uint16_t vhost_svq_last_desc_of_chain(const VhostShadowVirtqueue *svq, --2.31.1
Re: [PATCH 2/4] vhost: toggle device callbacks using used event idx
Quoted 39 lines · Jason Wang
Quoted 37 lines · Michael S. Tsirkin
Quoted 35 lines · Eugenio Perez Martin
Quoted 33 lines · Jason Wang
Quoted 16 lines · Eugenio Pérez
Actually use the new field of the used ring and tell the device if SVQ wants to be notified.
The code is not reachable at the moment.
Signed-off-by Eugenio Pérez <[email protected]>--- hw/virtio/vhost-shadow-virtqueue.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)hw/virtio/vhost-shadow-virtqueue.c +0
-0hw/virtio/vhost-shadow-virtqueue.c static bool vhost_svq_more_used(VhostShadowVirtqueue *svq) · line 374
Quoted 8 lines
hw/virtio/vhost-shadow-virtqueue.c +3
-2@@ -369,15 +369,27 @@ static bool vhost_svq_more_used(VhostShadowVirtqueue *svq)*/ static bool vhost_svq_enable_notification (VhostShadowVirtqueue *svq) { - svq->vring.avail->flags &= ~cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); -/* Make sure the flag is written before the read of used_idx */ +if (virtio_vdev_has_feature (svq->vdev,VIRTIO_RING_F_EVENT_IDX )) { +uint16_t *used_event = (uint16_t *)&svq->vring.avail->ring[svq->vring.num]; + *used_event = svq->shadow_used_idx;Do we need to care about the endian here?
E.g vduse has:
*((uint16_t *)&vq->vring.used->ring[vq->vring.num]) = htole16(val);
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
Good catch, I forgot about endianness.
I'll review the series, thanks!
It's generally a waste that we don't use endian-ness annotations the way linux does.
Yes, it's worth doing something similar sometime.
Maybe we could wrap them in some struct like virtio_le16 or virtio_16, avoiding at least integer direct assignment? Wrappers like cpu_to_virtio16 could return these structs and I think all compilers should emit the same code as direct assignment.
Thanks!
Quoted 34 lines
Thanks
Quoted 31 lines
Quoted 30 lines
Quoted 30 lines
Thanks
Quoted 25 lines
hw/virtio/vhost-shadow-virtqueue.c +12
-2+ }else { + svq->vring.avail->flags &= ~cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); + } + +/* Make sure the event is enabled before the read of used_idx */ smp_mb ();return !vhost_svq_more_used (svq); }static void vhost_svq_disable_notification (VhostShadowVirtqueue *svq) { - svq->vring.avail->flags |=cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); +/* +* No need to disable notification in the event idx case, since used event +* index is already an index too far away. +*/ +if (!virtio_vdev_has_feature (svq->vdev,VIRTIO_RING_F_EVENT_IDX )) { + svq->vring.avail->flags |=cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); + } } static uint16_t vhost_svq_last_desc_of_chain(const VhostShadowVirtqueue *svq, --2.31.1
Re: [PATCH 2/4] vhost: toggle device callbacks using used event idx
Quoted 6 lines · Eugenio Perez Martin
Quoted 5 lines
Quoted 2 lines
It's generally a waste that we don't use endian-ness annotations the way linux does.
Yes, it's worth doing something similar sometime.
Quoted 7 lines
Maybe we could wrap them in some struct like
virtio_le16orvirtio_16, avoiding at least integer direct assignment? Wrappers likecpu_to_virtio16could return these structs and I think all compilers should emit the same code as direct assignment.Thanks!
This will break bitwise operations such as | and &. Generally Linux has solved the problem and I don't think we should go look for another solution.
Quoted 36 lines
Quoted 36 lines
Thanks
Quoted 31 lines
Quoted 30 lines
Quoted 30 lines
Thanks
Quoted 25 lines
hw/virtio/vhost-shadow-virtqueue.c +12
-2+ }else { + svq->vring.avail->flags &= ~cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); + } + +/* Make sure the event is enabled before the read of used_idx */ smp_mb ();return !vhost_svq_more_used (svq); }static void vhost_svq_disable_notification (VhostShadowVirtqueue *svq) { - svq->vring.avail->flags |=cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); +/* +* No need to disable notification in the event idx case, since used event +* index is already an index too far away. +*/ +if (!virtio_vdev_has_feature (svq->vdev,VIRTIO_RING_F_EVENT_IDX )) { + svq->vring.avail->flags |=cpu_to_le16 (VRING_AVAIL_F_NO_INTERRUPT ); + } } static uint16_t vhost_svq_last_desc_of_chain(const VhostShadowVirtqueue *svq, --2.31.1
Re: [PATCH 2/4] vhost: toggle device callbacks using used event idx
Quoted 13 lines · Michael S. Tsirkin
Quoted 11 lines · Eugenio Perez Martin
Quoted 5 lines
Quoted 2 lines
It's generally a waste that we don't use endian-ness annotations the way linux does.
Yes, it's worth doing something similar sometime.
Maybe we could wrap them in some struct like
virtio_le16orvirtio_16, avoiding at least integer direct assignment? Wrappers likecpu_to_virtio16could return these structs and I think all compilers should emit the same code as direct assignment.
Quoted 6 lines
Quoted 2 lines
Thanks!
This will break bitwise operations such as | and &. Generally Linux has solved the problem and I don't think we should go look for another solution.
Yes, but it should not block this series (we can do that in the future if we had bandwidth).
Thanks
Quoted 39 lines
Quoted 38 lines
Quoted 38 lines
Thanks
Quoted 31 lines
Quoted 30 lines
Quoted 30 lines
Thanks
Quoted 25 lines
(quoted diff) +12
-2+ } else { + svq->vring.avail->flags &= ~cpu_to_le16(VRING_AVAIL_F_NO_INTERRUPT); + } + + /* Make sure the event is enabled before the read of used_idx */ smp_mb(); return !vhost_svq_more_used(svq); } static void vhost_svq_disable_notification(VhostShadowVirtqueue *svq) { - svq->vring.avail->flags |= cpu_to_le16(VRING_AVAIL_F_NO_INTERRUPT); + /* + * No need to disable notification in the event idx case, since used event + * index is already an index too far away. + */ + if (!virtio_vdev_has_feature(svq->vdev, VIRTIO_RING_F_EVENT_IDX)) { + svq->vring.avail->flags |= cpu_to_le16(VRING_AVAIL_F_NO_INTERRUPT); + } } static uint16_t vhost_svq_last_desc_of_chain(const VhostShadowVirtqueue *svq, --2.31.1
Re: [PATCH 2/4] vhost: toggle device callbacks using used event idx
Quoted 17 lines · Jason Wang
Quoted 16 lines · Michael S. Tsirkin
Quoted 13 lines · Eugenio Perez Martin
Quoted 5 lines
Quoted 2 lines
It's generally a waste that we don't use endian-ness annotations the way linux does.
Yes, it's worth doing something similar sometime.
Maybe we could wrap them in some struct like
virtio_le16orvirtio_16, avoiding at least integer direct assignment? Wrappers likecpu_to_virtio16could return these structs and I think all compilers should emit the same code as direct assignment.Thanks!
Quoted 8 lines
Quoted 3 lines
This will break bitwise operations such as | and &. Generally Linux has solved the problem and I don't think we should go look for another solution.
Yes, but it should not block this series (we can do that in the future if we had bandwidth).
Thanks
Sorry I don't get what you are saying. Which series?
Making LE tags structs is not going to fly, this is why sparse implements the __bitwise__ tag and in fact this is where the name comes from - bitwise operations need to work.
Quoted 39 lines
Quoted 39 lines
Quoted 38 lines
Quoted 38 lines
Thanks
Quoted 31 lines
Quoted 30 lines
Quoted 30 lines
Thanks
Quoted 25 lines
(quoted diff) +12
-2+ } else { + svq->vring.avail->flags &= ~cpu_to_le16(VRING_AVAIL_F_NO_INTERRUPT); + } + + /* Make sure the event is enabled before the read of used_idx */ smp_mb(); return !vhost_svq_more_used(svq); } static void vhost_svq_disable_notification(VhostShadowVirtqueue *svq) { - svq->vring.avail->flags |= cpu_to_le16(VRING_AVAIL_F_NO_INTERRUPT); + /* + * No need to disable notification in the event idx case, since used event + * index is already an index too far away. + */ + if (!virtio_vdev_has_feature(svq->vdev, VIRTIO_RING_F_EVENT_IDX)) { + svq->vring.avail->flags |= cpu_to_le16(VRING_AVAIL_F_NO_INTERRUPT); + } } static uint16_t vhost_svq_last_desc_of_chain(const VhostShadowVirtqueue *svq, --2.31.1
Re: [PATCH 2/4] vhost: toggle device callbacks using used event idx
Quoted 23 lines · Michael S. Tsirkin
Quoted 21 lines · Jason Wang
Quoted 19 lines · Michael S. Tsirkin
Quoted 6 lines · Eugenio Perez Martin
Quoted 5 lines
Quoted 2 lines
It's generally a waste that we don't use endian-ness annotations the way linux does.
Yes, it's worth doing something similar sometime.
Quoted 7 lines
Maybe we could wrap them in some struct like
virtio_le16orvirtio_16, avoiding at least integer direct assignment? Wrappers likecpu_to_virtio16could return these structs and I think all compilers should emit the same code as direct assignment.Thanks!
This will break bitwise operations such as | and &. Generally Linux has solved the problem and I don't think we should go look for another solution.
Quoted 6 lines
Quoted 4 lines
Yes, but it should not block this series (we can do that in the future if we had bandwidth).
Thanks
Sorry I don't get what you are saying. Which series?
I meant this series.
Quoted 3 lines
Making LE tags structs is not going to fly, this is why sparse implements the
__bitwise__tag and in fact this is where the name comes from - bitwise operations need to work.
Yes but do we want to add sparse checks in this series only for shadow virtqueue code?
Thanks
Quoted 43 lines
Quoted 42 lines
Quoted 42 lines
Quoted 41 lines
Quoted 41 lines
Thanks
Quoted 31 lines
Quoted 30 lines
Quoted 30 lines
Thanks
Quoted 25 lines
(quoted diff) +12
-2+ } else { + svq->vring.avail->flags &= ~cpu_to_le16(VRING_AVAIL_F_NO_INTERRUPT); + } + + /* Make sure the event is enabled before the read of used_idx */ smp_mb(); return !vhost_svq_more_used(svq); } static void vhost_svq_disable_notification(VhostShadowVirtqueue *svq) { - svq->vring.avail->flags |= cpu_to_le16(VRING_AVAIL_F_NO_INTERRUPT); + /* + * No need to disable notification in the event idx case, since used event + * index is already an index too far away. + */ + if (!virtio_vdev_has_feature(svq->vdev, VIRTIO_RING_F_EVENT_IDX)) { + svq->vring.avail->flags |= cpu_to_le16(VRING_AVAIL_F_NO_INTERRUPT); + } } static uint16_t vhost_svq_last_desc_of_chain(const VhostShadowVirtqueue *svq, --2.31.1
Re: [PATCH 2/4] vhost: toggle device callbacks using used event idx
Quoted 33 lines · Jason Wang
Quoted 23 lines · Michael S. Tsirkin
Quoted 21 lines · Jason Wang
Quoted 19 lines · Michael S. Tsirkin
Quoted 6 lines · Eugenio Perez Martin
Quoted 5 lines
Quoted 2 lines
It's generally a waste that we don't use endian-ness annotations the way linux does.
Yes, it's worth doing something similar sometime.
Quoted 7 lines
Maybe we could wrap them in some struct like
virtio_le16orvirtio_16, avoiding at least integer direct assignment? Wrappers likecpu_to_virtio16could return these structs and I think all compilers should emit the same code as direct assignment.Thanks!
This will break bitwise operations such as | and &. Generally Linux has solved the problem and I don't think we should go look for another solution.
Quoted 6 lines
Quoted 4 lines
Yes, but it should not block this series (we can do that in the future if we had bandwidth).
Thanks
Sorry I don't get what you are saying. Which series?
I meant this series.
Quoted 8 lines
Quoted 3 lines
Making LE tags structs is not going to fly, this is why sparse implements the
__bitwise__tag and in fact this is where the name comes from - bitwise operations need to work.Yes but do we want to add sparse checks in this series only for shadow virtqueue code?
Thanks
No, I think a concerted effort to add endian-ness tagging is more appropriate. Has nothing to do with shadow vq.
Quoted 43 lines
Quoted 43 lines
Quoted 42 lines
Quoted 42 lines
Quoted 41 lines
Quoted 41 lines
Thanks
Quoted 31 lines
Quoted 30 lines
Quoted 30 lines
Thanks
Quoted 25 lines
(quoted diff) +12
-2+ } else { + svq->vring.avail->flags &= ~cpu_to_le16(VRING_AVAIL_F_NO_INTERRUPT); + } + + /* Make sure the event is enabled before the read of used_idx */ smp_mb(); return !vhost_svq_more_used(svq); } static void vhost_svq_disable_notification(VhostShadowVirtqueue *svq) { - svq->vring.avail->flags |= cpu_to_le16(VRING_AVAIL_F_NO_INTERRUPT); + /* + * No need to disable notification in the event idx case, since used event + * index is already an index too far away. + */ + if (!virtio_vdev_has_feature(svq->vdev, VIRTIO_RING_F_EVENT_IDX)) { + svq->vring.avail->flags |= cpu_to_le16(VRING_AVAIL_F_NO_INTERRUPT); + } } static uint16_t vhost_svq_last_desc_of_chain(const VhostShadowVirtqueue *svq, --2.31.1
Re: [PATCH 2/4] vhost: toggle device callbacks using used event idx
In reply to Michael S. TsirkinQuoted 13 lines · Michael S. Tsirkin
Quoted 11 lines · Eugenio Perez Martin
Quoted 5 lines
Quoted 2 lines
It's generally a waste that we don't use endian-ness annotations the way linux does.
Yes, it's worth doing something similar sometime.
Maybe we could wrap them in some struct like
virtio_le16orvirtio_16, avoiding at least integer direct assignment? Wrappers likecpu_to_virtio16could return these structs and I think all compilers should emit the same code as direct assignment.
Quoted 7 lines
Quoted 2 lines
Thanks!
This will break bitwise operations such as | and &. Generally Linux has solved the problem and I don't think we should go look for another solution.
That's right, we would need to do it with functions like we do with the Int128 type. The idea is the same, do not mix operations with bitwise integers and cpu type ones.
But I totally agree a sparse tag or similar is way more clean and convenient.
Quoted 38 lines
Quoted 37 lines
Quoted 37 lines
Thanks
Quoted 31 lines
Quoted 30 lines
Quoted 30 lines
Thanks
Quoted 25 lines
(quoted diff) +12
-2+ } else { + svq->vring.avail->flags &= ~cpu_to_le16(VRING_AVAIL_F_NO_INTERRUPT); + } + + /* Make sure the event is enabled before the read of used_idx */ smp_mb(); return !vhost_svq_more_used(svq); } static void vhost_svq_disable_notification(VhostShadowVirtqueue *svq) { - svq->vring.avail->flags |= cpu_to_le16(VRING_AVAIL_F_NO_INTERRUPT); + /* + * No need to disable notification in the event idx case, since used event + * index is already an index too far away. + */ + if (!virtio_vdev_has_feature(svq->vdev, VIRTIO_RING_F_EVENT_IDX)) { + svq->vring.avail->flags |= cpu_to_le16(VRING_AVAIL_F_NO_INTERRUPT); + } } static uint16_t vhost_svq_last_desc_of_chain(const VhostShadowVirtqueue *svq, --2.31.1
PATCH 4/4 vhost: Accept event idx flag
In reply to Eugenio PérezEnabling all the code path created before.
--- hw/virtio/vhost-shadow-virtqueue.c | 1 + 1 file changed, 1 insertion(+)
hw/virtio/vhost-shadow-virtqueue.c +1 -0
@@ -33,6 +33,7 @@ bool vhost_svq_valid_features(uint64_t features, Error **errp) ++b) {switch (b) {case VIRTIO_F_ANY_LAYOUT : +case VIRTIO_RING_F_EVENT_IDX :continue ;case VIRTIO_F_ACCESS_PLATFORM :
Signature · 1 line
2.31.1
Quoted 6 lines · Eugenio Pérez
Event idx helps to reduce the number of notifications between the device and the driver. It allows them to specify an index on the circular descriptors rings where to issue the notification, instead of a single binary indicator.
Adding support for SVQ.
Jason seems to be taking this through net
Quoted 18 lines
These patches are sent on top of [1] series, so trivial conflicts could arise if it is applied directly on master. Future versions can be not based on it is more convenient.
[1] https://lists.nongnu.org/archive/html/qemu-devel/2022-10/msg03280.html
Eugenio Pérez (4):
vhost: allocateevent_idxfields on vring
vhost: toggle device callbacks using used event idx
vhost: use avail event idx onvhost_svq_kick
vhost: Accept event idx flaghw/virtio/vhost-shadow-virtqueue.c | 39 ++++++++++++++++++++++++------ 1 file changed, 31 insertions(+), 8 deletions(-)Signature · 1 line
2.31.1
Quoted 11 lines · Michael S. Tsirkin
Quoted 6 lines · Eugenio Pérez
Event idx helps to reduce the number of notifications between the device and the driver. It allows them to specify an index on the circular descriptors rings where to issue the notification, instead of a single binary indicator.
Adding support for SVQ.
Jason seems to be taking this through net
Reviewed-by Michael S. Tsirkin <[email protected]>
Ok, I've queued this.
Eugenio, please post the fix for endian on top.
Thanks
Quoted 19 lines
Quoted 19 lines
These patches are sent on top of [1] series, so trivial conflicts could arise if it is applied directly on master. Future versions can be not based on it is more convenient.
[1] https://lists.nongnu.org/archive/html/qemu-devel/2022-10/msg03280.html
Eugenio Pérez (4):
vhost: allocateevent_idxfields on vring
vhost: toggle device callbacks using used event idx
vhost: use avail event idx onvhost_svq_kick
vhost: Accept event idx flaghw/virtio/vhost-shadow-virtqueue.c | 39 ++++++++++++++++++++++++------ 1 file changed, 31 insertions(+), 8 deletions(-)Signature · 1 line
2.31.1
Quoted 11 lines · Jason Wang
Quoted 8 lines · Michael S. Tsirkin
Quoted 6 lines · Eugenio Pérez
Event idx helps to reduce the number of notifications between the device and the driver. It allows them to specify an index on the circular descriptors rings where to issue the notification, instead of a single binary indicator.
Adding support for SVQ.
Quoted 9 lines
Quoted 3 lines
Jason seems to be taking this through net
Reviewed-by Michael S. Tsirkin <[email protected]>Ok, I've queued this.
Eugenio, please post the fix for endian on top.
I've got a v2 ready to send, would it be possible to send it right now and send a v2 pull? That would save a few commits where the vdpa devices do not work with big endian architectures.
Thanks!
Quoted 18 lines · Eugenio Perez Martin
Quoted 16 lines · Jason Wang
Quoted 11 lines · Michael S. Tsirkin
Quoted 6 lines · Eugenio Pérez
Event idx helps to reduce the number of notifications between the device and the driver. It allows them to specify an index on the circular descriptors rings where to issue the notification, instead of a single binary indicator.
Adding support for SVQ.
Jason seems to be taking this through net
Reviewed-by Michael S. Tsirkin <[email protected]>
Quoted 8 lines
Quoted 4 lines
Ok, I've queued this.
Eugenio, please post the fix for endian on top.
I've got a v2 ready to send, would it be possible to send it right now and send a v2 pull? That would save a few commits where the vdpa devices do not work with big endian architectures.
Probably too late, but I queue the fixes on top to rc1.
Thanks
Quoted 3 lines
Thanks!