vhost_vdpa: better PACKED support
Subsystems (2)
v2 cover vhost_vdpa: better PACKED support
While testing our vDPA driver with QEMU we found that vhost_vdpa was missing some support for PACKED VQs. Adding these helped us get further in our testing. The first patch makes sure that the vhost_vdpa vqs are given the feature flags, as done in other vhost variants. The second and third patches use the feature flags to properly format and pass set/get_vq requests.
v1:
- included wrap counter in packing answers for VHOST_GET_VRING_BASE
- added comments to vhost_virtqueue fields last_avail_idx and last_used_idx
v0:
https://lists.linuxfoundation.org/pipermail/virtualization/2023-April/066253.html
- initial version
Shannon Nelson (3):
vhost_vdpa: tell vqs about the negotiated
vhost: support PACKED when setting-getting vring_base
vhost_vdpa: support PACKED when setting-getting vring_base
drivers/vhost/vdpa.c | 34 ++++++++++++++++++++++++++++++---- drivers/vhost/vhost.c | 18 +++++++++++++----- drivers/vhost/vhost.h | 8 ++++++-- 3 files changed, 49 insertions(+), 11 deletions(-)
Signature · 6 lines
2.17.1 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
PATCH v2 1/3 vhost_vdpa: tell vqs about the negotiated
As is done in the net, iscsi, and vsock vhost support, let the vdpa vqs know about the features that have been negotiated. This allows vhost to more safely make decisions based on the features, such as when using PACKED vs split queues.
--- drivers/vhost/vdpa.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
drivers/vhost/vdpa.c +13 -0
@@ -385,7 +385,10 @@ static long vhost_vdpa_set_features(struct vhost_vdpa *v, u64 __user *featurep) {struct vdpa_device *vdpa = v->vdpa;const struct vdpa_config_ops *ops = vdpa->config; +struct vhost_dev *d = &v->vdev; +u64 actual_features;u64 features; +int i;/* * It's not allowed to change the features after they have
@@ -400,6 +403,16 @@ static long vhost_vdpa_set_features(struct vhost_vdpa *v, u64 __user *featurep)if (vdpa_set_features (vdpa, features))return -EINVAL ; +/* let the vqs know what has been configured */ + actual_features = ops->get_driver_features (vdpa); +for (i =0 ; i < d->nvqs; ++i) { +struct vhost_virtqueue *vq = d->vqs[i]; + +mutex_lock (&vq->mutex); + vq->acked_features = actual_features; +mutex_unlock (&vq->mutex); + } +return 0 ; }
Signature · 6 lines
2.17.1 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
PATCH v2 2/3 vhost: support PACKED when setting-getting vring_base
In reply to Shannon Nelson via VirtualizationUse the right structs for PACKED or split vqs when setting and getting the vring base.
--- drivers/vhost/vhost.c | 18 +++++++++++++----- drivers/vhost/vhost.h | 8 ++++++-- 2 files changed, 19 insertions(+), 7 deletions(-)
drivers/vhost/vhost.c +13 -5
@@ -1633,17 +1633,25 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg r = -EFAULT ;break ; } -if (s.num >0xffff ) { - r = -EINVAL ; -break ; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED )) { + vq->last_avail_idx = s.num &0xffff ; + vq->last_used_idx = (s.num >>16 ) &0xffff ; + }else { +if (s.num >0xffff ) { + r = -EINVAL ; +break ; + } + vq->last_avail_idx = s.num; } - vq->last_avail_idx = s.num;/* Forget the cached index value. */ vq->avail_idx = vq->last_avail_idx;break ;case VHOST_GET_VRING_BASE : s.index = idx; - s.num = vq->last_avail_idx; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED )) + s.num = (u32 )vq->last_avail_idx | ((u32 )vq->last_used_idx <<16 ); +else + s.num = vq->last_avail_idx;if (copy_to_user (argp, &s,sizeof s)) r = -EFAULT ;break ;
drivers/vhost/vhost.h +6 -2
@@ -85,13 +85,17 @@ struct vhost_virtqueue {/* The routine to call when the Guest pings us, or timeout. */ vhost_work_fn_t handle_kick; -/* Last available index we saw. */ +/* Last available index we saw. +* Values are limited to 0x7fff, and the high bit is used as +* a wrap counter when using VIRTIO_F_RING_PACKED. */ u16 last_avail_idx;/* Caches available index value from user. */ u16 avail_idx; -/* Last index we used. */ +/* Last index we used. +* Values are limited to 0x7fff, and the high bit is used as +* a wrap counter when using VIRTIO_F_RING_PACKED. */ u16 last_used_idx;/* Used flags */
Signature · 6 lines
2.17.1 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v2 2/3] vhost: support PACKED when setting-getting vring_base
Quoted 5 lines · Shannon Nelson
Use the right structs for PACKED or split vqs when setting and getting the vring base.
Signed-off-by Shannon Nelson <[email protected]>
Thanks
Quoted 65 lines
--- drivers/vhost/vhost.c | 18 +++++++++++++----- drivers/vhost/vhost.h | 8 ++++++-- 2 files changed, 19 insertions(+), 7 deletions(-)drivers/vhost/vhost.c +13
-5@@ -1633,17 +1633,25 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg r = -EFAULT ;break ; } -if (s.num >0xffff ) { - r = -EINVAL ; -break ; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED )) { + vq->last_avail_idx = s.num &0xffff ; + vq->last_used_idx = (s.num >>16 ) &0xffff ; + }else { +if (s.num >0xffff ) { + r = -EINVAL ; +break ; + } + vq->last_avail_idx = s.num; } - vq->last_avail_idx = s.num;/* Forget the cached index value. */ vq->avail_idx = vq->last_avail_idx;break ;case VHOST_GET_VRING_BASE : s.index = idx; - s.num = vq->last_avail_idx; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED )) + s.num = (u32 )vq->last_avail_idx | ((u32 )vq->last_used_idx <<16 ); +else + s.num = vq->last_avail_idx;if (copy_to_user (argp, &s,sizeof s)) r = -EFAULT ;break ;drivers/vhost/vhost.h +6
-3@@ -85,13 +85,17 @@ struct vhost_virtqueue {/* The routine to call when the Guest pings us, or timeout. */ vhost_work_fn_t handle_kick; -/* Last available index we saw. */ +/* Last available index we saw. +* Values are limited to 0x7fff, and the high bit is used as +* a wrap counter when using VIRTIO_F_RING_PACKED. */ u16 last_avail_idx;/* Caches available index value from user. */ u16 avail_idx; -/* Last index we used. */ +/* Last index we used. +* Values are limited to 0x7fff, and the high bit is used as +* a wrap counter when using VIRTIO_F_RING_PACKED. */ u16 last_used_idx;/* Used flags */ --
Quoted 2 lines
2.17.1
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v2 2/3] vhost: support PACKED when setting-getting vring_base
In reply to Shannon Nelson via VirtualizationQuoted 31 lines · Shannon Nelson via Virtualization
Use the right structs for PACKED or split vqs when setting and getting the vring base.
Signed-off-by Shannon Nelson <[email protected]>---
drivers/vhost/vhost.c| 18 +++++++++++++-----drivers/vhost/vhost.h| 8 ++++++--2 files changed, 19 insertions(+), 7 deletions(-)drivers/vhost/vhost.c +9
-4@@ -1633,17 +1633,25 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg r = -EFAULT ;break ; } -if (s.num >0xffff ) { - r = -EINVAL ; -break ; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED )) { + vq->last_avail_idx = s.num &0xffff ; + vq->last_used_idx = (s.num >>16 ) &0xffff ; + }else { +if (s.num >0xffff ) { + r = -EINVAL ; +break ; + } + vq->last_avail_idx = s.num; } - vq->last_avail_idx = s.num;
drivers/vhost/vhost.c long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg · line 1654
Quoted 10 lines
drivers/vhost/vhost.c +4
-1/* Forget the cached index value. */ vq->avail_idx = vq->last_avail_idx;break ;case VHOST_GET_VRING_BASE : s.index = idx; - s.num = vq->last_avail_idx; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED )) + s.num = (u32 )vq->last_avail_idx | ((u32 )vq->last_used_idx <<16 ); +else + s.num = vq->last_avail_idx;
The changes LGTM, but since we are changing the UAPI, should we update the documentation of VHOST_SET_VRING_BASE and VHOST_GET_VRING_BASE in include/uapi/linux/vhost.h?
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v2 2/3] vhost: support PACKED when setting-getting vring_base
Quoted 40 lines · Stefano Garzarella
Quoted 39 lines · Shannon Nelson via Virtualization
Use the right structs for PACKED or split vqs when setting and getting the vring base.
Signed-off-by Shannon Nelson <[email protected]>---
drivers/vhost/vhost.c| 18 +++++++++++++-----drivers/vhost/vhost.h| 8 ++++++--2 files changed, 19 insertions(+), 7 deletions(-)drivers/vhost/vhost.c +0
-0@@ -1633,17 +1633,25 @@ long vhost_vring_ioctl(struct vhost_dev *d,unsigned int ioctl, void
__user*arg
r =-EFAULT;
break;
}
- if (s.num >0xffff) {
- r =-EINVAL;
- break;
+ if (vhost_has_feature(vq, VIRTIO_F_RING_PACKED)) { +vq->last_avail_idx= s.num &0xffff; +vq->last_used_idx= (s.num >> 16) &0xffff; + } else {
+ if (s.num >0xffff) {
+ r =-EINVAL;
+ break;
+ }
+vq->last_avail_idx= s.num;
}
-vq->last_avail_idx= s.num;
/* Forget the cached index value. */vq->avail_idx=vq->last_avail_idx; break;
caseVHOST_GET_VRING_BASE:
s.index = idx;
- s.num =vq->last_avail_idx;
+ if (vhost_has_feature(vq, VIRTIO_F_RING_PACKED))
Quoted 8 lines
Quoted 4 lines
+ s.num = (u32)vq->last_avail_idx | ((u32)vq->last_used_idx << 16);
+ else
+ s.num =vq->last_avail_idx;The changes LGTM, but since we are changing the UAPI, should we update the documentation of
VHOST_SET_VRING_BASEandVHOST_GET_VRING_BASEininclude/uapi/linux/vhost.h?
Correct me if I'm wrong, but I don't think we're changing anything in the UAPI here, just fixing code to work correctly with what is already happening.
sln
Quoted 4 lines
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v2 2/3] vhost: support PACKED when setting-getting vring_base
Quoted 50 lines · Shannon Nelson
Quoted 40 lines · Stefano Garzarella
Quoted 39 lines · Shannon Nelson via Virtualization
Use the right structs for PACKED or split vqs when setting and getting the vring base.
Signed-off-by Shannon Nelson <[email protected]>---
drivers/vhost/vhost.c| 18 +++++++++++++-----drivers/vhost/vhost.h| 8 ++++++--2 files changed, 19 insertions(+), 7 deletions(-)drivers/vhost/vhost.c +9
-4@@ -1633,17 +1633,25 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg r = -EFAULT; break; } - if (s.num > 0xffff) { - r = -EINVAL; - break; + if (vhost_has_feature(vq, VIRTIO_F_RING_PACKED)) { + vq->last_avail_idx = s.num & 0xffff; + vq->last_used_idx = (s.num >> 16) & 0xffff; + } else { + if (s.num > 0xffff) { + r = -EINVAL; + break; + } + vq->last_avail_idx = s.num; } - vq->last_avail_idx = s.num; /* Forget the cached index value. */ vq->avail_idx = vq->last_avail_idx; break;case
VHOST_GET_VRING_BASE:
s.index = idx;
- s.num =vq->last_avail_idx;
+ if (vhost_has_feature(vq, VIRTIO_F_RING_PACKED))Quoted 8 lines
Quoted 4 lines
+ s.num = (u32)vq->last_avail_idx | ((u32)vq->last_used_idx << 16);
+ else
+ s.num =vq->last_avail_idx;The changes LGTM, but since we are changing the UAPI, should we update the documentation of
VHOST_SET_VRING_BASEandVHOST_GET_VRING_BASEininclude/uapi/linux/vhost.h?
Quoted 1 line
Correct me if I'm wrong, but I don't think we're changing anything in the UAPI here, just fixing code to work correctly with what is already happening.
IIUC before this patch VHOST_GET_VRING_BASE and VHOST_SET_VRING_BASE never worked with packed virtqueue, since we were only handling last_avail_idx. Now we are supporting packed virtqueue, handling in vhost_vring_state.num both last_avail_idx and last_used_idx (with wrap counters).
For example for VHOST_GET_VRING_BASE where is documented that the first 15 bits are last_avail_idx, the 16th the avail_wrap_counter, and the others are last_used_idx and used_wrap_counter?
Maybe I missed something, but since this is UAPI, IMHO we should document the parameters of ioctls at least in include/uapi/linux/vhost.h.
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v2 2/3] vhost: support PACKED when setting-getting vring_base
Quoted 62 lines · Stefano Garzarella
Quoted 51 lines · Shannon Nelson
Quoted 41 lines · Stefano Garzarella
Quoted 39 lines · Shannon Nelson via Virtualization
Use the right structs for PACKED or split vqs when setting and getting the vring base.
Signed-off-by Shannon Nelson <[email protected]>---
drivers/vhost/vhost.c| 18 +++++++++++++-----drivers/vhost/vhost.h| 8 ++++++--2 files changed, 19 insertions(+), 7 deletions(-)drivers/vhost/vhost.c +9
-4@@ -1633,17 +1633,25 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg r = -EFAULT; break; } - if (s.num > 0xffff) { - r = -EINVAL; - break; + if (vhost_has_feature(vq, VIRTIO_F_RING_PACKED)) { + vq->last_avail_idx = s.num & 0xffff; + vq->last_used_idx = (s.num >> 16) & 0xffff; + } else { + if (s.num > 0xffff) { + r = -EINVAL; + break; + } + vq->last_avail_idx = s.num; } - vq->last_avail_idx = s.num; /* Forget the cached index value. */ vq->avail_idx = vq->last_avail_idx; break;case
VHOST_GET_VRING_BASE:
s.index = idx;
- s.num =vq->last_avail_idx;
+ if (vhost_has_feature(vq, VIRTIO_F_RING_PACKED))Quoted 8 lines
Quoted 4 lines
+ s.num = (u32)vq->last_avail_idx | ((u32)vq->last_used_idx << 16);
+ else
+ s.num =vq->last_avail_idx;The changes LGTM, but since we are changing the UAPI, should we update the documentation of
VHOST_SET_VRING_BASEandVHOST_GET_VRING_BASEininclude/uapi/linux/vhost.h?Quoted 3 lines
Correct me if I'm wrong, but I don't think we're changing anything in the UAPI here, just fixing code to work correctly with what is already happening.
IIUC before this patch
VHOST_GET_VRING_BASEandVHOST_SET_VRING_BASEnever worked with packed virtqueue, since we were only handlinglast_avail_idx. Now we are supporting packed virtqueue, handling in vhost_vring_state.num bothlast_avail_idxandlast_used_idx(with wrap counters).
Quoted 7 lines
For example for
VHOST_GET_VRING_BASEwhere is documented that the first 15 bits arelast_avail_idx, the 16th theavail_wrap_counter, and the others arelast_used_idxandused_wrap_counter?Maybe I missed something, but since this is UAPI, IMHO we should document the parameters of ioctls at least in
include/uapi/linux/vhost.h.
Perhaps Jason already has something written up that could be put in here from when he first added the wrap_counter a couple of years ago?
sln
Quoted 8 lines
Thanks,
StefanoSignature · 3 lines
You received this message because you are subscribed to the Google Groups "Pensando Drivers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/pensando.io/d/msgid/drivers/q6cmfha36sdkgflwwd3pr4sw7rgajag4ahgjbpfjrr76w4o2b6%403yc7zs5u65s4.
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v2 2/3] vhost: support PACKED when setting-getting vring_base
Quoted 68 lines · Shannon Nelson
Quoted 66 lines · Stefano Garzarella
Quoted 51 lines · Shannon Nelson
Quoted 41 lines · Stefano Garzarella
Quoted 39 lines · Shannon Nelson via Virtualization
Use the right structs for PACKED or split vqs when setting and getting the vring base.
Signed-off-by Shannon Nelson <[email protected]>---
drivers/vhost/vhost.c| 18 +++++++++++++-----drivers/vhost/vhost.h| 8 ++++++--2 files changed, 19 insertions(+), 7 deletions(-)drivers/vhost/vhost.c +10
-5@@ -1633,17 +1633,25 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg r = -EFAULT;break ; } -if (s.num >0xffff ) { - r = -EINVAL ; -break ; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED )) { + vq->last_avail_idx = s.num &0xffff ; + vq->last_used_idx = (s.num >>16 ) &0xffff ; + }else { +if (s.num >0xffff ) { + r = -EINVAL ; +break ; + } + vq->last_avail_idx = s.num; } - vq->last_avail_idx = s.num;/* Forget the cached index value. */ vq->avail_idx = vq->last_avail_idx;break ;case VHOST_GET_VRING_BASE : s.index = idx; - s.num = vq->last_avail_idx; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED ))Quoted 8 lines
drivers/vhost/vhost.c long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg · line 1654
Quoted 4 lines
drivers/vhost/vhost.c +3
-0+ s.num = (u32 )vq->last_avail_idx | ((u32 )vq->last_used_idx <<16 ); + else + s.num = vq->last_avail_idx;The changes LGTM, but since we are changing the UAPI, should we update the documentation of
VHOST_SET_VRING_BASEandVHOST_GET_VRING_BASEininclude/uapi/linux/vhost.h?Quoted 3 lines
Correct me if I'm wrong, but I don't think we're changing anything in the UAPI here, just fixing code to work correctly with what is already happening.
IIUC before this patch
VHOST_GET_VRING_BASEandVHOST_SET_VRING_BASEnever worked with packed virtqueue, since we were only handlinglast_avail_idx. Now we are supporting packed virtqueue, handling in vhost_vring_state.num bothlast_avail_idxandlast_used_idx(with wrap counters).For example for
VHOST_GET_VRING_BASEwhere is documented that the first 15 bits arelast_avail_idx, the 16th theavail_wrap_counter, and the others arelast_used_idxandused_wrap_counter?
Quoted 6 lines
Quoted 3 lines
Maybe I missed something, but since this is UAPI, IMHO we should document the parameters of ioctls at least in
include/uapi/linux/vhost.h.Perhaps Jason already has something written up that could be put in here from when he first added the
wrap_countera couple of years ago?
If you meant the virtio driver support for packed, I think it's different from the context which is vhost here.
I agree with Stefano that we need to update the comments around GET_VRING_BASE and SET_VRING_BASE, then we are fine.
Thanks
Quoted 15 lines
sln
Quoted 11 lines
Thanks,
Stefano--
You received this message because you are subscribed to the Google Groups "Pensando Drivers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion on the web visit
https://groups.google.com/a/pensando.io/d/msgid/drivers/q6cmfha36sdkgflwwd3pr4sw7rgajag4ahgjbpfjrr76w4o2b6%403yc7zs5u65s4.
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v2 2/3] vhost: support PACKED when setting-getting vring_base
Quoted 74 lines · Jason Wang
Quoted 72 lines · Shannon Nelson
Quoted 69 lines · Stefano Garzarella
Quoted 51 lines · Shannon Nelson
Quoted 41 lines · Stefano Garzarella
Quoted 39 lines · Shannon Nelson via Virtualization
Use the right structs for PACKED or split vqs when setting and getting the vring base.
Signed-off-by Shannon Nelson <[email protected]>---
drivers/vhost/vhost.c| 18 +++++++++++++-----drivers/vhost/vhost.h| 8 ++++++--2 files changed, 19 insertions(+), 7 deletions(-)drivers/vhost/vhost.c +10
-5@@ -1633,17 +1633,25 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg r = -EFAULT;break ; } -if (s.num >0xffff ) { - r = -EINVAL ; -break ; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED )) { + vq->last_avail_idx = s.num &0xffff ; + vq->last_used_idx = (s.num >>16 ) &0xffff ; + }else { +if (s.num >0xffff ) { + r = -EINVAL ; +break ; + } + vq->last_avail_idx = s.num; } - vq->last_avail_idx = s.num;/* Forget the cached index value. */ vq->avail_idx = vq->last_avail_idx;break ;case VHOST_GET_VRING_BASE : s.index = idx; - s.num = vq->last_avail_idx; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED ))Quoted 8 lines
drivers/vhost/vhost.c long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg · line 1654
Quoted 4 lines
drivers/vhost/vhost.c +3
-0+ s.num = (u32 )vq->last_avail_idx | ((u32 )vq->last_used_idx <<16 ); + else + s.num = vq->last_avail_idx;The changes LGTM, but since we are changing the UAPI, should we update the documentation of
VHOST_SET_VRING_BASEandVHOST_GET_VRING_BASEininclude/uapi/linux/vhost.h?Quoted 3 lines
Correct me if I'm wrong, but I don't think we're changing anything in the UAPI here, just fixing code to work correctly with what is already happening.
IIUC before this patch
VHOST_GET_VRING_BASEandVHOST_SET_VRING_BASEnever worked with packed virtqueue, since we were only handlinglast_avail_idx. Now we are supporting packed virtqueue, handling in vhost_vring_state.num bothlast_avail_idxandlast_used_idx(with wrap counters).For example for
VHOST_GET_VRING_BASEwhere is documented that the first 15 bits arelast_avail_idx, the 16th theavail_wrap_counter, and the others arelast_used_idxandused_wrap_counter?Maybe I missed something, but since this is UAPI, IMHO we should document the parameters of ioctls at least in
include/uapi/linux/vhost.h.
Quoted 8 lines
Quoted 2 lines
Perhaps Jason already has something written up that could be put in here from when he first added the
wrap_countera couple of years ago?If you meant the virtio driver support for packed, I think it's different from the context which is vhost here.
I agree with Stefano that we need to update the comments around
GET_VRING_BASEandSET_VRING_BASE, then we are fine.
I'm thinking if we should also add a new VHOST_BACKEND_F_RING_PACKED feature (or something similar) to inform the user space that now we are able to handle packed virtqueue through vhost IOCTLs, otherwise how can the userspace know if it is supported or not?
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v2 2/3] vhost: support PACKED when setting-getting vring_base
Quoted 82 lines · Stefano Garzarella
Quoted 80 lines · Jason Wang
Quoted 68 lines · Shannon Nelson
Quoted 66 lines · Stefano Garzarella
Quoted 51 lines · Shannon Nelson
Quoted 41 lines · Stefano Garzarella
Quoted 39 lines · Shannon Nelson via Virtualization
Use the right structs for PACKED or split vqs when setting and getting the vring base.
Signed-off-by Shannon Nelson <[email protected]>---
drivers/vhost/vhost.c| 18 +++++++++++++-----drivers/vhost/vhost.h| 8 ++++++--2 files changed, 19 insertions(+), 7 deletions(-)drivers/vhost/vhost.c +10
-5@@ -1633,17 +1633,25 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg r = -EFAULT;break ; } -if (s.num >0xffff ) { - r = -EINVAL ; -break ; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED )) { + vq->last_avail_idx = s.num &0xffff ; + vq->last_used_idx = (s.num >>16 ) &0xffff ; + }else { +if (s.num >0xffff ) { + r = -EINVAL ; +break ; + } + vq->last_avail_idx = s.num; } - vq->last_avail_idx = s.num;/* Forget the cached index value. */ vq->avail_idx = vq->last_avail_idx;break ;case VHOST_GET_VRING_BASE : s.index = idx; - s.num = vq->last_avail_idx; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED ))Quoted 8 lines
drivers/vhost/vhost.c long vhost_vring_ioctl(struct vhost_dev · line 1656
Quoted 4 lines
drivers/vhost/vhost.c +3
-0+ s.num = (u32 )vq->last_avail_idx | ((u32 )vq->last_used_idx <<16 ); + else + s.num = vq->last_avail_idx;The changes LGTM, but since we are changing the UAPI, should we update the documentation of
VHOST_SET_VRING_BASEandVHOST_GET_VRING_BASEininclude/uapi/linux/vhost.h?Quoted 3 lines
Correct me if I'm wrong, but I don't think we're changing anything in the UAPI here, just fixing code to work correctly with what is already happening.
IIUC before this patch
VHOST_GET_VRING_BASEandVHOST_SET_VRING_BASEnever worked with packed virtqueue, since we were only handlinglast_avail_idx. Now we are supporting packed virtqueue, handling in vhost_vring_state.num bothlast_avail_idxandlast_used_idx(with wrap counters).For example for
VHOST_GET_VRING_BASEwhere is documented that the first 15 bits arelast_avail_idx, the 16th theavail_wrap_counter, and the others arelast_used_idxandused_wrap_counter?Quoted 6 lines
Quoted 3 lines
Maybe I missed something, but since this is UAPI, IMHO we should document the parameters of ioctls at least in
include/uapi/linux/vhost.h.Perhaps Jason already has something written up that could be put in here from when he first added the
wrap_countera couple of years ago?If you meant the virtio driver support for packed, I think it's different from the context which is vhost here.
Quoted 7 lines
Quoted 2 lines
I agree with Stefano that we need to update the comments around
GET_VRING_BASEandSET_VRING_BASE, then we are fine.I'm thinking if we should also add a new
VHOST_BACKEND_F_RING_PACKEDfeature (or something similar) to inform the user space that now we are able to handle packed virtqueue through vhost IOCTLs, otherwise how can the userspace know if it is supported or not?
I probably understand this but I think it should be done via VHOST_GET_FEAETURES. It would be a burden if we matianing duplicated features.
Thanks
Quoted 4 lines
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v2 2/3] vhost: support PACKED when setting-getting vring_base
Quoted 87 lines · Jason Wang
Quoted 85 lines · Stefano Garzarella
Quoted 82 lines · Jason Wang
Quoted 68 lines · Shannon Nelson
Quoted 66 lines · Stefano Garzarella
Quoted 51 lines · Shannon Nelson
Quoted 41 lines · Stefano Garzarella
Quoted 39 lines · Shannon Nelson via Virtualization
Use the right structs for PACKED or split vqs when setting and getting the vring base.
Signed-off-by Shannon Nelson <[email protected]>---
drivers/vhost/vhost.c| 18 +++++++++++++-----drivers/vhost/vhost.h| 8 ++++++--2 files changed, 19 insertions(+), 7 deletions(-)drivers/vhost/vhost.c +10
-5@@ -1633,17 +1633,25 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg r = -EFAULT;break ; } -if (s.num >0xffff ) { - r = -EINVAL ; -break ; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED )) { + vq->last_avail_idx = s.num &0xffff ; + vq->last_used_idx = (s.num >>16 ) &0xffff ; + }else { +if (s.num >0xffff ) { + r = -EINVAL ; +break ; + } + vq->last_avail_idx = s.num; } - vq->last_avail_idx = s.num;/* Forget the cached index value. */ vq->avail_idx = vq->last_avail_idx;break ;case VHOST_GET_VRING_BASE : s.index = idx; - s.num = vq->last_avail_idx; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED ))Quoted 8 lines
drivers/vhost/vhost.c long vhost_vring_ioctl(struct vhost_dev · line 1656
Quoted 4 lines
drivers/vhost/vhost.c +3
-0+ s.num = (u32 )vq->last_avail_idx | ((u32 )vq->last_used_idx <<16 ); + else + s.num = vq->last_avail_idx;The changes LGTM, but since we are changing the UAPI, should we update the documentation of
VHOST_SET_VRING_BASEandVHOST_GET_VRING_BASEininclude/uapi/linux/vhost.h?Quoted 3 lines
Correct me if I'm wrong, but I don't think we're changing anything in the UAPI here, just fixing code to work correctly with what is already happening.
IIUC before this patch
VHOST_GET_VRING_BASEandVHOST_SET_VRING_BASEnever worked with packed virtqueue, since we were only handlinglast_avail_idx. Now we are supporting packed virtqueue, handling in vhost_vring_state.num bothlast_avail_idxandlast_used_idx(with wrap counters).For example for
VHOST_GET_VRING_BASEwhere is documented that the first 15 bits arelast_avail_idx, the 16th theavail_wrap_counter, and the others arelast_used_idxandused_wrap_counter?Quoted 6 lines
Quoted 3 lines
Maybe I missed something, but since this is UAPI, IMHO we should document the parameters of ioctls at least in
include/uapi/linux/vhost.h.Perhaps Jason already has something written up that could be put in here from when he first added the
wrap_countera couple of years ago?If you meant the virtio driver support for packed, I think it's different from the context which is vhost here.
I agree with Stefano that we need to update the comments around
GET_VRING_BASEandSET_VRING_BASE, then we are fine.
Quoted 8 lines
Quoted 4 lines
I'm thinking if we should also add a new
VHOST_BACKEND_F_RING_PACKEDfeature (or something similar) to inform the user space that now we are able to handle packed virtqueue through vhost IOCTLs, otherwise how can the userspace know if it is supported or not?I probably understand this but I think it should be done via
VHOST_GET_FEAETURES. It would be a burden if we matianing duplicated features.
Good point, I see.
I think we should do one of these things, though:
- mask VIRTIO_F_RING_PACKED in the stable kernels when VHOST_GET_FEAETURES is called
- backport this patch on all stable kernels that support vhost-vdpa
Maybe the last one makes more sense.
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v2 2/3] vhost: support PACKED when setting-getting vring_base
Quoted 98 lines · Stefano Garzarella
Quoted 87 lines · Jason Wang
Quoted 85 lines · Stefano Garzarella
Quoted 82 lines · Jason Wang
Quoted 68 lines · Shannon Nelson
Quoted 66 lines · Stefano Garzarella
Quoted 51 lines · Shannon Nelson
Quoted 41 lines · Stefano Garzarella
Quoted 39 lines · Shannon Nelson via Virtualization
Use the right structs for PACKED or split vqs when setting and getting the vring base.
Signed-off-by Shannon Nelson <[email protected]>---
drivers/vhost/vhost.c| 18 +++++++++++++-----drivers/vhost/vhost.h| 8 ++++++--2 files changed, 19 insertions(+), 7 deletions(-)drivers/vhost/vhost.c +10
-5@@ -1633,17 +1633,25 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg r = -EFAULT;break ; } -if (s.num >0xffff ) { - r = -EINVAL ; -break ; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED )) { + vq->last_avail_idx = s.num &0xffff ; + vq->last_used_idx = (s.num >>16 ) &0xffff ; + }else { +if (s.num >0xffff ) { + r = -EINVAL ; +break ; + } + vq->last_avail_idx = s.num; } - vq->last_avail_idx = s.num;/* Forget the cached index value. */ vq->avail_idx = vq->last_avail_idx;break ;case VHOST_GET_VRING_BASE : s.index = idx; - s.num = vq->last_avail_idx; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED ))Quoted 8 lines
drivers/vhost/vhost.c long vhost_vring_ioctl(struct vhost_dev · line 1656
Quoted 4 lines
drivers/vhost/vhost.c +3
-0+ s.num = (u32 )vq->last_avail_idx | ((u32 )vq->last_used_idx <<16 ); + else + s.num = vq->last_avail_idx;The changes LGTM, but since we are changing the UAPI, should we update the documentation of
VHOST_SET_VRING_BASEandVHOST_GET_VRING_BASEininclude/uapi/linux/vhost.h?Quoted 3 lines
Correct me if I'm wrong, but I don't think we're changing anything in the UAPI here, just fixing code to work correctly with what is already happening.
IIUC before this patch
VHOST_GET_VRING_BASEandVHOST_SET_VRING_BASEnever worked with packed virtqueue, since we were only handlinglast_avail_idx. Now we are supporting packed virtqueue, handling in vhost_vring_state.num bothlast_avail_idxandlast_used_idx(with wrap counters).For example for
VHOST_GET_VRING_BASEwhere is documented that the first 15 bits arelast_avail_idx, the 16th theavail_wrap_counter, and the others arelast_used_idxandused_wrap_counter?Quoted 6 lines
Quoted 3 lines
Maybe I missed something, but since this is UAPI, IMHO we should document the parameters of ioctls at least in
include/uapi/linux/vhost.h.Perhaps Jason already has something written up that could be put in here from when he first added the
wrap_countera couple of years ago?If you meant the virtio driver support for packed, I think it's different from the context which is vhost here.
I agree with Stefano that we need to update the comments around
GET_VRING_BASEandSET_VRING_BASE, then we are fine.Quoted 8 lines
Quoted 4 lines
I'm thinking if we should also add a new
VHOST_BACKEND_F_RING_PACKEDfeature (or something similar) to inform the user space that now we are able to handle packed virtqueue through vhost IOCTLs, otherwise how can the userspace know if it is supported or not?I probably understand this but I think it should be done via
VHOST_GET_FEAETURES. It would be a burden if we matianing duplicated features.
Quoted 8 lines
Good point, I see.
I think we should do one of these things, though:
- maskVIRTIO_F_RING_PACKEDin the stable kernels whenVHOST_GET_FEAETURESis called
- backport this patch on all stable kernels that support vhost-vdpaMaybe the last one makes more sense.
Not sure, it looks to me the packed support for vDPA was first added by Gautam. So it probably means that except for vp_vdpa, we don't have a vDPA parent that can do the packed virtuque now. Adding the relevant people here for more comment
Thanks
Quoted 4 lines
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Michael S. Tsirkin
maintainerRe: [PATCH v2 2/3] vhost: support PACKED when setting-getting vring_base
Quoted 106 lines · Jason Wang
Quoted 105 lines · Stefano Garzarella
Quoted 87 lines · Jason Wang
Quoted 85 lines · Stefano Garzarella
Quoted 82 lines · Jason Wang
Quoted 68 lines · Shannon Nelson
Quoted 66 lines · Stefano Garzarella
Quoted 51 lines · Shannon Nelson
Quoted 41 lines · Stefano Garzarella
Quoted 39 lines · Shannon Nelson via Virtualization
Use the right structs for PACKED or split vqs when setting and getting the vring base.
Signed-off-by Shannon Nelson <[email protected]>---
drivers/vhost/vhost.c| 18 +++++++++++++-----drivers/vhost/vhost.h| 8 ++++++--2 files changed, 19 insertions(+), 7 deletions(-)drivers/vhost/vhost.c +10
-5@@ -1633,17 +1633,25 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg r = -EFAULT;break ; } -if (s.num >0xffff ) { - r = -EINVAL ; -break ; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED )) { + vq->last_avail_idx = s.num &0xffff ; + vq->last_used_idx = (s.num >>16 ) &0xffff ; + }else { +if (s.num >0xffff ) { + r = -EINVAL ; +break ; + } + vq->last_avail_idx = s.num; } - vq->last_avail_idx = s.num;/* Forget the cached index value. */ vq->avail_idx = vq->last_avail_idx;break ;case VHOST_GET_VRING_BASE : s.index = idx; - s.num = vq->last_avail_idx; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED ))Quoted 8 lines
drivers/vhost/vhost.c long vhost_vring_ioctl(struct vhost_dev · line 1656
Quoted 4 lines
drivers/vhost/vhost.c +3
-0+ s.num = (u32 )vq->last_avail_idx | ((u32 )vq->last_used_idx <<16 ); + else + s.num = vq->last_avail_idx;The changes LGTM, but since we are changing the UAPI, should we update the documentation of
VHOST_SET_VRING_BASEandVHOST_GET_VRING_BASEininclude/uapi/linux/vhost.h?Quoted 3 lines
Correct me if I'm wrong, but I don't think we're changing anything in the UAPI here, just fixing code to work correctly with what is already happening.
IIUC before this patch
VHOST_GET_VRING_BASEandVHOST_SET_VRING_BASEnever worked with packed virtqueue, since we were only handlinglast_avail_idx. Now we are supporting packed virtqueue, handling in vhost_vring_state.num bothlast_avail_idxandlast_used_idx(with wrap counters).For example for
VHOST_GET_VRING_BASEwhere is documented that the first 15 bits arelast_avail_idx, the 16th theavail_wrap_counter, and the others arelast_used_idxandused_wrap_counter?Quoted 6 lines
Quoted 3 lines
Maybe I missed something, but since this is UAPI, IMHO we should document the parameters of ioctls at least in
include/uapi/linux/vhost.h.Perhaps Jason already has something written up that could be put in here from when he first added the
wrap_countera couple of years ago?If you meant the virtio driver support for packed, I think it's different from the context which is vhost here.
I agree with Stefano that we need to update the comments around
GET_VRING_BASEandSET_VRING_BASE, then we are fine.Quoted 8 lines
Quoted 4 lines
I'm thinking if we should also add a new
VHOST_BACKEND_F_RING_PACKEDfeature (or something similar) to inform the user space that now we are able to handle packed virtqueue through vhost IOCTLs, otherwise how can the userspace know if it is supported or not?I probably understand this but I think it should be done via
VHOST_GET_FEAETURES. It would be a burden if we matianing duplicated features.Good point, I see.
I think we should do one of these things, though:
- maskVIRTIO_F_RING_PACKEDin the stable kernels whenVHOST_GET_FEAETURESis called
- backport this patch on all stable kernels that support vhost-vdpa
Quoted 8 lines
Quoted 1 line
Maybe the last one makes more sense.
Not sure, it looks to me the packed support for vDPA was first added by Gautam. So it probably means that except for
vp_vdpa, we don't have a vDPA parent that can do the packed virtuque now. Adding the relevant people here for more commentThanks
BTW should we fix up vhost.c to support packed rings too? E.g. so we can migrate to vhost?
There's an old patchset of mine that started work on this:
https://lore.kernel.org/all/20200407011612.478226-1-mst%40redhat.com
Is there need for this now?
Quoted 4 lines
Quoted 4 lines
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v2 2/3] vhost: support PACKED when setting-getting vring_base
Quoted 115 lines · Michael S. Tsirkin
Quoted 113 lines · Jason Wang
Quoted 98 lines · Stefano Garzarella
Quoted 87 lines · Jason Wang
Quoted 85 lines · Stefano Garzarella
Quoted 82 lines · Jason Wang
Quoted 68 lines · Shannon Nelson
Quoted 66 lines · Stefano Garzarella
Quoted 51 lines · Shannon Nelson
Quoted 41 lines · Stefano Garzarella
Quoted 39 lines · Shannon Nelson via Virtualization
Use the right structs for PACKED or split vqs when setting and getting the vring base.
Signed-off-by Shannon Nelson <[email protected]>---
drivers/vhost/vhost.c| 18 +++++++++++++-----drivers/vhost/vhost.h| 8 ++++++--2 files changed, 19 insertions(+), 7 deletions(-)drivers/vhost/vhost.c +10
-5@@ -1633,17 +1633,25 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg r = -EFAULT;break ; } -if (s.num >0xffff ) { - r = -EINVAL ; -break ; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED )) { + vq->last_avail_idx = s.num &0xffff ; + vq->last_used_idx = (s.num >>16 ) &0xffff ; + }else { +if (s.num >0xffff ) { + r = -EINVAL ; +break ; + } + vq->last_avail_idx = s.num; } - vq->last_avail_idx = s.num;/* Forget the cached index value. */ vq->avail_idx = vq->last_avail_idx;break ;case VHOST_GET_VRING_BASE : s.index = idx; - s.num = vq->last_avail_idx; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED ))Quoted 8 lines
drivers/vhost/vhost.c long vhost_vring_ioctl(struct vhost_dev · line 1656
Quoted 4 lines
drivers/vhost/vhost.c +3
-0+ s.num = (u32 )vq->last_avail_idx | ((u32 )vq->last_used_idx <<16 ); + else + s.num = vq->last_avail_idx;The changes LGTM, but since we are changing the UAPI, should we update the documentation of
VHOST_SET_VRING_BASEandVHOST_GET_VRING_BASEininclude/uapi/linux/vhost.h?Quoted 3 lines
Correct me if I'm wrong, but I don't think we're changing anything in the UAPI here, just fixing code to work correctly with what is already happening.
IIUC before this patch
VHOST_GET_VRING_BASEandVHOST_SET_VRING_BASEnever worked with packed virtqueue, since we were only handlinglast_avail_idx. Now we are supporting packed virtqueue, handling in vhost_vring_state.num bothlast_avail_idxandlast_used_idx(with wrap counters).For example for
VHOST_GET_VRING_BASEwhere is documented that the first 15 bits arelast_avail_idx, the 16th theavail_wrap_counter, and the others arelast_used_idxandused_wrap_counter?Quoted 6 lines
Quoted 3 lines
Maybe I missed something, but since this is UAPI, IMHO we should document the parameters of ioctls at least in
include/uapi/linux/vhost.h.Perhaps Jason already has something written up that could be put in here from when he first added the
wrap_countera couple of years ago?If you meant the virtio driver support for packed, I think it's different from the context which is vhost here.
I agree with Stefano that we need to update the comments around
GET_VRING_BASEandSET_VRING_BASE, then we are fine.Quoted 8 lines
Quoted 4 lines
I'm thinking if we should also add a new
VHOST_BACKEND_F_RING_PACKEDfeature (or something similar) to inform the user space that now we are able to handle packed virtqueue through vhost IOCTLs, otherwise how can the userspace know if it is supported or not?I probably understand this but I think it should be done via
VHOST_GET_FEAETURES. It would be a burden if we matianing duplicated features.Quoted 8 lines
Good point, I see.
I think we should do one of these things, though:
- maskVIRTIO_F_RING_PACKEDin the stable kernels whenVHOST_GET_FEAETURESis called
- backport this patch on all stable kernels that support vhost-vdpaMaybe the last one makes more sense.
Not sure, it looks to me the packed support for vDPA was first added by Gautam. So it probably means that except for
vp_vdpa, we don't have a vDPA parent that can do the packed virtuque now. Adding the relevant people here for more comment
Quoted 4 lines
Quoted 1 line
Thanks
BTW should we fix up
vhost.cto support packed rings too? E.g. so we can migrate to vhost?
Better to have.
Quoted 5 lines
There's an old patchset of mine that started work on this:
https://lore.kernel.org/all/20200407011612.478226-1-mst%40redhat.com
Is there need for this now?
Is there any benchmark for that?
Thanks
Quoted 6 lines
Quoted 5 lines
Quoted 5 lines
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Michael S. Tsirkin
maintainerRe: [PATCH v2 2/3] vhost: support PACKED when setting-getting vring_base
Quoted 125 lines · Jason Wang
Quoted 115 lines · Michael S. Tsirkin
Quoted 113 lines · Jason Wang
Quoted 98 lines · Stefano Garzarella
Quoted 87 lines · Jason Wang
Quoted 85 lines · Stefano Garzarella
Quoted 82 lines · Jason Wang
Quoted 68 lines · Shannon Nelson
Quoted 66 lines · Stefano Garzarella
Quoted 51 lines · Shannon Nelson
Quoted 41 lines · Stefano Garzarella
Quoted 39 lines · Shannon Nelson via Virtualization
Use the right structs for PACKED or split vqs when setting and getting the vring base.
Signed-off-by Shannon Nelson <[email protected]>---
drivers/vhost/vhost.c| 18 +++++++++++++-----drivers/vhost/vhost.h| 8 ++++++--2 files changed, 19 insertions(+), 7 deletions(-)drivers/vhost/vhost.c +10
-5@@ -1633,17 +1633,25 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg r = -EFAULT;break ; } -if (s.num >0xffff ) { - r = -EINVAL ; -break ; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED )) { + vq->last_avail_idx = s.num &0xffff ; + vq->last_used_idx = (s.num >>16 ) &0xffff ; + }else { +if (s.num >0xffff ) { + r = -EINVAL ; +break ; + } + vq->last_avail_idx = s.num; } - vq->last_avail_idx = s.num;/* Forget the cached index value. */ vq->avail_idx = vq->last_avail_idx;break ;case VHOST_GET_VRING_BASE : s.index = idx; - s.num = vq->last_avail_idx; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED ))Quoted 8 lines
drivers/vhost/vhost.c long vhost_vring_ioctl(struct vhost_dev · line 1656
Quoted 4 lines
drivers/vhost/vhost.c +3
-0+ s.num = (u32 )vq->last_avail_idx | ((u32 )vq->last_used_idx <<16 ); + else + s.num = vq->last_avail_idx;The changes LGTM, but since we are changing the UAPI, should we update the documentation of
VHOST_SET_VRING_BASEandVHOST_GET_VRING_BASEininclude/uapi/linux/vhost.h?Quoted 3 lines
Correct me if I'm wrong, but I don't think we're changing anything in the UAPI here, just fixing code to work correctly with what is already happening.
IIUC before this patch
VHOST_GET_VRING_BASEandVHOST_SET_VRING_BASEnever worked with packed virtqueue, since we were only handlinglast_avail_idx. Now we are supporting packed virtqueue, handling in vhost_vring_state.num bothlast_avail_idxandlast_used_idx(with wrap counters).For example for
VHOST_GET_VRING_BASEwhere is documented that the first 15 bits arelast_avail_idx, the 16th theavail_wrap_counter, and the others arelast_used_idxandused_wrap_counter?Quoted 6 lines
Quoted 3 lines
Maybe I missed something, but since this is UAPI, IMHO we should document the parameters of ioctls at least in
include/uapi/linux/vhost.h.Perhaps Jason already has something written up that could be put in here from when he first added the
wrap_countera couple of years ago?If you meant the virtio driver support for packed, I think it's different from the context which is vhost here.
I agree with Stefano that we need to update the comments around
GET_VRING_BASEandSET_VRING_BASE, then we are fine.Quoted 8 lines
Quoted 4 lines
I'm thinking if we should also add a new
VHOST_BACKEND_F_RING_PACKEDfeature (or something similar) to inform the user space that now we are able to handle packed virtqueue through vhost IOCTLs, otherwise how can the userspace know if it is supported or not?I probably understand this but I think it should be done via
VHOST_GET_FEAETURES. It would be a burden if we matianing duplicated features.Quoted 8 lines
Good point, I see.
I think we should do one of these things, though:
- maskVIRTIO_F_RING_PACKEDin the stable kernels whenVHOST_GET_FEAETURESis called
- backport this patch on all stable kernels that support vhost-vdpaMaybe the last one makes more sense.
Not sure, it looks to me the packed support for vDPA was first added by Gautam. So it probably means that except for
vp_vdpa, we don't have a vDPA parent that can do the packed virtuque now. Adding the relevant people here for more commentQuoted 4 lines
Quoted 1 line
Thanks
BTW should we fix up
vhost.cto support packed rings too? E.g. so we can migrate to vhost?Better to have.
Quoted 2 lines
There's an old patchset of mine that started work on this:
Quoted 7 lines
Quoted 3 lines
https://lore.kernel.org/all/20200407011612.478226-1-mst%40redhat.com
Is there need for this now?
Is there any benchmark for that?
Thanks
I used one in tools/virtio, there was no real change. Might make sense to remake it since we fixed some bugs in EVENT_IDX support and things might have changed.
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Quoted 5 lines
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Michael S. Tsirkin
maintainerRe: [PATCH v2 2/3] vhost: support PACKED when setting-getting vring_base
In reply to Stefano GarzarellaQuoted 9 lines · Stefano Garzarella
I think we should do one of these things, though:
- maskVIRTIO_F_RING_PACKEDin the stable kernels whenVHOST_GET_FEAETURESis called
- backport this patch on all stable kernels that support vhost-vdpaMaybe the last one makes more sense.
Thanks,
Stefano
OK which patches do you want to go to stable exactly?
Signature · 6 lines
MST _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v2 2/3] vhost: support PACKED when setting-getting vring_base
Quoted 6 lines · Michael S. Tsirkin
Quoted 5 lines · Stefano Garzarella
I think we should do one of these things, though:
- maskVIRTIO_F_RING_PACKEDin the stable kernels whenVHOST_GET_FEAETURESis called
- backport this patch on all stable kernels that support vhost-vdpa
Quoted 6 lines
Quoted 4 lines
Maybe the last one makes more sense.
Thanks,
StefanoOK which patches do you want to go to stable exactly?
Initially I was thinking this entire series, but I think it's too risky, so on second thoughtwhat do you think about this:
drivers/vhost/vdpa.c +6 -0
@@ -397,6 +397,12 @@ static long vhost_vdpa_get_features(struct vhost_vdpa *v, u64 __user *featurep) features = ops->get_device_features (vdpa); +/* +* IOCTLs (eg. VHOST_GET_VRING_BASE, VHOST_SET_VRING_BASE) don't support +* packed virtqueue well yet, so let's filter the feature for now. +*/ + features &= ~BIT_ULL (VIRTIO_F_RING_PACKED ); +if (copy_to_user (featurep, &features,sizeof (features)))return -EFAULT ;
I can send the patch ASAP and we can apply it before this series.
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
PATCH v2 3/3 vhost_vdpa: support PACKED when setting-getting vring_base
In reply to Shannon Nelson via VirtualizationUse the right structs for PACKED or split vqs when setting and getting the vring base.
--- drivers/vhost/vdpa.c | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-)
drivers/vhost/vdpa.c +17 -4
@@ -585,7 +585,14 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,if (r)return r; - vq->last_avail_idx = vq_state.split.avail_index; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED )) { + vq->last_avail_idx = vq_state.packed.last_avail_idx | + (vq_state.packed.last_avail_counter <<15 ); + vq->last_used_idx = vq_state.packed.last_used_idx | + (vq_state.packed.last_used_counter <<15 ); + }else { + vq->last_avail_idx = vq_state.split.avail_index; + }break ; }
@@ -603,9 +610,15 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,break ;case VHOST_SET_VRING_BASE : - vq_state.split.avail_index = vq->last_avail_idx; -if (ops->set_vq_state (vdpa, idx, &vq_state)) - r = -EINVAL ; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED )) { + vq_state.packed.last_avail_idx = vq->last_avail_idx &0x7fff ; + vq_state.packed.last_avail_counter = !!(vq->last_avail_idx &0x8000 ); + vq_state.packed.last_used_idx = vq->last_used_idx &0x7fff ; + vq_state.packed.last_used_counter = !!(vq->last_used_idx &0x8000 ); + }else { + vq_state.split.avail_index = vq->last_avail_idx; + } + r = ops->set_vq_state (vdpa, idx, &vq_state);break ;case VHOST_SET_VRING_CALL :
Signature · 6 lines
2.17.1 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v2 3/3] vhost_vdpa: support PACKED when setting-getting vring_base
Quoted 5 lines · Shannon Nelson
Use the right structs for PACKED or split vqs when setting and getting the vring base.
Signed-off-by Shannon Nelson <[email protected]>
Thanks
Quoted 44 lines
--- drivers/vhost/vdpa.c | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-)drivers/vhost/vdpa.c +17
-5@@ -585,7 +585,14 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,if (r)return r; - vq->last_avail_idx = vq_state.split.avail_index; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED )) { + vq->last_avail_idx = vq_state.packed.last_avail_idx | + (vq_state.packed.last_avail_counter <<15 ); + vq->last_used_idx = vq_state.packed.last_used_idx | + (vq_state.packed.last_used_counter <<15 ); + }else { + vq->last_avail_idx = vq_state.split.avail_index; + }break ; }@@ -603,9 +610,15 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,break ;case VHOST_SET_VRING_BASE : - vq_state.split.avail_index = vq->last_avail_idx; -if (ops->set_vq_state (vdpa, idx, &vq_state)) - r = -EINVAL ; +if (vhost_has_feature (vq,VIRTIO_F_RING_PACKED )) { + vq_state.packed.last_avail_idx = vq->last_avail_idx &0x7fff ; + vq_state.packed.last_avail_counter = !!(vq->last_avail_idx &0x8000 ); + vq_state.packed.last_used_idx = vq->last_used_idx &0x7fff ; + vq_state.packed.last_used_counter = !!(vq->last_used_idx &0x8000 ); + }else { + vq_state.split.avail_index = vq->last_avail_idx; + } + r = ops->set_vq_state (vdpa, idx, &vq_state);break ;case VHOST_SET_VRING_CALL : --
Quoted 2 lines
2.17.1
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Michael S. Tsirkin
maintainerQuoted 6 lines · Shannon Nelson
While testing our vDPA driver with QEMU we found that
vhost_vdpawas missing some support for PACKED VQs. Adding these helped us get further in our testing. The first patch makes sure that thevhost_vdpavqs are given the feature flags, as done in other vhost variants. The second and third patches use the feature flags to properly format and pass set/get_vq requests.
This missed the merge window most likely, unless there's a reason to send a second pull towards the end of it. We'll see.
Quoted 13 lines
v1:
- included wrap counter in packing answers forVHOST_GET_VRING_BASE
- added comments tovhost_virtqueuefieldslast_avail_idxandlast_used_idxv0:
https://lists.linuxfoundation.org/pipermail/virtualization/2023-April/066253.html- initial version
Shannon Nelson (3):
vhost_vdpa: tell vqs about the negotiated
vhost: support PACKED when setting-gettingvring_base
vhost_vdpa: support PACKED when setting-gettingvring_base
Quoted 7 lines
drivers/vhost/vdpa.c | 34 ++++++++++++++++++++++++++++++---- drivers/vhost/vhost.c | 18 +++++++++++++----- drivers/vhost/vhost.h | 8 ++++++-- 3 files changed, 49 insertions(+), 11 deletions(-)Signature · 1 line
2.17.1
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 11 lines · Michael S. Tsirkin
Quoted 6 lines · Shannon Nelson
While testing our vDPA driver with QEMU we found that
vhost_vdpawas missing some support for PACKED VQs. Adding these helped us get further in our testing. The first patch makes sure that thevhost_vdpavqs are given the feature flags, as done in other vhost variants. The second and third patches use the feature flags to properly format and pass set/get_vq requests.This missed the merge window most likely, unless there's a reason to send a second pull towards the end of it. We'll see.
Understood - we'll keep our fingers crossed for good luck :-) Thanks for the help.
sln
Quoted 14 lines
Quoted 14 lines
v1:
- included wrap counter in packing answers forVHOST_GET_VRING_BASE
- added comments tovhost_virtqueuefieldslast_avail_idxandlast_used_idxv0:
https://lists.linuxfoundation.org/pipermail/virtualization/2023-April/066253.html- initial version
Shannon Nelson (3):
vhost_vdpa: tell vqs about the negotiated
vhost: support PACKED when setting-gettingvring_base
vhost_vdpa: support PACKED when setting-gettingvring_base
Quoted 8 lines
Quoted 7 lines
drivers/vhost/vdpa.c | 34 ++++++++++++++++++++++++++++++---- drivers/vhost/vhost.c | 18 +++++++++++++----- drivers/vhost/vhost.h | 8 ++++++-- 3 files changed, 49 insertions(+), 11 deletions(-)--
2.17.1
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization