Vhost-vdpa Shadow Virtqueue Offloads support
v2 cover Vhost-vdpa Shadow Virtqueue Offloads support
This series enables shadowed CVQ to intercept Offloads commands through shadowed CVQ, update the virtio NIC device model so qemu send it in a migration, and the restore of that Offloads state in the destination.
Changelog
=========
v2:
- make some function arguments const
- reuse virtio_vdev_has_feature() suggested by Eugenio and Jason
- avoid sending CVQ command in default state suggested by Eugenio
v1: https://lore.kernel.org/all/[email protected]/
Hawkins Jiawei (6):
include/hw/virtio: make some VirtIODevice const
vdpa: reuse virtio_vdev_has_feature() hw/net/virtio-net: make some VirtIONet const
virtio-net: expose virtio_net_supported_guest_offloads()
vdpa: Add vhost_vdpa_net_load_offloads()
vdpa: Allow VIRTIO_NET_F_CTRL_GUEST_OFFLOADS in SVQ
hw/net/virtio-net.c | 2 +- include/hw/virtio/virtio-net.h | 1 + include/hw/virtio/virtio.h | 2 +- net/vhost-vdpa.c | 45 +++++++++++++++++++++++++++++++--- 4 files changed, 44 insertions(+), 6 deletions(-)
Signature · 1 line
2.25.1
PATCH v2 1/6 include/hw/virtio: make some VirtIODevice const
The VirtIODevice structure is not modified in virtio_vdev_has_feature(). Therefore, make it const to allow this function to accept const variables.
--- include/hw/virtio/virtio.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
include/hw/virtio/virtio.h +1 -1
@@ -402,7 +402,7 @@ static inline bool virtio_has_feature(uint64_t features, unsigned int fbit)return !!(features & (1ULL << fbit)); } -static inline bool virtio_vdev_has_feature (VirtIODevice *vdev, +static inline bool virtio_vdev_has_feature (const VirtIODevice *vdev,unsigned int fbit) {return virtio_has_feature (vdev->guest_features, fbit);
Signature · 1 line
2.25.1
PATCH v2 2/6 vdpa: reuse virtio_vdev_has_feature()
In reply to Hawkins JiaweiWe can use virtio_vdev_has_feature() instead of manually accessing the features.
--- net/vhost-vdpa.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
net/vhost-vdpa.c +2 -4
@@ -643,8 +643,7 @@ static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, uint8_t class,static int vhost_vdpa_net_load_mac (VhostVDPAState *s,const VirtIONet *n) { -uint64_t features = n->parent_obj.guest_features; -if (features &BIT_ULL (VIRTIO_NET_F_CTRL_MAC_ADDR )) { +if (virtio_vdev_has_feature (&n->parent_obj,VIRTIO_NET_F_CTRL_MAC_ADDR )) {ssize_t dev_written =vhost_vdpa_net_load_cmd (s,VIRTIO_NET_CTRL_MAC ,VIRTIO_NET_CTRL_MAC_ADDR_SET , n->mac,sizeof (n->mac));
@@ -662,10 +661,9 @@ static int vhost_vdpa_net_load_mq(VhostVDPAState *s, const VirtIONet *n) {struct virtio_net_ctrl_mq mq; -uint64_t features = n->parent_obj.guest_features;ssize_t dev_written; -if (!(features &BIT_ULL (VIRTIO_NET_F_MQ ))) { +if (!virtio_vdev_has_feature (&n->parent_obj,VIRTIO_NET_F_MQ )) {return 0 ; }
Signature · 1 line
2.25.1
PATCH v2 3/6 hw/net/virtio-net: make some VirtIONet const
In reply to Hawkins JiaweiThe VirtIONet structure is not modified in virtio_net_supported_guest_offloads(). Therefore, make it const to allow this function to accept const variables.
--- hw/net/virtio-net.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
hw/net/virtio-net.c +1 -1
@@ -874,7 +874,7 @@ static uint64_t virtio_net_guest_offloads_by_features(uint32_t features)return guest_offloads_mask & features; } -static inline uint64_t virtio_net_supported_guest_offloads (VirtIONet *n) +static inline uint64_t virtio_net_supported_guest_offloads (const VirtIONet *n) {VirtIODevice *vdev =VIRTIO_DEVICE (n);return virtio_net_guest_offloads_by_features (vdev->guest_features);
Signature · 1 line
2.25.1
PATCH v2 4/6 virtio-net: expose virtio_net_supported_guest_offloads()
In reply to Hawkins JiaweiTo support restoring offloads state in vdpa, need to expose the function virtio_net_supported_guest_offloads(), then vdpa uses this function to get the guest supported offloads.
--- hw/net/virtio-net.c | 2 +- include/hw/virtio/virtio-net.h | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)
hw/net/virtio-net.c +1 -1
@@ -874,7 +874,7 @@ static uint64_t virtio_net_guest_offloads_by_features(uint32_t features)return guest_offloads_mask & features; } -static inline uint64_t virtio_net_supported_guest_offloads (const VirtIONet *n) +uint64_t virtio_net_supported_guest_offloads (const VirtIONet *n) {VirtIODevice *vdev =VIRTIO_DEVICE (n);return virtio_net_guest_offloads_by_features (vdev->guest_features);
include/hw/virtio/virtio-net.h +1 -0
@@ -227,5 +227,6 @@ size_t virtio_net_handle_ctrl_iov(VirtIODevice *vdev, unsigned out_num);void virtio_net_set_netclient_name (VirtIONet *n,const char *name,const char *type); +uint64_t virtio_net_supported_guest_offloads (const VirtIONet *n);#endif
Signature · 1 line
2.25.1
Re: [PATCH v2 4/6] virtio-net: expose virtio_net_supported_guest_offloads()
Quoted 4 lines · Hawkins Jiawei
To support restoring offloads state in vdpa, need to expose the function
virtio_net_supported_guest_offloads(), then vdpa uses this function to get the guest supported offloads.
Here it should be changed to "then QEMU uses this function to get the guest supported offloads.". I will correct the commit message in the v3 patch.
Thanks!
Quoted 34 lines
Signed-off-by Hawkins Jiawei <[email protected]>--- hw/net/virtio-net.c | 2 +- include/hw/virtio/virtio-net.h | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)hw/net/virtio-net.c +1
-1@@ -874,7 +874,7 @@ static uint64_t virtio_net_guest_offloads_by_features(uint32_t features)return guest_offloads_mask & features; } -static inline uint64_t virtio_net_supported_guest_offloads (const VirtIONet *n) +uint64_t virtio_net_supported_guest_offloads (const VirtIONet *n) {VirtIODevice *vdev =VIRTIO_DEVICE (n);return virtio_net_guest_offloads_by_features (vdev->guest_features);include/hw/virtio/virtio-net.h +1
-1@@ -227,5 +227,6 @@ size_t virtio_net_handle_ctrl_iov(VirtIODevice *vdev, unsigned out_num);void virtio_net_set_netclient_name (VirtIONet *n,const char *name,const char *type); +uint64_t virtio_net_supported_guest_offloads (const VirtIONet *n);#endif --2.25.1
Re: [PATCH v2 4/6] virtio-net: expose virtio_net_supported_guest_offloads()
Quoted 11 lines · Hawkins Jiawei
Quoted 4 lines · Hawkins Jiawei
To support restoring offloads state in vdpa, need to expose the function
virtio_net_supported_guest_offloads(), then vdpa uses this function to get the guest supported offloads.Here it should be changed to "then QEMU uses this function to get the guest supported offloads.". I will correct the commit message in the v3 patch.
Maybe "to get the device supported offloads allow qemu to know the defaults, so it can skip the control message sending if they match with the driver's configuration"?
We can also add "This will be the default at guest's startup, these values can mismatch only at live migration".
What do you think?
Thanks!
Quoted 37 lines
Thanks!
Quoted 34 lines
Signed-off-by Hawkins Jiawei <[email protected]>--- hw/net/virtio-net.c | 2 +- include/hw/virtio/virtio-net.h | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)hw/net/virtio-net.c +1
-1@@ -874,7 +874,7 @@ static uint64_t virtio_net_guest_offloads_by_features(uint32_t features)return guest_offloads_mask & features; } -static inline uint64_t virtio_net_supported_guest_offloads (const VirtIONet *n) +uint64_t virtio_net_supported_guest_offloads (const VirtIONet *n) {VirtIODevice *vdev =VIRTIO_DEVICE (n);return virtio_net_guest_offloads_by_features (vdev->guest_features);include/hw/virtio/virtio-net.h +1
-1@@ -227,5 +227,6 @@ size_t virtio_net_handle_ctrl_iov(VirtIODevice *vdev, unsigned out_num);void virtio_net_set_netclient_name (VirtIONet *n,const char *name,const char *type); +uint64_t virtio_net_supported_guest_offloads (const VirtIONet *n);#endif --2.25.1
Re: [PATCH v2 4/6] virtio-net: expose virtio_net_supported_guest_offloads()
Quoted 13 lines · Eugenio Perez Martin
Quoted 11 lines · Hawkins Jiawei
Quoted 4 lines · Hawkins Jiawei
To support restoring offloads state in vdpa, need to expose the function
virtio_net_supported_guest_offloads(), then vdpa uses this function to get the guest supported offloads.Here it should be changed to "then QEMU uses this function to get the guest supported offloads.". I will correct the commit message in the v3 patch.
Quoted 6 lines
Maybe "to get the device supported offloads allow qemu to know the defaults, so it can skip the control message sending if they match with the driver's configuration"?
We can also add "This will be the default at guest's startup, these values can mismatch only at live migration".
Hi Eugenio,
I sent the v3 patch before seeing this email:(.
I refactor the commit message to
"To support restoring offloads state in vdpa, need to expose the function virtio_net_supported_guest_offloads(). QEMU uses this function to get the guest supported offloads and no needs to send the corresponding CVQ command if guest enables all supported features." in v3 patch.
Do you think there is still some refactoring needed here?
Thanks!
Quoted 43 lines
What do you think?
Thanks!
Quoted 37 lines
Thanks!
Quoted 34 lines
Signed-off-by Hawkins Jiawei <[email protected]>--- hw/net/virtio-net.c | 2 +- include/hw/virtio/virtio-net.h | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)hw/net/virtio-net.c +1
-1@@ -874,7 +874,7 @@ static uint64_t virtio_net_guest_offloads_by_features(uint32_t features)return guest_offloads_mask & features; } -static inline uint64_t virtio_net_supported_guest_offloads (const VirtIONet *n) +uint64_t virtio_net_supported_guest_offloads (const VirtIONet *n) {VirtIODevice *vdev =VIRTIO_DEVICE (n);return virtio_net_guest_offloads_by_features (vdev->guest_features);include/hw/virtio/virtio-net.h +1
-1@@ -227,5 +227,6 @@ size_t virtio_net_handle_ctrl_iov(VirtIODevice *vdev, unsigned out_num);void virtio_net_set_netclient_name (VirtIONet *n,const char *name,const char *type); +uint64_t virtio_net_supported_guest_offloads (const VirtIONet *n);#endif --2.25.1
Re: [PATCH v2 4/6] virtio-net: expose virtio_net_supported_guest_offloads()
Quoted 19 lines · Hawkins Jiawei
Quoted 17 lines · Eugenio Perez Martin
Quoted 11 lines · Hawkins Jiawei
Quoted 4 lines · Hawkins Jiawei
To support restoring offloads state in vdpa, need to expose the function
virtio_net_supported_guest_offloads(), then vdpa uses this function to get the guest supported offloads.Here it should be changed to "then QEMU uses this function to get the guest supported offloads.". I will correct the commit message in the v3 patch.
Maybe "to get the device supported offloads allow qemu to know the defaults, so it can skip the control message sending if they match with the driver's configuration"?
Quoted 7 lines
Quoted 2 lines
We can also add "This will be the default at guest's startup, these values can mismatch only at live migration".
Hi Eugenio,
I sent the v3 patch before seeing this email:(.
Moving the discussion to v3 then :).
Thanks!
Quoted 56 lines
I refactor the commit message to
"To support restoring offloads state in vdpa, need to expose the functionvirtio_net_supported_guest_offloads(). QEMU uses this function to get the guest supported offloads and no needs to send the corresponding CVQ command if guest enables all supported features." in v3 patch.Do you think there is still some refactoring needed here?
Thanks!
Quoted 43 lines
What do you think?
Thanks!
Quoted 37 lines
Thanks!
Quoted 34 lines
Signed-off-by Hawkins Jiawei <[email protected]>--- hw/net/virtio-net.c | 2 +- include/hw/virtio/virtio-net.h | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)hw/net/virtio-net.c +1
-1@@ -874,7 +874,7 @@ static uint64_t virtio_net_guest_offloads_by_features(uint32_t features)return guest_offloads_mask & features; } -static inline uint64_t virtio_net_supported_guest_offloads (const VirtIONet *n) +uint64_t virtio_net_supported_guest_offloads (const VirtIONet *n) {VirtIODevice *vdev =VIRTIO_DEVICE (n);return virtio_net_guest_offloads_by_features (vdev->guest_features);include/hw/virtio/virtio-net.h +1
-1@@ -227,5 +227,6 @@ size_t virtio_net_handle_ctrl_iov(VirtIODevice *vdev, unsigned out_num);void virtio_net_set_netclient_name (VirtIONet *n,const char *name,const char *type); +uint64_t virtio_net_supported_guest_offloads (const VirtIONet *n);#endif --2.25.1
PATCH v2 5/6 vdpa: Add vhost_vdpa_net_load_offloads()
In reply to Hawkins JiaweiThis patch introduces vhost_vdpa_net_load_offloads() to restore offloads state at device's startup.
--- net/vhost-vdpa.c | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+)
net/vhost-vdpa.c +38 -0
@@ -678,6 +678,40 @@ static int vhost_vdpa_net_load_mq(VhostVDPAState *s,return *s->status !=VIRTIO_NET_OK ; } +static int vhost_vdpa_net_load_offloads (VhostVDPAState *s, +const VirtIONet *n) +{ +uint64_t offloads; +ssize_t dev_written; + +if (!virtio_vdev_has_feature (&n->parent_obj, +VIRTIO_NET_F_CTRL_GUEST_OFFLOADS )) { +return 0 ; + } + + offloads =cpu_to_le64 (n->curr_guest_offloads); + +if (offloads ==virtio_net_supported_guest_offloads (n)) { +/* +* According to VirtIO standard, "Upon feature negotiation +* corresponding offload gets enabled to preserve +* backward compatibility." +* So we do not need to send this CVQ command if the guest +* also enables all supported offloads. +*/ +return 0 ; + } + + dev_written =vhost_vdpa_net_load_cmd (s,VIRTIO_NET_CTRL_GUEST_OFFLOADS , +VIRTIO_NET_CTRL_GUEST_OFFLOADS_SET , + &offloads,sizeof (offloads)); +if (unlikely (dev_written <0 )) { +return dev_written; + } + +return *s->status !=VIRTIO_NET_OK ; +} +static int vhost_vdpa_net_load (NetClientState *nc) {VhostVDPAState *s =DO_UPCAST (VhostVDPAState, nc, nc);
@@ -700,6 +734,10 @@ static int vhost_vdpa_net_load(NetClientState *nc)if (unlikely (r)) {return r; } + r =vhost_vdpa_net_load_offloads (s, n); +if (unlikely (r)) { +return r; + }return 0 ; }
Signature · 1 line
2.25.1
PATCH v2 6/6 vdpa: Allow VIRTIO_NET_F_CTRL_GUEST_OFFLOADS in SVQ
In reply to Hawkins JiaweiEnable SVQ with VIRTIO_NET_F_CTRL_GUEST_OFFLOADS feature.
--- net/vhost-vdpa.c | 1 + 1 file changed, 1 insertion(+)
net/vhost-vdpa.c +1 -0
@@ -85,6 +85,7 @@ const int vdpa_feature_bits[] = {static const uint64_t vdpa_svq_device_features =BIT_ULL (VIRTIO_NET_F_CSUM ) |BIT_ULL (VIRTIO_NET_F_GUEST_CSUM ) | +BIT_ULL (VIRTIO_NET_F_CTRL_GUEST_OFFLOADS ) |BIT_ULL (VIRTIO_NET_F_MTU ) |BIT_ULL (VIRTIO_NET_F_MAC ) | BIT_ULL(VIRTIO_NET_F_GUEST_TSO4) |
Signature · 1 line
2.25.1
I'm a QE responsible for vhost_vdpa parts. Could you please provide me with the test steps for this series? I can test it in my environment and update the test results.
Quoted 33 lines · Hawkins Jiawei
This series enables shadowed CVQ to intercept Offloads commands through shadowed CVQ, update the virtio NIC device model so qemu send it in a migration, and the restore of that Offloads state in the destination.
Changelog
=========
v2:
- make some function arguments const
- reusevirtio_vdev_has_feature()suggested by Eugenio and Jason
- avoid sending CVQ command in default state suggested by Eugeniov1: https://lore.kernel.org/all/[email protected]/
Hawkins Jiawei (6):
include/hw/virtio: make some VirtIODevice const
vdpa: reusevirtio_vdev_has_feature()hw/net/virtio-net: make some VirtIONet const
virtio-net: exposevirtio_net_supported_guest_offloads()
vdpa: Addvhost_vdpa_net_load_offloads()
vdpa: AllowVIRTIO_NET_F_CTRL_GUEST_OFFLOADSin SVQhw/net/virtio-net.c | 2 +- include/hw/virtio/virtio-net.h | 1 + include/hw/virtio/virtio.h | 2 +- net/vhost-vdpa.c | 45 +++++++++++++++++++++++++++++++--- 4 files changed, 44 insertions(+), 6 deletions(-)--
2.25.1
Quoted 4 lines · Lei Yang
I'm a QE responsible for
vhost_vdpaparts. Could you please provide me with the test steps for this series? I can test it in my environment and update the test results.
Hi Lei,
I am not sure if these are enough, but you can test it like this:
1. you can merge these patches into the latest master branch and compile the source. It should not trigger any error or warning.
2. start QEMU with the vdpa device and enable the ctrl_guest_offloads feature on. The command line should like the following:
-netdev type=vhost-vdpa,id=vdpa0,vhostdev=/dev/vhost-vdpa-0,x-svq=true \
-device virtio-net-pci,netdev=vdpa0,ctrl_guest_offloads=on,...It should also not trigger any error or warning. Without this series, QEMU should fail with "vdpa svq does not work with features 4" error.
Regarding migration, it seems that testing it is not feasible at the moment due to the migration blocker.
And you can test on my v3 patch, I will send the v3 patch to the mailing lists later.
Thank you for your efforts!
Quoted 39 lines · Hawkins Jiawei
Quoted 33 lines
This series enables shadowed CVQ to intercept Offloads commands through shadowed CVQ, update the virtio NIC device model so qemu send it in a migration, and the restore of that Offloads state in the destination.
Changelog
=========
v2:
- make some function arguments const
- reusevirtio_vdev_has_feature()suggested by Eugenio and Jason
- avoid sending CVQ command in default state suggested by Eugeniov1: https://lore.kernel.org/all/[email protected]/
Hawkins Jiawei (6):
include/hw/virtio: make some VirtIODevice const
vdpa: reusevirtio_vdev_has_feature()hw/net/virtio-net: make some VirtIONet const
virtio-net: exposevirtio_net_supported_guest_offloads()
vdpa: Addvhost_vdpa_net_load_offloads()
vdpa: AllowVIRTIO_NET_F_CTRL_GUEST_OFFLOADSin SVQhw/net/virtio-net.c | 2 +- include/hw/virtio/virtio-net.h | 1 + include/hw/virtio/virtio.h | 2 +- net/vhost-vdpa.c | 45 +++++++++++++++++++++++++++++++--- 4 files changed, 44 insertions(+), 6 deletions(-)--
2.25.1