Vhost-vdpa Shadow Virtqueue Offloads support
PATCH 1/2 vdpa: Add vhost_vdpa_net_load_offloads
In reply to Hawkins JiaweiThis patch introduces vhost_vdpa_net_load_offloads() to restore offloads state at device's startup.
--- net/vhost-vdpa.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)
net/vhost-vdpa.c +26 -0
@@ -680,6 +680,28 @@ static int vhost_vdpa_net_load_mq(VhostVDPAState *s,return *s->status !=VIRTIO_NET_OK ; } +static int vhost_vdpa_net_load_offloads (VhostVDPAState *s, +const VirtIONet *n) +{ +uint64_t features, offloads; +ssize_t dev_written; + + features = n->parent_obj.guest_features; +if (!(features &BIT_ULL (VIRTIO_NET_F_CTRL_GUEST_OFFLOADS ))) { +return 0 ; + } + + offloads =cpu_to_le64 (n->curr_guest_offloads); + dev_written =vhost_vdpa_net_load_cmd (s,VIRTIO_NET_CTRL_GUEST_OFFLOADS , +VIRTIO_NET_CTRL_GUEST_OFFLOADS_SET , + &offloads,sizeof (offloads)); +if (unlikely (dev_written <0 )) { +return dev_written; + } + +return *s->status !=VIRTIO_NET_OK ; +} +static int vhost_vdpa_net_load (NetClientState *nc) {VhostVDPAState *s =DO_UPCAST (VhostVDPAState, nc, nc);
@@ -702,6 +724,10 @@ static int vhost_vdpa_net_load(NetClientState *nc)if (unlikely (r)) {return r; } + r =vhost_vdpa_net_load_offloads (s, n); +if (unlikely (r)) { +return r; + }return 0 ; }
Signature · 1 line
2.25.1
PATCH 2/2 vdpa: Allow VIRTIO_NET_F_CTRL_GUEST_OFFLOADS in SVQ
In reply to Hawkins JiaweiEnable SVQ with VIRTIO_NET_F_CTRL_GUEST_OFFLOADS feature.
--- net/vhost-vdpa.c | 1 + 1 file changed, 1 insertion(+)
net/vhost-vdpa.c +1 -0
@@ -85,6 +85,7 @@ const int vdpa_feature_bits[] = {static const uint64_t vdpa_svq_device_features =BIT_ULL (VIRTIO_NET_F_CSUM ) |BIT_ULL (VIRTIO_NET_F_GUEST_CSUM ) | +BIT_ULL (VIRTIO_NET_F_CTRL_GUEST_OFFLOADS ) |BIT_ULL (VIRTIO_NET_F_MTU ) |BIT_ULL (VIRTIO_NET_F_MAC ) | BIT_ULL(VIRTIO_NET_F_GUEST_TSO4) |
Signature · 1 line
2.25.1
cover Vhost-vdpa Shadow Virtqueue Offloads support
This series enables shadowed CVQ to intercept Offloads commands through shadowed CVQ, update the virtio NIC device model so qemu send it in a migration, and the restore of that Offloads state in the destination.
Hawkins Jiawei (2):
vdpa: Add vhost_vdpa_net_load_offloads
vdpa: Allow VIRTIO_NET_F_CTRL_GUEST_OFFLOADS in SVQ
net/vhost-vdpa.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)
Signature · 1 line
2.25.1
Re: [PATCH 1/2] vdpa: Add vhost_vdpa_net_load_offloads
In reply to Hawkins JiaweiQuoted 18 lines · Hawkins Jiawei
This patch introduces
vhost_vdpa_net_load_offloads()to restore offloads state at device's startup.Signed-off-by Hawkins Jiawei <[email protected]>--- net/vhost-vdpa.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)net/vhost-vdpa.c +1
-0@@ -680,6 +680,28 @@ static int vhost_vdpa_net_load_mq(VhostVDPAState *s,return *s->status !=VIRTIO_NET_OK ; } +static int vhost_vdpa_net_load_offloads(VhostVDPAState *s,
net/vhost-vdpa.c static int vhost_vdpa_net_load_mq(VhostVDPAState *s, · line 693
Quoted 10 lines
net/vhost-vdpa.c +10
-0+ const VirtIONet *n) +{ +uint64_t features, offloads; +ssize_t dev_written; + + features = n->parent_obj.guest_features; +if (!(features &BIT_ULL (VIRTIO_NET_F_CTRL_GUEST_OFFLOADS ))) { +return 0 ; + } +
Maybe we can avoid sending this CVQ command if the guest already uses the default values?
By default all features are enabled if I'm not wrong. I think the best way is to expose virtio_net_supported_guest_offloads or virtio_net_guest_offloads_by_features and then check if n->curr_guest_offloads is the same.
We should do the same with vhost_vdpa_net_load_mq, but that is out of the scope of this series.
Thanks!
Quoted 28 lines
net/vhost-vdpa.c +15
-1+ offloads =cpu_to_le64 (n->curr_guest_offloads); + dev_written =vhost_vdpa_net_load_cmd (s,VIRTIO_NET_CTRL_GUEST_OFFLOADS , +VIRTIO_NET_CTRL_GUEST_OFFLOADS_SET , + &offloads,sizeof (offloads)); +if (unlikely (dev_written <0 )) { +return dev_written; + } + +return *s->status !=VIRTIO_NET_OK ; +} +static int vhost_vdpa_net_load (NetClientState *nc) {VhostVDPAState *s =DO_UPCAST (VhostVDPAState, nc, nc);@@ -702,6 +724,10 @@ static int vhost_vdpa_net_load(NetClientState *nc)if (unlikely (r)) {return r; } + r =vhost_vdpa_net_load_offloads (s, n); +if (unlikely (r)) { +return r; + }return 0 ; } --2.25.1
Quoted 6 lines · Hawkins Jiawei
This series enables shadowed CVQ to intercept Offloads commands through shadowed CVQ, update the virtio NIC device model so qemu send it in a migration, and the restore of that Offloads state in the destination.
Quoted 7 lines
Hawkins Jiawei (2):
vdpa: Addvhost_vdpa_net_load_offloads
vdpa: AllowVIRTIO_NET_F_CTRL_GUEST_OFFLOADSin SVQnet/vhost-vdpa.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)
CCing MST too.
Thanks!
Quoted 11 lines · Eugenio Perez Martin
Quoted 10 lines · Hawkins Jiawei
This series enables shadowed CVQ to intercept Offloads commands through shadowed CVQ, update the virtio NIC device model so qemu send it in a migration, and the restore of that Offloads state in the destination.
Hawkins Jiawei (2):
vdpa: Addvhost_vdpa_net_load_offloads
vdpa: AllowVIRTIO_NET_F_CTRL_GUEST_OFFLOADSin SVQ
Quoted 5 lines
Quoted 3 lines
net/vhost-vdpa.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)CCing MST too.
OK, I will cc MST in the future patches for this part.
Thanks!
Quoted 3 lines
Thanks!
Re: [PATCH 1/2] vdpa: Add vhost_vdpa_net_load_offloads
In reply to Hawkins JiaweiQuoted 14 lines · Hawkins Jiawei
This patch introduces
vhost_vdpa_net_load_offloads()to restore offloads state at device's startup.Signed-off-by Hawkins Jiawei <[email protected]>--- net/vhost-vdpa.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)net/vhost-vdpa.c +0
-0
net/vhost-vdpa.c static int vhost_vdpa_net_load_mq(VhostVDPAState *s, · line 689
Quoted 10 lines
net/vhost-vdpa.c +7
-0@@ -680,6 +680,28 @@ static int vhost_vdpa_net_load_mq(VhostVDPAState *s,return *s->status !=VIRTIO_NET_OK ; } +static int vhost_vdpa_net_load_offloads (VhostVDPAState *s, +const VirtIONet *n) +{ +uint64_t features, offloads; +ssize_t dev_written; + + features = n->parent_obj.guest_features;
Any reason you need to do tricks like this instead of using virtio_xxx_has_features()?
Quoted 32 lines
net/vhost-vdpa.c +19
-1+if (!(features &BIT_ULL (VIRTIO_NET_F_CTRL_GUEST_OFFLOADS ))) { +return 0 ; + } + + offloads =cpu_to_le64 (n->curr_guest_offloads); + dev_written =vhost_vdpa_net_load_cmd (s,VIRTIO_NET_CTRL_GUEST_OFFLOADS , +VIRTIO_NET_CTRL_GUEST_OFFLOADS_SET , + &offloads,sizeof (offloads)); +if (unlikely (dev_written <0 )) { +return dev_written; + } + +return *s->status !=VIRTIO_NET_OK ; +} +static int vhost_vdpa_net_load (NetClientState *nc) {VhostVDPAState *s =DO_UPCAST (VhostVDPAState, nc, nc);@@ -702,6 +724,10 @@ static int vhost_vdpa_net_load(NetClientState *nc)if (unlikely (r)) {return r; } + r =vhost_vdpa_net_load_offloads (s, n); +if (unlikely (r)) { +return r; + }return 0 ; } --2.25.1
Re: [PATCH 2/2] vdpa: Allow VIRTIO_NET_F_CTRL_GUEST_OFFLOADS in SVQ
In reply to Hawkins JiaweiQuoted 4 lines · Hawkins Jiawei
Enable SVQ with
VIRTIO_NET_F_CTRL_GUEST_OFFLOADSfeature.Signed-off-by Hawkins Jiawei <[email protected]>
Thanks
Quoted 19 lines
--- net/vhost-vdpa.c | 1 + 1 file changed, 1 insertion(+)net/vhost-vdpa.c +1
-1@@ -85,6 +85,7 @@ const int vdpa_feature_bits[] = {static const uint64_t vdpa_svq_device_features =BIT_ULL (VIRTIO_NET_F_CSUM ) |BIT_ULL (VIRTIO_NET_F_GUEST_CSUM ) | +BIT_ULL (VIRTIO_NET_F_CTRL_GUEST_OFFLOADS ) |BIT_ULL (VIRTIO_NET_F_MTU ) |BIT_ULL (VIRTIO_NET_F_MAC ) | BIT_ULL(VIRTIO_NET_F_GUEST_TSO4) | --2.25.1
Re: [PATCH 1/2] vdpa: Add vhost_vdpa_net_load_offloads
In reply to Jason WangQuoted 19 lines · Jason Wang
Quoted 17 lines · Hawkins Jiawei
This patch introduces
vhost_vdpa_net_load_offloads()to restore offloads state at device's startup.Signed-off-by Hawkins Jiawei <[email protected]>--- net/vhost-vdpa.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)net/vhost-vdpa.c +0
-0@@ -680,6 +680,28 @@ static int vhost_vdpa_net_load_mq(VhostVDPAState *s,return *s->status !=VIRTIO_NET_OK ; }
Quoted 11 lines
net/vhost-vdpa.c static int vhost_vdpa_net_load_mq(VhostVDPAState *s, · line 689
Quoted 7 lines
net/vhost-vdpa.c +7
-0+static int vhost_vdpa_net_load_offloads (VhostVDPAState *s, +const VirtIONet *n) +{ +uint64_t features, offloads; +ssize_t dev_written; + + features = n->parent_obj.guest_features;Any reason you need to do tricks like this instead of using
virtio_xxx_has_features()?
It can be replaced by virtio_vdev_has_feature, yes.
Current code of vhost_vdpa_net_load_mac and vhost_vdpa_net_load_mq access to guest_features directly too, so I think we should change all of them at once.
Thanks!
Quoted 33 lines
Quoted 32 lines
net/vhost-vdpa.c +19
-1+if (!(features &BIT_ULL (VIRTIO_NET_F_CTRL_GUEST_OFFLOADS ))) { +return 0 ; + } + + offloads =cpu_to_le64 (n->curr_guest_offloads); + dev_written =vhost_vdpa_net_load_cmd (s,VIRTIO_NET_CTRL_GUEST_OFFLOADS , +VIRTIO_NET_CTRL_GUEST_OFFLOADS_SET , + &offloads,sizeof (offloads)); +if (unlikely (dev_written <0 )) { +return dev_written; + } + +return *s->status !=VIRTIO_NET_OK ; +} +static int vhost_vdpa_net_load (NetClientState *nc) {VhostVDPAState *s =DO_UPCAST (VhostVDPAState, nc, nc);@@ -702,6 +724,10 @@ static int vhost_vdpa_net_load(NetClientState *nc)if (unlikely (r)) {return r; } + r =vhost_vdpa_net_load_offloads (s, n); +if (unlikely (r)) { +return r; + }return 0 ; } --2.25.1
Re: [PATCH 1/2] vdpa: Add vhost_vdpa_net_load_offloads
In reply to Eugenio Perez MartinQuoted 18 lines · Eugenio Perez Martin
Quoted 17 lines · Hawkins Jiawei
This patch introduces
vhost_vdpa_net_load_offloads()to restore offloads state at device's startup.Signed-off-by Hawkins Jiawei <[email protected]>--- net/vhost-vdpa.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)net/vhost-vdpa.c +0
-0@@ -680,6 +680,28 @@ static int vhost_vdpa_net_load_mq(VhostVDPAState *s,return *s->status !=VIRTIO_NET_OK ; }
Quoted 14 lines
net/vhost-vdpa.c static int vhost_vdpa_net_load_mq(VhostVDPAState *s, · line 693
Quoted 11 lines
net/vhost-vdpa.c +11
-0+static int vhost_vdpa_net_load_offloads (VhostVDPAState *s, +const VirtIONet *n) +{ +uint64_t features, offloads; +ssize_t dev_written; + + features = n->parent_obj.guest_features; +if (!(features &BIT_ULL (VIRTIO_NET_F_CTRL_GUEST_OFFLOADS ))) { +return 0 ; + } +Maybe we can avoid sending this CVQ command if the guest already uses the default values?
Hi Eugenio,
Thanks for the review. However, I'm curious why we don't need to send this CVQ command if the guest is using the default values. Is it because the device automatically applies these default offloads, when the VIRTIO_NET_F_CTRL_GUEST_OFFLOADS feature is negotiated and QEMU doesn't send the CVQ command?
Thanks!
Quoted 40 lines
By default all features are enabled if I'm not wrong. I think the best way is to expose
virtio_net_supported_guest_offloadsorvirtio_net_guest_offloads_by_featuresand then check ifn->curr_guest_offloadsis the same.We should do the same with
vhost_vdpa_net_load_mq, but that is out of the scope of this series.Thanks!
Quoted 28 lines
net/vhost-vdpa.c +15
-1+ offloads =cpu_to_le64 (n->curr_guest_offloads); + dev_written =vhost_vdpa_net_load_cmd (s,VIRTIO_NET_CTRL_GUEST_OFFLOADS , +VIRTIO_NET_CTRL_GUEST_OFFLOADS_SET , + &offloads,sizeof (offloads)); +if (unlikely (dev_written <0 )) { +return dev_written; + } + +return *s->status !=VIRTIO_NET_OK ; +} +static int vhost_vdpa_net_load (NetClientState *nc) {VhostVDPAState *s =DO_UPCAST (VhostVDPAState, nc, nc);@@ -702,6 +724,10 @@ static int vhost_vdpa_net_load(NetClientState *nc)if (unlikely (r)) {return r; } + r =vhost_vdpa_net_load_offloads (s, n); +if (unlikely (r)) { +return r; + }return 0 ; } --2.25.1
Re: [PATCH 1/2] vdpa: Add vhost_vdpa_net_load_offloads
In reply to Eugenio Perez MartinQuoted 32 lines · Eugenio Perez Martin
Quoted 19 lines · Jason Wang
Quoted 17 lines · Hawkins Jiawei
This patch introduces
vhost_vdpa_net_load_offloads()to restore offloads state at device's startup.Signed-off-by Hawkins Jiawei <[email protected]>--- net/vhost-vdpa.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)net/vhost-vdpa.c +0
-0@@ -680,6 +680,28 @@ static int vhost_vdpa_net_load_mq(VhostVDPAState *s,return *s->status !=VIRTIO_NET_OK ; }Quoted 11 lines
net/vhost-vdpa.c static int vhost_vdpa_net_load_mq(VhostVDPAState *s, · line 689
Quoted 7 lines
net/vhost-vdpa.c +7
-0+static int vhost_vdpa_net_load_offloads (VhostVDPAState *s, +const VirtIONet *n) +{ +uint64_t features, offloads; +ssize_t dev_written; + + features = n->parent_obj.guest_features;Any reason you need to do tricks like this instead of using
virtio_xxx_has_features()?
Quoted 5 lines
It can be replaced by
virtio_vdev_has_feature, yes.Current code of
vhost_vdpa_net_load_macandvhost_vdpa_net_load_mqaccess toguest_featuresdirectly too, so I think we should change all of them at once.
Yes, I agree with you and Jason.
I will refactor the patch as you and Jason have suggested.
Thanks!
Quoted 37 lines
Thanks!
Quoted 33 lines
Quoted 32 lines
net/vhost-vdpa.c +19
-1+if (!(features &BIT_ULL (VIRTIO_NET_F_CTRL_GUEST_OFFLOADS ))) { +return 0 ; + } + + offloads =cpu_to_le64 (n->curr_guest_offloads); + dev_written =vhost_vdpa_net_load_cmd (s,VIRTIO_NET_CTRL_GUEST_OFFLOADS , +VIRTIO_NET_CTRL_GUEST_OFFLOADS_SET , + &offloads,sizeof (offloads)); +if (unlikely (dev_written <0 )) { +return dev_written; + } + +return *s->status !=VIRTIO_NET_OK ; +} +static int vhost_vdpa_net_load (NetClientState *nc) {VhostVDPAState *s =DO_UPCAST (VhostVDPAState, nc, nc);@@ -702,6 +724,10 @@ static int vhost_vdpa_net_load(NetClientState *nc)if (unlikely (r)) {return r; } + r =vhost_vdpa_net_load_offloads (s, n); +if (unlikely (r)) { +return r; + }return 0 ; } --2.25.1
Re: [PATCH 1/2] vdpa: Add vhost_vdpa_net_load_offloads
In reply to Hawkins JiaweiQuoted 35 lines · Hawkins Jiawei
Quoted 18 lines · Eugenio Perez Martin
Quoted 17 lines · Hawkins Jiawei
This patch introduces
vhost_vdpa_net_load_offloads()to restore offloads state at device's startup.Signed-off-by Hawkins Jiawei <[email protected]>--- net/vhost-vdpa.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)net/vhost-vdpa.c +0
-0@@ -680,6 +680,28 @@ static int vhost_vdpa_net_load_mq(VhostVDPAState *s,return *s->status !=VIRTIO_NET_OK ; }Quoted 14 lines
net/vhost-vdpa.c static int vhost_vdpa_net_load_mq(VhostVDPAState *s, · line 693
Quoted 11 lines
net/vhost-vdpa.c +11
-0+static int vhost_vdpa_net_load_offloads (VhostVDPAState *s, +const VirtIONet *n) +{ +uint64_t features, offloads; +ssize_t dev_written; + + features = n->parent_obj.guest_features; +if (!(features &BIT_ULL (VIRTIO_NET_F_CTRL_GUEST_OFFLOADS ))) { +return 0 ; + } +Maybe we can avoid sending this CVQ command if the guest already uses the default values?
Quoted 8 lines
Hi Eugenio,
Thanks for the review. However, I'm curious why we don't need to send this CVQ command if the guest is using the default values. Is it because the device automatically applies these default offloads, when the
VIRTIO_NET_F_CTRL_GUEST_OFFLOADSfeature is negotiated and QEMU doesn't send the CVQ command?
Exactly. You can check that either by the standard text or (sometimes easier) qemu virtio or virtio-net device code.
The standard text says that:
"Upon feature negotiation corresponding offload gets enabled to preserve backward compatibility."
And you can check in the qemu code by hw/net/virtio-net:virtio_net_set_features(vdev, features), this chunk of code:n->curr_guest_offloads = virtio_net_guest_offloads_by_features(features); virtio_net_apply_guest_offloads(n);
Thanks!
Quoted 44 lines
Thanks!
Quoted 40 lines
By default all features are enabled if I'm not wrong. I think the best way is to expose
virtio_net_supported_guest_offloadsorvirtio_net_guest_offloads_by_featuresand then check ifn->curr_guest_offloadsis the same.We should do the same with
vhost_vdpa_net_load_mq, but that is out of the scope of this series.Thanks!
Quoted 28 lines
net/vhost-vdpa.c +15
-1+ offloads =cpu_to_le64 (n->curr_guest_offloads); + dev_written =vhost_vdpa_net_load_cmd (s,VIRTIO_NET_CTRL_GUEST_OFFLOADS , +VIRTIO_NET_CTRL_GUEST_OFFLOADS_SET , + &offloads,sizeof (offloads)); +if (unlikely (dev_written <0 )) { +return dev_written; + } + +return *s->status !=VIRTIO_NET_OK ; +} +static int vhost_vdpa_net_load (NetClientState *nc) {VhostVDPAState *s =DO_UPCAST (VhostVDPAState, nc, nc);@@ -702,6 +724,10 @@ static int vhost_vdpa_net_load(NetClientState *nc)if (unlikely (r)) {return r; } + r =vhost_vdpa_net_load_offloads (s, n); +if (unlikely (r)) { +return r; + }return 0 ; } --2.25.1