Guest announce feature emulation using Shadow VirtQueue
v2 cover Guest announce feature emulation using Shadow VirtQueue
A gratuitous ARP is recommended after a live migration to reduce the amount of time needed by the network links to be aware of the new location. A hypervisor may not have the knowledge of the guest network configuration, and this is especially true on passthrough devices, so its simpler to ask the guest to do it.
However, the device control part of this feature can be totally emulated by qemu and shadow virtqueue, not needing any special feature from the actual vdpa device.
VIRTIO_NET_F_STATUS is also needed for the guest to access the status of virtio net config where announcement status bit is set. Emulating it as always active in case backend does not support it.
v2:
* Add VIRTIO_NET_F_STATUS emulation.
Eugenio Pérez (8):
vdpa: Delete duplicated vdpa_feature_bits entry
vdpa: Save emulated features list in vhost_vdpa
vhost_net: Emulate link state up if backend doesn't expose it
vdpa: Expose VIRTIO_NET_F_STATUS unconditionally
vdpa: Remove shadow CVQ command check
vdpa: handle VIRTIO_NET_CTRL_ANNOUNCE in
vhost_vdpa_net_handle_ctrl_avail
vhost_net: return VIRTIO_NET_S_ANNOUNCE is device model has it set
vdpa: Offer VIRTIO_NET_F_GUEST_ANNOUNCE feature if SVQ is enabled
include/hw/virtio/vhost-vdpa.h | 2 + hw/net/vhost_net.c | 35 +++++++++++++++- hw/virtio/vhost-vdpa.c | 8 ++-- net/vhost-vdpa.c | 74 ++++++++++------------------------ 4 files changed, 62 insertions(+), 57 deletions(-)
Signature · 1 line
2.31.1
RFC PATCH v2 1/8 vdpa: Delete duplicated vdpa_feature_bits entry
This entry was duplicated on referenced commit. Removing it.
--- net/vhost-vdpa.c | 1 - 1 file changed, 1 deletion(-)
net/vhost-vdpa.c +0 -1
@@ -63,7 +63,6 @@ const int vdpa_feature_bits[] = {VIRTIO_NET_F_CTRL_RX ,VIRTIO_NET_F_CTRL_RX_EXTRA ,VIRTIO_NET_F_CTRL_VLAN , -VIRTIO_NET_F_GUEST_ANNOUNCE ,VIRTIO_NET_F_CTRL_MAC_ADDR ,VIRTIO_NET_F_RSS , VIRTIO_NET_F_MQ,
Signature · 1 line
2.31.1
Re: [RFC PATCH v2 1/8] vdpa: Delete duplicated vdpa_feature_bits entry
Quoted 5 lines · Eugenio Pérez
This entry was duplicated on referenced commit. Removing it.
Fixes 402378407dbd ("vhost-vdpa: multiqueue support")Signed-off-by Eugenio Pérez <[email protected]>
Quoted 19 lines
--- net/vhost-vdpa.c | 1 - 1 file changed, 1 deletion(-)net/vhost-vdpa.c +0
-2@@ -63,7 +63,6 @@ const int vdpa_feature_bits[] = {VIRTIO_NET_F_CTRL_RX ,VIRTIO_NET_F_CTRL_RX_EXTRA ,VIRTIO_NET_F_CTRL_VLAN , -VIRTIO_NET_F_GUEST_ANNOUNCE ,VIRTIO_NET_F_CTRL_MAC_ADDR ,VIRTIO_NET_F_RSS , VIRTIO_NET_F_MQ, --2.31.1
RFC PATCH v2 2/8 vdpa: Save emulated features list in vhost_vdpa
In reply to Eugenio PérezAt this moment only _F_LOG is added there.
However future patches add features that depend on the kind of device. In particular, only net devices can add VIRTIO_F_GUEST_ANNOUNCE. So let's allow vhost_vdpa creator to set custom emulated device features.
--- include/hw/virtio/vhost-vdpa.h | 2 ++ hw/virtio/vhost-vdpa.c | 8 ++++---- net/vhost-vdpa.c | 4 ++++ 3 files changed, 10 insertions(+), 4 deletions(-)
include/hw/virtio/vhost-vdpa.h +2 -0
@@ -31,6 +31,8 @@ typedef struct vhost_vdpa {bool iotlb_batch_begin_sent;MemoryListener listener;struct vhost_vdpa_iova_range iova_range; +/* VirtIO device features that can be emulated by qemu */ +uint64_t added_features;uint64_t acked_features;bool shadow_vqs_enabled;/* IOVA mapping used by the Shadow Virtqueue */
hw/virtio/vhost-vdpa.c +4 -4
@@ -660,8 +660,8 @@ static int vhost_vdpa_set_features(struct vhost_dev *dev, v->acked_features = features; -/* We must not ack _F_LOG if SVQ is enabled */ - features &= ~BIT_ULL (VHOST_F_LOG_ALL ); +/* Do not ack features emulated by qemu */ + features &= ~v->added_features; }trace_vhost_vdpa_set_features (dev, features);
@@ -1244,8 +1244,8 @@ static int vhost_vdpa_get_features(struct vhost_dev *dev,int ret =vhost_vdpa_get_dev_features (dev, features);if (ret ==0 && v->shadow_vqs_enabled) { -/* Add SVQ logging capabilities */ - *features |=BIT_ULL (VHOST_F_LOG_ALL ); +/* Add emulated capabilities */ + *features |= v->added_features; }return ret;
net/vhost-vdpa.c +4 -0
@@ -599,6 +599,10 @@ static NetClientState *net_vhost_vdpa_init(NetClientState *peer, s->vhost_vdpa.index = queue_pair_index; s->vhost_vdpa.shadow_vqs_enabled = svq; s->vhost_vdpa.iova_tree = iova_tree; +if (svq) { +/* Add SVQ logging capabilities */ + s->vhost_vdpa.added_features |=BIT_ULL (VHOST_F_LOG_ALL ); + }if (!is_datapath) { s->cvq_cmd_out_buffer =qemu_memalign (qemu_real_host_page_size (),vhost_vdpa_net_cvq_cmd_page_len ());
Signature · 1 line
2.31.1
Re: [RFC PATCH v2 2/8] vdpa: Save emulated features list in vhost_vdpa
Quoted 19 lines · Eugenio Pérez
At this moment only
_F_LOGis added there.However future patches add features that depend on the kind of device. In particular, only net devices can add
VIRTIO_F_GUEST_ANNOUNCE. So let's allowvhost_vdpacreator to set custom emulated device features.Signed-off-by Eugenio Pérez <[email protected]>--- include/hw/virtio/vhost-vdpa.h | 2 ++ hw/virtio/vhost-vdpa.c | 8 ++++---- net/vhost-vdpa.c | 4 ++++ 3 files changed, 10 insertions(+), 4 deletions(-)include/hw/virtio/vhost-vdpa.h +0
-0
include/hw/virtio/vhost-vdpa.h typedef struct vhost_vdpa { · line 35
Quoted 5 lines
include/hw/virtio/vhost-vdpa.h +2
-0@@ -31,6 +31,8 @@ typedef struct vhost_vdpa {bool iotlb_batch_begin_sent;MemoryListener listener;struct vhost_vdpa_iova_range iova_range; +/* VirtIO device features that can be emulated by qemu */ +uint64_t added_features;
Any reason we need a per vhost_vdpa storage for this? Or is there a chance that this field could be different among the devices?
Thanks
Quoted 47 lines
include/hw/virtio/vhost-vdpa.h +0
-0uint64_t acked_features;bool shadow_vqs_enabled;/* IOVA mapping used by the Shadow Virtqueue */ hw/virtio/vhost-vdpa.c +4
-4@@ -660,8 +660,8 @@ static int vhost_vdpa_set_features(struct vhost_dev *dev, v->acked_features = features; -/* We must not ack _F_LOG if SVQ is enabled */ - features &= ~BIT_ULL (VHOST_F_LOG_ALL ); +/* Do not ack features emulated by qemu */ + features &= ~v->added_features; }trace_vhost_vdpa_set_features (dev, features);@@ -1244,8 +1244,8 @@ static int vhost_vdpa_get_features(struct vhost_dev *dev,int ret =vhost_vdpa_get_dev_features (dev, features);if (ret ==0 && v->shadow_vqs_enabled) { -/* Add SVQ logging capabilities */ - *features |=BIT_ULL (VHOST_F_LOG_ALL ); +/* Add emulated capabilities */ + *features |= v->added_features; }return ret;net/vhost-vdpa.c +4
-1@@ -599,6 +599,10 @@ static NetClientState *net_vhost_vdpa_init(NetClientState *peer, s->vhost_vdpa.index = queue_pair_index; s->vhost_vdpa.shadow_vqs_enabled = svq; s->vhost_vdpa.iova_tree = iova_tree; +if (svq) { +/* Add SVQ logging capabilities */ + s->vhost_vdpa.added_features |=BIT_ULL (VHOST_F_LOG_ALL ); + }if (!is_datapath) { s->cvq_cmd_out_buffer =qemu_memalign (qemu_real_host_page_size (),vhost_vdpa_net_cvq_cmd_page_len ()); --2.31.1
Re: [RFC PATCH v2 2/8] vdpa: Save emulated features list in vhost_vdpa
Quoted 16 lines · Jason Wang
Quoted 14 lines · Eugenio Pérez
At this moment only
_F_LOGis added there.However future patches add features that depend on the kind of device. In particular, only net devices can add
VIRTIO_F_GUEST_ANNOUNCE. So let's allowvhost_vdpacreator to set custom emulated device features.Signed-off-by Eugenio Pérez <[email protected]>--- include/hw/virtio/vhost-vdpa.h | 2 ++ hw/virtio/vhost-vdpa.c | 8 ++++---- net/vhost-vdpa.c | 4 ++++ 3 files changed, 10 insertions(+), 4 deletions(-)
Quoted 14 lines
include/hw/virtio/vhost-vdpa.h typedef struct vhost_vdpa { · line 35
Quoted 10 lines
include/hw/virtio/vhost-vdpa.h +2
-0@@ -31,6 +31,8 @@ typedef struct vhost_vdpa {bool iotlb_batch_begin_sent;MemoryListener listener;struct vhost_vdpa_iova_range iova_range; +/* VirtIO device features that can be emulated by qemu */ +uint64_t added_features;Any reason we need a per
vhost_vdpastorage for this? Or is there a chance that this field could be different among the devices?
Yes, one device could support SVQ and the other one could not support it because of different feature sets for example.
Thanks!
Quoted 50 lines
Thanks
Quoted 47 lines
include/hw/virtio/vhost-vdpa.h +0
-0uint64_t acked_features;bool shadow_vqs_enabled;/* IOVA mapping used by the Shadow Virtqueue */ hw/virtio/vhost-vdpa.c +4
-4@@ -660,8 +660,8 @@ static int vhost_vdpa_set_features(struct vhost_dev *dev, v->acked_features = features; -/* We must not ack _F_LOG if SVQ is enabled */ - features &= ~BIT_ULL (VHOST_F_LOG_ALL ); +/* Do not ack features emulated by qemu */ + features &= ~v->added_features; }trace_vhost_vdpa_set_features (dev, features);@@ -1244,8 +1244,8 @@ static int vhost_vdpa_get_features(struct vhost_dev *dev,int ret =vhost_vdpa_get_dev_features (dev, features);if (ret ==0 && v->shadow_vqs_enabled) { -/* Add SVQ logging capabilities */ - *features |=BIT_ULL (VHOST_F_LOG_ALL ); +/* Add emulated capabilities */ + *features |= v->added_features; }return ret;net/vhost-vdpa.c +4
-1@@ -599,6 +599,10 @@ static NetClientState *net_vhost_vdpa_init(NetClientState *peer, s->vhost_vdpa.index = queue_pair_index; s->vhost_vdpa.shadow_vqs_enabled = svq; s->vhost_vdpa.iova_tree = iova_tree; +if (svq) { +/* Add SVQ logging capabilities */ + s->vhost_vdpa.added_features |=BIT_ULL (VHOST_F_LOG_ALL ); + }if (!is_datapath) { s->cvq_cmd_out_buffer =qemu_memalign (qemu_real_host_page_size (),vhost_vdpa_net_cvq_cmd_page_len ()); --2.31.1
Re: [RFC PATCH v2 2/8] vdpa: Save emulated features list in vhost_vdpa
Quoted 33 lines · Eugenio Perez Martin
Quoted 16 lines · Jason Wang
Quoted 14 lines · Eugenio Pérez
At this moment only
_F_LOGis added there.However future patches add features that depend on the kind of device. In particular, only net devices can add
VIRTIO_F_GUEST_ANNOUNCE. So let's allowvhost_vdpacreator to set custom emulated device features.Signed-off-by Eugenio Pérez <[email protected]>--- include/hw/virtio/vhost-vdpa.h | 2 ++ hw/virtio/vhost-vdpa.c | 8 ++++---- net/vhost-vdpa.c | 4 ++++ 3 files changed, 10 insertions(+), 4 deletions(-)Quoted 14 lines
include/hw/virtio/vhost-vdpa.h typedef struct vhost_vdpa { · line 35
Quoted 10 lines
include/hw/virtio/vhost-vdpa.h +2
-0@@ -31,6 +31,8 @@ typedef struct vhost_vdpa {bool iotlb_batch_begin_sent;MemoryListener listener;struct vhost_vdpa_iova_range iova_range; +/* VirtIO device features that can be emulated by qemu */ +uint64_t added_features;Any reason we need a per
vhost_vdpastorage for this? Or is there a chance that this field could be different among the devices?
Quoted 2 lines
Yes, one device could support SVQ and the other one could not support it because of different feature sets for example.
Right, but for those devices that don't support SVQ, we don't even need mediation for feature like F_LOG and _F_STATUS?
Thanks
Quoted 54 lines
Thanks!
Quoted 50 lines
Thanks
Quoted 47 lines
include/hw/virtio/vhost-vdpa.h +0
-0uint64_t acked_features;bool shadow_vqs_enabled;/* IOVA mapping used by the Shadow Virtqueue */ hw/virtio/vhost-vdpa.c +4
-4@@ -660,8 +660,8 @@ static int vhost_vdpa_set_features(struct vhost_dev *dev, v->acked_features = features; -/* We must not ack _F_LOG if SVQ is enabled */ - features &= ~BIT_ULL (VHOST_F_LOG_ALL ); +/* Do not ack features emulated by qemu */ + features &= ~v->added_features; }trace_vhost_vdpa_set_features (dev, features);@@ -1244,8 +1244,8 @@ static int vhost_vdpa_get_features(struct vhost_dev *dev,int ret =vhost_vdpa_get_dev_features (dev, features);if (ret ==0 && v->shadow_vqs_enabled) { -/* Add SVQ logging capabilities */ - *features |=BIT_ULL (VHOST_F_LOG_ALL ); +/* Add emulated capabilities */ + *features |= v->added_features; }return ret;net/vhost-vdpa.c +4
-1@@ -599,6 +599,10 @@ static NetClientState *net_vhost_vdpa_init(NetClientState *peer, s->vhost_vdpa.index = queue_pair_index; s->vhost_vdpa.shadow_vqs_enabled = svq; s->vhost_vdpa.iova_tree = iova_tree; +if (svq) { +/* Add SVQ logging capabilities */ + s->vhost_vdpa.added_features |=BIT_ULL (VHOST_F_LOG_ALL ); + }if (!is_datapath) { s->cvq_cmd_out_buffer =qemu_memalign (qemu_real_host_page_size (),vhost_vdpa_net_cvq_cmd_page_len ()); --2.31.1
Re: [RFC PATCH v2 2/8] vdpa: Save emulated features list in vhost_vdpa
Quoted 36 lines · Jason Wang
Quoted 33 lines · Eugenio Perez Martin
Quoted 30 lines · Jason Wang
Quoted 19 lines · Eugenio Pérez
At this moment only
_F_LOGis added there.However future patches add features that depend on the kind of device. In particular, only net devices can add
VIRTIO_F_GUEST_ANNOUNCE. So let's allowvhost_vdpacreator to set custom emulated device features.Signed-off-by Eugenio Pérez <[email protected]>--- include/hw/virtio/vhost-vdpa.h | 2 ++ hw/virtio/vhost-vdpa.c | 8 ++++---- net/vhost-vdpa.c | 4 ++++ 3 files changed, 10 insertions(+), 4 deletions(-)include/hw/virtio/vhost-vdpa.h +0
-0include/hw/virtio/vhost-vdpa.h typedef struct vhost_vdpa { · line 35
Quoted 5 lines
include/hw/virtio/vhost-vdpa.h +2
-0@@ -31,6 +31,8 @@ typedef struct vhost_vdpa {bool iotlb_batch_begin_sent;MemoryListener listener;struct vhost_vdpa_iova_range iova_range; +/* VirtIO device features that can be emulated by qemu */ +uint64_t added_features;Any reason we need a per
vhost_vdpastorage for this? Or is there a chance that this field could be different among the devices?
Quoted 6 lines
Quoted 2 lines
Yes, one device could support SVQ and the other one could not support it because of different feature sets for example.
Right, but for those devices that don't support SVQ, we don't even need mediation for feature like
F_LOGand_F_STATUS?
No, and we cannot offer it to the guest either.
Quoted 57 lines
Thanks
Quoted 54 lines
Thanks!
Quoted 50 lines
Thanks
Quoted 47 lines
include/hw/virtio/vhost-vdpa.h +0
-0uint64_t acked_features;bool shadow_vqs_enabled;/* IOVA mapping used by the Shadow Virtqueue */ hw/virtio/vhost-vdpa.c +4
-4@@ -660,8 +660,8 @@ static int vhost_vdpa_set_features(struct vhost_dev *dev, v->acked_features = features; -/* We must not ack _F_LOG if SVQ is enabled */ - features &= ~BIT_ULL (VHOST_F_LOG_ALL ); +/* Do not ack features emulated by qemu */ + features &= ~v->added_features; }trace_vhost_vdpa_set_features (dev, features);@@ -1244,8 +1244,8 @@ static int vhost_vdpa_get_features(struct vhost_dev *dev,int ret =vhost_vdpa_get_dev_features (dev, features);if (ret ==0 && v->shadow_vqs_enabled) { -/* Add SVQ logging capabilities */ - *features |=BIT_ULL (VHOST_F_LOG_ALL ); +/* Add emulated capabilities */ + *features |= v->added_features; }return ret;net/vhost-vdpa.c +4
-1@@ -599,6 +599,10 @@ static NetClientState *net_vhost_vdpa_init(NetClientState *peer, s->vhost_vdpa.index = queue_pair_index; s->vhost_vdpa.shadow_vqs_enabled = svq; s->vhost_vdpa.iova_tree = iova_tree; +if (svq) { +/* Add SVQ logging capabilities */ + s->vhost_vdpa.added_features |=BIT_ULL (VHOST_F_LOG_ALL ); + }if (!is_datapath) { s->cvq_cmd_out_buffer =qemu_memalign (qemu_real_host_page_size (),vhost_vdpa_net_cvq_cmd_page_len ()); --2.31.1
Re: [RFC PATCH v2 2/8] vdpa: Save emulated features list in vhost_vdpa
Quoted 38 lines · Eugenio Perez Martin
Quoted 36 lines · Jason Wang
Quoted 33 lines · Eugenio Perez Martin
Quoted 30 lines · Jason Wang
Quoted 19 lines · Eugenio Pérez
At this moment only
_F_LOGis added there.However future patches add features that depend on the kind of device. In particular, only net devices can add
VIRTIO_F_GUEST_ANNOUNCE. So let's allowvhost_vdpacreator to set custom emulated device features.Signed-off-by Eugenio Pérez <[email protected]>--- include/hw/virtio/vhost-vdpa.h | 2 ++ hw/virtio/vhost-vdpa.c | 8 ++++---- net/vhost-vdpa.c | 4 ++++ 3 files changed, 10 insertions(+), 4 deletions(-)include/hw/virtio/vhost-vdpa.h +0
-0include/hw/virtio/vhost-vdpa.h typedef struct vhost_vdpa { · line 35
Quoted 5 lines
include/hw/virtio/vhost-vdpa.h +2
-0@@ -31,6 +31,8 @@ typedef struct vhost_vdpa {bool iotlb_batch_begin_sent;MemoryListener listener;struct vhost_vdpa_iova_range iova_range; +/* VirtIO device features that can be emulated by qemu */ +uint64_t added_features;Any reason we need a per
vhost_vdpastorage for this? Or is there a chance that this field could be different among the devices?
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
Yes, one device could support SVQ and the other one could not support it because of different feature sets for example.
Right, but for those devices that don't support SVQ, we don't even need mediation for feature like
F_LOGand_F_STATUS?No, and we cannot offer it to the guest either.
Just to make sure we are on the same page, what I meant is, consider in the future SVQ get the support of all features, so we can remove this field? This is because _F_STATUS can be mediated unconditionally anyhow.
Thanks
Quoted 59 lines
Quoted 58 lines
Thanks
Quoted 54 lines
Thanks!
Quoted 50 lines
Thanks
Quoted 47 lines
include/hw/virtio/vhost-vdpa.h +0
-0uint64_t acked_features;bool shadow_vqs_enabled;/* IOVA mapping used by the Shadow Virtqueue */ hw/virtio/vhost-vdpa.c +4
-4@@ -660,8 +660,8 @@ static int vhost_vdpa_set_features(struct vhost_dev *dev, v->acked_features = features; -/* We must not ack _F_LOG if SVQ is enabled */ - features &= ~BIT_ULL (VHOST_F_LOG_ALL ); +/* Do not ack features emulated by qemu */ + features &= ~v->added_features; }trace_vhost_vdpa_set_features (dev, features);@@ -1244,8 +1244,8 @@ static int vhost_vdpa_get_features(struct vhost_dev *dev,int ret =vhost_vdpa_get_dev_features (dev, features);if (ret ==0 && v->shadow_vqs_enabled) { -/* Add SVQ logging capabilities */ - *features |=BIT_ULL (VHOST_F_LOG_ALL ); +/* Add emulated capabilities */ + *features |= v->added_features; }return ret;net/vhost-vdpa.c +4
-1@@ -599,6 +599,10 @@ static NetClientState *net_vhost_vdpa_init(NetClientState *peer, s->vhost_vdpa.index = queue_pair_index; s->vhost_vdpa.shadow_vqs_enabled = svq; s->vhost_vdpa.iova_tree = iova_tree; +if (svq) { +/* Add SVQ logging capabilities */ + s->vhost_vdpa.added_features |=BIT_ULL (VHOST_F_LOG_ALL ); + }if (!is_datapath) { s->cvq_cmd_out_buffer =qemu_memalign (qemu_real_host_page_size (),vhost_vdpa_net_cvq_cmd_page_len ()); --2.31.1
Re: [RFC PATCH v2 2/8] vdpa: Save emulated features list in vhost_vdpa
Quoted 48 lines · Jason Wang
Quoted 45 lines · Eugenio Perez Martin
Quoted 42 lines · Jason Wang
Quoted 33 lines · Eugenio Perez Martin
Quoted 16 lines · Jason Wang
Quoted 14 lines · Eugenio Pérez
At this moment only
_F_LOGis added there.However future patches add features that depend on the kind of device. In particular, only net devices can add
VIRTIO_F_GUEST_ANNOUNCE. So let's allowvhost_vdpacreator to set custom emulated device features.Signed-off-by Eugenio Pérez <[email protected]>--- include/hw/virtio/vhost-vdpa.h | 2 ++ hw/virtio/vhost-vdpa.c | 8 ++++---- net/vhost-vdpa.c | 4 ++++ 3 files changed, 10 insertions(+), 4 deletions(-)Quoted 14 lines
include/hw/virtio/vhost-vdpa.h typedef struct vhost_vdpa { · line 35
Quoted 10 lines
include/hw/virtio/vhost-vdpa.h +2
-0@@ -31,6 +31,8 @@ typedef struct vhost_vdpa {bool iotlb_batch_begin_sent;MemoryListener listener;struct vhost_vdpa_iova_range iova_range; +/* VirtIO device features that can be emulated by qemu */ +uint64_t added_features;Any reason we need a per
vhost_vdpastorage for this? Or is there a chance that this field could be different among the devices?Quoted 2 lines
Yes, one device could support SVQ and the other one could not support it because of different feature sets for example.
Right, but for those devices that don't support SVQ, we don't even need mediation for feature like
F_LOGand_F_STATUS?
Quoted 7 lines
Quoted 1 line
No, and we cannot offer it to the guest either.
Just to make sure we are on the same page, what I meant is, consider in the future SVQ get the support of all features, so we can remove this field? This is because
_F_STATUScan be mediated unconditionally anyhow.
For _F_STATUS that is right. But we cannot handle full _F_GUEST_ANNOUNCE since control SVQ (will) needs features from the device that cannot be emulated, like ASID.
I think your point is "Since qemu cannot migrate these devices it will never set VIRTIO_NET_S_ANNOUNCE, so the guest will never send VIRTIO_NET_CTRL_ANNOUNCE messages". And I think that is totally right, but I still feel it is weird to expose it if we cannot handle it.
Maybe a good first step is to move added_features to vhost_net, or maybe to convert it to "bool guest_announce_emulated" or something similar? This way hw/virtio/vhost-vdpa is totally unaware of this and changes are more self contained.
Thanks!
Quoted 62 lines
Thanks
Quoted 59 lines
Quoted 58 lines
Thanks
Quoted 54 lines
Thanks!
Quoted 50 lines
Thanks
Quoted 47 lines
include/hw/virtio/vhost-vdpa.h +0
-0uint64_t acked_features;bool shadow_vqs_enabled;/* IOVA mapping used by the Shadow Virtqueue */ hw/virtio/vhost-vdpa.c +4
-4@@ -660,8 +660,8 @@ static int vhost_vdpa_set_features(struct vhost_dev *dev, v->acked_features = features; -/* We must not ack _F_LOG if SVQ is enabled */ - features &= ~BIT_ULL (VHOST_F_LOG_ALL ); +/* Do not ack features emulated by qemu */ + features &= ~v->added_features; }trace_vhost_vdpa_set_features (dev, features);@@ -1244,8 +1244,8 @@ static int vhost_vdpa_get_features(struct vhost_dev *dev,int ret =vhost_vdpa_get_dev_features (dev, features);if (ret ==0 && v->shadow_vqs_enabled) { -/* Add SVQ logging capabilities */ - *features |=BIT_ULL (VHOST_F_LOG_ALL ); +/* Add emulated capabilities */ + *features |= v->added_features; }return ret;net/vhost-vdpa.c +4
-1@@ -599,6 +599,10 @@ static NetClientState *net_vhost_vdpa_init(NetClientState *peer, s->vhost_vdpa.index = queue_pair_index; s->vhost_vdpa.shadow_vqs_enabled = svq; s->vhost_vdpa.iova_tree = iova_tree; +if (svq) { +/* Add SVQ logging capabilities */ + s->vhost_vdpa.added_features |=BIT_ULL (VHOST_F_LOG_ALL ); + }if (!is_datapath) { s->cvq_cmd_out_buffer =qemu_memalign (qemu_real_host_page_size (),vhost_vdpa_net_cvq_cmd_page_len ()); --2.31.1
Re: [RFC PATCH v2 2/8] vdpa: Save emulated features list in vhost_vdpa
Quoted 67 lines · Eugenio Perez Martin
Quoted 48 lines · Jason Wang
Quoted 45 lines · Eugenio Perez Martin
Quoted 42 lines · Jason Wang
Quoted 33 lines · Eugenio Perez Martin
Quoted 16 lines · Jason Wang
Quoted 14 lines · Eugenio Pérez
At this moment only
_F_LOGis added there.However future patches add features that depend on the kind of device. In particular, only net devices can add
VIRTIO_F_GUEST_ANNOUNCE. So let's allowvhost_vdpacreator to set custom emulated device features.Signed-off-by Eugenio Pérez <[email protected]>--- include/hw/virtio/vhost-vdpa.h | 2 ++ hw/virtio/vhost-vdpa.c | 8 ++++---- net/vhost-vdpa.c | 4 ++++ 3 files changed, 10 insertions(+), 4 deletions(-)Quoted 14 lines
include/hw/virtio/vhost-vdpa.h typedef struct vhost_vdpa { · line 35
Quoted 10 lines
include/hw/virtio/vhost-vdpa.h +2
-0@@ -31,6 +31,8 @@ typedef struct vhost_vdpa {bool iotlb_batch_begin_sent;MemoryListener listener;struct vhost_vdpa_iova_range iova_range; +/* VirtIO device features that can be emulated by qemu */ +uint64_t added_features;Any reason we need a per
vhost_vdpastorage for this? Or is there a chance that this field could be different among the devices?Quoted 2 lines
Yes, one device could support SVQ and the other one could not support it because of different feature sets for example.
Right, but for those devices that don't support SVQ, we don't even need mediation for feature like
F_LOGand_F_STATUS?Quoted 7 lines
Quoted 1 line
No, and we cannot offer it to the guest either.
Just to make sure we are on the same page, what I meant is, consider in the future SVQ get the support of all features, so we can remove this field? This is because
_F_STATUScan be mediated unconditionally anyhow.For
_F_STATUSthat is right. But we cannot handle full_F_GUEST_ANNOUNCEsince control SVQ (will) needs features from the device that cannot be emulated, like ASID.I think your point is "Since qemu cannot migrate these devices it will never set
VIRTIO_NET_S_ANNOUNCE, so the guest will never sendVIRTIO_NET_CTRL_ANNOUNCEmessages". And I think that is totally right, but I still feel it is weird to expose it if we cannot handle it.
Quoted 4 lines
Maybe a good first step is to move
added_featurestovhost_net, or maybe to convert it to "boolguest_announce_emulated" or something similar? This way hw/virtio/vhost-vdpa is totally unaware of this and changes are more self contained.
This reminds me of something. For vhost, if Qemu can handle some feature bits, we don't need to validate if vhost has such support.
E.g we don't have _F_SATAUS and _F_GUEST_ANNOUNCE in kernel_feature_bits.
I wonder if we can do something similar for vhost-vDPA? Then we don't need to bother new fields.
Thanks
Quoted 68 lines
Thanks!
Quoted 62 lines
Thanks
Quoted 59 lines
Quoted 58 lines
Thanks
Quoted 54 lines
Thanks!
Quoted 50 lines
Thanks
Quoted 47 lines
include/hw/virtio/vhost-vdpa.h +0
-0uint64_t acked_features;bool shadow_vqs_enabled;/* IOVA mapping used by the Shadow Virtqueue */ hw/virtio/vhost-vdpa.c +4
-4@@ -660,8 +660,8 @@ static int vhost_vdpa_set_features(struct vhost_dev *dev, v->acked_features = features; -/* We must not ack _F_LOG if SVQ is enabled */ - features &= ~BIT_ULL (VHOST_F_LOG_ALL ); +/* Do not ack features emulated by qemu */ + features &= ~v->added_features; }trace_vhost_vdpa_set_features (dev, features);@@ -1244,8 +1244,8 @@ static int vhost_vdpa_get_features(struct vhost_dev *dev,int ret =vhost_vdpa_get_dev_features (dev, features);if (ret ==0 && v->shadow_vqs_enabled) { -/* Add SVQ logging capabilities */ - *features |=BIT_ULL (VHOST_F_LOG_ALL ); +/* Add emulated capabilities */ + *features |= v->added_features; }return ret;net/vhost-vdpa.c +4
-1@@ -599,6 +599,10 @@ static NetClientState *net_vhost_vdpa_init(NetClientState *peer, s->vhost_vdpa.index = queue_pair_index; s->vhost_vdpa.shadow_vqs_enabled = svq; s->vhost_vdpa.iova_tree = iova_tree; +if (svq) { +/* Add SVQ logging capabilities */ + s->vhost_vdpa.added_features |=BIT_ULL (VHOST_F_LOG_ALL ); + }if (!is_datapath) { s->cvq_cmd_out_buffer =qemu_memalign (qemu_real_host_page_size (),vhost_vdpa_net_cvq_cmd_page_len ()); --2.31.1
Re: [RFC PATCH v2 2/8] vdpa: Save emulated features list in vhost_vdpa
Quoted 75 lines · Jason Wang
Quoted 67 lines · Eugenio Perez Martin
Quoted 48 lines · Jason Wang
Quoted 45 lines · Eugenio Perez Martin
Quoted 42 lines · Jason Wang
Quoted 33 lines · Eugenio Perez Martin
Quoted 16 lines · Jason Wang
Quoted 14 lines · Eugenio Pérez
At this moment only
_F_LOGis added there.However future patches add features that depend on the kind of device. In particular, only net devices can add
VIRTIO_F_GUEST_ANNOUNCE. So let's allowvhost_vdpacreator to set custom emulated device features.Signed-off-by Eugenio Pérez <[email protected]>--- include/hw/virtio/vhost-vdpa.h | 2 ++ hw/virtio/vhost-vdpa.c | 8 ++++---- net/vhost-vdpa.c | 4 ++++ 3 files changed, 10 insertions(+), 4 deletions(-)Quoted 14 lines
include/hw/virtio/vhost-vdpa.h typedef struct vhost_vdpa { · line 35
Quoted 10 lines
include/hw/virtio/vhost-vdpa.h +2
-0@@ -31,6 +31,8 @@ typedef struct vhost_vdpa {bool iotlb_batch_begin_sent;MemoryListener listener;struct vhost_vdpa_iova_range iova_range; +/* VirtIO device features that can be emulated by qemu */ +uint64_t added_features;Any reason we need a per
vhost_vdpastorage for this? Or is there a chance that this field could be different among the devices?Quoted 2 lines
Yes, one device could support SVQ and the other one could not support it because of different feature sets for example.
Right, but for those devices that don't support SVQ, we don't even need mediation for feature like
F_LOGand_F_STATUS?Quoted 7 lines
Quoted 1 line
No, and we cannot offer it to the guest either.
Just to make sure we are on the same page, what I meant is, consider in the future SVQ get the support of all features, so we can remove this field? This is because
_F_STATUScan be mediated unconditionally anyhow.For
_F_STATUSthat is right. But we cannot handle full_F_GUEST_ANNOUNCEsince control SVQ (will) needs features from the device that cannot be emulated, like ASID.I think your point is "Since qemu cannot migrate these devices it will never set
VIRTIO_NET_S_ANNOUNCE, so the guest will never sendVIRTIO_NET_CTRL_ANNOUNCEmessages". And I think that is totally right, but I still feel it is weird to expose it if we cannot handle it.Quoted 4 lines
Maybe a good first step is to move
added_featurestovhost_net, or maybe to convert it to "boolguest_announce_emulated" or something similar? This way hw/virtio/vhost-vdpa is totally unaware of this and changes are more self contained.
Quoted 8 lines
This reminds me of something. For vhost, if Qemu can handle some feature bits, we don't need to validate if vhost has such support.
E.g we don't have
_F_SATAUSand_F_GUEST_ANNOUNCEinkernel_feature_bits.I wonder if we can do something similar for vhost-vDPA? Then we don't need to bother new fields.
That is valid for _F_GUEST_ANNOUNCE, because all of it is emulated and we must forbid the ack of it for simplicity.
But _F_STATUS has two parts, one bit that must be exposed to the guest from the physical NIC, since qemu cannot know when to change it (VIRTIO_NET_S_LINK_UP) and another one that must be 100% emulated (VIRTIO_NET_S_ANNOUNCE). VIRTIO_NET_F_STATUS is 100% emulated only if it is not offered by the device.
I'm moving to vhost_net for the next series, and I'll try to make all of this more explicit.
Thanks!
RFC PATCH v2 3/8 vhost_net: Emulate link state up if backend doesn't expose it
In reply to Eugenio PérezAt this moment this code path is not reached, but vdpa devices can offer VIRTIO_NET_F_STATUS unconditionally. While the guest must assume that link is always up by the standard, qemu will set the status bit to 1 always in this case.
This makes little use by itself, but VIRTIO_NET_F_STATUS is needed for the guest to read status bit VIRTIO_NET_F_GUEST_ANNOUNCE, used by feature VIRTIO_NET_F_GUEST_ANNOUNCE. So qemu must emulate status feature in case it needs to emulate the guest announce feature.
--- hw/net/vhost_net.c | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-)
hw/net/vhost_net.c +26 -1
@@ -117,7 +117,32 @@ uint64_t vhost_net_get_features(struct vhost_net *net, uint64_t features)int vhost_net_get_config (struct vhost_net *net,uint8_t *config,uint32_t config_len) { -return vhost_dev_get_config (&net->dev, config, config_len,NULL ); +VirtIODevice *vdev; +int r =vhost_dev_get_config (&net->dev, config, config_len,NULL ); + +if (unlikely (r !=0 )) { +return r; + } + + if (config_len < endof(struct virtio_net_config, status)) { +return 0 ; + } + +/* +* TODO: Perform this only if vhost_vdpa. +*/ + vdev = net->dev.vdev; +if (!vdev) { +/* Device is starting */ +return 0 ; + } + +if ((net->dev.acked_features &BIT_ULL (VIRTIO_NET_F_STATUS )) && + !(net->dev.features &BIT_ULL (VIRTIO_NET_F_STATUS ))) { + ((struct virtio_net_config *)config)->status |=VIRTIO_NET_S_LINK_UP ; + } + +return 0 ; }int vhost_net_set_config (struct vhost_net *net,const uint8_t *data,uint32_t offset,uint32_t size,uint32_t flags)
Signature · 1 line
2.31.1
Re: [RFC PATCH v2 3/8] vhost_net: Emulate link state up if backend doesn't expose it
Quoted 3 lines · Eugenio Pérez
At this moment this code path is not reached, but vdpa devices can offer
VIRTIO_NET_F_STATUSunconditionally.
So I guess what you mean is that, for the parent that doesn't support VIRTIO_NET_F_STATUS, emulate one for making sure the ANNOUCNE to work. This is safe since the spec said the driver will assume the link is always up if without this feature.
Quoted 27 lines
While the guest must assume that
link is always up by the standard, qemu will set the status bit to 1 always in this case.This makes little use by itself, but
VIRTIO_NET_F_STATUSis needed for the guest to read status bitVIRTIO_NET_F_GUEST_ANNOUNCE, used by featureVIRTIO_NET_F_GUEST_ANNOUNCE. So qemu must emulate status feature in case it needs to emulate the guest announce feature.Signed-off-by Eugenio Pérez <[email protected]>--- hw/net/vhost_net.c | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-)hw/net/vhost_net.c +4
-1@@ -117,7 +117,32 @@ uint64_t vhost_net_get_features(struct vhost_net *net, uint64_t features)int vhost_net_get_config (struct vhost_net *net,uint8_t *config,uint32_t config_len) { -return vhost_dev_get_config (&net->dev, config, config_len,NULL ); +VirtIODevice *vdev; +int r =vhost_dev_get_config (&net->dev, config, config_len,NULL ); + +if (unlikely (r !=0 )) {
hw/net/vhost_net.c uint64_t vhost_net_get_features(struct vhost_net *net, uint64_t features) · line 133
Quoted 10 lines
hw/net/vhost_net.c +10
-0+return r; + } + + if (config_len < endof(struct virtio_net_config, status)) { +return 0 ; + } + +/* +* TODO: Perform this only if vhost_vdpa. +*/
Cindy adds some mediation codes for vhost-vDPA in virtio_net_get_config(), so I believe it can be done there?
Thanks
Quoted 18 lines
hw/net/vhost_net.c +12
-1+ vdev = net->dev.vdev; +if (!vdev) { +/* Device is starting */ +return 0 ; + } + +if ((net->dev.acked_features &BIT_ULL (VIRTIO_NET_F_STATUS )) && + !(net->dev.features &BIT_ULL (VIRTIO_NET_F_STATUS ))) { + ((struct virtio_net_config *)config)->status |=VIRTIO_NET_S_LINK_UP ; + } + +return 0 ; }int vhost_net_set_config (struct vhost_net *net,const uint8_t *data,uint32_t offset,uint32_t size,uint32_t flags) --2.31.1
Re: [RFC PATCH v2 3/8] vhost_net: Emulate link state up if backend doesn't expose it
Quoted 11 lines · Jason Wang
Quoted 3 lines · Eugenio Pérez
At this moment this code path is not reached, but vdpa devices can offer
VIRTIO_NET_F_STATUSunconditionally.So I guess what you mean is that, for the parent that doesn't support
VIRTIO_NET_F_STATUS, emulate one for making sure the ANNOUCNE to work. This is safe since the spec said the driver will assume the link is always up if without this feature.
Right.
Quoted 32 lines
Quoted 32 lines
While the guest must assume that
link is always up by the standard, qemu will set the status bit to 1 always in this case.This makes little use by itself, but
VIRTIO_NET_F_STATUSis needed for the guest to read status bitVIRTIO_NET_F_GUEST_ANNOUNCE, used by featureVIRTIO_NET_F_GUEST_ANNOUNCE. So qemu must emulate status feature in case it needs to emulate the guest announce feature.Signed-off-by Eugenio Pérez <[email protected]>--- hw/net/vhost_net.c | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-)hw/net/vhost_net.c +9
-1@@ -117,7 +117,32 @@ uint64_t vhost_net_get_features(struct vhost_net *net, uint64_t features)int vhost_net_get_config (struct vhost_net *net,uint8_t *config,uint32_t config_len) { -return vhost_dev_get_config (&net->dev, config, config_len,NULL ); +VirtIODevice *vdev; +int r =vhost_dev_get_config (&net->dev, config, config_len,NULL ); + +if (unlikely (r !=0 )) { +return r; + } + + if (config_len < endof(struct virtio_net_config, status)) { +return 0 ;
Quoted 9 lines
hw/net/vhost_net.c uint64_t vhost_net_get_features(struct vhost_net *net, uint64_t features) · line 133
Quoted 5 lines
hw/net/vhost_net.c +5
-0+ } + +/* +* TODO: Perform this only if vhost_vdpa. +*/ Cindy adds some mediation codes for vhost-vDPA in
virtio_net_get_config(), so I believe it can be done there?
Sure, let me take a look.
Thanks!
Quoted 21 lines
Thanks
Quoted 18 lines
hw/net/vhost_net.c +12
-1+ vdev = net->dev.vdev; +if (!vdev) { +/* Device is starting */ +return 0 ; + } + +if ((net->dev.acked_features &BIT_ULL (VIRTIO_NET_F_STATUS )) && + !(net->dev.features &BIT_ULL (VIRTIO_NET_F_STATUS ))) { + ((struct virtio_net_config *)config)->status |=VIRTIO_NET_S_LINK_UP ; + } + +return 0 ; }int vhost_net_set_config (struct vhost_net *net,const uint8_t *data,uint32_t offset,uint32_t size,uint32_t flags) --2.31.1
RFC PATCH v2 4/8 vdpa: Expose VIRTIO_NET_F_STATUS unconditionally
In reply to Eugenio PérezNow that qemu can handle and emulate it if the vdpa backend does not support it we can offer it always.
--- net/vhost-vdpa.c | 3 +++ 1 file changed, 3 insertions(+)
net/vhost-vdpa.c +3 -0
@@ -602,6 +602,9 @@ static NetClientState *net_vhost_vdpa_init(NetClientState *peer,if (svq) {/* Add SVQ logging capabilities */ s->vhost_vdpa.added_features |=BIT_ULL (VHOST_F_LOG_ALL ); + +/* VIRTIO_NET_F_STATUS is mandatory for _F_GUEST_ANNOUNCE. */ + s->vhost_vdpa.added_features |=BIT_ULL (VIRTIO_NET_F_STATUS ); }if (!is_datapath) { s->cvq_cmd_out_buffer = qemu_memalign(qemu_real_host_page_size(),
Signature · 1 line
2.31.1
RFC PATCH v2 5/8 vdpa: Remove shadow CVQ command check
In reply to Eugenio PérezThe guest will see undefined behavior if it issue not negotiate commands, bit it is expected somehow.
Simplify code deleting this check.
--- net/vhost-vdpa.c | 48 ------------------------------------------------ 1 file changed, 48 deletions(-)
net/vhost-vdpa.c +0 -48
@@ -461,48 +461,6 @@ static NetClientInfo net_vhost_vdpa_cvq_info = { .check_peer_type = vhost_vdpa_check_peer_type, }; -/** -* Do not forward commands not supported by SVQ. Otherwise, the device could -* accept it and qemu would not know how to update the device model. -*/ -static bool vhost_vdpa_net_cvq_validate_cmd (const void *out_buf,size_t len) -{ -struct virtio_net_ctrl_hdr ctrl; - -if (unlikely (len <sizeof (ctrl))) { -qemu_log_mask (LOG_GUEST_ERROR , -"%s: invalid legnth of out buffer %zu\n" , __func__, len); -return false; - } - -memcpy (&ctrl, out_buf,sizeof (ctrl)); -switch (ctrl.class) { -case VIRTIO_NET_CTRL_MAC : -switch (ctrl.cmd) { -case VIRTIO_NET_CTRL_MAC_ADDR_SET : -return true; -default : -qemu_log_mask (LOG_GUEST_ERROR ,"%s: invalid mac cmd %u\n" , - __func__, ctrl.cmd); - }; -break ; -case VIRTIO_NET_CTRL_MQ : -switch (ctrl.cmd) { -case VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET : -return true; -default : -qemu_log_mask (LOG_GUEST_ERROR ,"%s: invalid mq cmd %u\n" , - __func__, ctrl.cmd); - }; -break ; -default : -qemu_log_mask (LOG_GUEST_ERROR ,"%s: invalid control class %u\n" , - __func__, ctrl.class); - }; - -return false; -} -/** * Validate and copy control virtqueue commands. *
@@ -526,16 +484,10 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, .iov_len = sizeof(status), };ssize_t dev_written = -EINVAL ; -bool ok; out.iov_len =iov_to_buf (elem->out_sg, elem->out_num,0 , s->cvq_cmd_out_buffer,vhost_vdpa_net_cvq_cmd_len ()); - ok =vhost_vdpa_net_cvq_validate_cmd (s->cvq_cmd_out_buffer, out.iov_len); -if (unlikely (!ok)) { - goto out; - } - dev_written =vhost_vdpa_net_cvq_add (s, out.iov_len,sizeof (status));if (unlikely (dev_written <0 )) { goto out;
Signature · 1 line
2.31.1
Re: [RFC PATCH v2 5/8] vdpa: Remove shadow CVQ command check
Quoted 7 lines · Eugenio Pérez
The guest will see undefined behavior if it issue not negotiate commands, bit it is expected somehow.
Simplify code deleting this check.
Signed-off-by Eugenio Pérez <[email protected]>
Quoted 77 lines
--- net/vhost-vdpa.c | 48 ------------------------------------------------ 1 file changed, 48 deletions(-)net/vhost-vdpa.c +0
-49@@ -461,48 +461,6 @@ static NetClientInfo net_vhost_vdpa_cvq_info = { .check_peer_type = vhost_vdpa_check_peer_type, }; -/** -* Do not forward commands not supported by SVQ. Otherwise, the device could -* accept it and qemu would not know how to update the device model. -*/ -static bool vhost_vdpa_net_cvq_validate_cmd (const void *out_buf,size_t len) -{ -struct virtio_net_ctrl_hdr ctrl; - -if (unlikely (len <sizeof (ctrl))) { -qemu_log_mask (LOG_GUEST_ERROR , -"%s: invalid legnth of out buffer %zu\n" , __func__, len); -return false; - } - -memcpy (&ctrl, out_buf,sizeof (ctrl)); -switch (ctrl.class) { -case VIRTIO_NET_CTRL_MAC : -switch (ctrl.cmd) { -case VIRTIO_NET_CTRL_MAC_ADDR_SET : -return true; -default : -qemu_log_mask (LOG_GUEST_ERROR ,"%s: invalid mac cmd %u\n" , - __func__, ctrl.cmd); - }; -break ; -case VIRTIO_NET_CTRL_MQ : -switch (ctrl.cmd) { -case VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET : -return true; -default : -qemu_log_mask (LOG_GUEST_ERROR ,"%s: invalid mq cmd %u\n" , - __func__, ctrl.cmd); - }; -break ; -default : -qemu_log_mask (LOG_GUEST_ERROR ,"%s: invalid control class %u\n" , - __func__, ctrl.class); - }; - -return false; -} -/** * Validate and copy control virtqueue commands. * @@ -526,16 +484,10 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, .iov_len = sizeof(status), };ssize_t dev_written = -EINVAL ; -bool ok; out.iov_len =iov_to_buf (elem->out_sg, elem->out_num,0 , s->cvq_cmd_out_buffer,vhost_vdpa_net_cvq_cmd_len ()); - ok =vhost_vdpa_net_cvq_validate_cmd (s->cvq_cmd_out_buffer, out.iov_len); -if (unlikely (!ok)) { - goto out; - } - dev_written =vhost_vdpa_net_cvq_add (s, out.iov_len,sizeof (status));if (unlikely (dev_written <0 )) { goto out; --2.31.1
RFC PATCH v2 6/8 vdpa: handle VIRTIO_NET_CTRL_ANNOUNCE in vhost_vdpa_net_handle_ctrl_avail
In reply to Eugenio PérezSince this capability is emulated by qemu shadowed CVQ cannot forward it to the device. Process all that command within qemu.
--- net/vhost-vdpa.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-)
net/vhost-vdpa.c +12 -3
@@ -488,9 +488,18 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, out.iov_len =iov_to_buf (elem->out_sg, elem->out_num,0 , s->cvq_cmd_out_buffer,vhost_vdpa_net_cvq_cmd_len ()); - dev_written =vhost_vdpa_net_cvq_add (s, out.iov_len,sizeof (status)); -if (unlikely (dev_written <0 )) { - goto out; +if (*(uint8_t *)s->cvq_cmd_out_buffer ==VIRTIO_NET_CTRL_ANNOUNCE ) { +/* +* Guest announce capability is emulated by qemu, so dont forward to +* the device. +*/ + dev_written =sizeof (status); + *s->status =VIRTIO_NET_OK ; + }else { + dev_written =vhost_vdpa_net_cvq_add (s, out.iov_len,sizeof (status)); +if (unlikely (dev_written <0 )) { + goto out; + } }if (unlikely (dev_written <sizeof (status))) {
Signature · 1 line
2.31.1
Re: [RFC PATCH v2 6/8] vdpa: handle VIRTIO_NET_CTRL_ANNOUNCE in vhost_vdpa_net_handle_ctrl_avail
Quoted 3 lines · Eugenio Pérez
Since this capability is emulated by qemu shadowed CVQ cannot forward it to the device.
I wonder what happens for a device that has GUEST_ANNOUNCE support on its own?
Quoted 12 lines
Process all that command within qemu.
Signed-off-by Eugenio Pérez <[email protected]>--- net/vhost-vdpa.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-)net/vhost-vdpa.c +0
-0
net/vhost-vdpa.c static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, · line 491
Quoted 7 lines
net/vhost-vdpa.c +1
-3@@ -488,9 +488,18 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, out.iov_len =iov_to_buf (elem->out_sg, elem->out_num,0 , s->cvq_cmd_out_buffer,vhost_vdpa_net_cvq_cmd_len ()); - dev_written =vhost_vdpa_net_cvq_add (s, out.iov_len,sizeof (status)); -if (unlikely (dev_written <0 )) { - goto out; +if (*(uint8_t *)s->cvq_cmd_out_buffer ==VIRTIO_NET_CTRL_ANNOUNCE ) {
Interesting, I thought we can do better by forbidding the code that goes into vhost-vDPA specific code, everything should be set at virtio-net.c level.
Thanks
Quoted 17 lines
net/vhost-vdpa.c +11
-1+/* +* Guest announce capability is emulated by qemu, so dont forward to +* the device. +*/ + dev_written =sizeof (status); + *s->status =VIRTIO_NET_OK ; + }else { + dev_written =vhost_vdpa_net_cvq_add (s, out.iov_len,sizeof (status)); +if (unlikely (dev_written <0 )) { + goto out; + } }if (unlikely (dev_written <sizeof (status))) { --2.31.1
Re: [RFC PATCH v2 6/8] vdpa: handle VIRTIO_NET_CTRL_ANNOUNCE in vhost_vdpa_net_handle_ctrl_avail
Quoted 8 lines · Jason Wang
Quoted 3 lines · Eugenio Pérez
Since this capability is emulated by qemu shadowed CVQ cannot forward it to the device.
I wonder what happens for a device that has
GUEST_ANNOUNCEsupport on its own?
If SVQ is enabled the feature is always emulated by qemu by this series.
if SVQ is disabled then the device is the one in charge of all of it.
Quoted 7 lines
Quoted 7 lines
Process all that command within qemu.
Signed-off-by Eugenio Pérez <[email protected]>--- net/vhost-vdpa.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-)
Quoted 17 lines
net/vhost-vdpa.c static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, · line 491
Quoted 12 lines
net/vhost-vdpa.c +1
-3@@ -488,9 +488,18 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, out.iov_len =iov_to_buf (elem->out_sg, elem->out_num,0 , s->cvq_cmd_out_buffer,vhost_vdpa_net_cvq_cmd_len ()); - dev_written =vhost_vdpa_net_cvq_add (s, out.iov_len,sizeof (status)); -if (unlikely (dev_written <0 )) { - goto out; +if (*(uint8_t *)s->cvq_cmd_out_buffer ==VIRTIO_NET_CTRL_ANNOUNCE ) {Interesting, I thought we can do better by forbidding the code that goes into vhost-vDPA specific code, everything should be set at
virtio-net.clevel.
Do you mean to move the SVQ processing to each handle_output? It's somehow on the roadmap but I'm not sure if it has more priority than implementing the different features.
Thanks!
Quoted 20 lines
Thanks
Quoted 17 lines
net/vhost-vdpa.c +11
-1+/* +* Guest announce capability is emulated by qemu, so dont forward to +* the device. +*/ + dev_written =sizeof (status); + *s->status =VIRTIO_NET_OK ; + }else { + dev_written =vhost_vdpa_net_cvq_add (s, out.iov_len,sizeof (status)); +if (unlikely (dev_written <0 )) { + goto out; + } }if (unlikely (dev_written <sizeof (status))) { --2.31.1
Re: [RFC PATCH v2 6/8] vdpa: handle VIRTIO_NET_CTRL_ANNOUNCE in vhost_vdpa_net_handle_ctrl_avail
Quoted 8 lines · Eugenio Perez Martin
Quoted 6 lines · Jason Wang
Quoted 3 lines · Eugenio Pérez
Since this capability is emulated by qemu shadowed CVQ cannot forward it to the device.
Quoted 6 lines
Quoted 2 lines
I wonder what happens for a device that has
GUEST_ANNOUNCEsupport on its own?If SVQ is enabled the feature is always emulated by qemu by this series.
if SVQ is disabled then the device is the one in charge of all of it.
Ok, I see.
Quoted 26 lines
Quoted 8 lines
Quoted 8 lines
Process all that command within qemu.
Signed-off-by Eugenio Pérez <[email protected]>--- net/vhost-vdpa.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-)Quoted 17 lines
net/vhost-vdpa.c static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, · line 491
Quoted 12 lines
net/vhost-vdpa.c +1
-3@@ -488,9 +488,18 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, out.iov_len =iov_to_buf (elem->out_sg, elem->out_num,0 , s->cvq_cmd_out_buffer,vhost_vdpa_net_cvq_cmd_len ()); - dev_written =vhost_vdpa_net_cvq_add (s, out.iov_len,sizeof (status)); -if (unlikely (dev_written <0 )) { - goto out; +if (*(uint8_t *)s->cvq_cmd_out_buffer ==VIRTIO_NET_CTRL_ANNOUNCE ) {Interesting, I thought we can do better by forbidding the code that goes into vhost-vDPA specific code, everything should be set at
virtio-net.clevel.
Quoted 3 lines
Do you mean to move the SVQ processing to each
handle_output? It's somehow on the roadmap but I'm not sure if it has more priority than implementing the different features.
Right, but I think we need to find a way to eliminate the casting here.
Thanks
Quoted 25 lines
Thanks!
Quoted 20 lines
Thanks
Quoted 17 lines
net/vhost-vdpa.c +11
-1+/* +* Guest announce capability is emulated by qemu, so dont forward to +* the device. +*/ + dev_written =sizeof (status); + *s->status =VIRTIO_NET_OK ; + }else { + dev_written =vhost_vdpa_net_cvq_add (s, out.iov_len,sizeof (status)); +if (unlikely (dev_written <0 )) { + goto out; + } }if (unlikely (dev_written <sizeof (status))) { --2.31.1
Re: [RFC PATCH v2 6/8] vdpa: handle VIRTIO_NET_CTRL_ANNOUNCE in vhost_vdpa_net_handle_ctrl_avail
Quoted 46 lines · Jason Wang
Quoted 8 lines · Eugenio Perez Martin
Quoted 6 lines · Jason Wang
Quoted 3 lines · Eugenio Pérez
Since this capability is emulated by qemu shadowed CVQ cannot forward it to the device.
Quoted 6 lines
Quoted 2 lines
I wonder what happens for a device that has
GUEST_ANNOUNCEsupport on its own?If SVQ is enabled the feature is always emulated by qemu by this series.
if SVQ is disabled then the device is the one in charge of all of it.
Ok, I see.
Quoted 26 lines
Quoted 25 lines
Quoted 13 lines
Process all that command within qemu.
Signed-off-by Eugenio Pérez <[email protected]>--- net/vhost-vdpa.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-)net/vhost-vdpa.c +0
-0net/vhost-vdpa.c static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, · line 491
Quoted 7 lines
net/vhost-vdpa.c +1
-3@@ -488,9 +488,18 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, out.iov_len =iov_to_buf (elem->out_sg, elem->out_num,0 , s->cvq_cmd_out_buffer,vhost_vdpa_net_cvq_cmd_len ()); - dev_written =vhost_vdpa_net_cvq_add (s, out.iov_len,sizeof (status)); -if (unlikely (dev_written <0 )) { - goto out; +if (*(uint8_t *)s->cvq_cmd_out_buffer ==VIRTIO_NET_CTRL_ANNOUNCE ) {Interesting, I thought we can do better by forbidding the code that goes into vhost-vDPA specific code, everything should be set at
virtio-net.clevel.
Quoted 6 lines
Quoted 3 lines
Do you mean to move the SVQ processing to each
handle_output? It's somehow on the roadmap but I'm not sure if it has more priority than implementing the different features.Right, but I think we need to find a way to eliminate the casting here.
Would it work to use it this way?uint8_t *virtio_net_ctrl_class = s->cvq_cmd_out_buffer if (*virtio_net_ctrl_class == VIRTIO_NET_CTRL_ANNOUNCE) {
...
}
Quoted 28 lines
Thanks
Quoted 25 lines
Thanks!
Quoted 20 lines
Thanks
Quoted 17 lines
net/vhost-vdpa.c +11
-1+/* +* Guest announce capability is emulated by qemu, so dont forward to +* the device. +*/ + dev_written =sizeof (status); + *s->status =VIRTIO_NET_OK ; + }else { + dev_written =vhost_vdpa_net_cvq_add (s, out.iov_len,sizeof (status)); +if (unlikely (dev_written <0 )) { + goto out; + } }if (unlikely (dev_written <sizeof (status))) { --2.31.1
Re: [RFC PATCH v2 6/8] vdpa: handle VIRTIO_NET_CTRL_ANNOUNCE in vhost_vdpa_net_handle_ctrl_avail
Quoted 52 lines · Eugenio Perez Martin
Quoted 50 lines · Jason Wang
Quoted 8 lines · Eugenio Perez Martin
Quoted 6 lines · Jason Wang
Quoted 3 lines · Eugenio Pérez
Since this capability is emulated by qemu shadowed CVQ cannot forward it to the device.
Quoted 6 lines
Quoted 2 lines
I wonder what happens for a device that has
GUEST_ANNOUNCEsupport on its own?If SVQ is enabled the feature is always emulated by qemu by this series.
if SVQ is disabled then the device is the one in charge of all of it.
Ok, I see.
Quoted 29 lines
Quoted 8 lines
Quoted 8 lines
Process all that command within qemu.
Signed-off-by Eugenio Pérez <[email protected]>--- net/vhost-vdpa.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-)Quoted 17 lines
net/vhost-vdpa.c static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, · line 491
Quoted 12 lines
net/vhost-vdpa.c +1
-3@@ -488,9 +488,18 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, out.iov_len =iov_to_buf (elem->out_sg, elem->out_num,0 , s->cvq_cmd_out_buffer,vhost_vdpa_net_cvq_cmd_len ()); - dev_written =vhost_vdpa_net_cvq_add (s, out.iov_len,sizeof (status)); -if (unlikely (dev_written <0 )) { - goto out; +if (*(uint8_t *)s->cvq_cmd_out_buffer ==VIRTIO_NET_CTRL_ANNOUNCE ) {Interesting, I thought we can do better by forbidding the code that goes into vhost-vDPA specific code, everything should be set at
virtio-net.clevel.Do you mean to move the SVQ processing to each
handle_output? It's somehow on the roadmap but I'm not sure if it has more priority than implementing the different features.
Quoted 8 lines
Quoted 2 lines
Right, but I think we need to find a way to eliminate the casting here.
Would it work to use it this way?
uint8_t*virtio_net_ctrl_class=s->cvq_cmd_out_bufferif (*virtio_net_ctrl_class==VIRTIO_NET_CTRL_ANNOUNCE) {
...
}
Something like this, yes.
Thanks
Quoted 30 lines
Quoted 29 lines
Thanks
Quoted 25 lines
Thanks!
Quoted 20 lines
Thanks
Quoted 17 lines
net/vhost-vdpa.c +11
-1+/* +* Guest announce capability is emulated by qemu, so dont forward to +* the device. +*/ + dev_written =sizeof (status); + *s->status =VIRTIO_NET_OK ; + }else { + dev_written =vhost_vdpa_net_cvq_add (s, out.iov_len,sizeof (status)); +if (unlikely (dev_written <0 )) { + goto out; + } }if (unlikely (dev_written <sizeof (status))) { --2.31.1
RFC PATCH v2 7/8 vhost_net: return VIRTIO_NET_S_ANNOUNCE is device model has it set
In reply to Eugenio PérezTemporal, as this affects other vhost backends and we must check status feature first.
--- hw/net/vhost_net.c | 8 ++++++++ 1 file changed, 8 insertions(+)
hw/net/vhost_net.c +8 -0
@@ -118,6 +118,7 @@ int vhost_net_get_config(struct vhost_net *net, uint8_t *config, uint32_t config_len) {VirtIODevice *vdev; +VirtIONet *n;int r =vhost_dev_get_config (&net->dev, config, config_len,NULL );if (unlikely (r !=0 )) {
@@ -142,6 +143,13 @@ int vhost_net_get_config(struct vhost_net *net, uint8_t *config, ((struct virtio_net_config *)config)->status |=VIRTIO_NET_S_LINK_UP ; } +if (!(net->dev.acked_features &BIT_ULL (VIRTIO_NET_F_GUEST_ANNOUNCE ))) { +return 0 ; + } + + n =VIRTIO_NET (vdev); + ((struct virtio_net_config *)config)->status |= + (n->status &VIRTIO_NET_S_ANNOUNCE );return 0 ; } int vhost_net_set_config(struct vhost_net *net, const uint8_t *data,
Signature · 1 line
2.31.1
Re: [RFC PATCH v2 7/8] vhost_net: return VIRTIO_NET_S_ANNOUNCE is device model has it set
Quoted 22 lines · Eugenio Pérez
Temporal, as this affects other vhost backends and we must check status feature first.
Signed-off-by Eugenio Pérez <[email protected]>--- hw/net/vhost_net.c | 8 ++++++++ 1 file changed, 8 insertions(+)hw/net/vhost_net.c +1
-0@@ -118,6 +118,7 @@ int vhost_net_get_config(struct vhost_net *net, uint8_t *config, uint32_t config_len) {VirtIODevice *vdev; +VirtIONet *n;int r =vhost_dev_get_config (&net->dev, config, config_len,NULL );if (unlikely (r !=0 )) {
hw/net/vhost_net.c int vhost_net_get_config(struct vhost_net *net, uint8_t *config, · line 152
Quoted 10 lines
hw/net/vhost_net.c +7
-0@@ -142,6 +143,13 @@ int vhost_net_get_config(struct vhost_net *net, uint8_t *config, ((struct virtio_net_config *)config)->status |=VIRTIO_NET_S_LINK_UP ; } +if (!(net->dev.acked_features &BIT_ULL (VIRTIO_NET_F_GUEST_ANNOUNCE ))) { +return 0 ; + } + + n =VIRTIO_NET (vdev); + ((struct virtio_net_config *)config)->status |= + (n->status &VIRTIO_NET_S_ANNOUNCE );
Similar to the previous patch, it would be better to move this to virtio-net.c.
Thanks
Quoted 6 lines
hw/net/vhost_net.c +0
-1return 0 ; } int vhost_net_set_config(struct vhost_net *net, const uint8_t *data, --2.31.1
RFC PATCH v2 8/8 vdpa: Offer VIRTIO_NET_F_GUEST_ANNOUNCE feature if SVQ is enabled
In reply to Eugenio PérezSo qemu emulates it in case the device does not support it.
--- net/vhost-vdpa.c | 3 +++ 1 file changed, 3 insertions(+)
net/vhost-vdpa.c +3 -0
@@ -566,6 +566,9 @@ static NetClientState *net_vhost_vdpa_init(NetClientState *peer,/* VIRTIO_NET_F_STATUS is mandatory for _F_GUEST_ANNOUNCE. */ s->vhost_vdpa.added_features |=BIT_ULL (VIRTIO_NET_F_STATUS ); + +/* We can emulate guest announce shadowing CVQ */ + s->vhost_vdpa.added_features |=BIT_ULL (VIRTIO_NET_F_GUEST_ANNOUNCE ); }if (!is_datapath) { s->cvq_cmd_out_buffer = qemu_memalign(qemu_real_host_page_size(),
Signature · 1 line
2.31.1
Quoted 3 lines · Eugenio Pérez
A gratuitous ARP is recommended after a live migration to reduce the amount of time needed by the network links to be aware of the new location.
A question: I think we need to deal with the case when GUSET_ANNOUNCE is not negotiated? E.d sending the gARP by ourselves via vhost-vDPA?
Thanks
Quoted 37 lines
A hypervisor
may not have the knowledge of the guest network configuration, and this is especially true on passthrough devices, so its simpler to ask the guest to do it.However, the device control part of this feature can be totally emulated by qemu and shadow virtqueue, not needing any special feature from the actual vdpa device.
VIRTIO_NET_F_STATUSis also needed for the guest to access the status of virtio net config where announcement status bit is set. Emulating it as always active in case backend does not support it.v2:
* AddVIRTIO_NET_F_STATUSemulation.Eugenio Pérez (8):
vdpa: Delete duplicatedvdpa_feature_bitsentry
vdpa: Save emulated features list invhost_vdpa
vhost_net: Emulate link state up if backend doesn't expose it
vdpa: ExposeVIRTIO_NET_F_STATUSunconditionally
vdpa: Remove shadow CVQ command check
vdpa: handleVIRTIO_NET_CTRL_ANNOUNCEin
vhost_vdpa_net_handle_ctrl_avail
vhost_net: returnVIRTIO_NET_S_ANNOUNCEis device model has it set
vdpa: OfferVIRTIO_NET_F_GUEST_ANNOUNCEfeature if SVQ is enabledinclude/hw/virtio/vhost-vdpa.h | 2 + hw/net/vhost_net.c | 35 +++++++++++++++- hw/virtio/vhost-vdpa.c | 8 ++-- net/vhost-vdpa.c | 74 ++++++++++------------------------ 4 files changed, 62 insertions(+), 57 deletions(-)--
2.31.1
Quoted 9 lines · Jason Wang
Quoted 3 lines · Eugenio Pérez
A gratuitous ARP is recommended after a live migration to reduce the amount of time needed by the network links to be aware of the new location.
A question: I think we need to deal with the case when
GUSET_ANNOUNCEis not negotiated? E.d sending the gARP by ourselves via vhost-vDPA?
That is possible and I totally agree to implement it on top of this series, but it complicates the code in two ways:
1. The startup will be slower and more complicated.
We can only send the gARP from qemu using SVQ. At this point SVQ in dataplane is always enabled as long as cmdline x-svq is on, but in the final form it will not be the case, and dataplane will be passthrough.
If we want to send gARP from qemu, this will imply to start the device all in SVQ, and then reset it to its final configuration.
2. Qemu may not know the actual guest state
For example, regarding the mac filtering qemu will simply allow all unicast if too many macs are configured in the device, not saving them individually.
However, I think it is better than nothing for the guest that does not support GUEST_ANNOUNCE so, as said, I'm ok to implement it on top of this series for sure. But I think other features should have more priority, isn't it?
Thanks!
Quoted 40 lines
Thanks
Quoted 37 lines
A hypervisor
may not have the knowledge of the guest network configuration, and this is especially true on passthrough devices, so its simpler to ask the guest to do it.However, the device control part of this feature can be totally emulated by qemu and shadow virtqueue, not needing any special feature from the actual vdpa device.
VIRTIO_NET_F_STATUSis also needed for the guest to access the status of virtio net config where announcement status bit is set. Emulating it as always active in case backend does not support it.v2:
* AddVIRTIO_NET_F_STATUSemulation.Eugenio Pérez (8):
vdpa: Delete duplicatedvdpa_feature_bitsentry
vdpa: Save emulated features list invhost_vdpa
vhost_net: Emulate link state up if backend doesn't expose it
vdpa: ExposeVIRTIO_NET_F_STATUSunconditionally
vdpa: Remove shadow CVQ command check
vdpa: handleVIRTIO_NET_CTRL_ANNOUNCEin
vhost_vdpa_net_handle_ctrl_avail
vhost_net: returnVIRTIO_NET_S_ANNOUNCEis device model has it set
vdpa: OfferVIRTIO_NET_F_GUEST_ANNOUNCEfeature if SVQ is enabledinclude/hw/virtio/vhost-vdpa.h | 2 + hw/net/vhost_net.c | 35 +++++++++++++++- hw/virtio/vhost-vdpa.c | 8 ++-- net/vhost-vdpa.c | 74 ++++++++++------------------------ 4 files changed, 62 insertions(+), 57 deletions(-)--
2.31.1