Skip to contentSkip to conversation

Guest announce feature emulation using Shadow VirtQueue

PATCH Superseded Eugenio Pérez 3 participants 19 messages Nov 24, 2022 – Dec 20, 2022 Networking

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover Guest announce feature emulation using Shadow VirtQueue

A gratuitous ARP is recommended after a live migration to reduce the amount of time needed by the network links to be aware of the new location. A hypervisor may not have the knowledge of the guest network configuration, and this is especially true on passthrough devices, so its simpler to ask the guest to do it.

However, the device control part of this feature can be totally emulated by qemu and shadow virtqueue, not needing any special feature from the actual vdpa device.

The vdpa device must offer VIRTIO_NET_F_STATUS for the guest to access the status of virtio net config where announcement status bit is set. It is possible to emulate it as always active in case backend does not support it, but this is left for the future, as there are not many devices not offering it anyway.

Patch 1 is less useful now that we don't emulate _F_STATUS anymore but QEMU coding style seems to prefer early return so leaving it in this version.

v2:
* Actually remove VIRTIO_NET_F_STATUS emulation. Comparing with v1, we offer the feature with virtio instead of using virtio/vhost-vdpa.

v1:
* Move code from vhost_net_get_config to virtio_net_get_config.

RFC v2:
* Add VIRTIO_NET_F_STATUS emulation.

Eugenio Pérez (4):
virtio_net: Modify virtio_net_get_config to early return
virtio_net: copy VIRTIO_NET_S_ANNOUNCE if device model has it
vdpa: handle VIRTIO_NET_CTRL_ANNOUNCE in
vhost_vdpa_net_handle_ctrl_avail
vdpa: do not handle VIRTIO_NET_F_GUEST_ANNOUNCE in vhost-vdpa

 hw/net/virtio-net.c | 29 ++++++++++++++++-------------
 net/vhost-vdpa.c    | 16 ++++++++++++----
 2 files changed, 28 insertions(+), 17 deletions(-)
Signature · 1 line
2.31.1
#1 lore raw

PATCH v2 1/4 virtio_net: Modify virtio_net_get_config to early return

Next patches introduce more code on vhost-vdpa branch, with already have too much indentation.

---
 hw/net/virtio-net.c | 28 +++++++++++++++-------------
 1 file changed, 15 insertions(+), 13 deletions(-)
hw/net/virtio-net.c +15 -13
index aba12759d5..eed629766f 100644
@@ -168,20 +168,22 @@ static void virtio_net_get_config(VirtIODevice *vdev, uint8_t *config)
     if (nc->peer && nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_VDPA) {
         ret = vhost_net_get_config(get_vhost_net(nc->peer), (uint8_t *)&netcfg,
                                    n->config_size);
-        if (ret != -1) {
-            /*
-             * Some NIC/kernel combinations present 0 as the mac address.  As
-             * that is not a legal address, try to proceed with the
-             * address from the QEMU command line in the hope that the
-             * address has been configured correctly elsewhere - just not
-             * reported by the device.
-             */
-            if (memcmp(&netcfg.mac, &zero, sizeof(zero)) == 0) {
-                info_report("Zero hardware mac address detected. Ignoring.");
-                memcpy(netcfg.mac, n->mac, ETH_ALEN);
-            }
-            memcpy(config, &netcfg, n->config_size);
+        if (ret == -1) {
+            return;
         }
+
+        /*
+         * Some NIC/kernel combinations present 0 as the mac address.  As that
+         * is not a legal address, try to proceed with the address from the
+         * QEMU command line in the hope that the address has been configured
+         * correctly elsewhere - just not reported by the device.
+         */
+        if (memcmp(&netcfg.mac, &zero, sizeof(zero)) == 0) {
+            info_report("Zero hardware mac address detected. Ignoring.");
+            memcpy(netcfg.mac, n->mac, ETH_ALEN);
+        }
+
+        memcpy(config, &netcfg, n->config_size);
     }
 }
 
Signature · 1 line
2.31.1
#3 lore raw

PATCH v2 2/4 virtio_net: copy VIRTIO_NET_S_ANNOUNCE if device model has it

In reply to Eugenio Pérez

Status part of the emulated feature. It will follow device model, so we must copy it as long as NIC device model has it set.

---
 hw/net/virtio-net.c | 1 +
 1 file changed, 1 insertion(+)
hw/net/virtio-net.c +1 -0
index eed629766f..bf71ef33e8 100644
@@ -183,6 +183,7 @@ static void virtio_net_get_config(VirtIODevice *vdev, uint8_t *config)
             memcpy(netcfg.mac, n->mac, ETH_ALEN);
         }
 
+        netcfg.status |= (n->status & VIRTIO_NET_S_ANNOUNCE);
         memcpy(config, &netcfg, n->config_size);
     }
 }
Signature · 1 line
2.31.1
#7 lore raw

PATCH v2 3/4 vdpa: handle VIRTIO_NET_CTRL_ANNOUNCE in vhost_vdpa_net_handle_ctrl_avail

In reply to Eugenio Pérez

Since this capability is emulated by qemu shadowed CVQ cannot forward it to the device. Process all that command within qemu.

---
 net/vhost-vdpa.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
net/vhost-vdpa.c +12 -3
index 2b4b85d8f8..8172aa8449 100644
@@ -489,9 +489,18 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq,
     out.iov_len = iov_to_buf(elem->out_sg, elem->out_num, 0,
                              s->cvq_cmd_out_buffer,
                              vhost_vdpa_net_cvq_cmd_len());
-    dev_written = vhost_vdpa_net_cvq_add(s, out.iov_len, sizeof(status));
-    if (unlikely(dev_written < 0)) {
-        goto out;
+    if (*(uint8_t *)s->cvq_cmd_out_buffer == VIRTIO_NET_CTRL_ANNOUNCE) {
+        /*
+         * Guest announce capability is emulated by qemu, so dont forward to
+         * the device.
+         */
+        dev_written = sizeof(status);
+        *s->status = VIRTIO_NET_OK;
+    } else {
+        dev_written = vhost_vdpa_net_cvq_add(s, out.iov_len, sizeof(status));
+        if (unlikely(dev_written < 0)) {
+            goto out;
+        }
     }
 
     if (unlikely(dev_written < sizeof(status))) {
Signature · 1 line
2.31.1
#17 lore raw

PATCH v2 4/4 vdpa: do not handle VIRTIO_NET_F_GUEST_ANNOUNCE in vhost-vdpa

In reply to Eugenio Pérez

So qemu emulates it even in case the device does not support it.

---
 net/vhost-vdpa.c | 1 -
 1 file changed, 1 deletion(-)
net/vhost-vdpa.c +0 -1
index 8172aa8449..79f022c2bf 100644
@@ -72,7 +72,6 @@ const int vdpa_feature_bits[] = {
     VIRTIO_F_RING_RESET,
     VIRTIO_NET_F_RSS,
     VIRTIO_NET_F_HASH_REPORT,
-    VIRTIO_NET_F_GUEST_ANNOUNCE,
     VIRTIO_NET_F_STATUS,
     VHOST_INVALID_FEATURE_BIT
 };
Signature · 1 line
2.31.1
Acked-by on v2 1/4
#2 lore raw

Re: [PATCH v2 1/4] virtio_net: Modify virtio_net_get_config to early return

In reply to Eugenio Pérez
Quoted 7 lines · Eugenio Pérez

Next patches introduce more code on vhost-vdpa branch, with already have too much indentation.

Thanks

Quoted 47 lines
---
 hw/net/virtio-net.c | 28 +++++++++++++++-------------
 1 file changed, 15 insertions(+), 13 deletions(-)
hw/net/virtio-net.c +15 -14
index aba12759d5..eed629766f 100644
@@ -168,20 +168,22 @@ static void virtio_net_get_config(VirtIODevice *vdev, uint8_t *config)
     if (nc->peer && nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_VDPA) {
         ret = vhost_net_get_config(get_vhost_net(nc->peer), (uint8_t *)&netcfg,
                                    n->config_size);
-        if (ret != -1) {
-            /*
-             * Some NIC/kernel combinations present 0 as the mac address.  As
-             * that is not a legal address, try to proceed with the
-             * address from the QEMU command line in the hope that the
-             * address has been configured correctly elsewhere - just not
-             * reported by the device.
-             */
-            if (memcmp(&netcfg.mac, &zero, sizeof(zero)) == 0) {
-                info_report("Zero hardware mac address detected. Ignoring.");
-                memcpy(netcfg.mac, n->mac, ETH_ALEN);
-            }
-            memcpy(config, &netcfg, n->config_size);
+        if (ret == -1) {
+            return;
         }
+
+        /*
+         * Some NIC/kernel combinations present 0 as the mac address.  As that
+         * is not a legal address, try to proceed with the address from the
+         * QEMU command line in the hope that the address has been configured
+         * correctly elsewhere - just not reported by the device.
+         */
+        if (memcmp(&netcfg.mac, &zero, sizeof(zero)) == 0) {
+            info_report("Zero hardware mac address detected. Ignoring.");
+            memcpy(netcfg.mac, n->mac, ETH_ALEN);
+        }
+
+        memcpy(config, &netcfg, n->config_size);
     }
 }
 
--

2.31.1

#4 lore raw

Re: [PATCH v2 2/4] virtio_net: copy VIRTIO_NET_S_ANNOUNCE if device model has it

In reply to Eugenio Pérez
Quoted 14 lines · Eugenio Pérez

Status part of the emulated feature. It will follow device model, so we must copy it as long as NIC device model has it set.

---
 hw/net/virtio-net.c | 1 +
 1 file changed, 1 insertion(+)
hw/net/virtio-net.c +0 -0
index eed629766f..bf71ef33e8 100644

hw/net/virtio-net.c static void virtio_net_get_config(VirtIODevice *vdev, uint8_t *config) · line 186

Quoted 4 lines
hw/net/virtio-net.c +1 -0
@@ -183,6 +183,7 @@ static void virtio_net_get_config(VirtIODevice *vdev, uint8_t *config)
             memcpy(netcfg.mac, n->mac, ETH_ALEN);
         }
 
+        netcfg.status |= (n->status & VIRTIO_NET_S_ANNOUNCE);

Do we need to care about the endian here? We use:

    virtio_stw_p(vdev, &netcfg.status, n->status);

At the beginning of this function.

Thanks

Quoted 6 lines
hw/net/virtio-net.c +0 -1
         memcpy(config, &netcfg, n->config_size);
     }
 }
--

2.31.1

#8 lore raw

Re: [PATCH v2 3/4] vdpa: handle VIRTIO_NET_CTRL_ANNOUNCE in vhost_vdpa_net_handle_ctrl_avail

In reply to Eugenio Pérez
Quoted 14 lines · Eugenio Pérez

Since this capability is emulated by qemu shadowed CVQ cannot forward it to the device. Process all that command within qemu.

---
 net/vhost-vdpa.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
net/vhost-vdpa.c +0 -0
index 2b4b85d8f8..8172aa8449 100644

net/vhost-vdpa.c static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, · line 494

Quoted 9 lines
net/vhost-vdpa.c +3 -3
@@ -489,9 +489,18 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq,
     out.iov_len = iov_to_buf(elem->out_sg, elem->out_num, 0,
                              s->cvq_cmd_out_buffer,
                              vhost_vdpa_net_cvq_cmd_len());
-    dev_written = vhost_vdpa_net_cvq_add(s, out.iov_len, sizeof(status));
-    if (unlikely(dev_written < 0)) {
-        goto out;
+    if (*(uint8_t *)s->cvq_cmd_out_buffer == VIRTIO_NET_CTRL_ANNOUNCE) {
+        /*
+         * Guest announce capability is emulated by qemu, so dont forward to

s/dont/don't/

net/vhost-vdpa.c static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, · line 498

Quoted 4 lines
net/vhost-vdpa.c +4 -0
+         * the device.
+         */
+        dev_written = sizeof(status);
+        *s->status = VIRTIO_NET_OK;

I wonder if we should avoid negotiating ANNOUNCE with vDPA parents if we do this?

Thanks

Quoted 11 lines
net/vhost-vdpa.c +5 -1
+    } else {
+        dev_written = vhost_vdpa_net_cvq_add(s, out.iov_len, sizeof(status));
+        if (unlikely(dev_written < 0)) {
+            goto out;
+        }
     }
 
     if (unlikely(dev_written < sizeof(status))) {
--

2.31.1

Acked-by on v2 4/4
#18 lore raw

Re: [PATCH v2 4/4] vdpa: do not handle VIRTIO_NET_F_GUEST_ANNOUNCE in vhost-vdpa

In reply to Eugenio Pérez
Quoted 4 lines · Eugenio Pérez

So qemu emulates it even in case the device does not support it.

Thanks

Quoted 19 lines
---
 net/vhost-vdpa.c | 1 -
 1 file changed, 1 deletion(-)
net/vhost-vdpa.c +0 -2
index 8172aa8449..79f022c2bf 100644
@@ -72,7 +72,6 @@ const int vdpa_feature_bits[] = {
     VIRTIO_F_RING_RESET,
     VIRTIO_NET_F_RSS,
     VIRTIO_NET_F_HASH_REPORT,
-    VIRTIO_NET_F_GUEST_ANNOUNCE,
     VIRTIO_NET_F_STATUS,
     VHOST_INVALID_FEATURE_BIT
 };
--

2.31.1

#9 lore raw

Re: [PATCH v2 3/4] vdpa: handle VIRTIO_NET_CTRL_ANNOUNCE in vhost_vdpa_net_handle_ctrl_avail

In reply to Jason Wang
Quoted 11 lines · Jason Wang
Quoted 9 lines · Eugenio Pérez

Since this capability is emulated by qemu shadowed CVQ cannot forward it to the device. Process all that command within qemu.

---
 net/vhost-vdpa.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
Quoted 17 lines
Quoted 5 lines
net/vhost-vdpa.c +0 -0
index 2b4b85d8f8..8172aa8449 100644

net/vhost-vdpa.c static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, · line 494

Quoted 9 lines
net/vhost-vdpa.c +3 -3
@@ -489,9 +489,18 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq,
     out.iov_len = iov_to_buf(elem->out_sg, elem->out_num, 0,
                              s->cvq_cmd_out_buffer,
                              vhost_vdpa_net_cvq_cmd_len());
-    dev_written = vhost_vdpa_net_cvq_add(s, out.iov_len, sizeof(status));
-    if (unlikely(dev_written < 0)) {
-        goto out;
+    if (*(uint8_t *)s->cvq_cmd_out_buffer == VIRTIO_NET_CTRL_ANNOUNCE) {
+        /*
+         * Guest announce capability is emulated by qemu, so dont forward to

s/dont/don't/

I'll correct it, thanks!

Quoted 8 lines

net/vhost-vdpa.c static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, · line 498

Quoted 4 lines
net/vhost-vdpa.c +4 -0
+         * the device.
+         */
+        dev_written = sizeof(status);
+        *s->status = VIRTIO_NET_OK;

I wonder if we should avoid negotiating ANNOUNCE with vDPA parents if we do this?

I can re-check, but the next patch should avoid it. Even if negotiated, the parent should never set the announce status bit, since we never tell the device is a destination device.

But it's better to be on the safe side, I'll recheck.

Thanks!

Quoted 14 lines

Thanks

Quoted 11 lines
net/vhost-vdpa.c +5 -1
+    } else {
+        dev_written = vhost_vdpa_net_cvq_add(s, out.iov_len, sizeof(status));
+        if (unlikely(dev_written < 0)) {
+            goto out;
+        }
     }
 
     if (unlikely(dev_written < sizeof(status))) {
--

2.31.1

#15 lore raw

Re: [PATCH v2 3/4] vdpa: handle VIRTIO_NET_CTRL_ANNOUNCE in vhost_vdpa_net_handle_ctrl_avail

In reply to Eugenio Pérez
Quoted 14 lines · Eugenio Pérez

Since this capability is emulated by qemu shadowed CVQ cannot forward it to the device. Process all that command within qemu.

---
 net/vhost-vdpa.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
net/vhost-vdpa.c +0 -0
index 2b4b85d8f8..8172aa8449 100644
@@ -489,9 +489,18 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq,
     out.iov_len = iov_to_buf(elem->out_sg, elem->out_num, 0,

net/vhost-vdpa.c static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, · line 496

Quoted 10 lines
net/vhost-vdpa.c +5 -3
                              s->cvq_cmd_out_buffer,
                              vhost_vdpa_net_cvq_cmd_len());
-    dev_written = vhost_vdpa_net_cvq_add(s, out.iov_len, sizeof(status));
-    if (unlikely(dev_written < 0)) {
-        goto out;
+    if (*(uint8_t *)s->cvq_cmd_out_buffer == VIRTIO_NET_CTRL_ANNOUNCE) {
+        /*
+         * Guest announce capability is emulated by qemu, so dont forward to
+         * the device.
+         */

Hmm I'm not sure why. We don't forward the status bit to guest?

Quoted 12 lines
net/vhost-vdpa.c +7 -0
+        dev_written = sizeof(status);
+        *s->status = VIRTIO_NET_OK;
+    } else {
+        dev_written = vhost_vdpa_net_cvq_add(s, out.iov_len, sizeof(status));
+        if (unlikely(dev_written < 0)) {
+            goto out;
+        }
     }
 
     if (unlikely(dev_written < sizeof(status))) {
Signature · 1 line
2.31.1
#16 lore raw

Re: [PATCH v2 3/4] vdpa: handle VIRTIO_NET_CTRL_ANNOUNCE in vhost_vdpa_net_handle_ctrl_avail

Quoted 20 lines · Michael S. Tsirkin
Quoted 18 lines · Eugenio Pérez

Since this capability is emulated by qemu shadowed CVQ cannot forward it to the device. Process all that command within qemu.

---
 net/vhost-vdpa.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
net/vhost-vdpa.c +0 -2
index 2b4b85d8f8..8172aa8449 100644
@@ -489,9 +489,18 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq,
     out.iov_len = iov_to_buf(elem->out_sg, elem->out_num, 0,
                              s->cvq_cmd_out_buffer,
                              vhost_vdpa_net_cvq_cmd_len());
-    dev_written = vhost_vdpa_net_cvq_add(s, out.iov_len, sizeof(status));
-    if (unlikely(dev_written < 0)) {
Quoted 9 lines

net/vhost-vdpa.c static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, · line 496

Quoted 6 lines
net/vhost-vdpa.c +5 -1
-        goto out;
+    if (*(uint8_t *)s->cvq_cmd_out_buffer == VIRTIO_NET_CTRL_ANNOUNCE) {
+        /*
+         * Guest announce capability is emulated by qemu, so dont forward to
+         * the device.
+         */

Hmm I'm not sure why. We don't forward the status bit to guest?

No, the idea is to make this feature entirely emulated by qemu so it does not depend on device's features to support it.

Thanks!

Quoted 13 lines
Quoted 12 lines
net/vhost-vdpa.c +7 -1
+        dev_written = sizeof(status);
+        *s->status = VIRTIO_NET_OK;
+    } else {
+        dev_written = vhost_vdpa_net_cvq_add(s, out.iov_len, sizeof(status));
+        if (unlikely(dev_written < 0)) {
+            goto out;
+        }
     }
 
     if (unlikely(dev_written < sizeof(status))) {
--

2.31.1

#10 lore raw

Re: [PATCH v2 3/4] vdpa: handle VIRTIO_NET_CTRL_ANNOUNCE in vhost_vdpa_net_handle_ctrl_avail

In reply to Eugenio Perez Martin
Quoted 38 lines · Eugenio Perez Martin
Quoted 11 lines · Jason Wang
Quoted 9 lines · Eugenio Pérez

Since this capability is emulated by qemu shadowed CVQ cannot forward it to the device. Process all that command within qemu.

---
 net/vhost-vdpa.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
Quoted 17 lines
Quoted 5 lines
net/vhost-vdpa.c +0 -0
index 2b4b85d8f8..8172aa8449 100644

net/vhost-vdpa.c static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, · line 494

Quoted 9 lines
net/vhost-vdpa.c +3 -3
@@ -489,9 +489,18 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq,
     out.iov_len = iov_to_buf(elem->out_sg, elem->out_num, 0,
                              s->cvq_cmd_out_buffer,
                              vhost_vdpa_net_cvq_cmd_len());
-    dev_written = vhost_vdpa_net_cvq_add(s, out.iov_len, sizeof(status));
-    if (unlikely(dev_written < 0)) {
-        goto out;
+    if (*(uint8_t *)s->cvq_cmd_out_buffer == VIRTIO_NET_CTRL_ANNOUNCE) {
+        /*
+         * Guest announce capability is emulated by qemu, so dont forward to

s/dont/don't/

I'll correct it, thanks!

Quoted 5 lines
Quoted 4 lines
net/vhost-vdpa.c +4 -0
+         * the device.
+         */
+        dev_written = sizeof(status);
+        *s->status = VIRTIO_NET_OK;
Quoted 5 lines
Quoted 3 lines

I wonder if we should avoid negotiating ANNOUNCE with vDPA parents if we do this?

I can re-check, but the next patch should avoid it.

Kind of, it makes sure guest can always see _F_ANNOUNCE. But does it prevent _F_ANNOUNCE from being negotiated?

Quoted 3 lines

Even if
negotiated, the parent should never set the announce status bit, since we never tell the device is a destination device.

That's the point, do we have such a guarantee? Or I wonder if there's any parent that supports _F_ANNOUNCE if yes, how it is supposed to work?

Quoted 2 lines

But it's better to be on the safe side, I'll recheck.

Exactly.

Thanks

Quoted 18 lines

Thanks!

Quoted 14 lines

Thanks

Quoted 11 lines
net/vhost-vdpa.c +5 -1
+    } else {
+        dev_written = vhost_vdpa_net_cvq_add(s, out.iov_len, sizeof(status));
+        if (unlikely(dev_written < 0)) {
+            goto out;
+        }
     }
 
     if (unlikely(dev_written < sizeof(status))) {
--

2.31.1

#11 lore raw

Re: [PATCH v2 3/4] vdpa: handle VIRTIO_NET_CTRL_ANNOUNCE in vhost_vdpa_net_handle_ctrl_avail

Quoted 41 lines · Jason Wang
Quoted 38 lines · Eugenio Perez Martin
Quoted 11 lines · Jason Wang
Quoted 9 lines · Eugenio Pérez

Since this capability is emulated by qemu shadowed CVQ cannot forward it to the device. Process all that command within qemu.

---
 net/vhost-vdpa.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
Quoted 17 lines
Quoted 5 lines
net/vhost-vdpa.c +0 -0
index 2b4b85d8f8..8172aa8449 100644

net/vhost-vdpa.c static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, · line 494

Quoted 9 lines
net/vhost-vdpa.c +3 -3
@@ -489,9 +489,18 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq,
     out.iov_len = iov_to_buf(elem->out_sg, elem->out_num, 0,
                              s->cvq_cmd_out_buffer,
                              vhost_vdpa_net_cvq_cmd_len());
-    dev_written = vhost_vdpa_net_cvq_add(s, out.iov_len, sizeof(status));
-    if (unlikely(dev_written < 0)) {
-        goto out;
+    if (*(uint8_t *)s->cvq_cmd_out_buffer == VIRTIO_NET_CTRL_ANNOUNCE) {
+        /*
+         * Guest announce capability is emulated by qemu, so dont forward to

s/dont/don't/

I'll correct it, thanks!

Quoted 5 lines
Quoted 4 lines
net/vhost-vdpa.c +4 -0
+         * the device.
+         */
+        dev_written = sizeof(status);
+        *s->status = VIRTIO_NET_OK;
Quoted 9 lines
Quoted 5 lines
Quoted 3 lines

I wonder if we should avoid negotiating ANNOUNCE with vDPA parents if we do this?

I can re-check, but the next patch should avoid it.

Kind of, it makes sure guest can always see _F_ANNOUNCE. But does it prevent _F_ANNOUNCE from being negotiated?

It should go like:
* vhost_net_ack_features calls vhost_ack_features with feature_bits = vdpa_feature_bits and features = guest acked features. vhost_ack_features stores in hdev->acked_features only the features that met features & bit_mask, so it will not store _F_ANNOUNCE.
* vhost_vdpa_set_features is called from vhost_dev_set_features with features = dev->acked_features. Both functions can add features by themselves (VHOST_F_LOG_ALL, VIRTIO_F_IOMMU_PLATFORM), but no _F_ANNOUNCE.

Still untested.

Quoted 8 lines
Quoted 3 lines

Even if
negotiated, the parent should never set the announce status bit, since we never tell the device is a destination device.

That's the point, do we have such a guarantee? Or I wonder if there's any parent that supports _F_ANNOUNCE if yes, how it is supposed to work?

At the moment it is impossible to work since there is no support for config interrupt from the device. Even with config interrupt, something external from qemu should make the device enable the status bit, since the current migration protocol makes no difference between to be a migration destination and to start the device from scratch. Unless it enables the bit maliciously or by mistake.

Just for completion, the current method works with no need of vdpa device config interrupt support thanks to being 100% emulated in qemu, which has the support of injecting config interrupts.

Thanks!

#12 lore raw

Re: [PATCH v2 3/4] vdpa: handle VIRTIO_NET_CTRL_ANNOUNCE in vhost_vdpa_net_handle_ctrl_avail

Quoted 63 lines · Eugenio Perez Martin
Quoted 41 lines · Jason Wang
Quoted 38 lines · Eugenio Perez Martin
Quoted 11 lines · Jason Wang
Quoted 9 lines · Eugenio Pérez

Since this capability is emulated by qemu shadowed CVQ cannot forward it to the device. Process all that command within qemu.

---
 net/vhost-vdpa.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
Quoted 17 lines
Quoted 5 lines
net/vhost-vdpa.c +0 -0
index 2b4b85d8f8..8172aa8449 100644

net/vhost-vdpa.c static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, · line 494

Quoted 9 lines
net/vhost-vdpa.c +3 -3
@@ -489,9 +489,18 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq,
     out.iov_len = iov_to_buf(elem->out_sg, elem->out_num, 0,
                              s->cvq_cmd_out_buffer,
                              vhost_vdpa_net_cvq_cmd_len());
-    dev_written = vhost_vdpa_net_cvq_add(s, out.iov_len, sizeof(status));
-    if (unlikely(dev_written < 0)) {
-        goto out;
+    if (*(uint8_t *)s->cvq_cmd_out_buffer == VIRTIO_NET_CTRL_ANNOUNCE) {
+        /*
+         * Guest announce capability is emulated by qemu, so dont forward to

s/dont/don't/

I'll correct it, thanks!

Quoted 5 lines
Quoted 4 lines
net/vhost-vdpa.c +4 -0
+         * the device.
+         */
+        dev_written = sizeof(status);
+        *s->status = VIRTIO_NET_OK;
Quoted 9 lines
Quoted 5 lines
Quoted 3 lines

I wonder if we should avoid negotiating ANNOUNCE with vDPA parents if we do this?

I can re-check, but the next patch should avoid it.

Kind of, it makes sure guest can always see _F_ANNOUNCE. But does it prevent _F_ANNOUNCE from being negotiated?

It should go like:
* vhost_net_ack_features calls vhost_ack_features with feature_bits = vdpa_feature_bits and features = guest acked features. vhost_ack_features stores in hdev->acked_features only the features that met features & bit_mask, so it will not store _F_ANNOUNCE.
* vhost_vdpa_set_features is called from vhost_dev_set_features with features = dev->acked_features. Both functions can add features by themselves (VHOST_F_LOG_ALL, VIRTIO_F_IOMMU_PLATFORM), but no _F_ANNOUNCE.

Quoted 1 line

Still untested.

Ok.

Quoted 17 lines
Quoted 9 lines
Quoted 4 lines

Even if
negotiated, the parent should never set the announce status bit, since we never tell the device is a destination device.

That's the point, do we have such a guarantee? Or I wonder if there's any parent that supports _F_ANNOUNCE if yes, how it is supposed to work?

At the moment it is impossible to work since there is no support for config interrupt from the device. Even with config interrupt, something external from qemu should make the device enable the status bit, since the current migration protocol makes no difference between to be a migration destination and to start the device from scratch. Unless it enables the bit maliciously or by mistake.

Quoted 3 lines

Just for completion, the current method works with no need of vdpa device config interrupt support thanks to being 100% emulated in qemu, which has the support of injecting config interrupts.

Ok, rethink this feature, I think I can find one use case for _F_ANNOUNCE, that is, the migration is totally done through the vDPA device (DPU) itself.

I think we can go forward and revisit this issue in the future.

Thanks

Quoted 3 lines

Thanks!

#13 lore raw

Re: [PATCH v2 3/4] vdpa: handle VIRTIO_NET_CTRL_ANNOUNCE in vhost_vdpa_net_handle_ctrl_avail

Quoted 86 lines · Jason Wang
Quoted 63 lines · Eugenio Perez Martin
Quoted 41 lines · Jason Wang
Quoted 38 lines · Eugenio Perez Martin
Quoted 11 lines · Jason Wang
Quoted 9 lines · Eugenio Pérez

Since this capability is emulated by qemu shadowed CVQ cannot forward it to the device. Process all that command within qemu.

---
 net/vhost-vdpa.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
Quoted 17 lines
Quoted 5 lines
net/vhost-vdpa.c +0 -0
index 2b4b85d8f8..8172aa8449 100644

net/vhost-vdpa.c static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, · line 494

Quoted 9 lines
net/vhost-vdpa.c +3 -3
@@ -489,9 +489,18 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq,
     out.iov_len = iov_to_buf(elem->out_sg, elem->out_num, 0,
                              s->cvq_cmd_out_buffer,
                              vhost_vdpa_net_cvq_cmd_len());
-    dev_written = vhost_vdpa_net_cvq_add(s, out.iov_len, sizeof(status));
-    if (unlikely(dev_written < 0)) {
-        goto out;
+    if (*(uint8_t *)s->cvq_cmd_out_buffer == VIRTIO_NET_CTRL_ANNOUNCE) {
+        /*
+         * Guest announce capability is emulated by qemu, so dont forward to

s/dont/don't/

I'll correct it, thanks!

Quoted 5 lines
Quoted 4 lines
net/vhost-vdpa.c +4 -0
+         * the device.
+         */
+        dev_written = sizeof(status);
+        *s->status = VIRTIO_NET_OK;
Quoted 9 lines
Quoted 5 lines
Quoted 3 lines

I wonder if we should avoid negotiating ANNOUNCE with vDPA parents if we do this?

I can re-check, but the next patch should avoid it.

Kind of, it makes sure guest can always see _F_ANNOUNCE. But does it prevent _F_ANNOUNCE from being negotiated?

It should go like:
* vhost_net_ack_features calls vhost_ack_features with feature_bits = vdpa_feature_bits and features = guest acked features. vhost_ack_features stores in hdev->acked_features only the features that met features & bit_mask, so it will not store _F_ANNOUNCE.
* vhost_vdpa_set_features is called from vhost_dev_set_features with features = dev->acked_features. Both functions can add features by themselves (VHOST_F_LOG_ALL, VIRTIO_F_IOMMU_PLATFORM), but no _F_ANNOUNCE.

Quoted 1 line

Still untested.

Ok.

Quoted 17 lines
Quoted 9 lines
Quoted 4 lines

Even if
negotiated, the parent should never set the announce status bit, since we never tell the device is a destination device.

That's the point, do we have such a guarantee? Or I wonder if there's any parent that supports _F_ANNOUNCE if yes, how it is supposed to work?

At the moment it is impossible to work since there is no support for config interrupt from the device. Even with config interrupt, something external from qemu should make the device enable the status bit, since the current migration protocol makes no difference between to be a migration destination and to start the device from scratch. Unless it enables the bit maliciously or by mistake.

Quoted 8 lines
Quoted 3 lines

Just for completion, the current method works with no need of vdpa device config interrupt support thanks to being 100% emulated in qemu, which has the support of injecting config interrupts.

Ok, rethink this feature, I think I can find one use case for _F_ANNOUNCE, that is, the migration is totally done through the vDPA device (DPU) itself.

To make sure we are on the same page, this migration would save some things like transfer the status through qemu, but it is not possible at the moment. A few things need to be developed for that to make it possible.

The default behavior is to emulate the announce feature / status bit at the moment, so no ack to the device is needed. If we want that passthrough, a new parameter or similar needs to be developed, so the feature is negotiated with the device and not emulated in get_config.

Is that accurate?

Thanks!

Quoted 8 lines

I think we can go forward and revisit this issue in the future.

Thanks

Quoted 3 lines

Thanks!

#14 lore raw

Re: [PATCH v2 3/4] vdpa: handle VIRTIO_NET_CTRL_ANNOUNCE in vhost_vdpa_net_handle_ctrl_avail

Quoted 97 lines · Eugenio Perez Martin
Quoted 86 lines · Jason Wang
Quoted 63 lines · Eugenio Perez Martin
Quoted 41 lines · Jason Wang
Quoted 38 lines · Eugenio Perez Martin
Quoted 11 lines · Jason Wang
Quoted 9 lines · Eugenio Pérez

Since this capability is emulated by qemu shadowed CVQ cannot forward it to the device. Process all that command within qemu.

---
 net/vhost-vdpa.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
Quoted 17 lines
Quoted 5 lines
net/vhost-vdpa.c +0 -0
index 2b4b85d8f8..8172aa8449 100644

net/vhost-vdpa.c static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, · line 494

Quoted 9 lines
net/vhost-vdpa.c +3 -3
@@ -489,9 +489,18 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq,
     out.iov_len = iov_to_buf(elem->out_sg, elem->out_num, 0,
                              s->cvq_cmd_out_buffer,
                              vhost_vdpa_net_cvq_cmd_len());
-    dev_written = vhost_vdpa_net_cvq_add(s, out.iov_len, sizeof(status));
-    if (unlikely(dev_written < 0)) {
-        goto out;
+    if (*(uint8_t *)s->cvq_cmd_out_buffer == VIRTIO_NET_CTRL_ANNOUNCE) {
+        /*
+         * Guest announce capability is emulated by qemu, so dont forward to

s/dont/don't/

I'll correct it, thanks!

Quoted 5 lines
Quoted 4 lines
net/vhost-vdpa.c +4 -0
+         * the device.
+         */
+        dev_written = sizeof(status);
+        *s->status = VIRTIO_NET_OK;
Quoted 9 lines
Quoted 5 lines
Quoted 3 lines

I wonder if we should avoid negotiating ANNOUNCE with vDPA parents if we do this?

I can re-check, but the next patch should avoid it.

Kind of, it makes sure guest can always see _F_ANNOUNCE. But does it prevent _F_ANNOUNCE from being negotiated?

It should go like:
* vhost_net_ack_features calls vhost_ack_features with feature_bits = vdpa_feature_bits and features = guest acked features. vhost_ack_features stores in hdev->acked_features only the features that met features & bit_mask, so it will not store _F_ANNOUNCE.
* vhost_vdpa_set_features is called from vhost_dev_set_features with features = dev->acked_features. Both functions can add features by themselves (VHOST_F_LOG_ALL, VIRTIO_F_IOMMU_PLATFORM), but no _F_ANNOUNCE.

Quoted 1 line

Still untested.

Ok.

Quoted 17 lines
Quoted 9 lines
Quoted 4 lines

Even if
negotiated, the parent should never set the announce status bit, since we never tell the device is a destination device.

That's the point, do we have such a guarantee? Or I wonder if there's any parent that supports _F_ANNOUNCE if yes, how it is supposed to work?

At the moment it is impossible to work since there is no support for config interrupt from the device. Even with config interrupt, something external from qemu should make the device enable the status bit, since the current migration protocol makes no difference between to be a migration destination and to start the device from scratch. Unless it enables the bit maliciously or by mistake.

Quoted 8 lines
Quoted 3 lines

Just for completion, the current method works with no need of vdpa device config interrupt support thanks to being 100% emulated in qemu, which has the support of injecting config interrupts.

Ok, rethink this feature, I think I can find one use case for _F_ANNOUNCE, that is, the migration is totally done through the vDPA device (DPU) itself.

Quoted 4 lines

To make sure we are on the same page, this migration would save some things like transfer the status through qemu, but it is not possible at the moment. A few things need to be developed for that to make it possible.

Somehow, it means the DPU is in charge of doing all the migration.

Quoted 7 lines

The default behavior is to emulate the announce feature / status bit at the moment, so no ack to the device is needed. If we want that passthrough, a new parameter or similar needs to be developed, so the feature is negotiated with the device and not emulated in get_config.

Is that accurate?

Yes.

Thanks

Quoted 12 lines

Thanks!

Quoted 8 lines

I think we can go forward and revisit this issue in the future.

Thanks

Quoted 3 lines

Thanks!

#5 lore raw

Re: [PATCH v2 2/4] virtio_net: copy VIRTIO_NET_S_ANNOUNCE if device model has it

In reply to Jason Wang
Quoted 22 lines · Jason Wang
Quoted 14 lines · Eugenio Pérez

Status part of the emulated feature. It will follow device model, so we must copy it as long as NIC device model has it set.

---
 hw/net/virtio-net.c | 1 +
 1 file changed, 1 insertion(+)
hw/net/virtio-net.c +0 -0
index eed629766f..bf71ef33e8 100644

hw/net/virtio-net.c static void virtio_net_get_config(VirtIODevice *vdev, uint8_t *config) · line 186

Quoted 4 lines
hw/net/virtio-net.c +1 -0
@@ -183,6 +183,7 @@ static void virtio_net_get_config(VirtIODevice *vdev, uint8_t *config)
             memcpy(netcfg.mac, n->mac, ETH_ALEN);
         }
 
+        netcfg.status |= (n->status & VIRTIO_NET_S_ANNOUNCE);

Do we need to care about the endian here? We use:

Quoted 5 lines
    virtio_stw_p(vdev, &netcfg.status, n->status);

At the beginning of this function.

Thanks

Didn't see this answered.

Quoted 6 lines
Quoted 6 lines
hw/net/virtio-net.c +0 -1
         memcpy(config, &netcfg, n->config_size);
     }
 }
--

2.31.1

#6 lore raw

Re: [PATCH v2 2/4] virtio_net: copy VIRTIO_NET_S_ANNOUNCE if device model has it

In reply to Jason Wang
Quoted 19 lines · Jason Wang
Quoted 17 lines · Eugenio Pérez

Status part of the emulated feature. It will follow device model, so we must copy it as long as NIC device model has it set.

---
 hw/net/virtio-net.c | 1 +
 1 file changed, 1 insertion(+)
hw/net/virtio-net.c +0 -0
index eed629766f..bf71ef33e8 100644
@@ -183,6 +183,7 @@ static void virtio_net_get_config(VirtIODevice *vdev, uint8_t *config)
             memcpy(netcfg.mac, n->mac, ETH_ALEN);
         }
 
Quoted 8 lines

hw/net/virtio-net.c static void virtio_net_get_config(VirtIODevice *vdev, uint8_t *config) · line 186

Quoted 1 line
hw/net/virtio-net.c +1 -0
+        netcfg.status |= (n->status & VIRTIO_NET_S_ANNOUNCE);

Do we need to care about the endian here? We use:

    virtio_stw_p(vdev, &netcfg.status, n->status);

At the beginning of this function.

Right, this is a miss. I'll fix it in the next version.

Thanks!