Skip to contentSkip to conversation

Vhost-vdpa Shadow Virtqueue Hash calculation Support

RFC PATCH New Hawkins Jiawei 1 participant 3 messages Aug 11, 2023 Networking

lore
Share & tools
lore
#0 lore raw

cover Vhost-vdpa Shadow Virtqueue Hash calculation Support

This series enables shadowed CVQ to intercept VIRTIO_NET_CTRL_MQ_HASH_CONFIG command through shadowed CVQ, update the virtio NIC device model so qemu send it in a migration, and the restore of that Hash calculation state in the destination.

Note that this patch should be based on
patch "vdpa: Send all CVQ state load commands in parallel" at [1].

[1]. https://lore.kernel.org/all/[email protected]/

TestStep
========
1. test the migration using vp-vdpa device
- For L0 guest, boot QEMU with two virtio-net-pci net device with
ctrl_vq, mq, hash features on, command line like:

	-netdev tap,...
	-device virtio-net-pci,disable-legacy=on,disable-modern=off,

iommu_platform=on,mq=on,ctrl_vq=on,hash=on,guest_announce=off, indirect_desc=off,queue_reset=off,...

- For L1 guest, apply the relative patch series and compile the
source code, start QEMU with two vdpa device with svq mode on, enable the ctrl_vq, mq, hash features on, command line like:

      -netdev type=vhost-vdpa,x-svq=true,...
      -device virtio-net-pci,mq=on,guest_announce=off,ctrl_vq=on,

hash=on,...

- For L2 source guest, run the following bash command:

```bash
#!/bin/sh

ethtool -K eth0 rxhash on
```

- Gdb attach the destination VM and break at the
vhost_vdpa_net_load_rss()

- Execute the live migration in L2 source monitor

- Result
* with this series, gdb can hit the breakpoint and continue
the executing without triggering any error or warning.

Hawkins Jiawei (2):
vdpa: Restore hash calculation state
vdpa: Allow VIRTIO_NET_F_HASH_REPORT in SVQ

 net/vhost-vdpa.c | 89 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 89 insertions(+)
Signature · 1 line
2.25.1
#1 lore raw

RFC PATCH 1/2 vdpa: Restore hash calculation state

This patch introduces vhost_vdpa_net_load_rss() to restore the hash calculation state at device's startup.

Note that vhost_vdpa_net_load_rss() has do_rss argument, which allows future code to reuse this function to restore the receive-side scaling state when the VIRTIO_NET_F_RSS feature is enabled in SVQ. Currently, vhost_vdpa_net_load_rss() could only be invoked when do_rss is set to false.

---
Question:

It seems that virtio_net_handle_rss() currently does not restore the hash key length parsed from the CVQ command sent from the guest into n->rss_data and uses the maximum key length in other code.

So for hash_key_length field in VIRTIO_NET_CTRL_MQ_HASH_CONFIG command sent to device, is it okay to also use the maximum key length as its value? Or should we introduce the hash_key_length field in n->rss_data structure to record the key length from guest and use this value?

 net/vhost-vdpa.c | 88 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 88 insertions(+)
net/vhost-vdpa.c +88 -0
index 7bb29f6009..bd51020771 100644
@@ -788,6 +788,85 @@ static int vhost_vdpa_net_load_mac(VhostVDPAState *s, const VirtIONet *n,
     return 0;
 }
 
+static int vhost_vdpa_net_load_rss(VhostVDPAState *s, const VirtIONet *n,
+                                   void **out_cursor, void **in_cursor,
+                                   bool do_rss)
+{
+    struct virtio_net_rss_config cfg;
+    ssize_t r;
+
+    /*
+     * According to VirtIO standard, "Initially the device has all hash
+     * types disabled and reports only VIRTIO_NET_HASH_REPORT_NONE.".
+     *
+     * Therefore, there is no need to send this CVQ command if the
+     * driver disable the all hash types, which aligns with
+     * the device's defaults.
+     *
+     * Note that the device's defaults can mismatch the driver's
+     * configuration only at live migration.
+     */
+    if (!n->rss_data.enabled ||
+        n->rss_data.hash_types == VIRTIO_NET_HASH_REPORT_NONE) {
+        return 0;
+    }
+
+    cfg.hash_types = cpu_to_le32(n->rss_data.hash_types);
+    /*
+     * According to VirtIO standard, "Field reserved MUST contain zeroes.
+     * It is defined to make the structure to match the layout of
+     * virtio_net_rss_config structure, defined in 5.1.6.5.7.".
+     *
+     * Therefore, we need to zero the fields in struct virtio_net_rss_config,
+     * which corresponds the `reserved` field in
+     * struct virtio_net_hash_config.
+     */
+    memset(&cfg.indirection_table_mask, 0,
+           sizeof_field(struct virtio_net_hash_config, reserved));
+    /*
+     * Consider that virtio_net_handle_rss() currently does not restore the
+     * hash key length parsed from the CVQ command sent from the guest into
+     * n->rss_data and uses the maximum key length in other code, so we also
+     * employthe the maxium key length here.
+     */
+    cfg.hash_key_length = sizeof(n->rss_data.key);
+
+    g_autofree uint16_t *table = g_malloc_n(n->rss_data.indirections_len,
+                                    sizeof(n->rss_data.indirections_table[0]));
+    for (int i = 0; i < n->rss_data.indirections_len; ++i) {
+        table[i] = cpu_to_le16(n->rss_data.indirections_table[i]);
+    }
+
+    const struct iovec data[] = {
+        {
+            .iov_base = &cfg,
+            .iov_len = offsetof(struct virtio_net_rss_config,
+                                indirection_table),
+        }, {
+            .iov_base = table,
+            .iov_len = n->rss_data.indirections_len *
+                       sizeof(n->rss_data.indirections_table[0]),
+        }, {
+            .iov_base = &cfg.max_tx_vq,
+            .iov_len = offsetof(struct virtio_net_rss_config, hash_key_data) -
+                       offsetof(struct virtio_net_rss_config, max_tx_vq),
+        }, {
+            .iov_base = (void *)n->rss_data.key,
+            .iov_len = sizeof(n->rss_data.key),
+        }
+    };
+
+    r = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor,
+                                VIRTIO_NET_CTRL_MQ,
+                                VIRTIO_NET_CTRL_MQ_HASH_CONFIG,
+                                data, ARRAY_SIZE(data));
+    if (unlikely(r < 0)) {
+        return r;
+    }
+
+    return 0;
+}
+
 static int vhost_vdpa_net_load_mq(VhostVDPAState *s,
                                   const VirtIONet *n,
                                   void **out_cursor, void **in_cursor)
@@ -812,6 +891,15 @@ static int vhost_vdpa_net_load_mq(VhostVDPAState *s,
         return r;
     }
 
+    if (!virtio_vdev_has_feature(&n->parent_obj, VIRTIO_NET_F_HASH_REPORT)) {
+        return 0;
+    }
+
+    r = vhost_vdpa_net_load_rss(s, n, out_cursor, in_cursor, false);
+    if (unlikely(r < 0)) {
+        return r;
+    }
+
     return 0;
 }
 
Signature · 1 line
2.25.1
#2 lore raw

RFC PATCH 2/2 vdpa: Allow VIRTIO_NET_F_HASH_REPORT in SVQ

In reply to Hawkins Jiawei

Enable SVQ with VIRTIO_NET_F_HASH_REPORT feature.

---
 net/vhost-vdpa.c | 1 +
 1 file changed, 1 insertion(+)
net/vhost-vdpa.c +1 -0
index bd51020771..a13b267250 100644
@@ -118,6 +118,7 @@ static const uint64_t vdpa_svq_device_features =
     BIT_ULL(VIRTIO_NET_F_CTRL_MAC_ADDR) |
     /* VHOST_F_LOG_ALL is exposed by SVQ */
     BIT_ULL(VHOST_F_LOG_ALL) |
+    BIT_ULL(VIRTIO_NET_F_HASH_REPORT) |
     BIT_ULL(VIRTIO_NET_F_RSC_EXT) |
     BIT_ULL(VIRTIO_NET_F_STANDBY) |
     BIT_ULL(VIRTIO_NET_F_SPEED_DUPLEX);
Signature · 1 line
2.25.1