Vhost-vdpa Shadow Virtqueue VLAN support
cover Vhost-vdpa Shadow Virtqueue VLAN support
This series enables shadowed CVQ to intercept VLAN commands through shadowed CVQ, update the virtio NIC device model so qemu send it in a migration, and the restore of that VLAN state in the destination.
TestStep
========
1. test the migration using vp-vdpa device
- For L0 guest, boot QEMU with two virtio-net-pci net device withctrl_vq, ctrl_vlan features on, command line like:
-device virtio-net-pci,disable-legacy=on,disable-modern=off,
iommu_platform=on,mq=on,ctrl_vq=on,guest_announce=off, indirect_desc=off,queue_reset=off,ctrl_vlan=on,...
- For L1 guest, apply the patch series and compile the source code,
start QEMU with two vdpa device with svq mode on, enable the ctrl_vq, ctrl_vlan features on, command line like:
-netdev type=vhost-vdpa,x-svq=true,...
-device virtio-net-pci,mq=on,guest_announce=off,ctrl_vq=on,ctrl_vlan=on,...
- For L2 source guest, run the following bash command:
```bash
#!/bin/sh
for idx in {1..4094}
do
ip link add link eth0 name vlan$idx type vlan id $idx
done
``` - gdb attaches the L2 dest VM and break at thevhost_vdpa_net_load_single_vlan(), and execute the following gdbscript
```gdbscript ignore 1 4094 c ```
- Execute the live migration in L2 source monitor
- Result
* with this series, gdb can hit the breakpoint and continue
the executing without triggering any error or warning.
Eugenio Pérez (1):
virtio-net: do not reset vlan filtering at set_features
Hawkins Jiawei (3):
virtio-net: Expose MAX_VLAN
vdpa: Restore vlan filtering state
vdpa: Allow VIRTIO_NET_F_CTRL_VLAN in SVQ
hw/net/virtio-net.c | 6 +--- include/hw/virtio/virtio-net.h | 6 ++++ net/vhost-vdpa.c | 50 ++++++++++++++++++++++++++++++++++ 3 files changed, 57 insertions(+), 5 deletions(-)
Signature · 1 line
2.25.1
PATCH 1/4 virtio-net: do not reset vlan filtering at set_features
From: Eugenio Pérez <[email protected]>
This function is called after virtio_load, so all vlan configuration is lost in migration case.
Just allow all the vlan-tagged packets if vlan is not configured, and trust device reset to clear all filtered vlans.
--- hw/net/virtio-net.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
hw/net/virtio-net.c +1 -3
@@ -1006,9 +1006,7 @@ static void virtio_net_set_features(VirtIODevice *vdev, uint64_t features)vhost_net_save_acked_features (nc->peer); } -if (virtio_has_feature (features,VIRTIO_NET_F_CTRL_VLAN )) { -memset (n->vlans,0 ,MAX_VLAN >>3 ); - }else { +if (!virtio_has_feature (features,VIRTIO_NET_F_CTRL_VLAN )) {memset (n->vlans,0xff ,MAX_VLAN >>3 ); }
Signature · 1 line
2.25.1
PATCH 2/4 virtio-net: Expose MAX_VLAN
In reply to Hawkins Jiaweivhost-vdpa shadowed CVQ needs to know the maximum number of vlans supported by the virtio-net device, so QEMU can restore the VLAN state in a migration.
--- hw/net/virtio-net.c | 2 -- include/hw/virtio/virtio-net.h | 6 ++++++ 2 files changed, 6 insertions(+), 2 deletions(-)
hw/net/virtio-net.c +0 -2
@@ -49,8 +49,6 @@#define VIRTIO_NET_VM_VERSION 11 -#define MAX_VLAN (1 << 12)/* Per 802.1Q definition */ -/* previously fixed value */ #define VIRTIO_NET_RX_QUEUE_DEFAULT_SIZE 256#define VIRTIO_NET_TX_QUEUE_DEFAULT_SIZE 256
include/hw/virtio/virtio-net.h +6 -0
@@ -38,6 +38,12 @@ OBJECT_DECLARE_SIMPLE_TYPE(VirtIONet, VIRTIO_NET)/* Maximum VIRTIO_NET_CTRL_MAC_TABLE_SET unicast + multicast entries. */ #define MAC_TABLE_ENTRIES 64 +/* +* The maximum number of VLANs in the VLAN filter table +* added by VIRTIO_NET_CTRL_VLAN_ADD +*/ +#define MAX_VLAN (1 << 12)/* Per 802.1Q definition */ + typedef struct virtio_net_conf { uint32_t txtimer;
Signature · 1 line
2.25.1
PATCH 3/4 vdpa: Restore vlan filtering state
In reply to Hawkins JiaweiThis patch introduces vhost_vdpa_net_load_single_vlan() and vhost_vdpa_net_load_vlan() to restore the vlan filtering state at device's startup.
--- net/vhost-vdpa.c | 49 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+)
net/vhost-vdpa.c +49 -0
@@ -965,6 +965,51 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s,return 0 ; } +static int vhost_vdpa_net_load_single_vlan (VhostVDPAState *s, +const VirtIONet *n, +uint16_t vid) +{ +const struct iovec data = { + .iov_base = &vid, + .iov_len =sizeof (vid), + }; +ssize_t dev_written =vhost_vdpa_net_load_cmd (s,VIRTIO_NET_CTRL_VLAN , +VIRTIO_NET_CTRL_VLAN_ADD , + &data,1 ); +if (unlikely (dev_written <0 )) { +return dev_written; + } +if (unlikely (*s->status !=VIRTIO_NET_OK )) { +return -EIO ; + } + +return 0 ; +} + +static int vhost_vdpa_net_load_vlan (VhostVDPAState *s, +const VirtIONet *n) +{ +int r; + +if (!virtio_vdev_has_feature (&n->parent_obj,VIRTIO_NET_F_CTRL_VLAN )) { +return 0 ; + } + +for (int i =0 ; i <MAX_VLAN >>5 ; i++) { +for (int j =0 ; n->vlans[i] && j <=0x1f ; j++) { +if (n->vlans[i] & (1U << j)) { + r =vhost_vdpa_net_load_single_vlan (s, n, (i <<5 ) + j); +if (unlikely (r !=0 )) { +return r; + } + } + } + } + +return 0 ; +} + +static int vhost_vdpa_net_load (NetClientState *nc) {VhostVDPAState *s =DO_UPCAST (VhostVDPAState, nc, nc);
@@ -995,6 +1040,10 @@ static int vhost_vdpa_net_load(NetClientState *nc)if (unlikely (r)) {return r; } + r =vhost_vdpa_net_load_vlan (s, n); +if (unlikely (r)) { +return r; + }return 0 ; }
Signature · 1 line
2.25.1
Re: [PATCH 3/4] vdpa: Restore vlan filtering state
Quoted 55 lines · Hawkins Jiawei
This patch introduces
vhost_vdpa_net_load_single_vlan()andvhost_vdpa_net_load_vlan()to restore the vlan filtering state at device's startup.Co-developed-by Eugenio Pérez <[email protected]>Signed-off-by Eugenio Pérez <[email protected]>Signed-off-by Hawkins Jiawei <[email protected]>--- net/vhost-vdpa.c | 49 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+)net/vhost-vdpa.c +35
-0@@ -965,6 +965,51 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s,return 0 ; } +static int vhost_vdpa_net_load_single_vlan (VhostVDPAState *s, +const VirtIONet *n, +uint16_t vid) +{ +const struct iovec data = { + .iov_base = &vid, + .iov_len =sizeof (vid), + }; +ssize_t dev_written =vhost_vdpa_net_load_cmd (s,VIRTIO_NET_CTRL_VLAN , +VIRTIO_NET_CTRL_VLAN_ADD , + &data,1 ); +if (unlikely (dev_written <0 )) { +return dev_written; + } +if (unlikely (*s->status !=VIRTIO_NET_OK )) { +return -EIO ; + } + +return 0 ; +} + +static int vhost_vdpa_net_load_vlan (VhostVDPAState *s, +const VirtIONet *n) +{ +int r; + +if (!virtio_vdev_has_feature (&n->parent_obj,VIRTIO_NET_F_CTRL_VLAN )) { +return 0 ; + } + +for (int i =0 ; i <MAX_VLAN >>5 ; i++) { +for (int j =0 ; n->vlans[i] && j <=0x1f ; j++) { +if (n->vlans[i] & (1U << j)) { + r =vhost_vdpa_net_load_single_vlan (s, n, (i <<5 ) + j); +if (unlikely (r !=0 )) {
net/vhost-vdpa.c static int vhost_vdpa_net_load_rx(VhostVDPAState *s, · line 1012
Quoted 10 lines
net/vhost-vdpa.c +10
-0+return r; + } + } + } + } + +return 0 ; +} + +
Nit: I'm not sure if it was here originally, but there is an extra newline here.
Quoted 17 lines
net/vhost-vdpa.c +4
-1static int vhost_vdpa_net_load (NetClientState *nc) {VhostVDPAState *s =DO_UPCAST (VhostVDPAState, nc, nc);@@ -995,6 +1040,10 @@ static int vhost_vdpa_net_load(NetClientState *nc)if (unlikely (r)) {return r; } + r =vhost_vdpa_net_load_vlan (s, n); +if (unlikely (r)) { +return r; + }return 0 ; } --2.25.1
Re: [PATCH 3/4] vdpa: Restore vlan filtering state
Quoted 60 lines · Eugenio Perez Martin
Quoted 59 lines · Hawkins Jiawei
This patch introduces
vhost_vdpa_net_load_single_vlan()andvhost_vdpa_net_load_vlan()to restore the vlan filtering state at device's startup.Co-developed-by Eugenio Pérez <[email protected]>Signed-off-by Eugenio Pérez <[email protected]>Signed-off-by Hawkins Jiawei <[email protected]>--- net/vhost-vdpa.c | 49 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+)net/vhost-vdpa.c +39
-0@@ -965,6 +965,51 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s,return 0 ; } +static int vhost_vdpa_net_load_single_vlan (VhostVDPAState *s, +const VirtIONet *n, +uint16_t vid) +{ +const struct iovec data = { + .iov_base = &vid, + .iov_len =sizeof (vid), + }; +ssize_t dev_written =vhost_vdpa_net_load_cmd (s,VIRTIO_NET_CTRL_VLAN , +VIRTIO_NET_CTRL_VLAN_ADD , + &data,1 ); +if (unlikely (dev_written <0 )) { +return dev_written; + } +if (unlikely (*s->status !=VIRTIO_NET_OK )) { +return -EIO ; + } + +return 0 ; +} + +static int vhost_vdpa_net_load_vlan (VhostVDPAState *s, +const VirtIONet *n) +{ +int r; + +if (!virtio_vdev_has_feature (&n->parent_obj,VIRTIO_NET_F_CTRL_VLAN )) { +return 0 ; + } + +for (int i =0 ; i <MAX_VLAN >>5 ; i++) { +for (int j =0 ; n->vlans[i] && j <=0x1f ; j++) { +if (n->vlans[i] & (1U << j)) { + r =vhost_vdpa_net_load_single_vlan (s, n, (i <<5 ) + j); +if (unlikely (r !=0 )) { +return r; + } + } + }
Quoted 8 lines
net/vhost-vdpa.c static int vhost_vdpa_net_load_rx(VhostVDPAState *s, · line 993
Quoted 6 lines
net/vhost-vdpa.c +6
-0+ } + +return 0 ; +} + +Nit: I'm not sure if it was here originally, but there is an extra newline here.
Hi Eugenio,
It was not here originally, it was introduced mistakenly during the refactoring process.
I will fix this in the v2 version.
Thanks!
Quoted 19 lines
Quoted 18 lines
net/vhost-vdpa.c +4
-1static int vhost_vdpa_net_load (NetClientState *nc) {VhostVDPAState *s =DO_UPCAST (VhostVDPAState, nc, nc);@@ -995,6 +1040,10 @@ static int vhost_vdpa_net_load(NetClientState *nc)if (unlikely (r)) {return r; } + r =vhost_vdpa_net_load_vlan (s, n); +if (unlikely (r)) { +return r; + }return 0 ; } --2.25.1
PATCH 4/4 vdpa: Allow VIRTIO_NET_F_CTRL_VLAN in SVQ
In reply to Hawkins JiaweiEnable SVQ with VIRTIO_NET_F_CTRL_VLAN feature.
--- net/vhost-vdpa.c | 1 + 1 file changed, 1 insertion(+)
net/vhost-vdpa.c +1 -0
@@ -111,6 +111,7 @@ static const uint64_t vdpa_svq_device_features =BIT_ULL (VIRTIO_NET_F_STATUS ) |BIT_ULL (VIRTIO_NET_F_CTRL_VQ ) |BIT_ULL (VIRTIO_NET_F_CTRL_RX ) | +BIT_ULL (VIRTIO_NET_F_CTRL_VLAN ) |BIT_ULL (VIRTIO_NET_F_CTRL_RX_EXTRA ) |BIT_ULL (VIRTIO_NET_F_MQ ) | BIT_ULL(VIRTIO_F_ANY_LAYOUT) |
Signature · 1 line
2.25.1
Quoted 4 lines · Hawkins Jiawei
This series enables shadowed CVQ to intercept VLAN commands through shadowed CVQ, update the virtio NIC device model so qemu send it in a migration, and the restore of that VLAN state in the destination.
This patch series is based on
"[PATCH 0/3] Vhost-vdpa Shadow Virtqueue VLAN support" at [1], with these changes:
- move MAX_VLAN macro to include/hw/virtio/virtio-net.h
instead of net/vhost-vdpa.c
- fix conflicts with the master branch
Thanks!
[1]. https://lists.gnu.org/archive/html/qemu-devel/2022-09/msg01016.html
Quoted 54 lines
TestStep
========
1. test the migration using vp-vdpa device
- For L0 guest, boot QEMU with two virtio-net-pci net device withctrl_vq,ctrl_vlanfeatures on, command line like:
-device virtio-net-pci,disable-legacy=on,disable-modern=off,
iommu_platform=on,mq=on,ctrl_vq=on,guest_announce=off, indirect_desc=off,queue_reset=off,ctrl_vlan=on,...- For L1 guest, apply the patch series and compile the source code,
start QEMU with two vdpa device with svq mode on, enable thectrl_vq,ctrl_vlanfeatures on, command line like:-netdev type=vhost-vdpa,x-svq=true,... -device virtio-net-pci,mq=on,guest_announce=off,ctrl_vq=on,ctrl_vlan=on,...
- For L2 source guest, run the following bash command:
```bash #!/bin/sh for idx in {1..4094} do ip link add link eth0 name vlan$idx type vlan id $idx done ```- gdb attaches the L2 dest VM and break at the
vhost_vdpa_net_load_single_vlan(), and execute the following gdbscript```gdbscript ignore 1 4094 c ```- Execute the live migration in L2 source monitor
- Result
* with this series, gdb can hit the breakpoint and continue
the executing without triggering any error or warning.Eugenio Pérez (1):
virtio-net: do not reset vlan filtering atset_featuresHawkins Jiawei (3):
virtio-net: ExposeMAX_VLAN
vdpa: Restore vlan filtering state
vdpa: AllowVIRTIO_NET_F_CTRL_VLANin SVQhw/net/virtio-net.c | 6 +--- include/hw/virtio/virtio-net.h | 6 ++++ net/vhost-vdpa.c | 50 ++++++++++++++++++++++++++++++++++ 3 files changed, 57 insertions(+), 5 deletions(-)
Lei Yang
Tested-byQE tested this series patch with real hardware, it can support setup vlan for the nic, and the vlan id still can be found after finished migrate. In addition, this series patch also help to test another patch, can got the expect result.
Quoted 75 lines · Hawkins Jiawei
Quoted 4 lines · Hawkins Jiawei
This series enables shadowed CVQ to intercept VLAN commands through shadowed CVQ, update the virtio NIC device model so qemu send it in a migration, and the restore of that VLAN state in the destination.
This patch series is based on
"[PATCH 0/3] Vhost-vdpa Shadow Virtqueue VLAN support" at [1], with these changes:- move
MAX_VLANmacro toinclude/hw/virtio/virtio-net.h
instead ofnet/vhost-vdpa.c
- fix conflicts with the master branchThanks!
[1]. https://lists.gnu.org/archive/html/qemu-devel/2022-09/msg01016.html
Quoted 54 lines
TestStep
========
1. test the migration using vp-vdpa device
- For L0 guest, boot QEMU with two virtio-net-pci net device withctrl_vq,ctrl_vlanfeatures on, command line like:
-device virtio-net-pci,disable-legacy=on,disable-modern=off,
iommu_platform=on,mq=on,ctrl_vq=on,guest_announce=off, indirect_desc=off,queue_reset=off,ctrl_vlan=on,...- For L1 guest, apply the patch series and compile the source code,
start QEMU with two vdpa device with svq mode on, enable thectrl_vq,ctrl_vlanfeatures on, command line like:-netdev type=vhost-vdpa,x-svq=true,... -device virtio-net-pci,mq=on,guest_announce=off,ctrl_vq=on,ctrl_vlan=on,...
- For L2 source guest, run the following bash command:
```bash #!/bin/sh for idx in {1..4094} do ip link add link eth0 name vlan$idx type vlan id $idx done ```- gdb attaches the L2 dest VM and break at the
vhost_vdpa_net_load_single_vlan(), and execute the following gdbscript```gdbscript ignore 1 4094 c ```- Execute the live migration in L2 source monitor
- Result
* with this series, gdb can hit the breakpoint and continue
the executing without triggering any error or warning.Eugenio Pérez (1):
virtio-net: do not reset vlan filtering atset_featuresHawkins Jiawei (3):
virtio-net: ExposeMAX_VLAN
vdpa: Restore vlan filtering state
vdpa: AllowVIRTIO_NET_F_CTRL_VLANin SVQhw/net/virtio-net.c | 6 +--- include/hw/virtio/virtio-net.h | 6 ++++ net/vhost-vdpa.c | 50 ++++++++++++++++++++++++++++++++++ 3 files changed, 57 insertions(+), 5 deletions(-)