RFC - vdev_netvsc automatic blacklisting
When using DPDK on Azure it is common to have one non-DPDK interface. If that non-DPDK interface is present vdev_netvsc correctly skip it. But if the non-DPDK has accelerated networking the Mellanox driver will still get associated with DPDK (and break connectivity).
The current process is to tell users to do whitelist or blacklist the PCI device(s) not used for DPDK. But vdev_netvsc already is doing a lot of looking at devices and VF devices.
Could vdev_netvsc just do this automatically by setting devargs for the VF to blacklist?
Hi Stephen
From: Stephen Hemminger
Quoted 11 lines
When using DPDK on Azure it is common to have one non-DPDK interface. If that non-DPDK interface is present
vdev_netvsccorrectly skip it. But if the non-DPDK has accelerated networking the Mellanox driver will still get associated with DPDK (and break connectivity).The current process is to tell users to do whitelist or blacklist the PCI device(s) not used for DPDK. But
vdev_netvscalready is doing a lot of looking at devices and VF devices.Could
vdev_netvscjust do this automatically by setting devargs for the VF to blacklist?
There is way to blacklist a device by setting it a rout\IP\IPv6, from the VDEV_NETVSC doc:
"Not specifying either iface or mac makes this driver attach itself to all unrouted NetVSC interfaces found on the system. Specifying the device makes this driver attach itself to the device regardless the device routes."
So, we are expecting that used VFs will be with a rout and DPDK VFs will not be with a rout.
Doesn't it enough?
Matan
Quoted 19 lines · On Tue, 11 Jun 2019 06:21:21 +0000 Matan Azrad <[email protected]> wrote:
Hi Stephen
From: Stephen Hemminger
Quoted 11 lines
When using DPDK on Azure it is common to have one non-DPDK interface. If that non-DPDK interface is present
vdev_netvsccorrectly skip it. But if the non-DPDK has accelerated networking the Mellanox driver will still get associated with DPDK (and break connectivity).The current process is to tell users to do whitelist or blacklist the PCI device(s) not used for DPDK. But
vdev_netvscalready is doing a lot of looking at devices and VF devices.Could
vdev_netvscjust do this automatically by setting devargs for the VF to blacklist?There is way to blacklist a device by setting it a rout\IP\IPv6, from the
VDEV_NETVSCdoc:
"Not specifying either iface or mac makes this driver attach itself to all unrouted NetVSC interfaces found on the system. Specifying the device makes this driver attach itself to the device regardless the device routes."
Quoted 6 lines
So, we are expecting that used VFs will be with a rout and DPDK VFs will not be with a rout.
Doesn't it enough?
Matan
I am talking about if eth0 has a route, it gets skipped but the associated MLX SR-IOV device does not. When the MLX device is then configured for DPDK, it breaks it for use by kernel; and therefore connectivity with the VM is lost.
From: Stephen Hemminger
Quoted 27 lines
Quoted 4 lines
Hi Stephen
From: Stephen Hemminger
Quoted 1 line
When using DPDK on Azure it is common to have one non-DPDK
interface.
Quoted 9 lines
Quoted 9 lines
If that non-DPDK interface is present
vdev_netvsccorrectly skip it. But if the non-DPDK has accelerated networking the Mellanox driver will still get associated with DPDK (and break connectivity).The current process is to tell users to do whitelist or blacklist the PCI
device(s) not used for DPDK. Butvdev_netvscalready is doing a lot of looking at devices and VF devices.Quoted 5 lines
Quoted 2 lines
Could
vdev_netvscjust do this automatically by setting devargs for the VF to blacklist?There is way to blacklist a device by setting it a rout\IP\IPv6, from the
VDEV_NETVSCdoc:Quoted 1 line
"Not specifying either iface or mac makes this driver attach itself to all
unrouted NetVSC interfaces found on the system. Specifying the device makes this driver attach itself to the device regardless the device routes."
Quoted 2 lines
So, we are expecting that used VFs will be with a rout and DPDK VFs will not
be with a rout.
Quoted 1 line
Quoted 8 lines
Quoted 4 lines
Doesn't it enough?
Matan
I am talking about if eth0 has a route, it gets skipped but the associated MLX SR-IOV device does not. When the MLX device is then configured for DPDK, it breaks it for use by kernel; and therefore connectivity with the VM is lost.
Ok, I think I got you.
You want to blacklist the PCI device which its netvsc net-device is detected as routed. Do you?
If so,
I don't think that probing the pci device hurts the connectivity, only the configuration should hurt it.
It means that the application configures the device and hurt it. Doesn't it an application issue?
Matan
Quoted 40 lines · On Wed, 12 Jun 2019 05:15:47 +0000 Matan Azrad <[email protected]> wrote:
From: Stephen Hemminger
Quoted 27 lines
Quoted 4 lines
Hi Stephen
From: Stephen Hemminger
Quoted 1 line
When using DPDK on Azure it is common to have one non-DPDK
interface.
Quoted 9 lines
Quoted 9 lines
If that non-DPDK interface is present
vdev_netvsccorrectly skip it. But if the non-DPDK has accelerated networking the Mellanox driver will still get associated with DPDK (and break connectivity).The current process is to tell users to do whitelist or blacklist the PCI
device(s) not used for DPDK. Butvdev_netvscalready is doing a lot of looking at devices and VF devices.Quoted 5 lines
Quoted 2 lines
Could
vdev_netvscjust do this automatically by setting devargs for the VF to blacklist?There is way to blacklist a device by setting it a rout\IP\IPv6, from the
VDEV_NETVSCdoc:Quoted 1 line
"Not specifying either iface or mac makes this driver attach itself to all
unrouted NetVSC interfaces found on the system. Specifying the device makes this driver attach itself to the device regardless the device routes."
Quoted 2 lines
So, we are expecting that used VFs will be with a rout and DPDK VFs will not
be with a rout.
Quoted 1 line
Quoted 8 lines
Quoted 4 lines
Doesn't it enough?
Matan
I am talking about if eth0 has a route, it gets skipped but the associated MLX SR-IOV device does not. When the MLX device is then configured for DPDK, it breaks it for use by kernel; and therefore connectivity with the VM is lost.
Ok, I think I got you.
You want to blacklist the PCI device which its netvsc net-device is detected as routed. Do you?
Quoted 8 lines
If so,
I don't think that probing the pci device hurts the connectivity, only the configuration should hurt it.
It means that the application configures the device and hurt it. Doesn't it an application issue?
Matan
Actually probing does hurt, it corrupts the MLX driver.
In theory, the driver supports bifurcated but in practice it is greedy and grabs all flows.
Hi
+ Shahaf
From: Stephen Hemminger
Quoted 54 lines
Quoted 26 lines · On Wed, 12 Jun 2019 05:15:47 +0000 Matan Azrad <[email protected]> wrote:
From: Stephen Hemminger
Quoted 25 lines
Quoted 4 lines
Hi Stephen
From: Stephen Hemminger
Quoted 1 line
When using DPDK on Azure it is common to have one non-DPDK
interface.
Quoted 9 lines
Quoted 9 lines
If that non-DPDK interface is present
vdev_netvsccorrectly skip it. But if the non-DPDK has accelerated networking the Mellanox driver will still get associated with DPDK (and break connectivity).The current process is to tell users to do whitelist or blacklist the PCI
device(s) not used for DPDK. Butvdev_netvscalready is doing a lot of looking at devices and VF devices.Quoted 6 lines
Quoted 2 lines
Could
vdev_netvscjust do this automatically by setting devargs for the VF to blacklist?There is way to blacklist a device by setting it a rout\IP\IPv6, from the
VDEV_NETVSCdoc:Quoted 2 lines
"Not specifying either iface or mac makes this driver attach itself to all
unrouted NetVSC interfaces found on the system. Specifying the device makes this driver attach itself to the device regardless the device
routes."
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
So, we are expecting that used VFs will be with a rout and DPDK VFs will not
be with a rout.
Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
Doesn't it enough?
Matan
I am talking about if eth0 has a route, it gets skipped but the associated MLX SR-IOV device does not. When the MLX device is then configured for DPDK, it breaks it for use by kernel; and therefore
connectivity with the VM is lost.
Quoted 3 lines
Ok, I think I got you.
You want to blacklist the PCI device which its netvsc net-device is detectedas routed. Do you?
Quoted 4 lines
If so,
I don't think that probing the pci device hurts the connectivity, only the
configuration should hurt it.
Quoted 1 line
Quoted 8 lines
Quoted 4 lines
It means that the application configures the device and hurt it. Doesn't it an application issue?
Matan
Actually probing does hurt, it corrupts the MLX driver.
In theory, the driver supports bifurcated but in practice it is greedy and grabs all flows.
I can't see promiscuous configuration in the probing code. Looks like it is an application configuration.
Shahaf , are you agree?
Matan