Skip to contentSkip to conversation

RFC - vdev_netvsc automatic blacklisting

dpdk-dev Stephen Hemminger 2 participants 6 messages Jun 4, 2019 – Jun 12, 2019

lore
Share & tools
lore
#0 lore raw

When using DPDK on Azure it is common to have one non-DPDK interface. If that non-DPDK interface is present vdev_netvsc correctly skip it. But if the non-DPDK has accelerated networking the Mellanox driver will still get associated with DPDK (and break connectivity).

The current process is to tell users to do whitelist or blacklist the PCI device(s) not used for DPDK. But vdev_netvsc already is doing a lot of looking at devices and VF devices.

Could vdev_netvsc just do this automatically by setting devargs for the VF to blacklist?

#1 lore raw

Hi Stephen

From: Stephen Hemminger

Quoted 11 lines

When using DPDK on Azure it is common to have one non-DPDK interface. If that non-DPDK interface is present vdev_netvsc correctly skip it. But if the non-DPDK has accelerated networking the Mellanox driver will still get associated with DPDK (and break connectivity).

The current process is to tell users to do whitelist or blacklist the PCI device(s) not used for DPDK. But vdev_netvsc already is doing a lot of looking at devices and VF devices.

Could vdev_netvsc just do this automatically by setting devargs for the VF to blacklist?

There is way to blacklist a device by setting it a rout\IP\IPv6, from the VDEV_NETVSC doc:
"Not specifying either iface or mac makes this driver attach itself to all unrouted NetVSC interfaces found on the system. Specifying the device makes this driver attach itself to the device regardless the device routes."

So, we are expecting that used VFs will be with a rout and DPDK VFs will not be with a rout.

Doesn't it enough?

Matan

#2 lore raw
Quoted 19 lines · On Tue, 11 Jun 2019 06:21:21 +0000 Matan Azrad <[email protected]> wrote:

Hi Stephen

From: Stephen Hemminger

Quoted 11 lines

When using DPDK on Azure it is common to have one non-DPDK interface. If that non-DPDK interface is present vdev_netvsc correctly skip it. But if the non-DPDK has accelerated networking the Mellanox driver will still get associated with DPDK (and break connectivity).

The current process is to tell users to do whitelist or blacklist the PCI device(s) not used for DPDK. But vdev_netvsc already is doing a lot of looking at devices and VF devices.

Could vdev_netvsc just do this automatically by setting devargs for the VF to blacklist?

There is way to blacklist a device by setting it a rout\IP\IPv6, from the VDEV_NETVSC doc:
"Not specifying either iface or mac makes this driver attach itself to all unrouted NetVSC interfaces found on the system. Specifying the device makes this driver attach itself to the device regardless the device routes."

Quoted 6 lines

So, we are expecting that used VFs will be with a rout and DPDK VFs will not be with a rout.

Doesn't it enough?

Matan

I am talking about if eth0 has a route, it gets skipped but the associated MLX SR-IOV device does not. When the MLX device is then configured for DPDK, it breaks it for use by kernel; and therefore connectivity with the VM is lost.

#3 lore raw

From: Stephen Hemminger

Quoted 27 lines
Quoted 4 lines

Hi Stephen

From: Stephen Hemminger

Quoted 1 line

When using DPDK on Azure it is common to have one non-DPDK

interface.

Quoted 9 lines
Quoted 9 lines

If that non-DPDK interface is present vdev_netvsc correctly skip it. But if the non-DPDK has accelerated networking the Mellanox driver will still get associated with DPDK (and break connectivity).

The current process is to tell users to do whitelist or blacklist the PCI
device(s) not used for DPDK. But vdev_netvsc already is doing a lot of looking at devices and VF devices.

Quoted 5 lines
Quoted 2 lines

Could vdev_netvsc just do this automatically by setting devargs for the VF to blacklist?

There is way to blacklist a device by setting it a rout\IP\IPv6, from the

VDEV_NETVSC doc:

Quoted 1 line

"Not specifying either iface or mac makes this driver attach itself to all

unrouted NetVSC interfaces found on the system. Specifying the device makes this driver attach itself to the device regardless the device routes."

Quoted 2 lines

So, we are expecting that used VFs will be with a rout and DPDK VFs will not

be with a rout.

Quoted 1 line
Quoted 8 lines
Quoted 4 lines

Doesn't it enough?

Matan

I am talking about if eth0 has a route, it gets skipped but the associated MLX SR-IOV device does not. When the MLX device is then configured for DPDK, it breaks it for use by kernel; and therefore connectivity with the VM is lost.

Ok, I think I got you.
You want to blacklist the PCI device which its netvsc net-device is detected as routed. Do you?

If so,

I don't think that probing the pci device hurts the connectivity, only the configuration should hurt it.

It means that the application configures the device and hurt it. Doesn't it an application issue?

Matan

#4 lore raw
Quoted 40 lines · On Wed, 12 Jun 2019 05:15:47 +0000 Matan Azrad <[email protected]> wrote:

From: Stephen Hemminger

Quoted 27 lines
Quoted 4 lines

Hi Stephen

From: Stephen Hemminger

Quoted 1 line

When using DPDK on Azure it is common to have one non-DPDK

interface.

Quoted 9 lines
Quoted 9 lines

If that non-DPDK interface is present vdev_netvsc correctly skip it. But if the non-DPDK has accelerated networking the Mellanox driver will still get associated with DPDK (and break connectivity).

The current process is to tell users to do whitelist or blacklist the PCI
device(s) not used for DPDK. But vdev_netvsc already is doing a lot of looking at devices and VF devices.

Quoted 5 lines
Quoted 2 lines

Could vdev_netvsc just do this automatically by setting devargs for the VF to blacklist?

There is way to blacklist a device by setting it a rout\IP\IPv6, from the

VDEV_NETVSC doc:

Quoted 1 line

"Not specifying either iface or mac makes this driver attach itself to all

unrouted NetVSC interfaces found on the system. Specifying the device makes this driver attach itself to the device regardless the device routes."

Quoted 2 lines

So, we are expecting that used VFs will be with a rout and DPDK VFs will not

be with a rout.

Quoted 1 line
Quoted 8 lines
Quoted 4 lines

Doesn't it enough?

Matan

I am talking about if eth0 has a route, it gets skipped but the associated MLX SR-IOV device does not. When the MLX device is then configured for DPDK, it breaks it for use by kernel; and therefore connectivity with the VM is lost.

Ok, I think I got you.
You want to blacklist the PCI device which its netvsc net-device is detected as routed. Do you?

Quoted 8 lines

If so,

I don't think that probing the pci device hurts the connectivity, only the configuration should hurt it.

It means that the application configures the device and hurt it. Doesn't it an application issue?

Matan

Actually probing does hurt, it corrupts the MLX driver.
In theory, the driver supports bifurcated but in practice it is greedy and grabs all flows.

#5 lore raw

Hi

+ Shahaf

From: Stephen Hemminger

Quoted 54 lines
Quoted 26 lines · On Wed, 12 Jun 2019 05:15:47 +0000 Matan Azrad <[email protected]> wrote:

From: Stephen Hemminger

Quoted 25 lines
Quoted 4 lines

Hi Stephen

From: Stephen Hemminger

Quoted 1 line

When using DPDK on Azure it is common to have one non-DPDK

interface.

Quoted 9 lines
Quoted 9 lines

If that non-DPDK interface is present vdev_netvsc correctly skip it. But if the non-DPDK has accelerated networking the Mellanox driver will still get associated with DPDK (and break connectivity).

The current process is to tell users to do whitelist or blacklist the PCI
device(s) not used for DPDK. But vdev_netvsc already is doing a lot of looking at devices and VF devices.

Quoted 6 lines
Quoted 2 lines

Could vdev_netvsc just do this automatically by setting devargs for the VF to blacklist?

There is way to blacklist a device by setting it a rout\IP\IPv6, from the

VDEV_NETVSC doc:

Quoted 2 lines

"Not specifying either iface or mac makes this driver attach itself to all

unrouted NetVSC interfaces found on the system. Specifying the device makes this driver attach itself to the device regardless the device

routes."

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

So, we are expecting that used VFs will be with a rout and DPDK VFs will not

be with a rout.

Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines

Doesn't it enough?

Matan

I am talking about if eth0 has a route, it gets skipped but the associated MLX SR-IOV device does not. When the MLX device is then configured for DPDK, it breaks it for use by kernel; and therefore

connectivity with the VM is lost.

Quoted 3 lines

Ok, I think I got you.
You want to blacklist the PCI device which its netvsc net-device is detected

as routed. Do you?

Quoted 4 lines

If so,

I don't think that probing the pci device hurts the connectivity, only the

configuration should hurt it.

Quoted 1 line
Quoted 8 lines
Quoted 4 lines

It means that the application configures the device and hurt it. Doesn't it an application issue?

Matan

Actually probing does hurt, it corrupts the MLX driver.
In theory, the driver supports bifurcated but in practice it is greedy and grabs all flows.

I can't see promiscuous configuration in the probing code. Looks like it is an application configuration.

Shahaf , are you agree?

Matan