KNI alternatives
Hi,
As discussed today in the techboard meeting, KNI has probably better alternatives today without using an out-of-tree module. Virtio-user is a good candidate to replace KNI.
What is the performance of TAP?
Is there a way to leverage af_packet, af_xdp, or even pcap interfaces?
In order to avoid using the KNI out-of-tree module,
we should make the librte_kni compatible with an other interface. The big plan is then to move the KNI module out of the main DPDK repo, as we did for igb_uio.
The first step of this plan might be to document pros & cons of the KNI alternatives inside the KNI documentation:
https://doc.dpdk.org/guides/prog_guide/kernel_nic_interface.html
As a start, this doc should be better referenced:
https://doc.dpdk.org/guides/howto/virtio_user_as_exceptional_path.html
Note: I won't do this update myself, so feel free to step up! Thanks
Last time I tried.
Virtio user was as fast as KNI and consumed less cpu. Was seeing 10mpps Tap was much slower. Like 1mpps.
Vpp uses virtio user.
Sorry for top post. Only have phone internet
Quoted 26 lines · Thomas Monjalon
Hi,
As discussed today in the techboard meeting, KNI has probably better alternatives today without using an out-of-tree module. Virtio-user is a good candidate to replace KNI.
What is the performance of TAP?
Is there a way to leverageaf_packet,af_xdp, or even pcap interfaces?In order to avoid using the KNI out-of-tree module,
we should make thelibrte_knicompatible with an other interface. The big plan is then to move the KNI module out of the main DPDK repo, as we did forigb_uio.The first step of this plan might be to document pros & cons of the KNI alternatives inside the KNI documentation:
https://doc.dpdk.org/guides/prog_guide/kernel_nic_interface.htmlAs a start, this doc should be better referenced:
https://doc.dpdk.org/guides/howto/virtio_user_as_exceptional_path.html
Note: I won't do this update myself, so feel free to step up! Thanks
Quoted 4 lines · Stephen Hemminger
Last time I tried.
Virtio user was as fast as KNI and consumed less cpu. Was seeing 10mpps Tap was much slower. Like 1mpps.
Quoted 13 lines
Vpp uses virtio user.
Sorry for top post. Only have phone internet
Quoted 7 lines · Thomas Monjalon
Hi,
As discussed today in the techboard meeting, KNI has probably better alternatives today without using an out-of-tree module. Virtio-user is a good candidate to replace KNI.
What is the performance of TAP?
Is there a way to leverageaf_packet,af_xdp, or even pcap interfaces?
Performance is not the only question. The advantage of KNI we are currently using is
the ability to control the DPDK interfaces by the kernel.
For example, to implement bonding in the DPDK application, it is possible to create KNI
pair for each physical interface, create a bond interface in Linux over those KNI interfaces
and just pass LACP packets between the app and the kernel. The kernel itself will control
MACs, MTU, etc. of underlying interfaces. AFAIK it's not possible with virtio-user or tap.
Am I wrong?
Quoted 22 lines
Quoted 21 lines
In order to avoid using the KNI out-of-tree module,
we should make thelibrte_knicompatible with an other interface. The big plan is then to move the KNI module out of the main DPDK repo, as we did forigb_uio.The first step of this plan might be to document pros & cons of the KNI alternatives inside the KNI documentation:
https://doc.dpdk.org/guides/prog_guide/kernel_nic_interface.htmlAs a start, this doc should be better referenced:
https://doc.dpdk.org/guides/howto/virtio_user_as_exceptional_path.html
Note: I won't do this update myself, so feel free to step up! Thanks
13/01/2021 19:17, Igor Ryzhov:
Quoted 16 lines
Quoted 10 lines · Stephen Hemminger
Quoted 7 lines · Thomas Monjalon
Hi,
As discussed today in the techboard meeting, KNI has probably better alternatives today without using an out-of-tree module. Virtio-user is a good candidate to replace KNI.
What is the performance of TAP?
Is there a way to leverageaf_packet,af_xdp, or even pcap interfaces?Quoted 3 lines
Last time I tried.
Virtio user was as fast as KNI and consumed less cpu. Was seeing 10mpps Tap was much slower. Like 1mpps.
Quoted 12 lines
Performance is not the only question. The advantage of KNI we are currently using is
the ability to control the DPDK interfaces by the kernel.
For example, to implement bonding in the DPDK application, it is possible to create KNI
pair for each physical interface, create a bond interface in Linux over those KNI interfaces
and just pass LACP packets between the app and the kernel. The kernel itself will control
MACs, MTU, etc. of underlying interfaces. AFAIK it's not possible with virtio-user or tap.
Am I wrong?
I see at least 2 alternatives for bonding with kernel management:
- mlx5 bonding
- af_xdp interface for most of NICs
Quoted 33 lines · Thomas Monjalon
13/01/2021 19:17, Igor Ryzhov:
Quoted 3 lines
Quoted 1 line · Stephen Hemminger
On Wed, Jan 13, 2021, 9:06 AM Thomas Monjalon <[email protected]>
Quoted 8 lines · wrote:
Quoted 8 lines
Quoted 8 lines
Hi,
As discussed today in the techboard meeting, KNI has probably better alternatives today without using an out-of-tree module. Virtio-user is a good candidate to replace KNI.
What is the performance of TAP?
Is there a way to leverageaf_packet,af_xdp, or even pcapinterfaces?
Quoted 6 lines
Quoted 4 lines
Last time I tried.
Virtio user was as fast as KNI and consumed less cpu. Was seeing 10mpps Tap was much slower. Like 1mpps.Performance is not the only question. The advantage of KNI we are
currently
Quoted 11 lines
using is
the ability to control the DPDK interfaces by the kernel.
For example, to implement bonding in the DPDK application, it is possible to create KNI
pair for each physical interface, create a bond interface in Linux over those KNI interfaces
and just pass LACP packets between the app and the kernel. The kernel itself will control
MACs, MTU, etc. of underlying interfaces. AFAIK it's not possible with virtio-user or tap.
Am I wrong?
Quoted 4 lines
I see at least 2 alternatives for bonding with kernel management:
- mlx5 bonding
-af_xdpinterface for most of NICs
Bonding is just an example. My point is that KNI allows changing MAC, MTU, rx mode and admin status of a NIC used by DPDK application - all using standard Linux utilities without any need for additional APIs provided by the app.
Yes, Mellanox NICs also allow this but it's just one of many drivers supported by DPDK.
I'll dig into af_xdp, thanks.
Controlling DPDK interfaces with KNI is known to be broken. Kni calls usepspace with network mutex held. The problem is not fixable.
The kernel does provide netlink and devlink API's but there is no generic solution with DPDK.
Quoted 54 lines · Igor Ryzhov
Quoted 35 lines · Thomas Monjalon
13/01/2021 19:17, Igor Ryzhov:
Quoted 3 lines
Quoted 1 line · Stephen Hemminger
On Wed, Jan 13, 2021, 9:06 AM Thomas Monjalon <[email protected]>
Quoted 8 lines · wrote:
Quoted 8 lines
Quoted 8 lines
Hi,
As discussed today in the techboard meeting, KNI has probably better alternatives today without using an out-of-tree module. Virtio-user is a good candidate to replace KNI.
What is the performance of TAP?
Is there a way to leverageaf_packet,af_xdp, or even pcapinterfaces?
Quoted 3 lines
Quoted 3 lines
Last time I tried.
Virtio user was as fast as KNI and consumed less cpu. Was seeing10mpps
Quoted 3 lines
Quoted 1 line
Tap was much slower. Like 1mpps.
Performance is not the only question. The advantage of KNI we are
currently
Quoted 3 lines
using is
the ability to control the DPDK interfaces by the kernel. For example, to implement bonding in the DPDK application, it ispossible
Quoted 8 lines
to create KNI
pair for each physical interface, create a bond interface in Linux over those KNI interfaces
and just pass LACP packets between the app and the kernel. The kernel itself will control
MACs, MTU, etc. of underlying interfaces. AFAIK it's not possible with virtio-user or tap.
Am I wrong?Quoted 4 lines
I see at least 2 alternatives for bonding with kernel management:
- mlx5 bonding
-af_xdpinterface for most of NICsBonding is just an example. My point is that KNI allows changing MAC, MTU, rx mode and admin status of a NIC used by DPDK application - all using standard Linux utilities without any need for additional APIs provided by the app.
Yes, Mellanox NICs also allow this but it's just one of many drivers supported by DPDK.
I'll dig intoaf_xdp, thanks.