Skip to contentSkip to conversation

KNI performance is not what is claimed

DISCUSSION Stephen Hemminger 4 participants 7 messages Sep 20, 2018 – Oct 1, 2018

lore
Share & tools
lore
#0 lore raw

I wonder if KNI is claiming performance that was never measured on current CPU, OS, DPDK.

With single stream and TCP testing on IXGBE (DPDK), I see lowest performance with KNI.

		Rx		Tx
	KNI	3.2 Gbit/sec	1.3 Gbit/sec
	TAP	4.9		4.7
	Virtio	5.6		8.6

Perhaps for 18.11 we should change documentation to remove language claiming better performance with KNI, and then plan for future deprecation?

#1 lore raw
Quoted 6 lines · Stephen Hemminger

I wonder if KNI is claiming performance that was never measured on current CPU, OS, DPDK.

With single stream and TCP testing on IXGBE (DPDK), I see lowest performance with KNI.

Quoted 9 lines
                Rx              Tx
        KNI     3.2 Gbit/sec    1.3 Gbit/sec
        TAP     4.9             4.7
        Virtio  5.6             8.6

Perhaps for 18.11 we should change documentation to remove language claiming
better performance with KNI, and then plan for future deprecation?

Do TAP and Virtio provide equivalent function to KNI? I can't speak for any other products, but ours is dependent on KNI. The ability for control plane applications to use normal Linux sockets with DPDK is key even if it isn't performant.

Hopefully the answer is "yes", in which case I'll happily port over to using one of the faster mechanisms.

Thanks,
Jay

#2 lore raw
Quoted 19 lines · On Thu, 20 Sep 2018 15:02:53 -0500 Jay Rolette <[email protected]> wrote:
Quoted 6 lines · Stephen Hemminger

I wonder if KNI is claiming performance that was never measured on current CPU, OS, DPDK.

With single stream and TCP testing on IXGBE (DPDK), I see lowest performance with KNI.

Quoted 9 lines
                Rx              Tx
        KNI     3.2 Gbit/sec    1.3 Gbit/sec
        TAP     4.9             4.7
        Virtio  5.6             8.6

Perhaps for 18.11 we should change documentation to remove language claiming
better performance with KNI, and then plan for future deprecation?

Quoted 4 lines

Do TAP and Virtio provide equivalent function to KNI? I can't speak for any other products, but ours is dependent on KNI. The ability for control plane applications to use normal Linux sockets with DPDK is key even if it isn't performant.

Yes, all are equivalent. KNI was just invented as "faster version of TAP" but it isn't

Quoted 5 lines

Hopefully the answer is "yes", in which case I'll happily port over to using one of the faster mechanisms.

Thanks,
Jay

#3 lore raw
In reply to Jay Rolette
Quoted 24 lines · On Thu, 20 Sep 2018 15:02:53 -0500 Jay Rolette <[email protected]> wrote:
Quoted 6 lines · Stephen Hemminger

I wonder if KNI is claiming performance that was never measured on current CPU, OS, DPDK.

With single stream and TCP testing on IXGBE (DPDK), I see lowest performance with KNI.

Quoted 9 lines
                Rx              Tx
        KNI     3.2 Gbit/sec    1.3 Gbit/sec
        TAP     4.9             4.7
        Virtio  5.6             8.6

Perhaps for 18.11 we should change documentation to remove language claiming
better performance with KNI, and then plan for future deprecation?

Do TAP and Virtio provide equivalent function to KNI? I can't speak for any other products, but ours is dependent on KNI. The ability for control plane applications to use normal Linux sockets with DPDK is key even if it isn't performant.

Quoted 5 lines

Hopefully the answer is "yes", in which case I'll happily port over to using one of the faster mechanisms.

Thanks,
Jay

See:
https://doc.dpdk.org/guides-17.11/howto/virtio_user_as_exceptional_path.html

#4 lore raw
Quoted 34 lines · Stephen Hemminger
Quoted 4 lines · On Thu, 20 Sep 2018 15:02:53 -0500 Jay Rolette <[email protected]> wrote:
Quoted 1 line · Stephen Hemminger

I wonder if KNI is claiming performance that was never measured on

current

Quoted 10 lines
Quoted 10 lines

CPU, OS, DPDK.

With single stream and TCP testing on IXGBE (DPDK), I see lowest performance with KNI.

                Rx              Tx
        KNI     3.2 Gbit/sec    1.3 Gbit/sec
        TAP     4.9             4.7
        Virtio  5.6             8.6
Quoted 6 lines
Quoted 4 lines

Perhaps for 18.11 we should change documentation to remove language claiming
better performance with KNI, and then plan for future deprecation?

Do TAP and Virtio provide equivalent function to KNI? I can't speak for

any

Quoted 1 line

other products, but ours is dependent on KNI. The ability for control

plane

Quoted 1 line

applications to use normal Linux sockets with DPDK is key even if it

isn't

Quoted 5 lines

performant.

Hopefully the answer is "yes", in which case I'll happily port over to using one of the faster mechanisms.

Quoted 6 lines
Quoted 2 lines

Thanks,
Jay

See:

https://doc.dpdk.org/guides-17.11/howto/virtio_user_as_exceptional_path.html

Thanks. Looks like it's time to run some experiments again.

Jay

#5 lore raw
Quoted 50 lines

-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of Jay Rolette
Sent: Friday, September 21, 2018 5:39 AM
To: Stephen Hemminger <[email protected]>
Cc: DPDK <[email protected]>
Subject: Re: [dpdk-dev] KNI performance is not what is claimed

Quoted 19 lines · Stephen Hemminger
Quoted 4 lines · On Thu, 20 Sep 2018 15:02:53 -0500 Jay Rolette <[email protected]> wrote:
Quoted 1 line · Stephen Hemminger

I wonder if KNI is claiming performance that was never measured on

current

Quoted 11 lines
Quoted 11 lines

CPU, OS, DPDK.

With single stream and TCP testing on IXGBE (DPDK), I see lowest performance with KNI.

                Rx              Tx
        KNI     3.2 Gbit/sec    1.3 Gbit/sec
        TAP     4.9             4.7
        Virtio  5.6             8.6

Perhaps for 18.11 we should change documentation to remove

language

Quoted 18 lines
Quoted 5 lines
Quoted 3 lines

claiming
better performance with KNI, and then plan for future deprecation?

Do TAP and Virtio provide equivalent function to KNI? I can't speak for

any

Quoted 1 line

other products, but ours is dependent on KNI. The ability for control

plane

Quoted 1 line

applications to use normal Linux sockets with DPDK is key even if it

isn't

Quoted 7 lines

performant.

Hopefully the answer is "yes", in which case I'll happily port over to using one of the faster mechanisms.

Thanks,
Jay

Quoted 7 lines
Quoted 3 lines

See:

https://doc.dpdk.org/guides-

17.11/howto/virtio_user_as_exceptional_path.html

Thanks. Looks like it's time to run some experiments again.

To do the test with the latest DPDK 17.11 LTS, you'll need the below one-line fix (which was missed during backporting) to enable the offloads:
https://git.dpdk.org/dpdk/commit/?id=bce7e9050f9b

You can also refer to this paper for more details:
https://dl.acm.org/citation.cfm?id=3098586

-Zhihong

Quoted 2 lines

Jay

#6 lore raw
Quoted 43 lines
Quoted 43 lines
Quoted 20 lines · Stephen Hemminger
Quoted 5 lines · On Thu, 20 Sep 2018 15:02:53 -0500 Jay Rolette <[email protected]> wrote:
Quoted 2 lines · Stephen Hemminger

I wonder if KNI is claiming performance that was never measured on

current

Quoted 11 lines
Quoted 11 lines

CPU, OS, DPDK.

With single stream and TCP testing on IXGBE (DPDK), I see lowest performance with KNI.

                Rx              Tx
        KNI     3.2 Gbit/sec    1.3 Gbit/sec
        TAP     4.9             4.7
        Virtio  5.6             8.6

Perhaps for 18.11 we should change documentation to remove

language

Quoted 18 lines
Quoted 6 lines
Quoted 3 lines

claiming
better performance with KNI, and then plan for future deprecation?

Do TAP and Virtio provide equivalent function to KNI? I can't speak for

any

Quoted 2 lines

other products, but ours is dependent on KNI. The ability for control

plane

Quoted 2 lines

applications to use normal Linux sockets with DPDK is key even if it

isn't

Quoted 5 lines

performant.

Hopefully the answer is "yes", in which case I'll happily port over to using one of the faster mechanisms.

Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 2 lines

Thanks,
Jay

See:

https://doc.dpdk.org/guides-

17.11/howto/virtio_user_as_exceptional_path.html

This document says 'similar to KNI, this solution would use one or more kthreads to send/receive packets to/from user space DPDK applications, which has little impact on user space polling thread (except that it might enter into kernel space to wake up those kthreads if necessary)'. Does this mean the user space polling thread will make a system call? I am not a kernel expert. But, would this mean, the user space thread might block?

Quoted 16 lines
Quoted 3 lines

Thanks. Looks like it's time to run some experiments again.

To do the test with the latest DPDK 17.11 LTS, you'll need the below one-line fix (which was missed during backporting) to enable the
offloads:
https://git.dpdk.org/dpdk/commit/?id=bce7e9050f9b

You can also refer to this paper for more details:
https://dl.acm.org/citation.cfm?id=3098586

-Zhihong

Quoted 2 lines

Jay