Skip to content

Vhost_net TX batching

Jason Wang kvm lkml netdev virtualization 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest11 patches 49 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/11net: sock: introduce SOCK_XDPdrivers/net/tun.c include/net/sock.h+20 -04 replies
1/11net: sock: introduce SOCK_XDPdrivers/net/tun.c include/net/sock.h+20 -00 replies
2/11tuntap: switch to use XDP_PACKET_HEADROOMdrivers/net/tun.c+1 -24 replies
2/11tuntap: switch to use XDP_PACKET_HEADROOMdrivers/net/tun.c+1 -20 replies
3/11tuntap: enable bh early during processing XDPdrivers/net/tun.c+3 -72 replies
3/11tuntap: enable bh early during processing XDPdrivers/net/tun.c+3 -70 replies
4/11tuntap: simplify error handling in tun_build_skb()drivers/net/tun.c+16 -218 replies
4/11tuntap: simplify error handling in tun_build_skb()drivers/net/tun.c+16 -210 replies
5/11tuntap: tweak on the path of non-xdp case in tun_build_skb()drivers/net/tun.c+6 -33 replies
5/11tuntap: tweak on the path of non-xdp case in tun_build_skb()drivers/net/tun.c+6 -30 replies
6/11tuntap: split out XDP logicdrivers/net/tun.c+51 -336 replies
6/11tuntap: split out XDP logicdrivers/net/tun.c+51 -330 replies
7/11tuntap: move XDP flushing out of tun_do_xdp()drivers/net/tun.c+3 -14 replies
7/11tuntap: move XDP flushing out of tun_do_xdp()drivers/net/tun.c+3 -10 replies
8/11tun: switch to new type of msg_controldrivers/net/tap.c drivers/net/tun.c drivers/vhost/net.c +1 more+29 -94 replies
8/11tun: switch to new type of msg_controldrivers/net/tap.c drivers/net/tun.c drivers/vhost/net.c +1 more+29 -90 replies
9/11tuntap: accept an array of XDP buffs through sendmsg()drivers/net/tun.c+100 -33 replies
9/11tuntap: accept an array of XDP buffs through sendmsg()drivers/net/tun.c+100 -30 replies
10/11tap: accept an array of XDP buffs through sendmsg()drivers/net/tap.c+71 -24 replies
10/11tap: accept an array of XDP buffs through sendmsg()drivers/net/tap.c+71 -20 replies
11/11vhost_net: batch submitting XDP buffers to underlayer socketsdrivers/vhost/net.c+151 -137 replies
11/11vhost_net: batch submitting XDP buffers to underlayer socketsdrivers/vhost/net.c+151 -130 replies

Reviews trailers per patch

4 of 22 patches have a Reviewed-by or Acked-by.

PatchMichael S. Tsirkin
1/11
1/11
2/11A-b A-b
2/11A-b A-b
3/11A-b A-b
3/11A-b A-b
4/11
4/11
5/11
5/11
6/11
6/11
7/11
7/11
8/11
8/11
9/11
9/11
10/11
10/11
11/11
11/11

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v1 thread

Cover letter [PATCH net-next 00/11] Vhost_net TX batching

Hi all:

This series tries to batch submitting packets to underlayer socket through msg_control during sendmsg(). This is done by:

1) Doing userspace copy inside vhost_net
2) Build XDP buff
3) Batch at most 64 (VHOST_NET_BATCH) XDP buffs and submit them once through msg_control during sendmsg().
4) Underlayer sockets can use XDP buffs directly when XDP is enalbed, or build skb based on XDP buff.

For the packet that can not be built easily with XDP or for the case that batch submission is hard (e.g sndbuf is limited). We will go for the previous slow path, passing iov iterator to underlayer socket through sendmsg() once per packet.

This can help to improve cache utilization and avoid lots of indirect calls with sendmsg(). It can also co-operate with the batching support of the underlayer sockets (e.g the case of XDP redirection through maps).

Testpmd(txonly) in guest shows obvious improvements:

Test /+pps%
XDP_DROP on TAP /+44.8%
XDP_REDIRECT on TAP /+29%
macvtap (skb) /+26%

Netperf TCP_STREAM TX from guest shows obvious improvements on small packet:

    size/session/+thu%/+normalize%
       64/     1/   +2%/    0%
       64/     2/   +3%/   +1%
       64/     4/   +7%/   +5%
       64/     8/   +8%/   +6%
      256/     1/   +3%/    0%
      256/     2/  +10%/   +7%
      256/     4/  +26%/  +22%
      256/     8/  +27%/  +23%
      512/     1/   +3%/   +2%
      512/     2/  +19%/  +14%
      512/     4/  +43%/  +40%
      512/     8/  +45%/  +41%
     1024/     1/   +4%/    0%
     1024/     2/  +27%/  +21%
     1024/     4/  +38%/  +73%
     1024/     8/  +15%/  +24%
     2048/     1/  +10%/   +7%
     2048/     2/  +16%/  +12%
     2048/     4/    0%/   +2%
     2048/     8/    0%/   +2%
     4096/     1/  +36%/  +60%
     4096/     2/  -11%/  -26%
     4096/     4/    0%/  +14%
     4096/     8/    0%/   +4%
    16384/     1/   -1%/   +5%
    16384/     2/    0%/   +2%
    16384/     4/    0%/   -3%
    16384/     8/    0%/   +4%
    65535/     1/    0%/  +10%
    65535/     2/    0%/   +8%
    65535/     4/    0%/   +1%
    65535/     8/    0%/   +3%

Please review.

Thanks

Jason Wang (11):
net: sock: introduce SOCK_XDP
tuntap: switch to use XDP_PACKET_HEADROOM
tuntap: enable bh early during processing XDP
tuntap: simplify error handling in tun_build_skb()
tuntap: tweak on the path of non-xdp case in tun_build_skb()
tuntap: split out XDP logic
tuntap: move XDP flushing out of tun_do_xdp()
tun: switch to new type of msg_control
tuntap: accept an array of XDP buffs through sendmsg()
tap: accept an array of XDP buffs through sendmsg()
vhost_net: batch submitting XDP buffers to underlayer sockets

 drivers/net/tap.c      |  87 +++++++++++++-
 drivers/net/tun.c      | 251 +++++++++++++++++++++++++++++++----------
 drivers/vhost/net.c    | 171 +++++++++++++++++++++++++---
 include/linux/if_tun.h |   7 ++
 include/net/sock.h     |   1 +
 5 files changed, 437 insertions(+), 80 deletions(-)
Signature · 1 line
2.17.1