Skip to content

RFC macvtap TX zero copy between guest and host kernel

Shirley Ma kvm lkml netdev 2 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v2 RFC Latest5 patches 7 repliesIn review
  2. v1 RFC1 of 2 patches 42 repliesSuperseded

Patches in v2 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/5Add a new sock flag for zero-copyinclude/net/sock.h+1 -00 replies
2/5Add a new device flag for zero copyinclude/linux/netdevice.h+3 -00 replies
3/5Add userspace buffers callback in skb_share_infoinclude/linux/skbuff.h net/core/skbuff.c+24 -00 replies
4/5Add vhost zero copy callback to release guest kernel buffersdrivers/vhost/net.c drivers/vhost/vhost.c drivers/vhost/vhost.h+75 -10 replies
5/5Add TX zero copy in macvtapdrivers/net/macvtap.c+232 -244 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v2 yet.

Read the v2 thread

Cover letter [RFC PATCH V2 0/5] macvtap TX zero copy between guest and host kernel

This patchset add supports for TX zero-copy between guest and host kernel through vhost. It significantly reduces CPU utilization on the local host on which the guest is located (It reduced 30-50% CPU usage for vhost thread for single stream test). The patchset is based on previous submission and comments from the community regarding when/how to handle guest kernel buffers to be released. This is the simplest approach I can think of after comparing with several other solutions.

This patchset includes:

1. Induce a new sock zero-copy flag, SOCK_ZEROCOPY;

2. Induce a new device flag, NETIF_F_ZEROCOPY for device can support zero-copy;

3. Add a new struct skb_ubuf_info in skb_share_info for userspace buffers release callback when device DMA has done for that skb;

4. Add vhost zero-copy callback in vhost when skb last refcnt is gone; add vhost_zerocopy_add_used_and_signal to notify guest to release TX
skb buffers.

5. Add macvtap zero-copy in lower device when sending packet is greater than 128 bytes.

The patchset has passed netperf/netserver test on Chelsio, and continuing test on other 10GbE NICs, like Intel ixgbe, Mellanox mlx4... I will provide guest to host, host to guest performance data next week.

However when running stress test, vhost & virtio_net seems out of sync, and virito_net interrupt was disabled somehow, and it stopped to send any packet. This problem has bothered me for a long long time, I will continue to look at this.

Please review this.

Thanks
Shirley