RFC xdp: allow disable GRO per packet by XDP

Software GRO is currently controlled by a single switch, i.e.

ethtool -K dev gro on|off

However, this is not always desired. When GRO is enabled, even if the kernel cannot GRO certain traffic, it has to run through the GRO receive handlers with no benefit.

There are also scenarios that turning off GRO is a requirement. For example, our production environment has a scenario that a TC egress hook may add multiple encapsulation headers to forwarded skbs for load balancing and isolation purpose. The encapsulation is implemented via BPF. But the problem arises then: there is no way to properly offload a double-encapsulated packet, since skb only has network_header and inner_network_header to track one layer of encapsulation, but not two. On the other hand, not all the traffic through this device needs double encapsulation. But we have to turn off GRO completely for any ingress device as a result.

A natural approach to make this more flexible is to use XDP to control GRO behavior. But current semantic gap between XDP buffer/frame and socket buffer requires some new primitives.

This change set proposes a control bit gro_disabled on skbs to determine if GRO should work on an skb or not. To manipulate this bit, we introduce a new XDP kfunc bpf_xdp_disable_gro as well as generic helpers xdp_frame/buff_fixup_skb_offloading.

The expected working flow is that:
* XDP program examines packets and can call bpf_xdp_disable_gro to disable GRO on a packet
* Device drivers need to call xdp_buff_fixup_skb_offloading (or equivalent version for xdp_frame), to check if skb->gro_disabled needs to be set.
* The kernel will elide GRO later if this bit is used.

Initially we only modified a few drivers for demonstration purpose. The driver side changes is optional and also incremental depending on vendors' agenda. Any suggestions are welcome!

Jesper Dangaard Brouer (1):
mlx5: move xdp_buff scope one level up

Yan Zhai (8):
skb: introduce gro_disabled bit
xdp: add XDP_FLAGS_GRO_DISABLED flag
xdp: implement bpf_xdp_disable_gro kfunc
bnxt: apply XDP offloading fixup when building skb
ice: apply XDP offloading fixup when building skb
veth: apply XDP offloading fixup when building skb
mlx5: apply XDP offloading fixup when building skb
bpf: selftests: test disabling GRO by XDP

 drivers/net/ethernet/broadcom/bnxt/bnxt.c     |   4 +
 drivers/net/ethernet/intel/ice/ice_txrx.c     |   2 +
 drivers/net/ethernet/intel/ice/ice_xsk.c      |   6 +-
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |   6 +-
 .../ethernet/mellanox/mlx5/core/en/xsk/rx.c   |  10 +-
 .../ethernet/mellanox/mlx5/core/en/xsk/rx.h   |   6 +-
 .../net/ethernet/mellanox/mlx5/core/en_rx.c   | 117 ++++++++++-------
 drivers/net/veth.c                            |   4 +
 include/linux/netdevice.h                     |   9 +-
 include/linux/skbuff.h                        |  10 ++
 include/net/xdp.h                             |  38 +++++-
 include/net/xdp_sock_drv.h                    |   2 +-
 net/Kconfig                                   |  10 ++
 net/core/gro.c                                |   2 +-
 net/core/gro_cells.c                          |   2 +-
 net/core/skbuff.c                             |   4 +
 net/core/xdp.c                                |  27 +++-
 tools/testing/selftests/bpf/config            |   1 +
 .../selftests/bpf/prog_tests/xdp_offloading.c | 122 ++++++++++++++++++
 .../selftests/bpf/progs/xdp_offloading.c      |  50 +++++++
 20 files changed, 369 insertions(+), 63 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/xdp_offloading.c
 create mode 100644 tools/testing/selftests/bpf/progs/xdp_offloading.c

2.30.2