ovpn: construct and process GRO batches

Hi,

These patches were initially included in a joint GSO-GRO series [1] implementing both TX and RX batching mechanisms in the ovpn data path. Sashiko's review of v2 of that series identified fundamental flaws in the GRO part which required a broad redesign of the new RX path. So I decided to proceed with the GSO series separately [2] and keep this series GRO-specific since the two mechanisms are not functionally dependent. The series applies both on its own and on top of [2].

The redesign uses the standard SKB_GSO_UDP_L4 | SKB_GSO_FRAGLIST representation instead of giving an ovpn aggregate tunnel-specific completion semantics. This requires UDP to let an encapsulation callback opt into consuming that representation, while retaining a segment-and-redeliver fallback when the callback is absent or declines it. It also requires detachable GRO callbacks: unlike a conventional kernel-owned tunnel socket, ovpn attaches to a userspace socket which can outlive its association with the driver. The first patch fixes a pre-existing UDP frag-list GRO checksum problem on CHECKSUM_COMPLETE devices which the new receive path would otherwise hit; it carries a Fixes tag and does not depend on the rest of the series. The next three provide those core contracts and make UDP_ENCAP changes retire detachable callbacks before replacing the encapsulation state. Clearly, these four patches are more suited to netdev than openvpn-devel, but in order to keep this series applicable and reviewable, they are included in this openvpn-devel-targeted series. Patch 2 explains why this opt-in does not reopen commit 3d010c8031e3 and shows that consuming the aggregate directly is 22.7% to 25.2% faster than letting UDP segment it first.

On receive, the ovpn GRO callback collects compatible DATA_V2 records from one outer transport flow, matching their opcode, key and peer header while retaining every complete encrypted record as a separate frag-list entry. The aggregate continues through the ordinary outer receive stack. At the encapsulation callback, ovpn validates its layout, ownership, headers and outer tuple before detaching the records, propagating post-GRO metadata and authenticating each one independently. Any aggregate which ovpn cannot accept whole, such as a foreign or modified one, is declined, causing UDP to segment it and retry delivery one datagram at a time. The final patch uses the resulting lookahead to prefetch ciphertext two records before it is decrypted.

Performance was measured with the GSO series [2] applied solely to keep transmit from limiting the receive comparison. Five interleaved 60-second iperf3 -Z single-flow AES-128-GCM runs in each direction, after a 10-second warm-up, over two directly connected 100-Gbit/s mlx5 ports produced:

Forward Reverse

Without GRO             18.721 Gbit/s    18.777 Gbit/s
With GRO                21.542 Gbit/s    21.366 Gbit/s
With GRO + RX prefetch  25.741 Gbit/s    25.150 Gbit/s

The complete series improved throughput by 37.5% forward and 33.9% reverse. Across the monitored data-path CPUs, mean cycles per byte fell from 4.535 to 3.896 (-14.1%) and instructions per byte from 10.913 to 9.492 (-13.0%). Individual patch messages contain the incremental measurements and further test details.

Thanks,

Ralf Lici
Mandelbit Srl

[1] https://lore.kernel.org/openvpn-devel/[email protected]/
[2] https://lore.kernel.org/openvpn-devel/[email protected]/

---
Changes since v2 https://lore.kernel.org/openvpn-devel/[email protected]/
- Point the Fixes tag of the checksum fix at the commit which stopped
converting CHECKSUM_COMPLETE, and explain why only the checksum state
can carry the GRO verdict. (Sashiko)
- Restore the pre-encapsulation checksum bypass for accepted aggregates,
since a tunnel can reset their checksum state. (Sashiko)
- Refuse records inside FOU or GUE in the ovpn GRO callback. (Sashiko)
- Rebase on current ovpn-net-next.

Changes since v1 https://lore.kernel.org/openvpn-devel/[email protected]/
- Add a UDP fix which turns CHECKSUM_COMPLETE frag-list aggregates into CHECKSUM_UNNECESSARY at GRO completion, and drop the pre-encapsulation checksum bypass which it makes redundant. (Sashiko)
- Run the inbound XFRM policy check and conntrack reset once for an
accepted aggregate, so that a declined one is not checked again
without that state, and count every datagram of a dropped aggregate.
(Sashiko)
- Read frag-list acceptance with acquire ordering and warn when setting
up a socket which still accepts frag-lists. (Sashiko)
- Require a detachable GRO socket to have no registered GRO callback,
and refuse to attach ovpn to a socket which another encapsulation has configured. (Sashiko)
- Read a held aggregate's OpenVPN header without pulling it, and never
pull an aggregate before deciding whether to consume it. (Sashiko)
- Clear the redirect, classification and delivery time state which the
receive stack consumed on the aggregate head only. (Sashiko)

Ralf Lici (6):
udp: keep the GRO checksum verdict for CHECKSUM_COMPLETE frag-lists
udp: let encapsulation callbacks consume UDP frag-list GSO
udp_tunnel: support detachable GRO receive callbacks
udp: detach tunnel GRO before changing UDP encapsulation
ovpn: coalesce UDP data records with GRO
ovpn: prefetch encrypted records before GRO batch decryption

 drivers/net/ovpn/udp.c     | 368 +++++++++++++++++++++++++++++++++++--
 include/linux/udp.h        |  18 ++
 include/net/udp_tunnel.h   |   3 +
 net/core/gro.c             |   1 +
 net/ipv4/udp.c             |  79 ++++++--
 net/ipv4/udp_offload.c     |  67 ++++++-
 net/ipv4/udp_tunnel_core.c |  37 +++-
 net/ipv6/udp.c             |  73 ++++++--
 net/ipv6/udp_offload.c     |   8 +
 9 files changed, 608 insertions(+), 46 deletions(-)

base-commit: 7b6ab4dbc156e2f96419715130b3d5738a306c93

2.56.0



_______________________________________________
Openvpn-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/openvpn-devel