ovpn: preserve GSO batching

Hi,

This series is a respin of earlier work that combined GSO and GRO batching across ovpn's data path. After submitting v2, Sashiko demolished the GRO part by pointing out various lifetime and ownership issues that both I and previous Sashiko review rounds had missed. So this series now carries only the GSO patches, with the TX-specific findings addressed. The GRO work will be submitted separately since there's no functional dependency between the two batching mechanisms.

Another major difference from v2 is the exclusion of asynchronous crypto backends from the new GSO data path. This proved necessary after carefully re-reading the DMA API documentation and realizing that adjacent records in the output UDP GSO skb can share cache lines. Independently mapped asynchronous DMA operations cannot safely use that layout on non-coherent systems. Keeping asynchronous transforms on the existing per-record path also addresses Sabrina's concern about preserving aggregate lifetime across asynchronous crypto completions.

This series lets GSO skbs reach ovpn, completes each inner packet's checksum during software segmentation and, when the layout permits, encrypts the resulting packets directly into a UDP GSO aggregate. Ordinary packets, GSO_BY_FRAGS input, unsupported segment geometries and traffic using asynchronous crypto retain the existing in-place encryption path. Packet IDs for a UDP GSO batch are reserved as a single range instead of one segment at a time. The last two patches also construct each record's AAD directly in its output range when possible and reuse one AEAD request and its scratch storage across the synchronous batch.

Patch 5 changes net/ipv4/udp_offload.c and needs review from the UDP GSO maintainers when the series is forwarded to netdev.

Performance was measured with iperf3 -Z using one TCP flow over UDP ovpn with AES-128-GCM, a 1408-byte tunnel MTU and a 1500-byte physical MTU between two directly connected 100-Gbit/s mlx5 ports. Each 25-second run omitted the first 5 seconds, with a 15-second perf sample covering the data-path CPUs. Six runs of each tested commit were interleaved round-robin. Comparing the state immediately before patch 6 with the final series produced:

Without UDP GSO Final series
Throughput 13.431 Gbit/s 16.932 Gbit/s (+26.1%)
Cycles per byte 6.628 4.680 (-29.4%)
Instructions per byte 13.871 10.354 (-25.4%)

Once UDP GSO output was introduced, throughput remained on the same roughly 17 Gbit/s plateau within run-to-run variation. The subsequent patches reduced data-path cost from 4.899 to 4.680 cycles per byte (-4.5%) and from 10.653 to 10.354 instructions per byte (-2.8%).

To verify that this was not only a hardware-segmentation win, the main conversion was also tested in four order-balanced pairs with tx-udp-segmentation disabled on the lower device. Mean throughput rose from 13.647 to 16.278 Gbit/s (+19.3%) and cycles per byte fell from 6.609 to 6.199 (-6.2%), despite instructions per byte increasing from 13.858 to 14.582 (+5.2%) because the lower device segments the aggregate in software. The patches making individual performance claims carry their corresponding measurements.

Thanks,

Ralf Lici
Mandelbit Srl

---
Changes since v4 https://lore.kernel.org/openvpn-devel/[email protected]/
- Transfer write accounting from successfully encrypted source segments
to the replacement aggregate without running the source destructors,
so TCP Small Queues are not woken before the batch leaves the host.
(Sashiko)
- Preserve TX timestamp requests, neighbour-confirmation state and
security marks when replacing input segments.

Changes since v3 https://lore.kernel.org/openvpn-devel/[email protected]/
- Preserve socket write ownership across software UDP GSO segmentation
for every destructor recognized by is_skb_wmem(), and only coalesce
input segments with a consistent owner and destructor. (Sashiko)
- Reserve LL_MAX_HEADER when sizing output UDP GSO aggregates so that
they remain below the lower device's GSO size limit after its link
header is pushed. (Sashiko)

Changes since v2 https://lore.kernel.org/openvpn-devel/[email protected]/
- Drop the GRO and receive-prefetch patches; they will be submitted as a
separate series.
- Drop transmit prefetch after new measurements found no repeatable
benefit.
- Restrict UDP GSO output to synchronous AEAD transforms.
- Let fixed-size frag-list GSO input use the UDP GSO output path; only GSO_BY_FRAGS input remains on the in-place path.
- Replace asynchronous batch-completion machinery with serial
synchronous encryption, and reuse one AEAD request and its scratch
storage per batch.
- Count peer packet statistics for every segment which survives GSO
segmentation rather than once for the original GSO skb. (Sashiko)
- Transfer socket write ownership with the input skb's original
destructor so TCP Small Queues wakeups remain correct. (Sashiko)
- Preserve timestamp and ooo_okay metadata on the replacement aggregate.
- Reset the replacement skb's inner network header so tunnel byte
accounting excludes reserved headroom, and document that device
tx_packets counts output GSO skbs. (Sashiko)
- Use an unaligned-safe store when constructing AAD directly in the
output aggregate. (Sashiko)

Changes since v1 https://lore.kernel.org/openvpn-devel/[email protected]/
- Release successfully encrypted GSO source skbs with consume_skb after
their data has been transferred to the output aggregate. (Antonio)
- Complete a held GRO aggregate before delivering an unmergeable record,
and let a DATA_V2 record with a changed header start the next
aggregate. (Antonio)
- Document that RX prefetch primarily targets synchronous software AEAD; asynchronous transforms are not expected to benefit in the same way.
- Add direct placement of record AAD in the output aggregate, retaining
the scratch-and-copy path as a fallback.

Changes since RFC v4 https://lore.kernel.org/openvpn-devel/[email protected]/
- Drop the opt-in direct GRO mode and the generic GRO ownership change
it required. The series now retains the complete outer receive stack.
- Preserve safe page-backed payload fragments during GSO segmentation
and map them directly into the AEAD source scatterlist.
- Reserve one consecutive packet-ID range for each UDP GSO batch instead
of updating the counter once per segment.
- Extend transmit prefetch to the first payload fragment of segments
encrypted into a UDP GSO aggregate.
- Refresh the GSO, GRO and prefetch measurements using two directly
connected 100-Gbit/s mlx5 ports.

Ralf Lici (9):
ovpn: advertise checksum offload for GSO packets
ovpn: accept frag-list GSO input
ovpn: refactor AEAD encryption helpers
ovpn: record asynchronous transmit crypto
udp: preserve write ownership across GSO segmentation
ovpn: convert GSO input into UDP GSO output
ovpn: reserve packet-ID ranges for UDP GSO batches
ovpn: place GSO record AAD directly in output
ovpn: reuse one AEAD request per UDP GSO batch

 drivers/net/ovpn/crypto.h      |   2 +
 drivers/net/ovpn/crypto_aead.c | 226 +++++++++++++++----
 drivers/net/ovpn/crypto_aead.h |  21 ++
 drivers/net/ovpn/io.c          | 397 ++++++++++++++++++++++++++++-----
 drivers/net/ovpn/io.h          |   3 +-
 drivers/net/ovpn/main.c        |   5 +-
 drivers/net/ovpn/pktid.h       |  48 ++++
 drivers/net/ovpn/proto.h       |   4 +
 drivers/net/ovpn/stats.h       |  16 +-
 drivers/net/ovpn/tcp.c         |   4 +-
 drivers/net/ovpn/udp.c         |  26 ++-
 net/ipv4/udp_offload.c         |   7 +-
 12 files changed, 652 insertions(+), 107 deletions(-)

base-commit: 734bef548a96e8774d6eef5dfa2026c6edc4e80b

2.56.0



_______________________________________________
Openvpn-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/openvpn-devel