Skip to content

net: mana: Avoid DMA queue allocation failure under memory fragmentation

Aditya Garg linux-hyperv linux-rdma lkml netdev 1 version Applied

Status from this message: This series was applied to netdev/net-next.git (main)

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest2 patches 2 repliesApplied 2/2

Patches in v1 All 2 applied

#Subject and filesSizeRepliesApplied
1/2net: mana: Route ring-buffer access through offset-based helpersdrivers/net/ethernet/microsoft/mana/gdma_main.c+39 -240 repliesApplied
2/2net: mana: Fall back to scattered pages for GDMA queuesdrivers/net/ethernet/microsoft/mana/gdma_main.c drivers/net/ethernet/microsoft/mana/hw_channel.c drivers/net/ethernet/microsoft/mana/mana_en.c +1 more+168 -160 repliesApplied

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH net-next 0/2] net: mana: Avoid DMA queue allocation failure under memory fragmentation

The MANA driver can fail to bring up its queues on systems with high memory utilization because every GDMA queue ring is allocated as a single dma_alloc_coherent() of the whole power-of-2 ring size. Under memory fragmentation these high-order allocations may fail, preventing the driver from creating queues when opening the interface, after a VF reset, or when reconfiguring channels, ring parameters or MTU.

Per-queue sizes that are problematic, with depth and size given as (default, max) over the ethtool ring settings:

ring entry depth size
------------------------------------------------------------

  TX completion queue   64 B   (256, 16384)   (16 KB, 1024 KB)
  TX send queue         32 B   (256, 16384)   ( 8 KB,  512 KB)
  RX completion queue   64 B   (1024, 8192)   (64 KB,  512 KB)
  RX receive queue      32 B   (1024, 8192)   (32 KB,  256 KB)
  event queue           16 B   2048 (fixed)   32 KB

This series addresses the issue by:
1. Routing all CPU-side ring access through mana_gd_ring_ptr() and mana_gd_ring_contig_avail(). On a contiguous ring these reduce to simple arithmetic, so this patch is a pure refactor.
2. Falling back in mana_gd_alloc_memory() to a vector of scattered order-0 coherent pages when the contiguous allocation fails. The device sees the same page-list format either way, as mana_gd_create_dma_region() already describes a ring as a list of MANA_PAGE_SIZE addresses. The HW channel stays contiguous, as advertising a scattered page list needs the HW channel itself.

Throughput testing confirms no regression. Since the fallback only triggers under memory fragmentation, the scattered-page path was enabled unconditionally for all eligible GDMA queue rings during testing (iperf3, Gbit/s):

Baseline Patched Patched
Connections Contiguous Contiguous Scattered
-----------------------------------------------

  1                  46.1        46.2       46.1
  16                 182         182        182
  32                 182         182        182
  64                 182         182        182

Aditya Garg (2):
net: mana: Route ring-buffer access through offset-based helpers
net: mana: Fall back to scattered pages for GDMA queues

 .../net/ethernet/microsoft/mana/gdma_main.c   | 219 +++++++++++++++---
 .../net/ethernet/microsoft/mana/hw_channel.c  |   2 +-
 drivers/net/ethernet/microsoft/mana/mana_en.c |   3 +
 include/net/mana/gdma.h                       |  19 +-
 4 files changed, 205 insertions(+), 38 deletions(-)
Signature · 1 line
2.43.0