Skip to content

net/mlx5: enable sub-page allocations for mlx5_frag_buf

Tariq Toukan linux-rdma lkml netdev 2 versions – Applied

Status from this message: This series was applied to netdev/net-next.git (main)

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v2 Latest3 patches 0 repliesApplied 3/3
  2. v13 patches 1 replySuperseded

Changes in v2 from the cover letter

- Removed defensive WARN_ONCE() checks per Leon's review.
- Link to V1:
https://lore.kernel.org/r/[email protected]/

Patches in v2 All 3 applied

#Subject and filesSizeRepliesApplied
1/3net/mlx5: wire frag buf pools lifecycle hooksdrivers/net/ethernet/mellanox/mlx5/core/alloc.c drivers/net/ethernet/mellanox/mlx5/core/main.c drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h+20 -00 repliesApplied
2/3net/mlx5: add frag buf pools create/destroy pathsdrivers/net/ethernet/mellanox/mlx5/core/alloc.c include/linux/mlx5/driver.h+119 -40 repliesApplied
3/3net/mlx5: use internal dma pools for frag buf allocdrivers/net/ethernet/mellanox/mlx5/core/alloc.c include/linux/mlx5/driver.h+140 -320 repliesApplied

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v2 yet.

Read the v2 thread

Cover letter [PATCH net-next V2 0/3] net/mlx5: enable sub-page allocations for mlx5_frag_buf

Hi,

See detailed description by Nimrod below [1].

Regards,
Tariq

[1]
This series aims to improve memory utilization for DMA-coherent fragmented-buffer allocations on systems with large PAGE_SIZE.

Before this change, such allocations were page-granular, as they were backed by full pages. On large-page systems this caused significant internal waste for small objects. For example, a single 4K request consumed an entire 64K page.

The common kernel solution for sub-page coherent DMA allocations is the DMA pool API. However, those pools do not return pages to the system until teardown. That behavior is not a good fit for mlx5_frag_buf allocations, since they back interface resources (WQs and CQs). Interfaces may be removed dynamically, so their memory footprint should reflect live usage to avoid situations where large amounts of memory remain tied up in pools.

This series introduces a lightweight mlx5-local pool implementation for sub-page coherent DMA allocations, which immediately returns free backing pages. It wires mlx5_frag_buf allocations to use these internal pools, while keeping the mechanism reusable for other mlx5-internal coherent DMA allocation users in follow-up work.

V2:
- Removed defensive WARN_ONCE() checks per Leon's review.
- Link to V1:
https://lore.kernel.org/r/[email protected]/

Nimrod Oren (3):
net/mlx5: wire frag buf pools lifecycle hooks net/mlx5: add frag buf pools create/destroy paths net/mlx5: use internal dma pools for frag buf alloc

 .../net/ethernet/mellanox/mlx5/core/alloc.c   | 293 ++++++++++++++++--
 .../net/ethernet/mellanox/mlx5/core/main.c    |   7 +
 .../ethernet/mellanox/mlx5/core/mlx5_core.h   |   2 +
 include/linux/mlx5/driver.h                   |   9 +-
 4 files changed, 277 insertions(+), 34 deletions(-)

base-commit: 09942ddedcb960f9e78fd817ec33f501d1040c5b

Signature · 1 line
2.44.0