Skip to content

GPU Direct RDMA (P2P DMA) for Device Private Pages

Yonatan Maman dri-devel linux-mm linux-rdma lkml nouveau 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest4 patches 23 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/4mm/hmm: HMM API for P2P DMA to device zone pagesinclude/linux/hmm.h include/linux/memremap.h mm/hmm.c+37 -016 replies
2/4nouveau/dmem: HMM P2P DMA for private dev pagesdrivers/gpu/drm/nouveau/nouveau_dmem.c+115 -22 replies
3/4IB/core: P2P DMA for device private pagesdrivers/infiniband/core/umem_odp.c+1 -10 replies
4/4RDMA/mlx5: Enabling ATS for ODP memorydrivers/infiniband/hw/mlx5/mlx5_ib.h+3 -30 replies

Reviews trailers per patch

All 4 patches have a Reviewed-by or Acked-by.

PatchGal Shalom
1/4R-b
2/4R-b
3/4R-b
4/4R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v1 thread

Cover letter [PATCH v1 0/4] GPU Direct RDMA (P2P DMA) for Device Private Pages

From: Yonatan Maman <[email protected]>

This patch series aims to enable Peer-to-Peer (P2P) DMA access in GPU-centric applications that utilize RDMA and private device pages. This enhancement is crucial for minimizing data transfer overhead by allowing the GPU to directly expose device private page data to devices such as NICs, eliminating the need to traverse system RAM, which is the native method for exposing device private page data.

To fully support Peer-to-Peer for device private pages, the following changes are proposed:

Memory Management (MM)
* Leverage struct pagemap_ops to support P2P page operations: This
modification ensures that the GPU can directly map device private pages for P2P DMA.
* Utilize hmm_range_fault to support P2P connections for device private
pages (instead of Page fault)

IB Drivers
Add TRY_P2P_REQ flag for the hmm_range_fault call: This flag indicates the need for P2P mapping, enabling IB drivers to efficiently handle P2P DMA requests.

Nouveau driver
Add support for the Nouveau p2p_page callback function: This update integrates P2P DMA support into the Nouveau driver, allowing it to handle P2P page operations seamlessly.

MLX5 Driver
Optimize PCI Peer-to-Peer for private device pages, by enabling Address Translation service(ATS) for ODP memory.

Yonatan Maman (4):
mm/hmm: HMM API for P2P DMA to device zone pages nouveau/dmem: HMM P2P DMA for private dev pages IB/core: P2P DMA for device private pages RDMA/mlx5: Enabling ATS for ODP memory

 drivers/gpu/drm/nouveau/nouveau_dmem.c | 117 ++++++++++++++++++++++++-
 drivers/infiniband/core/umem_odp.c     |   2 +-
 drivers/infiniband/hw/mlx5/mlx5_ib.h   |   6 +-
 include/linux/hmm.h                    |   2 +
 include/linux/memremap.h               |   7 ++
 mm/hmm.c                               |  28 ++++++
 6 files changed, 156 insertions(+), 6 deletions(-)
Signature · 1 line
2.34.1