Skip to content

RDMA/mana_ib: Streamline kernel UD/GSI posting and completion handling

Konstantin Taranov linux-rdma lkml 4 versions – Applied

Status from this message: Applied, thanks!

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v4 Latest5 patches 1 replyApplied
  2. v35 patches 0 repliesSuperseded
  3. v25 patches 0 repliesSuperseded
Earlier versions · 1
VersionPostedPatchesRepliesStatus
v15 patches1 replySuperseded

Changes in v4 from the cover letter

- re-write all commit messages

Patches in v4 Applied

#Subject and filesSizeRepliesApplied
1/5RDMA/mana_ib: Optimize shadow queue bookkeepingdrivers/infiniband/hw/mana/cq.c drivers/infiniband/hw/mana/qp.c drivers/infiniband/hw/mana/shadow_queue.h +1 more+40 -430 replies
2/5RDMA/mana_ib: Revise UD send posting and WQE definitionsdrivers/infiniband/hw/mana/mana_ib.h drivers/infiniband/hw/mana/wr.c+72 -260 replies
3/5RDMA/mana_ib: Revise UD receive posting with GDMA_WR_IB_SGLdrivers/infiniband/hw/mana/wr.c drivers/net/ethernet/microsoft/mana/gdma_main.c include/net/mana/gdma.h+70 -300 replies
4/5RDMA/mana_ib: Make kernel CQ arming robustdrivers/infiniband/hw/mana/cq.c drivers/infiniband/hw/mana/mana_ib.h drivers/net/ethernet/microsoft/mana/gdma_main.c +1 more+72 -60 replies
5/5RDMA/mana_ib: Poll UD completions and flush software error QPsdrivers/infiniband/hw/mana/cq.c drivers/infiniband/hw/mana/mana_ib.h drivers/infiniband/hw/mana/qp.c +4 more+359 -1480 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v4 yet.

Read the v4 thread

Cover letter [PATCH rdma-next v4 0/5] RDMA/mana_ib: Streamline kernel UD/GSI posting and completion handling

From: Konstantin Taranov <[email protected]>

This series updates the kernel UD/GSI work-request and completion paths in the MANA RDMA driver. This series is required for upcoming kernel RC QP support.

The main idea is to fill in WCs directly from CQEs and do not use shadow Qs as proxy. The older code, a CQ had lists of QPs using it, which adds overhead on polling when CQs are shared. Instead, CQs now maintain only the list of QPs in error state.

The series also revises posting WQEs to reuse it for RC QPs in the next series. We also add GDMA_WR_IB_SGL flag for posting to remove sge copying for RQs. The RC QPs will use it for SQs and RQs.

Imporve CQ arming to avoid doorbell drops in hw. MANA hw requires occasional arming to update tail offest in hw.

Changes in v4:
- re-write all commit messages

Changes in v3:
- Patch 5: Track unsignaled send WRs in the shadow queue.
- Patch 5: Validate the CQE target before retiring a shadow wqe.
- Patch 5: Notify the send CQ after posting to an already-flushing QP, outside the CQ lock, so GSI draining does not need another hardware CQE.
- Patches 1-4 are unchanged.

Changes in v2:
- Patch 5: Hold the corresponding CQ lock while resetting each shadow queue and removing the QP from the error list, serializing RESET with hardware polling and software flushing.
- Patch 5: Return the unmasked WQE start offset from GDMA posting to discard mismatching CQEs without producing a WC.
- Patches 1-4 are unchanged.

Konstantin Taranov (5):
RDMA/mana_ib: Optimize shadow queue bookkeeping RDMA/mana_ib: Revise UD send posting and WQE definitions RDMA/mana_ib: Revise UD receive posting with GDMA_WR_IB_SGL RDMA/mana_ib: Make kernel CQ arming robust RDMA/mana_ib: Poll UD completions and flush software error QPs

 drivers/infiniband/hw/mana/cq.c               | 344 +++++++++++++-----
 drivers/infiniband/hw/mana/mana_ib.h          | 105 +++++-
 drivers/infiniband/hw/mana/qp.c               |  84 +++--
 drivers/infiniband/hw/mana/shadow_queue.h     |  61 ++--
 drivers/infiniband/hw/mana/wr.c               | 158 ++++----
 .../net/ethernet/microsoft/mana/gdma_main.c   |  49 ++-
 include/net/mana/gdma.h                       |  11 +-
 7 files changed, 586 insertions(+), 226 deletions(-)

base-commit: 63367df6e7255067ad6a83abe0d2799dfa491876

Signature · 1 line
2.43.0