RDMA/mana_ib: Add kernel RC and fast registration support
Status from this message: Applied, thanks!
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v11 Latest9 patches 2 repliesApplied
- v109 patches 2 repliesSuperseded
- v99 patches 3 repliesSuperseded
Earlier versions · 8
| Version | Posted | Patches | Replies | Status |
|---|---|---|---|---|
| v8 | 9 patches | 0 replies | Superseded | |
| v7 | 9 patches | 0 replies | Superseded | |
| v6 | 10 patches | 0 replies | Superseded | |
| v5 | 10 patches | 1 reply | Superseded | |
| v4 | 10 patches | 0 replies | Superseded | |
| v3 | 10 patches | 0 replies | Superseded | |
| v2 | 10 patches | 0 replies | Superseded | |
| v1 | 10 patches | 0 replies | Superseded |
Changes in v11 from the cover letter
- Patch 09: Reset kernel QPs before destruction unless already in RESET or INIT, and always clean CQEs. Keep shared-CQ cleanup under one uninterrupted CQ lock.
Patches in v11 Applied
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/9 | RDMA/mana_ib: Allocate and map fast-registration MRsdrivers/ drivers/ drivers/ +2 more | +150 | 0 replies | |
| 2/9 | RDMA/mana: Create and destroy kernel RC QPsdrivers/ drivers/ drivers/ | +167 | 0 replies | |
| 3/9 | net/mana: Extend GDMA encoding for new RDMA WQEsdrivers/ include/ | +63 | 0 replies | |
| 4/9 | RDMA/mana_ib: Maintain kernel RC QP statedrivers/ drivers/ include/ | +27 | 0 replies | |
| 5/9 | RDMA/mana_ib: Post receive WRs on kernel RC QPsdrivers/ | +37 | 0 replies | |
| 6/9 | RDMA/mana_ib: Post send and memory-management WRs on RC QPsdrivers/ drivers/ drivers/ | +350 | 0 replies | |
| 7/9 | RDMA/mana_ib: Poll RC completions using PSN and FSN progressdrivers/ drivers/ drivers/ +1 more | +203 | 0 replies | |
| 8/9 | RDMA/mana_ib: Flush and notify CQs when kernel QPs enter ERRdrivers/ drivers/ drivers/ | +105 | 0 replies | |
| 9/9 | RDMA/mana_ib: Handle error CQEs for kernel QPsdrivers/ drivers/ drivers/ | +311 | 1 reply |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v11 yet.
Cover letter [PATCH rdma-next v11 0/9] RDMA/mana_ib: Add kernel RC and fast registration support
From: Konstantin Taranov <[email protected]>
Add kernel RC QPs and fast-registration MRs to the MANA RDMA driver for kernel consumers such as NVMe/RDMA and NFS/RDMA.
Implement QP creation and state management, work request posting, completion processing, and error handling. Use the RDMA core's generic queue drains for kernel RC QPs, with software-flush notification in the posting paths. Kernel RC QPs require extended-WQE and power-of-two-SQ hardware support.
v11:
- Patch 09: Reset kernel QPs before destruction unless already in RESET or INIT, and always clean CQEs. Keep shared-CQ cleanup under one uninterrupted CQ lock.
v10:
- Patch 01: Remove the FMR page-count limit check
- Patch 02: Remove the minimum firmware send-SGE check, kernel QP queue prevalidation
- Patch 06: Remove send-SGE and READ-SGE limit checks and message- length overflow validation
v9:
- Patch 07 and 09: Retire acknowledged successful unsignaled WQEs with a full WC array to avoid deferring rearming.
v8:
- Patch 06: Annotate rc_send.immediate and immediate_ext as __le32
v7:
- Patch 09: Include GSI QPs in deferred ERR handling.
- Patch 09: Complete software-flush setup even if the deferred firmware ERR request fails. An error CQE guarantees whole-QP DMA shutdown for all QP types.
- Drop patch 10 to use the generic SQ/RQ drains for RC.
v6:
- Patch 08: Notify software flushes after posting to error-listed QPs. Add reporting pending software flush for IB_CQ_REPORT_MISSED_EVENTS.
- Patch 09: Check fenced WQEs on error cqes to prevent flushing completed WQEs. Add mana_ib_cq_clean on QP reset to discard stale CQEs. QID 0 is an invalid value for RNIC and safe dor discarding. Add error_pending flag to prevent a deffered ERR request to error reset QPs.
v5:
- Patch 01: Reject max_num_sg above MANA_MAX_FMR_PAGES.
- Patch 02: Validate kernel QP queue sizes and SGE limits to prevent overflows. Ensure that queue size cannnot be 0.
- Patch 06: Preserve fence flags on unsignaled WQEs. Check message length and read sges before posting a wqe.
- Patch 08: Remove the unused QP state field.
v4:
- Patch 01: Use an explicit 64-byte FMR MST alignment
- Patch 02: Validate that the RNIC firmware reports at least two send SGEs and subtract the reserved SGE only for RNIC-based mana_ib.
- Patch 07: Add missing wqe_size to shadow wqe for SEND/Write, and return a missed-event hint for cached CQEs when CQ is armed
v3:
- Patch 07: Mask arm_psn to 24 bits before ringing the doorbell.
- Patch 08: Exclude userspace QPs from kernel software-state handling and CQ error-list insertion, and unlink UC QPs during destruction.
v2:
- Patch 03: Validate fixed-WQE capacity for the header, OOB and SGEs.
- Patch 04: Preserve UC QP modification support.
- Patch 08: Skip software CQ notifications for direct-polling CQs.
- Patch 09: Flush that ERR work during QP teardown.
- Patch 10: Limit drain callbacks to RC QPs and avoid completion callbacks for direct-polling CQs.
Konstantin Taranov (9):
RDMA/mana_ib: Allocate and map fast-registration MRs RDMA/mana: Create and destroy kernel RC QPs net/mana: Extend GDMA encoding for new RDMA WQEs RDMA/mana_ib: Maintain kernel RC QP state RDMA/mana_ib: Post receive WRs on kernel RC QPs RDMA/mana_ib: Post send and memory-management WRs on RC QPs RDMA/mana_ib: Poll RC completions using PSN and FSN progress RDMA/mana_ib: Flush and notify CQs when kernel QPs enter ERR RDMA/mana_ib: Handle error CQEs for kernel QPs
drivers/infiniband/hw/mana/cq.c | 356 ++++++++++++++++++-- drivers/infiniband/hw/mana/device.c | 2 + drivers/infiniband/hw/mana/main.c | 8 +- drivers/infiniband/hw/mana/mana_ib.h | 142 +++++++- drivers/infiniband/hw/mana/mr.c | 84 ++++- drivers/infiniband/hw/mana/qp.c | 332 +++++++++++++++++-- drivers/infiniband/hw/mana/shadow_queue.h | 22 ++ drivers/infiniband/hw/mana/wr.c | 423 +++++++++++++++++++++++- drivers/net/ethernet/microsoft/mana/gdma_main.c | 55 ++- include/net/mana/gdma.h | 38 ++- 10 files changed, 1386 insertions(+), 76 deletions(-)
Signature · 1 line
2.43.0