net/smc: transition to RDMA core CQ pooling
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v4 Latest3 patches 10 repliesIn review
- v3 posting 2/23 patches 1 replySuperseded
- v2 posting 2/22 patches 3 repliesSuperseded
Earlier versions · 3
| Version | Posted | Patches | Replies | Status |
|---|---|---|---|---|
| v3 posting 1/2 | 1 patch | 3 replies | Superseded | |
| v2 posting 1/2 | 1 patch | 2 replies | Superseded | |
| v1 | 1 patch | 2 replies | Superseded |
Changes in v4 from the cover letter
https://lore.kernel.org/netdev/[email protected]/
1. Reorder: move the exclusive-wait patch before the CQ pooling patch.
2. Fix teardown stall: reintroduce smc_ib_modify_qp_error() and flush the QP in smc_wr_stop_link() before waiting for the refs.
3. smc_wr_reg_send(): take the ref before ib_post_send().
4. smc_wr_tx_process_cqe(): wake_up_all() on a flush completion.
5. Merge the three per-link percpu_refs into a single wr_refs.
Patches in v4 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/3 | net/smc: clear the correct v2 slot and buffer in smc_wr_tx_put_slot()net/ | +4 | 1 reply | |
| 2/3 | net/smc: reduce TX slot contention with exclusive waitnet/ | +26 | 1 reply | |
| 3/3 | net/smc: transition to RDMA core CQ poolingnet/ net/ net/ +4 more | +257 | 6 replies |
Reviews trailers per patch
All 3 patches have a Reviewed-by or Acked-by.
| Patch | Dust Li | Leon Romanovsky | Mahanta Jambigi | Wen Gu |
|---|---|---|---|---|
| 1/3 | R-b | R-b | ||
| 2/3 | R-b | |||
| 3/3 | R-b |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 4 reviewers.
Cover letter [PATCH net-next v4 0/3] net/smc: transition to RDMA core CQ pooling
This series transitions SMC-R completion handling to RDMA core CQ pooling via the ib_cqe API. The new completion model improves scalability by allowing per-link completion processing across multiple cores and enables DIM-based interrupt moderation.
The increased concurrency can amplify contention for TX slots on the shared wait queue, so the exclusive-wait change comes first as a standalone improvement that the CQ pooling teardown then builds on.
Patch 1 fixes smc_wr_tx_put_slot() to clear the v2 pending slot and buffer structures instead of the pointer variables.
Patch 2 reduces TX slot contention by switching TX slot allocation from non-exclusive wait_event() to prepare_to_wait_exclusive(). Patch 3 replaces the global per-device CQ and manual tasklet polling model with RDMA core CQ pooling.
---
Changes v1 -> v2:
https://lore.kernel.org/netdev/[email protected]/
1. remove unnecessary inline from static CQE init helpers.
2. Use ib_drain_qp() with +1 max_send_wr;
3. Fix v2 state clearing.
4. Add re-check after schedule_timeout() to fix timeout/signal races.
Changes v2 -> v3:
https://lore.kernel.org/netdev/[email protected]/
1. Reserve +3 instead of +1 for the SQ to cover the drain, FastReg and SMC-Rv2 SEND WRs, avoiding SQ exhaustion that breaks ib_drain_sq().
2. Guard the recv WR repost with a per-link percpu_ref so no WR is reposted after ib_drain_qp(), fixing the RX repost/drain use-after-free.
3. Split the smc_wr_tx_put_slot() v2 clearing fix into a separate patch (1/3) with a Fixes: tag and reworded to the verifiable root cause.
Changes v3 -> v4:
https://lore.kernel.org/netdev/[email protected]/
1. Reorder: move the exclusive-wait patch before the CQ pooling patch.
2. Fix teardown stall: reintroduce smc_ib_modify_qp_error() and flush the QP in smc_wr_stop_link() before waiting for the refs.
3. smc_wr_reg_send(): take the ref before ib_post_send().
4. smc_wr_tx_process_cqe(): wake_up_all() on a flush completion.
5. Merge the three per-link percpu_refs into a single wr_refs.
D. Wythe (3):
net/smc: clear the correct v2 slot and buffer in smc_wr_tx_put_slot() net/smc: reduce TX slot contention with exclusive wait net/smc: transition to RDMA core CQ pooling
net/smc/smc_core.c | 10 +- net/smc/smc_core.h | 36 ++-- net/smc/smc_ib.c | 106 ++++------- net/smc/smc_ib.h | 6 - net/smc/smc_tx.c | 1 - net/smc/smc_wr.c | 445 +++++++++++++++++++++------------------------ net/smc/smc_wr.h | 50 ++--- 7 files changed, 287 insertions(+), 367 deletions(-)
Signature · 1 line
2.45.0