net/smc: transition to RDMA core CQ pooling
Subsystems (5)
D. Wythe
maintainerv4 cover net/smc: transition to RDMA core CQ pooling
This series transitions SMC-R completion handling to RDMA core CQ pooling via the ib_cqe API. The new completion model improves scalability by allowing per-link completion processing across multiple cores and enables DIM-based interrupt moderation.
The increased concurrency can amplify contention for TX slots on the shared wait queue, so the exclusive-wait change comes first as a standalone improvement that the CQ pooling teardown then builds on.
Patch 1 fixes smc_wr_tx_put_slot() to clear the v2 pending slot and buffer structures instead of the pointer variables.
Patch 2 reduces TX slot contention by switching TX slot allocation from non-exclusive wait_event() to prepare_to_wait_exclusive(). Patch 3 replaces the global per-device CQ and manual tasklet polling model with RDMA core CQ pooling.
---
Changes v1 -> v2:
https://lore.kernel.org/netdev/[email protected]/
1. remove unnecessary inline from static CQE init helpers.
2. Use ib_drain_qp() with +1 max_send_wr;
3. Fix v2 state clearing.
4. Add re-check after schedule_timeout() to fix timeout/signal races.
Changes v2 -> v3:
https://lore.kernel.org/netdev/[email protected]/
1. Reserve +3 instead of +1 for the SQ to cover the drain, FastReg and SMC-Rv2 SEND WRs, avoiding SQ exhaustion that breaks ib_drain_sq().
2. Guard the recv WR repost with a per-link percpu_ref so no WR is reposted after ib_drain_qp(), fixing the RX repost/drain use-after-free.
3. Split the smc_wr_tx_put_slot() v2 clearing fix into a separate patch (1/3) with a Fixes: tag and reworded to the verifiable root cause.
Changes v3 -> v4:
https://lore.kernel.org/netdev/[email protected]/
1. Reorder: move the exclusive-wait patch before the CQ pooling patch.
2. Fix teardown stall: reintroduce smc_ib_modify_qp_error() and flush the QP in smc_wr_stop_link() before waiting for the refs.
3. smc_wr_reg_send(): take the ref before ib_post_send().
4. smc_wr_tx_process_cqe(): wake_up_all() on a flush completion.
5. Merge the three per-link percpu_refs into a single wr_refs.
D. Wythe (3):
net/smc: clear the correct v2 slot and buffer in smc_wr_tx_put_slot() net/smc: reduce TX slot contention with exclusive wait net/smc: transition to RDMA core CQ pooling
net/smc/smc_core.c | 10 +- net/smc/smc_core.h | 36 ++-- net/smc/smc_ib.c | 106 ++++------- net/smc/smc_ib.h | 6 - net/smc/smc_tx.c | 1 - net/smc/smc_wr.c | 445 +++++++++++++++++++++------------------------ net/smc/smc_wr.h | 50 ++--- 7 files changed, 287 insertions(+), 367 deletions(-)
Signature · 1 line
2.45.0
D. Wythe
maintainerPATCH v4 1/3 net/smc: clear the correct v2 slot and buffer in smc_wr_tx_put_slot()
smc_wr_tx_put_slot() tries to reset the v2 pending slot and buffer with memset(&link->wr_tx_v2_pend, 0, sizeof(link->wr_tx_v2_pend)) and the equivalent for wr_tx_buf_v2. Both are pointers, so this zeroes the 8-byte pointer variable instead of the structure it points to. The pending slot and buffer are therefore never actually cleared, and the pointers get overwritten with NULL.
Pass the pointers directly and use sizeof(*pointer) so the intended structures are cleared.
--- net/smc/smc_wr.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
net/smc/smc_wr.c +4 -4
@@ -288,10 +288,10 @@ int smc_wr_tx_put_slot(struct smc_link *link, }else if (link->lgr->smc_version ==SMC_V2 && pend->idx == link->wr_tx_cnt) {/* Large v2 buffer */ -memset (&link->wr_tx_v2_pend,0 , -sizeof (link->wr_tx_v2_pend)); -memset (&link->lgr->wr_tx_buf_v2,0 , -sizeof (link->lgr->wr_tx_buf_v2)); +memset (link->wr_tx_v2_pend,0 , +sizeof (*link->wr_tx_v2_pend)); +memset (link->lgr->wr_tx_buf_v2,0 , +sizeof (*link->lgr->wr_tx_buf_v2));return 1 ; }
Signature · 1 line
2.45.0
[email protected]
AI reviewRe: [PATCH net-next v4 1/3] net/smc: clear the correct v2 slot and buffer in smc_wr_tx_put_slot()
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Pre-existing issues:
- [High] Data race in V2 slot cleanup allowing a concurrent sender's data to be overwritten.
- [High] Unconditional zeroing of the shared lgr->wr_tx_buf_v2 in smc_wr_tx_process_cqe() on any link error corrupts in-progress LLC messages on other links. --
--- Patch [1]: [PATCH net-next v4 1/3] net/smc: clear the correct v2 slot and buffer in smc_wr_tx_put_slot() --- commit eb37bf5f30fd47da384e2535c48a0ad5576aa908
Author: D. Wythe <[email protected]>
net/smc: clear the correct v2 slot and buffer in smc_wr_tx_put_slot()
smc_wr_tx_put_slot() tries to reset the v2 pending slot and buffer with memset(&link->wr_tx_v2_pend, 0, sizeof(link->wr_tx_v2_pend)) and the equivalent for wr_tx_buf_v2. Both are pointers, so this zeroes the 8-byte pointer variable instead of the structure it points to. The pending slot and buffer are therefore never actually cleared, and the pointers get overwritten with NULL.
Pass the pointers directly and use sizeof(*pointer) so the intended structures are cleared.
net/smc/smc_wr.c int smc_wr_tx_put_slot(struct smc_link *link, · line 292
Quoted 12 lines
net/smc/smc_wr.c +2
-2@@ -288,10 +288,10 @@ int smc_wr_tx_put_slot(struct smc_link *link, }else if (link->lgr->smc_version ==SMC_V2 && pend->idx == link->wr_tx_cnt) {/* Large v2 buffer */ -memset (&link->wr_tx_v2_pend,0 , -sizeof (link->wr_tx_v2_pend)); +memset (link->wr_tx_v2_pend,0 , +sizeof (*link->wr_tx_v2_pend));
[Severity: High]
This isn't a bug introduced by this patch, but does clearing the control structure before the data buffer create a race window?
If a thread sends an LLC message and waits for a response, the Rx tasklet might process the Rx CQE first, waking up the LLC flow thread which proceeds to send the next message by calling smc_wr_tx_get_v2_slot().
Concurrently, the Tx tasklet could process the Tx CQE and clear the slot control structure here. The zeroed idx immediately publishes the slot as free, so the LLC thread could acquire it and begin formatting its message in link->lgr->wr_tx_buf_v2.
The same sequence appears to occur in smc_wr_tx_process_cqe().
Would the following memset then wipe out the newly written message?
net/smc/smc_wr.c int smc_wr_tx_put_slot(struct smc_link *link, · line 294
Quoted 4 lines
net/smc/smc_wr.c +2
-2-memset (&link->lgr->wr_tx_buf_v2,0 , -sizeof (link->lgr->wr_tx_buf_v2)); +memset (link->lgr->wr_tx_buf_v2,0 , +sizeof (*link->lgr->wr_tx_buf_v2));
[Severity: High]
This is a pre-existing issue, but could unconditionally zeroing the shared link->lgr->wr_tx_buf_v2 cause issues for other links in the LGR?
It looks like the same zeroing pattern happens in the error path of smc_wr_tx_process_cqe():
net/smc/smc_wr.c:smc_wr_tx_process_cqe() {
...
if (unlikely(wc->status)) {
...
if (link->lgr->smc_version == SMC_V2) {
memset(link->wr_tx_v2_pend, 0, sizeof(*link->wr_tx_v2_pend));
memset(link->lgr->wr_tx_buf_v2, 0, sizeof(*link->lgr->wr_tx_buf_v2));
}
}
}If Link B is actively formatting an LLC message in this shared LGR buffer, and a normal data work request concurrently fails on Link A, would Link A's tasklet incorrectly wipe out Link B's in-progress LLC message?
Quoted 2 lines
net/smc/smc_wr.c +0
-0return 1 ; }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Re: [PATCH net-next v4 1/3] net/smc: clear the correct v2 slot and buffer in smc_wr_tx_put_slot()
In reply to D. WytheQuoted 26 lines · D. Wythe
smc_wr_tx_put_slot()tries to reset the v2 pending slot and buffer withmemset(&link->wr_tx_v2_pend, 0, sizeof(link->wr_tx_v2_pend))and the equivalent forwr_tx_buf_v2. Both are pointers, so this zeroes the 8-byte pointer variable instead of the structure it points to. The pending slot and buffer are therefore never actually cleared, and the pointers get overwritten with NULL.Pass the pointers directly and use
sizeof(*pointer)so the intended structures are cleared.Fixes 8799e310fb3f ("net/smc: add v2 support to the work request layer")Signed-off-by D. Wythe <[email protected]>Reviewed-by Dust Li <[email protected]>--- net/smc/smc_wr.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)net/smc/smc_wr.c +0
-1@@ -288,10 +288,10 @@ int smc_wr_tx_put_slot(struct smc_link *link, }else if (link->lgr->smc_version ==SMC_V2 && pend->idx == link->wr_tx_cnt) {/* Large v2 buffer */ - memset(&link->wr_tx_v2_pend, 0,
net/smc/smc_wr.c int smc_wr_tx_put_slot(struct smc_link *link, · line 297
Quoted 10 lines
net/smc/smc_wr.c +4
-3- sizeof(link->wr_tx_v2_pend)); -memset (&link->lgr->wr_tx_buf_v2,0 , -sizeof (link->lgr->wr_tx_buf_v2)); +memset (link->wr_tx_v2_pend,0 , +sizeof (*link->wr_tx_v2_pend)); +memset (link->lgr->wr_tx_buf_v2,0 , +sizeof (*link->lgr->wr_tx_buf_v2));return 1 ; }
D. Wythe
maintainerPATCH v4 2/3 net/smc: reduce TX slot contention with exclusive wait
In reply to D. Wythesmc_wr_tx_get_free_slot() waits for a free TX slot with wait_event_interruptible_timeout(). Since the wait_event family enqueues waiters as non-exclusive, wake_up() may wake multiple waiters even though only one can use the slot, causing thundering-herd contention when slots are scarce.
Use an exclusive wait loop with prepare_to_wait_exclusive() so wake_up() wakes only one waiter per freed slot. smc_wr_wakeup_tx_wait() still uses wake_up_all() during link teardown, so teardown behavior is unchanged.
This also corrects the return value on a pending signal: the previous wait_event_interruptible_timeout() path fell through to the "no free slot" case and returned -EPIPE, masking the signal as a connection error. The open-coded loop now returns -ERESTARTSYS, matching the standard interruptible-wait semantics and letting the syscall restart machinery handle it.
--- net/smc/smc_wr.c | 36 ++++++++++++++++++++++++++---------- 1 file changed, 26 insertions(+), 10 deletions(-)
net/smc/smc_wr.c +26 -10
@@ -198,11 +198,13 @@ int smc_wr_tx_get_free_slot(struct smc_link *link,struct smc_rdma_wr **wr_rdma_buf, struct smc_wr_tx_pend_priv **wr_pend_priv) { + unsigned long timeout = SMC_WR_TX_WAIT_FREE_SLOT_TIME;struct smc_link_group *lgr =smc_get_lgr (link);struct smc_wr_tx_pend *wr_pend;u32 idx = link->wr_tx_cnt;struct ib_send_wr *wr_ib;u64 wr_id; +DEFINE_WAIT (wait);int rc; *wr_buf =NULL ;
@@ -212,17 +214,31 @@ int smc_wr_tx_get_free_slot(struct smc_link *link,if (rc)return rc; }else { - rc =wait_event_interruptible_timeout ( - link->wr_tx_wait, - !smc_link_sendable (link) || - lgr->terminating || - (smc_wr_tx_get_free_slot_index (link, &idx) != -EBUSY ), -SMC_WR_TX_WAIT_FREE_SLOT_TIME ); -if (!rc) { -/* timeout - terminate link */ -smcr_link_down_cond_sched (link); -return -EPIPE ; + rc =0 ; +for (;;) { +prepare_to_wait_exclusive (&link->wr_tx_wait, &wait, +TASK_INTERRUPTIBLE ); +if (!smc_link_sendable (link) || lgr->terminating || +smc_wr_tx_get_free_slot_index (link, &idx) != -EBUSY ) +break ; + timeout =schedule_timeout (timeout); +/* re-check */ +if (!smc_link_sendable (link) || lgr->terminating || +smc_wr_tx_get_free_slot_index (link, &idx) != -EBUSY ) +break ; +if (!timeout) { +/* timeout - terminate link */ +smcr_link_down_cond_sched (link); +break ; + } +if (signal_pending (current)) { + rc = -ERESTARTSYS ; +break ; + } } +finish_wait (&link->wr_tx_wait, &wait); +if (rc) +return rc;if (idx == link->wr_tx_cnt)return -EPIPE ; }
Signature · 1 line
2.45.0
Re: [PATCH net-next v4 2/3] net/smc: reduce TX slot contention with exclusive wait
Quoted 18 lines · D. Wythe
smc_wr_tx_get_free_slot()waits for a free TX slot withwait_event_interruptible_timeout(). Since thewait_eventfamily enqueues waiters as non-exclusive,wake_up()may wake multiple waiters even though only one can use the slot, causing thundering-herd contention when slots are scarce.Use an exclusive wait loop with
prepare_to_wait_exclusive()sowake_up()wakes only one waiter per freed slot.smc_wr_wakeup_tx_wait()still useswake_up_all()during link teardown, so teardown behavior is unchanged.This also corrects the return value on a pending signal: the previous
wait_event_interruptible_timeout()path fell through to the "no free slot" case and returned-EPIPE, masking the signal as a connection error. The open-coded loop now returns-ERESTARTSYS, matching the standard interruptible-wait semantics and letting the syscall restart machinery handle it.
Quoted 2 lines
Signed-off-by D. Wythe <[email protected]>---
Thanks.
Quoted 63 lines
net/smc/smc_wr.c | 36 ++++++++++++++++++++++++++---------- 1 file changed, 26 insertions(+), 10 deletions(-)net/smc/smc_wr.c +26
-10@@ -198,11 +198,13 @@ int smc_wr_tx_get_free_slot(struct smc_link *link,struct smc_rdma_wr **wr_rdma_buf, struct smc_wr_tx_pend_priv **wr_pend_priv) { + unsigned long timeout = SMC_WR_TX_WAIT_FREE_SLOT_TIME;struct smc_link_group *lgr =smc_get_lgr (link);struct smc_wr_tx_pend *wr_pend;u32 idx = link->wr_tx_cnt;struct ib_send_wr *wr_ib;u64 wr_id; +DEFINE_WAIT (wait);int rc; *wr_buf =NULL ;@@ -212,17 +214,31 @@ int smc_wr_tx_get_free_slot(struct smc_link *link,if (rc)return rc; }else { - rc =wait_event_interruptible_timeout ( - link->wr_tx_wait, - !smc_link_sendable (link) || - lgr->terminating || - (smc_wr_tx_get_free_slot_index (link, &idx) != -EBUSY ), -SMC_WR_TX_WAIT_FREE_SLOT_TIME ); -if (!rc) { -/* timeout - terminate link */ -smcr_link_down_cond_sched (link); -return -EPIPE ; + rc =0 ; +for (;;) { +prepare_to_wait_exclusive (&link->wr_tx_wait, &wait, +TASK_INTERRUPTIBLE ); +if (!smc_link_sendable (link) || lgr->terminating || +smc_wr_tx_get_free_slot_index (link, &idx) != -EBUSY ) +break ; + timeout =schedule_timeout (timeout); +/* re-check */ +if (!smc_link_sendable (link) || lgr->terminating || +smc_wr_tx_get_free_slot_index (link, &idx) != -EBUSY ) +break ; +if (!timeout) { +/* timeout - terminate link */ +smcr_link_down_cond_sched (link); +break ; + } +if (signal_pending (current)) { + rc = -ERESTARTSYS ; +break ; + } } +finish_wait (&link->wr_tx_wait, &wait); +if (rc) +return rc;if (idx == link->wr_tx_cnt)return -EPIPE ; }
D. Wythe
maintainerPATCH v4 3/3 net/smc: transition to RDMA core CQ pooling
In reply to D. WytheThe current SMC-R implementation relies on global per-device CQs and manual polling within tasklets, which introduces severe scalability bottlenecks due to global lock contention and tasklet scheduling overhead, resulting in poor performance as concurrency increases.
Refactor the completion handling to utilize the ib_cqe API and standard RDMA core CQ pooling. This transition provides several key advantages:
1. Multi-CQ: Shift from a single shared per-device CQ to multiple link-specific CQs via the CQ pool. This allows completion processing to be parallelized across multiple CPU cores, effectively eliminating the global CQ bottleneck.
2. Leverage DIM: Utilizing the standard CQ pool with IB_POLL_SOFTIRQ enables Dynamic Interrupt Moderation from the RDMA core, optimizing interrupt frequency and reducing CPU load under high pressure.
3. O(1) Context Retrieval: Replaces the expensive wr_id based lookup logic (e.g., smc_wr_tx_find_pending_index) with direct context retrieval using container_of() on the embedded ib_cqe.
4. Code Simplification: This refactoring results in a reduction of ~150 lines of code. It removes redundant sequence tracking, complex lookup helpers, and manual CQ management, significantly improving maintainability.
Performance Test: redis-benchmark with max 32 connections per QP Data format: Requests Per Second (RPS), Percentage in brackets represents the gain/loss compared to TCP.
| Clients | TCP | SMC (original) | SMC (cq_pool) | |---------|----------|---------------------|---------------------| | c = 1 | 24449 | 31172 (+27%) | 34039 (+39%) | | c = 2 | 46420 | 53216 (+14%) | 64391 (+38%) | | c = 16 | 159673 | 83668 (-48%) <-- | 216947 (+36%) | | c = 32 | 164956 | 97631 (-41%) <-- | 249376 (+51%) | | c = 64 | 166322 | 118192 (-29%) <-- | 249488 (+50%) | | c = 128 | 167700 | 121497 (-27%) <-- | 249480 (+48%) | | c = 256 | 175021 | 146109 (-16%) <-- | 240384 (+37%) | | c = 512 | 168987 | 101479 (-40%) <-- | 226634 (+34%) |
The results demonstrate that this optimization effectively resolves the scalability bottleneck, with RPS increasing by over 110% at c=64 compared to the original implementation.
--- net/smc/smc_core.c | 10 +- net/smc/smc_core.h | 36 ++-- net/smc/smc_ib.c | 106 ++++-------- net/smc/smc_ib.h | 6 - net/smc/smc_tx.c | 1 - net/smc/smc_wr.c | 401 ++++++++++++++++++++------------------------- net/smc/smc_wr.h | 50 ++---- 7 files changed, 257 insertions(+), 353 deletions(-)
net/smc/smc_core.c +3 -7
@@ -815,17 +815,11 @@ int smcr_link_init(struct smc_link_group *lgr, struct smc_link *lnk, lnk->lgr = lgr;smc_lgr_hold (lgr);/* lgr_put in smcr_link_clear() */ lnk->link_idx = link_idx; - lnk->wr_rx_id_compl =0 ;smc_ibdev_cnt_inc (lnk);smcr_copy_dev_info_to_link (lnk);atomic_set (&lnk->conn_cnt,0 );smc_llc_link_set_uid (lnk);INIT_WORK (&lnk->link_down_wrk, smc_link_down_work); -if (!lnk->smcibdev->initialized) { - rc = (int )smc_ib_setup_per_ibdev (lnk->smcibdev); -if (rc) - goto out; - }get_random_bytes (rndvec,sizeof (rndvec)); lnk->psn_initial = rndvec[0 ] + (rndvec[1 ] <<8 ) + (rndvec[2 ] <<16 );
@@ -863,6 +857,7 @@ int smcr_link_init(struct smc_link_group *lgr, struct smc_link *lnk,if (rc) goto free_link_mem; lnk->state =SMC_LNK_ACTIVATING ; +smc_wr_init_cqes (lnk);return 0 ; free_link_mem:
@@ -1373,7 +1368,8 @@ void smcr_link_clear(struct smc_link *lnk, bool log)smc_llc_link_clear (lnk, log);smcr_buf_unmap_lgr (lnk);smcr_rtoken_clear_link (lnk); -smc_ib_modify_qp_error (lnk); +smc_wr_stop_link (lnk); +smc_wr_drain_qp (lnk);smc_wr_free_link (lnk);smc_ib_destroy_queue_pair (lnk);smc_ib_dealloc_protection_domain (lnk);
net/smc/smc_core.h +22 -14
@@ -89,8 +89,21 @@ struct smc_rdma_sges { /* sges per message send */struct smc_rdma_wr {/* work requests per message * send */ +struct ib_cqe cqe;struct ib_rdma_wr wr_tx_rdma[SMC_MAX_RDMA_WRITES ]; -}; +}____cacheline_aligned_in_smp ; + +struct smc_ib_recv_wr { +struct ib_cqe cqe; +struct ib_recv_wr wr; +int idx; +}____cacheline_aligned_in_smp ; + +struct smc_ib_send_wr { +struct ib_cqe cqe; +struct ib_send_wr wr; +int idx; +}____cacheline_aligned_in_smp ;#define SMC_LGR_ID_SIZE 4
@@ -100,33 +113,34 @@ struct smc_link {struct ib_pd *roce_pd;/* IB protection domain, * unique for every RoCE QP */ +unsigned int nr_cqe;/* number of CQ entries */ +struct ib_cq *ib_cq;/* IB completion queue */ struct ib_qp *roce_qp;/* IB queue pair */ struct ib_qp_attr qp_attr;/* IB queue pair attributes */ struct smc_wr_buf *wr_tx_bufs;/* WR send payload buffers */ -struct ib_send_wr *wr_tx_ibs;/* WR send meta data */ +struct smc_ib_send_wr *wr_tx_ibs;/* WR send meta data */ struct ib_sge *wr_tx_sges;/* WR send gather meta data */ struct smc_rdma_sges *wr_tx_rdma_sges;/*RDMA WRITE gather meta data*/ struct smc_rdma_wr *wr_tx_rdmas;/* WR RDMA WRITE */ struct smc_wr_tx_pend *wr_tx_pends;/* WR send waiting for CQE */ struct completion *wr_tx_compl;/* WR send CQE completion */ /* above four vectors have wr_tx_cnt elements and use the same index */ -struct ib_send_wr *wr_tx_v2_ib;/* WR send v2 meta data */ +struct smc_ib_send_wr *wr_tx_v2_ib;/* WR send v2 meta data */ struct ib_sge *wr_tx_v2_sge;/* WR send v2 gather meta data*/ struct smc_wr_tx_pend *wr_tx_v2_pend;/* WR send v2 waiting for CQE */ dma_addr_t wr_tx_dma_addr;/* DMA address of wr_tx_bufs */ dma_addr_t wr_tx_v2_dma_addr;/* DMA address of v2 tx buf*/ -atomic_long_t wr_tx_id;/* seq # of last sent WR */ unsigned long *wr_tx_mask;/* bit mask of used indexes */ u32 wr_tx_cnt;/* number of WR send buffers */ wait_queue_head_t wr_tx_wait;/* wait for free WR send buf */ struct { -struct percpu_ref wr_tx_refs; +struct percpu_ref wr_refs;/* refs for all WR posting (tx/rx/reg) */ }____cacheline_aligned_in_smp ; -struct completion tx_ref_comp; +struct completion ref_comp;u8 *wr_rx_bufs;/* WR recv payload buffers */ -struct ib_recv_wr *wr_rx_ibs;/* WR recv meta data */ +struct smc_ib_recv_wr *wr_rx_ibs;/* WR recv meta data */ struct ib_sge *wr_rx_sges;/* WR recv scatter meta data */ /* above three vectors have wr_rx_cnt elements and use the same index */ int wr_rx_sge_cnt;/* rx sge, V1 is 1, V2 is either 2 or 1 */
@@ -135,18 +149,12 @@ struct smc_link {*/ dma_addr_t wr_rx_dma_addr;/* DMA address of wr_rx_bufs */ dma_addr_t wr_rx_v2_dma_addr;/* DMA address of v2 rx buf*/ -u64 wr_rx_id;/* seq # of last recv WR */ -u64 wr_rx_id_compl;/* seq # of last completed WR */ u32 wr_rx_cnt;/* number of WR recv buffers */ unsigned long wr_rx_tstamp;/* jiffies when last buf rx */ -wait_queue_head_t wr_rx_empty_wait;/* wait for RQ empty */ struct ib_reg_wr wr_reg;/* WR register memory region */ +struct ib_cqe wr_reg_cqe;/* ib_cqe for wr_reg */ wait_queue_head_t wr_reg_wait;/* wait for wr_reg result */ -struct { -struct percpu_ref wr_reg_refs; - }____cacheline_aligned_in_smp ; -struct completion reg_ref_comp;enum smc_wr_reg_state wr_reg_state;/* state of wr_reg request */ u8 gid[SMC_GID_SIZE ];/* gid matching used vlan id*/
net/smc/smc_ib.c +34 -72
158 lines not on this page · raw message
net/smc/smc_ib.h +0 -6
@@ -37,17 +37,12 @@ struct smc_ib_device { /* ib-device infos for smc */struct ib_device *ibdev;struct ib_port_attr pattr[SMC_MAX_PORTS ];/* ib dev. port attrs */ struct ib_event_handler event_handler;/* global ib_event handler */ -struct ib_cq *roce_cq_send;/* send completion queue */ -struct ib_cq *roce_cq_recv;/* recv completion queue */ -struct tasklet_struct send_tasklet;/* called by send cq handler */ -struct tasklet_struct recv_tasklet;/* called by recv cq handler */ char mac[SMC_MAX_PORTS ][ETH_ALEN ];/* mac address per port*/ u8 pnetid[SMC_MAX_PORTS ][SMC_MAX_PNETID_LEN ];/* pnetid per port */ bool pnetid_by_user[SMC_MAX_PORTS ];/* pnetid defined by user? */ - u8 initialized : 1; /* ib dev CQ, evthdl done */struct work_struct port_event_work;unsigned long port_event_mask;DECLARE_BITMAP (ports_going_away,SMC_MAX_PORTS );
@@ -97,7 +92,6 @@ int smc_ib_create_queue_pair(struct smc_link *lnk);int smc_ib_ready_link (struct smc_link *lnk);int smc_ib_modify_qp_rts (struct smc_link *lnk);int smc_ib_modify_qp_error (struct smc_link *lnk); -long smc_ib_setup_per_ibdev (struct smc_ib_device *smcibdev);int smc_ib_get_memory_region (struct ib_pd *pd,int access_flags,struct smc_buf_desc *buf_slot,u8 link_idx);void smc_ib_put_memory_region (struct ib_mr *mr);
net/smc/smc_tx.c +0 -1
@@ -321,7 +321,6 @@ static int smc_tx_rdma_write(struct smc_connection *conn, int peer_rmbe_offset,struct smc_link *link = conn->lnk;int rc; - rdma_wr->wr.wr_id =smc_wr_tx_get_next_wr_id (link); rdma_wr->wr.num_sge = num_sges; rdma_wr->remote_addr = lgr->rtokens[conn->rtoken_idx][link->link_idx].dma_addr +
net/smc/smc_wr.c +181 -220
603 lines not on this page · raw message
net/smc/smc_wr.h +17 -33
93 lines not on this page · raw message
Signature · 1 line
2.45.0
Mahanta Jambigi
maintainerQuoted 15 lines · D. Wythe
Performance Test: redis-benchmark with max 32 connections per QP Data format: Requests Per Second (RPS), Percentage in brackets represents the gain/loss compared to TCP.
| Clients | TCP | SMC (original) | SMC (cq_pool) | |---------|----------|---------------------|---------------------| | c = 1 | 24449 | 31172 (+27%) | 34039 (+39%) | | c = 2 | 46420 | 53216 (+14%) | 64391 (+38%) | | c = 16 | 159673 | 83668 (-48%) <-- | 216947 (+36%) | | c = 32 | 164956 | 97631 (-41%) <-- | 249376 (+51%) | | c = 64 | 166322 | 118192 (-29%) <-- | 249488 (+50%) | | c = 128 | 167700 | 121497 (-27%) <-- | 249480 (+48%) | | c = 256 | 175021 | 146109 (-16%) <-- | 240384 (+37%) | | c = 512 | 168987 | 101479 (-40%) <-- | 226634 (+34%) |
Quoted 3 lines
The results demonstrate that this optimization effectively resolves the scalability bottleneck, with RPS increasing by over 110% at c=64 compared to the original implementation.
Thanks for the performance numbers — the scalability improvement at high client counts is striking. A few questions to help reproduce these results:
1) Was net.smc.smcr_max_conns_per_lgr=32 the value used? The "max 32 connections per QP" in the description suggests so, but the sysctl wasn't explicitly listed.
2) Were net.smc.smcr_max_send_wr and net.smc.smcr_max_recv_wr left at their defaults, or tuned? With per-link CQs the optimal WR counts may differ from the global-CQ case, so knowing whether these were touched would help interpret the numbers.
3) Was this SMC-R v2? The v2 TX path (smc_wr_tx_get_v2_slot, wr_tx_v2_ib) has different concurrency characteristics, so confirming the version matters for the contention analysis in patch 2/3 as well.
4) Was uperf tested for v4 as well, as we did for v1[1] in February-2026? uperf exercises a wider range of message sizes and thread counts, so its numbers would strengthen the case — particularly for the intermediate client counts (c=16 to c=64) where the original implementation regresses below TCP.
Mahanta Jambigi
maintainerQuoted 38 lines · Mahanta Jambigi
Quoted 15 lines · D. Wythe
Performance Test: redis-benchmark with max 32 connections per QP Data format: Requests Per Second (RPS), Percentage in brackets represents the gain/loss compared to TCP.
| Clients | TCP | SMC (original) | SMC (cq_pool) | |---------|----------|---------------------|---------------------| | c = 1 | 24449 | 31172 (+27%) | 34039 (+39%) | | c = 2 | 46420 | 53216 (+14%) | 64391 (+38%) | | c = 16 | 159673 | 83668 (-48%) <-- | 216947 (+36%) | | c = 32 | 164956 | 97631 (-41%) <-- | 249376 (+51%) | | c = 64 | 166322 | 118192 (-29%) <-- | 249488 (+50%) | | c = 128 | 167700 | 121497 (-27%) <-- | 249480 (+48%) | | c = 256 | 175021 | 146109 (-16%) <-- | 240384 (+37%) | | c = 512 | 168987 | 101479 (-40%) <-- | 226634 (+34%) |Quoted 3 lines
The results demonstrate that this optimization effectively resolves the scalability bottleneck, with RPS increasing by over 110% at c=64 compared to the original implementation.
Thanks for the performance numbers — the scalability improvement at high client counts is striking. A few questions to help reproduce these results:
1) Was net.smc.smcr_max_conns_per_lgr=32 the value used? The "max 32 connections per QP" in the description suggests so, but the sysctl wasn't explicitly listed.
2) Were net.smc.smcr_max_send_wr and net.smc.smcr_max_recv_wr left at their defaults, or tuned? With per-link CQs the optimal WR counts may differ from the global-CQ case, so knowing whether these were touched would help interpret the numbers.
3) Was this SMC-R v2? The v2 TX path (
smc_wr_tx_get_v2_slot,wr_tx_v2_ib) has different concurrency characteristics, so confirming the version matters for the contention analysis in patch 2/3 as well.
Quoted 5 lines
4) Was uperf tested for v4 as well, as we did for v1[1] in February-2026? uperf exercises a wider range of message sizes and thread counts, so its numbers would strengthen the case — particularly for the intermediate client counts (c=16 to c=64) where the original implementation regresses below TCP.
@D. Wythe: Gentle reminder regarding my earlier questions. Have you had a chance to look into them?
Quoted 3 lines
D. Wythe
maintainerQuoted 29 lines · Mahanta Jambigi
Quoted 15 lines · D. Wythe
Performance Test: redis-benchmark with max 32 connections per QP Data format: Requests Per Second (RPS), Percentage in brackets represents the gain/loss compared to TCP.
| Clients | TCP | SMC (original) | SMC (cq_pool) | |---------|----------|---------------------|---------------------| | c = 1 | 24449 | 31172 (+27%) | 34039 (+39%) | | c = 2 | 46420 | 53216 (+14%) | 64391 (+38%) | | c = 16 | 159673 | 83668 (-48%) <-- | 216947 (+36%) | | c = 32 | 164956 | 97631 (-41%) <-- | 249376 (+51%) | | c = 64 | 166322 | 118192 (-29%) <-- | 249488 (+50%) | | c = 128 | 167700 | 121497 (-27%) <-- | 249480 (+48%) | | c = 256 | 175021 | 146109 (-16%) <-- | 240384 (+37%) | | c = 512 | 168987 | 101479 (-40%) <-- | 226634 (+34%) |Quoted 3 lines
The results demonstrate that this optimization effectively resolves the scalability bottleneck, with RPS increasing by over 110% at c=64 compared to the original implementation.
Thanks for the performance numbers — the scalability improvement at high client counts is striking. A few questions to help reproduce these results:
1) Was net.smc.smcr_max_conns_per_lgr=32 the value used? The "max 32 connections per QP" in the description suggests so, but the sysctl wasn't explicitly listed.
Quoted 8 lines
2) Were net.smc.smcr_max_send_wr and net.smc.smcr_max_recv_wr left at their defaults, or tuned? With per-link CQs the optimal WR counts may differ from the global-CQ case, so knowing whether these were touched would help interpret the numbers.
3) Was this SMC-R v2? The v2 TX path (
smc_wr_tx_get_v2_slot,wr_tx_v2_ib) has different concurrency characteristics, so confirming the version matters for the contention analysis in patch 2/3 as well.
In our tests, we set net.smc.smcr_max_conns_per_lgr=32 via sysctl. We also tuned net.smc.smcr_max_send_wr and net.smc.smcr_max_recv_wr from their default to 64 to provide sufficient work request depth.
All tests were run with SMC-R v2.
And the performance numbers do need a refresh: with recent improvements—including thundering herd mitigation in earlier patches, the original SMC implementation’s regression in the c=16–512 range is expected to be largely resolved.
Quoted 6 lines
4) Was uperf tested for v4 as well, as we did for v1[1] in February-2026? uperf exercises a wider range of message sizes and thread counts, so its numbers would strengthen the case — particularly for the intermediate client counts (c=16 to c=64) where the original implementation regresses below TCP.
Of course uperf was tested. We’ll include a comprehensive uperf evaluation in the next version.
Given that this CQ-pooling patch may require additional review time, I plan to send the first two patches of the series separately. They’ve already been reviewed and are ready to merge. This patch will follow as a standalone submission once the uperf results are ready.
D. Wythe
Quoted 3 lines
Mahanta Jambigi
maintainerQuoted 43 lines · D. Wythe
Quoted 29 lines · Mahanta Jambigi
Quoted 15 lines · D. Wythe
Performance Test: redis-benchmark with max 32 connections per QP Data format: Requests Per Second (RPS), Percentage in brackets represents the gain/loss compared to TCP.
| Clients | TCP | SMC (original) | SMC (cq_pool) | |---------|----------|---------------------|---------------------| | c = 1 | 24449 | 31172 (+27%) | 34039 (+39%) | | c = 2 | 46420 | 53216 (+14%) | 64391 (+38%) | | c = 16 | 159673 | 83668 (-48%) <-- | 216947 (+36%) | | c = 32 | 164956 | 97631 (-41%) <-- | 249376 (+51%) | | c = 64 | 166322 | 118192 (-29%) <-- | 249488 (+50%) | | c = 128 | 167700 | 121497 (-27%) <-- | 249480 (+48%) | | c = 256 | 175021 | 146109 (-16%) <-- | 240384 (+37%) | | c = 512 | 168987 | 101479 (-40%) <-- | 226634 (+34%) |Quoted 3 lines
The results demonstrate that this optimization effectively resolves the scalability bottleneck, with RPS increasing by over 110% at c=64 compared to the original implementation.
Thanks for the performance numbers — the scalability improvement at high client counts is striking. A few questions to help reproduce these results:
1) Was net.smc.smcr_max_conns_per_lgr=32 the value used? The "max 32 connections per QP" in the description suggests so, but the sysctl wasn't explicitly listed.
Quoted 8 lines
2) Were net.smc.smcr_max_send_wr and net.smc.smcr_max_recv_wr left at their defaults, or tuned? With per-link CQs the optimal WR counts may differ from the global-CQ case, so knowing whether these were touched would help interpret the numbers.
3) Was this SMC-R v2? The v2 TX path (
smc_wr_tx_get_v2_slot,wr_tx_v2_ib) has different concurrency characteristics, so confirming the version matters for the contention analysis in patch 2/3 as well.In our tests, we set net.smc.smcr_max_conns_per_lgr=32 via sysctl. We also tuned net.smc.smcr_max_send_wr and net.smc.smcr_max_recv_wr from their default to 64 to provide sufficient work request depth.
Quoted 5 lines
All tests were run with SMC-R v2.
And the performance numbers do need a refresh: with recent improvements—including thundering herd mitigation in earlier patches, the original SMC implementation’s regression in the c=16–512 range is expected to be largely resolved.
I believe that Patch 2/3, "net/smc: reduce TX slot contention with exclusive wait", largely mitigates the thundering herd effect.
Quoted 8 lines
Quoted 7 lines
4) Was uperf tested for v4 as well, as we did for v1[1] in February-2026? uperf exercises a wider range of message sizes and thread counts, so its numbers would strengthen the case — particularly for the intermediate client counts (c=16 to c=64) where the original implementation regresses below TCP.
Quoted 8 lines
Of course uperf was tested. We’ll include a comprehensive uperf evaluation in the next version.
Given that this CQ-pooling patch may require additional review time, I plan to send the first two patches of the series separately. They’ve already been reviewed and are ready to merge. This patch will follow as a standalone submission once the uperf results are ready.
Sounds good. I'll be looking forward to the standalone patch that introduces CQ pooling and the accompanying uperf results.
Quoted 6 lines
D. Wythe
Quoted 3 lines
Wen Gu
reviewernet/smc/smc_ib.c static int smc_ib_add_dev(struct ib_device *ibdev) · line 955
Quoted 6 lines · D. Wythe
static void
smc_copy_netdev_ifindex(struct smc_ib_device *smcibdev, int port)net/smc/smc_ib.c +0
-1@@ -952,7 +916,6 @@ static int smc_ib_add_dev(struct ib_device *ibdev)INIT_WORK (&smcibdev->port_event_work, smc_ib_port_event_work);atomic_set (&smcibdev->lnk_cnt,0 );init_waitqueue_head (&smcibdev->lnks_deleted); -mutex_init (&smcibdev->mutex);
Since smc_ib_setup_per_ibdev() and smc_ib_cleanup_per_ibdev() were removed, the .mutex field is now unused and can be dropped.
Thanks.
Quoted 11 lines
net/smc/smc_ib.c +0
-1mutex_lock (&smc_ib_devices.mutex);list_add_tail (&smcibdev->list, &smc_ib_devices.list);mutex_unlock (&smc_ib_devices.mutex);@@ -1001,7 +964,6 @@ static void smc_ib_remove_dev(struct ib_device *ibdev, void *client_data)pr_warn_ratelimited ("smc: removing ib device %s\n" , smcibdev->ibdev->name);smc_smcr_terminate_all (smcibdev); -smc_ib_cleanup_per_ibdev (smcibdev);ib_unregister_event_handler (&smcibdev->event_handler);cancel_work_sync (&smcibdev->port_event_work);kfree (smcibdev);
D. Wythe
maintainerQuoted 13 lines · Wen Gu
net/smc/smc_ib.c static int smc_ib_add_dev(struct ib_device *ibdev) · line 955
Quoted 6 lines · D. Wythe
static void
smc_copy_netdev_ifindex(struct smc_ib_device *smcibdev, int port)net/smc/smc_ib.c +0
-1@@ -952,7 +916,6 @@ static int smc_ib_add_dev(struct ib_device *ibdev)INIT_WORK (&smcibdev->port_event_work, smc_ib_port_event_work);atomic_set (&smcibdev->lnk_cnt,0 );init_waitqueue_head (&smcibdev->lnks_deleted); -mutex_init (&smcibdev->mutex);Since
smc_ib_setup_per_ibdev()andsmc_ib_cleanup_per_ibdev()were removed, the .mutex field is now unused and can be dropped.
Quoted 1 line
Thanks.
Good catch! The mutex is indeed unused now — will remove it in v5.
Quoted 13 lines
Quoted 12 lines
net/smc/smc_ib.c +0
-1mutex_lock (&smc_ib_devices.mutex);list_add_tail (&smcibdev->list, &smc_ib_devices.list);mutex_unlock (&smc_ib_devices.mutex);@@ -1001,7 +964,6 @@ static void smc_ib_remove_dev(struct ib_device *ibdev, void *client_data)pr_warn_ratelimited ("smc: removing ib device %s\n" , smcibdev->ibdev->name);smc_smcr_terminate_all (smcibdev); -smc_ib_cleanup_per_ibdev (smcibdev);ib_unregister_event_handler (&smcibdev->event_handler);cancel_work_sync (&smcibdev->port_event_work);kfree (smcibdev);
Jakub Kicinski
maintainerQuoted 4 lines · On Thu, 16 Jul 2026 19:37:42 +0800 D. Wythe wrote:
This series transitions SMC-R completion handling to RDMA core CQ pooling via the
ib_cqeAPI. The new completion model improves scalability by allowing per-link completion processing across multiple cores and enables DIM-based interrupt moderation.
There are 2 fixes for SMC on the list waiting for review. It'd also be great to get someone to review this series, preferably from IBM. There are 7 reviewers/maintainers for this code. Pinging y'all is getting really tedious.
Mahanta Jambigi
maintainerQuoted 10 lines · Jakub Kicinski
Quoted 4 lines · On Thu, 16 Jul 2026 19:37:42 +0800 D. Wythe wrote:
This series transitions SMC-R completion handling to RDMA core CQ pooling via the
ib_cqeAPI. The new completion model improves scalability by allowing per-link completion processing across multiple cores and enables DIM-based interrupt moderation.There are 2 fixes for SMC on the list waiting for review. It'd also be great to get someone to review this series, preferably from IBM. There are 7 reviewers/maintainers for this code. Pinging y'all is getting really tedious.
Sorry for the delay, I was tied up with other work. I'm reviewing the series now.