Skip to contentSkip to conversation

net/smc: transition to RDMA core CQ pooling

PATCH In review D. Wythe 5 participants 15 messages Jul 16 – Jul 30

Subsystems (5)
lore
Share & tools
lore

D. Wythe

author maintainer
#0 lore raw

v4 cover net/smc: transition to RDMA core CQ pooling

This series transitions SMC-R completion handling to RDMA core CQ pooling via the ib_cqe API. The new completion model improves scalability by allowing per-link completion processing across multiple cores and enables DIM-based interrupt moderation.

The increased concurrency can amplify contention for TX slots on the shared wait queue, so the exclusive-wait change comes first as a standalone improvement that the CQ pooling teardown then builds on.

Patch 1 fixes smc_wr_tx_put_slot() to clear the v2 pending slot and buffer structures instead of the pointer variables.
Patch 2 reduces TX slot contention by switching TX slot allocation from non-exclusive wait_event() to prepare_to_wait_exclusive(). Patch 3 replaces the global per-device CQ and manual tasklet polling model with RDMA core CQ pooling.

---
Changes v1 -> v2:
https://lore.kernel.org/netdev/[email protected]/
1. remove unnecessary inline from static CQE init helpers.
2. Use ib_drain_qp() with +1 max_send_wr;
3. Fix v2 state clearing.
4. Add re-check after schedule_timeout() to fix timeout/signal races.

Changes v2 -> v3:
https://lore.kernel.org/netdev/[email protected]/
1. Reserve +3 instead of +1 for the SQ to cover the drain, FastReg and SMC-Rv2 SEND WRs, avoiding SQ exhaustion that breaks ib_drain_sq().
2. Guard the recv WR repost with a per-link percpu_ref so no WR is reposted after ib_drain_qp(), fixing the RX repost/drain use-after-free.
3. Split the smc_wr_tx_put_slot() v2 clearing fix into a separate patch (1/3) with a Fixes: tag and reworded to the verifiable root cause.

Changes v3 -> v4:
https://lore.kernel.org/netdev/[email protected]/
1. Reorder: move the exclusive-wait patch before the CQ pooling patch.
2. Fix teardown stall: reintroduce smc_ib_modify_qp_error() and flush the QP in smc_wr_stop_link() before waiting for the refs.
3. smc_wr_reg_send(): take the ref before ib_post_send().
4. smc_wr_tx_process_cqe(): wake_up_all() on a flush completion.
5. Merge the three per-link percpu_refs into a single wr_refs.

D. Wythe (3):
net/smc: clear the correct v2 slot and buffer in smc_wr_tx_put_slot() net/smc: reduce TX slot contention with exclusive wait net/smc: transition to RDMA core CQ pooling

 net/smc/smc_core.c |  10 +-
 net/smc/smc_core.h |  36 ++--
 net/smc/smc_ib.c   | 106 ++++-------
 net/smc/smc_ib.h   |   6 -
 net/smc/smc_tx.c   |   1 -
 net/smc/smc_wr.c   | 445 +++++++++++++++++++++------------------------
 net/smc/smc_wr.h   |  50 ++---
 7 files changed, 287 insertions(+), 367 deletions(-)
Signature · 1 line
2.45.0
author maintainer
#1 lore raw

PATCH v4 1/3 net/smc: clear the correct v2 slot and buffer in smc_wr_tx_put_slot()

smc_wr_tx_put_slot() tries to reset the v2 pending slot and buffer with memset(&link->wr_tx_v2_pend, 0, sizeof(link->wr_tx_v2_pend)) and the equivalent for wr_tx_buf_v2. Both are pointers, so this zeroes the 8-byte pointer variable instead of the structure it points to. The pending slot and buffer are therefore never actually cleared, and the pointers get overwritten with NULL.

Pass the pointers directly and use sizeof(*pointer) so the intended structures are cleared.

Fixes 8799e310fb3f ("net/smc: add v2 support to the work request layer")
---
 net/smc/smc_wr.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
net/smc/smc_wr.c +4 -4
index 59c92b46945c..6b5add922993 100644
@@ -288,10 +288,10 @@ int smc_wr_tx_put_slot(struct smc_link *link,
 	} else if (link->lgr->smc_version == SMC_V2 &&
 		   pend->idx == link->wr_tx_cnt) {
 		/* Large v2 buffer */
-		memset(&link->wr_tx_v2_pend, 0,
-		       sizeof(link->wr_tx_v2_pend));
-		memset(&link->lgr->wr_tx_buf_v2, 0,
-		       sizeof(link->lgr->wr_tx_buf_v2));
+		memset(link->wr_tx_v2_pend, 0,
+		       sizeof(*link->wr_tx_v2_pend));
+		memset(link->lgr->wr_tx_buf_v2, 0,
+		       sizeof(*link->lgr->wr_tx_buf_v2));
 		return 1;
 	}
 
Signature · 1 line
2.45.0
#2 lore raw

Re: [PATCH net-next v4 1/3] net/smc: clear the correct v2 slot and buffer in smc_wr_tx_put_slot()

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [High] Data race in V2 slot cleanup allowing a concurrent sender's data to be overwritten.
- [High] Unconditional zeroing of the shared lgr->wr_tx_buf_v2 in smc_wr_tx_process_cqe() on any link error corrupts in-progress LLC messages on other links. --

--- Patch [1]: [PATCH net-next v4 1/3] net/smc: clear the correct v2 slot and buffer in smc_wr_tx_put_slot() --- commit eb37bf5f30fd47da384e2535c48a0ad5576aa908
Author: D. Wythe <[email protected]>

net/smc: clear the correct v2 slot and buffer in smc_wr_tx_put_slot()

smc_wr_tx_put_slot() tries to reset the v2 pending slot and buffer with memset(&link->wr_tx_v2_pend, 0, sizeof(link->wr_tx_v2_pend)) and the equivalent for wr_tx_buf_v2. Both are pointers, so this zeroes the 8-byte pointer variable instead of the structure it points to. The pending slot and buffer are therefore never actually cleared, and the pointers get overwritten with NULL.

Pass the pointers directly and use sizeof(*pointer) so the intended structures are cleared.

net/smc/smc_wr.c int smc_wr_tx_put_slot(struct smc_link *link, · line 292

Quoted 12 lines
net/smc/smc_wr.c +2 -2
index 59c92b46945c5..6b5add9229937 100644
@@ -288,10 +288,10 @@ int smc_wr_tx_put_slot(struct smc_link *link,
 	} else if (link->lgr->smc_version == SMC_V2 &&
 		   pend->idx == link->wr_tx_cnt) {
 		/* Large v2 buffer */
-		memset(&link->wr_tx_v2_pend, 0,
-		       sizeof(link->wr_tx_v2_pend));
+		memset(link->wr_tx_v2_pend, 0,
+		       sizeof(*link->wr_tx_v2_pend));

[Severity: High]
This isn't a bug introduced by this patch, but does clearing the control structure before the data buffer create a race window?

If a thread sends an LLC message and waits for a response, the Rx tasklet might process the Rx CQE first, waking up the LLC flow thread which proceeds to send the next message by calling smc_wr_tx_get_v2_slot().

Concurrently, the Tx tasklet could process the Tx CQE and clear the slot control structure here. The zeroed idx immediately publishes the slot as free, so the LLC thread could acquire it and begin formatting its message in link->lgr->wr_tx_buf_v2.

The same sequence appears to occur in smc_wr_tx_process_cqe().

Would the following memset then wipe out the newly written message?

net/smc/smc_wr.c int smc_wr_tx_put_slot(struct smc_link *link, · line 294

Quoted 4 lines
net/smc/smc_wr.c +2 -2
-		memset(&link->lgr->wr_tx_buf_v2, 0,
-		       sizeof(link->lgr->wr_tx_buf_v2));
+		memset(link->lgr->wr_tx_buf_v2, 0,
+		       sizeof(*link->lgr->wr_tx_buf_v2));

[Severity: High]
This is a pre-existing issue, but could unconditionally zeroing the shared link->lgr->wr_tx_buf_v2 cause issues for other links in the LGR?

It looks like the same zeroing pattern happens in the error path of smc_wr_tx_process_cqe():

net/smc/smc_wr.c:smc_wr_tx_process_cqe() {
    ...
    if (unlikely(wc->status)) {
        ...
        if (link->lgr->smc_version == SMC_V2) {
            memset(link->wr_tx_v2_pend, 0, sizeof(*link->wr_tx_v2_pend));
            memset(link->lgr->wr_tx_buf_v2, 0, sizeof(*link->lgr->wr_tx_buf_v2));
        }
    }
}

If Link B is actively formatting an LLC message in this shared LGR buffer, and a normal data work request concurrently fails on Link A, would Link A's tasklet incorrectly wipe out Link B's in-progress LLC message?

Quoted 2 lines
net/smc/smc_wr.c +0 -0
 		return 1;
 	}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
maintainer Reviewed-by on v4 1/3
#3 lore raw

Re: [PATCH net-next v4 1/3] net/smc: clear the correct v2 slot and buffer in smc_wr_tx_put_slot()

In reply to D. Wythe
Quoted 26 lines · D. Wythe

smc_wr_tx_put_slot() tries to reset the v2 pending slot and buffer with memset(&link->wr_tx_v2_pend, 0, sizeof(link->wr_tx_v2_pend)) and the equivalent for wr_tx_buf_v2. Both are pointers, so this zeroes the 8-byte pointer variable instead of the structure it points to. The pending slot and buffer are therefore never actually cleared, and the pointers get overwritten with NULL.

Pass the pointers directly and use sizeof(*pointer) so the intended structures are cleared.

Fixes 8799e310fb3f ("net/smc: add v2 support to the work request layer")
---
 net/smc/smc_wr.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
net/smc/smc_wr.c +0 -1
index 59c92b46945c..6b5add922993 100644
@@ -288,10 +288,10 @@ int smc_wr_tx_put_slot(struct smc_link *link,
 	} else if (link->lgr->smc_version == SMC_V2 &&
 		   pend->idx == link->wr_tx_cnt) {
 		/* Large v2 buffer */
-		memset(&link->wr_tx_v2_pend, 0,

net/smc/smc_wr.c int smc_wr_tx_put_slot(struct smc_link *link, · line 297

Quoted 10 lines
net/smc/smc_wr.c +4 -3
-		       sizeof(link->wr_tx_v2_pend));
-		memset(&link->lgr->wr_tx_buf_v2, 0,
-		       sizeof(link->lgr->wr_tx_buf_v2));
+		memset(link->wr_tx_v2_pend, 0,
+		       sizeof(*link->wr_tx_v2_pend));
+		memset(link->lgr->wr_tx_buf_v2, 0,
+		       sizeof(*link->lgr->wr_tx_buf_v2));
 		return 1;
 	}
 
author maintainer
#4 lore raw

PATCH v4 2/3 net/smc: reduce TX slot contention with exclusive wait

In reply to D. Wythe

smc_wr_tx_get_free_slot() waits for a free TX slot with wait_event_interruptible_timeout(). Since the wait_event family enqueues waiters as non-exclusive, wake_up() may wake multiple waiters even though only one can use the slot, causing thundering-herd contention when slots are scarce.

Use an exclusive wait loop with prepare_to_wait_exclusive() so wake_up() wakes only one waiter per freed slot. smc_wr_wakeup_tx_wait() still uses wake_up_all() during link teardown, so teardown behavior is unchanged.

This also corrects the return value on a pending signal: the previous wait_event_interruptible_timeout() path fell through to the "no free slot" case and returned -EPIPE, masking the signal as a connection error. The open-coded loop now returns -ERESTARTSYS, matching the standard interruptible-wait semantics and letting the syscall restart machinery handle it.

---
 net/smc/smc_wr.c | 36 ++++++++++++++++++++++++++----------
 1 file changed, 26 insertions(+), 10 deletions(-)
net/smc/smc_wr.c +26 -10
index 6b5add922993..2cdd4063c13d 100644
@@ -198,11 +198,13 @@ int smc_wr_tx_get_free_slot(struct smc_link *link,
 			    struct smc_rdma_wr **wr_rdma_buf,
 			    struct smc_wr_tx_pend_priv **wr_pend_priv)
 {
+	unsigned long timeout = SMC_WR_TX_WAIT_FREE_SLOT_TIME;
 	struct smc_link_group *lgr = smc_get_lgr(link);
 	struct smc_wr_tx_pend *wr_pend;
 	u32 idx = link->wr_tx_cnt;
 	struct ib_send_wr *wr_ib;
 	u64 wr_id;
+	DEFINE_WAIT(wait);
 	int rc;
 
 	*wr_buf = NULL;
@@ -212,17 +214,31 @@ int smc_wr_tx_get_free_slot(struct smc_link *link,
 		if (rc)
 			return rc;
 	} else {
-		rc = wait_event_interruptible_timeout(
-			link->wr_tx_wait,
-			!smc_link_sendable(link) ||
-			lgr->terminating ||
-			(smc_wr_tx_get_free_slot_index(link, &idx) != -EBUSY),
-			SMC_WR_TX_WAIT_FREE_SLOT_TIME);
-		if (!rc) {
-			/* timeout - terminate link */
-			smcr_link_down_cond_sched(link);
-			return -EPIPE;
+		rc = 0;
+		for (;;) {
+			prepare_to_wait_exclusive(&link->wr_tx_wait, &wait,
+						  TASK_INTERRUPTIBLE);
+			if (!smc_link_sendable(link) || lgr->terminating ||
+			    smc_wr_tx_get_free_slot_index(link, &idx) != -EBUSY)
+				break;
+			timeout = schedule_timeout(timeout);
+			/* re-check */
+			if (!smc_link_sendable(link) || lgr->terminating ||
+			    smc_wr_tx_get_free_slot_index(link, &idx) != -EBUSY)
+				break;
+			if (!timeout) {
+				/* timeout - terminate link */
+				smcr_link_down_cond_sched(link);
+				break;
+			}
+			if (signal_pending(current)) {
+				rc = -ERESTARTSYS;
+				break;
+			}
 		}
+		finish_wait(&link->wr_tx_wait, &wait);
+		if (rc)
+			return rc;
 		if (idx == link->wr_tx_cnt)
 			return -EPIPE;
 	}
Signature · 1 line
2.45.0
reviewer Reviewed-by on v4 2/3
#5 lore raw

Re: [PATCH net-next v4 2/3] net/smc: reduce TX slot contention with exclusive wait

Quoted 18 lines · D. Wythe

smc_wr_tx_get_free_slot() waits for a free TX slot with wait_event_interruptible_timeout(). Since the wait_event family enqueues waiters as non-exclusive, wake_up() may wake multiple waiters even though only one can use the slot, causing thundering-herd contention when slots are scarce.

Use an exclusive wait loop with prepare_to_wait_exclusive() so wake_up() wakes only one waiter per freed slot. smc_wr_wakeup_tx_wait() still uses wake_up_all() during link teardown, so teardown behavior is unchanged.

This also corrects the return value on a pending signal: the previous wait_event_interruptible_timeout() path fell through to the "no free slot" case and returned -EPIPE, masking the signal as a connection error. The open-coded loop now returns -ERESTARTSYS, matching the standard interruptible-wait semantics and letting the syscall restart machinery handle it.

Quoted 2 lines

---

Thanks.

Quoted 63 lines
  net/smc/smc_wr.c | 36 ++++++++++++++++++++++++++----------
  1 file changed, 26 insertions(+), 10 deletions(-)
net/smc/smc_wr.c +26 -10
index 6b5add922993..2cdd4063c13d 100644
@@ -198,11 +198,13 @@ int smc_wr_tx_get_free_slot(struct smc_link *link,
  			    struct smc_rdma_wr **wr_rdma_buf,
  			    struct smc_wr_tx_pend_priv **wr_pend_priv)
  {
+	unsigned long timeout = SMC_WR_TX_WAIT_FREE_SLOT_TIME;
  	struct smc_link_group *lgr = smc_get_lgr(link);
  	struct smc_wr_tx_pend *wr_pend;
  	u32 idx = link->wr_tx_cnt;
  	struct ib_send_wr *wr_ib;
  	u64 wr_id;
+	DEFINE_WAIT(wait);
  	int rc;
 
  	*wr_buf = NULL;
@@ -212,17 +214,31 @@ int smc_wr_tx_get_free_slot(struct smc_link *link,
  		if (rc)
  			return rc;
  	} else {
-		rc = wait_event_interruptible_timeout(
-			link->wr_tx_wait,
-			!smc_link_sendable(link) ||
-			lgr->terminating ||
-			(smc_wr_tx_get_free_slot_index(link, &idx) != -EBUSY),
-			SMC_WR_TX_WAIT_FREE_SLOT_TIME);
-		if (!rc) {
-			/* timeout - terminate link */
-			smcr_link_down_cond_sched(link);
-			return -EPIPE;
+		rc = 0;
+		for (;;) {
+			prepare_to_wait_exclusive(&link->wr_tx_wait, &wait,
+						  TASK_INTERRUPTIBLE);
+			if (!smc_link_sendable(link) || lgr->terminating ||
+			    smc_wr_tx_get_free_slot_index(link, &idx) != -EBUSY)
+				break;
+			timeout = schedule_timeout(timeout);
+			/* re-check */
+			if (!smc_link_sendable(link) || lgr->terminating ||
+			    smc_wr_tx_get_free_slot_index(link, &idx) != -EBUSY)
+				break;
+			if (!timeout) {
+				/* timeout - terminate link */
+				smcr_link_down_cond_sched(link);
+				break;
+			}
+			if (signal_pending(current)) {
+				rc = -ERESTARTSYS;
+				break;
+			}
  		}
+		finish_wait(&link->wr_tx_wait, &wait);
+		if (rc)
+			return rc;
  		if (idx == link->wr_tx_cnt)
  			return -EPIPE;
  	}
author maintainer
#6 lore raw

PATCH v4 3/3 net/smc: transition to RDMA core CQ pooling

In reply to D. Wythe

The current SMC-R implementation relies on global per-device CQs and manual polling within tasklets, which introduces severe scalability bottlenecks due to global lock contention and tasklet scheduling overhead, resulting in poor performance as concurrency increases.

Refactor the completion handling to utilize the ib_cqe API and standard RDMA core CQ pooling. This transition provides several key advantages:

1. Multi-CQ: Shift from a single shared per-device CQ to multiple link-specific CQs via the CQ pool. This allows completion processing to be parallelized across multiple CPU cores, effectively eliminating the global CQ bottleneck.

2. Leverage DIM: Utilizing the standard CQ pool with IB_POLL_SOFTIRQ enables Dynamic Interrupt Moderation from the RDMA core, optimizing interrupt frequency and reducing CPU load under high pressure.

3. O(1) Context Retrieval: Replaces the expensive wr_id based lookup logic (e.g., smc_wr_tx_find_pending_index) with direct context retrieval using container_of() on the embedded ib_cqe.

4. Code Simplification: This refactoring results in a reduction of ~150 lines of code. It removes redundant sequence tracking, complex lookup helpers, and manual CQ management, significantly improving maintainability.

Performance Test: redis-benchmark with max 32 connections per QP Data format: Requests Per Second (RPS), Percentage in brackets represents the gain/loss compared to TCP.

| Clients | TCP      | SMC (original)      | SMC (cq_pool)       |
|---------|----------|---------------------|---------------------|
| c = 1   | 24449    | 31172  (+27%)       | 34039  (+39%)       |
| c = 2   | 46420    | 53216  (+14%)       | 64391  (+38%)       |
| c = 16  | 159673   | 83668  (-48%)  <--  | 216947 (+36%)       |
| c = 32  | 164956   | 97631  (-41%)  <--  | 249376 (+51%)       |
| c = 64  | 166322   | 118192 (-29%)  <--  | 249488 (+50%)       |
| c = 128 | 167700   | 121497 (-27%)  <--  | 249480 (+48%)       |
| c = 256 | 175021   | 146109 (-16%)  <--  | 240384 (+37%)       |
| c = 512 | 168987   | 101479 (-40%)  <--  | 226634 (+34%)       |

The results demonstrate that this optimization effectively resolves the scalability bottleneck, with RPS increasing by over 110% at c=64 compared to the original implementation.

---
 net/smc/smc_core.c |  10 +-
 net/smc/smc_core.h |  36 ++--
 net/smc/smc_ib.c   | 106 ++++--------
 net/smc/smc_ib.h   |   6 -
 net/smc/smc_tx.c   |   1 -
 net/smc/smc_wr.c   | 401 ++++++++++++++++++++-------------------------
 net/smc/smc_wr.h   |  50 ++----
 7 files changed, 257 insertions(+), 353 deletions(-)
net/smc/smc_core.c +3 -7
index cf6b620fef05..5073af92e086 100644
@@ -815,17 +815,11 @@ int smcr_link_init(struct smc_link_group *lgr, struct smc_link *lnk,
 	lnk->lgr = lgr;
 	smc_lgr_hold(lgr); /* lgr_put in smcr_link_clear() */
 	lnk->link_idx = link_idx;
-	lnk->wr_rx_id_compl = 0;
 	smc_ibdev_cnt_inc(lnk);
 	smcr_copy_dev_info_to_link(lnk);
 	atomic_set(&lnk->conn_cnt, 0);
 	smc_llc_link_set_uid(lnk);
 	INIT_WORK(&lnk->link_down_wrk, smc_link_down_work);
-	if (!lnk->smcibdev->initialized) {
-		rc = (int)smc_ib_setup_per_ibdev(lnk->smcibdev);
-		if (rc)
-			goto out;
-	}
 	get_random_bytes(rndvec, sizeof(rndvec));
 	lnk->psn_initial = rndvec[0] + (rndvec[1] << 8) +
 		(rndvec[2] << 16);
@@ -863,6 +857,7 @@ int smcr_link_init(struct smc_link_group *lgr, struct smc_link *lnk,
 	if (rc)
 		goto free_link_mem;
 	lnk->state = SMC_LNK_ACTIVATING;
+	smc_wr_init_cqes(lnk);
 	return 0;
 
 free_link_mem:
@@ -1373,7 +1368,8 @@ void smcr_link_clear(struct smc_link *lnk, bool log)
 	smc_llc_link_clear(lnk, log);
 	smcr_buf_unmap_lgr(lnk);
 	smcr_rtoken_clear_link(lnk);
-	smc_ib_modify_qp_error(lnk);
+	smc_wr_stop_link(lnk);
+	smc_wr_drain_qp(lnk);
 	smc_wr_free_link(lnk);
 	smc_ib_destroy_queue_pair(lnk);
 	smc_ib_dealloc_protection_domain(lnk);
net/smc/smc_core.h +22 -14
index 5c18f08a4c8a..1612a8012280 100644
@@ -89,8 +89,21 @@ struct smc_rdma_sges {				/* sges per message send */
 struct smc_rdma_wr {				/* work requests per message
 						 * send
 						 */
+	struct ib_cqe		cqe;
 	struct ib_rdma_wr	wr_tx_rdma[SMC_MAX_RDMA_WRITES];
-};
+} ____cacheline_aligned_in_smp;
+
+struct smc_ib_recv_wr {
+	struct ib_cqe		cqe;
+	struct ib_recv_wr	wr;
+	int idx;
+} ____cacheline_aligned_in_smp;
+
+struct smc_ib_send_wr {
+	struct ib_cqe		cqe;
+	struct ib_send_wr	wr;
+	int idx;
+} ____cacheline_aligned_in_smp;
 
 #define SMC_LGR_ID_SIZE		4
 
@@ -100,33 +113,34 @@ struct smc_link {
 	struct ib_pd		*roce_pd;	/* IB protection domain,
 						 * unique for every RoCE QP
 						 */
+	unsigned int		nr_cqe;		/* number of CQ entries */
+	struct ib_cq		*ib_cq;		/* IB completion queue */
 	struct ib_qp		*roce_qp;	/* IB queue pair */
 	struct ib_qp_attr	qp_attr;	/* IB queue pair attributes */
 
 	struct smc_wr_buf	*wr_tx_bufs;	/* WR send payload buffers */
-	struct ib_send_wr	*wr_tx_ibs;	/* WR send meta data */
+	struct smc_ib_send_wr	*wr_tx_ibs;	/* WR send meta data */
 	struct ib_sge		*wr_tx_sges;	/* WR send gather meta data */
 	struct smc_rdma_sges	*wr_tx_rdma_sges;/*RDMA WRITE gather meta data*/
 	struct smc_rdma_wr	*wr_tx_rdmas;	/* WR RDMA WRITE */
 	struct smc_wr_tx_pend	*wr_tx_pends;	/* WR send waiting for CQE */
 	struct completion	*wr_tx_compl;	/* WR send CQE completion */
 	/* above four vectors have wr_tx_cnt elements and use the same index */
-	struct ib_send_wr	*wr_tx_v2_ib;	/* WR send v2 meta data */
+	struct smc_ib_send_wr	*wr_tx_v2_ib;	/* WR send v2 meta data */
 	struct ib_sge		*wr_tx_v2_sge;	/* WR send v2 gather meta data*/
 	struct smc_wr_tx_pend	*wr_tx_v2_pend;	/* WR send v2 waiting for CQE */
 	dma_addr_t		wr_tx_dma_addr;	/* DMA address of wr_tx_bufs */
 	dma_addr_t		wr_tx_v2_dma_addr; /* DMA address of v2 tx buf*/
-	atomic_long_t		wr_tx_id;	/* seq # of last sent WR */
 	unsigned long		*wr_tx_mask;	/* bit mask of used indexes */
 	u32			wr_tx_cnt;	/* number of WR send buffers */
 	wait_queue_head_t	wr_tx_wait;	/* wait for free WR send buf */
 	struct {
-		struct percpu_ref	wr_tx_refs;
+		struct percpu_ref	wr_refs;	/* refs for all WR posting (tx/rx/reg) */
 	} ____cacheline_aligned_in_smp;
-	struct completion	tx_ref_comp;
+	struct completion	ref_comp;
 
 	u8			*wr_rx_bufs;	/* WR recv payload buffers */
-	struct ib_recv_wr	*wr_rx_ibs;	/* WR recv meta data */
+	struct smc_ib_recv_wr	*wr_rx_ibs;	/* WR recv meta data */
 	struct ib_sge		*wr_rx_sges;	/* WR recv scatter meta data */
 	/* above three vectors have wr_rx_cnt elements and use the same index */
 	int			wr_rx_sge_cnt; /* rx sge, V1 is 1, V2 is either 2 or 1 */
@@ -135,18 +149,12 @@ struct smc_link {
 						 */
 	dma_addr_t		wr_rx_dma_addr;	/* DMA address of wr_rx_bufs */
 	dma_addr_t		wr_rx_v2_dma_addr; /* DMA address of v2 rx buf*/
-	u64			wr_rx_id;	/* seq # of last recv WR */
-	u64			wr_rx_id_compl; /* seq # of last completed WR */
 	u32			wr_rx_cnt;	/* number of WR recv buffers */
 	unsigned long		wr_rx_tstamp;	/* jiffies when last buf rx */
-	wait_queue_head_t       wr_rx_empty_wait; /* wait for RQ empty */
 
 	struct ib_reg_wr	wr_reg;		/* WR register memory region */
+	struct ib_cqe		wr_reg_cqe;	/* ib_cqe for wr_reg */
 	wait_queue_head_t	wr_reg_wait;	/* wait for wr_reg result */
-	struct {
-		struct percpu_ref	wr_reg_refs;
-	} ____cacheline_aligned_in_smp;
-	struct completion	reg_ref_comp;
 	enum smc_wr_reg_state	wr_reg_state;	/* state of wr_reg request */
 
 	u8			gid[SMC_GID_SIZE];/* gid matching used vlan id*/
net/smc/smc_ib.c +34 -72

158 lines not on this page · raw message

net/smc/smc_ib.h +0 -6
index ef8ac2b7546d..0e4c6a14151a 100644
@@ -37,17 +37,12 @@ struct smc_ib_device {				/* ib-device infos for smc */
 	struct ib_device	*ibdev;
 	struct ib_port_attr	pattr[SMC_MAX_PORTS];	/* ib dev. port attrs */
 	struct ib_event_handler	event_handler;	/* global ib_event handler */
-	struct ib_cq		*roce_cq_send;	/* send completion queue */
-	struct ib_cq		*roce_cq_recv;	/* recv completion queue */
-	struct tasklet_struct	send_tasklet;	/* called by send cq handler */
-	struct tasklet_struct	recv_tasklet;	/* called by recv cq handler */
 	char			mac[SMC_MAX_PORTS][ETH_ALEN];
 						/* mac address per port*/
 	u8			pnetid[SMC_MAX_PORTS][SMC_MAX_PNETID_LEN];
 						/* pnetid per port */
 	bool			pnetid_by_user[SMC_MAX_PORTS];
 						/* pnetid defined by user? */
-	u8			initialized : 1; /* ib dev CQ, evthdl done */
 	struct work_struct	port_event_work;
 	unsigned long		port_event_mask;
 	DECLARE_BITMAP(ports_going_away, SMC_MAX_PORTS);
@@ -97,7 +92,6 @@ int smc_ib_create_queue_pair(struct smc_link *lnk);
 int smc_ib_ready_link(struct smc_link *lnk);
 int smc_ib_modify_qp_rts(struct smc_link *lnk);
 int smc_ib_modify_qp_error(struct smc_link *lnk);
-long smc_ib_setup_per_ibdev(struct smc_ib_device *smcibdev);
 int smc_ib_get_memory_region(struct ib_pd *pd, int access_flags,
 			     struct smc_buf_desc *buf_slot, u8 link_idx);
 void smc_ib_put_memory_region(struct ib_mr *mr);
net/smc/smc_tx.c +0 -1
index 3144b4b1fe29..d301df9ed58b 100644
@@ -321,7 +321,6 @@ static int smc_tx_rdma_write(struct smc_connection *conn, int peer_rmbe_offset,
 	struct smc_link *link = conn->lnk;
 	int rc;
 
-	rdma_wr->wr.wr_id = smc_wr_tx_get_next_wr_id(link);
 	rdma_wr->wr.num_sge = num_sges;
 	rdma_wr->remote_addr =
 		lgr->rtokens[conn->rtoken_idx][link->link_idx].dma_addr +
net/smc/smc_wr.c +181 -220

603 lines not on this page · raw message

net/smc/smc_wr.h +17 -33

93 lines not on this page · raw message

Signature · 1 line
2.45.0
maintainer
#7 lore raw
Quoted 15 lines · D. Wythe

Performance Test: redis-benchmark with max 32 connections per QP Data format: Requests Per Second (RPS), Percentage in brackets represents the gain/loss compared to TCP.

| Clients | TCP      | SMC (original)      | SMC (cq_pool)       |
|---------|----------|---------------------|---------------------|
| c = 1   | 24449    | 31172  (+27%)       | 34039  (+39%)       |
| c = 2   | 46420    | 53216  (+14%)       | 64391  (+38%)       |
| c = 16  | 159673   | 83668  (-48%)  <--  | 216947 (+36%)       |
| c = 32  | 164956   | 97631  (-41%)  <--  | 249376 (+51%)       |
| c = 64  | 166322   | 118192 (-29%)  <--  | 249488 (+50%)       |
| c = 128 | 167700   | 121497 (-27%)  <--  | 249480 (+48%)       |
| c = 256 | 175021   | 146109 (-16%)  <--  | 240384 (+37%)       |
| c = 512 | 168987   | 101479 (-40%)  <--  | 226634 (+34%)       |
Quoted 3 lines

The results demonstrate that this optimization effectively resolves the scalability bottleneck, with RPS increasing by over 110% at c=64 compared to the original implementation.

Thanks for the performance numbers — the scalability improvement at high client counts is striking. A few questions to help reproduce these results:

1) Was net.smc.smcr_max_conns_per_lgr=32 the value used? The "max 32 connections per QP" in the description suggests so, but the sysctl wasn't explicitly listed.

2) Were net.smc.smcr_max_send_wr and net.smc.smcr_max_recv_wr left at their defaults, or tuned? With per-link CQs the optimal WR counts may differ from the global-CQ case, so knowing whether these were touched would help interpret the numbers.

3) Was this SMC-R v2? The v2 TX path (smc_wr_tx_get_v2_slot, wr_tx_v2_ib) has different concurrency characteristics, so confirming the version matters for the contention analysis in patch 2/3 as well.

4) Was uperf tested for v4 as well, as we did for v1[1] in February-2026? uperf exercises a wider range of message sizes and thread counts, so its numbers would strengthen the case — particularly for the intermediate client counts (c=16 to c=64) where the original implementation regresses below TCP.

[1]
https://lore.kernel.org/netdev/[email protected]/

maintainer
#8 lore raw
Quoted 38 lines · Mahanta Jambigi
Quoted 15 lines · D. Wythe

Performance Test: redis-benchmark with max 32 connections per QP Data format: Requests Per Second (RPS), Percentage in brackets represents the gain/loss compared to TCP.

| Clients | TCP      | SMC (original)      | SMC (cq_pool)       |
|---------|----------|---------------------|---------------------|
| c = 1   | 24449    | 31172  (+27%)       | 34039  (+39%)       |
| c = 2   | 46420    | 53216  (+14%)       | 64391  (+38%)       |
| c = 16  | 159673   | 83668  (-48%)  <--  | 216947 (+36%)       |
| c = 32  | 164956   | 97631  (-41%)  <--  | 249376 (+51%)       |
| c = 64  | 166322   | 118192 (-29%)  <--  | 249488 (+50%)       |
| c = 128 | 167700   | 121497 (-27%)  <--  | 249480 (+48%)       |
| c = 256 | 175021   | 146109 (-16%)  <--  | 240384 (+37%)       |
| c = 512 | 168987   | 101479 (-40%)  <--  | 226634 (+34%)       |
Quoted 3 lines

The results demonstrate that this optimization effectively resolves the scalability bottleneck, with RPS increasing by over 110% at c=64 compared to the original implementation.

Thanks for the performance numbers — the scalability improvement at high client counts is striking. A few questions to help reproduce these results:

1) Was net.smc.smcr_max_conns_per_lgr=32 the value used? The "max 32 connections per QP" in the description suggests so, but the sysctl wasn't explicitly listed.

2) Were net.smc.smcr_max_send_wr and net.smc.smcr_max_recv_wr left at their defaults, or tuned? With per-link CQs the optimal WR counts may differ from the global-CQ case, so knowing whether these were touched would help interpret the numbers.

3) Was this SMC-R v2? The v2 TX path (smc_wr_tx_get_v2_slot, wr_tx_v2_ib) has different concurrency characteristics, so confirming the version matters for the contention analysis in patch 2/3 as well.

Quoted 5 lines

4) Was uperf tested for v4 as well, as we did for v1[1] in February-2026? uperf exercises a wider range of message sizes and thread counts, so its numbers would strengthen the case — particularly for the intermediate client counts (c=16 to c=64) where the original implementation regresses below TCP.

@D. Wythe: Gentle reminder regarding my earlier questions. Have you had a chance to look into them?

Quoted 3 lines

[1]
https://lore.kernel.org/netdev/[email protected]/

author maintainer
#9 lore raw
In reply to Mahanta Jambigi
Quoted 29 lines · Mahanta Jambigi
Quoted 15 lines · D. Wythe

Performance Test: redis-benchmark with max 32 connections per QP Data format: Requests Per Second (RPS), Percentage in brackets represents the gain/loss compared to TCP.

| Clients | TCP      | SMC (original)      | SMC (cq_pool)       |
|---------|----------|---------------------|---------------------|
| c = 1   | 24449    | 31172  (+27%)       | 34039  (+39%)       |
| c = 2   | 46420    | 53216  (+14%)       | 64391  (+38%)       |
| c = 16  | 159673   | 83668  (-48%)  <--  | 216947 (+36%)       |
| c = 32  | 164956   | 97631  (-41%)  <--  | 249376 (+51%)       |
| c = 64  | 166322   | 118192 (-29%)  <--  | 249488 (+50%)       |
| c = 128 | 167700   | 121497 (-27%)  <--  | 249480 (+48%)       |
| c = 256 | 175021   | 146109 (-16%)  <--  | 240384 (+37%)       |
| c = 512 | 168987   | 101479 (-40%)  <--  | 226634 (+34%)       |
Quoted 3 lines

The results demonstrate that this optimization effectively resolves the scalability bottleneck, with RPS increasing by over 110% at c=64 compared to the original implementation.

Thanks for the performance numbers — the scalability improvement at high client counts is striking. A few questions to help reproduce these results:

1) Was net.smc.smcr_max_conns_per_lgr=32 the value used? The "max 32 connections per QP" in the description suggests so, but the sysctl wasn't explicitly listed.

Quoted 8 lines

2) Were net.smc.smcr_max_send_wr and net.smc.smcr_max_recv_wr left at their defaults, or tuned? With per-link CQs the optimal WR counts may differ from the global-CQ case, so knowing whether these were touched would help interpret the numbers.

3) Was this SMC-R v2? The v2 TX path (smc_wr_tx_get_v2_slot, wr_tx_v2_ib) has different concurrency characteristics, so confirming the version matters for the contention analysis in patch 2/3 as well.

In our tests, we set net.smc.smcr_max_conns_per_lgr=32 via sysctl. We also tuned net.smc.smcr_max_send_wr and net.smc.smcr_max_recv_wr from their default to 64 to provide sufficient work request depth.

All tests were run with SMC-R v2.

And the performance numbers do need a refresh: with recent improvements—including thundering herd mitigation in earlier patches, the original SMC implementation’s regression in the c=16–512 range is expected to be largely resolved.

Quoted 6 lines

4) Was uperf tested for v4 as well, as we did for v1[1] in February-2026? uperf exercises a wider range of message sizes and thread counts, so its numbers would strengthen the case — particularly for the intermediate client counts (c=16 to c=64) where the original implementation regresses below TCP.

Of course uperf was tested. We’ll include a comprehensive uperf evaluation in the next version.

Given that this CQ-pooling patch may require additional review time, I plan to send the first two patches of the series separately. They’ve already been reviewed and are ready to merge. This patch will follow as a standalone submission once the uperf results are ready.

D. Wythe

Quoted 3 lines

[1]
https://lore.kernel.org/netdev/[email protected]/

maintainer
#10 lore raw
Quoted 43 lines · D. Wythe
Quoted 29 lines · Mahanta Jambigi
Quoted 15 lines · D. Wythe

Performance Test: redis-benchmark with max 32 connections per QP Data format: Requests Per Second (RPS), Percentage in brackets represents the gain/loss compared to TCP.

| Clients | TCP      | SMC (original)      | SMC (cq_pool)       |
|---------|----------|---------------------|---------------------|
| c = 1   | 24449    | 31172  (+27%)       | 34039  (+39%)       |
| c = 2   | 46420    | 53216  (+14%)       | 64391  (+38%)       |
| c = 16  | 159673   | 83668  (-48%)  <--  | 216947 (+36%)       |
| c = 32  | 164956   | 97631  (-41%)  <--  | 249376 (+51%)       |
| c = 64  | 166322   | 118192 (-29%)  <--  | 249488 (+50%)       |
| c = 128 | 167700   | 121497 (-27%)  <--  | 249480 (+48%)       |
| c = 256 | 175021   | 146109 (-16%)  <--  | 240384 (+37%)       |
| c = 512 | 168987   | 101479 (-40%)  <--  | 226634 (+34%)       |
Quoted 3 lines

The results demonstrate that this optimization effectively resolves the scalability bottleneck, with RPS increasing by over 110% at c=64 compared to the original implementation.

Thanks for the performance numbers — the scalability improvement at high client counts is striking. A few questions to help reproduce these results:

1) Was net.smc.smcr_max_conns_per_lgr=32 the value used? The "max 32 connections per QP" in the description suggests so, but the sysctl wasn't explicitly listed.

Quoted 8 lines

2) Were net.smc.smcr_max_send_wr and net.smc.smcr_max_recv_wr left at their defaults, or tuned? With per-link CQs the optimal WR counts may differ from the global-CQ case, so knowing whether these were touched would help interpret the numbers.

3) Was this SMC-R v2? The v2 TX path (smc_wr_tx_get_v2_slot, wr_tx_v2_ib) has different concurrency characteristics, so confirming the version matters for the contention analysis in patch 2/3 as well.

In our tests, we set net.smc.smcr_max_conns_per_lgr=32 via sysctl. We also tuned net.smc.smcr_max_send_wr and net.smc.smcr_max_recv_wr from their default to 64 to provide sufficient work request depth.

Quoted 5 lines

All tests were run with SMC-R v2.

And the performance numbers do need a refresh: with recent improvements—including thundering herd mitigation in earlier patches, the original SMC implementation’s regression in the c=16–512 range is expected to be largely resolved.

I believe that Patch 2/3, "net/smc: reduce TX slot contention with exclusive wait", largely mitigates the thundering herd effect.

Quoted 8 lines
Quoted 7 lines

4) Was uperf tested for v4 as well, as we did for v1[1] in February-2026? uperf exercises a wider range of message sizes and thread counts, so its numbers would strengthen the case — particularly for the intermediate client counts (c=16 to c=64) where the original implementation regresses below TCP.

Quoted 8 lines

Of course uperf was tested. We’ll include a comprehensive uperf evaluation in the next version.

Given that this CQ-pooling patch may require additional review time, I plan to send the first two patches of the series separately. They’ve already been reviewed and are ready to merge. This patch will follow as a standalone submission once the uperf results are ready.

Sounds good. I'll be looking forward to the standalone patch that introduces CQ pooling and the accompanying uperf results.

Quoted 6 lines

D. Wythe

Quoted 3 lines

[1]
https://lore.kernel.org/netdev/[email protected]/

reviewer
#11 lore raw
In reply to D. Wythe

net/smc/smc_ib.c static int smc_ib_add_dev(struct ib_device *ibdev) · line 955

Quoted 6 lines · D. Wythe

static void smc_copy_netdev_ifindex(struct smc_ib_device *smcibdev, int port)

net/smc/smc_ib.c +0 -1
@@ -952,7 +916,6 @@ static int smc_ib_add_dev(struct ib_device *ibdev)
  	INIT_WORK(&smcibdev->port_event_work, smc_ib_port_event_work);
  	atomic_set(&smcibdev->lnk_cnt, 0);
  	init_waitqueue_head(&smcibdev->lnks_deleted);
-	mutex_init(&smcibdev->mutex);

Since smc_ib_setup_per_ibdev() and smc_ib_cleanup_per_ibdev() were removed, the .mutex field is now unused and can be dropped.

Thanks.

Quoted 11 lines
net/smc/smc_ib.c +0 -1
  	mutex_lock(&smc_ib_devices.mutex);
  	list_add_tail(&smcibdev->list, &smc_ib_devices.list);
  	mutex_unlock(&smc_ib_devices.mutex);
@@ -1001,7 +964,6 @@ static void smc_ib_remove_dev(struct ib_device *ibdev, void *client_data)
  	pr_warn_ratelimited("smc: removing ib device %s\n",
  			    smcibdev->ibdev->name);
  	smc_smcr_terminate_all(smcibdev);
-	smc_ib_cleanup_per_ibdev(smcibdev);
  	ib_unregister_event_handler(&smcibdev->event_handler);
  	cancel_work_sync(&smcibdev->port_event_work);
  	kfree(smcibdev);
author maintainer
#12 lore raw
Quoted 13 lines · Wen Gu

net/smc/smc_ib.c static int smc_ib_add_dev(struct ib_device *ibdev) · line 955

Quoted 6 lines · D. Wythe

static void smc_copy_netdev_ifindex(struct smc_ib_device *smcibdev, int port)

net/smc/smc_ib.c +0 -1
@@ -952,7 +916,6 @@ static int smc_ib_add_dev(struct ib_device *ibdev)
 	INIT_WORK(&smcibdev->port_event_work, smc_ib_port_event_work);
 	atomic_set(&smcibdev->lnk_cnt, 0);
 	init_waitqueue_head(&smcibdev->lnks_deleted);
-	mutex_init(&smcibdev->mutex);

Since smc_ib_setup_per_ibdev() and smc_ib_cleanup_per_ibdev() were removed, the .mutex field is now unused and can be dropped.

Quoted 1 line

Thanks.

Good catch! The mutex is indeed unused now — will remove it in v5.

Quoted 13 lines
Quoted 12 lines
net/smc/smc_ib.c +0 -1
 	mutex_lock(&smc_ib_devices.mutex);
 	list_add_tail(&smcibdev->list, &smc_ib_devices.list);
 	mutex_unlock(&smc_ib_devices.mutex);
@@ -1001,7 +964,6 @@ static void smc_ib_remove_dev(struct ib_device *ibdev, void *client_data)
 	pr_warn_ratelimited("smc: removing ib device %s\n",
 			    smcibdev->ibdev->name);
 	smc_smcr_terminate_all(smcibdev);
-	smc_ib_cleanup_per_ibdev(smcibdev);
 	ib_unregister_event_handler(&smcibdev->event_handler);
 	cancel_work_sync(&smcibdev->port_event_work);
 	kfree(smcibdev);
maintainer
#13 lore raw
In reply to D. Wythe
Quoted 4 lines · On Thu, 16 Jul 2026 19:37:42 +0800 D. Wythe wrote:

This series transitions SMC-R completion handling to RDMA core CQ pooling via the ib_cqe API. The new completion model improves scalability by allowing per-link completion processing across multiple cores and enables DIM-based interrupt moderation.

There are 2 fixes for SMC on the list waiting for review. It'd also be great to get someone to review this series, preferably from IBM. There are 7 reviewers/maintainers for this code. Pinging y'all is getting really tedious.

maintainer
#14 lore raw
Quoted 10 lines · Jakub Kicinski
Quoted 4 lines · On Thu, 16 Jul 2026 19:37:42 +0800 D. Wythe wrote:

This series transitions SMC-R completion handling to RDMA core CQ pooling via the ib_cqe API. The new completion model improves scalability by allowing per-link completion processing across multiple cores and enables DIM-based interrupt moderation.

There are 2 fixes for SMC on the list waiting for review. It'd also be great to get someone to review this series, preferably from IBM. There are 7 reviewers/maintainers for this code. Pinging y'all is getting really tedious.

Sorry for the delay, I was tied up with other work. I'm reviewing the series now.