Skip to contentSkip to conversation

net/smc: transition to RDMA core CQ pooling

PATCH Superseded D. Wythe 3 participants 6 messages Jul 10 – Jul 14

Subsystems (5)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
author maintainer
#3 lore raw

PATCH v3 2/3 net/smc: transition to RDMA core CQ pooling

In reply to D. Wythe

The current SMC-R implementation relies on global per-device CQs and manual polling within tasklets, which introduces severe scalability bottlenecks due to global lock contention and tasklet scheduling overhead, resulting in poor performance as concurrency increases.

Refactor the completion handling to utilize the ib_cqe API and standard RDMA core CQ pooling. This transition provides several key advantages:

1. Multi-CQ: Shift from a single shared per-device CQ to multiple link-specific CQs via the CQ pool. This allows completion processing to be parallelized across multiple CPU cores, effectively eliminating the global CQ bottleneck.

2. Leverage DIM: Utilizing the standard CQ pool with IB_POLL_SOFTIRQ enables Dynamic Interrupt Moderation from the RDMA core, optimizing interrupt frequency and reducing CPU load under high pressure.

3. O(1) Context Retrieval: Replaces the expensive wr_id based lookup logic (e.g., smc_wr_tx_find_pending_index) with direct context retrieval using container_of() on the embedded ib_cqe.

4. Code Simplification: This refactoring results in a reduction of ~150 lines of code. It removes redundant sequence tracking, complex lookup helpers, and manual CQ management, significantly improving maintainability.

Performance Test: redis-benchmark with max 32 connections per QP Data format: Requests Per Second (RPS), Percentage in brackets represents the gain/loss compared to TCP.

| Clients | TCP      | SMC (original)      | SMC (cq_pool)       |
|---------|----------|---------------------|---------------------|
| c = 1   | 24449    | 31172  (+27%)       | 34039  (+39%)       |
| c = 2   | 46420    | 53216  (+14%)       | 64391  (+38%)       |
| c = 16  | 159673   | 83668  (-48%)  <--  | 216947 (+36%)       |
| c = 32  | 164956   | 97631  (-41%)  <--  | 249376 (+51%)       |
| c = 64  | 166322   | 118192 (-29%)  <--  | 249488 (+50%)       |
| c = 128 | 167700   | 121497 (-27%)  <--  | 249480 (+48%)       |
| c = 256 | 175021   | 146109 (-16%)  <--  | 240384 (+37%)       |
| c = 512 | 168987   | 101479 (-40%)  <--  | 226634 (+34%)       |

The results demonstrate that this optimization effectively resolves the scalability bottleneck, with RPS increasing by over 110% at c=64 compared to the original implementation.

---
 net/smc/smc_core.c |  10 +-
 net/smc/smc_core.h |  32 +++-
 net/smc/smc_ib.c   | 115 ++++----------
 net/smc/smc_ib.h   |   7 -
 net/smc/smc_tx.c   |   1 -
 net/smc/smc_wr.c   | 376 +++++++++++++++++++++------------------------
 net/smc/smc_wr.h   |  48 ++----
 7 files changed, 255 insertions(+), 334 deletions(-)
net/smc/smc_core.c +3 -7
index cf6b620fef05..5073af92e086 100644
@@ -815,17 +815,11 @@ int smcr_link_init(struct smc_link_group *lgr, struct smc_link *lnk,
 	lnk->lgr = lgr;
 	smc_lgr_hold(lgr); /* lgr_put in smcr_link_clear() */
 	lnk->link_idx = link_idx;
-	lnk->wr_rx_id_compl = 0;
 	smc_ibdev_cnt_inc(lnk);
 	smcr_copy_dev_info_to_link(lnk);
 	atomic_set(&lnk->conn_cnt, 0);
 	smc_llc_link_set_uid(lnk);
 	INIT_WORK(&lnk->link_down_wrk, smc_link_down_work);
-	if (!lnk->smcibdev->initialized) {
-		rc = (int)smc_ib_setup_per_ibdev(lnk->smcibdev);
-		if (rc)
-			goto out;
-	}
 	get_random_bytes(rndvec, sizeof(rndvec));
 	lnk->psn_initial = rndvec[0] + (rndvec[1] << 8) +
 		(rndvec[2] << 16);
@@ -863,6 +857,7 @@ int smcr_link_init(struct smc_link_group *lgr, struct smc_link *lnk,
 	if (rc)
 		goto free_link_mem;
 	lnk->state = SMC_LNK_ACTIVATING;
+	smc_wr_init_cqes(lnk);
 	return 0;
 
 free_link_mem:
@@ -1373,7 +1368,8 @@ void smcr_link_clear(struct smc_link *lnk, bool log)
 	smc_llc_link_clear(lnk, log);
 	smcr_buf_unmap_lgr(lnk);
 	smcr_rtoken_clear_link(lnk);
-	smc_ib_modify_qp_error(lnk);
+	smc_wr_stop_link(lnk);
+	smc_wr_drain_qp(lnk);
 	smc_wr_free_link(lnk);
 	smc_ib_destroy_queue_pair(lnk);
 	smc_ib_dealloc_protection_domain(lnk);
net/smc/smc_core.h +24 -8
index 5c18f08a4c8a..9645105ac7ed 100644
@@ -89,8 +89,21 @@ struct smc_rdma_sges {				/* sges per message send */
 struct smc_rdma_wr {				/* work requests per message
 						 * send
 						 */
+	struct ib_cqe		cqe;
 	struct ib_rdma_wr	wr_tx_rdma[SMC_MAX_RDMA_WRITES];
-};
+} ____cacheline_aligned_in_smp;
+
+struct smc_ib_recv_wr {
+	struct ib_cqe		cqe;
+	struct ib_recv_wr	wr;
+	int idx;
+} ____cacheline_aligned_in_smp;
+
+struct smc_ib_send_wr {
+	struct ib_cqe		cqe;
+	struct ib_send_wr	wr;
+	int idx;
+} ____cacheline_aligned_in_smp;
 
 #define SMC_LGR_ID_SIZE		4
 
@@ -100,23 +113,24 @@ struct smc_link {
 	struct ib_pd		*roce_pd;	/* IB protection domain,
 						 * unique for every RoCE QP
 						 */
+	unsigned int		nr_cqe;		/* number of CQ entries */
+	struct ib_cq		*ib_cq;		/* IB completion queue */
 	struct ib_qp		*roce_qp;	/* IB queue pair */
 	struct ib_qp_attr	qp_attr;	/* IB queue pair attributes */
 
 	struct smc_wr_buf	*wr_tx_bufs;	/* WR send payload buffers */
-	struct ib_send_wr	*wr_tx_ibs;	/* WR send meta data */
+	struct smc_ib_send_wr	*wr_tx_ibs;	/* WR send meta data */
 	struct ib_sge		*wr_tx_sges;	/* WR send gather meta data */
 	struct smc_rdma_sges	*wr_tx_rdma_sges;/*RDMA WRITE gather meta data*/
 	struct smc_rdma_wr	*wr_tx_rdmas;	/* WR RDMA WRITE */
 	struct smc_wr_tx_pend	*wr_tx_pends;	/* WR send waiting for CQE */
 	struct completion	*wr_tx_compl;	/* WR send CQE completion */
 	/* above four vectors have wr_tx_cnt elements and use the same index */
-	struct ib_send_wr	*wr_tx_v2_ib;	/* WR send v2 meta data */
+	struct smc_ib_send_wr	*wr_tx_v2_ib;	/* WR send v2 meta data */
 	struct ib_sge		*wr_tx_v2_sge;	/* WR send v2 gather meta data*/
 	struct smc_wr_tx_pend	*wr_tx_v2_pend;	/* WR send v2 waiting for CQE */
 	dma_addr_t		wr_tx_dma_addr;	/* DMA address of wr_tx_bufs */
 	dma_addr_t		wr_tx_v2_dma_addr; /* DMA address of v2 tx buf*/
-	atomic_long_t		wr_tx_id;	/* seq # of last sent WR */
 	unsigned long		*wr_tx_mask;	/* bit mask of used indexes */
 	u32			wr_tx_cnt;	/* number of WR send buffers */
 	wait_queue_head_t	wr_tx_wait;	/* wait for free WR send buf */
@@ -126,7 +140,7 @@ struct smc_link {
 	struct completion	tx_ref_comp;
 
 	u8			*wr_rx_bufs;	/* WR recv payload buffers */
-	struct ib_recv_wr	*wr_rx_ibs;	/* WR recv meta data */
+	struct smc_ib_recv_wr	*wr_rx_ibs;	/* WR recv meta data */
 	struct ib_sge		*wr_rx_sges;	/* WR recv scatter meta data */
 	/* above three vectors have wr_rx_cnt elements and use the same index */
 	int			wr_rx_sge_cnt; /* rx sge, V1 is 1, V2 is either 2 or 1 */
@@ -135,13 +149,15 @@ struct smc_link {
 						 */
 	dma_addr_t		wr_rx_dma_addr;	/* DMA address of wr_rx_bufs */
 	dma_addr_t		wr_rx_v2_dma_addr; /* DMA address of v2 rx buf*/
-	u64			wr_rx_id;	/* seq # of last recv WR */
-	u64			wr_rx_id_compl; /* seq # of last completed WR */
 	u32			wr_rx_cnt;	/* number of WR recv buffers */
 	unsigned long		wr_rx_tstamp;	/* jiffies when last buf rx */
-	wait_queue_head_t       wr_rx_empty_wait; /* wait for RQ empty */
+	struct {
+		struct percpu_ref	wr_rx_refs;
+	} ____cacheline_aligned_in_smp;
+	struct completion	rx_ref_comp;
 
 	struct ib_reg_wr	wr_reg;		/* WR register memory region */
+	struct ib_cqe		wr_reg_cqe;	/* ib_cqe for wr_reg */
 	wait_queue_head_t	wr_reg_wait;	/* wait for wr_reg result */
 	struct {
 		struct percpu_ref	wr_reg_refs;
net/smc/smc_ib.c +34 -81

173 lines not on this page · raw message

net/smc/smc_ib.h +0 -7
index ef8ac2b7546d..a75fe8bcef3a 100644
@@ -37,17 +37,12 @@ struct smc_ib_device {				/* ib-device infos for smc */
 	struct ib_device	*ibdev;
 	struct ib_port_attr	pattr[SMC_MAX_PORTS];	/* ib dev. port attrs */
 	struct ib_event_handler	event_handler;	/* global ib_event handler */
-	struct ib_cq		*roce_cq_send;	/* send completion queue */
-	struct ib_cq		*roce_cq_recv;	/* recv completion queue */
-	struct tasklet_struct	send_tasklet;	/* called by send cq handler */
-	struct tasklet_struct	recv_tasklet;	/* called by recv cq handler */
 	char			mac[SMC_MAX_PORTS][ETH_ALEN];
 						/* mac address per port*/
 	u8			pnetid[SMC_MAX_PORTS][SMC_MAX_PNETID_LEN];
 						/* pnetid per port */
 	bool			pnetid_by_user[SMC_MAX_PORTS];
 						/* pnetid defined by user? */
-	u8			initialized : 1; /* ib dev CQ, evthdl done */
 	struct work_struct	port_event_work;
 	unsigned long		port_event_mask;
 	DECLARE_BITMAP(ports_going_away, SMC_MAX_PORTS);
@@ -96,8 +91,6 @@ void smc_ib_destroy_queue_pair(struct smc_link *lnk);
 int smc_ib_create_queue_pair(struct smc_link *lnk);
 int smc_ib_ready_link(struct smc_link *lnk);
 int smc_ib_modify_qp_rts(struct smc_link *lnk);
-int smc_ib_modify_qp_error(struct smc_link *lnk);
-long smc_ib_setup_per_ibdev(struct smc_ib_device *smcibdev);
 int smc_ib_get_memory_region(struct ib_pd *pd, int access_flags,
 			     struct smc_buf_desc *buf_slot, u8 link_idx);
 void smc_ib_put_memory_region(struct ib_mr *mr);
net/smc/smc_tx.c +0 -1
index 3144b4b1fe29..d301df9ed58b 100644
@@ -321,7 +321,6 @@ static int smc_tx_rdma_write(struct smc_connection *conn, int peer_rmbe_offset,
 	struct smc_link *link = conn->lnk;
 	int rc;
 
-	rdma_wr->wr.wr_id = smc_wr_tx_get_next_wr_id(link);
 	rdma_wr->wr.num_sge = num_sges;
 	rdma_wr->remote_addr =
 		lgr->rtokens[conn->rtoken_idx][link->link_idx].dma_addr +
net/smc/smc_wr.c +178 -198

579 lines not on this page · raw message

net/smc/smc_wr.h +16 -32

91 lines not on this page · raw message

Signature · 1 line
2.45.0
author maintainer
#5 lore raw

PATCH v3 3/3 net/smc: reduce TX slot contention with exclusive wait

In reply to D. Wythe

smc_wr_tx_get_free_slot() waits for a free TX slot with wait_event_interruptible_timeout(). Since the wait_event family enqueues waiters as non-exclusive, wake_up() may wake multiple waiters even though only one can use the slot, causing thundering-herd contention when slots are scarce.

Use an exclusive wait loop with prepare_to_wait_exclusive() so wake_up() wakes only one waiter per freed slot. smc_wr_wakeup_tx_wait() still uses wake_up_all() during link teardown, so teardown behavior is unchanged.

This also corrects the return value on a pending signal: the previous wait_event_interruptible_timeout() path fell through to the "no free slot" case and returned -EPIPE, masking the signal as a connection error. The open-coded loop now returns -ERESTARTSYS, matching the standard interruptible-wait semantics and letting the syscall restart machinery handle it.

Performance measured with netperf TCP_RR (63 flows, 200B write / 1000B read, 60s duration):

+-------------------------------+---------------+---------------+
| smcr_max_conns_per_lgr        | 32            | 255           |
|-------------------------------+---------------+---------------|
| before                        | 4.85 Gb/s     | 657.95 Mb/s   |
|-------------------------------+---------------+---------------+
| after                         | 5.01 Gb/s     | 2.2 Gb/s      |
+-------------------------------+---------------+---------------+
---
 net/smc/smc_wr.c | 36 ++++++++++++++++++++++++++----------
 1 file changed, 26 insertions(+), 10 deletions(-)
net/smc/smc_wr.c +26 -10
index 675bfe633e39..dfa242549876 100644
@@ -153,9 +153,11 @@ int smc_wr_tx_get_free_slot(struct smc_link *link,
 			    struct smc_rdma_wr **wr_rdma_buf,
 			    struct smc_wr_tx_pend_priv **wr_pend_priv)
 {
+	unsigned long timeout = SMC_WR_TX_WAIT_FREE_SLOT_TIME;
 	struct smc_link_group *lgr = smc_get_lgr(link);
 	struct smc_wr_tx_pend *wr_pend;
 	u32 idx = link->wr_tx_cnt;
+	DEFINE_WAIT(wait);
 	int rc;
 
 	*wr_buf = NULL;
@@ -165,17 +167,31 @@ int smc_wr_tx_get_free_slot(struct smc_link *link,
 		if (rc)
 			return rc;
 	} else {
-		rc = wait_event_interruptible_timeout(
-			link->wr_tx_wait,
-			!smc_link_sendable(link) ||
-			lgr->terminating ||
-			(smc_wr_tx_get_free_slot_index(link, &idx) != -EBUSY),
-			SMC_WR_TX_WAIT_FREE_SLOT_TIME);
-		if (!rc) {
-			/* timeout - terminate link */
-			smcr_link_down_cond_sched(link);
-			return -EPIPE;
+		rc = 0;
+		for (;;) {
+			prepare_to_wait_exclusive(&link->wr_tx_wait, &wait,
+						  TASK_INTERRUPTIBLE);
+			if (!smc_link_sendable(link) || lgr->terminating ||
+			    smc_wr_tx_get_free_slot_index(link, &idx) != -EBUSY)
+				break;
+			timeout = schedule_timeout(timeout);
+			/* re-check */
+			if (!smc_link_sendable(link) || lgr->terminating ||
+			    smc_wr_tx_get_free_slot_index(link, &idx) != -EBUSY)
+				break;
+			if (!timeout) {
+				/* timeout - terminate link */
+				smcr_link_down_cond_sched(link);
+				break;
+			}
+			if (signal_pending(current)) {
+				rc = -ERESTARTSYS;
+				break;
+			}
 		}
+		finish_wait(&link->wr_tx_wait, &wait);
+		if (rc)
+			return rc;
 		if (idx == link->wr_tx_cnt)
 			return -EPIPE;
 	}
Signature · 1 line
2.45.0

D. Wythe

author maintainer
#0 lore raw

v3 cover net/smc: transition to RDMA core CQ pooling

This series transitions SMC-R completion handling to RDMA core CQ pooling via the ib_cqe API. The new completion model improves scalability by allowing per-link completion processing across multiple cores and enables DIM-based interrupt moderation.

As a side effect, the increased concurrency can amplify contention for TX slots on the shared wait queue. Patch 3 addresses this by switching TX slot allocation from non-exclusive wait_event() to prepare_to_wait_exclusive(), which avoids thundering-herd wakeups under contention.

Patch 1 fixes smc_wr_tx_put_slot() to clear the v2 pending slot and buffer structures instead of the pointer variables.
Patch 2 replaces the global per-device CQ and manual tasklet polling model with RDMA core CQ pooling.
Patch 3 reduces TX slot contention by using exclusive wait queue entries during allocation.

---
Changes v1 -> v2:
https://lore.kernel.org/netdev/[email protected]/
1. remove unnecessary inline from static CQE init helpers.
2. Use ib_drain_qp() with +1 max_send_wr;
3. Fix v2 state clearing.
4. Add re-check after schedule_timeout() to fix timeout/signal races.

Changes v2 -> v3:
https://lore.kernel.org/netdev/[email protected]/
1. Reserve +3 instead of +1 for the SQ to cover the drain, FastReg and SMC-Rv2 SEND WRs, avoiding SQ exhaustion that breaks ib_drain_sq().
2. Guard the recv WR repost with a per-link percpu_ref so no WR is reposted after ib_drain_qp(), fixing the RX repost/drain use-after-free.
3. Split the smc_wr_tx_put_slot() v2 clearing fix into a separate patch (1/3) with a Fixes: tag and reworded to the verifiable root cause.

D. Wythe (3):
net/smc: clear the correct v2 slot and buffer in smc_wr_tx_put_slot() net/smc: transition to RDMA core CQ pooling
net/smc: reduce TX slot contention with exclusive wait

 net/smc/smc_core.c |  10 +-
 net/smc/smc_core.h |  32 +++-
 net/smc/smc_ib.c   | 115 ++++---------
 net/smc/smc_ib.h   |   7 -
 net/smc/smc_tx.c   |   1 -
 net/smc/smc_wr.c   | 420 ++++++++++++++++++++++-----------------------
 net/smc/smc_wr.h   |  48 ++----
 7 files changed, 285 insertions(+), 348 deletions(-)
Signature · 1 line
2.45.0
author maintainer
#1 lore raw

PATCH v3 1/3 net/smc: clear the correct v2 slot and buffer in smc_wr_tx_put_slot()

smc_wr_tx_put_slot() tries to reset the v2 pending slot and buffer with memset(&link->wr_tx_v2_pend, 0, sizeof(link->wr_tx_v2_pend)) and the equivalent for wr_tx_buf_v2. Both are pointers, so this zeroes the 8-byte pointer variable instead of the structure it points to. The pending slot and buffer are therefore never actually cleared, and the pointers get overwritten with NULL.

Pass the pointers directly and use sizeof(*pointer) so the intended structures are cleared.

Fixes 8799e310fb3f ("net/smc: add v2 support to the work request layer")
---
 net/smc/smc_wr.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
net/smc/smc_wr.c +4 -4
index 59c92b46945c..6b5add922993 100644
@@ -288,10 +288,10 @@ int smc_wr_tx_put_slot(struct smc_link *link,
 	} else if (link->lgr->smc_version == SMC_V2 &&
 		   pend->idx == link->wr_tx_cnt) {
 		/* Large v2 buffer */
-		memset(&link->wr_tx_v2_pend, 0,
-		       sizeof(link->wr_tx_v2_pend));
-		memset(&link->lgr->wr_tx_buf_v2, 0,
-		       sizeof(link->lgr->wr_tx_buf_v2));
+		memset(link->wr_tx_v2_pend, 0,
+		       sizeof(*link->wr_tx_v2_pend));
+		memset(link->lgr->wr_tx_buf_v2, 0,
+		       sizeof(*link->lgr->wr_tx_buf_v2));
 		return 1;
 	}
 
Signature · 1 line
2.45.0
#4 lore raw
In reply to D. Wythe
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] Race condition in `smc_wr_reg_send` causes a Use-After-Free when posting a work request to a…

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Race condition in smc_wr_reg_send causes a Use-After-Free when posting a work request to a terminating link.
- [High] Link teardown stall/deadlock due to inverted QP destruction sequence.
--

--- Patch [2]: [PATCH net-next v3 2/3] net/smc: transition to RDMA core CQ pooling --- commit 7cb20cb1fe260499ce1bc5f3d7c63d07feedaebd
Author: D. Wythe <[email protected]>
Subject: net/smc: transition to RDMA core CQ pooling

Refactors completion handling to use the ib_cqe API and standard RDMA core CQ pooling. This parallelizes completion processing across multiple CPU cores and removes redundant manual CQ management logic.

net/smc/smc_core.c void smcr_link_clear(struct smc_link *lnk, bool log) · line 1372

Quoted 11 lines
net/smc/smc_core.c +2 -1
index cf6b620fef05f..5073af92e0868 100644
@@ -1373,7 +1368,8 @@ void smcr_link_clear(struct smc_link *lnk, bool log)
 	smc_llc_link_clear(lnk, log);
 	smcr_buf_unmap_lgr(lnk);
 	smcr_rtoken_clear_link(lnk);
-	smc_ib_modify_qp_error(lnk);
+	smc_wr_stop_link(lnk);
+	smc_wr_drain_qp(lnk);

[Severity: High]
Will this order of operations cause a multi-second stall during link teardown?

In smc_wr_stop_link, the teardown thread blocks on wait_for_completion for the sender refcounts to drop. However, the queue pair is no longer put into an error state before this wait, as that now happens in smc_wr_drain_qp after the wait completes.

Without the queue pair in an error state, completion queue events are not generated. The quick wakeups in smc_wr_stop_link like smc_wr_wakeup_reg_wait appear ineffective because the condition check (such as link->wr_reg_state != POSTED) will remain false.

Could this cause the teardown thread to stall until the senders hit their timeout?

Quoted 16 lines
net/smc/smc_core.c +0 -0
 	smc_wr_free_link(lnk);
 	smc_ib_destroy_queue_pair(lnk);
 	smc_ib_dealloc_protection_domain(lnk);
net/smc/smc_wr.c +0 -3
index 6b5add9229937..675bfe633e395 100644
[ ... ]
@@ -367,17 +306,16 @@ int smc_wr_reg_send(struct smc_link *link, struct ib_mr *mr)
 {
 	int rc;
 
-	ib_req_notify_cq(link->smcibdev->roce_cq_send,
-			 IB_CQ_NEXT_COMP | IB_CQ_REPORT_MISSED_EVENTS);
 	link->wr_reg_state = POSTED;
-	link->wr_reg.wr.wr_id = (u64)(uintptr_t)mr;

net/smc/smc_wr.c int smc_wr_reg_send(struct smc_link *link, struct ib_mr *mr) · line 318

Quoted 10 lines
net/smc/smc_wr.c +3 -1
 	link->wr_reg.mr = mr;
 	link->wr_reg.key = mr->rkey;
 	rc = ib_post_send(link->roce_qp, &link->wr_reg.wr, NULL);
 	if (rc)
 		return rc;
 
-	percpu_ref_get(&link->wr_reg_refs);
+	if (!percpu_ref_tryget_live(&link->wr_reg_refs))
+		return -EPIPE;
+

[Severity: High]
Can this lead to a use-after-free if the link is being torn down concurrently?

If percpu_ref_tryget_live fails, the function returns -EPIPE immediately and the caller may proceed to free the link memory. However, the work request has already been posted to the hardware queue via ib_post_send on the previous lines.

When the completion event eventually arrives, the completion handler smc_wr_reg_process_cqe executes:

smc_wr_reg_process_cqe()

        ...
        if (wc->status)
                link->wr_reg_state = FAILED;

If the link memory was already freed, won't writing to link->wr_reg_state corrupt memory?

Quoted 3 lines
net/smc/smc_wr.c +0 -0
 	rc = wait_event_interruptible_timeout(link->wr_reg_wait,
 					      (link->wr_reg_state != POSTED),
 					      SMC_WR_REG_MR_WAIT_TIME);
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
maintainer Reviewed-by on v3 1/3
#2 lore raw

Re: [PATCH net-next v3 1/3] net/smc: clear the correct v2 slot and buffer in smc_wr_tx_put_slot()

In reply to D. Wythe
Quoted 7 lines · D. Wythe

smc_wr_tx_put_slot() tries to reset the v2 pending slot and buffer with memset(&link->wr_tx_v2_pend, 0, sizeof(link->wr_tx_v2_pend)) and the equivalent for wr_tx_buf_v2. Both are pointers, so this zeroes the 8-byte pointer variable instead of the structure it points to. The pending slot and buffer are therefore never actually cleared, and the pointers get overwritten with NULL.

Quoted 5 lines

Pass the pointers directly and use sizeof(*pointer) so the intended structures are cleared.

Fixes 8799e310fb3f ("net/smc: add v2 support to the work request layer")

Best regards,
Dust

Quoted 25 lines

---
net/smc/smc_wr.c | 8 ++++----

1 file changed, 4 insertions(+), 4 deletions(-)
net/smc/smc_wr.c +4 -4
index 59c92b46945c..6b5add922993 100644
@@ -288,10 +288,10 @@ int smc_wr_tx_put_slot(struct smc_link *link,
 	} else if (link->lgr->smc_version == SMC_V2 &&
 		   pend->idx == link->wr_tx_cnt) {
 		/* Large v2 buffer */
-		memset(&link->wr_tx_v2_pend, 0,
-		       sizeof(link->wr_tx_v2_pend));
-		memset(&link->lgr->wr_tx_buf_v2, 0,
-		       sizeof(link->lgr->wr_tx_buf_v2));
+		memset(link->wr_tx_v2_pend, 0,
+		       sizeof(*link->wr_tx_v2_pend));
+		memset(link->lgr->wr_tx_buf_v2, 0,
+		       sizeof(*link->lgr->wr_tx_buf_v2));
 		return 1;
 	}
 
Signature · 1 line
2.45.0