Skip to contentSkip to conversation

RDMA/rxe: Fix rxe_cq_post

PATCH Applied Bob Pearson 3 participants 3 messages Jun 12, 2023

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 RDMA/rxe: Fix rxe_cq_post

A recent patch replaced a tasklet execution of cq->comp_handler by a direct call. While this made sense it let changes to cq->notify state be unprotected and assumed that the cq completion machinery and the ulp done callbacks were reentrant. The result is that in some cases completion events can be lost. This patch moves the cq->comp_handler call inside of the spinlock in rxe_cq_post which solves both issues. This is compatible with the matching code in the request notify verb.

Fixes 78b26a335310 ("RDMA/rxe: Remove tasklet call from rxe_cq.c")
---
 drivers/infiniband/sw/rxe/rxe_cq.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/infiniband/sw/rxe/rxe_cq.c +2 -2
index 20ff0c0c4605..6ca2a05b6a2a 100644
@@ -113,8 +113,6 @@ int rxe_cq_post(struct rxe_cq *cq, struct rxe_cqe *cqe, int solicited)
 
 	queue_advance_producer(cq->queue, QUEUE_TYPE_TO_CLIENT);
 
-	spin_unlock_irqrestore(&cq->cq_lock, flags);
-
 	if ((cq->notify == IB_CQ_NEXT_COMP) ||
 	    (cq->notify == IB_CQ_SOLICITED && solicited)) {
 		cq->notify = 0;
@@ -122,6 +120,8 @@ int rxe_cq_post(struct rxe_cq *cq, struct rxe_cqe *cqe, int solicited)
 		cq->ibcq.comp_handler(&cq->ibcq, cq->ibcq.cq_context);
 	}
 
+	spin_unlock_irqrestore(&cq->cq_lock, flags);
+
 	return 0;
 }
 

base-commit: 830f93f47068b1632cc127871fbf27e918efdf46

Signature · 1 line
2.39.2
maintainer Applied
#1 lore raw
Quoted 9 lines · Bob Pearson

A recent patch replaced a tasklet execution of cq->comp_handler by a direct call. While this made sense it let changes to cq->notify state be unprotected and assumed that the cq completion machinery and the ulp done callbacks were reentrant. The result is that in some cases completion events can be lost. This patch moves the cq->comp_handler call inside of the spinlock in rxe_cq_post which solves both issues. This is compatible with the matching code in the request notify verb.

Quoted 5 lines
Fixes 78b26a335310 ("RDMA/rxe: Remove tasklet call from rxe_cq.c")
---
 drivers/infiniband/sw/rxe/rxe_cq.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Applied to for-rc, thanks

Jason

#2 lore raw

Jason, thanks. - Bob

-----Original Message-----
From: Jason Gunthorpe <[email protected]>
Sent: Monday, June 12, 2023 1:28 PM
To: Bob Pearson <[email protected]>
Cc: [email protected]; [email protected]
Subject: Re: [PATCH for-next] RDMA/rxe: Fix rxe_cq_post

Quoted 9 lines · Bob Pearson

A recent patch replaced a tasklet execution of cq->comp_handler by a direct call. While this made sense it let changes to cq->notify state be unprotected and assumed that the cq completion machinery and the ulp done callbacks were reentrant. The result is that in some cases completion events can be lost. This patch moves the cq->comp_handler call inside of the spinlock in rxe_cq_post which solves both issues. This is compatible with the matching code in the request notify verb.

Quoted 5 lines
Fixes 78b26a335310 ("RDMA/rxe: Remove tasklet call from rxe_cq.c")
---
 drivers/infiniband/sw/rxe/rxe_cq.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Applied to for-rc, thanks

Jason