Skip to contentSkip to conversation

RPING: Make sure CQ event thread exits before destroying the CQ.

PATCH In review Steve Wise 2 participants 4 messages Oct 25, 2010 – Nov 1, 2010 2 messages not in the archive InfiniBand

lore
Share & tools
lore

Steve Wise

author
#0 lore raw

PATCH v2 1/1 RPING: Make sure CQ event thread exits before destroying the CQ.

It is possible for the CQ event thread to poll the CQ after it has been destroyed which can result in a seg fault on T3 interfaces. This patch waits for the thread to exit before destroying the CQ.

---

 examples/rping.c |   10 +++++-----
 1 files changed, 5 insertions(+), 5 deletions(-)
examples/rping.c +5 -5
index b58ea74..a8af4d6 100644
@@ -280,12 +280,11 @@ static int rping_cq_event_handler(struct rping_cb *cb)
 		ret = 0;
 
 		if (wc.status) {
-			if (wc.status != IBV_WC_WR_FLUSH_ERR) {
+			if (wc.status != IBV_WC_WR_FLUSH_ERR)
 				fprintf(stderr, 
 					"cq completion failed status %d\n",
 					wc.status);
-				ret = -1;
-			}
+			ret = -1;
 			goto error;
 		}
 
@@ -802,10 +801,9 @@ static void *rping_persistent_server_thread(void *arg)
 
 	rping_test_server(cb);
 	rdma_disconnect(cb->child_cm_id);
+	pthread_join(cb->cqthread, NULL);
 	rping_free_buffers(cb);
 	rping_free_qp(cb);
-	pthread_cancel(cb->cqthread);
-	pthread_join(cb->cqthread, NULL);
 	rdma_destroy_id(cb->child_cm_id);
 	free_cb(cb);
 	return NULL;
@@ -890,6 +888,7 @@ static int rping_run_server(struct rping_cb *cb)
 
 	rping_test_server(cb);
 	rdma_disconnect(cb->child_cm_id);
+	pthread_join(cb->cqthread, NULL);
 	rdma_destroy_id(cb->child_cm_id);
 err2:
 	rping_free_buffers(cb);
@@ -1057,6 +1056,7 @@ static int rping_run_client(struct rping_cb *cb)
 
 	rping_test_client(cb);
 	rdma_disconnect(cb->cm_id);
+	pthread_join(cb->cqthread, NULL);
 err2:
 	rping_free_buffers(cb);
 err1:

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20101025194309.11127.29600.stgit-T4OLL4TyM9aNDNWfRnPdfg@public.gmane.org

author
#2 lore raw

Hey Sean,

Thoughts on this?

I did quick rping tests on mthca, mlx4 and T3. Tested out ok.

Steve.

Quoted 59 lines · Steve Wise

It is possible for the CQ event thread to poll the CQ after it has been destroyed which can result in a seg fault on T3 interfaces. This patch waits for the thread to exit before destroying the CQ.

---

  examples/rping.c |   10 +++++-----
  1 files changed, 5 insertions(+), 5 deletions(-)
examples/rping.c +5 -6
index b58ea74..a8af4d6 100644
@@ -280,12 +280,11 @@ static int rping_cq_event_handler(struct rping_cb *cb)
  		ret = 0;
 
  		if (wc.status) {
-			if (wc.status != IBV_WC_WR_FLUSH_ERR) {
+			if (wc.status != IBV_WC_WR_FLUSH_ERR)
  				fprintf(stderr,
  					"cq completion failed status %d\n",
  					wc.status);
-				ret = -1;
-			}
+			ret = -1;
  			goto error;
  		}
 
@@ -802,10 +801,9 @@ static void *rping_persistent_server_thread(void *arg)
 
  	rping_test_server(cb);
  	rdma_disconnect(cb->child_cm_id);
+	pthread_join(cb->cqthread, NULL);
  	rping_free_buffers(cb);
  	rping_free_qp(cb);
-	pthread_cancel(cb->cqthread);
-	pthread_join(cb->cqthread, NULL);
  	rdma_destroy_id(cb->child_cm_id);
  	free_cb(cb);
  	return NULL;
@@ -890,6 +888,7 @@ static int rping_run_server(struct rping_cb *cb)
 
  	rping_test_server(cb);
  	rdma_disconnect(cb->child_cm_id);
+	pthread_join(cb->cqthread, NULL);
  	rdma_destroy_id(cb->child_cm_id);
  err2:
  	rping_free_buffers(cb);
@@ -1057,6 +1056,7 @@ static int rping_run_client(struct rping_cb *cb)
 
  	rping_test_client(cb);
  	rdma_disconnect(cb->cm_id);
+	pthread_join(cb->cqthread, NULL);
  err2:
  	rping_free_buffers(cb);
  err1:
 
--
Quoted 4 lines

To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]