Skip to contentSkip to conversation

rsockets: acknowledge completion queue events in batch

PATCH In review Sreedhar Kodali 2 participants 3 messages Sep 8, 2014 – Sep 18, 2014 2 messages not in the archive InfiniBand

lore
Share & tools
lore
#0 lore raw

PATCH v3 4/4 rsockets: acknowledge completion queue events in batch

Author: Sreedhar Kodali <[email protected]> Change note from v2 to v3: eliminated setting through option as suggested by Sean
Date: Mon Sep 8 15:38:54 2014 +0530

Perform completion event acknowledgments in batches instead of individually to minimze locking overheads. Size of the completion queue decides the size of a batch.

---

src/rsocket.c +11 -2
index ffea0ca..d201e0a 100644
@@ -384,6 +384,7 @@ struct rsocket {
 	dlist_entry	  iomap_list;
 	dlist_entry	  iomap_queue;
 	int		  iomap_pending;
+	int	      unack_cqe;
 };
 
 #define DS_UDP_TAG 0x55555555
@@ -638,6 +639,7 @@ static struct rsocket *rs_alloc(struct rsocket *inherited_rs, int type)
 			rs->target_iomap_size = def_iomap_size;
 		}
 	}
+	rs->unack_cqe = 0;
 	fastlock_init(&rs->slock);
 	fastlock_init(&rs->rlock);
 	fastlock_init(&rs->cq_lock);
@@ -1044,8 +1046,11 @@ static void rs_free(struct rsocket *rs)
 
 	if (rs->cm_id) {
 		rs_free_iomappings(rs);
-		if (rs->cm_id->qp)
+		if (rs->cm_id->qp) {
+			if (rs->unack_cqe > 0)
+				ibv_ack_cq_events(rs->cm_id->recv_cq, rs->unack_cqe);
 			rdma_destroy_qp(rs->cm_id);
+		}
 		rdma_destroy_id(rs->cm_id);
 	}
 
@@ -2026,7 +2031,11 @@ static int rs_get_cq_event(struct rsocket *rs)
 resume_get_cq_event:
 	ret = ibv_get_cq_event(rs->cm_id->recv_cq_channel, &cq, &context);
 	if (!ret) {
-		ibv_ack_cq_events(rs->cm_id->recv_cq, 1);
+		rs->unack_cqe += 1;
+		if (rs->unack_cqe == rs->sq_size + rs->rq_size) {
+			ibv_ack_cq_events(rs->cm_id->recv_cq, rs->unack_cqe);
+			rs->unack_cqe = 0;
+		}
 		rs->cq_armed = 0;
 	} else if (restart_onintr == 1 && errno == EINTR) {
 		errno = 0;

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): c34f0dcd116683c05c7c1836c381a09f-FJGp5E75HVmZamtmwQBW5tBPR1lH4CV8@public.gmane.org

A message that is not in the archive (sent off-list or not yet received): 1828884A29C6694DAF28B7E6B8A8237399DD4376-P5GAC/[email protected]

#4 lore raw
Quoted 2 lines · Sean

Can you please resend this patch as well? There's something off with the formatting.

Hi Sean,

I have resent the patch in v5. Also attached the patch file to message for convenience.

Thank You.

- Sreedhar

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html