Bugfixes for send queue overflow by heartbeat
Subsystems (2)
Jack Wang
maintainerv2 cover Bugfixes for send queue overflow by heartbeat
Hi Jason, hi Doug,
Please consider to include following changes to upstream.
This patchset fix a regression since b38041d50add ("RDMA/rtrs: Do not signal for heatbeat").
In commit b38041d50add, the signal flag is droped to fix the send queue full logic, but introduced a worse bug the send queue overflow on both clt and srv by heartbeat, sorry.
The patchset is orgnized as:
- patch1 debug patch.
- patch2 preparation.
- patch3 signal both IO and heartbeat.
- patch4 cleanup.
- patch5 cleanup
- patch6 move sq_wr_avail to account send queue full correctly.
The patches are created base v5.14-rc1.
Since v1:
* rebased to latest v5.14-rc1, target rc instread of for-next.
v1: https://lore.kernel.org/linux-rdma/[email protected]/T/#t
Jack Wang (6):
RDMA/rtrs: Add error messages for failed operations. RDMA/rtrs: move wr_cnt from rtrs_srv_con to rtrs_con RDMA/rtrs: Enable the same selective signal for heartbeat and IO RDMA/rtrs: Make rtrs_post_rdma_write_imm_empty static RDMA/rtrs: Remove unused flags parameter
RDMA/rtrs: Move sq_wr_avail to rtrs_con
drivers/infiniband/ulp/rtrs/rtrs-clt.c | 11 ++++++++--- drivers/infiniband/ulp/rtrs/rtrs-clt.h | 1 - drivers/infiniband/ulp/rtrs/rtrs-pri.h | 6 +++--- drivers/infiniband/ulp/rtrs/rtrs-srv.c | 19 ++++++++++--------- drivers/infiniband/ulp/rtrs/rtrs-srv.h | 2 -- drivers/infiniband/ulp/rtrs/rtrs.c | 23 ++++++++++++++++------- 6 files changed, 37 insertions(+), 25 deletions(-)
Signature · 1 line
2.25.1
Jack Wang
maintainerPATCH v2 1/6 RDMA/rtrs: Add error messages for failed operations.
It could help debugging in case of error happens.
--- drivers/infiniband/ulp/rtrs/rtrs.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/infiniband/ulp/rtrs/rtrs.c +3 -0
@@ -316,6 +316,7 @@ void rtrs_send_hb_ack(struct rtrs_sess *sess) err =rtrs_post_rdma_write_imm_empty (usr_con, sess->hb_cqe, imm,0 ,NULL );if (err) { +rtrs_err (sess,"send HB ACK failed, errno: %d\n" , err); sess->hb_err_handler (usr_con);return ; }
@@ -333,6 +334,7 @@ static void hb_work(struct work_struct *work) usr_con = sess->con[0 ];if (sess->hb_missed_cnt > sess->hb_missed_max) { +rtrs_err (sess,"HB missed max reached.\n" ); sess->hb_err_handler (usr_con);return ; }
@@ -348,6 +350,7 @@ static void hb_work(struct work_struct *work) err =rtrs_post_rdma_write_imm_empty (usr_con, sess->hb_cqe, imm,0 ,NULL );if (err) { +rtrs_err (sess,"HB send failed, errno: %d\n" , err); sess->hb_err_handler (usr_con);return ; }
Signature · 1 line
2.25.1
Jack Wang
maintainerPATCH v2 2/6 RDMA/rtrs: move wr_cnt from rtrs_srv_con to rtrs_con
In reply to Jack WangWe need to track also the wr used for heatbeat. This is a preparation for that, will be used in later patch.
The io_cnt in rtrs_clt is removed, use wr_cnt instead.
--- drivers/infiniband/ulp/rtrs/rtrs-clt.c | 7 ++++--- drivers/infiniband/ulp/rtrs/rtrs-clt.h | 1 - drivers/infiniband/ulp/rtrs/rtrs-pri.h | 1 + drivers/infiniband/ulp/rtrs/rtrs-srv.c | 6 +++--- drivers/infiniband/ulp/rtrs/rtrs-srv.h | 1 - 5 files changed, 8 insertions(+), 8 deletions(-)
drivers/infiniband/ulp/rtrs/rtrs-clt.c +4 -3
@@ -478,7 +478,7 @@ static int rtrs_post_send_rdma(struct rtrs_clt_con *con,* From time to time we have to post signalled sends, * or send queue will fill up and only QP reset can help. */ - flags =atomic_inc_return (&con->io_cnt) % sess->queue_depth ? + flags =atomic_inc_return (&con->c.wr_cnt) % sess->queue_depth ?0 :IB_SEND_SIGNALED ; ib_dma_sync_single_for_device(sess->s.dev->ib_dev, req->iu->dma_addr,
@@ -1043,7 +1043,7 @@ static int rtrs_post_rdma_write_sg(struct rtrs_clt_con *con,* From time to time we have to post signalled sends, * or send queue will fill up and only QP reset can help. */ - flags =atomic_inc_return (&con->io_cnt) % sess->queue_depth ? + flags =atomic_inc_return (&con->c.wr_cnt) % sess->queue_depth ?0 :IB_SEND_SIGNALED ; ib_dma_sync_single_for_device(sess->s.dev->ib_dev, req->iu->dma_addr,
@@ -1601,7 +1601,8 @@ static int create_con(struct rtrs_clt_sess *sess, unsigned int cid) con->cpu = (cid ? cid -1 :0 ) % nr_cpu_ids; con->c.cid = cid; con->c.sess = &sess->s; -atomic_set (&con->io_cnt,0 ); +/* Align with srv, init as 1 */ +atomic_set (&con->c.wr_cnt,1 );mutex_init (&con->con_mutex); sess->s.con[cid] = &con->c;
drivers/infiniband/ulp/rtrs/rtrs-clt.h +0 -1
@@ -74,7 +74,6 @@ struct rtrs_clt_con {u32 queue_num;unsigned int cpu;struct mutex con_mutex; -atomic_t io_cnt;int cm_err; };
drivers/infiniband/ulp/rtrs/rtrs-pri.h +1 -0
@@ -96,6 +96,7 @@ struct rtrs_con {struct rdma_cm_id *cm_id;unsigned int cid;int nr_cqe; +atomic_t wr_cnt; };struct rtrs_sess {
drivers/infiniband/ulp/rtrs/rtrs-srv.c +3 -3
@@ -269,7 +269,7 @@ static int rdma_write_sg(struct rtrs_srv_op *id)* From time to time we have to post signaled sends, * or send queue will fill up and only QP reset can help. */ - flags = (atomic_inc_return (&id->con->wr_cnt) % srv->queue_depth) ? + flags = (atomic_inc_return (&id->con->c.wr_cnt) % srv->queue_depth) ?0 :IB_SEND_SIGNALED ;if (need_inval) {
@@ -396,7 +396,7 @@ static int send_io_resp_imm(struct rtrs_srv_con *con, struct rtrs_srv_op *id,* From time to time we have to post signalled sends, * or send queue will fill up and only QP reset can help. */ - flags = (atomic_inc_return (&con->wr_cnt) % srv->queue_depth) ? + flags = (atomic_inc_return (&con->c.wr_cnt) % srv->queue_depth) ?0 :IB_SEND_SIGNALED ; imm =rtrs_to_io_rsp_imm (id->msg_id, errno, need_inval); imm_wr.wr.next =NULL ;
@@ -1648,7 +1648,7 @@ static int create_con(struct rtrs_srv_sess *sess, con->c.cm_id = cm_id; con->c.sess = &sess->s; con->c.cid = cid; -atomic_set (&con->wr_cnt,1 ); +atomic_set (&con->c.wr_cnt,1 ); wr_limit = sess->s.dev->ib_dev->attrs.max_qp_wr;if (con->c.cid ==0 ) {
drivers/infiniband/ulp/rtrs/rtrs-srv.h +0 -1
@@ -42,7 +42,6 @@ struct rtrs_srv_stats {struct rtrs_srv_con {struct rtrs_con c; -atomic_t wr_cnt;atomic_t sq_wr_avail;struct list_head rsp_wr_wait_list;spinlock_t rsp_wr_wait_lock;
Signature · 1 line
2.25.1
Jack Wang
maintainerPATCH v2 3/6 RDMA/rtrs: Enable the same selective signal for heartbeat and IO
In reply to Jack WangOn idle session, because we do not do signal for heartbeat, it will overflow the send queue after sometime.
To avoid that, we need to enable the signal for heartbeat. To do that, add a new member signal_interval in rtrs_path, which will set min of queue_depth and SERVICE_CON_QUEUE_DEPTH, and track it for both heartbeat and IO, so the sq queue full accounting is correct.
--- drivers/infiniband/ulp/rtrs/rtrs-clt.c | 7 +++++-- drivers/infiniband/ulp/rtrs/rtrs-pri.h | 1 + drivers/infiniband/ulp/rtrs/rtrs-srv.c | 11 ++++++----- drivers/infiniband/ulp/rtrs/rtrs.c | 7 ++++++- 4 files changed, 18 insertions(+), 8 deletions(-)
drivers/infiniband/ulp/rtrs/rtrs-clt.c +5 -2
@@ -478,7 +478,7 @@ static int rtrs_post_send_rdma(struct rtrs_clt_con *con,* From time to time we have to post signalled sends, * or send queue will fill up and only QP reset can help. */ - flags =atomic_inc_return (&con->c.wr_cnt) % sess->queue_depth ? + flags =atomic_inc_return (&con->c.wr_cnt) % sess->s.signal_interval ?0 :IB_SEND_SIGNALED ; ib_dma_sync_single_for_device(sess->s.dev->ib_dev, req->iu->dma_addr,
@@ -680,6 +680,7 @@ static void rtrs_clt_rdma_done(struct ib_cq *cq, struct ib_wc *wc)case IB_WC_RDMA_WRITE :/* * post_send() RDMA write completions of IO reqs (read/write) +* and hb. */ break ;
@@ -1043,7 +1044,7 @@ static int rtrs_post_rdma_write_sg(struct rtrs_clt_con *con,* From time to time we have to post signalled sends, * or send queue will fill up and only QP reset can help. */ - flags =atomic_inc_return (&con->c.wr_cnt) % sess->queue_depth ? + flags =atomic_inc_return (&con->c.wr_cnt) % sess->s.signal_interval ?0 :IB_SEND_SIGNALED ; ib_dma_sync_single_for_device(sess->s.dev->ib_dev, req->iu->dma_addr,
@@ -1849,6 +1850,8 @@ static int rtrs_rdma_conn_established(struct rtrs_clt_con *con,return -ENOMEM ; } sess->queue_depth = queue_depth; + sess->s.signal_interval =min_not_zero (queue_depth, + (unsigned short )SERVICE_CON_QUEUE_DEPTH ); sess->max_hdr_size =le32_to_cpu (msg->max_hdr_size); sess->max_io_size =le32_to_cpu (msg->max_io_size); sess->flags =le32_to_cpu (msg->flags);
drivers/infiniband/ulp/rtrs/rtrs-pri.h +1 -0
@@ -109,6 +109,7 @@ struct rtrs_sess {unsigned int con_num;unsigned int irq_con_num;unsigned int recon_cnt; +unsigned int signal_interval;struct rtrs_ib_dev *dev;int dev_ref;struct ib_cqe *hb_cqe;
drivers/infiniband/ulp/rtrs/rtrs-srv.c +6 -5
@@ -201,7 +201,6 @@ static int rdma_write_sg(struct rtrs_srv_op *id)struct rtrs_srv_sess *sess =to_srv_sess (s);dma_addr_t dma_addr = sess->dma_addr[id->msg_id];struct rtrs_srv_mr *srv_mr; -struct rtrs_srv *srv = sess->srv;struct ib_send_wr inv_wr;struct ib_rdma_wr imm_wr;struct ib_rdma_wr *wr =NULL ;
@@ -269,7 +268,7 @@ static int rdma_write_sg(struct rtrs_srv_op *id)* From time to time we have to post signaled sends, * or send queue will fill up and only QP reset can help. */ - flags = (atomic_inc_return (&id->con->c.wr_cnt) % srv->queue_depth) ? + flags = (atomic_inc_return (&id->con->c.wr_cnt) % s->signal_interval) ?0 :IB_SEND_SIGNALED ;if (need_inval) {
@@ -347,7 +346,6 @@ static int send_io_resp_imm(struct rtrs_srv_con *con, struct rtrs_srv_op *id,struct ib_send_wr inv_wr, *wr =NULL ;struct ib_rdma_wr imm_wr;struct ib_reg_wr rwr; -struct rtrs_srv *srv = sess->srv;struct rtrs_srv_mr *srv_mr;bool need_inval = false;enum ib_send_flags flags;
@@ -396,7 +394,7 @@ static int send_io_resp_imm(struct rtrs_srv_con *con, struct rtrs_srv_op *id,* From time to time we have to post signalled sends, * or send queue will fill up and only QP reset can help. */ - flags = (atomic_inc_return (&con->c.wr_cnt) % srv->queue_depth) ? + flags = (atomic_inc_return (&con->c.wr_cnt) % s->signal_interval) ?0 :IB_SEND_SIGNALED ; imm =rtrs_to_io_rsp_imm (id->msg_id, errno, need_inval); imm_wr.wr.next =NULL ;
@@ -1268,8 +1266,9 @@ static void rtrs_srv_rdma_done(struct ib_cq *cq, struct ib_wc *wc)case IB_WC_SEND :/* * post_send() RDMA write completions of IO reqs (read/write) +* and hb. */ -atomic_add (srv->queue_depth, &con->sq_wr_avail); +atomic_add (s->signal_interval, &con->sq_wr_avail);if (unlikely (!list_empty_careful (&con->rsp_wr_wait_list)))rtrs_rdma_process_wr_wait_list (con);
@@ -1659,6 +1658,8 @@ static int create_con(struct rtrs_srv_sess *sess, max_send_wr =min_t (int, wr_limit,SERVICE_CON_QUEUE_DEPTH *2 +2 ); max_recv_wr = max_send_wr; + s->signal_interval =min_not_zero (srv->queue_depth, + (size_t )SERVICE_CON_QUEUE_DEPTH ); }else {/* when always_invlaidate enalbed, we need linv+rinv+mr+imm */ if (always_invalidate)
drivers/infiniband/ulp/rtrs/rtrs.c +6 -1
@@ -187,10 +187,15 @@ int rtrs_post_rdma_write_imm_empty(struct rtrs_con *con, struct ib_cqe *cqe, struct ib_send_wr *head) {struct ib_rdma_wr wr; +struct rtrs_sess *sess = con->sess; +enum ib_send_flags sflags; + + sflags = (atomic_inc_return (&con->wr_cnt) % sess->signal_interval) ? +0 :IB_SEND_SIGNALED ; wr = (struct ib_rdma_wr ) { .wr.wr_cqe = cqe, - .wr.send_flags = flags, + .wr.send_flags = sflags, .wr.opcode =IB_WR_RDMA_WRITE_WITH_IMM , .wr.ex.imm_data =cpu_to_be32 (imm_data), };
Signature · 1 line
2.25.1
Jack Wang
maintainerPATCH v2 4/6 RDMA/rtrs: Make rtrs_post_rdma_write_imm_empty static
In reply to Jack WangIt's only used in rtrs.c, so no need to export.
--- drivers/infiniband/ulp/rtrs/rtrs-pri.h | 3 --- drivers/infiniband/ulp/rtrs/rtrs.c | 9 +++++---- 2 files changed, 5 insertions(+), 7 deletions(-)
drivers/infiniband/ulp/rtrs/rtrs-pri.h +0 -3
@@ -311,9 +311,6 @@ int rtrs_iu_post_rdma_write_imm(struct rtrs_con *con, struct rtrs_iu *iu, struct ib_send_wr *tail);int rtrs_post_recv_empty (struct rtrs_con *con,struct ib_cqe *cqe); -int rtrs_post_rdma_write_imm_empty (struct rtrs_con *con,struct ib_cqe *cqe, -u32 imm_data,enum ib_send_flags flags, -struct ib_send_wr *head); int rtrs_cq_qp_create(struct rtrs_sess *sess, struct rtrs_con *con, u32 max_send_sge, int cq_vector, int nr_cqe,
drivers/infiniband/ulp/rtrs/rtrs.c +5 -4
@@ -182,9 +182,11 @@ int rtrs_iu_post_rdma_write_imm(struct rtrs_con *con, struct rtrs_iu *iu, }EXPORT_SYMBOL_GPL (rtrs_iu_post_rdma_write_imm); -int rtrs_post_rdma_write_imm_empty (struct rtrs_con *con,struct ib_cqe *cqe, -u32 imm_data,enum ib_send_flags flags, -struct ib_send_wr *head) +static int rtrs_post_rdma_write_imm_empty (struct rtrs_con *con, +struct ib_cqe *cqe, +u32 imm_data, +enum ib_send_flags flags, +struct ib_send_wr *head) {struct ib_rdma_wr wr;struct rtrs_sess *sess = con->sess;
@@ -202,7 +204,6 @@ int rtrs_post_rdma_write_imm_empty(struct rtrs_con *con, struct ib_cqe *cqe,return rtrs_post_send (con->qp, head, &wr.wr,NULL ); } -EXPORT_SYMBOL_GPL (rtrs_post_rdma_write_imm_empty);static void qp_event_handler (struct ib_event *ev,void *ctx) {
Signature · 1 line
2.25.1
Jack Wang
maintainerPATCH v2 5/6 RDMA/rtrs: Remove unused flags parameter
In reply to Jack Wangflags is not used, so remove it from rtrs_post_rdma_write_imm_empty.
--- drivers/infiniband/ulp/rtrs/rtrs.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
drivers/infiniband/ulp/rtrs/rtrs.c +2 -3
@@ -185,7 +185,6 @@ EXPORT_SYMBOL_GPL(rtrs_iu_post_rdma_write_imm);static int rtrs_post_rdma_write_imm_empty (struct rtrs_con *con,struct ib_cqe *cqe,u32 imm_data, -enum ib_send_flags flags,struct ib_send_wr *head) {struct ib_rdma_wr wr;
@@ -320,7 +319,7 @@ void rtrs_send_hb_ack(struct rtrs_sess *sess) imm =rtrs_to_imm (RTRS_HB_ACK_IMM ,0 ); err =rtrs_post_rdma_write_imm_empty (usr_con, sess->hb_cqe, imm, -0 ,NULL ); +NULL );if (err) {rtrs_err (sess,"send HB ACK failed, errno: %d\n" , err); sess->hb_err_handler (usr_con);
@@ -354,7 +353,7 @@ static void hb_work(struct work_struct *work) imm =rtrs_to_imm (RTRS_HB_MSG_IMM ,0 ); err =rtrs_post_rdma_write_imm_empty (usr_con, sess->hb_cqe, imm, -0 ,NULL ); +NULL );if (err) {rtrs_err (sess,"HB send failed, errno: %d\n" , err); sess->hb_err_handler (usr_con);
Signature · 1 line
2.25.1
Jack Wang
maintainerPATCH v2 6/6 RDMA/rtrs: Move sq_wr_avail to rtrs_con
In reply to Jack WangIn order to account HB for sq_wr_avail properly,
move sq_wr_avail from rtrs_srv_con to rtrs_con.
Although rtrs-clt do not care sq_wr_avail, but still init it to max_send_wr.
--- drivers/infiniband/ulp/rtrs/rtrs-clt.c | 1 + drivers/infiniband/ulp/rtrs/rtrs-pri.h | 1 + drivers/infiniband/ulp/rtrs/rtrs-srv.c | 8 ++++---- drivers/infiniband/ulp/rtrs/rtrs-srv.h | 1 - drivers/infiniband/ulp/rtrs/rtrs.c | 1 + 5 files changed, 7 insertions(+), 5 deletions(-)
drivers/infiniband/ulp/rtrs/rtrs-clt.c +1 -0
@@ -1680,6 +1680,7 @@ static int create_con_cq_qp(struct rtrs_clt_con *con) sess->queue_depth * 3 + 1); max_send_sge =2 ; } +atomic_set (&con->c.sq_wr_avail, max_send_wr); cq_num = max_send_wr + max_recv_wr;/* alloc iu to recv new rkey reply when server reports flags set */ if (sess->flags &RTRS_MSG_NEW_RKEY_F || con->c.cid ==0 ) {
drivers/infiniband/ulp/rtrs/rtrs-pri.h +1 -0
@@ -97,6 +97,7 @@ struct rtrs_con {unsigned int cid;int nr_cqe;atomic_t wr_cnt; +atomic_t sq_wr_avail; };struct rtrs_sess {
drivers/infiniband/ulp/rtrs/rtrs-srv.c +4 -4
@@ -507,11 +507,11 @@ bool rtrs_srv_resp_rdma(struct rtrs_srv_op *id, int status)ib_update_fast_reg_key (mr->mr,ib_inc_rkey (mr->mr->rkey)); }if (unlikely (atomic_sub_return (1 , - &con->sq_wr_avail) <0 )) { + &con->c.sq_wr_avail) <0 )) {rtrs_err (s,"IB send queue full: sess=%s cid=%d\n" ,kobject_name (&sess->kobj), con->c.cid); -atomic_add (1 , &con->sq_wr_avail); +atomic_add (1 , &con->c.sq_wr_avail);spin_lock (&con->rsp_wr_wait_lock);list_add_tail (&id->wait_list, &con->rsp_wr_wait_list);spin_unlock (&con->rsp_wr_wait_lock);
@@ -1268,7 +1268,7 @@ static void rtrs_srv_rdma_done(struct ib_cq *cq, struct ib_wc *wc)* post_send() RDMA write completions of IO reqs (read/write) * and hb. */ -atomic_add (s->signal_interval, &con->sq_wr_avail); +atomic_add (s->signal_interval, &con->c.sq_wr_avail);if (unlikely (!list_empty_careful (&con->rsp_wr_wait_list)))rtrs_rdma_process_wr_wait_list (con);
@@ -1680,7 +1680,7 @@ static int create_con(struct rtrs_srv_sess *sess,*/ } cq_num = max_send_wr + max_recv_wr; -atomic_set (&con->sq_wr_avail, max_send_wr); +atomic_set (&con->c.sq_wr_avail, max_send_wr); cq_vector =rtrs_srv_get_next_cq_vector (sess);/* TODO: SOFTIRQ can be faster, but be careful with softirq context */
drivers/infiniband/ulp/rtrs/rtrs-srv.h +0 -1
@@ -42,7 +42,6 @@ struct rtrs_srv_stats {struct rtrs_srv_con {struct rtrs_con c; -atomic_t sq_wr_avail;struct list_head rsp_wr_wait_list;spinlock_t rsp_wr_wait_lock; };
drivers/infiniband/ulp/rtrs/rtrs.c +1 -0
@@ -191,6 +191,7 @@ static int rtrs_post_rdma_write_imm_empty(struct rtrs_con *con,struct rtrs_sess *sess = con->sess;enum ib_send_flags sflags; +atomic_dec_if_positive (&con->sq_wr_avail); sflags = (atomic_inc_return (&con->wr_cnt) % sess->signal_interval) ?0 :IB_SEND_SIGNALED ;
Signature · 1 line
2.25.1
Jason Gunthorpe
maintainerQuoted 25 lines · Jack Wang
Hi Jason, hi Doug,
Please consider to include following changes to upstream.
This patchset fix a regression since
b38041d50add("RDMA/rtrs: Do not signal for heatbeat").In commit
b38041d50add, the signal flag is droped to fix the send queue full logic, but introduced a worse bug the send queue overflow on both clt and srv by heartbeat, sorry.The patchset is orgnized as:
- patch1 debug patch.
- patch2 preparation.
- patch3 signal both IO and heartbeat.
- patch4 cleanup.
- patch5 cleanup
- patch6 movesq_wr_availto account send queue full correctly.The patches are created base v5.14-rc1.
Since v1:
* rebased to latest v5.14-rc1, target rc instread of for-next.v1: https://lore.kernel.org/linux-rdma/[email protected]/T/#t
Quoted 7 lines
Jack Wang (6):
RDMA/rtrs: Add error messages for failed operations. RDMA/rtrs: movewr_cntfromrtrs_srv_contortrs_conRDMA/rtrs: Enable the same selective signal for heartbeat and IO RDMA/rtrs: Makertrs_post_rdma_write_imm_emptystatic RDMA/rtrs: Remove unused flags parameter
RDMA/rtrs: Movesq_wr_availtortrs_con
This is not really structured well enough for a -rc patch. There should be no unncessary changes and each patch should try to be self contained. Avoid "cleanup". Carefully describe what user visible defect each patch is fixing.
If you really want it to be in -rc then it needs reorganizing, otherwise I can put it in -next
Jason
Jinpu Wang
maintainerQuoted 40 lines · Jason Gunthorpe
Quoted 25 lines · Jack Wang
Hi Jason, hi Doug,
Please consider to include following changes to upstream.
This patchset fix a regression since
b38041d50add("RDMA/rtrs: Do not signal for heatbeat").In commit
b38041d50add, the signal flag is droped to fix the send queue full logic, but introduced a worse bug the send queue overflow on both clt and srv by heartbeat, sorry.The patchset is orgnized as:
- patch1 debug patch.
- patch2 preparation.
- patch3 signal both IO and heartbeat.
- patch4 cleanup.
- patch5 cleanup
- patch6 movesq_wr_availto account send queue full correctly.The patches are created base v5.14-rc1.
Since v1:
* rebased to latest v5.14-rc1, target rc instread of for-next.v1: https://lore.kernel.org/linux-rdma/[email protected]/T/#t
Quoted 7 lines
Jack Wang (6):
RDMA/rtrs: Add error messages for failed operations. RDMA/rtrs: movewr_cntfromrtrs_srv_contortrs_conRDMA/rtrs: Enable the same selective signal for heartbeat and IO RDMA/rtrs: Makertrs_post_rdma_write_imm_emptystatic RDMA/rtrs: Remove unused flags parameter
RDMA/rtrs: Movesq_wr_availtortrs_conThis is not really structured well enough for a -rc patch. There should be no unncessary changes and each patch should try to be self contained. Avoid "cleanup". Carefully describe what user visible defect each patch is fixing.
Quoted 4 lines
If you really want it to be in -rc then it needs reorganizing, otherwise I can put it in -next
Jason
Hi Jason,
Thanks for your suggestion, I think it would be ok to put them in for-next.
Regards!
Quoted 25 lines · Jack Wang
Hi Jason, hi Doug,
Please consider to include following changes to upstream.
This patchset fix a regression since
b38041d50add("RDMA/rtrs: Do not signal for heatbeat").In commit
b38041d50add, the signal flag is droped to fix the send queue full logic, but introduced a worse bug the send queue overflow on both clt and srv by heartbeat, sorry.The patchset is orgnized as:
- patch1 debug patch.
- patch2 preparation.
- patch3 signal both IO and heartbeat.
- patch4 cleanup.
- patch5 cleanup
- patch6 movesq_wr_availto account send queue full correctly.The patches are created base v5.14-rc1.
Since v1:
* rebased to latest v5.14-rc1, target rc instread of for-next.v1: https://lore.kernel.org/linux-rdma/[email protected]/T/#t
Quoted 7 lines
Jack Wang (6):
RDMA/rtrs: Add error messages for failed operations. RDMA/rtrs: movewr_cntfromrtrs_srv_contortrs_conRDMA/rtrs: Enable the same selective signal for heartbeat and IO RDMA/rtrs: Makertrs_post_rdma_write_imm_emptystatic RDMA/rtrs: Remove unused flags parameter
RDMA/rtrs: Movesq_wr_availtortrs_con
Applied to for-next, thanks
Jason