net/smc: fixes 2020-07-16
v2 cover net/smc: fixes 2020-07-16
Please apply the following patch series for smc to netdev's net tree.
The patches address problems caused by late or unexpected link layer control packets, dma sync calls for unmapped memory, freed buffers that are not removed from the buffer list and a possible null pointer access that results in a crash.
v1->v2: in patch 4, improve patch description and correct the comment
for the new mutex
Thanks,
Karsten
Karsten Graul (10):
net/smc: handle unexpected response types for confirm link net/smc: clear link during SMC client link down processing net/smc: fix link lookup for new rdma connections net/smc: protect smc ib device initialization net/smc: drop out-of-flow llc response messages net/smc: move add link processing for new device into llc layer net/smc: fix handling of delete link requests net/smc: do not call dma sync for unmapped memory net/smc: remove freed buffer from list
net/smc: fix restoring of fallback changes
net/smc/af_smc.c | 12 +++-- net/smc/smc_core.c | 105 ++++++++----------------------------- net/smc/smc_core.h | 5 ++ net/smc/smc_ib.c | 16 ++++-- net/smc/smc_ib.h | 1 + net/smc/smc_llc.c | 127 ++++++++++++++++++++++++++++++--------------- net/smc/smc_llc.h | 2 +- 7 files changed, 135 insertions(+), 133 deletions(-)
Signature · 1 line
2.17.1
PATCH v2 1/10 net/smc: handle unexpected response types for confirm link
A delete link could arrive during confirm link processing. Handle this situation directly in smc_llc_srv_conf_link() rather than using the logic in smc_llc_wait() to avoid the unexpected message handling there.
--- net/smc/smc_llc.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
net/smc/smc_llc.c +5 -3
@@ -1051,12 +1051,14 @@ static int smc_llc_srv_conf_link(struct smc_link *link,if (rc)return -ENOLINK ;/* receive CONFIRM LINK response over the RoCE fabric */ - qentry =smc_llc_wait (lgr, link,SMC_LLC_WAIT_FIRST_TIME , -SMC_LLC_CONFIRM_LINK ); -if (!qentry) { + qentry =smc_llc_wait (lgr, link,SMC_LLC_WAIT_FIRST_TIME ,0 ); +if (!qentry || + qentry->msg.raw.hdr.common.type !=SMC_LLC_CONFIRM_LINK ) {/* send DELETE LINK */ smc_llc_send_delete_link (link, link_new->link_id,SMC_LLC_REQ , false,SMC_LLC_DEL_LOST_PATH ); +if (qentry) +smc_llc_flow_qentry_del (&lgr->llc_flow_lcl);return -ENOLINK ; }smc_llc_save_peer_uid (qentry);
Signature · 1 line
2.17.1
PATCH v2 2/10 net/smc: clear link during SMC client link down processing
In reply to Karsten GraulIn a link-down condition we notify the SMC server and expect that the server will finally trigger the link clear processing on the client side. This could fail when anything along this notification path goes wrong. Clear the link as part of SMC client link-down processing to prevent dangling links.
--- net/smc/smc_core.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
net/smc/smc_core.c +3 -1
@@ -1204,10 +1204,12 @@ static void smcr_link_down(struct smc_link *lnk) SMC_LLC_WAIT_TIME);mutex_lock (&lgr->llc_conf_mutex); } -if (!list_empty (&lgr->list)) +if (!list_empty (&lgr->list)) {smc_llc_send_delete_link (to_lnk, del_link_id,SMC_LLC_REQ , true,SMC_LLC_DEL_LOST_PATH ); +smcr_link_clear (lnk, true); + }wake_up (&lgr->llc_flow_waiter);/* wake up next waiter */ } }
Signature · 1 line
2.17.1
PATCH v2 3/10 net/smc: fix link lookup for new rdma connections
In reply to Karsten GraulFor new rdma connections the SMC server assigns the link and sends the link data in the clc accept message. To match the correct link use not only the qp_num but also the gid and the mac of the links. If there are equal qp_nums for different links the wrong link would be chosen.
--- net/smc/af_smc.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
net/smc/af_smc.c +3 -1
@@ -632,7 +632,9 @@ static int smc_connect_rdma(struct smc_sock *smc,for (i =0 ; i <SMC_LINKS_PER_LGR_MAX ; i++) {struct smc_link *l = &smc->conn.lgr->lnk[i]; -if (l->peer_qpn ==ntoh24 (aclc->qpn)) { +if (l->peer_qpn ==ntoh24 (aclc->qpn) && + !memcmp (l->peer_gid, &aclc->lcl.gid,SMC_GID_SIZE ) && + !memcmp (l->peer_mac, &aclc->lcl.mac,sizeof (l->peer_mac))) { link = l;break ; }
Signature · 1 line
2.17.1
PATCH v2 4/10 net/smc: protect smc ib device initialization
In reply to Karsten GraulBefore an smc ib device is used the first time for an smc link it is lazily initialized. When there are 2 active link groups and a new ib device is brought online then it might happen that 2 link creations run in parallel and enter smc_ib_setup_per_ibdev(). Both allocate new send and receive completion queues on the device, but only one set of them keeps assigned and the other leaks.
Fix that by protecting the setup and cleanup code using a mutex.
--- net/smc/smc_ib.c | 16 +++++++++++++--- net/smc/smc_ib.h | 1 + 2 files changed, 14 insertions(+), 3 deletions(-)
net/smc/smc_ib.c +13 -3
@@ -506,6 +506,10 @@ long smc_ib_setup_per_ibdev(struct smc_ib_device *smcibdev)int cqe_size_order, smc_order;long rc; +mutex_lock (&smcibdev->mutex); + rc =0 ; +if (smcibdev->initialized) + goto out;/* the calculated number of cq entries fits to mlx5 cq allocation */ cqe_size_order =cache_line_size () ==128 ?7 :6 ; smc_order =MAX_ORDER - cqe_size_order -1 ;
@@ -517,7 +521,7 @@ long smc_ib_setup_per_ibdev(struct smc_ib_device *smcibdev) rc =PTR_ERR_OR_ZERO (smcibdev->roce_cq_send);if (IS_ERR (smcibdev->roce_cq_send)) { smcibdev->roce_cq_send =NULL ; -return rc; + goto out; } smcibdev->roce_cq_recv = ib_create_cq(smcibdev->ibdev, smc_wr_rx_cq_handler, NULL,
@@ -529,21 +533,26 @@ long smc_ib_setup_per_ibdev(struct smc_ib_device *smcibdev) }smc_wr_add_dev (smcibdev); smcibdev->initialized =1 ; -return rc; + goto out; err:ib_destroy_cq (smcibdev->roce_cq_send); +out: +mutex_unlock (&smcibdev->mutex);return rc; }static void smc_ib_cleanup_per_ibdev (struct smc_ib_device *smcibdev) { +mutex_lock (&smcibdev->mutex);if (!smcibdev->initialized) -return ; + goto out; smcibdev->initialized =0 ;ib_destroy_cq (smcibdev->roce_cq_recv);ib_destroy_cq (smcibdev->roce_cq_send);smc_wr_remove_dev (smcibdev); +out: +mutex_unlock (&smcibdev->mutex); }static struct ib_client smc_ib_client;
@@ -566,6 +575,7 @@ static int smc_ib_add_dev(struct ib_device *ibdev)INIT_WORK (&smcibdev->port_event_work, smc_ib_port_event_work);atomic_set (&smcibdev->lnk_cnt,0 );init_waitqueue_head (&smcibdev->lnks_deleted); +mutex_init (&smcibdev->mutex);mutex_lock (&smc_ib_devices.mutex);list_add_tail (&smcibdev->list, &smc_ib_devices.list);mutex_unlock (&smc_ib_devices.mutex);
net/smc/smc_ib.h +1 -0
@@ -52,6 +52,7 @@ struct smc_ib_device { /* ib-device infos for smc */DECLARE_BITMAP (ports_going_away,SMC_MAX_PORTS );atomic_t lnk_cnt;/* number of links on ibdev */ wait_queue_head_t lnks_deleted;/* wait 4 removal of all links*/ +struct mutex mutex;/* protect dev setup+cleanup */ };struct smc_buf_desc ;
Signature · 1 line
2.17.1
PATCH v2 5/10 net/smc: drop out-of-flow llc response messages
In reply to Karsten GraulTo be save from unexpected or late llc response messages check if the arrived message fits to the current flow type and drop out-of-flow messages. And drop it when there is already a response assigned to the flow.
--- net/smc/smc_llc.c | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-)
net/smc/smc_llc.c +18 -6
@@ -1587,6 +1587,8 @@ static void smc_llc_event_work(struct work_struct *work)static void smc_llc_rx_response (struct smc_link *link,struct smc_llc_qentry *qentry) { +enum smc_llc_flowtype flowtype = link->lgr->llc_flow_lcl.type; +struct smc_llc_flow *flow = &link->lgr->llc_flow_lcl;u8 llc_type = qentry->msg.raw.hdr.common.type;switch (llc_type) {
@@ -1595,15 +1597,20 @@ static void smc_llc_rx_response(struct smc_link *link,complete (&link->llc_testlink_resp);break ;case SMC_LLC_ADD_LINK : -case SMC_LLC_DELETE_LINK : -case SMC_LLC_CONFIRM_LINK :case SMC_LLC_ADD_LINK_CONT : +case SMC_LLC_CONFIRM_LINK : +if (flowtype !=SMC_LLC_FLOW_ADD_LINK || flow->qentry) +break ;/* drop out-of-flow response */ + goto assign; +case SMC_LLC_DELETE_LINK : +if (flowtype !=SMC_LLC_FLOW_DEL_LINK || flow->qentry) +break ;/* drop out-of-flow response */ + goto assign;case SMC_LLC_CONFIRM_RKEY :case SMC_LLC_DELETE_RKEY : -/* assign responses to the local flow, we requested them */ -smc_llc_flow_qentry_set (&link->lgr->llc_flow_lcl, qentry); -wake_up (&link->lgr->llc_msg_waiter); -return ; +if (flowtype !=SMC_LLC_FLOW_RKEY || flow->qentry) +break ;/* drop out-of-flow response */ + goto assign;case SMC_LLC_CONFIRM_RKEY_CONT :/* not used because max links is 3 */ break ;
@@ -1612,6 +1619,11 @@ static void smc_llc_rx_response(struct smc_link *link,break ; }kfree (qentry); +return ; +assign: +/* assign responses to the local flow, we requested them */ +smc_llc_flow_qentry_set (&link->lgr->llc_flow_lcl, qentry); +wake_up (&link->lgr->llc_msg_waiter); }static void smc_llc_enqueue (struct smc_link *link,union smc_llc_msg *llc)
Signature · 1 line
2.17.1
PATCH v2 6/10 net/smc: move add link processing for new device into llc layer
In reply to Karsten GraulWhen a new ib device is up smc will send an add link invitation to the peer if needed. This is currently done with rudimentary flow control. Under high workload these add link invitations can disturb other llc flows because they arrive unexpected. Fix this by integrating the invitations into the normal llc event flow and handle them as a flow. While at it, check for already assigned requests in the flow before the new add link request is assigned.
--- net/smc/smc_core.c | 80 ++++------------------------------------------ net/smc/smc_llc.c | 56 ++++++++++++++++++++++++++++---- net/smc/smc_llc.h | 2 +- 3 files changed, 58 insertions(+), 80 deletions(-)
net/smc/smc_core.c +7 -73
@@ -45,18 +45,10 @@ static struct smc_lgr_list smc_lgr_list = { /* established link groups */static atomic_t lgr_cnt =ATOMIC_INIT (0 );/* number of existing link groups */ static DECLARE_WAIT_QUEUE_HEAD (lgrs_deleted); -struct smc_ib_up_work { -struct work_struct work; -struct smc_link_group *lgr; -struct smc_ib_device *smcibdev; -u8 ibport; -}; -static void smc_buf_free (struct smc_link_group *lgr,bool is_rmb,struct smc_buf_desc *buf_desc);static void __smc_lgr_terminate (struct smc_link_group *lgr,bool soft); -static void smc_link_up_work (struct work_struct *work);static void smc_link_down_work (struct work_struct *work);/* return head of link group list and its lock for a given link group */
@@ -1106,67 +1098,23 @@ static void smc_conn_abort_work(struct work_struct *work)sock_put (&smc->sk);/* sock_hold done by schedulers of abort_work */ } -/* link is up - establish alternate link if applicable */ -static void smcr_link_up (struct smc_link_group *lgr, -struct smc_ib_device *smcibdev,u8 ibport) -{ -struct smc_link *link =NULL ; - -if (list_empty (&lgr->list) || - lgr->type ==SMC_LGR_SYMMETRIC || - lgr->type ==SMC_LGR_ASYMMETRIC_PEER ) -return ; - -if (lgr->role ==SMC_SERV ) { -/* trigger local add link processing */ - link =smc_llc_usable_link (lgr); -if (!link) -return ; -smc_llc_srv_add_link_local (link); - }else { -/* invite server to start add link processing */ -u8 gid[SMC_GID_SIZE ]; - -if (smc_ib_determine_gid (smcibdev, ibport, lgr->vlan_id, gid, -NULL )) -return ; -if (lgr->llc_flow_lcl.type !=SMC_LLC_FLOW_NONE ) { -/* some other llc task is ongoing */ -wait_event_timeout (lgr->llc_flow_waiter, - (list_empty (&lgr->list) || - lgr->llc_flow_lcl.type ==SMC_LLC_FLOW_NONE ), -SMC_LLC_WAIT_TIME ); - } -/* lgr or device no longer active? */ -if (!list_empty (&lgr->list) && -smc_ib_port_active (smcibdev, ibport)) - link =smc_llc_usable_link (lgr); -if (link) -smc_llc_send_add_link (link, smcibdev->mac[ibport -1 ], - gid,NULL ,SMC_LLC_REQ ); -wake_up (&lgr->llc_flow_waiter);/* wake up next waiter */ - } -} -void smcr_port_add (struct smc_ib_device *smcibdev,u8 ibport) { -struct smc_ib_up_work *ib_work;struct smc_link_group *lgr, *n;list_for_each_entry_safe (lgr, n, &smc_lgr_list.list, list) { +struct smc_link *link; +if (strncmp (smcibdev->pnetid[ibport -1 ], lgr->pnet_id,SMC_MAX_PNETID_LEN ) || lgr->type ==SMC_LGR_SYMMETRIC || lgr->type ==SMC_LGR_ASYMMETRIC_PEER )continue ; - ib_work =kmalloc (sizeof (*ib_work),GFP_KERNEL ); -if (!ib_work) -continue ; -INIT_WORK (&ib_work->work, smc_link_up_work); - ib_work->lgr = lgr; - ib_work->smcibdev = smcibdev; - ib_work->ibport = ibport; -schedule_work (&ib_work->work); + +/* trigger local add link processing */ + link =smc_llc_usable_link (lgr); +if (link) +smc_llc_add_link_local (link); } }
@@ -1249,20 +1197,6 @@ void smcr_port_err(struct smc_ib_device *smcibdev, u8 ibport) } } -static void smc_link_up_work (struct work_struct *work) -{ -struct smc_ib_up_work *ib_work =container_of (work, - struct smc_ib_up_work, - work); -struct smc_link_group *lgr = ib_work->lgr; - -if (list_empty (&lgr->list)) - goto out; -smcr_link_up (lgr, ib_work->smcibdev, ib_work->ibport); -out: -kfree (ib_work); -} -static void smc_link_down_work (struct work_struct *work) { struct smc_link *link = container_of(work, struct smc_link,
net/smc/smc_llc.c +50 -6
91 lines not on this page · raw message
net/smc/smc_llc.h +1 -1
@@ -103,7 +103,7 @@ void smc_llc_send_link_delete_all(struct smc_link_group *lgr, bool ord, u32 rsn);int smc_llc_cli_add_link (struct smc_link *link,struct smc_llc_qentry *qentry);int smc_llc_srv_add_link (struct smc_link *link); -void smc_llc_srv_add_link_local (struct smc_link *link); +void smc_llc_add_link_local (struct smc_link *link);int smc_llc_init (void )__init ;#endif /* SMC_LLC_H */
Signature · 1 line
2.17.1
PATCH v2 7/10 net/smc: fix handling of delete link requests
In reply to Karsten GraulAs smc client the delete link requests are assigned to the flow when _any_ flow is active. This may break other flows that do not expect delete link requests during their handling. Fix that by assigning the request only when an add link flow is active. With that fix the code for smc client and smc server is the same, so remove the separate handling.
--- net/smc/smc_llc.c | 29 +++++++---------------------- 1 file changed, 7 insertions(+), 22 deletions(-)
net/smc/smc_llc.c +7 -22
@@ -1544,28 +1544,13 @@ static void smc_llc_event_handler(struct smc_llc_qentry *qentry) }break ;case SMC_LLC_DELETE_LINK : -if (lgr->role ==SMC_CLNT ) { -/* server requests to delete this link, send response */ -if (lgr->llc_flow_lcl.type !=SMC_LLC_FLOW_NONE ) { -/* DEL LINK REQ during ADD LINK SEQ */ -smc_llc_flow_qentry_set (&lgr->llc_flow_lcl, - qentry); -wake_up (&lgr->llc_msg_waiter); - }else if (smc_llc_flow_start (&lgr->llc_flow_lcl, - qentry)) { -schedule_work (&lgr->llc_del_link_work); - } - }else { -if (lgr->llc_flow_lcl.type ==SMC_LLC_FLOW_ADD_LINK && - !lgr->llc_flow_lcl.qentry) { -/* DEL LINK REQ during ADD LINK SEQ */ -smc_llc_flow_qentry_set (&lgr->llc_flow_lcl, - qentry); -wake_up (&lgr->llc_msg_waiter); - }else if (smc_llc_flow_start (&lgr->llc_flow_lcl, - qentry)) { -schedule_work (&lgr->llc_del_link_work); - } +if (lgr->llc_flow_lcl.type ==SMC_LLC_FLOW_ADD_LINK && + !lgr->llc_flow_lcl.qentry) { +/* DEL LINK REQ during ADD LINK SEQ */ +smc_llc_flow_qentry_set (&lgr->llc_flow_lcl, qentry); +wake_up (&lgr->llc_msg_waiter); + }else if (smc_llc_flow_start (&lgr->llc_flow_lcl, qentry)) { +schedule_work (&lgr->llc_del_link_work); }return ;case SMC_LLC_CONFIRM_RKEY :
Signature · 1 line
2.17.1
PATCH v2 8/10 net/smc: do not call dma sync for unmapped memory
In reply to Karsten GraulThe dma related ...sync_sg... functions check the link state before the dma function is actually called. But the check in smc_link_usable() allows links in ACTIVATING state which are not yet mapped to dma memory. Under high load it may happen that the sync_sg functions are called for such a link which results in an debug output like
DMA-API: mlx5_core 0002:00:00.0: device driver tries to sync DMA memory it has not allocated [device address=0x0000000103370000] [size=65536 bytes]
To fix that introduce a helper to check for the link state ACTIVE and use it where appropriate. And move the link state update to ACTIVATING to the end of smcr_link_init() when most initial setup is done.
--- net/smc/af_smc.c | 2 +- net/smc/smc_core.c | 15 +++++++-------- net/smc/smc_core.h | 5 +++++ net/smc/smc_llc.c | 10 +++++----- 4 files changed, 18 insertions(+), 14 deletions(-)
net/smc/af_smc.c +1 -1
@@ -352,7 +352,7 @@ static int smcr_lgr_reg_rmbs(struct smc_link *link,*/ mutex_lock (&lgr->llc_conf_mutex);for (i =0 ; i <SMC_LINKS_PER_LGR_MAX ; i++) { -if (lgr->lnk[i].state !=SMC_LNK_ACTIVE ) +if (!smc_link_active (&lgr->lnk[i]))continue ; rc =smcr_link_reg_rmb (&lgr->lnk[i], rmb_desc);if (rc)
net/smc/smc_core.c +7 -8
@@ -318,7 +318,6 @@ int smcr_link_init(struct smc_link_group *lgr, struct smc_link *lnk,get_device (&ini->ib_dev->ibdev->dev);atomic_inc (&ini->ib_dev->lnk_cnt); - lnk->state =SMC_LNK_ACTIVATING ; lnk->link_id =smcr_next_link_id (lgr); lnk->lgr = lgr; lnk->link_idx = link_idx;
@@ -354,6 +353,7 @@ int smcr_link_init(struct smc_link_group *lgr, struct smc_link *lnk, rc =smc_wr_create_link (lnk);if (rc) goto destroy_qp; + lnk->state =SMC_LNK_ACTIVATING ;return 0 ; destroy_qp:
@@ -542,8 +542,7 @@ struct smc_link *smc_switch_conns(struct smc_link_group *lgr,smc_wr_wakeup_tx_wait (from_lnk);for (i =0 ; i <SMC_LINKS_PER_LGR_MAX ; i++) { -if (lgr->lnk[i].state !=SMC_LNK_ACTIVE || - i == from_lnk->link_idx) +if (!smc_link_active (&lgr->lnk[i]) || i == from_lnk->link_idx)continue ;if (is_dev_err && from_lnk->smcibdev == lgr->lnk[i].smcibdev && from_lnk->ibport == lgr->lnk[i].ibport) {
@@ -1269,7 +1268,7 @@ static bool smcr_lgr_match(struct smc_link_group *lgr,return false;for (i =0 ; i <SMC_LINKS_PER_LGR_MAX ; i++) { -if (lgr->lnk[i].state !=SMC_LNK_ACTIVE ) +if (!smc_link_active (&lgr->lnk[i]))continue ; if ((lgr->role == SMC_SERV || lgr->lnk[i].peer_qpn == clcqpn) && !memcmp(lgr->lnk[i].peer_gid, &lcl->gid, SMC_GID_SIZE) &&
@@ -1717,14 +1716,14 @@ static int __smc_buf_create(struct smc_sock *smc, bool is_smcd, bool is_rmb)void smc_sndbuf_sync_sg_for_cpu (struct smc_connection *conn) { -if (!conn->lgr || conn->lgr->is_smcd || !smc_link_usable (conn->lnk)) +if (!conn->lgr || conn->lgr->is_smcd || !smc_link_active (conn->lnk))return ;smc_ib_sync_sg_for_cpu (conn->lnk, conn->sndbuf_desc,DMA_TO_DEVICE ); }void smc_sndbuf_sync_sg_for_device (struct smc_connection *conn) { -if (!conn->lgr || conn->lgr->is_smcd || !smc_link_usable (conn->lnk)) +if (!conn->lgr || conn->lgr->is_smcd || !smc_link_active (conn->lnk))return ;smc_ib_sync_sg_for_device (conn->lnk, conn->sndbuf_desc,DMA_TO_DEVICE ); }
@@ -1736,7 +1735,7 @@ void smc_rmb_sync_sg_for_cpu(struct smc_connection *conn)if (!conn->lgr || conn->lgr->is_smcd)return ;for (i =0 ; i <SMC_LINKS_PER_LGR_MAX ; i++) { -if (!smc_link_usable (&conn->lgr->lnk[i])) +if (!smc_link_active (&conn->lgr->lnk[i]))continue ;smc_ib_sync_sg_for_cpu (&conn->lgr->lnk[i], conn->rmb_desc,DMA_FROM_DEVICE );
@@ -1750,7 +1749,7 @@ void smc_rmb_sync_sg_for_device(struct smc_connection *conn)if (!conn->lgr || conn->lgr->is_smcd)return ;for (i =0 ; i <SMC_LINKS_PER_LGR_MAX ; i++) { -if (!smc_link_usable (&conn->lgr->lnk[i])) +if (!smc_link_active (&conn->lgr->lnk[i]))continue ;smc_ib_sync_sg_for_device (&conn->lgr->lnk[i], conn->rmb_desc,DMA_FROM_DEVICE );
net/smc/smc_core.h +5 -0
@@ -349,6 +349,11 @@ static inline bool smc_link_usable(struct smc_link *lnk)return true; } +static inline bool smc_link_active (struct smc_link *lnk) +{ +return lnk->state ==SMC_LNK_ACTIVE ; +} +struct smc_sock ;struct smc_clc_msg_accept_confirm ;struct smc_clc_msg_local ;
net/smc/smc_llc.c +5 -5
@@ -428,7 +428,7 @@ static int smc_llc_send_confirm_rkey(struct smc_link *send_link, rtok_ix =1 ;for (i =0 ; i <SMC_LINKS_PER_LGR_MAX ; i++) { link = &send_link->lgr->lnk[i]; -if (link->state ==SMC_LNK_ACTIVE && link != send_link) { +if (smc_link_active (link) && link != send_link) { rkeyllc->rtoken[rtok_ix].link_id = link->link_id; rkeyllc->rtoken[rtok_ix].rmb_key =htonl (rmb_desc->mr_rx[link->link_idx]->rkey);
@@ -944,7 +944,7 @@ static int smc_llc_active_link_count(struct smc_link_group *lgr)int i, link_count =0 ;for (i =0 ; i <SMC_LINKS_PER_LGR_MAX ; i++) { -if (!smc_link_usable (&lgr->lnk[i])) +if (!smc_link_active (&lgr->lnk[i]))continue ; link_count++; }
@@ -1622,7 +1622,7 @@ static void smc_llc_rx_response(struct smc_link *link,switch (llc_type) {case SMC_LLC_TEST_LINK : -if (link->state ==SMC_LNK_ACTIVE ) +if (smc_link_active (link))complete (&link->llc_testlink_resp);break ;case SMC_LLC_ADD_LINK :
@@ -1706,7 +1706,7 @@ static void smc_llc_testlink_work(struct work_struct *work)u8 user_data[16 ] = {0 };int rc; -if (link->state !=SMC_LNK_ACTIVE ) +if (!smc_link_active (link))return ;/* don't reschedule worker */ expire_time = link->wr_rx_tstamp + link->llc_testlink_time;if (time_is_after_jiffies (expire_time)) {
@@ -1718,7 +1718,7 @@ static void smc_llc_testlink_work(struct work_struct *work)/* receive TEST LINK response over RoCE fabric */ rc =wait_for_completion_interruptible_timeout (&link->llc_testlink_resp,SMC_LLC_WAIT_TIME ); -if (link->state !=SMC_LNK_ACTIVE ) +if (!smc_link_active (link))return ;/* link state changed */ if (rc <=0 ) {smcr_link_down_cond_sched (link);
Signature · 1 line
2.17.1
PATCH v2 9/10 net/smc: remove freed buffer from list
In reply to Karsten GraulTwo buffers are allocated for each SMC connection. Each buffer is added to a buffer list after creation. When the second buffer allocation fails, the first buffer is freed but not deleted from the list. This might result in crashes when another connection picks up the freed buffer later and starts to work with it.
--- net/smc/smc_core.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
net/smc/smc_core.c +5 -1
@@ -1772,8 +1772,12 @@ int smc_buf_create(struct smc_sock *smc, bool is_smcd)return rc;/* create rmb */ rc =__smc_buf_create (smc, is_smcd, true); -if (rc) +if (rc) { +mutex_lock (&smc->conn.lgr->sndbufs_lock); +list_del (&smc->conn.sndbuf_desc->list); +mutex_unlock (&smc->conn.lgr->sndbufs_lock);smc_buf_free (smc->conn.lgr, false, smc->conn.sndbuf_desc); + }return rc; }
Signature · 1 line
2.17.1
PATCH v2 10/10 net/smc: fix restoring of fallback changes
In reply to Karsten GraulWhen a listen socket is closed then all non-accepted sockets in its accept queue are to be released. Inside __smc_release() the helper smc_restore_fallback_changes() restores the changes done to the socket without to check if the clcsocket has a file set. This can result in a crash. Fix this by checking the file pointer first.
--- net/smc/af_smc.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
net/smc/af_smc.c +4 -2
@@ -126,8 +126,10 @@ EXPORT_SYMBOL_GPL(smc_proto6);static void smc_restore_fallback_changes (struct smc_sock *smc) { - smc->clcsock->file->private_data = smc->sk.sk_socket; - smc->clcsock->file =NULL ; +if (smc->clcsock->file) {/* non-accepted sockets have no file yet */ + smc->clcsock->file->private_data = smc->sk.sk_socket; + smc->clcsock->file =NULL ; + } }static int __smc_release (struct smc_sock *smc)
Signature · 1 line
2.17.1
David Miller
maintainerFrom: Karsten Graul <[email protected]>
Date: Sat, 18 Jul 2020 15:06:08 +0200
Quoted 9 lines
Please apply the following patch series for smc to netdev's net tree.
The patches address problems caused by late or unexpected link layer control packets, dma sync calls for unmapped memory, freed buffers that are not removed from the buffer list and a possible null pointer access that results in a crash.
v1->v2: in patch 4, improve patch description and correct the comment
for the new mutex
Series applied, thanks.