Skip to contentSkip to conversation

net/smc: Stop the CLC flow if no link to map buffers on

PATCH Applied Wen Gu 3 participants 4 messages Sep 20, 2022 – Sep 22, 2022

Subsystems (4)
lore
Share & tools
lore

Wen Gu

author reviewer
#0 lore raw

PATCH 1/1 net/smc: Stop the CLC flow if no link to map buffers on

There might be a potential race between SMC-R buffer map and link group termination.

smc_smcr_terminate_all() | smc_connect_rdma()
--------------------------------------------------------------

                             | smc_conn_create()
for links in smcibdev        |
        schedule links down  |
                             | smc_buf_create()
                             |  \- smcr_buf_map_usable_links()
                             |      \- no usable links found,
                             |         (rmb->mr = NULL)
                             |
                             | smc_clc_send_confirm()
                             |  \- access conn->rmb_desc->mr[]->rkey
                             |     (panic)

During reboot and IB device module remove, all links will be set down and no usable links remain in link groups. In such situation smcr_buf_map_usable_links() should return an error and stop the CLC flow accessing to uninitialized mr.

Fixes b9247544c1bc ("net/smc: convert static link ID instances to support multiple links")
---
 net/smc/smc_core.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
net/smc/smc_core.c +4 -1
index ebf56cd..df89c2e 100644
@@ -2239,7 +2239,7 @@ static struct smc_buf_desc *smcr_new_buf_create(struct smc_link_group *lgr,
 static int smcr_buf_map_usable_links(struct smc_link_group *lgr,
 				     struct smc_buf_desc *buf_desc, bool is_rmb)
 {
-	int i, rc = 0;
+	int i, rc = 0, cnt = 0;
 
 	/* protect against parallel link reconfiguration */
 	mutex_lock(&lgr->llc_conf_mutex);
@@ -2252,9 +2252,12 @@ static int smcr_buf_map_usable_links(struct smc_link_group *lgr,
 			rc = -ENOMEM;
 			goto out;
 		}
+		cnt++;
 	}
 out:
 	mutex_unlock(&lgr->llc_conf_mutex);
+	if (!rc && !cnt)
+		rc = -EINVAL;
 	return rc;
 }
 
Signature · 1 line
1.8.3.1
author reviewer
#1 lore raw
Quoted 44 lines · Wen Gu

There might be a potential race between SMC-R buffer map and link group termination.

smc_smcr_terminate_all() | smc_connect_rdma()
--------------------------------------------------------------

                              | smc_conn_create()
for links in smcibdev        |
         schedule links down  |
                              | smc_buf_create()
                              |  \- smcr_buf_map_usable_links()
                              |      \- no usable links found,
                              |         (rmb->mr = NULL)
                              |
                              | smc_clc_send_confirm()
                              |  \- access conn->rmb_desc->mr[]->rkey
                              |     (panic)

During reboot and IB device module remove, all links will be set down and no usable links remain in link groups. In such situation smcr_buf_map_usable_links() should return an error and stop the CLC flow accessing to uninitialized mr.

Fixes b9247544c1bc ("net/smc: convert static link ID instances to support multiple links")
---
  net/smc/smc_core.c | 5 ++++-
  1 file changed, 4 insertions(+), 1 deletion(-)
net/smc/smc_core.c +1 -1
index ebf56cd..df89c2e 100644
@@ -2239,7 +2239,7 @@ static struct smc_buf_desc *smcr_new_buf_create(struct smc_link_group *lgr,
  static int smcr_buf_map_usable_links(struct smc_link_group *lgr,
  				     struct smc_buf_desc *buf_desc, bool is_rmb)
  {
-	int i, rc = 0;
+	int i, rc = 0, cnt = 0;
 
  	/* protect against parallel link reconfiguration */
  	mutex_lock(&lgr->llc_conf_mutex);
@@ -2252,9 +2252,12 @@ static int smcr_buf_map_usable_links(struct smc_link_group *lgr,
  			rc = -ENOMEM;
  			goto out;

net/smc/smc_core.c static int smcr_buf_map_usable_links(struct smc_link_group *lgr, · line 2263

Quoted 10 lines
net/smc/smc_core.c +3 -0
  		}
+		cnt++;
  	}
  out:
  	mutex_unlock(&lgr->llc_conf_mutex);
+	if (!rc && !cnt)
+		rc = -EINVAL;
  	return rc;
  }
 

Any comments or reviews are welcome and appreciated.

Thanks,
Wen Gu

Acked-by
#2 lore raw
Quoted 61 lines · Wen Gu
Quoted 48 lines · Wen Gu

There might be a potential race between SMC-R buffer map and link group termination.

smc_smcr_terminate_all()     | smc_connect_rdma()
--------------------------------------------------------------
                              | smc_conn_create()
for links in smcibdev        |
         schedule links down  |

                              | smc_buf_create()
                              |  \- smcr_buf_map_usable_links()
                              |      \- no usable links found,
                              |         (rmb->mr = NULL)
                              |
                              | smc_clc_send_confirm()
                              |  \- access conn->rmb_desc->mr[]->rkey
                              |     (panic)

During reboot and IB device module remove, all links will be set down and no usable links remain in link groups. In such situation smcr_buf_map_usable_links() should return an error and stop the CLC flow accessing to uninitialized mr.

Fixes b9247544c1bc ("net/smc: convert static link ID instances to support multiple links")

---
  net/smc/smc_core.c | 5 ++++-

  1 file changed, 4 insertions(+), 1 deletion(-)
net/smc/smc_core.c +2 -1
index ebf56cd..df89c2e 100644
@@ -2239,7 +2239,7 @@ static struct smc_buf_desc 
 *smcr_new_buf_create(struct smc_link_group *lgr,
   static int smcr_buf_map_usable_links(struct smc_link_group *lgr,
                        struct smc_buf_desc *buf_desc, bool is_rmb)
   {
-    int i, rc = 0;
+    int i, rc = 0, cnt = 0;
       /* protect against parallel link reconfiguration */
       mutex_lock(&lgr->llc_conf_mutex);
@@ -2252,9 +2252,12 @@ static int smcr_buf_map_usable_links(struct 
 smc_link_group *lgr,
               rc = -ENOMEM;
               goto out;
           }
+        cnt++;
       }
Quoted 6 lines

  out:
      mutex_unlock(&lgr->llc_conf_mutex);
+    if (!rc && !cnt)
+        rc = -EINVAL;
      return rc;
  }

Any comments or reviews are welcome and appreciated.

Quoted 2 lines

Thanks,
Wen Gu

Sorry for the late answer!
Good catch! Thank you!

#3 lore raw
In reply to Wen Gu
This patch was applied to netdev/net.git (master) by Paolo Abeni <[email protected]>:

Hello:

This patch was applied to netdev/net.git (master)
by Paolo Abeni <[email protected]>:

Quoted 17 lines · On Tue, 20 Sep 2022 14:43:09 +0800 you wrote:

There might be a potential race between SMC-R buffer map and link group termination.

smc_smcr_terminate_all() | smc_connect_rdma()
--------------------------------------------------------------

                             | smc_conn_create()
for links in smcibdev        |
        schedule links down  |
                             | smc_buf_create()
                             |  \- smcr_buf_map_usable_links()
                             |      \- no usable links found,
                             |         (rmb->mr = NULL)
                             |
                             | smc_clc_send_confirm()
                             |  \- access conn->rmb_desc->mr[]->rkey
                             |     (panic)
Quoted 1 line

[...]

Here is the summary with links:
- [net] net/smc: Stop the CLC flow if no link to map buffers on

    https://git.kernel.org/netdev/net/c/e738455b2c6d

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html