Skip to contentSkip to conversation

net/smc: add sysctl for smc_limit_hs

PATCH In review D. Wythe 5 participants 9 messages Aug 21, 2024 – Sep 6, 2024

Subsystems (5)
lore
Share & tools
lore

D. Wythe

author maintainer
#0 lore raw

PATCH v2 1/1 net/smc: add sysctl for smc_limit_hs

From: "D. Wythe" <[email protected]>

In commit 48b6190a0042 ("net/smc: Limit SMC visits when handshake workqueue congested"), we introduce a mechanism to put constraint on SMC connections visit according to the pressure of SMC handshake process.

At that time, we believed that controlling the feature through netlink was sufficient. However, most people have realized now that netlink is not convenient in container scenarios, and sysctl is a more suitable approach.

In addition, since commit 462791bbfa35 ("net/smc: add sysctl interface for SMC") had introcuded smc_sysctl_net_init(), it is reasonable for us to initialize limit_smc_hs in it instead of initializing it in smc_pnet_net_int().

---
v1 -> v2:

Modified the description in the commit and removed the incorrect spelling.

 net/smc/smc_pnet.c   |  3 ---
 net/smc/smc_sysctl.c | 11 +++++++++++
 2 files changed, 11 insertions(+), 3 deletions(-)
net/smc/smc_pnet.c +0 -3
index 2adb92b..1dd3623 100644
@@ -887,9 +887,6 @@ int smc_pnet_net_init(struct net *net)
 
 	smc_pnet_create_pnetids_list(net);
 
-	/* disable handshake limitation by default */
-	net->smc.limit_smc_hs = 0;
-
 	return 0;
 }
 
net/smc/smc_sysctl.c +11 -0
index 13f2bc0..2fab645 100644
@@ -90,6 +90,15 @@
 		.extra1		= &conns_per_lgr_min,
 		.extra2		= &conns_per_lgr_max,
 	},
+	{
+		.procname	= "limit_smc_hs",
+		.data		= &init_net.smc.limit_smc_hs,
+		.maxlen		= sizeof(int),
+		.mode		= 0644,
+		.proc_handler	= proc_dointvec_minmax,
+		.extra1		= SYSCTL_ZERO,
+		.extra2		= SYSCTL_ONE,
+	},
 };
 
 int __net_init smc_sysctl_net_init(struct net *net)
@@ -121,6 +130,8 @@ int __net_init smc_sysctl_net_init(struct net *net)
 	WRITE_ONCE(net->smc.sysctl_rmem, net_smc_rmem_init);
 	net->smc.sysctl_max_links_per_lgr = SMC_LINKS_PER_LGR_MAX_PREFER;
 	net->smc.sysctl_max_conns_per_lgr = SMC_CONN_PER_LGR_PREFER;
+	/* disable handshake limitation by default */
+	net->smc.limit_smc_hs = 0;
 
 	return 0;
 
Signature · 1 line
1.8.3.1
reviewer Reviewed-by on v2 1/1
#1 lore raw
Quoted 11 lines · D. Wythe

From: "D. Wythe" <[email protected]>

In commit 48b6190a0042 ("net/smc: Limit SMC visits when handshake workqueue congested"), we introduce a mechanism to put constraint on SMC connections visit according to the pressure of SMC handshake process.

At that time, we believed that controlling the feature through netlink was sufficient. However, most people have realized now that netlink is not convenient in container scenarios, and sysctl is a more suitable approach.

Quoted 6 lines

In addition, since commit 462791bbfa35 ("net/smc: add sysctl interface for SMC") had introcuded smc_sysctl_net_init(), it is reasonable for us to initialize limit_smc_hs in it instead of initializing it in smc_pnet_net_int().

LGTM. Thanks!

Quoted 53 lines

---
v1 -> v2:

Modified the description in the commit and removed the incorrect spelling.

  net/smc/smc_pnet.c   |  3 ---
  net/smc/smc_sysctl.c | 11 +++++++++++
  2 files changed, 11 insertions(+), 3 deletions(-)
net/smc/smc_pnet.c +0 -3
index 2adb92b..1dd3623 100644
@@ -887,9 +887,6 @@ int smc_pnet_net_init(struct net *net)
 
  	smc_pnet_create_pnetids_list(net);
 
-	/* disable handshake limitation by default */
-	net->smc.limit_smc_hs = 0;
-
  	return 0;
  }
 
net/smc/smc_sysctl.c +11 -0
index 13f2bc0..2fab645 100644
@@ -90,6 +90,15 @@
  		.extra1		= &conns_per_lgr_min,
  		.extra2		= &conns_per_lgr_max,
  	},
+	{
+		.procname	= "limit_smc_hs",
+		.data		= &init_net.smc.limit_smc_hs,
+		.maxlen		= sizeof(int),
+		.mode		= 0644,
+		.proc_handler	= proc_dointvec_minmax,
+		.extra1		= SYSCTL_ZERO,
+		.extra2		= SYSCTL_ONE,
+	},
  };
 
  int __net_init smc_sysctl_net_init(struct net *net)
@@ -121,6 +130,8 @@ int __net_init smc_sysctl_net_init(struct net *net)
  	WRITE_ONCE(net->smc.sysctl_rmem, net_smc_rmem_init);
  	net->smc.sysctl_max_links_per_lgr = SMC_LINKS_PER_LGR_MAX_PREFER;
  	net->smc.sysctl_max_conns_per_lgr = SMC_CONN_PER_LGR_PREFER;
+	/* disable handshake limitation by default */
+	net->smc.limit_smc_hs = 0;
 
  	return 0;
 
Reviewed-by on v2 1/1
#2 lore raw
In reply to D. Wythe
Quoted 10 lines · D. Wythe

From: "D. Wythe" <[email protected]>

In commit 48b6190a0042 ("net/smc: Limit SMC visits when handshake workqueue congested"), we introduce a mechanism to put constraint on SMC connections visit according to the pressure of SMC handshake process.

At that time, we believed that controlling the feature through netlink was sufficient. However, most people have realized now that netlink is not convenient in container scenarios, and sysctl is a more suitable approach.

Hi D.

thanks for your contribution.
What i wonder is should we prefer the use of netlink > sysctl or not? To the upstream maintainers: Is there a prefernce for the net tree?

My impression from past discussions is that netlink should be chosen over sysctl. If so, why is it inconvenient to use netlink in containers?
Can this be changed?

Other then the general discussion the changhes look good to me.

Quoted 60 lines

In addition, since commit 462791bbfa35 ("net/smc: add sysctl interface for SMC") had introcuded smc_sysctl_net_init(), it is reasonable for us to initialize limit_smc_hs in it instead of initializing it in smc_pnet_net_int().

---
v1 -> v2:

Modified the description in the commit and removed the incorrect spelling.

  net/smc/smc_pnet.c   |  3 ---
  net/smc/smc_sysctl.c | 11 +++++++++++
  2 files changed, 11 insertions(+), 3 deletions(-)
net/smc/smc_pnet.c +0 -3
index 2adb92b..1dd3623 100644
@@ -887,9 +887,6 @@ int smc_pnet_net_init(struct net *net)
 
  	smc_pnet_create_pnetids_list(net);
 
-	/* disable handshake limitation by default */
-	net->smc.limit_smc_hs = 0;
-
  	return 0;
  }
 
net/smc/smc_sysctl.c +11 -0
index 13f2bc0..2fab645 100644
@@ -90,6 +90,15 @@
  		.extra1		= &conns_per_lgr_min,
  		.extra2		= &conns_per_lgr_max,
  	},
+	{
+		.procname	= "limit_smc_hs",
+		.data		= &init_net.smc.limit_smc_hs,
+		.maxlen		= sizeof(int),
+		.mode		= 0644,
+		.proc_handler	= proc_dointvec_minmax,
+		.extra1		= SYSCTL_ZERO,
+		.extra2		= SYSCTL_ONE,
+	},
  };
 
  int __net_init smc_sysctl_net_init(struct net *net)
@@ -121,6 +130,8 @@ int __net_init smc_sysctl_net_init(struct net *net)
  	WRITE_ONCE(net->smc.sysctl_rmem, net_smc_rmem_init);
  	net->smc.sysctl_max_links_per_lgr = SMC_LINKS_PER_LGR_MAX_PREFER;
  	net->smc.sysctl_max_conns_per_lgr = SMC_CONN_PER_LGR_PREFER;
+	/* disable handshake limitation by default */
+	net->smc.limit_smc_hs = 0;
 
  	return 0;
 
maintainer
#3 lore raw
Quoted 2 lines · On Wed, 21 Aug 2024 10:03:49 +0200 Jan Karcher wrote:

If so, why is it inconvenient to use netlink in containers? Can this be changed?

Adding a YAML spec for the link to make it easily accessible from C/C++/Python applications seems like a better direction.

author maintainer
#4 lore raw
In reply to Jan Karcher
Quoted 20 lines · Jan Karcher
Quoted 10 lines · D. Wythe

From: "D. Wythe" <[email protected]>

In commit 48b6190a0042 ("net/smc: Limit SMC visits when handshake workqueue congested"), we introduce a mechanism to put constraint on SMC connections visit according to the pressure of SMC handshake process.

At that time, we believed that controlling the feature through netlink was sufficient. However, most people have realized now that netlink is not convenient in container scenarios, and sysctl is a more suitable approach.

Hi D.

thanks for your contribution.
What i wonder is should we prefer the use of netlink > sysctl or not? To the upstream maintainers: Is there a prefernce for the net tree?

Quoted 8 lines

My impression from past discussions is that netlink should be chosen over sysctl. If so, why is it inconvenient to use netlink in containers?
Can this be changed?

Other then the general discussion the changhes look good to me.

Hi Jan,

I noticed that there have been relevant discussions before, perhaps this will be helpful to you.

Best wishes,
D. Wythe

Quoted 55 lines
Quoted 55 lines

In addition, since commit 462791bbfa35 ("net/smc: add sysctl interface for SMC") had introcuded smc_sysctl_net_init(), it is reasonable for us to initialize limit_smc_hs in it instead of initializing it in smc_pnet_net_int().

---
v1 -> v2:

Modified the description in the commit and removed the incorrect spelling.

  net/smc/smc_pnet.c   |  3 ---
  net/smc/smc_sysctl.c | 11 +++++++++++

  2 files changed, 11 insertions(+), 3 deletions(-)
net/smc/smc_pnet.c +0 -0
index 2adb92b..1dd3623 100644
@@ -887,9 +887,6 @@ int smc_pnet_net_init(struct net *net)
         smc_pnet_create_pnetids_list(net);

  -    /* disable handshake limitation by default */ -    net->smc.limit_smc_hs = 0;
-
      return 0;
  }
  diff --git a/net/smc/smc_sysctl.c b/net/smc/smc_sysctl.c index 13f2bc0..2fab645 100644

net/smc/smc_sysctl.c +11 -0
@@ -90,6 +90,15 @@
           .extra1        = &conns_per_lgr_min,
           .extra2        = &conns_per_lgr_max,
       },
+    {
+        .procname    = "limit_smc_hs",
+        .data        = &init_net.smc.limit_smc_hs,
+        .maxlen        = sizeof(int),
+        .mode        = 0644,
+        .proc_handler    = proc_dointvec_minmax,
+        .extra1        = SYSCTL_ZERO,
+        .extra2        = SYSCTL_ONE,
+    },
   };
     int __net_init smc_sysctl_net_init(struct net *net)
@@ -121,6 +130,8 @@ int __net_init smc_sysctl_net_init(struct net *net)
       WRITE_ONCE(net->smc.sysctl_rmem, net_smc_rmem_init);
       net->smc.sysctl_max_links_per_lgr = SMC_LINKS_PER_LGR_MAX_PREFER;
       net->smc.sysctl_max_conns_per_lgr = SMC_CONN_PER_LGR_PREFER;
+    /* disable handshake limitation by default */
+    net->smc.limit_smc_hs = 0;
         return 0;
#5 lore raw
Quoted 29 lines · D. Wythe
Quoted 26 lines · Jan Karcher
Quoted 10 lines · D. Wythe

From: "D. Wythe" <[email protected]>

In commit 48b6190a0042 ("net/smc: Limit SMC visits when handshake workqueue congested"), we introduce a mechanism to put constraint on SMC connections visit according to the pressure of SMC handshake process.

At that time, we believed that controlling the feature through netlink was sufficient. However, most people have realized now that netlink is not convenient in container scenarios, and sysctl is a more suitable approach.

Hi D.

thanks for your contribution.
What i wonder is should we prefer the use of netlink > sysctl or not? To the upstream maintainers: Is there a prefernce for the net tree?

My impression from past discussions is that netlink should be chosen over sysctl. If so, why is it inconvenient to use netlink in containers?
Can this be changed?

Other then the general discussion the changhes look good to me.

Quoted 8 lines
Quoted 2 lines

Hi Jan,

I noticed that there have been relevant discussions before, perhaps this will be helpful to you.

Hi D.,

thanks for the pointer! If i understood Jakub correct it should be possible to add a yaml definition for the SMC netlink anbd modify it vie a small program.

That said I'm not to familia with the use of them in containers. So if you say this is the better solution and everyone is fine with yet another sysctl it is fine by me.

Thanks
- Jan

Quoted 62 lines

Best wishes,
D. Wythe

Quoted 55 lines
Quoted 55 lines

In addition, since commit 462791bbfa35 ("net/smc: add sysctl interface for SMC") had introcuded smc_sysctl_net_init(), it is reasonable for us to initialize limit_smc_hs in it instead of initializing it in smc_pnet_net_int().

---
v1 -> v2:

Modified the description in the commit and removed the incorrect spelling.

  net/smc/smc_pnet.c   |  3 ---
  net/smc/smc_sysctl.c | 11 +++++++++++

  2 files changed, 11 insertions(+), 3 deletions(-)
net/smc/smc_pnet.c +0 -0
index 2adb92b..1dd3623 100644
@@ -887,9 +887,6 @@ int smc_pnet_net_init(struct net *net)
         smc_pnet_create_pnetids_list(net);

  -    /* disable handshake limitation by default */ -    net->smc.limit_smc_hs = 0;
-
      return 0;
  }
  diff --git a/net/smc/smc_sysctl.c b/net/smc/smc_sysctl.c index 13f2bc0..2fab645 100644

net/smc/smc_sysctl.c +11 -0
@@ -90,6 +90,15 @@
           .extra1        = &conns_per_lgr_min,
           .extra2        = &conns_per_lgr_max,
       },
+    {
+        .procname    = "limit_smc_hs",
+        .data        = &init_net.smc.limit_smc_hs,
+        .maxlen        = sizeof(int),
+        .mode        = 0644,
+        .proc_handler    = proc_dointvec_minmax,
+        .extra1        = SYSCTL_ZERO,
+        .extra2        = SYSCTL_ONE,
+    },
   };
     int __net_init smc_sysctl_net_init(struct net *net)
@@ -121,6 +130,8 @@ int __net_init smc_sysctl_net_init(struct net *net)
       WRITE_ONCE(net->smc.sysctl_rmem, net_smc_rmem_init);
       net->smc.sysctl_max_links_per_lgr = SMC_LINKS_PER_LGR_MAX_PREFER;
       net->smc.sysctl_max_conns_per_lgr = SMC_CONN_PER_LGR_PREFER;
+    /* disable handshake limitation by default */
+    net->smc.limit_smc_hs = 0;
         return 0;
author maintainer
#6 lore raw
In reply to D. Wythe
Quoted 11 lines · D. Wythe

From: "D. Wythe" <[email protected]>

In commit 48b6190a0042 ("net/smc: Limit SMC visits when handshake workqueue congested"), we introduce a mechanism to put constraint on SMC connections visit according to the pressure of SMC handshake process.

At that time, we believed that controlling the feature through netlink was sufficient. However, most people have realized now that netlink is not convenient in container scenarios, and sysctl is a more suitable approach.

Hi everyone,

Just a quick reminder regarding the patch that seems to have been overlooked, possibly dues to its status was mistakenly updated to change request ? It seems that no further modifications are needed.

Best wishes,
D. Wythe

#7 lore raw
Quoted 5 lines

Hi everyone,

Just a quick reminder regarding the patch that seems to have been overlooked, possibly dues to its status was
mistakenly updated to change request ?

Once it gets set to change request, it is effectively dead. Please repost.

Andrew

author maintainer
#8 lore raw
Quoted 9 lines · Andrew Lunn
Quoted 5 lines

Hi everyone,

Just a quick reminder regarding the patch that seems to have been overlooked, possibly dues to its status was
mistakenly updated to change request ?

Once it gets set to change request, it is effectively dead. Please repost.

Andrew

Thanks for the reminder,  I will repost it later.

Best wishes,
D. Wythe