Skip to contentSkip to conversation

dlm: prompt the user SCTP is experimental

PATCH In review Gang He 3 participants 7 messages Mar 20, 2018 – Apr 10, 2018 1 message not in the archive Distributed lock manager (DLM)

lore
Share & tools
lore

Gang He

author
#0 lore raw

PATCH 1/1 dlm: prompt the user SCTP is experimental

As you know, DLM module can use TCP or SCTP protocols to communicate among the cluster.
But, according to our testing, SCTP protocol is still considered experimental, since not all aspects are working correctly and it is not full tested.
e.g. SCTP connection channel switch needs about 5mins hang in case one connection(ring) is broken.
Then, I suggest to add a kernel print, which prompts the user SCTP protocol for DLM should be considered experimental, it is not recommended in production environment.

---
 fs/dlm/lowcomms.c | 1 +
 1 file changed, 1 insertion(+)
fs/dlm/lowcomms.c +1 -0
index cff79ea..18fd85d 100644
@@ -1307,6 +1307,7 @@ static int sctp_listen_for_all(void)
 		return -ENOMEM;
 
 	log_print("Using SCTP for communications");
+	log_print("SCTP protocol is experimental, use at your own risk");
 
 	result = sock_create_kern(&init_net, dlm_local_addr[0]->ss_family,
 				  SOCK_STREAM, IPPROTO_SCTP, &sock);
Signature · 1 line
1.8.5.6
author
#1 lore raw

Hello David,

Do you agree to add this prompt to the user?
Since sometimes customers attempted to setup SCTP protocol with two rings,
but they could not get the expected result, then it maybe bring some concerns to the customer for DLM qualities.

Thanks
Gang

Quoted 31 lines
Quoted 1 line
Quoted 1 line

As you know, DLM module can use TCP or SCTP protocols to communicate among the cluster.
But, according to our testing, SCTP protocol is still considered experimental, since not all aspects are working correctly and it is not full tested.
e.g. SCTP connection channel switch needs about 5mins hang in case one connection(ring) is broken.
Then, I suggest to add a kernel print, which prompts the user SCTP protocol for DLM should be considered experimental, it is not recommended in production environment.

---
 fs/dlm/lowcomms.c | 1 +
 1 file changed, 1 insertion(+)
fs/dlm/lowcomms.c +1 -0
index cff79ea..18fd85d 100644
@@ -1307,6 +1307,7 @@ static int sctp_listen_for_all(void)
 		return -ENOMEM;
 
 	log_print("Using SCTP for communications");
+	log_print("SCTP protocol is experimental, use at your own risk");
 
 	result = sock_create_kern(&init_net, dlm_local_addr[0]->ss_family,
 				  SOCK_STREAM, IPPROTO_SCTP, &sock);
Signature · 1 line
1.8.5.6
maintainer
#2 lore raw
Quoted 5 lines · Gang He

Hello David,

Do you agree to add this prompt to the user?
Since sometimes customers attempted to setup SCTP protocol with two rings,
but they could not get the expected result, then it maybe bring some concerns to the customer for DLM qualities.

I don't think the kernel message is a good way to communicate this to users. Dave

Quoted 30 lines
Quoted 30 lines

As you know, DLM module can use TCP or SCTP protocols to communicate among the cluster.
But, according to our testing, SCTP protocol is still considered experimental, since not all aspects are working correctly and it is not full tested.
e.g. SCTP connection channel switch needs about 5mins hang in case one connection(ring) is broken.
Then, I suggest to add a kernel print, which prompts the user SCTP protocol for DLM should be considered experimental, it is not recommended in production environment.

---
 fs/dlm/lowcomms.c | 1 +
 1 file changed, 1 insertion(+)
fs/dlm/lowcomms.c +1 -0
index cff79ea..18fd85d 100644
@@ -1307,6 +1307,7 @@ static int sctp_listen_for_all(void)
 		return -ENOMEM;
 
 	log_print("Using SCTP for communications");
+	log_print("SCTP protocol is experimental, use at your own risk");
 
 	result = sock_create_kern(&init_net, dlm_local_addr[0]->ss_family,
 				  SOCK_STREAM, IPPROTO_SCTP, &sock);
Signature · 1 line
1.8.5.6
author
#3 lore raw

Hi David,

Quoted 11 lines
Quoted 1 line
Quoted 1 line
Quoted 5 lines · Gang He

Hello David,

Do you agree to add this prompt to the user?
Since sometimes customers attempted to setup SCTP protocol with two rings, but they could not get the expected result, then it maybe bring some

concerns to the customer for DLM qualities.

I don't think the kernel message is a good way to communicate this to users. Dave

OK, I got your point.
But, could we have a appropriate way to let the users know SCTP protocol status?

Thanks
Gang

Quoted 32 lines
Quoted 32 lines
Quoted 32 lines

As you know, DLM module can use TCP or SCTP protocols to communicate among the cluster.
But, according to our testing, SCTP protocol is still considered experimental, since not all aspects are working correctly and it is not full tested.
e.g. SCTP connection channel switch needs about 5mins hang in case one connection(ring) is broken.
Then, I suggest to add a kernel print, which prompts the user SCTP protocol for DLM should be considered experimental, it is not recommended in production environment.

---
 fs/dlm/lowcomms.c | 1 +
 1 file changed, 1 insertion(+)
fs/dlm/lowcomms.c +1 -0
index cff79ea..18fd85d 100644
@@ -1307,6 +1307,7 @@ static int sctp_listen_for_all(void)
 		return -ENOMEM;
 
 	log_print("Using SCTP for communications");
+	log_print("SCTP protocol is experimental, use at your own risk");
 
 	result = sock_create_kern(&init_net, dlm_local_addr[0]->ss_family,
 				  SOCK_STREAM, IPPROTO_SCTP, &sock);
Signature · 1 line
1.8.5.6

A message that is not in the archive (sent off-list or not yet received): [email protected]

author
#5 lore raw

PATCH 1/1 dlm: prompt the user SCTP is experimental

Hello David,

If the user sets "protocol=tcp" in the configuration file /etc/dlm/dlm.conf under two-rings cluster environment, DLM kernel module will not work with the below error message,

[   43.696924] DLM installed
[  149.552039] ocfs2: Registered cluster interface user
[  149.559579] dlm: TCP protocol can't handle multi-homed hosts, try SCTP  <<== here, failed
[  149.559589] dlm: cannot start dlm lowcomms -22
[  149.559612] (mount.ocfs2,2593,3):ocfs2_dlm_init:3120 ERROR: status = -22
[  149.559629] (mount.ocfs2,2593,3):ocfs2_mount_volume:1845 ERROR: status = -22 

Then, could we modify the code, let this case still work via only using one ring address? or the code is written by purpose. in lowcomms.c
1358 static int tcp_listen_for_all(void)
1359 {

1360         struct socket *sock = NULL;
1361         struct connection *con = nodeid2con(0, GFP_NOFS);
1362         int result = -EINVAL;

1363
1364 if (!con)
1365 return -ENOMEM;
1366
1367 /* We don't support multi-homed hosts */
1368 if (dlm_local_addr[1] != NULL) { <<== here, could we get ride of this limitation?
1369 log_print("TCP protocol can't handle multi-homed hosts, "
1370 "try SCTP");
1371 return -EINVAL;
1372 }
1373
1374 log_print("Using TCP for communications");
1375
1376 sock = tcp_create_listen_sock(con, dlm_local_addr[0]);
1377 if (sock) {
1378 add_sock(sock, con);
1379 result = 0;
1380 }
1381 else {
1382 result = -EADDRINUSE;
1383 }

Thanks
Gang

Quoted 13 lines
Quoted 1 line
Quoted 1 line
Quoted 2 lines · Gang He

OK, I got your point.
But, could we have a appropriate way to let the users know SCTP protocol

status?

I think this is a case where suse/rh/etc need to have their own distro-specific approaches for specifying the usage parameters that they have tested and found to be supportable. Other companies have previously found their specific use of SCTP to be acceptable. RH does not properly support dlm+SCTP for similar reasons as you've found, although I've more recently encouraged customers to try dlm+SCTP with a single path in order debug or diagnose potential networking issues.

#6 lore raw

PATCH 1/1 dlm: prompt the user SCTP is experimental

Hi,

Quoted 41 lines · Gang He

Hello David,

If the user sets "protocol=tcp" in the configuration file /etc/dlm/dlm.conf under two-rings cluster environment, DLM kernel module will not work with the below error message,

[   43.696924] DLM installed
[  149.552039] ocfs2: Registered cluster interface user
[  149.559579] dlm: TCP protocol can't handle multi-homed hosts, try SCTP  <<== here, failed
[  149.559589] dlm: cannot start dlm lowcomms -22
[  149.559612] (mount.ocfs2,2593,3):ocfs2_dlm_init:3120 ERROR: status = -22
[  149.559629] (mount.ocfs2,2593,3):ocfs2_mount_volume:1845 ERROR: status = -22

Then, could we modify the code, let this case still work via only using one ring address? or the code is written by purpose. in lowcomms.c
1358 static int tcp_listen_for_all(void)
1359 {

1360         struct socket *sock = NULL;
1361         struct connection *con = nodeid2con(0, GFP_NOFS);
1362         int result = -EINVAL;

1363
1364 if (!con)
1365 return -ENOMEM;
1366
1367 /* We don't support multi-homed hosts */
1368 if (dlm_local_addr[1] != NULL) { <<== here, could we get ride of this limitation?
1369 log_print("TCP protocol can't handle multi-homed hosts, "
1370 "try SCTP");
1371 return -EINVAL;
1372 }
1373
1374 log_print("Using TCP for communications");
1375
1376 sock = tcp_create_listen_sock(con, dlm_local_addr[0]);
1377 if (sock) {
1378 add_sock(sock, con);
1379 result = 0;
1380 }
1381 else {
1382 result = -EADDRINUSE;
1383 }

Quoted 3 lines

Thanks
Gang

There is already a patch set to allow multi-homing for TCP. Mark and Dave can comment on the current status and how far from merging it currently is,

Steve.

author
#7 lore raw

PATCH 1/1 dlm: prompt the user SCTP is experimental

Hi Steven and David,

Quoted 52 lines
Quoted 1 line
Quoted 1 line

Hi,

Quoted 3 lines · Gang He

Hello David,

If the user sets "protocol=tcp" in the configuration file /etc/dlm/dlm.conf

under two-rings cluster environment,

Quoted 4 lines

DLM kernel module will not work with the below error message,

[   43.696924] DLM installed
[  149.552039] ocfs2: Registered cluster interface user
[  149.559579] dlm: TCP protocol can't handle multi-homed hosts, try SCTP  

<<== here, failed

Quoted 3 lines
[  149.559589] dlm: cannot start dlm lowcomms -22
[  149.559612] (mount.ocfs2,2593,3):ocfs2_dlm_init:3120 ERROR: status = -22
[  149.559629] (mount.ocfs2,2593,3):ocfs2_mount_volume:1845 ERROR: status = 

-22

Quoted 2 lines

Then, could we modify the code, let this case still work via only using one

ring address? or the code is written by purpose.

Quoted 6 lines

in lowcomms.c
1358 static int tcp_listen_for_all(void)
1359 {

1360         struct socket *sock = NULL;
1361         struct connection *con = nodeid2con(0, GFP_NOFS);
1362         int result = -EINVAL;
Quoted 6 lines

1363
1364 if (!con)
1365 return -ENOMEM;
1366
1367 /* We don't support multi-homed hosts */
1368 if (dlm_local_addr[1] != NULL) { <<== here, could we get ride

of this limitation?

Quoted 1 line

1369 log_print("TCP protocol can't handle multi-homed hosts,

"

Quoted 16 lines

1370 "try SCTP");
1371 return -EINVAL;
1372 }
1373
1374 log_print("Using TCP for communications");
1375
1376 sock = tcp_create_listen_sock(con, dlm_local_addr[0]);
1377 if (sock) {
1378 add_sock(sock, con);
1379 result = 0;
1380 }
1381 else {
1382 result = -EADDRINUSE;
1383 }

Quoted 6 lines
Quoted 3 lines

Thanks
Gang

There is already a patch set to allow multi-homing for TCP. Mark and Dave can comment on the current status and how far from merging it currently is,

Thanks for your update on this problem, hopefully we can the related patches in the Linus git tree soon.

Thanks
Gang

Quoted 2 lines

Steve.