Skip to contentSkip to conversation

dlm fix for 2.6.38

GIT PULL David Teigland 3 participants 7 messages Feb 11, 2011 – Feb 14, 2011 Distributed lock manager (DLM)

lore
Share & tools
lore

David Teigland

author maintainer
#0 lore raw

Linus,

Please pull this fix for a dlm regression from:

git://git.kernel.org/pub/scm/linux/kernel/git/teigland/dlm.git for-linus

Thanks,
Dave

Author: David Teigland <[email protected]>
Date:   Fri Feb 11 16:44:31 2011 -0600

dlm: use single thread workqueues

The recent commit to use cmwq for send and recv threads dcce240ead802d42b1e45ad2fcb2ed4a399cb255 introduced problems, apparently due to multiple workqueue threads. Single threads make the problems go away, so return to that until we fully understand the concurrency issues with multiple threads.

fs/dlm/lowcomms.c +2 -4
index 9c64ae9..2d8c87b 100644
@@ -1468,15 +1468,13 @@ static void work_stop(void)
 
 static int work_start(void)
 {
-	recv_workqueue = alloc_workqueue("dlm_recv", WQ_MEM_RECLAIM |
-					 WQ_HIGHPRI | WQ_FREEZEABLE, 0);
+	recv_workqueue = create_singlethread_workqueue("dlm_recv");
 	if (!recv_workqueue) {
 		log_print("can't start dlm_recv");
 		return -ENOMEM;
 	}
 
-	send_workqueue = alloc_workqueue("dlm_send", WQ_MEM_RECLAIM |
-					 WQ_HIGHPRI | WQ_FREEZEABLE, 0);
+	send_workqueue = create_singlethread_workqueue("dlm_send");
 	if (!send_workqueue) {
 		log_print("can't start dlm_send");
 		destroy_workqueue(recv_workqueue);
#1 lore raw

Hi,

Quoted 36 lines · David Teigland

Linus,

Please pull this fix for a dlm regression from:

git://git.kernel.org/pub/scm/linux/kernel/git/teigland/dlm.git for-linus

Thanks,
Dave

Author: David Teigland <[email protected]>
Date:   Fri Feb 11 16:44:31 2011 -0600

dlm: use single thread workqueues

The recent commit to use cmwq for send and recv threads dcce240ead802d42b1e45ad2fcb2ed4a399cb255 introduced problems, apparently due to multiple workqueue threads. Single threads make the problems go away, so return to that until we fully understand the concurrency issues with multiple threads.

fs/dlm/lowcomms.c +1 -2
index 9c64ae9..2d8c87b 100644
@@ -1468,15 +1468,13 @@ static void work_stop(void)
 
 static int work_start(void)
 {
-	recv_workqueue = alloc_workqueue("dlm_recv", WQ_MEM_RECLAIM |
-					 WQ_HIGHPRI | WQ_FREEZEABLE, 0);
+	recv_workqueue = create_singlethread_workqueue("dlm_recv");
 	if (!recv_workqueue) {

fs/dlm/lowcomms.c static void work_stop(void) · line 1480

Quoted 10 lines
fs/dlm/lowcomms.c +1 -2
 		log_print("can't start dlm_recv");
 		return -ENOMEM;
 	}
 
-	send_workqueue = alloc_workqueue("dlm_send", WQ_MEM_RECLAIM |
-					 WQ_HIGHPRI | WQ_FREEZEABLE, 0);
+	send_workqueue = create_singlethread_workqueue("dlm_send");
 	if (!send_workqueue) {
 		log_print("can't start dlm_send");
 		destroy_workqueue(recv_workqueue);

What is the issue here? If there is a problem with the workqueues then we should ask Tejun about it,

Steve.

#2 lore raw

Hello,

Quoted 11 lines · Steven Whitehouse

fs/dlm/lowcomms.c static void work_stop(void) · line 1471

Quoted 11 lines
fs/dlm/lowcomms.c +1 -2
index 9c64ae9..2d8c87b 100644
@@ -1468,15 +1468,13 @@ static void work_stop(void)
 
 static int work_start(void)
 {
-	recv_workqueue = alloc_workqueue("dlm_recv", WQ_MEM_RECLAIM |
-					 WQ_HIGHPRI | WQ_FREEZEABLE, 0);
+	recv_workqueue = create_singlethread_workqueue("dlm_recv");

recv_workqueue was multithread one even before the conversion. It probably is best to leave this part alone.

Quoted 3 lines

fs/dlm/lowcomms.c static void work_stop(void) · line 1477

Quoted 3 lines
fs/dlm/lowcomms.c +1 -2
-	send_workqueue = alloc_workqueue("dlm_send", WQ_MEM_RECLAIM |
-					 WQ_HIGHPRI | WQ_FREEZEABLE, 0);
+	send_workqueue = create_singlethread_workqueue("dlm_send");

send_workqueue was converted from ST to MT but the correct way at this point would be,

	alloc_ordered_workqueue("dlm_send",
				WQ_MEM_RECLAIM | WQ_HIGHPRI | WQ_FREEZEABLE);
Quoted 8 lines

fs/dlm/lowcomms.c static void work_stop(void)

Quoted 5 lines
fs/dlm/lowcomms.c +0 -0
 	if (!send_workqueue) {
 		log_print("can't start dlm_send");
 		destroy_workqueue(recv_workqueue);
 
 

What is the issue here? If there is a problem with the workqueues then we should ask Tejun about it,

Yeah, what kind of problem was it? There's only one work per connection so reordering is not a problem. All the workqueue operations use proper locking, so the conversion seemed safe to me. What am I missing?

Thanks.

Signature · 1 line
tejun
author maintainer
#3 lore raw
Quoted 8 lines · Tejun Heo
Quoted 2 lines · Steven Whitehouse

What is the issue here? If there is a problem with the workqueues then we should ask Tejun about it,

Yeah, what kind of problem was it? There's only one work per connection so reordering is not a problem. All the workqueue operations use proper locking, so the conversion seemed safe to me. What am I missing?

find_lkb seems to be getting an actual, but wrong lkid, so it's returning the wrong lkb in the receive routines. It happens fairly quickly with multiple wq threads, but not at all with single. One suspect I'm going to look at are the ls_stub and fields in the lockspace struct. I'm not convinced extra send/recv threads give us that much benefit in practice, so it's not my top priority at the moment.

Dave

#4 lore raw

Hello,

Quoted 6 lines · David Teigland

find_lkb seems to be getting an actual, but wrong lkid, so it's returning the wrong lkb in the receive routines. It happens fairly quickly with multiple wq threads, but not at all with single. One suspect I'm going to look at are the ls_stub and fields in the lockspace struct. I'm not convinced extra send/recv threads give us that much benefit in practice, so it's not my top priority at the moment.

Hmmm... okay. At any rate, please use alloc[_ordered]_workqueue() interface. create[_singlethread]_workqueue() will be deprecated soon.

Thanks.

Signature · 1 line
tejun
author maintainer
#5 lore raw
Quoted 10 lines · Tejun Heo

Hello,

Quoted 6 lines · David Teigland

find_lkb seems to be getting an actual, but wrong lkid, so it's returning the wrong lkb in the receive routines. It happens fairly quickly with multiple wq threads, but not at all with single. One suspect I'm going to look at are the ls_stub and fields in the lockspace struct. I'm not convinced extra send/recv threads give us that much benefit in practice, so it's not my top priority at the moment.

Quoted 2 lines

Hmmm... okay. At any rate, please use alloc[_ordered]_workqueue() interface. create[_singlethread]_workqueue() will be deprecated soon.

Oh, I'll switch to the other API in the patch set for the next merge. Dave

#6 lore raw
Quoted 5 lines · David Teigland
Quoted 2 lines

Hmmm... okay. At any rate, please use alloc[_ordered]_workqueue() interface. create[_singlethread]_workqueue() will be deprecated soon.

Oh, I'll switch to the other API in the patch set for the next merge. Dave

Alright. Sounds good. Thanks.

Signature · 1 line
tejun