Problem with freezable workqueues
Subsystems (2)
Hi,
We have a problem with freezable workqueues in 2.6.21-rc1 and in -mm (there are only two of them, in XFS, but still). Namely, their worker threads deadlock with workqueue_cpu_callback() that gets called during the CPU hotplug, becuase workqueue_cpu_callback() tries to stop these threads while they are frozen (disable_nonboot_cpus() happens after we've frozen tasks).
For 2.6.21-rc1 I've invented the appended workaround (works for me, waiting for Johannes to confirm it works for him too), but I think we need something better for -mm and future kernels.
Greetings,
Rafael
kernel/workqueue.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-)
kernel/workqueue.c +16 -2
@@ -376,8 +376,19 @@ static int worker_thread(void *__cwq)set_current_state (TASK_INTERRUPTIBLE );while (!kthread_should_stop ()) { -if (cwq->freezeable) -try_to_freeze (); +if (try_to_freeze ()) { +/* We've just left the refrigerator. If our CPU is +* a nonboot one, we might have been replaced. +* The lock is taken to prevent the race with +* cleanup_workqueue_thread() from happening +*/ +spin_lock_irq (&cwq->lock); +if (cwq->thread != current) { +spin_unlock_irq (&cwq->lock); +break ; + } +spin_unlock_irq (&cwq->lock); + }add_wait_queue (&cwq->more_work, &wait);if (list_empty (&cwq->worklist))
@@ -539,6 +550,9 @@ static void cleanup_workqueue_thread(str cwq =per_cpu_ptr (wq->cpu_wq, cpu);spin_lock_irqsave (&cwq->lock, flags); p = cwq->thread; +if (p &&frozen (p)) + p =NULL ; + cwq->thread =NULL ;spin_unlock_irqrestore (&cwq->lock, flags);if (p)
Quoted 6 lines · Rafael J. Wysocki
We have a problem with freezable workqueues in 2.6.21-rc1 and in -mm (there are only two of them, in XFS, but still). Namely, their worker threads deadlock with
workqueue_cpu_callback()that gets called during the CPU hotplug, becuaseworkqueue_cpu_callback()tries to stop these threads while they are frozen (disable_nonboot_cpus()happens after we've frozen tasks).
Ugh. I know nothing, nothing, nothing about suspend. I'll try to guess.
Commit: ed746e3b18f4df18afa3763155972c5835f284c5
[PATCH] swsusp: Change code ordering in disk.c
Change the ordering of code in kernel/power/disk.c so that device_suspend() is called before disable_nonboot_cpus() and platform_finish() is called after enable_nonboot_cpus() and before device_resume(), as indicated by the recent discussion on Linux-PM (cf.
http://lists.osdl.org/pipermail/linux-pm/2006-November/004164.html).
The changes here only affect the built-in swsusp.
Yes? with the patch above, _cpu_down() called _after_ freeze_processes() ??? Honestly, I can't understand this (yes, I know nothing, nothing, nothing...).
Quoted 3 lines
For 2.6.21-rc1 I've invented the appended workaround (works for me, waiting for Johannes to confirm it works for him too), but I think we need something better for -mm and future kernels.
How about other kthread_stop()s ? For example, kernel/softirq.c:cpu_callback() ?
I think we need a general "cpu_down() after freeze" implementation, this is what Gautham and Srivatsa are working on, right?
Quoted 5 lines
kernel/workqueue.c +0
-0@@ -376,8 +376,19 @@ static int worker_thread(void *__cwq)set_current_state (TASK_INTERRUPTIBLE );
kernel/workqueue.c static int worker_thread(void *__cwq) · line 385
Quoted 10 lines
kernel/workqueue.c +7
-2while (!kthread_should_stop ()) { -if (cwq->freezeable) -try_to_freeze (); +if (try_to_freeze ()) { +/* We've just left the refrigerator. If our CPU is +* a nonboot one, we might have been replaced. +* The lock is taken to prevent the race with +* cleanup_workqueue_thread() from happening +*/ +spin_lock_irq (&cwq->lock);
I'm afraid this is racy. We can't touch *cwq, it may be freed. Suppose that another thread does destroy_workqueue(), and we thaw that thread before cwq->thread.
Oleg.
Quoted 5 lines · Oleg Nesterov
Ugh. I know nothing, nothing, nothing about suspend. I'll try to guess.
Commit: ed746e3b18f4df18afa3763155972c5835f284c5Yes? with the patch above,
_cpu_down()called _after_freeze_processes()???
perfect :)
See also my original mail and the thread:
http://thread.gmane.org/gmane.linux.power-management.general/4314
Quoted 1 line
How about other kthread_stop()s ? For example, kernel/softirq.c:cpu_callback() ?
I'd they should be affected as well. For me, I guess I haven't run into them because xfs is enough to freeze the box ;) As for Rafael, I don't know why he isn't seeing that... I haven't been able to test this patch on my quad powermac so far unfortunately, been sick and away from the machine. I'll probably be able to test tomorrow.
Quoted 1 line
I think we need a general "
cpu_down()after freeze" implementation,
Yeah, I'd think so.
Quoted 2 lines
this is what
Gautham and Srivatsa are working on, right?
That I don't know.
johannes
Quoted 8 lines · Johannes Berg
Quoted 5 lines · Oleg Nesterov
Ugh. I know nothing, nothing, nothing about suspend. I'll try to guess.
Commit: ed746e3b18f4df18afa3763155972c5835f284c5Yes? with the patch above,
_cpu_down()called _after_freeze_processes()???
Quoted 7 lines
perfect :)
See also my original mail and the thread:
http://thread.gmane.org/gmane.linux.power-management.general/4314Quoted 1 line
How about other kthread_stop()s ? For example, kernel/softirq.c:cpu_callback() ?
I'd they should be affected as well.
They won't be, if they have PF_NOFREEZE set.
Quoted 2 lines
For me, I guess I haven't run into them because xfs is enough to freeze the box ;)
I think you've hit the only thing that could have triggered the issue. ;-)
Greetings,
Rafael
Quoted 5 lines · Rafael J. Wysocki
Quoted 3 lines
Quoted 1 line
How about other kthread_stop()s ? For example, kernel/softirq.c:cpu_callback() ?
I'd they should be affected as well.
They won't be, if they have
PF_NOFREEZEset.
Yup, I missed that.
johannes
Quoted 12 lines · Johannes Berg
Quoted 5 lines · Oleg Nesterov
Ugh. I know nothing, nothing, nothing about suspend. I'll try to guess.
Commit: ed746e3b18f4df18afa3763155972c5835f284c5Yes? with the patch above,
_cpu_down()called _after_freeze_processes()???perfect :)
See also my original mail and the thread:
http://thread.gmane.org/gmane.linux.power-management.general/4314
Quoted 7 lines
Quoted 1 line
How about other kthread_stop()s ? For example, kernel/softirq.c:cpu_callback() ?
I'd they should be affected as well. For me, I guess I haven't run into them because xfs is enough to freeze the box ;) As for Rafael, I don't know why he isn't seeing that... I haven't been able to test this patch on my quad powermac so far unfortunately, been sick and away from the machine. I'll probably be able to test tomorrow.
They will be affected. In our first implemetentation of general "cpu_down after freeze"(http://lkml.org/lkml/2007/2/14/107) we had a new state known CPU_DEAD_KILL_THREADS,
the notifications for which were being sent
*after* we thawed the processes in __cpu_down.
However, in the version which we are working on, we are thawing processes individually in CPU_DEAD before cleaning/stopping them.
I haven't observed any bad lockups/freeze chills with this approach. But I need to test it a bit more before posting them.
regards
gautham.
Signature · 5 lines
Gautham R Shenoy Linux Technology Center IBM India. "Freedom comes with a price tag of responsibility, which is still a bargain, because Freedom is priceless!"
Quoted 11 lines · Oleg Nesterov
Quoted 6 lines · Rafael J. Wysocki
We have a problem with freezable workqueues in 2.6.21-rc1 and in -mm (there are only two of them, in XFS, but still). Namely, their worker threads deadlock with
workqueue_cpu_callback()that gets called during the CPU hotplug, becuaseworkqueue_cpu_callback()tries to stop these threads while they are frozen (disable_nonboot_cpus()happens after we've frozen tasks).Ugh. I know nothing, nothing, nothing about suspend. I'll try to guess.
Commit: ed746e3b18f4df18afa3763155972c5835f284c5
Yes, but not only this one. Please see:
Quoted 11 lines
[PATCH] swsusp: Change code ordering in
disk.cChange the ordering of code in
kernel/power/disk.cso thatdevice_suspend()is called beforedisable_nonboot_cpus()andplatform_finish()is called afterenable_nonboot_cpus()and beforedevice_resume(), as indicated by the recent discussion on Linux-PM (cf.
http://lists.osdl.org/pipermail/linux-pm/2006-November/004164.html).The changes here only affect the built-in swsusp.
Yes? with the patch above,
_cpu_down()called _after_freeze_processes()???
Yes, that's what we want to do.
Quoted 1 line
Honestly, I can't understand this (yes, I know nothing, nothing, nothing...).
Well, we used the CPU hotplug for disabling nonboot CPUs before freeze_processes() was called, because the freezer used to be totally unsafe on SMP. However, what we wanted from the beginning was to freeze tasks with all CPUs on line (this way, for example, userland tasks should not notice that we have played with the CPUs under them, but there are other reasons).
Quoted 5 lines
Quoted 3 lines
For 2.6.21-rc1 I've invented the appended workaround (works for me, waiting for Johannes to confirm it works for him too), but I think we need something better for -mm and future kernels.
How about other kthread_stop()s ? For example, kernel/softirq.c:cpu_callback() ?
They all are PF_NOFREEZE, I suppose. If we make all workqueues nonfreezable (as they were before), the problem won't appear.
Quoted 2 lines
I think we need a general "
cpu_down()after freeze" implementation, this is what Gautham and Srivatsa are working on, right?
Yes, certainly.
Quoted 4 lines
Quoted 4 lines
kernel/workqueue.c +0
-0@@ -376,8 +376,19 @@ static int worker_thread(void *__cwq)
Quoted 15 lines
kernel/workqueue.c static int worker_thread(void *__cwq) · line 385
Quoted 11 lines
kernel/workqueue.c +7
-2set_current_state (TASK_INTERRUPTIBLE );while (!kthread_should_stop ()) { -if (cwq->freezeable) -try_to_freeze (); +if (try_to_freeze ()) { +/* We've just left the refrigerator. If our CPU is +* a nonboot one, we might have been replaced. +* The lock is taken to prevent the race with +* cleanup_workqueue_thread() from happening +*/ +spin_lock_irq (&cwq->lock);I'm afraid this is racy. We can't touch *cwq, it may be freed. Suppose that another thread does
destroy_workqueue(), and we thaw that thread beforecwq->thread.
Okay, in that case I'd suggest removing create_freezeable_workqueue() and make all workqueues nonfreezable once again for 2.6.21 (as far as I know, only the two XFS workqueues are affected).
Pavel, would that be acceptable?
Rafael
Quoted 3 lines · Rafael J. Wysocki
Okay, in that case I'd suggest removing
create_freezeable_workqueue()and make all workqueues nonfreezable once again for 2.6.21 (as far as I know, only the two XFS workqueues are affected).
I think Nigel might object but I forgot what specific trouble XFS was causing him.
johannes
Quoted 8 lines · Johannes Berg
Quoted 3 lines · Rafael J. Wysocki
Okay, in that case I'd suggest removing
create_freezeable_workqueue()and make all workqueues nonfreezable once again for 2.6.21 (as far as I know, only the two XFS workqueues are affected).I think Nigel might object but I forgot what specific trouble XFS was causing him.
We suspected that the XFS' worker threads might commit I/O after freeze_processes() has returned, but that hasn't been supported by evidence, as far as I can recall.
Also, making them freezable was controversial ...
Greetings,
Rafael
Hi.
Quoted 7 lines · Rafael J. Wysocki
Quoted 6 lines · Johannes Berg
Quoted 3 lines · Rafael J. Wysocki
Okay, in that case I'd suggest removing
create_freezeable_workqueue()and make all workqueues nonfreezable once again for 2.6.21 (as far as I know, only the two XFS workqueues are affected).
Quoted 8 lines
Quoted 2 lines
I think Nigel might object but I forgot what specific trouble XFS was causing him.
We suspected that the XFS' worker threads might commit I/O after
freeze_processes()has returned, but that hasn't been supported by evidence, as far as I can recall.Also, making them freezable was controversial ...
Controversy is no reason to give in! Nevertheless, I think you're right
- I believe the XFS guys said they fixed the issue that had caused I/O to be submitted post-freeze. Well, we'll see if it appears again, won't we?
Regards,
Nigel
Quoted 17 lines · Nigel Cunningham
Hi.
Quoted 14 lines · Rafael J. Wysocki
Quoted 8 lines · Johannes Berg
Quoted 3 lines · Rafael J. Wysocki
Okay, in that case I'd suggest removing
create_freezeable_workqueue()and make all workqueues nonfreezable once again for 2.6.21 (as far as I know, only the two XFS workqueues are affected).I think Nigel might object but I forgot what specific trouble XFS was causing him.
We suspected that the XFS' worker threads might commit I/O after
freeze_processes()has returned, but that hasn't been supported by evidence, as far as I can recall.
Quoted 6 lines
Quoted 1 line
Also, making them freezable was controversial ...
Controversy is no reason to give in! Nevertheless, I think you're right
- I believe the XFS guys said they fixed the issue that had caused I/O to be submitted post-freeze. Well, we'll see if it appears again, won't we?
Sure, we will. :-)
Rafael
Quoted 4 lines · Nigel Cunningham
Controversy is no reason to give in! Nevertheless, I think you're right
- I believe the XFS guys said they fixed the issue that had caused I/O to be submitted post-freeze. Well, we'll see if it appears again, won't we?
I get to be the guinea pig, right? :P Unfortunately I was sick for the better part of the past few days and can only test all this stuff early next week.
johannes
Quoted 4 lines · Rafael J. Wysocki
Quoted 1 line
How about other kthread_stop()s ? For example, kernel/softirq.c:cpu_callback() ?
They all are
PF_NOFREEZE, I suppose. If we make all workqueues nonfreezable (as they were before), the problem won't appear.
We can just thaw the worker thread selectively before kthread_stopping them. This will let us freeze all worker threads (which we want to for hotplug anyway).
Quoted 4 lines
Quoted 2 lines
I think we need a general "
cpu_down()after freeze" implementation, this is what Gautham and Srivatsa are working on, right?Yes, certainly.
Hmm ..good point. So can we assume that disable/enable_nonboot_cpus() are called with processes frozen already?
Gautham, you need to take this into account in your patchset!
Quoted 7 lines
Quoted 3 lines
I'm afraid this is racy. We can't touch *cwq, it may be freed. Suppose that another thread does
destroy_workqueue(), and we thaw that thread beforecwq->thread.Okay, in that case I'd suggest removing
create_freezeable_workqueue()and make all workqueues nonfreezable once again for 2.6.21 (as far as I know, only the two XFS workqueues are affected).
See above suggestion of thawing worker thread before kthread_stopping it.
Signature · 2 lines
Regards, vatsa
Quoted 10 lines · Srivatsa Vaddagiri
Quoted 4 lines · Rafael J. Wysocki
Quoted 1 line
How about other kthread_stop()s ? For example, kernel/softirq.c:cpu_callback() ?
They all are
PF_NOFREEZE, I suppose. If we make all workqueues nonfreezable (as they were before), the problem won't appear.We can just thaw the worker thread selectively before
kthread_stoppingthem. This will let us freeze all worker threads (which we want to for hotplug anyway).
I am not sure this is a good change for 2.6.21.
I strongly believe it is better to change XFS so that it doesn't use create_freezeable_workqueue() as Rafael suggested. Besides, freezeable workqueues are buggy anyway in 2.6.21-rc,
http://marc.theaimsgroup.com/?l=linux-kernel&m=116855740612755
This means that workqueues become non-freezeable after suspend/resume anyway (if I understand disable_nonboot_cpus() correctly).
Oleg.
Quoted 6 lines · Oleg Nesterov
Quoted 3 lines · Srivatsa Vaddagiri
We can just thaw the worker thread selectively before
kthread_stoppingthem. This will let us freeze all worker threads (which we want to for hotplug anyway).I am not sure this is a good change for 2.6.21.
So we make that change when merging the freezer-based hotplug patchset?
Quoted 2 lines
I strongly believe it is better to change XFS so that it doesn't use
create_freezeable_workqueue()as Rafael suggested.
Ok no issues. But when we enable freezer-based hotplug, we expect all non-singlethreaded worker threads to be frozen (for hotplug atleast).
Quoted 6 lines
Besides, freezeable workqueues are buggy anyway in 2.6.21-rc,
http://marc.theaimsgroup.com/?l=linux-kernel&m=116855740612755This means that workqueues become non-freezeable after suspend/resume anyway (if I understand
disable_nonboot_cpus()correctly).
Ah ok. When is the above patch expected to be merged?
Signature · 2 lines
Regards, vatsa
Quoted 9 lines · Srivatsa Vaddagiri
Quoted 6 lines · Oleg Nesterov
Quoted 3 lines · Srivatsa Vaddagiri
We can just thaw the worker thread selectively before
kthread_stoppingthem. This will let us freeze all worker threads (which we want to for hotplug anyway).I am not sure this is a good change for 2.6.21.
Quoted 7 lines
So we make that change when merging the freezer-based hotplug patchset?
Quoted 2 lines
I strongly believe it is better to change XFS so that it doesn't use
create_freezeable_workqueue()as Rafael suggested.Ok no issues. But when we enable freezer-based hotplug, we expect all non-singlethreaded worker threads to be frozen (for hotplug atleast).
Yes, we already discussed this :) This is btw another indication we should kill create_freezeable_workqueue() eventually, so it would be nice to change the only one user not to use it.
Quoted 8 lines
Quoted 6 lines
Besides, freezeable workqueues are buggy anyway in 2.6.21-rc,
http://marc.theaimsgroup.com/?l=linux-kernel&m=116855740612755This means that workqueues become non-freezeable after suspend/resume anyway (if I understand
disable_nonboot_cpus()correctly).Ah ok. When is the above patch expected to be merged?
Oh, I don't know. Note that this particular patch makes little sense if we change XFS now (because we have no other users), but we can't drop it, this will break subsequent patches.
Oleg.
Quoted 9 lines · Srivatsa Vaddagiri
Quoted 6 lines · Oleg Nesterov
Quoted 3 lines · Srivatsa Vaddagiri
We can just thaw the worker thread selectively before
kthread_stoppingthem. This will let us freeze all worker threads (which we want to for hotplug anyway).I am not sure this is a good change for 2.6.21.
So we make that change when merging the freezer-based hotplug patchset?
Well, if we want to have freezable workqueues eventually, and I think we do, they should be taken into consideration in designing this patchset, as well as the suspend code ordering (ie. freeze_processes(SUSPEND) before freeze_processes(CPU_HOTPLUG)).
Greetings,
Rafael
Quoted 5 lines · Srivatsa Vaddagiri
Hmm ..good point. So can we assume that disable/enable_nonboot_cpus() are called with processes frozen already?
Gautham, you need to take this into account in your patchset!
Yup. That would mean making the freezer reentrant since we will be freezing twice (once for suspend and later on for hotplug). This is ok since the api in my patches looks like freeze_processes(int freeze_event);
But thaw will be interesting. If we are thawing for hotplug, we gotta only thaw processes which were frozen *only* for hotplug.
Rafael, does that mean more status flags?!
Thanks and Regards
gautham.
Signature · 5 lines
Gautham R Shenoy Linux Technology Center IBM India. "Freedom comes with a price tag of responsibility, which is still a bargain, because Freedom is priceless!"
Quoted 12 lines · Gautham R Shenoy
Quoted 5 lines · Srivatsa Vaddagiri
Hmm ..good point. So can we assume that disable/enable_nonboot_cpus() are called with processes frozen already?
Gautham, you need to take this into account in your patchset!
Yup. That would mean making the freezer reentrant since we will be freezing twice (once for suspend and later on for hotplug). This is ok since the api in my patches looks like
freeze_processes(int freeze_event);
Quoted 4 lines
But thaw will be interesting. If we are thawing for hotplug, we gotta only thaw processes which were frozen *only* for hotplug.
Rafael, does that mean more status flags?!
Well, I don't really think so, but we need to store some information in the freezer (eg. in a status variable). Namely, we can define a variable, say tasks_frozen, the value of which will be the bitwise or of the flags SPE_SUSPEND, SPE_HOTPLUG etc. In a fully functional system, tasks_frozen is equal to zero. If freeze_processes(SPE_SUSPEND) is run, it does tasks_frozen |= SPE_SUSPEND and analogously for SPE_HOTPLUG etc. If tasks_frozen is equal to SPE_SUSPEND|SPE_HOTPLUG, for example, and thaw_tasks(SPE_HOTPLUG) runs, it only thaws the tasks that need not stay frozen for the suspend and does tasks_frozen &= ~SPE_SUSPEND etc.
I think something like this should work.
Greetings,
Rafael
Hi!
Quoted 9 lines
Quoted 3 lines
I'm afraid this is racy. We can't touch *cwq, it may be freed. Suppose that another thread does
destroy_workqueue(), and we thaw that thread beforecwq->thread.Okay, in that case I'd suggest removing
create_freezeable_workqueue()and make all workqueues nonfreezable once again for 2.6.21 (as far as I know, only the two XFS workqueues are affected).Pavel, would that be acceptable?
Not sure... I really dislike XFS running while we are doing swsusp. I'd like to move in direction of freezeable workqueues in the future.
Anyway, if it gets us out of current trouble... yes I guess we can do it.
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Quoted 5 lines · Rafael J. Wysocki
We have a problem with freezable workqueues in 2.6.21-rc1 and in -mm (there are only two of them, in XFS, but still). Namely, their worker threads deadlock with
workqueue_cpu_callback()that gets called during the CPU hotplug, becuaseworkqueue_cpu_callback()tries to stop these threads while they are frozen (disable_nonboot_cpus()happens after we've frozen tasks).
This problem (of kthread_stopping a frozen thread) was there when we implemented freezer-based cpu hotplug. We worked around that in the callbacks by thawing the worker thread first before kthread_stopping it, which is working pretty neatly.
Should that fix the issue?
Signature · 2 lines
Regards, vatsa
Quoted 6 lines · Srivatsa Vaddagiri
This problem (of
kthread_stoppinga frozen thread) was there when we implemented freezer-based cpu hotplug. We worked around that in the callbacks by thawing the worker thread first beforekthread_stoppingit, which is working pretty neatly.Should that fix the issue?
In addition to thawing worker thread before kthread_stopping it, there are minor changes required in worker threads, to check for is_cpu_offline(bind_cpu) when they come out of refrigerator and jump to wait_to_die if so (ex: softirq.c).
I guess you would need these changes before freezer-based hotplug is merged, in which case Gautham can send those patches out first.
Signature · 2 lines
Regards, vatsa
Quoted 8 lines · Srivatsa Vaddagiri
Quoted 6 lines · Srivatsa Vaddagiri
This problem (of
kthread_stoppinga frozen thread) was there when we implemented freezer-based cpu hotplug. We worked around that in the callbacks by thawing the worker thread first beforekthread_stoppingit, which is working pretty neatly.Should that fix the issue?
Quoted 7 lines
In addition to thawing worker thread before
kthread_stoppingit, there are minor changes required in worker threads, to check foris_cpu_offline(bind_cpu)when they come out of refrigerator and jump towait_to_dieif so (ex:softirq.c).I guess you would need these changes before freezer-based hotplug is merged, in which case Gautham can send those patches out first.
Yes, please, if that's possible.
Quoted 9 lines · Rafael J. Wysocki
Quoted 7 lines
In addition to thawing worker thread before
kthread_stoppingit, there are minor changes required in worker threads, to check foris_cpu_offline(bind_cpu)when they come out of refrigerator and jump towait_to_dieif so (ex:softirq.c).I guess you would need these changes before freezer-based hotplug is merged, in which case Gautham can send those patches out first.
Yes, please, if that's possible.
After looking at the current workqueue code, the above minor change I suggested is not required.
So you should be able to fix your "kthread_stop on a frozen worker thread hangs" problem by just a simple patch like this (against 2.6.20-mm2):
workqueue.c renamed from workqueue.c.org +2 -0
@@ -718,6 +718,8 @@ static void cleanup_workqueue_thread(strinsert_wq_barrier (cwq, &barr,1 ); cwq->should_stop =1 ; alive =1 ; +if (frozen (cwq->thread)) +thaw (cwq->thread); }spin_unlock_irq (&cwq->lock);
Can you test with this?
Note that as Oleg commented, freezable workqueues are broken w/o his patch here:
http://marc.theaimsgroup.com/?l=linux-kernel&m=116855740612755
So you need to have Andrew/Linux pick up the above patch first to have correctly functioning freezable workqueues.
Signature · 2 lines
Regards, vatsa
workqueue.c static void cleanup_workqueue_thread(str · line 722
Quoted 8 lines · Srivatsa Vaddagiri
workqueue.c renamed from workqueue.c.org +2
-0@@ -718,6 +718,8 @@ static void cleanup_workqueue_thread(strinsert_wq_barrier (cwq, &barr,1 ); cwq->should_stop =1 ; alive =1 ; +if (frozen (cwq->thread)) +thaw (cwq->thread);
I meant thaw_process(cwq->thread)
Quoted 2 lines
workqueue.c renamed from workqueue.c.org +0
-0}spin_unlock_irq (&cwq->lock);
Signature · 2 lines
Regards, vatsa
Quoted 11 lines · Srivatsa Vaddagiri
workqueue.c static void cleanup_workqueue_thread(str · line 722
Quoted 8 lines · Srivatsa Vaddagiri
workqueue.c renamed from workqueue.c.org +2
-0@@ -718,6 +718,8 @@ static void cleanup_workqueue_thread(strinsert_wq_barrier (cwq, &barr,1 ); cwq->should_stop =1 ; alive =1 ; +if (frozen (cwq->thread)) +thaw (cwq->thread);I meant
thaw_process(cwq->thread)
I thought so, but thanks anyway. ;-)
Quoted 4 lines
Quoted 3 lines
workqueue.c renamed from workqueue.c.org +0
-0}spin_unlock_irq (&cwq->lock);
Quoted 31 lines · Srivatsa Vaddagiri
Quoted 9 lines · Rafael J. Wysocki
Quoted 7 lines
In addition to thawing worker thread before
kthread_stoppingit, there are minor changes required in worker threads, to check foris_cpu_offline(bind_cpu)when they come out of refrigerator and jump towait_to_dieif so (ex:softirq.c).I guess you would need these changes before freezer-based hotplug is merged, in which case Gautham can send those patches out first.
Yes, please, if that's possible.
After looking at the current workqueue code, the above minor change I suggested is not required.
So you should be able to fix your "
kthread_stopon a frozen worker thread hangs" problem by just a simple patch like this (against 2.6.20-mm2):workqueue.c renamed from workqueue.c.org +2
-0@@ -718,6 +718,8 @@ static void cleanup_workqueue_thread(strinsert_wq_barrier (cwq, &barr,1 ); cwq->should_stop =1 ; alive =1 ; +if (frozen (cwq->thread)) +thaw (cwq->thread); }spin_unlock_irq (&cwq->lock);
Quoted 1 line
Can you test with this?
Sure, I will.
Quoted 7 lines
Note that as Oleg commented, freezable workqueues are broken w/o his patch here:
http://marc.theaimsgroup.com/?l=linux-kernel&m=116855740612755So you need to have Andrew/Linux pick up the above patch first to have correctly functioning freezable workqueues.
This one already is in -mm.
Quoted 31 lines · Srivatsa Vaddagiri
Quoted 9 lines · Rafael J. Wysocki
Quoted 7 lines
In addition to thawing worker thread before
kthread_stoppingit, there are minor changes required in worker threads, to check foris_cpu_offline(bind_cpu)when they come out of refrigerator and jump towait_to_dieif so (ex:softirq.c).I guess you would need these changes before freezer-based hotplug is merged, in which case Gautham can send those patches out first.
Yes, please, if that's possible.
After looking at the current workqueue code, the above minor change I suggested is not required.
So you should be able to fix your "
kthread_stopon a frozen worker thread hangs" problem by just a simple patch like this (against 2.6.20-mm2):workqueue.c renamed from workqueue.c.org +2
-0@@ -718,6 +718,8 @@ static void cleanup_workqueue_thread(strinsert_wq_barrier (cwq, &barr,1 ); cwq->should_stop =1 ; alive =1 ; +if (frozen (cwq->thread)) +thaw (cwq->thread); }spin_unlock_irq (&cwq->lock);
Quoted 1 line
Can you test with this?
Unfortunately, the above code is mm-only. Is the analogous fix for 2.6.21-rc2 viable?
Rafael
Quoted 14 lines · Rafael J. Wysocki
workqueue.c static void cleanup_workqueue_thread(str · line 724
Quoted 11 lines
workqueue.c renamed from workqueue.c.org +2
-0@@ -718,6 +718,8 @@ static void cleanup_workqueue_thread(strinsert_wq_barrier (cwq, &barr,1 ); cwq->should_stop =1 ; alive =1 ; +if (frozen (cwq->thread)) +thaw (cwq->thread); }spin_unlock_irq (&cwq->lock);Unfortunately, the above code is mm-only. Is the analogous fix for 2.6.21-rc2 viable?
I am sorry, I lost track of this problem. As for 2.6.21, create_freezeable_workqueue doesn't work and conflict with suspend. Why can't we remove it from XFS as you suggested before?
Iirc,
On 02/28, Nigel Cunningham wrote:
>
> On Wed, 2007-02-28 at 01:08 +0100, Rafael J. Wysocki wrote:
> > On Wednesday, 28 February 2007 01:01, Johannes Berg wrote:
> > > On Wed, 2007-02-28 at 00:57 +0100, Rafael J. Wysocki wrote:
> > >
> > > > Okay, in that case I'd suggest removing create_freezeable_workqueue() and > > > > make all workqueues nonfreezable once again for 2.6.21 (as far as I know, only > > > > the two XFS workqueues are affected).
> > >
> > > I think Nigel might object but I forgot what specific trouble XFS was > > > causing him.
> >
> > We suspected that the XFS' worker threads might commit I/O after
> > freeze_processes() has returned, but that hasn't been supported by evidence, > > as far as I can recall.
> >
> > Also, making them freezable was controversial ...
>
> Controversy is no reason to give in! Nevertheless, I think you're right > - I believe the XFS guys said they fixed the issue that had caused I/O > to be submitted post-freeze. Well, we'll see if it appears again, won't > we?
Oleg.
Quoted 16 lines · Oleg Nesterov
Quoted 14 lines · Rafael J. Wysocki
workqueue.c static void cleanup_workqueue_thread(str · line 724
Quoted 11 lines
workqueue.c renamed from workqueue.c.org +2
-0@@ -718,6 +718,8 @@ static void cleanup_workqueue_thread(strinsert_wq_barrier (cwq, &barr,1 ); cwq->should_stop =1 ; alive =1 ; +if (frozen (cwq->thread)) +thaw (cwq->thread); }spin_unlock_irq (&cwq->lock);Unfortunately, the above code is mm-only. Is the analogous fix for 2.6.21-rc2 viable?
Quoted 3 lines
I am sorry, I lost track of this problem. As for 2.6.21,
create_freezeable_workqueuedoesn't work and conflict with suspend. Why can't we remove it from XFS as you suggested before?
Yes, we can (preparing a patch). I was just curious. :-)
Rafael
Quoted 8 lines · Rafael J. Wysocki
Quoted 4 lines · Oleg Nesterov
I am sorry, I lost track of this problem. As for 2.6.21,
create_freezeable_workqueuedoesn't work and conflict with suspend. Why can't we remove it from XFS as you suggested before?Yes, we can (preparing a patch). I was just curious. :-)
OK, thanks.
We can (I think) do pretty much the same with some additional complications in worker_thread() (check !cpu_online() after try_to_freeze() and break).
Oleg.
Quoted 8 lines · Oleg Nesterov
Quoted 7 lines · Rafael J. Wysocki
Quoted 4 lines · Oleg Nesterov
I am sorry, I lost track of this problem. As for 2.6.21,
create_freezeable_workqueuedoesn't work and conflict with suspend. Why can't we remove it from XFS as you suggested before?
Quoted 6 lines
Quoted 1 line
Yes, we can (preparing a patch). I was just curious. :-)
OK, thanks.
We can (I think) do pretty much the same with some additional complications in
worker_thread()(check !cpu_online() aftertry_to_freeze()and break).
Okay, but I've just finished the patch that removes the freezability of workqueues (appended), so can we please do this in a separate one?
Rafael
---
Since freezable workqueues are broken in 2.6.21-rc
(cf. http://marc.theaimsgroup.com/?l=linux-kernel&m=116855740612755,
http://marc.theaimsgroup.com/?l=linux-kernel&m=117261312523921&w=2)
it's better to remove them altogether for 2.6.21 and change the only user of them (XFS) accordingly.
--- fs/xfs/linux-2.6/xfs_buf.c | 4 ++-- include/linux/workqueue.h | 8 +++----- kernel/workqueue.c | 21 +++++++-------------- 3 files changed, 12 insertions(+), 21 deletions(-)
kernel/workqueue.c +7 -14
@@ -59,7 +59,6 @@ struct cpu_workqueue_struct {int run_depth;/* Detect run_workqueue() recursion depth */ -int freezeable;/* Freeze the thread during suspend */ }____cacheline_aligned ;/*
@@ -352,8 +351,7 @@ static int worker_thread(void *__cwq)struct k_sigaction sa;sigset_t blocked; -if (!cwq->freezeable) - current->flags |=PF_NOFREEZE ; + current->flags |=PF_NOFREEZE ;set_user_nice (current,-5 );
@@ -376,9 +374,6 @@ static int worker_thread(void *__cwq)set_current_state (TASK_INTERRUPTIBLE );while (!kthread_should_stop ()) { -if (cwq->freezeable) -try_to_freeze (); -add_wait_queue (&cwq->more_work, &wait);if (list_empty (&cwq->worklist))schedule ();
@@ -454,8 +449,8 @@ void fastcall flush_workqueue(struct wor }EXPORT_SYMBOL_GPL (flush_workqueue); -static struct task_struct *create_workqueue_thread (struct workqueue_struct *wq, -int cpu,int freezeable) +static struct task_struct +*create_workqueue_thread (struct workqueue_struct *wq,int cpu) {struct cpu_workqueue_struct *cwq =per_cpu_ptr (wq->cpu_wq, cpu);struct task_struct *p;
@@ -465,7 +460,6 @@ static struct task_struct *create_workqu cwq->thread =NULL ; cwq->insert_sequence =0 ; cwq->remove_sequence =0 ; - cwq->freezeable = freezeable;INIT_LIST_HEAD (&cwq->worklist);init_waitqueue_head (&cwq->more_work);init_waitqueue_head (&cwq->work_done);
@@ -480,8 +474,7 @@ static struct task_struct *create_workqureturn p; } -struct workqueue_struct *__create_workqueue (const char *name, -int singlethread,int freezeable) +struct workqueue_struct *__create_workqueue (const char *name,int singlethread) {int cpu, destroy =0 ;struct workqueue_struct *wq;
@@ -501,7 +494,7 @@ struct workqueue_struct *__create_workqumutex_lock (&workqueue_mutex);if (singlethread) {INIT_LIST_HEAD (&wq->list); - p =create_workqueue_thread (wq, singlethread_cpu, freezeable); + p =create_workqueue_thread (wq, singlethread_cpu);if (!p) destroy =1 ;else
@@ -509,7 +502,7 @@ struct workqueue_struct *__create_workqu }else {list_add (&wq->list, &workqueues);for_each_online_cpu (cpu) { - p =create_workqueue_thread (wq, cpu, freezeable); + p =create_workqueue_thread (wq, cpu);if (p) {kthread_bind (p, cpu);wake_up_process (p);
@@ -760,7 +753,7 @@ static int __devinit workqueue_cpu_callbmutex_lock (&workqueue_mutex);/* Create a new workqueue thread for it. */ list_for_each_entry (wq, &workqueues, list) { -if (!create_workqueue_thread (wq, hotcpu,0 )) { +if (!create_workqueue_thread (wq, hotcpu)) {printk ("workqueue for %i failed\n" , hotcpu);return NOTIFY_BAD ; }
include/linux/workqueue.h +3 -5
@@ -159,11 +159,9 @@ struct execute_work {extern struct workqueue_struct *__create_workqueue (const char *name, -int singlethread, -int freezeable); -#define create_workqueue (name) __create_workqueue((name), 0, 0) -#define create_freezeable_workqueue (name) __create_workqueue((name), 0, 1) -#define create_singlethread_workqueue (name) __create_workqueue((name), 1, 0) +int singlethread); +#define create_workqueue (name) __create_workqueue((name), 0) +#define create_singlethread_workqueue (name) __create_workqueue((name), 1)extern void destroy_workqueue (struct workqueue_struct *wq);
fs/xfs/linux-2.6/xfs_buf.c +2 -2
@@ -1829,11 +1829,11 @@ xfs_buf_init(void)if (!xfs_buf_zone) goto out_free_trace_buf; - xfslogd_workqueue =create_freezeable_workqueue ("xfslogd" ); + xfslogd_workqueue =create_workqueue ("xfslogd" );if (!xfslogd_workqueue) goto out_free_buf_zone; - xfsdatad_workqueue =create_freezeable_workqueue ("xfsdatad" ); + xfsdatad_workqueue =create_workqueue ("xfsdatad" );if (!xfsdatad_workqueue) goto out_destroy_xfslogd_workqueue;
Quoted 3 lines · Rafael J. Wysocki
Okay, but I've just finished the patch that removes the freezability of workqueues (appended), so can we please do this in a separate one?
Please, please, no. This patch is of course correct, but it breaks _a lot_ of patches in -mm tree.
May I ask you to send just
Quoted 8 lines
===================================================================
fs/xfs/linux-2.6/xfs_buf.c +0
-0@@ -1829,11 +1829,11 @@ xfs_buf_init(void)if (!xfs_buf_zone) goto out_free_trace_buf;
fs/xfs/linux-2.6/xfs_buf.c xfs_buf_init(void) · line 1839
Quoted 10 lines
fs/xfs/linux-2.6/xfs_buf.c +2
-2- xfslogd_workqueue =create_freezeable_workqueue ("xfslogd" ); + xfslogd_workqueue =create_workqueue ("xfslogd" );if (!xfslogd_workqueue) goto out_free_buf_zone; - xfsdatad_workqueue =create_freezeable_workqueue ("xfsdatad" ); + xfsdatad_workqueue =create_workqueue ("xfsdatad" );if (!xfsdatad_workqueue) goto out_destroy_xfslogd_workqueue;
this bit?
After that, we can do the "removes the freezability of workqueues" patch against -mm tree.
Oleg.
Quoted 31 lines · Oleg Nesterov
Quoted 3 lines · Rafael J. Wysocki
Okay, but I've just finished the patch that removes the freezability of workqueues (appended), so can we please do this in a separate one?
Please, please, no. This patch is of course correct, but it breaks _a
lot_of patches in -mm tree.May I ask you to send just
Quoted 8 lines
===================================================================
fs/xfs/linux-2.6/xfs_buf.c +0
-0@@ -1829,11 +1829,11 @@ xfs_buf_init(void)if (!xfs_buf_zone) goto out_free_trace_buf;fs/xfs/linux-2.6/xfs_buf.c xfs_buf_init(void) · line 1839
Quoted 10 lines
fs/xfs/linux-2.6/xfs_buf.c +2
-2- xfslogd_workqueue =create_freezeable_workqueue ("xfslogd" ); + xfslogd_workqueue =create_workqueue ("xfslogd" );if (!xfslogd_workqueue) goto out_free_buf_zone; - xfsdatad_workqueue =create_freezeable_workqueue ("xfsdatad" ); + xfsdatad_workqueue =create_workqueue ("xfsdatad" );if (!xfsdatad_workqueue) goto out_destroy_xfslogd_workqueue;this bit?
Quoted 2 lines
After that, we can do the "removes the freezability of workqueues" patch against -mm tree.
Okay, if that's better.
Pavel, is that acceptable to you?
Rafael
Quoted 32 lines · Rafael J. Wysocki
Quoted 31 lines · Oleg Nesterov
Quoted 3 lines · Rafael J. Wysocki
Okay, but I've just finished the patch that removes the freezability of workqueues (appended), so can we please do this in a separate one?
Please, please, no. This patch is of course correct, but it breaks _a
lot_of patches in -mm tree.May I ask you to send just
Quoted 8 lines
===================================================================
fs/xfs/linux-2.6/xfs_buf.c +0
-0@@ -1829,11 +1829,11 @@ xfs_buf_init(void)if (!xfs_buf_zone) goto out_free_trace_buf;fs/xfs/linux-2.6/xfs_buf.c xfs_buf_init(void) · line 1839
Quoted 10 lines
fs/xfs/linux-2.6/xfs_buf.c +2
-2- xfslogd_workqueue =create_freezeable_workqueue ("xfslogd" ); + xfslogd_workqueue =create_workqueue ("xfslogd" );if (!xfslogd_workqueue) goto out_free_buf_zone; - xfsdatad_workqueue =create_freezeable_workqueue ("xfsdatad" ); + xfsdatad_workqueue =create_workqueue ("xfsdatad" );if (!xfsdatad_workqueue) goto out_destroy_xfslogd_workqueue;this bit?
Quoted 6 lines
Quoted 2 lines
After that, we can do the "removes the freezability of workqueues" patch against -mm tree.
Okay, if that's better.
Pavel, is that acceptable to you?
No problem, but get that acked-by: from XFS people ;-).
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
PATCH 1/1 Make XFS workqueues nonfreezable
Since freezable workqueues are broken in 2.6.21-rc
(cf. http://marc.theaimsgroup.com/?l=linux-kernel&m=116855740612755,
http://marc.theaimsgroup.com/?l=linux-kernel&m=117261312523921&w=2)
it's better to change the only user of them, which is XFS, to use "normal" nonfreezable workqueues.
--- fs/xfs/linux-2.6/xfs_buf.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
fs/xfs/linux-2.6/xfs_buf.c +2 -2
@@ -1829,11 +1829,11 @@ xfs_buf_init(void)if (!xfs_buf_zone) goto out_free_trace_buf; - xfslogd_workqueue =create_freezeable_workqueue ("xfslogd" ); + xfslogd_workqueue =create_workqueue ("xfslogd" );if (!xfslogd_workqueue) goto out_free_buf_zone; - xfsdatad_workqueue =create_freezeable_workqueue ("xfsdatad" ); + xfsdatad_workqueue =create_workqueue ("xfsdatad" );if (!xfsdatad_workqueue) goto out_destroy_xfslogd_workqueue;
Re: [PATCH] Make XFS workqueues nonfreezable
Quoted 19 lines · On Thu, 1 Mar 2007 00:54:32 +0100 "Rafael J. Wysocki" <[email protected]> wrote:
Since freezable workqueues are broken in 2.6.21-rc
(cf. http://marc.theaimsgroup.com/?l=linux-kernel&m=116855740612755,
http://marc.theaimsgroup.com/?l=linux-kernel&m=117261312523921&w=2)
it's better to change the only user of them, which is XFS, to use "normal" nonfreezable workqueues.Signed-off-by Rafael J. Wysocki <[email protected]>--- fs/xfs/linux-2.6/xfs_buf.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)Index:
linux-2.6.21-rc2/fs/xfs/linux-2.6/xfs_buf.c
===================================================================fs/xfs/linux-2.6/xfs_buf.c +0
-0@@ -1829,11 +1829,11 @@ xfs_buf_init(void)if (!xfs_buf_zone) goto out_free_trace_buf;
fs/xfs/linux-2.6/xfs_buf.c xfs_buf_init(void) · line 1839
Quoted 10 lines
fs/xfs/linux-2.6/xfs_buf.c +2
-2- xfslogd_workqueue =create_freezeable_workqueue ("xfslogd" ); + xfslogd_workqueue =create_workqueue ("xfslogd" );if (!xfslogd_workqueue) goto out_free_buf_zone; - xfsdatad_workqueue =create_freezeable_workqueue ("xfsdatad" ); + xfsdatad_workqueue =create_workqueue ("xfsdatad" );if (!xfsdatad_workqueue) goto out_destroy_xfslogd_workqueue;
Won't this break suspend+XFS?
If so, and given that nobody seems to be reporting this deadlock, perhaps we'd be better off leaving things as-is for the while?
Re: [PATCH] Make XFS workqueues nonfreezable
Hi!
Quoted 32 lines
Quoted 19 lines
Since freezable workqueues are broken in 2.6.21-rc
(cf. http://marc.theaimsgroup.com/?l=linux-kernel&m=116855740612755,
http://marc.theaimsgroup.com/?l=linux-kernel&m=117261312523921&w=2)
it's better to change the only user of them, which is XFS, to use "normal" nonfreezable workqueues.Signed-off-by Rafael J. Wysocki <[email protected]>--- fs/xfs/linux-2.6/xfs_buf.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)Index:
linux-2.6.21-rc2/fs/xfs/linux-2.6/xfs_buf.c
===================================================================fs/xfs/linux-2.6/xfs_buf.c +0
-0@@ -1829,11 +1829,11 @@ xfs_buf_init(void)if (!xfs_buf_zone) goto out_free_trace_buf;fs/xfs/linux-2.6/xfs_buf.c xfs_buf_init(void) · line 1839
Quoted 10 lines
fs/xfs/linux-2.6/xfs_buf.c +2
-2- xfslogd_workqueue =create_freezeable_workqueue ("xfslogd" ); + xfslogd_workqueue =create_workqueue ("xfslogd" );if (!xfslogd_workqueue) goto out_free_buf_zone; - xfsdatad_workqueue =create_freezeable_workqueue ("xfsdatad" ); + xfsdatad_workqueue =create_workqueue ("xfsdatad" );if (!xfsdatad_workqueue) goto out_destroy_xfslogd_workqueue;Won't this break suspend+XFS?
Quoted 2 lines
If so, and given that nobody seems to be reporting this deadlock, perhaps we'd be better off leaving things as-is for the while?
Worst case is not breaking suspend+XFS, worst case is XFS writing to disk after freeze(), leading to subtle fs corruption.
(But noone could reproduce corruption before, and I was told XFS will not do those writes these days).
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Re: [PATCH] Make XFS workqueues nonfreezable
Quoted 13 lines · On Thu, 1 Mar 2007 10:15:21 +0100 Pavel Machek <[email protected]> wrote:
Quoted 13 lines
fs/xfs/linux-2.6/xfs_buf.c xfs_buf_init(void) · line 1838
Quoted 10 lines
fs/xfs/linux-2.6/xfs_buf.c +2
-2- xfslogd_workqueue =create_freezeable_workqueue ("xfslogd" ); + xfslogd_workqueue =create_workqueue ("xfslogd" );if (!xfslogd_workqueue) goto out_free_buf_zone; - xfsdatad_workqueue =create_freezeable_workqueue ("xfsdatad" ); + xfsdatad_workqueue =create_workqueue ("xfsdatad" );if (!xfsdatad_workqueue) goto out_destroy_xfslogd_workqueue;Won't this break suspend+XFS?
Quoted 8 lines
Quoted 2 lines
If so, and given that nobody seems to be reporting this deadlock, perhaps we'd be better off leaving things as-is for the while?
Worst case is not breaking suspend+XFS, worst case is XFS writing to disk after
freeze(), leading to subtle fs corruption.(But noone could reproduce corruption before, and I was told XFS will not do those writes these days).
hm, OK. To avoid making a decision I sent the patch to David ;)
Hi!
Quoted 7 lines
Quoted 4 lines
OK, thanks.
We can (I think) do pretty much the same with some additional complications in
worker_thread()(check !cpu_online() aftertry_to_freeze()and break).Okay, but I've just finished the patch that removes the freezability of workqueues (appended), so can we please do this in a separate one?
Hmm, nothing obviously wrong with the patch (ACK), but xfs people should ack this one, too: 'is it okay to let xfs run while suspending' is not a trivial question.
Quoted 5 lines
Since freezable workqueues are broken in 2.6.21-rc
(cf. http://marc.theaimsgroup.com/?l=linux-kernel&m=116855740612755,
http://marc.theaimsgroup.com/?l=linux-kernel&m=117261312523921&w=2)
it's better to remove them altogether for 2.6.21 and change the only user of them (XFS) accordingly.
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Quoted 3 lines · Rafael J. Wysocki
For 2.6.21-rc1 I've invented the appended workaround (works for me, waiting for Johannes to confirm it works for him too), but I think we need something better for -mm and future kernels.
Finally I could get back to this but after reading the thread I figured it might not be necessary to test this. Please let me know ASAP if you want this patch tested as well or it'll take quite a long time (going skiing for a week on Saturday)
In any case, I made the two xfs workqueues non-freezable and everything on my quad powermac works again, I also couldn't detect any filesystem correction. I wanted to adapt the BUG_ON(block IO not from suspend code) patch from suspend2 but haven't gotten around to it yet.
johannes
Hi,
Quoted 10 lines · Johannes Berg
Quoted 3 lines · Rafael J. Wysocki
For 2.6.21-rc1 I've invented the appended workaround (works for me, waiting for Johannes to confirm it works for him too), but I think we need something better for -mm and future kernels.
Finally I could get back to this but after reading the thread I figured it might not be necessary to test this. Please let me know ASAP if you want this patch tested as well or it'll take quite a long time (going skiing for a week on Saturday)
I think it won't be necessary.
For now, we have decided to make the workqueues nonfreezable (the patch for that has already been merged, AFAICT).
Quoted 3 lines
In any case, I made the two xfs workqueues non-freezable and everything on my quad powermac works again, I also couldn't detect any filesystem correction.
Good, thanks for the confirmation.
Quoted 2 lines
I wanted to adapt the
BUG_ON(block IO not from suspend code)patch from suspend2 but haven't gotten around to it yet.
That might be a good idea for other reasons too, but I'd prefer WARN_ON() instead of BUG_ON() when you're at it. ;-)
Greetings,
Rafael
Hi.
Quoted 23 lines · Rafael J. Wysocki
Hi,
Quoted 10 lines · Johannes Berg
Quoted 3 lines · Rafael J. Wysocki
For 2.6.21-rc1 I've invented the appended workaround (works for me, waiting for Johannes to confirm it works for him too), but I think we need something better for -mm and future kernels.
Finally I could get back to this but after reading the thread I figured it might not be necessary to test this. Please let me know ASAP if you want this patch tested as well or it'll take quite a long time (going skiing for a week on Saturday)
I think it won't be necessary.
For now, we have decided to make the workqueues nonfreezable (the patch for that has already been merged, AFAICT).
Quoted 3 lines
In any case, I made the two xfs workqueues non-freezable and everything on my quad powermac works again, I also couldn't detect any filesystem correction.
Quoted 7 lines
Good, thanks for the confirmation.
Quoted 2 lines
I wanted to adapt the
BUG_ON(block IO not from suspend code)patch from suspend2 but haven't gotten around to it yet.That might be a good idea for other reasons too, but I'd prefer
WARN_ON()instead ofBUG_ON()when you're at it. ;-)
I made it BUG_ON() because if Suspend2 is running any I/O coming from another source besides Suspend2 may be I/O on a page that's been used for the atomic copy, and in that case it would definitely be bad to write it to disk. If swsusp is running, the BUG_ON() won't trigger IIRC.
Regards,
Nigel
Hi,
Quoted 27 lines · Nigel Cunningham
Quoted 26 lines · Rafael J. Wysocki
Quoted 10 lines · Johannes Berg
Quoted 3 lines · Rafael J. Wysocki
For 2.6.21-rc1 I've invented the appended workaround (works for me, waiting for Johannes to confirm it works for him too), but I think we need something better for -mm and future kernels.
Finally I could get back to this but after reading the thread I figured it might not be necessary to test this. Please let me know ASAP if you want this patch tested as well or it'll take quite a long time (going skiing for a week on Saturday)
I think it won't be necessary.
For now, we have decided to make the workqueues nonfreezable (the patch for that has already been merged, AFAICT).
Quoted 3 lines
In any case, I made the two xfs workqueues non-freezable and everything on my quad powermac works again, I also couldn't detect any filesystem correction.
Good, thanks for the confirmation.
Quoted 2 lines
I wanted to adapt the
BUG_ON(block IO not from suspend code)patch from suspend2 but haven't gotten around to it yet.
Quoted 7 lines
Quoted 2 lines
That might be a good idea for other reasons too, but I'd prefer
WARN_ON()instead ofBUG_ON()when you're at it. ;-)I made it
BUG_ON()because if Suspend2 is running any I/O coming from another source besides Suspend2 may be I/O on a page that's been used for the atomic copy, and in that case it would definitely be bad to write it to disk. If swsusp is running, theBUG_ON()won't trigger IIRC.
Okay, but I thought the idea was to use something similar that would trigger for swsusp too (we would have to skip the IO from the userland suspend somehow, BTW), and that I'd prefer to be WARN_ON() rather than BUG_ON().
Greetings,
Rafael
Quoted 2 lines · Rafael J. Wysocki
For now, we have decided to make the workqueues nonfreezable (the patch for that has already been merged, AFAICT).
It isn't in 2.6.21-rc3.
Quoted 5 lines
Quoted 2 lines
I wanted to adapt the
BUG_ON(block IO not from suspend code)patch from suspend2 but haven't gotten around to it yet.That might be a good idea for other reasons too, but I'd prefer
WARN_ON()instead ofBUG_ON()when you're at it. ;-)
I won't be able to do that for quite a while... It's just on my "future projects that would be nice" list :P
johannes