Preemption Broken: centrino_target busted under SMP on 2.6.20.4
BUG: using smp_processor_id() in preemptible [00000001] code: kondemand/0/2473 caller is centrino_target+0xfb/0x600 [<401e3646>] debug_smp_processor_id+0x9e/0xb0 [<40112afb>] centrino_target+0xfb/0x600 [<40112a00>] centrino_target+0x0/0x600 [<40305bd9>] __cpufreq_driver_target+0x5c/0x6b [<f897a537>] do_dbs_timer+0x1bc/0x208 [cpufreq_ondemand] [<40134a46>] run_workqueue+0x85/0x125 [<40374f7f>] _spin_lock_irqsave+0x18/0x66 [<f897a37b>] do_dbs_timer+0x0/0x208 [cpufreq_ondemand] [<401353fb>] worker_thread+0xf9/0x124 [<401213b9>] default_wake_function+0x0/0xc [<40135302>] worker_thread+0x0/0x124 [<40137b37>] kthread+0xb0/0xd9 [<40137a87>] kthread+0x0/0xd9 [<40104b2f>] kernel_thread_helper+0x7/0x10 =======================
Jeff
Quoted 10 lines · Jeff V. Merkey
BUG: using smp_processor_id() in preemptible [00000001] code: kondemand/0/2473 caller is centrino_target+0xfb/0x600 [<401e3646>] debug_smp_processor_id+0x9e/0xb0 [<40112afb>] centrino_target+0xfb/0x600 [<40112a00>] centrino_target+0x0/0x600 [<40305bd9>] __cpufreq_driver_target+0x5c/0x6b [<f897a537>] do_dbs_timer+0x1bc/0x208 [cpufreq_ondemand] [<40134a46>] run_workqueue+0x85/0x125
Quoted 8 lines
[<40374f7f>] _spin_lock_irqsave+0x18/0x66 [<f897a37b>] do_dbs_timer+0x0/0x208 [cpufreq_ondemand] [<401353fb>] worker_thread+0xf9/0x124 [<401213b9>] default_wake_function+0x0/0xc [<40135302>] worker_thread+0x0/0x124 [<40137b37>] kthread+0xb0/0xd9 [<40137a87>] kthread+0x0/0xd9 [<40104b2f>] kernel_thread_helper+0x7/0x10
Given speedstep-centrino is obsoleted in favour of acpi-cpufreq, I'm reluctant to spend too much time turd-polishing. This big-hammer diff should fix this..
Dave
arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c +2 -0
@@ -708,6 +708,7 @@ static int centrino_target (struct cpufreq_policy *policy, saved_mask = current->cpus_allowed; first_cpu =1 ;cpus_clear (covered_cpus); +preempt_disable ();for_each_cpu_mask (j, online_policy_cpus) {/* * Support for SMP systems.
@@ -798,6 +799,7 @@ static int centrino_target (struct cpufreq_policy *policy, } migrate_end: +preempt_enable ();set_cpus_allowed (current, saved_mask);return 0 ; }
Signature · 1 line
http://www.codemonkey.org.uk
Quoted 39 lines · On Thu, 5 Apr 2007 16:50:34 -0400 Dave Jones <[email protected]> wrote:
Quoted 10 lines · Jeff V. Merkey
BUG: using smp_processor_id() in preemptible [00000001] code: kondemand/0/2473 caller is centrino_target+0xfb/0x600 [<401e3646>] debug_smp_processor_id+0x9e/0xb0 [<40112afb>] centrino_target+0xfb/0x600 [<40112a00>] centrino_target+0x0/0x600 [<40305bd9>] __cpufreq_driver_target+0x5c/0x6b [<f897a537>] do_dbs_timer+0x1bc/0x208 [cpufreq_ondemand] [<40134a46>] run_workqueue+0x85/0x125Quoted 8 lines
[<40374f7f>] _spin_lock_irqsave+0x18/0x66 [<f897a37b>] do_dbs_timer+0x0/0x208 [cpufreq_ondemand] [<401353fb>] worker_thread+0xf9/0x124 [<401213b9>] default_wake_function+0x0/0xc [<40135302>] worker_thread+0x0/0x124 [<40137b37>] kthread+0xb0/0xd9 [<40137a87>] kthread+0x0/0xd9 [<40104b2f>] kernel_thread_helper+0x7/0x10Given speedstep-centrino is obsoleted in favour of acpi-cpufreq, I'm reluctant to spend too much time turd-polishing. This big-hammer diff should fix this..
Dave
arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c +1
-0@@ -708,6 +708,7 @@ static int centrino_target (struct cpufreq_policy *policy, saved_mask = current->cpus_allowed; first_cpu =1 ;cpus_clear (covered_cpus); +preempt_disable ();for_each_cpu_mask (j, online_policy_cpus) {/* * Support for SMP systems.
arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c static int centrino_target (struct cpufreq_policy *policy, · line 805
Quoted 7 lines
arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c +1
-0@@ -798,6 +799,7 @@ static int centrino_target (struct cpufreq_policy *policy, } migrate_end: +preempt_enable ();set_cpus_allowed (current, saved_mask);return 0 ; }
This means we'll call set_cpus_allowed() while in atomic state, but set_cpus_allowed() does sleepy stuff.
Quoted 18 lines · Andrew Morton
Quoted 15 lines · On Thu, 5 Apr 2007 16:50:34 -0400 Dave Jones <[email protected]> wrote:
arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c +1
-0@@ -708,6 +708,7 @@ static int centrino_target (struct cpufreq_policy *policy, saved_mask = current->cpus_allowed; first_cpu =1 ;cpus_clear (covered_cpus); +preempt_disable ();for_each_cpu_mask (j, online_policy_cpus) {/* * Support for SMP systems. @@ -798,6 +799,7 @@ static int centrino_target (struct cpufreq_policy *policy, }
Quoted 8 lines
arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c static int centrino_target (struct cpufreq_policy *policy, · line 805
Quoted 5 lines
arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c +1
-0migrate_end: +preempt_enable ();set_cpus_allowed (current, saved_mask);return 0 ; }This means we'll call
set_cpus_allowed()while in atomic state, butset_cpus_allowed()does sleepy stuff.
Puzzled. This diff shouldn't change anything about the context we're in when we call set_cpus_allowed, and as we're not seeing warnings now, I'm not sure what I'm missing?
[which may be 'the obvious', you wouldn't believe the evening I've had
involving gas leaks and noxious fumes. Wheee, floaty head.]
Dave
Signature · 1 line
http://www.codemonkey.org.uk
Quoted 28 lines · On Mon, 9 Apr 2007 22:31:08 -0400 Dave Jones <[email protected]> wrote:
Quoted 18 lines · Andrew Morton
Quoted 15 lines · On Thu, 5 Apr 2007 16:50:34 -0400 Dave Jones <[email protected]> wrote:
arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c +1
-0@@ -708,6 +708,7 @@ static int centrino_target (struct cpufreq_policy *policy, saved_mask = current->cpus_allowed; first_cpu =1 ;cpus_clear (covered_cpus); +preempt_disable ();for_each_cpu_mask (j, online_policy_cpus) {/* * Support for SMP systems. @@ -798,6 +799,7 @@ static int centrino_target (struct cpufreq_policy *policy, }Quoted 8 lines
arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c static int centrino_target (struct cpufreq_policy *policy, · line 805
Quoted 5 lines
arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c +1
-0migrate_end: +preempt_enable ();set_cpus_allowed (current, saved_mask);return 0 ; }This means we'll call
set_cpus_allowed()while in atomic state, butset_cpus_allowed()does sleepy stuff.
Quoted 3 lines
Puzzled. This diff shouldn't change anything about the context we're in when we call
set_cpus_allowed, and as we're not seeing warnings now, I'm not sure what I'm missing?
set_cpus_allowed() will only sleep in special circumstances: when we're telling the target task that it is not allwed to run on a CPU upon which it is presently executing. So it needs to be synchronously migrated off that CPU, which requires that the set_cpus_allowed() caller block.
You're probably just not hitting that case.
Probably we should have a might_sleep() in set_cpus_allowed(), although there might be callers who are guaranteeed to never hit that case and who might legitimately want special treatment to avoid the warning.
Quoted 2 lines
[which may be 'the obvious', you wouldn't believe the evening I've had
involving gas leaks and noxious fumes. Wheee, floaty head.]
Yeah, I get a lot of patches like that.
Quoted 7 lines · Andrew Morton
Quoted 6 lines
Quoted 2 lines
This means we'll call
set_cpus_allowed()while in atomic state, butset_cpus_allowed()does sleepy stuff.Puzzled. This diff shouldn't change anything about the context we're in when we call
set_cpus_allowed, and as we're not seeing warnings now, I'm not sure what I'm missing?
Quoted 6 lines
set_cpus_allowed()will only sleep in special circumstances: when we're telling the target task that it is not allwed to run on a CPU upon which it is presently executing. So it needs to be synchronously migrated off that CPU, which requires that theset_cpus_allowed()caller block.You're probably just not hitting that case.
Oh, now I see it. The set_cpus_allowed that was inside the preempt stuff I was adding. (that the diff elided). Yeah, that's a problem. Bugger.
Quoted 3 lines
Probably we should have a
might_sleep()inset_cpus_allowed(), although there might be callers who are guaranteeed to never hit that case and who might legitimately want special treatment to avoid the warning.
This whole file is going away in .22, and we have a viable alternative in .21 (acpi-cpufreq), so I'm not overly worried about fixing this up given it only shows up in debug kernels, especially at this stage in -rc.
(Yeah, it's a cop-out, but unless someone with more interest in this problem
steps up, I've bigger fishes to fry).
Dave
Signature · 1 line
http://www.codemonkey.org.uk
Quoted 6 lines · Dave Jones
This whole file is going away in .22, and we have a viable alternative in .21 (acpi-cpufreq), so I'm not overly worried about fixing this up given it only shows up in debug kernels, especially at this stage in -rc.
(Yeah, it's a cop-out, but unless someone with more interest in this problem
steps up, I've bigger fishes to fry).
One last try...
(I didn't think too long about this, so this might be equally busted,
but if so, see comment above).
Dave
arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c +3 -0
@@ -720,11 +720,13 @@ static int centrino_target (struct cpufreq_policy *policy,cpu_set (j, set_mask);set_cpus_allowed (current, set_mask); +preempt_disable ();if (unlikely (!cpu_isset (smp_processor_id (), set_mask))) {dprintk ("couldn't limit to CPUs in this domain\n" ); retval = -EAGAIN ;if (first_cpu) {/* We haven't started the transition yet. */ +preempt_enable (); goto migrate_end; }break ;
@@ -765,6 +767,7 @@ static int centrino_target (struct cpufreq_policy *policy,break ;cpu_set (j, covered_cpus); +preempt_enable (); }for_each_cpu_mask (k, online_policy_cpus) {
Signature · 1 line
http://www.codemonkey.org.uk
Quoted 32 lines · On Mon, 9 Apr 2007 23:08:23 -0400 Dave Jones <[email protected]> wrote:
Quoted 6 lines
This whole file is going away in .22, and we have a viable alternative in .21 (acpi-cpufreq), so I'm not overly worried about fixing this up given it only shows up in debug kernels, especially at this stage in -rc.
(Yeah, it's a cop-out, but unless someone with more interest in this problem
steps up, I've bigger fishes to fry).One last try...
(I didn't think too long about this, so this might be equally busted,
but if so, see comment above).Dave
arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c +2
-0@@ -720,11 +720,13 @@ static int centrino_target (struct cpufreq_policy *policy,cpu_set (j, set_mask);set_cpus_allowed (current, set_mask); +preempt_disable ();if (unlikely (!cpu_isset (smp_processor_id (), set_mask))) {dprintk ("couldn't limit to CPUs in this domain\n" ); retval = -EAGAIN ;if (first_cpu) {/* We haven't started the transition yet. */ +preempt_enable (); goto migrate_end; }break ;
arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c static int centrino_target (struct cpufreq_policy *policy, · line 772
Quoted 6 lines
arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c +1
-0@@ -765,6 +767,7 @@ static int centrino_target (struct cpufreq_policy *policy,break ;cpu_set (j, covered_cpus); +preempt_enable (); }
Yes, I expect that should squish the warnings. It looks all racy wrt cpu hotplug and against async set_cpus_allowed(), but if those are our worst problems, we're good.
Quoted 9 lines · Andrew Morton
Quoted 7 lines · On Mon, 9 Apr 2007 23:08:23 -0400 Dave Jones <[email protected]> wrote:
Quoted 6 lines
This whole file is going away in .22, and we have a viable alternative in .21 (acpi-cpufreq), so I'm not overly worried about fixing this up given it only shows up in debug kernels, especially at this stage in -rc.
(Yeah, it's a cop-out, but unless someone with more interest in this problem
steps up, I've bigger fishes to fry).
Quoted 7 lines
Quoted 3 lines
One last try...
(I didn't think too long about this, so this might be equally busted,
but if so, see comment above).Yes, I expect that should squish the warnings. It looks all racy wrt cpu hotplug and against async
set_cpus_allowed(), but if those are our worst problems, we're good.
It probably needs a couple more preempt_enable()'s sprinkled throughout the function to take care of the break's. I also missed a goto case.
Meh, this cure is as bad as the disease.
Dave
Signature · 1 line
http://www.codemonkey.org.uk