Per cpu thread hotplug infrastructure - V3
Subsystems (2)
cover Per cpu thread hotplug infrastructure - V3
The following series implements the infrastructure for parking and unparking kernel threads to avoid the full teardown and fork on cpu hotplug operations along with management infrastructure for hotplug and users.
Changes vs. V2:
Use callbacks for all functionality. Thanks to Rusty for pointing that out. It makes the use sites nice and simple and keeps all the code which would be duplicated otherwise on the core.
Thanks,
tglx
PATCH 1/7 rcu: Yield simpler
The rcu_yield() code is amazing. It's there to avoid starvation of the system when lots of (boosting) work is to be done.
Now looking at the code it's functionality is:
Make the thread SCHED_OTHER and very nice, i.e. get it out of the way Arm a timer with 2 ticks
schedule()
Now if the system goes idle the rcu task returns, regains SCHED_FIFO and plugs on. If the systems stays busy the timer fires and wakes a per node kthread which in turn makes the per cpu thread SCHED_FIFO and brings it back on the cpu. For the boosting thread the "make it FIFO" bit is missing and it just runs some magic boost checks. Now this is a lot of code with extra threads and complexity.
It's way simpler to let the tasks when they detect overload schedule away for 2 ticks and defer the normal wakeup as long as they are in yielded state and the cpu is not idle.
That solves the same problem and the only difference is that when the cpu goes idle it's not guaranteed that the thread returns right away, but it won't be longer out than two ticks, so no harm is done. If that's an issue than it is way simpler just to wake the task from idle as RCU has callbacks there anyway.
Signed-off-by Thomas Gleixner <[email protected]>
--- kernel/rcutree.c | 8 - kernel/rcutree.h | 7 - kernel/rcutree_plugin.h | 210 ++++++++---------------------------------------- 3 files changed, 41 insertions(+), 184 deletions(-)
kernel/rcutree.c +4 -4
@@ -131,7 +131,7 @@ DEFINE_PER_CPU(char, rcu_cpu_has_work);#endif /* #ifdef CONFIG_RCU_BOOST */ -static void rcu_node_kthread_setaffinity (struct rcu_node *rnp,int outgoingcpu); +static void rcu_boost_kthread_setaffinity (struct rcu_node *rnp,int outgoingcpu);static void invoke_rcu_core (void );static void invoke_rcu_callbacks (struct rcu_state *rsp,struct rcu_data *rdp);
@@ -1459,7 +1459,7 @@ static void rcu_cleanup_dead_cpu(int cpu/* Adjust any no-longer-needed kthreads. */ rcu_stop_cpu_kthread (cpu); -rcu_node_kthread_setaffinity (rnp,-1 ); +rcu_boost_kthread_setaffinity (rnp,-1 );/* Remove the dead CPU from the bitmasks in the rcu_node hierarchy. */
@@ -2513,11 +2513,11 @@ static int __cpuinit rcu_cpu_notify(strubreak ;case CPU_ONLINE :case CPU_DOWN_FAILED : -rcu_node_kthread_setaffinity (rnp,-1 ); +rcu_boost_kthread_setaffinity (rnp,-1 );rcu_cpu_kthread_setrt (cpu,1 );break ;case CPU_DOWN_PREPARE : -rcu_node_kthread_setaffinity (rnp, cpu); +rcu_boost_kthread_setaffinity (rnp, cpu);rcu_cpu_kthread_setrt (cpu,0 );break ;case CPU_DYING :
kernel/rcutree.h +1 -6
@@ -464,13 +464,8 @@ static void invoke_rcu_callbacks_kthreadstatic bool rcu_is_callbacks_kthread (void );#ifdef CONFIG_RCU_BOOST static void rcu_preempt_do_callbacks (void ); -static void rcu_boost_kthread_setaffinity (struct rcu_node *rnp, -cpumask_var_t cm);static int __cpuinit rcu_spawn_one_boost_kthread (struct rcu_state *rsp, -struct rcu_node *rnp, -int rnp_index); -static void invoke_rcu_node_kthread (struct rcu_node *rnp); -static void rcu_yield (void (*f)(unsigned long ),unsigned long arg); +struct rcu_node *rnp);#endif /* #ifdef CONFIG_RCU_BOOST */ static void rcu_cpu_kthread_setrt (int cpu,int to_rt);static void __cpuinit rcu_prepare_kthreads (int cpu);
kernel/rcutree_plugin.h +36 -174
331 lines not on this page · raw message
[tip:smp/hotplug] rcu: Yield simpler
Commit-ID: 5d01bbd111d6ff9ea9d9847774f66dff39633776 Gitweb: http://git.kernel.org/tip/5d01bbd111d6ff9ea9d9847774f66dff39633776 Author: Thomas Gleixner <[email protected]> AuthorDate: Mon, 16 Jul 2012 10:42:35 +0000 Committer: Thomas Gleixner <[email protected]> CommitDate: Mon, 13 Aug 2012 17:01:06 +0200
rcu: Yield simpler
The rcu_yield() code is amazing. It's there to avoid starvation of the system when lots of (boosting) work is to be done.
Now looking at the code it's functionality is:
Make the thread SCHED_OTHER and very nice, i.e. get it out of the way Arm a timer with 2 ticks
schedule()
Now if the system goes idle the rcu task returns, regains SCHED_FIFO and plugs on. If the systems stays busy the timer fires and wakes a per node kthread which in turn makes the per cpu thread SCHED_FIFO and brings it back on the cpu. For the boosting thread the "make it FIFO" bit is missing and it just runs some magic boost checks. Now this is a lot of code with extra threads and complexity.
It's way simpler to let the tasks when they detect overload schedule away for 2 ticks and defer the normal wakeup as long as they are in yielded state and the cpu is not idle.
That solves the same problem and the only difference is that when the cpu goes idle it's not guaranteed that the thread returns right away, but it won't be longer out than two ticks, so no harm is done. If that's an issue than it is way simpler just to wake the task from idle as RCU has callbacks there anyway.
Signed-off-by Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Srivatsa S. Bhat <[email protected]>
Cc Rusty Russell <[email protected]>
Cc Namhyung Kim <[email protected]>
Reviewed-by Paul E. McKenney <[email protected]>
Signed-off-by Thomas Gleixner <[email protected]>
--- kernel/rcutree.c | 8 +- kernel/rcutree.h | 7 +-- kernel/rcutree_plugin.h | 210 ++++++++--------------------------------------- 3 files changed, 41 insertions(+), 184 deletions(-)
kernel/rcutree.c +4 -4
@@ -139,7 +139,7 @@ DEFINE_PER_CPU(char, rcu_cpu_has_work);#endif /* #ifdef CONFIG_RCU_BOOST */ -static void rcu_node_kthread_setaffinity (struct rcu_node *rnp,int outgoingcpu); +static void rcu_boost_kthread_setaffinity (struct rcu_node *rnp,int outgoingcpu);static void invoke_rcu_core (void );static void invoke_rcu_callbacks (struct rcu_state *rsp,struct rcu_data *rdp);
@@ -1469,7 +1469,7 @@ static void rcu_cleanup_dead_cpu(int cpu, struct rcu_state *rsp)/* Adjust any no-longer-needed kthreads. */ rcu_stop_cpu_kthread (cpu); -rcu_node_kthread_setaffinity (rnp,-1 ); +rcu_boost_kthread_setaffinity (rnp,-1 );/* Remove the dead CPU from the bitmasks in the rcu_node hierarchy. */
@@ -2594,11 +2594,11 @@ static int __cpuinit rcu_cpu_notify(struct notifier_block *self,break ;case CPU_ONLINE :case CPU_DOWN_FAILED : -rcu_node_kthread_setaffinity (rnp,-1 ); +rcu_boost_kthread_setaffinity (rnp,-1 );rcu_cpu_kthread_setrt (cpu,1 );break ;case CPU_DOWN_PREPARE : -rcu_node_kthread_setaffinity (rnp, cpu); +rcu_boost_kthread_setaffinity (rnp, cpu);rcu_cpu_kthread_setrt (cpu,0 );break ;case CPU_DYING :
kernel/rcutree.h +1 -6
@@ -491,13 +491,8 @@ static void invoke_rcu_callbacks_kthread(void);static bool rcu_is_callbacks_kthread (void );#ifdef CONFIG_RCU_BOOST static void rcu_preempt_do_callbacks (void ); -static void rcu_boost_kthread_setaffinity (struct rcu_node *rnp, -cpumask_var_t cm);static int __cpuinit rcu_spawn_one_boost_kthread (struct rcu_state *rsp, -struct rcu_node *rnp, -int rnp_index); -static void invoke_rcu_node_kthread (struct rcu_node *rnp); -static void rcu_yield (void (*f)(unsigned long ),unsigned long arg); +struct rcu_node *rnp);#endif /* #ifdef CONFIG_RCU_BOOST */ static void rcu_cpu_kthread_setrt (int cpu,int to_rt);static void __cpuinit rcu_prepare_kthreads (int cpu);
kernel/rcutree_plugin.h +36 -174
331 lines not on this page · raw message
PATCH 2/7 kthread: Implement park/unpark facility
In reply to Thomas GleixnerTo avoid the full teardown/setup of per cpu kthreads in the case of cpu hot(un)plug, provide a facility which allows to put the kthread into a park position and unpark it when the cpu comes online again.
Signed-off-by Thomas Gleixner <[email protected]>
Reviewed-by Namhyung Kim <[email protected]>
--- include/linux/kthread.h | 11 ++ kernel/kthread.c | 185 +++++++++++++++++++++++++++++++++++++++++++----- 2 files changed, 176 insertions(+), 20 deletions(-)
include/linux/kthread.h +10 -1
@@ -14,6 +14,11 @@ struct task_struct *kthread_create_on_no kthread_create_on_node(threadfn, data, -1, namefmt, ##arg) +struct task_struct *kthread_create_on_cpu(int (*threadfn)(void *data), + void *data, + unsigned int cpu, + const char *namefmt); +/** * kthread_run - create and wake a thread. * @threadfn: the function to run until signal_pending(current).
@@ -34,9 +39,13 @@ struct task_struct *kthread_create_on_novoid kthread_bind (struct task_struct *k,unsigned int cpu);int kthread_stop (struct task_struct *k); -int kthread_should_stop (void ); +bool kthread_should_stop (void ); +bool kthread_should_park (void );bool kthread_freezable_should_stop (bool *was_frozen);void *kthread_data (struct task_struct *k); +int kthread_park (struct task_struct *k); +void kthread_unpark (struct task_struct *k); +void kthread_parkme (void );int kthreadd (void *unused);extern struct task_struct *kthreadd_task;
kernel/kthread.c +166 -19
258 lines not on this page · raw message
Re: [Patch 2/7] kthread: Implement park/unpark facility
Quoted 6 lines · Thomas Gleixner
To avoid the full teardown/setup of per cpu kthreads in the case of cpu hot(un)plug, provide a facility which allows to put the kthread into a park position and unpark it when the cpu comes online again.
Signed-off-by Thomas Gleixner <[email protected]>Reviewed-by Namhyung Kim <[email protected]>
Reviewed-by Srivatsa S. Bhat <[email protected]>
Regards,
Srivatsa S. Bhat
Quoted 309 lines
--- include/linux/kthread.h | 11 ++ kernel/kthread.c | 185 +++++++++++++++++++++++++++++++++++++++++++----- 2 files changed, 176 insertions(+), 20 deletions(-)Index:
tip/include/linux/kthread.h
===================================================================include/linux/kthread.h +10
-1@@ -14,6 +14,11 @@ struct task_struct *kthread_create_on_no kthread_create_on_node(threadfn, data, -1, namefmt, ##arg) +struct task_struct *kthread_create_on_cpu(int (*threadfn)(void *data), + void *data, + unsigned int cpu, + const char *namefmt); +/** * kthread_run - create and wake a thread. * @threadfn: the function to run until signal_pending(current). @@ -34,9 +39,13 @@ struct task_struct *kthread_create_on_novoid kthread_bind (struct task_struct *k,unsigned int cpu);int kthread_stop (struct task_struct *k); -int kthread_should_stop (void ); +bool kthread_should_stop (void ); +bool kthread_should_park (void );bool kthread_freezable_should_stop (bool *was_frozen);void *kthread_data (struct task_struct *k); +int kthread_park (struct task_struct *k); +void kthread_unpark (struct task_struct *k); +void kthread_parkme (void );int kthreadd (void *unused);extern struct task_struct *kthreadd_task;Index:
tip/kernel/kthread.c
===================================================================kernel/kthread.c +166
-19@@ -37,11 +37,20 @@ struct kthread_create_info };struct kthread { -int should_stop; +unsigned long flags; +unsigned int cpu;void *data; +struct completion parked;struct completion exited; }; +enum KTHREAD_BITS { +KTHREAD_IS_PER_CPU =0 , +KTHREAD_SHOULD_STOP , +KTHREAD_SHOULD_PARK , +KTHREAD_IS_PARKED , +}; +#define to_kthread (tsk) \ container_of((tsk)->vfork_done, struct kthread, exited)@@ -52,13 +61,29 @@ struct kthread {* and this will return true. You should then return, and your return * value will be passed through to kthread_stop(). */ -int kthread_should_stop (void ) +bool kthread_should_stop (void ) { -return to_kthread (current)->should_stop; +return test_bit (KTHREAD_SHOULD_STOP , &to_kthread (current)->flags); }EXPORT_SYMBOL (kthread_should_stop);/** +* kthread_should_park - should this kthread park now? +* +* When someone calls kthread_park() on your kthread, it will be woken +* and this will return true. You should then do the necessary +* cleanup and call kthread_parkme() +* +* Similar to kthread_should_stop(), but this keeps the thread alive +* and in a park position. kthread_unpark() "restarts" the thread and +* calls the thread function again. +*/ +bool kthread_should_park (void ) +{ +return test_bit (KTHREAD_SHOULD_PARK , &to_kthread (current)->flags); +} + +/** * kthread_freezable_should_stop - should this freezable kthread return now? * @was_frozen: optional out parameter, indicates whether %current was frozen * @@ -96,6 +121,24 @@ void *kthread_data(struct task_struct *treturn to_kthread (task)->data; } +static void __kthread_parkme (struct kthread *self) +{ +__set_current_state (TASK_INTERRUPTIBLE ); +while (test_bit (KTHREAD_SHOULD_PARK , &self->flags)) { +if (!test_and_set_bit (KTHREAD_IS_PARKED , &self->flags)) +complete (&self->parked); +schedule (); +__set_current_state (TASK_INTERRUPTIBLE ); + } +clear_bit (KTHREAD_IS_PARKED , &self->flags); +__set_current_state (TASK_RUNNING ); +} + +void kthread_parkme (void ) +{ +__kthread_parkme (to_kthread (current)); +} +static int kthread (void *_create) {/* Copy data: it's on kthread's stack */ @@ -105,9 +148,10 @@ static int kthread(void *_create)struct kthread self;int ret; - self.should_stop =0 ; + self.flags =0 ; self.data = data;init_completion (&self.exited); +init_completion (&self.parked); current->vfork_done = &self.exited;/* OK, tell user we're spawned, wait for stop or wakeup */ @@ -117,9 +161,11 @@ static int kthread(void *_create)schedule (); ret = -EINTR ; -if (!self.should_stop) - ret =threadfn (data); +if (!test_bit (KTHREAD_SHOULD_STOP , &self.flags)) { +__kthread_parkme (&self); + ret =threadfn (data); + }/* we can't just return, we must preserve "self" on stack */ do_exit (ret); }@@ -172,8 +218,7 @@ static void create_kthread(struct kthrea* Returns a task_struct or ERR_PTR(-ENOMEM). */ struct task_struct *kthread_create_on_node (int (*threadfn)(void *data), -void *data, -int node, +void *data,int node,const char namefmt[], ...) {@@ -210,6 +255,13 @@ struct task_struct *kthread_create_on_no }EXPORT_SYMBOL (kthread_create_on_node); +static void __kthread_bind (struct task_struct *p,unsigned int cpu) +{ +/* It's safe because the task is inactive. */ +do_set_cpus_allowed (p,cpumask_of (cpu)); + p->flags |=PF_THREAD_BOUND ; +} +/** * kthread_bind - bind a just-created kthread to a cpu. * @p: thread created by kthread_create(). @@ -226,14 +278,112 @@ void kthread_bind(struct task_struct *p,WARN_ON (1 );return ; } - -/* It's safe because the task is inactive. */ -do_set_cpus_allowed (p,cpumask_of (cpu)); - p->flags |=PF_THREAD_BOUND ; +__kthread_bind (p, cpu); }EXPORT_SYMBOL (kthread_bind);/** +* kthread_create_on_cpu - Create a cpu bound kthread +* @threadfn: the function to run until signal_pending(current). +* @data: data ptr for @threadfn. +* @cpu: The cpu on which the thread should be bound, +* @namefmt: printf-style name for the thread. Format is restricted +* to "name.*%u". Code fills in cpu number. +* +* Description: This helper function creates and names a kernel thread +* The thread will be woken and put into park mode. +*/ +struct task_struct *kthread_create_on_cpu (int (*threadfn)(void *data), +void *data,unsigned int cpu, +const char *namefmt) +{ +struct task_struct *p; + + p =kthread_create_on_node (threadfn, data,cpu_to_node (cpu), namefmt, + cpu); +if (IS_ERR (p)) +return p; +set_bit (KTHREAD_IS_PER_CPU , &to_kthread (p)->flags); +to_kthread (p)->cpu = cpu; +/* Park the thread to get it out of TASK_UNINTERRUPTIBLE state */ +kthread_park (p); +return p; +} + +static struct kthread *task_get_live_kthread (struct task_struct *k) +{ +struct kthread *kthread; + +get_task_struct (k); + kthread =to_kthread (k); +/* It might have exited */ +barrier (); +if (k->vfork_done !=NULL ) +return kthread; +return NULL ; +} + +/** +* kthread_unpark - unpark a thread created by kthread_create(). +* @k: thread created by kthread_create(). +* +* Sets kthread_should_park() for @k to return false, wakes it, and +* waits for it to return. If the thread is marked percpu then its +* bound to the cpu again. +*/ +void kthread_unpark (struct task_struct *k) +{ +struct kthread *kthread =task_get_live_kthread (k); + +if (kthread) { +clear_bit (KTHREAD_SHOULD_PARK , &kthread->flags); +/* +* We clear the IS_PARKED bit here as we don't wait +* until the task has left the park code. So if we'd +* park before that happens we'd see the IS_PARKED bit +* which might be about to be cleared. +*/ +if (test_and_clear_bit (KTHREAD_IS_PARKED , &kthread->flags)) { +if (test_bit (KTHREAD_IS_PER_CPU , &kthread->flags)) +__kthread_bind (k, kthread->cpu); +wake_up_process (k); + } + } +put_task_struct (k); +} + +/** +* kthread_park - park a thread created by kthread_create(). +* @k: thread created by kthread_create(). +* +* Sets kthread_should_park() for @k to return true, wakes it, and +* waits for it to return. This can also be called after kthread_create() +* instead of calling wake_up_process(): the thread will park without +* calling threadfn(). +* +* Returns 0 if the thread is parked, -ENOSYS if the thread exited. +* If called by the kthread itself just the park bit is set. +*/ +int kthread_park (struct task_struct *k) +{ +struct kthread *kthread =task_get_live_kthread (k); +int ret = -ENOSYS ; + +if (kthread) { +if (!test_bit (KTHREAD_IS_PARKED , &kthread->flags)) { +set_bit (KTHREAD_SHOULD_PARK , &kthread->flags); +if (k != current) { +wake_up_process (k); +wait_for_completion (&kthread->parked); + } + } + ret =0 ; + } +put_task_struct (k); +return ret; +} + +/** * kthread_stop - stop a thread created by kthread_create(). * @k: thread created by kthread_create(). * @@ -250,16 +400,13 @@ EXPORT_SYMBOL(kthread_bind);*/ int kthread_stop (struct task_struct *k) { -struct kthread *kthread; +struct kthread *kthread =task_get_live_kthread (k);int ret;trace_sched_kthread_stop (k); -get_task_struct (k); - - kthread =to_kthread (k); -barrier ();/* it might have exited */ -if (k->vfork_done !=NULL ) { - kthread->should_stop =1 ; +if (kthread) { +set_bit (KTHREAD_SHOULD_STOP , &kthread->flags); +clear_bit (KTHREAD_SHOULD_PARK , &kthread->flags);wake_up_process (k);wait_for_completion (&kthread->exited); }
[tip:smp/hotplug] kthread: Implement park/unpark facility
In reply to Thomas GleixnerCommit-ID: 2a1d446019f9a5983ec5a335b95e8593fdb6fa2e Gitweb: http://git.kernel.org/tip/2a1d446019f9a5983ec5a335b95e8593fdb6fa2e Author: Thomas Gleixner <[email protected]> AuthorDate: Mon, 16 Jul 2012 10:42:36 +0000 Committer: Thomas Gleixner <[email protected]> CommitDate: Mon, 13 Aug 2012 17:01:06 +0200
kthread: Implement park/unpark facility
To avoid the full teardown/setup of per cpu kthreads in the case of cpu hot(un)plug, provide a facility which allows to put the kthread into a park position and unpark it when the cpu comes online again.
Signed-off-by Thomas Gleixner <[email protected]>
Reviewed-by Namhyung Kim <[email protected]>
Cc Peter Zijlstra <[email protected]>
Reviewed-by Srivatsa S. Bhat <[email protected]>
Cc Rusty Russell <[email protected]>
Reviewed-by Paul E. McKenney <[email protected]>
Signed-off-by Thomas Gleixner <[email protected]>
--- include/linux/kthread.h | 11 +++- kernel/kthread.c | 185 ++++++++++++++++++++++++++++++++++++++++++----- 2 files changed, 176 insertions(+), 20 deletions(-)
include/linux/kthread.h +10 -1
@@ -14,6 +14,11 @@ struct task_struct *kthread_create_on_node(int (*threadfn)(void *data), kthread_create_on_node(threadfn, data, -1, namefmt, ##arg) +struct task_struct *kthread_create_on_cpu(int (*threadfn)(void *data), + void *data, + unsigned int cpu, + const char *namefmt); +/** * kthread_run - create and wake a thread. * @threadfn: the function to run until signal_pending(current).
@@ -34,9 +39,13 @@ struct task_struct *kthread_create_on_node(int (*threadfn)(void *data),void kthread_bind (struct task_struct *k,unsigned int cpu);int kthread_stop (struct task_struct *k); -int kthread_should_stop (void ); +bool kthread_should_stop (void ); +bool kthread_should_park (void );bool kthread_freezable_should_stop (bool *was_frozen);void *kthread_data (struct task_struct *k); +int kthread_park (struct task_struct *k); +void kthread_unpark (struct task_struct *k); +void kthread_parkme (void );int kthreadd (void *unused);extern struct task_struct *kthreadd_task;
kernel/kthread.c +166 -19
258 lines not on this page · raw message
PATCH 3/7 smpboot: Provide infrastructure for percpu hotplug threads
In reply to Thomas GleixnerProvide a generic interface for setting up and tearing down percpu threads.
On registration the threads for already online cpus are created and started. On deregistration (modules) the threads are stoppped.
During hotplug operations the threads are created, started, parked and unparked. The datastructure for registration provides a pointer to percpu storage space and optional setup, cleanup, park, unpark functions. These functions are called when the thread state changes.
Each implementation has to provide a function which is queried and returns whether the thread should run and the thread function itself.
The core code handles all state transitions and avoids duplicated code in the call sites.
Signed-off-by Thomas Gleixner <[email protected]>
--- include/linux/smpboot.h | 43 +++++++++ kernel/cpu.c | 10 +- kernel/smpboot.c | 229 ++++++++++++++++++++++++++++++++++++++++++++++++ kernel/smpboot.h | 4 4 files changed, 285 insertions(+), 1 deletion(-)
include/linux/smpboot.h new file +43 -0
@@ -0,0 +1,43 @@ +#ifndef _LINUX_SMPBOOT_H +#define _LINUX_SMPBOOT_H + +#include <linux/types.h> + +struct task_struct ; +/* Cookie handed to the thread_fn*/ +struct smpboot_thread_data ; + +/** +* struct smp_hotplug_thread - CPU hotplug related thread descriptor +* @store: Pointer to per cpu storage for the task pointers +* @list: List head for core management +* @thread_should_run: Check whether the thread should run or not. Called with +* preemption disabled. +* @thread_fn: The associated thread function +* @setup: Optional setup function, called when the thread gets +* operational the first time +* @cleanup: Optional cleanup function, called when the thread +* should stop (module exit) +* @park: Optional park function, called when the thread is +* parked (cpu offline) +* @unpark: Optional unpark function, called when the thread is +* unparked (cpu online) +* @thread_comm: The base name of the thread +*/ +struct smp_hotplug_thread { +struct task_struct __percpu **store; +struct list_head list; +int (*thread_should_run)(unsigned int cpu); +void (*thread_fn)(unsigned int cpu); +void (*setup)(unsigned int cpu); +void (*cleanup)(unsigned int cpu,bool online); +void (*park)(unsigned int cpu); +void (*unpark)(unsigned int cpu); +const char *thread_comm; +}; + +int smpboot_register_percpu_thread (struct smp_hotplug_thread *plug_thread); +void smpboot_unregister_percpu_thread (struct smp_hotplug_thread *plug_thread); +int smpboot_thread_schedule (void ); + +#endif
kernel/cpu.c +9 -1
@@ -280,12 +280,13 @@ static int __ref _cpu_down(unsigned int __func__, cpu); goto out_release; } +smpboot_park_threads (cpu); err =__stop_machine (take_cpu_down, &tcd_param,cpumask_of (cpu));if (err) {/* CPU didn't die: tell everyone. Can't complain. */ +smpboot_unpark_threads (cpu);cpu_notify_nofail (CPU_DOWN_FAILED | mod, hcpu); - goto out_release; }BUG_ON (cpu_online (cpu));
@@ -354,6 +355,10 @@ static int __cpuinit _cpu_up(unsigned in goto out; } + ret =smpboot_create_threads (cpu); +if (ret) + goto out; + ret =__cpu_notify (CPU_UP_PREPARE | mod, hcpu,-1 , &nr_calls);if (ret) { nr_calls--;
@@ -368,6 +373,9 @@ static int __cpuinit _cpu_up(unsigned in goto out_notify;BUG_ON (!cpu_online (cpu)); +/* Wake the per cpu threads */ +smpboot_unpark_threads (cpu); +/* Now call notifier in preparation. */ cpu_notify (CPU_ONLINE | mod, hcpu);
kernel/smpboot.c +229 -0
243 lines not on this page · raw message
kernel/smpboot.h +4 -0
@@ -13,4 +13,8 @@ static inline void idle_thread_set_boot_static inline void idle_threads_init (void ) { }#endif +int smpboot_create_threads (unsigned int cpu); +void smpboot_park_threads (unsigned int cpu); +void smpboot_unpark_threads (unsigned int cpu); +#endif
Re: [Patch 3/7] smpboot: Provide infrastructure for percpu hotplug threads
Quoted 11 lines · Thomas Gleixner
Provide a generic interface for setting up and tearing down percpu threads.
On registration the threads for already online cpus are created and started. On deregistration (modules) the threads are stoppped.
During hotplug operations the threads are created, started, parked and unparked. The datastructure for registration provides a pointer to percpu storage space and optional setup, cleanup, park, unpark functions. These functions are called when the thread state changes.
Quoted 7 lines
Each implementation has to provide a function which is queried and returns whether the thread should run and the thread function itself.
The core code handles all state transitions and avoids duplicated code in the call sites.
Signed-off-by Thomas Gleixner <[email protected]>
Elegant design and very beautiful code!
It was such a pleasure to read and review it :-)
Reviewed-by Srivatsa S. Bhat <[email protected]>
[ A minor nit below ]
kernel/smpboot.c void __init idle_threads_init(void) · line 92
Quoted 11 lines
--- include/linux/smpboot.h | 43 +++++++++ kernel/cpu.c | 10 +- kernel/smpboot.c | 229 ++++++++++++++++++++++++++++++++++++++++++++++++ kernel/smpboot.h | 4 4 files changed, 285 insertions(+), 1 deletion(-)kernel/smpboot.c +4
-0+ +/** +* smpboot_thread_fn - percpu hotplug thread loop function +* @void: thread data pointer
s/void/data
kernel/smpboot.c void __init idle_threads_init(void) · line 104
Quoted 12 lines
+ *
+ * Checks for thread stop and park conditions. Calls the necessary
+ * setup, cleanup, park and unpark functions for the registered
+ * thread.
+ *
+ * Returns 1 when the thread should exit, 0 otherwise.
+ */kernel/smpboot.c +5
-0+static int smpboot_thread_fn (void *data) +{ +struct smpboot_thread_data *td = data; +struct smp_hotplug_thread *ht = td->ht; +
Regards,
Srivatsa S. Bhat
Re: [Patch 3/7] smpboot: Provide infrastructure for percpu hotplug threads
Quoted 18 lines · Srivatsa S. Bhat
Quoted 17 lines · Thomas Gleixner
Provide a generic interface for setting up and tearing down percpu threads.
On registration the threads for already online cpus are created and started. On deregistration (modules) the threads are stoppped.
During hotplug operations the threads are created, started, parked and unparked. The datastructure for registration provides a pointer to percpu storage space and optional setup, cleanup, park, unpark functions. These functions are called when the thread state changes.
Each implementation has to provide a function which is queried and returns whether the thread should run and the thread function itself.
The core code handles all state transitions and avoids duplicated code in the call sites.
Quoted 7 lines
Quoted 1 line
Signed-off-by Thomas Gleixner <[email protected]>Elegant design and very beautiful code!
It was such a pleasure to read and review it :-)Reviewed-by Srivatsa S. Bhat <[email protected]>
Of course, the preempt imbalance needs to get resolved, as pointed out by Paul... Your patchset + his fix worked fine, without throwing any scheduling-while-atomic splats.
Regards,
Srivatsa S. Bhat
Quoted 32 lines
[ A minor nit below ]
kernel/smpboot.c void __init idle_threads_init(void) · line 92
Quoted 11 lines
--- include/linux/smpboot.h | 43 +++++++++ kernel/cpu.c | 10 +- kernel/smpboot.c | 229 ++++++++++++++++++++++++++++++++++++++++++++++++ kernel/smpboot.h | 4 4 files changed, 285 insertions(+), 1 deletion(-)kernel/smpboot.c +4
-0+ +/** +* smpboot_thread_fn - percpu hotplug thread loop function +* @void: thread data pointer s/void/data
kernel/smpboot.c void __init idle_threads_init(void) · line 104
Quoted 12 lines
+ *
+ * Checks for thread stop and park conditions. Calls the necessary
+ * setup, cleanup, park and unpark functions for the registered
+ * thread.
+ *
+ * Returns 1 when the thread should exit, 0 otherwise.
+ */kernel/smpboot.c +5
-0+static int smpboot_thread_fn (void *data) +{ +struct smpboot_thread_data *td = data; +struct smp_hotplug_thread *ht = td->ht; +Regards,
Srivatsa S. Bhat
Signature · 3 lines
Regards, Srivatsa S. Bhat IBM Linux Technology Center
Re: [Patch 3/7] smpboot: Provide infrastructure for percpu hotplug threads
In reply to Thomas GleixnerQuoted 11 lines · Thomas Gleixner
Provide a generic interface for setting up and tearing down percpu threads.
On registration the threads for already online cpus are created and started. On deregistration (modules) the threads are stoppped.
During hotplug operations the threads are created, started, parked and unparked. The datastructure for registration provides a pointer to percpu storage space and optional setup, cleanup, park, unpark functions. These functions are called when the thread state changes.
Quoted 7 lines
Each implementation has to provide a function which is queried and returns whether the thread should run and the thread function itself.
The core code handles all state transitions and avoids duplicated code in the call sites.
Signed-off-by Thomas Gleixner <[email protected]>
This approach certainly results in much nicer code!
One issue noted below.
Thanx, Paul
Quoted 48 lines
kernel/smpboot.c +48
-0+static int smpboot_thread_fn (void *data) +{ +struct smpboot_thread_data *td = data; +struct smp_hotplug_thread *ht = td->ht; + +while (1 ) { +set_current_state (TASK_INTERRUPTIBLE ); +preempt_disable (); +if (kthread_should_stop ()) { +set_current_state (TASK_RUNNING ); +preempt_enable (); +if (ht->cleanup) + ht->cleanup (td->cpu,cpu_online (td->cpu)); +kfree (td); +return 0 ; + } + +if (kthread_should_park ()) { +__set_current_state (TASK_RUNNING ); +preempt_enable (); +if (ht->park && td->status ==HP_THREAD_ACTIVE ) { +BUG_ON (td->cpu !=smp_processor_id ()); + ht->park (td->cpu); + td->status =HP_THREAD_PARKED ; + } +kthread_parkme (); +/* We might have been woken for stop */ +continue ; + } + +BUG_ON (td->cpu !=smp_processor_id ()); + +/* Check for state change setup */ +switch (td->status) { +case HP_THREAD_NONE : +preempt_enable (); +if (ht->setup) + ht->setup (td->cpu); + td->status =HP_THREAD_ACTIVE ; +preempt_disable (); +break ; +case HP_THREAD_PARKED : +preempt_enable (); +if (ht->unpark) + ht->unpark (td->cpu); + td->status =HP_THREAD_ACTIVE ; +preempt_disable (); +break ;
kernel/smpboot.c void __init idle_threads_init(void) · line 157
Quoted 10 lines
kernel/smpboot.c +10
-0+ } + +if (!ht->thread_should_run (td->cpu)) { +schedule_preempt_disabled (); + }else { +set_current_state (TASK_RUNNING ); +preempt_enable (); + ht->thread_fn (td->cpu); +preempt_disable (); + }
At this point, preemption is disabled, but the code at the beginning of the loop will disable it again. This results in "scheduling while atomic" splats. I did the following to clear them up:
+ if (!ht->thread_should_run(td->cpu)) {
+ preempt_enable();
+ schedule();
+ } else {
+ set_current_state(TASK_RUNNING);
+ preempt_enable();
+ ht->thread_fn(td->cpu);
+ }
I also placed the updated series on -rcu at branch rcu/smp/hotplug
(git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git),
based on tip/smp/hotplug, for Linaro testing purposes.
Thanx, Paul
[tip:smp/hotplug] smpboot: Provide infrastructure for percpu hotplug threads
In reply to Thomas GleixnerCommit-ID: f97f8f06a49febbc3cb3635172efbe64ddc79700 Gitweb: http://git.kernel.org/tip/f97f8f06a49febbc3cb3635172efbe64ddc79700 Author: Thomas Gleixner <[email protected]> AuthorDate: Mon, 16 Jul 2012 10:42:36 +0000 Committer: Thomas Gleixner <[email protected]> CommitDate: Mon, 13 Aug 2012 17:01:07 +0200
smpboot: Provide infrastructure for percpu hotplug threads
Provide a generic interface for setting up and tearing down percpu threads.
On registration the threads for already online cpus are created and started. On deregistration (modules) the threads are stoppped.
During hotplug operations the threads are created, started, parked and unparked. The datastructure for registration provides a pointer to percpu storage space and optional setup, cleanup, park, unpark functions. These functions are called when the thread state changes.
Each implementation has to provide a function which is queried and returns whether the thread should run and the thread function itself.
The core code handles all state transitions and avoids duplicated code in the call sites.
[ paulmck: Preemption leak fix ]
Signed-off-by Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Reviewed-by Srivatsa S. Bhat <[email protected]>
Cc Rusty Russell <[email protected]>
Reviewed-by Paul E. McKenney <[email protected]>
Cc Namhyung Kim <[email protected]>
Signed-off-by Thomas Gleixner <[email protected]>
--- include/linux/smpboot.h | 43 +++++++++ kernel/cpu.c | 10 ++- kernel/smpboot.c | 229 +++++++++++++++++++++++++++++++++++++++++++++++ kernel/smpboot.h | 4 + 4 files changed, 285 insertions(+), 1 deletions(-)
include/linux/smpboot.h new file +43 -0
@@ -0,0 +1,43 @@ +#ifndef _LINUX_SMPBOOT_H +#define _LINUX_SMPBOOT_H + +#include <linux/types.h> + +struct task_struct ; +/* Cookie handed to the thread_fn*/ +struct smpboot_thread_data ; + +/** +* struct smp_hotplug_thread - CPU hotplug related thread descriptor +* @store: Pointer to per cpu storage for the task pointers +* @list: List head for core management +* @thread_should_run: Check whether the thread should run or not. Called with +* preemption disabled. +* @thread_fn: The associated thread function +* @setup: Optional setup function, called when the thread gets +* operational the first time +* @cleanup: Optional cleanup function, called when the thread +* should stop (module exit) +* @park: Optional park function, called when the thread is +* parked (cpu offline) +* @unpark: Optional unpark function, called when the thread is +* unparked (cpu online) +* @thread_comm: The base name of the thread +*/ +struct smp_hotplug_thread { +struct task_struct __percpu **store; +struct list_head list; +int (*thread_should_run)(unsigned int cpu); +void (*thread_fn)(unsigned int cpu); +void (*setup)(unsigned int cpu); +void (*cleanup)(unsigned int cpu,bool online); +void (*park)(unsigned int cpu); +void (*unpark)(unsigned int cpu); +const char *thread_comm; +}; + +int smpboot_register_percpu_thread (struct smp_hotplug_thread *plug_thread); +void smpboot_unregister_percpu_thread (struct smp_hotplug_thread *plug_thread); +int smpboot_thread_schedule (void ); + +#endif
kernel/cpu.c +9 -1
@@ -280,12 +280,13 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen) __func__, cpu); goto out_release; } +smpboot_park_threads (cpu); err =__stop_machine (take_cpu_down, &tcd_param,cpumask_of (cpu));if (err) {/* CPU didn't die: tell everyone. Can't complain. */ +smpboot_unpark_threads (cpu);cpu_notify_nofail (CPU_DOWN_FAILED | mod, hcpu); - goto out_release; }BUG_ON (cpu_online (cpu));
@@ -354,6 +355,10 @@ static int __cpuinit _cpu_up(unsigned int cpu, int tasks_frozen) goto out; } + ret =smpboot_create_threads (cpu); +if (ret) + goto out; + ret =__cpu_notify (CPU_UP_PREPARE | mod, hcpu,-1 , &nr_calls);if (ret) { nr_calls--;
@@ -368,6 +373,9 @@ static int __cpuinit _cpu_up(unsigned int cpu, int tasks_frozen) goto out_notify;BUG_ON (!cpu_online (cpu)); +/* Wake the per cpu threads */ +smpboot_unpark_threads (cpu); +/* Now call notifier in preparation. */ cpu_notify (CPU_ONLINE | mod, hcpu);
kernel/smpboot.c +229 -0
243 lines not on this page · raw message
kernel/smpboot.h +4 -0
@@ -13,4 +13,8 @@ static inline void idle_thread_set_boot_cpu(void) { }static inline void idle_threads_init (void ) { }#endif +int smpboot_create_threads (unsigned int cpu); +void smpboot_park_threads (unsigned int cpu); +void smpboot_unpark_threads (unsigned int cpu); +#endif
Re: [Patch 3/7] smpboot: Provide infrastructure for percpu hotplug threads
In reply to Thomas GleixnerHi Thomas,
Quoted 11 lines · Thomas Gleixner
Provide a generic interface for setting up and tearing down percpu threads.
On registration the threads for already online cpus are created and started. On deregistration (modules) the threads are stoppped.
During hotplug operations the threads are created, started, parked and unparked. The datastructure for registration provides a pointer to percpu storage space and optional setup, cleanup, park, unpark functions. These functions are called when the thread state changes.
Quoted 8 lines
Each implementation has to provide a function which is queried and returns whether the thread should run and the thread function itself.
The core code handles all state transitions and avoids duplicated code in the call sites.
Signed-off-by Thomas Gleixner <[email protected]>---
This patch seems to cause the following BUG() on KVM guests with large amount of VCPUs:
[ 0.511760] ------------[ cut here ]------------ [ 0.511761] kernel BUG at kernel/smpboot.c:134! [ 0.511764] invalid opcode: 0000 [#3] PREEMPT SMP DEBUG_PAGEALLOC [ 0.511779] CPU 0 [ 0.511780] Pid: 70, comm: watchdog/10 Tainted: G D W 3.6.0-rc6-next-20120919-sasha-00001-gb54aafe #365 [ 0.511783] RIP: 0010:[<ffffffff81141676>] [<ffffffff81141676>] smpboot_thread_fn+0x196/0x2e0 [ 0.511785] RSP: 0018:ffff88000cf4bdd0 EFLAGS: 00010206 [ 0.511786] RAX: 0000000000000000 RBX: ffff88000cf58000 RCX: 0000000000000000 [ 0.511787] RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000001 [ 0.511788] RBP: ffff88000cf4be30 R08: 0000000000000000 R09: 0000000000000001 [ 0.511789] R10: 0000000000000000 R11: 0000000000000000 R12: ffff88000cdb9ff0 [ 0.511790] R13: ffffffff84c60920 R14: 000000000000000a R15: ffff88000cf58000 [ 0.511792] FS: 0000000000000000(0000) GS:ffff88000d200000(0000) knlGS:0000000000000000 [ 0.511794] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 0.511795] CR2: 00000000ffffffff CR3: 0000000004c26000 CR4: 00000000000406f0 [ 0.511801] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 0.511805] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 0.511807] Process watchdog/10 (pid: 70, threadinfo ffff88000cf4a000, task ffff88000cf58000) [ 0.511808] Stack: [ 0.511822] ffff88000cf4bfd8 ffff88000cf4bfd8 0000000000000000 0000000000000000 [ 0.511833] ffff88000cf4be00 ffffffff839eace5 ffff88000cf4be30 ffff88000cdd1c68 [ 0.511844] ffff88000cdb9ff0 ffffffff811414e0 0000000000000000 0000000000000000 [ 0.511845] Call Trace: [ 0.511852] [<ffffffff839eace5>] ? schedule+0x55/0x60 [ 0.511857] [<ffffffff811414e0>] ? __smpboot_create_thread+0xf0/0xf0 [ 0.511863] [<ffffffff81135c13>] kthread+0xe3/0xf0 [ 0.511867] [<ffffffff839eb463>] ? wait_for_common+0x143/0x180 [ 0.511873] [<ffffffff839ef044>] kernel_thread_helper+0x4/0x10 [ 0.511878] [<ffffffff839ed3b4>] ? retint_restore_args+0x13/0x13 [ 0.511883] [<ffffffff81135b30>] ? insert_kthread_work+0x90/0x90 [ 0.511888] [<ffffffff839ef040>] ? gs_change+0x13/0x13 [ 0.511916] Code: 24 04 02 00 00 00 0f 1f 80 00 00 00 00 e8 b3 46 ff ff e9 b6 fe ff ff 66 0f 1f 44 00 00 45 8b 34 24 e8 ff 72 8a 00 41 39 c6 74 0a <0f> 0b 0f 1f 84 00 00 00 00 00 41 8b 44 24 04 85 c0 74 0f 83 f8 [ 0.511919] RIP [<ffffffff81141676>] smpboot_thread_fn+0x196/0x2e0 [ 0.511920] RSP <ffff88000cf4bdd0> [ 0.511922] ---[ end trace 127920ef70923ae1 ]---
I'm starting the guest with numa=fake=10, so vcpu 0 ends up on the same (fake) node as vcpu 10, and while digging into the bug, it seems that the issue is that vcpu10's thread gets scheduled on vcpu0.
Beyond that I don't really understand what's wrong...
Thanks,
Sasha
Re: [Patch 3/7] smpboot: Provide infrastructure for percpu hotplug threads
Quoted 68 lines · Sasha Levin
Hi Thomas,
Quoted 11 lines · Thomas Gleixner
Provide a generic interface for setting up and tearing down percpu threads.
On registration the threads for already online cpus are created and started. On deregistration (modules) the threads are stoppped.
During hotplug operations the threads are created, started, parked and unparked. The datastructure for registration provides a pointer to percpu storage space and optional setup, cleanup, park, unpark functions. These functions are called when the thread state changes.
Quoted 8 lines
Each implementation has to provide a function which is queried and returns whether the thread should run and the thread function itself.
The core code handles all state transitions and avoids duplicated code in the call sites.
Signed-off-by Thomas Gleixner <[email protected]>---
This patch seems to cause the following
BUG()on KVM guests with large amount of VCPUs:[ 0.511760] ------------[ cut here ]------------ [ 0.511761] kernel BUG at kernel/smpboot.c:134! [ 0.511764] invalid opcode: 0000 [#3] PREEMPT SMP DEBUG_PAGEALLOC [ 0.511779] CPU 0 [ 0.511780] Pid: 70, comm: watchdog/10 Tainted: G D W 3.6.0-rc6-next-20120919-sasha-00001-gb54aafe #365 [ 0.511783] RIP: 0010:[<ffffffff81141676>] [<ffffffff81141676>] smpboot_thread_fn+0x196/0x2e0 [ 0.511785] RSP: 0018:ffff88000cf4bdd0 EFLAGS: 00010206 [ 0.511786] RAX: 0000000000000000 RBX: ffff88000cf58000 RCX: 0000000000000000 [ 0.511787] RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000001 [ 0.511788] RBP: ffff88000cf4be30 R08: 0000000000000000 R09: 0000000000000001 [ 0.511789] R10: 0000000000000000 R11: 0000000000000000 R12: ffff88000cdb9ff0 [ 0.511790] R13: ffffffff84c60920 R14: 000000000000000a R15: ffff88000cf58000 [ 0.511792] FS: 0000000000000000(0000) GS:ffff88000d200000(0000) knlGS:0000000000000000 [ 0.511794] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 0.511795] CR2: 00000000ffffffff CR3: 0000000004c26000 CR4: 00000000000406f0 [ 0.511801] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 0.511805] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 0.511807] Process watchdog/10 (pid: 70, threadinfo ffff88000cf4a000, task ffff88000cf58000) [ 0.511808] Stack: [ 0.511822] ffff88000cf4bfd8 ffff88000cf4bfd8 0000000000000000 0000000000000000 [ 0.511833] ffff88000cf4be00 ffffffff839eace5 ffff88000cf4be30 ffff88000cdd1c68 [ 0.511844] ffff88000cdb9ff0 ffffffff811414e0 0000000000000000 0000000000000000 [ 0.511845] Call Trace: [ 0.511852] [<ffffffff839eace5>] ? schedule+0x55/0x60 [ 0.511857] [<ffffffff811414e0>] ? __smpboot_create_thread+0xf0/0xf0 [ 0.511863] [<ffffffff81135c13>] kthread+0xe3/0xf0 [ 0.511867] [<ffffffff839eb463>] ? wait_for_common+0x143/0x180 [ 0.511873] [<ffffffff839ef044>] kernel_thread_helper+0x4/0x10 [ 0.511878] [<ffffffff839ed3b4>] ? retint_restore_args+0x13/0x13 [ 0.511883] [<ffffffff81135b30>] ? insert_kthread_work+0x90/0x90 [ 0.511888] [<ffffffff839ef040>] ? gs_change+0x13/0x13 [ 0.511916] Code: 24 04 02 00 00 00 0f 1f 80 00 00 00 00 e8 b3 46 ff ff e9 b6 fe ff ff 66 0f 1f 44 00 00 45 8b 34 24 e8 ff 72 8a 00 41 39 c6 74 0a <0f> 0b 0f 1f 84 00 00 00 00 00 41 8b 44 24 04 85 c0 74 0f 83 f8 [ 0.511919] RIP [<ffffffff81141676>] smpboot_thread_fn+0x196/0x2e0 [ 0.511920] RSP <ffff88000cf4bdd0> [ 0.511922] ---[ end trace 127920ef70923ae1 ]---
Quoted 5 lines
I'm starting the guest with numa=fake=10, so vcpu 0 ends up on the same (fake) node as vcpu 10, and while digging into the bug, it seems that the issue is that vcpu10's thread gets scheduled on vcpu0.
Beyond that I don't really understand what's wrong...
Ping? Still seeing that with linux-next.
PATCH 4/7 softirq: Use hotplug thread infrastructure
In reply to Thomas GleixnerSigned-off-by Thomas Gleixner <[email protected]>
--- kernel/softirq.c | 107 +++++++++++++------------------------------------------ 1 file changed, 26 insertions(+), 81 deletions(-)
kernel/softirq.c +26 -81
@@ -23,6 +23,7 @@#include <linux/rcupdate.h> #include <linux/ftrace.h> #include <linux/smp.h> +#include <linux/smpboot.h> #include <linux/tick.h> #define CREATE_TRACE_POINTS
@@ -733,49 +734,21 @@ void __init softirq_init(void)open_softirq (HI_SOFTIRQ , tasklet_hi_action); } -static int run_ksoftirqd (void * __bind_cpu) +static int ksoftirqd_should_run (unsigned int cpu) { -set_current_state (TASK_INTERRUPTIBLE ); - -while (!kthread_should_stop ()) { -preempt_disable (); -if (!local_softirq_pending ()) { -schedule_preempt_disabled (); - } - -__set_current_state (TASK_RUNNING ); - -while (local_softirq_pending ()) { -/* Preempt disable stops cpu going offline. -If already offline, we'll be on wrong CPU: -don't process */ -if (cpu_is_offline ((long )__bind_cpu)) - goto wait_to_die; -local_irq_disable (); -if (local_softirq_pending ()) -__do_softirq (); -local_irq_enable (); -sched_preempt_enable_no_resched (); -cond_resched (); -preempt_disable (); -rcu_note_context_switch ((long )__bind_cpu); - } -preempt_enable (); -set_current_state (TASK_INTERRUPTIBLE ); - } -__set_current_state (TASK_RUNNING ); -return 0 ; +return local_softirq_pending (); +} -wait_to_die: -preempt_enable (); -/* Wait for kthread_stop */ -set_current_state (TASK_INTERRUPTIBLE ); -while (!kthread_should_stop ()) { -schedule (); -set_current_state (TASK_INTERRUPTIBLE ); +static void run_ksoftirqd (unsigned int cpu) +{ +local_irq_disable (); +if (local_softirq_pending ()) { +__do_softirq (); +local_irq_enable (); +cond_resched (); +rcu_note_context_switch (cpu); } -__set_current_state (TASK_RUNNING ); -return 0 ; +local_irq_enable (); }#ifdef CONFIG_HOTPLUG_CPU
@@ -841,50 +814,17 @@ static int __cpuinit cpu_callback(struct unsigned long action, void *hcpu) { - int hotcpu = (unsigned long)hcpu; -struct task_struct *p; -switch (action) { -case CPU_UP_PREPARE : -case CPU_UP_PREPARE_FROZEN : - p =kthread_create_on_node (run_ksoftirqd, - hcpu, -cpu_to_node (hotcpu), -"ksoftirqd/%d" , hotcpu); -if (IS_ERR (p)) { -printk ("ksoftirqd for %i failed\n" , hotcpu); -return notifier_from_errno (PTR_ERR (p)); - } -kthread_bind (p, hotcpu); -per_cpu (ksoftirqd, hotcpu) = p; -break ; -case CPU_ONLINE : -case CPU_ONLINE_FROZEN : -wake_up_process (per_cpu (ksoftirqd, hotcpu)); -break ;#ifdef CONFIG_HOTPLUG_CPU -case CPU_UP_CANCELED : -case CPU_UP_CANCELED_FROZEN : -if (!per_cpu (ksoftirqd, hotcpu)) -break ; -/* Unbind so it can run. Fall thru. */ -kthread_bind (per_cpu (ksoftirqd, hotcpu), -cpumask_any (cpu_online_mask));case CPU_DEAD :case CPU_DEAD_FROZEN : { -static const struct sched_param param = { - .sched_priority =MAX_RT_PRIO -1 - }; - - p =per_cpu (ksoftirqd, hotcpu); -per_cpu (ksoftirqd, hotcpu) =NULL ; -sched_setscheduler_nocheck (p,SCHED_FIFO , ¶m); -kthread_stop (p); +int hotcpu = (unsigned long )hcpu; +takeover_tasklets (hotcpu);break ; }#endif /* CONFIG_HOTPLUG_CPU */ - } + }return NOTIFY_OK ; }
@@ -892,14 +832,19 @@ static struct notifier_block __cpuinitda .notifier_call = cpu_callback }; +static struct smp_hotplug_thread softirq_threads = { + .store = &ksoftirqd, + .thread_should_run = ksoftirqd_should_run, + .thread_fn = run_ksoftirqd, + .thread_comm ="ksoftirqd/%u" , +}; +static __init int spawn_ksoftirqd (void ) { -void *cpu = (void *)(long )smp_processor_id (); -int err =cpu_callback (&cpu_nfb,CPU_UP_PREPARE , cpu); - -BUG_ON (err !=NOTIFY_OK ); -cpu_callback (&cpu_nfb,CPU_ONLINE , cpu);register_cpu_notifier (&cpu_nfb); + +BUG_ON (smpboot_register_percpu_thread (&softirq_threads)); +return 0 ; }early_initcall (spawn_ksoftirqd);
Re: [Patch 4/7] softirq: Use hotplug thread infrastructure
Quoted 4 lines · Thomas Gleixner
Signed-off-by Thomas Gleixner <[email protected]>--- kernel/softirq.c | 107 +++++++++++++------------------------------------------ 1 file changed, 26 insertions(+), 81 deletions(-)
This is also much more compact! One issue with fix below.
Thanx, Paul
[ . . . ]
kernel/softirq.c void __init softirq_init(void) · line 749
Quoted 8 lines
kernel/softirq.c +8
-0+static void run_ksoftirqd (unsigned int cpu) +{ +local_irq_disable (); +if (local_softirq_pending ()) { +__do_softirq (); +local_irq_enable (); +cond_resched (); +rcu_note_context_switch (cpu);
The rcu_note_context_switch() function must run with preemption disabled. I fixed this as follows:
+static void run_ksoftirqd(unsigned int cpu)
+{
+ local_irq_disable();
+ if (local_softirq_pending()) {
+ __do_softirq();
+ rcu_note_context_switch(cpu);
+ local_irq_enable();
+ cond_resched();
+ return;
}
Again, I also placed the updated series on -rcu at branch rcu/smp/hotplug
(git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git),
based on tip/smp/hotplug, for Linaro testing purposes.
With these two changes, this series merged with Tejun's workqueue hotplug changes merged with my -rcu tree passed moderate rcutorture and hotplug testing, with the exception of some apparently unrelated build problems in UP configuration:
warning: call to ‘copy_from_user_overflow’ declared with attribute warning: copy_from_user() buffer size is not provably correct [enabled by default
I have a fix for an earlier occurrence of this problem that I will try next.
Thanx, Paul
Re: [Patch 4/7] softirq: Use hotplug thread infrastructure
Quoted 5 lines · Paul E. McKenney
Quoted 4 lines · Thomas Gleixner
Signed-off-by Thomas Gleixner <[email protected]>--- kernel/softirq.c | 107 +++++++++++++------------------------------------------ 1 file changed, 26 insertions(+), 81 deletions(-)
[ . . . ]
Quoted 4 lines
Again, I also placed the updated series on -rcu at branch rcu/smp/hotplug
(git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git),
based on tip/smp/hotplug, for Linaro testing purposes.
Quoted 8 lines
With these two changes, this series merged with Tejun's workqueue hotplug changes merged with my -rcu tree passed moderate rcutorture and hotplug testing, with the exception of some apparently unrelated build problems in UP configuration:
warning: call to ‘copy_from_user_overflow’ declared with attribute warning:
copy_from_user()buffer size is not provably correct [enabled by defaultI have a fix for an earlier occurrence of this problem that I will try next.
And the fix worked, but uncovered a !SMP build problem in the underlying patchset. I fixed this with the following patch, which is also available as noted above.
Thanx, Paul
------------------------------------------------------------------------
Makefile | 3 +-- smpboot.c | 4 ++++ 2 files changed, 5 insertions(+), 2 deletions(-)
hotplug: Fix UP bug in smpboot hotplug code
Because kernel subsystems need their per-CPU kthreads on UP systems as well as on SMP systems, the smpboot hotplug kthread functions must be provided in UP builds as well as in SMP builds. This commit therefore adds smpboot.c to UP builds and excludes irrelevant code via #ifdef.
Signed-off-by Paul E. McKenney <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
kernel/Makefile +1 -2
@@ -10,7 +10,7 @@ obj-y = fork.o exec_domain.o panic.o printk.o \ kthread.o wait.o kfifo.o sys_ni.o posix-cpu-timers.o mutex.o \ hrtimer.o rwsem.o nsproxy.o srcu.o semaphore.o \ notifier.o ksysfs.o cred.o \ - async.o range.o groups.o lglock.o + async.o range.o groups.o lglock.o smpboot.o ifdef CONFIG_FUNCTION_TRACER # Do not trace debug files and internal ftrace files
@@ -46,7 +46,6 @@ obj-$(CONFIG_DEBUG_RT_MUTEXES) += rtmutex-debug.o obj-$(CONFIG_RT_MUTEX_TESTER) += rtmutex-tester.o obj-$(CONFIG_GENERIC_ISA_DMA) += dma.o obj-$(CONFIG_SMP) += smp.o -obj-$(CONFIG_SMP) += smpboot.o ifneq ($(CONFIG_SMP),y) obj-y += up.o endif
kernel/smpboot.c +4 -0
@@ -15,6 +15,8 @@#include "smpboot.h" +#ifdef CONFIG_SMP +#ifdef CONFIG_GENERIC_SMP_IDLE_THREAD /* * For the hotplug case we keep the task structs around and reuse
@@ -72,6 +74,8 @@ void __init idle_threads_init(void) }#endif +#endif /* #ifdef CONFIG_SMP */ +static LIST_HEAD (hotplug_threads);static DEFINE_MUTEX (smpboot_threads_lock);
Re: [Patch 4/7] softirq: Use hotplug thread infrastructure
In reply to Thomas Gleixner2012/7/16 Thomas Gleixner <[email protected]>:
kernel/softirq.c static int __cpuinit cpu_callback(struct · line 824
Quoted 12 lines
kernel/softirq.c +2
-8-static const struct sched_param param = { - .sched_priority =MAX_RT_PRIO -1 - }; - - p =per_cpu (ksoftirqd, hotcpu); -per_cpu (ksoftirqd, hotcpu) =NULL ; -sched_setscheduler_nocheck (p,SCHED_FIFO , ¶m); -kthread_stop (p); +int hotcpu = (unsigned long )hcpu; +takeover_tasklets (hotcpu);break ;
Currently, "int hotcpu = (unsigned long)hcpu;" is somewhat strange. "takeover_tasklets((unsigned long)hcpu)" is sufficient for me.
[tip:smp/hotplug] softirq: Use hotplug thread infrastructure
In reply to Thomas GleixnerCommit-ID: 3e339b5dae24a7065e196eb8d0145ab2f8cc2d2d Gitweb: http://git.kernel.org/tip/3e339b5dae24a7065e196eb8d0145ab2f8cc2d2d Author: Thomas Gleixner <[email protected]> AuthorDate: Mon, 16 Jul 2012 10:42:37 +0000 Committer: Thomas Gleixner <[email protected]> CommitDate: Mon, 13 Aug 2012 17:01:07 +0200
softirq: Use hotplug thread infrastructure
[ paulmck: Call rcu_note_context_switch() with interrupts enabled. ]
Signed-off-by Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Srivatsa S. Bhat <[email protected]>
Cc Rusty Russell <[email protected]>
Reviewed-by Paul E. McKenney <[email protected]>
Cc Namhyung Kim <[email protected]>
Signed-off-by Thomas Gleixner <[email protected]>
--- kernel/softirq.c | 111 +++++++++++++---------------------------------------- 1 files changed, 27 insertions(+), 84 deletions(-)
kernel/softirq.c +27 -84
@@ -23,6 +23,7 @@#include <linux/rcupdate.h> #include <linux/ftrace.h> #include <linux/smp.h> +#include <linux/smpboot.h> #include <linux/tick.h> #define CREATE_TRACE_POINTS
@@ -742,49 +743,22 @@ void __init softirq_init(void)open_softirq (HI_SOFTIRQ , tasklet_hi_action); } -static int run_ksoftirqd (void * __bind_cpu) +static int ksoftirqd_should_run (unsigned int cpu) { -set_current_state (TASK_INTERRUPTIBLE ); - -while (!kthread_should_stop ()) { -preempt_disable (); -if (!local_softirq_pending ()) { -schedule_preempt_disabled (); - } - -__set_current_state (TASK_RUNNING ); - -while (local_softirq_pending ()) { -/* Preempt disable stops cpu going offline. -If already offline, we'll be on wrong CPU: -don't process */ -if (cpu_is_offline ((long )__bind_cpu)) - goto wait_to_die; -local_irq_disable (); -if (local_softirq_pending ()) -__do_softirq (); -local_irq_enable (); -sched_preempt_enable_no_resched (); -cond_resched (); -preempt_disable (); -rcu_note_context_switch ((long )__bind_cpu); - } -preempt_enable (); -set_current_state (TASK_INTERRUPTIBLE ); - } -__set_current_state (TASK_RUNNING ); -return 0 ; +return local_softirq_pending (); +} -wait_to_die: -preempt_enable (); -/* Wait for kthread_stop */ -set_current_state (TASK_INTERRUPTIBLE ); -while (!kthread_should_stop ()) { -schedule (); -set_current_state (TASK_INTERRUPTIBLE ); +static void run_ksoftirqd (unsigned int cpu) +{ +local_irq_disable (); +if (local_softirq_pending ()) { +__do_softirq (); +rcu_note_context_switch (cpu); +local_irq_enable (); +cond_resched (); +return ; } -__set_current_state (TASK_RUNNING ); -return 0 ; +local_irq_enable (); }#ifdef CONFIG_HOTPLUG_CPU
@@ -850,50 +824,14 @@ static int __cpuinit cpu_callback(struct notifier_block *nfb, unsigned long action, void *hcpu) { - int hotcpu = (unsigned long)hcpu; -struct task_struct *p; -switch (action) { -case CPU_UP_PREPARE : -case CPU_UP_PREPARE_FROZEN : - p =kthread_create_on_node (run_ksoftirqd, - hcpu, -cpu_to_node (hotcpu), -"ksoftirqd/%d" , hotcpu); -if (IS_ERR (p)) { -printk ("ksoftirqd for %i failed\n" , hotcpu); -return notifier_from_errno (PTR_ERR (p)); - } -kthread_bind (p, hotcpu); -per_cpu (ksoftirqd, hotcpu) = p; -break ; -case CPU_ONLINE : -case CPU_ONLINE_FROZEN : -wake_up_process (per_cpu (ksoftirqd, hotcpu)); -break ;#ifdef CONFIG_HOTPLUG_CPU -case CPU_UP_CANCELED : -case CPU_UP_CANCELED_FROZEN : -if (!per_cpu (ksoftirqd, hotcpu)) -break ; -/* Unbind so it can run. Fall thru. */ -kthread_bind (per_cpu (ksoftirqd, hotcpu), -cpumask_any (cpu_online_mask));case CPU_DEAD : -case CPU_DEAD_FROZEN : { -static const struct sched_param param = { - .sched_priority =MAX_RT_PRIO -1 - }; - - p =per_cpu (ksoftirqd, hotcpu); -per_cpu (ksoftirqd, hotcpu) =NULL ; -sched_setscheduler_nocheck (p,SCHED_FIFO , ¶m); -kthread_stop (p); -takeover_tasklets (hotcpu); +case CPU_DEAD_FROZEN : +takeover_tasklets ((unsigned long )hcpu);break ; - }#endif /* CONFIG_HOTPLUG_CPU */ - } + }return NOTIFY_OK ; }
@@ -901,14 +839,19 @@ static struct notifier_block __cpuinitdata cpu_nfb = { .notifier_call = cpu_callback }; +static struct smp_hotplug_thread softirq_threads = { + .store = &ksoftirqd, + .thread_should_run = ksoftirqd_should_run, + .thread_fn = run_ksoftirqd, + .thread_comm ="ksoftirqd/%u" , +}; +static __init int spawn_ksoftirqd (void ) { -void *cpu = (void *)(long )smp_processor_id (); -int err =cpu_callback (&cpu_nfb,CPU_UP_PREPARE , cpu); - -BUG_ON (err !=NOTIFY_OK ); -cpu_callback (&cpu_nfb,CPU_ONLINE , cpu);register_cpu_notifier (&cpu_nfb); + +BUG_ON (smpboot_register_percpu_thread (&softirq_threads)); +return 0 ; }early_initcall (spawn_ksoftirqd);
PATCH 5/7 watchdog: Use hotplug thread infrastructure
In reply to Thomas GleixnerSigned-off-by Thomas Gleixner <[email protected]>
--- kernel/watchdog.c | 264 ++++++++++++++++++------------------------------------ 1 file changed, 89 insertions(+), 175 deletions(-)
kernel/watchdog.c +89 -175
@@ -22,6 +22,7 @@#include <linux/notifier.h> #include <linux/module.h> #include <linux/sysctl.h> +#include <linux/smpboot.h> #include <asm/irq_regs.h> #include <linux/kvm_para.h>
@@ -29,16 +30,18 @@int watchdog_enabled =1 ;int __read_mostly watchdog_thresh =10 ; +static int __read_mostly watchdog_disabled; static DEFINE_PER_CPU(unsigned long, watchdog_touch_ts); static DEFINE_PER_CPU(struct task_struct *, softlockup_watchdog); static DEFINE_PER_CPU(struct hrtimer, watchdog_hrtimer); static DEFINE_PER_CPU(bool, softlockup_touch_sync); static DEFINE_PER_CPU(bool, soft_watchdog_warn); +static DEFINE_PER_CPU(unsigned long, hrtimer_interrupts); +static DEFINE_PER_CPU(unsigned long, soft_lockup_hrtimer_cnt);#ifdef CONFIG_HARDLOCKUP_DETECTOR static DEFINE_PER_CPU(bool, hard_watchdog_warn); static DEFINE_PER_CPU(bool, watchdog_nmi_touch); -static DEFINE_PER_CPU(unsigned long, hrtimer_interrupts); static DEFINE_PER_CPU(unsigned long, hrtimer_interrupts_saved); static DEFINE_PER_CPU(struct perf_event *, watchdog_ev);#endif
@@ -248,13 +251,15 @@ static void watchdog_overflow_callback(s__this_cpu_write (hard_watchdog_warn, false); return; } +#endif /* CONFIG_HARDLOCKUP_DETECTOR */ +static void watchdog_interrupt_count (void ) {__this_cpu_inc (hrtimer_interrupts); } -#else -static inline void watchdog_interrupt_count (void ) {return ; } -#endif /* CONFIG_HARDLOCKUP_DETECTOR */ + +static int watchdog_nmi_enable (unsigned int cpu); +static void watchdog_nmi_disable (unsigned int cpu);/* watchdog kicker functions */ static enum hrtimer_restart watchdog_timer_fn (struct hrtimer *hrtimer)
@@ -327,52 +332,71 @@ static enum hrtimer_restart watchdog_timreturn HRTIMER_RESTART ; } +static void watchdog_set_prio (unsigned int policy,unsigned int prio) +{ +struct sched_param param = { .sched_priority = prio }; -/* -* The watchdog thread - touches the timestamp. -*/ -static int watchdog (void *unused) +sched_setscheduler (current, policy, ¶m); +} + +static void watchdog_enable (unsigned int cpu) { -struct sched_param param = { .sched_priority =0 };struct hrtimer *hrtimer = &__raw_get_cpu_var (watchdog_hrtimer); -/* initialize timestamp */ -__touch_watchdog (); +if (!watchdog_enabled) { +kthread_park (current); +return ; + } + +/* Enable the perf event */ +watchdog_nmi_enable (cpu);/* kick off the timer for the hardlockup detector */ +hrtimer_init (hrtimer,CLOCK_MONOTONIC ,HRTIMER_MODE_REL ); + hrtimer->function = watchdog_timer_fn; +/* done here because hrtimer_start can only pin to smp_processor_id() */ hrtimer_start (hrtimer,ns_to_ktime (get_sample_period ()),HRTIMER_MODE_REL_PINNED ); -set_current_state (TASK_INTERRUPTIBLE ); -/* -* Run briefly (kicked by the hrtimer callback function) once every -* get_sample_period() seconds (4 seconds by default) to reset the -* softlockup timestamp. If this gets delayed for more than -* 2*watchdog_thresh seconds then the debug-printout triggers in -* watchdog_timer_fn(). -*/ -while (!kthread_should_stop ()) { -__touch_watchdog (); -schedule (); +/* initialize timestamp */ +watchdog_set_prio (SCHED_FIFO ,MAX_RT_PRIO -1 ); +__touch_watchdog (); +} -if (kthread_should_stop ()) -break ; +static void watchdog_disable (unsigned int cpu) +{ +struct hrtimer *hrtimer = &__raw_get_cpu_var (watchdog_hrtimer); -set_current_state (TASK_INTERRUPTIBLE ); - } -/* -* Drop the policy/priority elevation during thread exit to avoid a -* scheduling latency spike. -*/ -__set_current_state (TASK_RUNNING ); -sched_setscheduler (current,SCHED_NORMAL , ¶m); -return 0 ; +watchdog_set_prio (SCHED_NORMAL ,0 ); +hrtimer_cancel (hrtimer); +/* disable the perf event */ +watchdog_nmi_disable (cpu); +} + +static int watchdog_should_run (unsigned int cpu) +{ +return __this_cpu_read (hrtimer_interrupts) != +__this_cpu_read (soft_lockup_hrtimer_cnt); } +/* +* The watchdog thread function - touches the timestamp. +* +* It only runs once every get_sample_period() seconds (4 seconds by +* default) to reset the softlockup timestamp. If this gets delayed +* for more than 2*watchdog_thresh seconds then the debug-printout +* triggers in watchdog_timer_fn(). +*/ +static void watchdog (unsigned int cpu) +{ +__this_cpu_write (soft_lockup_hrtimer_cnt, +__this_cpu_read (hrtimer_interrupts)); +__touch_watchdog (); +}#ifdef CONFIG_HARDLOCKUP_DETECTOR -static int watchdog_nmi_enable (int cpu) +static int watchdog_nmi_enable (unsigned int cpu) {struct perf_event_attr *wd_attr;struct perf_event *event =per_cpu (watchdog_ev, cpu);
@@ -395,7 +419,6 @@ static int watchdog_nmi_enable(int cpu) goto out_save; } -/* vary the KERN level based on the returned errno */ if (PTR_ERR (event) == -EOPNOTSUPP )pr_info ("disabled (cpu%i): not supported (no LAPIC?)\n" , cpu);
@@ -416,7 +439,7 @@ out:return 0 ; } -static void watchdog_nmi_disable (int cpu) +static void watchdog_nmi_disable (unsigned int cpu) {struct perf_event *event =per_cpu (watchdog_ev, cpu);
@@ -430,107 +453,35 @@ static void watchdog_nmi_disable(int cpu return; }#else -static int watchdog_nmi_enable (int cpu) {return 0 ; } -static void watchdog_nmi_disable (int cpu) {return ; } +static int watchdog_nmi_enable (unsigned int cpu) {return 0 ; } +static void watchdog_nmi_disable (unsigned int cpu) {return ; }#endif /* CONFIG_HARDLOCKUP_DETECTOR */ /* prepare/enable/disable routines */ -static void watchdog_prepare_cpu (int cpu) -{ -struct hrtimer *hrtimer = &per_cpu (watchdog_hrtimer, cpu); - -WARN_ON (per_cpu (softlockup_watchdog, cpu)); -hrtimer_init (hrtimer,CLOCK_MONOTONIC ,HRTIMER_MODE_REL ); - hrtimer->function = watchdog_timer_fn; -} - -static int watchdog_enable (int cpu) -{ -struct task_struct *p =per_cpu (softlockup_watchdog, cpu); -int err =0 ; - -/* enable the perf event */ - err =watchdog_nmi_enable (cpu); - -/* Regardless of err above, fall through and start softlockup */ - -/* create the watchdog thread */ -if (!p) { -struct sched_param param = { .sched_priority =MAX_RT_PRIO -1 }; - p =kthread_create_on_node (watchdog,NULL ,cpu_to_node (cpu),"watchdog/%d" , cpu); -if (IS_ERR (p)) { -pr_err ("softlockup watchdog for %i failed\n" , cpu); -if (!err) { -/* if hardlockup hasn't already set this */ - err =PTR_ERR (p); -/* and disable the perf event */ -watchdog_nmi_disable (cpu); - } - goto out; - } -sched_setscheduler (p,SCHED_FIFO , ¶m); -kthread_bind (p, cpu); -per_cpu (watchdog_touch_ts, cpu) =0 ; -per_cpu (softlockup_watchdog, cpu) = p; -wake_up_process (p); - } - -out: -return err; -} - -static void watchdog_disable (int cpu) -{ -struct task_struct *p =per_cpu (softlockup_watchdog, cpu); -struct hrtimer *hrtimer = &per_cpu (watchdog_hrtimer, cpu); - -/* -* cancel the timer first to stop incrementing the stats -* and waking up the kthread -*/ -hrtimer_cancel (hrtimer); - -/* disable the perf event */ -watchdog_nmi_disable (cpu); - -/* stop the watchdog thread */ -if (p) { -per_cpu (softlockup_watchdog, cpu) =NULL ; -kthread_stop (p); - } -} -/* sysctl functions */ #ifdef CONFIG_SYSCTL static void watchdog_enable_all_cpus (void ) { -int cpu; - - watchdog_enabled =0 ; - -for_each_online_cpu (cpu) -if (!watchdog_enable (cpu)) -/* if any cpu succeeds, watchdog is considered -enabled for the system */ - watchdog_enabled =1 ; - -if (!watchdog_enabled) -pr_err ("failed to be enabled on some cpus\n" ); +unsigned int cpu; +if (watchdog_disabled) { + watchdog_disabled =0 ; +for_each_online_cpu (cpu) +kthread_unpark (per_cpu (softlockup_watchdog, cpu)); + } }static void watchdog_disable_all_cpus (void ) { -int cpu; - -for_each_online_cpu (cpu) -watchdog_disable (cpu); +unsigned int cpu; -/* if all watchdogs are disabled, then they are disabled for the system */ - watchdog_enabled =0 ; +if (!watchdog_disabled) { + watchdog_disabled =1 ; +for_each_online_cpu (cpu) +kthread_park (per_cpu (softlockup_watchdog, cpu)); + } } -/* * proc handler for /proc/sys/kernel/nmi_watchdog,watchdog_thresh */
@@ -540,73 +491,36 @@ int proc_dowatchdog(struct ctl_table *ta {int ret; +if (watchdog_disabled <0 ) +return -ENODEV ; + ret =proc_dointvec_minmax (table, write, buffer, lenp, ppos);if (ret || !write) - goto out; +return ret;if (watchdog_enabled && watchdog_thresh)watchdog_enable_all_cpus ();else watchdog_disable_all_cpus (); -out:return ret; }#endif /* CONFIG_SYSCTL */ - -/* -* Create/destroy watchdog threads as CPUs come and go: -*/ -static int __cpuinit -cpu_callback (struct notifier_block *nfb,unsigned long action,void *hcpu) -{ -int hotcpu = (unsigned long )hcpu; - -switch (action) { -case CPU_UP_PREPARE : -case CPU_UP_PREPARE_FROZEN : -watchdog_prepare_cpu (hotcpu); -break ; -case CPU_ONLINE : -case CPU_ONLINE_FROZEN : -if (watchdog_enabled) -watchdog_enable (hotcpu); -break ; -#ifdef CONFIG_HOTPLUG_CPU -case CPU_UP_CANCELED : -case CPU_UP_CANCELED_FROZEN : -watchdog_disable (hotcpu); -break ; -case CPU_DEAD : -case CPU_DEAD_FROZEN : -watchdog_disable (hotcpu); -break ; -#endif /* CONFIG_HOTPLUG_CPU */ - } - -/* -* hardlockup and softlockup are not important enough -* to block cpu bring up. Just always succeed and -* rely on printk output to flag problems. -*/ -return NOTIFY_OK ; -} - -static struct notifier_block __cpuinitdata cpu_nfb = { - .notifier_call = cpu_callback +static struct smp_hotplug_thread watchdog_threads = { + .store = &softlockup_watchdog, + .thread_should_run = watchdog_should_run, + .thread_fn = watchdog, + .thread_comm ="watchdog/%u" , + .setup = watchdog_enable, + .park = watchdog_disable, + .unpark = watchdog_enable, };void __init lockup_detector_init (void ) { -void *cpu = (void *)(long )smp_processor_id (); -int err; - - err =cpu_callback (&cpu_nfb,CPU_UP_PREPARE , cpu); -WARN_ON (notifier_to_errno (err)); - -cpu_callback (&cpu_nfb,CPU_ONLINE , cpu); -register_cpu_notifier (&cpu_nfb); - -return ; +if (smpboot_register_percpu_thread (&watchdog_threads)) { +pr_err ("Failed to create watchdog threads, disabled\n" ); + watchdog_disabled = -ENODEV ; + } }
[tip:smp/hotplug] watchdog: Use hotplug thread infrastructure
Commit-ID: bcd951cf10f24e341defcd002c15a1f4eea13ddb Gitweb: http://git.kernel.org/tip/bcd951cf10f24e341defcd002c15a1f4eea13ddb Author: Thomas Gleixner <[email protected]> AuthorDate: Mon, 16 Jul 2012 10:42:38 +0000 Committer: Thomas Gleixner <[email protected]> CommitDate: Mon, 13 Aug 2012 17:01:07 +0200
watchdog: Use hotplug thread infrastructure
Signed-off-by Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Srivatsa S. Bhat <[email protected]>
Cc Rusty Russell <[email protected]>
Reviewed-by Paul E. McKenney <[email protected]>
Cc Namhyung Kim <[email protected]>
Signed-off-by Thomas Gleixner <[email protected]>
--- kernel/watchdog.c | 263 ++++++++++++++++++----------------------------------- 1 files changed, 89 insertions(+), 174 deletions(-)
kernel/watchdog.c +89 -174
@@ -22,6 +22,7 @@#include <linux/notifier.h> #include <linux/module.h> #include <linux/sysctl.h> +#include <linux/smpboot.h> #include <asm/irq_regs.h> #include <linux/kvm_para.h>
@@ -29,16 +30,18 @@int watchdog_enabled =1 ;int __read_mostly watchdog_thresh =10 ; +static int __read_mostly watchdog_disabled; static DEFINE_PER_CPU(unsigned long, watchdog_touch_ts); static DEFINE_PER_CPU(struct task_struct *, softlockup_watchdog); static DEFINE_PER_CPU(struct hrtimer, watchdog_hrtimer); static DEFINE_PER_CPU(bool, softlockup_touch_sync); static DEFINE_PER_CPU(bool, soft_watchdog_warn); +static DEFINE_PER_CPU(unsigned long, hrtimer_interrupts); +static DEFINE_PER_CPU(unsigned long, soft_lockup_hrtimer_cnt);#ifdef CONFIG_HARDLOCKUP_DETECTOR static DEFINE_PER_CPU(bool, hard_watchdog_warn); static DEFINE_PER_CPU(bool, watchdog_nmi_touch); -static DEFINE_PER_CPU(unsigned long, hrtimer_interrupts); static DEFINE_PER_CPU(unsigned long, hrtimer_interrupts_saved); static DEFINE_PER_CPU(struct perf_event *, watchdog_ev);#endif
@@ -248,13 +251,15 @@ static void watchdog_overflow_callback(struct perf_event *event,__this_cpu_write (hard_watchdog_warn, false); return; } +#endif /* CONFIG_HARDLOCKUP_DETECTOR */ +static void watchdog_interrupt_count (void ) {__this_cpu_inc (hrtimer_interrupts); } -#else -static inline void watchdog_interrupt_count (void ) {return ; } -#endif /* CONFIG_HARDLOCKUP_DETECTOR */ + +static int watchdog_nmi_enable (unsigned int cpu); +static void watchdog_nmi_disable (unsigned int cpu);/* watchdog kicker functions */ static enum hrtimer_restart watchdog_timer_fn (struct hrtimer *hrtimer)
@@ -327,49 +332,68 @@ static enum hrtimer_restart watchdog_timer_fn(struct hrtimer *hrtimer)return HRTIMER_RESTART ; } +static void watchdog_set_prio (unsigned int policy,unsigned int prio) +{ +struct sched_param param = { .sched_priority = prio }; -/* -* The watchdog thread - touches the timestamp. -*/ -static int watchdog (void *unused) +sched_setscheduler (current, policy, ¶m); +} + +static void watchdog_enable (unsigned int cpu) { -struct sched_param param = { .sched_priority =0 };struct hrtimer *hrtimer = &__raw_get_cpu_var (watchdog_hrtimer); -/* initialize timestamp */ -__touch_watchdog (); +if (!watchdog_enabled) { +kthread_park (current); +return ; + } + +/* Enable the perf event */ +watchdog_nmi_enable (cpu);/* kick off the timer for the hardlockup detector */ +hrtimer_init (hrtimer,CLOCK_MONOTONIC ,HRTIMER_MODE_REL ); + hrtimer->function = watchdog_timer_fn; +/* done here because hrtimer_start can only pin to smp_processor_id() */ hrtimer_start (hrtimer,ns_to_ktime (get_sample_period ()),HRTIMER_MODE_REL_PINNED ); -set_current_state (TASK_INTERRUPTIBLE ); -/* -* Run briefly (kicked by the hrtimer callback function) once every -* get_sample_period() seconds (4 seconds by default) to reset the -* softlockup timestamp. If this gets delayed for more than -* 2*watchdog_thresh seconds then the debug-printout triggers in -* watchdog_timer_fn(). -*/ -while (!kthread_should_stop ()) { -__touch_watchdog (); -schedule (); +/* initialize timestamp */ +watchdog_set_prio (SCHED_FIFO ,MAX_RT_PRIO -1 ); +__touch_watchdog (); +} -if (kthread_should_stop ()) -break ; +static void watchdog_disable (unsigned int cpu) +{ +struct hrtimer *hrtimer = &__raw_get_cpu_var (watchdog_hrtimer); -set_current_state (TASK_INTERRUPTIBLE ); - } -/* -* Drop the policy/priority elevation during thread exit to avoid a -* scheduling latency spike. -*/ -__set_current_state (TASK_RUNNING ); -sched_setscheduler (current,SCHED_NORMAL , ¶m); -return 0 ; +watchdog_set_prio (SCHED_NORMAL ,0 ); +hrtimer_cancel (hrtimer); +/* disable the perf event */ +watchdog_nmi_disable (cpu); } +static int watchdog_should_run (unsigned int cpu) +{ +return __this_cpu_read (hrtimer_interrupts) != +__this_cpu_read (soft_lockup_hrtimer_cnt); +} + +/* +* The watchdog thread function - touches the timestamp. +* +* It only runs once every get_sample_period() seconds (4 seconds by +* default) to reset the softlockup timestamp. If this gets delayed +* for more than 2*watchdog_thresh seconds then the debug-printout +* triggers in watchdog_timer_fn(). +*/ +static void watchdog (unsigned int cpu) +{ +__this_cpu_write (soft_lockup_hrtimer_cnt, +__this_cpu_read (hrtimer_interrupts)); +__touch_watchdog (); +}#ifdef CONFIG_HARDLOCKUP_DETECTOR /*
@@ -379,7 +403,7 @@ static int watchdog(void *unused)*/ static unsigned long cpu0_err; -static int watchdog_nmi_enable (int cpu) +static int watchdog_nmi_enable (unsigned int cpu) {struct perf_event_attr *wd_attr;struct perf_event *event =per_cpu (watchdog_ev, cpu);
@@ -433,7 +457,7 @@ out:return 0 ; } -static void watchdog_nmi_disable (int cpu) +static void watchdog_nmi_disable (unsigned int cpu) {struct perf_event *event =per_cpu (watchdog_ev, cpu);
@@ -447,107 +471,35 @@ static void watchdog_nmi_disable(int cpu) return; }#else -static int watchdog_nmi_enable (int cpu) {return 0 ; } -static void watchdog_nmi_disable (int cpu) {return ; } +static int watchdog_nmi_enable (unsigned int cpu) {return 0 ; } +static void watchdog_nmi_disable (unsigned int cpu) {return ; }#endif /* CONFIG_HARDLOCKUP_DETECTOR */ /* prepare/enable/disable routines */ -static void watchdog_prepare_cpu (int cpu) -{ -struct hrtimer *hrtimer = &per_cpu (watchdog_hrtimer, cpu); - -WARN_ON (per_cpu (softlockup_watchdog, cpu)); -hrtimer_init (hrtimer,CLOCK_MONOTONIC ,HRTIMER_MODE_REL ); - hrtimer->function = watchdog_timer_fn; -} - -static int watchdog_enable (int cpu) -{ -struct task_struct *p =per_cpu (softlockup_watchdog, cpu); -int err =0 ; - -/* enable the perf event */ - err =watchdog_nmi_enable (cpu); - -/* Regardless of err above, fall through and start softlockup */ - -/* create the watchdog thread */ -if (!p) { -struct sched_param param = { .sched_priority =MAX_RT_PRIO -1 }; - p =kthread_create_on_node (watchdog,NULL ,cpu_to_node (cpu),"watchdog/%d" , cpu); -if (IS_ERR (p)) { -pr_err ("softlockup watchdog for %i failed\n" , cpu); -if (!err) { -/* if hardlockup hasn't already set this */ - err =PTR_ERR (p); -/* and disable the perf event */ -watchdog_nmi_disable (cpu); - } - goto out; - } -sched_setscheduler (p,SCHED_FIFO , ¶m); -kthread_bind (p, cpu); -per_cpu (watchdog_touch_ts, cpu) =0 ; -per_cpu (softlockup_watchdog, cpu) = p; -wake_up_process (p); - } - -out: -return err; -} - -static void watchdog_disable (int cpu) -{ -struct task_struct *p =per_cpu (softlockup_watchdog, cpu); -struct hrtimer *hrtimer = &per_cpu (watchdog_hrtimer, cpu); - -/* -* cancel the timer first to stop incrementing the stats -* and waking up the kthread -*/ -hrtimer_cancel (hrtimer); - -/* disable the perf event */ -watchdog_nmi_disable (cpu); - -/* stop the watchdog thread */ -if (p) { -per_cpu (softlockup_watchdog, cpu) =NULL ; -kthread_stop (p); - } -} -/* sysctl functions */ #ifdef CONFIG_SYSCTL static void watchdog_enable_all_cpus (void ) { -int cpu; - - watchdog_enabled =0 ; - -for_each_online_cpu (cpu) -if (!watchdog_enable (cpu)) -/* if any cpu succeeds, watchdog is considered -enabled for the system */ - watchdog_enabled =1 ; - -if (!watchdog_enabled) -pr_err ("failed to be enabled on some cpus\n" ); +unsigned int cpu; +if (watchdog_disabled) { + watchdog_disabled =0 ; +for_each_online_cpu (cpu) +kthread_unpark (per_cpu (softlockup_watchdog, cpu)); + } }static void watchdog_disable_all_cpus (void ) { -int cpu; - -for_each_online_cpu (cpu) -watchdog_disable (cpu); +unsigned int cpu; -/* if all watchdogs are disabled, then they are disabled for the system */ - watchdog_enabled =0 ; +if (!watchdog_disabled) { + watchdog_disabled =1 ; +for_each_online_cpu (cpu) +kthread_park (per_cpu (softlockup_watchdog, cpu)); + } } -/* * proc handler for /proc/sys/kernel/nmi_watchdog,watchdog_thresh */
@@ -557,73 +509,36 @@ int proc_dowatchdog(struct ctl_table *table, int write, {int ret; +if (watchdog_disabled <0 ) +return -ENODEV ; + ret =proc_dointvec_minmax (table, write, buffer, lenp, ppos);if (ret || !write) - goto out; +return ret;if (watchdog_enabled && watchdog_thresh)watchdog_enable_all_cpus ();else watchdog_disable_all_cpus (); -out:return ret; }#endif /* CONFIG_SYSCTL */ - -/* -* Create/destroy watchdog threads as CPUs come and go: -*/ -static int __cpuinit -cpu_callback (struct notifier_block *nfb,unsigned long action,void *hcpu) -{ -int hotcpu = (unsigned long )hcpu; - -switch (action) { -case CPU_UP_PREPARE : -case CPU_UP_PREPARE_FROZEN : -watchdog_prepare_cpu (hotcpu); -break ; -case CPU_ONLINE : -case CPU_ONLINE_FROZEN : -if (watchdog_enabled) -watchdog_enable (hotcpu); -break ; -#ifdef CONFIG_HOTPLUG_CPU -case CPU_UP_CANCELED : -case CPU_UP_CANCELED_FROZEN : -watchdog_disable (hotcpu); -break ; -case CPU_DEAD : -case CPU_DEAD_FROZEN : -watchdog_disable (hotcpu); -break ; -#endif /* CONFIG_HOTPLUG_CPU */ - } - -/* -* hardlockup and softlockup are not important enough -* to block cpu bring up. Just always succeed and -* rely on printk output to flag problems. -*/ -return NOTIFY_OK ; -} - -static struct notifier_block __cpuinitdata cpu_nfb = { - .notifier_call = cpu_callback +static struct smp_hotplug_thread watchdog_threads = { + .store = &softlockup_watchdog, + .thread_should_run = watchdog_should_run, + .thread_fn = watchdog, + .thread_comm ="watchdog/%u" , + .setup = watchdog_enable, + .park = watchdog_disable, + .unpark = watchdog_enable, };void __init lockup_detector_init (void ) { -void *cpu = (void *)(long )smp_processor_id (); -int err; - - err =cpu_callback (&cpu_nfb,CPU_UP_PREPARE , cpu); -WARN_ON (notifier_to_errno (err)); - -cpu_callback (&cpu_nfb,CPU_ONLINE , cpu); -register_cpu_notifier (&cpu_nfb); - -return ; +if (smpboot_register_percpu_thread (&watchdog_threads)) { +pr_err ("Failed to create watchdog threads, disabled\n" ); + watchdog_disabled = -ENODEV ; + } }
Re: [tip:smp/hotplug] watchdog: Use hotplug thread infrastructure
Quoted 8 lines · tip-bot for Thomas Gleixner
Commit-ID: bcd951cf10f24e341defcd002c15a1f4eea13ddb Gitweb: http://git.kernel.org/tip/bcd951cf10f24e341defcd002c15a1f4eea13ddb Author: Thomas Gleixner <[email protected]> AuthorDate: Mon, 16 Jul 2012 10:42:38 +0000 Committer: Thomas Gleixner <[email protected]> CommitDate: Mon, 13 Aug 2012 17:01:07 +0200watchdog: Use hotplug thread infrastructure
Hi Thomas,
I am maintaining http://git.linaro.org/gitweb?p=arm/big.LITTLE/mp.git;a=summary
linaro tree for big.LITTLE MP work. I had a branch per-cpu-thread-hotplug-v3-fixed
with your stuff in it + Paul's fix.
I was rebasing it over 3.6-rc1 and got conflicts, actually a routine is added in kernel/watchdog.c file, which uses functions removed in your patch:
commit 45226e944ce071d0231949f2fea90969437cd2dc
Author: Sameer Nanda <[email protected]> Date: Mon Jul 30 14:40:00 2012 -0700
NMI watchdog: fix for lockup detector breakage on resume
+#ifdef CONFIG_SUSPEND
+/*
+ * On exit from suspend we force an offline->online transition on the boot CPU
+ * so that the PMU state that was lost while in suspended state gets set up
+ * properly for the boot CPU. This information is required for restarting the
+ * NMI watchdog.
+ */
+void lockup_detector_bootcpu_resume(void)
+{
+ void *cpu = (void *)(long)smp_processor_id();
+
+ cpu_callback(&cpu_nfb, CPU_DEAD_FROZEN, cpu); + cpu_callback(&cpu_nfb, CPU_UP_PREPARE_FROZEN, cpu); + cpu_callback(&cpu_nfb, CPU_ONLINE_FROZEN, cpu);
+}
+#endif
+
Now, cpu_callback is removed in your patch. Can you please share an updated patch to fix this issue? Or let me know how to handle it?
viresh
Re: [tip:smp/hotplug] watchdog: Use hotplug thread infrastructure
Quoted 22 lines · viresh kumar
Quoted 8 lines · tip-bot for Thomas Gleixner
Commit-ID: bcd951cf10f24e341defcd002c15a1f4eea13ddb Gitweb: http://git.kernel.org/tip/bcd951cf10f24e341defcd002c15a1f4eea13ddb Author: Thomas Gleixner <[email protected]> AuthorDate: Mon, 16 Jul 2012 10:42:38 +0000 Committer: Thomas Gleixner <[email protected]> CommitDate: Mon, 13 Aug 2012 17:01:07 +0200watchdog: Use hotplug thread infrastructure
Hi Thomas,
I am maintaining http://git.linaro.org/gitweb?p=arm/big.LITTLE/mp.git;a=summary
linaro tree for big.LITTLE MP work. I had a branch per-cpu-thread-hotplug-v3-fixed
with your stuff in it + Paul's fix.I was rebasing it over 3.6-rc1 and got conflicts, actually a routine is added in
kernel/watchdog.cfile, which uses functions removed in your patch:
Quoted 5 lines
commit
45226e944ce071d0231949f2fea90969437cd2dcAuthor: Sameer Nanda <[email protected]> Date: Mon Jul 30 14:40:00 2012 -0700NMI watchdog: fix for lockup detector breakage on resume
This patch got reverted in mainline. See commit commit 300d3739 (Revert "NMI watchdog: fix for lockup detector breakage on resume")
So nothing to fix there.
Thanks,
tglx
PATCH 6/7 rcu: Use smp_hotplug_thread facility for RCUs per-CPU kthread
In reply to Thomas GleixnerFrom: "Paul E. McKenney" <[email protected]>
Bring RCU into the new-age CPU-hotplug fold by modifying RCU's per-CPU kthread code to use the new smp_hotplug_thread facility.
[ tglx: Adapted it to use callbacks and to the simplified rcu yield ]
Signed-off-by Paul E. McKenney <[email protected]>
Signed-off-by Thomas Gleixner <[email protected]>
--- kernel/rcutree.c | 4 kernel/rcutree.h | 8 - kernel/rcutree_plugin.h | 203 +++++++++--------------------------------------- kernel/rcutree_trace.c | 3 4 files changed, 41 insertions(+), 177 deletions(-)
kernel/rcutree.c +0 -4
@@ -125,7 +125,6 @@ static int rcu_scheduler_fully_active __*/ static DEFINE_PER_CPU(struct task_struct *, rcu_cpu_kthread_task);DEFINE_PER_CPU (unsigned int , rcu_cpu_kthread_status); -DEFINE_PER_CPU (int, rcu_cpu_kthread_cpu); DEFINE_PER_CPU(unsigned int, rcu_cpu_kthread_loops);DEFINE_PER_CPU (char, rcu_cpu_has_work);
@@ -1458,7 +1457,6 @@ static void rcu_cleanup_dead_cpu(int cpustruct rcu_node *rnp = rdp->mynode;/* Outgoing CPU's rdp & rnp. */ /* Adjust any no-longer-needed kthreads. */ -rcu_stop_cpu_kthread (cpu);rcu_boost_kthread_setaffinity (rnp,-1 );/* Remove the dead CPU from the bitmasks in the rcu_node hierarchy. */
@@ -2514,11 +2512,9 @@ static int __cpuinit rcu_cpu_notify(strucase CPU_ONLINE :case CPU_DOWN_FAILED :rcu_boost_kthread_setaffinity (rnp,-1 ); -rcu_cpu_kthread_setrt (cpu,1 );break ;case CPU_DOWN_PREPARE :rcu_boost_kthread_setaffinity (rnp, cpu); -rcu_cpu_kthread_setrt (cpu,0 );break ;case CPU_DYING :case CPU_DYING_FROZEN :
kernel/rcutree.h +0 -8
@@ -178,12 +178,6 @@ struct rcu_node {/* Refused to boost: not sure why, though. */ /* This can happen due to race conditions. */ #endif /* #ifdef CONFIG_RCU_BOOST */ -struct task_struct *node_kthread_task; -/* kthread that takes care of this rcu_node */ -/* structure, for example, awakening the */ -/* per-CPU kthreads as needed. */ -unsigned int node_kthread_status; -/* State of node_kthread_task for tracing. */ }____cacheline_internodealigned_in_smp ;/*
@@ -434,7 +428,6 @@ static int rcu_preempt_blocked_readers_c#ifdef CONFIG_HOTPLUG_CPU static void rcu_report_unblock_qs_rnp (struct rcu_node *rnp,unsigned long flags); -static void rcu_stop_cpu_kthread (int cpu);#endif /* #ifdef CONFIG_HOTPLUG_CPU */ static void rcu_print_detail_task_stall (struct rcu_state *rsp);static int rcu_print_task_stall (struct rcu_node *rnp);
@@ -467,7 +460,6 @@ static void rcu_preempt_do_callbacks(voistatic int __cpuinit rcu_spawn_one_boost_kthread (struct rcu_state *rsp,struct rcu_node *rnp);#endif /* #ifdef CONFIG_RCU_BOOST */ -static void rcu_cpu_kthread_setrt (int cpu,int to_rt);static void __cpuinit rcu_prepare_kthreads (int cpu);static void rcu_prepare_for_idle_init (int cpu);static void rcu_cleanup_after_idle (int cpu);
kernel/rcutree_plugin.h +40 -163
275 lines not on this page · raw message
kernel/rcutree_trace.c +1 -2
@@ -83,11 +83,10 @@ static void print_one_rcu_data(struct se rdp->nxttail[RCU_WAIT_TAIL]], ".D"[&rdp->nxtlist != rdp->nxttail[RCU_DONE_TAIL]]);#ifdef CONFIG_RCU_BOOST -seq_printf (m," kt=%d/%c/%d ktl=%x" , +seq_printf (m," kt=%d/%c ktl=%x" ,per_cpu (rcu_cpu_has_work, rdp->cpu),convert_kthread_status (per_cpu (rcu_cpu_kthread_status, rdp->cpu)), -per_cpu (rcu_cpu_kthread_cpu, rdp->cpu),per_cpu (rcu_cpu_kthread_loops, rdp->cpu) &0xffff );#endif /* #ifdef CONFIG_RCU_BOOST */ seq_printf (m," b=%ld" , rdp->blimit);
Re: [Patch 6/7] rcu: Use smp_hotplug_thread facility for RCUs per-CPU kthread
Quoted 6 lines · Thomas Gleixner
From: "Paul E. McKenney" <[email protected]>
Bring RCU into the new-age CPU-hotplug fold by modifying RCU's per-CPU kthread code to use the new
smp_hotplug_threadfacility.[ tglx: Adapted it to use callbacks and to the simplified rcu yield ]
This version does look a lot nicer!
Thanx, Paul
Quoted 380 lines
Signed-off-by Paul E. McKenney <[email protected]>Signed-off-by Thomas Gleixner <[email protected]>--- kernel/rcutree.c | 4 kernel/rcutree.h | 8 - kernel/rcutree_plugin.h | 203 +++++++++--------------------------------------- kernel/rcutree_trace.c | 3 4 files changed, 41 insertions(+), 177 deletions(-)Index:
tip/kernel/rcutree.c
===================================================================kernel/rcutree.c +0
-4@@ -125,7 +125,6 @@ static int rcu_scheduler_fully_active __*/ static DEFINE_PER_CPU(struct task_struct *, rcu_cpu_kthread_task);DEFINE_PER_CPU (unsigned int , rcu_cpu_kthread_status); -DEFINE_PER_CPU (int, rcu_cpu_kthread_cpu); DEFINE_PER_CPU(unsigned int, rcu_cpu_kthread_loops);DEFINE_PER_CPU (char, rcu_cpu_has_work);@@ -1458,7 +1457,6 @@ static void rcu_cleanup_dead_cpu(int cpustruct rcu_node *rnp = rdp->mynode;/* Outgoing CPU's rdp & rnp. */ /* Adjust any no-longer-needed kthreads. */ -rcu_stop_cpu_kthread (cpu);rcu_boost_kthread_setaffinity (rnp,-1 );/* Remove the dead CPU from the bitmasks in the rcu_node hierarchy. */ @@ -2514,11 +2512,9 @@ static int __cpuinit rcu_cpu_notify(strucase CPU_ONLINE :case CPU_DOWN_FAILED :rcu_boost_kthread_setaffinity (rnp,-1 ); -rcu_cpu_kthread_setrt (cpu,1 );break ;case CPU_DOWN_PREPARE :rcu_boost_kthread_setaffinity (rnp, cpu); -rcu_cpu_kthread_setrt (cpu,0 );break ;case CPU_DYING :case CPU_DYING_FROZEN :Index:
tip/kernel/rcutree.h
===================================================================kernel/rcutree.h +0
-8@@ -178,12 +178,6 @@ struct rcu_node {/* Refused to boost: not sure why, though. */ /* This can happen due to race conditions. */ #endif /* #ifdef CONFIG_RCU_BOOST */ -struct task_struct *node_kthread_task; -/* kthread that takes care of this rcu_node */ -/* structure, for example, awakening the */ -/* per-CPU kthreads as needed. */ -unsigned int node_kthread_status; -/* State of node_kthread_task for tracing. */ }____cacheline_internodealigned_in_smp ;/* @@ -434,7 +428,6 @@ static int rcu_preempt_blocked_readers_c#ifdef CONFIG_HOTPLUG_CPU static void rcu_report_unblock_qs_rnp (struct rcu_node *rnp,unsigned long flags); -static void rcu_stop_cpu_kthread (int cpu);#endif /* #ifdef CONFIG_HOTPLUG_CPU */ static void rcu_print_detail_task_stall (struct rcu_state *rsp);static int rcu_print_task_stall (struct rcu_node *rnp);@@ -467,7 +460,6 @@ static void rcu_preempt_do_callbacks(voistatic int __cpuinit rcu_spawn_one_boost_kthread (struct rcu_state *rsp,struct rcu_node *rnp);#endif /* #ifdef CONFIG_RCU_BOOST */ -static void rcu_cpu_kthread_setrt (int cpu,int to_rt);static void __cpuinit rcu_prepare_kthreads (int cpu);static void rcu_prepare_for_idle_init (int cpu);static void rcu_cleanup_after_idle (int cpu);Index:
tip/kernel/rcutree_plugin.h
===================================================================kernel/rcutree_plugin.h +40
-163@@ -25,6 +25,7 @@*/ #include <linux/delay.h> +#include <linux/smpboot.h> #define RCU_KTHREAD_PRIO 1@@ -1440,25 +1441,6 @@ static int __cpuinit rcu_spawn_one_boostreturn 0 ; } -#ifdef CONFIG_HOTPLUG_CPU - -/* -* Stop the RCU's per-CPU kthread when its CPU goes offline,. -*/ -static void rcu_stop_cpu_kthread (int cpu) -{ -struct task_struct *t; - -/* Stop the CPU's kthread. */ - t =per_cpu (rcu_cpu_kthread_task, cpu); -if (t !=NULL ) { -per_cpu (rcu_cpu_kthread_task, cpu) =NULL ; -kthread_stop (t); - } -} - -#endif /* #ifdef CONFIG_HOTPLUG_CPU */ -static void rcu_kthread_do_work (void ) {rcu_do_batch (&rcu_sched_state, &__get_cpu_var (rcu_sched_data));@@ -1466,59 +1448,22 @@ static void rcu_kthread_do_work(void)rcu_preempt_do_callbacks (); } -/* -* Set the specified CPU's kthread to run RT or not, as specified by -* the to_rt argument. The CPU-hotplug locks are held, so the task -* is not going away. -*/ -static void rcu_cpu_kthread_setrt (int cpu,int to_rt) +static void rcu_cpu_kthread_setup (unsigned int cpu) { -int policy;struct sched_param sp; -struct task_struct *t; - t =per_cpu (rcu_cpu_kthread_task, cpu); -if (t ==NULL ) -return ; -if (to_rt) { - policy =SCHED_FIFO ; - sp.sched_priority =RCU_KTHREAD_PRIO ; - }else { - policy =SCHED_NORMAL ; - sp.sched_priority =0 ; - } -sched_setscheduler_nocheck (t, policy, &sp); + sp.sched_priority =RCU_KTHREAD_PRIO ; +sched_setscheduler_nocheck (current,SCHED_FIFO , &sp); } -/* -* Handle cases where the rcu_cpu_kthread() ends up on the wrong CPU. -* This can happen while the corresponding CPU is either coming online -* or going offline. We cannot wait until the CPU is fully online -* before starting the kthread, because the various notifier functions -* can wait for RCU grace periods. So we park rcu_cpu_kthread() until -* the corresponding CPU is online. -* -* Return 1 if the kthread needs to stop, 0 otherwise. -* -* Caller must disable bh. This function can momentarily enable it. -*/ -static int rcu_cpu_kthread_should_stop (int cpu) +static void rcu_cpu_kthread_park (unsigned int cpu) { -while (cpu_is_offline (cpu) || - !cpumask_equal (¤t->cpus_allowed,cpumask_of (cpu)) || -smp_processor_id () != cpu) { -if (kthread_should_stop ()) -return 1 ; -per_cpu (rcu_cpu_kthread_status, cpu) =RCU_KTHREAD_OFFCPU ; -per_cpu (rcu_cpu_kthread_cpu, cpu) =raw_smp_processor_id (); -local_bh_enable (); -schedule_timeout_uninterruptible (1 ); -if (!cpumask_equal (¤t->cpus_allowed,cpumask_of (cpu))) -set_cpus_allowed_ptr (current,cpumask_of (cpu)); -local_bh_disable (); - } -per_cpu (rcu_cpu_kthread_cpu, cpu) = cpu; -return 0 ; +per_cpu (rcu_cpu_kthread_status, cpu) =RCU_KTHREAD_OFFCPU ; +} + +static int rcu_cpu_kthread_should_run (unsigned int cpu) +{ +return __get_cpu_var (rcu_cpu_has_work); }/* @@ -1526,96 +1471,35 @@ static int rcu_cpu_kthread_should_stop(i* RCU softirq used in flavors and configurations of RCU that do not * support RCU priority boosting. */ -static int rcu_cpu_kthread (void *arg) +static void rcu_cpu_kthread (unsigned int cpu) { -int cpu = (int )(long )arg; -unsigned long flags; -int spincnt =0 ; -unsigned int *statusp = &per_cpu (rcu_cpu_kthread_status, cpu); -char work; -char *workp = &per_cpu (rcu_cpu_has_work, cpu); +unsigned int *statusp = &__get_cpu_var (rcu_cpu_kthread_status); +char work, *workp = &__get_cpu_var (rcu_cpu_has_work); +int spincnt; -trace_rcu_utilization ("Start CPU kthread@init" ); -for (;;) { - *statusp =RCU_KTHREAD_WAITING ; -trace_rcu_utilization ("End CPU kthread@rcu_wait" ); -rcu_wait (*workp !=0 ||kthread_should_stop ()); +for (spincnt =0 ; spincnt <10 ; spincnt++) {trace_rcu_utilization ("Start CPU kthread@rcu_wait" );local_bh_disable (); -if (rcu_cpu_kthread_should_stop (cpu)) { -local_bh_enable (); -break ; - } *statusp =RCU_KTHREAD_RUNNING ; -per_cpu (rcu_cpu_kthread_loops, cpu)++; -local_irq_save (flags); +this_cpu_inc (rcu_cpu_kthread_loops); +local_irq_disable (); work = *workp; *workp =0 ; -local_irq_restore (flags); +local_irq_enable ();if (work)rcu_kthread_do_work ();local_bh_enable (); -if (*workp !=0 ) - spincnt++; -else - spincnt =0 ; -if (spincnt >10 ) { - *statusp =RCU_KTHREAD_YIELDING ; -trace_rcu_utilization ("End CPU kthread@rcu_yield" ); -schedule_timeout_interruptible (2 ); -trace_rcu_utilization ("Start CPU kthread@rcu_yield" ); - spincnt =0 ; +if (*workp ==0 ) { +trace_rcu_utilization ("End CPU kthread@rcu_wait" ); + *statusp =RCU_KTHREAD_WAITING ; +return ; } } - *statusp =RCU_KTHREAD_STOPPED ; -trace_rcu_utilization ("End CPU kthread@term" ); -return 0 ; -} - -/* -* Spawn a per-CPU kthread, setting up affinity and priority. -* Because the CPU hotplug lock is held, no other CPU will be attempting -* to manipulate rcu_cpu_kthread_task. There might be another CPU -* attempting to access it during boot, but the locking in kthread_bind() -* will enforce sufficient ordering. -* -* Please note that we cannot simply refuse to wake up the per-CPU -* kthread because kthreads are created in TASK_UNINTERRUPTIBLE state, -* which can result in softlockup complaints if the task ends up being -* idle for more than a couple of minutes. -* -* However, please note also that we cannot bind the per-CPU kthread to its -* CPU until that CPU is fully online. We also cannot wait until the -* CPU is fully online before we create its per-CPU kthread, as this would -* deadlock the system when CPU notifiers tried waiting for grace -* periods. So we bind the per-CPU kthread to its CPU only if the CPU -* is online. If its CPU is not yet fully online, then the code in -* rcu_cpu_kthread() will wait until it is fully online, and then do -* the binding. -*/ -static int __cpuinit rcu_spawn_one_cpu_kthread (int cpu) -{ -struct sched_param sp; -struct task_struct *t; - -if (!rcu_scheduler_fully_active || -per_cpu (rcu_cpu_kthread_task, cpu) !=NULL ) -return 0 ; - t =kthread_create_on_node (rcu_cpu_kthread, - (void *)(long )cpu, -cpu_to_node (cpu), -"rcuc/%d" , cpu); -if (IS_ERR (t)) -return PTR_ERR (t); -if (cpu_online (cpu)) -kthread_bind (t, cpu); -per_cpu (rcu_cpu_kthread_cpu, cpu) = cpu; -WARN_ON_ONCE (per_cpu (rcu_cpu_kthread_task, cpu) !=NULL ); - sp.sched_priority =RCU_KTHREAD_PRIO ; -sched_setscheduler_nocheck (t,SCHED_FIFO , &sp); -per_cpu (rcu_cpu_kthread_task, cpu) = t; -wake_up_process (t);/* Get to TASK_INTERRUPTIBLE quickly. */ -return 0 ; + *statusp =RCU_KTHREAD_YIELDING ; +trace_rcu_utilization ("Start CPU kthread@rcu_yield" ); +schedule_timeout_interruptible (2 ); +trace_rcu_utilization ("End CPU kthread@rcu_yield" ); + *statusp =RCU_KTHREAD_WAITING ; }/* @@ -1651,6 +1535,15 @@ static void rcu_boost_kthread_setaffinitfree_cpumask_var (cm); } +static struct smp_hotplug_thread rcu_cpu_thread_spec = { + .store = &rcu_cpu_kthread_task, + .thread_should_run = rcu_cpu_kthread_should_run, + .thread_fn = rcu_cpu_kthread, + .thread_comm ="rcuc/%u" , + .setup = rcu_cpu_kthread_setup, + .park = rcu_cpu_kthread_park, +}; +/* * Spawn all kthreads -- called as soon as the scheduler is running. */ @@ -1660,11 +1553,9 @@ static int __init rcu_spawn_kthreads(voiint cpu; rcu_scheduler_fully_active =1 ; -for_each_possible_cpu (cpu) { +for_each_possible_cpu (cpu)per_cpu (rcu_cpu_has_work, cpu) =0 ; -if (cpu_online (cpu)) - (void )rcu_spawn_one_cpu_kthread (cpu); - } +BUG_ON (smpboot_register_percpu_thread (&rcu_cpu_thread_spec)); rnp =rcu_get_root (rcu_state); (void )rcu_spawn_one_boost_kthread (rcu_state, rnp);if (NUM_RCU_NODES >1 ) {@@ -1681,10 +1572,8 @@ static void __cpuinit rcu_prepare_kthreastruct rcu_node *rnp = rdp->mynode;/* Fire up the incoming CPU's kthread and leaf rcu_node kthread. */ -if (rcu_scheduler_fully_active) { - (void )rcu_spawn_one_cpu_kthread (cpu); +if (rcu_scheduler_fully_active) (void)rcu_spawn_one_boost_kthread(rcu_state, rnp); - } }#else /* #ifdef CONFIG_RCU_BOOST */ @@ -1708,22 +1597,10 @@ static void rcu_preempt_boost_start_gp(s { } -#ifdef CONFIG_HOTPLUG_CPU - -static void rcu_stop_cpu_kthread (int cpu) -{ -} - -#endif /* #ifdef CONFIG_HOTPLUG_CPU */ -static void rcu_boost_kthread_setaffinity (struct rcu_node *rnp,int outgoingcpu) { } -static void rcu_cpu_kthread_setrt (int cpu,int to_rt) -{ -} -static int __init rcu_scheduler_really_started (void ) { rcu_scheduler_fully_active =1 ;Index:
tip/kernel/rcutree_trace.c
===================================================================kernel/rcutree_trace.c +1
-2@@ -83,11 +83,10 @@ static void print_one_rcu_data(struct se rdp->nxttail[RCU_WAIT_TAIL]], ".D"[&rdp->nxtlist != rdp->nxttail[RCU_DONE_TAIL]]);#ifdef CONFIG_RCU_BOOST -seq_printf (m," kt=%d/%c/%d ktl=%x" , +seq_printf (m," kt=%d/%c ktl=%x" ,per_cpu (rcu_cpu_has_work, rdp->cpu),convert_kthread_status (per_cpu (rcu_cpu_kthread_status, rdp->cpu)), -per_cpu (rcu_cpu_kthread_cpu, rdp->cpu),per_cpu (rcu_cpu_kthread_loops, rdp->cpu) &0xffff );#endif /* #ifdef CONFIG_RCU_BOOST */ seq_printf (m," b=%ld" , rdp->blimit);
[tip:smp/hotplug] rcu: Use smp_hotplug_thread facility for RCUs per-CPU kthread
In reply to Thomas GleixnerCommit-ID: 62ab7072476ae1600e877cc62b43758e485f4f1e Gitweb: http://git.kernel.org/tip/62ab7072476ae1600e877cc62b43758e485f4f1e Author: Paul E. McKenney <[email protected]> AuthorDate: Mon, 16 Jul 2012 10:42:38 +0000 Committer: Thomas Gleixner <[email protected]> CommitDate: Mon, 13 Aug 2012 17:01:08 +0200
rcu: Use smp_hotplug_thread facility for RCUs per-CPU kthread
Bring RCU into the new-age CPU-hotplug fold by modifying RCU's per-CPU kthread code to use the new smp_hotplug_thread facility.
[ tglx: Adapted it to use callbacks and to the simplified rcu yield ]
Signed-off-by Paul E. McKenney <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Srivatsa S. Bhat <[email protected]>
Cc Rusty Russell <[email protected]>
Cc Namhyung Kim <[email protected]>
Signed-off-by Thomas Gleixner <[email protected]>
--- kernel/rcutree.c | 4 - kernel/rcutree.h | 8 -- kernel/rcutree_plugin.h | 205 ++++++++++------------------------------------- kernel/rcutree_trace.c | 3 +- 4 files changed, 42 insertions(+), 178 deletions(-)
kernel/rcutree.c +0 -4
@@ -133,7 +133,6 @@ static int rcu_scheduler_fully_active __read_mostly;*/ static DEFINE_PER_CPU(struct task_struct *, rcu_cpu_kthread_task);DEFINE_PER_CPU (unsigned int , rcu_cpu_kthread_status); -DEFINE_PER_CPU (int, rcu_cpu_kthread_cpu); DEFINE_PER_CPU(unsigned int, rcu_cpu_kthread_loops);DEFINE_PER_CPU (char, rcu_cpu_has_work);
@@ -1468,7 +1467,6 @@ static void rcu_cleanup_dead_cpu(int cpu, struct rcu_state *rsp)struct rcu_node *rnp = rdp->mynode;/* Outgoing CPU's rdp & rnp. */ /* Adjust any no-longer-needed kthreads. */ -rcu_stop_cpu_kthread (cpu);rcu_boost_kthread_setaffinity (rnp,-1 );/* Remove the dead CPU from the bitmasks in the rcu_node hierarchy. */
@@ -2595,11 +2593,9 @@ static int __cpuinit rcu_cpu_notify(struct notifier_block *self,case CPU_ONLINE :case CPU_DOWN_FAILED :rcu_boost_kthread_setaffinity (rnp,-1 ); -rcu_cpu_kthread_setrt (cpu,1 );break ;case CPU_DOWN_PREPARE :rcu_boost_kthread_setaffinity (rnp, cpu); -rcu_cpu_kthread_setrt (cpu,0 );break ;case CPU_DYING :case CPU_DYING_FROZEN :
kernel/rcutree.h +0 -8
@@ -196,12 +196,6 @@ struct rcu_node {/* Refused to boost: not sure why, though. */ /* This can happen due to race conditions. */ #endif /* #ifdef CONFIG_RCU_BOOST */ -struct task_struct *node_kthread_task; -/* kthread that takes care of this rcu_node */ -/* structure, for example, awakening the */ -/* per-CPU kthreads as needed. */ -unsigned int node_kthread_status; -/* State of node_kthread_task for tracing. */ }____cacheline_internodealigned_in_smp ;/*
@@ -468,7 +462,6 @@ static int rcu_preempt_blocked_readers_cgp(struct rcu_node *rnp);#ifdef CONFIG_HOTPLUG_CPU static void rcu_report_unblock_qs_rnp (struct rcu_node *rnp,unsigned long flags); -static void rcu_stop_cpu_kthread (int cpu);#endif /* #ifdef CONFIG_HOTPLUG_CPU */ static void rcu_print_detail_task_stall (struct rcu_state *rsp);static int rcu_print_task_stall (struct rcu_node *rnp);
@@ -494,7 +487,6 @@ static void rcu_preempt_do_callbacks(void);static int __cpuinit rcu_spawn_one_boost_kthread (struct rcu_state *rsp,struct rcu_node *rnp);#endif /* #ifdef CONFIG_RCU_BOOST */ -static void rcu_cpu_kthread_setrt (int cpu,int to_rt);static void __cpuinit rcu_prepare_kthreads (int cpu);static void rcu_prepare_for_idle_init (int cpu);static void rcu_cleanup_after_idle (int cpu);
kernel/rcutree_plugin.h +40 -163
275 lines not on this page · raw message
kernel/rcutree_trace.c +1 -2
@@ -108,11 +108,10 @@ static void print_one_rcu_data(struct seq_file *m, struct rcu_data *rdp) rdp->nxttail[RCU_WAIT_TAIL]], ".D"[&rdp->nxtlist != rdp->nxttail[RCU_DONE_TAIL]]);#ifdef CONFIG_RCU_BOOST -seq_printf (m," kt=%d/%c/%d ktl=%x" , +seq_printf (m," kt=%d/%c ktl=%x" ,per_cpu (rcu_cpu_has_work, rdp->cpu),convert_kthread_status (per_cpu (rcu_cpu_kthread_status, rdp->cpu)), -per_cpu (rcu_cpu_kthread_cpu, rdp->cpu),per_cpu (rcu_cpu_kthread_loops, rdp->cpu) &0xffff );#endif /* #ifdef CONFIG_RCU_BOOST */ seq_printf (m," b=%ld" , rdp->blimit);
PATCH 7/7 infiniband: ehca: Use hotplug thread infrastructure
In reply to Thomas GleixnerGet rid of the hotplug notifiers and use the generic hotplug thread infrastructure.
Signed-off-by Thomas Gleixner <[email protected]>
--- drivers/infiniband/hw/ehca/ehca_irq.c | 255 ++++++++++++---------------------- drivers/infiniband/hw/ehca/ehca_irq.h | 6 2 files changed, 96 insertions(+), 165 deletions(-)
drivers/infiniband/hw/ehca/ehca_irq.c +93 -162
@@ -42,6 +42,7 @@*/ #include <linux/slab.h> +#include <linux/smpboot.h> #include "ehca_classes.h" #include "ehca_irq.h"
@@ -652,7 +653,7 @@ void ehca_tasklet_eq(unsigned long data)ehca_process_eq ((struct ehca_shca *)data,1 ); } -static inline int find_next_online_cpu (struct ehca_comp_pool *pool) +static int find_next_online_cpu (struct ehca_comp_pool *pool) {int cpu;unsigned long flags;
@@ -662,17 +663,23 @@ static inline int find_next_online_cpu(sehca_dmp (cpu_online_mask,cpumask_size (),"" );spin_lock_irqsave (&pool->last_cpu_lock, flags); - cpu =cpumask_next (pool->last_cpu, cpu_online_mask); -if (cpu >= nr_cpu_ids) - cpu =cpumask_first (cpu_online_mask); - pool->last_cpu = cpu; +while (1 ) { + cpu =cpumask_next (pool->last_cpu, cpu_online_mask); +if (cpu >= nr_cpu_ids) + cpu =cpumask_first (cpu_online_mask); + pool->last_cpu = cpu; +/* Might be on the way out */ +if (per_cpu_ptr (pool->cpu_comp_tasks, cpu)->active) +break ; + }spin_unlock_irqrestore (&pool->last_cpu_lock, flags);return cpu; }static void __queue_comp_task (struct ehca_cq *__cq, -struct ehca_cpu_comp_task *cct) +struct ehca_cpu_comp_task *cct, +struct task_struct *thread) {unsigned long flags;
@@ -683,7 +690,7 @@ static void __queue_comp_task(struct ehc __cq->nr_callbacks++;list_add_tail (&__cq->entry, &cct->cq_list); cct->cq_jobs++; -wake_up (&cct->wait_queue); +wake_up_process (thread); }else __cq->nr_callbacks++;
@@ -695,6 +702,7 @@ static void queue_comp_task(struct ehca_ {int cpu_id;struct ehca_cpu_comp_task *cct; +struct task_struct *thread;int cq_jobs;unsigned long flags;
@@ -702,7 +710,8 @@ static void queue_comp_task(struct ehca_BUG_ON (!cpu_online (cpu_id)); cct =per_cpu_ptr (pool->cpu_comp_tasks, cpu_id); -BUG_ON (!cct); + thread =per_cpu_ptr (pool->cpu_comp_threads, cpu_id); +BUG_ON (!cct || !thread);spin_lock_irqsave (&cct->task_lock, flags); cq_jobs = cct->cq_jobs;
@@ -710,28 +719,25 @@ static void queue_comp_task(struct ehca_if (cq_jobs >0 ) { cpu_id =find_next_online_cpu (pool); cct =per_cpu_ptr (pool->cpu_comp_tasks, cpu_id); -BUG_ON (!cct); + thread =per_cpu_ptr (pool->cpu_comp_threads, cpu_id); +BUG_ON (!cct || !thread); } - -__queue_comp_task (__cq, cct); +__queue_comp_task (__cq, cct, thread); }static void run_comp_task (struct ehca_cpu_comp_task *cct) {struct ehca_cq *cq; -unsigned long flags; - -spin_lock_irqsave (&cct->task_lock, flags);while (!list_empty (&cct->cq_list)) { cq = list_entry(cct->cq_list.next, struct ehca_cq, entry); -spin_unlock_irqrestore (&cct->task_lock, flags); +spin_unlock_irq (&cct->task_lock);comp_event_callback (cq);if (atomic_dec_and_test (&cq->nr_events))wake_up (&cq->wait_completion); -spin_lock_irqsave (&cct->task_lock, flags); +spin_lock_irq (&cct->task_lock);spin_lock (&cq->task_lock); cq->nr_callbacks--;if (!cq->nr_callbacks) {
@@ -740,159 +746,76 @@ static void run_comp_task(struct ehca_cp }spin_unlock (&cq->task_lock); } - -spin_unlock_irqrestore (&cct->task_lock, flags); } -static int comp_task (void *__cct) +static void comp_task_park (unsigned int cpu) { -struct ehca_cpu_comp_task *cct = __cct; -int cql_empty; -DECLARE_WAITQUEUE (wait, current); - -set_current_state (TASK_INTERRUPTIBLE ); -while (!kthread_should_stop ()) { -add_wait_queue (&cct->wait_queue, &wait); - -spin_lock_irq (&cct->task_lock); - cql_empty =list_empty (&cct->cq_list); -spin_unlock_irq (&cct->task_lock); -if (cql_empty) -schedule (); -else -__set_current_state (TASK_RUNNING ); - -remove_wait_queue (&cct->wait_queue, &wait); +struct ehca_cpu_comp_task *cct =per_cpu_ptr (pool->cpu_comp_tasks, cpu); +struct ehca_cpu_comp_task *target; +struct task_struct *thread; +struct ehca_cq *cq, *tmp; +LIST_HEAD (list); -spin_lock_irq (&cct->task_lock); - cql_empty =list_empty (&cct->cq_list); -spin_unlock_irq (&cct->task_lock); -if (!cql_empty) -run_comp_task (__cct); +spin_lock_irq (&cct->task_lock); + cct->cq_jobs =0 ; + cct->active =0 ; +list_splice_init (&cct->cq_list, &list); +spin_unlock_irq (&cct->task_lock); -set_current_state (TASK_INTERRUPTIBLE ); + cpu =find_next_online_cpu (pool); + target =per_cpu_ptr (pool->cpu_comp_tasks, cpu); + thread =per_cpu_ptr (pool->cpu_comp_threads, cpu); +spin_lock_irq (&target->task_lock); +list_for_each_entry_safe (cq, tmp, &list, entry) { +list_del (&cq->entry); +__queue_comp_task (cq, target, thread); } -__set_current_state (TASK_RUNNING ); - -return 0 ; -} - -static struct task_struct *create_comp_task (struct ehca_comp_pool *pool, -int cpu) -{ -struct ehca_cpu_comp_task *cct; - - cct =per_cpu_ptr (pool->cpu_comp_tasks, cpu); -spin_lock_init (&cct->task_lock); -INIT_LIST_HEAD (&cct->cq_list); -init_waitqueue_head (&cct->wait_queue); - cct->task =kthread_create_on_node (comp_task, cct,cpu_to_node (cpu), -"ehca_comp/%d" , cpu); - -return cct->task; +spin_unlock_irq (&target->task_lock); } -static void destroy_comp_task (struct ehca_comp_pool *pool, -int cpu) +static void comp_task_stop (unsigned int cpu,bool online) { -struct ehca_cpu_comp_task *cct; -struct task_struct *task; -unsigned long flags_cct; - - cct =per_cpu_ptr (pool->cpu_comp_tasks, cpu); - -spin_lock_irqsave (&cct->task_lock, flags_cct); +struct ehca_cpu_comp_task *cct =per_cpu_ptr (pool->cpu_comp_tasks, cpu); - task = cct->task; - cct->task =NULL ; +spin_lock_irq (&cct->task_lock); cct->cq_jobs =0 ; - -spin_unlock_irqrestore (&cct->task_lock, flags_cct); - -if (task) -kthread_stop (task); + cct->active =0 ; +WARN_ON (!list_empty (&cct->cq_list)); +spin_unlock_irq (&cct->task_lock); } -static void __cpuinit take_over_work (struct ehca_comp_pool *pool,int cpu) +static int comp_task_should_run (unsigned int cpu) {struct ehca_cpu_comp_task *cct =per_cpu_ptr (pool->cpu_comp_tasks, cpu); -LIST_HEAD (list); -struct ehca_cq *cq; -unsigned long flags_cct; - -spin_lock_irqsave (&cct->task_lock, flags_cct); - -list_splice_init (&cct->cq_list, &list); - -while (!list_empty (&list)) { - cq = list_entry(cct->cq_list.next, struct ehca_cq, entry); - -list_del (&cq->entry); -__queue_comp_task (cq,this_cpu_ptr (pool->cpu_comp_tasks)); - } - -spin_unlock_irqrestore (&cct->task_lock, flags_cct); +return cct->cq_jobs; } -static int __cpuinit comp_pool_callback (struct notifier_block *nfb, -unsigned long action, -void *hcpu) +static int comp_task (unsigned int cpu) { -unsigned int cpu = (unsigned long )hcpu; -struct ehca_cpu_comp_task *cct; - -switch (action) { -case CPU_UP_PREPARE : -case CPU_UP_PREPARE_FROZEN : -ehca_gen_dbg ("CPU: %x (CPU_PREPARE)" , cpu); -if (!create_comp_task (pool, cpu)) { -ehca_gen_err ("Can't create comp_task for cpu: %x" , cpu); -return notifier_from_errno (-ENOMEM ); - } -break ; -case CPU_UP_CANCELED : -case CPU_UP_CANCELED_FROZEN : -ehca_gen_dbg ("CPU: %x (CPU_CANCELED)" , cpu); - cct =per_cpu_ptr (pool->cpu_comp_tasks, cpu); -kthread_bind (cct->task,cpumask_any (cpu_online_mask)); -destroy_comp_task (pool, cpu); -break ; -case CPU_ONLINE : -case CPU_ONLINE_FROZEN : -ehca_gen_dbg ("CPU: %x (CPU_ONLINE)" , cpu); - cct =per_cpu_ptr (pool->cpu_comp_tasks, cpu); -kthread_bind (cct->task, cpu); -wake_up_process (cct->task); -break ; -case CPU_DOWN_PREPARE : -case CPU_DOWN_PREPARE_FROZEN : -ehca_gen_dbg ("CPU: %x (CPU_DOWN_PREPARE)" , cpu); -break ; -case CPU_DOWN_FAILED : -case CPU_DOWN_FAILED_FROZEN : -ehca_gen_dbg ("CPU: %x (CPU_DOWN_FAILED)" , cpu); -break ; -case CPU_DEAD : -case CPU_DEAD_FROZEN : -ehca_gen_dbg ("CPU: %x (CPU_DEAD)" , cpu); -destroy_comp_task (pool, cpu); -take_over_work (pool, cpu); -break ; - } +struct ehca_cpu_comp_task *cct =this_cpu_ptr (pool->cpu_comp_tasks); +int cql_empty; -return NOTIFY_OK ; +spin_lock_irq (&cct->task_lock); + cql_empty =list_empty (&cct->cq_list); +if (!cql_empty) { +__set_current_state (TASK_RUNNING ); +run_comp_task (cct); + } +spin_unlock_irq (&cct->task_lock); } -static struct notifier_block comp_pool_callback_nb__cpuinitdata = { - .notifier_call = comp_pool_callback, - .priority =0 , +static struct smp_hotplug_thread comp_pool_threads = { + .thread_should_run = comp_task_should_run, + .thread_fn = comp_task, + .thread_comm ="ehca_comp/%u" , + .cleanup = comp_task_stop, + .park = comp_task_park, };int ehca_create_comp_pool (void ) { -int cpu; -struct task_struct *task; +int cpu, ret = -ENOMEM ;if (!ehca_scaling_code)return 0 ;
@@ -905,38 +828,46 @@ int ehca_create_comp_pool(void) pool->last_cpu =cpumask_any (cpu_online_mask); pool->cpu_comp_tasks = alloc_percpu(struct ehca_cpu_comp_task); -if (pool->cpu_comp_tasks ==NULL ) { -kfree (pool); -return -EINVAL ; - } +if (!pool->cpu_comp_tasks) + goto out_pool; -for_each_online_cpu (cpu) { - task =create_comp_task (pool, cpu); -if (task) { -kthread_bind (task, cpu); -wake_up_process (task); - } + pool->cpu_comp_threads = alloc_percpu(struct task_struct *); +if (!pool->cpu_comp_threads) + goto out_tasks; + +for_each_present_cpu (cpu) { +struct ehca_cpu_comp_task *cct; + + cct =per_cpu_ptr (pool->cpu_comp_tasks, cpu); +spin_lock_init (&cct->task_lock); +INIT_LIST_HEAD (&cct->cq_list); } -register_hotcpu_notifier (&comp_pool_callback_nb); + comp_pool_threads.store = pool->cpu_comp_threads; + ret =smpboot_register_percpu_thread (&comp_pool_threads); +if (ret) + goto out_threads; -printk (KERN_INFO "eHCA scaling code enabled\n" ); +pr_info ("eHCA scaling code enabled\n" ); +return ret; -return 0 ; +out_threads: +free_percpu (pool->cpu_comp_threads); +out_tasks: +free_percpu (pool->cpu_comp_tasks); +out_pool: +kfree (pool); +return ret; }void ehca_destroy_comp_pool (void ) { -int i; -if (!ehca_scaling_code)return ; -unregister_hotcpu_notifier (&comp_pool_callback_nb); - -for_each_online_cpu (i) -destroy_comp_task (pool, i); +smpboot_unregister_percpu_thread (&comp_pool_threads); +free_percpu (pool->cpu_comp_threads);free_percpu (pool->cpu_comp_tasks);kfree (pool); }
drivers/infiniband/hw/ehca/ehca_irq.h +3 -3
@@ -58,15 +58,15 @@ void ehca_tasklet_eq(unsigned long data)void ehca_process_eq (struct ehca_shca *shca,int is_irq);struct ehca_cpu_comp_task { -wait_queue_head_t wait_queue;struct list_head cq_list; -struct task_struct *task;spinlock_t task_lock;int cq_jobs; +int active; };struct ehca_comp_pool { -struct ehca_cpu_comp_task *cpu_comp_tasks; +struct ehca_cpu_comp_task __percpu *cpu_comp_tasks; +struct task_struct *__percpu *cpu_comp_threads;int last_cpu;spinlock_t last_cpu_lock; };
Re: [Patch 7/7] infiniband: ehca: Use hotplug thread infrastructure
drivers/infiniband/hw/ehca/ehca_irq.c static inline int find_next_online_cpu(s · line 674
Quoted 9 lines · Thomas Gleixner
drivers/infiniband/hw/ehca/ehca_irq.c +9
-0+while (1 ) { + cpu =cpumask_next (pool->last_cpu, cpu_online_mask); +if (cpu >= nr_cpu_ids) + cpu =cpumask_first (cpu_online_mask); + pool->last_cpu = cpu; +/* Might be on the way out */ +if (per_cpu_ptr (pool->cpu_comp_tasks, cpu)->active) +break ; + }
I always look twice at while(1) loops; I'm sure there's a do-while hiding in there.
Overall, the patch series looks like a very nice cleanup.
Thanks!
Rusty.
[tip:smp/hotplug] infiniband: Ehca: Use hotplug thread infrastructure
In reply to Thomas GleixnerCommit-ID: 81942621bd6b70b1a1ac4692b3f8f3be65a91b44 Gitweb: http://git.kernel.org/tip/81942621bd6b70b1a1ac4692b3f8f3be65a91b44 Author: Thomas Gleixner <[email protected]> AuthorDate: Mon, 16 Jul 2012 10:42:39 +0000 Committer: Thomas Gleixner <[email protected]> CommitDate: Mon, 13 Aug 2012 17:01:08 +0200
infiniband: Ehca: Use hotplug thread infrastructure
Get rid of the hotplug notifiers and use the generic hotplug thread infrastructure.
Signed-off-by Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Srivatsa S. Bhat <[email protected]>
Cc Rusty Russell <[email protected]>
Reviewed-by Paul E. McKenney <[email protected]>
Cc Namhyung Kim <[email protected]>
Signed-off-by Thomas Gleixner <[email protected]>
--- drivers/infiniband/hw/ehca/ehca_irq.c | 250 ++++++++++++--------------------- drivers/infiniband/hw/ehca/ehca_irq.h | 6 +- 2 files changed, 92 insertions(+), 164 deletions(-)
drivers/infiniband/hw/ehca/ehca_irq.c +89 -161
@@ -42,6 +42,7 @@*/ #include <linux/slab.h> +#include <linux/smpboot.h> #include "ehca_classes.h" #include "ehca_irq.h"
@@ -652,7 +653,7 @@ void ehca_tasklet_eq(unsigned long data)ehca_process_eq ((struct ehca_shca *)data,1 ); } -static inline int find_next_online_cpu (struct ehca_comp_pool *pool) +static int find_next_online_cpu (struct ehca_comp_pool *pool) {int cpu;unsigned long flags;
@@ -662,17 +663,20 @@ static inline int find_next_online_cpu(struct ehca_comp_pool *pool)ehca_dmp (cpu_online_mask,cpumask_size (),"" );spin_lock_irqsave (&pool->last_cpu_lock, flags); - cpu =cpumask_next (pool->last_cpu, cpu_online_mask); -if (cpu >= nr_cpu_ids) - cpu =cpumask_first (cpu_online_mask); - pool->last_cpu = cpu; +do { + cpu =cpumask_next (pool->last_cpu, cpu_online_mask); +if (cpu >= nr_cpu_ids) + cpu =cpumask_first (cpu_online_mask); + pool->last_cpu = cpu; + } while (!per_cpu_ptr(pool->cpu_comp_tasks, cpu)->active)spin_unlock_irqrestore (&pool->last_cpu_lock, flags);return cpu; }static void __queue_comp_task (struct ehca_cq *__cq, -struct ehca_cpu_comp_task *cct) +struct ehca_cpu_comp_task *cct, +struct task_struct *thread) {unsigned long flags;
@@ -683,7 +687,7 @@ static void __queue_comp_task(struct ehca_cq *__cq, __cq->nr_callbacks++;list_add_tail (&__cq->entry, &cct->cq_list); cct->cq_jobs++; -wake_up (&cct->wait_queue); +wake_up_process (thread); }else __cq->nr_callbacks++;
@@ -695,6 +699,7 @@ static void queue_comp_task(struct ehca_cq *__cq) {int cpu_id;struct ehca_cpu_comp_task *cct; +struct task_struct *thread;int cq_jobs;unsigned long flags;
@@ -702,7 +707,8 @@ static void queue_comp_task(struct ehca_cq *__cq)BUG_ON (!cpu_online (cpu_id)); cct =per_cpu_ptr (pool->cpu_comp_tasks, cpu_id); -BUG_ON (!cct); + thread =per_cpu_ptr (pool->cpu_comp_threads, cpu_id); +BUG_ON (!cct || !thread);spin_lock_irqsave (&cct->task_lock, flags); cq_jobs = cct->cq_jobs;
@@ -710,28 +716,25 @@ static void queue_comp_task(struct ehca_cq *__cq)if (cq_jobs >0 ) { cpu_id =find_next_online_cpu (pool); cct =per_cpu_ptr (pool->cpu_comp_tasks, cpu_id); -BUG_ON (!cct); + thread =per_cpu_ptr (pool->cpu_comp_threads, cpu_id); +BUG_ON (!cct || !thread); } - -__queue_comp_task (__cq, cct); +__queue_comp_task (__cq, cct, thread); }static void run_comp_task (struct ehca_cpu_comp_task *cct) {struct ehca_cq *cq; -unsigned long flags; - -spin_lock_irqsave (&cct->task_lock, flags);while (!list_empty (&cct->cq_list)) { cq = list_entry(cct->cq_list.next, struct ehca_cq, entry); -spin_unlock_irqrestore (&cct->task_lock, flags); +spin_unlock_irq (&cct->task_lock);comp_event_callback (cq);if (atomic_dec_and_test (&cq->nr_events))wake_up (&cq->wait_completion); -spin_lock_irqsave (&cct->task_lock, flags); +spin_lock_irq (&cct->task_lock);spin_lock (&cq->task_lock); cq->nr_callbacks--;if (!cq->nr_callbacks) {
@@ -740,159 +743,76 @@ static void run_comp_task(struct ehca_cpu_comp_task *cct) }spin_unlock (&cq->task_lock); } - -spin_unlock_irqrestore (&cct->task_lock, flags); } -static int comp_task (void *__cct) +static void comp_task_park (unsigned int cpu) { -struct ehca_cpu_comp_task *cct = __cct; -int cql_empty; -DECLARE_WAITQUEUE (wait, current); - -set_current_state (TASK_INTERRUPTIBLE ); -while (!kthread_should_stop ()) { -add_wait_queue (&cct->wait_queue, &wait); - -spin_lock_irq (&cct->task_lock); - cql_empty =list_empty (&cct->cq_list); -spin_unlock_irq (&cct->task_lock); -if (cql_empty) -schedule (); -else -__set_current_state (TASK_RUNNING ); - -remove_wait_queue (&cct->wait_queue, &wait); +struct ehca_cpu_comp_task *cct =per_cpu_ptr (pool->cpu_comp_tasks, cpu); +struct ehca_cpu_comp_task *target; +struct task_struct *thread; +struct ehca_cq *cq, *tmp; +LIST_HEAD (list); -spin_lock_irq (&cct->task_lock); - cql_empty =list_empty (&cct->cq_list); -spin_unlock_irq (&cct->task_lock); -if (!cql_empty) -run_comp_task (__cct); +spin_lock_irq (&cct->task_lock); + cct->cq_jobs =0 ; + cct->active =0 ; +list_splice_init (&cct->cq_list, &list); +spin_unlock_irq (&cct->task_lock); -set_current_state (TASK_INTERRUPTIBLE ); + cpu =find_next_online_cpu (pool); + target =per_cpu_ptr (pool->cpu_comp_tasks, cpu); + thread =per_cpu_ptr (pool->cpu_comp_threads, cpu); +spin_lock_irq (&target->task_lock); +list_for_each_entry_safe (cq, tmp, &list, entry) { +list_del (&cq->entry); +__queue_comp_task (cq, target, thread); } -__set_current_state (TASK_RUNNING ); - -return 0 ; -} - -static struct task_struct *create_comp_task (struct ehca_comp_pool *pool, -int cpu) -{ -struct ehca_cpu_comp_task *cct; - - cct =per_cpu_ptr (pool->cpu_comp_tasks, cpu); -spin_lock_init (&cct->task_lock); -INIT_LIST_HEAD (&cct->cq_list); -init_waitqueue_head (&cct->wait_queue); - cct->task =kthread_create_on_node (comp_task, cct,cpu_to_node (cpu), -"ehca_comp/%d" , cpu); - -return cct->task; +spin_unlock_irq (&target->task_lock); } -static void destroy_comp_task (struct ehca_comp_pool *pool, -int cpu) +static void comp_task_stop (unsigned int cpu,bool online) { -struct ehca_cpu_comp_task *cct; -struct task_struct *task; -unsigned long flags_cct; - - cct =per_cpu_ptr (pool->cpu_comp_tasks, cpu); - -spin_lock_irqsave (&cct->task_lock, flags_cct); +struct ehca_cpu_comp_task *cct =per_cpu_ptr (pool->cpu_comp_tasks, cpu); - task = cct->task; - cct->task =NULL ; +spin_lock_irq (&cct->task_lock); cct->cq_jobs =0 ; - -spin_unlock_irqrestore (&cct->task_lock, flags_cct); - -if (task) -kthread_stop (task); + cct->active =0 ; +WARN_ON (!list_empty (&cct->cq_list)); +spin_unlock_irq (&cct->task_lock); } -static void __cpuinit take_over_work (struct ehca_comp_pool *pool,int cpu) +static int comp_task_should_run (unsigned int cpu) {struct ehca_cpu_comp_task *cct =per_cpu_ptr (pool->cpu_comp_tasks, cpu); -LIST_HEAD (list); -struct ehca_cq *cq; -unsigned long flags_cct; - -spin_lock_irqsave (&cct->task_lock, flags_cct); - -list_splice_init (&cct->cq_list, &list); - -while (!list_empty (&list)) { - cq = list_entry(cct->cq_list.next, struct ehca_cq, entry); - -list_del (&cq->entry); -__queue_comp_task (cq,this_cpu_ptr (pool->cpu_comp_tasks)); - } - -spin_unlock_irqrestore (&cct->task_lock, flags_cct); +return cct->cq_jobs; } -static int __cpuinit comp_pool_callback (struct notifier_block *nfb, -unsigned long action, -void *hcpu) +static int comp_task (unsigned int cpu) { -unsigned int cpu = (unsigned long )hcpu; -struct ehca_cpu_comp_task *cct; +struct ehca_cpu_comp_task *cct =this_cpu_ptr (pool->cpu_comp_tasks); +int cql_empty; -switch (action) { -case CPU_UP_PREPARE : -case CPU_UP_PREPARE_FROZEN : -ehca_gen_dbg ("CPU: %x (CPU_PREPARE)" , cpu); -if (!create_comp_task (pool, cpu)) { -ehca_gen_err ("Can't create comp_task for cpu: %x" , cpu); -return notifier_from_errno (-ENOMEM ); - } -break ; -case CPU_UP_CANCELED : -case CPU_UP_CANCELED_FROZEN : -ehca_gen_dbg ("CPU: %x (CPU_CANCELED)" , cpu); - cct =per_cpu_ptr (pool->cpu_comp_tasks, cpu); -kthread_bind (cct->task,cpumask_any (cpu_online_mask)); -destroy_comp_task (pool, cpu); -break ; -case CPU_ONLINE : -case CPU_ONLINE_FROZEN : -ehca_gen_dbg ("CPU: %x (CPU_ONLINE)" , cpu); - cct =per_cpu_ptr (pool->cpu_comp_tasks, cpu); -kthread_bind (cct->task, cpu); -wake_up_process (cct->task); -break ; -case CPU_DOWN_PREPARE : -case CPU_DOWN_PREPARE_FROZEN : -ehca_gen_dbg ("CPU: %x (CPU_DOWN_PREPARE)" , cpu); -break ; -case CPU_DOWN_FAILED : -case CPU_DOWN_FAILED_FROZEN : -ehca_gen_dbg ("CPU: %x (CPU_DOWN_FAILED)" , cpu); -break ; -case CPU_DEAD : -case CPU_DEAD_FROZEN : -ehca_gen_dbg ("CPU: %x (CPU_DEAD)" , cpu); -destroy_comp_task (pool, cpu); -take_over_work (pool, cpu); -break ; +spin_lock_irq (&cct->task_lock); + cql_empty =list_empty (&cct->cq_list); +if (!cql_empty) { +__set_current_state (TASK_RUNNING ); +run_comp_task (cct); } - -return NOTIFY_OK ; +spin_unlock_irq (&cct->task_lock); } -static struct notifier_block comp_pool_callback_nb__cpuinitdata = { - .notifier_call = comp_pool_callback, - .priority =0 , +static struct smp_hotplug_thread comp_pool_threads = { + .thread_should_run = comp_task_should_run, + .thread_fn = comp_task, + .thread_comm ="ehca_comp/%u" , + .cleanup = comp_task_stop, + .park = comp_task_park, };int ehca_create_comp_pool (void ) { -int cpu; -struct task_struct *task; +int cpu, ret = -ENOMEM ;if (!ehca_scaling_code)return 0 ;
@@ -905,38 +825,46 @@ int ehca_create_comp_pool(void) pool->last_cpu =cpumask_any (cpu_online_mask); pool->cpu_comp_tasks = alloc_percpu(struct ehca_cpu_comp_task); -if (pool->cpu_comp_tasks ==NULL ) { -kfree (pool); -return -EINVAL ; - } +if (!pool->cpu_comp_tasks) + goto out_pool; -for_each_online_cpu (cpu) { - task =create_comp_task (pool, cpu); -if (task) { -kthread_bind (task, cpu); -wake_up_process (task); - } + pool->cpu_comp_threads = alloc_percpu(struct task_struct *); +if (!pool->cpu_comp_threads) + goto out_tasks; + +for_each_present_cpu (cpu) { +struct ehca_cpu_comp_task *cct; + + cct =per_cpu_ptr (pool->cpu_comp_tasks, cpu); +spin_lock_init (&cct->task_lock); +INIT_LIST_HEAD (&cct->cq_list); } -register_hotcpu_notifier (&comp_pool_callback_nb); + comp_pool_threads.store = pool->cpu_comp_threads; + ret =smpboot_register_percpu_thread (&comp_pool_threads); +if (ret) + goto out_threads; -printk (KERN_INFO "eHCA scaling code enabled\n" ); +pr_info ("eHCA scaling code enabled\n" ); +return ret; -return 0 ; +out_threads: +free_percpu (pool->cpu_comp_threads); +out_tasks: +free_percpu (pool->cpu_comp_tasks); +out_pool: +kfree (pool); +return ret; }void ehca_destroy_comp_pool (void ) { -int i; -if (!ehca_scaling_code)return ; -unregister_hotcpu_notifier (&comp_pool_callback_nb); - -for_each_online_cpu (i) -destroy_comp_task (pool, i); +smpboot_unregister_percpu_thread (&comp_pool_threads); +free_percpu (pool->cpu_comp_threads);free_percpu (pool->cpu_comp_tasks);kfree (pool); }
drivers/infiniband/hw/ehca/ehca_irq.h +3 -3
@@ -58,15 +58,15 @@ void ehca_tasklet_eq(unsigned long data);void ehca_process_eq (struct ehca_shca *shca,int is_irq);struct ehca_cpu_comp_task { -wait_queue_head_t wait_queue;struct list_head cq_list; -struct task_struct *task;spinlock_t task_lock;int cq_jobs; +int active; };struct ehca_comp_pool { -struct ehca_cpu_comp_task *cpu_comp_tasks; +struct ehca_cpu_comp_task __percpu *cpu_comp_tasks; +struct task_struct *__percpu *cpu_comp_threads;int last_cpu;spinlock_t last_cpu_lock; };
Quoted 10 lines · Thomas Gleixner
The following series implements the infrastructure for parking and unparking kernel threads to avoid the full teardown and fork on cpu hotplug operations along with management infrastructure for hotplug and users.
Changes vs. V2:
Use callbacks for all functionality. Thanks to Rusty for pointing that out. It makes the use sites nice and simple and keeps all the code which would be duplicated otherwise on the core.
Hello, Thomas,
What version should I apply this patchset to? I tried v3.5-rc7, but got lots of warnings (one shown below) and the watchdog patch did not apply.
Thanx, Paul
------------------------------------------------------------------------
[ 0.280196] BUG: scheduling while atomic: ksoftirqd/1/7/0x00000004 [ 0.280968] Modules linked in: [ 0.282064] Pid: 7, comm: ksoftirqd/1 Not tainted 3.5.0-rc7+ #1559 [ 0.283990] Call Trace: [ 0.284987] [<c16314f1>] __schedule_bug+0x49/0x55 [ 0.285975] [<c1638e1a>] __schedule+0x6ca/0x780 [ 0.286977] [<c109b556>] ? rcu_process_callbacks+0x66/0x130 [ 0.288983] [<c10362c4>] ? __local_bh_enable+0x24/0x70 [ 0.289975] [<c105f02b>] ? default_wake_function+0xb/0x10 [ 0.291972] [<c1638fbe>] schedule+0x1e/0x50 [ 0.292969] [<c1639202>] schedule_preempt_disabled+0x12/0x20 [ 0.294975] [<c1075ff8>] smpboot_thread_fn+0xe8/0x240 [ 0.295968] [<c1075f10>] ? smpboot_register_percpu_thread+0xc0/0xc0 [ 0.295969] [<c104fe5f>] kthread+0x8f/0xa0 [ 0.295969] [<c104fdd0>] ? flush_kthread_worker+0x80/0x80 [ 0.295969] [<c163ffb6>] kernel_thread_helper+0x6/0xd
Quoted 12 lines · Paul E. McKenney
Quoted 10 lines · Thomas Gleixner
The following series implements the infrastructure for parking and unparking kernel threads to avoid the full teardown and fork on cpu hotplug operations along with management infrastructure for hotplug and users.
Changes vs. V2:
Use callbacks for all functionality. Thanks to Rusty for pointing that out. It makes the use sites nice and simple and keeps all the code which would be duplicated otherwise on the core.
Quoted 6 lines
Hello, Thomas,
What version should I apply this patchset to? I tried v3.5-rc7, but got lots of warnings (one shown below) and the watchdog patch did not apply.
Hi Paul,
This patchset applies cleanly on Thomas' smp/hotplug branch in the -tip tree.
Regards,
Srivatsa S. Bhat
Quoted 20 lines · Srivatsa S. Bhat
Quoted 12 lines · Paul E. McKenney
Quoted 10 lines · Thomas Gleixner
The following series implements the infrastructure for parking and unparking kernel threads to avoid the full teardown and fork on cpu hotplug operations along with management infrastructure for hotplug and users.
Changes vs. V2:
Use callbacks for all functionality. Thanks to Rusty for pointing that out. It makes the use sites nice and simple and keeps all the code which would be duplicated otherwise on the core.
Quoted 6 lines
Hello, Thomas,
What version should I apply this patchset to? I tried v3.5-rc7, but got lots of warnings (one shown below) and the watchdog patch did not apply.
Quoted 4 lines
Hi Paul,
This patchset applies cleanly on Thomas' smp/hotplug branch in the -tip tree.
Thank you, Srivatsa, works much better. Still get "scheduling while atomic", looking into that.
I suppose I should have been able to guess where it was based... Ah, well...
Thanx, Paul
Quoted 21 lines · Paul E. McKenney
Quoted 20 lines · Srivatsa S. Bhat
Quoted 18 lines · Paul E. McKenney
Quoted 10 lines · Thomas Gleixner
The following series implements the infrastructure for parking and unparking kernel threads to avoid the full teardown and fork on cpu hotplug operations along with management infrastructure for hotplug and users.
Changes vs. V2:
Use callbacks for all functionality. Thanks to Rusty for pointing that out. It makes the use sites nice and simple and keeps all the code which would be duplicated otherwise on the core.
Hello, Thomas,
What version should I apply this patchset to? I tried v3.5-rc7, but got lots of warnings (one shown below) and the watchdog patch did not apply.
Quoted 8 lines
Quoted 4 lines
Hi Paul,
This patchset applies cleanly on Thomas' smp/hotplug branch in the -tip tree.
Thank you, Srivatsa, works much better. Still get "scheduling while atomic", looking into that.
Got a chance to run this patchset now.. Even I am getting "scheduling while atomic" messages like shown below.. Hmmm...
Regards,
Srivatsa S. Bhat
-----------------
BUG: sleeping function called from invalid context at kernel/softirq.c:748 in_atomic(): 1, irqs_disabled(): 0, pid: 3, name: ksoftirqd/0 INFO: lockdep is turned off. Pid: 3, comm: ksoftirqd/0 Tainted: G W 3.5.0-rc1-threadhotplugv3-debug #2 Call Trace: [<ffffffff8108e7ac>] __might_sleep+0x18c/0x250 [<ffffffff8105ea4f>] run_ksoftirqd+0x4f/0x70 [<ffffffff810c8b97>] smpboot_thread_fn+0x157/0x1f0 [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80 [<ffffffff8107e7ee>] kthread+0xee/0x100 [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10 [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13 [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8158eb70>] ? gs_change+0x13/0x13 BUG: scheduling while atomic: ksoftirqd/0/3/0x1000000b INFO: lockdep is turned off. Modules linked in: ext3 jbd mbcache sd_mod crc_t10dif qla2xxx scsi_transport_fc scsi_tgt mptsas mptscsih mptbase scsi_transport_sas [last unloaded: scsi_wait_scan] Pid: 3, comm: ksoftirqd/0 Tainted: G W 3.5.0-rc1-threadhotplugv3-debug #2 Call Trace: [<ffffffff8108d1cc>] __schedule_bug+0x6c/0x90 [<ffffffff81582818>] __schedule+0x7c8/0x880 [<ffffffff81092cea>] __cond_resched+0x2a/0x40 [<ffffffff81582960>] _cond_resched+0x30/0x40 SELinux: 2048 avtab hash slots, 250818 rules. [<ffffffff8105ea54>] run_ksoftirqd+0x54/0x70 [<ffffffff810c8b97>] smpboot_thread_fn+0x157/0x1f0 [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80 [<ffffffff8107e7ee>] kthread+0xee/0x100 [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10 [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13 [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8158eb70>] ? gs_change+0x13/0x13
[...]
BUG: sleeping function called from invalid context at kernel/softirq.c:748 in_atomic(): 1, irqs_disabled(): 0, pid: 37, name: ksoftirqd/8 INFO: lockdep is turned off. Pid: 37, comm: ksoftirqd/8 Tainted: G W 3.5.0-rc1-threadhotplugv3-debug #2 Call Trace: [<ffffffff8108e7ac>] __might_sleep+0x18c/0x250 [<ffffffff8105ea4f>] run_ksoftirqd+0x4f/0x70 [<ffffffff810c8b97>] smpboot_thread_fn+0x157/0x1f0 Welcome to [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80 [<ffffffff8107e7ee>] kthread+0xee/0x100 Red Hat [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10 [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13 [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8158eb70>] ? gs_change+0x13/0x13 BUG: scheduling while atomic: ksoftirqd/8/37/0x10000004 INFO: lockdep is turned off. Modules linked in: ext3 jbd mbcache sd_mod crc_t10dif qla2xxx scsi_transport_fc scsi_tgt mptsas mptscsih mptbase scsi_transport_sas [last unloaded: scsi_wait_scan] Pid: 37, comm: ksoftirqd/8 Tainted: G W 3.5.0-rc1-threadhotplugv3-debug #2 Call Trace: Enterprise Linu [<ffffffff8108d1cc>] __schedule_bug+0x6c/0x90 x Server [<ffffffff81582818>] __schedule+0x7c8/0x880 [<ffffffff81092cea>] __cond_resched+0x2a/0x40 [<ffffffff81582960>] _cond_resched+0x30/0x40 [<ffffffff8105ea54>] run_ksoftirqd+0x54/0x70 [<ffffffff810c8b97>] smpboot_thread_fn+0x157/0x1f0 [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80 [<ffffffff8107e7ee>] kthread+0xee/0x100 [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10 [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13 [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8158eb70>] ? gs_change+0x13/0x13
[...]
BUG: scheduling while atomic: watchdog/15/64/0x0000009b INFO: lockdep is turned off. Modules linked in: ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat xt_CHECKSUM iptable_mangle bridge stp llc autofs4 cpufreq_powersave sunrpc cpufreq_ondemand acpi_cpufreq freq_table mperf ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 dm_mirror dm_region_hash dm_log dm_round_robin scsi_dh_rdac dm_multipath vhost_net macvtap macvlan tun uinput cdc_ether usbnet mii serio_raw kvm_intel kvm microcode i2c_i801 i2c_core lpc_ich mfd_core shpchp ioatdma dca i7core_edac edac_core bnx2 dm_mod sg ext3 jbd mbcache sd_mod crc_t10dif qla2xxx scsi_transport_fc scsi_tgt mptsas mptscsih mptbase scsi_transport_sas [last unloaded: scsi_wait_scan] Pid: 64, comm: watchdog/15 Tainted: G W 3.5.0-rc1-threadhotplugv3-debug #2 Call Trace: [<ffffffff8108d1cc>] __schedule_bug+0x6c/0x90 [<ffffffff81582818>] __schedule+0x7c8/0x880 [<ffffffff81582bf9>] schedule+0x29/0x70 [<ffffffff81582f77>] schedule_preempt_disabled+0x27/0x40 [<ffffffff810c8b33>] smpboot_thread_fn+0xf3/0x1f0 [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80 [<ffffffff8107e7ee>] kthread+0xee/0x100 [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10 [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13 [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8158eb70>] ? gs_change+0x13/0x13
Quoted 27 lines · Srivatsa S. Bhat
Quoted 26 lines · Paul E. McKenney
Quoted 24 lines · Srivatsa S. Bhat
Quoted 12 lines · Paul E. McKenney
Quoted 10 lines · Thomas Gleixner
The following series implements the infrastructure for parking and unparking kernel threads to avoid the full teardown and fork on cpu hotplug operations along with management infrastructure for hotplug and users.
Changes vs. V2:
Use callbacks for all functionality. Thanks to Rusty for pointing that out. It makes the use sites nice and simple and keeps all the code which would be duplicated otherwise on the core.
Quoted 6 lines
Hello, Thomas,
What version should I apply this patchset to? I tried v3.5-rc7, but got lots of warnings (one shown below) and the watchdog patch did not apply.
Hi Paul,
This patchset applies cleanly on Thomas' smp/hotplug branch in the -tip tree.
Quoted 6 lines
Quoted 3 lines
Thank you, Srivatsa, works much better. Still get "scheduling while atomic", looking into that.
Got a chance to run this patchset now.. Even I am getting "scheduling while atomic" messages like shown below.. Hmmm...
Here is what little I have done so far (lots of completing demands on time this week, but I should have a goodly block of time to focus on this today):
1. The failure is from the softirq modifications. Reverting that
commit gets rid of the failures.
2. As one would expect, CONFIG_PREEMPT=n kernels do not have the
problem, which of course indicates a preempt_disable() imbalance.
3. I was unable to spot the problem by inspection, but this is not
too surprising given the high level of distraction this week.
4. Instrumentation shows that preempt_count() grows slowly with
time, but with the upper bits zero. This confirms the preempt_disable imbalance.
5. I am currently placing WARN_ONCE() calls in the code to track
this down. When I do find it, I fully expect to feel very stupid about my efforts on #3 above. ;-)
Thanx, Paul
Quoted 92 lines
Regards,
Srivatsa S. Bhat-----------------
BUG: sleeping function called from invalid context at kernel/softirq.c:748 in_atomic(): 1, irqs_disabled(): 0, pid: 3, name: ksoftirqd/0 INFO: lockdep is turned off. Pid: 3, comm: ksoftirqd/0 Tainted: G W 3.5.0-rc1-threadhotplugv3-debug #2 Call Trace: [<ffffffff8108e7ac>] __might_sleep+0x18c/0x250 [<ffffffff8105ea4f>] run_ksoftirqd+0x4f/0x70 [<ffffffff810c8b97>] smpboot_thread_fn+0x157/0x1f0 [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80 [<ffffffff8107e7ee>] kthread+0xee/0x100 [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10 [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13 [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8158eb70>] ? gs_change+0x13/0x13 BUG: scheduling while atomic: ksoftirqd/0/3/0x1000000b INFO: lockdep is turned off. Modules linked in: ext3 jbd mbcache sd_mod crc_t10dif qla2xxx scsi_transport_fc scsi_tgt mptsas mptscsih mptbase scsi_transport_sas [last unloaded: scsi_wait_scan] Pid: 3, comm: ksoftirqd/0 Tainted: G W 3.5.0-rc1-threadhotplugv3-debug #2 Call Trace: [<ffffffff8108d1cc>] __schedule_bug+0x6c/0x90 [<ffffffff81582818>] __schedule+0x7c8/0x880 [<ffffffff81092cea>] __cond_resched+0x2a/0x40 [<ffffffff81582960>] _cond_resched+0x30/0x40 SELinux: 2048 avtab hash slots, 250818 rules. [<ffffffff8105ea54>] run_ksoftirqd+0x54/0x70 [<ffffffff810c8b97>] smpboot_thread_fn+0x157/0x1f0 [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80 [<ffffffff8107e7ee>] kthread+0xee/0x100 [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10 [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13 [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8158eb70>] ? gs_change+0x13/0x13[...]
BUG: sleeping function called from invalid context at kernel/softirq.c:748 in_atomic(): 1, irqs_disabled(): 0, pid: 37, name: ksoftirqd/8 INFO: lockdep is turned off. Pid: 37, comm: ksoftirqd/8 Tainted: G W 3.5.0-rc1-threadhotplugv3-debug #2 Call Trace: [<ffffffff8108e7ac>] __might_sleep+0x18c/0x250 [<ffffffff8105ea4f>] run_ksoftirqd+0x4f/0x70 [<ffffffff810c8b97>] smpboot_thread_fn+0x157/0x1f0 Welcome to [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80 [<ffffffff8107e7ee>] kthread+0xee/0x100 Red Hat [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10 [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13 [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8158eb70>] ? gs_change+0x13/0x13 BUG: scheduling while atomic: ksoftirqd/8/37/0x10000004 INFO: lockdep is turned off. Modules linked in: ext3 jbd mbcache sd_mod crc_t10dif qla2xxx scsi_transport_fc scsi_tgt mptsas mptscsih mptbase scsi_transport_sas [last unloaded: scsi_wait_scan] Pid: 37, comm: ksoftirqd/8 Tainted: G W 3.5.0-rc1-threadhotplugv3-debug #2 Call Trace: Enterprise Linu [<ffffffff8108d1cc>] __schedule_bug+0x6c/0x90 x Server [<ffffffff81582818>] __schedule+0x7c8/0x880 [<ffffffff81092cea>] __cond_resched+0x2a/0x40 [<ffffffff81582960>] _cond_resched+0x30/0x40 [<ffffffff8105ea54>] run_ksoftirqd+0x54/0x70 [<ffffffff810c8b97>] smpboot_thread_fn+0x157/0x1f0 [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80 [<ffffffff8107e7ee>] kthread+0xee/0x100 [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10 [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13 [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8158eb70>] ? gs_change+0x13/0x13[...]
BUG: scheduling while atomic: watchdog/15/64/0x0000009b INFO: lockdep is turned off. Modules linked in: ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat xt_CHECKSUM iptable_mangle bridge stp llc autofs4 cpufreq_powersave sunrpc cpufreq_ondemand acpi_cpufreq freq_table mperf ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 dm_mirror dm_region_hash dm_log dm_round_robin scsi_dh_rdac dm_multipath vhost_net macvtap macvlan tun uinput cdc_ether usbnet mii serio_raw kvm_intel kvm microcode i2c_i801 i2c_core lpc_ich mfd_core shpchp ioatdma dca i7core_edac edac_core bnx2 dm_mod sg ext3 jbd mbcache sd_mod crc_t10dif qla2xxx scsi_transport_fc scsi_tgt mptsas mptscsih mptbase scsi_transport_sas [last unloaded: scsi_wait_scan] Pid: 64, comm: watchdog/15 Tainted: G W 3.5.0-rc1-threadhotplugv3-debug #2 Call Trace: [<ffffffff8108d1cc>] __schedule_bug+0x6c/0x90 [<ffffffff81582818>] __schedule+0x7c8/0x880 [<ffffffff81582bf9>] schedule+0x29/0x70 [<ffffffff81582f77>] schedule_preempt_disabled+0x27/0x40 [<ffffffff810c8b33>] smpboot_thread_fn+0xf3/0x1f0 [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80 [<ffffffff8107e7ee>] kthread+0xee/0x100 [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10 [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13 [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8158eb70>] ? gs_change+0x13/0x13
Quoted 35 lines · Paul E. McKenney
Quoted 27 lines · Srivatsa S. Bhat
Quoted 26 lines · Paul E. McKenney
Quoted 24 lines · Srivatsa S. Bhat
Quoted 12 lines · Paul E. McKenney
Quoted 10 lines · Thomas Gleixner
The following series implements the infrastructure for parking and unparking kernel threads to avoid the full teardown and fork on cpu hotplug operations along with management infrastructure for hotplug and users.
Changes vs. V2:
Use callbacks for all functionality. Thanks to Rusty for pointing that out. It makes the use sites nice and simple and keeps all the code which would be duplicated otherwise on the core.
Quoted 6 lines
Hello, Thomas,
What version should I apply this patchset to? I tried v3.5-rc7, but got lots of warnings (one shown below) and the watchdog patch did not apply.
Hi Paul,
This patchset applies cleanly on Thomas' smp/hotplug branch in the -tip tree.
Quoted 6 lines
Quoted 3 lines
Thank you, Srivatsa, works much better. Still get "scheduling while atomic", looking into that.
Got a chance to run this patchset now.. Even I am getting "scheduling while atomic" messages like shown below.. Hmmm...
Quoted 9 lines
Here is what little I have done so far (lots of completing demands on time this week, but I should have a goodly block of time to focus on this today):
1. The failure is from the softirq modifications. Reverting that
commit gets rid of the failures.2. As one would expect,
CONFIG_PREEMPT=nkernels do not have the
problem, which of course indicates apreempt_disable()imbalance.
Right..
Quoted 11 lines
3. I was unable to spot the problem by inspection, but this is not
too surprising given the high level of distraction this week.4. Instrumentation shows that
preempt_count()grows slowly with
time, but with the upper bits zero. This confirms thepreempt_disableimbalance.5. I am currently placing
WARN_ONCE()calls in the code to track
this down. When I do find it, I fully expect to feel very stupid about my efforts on #3 above. ;-)
Hehe :-) I'll also see if I can dig out the problem..
Regards,
Srivatsa S. Bhat
Quoted 89 lines
Quoted 89 lines
-----------------
BUG: sleeping function called from invalid context at kernel/softirq.c:748 in_atomic(): 1, irqs_disabled(): 0, pid: 3, name: ksoftirqd/0 INFO: lockdep is turned off. Pid: 3, comm: ksoftirqd/0 Tainted: G W 3.5.0-rc1-threadhotplugv3-debug #2 Call Trace: [<ffffffff8108e7ac>] __might_sleep+0x18c/0x250 [<ffffffff8105ea4f>] run_ksoftirqd+0x4f/0x70 [<ffffffff810c8b97>] smpboot_thread_fn+0x157/0x1f0 [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80 [<ffffffff8107e7ee>] kthread+0xee/0x100 [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10 [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13 [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8158eb70>] ? gs_change+0x13/0x13 BUG: scheduling while atomic: ksoftirqd/0/3/0x1000000b INFO: lockdep is turned off. Modules linked in: ext3 jbd mbcache sd_mod crc_t10dif qla2xxx scsi_transport_fc scsi_tgt mptsas mptscsih mptbase scsi_transport_sas [last unloaded: scsi_wait_scan] Pid: 3, comm: ksoftirqd/0 Tainted: G W 3.5.0-rc1-threadhotplugv3-debug #2 Call Trace: [<ffffffff8108d1cc>] __schedule_bug+0x6c/0x90 [<ffffffff81582818>] __schedule+0x7c8/0x880 [<ffffffff81092cea>] __cond_resched+0x2a/0x40 [<ffffffff81582960>] _cond_resched+0x30/0x40 SELinux: 2048 avtab hash slots, 250818 rules. [<ffffffff8105ea54>] run_ksoftirqd+0x54/0x70 [<ffffffff810c8b97>] smpboot_thread_fn+0x157/0x1f0 [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80 [<ffffffff8107e7ee>] kthread+0xee/0x100 [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10 [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13 [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8158eb70>] ? gs_change+0x13/0x13[...]
BUG: sleeping function called from invalid context at kernel/softirq.c:748 in_atomic(): 1, irqs_disabled(): 0, pid: 37, name: ksoftirqd/8 INFO: lockdep is turned off. Pid: 37, comm: ksoftirqd/8 Tainted: G W 3.5.0-rc1-threadhotplugv3-debug #2 Call Trace: [<ffffffff8108e7ac>] __might_sleep+0x18c/0x250 [<ffffffff8105ea4f>] run_ksoftirqd+0x4f/0x70 [<ffffffff810c8b97>] smpboot_thread_fn+0x157/0x1f0 Welcome to [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80 [<ffffffff8107e7ee>] kthread+0xee/0x100 Red Hat [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10 [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13 [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8158eb70>] ? gs_change+0x13/0x13 BUG: scheduling while atomic: ksoftirqd/8/37/0x10000004 INFO: lockdep is turned off. Modules linked in: ext3 jbd mbcache sd_mod crc_t10dif qla2xxx scsi_transport_fc scsi_tgt mptsas mptscsih mptbase scsi_transport_sas [last unloaded: scsi_wait_scan] Pid: 37, comm: ksoftirqd/8 Tainted: G W 3.5.0-rc1-threadhotplugv3-debug #2 Call Trace: Enterprise Linu [<ffffffff8108d1cc>] __schedule_bug+0x6c/0x90 x Server [<ffffffff81582818>] __schedule+0x7c8/0x880 [<ffffffff81092cea>] __cond_resched+0x2a/0x40 [<ffffffff81582960>] _cond_resched+0x30/0x40 [<ffffffff8105ea54>] run_ksoftirqd+0x54/0x70 [<ffffffff810c8b97>] smpboot_thread_fn+0x157/0x1f0 [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80 [<ffffffff8107e7ee>] kthread+0xee/0x100 [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10 [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13 [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8158eb70>] ? gs_change+0x13/0x13[...]
BUG: scheduling while atomic: watchdog/15/64/0x0000009b INFO: lockdep is turned off. Modules linked in: ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat xt_CHECKSUM iptable_mangle bridge stp llc autofs4 cpufreq_powersave sunrpc cpufreq_ondemand acpi_cpufreq freq_table mperf ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 dm_mirror dm_region_hash dm_log dm_round_robin scsi_dh_rdac dm_multipath vhost_net macvtap macvlan tun uinput cdc_ether usbnet mii serio_raw kvm_intel kvm microcode i2c_i801 i2c_core lpc_ich mfd_core shpchp ioatdma dca i7core_edac edac_core bnx2 dm_mod sg ext3 jbd mbcache sd_mod crc_t10dif qla2xxx scsi_transport_fc scsi_tgt mptsas mptscsih mptbase scsi_transport_sas [last unloaded: scsi_wait_scan] Pid: 64, comm: watchdog/15 Tainted: G W 3.5.0-rc1-threadhotplugv3-debug #2 Call Trace: [<ffffffff8108d1cc>] __schedule_bug+0x6c/0x90 [<ffffffff81582818>] __schedule+0x7c8/0x880 [<ffffffff81582bf9>] schedule+0x29/0x70 [<ffffffff81582f77>] schedule_preempt_disabled+0x27/0x40 [<ffffffff810c8b33>] smpboot_thread_fn+0xf3/0x1f0 [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80 [<ffffffff8107e7ee>] kthread+0xee/0x100 [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10 [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13 [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8158eb70>] ? gs_change+0x13/0x13
Quoted 39 lines · Srivatsa S. Bhat
Quoted 38 lines · Paul E. McKenney
Quoted 27 lines · Srivatsa S. Bhat
Quoted 26 lines · Paul E. McKenney
Quoted 24 lines · Srivatsa S. Bhat
Quoted 12 lines · Paul E. McKenney
Quoted 10 lines · Thomas Gleixner
The following series implements the infrastructure for parking and unparking kernel threads to avoid the full teardown and fork on cpu hotplug operations along with management infrastructure for hotplug and users.
Changes vs. V2:
Use callbacks for all functionality. Thanks to Rusty for pointing that out. It makes the use sites nice and simple and keeps all the code which would be duplicated otherwise on the core.
Quoted 6 lines
Hello, Thomas,
What version should I apply this patchset to? I tried v3.5-rc7, but got lots of warnings (one shown below) and the watchdog patch did not apply.
Hi Paul,
This patchset applies cleanly on Thomas' smp/hotplug branch in the -tip tree.
Quoted 6 lines
Quoted 3 lines
Thank you, Srivatsa, works much better. Still get "scheduling while atomic", looking into that.
Got a chance to run this patchset now.. Even I am getting "scheduling while atomic" messages like shown below.. Hmmm...
Here is what little I have done so far (lots of completing demands on time this week, but I should have a goodly block of time to focus on this today):
Quoted 7 lines
Quoted 5 lines
1. The failure is from the softirq modifications. Reverting that
commit gets rid of the failures.2. As one would expect,
CONFIG_PREEMPT=nkernels do not have the
problem, which of course indicates apreempt_disable()imbalance.Right..
Except that the imbalance is not in softirq like I was thinking, but rather in smpboot. See patch below, which clears this up for me.
Thanx, Paul
Quoted 7 lines
Quoted 7 lines
3. I was unable to spot the problem by inspection, but this is not
too surprising given the high level of distraction this week.4. Instrumentation shows that
preempt_count()grows slowly with
time, but with the upper bits zero. This confirms thepreempt_disableimbalance.
Quoted 6 lines
Quoted 4 lines
5. I am currently placing
WARN_ONCE()calls in the code to track
this down. When I do find it, I fully expect to feel very stupid about my efforts on #3 above. ;-)Hehe :-) I'll also see if I can dig out the problem..
smpboot.c | 4 ++-- softirq.c | 3 ++- 2 files changed, 4 insertions(+), 3 deletions(-)
kernel/smpboot.c +2 -2
@@ -148,12 +148,12 @@ static int smpboot_thread_fn(void *data) }if (!ht->thread_should_run (td->cpu)) { -schedule_preempt_disabled (); +preempt_enable (); +schedule (); }else {set_current_state (TASK_RUNNING );preempt_enable (); ht->thread_fn (td->cpu); -preempt_disable (); } } }
kernel/softirq.c +2 -1
@@ -744,9 +744,10 @@ static void run_ksoftirqd(unsigned int cpu)local_irq_disable ();if (local_softirq_pending ()) {__do_softirq (); +rcu_note_context_switch (cpu);local_irq_enable ();cond_resched (); -rcu_note_context_switch (cpu); +return ; }local_irq_enable (); }
Quoted 76 lines · Paul E. McKenney
Quoted 39 lines · Srivatsa S. Bhat
Quoted 38 lines · Paul E. McKenney
Quoted 27 lines · Srivatsa S. Bhat
Quoted 26 lines · Paul E. McKenney
Quoted 24 lines · Srivatsa S. Bhat
Quoted 12 lines · Paul E. McKenney
Quoted 10 lines · Thomas Gleixner
The following series implements the infrastructure for parking and unparking kernel threads to avoid the full teardown and fork on cpu hotplug operations along with management infrastructure for hotplug and users.
Changes vs. V2:
Use callbacks for all functionality. Thanks to Rusty for pointing that out. It makes the use sites nice and simple and keeps all the code which would be duplicated otherwise on the core.
Quoted 6 lines
Hello, Thomas,
What version should I apply this patchset to? I tried v3.5-rc7, but got lots of warnings (one shown below) and the watchdog patch did not apply.
Hi Paul,
This patchset applies cleanly on Thomas' smp/hotplug branch in the -tip tree.
Quoted 6 lines
Quoted 3 lines
Thank you, Srivatsa, works much better. Still get "scheduling while atomic", looking into that.
Got a chance to run this patchset now.. Even I am getting "scheduling while atomic" messages like shown below.. Hmmm...
Here is what little I have done so far (lots of completing demands on time this week, but I should have a goodly block of time to focus on this today):
Quoted 7 lines
Quoted 5 lines
1. The failure is from the softirq modifications. Reverting that
commit gets rid of the failures.2. As one would expect,
CONFIG_PREEMPT=nkernels do not have the
problem, which of course indicates apreempt_disable()imbalance.Right..
Except that the imbalance is not in softirq like I was thinking, but rather in smpboot. See patch below, which clears this up for me.
Thanx, Paul
Quoted 7 lines
Quoted 7 lines
3. I was unable to spot the problem by inspection, but this is not
too surprising given the high level of distraction this week.4. Instrumentation shows that
preempt_count()grows slowly with
time, but with the upper bits zero. This confirms thepreempt_disableimbalance.Quoted 6 lines
Quoted 4 lines
5. I am currently placing
WARN_ONCE()calls in the code to track
this down. When I do find it, I fully expect to feel very stupid about my efforts on #3 above. ;-)Hehe :-) I'll also see if I can dig out the problem..
smpboot.c | 4 ++-- softirq.c | 3 ++- 2 files changed, 4 insertions(+), 3 deletions(-)kernel/smpboot.c +0
-0
kernel/smpboot.c static int smpboot_thread_fn(void *data) · line 152
Quoted 6 lines
kernel/smpboot.c +2
-1@@ -148,12 +148,12 @@ static int smpboot_thread_fn(void *data) }if (!ht->thread_should_run (td->cpu)) { -schedule_preempt_disabled (); +preempt_enable (); +schedule ();
Oh, of *course*! The trailing preempt_disable() would end up causing trouble as we go into the next iteration of the loop..
Quoted 15 lines
kernel/smpboot.c +0
-1}else {set_current_state (TASK_RUNNING );preempt_enable (); ht->thread_fn (td->cpu); -preempt_disable (); } } }kernel/softirq.c +0
-0@@ -744,9 +744,10 @@ static void run_ksoftirqd(unsigned int cpu)local_irq_disable ();
kernel/softirq.c static void run_ksoftirqd(unsigned int cpu) · line 753
Quoted 10 lines
kernel/softirq.c +2
-1if (local_softirq_pending ()) {__do_softirq (); +rcu_note_context_switch (cpu);local_irq_enable ();cond_resched (); -rcu_note_context_switch (cpu); +return ; }local_irq_enable (); }
Tested your fix, it works great!
Regards,
Srivatsa S. Bhat
Quoted 11 lines · Thomas Gleixner
The following series implements the infrastructure for parking and unparking kernel threads to avoid the full teardown and fork on cpu hotplug operations along with management infrastructure for hotplug and users.
Changes vs. V2:
Use callbacks for all functionality. Thanks to Rusty for pointing that out. It makes the use sites nice and simple and keeps all the code which would be duplicated otherwise on the core.
Hi Thomas,
I am observing stalls detected by RCU and the subsequent NMI backtraces while running CPU hotplug stress tests on this patchset. Will check whether I hit this without your patches too... (But I think the baseline is a little bit old.. 3.5-rc1..)
----------------------------
Booting Node 1 Processor 7 APIC 0x16
NMI watchdog: enabled, takes one hw-pmu counter.
lockdep: fixing up alternatives.
Booting Node 0 Processor 8 APIC 0x1
NMI watchdog: enabled, takes one hw-pmu counter.
INFO: task stress_cpu_hotp:6903 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.stress_cpu_hotp D 0000000000000000 0 6903 6850 0x00000000
ffff8808cda09ad8 0000000000000046 0000000000000001 ffff8808cda08010 ffff8808cda08000 ffff8808cda08000 ffff8808cda08000 ffff8808cda09fd8 ffff8808cda08000 ffff8808cda09fd8 ffff8808d04f45c0 ffff8808d2d8a200
Call Trace:
[<ffffffff81465164>] ? cpufreq_notify_transition+0x174/0x2f0
[<ffffffffa00b1200>] ? acpi_cpufreq_cpu_exit+0xb0/0xb0 [acpi_cpufreq]
[<ffffffff810c21f0>] ? mark_held_locks+0x80/0x120
[<ffffffff81591ab7>] ? __mutex_unlock_slowpath+0xf7/0x190
[<ffffffff815939c9>] schedule+0x29/0x70
[<ffffffff815915e5>] schedule_timeout+0x2d5/0x3f0
[<ffffffff810c21f0>] ? mark_held_locks+0x80/0x120
[<ffffffff81594eb0>] ? _raw_spin_unlock_irq+0x30/0x50
[<ffffffff810c259d>] ? trace_hardirqs_on_caller+0x12d/0x1b0
[<ffffffff81593845>] wait_for_common+0x115/0x190
[<ffffffff810983d0>] ? try_to_wake_up+0x2f0/0x2f0
[<ffffffff81078bd2>] ? queue_delayed_work_on+0x132/0x140
[<ffffffff8159399d>] wait_for_completion+0x1d/0x20
[<ffffffff81043839>] hpet_cpuhp_notify+0x119/0x160
[<ffffffff81043a40>] ? init_one_hpet_msi_clockevent+0x1c0/0x1c0
[<ffffffff810784b0>] ? __queue_work+0x500/0x500
[<ffffffff81599512>] notifier_call_chain+0x72/0x130
[<ffffffff810890fe>] __raw_notifier_call_chain+0xe/0x10
[<ffffffff8158af88>] _cpu_up+0x15b/0x164
[<ffffffff8158b04d>] cpu_up+0xbc/0x11f
[<ffffffff8157a87d>] store_online+0x9d/0xe0
[<ffffffff81396be0>] dev_attr_store+0x20/0x30
[<ffffffff81222119>] sysfs_write_file+0xd9/0x150
[<ffffffff811a6a5b>] vfs_write+0xcb/0x130
[<ffffffff811a6bb5>] sys_write+0x55/0x90
[<ffffffff8159e629>] system_call_fastpath+0x16/0x1b
5 locks held by stress_cpu_hotp/6903:
#0: (&buffer->mutex){+.+.+.}, at: [<ffffffff81222089>] sysfs_write_file+0x49/0x150
#1: (s_active#111){.+.+.+}, at: [<ffffffff81222102>] sysfs_write_file+0xc2/0x150
#2: (x86_cpu_hotplug_driver_mutex){+.+.+.}, at: [<ffffffff81039df7>] cpu_hotplug_driver_lock+0x17/0x20
#3: (cpu_add_remove_lock){+.+.+.}, at: [<ffffffff8158b009>] cpu_up+0x78/0x11f
#4: (cpu_hotplug.lock){+.+.+.}, at: [<ffffffff81059cef>] cpu_hotplug_begin+0x2f/0x60lockdep: fixing up alternatives.
Booting Node 0 Processor 9 APIC 0x3
NMI watchdog: enabled, takes one hw-pmu counter.
lockdep: fixing up alternatives.
Booting Node 0 Processor 10 APIC 0x5
CPU 2 is now offline
CPU 10 MCA banks CMCI:2 CMCI:3 CMCI:5
CPU 3 is now offline
CPU 11 MCA banks CMCI:2 CMCI:3 CMCI:5
CPU 4 is now offline
CPU 5 MCA banks CMCI:6 CMCI:8
CPU 12 MCA banks CMCI:2 CMCI:3 CMCI:5
CPU 5 is now offline
CPU 6 MCA banks CMCI:6 CMCI:8
CPU 13 MCA banks CMCI:2 CMCI:3 CMCI:5
CPU 6 is now offline
CPU 7 MCA banks CMCI:6 CMCI:8
CPU 14 MCA banks CMCI:2 CMCI:3 CMCI:5
CPU 7 is now offline
CPU 12 MCA banks CMCI:6 CMCI:8
CPU 15 MCA banks CMCI:2 CMCI:3 CMCI:5
CPU 8 is now offline
------------[ cut here ]------------ WARNING: at kernel/rcutree.c:1545 rcu_do_batch+0x35d/0x370() Hardware name: IBM System x -[7870C4Q]- Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas Pid: 11788, comm: migration/9 Not tainted 3.5.0-rc1-threadhotplugv3-paulfix #1 Call Trace: <IRQ> [<ffffffff810fe58d>] ? rcu_do_batch+0x35d/0x370 [<ffffffff810fe58d>] ? rcu_do_batch+0x35d/0x370 [<ffffffff8105672f>] warn_slowpath_common+0x7f/0xc0 [<ffffffff8105678a>] warn_slowpath_null+0x1a/0x20 [<ffffffff810fe58d>] rcu_do_batch+0x35d/0x370 [<ffffffff810c24e0>] ? trace_hardirqs_on_caller+0x70/0x1b0 [<ffffffff810c262d>] ? trace_hardirqs_on+0xd/0x10 [<ffffffff811000a3>] __rcu_process_callbacks+0x193/0x1f0 [<ffffffff8110018d>] rcu_process_callbacks+0x8d/0x260 [<ffffffff81060e71>] __do_softirq+0x151/0x3e0 [<ffffffff8159fb6c>] call_softirq+0x1c/0x30 [<ffffffff810194fd>] do_softirq+0x9d/0xd0 [<ffffffff81060555>] irq_exit+0xe5/0x100 [<ffffffff815a04ee>] smp_apic_timer_interrupt+0x6e/0x99 [<ffffffff8159f16f>] apic_timer_interrupt+0x6f/0x80 <EOI> [<ffffffff810e6232>] ? stop_machine_cpu_stop+0xe2/0x140 [<ffffffff810e6150>] ? stop_one_cpu_nowait+0x50/0x50 [<ffffffff810e5e53>] cpu_stopper_thread+0xf3/0x1e0 [<ffffffff81594f14>] ? _raw_spin_unlock_irqrestore+0x44/0x90 [<ffffffff810e5d60>] ? res_counter_init+0x60/0x60 [<ffffffff810c259d>] ? trace_hardirqs_on_caller+0x12d/0x1b0 [<ffffffff810c262d>] ? trace_hardirqs_on+0xd/0x10 [<ffffffff810e5d60>] ? res_counter_init+0x60/0x60 [<ffffffff810811ee>] kthread+0xee/0x100 [<ffffffff8159fa74>] kernel_thread_helper+0x4/0x10 [<ffffffff815952b0>] ? retint_restore_args+0x13/0x13 [<ffffffff81081100>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8159fa70>] ? gs_change+0x13/0x13 ---[ end trace e18e7d52508d4205 ]---
CPU 9 is now offline
CPU 10 is now offline
Booting Node 1 Processor 15 APIC 0x17
NMI watchdog: enabled, takes one hw-pmu counter.
CPU 1 is now offline
CPU 9 MCA banks CMCI:2 CMCI:3 CMCI:5
CPU 2 is now offline
CPU 10 MCA banks CMCI:2 CMCI:3 CMCI:5
CPU 3 is now offline
CPU 11 MCA banks CMCI:2 CMCI:3 CMCI:5
NOHZ: local_softirq_pending 100
CPU 4 is now offline
CPU 5 MCA banks CMCI:6 CMCI:8
CPU 12 MCA banks CMCI:2 CMCI:3 CMCI:5
Broke affinity for irq 81
CPU 5 is now offline
CPU 6 MCA banks CMCI:6 CMCI:8
CPU 13 MCA banks CMCI:2 CMCI:3 CMCI:5
CPU 6 is now offline
CPU 7 MCA banks CMCI:6 CMCI:8
CPU 14 MCA banks CMCI:2 CMCI:3 CMCI:5
NOHZ: local_softirq_pending 200
CPU 7 is now offline
CPU 12 MCA banks CMCI:6 CMCI:8
CPU 15 MCA banks CMCI:2 CMCI:3 CMCI:5
CPU 8 is now offline
CPU 9 is now offline
CPU 10 is now offline
CPU 11 is now offline
CPU 12 is now offline
CPU 13 MCA banks CMCI:6 CMCI:8
CPU 13 is now offline
CPU 14 MCA banks CMCI:6 CMCI:8
CPU 14 is now offline
CPU 15 MCA banks CMCI:6 CMCI:8
CPU 15 is now offline
Booting Node 0 Processor 10 APIC 0x5
NMI watchdog: enabled, takes one hw-pmu counter.
lockdep: fixing up alternatives.
Booting Node 0 Processor 11 APIC 0x7
NMI watchdog: enabled, takes one hw-pmu counter.
INFO: rcu_sched self-detected stall on CPU { 3} (t=91070 jiffies)
INFO: rcu_sched self-detected stall on CPU { 4} (t=91070 jiffies)
sending NMI to all CPUs:
NMI backtrace for cpu 0
CPU 0
Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas
INFO: rcu_sched self-detected stall on CPUPid: 0, comm: swapper/0 Tainted: G W 3.5.0-rc1-threadhotplugv3-paulfix #1 IBM IBM System x -[7870C4Q]-
{
/68Y8033
5
RIP: 0010:[<ffffffff8103b741>] [<ffffffff8103b741>] default_send_IPI_mask_sequence_phys+0xa1/0x130 RSP: 0018:ffff8808ffc03d98 EFLAGS: 00000002 RAX: 00000000000010ef RBX: 0000000000000006 RCX: 0000000000000010 } (t=91070 jiffies) RDX: 0000000000000fc0 RSI: 0000000000000006 RDI: 0000000000000000 RBP: ffff8808ffc03dd8 R08: 0000000000000010 R09: ffffffff81da9d60 R10: 0000000000000001 R11: 0000000000000000 R12: 0000000000000014 R13: 00000000000000ef R14: 000000000000dd40 R15: 00000000000000ef FS: 0000000000000000(0000) GS:ffff8808ffc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: ffffffffff600400 CR3: 0000000001a0c000 CR4: 00000000000007f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process swapper/0 (pid: 0, threadinfo ffffffff81a00000, task ffffffff81a14420) Stack: 0000000000000000 ffffffff81da9d60 0000000000000092 000000000000ea20 ffffffff81da9d60 000001732e239870 ffffffff81a1ef00 7fffffffffffffff ffff8808ffc03de8 ffffffff8103e80e ffff8808ffc03df8 ffffffff8103a43b Call Trace: <IRQ> [<ffffffff8103e80e>] physflat_send_IPI_mask+0xe/0x10 [<ffffffff8103a43b>] lapic_timer_broadcast+0x1b/0x20 [<ffffffff810bb126>] tick_do_broadcast+0x66/0xa0 [<ffffffff810bb254>] tick_handle_oneshot_broadcast+0xf4/0x130 [<ffffffff81019de5>] timer_interrupt+0x15/0x20 [<ffffffff810f7cbc>] handle_irq_event_percpu+0xcc/0x3e0 [<ffffffff810f8018>] handle_irq_event+0x48/0x70 [<ffffffff810fb755>] handle_edge_irq+0x65/0x130 [<ffffffff8101958e>] handle_irq+0x5e/0x150 [<ffffffff81599736>] ? atomic_notifier_call_chain+0x16/0x20 [<ffffffff815a0400>] do_IRQ+0x60/0xe0 [<ffffffff815951ef>] common_interrupt+0x6f/0x6f <EOI> [<ffffffff81306a18>] ? intel_idle+0xc8/0x170 [<ffffffff81306a14>] ? intel_idle+0xc4/0x170 [<ffffffff81467107>] cpuidle_enter+0x17/0x20 [<ffffffff81467ab3>] cpuidle_idle_call+0x143/0x290 [<ffffffff810205c5>] cpu_idle+0xe5/0x140 [<ffffffff815939c9>] ? schedule+0x29/0x70 [<ffffffff815769af>] rest_init+0xff/0x180 [<ffffffff815768b0>] ? csum_partial_copy_generic+0x170/0x170 [<ffffffff81b17f2a>] start_kernel+0x3be/0x48c [<ffffffff81b17965>] ? kernel_init+0x290/0x290 [<ffffffff8158f9be>] ? memblock_reserve+0x2e/0x4e [<ffffffff81b172d5>] x86_64_start_reservations+0xa5/0xc2 [<ffffffff81b17401>] x86_64_start_kernel+0x10f/0x12a [<ffffffff81b17120>] ? early_idt_handlers+0x120/0x120 Code: a0 e3 ae 81 41 83 fd 02 45 0f b7 24 06 74 4b 8b 04 25 00 a3 5f ff f6 c4 10 74 1b 0f 1f 80 00 00 00 00 f3 90 8b 04 25 00 a3 5f ff <f6> c4 10 75 f2 8b 0d 5c ea ab 00 44 89 e0 c1 e0 18 89 04 25 10 NMI backtrace for cpu 1 CPU 1 Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas
Pid: 0, comm: swapper/1 Tainted: G W 3.5.0-rc1-threadhotplugv3-paulfix #1 IBM IBM System x -[7870C4Q]-/68Y8033
RIP: 0010:[<ffffffff8101fc7b>] [<ffffffff8101fc7b>] native_read_tsc+0xb/0x20 RSP: 0018:ffff8808d2d8dc70 EFLAGS: 00000002 RAX: 00000000000004d1 RBX: 0000000000000001 RCX: 00000000f9c4c38a RDX: 00000000000004d1 RSI: 01a8000000000000 RDI: 0000000000000001 RBP: ffff8808d2d8dc70 R08: 00000000f9c4c356 R09: ffff8808d2d8aab8 R10: ffff8808d2d8aab8 R11: 0000000000000000 R12: 00000000000011dc R13: 00000000aeda8ed8 R14: ffff8808d2d8c000 R15: 0000000000000001 FS: 0000000000000000(0000) GS:ffff8808ffc40000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: ffffffffff600400 CR3: 0000000001a0c000 CR4: 00000000000007e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process swapper/1 (pid: 0, threadinfo ffff8808d2d8c000, task ffff8808d2d8a200) Stack: ffff8808d2d8dcb8 ffffffff812aff41 ffff8808d2d8c000 00000001f9c4c32e ffffffff81a34580 00000000000011dc 00000000aeda8ed8 0000000000000001 ffff8808d2d8de64 ffff8808d2d8dcc8 ffffffff812affef ffff8808d2d8dcf8 Call Trace: [<ffffffff812aff41>] delay_tsc+0x41/0xc0 [<ffffffff812affef>] __delay+0xf/0x20 [<ffffffff812b769b>] do_raw_spin_lock+0xab/0x150 [<ffffffff815946b1>] _raw_spin_lock_irqsave+0x61/0x70 [<ffffffff810baf03>] ? tick_broadcast_oneshot_control+0x83/0x130 [<ffffffff810baf03>] tick_broadcast_oneshot_control+0x83/0x130 [<ffffffff810bab68>] tick_notify+0xf8/0x200 [<ffffffff81599512>] notifier_call_chain+0x72/0x130 [<ffffffff81089116>] raw_notifier_call_chain+0x16/0x20 [<ffffffff810ba48d>] clockevents_notify+0x3d/0x150 [<ffffffff81306a32>] intel_idle+0xe2/0x170 [<ffffffff81467107>] cpuidle_enter+0x17/0x20 [<ffffffff81467ab3>] cpuidle_idle_call+0x143/0x290 [<ffffffff810205c5>] cpu_idle+0xe5/0x140 [<ffffffff8158991d>] start_secondary+0xd1/0xd4 Code: 9f 00 48 c7 43 08 00 00 00 00 48 89 03 48 83 c4 08 5b c9 c3 66 66 66 2e 0f 1f 84 00 00 00 00 00 55 48 89 e5 0f 31 89 c1 48 89 d0 <48> c1 e0 20 89 ca 48 09 d0 c9 c3 66 2e 0f 1f 84 00 00 00 00 00 NMI backtrace for cpu 2 CPU 2 Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas
Pid: 0, comm: swapper/2 Tainted: G W 3.5.0-rc1-threadhotplugv3-paulfix #1 IBM IBM System x -[7870C4Q]-/68Y8033
RIP: 0010:[<ffffffff812aff3c>] [<ffffffff812aff3c>] delay_tsc+0x3c/0xc0 RSP: 0018:ffff8808d2db1d40 EFLAGS: 00000006 RAX: 0000000000005f7d RBX: 0000000000000001 RCX: 000000005f7d5f7c RDX: ffff8808d2db1fd8 RSI: 0018000000000000 RDI: 0000000000000001 RBP: ffff8808d2db1d78 R08: 00000000f9c93f00 R09: ffff8808d2daead0 R10: ffff8808d2daead0 R11: 0000000000000001 R12: 000000000000117f R13: 00000000aeda8ed8 R14: ffff8808d2db0000 R15: 0000000000000002 FS: 0000000000000000(0000) GS:ffff8808ffc80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: ffffffffff600400 CR3: 0000000001a0c000 CR4: 00000000000007e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process swapper/2 (pid: 0, threadinfo ffff8808d2db0000, task ffff8808d2dae240) Stack: ffff8808d2db0000 00000002f9c93ecc ffffffff81a344c0 000000000000117f 00000000aeda8ed8 0000000000000001 ffff8808ffc98970 ffff8808d2db1d88 ffffffff812affef ffff8808d2db1db8 ffffffff812b769b ffffffff81a344c0 Call Trace: [<ffffffff812affef>] __delay+0xf/0x20 [<ffffffff812b769b>] do_raw_spin_lock+0xab/0x150 [<ffffffff815946b1>] _raw_spin_lock_irqsave+0x61/0x70 [<ffffffff810ba478>] ? clockevents_notify+0x28/0x150 [<ffffffff810ba478>] clockevents_notify+0x28/0x150 [<ffffffff81306a32>] intel_idle+0xe2/0x170 [<ffffffff81467107>] cpuidle_enter+0x17/0x20 [<ffffffff81467ab3>] cpuidle_idle_call+0x143/0x290 [<ffffffff810205c5>] cpu_idle+0xe5/0x140 [<ffffffff8158991d>] start_secondary+0xd1/0xd4 Code: 66 66 66 66 90 65 48 8b 14 25 c8 c7 00 00 4c 8d b2 28 e0 ff ff 89 fb 41 83 46 1c 01 65 44 8b 3c 25 08 dd 00 00 66 66 90 0f ae e8 <e8> 2f fd d6 ff 66 90 89 45 d0 44 89 7d d4 4c 89 75 c8 eb 1d 48 NMI backtrace for cpu 3 CPU 3 Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas
Pid: 0, comm: swapper/3 Tainted: G W 3.5.0-rc1-threadhotplugv3-paulfix #1 IBM IBM System x -[7870C4Q]-/68Y8033
RIP: 0010:[<ffffffff8137c0f8>] [<ffffffff8137c0f8>] io_serial_in+0x18/0x20 RSP: 0018:ffff8808ffcc3be8 EFLAGS: 00000002 RAX: ffff8808d2db7f00 RBX: ffffffff826876b0 RCX: 0000000000000000 RDX: 00000000000002fd RSI: 0000000000000005 RDI: ffffffff826876b0 RBP: ffff8808ffcc3be8 R08: 00000000f9cdb3aa R09: 0000000000000000 R10: 0000000000000001 R11: 0000000000000f32 R12: 000000000000270b R13: 0000000000000020 R14: 0000000000000043 R15: ffffffff826876b0 FS: 0000000000000000(0000) GS:ffff8808ffcc0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: ffffffffff600400 CR3: 0000000001a0c000 CR4: 00000000000007e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process swapper/3 (pid: 0, threadinfo ffff8808d2db6000, task ffff8808d2db4280) Stack: ffff8808ffcc3c18 ffffffff8137c96b 000000000000ffff ffffffff826876b0 0000000000000020 ffffffff81d1f740 ffff8808ffcc3c38 ffffffff8137c9f6 0000000000000006 ffffffff8137c9d0 ffff8808ffcc3c78 ffffffff81377fbd Call Trace: <IRQ> [<ffffffff8137c96b>] wait_for_xmitr+0x3b/0xa0 [<ffffffff8137c9f6>] serial8250_console_putchar+0x26/0x40 [<ffffffff8137c9d0>] ? wait_for_xmitr+0xa0/0xa0 [<ffffffff81377fbd>] uart_console_write+0x3d/0x70 [<ffffffff8137f423>] serial8250_console_write+0xc3/0x1a0 [<ffffffff810577b7>] call_console_drivers+0xa7/0x1b0 [<ffffffff81057ced>] console_unlock+0x10d/0x220 [<ffffffff81058436>] vprintk_emit+0x1a6/0x610 [<ffffffff810c36c1>] ? __lock_acquire+0x301/0x520 [<ffffffff8159128c>] printk+0x4f/0x51 [<ffffffff810fe8a4>] print_cpu_stall+0x74/0x130 [<ffffffff8110064e>] __rcu_pending+0xee/0x210 [<ffffffff81100878>] rcu_check_callbacks+0x108/0x310 [<ffffffff81068c66>] update_process_times+0x46/0x90 [<ffffffff810bbf66>] tick_sched_timer+0x66/0xc0 [<ffffffff810bbf00>] ? tick_nohz_handler+0x100/0x100 [<ffffffff81086a4e>] __run_hrtimer+0x12e/0x330 [<ffffffff81086e9f>] hrtimer_interrupt+0xcf/0x200 [<ffffffff815a04e9>] smp_apic_timer_interrupt+0x69/0x99 [<ffffffff8159f16f>] apic_timer_interrupt+0x6f/0x80 <EOI> [<ffffffff81306a18>] ? intel_idle+0xc8/0x170 [<ffffffff81306a14>] ? intel_idle+0xc4/0x170 [<ffffffff81467107>] cpuidle_enter+0x17/0x20 [<ffffffff81467ab3>] cpuidle_idle_call+0x143/0x290 [<ffffffff810205c5>] cpu_idle+0xe5/0x140 [<ffffffff8158991d>] start_secondary+0xd1/0xd4 Code: 00 00 00 d3 e6 48 63 f6 48 03 77 40 8b 06 c9 c3 0f 1f 00 55 48 89 e5 66 66 66 66 90 0f b6 8f 91 00 00 00 89 f2 d3 e2 03 57 38 ec <0f> b6 c0 c9 c3 0f 1f 00 55 48 89 e5 66 66 66 66 90 0f b6 8f 91 NMI backtrace for cpu 4 CPU 4 Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas
Pid: 0, comm: swapper/4 Tainted: G W 3.5.0-rc1-threadhotplugv3-paulfix #1 IBM IBM System x -[7870C4Q]-/68Y8033
RIP: 0010:[<ffffffff812aff5e>] [<ffffffff812aff5e>] delay_tsc+0x5e/0xc0 RSP: 0018:ffff88117fc03d10 EFLAGS: 00000002 RAX: ffff8808d2dba000 RBX: 00000000000479ad RCX: 00000000f9d24aaa RDX: 00000000f9d24aaa RSI: 0000000000000005 RDI: 00000000000479ad RBP: ffff88117fc03d48 R08: 00000000f9d24aaa R09: ffffffff81af8a60 R10: 0000000000000001 R11: 0000000000000f34 R12: 0000000000001000 R13: 0000000000000002 R14: ffff8808d2dba000 R15: 0000000000000004 FS: 0000000000000000(0000) GS:ffff88117fc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 00007f9f8b3ba000 CR3: 0000000001a0c000 CR4: 00000000000007e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process swapper/4 (pid: 0, threadinfo ffff8808d2dba000, task ffff8808d2db82c0) Stack: ffff8808d2dba000 00000004f9d24a08 0000000000000000 0000000000001000 0000000000000002 000000000000dd40 0000000000000002 ffff88117fc03d58 ffffffff812b0028 ffff88117fc03d78 ffffffff8103b212 0000000000000005 Call Trace: <IRQ> [<ffffffff812b0028>] __const_udelay+0x28/0x30 [<ffffffff8103b212>] native_safe_apic_wait_icr_idle+0x22/0x60 [<ffffffff8103b77d>] default_send_IPI_mask_sequence_phys+0xdd/0x130 [<ffffffff8103e829>] physflat_send_IPI_all+0x19/0x20 [<ffffffff8103b85c>] arch_trigger_all_cpu_backtrace+0x6c/0xb0 [<ffffffff810fe8a9>] print_cpu_stall+0x79/0x130 [<ffffffff8110064e>] __rcu_pending+0xee/0x210 [<ffffffff81100878>] rcu_check_callbacks+0x108/0x310 [<ffffffff81068c66>] update_process_times+0x46/0x90 [<ffffffff810bbf66>] tick_sched_timer+0x66/0xc0 [<ffffffff810bbf00>] ? tick_nohz_handler+0x100/0x100 [<ffffffff81086a4e>] __run_hrtimer+0x12e/0x330 [<ffffffff81086e9f>] hrtimer_interrupt+0xcf/0x200 [<ffffffff815a04e9>] smp_apic_timer_interrupt+0x69/0x99 [<ffffffff8159f16f>] apic_timer_interrupt+0x6f/0x80 <EOI> [<ffffffff81306a18>] ? intel_idle+0xc8/0x170 [<ffffffff81306a14>] ? intel_idle+0xc4/0x170 [<ffffffff81467107>] cpuidle_enter+0x17/0x20 [<ffffffff81467ab3>] cpuidle_idle_call+0x143/0x290 [<ffffffff810205c5>] cpu_idle+0xe5/0x140 [<ffffffff8158991d>] start_secondary+0xd1/0xd4 Code: dd 00 00 66 66 90 0f ae e8 e8 2f fd d6 ff 66 90 89 45 d0 44 89 7d d4 4c 89 75 c8 eb 1d 48 8b 45 c8 83 68 1c 01 f3 90 83 40 1c 01 <65> 44 8b 3c 25 08 dd 00 00 44 3b 7d d4 75 36 66 66 90 0f ae e8 NMI backtrace for cpu 5 CPU 5 Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas
Pid: 0, comm: swapper/5 Tainted: G W 3.5.0-rc1-threadhotplugv3-paulfix #1 IBM IBM System x -[7870C4Q]-/68Y8033
RIP: 0010:[<ffffffff810699e7>] [<ffffffff810699e7>] run_timer_softirq+0xb7/0x2c0 RSP: 0000:ffff88117fc43e30 EFLAGS: 00000002 RAX: 0000000100139276 RBX: 0000000000000075 RCX: ffff881152e08790 RDX: 0000000100139275 RSI: ffff88117fc43df0 RDI: ffff881152e08000 RBP: ffff88117fc43eb0 R08: ffff8808d0893d70 R09: ffff8808d2dbcb90 R10: ffff8808d2dbcb90 R11: 0000000000000001 R12: ffff881152e08000 R13: 0000000000000141 R14: ffff88117fc43e70 R15: ffffffff810784b0 FS: 0000000000000000(0000) GS:ffff88117fc40000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001a0c000 CR4: 00000000000007e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process swapper/5 (pid: 0, threadinfo ffff8808d2dbe000, task ffff8808d2dbc300) Stack: 0000000000000002 ffff8808d2dbe000 ffff8808d2dbe000 ffff88117fc50ac0 ffff881152e09c50 ffff881152e09850 ffff881152e09450 ffff881152e09050 ffff88117fc43e70 ffff88117fc43e70 ffff88117fc43ea0 0000000000000282 Call Trace: <IRQ> [<ffffffff81060e71>] __do_softirq+0x151/0x3e0 [<ffffffff8159fb6c>] call_softirq+0x1c/0x30 [<ffffffff810194fd>] do_softirq+0x9d/0xd0 [<ffffffff81060555>] irq_exit+0xe5/0x100 [<ffffffff815a04ee>] smp_apic_timer_interrupt+0x6e/0x99 [<ffffffff8159f16f>] apic_timer_interrupt+0x6f/0x80 <EOI> [<ffffffff81306a18>] ? intel_idle+0xc8/0x170 [<ffffffff81306a14>] ? intel_idle+0xc4/0x170 [<ffffffff81467107>] cpuidle_enter+0x17/0x20 [<ffffffff81467ab3>] cpuidle_idle_call+0x143/0x290 [<ffffffff810205c5>] cpu_idle+0xe5/0x140 [<ffffffff8158991d>] start_secondary+0xd1/0xd4 Code: 00 00 00 00 48 8b 05 39 76 a9 00 49 8b 54 24 40 48 39 d0 0f 88 6b 01 00 00 89 d3 81 e3 ff 00 00 00 0f 84 35 01 00 00 48 8d 42 01 <49> 89 44 24 40 48 63 c3 48 83 c0 04 48 c1 e0 04 4a 8b 54 20 10 NMI backtrace for cpu 6 CPU 6 Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas
Pid: 0, comm: swapper/6 Tainted: G W 3.5.0-rc1-threadhotplugv3-paulfix #1 IBM IBM System x -[7870C4Q]-/68Y8033
RIP: 0010:[<ffffffff81069a09>] [<ffffffff81069a09>] run_timer_softirq+0xd9/0x2c0 RSP: 0000:ffff88117fc83e30 EFLAGS: 00000006 RAX: 0000000000000550 RBX: 0000000000000051 RCX: ffff881152e58560 RDX: ffff881152e58560 RSI: ffff88117fc83df0 RDI: ffff881152e58000 RBP: ffff88117fc83eb0 R08: ffffffffff640027 R09: ffff8808d2dc0bd0 R10: ffff8808d2dc0bd0 R11: 0000000000000001 R12: ffff881152e58000 R13: 0000000000000141 R14: ffff88117fc83e70 R15: ffffffff810689f0 FS: 0000000000000000(0000) GS:ffff88117fc80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001a0c000 CR4: 00000000000007e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process swapper/6 (pid: 0, threadinfo ffff8808d2dc2000, task ffff8808d2dc0340) Stack: 0000000000000002 ffff8808d2dc2000 ffff8808d2dc2000 ffff8808d28d4740 ffff881152e59c50 ffff881152e59850 ffff881152e59450 ffff881152e59050 ffff881152e58560 ffff88117fc83e70 ffff88117fc83ea0 0000000000000282 Call Trace: <IRQ> [<ffffffff81060e71>] __do_softirq+0x151/0x3e0 [<ffffffff8159fb6c>] call_softirq+0x1c/0x30 [<ffffffff810194fd>] do_softirq+0x9d/0xd0 [<ffffffff81060555>] irq_exit+0xe5/0x100 [<ffffffff815a04ee>] smp_apic_timer_interrupt+0x6e/0x99 [<ffffffff8159f16f>] apic_timer_interrupt+0x6f/0x80 <EOI> [<ffffffff81306a18>] ? intel_idle+0xc8/0x170 [<ffffffff81306a14>] ? intel_idle+0xc4/0x170 [<ffffffff81467107>] cpuidle_enter+0x17/0x20 [<ffffffff81467ab3>] cpuidle_idle_call+0x143/0x290 [<ffffffff810205c5>] cpu_idle+0xe5/0x140 [<ffffffff8158991d>] start_secondary+0xd1/0xd4 Code: 84 35 01 00 00 48 8d 42 01 49 89 44 24 40 48 63 c3 48 83 c0 04 48 c1 e0 04 4a 8b 54 20 10 49 8d 4c 04 10 48 89 55 c0 4c 89 72 08 <4a> 8b 54 20 18 48 89 55 c8 4c 89 32 4a 89 4c 20 10 4a 89 4c 20 NMI backtrace for cpu 7 CPU 7 Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas
Pid: 6687, comm: pickup Tainted: G W 3.5.0-rc1-threadhotplugv3-paulfix #1 IBM IBM System x -[7870C4Q]-/68Y8033
RIP: 0010:[<ffffffff81043018>] [<ffffffff81043018>] read_hpet+0x18/0x20 RSP: 0018:ffff88117fcc3e98 EFLAGS: 00000086 INFO: task stress_cpu_hotp:6903 blocked for more than 120 seconds. RAX: 00000000516a74c9 RBX: ffffffff81a1efc0 RCX: 7fffffffffffffff
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
RDX: 0000000000000000 RSI: 000001732e339800 RDI: ffffffff81a1efc0
RBP: ffff88117fcc3e98 R08: fffffffffe04007f R09: ffff8808cdb43238
R10: ffff8808cdb43238 R11: 0000000000000000 R12: 00000000238c1d76stress_cpu_hotp D 0000000000000000 0 6903 6850 0x00000000
R13: 00000000245c0de7 R14: 0000000000000639 R15: 000000000001dcbc FS: 0000000000000000(0000) GS:ffff88117fcc0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 00007f63164d8dd8 CR3: 00000008c9745000 CR4: 00000000000007e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 ffff8808cda09ad8 0000000000000046 0000000000000001 ffff8808cda08010 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process pickup (pid: 6687, threadinfo ffff8808cc59a000, task ffff8808cdb42980) Stack: ffff8808cda08000 ffff8808cda08000 ffff8808cda08000 ffff88117fcc3ed8 ffff8808cda09fd8
ffffffff810b446c ffff88117fcce818
ffff8808cda08000
000001732e339800
ffff8808cda09fd8 ffff8808d04f45c0
ffff88117fccdd80
ffff8808d2dae240
Call Trace: 0000000000000000 000001732e339800 000001732e339800 [<ffffffff81465164>] ? cpufreq_notify_transition+0x174/0x2f0 ffff88117fcc3f08 ffffffff810ba353 000001732e339800 [<ffffffffa00b1200>] ? acpi_cpufreq_cpu_exit+0xb0/0xb0 [acpi_cpufreq] ffff88117fcce8c0 Call Trace: [<ffffffff810c21f0>] ? mark_held_locks+0x80/0x120 <IRQ> [<ffffffff81591ab7>] ? __mutex_unlock_slowpath+0xf7/0x190 [<ffffffff810b446c>] ktime_get+0x6c/0xe0 [<ffffffff815939c9>] schedule+0x29/0x70 [<ffffffff810ba353>] clockevents_program_event+0x43/0x100 [<ffffffff815915e5>] schedule_timeout+0x2d5/0x3f0 [<ffffffff810bbae4>] tick_program_event+0x24/0x30 [<ffffffff810c21f0>] ? mark_held_locks+0x80/0x120 [<ffffffff81086ee6>] hrtimer_interrupt+0x116/0x200 [<ffffffff81594eb0>] ? _raw_spin_unlock_irq+0x30/0x50 [<ffffffff815a04e9>] smp_apic_timer_interrupt+0x69/0x99 [<ffffffff810c259d>] ? trace_hardirqs_on_caller+0x12d/0x1b0 [<ffffffff8159f16f>] apic_timer_interrupt+0x6f/0x80 [<ffffffff81593845>] wait_for_common+0x115/0x190 <EOI> [<ffffffff8118cd93>] ? kmem_cache_free+0x183/0x280 [<ffffffff810983d0>] ? try_to_wake_up+0x2f0/0x2f0 [<ffffffff81171e8e>] unlink_anon_vmas+0xae/0x180 [<ffffffff81078bd2>] ? queue_delayed_work_on+0x132/0x140 [<ffffffff81167039>] free_pgtables+0xa9/0x110 [<ffffffff8159399d>] wait_for_completion+0x1d/0x20 [<ffffffff8116c766>] exit_mmap+0x96/0x140 [<ffffffff81043839>] hpet_cpuhp_notify+0x119/0x160 [<ffffffff810545e7>] mmput+0x87/0x110 [<ffffffff8105affa>] exit_mm+0x10a/0x140 [<ffffffff81043a40>] ? init_one_hpet_msi_clockevent+0x1c0/0x1c0 [<ffffffff81594eb0>] ? _raw_spin_unlock_irq+0x30/0x50 [<ffffffff810784b0>] ? __queue_work+0x500/0x500 [<ffffffff81365598>] ? tty_audit_exit+0x28/0xa0 [<ffffffff81599512>] notifier_call_chain+0x72/0x130 [<ffffffff8105daa8>] do_exit+0x168/0x4b0 [<ffffffff810890fe>] __raw_notifier_call_chain+0xe/0x10 [<ffffffff810c259d>] ? trace_hardirqs_on_caller+0x12d/0x1b0 [<ffffffff8158af88>] _cpu_up+0x15b/0x164 [<ffffffff8105de3c>] do_group_exit+0x4c/0xc0 [<ffffffff8158b04d>] cpu_up+0xbc/0x11f [<ffffffff8105dec7>] sys_exit_group+0x17/0x20 [<ffffffff8157a87d>] store_online+0x9d/0xe0 [<ffffffff8159e629>] system_call_fastpath+0x16/0x1b [<ffffffff81396be0>] dev_attr_store+0x20/0x30 Code: [<ffffffff81222119>] sysfs_write_file+0xd9/0x150
c1 80 c9
[<ffffffff811a6a5b>] vfs_write+0xcb/0x130
0f 4c c2
[<ffffffff811a6bb5>] sys_write+0x55/0x90
c3 66 66
[<ffffffff8159e629>] system_call_fastpath+0x16/0x1b 5 locks held by stress_cpu_hotp/6903:
66 2e 0f 1f
#0:
84
(
00&buffer->mutex
00
){+.+.+.}
00
, at:
[<ffffffff81222089>] sysfs_write_file+0x49/0x150
00 00
#1:
55
(
48s_active
89
#111
e5
){.+.+.+}
66
, at:
[<ffffffff81222102>] sysfs_write_file+0xc2/0x150
66 66
#2:
66
(
90x86_cpu_hotplug_driver_mutex
48
){+.+.+.}
8b
, at:
[<ffffffff81039df7>] cpu_hotplug_driver_lock+0x17/0x20 05 a8
#3:
36
(
cccpu_add_remove_lock
00
){+.+.+.}
48
, at:
[<ffffffff8158b009>] cpu_up+0x78/0x11f
05 f0
#4:
00
(
00
cpu_hotplug.lock
00
){+.+.+.}
8b
, at:
[<ffffffff81059cef>] cpu_hotplug_begin+0x2f/0x60
00 <89> c0 c9 c3 0f 1f 40 00 55 48 89 e5 66 66 66 66 90 8b 35 91 36 NMI backtrace for cpu 8
CPU 8
Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas
Pid: 0, comm: swapper/8 Tainted: G W 3.5.0-rc1-threadhotplugv3-paulfix #1 IBM IBM System x -[7870C4Q]-/68Y8033
RIP: 0010:[<ffffffff81069a15>] [<ffffffff81069a15>] run_timer_softirq+0xe5/0x2c0 RSP: 0018:ffff8808ffd03e30 EFLAGS: 00000002 RAX: 0000000000000ce0 RBX: 00000000000000ca RCX: ffff8808d2f98cf0 RDX: ffff8808ffd03e70 RSI: ffff8808ffd03df0 RDI: ffff8808d2f98000 RBP: ffff8808ffd03eb0 R08: ffffffffff640027 R09: ffff8808d2dcac50 R10: ffff8808d2dcac50 R11: 0000000000000001 R12: ffff8808d2f98000 R13: 0000000000000141 R14: ffff8808ffd03e70 R15: ffffffff810784b0 FS: 0000000000000000(0000) GS:ffff8808ffd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 00007fee875cf0a0 CR3: 0000000001a0c000 CR4: 00000000000007e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process swapper/8 (pid: 0, threadinfo ffff8808d2dcc000, task ffff8808d2dca3c0) Stack: 0000000000000002 ffff8808d2dcc000 ffff8808d2dcc000 ffff8808ffd109a0 ffff8808d2f99c50 ffff8808d2f99850 ffff8808d2f99450 ffff8808d2f99050 ffff8808ffd03e70 ffff8808ffd03e70 ffff8808ffd03ea0 0000000000000282 Call Trace: <IRQ> [<ffffffff81060e71>] __do_softirq+0x151/0x3e0 [<ffffffff8159fb6c>] call_softirq+0x1c/0x30 [<ffffffff810194fd>] do_softirq+0x9d/0xd0 [<ffffffff81060555>] irq_exit+0xe5/0x100 [<ffffffff815a04ee>] smp_apic_timer_interrupt+0x6e/0x99 [<ffffffff8159f16f>] apic_timer_interrupt+0x6f/0x80 <EOI> [<ffffffff81306a18>] ? intel_idle+0xc8/0x170 [<ffffffff81306a14>] ? intel_idle+0xc4/0x170 [<ffffffff81467107>] cpuidle_enter+0x17/0x20 [<ffffffff81467ab3>] cpuidle_idle_call+0x143/0x290 [<ffffffff810205c5>] cpu_idle+0xe5/0x140 [<ffffffff8158991d>] start_secondary+0xd1/0xd4 Code: 24 40 48 63 c3 48 83 c0 04 48 c1 e0 04 4a 8b 54 20 10 49 8d 4c 04 10 48 89 55 c0 4c 89 72 08 4a 8b 54 20 18 48 89 55 c8 4c 89 32 <4a> 89 4c 20 10 4a 89 4c 20 18 48 8b 5d c0 4c 39 f3 74 98 48 8b NMI backtrace for cpu 9 CPU 9 Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas
Pid: 5, comm: kworker/u:0 Tainted: G W 3.5.0-rc1-threadhotplugv3-paulfix #1 IBM IBM System x -[7870C4Q]-/68Y8033
RIP: 0010:[<ffffffff8107a6d2>] [<ffffffff8107a6d2>] process_one_work+0x312/0x480 RSP: 0018:ffff8808d2d3dd40 EFLAGS: 00000046 RAX: 0000000000000000 RBX: ffff8808cfb34030 RCX: 000000002dfe2dfe RDX: 0000000000000000 RSI: 000000008d0b56c0 RDI: ffffffff81da2d00 RBP: ffff8808d2d3ddd0 R08: fffffffffe340073 R09: ffff8808d2d3a9d0 R10: ffff8808d2d3a9d0 R11: 0000000000000001 R12: ffff8808cfb34028 R13: ffff881152ca8980 R14: ffffffff81da2d00 R15: ffff8808d0b56c00 FS: 0000000000000000(0000) GS:ffff8808ffd40000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 00007f28bade13e0 CR3: 0000000001a0c000 CR4: 00000000000007e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process kworker/u:0 (pid: 5, threadinfo ffff8808d2d3c000, task ffff8808d2d3a140) Stack: ffffffff8107a4ff 0000000000000046 0000000000000004 ffffffff8107b2d8 ffffffff00000000 0000000000000000 ffffffffa0b23b00 0ffff8808d0b56c0 ffffffffa0b26f98 ffffffff820155d0 0000000000000000 ffffffffa0b25353 Call Trace: [<ffffffff8107a4ff>] ? process_one_work+0x13f/0x480 [<ffffffff8107b2d8>] ? worker_thread+0x68/0x3b0 [<ffffffffa0b23b00>] ? edac_pci_reset_delay_period+0x60/0x60 [edac_core] [<ffffffff8107b3f6>] worker_thread+0x186/0x3b0 [<ffffffff8107b270>] ? manage_workers+0x120/0x120 [<ffffffff810811ee>] kthread+0xee/0x100 [<ffffffff8159fa74>] kernel_thread_helper+0x4/0x10 [<ffffffff815952b0>] ? retint_restore_args+0x13/0x13 [<ffffffff81081100>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8159fa70>] ? gs_change+0x13/0x13 Code: 48 89 02 74 04 48 89 50 08 49 c7 45 00 00 00 00 00 49 c7 45 08 00 00 00 00 48 c1 6d a8 04 49 c7 45 10 00 00 00 00 31 d2 8b 75 a8 <49> c7 45 18 00 00 00 00 4c 89 ff 83 e6 0f e8 2b fc ff ff 48 83 NMI backtrace for cpu 10 CPU 10 Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas
Pid: 16471, comm: kworker/10:1 Tainted: G W 3.5.0-rc1-threadhotplugv3-paulfix #1 IBM IBM System x -[7870C4Q]-/68Y8033
RIP: 0010:[<ffffffff8108fe29>] [<ffffffff8108fe29>] __might_sleep+0x19/0x240 RSP: 0018:ffff8808c9657cc0 EFLAGS: 00000286 RAX: 000000010013bf80 RBX: ffff8808d253d900 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 000000000000109d RDI: ffffffff817c61e4 RBP: ffff8808c9657ce0 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000003 R11: 0000000000000001 R12: ffff8808d253d840 R13: ffff8808d25309c0 R14: 0000000000000001 R15: 0000000000000000 FS: 0000000000000000(0000) GS:ffff8808ffd80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 00007fee87eccb60 CR3: 0000000001a0c000 CR4: 00000000000007e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process kworker/10:1 (pid: 16471, threadinfo ffff8808c9656000, task ffff8808cf8e4380) Stack: ffff8808d253d840 ffff8808d253d900 ffff8808d253d840 ffff8808d25309c0 ffff8808c9657d30 ffffffff8118d7c4 ffff8808ffd90ac0 00000000c9657d80 0000000000000000 ffff8808ffd90ac8 ffff8808ffd90ac0 ffff8808cf888140 Call Trace: [<ffffffff8118d7c4>] cache_reap+0xf4/0x270 [<ffffffff8107a5c1>] process_one_work+0x201/0x480 [<ffffffff8107a4ff>] ? process_one_work+0x13f/0x480 [<ffffffff8107b2d8>] ? worker_thread+0x68/0x3b0 [<ffffffff8118d6d0>] ? drain_freelist+0xb0/0xb0 [<ffffffff8107b3f6>] worker_thread+0x186/0x3b0 [<ffffffff8107b270>] ? manage_workers+0x120/0x120 [<ffffffff810811ee>] kthread+0xee/0x100 [<ffffffff8159fa74>] kernel_thread_helper+0x4/0x10 [<ffffffff815952b0>] ? retint_restore_args+0x13/0x13 [<ffffffff81081100>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8159fa70>] ? gs_change+0x13/0x13 Code: 55 55 48 83 c4 08 5b c9 c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 48 83 ec 20 48 89 5d e8 4c 89 65 f0 4c 89 6d f8 66 66 66 66 90 <49> 89 fd 41 89 f4 89 d3 e8 2a d6 fe ff 85 c0 74 0d 80 3d d8 e6 NMI backtrace for cpu 11 CPU 11 Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas
Pid: 0, comm: swapper/11 Tainted: G W 3.5.0-rc1-threadhotplugv3-paulfix #1 IBM IBM System x -[7870C4Q]-/68Y8033
RIP: 0010:[<ffffffff810699f7>] [<ffffffff810699f7>] run_timer_softirq+0xc7/0x2c0 RSP: 0018:ffff8808ffdc3e30 EFLAGS: 00000002 RAX: 0000000000000e70 RBX: 00000000000000e3 RCX: ffff8808d2898e70 RDX: 0000000100131fe3 RSI: ffff8808ffdc3df0 RDI: ffff8808d2898000 RBP: ffff8808ffdc3eb0 R08: ffffffffff640027 R09: ffff8808d2dd8d10 R10: ffff8808d2dd8d10 R11: 0000000000000001 R12: ffff8808d2898000 R13: 0000000000000141 R14: ffff8808ffdc3e70 R15: ffffffff810784b0 FS: 0000000000000000(0000) GS:ffff8808ffdc0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 00007fee875cf0a0 CR3: 0000000001a0c000 CR4: 00000000000007e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process swapper/11 (pid: 0, threadinfo ffff8808d2dda000, task ffff8808d2dd8480) Stack: 0000000000000002 ffff8808d2dda000 ffff8808d2dda000 ffff8808ffdd09a0 ffff8808d2899c50 ffff8808d2899850 ffff8808d2899450 ffff8808d2899050 ffff8808ffdc3e70 ffff8808ffdc3e70 ffff8808ffdc3ea0 0000000000000282 Call Trace: <IRQ> [<ffffffff81060e71>] __do_softirq+0x151/0x3e0 [<ffffffff8159fb6c>] call_softirq+0x1c/0x30 [<ffffffff810194fd>] do_softirq+0x9d/0xd0 [<ffffffff81060555>] irq_exit+0xe5/0x100 [<ffffffff815a04ee>] smp_apic_timer_interrupt+0x6e/0x99 [<ffffffff8159f16f>] apic_timer_interrupt+0x6f/0x80 <EOI> [<ffffffff81306a18>] ? intel_idle+0xc8/0x170 [<ffffffff81306a14>] ? intel_idle+0xc4/0x170 [<ffffffff81467107>] cpuidle_enter+0x17/0x20 [<ffffffff81467ab3>] cpuidle_idle_call+0x143/0x290 [<ffffffff810205c5>] cpu_idle+0xe5/0x140 [<ffffffff8158991d>] start_secondary+0xd1/0xd4 Code: 48 39 d0 0f 88 6b 01 00 00 89 d3 81 e3 ff 00 00 00 0f 84 35 01 00 00 48 8d 42 01 49 89 44 24 40 48 63 c3 48 83 c0 04 48 c1 e0 04 <4a> 8b 54 20 10 49 8d 4c 04 10 48 89 55 c0 4c 89 72 08 4a 8b 54
lockdep: fixing up alternatives.
Booting Node 1 Processor 12 APIC 0x11
sending NMI to all CPUs:
NMI backtrace for cpu 9
CPU 9
Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas
Pid: 16456, comm: migration/9 Tainted: G W 3.5.0-rc1-threadhotplugv3-paulfix #1 IBM IBM System x -[7870C4Q]-/68Y8033
RIP: 0010:[<ffffffff810e61d5>] [<ffffffff810e61d5>] stop_machine_cpu_stop+0x85/0x140 RSP: 0018:ffff8808d2f93d20 EFLAGS: 00000293 RAX: 0000000000000001 RBX: 0000000000000001 RCX: ffff8808d2da0400 RDX: ffff8808d2da0400 RSI: 0000000000000001 RDI: ffff8808d2ddfdf8 RBP: ffff8808d2f93d50 R08: ffff8808ffd4ebc0 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: ffff8808d2ddfdf8 R13: ffff8808d2ddfe1c R14: 0000000000000282 R15: ffff8808d2ddfd01 FS: 0000000000000000(0000) GS:ffff8808ffd40000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: ffffffffff600400 CR3: 0000000001a0c000 CR4: 00000000000007e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process migration/9 (pid: 16456, threadinfo ffff8808d2f92000, task ffff8808d2da0400) Stack: 00000000d2f93d30 ffff8808d2ddfe28 ffff8808d2da0400 ffff8808ffd4eb80 ffffffff810e6150 ffff8808d2ddfdf8 ffff8808d2f93e50 ffffffff810e5e53 ffff8808d2f92000 ffff8808d2f93fd8 ffff8808d2f92000 ffff8808ffd4ebc0 Call Trace: [<ffffffff810e6150>] ? stop_one_cpu_nowait+0x50/0x50 [<ffffffff810e5e53>] cpu_stopper_thread+0xf3/0x1e0 [<ffffffff81594f14>] ? _raw_spin_unlock_irqrestore+0x44/0x90 [<ffffffff810e5d60>] ? res_counter_init+0x60/0x60 [<ffffffff810c259d>] ? trace_hardirqs_on_caller+0x12d/0x1b0 [<ffffffff810c262d>] ? trace_hardirqs_on+0xd/0x10 [<ffffffff810e5d60>] ? res_counter_init+0x60/0x60 [<ffffffff810811ee>] kthread+0xee/0x100 [<ffffffff8159fa74>] kernel_thread_helper+0x4/0x10 [<ffffffff815952b0>] ? retint_restore_args+0x13/0x13 [<ffffffff81081100>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8159fa70>] ? gs_change+0x13/0x13 Code: 03 74 4b f0 41 ff 4d 00 0f 94 c0 84 c0 74 17 41 8b 54 24 20 41 8b 44 24 10 83 c2 01 41 89 44 24 24 41 89 54 24 20 83 fb 04 74 43 <89> d8 f3 90 41 8b 5c 24 20 39 c3 74 ee 83 fb 02 75 c1 fa 66 66 NMI backtrace for cpu 10 CPU 10 Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas
Pid: 16466, comm: migration/10 Tainted: G W 3.5.0-rc1-threadhotplugv3-paulfix #1 IBM IBM System x -[7870C4Q]-/68Y8033
RIP: 0010:[<ffffffff810e61d9>] [<ffffffff810e61d9>] stop_machine_cpu_stop+0x89/0x140 RSP: 0018:ffff8808cfa63d20 EFLAGS: 00000293 RAX: 0000000000000001 RBX: 0000000000000001 RCX: ffff8808cbcc81c0 RDX: ffff8808cbcc81c0 RSI: 0000000000000001 RDI: ffff8808d2ddfdf8 RBP: ffff8808cfa63d50 R08: ffff8808ffd8ebc0 R09: 0000000000000000 R10: 0000000000000000 R11: 000000000000000 R12: ffff8808d2ddfdf8 R13: ffff8808d2ddfe1c R14: 0000000000000282 R15: ffff8808d2ddfd01 FS: 0000000000000000(0000) GS:ffff8808ffd80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 00007fee875cf0a0 CR3: 0000000001a0c000 CR4: 00000000000007e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process migration/10 (pid: 16466, threadinfo ffff8808cfa62000, task ffff8808cbcc81c0) Stack: 00000000cfa63d30 ffff8808d2ddfe28 ffff8808cbcc81c0 ffff8808ffd8eb80 ffffffff810e6150 ffff8808d2ddfdf8 ffff8808cfa63e50 ffffffff810e5e53 ffff8808cfa62000 ffff8808cfa63fd8 ffff8808cfa62000 ffff8808ffd8ebc0 Call Trace: [<ffffffff810e6150>] ? stop_one_cpu_nowait+0x50/0x50 [<ffffffff810e5e53>] cpu_stopper_thread+0xf3/0x1e0 [<ffffffff81594f14>] ? _raw_spin_unlock_irqrestore+0x44/0x90 [<ffffffff810e5d60>] ? res_counter_init+0x60/0x60 [<ffffffff810c259d>] ? trace_hardirqs_on_caller+0x12d/0x1b0 [<ffffffff810c262d>] ? trace_hardirqs_on+0xd/0x10 [<ffffffff810e5d60>] ? res_counter_init+0x60/0x60 [<ffffffff810811ee>] kthread+0xee/0x100 [<ffffffff8159fa74>] kernel_thread_helper+0x4/0x10 [<ffffffff815952b0>] ? retint_restore_args+0x13/0x13 [<ffffffff81081100>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8159fa70>] ? gs_change+0x13/0x13 Code: 41 ff 4d 00 0f 94 c0 84 c0 74 17 41 8b 54 24 20 41 8b 44 24 10 83 c2 01 41 89 44 24 24 41 89 54 24 20 83 fb 04 74 43 89 d8 f3 90 <41> 8b 5c 24 20 39 c3 74 ee 83 fb 02 75 c1 fa 66 66 90 66 66 90 NMI backtrace for cpu 11 CPU 11 Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas
Pid: 16475, comm: migration/11 Tainted: G W 3.5.0-rc1-threadhotplugv3-paulfix #1 IBM IBM System x -[7870C4Q]-/68Y8033
RIP: 0010:[<ffffffff810e61d0>] [<ffffffff810e61d0>] stop_machine_cpu_stop+0x80/0x140 RSP: 0018:ffff8808ce103d20 EFLAGS: 00000246 RAX: 0000000000000001 RBX: 0000000000000001 RCX: ffff8808ce0a4640 RDX: ffff8808ce0a4640 RSI: 0000000000000001 RDI: ffff8808d2ddfdf8 RBP: ffff8808ce103d50 R08: ffff8808ffdcebc0 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: ffff8808d2ddfdf8 R13: ffff8808d2ddfe1c R14: 0000000000000282 R15: ffff8808d2ddfd01 FS: 0000000000000000(0000) GS:ffff8808ffdc0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 00007fee875cf0a0 CR3: 0000000001a0c000 CR4: 00000000000007e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process migration/11 (pid: 16475, threadinfo ffff8808ce102000, task ffff8808ce0a4640) Stack: 00000000ce103d30 ffff8808d2ddfe28 ffff8808ce0a4640 ffff8808ffdceb80 ffffffff810e6150 ffff8808d2ddfdf8 ffff8808ce103e50 ffffffff810e5e53 ffff8808ce102000 ffff8808ce103fd8 ffff8808ce102000 ffff8808ffdcebc0 Call Trace: [<ffffffff810e6150>] ? stop_one_cpu_nowait+0x50/0x50 [<ffffffff810e5e53>] cpu_stopper_thread+0xf3/0x1e0 [<ffffffff81594f14>] ? _raw_spin_unlock_irqrestore+0x44/0x90 [<ffffffff810e5d60>] ? res_counter_init+0x60/0x60 [<ffffffff810c259d>] ? trace_hardirqs_on_caller+0x12d/0x1b0 [<ffffffff810c262d>] ? trace_hardirqs_on+0xd/0x10 [<ffffffff810e5d60>] ? res_counter_init+0x60/0x60 [<ffffffff810811ee>] kthread+0xee/0x100 [<ffffffff8159fa74>] kernel_thread_helper+0x4/0x10 [<ffffffff815952b0>] ? retint_restore_args+0x13/0x13 [<ffffffff81081100>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8159fa70>] ? gs_change+0x13/0x13 Code: 44 00 00 83 fb 03 74 4b f0 41 ff 4d 00 0f 94 c0 84 c0 74 17 41 8b 54 24 20 41 8b 44 24 10 83 c2 01 41 89 44 24 24 41 89 54 24 20 <83> fb 04 74 43 89 d8 f3 90 41 8b 5c 24 20 39 c3 74 ee 83 fb 02 NMI backtrace for cpu 2 CPU 2 Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas
Pid: 16393, comm: migration/2 Tainted: G W 3.5.0-rc1-threadhotplugv3-paulfix #1 IBM IBM System x -[7870C4Q]-/68Y8033
RIP: 0010:[<ffffffff810e61d9>] [<ffffffff810e61d9>] stop_machine_cpu_stop+0x89/0x140 RSP: 0018:ffff8808cdaf1d20 EFLAGS: 00000293 RAX: 0000000000000001 RBX: 0000000000000001 RCX: ffff8808cb53a400 RDX: ffff8808cb53a400 RSI: 0000000000000001 RDI: ffff8808d2ddfdf8 RBP: ffff8808cdaf1d50 R08: ffff8808ffc8ebc0 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: ffff8808d2ddfdf8 R13: ffff8808d2ddfe1c R14: 0000000000000282 R15: ffff8808d2ddfd01 FS: 0000000000000000(0000) GS:ffff8808ffc80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 00007f97183ca000 CR3: 00000008c965d000 CR4: 00000000000007e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process migration/2 (pid: 16393, threadinfo ffff8808cdaf0000, task ffff8808cb53a400) Stack: 00000000cdaf1d30 ffff8808d2ddfe28 ffff8808cb53a400 ffff8808ffc8eb80 ffffffff810e6150 ffff8808d2ddfdf8 ffff8808cdaf1e50 ffffffff810e5e53 ffff8808cdaf0000 ffff8808cdaf1fd8 ffff8808cdaf0000 ffff8808ffc8ebc0 Call Trace: [<ffffffff810e6150>] ? stop_one_cpu_nowait+0x50/0x50 [<ffffffff810e5e53>] cpu_stopper_thread+0xf3/0x1e0 [<ffffffff81594f14>] ? _raw_spin_unlock_irqrestore+0x44/0x90 [<ffffffff810e5d60>] ? res_counter_init+0x60/0x60 [<ffffffff810c259d>] ? trace_hardirqs_on_caller+0x12d/0x1b0 [<ffffffff810c262d>] ? trace_hardirqs_on+0xd/0x10 [<ffffffff810e5d60>] ? res_counter_init+0x60/0x60 [<ffffffff810811ee>] kthread+0xee/0x100 [<ffffffff8159fa74>] kernel_thread_helper+0x4/0x10 [<ffffffff815952b0>] ? retint_restore_args+0x13/0x13 [<ffffffff81081100>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8159fa70>] ? gs_change+0x13/0x13 Code: 41 ff 4d 00 0f 94 c0 84 c0 74 17 41 8b 54 24 20 41 8b 44 24 10 83 c2 01 41 89 44 24 24 41 89 54 24 20 83 fb 04 74 43 89 d8 f3 90 <41> 8b 5c 24 20 39 c3 74 ee 83 fb 02 75 c1 fa 66 66 90 66 66 90 NMI backtrace for cpu 0 CPU 0 Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas
Pid: 6, comm: migration/0 Tainted: G W 3.5.0-rc1-threadhotplugv3-paulfix #1 IBM IBM System x -[7870C4Q]-/68Y8033
RIP: 0010:[<ffffffff810e61d9>] [<ffffffff810e61d9>] stop_machine_cpu_stop+0x89/0x140 RSP: 0018:ffff8808d2d83d20 EFLAGS: 00000293 RAX: 0000000000000001 RBX: 0000000000000001 RCX: ffff8808d2d80180 RDX: ffff8808d2d80180 RSI: 0000000000000001 RDI: ffff8808d2ddfdf8 RBP: ffff8808d2d83d50 R08: ffff8808ffc0ebc0 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: ffff8808d2ddfdf8 R13: ffff8808d2ddfe1c R14: 0000000000000282 R15: ffff8808d2ddfd01 FS: 0000000000000000(0000) GS:ffff8808ffc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: ffffffffff600400 CR3: 0000000001a0c000 CR4: 00000000000007f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process migration/0 (pid: 6, threadinfo ffff8808d2d82000, task ffff8808d2d80180) Stack: 00000000d2d83d30 ffff8808d2ddfe28 ffff8808d2d80180 ffff8808ffc0eb80 ffffffff810e6150 ffff8808d2ddfdf8 ffff8808d2d83e50 ffffffff810e5e53 ffff8808d2d82000 ffff8808d2d83fd8 ffff8808d2d82000 ffff8808ffc0ebc0 Call Trace: [<ffffffff810e6150>] ? stop_one_cpu_nowait+0x50/0x50 [<ffffffff810e5e53>] cpu_stopper_thread+0xf3/0x1e0 [<ffffffff81594f14>] ? _raw_spin_unlock_irqrestore+0x44/0x90 [<ffffffff810e5d60>] ? res_counter_init+0x60/0x60 [<ffffffff810c259d>] ? trace_hardirqs_on_caller+0x12d/0x1b0 [<ffffffff810c262d>] ? trace_hardirqs_on+0xd/0x10 [<ffffffff810e5d60>] ? res_counter_init+0x60/0x60 [<ffffffff810811ee>] kthread+0xee/0x100 [<ffffffff8159fa74>] kernel_thread_helper+0x4/0x10 [<ffffffff815952b0>] ? retint_restore_args+0x13/0x13 [<ffffffff81081100>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8159fa70>] ? gs_change+0x13/0x13 Code: 41 ff 4d 00 0f 94 c0 84 c0 74 17 41 8b 54 24 20 41 8b 44 24 10 83 c2 01 41 89 44 24 24 41 89 54 24 20 83 fb 04 74 43 89 d8 f3 90 <41> 8b 5c 24 20 39 c3 74 ee 83 fb 02 75 c1 fa 66 66 90 66 66 90 NMI backtrace for cpu 3 CPU 3 Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas
Pid: 0, comm: swapper/3 Tainted: G W 3.5.0-rc1-threadhotplugv3-paulfix #1 IBM IBM System x -[7870C4Q]-/68Y8033
RIP: 0010:[<ffffffff8137c0f8>] [<ffffffff8137c0f8>] io_serial_in+0x18/0x20 RSP: 0018:ffff8808ffcc3bc8 EFLAGS: 00000002 RAX: ffff8808d2db7f00 RBX: ffffffff826876b0 RCX: 0000000000000000 RDX: 00000000000002fd RSI: 0000000000000005 RDI: ffffffff826876b0 RBP: ffff8808ffcc3bc8 R08: 0000000067ca504e R09: 0000000000000000 R10: 0000000000000001 R11: 0000000000001191 R12: 0000000000002709 R13: 0000000000000020 R14: 00000000000000c9 R15: ffffffff826876b0 FS: 0000000000000000(0000) GS:ffff8808ffcc0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: ffffffffff600400 CR3: 0000000001a0c000 CR4: 00000000000007e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process swapper/3 (pid: 0, threadinfo ffff8808d2db6000, task ffff8808d2db4280) Stack: ffff8808ffcc3bf8 ffffffff8137c96b ffffffff812aff78 ffffffff826876b0 0000000000000030 ffffffff81d1f740 ffff8808ffcc3c18 ffffffff8137c9f6 000000000000000a ffffffff8137c9d0 ffff8808ffcc3c58 ffffffff81377fbd Call Trace: <IRQ> [<ffffffff8137c96b>] wait_for_xmitr+0x3b/0xa0 [<ffffffff812aff78>] ? delay_tsc+0x78/0xc0 [<ffffffff8137c9f6>] serial8250_console_putchar+0x26/0x40 [<ffffffff8137c9d0>] ? wait_for_xmitr+0xa0/0xa0 [<ffffffff81377fbd>] uart_console_write+0x3d/0x70 [<ffffffff8137f423>] serial8250_console_write+0xc3/0x1a0 [<ffffffff810577b7>] call_console_drivers+0xa7/0x1b0 [<ffffffff81057ced>] console_unlock+0x10d/0x220 [<ffffffff81058436>] vprintk_emit+0x1a6/0x610 [<ffffffff8159128c>] printk+0x4f/0x51 [<ffffffff8159128c>] ? printk+0x4f/0x51 [<ffffffff8103b84a>] arch_trigger_all_cpu_backtrace+0x5a/0xb0 [<ffffffff810fe8a9>] print_cpu_stall+0x79/0x130 [<ffffffff8110064e>] __rcu_pending+0xee/0x210 [<ffffffff81100878>] rcu_check_callbacks+0x108/0x310 [<ffffffff81068c66>] update_process_times+0x46/0x90 [<ffffffff810bbf66>] tick_sched_timer+0x66/0xc0 [<ffffffff810bbf00>] ? tick_nohz_handler+0x100/0x100 [<ffffffff81086a4e>] __run_hrtimer+0x12e/0x330 [<ffffffff81086e9f>] hrtimer_interrupt+0xcf/0x200 [<ffffffff815a04e9>] smp_apic_timer_interrupt+0x69/0x99 [<ffffffff8159f16f>] apic_timer_interrupt+0x6f/0x80 <EOI> [<ffffffff81306a18>] ? intel_idle+0xc8/0x170 [<ffffffff81306a14>] ? intel_idle+0xc4/0x170 [<ffffffff81467107>] cpuidle_enter+0x17/0x20 [<ffffffff81467ab3>] cpuidle_idle_call+0x143/0x290 [<ffffffff810205c5>] cpu_idle+0xe5/0x140 [<ffffffff8158991d>] start_secondary+0xd1/0xd4 Code: 00 00 00 d3 e6 48 63 f6 48 03 77 40 8b 06 c9 c3 0f 1f 00 55 48 89 e5 66 66 66 66 90 0f b6 8f 91 00 00 00 89 f2 d3 e2 03 57 38 ec <0f> b6 c0 c9 c3 0f 1f 00 55 48 89 e5 66 66 66 66 90 0f b6 8f 91 NMI backtrace for cpu 1 CPU 1 Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas
Pid: 16384, comm: migration/1 Tainted: G W 3.5.0-rc1-threadhotplugv3-paulfix #1 IBM IBM System x -[7870C4Q]-/68Y8033
RIP: 0010:[<ffffffff810e61d9>] [<ffffffff810e61d9>] stop_machine_cpu_stop+0x89/0x140 RSP: 0018:ffff8808cda55d20 EFLAGS: 00000293 RAX: 0000000000000001 RBX: 0000000000000001 RCX: ffff8808d0db23c0 RDX: ffff8808d0db23c0 RSI: 0000000000000001 RDI: ffff8808d2ddfdf8 RBP: ffff8808cda55d50 R08: ffff8808ffc4ebc0 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: ffff8808d2ddfdf8 R13: ffff8808d2ddfe1c R14: 0000000000000282 R15: ffff8808d2ddfd01 FS: 0000000000000000(0000) GS:ffff8808ffc40000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 00007f8d3cfa8570 CR3: 0000000001a0c000 CR4: 00000000000007e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process migration/1 (pid: 16384, threadinfo ffff8808cda54000, task ffff8808d0db23c0) Stack: 00000000cda55d30 ffff8808d2ddfe28 ffff8808d0db23c0 ffff8808ffc4eb80 ffffffff810e6150 ffff8808d2ddfdf8 ffff8808cda55e50 ffffffff810e5e53 ffff8808cda54000 ffff8808cda55fd8 ffff8808cda54000 ffff8808ffc4ebc0 Call Trace: [<ffffffff810e6150>] ? stop_one_cpu_nowait+0x50/0x50 [<ffffffff810e5e53>] cpu_stopper_thread+0xf3/0x1e0 [<ffffffff81594f14>] ? _raw_spin_unlock_irqrestore+0x44/0x90 [<ffffffff810e5d60>] ? res_counter_init+0x60/0x60 [<ffffffff810c259d>] ? trace_hardirqs_on_caller+0x12d/0x1b0 [<ffffffff810c262d>] ? trace_hardirqs_on+0xd/0x10 [<ffffffff810e5d60>] ? res_counter_init+0x60/0x60 [<ffffffff810811ee>] kthread+0xee/0x100 [<ffffffff8159fa74>] kernel_thread_helper+0x4/0x10 [<ffffffff815952b0>] ? retint_restore_args+0x13/0x13 [<ffffffff81081100>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8159fa70>] ? gs_change+0x13/0x13 Code: 41 ff 4d 00 0f 94 c0 84 c0 74 17 41 8b 54 24 20 41 8b 44 24 10 83 c2 01 41 89 44 24 24 41 89 54 24 20 83 fb 04 74 43 89 d8 f3 90 <41> 8b 5c 24 20 39 c3 74 ee 83 fb 02 75 c1 fa 66 66 90 66 66 90 NMI backtrace for cpu 8 CPU 8 Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas
Pid: 16447, comm: migration/8 Tainted: G W 3.5.0-rc1-threadhotplugv3-paulfix #1 IBM IBM System x -[7870C4Q]-/68Y8033
RIP: 0010:[<ffffffff810e61d9>] [<ffffffff810e61d9>] stop_machine_cpu_stop+0x89/0x140 RSP: 0018:ffff8808d0819d20 EFLAGS: 00000293 RAX: 0000000000000001 RBX: 0000000000000001 RCX: ffff8808cf92e040 RDX: ffff8808cf92e040 RSI: 0000000000000001 RDI: ffff8808d2ddfdf8 RBP: ffff8808d0819d50 R08: ffff8808ffd0ebc0 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: ffff8808d2ddfdf8 R13: ffff8808d2ddfe1c R14: 0000000000000282 R15: ffff8808d2ddfd01 FS: 0000000000000000(0000) GS:ffff8808ffd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 00007fee875cf0a0 CR3: 0000000001a0c000 CR4: 00000000000007e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process migration/8 (pid: 16447, threadinfo ffff8808d0818000, task ffff8808cf92e040) Stack: 00000000d0819d30 ffff8808d2ddfe28 ffff8808cf92e040 ffff8808ffd0eb80 ffffffff810e6150 ffff8808d2ddfdf8 ffff8808d0819e50 ffffffff810e5e53 ffff8808d0818000 ffff8808d0819fd8 ffff8808d0818000 ffff8808ffd0ebc0 Call Trace: [<ffffffff810e6150>] ? stop_one_cpu_nowait+0x50/0x50 [<ffffffff810e5e53>] cpu_stopper_thread+0xf3/0x1e0 [<ffffffff81594f14>] ? _raw_spin_unlock_irqrestore+0x44/0x90 [<ffffffff810e5d60>] ? res_counter_init+0x60/0x60 [<ffffffff810c259d>] ? trace_hardirqs_on_caller+0x12d/0x1b0 [<ffffffff810c262d>] ? trace_hardirqs_on+0xd/0x10 [<ffffffff810e5d60>] ? res_counter_init+0x60/0x60 [<ffffffff810811ee>] kthread+0xee/0x100 [<ffffffff8159fa74>] kernel_thread_helper+0x4/0x10 [<ffffffff815952b0>] ? retint_restore_args+0x13/0x13 [<ffffffff81081100>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8159fa70>] ? gs_change+0x13/0x13 Code: 41 ff 4d 00 0f 94 c0 84 c0 74 17 41 8b 54 24 20 41 8b 44 24 10 83 c2 01 41 89 44 24 24 41 89 54 24 20 83 fb 04 74 43 89 d8 f3 90 <41> 8b 5c 24 20 39 c3 74 ee 83 fb 02 75 c1 fa 66 66 90 66 66 90 NMI backtrace for cpu 4 CPU 4 Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas
Pid: 16411, comm: migration/4 Tainted: G W 3.5.0-rc1-threadhotplugv3-paulfix #1 IBM IBM System x -[7870C4Q]-/68Y8033
RIP: 0010:[<ffffffff810e61d9>] [<ffffffff810e61d9>] stop_machine_cpu_stop+0x89/0x140 RSP: 0018:ffff88114d66dd20 EFLAGS: 00000293 RAX: 0000000000000001 RBX: 0000000000000001 RCX: ffff881152f0aa00 RDX: ffff881152f0aa00 RSI: 0000000000000001 RDI: ffff8808d2ddfdf8 RBP: ffff88114d66dd50 R08: ffff88117fc0ebc0 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: ffff8808d2ddfdf8 R13: ffff8808d2ddfe1c R14: 0000000000000282 R15: ffff8808d2ddfd01 FS: 0000000000000000(0000) GS:ffff88117fc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 00007f9f8b3ba000 CR3: 0000000001a0c000 CR4: 00000000000007e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process migration/4 (pid: 16411, threadinfo ffff88114d66c000, task ffff881152f0aa00) Stack: 000000004d66dd30 ffff8808d2ddfe28 ffff881152f0aa00 ffff88117fc0eb80 ffffffff810e6150 ffff8808d2ddfdf8 ffff88114d66de50 ffffffff810e5e53 ffff88114d66c000 ffff88114d66dfd8 ffff88114d66c000 ffff88117fc0ebc0 Call Trace: [<ffffffff810e6150>] ? stop_one_cpu_nowait+0x50/0x50 [<ffffffff810e5e53>] cpu_stopper_thread+0xf3/0x1e0 [<ffffffff81594f14>] ? _raw_spin_unlock_irqrestore+0x44/0x90 [<ffffffff810e5d60>] ? res_counter_init+0x60/0x60 [<ffffffff810c259d>] ? trace_hardirqs_on_caller+0x12d/0x1b0 [<ffffffff810c262d>] ? trace_hardirqs_on+0xd/0x10 [<ffffffff810e5d60>] ? res_counter_init+0x60/0x60 [<ffffffff810811ee>] kthread+0xee/0x100 [<ffffffff8159fa74>] kernel_thread_helper+0x4/0x10 [<ffffffff815952b0>] ? retint_restore_args+0x13/0x13 [<ffffffff81081100>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8159fa70>] ? gs_change+0x13/0x13 Code: 41 ff 4d 00 0f 94 c0 84 c0 74 17 41 8b 54 24 20 41 8b 44 24 10 83 c2 01 41 89 44 24 24 41 89 54 24 20 83 fb 04 74 43 89 d8 f3 90 <41> 8b 5c 24 20 39 c3 74 ee 83 fb 02 75 c1 fa 66 66 90 66 66 90 NMI backtrace for cpu 5 CPU 5 Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas
Pid: 16420, comm: migration/5 Tainted: G W 3.5.0-rc1-threadhotplugv3-paulfix #1 IBM IBM System x -[7870C4Q]-/68Y8033
RIP: 0010:[<ffffffff810e61d0>] [<ffffffff810e61d0>] stop_machine_cpu_stop+0x80/0x140 RSP: 0018:ffff881150bfdd20 EFLAGS: 00000246 RAX: 0000000000000001 RBX: 0000000000000001 RCX: ffff881152cc2780 RDX: ffff881152cc2780 RSI: 0000000000000001 RDI: ffff8808d2ddfdf8 RBP: ffff881150bfdd50 R08: ffff88117fc4ebc0 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: ffff8808d2ddfdf8 R13: ffff8808d2ddfe1c R14: 0000000000000282 R15: ffff8808d2ddfd01 FS: 0000000000000000(0000) GS:ffff88117fc40000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 00007f9f8b3ba000 CR3: 0000000001a0c000 CR4: 00000000000007e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process migration/5 (pid: 16420, threadinfo ffff881150bfc000, task ffff881152cc2780) Stack: 0000000050bfdd30 ffff8808d2ddfe28 ffff881152cc2780 ffff88117fc4eb80 ffffffff810e6150 ffff8808d2ddfdf8 ffff881150bfde50 ffffffff810e5e53 ffff881150bfc000 ffff881150bfdfd8 ffff881150bfc000 ffff88117fc4ebc0 Call Trace: [<ffffffff810e6150>] ? stop_one_cpu_nowait+0x50/0x50 [<ffffffff810e5e53>] cpu_stopper_thread+0xf3/0x1e0 [<ffffffff81594f14>] ? _raw_spin_unlock_irqrestore+0x44/0x90 [<ffffffff810e5d60>] ? res_counter_init+0x60/0x60 [<ffffffff810c259d>] ? trace_hardirqs_on_caller+0x12d/0x1b0 [<ffffffff810c262d>] ? trace_hardirqs_on+0xd/0x10 [<ffffffff810e5d60>] ? res_counter_init+0x60/0x60 [<ffffffff810811ee>] kthread+0xee/0x100 [<ffffffff8159fa74>] kernel_thread_helper+0x4/0x10 [<ffffffff815952b0>] ? retint_restore_args+0x13/0x13 [<ffffffff81081100>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8159fa70>] ? gs_change+0x13/0x13 Code: 44 00 00 83 fb 03 74 4b f0 41 ff 4d 00 0f 94 c0 84 c0 74 17 41 8b 54 24 20 41 8b 44 24 10 83 c2 01 41 89 44 24 24 41 89 54 24 20 <83> fb 04 74 43 89 d8 f3 90 41 8b 5c 24 20 39 c3 74 ee 83 fb 02 NMI backtrace for cpu 7 CPU 7 Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas
Pid: 16438, comm: migration/7 Tainted: G W 3.5.0-rc1-threadhotplugv3-paulfix #1 IBM IBM System x -[7870C4Q]-/68Y8033
RIP: 0010:[<ffffffff810e61d0>] [<ffffffff810e61d0>] stop_machine_cpu_stop+0x80/0x140 RSP: 0018:ffff88114d437d20 EFLAGS: 00000246 RAX: 0000000000000001 RBX: 0000000000000001 RCX: ffff881152fac900 RDX: ffff881152fac900 RSI: 0000000000000001 RDI: ffff8808d2ddfdf8 RBP: ffff88114d437d50 R08: ffff88117fccebc0 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: ffff8808d2ddfdf8 R13: ffff8808d2ddfe1c R14: 0000000000000282 R15: ffff8808d2ddfd01 FS: 0000000000000000(0000) GS:ffff88117fcc0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 00007f63164d8dd8 CR3: 0000000001a0c000 CR4: 00000000000007e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process migration/7 (pid: 16438, threadinfo ffff88114d436000, task ffff881152fac900) Stack: 000000004d437d30 ffff8808d2ddfe28 ffff881152fac900 ffff88117fcceb80 ffffffff810e6150 ffff8808d2ddfdf8 ffff88114d437e50 ffffffff810e5e53 ffff88114d436000 ffff88114d437fd8 ffff88114d436000 ffff88117fccebc0 Call Trace: [<ffffffff810e6150>] ? stop_one_cpu_nowait+0x50/0x50 [<ffffffff810e5e53>] cpu_stopper_thread+0xf3/0x1e0 [<ffffffff81594f14>] ? _raw_spin_unlock_irqrestore+0x44/0x90 [<ffffffff810e5d60>] ? res_counter_init+0x60/0x60 [<ffffffff810c259d>] ? trace_hardirqs_on_caller+0x12d/0x1b0 [<ffffffff810c262d>] ? trace_hardirqs_on+0xd/0x10 [<ffffffff810e5d60>] ? res_counter_init+0x60/0x60 [<ffffffff810811ee>] kthread+0xee/0x100 [<ffffffff8159fa74>] kernel_thread_helper+0x4/0x10 [<ffffffff815952b0>] ? retint_restore_args+0x13/0x13 [<ffffffff81081100>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8159fa70>] ? gs_change+0x13/0x13 Code: 44 00 00 83 fb 03 74 4b f0 41 ff 4d 00 0f 94 c0 84 c0 74 17 41 8b 54 24 20 41 8b 44 24 10 83 c2 01 41 89 44 24 24 41 89 54 24 20 <83> fb 04 74 43 89 d8 f3 90 41 8b 5c 24 20 39 c3 74 ee 83 fb 02 NMI backtrace for cpu 6 CPU 6 Modules linked in: ipv6 cpufreq_ondemand cpufreq_powersave acpi_cpufreq freq_table mperf dm_mod ioatdma cdc_ether usbnet mii bnx2 i7core_edac kvm_intel i2c_i801 dca shpchp edac_core i2c_core sg kvm lpc_ich mfd_core microcode serio_raw sd_mod crc_t10dif ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas
Pid: 16429, comm: migration/6 Tainted: G W 3.5.0-rc1-threadhotplugv3-paulfix #1 IBM IBM System x -[7870C4Q]-/68Y8033
RIP: 0010:[<ffffffff810e61d5>] [<ffffffff810e61d5>] stop_machine_cpu_stop+0x85/0x140 RSP: 0000:ffff881152e93d20 EFLAGS: 00000293 RAX: 0000000000000001 RBX: 0000000000000001 RCX: ffff881152664840 RDX: ffff881152664840 RSI: 0000000000000001 RDI: ffff8808d2ddfdf8 RBP: ffff881152e93d50 R08: ffff88117fc8ebc0 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: ffff8808d2ddfdf8 R13: ffff8808d2ddfe1c R14: 0000000000000282 R15: ffff8808d2ddfd01 FS: 0000000000000000(0000) GS:ffff88117fc80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000001a0c000 CR4: 00000000000007e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process migration/6 (pid: 16429, threadinfo ffff881152e92000, task ffff881152664840) Stack: 0000000052e93d30 ffff8808d2ddfe28 ffff881152664840 ffff88117fc8eb80 ffffffff810e6150 ffff8808d2ddfdf8 ffff881152e93e50 ffffffff810e5e53 ffff881152e92000 ffff881152e93fd8 ffff881152e92000 ffff88117fc8ebc0 Call Trace: [<ffffffff810e6150>] ? stop_one_cpu_nowait+0x50/0x50 [<ffffffff810e5e53>] cpu_stopper_thread+0xf3/0x1e0 [<ffffffff81594f14>] ? _raw_spin_unlock_irqrestore+0x44/0x90 [<ffffffff810e5d60>] ? res_counter_init+0x60/0x60 [<ffffffff810c259d>] ? trace_hardirqs_on_caller+0x12d/0x1b0 [<ffffffff810c262d>] ? trace_hardirqs_on+0xd/0x10 [<ffffffff810e5d60>] ? res_counter_init+0x60/0x60 [<ffffffff810811ee>] kthread+0xee/0x100 [<ffffffff8159fa74>] kernel_thread_helper+0x4/0x10 [<ffffffff815952b0>] ? retint_restore_args+0x13/0x13 [<ffffffff81081100>] ? __init_kthread_worker+0x70/0x70 [<ffffffff8159fa70>] ? gs_change+0x13/0x13 Code: 03 74 4b f0 41 ff 4d 00 0f 94 c0 84 c0 74 17 41 8b 54 24 20 41 8b 44 24 10 83 c2 01 41 89 44 24 24 41 89 54 24 20 83 fb 04 74 43 <89> d8 f3 90 41 8b 5c 24 20 39 c3 74 ee 83 fb 02 75 c1 fa 66 66
Uhhuh. NMI received for unknown reason 3d on CPU 2. Uhhuh. NMI received for unknown reason 2d on CPU 3. Do you have a strange power saving mode enabled? Dazed and confused, but trying to continue Uhhuh. NMI received for unknown reason 3d on CPU 4. Do you have a strange power saving mode enabled? Dazed and confused, but trying to continue Uhhuh. NMI received for unknown reason 3d on CPU 5. Do you have a strange power saving mode enabled? Dazed and confused, but trying to continue Uhhuh. NMI received for unknown reason 2d on CPU 6. Do you have a strange power saving mode enabled? Dazed and confused, but trying to continue Uhhuh. NMI received for unknown reason 2d on CPU 7. Do you have a strange power saving mode enabled? Dazed and confused, but trying to continue Uhhuh. NMI received for unknown reason 3d on CPU 8. Do you have a strange power saving mode enabled? Dazed and confused, but trying to continue Uhhuh. NMI received for unknown reason 2d on CPU 9. Do you have a strange power saving mode enabled? Dazed and confused, but trying to continue Uhhuh. NMI received for unknown reason 2d on CPU 10. Do you have a strange power saving mode enabled? Dazed and confused, but trying to continue Uhhuh. NMI received for unknown reason 3d on CPU 11. Do you have a strange power saving mode enabled? Dazed and confused, but trying to continue Do you have a strange power saving mode enabled? Dazed and confused, but trying to continue NMI watchdog: enabled, takes one hw-pmu counter. Booting Node 1 Processor 12 APIC 0x11
NMI watchdog: enabled, takes one hw-pmu counter.
INFO: task stress_cpu_hotp:6903 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.stress_cpu_hotp D 0000000000000000 0 6903 6850 0x00000000
ffff8808cda09ad8 0000000000000046 0000000000000001 ffff8808cda08010 ffff8808cda08000 ffff8808cda08000 ffff8808cda08000 ffff8808cda09fd8 ffff8808cda08000 ffff8808cda09fd8 ffff8808d04f45c0 ffff8808d2dbc300
Call Trace:
[<ffffffff81465164>] ? cpufreq_notify_transition+0x174/0x2f0
[<ffffffffa00b1200>] ? acpi_cpufreq_cpu_exit+0xb0/0xb0 [acpi_cpufreq]
[<ffffffff810c21f0>] ? mark_held_locks+0x80/0x120
[<ffffffff81591ab7>] ? __mutex_unlock_slowpath+0xf7/0x190
[<ffffffff815939c9>] schedule+0x29/0x70
[<ffffffff815915e5>] schedule_timeout+0x2d5/0x3f0
[<ffffffff810c21f0>] ? mark_held_locks+0x80/0x120
[<ffffffff81594eb0>] ? _raw_spin_unlock_irq+0x30/0x50
[<ffffffff810c259d>] ? trace_hardirqs_on_caller+0x12d/0x1b0
[<ffffffff81593845>] wait_for_common+0x115/0x190
[<ffffffff810983d0>] ? try_to_wake_up+0x2f0/0x2f0
[<ffffffff81078bd2>] ? queue_delayed_work_on+0x132/0x140
[<ffffffff8159399d>] wait_for_completion+0x1d/0x20
[<ffffffff81043839>] hpet_cpuhp_notify+0x119/0x160
[<ffffffff81043a40>] ? init_one_hpet_msi_clockevent+0x1c0/0x1c0
[<ffffffff810784b0>] ? __queue_work+0x500/0x500
[<ffffffff81599512>] notifier_call_chain+0x72/0x130
[<ffffffff810890fe>] __raw_notifier_call_chain+0xe/0x10
[<ffffffff8158af88>] _cpu_up+0x15b/0x164
[<ffffffff8158b04d>] cpu_up+0xbc/0x11f
[<ffffffff8157a87d>] store_online+0x9d/0xe0
[<ffffffff81396be0>] dev_attr_store+0x20/0x30
[<ffffffff81222119>] sysfs_write_file+0xd9/0x150
[<ffffffff811a6a5b>] vfs_write+0xcb/0x130
[<ffffffff811a6bb5>] sys_write+0x55/0x90
[<ffffffff8159e629>] system_call_fastpath+0x16/0x1b
5 locks held by stress_cpu_hotp/6903:
#0: (&buffer->mutex){+.+.+.}, at: [<ffffffff81222089>] sysfs_write_file+0x49/0x150
#1: (s_active#111){.+.+.+}, at: [<ffffffff81222102>] sysfs_write_file+0xc2/0x150
#2: (x86_cpu_hotplug_driver_mutex){+.+.+.}, at: [<ffffffff81039df7>] cpu_hotplug_driver_lock+0x17/0x20
#3: (cpu_add_remove_lock){+.+.+.}, at: [<ffffffff8158b009>] cpu_up+0x78/0x11f
#4: (cpu_hotplug.lock){+.+.+.}, at: [<ffffffff81059cef>] cpu_hotplug_begin+0x2f/0x60lockdep: fixing up alternatives.
Booting Node 1 Processor 13 APIC 0x13
CPU 1 is now offline
CPU 9 MCA banks CMCI:2 CMCI:3 CMCI:5
NOHZ: local_softirq_pending 200
NOHZ: local_softirq_pending 200
2012/7/16 Thomas Gleixner <[email protected]>:
Quoted 7 lines
The following series implements the infrastructure for parking and unparking kernel threads to avoid the full teardown and fork on cpu hotplug operations along with management infrastructure for hotplug and users.
Changes vs. V2:
Quoted 8 lines
Use callbacks for all functionality. Thanks to Rusty for pointing that out. It makes the use sites nice and simple and keeps all the code which would be duplicated otherwise on the core.
Thanks,
tglx
I'm just a kernel newbie, so I have a dumb question.
Why this patchset doesn't include patch for migrationd in kernel/stop_machine.c? Is there any reason for that?