2.6.2-rc2-mm2 CPU Hotplug: The Core
Subsystems (12)
PATCH 3/4 2.6.2-rc2-mm2 CPU Hotplug: The Core
The actual CPU patch. It's big, but almost all under CONFIG_HOTPLUG_CPU, or macros which have same effect.
Particularly interested in Nick's views here: I tried to catch all the places where the new scheduler might pull tasks onto an offline cpu.
Name: Hotplug CPU Patch: -mm Version
Author: Almost Everyone on the lhcs List
Status: Experimental
Depends: Hotcpu/use-kthread-simple.patch.gz
Depends: Hotcpu/drain_local_pages.patch.gz
Depends: Hotcpu/kstat-notifier-removal.patch.gz
Depends: Hotcpu/cpucontrol_macros.patch.gz
Depends: Module/module_kthread.patch.gz
Depends: Hotcpu/cpu_prepare.patch.gz
Depends: Hotcpu/zero-initializer-notifier-removal.patch.gz
Depends: Hotcpu/workqueue-cleanup.patch.gz
Depends: Hotcpu/kmod-affinity.patch.gz
Version: -mm
D: This is the arch-indep hotplug cpu code, contributed by Matt
D: Fleming, Zwane Mwaikambo, Dipankar Sarma, Joel Schopp,
D: Vatsa Vaddagiri and me.
D:
D: Changes are designed to be NOOPs: usually by explicit use of
D: CONFIG_HOTPLUG_CPU
D:
D: The way CPUs go down is:
D: 1) Userspace writes 0 to sysfs ("online" attrib).
D: 2) The cpucontrol mutex is grabbed.
D: 3) The task moves onto the cpu.
D: 4) cpu_online(cpu) is set to false.
D: 5) Tasks are migrated, cpus_allowed broken if required (not kernel threads)
D: 6) We move back off the cpu.
D: 7) CPU_OFFLINE notifier is called for the cpu (kernel threads clean up).
D: 8) Arch-specific __cpu_die() is called.
D: 9) CPU_DEAD notifier is called (caches drained, etc).
D: 10) hotplug script is run
D: 11) cpucontrol mutex released.
D:
D: CPUs go up as before, except with CONFIG_HOTPLUG_CPU, they can go
D: up after boot.
D:
D: Changes are:
D: - drivers/base/cpu.c: Add an "online" attribute.
D:
D: - drivers/scsi/scsi.c: drain the scsi_done_q into current CPU on CPU_DEAD D:
D: - fs/buffer.c: release buffer heads from bh_lrus on CPU_DEAD D:
D: - linux/cpu.h: Add cpu_down(). Add hotcpu_notifier convenience macro. D: Add cpu_is_offline() for when you know a cpu was (once) online: this D: is a constant if !CONFIG_HOTPLUG_CPU.
D:
D: - linux/interrupt.h: Add tasklet_kill_immediate for RCU tasklet takedown
D: on dead CPU.
D:
D: - linux/mmzip.h: keep pointer to kswapd, in case we need to move it. D:
D: - linux/sched.h: Add migrate_all_tasks() for cpu-down. Add migrate_to_cpu
D: for kernel threads to escape. Add wake_idle_cpu() for i386 hotplug code.
D:
D: - kernel/cpu.c: Take cpucontrol lock around notifiers: notifiers have D: no locking themselves. Implement cpu_down() and code to call D: sbin_hotplug. D: D: - kernel/kmod.c: As a bound kernel thread, take care of case where D: thread is going down, before keventd() CPU_OFFLINE notifier can D: return.
D:
D: - kernel/kthread.c: Same as kmod.c, take care of race where CPU goes
D: down and we didn't get migrated.
D:
D: - kernel/rcupdate.c: Move rcu batch when cpu dies.
D:
D: - kernel/sched.c: Add wake_idle_cpu() for i386 hotplug. D:
D: Check everywhere to make sure we never move tasks onto an D: offline cpu: we'll just be fighting migrate_all_tasks(). D: D: Change sched_migrate_task() to migrate_to_cpu and expose it D: for hotplug cpu. D: D: Take hotplug lock in sys_sched_setaffinity. D: D: Return cpus_allowed masked by cpu_possible_map, not D: cpu_online_map in sys_sched_getaffinity, otherwise tasks can't D: know about whether they can run on currently-offline cpus.
D:
D: Implement migrate_all_tasks() to push tasks off the dying cpu.
D:
D: Add callbacks to stop migration thread.
D:
D: - kernel/softirq.c:
D: Handle case where interrupt tries to kick softirqd after it has exited.
D:
D: Implement tasklet_kill_immediate for RCU.
D:
D: Code to shut down ksoftirqd, and take over dead cpu's tasklets.
D:
D: - kernel/timer.c:
D: Code to migrate timers off dead CPU.
D:
D: - kernel/workqueue.c: D: Keep all workqueue is list, with their name: when cpus come up D: we need to create new threads for each one.
D:
D: Lock cpu hotplug in flush_workqueue for simplicity.
D:
D: create_workqueue_thread doesn't need name arg, now in struct.
D:
D: Lock cpu hotplug when creating and destroying workqueues.
D:
D: cleanup_workqueue_thread needs to block irqs: can now be called D: on live workqueues.
D:
D: Implement callbacks to add and delete threads, and take over work.
D:
D: - mm/page_alloc.c:
D: Drain local pages on CPU which is dead.
D:
D: - mm/slab.c:
D: Move free_block decl, ac_entry and ac_data earlier in file.
D:
D: Stop reap timer on as cpu goes offline.
D:
D: Neaten list_for_each into list_for_each_entry.
D:
D: Free cache blocks and correct stats one CPU is dead.
D:
D: Make reap_timer_fnc terminate if cpu goes offline.
D:
D: - mm/swap.c:
D: Fix up committed stats and drain lru cache when cpu dead.
D:
D: - mm/vmscan.c: D: Keep kswapd on cpus within node unless all CPUs go offline, and D: restore when they come back.
D:
D: Record kswapd tasks for each pgdat upon creation.
D:
D: - net/core/dev.c:
D: Drain skb queues to this cpu when another cpu is dead.
D:
D: - net/core/flow.c:
D: Call __flow_cache_shrink() to shrink cache to zero when CPU dead.
drivers/base/cpu.c +43 -0
@@ -7,6 +7,7 @@#include <linux/init.h> #include <linux/cpu.h> #include <linux/topology.h> +#include <linux/device.h> struct sysdev_class cpu_sysdev_class = {
@@ -14,6 +15,46 @@ struct sysdev_class cpu_sysdev_class = { };EXPORT_SYMBOL (cpu_sysdev_class); +#ifdef CONFIG_HOTPLUG_CPU +static ssize_t show_online (struct sys_device *dev,char *buf) +{ + struct cpu *cpu = container_of(dev, struct cpu, sysdev); + +return sprintf (buf,"%u\n" , !!cpu_online (cpu->sysdev.id)); +} + +static ssize_t store_online (struct sys_device *dev,const char *buf, +size_t count) +{ + struct cpu *cpu = container_of(dev, struct cpu, sysdev); +ssize_t ret; + +switch (buf[0 ]) { +case '0' : + ret =cpu_down (cpu->sysdev.id); +break ; +case '1' : + ret =cpu_up (cpu->sysdev.id); +break ; +default : + ret = -EINVAL ; + } + +if (ret >=0 ) + ret = count; +return ret; +} +static SYSDEV_ATTR(online, 0600, show_online, store_online); + +static void __init register_cpu_control (struct cpu *cpu) +{ +sysdev_create_file (&cpu->sysdev, &attr_online); +} +#else /* ... !CONFIG_HOTPLUG_CPU */ +static void __init register_cpu_control (struct cpu *cpu) +{ +} +#endif /* CONFIG_HOTPLUG_CPU */ /* * register_cpu - Setup a driverfs device for a CPU.
@@ -34,6 +75,8 @@ int __init register_cpu(struct cpu *cpu, error =sysfs_create_link (&root->sysdev.kobj, &cpu->sysdev.kobj,kobject_name (&cpu->sysdev.kobj)); +if (!error) +register_cpu_control (cpu);return error; }
drivers/scsi/scsi.c +36 -0
@@ -53,6 +53,8 @@#include <linux/spinlock.h> #include <linux/kmod.h> #include <linux/interrupt.h> +#include <linux/notifier.h> +#include <linux/cpu.h> #include <scsi/scsi_host.h> #include "scsi.h"
@@ -1129,6 +1131,38 @@ int scsi_device_cancel(struct scsi_devicreturn 0 ; } +#ifdef CONFIG_HOTPLUG_CPU +static int scsi_cpu_notify (struct notifier_block *self, +unsigned long action,void *hcpu) +{ +int cpu = (unsigned long )hcpu; + +switch (action) { +case CPU_DEAD : +/* Drain scsi_done_q. */ +local_irq_disable (); +list_splice_init (&per_cpu (scsi_done_q, cpu), + &__get_cpu_var (scsi_done_q)); +raise_softirq_irqoff (SCSI_SOFTIRQ ); +local_irq_enable (); +break ; +default : +break ; + } +return NOTIFY_OK ; +} + +static struct notifier_block __devinitdata scsi_cpu_nb = { + .notifier_call = scsi_cpu_notify, +}; + +#define register_scsi_cpu () register_cpu_notifier(&scsi_cpu_nb) +#define unregister_scsi_cpu () unregister_cpu_notifier(&scsi_cpu_nb) +#else +#define register_scsi_cpu () +#define unregister_scsi_cpu () +#endif /* CONFIG_HOTPLUG_CPU */ +MODULE_DESCRIPTION ("SCSI core" );MODULE_LICENSE ("GPL" );
@@ -1163,6 +1197,7 @@ static int __init init_scsi(void)devfs_mk_dir ("scsi" );open_softirq (SCSI_SOFTIRQ , scsi_softirq,NULL ); +register_scsi_cpu ();printk (KERN_NOTICE "SCSI subsystem initialized\n" );return 0 ;
@@ -1190,6 +1225,7 @@ static void __exit exit_scsi(void)devfs_remove ("scsi" );scsi_exit_procfs ();scsi_exit_queue (); +unregister_scsi_cpu (); }subsys_initcall (init_scsi);
fs/buffer.c +21 -0
@@ -2991,6 +2991,26 @@ init_buffer_head(void *data, kmem_cache_ } } +#ifdef CONFIG_HOTPLUG_CPU +static void buffer_exit_cpu (int cpu) +{ +int i; +struct bh_lru *b = &per_cpu (bh_lrus, cpu); + +for (i =0 ; i <BH_LRU_SIZE ; i++) { +brelse (b->bhs[i]); + b->bhs[i] =NULL ; + } +} + +static int buffer_cpu_notify (struct notifier_block *self, +unsigned long action,void *hcpu) +{ +if (action ==CPU_DEAD ) +buffer_exit_cpu ((unsigned long )hcpu); +return NOTIFY_OK ; +} +#endif /* CONFIG_HOTPLUG_CPU */ void __init buffer_init (void ) {
@@ -3008,6 +3028,7 @@ void __init buffer_init(void)*/ nrpages = (nr_free_buffer_pages () *10 ) /100 ; max_buffer_heads = nrpages * (PAGE_SIZE /sizeof (struct buffer_head )); +hotcpu_notifier (buffer_cpu_notify,0 ); }EXPORT_SYMBOL (__bforget);
include/linux/cpu.h +12 -0
@@ -21,6 +21,8 @@#include <linux/sysdev.h> #include <linux/node.h> +#include <linux/compiler.h> +#include <linux/cpumask.h> #include <asm/semaphore.h> struct cpu {
@@ -56,9 +58,19 @@ extern struct sysdev_class cpu_sysdev_clextern struct semaphore cpucontrol;#define lock_cpu_hotplug () down(&cpucontrol)#define unlock_cpu_hotplug () up(&cpucontrol) +int cpu_down (unsigned int cpu); +#define hotcpu_notifier (fn, pri) { \ + static struct notifier_block fn##_nb = { fn, pri }; \ + register_cpu_notifier(&fn##_nb); \ +} +#define cpu_is_offline (cpu) unlikely(!cpu_online(cpu))#else #define lock_cpu_hotplug () do { } while (0)#define unlock_cpu_hotplug () do { } while (0) +#define hotcpu_notifier (fn, pri) + +/* CPUs don't go offline once they're online w/o CONFIG_HOTPLUG_CPU */ +#define cpu_is_offline (cpu) 0#endif #endif /* _LINUX_CPU_H_ */
include/linux/interrupt.h +1 -0
@@ -211,6 +211,7 @@ static inline void tasklet_hi_enable(str }extern void tasklet_kill (struct tasklet_struct *t); +extern void tasklet_kill_immediate (struct tasklet_struct *t,unsigned int cpu);extern void tasklet_init (struct tasklet_struct *t,void (*func)(unsigned long ),unsigned long data);
include/linux/mmzone.h +1 -0
@@ -213,6 +213,7 @@ typedef struct pglist_data {int node_id;struct pglist_data *pgdat_next;wait_queue_head_t kswapd_wait; +struct task_struct *kswapd; } pg_data_t;#define node_present_pages (nid) (NODE_DATA(nid)->node_present_pages)
include/linux/sched.h +6 -0
@@ -634,6 +634,10 @@ extern void sched_balance_exec(void);#define sched_balance_exec () {}#endif +/* Move tasks off this (offline) CPU onto another. */ +extern void migrate_all_tasks (void ); +/* Try to move me here, if possible. */ +void migrate_to_cpu (int dest_cpu);extern void set_user_nice (task_t *p,long nice);extern int task_prio (task_t *p);extern int task_nice (task_t *p);
@@ -1010,6 +1014,8 @@ static inline void set_task_cpu(struct t p->thread_info->cpu = cpu; } +/* Wake up a CPU from idle */ +void wake_idle_cpu (unsigned int cpu);#else static inline unsigned int task_cpu (struct task_struct *p)
kernel/cpu.c +143 -4
171 lines not on this page · raw message
kernel/kmod.c +7 -0
@@ -34,6 +34,7 @@#include <linux/security.h> #include <linux/mount.h> #include <linux/kernel.h> +#include <linux/cpu.h> #include <asm/uaccess.h> extern int max_threads;
@@ -170,6 +171,12 @@ static int ____call_usermodehelper(void/* We can run anywhere, unlike our parent keventd(). */ set_cpus_allowed (current,CPU_MASK_ALL ); +/* As a kernel thread which was bound to a specific cpu, +migrate_all_tasks wouldn't touch us. Avoid running child +on dying CPU. */ +if (cpu_is_offline (smp_processor_id ())) +migrate_to_cpu (any_online_cpu (CPU_MASK_ALL )); + retval = -EPERM ;if (current->fs->root) retval =execve (sub_info->path, sub_info->argv,sub_info->envp);
kernel/kthread.c +8 -1
@@ -12,6 +12,7 @@#include <linux/completion.h> #include <linux/err.h> #include <linux/unistd.h> +#include <linux/cpu.h> #include <asm/semaphore.h> struct kthread_create_info
@@ -39,7 +40,7 @@ static int kthread(void *_create) threadfn = create->threadfn; data = create->data; -/* Block and flush all signals (in case we're not from keventd). */ +/* Block and flush all signals. */ sigfillset (&blocked);sigprocmask (SIG_BLOCK , &blocked,NULL );flush_signals (current);
@@ -47,6 +48,12 @@ static int kthread(void *_create)/* By default we can run anywhere, unlike keventd. */ set_cpus_allowed (current,CPU_MASK_ALL ); +/* As a kernel thread which was bound to a specific cpu, +migrate_all_tasks wouldn't touch us. Avoid running on +dying CPU. */ +if (cpu_is_offline (smp_processor_id ())) +migrate_to_cpu (any_online_cpu (CPU_MASK_ALL )); +/* OK, tell user we're spawned, wait for stop or wakeup */ __set_current_state (TASK_INTERRUPTIBLE );complete (&create->started);
kernel/rcupdate.c +59 -1
72 lines not on this page · raw message
kernel/sched.c +115 -18
229 lines not on this page · raw message
kernel/softirq.c +72 -1
91 lines not on this page · raw message
kernel/timer.c +73 -1
86 lines not on this page · raw message
kernel/workqueue.c +109 -7
195 lines not on this page · raw message
mm/page_alloc.c +18 -0
@@ -1538,9 +1538,27 @@ struct seq_operations vmstat_op = {#endif /* CONFIG_PROC_FS */ +#ifdef CONFIG_HOTPLUG_CPU +static int page_alloc_cpu_notify (struct notifier_block *self, +unsigned long action,void *hcpu) +{ +int cpu = (unsigned long )hcpu; +long *count; + +if (action ==CPU_DEAD ) { +/* Drain local pagecache count. */ + count = &per_cpu (nr_pagecache_local, cpu); +atomic_add (*count, &nr_pagecache); + *count =0 ; +drain_local_pages (cpu); + } +return NOTIFY_OK ; +} +#endif /* CONFIG_HOTPLUG_CPU */ void __init page_alloc_init (void ) { +hotcpu_notifier (page_alloc_cpu_notify,0 ); }/*
mm/slab.c +46 -27
140 lines not on this page · raw message
mm/swap.c +41 -1
58 lines not on this page · raw message
mm/vmscan.c +44 -1
63 lines not on this page · raw message
net/core/dev.c +48 -0
66 lines not on this page · raw message
net/core/flow.c +12 -0
@@ -326,6 +326,17 @@ static void __devinit flow_cache_cpu_pretasklet_init (tasklet, flow_cache_flush_tasklet,0 ); } +#ifdef CONFIG_HOTPLUG_CPU +static int flow_cache_cpu (struct notifier_block *nfb, +unsigned long action, +void *hcpu) +{ +if (action ==CPU_DEAD ) +__flow_cache_shrink ((unsigned long )hcpu,0 ); +return NOTIFY_OK ; +} +#endif /* CONFIG_HOTPLUG_CPU */ +static int __init flow_cache_init (void ) {int i;
@@ -350,6 +361,7 @@ static int __init flow_cache_init(void)for_each_cpu (i)flow_cache_cpu_prepare (i); +hotcpu_notifier (flow_cache_cpu,0 );return 0 ; }
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.
Quoted 3 lines · Rusty Russell
The actual CPU patch. It's big, but almost all under
CONFIG_HOTPLUG_CPU, or macros which have same effect.
Needs a fixup.
CPU_MASK_ALL and CPU_MASK_NONE may only be used for initialisers. It doesn't compile if NR_CPUS>4*BITS_PER_LONG. Fixes to the cpumask infrastructure for this remain welcome.
---
25-akpm/kernel/kmod.c | 5 +++-- 25-akpm/kernel/kthread.c | 5 +++-- 2 files changed, 6 insertions(+), 4 deletions(-)
kernel/kmod.c renamed from kernel/kmod.c~set_cpus_allowed-fix +3 -2
@@ -159,6 +159,7 @@ struct subprocess_info {static int ____call_usermodehelper (void *data) {struct subprocess_info *sub_info = data; +static cpumask_t all_cpus =CPU_MASK_ALL ;int retval;/* Unblock all signals. */
@@ -170,12 +171,12 @@ static int ____call_usermodehelper(voidspin_unlock_irq (¤t->sighand->siglock);/* We can run anywhere, unlike our parent keventd(). */ -set_cpus_allowed (current,CPU_MASK_ALL ); +set_cpus_allowed (current, all_cpus);/* As a kernel thread which was bound to a specific cpu, migrate_all_tasks wouldn't touch us. Avoid running child on dying CPU. */ if (cpu_is_offline (smp_processor_id ())) -migrate_to_cpu (any_online_cpu (CPU_MASK_ALL )); +migrate_to_cpu (any_online_cpu (all_cpus)); retval = -EPERM ;if (current->fs->root)
kernel/kthread.c renamed from kernel/kthread.c~set_cpus_allowed-fix +3 -2
@@ -35,6 +35,7 @@ static int kthread(void *_create)void *data;sigset_t blocked;int ret = -EINTR ; +static cpumask_t all_cpus =CPU_MASK_ALL ;/* Copy data: it's on keventd's stack */ threadfn = create->threadfn;
@@ -46,13 +47,13 @@ static int kthread(void *_create)flush_signals (current);/* By default we can run anywhere, unlike keventd. */ -set_cpus_allowed (current,CPU_MASK_ALL ); +set_cpus_allowed (current, all_cpus);/* As a kernel thread which was bound to a specific cpu, migrate_all_tasks wouldn't touch us. Avoid running on dying CPU. */ if (cpu_is_offline (smp_processor_id ())) -migrate_to_cpu (any_online_cpu (CPU_MASK_ALL )); +migrate_to_cpu (any_online_cpu (all_cpus));/* OK, tell user we're spawned, wait for stop or wakeup */ __set_current_state (TASK_INTERRUPTIBLE );
_
Quoted 6 lines · Andrew Morton
Quoted 3 lines · Rusty Russell
The actual CPU patch. It's big, but almost all under
CONFIG_HOTPLUG_CPU, or macros which have same effect.
Quoted 9 lines
Needs a fixup.
CPU_MASK_ALLandCPU_MASK_NONEmay only be used for initialisers. It doesn't compile if NR_CPUS>4*BITS_PER_LONG. Fixes to the cpumask infrastructure for this remain welcome.
The kernel/sched.c stuff otherwise looks pretty straightforward at a quick glance. I can't see any fundamental problem with it.
In message <[email protected]> you write:
Quoted 3 lines
CPU_MASK_ALLandCPU_MASK_NONEmay only be used for initialisers. It doesn't compile if NR_CPUS>4*BITS_PER_LONG. Fixes to the cpumask infrastructure for this remain welcome.
Of course: *THAT* makes sense.
I dislike the "flexibility" of the cpumask code immensley.
Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.
Ingo Molnar
maintainerQuoted 13 lines · Rusty Russell
D: -
kernel/sched.c: Addwake_idle_cpu()for i386 hotplug. D:D: Check everywhere to make sure we never move tasks onto an D: offline cpu: we'll just be fighting migrate_all_tasks(). D: D: Change sched_migrate_task() to migrate_to_cpu and expose it D: for hotplug cpu. D: D: Take hotplug lock in sys_sched_setaffinity. D: D: Return cpus_allowed masked by cpu_possible_map, not D: cpu_online_map in sys_sched_getaffinity, otherwise tasks can't D: know about whether they can run on currently-offline cpus.
Quoted 4 lines
D:
D: Implementmigrate_all_tasks()to push tasks off the dying cpu.
D:
D: Add callbacks to stop migration thread.
the sched.c bits look good. A question: why is the migrate_all_tasks() code nonatomic? If you run this code in the context of the migration thread of the downed CPU then all of the migration can be done in one big atomic locked section which holds all runqueue locks and just moves all tasks off the current CPU. If it were done this way then eg. the __migrate_task() race-avoidance check (cpu_is_offline()) could go away and the whole thing would be more robust i believe.
Ingo
In message <[email protected]> you write:
Quoted 7 lines
the
sched.cbits look good. A question: why is themigrate_all_tasks()code nonatomic? If you run this code in the context of the migration thread of the downed CPU then all of the migration can be done in one big atomic locked section which holds all runqueue locks and just moves all tasks off the current CPU. If it were done this way then eg. the__migrate_task()race-avoidance check (cpu_is_offline()) could go away and the whole thing would be more robust i believe.
Hmm...
I can code it up and we can see what it looks like.
Unfortunately the __migrate_task() check won't go away: someone may have asked to move from CPU 0 to 1, and by the time migration thread on 0 gets to the request, 1 has gone down. We don't want all the callers to hold the cpucontrol lock, because now the NUMA scheduler uses migration as a common case 8(
Thanks,
Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.
Ingo Molnar
maintainerQuoted 5 lines · Rusty Russell
Unfortunately the
__migrate_task()check won't go away: someone may have asked to move from CPU 0 to 1, and by the time migration thread on 0 gets to the request, 1 has gone down. We don't want all the callers to hold the cpucontrol lock, because now the NUMA scheduler uses migration as a common case 8(
well, when a CPU goes down it could process the migration request queue as well. (this would be a pretty natural thing to do if CPU-down executes in the migration-thread context.)
Another question is user-space semantics - if user-space relies on CPU affinity, is the kernel allowed to violate it or should the process be notified. Sending it a signal (SIGTERM or anything similar) if the affinity was non-generic might be a good thing to do.
Ingo
Quoted 9 lines · Ingo Molnar
Quoted 5 lines · Rusty Russell
Unfortunately the
__migrate_task()check won't go away: someone may have asked to move from CPU 0 to 1, and by the time migration thread on 0 gets to the request, 1 has gone down. We don't want all the callers to hold the cpucontrol lock, because now the NUMA scheduler uses migration as a common case 8(
Quoted 3 lines
well, when a CPU goes down it could process the migration request queue as well. (this would be a pretty natural thing to do if CPU-down executes in the migration-thread context.)
One possibility is the migration request will be in
CPU 0's queue and not in CPU1's? Hence there won't be anything (in CPU1's migration request queue) to process when CPU1 is brought down. However by the time migration thread on CPU 0 gets around to processing the request, CPU1 is already down and hence the check.
Signature · 5 lines
Thanks and Regards, Srivatsa Vaddagiri, Linux Technology Center, IBM Software Labs, Bangalore, INDIA - 560017
Quoted 5 lines · Srivatsa Vaddagiri
One possibility is the migration request will be in CPU 0's queue and not in CPU1's? Hence there won't be anything (in CPU1's migration request queue) to process when CPU1 is brought down. However by the time migration thread on CPU 0 gets around to processing the request, CPU1 is already down and hence the check.
IMO the semantics of this area are not defined clearly enough, hence these problems. __migrate_task(cpu) is a wrapper around set_cpus_allowed(). If a CPU goes away atomically then the only source of some non-online CPU is from user-space or unsafe references of smp_processor_id() [which are also preempt-unsafe so need to be fixed anyway]. set_cpus_allowed() deals with invalid masks just fine: it does nothing and returns an error. The following sequence shouldnt be possible to trigger if CPU-down is an atomic event:
/* Affinity changed (again). */
if (!cpu_isset(dest_cpu, p->cpus_allowed))
goto out;
/* CPU went down. */
if (cpu_is_offline(dest_cpu))
goto out;because p->cpus_allowed is fixed up atomically. If this task is in CPU1's migration queue then the CPU-down mechanism takes care of it. If this task is in CPU0's migration queue then the task will be migrated off CPU1 by the CPU-down mechanism and CPU0's migration wont have alot of work left.
kernel-space threads that rely on running on a specific CPU need the callback mechanism on CPU-down, so there's no problem with them.
user-space tasks that rely on running on a specific CPU need a callback too. (probably in the form of a signal, which, if unhandled, terminates the task.) Eg. if a webserver has a mode to run one thread per CPU, then the server needs to adapt to the new situation when a CPU goes away. We cannot just unilaterally migrate a task and violate its affinity.
another thing: if the migrate-irqs op is done atomically too (together with the migrate-tasks op) then the special-cases in idle_balance() and rebalance_tick() could go away too.
Ingo
In message <[email protected]> you write:
Quoted 5 lines
user-space tasks that rely on running on a specific CPU need a callback too. (probably in the form of a signal, which, if unhandled, terminates the task.) Eg. if a webserver has a mode to run one thread per CPU, then the server needs to adapt to the new situation when a CPU goes away. We cannot just unilaterally migrate a task and violate its affinity.
Well, that's what we'd do anyway, to deliver the signal.
This terminating signal idea is simply flawed: affinity is inherited, so you're killing a process which knows nothing anyway.
If we can't do it well, leave it to userspace to sort out 8)
Quoted 3 lines
another thing: if the migrate-irqs op is done atomically too (together with the migrate-tasks op) then the special-cases in
idle_balance()andrebalance_tick()could go away too.
Exactly. It simplifies a number of things.
Thanks,
Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.
Quoted 4 lines · Rusty Russell
This terminating signal idea is simply flawed: affinity is inherited, so you're killing a process which knows nothing anyway.
If we can't do it well, leave it to userspace to sort out 8)
yes, but currently there's no mechanism for userspace to get notified - hence no clean way for userspace to sort it out. (other than userspace continuously polling the CPU mask - bleh.) But this is a separate issue.
Ingo
In message <[email protected]> you write:
Quoted 11 lines · Rusty Russell
Quoted 5 lines
This terminating signal idea is simply flawed: affinity is inherited, so you're killing a process which knows nothing anyway.
If we can't do it well, leave it to userspace to sort out 8)
yes, but currently there's no mechanism for userspace to get notified - hence no clean way for userspace to sort it out. (other than userspace continuously polling the CPU mask - bleh.) But this is a separate issue.
We use /sbin/hotplug at the moment. I'm hoping that DBUS turn this from "possible" into "easy" some day...
Thanks,
Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.
Quoted 8 lines · Rusty Russell
Quoted 5 lines
Quoted 1 line
If we can't do it well, leave it to userspace to sort out 8)
yes, but currently there's no mechanism for userspace to get notified - hence no clean way for userspace to sort it out. (other than userspace continuously polling the CPU mask - bleh.) But this is a separate issue.
We use
/sbin/hotplugat the moment. I'm hoping that DBUS turn this from "possible" into "easy" some day...
ok. Anyway, it should be relatively rare for userspace to care about this.
Ingo
New v. v. experimental HOTPLUG CPU megapatch.
In reply to Ingo MolnarThis is my first cut of a patch, still has some old code in it. As an attachment since it's 70k (I'll split into multiple parts later, this is the x86 part, too).
Patch against 2.6.2-rc2-mm2. Works basically, gives "APIC error on
CPU1: 08(08)" under stress. Clues welcome.
Basically consists of:
1) New file stop_machine.[ch] which takes logic out of module.c (I haven't converted module.c code over yet though).
2) x86: arch_cpu_down_check (called before offlining) and arch_cpu_down (called when machine stopped, moves irqs).
3) x86: idle loop code to play dead.
4) migrate_all_tasks() called with machine stopped, and migrates kernel threads as well.
5) cpu.c fires off a thread to do the dirty work: it schedules with interrupts still disabled on dead cpu.
6) Most threads are happy to run on "wrong" CPUs, but slab.c reap timer needs a little fixing, and still needs to stop when CPU goes offline. softirq threads are tied to CPU: I just hacked in a check so the do nothing if CPU is offline. Moving the migration thread is safe since it should have nothing to do.
7) Ugly change to finish_arch_switch so it doesn't re-enable interrupts if the CPU is down (switching from take_cpu_down kthread to idle task). __migrate_task() still needs check for cpu down, AFAICT.
Given it was about a day's work, I'm happy it works at all...
Cheers,
Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.
Re: New v. v. experimental HOTPLUG CPU megapatch.
Quoted 2 lines · Rusty Russell
Patch against 2.6.2-rc2-mm2. Works basically, gives "APIC error on
CPU1: 08(08)" under stress. Clues welcome.
APIC error 08 is receive error. Ie. most likely there was a pending IPI (or pending hwirq) to that CPU but the CPU was zapped and the APIC reset. I'd suggest to add "sti;nop;cli" instructions after the IO-APIC masks have been redirected [note the nop - the interrupt-enable boundary on x86 is two instructions from sti] - to flush out pending hardirqs and IPIs. After this point nothing is supposed to reach this CPU. Enabling irqs at this point should not cause any races, because you do this first, right?
the pending cross-CPU-IPI case should not happen if the infrastructure is correct, external hardirqs are not an issue unless it's an edge-triggered device. So the worst-case with your current code could be a lost timer IRQ or a lost edge-triggered PCI irq (old ne2k cards).
Ingo
Re: New v. v. experimental HOTPLUG CPU megapatch.
In reply to Rusty RussellHi!
Quoted 11 lines
This is my first cut of a patch, still has some old code in it. As an attachment since it's 70k (I'll split into multiple parts later, this is the x86 part, too).
Patch against 2.6.2-rc2-mm2. Works basically, gives "APIC error on
CPU1: 08(08)" under stress. Clues welcome.Basically consists of:
1) New file stop_machine.[ch] which takes logic out of
module.c(I haven't convertedmodule.ccode over yet though).
I like this one. Something like this will be handy for SMP sleep support.
Pavel
Signature · 2 lines
When do you have a heart between your knees? [Johanka's followup: and *two* hearts?]
In message <[email protected]> you wri te:
Quoted 9 lines · Rusty Russell
Quoted 6 lines
Unfortunately the
__migrate_task()check won't go away: someone may have asked to move from CPU 0 to 1, and by the time migration thread on 0 gets to the request, 1 has gone down. We don't want all the callers to hold the cpucontrol lock, because now the NUMA scheduler uses migration as a common case 8(
Quoted 3 lines
well, when a CPU goes down it could process the migration request queue as well. (this would be a pretty natural thing to do if CPU-down executes in the migration-thread context.)
Wrong migration thread. The migration thread on CPU 1 has been asked to push into CPU 0, which is now going down.
Now I've slept on the "do it atomically" idea, I think it's a good one. I've even worked out how to maintain the "last thread on CPU is the idle thread", although I'd need to test in code.
Quoted 4 lines
Another question is user-space semantics - if user-space relies on CPU affinity, is the kernel allowed to violate it or should the process be notified. Sending it a signal (SIGTERM or anything similar) if the affinity was non-generic might be a good thing to do.
We've been there: we used to send SIGPWR with a new si_info field to say what CPU it was. But it turns out not to be useful, so we took it out.
In practice, any app which wants to scale with # CPUs needs to know when CPUs are coming up, as well as going down. Ditto memory, etc. This means they need to listen for the hotplug event (DBUS anyone?), or we introduce a SIGRECONF (default ignored). But AFAICT, introducing a new signal isn't possible (at least on x86) without breaking glibc.
Thanks!
Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.
Ingo Molnar
maintainerQuoted 2 lines · Rusty Russell
Wrong migration thread. The migration thread on CPU 1 has been asked to push into CPU 0, which is now going down.
you are right - this one place would have to be aware of CPUs going away.
Quoted 3 lines
Now I've slept on the "do it atomically" idea, I think it's a good one. I've even worked out how to maintain the "last thread on CPU is the idle thread", although I'd need to test in code.
good!
Quoted 5 lines
In practice, any app which wants to scale with # CPUs needs to know when CPUs are coming up, as well as going down. Ditto memory, etc. This means they need to listen for the hotplug event (DBUS anyone?), or we introduce a SIGRECONF (default ignored). But AFAICT, introducing a new signal isn't possible (at least on x86) without breaking glibc.
you can introduce a variable RT signal for this purpose no problem - and one would want to have a queued signal for this anyway. Ie. by default the notification is disabled, but a new syscall sets up the process to be notified of CPU up/down events, on a signal # picked by the app. But this this indeed is dbus domain ...
Ingo
In message <[email protected]> you write:
Quoted 8 lines
Quoted 2 lines
introduce a SIGRECONF (default ignored). But AFAICT, introducing a new signal isn't possible (at least on x86) without breaking glibc.
you can introduce a variable RT signal for this purpose no problem - and one would want to have a queued signal for this anyway. Ie. by default the notification is disabled, but a new syscall sets up the process to be notified of CPU up/down events, on a signal # picked by the app. But this this indeed is dbus domain ...
Yeah, this workaround because we can't add a new signal isn't very good, though ("I can't add a new signal, so you choose!"). In practice, it sucks for libraries and it sucks for children.
As you point out, DBUS is a better solution anyway. Of course, proposing that something which doesn't yet exist will solve all our problems is usually a sign of laziness or optimism 8).
Cheers,
Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.