scheduler changes for v2.6.27
Subsystems (5)
Linus,
Please pull the latest scheduler git tree from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip.git sched/for-linus
Thanks,
Ingo
------------------>
Ankita Garg (1):
sched: fix accounting in task delay accounting & migration
Arjan van de Ven (1):
sched: print module list in the "scheduling while atomic" warning
David Rientjes (1):
sched: prevent bound kthreads from changing cpus_allowed
Dhaval Giani (2):
sched: add comments for ifdefs in sched.c
sched: make sched_{rt,fair}.c ifdefs more readable
Dmitry Adamushko (2):
sched: check for SD_SERIALIZE atomically in rebalance_domains()
sched: rework of "prioritize non-migratable tasks over migratable ones"
Gautham R Shenoy (2):
sched: update the sched-domains debug documentation
sched: print the sd->level in sched_domain_debug code
Gregory Haskins (7):
sched: prioritize non-migratable tasks over migratable ones
sched: use a 2-d bitmap for searching lowest-pri CPU
sched: fix SCHED_OTHER balance iterator to include all tasks
sched: fix cpupri hotplug support
sched: fix cpupri priocount
sched: terminate newidle balancing once at least one task has moved over
sched: add avg-overlap support to RT tasks
Hiroshi Shimamoto (1):
sched: fix typo in Documentation/scheduler/sched-rt-group.txt
Ingo Molnar (5):
namespacecheck: more sched.c fixes
sched: fix cpuprio build bug
sched: export cpu_clock
sched: sched_clock_cpu() based cpu_clock(), lockdep fix
sched: build fix
Max Krasnyansky (2):
sched: CPU hotplug events must not destroy scheduler domains created by the cpusets
sched: Move cpu masks from kernel/sched.c into kernel/cpu.c
Mike Galbraith (2):
sched: make !hrtick faster
sched: trivial sched_features cleanup
Paul Mundt (1):
sched: kill off dead cfs_rq_set_shares()
Peter Zijlstra (34):
sched: fix hotplug cpus on ia64
sched: debug: add some rt debug output
sched: rt: fix SMP bandwidth balancing for throttled groups
sched: rt: move some code around
sched: rt: fix the bandwidth contraint computations
sched: rt: dont stop the period timer when there are tasks wanting to run
sched: clean up some unused variables
sched: revert the revert of: weight calculations
sched: fix calc_delta_asym()
sched: fix calc_delta_asym, #2
sched: revert revert of: fair-group: SMP-nice for group scheduling
sched: sched_clock_cpu() based cpu_clock()
sched: fix wakeup granularity and buddy granularity
sched: add full schedstats to /proc/sched_debug
sched: fix sched_domain aggregation
sched: update aggregate when holding the RQs
sched: dont micro manage share losses
sched: no need to aggregate task_weight
sched: simplify the group load balancer
sched: fix newidle smp group balancing
sched: fix sched_balance_self() smp group balancing
sched: persistent average load per task
sched: hierarchical load vs affine wakeups
sched: hierarchical load vs find_busiest_group
sched: fix load scaling in group balancing
sched: fix task_h_load()
sched: remove prio preference from balance decisions
sched: optimize effective_load()
sched: disable source/target_load bias
sched: fix shares boost logic
sched: update shares on wakeup
sched: correct wakeup weight calculations
sched: incremental effective_load()
sched: bias effective_load() error towards failing wake_affine().
Rabin Vincent (1):
sched: fix defined-but-unused warning
Richard Kennedy (1):
sched: reorder task_struct to reduce padding on 64bit builds
Srivatsa Vaddagiri (2):
sched: kill task_group balancing
sched: fix mult overflow
Steven Rostedt (7):
sched_clock: record from last tick
sched_clock: widen the max and min time
sched_clock: stop maximum check on NO HZ
sched_clock: fix calculation of other CPU
sched_clock: only update deltas with local reads.
sched_clock: record TSC after gtod
sched_clock: and multiplier for TSC to gtod drift
Thomas Gleixner (3):
namespacecheck: fixes in kernel/sched.c
sched: fix the cpuprio count really
sched: move weighted_cpuload into #ifdef CONFIG_SMP section
Vegard Nossum (1):
sched: fix warning
Documentation/scheduler/sched-domains.txt | 7 +- Documentation/scheduler/sched-rt-group.txt | 4 +- include/linux/sched.h | 59 ++-- kernel/Makefile | 5 +- kernel/cpu.c | 24 + kernel/cpuset.c | 14 +- kernel/kthread.c | 1 + kernel/sched.c | 723 +++++++++++++++++++--------- kernel/sched_clock.c | 137 +++++- kernel/sched_cpupri.c | 174 +++++++ kernel/sched_cpupri.h | 36 ++ kernel/sched_debug.c | 64 +++- kernel/sched_fair.c | 413 +++++++++++----- kernel/sched_features.h | 7 +- kernel/sched_rt.c | 405 ++++++++++------ kernel/sched_stats.h | 42 ++- kernel/sysctl.c | 8 + kernel/time/tick-sched.c | 2 + 18 files changed, 1555 insertions(+), 570 deletions(-) create mode 100644 kernel/sched_cpupri.c create mode 100644 kernel/sched_cpupri.h
Documentation/scheduler/sched-domains.txt +2 -5
@@ -61,10 +61,7 @@ builder by #define'ing ARCH_HASH_SCHED_DOMAIN, and exporting your arch_init_sched_domains function. This function will attach domains to all CPUs using cpu_attach_domain. -Implementors should change the line -#undef SCHED_DOMAIN_DEBUG -to -#define SCHED_DOMAIN_DEBUG -in kernel/sched.c as this enables an error checking parse of the sched domains +The sched-domains debugging infrastructure can be enabled by enabling +CONFIG_SCHED_DEBUG. This enables an error checking parse of the sched domains which should catch most possible errors (described above). It also prints out the domain structure in a visual format.
Documentation/scheduler/sched-rt-group.txt +2 -2
@@ -51,9 +51,9 @@ needs only about 3% CPU time to do so, it can do with a 0.03 * 0.005s = 0.00015s. So this group can be scheduled with a period of 0.005s and a run time of 0.00015s. -The remaining CPU time will be used for user input and other tass. Because +The remaining CPU time will be used for user input and other tasks. Because realtime tasks have explicitly allocated the CPU time they need to perform -their tasks, buffer underruns in the graphocs or audio can be eliminated. +their tasks, buffer underruns in the graphics or audio can be eliminated. NOTE: the above example is not fully implemented as of yet (2.6.25). We still lack an EDF scheduler to make non-uniform periods usable.
include/linux/sched.h +30 -29
148 lines not on this page · raw message
kernel/Makefile +3 -2
@@ -3,7 +3,7 @@ # obj-y = sched.o fork.o exec_domain.o panic.o printk.o profile.o \ - exit.o itimer.o time.o softirq.o resource.o \ + cpu.o exit.o itimer.o time.o softirq.o resource.o \ sysctl.o capability.o ptrace.o timer.o user.o \ signal.o sys.o kmod.o workqueue.o pid.o \ rcupdate.o extable.o params.o posix-timers.o \
@@ -27,7 +27,7 @@ obj-$(CONFIG_RT_MUTEXES) += rtmutex.o obj-$(CONFIG_DEBUG_RT_MUTEXES) += rtmutex-debug.o obj-$(CONFIG_RT_MUTEX_TESTER) += rtmutex-tester.o obj-$(CONFIG_GENERIC_ISA_DMA) += dma.o -obj-$(CONFIG_SMP) += cpu.o spinlock.o +obj-$(CONFIG_SMP) += spinlock.o obj-$(CONFIG_DEBUG_SPINLOCK) += spinlock.o obj-$(CONFIG_PROVE_LOCKING) += spinlock.o obj-$(CONFIG_UID16) += uid16.o
@@ -69,6 +69,7 @@ obj-$(CONFIG_TASK_DELAY_ACCT) += delayacct.o obj-$(CONFIG_TASKSTATS) += taskstats.o tsacct.o obj-$(CONFIG_MARKERS) += marker.o obj-$(CONFIG_LATENCYTOP) += latencytop.o +obj-$(CONFIG_SMP) += sched_cpupri.o ifneq ($(CONFIG_SCHED_NO_NO_OMIT_FRAME_POINTER),y) # According to Alan Modra <[email protected]>, the -fno-omit-frame-pointer is
kernel/cpu.c +24 -0
@@ -15,6 +15,28 @@#include <linux/stop_machine.h> #include <linux/mutex.h> +/* +* Represents all cpu's present in the system +* In systems capable of hotplug, this map could dynamically grow +* as new cpu's are detected in the system via any platform specific +* method, such as ACPI for e.g. +*/ +cpumask_t cpu_present_map__read_mostly ; +EXPORT_SYMBOL (cpu_present_map); + +#ifndef CONFIG_SMP + +/* +* Represents all cpu's that are currently online. +*/ +cpumask_t cpu_online_map__read_mostly =CPU_MASK_ALL ; +EXPORT_SYMBOL (cpu_online_map); + +cpumask_t cpu_possible_map__read_mostly =CPU_MASK_ALL ; +EXPORT_SYMBOL (cpu_possible_map); + +#else /* CONFIG_SMP */ +/* Serializes the updates to cpu_online_map, cpu_present_map */ static DEFINE_MUTEX (cpu_add_remove_lock);
@@ -403,3 +425,5 @@ out:cpu_maps_update_done (); }#endif /* CONFIG_PM_SLEEP_SMP */ + +#endif /* CONFIG_SMP */
kernel/cpuset.c +13 -1
@@ -1194,6 +1194,15 @@ static int cpuset_can_attach(struct cgroup_subsys *ss,if (cpus_empty (cs->cpus_allowed) ||nodes_empty (cs->mems_allowed))return -ENOSPC ; +if (tsk->flags &PF_THREAD_BOUND ) { +cpumask_t mask; + +mutex_lock (&callback_mutex); + mask = cs->cpus_allowed; +mutex_unlock (&callback_mutex); +if (!cpus_equal (tsk->cpus_allowed, mask)) +return -EINVAL ; + }return security_task_setscheduler (tsk,0 ,NULL ); }
@@ -1207,11 +1216,14 @@ static void cpuset_attach(struct cgroup_subsys *ss,struct mm_struct *mm;struct cpuset *cs =cgroup_cs (cont);struct cpuset *oldcs =cgroup_cs (oldcont); +int err;mutex_lock (&callback_mutex);guarantee_online_cpus (cs, &cpus); -set_cpus_allowed_ptr (tsk, &cpus); + err =set_cpus_allowed_ptr (tsk, &cpus);mutex_unlock (&callback_mutex); +if (err) +return ; from = oldcs->mems_allowed; to = cs->mems_allowed;
kernel/kthread.c +1 -0
@@ -180,6 +180,7 @@ void kthread_bind(struct task_struct *k, unsigned int cpu)set_task_cpu (k, cpu); k->cpus_allowed =cpumask_of_cpu (cpu); k->rt.nr_cpus_allowed =1 ; + k->flags |=PF_THREAD_BOUND ; }EXPORT_SYMBOL (kthread_bind);
kernel/sched.c +491 -232
1,331 lines not on this page · raw message
kernel/sched_clock.c +118 -19
243 lines not on this page · raw message
kernel/sched_cpupri.c new file +174 -0
174 lines not on this page · raw message
kernel/sched_cpupri.h new file +36 -0
@@ -0,0 +1,36 @@ +#ifndef _LINUX_CPUPRI_H +#define _LINUX_CPUPRI_H + +#include <linux/sched.h> + +#define CPUPRI_NR_PRIORITIES (MAX_RT_PRIO + 2) +#define CPUPRI_NR_PRI_WORDS BITS_TO_LONGS(CPUPRI_NR_PRIORITIES) + +#define CPUPRI_INVALID -1 +#define CPUPRI_IDLE 0 +#define CPUPRI_NORMAL 1 +/* values 2-101 are RT priorities 0-99 */ + +struct cpupri_vec { +spinlock_t lock; +int count; +cpumask_t mask; +}; + +struct cpupri { +struct cpupri_vec pri_to_cpu[CPUPRI_NR_PRIORITIES ]; +long pri_active[CPUPRI_NR_PRI_WORDS ]; +int cpu_to_pri[NR_CPUS ]; +}; + +#ifdef CONFIG_SMP +int cpupri_find (struct cpupri *cp, +struct task_struct *p,cpumask_t *lowest_mask); +void cpupri_set (struct cpupri *cp,int cpu,int pri); +void cpupri_init (struct cpupri *cp); +#else +#define cpupri_set (cp, cpu, pri) do { } while (0) +#define cpupri_init () do { } while (0) +#endif + +#endif /* _LINUX_CPUPRI_H */
kernel/sched_debug.c +59 -5
91 lines not on this page · raw message
kernel/sched_fair.c +290 -123
606 lines not on this page · raw message
kernel/sched_features.h +5 -2
@@ -1,4 +1,5 @@ SCHED_FEAT(NEW_FAIR_SLEEPERS, 1) +SCHED_FEAT(NORMALIZED_SLEEPER, 1) SCHED_FEAT(WAKEUP_PREEMPT, 1) SCHED_FEAT(START_DEBIT, 1)SCHED_FEAT (AFFINE_WAKEUPS ,1 )
@@ -6,5 +7,7 @@ SCHED_FEAT(CACHE_HOT_BUDDY, 1) SCHED_FEAT(SYNC_WAKEUPS, 1) SCHED_FEAT(HRTICK, 1) SCHED_FEAT(DOUBLE_TICK, 0) -SCHED_FEAT(NORMALIZED_SLEEPER, 1) -SCHED_FEAT (DEADLINE ,1 ) +SCHED_FEAT(ASYM_GRAN, 1) +SCHED_FEAT(LB_BIAS, 0) +SCHED_FEAT(LB_WAKEUP_UPDATE, 1) +SCHED_FEAT (ASYM_EFF_LOAD ,1 )
kernel/sched_rt.c +264 -141
241 lines not on this page · raw message
kernel/sched_stats.h +33 -9
42 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Part of the diff is not shown here. View the raw message.