Skip to contentSkip to conversation

scheduler changes for v2.6.27

GIT PULL Ingo Molnar 1 participant 1 message Jul 14, 2008

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

Linus,

Please pull the latest scheduler git tree from:

git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip.git sched/for-linus

Thanks,

Ingo

------------------>
Ankita Garg (1):
sched: fix accounting in task delay accounting & migration

Arjan van de Ven (1):
sched: print module list in the "scheduling while atomic" warning

David Rientjes (1):
sched: prevent bound kthreads from changing cpus_allowed

Dhaval Giani (2):
sched: add comments for ifdefs in sched.c
sched: make sched_{rt,fair}.c ifdefs more readable

Dmitry Adamushko (2):
sched: check for SD_SERIALIZE atomically in rebalance_domains()
sched: rework of "prioritize non-migratable tasks over migratable ones"

Gautham R Shenoy (2):
sched: update the sched-domains debug documentation
sched: print the sd->level in sched_domain_debug code

Gregory Haskins (7):
sched: prioritize non-migratable tasks over migratable ones
sched: use a 2-d bitmap for searching lowest-pri CPU
sched: fix SCHED_OTHER balance iterator to include all tasks
sched: fix cpupri hotplug support
sched: fix cpupri priocount
sched: terminate newidle balancing once at least one task has moved over
sched: add avg-overlap support to RT tasks

Hiroshi Shimamoto (1):
sched: fix typo in Documentation/scheduler/sched-rt-group.txt

Ingo Molnar (5):
namespacecheck: more sched.c fixes
sched: fix cpuprio build bug
sched: export cpu_clock
sched: sched_clock_cpu() based cpu_clock(), lockdep fix
sched: build fix

Max Krasnyansky (2):
sched: CPU hotplug events must not destroy scheduler domains created by the cpusets
sched: Move cpu masks from kernel/sched.c into kernel/cpu.c

Mike Galbraith (2):
sched: make !hrtick faster
sched: trivial sched_features cleanup

Paul Mundt (1):
sched: kill off dead cfs_rq_set_shares()

Peter Zijlstra (34):
sched: fix hotplug cpus on ia64
sched: debug: add some rt debug output
sched: rt: fix SMP bandwidth balancing for throttled groups
sched: rt: move some code around
sched: rt: fix the bandwidth contraint computations
sched: rt: dont stop the period timer when there are tasks wanting to run
sched: clean up some unused variables
sched: revert the revert of: weight calculations
sched: fix calc_delta_asym()
sched: fix calc_delta_asym, #2
sched: revert revert of: fair-group: SMP-nice for group scheduling
sched: sched_clock_cpu() based cpu_clock()
sched: fix wakeup granularity and buddy granularity
sched: add full schedstats to /proc/sched_debug
sched: fix sched_domain aggregation
sched: update aggregate when holding the RQs
sched: dont micro manage share losses
sched: no need to aggregate task_weight
sched: simplify the group load balancer
sched: fix newidle smp group balancing
sched: fix sched_balance_self() smp group balancing
sched: persistent average load per task
sched: hierarchical load vs affine wakeups
sched: hierarchical load vs find_busiest_group
sched: fix load scaling in group balancing
sched: fix task_h_load()
sched: remove prio preference from balance decisions
sched: optimize effective_load()
sched: disable source/target_load bias
sched: fix shares boost logic
sched: update shares on wakeup
sched: correct wakeup weight calculations
sched: incremental effective_load()
sched: bias effective_load() error towards failing wake_affine().

Rabin Vincent (1):
sched: fix defined-but-unused warning

Richard Kennedy (1):
sched: reorder task_struct to reduce padding on 64bit builds

Srivatsa Vaddagiri (2):
sched: kill task_group balancing
sched: fix mult overflow

Steven Rostedt (7):
sched_clock: record from last tick
sched_clock: widen the max and min time
sched_clock: stop maximum check on NO HZ
sched_clock: fix calculation of other CPU
sched_clock: only update deltas with local reads.
sched_clock: record TSC after gtod
sched_clock: and multiplier for TSC to gtod drift

Thomas Gleixner (3):
namespacecheck: fixes in kernel/sched.c
sched: fix the cpuprio count really
sched: move weighted_cpuload into #ifdef CONFIG_SMP section

Vegard Nossum (1):
sched: fix warning

 Documentation/scheduler/sched-domains.txt  |    7 +-
 Documentation/scheduler/sched-rt-group.txt |    4 +-
 include/linux/sched.h                      |   59 ++--
 kernel/Makefile                            |    5 +-
 kernel/cpu.c                               |   24 +
 kernel/cpuset.c                            |   14 +-
 kernel/kthread.c                           |    1 +
 kernel/sched.c                             |  723 +++++++++++++++++++---------
 kernel/sched_clock.c                       |  137 +++++-
 kernel/sched_cpupri.c                      |  174 +++++++
 kernel/sched_cpupri.h                      |   36 ++
 kernel/sched_debug.c                       |   64 +++-
 kernel/sched_fair.c                        |  413 +++++++++++-----
 kernel/sched_features.h                    |    7 +-
 kernel/sched_rt.c                          |  405 ++++++++++------
 kernel/sched_stats.h                       |   42 ++-
 kernel/sysctl.c                            |    8 +
 kernel/time/tick-sched.c                   |    2 +
 18 files changed, 1555 insertions(+), 570 deletions(-)
 create mode 100644 kernel/sched_cpupri.c
 create mode 100644 kernel/sched_cpupri.h
Documentation/scheduler/sched-domains.txt +2 -5
index a9e990a..373ceac 100644
@@ -61,10 +61,7 @@ builder by #define'ing ARCH_HASH_SCHED_DOMAIN, and exporting your
 arch_init_sched_domains function. This function will attach domains to all
 CPUs using cpu_attach_domain.
 
-Implementors should change the line
-#undef SCHED_DOMAIN_DEBUG
-to
-#define SCHED_DOMAIN_DEBUG
-in kernel/sched.c as this enables an error checking parse of the sched domains
+The sched-domains debugging infrastructure can be enabled by enabling
+CONFIG_SCHED_DEBUG. This enables an error checking parse of the sched domains
 which should catch most possible errors (described above). It also prints out
 the domain structure in a visual format.
Documentation/scheduler/sched-rt-group.txt +2 -2
index 14f901f..3ef339f 100644
@@ -51,9 +51,9 @@ needs only about 3% CPU time to do so, it can do with a 0.03 * 0.005s =
 0.00015s. So this group can be scheduled with a period of 0.005s and a run time
 of 0.00015s.
 
-The remaining CPU time will be used for user input and other tass. Because
+The remaining CPU time will be used for user input and other tasks. Because
 realtime tasks have explicitly allocated the CPU time they need to perform
-their tasks, buffer underruns in the graphocs or audio can be eliminated.
+their tasks, buffer underruns in the graphics or audio can be eliminated.
 
 NOTE: the above example is not fully implemented as of yet (2.6.25). We still
 lack an EDF scheduler to make non-uniform periods usable.
include/linux/sched.h +30 -29

148 lines not on this page · raw message

kernel/Makefile +3 -2
index 1c9938a..6c55301 100644
@@ -3,7 +3,7 @@
 #
 
 obj-y     = sched.o fork.o exec_domain.o panic.o printk.o profile.o \
-	    exit.o itimer.o time.o softirq.o resource.o \
+	    cpu.o exit.o itimer.o time.o softirq.o resource.o \
 	    sysctl.o capability.o ptrace.o timer.o user.o \
 	    signal.o sys.o kmod.o workqueue.o pid.o \
 	    rcupdate.o extable.o params.o posix-timers.o \
@@ -27,7 +27,7 @@ obj-$(CONFIG_RT_MUTEXES) += rtmutex.o
 obj-$(CONFIG_DEBUG_RT_MUTEXES) += rtmutex-debug.o
 obj-$(CONFIG_RT_MUTEX_TESTER) += rtmutex-tester.o
 obj-$(CONFIG_GENERIC_ISA_DMA) += dma.o
-obj-$(CONFIG_SMP) += cpu.o spinlock.o
+obj-$(CONFIG_SMP) += spinlock.o
 obj-$(CONFIG_DEBUG_SPINLOCK) += spinlock.o
 obj-$(CONFIG_PROVE_LOCKING) += spinlock.o
 obj-$(CONFIG_UID16) += uid16.o
@@ -69,6 +69,7 @@ obj-$(CONFIG_TASK_DELAY_ACCT) += delayacct.o
 obj-$(CONFIG_TASKSTATS) += taskstats.o tsacct.o
 obj-$(CONFIG_MARKERS) += marker.o
 obj-$(CONFIG_LATENCYTOP) += latencytop.o
+obj-$(CONFIG_SMP) += sched_cpupri.o
 
 ifneq ($(CONFIG_SCHED_NO_NO_OMIT_FRAME_POINTER),y)
 # According to Alan Modra <[email protected]>, the -fno-omit-frame-pointer is
kernel/cpu.c +24 -0
index c77bc3a..b11f06d 100644
@@ -15,6 +15,28 @@
 #include <linux/stop_machine.h>
 #include <linux/mutex.h>
 
+/*
+ * Represents all cpu's present in the system
+ * In systems capable of hotplug, this map could dynamically grow
+ * as new cpu's are detected in the system via any platform specific
+ * method, such as ACPI for e.g.
+ */
+cpumask_t cpu_present_map __read_mostly;
+EXPORT_SYMBOL(cpu_present_map);
+
+#ifndef CONFIG_SMP
+
+/*
+ * Represents all cpu's that are currently online.
+ */
+cpumask_t cpu_online_map __read_mostly = CPU_MASK_ALL;
+EXPORT_SYMBOL(cpu_online_map);
+
+cpumask_t cpu_possible_map __read_mostly = CPU_MASK_ALL;
+EXPORT_SYMBOL(cpu_possible_map);
+
+#else /* CONFIG_SMP */
+
 /* Serializes the updates to cpu_online_map, cpu_present_map */
 static DEFINE_MUTEX(cpu_add_remove_lock);
 
@@ -403,3 +425,5 @@ out:
 	cpu_maps_update_done();
 }
 #endif /* CONFIG_PM_SLEEP_SMP */
+
+#endif /* CONFIG_SMP */
kernel/cpuset.c +13 -1
index 798b3ab..459d601 100644
@@ -1194,6 +1194,15 @@ static int cpuset_can_attach(struct cgroup_subsys *ss,
 
 	if (cpus_empty(cs->cpus_allowed) || nodes_empty(cs->mems_allowed))
 		return -ENOSPC;
+	if (tsk->flags & PF_THREAD_BOUND) {
+		cpumask_t mask;
+
+		mutex_lock(&callback_mutex);
+		mask = cs->cpus_allowed;
+		mutex_unlock(&callback_mutex);
+		if (!cpus_equal(tsk->cpus_allowed, mask))
+			return -EINVAL;
+	}
 
 	return security_task_setscheduler(tsk, 0, NULL);
 }
@@ -1207,11 +1216,14 @@ static void cpuset_attach(struct cgroup_subsys *ss,
 	struct mm_struct *mm;
 	struct cpuset *cs = cgroup_cs(cont);
 	struct cpuset *oldcs = cgroup_cs(oldcont);
+	int err;
 
 	mutex_lock(&callback_mutex);
 	guarantee_online_cpus(cs, &cpus);
-	set_cpus_allowed_ptr(tsk, &cpus);
+	err = set_cpus_allowed_ptr(tsk, &cpus);
 	mutex_unlock(&callback_mutex);
+	if (err)
+		return;
 
 	from = oldcs->mems_allowed;
 	to = cs->mems_allowed;
kernel/kthread.c +1 -0
index bd1b9ea..97747cd 100644
@@ -180,6 +180,7 @@ void kthread_bind(struct task_struct *k, unsigned int cpu)
 	set_task_cpu(k, cpu);
 	k->cpus_allowed = cpumask_of_cpu(cpu);
 	k->rt.nr_cpus_allowed = 1;
+	k->flags |= PF_THREAD_BOUND;
 }
 EXPORT_SYMBOL(kthread_bind);
 
kernel/sched.c +491 -232

1,331 lines not on this page · raw message

kernel/sched_clock.c +118 -19

243 lines not on this page · raw message

kernel/sched_cpupri.c new file +174 -0

174 lines not on this page · raw message

kernel/sched_cpupri.h new file +36 -0
new file mode 100644
index 0000000..f25811b
@@ -0,0 +1,36 @@
+#ifndef _LINUX_CPUPRI_H
+#define _LINUX_CPUPRI_H
+
+#include <linux/sched.h>
+
+#define CPUPRI_NR_PRIORITIES	(MAX_RT_PRIO + 2)
+#define CPUPRI_NR_PRI_WORDS	BITS_TO_LONGS(CPUPRI_NR_PRIORITIES)
+
+#define CPUPRI_INVALID -1
+#define CPUPRI_IDLE     0
+#define CPUPRI_NORMAL   1
+/* values 2-101 are RT priorities 0-99 */
+
+struct cpupri_vec {
+	spinlock_t lock;
+	int        count;
+	cpumask_t  mask;
+};
+
+struct cpupri {
+	struct cpupri_vec pri_to_cpu[CPUPRI_NR_PRIORITIES];
+	long              pri_active[CPUPRI_NR_PRI_WORDS];
+	int               cpu_to_pri[NR_CPUS];
+};
+
+#ifdef CONFIG_SMP
+int  cpupri_find(struct cpupri *cp,
+		 struct task_struct *p, cpumask_t *lowest_mask);
+void cpupri_set(struct cpupri *cp, int cpu, int pri);
+void cpupri_init(struct cpupri *cp);
+#else
+#define cpupri_set(cp, cpu, pri) do { } while (0)
+#define cpupri_init() do { } while (0)
+#endif
+
+#endif /* _LINUX_CPUPRI_H */
kernel/sched_debug.c +59 -5

91 lines not on this page · raw message

kernel/sched_fair.c +290 -123

606 lines not on this page · raw message

kernel/sched_features.h +5 -2
index 1c7283c..862b06b 100644
@@ -1,4 +1,5 @@
 SCHED_FEAT(NEW_FAIR_SLEEPERS, 1)
+SCHED_FEAT(NORMALIZED_SLEEPER, 1)
 SCHED_FEAT(WAKEUP_PREEMPT, 1)
 SCHED_FEAT(START_DEBIT, 1)
 SCHED_FEAT(AFFINE_WAKEUPS, 1)
@@ -6,5 +7,7 @@ SCHED_FEAT(CACHE_HOT_BUDDY, 1)
 SCHED_FEAT(SYNC_WAKEUPS, 1)
 SCHED_FEAT(HRTICK, 1)
 SCHED_FEAT(DOUBLE_TICK, 0)
-SCHED_FEAT(NORMALIZED_SLEEPER, 1)
-SCHED_FEAT(DEADLINE, 1)
+SCHED_FEAT(ASYM_GRAN, 1)
+SCHED_FEAT(LB_BIAS, 0)
+SCHED_FEAT(LB_WAKEUP_UPDATE, 1)
+SCHED_FEAT(ASYM_EFF_LOAD, 1)
kernel/sched_rt.c +264 -141

241 lines not on this page · raw message

kernel/sched_stats.h +33 -9

42 lines not on this page · raw message

kernel/sysctl.c +8 -0
index 2911665..fe8cdc8 100644

Lines of this file are not shown. View the raw message.

kernel/time/tick-sched.c +2 -0
index b854a89..d63008b 100644

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

Part of the diff is not shown here. View the raw message.