Skip to contentSkip to conversation

sched/core for v2.6.32

GIT PULL Ingo Molnar 5 participants 9 messages Sep 11, 2009 – Sep 13, 2009

Subsystems (7)
x86 VFS Tracing Scheduler +3 more subsystems
lore
Share & tools
lore
#0 lore raw

Linus,

Please pull the sched-core-for-linus git tree from:

git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip.git sched-core-for-linus

Highlights:

- Child-runs-first is now off - i.e. we run parent first. [ Warning: this might trigger races in user-space. ]

- Rewritten and cleaned up load-balancer

- Re-tuned and fixed the scheduler core for more interactivity

Thanks,

Ingo

------------------>
Andreas Herrmann (12):
sched: Use structure to store local data in __build_sched_domains
sched: Separate out allocation/free/goto-hell from __build_sched_domains
sched: Separate out build of NUMA sched domain from __build_sched_domains
sched: Separate out build of CPU sched domain from __build_sched_domains
sched: Separate out build of MC sched domain from __build_sched_domains
sched: Separate out build of SMT sched domain from __build_sched_domains
sched: Separate out build of SMT sched groups from __build_sched_domains
sched: Separate out build of MC sched groups from __build_sched_domains
sched: Separate out build of CPU sched groups from __build_sched_domains
sched: Separate out build of ALLNODES sched groups from __build_sched_domains
sched: Separate out build of NUMA sched groups from __build_sched_domains
sched: Consolidate definition of variable sd in __build_sched_domains

Anirban Sinha (1):
sched: Rename init_cfs_rq => init_tg_cfs_rq

Arjan van de Ven (1):
sched: Provide iowait counters

Arnd Bergmann (1):
sched: Add default defines for PREEMPT_ACTIVE

Frederic Weisbecker (8):
sched: Drop the need_resched() loop from cond_resched()
sched: Remove obsolete comment in __cond_resched()
sched: Cover the CONFIG_DEBUG_SPINLOCK_SLEEP off-case for __might_sleep()
sched: Add a preempt count base offset to __might_sleep()
sched: Remove the CONFIG_PREEMPT_BKL case definition of cond_resched()
sched: Pull up the might_sleep() check into cond_resched()
sched: Convert the only user of cond_resched_bkl to use cond_resched()
sched: Fix cond_resched_lock() in !CONFIG_PREEMPT

Gautham R Shenoy (1):
sched: Try to deal with low capacity, fix update_sd_power_savings_stats()

Gregory Haskins (2):
sched: Enhance the pre/post scheduling logic
sched: Fully integrate cpus_active_map and root-domain code

Hiroshi Shimamoto (2):
sched, task_struct: stack_canary is not needed without CC_STACKPROTECTOR
sched: Use for_each_class macro in move_one_task()

Hitoshi Mitake (1):
sched: Hide runqueues from direct reference at source code level for __raw_get_cpu_var()

Ingo Molnar (6):
sched: Fix cpupri build on !CONFIG_SMP
sched: Fix dynamic power-balancing crash
sched: Clean up topology.h
sched: Turn on SD_BALANCE_NEWIDLE
sched: Disable NEW_FAIR_SLEEPERS for now
sched: Fix sched::sched_stat_wait tracepoint field

Lennart Poettering (1):
sched: Introduce SCHED_RESET_ON_FORK scheduling policy flag

Mike Galbraith (6):
sched: Clean up SCHED_RESET_ON_FORK
sched: Add SCHED_RESET_ON_FORK functionality for nice < 0 tasks
sched: Ensure that a child can't gain time over it's parent after fork()
sched: Turn off child_runs_first
sched: Re-tune the scheduler latency defaults to decrease worst-case latencies
sched: Keep kthreads at default priority

Peter Zijlstra (18):
sched: Fix cgroup smp fairness
sched: Optimize unused cgroup configuration
sched: Add debug check to task_of()
sched: Ensure the migration task doesn't go away during use
sched: Avoid division by zero
sched: Fix division by zero - really
sched: Add wait, sleep and iowait accounting tracepoints
sched: Restore __cpu_power to a straight sum of power
sched: Add SD_PREFER_SIBLING
sched: Update the cpu_power sum during load-balance
sched: Add smt_gain
sched: Implement dynamic cpu_power
sched: Scale down cpu_power due to RT tasks
sched: Try to deal with low capacity
sched: Remove reciprocal for cpu_power
sched: Remove short cut from select_task_rq_fair()
sched: Deal with low-load in wake_affine()
sched: enable SD_WAKE_IDLE

Steven Rostedt (2):
sched: Check for pushing rt tasks after all scheduling
sched: Add new prio to cpupri before removing old prio

Thomas Gleixner (2):
sched: Fix return value of migration_init()
init: Move sched_clock_init after late_time_init

 arch/x86/include/asm/topology.h |   47 +-
 fs/dcache.c                     |    1 +
 fs/locks.c                      |    2 +-
 include/linux/hardirq.h         |    6 +
 include/linux/kernel.h          |    5 +-
 include/linux/sched.h           |   94 ++--
 include/linux/topology.h        |  168 ++++---
 include/trace/events/sched.h    |   95 ++++
 init/main.c                     |    2 +-
 kernel/kthread.c                |    4 -
 kernel/sched.c                  | 1099 +++++++++++++++++++++++++--------------
 kernel/sched_cpupri.c           |   30 +-
 kernel/sched_debug.c            |    4 +
 kernel/sched_fair.c             |   84 +++-
 kernel/sched_features.h         |    2 +-
 kernel/sched_rt.c               |   62 +--
 kernel/sysctl.c                 |   24 +-
 kernel/workqueue.c              |    2 -
 18 files changed, 1117 insertions(+), 614 deletions(-)
arch/x86/include/asm/topology.h +28 -19
index 066ef59..26d06e0 100644
@@ -129,25 +129,34 @@ extern unsigned long node_remap_size[];
 #endif
 
 /* sched_domains SD_NODE_INIT for NUMA machines */
-#define SD_NODE_INIT (struct sched_domain) {		\
-	.min_interval		= 8,			\
-	.max_interval		= 32,			\
-	.busy_factor		= 32,			\
-	.imbalance_pct		= 125,			\
-	.cache_nice_tries	= SD_CACHE_NICE_TRIES,	\
-	.busy_idx		= 3,			\
-	.idle_idx		= SD_IDLE_IDX,		\
-	.newidle_idx		= SD_NEWIDLE_IDX,	\
-	.wake_idx		= 1,			\
-	.forkexec_idx		= SD_FORKEXEC_IDX,	\
-	.flags			= SD_LOAD_BALANCE	\
-				| SD_BALANCE_EXEC	\
-				| SD_BALANCE_FORK	\
-				| SD_WAKE_AFFINE	\
-				| SD_WAKE_BALANCE	\
-				| SD_SERIALIZE,		\
-	.last_balance		= jiffies,		\
-	.balance_interval	= 1,			\
+#define SD_NODE_INIT (struct sched_domain) {				\
+	.min_interval		= 8,					\
+	.max_interval		= 32,					\
+	.busy_factor		= 32,					\
+	.imbalance_pct		= 125,					\
+	.cache_nice_tries	= SD_CACHE_NICE_TRIES,			\
+	.busy_idx		= 3,					\
+	.idle_idx		= SD_IDLE_IDX,				\
+	.newidle_idx		= SD_NEWIDLE_IDX,			\
+	.wake_idx		= 1,					\
+	.forkexec_idx		= SD_FORKEXEC_IDX,			\
+									\
+	.flags			= 1*SD_LOAD_BALANCE			\
+				| 1*SD_BALANCE_NEWIDLE			\
+				| 1*SD_BALANCE_EXEC			\
+				| 1*SD_BALANCE_FORK			\
+				| 0*SD_WAKE_IDLE			\
+				| 1*SD_WAKE_AFFINE			\
+				| 1*SD_WAKE_BALANCE			\
+				| 0*SD_SHARE_CPUPOWER			\
+				| 0*SD_POWERSAVINGS_BALANCE		\
+				| 0*SD_SHARE_PKG_RESOURCES		\
+				| 1*SD_SERIALIZE			\
+				| 1*SD_WAKE_IDLE_FAR			\
+				| 0*SD_PREFER_SIBLING			\
+				,					\
+	.last_balance		= jiffies,				\
+	.balance_interval	= 1,					\
 }
 
 #ifdef CONFIG_X86_64_ACPI_NUMA
fs/dcache.c +1 -0
index 9e5cd3c..a100fa3 100644
@@ -32,6 +32,7 @@
 #include <linux/swap.h>
 #include <linux/bootmem.h>
 #include <linux/fs_struct.h>
+#include <linux/hardirq.h>
 #include "internal.h"
 
 int sysctl_vfs_cache_pressure __read_mostly = 100;
fs/locks.c +1 -1
index b6440f5..2eb8197 100644
@@ -768,7 +768,7 @@ static int flock_lock_file(struct file *filp, struct file_lock *request)
 	 * give it the opportunity to lock the file.
 	 */
 	if (found)
-		cond_resched_bkl();
+		cond_resched();
 
 find_conflict:
 	for_each_lock(inode, before) {
include/linux/hardirq.h +6 -0
index 8246c69..0d885fd 100644
@@ -64,6 +64,12 @@
 #define HARDIRQ_OFFSET	(1UL << HARDIRQ_SHIFT)
 #define NMI_OFFSET	(1UL << NMI_SHIFT)
 
+#ifndef PREEMPT_ACTIVE
+#define PREEMPT_ACTIVE_BITS	1
+#define PREEMPT_ACTIVE_SHIFT	(NMI_SHIFT + NMI_BITS)
+#define PREEMPT_ACTIVE	(__IRQ_MASK(PREEMPT_ACTIVE_BITS) << PREEMPT_ACTIVE_SHIFT)
+#endif
+
 #if PREEMPT_ACTIVE < (1 << (NMI_SHIFT + NMI_BITS))
 #error PREEMPT_ACTIVE is too low!
 #endif
include/linux/kernel.h +3 -2
index d6320a3..2b5b1e0 100644
@@ -125,7 +125,7 @@ extern int _cond_resched(void);
 #endif
 
 #ifdef CONFIG_DEBUG_SPINLOCK_SLEEP
-  void __might_sleep(char *file, int line);
+  void __might_sleep(char *file, int line, int preempt_offset);
 /**
  * might_sleep - annotation for functions that can sleep
  *
@@ -137,8 +137,9 @@ extern int _cond_resched(void);
  * supposed to.
  */
 # define might_sleep() \
-	do { __might_sleep(__FILE__, __LINE__); might_resched(); } while (0)
+	do { __might_sleep(__FILE__, __LINE__, 0); might_resched(); } while (0)
 #else
+  static inline void __might_sleep(char *file, int line, int preempt_offset) { }
 # define might_sleep() do { might_resched(); } while (0)
 #endif
 
include/linux/sched.h +55 -39

172 lines not on this page · raw message

include/linux/topology.h +101 -67

190 lines not on this page · raw message

include/trace/events/sched.h +95 -0

101 lines not on this page · raw message

init/main.c +1 -1
index 11f4f14..0ec75ce 100644
@@ -631,7 +631,6 @@ asmlinkage void __init start_kernel(void)
 	softirq_init();
 	timekeeping_init();
 	time_init();
-	sched_clock_init();
 	profile_init();
 	if (!irqs_disabled())
 		printk(KERN_CRIT "start_kernel(): bug: interrupts were "
@@ -682,6 +681,7 @@ asmlinkage void __init start_kernel(void)
 	numa_policy_init();
 	if (late_time_init)
 		late_time_init();
+	sched_clock_init();
 	calibrate_delay();
 	pidmap_init();
 	anon_vma_init();
kernel/kthread.c +0 -4
index eb8751a..5fe7099 100644
@@ -16,8 +16,6 @@
 #include <linux/mutex.h>
 #include <trace/events/sched.h>
 
-#define KTHREAD_NICE_LEVEL (-5)
-
 static DEFINE_SPINLOCK(kthread_create_lock);
 static LIST_HEAD(kthread_create_list);
 struct task_struct *kthreadd_task;
@@ -145,7 +143,6 @@ struct task_struct *kthread_create(int (*threadfn)(void *data),
 		 * The kernel thread should not inherit these properties.
 		 */
 		sched_setscheduler_nocheck(create.result, SCHED_NORMAL, &param);
-		set_user_nice(create.result, KTHREAD_NICE_LEVEL);
 		set_cpus_allowed_ptr(create.result, cpu_all_mask);
 	}
 	return create.result;
@@ -221,7 +218,6 @@ int kthreadd(void *unused)
 	/* Setup a clean context for our children to inherit. */
 	set_task_comm(tsk, "kthreadd");
 	ignore_signals(tsk);
-	set_user_nice(tsk, KTHREAD_NICE_LEVEL);
 	set_cpus_allowed_ptr(tsk, cpu_all_mask);
 	set_mems_allowed(node_possible_map);
 
kernel/sched.c +702 -397

1,662 lines not on this page · raw message

kernel/sched_cpupri.c +16 -14
index d014efb..0f052fc 100644
@@ -127,21 +127,11 @@ void cpupri_set(struct cpupri *cp, int cpu, int newpri)
 
 	/*
 	 * If the cpu was currently mapped to a different value, we
-	 * first need to unmap the old value
+	 * need to map it to the new value then remove the old value.
+	 * Note, we must add the new value first, otherwise we risk the
+	 * cpu being cleared from pri_active, and this cpu could be
+	 * missed for a push or pull.
 	 */
-	if (likely(oldpri != CPUPRI_INVALID)) {
-		struct cpupri_vec *vec  = &cp->pri_to_cpu[oldpri];
-
-		spin_lock_irqsave(&vec->lock, flags);
-
-		vec->count--;
-		if (!vec->count)
-			clear_bit(oldpri, cp->pri_active);
-		cpumask_clear_cpu(cpu, vec->mask);
-
-		spin_unlock_irqrestore(&vec->lock, flags);
-	}
-
 	if (likely(newpri != CPUPRI_INVALID)) {
 		struct cpupri_vec *vec = &cp->pri_to_cpu[newpri];
 
@@ -154,6 +144,18 @@ void cpupri_set(struct cpupri *cp, int cpu, int newpri)
 
 		spin_unlock_irqrestore(&vec->lock, flags);
 	}
+	if (likely(oldpri != CPUPRI_INVALID)) {
+		struct cpupri_vec *vec  = &cp->pri_to_cpu[oldpri];
+
+		spin_lock_irqsave(&vec->lock, flags);
+
+		vec->count--;
+		if (!vec->count)
+			clear_bit(oldpri, cp->pri_active);
+		cpumask_clear_cpu(cpu, vec->mask);
+
+		spin_unlock_irqrestore(&vec->lock, flags);
+	}
 
 	*currpri = newpri;
 }
kernel/sched_debug.c +4 -0
index 70c7e0b..5ddbd08 100644
@@ -409,6 +409,8 @@ void proc_sched_show_task(struct task_struct *p, struct seq_file *m)
 	PN(se.wait_max);
 	PN(se.wait_sum);
 	P(se.wait_count);
+	PN(se.iowait_sum);
+	P(se.iowait_count);
 	P(sched_info.bkl_count);
 	P(se.nr_migrations);
 	P(se.nr_migrations_cold);
@@ -479,6 +481,8 @@ void proc_sched_set_task(struct task_struct *p)
 	p->se.wait_max				= 0;
 	p->se.wait_sum				= 0;
 	p->se.wait_count			= 0;
+	p->se.iowait_sum			= 0;
+	p->se.iowait_count			= 0;
 	p->se.sleep_max				= 0;
 	p->se.sum_sleep_runtime			= 0;
 	p->se.block_max				= 0;
kernel/sched_fair.c +60 -24

206 lines not on this page · raw message

kernel/sched_features.h +1 -1
index 4569bfa..e2dc63a 100644
@@ -1,4 +1,4 @@
-SCHED_FEAT(NEW_FAIR_SLEEPERS, 1)
+SCHED_FEAT(NEW_FAIR_SLEEPERS, 0)
 SCHED_FEAT(NORMALIZED_SLEEPER, 0)
 SCHED_FEAT(ADAPTIVE_GRAN, 1)
 SCHED_FEAT(WAKEUP_PREEMPT, 1)
kernel/sched_rt.c +27 -35

136 lines not on this page · raw message

kernel/sysctl.c +16 -8
index 58be760..25d6bf3 100644
@@ -246,6 +246,14 @@ static int max_wakeup_granularity_ns = NSEC_PER_SEC;	/* 1 second */
 #endif
 
 static struct ctl_table kern_table[] = {
+	{
+		.ctl_name	= CTL_UNNUMBERED,
+		.procname	= "sched_child_runs_first",
+		.data		= &sysctl_sched_child_runs_first,
+		.maxlen		= sizeof(unsigned int),
+		.mode		= 0644,
+		.proc_handler	= &proc_dointvec,
+	},
 #ifdef CONFIG_SCHED_DEBUG
 	{
 		.ctl_name	= CTL_UNNUMBERED,
@@ -300,14 +308,6 @@ static struct ctl_table kern_table[] = {
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
-		.procname	= "sched_child_runs_first",
-		.data		= &sysctl_sched_child_runs_first,
-		.maxlen		= sizeof(unsigned int),
-		.mode		= 0644,
-		.proc_handler	= &proc_dointvec,
-	},
-	{
-		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "sched_features",
 		.data		= &sysctl_sched_features,
 		.maxlen		= sizeof(unsigned int),
@@ -332,6 +332,14 @@ static struct ctl_table kern_table[] = {
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
+		.procname	= "sched_time_avg",
+		.data		= &sysctl_sched_time_avg,
+		.maxlen		= sizeof(unsigned int),
+		.mode		= 0644,
+		.proc_handler	= &proc_dointvec,
+	},
+	{
+		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "timer_migration",
 		.data		= &sysctl_timer_migration,
 		.maxlen		= sizeof(unsigned int),
kernel/workqueue.c +0 -2
index 0668795..ea1b4e7 100644
@@ -317,8 +317,6 @@ static int worker_thread(void *__cwq)
 	if (cwq->wq->freezeable)
 		set_freezable();
 
-	set_user_nice(current, -5);
-
 	for (;;) {
 		prepare_to_wait(&cwq->more_work, &wait, TASK_INTERRUPTIBLE);
 		if (!freezing(current) &&
#1 lore raw

[...]

Quoted 4 lines

Highlights:

- Child-runs-first is now off - i.e. we run parent first. [ Warning: this might trigger races in user-space. ]

[...]

Ouch. Do we dare do that?

vfork() is supposed to always run the child first.
Most people I've talked to over the years assume that using fork(), the child runs first (yes, I know, that's not guaranteed, but people have come to believe that it is so and some may even depend on it).

I must admit that I do not have the understanding of the code needed to see why this is done, I'm just scared that this may cause lots of problems for userspace programs that (for whatever reason) have come to rely on child-runs-first semantics.

I assume there's a very good reason to do this, I'm just not sure we want to do it regardless of what (I assume) performance bennefits it brings. Will userspace apps really bennefit from this (In the real world, not in some fantasy world where everyone reads standards and do things then "proper" way)?

Signature · 3 lines
Jesper Juhl <[email protected]>             http://www.chaosbits.net/
Plain text mails only, please      http://www.expita.com/nomime.html
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
#2 lore raw
Quoted 8 lines · Jesper Juhl

[...]

Quoted 4 lines

Highlights:

- Child-runs-first is now off - i.e. we run parent first. [ Warning: this might trigger races in user-space. ]

[...]

Ouch. Do we dare do that?

We would want to at least try.

There are various reasons why we'd like to run the child first, ranging from just pure latency (quite often, the child is the one that is critical) to getting rid of page sharing for COW early thanks to execve etc.

But similarly, there are various reasons to run the parent first, like just the fact that we already have the state active in the TLB's and caches.

Finally, we've never made any guarantees, because the timeslice for the parent might be just about to end, so child-first vs parent-first is never a guarantee, it's always just a preference.

[ And we _have_ had that preference expose user-level bugs. Long long ago
we hit some problem with child-runs-first and 'bash' being unhappy about a really low-cost and quick child process exiting even _before_ bash itself had had time to fill in the process tables, and then when the SIGCHLD handler ran bash said "I got a SIGCHLD for something I don't even know about".

That was very much a bash bug, but it was a bash bug that forced us to do 'parent-runs-first' for a while. So the heuristic can show problems ]

Quoted 1 line

vfork() is supposed to always run the child first.

vfork() has always run the child first, since the parent won't even be runnable. The parent will get stuck in

	wait_for_completion(&vfork);

so the "child-runs-first" is just an issue for regular fork or clone, not vfork. For vfork there is never any question about it.

Quoted 3 lines

Most people I've talked to over the years assume that using fork(), the child runs first (yes, I know, that's not guaranteed, but people have come to believe that it is so and some may even depend on it).

It really hasn't been that way in Linux. We've done it both ways.

Linus

#3 lore raw
Quoted 19 lines · Linus Torvalds
Quoted 8 lines · Jesper Juhl

[...]

Quoted 4 lines

Highlights:

- Child-runs-first is now off - i.e. we run parent first. [ Warning: this might trigger races in user-space. ]

[...]

Ouch. Do we dare do that?

We would want to at least try.

There are various reasons why we'd like to run the child first, ranging from just pure latency (quite often, the child is the one that is critical) to getting rid of page sharing for COW early thanks to execve etc.

Quoted 8 lines

But similarly, there are various reasons to run the parent first, like just the fact that we already have the state active in the TLB's and caches.

Finally, we've never made any guarantees, because the timeslice for the parent might be just about to end, so child-first vs parent-first is never a guarantee, it's always just a preference.

Sure. I'm aware that it has never been a guarantee. I'm just worried that userspace programs have come to rely on a certain behaviour and changing that behaviour may result in undesired results for some apps. In a perfect world people would just fix those apps that (incorrectly) relied on a certain child-/parent-runs-first behaviour, but the world is not perfect, and many apps may not even have source available.

Quoted 10 lines

[ And we _have_ had that preference expose user-level bugs. Long long ago
we hit some problem with child-runs-first and 'bash' being unhappy about a really low-cost and quick child process exiting even _before_ bash itself had had time to fill in the process tables, and then when the SIGCHLD handler ran bash said "I got a SIGCHLD for something I don't even know about".

That was very much a bash bug, but it was a bash bug that forced us to do 'parent-runs-first' for a while. So the heuristic can show problems ]

In some far off corner of my mind I think I actually remember that one... But, do we really want to risk that sort of problems again? Since then Linux has moved from being a toy to being a first-class operating system that people use to do real work. And like it or not, most people who write apps do not read documentation, they do not read standards, they do not care how things are supposed to behave - they just see a certain behaviour when testing and assume that it is always so. Those people may be in for a nasty surprise with this change. I'm not saying that people who assume specific behaviour that is not guaranteed are not wrong and that their code is not broken - all I'm saying is that we should have a pretty damn good reason for breaking their assumptions - if we can allow their assumptions to be true and their code to work without it costing us much, then I think we should...

Quoted 10 lines
Quoted 1 line

vfork() is supposed to always run the child first.

vfork() has always run the child first, since the parent won't even be runnable. The parent will get stuck in

	wait_for_completion(&vfork);

so the "child-runs-first" is just an issue for regular fork or clone, not vfork. For vfork there is never any question about it.

Good. Then at least one can use vfork() if one knows that one needs child-runs-first semantics (and knows the differences between vfork() and fork()). Good to hear that that's not going to break - that would have been bad.

Quoted 6 lines
Quoted 3 lines

Most people I've talked to over the years assume that using fork(), the child runs first (yes, I know, that's not guaranteed, but people have come to believe that it is so and some may even depend on it).

It really hasn't been that way in Linux. We've done it both ways.

That may be so; but most people I've ever talked to about multiple processes, fork, vfork and the like, have mostly assumed child-runs-first. That is just my personal experience.
So I get worried when that assumption is made false.

Signature · 3 lines
Jesper Juhl <[email protected]>             http://www.chaosbits.net/
Plain text mails only, please      http://www.expita.com/nomime.html
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
#4 lore raw
Quoted 4 lines · Jesper Juhl

That may be so; but most people I've ever talked to about multiple processes, fork, vfork and the like, have mostly assumed child-runs-first. That is just my personal experience.
So I get worried when that assumption is made false.

With multi-core cpus becoming (being?) the norm, almost all systems are SMP now. So child and parent can surely end up running in parallel very often. So applications that make assumptions about child running first are going to be frequently surprised. Aren't they?

-Tony

#5 lore raw
Quoted 11 lines · Tony Luck
Quoted 4 lines · Jesper Juhl

That may be so; but most people I've ever talked to about multiple processes, fork, vfork and the like, have mostly assumed child-runs-first. That is just my personal experience.
So I get worried when that assumption is made false.

With multi-core cpus becoming (being?) the norm, almost all systems are SMP now. So child and parent can surely end up running in parallel very often. So applications that make assumptions about child running first are going to be frequently surprised. Aren't they?

We had parent-runs-first briefly, in v2.6.23 - this got changed by v2.6.24 - but yes, it did trigger at least one app bug that i remember (dont remember which one it was though).

We are almost two years later now - maybe it works fine now.

In any case, as a precaution i made the sched_child_runs_first sysctl knob unconditional (previously it was under CONFIG_SCHED_DEBUG).

So if an old distro is upgraded with a new kernel (and user-space is not updated), it can be worked around by putting this into /etc/sysctl.conf:

kernel.sched_child_runs_first = 1

You are right to suggest that due to SMP and due to the general non-determinism of preemption we _never_ made any 'promise' to run the child first.

It was a statistical property based on performance considerations - and now we flipped it around based on latency and for kbuild performance/throughput reasons: Serge Belyshev reported a 7% increase on a quad due to this change and i measured a 1.5% peak-kbuild performance increase.

So it's worth it for multiple reasons and even in the worst-case problems can be worked around easily and without rebooting the system.

Ingo

#6 lore raw
Quoted 25 lines · Ingo Molnar
Quoted 11 lines · Tony Luck
Quoted 4 lines · Jesper Juhl

That may be so; but most people I've ever talked to about multiple processes, fork, vfork and the like, have mostly assumed child-runs-first. That is just my personal experience.
So I get worried when that assumption is made false.

With multi-core cpus becoming (being?) the norm, almost all systems are SMP now. So child and parent can surely end up running in parallel very often. So applications that make assumptions about child running first are going to be frequently surprised. Aren't they?

We had parent-runs-first briefly, in v2.6.23 - this got changed by v2.6.24 - but yes, it did trigger at least one app bug that i remember (dont remember which one it was though).

We are almost two years later now - maybe it works fine now.

In any case, as a precaution i made the sched_child_runs_first sysctl knob unconditional (previously it was under CONFIG_SCHED_DEBUG).

Quoted 6 lines

So if an old distro is upgraded with a new kernel (and user-space is not updated), it can be worked around by putting this into /etc/sysctl.conf:

kernel.sched_child_runs_first = 1

Always goo to have a workaround if something unexpectedly breaks. Makes perfect sense for that knob to now be unconditional - and with it, changing the behaviour is a much less risky operation. Glad to have it.

Quoted 10 lines

You are right to suggest that due to SMP and due to the general non-determinism of preemption we _never_ made any 'promise' to run the child first.

It was a statistical property based on performance considerations - and now we flipped it around based on latency and for kbuild performance/throughput reasons: Serge Belyshev reported a 7% increase on a quad due to this change and i measured a 1.5% peak-kbuild performance increase.

Impressive. I wouldn't have expected that much gain by running the parent first. Actually I personally would have expected child-first to perform better since (in my experience) it's usually the child that's just forked that matters the most. A server process that spawns a child to service a request wants the child to run asap and process that forks, then execs cares primarily about the child.

Quoted 4 lines

So it's worth it for multiple reasons and even in the worst-case problems can be worked around easily and without rebooting the system.

Perfect :-)

Signature · 3 lines
Jesper Juhl <[email protected]>             http://www.chaosbits.net/
Plain text mails only, please      http://www.expita.com/nomime.html
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
#7 lore raw
Quoted 9 lines · Jesper Juhl
Quoted 6 lines · Ingo Molnar

It was a statistical property based on performance considerations - and now we flipped it around based on latency and for kbuild performance/throughput reasons: Serge Belyshev reported a 7% increase on a quad due to this change and i measured a 1.5% peak-kbuild performance increase.

Quoted 4 lines

Impressive. I wouldn't have expected that much gain by running the parent first. Actually I personally would have expected child-first to perform better since (in my experience) it's usually the child that's just forked that matters the most.

How can waiting for child1 to run a bit before forking off child2 _not_ hurt? The parent is the worker bee creator, the queen bee if you will. Seems to me that making the queen wait until one egg hatches and ages a bit before laying another egg is a very bad plan if the goal is to have a hive full of short lived worker bees.

-Mike

#8 lore raw
Quoted 15 lines · Mike Galbraith
Quoted 9 lines · Jesper Juhl
Quoted 6 lines · Ingo Molnar

It was a statistical property based on performance considerations - and now we flipped it around based on latency and for kbuild performance/throughput reasons: Serge Belyshev reported a 7% increase on a quad due to this change and i measured a 1.5% peak-kbuild performance increase.

Quoted 4 lines

Impressive. I wouldn't have expected that much gain by running the parent first. Actually I personally would have expected child-first to perform better since (in my experience) it's usually the child that's just forked that matters the most.

Quoted 6 lines

How can waiting for child1 to run a bit before forking off child2 _not_ hurt? The parent is the worker bee creator, the queen bee if you will. Seems to me that making the queen wait until one egg hatches and ages a bit before laying another egg is a very bad plan if the goal is to have a hive full of short lived worker bees.

When I wrote the above, I was thinking of one specific case; a server that receives a request, forks off a child to handle the request and the performance we measure is of how long it takes to handle that single request - and in that specific scenario we want the child to run as soon as possible.

Signature · 3 lines
Jesper Juhl <[email protected]>             http://www.chaosbits.net/
Plain text mails only, please      http://www.expita.com/nomime.html
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html