fix the itimer regression (BZ 12618)
Subsystems (5)
cover fix the itimer regression (BZ 12618)
This should hopefully address all the itimer borkage.
PATCH 1/2 signal: re-add dead task accumulation stats.
We're going to split the process wide cpu accounting into two parts:
- clocks; which can take all the time they want since they run from user context.
- timers; which need constant time tracing but can affort the overhead because they're default off -- and rare.
The clock readout will go back to a full sum of the thread group, for this we need to re-add the exit stats that were removed in the initial itimer rework (f06febc9: timers: fix itimer/many thread hang).
Furthermore, since that full sum can be rather slow for large thread groups and we have the complete dead task stats, revert the do_notify_parent time computation.
--- include/linux/sched.h | 10 +++++++++- kernel/exit.c | 3 +++ kernel/fork.c | 3 ++- 3 files changed, 14 insertions(+), 2 deletions(-)
include/linux/sched.h +9 -1
@@ -567,7 +567,7 @@ struct signal_struct {* Live threads maintain their own counters and add to these * in __exit_signal, except for the group leader. */ -cputime_t cutime, cstime; +cputime_t utime, stime, cutime, cstime;cputime_t gtime;cputime_t cgtime;unsigned long nvcsw, nivcsw, cnvcsw, cnivcsw;
@@ -576,6 +576,14 @@ struct signal_struct {struct task_io_accounting ioac;/* +* Cumulative ns of schedule CPU time fo dead threads in the +* group, not including a zombie group leader, (This only differs +* from jiffies_to_ns(utime + stime) if sched_clock uses something +* other than jiffies.) +*/ +unsigned long long sum_sched_runtime; + +/* * We don't bother to synchronize most readers of this at all, * because there is no reader checking a limit that actually needs * to get both rlim_cur and rlim_max atomically, and either one
kernel/exit.c +3 -0
@@ -118,6 +118,8 @@ static void __exit_signal(struct task_st* We won't ever get here for the group leader, since it * will have been the last reference on the signal_struct. */ + sig->utime =cputime_add (sig->utime,task_utime (tsk)); + sig->stime =cputime_add (sig->stime,task_stime (tsk)); sig->gtime =cputime_add (sig->gtime,task_gtime (tsk)); sig->min_flt += tsk->min_flt; sig->maj_flt += tsk->maj_flt;
@@ -126,6 +128,7 @@ static void __exit_signal(struct task_st sig->inblock +=task_io_get_inblock (tsk); sig->oublock +=task_io_get_oublock (tsk);task_io_accounting_add (&sig->ioac, &tsk->ioac); + sig->sum_sched_runtime += tsk->se.sum_exec_runtime; sig =NULL ;/* Marker for below. */ }
kernel/fork.c +2 -1
@@ -856,13 +856,14 @@ static int copy_signal(unsigned long clo sig->tty_old_pgrp =NULL ; sig->tty =NULL ; - sig->cutime = sig->cstime = cputime_zero; + sig->utime = sig->stime = sig->cutime = sig->cstime = cputime_zero; sig->gtime = cputime_zero; sig->cgtime = cputime_zero; sig->nvcsw = sig->nivcsw = sig->cnvcsw = sig->cnivcsw =0 ; sig->min_flt = sig->maj_flt = sig->cmin_flt = sig->cmaj_flt =0 ; sig->inblock = sig->oublock = sig->cinblock = sig->coublock =0 ;task_io_accounting_init (&sig->ioac); + sig->sum_sched_runtime =0 ;taskstats_tgid_init (sig);task_lock (current->group_leader);
kernel/signal.c +4 -4
@@ -1367,7 +1367,6 @@ int do_notify_parent(struct task_structstruct siginfo info;unsigned long flags;struct sighand_struct *psig; -struct task_cputime cputime;int ret = sig;BUG_ON (sig ==-1 );
@@ -1397,9 +1396,10 @@ int do_notify_parent(struct task_struct info.si_uid =__task_cred (tsk)->uid;rcu_read_unlock (); -thread_group_cputime (tsk, &cputime); - info.si_utime =cputime_to_jiffies (cputime.utime); - info.si_stime =cputime_to_jiffies (cputime.stime); + info.si_utime =cputime_to_clock_t (cputime_add (tsk->utime, + tsk->signal->utime)); + info.si_stime =cputime_to_clock_t (cputime_add (tsk->stime, + tsk->signal->stime)); info.si_status = tsk->exit_code &0x7f ;if (tsk->exit_code &0x80 )
PATCH 2/2 timers: split process wide cpu clocks/timers
In reply to Peter ZijlstraChange the process wide cpu timers/clocks so that we:
1) don't mess up the kernel with too many threads,
2) don't have a per-cpu allocation for each process,
3) have no impact when not used.
In order to accomplish this we're going to split it into two parts:
- clocks; which can take all the time they want since they run from user context -- ie. sys_clock_gettime(CLOCK_PROCESS_CPUTIME_ID)
- timers; which need constant time sampling but since they're explicity used, the user can pay the overhead.
The clock readout will go back to a full sum of the thread group, while the timers will run of a global 'clock' that only runs when needed, so only programs that make use of the facility pay the price.
--- include/linux/init_task.h | 11 ++--- include/linux/sched.h | 54 +++++++++++++++----------- kernel/itimer.c | 4 - kernel/posix-cpu-timers.c | 95 ++++++++++++++++++++++++++++++++++++++++++++-- kernel/sched_stats.h | 45 ++++++++++++--------- 5 files changed, 155 insertions(+), 54 deletions(-)
include/linux/init_task.h +5 -6
@@ -48,12 +48,11 @@ extern struct fs_struct init_fs; .posix_timers = LIST_HEAD_INIT(sig.posix_timers), \ .cpu_timers = INIT_CPU_TIMERS(sig.cpu_timers), \ .rlim = INIT_RLIMITS, \ - .cputime = { .totals = { \ - .utime = cputime_zero, \ - .stime = cputime_zero, \ - .sum_exec_runtime = 0, \ - .lock = __SPIN_LOCK_UNLOCKED(sig.cputime.totals.lock), \ - }, }, \ + .cputimer = { \ + .cputime = INIT_CPUTIME, \ + .running = 0, \ + .lock = __SPIN_LOCK_UNLOCKED(sig.cputimer.lock), \ + }, \ }extern struct nsproxy init_nsproxy;
include/linux/sched.h +31 -23
98 lines not on this page · raw message
kernel/itimer.c +2 -2
@@ -62,7 +62,7 @@ int do_getitimer(int which, struct itimestruct task_cputime cputime;cputime_t utime; -thread_group_cputime (tsk, &cputime); +thread_group_cputimer (tsk, &cputime); utime = cputime.utime;if (cputime_le (cval, utime)) {/* about to fire */ cval =jiffies_to_cputime (1 );
@@ -82,7 +82,7 @@ int do_getitimer(int which, struct itimestruct task_cputime times;cputime_t ptime; -thread_group_cputime (tsk, ×); +thread_group_cputimer (tsk, ×); ptime =cputime_add (times.utime, times.stime);if (cputime_le (cval, ptime)) {/* about to fire */ cval =jiffies_to_cputime (1 );
kernel/posix-cpu-timers.c +91 -4
142 lines not on this page · raw message
kernel/sched_stats.h +26 -19
80 lines not on this page · raw message
Re: [PATCH 2/2] timers: split process wide cpu clocks/timers
Quoted 12 lines · Peter Zijlstra
Change the process wide cpu timers/clocks so that we:
1) don't mess up the kernel with too many threads,
2) don't have a per-cpu allocation for each process,
3) have no impact when not used.In order to accomplish this we're going to split it into two parts:
- clocks; which can take all the time they want since they run from user context -- ie.
sys_clock_gettime(CLOCK_PROCESS_CPUTIME_ID)
Quoted 6 lines
- timers; which need constant time sampling but since they're explicity used, the user can pay the overhead.
The clock readout will go back to a full sum of the thread group, while the timers will run of a global 'clock' that only runs when needed, so only programs that make use of the facility pay the price.
Ah, personally I think this is a very nice idea!
A couple of minor questions, before I try to read this patch more...
include/linux/sched.h static inline int spin_needbreak(spinloc · line 2233
Quoted 3 lines
static inline
include/linux/sched.h +1
-1-void thread_group_cputime (struct task_struct *tsk,struct task_cputime *times) +void thread_group_cputimer (struct task_struct *tsk,struct task_cputime *times)
I know, it is silly to complain about the naming, but can't resist.
Now we have both thread_group_cputime() and thread_group_cputimer(). But it is not possible to distinguish them while reading the code. For example, looks like posix_cpu_timers_exit_group() needs thread_group_cputimer, not thread_group_cputime, no? But then we can hit the WARN_ON(!cputimer->running). Afaics.
Hmm... Can't fastpath_timer_check()->thread_group_cputimer() have the false warning too? Suppose we had the timer, then posix_cpu_timer_del() removes this timer, but task_cputime_zero(&sig->cputime_expires) still not true.
Quoted 4 lines
kernel/posix-cpu-timers.c +4
-0+void thread_group_cputime (struct task_struct *tsk,struct task_cputime *times) +{ +struct sighand_struct *sighand; +struct signal_struct *sig;
kernel/posix-cpu-timers.c static int cpu_clock_sample(const clocki · line 246
Quoted 10 lines
kernel/posix-cpu-timers.c +10
-0+struct task_struct *t; + + *times =INIT_CPUTIME ; + +rcu_read_lock (); + sighand =rcu_dereference (tsk->sighand); +if (!sighand) + goto out; + + sig = tsk->signal;
I am afraid to be wrong, but it looks like we always call this function when we know we must have a valid ->sighand/siglock. Perhaps we do not need any check?
IOW, unless I missed something we should not just return if there is no ->sighand or ->signal, this just hides the problem while we should fix the caller.
Quoted 11 lines
+ * Enable the process wide cpu timer accounting.
+ *
+ * serialized using ->sighand->siglock
+ */kernel/posix-cpu-timers.c +7
-0+static void start_process_timers (struct task_struct *tsk) +{ + tsk->signal->cputimer.running =1 ; +barrier (); +} + +/*
kernel/posix-cpu-timers.c static void clear_dead_task(struct k_iti · line 530
Quoted 10 lines
kernel/posix-cpu-timers.c +10
-0+* Release the process wide timer accounting -- timer stops ticking when +* nobody cares about it. +* +* serialized using ->sighand->siglock +*/ +static void stop_process_timers (struct task_struct *tsk) +{ + tsk->signal->cputimer.running =0 ; +barrier (); +}
Could you explain these barriers?
And, I am a bit lost... set_process_cpu_timer() does cpu_timer_sample_group(), but posix_cpu_timer_set() does cpu_clock_sample_group() in !CPUCLOCK_PERTHREAD case, could you confirm this is correct?
Oleg.
Re: [PATCH 2/2] timers: split process wide cpu clocks/timers
Quoted 13 lines · Oleg Nesterov
Quoted 12 lines · Peter Zijlstra
Change the process wide cpu timers/clocks so that we:
1) don't mess up the kernel with too many threads,
2) don't have a per-cpu allocation for each process,
3) have no impact when not used.In order to accomplish this we're going to split it into two parts:
- clocks; which can take all the time they want since they run from user context -- ie.
sys_clock_gettime(CLOCK_PROCESS_CPUTIME_ID)
Quoted 8 lines
Quoted 6 lines
- timers; which need constant time sampling but since they're explicity used, the user can pay the overhead.
The clock readout will go back to a full sum of the thread group, while the timers will run of a global 'clock' that only runs when needed, so only programs that make use of the facility pay the price.
Ah, personally I think this is a very nice idea!
Thanks.
Quoted 10 lines
A couple of minor questions, before I try to read this patch more...
Review
Quoted 3 lines
static inline
(quoted diff) +1
-1-void thread_group_cputime(struct task_struct *tsk, struct task_cputime *times) +void thread_group_cputimer(struct task_struct *tsk, struct task_cputime *times)I know, it is silly to complain about the naming, but can't resist.
Now we have both
thread_group_cputime()andthread_group_cputimer(). But it is not possible to distinguish them while reading the code.
Good point, s/thread_group_cputime/thread_group_sum_time/g ?
Quoted 3 lines
For example, looks like
posix_cpu_timers_exit_group()needsthread_group_cputimer, notthread_group_cputime, no? But then we can hit theWARN_ON(!cputimer->running). Afaics.
Yes, I think you're right. Although does it really matter what we do here, can anybody still access the remaining time after we clean up the whole thread group?
Quoted 4 lines
Hmm... Can't fastpath_timer_check()->thread_group_cputimer() have the false warning too? Suppose we had the timer, then
posix_cpu_timer_del()removes this timer, buttask_cputime_zero(&sig->cputime_expires)still not true.
Another good spotting, this appears to be the only site not under siglock or tasklist_lock. Yes in that case there can be a false positive.
I'd better remove that warning.
Quoted 10 lines
Quoted 10 lines
(quoted diff) +10
-0+void thread_group_cputime(struct task_struct *tsk, struct task_cputime *times) +{ + struct sighand_struct *sighand; + struct signal_struct *sig; + struct task_struct *t; + + *times = INIT_CPUTIME; + + rcu_read_lock(); + sighand = rcu_dereference(tsk->sighand);
Quoted 8 lines
Review
Quoted 4 lines
(quoted diff) +4
-0+ if (!sighand) + goto out; + + sig = tsk->signal;I am afraid to be wrong, but it looks like we always call this function when we know we must have a valid ->sighand/siglock. Perhaps we do not need any check?
I think you're right, but paranoia won out.
Quoted 3 lines
IOW, unless I missed something we should not just return if there is no ->sighand or ->signal, this just hides the problem while we should fix the caller.
Might be a patch for .30-rc1 and then fix up everything that falls over. Agreed.
Quoted 15 lines
Quoted 15 lines
+ * Enable the process wide cpu timer accounting.
+ *
+ * serialized using ->sighand->siglock
+ */(quoted diff) +11
-0+static void start_process_timers(struct task_struct *tsk) +{ + tsk->signal->cputimer.running = 1; + barrier(); +} + +/* + * Release the process wide timer accounting -- timer stops ticking when + * nobody cares about it. + * + * serialized using ->sighand->siglock
Quoted 8 lines
Review
Quoted 6 lines
+ */
(quoted diff) +5
-0+static void stop_process_timers(struct task_struct *tsk) +{ + tsk->signal->cputimer.running = 0; + barrier(); +}Could you explain these barriers?
I was thinking that since account_group_*time() can run concurrently, its best to issue the write asap, smp barriers seemed overkill since its not a correctness issue.
Quoted 3 lines
And, I am a bit lost...
set_process_cpu_timer()doescpu_timer_sample_group(), butposix_cpu_timer_set()doescpu_clock_sample_group()in !CPUCLOCK_PERTHREAD case, could you confirm this is correct?
Ah, you're saying I missed an start_process_timers() instance?
Ok, so how about this -- it does not yet address the posix_cpu_timers_exit_group() issue, but should clarify the rest a bit, agreed?
---
Subject: timer: cleanup the clock/timer separation
Rename thread_group_cputime() to thread_group_sum_time() to increase the visual difference to thread_group_cputimer().
Furthermore, to decrease the chance of a missed enable, always enable the timer when we sample it, we'll always disable it when we find that there are no active timers in the jiffy tick.
--- fs/binfmt_elf.c | 2 +- fs/binfmt_elf_fdpic.c | 2 +- fs/proc/array.c | 2 +- include/linux/sched.h | 4 ++-- kernel/exit.c | 4 ++-- kernel/posix-cpu-timers.c | 35 ++++------------------------------- kernel/sys.c | 4 ++-- 7 files changed, 13 insertions(+), 40 deletions(-)
fs/binfmt_elf.c +1 -1
@@ -1359,7 +1359,7 @@ static void fill_prstatus(struct elf_prstatus *prstatus,* This is the record for the group leader. It shows the * group-wide total, not its individual thread total. */ -thread_group_cputime (p, &cputime); +thread_group_sum_time (p, &cputime);cputime_to_timeval (cputime.utime, &prstatus->pr_utime);cputime_to_timeval (cputime.stime, &prstatus->pr_stime); }else {
fs/binfmt_elf_fdpic.c +1 -1
@@ -1389,7 +1389,7 @@ static void fill_prstatus(struct elf_prstatus *prstatus,* This is the record for the group leader. It shows the * group-wide total, not its individual thread total. */ -thread_group_cputime (p, &cputime); +thread_group_sum_time (p, &cputime);cputime_to_timeval (cputime.utime, &prstatus->pr_utime);cputime_to_timeval (cputime.stime, &prstatus->pr_stime); }else {
fs/proc/array.c +1 -1
@@ -411,7 +411,7 @@ static int do_task_stat(struct seq_file *m, struct pid_namespace *ns, min_flt += sig->min_flt; maj_flt += sig->maj_flt; -thread_group_cputime (task, &cputime); +thread_group_sum_time (task, &cputime); utime = cputime.utime; stime = cputime.stime; gtime =cputime_add (gtime, sig->gtime);
include/linux/sched.h +2 -2
@@ -2227,7 +2227,7 @@ static inline int spin_needbreak(spinlock_t *lock)/* * Thread group CPU time accounting. */ -void thread_group_cputime (struct task_struct *tsk,struct task_cputime *times); +void thread_group_sum_time (struct task_struct *tsk,struct task_cputime *times);static inline void thread_group_cputimer (struct task_struct *tsk,struct task_cputime *times)
@@ -2235,7 +2235,7 @@ void thread_group_cputimer(struct task_struct *tsk, struct task_cputime *times)struct thread_group_cputimer *cputimer = &tsk->signal->cputimer;unsigned long flags; -WARN_ON (!cputimer->running); + cputimer->running =1 ;spin_lock_irqsave (&cputimer->lock, flags); *times = cputimer->cputime;
kernel/exit.c +2 -2
@@ -1316,11 +1316,11 @@ static int wait_task_zombie(struct task_struct *p, int options,* as other threads in the parent group can be right * here reaping other children at the same time. * -* We use thread_group_cputime() to get times for the thread +* We use thread_group_sum_time() to get times for the thread * group, which consolidates times for all threads in the * group including the group leader. */ -thread_group_cputime (p, &cputime); +thread_group_sum_time (p, &cputime);spin_lock_irq (&p->parent->sighand->siglock); psig = p->parent->signal; sig = p->signal;
kernel/posix-cpu-timers.c +4 -31
@@ -230,7 +230,7 @@ static int cpu_clock_sample(const clockid_t which_clock, struct task_struct *p,return 0 ; } -void thread_group_cputime (struct task_struct *tsk,struct task_cputime *times) +void thread_group_sum_time (struct task_struct *tsk,struct task_cputime *times) {struct sighand_struct *sighand;struct signal_struct *sig;
@@ -271,7 +271,7 @@ static int cpu_clock_sample_group(const clockid_t which_clock, {struct task_cputime cputime; -thread_group_cputime (p, &cputime); +thread_group_sum_time (p, &cputime);switch (CPUCLOCK_WHICH (which_clock)) {default :return -EINVAL ;
@@ -488,7 +488,7 @@ void posix_cpu_timers_exit_group(struct task_struct *tsk) {struct task_cputime cputime; -thread_group_cputime (tsk, &cputime); +thread_group_sum_time (tsk, &cputime);cleanup_timers (tsk->signal->cpu_timers, cputime.utime, cputime.stime, cputime.sum_exec_runtime); }
@@ -507,29 +507,6 @@ static void clear_dead_task(struct k_itimer *timer, union cpu_time_count now) }/* -* Enable the process wide cpu timer accounting. -* -* serialized using ->sighand->siglock -*/ -static void start_process_timers (struct task_struct *tsk) -{ - tsk->signal->cputimer.running =1 ; -barrier (); -} - -/* -* Release the process wide timer accounting -- timer stops ticking when -* nobody cares about it. -* -* serialized using ->sighand->siglock -*/ -static void stop_process_timers (struct task_struct *tsk) -{ - tsk->signal->cputimer.running =0 ; -barrier (); -} - -/* * Insert the timer on the appropriate list before any timers that * expire later. This must be called with the tasklist_lock held * for reading, and interrupts disabled.
@@ -549,9 +526,6 @@ static void arm_timer(struct k_itimer *timer, union cpu_time_count now)BUG_ON (!irqs_disabled ());spin_lock (&p->sighand->siglock); -if (!CPUCLOCK_PERTHREAD (timer->it_clock)) -start_process_timers (p); - listpos = head;if (CPUCLOCK_WHICH (timer->it_clock) ==CPUCLOCK_SCHED ) {list_for_each_entry (next, head, entry) {
@@ -1045,7 +1019,7 @@ static void check_process_timers(struct task_struct *tsk, list_empty(&timers[CPUCLOCK_VIRT]) && cputime_eq(sig->it_virt_expires, cputime_zero) && list_empty(&timers[CPUCLOCK_SCHED])) { -stop_process_timers (tsk); + tsk->signal->cputimer.running =0 ;return ; }
@@ -1427,7 +1401,6 @@ void set_process_cpu_timer(struct task_struct *tsk, unsigned int clock_idx,struct list_head *head;BUG_ON (clock_idx ==CPUCLOCK_SCHED ); -start_process_timers (tsk);cpu_timer_sample_group (clock_idx, tsk, &now);if (oldval) {
kernel/sys.c +2 -2
@@ -910,7 +910,7 @@ void do_sys_times(struct tms *tms)struct task_cputime cputime;cputime_t cutime, cstime; -thread_group_cputime (current, &cputime); +thread_group_sum_time (current, &cputime);spin_lock_irq (¤t->sighand->siglock); cutime = current->signal->cutime; cstime = current->signal->cstime;
@@ -1657,7 +1657,7 @@ static void k_getrusage(struct task_struct *p, int who, struct rusage *r)break ;case RUSAGE_SELF : -thread_group_cputime (p, &cputime); +thread_group_sum_time (p, &cputime); utime =cputime_add (utime, cputime.utime); stime =cputime_add (stime, cputime.stime); r->ru_nvcsw += p->signal->nvcsw;
Ingo Molnar
AppliedQuoted 1 line · Peter Zijlstra
This should hopefully address all the itimer borkage.
Applied to tip:timers/urgent, thanks Peter!
Yanmin: could you check hacbench_pth with latest tip/master, do these fixes resolve that 3% regression you reported?
Ingo
Quoted 8 lines · Ingo Molnar
Quoted 1 line · Peter Zijlstra
This should hopefully address all the itimer borkage.
Applied to tip:timers/urgent, thanks Peter!
Yanmin: could you check
hacbench_pthwith latest tip/master, do these fixes resolve that 3% regression you reported?
Lin Ming tested it and hackbench_pth/volanoMark regression all disappear. But oltp has a regression. We think oltp new regression isn't related to the patch. Ming is investigating it.
Yanmin
Quoted 5 lines · Zhang, Yanmin
Quoted 4 lines · Ingo Molnar
Quoted 1 line · Peter Zijlstra
This should hopefully address all the itimer borkage.
Quoted 8 lines
Quoted 4 lines
Applied to tip:timers/urgent, thanks Peter!
Yanmin: could you check
hacbench_pthwith latest tip/master, do these fixes resolve that 3% regression you reported?Lin Ming tested it and hackbench_pth/volanoMark regression all disappear. But oltp has a regression. We think oltp new regression isn't related to the patch. Ming is investigating it.
Potential suspects for oltp regression would be:
3d39870: sched_rt: don't use first_cpu on cpumask created with cpumask_and
a571bbe: sched: fix buddie group latency
a9f3e2b: sched: clear buddies more aggressively
1596e29: sched: symmetric sync vs avg_overlap
d942fb6: sched: fix sync wakeups
Ingo
Quoted 16 lines · Ingo Molnar
Quoted 5 lines · Zhang, Yanmin
Quoted 4 lines · Ingo Molnar
Quoted 1 line · Peter Zijlstra
This should hopefully address all the itimer borkage.
Quoted 8 lines
Quoted 4 lines
Applied to tip:timers/urgent, thanks Peter!
Yanmin: could you check
hacbench_pthwith latest tip/master, do these fixes resolve that 3% regression you reported?Lin Ming tested it and hackbench_pth/volanoMark regression all disappear. But oltp has a regression. We think oltp new regression isn't related to the patch. Ming is investigating it.
Quoted 7 lines
Potential suspects for oltp regression would be:
3d39870:
sched_rt: don't usefirst_cpuon cpumask created withcpumask_and
a571bbe: sched: fix buddie group latency
a9f3e2b: sched: clear buddies more aggressively
1596e29: sched: symmetric sync vsavg_overlap
d942fb6: sched: fix sync wakeups
I tested the latest tip-master branch.
After reverting "d942fb6: sched: fix sync wakeups", the oltp regression on the 8cores Stockley machine is mostly fixed.
On another 4*4 cores Tigerton machine, oltp has more than 10% regression with 2.6.29-rc4 compared with 2.6.29-rc3.
Lin Ming
Quoted 2 lines
Ingo
Quoted 25 lines · Lin Ming
Quoted 16 lines · Ingo Molnar
Quoted 5 lines · Zhang, Yanmin
Quoted 4 lines · Ingo Molnar
Quoted 1 line · Peter Zijlstra
This should hopefully address all the itimer borkage.
Quoted 8 lines
Quoted 4 lines
Applied to tip:timers/urgent, thanks Peter!
Yanmin: could you check
hacbench_pthwith latest tip/master, do these fixes resolve that 3% regression you reported?Lin Ming tested it and hackbench_pth/volanoMark regression all disappear. But oltp has a regression. We think oltp new regression isn't related to the patch. Ming is investigating it.
Quoted 7 lines
Potential suspects for oltp regression would be:
3d39870:
sched_rt: don't usefirst_cpuon cpumask created withcpumask_and
a571bbe: sched: fix buddie group latency
a9f3e2b: sched: clear buddies more aggressively
1596e29: sched: symmetric sync vsavg_overlap
d942fb6: sched: fix sync wakeups
Quoted 6 lines
I tested the latest tip-master branch.
After reverting "d942fb6: sched: fix sync wakeups", the oltp regression on the 8cores Stockley machine is mostly fixed.On another 4*4 cores Tigerton machine, oltp has more than 10% regression with 2.6.29-rc4 compared with 2.6.29-rc3.
ok, that commit needs fixed or reverted. Peter, Mike?
Ingo
Quoted 27 lines · Ingo Molnar
Quoted 25 lines · Lin Ming
Quoted 23 lines · Ingo Molnar
Quoted 5 lines · Zhang, Yanmin
Quoted 4 lines · Ingo Molnar
Quoted 1 line · Peter Zijlstra
This should hopefully address all the itimer borkage.
Quoted 8 lines
Quoted 4 lines
Applied to tip:timers/urgent, thanks Peter!
Yanmin: could you check
hacbench_pthwith latest tip/master, do these fixes resolve that 3% regression you reported?Lin Ming tested it and hackbench_pth/volanoMark regression all disappear. But oltp has a regression. We think oltp new regression isn't related to the patch. Ming is investigating it.
Potential suspects for oltp regression would be:
3d39870:
sched_rt: don't usefirst_cpuon cpumask created withcpumask_and
a571bbe: sched: fix buddie group latency
a9f3e2b: sched: clear buddies more aggressively
1596e29: sched: symmetric sync vsavg_overlap
d942fb6: sched: fix sync wakeups
Quoted 8 lines
Quoted 6 lines
I tested the latest tip-master branch.
After reverting "d942fb6: sched: fix sync wakeups", the oltp regression on the 8cores Stockley machine is mostly fixed.On another 4*4 cores Tigerton machine, oltp has more than 10% regression with 2.6.29-rc4 compared with 2.6.29-rc3.
ok, that commit needs fixed or reverted. Peter, Mike?
I see some ~problems.
Looking at the tasks sitting in my ~idle box right now:
tasks 284, avg_overlap = 0.000000 196
starts make -j30
tasks 401, avg_overlap = 0.000000 285
0.0 (should) means zero wakeups since birth, it does not mean this task is showing synchronous behavior until it's non-zero. New tasks start with zero, so until they grow an avg_overlap, when they wake, at least half of the decision making data is bogus/non-existent. With make -j30, I added 117 tasks, 89 are unknown, 28 known. This parallel load _tries_ to go affine. On an nfs mount where runners are also frequent (and truly synchronous) wakers, it tries really hard.
IOW, I think the affinity logic may become too strong when strengthened by flipping the hint. I originally inverted that test to filter out the case where we _have_ behavioral data indicating that the tasks in question were definitely not synchronous despite the sync wakeup hint.
Another ~problem is that a task with low avg_overlap can change behavior to cpu hog, and retain it's stale avg_overlap up to forever.
Maybe we shouldn't use avg_overlap until it's been established.
But..
Flip-side: I have a strong feeling that that _not_ using it would have negative impact. Freshly forked task generates red-hot data for a yet to be awakened partner...
Sigh. Damned if ya do, damned if ya don't.
-Mike
Quoted 27 lines · Ingo Molnar
Quoted 25 lines · Lin Ming
Quoted 23 lines · Ingo Molnar
Quoted 5 lines · Zhang, Yanmin
Quoted 4 lines · Ingo Molnar
Quoted 1 line · Peter Zijlstra
This should hopefully address all the itimer borkage.
Quoted 8 lines
Quoted 4 lines
Applied to tip:timers/urgent, thanks Peter!
Yanmin: could you check
hacbench_pthwith latest tip/master, do these fixes resolve that 3% regression you reported?Lin Ming tested it and hackbench_pth/volanoMark regression all disappear. But oltp has a regression. We think oltp new regression isn't related to the patch. Ming is investigating it.
Potential suspects for oltp regression would be:
3d39870:
sched_rt: don't usefirst_cpuon cpumask created withcpumask_and
a571bbe: sched: fix buddie group latency
a9f3e2b: sched: clear buddies more aggressively
1596e29: sched: symmetric sync vsavg_overlap
d942fb6: sched: fix sync wakeups
Quoted 8 lines
Quoted 6 lines
I tested the latest tip-master branch.
After reverting "d942fb6: sched: fix sync wakeups", the oltp regression on the 8cores Stockley machine is mostly fixed.On another 4*4 cores Tigerton machine, oltp has more than 10% regression with 2.6.29-rc4 compared with 2.6.29-rc3.
ok, that commit needs fixed or reverted. Peter, Mike?
Yanmin, is that tigerton regression also due to the sync changes?
That is, if you revert both d942fb6 and 1596e29, does it get back to -rc3 state, or is the tigerton regression due to something else?
This isn't quite clear to me.
Ingo, if that is the case, I'm fine with reverting those changes for now, and have another look at them later on -- preferably when someone ships me a 4*4 machine so I can validate :-)
Quoted 32 lines · Peter Zijlstra
Quoted 31 lines · Ingo Molnar
Quoted 29 lines · Lin Ming
Quoted 16 lines · Ingo Molnar
Quoted 5 lines · Zhang, Yanmin
Quoted 4 lines · Ingo Molnar
Quoted 1 line · Peter Zijlstra
This should hopefully address all the itimer borkage.
Quoted 8 lines
Quoted 4 lines
Applied to tip:timers/urgent, thanks Peter!
Yanmin: could you check
hacbench_pthwith latest tip/master, do these fixes resolve that 3% regression you reported?Lin Ming tested it and hackbench_pth/volanoMark regression all disappear. But oltp has a regression. We think oltp new regression isn't related to the patch. Ming is investigating it.
Quoted 7 lines
Potential suspects for oltp regression would be:
3d39870:
sched_rt: don't usefirst_cpuon cpumask created withcpumask_and
a571bbe: sched: fix buddie group latency
a9f3e2b: sched: clear buddies more aggressively
1596e29: sched: symmetric sync vsavg_overlap
d942fb6: sched: fix sync wakeupsI tested the latest tip-master branch.
After reverting "d942fb6: sched: fix sync wakeups", the oltp regression on the 8cores Stockley machine is mostly fixed.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
On another 4*4 cores Tigerton machine, oltp has more than 10% regression with 2.6.29-rc4 compared with 2.6.29-rc3.
ok, that commit needs fixed or reverted. Peter, Mike?
Yanmin, is that tigerton regression also due to the sync changes?
Yes.
Quoted 3 lines
That is, if you revert both d942fb6 and 1596e29, does it get back to -rc3 state,
Yes.
Quoted 6 lines
or is the tigerton regression due to something else?
This isn't quite clear to me.Ingo, if that is the case, I'm fine with reverting those changes for now, and have another look at them later on -- preferably when someone ships me a 4*4 machine so I can validate :-)
2*4 stoakley has the similiar regression. To find potential scalability issues, I run sysbench+mysql(oltp) with many thread numbers, such like 8,12,16,32,64,128, then get an average value.
yanmin
Quoted 49 lines · Zhang, Yanmin
Quoted 32 lines · Peter Zijlstra
Quoted 31 lines · Ingo Molnar
Quoted 29 lines · Lin Ming
Quoted 16 lines · Ingo Molnar
Quoted 5 lines · Zhang, Yanmin
Quoted 4 lines · Ingo Molnar
Quoted 1 line · Peter Zijlstra
This should hopefully address all the itimer borkage.
Quoted 8 lines
Quoted 4 lines
Applied to tip:timers/urgent, thanks Peter!
Yanmin: could you check
hacbench_pthwith latest tip/master, do these fixes resolve that 3% regression you reported?Lin Ming tested it and hackbench_pth/volanoMark regression all disappear. But oltp has a regression. We think oltp new regression isn't related to the patch. Ming is investigating it.
Quoted 7 lines
Potential suspects for oltp regression would be:
3d39870:
sched_rt: don't usefirst_cpuon cpumask created withcpumask_and
a571bbe: sched: fix buddie group latency
a9f3e2b: sched: clear buddies more aggressively
1596e29: sched: symmetric sync vsavg_overlap
d942fb6: sched: fix sync wakeupsI tested the latest tip-master branch.
After reverting "d942fb6: sched: fix sync wakeups", the oltp regression on the 8cores Stockley machine is mostly fixed.Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
On another 4*4 cores Tigerton machine, oltp has more than 10% regression with 2.6.29-rc4 compared with 2.6.29-rc3.
ok, that commit needs fixed or reverted. Peter, Mike?
Yanmin, is that tigerton regression also due to the sync changes?
Yes.
Quoted 3 lines
That is, if you revert both d942fb6 and 1596e29, does it get back to -rc3 state,
Yes.
Quoted 3 lines
or is the tigerton regression due to something else?
This isn't quite clear to me.
Quoted 6 lines
Quoted 3 lines
Ingo, if that is the case, I'm fine with reverting those changes for now, and have another look at them later on -- preferably when someone ships me a 4*4 machine so I can validate :-)
2*4 stoakley has the similiar regression. To find potential scalability issues, I run sysbench+mysql(oltp) with many thread numbers, such like 8,12,16,32,64,128, then get an average value.
FYI, in Linus's latest tree (v2.6.29-rc4-175-gb578f3f or later), all the scheduler related performance regressions should be addressed.
Could you please double-check that there's no performance regression remaining?
Thanks,
Ingo
Quoted 6 lines
FYI, in Linus's latest tree (v2.6.29-rc4-175-gb578f3f or later), all the scheduler related performance regressions should be addressed.
Could you please double-check that there's no performance regression remaining?
I have tested the latest git tree(071a0bc2).
hackbench, volanoMark, oltp, specjbb2005 regressions are all fixed.
Thanks,
Lin Ming
Quoted 9 lines · Lin Ming
Quoted 6 lines
FYI, in Linus's latest tree (v2.6.29-rc4-175-gb578f3f or later), all the scheduler related performance regressions should be addressed.
Could you please double-check that there's no performance regression remaining?
I have tested the latest git tree(071a0bc2).
hackbench, volanoMark, oltp, specjbb2005 regressions are all fixed.
Great - thanks for testing it!
Ingo
Quoted 39 lines · Peter Zijlstra
Quoted 27 lines · Ingo Molnar
Quoted 25 lines · Lin Ming
Quoted 23 lines · Ingo Molnar
Quoted 5 lines · Zhang, Yanmin
Quoted 4 lines · Ingo Molnar
Quoted 1 line · Peter Zijlstra
This should hopefully address all the itimer borkage.
Quoted 8 lines
Quoted 4 lines
Applied to tip:timers/urgent, thanks Peter!
Yanmin: could you check
hacbench_pthwith latest tip/master, do these fixes resolve that 3% regression you reported?Lin Ming tested it and hackbench_pth/volanoMark regression all disappear. But oltp has a regression. We think oltp new regression isn't related to the patch. Ming is investigating it.
Potential suspects for oltp regression would be:
3d39870:
sched_rt: don't usefirst_cpuon cpumask created withcpumask_and
a571bbe: sched: fix buddie group latency
a9f3e2b: sched: clear buddies more aggressively
1596e29: sched: symmetric sync vsavg_overlap
d942fb6: sched: fix sync wakeupsQuoted 8 lines
Quoted 6 lines
I tested the latest tip-master branch.
After reverting "d942fb6: sched: fix sync wakeups", the oltp regression on the 8cores Stockley machine is mostly fixed.On another 4*4 cores Tigerton machine, oltp has more than 10% regression with 2.6.29-rc4 compared with 2.6.29-rc3.
ok, that commit needs fixed or reverted. Peter, Mike?
Yanmin, is that tigerton regression also due to the sync changes?
Quoted 8 lines
That is, if you revert both d942fb6 and 1596e29, does it get back to -rc3 state, or is the tigerton regression due to something else?
This isn't quite clear to me.
Ingo, if that is the case, I'm fine with reverting those changes for now, and have another look at them later on -- preferably when someone ships me a 4*4 machine so I can validate :-)
Could you please send a changelogged minimal reverter patch with Reported-by and Bisected-by tags in place, etc? It does not have to be a full revert, if you think we can do with less that's fine too.
Thanks,
Ingo
Quoted 3 lines · Ingo Molnar
Could you please send a changelogged minimal reverter patch with Reported-by and Bisected-by tags in place, etc? It does not have to be a full revert, if you think we can do with less that's fine too.
I don't think its worth playing games with at this point, lets just do a full revert.
---
Subject: sched: revert recent sync wakeup changes
From: Peter Zijlstra <[email protected]>
Date: Wed Feb 11 14:22:24 CET 2009
Intel reported a hefty 10% regression on a 16-way machine with these patches:
1596e29: sched: symmetric sync vs avg_overlap
d942fb6: sched: fix sync wakeups
Revert them.
--- kernel/sched.c | 10 ---------- kernel/sched_fair.c | 11 +++++++++-- 2 files changed, 9 insertions(+), 12 deletions(-)
kernel/sched.c +0 -10
@@ -2362,16 +2362,6 @@ static int try_to_wake_up(struct task_stif (!sched_feat (SYNC_WAKEUPS )) sync =0 ; -if (!sync) { -if (current->se.avg_overlap < sysctl_sched_migration_cost && - p->se.avg_overlap < sysctl_sched_migration_cost) - sync =1 ; - }else { -if (current->se.avg_overlap >= sysctl_sched_migration_cost || - p->se.avg_overlap >= sysctl_sched_migration_cost) - sync =0 ; - } -#ifdef CONFIG_SMP if (sched_feat (LB_WAKEUP_UPDATE )) {struct sched_domain *sd;
kernel/sched_fair.c +9 -2
@@ -1191,15 +1191,20 @@ wake_affine(struct sched_domain *this_sd int idx, unsigned long load, unsigned long this_load, unsigned int imbalance) { + struct task_struct *curr = this_rq->curr; +struct task_group *tg;unsigned long tl = this_load;unsigned long tl_per_task; -struct task_group *tg;unsigned long weight;int balanced;if (!(this_sd->flags &SD_WAKE_AFFINE ) || !sched_feat (AFFINE_WAKEUPS ))return 0 ; +if (sync && (curr->se.avg_overlap > sysctl_sched_migration_cost || + p->se.avg_overlap > sysctl_sched_migration_cost)) + sync =0 ; +/* * If sync wakeup then subtract the (maximum possible) * effect of the currently running task from the load
@@ -1473,7 +1478,9 @@ static void check_preempt_wakeup(structif (!sched_feat (WAKEUP_PREEMPT ))return ; -if (sched_feat (WAKEUP_OVERLAP ) && sync) { +if (sched_feat (WAKEUP_OVERLAP ) && (sync || + (se->avg_overlap < sysctl_sched_migration_cost && + pse->avg_overlap < sysctl_sched_migration_cost))) {resched_task (curr);return ; }
Quoted 9 lines · Ingo Molnar
Quoted 1 line · Peter Zijlstra
This should hopefully address all the itimer borkage.
Applied to tip:timers/urgent, thanks Peter!
Yanmin: could you check
hacbench_pthwith latest tip/master, do these fixes resolve that 3% regression you reported?
hacbench_pth still has regression with 2.6.29-rc4.
Below 2 patches that fixed the regression are not merged into 2.6.29-rc4.
commit 32bd671d6cbeda60dc73be77fa2b9037d9a9bfa0
Author: Peter Zijlstra <[email protected]> Date: Thu Feb 5 12:24:15 2009 +0100
signal: re-add dead task accumulation stats.
commit 4cd4c1b40d40447fb5e7ba80746c6d7ba91d7a53
Author: Peter Zijlstra <[email protected]> Date: Thu Feb 5 12:24:16 2009 +0100
timers: split process wide cpu clocks/timers
Lin Ming
Quoted 18 lines · Lin Ming
Quoted 9 lines · Ingo Molnar
Quoted 1 line · Peter Zijlstra
This should hopefully address all the itimer borkage.
Applied to tip:timers/urgent, thanks Peter!
Yanmin: could you check
hacbench_pthwith latest tip/master, do these fixes resolve that 3% regression you reported?
hacbench_pthstill has regression with 2.6.29-rc4.
Below 2 patches that fixed the regression are not merged into 2.6.29-rc4.commit
32bd671d6cbeda60dc73be77fa2b9037d9a9bfa0Author: Peter Zijlstra <[email protected]> Date: Thu Feb 5 12:24:15 2009 +0100
Quoted 7 lines
signal: re-add dead task accumulation stats.
commit
4cd4c1b40d40447fb5e7ba80746c6d7ba91d7a53Author: Peter Zijlstra <[email protected]> Date: Thu Feb 5 12:24:16 2009 +0100timers: split process wide cpu clocks/timers
they are queued up but not fully baken yet. Maybe in -rc5.
Ingo