high power consumption in recent kernels
Subsystems (2)
CC to lkml since maybe other guys also have interesting on this.
Norbert:
In the powertop source code, the "extra timer interrupt" come from /proc/timer_stats. If my understanding is correct, so many "extra timer interrupt" and the nohz_ratemlimit checking make your system can not go to deep C states. but don't know what the effect for "load balancing tick" come from "extra timer interrupt". So maybe reduce the "extra timer interrupt" is helpful on your system. Could you like to try this?
Regards!
Alex
Quoted 36 lines
-----Original Message-----
From: Peter Zijlstra [mailto:[email protected]]
Sent: Friday, September 10, 2010 4:25 PM
To: Norbert Preining
Cc: Chen, Tim C; Shi, Alex; [email protected]; [email protected]; Li, Shaohua; tglx
Subject: Re: high power consumption in recent kernelsQuoted 6 lines · Norbert Preining
Quoted 1 line · Tim C
Have you tried to run powertop to see the causes of the wakeups
Of course, in the very first email of the thread I posted the results, here again for your convenience:
Top causes for wakeups:Quoted 8 lines
34.2% (185.3) [kernel scheduler] Load balancing tick 23.9% (129.6) [extra timer interrupt] 10.8% ( 58.6) firefox-bin 9.2% ( 49.7) [iwlagn] <interrupt> 7.2% ( 39.1) [kernel core] hrtimer_start (tick_sched_timer) 3.9% ( 20.9) PS/2 keyboard/mouse/touchpad interruptThe problem is the "Load balancing tick" that is far far to high.
Right, which makes perfect sense, delaying going into nohz state simply costs power.
One thing we could do is hand over nohz control to the power governor (I'm sure Arjan will be thrilled), it could compare the cost of nohz enter/exit against the projected sleep time and make an informed decision.
Because from what I understand, the nohz enter/exit cycle is what is costing you performance on this workload, its simply very expensive to prod at timer hardware (which is of course totally intel's own fault for giving us crummy timers to begin with).
Quoted 1 line · Alex
In the powertop source code, the "extra timer interrupt" come from
/proc/timer_stats. If my understanding is correct, so many "extra timer interrupt" and thenohz_ratemlimitchecking make your system can not go to deep C states. but don't know what the effect for "load balancing tick" come from "extra timer interrupt". So maybe reduce the "extra timer interrupt" is helpful on your system. Could you like to try this?
Can you be more specific how to do that?
- revert the nohz patch (can you send me a patch for that)
- how to reduce the "extra timer interrupt"?
Thanks
Best wishes
Norbert
------------------------------------------------------------------------
Norbert Preining preining@{jaist.ac.jp, logic.at, debian.org}
JAIST, Japan TeX Live & Debian Developer
DSA: 0x09C5B094 fp: 14DF 2E6C 0307 BE6D AD76 A9C0 D2BF 4AA3 09C5 B094
------------------------------------------------------------------------
CAIRNPAT (n.)
A large piece of dried dung found in mountainous terrain above the cowline which leads the experienced tracker to believe that hikers have recently passed.
--- Douglas Adams, The Meaning of Liff
Quoted 6 lines · Norbert Preining
Quoted 1 line · Alex
In the powertop source code, the "extra timer interrupt" come from
/proc/timer_stats. If my understanding is correct, so many "extra timer interrupt" and thenohz_ratemlimitchecking make your system can not go to deep C states. but don't know what the effect for "load balancing tick" come from "extra timer interrupt". So maybe reduce the "extra timer interrupt" is helpful on your system. Could you like to try this?Can you be more specific how to do that?
- revert the nohz patch (can you send me a patch for that)
include/linux/sched.h +6 -0
@@ -274,8 +274,14 @@ extern cpumask_var_t nohz_cpu_mask;#if defined(CONFIG_SMP ) && defined(CONFIG_NO_HZ )extern void select_nohz_load_balancer (int stop_tick);extern int get_nohz_timer_target (void ); +extern int nohz_ratelimit (int cpu);#else static inline void select_nohz_load_balancer (int stop_tick) { } +static inline int nohz_ratelimit (int cpu) +{ +return 0 ; +} +#endif /*
kernel/sched.c +11 -0
@@ -1182,6 +1182,17 @@ static void resched_task(struct task_struct *p)smp_send_reschedule (cpu); } +int nohz_ratelimit (int cpu) +{ +struct rq *rq =cpu_rq (cpu); +u64 diff = rq->clock - rq->nohz_stamp; + + rq->nohz_stamp = rq->clock; + +return diff < (NSEC_PER_SEC /HZ ) >>1 ; +} + +static void resched_cpu (int cpu) {struct rq *rq =cpu_rq (cpu);
kernel/time/tick-sched.c +1 -1
@@ -325,7 +325,7 @@ void tick_nohz_stop_sched_tick(int inidle) }while (read_seqretry (&xtime_lock, seq));if (rcu_needs_cpu (cpu) ||printk_needs_cpu (cpu) || -arch_needs_cpu (cpu)) { +arch_needs_cpu (cpu)||nohz_ratelimit (cpu)) { next_jiffies = last_jiffies +1 ; delta_jiffies =1 ; }else {
Quoted 1 line
- how to reduce the "extra timer interrupt"?
The timer was listed in /proc/timer_stats, you can post its contents, and let's see what is abnormal here.
Quoted 16 lines
Thanks
Best wishes
Norbert
------------------------------------------------------------------------
Norbert Preining preining@{jaist.ac.jp, logic.at, debian.org}
JAIST, Japan TeX Live & Debian Developer
DSA:0x09C5B094fp: 14DF 2E6C 0307 BE6D AD76 A9C0 D2BF 4AA3 09C5 B094
------------------------------------------------------------------------
CAIRNPAT (n.)
A large piece of dried dung found in mountainous terrain above the cowline which leads the experienced tracker to believe that hikers have recently passed.
--- Douglas Adams, The Meaning of Liff
Quoted 5 lines
Quoted 2 lines
- how to reduce the "extra timer interrupt"?
The timer was listed in
/proc/timer_stats, you can post its contents, and let's see what is abnormal here.
Norbert, did you try the patch?
We are still thinking the too many "extra timer interrupts" on your system, maybe the following message can give some help. So, could you like to check if your system is using pit timer?
#cat /proc/timer_list
Or some APP/system services in your system is using lots of timer?
===
http://www.linuxpowertop.org/faq.php
Even with a tickless kernel, there are some timer ticks still happening:
1.) For userspace events (also shown in the lower "top 10 events" list). 2.) The kernel still has timer ticks when userspace is executing code, to sample and get data for the "top" program.
3.) Hardware timers have a maximum time they can be set for, for the PIT timer (the one used if you don't have HPET active). This is around 27 milliseconds. If you would have longer sleep times than the maximum, the kernel has no choice but to set the timer to the maximum repeatedly until the actual event is approaching.
The "extra timer interrupts" data is a combination fo 1 and 3.
Hi Alex, hi all,
sorry for the long silence, I was on business travel and mountaineering for some weeks.
Quoted 1 line · Alex,Shi
Quoted 1 line
- revert the nohz patch (can you send me a patch for that)
I am now running 2.6.36-rc5 with this patch:
Quoted 46 lines
include/linux/sched.h +6
-0@@ -274,8 +274,14 @@ extern cpumask_var_t nohz_cpu_mask;#if defined(CONFIG_SMP ) && defined(CONFIG_NO_HZ )extern void select_nohz_load_balancer (int stop_tick);extern int get_nohz_timer_target (void ); +extern int nohz_ratelimit (int cpu);#else static inline void select_nohz_load_balancer (int stop_tick) { } +static inline int nohz_ratelimit (int cpu) +{ +return 0 ; +} +#endif /* kernel/sched.c +11
-0@@ -1182,6 +1182,17 @@ static void resched_task(struct task_struct *p)smp_send_reschedule (cpu); } +int nohz_ratelimit (int cpu) +{ +struct rq *rq =cpu_rq (cpu); +u64 diff = rq->clock - rq->nohz_stamp; + + rq->nohz_stamp = rq->clock; + +return diff < (NSEC_PER_SEC /HZ ) >>1 ; +} + +static void resched_cpu (int cpu) {struct rq *rq =cpu_rq (cpu);kernel/time/tick-sched.c +0
-0
kernel/time/tick-sched.c void tick_nohz_stop_sched_tick(int inidle) · line 331
Quoted 8 lines
kernel/time/tick-sched.c +1
-1@@ -325,7 +325,7 @@ void tick_nohz_stop_sched_tick(int inidle) }while (read_seqretry (&xtime_lock, seq));if (rcu_needs_cpu (cpu) ||printk_needs_cpu (cpu) || -arch_needs_cpu (cpu)) { +arch_needs_cpu (cpu)||nohz_ratelimit (cpu)) { next_jiffies = last_jiffies +1 ; delta_jiffies =1 ; }else {
and it seems that still works nicely. I am now running on battery power, connected to the internet over bluetooth to my mobile, and get:
Wakeups-from-idle per second : 452.2 interval: 10.0s
Power usage (ACPI estimate): 12.4W (3.6 hours) (long term: 8.0W,/5.6h)
Top causes for wakeups:
16.1% ( 79.5) [yenta, ehci_hcd:usb2, uhci_hcd:usb6, uhci_hcd:usb7, uhci_hcd: 14.3% ( 70.7) [kernel scheduler] Load balancing tick 14.1% ( 69.8) [extra timer interrupt] 13.1% ( 64.9) kworker/0:0 10.7% ( 53.0) USB device 8-2 : BCM2046 Bluetooth Device (Broadcom Corp) 8.4% ( 41.8) [kernel core] hrtimer_start (tick_sched_timer) 3.8% ( 19.0) gnome-terminal 2.9% ( 14.6) icedove-bin 2.2% ( 10.7) PS/2 keyboard/mouse/touchpad interrupt
Which shows that the "Load balancing tick" and "extra timer interrupt" are down to halfway normal levels. I guess when I shutdown the usb/bluetooth connection both of them will drop even further.
Quoted 3 lines
Quoted 1 line
- how to reduce the "extra timer interrupt"?
The timer was listed in
/proc/timer_stats, you can post its contents, and let's see what is abnormal here.
Here is the output of /proc/timer_stats:
Timer Stats Version: v0.2
Sample period: 7.828 s
66, 4378 firefox-bin hrtimer_start_range_ns (hrtimer_wakeup)
15, 12527 aptitude hrtimer_start_range_ns (hrtimer_wakeup)
8, 3629 cpufreq-applet hrtimer_start_range_ns (hrtimer_wakeup)
612, 0 swapper hrtimer_start_range_ns (tick_sched_timer)
483D, 6082 kworker/0:0 do_dbs_timer (delayed_work_timer_fn)
17, 12527 aptitude queue_delayed_work (delayed_work_timer_fn)
376, 0 kworker/0:0 hrtimer_start_range_ns (tick_sched_timer)
237D, 6810 kworker/1:1 do_dbs_timer (delayed_work_timer_fn)
51, 7843 sakura hrtimer_start_range_ns (hrtimer_wakeup)
8, 2665 Xorg queue_delayed_work (delayed_work_timer_fn)
387, 0 swapper hrtimer_start (tick_sched_timer)
39, 3510 syndaemon hrtimer_start_range_ns (hrtimer_wakeup)
8, 2705 gpsd hrtimer_start_range_ns (hrtimer_wakeup)
32, 0 swapper __mod_timer (rh_timer_func)
8, 7572 icedove-bin hrtimer_start_range_ns (hrtimer_wakeup)
8, 12916 icedove-bin hrtimer_start_range_ns (hrtimer_wakeup)
8, 12949 icedove-bin hrtimer_start_range_ns (hrtimer_wakeup)
8, 2080 apache2 hrtimer_start_range_ns (hrtimer_wakeup)
8, 2795 ntpd hrtimer_start_range_ns (posix_timer_fn)
26, 3555 yarssr hrtimer_start_range_ns (hrtimer_wakeup)
91, 12790 sakura hrtimer_start_range_ns (hrtimer_wakeup)
15, 3583 multiload-apple hrtimer_start_range_ns (hrtimer_wakeup)
8, 4376 firefox-bin hrtimer_start_range_ns (hrtimer_wakeup)
8, 7579 icedove-bin hrtimer_start_range_ns (hrtimer_wakeup)
8D, 6810 kworker/1:1 queue_delayed_work (delayed_work_timer_fn)
4D, 6810 kworker/1:1 queue_delayed_work (delayed_work_timer_fn)
8D, 6082 kworker/0:0 queue_delayed_work (delayed_work_timer_fn)
4D, 6082 kworker/0:0 queue_delayed_work (delayed_work_timer_fn)
8, 12895 icedove-bin hrtimer_start_range_ns (hrtimer_wakeup)
4, 3516 udisks-daemon hrtimer_start_range_ns (hrtimer_wakeup)
4, 2642 hald-addon-stor hrtimer_start_range_ns (hrtimer_wakeup)
8, 7540 icedove-bin hrtimer_start_range_ns (hrtimer_wakeup)
8, 7777 icedove-bin hrtimer_start_range_ns (hrtimer_wakeup)
9, 0 kworker/0:0 hrtimer_start (tick_sched_timer)
6, 2665 Xorg __mod_timer (i915_hangcheck_elapsed)
8, 7550 icedove-bin hrtimer_start_range_ns (hrtimer_wakeup)
8, 3531 gvfs-afc-volume hrtimer_start_range_ns (hrtimer_wakeup)
7, 7551 icedove-bin hrtimer_start_range_ns (hrtimer_wakeup)
7, 7549 icedove-bin hrtimer_start_range_ns (hrtimer_wakeup)
7, 7578 icedove-bin hrtimer_start_range_ns (hrtimer_wakeup)
1, 0 swapper __mod_timer (uhci_fsbr_timeout)
7, 7527 icedove-bin hrtimer_start_range_ns (hrtimer_wakeup)
7, 12890 icedove-bin hrtimer_start_range_ns (hrtimer_wakeup)
7, 7536 icedove-bin hrtimer_start_range_ns (hrtimer_wakeup)
18, 12790 sakura hrtimer_start_range_ns (hrtimer_wakeup)
4, 3582 sensors-applet hrtimer_start_range_ns (hrtimer_wakeup)
4, 0 swapper __mod_timer (laptop_mode_timer_fn)
4, 0 swapper __mod_timer (dev_watchdog)
11, 2868 tor __mod_timer (process_timeout)
5, 7528 icedove-bin hrtimer_start_range_ns (hrtimer_wakeup)
4, 4807 dropbox hrtimer_start_range_ns (hrtimer_wakeup)
1, 12537 http __mod_timer (tcp_write_timer)
5, 2825 nmbd hrtimer_start_range_ns (hrtimer_wakeup)
1, 3439 gconfd-2 hrtimer_start_range_ns (hrtimer_wakeup)
4, 7403 kbnepd bnep0 __mod_timer (tcp_delack_timer)
1, 3422 ssh-agent hrtimer_start_range_ns (hrtimer_wakeup)
1, 2195 hald hrtimer_start_range_ns (hrtimer_wakeup)
1, 2195 hald __mod_timer (process_timeout)
1, 7 watchdog/0 hrtimer_start (watchdog_timer_fn)
1, 3444 gnome-power-man hrtimer_start_range_ns (hrtimer_wakeup)
3, 816 flush-8:0 __mod_timer (blk_unplug_timeout)
1, 1 init hrtimer_start_range_ns (hrtimer_wakeup)
1, 1 swapper enqueue_task_rt (sched_rt_period_timer)
16, 12790 sakura queue_delayed_work (delayed_work_timer_fn)
14, 12791 bash queue_delayed_work (delayed_work_timer_fn)
5, 2665 Xorg hrtimer_start_range_ns (hrtimer_wakeup)
2, 3582 sensors-applet __mod_timer (process_timeout)
1, 3569 gnome-volume-ma hrtimer_start_range_ns (hrtimer_wakeup)
1, 2247 NetworkManager hrtimer_start_range_ns (hrtimer_wakeup)
1D, 6082 kworker/0:0 queue_delayed_work (delayed_work_timer_fn)
1, 7403 kbnepd bnep0 __mod_timer (tcp_delack_timer)
1, 2642 hald-addon-stor __mod_timer (blk_rq_timed_out_timer)
1, 4804 dropbox __mod_timer (tcp_write_timer)
1, 3511 gnome-panel hrtimer_start_range_ns (hrtimer_wakeup)2772 total events, 354.113 events/sec
Hope you can do something with it!
Let me know if I should do more testing.
Best wishes
Norbert
------------------------------------------------------------------------
Norbert Preining preining@{jaist.ac.jp, logic.at, debian.org}
JAIST, Japan TeX Live & Debian Developer
DSA: 0x09C5B094 fp: 14DF 2E6C 0307 BE6D AD76 A9C0 D2BF 4AA3 09C5 B094
------------------------------------------------------------------------
BERKHAMSTED
The massive three-course midmorning blow-out enjoyed by a dieter who has already done his or her slimming duty by having a teaspoonful of cottage cheese for breakfast.
--- Douglas Adams, The Meaning of Liff
Quoted 17 lines
and it seems that still works nicely. I am now running on battery power, connected to the internet over bluetooth to my mobile, and get:
Wakeups-from-idle per second : 452.2 interval: 10.0s
Power usage (ACPI estimate): 12.4W (3.6 hours) (long term: 8.0W,/5.6h)Top causes for wakeups:
16.1% ( 79.5) [yenta, ehci_hcd:usb2, uhci_hcd:usb6, uhci_hcd:usb7, uhci_hcd: 14.3% ( 70.7) [kernel scheduler] Load balancing tick 14.1% ( 69.8) [extra timer interrupt] 13.1% ( 64.9) kworker/0:0 10.7% ( 53.0) USB device 8-2 : BCM2046 Bluetooth Device (Broadcom Corp) 8.4% ( 41.8) [kernel core] hrtimer_start (tick_sched_timer) 3.8% ( 19.0) gnome-terminal 2.9% ( 14.6) icedove-bin 2.2% ( 10.7) PS/2 keyboard/mouse/touchpad interrupt
Quoted 8 lines
Which shows that the "Load balancing tick" and "extra timer interrupt" are down to halfway normal levels. I guess when I shutdown the usb/bluetooth connection both of them will drop even further.
Hope you can do something with it!
Let me know if I should do more testing.
Seems in .36 kernel the extra timer reduced. Guess the following patch worked. http://lkml.org/lkml/2010/6/9/109
Tim,Yakui and I talked with the issue. We thought the nohz_ratelimit() only hart the power when lots of interrupts with light load in system. Since the tick_nohz_stop_sched_tick() only be checked in cpu_idle() and in irq_exit(), we remove the nohz checking in irq_exit(). The TCP/UDP RR loop back testing of netperf also benefit from this. But it have no effect on real network testing.
Could you like to try the following patch on your system and see how many power it consumption now?
include/linux/sched.h +5 -0
@@ -274,8 +274,13 @@ extern cpumask_var_t nohz_cpu_mask;#if defined(CONFIG_SMP ) && defined(CONFIG_NO_HZ )extern void select_nohz_load_balancer (int stop_tick);extern int get_nohz_timer_target (void ); +extern int nohz_ratelimit (int cpu);#else static inline void select_nohz_load_balancer (int stop_tick) { } +static inline int nohz_ratelimit (int cpu) +{ +return 0 ; +}#endif /*
kernel/sched.c +10 -0
@@ -1182,6 +1182,16 @@ static void resched_task(struct task_struct *p)smp_send_reschedule (cpu); } +int nohz_ratelimit (int cpu) +{ +struct rq *rq =cpu_rq (cpu); +u64 diff = rq->clock - rq->nohz_stamp; + + rq->nohz_stamp = rq->clock; + +return diff < (NSEC_PER_SEC /HZ ) >>1 ; +} +static void resched_cpu (int cpu) {struct rq *rq =cpu_rq (cpu);
kernel/time/tick-sched.c +1 -1
@@ -325,7 +325,7 @@ void tick_nohz_stop_sched_tick(int inidle) }while (read_seqretry (&xtime_lock, seq));if (rcu_needs_cpu (cpu) ||printk_needs_cpu (cpu) || -arch_needs_cpu (cpu)) { +arch_needs_cpu (cpu)|| (inidle &&nohz_ratelimit (cpu))) { next_jiffies = last_jiffies +1 ; delta_jiffies =1 ; }else {
Hi Alex,
Quoted 2 lines
Could you like to try the following patch on your system and see how many power it consumption now?
But this patch is excately what you send me already and what I tried and I answered in the last email?
Do I miss something here?
All the best
Norbert
Norbert,
Can you try with the no-hz ratelimit now code REMOVED, and see if that will reduce your system power and cpu wakeups further. Your last testing has the no-hz ratelimit code and we will like to compare the effect with and without the code.
Sorry for the confusion.
Tim
Quoted 23 lines
-----Original Message-----
From: Norbert Preining [mailto:[email protected]]
Sent: Wednesday, September 29, 2010 7:39 AM
To: Shi, Alex
Cc: Peter Zijlstra; Chen, Tim C; [email protected]; [email protected]; Li, Shaohua; tglx; [email protected]; Zhao, Yakui
Subject: Re: high power consumption in recent kernelsHi Alex,
Quoted 2 lines
Could you like to try the following patch on your system and see how many power it consumption now?
But this patch is excately what you send me already and what I tried and I answered in the last email?
Do I miss something here?
All the best
Norbert
Quoted 5 lines · Norbert Preining
Hi Alex,
Quoted 2 lines
Could you like to try the following patch on your system and see how many power it consumption now?
Quoted 8 lines
But this patch is excately what you send me already and what I tried and I answered in the last email?
Do I miss something here?
All the best
Norbert
The following line is different, the new code remove the nohz_ratelimit from irq_exit(). That may resolve your system too many interrupts issue.
+ arch_needs_cpu(cpu)|| (inidle && nohz_ratelimit(cpu))) {
Quoted 1 line · Alex,Shi
+ arch_needs_cpu(cpu)|| (inidle &&
nohz_ratelimit(cpu))) {
Sorry right, the
inidle
is new, missed that one. I will rebuild in a minute and test.
Best wishes
Norbert
------------------------------------------------------------------------
Norbert Preining preining@{jaist.ac.jp, logic.at, debian.org}
JAIST, Japan TeX Live & Debian Developer
DSA: 0x09C5B094 fp: 14DF 2E6C 0307 BE6D AD76 A9C0 D2BF 4AA3 09C5 B094
------------------------------------------------------------------------
AITH (n.)
The single bristle that sticks out sideways on a cheap paintbrush.
--- Douglas Adams, The Meaning of Liff
Quoted 5 lines
The following line is different, the new code remove the
nohz_ratelimitfromirq_exit(). That may resolve your system too many interrupts issue.+ arch_needs_cpu(cpu)|| (inidle &&
nohz_ratelimit(cpu))) {
That is a very nice one ..., first time I came down with inactive laptop, lowest brightness, down to below 8W:
Cn Avg residency P-states (frequencies)
C0 (cpu running) ( 1.1%) Turbo Mode 0.0% C0 0.0ms ( 0.0%) 2.54 Ghz 0.0% C1 mwait 0.0ms ( 0.0%) 1.60 Ghz 0.0%
C2 mwait 0.3ms ( 0.8%) 800 Mhz 99.9%
C6 mwait 16.7ms (98.2%)
Wakeups-from-idle per second : 81.3 interval: 15.0s
Power usage (ACPI estimate): 7.9W (11.0 hours)
Top causes for wakeups:
23.8% ( 18.1) kworker/0:0 14.0% ( 10.7) [ahci] <interrupt> 13.4% ( 10.2) [kernel scheduler] Load balancing tick 6.6% ( 5.0) syndaemon 5.5% ( 4.2) Xorg 5.5% ( 4.2) [extra timer interrupt] 4.6% ( 3.5) [acpi] <interrupt>
Very nice, with activity and bluetooth and internet over bluetooth I am back at 10.3W, but that is also fine.
Anything else you need to know (kernel was latest git, -rc6 AFAIR).
Best wishes
Norbert
------------------------------------------------------------------------
Norbert Preining preining@{jaist.ac.jp, logic.at, debian.org}
JAIST, Japan TeX Live & Debian Developer
DSA: 0x09C5B094 fp: 14DF 2E6C 0307 BE6D AD76 A9C0 D2BF 4AA3 09C5 B094
------------------------------------------------------------------------
The main reception foyer was almost empty but Ford nevertheless weaved his way through it.
--- Ford making his way out of Milliways whilst under the --- influence of enough alchol to make a rhino sing. --- Douglas Adams, The Hitchhikers Guide to the Galaxy
Alex,Shi
Tested-by Norbert PreiningThanks Norbert!
Mike&Peter:
would you like to add signed-off for the following patch?
---
sched: nohz_ratelimit function refresh
The nohz_ratelimit() function that written by Mike Galbraith can bring about more than 10% throughput for netperf TCP/UDP RR when scheduling cross-cpu. It did this by reducing down to nohz mode chance.
But the patch also reduce CPU chance to nohz mode after interrupt processed, that cause Norbert's system have 4 watts power increase(the system have about 100 int/sec and with a light load). That is not acceptable for a laptop.
So, I remove the nohz_ratelimit from irq_exit(). and then the Norbert's system back to low power consumption.
include/linux/sched.h +5 -0
@@ -274,8 +274,13 @@ extern cpumask_var_t nohz_cpu_mask;#if defined(CONFIG_SMP ) && defined(CONFIG_NO_HZ )extern void select_nohz_load_balancer (int stop_tick);extern int get_nohz_timer_target (void ); +extern int nohz_ratelimit (int cpu);#else static inline void select_nohz_load_balancer (int stop_tick) { } +static inline int nohz_ratelimit (int cpu) +{ +return 0 ; +}#endif /*
kernel/sched.c +10 -0
@@ -1182,6 +1182,16 @@ static void resched_task(struct task_struct *p)smp_send_reschedule (cpu); } +int nohz_ratelimit (int cpu) +{ +struct rq *rq =cpu_rq (cpu); +u64 diff = rq->clock - rq->nohz_stamp; + + rq->nohz_stamp = rq->clock; + +return diff < (NSEC_PER_SEC /HZ ) >>1 ; +} +static void resched_cpu (int cpu) {struct rq *rq =cpu_rq (cpu);
kernel/time/tick-sched.c +1 -1
@@ -325,7 +325,7 @@ void tick_nohz_stop_sched_tick(int inidle) }while (read_seqretry (&xtime_lock, seq));if (rcu_needs_cpu (cpu) ||printk_needs_cpu (cpu) || -arch_needs_cpu (cpu)) { +arch_needs_cpu (cpu) || (inidle &&nohz_ratelimit (cpu))) { next_jiffies = last_jiffies +1 ; delta_jiffies =1 ; }else {
Peter Zijlstra
maintainerQuoted 3 lines · Alex,Shi
Thanks Norbert!
Mike&Peter:
would you like to add signed-off for the following patch?
I would really rather see the nohz decision move into the whole cpuidle governor thing, which can make a much better cost vs benefit decision.
Thomas, Arjan?
Quoted 72 lines
---
sched:nohz_ratelimitfunction refreshThe
nohz_ratelimit()function that written by Mike Galbraith can bring about more than 10% throughput for netperf TCP/UDP RR when scheduling cross-cpu. It did this by reducing down to nohz mode chance.
But the patch also reduce CPU chance to nohz mode after interrupt processed, that cause Norbert's system have 4 watts power increase(the system have about 100 int/sec and with a light load). That is not acceptable for a laptop.So, I remove the
nohz_ratelimitfromirq_exit(). and then the Norbert's system back to low power consumption.Tested-by Norbert Preining <[email protected]>Signed-off-by Alex Shi <[email protected]>include/linux/sched.h +5
-0@@ -274,8 +274,13 @@ extern cpumask_var_t nohz_cpu_mask;#if defined(CONFIG_SMP ) && defined(CONFIG_NO_HZ )extern void select_nohz_load_balancer (int stop_tick);extern int get_nohz_timer_target (void ); +extern int nohz_ratelimit (int cpu);#else static inline void select_nohz_load_balancer (int stop_tick) { } +static inline int nohz_ratelimit (int cpu) +{ +return 0 ; +}#endif /* kernel/sched.c +10
-0@@ -1182,6 +1182,16 @@ static void resched_task(struct task_struct *p)smp_send_reschedule (cpu); } +int nohz_ratelimit (int cpu) +{ +struct rq *rq =cpu_rq (cpu); +u64 diff = rq->clock - rq->nohz_stamp; + + rq->nohz_stamp = rq->clock; + +return diff < (NSEC_PER_SEC /HZ ) >>1 ; +} +static void resched_cpu (int cpu) {struct rq *rq =cpu_rq (cpu);kernel/time/tick-sched.c +1
-1@@ -325,7 +325,7 @@ void tick_nohz_stop_sched_tick(int inidle) }while (read_seqretry (&xtime_lock, seq));if (rcu_needs_cpu (cpu) ||printk_needs_cpu (cpu) || -arch_needs_cpu (cpu)) { +arch_needs_cpu (cpu) || (inidle &&nohz_ratelimit (cpu))) { next_jiffies = last_jiffies +1 ; delta_jiffies =1 ; }else {
Quoted 7 lines
Quoted 3 lines
Thanks Norbert!
Mike&Peter:
would you like to add signed-off for the following patch?I would really rather see the nohz decision move into the whole cpuidle governor thing, which can make a much better cost vs benefit decision.
See!