i386 No Idle HZ aka dynticks 051221
Subsystems (5)
PATCH 1/1 i386 No Idle HZ aka dynticks 051221
Here is the latest version of the dynticks code incorporating a huge rewrite correcting all the known problems with the previous code.
Major Changes:
All the tick skipping data is per cpu now, allowing each cpu to skip ticks according to its own timers. The data for the next scheduled tick is also available based on this information allowing it to be used when all cpus are idle (eg to disable the PIT timer).
Management of APIC on SMP and UP now has a boottime check that determines the best way to reprogram the timer, and selects the most reliable one. Unfortunately, being i386 this is almost always PIT reprogramming on UP (now the APIC is disabled when this is used) and APIC reprogramming per CPU but PIT reprogramming when all cpus are idle.
Bugs fixed:
This corrects the problem of too many interrupts on SMP
SMP reprogramming now is actually timer correct
The cpu accounting is now correct
The stalling on UP machines with local apic requiring keypresses was abolished The maximum number of APIC ticks to skip is allowed to be as high as possible instead of some arbitrary maximum
Runtime overhead has been reduced
Tick rates are lower than ever (on a single user console on SMP it idles at 18HZ)
There are now no known bugs. Of course that doesn't mean there won't be...
Please test and comment.
Power savings on laptops vary from .3 to 2W. The higher the cpu frequency, the more the power savings are (when idle).
Latest patch, split patches and timer utilties available here:
http://ck.kolivas.org/patches/dyn-ticks/
Rolled up patch in this email.
Cheers,
Con
---
Implement i386 no-idle-hz aka dynticks rolled up patch.
Original code by Tony Lindgen <[email protected]>,
Tuukka Tikkanen <[email protected]>,
Srivatsa Vaddagiri <[email protected]>,
Daniel Petrini <[email protected]> and
Dominik Brodowski <[email protected]>.
Rewritten and updated by Con Kolivas <[email protected]>
arch/i386/Kconfig | 20 + arch/i386/defconfig | 1 arch/i386/kernel/Makefile | 1 arch/i386/kernel/apic.c | 31 ++ arch/i386/kernel/dyn-tick.c | 340 ++++++++++++++++++++++++++++++++ arch/i386/kernel/io_apic.c | 26 ++ arch/i386/kernel/irq.c | 4 arch/i386/kernel/process.c | 4 arch/i386/kernel/smp.c | 9 arch/i386/kernel/time.c | 19 + arch/i386/kernel/timers/timer_cyclone.c | 4 arch/i386/kernel/timers/timer_hpet.c | 4 arch/i386/kernel/timers/timer_none.c | 3 arch/i386/kernel/timers/timer_pit.c | 35 +++ arch/i386/kernel/timers/timer_pm.c | 32 +-- arch/i386/kernel/timers/timer_tsc.c | 62 ++--- drivers/acpi/Kconfig | 2 drivers/acpi/processor_idle.c | 75 ++++++- drivers/input/serio/i8042.h | 2 include/acpi/processor.h | 3 include/asm-i386/apic.h | 2 include/asm-i386/dyn-tick.h | 76 +++++++ include/asm-i386/timer.h | 39 +++ include/linux/dyn-tick.h | 113 ++++++++++ include/linux/timer.h | 1 kernel/Makefile | 2 kernel/dyn-tick.c | 313 +++++++++++++++++++++++++++++ kernel/timer.c | 10 kernel/timer_top.c | 234 ++++++++++++++++++++++ lib/Kconfig.debug | 13 + 30 files changed, 1406 insertions(+), 74 deletions(-)
include/linux/dyn-tick.h new file +113 -0
@@ -0,0 +1,113 @@ +/* +* linux/include/linux/dyn-tick.h +* +* Copyright (C) 2004 Nokia Corporation +* Written by Tony Lindgen <[email protected]> and +* Tuukka Tikkanen <[email protected]> +* Rewritten by Con Kolivas <[email protected]> +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License version 2 as +* published by the Free Software Foundation. +*/ + +#ifndef _DYN_TICK_TIMER_H +#define _DYN_TICK_TIMER_H + +#include <linux/interrupt.h> + +#define DYN_TICK_SKIPPING (1 << 2) +#define DYN_TICK_ENABLED (1 << 1) +#define DYN_TICK_SUITABLE (1 << 0) + +/* Don't skip longer than NMI */ +#define DYN_TICK_MAX_SKIP (HZ * 5) + +struct dyn_tick_timer { +spinlock_t lock; + +/* dyn_tick init */ +int (*arch_init) (void ); +/* Enables dynamic tick */ +int (*arch_enable)(void ); +/* Disables dynamic tick */ +int (*arch_disable)(void ); +/* Reprograms the timer */ +void (*arch_reprogram)(unsigned long ); +/* Function called when all cpus are idle */ +void (*arch_all_cpus_idle) (int ); + +unsigned short state;/* Current state */ +unsigned int min_skip;/* Min number of ticks to skip */ +unsigned int max_skip;/* Max number of ticks to skip */ +unsigned long tick;/* The next earliest tick */ +}; + +typedef struct { +unsigned long next_tick;/* Next tick we're skipping to */ +unsigned long skip;/* Ticks we're currently skipping */ +unsigned int nohz_cpu;/* This cpu is idle */ +}____cacheline_aligned dyn_tick_data ; + +extern struct dyn_tick_timer *dyn_tick; +extern spinlock_t *dyn_tick_lock; + +extern void dyn_tick_register (struct dyn_tick_timer *new_timer); + +#ifdef CONFIG_NO_IDLE_HZ +DECLARE_PER_CPU (dyn_tick_data, dyn_cpu); +extern dyn_tick_data dyn_cpu; +extern int dyn_tick_enabled (void ); +extern int dyn_tick_skipping (void ); +extern int dyn_tick_current_skip (void ); +extern unsigned long dyn_tick_next_tick (void ); +extern void timer_dyn_reprogram (void ); +extern void dyn_early_reprogram (unsigned int delta); +extern void set_dyn_tick_limits (unsigned int max_skip,unsigned int min_skip); + +#ifdef CONFIG_TIMER_INFO +extern int account_timer (unsigned long function,unsigned long data); +#else +static inline int account_timer (unsigned long function,unsigned long data) +{ +return 0 ; +} +#endif + +#else /* CONFIG_NO_IDLE_HZ */ +static inline int dyn_tick_enabled (void ) +{ +return 0 ; +} + +static inline int dyn_tick_skipping (void ) +{ +return 0 ; +} + +static inline int dyn_tick_current_skip (void ) +{ +return 0 ; +} + +static inline unsigned long dyn_tick_next_tick (void ) +{ +return 0 ; +} + +static inline void set_dyn_tick_limits (unsigned int max_skip, +unsigned int min_skip) +{ +} + +static inline void dyn_early_reprogram (unsigned int delta) +{ +} + +static inline int account_timer (unsigned long function,unsigned long data) +{ +return 0 ; +} +#endif /* CONFIG_NO_IDLE_HZ */ + +#endif /* _DYN_TICK_TIMER_H */
include/linux/timer.h +1 -0
@@ -97,5 +97,6 @@ static inline void add_timer(struct timeextern void init_timers (void );extern void run_local_timers (void );extern void it_real_fn (unsigned long ); +extern void conditional_run_local_timers (void );#endif
kernel/dyn-tick.c new file +313 -0
313 lines not on this page · raw message
kernel/Makefile +2 -0
@@ -32,6 +32,8 @@ obj-$(CONFIG_GENERIC_HARDIRQS) += irq/ obj-$(CONFIG_CRASH_DUMP) += crash_dump.o obj-$(CONFIG_SECCOMP) += seccomp.o obj-$(CONFIG_RCU_TORTURE_TEST) += rcutorture.o +obj-$(CONFIG_NO_IDLE_HZ) += dyn-tick.o +obj-$(CONFIG_TIMER_INFO) += timer_top.o ifneq ($(CONFIG_SCHED_NO_NO_OMIT_FRAME_POINTER),y) # According to Alan Modra <[email protected]>, the -fno-omit-frame-pointer is
kernel/timer.c +10 -0
@@ -33,6 +33,7 @@#include <linux/posix-timers.h> #include <linux/cpu.h> #include <linux/syscalls.h> +#include <linux/dyn-tick.h> #include <asm/uaccess.h> #include <asm/unistd.h>
@@ -540,6 +541,8 @@ found: expires = nte->expires; } } +account_timer ((unsigned long )nte->function, nte->data); +spin_unlock (&base->t_base.lock);return expires; }
@@ -869,6 +872,13 @@ void run_local_timers(void)raise_softirq (TIMER_SOFTIRQ ); } +void conditional_run_local_timers (void ) +{ +tvec_base_t *base = &__get_cpu_var (tvec_bases); + +if (base->timer_jiffies != jiffies) +run_local_timers (); +}/* * Called by the timer interrupt. xtime_lock must already be taken * by the timer IRQ!
drivers/acpi/Kconfig +2 -0
@@ -301,6 +301,8 @@ config X86_PM_TIMER voltage scaling, unlike the commonly used Time Stamp Counter (TSC) timing source. + This timer is selected by dyntick (NO_IDLE_HZ). + So, if you see messages like 'Losing too many ticks!' in the kernel logs, and/or you are using this on a notebook which does not yet have an HPET, you should say "Y" here.
arch/i386/defconfig +1 -0
@@ -91,6 +91,7 @@ CONFIG_X86_INTEL_USERCOPY=y CONFIG_X86_USE_PPRO_CHECKSUM=y # CONFIG_HPET_TIMER is not set # CONFIG_HPET_EMULATE_RTC is not set +# CONFIG_NO_IDLE_HZ is not set CONFIG_SMP=y CONFIG_NR_CPUS=8 CONFIG_SCHED_SMT=y
arch/i386/Kconfig +20 -0
@@ -173,6 +173,26 @@ config HPET_EMULATE_RTC depends on HPET_TIMER && RTC=y default y +config NO_IDLE_HZ + bool "Dynamic Tick Timer - Skip timer ticks during idle" + depends on EXPERIMENTAL + select X86_PM_TIMER + help + This option enables support for skipping timer ticks when the + processor is idle. During system load, timer is continuous. + This option saves power, as it allows the system to stay in + idle mode longer. Currently the only supported timer is ACPI PM + timer. + + Note that you can disable dynamic tick timer either by + passing dyntick=disable command line option, or via sysfs: + + # echo 0 > /sys/devices/system/timer/timer0/dyn_tick + + Most users wishing to lower their power usage while retaining + low latencies will most likely want to say Y here in combination + with a high HZ value (eg 1000). + config SMP bool "Symmetric multi-processing support" ---help---
arch/i386/kernel/apic.c +30 -1
@@ -26,6 +26,7 @@#include <linux/kernel_stat.h> #include <linux/sysdev.h> #include <linux/cpu.h> +#include <linux/dyn-tick.h> #include <asm/atomic.h> #include <asm/smp.h>
@@ -35,6 +36,7 @@#include <asm/arch_hooks.h> #include <asm/hpet.h> #include <asm/i8253.h> +#include <asm/dyn-tick.h> #include <mach_apic.h>
@@ -932,6 +934,8 @@ void (*wait_timer_tick)(void) __devinitd#define APIC_DIVISOR 16 +static u32 apic_timer_val__read_mostly ; +static void __setup_APIC_LVTT (unsigned int clocks) {unsigned int lvtt_value, tmp_value, ver;
@@ -950,7 +954,9 @@ static void __setup_APIC_LVTT(unsigned i & ~(APIC_TDR_DIV_1 | APIC_TDR_DIV_TMBASE)) | APIC_TDR_DIV_16); -apic_write_around (APIC_TMICT , clocks/APIC_DIVISOR ); + apic_timer_val = clocks /APIC_DIVISOR ; + +apic_write_around (APIC_TMICT , apic_timer_val); }static void __devinit setup_APIC_timer (unsigned int clocks)
@@ -969,6 +975,17 @@ static void __devinit setup_APIC_timer(ulocal_irq_restore (flags); } +/* Used by NO_IDLE_HZ to skip ticks on idle CPUs */ +void reprogram_apic_timer (unsigned long count) +{ +unsigned long flags; + + count = count * apic_timer_val; +local_irq_save (flags); +apic_write_around (APIC_TMICT , count); +local_irq_restore (flags); +} +/* * In this function we calibrate APIC bus clocks to the external * timer. Unfortunately we cannot use jiffies and the timer irq
@@ -1064,6 +1081,9 @@ void __init setup_boot_APIC_clock(void)*/ setup_APIC_timer (calibration_result); +setup_dyn_tick_use_apic (calibration_result); +set_dyn_tick_limits ((0xFFFFFFFF / calibration_result) *APIC_DIVISOR ,0 ); +local_irq_restore (flags); }
@@ -1202,6 +1222,9 @@ fastcall void smp_apic_timer_interrupt(s* interrupt lock, which is the WrongThing (tm) to do. */ irq_enter (); + +dyn_tick_interrupt (regs); +smp_local_timer_interrupt (regs);irq_exit (); }
@@ -1214,6 +1237,9 @@ fastcall void smp_spurious_interrupt(strunsigned long v;irq_enter (); + +dyn_tick_interrupt (regs); +/* * Check if this really is a spurious interrupt and ACK it * if it is a vectored one. Just in case...
@@ -1238,6 +1264,9 @@ fastcall void smp_error_interrupt(structunsigned long v, v1;irq_enter (); + +dyn_tick_interrupt (regs); +/* First tickle the hardware, only then report what went on. -- REW */ v =apic_read (APIC_ESR );apic_write (APIC_ESR ,0 );
arch/i386/kernel/dyn-tick.c new file +340 -0
340 lines not on this page · raw message
arch/i386/kernel/io_apic.c +23 -3
@@ -32,6 +32,7 @@#include <linux/acpi.h> #include <linux/module.h> #include <linux/sysdev.h> +#include <linux/dyn-tick.h> #include <asm/io.h> #include <asm/smp.h>
@@ -1179,6 +1180,7 @@ next:static struct hw_interrupt_type ioapic_level_type;static struct hw_interrupt_type ioapic_edge_type; +static struct hw_interrupt_type ioapic_edge_type_irq0;#define IOAPIC_AUTO -1#define IOAPIC_EDGE 0
@@ -1190,15 +1192,19 @@ static inline void ioapic_register_intr(if ((trigger ==IOAPIC_AUTO &&IO_APIC_irq_trigger (irq)) || trigger ==IOAPIC_LEVEL ) irq_desc[vector].handler = &ioapic_level_type; -else +else if (vector) irq_desc[vector].handler = &ioapic_edge_type; +else + irq_desc[vector].handler = &ioapic_edge_type_irq0;set_intr_gate (vector, interrupt[vector]); }else {if ((trigger ==IOAPIC_AUTO &&IO_APIC_irq_trigger (irq)) || trigger ==IOAPIC_LEVEL ) irq_desc[irq].handler = &ioapic_level_type; -else +else if (irq) irq_desc[irq].handler = &ioapic_edge_type; +else + irq_desc[irq].handler = &ioapic_edge_type_irq0;set_intr_gate (vector, interrupt[irq]); } }
@@ -1311,7 +1317,7 @@ static void __init setup_ExtINT_IRQ0_pin* The timer IRQ doesn't have to know that behind the * scene we have a 8259A-master in AEOI mode ... */ - irq_desc[0 ].handler = &ioapic_edge_type; + irq_desc[0 ].handler = &ioapic_edge_type_irq0;/* * Add it to the IO-APIC irq-routing table:
@@ -2088,6 +2094,20 @@ static struct hw_interrupt_type ioapic_l#endif }; +/* Needed to disable PIT interrupts when all CPUs sleep */ +static struct hw_interrupt_type ioapic_edge_type_irq0__read_mostly = { + .typename ="IO-APIC-edge-irq0" , + .startup = startup_edge_ioapic, + .shutdown = shutdown_edge_ioapic, + .enable = unmask_IO_APIC_irq, + .disable = mask_IO_APIC_irq, + .ack = ack_edge_ioapic, + .end = end_edge_ioapic, +#ifdef CONFIG_SMP + .set_affinity = set_ioapic_affinity, +#endif +}; +static inline void init_IO_APIC_traps (void ) {int irq;
arch/i386/kernel/irq.c +4 -0
@@ -18,6 +18,8 @@#include <linux/notifier.h> #include <linux/cpu.h> #include <linux/delay.h> +#include <linux/dyn-tick.h> +#include <asm/dyn-tick.h> DEFINE_PER_CPU(irq_cpustat_t, irq_stat) ____cacheline_maxaligned_in_smp;EXPORT_PER_CPU_SYMBOL (irq_stat);
@@ -76,6 +78,8 @@ fastcall unsigned int do_IRQ(struct pt_r }#endif +dyn_tick_interrupt (regs); +#ifdef CONFIG_4KSTACKS curctx = (union irq_ctx *)current_thread_info ();
arch/i386/kernel/Makefile +1 -0
@@ -32,6 +32,7 @@ obj-$(CONFIG_MODULES) += module.o obj-y += sysenter.o vsyscall.o obj-$(CONFIG_ACPI_SRAT) += srat.o obj-$(CONFIG_HPET_TIMER) += time_hpet.o +obj-$(CONFIG_NO_IDLE_HZ) += dyn-tick.o obj-$(CONFIG_EFI) += efi.o efi_stub.o obj-$(CONFIG_EARLY_PRINTK) += early_printk.o
arch/i386/kernel/process.c +4 -0
@@ -39,6 +39,7 @@#include <linux/ptrace.h> #include <linux/random.h> #include <linux/kprobes.h> +#include <linux/dyn-tick.h> #include <asm/uaccess.h> #include <asm/pgtable.h>
@@ -56,6 +57,7 @@#include <asm/tlbflush.h> #include <asm/cpu.h> +#include <asm/dyn-tick.h> asmlinkage void ret_from_fork(void) __asm__("ret_from_fork");
@@ -194,6 +196,8 @@ void cpu_idle(void)play_dead ();__get_cpu_var (irq_stat).idle_timestamp = jiffies; +idle_reprogram_timer (); +idle (); }preempt_enable_no_resched ();
arch/i386/kernel/smp.c +9 -0
@@ -20,9 +20,11 @@#include <linux/interrupt.h> #include <linux/cpu.h> #include <linux/module.h> +#include <linux/dyn-tick.h> #include <asm/mtrr.h> #include <asm/tlbflush.h> +#include <asm/dyn-tick.h> #include <mach_apic.h> /*
@@ -313,6 +315,8 @@ fastcall void smp_invalidate_interrupt(s {unsigned long cpu; +dyn_tick_interrupt (regs); + cpu =get_cpu ();if (!cpu_isset (cpu, flush_cpumask))
@@ -600,6 +604,8 @@ void smp_send_stop(void) fastcall void smp_reschedule_interrupt(struct pt_regs *regs) {ack_APIC_irq (); + +dyn_tick_interrupt (regs); } fastcall void smp_call_function_interrupt(struct pt_regs *regs)
@@ -609,6 +615,9 @@ fastcall void smp_call_function_interrupint wait = call_data->wait;ack_APIC_irq (); + +dyn_tick_interrupt (regs); +/* * Notify initiating CPU that I've grabbed the data and am * about to execute the function
arch/i386/kernel/time.c +13 -6
@@ -46,6 +46,7 @@#include <linux/bcd.h> #include <linux/efi.h> #include <linux/mca.h> +#include <linux/dyn-tick.h> #include <asm/io.h> #include <asm/smp.h>
@@ -56,6 +57,7 @@#include <asm/uaccess.h> #include <asm/processor.h> #include <asm/timer.h> +#include <asm/dyn-tick.h> #include "mach_time.h"
@@ -245,7 +247,7 @@ EXPORT_SYMBOL(profile_pc);* timer_interrupt() needs to keep up the real-time clock, * as well as call the "do_timer()" routine every clocktick */ -static inline void do_timer_interrupt (int irq,struct pt_regs *regs) +static inline void do_timer_interrupt (int irq,struct pt_regs *regs,int lost) {#ifdef CONFIG_X86_IO_APIC if (timer_ack) {
@@ -263,7 +265,8 @@ static inline void do_timer_interrupt(in }#endif -do_timer_interrupt_hook (regs); +if (!dyn_tick_enabled () || lost) +do_timer_interrupt_hook (regs);if (MCA_bus) {
@@ -288,6 +291,8 @@ static inline void do_timer_interrupt(in*/ irqreturn_t timer_interrupt (int irq,void *dev_id,struct pt_regs *regs) { +int lost; +/* * Here we are in the timer irq handler. We just have irqs locally * disabled but we don't know if the timer_bh is running on the other
@@ -297,9 +302,9 @@ irqreturn_t timer_interrupt(int irq, voi*/ write_seqlock (&xtime_lock); - cur_timer->mark_offset (); - -do_timer_interrupt (irq, regs); + lost = cur_timer->mark_offset (); + +do_timer_interrupt (irq, regs, lost);write_sequnlock (&xtime_lock);return IRQ_HANDLED ;
@@ -425,7 +430,7 @@ static struct sysdev_class timer_sysclas/* XXX this driverfs stuff should probably go elsewhere later -john */ -static struct sys_device device_timer = { +struct sys_device device_timer = { .id =0 , .cls = &timer_sysclass, };
@@ -481,5 +486,7 @@ void __init time_init(void) cur_timer =select_timer ();printk (KERN_INFO "Using %s for high-res timesource\n" ,cur_timer->name); +dyn_tick_time_init (cur_timer); +time_init_hook (); }
arch/i386/kernel/timers/timer_pit.c +34 -1
@@ -32,9 +32,10 @@ static int __init init_pit(char* overridreturn 0 ; } -static void mark_offset_pit (void ) +static int mark_offset_pit (void ) {/* nothing needed */ +return 1 ; }static unsigned long long monotonic_clock_pit (void )
@@ -148,6 +149,38 @@ static unsigned long get_offset_pit(voidreturn count; } +void disable_pit_timer (void ) +{ + irq_desc[0 ].handler->disable (0 ); +} + +void enable_pit_timer (void ) +{ + irq_desc[0 ].handler->enable (0 ); +} + +/* +* Reprograms the next timer interrupt +* PIT timer reprogramming code taken from APM code. +* Note that PIT timer is a 16-bit timer, which allows max +* skip of only few seconds. +*/ +void reprogram_pit_timer (unsigned long jiffies_to_skip) +{ +int skip; +extern spinlock_t i8253_lock; +unsigned long flags; + + skip = jiffies_to_skip *LATCH ; +if (skip >0xffff ) + skip =0xffff ; + +spin_lock_irqsave (&i8253_lock, flags); +outb_p (0x34 ,PIT_MODE );/* binary, mode 2, LSB/MSB, ch 0 */ +outb_p (skip &0xff ,PIT_CH0 );/* LSB */ +outb (skip >>8 ,PIT_CH0 );/* MSB */ +spin_unlock_irqrestore (&i8253_lock, flags); +}/* tsc timer_opts struct */ struct timer_opts timer_pit = {
arch/i386/kernel/timers/timer_pm.c +15 -17
@@ -15,6 +15,7 @@#include <linux/module.h> #include <linux/device.h> #include <linux/init.h> +#include <linux/dyn-tick.h> #include <asm/types.h> #include <asm/timer.h> #include <asm/smp.h>
@@ -28,7 +29,7 @@#define PMTMR_TICKS_PER_SEC 3579545#define PMTMR_EXPECTED_RATE \ ((CALIBRATE_LATCH * (PMTMR_TICKS_PER_SEC >> 10)) / (CLOCK_TICK_RATE>>10)) - +#define PMTMR_TICKS_PER_JIFFY (PMTMR_EXPECTED_RATE / (CALIBRATE_LATCH/LATCH))/* The I/O port the PMTMR resides at. * The location is detected during setup_arch(),
@@ -127,7 +128,13 @@ pm_good:if (verify_pmtmr_rate () !=0 )return -ENODEV ; +printk ("Using %u PM timer ticks per jiffy \n" ,PMTMR_TICKS_PER_JIFFY ); + + offset_tick =read_pmtmr (); +setup_pit_timer (); +init_cpu_khz (); +set_dyn_tick_limits (((0xFFFFFF /1000000 ) *286 *HZ ) >>10 ,0 );return 0 ; }
@@ -148,10 +155,9 @@ static inline u32 cyc2us(u32 cycles)* this gets called during each timer interrupt * - Called while holding the writer xtime_lock */ -static void mark_offset_pmtmr (void ) +static int mark_offset_pmtmr (void ) {u32 lost, delta, last_offset; -static int first_run =1 ; last_offset = offset_tick;write_seqlock (&monotonic_lock);
@@ -161,29 +167,20 @@ static void mark_offset_pmtmr(void)/* calculate tick interval */ delta = (offset_tick - last_offset) &ACPI_PM_MASK ; -/* convert to usecs */ - delta =cyc2us (delta); -/* update the monotonic base value */ - monotonic_base += delta *NSEC_PER_USEC ; + monotonic_base +=cyc2us (delta) *NSEC_PER_USEC ;write_sequnlock (&monotonic_lock);/* convert to ticks */ delta += offset_delay; - lost = delta / (USEC_PER_SEC /HZ ); - offset_delay = delta % (USEC_PER_SEC /HZ ); - + lost = delta /PMTMR_TICKS_PER_JIFFY ; + offset_delay = delta %PMTMR_TICKS_PER_JIFFY ;/* compensate for lost ticks */ if (lost >=2 ) jiffies_64 += lost -1 ; -/* don't calculate delay for first run, -or if we've got less then a tick */ -if (first_run || (lost <1 )) { - first_run =0 ; - offset_delay =0 ; - } +return lost; }static int pmtmr_resume (void )
@@ -192,6 +189,7 @@ static int pmtmr_resume(void)/* Assume this is the last mark offset time */ offset_tick =read_pmtmr ();write_sequnlock (&monotonic_lock); + offset_delay =0 ;return 0 ; }
@@ -243,7 +241,7 @@ static unsigned long get_offset_pmtmr(vo now =read_pmtmr (); delta = (now - offset)&ACPI_PM_MASK ; -return (unsigned long ) offset_delay +cyc2us (delta); +return (unsigned long )cyc2us (delta + offset_delay); }
arch/i386/kernel/timers/timer_tsc.c +27 -35
142 lines not on this page · raw message
include/asm-i386/apic.h +2 -0
@@ -121,6 +121,7 @@ extern void nmi_watchdog_tick (struct ptextern int APIC_init_uniprocessor (void );extern void disable_APIC_timer (void );extern void enable_APIC_timer (void ); +extern void reprogram_apic_timer (unsigned long count);extern void enable_NMI_through_LVT0 (void * dummy);
@@ -134,6 +135,7 @@ extern int disable_timer_pin_1;#else /* !CONFIG_X86_LOCAL_APIC */ static inline void lapic_shutdown (void ) { } +static inline void reprogram_apic_timer (unsigned long count) { }#endif /* !CONFIG_X86_LOCAL_APIC */
include/asm-i386/dyn-tick.h new file +76 -0
76 lines not on this page · raw message
include/asm-i386/timer.h +38 -1
@@ -1,5 +1,6 @@#ifndef _ASMi386_TIMER_H#define _ASMi386_TIMER_H +#include <linux/jiffies.h> #include <linux/init.h> #include <linux/pm.h>
@@ -19,7 +20,7 @@*/ struct timer_opts {char * name; -void (*mark_offset)(void ); +int (*mark_offset)(void );unsigned long (*get_offset)(void );unsigned long long (*monotonic_clock)(void );void (*delay)(unsigned long );
@@ -38,6 +39,9 @@ struct init_timer_opts {extern struct timer_opts *__init select_timer (void );extern void clock_fallback (void );void setup_pit_timer (void ); +extern void disable_pit_timer (void ); +extern void enable_pit_timer (void ); +extern void reprogram_pit_timer (unsigned long jiffies_to_skip);/* Modifiers for buggy PIT handling */
@@ -67,4 +71,37 @@ extern unsigned long calibrate_tsc_hpet(#ifdef CONFIG_X86_PM_TIMER extern struct init_timer_opts timer_pmtmr_init;#endif +#ifdef CONFIG_NO_IDLE_HZ +static inline void tsc_sanity_check (int lost) +{ +} +#else /* CONFIG_NO_IDLE_HZ */ +extern void cpufreq_delayed_get (void ); + +static inline void tsc_sanity_check (int lost) +{ +static int lost_count =0 ; + +if (lost >=2 ) { + jiffies_64 += lost-1 ; + +/* sanity check to ensure we're not always losing ticks */ +if (lost_count++ >100 ) { +printk (KERN_WARNING "Losing too many ticks!\n" ); +printk (KERN_WARNING "TSC cannot be used as a timesource. \n" ); +printk (KERN_WARNING "Possible reasons for this are:\n" ); +printk (KERN_WARNING " You're running with Speedstep,\n" ); +printk (KERN_WARNING " You don't have DMA enabled for your hard disk (see hdparm),\n" ); +printk (KERN_WARNING " Incorrect TSC synchronization on an SMP system (see dmesg).\n" ); +printk (KERN_WARNING "Falling back to a sane timesource now.\n" ); + +clock_fallback (); + } +/* ... but give the TSC a fair chance */ +if (lost_count >25 ) +cpufreq_delayed_get (); + }else + lost_count =0 ; +} +#endif /* CONFIG_NO_IDLE_HZ */ #endif
arch/i386/kernel/timers/timer_cyclone.c +3 -1
@@ -45,7 +45,7 @@ static seqlock_t monotonic_lock = SEQLOC }while (high != cyclone_timer[1 ]); -static void mark_offset_cyclone (void ) +static int mark_offset_cyclone (void ) {unsigned long lost, delay;unsigned long delta = last_cyclone_low;
@@ -101,6 +101,8 @@ static void mark_offset_cyclone(void)*/ if (lost &&abs (delay - delay_at_last_interrupt) > (900000 /HZ )) jiffies_64++; + +return lost; }static unsigned long get_offset_cyclone (void )
arch/i386/kernel/timers/timer_hpet.c +3 -1
@@ -101,7 +101,7 @@ static unsigned long get_offset_hpet(voireturn edx; } -static void mark_offset_hpet (void ) +static int mark_offset_hpet (void ) {unsigned long long this_offset, last_offset;unsigned long offset;
@@ -124,6 +124,8 @@ static void mark_offset_hpet(void) this_offset = ((unsigned long long )last_tsc_high<<32 )|last_tsc_low; monotonic_base +=cycles_2_ns (this_offset - last_offset);write_sequnlock (&monotonic_lock); + +return 1 ; }static void delay_hpet (unsigned long loops)
arch/i386/kernel/timers/timer_none.c +2 -1
@@ -1,9 +1,10 @@#include <linux/init.h> #include <asm/timer.h> -static void mark_offset_none (void ) +static int mark_offset_none (void ) {/* nothing needed */ +return 1 ; }static unsigned long get_offset_none (void )
kernel/timer_top.c new file +234 -0
234 lines not on this page · raw message
lib/Kconfig.debug +13 -0
@@ -77,6 +77,19 @@ config SCHEDSTATS application, you can say N to avoid the very slight overhead this adds. +config TIMER_INFO + bool "Collect kernel timers statistics" + depends on DEBUG_KERNEL && PROC_FS && NO_IDLE_HZ + help + If you say Y here, additional code will be inserted into the + timer routines to collect statistics about kernel timers being + reprogrammed through dynamic ticks feature. The statistics + will be provided in /proc/timer_info and the behavior of this + feature can be controlled through /proc/timer_input. + The goal is to offer some output to let user applications show + timer pattern usage and allow some tuning in them to + maximise idle time. + config DEBUG_SLAB bool "Debug memory allocations" depends on DEBUG_KERNEL
drivers/acpi/processor_idle.c +69 -6
123 lines not on this page · raw message
include/acpi/processor.h +3 -0
@@ -51,6 +51,7 @@ struct acpi_processor_cx {u32 latency_ticks;u32 power;u32 usage; +u64 time;struct acpi_processor_cx_policy promotion;struct acpi_processor_cx_policy demotion; };
@@ -60,6 +61,8 @@ struct acpi_processor_power {unsigned long bm_check_timestamp;u32 default_state;u32 bm_activity; +u16 last_sleep; +u16 quick_promotion;int count;struct acpi_processor_cx states[ACPI_PROCESSOR_MAX_POWER ];
drivers/input/serio/i8042.h +1 -1
@@ -44,7 +44,7 @@* polling. */ -#define I8042_POLL_PERIOD HZ/20 +#define I8042_POLL_PERIOD HZ/5/* * Status register bits.
Quoted 2 lines · Con Kolivas
Here is the latest version of the dynticks code incorporating a huge rewrite correcting all the known problems with the previous code.
Works nicely sofar. One slight bug in the pmstats code, the interval setting doesn't work because of a missing else clause.
One thing I'm curious about (and haven't tested yet): does this also work with S3 suspend to ram? Last dynticks I tried had issues with that...
Thanks,
Jan
Signature · 2 lines
Algebraic symbols are used when you do not know what you are talking about. -- Philippe Schnoebelen
Hi Jan,
Quoted 9 lines · Jan De Luyck
Quoted 2 lines · Con Kolivas
Here is the latest version of the dynticks code incorporating a huge rewrite correcting all the known problems with the previous code.
Works nicely sofar. One slight bug in the pmstats code, the interval setting doesn't work because of a missing else clause.
One thing I'm curious about (and haven't tested yet): does this also work with S3 suspend to ram? Last dynticks I tried had issues with that...
I'm testing suspend to ram and it is working normally with dyn-ticks. What kind of problems are you facing?
Daniel
Quoted 8 lines · Daniel Petrini
Hi Jan,
Quoted 2 lines · Jan De Luyck
One thing I'm curious about (and haven't tested yet): does this also work with S3 suspend to ram? Last dynticks I tried had issues with that...
I'm testing suspend to ram and it is working normally with dyn-ticks. What kind of problems are you facing?
I am facing none :) I just was facing (with an earlier dyntick patch, around 2.6.12.x) problems resuming with it. They seem to have been fixed completely :)
In X i'm around 180HZ with everything up and running. Nice :) Very good work everyone :)
Jan
Signature · 3 lines
Tu bi or not tu bi DICK -- Começo de musica brega -- DICK vale o céu azul ...
So I took Linus's advice and am the process of trying out 2.6.15rc7, and in I needed to rebase the dyntick patch to apply against rc7. Here's what I came up with....
- Ted
Here is the latest version of the dynticks code incorporating a huge rewrite correcting all the known problems with the previous code.
Major Changes:
All the tick skipping data is per cpu now, allowing each cpu to skip ticks according to its own timers. The data for the next scheduled tick is also available based on this information allowing it to be used when all cpus are idle (eg to disable the PIT timer).
Management of APIC on SMP and UP now has a boottime check that determines the best way to reprogram the timer, and selects the most reliable one. Unfortunately, being i386 this is almost always PIT reprogramming on UP (now the APIC is disabled when this is used) and APIC reprogramming per CPU but PIT reprogramming when all cpus are idle.
Bugs fixed:
This corrects the problem of too many interrupts on SMP
SMP reprogramming now is actually timer correct
The cpu accounting is now correct
The stalling on UP machines with local apic requiring keypresses was abolished The maximum number of APIC ticks to skip is allowed to be as high as possible instead of some arbitrary maximum
Runtime overhead has been reduced
Tick rates are lower than ever (on a single user console on SMP it idles at 18HZ)
There are now no known bugs. Of course that doesn't mean there won't be...
Please test and comment.
Power savings on laptops vary from .3 to 2W. The higher the cpu frequency, the more the power savings are (when idle).
Latest patch, split patches and timer utilties available here:
http://ck.kolivas.org/patches/dyn-ticks/
Rolled up patch in this email.
Cheers,
Con
include/linux/dyn-tick.h new file +113 -0
@@ -0,0 +1,113 @@ +/* +* linux/include/linux/dyn-tick.h +* +* Copyright (C) 2004 Nokia Corporation +* Written by Tony Lindgen <[email protected]> and +* Tuukka Tikkanen <[email protected]> +* Rewritten by Con Kolivas <[email protected]> +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License version 2 as +* published by the Free Software Foundation. +*/ + +#ifndef _DYN_TICK_TIMER_H +#define _DYN_TICK_TIMER_H + +#include <linux/interrupt.h> + +#define DYN_TICK_SKIPPING (1 << 2) +#define DYN_TICK_ENABLED (1 << 1) +#define DYN_TICK_SUITABLE (1 << 0) + +/* Don't skip longer than NMI */ +#define DYN_TICK_MAX_SKIP (HZ * 5) + +struct dyn_tick_timer { +spinlock_t lock; + +/* dyn_tick init */ +int (*arch_init) (void ); +/* Enables dynamic tick */ +int (*arch_enable)(void ); +/* Disables dynamic tick */ +int (*arch_disable)(void ); +/* Reprograms the timer */ +void (*arch_reprogram)(unsigned long ); +/* Function called when all cpus are idle */ +void (*arch_all_cpus_idle) (int ); + +unsigned short state;/* Current state */ +unsigned int min_skip;/* Min number of ticks to skip */ +unsigned int max_skip;/* Max number of ticks to skip */ +unsigned long tick;/* The next earliest tick */ +}; + +typedef struct { +unsigned long next_tick;/* Next tick we're skipping to */ +unsigned long skip;/* Ticks we're currently skipping */ +unsigned int nohz_cpu;/* This cpu is idle */ +}____cacheline_aligned dyn_tick_data ; + +extern struct dyn_tick_timer *dyn_tick; +extern spinlock_t *dyn_tick_lock; + +extern void dyn_tick_register (struct dyn_tick_timer *new_timer); + +#ifdef CONFIG_NO_IDLE_HZ +DECLARE_PER_CPU (dyn_tick_data, dyn_cpu); +extern dyn_tick_data dyn_cpu; +extern int dyn_tick_enabled (void ); +extern int dyn_tick_skipping (void ); +extern int dyn_tick_current_skip (void ); +extern unsigned long dyn_tick_next_tick (void ); +extern void timer_dyn_reprogram (void ); +extern void dyn_early_reprogram (unsigned int delta); +extern void set_dyn_tick_limits (unsigned int max_skip,unsigned int min_skip); + +#ifdef CONFIG_TIMER_INFO +extern int account_timer (unsigned long function,unsigned long data); +#else +static inline int account_timer (unsigned long function,unsigned long data) +{ +return 0 ; +} +#endif + +#else /* CONFIG_NO_IDLE_HZ */ +static inline int dyn_tick_enabled (void ) +{ +return 0 ; +} + +static inline int dyn_tick_skipping (void ) +{ +return 0 ; +} + +static inline int dyn_tick_current_skip (void ) +{ +return 0 ; +} + +static inline unsigned long dyn_tick_next_tick (void ) +{ +return 0 ; +} + +static inline void set_dyn_tick_limits (unsigned int max_skip, +unsigned int min_skip) +{ +} + +static inline void dyn_early_reprogram (unsigned int delta) +{ +} + +static inline int account_timer (unsigned long function,unsigned long data) +{ +return 0 ; +} +#endif /* CONFIG_NO_IDLE_HZ */ + +#endif /* _DYN_TICK_TIMER_H */
include/linux/timer.h +1 -0
@@ -97,5 +97,6 @@extern void init_timers (void );extern void run_local_timers (void );extern void it_real_fn (unsigned long ); +extern void conditional_run_local_timers (void );#endif
kernel/dyn-tick.c new file +313 -0
313 lines not on this page · raw message
kernel/Makefile +2 -0
@@ -32,6 +32,8 @@ obj-$(CONFIG_CRASH_DUMP) += crash_dump.o obj-$(CONFIG_SECCOMP) += seccomp.o obj-$(CONFIG_RCU_TORTURE_TEST) += rcutorture.o +obj-$(CONFIG_NO_IDLE_HZ) += dyn-tick.o +obj-$(CONFIG_TIMER_INFO) += timer_top.o ifneq ($(CONFIG_SCHED_NO_NO_OMIT_FRAME_POINTER),y) # According to Alan Modra <[email protected]>, the -fno-omit-frame-pointer is
kernel/timer.c +10 -0
@@ -33,6 +33,7 @@#include <linux/posix-timers.h> #include <linux/cpu.h> #include <linux/syscalls.h> +#include <linux/dyn-tick.h> #include <asm/uaccess.h> #include <asm/unistd.h>
@@ -540,6 +541,8 @@ expires = nte->expires; } } +account_timer ((unsigned long )nte->function, nte->data); +spin_unlock (&base->t_base.lock);return expires; }
@@ -869,6 +872,13 @@raise_softirq (TIMER_SOFTIRQ ); } +void conditional_run_local_timers (void ) +{ +tvec_base_t *base = &__get_cpu_var (tvec_bases); + +if (base->timer_jiffies != jiffies) +run_local_timers (); +}/* * Called by the timer interrupt. xtime_lock must already be taken * by the timer IRQ!
drivers/acpi/Kconfig +2 -0
@@ -301,6 +301,8 @@ voltage scaling, unlike the commonly used Time Stamp Counter (TSC) timing source. + This timer is selected by dyntick (NO_IDLE_HZ). + So, if you see messages like 'Losing too many ticks!' in the kernel logs, and/or you are using this on a notebook which does not yet have an HPET, you should say "Y" here.
arch/i386/defconfig +1 -0
@@ -91,6 +91,7 @@ CONFIG_X86_USE_PPRO_CHECKSUM=y # CONFIG_HPET_TIMER is not set # CONFIG_HPET_EMULATE_RTC is not set +# CONFIG_NO_IDLE_HZ is not set CONFIG_SMP=y CONFIG_NR_CPUS=8 CONFIG_SCHED_SMT=y
arch/i386/Kconfig +20 -0
@@ -173,6 +173,26 @@ depends on HPET_TIMER && RTC=y default y +config NO_IDLE_HZ + bool "Dynamic Tick Timer - Skip timer ticks during idle" + depends on EXPERIMENTAL + select X86_PM_TIMER + help + This option enables support for skipping timer ticks when the + processor is idle. During system load, timer is continuous. + This option saves power, as it allows the system to stay in + idle mode longer. Currently the only supported timer is ACPI PM + timer. + + Note that you can disable dynamic tick timer either by + passing dyntick=disable command line option, or via sysfs: + + # echo 0 > /sys/devices/system/timer/timer0/dyn_tick + + Most users wishing to lower their power usage while retaining + low latencies will most likely want to say Y here in combination + with a high HZ value (eg 1000). + config SMP bool "Symmetric multi-processing support" ---help---
arch/i386/kernel/apic.c +30 -1
@@ -26,6 +26,7 @@#include <linux/kernel_stat.h> #include <linux/sysdev.h> #include <linux/cpu.h> +#include <linux/dyn-tick.h> #include <asm/atomic.h> #include <asm/smp.h>
@@ -35,6 +36,7 @@#include <asm/arch_hooks.h> #include <asm/hpet.h> #include <asm/i8253.h> +#include <asm/dyn-tick.h> #include <mach_apic.h>
@@ -932,6 +934,8 @@#define APIC_DIVISOR 16 +static u32 apic_timer_val__read_mostly ; +static void __setup_APIC_LVTT (unsigned int clocks) {unsigned int lvtt_value, tmp_value, ver;
@@ -950,7 +954,9 @@ & ~(APIC_TDR_DIV_1 | APIC_TDR_DIV_TMBASE)) | APIC_TDR_DIV_16); -apic_write_around (APIC_TMICT , clocks/APIC_DIVISOR ); + apic_timer_val = clocks /APIC_DIVISOR ; + +apic_write_around (APIC_TMICT , apic_timer_val); }static void __devinit setup_APIC_timer (unsigned int clocks)
@@ -969,6 +975,17 @@local_irq_restore (flags); } +/* Used by NO_IDLE_HZ to skip ticks on idle CPUs */ +void reprogram_apic_timer (unsigned long count) +{ +unsigned long flags; + + count = count * apic_timer_val; +local_irq_save (flags); +apic_write_around (APIC_TMICT , count); +local_irq_restore (flags); +} +/* * In this function we calibrate APIC bus clocks to the external * timer. Unfortunately we cannot use jiffies and the timer irq
@@ -1064,6 +1081,9 @@*/ setup_APIC_timer (calibration_result); +setup_dyn_tick_use_apic (calibration_result); +set_dyn_tick_limits ((0xFFFFFFFF / calibration_result) *APIC_DIVISOR ,0 ); +local_irq_restore (flags); }
@@ -1202,6 +1222,9 @@* interrupt lock, which is the WrongThing (tm) to do. */ irq_enter (); + +dyn_tick_interrupt (regs); +smp_local_timer_interrupt (regs);irq_exit (); }
@@ -1214,6 +1237,9 @@unsigned long v;irq_enter (); + +dyn_tick_interrupt (regs); +/* * Check if this really is a spurious interrupt and ACK it * if it is a vectored one. Just in case...
@@ -1238,6 +1264,9 @@unsigned long v, v1;irq_enter (); + +dyn_tick_interrupt (regs); +/* First tickle the hardware, only then report what went on. -- REW */ v =apic_read (APIC_ESR );apic_write (APIC_ESR ,0 );
arch/i386/kernel/dyn-tick.c new file +340 -0
340 lines not on this page · raw message
arch/i386/kernel/io_apic.c +23 -3
@@ -32,6 +32,7 @@#include <linux/acpi.h> #include <linux/module.h> #include <linux/sysdev.h> +#include <linux/dyn-tick.h> #include <asm/io.h> #include <asm/smp.h>
@@ -1179,6 +1180,7 @@static struct hw_interrupt_type ioapic_level_type;static struct hw_interrupt_type ioapic_edge_type; +static struct hw_interrupt_type ioapic_edge_type_irq0;#define IOAPIC_AUTO -1#define IOAPIC_EDGE 0
@@ -1190,15 +1192,19 @@if ((trigger ==IOAPIC_AUTO &&IO_APIC_irq_trigger (irq)) || trigger ==IOAPIC_LEVEL ) irq_desc[vector].handler = &ioapic_level_type; -else +else if (vector) irq_desc[vector].handler = &ioapic_edge_type; +else + irq_desc[vector].handler = &ioapic_edge_type_irq0;set_intr_gate (vector, interrupt[vector]); }else {if ((trigger ==IOAPIC_AUTO &&IO_APIC_irq_trigger (irq)) || trigger ==IOAPIC_LEVEL ) irq_desc[irq].handler = &ioapic_level_type; -else +else if (irq) irq_desc[irq].handler = &ioapic_edge_type; +else + irq_desc[irq].handler = &ioapic_edge_type_irq0;set_intr_gate (vector, interrupt[irq]); } }
@@ -1311,7 +1317,7 @@* The timer IRQ doesn't have to know that behind the * scene we have a 8259A-master in AEOI mode ... */ - irq_desc[0 ].handler = &ioapic_edge_type; + irq_desc[0 ].handler = &ioapic_edge_type_irq0;/* * Add it to the IO-APIC irq-routing table:
@@ -2088,6 +2094,20 @@#endif }; +/* Needed to disable PIT interrupts when all CPUs sleep */ +static struct hw_interrupt_type ioapic_edge_type_irq0__read_mostly = { + .typename ="IO-APIC-edge-irq0" , + .startup = startup_edge_ioapic, + .shutdown = shutdown_edge_ioapic, + .enable = unmask_IO_APIC_irq, + .disable = mask_IO_APIC_irq, + .ack = ack_edge_ioapic, + .end = end_edge_ioapic, +#ifdef CONFIG_SMP + .set_affinity = set_ioapic_affinity, +#endif +}; +static inline void init_IO_APIC_traps (void ) {int irq;
arch/i386/kernel/irq.c +4 -0
@@ -18,6 +18,8 @@#include <linux/notifier.h> #include <linux/cpu.h> #include <linux/delay.h> +#include <linux/dyn-tick.h> +#include <asm/dyn-tick.h> DEFINE_PER_CPU(irq_cpustat_t, irq_stat) ____cacheline_maxaligned_in_smp;EXPORT_PER_CPU_SYMBOL (irq_stat);
@@ -76,6 +78,8 @@ }#endif +dyn_tick_interrupt (regs); +#ifdef CONFIG_4KSTACKS curctx = (union irq_ctx *)current_thread_info ();
arch/i386/kernel/Makefile +1 -0
@@ -32,6 +32,7 @@ obj-y += sysenter.o vsyscall.o obj-$(CONFIG_ACPI_SRAT) += srat.o obj-$(CONFIG_HPET_TIMER) += time_hpet.o +obj-$(CONFIG_NO_IDLE_HZ) += dyn-tick.o obj-$(CONFIG_EFI) += efi.o efi_stub.o obj-$(CONFIG_EARLY_PRINTK) += early_printk.o
arch/i386/kernel/process.c +4 -0
@@ -39,6 +39,7 @@#include <linux/ptrace.h> #include <linux/random.h> #include <linux/kprobes.h> +#include <linux/dyn-tick.h> #include <asm/uaccess.h> #include <asm/pgtable.h>
@@ -56,6 +57,7 @@#include <asm/tlbflush.h> #include <asm/cpu.h> +#include <asm/dyn-tick.h> asmlinkage void ret_from_fork(void) __asm__("ret_from_fork");
@@ -194,6 +196,8 @@play_dead ();__get_cpu_var (irq_stat).idle_timestamp = jiffies; +idle_reprogram_timer (); +idle (); }preempt_enable_no_resched ();
arch/i386/kernel/smp.c +9 -0
@@ -20,9 +20,11 @@#include <linux/interrupt.h> #include <linux/cpu.h> #include <linux/module.h> +#include <linux/dyn-tick.h> #include <asm/mtrr.h> #include <asm/tlbflush.h> +#include <asm/dyn-tick.h> #include <mach_apic.h> /*
@@ -313,6 +315,8 @@ {unsigned long cpu; +dyn_tick_interrupt (regs); + cpu =get_cpu ();if (!cpu_isset (cpu, flush_cpumask))
@@ -600,6 +604,8 @@ fastcall void smp_reschedule_interrupt(struct pt_regs *regs) {ack_APIC_irq (); + +dyn_tick_interrupt (regs); } fastcall void smp_call_function_interrupt(struct pt_regs *regs)
@@ -609,6 +615,9 @@int wait = call_data->wait;ack_APIC_irq (); + +dyn_tick_interrupt (regs); +/* * Notify initiating CPU that I've grabbed the data and am * about to execute the function
arch/i386/kernel/time.c +13 -6
@@ -46,6 +46,7 @@#include <linux/bcd.h> #include <linux/efi.h> #include <linux/mca.h> +#include <linux/dyn-tick.h> #include <asm/io.h> #include <asm/smp.h>
@@ -56,6 +57,7 @@#include <asm/uaccess.h> #include <asm/processor.h> #include <asm/timer.h> +#include <asm/dyn-tick.h> #include "mach_time.h"
@@ -245,7 +247,7 @@* timer_interrupt() needs to keep up the real-time clock, * as well as call the "do_timer()" routine every clocktick */ -static inline void do_timer_interrupt (int irq,struct pt_regs *regs) +static inline void do_timer_interrupt (int irq,struct pt_regs *regs,int lost) {#ifdef CONFIG_X86_IO_APIC if (timer_ack) {
@@ -263,7 +265,8 @@ }#endif -do_timer_interrupt_hook (regs); +if (!dyn_tick_enabled () || lost) +do_timer_interrupt_hook (regs);if (MCA_bus) {
@@ -288,6 +291,8 @@*/ irqreturn_t timer_interrupt (int irq,void *dev_id,struct pt_regs *regs) { +int lost; +/* * Here we are in the timer irq handler. We just have irqs locally * disabled but we don't know if the timer_bh is running on the other
@@ -297,9 +302,9 @@*/ write_seqlock (&xtime_lock); - cur_timer->mark_offset (); - -do_timer_interrupt (irq, regs); + lost = cur_timer->mark_offset (); + +do_timer_interrupt (irq, regs, lost);write_sequnlock (&xtime_lock);return IRQ_HANDLED ;
@@ -430,7 +435,7 @@/* XXX this driverfs stuff should probably go elsewhere later -john */ -static struct sys_device device_timer = { +struct sys_device device_timer = { .id =0 , .cls = &timer_sysclass, };
@@ -486,5 +491,7 @@ cur_timer =select_timer ();printk (KERN_INFO "Using %s for high-res timesource\n" ,cur_timer->name); +dyn_tick_time_init (cur_timer); +time_init_hook (); }
arch/i386/kernel/timers/timer_pit.c +34 -1
@@ -32,9 +32,10 @@return 0 ; } -static void mark_offset_pit (void ) +static int mark_offset_pit (void ) {/* nothing needed */ +return 1 ; }static unsigned long long monotonic_clock_pit (void )
@@ -148,6 +149,38 @@return count; } +void disable_pit_timer (void ) +{ + irq_desc[0 ].handler->disable (0 ); +} + +void enable_pit_timer (void ) +{ + irq_desc[0 ].handler->enable (0 ); +} + +/* +* Reprograms the next timer interrupt +* PIT timer reprogramming code taken from APM code. +* Note that PIT timer is a 16-bit timer, which allows max +* skip of only few seconds. +*/ +void reprogram_pit_timer (unsigned long jiffies_to_skip) +{ +int skip; +extern spinlock_t i8253_lock; +unsigned long flags; + + skip = jiffies_to_skip *LATCH ; +if (skip >0xffff ) + skip =0xffff ; + +spin_lock_irqsave (&i8253_lock, flags); +outb_p (0x34 ,PIT_MODE );/* binary, mode 2, LSB/MSB, ch 0 */ +outb_p (skip &0xff ,PIT_CH0 );/* LSB */ +outb (skip >>8 ,PIT_CH0 );/* MSB */ +spin_unlock_irqrestore (&i8253_lock, flags); +}/* tsc timer_opts struct */ struct timer_opts timer_pit = {
arch/i386/kernel/timers/timer_pm.c +15 -17
@@ -15,6 +15,7 @@#include <linux/module.h> #include <linux/device.h> #include <linux/init.h> +#include <linux/dyn-tick.h> #include <asm/types.h> #include <asm/timer.h> #include <asm/smp.h>
@@ -28,7 +29,7 @@#define PMTMR_TICKS_PER_SEC 3579545#define PMTMR_EXPECTED_RATE \ ((CALIBRATE_LATCH * (PMTMR_TICKS_PER_SEC >> 10)) / (CLOCK_TICK_RATE>>10)) - +#define PMTMR_TICKS_PER_JIFFY (PMTMR_EXPECTED_RATE / (CALIBRATE_LATCH/LATCH))/* The I/O port the PMTMR resides at. * The location is detected during setup_arch(),
@@ -127,7 +128,13 @@if (verify_pmtmr_rate () !=0 )return -ENODEV ; +printk ("Using %u PM timer ticks per jiffy \n" ,PMTMR_TICKS_PER_JIFFY ); + + offset_tick =read_pmtmr (); +setup_pit_timer (); +init_cpu_khz (); +set_dyn_tick_limits (((0xFFFFFF /1000000 ) *286 *HZ ) >>10 ,0 );return 0 ; }
@@ -148,10 +155,9 @@* this gets called during each timer interrupt * - Called while holding the writer xtime_lock */ -static void mark_offset_pmtmr (void ) +static int mark_offset_pmtmr (void ) {u32 lost, delta, last_offset; -static int first_run =1 ; last_offset = offset_tick;write_seqlock (&monotonic_lock);
@@ -161,29 +167,20 @@/* calculate tick interval */ delta = (offset_tick - last_offset) &ACPI_PM_MASK ; -/* convert to usecs */ - delta =cyc2us (delta); -/* update the monotonic base value */ - monotonic_base += delta *NSEC_PER_USEC ; + monotonic_base +=cyc2us (delta) *NSEC_PER_USEC ;write_sequnlock (&monotonic_lock);/* convert to ticks */ delta += offset_delay; - lost = delta / (USEC_PER_SEC /HZ ); - offset_delay = delta % (USEC_PER_SEC /HZ ); - + lost = delta /PMTMR_TICKS_PER_JIFFY ; + offset_delay = delta %PMTMR_TICKS_PER_JIFFY ;/* compensate for lost ticks */ if (lost >=2 ) jiffies_64 += lost -1 ; -/* don't calculate delay for first run, -or if we've got less then a tick */ -if (first_run || (lost <1 )) { - first_run =0 ; - offset_delay =0 ; - } +return lost; }static int pmtmr_resume (void )
@@ -192,6 +189,7 @@/* Assume this is the last mark offset time */ offset_tick =read_pmtmr ();write_sequnlock (&monotonic_lock); + offset_delay =0 ;return 0 ; }
@@ -243,7 +241,7 @@ now =read_pmtmr (); delta = (now - offset)&ACPI_PM_MASK ; -return (unsigned long ) offset_delay +cyc2us (delta); +return (unsigned long )cyc2us (delta + offset_delay); }
arch/i386/kernel/timers/timer_tsc.c +27 -35
142 lines not on this page · raw message
include/asm-i386/apic.h +2 -0
@@ -121,6 +121,7 @@extern int APIC_init_uniprocessor (void );extern void disable_APIC_timer (void );extern void enable_APIC_timer (void ); +extern void reprogram_apic_timer (unsigned long count);extern void enable_NMI_through_LVT0 (void * dummy);
@@ -134,6 +135,7 @@#else /* !CONFIG_X86_LOCAL_APIC */ static inline void lapic_shutdown (void ) { } +static inline void reprogram_apic_timer (unsigned long count) { }#endif /* !CONFIG_X86_LOCAL_APIC */
include/asm-i386/dyn-tick.h new file +76 -0
76 lines not on this page · raw message
include/asm-i386/timer.h +38 -1
@@ -1,5 +1,6 @@#ifndef _ASMi386_TIMER_H#define _ASMi386_TIMER_H +#include <linux/jiffies.h> #include <linux/init.h> #include <linux/pm.h>
@@ -19,7 +20,7 @@*/ struct timer_opts {char * name; -void (*mark_offset)(void ); +int (*mark_offset)(void );unsigned long (*get_offset)(void );unsigned long long (*monotonic_clock)(void );void (*delay)(unsigned long );
@@ -38,6 +39,9 @@extern struct timer_opts *__init select_timer (void );extern void clock_fallback (void );void setup_pit_timer (void ); +extern void disable_pit_timer (void ); +extern void enable_pit_timer (void ); +extern void reprogram_pit_timer (unsigned long jiffies_to_skip);/* Modifiers for buggy PIT handling */
@@ -67,4 +71,37 @@#ifdef CONFIG_X86_PM_TIMER extern struct init_timer_opts timer_pmtmr_init;#endif +#ifdef CONFIG_NO_IDLE_HZ +static inline void tsc_sanity_check (int lost) +{ +} +#else /* CONFIG_NO_IDLE_HZ */ +extern void cpufreq_delayed_get (void ); + +static inline void tsc_sanity_check (int lost) +{ +static int lost_count =0 ; + +if (lost >=2 ) { + jiffies_64 += lost-1 ; + +/* sanity check to ensure we're not always losing ticks */ +if (lost_count++ >100 ) { +printk (KERN_WARNING "Losing too many ticks!\n" ); +printk (KERN_WARNING "TSC cannot be used as a timesource. \n" ); +printk (KERN_WARNING "Possible reasons for this are:\n" ); +printk (KERN_WARNING " You're running with Speedstep,\n" ); +printk (KERN_WARNING " You don't have DMA enabled for your hard disk (see hdparm),\n" ); +printk (KERN_WARNING " Incorrect TSC synchronization on an SMP system (see dmesg).\n" ); +printk (KERN_WARNING "Falling back to a sane timesource now.\n" ); + +clock_fallback (); + } +/* ... but give the TSC a fair chance */ +if (lost_count >25 ) +cpufreq_delayed_get (); + }else + lost_count =0 ; +} +#endif /* CONFIG_NO_IDLE_HZ */ #endif
arch/i386/kernel/timers/timer_cyclone.c +3 -1
@@ -45,7 +45,7 @@ }while (high != cyclone_timer[1 ]); -static void mark_offset_cyclone (void ) +static int mark_offset_cyclone (void ) {unsigned long lost, delay;unsigned long delta = last_cyclone_low;
@@ -101,6 +101,8 @@*/ if (lost &&abs (delay - delay_at_last_interrupt) > (900000 /HZ )) jiffies_64++; + +return lost; }static unsigned long get_offset_cyclone (void )
arch/i386/kernel/timers/timer_hpet.c +3 -1
@@ -101,7 +101,7 @@return edx; } -static void mark_offset_hpet (void ) +static int mark_offset_hpet (void ) {unsigned long long this_offset, last_offset;unsigned long offset;
@@ -124,6 +124,8 @@ this_offset = ((unsigned long long )last_tsc_high<<32 )|last_tsc_low; monotonic_base +=cycles_2_ns (this_offset - last_offset);write_sequnlock (&monotonic_lock); + +return 1 ; }static void delay_hpet (unsigned long loops)
arch/i386/kernel/timers/timer_none.c +2 -1
@@ -1,9 +1,10 @@#include <linux/init.h> #include <asm/timer.h> -static void mark_offset_none (void ) +static int mark_offset_none (void ) {/* nothing needed */ +return 1 ; }static unsigned long get_offset_none (void )
kernel/timer_top.c new file +234 -0
234 lines not on this page · raw message
lib/Kconfig.debug +13 -0
@@ -77,6 +77,19 @@ application, you can say N to avoid the very slight overhead this adds. +config TIMER_INFO + bool "Collect kernel timers statistics" + depends on DEBUG_KERNEL && PROC_FS && NO_IDLE_HZ + help + If you say Y here, additional code will be inserted into the + timer routines to collect statistics about kernel timers being + reprogrammed through dynamic ticks feature. The statistics + will be provided in /proc/timer_info and the behavior of this + feature can be controlled through /proc/timer_input. + The goal is to offer some output to let user applications show + timer pattern usage and allow some tuning in them to + maximise idle time. + config DEBUG_SLAB bool "Debug memory allocations" depends on DEBUG_KERNEL
drivers/acpi/processor_idle.c +70 -7
131 lines not on this page · raw message
include/acpi/processor.h +3 -0
@@ -51,6 +51,7 @@u32 latency_ticks;u32 power;u32 usage; +u64 time;struct acpi_processor_cx_policy promotion;struct acpi_processor_cx_policy demotion; };
@@ -60,6 +61,8 @@unsigned long bm_check_timestamp;u32 default_state;u32 bm_activity; +u16 last_sleep; +u16 quick_promotion;int count;struct acpi_processor_cx states[ACPI_PROCESSOR_MAX_POWER ];
drivers/input/serio/i8042.h +1 -1
@@ -44,7 +44,7 @@* polling. */ -#define I8042_POLL_PERIOD HZ/20 +#define I8042_POLL_PERIOD HZ/5/* * Status register bits.
I've been doing some extended and more careful measurements of power consumption with and without dynticks, and it appears that at least on my Thinkpad T40p laptop (1.6 GHz Pentium-M, model #2373G1U), dyntick isn't helping the power consumption by any appreciable amount. It's not hurting, but it's not helping.
I believe the reason why is that the T40 has an extra C state which only shows up if you are running on battery; if you are running on the AC mains, C4 disappears:
*C1: type[C1] promotion[C2] demotion[--] latency[001] usage[00000000] time[00000000000000000000]
C2: type[C2] promotion[C3] demotion[C1] latency[001] usage[00000000] time[00000000000000000000]
C3: type[C3] promotion[C4] demotion[C2] latency[085] usage[00000000] time[00000000000000000000]
C4: type[C3] promotion[--] demotion[C3] latency[185] usage[00000000] time[00000000000000000000]With dyntick enabled, the laptop never enters the C4 state, but instead bounces back and forth between C2 and C3 (and I notice that we never enter C1 state, even when the CPU is completely pegged, but that's true with or without dyntick).
If dyntick is enabled, the laptop enters C4 state, which presumably is a deeper, more power saving state, and it appears power saving effects of dyntick is getting balanced off against the fact that C4 is never getting entered when it is enabled.
Looking at acpi/processor_idle.c, there is all sorts of magic special cases code for the C2 and C3 states (both for promotion/demotion polcies, as well as what to do when idling in those particular states), and which doesn't exist for other states, such as C4. Presumably this explains why we are only never entering C1, and why dyntick enabled C4 never gets reached. What I don't understand is _why_ all of the magic is present for those two states, but not for any of the others.
For future work when I have time, is to actually do some performance benchmarks; given that the power consumption doesn't appear to be changed either way with dyntick enabled or disabled, does the time needed to compile a kernel change significantly with or without dyntick?
- Ted
Quoted 4 lines · Theodore Ts'o
With dyntick enabled, the laptop never enters the C4 state, but instead bounces back and forth between C2 and C3 (and I notice that we never enter C1 state, even when the CPU is completely pegged, but that's true with or without dyntick).
C1 is a power-saving mode - if your CPU is pegged, you won't be in any C state (the acpi code seems to make this confusing by showing you the last mode you were in - however, the usage count won't change). If the C2 latency isn't significantly larger than the C1 latency, I don't think there's ever any reason to want to use C1.
To further confuse things, the ipw2100 driver will silently disable anything higher than C2 if it ever receives a corrupt packet. I'm not convinced this is a good thing.
Signature · 1 line
Matthew Garrett | [email protected]
Quoted 6 lines · Theodore Ts'o
With dyntick enabled, the laptop never enters the C4 state, but instead bounces back and forth between C2 and C3 (and I notice that we never enter C1 state, even when the CPU is completely pegged, but that's true with or without dyntick).
If dyntick is enabled, the laptop enters C4 state, which presumably is
^^^^^^^ this should be "disabled"
Quoted 3 lines
a deeper, more power saving state, and it appears power saving effects of dyntick is getting balanced off against the fact that C4 is never getting entered when it is enabled.
My apologies for any confusion, and for not doing better copy-editing before hitting the send butter.
-Ted
Hi,
Quoted 4 lines · Theodore Ts'o
Looking at
acpi/processor_idle.c, there is all sorts of magic special cases code for the C2 and C3 states (both for promotion/demotion polcies, as well as what to do when idling in those particular states), and which doesn't exist for other states, such as C4.
The magic is for C2- and C3-_type_ sleep, and C4 is of type C3... I know, this numbering is very confusing, but that's the way it is... However, the sepcially added "magic" for dyntick still needs some tweaking; I'll try to improve (and hopefully streamline) it a bit. As a first test, could you remove the "Fast-path demotion" case and re-test?
Thanks,
Dominik
Hi!
Quoted 3 lines
I believe the reason why is that the T40 has an extra C state which only shows up if you are running on battery; if you are running on the AC mains, C4 disappears:
Stupid IBM. I've seen it appearing/disappearing, but did not work out when.
No-C4-on-AC is bad -- if you just disconnect AC and walk away, you are running without benefits of C4. Bad. Changing benchmarks depending on you booting on AC or battery also look nasty.
Quoted 10 lines
*C1: type[C1] promotion[C2] demotion[--] latency[001] usage[00000000] time[00000000000000000000] C2: type[C2] promotion[C3] demotion[C1] latency[001] usage[00000000] time[00000000000000000000] C3: type[C3] promotion[C4] demotion[C2] latency[085] usage[00000000] time[00000000000000000000] C4: type[C3] promotion[--] demotion[C3] latency[185] usage[00000000] time[00000000000000000000]With dyntick enabled, the laptop never enters the C4 state, but instead bounces back and forth between C2 and C3 (and I notice that we never enter C1 state, even when the CPU is completely pegged, but that's true with or without dyntick).
C1 is halt. If your cpu is fully loaded, you don't want to enter any sleep state, not even C1.
Quoted 4 lines
If dyntick is enabled, the laptop enters C4 state, which presumably is a deeper, more power saving state, and it appears power saving effects of dyntick is getting balanced off against the fact that C4 is never getting entered when it is enabled.
Can you boot on AC power, then go to battery power to verify this theory?
Quoted 9 lines
Looking at
acpi/processor_idle.c, there is all sorts of magic special cases code for the C2 and C3 states (both for promotion/demotion polcies, as well as what to do when idling in those particular states), and which doesn't exist for other states, such as C4. Presumably this explains why we are only never entering C1, and why dyntick enabled C4 never gets reached. What I don't understand is _why_ all of the magic is present for those two states, but not for any of the others.
Quoted 5 lines
For future work when I have time, is to actually do some performance benchmarks; given that the power consumption doesn't appear to be changed either way with dyntick enabled or disabled, does the time needed to compile a kernel change significantly with or without dyntick?
It should not change at all. If your cpu is loaded, timers should tick as usual, so that you can properly account user/system times of processes etc.
Pavel
Signature · 1 line
Thanks, Sharp!
Quoted 6 lines · Pavel Machek
Stupid IBM. I've seen it appearing/disappearing, but did not work out when.
No-C4-on-AC is bad -- if you just disconnect AC and walk away, you are running without benefits of C4. Bad. Changing benchmarks depending on you booting on AC or battery also look nasty.
The moment you disconnect AC, it C4 automagically appears. When you reconnect to the AC mains, the C4 state disappears again, at least from the listing displayed by /proc/acpi/processor/CPU0/power. So the first issue you brought up isn't a problem.
The fact that it could change the benchmarks results depending on whether or you're running on battery or not is unfortunate, but in the real world does it matter?
More of an issue is that there are times when the laptop might think that it's running of the AC mains, but in fact the owner may have connected an external battery, and might _want_ the system to be as frugal as possible with the power.
Quoted 6 lines
Quoted 4 lines
If dyntick is enabled, the laptop enters C4 state, which presumably is a deeper, more power saving state, and it appears power saving effects of dyntick is getting balanced off against the fact that C4 is never getting entered when it is enabled.
Can you boot on AC power, then go to battery power to verify this theory?
Yep. As I said, at least on my laptop, this is what /proc/acpi/processor/CPU/power looks like when I am running on the AC mains:
active state: C2
max_cstate: C8
bus master activity: e7d24db4
states:
C1: type[C1] promotion[C2] demotion[--] latency[001] usage[00000010] time[00000000000000000000]
*C2: type[C2] promotion[C3] demotion[C1] latency[001] usage[00084296] time[00000000000306642441]
C3: type[C3] promotion[--] demotion[C2] latency[085] usage[00000000] time[00000000000000000000]... and this is what it looks like when I am on battery:
active state: C2
max_cstate: C8
bus master activity: 6ef5bbbd
states:
C1: type[C1] promotion[C2] demotion[--] latency[001] usage[00000010] time[00000000000000000000]
*C2: type[C2] promotion[C3] demotion[C1] latency[001] usage[00001162] time[00000000000004425803]
C3: type[C3] promotion[C4] demotion[C2] latency[085] usage[00000000] time[00000000000000000000]
C4: type[C3] promotion[--] demotion[C3] latency[185] usage[00000000] time[00000000000000000000]Whether I boot from AC power battery seems to be immaterial; what seems to matter is whether or not the laptop is running on battery at the moment that /proc/acpi/processor/CPU0/power is sampled.
- Ted
Quoted 4 lines · Theodore Ts'o
Quoted 3 lines · Pavel Machek
Stupid IBM. I've seen it appearing/disappearing, but did not work out when.
Quoted 8 lines
Quoted 3 lines
No-C4-on-AC is bad -- if you just disconnect AC and walk away, you are running without benefits of C4. Bad. Changing benchmarks depending on you booting on AC or battery also look nasty.
The moment you disconnect AC, it C4 automagically appears. When you reconnect to the AC mains, the C4 state disappears again, at least from the listing displayed by
/proc/acpi/processor/CPU0/power. So the first issue you brought up isn't a problem.
It does not seem to work like that here. I'm not sure what the exact rules are, but I know that I sometimes have C4 and sometimes not. I have C4 now, and it is used, even when I'm on AC power. Thinkpad X32.
Quoted 4 lines
More of an issue is that there are times when the laptop might think that it's running of the AC mains, but in fact the owner may have connected an external battery, and might _want_ the system to be as frugal as possible with the power.
Yes... OTOH it is not that bad -- we could simply hardcode data from acpi BIOS into kernel, ignore what ACPI says, and drive C4 even on AC power. But that would be hack (tm).
Quoted 3 lines
Whether I boot from AC power battery seems to be immaterial; what seems to matter is whether or not the laptop is running on battery at the moment that
/proc/acpi/processor/CPU0/poweris sampled.
I did not realize ACPI was _that_ clever. Anyway, it is different on my machine...
Pavel
Signature · 1 line
Thanks, Sharp!
Quoted 14 lines · Pavel Machek
Quoted 4 lines · Theodore Ts'o
Quoted 3 lines · Pavel Machek
Stupid IBM. I've seen it appearing/disappearing, but did not work out when.
Quoted 8 lines
Quoted 3 lines
No-C4-on-AC is bad -- if you just disconnect AC and walk away, you are running without benefits of C4. Bad. Changing benchmarks depending on you booting on AC or battery also look nasty.
The moment you disconnect AC, it C4 automagically appears. When you reconnect to the AC mains, the C4 state disappears again, at least from the listing displayed by
/proc/acpi/processor/CPU0/power. So the first issue you brought up isn't a problem.
Quoted 4 lines
It does not seem to work like that here. I'm not sure what the exact rules are, but I know that I sometimes have C4 and sometimes not. I have C4 now, and it is used, even when I'm on AC power. Thinkpad X32.
Well, today it _does_ behave like Theodore described (slightly different kernel, and I'm using power supply, not docking station). Strange.
Pavel
Signature · 1 line
Thanks, Sharp!
C4 strangeness [was Re: [PATCH] i386 No Idle HZ aka dynticks 051221]
Quoted 15 lines · Pavel Machek
Quoted 14 lines · Pavel Machek
Quoted 12 lines · Theodore Ts'o
Quoted 6 lines · Pavel Machek
Stupid IBM. I've seen it appearing/disappearing, but did not work out when.
No-C4-on-AC is bad -- if you just disconnect AC and walk away, you are running without benefits of C4. Bad. Changing benchmarks depending on you booting on AC or battery also look nasty.
The moment you disconnect AC, it C4 automagically appears. When you reconnect to the AC mains, the C4 state disappears again, at least from the listing displayed by
/proc/acpi/processor/CPU0/power. So the first issue you brought up isn't a problem.
Quoted 8 lines
Quoted 4 lines
It does not seem to work like that here. I'm not sure what the exact rules are, but I know that I sometimes have C4 and sometimes not. I have C4 now, and it is used, even when I'm on AC power. Thinkpad X32.
Well, today it _does_ behave like Theodore described (slightly different kernel, and I'm using power supply, not docking station). Strange.
So... I guess I found out what is going on.
When power is unplugged, X32 adds C4 state. When power is plugged, X32 removes C4 state (behaviour Ted seen). When I load ipw2200, this behaviour stops, and I see everything up-to C4. Strange. I remember ipw had some problems with C3 and C4, perhaps this is related?
Pavel
Signature · 1 line
Thanks, Sharp!
C4 non-strangeness [was: C4 strangeness [was Re: [PATCH] i386 No Idle HZ aka dynticks 051221]
Hi,
Quoted 6 lines · Pavel Machek
So... I guess I found out what is going on.
When power is unplugged, X32 adds C4 state. When power is plugged, X32 removes C4 state (behaviour Ted seen). When I load ipw2200, this behaviour stops, and I see everything up-to C4. Strange. I remember ipw had some problems with C3 and C4, perhaps this is related?
Nothing strange at all. The C-States are exported by the BIOS to the OS using the _CST method/object/whatever. This can change on runtime. When the BIOS recognizes it is on battery power, the _CST contains the C4 state, if it is on AC power, the _CST doesn't contain it. The ACPI code follows what it is told by the BIOS, for it has no chance to know about this additional C-State if on AC power, and it wouldn't be wise to second-guess the BIOS.
Ipw does limit the max_cstate setting dynamically if it recognizes problems; however I haven't seen _any_ such things lately on my own system. Might be related to dyntick being _enabled_, though ;-)
Dominik
Re: C4 non-strangeness [was: C4 strangeness [was Re: [PATCH] i386 No Idle HZ aka dynticks 051221]
Quoted 17 lines · Dominik Brodowski
Hi,
Quoted 6 lines · Pavel Machek
So... I guess I found out what is going on.
When power is unplugged, X32 adds C4 state. When power is plugged, X32 removes C4 state (behaviour Ted seen). When I load ipw2200, this behaviour stops, and I see everything up-to C4. Strange. I remember ipw had some problems with C3 and C4, perhaps this is related?
Nothing strange at all. The C-States are exported by the BIOS to the OS using the
_CSTmethod/object/whatever. This can change on runtime. When the BIOS recognizes it is on battery power, the_CSTcontains the C4 state, if it is on AC power, the_CSTdoesn't contain it. The ACPI code follows what it is told by the BIOS, for it has no chance to know about this additional C-State if on AC power, and it wouldn't be wise to second-guess the BIOS.
Quoted 3 lines
Ipw does limit the
max_cstatesetting dynamically if it recognizes problems; however I haven't seen _any_ such things lately on my own system. Might be related to dyntick being _enabled_, though ;-)
This was without dynticks... But why C4 availability no longer changes (between AC and battery power) with ipw2200 loaded? I'd understand higher C states being unavailable...
Pavel
Signature · 1 line
Thanks, Sharp!
Hi,
Quoted 4 lines · Theodore Ts'o
With dyntick enabled, the laptop never enters the C4 state, but instead bounces back and forth between C2 and C3 (and I notice that we never enter C1 state, even when the CPU is completely pegged, but that's true with or without dyntick).
Could you try out this patch on top of it? It also adds some debug output to /proc/acpi/processor/CPU0/power which I'd be interested in during normal usage on your notebook.
Hope this helps a bit...
Dominik
PS: this patch isn't yet in mergeable shape...
drivers/acpi/processor_idle.c +70 -26
@@ -61,7 +61,7 @@ module_param(max_cstate, uint, 0644);static unsigned int nocst =0 ;module_param (nocst, uint,0000 ); -#define BM_JIFFIES (HZ >= 800 ? 4 : 2) +#define BM_JIFFIES (HZ >= 800 ? 2 : 1)/* * bm_history -- bit-mask with a bit per jiffy of bus-master activity
@@ -181,6 +181,14 @@ static void acpi_safe_halt(void)static atomic_t c3_cpu_count; +u32 super_fastpath; +u32 fastpath; +u32 fastpath_demotion; +u32 kickback; +u32 promotion; +u32 demotion; +u32 bm_demotion; +static void acpi_processor_idle (void ) {struct acpi_processor *pr =NULL ;
@@ -237,7 +245,6 @@ static void acpi_processor_idle(void) pr->power.bm_activity |=0x1 ; pr->power.bm_activity <<=1 ; } -acpi_get_register (ACPI_BITREG_BUS_MASTER_STATUS , &bm_status,ACPI_MTX_DO_NOT_LOCK );if (bm_status) {
@@ -273,11 +280,12 @@ static void acpi_processor_idle(void)if ((pr->power.bm_activity &0x01 ) && cx->demotion.threshold.bm) {local_irq_enable (); +if (!pr->power.pre_bm_state) + pr->power.pre_bm_state = cx; + bm_demotion++; next_state = cx->demotion.state;if (dyn_tick_enabled ()) -dyn_early_reprogram (BM_JIFFIES ); -/* do not promote in fast-path */ - pr->power.last_sleep =0 ; +dyn_early_reprogram (1 ); goto end; } }
@@ -287,38 +295,64 @@ static void acpi_processor_idle(void)*/ if (dyn_tick_enabled ()) {/* -* Fast-path promotion: if we slept for more than 2 jiffies the -* last time, and we're intended to sleep for more than 2 -* jiffies now, promote. Note that the processor won't enter a -* low-power state during this call (to this funciton) but -* should upon the next. +* Fast-path promotion: promote to higher state if +* - we can promote +* - we did sleep at least "threshold" pm_timer ticks last time +* - there is no bm_activity this tick +* - we slept more than BM_JIFFIES ticks last time +* - we're scheduled to sleep at least BM_JIFFIES ticks now */ -if (cx->promotion.state && cx->promotion.count && -dyn_tick_skipping () && ((pr->power.last_sleep > - (BM_JIFFIES * cx->promotion.threshold.ticks)) || - (pr->power.quick_promotion ==1 )) && - (dyn_tick_current_skip () >BM_JIFFIES )) { +if (cx->promotion.state && + cx->promotion.count && + !(pr->power.bm_activity &0x01 ) && + (pr->power.last_sleep >BM_JIFFIES ) && + (dyn_tick_current_skip () >=BM_JIFFIES )) {local_irq_enable (); next_state = cx->promotion.state; +/* +* Super-fast-path: promote to next higher state if +* - we can promote +* - we did sleep longer than 2 * BM_JIFFIES times last time +*/ +if (next_state->promotion.state && + ((cx->promotion.threshold.bm && next_state->promotion.threshold.bm) || + (!cx->promotion.threshold.bm && !next_state->promotion.threshold.bm)) && + (pr->power.last_sleep > (2 *BM_JIFFIES ))) { + next_state = next_state->promotion.state; + super_fastpath++; + }else + fastpath++; goto end; } +/* +* Kick-back promotion: promote to C-State used before bm activity was detected if +* - we have a pre-bm-state +* - we had bus mastering activity last jiffy, but not this jiffy +* - we're scheduled to sleep at least BM_JIFFIES now +*/ +if (pr->power.pre_bm_state &&/* we have a pre-bm-state */ + ((pr->power.bm_activity &0x11 ) ==0x10 ) &&/* we had bm activity last jiffy, not this jiffy */ + (dyn_tick_current_skip () >=BM_JIFFIES )) { +local_irq_enable (); + next_state = pr->power.pre_bm_state; + pr->power.pre_bm_state =NULL ; + kickback++; + goto end; + }/* * Fast-path demotion: if we're going to sleep for only one -* jiffy, and we'd enter C3-type sleep, and the wakeup latency -* is high, don't use C3-type sleep but (temporarily) C2. +* jiffy, and the wakeup latency is high, demote. */ -if (cx->demotion.state && cx->demotion.threshold.bm && - (dyn_tick_next_tick () < (jiffies +1 )) - && (cx->latency >150 )) { +if (cx->demotion.state && + (dyn_tick_next_tick () < (jiffies +1 )) && + (cx->latency >150 )) {local_irq_enable (); next_state = cx->demotion.state; -/* don't do a fast-path promotion next time... */ pr->power.last_sleep =0 ; -/* ... but thereafter. */ - pr->power.quick_promotion =2 ; + fastpath_demotion++; goto end; } }
@@ -446,9 +480,6 @@ static void acpi_processor_idle(void) next_state = pr->power.state; -if (pr->power.quick_promotion) - pr->power.quick_promotion--; -#ifdef CONFIG_HOTPLUG_CPU /* Don't do promotion/demotion */ if ((cx->type == ACPI_STATE_C1) && (num_online_cpus() > 1) &&
@@ -477,11 +508,13 @@ static void acpi_processor_idle(void)if (! (pr->power.bm_activity & cx-> promotion.threshold.bm)) { + promotion++; next_state = cx->promotion.state; goto end; } }else { + promotion++; next_state = cx->promotion.state; goto end; }
@@ -500,6 +533,7 @@ static void acpi_processor_idle(void) cx->demotion.count++; cx->promotion.count =0 ;if (cx->demotion.count >= cx->demotion.threshold.count) { + demotion++; next_state = cx->demotion.state; goto end; }
@@ -1019,6 +1053,16 @@ static int acpi_processor_power_seq_show pr->power.state ? pr->power.state - pr->power.states :0 , max_cstate, (unsigned)pr->power.bm_activity); +seq_printf (seq, +"promotion: %08d\n" +"demotion: %08d\n" +"bm_demotion: %08d\n" +"fastpath_demotion: %08d\n" +"fastpath: %08d\n" +"sfastpath: %08d\n" +"kickback: %08d\n" , + promotion, demotion, bm_demotion, fastpath_demotion, fastpath, super_fastpath, kickback); +seq_puts (seq,"states:\n" );for (i =1 ; i <= pr->power.count; i++) {
include/acpi/processor.h +1 -1
@@ -58,11 +58,11 @@ struct acpi_processor_cx {struct acpi_processor_power {struct acpi_processor_cx *state; +struct acpi_processor_cx *pre_bm_state;unsigned long bm_check_timestamp;u32 default_state;u32 bm_activity;u16 last_sleep; -u16 quick_promotion;int count;struct acpi_processor_cx states[ACPI_PROCESSOR_MAX_POWER ];
Quoted 11 lines · Dominik Brodowski
Hi,
Quoted 4 lines · Theodore Ts'o
With dyntick enabled, the laptop never enters the C4 state, but instead bounces back and forth between C2 and C3 (and I notice that we never enter C1 state, even when the CPU is completely pegged, but that's true with or without dyntick).
Could you try out this patch on top of it? It also adds some debug output to
/proc/acpi/processor/CPU0/powerwhich I'd be interested in during normal usage on your notebook.
Sure! Before I tried your patch, I first tried commenting out the fast demotion test, as you had suggested, but that had essentially no effect.
The patch which you sent does seem to help. Now with dynticks enabled, my T40p is now entering C4, and although I haven't done a careful controlled power test (that takes more time than I had at the time), I can say that in maximal power saving configuration (USB and sound drivers disabled and unloaded, network ifconfig'ed down, laptop_mode enabled, screen brightness turned all the way down, etc.) the minimal power utilization went from approximately 9900 mW/hr (dynticks disabled) to approximately 9500 mW/hr (dynticks enabled).
If I could keep the system at 9500 mW/hr continuously, that would translate to approximately 7.5 hours on the extended battery, which wouldn't be bad; unfortunately this doesn't take into account random things like cron waking up and driving the power utilization up, and it's not fair to do a power test when the laptop is not doing anything at all. A much more interesting number would be with my entire mailbox in cache, how long could I read and respond to email while on battery and with minimal disk usage.
Here's the status of /proc/acpi/processor/CPU/power after a number of tests, with and without dynticks enabled:
<tytso@think> {/home/tytso}, level 3
515% cat /proc/acpi/processor/CPU/power
active state: C4
max_cstate: C8
bus master activity: 68890a90
promotion: 00002463
demotion: 00000621
bm_demotion: 00019522
fastpath_demotion: 00000214
fastpath: 00004495
sfastpath: 00001775
kickback: 00012525
states:
C1: type[C1] promotion[C2] demotion[--] latency[001] usage[00000010] time[00000000000000000000]
C2: type[C2] promotion[C3] demotion[C1] latency[001] usage[00002129] time[00000000000036902671]
C3: type[C3] promotion[C4] demotion[C2] latency[085] usage[00000430] time[00000000000003215582]
*C4: type[C3] promotion[--] demotion[C3] latency[185] usage[00001486] time[00000000000036966581]BTW, I noticed this time around at least on my T40p, when I go from on to off AC mains, and C4 disappears/appears, all of the ACPI C states usage/time counters reset; however, the promotion/demotion/ bm_demotion/fast_demotion/fastpath/sfastpath/kickback counters do _not_ reset. Probably not a serious issue, but I thought I should mention it in case the numbers looked inconsistent.
One other strange thing happened I noticed while testing your patch. One of the times when I was switching between on and off battery power, I ended up in a wierd state where I was on AC mains, but the bus master activity was reading something like fffffffe, and the asterick by the last active C state completely disappeared, presumably because the system wasn't entering sleep state at all. (And I from the OS level, I couldn't see any reason for this behavior; the disk wasn't active, and as near as I could tell the system was completely idle.) Scary, but I was only able to reproduce it once or twice, and the only times I saw it was when I was on AC power, so if in fact I was drawing extra power, the only thing that would have noticed would be the heatsink fan....
I'll keep an eye on it, but for now, your patch definitely seems to be helpful. I'll keep using it and see what else I might find.
Thanks, regards,
- Ted