64-bit sched_clock
Subsystems (9)
cover 64-bit sched_clock
This patch series replaces existing sched_clock() implementations with a version which gives a full 64-bit nanosecond time from counters with less than 33 bits.
This series depends on the prevously posted clocksource patch series, and the notrace sched_clock patch, and the ftrace tree previously merged into my devel-stable branch. It also depends on a fix to the clocksource code to achieve the accuracy quoted below - though it'll build and work without.
We achieve the 64-bit nanosecond time by maintaining a pair of epochs - one for the counter and a corresponding one for the ns value. This allows us to calculate how many counter ticks have happened since the ns epoch was last updated, and therefore how much time has passed.
We use a lockless method to read and update the epoch values, eliminating any locking overhead. The epoch update happens at about 90% of the wrap period, resulting in less maintainence overhead.
Lastly, we allow platforms to provide constants for the counter->ns conversion.
The PXA sched_clock implementation using cnt32_to_63() builds to 72 bytes of code and 12 bytes of data, and provides 54 bits of nanoseconds, with an error of 0.00008%.
The PXA sched_clock implementation using this infrastructure builds to 84 bytes of code and 8 bytes of data, and provides 64 bits of nanoseconds, with an error of 0.00000002%.
The epoch updates in action on Versatile Express (24MHz 32-bit counter):
sched_clock: update 68391420 -> 50e0a246 0ns -> 162637292211nssched_clock: update 50e0a246 -> 372c24b7 162637292211ns -> 323625136880nssched_clock: update 372c24b7 -> 1d35c46a 323625136880ns -> 484433069633nssched_clock: update 1d35c46a -> 0367a736 484433069633ns -> 645350944761ns
...sched_clock: update f6f7fd55 -> dd2d882e 32669955162290ns -> 32830883021959nssched_clock: update dd2d882e -> c36312cd 32830883021959ns -> 32991810879212nssched_clock: update c36312cd -> a9989d41 32991810879212ns -> 33152738734673nssched_clock: update a9989d41 -> 8fce2781 33152738734673ns -> 33313666587967ns
PATCH 1/13 ARM: sched_clock: provide common infrastructure for sched_clock()
Provide common sched_clock() infrastructure for platforms to use to create a 64-bit ns based sched_clock() implementation from a counter running at a non-variable clock rate.
This implementation is based upon maintaining an epoch for the counter and an epoch for the nanosecond time. When we desire a sched_clock() time, we calculate the number of counter ticks since the last epoch update, convert this to nanoseconds and add to the epoch nanoseconds.
We regularly refresh these epochs within the counter wrap interval. We perform a similar calculation as above, and store the new epochs.
We read and write the epochs in such a way that sched_clock() can easily (and locklessly) detect when an update is in progress, and repeat the loading of these constants when they're known not to be stable. The one caveat is that sched_clock() is not called in the middle of an update.
Finally, if the clock rate is known at compile time, the counter to ns conversion factors can be specified, allowing sched_clock() to be tightly optimized. We ensure that these factors are correct by providing an initialization function which performs a run-time check.
--- arch/arm/Kconfig | 3 + arch/arm/include/asm/sched_clock.h | 116 ++++++++++++++++++++++++++++++++++++ arch/arm/kernel/Makefile | 1 + arch/arm/kernel/sched_clock.c | 69 +++++++++++++++++++++ 4 files changed, 189 insertions(+), 0 deletions(-) create mode 100644 arch/arm/include/asm/sched_clock.h create mode 100644 arch/arm/kernel/sched_clock.c
arch/arm/Kconfig +3 -0
@@ -38,6 +38,9 @@ config HAVE_PWM config SYS_SUPPORTS_APM_EMULATION bool +config HAVE_SCHED_CLOCK + bool + config GENERIC_GPIO bool
arch/arm/include/asm/sched_clock.h new file +116 -0
116 lines not on this page · raw message
arch/arm/kernel/Makefile +1 -0
@@ -29,6 +29,7 @@ obj-$(CONFIG_MODULES) += armksyms.o module.o obj-$(CONFIG_ARTHUR) += arthur.o obj-$(CONFIG_ISA_DMA) += dma-isa.o obj-$(CONFIG_PCI) += bios32.o isa.o +obj-$(CONFIG_HAVE_SCHED_CLOCK) += sched_clock.o obj-$(CONFIG_SMP) += smp.o obj-$(CONFIG_HAVE_ARM_SCU) += smp_scu.o obj-$(CONFIG_HAVE_ARM_TWD) += smp_twd.o
arch/arm/kernel/sched_clock.c new file +69 -0
69 lines not on this page · raw message
Signature · 1 line
1.6.2.5
PATCH 1/13 ARM: sched_clock: provide common infrastructure for sched_clock()
Quoted 3 lines · Russell King - ARM Linux
Provide common
sched_clock()infrastructure for platforms to use to create a 64-bit ns basedsched_clock()implementation from a counter running at a non-variable clock rate.
Some of the comments in the patch were only relevant to a previous version. He's an updated version with proper comments to avoid confusion.
arch/arm/Kconfig | 3 + arch/arm/include/asm/sched_clock.h | 115 ++++++++++++++++++++++++++++++++++++ arch/arm/kernel/Makefile | 1 + arch/arm/kernel/sched_clock.c | 69 +++++++++++++++++++++ 4 files changed, 188 insertions(+), 0 deletions(-)
arch/arm/Kconfig +3 -0
@@ -38,6 +38,9 @@ config HAVE_PWM config SYS_SUPPORTS_APM_EMULATION bool +config HAVE_SCHED_CLOCK + bool + config GENERIC_GPIO bool
arch/arm/include/asm/sched_clock.h new file +115 -0
115 lines not on this page · raw message
arch/arm/kernel/Makefile +1 -0
@@ -29,6 +29,7 @@ obj-$(CONFIG_MODULES) += armksyms.o module.o obj-$(CONFIG_ARTHUR) += arthur.o obj-$(CONFIG_ISA_DMA) += dma-isa.o obj-$(CONFIG_PCI) += bios32.o isa.o +obj-$(CONFIG_HAVE_SCHED_CLOCK) += sched_clock.o obj-$(CONFIG_SMP) += smp.o obj-$(CONFIG_HAVE_ARM_SCU) += smp_scu.o obj-$(CONFIG_HAVE_ARM_TWD) += smp_twd.o
arch/arm/kernel/sched_clock.c new file +69 -0
69 lines not on this page · raw message
PATCH 2/13 ARM: ixp4xx: convert sched_clock() to use new infrastructure
In reply to Russell King - ARM LinuxConvert ixp4xx to use the new sched_clock() infrastructure for extending 32bit counters to full 64-bit nanoseconds.
--- arch/arm/Kconfig | 1 + arch/arm/mach-ixp4xx/common.c | 31 ++++++++++++++++++++----------- 2 files changed, 21 insertions(+), 11 deletions(-)
arch/arm/Kconfig +1 -0
@@ -437,6 +437,7 @@ config ARCH_IXP4XX select CPU_XSCALE select GENERIC_GPIO select GENERIC_CLOCKEVENTS + select HAVE_SCHED_CLOCK select DMABOUNCE if PCI help Support for Intel's IXP4XX (XScale) family of processors.
arch/arm/mach-ixp4xx/common.c +20 -11
@@ -35,6 +35,7 @@#include <asm/pgtable.h> #include <asm/page.h> #include <asm/irq.h> +#include <asm/sched_clock.h> #include <asm/mach/map.h> #include <asm/mach/irq.h>
@@ -399,6 +400,23 @@ void __init ixp4xx_sys_init(void) }/* +* sched_clock() +*/ +static DEFINE_CLOCK_DATA (cd); + +unsigned long long notrace sched_clock (void ) +{ +u32 cyc = *IXP4XX_OSTS ; +return cyc_to_sched_clock (&cd, cyc, (u32 )~0 ); +} + +static void notrace ixp4xx_update_sched_clock(void) +{ +u32 cyc = *IXP4XX_OSTS ; +update_sched_clock (&cd, cyc, (u32 )~0 ); +} + +/* * clocksource */ static cycle_t ixp4xx_get_cycles (struct clocksource *cs)
@@ -418,18 +436,9 @@ unsigned long ixp4xx_timer_freq = FREQ;EXPORT_SYMBOL (ixp4xx_timer_freq);static void __init ixp4xx_clocksource_init (void ) { -clocksource_register_hz (&clocksource_ixp4xx, ixp4xx_timer_freq); -} - -/* -* sched_clock() -*/ -unsigned long long notrace sched_clock (void ) -{ -cycle_t cyc =ixp4xx_get_cycles (NULL ); -struct clocksource *cs = &clocksource_ixp4xx; +init_sched_clock (&cd, ixp4xx_update_sched_clock,32 , ixp4xx_timer_freq); -return clocksource_cyc2ns (cyc, cs->mult, cs->shift); +clocksource_register_hz (&clocksource_ixp4xx, ixp4xx_timer_freq); }/*
Signature · 1 line
1.6.2.5
PATCH 3/13 ARM: mmp: convert sched_clock() to use new infrastructure
In reply to Russell King - ARM LinuxConvert mmp to use the new sched_clock() infrastructure for extending 32bit counters to full 64-bit nanoseconds.
--- arch/arm/Kconfig | 1 + arch/arm/mach-mmp/time.c | 32 +++++++++++--------------------- 2 files changed, 12 insertions(+), 21 deletions(-)
arch/arm/Kconfig +1 -0
@@ -513,6 +513,7 @@ config ARCH_MMP select ARCH_REQUIRE_GPIOLIB select COMMON_CLKDEV select GENERIC_CLOCKEVENTS + select HAVE_SCHED_CLOCK select TICK_ONESHOT select PLAT_PXA select SPARSE_IRQ
arch/arm/mach-mmp/time.c +11 -21
@@ -26,8 +26,8 @@#include <linux/io.h> #include <linux/irq.h> #include <linux/sched.h> -#include <linux/cnt32_to_63.h> +#include <asm/sched_clock.h> #include <mach/addr-map.h> #include <mach/regs-timers.h> #include <mach/regs-apbc.h>
@@ -42,23 +42,7 @@#define MAX_DELTA (0xfffffffe)#define MIN_DELTA (16) -#define TCR2NS_SCALE_FACTOR 10 - -static unsigned long tcr2ns_scale; - -static void __init set_tcr2ns_scale (unsigned long tcr_rate) -{ -unsigned long long v =1000000000ULL <<TCR2NS_SCALE_FACTOR ; -do_div (v, tcr_rate); - tcr2ns_scale = v; -/* -* We want an even value to automatically clear the top bit -* returned by cnt32_to_63() without an additional run time -* instruction. So if the LSB is 1 then round it up. -*/ -if (tcr2ns_scale &1 ) - tcr2ns_scale++; -} +static DEFINE_CLOCK_DATA (cd);/* * FIXME: the timer needs some delay to stablize the counter capture
@@ -77,8 +61,14 @@ static inline uint32_t timer_read(void)unsigned long long notrace sched_clock (void ) { -unsigned long long v =cnt32_to_63 (timer_read ()); -return (v * tcr2ns_scale) >>TCR2NS_SCALE_FACTOR ; +u32 cyc =timer_read (); +return cyc_to_sched_clock (&cd, cyc, (u32 )~0 ); +} + +static void notrace mmp_update_sched_clock(void) +{ +u32 cyc =timer_read (); +update_sched_clock (&cd, cyc, (u32 )~0 ); }static irqreturn_t timer_interrupt (int irq,void *dev_id)
@@ -185,7 +175,7 @@ void __init timer_init(int irq) {timer_config (); -set_tcr2ns_scale (CLOCK_TICK_RATE ); +init_sched_clock (&cd, mmp_update_sched_clock,32 ,CLOCK_TICK_RATE ); ckevt.mult =div_sc (CLOCK_TICK_RATE ,NSEC_PER_SEC , ckevt.shift); ckevt.max_delta_ns =clockevent_delta2ns (MAX_DELTA , &ckevt);
Signature · 1 line
1.6.2.5
PATCH 4/13 ARM: pxa: convert sched_clock() to use new infrastructure
In reply to Russell King - ARM LinuxConvert pxa to use the new sched_clock() infrastructure for extending 32bit counters to full 64-bit nanoseconds.
--- arch/arm/Kconfig | 1 + arch/arm/mach-pxa/time.c | 29 +++++++++-------------------- 2 files changed, 10 insertions(+), 20 deletions(-)
arch/arm/Kconfig +1 -0
@@ -593,6 +593,7 @@ config ARCH_PXA select COMMON_CLKDEV select ARCH_REQUIRE_GPIOLIB select GENERIC_CLOCKEVENTS + select HAVE_SCHED_CLOCK select TICK_ONESHOT select PLAT_PXA select SPARSE_IRQ
arch/arm/mach-pxa/time.c +9 -20
@@ -17,11 +17,11 @@#include <linux/interrupt.h> #include <linux/clockchips.h> #include <linux/sched.h> -#include <linux/cnt32_to_63.h> #include <asm/div64.h> #include <asm/mach/irq.h> #include <asm/mach/time.h> +#include <asm/sched_clock.h> #include <mach/regs-ost.h> /*
@@ -32,29 +32,18 @@* long as there is always less than 582 seconds between successive * calls to sched_clock() which should always be the case in practice. */ +static DEFINE_CLOCK_DATA (cd); -#define OSCR2NS_SCALE_FACTOR 10 - -static unsigned long oscr2ns_scale; - -static void __init set_oscr2ns_scale (unsigned long oscr_rate) +unsigned long long notrace sched_clock (void ) { -unsigned long long v =1000000000ULL <<OSCR2NS_SCALE_FACTOR ; -do_div (v, oscr_rate); - oscr2ns_scale = v; -/* -* We want an even value to automatically clear the top bit -* returned by cnt32_to_63() without an additional run time -* instruction. So if the LSB is 1 then round it up. -*/ -if (oscr2ns_scale &1 ) - oscr2ns_scale++; +u32 cyc =OSCR ; +return cyc_to_sched_clock (&cd, cyc, (u32 )~0 ); } -unsigned long long notrace sched_clock (void ) +static void notrace pxa_update_sched_clock(void) { -unsigned long long v =cnt32_to_63 (OSCR ); -return (v * oscr2ns_scale) >>OSCR2NS_SCALE_FACTOR ; +u32 cyc =OSCR ; +update_sched_clock (&cd, cyc, (u32 )~0 ); }
@@ -144,7 +133,7 @@ static void __init pxa_timer_init(void)OIER =0 ;OSSR =OSSR_M0 |OSSR_M1 |OSSR_M2 |OSSR_M3 ; -set_oscr2ns_scale (clock_tick_rate); +init_sched_clock (&cd, pxa_update_sched_clock,32 , clock_tick_rate); ckevt_pxa_osmr0.mult =div_sc (clock_tick_rate,NSEC_PER_SEC , ckevt_pxa_osmr0.shift);
Signature · 1 line
1.6.2.5
PATCH 4/13 ARM: pxa: convert sched_clock() to use new infrastructure
Hello Russell,
a very minor nit ...
arch/arm/mach-pxa/time.c line 40
Quoted 1 line · Russell King - ARM Linux
arch/arm/mach-pxa/time.c +1
-0+return cyc_to_sched_clock (&cd, cyc, (u32 )~0 );
The value of ~0 is compiler implementation defined, so in general ~((u32)0) is saver. (Though I admit that on arm ~0 will probably always be -1, still I think using the save construct doesn't hurt here.)
Best regards
Uwe
Signature · 2 lines
Pengutronix e.K. | Uwe Kleine-K?nig | Industrial Linux Solutions | http://www.pengutronix.de/ |
PATCH 4/13 ARM: pxa: convert sched_clock() to use new infrastructure
Quoted 9 lines · Uwe Kleine-K?nig
Hello Russell,
a very minor nit ...
arch/arm/mach-pxa/time.c line 40
Quoted 1 line · Russell King - ARM Linux
arch/arm/mach-pxa/time.c +1
-0+return cyc_to_sched_clock (&cd, cyc, (u32 )~0 );The value of ~0 is compiler implementation defined, so in general ~((u32)0) is saver. (Though I admit that on arm ~0 will probably always be -1, still I think using the save construct doesn't hurt here.)
I've been debating about using something like CLOCKSOURCE_MASK() here, but haven't yet decided it to be worth it. I also toyed with the idea of storing the mask in the struct, but that adds extra expense when it's not needed - everyone should know how many bits are in their counter at build time, and if not, they shouldn't penalize those who do.
PATCH 5/13 ARM: sa1100: convert sched_clock() to use new infrastructure
In reply to Russell King - ARM LinuxConvert sa1100 to use the new sched_clock() infrastructure for extending 32bit counters to full 64-bit nanoseconds.
--- arch/arm/Kconfig | 1 + arch/arm/mach-sa1100/generic.c | 23 ----------------------- arch/arm/mach-sa1100/time.c | 30 ++++++++++++++++++++++++++++++ 3 files changed, 31 insertions(+), 23 deletions(-)
arch/arm/Kconfig +1 -0
@@ -642,6 +642,7 @@ config ARCH_SA1100 select CPU_FREQ select GENERIC_CLOCKEVENTS select HAVE_CLK + select HAVE_SCHED_CLOCK select TICK_ONESHOT select ARCH_REQUIRE_GPIOLIB help
arch/arm/mach-sa1100/generic.c +0 -23
@@ -16,9 +16,7 @@#include <linux/pm.h> #include <linux/cpufreq.h> #include <linux/ioport.h> -#include <linux/sched.h> /* just for sched_clock() - funny that */ #include <linux/platform_device.h> -#include <linux/cnt32_to_63.h> #include <asm/div64.h> #include <mach/hardware.h>
@@ -110,27 +108,6 @@ unsigned int sa11x0_getspeed(unsigned int cpu) }/* -* This is the SA11x0 sched_clock implementation. This has -* a resolution of 271ns, and a maximum value of 32025597s (370 days). -* -* The return value is guaranteed to be monotonic in that range as -* long as there is always less than 582 seconds between successive -* calls to this function. -* -* ( * 1E9 / 3686400 => * 78125 / 288) -*/ -unsigned long long notrace sched_clock (void ) -{ -unsigned long long v =cnt32_to_63 (OSCR ); - -/* the <<1 gets rid of the cnt_32_to_63 top bit saving on a bic insn */ - v *=78125 <<1 ; -do_div (v,288 <<1 ); - -return v; -} - -/* * Default power-off for SA1100 */ static void sa1100_power_off (void )
arch/arm/mach-sa1100/time.c +30 -0
@@ -12,12 +12,39 @@#include <linux/errno.h> #include <linux/interrupt.h> #include <linux/irq.h> +#include <linux/sched.h> /* just for sched_clock() - funny that */ #include <linux/timex.h> #include <linux/clockchips.h> #include <asm/mach/time.h> +#include <asm/sched_clock.h> #include <mach/hardware.h> +/* +* This is the SA11x0 sched_clock implementation. +*/ +static DEFINE_CLOCK_DATA (cd); + +/* +* Constants generated by clocks_calc_mult_shift(m, s, 3.6864MHz, +* NSEC_PER_SEC, 60). +* This gives a resolution of about 271ns and a wrap period of about 19min. +*/ +#define SC_MULT 2275555556u +#define SC_SHIFT 23 + +unsigned long long notrace sched_clock (void ) +{ +u32 cyc =OSCR ; +return cyc_to_fixed_sched_clock (&cd, cyc, (u32 )~0 ,SC_MULT ,SC_SHIFT ); +} + +static void notrace sa1100_update_sched_clock(void) +{ +u32 cyc =OSCR ; +update_sched_clock (&cd, cyc, (u32 )~0 ); +} +#define MIN_OSCR_DELTA 2static irqreturn_t sa1100_ost0_interrupt (int irq,void *dev_id)
@@ -96,6 +123,9 @@ static void __init sa1100_timer_init(void)OIER =0 ;/* disable any timer interrupts */ OSSR =0xf ;/* clear status on all timers */ +init_fixed_sched_clock (&cd, sa1100_update_sched_clock,32 , +3686400 ,SC_MULT ,SC_SHIFT ); + ckevt_sa1100_osmr0.mult =div_sc (3686400 ,NSEC_PER_SEC , ckevt_sa1100_osmr0.shift); ckevt_sa1100_osmr0.max_delta_ns =
Signature · 1 line
1.6.2.5
PATCH 6/13 ARM: tegra: convert sched_clock() to use new infrastructure
In reply to Russell King - ARM LinuxConvert tegra to use the new sched_clock() infrastructure for extending 32bit counters to full 64-bit nanoseconds.
--- arch/arm/Kconfig | 1 + arch/arm/mach-tegra/timer.c | 22 ++++++++++++++++++++-- 2 files changed, 21 insertions(+), 2 deletions(-)
arch/arm/Kconfig +1 -0
@@ -570,6 +570,7 @@ config ARCH_TEGRA select GENERIC_CLOCKEVENTS select GENERIC_GPIO select HAVE_CLK + select HAVE_SCHED_CLOCK select COMMON_CLKDEV select ARCH_HAS_BARRIERS if CACHE_L2X0 select ARCH_HAS_CPUFREQ
arch/arm/mach-tegra/timer.c +20 -2
@@ -25,7 +25,6 @@#include <linux/clocksource.h> #include <linux/clk.h> #include <linux/io.h> -#include <linux/cnt32_to_63.h> #include <asm/mach/time.h> #include <asm/localtimer.h>
@@ -110,9 +109,25 @@ static struct clocksource tegra_clocksource = { .flags = CLOCK_SOURCE_IS_CONTINUOUS, }; +static DEFINE_CLOCK_DATA (cd); + +/* +* Constants generated by clocks_calc_mult_shift(m, s, 1MHz, NSEC_PER_SEC, 60). +* This gives a resolution of about 1us and a wrap period of about 1h11min. +*/ +#define SC_MULT 4194304000u +#define SC_SHIFT 22 +unsigned long long notrace sched_clock (void ) { -return cnt32_to_63 (timer_readl (TIMERUS_CNTR_1US )) *1000 ; +u32 cyc =timer_readl (TIMERUS_CNTR_1US ); +return cyc_to_fixed_sched_clock (&cd, cyc, (u32 )~0 ,SC_MULT ,SC_SHIFT ); +} + +static void notrace tegra_update_sched_clock(void) +{ +u32 cyc =timer_readl (TIMERUS_CNTR_1US ); +update_sched_clock (&cd, cyc, (u32 )~0 ); }static irqreturn_t tegra_timer_interrupt (int irq,void *dev_id)
@@ -157,6 +172,9 @@ static void __init tegra_init_timer(void)WARN (1 ,"Unknown clock rate" ); } +init_fixed_sched_clock (&cd, tegra_update_sched_clock,32 , +1000000 ,SC_MULT ,SC_SHIFT ); +if (clocksource_register_hz (&tegra_clocksource,1000000 )) {printk (KERN_ERR "Failed to register clocksource\n" );BUG ();
Signature · 1 line
1.6.2.5
PATCH 7/13 ARM: u300: convert sched_clock() to use new infrastructure
In reply to Russell King - ARM LinuxConvert u300 to use the new sched_clock() infrastructure for extending 32bit counters to full 64-bit nanoseconds.
--- arch/arm/Kconfig | 1 + arch/arm/mach-u300/timer.c | 17 +++++++++++++---- 2 files changed, 14 insertions(+), 4 deletions(-)
arch/arm/Kconfig +1 -0
@@ -790,6 +790,7 @@ config ARCH_U300 bool "ST-Ericsson U300 Series" depends on MMU select CPU_ARM926T + select HAVE_SCHED_CLOCK select HAVE_TCM select ARM_AMBA select ARM_VIC
arch/arm/mach-u300/timer.c +13 -4
@@ -21,6 +21,7 @@#include <mach/hardware.h> /* Generic stuff */ +#include <asm/sched_clock.h> #include <asm/mach/map.h> #include <asm/mach/time.h> #include <asm/mach/irq.h>
@@ -352,12 +353,18 @@ static struct clocksource clocksource_u300_1mhz = {* this wraps around for now, since it is just a relative time * stamp. (Inspired by OMAP implementation.) */ +static DEFINE_CLOCK_DATA (cd); +unsigned long long notrace sched_clock (void ) { -return clocksource_cyc2ns (clocksource_u300_1mhz.read ( - &clocksource_u300_1mhz), - clocksource_u300_1mhz.mult, - clocksource_u300_1mhz.shift); +u32 cyc =readl (U300_TIMER_APP_VBASE +U300_TIMER_APP_GPT2CC ); +return cyc_to_sched_clock (&cd, cyc, (u32 )~0 ); +} + +static void notrace u300_update_sched_clock(void) +{ +u32 cyc =readl (U300_TIMER_APP_VBASE +U300_TIMER_APP_GPT2CC ); +update_sched_clock (&cd, cyc, (u32 )~0 ); }
@@ -375,6 +382,8 @@ static void __init u300_timer_init(void)clk_enable (clk); rate =clk_get_rate (clk); +init_sched_clock (&cd, u300_update_sched_clock,32 , rate); +/* * Disable the "OS" and "DD" timers - these are designed for Symbian! * Example usage in cnh1601578 cpu subsystem pd_timer_app.c
Signature · 1 line
1.6.2.5
PATCH 8/13 ARM: iop: convert sched_clock() to use new infrastructure
In reply to Russell King - ARM LinuxConvert iop platforms to use the new sched_clock() infrastructure for extending 32bit counters to full 64-bit nanoseconds.
--- arch/arm/Kconfig | 1 + arch/arm/plat-iop/time.c | 14 +++++++++++--- 2 files changed, 12 insertions(+), 3 deletions(-)
arch/arm/Kconfig +1 -0
@@ -992,6 +992,7 @@ config ARCH_ACORN config PLAT_IOP bool select GENERIC_CLOCKEVENTS + select HAVE_SCHED_CLOCK config PLAT_ORION bool
arch/arm/plat-iop/time.c +11 -3
@@ -49,15 +49,21 @@ static struct clocksource iop_clocksource = { .flags = CLOCK_SOURCE_IS_CONTINUOUS, }; +static DEFINE_CLOCK_DATA (cd); +/* * IOP sched_clock() implementation via its clocksource. */ unsigned long long notrace sched_clock (void ) { -cycle_t cyc =iop_clocksource_read (NULL ); -struct clocksource *cs = &iop_clocksource; +u32 cyc =0xffffffffu -read_tcr1 (); +return cyc_to_sched_clock (&cd, cyc, (u32 )~0 ); +} -return clocksource_cyc2ns (cyc, cs->mult, cs->shift); +static void notrace iop_update_sched_clock(void) +{ +u32 cyc =0xffffffffu -read_tcr1 (); +update_sched_clock (&cd, cyc, (u32 )~0 ); }/*
@@ -142,6 +148,8 @@ void __init iop_init_time(unsigned long tick_rate) {u32 timer_ctl; +init_sched_clock (&cd, iop_update_sched_clock,32 , tick_rate); + ticks_per_jiffy =DIV_ROUND_CLOSEST (tick_rate,HZ ); iop_tick_rate = tick_rate;
Signature · 1 line
1.6.2.5
PATCH 9/13 ARM: nomadik: convert sched_clock() to use new infrastructure
In reply to Russell King - ARM LinuxConvert nomadik platforms to use the new sched_clock() infrastructure for extending 32bit counters to full 64-bit nanoseconds.
--- arch/arm/plat-nomadik/Kconfig | 1 + arch/arm/plat-nomadik/timer.c | 76 +++++------------------------------------ 2 files changed, 10 insertions(+), 67 deletions(-)
arch/arm/plat-nomadik/Kconfig +1 -0
@@ -14,6 +14,7 @@ if PLAT_NOMADIK config HAS_MTU bool + select HAVE_SCHED_CLOCK help Support for Multi Timer Unit. MTU provides access to multiple interrupt generating programmable
arch/arm/plat-nomadik/timer.c +9 -67
106 lines not on this page · raw message
Signature · 1 line
1.6.2.5
PATCH 10/13 ARM: omap: convert sched_clock() to use new infrastructure
In reply to Russell King - ARM LinuxConvert omap to use the new sched_clock() infrastructure for extending 32bit counters to full 64-bit nanoseconds.
--- arch/arm/Kconfig | 1 + arch/arm/plat-omap/counter_32k.c | 24 ++++++++++++++++++++++-- 2 files changed, 23 insertions(+), 2 deletions(-)
arch/arm/Kconfig +1 -0
@@ -839,6 +839,7 @@ config ARCH_OMAP select ARCH_REQUIRE_GPIOLIB select ARCH_HAS_CPUFREQ select GENERIC_CLOCKEVENTS + select HAVE_SCHED_CLOCK select ARCH_HAS_HOLES_MEMORYMODEL help Support for TI's OMAP platform (OMAP1/2/3/4).
arch/arm/plat-omap/counter_32k.c +22 -2
@@ -17,6 +17,8 @@#include <linux/clk.h> #include <linux/io.h> +#include <asm/sched_clock.h> +#include <plat/common.h> #include <plat/board.h>
@@ -109,10 +111,25 @@ static struct clocksource clocksource_32k = {* Returns current time from boot in nsecs. It's OK for this to wrap * around for now, as it's just a relative time stamp. */ +static DEFINE_CLOCK_DATA (cd); + +/* +* Constants generated by clocks_calc_mult_shift(m, s, 32768, NSEC_PER_SEC, 60). +* This gives a resolution of about 30us and a wrap period of about 36hrs. +*/ +#define SC_MULT 4000000000u +#define SC_SHIFT 17 +unsigned long long notrace sched_clock (void ) { -return clocksource_cyc2ns (clocksource_32k.read (&clocksource_32k), - clocksource_32k.mult, clocksource_32k.shift); +u32 cyc = clocksource_32k.read (&clocksource_32k); +return cyc_to_fixed_sched_clock (&cd, cyc, (u32 )~0 ,SC_MULT ,SC_SHIFT ); +} + +static void notrace omap_update_sched_clock(void) +{ +u32 cyc = clocksource_32k.read (&clocksource_32k); +update_sched_clock (&cd, cyc, (u32 )~0 ); }/**
@@ -170,6 +187,9 @@ static int __init omap_init_clocksource_32k(void)if (clocksource_register_hz (&clocksource_32k,32768 ))printk (err, clocksource_32k.name); + +init_fixed_sched_clock (&cd, omap_update_sched_clock,32 , +32768 ,SC_MULT ,SC_SHIFT ); }return 0 ; }
Signature · 1 line
1.6.2.5
PATCH 11/13 ARM: orion: convert sched_clock() to use new infrastructure
In reply to Russell King - ARM LinuxConvert orion platforms to use the new sched_clock() infrastructure for extending 32bit counters to full 64-bit nanoseconds.
--- arch/arm/Kconfig | 1 + arch/arm/plat-orion/time.c | 44 +++++++++----------------------------------- 2 files changed, 10 insertions(+), 35 deletions(-)
arch/arm/Kconfig +1 -0
@@ -997,6 +997,7 @@ config PLAT_IOP config PLAT_ORION bool + select HAVE_SCHED_CLOCK config PLAT_PXA bool
arch/arm/plat-orion/time.c +9 -35
72 lines not on this page · raw message
Signature · 1 line
1.6.2.5
PATCH 12/13 ARM: versatile: convert sched_clock() to use new infrastructure
In reply to Russell King - ARM LinuxConvert versatile platforms to use the new sched_clock() infrastructure for extending 32bit counters to full 64-bit nanoseconds.
--- arch/arm/Kconfig | 2 + arch/arm/mach-realview/core.c | 10 ++++ arch/arm/mach-versatile/core.c | 10 ++++ arch/arm/plat-versatile/include/plat/sched_clock.h | 6 +++ arch/arm/plat-versatile/sched-clock.c | 48 +++++++++++--------- 5 files changed, 54 insertions(+), 22 deletions(-) create mode 100644 arch/arm/plat-versatile/include/plat/sched_clock.h
arch/arm/Kconfig +2 -0
@@ -236,6 +236,7 @@ config ARCH_REALVIEW bool "ARM Ltd. RealView family" select ARM_AMBA select COMMON_CLKDEV + select HAVE_SCHED_CLOCK select ICST select GENERIC_CLOCKEVENTS select ARCH_WANT_OPTIONAL_GPIOLIB
@@ -250,6 +251,7 @@ config ARCH_VERSATILE select ARM_AMBA select ARM_VIC select COMMON_CLKDEV + select HAVE_SCHED_CLOCK select ICST select GENERIC_CLOCKEVENTS select ARCH_WANT_OPTIONAL_GPIOLIB
arch/arm/mach-realview/core.c +10 -0
@@ -52,6 +52,8 @@#include <mach/irqs.h> #include <plat/timer-sp.h> +#include <plat/sched_clock.h> +#include "core.h" /* used by entry-macro.S and platsmp.c */
@@ -658,6 +660,12 @@ void realview_leds_event(led_event_t ledevt)#endif /* CONFIG_LEDS */ /* +* The sched_clock counter +*/ +#define REFCOUNTER (__io_address(REALVIEW_SYS_BASE) + \ + REALVIEW_SYS_24MHz_OFFSET) + +/* * Where is the timer (VA)? */ void __iomem *timer0_va_base;
@@ -672,6 +680,8 @@ void __init realview_timer_init(unsigned int timer_irq) {u32 val; +versatile_sched_clock_init (REFCOUNTER ,24000000 ); +/* * set clock frequency: * REALVIEW_REFCLK is 32KHz
arch/arm/mach-versatile/core.c +10 -0
@@ -51,6 +51,8 @@#include <mach/platform.h> #include <plat/timer-sp.h> +#include <plat/sched_clock.h> +#include "core.h" /*
@@ -886,6 +888,12 @@ void __init versatile_init(void) }/* +* The sched_clock counter +*/ +#define REFCOUNTER (__io_address(VERSATILE_SYS_BASE) + \ + VERSATILE_SYS_24MHz_OFFSET) + +/* * Where is the timer (VA)? */ #define TIMER0_VA_BASE __io_address(VERSATILE_TIMER0_1_BASE)
@@ -900,6 +908,8 @@ static void __init versatile_timer_init(void) {u32 val; +versatile_sched_clock_init (REFCOUNTER ,24000000 ); +/* * set clock frequency: * VERSATILE_REFCLK is 32KHz
arch/arm/plat-versatile/include/plat/sched_clock.h new file +6 -0
@@ -0,0 +1,6 @@ +#ifndef ARM_PLAT_SCHED_CLOCK_H +#define ARM_PLAT_SCHED_CLOCK_H + +void versatile_sched_clock_init (void __iomem *,unsigned long ); + +#endif
arch/arm/plat-versatile/sched-clock.c +26 -22
63 lines not on this page · raw message
Signature · 1 line
1.6.2.5
PATCH 13/13 ARM: vexpress: add sched_clock() for Versatile Express
In reply to Russell King - ARM LinuxAdd a sched_clock() implementation to Versatile Express using the new sched_clock() infrastructure for extending 32bit counters to full 64-bit nanoseconds.
--- arch/arm/Kconfig | 1 + arch/arm/mach-vexpress/v2m.c | 4 ++++ arch/arm/plat-versatile/Makefile | 4 +--- 3 files changed, 6 insertions(+), 3 deletions(-)
arch/arm/Kconfig +1 -0
@@ -268,6 +268,7 @@ config ARCH_VEXPRESS select COMMON_CLKDEV select GENERIC_CLOCKEVENTS select HAVE_CLK + select HAVE_SCHED_CLOCK select ICST select PLAT_VERSATILE help
arch/arm/mach-vexpress/v2m.c +4 -0
@@ -22,6 +22,8 @@#include <mach/clkdev.h> #include <mach/motherboard.h> +#include <plat/sched_clock.h> +#include <plat/timer-sp.h> #include "core.h"
@@ -50,6 +52,8 @@ void __init v2m_map_io(struct map_desc *tile, size_t num)static void __init v2m_timer_init (void ) { +versatile_sched_clock_init (MMIO_P2V (V2M_SYS_24MHZ ),24000000 ); +writel (0 ,MMIO_P2V (V2M_TIMER0 ) +TIMER_CTRL );writel (0 ,MMIO_P2V (V2M_TIMER1 ) +TIMER_CTRL );
arch/arm/plat-versatile/Makefile +1 -3
@@ -1,7 +1,5 @@ -obj-y := clock.o +obj-y := clock.o sched-clock.o obj-$(CONFIG_ARM_TIMER_SP804) += timer-sp.o -obj-$(CONFIG_ARCH_REALVIEW) += sched-clock.o -obj-$(CONFIG_ARCH_VERSATILE) += sched-clock.o ifeq ($(CONFIG_LEDS_CLASS),y) obj-$(CONFIG_ARCH_REALVIEW) += leds.o obj-$(CONFIG_ARCH_VERSATILE) += leds.o
Signature · 1 line
1.6.2.5
cover 64-bit sched_clock
In reply to Russell King - ARM LinuxThere is one issue which remains with this code - that is the initialization ordering.
init/main.c:
setup_arch(&command_line); ... sched_init(); ... early_irq_init(); init_IRQ(); ... timekeeping_init(); time_init();
sched_init() calls init_idle() which calls sched_clock() to set the start time for the idle thread. However, our sched_clock() is not initialized (in general) until init/main.c calls time_init().
This results in a big step in the sched_clock() return value during boot, which is probably not desirable. At the moment, I work around this by explicitly setting the epoch_ns to zero after the first update callback - but this means that sched_clock() will only start at this point, which may not be desirable for everyone.
Some sched_clock() implementations rely upon clocksource code (they use the clocksource read function) and this is only initialized when the clocksource is registered. There's also the matter of the clock rate.
It's probably not a good idea to register clocksources before timekeeping_init() has happened either.
I can provide an init_early() method in the machine description which setup_arch() can call as the last thing it does - this will happen after the ->reserve and ->map_io callbacks, but before interrupts and timers have been setup. This is the ideal time to setup the sched_clock(). However, that means digging through lots of arch code to sort out what can be moved where, and that's going to be inherently unreliable.
I think init_early() has merit in other ways - it'll allow us to remove crap from the ->map_io callbacks which is doing stuff like registering clk structures and so forth.
Any other suggestions?
cover 64-bit sched_clock
2010/12/16 Russell King - ARM Linux <[email protected]>:
Quoted 11 lines
I can provide an
init_early()method in the machine description whichsetup_arch()can call as the last thing it does - this will happen after the ->reserve and ->map_io callbacks, but before interrupts and timers have been setup. ?This is the ideal time to setup thesched_clock(). ?However, that means digging through lots of arch code to sort out what can be moved where, and that's going to be inherently unreliable.I think
init_early()has merit in other ways - it'll allow us to remove crap from the ->map_io callbacks which is doing stuff like registering clk structures and so forth.
I agree. I was contemplating a clock_init() or similar a way back for setting up clock trees early. We would have to move clock tree registration in front of the init_sched_clock() call in the init_early() handler in many cases since the rate often comes from the clock framework, but this is indeed where it belongs.
So a clear ACK for this approach.
Yours,
Linus Walleij
cover 64-bit sched_clock
Quoted 14 lines · Linus Walleij
2010/12/16 Russell King - ARM Linux <[email protected]>:
Quoted 11 lines
I can provide an
init_early()method in the machine description whichsetup_arch()can call as the last thing it does - this will happen after the ->reserve and ->map_io callbacks, but before interrupts and timers have been setup. ?This is the ideal time to setup thesched_clock(). ?However, that means digging through lots of arch code to sort out what can be moved where, and that's going to be inherently unreliable.I think
init_early()has merit in other ways - it'll allow us to remove crap from the ->map_io callbacks which is doing stuff like registering clk structures and so forth.
Quoted 5 lines
I agree. I was contemplating a
clock_init()or similar a way back for setting up clock trees early. We would have to move clock tree registration in front of theinit_sched_clock()call in theinit_early()handler in many cases since the rate often comes from the clock framework, but this is indeed where it belongs.
Yes, that was indeed one of the subsystems that I thought platforms should initialize there - one of my motivations is in connection with 6433/1 and 6434/1.
If we have everyone setting up their clock subsystems in init_early, then we can have both the TWD and SP804 always getting their clock rates from the clk API, rather than having the conditional approach in those two patches.