Skip to contentSkip to conversation

[patch 00/23]

PATCH In review Thomas Gleixner 7 participants 55 messages Sep 30, 2006 – Oct 3, 2006

Subsystems (10)
lore
Share & tools
lore
#0 lore raw

cover [patch 00/23]

We are pleased to announce the next version of our "high resolution timers" and "dynticks" patchset, which implements two important features that Linux lacked for many years.

The patchset is against 2.6.18-mm2. (Since our last release there were no big changes, other than bugfixes and internal releasification cleanups, and the merge to -mm. The queue is bisect-friendly.)

If review and feedback is positive we'd like this patchset to be added to the 2.6.19 kernel. It has been maintained ontop of ktimers initially (more than a year ago), and then ontop of hrtimers (after ktimers were renamed to hrtimers and the hrtimer subsystem went upstream in January). Various -hrt iterations have been announced on lkml numerous times in the past year.

Now that the hrtimers subsystem and most of John Stultz Generic Time Of Day work is upstream, this patchset is straightforward and carries little risks if high-res timers are turned off (which is the default).

This patchset has been tested on various i686 systems. (We have the x86_64 patches too, but we'd like to concentrate on this first wave initially.)

The patchset can also be found at:

http://www.tglx.de/projects/hrtimers/2.6.18-mm2/patch-2.6.18-mm2-hrt-dyntick1.patches.tar.bz2

Thomas, Ingo

--

#1 lore raw

PATCH 1/23 GTOD: exponential update_wall_time

From: John Stultz <[email protected]>

Accumulate time in update_wall_time() exponentially. This avoids long running loops seen with the dynticks patch as well as the problematic hang seen on systems with broken clocksources.

NOTE: this only has relevance on dyntick kernels, so the quality of NTP updates on jiffies-tick systems is unaffected. (non-dyntick kernels call update_wall_time() in every timer tick)

--

 kernel/timer.c |   28 ++++++++++++++++++++--------
 1 file changed, 20 insertions(+), 8 deletions(-)
kernel/timer.c +20 -8
Index: linux-2.6.18-mm2/kernel/timer.c
===================================================================
@@ -907,6 +907,7 @@ static void clocksource_adjust(struct cl
 static void update_wall_time(void)
 {
 	cycle_t offset;
+	int shift = 0;
 
 	/* Make sure we're fully resumed: */
 	if (unlikely(timekeeping_suspended))
@@ -919,28 +920,39 @@ static void update_wall_time(void)
 #endif
 	clock->xtime_nsec += (s64)xtime.tv_nsec << clock->shift;
 
+	while (offset > clock->cycle_interval << (shift + 1))
+		shift++;
+
 	/* normally this loop will run just once, however in the
 	 * case of lost or late ticks, it will accumulate correctly.
 	 */
 	while (offset >= clock->cycle_interval) {
+		if (offset < (clock->cycle_interval << shift)) {
+			shift--;
+			continue;
+		}
+
 		/* accumulate one interval */
-		clock->xtime_nsec += clock->xtime_interval;
-		clock->cycle_last += clock->cycle_interval;
-		offset -= clock->cycle_interval;
+		clock->xtime_nsec += clock->xtime_interval << shift;
+		clock->cycle_last += clock->cycle_interval << shift;
+		offset -= clock->cycle_interval << shift;
 
-		if (clock->xtime_nsec >= (u64)NSEC_PER_SEC << clock->shift) {
+		while (clock->xtime_nsec >= (u64)NSEC_PER_SEC << clock->shift) {
 			clock->xtime_nsec -= (u64)NSEC_PER_SEC << clock->shift;
 			xtime.tv_sec++;
 			second_overflow();
 		}
 
 		/* interpolator bits */
-		time_interpolator_update(clock->xtime_interval
-						>> clock->shift);
+		time_interpolator_update((clock->xtime_interval
+						>> clock->shift)<<shift);
 
 		/* accumulate error between NTP and clock interval */
-		clock->error += current_tick_length();
-		clock->error -= clock->xtime_interval << (TICK_LENGTH_SHIFT - clock->shift);
+		clock->error += current_tick_length() << shift;
+		clock->error -= (clock->xtime_interval
+			<< (TICK_LENGTH_SHIFT - clock->shift))<<shift;
+
+		shift--;
 	}
 
 	/* correct the clock when NTP error is too big */

--

#2 lore raw

PATCH 2/23 GTOD: persistent clock support, core

In reply to Thomas Gleixner

From: John Stultz <[email protected]>

persistent clock support: do proper timekeeping across suspend/resume.

--

 include/linux/hrtimer.h |    3 +++
 include/linux/time.h    |    1 +
 kernel/hrtimer.c        |    8 ++++++++
 kernel/timer.c          |   34 +++++++++++++++++++++++++++++++---
 4 files changed, 43 insertions(+), 3 deletions(-)

linux-2.6.18-rc6_timeofday-persistent-clock-generic_C6.patch

include/linux/hrtimer.h +3 -0
Index: linux-2.6.18-mm2/include/linux/hrtimer.h
===================================================================
@@ -146,6 +146,9 @@ extern void hrtimer_init_sleeper(struct 
 /* Soft interrupt function to run the hrtimer queues: */
 extern void hrtimer_run_queues(void);
 
+/* Resume notification */
+void hrtimer_notify_resume(void);
+
 /* Bootup initialization: */
 extern void __init hrtimers_init(void);
 
include/linux/time.h +1 -0
Index: linux-2.6.18-mm2/include/linux/time.h
===================================================================
@@ -92,6 +92,7 @@ extern struct timespec xtime;
 extern struct timespec wall_to_monotonic;
 extern seqlock_t xtime_lock;
 
+extern unsigned long read_persistent_clock(void);
 void timekeeping_init(void);
 
 static inline unsigned long get_seconds(void)
kernel/hrtimer.c +8 -0
Index: linux-2.6.18-mm2/kernel/hrtimer.c
===================================================================
@@ -287,6 +287,14 @@ static unsigned long ktime_divns(const k
 #endif /* BITS_PER_LONG >= 64 */
 
 /*
+ * Timekeeping resumed notification
+ */
+void hrtimer_notify_resume(void)
+{
+	clock_was_set();
+}
+
+/*
  * Counterpart to lock_timer_base above:
  */
 static inline
kernel/timer.c +31 -3
Index: linux-2.6.18-mm2/kernel/timer.c
===================================================================
@@ -41,6 +41,9 @@
 #include <asm/timex.h>
 #include <asm/io.h>
 
+/* jiffies at the most recent update of wall time */
+unsigned long wall_jiffies = INITIAL_JIFFIES;
+
 u64 jiffies_64 __cacheline_aligned_in_smp = INITIAL_JIFFIES;
 
 EXPORT_SYMBOL(jiffies_64);
@@ -743,12 +746,20 @@ int timekeeping_is_continuous(void)
 	return ret;
 }
 
+/* Weak dummy function for arches that do not yet support it.
+ * XXX - Do be sure to remove it once all arches implement it.
+ */
+unsigned long __attribute__((weak)) read_persistent_clock(void)
+{
+	return 0;
+}
+
 /*
  * timekeeping_init - Initializes the clocksource and common timekeeping values
  */
 void __init timekeeping_init(void)
 {
-	unsigned long flags;
+	unsigned long flags, sec = read_persistent_clock();
 
 	write_seqlock_irqsave(&xtime_lock, flags);
 
@@ -758,11 +769,18 @@ void __init timekeeping_init(void)
 	clocksource_calculate_interval(clock, tick_nsec);
 	clock->cycle_last = clocksource_read(clock);
 
+	xtime.tv_sec = sec;
+	xtime.tv_nsec = (jiffies % HZ) * (NSEC_PER_SEC / HZ);
+	set_normalized_timespec(&wall_to_monotonic,
+		-xtime.tv_sec, -xtime.tv_nsec);
+
 	write_sequnlock_irqrestore(&xtime_lock, flags);
 }
 
 
 static int timekeeping_suspended;
+static unsigned long timekeeping_suspend_time;
+
 /**
  * timekeeping_resume - Resumes the generic timekeeping subsystem.
  * @dev:	unused
@@ -773,14 +791,23 @@ static int timekeeping_suspended;
  */
 static int timekeeping_resume(struct sys_device *dev)
 {
-	unsigned long flags;
+	unsigned long flags, now = read_persistent_clock();
 
 	write_seqlock_irqsave(&xtime_lock, flags);
-	/* restart the last cycle value */
+
+	if (now && (now > timekeeping_suspend_time)) {
+		unsigned long sleep_length = now - timekeeping_suspend_time;
+		xtime.tv_sec += sleep_length;
+		jiffies_64 += sleep_length * HZ;
+	}
+	/* re-base the last cycle value */
 	clock->cycle_last = clocksource_read(clock);
 	clock->error = 0;
 	timekeeping_suspended = 0;
 	write_sequnlock_irqrestore(&xtime_lock, flags);
+
+	hrtimer_notify_resume();
+
 	return 0;
 }
 
@@ -790,6 +817,7 @@ static int timekeeping_suspend(struct sy
 
 	write_seqlock_irqsave(&xtime_lock, flags);
 	timekeeping_suspended = 1;
+	timekeeping_suspend_time = read_persistent_clock();
 	write_sequnlock_irqrestore(&xtime_lock, flags);
 	return 0;
 }

--

#3 lore raw

Re: [patch 02/23] GTOD: persistent clock support, core

Quoted 3 lines · On Fri, 29 Sep 2006 23:58:21 -0000 Thomas Gleixner <[email protected]> wrote:

From: John Stultz <[email protected]>

persistent clock support: do proper timekeeping across suspend/resume.

How?

kernel/timer.c int timekeeping_is_continuous(void) · line 755

Quoted 7 lines
kernel/timer.c +7 -0
+/* Weak dummy function for arches that do not yet support it.
+ * XXX - Do be sure to remove it once all arches implement it.
+ */
+unsigned long __attribute__((weak)) read_persistent_clock(void)
+{
+	return 0;
+}

Seconds? microseconds? jiffies? walltime? uptime?

Needs some comments.

kernel/timer.c int timekeeping_is_continuous(void) · line 762

Quoted 4 lines

void __init timekeeping_init(void)
{

kernel/timer.c +1 -1
-	unsigned long flags;
+	unsigned long flags, sec = read_persistent_clock();

So it apparently returns seconds-since-epoch?

If so, why?

kernel/timer.c void __init timekeeping_init(void) · line 773

Quoted 8 lines
kernel/timer.c +2 -0
 	write_seqlock_irqsave(&xtime_lock, flags);
@@ -758,11 +769,18 @@ void __init timekeeping_init(void)
 	clocksource_calculate_interval(clock, tick_nsec);
 	clock->cycle_last = clocksource_read(clock);
 
+	xtime.tv_sec = sec;
+	xtime.tv_nsec = (jiffies % HZ) * (NSEC_PER_SEC / HZ);

Why is it valid to take the second from the persistent clock and the fraction-of-a-second from jiffies? Some comments describing the implementation would improve its understandability and maintainability.

This statement can set xtime.tv_nsec to a value >= NSEC_PER_SEC. Should it not be normalised?

kernel/timer.c void __init timekeeping_init(void) · line 782

Quoted 9 lines
kernel/timer.c +4 -0
+	set_normalized_timespec(&wall_to_monotonic,
+		-xtime.tv_sec, -xtime.tv_nsec);
+
 	write_sequnlock_irqrestore(&xtime_lock, flags);
 }
 
 
 static int timekeeping_suspended;
+static unsigned long timekeeping_suspend_time;

In what units?

kernel/timer.c static int timekeeping_suspended; · line 794

Quoted 10 lines
kernel/timer.c +2 -1
+
 /**
  * timekeeping_resume - Resumes the generic timekeeping subsystem.
  * @dev:	unused
@@ -773,14 +791,23 @@ static int timekeeping_suspended;
  */
 static int timekeeping_resume(struct sys_device *dev)
 {
-	unsigned long flags;
+	unsigned long flags, now = read_persistent_clock();

Would whoever keeps doing that please stop it? This:

	unsigned long flags;
	unsigned long now = read_persistent_clock();

is more readable and makes for more readable patches in the future.

kernel/timer.c static int timekeeping_suspended; · line 801

Quoted 7 lines
kernel/timer.c +5 -1
 	write_seqlock_irqsave(&xtime_lock, flags);
-	/* restart the last cycle value */
+
+	if (now && (now > timekeeping_suspend_time)) {
+		unsigned long sleep_length = now - timekeeping_suspend_time;
+		xtime.tv_sec += sleep_length;
+		jiffies_64 += sleep_length * HZ;

sleep_length will overflow if we slept for more than 49 days, and HZ=1000.

Quoted 22 lines
kernel/timer.c +6 -1
+	}
+	/* re-base the last cycle value */
 	clock->cycle_last = clocksource_read(clock);
 	clock->error = 0;
 	timekeeping_suspended = 0;
 	write_sequnlock_irqrestore(&xtime_lock, flags);
+
+	hrtimer_notify_resume();
+
 	return 0;
 }
 
@@ -790,6 +817,7 @@ static int timekeeping_suspend(struct sy
 
 	write_seqlock_irqsave(&xtime_lock, flags);
 	timekeeping_suspended = 1;
+	timekeeping_suspend_time = read_persistent_clock();
 	write_sequnlock_irqrestore(&xtime_lock, flags);
 	return 0;
 }
 
--
#4 lore raw

Re: [patch 02/23] GTOD: persistent clock support, core

Quoted 4 lines
Quoted 2 lines

persistent clock support: do proper timekeeping across suspend/resume.

How?

Rereading the RTC seems the only way to me. Someone prove me wrong, and do it fast! :)

Jan Engelhardt

#5 lore raw

Re: [patch 02/23] GTOD: persistent clock support, core

In reply to Andrew Morton
Quoted 8 lines · Andrew Morton
Quoted 3 lines · On Fri, 29 Sep 2006 23:58:21 -0000 Thomas Gleixner <[email protected]> wrote:

From: John Stultz <[email protected]>

persistent clock support: do proper timekeeping across suspend/resume.

How?

Improved description included below.

Quoted 11 lines

Review

Quoted 7 lines
(quoted diff) +7 -0
+/* Weak dummy function for arches that do not yet support it.
+ * XXX - Do be sure to remove it once all arches implement it.
+ */
+unsigned long __attribute__((weak)) read_persistent_clock(void)
+{
+	return 0;
+}

Seconds? microseconds? jiffies? walltime? uptime?

Needs some comments.

Agreed. Thanks for pointing it out.

Quoted 16 lines

Review

Quoted 5 lines

void __init timekeeping_init(void)
{

(quoted diff) +1 -1
-	unsigned long flags;
+	unsigned long flags, sec = read_persistent_clock();

So it apparently returns seconds-since-epoch?

If so, why?

Quoted 6 lines
(quoted diff) +0 -0
 	write_seqlock_irqsave(&xtime_lock, flags);
@@ -758,11 +769,18 @@ void __init timekeeping_init(void)
 	clocksource_calculate_interval(clock, tick_nsec);
 	clock->cycle_last = clocksource_read(clock);
 
Quoted 6 lines

Review void __init timekeeping_init(void) · line 773

Quoted 2 lines
(quoted diff) +2 -0
+	xtime.tv_sec = sec;
+	xtime.tv_nsec = (jiffies % HZ) * (NSEC_PER_SEC / HZ);

Why is it valid to take the second from the persistent clock and the fraction-of-a-second from jiffies? Some comments describing the implementation would improve its understandability and maintainability.

Yea. i386 and other arches have done this for awhile, so I preserved it. However on further inspection, it really doesn't make much sense. We're pre-timer interurpts anyway, so jiffies won't have begun yet. So now I just initialize it to zero.

Quoted 2 lines

This statement can set xtime.tv_nsec to a value >= NSEC_PER_SEC. Should it not be normalised?

Yep, it is, and you commented just above it.:)

Quoted 10 lines

Review void __init timekeeping_init(void) · line 781

Quoted 8 lines
(quoted diff) +4 -0
+	set_normalized_timespec(&wall_to_monotonic,
+		-xtime.tv_sec, -xtime.tv_nsec);
+
 	write_sequnlock_irqrestore(&xtime_lock, flags);
 }
 
 static int timekeeping_suspended;
+static unsigned long timekeeping_suspend_time;

In what units?

Fixed.

Quoted 12 lines

Review static int timekeeping_suspended; · line 794

Quoted 10 lines
(quoted diff) +2 -1
+
 /**
  * timekeeping_resume - Resumes the generic timekeeping subsystem.
  * @dev:	unused
@@ -773,14 +791,23 @@ static int timekeeping_suspended;
  */
 static int timekeeping_resume(struct sys_device *dev)
 {
-	unsigned long flags;
+	unsigned long flags, now = read_persistent_clock();

Would whoever keeps doing that please stop it? This:

Quoted 4 lines
	unsigned long flags;
	unsigned long now = read_persistent_clock();

is more readable and makes for more readable patches in the future.

Fixed.

Quoted 9 lines

Review static int timekeeping_suspended; · line 800

Quoted 7 lines
(quoted diff) +5 -1
 	write_seqlock_irqsave(&xtime_lock, flags);
-	/* restart the last cycle value */
+
+	if (now && (now > timekeeping_suspend_time)) {
+		unsigned long sleep_length = now - timekeeping_suspend_time;
+		xtime.tv_sec += sleep_length;
+		jiffies_64 += sleep_length * HZ;

sleep_length will overflow if we slept for more than 49 days, and HZ=1000.

Oh! Great catch! Fixed.

Thanks so much for the thorough review!

Updated patch follows:

thanks
-john

Implement generic timekeeping suspend/resume accounting by introducing the read_persistent_clock() interface. This is an arch specific function that returns the seconds since the epoch using the arch defined battery backed clock.

Aside from allowing the removal of duplicate arch specific resume logic, this change helps avoid potential resume time ordering issues between generic and arch specific time code.

This patch only provides the generic usage of this new function and a weak dummy function, that always returns zero if no arch specific function is defined. Thus if no persistent clock is present, no change in behavior should be seen with this patch.

--

 include/linux/hrtimer.h |    3 +++
 include/linux/time.h    |    1 +
 kernel/hrtimer.c        |    8 ++++++++
 kernel/timer.c          |   40 +++++++++++++++++++++++++++++++++++++++-
 4 files changed, 51 insertions(+), 1 deletion(-)

linux-2.6.18_timeofday-persistent-clock-generic_C7.patch
============================================

include/linux/hrtimer.h +3 -0
index fca9302..660d91d 100644
@@ -146,6 +146,9 @@ extern void hrtimer_init_sleeper(struct 
 /* Soft interrupt function to run the hrtimer queues: */
 extern void hrtimer_run_queues(void);
 
+/* Resume notification */
+void hrtimer_notify_resume(void);
+
 /* Bootup initialization: */
 extern void __init hrtimers_init(void);
 
include/linux/time.h +1 -0
index a5b7399..db31d2a 100644
@@ -92,6 +92,7 @@ extern struct timespec xtime;
 extern struct timespec wall_to_monotonic;
 extern seqlock_t xtime_lock;
 
+extern unsigned long read_persistent_clock(void);
 void timekeeping_init(void);
 
 static inline unsigned long get_seconds(void)
kernel/hrtimer.c +8 -0
index d0ba190..090b752 100644
@@ -287,6 +287,14 @@ static unsigned long ktime_divns(const k
 #endif /* BITS_PER_LONG >= 64 */
 
 /*
+ * Timekeeping resumed notification
+ */
+void hrtimer_notify_resume(void)
+{
+	clock_was_set();
+}
+
+/*
  * Counterpart to lock_timer_base above:
  */
 static inline
kernel/timer.c +39 -1
index c1c7fbc..5069139 100644
@@ -41,6 +41,9 @@
 #include <asm/timex.h>
 #include <asm/io.h>
 
+/* jiffies at the most recent update of wall time */
+unsigned long wall_jiffies = INITIAL_JIFFIES;
+
 u64 jiffies_64 __cacheline_aligned_in_smp = INITIAL_JIFFIES;
 
 EXPORT_SYMBOL(jiffies_64);
@@ -743,12 +746,27 @@ int timekeeping_is_continuous(void)
 	return ret;
 }
 
+/**
+ * read_persistent_clock -  Return time in seconds from the persistent clock.
+ *
+ * Weak dummy function for arches that do not yet support it.
+ * Returns seconds from epoch using the battery backed persistent clock.
+ * Returns zero if unsupported.
+ *
+ *  XXX - Do be sure to remove it once all arches implement it.
+ */
+unsigned long __attribute__((weak)) read_persistent_clock(void)
+{
+	return 0;
+}
+
 /*
  * timekeeping_init - Initializes the clocksource and common timekeeping values
  */
 void __init timekeeping_init(void)
 {
 	unsigned long flags;
+	unsigned long sec = read_persistent_clock();
 
 	write_seqlock_irqsave(&xtime_lock, flags);
 
@@ -758,11 +776,20 @@ void __init timekeeping_init(void)
 	clocksource_calculate_interval(clock, tick_nsec);
 	clock->cycle_last = clocksource_read(clock);
 
+	xtime.tv_sec = sec;
+	xtime.tv_nsec = 0;
+	set_normalized_timespec(&wall_to_monotonic,
+		-xtime.tv_sec, -xtime.tv_nsec);
+
 	write_sequnlock_irqrestore(&xtime_lock, flags);
 }
 
 
+/* flag for if timekeeping is suspended */
 static int timekeeping_suspended;
+/* time in seconds when suspend began */
+static unsigned long timekeeping_suspend_time;
+
 /**
  * timekeeping_resume - Resumes the generic timekeeping subsystem.
  * @dev:	unused
@@ -774,13 +801,23 @@ static int timekeeping_suspended;
 static int timekeeping_resume(struct sys_device *dev)
 {
 	unsigned long flags;
+	unsigned long now = read_persistent_clock();
 
 	write_seqlock_irqsave(&xtime_lock, flags);
-	/* restart the last cycle value */
+
+	if (now && (now > timekeeping_suspend_time)) {
+		unsigned long sleep_length = now - timekeeping_suspend_time;
+		xtime.tv_sec += sleep_length;
+		jiffies_64 += (u64)sleep_length * HZ;
+	}
+	/* re-base the last cycle value */
 	clock->cycle_last = clocksource_read(clock);
 	clock->error = 0;
 	timekeeping_suspended = 0;
 	write_sequnlock_irqrestore(&xtime_lock, flags);
+
+	hrtimer_notify_resume();
+
 	return 0;
 }
 
@@ -790,6 +827,7 @@ static int timekeeping_suspend(struct sy
 
 	write_seqlock_irqsave(&xtime_lock, flags);
 	timekeeping_suspended = 1;
+	timekeeping_suspend_time = read_persistent_clock();
 	write_sequnlock_irqrestore(&xtime_lock, flags);
 	return 0;
 }
#6 lore raw

PATCH 3/23 GTOD: persistent clock support, i386

In reply to Thomas Gleixner

From: John Stultz <[email protected]>

persistent clock support: do proper timekeeping across suspend/resume, i386 arch support.

--

 arch/i386/kernel/apm.c  |   44 ---------------------------------------
 arch/i386/kernel/time.c |   54 +-----------------------------------------------
 2 files changed, 2 insertions(+), 96 deletions(-)

linux-2.6.18-rc6_timeofday-persistent-clock-i386_C6.patch

arch/i386/kernel/apm.c +0 -44
Index: linux-2.6.18-mm2/arch/i386/kernel/apm.c
===================================================================
@@ -234,7 +234,6 @@
 
 #include "io_ports.h"
 
-extern unsigned long get_cmos_time(void);
 extern void machine_real_restart(unsigned char *, int);
 
 #if defined(CONFIG_APM_DISPLAY_BLANK) && defined(CONFIG_VT)
@@ -1153,28 +1152,6 @@ out:
 	spin_unlock(&user_list_lock);
 }
 
-static void set_time(void)
-{
-	struct timespec ts;
-	if (got_clock_diff) {	/* Must know time zone in order to set clock */
-		ts.tv_sec = get_cmos_time() + clock_cmos_diff;
-		ts.tv_nsec = 0;
-		do_settimeofday(&ts);
-	} 
-}
-
-static void get_time_diff(void)
-{
-#ifndef CONFIG_APM_RTC_IS_GMT
-	/*
-	 * Estimate time zone so that set_time can update the clock
-	 */
-	clock_cmos_diff = -get_cmos_time();
-	clock_cmos_diff += get_seconds();
-	got_clock_diff = 1;
-#endif
-}
-
 static void reinit_timer(void)
 {
 #ifdef INIT_TIMER_AFTER_SUSPEND
@@ -1214,19 +1191,6 @@ static int suspend(int vetoable)
 	local_irq_disable();
 	device_power_down(PMSG_SUSPEND);
 
-	/* serialize with the timer interrupt */
-	write_seqlock(&xtime_lock);
-
-	/* protect against access to timer chip registers */
-	spin_lock(&i8253_lock);
-
-	get_time_diff();
-	/*
-	 * Irq spinlock must be dropped around set_system_power_state.
-	 * We'll undo any timer changes due to interrupts below.
-	 */
-	spin_unlock(&i8253_lock);
-	write_sequnlock(&xtime_lock);
 	local_irq_enable();
 
 	save_processor_state();
@@ -1235,7 +1199,6 @@ static int suspend(int vetoable)
 	restore_processor_state();
 
 	local_irq_disable();
-	set_time();
 	reinit_timer();
 
 	if (err == APM_NO_ERROR)
@@ -1265,11 +1228,6 @@ static void standby(void)
 
 	local_irq_disable();
 	device_power_down(PMSG_SUSPEND);
-	/* serialize with the timer interrupt */
-	write_seqlock(&xtime_lock);
-	/* If needed, notify drivers here */
-	get_time_diff();
-	write_sequnlock(&xtime_lock);
 	local_irq_enable();
 
 	err = set_system_power_state(APM_STATE_STANDBY);
@@ -1363,7 +1321,6 @@ static void check_events(void)
 			ignore_bounce = 1;
 			if ((event != APM_NORMAL_RESUME)
 			    || (ignore_normal_resume == 0)) {
-				set_time();
 				device_resume();
 				pm_send_all(PM_RESUME, (void *)0);
 				queue_event(event, NULL);
@@ -1379,7 +1336,6 @@ static void check_events(void)
 			break;
 
 		case APM_UPDATE_TIME:
-			set_time();
 			break;
 
 		case APM_CRITICAL_SUSPEND:
arch/i386/kernel/time.c +2 -52

103 lines not on this page · raw message

--

#7 lore raw

Re: [patch 03/23] GTOD: persistent clock support, i386

Quoted 3 lines · On Fri, 29 Sep 2006 23:58:22 -0000 Thomas Gleixner <[email protected]> wrote:

persistent clock support: do proper timekeeping across suspend/resume, i386 arch support.

This description implies that the patch implements something for i386

Quoted 2 lines
 arch/i386/kernel/apm.c  |   44 ---------------------------------------
 arch/i386/kernel/time.c |   54 +---------------------------------------------

but all it does is delete stuff.

I _assume_ that it switches i386 over to using the (undescribed) generic core, and it does that merely by implementing read_persistent_clock().

But I'd have expected to see some Kconfig change in there as well?

Does this implementation support all forms of persistent clock which are known to exist on i386 platforms?

If/when you issue new changelogs, please describe what has to be done to port other architectures over to use this overall framework.

Do ports for other architectures exist?

#8 lore raw

Re: [patch 03/23] GTOD: persistent clock support, i386

Quoted 7 lines · Andrew Morton
Quoted 3 lines · On Fri, 29 Sep 2006 23:58:22 -0000 Thomas Gleixner <[email protected]> wrote:

persistent clock support: do proper timekeeping across suspend/resume, i386 arch support.

Quoted 6 lines

This description implies that the patch implements something for i386

Quoted 2 lines
 arch/i386/kernel/apm.c  |   44 ---------------------------------------
 arch/i386/kernel/time.c |   54 +---------------------------------------------

but all it does is delete stuff.

Improved description included.

Quoted 4 lines

I _assume_ that it switches i386 over to using the (undescribed) generic core, and it does that merely by implementing read_persistent_clock().

But I'd have expected to see some Kconfig change in there as well?

Since there is a generic weak read_persistent_clock function, all that is needed is for an arch to implement the read_persistent_clock function and remove its arch specific suspend/resume code.

Quoted 2 lines

Does this implementation support all forms of persistent clock which are known to exist on i386 platforms?

Yep. It converts the read_cmos_clock() function, which covers legacy CMOS and EFI clocks.

Quoted 2 lines

If/when you issue new changelogs, please describe what has to be done to port other architectures over to use this overall framework.

Included below.

Quoted 1 line

Do ports for other architectures exist?

I made a quick attempt earlier and covered most of the arches. However, I'm sort of wrapping this up w/ the generic time conversion (it was one of the changes I dropped in the GTOD rework earlier this year). I was going to re-add this later, but then Thomas started seeing resume ordering issues w/ the dynticks patch, so I raised the patches again.

Updated patch below:

thanks
-john

Convert i386's read_cmos_clock to the read_persistent_clock interface and remove the arch specific suspend/resume code, as the generic timekeeping code will now handle it.

If you wish to convert your arch to the read_persistent_clock code:
1) Implement read_persistent_clock in your arch.
2) Kill off xtime and jiffies modification in arch specific initialization and suspend/resume.

--

 apm.c  |   43 -------------------------------------------
 time.c |   55 ++-----------------------------------------------------
 2 files changed, 2 insertions(+), 96 deletions(-)

linux-2.6.18_timeofday-persistent-clock-i386_C7.patch
============================================

arch/i386/kernel/apm.c +0 -43
index b42f2d9..e40e7ef 100644
@@ -1153,28 +1153,6 @@ out:
 	spin_unlock(&user_list_lock);
 }
 
-static void set_time(void)
-{
-	struct timespec ts;
-	if (got_clock_diff) {	/* Must know time zone in order to set clock */
-		ts.tv_sec = get_cmos_time() + clock_cmos_diff;
-		ts.tv_nsec = 0;
-		do_settimeofday(&ts);
-	} 
-}
-
-static void get_time_diff(void)
-{
-#ifndef CONFIG_APM_RTC_IS_GMT
-	/*
-	 * Estimate time zone so that set_time can update the clock
-	 */
-	clock_cmos_diff = -get_cmos_time();
-	clock_cmos_diff += get_seconds();
-	got_clock_diff = 1;
-#endif
-}
-
 static void reinit_timer(void)
 {
 #ifdef INIT_TIMER_AFTER_SUSPEND
@@ -1214,19 +1192,6 @@ static int suspend(int vetoable)
 	local_irq_disable();
 	device_power_down(PMSG_SUSPEND);
 
-	/* serialize with the timer interrupt */
-	write_seqlock(&xtime_lock);
-
-	/* protect against access to timer chip registers */
-	spin_lock(&i8253_lock);
-
-	get_time_diff();
-	/*
-	 * Irq spinlock must be dropped around set_system_power_state.
-	 * We'll undo any timer changes due to interrupts below.
-	 */
-	spin_unlock(&i8253_lock);
-	write_sequnlock(&xtime_lock);
 	local_irq_enable();
 
 	save_processor_state();
@@ -1235,7 +1200,6 @@ static int suspend(int vetoable)
 	restore_processor_state();
 
 	local_irq_disable();
-	set_time();
 	reinit_timer();
 
 	if (err == APM_NO_ERROR)
@@ -1265,11 +1229,6 @@ static void standby(void)
 
 	local_irq_disable();
 	device_power_down(PMSG_SUSPEND);
-	/* serialize with the timer interrupt */
-	write_seqlock(&xtime_lock);
-	/* If needed, notify drivers here */
-	get_time_diff();
-	write_sequnlock(&xtime_lock);
 	local_irq_enable();
 
 	err = set_system_power_state(APM_STATE_STANDBY);
@@ -1363,7 +1322,6 @@ static void check_events(void)
 			ignore_bounce = 1;
 			if ((event != APM_NORMAL_RESUME)
 			    || (ignore_normal_resume == 0)) {
-				set_time();
 				device_resume();
 				pm_send_all(PM_RESUME, (void *)0);
 				queue_event(event, NULL);
@@ -1379,7 +1337,6 @@ static void check_events(void)
 			break;
 
 		case APM_UPDATE_TIME:
-			set_time();
 			break;
 
 		case APM_CRITICAL_SUSPEND:
arch/i386/kernel/time.c +2 -53

103 lines not on this page · raw message

#9 lore raw

Re: [patch 03/23] GTOD: persistent clock support, i386

Quoted 22 lines · On Mon, 02 Oct 2006 15:03:37 -0700 john stultz <[email protected]> wrote:

static int timer_resume(struct sys_device *dev)
{

arch/i386/kernel/time.c +0 -15
-	unsigned long flags;
-	unsigned long sec;
-	unsigned long ctime = get_cmos_time();
-	long sleep_length = (ctime - sleep_start) * HZ;
-	struct timespec ts;
-
-	if (sleep_length < 0) {
-		printk(KERN_WARNING "CMOS clock skew detected in timer resume!\n");
-		/* The time after the resume must not be earlier than the time
-		 * before the suspend or some nasty things will happen
-		 */
-		sleep_length = 0;
-		ctime = sleep_start;
-	}
 #ifdef CONFIG_HPET_TIMER
 	if (is_hpet_enabled())
 		hpet_reenable();
 #endif
 	setup_pit_timer();
-

arch/i386/kernel/time.c void notify_arch_cmos_timer(void) · line 295

Quoted 10 lines
arch/i386/kernel/time.c +0 -7
-	sec = ctime + clock_cmos_diff;
-	ts.tv_sec = sec;
-	ts.tv_nsec = 0;
-	do_settimeofday(&ts);
-	write_seqlock_irqsave(&xtime_lock, flags);
-	jiffies_64 += sleep_length;
-	write_sequnlock_irqrestore(&xtime_lock, flags);
 	touch_softlockup_watchdog();
 	return 0;
 }

In this version of the patch, you no longer remove the touch_softlockup_watchdog() call from timer_resume().

But clockevents-drivers-for-i386.patch deletes timer_resume() altogether.

Hence we might need to put that re-added touch_softlockup_watchdog() call into somewhere else now.

#10 lore raw

Re: [patch 03/23] GTOD: persistent clock support, i386

Quoted 30 lines · Andrew Morton
Quoted 27 lines · On Mon, 02 Oct 2006 15:03:37 -0700 john stultz <[email protected]> wrote:

static int timer_resume(struct sys_device *dev)
{

arch/i386/kernel/time.c +0 -20
-	unsigned long flags;
-	unsigned long sec;
-	unsigned long ctime = get_cmos_time();
-	long sleep_length = (ctime - sleep_start) * HZ;
-	struct timespec ts;
-
-	if (sleep_length < 0) {
-		printk(KERN_WARNING "CMOS clock skew detected in timer resume!\n");
-		/* The time after the resume must not be earlier than the time
-		 * before the suspend or some nasty things will happen
-		 */
-		sleep_length = 0;
-		ctime = sleep_start;
-	}
 #ifdef CONFIG_HPET_TIMER
 	if (is_hpet_enabled())
 		hpet_reenable();
 #endif
 	setup_pit_timer();
-
-	sec = ctime + clock_cmos_diff;
-	ts.tv_sec = sec;
-	ts.tv_nsec = 0;
-	do_settimeofday(&ts);
-	write_seqlock_irqsave(&xtime_lock, flags);
Quoted 8 lines

arch/i386/kernel/time.c void notify_arch_cmos_timer(void) · line 295

Quoted 5 lines
arch/i386/kernel/time.c +0 -2
-	jiffies_64 += sleep_length;
-	write_sequnlock_irqrestore(&xtime_lock, flags);
 	touch_softlockup_watchdog();
 	return 0;
 }

In this version of the patch, you no longer remove the touch_softlockup_watchdog() call from timer_resume().

Yea. That removal was added by Thomas, and I didn't merge it into my tree.

Quoted 5 lines

But clockevents-drivers-for-i386.patch deletes timer_resume() altogether.

Hence we might need to put that re-added touch_softlockup_watchdog() call into somewhere else now.

Yea. While my dropping the change wasn't intentional, it seems the change isn't really part of the persistent_clock changes, so the removal should be done in one of the clockevents patches.

But I'll have to defer to tglx as to which one.

thanks
-john

#11 lore raw

Re: [patch 03/23] GTOD: persistent clock support, i386

In reply to Andrew Morton
Quoted 10 lines · Andrew Morton

arch/i386/kernel/time.c void notify_arch_cmos_timer(void) · line 295

Quoted 6 lines
arch/i386/kernel/time.c +0 -3
-	write_seqlock_irqsave(&xtime_lock, flags);
-	jiffies_64 += sleep_length;
-	write_sequnlock_irqrestore(&xtime_lock, flags);
 	touch_softlockup_watchdog();
 	return 0;
 }

In this version of the patch, you no longer remove the touch_softlockup_watchdog() call from timer_resume().

Quoted 5 lines

But clockevents-drivers-for-i386.patch deletes timer_resume() altogether.

Hence we might need to put that re-added touch_softlockup_watchdog() call into somewhere else now.

clockevents has is it in the resume path.

static void clockevents_resume_local_events(void *arg) {
....

        touch_softlockup_watchdog();
}

tglx

#12 lore raw

PATCH 4/23 time: uninline jiffies.h

In reply to Thomas Gleixner

From: Ingo Molnar <[email protected]>

there are load of fat functions hidden in jiffies.h. Uninline them. No code changes.

--

 include/linux/jiffies.h |  223 +++---------------------------------------------
 kernel/time.c           |  218 ++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 234 insertions(+), 207 deletions(-)
include/linux/jiffies.h +16 -207
Index: linux-2.6.18-mm2/include/linux/jiffies.h
===================================================================
@@ -259,215 +259,24 @@ static inline u64 get_jiffies_64(void)
 #endif
 
 /*
- * Convert jiffies to milliseconds and back.
- *
- * Avoid unnecessary multiplications/divisions in the
- * two most common HZ cases:
+ * Convert various time units to each other:
  */
-static inline unsigned int jiffies_to_msecs(const unsigned long j)
-{
-#if HZ <= MSEC_PER_SEC && !(MSEC_PER_SEC % HZ)
-	return (MSEC_PER_SEC / HZ) * j;
-#elif HZ > MSEC_PER_SEC && !(HZ % MSEC_PER_SEC)
-	return (j + (HZ / MSEC_PER_SEC) - 1)/(HZ / MSEC_PER_SEC);
-#else
-	return (j * MSEC_PER_SEC) / HZ;
-#endif
-}
-
-static inline unsigned int jiffies_to_usecs(const unsigned long j)
-{
-#if HZ <= USEC_PER_SEC && !(USEC_PER_SEC % HZ)
-	return (USEC_PER_SEC / HZ) * j;
-#elif HZ > USEC_PER_SEC && !(HZ % USEC_PER_SEC)
-	return (j + (HZ / USEC_PER_SEC) - 1)/(HZ / USEC_PER_SEC);
-#else
-	return (j * USEC_PER_SEC) / HZ;
-#endif
-}
-
-static inline unsigned long msecs_to_jiffies(const unsigned int m)
-{
-	if (m > jiffies_to_msecs(MAX_JIFFY_OFFSET))
-		return MAX_JIFFY_OFFSET;
-#if HZ <= MSEC_PER_SEC && !(MSEC_PER_SEC % HZ)
-	return (m + (MSEC_PER_SEC / HZ) - 1) / (MSEC_PER_SEC / HZ);
-#elif HZ > MSEC_PER_SEC && !(HZ % MSEC_PER_SEC)
-	return m * (HZ / MSEC_PER_SEC);
-#else
-	return (m * HZ + MSEC_PER_SEC - 1) / MSEC_PER_SEC;
-#endif
-}
-
-static inline unsigned long usecs_to_jiffies(const unsigned int u)
-{
-	if (u > jiffies_to_usecs(MAX_JIFFY_OFFSET))
-		return MAX_JIFFY_OFFSET;
-#if HZ <= USEC_PER_SEC && !(USEC_PER_SEC % HZ)
-	return (u + (USEC_PER_SEC / HZ) - 1) / (USEC_PER_SEC / HZ);
-#elif HZ > USEC_PER_SEC && !(HZ % USEC_PER_SEC)
-	return u * (HZ / USEC_PER_SEC);
-#else
-	return (u * HZ + USEC_PER_SEC - 1) / USEC_PER_SEC;
-#endif
-}
-
-/*
- * The TICK_NSEC - 1 rounds up the value to the next resolution.  Note
- * that a remainder subtract here would not do the right thing as the
- * resolution values don't fall on second boundries.  I.e. the line:
- * nsec -= nsec % TICK_NSEC; is NOT a correct resolution rounding.
- *
- * Rather, we just shift the bits off the right.
- *
- * The >> (NSEC_JIFFIE_SC - SEC_JIFFIE_SC) converts the scaled nsec
- * value to a scaled second value.
- */
-static __inline__ unsigned long
-timespec_to_jiffies(const struct timespec *value)
-{
-	unsigned long sec = value->tv_sec;
-	long nsec = value->tv_nsec + TICK_NSEC - 1;
-
-	if (sec >= MAX_SEC_IN_JIFFIES){
-		sec = MAX_SEC_IN_JIFFIES;
-		nsec = 0;
-	}
-	return (((u64)sec * SEC_CONVERSION) +
-		(((u64)nsec * NSEC_CONVERSION) >>
-		 (NSEC_JIFFIE_SC - SEC_JIFFIE_SC))) >> SEC_JIFFIE_SC;
-
-}
-
-static __inline__ void
-jiffies_to_timespec(const unsigned long jiffies, struct timespec *value)
-{
-	/*
-	 * Convert jiffies to nanoseconds and separate with
-	 * one divide.
-	 */
-	u64 nsec = (u64)jiffies * TICK_NSEC;
-	value->tv_sec = div_long_long_rem(nsec, NSEC_PER_SEC, &value->tv_nsec);
-}
-
-/* Same for "timeval"
- *
- * Well, almost.  The problem here is that the real system resolution is
- * in nanoseconds and the value being converted is in micro seconds.
- * Also for some machines (those that use HZ = 1024, in-particular),
- * there is a LARGE error in the tick size in microseconds.
-
- * The solution we use is to do the rounding AFTER we convert the
- * microsecond part.  Thus the USEC_ROUND, the bits to be shifted off.
- * Instruction wise, this should cost only an additional add with carry
- * instruction above the way it was done above.
- */
-static __inline__ unsigned long
-timeval_to_jiffies(const struct timeval *value)
-{
-	unsigned long sec = value->tv_sec;
-	long usec = value->tv_usec;
-
-	if (sec >= MAX_SEC_IN_JIFFIES){
-		sec = MAX_SEC_IN_JIFFIES;
-		usec = 0;
-	}
-	return (((u64)sec * SEC_CONVERSION) +
-		(((u64)usec * USEC_CONVERSION + USEC_ROUND) >>
-		 (USEC_JIFFIE_SC - SEC_JIFFIE_SC))) >> SEC_JIFFIE_SC;
-}
-
-static __inline__ void
-jiffies_to_timeval(const unsigned long jiffies, struct timeval *value)
-{
-	/*
-	 * Convert jiffies to nanoseconds and separate with
-	 * one divide.
-	 */
-	u64 nsec = (u64)jiffies * TICK_NSEC;
-	long tv_usec;
-
-	value->tv_sec = div_long_long_rem(nsec, NSEC_PER_SEC, &tv_usec);
-	tv_usec /= NSEC_PER_USEC;
-	value->tv_usec = tv_usec;
-}
-
-/*
- * Convert jiffies/jiffies_64 to clock_t and back.
- */
-static inline clock_t jiffies_to_clock_t(long x)
-{
-#if (TICK_NSEC % (NSEC_PER_SEC / USER_HZ)) == 0
-	return x / (HZ / USER_HZ);
-#else
-	u64 tmp = (u64)x * TICK_NSEC;
-	do_div(tmp, (NSEC_PER_SEC / USER_HZ));
-	return (long)tmp;
-#endif
-}
-
-static inline unsigned long clock_t_to_jiffies(unsigned long x)
-{
-#if (HZ % USER_HZ)==0
-	if (x >= ~0UL / (HZ / USER_HZ))
-		return ~0UL;
-	return x * (HZ / USER_HZ);
-#else
-	u64 jif;
-
-	/* Don't worry about loss of precision here .. */
-	if (x >= ~0UL / HZ * USER_HZ)
-		return ~0UL;
-
-	/* .. but do try to contain it here */
-	jif = x * (u64) HZ;
-	do_div(jif, USER_HZ);
-	return jif;
-#endif
-}
-
-static inline u64 jiffies_64_to_clock_t(u64 x)
-{
-#if (TICK_NSEC % (NSEC_PER_SEC / USER_HZ)) == 0
-	do_div(x, HZ / USER_HZ);
-#else
-	/*
-	 * There are better ways that don't overflow early,
-	 * but even this doesn't overflow in hundreds of years
-	 * in 64 bits, so..
-	 */
-	x *= TICK_NSEC;
-	do_div(x, (NSEC_PER_SEC / USER_HZ));
-#endif
-	return x;
-}
-
-static inline u64 nsec_to_clock_t(u64 x)
-{
-#if (NSEC_PER_SEC % USER_HZ) == 0
-	do_div(x, (NSEC_PER_SEC / USER_HZ));
-#elif (USER_HZ % 512) == 0
-	x *= USER_HZ/512;
-	do_div(x, (NSEC_PER_SEC / 512));
-#else
-	/*
-         * max relative error 5.7e-8 (1.8s per year) for USER_HZ <= 1024,
-         * overflow after 64.99 years.
-         * exact for HZ=60, 72, 90, 120, 144, 180, 300, 600, 900, ...
-         */
-	x *= 9;
-	do_div(x, (unsigned long)((9ull * NSEC_PER_SEC + (USER_HZ/2))
-	                          / USER_HZ));
-#endif
-	return x;
-}
+extern unsigned int jiffies_to_msecs(const unsigned long j);
+extern unsigned int jiffies_to_usecs(const unsigned long j);
+extern unsigned long msecs_to_jiffies(const unsigned int m);
+extern unsigned long usecs_to_jiffies(const unsigned int u);
+extern unsigned long timespec_to_jiffies(const struct timespec *value);
+extern void jiffies_to_timespec(const unsigned long jiffies,
+				struct timespec *value);
+extern unsigned long timeval_to_jiffies(const struct timeval *value);
+extern void jiffies_to_timeval(const unsigned long jiffies,
+			       struct timeval *value);
+extern clock_t jiffies_to_clock_t(long x);
+extern unsigned long clock_t_to_jiffies(unsigned long x);
+extern u64 jiffies_64_to_clock_t(u64 x);
+extern u64 nsec_to_clock_t(u64 x);
+extern int nsec_to_timestamp(char *s, u64 t);
 
-static inline int nsec_to_timestamp(char *s, u64 t)
-{
-	unsigned long nsec_rem = do_div(t, NSEC_PER_SEC);
-	return sprintf(s, "[%5lu.%06lu]", (unsigned long)t,
-		       nsec_rem/NSEC_PER_USEC);
-}
 #define TIMESTAMP_SIZE	30
 
 #endif
kernel/time.c +218 -0

224 lines not on this page · raw message

--

#13 lore raw

PATCH 5/23 time: fix msecs_to_jiffies() bug

In reply to Thomas Gleixner

From: Ingo Molnar <[email protected]>

fix multiple conversion bugs in msecs_to_jiffies().

the main problem is that this condition:

if (m > jiffies_to_msecs(MAX_JIFFY_OFFSET))

overflows if HZ is smaller than 1000!

this change is user-visible: for HZ=250 SUS-compliant poll()-timeout value of -20 is mistakenly converted to 'immediate timeout'.

(the new dyntick code also triggered this, as it frequently creates 'lagging timer wheel' scenarios.)

--

 kernel/time.c |   43 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 42 insertions(+), 1 deletion(-)
kernel/time.c +42 -1
Index: linux-2.6.18-mm2/kernel/time.c
===================================================================
@@ -500,15 +500,56 @@ unsigned int jiffies_to_usecs(const unsi
 }
 EXPORT_SYMBOL(jiffies_to_usecs);
 
+/*
+ * When we convert to jiffies then we interpret incoming values
+ * the following way:
+ *
+ * - negative values mean 'infinite timeout' (MAX_JIFFY_OFFSET)
+ *
+ * - 'too large' values [that would result in larger than
+ *   MAX_JIFFY_OFFSET values] mean 'infinite timeout' too.
+ *
+ * - all other values are converted to jiffies by either multiplying
+ *   the input value by a factor or dividing it with a factor
+ *
+ * We must also be careful about 32-bit overflows.
+ */
 unsigned long msecs_to_jiffies(const unsigned int m)
 {
-	if (m > jiffies_to_msecs(MAX_JIFFY_OFFSET))
+	/*
+	 * Negative value, means infinite timeout:
+	 */
+	if ((int)m < 0)
 		return MAX_JIFFY_OFFSET;
+
 #if HZ <= MSEC_PER_SEC && !(MSEC_PER_SEC % HZ)
+	/*
+	 * HZ is equal to or smaller than 1000, and 1000 is a nice
+	 * round multiple of HZ, divide with the factor between them,
+	 * but round upwards:
+	 */
 	return (m + (MSEC_PER_SEC / HZ) - 1) / (MSEC_PER_SEC / HZ);
 #elif HZ > MSEC_PER_SEC && !(HZ % MSEC_PER_SEC)
+	/*
+	 * HZ is larger than 1000, and HZ is a nice round multiple of
+	 * 1000 - simply multiply with the factor between them.
+	 *
+	 * But first make sure the multiplication result cannot
+	 * overflow:
+	 */
+	if (m > jiffies_to_msecs(MAX_JIFFY_OFFSET))
+		return MAX_JIFFY_OFFSET;
+
 	return m * (HZ / MSEC_PER_SEC);
 #else
+	/*
+	 * Generic case - multiply, round and divide. But first
+	 * check that if we are doing a net multiplication, that
+	 * we wouldnt overflow:
+	 */
+	if (HZ > MSEC_PER_SEC && m > jiffies_to_msecs(MAX_JIFFY_OFFSET))
+		return MAX_JIFFY_OFFSET;
+
 	return (m * HZ + MSEC_PER_SEC - 1) / MSEC_PER_SEC;
 #endif
 }

--

#14 lore raw

PATCH 6/23 time: fix timeout overflow

In reply to Thomas Gleixner

From: Ingo Molnar <[email protected]>

prevent timeout overflow if timer ticks are behind jiffies (due to high softirq load or due to dyntick), by limiting the valid timeout range to MAX_LONG/2.

--

 include/linux/jiffies.h |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
include/linux/jiffies.h +2 -2
Index: linux-2.6.18-mm2/include/linux/jiffies.h
===================================================================
@@ -142,13 +142,13 @@ static inline u64 get_jiffies_64(void)
  *
  * And some not so obvious.
  *
- * Note that we don't want to return MAX_LONG, because
+ * Note that we don't want to return LONG_MAX, because
  * for various timeout reasons we often end up having
  * to wait "jiffies+1" in order to guarantee that we wait
  * at _least_ "jiffies" - so "jiffies+1" had better still
  * be positive.
  */
-#define MAX_JIFFY_OFFSET ((~0UL >> 1)-1)
+#define MAX_JIFFY_OFFSET ((LONG_MAX >> 1)-1)
 
 /*
  * We want to do realistic conversions of time so we need to use the same

--

#15 lore raw

PATCH 7/23 cleanup: uninline irq_enter() and move it into a function

In reply to Thomas Gleixner

From: Ingo Molnar <[email protected]>

uninline irq_enter(). [dynticks adds more stuff to it]

--

 include/linux/hardirq.h |    7 +------
 kernel/softirq.c        |    7 +++++++
 2 files changed, 8 insertions(+), 6 deletions(-)
include/linux/hardirq.h +1 -6
Index: linux-2.6.18-mm2/include/linux/hardirq.h
===================================================================
@@ -106,12 +106,7 @@ static inline void account_system_vtime(
  * always balanced, so the interrupted value of ->hardirq_context
  * will always be restored.
  */
-#define irq_enter()					\
-	do {						\
-		account_system_vtime(current);		\
-		add_preempt_count(HARDIRQ_OFFSET);	\
-		trace_hardirq_enter();			\
-	} while (0)
+extern void irq_enter(void);
 
 /*
  * Exit irq context without processing softirqs:
kernel/softirq.c +7 -0
Index: linux-2.6.18-mm2/kernel/softirq.c
===================================================================
@@ -279,6 +279,13 @@ EXPORT_SYMBOL(do_softirq);
 # define invoke_softirq()	do_softirq()
 #endif
 
+extern void irq_enter(void)
+{
+	account_system_vtime(current);
+	add_preempt_count(HARDIRQ_OFFSET);
+	trace_hardirq_enter();
+}
+
 /*
  * Exit an interrupt context. Process softirqs if needed and possible:
  */

--

#16 lore raw

Re: [patch 07/23] cleanup: uninline irq_enter() and move it into a function

kernel/softirq.c EXPORT_SYMBOL(do_softirq); · line 282

Quoted 7 lines · On Fri, 29 Sep 2006 23:58:26 -0000 Thomas Gleixner <[email protected]> wrote:
kernel/softirq.c +1 -0
@@ -279,6 +279,13 @@ EXPORT_SYMBOL(do_softirq);
 # define invoke_softirq()	do_softirq()
 #endif
 
+extern void irq_enter(void)

unneeded `extern'.

#17 lore raw

PATCH 8/23 dynticks: prepare the RCU code

In reply to Thomas Gleixner

From: Ingo Molnar <[email protected]>

prepare the RCU code for dynticks/nohz. Since on nohz kernels there is no guaranteed timer IRQ that processes RCU callbacks, the idle code has to make sure that all RCU callbacks that can be finished off are indeed finished off. This patch adds the necessary APIs:
rcu_advance_callbacks() [to register quiescent state] and rcu_process_callbacks() [to finish finishable RCU callbacks].

--

 include/linux/rcupdate.h |    2 ++
 kernel/rcupdate.c        |   13 ++++++++++++-
 2 files changed, 14 insertions(+), 1 deletion(-)
include/linux/rcupdate.h +2 -0
Index: linux-2.6.18-mm2/include/linux/rcupdate.h
===================================================================
@@ -271,6 +271,7 @@ extern int rcu_needs_cpu(int cpu);
 
 extern void rcu_init(void);
 extern void rcu_check_callbacks(int cpu, int user);
+extern void rcu_advance_callbacks(int cpu, int user);
 extern void rcu_restart_cpu(int cpu);
 extern long rcu_batches_completed(void);
 extern long rcu_batches_completed_bh(void);
@@ -283,6 +284,7 @@ extern void FASTCALL(call_rcu_bh(struct 
 extern void synchronize_rcu(void);
 void synchronize_idle(void);
 extern void rcu_barrier(void);
+extern void rcu_process_callbacks(unsigned long unused);
 
 #endif /* __KERNEL__ */
 #endif /* __LINUX_RCUPDATE_H */
kernel/rcupdate.c +12 -1
Index: linux-2.6.18-mm2/kernel/rcupdate.c
===================================================================
@@ -460,7 +460,7 @@ static void __rcu_process_callbacks(stru
 		rcu_do_batch(rdp);
 }
 
-static void rcu_process_callbacks(unsigned long unused)
+void rcu_process_callbacks(unsigned long unused)
 {
 	__rcu_process_callbacks(&rcu_ctrlblk, &__get_cpu_var(rcu_data));
 	__rcu_process_callbacks(&rcu_bh_ctrlblk, &__get_cpu_var(rcu_bh_data));
@@ -515,6 +515,17 @@ int rcu_needs_cpu(int cpu)
 	return (!!rdp->curlist || !!rdp_bh->curlist || rcu_pending(cpu));
 }
 
+void rcu_advance_callbacks(int cpu, int user)
+{
+	if (user ||
+	    (idle_cpu(cpu) && !in_softirq() &&
+				hardirq_count() <= (1 << HARDIRQ_SHIFT))) {
+		rcu_qsctr_inc(cpu);
+		rcu_bh_qsctr_inc(cpu);
+	} else if (!in_softirq())
+		rcu_bh_qsctr_inc(cpu);
+}
+
 void rcu_check_callbacks(int cpu, int user)
 {
 	if (user || 

--

#18 lore raw

Re: [patch 08/23] dynticks: prepare the RCU code

Quoted 12 lines · On Fri, 29 Sep 2006 23:58:27 -0000 Thomas Gleixner <[email protected]> wrote:

From: Ingo Molnar <[email protected]>

prepare the RCU code for dynticks/nohz. Since on nohz kernels there is no guaranteed timer IRQ that processes RCU callbacks, the idle code has to make sure that all RCU callbacks that can be finished off are indeed finished off. This patch adds the necessary APIs:
rcu_advance_callbacks() [to register quiescent state] and rcu_process_callbacks() [to finish finishable RCU callbacks].

...

kernel/rcupdate.c +1 -0
+void rcu_advance_callbacks(int cpu, int user)

kernel/rcupdate.c int rcu_needs_cpu(int cpu) · line 528

Quoted 10 lines
kernel/rcupdate.c +10 -0
+{
+	if (user ||
+	    (idle_cpu(cpu) && !in_softirq() &&
+				hardirq_count() <= (1 << HARDIRQ_SHIFT))) {
+		rcu_qsctr_inc(cpu);
+		rcu_bh_qsctr_inc(cpu);
+	} else if (!in_softirq())
+		rcu_bh_qsctr_inc(cpu);
+}
+

I hope this function is immediately clear to the RCU maintainers, because it's complete mud to me.

An introductory comment which describes what this function does and how it does it seems appropriate. And some words which decrypt the tests in there might be needed too.

#19 lore raw

Re: [patch 08/23] dynticks: prepare the RCU code

Quoted 14 lines · Andrew Morton
Quoted 11 lines · On Fri, 29 Sep 2006 23:58:27 -0000 Thomas Gleixner <[email protected]> wrote:

From: Ingo Molnar <[email protected]>

prepare the RCU code for dynticks/nohz. Since on nohz kernels there is no guaranteed timer IRQ that processes RCU callbacks, the idle code has to make sure that all RCU callbacks that can be finished off are indeed finished off. This patch adds the necessary APIs:
rcu_advance_callbacks() [to register quiescent state] and rcu_process_callbacks() [to finish finishable RCU callbacks].

...

Quoted 15 lines

kernel/rcupdate.c int rcu_needs_cpu(int cpu) · line 528

Quoted 11 lines
kernel/rcupdate.c +11 -0
+void rcu_advance_callbacks(int cpu, int user)
+{
+	if (user ||
+	    (idle_cpu(cpu) && !in_softirq() &&
+				hardirq_count() <= (1 << HARDIRQ_SHIFT))) {
+		rcu_qsctr_inc(cpu);
+		rcu_bh_qsctr_inc(cpu);
+	} else if (!in_softirq())
+		rcu_bh_qsctr_inc(cpu);
+}
+

I hope this function is immediately clear to the RCU maintainers, because it's complete mud to me.

Ingo,

It is duplicating code. That can be easily fixed, but we need to figure out what we really want from RCU when we are about to switch off the ticks. It is hard if you want to finish off all the pending RCUs and go to nohz state. Can you live with backing out if there are pending RCUs ?

Thanks
Dipankar

#20 lore raw

Re: [patch 08/23] dynticks: prepare the RCU code

Quoted 5 lines · Dipankar Sarma

It is duplicating code. That can be easily fixed, but we need to figure out what we really want from RCU when we are about to switch off the ticks. It is hard if you want to finish off all the pending RCUs and go to nohz state. Can you live with backing out if there are pending RCUs ?

the thing is that when we go idle we /want/ to process whatever delayed work there might be - rate limited or not. Do you agree with that approach? I consider this a performance feature as well: this way we can utilize otherwise lost idle time. It is not a problem that we dont 'batch' this processing: we are really idle and we've got free cycles to burn. We could even do an RCU processing loop that immediately breaks out if need_resched() gets set [by an IRQ or by another CPU].

secondly, i think i saw functionality problems when RCU was not completed before going idle - for example synchronize_rcu() on another CPU would hang.

what approach would you suggest to achieve these goals?

Ingo

#21 lore raw

Re: [patch 08/23] dynticks: prepare the RCU code

Quoted 8 lines · Ingo Molnar
Quoted 5 lines · Dipankar Sarma

It is duplicating code. That can be easily fixed, but we need to figure out what we really want from RCU when we are about to switch off the ticks. It is hard if you want to finish off all the pending RCUs and go to nohz state. Can you live with backing out if there are pending RCUs ?

Quoted 7 lines

the thing is that when we go idle we /want/ to process whatever delayed work there might be - rate limited or not. Do you agree with that approach? I consider this a performance feature as well: this way we can utilize otherwise lost idle time. It is not a problem that we dont 'batch' this processing: we are really idle and we've got free cycles to burn. We could even do an RCU processing loop that immediately breaks out if need_resched() gets set [by an IRQ or by another CPU].

If you don't care about rate limiting RCU processing (you wouldn't in CONFIG_PREEMPT_RT), you still have to deal with the situation that one CPU going idle doesn't guarantee that you can process all pending RCUs. You can process the finished ones, but what about the ones that are still waiting for the grace period beyond the current cpu ?

Quoted 4 lines

secondly, i think i saw functionality problems when RCU was not completed before going idle - for example synchronize_rcu() on another CPU would hang.

That is probably because of what I mention above. In the original CONFIG_NO_IDLE_HZ, we don't go into a nohz state if there are RCUs pending in that cpu.

Quoted 2 lines

what approach would you suggest to achieve these goals?

There is no way to finish all the RCUs in a given cpu unless you are prepared to wait for a grace period or so. So, you go to idle, keep checking in the timer tick and as soon as all RCUs are done, go to nohz state. You can keep processing RCUs in every idle tick so that if you have only finished RCU callbacks on that cpu, you can go to nohz rightaway. I can add that API.

Of course, you can do what cpu hotplug does - move the RCUs to another CPU. But that is an expensive operaation.

Thanks
Dipankar

#22 lore raw

Re: [patch 08/23] dynticks: prepare the RCU code

Quoted 7 lines · Dipankar Sarma
Quoted 3 lines

secondly, i think i saw functionality problems when RCU was not completed before going idle - for example synchronize_rcu() on another CPU would hang.

That is probably because of what I mention above. In the original CONFIG_NO_IDLE_HZ, we don't go into a nohz state if there are RCUs pending in that cpu.

hm. I just tried it and it seems completing RCU processing isnt even necessary. I'll drop the RCU hackery. If we need anything then in synchronize_rcu [which is a rare and slowpath op]: there (on NO_HZ) we should tickle all cpus via an smp_call_function().

Ingo

#23 lore raw

PATCH 9/23 dynticks: extend next_timer_interrupt() to use a reference jiffie

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

For CONFIG_NO_HZ we need to calculate the next timer wheel event based to a given jiffie value. Extend the existing code to allow the extra now argument. Provide a compability function for the existing implementations to call the function with now = jiffies.
This also solves the racyness of the original code vs. jiffies changing during the iteration.

----

 include/linux/timer.h |   10 +++++++
 kernel/timer.c        |   64 +++++++++++++++++++++++++++++++++++---------------
 2 files changed, 56 insertions(+), 18 deletions(-)
include/linux/timer.h +10 -0
Index: linux-2.6.18-mm2/include/linux/timer.h
===================================================================
@@ -61,7 +61,17 @@ extern int del_timer(struct timer_list *
 extern int __mod_timer(struct timer_list *timer, unsigned long expires);
 extern int mod_timer(struct timer_list *timer, unsigned long expires);
 
+/*
+ * Return when the next timer-wheel timeout occurs (in absolute jiffies),
+ * locks the timer base:
+ */
 extern unsigned long next_timer_interrupt(void);
+/*
+ * Return when the next timer-wheel timeout occurs (in absolute jiffies),
+ * locks the timer base and does the comparison against the given
+ * jiffie.
+ */
+extern unsigned long get_next_timer_interrupt(unsigned long now);
 
 /***
  * add_timer - start a timer
kernel/timer.c +46 -18

116 lines not on this page · raw message

--

#24 lore raw

Re: [patch 09/23] dynticks: extend next_timer_interrupt() to use a reference jiffie

Quoted 9 lines · On Fri, 29 Sep 2006 23:58:28 -0000 Thomas Gleixner <[email protected]> wrote:

From: Thomas Gleixner <[email protected]>

For CONFIG_NO_HZ we need to calculate the next timer wheel event based to a given jiffie value. Extend the existing code to allow the extra now argument. Provide a compability function for the existing implementations to call the function with now = jiffies.
This also solves the racyness of the original code vs. jiffies changing during the iteration.

I think this change has the potential to significantly increase the hold time of tvec_base_t.lock. Quite a lot of code has been moved inside that lock, but most worrisome is that hrtimer_get_next_event() is also now inside it.

What workloads is this change likely to impact, and how can we set about verifying that we aren't introducing a problem?

Was that (unchangelogged) locking change even needed? If so, why?

#25 lore raw

PATCH 10/23 hrtimers: clean up locking

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

improve kernel/hrtimers.c locking: use a per-CPU base with a lock to control locking of all clocks belonging to a CPU. This simplifies code that needs to lock all clocks at once. This makes life easier for high-res timers and dyntick. No functional change should happen due to this.

--

 include/linux/hrtimer.h |   37 ++++++---
 kernel/hrtimer.c        |  181 +++++++++++++++++++++++++-----------------------
 2 files changed, 121 insertions(+), 97 deletions(-)
include/linux/hrtimer.h +25 -12
Index: linux-2.6.18-mm2/include/linux/hrtimer.h
===================================================================
@@ -36,7 +36,7 @@ enum hrtimer_restart {
 
 #define HRTIMER_INACTIVE	((void *)1UL)
 
-struct hrtimer_base;
+struct hrtimer_clock_base;
 
 /**
  * struct hrtimer - the basic hrtimer structure
@@ -50,10 +50,10 @@ struct hrtimer_base;
  * The hrtimer structure must be initialized by init_hrtimer_#CLOCKTYPE()
  */
 struct hrtimer {
-	struct rb_node		node;
-	ktime_t			expires;
-	int			(*function)(struct hrtimer *);
-	struct hrtimer_base	*base;
+	struct rb_node			node;
+	ktime_t				expires;
+	int				(*function)(struct hrtimer *);
+	struct hrtimer_clock_base	*base;
 };
 
 /**
@@ -68,31 +68,44 @@ struct hrtimer_sleeper {
 	struct task_struct *task;
 };
 
+struct hrtimer_cpu_base;
+
 /**
  * struct hrtimer_base - the timer base for a specific clock
  * @index:		clock type index for per_cpu support when moving a timer
  *			to a base on another cpu.
- * @lock:		lock protecting the base and associated timers
  * @active:		red black tree root node for the active timers
  * @first:		pointer to the timer node which expires first
  * @resolution:		the resolution of the clock, in nanoseconds
  * @get_time:		function to retrieve the current time of the clock
  * @get_softirq_time:	function to retrieve the current time from the softirq
- * @curr_timer:		the timer which is executing a callback right now
  * @softirq_time:	the time when running the hrtimer queue in the softirq
- * @lock_key:		the lock_class_key for use with lockdep
  */
-struct hrtimer_base {
+struct hrtimer_clock_base {
+	struct hrtimer_cpu_base	*cpu_base;
 	clockid_t		index;
-	spinlock_t		lock;
 	struct rb_root		active;
 	struct rb_node		*first;
 	ktime_t			resolution;
 	ktime_t			(*get_time)(void);
 	ktime_t			(*get_softirq_time)(void);
-	struct hrtimer		*curr_timer;
 	ktime_t			softirq_time;
-	struct lock_class_key lock_key;
+};
+
+#define HRTIMER_MAX_CLOCK_BASES 2
+
+/*
+ * struct hrtimer_cpu_base - the per cpu clock bases
+ * @lock:		lock protecting the base and associated clock bases and timers
+ * @lock_key:		the lock_class_key for use with lockdep
+ * @clock_base:		array of clock bases for this cpu
+ * @curr_timer:		the timer which is executing a callback right now
+ */
+struct hrtimer_cpu_base {
+	spinlock_t			lock;
+	struct lock_class_key		lock_key;
+	struct hrtimer_clock_base	clock_base[HRTIMER_MAX_CLOCK_BASES];
+	struct hrtimer			*curr_timer;
 };
 
 /*
kernel/hrtimer.c +96 -85

401 lines not on this page · raw message

--

#26 lore raw

Re: [patch 10/23] hrtimers: clean up locking

include/linux/hrtimer.h enum hrtimer_restart { · line 39

Quoted 12 lines · On Fri, 29 Sep 2006 23:58:29 -0000 Thomas Gleixner <[email protected]> wrote:

From: Thomas Gleixner <[email protected]>

improve kernel/hrtimers.c locking: use a per-CPU base with a lock to control locking of all clocks belonging to a CPU. This simplifies code that needs to lock all clocks at once. This makes life easier for high-res timers and dyntick. No functional change should happen due to this.

..

include/linux/hrtimer.h +1 -1
-struct hrtimer_base;
+struct hrtimer_clock_base;

It is better to place these forward declarations right at the top of the include file. That prevents people from later accidentally adding another forward declaration of the same structure at an earlier point in the file, and keeps all the same types of thing in the same place.

(two instances in this file)

Quoted 2 lines

+ * struct hrtimer_cpu_base - the per cpu clock bases
+ * @lock: lock protecting the base and associated clock bases and timers

Looks crappy in 80-cols. But I don't know if breaking this line will break kerneldoc?

#27 lore raw

PATCH 11/23 hrtimers: state tracking

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

reintroduce ktimers feature "optimized away" by the ktimers review process: multiple hrtimer states to enable the running of hrtimers without holding the cpu-base-lock.

(the "optimized" rbtree hack carried only 2 states worth of information and we need 3.)

--

 include/linux/hrtimer.h |    7 +++++--
 kernel/hrtimer.c        |   17 ++++++++++-------
 2 files changed, 15 insertions(+), 9 deletions(-)
include/linux/hrtimer.h +5 -2
Index: linux-2.6.18-mm2/include/linux/hrtimer.h
===================================================================
@@ -34,7 +34,9 @@ enum hrtimer_restart {
 	HRTIMER_RESTART,
 };
 
-#define HRTIMER_INACTIVE	((void *)1UL)
+#define HRTIMER_INACTIVE	0x00
+#define HRTIMER_ACTIVE		0x01
+#define HRTIMER_CALLBACK	0x02
 
 struct hrtimer_clock_base;
 
@@ -54,6 +56,7 @@ struct hrtimer {
 	ktime_t				expires;
 	int				(*function)(struct hrtimer *);
 	struct hrtimer_clock_base	*base;
+	unsigned long			state;
 };
 
 /**
@@ -139,7 +142,7 @@ extern ktime_t hrtimer_get_next_event(vo
 
 static inline int hrtimer_active(const struct hrtimer *timer)
 {
-	return rb_parent(&timer->node) != &timer->node;
+	return timer->state != HRTIMER_INACTIVE;
 }
 
 /* Forward a hrtimer so it expires after now: */
kernel/hrtimer.c +10 -7
Index: linux-2.6.18-mm2/kernel/hrtimer.c
===================================================================
@@ -384,6 +384,7 @@ static void enqueue_hrtimer(struct hrtim
 	 */
 	rb_link_node(&timer->node, parent, link);
 	rb_insert_color(&timer->node, &base->active);
+	timer->state |= HRTIMER_ACTIVE;
 
 	if (!base->first || timer->expires.tv64 <
 	    rb_entry(base->first, struct hrtimer, node)->expires.tv64)
@@ -396,7 +397,8 @@ static void enqueue_hrtimer(struct hrtim
  * Caller must hold the base lock.
  */
 static void __remove_hrtimer(struct hrtimer *timer,
-			     struct hrtimer_clock_base *base)
+			     struct hrtimer_clock_base *base,
+			     unsigned long newstate)
 {
 	/*
 	 * Remove the timer from the rbtree and replace the
@@ -405,7 +407,7 @@ static void __remove_hrtimer(struct hrti
 	if (base->first == &timer->node)
 		base->first = rb_next(&timer->node);
 	rb_erase(&timer->node, &base->active);
-	rb_set_parent(&timer->node, &timer->node);
+	timer->state = newstate;
 }
 
 /*
@@ -415,7 +417,7 @@ static inline int
 remove_hrtimer(struct hrtimer *timer, struct hrtimer_clock_base *base)
 {
 	if (hrtimer_active(timer)) {
-		__remove_hrtimer(timer, base);
+		__remove_hrtimer(timer, base, HRTIMER_INACTIVE);
 		return 1;
 	}
 	return 0;
@@ -487,7 +489,7 @@ int hrtimer_try_to_cancel(struct hrtimer
 
 	base = lock_hrtimer_base(timer, &flags);
 
-	if (base->cpu_base->curr_timer != timer)
+	if (!(timer->state & HRTIMER_CALLBACK))
 		ret = remove_hrtimer(timer, base);
 
 	unlock_hrtimer_base(timer, &flags);
@@ -592,7 +594,6 @@ void hrtimer_init(struct hrtimer *timer,
 		clock_id = CLOCK_MONOTONIC;
 
 	timer->base = &cpu_base->clock_base[clock_id];
-	rb_set_parent(&timer->node, &timer->node);
 }
 EXPORT_SYMBOL_GPL(hrtimer_init);
 
@@ -643,13 +644,14 @@ static inline void run_hrtimer_queue(str
 
 		fn = timer->function;
 		set_curr_timer(cpu_base, timer);
-		__remove_hrtimer(timer, base);
+		__remove_hrtimer(timer, base, HRTIMER_CALLBACK);
 		spin_unlock_irq(&cpu_base->lock);
 
 		restart = fn(timer);
 
 		spin_lock_irq(&cpu_base->lock);
 
+		timer->state &= ~HRTIMER_CALLBACK;
 		if (restart != HRTIMER_NORESTART) {
 			BUG_ON(hrtimer_active(timer));
 			enqueue_hrtimer(timer, base);
@@ -820,7 +822,8 @@ static void migrate_hrtimer_list(struct 
 
 	while ((node = rb_first(&old_base->active))) {
 		timer = rb_entry(node, struct hrtimer, node);
-		__remove_hrtimer(timer, old_base);
+		BUG_ON(timer->state & HRTIMER_CALLBACK);
+		__remove_hrtimer(timer, old_base, HRTIMER_INACTIVE);
 		timer->base = new_base;
 		enqueue_hrtimer(timer, new_base);
 	}

--

#28 lore raw

Re: [patch 11/23] hrtimers: state tracking

Quoted 9 lines · On Fri, 29 Sep 2006 23:58:30 -0000 Thomas Gleixner <[email protected]> wrote:

From: Thomas Gleixner <[email protected]>

reintroduce ktimers feature "optimized away" by the ktimers review process: multiple hrtimer states to enable the running of hrtimers without holding the cpu-base-lock.

(the "optimized" rbtree hack carried only 2 states worth of information and we need 3.)

uh, I'll believe you ;)

include/linux/hrtimer.h struct hrtimer { · line 59

Quoted 12 lines
include/linux/hrtimer.h +4 -1
-#define HRTIMER_INACTIVE	((void *)1UL)
+#define HRTIMER_INACTIVE	0x00
+#define HRTIMER_ACTIVE		0x01
+#define HRTIMER_CALLBACK	0x02
 
 struct hrtimer_clock_base;
@@ -54,6 +56,7 @@ struct hrtimer {
 	ktime_t				expires;
 	int				(*function)(struct hrtimer *);
 	struct hrtimer_clock_base	*base;
+	unsigned long			state;

I assume that `state' here takes the above enumerated values HRTIMER_*?

Using an enum would make that explicit, and more understandable.

Does it really need to be a long type?

Quoted 5 lines
include/linux/hrtimer.h +1 -1
 static inline int hrtimer_active(const struct hrtimer *timer)
 {
-	return rb_parent(&timer->node) != &timer->node;
+	return timer->state != HRTIMER_INACTIVE;

}

This implies that HRTIMER_CALLBACK is an "active" state, yes? If so, how come? Perhaps a comment here would aid understandability.

kernel/hrtimer.c static void enqueue_hrtimer(struct hrtim · line 387

Quoted 1 line
kernel/hrtimer.c +1 -0
+	timer->state |= HRTIMER_ACTIVE;

No! It's a bitfield! The plot thickens.

How come hrtimer_active() tests for equality of all bits if it's a bitfield?

kernel/hrtimer.c static void __remove_hrtimer(struct hrti · line 410

Quoted 1 line
kernel/hrtimer.c +1 -0
+	timer->state = newstate;

No, it's not a bitfield. It's a scalar.

kernel/hrtimer.c int hrtimer_try_to_cancel(struct hrtimer · line 492

Quoted 1 line
kernel/hrtimer.c +1 -0
+	if (!(timer->state & HRTIMER_CALLBACK))

whoop, it's a bitfield again.

Quoted 12 lines
kernel/hrtimer.c +0 -1
 		ret = remove_hrtimer(timer, base);
 
 	unlock_hrtimer_base(timer, &flags);
@@ -592,7 +594,6 @@ void hrtimer_init(struct hrtimer *timer,
 		clock_id = CLOCK_MONOTONIC;
 
 	timer->base = &cpu_base->clock_base[clock_id];
-	rb_set_parent(&timer->node, &timer->node);
 }
 EXPORT_SYMBOL_GPL(hrtimer_init);
 

kernel/hrtimer.c static inline void run_hrtimer_queue(str · line 647

Quoted 5 lines
kernel/hrtimer.c +1 -1
@@ -643,13 +644,14 @@ static inline void run_hrtimer_queue(str
 
 		fn = timer->function;
 		set_curr_timer(cpu_base, timer);
-		__remove_hrtimer(timer, base);
+		__remove_hrtimer(timer, base, HRTIMER_CALLBACK);

How come this was assigned to state, and not or-ed into it?

kernel/hrtimer.c static inline void run_hrtimer_queue(str · line 654

Quoted 1 line
kernel/hrtimer.c +1 -0
+		timer->state &= ~HRTIMER_CALLBACK;

Please document the locking for timer->state.

Please also document its various states.

#29 lore raw

PATCH 12/23 hrtimers: clean up callback tracking

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

reintroduce ktimers feature "optimized away" by the ktimers review process: remove the curr_timer pointer from the cpu-base and use the hrtimer state.

--

 include/linux/hrtimer.h |    1 -
 kernel/hrtimer.c        |   10 +---------
 2 files changed, 1 insertion(+), 10 deletions(-)
include/linux/hrtimer.h +0 -1
Index: linux-2.6.18-mm2/include/linux/hrtimer.h
===================================================================
@@ -108,7 +108,6 @@ struct hrtimer_cpu_base {
 	spinlock_t			lock;
 	struct lock_class_key		lock_key;
 	struct hrtimer_clock_base	clock_base[HRTIMER_MAX_CLOCK_BASES];
-	struct hrtimer			*curr_timer;
 };
 
 /*
kernel/hrtimer.c +1 -9
Index: linux-2.6.18-mm2/kernel/hrtimer.c
===================================================================
@@ -149,8 +149,6 @@ static void hrtimer_get_softirq_time(str
  */
 #ifdef CONFIG_SMP
 
-#define set_curr_timer(b, t)		do { (b)->curr_timer = (t); } while (0)
-
 /*
  * We are using hashed locking: holding per_cpu(hrtimer_bases)[n].lock
  * means that all timers which are tied to this base via timer->base are
@@ -204,7 +202,7 @@ switch_hrtimer_base(struct hrtimer *time
 		 * completed. There is no conflict as we hold the lock until
 		 * the timer is enqueued.
 		 */
-		if (unlikely(base->cpu_base->curr_timer == timer))
+		if (unlikely(timer->state & HRTIMER_CALLBACK))
 			return base;
 
 		/* See the comment in lock_timer_base() */
@@ -218,8 +216,6 @@ switch_hrtimer_base(struct hrtimer *time
 
 #else /* CONFIG_SMP */
 
-#define set_curr_timer(b, t)		do { } while (0)
-
 static inline struct hrtimer_clock_base *
 lock_hrtimer_base(const struct hrtimer *timer, unsigned long *flags)
 {
@@ -643,7 +639,6 @@ static inline void run_hrtimer_queue(str
 			break;
 
 		fn = timer->function;
-		set_curr_timer(cpu_base, timer);
 		__remove_hrtimer(timer, base, HRTIMER_CALLBACK);
 		spin_unlock_irq(&cpu_base->lock);
 
@@ -657,7 +652,6 @@ static inline void run_hrtimer_queue(str
 			enqueue_hrtimer(timer, base);
 		}
 	}
-	set_curr_timer(cpu_base, NULL);
 	spin_unlock_irq(&cpu_base->lock);
 }
 
@@ -844,8 +838,6 @@ static void migrate_hrtimers(int cpu)
 	spin_lock(&old_base->lock);
 
 	for (i = 0; i < HRTIMER_MAX_CLOCK_BASES; i++) {
-		BUG_ON(old_base->curr_timer);
-
 		migrate_hrtimer_list(&old_base->clock_base[i],
 				     &new_base->clock_base[i]);
 	}

--

#30 lore raw

PATCH 13/23 clockevents: core

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

We have two types of clock event devices:
- global events (one device per system)
- local events (one device per cpu)

We assign the various time(r) related interrupts to those devices:

- global tick
- profiling (per cpu)
- next timer events (per cpu)

architectures register their clockevent sources, with specific capability masks set, and the generic high-res-timers code picks the best one, without the architecture having to worry about that.

here are the capabilities a clockevent driver can register:

#define CLOCK_CAP_TICK 0x000001
#define CLOCK_CAP_UPDATE 0x000002
#define CLOCK_CAP_PROFILE 0x000004
#define CLOCK_CAP_NEXTEVT 0x000008

--

 include/linux/clockchips.h |  104 ++++++++
 include/linux/hrtimer.h    |    3 
 init/main.c                |    2 
 kernel/hrtimer.c           |    6 
 kernel/time/Makefile       |    2 
 kernel/time/clockevents.c  |  527 +++++++++++++++++++++++++++++++++++++++++++++
 6 files changed, 642 insertions(+), 2 deletions(-)
include/linux/clockchips.h new file +104 -0
Index: linux-2.6.18-mm2/include/linux/clockchips.h
===================================================================
@@ -0,0 +1,104 @@
+/*  linux/include/linux/clockchips.h
+ *
+ *  This file contains the structure definitions for clockchips.
+ *
+ *  If you are not a clockchip, or the time of day code, you should
+ *  not be including this file!
+ */
+#ifndef _LINUX_CLOCKCHIPS_H
+#define _LINUX_CLOCKCHIPS_H
+
+#include <linux/config.h>
+
+#ifdef CONFIG_GENERIC_TIME
+
+#include <linux/clocksource.h>
+#include <linux/interrupt.h>
+
+/* Clock event mode commands */
+enum {
+	CLOCK_EVT_PERIODIC,
+	CLOCK_EVT_ONESHOT,
+	CLOCK_EVT_SHUTDOWN,
+};
+
+/* Clock event capability flags */
+#define CLOCK_CAP_TICK		0x000001
+#define CLOCK_CAP_UPDATE	0x000002
+#ifndef CONFIG_PROFILE_NMI
+# define CLOCK_CAP_PROFILE	0x000004
+#else
+# define CLOCK_CAP_PROFILE	0x000000
+#endif
+#ifdef CONFIG_HIGH_RES_TIMERS
+# define CLOCK_CAP_NEXTEVT	0x000008
+#else
+# define CLOCK_CAP_NEXTEVT	0x000000
+#endif
+
+#define CLOCK_BASE_CAPS_MASK	(CLOCK_CAP_TICK | CLOCK_CAP_PROFILE | \
+				 CLOCK_CAP_UPDATE)
+#define CLOCK_CAPS_MASK		(CLOCK_BASE_CAPS_MASK | CLOCK_CAP_NEXTEVT)
+
+struct clock_event;
+
+/**
+ * struct clock_event - clock event descriptor
+ *
+ * @name:		ptr to clock event name
+ * @capabilities:	capabilities of the event chip
+ * @max_delta_ns:	maximum delta value in ns
+ * @min_delta_ns:	minimum delta value in ns
+ * @mult:		nanosecond to cycles multiplier
+ * @shift:		nanoseconds to cycles divisor (power of two)
+ * @set_next_event:	set next event
+ * @set_mode:		set mode function
+ * @suspend:		suspend function (optional)
+ * @resume:		resume function (optional)
+ * @evthandler:		Assigned by the framework to be called by the low
+ *			level handler of the event source
+ */
+struct clock_event {
+	const char	*name;
+	unsigned int	capabilities;
+	unsigned long	max_delta_ns;
+	unsigned long	min_delta_ns;
+	unsigned long	mult;
+	int		shift;
+	void		(*set_next_event)(unsigned long evt,
+					  struct clock_event *);
+	void		(*set_mode)(int mode, struct clock_event *);
+	int		(*suspend)(struct clock_event *);
+	int		(*resume)(struct clock_event *);
+	void		(*event_handler)(struct pt_regs *regs);
+};
+
+/*
+ * Calculate a multiplication factor
+ */
+static inline unsigned long div_sc(unsigned long a, unsigned long b,
+				   int shift)
+{
+	uint64_t tmp = ((uint64_t)a) << shift;
+	do_div(tmp, b);
+	return (unsigned long) tmp;
+}
+
+/* Clock event layer functions */
+extern int register_local_clockevent(struct clock_event *);
+extern int register_global_clockevent(struct clock_event *);
+extern unsigned long clockevent_delta2ns(unsigned long latch,
+					 struct clock_event *evt);
+extern void clockevents_init(void);
+
+extern int clockevents_init_next_event(void);
+extern int clockevents_set_next_event(ktime_t expires, int force);
+extern int clockevents_next_event_available(void);
+extern void clockevents_resume_events(void);
+
+#else
+# define clockevents_init()		do { } while(0)
+# define clockevents_resume_events()	do { } while(0)
+#endif
+
+#endif
include/linux/hrtimer.h +3 -0
Index: linux-2.6.18-mm2/include/linux/hrtimer.h
===================================================================
@@ -116,6 +116,9 @@ struct hrtimer_cpu_base {
  * is expired in the next softirq when the clock was advanced.
  */
 #define clock_was_set()		do { } while (0)
+#define hrtimer_clock_notify()	do { } while (0)
+extern ktime_t ktime_get(void);
+extern ktime_t ktime_get_real(void);
 
 /* Exported timer functions: */
 
init/main.c +2 -0
Index: linux-2.6.18-mm2/init/main.c
===================================================================
@@ -36,6 +36,7 @@
 #include <linux/moduleparam.h>
 #include <linux/kallsyms.h>
 #include <linux/writeback.h>
+#include <linux/clockchips.h>
 #include <linux/cpu.h>
 #include <linux/cpuset.h>
 #include <linux/efi.h>
@@ -529,6 +530,7 @@ asmlinkage void __init start_kernel(void
 	rcu_init();
 	init_IRQ();
 	pidhash_init();
+	clockevents_init();
 	init_timers();
 	hrtimers_init();
 	softirq_init();
kernel/hrtimer.c +4 -2
Index: linux-2.6.18-mm2/kernel/hrtimer.c
===================================================================
@@ -30,6 +30,7 @@
  *  For licencing details see kernel-base/COPYING
  */
 
+#include <linux/clockchips.h>
 #include <linux/cpu.h>
 #include <linux/module.h>
 #include <linux/percpu.h>
@@ -45,7 +46,7 @@
  *
  * returns the time in ktime_t format
  */
-static ktime_t ktime_get(void)
+ktime_t ktime_get(void)
 {
 	struct timespec now;
 
@@ -59,7 +60,7 @@ static ktime_t ktime_get(void)
  *
  * returns the time in ktime_t format
  */
-static ktime_t ktime_get_real(void)
+ktime_t ktime_get_real(void)
 {
 	struct timespec now;
 
@@ -292,6 +293,7 @@ static unsigned long ktime_divns(const k
  */
 void hrtimer_notify_resume(void)
 {
+	clockevents_resume_events();
 	clock_was_set();
 }
 
kernel/time/Makefile +2 -0
Index: linux-2.6.18-mm2/kernel/time/Makefile
===================================================================
@@ -1 +1,3 @@
 obj-y += ntp.o clocksource.o jiffies.o
+
+obj-$(CONFIG_GENERIC_TIME) += clockevents.o
kernel/time/clockevents.c new file +527 -0

527 lines not on this page · raw message

--

#31 lore raw

Re: [patch 13/23] clockevents: core

Quoted 16 lines · On Fri, 29 Sep 2006 23:58:32 -0000 Thomas Gleixner <[email protected]> wrote:

From: Thomas Gleixner <[email protected]>

We have two types of clock event devices:
- global events (one device per system)
- local events (one device per cpu)

We assign the various time(r) related interrupts to those devices:

- global tick
- profiling (per cpu)
- next timer events (per cpu)

architectures register their clockevent sources, with specific capability masks set, and the generic high-res-timers code picks the best one, without the architecture having to worry about that.

Quoted 6 lines

here are the capabilities a clockevent driver can register:

#define CLOCK_CAP_TICK 0x000001
#define CLOCK_CAP_UPDATE 0x000002
#define CLOCK_CAP_PROFILE 0x000004
#define CLOCK_CAP_NEXTEVT 0x000008

OK.. Perhaps this info is worth promoting to a code comment.

include/linux/clockchips.h line 11

Quoted 13 lines

+++ linux-2.6.18-mm2/include/linux/clockchips.h 2006-09-30 01:41:17.000000000 +0200

include/linux/clockchips.h new file +11 -0
@@ -0,0 +1,104 @@
+/*  linux/include/linux/clockchips.h
+ *
+ *  This file contains the structure definitions for clockchips.
+ *
+ *  If you are not a clockchip, or the time of day code, you should
+ *  not be including this file!
+ */
+#ifndef _LINUX_CLOCKCHIPS_H
+#define _LINUX_CLOCKCHIPS_H
+
+#include <linux/config.h>

The build system includes config.h for you.

Quoted 15 lines
include/linux/clockchips.h new file +15 -0
+#ifdef CONFIG_GENERIC_TIME
+
+#include <linux/clocksource.h>
+#include <linux/interrupt.h>
+
+/* Clock event mode commands */
+enum {
+	CLOCK_EVT_PERIODIC,
+	CLOCK_EVT_ONESHOT,
+	CLOCK_EVT_SHUTDOWN,
+};
+
+/* Clock event capability flags */
+#define CLOCK_CAP_TICK		0x000001
+#define CLOCK_CAP_UPDATE	0x000002

include/linux/clockchips.h line 37

Quoted 10 lines
include/linux/clockchips.h new file +10 -0
+#ifndef CONFIG_PROFILE_NMI
+# define CLOCK_CAP_PROFILE	0x000004
+#else
+# define CLOCK_CAP_PROFILE	0x000000
+#endif
+#ifdef CONFIG_HIGH_RES_TIMERS
+# define CLOCK_CAP_NEXTEVT	0x000008
+#else
+# define CLOCK_CAP_NEXTEVT	0x000000
+#endif

There is no CONFIG_PROFILE_NMI in the kernel nor anywhere else in this patchset.

Quoted 26 lines
include/linux/clockchips.h new file +26 -0
+#define CLOCK_BASE_CAPS_MASK	(CLOCK_CAP_TICK | CLOCK_CAP_PROFILE | \
+				 CLOCK_CAP_UPDATE)
+#define CLOCK_CAPS_MASK		(CLOCK_BASE_CAPS_MASK | CLOCK_CAP_NEXTEVT)
+
+struct clock_event;
+
+/**
+ * struct clock_event - clock event descriptor
+ *
+ * @name:		ptr to clock event name
+ * @capabilities:	capabilities of the event chip
+ * @max_delta_ns:	maximum delta value in ns
+ * @min_delta_ns:	minimum delta value in ns
+ * @mult:		nanosecond to cycles multiplier
+ * @shift:		nanoseconds to cycles divisor (power of two)
+ * @set_next_event:	set next event
+ * @set_mode:		set mode function
+ * @suspend:		suspend function (optional)
+ * @resume:		resume function (optional)
+ * @evthandler:		Assigned by the framework to be called by the low
+ *			level handler of the event source
+ */
+struct clock_event {
+	const char	*name;
+	unsigned int	capabilities;
+	unsigned long	max_delta_ns;

include/linux/clockchips.h line 74

Quoted 10 lines
include/linux/clockchips.h new file +10 -0
+	unsigned long	min_delta_ns;
+	unsigned long	mult;
+	int		shift;
+	void		(*set_next_event)(unsigned long evt,
+					  struct clock_event *);
+	void		(*set_mode)(int mode, struct clock_event *);
+	int		(*suspend)(struct clock_event *);
+	int		(*resume)(struct clock_event *);
+	void		(*event_handler)(struct pt_regs *regs);
+};

hm. The term "clock_event" implies "something which happens": ie, an event.

But a `struct clock_event' is, what? Actually a source of events?

Is this a well-chosen name?

include/linux/clockchips.h line 85

Quoted 10 lines
include/linux/clockchips.h new file +10 -0
+/*
+ * Calculate a multiplication factor
+ */
+static inline unsigned long div_sc(unsigned long a, unsigned long b,
+				   int shift)
+{
+	uint64_t tmp = ((uint64_t)a) << shift;
+	do_div(tmp, b);
+	return (unsigned long) tmp;
+}

What does "div_sc" stand for??

Quoted 14 lines

Index: linux-2.6.18-mm2/kernel/time/Makefile
===================================================================

kernel/time/Makefile +2 -0
@@ -1 +1,3 @@
 obj-y += ntp.o clocksource.o jiffies.o
+
+obj-$(CONFIG_GENERIC_TIME) += clockevents.o

Index: linux-2.6.18-mm2/kernel/time/clockevents.c
===================================================================

kernel/time/clockevents.c new file +1 -0
@@ -0,0 +1,527 @@
+/*

kernel/time/clockevents.c line 11

Quoted 10 lines
kernel/time/clockevents.c new file +10 -0
+ * linux/kernel/time/clockevents.c
+ *
+ * This file contains functions which manage clock event drivers.
+ *
+ * Copyright(C) 2005-2006, Thomas Gleixner <[email protected]>
+ * Copyright(C) 2005-2006, Red Hat, Inc., Ingo Molnar
+ *
+ * We have two types of clock event devices:
+ * - global events (one device per system)
+ * - local events (one device per cpu)

So perhaps s/clock_event/clock_event_driver/.

Or clock_event_device?

Quoted 38 lines
kernel/time/clockevents.c new file +38 -0
+ * We assign the various time(r) related interrupts to those devices
+ *
+ * - global tick
+ * - profiling (per cpu)
+ * - next timer events (per cpu)
+ *
+ * TODO:
+ * - implement variable frequency profiling
+ *
+ * This code is licenced under the GPL version 2. For details see
+ * kernel-base/COPYING.
+ */
+
+#include <linux/clockchips.h>
+#include <linux/cpu.h>
+#include <linux/irq.h>
+#include <linux/init.h>
+#include <linux/notifier.h>
+#include <linux/module.h>
+#include <linux/percpu.h>
+#include <linux/profile.h>
+#include <linux/sysdev.h>
+#include <linux/hrtimer.h>
+
+#define MAX_CLOCK_EVENTS	4
+#define GLOBAL_CLOCK_EVENT	MAX_CLOCK_EVENTS
+
+struct event_descr {
+	struct clock_event *event;
+	unsigned int mode;
+	unsigned int real_caps;
+	struct irqaction action;
+};
+
+struct local_events {
+	int installed;
+	struct event_descr events[MAX_CLOCK_EVENTS];
+	struct clock_event *nextevt;

kernel/time/clockevents.c line 60

Quoted 10 lines
kernel/time/clockevents.c new file +10 -0
+};
+
+/* Variables related to the global event source */
+static __read_mostly struct event_descr global_eventsource;
+
+/* Variables related to the per cpu local event sources */
+static DEFINE_PER_CPU(struct local_events, local_eventsources);
+
+/* lock to protect the above */
+static DEFINE_SPINLOCK(events_lock);

Does "the above" really refer to cpu-local storage?

kernel/time/clockevents.c line 292

Quoted 5 lines
kernel/time/clockevents.c new file +5 -0
+/*
+ * Recalc the events and reassign the handlers if necessary
+ */
+static int recalc_events(struct local_events *sources, struct clock_event *evt,
+			 unsigned int caps, int new)

It's good to document the caller-provided environmental requirements. I see from the callers that this requires spin_lock_irq(&events_lock).

kernel/time/clockevents.c line 303

Quoted 11 lines
kernel/time/clockevents.c new file +11 -0
+{
+	int i;
+
+	if (new && sources->installed == MAX_CLOCK_EVENTS)
+		return -ENOSPC;
+
+	/*
+	 * If there is no handler and this is not a next-event capable
+	 * event source, refuse to handle it
+	 */
+	if (!evt->capabilities & CLOCK_CAP_NEXTEVT && !event_handlers[caps]) {

bug - needs parentheses.

Quoted 21 lines
kernel/time/clockevents.c new file +21 -0
+		printk(KERN_ERR "Unsupported event source %s\n", evt->name);
+		return -EINVAL;
+	}
+
+	if (caps && global_eventsource.event && global_eventsource.event != evt)
+		recalc_active_event(&global_eventsource, caps);
+
+	for (i = 0; i < sources->installed; i++) {
+		if (sources->events[i].event != evt)
+			recalc_active_event(&sources->events[i], caps);
+	}
+
+	if (new)
+		sources->events[sources->installed++].event = evt;
+
+	if (caps) {
+		/* Is next_event event source going to be installed? */
+		if (caps & CLOCK_CAP_NEXTEVT)
+			caps = CLOCK_CAP_NEXTEVT;
+
+		setup_event(&sources->events[sources->installed],

kernel/time/clockevents.c line 334

Quoted 10 lines
kernel/time/clockevents.c new file +10 -0
+			    evt, caps);
+	} else
+		printk(KERN_INFO "Inactive event source %s registered\n",
+		       evt->name);
+
+	return 0;
+}
+
+/**
+ * register_local_clockevent - Set up a cpu local clock event device

We have a mixture of clock_event and clockevent.

kernel/time/clockevents.c line 340

Quoted 6 lines
kernel/time/clockevents.c new file +6 -0
+ *
+ * @evt:	event device to be registered
+ */
+int register_local_clockevent(struct clock_event *evt)
+{
+	struct local_events *sources = &__get_cpu_var(local_eventsources);

event_sources?

Quoted 35 lines
kernel/time/clockevents.c new file +35 -0
+	unsigned long flags;
+	int ret;
+
+	spin_lock_irqsave(&events_lock, flags);
+
+	/* Preset the handler in any case */
+	evt->event_handler = handle_noop;
+
+	/* Recalc event sources and maybe reassign handlers */
+	ret = recalc_events(sources, evt,
+			    evt->capabilities & CLOCK_BASE_CAPS_MASK, 1);
+
+	spin_unlock_irqrestore(&events_lock, flags);
+
+	/*
+	 * Trigger hrtimers, when the event source is next-event
+	 * capable
+	 */
+	if (!ret && (evt->capabilities & CLOCK_CAP_NEXTEVT))
+		hrtimer_clock_notify();
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(register_local_clockevent);
+
+/*
+ * Find a next-event capable event source
+ */
+static int get_next_event_source(void)
+{
+	struct local_events *sources = &__get_cpu_var(local_eventsources);
+	int i;
+
+	for (i = 0; i < sources->installed; i++) {
+		struct clock_event *evt;

kernel/time/clockevents.c line 385

Quoted 10 lines
kernel/time/clockevents.c new file +10 -0
+
+		evt = sources->events[i].event;
+		if (evt->capabilities & CLOCK_CAP_NEXTEVT)
+			return i;
+	}
+
+#ifndef CONFIG_SMP
+	if (global_eventsource.event->capabilities & CLOCK_CAP_NEXTEVT)
+		return GLOBAL_CLOCK_EVENT;
+#endif

How come this is UP-only? Perhaps a comment describing what's going on here.

Quoted 6 lines
kernel/time/clockevents.c new file +6 -0
+	return -ENODEV;
+}
+
+/**
+ * clockevents_next_event_available - Check for a installed next-event source
+ */

kernel/time/clockevents.c line 401

Quoted 10 lines
kernel/time/clockevents.c new file +10 -0
+int clockevents_next_event_available(void)
+{
+	unsigned long flags;
+	int idx;
+
+	spin_lock_irqsave(&events_lock, flags);
+	idx = get_next_event_source();
+	spin_unlock_irqrestore(&events_lock, flags);
+
+	return idx < 0 ? 0 : 1;

Perhaps IS_ERR_VALUE() could be used to make this code clearer.

I really wish kerneldoc had a standard way of describing return values. People often leave it out, and it's important.

(Although it's fairly obvious here due to the well-chosen function name)

(But it'll be even better when generic-boolean is merged, and people start using it).

Quoted 43 lines
kernel/time/clockevents.c new file +43 -0
+}
+
+int clockevents_init_next_event(void)
+{
+	struct local_events *sources = &__get_cpu_var(local_eventsources);
+	struct clock_event *nextevt;
+	unsigned long flags;
+	int idx, ret = -ENODEV;
+
+	if (sources->nextevt)
+		return -EBUSY;
+
+	spin_lock_irqsave(&events_lock, flags);
+
+	idx = get_next_event_source();
+	if (idx < 0)
+		goto out_unlock;
+
+	if (idx == GLOBAL_CLOCK_EVENT)
+		nextevt = global_eventsource.event;
+	else
+		nextevt = sources->events[idx].event;
+
+	ret = recalc_events(sources, nextevt, CLOCK_CAPS_MASK, 0);
+	if (!ret)
+		sources->nextevt = nextevt;
+ out_unlock:
+	spin_unlock_irqrestore(&events_lock, flags);
+
+	return ret;
+}
+
+int clockevents_set_next_event(ktime_t expires, int force)
+{
+	struct local_events *sources = &__get_cpu_var(local_eventsources);
+	int64_t delta = ktime_to_ns(ktime_sub(expires, ktime_get()));
+	struct clock_event *nextevt = sources->nextevt;
+	unsigned long long clc;
+
+	if (delta <= 0 && !force)
+		return -ETIME;
+
+	if (delta > nextevt->max_delta_ns)

kernel/time/clockevents.c line 454

Quoted 10 lines
kernel/time/clockevents.c new file +10 -0
+		delta = nextevt->max_delta_ns;
+	if (delta < nextevt->min_delta_ns)
+		delta = nextevt->min_delta_ns;
+
+	clc = delta * nextevt->mult;
+	clc >>= nextevt->shift;
+	nextevt->set_next_event((unsigned long)clc, sources->nextevt);
+
+	return 0;
+}

These functions are exported to the whole kernel, but are undocumented.

AFAIK the timer and hrtimer code consistently uses s64's and u64's. But someone has snuck a couple of int64_t's in here. Please review all the patches for that.

Quoted 28 lines
kernel/time/clockevents.c new file +28 -0
+/*
+ * Resume the cpu local clock events
+ */
+static void clockevents_resume_local_events(void *arg)
+{
+	struct local_events *sources = &__get_cpu_var(local_eventsources);
+	int i;
+
+	for (i = 0; i < sources->installed; i++) {
+		if (sources->events[i].real_caps)
+			startup_event(sources->events[i].event,
+				      sources->events[i].real_caps);
+	}
+}
+
+/*
+ * Called after timekeeping is functional again
+ */
+void clockevents_resume_events(void)
+{
+	unsigned long flags;
+
+	local_irq_save(flags);
+
+	/* Resume global event source */
+	if (global_eventsource.real_caps)
+		startup_event(global_eventsource.event,
+			      global_eventsource.real_caps);

kernel/time/clockevents.c line 493

Quoted 10 lines
kernel/time/clockevents.c new file +10 -0
+
+	clockevents_resume_local_events(NULL);
+	local_irq_restore(flags);
+
+	touch_softlockup_watchdog();
+
+	if (smp_call_function(clockevents_resume_local_events, NULL, 1, 1))
+		BUG();
+
+}

hm. The kernel's core resume code likes to call resume handlers under local_irq_disable(). Does that happen here? A BUG_ON(irqs_disabled()) would tell.

The above code can be simplified via on_each_cpu().

Quoted 24 lines
kernel/time/clockevents.c new file +23 -0
+/*
+ * Functions related to initialization and hotplug
+ */
+static int clockevents_cpu_notify(struct notifier_block *self,
+				  unsigned long action, void *hcpu)
+{
+	switch(action) {
+	case CPU_UP_PREPARE:
+		break;
+#ifdef CONFIG_HOTPLUG_CPU
+	case CPU_DEAD:
+		/*
+		 * Do something sensible here !
+		 * Disable the cpu local clocksources
+		 */
+		break;
+#endif
+	default:
+		break;
+	}
+	return NOTIFY_OK;
+}
+

kernel/time/clockevents.c line 527

Quoted 10 lines
kernel/time/clockevents.c new file +10 -0
+static struct notifier_block __devinitdata clockevents_nb = {
+	.notifier_call	= clockevents_cpu_notify,
+};
+
+void __init clockevents_init(void)
+{
+	clockevents_cpu_notify(&clockevents_nb, (unsigned long)CPU_UP_PREPARE,
+				(void *)(long)smp_processor_id());
+	register_cpu_notifier(&clockevents_nb);
+}

No... None of this code should be present if !CONFIG_HOTPLUG_CPU. See cpuid_class_cpu_callback() for an example
.

#32 lore raw

Re: [patch 13/23] clockevents: core

Quoted 64 lines · Andrew Morton
Quoted 16 lines · On Fri, 29 Sep 2006 23:58:32 -0000 Thomas Gleixner <[email protected]> wrote:

From: Thomas Gleixner <[email protected]>

We have two types of clock event devices:
- global events (one device per system)
- local events (one device per cpu)

We assign the various time(r) related interrupts to those devices:

- global tick
- profiling (per cpu)
- next timer events (per cpu)

architectures register their clockevent sources, with specific capability masks set, and the generic high-res-timers code picks the best one, without the architecture having to worry about that.

Quoted 6 lines

here are the capabilities a clockevent driver can register:

#define CLOCK_CAP_TICK 0x000001
#define CLOCK_CAP_UPDATE 0x000002
#define CLOCK_CAP_PROFILE 0x000004
#define CLOCK_CAP_NEXTEVT 0x000008

OK.. Perhaps this info is worth promoting to a code comment.

include/linux/clockchips.h line 11

Quoted 13 lines

+++ linux-2.6.18-mm2/include/linux/clockchips.h 2006-09-30 01:41:17.000000000 +0200

include/linux/clockchips.h new file +11 -0
@@ -0,0 +1,104 @@
+/*  linux/include/linux/clockchips.h
+ *
+ *  This file contains the structure definitions for clockchips.
+ *
+ *  If you are not a clockchip, or the time of day code, you should
+ *  not be including this file!
+ */
+#ifndef _LINUX_CLOCKCHIPS_H
+#define _LINUX_CLOCKCHIPS_H
+
+#include <linux/config.h>

The build system includes config.h for you.

Quoted 20 lines
include/linux/clockchips.h new file +20 -0
+#ifdef CONFIG_GENERIC_TIME
+
+#include <linux/clocksource.h>
+#include <linux/interrupt.h>
+
+/* Clock event mode commands */
+enum {
+	CLOCK_EVT_PERIODIC,
+	CLOCK_EVT_ONESHOT,
+	CLOCK_EVT_SHUTDOWN,
+};
+
+/* Clock event capability flags */
+#define CLOCK_CAP_TICK		0x000001
+#define CLOCK_CAP_UPDATE	0x000002
+#ifndef CONFIG_PROFILE_NMI
+# define CLOCK_CAP_PROFILE	0x000004
+#else
+# define CLOCK_CAP_PROFILE	0x000000
+#endif
Quoted 9 lines

include/linux/clockchips.h line 37

Quoted 5 lines
include/linux/clockchips.h new file +5 -0
+#ifdef CONFIG_HIGH_RES_TIMERS
+# define CLOCK_CAP_NEXTEVT	0x000008
+#else
+# define CLOCK_CAP_NEXTEVT	0x000000
+#endif

There is no CONFIG_PROFILE_NMI in the kernel nor anywhere else in this patchset.

---SNIP----

As I've pointed out - this breaks the ability to do timer tick profiling too.
http://marc.theaimsgroup.com/?l=linux-kernel&m=115484411119770&w=2
http://marc.theaimsgroup.com/?l=linux-kernel&m=115484446530853&w=2

#33 lore raw

PATCH 14/23 clockevents: drivers for i386

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

add clockevent drivers for i386: lapic (local) and PIT (global). Update the timer IRQ to call into the PIT driver's event handler and the lapic-timer IRQ to call into the lapic clockevent driver.

--

 arch/i386/kernel/apic.c                  |   81 ++++++++++++++++++++++++-------
 arch/i386/kernel/i8253.c                 |   60 +++++++++++++++++++---
 arch/i386/kernel/time.c                  |   45 -----------------
 include/asm-i386/i8253.h                 |    1 
 include/asm-i386/mach-default/do_timer.h |    3 -
 5 files changed, 120 insertions(+), 70 deletions(-)
arch/i386/kernel/apic.c +65 -16
Index: linux-2.6.18-mm2/arch/i386/kernel/apic.c
===================================================================
@@ -25,6 +25,7 @@
 #include <linux/kernel_stat.h>
 #include <linux/sysdev.h>
 #include <linux/cpu.h>
+#include <linux/clockchips.h>
 #include <linux/module.h>
 
 #include <asm/atomic.h>
@@ -70,6 +71,23 @@ static inline void lapic_enable(void)
  */
 int apic_verbosity;
 
+static unsigned int calibration_result;
+
+static void lapic_next_event(unsigned long delta, struct clock_event *evt);
+static void lapic_timer_setup(int mode, struct clock_event *evt);
+
+static struct clock_event lapic_clockevent = {
+	.name = "lapic",
+	.capabilities = CLOCK_CAP_NEXTEVT | CLOCK_CAP_PROFILE
+#ifdef CONFIG_SMP
+			| CLOCK_CAP_UPDATE
+#endif
+	,
+	.shift = 32,
+	.set_mode = lapic_timer_setup,
+	.set_next_event = lapic_next_event,
+};
+static DEFINE_PER_CPU(struct clock_event, lapic_events);
 
 static void apic_pm_activate(void);
 
@@ -919,6 +937,11 @@ fake_ioapic_page:
  */
 
 /*
+ * FIXME: Move this to i8253.h. There is no need to keep the access to
+ * the PIT scattered all around the place -tglx
+ */
+
+/*
  * The timer chip is already set up at HZ interrupts per second here,
  * but we do not accept timer interrupts yet. We only allow the BP
  * to calibrate.
@@ -976,13 +999,15 @@ void (*wait_timer_tick)(void) __devinitd
 
 #define APIC_DIVISOR 16
 
-static void __setup_APIC_LVTT(unsigned int clocks)
+static void __setup_APIC_LVTT(unsigned int clocks, int oneshot)
 {
 	unsigned int lvtt_value, tmp_value, ver;
 	int cpu = smp_processor_id();
 
 	ver = GET_APIC_VERSION(apic_read(APIC_LVR));
-	lvtt_value = APIC_LVT_TIMER_PERIODIC | LOCAL_TIMER_VECTOR;
+	lvtt_value = LOCAL_TIMER_VECTOR;
+	if (!oneshot)
+		lvtt_value |= APIC_LVT_TIMER_PERIODIC;
 	if (!APIC_INTEGRATED(ver))
 		lvtt_value |= SET_APIC_TIMER_BASE(APIC_TIMER_BASE_DIV);
 
@@ -999,23 +1024,31 @@ static void __setup_APIC_LVTT(unsigned i
 				& ~(APIC_TDR_DIV_1 | APIC_TDR_DIV_TMBASE))
 				| APIC_TDR_DIV_16);
 
-	apic_write_around(APIC_TMICT, clocks/APIC_DIVISOR);
+	if (!oneshot)
+		apic_write_around(APIC_TMICT, clocks/APIC_DIVISOR);
+}
+
+static void lapic_next_event(unsigned long delta, struct clock_event *evt)
+{
+	apic_write_around(APIC_TMICT, delta);
 }
 
-static void __devinit setup_APIC_timer(unsigned int clocks)
+static void lapic_timer_setup(int mode, struct clock_event *evt)
 {
 	unsigned long flags;
 
 	local_irq_save(flags);
+	__setup_APIC_LVTT(calibration_result, mode != CLOCK_EVT_PERIODIC);
+	local_irq_restore(flags);
+}
 
-	/*
-	 * Wait for IRQ0's slice:
-	 */
-	wait_timer_tick();
+static void __devinit setup_APIC_timer(void)
+{
+	struct clock_event *levt = &__get_cpu_var(lapic_events);
 
-	__setup_APIC_LVTT(clocks);
+	memcpy(levt, &lapic_clockevent, sizeof(*levt));
 
-	local_irq_restore(flags);
+	register_local_clockevent(levt);
 }
 
 /*
@@ -1024,6 +1057,8 @@ static void __devinit setup_APIC_timer(u
  * to calibrate, since some later bootup code depends on getting
  * the first irq? Ugh.
  *
+ * TODO: Fix this rather than saying "Ugh" -tglx
+ *
  * We want to do the calibration only once since we
  * want to have local timer irqs syncron. CPUs connected
  * by the same APIC bus have the very same bus frequency.
@@ -1046,7 +1081,7 @@ static int __init calibrate_APIC_clock(v
 	 * value into the APIC clock, we just want to get the
 	 * counter running for calibration.
 	 */
-	__setup_APIC_LVTT(1000000000);
+	__setup_APIC_LVTT(1000000000, 0);
 
 	/*
 	 * The timer chip counts down to zero. Let's wait
@@ -1083,6 +1118,14 @@ static int __init calibrate_APIC_clock(v
 
 	result = (tt1-tt2)*APIC_DIVISOR/LOOPS;
 
+	/* Calculate the scaled math multiplication factor */
+	lapic_clockevent.mult = div_sc(tt1-tt2, TICK_NSEC * LOOPS, 32);
+	lapic_clockevent.max_delta_ns =
+		clockevent_delta2ns(0x7FFFFF, &lapic_clockevent);
+	printk("lapic max_delta_ns: %ld\n", lapic_clockevent.max_delta_ns);
+	lapic_clockevent.min_delta_ns =
+		clockevent_delta2ns(0xF, &lapic_clockevent);
+
 	if (cpu_has_tsc)
 		apic_printk(APIC_VERBOSE, "..... CPU clock speed is "
 			"%ld.%04ld MHz.\n",
@@ -1097,8 +1140,6 @@ static int __init calibrate_APIC_clock(v
 	return result;
 }
 
-static unsigned int calibration_result;
-
 void __init setup_boot_APIC_clock(void)
 {
 	unsigned long flags;
@@ -1111,14 +1152,14 @@ void __init setup_boot_APIC_clock(void)
 	/*
 	 * Now set up the timer for real.
 	 */
-	setup_APIC_timer(calibration_result);
+	setup_APIC_timer();
 
 	local_irq_restore(flags);
 }
 
 void __devinit setup_secondary_APIC_clock(void)
 {
-	setup_APIC_timer(calibration_result);
+	setup_APIC_timer();
 }
 
 void disable_APIC_timer(void)
@@ -1164,6 +1205,13 @@ void switch_APIC_timer_to_ipi(void *cpum
 	    !cpu_isset(cpu, timer_bcast_ipi)) {
 		disable_APIC_timer();
 		cpu_set(cpu, timer_bcast_ipi);
+#ifdef CONFIG_HIGH_RES_TIMERS
+		printk("Disabling NO_HZ and high resolution timers "
+		       "due to timer broadcasting\n");
+		for_each_possible_cpu(cpu)
+			per_cpu(lapic_events, cpu).capabilities &=
+				~CLOCK_CAP_NEXTEVT;
+#endif
 	}
 }
 EXPORT_SYMBOL(switch_APIC_timer_to_ipi);
@@ -1224,6 +1272,7 @@ inline void smp_local_timer_interrupt(st
 fastcall void smp_apic_timer_interrupt(struct pt_regs *regs)
 {
 	int cpu = smp_processor_id();
+	struct clock_event *evt = &per_cpu(lapic_events, cpu);
 
 	/*
 	 * the NMI deadlock-detector uses this.
@@ -1241,7 +1290,7 @@ fastcall void smp_apic_timer_interrupt(s
 	 * interrupt lock, which is the WrongThing (tm) to do.
 	 */
 	irq_enter();
-	smp_local_timer_interrupt(regs);
+	evt->event_handler(regs);
 	irq_exit();
 }
 
arch/i386/kernel/i8253.c +52 -8

85 lines not on this page · raw message

arch/i386/kernel/time.c +0 -45

69 lines not on this page · raw message

include/asm-i386/i8253.h +1 -0
Index: linux-2.6.18-mm2/include/asm-i386/i8253.h
===================================================================
@@ -2,5 +2,6 @@
 #define __ASM_I8253_H__
 
 extern spinlock_t i8253_lock;
+extern struct clock_event pit_clockevent;
 
 #endif	/* __ASM_I8253_H__ */
include/asm-i386/mach-default/do_timer.h +2 -1
Index: linux-2.6.18-mm2/include/asm-i386/mach-default/do_timer.h
===================================================================
@@ -1,7 +1,8 @@
 /* defines for inline arch setup functions */
-
+#include <linux/clockchips.h>
 #include <asm/apic.h>
 #include <asm/i8259.h>
+#include <asm/i8253.h>
 
 /**
  * do_timer_interrupt_hook - hook into timer tick

--

#34 lore raw

Re: [patch 14/23] clockevents: drivers for i386

Quoted 6 lines · On Fri, 29 Sep 2006 23:58:33 -0000 Thomas Gleixner <[email protected]> wrote:

From: Thomas Gleixner <[email protected]>

add clockevent drivers for i386: lapic (local) and PIT (global). Update the timer IRQ to call into the PIT driver's event handler and the lapic-timer IRQ to call into the lapic clockevent driver.

What's the story on other clock sources? hpet, pm-timer?

Quoted 15 lines
arch/i386/kernel/apic.c +2 -0
@@ -25,6 +25,7 @@
 #include <linux/kernel_stat.h>
 #include <linux/sysdev.h>
 #include <linux/cpu.h>
+#include <linux/clockchips.h>
 #include <linux/module.h>
 
 #include <asm/atomic.h>
@@ -70,6 +71,23 @@ static inline void lapic_enable(void)
  */
 int apic_verbosity;
 
+static unsigned int calibration_result;

arch/i386/kernel/apic.c static inline void lapic_enable(void) · line 84

Quoted 10 lines
arch/i386/kernel/apic.c +10 -0
+
+static void lapic_next_event(unsigned long delta, struct clock_event *evt);
+static void lapic_timer_setup(int mode, struct clock_event *evt);
+
+static struct clock_event lapic_clockevent = {
+	.name = "lapic",
+	.capabilities = CLOCK_CAP_NEXTEVT | CLOCK_CAP_PROFILE
+#ifdef CONFIG_SMP
+			| CLOCK_CAP_UPDATE
+#endif

I can't work out why this is SMP-only. Please add changelog information or, better, a comment explaining this.

Quoted 5 lines
arch/i386/kernel/apic.c +1 -1
-static void __devinit setup_APIC_timer(unsigned int clocks)
+static void lapic_timer_setup(int mode, struct clock_event *evt)
 {
 	unsigned long flags;
 

arch/i386/kernel/apic.c static void __setup_APIC_LVTT(unsigned i

Quoted 10 lines
arch/i386/kernel/apic.c +3 -5
 	local_irq_save(flags);
+	__setup_APIC_LVTT(calibration_result, mode != CLOCK_EVT_PERIODIC);
+	local_irq_restore(flags);
+}
 
-	/*
-	 * Wait for IRQ0's slice:
-	 */
-	wait_timer_tick();
-	wait_timer_tick();

For what reason was setup_APIC_timer() "Waiting for IRQ0's slice" and why is it safe to remove that?

arch/i386/kernel/apic.c void switch_APIC_timer_to_ipi(void *cpum · line 1214

Quoted 7 lines
arch/i386/kernel/apic.c +7 -0
+#ifdef CONFIG_HIGH_RES_TIMERS
+		printk("Disabling NO_HZ and high resolution timers "
+		       "due to timer broadcasting\n");
+		for_each_possible_cpu(cpu)
+			per_cpu(lapic_events, cpu).capabilities &=
+				~CLOCK_CAP_NEXTEVT;
+#endif

I think you mean "due to lack of timer broadcasting"?

No comment, no changelog entry -> I don't understand why this code is here.

arch/i386/kernel/apic.c line 22

Quoted 13 lines

*
*/

arch/i386/kernel/apic.c +2 -2
-#include <linux/clocksource.h>
+#include <linux/clockchips.h>
 #include <linux/spinlock.h>
 #include <linux/jiffies.h>
 #include <linux/sysdev.h>
@@ -19,19 +19,63 @@
 DEFINE_SPINLOCK(i8253_lock);
 EXPORT_SYMBOL(i8253_lock);
 
-void setup_pit_timer(void)
+static void init_pit_timer(int mode, struct clock_event *evt)

No. mode' is not an integer. It has type enum you_forgot_to_give_it_a_name'.

In several places the timer code is using enums as integers - basically using them as a glorified #define.

This loses the main advantages of enums: readability. They permit the reader to say "ah, that's a clock event type" instead of "oh, thats an integer".

Please give those enums a name, and use it everywhere, rather than relying upon implicit conversion to `int'.

(C sucks)

Quoted 35 lines
arch/i386/kernel/apic.c +26 -5
+{
+	unsigned long flags;
+
+	spin_lock_irqsave(&i8253_lock, flags);
+
+	switch(mode) {
+	case CLOCK_EVT_PERIODIC:
+		/* binary, mode 2, LSB/MSB, ch 0 */
+		outb_p(0x34, PIT_MODE);
+		udelay(10);
+		outb_p(LATCH & 0xff , PIT_CH0);	/* LSB */
+		outb(LATCH >> 8 , PIT_CH0);	/* MSB */
+		break;
+
+	case CLOCK_EVT_ONESHOT:
+	case CLOCK_EVT_SHUTDOWN:
+		/* One shot setup */
+		outb_p(0x38, PIT_MODE);
+		udelay(10);
+		break;
+	}
+	spin_unlock_irqrestore(&i8253_lock, flags);
+}
+
+static void pit_next_event(unsigned long delta, struct clock_event *evt)
 {
 	unsigned long flags;
 
 	spin_lock_irqsave(&i8253_lock, flags);
-	outb_p(0x34,PIT_MODE);		/* binary, mode 2, LSB/MSB, ch 0 */
-	udelay(10);
-	outb_p(LATCH & 0xff , PIT_CH0);	/* LSB */
-	udelay(10);
-	outb(LATCH >> 8 , PIT_CH0);	/* MSB */
+	outb_p(delta & 0xff , PIT_CH0);	/* LSB */

arch/i386/kernel/apic.c line 62

Quoted 10 lines
arch/i386/kernel/apic.c +7 -0
+	outb(delta >> 8 , PIT_CH0);	/* MSB */
 	spin_unlock_irqrestore(&i8253_lock, flags);
 }
 
+struct clock_event pit_clockevent = {
+	.name		= "pit",
+	.capabilities	= CLOCK_CAP_TICK | CLOCK_CAP_PROFILE | CLOCK_CAP_UPDATE
+#ifndef CONFIG_SMP
+			| CLOCK_CAP_NEXTEVT
+#endif

Again, the CONFIG_SMP conditionality is a complete mystery to this reader.

#35 lore raw

PATCH 15/23 high-res timers: core

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

add the core bits of high-res timers support.

the design makes use of the existing hrtimers subsystem which manages a per-CPU and per-clock tree of timers, and the clockevents framework, which provides a standard API to request programmable clock events from. The core code does not have to know about the clock details - it makes use of clockevents_set_next_event().

the code also provides dyntick functionality: it is implemented via a per-cpu sched_tick hrtimer that is set to HZ frequency, but which is reprogrammed to a longer timeout before going idle, and reprogrammed to HZ again once the CPU goes busy again. (If an non-timer IRQ hits the idle task then it will process jiffies before calling the IRQ code.)

the impact to non-high-res architectures is intended to be minimal.

--

 include/linux/hrtimer.h   |   70 +++++
 include/linux/interrupt.h |    5 
 include/linux/ktime.h     |    3 
 kernel/hrtimer.c          |  576 ++++++++++++++++++++++++++++++++++++++++++++--
 kernel/itimer.c           |    2 
 kernel/posix-timers.c     |    2 
 kernel/time/Kconfig       |   22 +
 kernel/timer.c            |    1 
 8 files changed, 649 insertions(+), 32 deletions(-)
include/linux/hrtimer.h +66 -4
Index: linux-2.6.18-mm2/include/linux/hrtimer.h
===================================================================
@@ -17,6 +17,7 @@
 
 #include <linux/rbtree.h>
 #include <linux/ktime.h>
+#include <linux/timer.h>
 #include <linux/init.h>
 #include <linux/list.h>
 #include <linux/wait.h>
@@ -34,9 +35,17 @@ enum hrtimer_restart {
 	HRTIMER_RESTART,
 };
 
+enum hrtimer_cb_mode {
+	HRTIMER_CB_SOFTIRQ,
+	HRTIMER_CB_IRQSAFE,
+	HRTIMER_CB_IRQSAFE_NO_RESTART,
+	HRTIMER_CB_IRQSAFE_NO_SOFTIRQ,
+};
+
 #define HRTIMER_INACTIVE	0x00
 #define HRTIMER_ACTIVE		0x01
 #define HRTIMER_CALLBACK	0x02
+#define HRTIMER_PENDING		0x04
 
 struct hrtimer_clock_base;
 
@@ -49,6 +58,10 @@ struct hrtimer_clock_base;
  * @function:	timer expiry callback function
  * @base:	pointer to the timer base (per cpu and per clock)
  *
+ * @mode:	high resolution timer feature to allow executing the
+ *		callback in the hardirq context (wakeups)
+ * @cb_entry:	list head to enqueue an expired timer into the callback list
+ *
  * The hrtimer structure must be initialized by init_hrtimer_#CLOCKTYPE()
  */
 struct hrtimer {
@@ -57,6 +70,10 @@ struct hrtimer {
 	int				(*function)(struct hrtimer *);
 	struct hrtimer_clock_base	*base;
 	unsigned long			state;
+#ifdef CONFIG_HIGH_RES_TIMERS
+	int				cb_mode;
+	struct list_head		cb_entry;
+#endif
 };
 
 /**
@@ -83,6 +100,9 @@ struct hrtimer_cpu_base;
  * @get_time:		function to retrieve the current time of the clock
  * @get_softirq_time:	function to retrieve the current time from the softirq
  * @softirq_time:	the time when running the hrtimer queue in the softirq
+ * @cb_pending:		list of timers where the callback is pending
+ * @offset:		offset of this clock to the monotonic base
+ * @reprogram:		function to reprogram the timer event
  */
 struct hrtimer_clock_base {
 	struct hrtimer_cpu_base	*cpu_base;
@@ -93,6 +113,12 @@ struct hrtimer_clock_base {
 	ktime_t			(*get_time)(void);
 	ktime_t			(*get_softirq_time)(void);
 	ktime_t			softirq_time;
+#ifdef CONFIG_HIGH_RES_TIMERS
+	ktime_t			offset;
+	int			(*reprogram)(struct hrtimer *t,
+					     struct hrtimer_clock_base *b,
+					     ktime_t n);
+#endif
 };
 
 #define HRTIMER_MAX_CLOCK_BASES 2
@@ -108,17 +134,53 @@ struct hrtimer_cpu_base {
 	spinlock_t			lock;
 	struct lock_class_key		lock_key;
 	struct hrtimer_clock_base	clock_base[HRTIMER_MAX_CLOCK_BASES];
+#ifdef CONFIG_HIGH_RES_TIMERS
+	ktime_t				expires_next;
+	int				hres_active;
+	unsigned long			check_clocks;
+	struct list_head		cb_pending;
+	struct hrtimer			sched_timer;
+	struct pt_regs			*sched_regs;
+	unsigned long			events;
+#endif
 };
 
+#ifdef CONFIG_HIGH_RES_TIMERS
+
+extern void hrtimer_clock_notify(void);
+extern void clock_was_set(void);
+extern void hrtimer_interrupt(struct pt_regs *regs);
+
+# define hrtimer_cb_get_time(t)	(t)->base->get_time()
+# define hrtimer_hres_active	(__get_cpu_var(hrtimer_bases).hres_active)
+/*
+ * The resolution of the clocks. The resolution value is returned in
+ * the clock_getres() system call to give application programmers an
+ * idea of the (in)accuracy of timers. Timer values are rounded up to
+ * this resolution values.
+ */
+# define KTIME_HIGH_RES		(ktime_t) { .tv64 = CONFIG_HIGH_RES_RESOLUTION }
+# define KTIME_MONOTONIC_RES	KTIME_HIGH_RES
+
+#else
+
+# define KTIME_MONOTONIC_RES	KTIME_LOW_RES
+
 /*
  * clock_was_set() is a NOP for non- high-resolution systems. The
  * time-sorted order guarantees that a timer does not expire early and
  * is expired in the next softirq when the clock was advanced.
  */
-#define clock_was_set()		do { } while (0)
-#define hrtimer_clock_notify()	do { } while (0)
-extern ktime_t ktime_get(void);
-extern ktime_t ktime_get_real(void);
+# define clock_was_set()		do { } while (0)
+# define hrtimer_clock_notify()		do { } while (0)
+
+# define hrtimer_cb_get_time(t)		(t)->base->softirq_time
+# define hrtimer_hres_active		0
+
+#endif
+
+  extern ktime_t ktime_get(void);
+  extern ktime_t ktime_get_real(void);
 
 /* Exported timer functions: */
 
include/linux/interrupt.h +4 -1
Index: linux-2.6.18-mm2/include/linux/interrupt.h
===================================================================
@@ -235,7 +235,10 @@ enum
 	NET_TX_SOFTIRQ,
 	NET_RX_SOFTIRQ,
 	BLOCK_SOFTIRQ,
-	TASKLET_SOFTIRQ
+	TASKLET_SOFTIRQ,
+#ifdef CONFIG_HIGH_RES_TIMERS
+	HRTIMER_SOFTIRQ,
+#endif
 };
 
 /* softirq mask and active fields moved to irq_cpustat_t in
include/linux/ktime.h +1 -2
Index: linux-2.6.18-mm2/include/linux/ktime.h
===================================================================
@@ -261,8 +261,7 @@ static inline u64 ktime_to_ns(const ktim
  * idea of the (in)accuracy of timers. Timer values are rounded up to
  * this resolution values.
  */
-#define KTIME_REALTIME_RES	(ktime_t){ .tv64 = TICK_NSEC }
-#define KTIME_MONOTONIC_RES	(ktime_t){ .tv64 = TICK_NSEC }
+#define KTIME_LOW_RES		(ktime_t){ .tv64 = TICK_NSEC }
 
 /* Get the monotonic time in timespec format: */
 extern void ktime_get_ts(struct timespec *ts);
kernel/hrtimer.c +553 -23

709 lines not on this page · raw message

kernel/itimer.c +1 -1
Index: linux-2.6.18-mm2/kernel/itimer.c
===================================================================
@@ -136,7 +136,7 @@ int it_real_fn(struct hrtimer *timer)
 	send_group_sig_info(SIGALRM, SEND_SIG_PRIV, sig->tsk);
 
 	if (sig->it_real_incr.tv64 != 0) {
-		hrtimer_forward(timer, timer->base->softirq_time,
+		hrtimer_forward(timer, hrtimer_cb_get_time(timer),
 				sig->it_real_incr);
 		return HRTIMER_RESTART;
 	}
kernel/posix-timers.c +1 -1
Index: linux-2.6.18-mm2/kernel/posix-timers.c
===================================================================
@@ -356,7 +356,7 @@ static int posix_timer_fn(struct hrtimer
 		if (timr->it.real.interval.tv64 != 0) {
 			timr->it_overrun +=
 				hrtimer_forward(timer,
-						timer->base->softirq_time,
+						hrtimer_cb_get_time(timer),
 						timr->it.real.interval);
 			ret = HRTIMER_RESTART;
 			++timr->it_requeue_pending;
kernel/time/Kconfig new file +22 -0
Index: linux-2.6.18-mm2/kernel/time/Kconfig
===================================================================
@@ -0,0 +1,22 @@
+#
+# Timer subsystem related configuration options
+#
+config HIGH_RES_TIMERS
+	bool "High Resolution Timer Support"
+	depends on GENERIC_TIME
+	help
+	  This option enables high resolution timer support. If your
+	  hardware is not capable then this option only increases
+	  the size of the kernel image.
+
+config HIGH_RES_RESOLUTION
+	int "High Resolution Timer resolution (nanoseconds)"
+	depends on HIGH_RES_TIMERS
+	default 1000
+	help
+	  This sets the resolution in nanoseconds of the high resolution
+	  timers. Too fine a resolution (small a number) will usually
+	  not be observable due to normal system latencies.  For an
+          800 MHz processor about 10,000 (10 microseconds) is recommended as a
+	  finest resolution.  If you don't need that sort of resolution,
+	  larger values may generate less overhead.
kernel/timer.c +1 -0
Index: linux-2.6.18-mm2/kernel/timer.c
===================================================================
@@ -1022,6 +1022,7 @@ static void update_wall_time(void)
 	if (change_clocksource()) {
 		clock->error = 0;
 		clock->xtime_nsec = 0;
+		hrtimer_clock_notify();
 		clocksource_calculate_interval(clock, tick_nsec);
 	}
 }

--

#36 lore raw

Re: [patch 15/23] high-res timers: core

Quoted 23 lines · On Fri, 29 Sep 2006 23:58:34 -0000 Thomas Gleixner <[email protected]> wrote:

From: Thomas Gleixner <[email protected]>

add the core bits of high-res timers support.

the design makes use of the existing hrtimers subsystem which manages a per-CPU and per-clock tree of timers, and the clockevents framework, which provides a standard API to request programmable clock events from. The core code does not have to know about the clock details - it makes use of clockevents_set_next_event().

the code also provides dyntick functionality: it is implemented via a per-cpu sched_tick hrtimer that is set to HZ frequency, but which is reprogrammed to a longer timeout before going idle, and reprogrammed to HZ again once the CPU goes busy again. (If an non-timer IRQ hits the idle task then it will process jiffies before calling the IRQ code.)

the impact to non-high-res architectures is intended to be minimal.

...

include/linux/hrtimer.h +0 -0
@@ -108,17 +134,53 @@ struct hrtimer_cpu_base {
 	spinlock_t			lock;
 	struct lock_class_key		lock_key;

include/linux/hrtimer.h struct hrtimer_cpu_base { · line 145

Quoted 10 lines
include/linux/hrtimer.h +9 -0
 	struct hrtimer_clock_base	clock_base[HRTIMER_MAX_CLOCK_BASES];
+#ifdef CONFIG_HIGH_RES_TIMERS
+	ktime_t				expires_next;
+	int				hres_active;
+	unsigned long			check_clocks;
+	struct list_head		cb_pending;
+	struct hrtimer			sched_timer;
+	struct pt_regs			*sched_regs;
+	unsigned long			events;
+#endif

You forgot to update the kerneldoc for this struct.

Does `events' needs to be long?

<looks>

oh, it's a scalar this time ;)

include/linux/hrtimer.h struct hrtimer_cpu_base { · line 155

Quoted 8 lines
include/linux/hrtimer.h +8 -0
+#ifdef CONFIG_HIGH_RES_TIMERS
+
+extern void hrtimer_clock_notify(void);
+extern void clock_was_set(void);
+extern void hrtimer_interrupt(struct pt_regs *regs);
+
+# define hrtimer_cb_get_time(t)	(t)->base->get_time()
+# define hrtimer_hres_active	(__get_cpu_var(hrtimer_bases).hres_active)

These two could be inline functions?

That might cause include file ordering problems I guess.

Quoted 14 lines
include/linux/hrtimer.h +13 -0
+/*
+ * The resolution of the clocks. The resolution value is returned in
+ * the clock_getres() system call to give application programmers an
+ * idea of the (in)accuracy of timers. Timer values are rounded up to
+ * this resolution values.
+ */
+# define KTIME_HIGH_RES		(ktime_t) { .tv64 = CONFIG_HIGH_RES_RESOLUTION }
+# define KTIME_MONOTONIC_RES	KTIME_HIGH_RES
+
+#else
+
+# define KTIME_MONOTONIC_RES	KTIME_LOW_RES
+
 /*

include/linux/hrtimer.h struct hrtimer_cpu_base { · line 175

Quoted 10 lines
include/linux/hrtimer.h +2 -4
  * clock_was_set() is a NOP for non- high-resolution systems. The
  * time-sorted order guarantees that a timer does not expire early and
  * is expired in the next softirq when the clock was advanced.
  */
-#define clock_was_set()		do { } while (0)
-#define hrtimer_clock_notify()	do { } while (0)
-extern ktime_t ktime_get(void);
-extern ktime_t ktime_get_real(void);
+# define clock_was_set()		do { } while (0)
+# define hrtimer_clock_notify()		do { } while (0)

these could be inlines.

include/linux/hrtimer.h struct hrtimer_cpu_base { · line 177

Quoted 1 line
include/linux/hrtimer.h +1 -0
+# define hrtimer_cb_get_time(t)		(t)->base->softirq_time

Does this need parenthesisation? Probably it's OK.. An inline function would be nicer.

Review struct hrtimer_cpu_base { · line 175

Quoted 1 line
(quoted diff) +1 -0
+# define hrtimer_hres_active		0

Perhaps this would be better if it was presented as a function.

kernel/hrtimer.c static unsigned long ktime_divns(const k · line 295

Quoted 4 lines
kernel/hrtimer.c +4 -0
+/* High resolution timer related functions */
+#ifdef CONFIG_HIGH_RES_TIMERS
+
+static ktime_t last_jiffies_update;

What's this do?

kernel/hrtimer.c static unsigned long ktime_divns(const k · line 298

Quoted 2 lines
kernel/hrtimer.c +2 -0
+/*
+ * Reprogramm the event source with checking both queues for the

"Reprogramme" ;)

kernel/hrtimer.c static unsigned long ktime_divns(const k · line 310

Quoted 12 lines
kernel/hrtimer.c +12 -0
+ * next event
+ * Called with interrupts disabled and base->lock held
+ */
+static void hrtimer_force_reprogram(struct hrtimer_cpu_base *cpu_base)
+{
+	int i;
+	struct hrtimer_clock_base *base = cpu_base->clock_base;
+	ktime_t expires;
+
+	cpu_base->expires_next.tv64 = KTIME_MAX;
+
+	for (i = HRTIMER_MAX_CLOCK_BASES; i ; i--, base++) {

Downcounting loops hurt my brain. Does it actually generate better code?

Quoted 8 lines
kernel/hrtimer.c +8 -0
+		struct hrtimer *timer;
+
+		if (!base->first)
+			continue;
+		timer = rb_entry(base->first, struct hrtimer, node);
+		expires = ktime_sub(timer->expires, base->offset);
+		if (expires.tv64 < cpu_base->expires_next.tv64)
+			cpu_base->expires_next = expires;

kernel/hrtimer.c static unsigned long ktime_divns(const k · line 328

Quoted 10 lines
kernel/hrtimer.c +10 -0
+	}
+
+	if (cpu_base->expires_next.tv64 != KTIME_MAX)
+		clockevents_set_next_event(cpu_base->expires_next, 1);
+}
+
+/*
+ * Shared reprogramming for clock_realtime and clock_monotonic
+ *
+ * When a new expires first timer is enqueued, we have

That sentence might need work.

Quoted 39 lines
kernel/hrtimer.c +39 -0
+/*
+ * Retrigger next event is called after clock was set
+ */
+static void retrigger_next_event(void *arg)
+{
+	struct hrtimer_cpu_base *base;
+	struct timespec realtime_offset;
+	unsigned long flags, seq;
+
+	do {
+		seq = read_seqbegin(&xtime_lock);
+		set_normalized_timespec(&realtime_offset,
+					-wall_to_monotonic.tv_sec,
+					-wall_to_monotonic.tv_nsec);
+	} while (read_seqretry(&xtime_lock, seq));
+
+	base = &per_cpu(hrtimer_bases, smp_processor_id());
+
+	/* Adjust CLOCK_REALTIME offset */
+	spin_lock_irqsave(&base->lock, flags);
+	base->clock_base[CLOCK_REALTIME].offset =
+		timespec_to_ktime(realtime_offset);
+
+	hrtimer_force_reprogram(base);
+	spin_unlock_irqrestore(&base->lock, flags);
+}
+
+/*
+ * Clock realtime was set
+ *
+ * Change the offset of the realtime clock vs. the monotonic
+ * clock.
+ *
+ * We might have to reprogram the high resolution timer interrupt. On
+ * SMP we call the architecture specific code to retrigger _all_ high
+ * resolution timer interrupts. On UP we just disable interrupts and
+ * call the high resolution interrupt code.
+ */
+void clock_was_set(void)

kernel/hrtimer.c static unsigned long ktime_divns(const k · line 403

Quoted 10 lines
kernel/hrtimer.c +10 -0
+{
+	preempt_disable();
+	if (hrtimer_hres_active) {
+		retrigger_next_event(NULL);
+
+		if (smp_call_function(retrigger_next_event, NULL, 1, 1))
+			BUG();
+	}
+	preempt_enable();
+}

If you use on_each_cpu() here you know that retrigger_next_event() will be called under local_irq_disable(). The preempt_disable() goes away and the spin_lock_irqsave() in retrigger_next_event() becomes a spin_lock() and everything becomes simpler.

kernel/hrtimer.c static unsigned long ktime_divns(const k · line 416

Quoted 12 lines
kernel/hrtimer.c +12 -0
+/**
+ * hrtimer_clock_notify - A clock source or a clock event has been installed
+ *
+ * Notify the per cpu softirqs to recheck the clock sources and events
+ */
+void hrtimer_clock_notify(void)
+{
+	int i;
+
+	for (i = 0; i < NR_CPUS; i++)
+		set_bit(0, &per_cpu(hrtimer_bases, i).check_clocks);
+}

This will go splat if/when the arch chooses to not implement per-cpu storage for not-possible CPUs. Use for_each_possible_cpu().

kernel/hrtimer.c static unsigned long ktime_divns(const k · line 420

Quoted 3 lines
kernel/hrtimer.c +3 -0
+
+static const ktime_t nsec_per_hz = { .tv64 = NSEC_PER_SEC / HZ };
+

This could use the same trick as KTIME_HIGH_RES and friends. But perhaps the compiler will generate the same code..

Quoted 10 lines
kernel/hrtimer.c +10 -0
+/*
+ * We switched off the global tick source when switching to high resolution
+ * mode. Update jiffies64.
+ *
+ * Must be called with interrupts disabled !
+ *
+ * FIXME: We need a mechanism to assign the update to a CPU. In principle this
+ * is not hard, but when dynamic ticks come into play it starts to be. We don't
+ * want to wake up a complete idle cpu just to update jiffies, so we need
+ * something more intellegent than a mere "do this only on CPUx".

kernel/hrtimer.c static unsigned long ktime_divns(const k · line 440

Quoted 10 lines
kernel/hrtimer.c +10 -0
+ */
+static void update_jiffies64(ktime_t now)
+{
+	ktime_t delta;
+
+	write_seqlock(&xtime_lock);
+
+	delta = ktime_sub(now, last_jiffies_update);
+	if (delta.tv64 >= nsec_per_hz.tv64) {
+

stray blank line.

kernel/hrtimer.c static unsigned long ktime_divns(const k · line 441

Quoted 1 line
kernel/hrtimer.c +1 -0
+		unsigned long orun = 1;

"orun"?

Quoted 5 lines
kernel/hrtimer.c +5 -0
+
+		delta = ktime_sub(delta, nsec_per_hz);
+		last_jiffies_update = ktime_add(last_jiffies_update,
+						nsec_per_hz);
+

kernel/hrtimer.c static unsigned long ktime_divns(const k · line 456

Quoted 10 lines
kernel/hrtimer.c +10 -0
+		/* Slow path for long timeouts */
+		if (unlikely(delta.tv64 >= nsec_per_hz.tv64)) {
+			s64 incr = ktime_to_ns(nsec_per_hz);
+			orun = ktime_divns(delta, incr);
+
+			last_jiffies_update = ktime_add_ns(last_jiffies_update,
+							   incr * orun);
+			jiffies_64 += orun;
+			orun++;
+		}

That's a bit of a hack isn't it? do_timer() owns the modification of jiffies_64, so why is this code modifying it as well?

kernel/hrtimer.c static unsigned long ktime_divns(const k · line 457

Quoted 1 line
kernel/hrtimer.c +1 -0
+		do_timer(orun);

twice?

I suspect a bug.

Quoted 17 lines
kernel/hrtimer.c +17 -0
+	}
+	write_sequnlock(&xtime_lock);
+}
+
+/*
+ * We rearm the timer until we get disabled by the idle code
+ */
+static int hrtimer_sched_tick(struct hrtimer *timer)
+{
+	unsigned long flags;
+	struct hrtimer_cpu_base *cpu_base =
+		container_of(timer, struct hrtimer_cpu_base, sched_timer);
+
+	local_irq_save(flags);
+	/*
+	 * Do not call, when we are not in irq context and have
+	 * no valid regs pointer

kernel/hrtimer.c static unsigned long ktime_divns(const k · line 484

Quoted 10 lines
kernel/hrtimer.c +10 -0
+	 */
+	if (cpu_base->sched_regs) {
+		update_process_times(user_mode(cpu_base->sched_regs));
+		profile_tick(CPU_PROFILING, cpu_base->sched_regs);
+	}
+
+	hrtimer_forward(timer, hrtimer_cb_get_time(timer), nsec_per_hz);
+	local_irq_restore(flags);
+
+	return HRTIMER_RESTART;

bah. hrtimer_restart is an `enum hrtimer_restart', not an integer.

kernel/hrtimer.c static unsigned long ktime_divns(const k · line 544

Quoted 2 lines
kernel/hrtimer.c +2 -0
+	printk(KERN_INFO "hrtimers: Switched to high resolution mode CPU %d\n",
+	       smp_processor_id());

"on CPU"

kernel/hrtimer.c static unsigned long ktime_divns(const k · line 586

Quoted 11 lines
kernel/hrtimer.c +11 -0
+
+static inline int hrtimer_enqueue_reprogram(struct hrtimer *timer,
+					    struct hrtimer_clock_base *base)
+{
+	/*
+	 * When High resolution timers are active try to reprogram. Note, that
+	 * in case the state has HRTIMER_CALLBACK set, no reprogramming and no
+	 * expiry check happens. The timer gets enqueued into the rbtree and
+	 * the reprogramming / expiry check is done in the hrtimer_interrupt or
+	 * in the softirq.
+	 */

This (useful) comment should be above the function, not inside it.

kernel/hrtimer.c static unsigned long ktime_divns(const k · line 596

Quoted 10 lines
kernel/hrtimer.c +10 -0
+	if (hrtimer_hres_active && hrtimer_reprogram(timer, base)) {
+
+		/* Timer is expired, act upon the callback mode */
+		switch(timer->cb_mode) {
+		case HRTIMER_CB_IRQSAFE_NO_RESTART:
+			/*
+			 * We can call the callback from here. No restart
+			 * happens, so no danger of recursion
+			 */
+			BUG_ON(timer->function(timer) != HRTIMER_NORESTART);

Doing assert(thing-which-has-side-effects) is poor form.

I doubt if the kernel will work if someone goes and disables BUG_ON, but it's a laudable objective.

Quoted 16 lines
kernel/hrtimer.c +16 -0
+			return 1;
+		case HRTIMER_CB_IRQSAFE_NO_SOFTIRQ:
+			/*
+			 * This is solely for the sched tick emulation with
+			 * dynamic tick support to ensure that we do not
+			 * restart the tick right on the edge and end up with
+			 * the tick timer in the softirq ! The calling site
+			 * takes care of this.
+			 */
+			return 1;
+		case HRTIMER_CB_IRQSAFE:
+		case HRTIMER_CB_SOFTIRQ:
+			/*
+			 * Move everything else into the softirq pending list !
+			 */
+			hrtimer_add_cb_pending(timer, base);

kernel/hrtimer.c static unsigned long ktime_divns(const k · line 622

Quoted 10 lines
kernel/hrtimer.c +10 -0
+			raise_softirq(HRTIMER_SOFTIRQ);
+			return 1;
+		default:
+			BUG();
+		}
+	}
+	return 0;
+}
+
+static inline void hrtimer_resume_jiffie_update(void)

hrtimer_resume_jiffy_update

Quoted 15 lines
kernel/hrtimer.c +15 -0
+{
+	unsigned long flags;
+	ktime_t now = ktime_get();
+
+	write_seqlock_irqsave(&xtime_lock, flags);
+	last_jiffies_update = now;
+	write_sequnlock_irqrestore(&xtime_lock, flags);
+}
+
+#else
+
+# define hrtimer_hres_active		0
+# define hrtimer_check_clocks()		do { } while (0)
+# define hrtimer_enqueue_reprogram(t,b)	0
+# define hrtimer_force_reprogram(b)	do { } while (0)

kernel/hrtimer.c static unsigned long ktime_divns(const k · line 647

Quoted 10 lines
kernel/hrtimer.c +8 -0
+# define hrtimer_cb_pending(t)		0
+# define hrtimer_remove_cb_pending(t)	do { } while (0)
+# define hrtimer_init_hres(c)		do { } while (0)
+# define hrtimer_init_timer_hres(t)	do { } while (0)
+# define hrtimer_resume_jiffie_update()	do { } while (0)
+
+#endif /* CONFIG_HIGH_RES_TIMERS */
+
 /*
  * Timekeeping resumed notification

resume

Quoted 26 lines
kernel/hrtimer.c +26 -0
+#ifdef CONFIG_HIGH_RES_TIMERS
+
+/*
+ * High resolution timer interrupt
+ * Called with interrupts disabled
+ */
+void hrtimer_interrupt(struct pt_regs *regs)
+{
+	struct hrtimer_clock_base *base;
+	ktime_t expires_next, now;
+	int i, raise = 0, cpu = smp_processor_id();
+	struct hrtimer_cpu_base *cpu_base = &per_cpu(hrtimer_bases, cpu);
+
+	BUG_ON(!cpu_base->hres_active);
+
+	/* Store the regs for an possible sched_timer callback */
+	cpu_base->sched_regs = regs;
+	cpu_base->events++;
+
+ retry:
+	now = ktime_get();
+
+	/* Check, if the jiffies need an update */
+	update_jiffies64(now);
+
+	expires_next.tv64 = KTIME_MAX;

kernel/hrtimer.c int hrtimer_get_res(const clockid_t whic · line 1033

Quoted 10 lines
kernel/hrtimer.c +10 -0
+
+	base = cpu_base->clock_base;
+
+	for (i = 0; i < HRTIMER_MAX_CLOCK_BASES; i++) {
+		ktime_t basenow;
+		struct rb_node *node;
+
+		spin_lock(&cpu_base->lock);
+
+		basenow = ktime_add(now, base->offset);

Would it be better to take the lock outside the loop, rather than hammering on it like this?

Quoted 61 lines
kernel/hrtimer.c +56 -0
+		while ((node = base->first)) {
+			struct hrtimer *timer;
+
+			timer = rb_entry(node, struct hrtimer, node);
+
+			if (basenow.tv64 < timer->expires.tv64) {
+				ktime_t expires;
+
+				expires = ktime_sub(timer->expires,
+						    base->offset);
+				if (expires.tv64 < expires_next.tv64)
+					expires_next = expires;
+				break;
+			}
+
+			/* Move softirq callbacks to the pending list */
+			if (timer->cb_mode == HRTIMER_CB_SOFTIRQ) {
+				__remove_hrtimer(timer, base, HRTIMER_PENDING, 0);
+				hrtimer_add_cb_pending(timer, base);
+				raise = 1;
+				continue;
+			}
+
+			__remove_hrtimer(timer, base, HRTIMER_CALLBACK, 0);
+
+			if (timer->function(timer) != HRTIMER_NORESTART) {
+				BUG_ON(timer->state != HRTIMER_CALLBACK);
+				/*
+				 * state == HRTIMER_CALLBACK prevents
+				 * reprogramming. We do this when we break out
+				 * of the loop !
+				 */
+				enqueue_hrtimer(timer, base);
+			}
+			timer->state &= ~HRTIMER_CALLBACK;
+		}
+		spin_unlock(&cpu_base->lock);
+		base++;
+	}
+
+	cpu_base->expires_next = expires_next;
+
+	/* Reprogramming necessary ? */
+	if (expires_next.tv64 != KTIME_MAX) {
+		if (clockevents_set_next_event(expires_next, 0))
+			goto retry;
+	}
+
+	/* Invalidate regs */
+	cpu_base->sched_regs = NULL;
+
+	/* Raise softirq ? */
+	if (raise)
+		raise_softirq(HRTIMER_SOFTIRQ);
+}
+
 
...
 
 static int __sched do_nanosleep(struct hrtimer_sleeper *t, enum hrtimer_mode mode)

kernel/hrtimer.c static int __sched do_nanosleep(struct h · line 1230

Quoted 6 lines
kernel/hrtimer.c +2 -1
@@ -701,7 +1226,8 @@ static int __sched do_nanosleep(struct h
 		set_current_state(TASK_INTERRUPTIBLE);
 		hrtimer_start(&t->timer, t->timer.expires, mode);
 
-		schedule();
+		if (likely(t->task))
+			schedule();

Why? Needs a comment.

Quoted 13 lines
kernel/time/Kconfig new file +12 -0
@@ -0,0 +1,22 @@
+#
+# Timer subsystem related configuration options
+#
+config HIGH_RES_TIMERS
+	bool "High Resolution Timer Support"
+	depends on GENERIC_TIME
+	help
+	  This option enables high resolution timer support. If your
+	  hardware is not capable then this option only increases
+	  the size of the kernel image.
+
+config HIGH_RES_RESOLUTION

kernel/time/Kconfig line 22

Quoted 10 lines
kernel/time/Kconfig new file +10 -0
+	int "High Resolution Timer resolution (nanoseconds)"
+	depends on HIGH_RES_TIMERS
+	default 1000
+	help
+	  This sets the resolution in nanoseconds of the high resolution
+	  timers. Too fine a resolution (small a number) will usually
+	  not be observable due to normal system latencies.  For an
+          800 MHz processor about 10,000 (10 microseconds) is recommended as a
+	  finest resolution.  If you don't need that sort of resolution,
+	  larger values may generate less overhead.

In that case the default is far too low.

What value are you suggesting that users and vendors set it to?

#37 lore raw

PATCH 16/23 dynticks: core

In reply to Thomas Gleixner

From: Ingo Molnar <[email protected]>

dynticks core code.

Add idling-stats to the cpu base (to be used to optimize power management decisions), add the scheduler tick and its stop/restart functions, and the jiffies-update function to be called when an irq context hits the idle context.

--

 include/linux/hrtimer.h |   28 +++++++
 kernel/hrtimer.c        |  185 ++++++++++++++++++++++++++++++++++++++++++++++--
 kernel/softirq.c        |   11 ++
 kernel/time/Kconfig     |    8 ++
 kernel/timer.c          |    2 
 5 files changed, 226 insertions(+), 8 deletions(-)
include/linux/hrtimer.h +27 -1
Index: linux-2.6.18-mm2/include/linux/hrtimer.h
===================================================================
@@ -142,6 +142,14 @@ struct hrtimer_cpu_base {
 	struct hrtimer			sched_timer;
 	struct pt_regs			*sched_regs;
 	unsigned long			events;
+#ifdef CONFIG_NO_HZ
+	ktime_t				idle_tick;
+	int				tick_stopped;
+	unsigned long			idle_jiffies;
+	unsigned long			idle_calls;
+	unsigned long			idle_sleeps;
+	unsigned long			idle_sleeptime;
+#endif
 #endif
 };
 
@@ -200,7 +208,7 @@ extern int hrtimer_try_to_cancel(struct 
 extern ktime_t hrtimer_get_remaining(const struct hrtimer *timer);
 extern int hrtimer_get_res(const clockid_t which_clock, struct timespec *tp);
 
-#ifdef CONFIG_NO_IDLE_HZ
+#if defined(CONFIG_NO_IDLE_HZ) || defined(CONFIG_NO_HZ)
 extern ktime_t hrtimer_get_next_event(void);
 #endif
 
@@ -229,6 +237,24 @@ extern void hrtimer_run_queues(void);
 /* Resume notification */
 void hrtimer_notify_resume(void);
 
+#ifdef CONFIG_NO_HZ
+extern void hrtimer_trigger_next_hz_tick(struct tvec_t_base_s *base);
+extern int hrtimer_stop_sched_tick(void);
+extern void hrtimer_restart_sched_tick(void);
+extern void update_jiffies(void);
+struct seq_file;
+extern void show_no_hz_stats(struct seq_file *p);
+#else
+# define hrtimer_trigger_next_hz_tick(base)	do { } while (0)
+static inline int hrtimer_stop_sched_tick(void)
+{
+	return 0;
+}
+# define hrtimer_restart_sched_tick()		do { } while (0)
+# define update_jiffies()			do { } while (0)
+# define show_no_hz_stats(p)			do { } while (0)
+#endif
+
 /* Bootup initialization: */
 extern void __init hrtimers_init(void);
 
kernel/hrtimer.c +179 -6

240 lines not on this page · raw message

kernel/softirq.c +11 -0
Index: linux-2.6.18-mm2/kernel/softirq.c
===================================================================
@@ -284,6 +284,11 @@ extern void irq_enter(void)
 	account_system_vtime(current);
 	add_preempt_count(HARDIRQ_OFFSET);
 	trace_hardirq_enter();
+
+#ifdef CONFIG_NO_HZ
+	if (idle_cpu(smp_processor_id()))
+		update_jiffies();
+#endif
 }
 
 /*
@@ -296,6 +301,12 @@ void irq_exit(void)
 	sub_preempt_count(IRQ_EXIT_OFFSET);
 	if (!in_interrupt() && local_softirq_pending())
 		invoke_softirq();
+
+#ifdef CONFIG_NO_HZ
+	/* Make sure that timer wheel updates are propagated */
+	if (!in_interrupt() && idle_cpu(smp_processor_id()) && !need_resched())
+		hrtimer_stop_sched_tick();
+#endif
 	preempt_enable_no_resched();
 }
 
kernel/time/Kconfig +8 -0
Index: linux-2.6.18-mm2/kernel/time/Kconfig
===================================================================
@@ -20,3 +20,11 @@ config HIGH_RES_RESOLUTION
           800 MHz processor about 10,000 (10 microseconds) is recommended as a
 	  finest resolution.  If you don't need that sort of resolution,
 	  larger values may generate less overhead.
+
+config NO_HZ
+	bool "Tickless System (Dynamic Ticks)"
+	depends on GENERIC_TIME && HIGH_RES_TIMERS
+	help
+	  This option enables a tickless system: timer interrupts will
+	  only trigger on an as-needed basis both when the system is
+	  busy and when the system is idle.
kernel/timer.c +1 -1
Index: linux-2.6.18-mm2/kernel/timer.c
===================================================================
@@ -462,7 +462,7 @@ static inline void __run_timers(tvec_bas
 	spin_unlock_irq(&base->lock);
 }
 
-#ifdef CONFIG_NO_IDLE_HZ
+#if defined(CONFIG_NO_IDLE_HZ) || defined(CONFIG_NO_HZ)
 /*
  * Find out when the next timer event is due to happen. This
  * is used on S/390 to stop all activity when a cpus is idle.

--

#38 lore raw

Re: [patch 16/23] dynticks: core

Quoted 9 lines · On Fri, 29 Sep 2006 23:58:35 -0000 Thomas Gleixner <[email protected]> wrote:

From: Ingo Molnar <[email protected]>

dynticks core code.

Add idling-stats to the cpu base (to be used to optimize power management decisions), add the scheduler tick and its stop/restart functions, and the jiffies-update function to be called when an irq context hits the idle context.

I worry that we're making this feature optional.

Certainly for the public testing period we should wire these new features to "on".

But long-term this is yet another question which we'll need to ask when we're trying to work out why someone's computer failed.

Quoted 4 lines
include/linux/hrtimer.h +0 -0
@@ -142,6 +142,14 @@ struct hrtimer_cpu_base {
 	struct hrtimer			sched_timer;

include/linux/hrtimer.h struct hrtimer_cpu_base { · line 152

Quoted 10 lines
include/linux/hrtimer.h +8 -0
 	struct pt_regs			*sched_regs;
 	unsigned long			events;
+#ifdef CONFIG_NO_HZ
+	ktime_t				idle_tick;
+	int				tick_stopped;
+	unsigned long			idle_jiffies;
+	unsigned long			idle_calls;
+	unsigned long			idle_sleeps;
+	unsigned long			idle_sleeptime;
+#endif

Forgot to update this structure's kerneldoc.

include/linux/hrtimer.h extern void hrtimer_run_queues(void); · line 255

Quoted 1 line
include/linux/hrtimer.h +1 -0
+# define show_no_hz_stats(p)			do { } while (0)

static inlines provide type checking.

include/linux/hrtimer.h static void update_jiffies64(ktime_t now · line 454

Quoted 7 lines
include/linux/hrtimer.h +0 -1
@@ -451,7 +450,6 @@ static void update_jiffies64(ktime_t now
 
 			last_jiffies_update = ktime_add_ns(last_jiffies_update,
 							   incr * orun);
-			jiffies_64 += orun;
 			orun++;
 		}

I think we just fixed that bug I might have seen.

include/linux/hrtimer.h static void update_jiffies64(ktime_t now · line 462

Quoted 8 lines
include/linux/hrtimer.h +3 -0
 		do_timer(orun);
@@ -459,28 +457,201 @@ static void update_jiffies64(ktime_t now
 	write_sequnlock(&xtime_lock);
 }
 
+#ifdef CONFIG_NO_HZ
+/*
+ * Called from interrupt entry when then CPU was idle

tpyo

Quoted 8 lines
include/linux/hrtimer.h +8 -0
+ */
+void update_jiffies(void)
+{
+	unsigned long flags;
+	ktime_t now;
+
+	if (unlikely(!hrtimer_hres_active))
+		return;

include/linux/hrtimer.h static void update_jiffies64(ktime_t now · line 480

Quoted 10 lines
include/linux/hrtimer.h +10 -0
+
+	now = ktime_get();
+
+	local_irq_save(flags);
+	update_jiffies64(now);
+	local_irq_restore(flags);
+}
+
+/*
+ * Called from the idle thread so careful!

about what?

Quoted 4 lines
include/linux/hrtimer.h +4 -0
+ */
+int hrtimer_stop_sched_tick(void)
+{
+	int cpu = smp_processor_id();

include/linux/hrtimer.h static void update_jiffies64(ktime_t now · line 494

Quoted 10 lines
include/linux/hrtimer.h +10 -0
+	struct hrtimer_cpu_base *cpu_base = &per_cpu(hrtimer_bases, cpu);
+	unsigned long seq, last_jiffies, next_jiffies;
+	ktime_t last_update, expires;
+	unsigned long delta_jiffies;
+	unsigned long flags;
+
+	if (unlikely(!hrtimer_hres_active))
+		return 0;
+
+	local_irq_save(flags);

Do we really need local_irq_save() here? If it's called from the idle thread then presumably local IRQs are enabled already. They'd better be, because this function unconditionally enables them in a couple of places.

Quoted 22 lines
include/linux/hrtimer.h +22 -0
+	do {
+		seq = read_seqbegin(&xtime_lock);
+		last_update = last_jiffies_update;
+		last_jiffies = jiffies;
+	} while (read_seqretry(&xtime_lock, seq));
+
+	next_jiffies = get_next_timer_interrupt(last_jiffies);
+	delta_jiffies = next_jiffies - last_jiffies;
+
+	cpu_base->idle_calls++;
+
+	if ((long)delta_jiffies >= 1) {
+		/*
+		 * Save the current tick time, so we can restart the
+		 * scheduler tick when we get woken up before the next
+		 * wheel timer expires
+		 */
+		cpu_base->idle_tick = cpu_base->sched_timer.expires;
+		expires = ktime_add_ns(last_update,
+				       nsec_per_hz.tv64 * delta_jiffies);
+		hrtimer_start(&cpu_base->sched_timer, expires, HRTIMER_ABS);
+		cpu_base->idle_sleeps++;

include/linux/hrtimer.h static void update_jiffies64(ktime_t now · line 526

Quoted 10 lines
include/linux/hrtimer.h +10 -0
+		cpu_base->idle_jiffies = last_jiffies;
+		cpu_base->tick_stopped = 1;
+	} else {
+		/* Keep the timer alive */
+		if ((long) delta_jiffies < 0)
+			raise_softirq(TIMER_SOFTIRQ);
+	}
+
+	if (local_softirq_pending()) {
+		inc_preempt_count();

I am unable to work out why the inc_preempt_count() is there. Please add comment.

Quoted 14 lines
include/linux/hrtimer.h +14 -0
+		do_softirq();
+		dec_preempt_count();
+	}
+
+	WARN_ON(!idle_cpu(cpu));
+	/*
+	 * RCU normally depends on the timer IRQ kicking completion
+	 * in every tick. We have to do this here now:
+	 */
+	if (rcu_pending(cpu)) {
+		/*
+		 * We are in quiescent state, so advance callbacks:
+		 */
+		rcu_advance_callbacks(cpu, 1);

include/linux/hrtimer.h static void update_jiffies64(ktime_t now · line 550

Quoted 10 lines
include/linux/hrtimer.h +10 -0
+		local_irq_enable();
+		local_bh_disable();
+		rcu_process_callbacks(0);
+		local_bh_enable();
+	}
+
+	local_irq_restore(flags);
+
+	return need_resched();
+}

Are the RCU guys OK with this?

include/linux/hrtimer.h static void update_jiffies64(ktime_t now · line 551

Quoted 1 line
include/linux/hrtimer.h +1 -0
+void hrtimer_restart_sched_tick(void)

Am unable to work out what this does from its implementation and from its caller. Please document it.

include/linux/hrtimer.h static void update_jiffies64(ktime_t now · line 563

Quoted 12 lines
include/linux/hrtimer.h +12 -0
+{
+	struct hrtimer_cpu_base *cpu_base = &__get_cpu_var(hrtimer_bases);
+	unsigned long flags;
+	ktime_t now;
+
+	if (!hrtimer_hres_active || !cpu_base->tick_stopped)
+		return;
+
+	/* Update jiffies first */
+	now = ktime_get();
+
+	local_irq_save(flags);

The sole caller of this function calls it with local interrupts enabled. local_irq_disable() could be used here.

Quoted 43 lines
include/linux/hrtimer.h +43 -0
+	update_jiffies64(now);
+
+	/*
+	 * Update process times would randomly account the time we slept to
+	 * whatever the context of the next sched tick is.  Enforce that this
+	 * is accounted to idle !
+	 */
+	add_preempt_count(HARDIRQ_OFFSET);
+	update_process_times(0);
+	sub_preempt_count(HARDIRQ_OFFSET);
+
+	cpu_base->idle_sleeptime += jiffies - cpu_base->idle_jiffies;
+
+	cpu_base->tick_stopped  = 0;
+	hrtimer_cancel(&cpu_base->sched_timer);
+	cpu_base->sched_timer.expires = cpu_base->idle_tick;
+
+	while (1) {
+		hrtimer_forward(&cpu_base->sched_timer, now, nsec_per_hz);
+		hrtimer_start(&cpu_base->sched_timer,
+			      cpu_base->sched_timer.expires, HRTIMER_ABS);
+		if (hrtimer_active(&cpu_base->sched_timer))
+			break;
+		/* We missed an update */
+		update_jiffies64(now);
+		now = ktime_get();
+	}
+	local_irq_restore(flags);
+}
+
+void show_no_hz_stats(struct seq_file *p)
+{
+	int cpu;
+	unsigned long calls = 0, sleeps = 0, time = 0, events = 0;
+
+	for_each_online_cpu(cpu) {
+		struct hrtimer_cpu_base *base = &per_cpu(hrtimer_bases, cpu);
+
+		calls += base->idle_calls;
+		sleeps += base->idle_sleeps;
+		time += base->idle_sleeptime;
+		events += base->events;
+

include/linux/hrtimer.h static void update_jiffies64(ktime_t now · line 616

Quoted 10 lines
include/linux/hrtimer.h +10 -0
+		seq_printf(p, "nohz cpu%d I:%lu S:%lu T:%lu A:%lu E: %lu\n",
+			   cpu, base->idle_calls, base->idle_sleeps,
+			   base->idle_sleeptime, base->idle_sleeps ?
+			   base->idle_sleeptime / sleeps : 0, base->events);
+	}
+#ifdef CONFIG_SMP
+	seq_printf(p, "nohz total I:%lu S:%lu T:%lu A:%lu E:%lu\n",
+		   calls, sleeps, time, sleeps ? time / sleeps : 0, events);
+#endif
+}

Wouldn't it be better to display the "total" line on UP rather than cpu0?

#39 lore raw

Re: [patch 16/23] dynticks: core

Quoted 27 lines · Andrew Morton
Quoted 9 lines · On Fri, 29 Sep 2006 23:58:35 -0000 Thomas Gleixner <[email protected]> wrote:

From: Ingo Molnar <[email protected]>

dynticks core code.

Add idling-stats to the cpu base (to be used to optimize power management decisions), add the scheduler tick and its stop/restart functions, and the jiffies-update function to be called when an irq context hits the idle context.

I worry that we're making this feature optional.

Quoted 13 lines
kernel/hrtimer.c +13 -0
+	/*
+	 * RCU normally depends on the timer IRQ kicking completion
+	 * in every tick. We have to do this here now:
+	 */
+	if (rcu_pending(cpu)) {
+		/*
+		 * We are in quiescent state, so advance callbacks:
+		 */
+		rcu_advance_callbacks(cpu, 1);
+		local_irq_enable(); <----------------- Here
+		local_bh_disable();
+		rcu_process_callbacks(0);
+		local_bh_enable();
Quoted 8 lines

kernel/hrtimer.c static void update_jiffies64(ktime_t now · line 551

Quoted 6 lines
kernel/hrtimer.c +6 -0
+	}
+
+	local_irq_restore(flags);
+
+	return need_resched();
+}

Are the RCU guys OK with this?

What prevents more RCU callbacks getting queued up by an irq after irqs are enabled (marked Here) ? This seems racy. The s390 implementation is correct - there we back out if RCU is pending. Also, one call
to rcu_process_callbacks() doesn't guarantee that all the RCUs are processed. They can be rate limited.

Thanks
Dipankar

#40 lore raw

PATCH 17/23 dyntick: add nohz stats to /proc/stat

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

add nohz stats to /proc/stat.

--

 fs/proc/proc_misc.c |    2 ++
 1 file changed, 2 insertions(+)
fs/proc/proc_misc.c +2 -0
Index: linux-2.6.18-mm2/fs/proc/proc_misc.c
===================================================================
@@ -527,6 +527,8 @@ static int show_stat(struct seq_file *p,
 		nr_running(),
 		nr_iowait());
 
+	show_no_hz_stats(p);
+
 	return 0;
 }
 

--

#41 lore raw

PATCH 18/23 dynticks: i386 arch code

In reply to Thomas Gleixner

From: Ingo Molnar <[email protected]>

prepare i386 for dyntick: idle handler callbacks and IRQ callback.

----

 arch/i386/kernel/nmi.c     |    3 ++-
 arch/i386/kernel/process.c |   27 +++++++++++++++------------
 2 files changed, 17 insertions(+), 13 deletions(-)
arch/i386/kernel/nmi.c +2 -1
Index: linux-2.6.18-mm2/arch/i386/kernel/nmi.c
===================================================================
@@ -20,6 +20,7 @@
 #include <linux/sysdev.h>
 #include <linux/sysctl.h>
 #include <linux/percpu.h>
+#include <linux/kernel_stat.h>
 #include <linux/dmi.h>
 #include <linux/kprobes.h>
 
@@ -908,7 +909,7 @@ __kprobes int nmi_watchdog_tick(struct p
 		touched = 1;
 	}
 
-	sum = per_cpu(irq_stat, cpu).apic_timer_irqs;
+	sum = per_cpu(irq_stat, cpu).apic_timer_irqs + kstat_irqs(0);
 
 	/* if the apic timer isn't firing, this cpu isn't doing much */
 	if (!touched && last_irq_sums[cpu] == sum) {
arch/i386/kernel/process.c +15 -12
Index: linux-2.6.18-mm2/arch/i386/kernel/process.c
===================================================================
@@ -178,24 +178,27 @@ void cpu_idle(void)
 
 	/* endless idle loop with no priority at all */
 	while (1) {
-		while (!need_resched()) {
-			void (*idle)(void);
+		if (!hrtimer_stop_sched_tick()) {
+			while (!need_resched()) {
+				void (*idle)(void);
 
-			if (__get_cpu_var(cpu_idle_state))
-				__get_cpu_var(cpu_idle_state) = 0;
+				if (__get_cpu_var(cpu_idle_state))
+					__get_cpu_var(cpu_idle_state) = 0;
 
-			rmb();
-			idle = pm_idle;
+				rmb();
+				idle = pm_idle;
 
-			if (!idle)
-				idle = default_idle;
+				if (!idle)
+					idle = default_idle;
 
-			if (cpu_is_offline(cpu))
-				play_dead();
+				if (cpu_is_offline(cpu))
+					play_dead();
 
-			__get_cpu_var(irq_stat).idle_timestamp = jiffies;
-			idle();
+				__get_cpu_var(irq_stat).idle_timestamp = jiffies;
+				idle();
+			}
 		}
+		hrtimer_restart_sched_tick();
 		preempt_enable_no_resched();
 		schedule();
 		preempt_disable();

--

#42 lore raw

Re: [patch 18/23] dynticks: i386 arch code

Quoted 17 lines · On Fri, 29 Sep 2006 23:58:37 -0000 Thomas Gleixner <[email protected]> wrote:

From: Ingo Molnar <[email protected]>

prepare i386 for dyntick: idle handler callbacks and IRQ callback.

Index: linux-2.6.18-mm2/arch/i386/kernel/nmi.c
===================================================================

arch/i386/kernel/nmi.c +1 -0
@@ -20,6 +20,7 @@
 #include <linux/sysdev.h>
 #include <linux/sysctl.h>
 #include <linux/percpu.h>
+#include <linux/kernel_stat.h>
 #include <linux/dmi.h>
 #include <linux/kprobes.h>
 

arch/i386/kernel/nmi.c __kprobes int nmi_watchdog_tick(struct p · line 912

Quoted 5 lines
arch/i386/kernel/nmi.c +1 -1
@@ -908,7 +909,7 @@ __kprobes int nmi_watchdog_tick(struct p
 		touched = 1;
 	}
 
-	sum = per_cpu(irq_stat, cpu).apic_timer_irqs;
+	sum = per_cpu(irq_stat, cpu).apic_timer_irqs + kstat_irqs(0);

Why?

arch/i386/kernel/process.c void cpu_idle(void) · line 182

Quoted 11 lines

===================================================================

arch/i386/kernel/process.c +2 -2
@@ -178,24 +178,27 @@ void cpu_idle(void)
 
 	/* endless idle loop with no priority at all */
 	while (1) {
-		while (!need_resched()) {
-			void (*idle)(void);
+		if (!hrtimer_stop_sched_tick()) {
+			while (!need_resched()) {

I don't see why hrtimer_stop_sched_tick() returns need_resched(). We immediately reevaluate it anyway. hrtimer_stop_sched_tick() could return 1.

Quoted 33 lines
arch/i386/kernel/process.c +13 -10
+				void (*idle)(void);
 
-			if (__get_cpu_var(cpu_idle_state))
-				__get_cpu_var(cpu_idle_state) = 0;
+				if (__get_cpu_var(cpu_idle_state))
+					__get_cpu_var(cpu_idle_state) = 0;
 
-			rmb();
-			idle = pm_idle;
+				rmb();
+				idle = pm_idle;
 
-			if (!idle)
-				idle = default_idle;
+				if (!idle)
+					idle = default_idle;
 
-			if (cpu_is_offline(cpu))
-				play_dead();
+				if (cpu_is_offline(cpu))
+					play_dead();
 
-			__get_cpu_var(irq_stat).idle_timestamp = jiffies;
-			idle();
+				__get_cpu_var(irq_stat).idle_timestamp = jiffies;
+				idle();
+			}
 		}
+		hrtimer_restart_sched_tick();
 		preempt_enable_no_resched();
 		schedule();
 		preempt_disable();
#43 lore raw

PATCH 19/23 high-res timers, dynticks: enable i386 support

In reply to Thomas Gleixner

From: Ingo Molnar <[email protected]>

enable high-res timers and dyntick on i386.

--

 arch/i386/Kconfig |    2 ++
 1 file changed, 2 insertions(+)
arch/i386/Kconfig +2 -0
Index: linux-2.6.18-mm2/arch/i386/Kconfig
===================================================================
@@ -61,6 +61,8 @@ source "init/Kconfig"
 
 menu "Processor type and features"
 
+source "kernel/time/Kconfig"
+
 config SMP
 	bool "Symmetric multi-processing support"
 	---help---

--

#44 lore raw

PATCH 20/23 add /proc/sys/kernel/timeout_granularity

In reply to Thomas Gleixner

From: Ingo Molnar <[email protected]>

Introduce timeout granularity: process timer wheel timers every timeout_granularity jiffies. Defaults to 1 (process timers HZ times per second - most finegrained).

--

 Documentation/kernel-parameters.txt |    6 ++++++
 include/linux/sysctl.h              |    1 +
 include/linux/timer.h               |    1 +
 kernel/sysctl.c                     |   10 ++++++++++
 kernel/timer.c                      |   24 +++++++++++++++++++++---
 5 files changed, 39 insertions(+), 3 deletions(-)
Documentation/kernel-parameters.txt +6 -0
Index: linux-2.6.18-mm2/Documentation/kernel-parameters.txt
===================================================================
@@ -1637,6 +1637,12 @@ and is between 256 and 4096 characters. 
 
 	time		Show timing data prefixed to each printk message line
 
+	timeout_granularity=
+			[KNL]
+			Timeout granularity: process timer wheel timers every
+			timeout_granularity jiffies. Defaults to 1 (process
+			timers HZ times per second - most finegrained).
+
 	clocksource=	[GENERIC_TIME] Override the default clocksource
 			Override the default clocksource and use the clocksource
 			with the name specified.
include/linux/sysctl.h +1 -0
Index: linux-2.6.18-mm2/include/linux/sysctl.h
===================================================================
@@ -153,6 +153,7 @@ enum
 	KERN_MAX_LOCK_DEPTH=74,
 	KERN_NMI_WATCHDOG=75, /* int: enable/disable nmi watchdog */
 	KERN_PANIC_ON_NMI=76, /* int: whether we will panic on an unrecovered */
+	KERN_TIMEOUT_GRANULARITY=77, /* int: timeout granularity in jiffies */
 };
 
 
include/linux/timer.h +1 -0
Index: linux-2.6.18-mm2/include/linux/timer.h
===================================================================
@@ -18,6 +18,7 @@ struct timer_list {
 };
 
 extern struct tvec_t_base_s boot_tvec_bases;
+extern unsigned int timeout_granularity;
 
 #define TIMER_INITIALIZER(_function, _expires, _data) {		\
 		.function = (_function),			\
kernel/sysctl.c +10 -0
Index: linux-2.6.18-mm2/kernel/sysctl.c
===================================================================
@@ -640,6 +640,16 @@ static ctl_table kern_table[] = {
 		.proc_handler	= &proc_dointvec,
 	},
 #endif
+#ifdef CONFIG_NO_HZ
+	{
+		.ctl_name       = KERN_TIMEOUT_GRANULARITY,
+		.procname       = "timeout_granularity",
+		.data           = &timeout_granularity,
+		.maxlen         = sizeof(int),
+		.mode           = 0644,
+		.proc_handler   = &proc_dointvec,
+	},
+#endif
 	{
 		.ctl_name	= KERN_PIDMAX,
 		.procname	= "pid_max",
kernel/timer.c +21 -3
Index: linux-2.6.18-mm2/kernel/timer.c
===================================================================
@@ -66,6 +66,8 @@ typedef struct tvec_root_s {
 	struct list_head vec[TVR_SIZE];
 } tvec_root_t;
 
+unsigned int __read_mostly timeout_granularity = 1;
+
 struct tvec_t_base_s {
 	spinlock_t lock;
 	struct timer_list *running_timer;
@@ -417,7 +419,9 @@ static inline void __run_timers(tvec_bas
 	struct timer_list *timer;
 
 	spin_lock_irq(&base->lock);
-	while (time_after_eq(jiffies, base->timer_jiffies)) {
+
+	while (time_before_eq(base->timer_jiffies, jiffies)) {
+
 		struct list_head work_list;
 		struct list_head *head = &work_list;
  		int index = base->timer_jiffies & TVR_MASK;
@@ -569,7 +573,15 @@ found:
 	 * delayed processing, so make sure we return a value that
 	 * makes sense externally:
 	 */
-	return expires - (now - base->timer_jiffies);
+	expires -= (now - base->timer_jiffies);
+
+	/*
+	 * Round it up per timeout_granularity:
+	 */
+	expires += timeout_granularity - 1;
+	expires -= expires % timeout_granularity;
+
+	return expires;
 }
 
 unsigned long get_next_timer_interrupt(unsigned long now)
@@ -1112,7 +1124,13 @@ static void run_timer_softirq(struct sof
  */
 void run_local_timers(void)
 {
-	raise_softirq(TIMER_SOFTIRQ);
+	tvec_base_t *base = per_cpu(tvec_bases, smp_processor_id());
+	/*
+	 * Only wake up the TIMER_SOFTIRQ every timeout_granularity
+	 * jiffies:
+	 */
+	if (time_before_eq(base->timer_jiffies + timeout_granularity, jiffies))
+		raise_softirq(TIMER_SOFTIRQ);
 	softlockup_tick();
 }
 

--

#45 lore raw

Re: [patch 20/23] add /proc/sys/kernel/timeout_granularity

Quoted 8 lines · On Fri, 29 Sep 2006 23:58:39 -0000 Thomas Gleixner <[email protected]> wrote:

From: Ingo Molnar <[email protected]>

Introduce timeout granularity: process timer wheel timers every timeout_granularity jiffies. Defaults to 1 (process timers HZ times per second - most finegrained).

...

Documentation/kernel-parameters.txt and is between 256 and 4096 characters. · line 1645

Quoted 6 lines
Documentation/kernel-parameters.txt +6 -0
+	timeout_granularity=
+			[KNL]
+			Timeout granularity: process timer wheel timers every
+			timeout_granularity jiffies. Defaults to 1 (process
+			timers HZ times per second - most finegrained).
+

Please do not expose HZ to userspace in this fashion. It means that an application which was developed and tested on a 1000Hz kernel might fail on a 250Hz kernel.

#46 lore raw

PATCH 21/23 debugging feature: timer stats

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

add /proc/tstats support: debugging feature to profile timer expiration. Both the starting site, process/PID and the expiration function is captured. This allows the quick identification of timer event sources in a system.

sample output:

 # echo 1 > /proc/tstats
 # cat /proc/tstats
 Timerstats sample period: 3.888770 s
   12,     0 swapper          hrtimer_stop_sched_tick (hrtimer_sched_tick)
   15,     1 swapper          hcd_submit_urb (rh_timer_func)
    4,   959 kedac            schedule_timeout (process_timeout)
    1,     0 swapper          page_writeback_init (wb_timer_fn)
   28,     0 swapper          hrtimer_stop_sched_tick (hrtimer_sched_tick)
   22,  2948 IRQ 4            tty_flip_buffer_push (delayed_work_timer_fn)
    3,  3100 bash             schedule_timeout (process_timeout)
    1,     1 swapper          queue_delayed_work_on (delayed_work_timer_fn)
    1,     1 swapper          queue_delayed_work_on (delayed_work_timer_fn)
    1,     1 swapper          neigh_table_init_no_netlink (neigh_periodic_timer)
    1,  2292 ip               __netdev_watchdog_up (dev_watchdog)
    1,    23 events/1         do_cache_clean (delayed_work_timer_fn)
 90 total events, 30.0 events/sec

--

 include/linux/hrtimer.h   |   44 ++++++++
 include/linux/timer.h     |   48 +++++++++
 kernel/hrtimer.c          |   26 +++++
 kernel/time/Makefile      |    3 
 kernel/time/timer_stats.c |  227 ++++++++++++++++++++++++++++++++++++++++++++++
 kernel/timer.c            |   29 +++++
 kernel/workqueue.c        |    8 +
 lib/Kconfig.debug         |   11 ++
 8 files changed, 391 insertions(+), 5 deletions(-)
include/linux/hrtimer.h +44 -0
Index: linux-2.6.18-mm2/include/linux/hrtimer.h
===================================================================
@@ -74,6 +74,11 @@ struct hrtimer {
 	int				cb_mode;
 	struct list_head		cb_entry;
 #endif
+#ifdef CONFIG_TIMER_STATS
+	void				*start_site;
+	char				start_comm[16];
+	int				start_pid;
+#endif
 };
 
 /**
@@ -258,4 +263,43 @@ static inline int hrtimer_stop_sched_tic
 /* Bootup initialization: */
 extern void __init hrtimers_init(void);
 
+/*
+ * Timer-statistics info:
+ */
+#ifdef CONFIG_TIMER_STATS
+
+extern void tstats_update_stats(void *timer, pid_t pid, void *startf,
+				void *timerf, char * comm);
+
+static inline void tstats_account_hrtimer(struct hrtimer *timer)
+{
+	tstats_update_stats(timer, timer->start_pid, timer->start_site,
+			    timer->function, timer->start_comm);
+}
+
+extern void __tstats_hrtimer_set_start_info(struct hrtimer *timer, void *addr);
+
+static inline void tstats_hrtimer_set_start_info(struct hrtimer *timer)
+{
+	__tstats_hrtimer_set_start_info(timer, __builtin_return_address(0));
+}
+
+static inline void tstats_hrtimer_clear_start_info(struct hrtimer *timer)
+{
+	timer->start_site = NULL;
+}
+#else
+static inline void tstats_account_hrtimer(struct hrtimer *timer)
+{
+}
+
+static inline void tstats_hrtimer_set_start_info(struct hrtimer *timer)
+{
+}
+
+static inline void tstats_hrtimer_clear_start_info(struct hrtimer *timer)
+{
+}
+#endif
+
 #endif
include/linux/timer.h +48 -0

66 lines not on this page · raw message

kernel/hrtimer.c +26 -0
Index: linux-2.6.18-mm2/kernel/hrtimer.c
===================================================================
@@ -814,6 +814,18 @@ static inline void hrtimer_resume_jiffie
 
 #endif /* CONFIG_HIGH_RES_TIMERS */
 
+#ifdef CONFIG_TIMER_STATS
+void __tstats_hrtimer_set_start_info(struct hrtimer *timer, void *addr)
+{
+	if (timer->start_site)
+		return;
+
+	timer->start_site = addr;
+	memcpy(timer->start_comm, current->comm, TASK_COMM_LEN);
+	timer->start_pid = current->pid;
+}
+#endif
+
 /*
  * Timekeeping resumed notification
  */
@@ -959,6 +971,7 @@ remove_hrtimer(struct hrtimer *timer, st
 		 * Remove the timer and force reprogramming when high
 		 * resolution mode is active
 		 */
+		tstats_hrtimer_clear_start_info(timer);
 		__remove_hrtimer(timer, base, HRTIMER_INACTIVE, 1);
 		return 1;
 	}
@@ -1005,6 +1018,8 @@ hrtimer_start(struct hrtimer *timer, kti
 	}
 	timer->expires = tim;
 
+	tstats_hrtimer_set_start_info(timer);
+
 	enqueue_hrtimer(timer, new_base);
 
 	unlock_hrtimer_base(timer, &flags);
@@ -1146,6 +1161,12 @@ void hrtimer_init(struct hrtimer *timer,
 
 	timer->base = &cpu_base->clock_base[clock_id];
 	hrtimer_init_timer_hres(timer);
+
+#ifdef CONFIG_TIMER_STATS
+	timer->start_site = NULL;
+	timer->start_pid = -1;
+	memset(timer->start_comm, 0, TASK_COMM_LEN);
+#endif
 }
 EXPORT_SYMBOL_GPL(hrtimer_init);
 
@@ -1229,6 +1250,7 @@ void hrtimer_interrupt(struct pt_regs *r
 			}
 
 			__remove_hrtimer(timer, base, HRTIMER_CALLBACK, 0);
+			tstats_account_hrtimer(timer);
 
 			if (timer->function(timer) != HRTIMER_NORESTART) {
 				BUG_ON(timer->state != HRTIMER_CALLBACK);
@@ -1277,6 +1299,8 @@ static void run_hrtimer_softirq(struct s
 		timer = list_entry(cpu_base->cb_pending.next,
 				   struct hrtimer, cb_entry);
 
+		tstats_account_hrtimer(timer);
+
 		fn = timer->function;
 		__remove_hrtimer(timer, timer->base, HRTIMER_CALLBACK, 0);
 		spin_unlock_irq(&cpu_base->lock);
@@ -1326,6 +1350,8 @@ static inline void run_hrtimer_queue(str
 		if (base->softirq_time.tv64 <= timer->expires.tv64)
 			break;
 
+		tstats_account_hrtimer(timer);
+
 		fn = timer->function;
 		__remove_hrtimer(timer, base, HRTIMER_CALLBACK, 0);
 		spin_unlock_irq(&cpu_base->lock);
kernel/time/Makefile +2 -1
Index: linux-2.6.18-mm2/kernel/time/Makefile
===================================================================
@@ -1,3 +1,4 @@
 obj-y += ntp.o clocksource.o jiffies.o
 
-obj-$(CONFIG_GENERIC_TIME) += clockevents.o
+obj-$(CONFIG_GENERIC_TIME)	+= clockevents.o
+obj-$(CONFIG_TIMER_STATS)	+= timer_stats.o
kernel/time/timer_stats.c new file +227 -0

227 lines not on this page · raw message

kernel/timer.c +27 -2
Index: linux-2.6.18-mm2/kernel/timer.c
===================================================================
@@ -34,6 +34,7 @@
 #include <linux/cpu.h>
 #include <linux/syscalls.h>
 #include <linux/delay.h>
+#include <linux/kallsyms.h>
 
 #include <asm/uaccess.h>
 #include <asm/unistd.h>
@@ -135,6 +136,18 @@ static void internal_add_timer(tvec_base
 	list_add_tail(&timer->entry, vec);
 }
 
+#ifdef CONFIG_TIMER_STATS
+void __tstats_timer_set_start_info(struct timer_list *timer, void *addr)
+{
+	if (timer->start_site)
+		return;
+
+	timer->start_site = addr;
+	memcpy(timer->start_comm, current->comm, TASK_COMM_LEN);
+	timer->start_pid = current->pid;
+}
+#endif
+
 /**
  * init_timer - initialize a timer.
  * @timer: the timer to be initialized
@@ -146,11 +159,16 @@ void fastcall init_timer(struct timer_li
 {
 	timer->entry.next = NULL;
 	timer->base = __raw_get_cpu_var(tvec_bases);
+#ifdef CONFIG_TIMER_STATS
+	timer->start_site = NULL;
+	timer->start_pid = -1;
+	memset(timer->start_comm, 0, TASK_COMM_LEN);
+#endif
 }
 EXPORT_SYMBOL(init_timer);
 
 static inline void detach_timer(struct timer_list *timer,
-					int clear_pending)
+				int clear_pending)
 {
 	struct list_head *entry = &timer->entry;
 
@@ -197,6 +215,7 @@ int __mod_timer(struct timer_list *timer
 	unsigned long flags;
 	int ret = 0;
 
+	tstats_timer_set_start_info(timer);
 	BUG_ON(!timer->function);
 
 	base = lock_timer_base(timer, &flags);
@@ -247,6 +266,7 @@ void add_timer_on(struct timer_list *tim
 	tvec_base_t *base = per_cpu(tvec_bases, cpu);
   	unsigned long flags;
 
+	tstats_timer_set_start_info(timer);
   	BUG_ON(timer_pending(timer) || !timer->function);
 	spin_lock_irqsave(&base->lock, flags);
 	timer->base = base;
@@ -279,6 +299,7 @@ int mod_timer(struct timer_list *timer, 
 {
 	BUG_ON(!timer->function);
 
+	tstats_timer_set_start_info(timer);
 	/*
 	 * This is a common optimization triggered by the
 	 * networking code - if the timer is re-modified
@@ -309,6 +330,7 @@ int del_timer(struct timer_list *timer)
 	unsigned long flags;
 	int ret = 0;
 
+	tstats_timer_clear_start_info(timer);
 	if (timer_pending(timer)) {
 		base = lock_timer_base(timer, &flags);
 		if (timer_pending(timer)) {
@@ -444,6 +466,8 @@ static inline void __run_timers(tvec_bas
  			fn = timer->function;
  			data = timer->data;
 
+			tstats_account_timer(timer);
+
 			set_running_timer(base, timer);
 			detach_timer(timer, 1);
 			spin_unlock_irq(&base->lock);
@@ -1114,7 +1138,8 @@ static void run_timer_softirq(struct sof
 {
 	tvec_base_t *base = __get_cpu_var(tvec_bases);
 
- 	hrtimer_run_queues();
+	hrtimer_run_queues();
+
 	if (time_after_eq(jiffies, base->timer_jiffies))
 		__run_timers(base);
 }
kernel/workqueue.c +6 -2
Index: linux-2.6.18-mm2/kernel/workqueue.c
===================================================================
@@ -119,12 +119,14 @@ int fastcall queue_work(struct workqueue
 }
 EXPORT_SYMBOL_GPL(queue_work);
 
-static void delayed_work_timer_fn(unsigned long __data)
+void delayed_work_timer_fn(unsigned long __data)
 {
 	struct work_struct *work = (struct work_struct *)__data;
 	struct workqueue_struct *wq = work->wq_data;
 	int cpu = smp_processor_id();
+	struct list_head *head;
 
+	head = &per_cpu_ptr(wq->cpu_wq, cpu)->more_work.task_list;
 	if (unlikely(is_single_threaded(wq)))
 		cpu = singlethread_cpu;
 
@@ -140,11 +142,12 @@ static void delayed_work_timer_fn(unsign
  * Returns non-zero if it was successfully added.
  */
 int fastcall queue_delayed_work(struct workqueue_struct *wq,
-			struct work_struct *work, unsigned long delay)
+				struct work_struct *work, unsigned long delay)
 {
 	int ret = 0;
 	struct timer_list *timer = &work->timer;
 
+	tstats_timer_set_start_info(&work->timer);
 	if (!test_and_set_bit(0, &work->pending)) {
 		BUG_ON(timer_pending(timer));
 		BUG_ON(!list_empty(&work->entry));
@@ -469,6 +472,7 @@ EXPORT_SYMBOL(schedule_work);
  */
 int fastcall schedule_delayed_work(struct work_struct *work, unsigned long delay)
 {
+	tstats_timer_set_start_info(&work->timer);
 	return queue_delayed_work(keventd_wq, work, delay);
 }
 EXPORT_SYMBOL(schedule_delayed_work);
lib/Kconfig.debug +11 -0
Index: linux-2.6.18-mm2/lib/Kconfig.debug
===================================================================
@@ -109,6 +109,17 @@ config SCHEDSTATS
 	  application, you can say N to avoid the very slight overhead
 	  this adds.
 
+config TIMER_STATS
+	bool "Collect kernel timers statistics"
+	depends on DEBUG_KERNEL && PROC_FS
+	help
+	  If you say Y here, additional code will be inserted into the
+	  timer routines to collect statistics about kernel timers being
+	  reprogrammed. The statistics can be read from /proc/tstats.
+	  The statistics collection is started by writing 1 to /proc/tstats,
+	  writing 0 stops it. This feature is useful to collect information
+	  about timer usage patterns in kernel and userspace.
+
 config DEBUG_SLAB
 	bool "Debug slab memory allocations"
 	depends on DEBUG_KERNEL && SLAB

--

#47 lore raw

Re: [patch 21/23] debugging feature: timer stats

Quoted 3 lines · On Fri, 29 Sep 2006 23:58:41 -0000 Thomas Gleixner <[email protected]> wrote:

From: Thomas Gleixner <[email protected]>

add /proc/tstats support:

/proc/timer_stats would be a better name.

Is this intended as a mainlineable feature? If so, some documentation would be nice.

Quoted 27 lines

debugging feature to profile timer expiration.
Both the starting site, process/PID and the expiration function is captured. This allows the quick identification of timer event sources in a system.

sample output:

 # echo 1 > /proc/tstats
 # cat /proc/tstats
 Timerstats sample period: 3.888770 s
   12,     0 swapper          hrtimer_stop_sched_tick (hrtimer_sched_tick)
   15,     1 swapper          hcd_submit_urb (rh_timer_func)
    4,   959 kedac            schedule_timeout (process_timeout)
    1,     0 swapper          page_writeback_init (wb_timer_fn)
   28,     0 swapper          hrtimer_stop_sched_tick (hrtimer_sched_tick)
   22,  2948 IRQ 4            tty_flip_buffer_push (delayed_work_timer_fn)
    3,  3100 bash             schedule_timeout (process_timeout)
    1,     1 swapper          queue_delayed_work_on (delayed_work_timer_fn)
    1,     1 swapper          queue_delayed_work_on (delayed_work_timer_fn)
    1,     1 swapper          neigh_table_init_no_netlink (neigh_periodic_timer)
    1,  2292 ip               __netdev_watchdog_up (dev_watchdog)
    1,    23 events/1         do_cache_clean (delayed_work_timer_fn)
 90 total events, 30.0 events/sec

...

include/linux/hrtimer.h struct hrtimer { · line 82

Quoted 9 lines
include/linux/hrtimer.h +5 -0
@@ -74,6 +74,11 @@ struct hrtimer {
 	int				cb_mode;
 	struct list_head		cb_entry;
 #endif
+#ifdef CONFIG_TIMER_STATS
+	void				*start_site;
+	char				start_comm[16];
+	int				start_pid;
+#endif
 };

Forgot to update this struct's kerneldoc.

Review struct hrtimer { · line 83

Quoted 1 line
(quoted diff) +1 -0
+extern void tstats_update_stats(void *timer, pid_t pid, void *startf,

timer_stats_* would be nicer...

Quoted 13 lines

Index: linux-2.6.18-mm2/include/linux/timer.h
===================================================================

include/linux/timer.h +1 -0
@@ -2,6 +2,7 @@
 #define _LINUX_TIMER_H
 
 #include <linux/list.h>
+#include <linux/ktime.h>
 #include <linux/spinlock.h>
 #include <linux/stddef.h>
 

include/linux/timer.h struct timer_list { · line 24

Quoted 9 lines
include/linux/timer.h +5 -0
@@ -15,6 +16,11 @@ struct timer_list {
 	unsigned long data;
 
 	struct tvec_t_base_s *base;
+#ifdef CONFIG_TIMER_STATS
+	void *start_site;
+	char start_comm[16];
+	int start_pid;
+#endif
 };

hm. So it's very much a developer thing.

kernel/time/timer_stats.c line 9

Quoted 13 lines

===================================================================

kernel/time/timer_stats.c new file +9 -0
@@ -0,0 +1,227 @@
+/*
+ * kernel/time/timer_stats.c
+ *
+ * Copyright(C) 2006, Red Hat, Inc., Ingo Molnar
+ * Copyright(C) 2006, Thomas Gleixner <[email protected]>
+ *
+ * Based on: timer_top.c
+ *	Copyright (C) 2005 Instituto Nokia de Tecnologia - INdT - Manaus
+ *	Written by Daniel Petrini <[email protected]>

We're missing a Signed-off-by:?

Quoted 18 lines
kernel/time/timer_stats.c new file +18 -0
+ * Collect timer usage statistics.
+ *
+ * We export the addresses and counting of timer functions being called,
+ * the pid and cmdline from the owner process if applicable.
+ *
+ * Start/stop data collection:
+ * # echo 1[0] >/proc/tstats
+ *
+ * Display the collected information:
+ * # cat /proc/tstats
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/list.h>
+#include <linux/proc_fs.h>

kernel/time/timer_stats.c line 38

Quoted 10 lines
kernel/time/timer_stats.c new file +10 -0
+#include <linux/module.h>
+#include <linux/spinlock.h>
+#include <linux/sched.h>
+#include <linux/seq_file.h>
+#include <linux/kallsyms.h>
+
+#include <asm/uaccess.h>
+
+static DEFINE_SPINLOCK(tstats_lock);
+static int tstats_status;

This is not an integer. It has type `enum tstats_stat'.

Quoted 6 lines
kernel/time/timer_stats.c new file +6 -0
+static ktime_t tstats_time;
+
+enum tstats_stat {
+	TSTATS_INACTIVE,
+	TSTATS_ACTIVE,
+	TSTATS_READOUT,

kernel/time/timer_stats.c line 100

Quoted 10 lines
kernel/time/timer_stats.c new file +7 -0
+	TSTATS_RESET,
+};
+
 
...
 
+static void tstats_reset(void)
+{
+	memset(tstats, 0, sizeof(tstats));
+}

This is called without the lock held, which looks wrong.

kernel/time/timer_stats.c line 171

Quoted 7 lines
kernel/time/timer_stats.c new file +7 -0
+static ssize_t tstats_write(struct file *file, const char __user *buf,
+			    size_t count, loff_t *offs)
+{
+	char ctl[2];
+
+	if (count != 2 || *offs)
+		return -EINVAL;

Let's hope nobody's echo command does while(n) write(fd, *p++, 1);

kernel/time/timer_stats.c

Quoted 9 lines
kernel/time/timer_stats.c new file +3 -1
-static void delayed_work_timer_fn(unsigned long __data)
+void delayed_work_timer_fn(unsigned long __data)
 {
 	struct work_struct *work = (struct work_struct *)__data;
 	struct workqueue_struct *wq = work->wq_data;
 	int cpu = smp_processor_id();
+	struct list_head *head;
 
+	head = &per_cpu_ptr(wq->cpu_wq, cpu)->more_work.task_list;

This doesn't do anything.

#48 lore raw

PATCH 22/23 dynticks: increase SLAB timeouts

In reply to Thomas Gleixner

From: Ingo Molnar <[email protected]>

decrease the rate of SLAB timers going off. Reduces the amount of timers going off in an idle system.

--

 mm/slab.c |    9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
mm/slab.c +7 -2
Index: linux-2.6.18-mm2/mm/slab.c
===================================================================
@@ -457,8 +457,13 @@ struct kmem_cache {
  * OTOH the cpuarrays can contain lots of objects,
  * which could lock up otherwise freeable slabs.
  */
-#define REAPTIMEOUT_CPUC	(2*HZ)
-#define REAPTIMEOUT_LIST3	(4*HZ)
+#ifdef CONFIG_NO_HZ
+# define REAPTIMEOUT_CPUC	(4*HZ)
+# define REAPTIMEOUT_LIST3	(8*HZ)
+#else
+# define REAPTIMEOUT_CPUC	(2*HZ)
+# define REAPTIMEOUT_LIST3	(4*HZ)
+#endif
 
 #if STATS
 #define	STATS_INC_ACTIVE(x)	((x)->num_active++)

--

#49 lore raw

Re: [patch 22/23] dynticks: increase SLAB timeouts

Quoted 22 lines · On Fri, 29 Sep 2006 23:58:42 -0000 Thomas Gleixner <[email protected]> wrote:

From: Ingo Molnar <[email protected]>

decrease the rate of SLAB timers going off. Reduces the amount of timers going off in an idle system.

--

 mm/slab.c |    9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

Index: linux-2.6.18-mm2/mm/slab.c
===================================================================

mm/slab.c +0 -2
@@ -457,8 +457,13 @@ struct kmem_cache {
  * OTOH the cpuarrays can contain lots of objects,
  * which could lock up otherwise freeable slabs.
  */
-#define REAPTIMEOUT_CPUC	(2*HZ)
-#define REAPTIMEOUT_LIST3	(4*HZ)

mm/slab.c struct kmem_cache { · line 469

Quoted 10 lines
mm/slab.c +7 -0
+#ifdef CONFIG_NO_HZ
+# define REAPTIMEOUT_CPUC	(4*HZ)
+# define REAPTIMEOUT_LIST3	(8*HZ)
+#else
+# define REAPTIMEOUT_CPUC	(2*HZ)
+# define REAPTIMEOUT_LIST3	(4*HZ)
+#endif
 
 #if STATS
 #define	STATS_INC_ACTIVE(x)	((x)->num_active++)

err, no.

a) We shouldn't go and assume that "No Hz" implies "I want the CPU to remain idle for long periods".

It's a good assumption, but that is an *application* of NO_HZ and the above should be a separate configuration option. Or, better, runtime configurable.

b) This reap timeout is there for a reason. We shouldn't just go and modify memory management behaviour because someone selected NO_HZ.

Again, a runtime tunable is preferable.

Then again, two seconds is quite a long time, surely? And increasing it to just four seconds hardly seems worth the effort.

Still, the code you're patching is pretty lame anyway. It shouldn't be using time. Time is meaningless in the mm context. I'm not sure what it _should_ be using though. Perhaps every-Nth-kmem_cache_alloc or something.

It's trying to measure "is this memory I'm holding likely to be in the CPU's cache any more". So perhaps time is a close-enough basis.

#50 lore raw

PATCH 23/23 dynticks: decrease I8042_POLL_PERIOD

In reply to Thomas Gleixner

From: Ingo Molnar <[email protected]>

decrease the rate of timers going off. Also work around apparent kbd-init bug by making the first timeout short.

--

 drivers/input/serio/i8042.c |    2 +-
 drivers/input/serio/i8042.h |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
drivers/input/serio/i8042.c +1 -1
Index: linux-2.6.18-mm2/drivers/input/serio/i8042.c
===================================================================
@@ -1101,7 +1101,7 @@ static int __devinit i8042_probe(struct 
 		goto err_controller_cleanup;
 	}
 
-	mod_timer(&i8042_timer, jiffies + I8042_POLL_PERIOD);
+	mod_timer(&i8042_timer, jiffies + 2); //I8042_POLL_PERIOD);
 	return 0;
 
  err_unregister_ports:
drivers/input/serio/i8042.h +1 -1
Index: linux-2.6.18-mm2/drivers/input/serio/i8042.h
===================================================================
@@ -43,7 +43,7 @@
  * polling.
  */
 
-#define I8042_POLL_PERIOD	HZ/20
+#define I8042_POLL_PERIOD	(10*HZ)
 
 /*
  * Status register bits.

--

#51 lore raw

Re: [patch 23/23] dynticks: decrease I8042_POLL_PERIOD

Quoted 5 lines · On Fri, 29 Sep 2006 23:58:43 -0000 Thomas Gleixner <[email protected]> wrote:

From: Ingo Molnar <[email protected]>

decrease the rate of timers going off. Also work around apparent kbd-init bug by making the first timeout short.

Again, please don't make unrelated kernel functions behave differently like this.

Quoted 19 lines

Index: linux-2.6.18-mm2/drivers/input/serio/i8042.c
===================================================================

drivers/input/serio/i8042.c +1 -1
@@ -1101,7 +1101,7 @@ static int __devinit i8042_probe(struct 
 		goto err_controller_cleanup;
 	}
 
-	mod_timer(&i8042_timer, jiffies + I8042_POLL_PERIOD);
+	mod_timer(&i8042_timer, jiffies + 2); //I8042_POLL_PERIOD);
 	return 0;
 
  err_unregister_ports:

Index: linux-2.6.18-mm2/drivers/input/serio/i8042.h
===================================================================

drivers/input/serio/i8042.h line 46

Quoted 5 lines
drivers/input/serio/i8042.h +1 -1
@@ -43,7 +43,7 @@
  * polling.
  */
 
-#define I8042_POLL_PERIOD	HZ/20
+#define I8042_POLL_PERIOD	(10*HZ)

That's a huge change. Perhaps the interval was too short in the first case. I guess waiting ten seconds for your keyboard or mouse to come to life after hot-add is liveable with.

But whatever. This timer gets deleted in Dmitry's current development tree:

commit de9ce703c6b807b1dfef5942df4f2fadd0fdb67a

Author: Dmitry Torokhov <[email protected]>
Date:   Sun Sep 10 21:57:21 2006 -0400

Input: i8042 - get rid of polling timer

Remove polling timer that was used to detect keybord/mice hotplug and register both IRQs right away instead of waiting for a driver to attach to a port.

so problem solved.

#52 lore raw
In reply to Thomas Gleixner
Quoted 3 lines · On Fri, 29 Sep 2006 23:58:18 -0000 Thomas Gleixner <[email protected]> wrote:

We are pleased to announce the next version of our "high resolution timers" and "dynticks" patchset, which implements two important features that Linux lacked for many years.

Could we please have a full description of these features? All we have at present is "high resolution timers" and "dynticks", which is ludicrously terse. Please also describe the design and implementation. This is basic stuff for a run-of-the-mill patch, let alone a feature like this one.

I don't believe I can adequately review this work without that information. I can try, but obviously such a review will not be as beneficial - it can only cover trivial matters.

With all the work which has gone into this, and with the impact which it will have upon us all it is totally disproportionate that no more than a few minutes were spent telling the rest of us what it does and how it does it.

We've had a lot of problems with timekeeping in recent years, and they have been hard and slow to solve. Hence I'd like to set the bar very high on the maintainability and understandability of this work. And what I see here doesn't look really great from that POV.

Thanks.

#53 lore raw
Quoted 14 lines · Andrew Morton

Could we please have a full description of these features? All we have at present is "high resolution timers" and "dynticks", which is ludicrously terse. Please also describe the design and implementation. This is basic stuff for a run-of-the-mill patch, let alone a feature like this one.

I don't believe I can adequately review this work without that information. I can try, but obviously such a review will not be as beneficial - it can only cover trivial matters.

With all the work which has gone into this, and with the impact which it will have upon us all it is totally disproportionate that no more than a few minutes were spent telling the rest of us what it does and how it does it.

Quoted 4 lines

We've had a lot of problems with timekeeping in recent years, and they have been hard and slow to solve. Hence I'd like to set the bar very high on the maintainability and understandability of this work. And what I see here doesn't look really great from that POV.

Fair enough. Point taken. We're working on it and on the comments you gave. Thanks for taking the time to go through it nevertheless.

The OLS proceedings
http://www.linuxsymposium.org/2006/linuxsymposium_procv1.pdf

and the slides of my talk
http://tglx.de/projects/hrtimers/ols2006-hrtimers.pdf

might also shed some light on the design.

tglx

#54 lore raw
In reply to Thomas Gleixner

General readability comment. This:

enum hrtimer_mode {
HRTIMER_ABS, /* Time value is absolute */
HRTIMER_REL, /* Time value is relative to now */
};

enum hrtimer_restart {
HRTIMER_NORESTART,
HRTIMER_RESTART,
};

#define HRTIMER_INACTIVE 0x00
#define HRTIMER_ACTIVE 0x01
#define HRTIMER_CALLBACK 0x02
#define HRTIMER_PENDING 0x04

is quite bad. They enumerate different conceptual things but they are all in the sane namespace. The reader sees code which is using a mixture of HRTIMER_ABS, HRTIMER_RESTART and HRTIMER_ACTIVE and gets needlessly confused over what they all signify.

This:

enum hrtimer_cb_mode {
HRTIMER_CB_SOFTIRQ,
HRTIMER_CB_IRQSAFE,
HRTIMER_CB_IRQSAFE_NO_RESTART,
HRTIMER_CB_IRQSAFE_NO_SOFTIRQ,
};

is better.