final clockevents/hpet fixes
cover final clockevents/hpet fixes
This is the final version of the various clockevents fixes, which I will send to Linus. Can you please give it a try on your machines, so we can be sure that I did not mess up anything again.
Thanks for your patience and help
tglx
PATCH 1/6 clockevents: Prevent clockevent event_handler ending up handler_noop
There is a ordering related problem with clockevents code, due to which clockevents_register_device() called after tickless/highres switch will not work. The new clockevent ends up with clockevents_handle_noop as event handler, resulting in no timer activity.
The problematic path seems to be
* old device already has hrtimer_interrupt as the event_handler
* new clockevent device registers with a higher rating
* tick_check_new_device() is called
* clockevents_exchange_device() gets called
* old->event_handler is set to clockevents_handle_noop
* tick_setup_device() is called for the new device
* which sets new->event_handler using the old->event_handler which is noop.
Change the ordering so that new device inherits the proper handler.
This does not have any issue in normal case as most likely all the clockevent devices are setup before the highres switch. But, can potentially be affecting some corner case where HPET force detect happens after the highres switch. This was a problem with HPET in MSI mode code that we have been experimenting with.
--- include/linux/clockchips.h | 2 ++ kernel/time/clockevents.c | 3 +-- kernel/time/tick-common.c | 1 + 3 files changed, 4 insertions(+), 2 deletions(-)
include/linux/clockchips.h +2 -0
@@ -127,6 +127,8 @@ extern int clockevents_register_notifierextern int clockevents_program_event (struct clock_event_device *dev,ktime_t expires,ktime_t now); +extern void clockevents_handle_noop (struct clock_event_device *dev); +#ifdef CONFIG_GENERIC_CLOCKEVENTS extern void clockevents_notify (unsigned long reason,void *arg);#else
kernel/time/clockevents.c +1 -2
@@ -177,7 +177,7 @@ void clockevents_register_device(struct/* * Noop handler when we shut down an event device */ -static void clockevents_handle_noop (struct clock_event_device *dev) +void clockevents_handle_noop (struct clock_event_device *dev) { }
@@ -199,7 +199,6 @@ void clockevents_exchange_device(struct* released list and do a notify add later. */ if (old) { - old->event_handler = clockevents_handle_noop;clockevents_set_mode (old,CLOCK_EVT_MODE_UNUSED );list_del (&old->list);list_add (&old->list, &clockevents_released);
kernel/time/tick-common.c +1 -0
@@ -161,6 +161,7 @@ static void tick_setup_device(struct tic }else { handler = td->evtdev->event_handler; next_event = td->evtdev->next_event; + td->evtdev->event_handler = clockevents_handle_noop; } td->evtdev = newdev;
PATCH 2/6 clockevents: prevent endless loop in periodic broadcast handler
In reply to Thomas GleixnerThe reprogramming of the periodic broadcast handler was broken, when the first programming returned -ETIME. The clockevents code stores the new expiry value in the clock events device next_event field only when the programming time has not been elapsed yet. The loop in question calculates the new expiry value from the next_event value and therefor never increases.
--- kernel/time/tick-broadcast.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
kernel/time/tick-broadcast.c +8 -3
@@ -175,6 +175,8 @@ static void tick_do_periodic_broadcast(v*/ static void tick_handle_periodic_broadcast (struct clock_event_device *dev) { +ktime_t next; +tick_do_periodic_broadcast ();/*
@@ -185,10 +187,13 @@ static void tick_handle_periodic_broadca/* * Setup the next period for devices, which do not have -* periodic mode: +* periodic mode. We read dev->next_event first and add to it +* when the event alrady expired. clockevents_program_event() +* sets dev->next_event only when the event is really +* programmed to the device. */ -for (;;) { -ktime_t next =ktime_add (dev->next_event, tick_period); +for (next = dev->next_event; ;) { + next =ktime_add (next, tick_period);if (!clockevents_program_event (dev, next,ktime_get ()))return ;
PATCH 3/6 clockevents: enforce reprogram in oneshot setup
In reply to Thomas GleixnerIn tick_oneshot_setup we program the device to the given next_event, but we do not check the return value. We need to make sure that the device is programmed enforced so the interrupt handler engine starts working. Split out the reprogramming function from tick_program_event() and call it with the device, which was handed in to tick_setup_oneshot(). Set the force argument, so the devices is firing an interrupt.
--- kernel/time/tick-oneshot.c | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-)
kernel/time/tick-oneshot.c +14 -4
@@ -23,11 +23,11 @@#include "tick-internal.h" /** -* tick_program_event +* tick_program_event internal worker function */ -int tick_program_event (ktime_t expires,int force) +static int __tick_program_event (struct clock_event_device *dev, +ktime_t expires,int force) { -struct clock_event_device *dev =__get_cpu_var (tick_cpu_device).evtdev;ktime_t now =ktime_get ();while (1 ) {
@@ -41,6 +41,16 @@ int tick_program_event(ktime_t expires, }/** +* tick_program_event +*/ +int tick_program_event (ktime_t expires,int force) +{ +struct clock_event_device *dev =__get_cpu_var (tick_cpu_device).evtdev; + +return __tick_program_event (dev, expires, force); +} + +/** * tick_resume_onshot - resume oneshot mode */ void tick_resume_oneshot (void )
@@ -61,7 +71,7 @@ void tick_setup_oneshot(struct clock_eve { newdev->event_handler = handler;clockevents_set_mode (newdev,CLOCK_EVT_MODE_ONESHOT ); -clockevents_program_event (newdev, next_event,ktime_get ()); +__tick_program_event (newdev, next_event,1 ); }/**
PATCH 4/6 clockevents: prevent multiple init/shutdown
In reply to Thomas GleixnerWhile chasing the C1E/HPET bugreports I went through the clock events code inch by inch and found that the broadcast device can be initialized and shutdown multiple times. Multiple shutdowns are not critical, but useless waste of time. Multiple initializations are simply broken. Another CPU might have the device in use already after the first initialization and the second init could just render it unusable again.
--- kernel/time/tick-broadcast.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-)
kernel/time/tick-broadcast.c +13 -7
@@ -210,7 +210,7 @@ static void tick_do_broadcast_on_off(voistruct clock_event_device *bc, *dev;struct tick_device *td;unsigned long flags, *reason = why; -int cpu; +int cpu, bc_stopped;spin_lock_irqsave (&tick_broadcast_lock, flags);
@@ -228,6 +228,8 @@ static void tick_do_broadcast_on_off(voiif (!tick_device_is_functional (dev)) goto out; + bc_stopped =cpus_empty (tick_broadcast_mask); +switch (*reason) {case CLOCK_EVT_NOTIFY_BROADCAST_ON :case CLOCK_EVT_NOTIFY_BROADCAST_FORCE :
@@ -250,9 +252,10 @@ static void tick_do_broadcast_on_off(voibreak ; } -if (cpus_empty (tick_broadcast_mask)) -clockevents_set_mode (bc,CLOCK_EVT_MODE_SHUTDOWN ); -else { +if (cpus_empty (tick_broadcast_mask)) { +if (!bc_stopped) +clockevents_set_mode (bc,CLOCK_EVT_MODE_SHUTDOWN ); + }else if (bc_stopped) {if (tick_broadcast_device.mode ==TICKDEV_MODE_PERIODIC )tick_broadcast_start_periodic (bc);else
@@ -501,9 +504,12 @@ static void tick_broadcast_clear_oneshot*/ void tick_broadcast_setup_oneshot (struct clock_event_device *bc) { - bc->event_handler = tick_handle_oneshot_broadcast; -clockevents_set_mode (bc,CLOCK_EVT_MODE_ONESHOT ); - bc->next_event.tv64 =KTIME_MAX ; +/* Set it up only once ! */ +if (bc->event_handler != tick_handle_oneshot_broadcast) { + bc->event_handler = tick_handle_oneshot_broadcast; +clockevents_set_mode (bc,CLOCK_EVT_MODE_ONESHOT ); + bc->next_event.tv64 =KTIME_MAX ; + } }/*
PATCH 5/6 clockevents: prevent endless loop lockup
In reply to Thomas GleixnerThe C1E/HPET bug reports on AMDX2/RS690 systems where tracked down to a too small value of the HPET minumum delta for programming an event.
The clockevents code needs to enforce an interrupt event on the clock event device in some cases. The enforcement code was stupid and naive, as it just added the minimum delta to the current time and tried to reprogram the device. When the minimum delta is too small, then this loops forever.
Add a sanity check. Allow reprogramming to fail 3 times, then print a warning and double the minimum delta value to make sure, that this does not happen again. Use the same function for both tick-oneshot and tick-broadcast code.
--- kernel/time/tick-broadcast.c | 10 +--------- kernel/time/tick-internal.h | 2 ++ kernel/time/tick-oneshot.c | 36 ++++++++++++++++++++++++++++++------ 3 files changed, 33 insertions(+), 15 deletions(-)
kernel/time/tick-broadcast.c +1 -9
@@ -372,16 +372,8 @@ cpumask_t *tick_get_broadcast_oneshot_mastatic int tick_broadcast_set_event (ktime_t expires,int force) {struct clock_event_device *bc = tick_broadcast_device.evtdev; -ktime_t now =ktime_get (); -int res; -for (;;) { - res =clockevents_program_event (bc, expires, now); -if (!res || !force) -return res; - now =ktime_get (); - expires =ktime_add (now,ktime_set (0 , bc->min_delta_ns)); - } +return tick_dev_program_event (bc, expires, force); }int tick_resume_broadcast_oneshot (struct clock_event_device *bc)
kernel/time/tick-internal.h +2 -0
@@ -17,6 +17,8 @@ extern void tick_handle_periodic(structextern void tick_setup_oneshot (struct clock_event_device *newdev,void (*handler)(struct clock_event_device *),ktime_t nextevt); +extern int tick_dev_program_event (struct clock_event_device *dev, +ktime_t expires,int force);extern int tick_program_event (ktime_t expires,int force);extern void tick_oneshot_notify (void );extern int tick_switch_to_oneshot (void (*handler)(struct clock_event_device *));
kernel/time/tick-oneshot.c +30 -6
@@ -25,18 +25,42 @@/** * tick_program_event internal worker function */ -static int __tick_program_event (struct clock_event_device *dev, -ktime_t expires,int force) +int tick_dev_program_event (struct clock_event_device *dev,ktime_t expires, +int force) {ktime_t now =ktime_get (); +int i; -while (1 ) { +for (i =0 ;;) {int ret =clockevents_program_event (dev, expires, now);if (!ret || !force)return ret; + +/* +* We tried 2 times to program the device with the given +* min_delta_ns. If that's not working then we double it +* and emit a warning. +*/ +if (++i >2 ) { +printk (KERN_WARNING "CE: __tick_program_event of %s is " +"stuck %llx %llx\n" , dev->name ? dev->name :"?" , + now.tv64, expires.tv64); +printk (KERN_WARNING +"CE: increasing min_delta_ns %ld to %ld nsec\n" , + dev->min_delta_ns, dev->min_delta_ns <<1 ); +WARN_ON (1 ); + +/* Double the min. delta and try again */ +if (!dev->min_delta_ns) + dev->min_delta_ns =5000 ; +else + dev->min_delta_ns <<=1 ; + i =0 ; + } + now =ktime_get (); - expires =ktime_add (now,ktime_set (0 , dev->min_delta_ns)); + expires =ktime_add_ns (now, dev->min_delta_ns); } }
@@ -47,7 +71,7 @@ int tick_program_event(ktime_t expires, {struct clock_event_device *dev =__get_cpu_var (tick_cpu_device).evtdev; -return __tick_program_event (dev, expires, force); +return tick_dev_program_event (dev, expires, force); }/**
@@ -71,7 +95,7 @@ void tick_setup_oneshot(struct clock_eve { newdev->event_handler = handler;clockevents_set_mode (newdev,CLOCK_EVT_MODE_ONESHOT ); -__tick_program_event (newdev, next_event,1 ); +tick_dev_program_event (newdev, next_event,1 ); }/**
PATCH 6/6 HPET: make minimum reprogramming delta useful
In reply to Thomas GleixnerThe minimum reprogramming delta was hardcoded in HPET ticks, which is stupid as it does not work with faster running HPETs. The C1E idle patches made this prominent on AMD/RS690 chipsets, where the HPET runs with 25MHz. Set it to 5us which seems to be a reasonable value and fixes the problems on the bug reporters machines. We have a further sanity check now in the clock events, which increases the delta when it is not sufficient.
--- arch/x86/kernel/hpet.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/kernel/hpet.c +2 -2
@@ -210,8 +210,8 @@ static void hpet_legacy_clockevent_regis/* Calculate the min / max delta */ hpet_clockevent.max_delta_ns =clockevent_delta2ns (0x7FFFFFFF , &hpet_clockevent); - hpet_clockevent.min_delta_ns =clockevent_delta2ns (0x30 , - &hpet_clockevent); +/* 5 usec minimum reprogramming delta. */ + hpet_clockevent.min_delta_ns =5000 ;/* * Start hpet with the boot cpu mask and make it
Quoted 4 lines · Thomas Gleixner
This is the final version of the various clockevents fixes, which I will send to Linus. Can you please give it a try on your machines, so we can be sure that I did not mess up anything again.
Applied full patchset on top of 2.6.27-rc5. Works great on my affected system.
Signature · 1 line
WBR, Dmitry
Quoted 9 lines · Dmitry Nezhevenko
Quoted 4 lines · Thomas Gleixner
This is the final version of the various clockevents fixes, which I will send to Linus. Can you please give it a try on your machines, so we can be sure that I did not mess up anything again.
Applied full patchset on top of 2.6.27-rc5. Works great on my affected system.
Tested-by Dmitry Nezhevenko <[email protected]>
Dimitry, thanks for testing.
Are you up for another debug session in the next days to figure out why the x86 tree exposes this IPI warning ?
Thanks,
tglx
Quoted 5 lines · Thomas Gleixner
Dimitry, thanks for testing.
Are you up for another debug session in the next days to figure out why the x86 tree exposes this IPI warning ?
Sure. I'm subscribed to LKML however don't read it often. So please CC me
Signature · 1 line
WBR, Dmitry
Em Thu, 4 Sep 2008 02:00:03 +0300
Dmitry Nezhevenko <[email protected]> escreveu:
| On Wed, Sep 03, 2008 at 09:36:25PM -0000, Thomas Gleixner wrote: | > This is the final version of the various clockevents fixes, which I will | > send to Linus. Can you please give it a try on your machines, so we can | > be sure that I did not mess up anything again. | > | | Applied full patchset on top of 2.6.27-rc5. Works great on my affected system.
Worked for me too, sorry for the delay I was testing this on more
machines.
Signature · 1 line
Luiz Fernando N. Capitulino