Skip to contentSkip to conversation

tick: make sleep length calculation more accurate

PATCH Superseded Daniel Lezcano 2 participants 3 messages Sep 27, 2013 – Oct 2, 2013 Nohz, dynticks support

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 tick: make sleep length calculation more accurate

The sleep_length is computed in the tick_nohz_stop_sched_tick function but it is used later in the code with in between the local irq enabled.

cpu_idle_loop
tick_nohz_idle_enter [ exits with local irq enabled ]
__tick_nohz_idle_enter
tick_nohz_stop_sched_tick
...

arch_cpu_idle
menu_select [ uses here 'sleep_length' ]
...

Between the computation of the sleep length and its usage, some interrupts can occur, making the sleep length shorter than actually it is.

This patch fixes that by moving the sleep_length computation in the tick_nohz_get_sleep_length function and store the next_event for the device instead of the sleep_length.

---
 include/linux/tick.h     |    2 +-
 kernel/time/tick-sched.c |    5 +++--
 2 files changed, 4 insertions(+), 3 deletions(-)
include/linux/tick.h +1 -1
index 5128d33..4932004 100644
@@ -67,7 +67,7 @@ struct tick_sched {
 	ktime_t				idle_exittime;
 	ktime_t				idle_sleeptime;
 	ktime_t				iowait_sleeptime;
-	ktime_t				sleep_length;
+	ktime_t				next_event;
 	unsigned long			last_jiffies;
 	unsigned long			next_jiffies;
 	ktime_t				idle_expires;
kernel/time/tick-sched.c +3 -2
index 3612fc7..2007a7f 100644
@@ -673,7 +673,7 @@ static ktime_t tick_nohz_stop_sched_tick(struct tick_sched *ts,
 out:
 	ts->next_jiffies = next_jiffies;
 	ts->last_jiffies = last_jiffies;
-	ts->sleep_length = ktime_sub(dev->next_event, now);
+	ts->next_event = dev->next_event;
 
 	return ret;
 }
@@ -837,8 +837,9 @@ void tick_nohz_irq_exit(void)
 ktime_t tick_nohz_get_sleep_length(void)
 {
 	struct tick_sched *ts = &__get_cpu_var(tick_cpu_sched);
+	ktime_t now = ktime_get();
 
-	return ts->sleep_length;
+	return ktime_sub(ts->next_event, now);
 }
 
 static void tick_nohz_restart(struct tick_sched *ts, ktime_t now)
Signature · 1 line
1.7.9.5
#1 lore raw
Quoted 31 lines · Daniel Lezcano

The sleep_length is computed in the tick_nohz_stop_sched_tick function but it is used later in the code with in between the local irq enabled.

cpu_idle_loop
tick_nohz_idle_enter [ exits with local irq enabled ]
__tick_nohz_idle_enter
tick_nohz_stop_sched_tick
...

arch_cpu_idle
menu_select [ uses here 'sleep_length' ]
...

Between the computation of the sleep length and its usage, some interrupts can occur, making the sleep length shorter than actually it is.

This patch fixes that by moving the sleep_length computation in the tick_nohz_get_sleep_length function and store the next_event for the device instead of the sleep_length.

---
 include/linux/tick.h     |    2 +-
 kernel/time/tick-sched.c |    5 +++--
 2 files changed, 4 insertions(+), 3 deletions(-)
include/linux/tick.h +0 -0
index 5128d33..4932004 100644

include/linux/tick.h struct tick_sched { · line 73

Quoted 8 lines
include/linux/tick.h +1 -1
@@ -67,7 +67,7 @@ struct tick_sched {
 	ktime_t				idle_exittime;
 	ktime_t				idle_sleeptime;
 	ktime_t				iowait_sleeptime;
-	ktime_t				sleep_length;
+	ktime_t				next_event;
 	unsigned long			last_jiffies;
 	unsigned long			next_jiffies;
 	ktime_t				idle_expires;

Documentation update?

Quoted 14 lines
kernel/time/tick-sched.c +1 -1
index 3612fc7..2007a7f 100644
@@ -673,7 +673,7 @@ static ktime_t tick_nohz_stop_sched_tick(struct tick_sched *ts,
 out:
 	ts->next_jiffies = next_jiffies;
 	ts->last_jiffies = last_jiffies;
-	ts->sleep_length = ktime_sub(dev->next_event, now);
+	ts->next_event = dev->next_event;
 
 	return ret;
 }

kernel/time/tick-sched.c void tick_nohz_irq_exit(void) · line 845

Quoted 10 lines
kernel/time/tick-sched.c +2 -1
@@ -837,8 +837,9 @@ void tick_nohz_irq_exit(void)
 ktime_t tick_nohz_get_sleep_length(void)
 {
 	struct tick_sched *ts = &__get_cpu_var(tick_cpu_sched);
+	ktime_t now = ktime_get();
 
-	return ts->sleep_length;
+	return ktime_sub(ts->next_event, now);
 }
 
 static void tick_nohz_restart(struct tick_sched *ts, ktime_t now)

What happens if the idling CPU's next_event is updated via that interrupt? Say if the interrupt handler schedules a timer to fire before the next timer on the CPU? It looks like we won't notice that.

Perhaps it's better to do this instead?

ktime_t tick_nohz_get_sleep_length(void)
{
struct tick_sched *ts = &__get_cpu_var(tick_cpu_sched);
+ ktime_t now = ktime_get();
+ struct clock_event_device *dev = __get_cpu_var(tick_cpu_device).evtdev;

- return ts->sleep_length;
+ return ktime_sub(dev->next_event, now);
}

Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation
#2 lore raw
Quoted 66 lines · Stephen Boyd
Quoted 31 lines · Daniel Lezcano

The sleep_length is computed in the tick_nohz_stop_sched_tick function but it is used later in the code with in between the local irq enabled.

cpu_idle_loop
tick_nohz_idle_enter [ exits with local irq enabled ]
__tick_nohz_idle_enter
tick_nohz_stop_sched_tick
...

arch_cpu_idle
menu_select [ uses here 'sleep_length' ]
...

Between the computation of the sleep length and its usage, some interrupts can occur, making the sleep length shorter than actually it is.

This patch fixes that by moving the sleep_length computation in the tick_nohz_get_sleep_length function and store the next_event for the device instead of the sleep_length.

---
  include/linux/tick.h     |    2 +-
  kernel/time/tick-sched.c |    5 +++--
  2 files changed, 4 insertions(+), 3 deletions(-)
include/linux/tick.h +0 -0
index 5128d33..4932004 100644

include/linux/tick.h struct tick_sched { · line 73

Quoted 8 lines
include/linux/tick.h +1 -1
@@ -67,7 +67,7 @@ struct tick_sched {
  	ktime_t				idle_exittime;
  	ktime_t				idle_sleeptime;
  	ktime_t				iowait_sleeptime;
-	ktime_t				sleep_length;
+	ktime_t				next_event;
  	unsigned long			last_jiffies;
  	unsigned long			next_jiffies;
  	ktime_t				idle_expires;

Documentation update?

Quoted 23 lines
kernel/time/tick-sched.c +3 -2
index 3612fc7..2007a7f 100644
@@ -673,7 +673,7 @@ static ktime_t tick_nohz_stop_sched_tick(struct tick_sched *ts,
  out:
  	ts->next_jiffies = next_jiffies;
  	ts->last_jiffies = last_jiffies;
-	ts->sleep_length = ktime_sub(dev->next_event, now);
+	ts->next_event = dev->next_event;
 
  	return ret;
  }
@@ -837,8 +837,9 @@ void tick_nohz_irq_exit(void)
  ktime_t tick_nohz_get_sleep_length(void)
  {
  	struct tick_sched *ts = &__get_cpu_var(tick_cpu_sched);
+	ktime_t now = ktime_get();
 
-	return ts->sleep_length;
+	return ktime_sub(ts->next_event, now);
  }
 
Quoted 5 lines

kernel/time/tick-sched.c void tick_nohz_irq_exit(void) · line 845

Quoted 1 line
kernel/time/tick-sched.c +0 -0
  static void tick_nohz_restart(struct tick_sched *ts, ktime_t now)

What happens if the idling CPU's next_event is updated via that interrupt? Say if the interrupt handler schedules a timer to fire before the next timer on the CPU? It looks like we won't notice that.

Yes, or after.

It sounds like this issue also occurs with the current code, no ?

kernel/time/tick-sched.c void tick_nohz_irq_exit(void) · line 843

Quoted 11 lines

Perhaps it's better to do this instead?

  ktime_t tick_nohz_get_sleep_length(void)
  {
  	struct tick_sched *ts = &__get_cpu_var(tick_cpu_sched);
kernel/time/tick-sched.c +3 -1
+	ktime_t now = ktime_get();
+	struct clock_event_device *dev = __get_cpu_var(tick_cpu_device).evtdev;
 
-	return ts->sleep_length;
+	return ktime_sub(dev->next_event, now);
  }

Yes, I agree.

Thanks for the review.

-- Daniel

Signature · 5 lines
 <http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs

Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog