Skip to contentSkip to conversation

tick: Make sleep length calculation more accurate

PATCH In review Daniel Lezcano 2 participants 7 messages Oct 2, 2013 – Oct 5, 2013 Nohz, dynticks support

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 tick: Make sleep length calculation more accurate

The sleep_length is computed in the tick_nohz_stop_sched_tick function but it is used later in the code with in between the local irq enabled.

cpu_idle_loop
tick_nohz_idle_enter [ exits with local irq enabled ]
__tick_nohz_idle_enter
tick_nohz_stop_sched_tick
...

arch_cpu_idle
menu_select [ uses here 'sleep_length' ]
...

Between the computation of the sleep length and its usage, some interrupts may occur, making the sleep length shorter than actually it is because of the interrupt processing, or different if the timer itself expired.

This patch fixes that by moving the sleep_length computation in the tick_nohz_get_sleep_length function and using the tick device's next_event.

As the sleep_length field is no longer needed, it is removed from the tick_sched structure.

---
 include/linux/tick.h     |    2 --
 kernel/time/tick-sched.c |    5 +++--
 2 files changed, 3 insertions(+), 4 deletions(-)
include/linux/tick.h +0 -2
index 5128d33..53dbbd7 100644
@@ -48,7 +48,6 @@ enum tick_nohz_mode {
  * @idle_exittime:	Time when the idle state was left
  * @idle_sleeptime:	Sum of the time slept in idle with sched tick stopped
  * @iowait_sleeptime:	Sum of the time slept in idle with sched tick stopped, with IO outstanding
- * @sleep_length:	Duration of the current idle sleep
  * @do_timer_lst:	CPU was the last one doing do_timer before going idle
  */
 struct tick_sched {
@@ -67,7 +66,6 @@ struct tick_sched {
 	ktime_t				idle_exittime;
 	ktime_t				idle_sleeptime;
 	ktime_t				iowait_sleeptime;
-	ktime_t				sleep_length;
 	unsigned long			last_jiffies;
 	unsigned long			next_jiffies;
 	ktime_t				idle_expires;
kernel/time/tick-sched.c +3 -2
index 3612fc7..60b1dcd 100644
@@ -673,7 +673,6 @@ static ktime_t tick_nohz_stop_sched_tick(struct tick_sched *ts,
 out:
 	ts->next_jiffies = next_jiffies;
 	ts->last_jiffies = last_jiffies;
-	ts->sleep_length = ktime_sub(dev->next_event, now);
 
 	return ret;
 }
@@ -837,8 +836,10 @@ void tick_nohz_irq_exit(void)
 ktime_t tick_nohz_get_sleep_length(void)
 {
 	struct tick_sched *ts = &__get_cpu_var(tick_cpu_sched);
+	struct clock_event_device *dev = __get_cpu_var(tick_cpu_device).evtdev;
+	ktime_t now = ktime_get();
 
-	return ts->sleep_length;
+	return ktime_sub(dev->next_event, now);
 }
 
 static void tick_nohz_restart(struct tick_sched *ts, ktime_t now)
Signature · 1 line
1.7.9.5
#1 lore raw
Quoted 17 lines · Daniel Lezcano

The sleep_length is computed in the tick_nohz_stop_sched_tick function but it is used later in the code with in between the local irq enabled.

cpu_idle_loop
tick_nohz_idle_enter [ exits with local irq enabled ]
__tick_nohz_idle_enter
tick_nohz_stop_sched_tick
...

arch_cpu_idle
menu_select [ uses here 'sleep_length' ]
...

Between the computation of the sleep length and its usage, some interrupts may occur, making the sleep length shorter than actually it is because of the interrupt processing, or different if the timer itself expired.

Quoted 8 lines

This patch fixes that by moving the sleep_length computation in the tick_nohz_get_sleep_length function and using the tick device's next_event.

As the sleep_length field is no longer needed, it is removed from the tick_sched structure.

Stephen,

I added your signed-off as the change in tick_nohz_get_sleep_length is the one you proposed previously.

-- Daniel

Quoted 50 lines
---
  include/linux/tick.h     |    2 --
  kernel/time/tick-sched.c |    5 +++--
  2 files changed, 3 insertions(+), 4 deletions(-)
include/linux/tick.h +0 -2
index 5128d33..53dbbd7 100644
@@ -48,7 +48,6 @@ enum tick_nohz_mode {
   * @idle_exittime:	Time when the idle state was left
   * @idle_sleeptime:	Sum of the time slept in idle with sched tick stopped
   * @iowait_sleeptime:	Sum of the time slept in idle with sched tick stopped, with IO outstanding
- * @sleep_length:	Duration of the current idle sleep
   * @do_timer_lst:	CPU was the last one doing do_timer before going idle
   */
  struct tick_sched {
@@ -67,7 +66,6 @@ struct tick_sched {
  	ktime_t				idle_exittime;
  	ktime_t				idle_sleeptime;
  	ktime_t				iowait_sleeptime;
-	ktime_t				sleep_length;
  	unsigned long			last_jiffies;
  	unsigned long			next_jiffies;
  	ktime_t				idle_expires;
kernel/time/tick-sched.c +3 -2
index 3612fc7..60b1dcd 100644
@@ -673,7 +673,6 @@ static ktime_t tick_nohz_stop_sched_tick(struct tick_sched *ts,
  out:
  	ts->next_jiffies = next_jiffies;
  	ts->last_jiffies = last_jiffies;
-	ts->sleep_length = ktime_sub(dev->next_event, now);
 
  	return ret;
  }
@@ -837,8 +836,10 @@ void tick_nohz_irq_exit(void)
  ktime_t tick_nohz_get_sleep_length(void)
  {
  	struct tick_sched *ts = &__get_cpu_var(tick_cpu_sched);
+	struct clock_event_device *dev = __get_cpu_var(tick_cpu_device).evtdev;
+	ktime_t now = ktime_get();
 
-	return ts->sleep_length;
+	return ktime_sub(dev->next_event, now);
  }
 
  static void tick_nohz_restart(struct tick_sched *ts, ktime_t now)
Signature · 5 lines
 <http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs

Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog
#2 lore raw
In reply to Daniel Lezcano

2013/10/2 Daniel Lezcano <[email protected]>:

Quoted 9 lines

The sleep_length is computed in the tick_nohz_stop_sched_tick function but it is used later in the code with in between the local irq enabled.

cpu_idle_loop
tick_nohz_idle_enter [ exits with local irq enabled ]
__tick_nohz_idle_enter
tick_nohz_stop_sched_tick
...

Quoted 7 lines

arch_cpu_idle
menu_select [ uses here 'sleep_length' ]
...

Between the computation of the sleep length and its usage, some interrupts may occur, making the sleep length shorter than actually it is because of the interrupt processing

So, do you mean that the ts->sleep_length would return a value that is too long given that the CPU already spent some time to service the irqs since we computed the sleep length in tick_nohz_idle_enter()?

But then tick_nohz_irq_exit() should take care of that as it calls again tick_nohz_stop_sched_tick().
So I'm a bit confused.

Quoted 1 line

or different if the timer itself expired.

Same here, if the timer expired, it triggers an interrupt which can do two things:

1) reprogram a new timer and this recompute sleep_length
2) set_need_resched() and then exit the idle loop, so arch_cpu_idle() won't even be called. Or the timer interrupts hlt, but then menu_select() was called before.

So I probably missed something here.

Thanks.

#3 lore raw
Quoted 31 lines · Frederic Weisbecker

2013/10/2 Daniel Lezcano <[email protected]>:

Quoted 9 lines

The sleep_length is computed in the tick_nohz_stop_sched_tick function but it is used later in the code with in between the local irq enabled.

cpu_idle_loop
tick_nohz_idle_enter [ exits with local irq enabled ]
__tick_nohz_idle_enter
tick_nohz_stop_sched_tick
...

Quoted 7 lines

arch_cpu_idle
menu_select [ uses here 'sleep_length' ]
...

Between the computation of the sleep length and its usage, some interrupts may occur, making the sleep length shorter than actually it is because of the interrupt processing

So, do you mean that the ts->sleep_length would return a value that is too long given that the CPU already spent some time to service the irqs since we computed the sleep length in tick_nohz_idle_enter()?

But then tick_nohz_irq_exit() should take care of that as it calls again tick_nohz_stop_sched_tick().
So I'm a bit confused.

Quoted 1 line

or different if the timer itself expired.

Same here, if the timer expired, it triggers an interrupt which can do two things:

Quoted 6 lines

1) reprogram a new timer and this recompute sleep_length
2) set_need_resched() and then exit the idle loop, so arch_cpu_idle() won't even be called. Or the timer interrupts hlt, but then menu_select() was called before.

So I probably missed something here.

No you did not :)

Indeed... At the first glance, this issue sounded so obvious I suspected there must be a trick somewhere but I did not think to look at the irq_exit, the code is very complex. Thanks for clarifying this.

For my personal information, is there any particular reason to set an intermediate 'sleep_length' in tick_nohz_stop_sched_tick instead of doing what does this patch ?

Thanks
-- Daniel

Signature · 5 lines
 <http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs

Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog
#4 lore raw
Quoted 41 lines · Daniel Lezcano
Quoted 31 lines · Frederic Weisbecker

2013/10/2 Daniel Lezcano <[email protected]>:

Quoted 9 lines

The sleep_length is computed in the tick_nohz_stop_sched_tick function but it is used later in the code with in between the local irq enabled.

cpu_idle_loop
tick_nohz_idle_enter [ exits with local irq enabled ]
__tick_nohz_idle_enter
tick_nohz_stop_sched_tick
...

Quoted 7 lines

arch_cpu_idle
menu_select [ uses here 'sleep_length' ]
...

Between the computation of the sleep length and its usage, some interrupts may occur, making the sleep length shorter than actually it is because of the interrupt processing

So, do you mean that the ts->sleep_length would return a value that is too long given that the CPU already spent some time to service the irqs since we computed the sleep length in tick_nohz_idle_enter()?

But then tick_nohz_irq_exit() should take care of that as it calls again tick_nohz_stop_sched_tick().
So I'm a bit confused.

Quoted 1 line

or different if the timer itself expired.

Same here, if the timer expired, it triggers an interrupt which can do two things:

Quoted 6 lines

1) reprogram a new timer and this recompute sleep_length
2) set_need_resched() and then exit the idle loop, so arch_cpu_idle() won't even be called. Or the timer interrupts hlt, but then menu_select() was called before.

So I probably missed something here.

No you did not :)

Quoted 8 lines

Indeed... At the first glance, this issue sounded so obvious I suspected there must be a trick somewhere but I did not think to look at the irq_exit, the code is very complex. Thanks for clarifying this.

For my personal information, is there any particular reason to set an intermediate 'sleep_length' in tick_nohz_stop_sched_tick instead of doing what does this patch ?

May be we could do it that way yeah. Is menu_select() called only there? I don't know how much difference that would make.

Quoted 11 lines

Thanks
-- Daniel

Signature · 5 lines
 <http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs

Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog
#5 lore raw
Quoted 47 lines · Frederic Weisbecker
Quoted 46 lines · Daniel Lezcano
Quoted 31 lines · Frederic Weisbecker

2013/10/2 Daniel Lezcano <[email protected]>:

Quoted 9 lines

The sleep_length is computed in the tick_nohz_stop_sched_tick function but it is used later in the code with in between the local irq enabled.

cpu_idle_loop
tick_nohz_idle_enter [ exits with local irq enabled ]
__tick_nohz_idle_enter
tick_nohz_stop_sched_tick
...

Quoted 7 lines

arch_cpu_idle
menu_select [ uses here 'sleep_length' ]
...

Between the computation of the sleep length and its usage, some interrupts may occur, making the sleep length shorter than actually it is because of the interrupt processing

So, do you mean that the ts->sleep_length would return a value that is too long given that the CPU already spent some time to service the irqs since we computed the sleep length in tick_nohz_idle_enter()?

But then tick_nohz_irq_exit() should take care of that as it calls again tick_nohz_stop_sched_tick().
So I'm a bit confused.

Quoted 1 line

or different if the timer itself expired.

Same here, if the timer expired, it triggers an interrupt which can do two things:

Quoted 6 lines

1) reprogram a new timer and this recompute sleep_length
2) set_need_resched() and then exit the idle loop, so arch_cpu_idle() won't even be called. Or the timer interrupts hlt, but then menu_select() was called before.

So I probably missed something here.

No you did not :)

Indeed... At the first glance, this issue sounded so obvious I suspected there must be a trick somewhere but I did not think to look at the irq_exit, the code is very complex. Thanks for clarifying this.

Quoted 6 lines
Quoted 3 lines

For my personal information, is there any particular reason to set an intermediate 'sleep_length' in tick_nohz_stop_sched_tick instead of doing what does this patch ?

May be we could do it that way yeah. Is menu_select() called only there? I don't know how much difference that would make.

Yes, it is called just one time in all the code. The benefit would be just to cleanup a field in the struct tick_sched.

Signature · 5 lines
 <http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs

Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog
#6 lore raw
Quoted 52 lines · Daniel Lezcano
Quoted 51 lines · Frederic Weisbecker
Quoted 49 lines · Daniel Lezcano
Quoted 31 lines · Frederic Weisbecker

2013/10/2 Daniel Lezcano <[email protected]>:

Quoted 9 lines

The sleep_length is computed in the tick_nohz_stop_sched_tick function but it is used later in the code with in between the local irq enabled.

cpu_idle_loop
tick_nohz_idle_enter [ exits with local irq enabled ]
__tick_nohz_idle_enter
tick_nohz_stop_sched_tick
...

Quoted 7 lines

arch_cpu_idle
menu_select [ uses here 'sleep_length' ]
...

Between the computation of the sleep length and its usage, some interrupts may occur, making the sleep length shorter than actually it is because of the interrupt processing

So, do you mean that the ts->sleep_length would return a value that is too long given that the CPU already spent some time to service the irqs since we computed the sleep length in tick_nohz_idle_enter()?

But then tick_nohz_irq_exit() should take care of that as it calls again tick_nohz_stop_sched_tick().
So I'm a bit confused.

Quoted 1 line

or different if the timer itself expired.

Same here, if the timer expired, it triggers an interrupt which can do two things:

Quoted 6 lines

1) reprogram a new timer and this recompute sleep_length
2) set_need_resched() and then exit the idle loop, so arch_cpu_idle() won't even be called. Or the timer interrupts hlt, but then menu_select() was called before.

So I probably missed something here.

No you did not :)

Indeed... At the first glance, this issue sounded so obvious I suspected there must be a trick somewhere but I did not think to look at the irq_exit, the code is very complex. Thanks for clarifying this.

For my personal information, is there any particular reason to set an intermediate 'sleep_length' in tick_nohz_stop_sched_tick instead of doing what does this patch ?

Quoted 5 lines
Quoted 2 lines

May be we could do it that way yeah. Is menu_select() called only there? I don't know how much difference that would make.

Yes, it is called just one time in all the code. The benefit would be just to cleanup a field in the struct tick_sched.

Yeah, why not.

Thanks.

Quoted 9 lines
Signature · 6 lines
-- 
 <http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs

Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog