Skip to contentSkip to conversation

tile: ns2cycles must be called with preempt disabled.

PATCH In review Henrik Austad 2 participants 6 messages Mar 22, 2013 – Mar 26, 2013

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 tile: ns2cycles must be called with preempt disabled.

ns2cycles use per_cpu variables, and will, eventually, find its way into smp_processord_id(). The latter must be called with either preempt disabled, irq's off, from a thread locked to a single core or early in the bootprocess (according to debug_smp_processor_id())

BUG: using smp_processor_id() in preemptible [00000000] code: systemd-modules/367
caller is ns2cycles+0x40/0xb8

Starting stack dump of tid 367, pid 367 (systemd-modules) on cpu 2 at cycle 20969956421
frame 0: 0xfffffff70004b860 dump_stack+0x0/0x20 (sp 0xfffffe407993fa90) frame 1: 0xfffffff7006abc28 debug_smp_processor_id+0x1a8/0x1e0 (sp 0xfffffe407993fa90) frame 2: 0xfffffff7004d7b40 ns2cycles+0x40/0xb8 (sp 0xfffffe407993fab8) frame 3: 0xfffffff7004dc578 __ndelay+0x38/0x80 (sp 0xfffffe407993fae0)

Note: wrapping the -entire- delay-loop in a preempt-off section can be a bad idea as long delays would case kernel lockups without being able to preempt the thread at all.

---
 arch/tile/lib/delay.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
arch/tile/lib/delay.c +2 -0
index cdacdd1..a53cdcd 100644
@@ -30,7 +30,9 @@ EXPORT_SYMBOL(__udelay);
 void __ndelay(unsigned long nsecs)
 {
 	cycles_t target = get_cycles();
+	preempt_disable();
 	target += ns2cycles(nsecs);
+	preempt_enable();
 	while (get_cycles() < target)
 		cpu_relax();
 }
Signature · 1 line
1.7.2.5
#1 lore raw
Quoted 4 lines · Henrik Austad

ns2cycles use per_cpu variables, and will, eventually, find its way into smp_processord_id(). The latter must be called with either preempt disabled, irq's off, from a thread locked to a single core or early in the bootprocess (according to debug_smp_processor_id())

A better fix would be to use __raw_get_cpu_var() in ns2cycles(), along with a comment saying that the frequency of the tile timer is the same on all cores, so it doesn't matter which copy of the per-cpu structure we look at.

I don't object to the performance hit of the preemption calls so much as the confusing aspect of why it appears we need to disable preemption to compute the value, but not to use it :-)

Signature · 2 lines
Chris Metcalf, Tilera Corp.
http://www.tilera.com
#2 lore raw

PATCH 1/1 tile: ns2cycles should use __raw_get_cpu_var

ns2cycles use per_cpu variables, and will, eventually, find its way into smp_processord_id(). This is not safe in a preemptible kernel, preemption should ideally be disabled.

BUG: using smp_processor_id() in preemptible [00000000] code:

systemd-modules/367
caller is ns2cycles+0x40/0xb8

Starting stack dump of tid 367, pid 367 (systemd-modules) on cpu 2 at cycle 20969956421
frame 0: 0xfffffff70004b860 dump_stack+0x0/0x20 (sp 0xfffffe407993fa90) frame 1: 0xfffffff7006abc28 debug_smp_processor_id+0x1a8/0x1e0 (sp
0xfffffe407993fa90)
frame 2: 0xfffffff7004d7b40 ns2cycles+0x40/0xb8 (sp 0xfffffe407993fab8) frame 3: 0xfffffff7004dc578 __ndelay+0x38/0x80 (sp 0xfffffe407993fae0)

However, in this case:

- the frequency is the same accross all cores
- we use the data read-only
- we do not scale the frequency

Which means that we can use the __raw_get_cpu_var instead.

---
 arch/tile/kernel/time.c |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)
arch/tile/kernel/time.c +3 -1
index 42297f6..4e87cb9 100644
@@ -234,7 +234,9 @@ int setup_profiling_timer(unsigned int multiplier)
  */
 cycles_t ns2cycles(unsigned long nsecs)
 {
-	struct clock_event_device *dev = &__get_cpu_var(tile_timer);
+	/* we don't have to disable preemptions here as each core has the same
+	 * clock-frequency */
+	struct clock_event_device *dev = &__raw_get_cpu_var(tile_timer);
 	return ((u64)nsecs * dev->mult) >> dev->shift;
 }
 
Signature · 1 line
1.7.1
#3 lore raw

Re: [PATCH] tile: ns2cycles should use __raw_get_cpu_var

arch/tile/kernel/time.c int setup_profiling_timer(unsigned int multiplier) · line 238

Quoted 2 lines · Henrik Austad
arch/tile/kernel/time.c +2 -0
+	/* we don't have to disable preemptions here as each core has the same
+	 * clock-frequency */

Looks OK except the comment is in the wrong style. Kernel multiline comments

/*
* Should look
* like this.
*/

and use full sentences with punctuation. (Also, don't hyphenate clock frequency here.) Thanks!

Signature · 2 lines
Chris Metcalf, Tilera Corp.
http://www.tilera.com
#4 lore raw

PATCH 1/1 tile: ns2cycles should use __raw_get_cpu_var

ns2cycles use per_cpu variables, and will, eventually, find its way into smp_processord_id(). This is not safe in a preemptible kernel, preemption should ideally be disabled.

BUG: using smp_processor_id() in preemptible [00000000] code:

systemd-modules/367
caller is ns2cycles+0x40/0xb8

Starting stack dump of tid 367, pid 367 (systemd-modules) on cpu 2 at cycle 20969956421
frame 0: 0xfffffff70004b860 dump_stack+0x0/0x20 (sp 0xfffffe407993fa90) frame 1: 0xfffffff7006abc28 debug_smp_processor_id+0x1a8/0x1e0 (sp
0xfffffe407993fa90)
frame 2: 0xfffffff7004d7b40 ns2cycles+0x40/0xb8 (sp 0xfffffe407993fab8) frame 3: 0xfffffff7004dc578 __ndelay+0x38/0x80 (sp 0xfffffe407993fae0)

However, in this case:

- the frequency is the same accross all cores
- we use the data read-only
- we do not scale the frequency

Which means that we can use the __raw_get_cpu_var instead.

---
 arch/tile/kernel/time.c |    6 +++++-
 1 files changed, 5 insertions(+), 1 deletions(-)
arch/tile/kernel/time.c +5 -1
index 42297f6..1dc1d61 100644
@@ -234,7 +234,11 @@ int setup_profiling_timer(unsigned int multiplier)
  */
 cycles_t ns2cycles(unsigned long nsecs)
 {
-	struct clock_event_device *dev = &__get_cpu_var(tile_timer);
+	/*
+	 * We do not have to disable preemptions here as each core has the same
+	 * clock frequency.
+	 */
+	struct clock_event_device *dev = &__raw_get_cpu_var(tile_timer);
 	return ((u64)nsecs * dev->mult) >> dev->shift;
 }
 
Signature · 1 line
1.7.1
#5 lore raw

Re: [PATCH] tile: ns2cycles should use __raw_get_cpu_var

Quoted 22 lines · Henrik Austad

ns2cycles use per_cpu variables, and will, eventually, find its way into smp_processord_id(). This is not safe in a preemptible kernel, preemption should ideally be disabled.

BUG: using smp_processor_id() in preemptible [00000000] code:

systemd-modules/367
caller is ns2cycles+0x40/0xb8

Starting stack dump of tid 367, pid 367 (systemd-modules) on cpu 2 at cycle 20969956421
frame 0: 0xfffffff70004b860 dump_stack+0x0/0x20 (sp 0xfffffe407993fa90) frame 1: 0xfffffff7006abc28 debug_smp_processor_id+0x1a8/0x1e0 (sp
0xfffffe407993fa90)
frame 2: 0xfffffff7004d7b40 ns2cycles+0x40/0xb8 (sp 0xfffffe407993fab8) frame 3: 0xfffffff7004dc578 __ndelay+0x38/0x80 (sp 0xfffffe407993fae0)

However, in this case:

- the frequency is the same accross all cores
- we use the data read-only
- we do not scale the frequency

Quoted 6 lines

Which means that we can use the __raw_get_cpu_var instead.

---
 arch/tile/kernel/time.c |    6 +++++-
 1 files changed, 5 insertions(+), 1 deletions(-)

Thanks - taken into the tile tree!

Signature · 2 lines
Chris Metcalf, Tilera Corp.
http://www.tilera.com