Skip to contentSkip to conversation

x86_udelay_tsc not honoring notsc

BUG William Lee Irwin III 2 participants 6 messages Sep 20, 2002

lore
Share & tools
lore
#0 lore raw

Even though I booted with notsc, rdtsc happens here and appears to generate #GP or some such nonsense:

Restoring NMI vector
Booting processor 14/52 eip 2000
Initializing CPU#14
masked ExtINT on CPU#14
Leaving ESR disabled.
Calibrating delay loop...
Program received signal SIGEMT, Emulation trap.
__rdtsc_delay (loops=69700) at delay.c:40
40 delay.c: No such file or directory.
in delay.c
(gdb) bt

#0  __rdtsc_delay (loops=69700) at delay.c:40
#1  0xc019f31d in __delay (loops=69700) at delay.c:63
#2  0xc019f384 in __const_udelay (xloops=429400) at delay.c:74
#3  0xc02c1709 in do_boot_cpu (apicid=52) at smpboot.c:865
#4  0xc02c1a4e in smp_boot_cpus (max_cpus=4294967295) at smpboot.c:1095
#5  0xc02c1c70 in smp_prepare_cpus (max_cpus=4294967295) at smpboot.c:1199
#6  0xc01050d7 in init (unused=0x0) at main.c:547

(gdb) p $eip
$3 = (void *) 0xc019f2d4
(gdb) disassemble $eip
Dump of assembler code for function __rdtsc_delay:

0xc019f2c0 <__rdtsc_delay>:     push   %ebp
0xc019f2c1 <__rdtsc_delay+1>:   mov    %esp,%ebp
0xc019f2c3 <__rdtsc_delay+3>:   push   %ebx
0xc019f2c4 <__rdtsc_delay+4>:   mov    0x8(%ebp),%ebx
0xc019f2c7 <__rdtsc_delay+7>:   rdtsc  
0xc019f2c9 <__rdtsc_delay+9>:   mov    %eax,%ecx
0xc019f2cb <__rdtsc_delay+11>:  nop    
0xc019f2cc <__rdtsc_delay+12>:  lea    0x0(%esi,1),%esi
0xc019f2d0 <__rdtsc_delay+16>:  repz nop 
0xc019f2d2 <__rdtsc_delay+18>:  rdtsc  
0xc019f2d4 <__rdtsc_delay+20>:  sub    %ecx,%eax
0xc019f2d6 <__rdtsc_delay+22>:  cmp    %ebx,%eax
0xc019f2d8 <__rdtsc_delay+24>:  jb     0xc019f2d0 <__rdtsc_delay+16>
0xc019f2da <__rdtsc_delay+26>:  pop    %ebx
0xc019f2db <__rdtsc_delay+27>:  mov    %ebp,%esp
0xc019f2dd <__rdtsc_delay+29>:  pop    %ebp
0xc019f2de <__rdtsc_delay+30>:  ret    

End of assembler dump.

#1 lore raw

Does this help (on top of John's TSC patch in rollup 0)?

include/asm-i386/cpufeature.h +5 -1
diff -urN -X /home/fletch/.diff.exclude virgin/include/asm-i386/cpufeature.h cpu_has_tsc/include/asm-i386/cpufeature.h
@@ -70,7 +70,11 @@
 #define cpu_has_vme		boot_cpu_has(X86_FEATURE_VME)
 #define cpu_has_de		boot_cpu_has(X86_FEATURE_DE)
 #define cpu_has_pse		boot_cpu_has(X86_FEATURE_PSE)
-#define cpu_has_tsc		boot_cpu_has(X86_FEATURE_TSC)
+#ifdef CONFIG_X86_TSC
+#define cpu_has_tsc             boot_cpu_has(X86_FEATURE_TSC)
+#else
+#define cpu_has_tsc             (0)
+#endif
 #define cpu_has_pae		boot_cpu_has(X86_FEATURE_PAE)
 #define cpu_has_pge		boot_cpu_has(X86_FEATURE_PGE)
 #define cpu_has_apic
	boot_cpu_has(X86_FEATURE_APIC)
Quoted 51 lines · --On Thursday, September 19, 2002 5:10 PM -0700 William Lee Irwin III…

Even though I booted with notsc, rdtsc happens here and appears to generate #GP or some such nonsense:

Restoring NMI vector
Booting processor 14/52 eip 2000
Initializing CPU#14
masked ExtINT on CPU#14
Leaving ESR disabled.
Calibrating delay loop...
Program received signal SIGEMT, Emulation trap.
__rdtsc_delay (loops=69700) at delay.c:40
40 delay.c: No such file or directory.
in delay.c
(gdb) bt

# 0  __rdtsc_delay (loops=69700) at delay.c:40
# 1  0xc019f31d in __delay (loops=69700) at delay.c:63
# 2  0xc019f384 in __const_udelay (xloops=429400) at delay.c:74
# 3  0xc02c1709 in do_boot_cpu (apicid=52) at smpboot.c:865
# 4  0xc02c1a4e in smp_boot_cpus (max_cpus=4294967295) at smpboot.c:1095
# 5  0xc02c1c70 in smp_prepare_cpus (max_cpus=4294967295) at smpboot.c:1199
# 6  0xc01050d7 in init (unused=0x0) at main.c:547

(gdb) p $eip
$3 = (void *) 0xc019f2d4
(gdb) disassemble $eip
Dump of assembler code for function __rdtsc_delay:

0xc019f2c0 <__rdtsc_delay>:     push   %ebp
0xc019f2c1 <__rdtsc_delay+1>:   mov    %esp,%ebp
0xc019f2c3 <__rdtsc_delay+3>:   push   %ebx
0xc019f2c4 <__rdtsc_delay+4>:   mov    0x8(%ebp),%ebx
0xc019f2c7 <__rdtsc_delay+7>:   rdtsc  
0xc019f2c9 <__rdtsc_delay+9>:   mov    %eax,%ecx
0xc019f2cb <__rdtsc_delay+11>:  nop    
0xc019f2cc <__rdtsc_delay+12>:  lea    0x0(%esi,1),%esi
0xc019f2d0 <__rdtsc_delay+16>:  repz nop 
0xc019f2d2 <__rdtsc_delay+18>:  rdtsc  
0xc019f2d4 <__rdtsc_delay+20>:  sub    %ecx,%eax
0xc019f2d6 <__rdtsc_delay+22>:  cmp    %ebx,%eax
0xc019f2d8 <__rdtsc_delay+24>:  jb     0xc019f2d0 <__rdtsc_delay+16>
0xc019f2da <__rdtsc_delay+26>:  pop    %ebx
0xc019f2db <__rdtsc_delay+27>:  mov    %ebp,%esp
0xc019f2dd <__rdtsc_delay+29>:  pop    %ebp
0xc019f2de <__rdtsc_delay+30>:  ret    

End of assembler dump.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#2 lore raw
Quoted 1 line · Martin J. Bligh

Does this help (on top of John's TSC patch in rollup 0)?

Nope. But I believe I found the root cause: it apparently takes long enough to kick all the cpus the NMI oopser goes off during one of the many long ints-off sections in the bootstrap phase. My burning question now is why this only showed up in 2.5.36. Somehow I mistook a rather blatant (c.f. SIGEMT) NMI oops for TSD %cr4 #GP.

I'm going to guess the NMI oopser was not eager enough to trip beforehand and recent changes repaired that. Is this close?

If so, it's probably not worth mucking around with the bootstrap sequence to deal with something this minor. It's not like it can be mistaken for having hung, as console output is very consistent. Maybe we should give NUMA-Q a couple of minutes instead of 5s?

Thanks,
Bill

#3 lore raw
Quoted 8 lines
Quoted 1 line

Does this help (on top of John's TSC patch in rollup 0)?

Nope. But I believe I found the root cause: it apparently takes long enough to kick all the cpus the NMI oopser goes off during one of the many long ints-off sections in the bootstrap phase. My burning question now is why this only showed up in 2.5.36. Somehow I mistook a rather blatant (c.f. SIGEMT) NMI oops for TSD %cr4 #GP.

Quoted 7 lines

I'm going to guess the NMI oopser was not eager enough to trip beforehand and recent changes repaired that. Is this close?

If so, it's probably not worth mucking around with the bootstrap sequence to deal with something this minor. It's not like it can be mistaken for having hung, as console output is very consistent. Maybe we should give NUMA-Q a couple of minutes instead of 5s?

Nah, just recode the boot sequence to make them all boot in parallel ;-)

M.

#4 lore raw
Quoted 4 lines · At some point in the past, I wrote:
Quoted 4 lines

If so, it's probably not worth mucking around with the bootstrap sequence to deal with something this minor. It's not like it can be mistaken for having hung, as console output is very consistent. Maybe we should give NUMA-Q a couple of minutes instead of 5s?

Quoted 3 lines · Martin J. Bligh

Nah, just recode the boot sequence to make them all boot in parallel ;-)
M.

Do you think cpu wakeup alone could be doing this? If so, then doing that bit would be relatively isolated (though a slightly larger diff than changing an NMI oopser timeout).

Does 0xFF broadcast cluster, broadcast low nybble work or is waking them a cluster at a time required? This thing is not swift to boot...

Thanks,
Bill

#5 lore raw
Quoted 6 lines
Quoted 4 lines · At some point in the past, I wrote:
Quoted 4 lines

If so, it's probably not worth mucking around with the bootstrap sequence to deal with something this minor. It's not like it can be mistaken for having hung, as console output is very consistent. Maybe we should give NUMA-Q a couple of minutes instead of 5s?

Quoted 7 lines
Quoted 2 lines · Martin J. Bligh

Nah, just recode the boot sequence to make them all boot in parallel ;-)

Do you think cpu wakeup alone could be doing this? If so, then doing that bit would be relatively isolated (though a slightly larger diff than changing an NMI oopser timeout).

Seems odd that you didn't get it in previous versions ... ? But running the NMI oops detection whilst stealing NMIs in order to bootstrap the CPUs is probably a Bad Idea (tm).

Quoted 2 lines

Does 0xFF broadcast cluster, broadcast low nybble work or is waking them a cluster at a time required? This thing is not swift to boot...

Well, you don't want to send an NMI to yourself (the BSP), I presume?
That would probably make it unhappy ;-) The NMIs have to be logically addressed, which precludes the allbutself thingy, IIRC. So you have a choice of serialised unicast (which is probably quite fast enough)
or cluster at a time, being careful to exclude yourself when you do
the BSP's cluster.

But I think you'd be far better off just disabling the NMI oopser until we've booted - the dual use is too much nefarious incest for my liking.

M.