x86_udelay_tsc not honoring notsc
Even though I booted with notsc, rdtsc happens here and appears to generate #GP or some such nonsense:
Restoring NMI vector
Booting processor 14/52 eip 2000
Initializing CPU#14
masked ExtINT on CPU#14
Leaving ESR disabled.
Calibrating delay loop...
Program received signal SIGEMT, Emulation trap.__rdtsc_delay (loops=69700) at delay.c:40
40 delay.c: No such file or directory.
in delay.c
(gdb) bt
#0 __rdtsc_delay (loops=69700) at delay.c:40 #1 0xc019f31d in __delay (loops=69700) at delay.c:63 #2 0xc019f384 in __const_udelay (xloops=429400) at delay.c:74 #3 0xc02c1709 in do_boot_cpu (apicid=52) at smpboot.c:865 #4 0xc02c1a4e in smp_boot_cpus (max_cpus=4294967295) at smpboot.c:1095 #5 0xc02c1c70 in smp_prepare_cpus (max_cpus=4294967295) at smpboot.c:1199 #6 0xc01050d7 in init (unused=0x0) at main.c:547
(gdb) p $eip
$3 = (void *) 0xc019f2d4
(gdb) disassemble $eip
Dump of assembler code for function __rdtsc_delay:
0xc019f2c0 <__rdtsc_delay>: push %ebp 0xc019f2c1 <__rdtsc_delay+1>: mov %esp,%ebp 0xc019f2c3 <__rdtsc_delay+3>: push %ebx 0xc019f2c4 <__rdtsc_delay+4>: mov 0x8(%ebp),%ebx 0xc019f2c7 <__rdtsc_delay+7>: rdtsc 0xc019f2c9 <__rdtsc_delay+9>: mov %eax,%ecx 0xc019f2cb <__rdtsc_delay+11>: nop 0xc019f2cc <__rdtsc_delay+12>: lea 0x0(%esi,1),%esi 0xc019f2d0 <__rdtsc_delay+16>: repz nop 0xc019f2d2 <__rdtsc_delay+18>: rdtsc 0xc019f2d4 <__rdtsc_delay+20>: sub %ecx,%eax 0xc019f2d6 <__rdtsc_delay+22>: cmp %ebx,%eax 0xc019f2d8 <__rdtsc_delay+24>: jb 0xc019f2d0 <__rdtsc_delay+16> 0xc019f2da <__rdtsc_delay+26>: pop %ebx 0xc019f2db <__rdtsc_delay+27>: mov %ebp,%esp 0xc019f2dd <__rdtsc_delay+29>: pop %ebp 0xc019f2de <__rdtsc_delay+30>: ret
End of assembler dump.
Does this help (on top of John's TSC patch in rollup 0)?
include/asm-i386/cpufeature.h +5 -1
@@ -70,7 +70,11 @@#define cpu_has_vme boot_cpu_has(X86_FEATURE_VME)#define cpu_has_de boot_cpu_has(X86_FEATURE_DE)#define cpu_has_pse boot_cpu_has(X86_FEATURE_PSE) -#define cpu_has_tsc boot_cpu_has(X86_FEATURE_TSC) +#ifdef CONFIG_X86_TSC +#define cpu_has_tsc boot_cpu_has(X86_FEATURE_TSC) +#else +#define cpu_has_tsc (0) +#endif #define cpu_has_pae boot_cpu_has(X86_FEATURE_PAE)#define cpu_has_pge boot_cpu_has(X86_FEATURE_PGE)#define cpu_has_apic
boot_cpu_has(X86_FEATURE_APIC)
Quoted 51 lines · --On Thursday, September 19, 2002 5:10 PM -0700 William Lee Irwin III…
Even though I booted with notsc, rdtsc happens here and appears to generate #GP or some such nonsense:
Restoring NMI vector
Booting processor 14/52 eip 2000
Initializing CPU#14
masked ExtINT on CPU#14
Leaving ESR disabled.
Calibrating delay loop...
Program received signal SIGEMT, Emulation trap.__rdtsc_delay(loops=69700) at delay.c:40
40delay.c: No such file or directory.
indelay.c
(gdb) bt# 0 __rdtsc_delay (loops=69700) at delay.c:40 # 1 0xc019f31d in __delay (loops=69700) at delay.c:63 # 2 0xc019f384 in __const_udelay (xloops=429400) at delay.c:74 # 3 0xc02c1709 in do_boot_cpu (apicid=52) at smpboot.c:865 # 4 0xc02c1a4e in smp_boot_cpus (max_cpus=4294967295) at smpboot.c:1095 # 5 0xc02c1c70 in smp_prepare_cpus (max_cpus=4294967295) at smpboot.c:1199 # 6 0xc01050d7 in init (unused=0x0) at main.c:547(gdb) p $eip
$3 = (void *)0xc019f2d4
(gdb) disassemble $eip
Dump of assembler code for function__rdtsc_delay:0xc019f2c0 <__rdtsc_delay>: push %ebp 0xc019f2c1 <__rdtsc_delay+1>: mov %esp,%ebp 0xc019f2c3 <__rdtsc_delay+3>: push %ebx 0xc019f2c4 <__rdtsc_delay+4>: mov 0x8(%ebp),%ebx 0xc019f2c7 <__rdtsc_delay+7>: rdtsc 0xc019f2c9 <__rdtsc_delay+9>: mov %eax,%ecx 0xc019f2cb <__rdtsc_delay+11>: nop 0xc019f2cc <__rdtsc_delay+12>: lea 0x0(%esi,1),%esi 0xc019f2d0 <__rdtsc_delay+16>: repz nop 0xc019f2d2 <__rdtsc_delay+18>: rdtsc 0xc019f2d4 <__rdtsc_delay+20>: sub %ecx,%eax 0xc019f2d6 <__rdtsc_delay+22>: cmp %ebx,%eax 0xc019f2d8 <__rdtsc_delay+24>: jb 0xc019f2d0 <__rdtsc_delay+16> 0xc019f2da <__rdtsc_delay+26>: pop %ebx 0xc019f2db <__rdtsc_delay+27>: mov %ebp,%esp 0xc019f2dd <__rdtsc_delay+29>: pop %ebp 0xc019f2de <__rdtsc_delay+30>: retEnd of assembler dump.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 1 line · Martin J. Bligh
Does this help (on top of John's TSC patch in rollup 0)?
Nope. But I believe I found the root cause: it apparently takes long enough to kick all the cpus the NMI oopser goes off during one of the many long ints-off sections in the bootstrap phase. My burning question now is why this only showed up in 2.5.36. Somehow I mistook a rather blatant (c.f. SIGEMT) NMI oops for TSD %cr4 #GP.
I'm going to guess the NMI oopser was not eager enough to trip beforehand and recent changes repaired that. Is this close?
If so, it's probably not worth mucking around with the bootstrap sequence to deal with something this minor. It's not like it can be mistaken for having hung, as console output is very consistent. Maybe we should give NUMA-Q a couple of minutes instead of 5s?
Thanks,
Bill
Quoted 8 lines
Quoted 1 line
Does this help (on top of John's TSC patch in rollup 0)?
Nope. But I believe I found the root cause: it apparently takes long enough to kick all the cpus the NMI oopser goes off during one of the many long ints-off sections in the bootstrap phase. My burning question now is why this only showed up in 2.5.36. Somehow I mistook a rather blatant (c.f. SIGEMT) NMI oops for TSD %cr4 #GP.
Quoted 7 lines
I'm going to guess the NMI oopser was not eager enough to trip beforehand and recent changes repaired that. Is this close?
If so, it's probably not worth mucking around with the bootstrap sequence to deal with something this minor. It's not like it can be mistaken for having hung, as console output is very consistent. Maybe we should give NUMA-Q a couple of minutes instead of 5s?
Nah, just recode the boot sequence to make them all boot in parallel ;-)
M.
Quoted 4 lines · At some point in the past, I wrote:
Quoted 4 lines
If so, it's probably not worth mucking around with the bootstrap sequence to deal with something this minor. It's not like it can be mistaken for having hung, as console output is very consistent. Maybe we should give NUMA-Q a couple of minutes instead of 5s?
Quoted 3 lines · Martin J. Bligh
Nah, just recode the boot sequence to make them all boot in parallel ;-)
M.
Do you think cpu wakeup alone could be doing this? If so, then doing that bit would be relatively isolated (though a slightly larger diff than changing an NMI oopser timeout).
Does 0xFF broadcast cluster, broadcast low nybble work or is waking them a cluster at a time required? This thing is not swift to boot...
Thanks,
Bill
Quoted 6 lines
Quoted 4 lines · At some point in the past, I wrote:
Quoted 4 lines
If so, it's probably not worth mucking around with the bootstrap sequence to deal with something this minor. It's not like it can be mistaken for having hung, as console output is very consistent. Maybe we should give NUMA-Q a couple of minutes instead of 5s?
Quoted 7 lines
Quoted 2 lines · Martin J. Bligh
Nah, just recode the boot sequence to make them all boot in parallel ;-)
Do you think cpu wakeup alone could be doing this? If so, then doing that bit would be relatively isolated (though a slightly larger diff than changing an NMI oopser timeout).
Seems odd that you didn't get it in previous versions ... ? But running the NMI oops detection whilst stealing NMIs in order to bootstrap the CPUs is probably a Bad Idea (tm).
Quoted 2 lines
Does
0xFFbroadcast cluster, broadcast low nybble work or is waking them a cluster at a time required? This thing is not swift to boot...
Well, you don't want to send an NMI to yourself (the BSP), I presume?
That would probably make it unhappy ;-) The NMIs have to be logically addressed, which precludes the allbutself thingy, IIRC. So you have a choice of serialised unicast (which is probably quite fast enough)
or cluster at a time, being careful to exclude yourself when you do
the BSP's cluster.
But I think you'd be far better off just disabling the NMI oopser until we've booted - the dual use is too much nefarious incest for my liking.
M.