x86: 2.6.31-rc7 crash due to buggy flat_phys_pkg_id
PATCH 1/1 x86: 2.6.31-rc7 crash due to buggy flat_phys_pkg_id
2.6.31-rc7 does not boot on vSMPowered systems. The sched domains seem to build incorrectly with error messages of the sort:
[ 8.501108] CPU31: Thermal monitoring enabled (TM1) [ 8.501127] CPU 31 MCA banks SHD:2 SHD:3 SHD:5 SHD:6 SHD:8 [ 8.650254] CPU31: Intel(R) Xeon(R) CPU E5540 @ 2.53GHz stepping 04 [ 8.710324] Brought up 32 CPUs [ 8.713916] Total of 32 processors activated (162314.96 BogoMIPS). [ 8.721489] ERROR: parent span is not a superset of domain->span [ 8.727686] ERROR: domain->groups does not contain CPU0 [ 8.733091] ERROR: groups don't span domain->span [ 8.737975] ERROR: domain->cpu_power not set [ 8.742416]
This is followed by oopsen in the scheduler code with NULL pointer deference in find_busiest_group.
Git bisection pointed to the following commit:
commit 2759c3287de27266e06f1f4e82cbd2d65f6a044c
x86: don't call read_apic_id if !cpu_has_apic
Upon examining the history of the commit, the above commit seems to be a fix for:
commit 4797f6b021a3fa399942245d07a1feb30df81bb8
x86: read apic ID in the !acpi_lapic case
However, there appears to be bug in the commit 2759c3287de27266e06f1f4e82cbd2d65f6a044c, where flat_phys_pkg_id uses initial apic id instead of hard_smp_processor_id() on SMP machines.
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=2759c3287de27266e06f1f4e82cbd2d65f6a044c
This patch fixes the bug and causes vSMPowered systems to boot up correctly,
arch/x86/kernel/apic/apic_flat_64.c +2 -1
@@ -161,7 +161,8 @@ static int flat_apic_id_registered(void)static int flat_phys_pkg_id (int initial_apic_id,int index_msb) { -return initial_apic_id >> index_msb; +return cpu_has_apic ?hard_smp_processor_id () >> index_msb : + initial_apic_id >> index_msb; }struct apic apic_flat = {
Quoted 45 lines · Ravikiran G Thirumalai
2.6.31-rc7 does not boot on vSMPowered systems. The sched domains seem to build incorrectly with error messages of the sort:
[ 8.501108] CPU31: Thermal monitoring enabled (TM1) [ 8.501127] CPU 31 MCA banks SHD:2 SHD:3 SHD:5 SHD:6 SHD:8 [ 8.650254] CPU31: Intel(R) Xeon(R) CPU E5540 @ 2.53GHz stepping 04 [ 8.710324] Brought up 32 CPUs [ 8.713916] Total of 32 processors activated (162314.96 BogoMIPS). [ 8.721489] ERROR: parent span is not a superset of domain->span [ 8.727686] ERROR: domain->groups does not contain CPU0 [ 8.733091] ERROR: groups don't span domain->span [ 8.737975] ERROR: domain->cpu_power not set [ 8.742416]This is followed by oopsen in the scheduler code with NULL pointer deference in
find_busiest_group.Git bisection pointed to the following commit:
commit
2759c3287de27266e06f1f4e82cbd2d65f6a044c
x86: don't callread_apic_idif !cpu_has_apicUpon examining the history of the commit, the above commit seems to be a fix for:
commit
4797f6b021a3fa399942245d07a1feb30df81bb8
x86: read apic ID in the !acpi_lapic caseHowever, there appears to be bug in the commit
2759c3287de27266e06f1f4e82cbd2d65f6a044c, whereflat_phys_pkg_iduses initial apic id instead ofhard_smp_processor_id()on SMP machines.
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=2759c3287de27266e06f1f4e82cbd2d65f6a044cThis patch fixes the bug and causes vSMPowered systems to boot up correctly,
Signed-off-by Ravikiran Thirumalai <[email protected]>Cc Yinghai Lu <[email protected]>Index:
linux-2.6.31-rc6/arch/x86/kernel/apic/apic_flat_64.c
===================================================================
arch/x86/kernel/apic/apic_flat_64.c static int flat_apic_id_registered(void) · line 166
Quoted 7 lines
arch/x86/kernel/apic/apic_flat_64.c +2
-1@@ -161,7 +161,8 @@ static int flat_apic_id_registered(void)static int flat_phys_pkg_id (int initial_apic_id,int index_msb) { -return initial_apic_id >> index_msb; +return cpu_has_apic ?hard_smp_processor_id () >> index_msb : + initial_apic_id >> index_msb; }
it seems we should use initial apic id here.
can you check which calling for vsmp cause the problem?arch/x86/kernel/cpu/addon_cpuid_features.c: c->cpu_core_id = apic->phys_pkg_id(c->initial_apicid, ht_mask_width)arch/x86/kernel/cpu/addon_cpuid_features.c: c->phys_proc_id = apic->phys_pkg_id(c->initial_apicid, core_plus_mask_width
);arch/x86/kernel/cpu/addon_cpuid_features.c: c->apicid = apic->phys_pkg_id(c->initial_apicid, 0);
arch/x86/kernel/cpu/common.c: c->phys_proc_id = apic->phys_pkg_id(c->initial_apicid, index_msb); arch/x86/kernel/cpu/common.c: c->cpu_core_id = apic->phys_pkg_id(c->initial_apicid, index_msb) & arch/x86/kernel/cpu/common.c: c->apicid = apic->phys_pkg_id(c->initial_apicid, 0); arch/x86/kernel/cpu/common.c: c->apicid = apic->phys_pkg_id(c->initial_apicid, 0);
YH
Quoted 30 lines · Yinghai Lu
Quoted 29 lines · Ravikiran G Thirumalai
2.6.31-rc7 does not boot on vSMPowered systems. The sched domains seem to build incorrectly with error messages of the sort:
[ 8.501108] CPU31: Thermal monitoring enabled (TM1) [ 8.501127] CPU 31 MCA banks SHD:2 SHD:3 SHD:5 SHD:6 SHD:8 [ 8.650254] CPU31: Intel(R) Xeon(R) CPU E5540 @ 2.53GHz stepping 04 [ 8.710324] Brought up 32 CPUs [ 8.713916] Total of 32 processors activated (162314.96 BogoMIPS). [ 8.721489] ERROR: parent span is not a superset of domain->span [ 8.727686] ERROR: domain->groups does not contain CPU0 [ 8.733091] ERROR: groups don't span domain->span [ 8.737975] ERROR: domain->cpu_power not set [ 8.742416]This is followed by oopsen in the scheduler code with NULL pointer deference in
find_busiest_group.Git bisection pointed to the following commit:
commit
2759c3287de27266e06f1f4e82cbd2d65f6a044c
x86: don't callread_apic_idif !cpu_has_apicUpon examining the history of the commit, the above commit seems to be a fix for:
commit
4797f6b021a3fa399942245d07a1feb30df81bb8
x86: read apic ID in the !acpi_lapic case
Quoted 7 lines
Quoted 7 lines
However, there appears to be bug in the commit
2759c3287de27266e06f1f4e82cbd2d65f6a044c, whereflat_phys_pkg_iduses initial apic id instead ofhard_smp_processor_id()on SMP machines.
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=2759c3287de27266e06f1f4e82cbd2d65f6a044cThis patch fixes the bug and causes vSMPowered systems to boot up correctly,
can you check if this one fix your problem?
BTW: what does your /proc/cpuinfo print about apic id and initial apic id?
[PATCH] x86: use read_apic_id to get apic id
and leave phys_proc_id to use initial apic id.
--- arch/x86/kernel/cpu/common.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/common.c +2 -2
@@ -715,7 +715,7 @@ static void __cpuinit generic_identify(s c->initial_apicid = (cpuid_ebx (1 ) >>24 ) &0xFF ;#ifdef CONFIG_X86_32 # ifdef CONFIG_X86_HT - c->apicid = apic->phys_pkg_id (c->initial_apicid,0 ); + c->apicid =read_apic_id ();# else c->apicid = c->initial_apicid;# endif
@@ -772,7 +772,7 @@ static void __cpuinit identify_cpu(struc }#ifdef CONFIG_X86_64 - c->apicid = apic->phys_pkg_id (c->initial_apicid,0 ); + c->apicid =read_apic_id ();#endif /*
Quoted 4 lines · Yinghai Lu
Quoted 1 line · Yinghai Lu
Ravikiran G Thirumalai wrote:
can you check if this one fix your problem?
Nope it doesn't :(
Quoted 2 lines
BTW: what does your
/proc/cpuinfoprint about apic id and initial apic id?
Here's a sample cpuinfo from an older kernel (attached)
processor : 0vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 4
cpu cores : 4
apicid : 0
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5871.22
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 1vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 0
siblings : 4
core id : 1
cpu cores : 4
cpu cores : 4
apicid : 2
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5852.70
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 2vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 0
siblings : 4
core id : 2
cpu cores : 4
cpu cores : 4
apicid : 4
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5852.83
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 3vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 0
siblings : 4
core id : 3
cpu cores : 4
cpu cores : 4
apicid : 6
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5852.59
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 4vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 1
siblings : 4
core id : 0
cpu cores : 4
cpu cores : 4
apicid : 16
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5853.76
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 5vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 1
siblings : 4
core id : 1
cpu cores : 4
cpu cores : 4
apicid : 18
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5853.72
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 6vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 1
siblings : 4
core id : 2
cpu cores : 4
cpu cores : 4
apicid : 20
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5856.30
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 7vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 1
siblings : 4
core id : 3
cpu cores : 4
cpu cores : 4
apicid : 22
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5856.34
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 8vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 2
siblings : 4
core id : 0
cpu cores : 4
cpu cores : 4
apicid : 32
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5865.94
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 9vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 2
siblings : 4
core id : 1
cpu cores : 4
cpu cores : 4
apicid : 34
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5866.71
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 10vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 2
siblings : 4
core id : 2
cpu cores : 4
cpu cores : 4
apicid : 36
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5867.00
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 11vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 2
siblings : 4
core id : 3
cpu cores : 4
cpu cores : 4
apicid : 38
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5866.25
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 12vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 3
siblings : 4
core id : 0
cpu cores : 4
cpu cores : 4
apicid : 48
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5867.65
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 13vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 3
siblings : 4
core id : 1
cpu cores : 4
cpu cores : 4
apicid : 50
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5867.00
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 14vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 3
siblings : 4
core id : 2
cpu cores : 4
cpu cores : 4
apicid : 52
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5869.20
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 15vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 3
siblings : 4
core id : 3
cpu cores : 4
cpu cores : 4
apicid : 54
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5868.63
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 16vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 4
siblings : 4
core id : 0
cpu cores : 4
cpu cores : 4
apicid : 64
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5866.64
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 17vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 4
siblings : 4
core id : 1
cpu cores : 4
cpu cores : 4
apicid : 66
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5867.55
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 18vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 4
siblings : 4
core id : 2
cpu cores : 4
cpu cores : 4
apicid : 68
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5867.31
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 19vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 4
siblings : 4
core id : 3
cpu cores : 4
cpu cores : 4
apicid : 70
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5867.64
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 20vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 5
siblings : 4
core id : 0
cpu cores : 4
cpu cores : 4
apicid : 80
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5868.43
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 21vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 5
siblings : 4
core id : 1
cpu cores : 4
cpu cores : 4
apicid : 82
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5869.26
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 22vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 5
siblings : 4
core id : 2
cpu cores : 4
cpu cores : 4
apicid : 84
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5867.89
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 23vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 5
siblings : 4
core id : 3
cpu cores : 4
cpu cores : 4
apicid : 86
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5870.31
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 24vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 6
siblings : 4
core id : 0
cpu cores : 4
cpu cores : 4
apicid : 96
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5867.04
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 25vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 6
siblings : 4
core id : 1
cpu cores : 4
cpu cores : 4
apicid : 98
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5866.62
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 26vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 6
siblings : 4
core id : 2
cpu cores : 4
cpu cores : 4
apicid : 100
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5865.17
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 27vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 6
siblings : 4
core id : 3
cpu cores : 4
cpu cores : 4
apicid : 102
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5864.31
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 28vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 7
siblings : 4
core id : 0
cpu cores : 4
cpu cores : 4
apicid : 112
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5867.86
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 29vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 7
siblings : 4
core id : 1
cpu cores : 4
cpu cores : 4
apicid : 114
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5868.98
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 30vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 7
siblings : 4
core id : 2
cpu cores : 4
cpu cores : 4
apicid : 116
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5869.39
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 31vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 7
siblings : 4
core id : 3
cpu cores : 4
cpu cores : 4
apicid : 118
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5866.46
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 32vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 8
siblings : 4
core id : 0
cpu cores : 4
cpu cores : 4
apicid : 128
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5868.26
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 33vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 8
siblings : 4
core id : 1
cpu cores : 4
cpu cores : 4
apicid : 130
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5867.83
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 34vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 8
siblings : 4
core id : 2
cpu cores : 4
cpu cores : 4
apicid : 132
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5866.41
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 35vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 8
siblings : 4
core id : 3
cpu cores : 4
cpu cores : 4
apicid : 134
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5866.10
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 36vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 9
siblings : 4
core id : 0
cpu cores : 4
cpu cores : 4
apicid : 144
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5867.94
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 37vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 9
siblings : 4
core id : 1
cpu cores : 4
cpu cores : 4
apicid : 146
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5868.10
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 38vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 9
siblings : 4
core id : 2
cpu cores : 4
cpu cores : 4
apicid : 148
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5868.16
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 39vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 9
siblings : 4
core id : 3
cpu cores : 4
cpu cores : 4
apicid : 150
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5870.76
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 40vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 10
siblings : 4
core id : 0
cpu cores : 4
cpu cores : 4
apicid : 160
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5867.80
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 41vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 10
siblings : 4
core id : 1
cpu cores : 4
cpu cores : 4
apicid : 162
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5868.70
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 42vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 10
siblings : 4
core id : 2
cpu cores : 4
cpu cores : 4
apicid : 164
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5868.25
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 43vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 10
siblings : 4
core id : 3
cpu cores : 4
cpu cores : 4
apicid : 166
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5866.27
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 44vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 11
siblings : 4
core id : 0
cpu cores : 4
cpu cores : 4
apicid : 176
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5869.09
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 45vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 11
siblings : 4
core id : 1
cpu cores : 4
cpu cores : 4
apicid : 178
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5869.89
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 46vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 11
siblings : 4
core id : 2
cpu cores : 4
cpu cores : 4
apicid : 180
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5871.30
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 47vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 11
siblings : 4
core id : 3
cpu cores : 4
cpu cores : 4
apicid : 182
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5869.35
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 48vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 12
siblings : 4
core id : 0
cpu cores : 4
cpu cores : 4
apicid : 192
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5868.68
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 49vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 12
siblings : 4
core id : 1
cpu cores : 4
cpu cores : 4
apicid : 194
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5867.43
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 50vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 12
siblings : 4
core id : 2
cpu cores : 4
cpu cores : 4
apicid : 196
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5865.60
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 51vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 12
siblings : 4
core id : 3
cpu cores : 4
cpu cores : 4
apicid : 198
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5866.01
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 52vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 13
siblings : 4
core id : 0
cpu cores : 4
cpu cores : 4
apicid : 208
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5868.82
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 53vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 13
siblings : 4
core id : 1
cpu cores : 4
cpu cores : 4
apicid : 210
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5869.69
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 54vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 13
siblings : 4
core id : 2
cpu cores : 4
cpu cores : 4
apicid : 212
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5867.73
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 55vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 13
siblings : 4
core id : 3
cpu cores : 4
cpu cores : 4
apicid : 214
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5866.55
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 56vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 14
siblings : 4
core id : 0
cpu cores : 4
cpu cores : 4
apicid : 224
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5868.81
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 57vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 14
siblings : 4
core id : 1
cpu cores : 4
cpu cores : 4
apicid : 226
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5869.90
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 58vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 14
siblings : 4
core id : 2
cpu cores : 4
cpu cores : 4
apicid : 228
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5870.19
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 59vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 14
siblings : 4
core id : 3
cpu cores : 4
cpu cores : 4
apicid : 230
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5870.07
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 60vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 15
siblings : 4
core id : 0
cpu cores : 4
cpu cores : 4
apicid : 240
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5869.63
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 61vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 15
siblings : 4
core id : 1
cpu cores : 4
cpu cores : 4
apicid : 242
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5870.38
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 62vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 15
siblings : 4
core id : 2
cpu cores : 4
cpu cores : 4
apicid : 244
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5870.10
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 63vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz
stepping : 5
cpu MHz : 2926.327
cache size : 8192 KB
physical id : 15
siblings : 4
core id : 3
cpu cores : 4
cpu cores : 4
apicid : 246
fpu : yesfpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc pni ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips : 5870.64
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
Quoted 7 lines · Ravikiran G Thirumalai
Quoted 4 lines · Yinghai Lu
Quoted 1 line · Yinghai Lu
Ravikiran G Thirumalai wrote:
can you check if this one fix your problem?
Nope it doesn't :(
then there is other path
Quoted 8 lines
Quoted 4 lines
BTW: what does your
/proc/cpuinfoprint about apic id and initial apic id?Here's a sample cpuinfo from an older kernel (attached)
why there is two lines cores?
also we should have initial apic id field...
YH
Quoted 11 lines · Yinghai Lu
Quoted 10 lines · Ravikiran G Thirumalai
Signed-off-by Ravikiran Thirumalai <[email protected]>Cc Yinghai Lu <[email protected]>Index:
linux-2.6.31-rc6/arch/x86/kernel/apic/apic_flat_64.c
===================================================================arch/x86/kernel/apic/apic_flat_64.c +0
-0@@ -161,7 +161,8 @@ static int flat_apic_id_registered(void)
Quoted 9 lines
arch/x86/kernel/apic/apic_flat_64.c static int flat_apic_id_registered(void) · line 166
Quoted 6 lines
arch/x86/kernel/apic/apic_flat_64.c +2
-1static int flat_phys_pkg_id (int initial_apic_id,int index_msb) { -return initial_apic_id >> index_msb; +return cpu_has_apic ?hard_smp_processor_id () >> index_msb : + initial_apic_id >> index_msb; }it seems we should use initial apic id here.
Why? The specs seem to indicate otherwise unless I am mistaken -- Intel systems programming guide, Vol 3A Part1, chapter 7 section 7.5.5 - Identifying Logical Processors in a MP system:
<quote>
After the BIOS has completed the MP initialization protocol, each logical processor can be uniquely identified by its local APIC ID. Software can access these APIC IDs in either of the following ways </quote>phys_pkg_id() indicates that the logical package id is being looked up, so local apic id should be used here no?
What am I missing?
Quoted 5 lines
can you check which calling for vsmp cause the problem?
arch/x86/kernel/cpu/addon_cpuid_features.c:c->cpu_core_id=apic->phys_pkg_id(c->initial_apicid, ht_mask_width)arch/x86/kernel/cpu/addon_cpuid_features.c:c->phys_proc_id= apic->phys_pkg_id(c->initial_apicid,core_plus_mask_width
);arch/x86/kernel/cpu/addon_cpuid_features.c:c->apicid=apic->phys_pkg_id(c->initial_apicid, 0);
The above are definitely problematic. Anyplace that uses phys_pkg_id to get the unique pkg id of a logical processor will have problems.
Quoted 20 lines · Ravikiran G Thirumalai
Quoted 10 lines · Yinghai Lu
Quoted 9 lines · Ravikiran G Thirumalai
Signed-off-by Ravikiran Thirumalai <[email protected]>Cc Yinghai Lu <[email protected]>Index:
linux-2.6.31-rc6/arch/x86/kernel/apic/apic_flat_64.c
===================================================================arch/x86/kernel/apic/apic_flat_64.c +0
-0@@ -161,7 +161,8 @@ static int flat_apic_id_registered(void)Quoted 8 lines
arch/x86/kernel/apic/apic_flat_64.c static int flat_apic_id_registered(void) · line 166
Quoted 6 lines
arch/x86/kernel/apic/apic_flat_64.c +2
-1static int flat_phys_pkg_id (int initial_apic_id,int index_msb) { -return initial_apic_id >> index_msb; +return cpu_has_apic ?hard_smp_processor_id () >> index_msb : + initial_apic_id >> index_msb; }it seems we should use initial apic id here.
Quoted 11 lines
Why? The specs seem to indicate otherwise unless I am mistaken -- Intel systems programming guide, Vol 3A Part1, chapter 7 section 7.5.5 - Identifying Logical Processors in a MP system:
<quote>
After the BIOS has completed the MP initialization protocol, each logical processor can be uniquely identified by its local APIC ID. Software can access these APIC IDs in either of the following ways </quote>phys_pkg_id()indicates that the logical package id is being looked up, so local apic id should be used here no?
What am I missing?
initial apic id : it can not changed, there is fixed mapping from that to physical processor id aka socket id / node id.
apic id: could be changed by BIOS to any value. there is no good way to get phys_pkg_id from that.
Quoted 10 lines
Quoted 7 lines
can you check which calling for vsmp cause the problem?
arch/x86/kernel/cpu/addon_cpuid_features.c:c->cpu_core_id=apic->phys_pkg_id(c->initial_apicid, ht_mask_width)arch/x86/kernel/cpu/addon_cpuid_features.c:c->phys_proc_id= apic->phys_pkg_id(c->initial_apicid,core_plus_mask_width
);arch/x86/kernel/cpu/addon_cpuid_features.c:c->apicid=apic->phys_pkg_id(c->initial_apicid, 0);The above are definitely problematic. Anyplace that uses
phys_pkg_idto get the unique pkg id of a logical processor will have problems.
that is from intel new cpuid(0x0b) leaf, that is with initial_apicid according to intel EDS.
we need to figure out your initial apic id. and find out good way for phys_pkg_id. and may need to modify apic->phys_pkg_id member according to dmi info.
YH
Quoted 20 lines · Yinghai Lu
Quoted 7 lines · Ravikiran G Thirumalai
Quoted 5 lines · Yinghai Lu
Quoted 3 lines · Ravikiran G Thirumalai
Signed-off-by Ravikiran Thirumalai <[email protected]>Cc Yinghai Lu <[email protected]>Quoted 11 lines
Why? The specs seem to indicate otherwise unless I am mistaken -- Intel systems programming guide, Vol 3A Part1, chapter 7 section 7.5.5 - Identifying Logical Processors in a MP system:
<quote>
After the BIOS has completed the MP initialization protocol, each logical processor can be uniquely identified by its local APIC ID. Software can access these APIC IDs in either of the following ways </quote>phys_pkg_id()indicates that the logical package id is being looked up, so local apic id should be used here no?
What am I missing?
Quoted 4 lines
initial apic id : it can not changed, there is fixed mapping from that to physical processor id aka socket id / node id.
apic id: could be changed by BIOS to any value. there is no good way to get
phys_pkg_idfrom that.
But BIOS is supposed to change it to a sane value. Until 2.6.30, local apic id has been used to get phys_pkg_id for the 'flat' apics! What changed? Was this changed for a BIOS bug? Even the intel books seem to indicate local apic usage!
Quoted 23 lines · Ravikiran G Thirumalai
Quoted 22 lines · Yinghai Lu
Quoted 7 lines · Ravikiran G Thirumalai
Quoted 5 lines · Yinghai Lu
Quoted 3 lines · Ravikiran G Thirumalai
Signed-off-by Ravikiran Thirumalai <[email protected]>Cc Yinghai Lu <[email protected]>Quoted 11 lines
Why? The specs seem to indicate otherwise unless I am mistaken -- Intel systems programming guide, Vol 3A Part1, chapter 7 section 7.5.5 - Identifying Logical Processors in a MP system:
<quote>
After the BIOS has completed the MP initialization protocol, each logical processor can be uniquely identified by its local APIC ID. Software can access these APIC IDs in either of the following ways </quote>phys_pkg_id()indicates that the logical package id is being looked up, so local apic id should be used here no?
What am I missing?initial apic id : it can not changed, there is fixed mapping from that to physical processor id aka socket id / node id.
Quoted 7 lines
Quoted 2 lines
apic id: could be changed by BIOS to any value. there is no good way to get
phys_pkg_idfrom that.But BIOS is supposed to change it to a sane value. Until 2.6.30, local apic id has been used to get
phys_pkg_idfor the 'flat' apics! What changed? Was this changed for a BIOS bug? Even the intel books seem to indicate local apic usage!
We should revert to the .30 behavior unless there's a good reason (even in that case we'll solve the regression and do a workaround for vSMP). Yinghai?
Ingo
[Ingo Molnar - Tue, Aug 25, 2009 at 08:15:00PM +0200]
| | * Ravikiran G Thirumalai <[email protected]> wrote: | | > On Mon, Aug 24, 2009 at 10:12:01PM -0700, Yinghai Lu wrote: | > >Ravikiran G Thirumalai wrote: | > >> On Mon, Aug 24, 2009 at 04:53:45PM -0700, Yinghai Lu wrote: | > >>> Ravikiran G Thirumalai wrote: | > >>>> Signed-off-by: Ravikiran Thirumalai <[email protected]> | > >>>> Cc: Yinghai Lu <[email protected]> | > >>>> | > >>> | > >> | > >> Why? The specs seem to indicate otherwise unless I am mistaken -- | > >> Intel systems programming guide, Vol 3A Part1, chapter 7 section | > >> 7.5.5 - Identifying Logical Processors in a MP system: | > >> <quote> | > >> After the BIOS has completed the MP initialization protocol, each logical | > >> processor can be uniquely identified by its local APIC ID. Software can | > >> access these APIC IDs in either of the following ways | > >> </quote> | > >> phys_pkg_id() indicates that the logical package id is being looked up, | > >> so local apic id should be used here no? | > >> What am I missing? | > > | > >initial apic id : it can not changed, there is fixed mapping from that to physical processor id aka socket id / node id. | > > | > >apic id: could be changed by BIOS to any value. there is no good way to get phys_pkg_id from that. | > > | > | > But BIOS is supposed to change it to a sane value. Until 2.6.30, | > local apic id has been used to get phys_pkg_id for the 'flat' | > apics! What changed? Was this changed for a BIOS bug? Even the | > intel books seem to indicate local apic usage! | | We should revert to the .30 behavior unless there's a good reason | (even in that case we'll solve the regression and do a workaround | for vSMP). Yinghai? | | Ingo
I'm definitely not APIC expert but since I was partially involved letme turn in.
Original commit which causes problem for vSMP seems to be due to cpu_has_apic bit turned off (ie due to being manually disabled or acpi table broken) so further read apic id will return plain zero (we're talking about 64 bits now). So frnakly I don't understand what is wrong with Ravikiran's patch. In case of apic disabled initial apic value will be used anyway (which is latched but actually may be changed, but it's not our case).
Or perhaps there is an issue in srat numa nodes numbering?
-- Cyrill
Quoted 44 lines · Cyrill Gorcunov
[Ingo Molnar - Tue, Aug 25, 2009 at 08:15:00PM +0200]
| | * Ravikiran G Thirumalai <[email protected]> wrote: | | > On Mon, Aug 24, 2009 at 10:12:01PM -0700, Yinghai Lu wrote: | > >Ravikiran G Thirumalai wrote: | > >> On Mon, Aug 24, 2009 at 04:53:45PM -0700, Yinghai Lu wrote: | > >>> Ravikiran G Thirumalai wrote: | > >>>> Signed-off-by: Ravikiran Thirumalai <[email protected]> | > >>>> Cc: Yinghai Lu <[email protected]> | > >>>> | > >>> | > >> | > >> Why? The specs seem to indicate otherwise unless I am mistaken -- | > >> Intel systems programming guide, Vol 3A Part1, chapter 7 section | > >> 7.5.5 - Identifying Logical Processors in a MP system: | > >> <quote> | > >> After the BIOS has completed the MP initialization protocol, each logical | > >> processor can be uniquely identified by its local APIC ID. Software can | > >> access these APIC IDs in either of the following ways | > >> </quote> | > >> phys_pkg_id() indicates that the logical package id is being looked up, | > >> so local apic id should be used here no? | > >> What am I missing? | > > | > >initial apic id : it can not changed, there is fixed mapping from that to physical processor id aka socket id / node id. | > > | > >apic id: could be changed by BIOS to any value. there is no good way to get phys_pkg_id from that. | > > | > | > But BIOS is supposed to change it to a sane value. Until 2.6.30, | > local apic id has been used to get phys_pkg_id for the 'flat' | > apics! What changed? Was this changed for a BIOS bug? Even the | > intel books seem to indicate local apic usage! | | We should revert to the .30 behavior unless there's a good reason | (even in that case we'll solve the regression and do a workaround | for vSMP). Yinghai? | | IngoI'm definitely not APIC expert but since I was partially involved letme turn in.
Quoted 7 lines
Original commit which causes problem for vSMP seems to be due to
cpu_has_apicbit turned off (ie due to being manually disabled or acpi table broken) so further read apic id will return plain zero (we're talking about 64 bits now). So frnakly I don't understand what is wrong with Ravikiran's patch. In case of apic disabled initial apic value will be used anyway (which is latched but actually may be changed, but it's not our case).
initial apic id and apic id could be different.
and we should use initial apic id to get correct phys pkg id in case BIOS set crazy apic id.
YH
Quoted 4 lines · Yinghai Lu
initial apic id and apic id could be different.
and we should use initial apic id to get correct phys pkg id in case BIOS set crazy apic id.
Yinghai - I think you missed Cyrills' point. Let me repeat it:
"cpu_has_apic bit turned off"
there's no apic. No "initial apic id". No "phys pkg id". No nothing.
Discussions about "correct phys pkg id" are pointless.
Linus
Quoted 7 lines · Linus Torvalds
Quoted 5 lines · Yinghai Lu
initial apic id and apic id could be different.
and we should use initial apic id to get correct phys pkg id in case BIOS set crazy apic id.
Quoted 8 lines
Yinghai - I think you missed Cyrills' point. Let me repeat it:
"
cpu_has_apicbit turned off"there's no apic. No "initial apic id". No "phys pkg id". No nothing.
Discussions about "correct phys pkg id" are pointless.
that's not the case here though:
[ 8.713916] Total of 32 processors activated (162314.96 BogoMIPS).
so APICs are active. The real difference is i think this aspect of commit 2759c3287:
static int flat_phys_pkg_id(int initial_apic_id, int index_msb) {
- return hard_smp_processor_id() >> index_msb;
+ return initial_apic_id >> index_msb;
}
We need to revert back to .30 behavior here. (In case of which environment to trust we almost always trust whatever booted millions of Linux boxes in the past already.)
Furthermore, commit 2759c3287 did not declare any side-effects and clearly causes a side-effect on vSMP which apparently has an overlapping set of initial APIC ids.
Ravikiran, your patch does not do a clear revert of this bit though. If you do a plain revert of the line above alone, does that fix the problem too?
Ingo
Quoted 33 lines · Ingo Molnar
Quoted 5 lines · Linus Torvalds
Quoted 4 lines · Yinghai Lu
initial apic id and apic id could be different.
and we should use initial apic id to get correct phys pkg id in case BIOS set crazy apic id.
Quoted 8 lines
Yinghai - I think you missed Cyrills' point. Let me repeat it:
"
cpu_has_apicbit turned off"there's no apic. No "initial apic id". No "phys pkg id". No nothing.
Discussions about "correct phys pkg id" are pointless.
that's not the case here though:
[ 8.713916] Total of 32 processors activated (162314.96 BogoMIPS).so APICs are active. The real difference is i think this aspect of commit
2759c3287:static int
flat_phys_pkg_id(int initial_apic_id, int index_msb){(quoted diff) +1
-1- return hard_smp_processor_id() >> index_msb; + return initial_apic_id >> index_msb; }We need to revert back to .30 behavior here. (In case of which environment to trust we almost always trust whatever booted millions of Linux boxes in the past already.)
Quoted 7 lines
Furthermore, commit
2759c3287did not declare any side-effects and clearly causes a side-effect on vSMP which apparently has an overlapping set of initial APIC ids.Ravikiran, your patch does not do a clear revert of this bit though. If you do a plain revert of the line above alone, does that fix the problem too?
how about patch phys_pkg_id for vsmp?
arch/x86/kernel/apic/probe_64.c +10 -0
@@ -44,6 +44,11 @@ static struct apic *apic_probe[] __initdata = { NULL, }; +static int apicid_phys_pkg_id (int initial_apic_id,int index_msb) +{ +return hard_smp_processor_id () >> index_msb; +} +/* * Check the APIC IDs in bios_cpu_apicid and choose the APIC mode. */
@@ -69,6 +74,11 @@ void __init default_setup_apic_routing(void)printk (KERN_INFO "Setting APIC routing to %s\n" , apic->name); } +if (is_vsmp_box ()) { +/* need to update phys_pkg_id */ + apic->phys_pkg_id = apicid_phys_pkg_id; + } +/* * Now that apic routing model is selected, configure the * fault handling for intr remapping.
Quoted 61 lines · Yinghai Lu
Quoted 33 lines · Ingo Molnar
Quoted 5 lines · Linus Torvalds
Quoted 4 lines · Yinghai Lu
initial apic id and apic id could be different.
and we should use initial apic id to get correct phys pkg id in case BIOS set crazy apic id.
Quoted 8 lines
Yinghai - I think you missed Cyrills' point. Let me repeat it:
"
cpu_has_apicbit turned off"there's no apic. No "initial apic id". No "phys pkg id". No nothing.
Discussions about "correct phys pkg id" are pointless.
that's not the case here though:
[ 8.713916] Total of 32 processors activated (162314.96 BogoMIPS).so APICs are active. The real difference is i think this aspect of commit
2759c3287:static int
flat_phys_pkg_id(int initial_apic_id, int index_msb){(quoted diff) +1
-1- return hard_smp_processor_id() >> index_msb; + return initial_apic_id >> index_msb; }We need to revert back to .30 behavior here. (In case of which environment to trust we almost always trust whatever booted millions of Linux boxes in the past already.)
Quoted 7 lines
Furthermore, commit
2759c3287did not declare any side-effects and clearly causes a side-effect on vSMP which apparently has an overlapping set of initial APIC ids.Ravikiran, your patch does not do a clear revert of this bit though. If you do a plain revert of the line above alone, does that fix the problem too?
how about patch
phys_pkg_idfor vsmp?arch/x86/kernel/apic/probe_64.c +5
-0@@ -44,6 +44,11 @@ static struct apic *apic_probe[] __initdata = { NULL, }; +static int apicid_phys_pkg_id (int initial_apic_id,int index_msb) +{ +return hard_smp_processor_id () >> index_msb; +} +/* * Check the APIC IDs in bios_cpu_apicid and choose the APIC mode. */
arch/x86/kernel/apic/probe_64.c void __init default_setup_apic_routing(void) · line 80
Quoted 7 lines
arch/x86/kernel/apic/probe_64.c +4
-0@@ -69,6 +74,11 @@ void __init default_setup_apic_routing(void)printk (KERN_INFO "Setting APIC routing to %s\n" , apic->name); } +if (is_vsmp_box ()) { +/* need to update phys_pkg_id */ + apic->phys_pkg_id = apicid_phys_pkg_id; + }
Hm, this is rather tempting simply because it only affects vSMP systems and we are in late -rc's. Ravikiran, does Yinghai's patch solve the crash for you too?
Ingo
Quoted 31 lines · Ingo Molnar
Quoted 31 lines
...
Quoted 7 lines
Furthermore, commit
2759c3287did not declare any side-effects and clearly causes a side-effect on vSMP which apparently has an overlapping set of initial APIC ids.Ravikiran, your patch does not do a clear revert of this bit though. If you do a plain revert of the line above alone, does that fix the problem too?
how about patch
phys_pkg_idfor vsmp?arch/x86/kernel/apic/probe_64.c +5
-0@@ -44,6 +44,11 @@ static struct apic *apic_probe[] __initdata = { NULL, }; +static int apicid_phys_pkg_id (int initial_apic_id,int index_msb) +{ +return hard_smp_processor_id () >> index_msb; +} +/* * Check the APIC IDs in bios_cpu_apicid and choose the APIC mode. */ @@ -69,6 +74,11 @@ void __init default_setup_apic_routing(void)printk (KERN_INFO "Setting APIC routing to %s\n" , apic->name); }
Quoted 9 lines
arch/x86/kernel/apic/probe_64.c void __init default_setup_apic_routing(void) · line 80
Quoted 4 lines
arch/x86/kernel/apic/probe_64.c +4
-0+if (is_vsmp_box ()) { +/* need to update phys_pkg_id */ + apic->phys_pkg_id = apicid_phys_pkg_id; + }Hm, this is rather tempting simply because it only affects vSMP systems and we are in late -rc's. Ravikiran, does Yinghai's patch solve the crash for you too?
Yes it does. Tested and confirmed.
Thanks,
Kiran
PATCH 1/1 x86: fix vsmp booting with phys_pkg_id changing
2.6.31-rc7 does not boot on vSMPowered systems.
[ 8.501108] CPU31: Thermal monitoring enabled (TM1) [ 8.501127] CPU 31 MCA banks SHD:2 SHD:3 SHD:5 SHD:6 SHD:8 [ 8.650254] CPU31: Intel(R) Xeon(R) CPU E5540 @ 2.53GHz stepping 04 [ 8.710324] Brought up 32 CPUs [ 8.713916] Total of 32 processors activated (162314.96 BogoMIPS). [ 8.721489] ERROR: parent span is not a superset of domain->span [ 8.727686] ERROR: domain->groups does not contain CPU0 [ 8.733091] ERROR: groups don't span domain->span [ 8.737975] ERROR: domain->cpu_power not set [ 8.742416]
bisected to
| | commit 2759c3287de27266e06f1f4e82cbd2d65f6a044c | x86: don't call read_apic_id if !cpu_has_apic |
need to use apic id in phys_pkg_id for vsmp...
--- arch/x86/kernel/apic/probe_64.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
arch/x86/kernel/apic/probe_64.c +10 -0
@@ -44,6 +44,11 @@ static struct apic *apic_probe[] __initd NULL, }; +static int apicid_phys_pkg_id (int initial_apic_id,int index_msb) +{ +return hard_smp_processor_id () >> index_msb; +} +/* * Check the APIC IDs in bios_cpu_apicid and choose the APIC mode. */
@@ -69,6 +74,11 @@ void __init default_setup_apic_routing(vprintk (KERN_INFO "Setting APIC routing to %s\n" , apic->name); } +if (is_vsmp_box ()) { +/* need to update phys_pkg_id */ + apic->phys_pkg_id = apicid_phys_pkg_id; + } +/* * Now that apic routing model is selected, configure the * fault handling for intr remapping.
Re: [PATCH] x86: fix vsmp booting with phys_pkg_id changing
Quoted 45 lines · Yinghai Lu
2.6.31-rc7 does not boot on vSMPowered systems.
[ 8.501108] CPU31: Thermal monitoring enabled (TM1) [ 8.501127] CPU 31 MCA banks SHD:2 SHD:3 SHD:5 SHD:6 SHD:8 [ 8.650254] CPU31: Intel(R) Xeon(R) CPU E5540 @ 2.53GHz stepping 04 [ 8.710324] Brought up 32 CPUs [ 8.713916] Total of 32 processors activated (162314.96 BogoMIPS). [ 8.721489] ERROR: parent span is not a superset of domain->span [ 8.727686] ERROR: domain->groups does not contain CPU0 [ 8.733091] ERROR: groups don't span domain->span [ 8.737975] ERROR: domain->cpu_power not set [ 8.742416]bisected to
| | commit 2759c3287de27266e06f1f4e82cbd2d65f6a044c | x86: don't call read_apic_id if !cpu_has_apic |need to use apic id in
phys_pkg_idfor vsmp...Reported-and-Tested-by Ravikiran Thirumalai <[email protected]>Signed-off-by Yinghai Lu <[email protected]>--- arch/x86/kernel/apic/probe_64.c | 10 ++++++++++ 1 file changed, 10 insertions(+)Index:
linux-2.6/arch/x86/kernel/apic/probe_64.c
===================================================================arch/x86/kernel/apic/probe_64.c +5
-0@@ -44,6 +44,11 @@ static struct apic *apic_probe[] __initd NULL, }; +static int apicid_phys_pkg_id (int initial_apic_id,int index_msb) +{ +return hard_smp_processor_id () >> index_msb; +} +/* * Check the APIC IDs in bios_cpu_apicid and choose the APIC mode. */
arch/x86/kernel/apic/probe_64.c void __init default_setup_apic_routing(v · line 80
Quoted 7 lines
arch/x86/kernel/apic/probe_64.c +4
-0@@ -69,6 +74,11 @@ void __init default_setup_apic_routing(vprintk (KERN_INFO "Setting APIC routing to %s\n" , apic->name); } +if (is_vsmp_box ()) { +/* need to update phys_pkg_id */ + apic->phys_pkg_id = apicid_phys_pkg_id; + }
Ok, this looks like a rather straightforward quirk - and vSMP is indeed special in that it sets up overlapping APIC ids. It's also the smaller patch with basically zero cross section to other systems, so a lot more -rc7 worthy.
So i've applied it to x86/urgent (with small edits to the changelog). Any objections from anyone?
Note, should any other, real hardware show problems with , we might still have to consider reverting the original change in 2759c32. It's borderline.
Btw., the code is still not fully clean. For example this bit in generic_identify():
#ifdef CONFIG_X86_32
# ifdef CONFIG_X86_HT
c->apicid = apic->phys_pkg_id(c->initial_apicid, 0);
# else
c->apicid = c->initial_apicid;
# endif
#endifand in another place we have:
#ifdef CONFIG_X86_64
c->apicid = apic->phys_pkg_id(c->initial_apicid, 0);#endif
This ifdeffery could probably be removed and we could call ->phys_pkg_id() all the time, right?
Ingo
[tip:x86/urgent] x86: Fix vSMP boot crash
In reply to Yinghai LuCommit-ID: 295594e9cf6ae2efd73371777aa8feba0f87f42f Gitweb: http://git.kernel.org/tip/295594e9cf6ae2efd73371777aa8feba0f87f42f Author: Yinghai Lu <[email protected]> AuthorDate: Tue, 25 Aug 2009 13:44:44 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Wed, 26 Aug 2009 10:13:17 +0200
x86: Fix vSMP boot crash
2.6.31-rc7 does not boot on vSMP systems:
[ 8.501108] CPU31: Thermal monitoring enabled (TM1) [ 8.501127] CPU 31 MCA banks SHD:2 SHD:3 SHD:5 SHD:6 SHD:8 [ 8.650254] CPU31: Intel(R) Xeon(R) CPU E5540 @ 2.53GHz stepping 04 [ 8.710324] Brought up 32 CPUs [ 8.713916] Total of 32 processors activated (162314.96 BogoMIPS). [ 8.721489] ERROR: parent span is not a superset of domain->span [ 8.727686] ERROR: domain->groups does not contain CPU0 [ 8.733091] ERROR: groups don't span domain->span [ 8.737975] ERROR: domain->cpu_power not set [ 8.742416]
Ravikiran Thirumalai bisected it to:
| commit 2759c3287de27266e06f1f4e82cbd2d65f6a044c | x86: don't call read_apic_id if !cpu_has_apic
The problem is that on vSMP systems the CPUID derived initial-APICIDs are overlapping - so we need to fall back on hard_smp_processor_id() which reads the local APIC.
Both come from the hardware (influenced by firmware though) so it's a tough call which one to trust.
Doing the quirk expresses the vSMP property properly and also does not affect other systems, so we go for this solution instead of a revert.
LKML-Reference: <[email protected]>
--- arch/x86/kernel/apic/probe_64.c | 10 ++++++++++ 1 files changed, 10 insertions(+), 0 deletions(-)
arch/x86/kernel/apic/probe_64.c +10 -0
@@ -44,6 +44,11 @@ static struct apic *apic_probe[] __initdata = { NULL, }; +static int apicid_phys_pkg_id (int initial_apic_id,int index_msb) +{ +return hard_smp_processor_id () >> index_msb; +} +/* * Check the APIC IDs in bios_cpu_apicid and choose the APIC mode. */
@@ -69,6 +74,11 @@ void __init default_setup_apic_routing(void)printk (KERN_INFO "Setting APIC routing to %s\n" , apic->name); } +if (is_vsmp_box ()) { +/* need to update phys_pkg_id */ + apic->phys_pkg_id = apicid_phys_pkg_id; + } +/* * Now that apic routing model is selected, configure the * fault handling for intr remapping.
Quoted 4 lines · Ingo Molnar
* Linus Torvalds <[email protected]> wrote:
Quoted 8 lines
Furthermore, commit
2759c3287did not declare any side-effects and clearly causes a side-effect on vSMP which apparently has an overlapping set of initial APIC ids.Ravikiran, your patch does not do a clear revert of this bit though. If you do a plain revert of the line above alone, does that fix the problem too?
Yes it does. However, it probably reads local apic id when it shouldn't.
I am right now trying out the other patch Yinghai posted (based on is_vsmp_box()), and report results in a few.
[Yinghai Lu - Tue, Aug 25, 2009 at 11:50:56AM -0700]
...
| > | We should revert to the .30 behavior unless there's a good reason | > | (even in that case we'll solve the regression and do a workaround | > | for vSMP). Yinghai? | > | | > | Ingo | > | > I'm definitely not APIC expert but since I was partially involved | > letme turn in. | > | > Original commit which causes problem for vSMP seems to be due | > to cpu_has_apic bit turned off (ie due to being manually disabled | > or acpi table broken) so further read apic id will return plain | > zero (we're talking about 64 bits now). So frnakly I don't understand | > what is wrong with Ravikiran's patch. In case of apic disabled | > initial apic value will be used anyway (which is latched but | > actually may be changed, but it's not our case). | | initial apic id and apic id could be different.
yes, I know. initial one is latched by pins though still could be changed (not always).
|
| and we should use initial apic id to get correct phys pkg id in case BIOS set crazy apic id.
indeed, as function name implies. Need more code reading...
So (64 bits) at least generic_identify is not guilty since it uses initial one from scratch. Need more code reading...
| | YH |
-- Cyrill
Quoted 6 lines · Yinghai Lu
Cyrill Gorcunov wrote:
initial apic id and apic id could be different.
and we should use initial apic id to get correct phys pkg id in case BIOS set crazy apic id.
Is there a specific bug we are fixing here? Again, this behavior was present from atleast 2.6.15-2.6.30 on 'flat' and older 'cluster' apics, and intel documentation suggests local apic be used. Is there a specific bios bug that is being fixed?
(The BIOS is not supposed to set a crazy apic ID. The BIOS is buggy in that case -- Quoting section 9.4.6 from Intel System programming guide vol 3A part 1:
<quote>
9.4.6 Local APIC ID
At power up, system hardware assigns a unique APIC ID to each local APIC on the system bus (for Pentium 4 and Intel Xeon processors) or on the APIC bus (for P6 family and Pentium processors). The hardware assigned APIC ID is based on system topology and includes encoding for socket position and cluster information (see Figure 7-2).
In MP systems, the local APIC ID is also used as a processor ID by the BIOS and the operating system....
</quote>)
Thanks,
Kiran
Quoted 8 lines · Ravikiran G Thirumalai
Quoted 6 lines · Yinghai Lu
Cyrill Gorcunov wrote:
initial apic id and apic id could be different.
and we should use initial apic id to get correct phys pkg id in case BIOS set crazy apic id.
Quoted 8 lines
Is there a specific bug we are fixing here? Again, this behavior was present from atleast 2.6.15-2.6.30 on 'flat' and older 'cluster' apics, and intel documentation suggests local apic be used. Is there a specific bios bug that is being fixed?
(The BIOS is not supposed to set a crazy apic ID. The BIOS is buggy in that case -- Quoting section 9.4.6 from Intel System programming guide vol 3A part 1:
That means very little. If Yinghai fixed a real bug in an existing system we should consider it. Yinghai?
Ingo
Quoted 8 lines · Ravikiran G Thirumalai
Quoted 6 lines · Yinghai Lu
Cyrill Gorcunov wrote:
initial apic id and apic id could be different.
and we should use initial apic id to get correct phys pkg id in case BIOS set crazy apic id.
Quoted 6 lines
Is there a specific bug we are fixing here? Again, this behavior was present from atleast 2.6.15-2.6.30 on 'flat' and older 'cluster' apics, and intel documentation suggests local apic be used. Is there a specific bios bug that is being fixed?
(The BIOS is not supposed to set a crazy apic ID. The BIOS is buggy in that
we can not stop BIOS doing crazy thing.
for AMD 8 socket 4 cores system. BSP apic id : 4 or 8, and initial apic id is 0:
with apic id in phys_pkg_id, will get pkg id: 1, 2, ...
with initial apic id will get pkg id: 0, 1, 2, 3.
when I was working on LinuxBIOS, I did have one porting to have
BSP: 0
AP: start from 5, 6, 7...
that will have strange in pkg_id.
YH
Quoted 6 lines · Cyrill Gorcunov
[Ingo Molnar - Tue, Aug 25, 2009 at 08:15:00PM +0200] |
I'm definitely not APIC expert but since I was partially involved letme turn in.
Quoted 8 lines
Original commit which causes problem for vSMP seems to be due to
cpu_has_apicbit turned off (ie due to being manually disabled or acpi table broken) so further read apic id will return plain zero (we're talking about 64 bits now). So frnakly I don't understand what is wrong with Ravikiran's patch. In case of apic disabled initial apic value will be used anyway (which is latched but actually may be changed, but it's not our case).
Exactly my thinking. I hoped the patch I posted solves both cases -- does not depend on local apic id for the "fix crash on certain UP configs" case in the commit here:
And fixes vsmp too.
Quoted 2 lines
Or perhaps there is an issue in srat numa nodes numbering?
Don't think so, local apic id has been used for for 'flat' and 'cluster' apic (which was used prior to 'flat') for atleast 15 major releases.
Cyrill/Yinghai, can you test and confirm if the patch attached does not regress the 'UP crash case' mentioned in
commit 2759c3287de27266e06f1f4e82cbd2d65f6a044c please?
---
2.6.31-rc7 does not boot on vSMPowered systems. The sched domains seem to build incorrectly with error messages of the sort:
[ 8.501108] CPU31: Thermal monitoring enabled (TM1) [ 8.501127] CPU 31 MCA banks SHD:2 SHD:3 SHD:5 SHD:6 SHD:8 [ 8.650254] CPU31: Intel(R) Xeon(R) CPU E5540 @ 2.53GHz stepping 04 [ 8.710324] Brought up 32 CPUs [ 8.713916] Total of 32 processors activated (162314.96 BogoMIPS). [ 8.721489] ERROR: parent span is not a superset of domain->span [ 8.727686] ERROR: domain->groups does not contain CPU0 [ 8.733091] ERROR: groups don't span domain->span [ 8.737975] ERROR: domain->cpu_power not set [ 8.742416]
This is followed by oopsen in the scheduler code with NULL pointer deference in find_busiest_group.
Git bisection pointed to the following commit:
commit 2759c3287de27266e06f1f4e82cbd2d65f6a044c
x86: don't call read_apic_id if !cpu_has_apic
Upon examining the history of the commit, the above commit seems to be a fix for:
commit 4797f6b021a3fa399942245d07a1feb30df81bb8
x86: read apic ID in the !acpi_lapic case
However, there appears to be bug in the commit 2759c3287de27266e06f1f4e82cbd2d65f6a044c, where flat_phys_pkg_id uses initial apic id instead of hard_smp_processor_id() on SMP machines.
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=2759c3287de27266e06f1f4e82cbd2d65f6a044c
This patch fixes the bug and causes vSMPowered systems to boot up correctly,
arch/x86/kernel/apic/apic_flat_64.c +2 -1
@@ -161,7 +161,8 @@ static int flat_apic_id_registered(void)static int flat_phys_pkg_id (int initial_apic_id,int index_msb) { -return initial_apic_id >> index_msb; +return cpu_has_apic ?hard_smp_processor_id () >> index_msb : + initial_apic_id >> index_msb; }struct apic apic_flat = {
Quoted 16 lines · Ravikiran G Thirumalai
Quoted 6 lines · Cyrill Gorcunov
[Ingo Molnar - Tue, Aug 25, 2009 at 08:15:00PM +0200] |
I'm definitely not APIC expert but since I was partially involved letme turn in.
Quoted 8 lines
Original commit which causes problem for vSMP seems to be due to
cpu_has_apicbit turned off (ie due to being manually disabled or acpi table broken) so further read apic id will return plain zero (we're talking about 64 bits now). So frnakly I don't understand what is wrong with Ravikiran's patch. In case of apic disabled initial apic value will be used anyway (which is latched but actually may be changed, but it's not our case).
Quoted 7 lines
Exactly my thinking. I hoped the patch I posted solves both cases -- does not depend on local apic id for the "fix crash on certain UP configs" case in the commit here:
And fixes vsmp too.
Sidenote for .32: i am somewhat worried about the spreading of various cpu_has_apic checks. We should separate it out more cleanly and provide an apic-> callback instead and avoid this ugly flaggery.
Ingo
[Ravikiran G Thirumalai - Tue, Aug 25, 2009 at 12:20:55PM -0700]
| On Tue, Aug 25, 2009 at 10:31:30PM +0400, Cyrill Gorcunov wrote: | >[Ingo Molnar - Tue, Aug 25, 2009 at 08:15:00PM +0200] | >| | > | >I'm definitely not APIC expert but since I was partially involved | >letme turn in. | > | >Original commit which causes problem for vSMP seems to be due | >to cpu_has_apic bit turned off (ie due to being manually disabled | >or acpi table broken) so further read apic id will return plain | >zero (we're talking about 64 bits now). So frnakly I don't understand | >what is wrong with Ravikiran's patch. In case of apic disabled | >initial apic value will be used anyway (which is latched but | >actually may be changed, but it's not our case). | > | | Exactly my thinking. I hoped the patch I posted solves both cases -- | does not depend on local apic id for the "fix crash on certain UP configs" | case in the commit here: | | http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=2759c3287de27266e06f1f4e82cbd2d65f6a044c | | And fixes vsmp too. | | >Or perhaps there is an issue in srat numa nodes numbering? | > | | Don't think so, local apic id has been used for for 'flat' and 'cluster' | apic (which was used prior to 'flat') for atleast 15 major releases. | | Cyrill/Yinghai, can you test and confirm if the patch attached does not | regress the 'UP crash case' mentioned in | commit 2759c3287de27266e06f1f4e82cbd2d65f6a044c please? | | --- |
I would like to but I can;t. I don;t even know where were we hanging at this crash case. I was only reading commit changelog and since it was said about "not apic bit set" I thought about disabled apic case.
Personally I like much more quirk Yinghai posted which leaves phys id function _bared_ and simple as it should be :)
And since it's too late before kernel release -- introducing read apic back seems to be dangerous I suppose: there are a number of configs which could be having side effect. At moment they are tested _with_ Yinghai patch and only vSMP is issued. Though the problem need to be investigated more closely. But I'm not knowing that much in this area.
-- Cyrill
Quoted 24 lines · Ingo Molnar
Quoted 22 lines · Ravikiran G Thirumalai
Quoted 21 lines · Yinghai Lu
Quoted 5 lines · Ravikiran G Thirumalai
Quoted 4 lines · Yinghai Lu
Quoted 3 lines · Ravikiran G Thirumalai
Signed-off-by Ravikiran Thirumalai <[email protected]>Cc Yinghai Lu <[email protected]>Quoted 11 lines
Why? The specs seem to indicate otherwise unless I am mistaken -- Intel systems programming guide, Vol 3A Part1, chapter 7 section 7.5.5 - Identifying Logical Processors in a MP system:
<quote>
After the BIOS has completed the MP initialization protocol, each logical processor can be uniquely identified by its local APIC ID. Software can access these APIC IDs in either of the following ways </quote>phys_pkg_id()indicates that the logical package id is being looked up, so local apic id should be used here no?
What am I missing?initial apic id : it can not changed, there is fixed mapping from that to physical processor id aka socket id / node id.
apic id: could be changed by BIOS to any value. there is no good way to get
phys_pkg_idfrom that.
Quoted 8 lines
Quoted 4 lines
But BIOS is supposed to change it to a sane value. Until 2.6.30, local apic id has been used to get
phys_pkg_idfor the 'flat' apics! What changed? Was this changed for a BIOS bug? Even the intel books seem to indicate local apic usage!We should revert to the .30 behavior unless there's a good reason (even in that case we'll solve the regression and do a workaround for vSMP). Yinghai?
we should stop overloading phys_pkg_id to get physical processor id (socket id ?) and apic id.
we need to find out what vSMP initial apic id looks like. and have one better phys_pkg_id for it.
otherwise we have problem with intel system that have apic id changing by BIOS too.
YH