x86: support for new UV apic
Hey,
I am getting two following warnings (probable one per each CPU core) using the latest Linus's git:
WARNING: at arch/x86/kernel/genapic_64.c:86 read_apic_id+0x30/0x62() Modules linked in:
Pid: 1, comm: swapper Not tainted 2.6.25-03563-gf7288b1 #20
Call Trace: [<ffffffff8022f1e1>] warn_on_slowpath+0x51/0x63 [<ffffffff80282e48>] sys_ioctl+0x2d/0x5d [<ffffffff805185ff>] _spin_lock+0xe/0x24 [<ffffffff80227459>] task_rq_lock+0x3d/0x73 [<ffffffff805133c3>] set_cpu_sibling_map+0x336/0x350 [<ffffffff8021c1b8>] read_apic_id+0x30/0x62 [<ffffffff806d921d>] verify_local_APIC+0x90/0x138 [<ffffffff806d84b5>] native_smp_prepare_cpus+0x1f9/0x305 [<ffffffff806ce7b1>] kernel_init+0x59/0x2d9 [<ffffffff80518a26>] _spin_unlock_irq+0x11/0x2b [<ffffffff8020bf48>] child_rip+0xa/0x12 [<ffffffff806ce758>] kernel_init+0x0/0x2d9 [<ffffffff8020bf3e>] child_rip+0x0/0x12 ---[ end trace ca143223eefdc828 ]---
Few hours of bisecting gave me following commit as a suspect
ac23d4ee3f84de33c16ed7e68f9adee2386e74fb is first bad commit commit ac23d4ee3f84de33c16ed7e68f9adee2386e74fb
Author: Jack Steiner <[email protected]> Date: Fri Mar 28 14:12:16 2008 -0500
x86: support for new UV apic
Quoted 8 lines · Priit Laes
Hey,
I am getting two following warnings (probable one per each CPU core) using the latest Linus's git:
WARNING: at arch/x86/kernel/genapic_64.c:86 read_apic_id+0x30/0x62() Modules linked in:Pid: 1, comm: swapper Not tainted 2.6.25-03563-gf7288b1 #20
thanks - could you check the patch below, does it fix the warnings?
Ingo
--------------->
Subject: x86: disable preemption in native_smp_prepare_cpus
From: Ingo Molnar <[email protected]>
Date: Fri Apr 18 11:07:10 CEST 2008
Priit Laes reported the following warning:
Call Trace: [<ffffffff8022f1e1>] warn_on_slowpath+0x51/0x63 [<ffffffff80282e48>] sys_ioctl+0x2d/0x5d [<ffffffff805185ff>] _spin_lock+0xe/0x24 [<ffffffff80227459>] task_rq_lock+0x3d/0x73 [<ffffffff805133c3>] set_cpu_sibling_map+0x336/0x350 [<ffffffff8021c1b8>] read_apic_id+0x30/0x62 [<ffffffff806d921d>] verify_local_APIC+0x90/0x138 [<ffffffff806d84b5>] native_smp_prepare_cpus+0x1f9/0x305 [<ffffffff806ce7b1>] kernel_init+0x59/0x2d9 [<ffffffff80518a26>] _spin_unlock_irq+0x11/0x2b [<ffffffff8020bf48>] child_rip+0xa/0x12 [<ffffffff806ce758>] kernel_init+0x0/0x2d9 [<ffffffff8020bf3e>] child_rip+0x0/0x12
fix this by generally disabling preemption in native_smp_prepare_cpus().
--- arch/x86/kernel/smpboot.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
arch/x86/kernel/smpboot.c +4 -1
@@ -1181,6 +1181,7 @@ static void __init smp_cpu_index_default*/ void __init native_smp_prepare_cpus (unsigned int max_cpus) { +preempt_disable ();nmi_watchdog_default ();smp_cpu_index_default (); current_cpu_data = boot_cpu_data;
@@ -1197,7 +1198,7 @@ void __init native_smp_prepare_cpus(unsiif (smp_sanity_check (max_cpus) <0 ) {printk (KERN_INFO "SMP disabled\n" );disable_smp (); -return ; + goto out; }preempt_disable ();
@@ -1237,6 +1238,8 @@ void __init native_smp_prepare_cpus(unsiprintk (KERN_INFO "CPU%d: " ,0 );print_cpu_info (&cpu_data (0 ));setup_boot_clock (); +out: +preempt_enable (); }/* * Early setup to make printk work.
Quoted 4 lines · Ingo Molnar
Quoted 2 lines · Priit Laes
Hey,
Quoted 8 lines
Quoted 6 lines
I am getting two following warnings (probable one per each CPU core) using the latest Linus's git:
WARNING: at arch/x86/kernel/genapic_64.c:86 read_apic_id+0x30/0x62() Modules linked in:Pid: 1, comm: swapper Not tainted 2.6.25-03563-gf7288b1 #20
thanks - could you check the patch below, does it fix the warnings?
Hi Ingo ,
I got the same warning on my Core2Quad Q9300 box , your patch fixes the warning for me.
Quoted 56 lines
Ingo
--------------->
Subject: x86: disable preemption innative_smp_prepare_cpus
From: Ingo Molnar <[email protected]>
Date: Fri Apr 18 11:07:10 CEST 2008Priit Laes reported the following warning:
Call Trace: [<ffffffff8022f1e1>] warn_on_slowpath+0x51/0x63 [<ffffffff80282e48>] sys_ioctl+0x2d/0x5d [<ffffffff805185ff>] _spin_lock+0xe/0x24 [<ffffffff80227459>] task_rq_lock+0x3d/0x73 [<ffffffff805133c3>] set_cpu_sibling_map+0x336/0x350 [<ffffffff8021c1b8>] read_apic_id+0x30/0x62 [<ffffffff806d921d>] verify_local_APIC+0x90/0x138 [<ffffffff806d84b5>] native_smp_prepare_cpus+0x1f9/0x305 [<ffffffff806ce7b1>] kernel_init+0x59/0x2d9 [<ffffffff80518a26>] _spin_unlock_irq+0x11/0x2b [<ffffffff8020bf48>] child_rip+0xa/0x12 [<ffffffff806ce758>] kernel_init+0x0/0x2d9 [<ffffffff8020bf3e>] child_rip+0x0/0x12fix this by generally disabling preemption in
native_smp_prepare_cpus().Reported-and-bisected-by Priit Laes <[email protected]>Signed-off-by Ingo Molnar <[email protected]>Signed-off-by Thomas Gleixner <[email protected]>--- arch/x86/kernel/smpboot.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)Index:
linux-x86.q/arch/x86/kernel/smpboot.c
===================================================================arch/x86/kernel/smpboot.c +2
-1@@ -1181,6 +1181,7 @@ static void __init smp_cpu_index_default*/ void __init native_smp_prepare_cpus (unsigned int max_cpus) { +preempt_disable ();nmi_watchdog_default ();smp_cpu_index_default (); current_cpu_data = boot_cpu_data;@@ -1197,7 +1198,7 @@ void __init native_smp_prepare_cpus(unsiif (smp_sanity_check (max_cpus) <0 ) {printk (KERN_INFO "SMP disabled\n" );disable_smp (); -return ; + goto out; }preempt_disable ();
arch/x86/kernel/smpboot.c void __init native_smp_prepare_cpus(unsi · line 1245
Quoted 8 lines
arch/x86/kernel/smpboot.c +2
-0@@ -1237,6 +1238,8 @@ void __init native_smp_prepare_cpus(unsiprintk (KERN_INFO "CPU%d: " ,0 );print_cpu_info (&cpu_data (0 ));setup_boot_clock (); +out: +preempt_enable (); }/* * Early setup to make printk work.
Regards,
Gabriel
Ühel kenal päeval, R, 2008-04-25 kell 08:44, kirjutas Ingo Molnar:
Quoted 12 lines · Priit Laes
Quoted 9 lines
Hey,
I am getting two following warnings (probable one per each CPU core) using the latest Linus's git:
WARNING: at arch/x86/kernel/genapic_64.c:86 read_apic_id+0x30/0x62() Modules linked in:Pid: 1, comm: swapper Not tainted 2.6.25-03563-gf7288b1 #20
thanks - could you check the patch below, does it fix the warnings?
Works for me!
Thanks :)
--
Priit Laes
Quoted 4 lines · Ingo Molnar
Quoted 2 lines · Priit Laes
Hey,
Quoted 8 lines
Quoted 6 lines
I am getting two following warnings (probable one per each CPU core) using the latest Linus's git:
WARNING: at arch/x86/kernel/genapic_64.c:86 read_apic_id+0x30/0x62() Modules linked in:Pid: 1, comm: swapper Not tainted 2.6.25-03563-gf7288b1 #20
thanks - could you check the patch below, does it fix the warnings?
I think it would have been better to change the WARN_ON to
WARN_ON(preemptible() && num_online_cpus() > 1)
Preemption is actually ok here because this runs before the other CPUs are started
Alternative patch appended.
-Andi
---
Don't warn in read_apic_id() when preemptible but only one CPU online.
arch/x86/kernel/genapic_64.c +1 -1
@@ -83,7 +83,7 @@ unsigned int read_apic_id(void) {unsigned int id; -WARN_ON (preemptible ()); +WARN_ON (preemptible () &&num_online_cpus () >1 ); id =apic_read (APIC_ID );if (uv_system_type >=UV_X2APIC ) id |=__get_cpu_var (x2apic_extra_bits);
Quoted 9 lines · Andi Kleen
Quoted 7 lines · Ingo Molnar
Quoted 5 lines · Priit Laes
Hey,
I am getting two following warnings (probable one per each CPU core) using the latest Linus's git:
Quoted 8 lines
Quoted 4 lines
Quoted 3 lines
WARNING: at arch/x86/kernel/genapic_64.c:86 read_apic_id+0x30/0x62() Modules linked in:Pid: 1, comm: swapper Not tainted 2.6.25-03563-gf7288b1 #20
thanks - could you check the patch below, does it fix the warnings?
I think it would have been better to change the
WARN_ONto
WARN_ON(preemptible() && num_online_cpus() > 1)
Hmm , I think you are right , Ingo's patch fixes the warning on boot but now I get one when I reboot the box.
Quoted 5 lines
Preemption is actually ok here because this runs before the other CPUs are started
Alternative patch appended.
I will test your patch in some minutes.
Quoted 2 lines
-Andi
Gabriel
Quoted 20 lines
---
Don't warn in
read_apic_id()when preemptible but only one CPU online.Signed-off-by Andi Kleen <[email protected]>Index:
linux/arch/x86/kernel/genapic_64.c
===================================================================arch/x86/kernel/genapic_64.c +1
-1@@ -83,7 +83,7 @@ unsigned int read_apic_id(void) {unsigned int id; -WARN_ON (preemptible ()); +WARN_ON (preemptible () &&num_online_cpus () >1 ); id =apic_read (APIC_ID );if (uv_system_type >=UV_X2APIC ) id |=__get_cpu_var (x2apic_extra_bits);
Quoted 18 lines · Gabriel C
Quoted 9 lines · Andi Kleen
Quoted 7 lines · Ingo Molnar
Quoted 5 lines · Priit Laes
Hey,
I am getting two following warnings (probable one per each CPU core) using the latest Linus's git:
Quoted 7 lines
Quoted 4 lines
Quoted 3 lines
WARNING: at arch/x86/kernel/genapic_64.c:86 read_apic_id+0x30/0x62() Modules linked in:Pid: 1, comm: swapper Not tainted 2.6.25-03563-gf7288b1 #20
thanks - could you check the patch below, does it fix the warnings?
I think it would have been better to change the
WARN_ONto
WARN_ON(preemptible() && num_online_cpus() > 1)
Quoted 8 lines
Hmm , I think you are right , Ingo's patch fixes the warning on boot but now I get one when I reboot the box.
Quoted 4 lines
Preemption is actually ok here because this runs before the other CPUs are started
Alternative patch appended.
I will test your patch in some minutes.
Your patch works better , at least for my box , 4 reboots with your patch and boot/reboot warnings are all gone.
Quoted 20 lines
Quoted 18 lines
Don't warn in
read_apic_id()when preemptible but only one CPU online.Signed-off-by Andi Kleen <[email protected]>Index:
linux/arch/x86/kernel/genapic_64.c
===================================================================arch/x86/kernel/genapic_64.c +1
-1@@ -83,7 +83,7 @@ unsigned int read_apic_id(void) {unsigned int id; -WARN_ON (preemptible ()); +WARN_ON (preemptible () &&num_online_cpus () >1 ); id =apic_read (APIC_ID );if (uv_system_type >=UV_X2APIC ) id |=__get_cpu_var (x2apic_extra_bits);
Ühel kenal päeval, R, 2008-04-25 kell 11:45, kirjutas Andi Kleen:
Quoted 15 lines
Quoted 4 lines · Ingo Molnar
Quoted 2 lines · Priit Laes
Hey,
Quoted 8 lines
Quoted 6 lines
I am getting two following warnings (probable one per each CPU core) using the latest Linus's git:
WARNING: at arch/x86/kernel/genapic_64.c:86 read_apic_id+0x30/0x62() Modules linked in:Pid: 1, comm: swapper Not tainted 2.6.25-03563-gf7288b1 #20
thanks - could you check the patch below, does it fix the warnings?
Quoted 8 lines
I think it would have been better to change the
WARN_ONto
WARN_ON(preemptible() && num_online_cpus() > 1)Preemption is actually ok here because this runs before the other CPUs are started
Alternative patch appended.
Worksforme :)
--
Priit Laes
Ingo Molnar
AppliedQuoted 3 lines · Andi Kleen
Alternative patch appended.
Don't warn in
read_apic_id()when preemptible but only one CPU online.
applied, thanks Andi.
Ingo