linux-next: WARNING: at kernel/lockdep.c:2680 check_flags+0x98/0x151()
In next-20080530 and next-20080602 (and possibly earlier - I can't remember the linux-next tree before that I tried) I get the following:
[ 12.885153] ------------[ cut here ]------------ [ 12.885203] WARNING: at kernel/lockdep.c:2680 check_flags+0x98/0x151() [ 12.885248] Pid: 4, comm: watchdog/0 Not tainted 2.6.26-rc4-next-20080602 #13 [ 12.885292] [ 12.885293] Call Trace: [ 12.885364] [<ffffffff8022bbd5>] warn_on_slowpath+0x58/0x8a [ 12.885410] [<ffffffff804c9cfe>] ? _spin_unlock_irqrestore+0x51/0x6d [ 12.885455] [<ffffffff8032ff41>] ? debug_locks_off+0x9/0x3c [ 12.885498] [<ffffffff802582dd>] ? ftrace_record_ip+0x1fa/0x272 [ 12.885542] [<ffffffff8025278a>] ? watchdog+0xc5/0x1ff [ 12.885584] [<ffffffff8020b2c0>] ? mcount_call+0x5/0x35 [ 12.885627] [<ffffffff8025278a>] ? watchdog+0xc5/0x1ff [ 12.885668] [<ffffffff80247c80>] check_flags+0x98/0x151 [ 12.885710] [<ffffffff8024ae72>] lock_acquire+0x4a/0xa9 [ 12.885753] [<ffffffff8025278a>] ? watchdog+0xc5/0x1ff [ 12.885795] [<ffffffff802526c5>] ? watchdog+0x0/0x1ff [ 12.885837] [<ffffffff804c98da>] _read_lock+0x37/0x43 [ 12.885879] [<ffffffff8025278a>] watchdog+0xc5/0x1ff [ 12.885921] [<ffffffff802526c5>] ? watchdog+0x0/0x1ff [ 12.885963] [<ffffffff8023e48b>] kthread+0x4e/0x7b [ 12.886005] [<ffffffff8020bf18>] child_rip+0xa/0x12 [ 12.886046] [<ffffffff80227d8f>] ? finish_task_switch+0x57/0x92 [ 12.886090] [<ffffffff804c9d55>] ? _spin_unlock_irq+0x3b/0x57 [ 12.886133] [<ffffffff8020bad3>] ? restore_args+0x0/0x30 [ 12.886137] [<ffffffff8023e43d>] ? kthread+0x0/0x7b [ 12.886137] [<ffffffff8020bf0e>] ? child_rip+0x0/0x12 [ 12.886137] [ 12.886137] ---[ end trace 60e7f098a6913839 ]--- [ 12.886137] possible reason: unannotated irqs-on. [ 12.886137] irq event stamp: 20 [ 12.886137] hardirqs last enabled at (19): [<ffffffff80249cc3>] trace_hardirqs_on+0xd/0xf [ 12.886137] hardirqs last disabled at (20): [<ffffffff80248565>] trace_hardirqs_off+0xd/0xf [ 12.886137] softirqs last enabled at (0): [<ffffffff80229fef>] copy_process+0x2da/0x109e [ 12.886137] softirqs last disabled at (0): [<0000000000000000>] 0x0
Do I understand this correctly that something enabled irqs in a way that got around lockdep? I assume the problem is not in watchdog, just that the watchdog was the next thing to run that interacted with irqs and thus lockdep picked up the situation then?
Is there additional debugging I can do, given some instructions? If the cause is readily apparent to anyone, could they let me know (for my own interest) why it is apparent so that I can investigate things like this further next time?
This is completely reproducible on every boot - should I try to bisect it?
Signature · 1 line
Kevin Winchester
Quoted 7 lines · Kevin Winchester
In next-20080530 and next-20080602 (and possibly earlier - I can't remember the linux-next tree before that I tried) I get the following:
[ 12.885153] ------------[ cut here ]------------ [ 12.885203] WARNING: at kernel/lockdep.c:2680 check_flags+0x98/0x151() [ 12.885248] Pid: 4, comm: watchdog/0 Not tainted2.6.26-rc4-next-20080602 #13
I saw this and mentioned it too -
http://groups.google.co.uk/group/fa.linux.kernel/browse_thread/thread/77d4cedb7b27fa3b/fc66f2d849b42031?lnk=gst&q=watchdog+sitsofe#fc66f2d849b42031 .
Quoted 1 line
This is completely reproducible on every boot - should I try to bisect it?
Things that have been bisected seem far more likely to be looked at so if you have the stomach... If you do start bisection I can confirm that it is in next-20080529 . I can also mention that on my system the warning seems to go away if I boot with nmi_watchdog=2 .
Signature · 1 line
Sitsofe | http://sucs.org/~sits/
Quoted 47 lines · Kevin Winchester
In next-20080530 and next-20080602 (and possibly earlier - I can't remember the linux-next tree before that I tried) I get the following:
[ 12.885153] ------------[ cut here ]------------ [ 12.885203] WARNING: at kernel/lockdep.c:2680 check_flags+0x98/0x151() [ 12.885248] Pid: 4, comm: watchdog/0 Not tainted 2.6.26-rc4-next-20080602 #13 [ 12.885292] [ 12.885293] Call Trace: [ 12.885364] [<ffffffff8022bbd5>] warn_on_slowpath+0x58/0x8a [ 12.885410] [<ffffffff804c9cfe>] ? _spin_unlock_irqrestore+0x51/0x6d [ 12.885455] [<ffffffff8032ff41>] ? debug_locks_off+0x9/0x3c [ 12.885498] [<ffffffff802582dd>] ? ftrace_record_ip+0x1fa/0x272 [ 12.885542] [<ffffffff8025278a>] ? watchdog+0xc5/0x1ff [ 12.885584] [<ffffffff8020b2c0>] ? mcount_call+0x5/0x35 [ 12.885627] [<ffffffff8025278a>] ? watchdog+0xc5/0x1ff [ 12.885668] [<ffffffff80247c80>] check_flags+0x98/0x151 [ 12.885710] [<ffffffff8024ae72>] lock_acquire+0x4a/0xa9 [ 12.885753] [<ffffffff8025278a>] ? watchdog+0xc5/0x1ff [ 12.885795] [<ffffffff802526c5>] ? watchdog+0x0/0x1ff [ 12.885837] [<ffffffff804c98da>] _read_lock+0x37/0x43 [ 12.885879] [<ffffffff8025278a>] watchdog+0xc5/0x1ff [ 12.885921] [<ffffffff802526c5>] ? watchdog+0x0/0x1ff [ 12.885963] [<ffffffff8023e48b>] kthread+0x4e/0x7b [ 12.886005] [<ffffffff8020bf18>] child_rip+0xa/0x12 [ 12.886046] [<ffffffff80227d8f>] ? finish_task_switch+0x57/0x92 [ 12.886090] [<ffffffff804c9d55>] ? _spin_unlock_irq+0x3b/0x57 [ 12.886133] [<ffffffff8020bad3>] ? restore_args+0x0/0x30 [ 12.886137] [<ffffffff8023e43d>] ? kthread+0x0/0x7b [ 12.886137] [<ffffffff8020bf0e>] ? child_rip+0x0/0x12 [ 12.886137] [ 12.886137] ---[ end trace 60e7f098a6913839 ]--- [ 12.886137] possible reason: unannotated irqs-on. [ 12.886137] irq event stamp: 20 [ 12.886137] hardirqs last enabled at (19): [<ffffffff80249cc3>] trace_hardirqs_on+0xd/0xf [ 12.886137] hardirqs last disabled at (20): [<ffffffff80248565>] trace_hardirqs_off+0xd/0xf [ 12.886137] softirqs last enabled at (0): [<ffffffff80229fef>] copy_process+0x2da/0x109e [ 12.886137] softirqs last disabled at (0): [<0000000000000000>] 0x0Do I understand this correctly that something enabled irqs in a way that got around lockdep? I assume the problem is not in watchdog, just that the watchdog was the next thing to run that interacted with irqs and thus lockdep picked up the situation then?
Quoted 4 lines
Is there additional debugging I can do, given some instructions? If the cause is readily apparent to anyone, could they let me know (for my own interest) why it is apparent so that I can investigate things like this further next time?
You are correct - someone did sti but failed to call trace_hardirqs_on(). Frequently its possible to isolate the code from knowing the last recorded event:
[ 12.886137] irq event stamp: 20 [ 12.886137] hardirqs last disabled at (20): [<ffffffff80248565>] trace_hardirqs_off+0xd/0xf
However your compilation seems to have lost the caller IP:
void trace_hardirqs_off(void)
{
trace_hardirqs_off_caller(CALLER_ADDR0);
}EXPORT_SYMBOL(trace_hardirqs_off);
So that is of little help here. (Not sure how that happened, nor how you could fix that - perhaps turn on some debugging knobs like build with debug info etc..)
Quoted 1 line
This is completely reproducible on every boot - should I try to bisect it?
That is a possibility yes - if you can find the offending patch it should be relatively straight forward to find the offending sti.
Quoted 64 lines · Peter Zijlstra
Quoted 47 lines · Kevin Winchester
In next-20080530 and next-20080602 (and possibly earlier - I can't remember the linux-next tree before that I tried) I get the following:
[ 12.885153] ------------[ cut here ]------------ [ 12.885203] WARNING: at kernel/lockdep.c:2680 check_flags+0x98/0x151() [ 12.885248] Pid: 4, comm: watchdog/0 Not tainted 2.6.26-rc4-next-20080602 #13 [ 12.885292] [ 12.885293] Call Trace: [ 12.885364] [<ffffffff8022bbd5>] warn_on_slowpath+0x58/0x8a [ 12.885410] [<ffffffff804c9cfe>] ? _spin_unlock_irqrestore+0x51/0x6d [ 12.885455] [<ffffffff8032ff41>] ? debug_locks_off+0x9/0x3c [ 12.885498] [<ffffffff802582dd>] ? ftrace_record_ip+0x1fa/0x272 [ 12.885542] [<ffffffff8025278a>] ? watchdog+0xc5/0x1ff [ 12.885584] [<ffffffff8020b2c0>] ? mcount_call+0x5/0x35 [ 12.885627] [<ffffffff8025278a>] ? watchdog+0xc5/0x1ff [ 12.885668] [<ffffffff80247c80>] check_flags+0x98/0x151 [ 12.885710] [<ffffffff8024ae72>] lock_acquire+0x4a/0xa9 [ 12.885753] [<ffffffff8025278a>] ? watchdog+0xc5/0x1ff [ 12.885795] [<ffffffff802526c5>] ? watchdog+0x0/0x1ff [ 12.885837] [<ffffffff804c98da>] _read_lock+0x37/0x43 [ 12.885879] [<ffffffff8025278a>] watchdog+0xc5/0x1ff [ 12.885921] [<ffffffff802526c5>] ? watchdog+0x0/0x1ff [ 12.885963] [<ffffffff8023e48b>] kthread+0x4e/0x7b [ 12.886005] [<ffffffff8020bf18>] child_rip+0xa/0x12 [ 12.886046] [<ffffffff80227d8f>] ? finish_task_switch+0x57/0x92 [ 12.886090] [<ffffffff804c9d55>] ? _spin_unlock_irq+0x3b/0x57 [ 12.886133] [<ffffffff8020bad3>] ? restore_args+0x0/0x30 [ 12.886137] [<ffffffff8023e43d>] ? kthread+0x0/0x7b [ 12.886137] [<ffffffff8020bf0e>] ? child_rip+0x0/0x12 [ 12.886137] [ 12.886137] ---[ end trace 60e7f098a6913839 ]--- [ 12.886137] possible reason: unannotated irqs-on. [ 12.886137] irq event stamp: 20 [ 12.886137] hardirqs last enabled at (19): [<ffffffff80249cc3>] trace_hardirqs_on+0xd/0xf [ 12.886137] hardirqs last disabled at (20): [<ffffffff80248565>] trace_hardirqs_off+0xd/0xf [ 12.886137] softirqs last enabled at (0): [<ffffffff80229fef>] copy_process+0x2da/0x109e [ 12.886137] softirqs last disabled at (0): [<0000000000000000>] 0x0Do I understand this correctly that something enabled irqs in a way that got around lockdep? I assume the problem is not in watchdog, just that the watchdog was the next thing to run that interacted with irqs and thus lockdep picked up the situation then?
Quoted 4 lines
Is there additional debugging I can do, given some instructions? If the cause is readily apparent to anyone, could they let me know (for my own interest) why it is apparent so that I can investigate things like this further next time?
You are correct - someone did sti but failed to call
trace_hardirqs_on(). Frequently its possible to isolate the code from knowing the last recorded event:[ 12.886137] irq event stamp: 20 [ 12.886137] hardirqs last disabled at (20): [<ffffffff80248565>] trace_hardirqs_off+0xd/0xfHowever your compilation seems to have lost the caller IP:
void
trace_hardirqs_off(void)
Quoted 9 lines
{ trace_hardirqs_off_caller(CALLER_ADDR0); }
EXPORT_SYMBOL(trace_hardirqs_off);So that is of little help here. (Not sure how that happened, nor how you could fix that - perhaps turn on some debugging knobs like build with debug info etc..)
Could this be related to CONFIG_CC_OPTIMIZE_INLINING (or whatever it is called now)? I am pretty sure I have that enabled. I will check the rest of my config this evening to see if there is anything else I can turn on/off to help.
Quoted 6 lines
Quoted 2 lines
This is completely reproducible on every boot - should I try to bisect it?
That is a possibility yes - if you can find the offending patch it should be relatively straight forward to find the offending sti.
Sure - I can try it this evening. Also, is calling sti the only way this could have happened? And is linux-next broken out into a single patch? I wouldn't expect there to be too many calls to sti, so a grep in the patch file might be quicker than bisection.
Thanks,
Signature · 1 line
Kevin Winchester
Hi,
Quoted 83 lines · Kevin Winchester
Quoted 64 lines · Peter Zijlstra
Quoted 47 lines · Kevin Winchester
In next-20080530 and next-20080602 (and possibly earlier - I can't remember the linux-next tree before that I tried) I get the following:
[ 12.885153] ------------[ cut here ]------------ [ 12.885203] WARNING: at kernel/lockdep.c:2680 check_flags+0x98/0x151() [ 12.885248] Pid: 4, comm: watchdog/0 Not tainted 2.6.26-rc4-next-20080602 #13 [ 12.885292] [ 12.885293] Call Trace: [ 12.885364] [<ffffffff8022bbd5>] warn_on_slowpath+0x58/0x8a [ 12.885410] [<ffffffff804c9cfe>] ? _spin_unlock_irqrestore+0x51/0x6d [ 12.885455] [<ffffffff8032ff41>] ? debug_locks_off+0x9/0x3c [ 12.885498] [<ffffffff802582dd>] ? ftrace_record_ip+0x1fa/0x272 [ 12.885542] [<ffffffff8025278a>] ? watchdog+0xc5/0x1ff [ 12.885584] [<ffffffff8020b2c0>] ? mcount_call+0x5/0x35 [ 12.885627] [<ffffffff8025278a>] ? watchdog+0xc5/0x1ff [ 12.885668] [<ffffffff80247c80>] check_flags+0x98/0x151 [ 12.885710] [<ffffffff8024ae72>] lock_acquire+0x4a/0xa9 [ 12.885753] [<ffffffff8025278a>] ? watchdog+0xc5/0x1ff [ 12.885795] [<ffffffff802526c5>] ? watchdog+0x0/0x1ff [ 12.885837] [<ffffffff804c98da>] _read_lock+0x37/0x43 [ 12.885879] [<ffffffff8025278a>] watchdog+0xc5/0x1ff [ 12.885921] [<ffffffff802526c5>] ? watchdog+0x0/0x1ff [ 12.885963] [<ffffffff8023e48b>] kthread+0x4e/0x7b [ 12.886005] [<ffffffff8020bf18>] child_rip+0xa/0x12 [ 12.886046] [<ffffffff80227d8f>] ? finish_task_switch+0x57/0x92 [ 12.886090] [<ffffffff804c9d55>] ? _spin_unlock_irq+0x3b/0x57 [ 12.886133] [<ffffffff8020bad3>] ? restore_args+0x0/0x30 [ 12.886137] [<ffffffff8023e43d>] ? kthread+0x0/0x7b [ 12.886137] [<ffffffff8020bf0e>] ? child_rip+0x0/0x12 [ 12.886137] [ 12.886137] ---[ end trace 60e7f098a6913839 ]--- [ 12.886137] possible reason: unannotated irqs-on. [ 12.886137] irq event stamp: 20 [ 12.886137] hardirqs last enabled at (19): [<ffffffff80249cc3>] trace_hardirqs_on+0xd/0xf [ 12.886137] hardirqs last disabled at (20): [<ffffffff80248565>] trace_hardirqs_off+0xd/0xf [ 12.886137] softirqs last enabled at (0): [<ffffffff80229fef>] copy_process+0x2da/0x109e [ 12.886137] softirqs last disabled at (0): [<0000000000000000>] 0x0Do I understand this correctly that something enabled irqs in a way that got around lockdep? I assume the problem is not in watchdog, just that the watchdog was the next thing to run that interacted with irqs and thus lockdep picked up the situation then?
Quoted 4 lines
Is there additional debugging I can do, given some instructions? If the cause is readily apparent to anyone, could they let me know (for my own interest) why it is apparent so that I can investigate things like this further next time?
You are correct - someone did sti but failed to call
trace_hardirqs_on(). Frequently its possible to isolate the code from knowing the last recorded event:[ 12.886137] irq event stamp: 20 [ 12.886137] hardirqs last disabled at (20): [<ffffffff80248565>] trace_hardirqs_off+0xd/0xfHowever your compilation seems to have lost the caller IP:
void
trace_hardirqs_off(void)Quoted 9 lines
{ trace_hardirqs_off_caller(CALLER_ADDR0); }
EXPORT_SYMBOL(trace_hardirqs_off);So that is of little help here. (Not sure how that happened, nor how you could fix that - perhaps turn on some debugging knobs like build with debug info etc..)
Could this be related to
CONFIG_CC_OPTIMIZE_INLINING(or whatever it is called now)? I am pretty sure I have that enabled. I will check the rest of my config this evening to see if there is anything else I can turn on/off to help.Quoted 3 lines
Quoted 2 lines
This is completely reproducible on every boot - should I try to bisect it?
Quoted 8 lines
Quoted 3 lines
That is a possibility yes - if you can find the offending patch it should be relatively straight forward to find the offending sti.
Sure - I can try it this evening. Also, is calling sti the only way this could have happened? And is linux-next broken out into a single patch? I wouldn't expect there to be too many calls to sti, so a grep in the patch file might be quicker than bisection.
git-log -p origin/master..next/master showed nothing interesting. Grepping for sti() would also not find indirect calls.
Hannes
Quoted 91 lines · Johannes Weiner
Hi,
Quoted 87 lines · Kevin Winchester
Quoted 64 lines · Peter Zijlstra
Quoted 47 lines · Kevin Winchester
In next-20080530 and next-20080602 (and possibly earlier - I can't remember the linux-next tree before that I tried) I get the following:
[ 12.885153] ------------[ cut here ]------------ [ 12.885203] WARNING: at kernel/lockdep.c:2680 check_flags+0x98/0x151() [ 12.885248] Pid: 4, comm: watchdog/0 Not tainted 2.6.26-rc4-next-20080602 #13 [ 12.885292] [ 12.885293] Call Trace: [ 12.885364] [<ffffffff8022bbd5>] warn_on_slowpath+0x58/0x8a [ 12.885410] [<ffffffff804c9cfe>] ? _spin_unlock_irqrestore+0x51/0x6d [ 12.885455] [<ffffffff8032ff41>] ? debug_locks_off+0x9/0x3c [ 12.885498] [<ffffffff802582dd>] ? ftrace_record_ip+0x1fa/0x272 [ 12.885542] [<ffffffff8025278a>] ? watchdog+0xc5/0x1ff [ 12.885584] [<ffffffff8020b2c0>] ? mcount_call+0x5/0x35 [ 12.885627] [<ffffffff8025278a>] ? watchdog+0xc5/0x1ff [ 12.885668] [<ffffffff80247c80>] check_flags+0x98/0x151 [ 12.885710] [<ffffffff8024ae72>] lock_acquire+0x4a/0xa9 [ 12.885753] [<ffffffff8025278a>] ? watchdog+0xc5/0x1ff [ 12.885795] [<ffffffff802526c5>] ? watchdog+0x0/0x1ff [ 12.885837] [<ffffffff804c98da>] _read_lock+0x37/0x43 [ 12.885879] [<ffffffff8025278a>] watchdog+0xc5/0x1ff [ 12.885921] [<ffffffff802526c5>] ? watchdog+0x0/0x1ff [ 12.885963] [<ffffffff8023e48b>] kthread+0x4e/0x7b [ 12.886005] [<ffffffff8020bf18>] child_rip+0xa/0x12 [ 12.886046] [<ffffffff80227d8f>] ? finish_task_switch+0x57/0x92 [ 12.886090] [<ffffffff804c9d55>] ? _spin_unlock_irq+0x3b/0x57 [ 12.886133] [<ffffffff8020bad3>] ? restore_args+0x0/0x30 [ 12.886137] [<ffffffff8023e43d>] ? kthread+0x0/0x7b [ 12.886137] [<ffffffff8020bf0e>] ? child_rip+0x0/0x12 [ 12.886137] [ 12.886137] ---[ end trace 60e7f098a6913839 ]--- [ 12.886137] possible reason: unannotated irqs-on. [ 12.886137] irq event stamp: 20 [ 12.886137] hardirqs last enabled at (19): [<ffffffff80249cc3>] trace_hardirqs_on+0xd/0xf [ 12.886137] hardirqs last disabled at (20): [<ffffffff80248565>] trace_hardirqs_off+0xd/0xf [ 12.886137] softirqs last enabled at (0): [<ffffffff80229fef>] copy_process+0x2da/0x109e [ 12.886137] softirqs last disabled at (0): [<0000000000000000>] 0x0Do I understand this correctly that something enabled irqs in a way that got around lockdep? I assume the problem is not in watchdog, just that the watchdog was the next thing to run that interacted with irqs and thus lockdep picked up the situation then?
Quoted 4 lines
Is there additional debugging I can do, given some instructions? If the cause is readily apparent to anyone, could they let me know (for my own interest) why it is apparent so that I can investigate things like this further next time?
You are correct - someone did sti but failed to call
trace_hardirqs_on(). Frequently its possible to isolate the code from knowing the last recorded event:[ 12.886137] irq event stamp: 20 [ 12.886137] hardirqs last disabled at (20): [<ffffffff80248565>] trace_hardirqs_off+0xd/0xfHowever your compilation seems to have lost the caller IP:
void
trace_hardirqs_off(void)Quoted 9 lines
{ trace_hardirqs_off_caller(CALLER_ADDR0); }
EXPORT_SYMBOL(trace_hardirqs_off);So that is of little help here. (Not sure how that happened, nor how you could fix that - perhaps turn on some debugging knobs like build with debug info etc..)
Could this be related to
CONFIG_CC_OPTIMIZE_INLINING(or whatever it is called now)? I am pretty sure I have that enabled. I will check the rest of my config this evening to see if there is anything else I can turn on/off to help.Quoted 6 lines
Quoted 2 lines
This is completely reproducible on every boot - should I try to bisect it?
That is a possibility yes - if you can find the offending patch it should be relatively straight forward to find the offending sti.
Quoted 7 lines
Quoted 4 lines
Sure - I can try it this evening. Also, is calling sti the only way this could have happened? And is linux-next broken out into a single patch? I wouldn't expect there to be too many calls to sti, so a grep in the patch file might be quicker than bisection.
git-log -p origin/master..next/master showed nothing interesting. Grepping for
sti()would also not find indirect calls.
Yeah - except you'd miss all the (inline) asm. Not sure there are many calls that implicity enable interrupts, but a popf can also enable them.
Re: [X86][BISECTED] WARNING: at kernel/lockdep.c:2680
In reply to Kevin WinchesterQuoted 7 lines · Kevin Winchester
Quoted 7 lines · Kevin Winchester
Quoted 6 lines
In next-20080530 and next-20080602 (and possibly earlier - I can't remember the linux-next tree before that I tried) I get the following:
[ 12.885153] ------------[ cut here ]------------ [ 12.885203] WARNING: at kernel/lockdep.c:2680[ check_flags+0x98/0x151() 12.885248] Pid: 4, comm: watchdog/0 Not
OK I've managed to bisect this down to commit [c6531cce6e6e4b99bcda46b6268d6f2d9e30aea4] (sched: do not trace sched_clock):
commit c6531cce6e6e4b99bcda46b6268d6f2d9e30aea4
Author: Ingo Molnar <[email protected]> Date: Mon May 12 21:21:14 2008 +0200
sched: do not trace sched_clock
The tracer uses sched_clock, so do not trace it.
Reverting this made the lockdep warning go away. Here is the bisection log:
# bad: [cb689015ce306647ea9e00c2412b852b1dd1580d] Add linux-next specific files
# good: [5ffeee1785c30303007b3d2dce85402b0bbf9b00] Add linux-next specific files
git-bisect start 'next-20080527' 'next-20080526'
# bad: [357e7b54308101878f7e93e04fa8d7f06eea57a8] Merge
commit 'avr32/avr32-arch'
git-bisect bad 357e7b54308101878f7e93e04fa8d7f06eea57a8
# bad: [048f32bfd577d3c0c2c5f2e4b03690a2902be1c2] Merge
branch 'tracing/mmiotrace' into auto-ftrace-next
git-bisect bad 048f32bfd577d3c0c2c5f2e4b03690a2902be1c2
# good: [4d9493c90f8e6e1b164aede3814010a290161abb] ftrace: remove add-hoc code
git-bisect good 4d9493c90f8e6e1b164aede3814010a290161abb
# good: [37135677e653537ffc6e7def679443272a1c03c3] ftrace: fix mcount export bug
git-bisect good 37135677e653537ffc6e7def679443272a1c03c3
# good: [801a175bf601f9a9d5e86e92dee9adeeb6625da8] mmiotrace: ftrace fix git-bisect good 801a175bf601f9a9d5e86e92dee9adeeb6625da8
# good: [e0fd5c2fa188311667267c02a702ae699a9fc2bd] mmiotrace: do not print bogus pid for maps either
git-bisect good e0fd5c2fa188311667267c02a702ae699a9fc2bd
# good: [790e2a290b499b0400254e6870ec27969065d122] x86 mmiotrace: page level is unsigned
git-bisect good 790e2a290b499b0400254e6870ec27969065d122
# good: [7b3ce4a410ccdc8ca0be361d4d4fd61c3830f52a] sched: clean up scheduler markers
git-bisect good 7b3ce4a410ccdc8ca0be361d4d4fd61c3830f52a
# bad: [7fa13a1e8501b96ff11d4a19a5401f61ccd3d98c] Merge
branch 'tracing/ftrace' into auto-ftrace-next
git-bisect bad 7fa13a1e8501b96ff11d4a19a5401f61ccd3d98c
# bad: [c6531cce6e6e4b99bcda46b6268d6f2d9e30aea4] sched: do not trace sched_clock
git-bisect bad c6531cce6e6e4b99bcda46b6268d6f2d9e30aea4
Signature · 1 line
Sitsofe | http://sucs.org/~sits/
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [X86][BISECTED] WARNING: at kernel/lockdep.c:2680
Quoted 58 lines · Ingo Molnar
Quoted 6 lines · Sitsofe Wheeler
OK I've managed to bisect this down to commit [
c6531cce6e6e4b99bcda46b6268d6f2d9e30aea4] (sched: do not tracesched_clock):commit
c6531cce6e6e4b99bcda46b6268d6f2d9e30aea4Author: Ingo Molnar <[email protected]> Date: Mon May 12 21:21:14 2008 +0200Quoted 8 lines
sched: do not trace
sched_clockThe tracer uses
sched_clock, so do not trace it.Signed-off-by Ingo Molnar <[email protected]>Signed-off-by Thomas Gleixner <[email protected]>Reverting this made the lockdep warning go away. Here is the bisection log:
ah, good find! Does the patch below (instead of the full revert) fix it as well? I've queued the fix up into tip/tracing/ftrace.
Ingo
---------------->
Subject: sched:sched_clock()lockdep fix
From: Ingo Molnar <[email protected]>
Date: Thu Jun 05 15:04:17 CEST 2008Sitsofe Wheeler reported a lockdep warning and bisected it down to:
Quoted 5 lines
commit
c6531cce6e6e4b99bcda46b6268d6f2d9e30aea4Author: Ingo Molnar <[email protected]> Date: Mon May 12 21:21:14 2008 +0200sched: do not trace
sched_clockdo not use raw irq flags in
cpu_clock()as it causes lockdep to lose track of the true state of the IRQ flag.Reported-and-bisected-by Sitsofe Wheeler <[email protected]>Signed-off-by Ingo Molnar <[email protected]>--- kernel/sched.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)Index:
linux/kernel/sched.c
===================================================================kernel/sched.c +1
-1@@ -862,7 +862,7 @@ unsigned long long notrace cpu_clock(intunsigned long long prev_cpu_time, time, delta_time;unsigned long flags; -raw_local_irq_save (flags); +local_irq_save (flags); prev_cpu_time =per_cpu (prev_cpu_time, cpu); time =__cpu_clock (cpu) +per_cpu (time_offset, cpu); delta_time = time-prev_cpu_time;
kernel/sched.c unsigned long long notrace cpu_clock(int · line 877
Quoted 8 lines
kernel/sched.c +1
-1@@ -871,7 +871,7 @@ unsigned long long notrace cpu_clock(int time =__sync_cpu_clock (time, cpu);per_cpu (prev_cpu_time, cpu) = time; } -raw_local_irq_restore (flags); +local_irq_restore (flags);return time; }
I didn't manage to get through a bisection run for the warning - thanks to Sitsofe for doing it. The patch does eliminate the warning for me.
Thanks,
Signature · 1 line
Kevin Winchester
Quoted 8 lines · Kevin Winchester
In next-20080530 and next-20080602 (and possibly earlier - I can't remember the linux-next tree before that I tried) I get the following:
[ 12.885153] ------------[ cut here ]------------ [ 12.885203] WARNING: at kernel/lockdep.c:2680 check_flags+0x98/0x151() [ 12.885248] Pid: 4, comm: watchdog/0 Not taintedThis is completely reproducible on every boot - should I try to bisect it?
If you are going to bisect it seems to me that the problem seems to have occurred between next-20080526 and next-20080527...
Signature · 1 line
Sitsofe | http://sucs.org/~sits/