Skip to contentSkip to conversation

a non producible kernel panic

DISCUSSION Alex Shi 1 participant 2 messages Feb 5, 2012 – Feb 6, 2012

lore
Share & tools
lore

Alex Shi

author
#0 lore raw

When do performance testing for rc2 kernel on our NHM EX machine( 4 sockets * 8 cores), I find a kernel panic. but never reproduced it. and didn't find some useful info after enable lockdep, rcu debug etc.

Any ideas on this?

INFO: task umount:4280 blocked for more than 120 seconds.

"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
umount D ffff883025443198 0 4280 4235 0x00000080
ffff883027805d28 0000000000000086 ffff883025442de0 ffff883027805fd8 ffff883027805fd8 0000000000013100 ffff8810292eade0 ffff883025442de0 0000000000000018 7fffffffffffffff 7fffffffffffffff ffffffff81c180c8

Call Trace:
 [<ffffffff81452caa>] schedule+0x55/0x57
 [<ffffffff81451a93>] schedule_timeout+0x2f/0xd9
 [<ffffffff8109dfe0>] ? rcu_panic+0x12/0x12
 [<ffffffff81028259>] ? physflat_send_IPI_allbutself+0x12/0x14
 [<ffffffff81024a33>] ? native_send_call_func_ipi+0x71/0x91
 [<ffffffff81452b67>] wait_for_common+0x95/0xeb
 [<ffffffff8105dfe7>] ? try_to_wake_up+0x1e0/0x1e0
 [<ffffffff8109dfe0>] ? rcu_panic+0x12/0x12
 [<ffffffff8109eb41>] ? call_rcu_bh+0x12/0x12
 [<ffffffff81452c53>] wait_for_completion+0x18/0x1a
 [<ffffffff8109ef36>] _rcu_barrier+0x96/0xa6
 [<ffffffff8109ef5d>] rcu_barrier_sched+0x17/0x19
 [<ffffffff8109ef68>] rcu_barrier+0x9/0xb
 [<ffffffff8110bac0>] deactivate_locked_super+0x32/0x52
 [<ffffffff8110c536>] deactivate_super+0x32/0x36
 [<ffffffff81120691>] mntput_no_expire+0xde/0xe7
 [<ffffffff811212f2>] sys_umount+0x2cd/0x2ff
 [<ffffffff81112699>] ? path_put+0x1a/0x1e
 [<ffffffff81459a79>] system_call_fastpath+0x16/0x1b
INFO: task umount:4280 blocked for more than 120 seconds.

"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
umount D ffff883025443198 0 4280 4235 0x00000080
ffff883027805d28 0000000000000086 ffff883025442de0 ffff883027805fd8 ffff883027805fd8 0000000000013100 ffff8810292eade0 ffff883025442de0 0000000000000018 7fffffffffffffff 7fffffffffffffff ffffffff81c18

author
#1 lore raw
Quoted 3 lines · Alex Shi

When do performance testing for rc2 kernel on our NHM EX machine( 4 sockets * 8 cores), I find a kernel panic. but never reproduced it. and didn't find some useful info after enable lockdep, rcu debug etc.

BTW, the panic caused my machine hang.

Quoted 35 lines

Any ideas on this?

INFO: task umount:4280 blocked for more than 120 seconds.

"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
umount D ffff883025443198 0 4280 4235 0x00000080
ffff883027805d28 0000000000000086 ffff883025442de0 ffff883027805fd8 ffff883027805fd8 0000000000013100 ffff8810292eade0 ffff883025442de0 0000000000000018 7fffffffffffffff 7fffffffffffffff ffffffff81c180c8

Call Trace:
 [<ffffffff81452caa>] schedule+0x55/0x57
 [<ffffffff81451a93>] schedule_timeout+0x2f/0xd9
 [<ffffffff8109dfe0>] ? rcu_panic+0x12/0x12
 [<ffffffff81028259>] ? physflat_send_IPI_allbutself+0x12/0x14
 [<ffffffff81024a33>] ? native_send_call_func_ipi+0x71/0x91
 [<ffffffff81452b67>] wait_for_common+0x95/0xeb
 [<ffffffff8105dfe7>] ? try_to_wake_up+0x1e0/0x1e0
 [<ffffffff8109dfe0>] ? rcu_panic+0x12/0x12
 [<ffffffff8109eb41>] ? call_rcu_bh+0x12/0x12
 [<ffffffff81452c53>] wait_for_completion+0x18/0x1a
 [<ffffffff8109ef36>] _rcu_barrier+0x96/0xa6
 [<ffffffff8109ef5d>] rcu_barrier_sched+0x17/0x19
 [<ffffffff8109ef68>] rcu_barrier+0x9/0xb
 [<ffffffff8110bac0>] deactivate_locked_super+0x32/0x52
 [<ffffffff8110c536>] deactivate_super+0x32/0x36
 [<ffffffff81120691>] mntput_no_expire+0xde/0xe7
 [<ffffffff811212f2>] sys_umount+0x2cd/0x2ff
 [<ffffffff81112699>] ? path_put+0x1a/0x1e
 [<ffffffff81459a79>] system_call_fastpath+0x16/0x1b
INFO: task umount:4280 blocked for more than 120 seconds.

"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
umount D ffff883025443198 0 4280 4235 0x00000080
ffff883027805d28 0000000000000086 ffff883025442de0 ffff883027805fd8 ffff883027805fd8 0000000000013100 ffff8810292eade0 ffff883025442de0 0000000000000018 7fffffffffffffff 7fffffffffffffff ffffffff81c18