Skip to contentSkip to conversation

tty ldisc lockups in linux-next

DISCUSSION Sasha Levin 3 participants 5 messages Sep 25, 2012

lore
Share & tools
lore
#0 lore raw

Hi all,

While fuzzing with trinity in a KVM tools guest running linux-next kernel, I keep hitting the following lockup:

[  842.780242] INFO: task init:1 blocked for more than 120 seconds.
[  842.780732] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  842.781559] init            D ffff88000d5b0000  3344     1      0 0x00000002
[  842.783226]  ffff88000d5adc28 0000000000000082 ffff88000d5adbe8 ffffffff81150ac5
[  842.784714]  ffff88000d5adfd8 ffff88000d5adfd8 ffff88000d5adfd8 ffff88000d5adfd8
[  842.785737]  ffffffff84e2e420 ffff88000d5b0000 ffff88000d5b08f0 7fffffffffffffff
[  842.786764] Call Trace:
[  842.787102]  [<ffffffff81150ac5>] ? sched_clock_local+0x25/0xa0
[  842.787858]  [<ffffffff83a0be45>] schedule+0x55/0x60
[  842.788511]  [<ffffffff83a09dd5>] schedule_timeout+0x45/0x360
[  842.789251]  [<ffffffff83a0d54d>] ? _raw_spin_unlock_irqrestore+0x5d/0xb0
[  842.790149]  [<ffffffff8117b13d>] ? trace_hardirqs_on+0xd/0x10
[  842.790594]  [<ffffffff83a0d574>] ? _raw_spin_unlock_irqrestore+0x84/0xb0
[  842.791096]  [<ffffffff81137af7>] ? prepare_to_wait+0x77/0x90
[  842.791535]  [<ffffffff81b9b2c6>] tty_ldisc_wait_idle.isra.7+0x76/0xb0
[  842.792016]  [<ffffffff81137cd0>] ? abort_exclusive_wait+0xb0/0xb0
[  842.792490]  [<ffffffff81b9c03b>] tty_ldisc_hangup+0x1cb/0x320
[  842.792924]  [<ffffffff81b933a2>] ? __tty_hangup+0x122/0x430
[  842.793364]  [<ffffffff81b933aa>] __tty_hangup+0x12a/0x430
[  842.794077]  [<ffffffff83a0d574>] ? _raw_spin_unlock_irqrestore+0x84/0xb0
[  842.794942]  [<ffffffff81b955cc>] disassociate_ctty+0x6c/0x230
[  842.795693]  [<ffffffff8110e7e8>] do_exit+0x3d8/0xa90
[  842.796361]  [<ffffffff83a0e4d9>] ? retint_swapgs+0x13/0x1b
[  842.797079]  [<ffffffff8110ef64>] do_group_exit+0x84/0xd0
[  842.797818]  [<ffffffff8110efc2>] sys_exit_group+0x12/0x20
[  842.798524]  [<ffffffff83a0edcd>] system_call_fastpath+0x1a/0x1f
[  842.799294] 1 lock held by init/1:
[  842.799734]  #0:  (&tty->ldisc_mutex){+.+.+.}, at: [<ffffffff81b9bf92>] tty_ldisc_hangup+0x122/0x320

Thanks,
Sasha

#1 lore raw
Quoted 3 lines · Sasha Levin

Hi all,

While fuzzing with trinity in a KVM tools guest running linux-next kernel, I keep hitting the following lockup:

Hi, I'm confused here. Is this different to what you reported a couple days ago? Doesn't reverting aa3c8af86382 help in the end?

Quoted 28 lines
[  842.780242] INFO: task init:1 blocked for more than 120 seconds.
[  842.780732] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  842.781559] init            D ffff88000d5b0000  3344     1      0 0x00000002
[  842.783226]  ffff88000d5adc28 0000000000000082 ffff88000d5adbe8 ffffffff81150ac5
[  842.784714]  ffff88000d5adfd8 ffff88000d5adfd8 ffff88000d5adfd8 ffff88000d5adfd8
[  842.785737]  ffffffff84e2e420 ffff88000d5b0000 ffff88000d5b08f0 7fffffffffffffff
[  842.786764] Call Trace:
[  842.787102]  [<ffffffff81150ac5>] ? sched_clock_local+0x25/0xa0
[  842.787858]  [<ffffffff83a0be45>] schedule+0x55/0x60
[  842.788511]  [<ffffffff83a09dd5>] schedule_timeout+0x45/0x360
[  842.789251]  [<ffffffff83a0d54d>] ? _raw_spin_unlock_irqrestore+0x5d/0xb0
[  842.790149]  [<ffffffff8117b13d>] ? trace_hardirqs_on+0xd/0x10
[  842.790594]  [<ffffffff83a0d574>] ? _raw_spin_unlock_irqrestore+0x84/0xb0
[  842.791096]  [<ffffffff81137af7>] ? prepare_to_wait+0x77/0x90
[  842.791535]  [<ffffffff81b9b2c6>] tty_ldisc_wait_idle.isra.7+0x76/0xb0
[  842.792016]  [<ffffffff81137cd0>] ? abort_exclusive_wait+0xb0/0xb0
[  842.792490]  [<ffffffff81b9c03b>] tty_ldisc_hangup+0x1cb/0x320
[  842.792924]  [<ffffffff81b933a2>] ? __tty_hangup+0x122/0x430
[  842.793364]  [<ffffffff81b933aa>] __tty_hangup+0x12a/0x430
[  842.794077]  [<ffffffff83a0d574>] ? _raw_spin_unlock_irqrestore+0x84/0xb0
[  842.794942]  [<ffffffff81b955cc>] disassociate_ctty+0x6c/0x230
[  842.795693]  [<ffffffff8110e7e8>] do_exit+0x3d8/0xa90
[  842.796361]  [<ffffffff83a0e4d9>] ? retint_swapgs+0x13/0x1b
[  842.797079]  [<ffffffff8110ef64>] do_group_exit+0x84/0xd0
[  842.797818]  [<ffffffff8110efc2>] sys_exit_group+0x12/0x20
[  842.798524]  [<ffffffff83a0edcd>] system_call_fastpath+0x1a/0x1f
[  842.799294] 1 lock held by init/1:
[  842.799734]  #0:  (&tty->ldisc_mutex){+.+.+.}, at: [<ffffffff81b9bf92>] tty_ldisc_hangup+0x122/0x320
Signature · 2 lines
js
suse labs
#2 lore raw
Quoted 5 lines · Jiri Slaby
Quoted 3 lines

Hi all,

Quoted 2 lines

While fuzzing with trinity in a KVM tools guest running linux-next kernel, I keep hitting the following lockup:

Hi, I'm confused here. Is this different to what you reported a couple days ago? Doesn't reverting aa3c8af86382 help in the end?

I was just about to send a reply to that mail saying that while reverting aa3c8af86382 reduces the odds for seeing it, it still happens. You were faster than me :)

But yes, it still happens even if I revert aa3c8af86382 or try applying your patch in that thread.

Thanks,
Sasha

#3 lore raw
Quoted 11 lines · Sasha Levin
Quoted 5 lines · Jiri Slaby
Quoted 3 lines

Hi all,

Quoted 2 lines

While fuzzing with trinity in a KVM tools guest running linux-next kernel, I keep hitting the following lockup:

Hi, I'm confused here. Is this different to what you reported a couple days ago? Doesn't reverting aa3c8af86382 help in the end?

I was just about to send a reply to that mail saying that while reverting aa3c8af86382 reduces the odds for seeing it, it still happens. You were faster than me :)

But yes, it still happens even if I revert aa3c8af86382 or try applying your patch in that thread.

The patch won't help, it's kind of certain.

Instead I still wonder what process sits on the terminal. Could you investigate?

Signature · 2 lines
js
suse labs
#4 lore raw
Quoted 11 lines · Jiri Slaby
Quoted 10 lines · Sasha Levin
Quoted 5 lines · Jiri Slaby
Quoted 3 lines

Hi all,

Quoted 2 lines

While fuzzing with trinity in a KVM tools guest running linux-next kernel, I keep hitting the following lockup:

Hi, I'm confused here. Is this different to what you reported a couple days ago? Doesn't reverting aa3c8af86382 help in the end?

I was just about to send a reply to that mail saying that while reverting aa3c8af86382 reduces the odds for seeing it, it still happens. You were faster than me :)

Quoted 7 lines
Quoted 1 line

But yes, it still happens even if I revert aa3c8af86382 or try applying your patch in that thread.

The patch won't help, it's kind of certain.

Instead I still wonder what process sits on the terminal. Could you investigate?

It looks like sh is trying to read:

[  606.950194] sh              S 0000000000000001  4800  6260      1 0x00000000
[  606.950194]  ffff88000c0ddcc8 0000000000000082 ffffffff847baa68 0000000000000b02
[  606.950194]  ffff88000c0ddfd8 ffff88000c0ddfd8 ffff88000c0ddfd8 ffff88000c0ddfd8
[  606.950194]  ffff88000f578000 ffff88000c0bb000 ffff88000c0ddd98 ffff880040b4d000
[  606.950194] Call Trace:
[  606.950194]  [<ffffffff83a0be45>] schedule+0x55/0x60
[  606.950194]  [<ffffffff83a09dd5>] schedule_timeout+0x45/0x360
[  606.950194]  [<ffffffff83a0d54d>] ? _raw_spin_unlock_irqrestore+0x5d/0xb0
[  606.950194]  [<ffffffff8117b13d>] ? trace_hardirqs_on+0xd/0x10
[  606.950194]  [<ffffffff83a0d574>] ? _raw_spin_unlock_irqrestore+0x84/0xb0
[  606.950194]  [<ffffffff81b98271>] n_tty_read+0x4c1/0x9a0
[  606.950194]  [<ffffffff83a0d54d>] ? _raw_spin_unlock_irqrestore+0x5d/0xb0
[  606.950194]  [<ffffffff8114d760>] ? try_to_wake_up+0x360/0x360
[  606.950194]  [<ffffffff81b922cf>] tty_read+0x8f/0x100
[  606.950194]  [<ffffffff8127187d>] vfs_read+0xad/0x180
[  606.950194]  [<ffffffff81271c10>] sys_read+0x50/0xa0
[  606.950194]  [<ffffffff83a0edcd>] system_call_fastpath+0x1a/0x1f

While init is trying to exit:

[  605.524940] init            D ffff88000d5b0000  3376     1      0 0x00000002
[  605.527502]  ffff88000d5adc28 0000000000000082 ffff88000d5adbe8 ffffffff81150ac5
[  605.529685]  ffff88000d5adfd8 ffff88000d5adfd8 ffff88000d5adfd8 ffff88000d5adfd8
[  605.530939]  ffff88000d613000 ffff88000d5b0000 ffff88000d5b08f0 7fffffffffffffff
[  605.532064] Call Trace:
[  605.532064]  [<ffffffff81150ac5>] ? sched_clock_local+0x25/0xa0
[  605.532064]  [<ffffffff83a0be45>] schedule+0x55/0x60
[  605.532064]  [<ffffffff83a09dd5>] schedule_timeout+0x45/0x360
[  605.532064]  [<ffffffff83a0d54d>] ? _raw_spin_unlock_irqrestore+0x5d/0xb0
[  605.532064]  [<ffffffff8117b13d>] ? trace_hardirqs_on+0xd/0x10
[  605.532064]  [<ffffffff83a0d574>] ? _raw_spin_unlock_irqrestore+0x84/0xb0
[  605.532064]  [<ffffffff81137af7>] ? prepare_to_wait+0x77/0x90
[  605.532064]  [<ffffffff81b9b2c6>] tty_ldisc_wait_idle.isra.7+0x76/0xb0
[  605.532064]  [<ffffffff81137cd0>] ? abort_exclusive_wait+0xb0/0xb0
[  605.532064]  [<ffffffff81b9c03b>] tty_ldisc_hangup+0x1cb/0x320
[  605.532064]  [<ffffffff81b933a2>] ? __tty_hangup+0x122/0x430
[  605.532064]  [<ffffffff81b933aa>] __tty_hangup+0x12a/0x430
[  605.532064]  [<ffffffff83a0d574>] ? _raw_spin_unlock_irqrestore+0x84/0xb0
[  605.532064]  [<ffffffff81b955cc>] disassociate_ctty+0x6c/0x230
[  605.532064]  [<ffffffff8110e7e8>] do_exit+0x3d8/0xa90
[  605.532064]  [<ffffffff83a0e4d9>] ? retint_swapgs+0x13/0x1b
[  605.532064]  [<ffffffff8110ef64>] do_group_exit+0x84/0xd0
[  605.532064]  [<ffffffff8110efc2>] sys_exit_group+0x12/0x20
[  605.532064]  [<ffffffff83a0edcd>] system_call_fastpath+0x1a/0x1f

And the corresponding lock info:

[  606.950194] Showing all locks held in the system:
[  606.950194] 1 lock held by init/1:
[  606.950194]  #0:  (&tty->ldisc_mutex){+.+.+.}, at: [<ffffffff81b9bf92>] tty_ldisc_hangup+0x122/0x320
[  606.950194] 1 lock held by sh/6260:
[  606.950194]  #0:  (&tty->atomic_read_lock){+.+...}, at: [<ffffffff81b98078>] n_tty_read+0x2c8/0x9a0