linux-next: lockdep whinge in cgroup_rmdir

DISCUSSION [email protected] 7 participants 9 messages Jan 13, 2011 – Jan 14, 2011 2 messages not in the archive

Share & tools

[email protected]

author

2011-01-13 15:35 UTC #0 lore raw

Seen booting yesterday's linux-next, was not present in 2.6.37-rc7-mmotm1202.

Not sure if it's an selinux or cgroup issue, so I'm throwing it at every address I can find for either. This is easily replicatable and happens at every boot, so I can test patches if needed. Am willing to bisect it down if nobody knows right off the bat what the problem is.

The 'W' taint is from the already-reported kernel/workqueue.c worker_enter_idle issue.

[   85.100795] systemd[1]: readahead-replay.service: main process exited, code=exited, status=1
[   85.101530] 
[   85.101531] =============================================
[   85.101796] [ INFO: possible recursive locking detected ]
[   85.102002] 2.6.37-next-20110111 #1
[   85.102009] ---------------------------------------------
[   85.102009] systemd/1 is trying to acquire lock:
[   85.102009]  (&(&dentry->d_lock)->rlock){+.+...}, at: [<ffffffff8107ca5c>] cgroup_rmdir+0x339/0x479
[   85.102009] 
[   85.102009] but task is already holding lock:
[   85.102009]  (&(&dentry->d_lock)->rlock){+.+...}, at: [<ffffffff8107ca54>] cgroup_rmdir+0x331/0x479
[   85.102009] 
[   85.102009] other info that might help us debug this:
[   85.102009] 4 locks held by systemd/1:
[   85.102009]  #0:  (&sb->s_type->i_mutex_key#14/1){+.+.+.}, at: [<ffffffff810fea4d>] do_rmdir+0x7d/0x121
[   85.102009]  #1:  (&sb->s_type->i_mutex_key#14){+.+.+.}, at: [<ffffffff810fd4bc>] vfs_rmdir+0x4a/0xbe
[   85.102009]  #2:  (cgroup_mutex){+.+.+.}, at: [<ffffffff8107cb84>] cgroup_rmdir+0x461/0x479
[   85.102009]  #3:  (&(&dentry->d_lock)->rlock){+.+...}, at: [<ffffffff8107ca54>] cgroup_rmdir+0x331/0x479
[   85.102009] 
[   85.102009] stack backtrace:
[   85.102009] Pid: 1, comm: systemd Tainted: G        W   2.6.37-next-20110111 #1
[   85.102009] Call Trace:
[   85.102009]  [<ffffffff81069f22>] ? __lock_acquire+0x929/0xd4e
[   85.102009]  [<ffffffff8107c6f1>] ? cgroup_clear_directory+0xff/0x131
[   85.102009]  [<ffffffff8107c6f1>] ? cgroup_clear_directory+0xff/0x131
[   85.102009]  [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[   85.102009]  [<ffffffff8106a859>] ? lock_acquire+0x100/0x126
[   85.102009]  [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[   85.102009]  [<ffffffff815521ef>] ? sub_preempt_count+0x35/0x48
[   85.102009]  [<ffffffff8154e401>] ? _raw_spin_lock+0x36/0x45
[   85.102009]  [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[   85.102009]  [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[   85.102009]  [<ffffffff810579cd>] ? autoremove_wake_function+0x0/0x34
[   85.102009]  [<ffffffff811e1839>] ? selinux_inode_rmdir+0x15/0x17
[   85.102009]  [<ffffffff810fd4eb>] ? vfs_rmdir+0x79/0xbe
[   85.102009]  [<ffffffff810feaa0>] ? do_rmdir+0xd0/0x121
[   85.102009]  [<ffffffff8100256c>] ? sysret_check+0x27/0x62
[   85.102009]  [<ffffffff8106ac79>] ? trace_hardirqs_on_caller+0x117/0x13b
[   85.102009]  [<ffffffff8154e201>] ? trace_hardirqs_on_thunk+0x3a/0x3f
[   85.102009]  [<ffffffff8110040b>] ? sys_rmdir+0x11/0x13
[   85.102009]  [<ffffffff8100253b>] ? system_call_fastpath+0x16/0x1b
[   85.268272] systemd[1]: readahead-collect.service: main process exited, code=exited, status=1

Any ideas?

Eric Paris

2011-01-14 02:47 UTC #1 lore raw

Quoted 6 lines · On Thu, 2011-01-13 at 10:34 -0500, [email protected]…

Seen booting yesterday's linux-next, was not present in 2.6.37-rc7-mmotm1202.
Not sure if it's an selinux or cgroup issue, so I'm throwing it at every address I can find for either. This is easily replicatable and happens at every boot, so I can test patches if needed. Am willing to bisect it down if nobody knows right off the bat what the problem is.

Not an SELinux issue. selinux_inode_rmdir() on the stack trace is just left over junk. The vfs doesn't call into the filesystem rmdir code (cgroup_rmdir()) until after the selinux code has already returned. So cgroup people, you might as well pretend that wasn't on the stack trace at all.

-Eric

Quoted 48 lines

The 'W' taint is from the already-reported kernel/workqueue.c worker_enter_idle issue.

[   85.100795] systemd[1]: readahead-replay.service: main process exited, code=exited, status=1
[   85.101530] 
[   85.101531] =============================================
[   85.101796] [ INFO: possible recursive locking detected ]
[   85.102002] 2.6.37-next-20110111 #1
[   85.102009] ---------------------------------------------
[   85.102009] systemd/1 is trying to acquire lock:
[   85.102009]  (&(&dentry->d_lock)->rlock){+.+...}, at: [<ffffffff8107ca5c>] cgroup_rmdir+0x339/0x479
[   85.102009] 
[   85.102009] but task is already holding lock:
[   85.102009]  (&(&dentry->d_lock)->rlock){+.+...}, at: [<ffffffff8107ca54>] cgroup_rmdir+0x331/0x479
[   85.102009] 
[   85.102009] other info that might help us debug this:
[   85.102009] 4 locks held by systemd/1:
[   85.102009]  #0:  (&sb->s_type->i_mutex_key#14/1){+.+.+.}, at: [<ffffffff810fea4d>] do_rmdir+0x7d/0x121
[   85.102009]  #1:  (&sb->s_type->i_mutex_key#14){+.+.+.}, at: [<ffffffff810fd4bc>] vfs_rmdir+0x4a/0xbe
[   85.102009]  #2:  (cgroup_mutex){+.+.+.}, at: [<ffffffff8107cb84>] cgroup_rmdir+0x461/0x479
[   85.102009]  #3:  (&(&dentry->d_lock)->rlock){+.+...}, at: [<ffffffff8107ca54>] cgroup_rmdir+0x331/0x479
[   85.102009] 
[   85.102009] stack backtrace:
[   85.102009] Pid: 1, comm: systemd Tainted: G        W   2.6.37-next-20110111 #1
[   85.102009] Call Trace:
[   85.102009]  [<ffffffff81069f22>] ? __lock_acquire+0x929/0xd4e
[   85.102009]  [<ffffffff8107c6f1>] ? cgroup_clear_directory+0xff/0x131
[   85.102009]  [<ffffffff8107c6f1>] ? cgroup_clear_directory+0xff/0x131
[   85.102009]  [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[   85.102009]  [<ffffffff8106a859>] ? lock_acquire+0x100/0x126
[   85.102009]  [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[   85.102009]  [<ffffffff815521ef>] ? sub_preempt_count+0x35/0x48
[   85.102009]  [<ffffffff8154e401>] ? _raw_spin_lock+0x36/0x45
[   85.102009]  [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[   85.102009]  [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[   85.102009]  [<ffffffff810579cd>] ? autoremove_wake_function+0x0/0x34
[   85.102009]  [<ffffffff811e1839>] ? selinux_inode_rmdir+0x15/0x17
[   85.102009]  [<ffffffff810fd4eb>] ? vfs_rmdir+0x79/0xbe
[   85.102009]  [<ffffffff810feaa0>] ? do_rmdir+0xd0/0x121
[   85.102009]  [<ffffffff8100256c>] ? sysret_check+0x27/0x62
[   85.102009]  [<ffffffff8106ac79>] ? trace_hardirqs_on_caller+0x117/0x13b
[   85.102009]  [<ffffffff8154e201>] ? trace_hardirqs_on_thunk+0x3a/0x3f
[   85.102009]  [<ffffffff8110040b>] ? sys_rmdir+0x11/0x13
[   85.102009]  [<ffffffff8100253b>] ? system_call_fastpath+0x16/0x1b
[   85.268272] systemd[1]: readahead-collect.service: main process exited, code=exited, status=1

Any ideas?

Eric Paris

2011-01-14 02:48 UTC #2 lore raw

In reply to [email protected]

Quoted 6 lines · On Thu, 2011-01-13 at 10:34 -0500, [email protected] wrote:

Seen booting yesterday's linux-next, was not present in 2.6.37-rc7-mmotm1202.
Not sure if it's an selinux or cgroup issue, so I'm throwing it at every address I can find for either. This is easily replicatable and happens at every boot, so I can test patches if needed. Am willing to bisect it down if nobody knows right off the bat what the problem is.

-Eric

Quoted 48 lines

The 'W' taint is from the already-reported kernel/workqueue.c worker_enter_idle issue.

[   85.100795] systemd[1]: readahead-replay.service: main process exited, code=exited, status=1
[   85.101530] 
[   85.101531] =============================================
[   85.101796] [ INFO: possible recursive locking detected ]
[   85.102002] 2.6.37-next-20110111 #1
[   85.102009] ---------------------------------------------
[   85.102009] systemd/1 is trying to acquire lock:
[   85.102009]  (&(&dentry->d_lock)->rlock){+.+...}, at: [<ffffffff8107ca5c>] cgroup_rmdir+0x339/0x479
[   85.102009] 
[   85.102009] but task is already holding lock:
[   85.102009]  (&(&dentry->d_lock)->rlock){+.+...}, at: [<ffffffff8107ca54>] cgroup_rmdir+0x331/0x479
[   85.102009] 
[   85.102009] other info that might help us debug this:
[   85.102009] 4 locks held by systemd/1:
[   85.102009]  #0:  (&sb->s_type->i_mutex_key#14/1){+.+.+.}, at: [<ffffffff810fea4d>] do_rmdir+0x7d/0x121
[   85.102009]  #1:  (&sb->s_type->i_mutex_key#14){+.+.+.}, at: [<ffffffff810fd4bc>] vfs_rmdir+0x4a/0xbe
[   85.102009]  #2:  (cgroup_mutex){+.+.+.}, at: [<ffffffff8107cb84>] cgroup_rmdir+0x461/0x479
[   85.102009]  #3:  (&(&dentry->d_lock)->rlock){+.+...}, at: [<ffffffff8107ca54>] cgroup_rmdir+0x331/0x479
[   85.102009] 
[   85.102009] stack backtrace:
[   85.102009] Pid: 1, comm: systemd Tainted: G        W   2.6.37-next-20110111 #1
[   85.102009] Call Trace:
[   85.102009]  [<ffffffff81069f22>] ? __lock_acquire+0x929/0xd4e
[   85.102009]  [<ffffffff8107c6f1>] ? cgroup_clear_directory+0xff/0x131
[   85.102009]  [<ffffffff8107c6f1>] ? cgroup_clear_directory+0xff/0x131
[   85.102009]  [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[   85.102009]  [<ffffffff8106a859>] ? lock_acquire+0x100/0x126
[   85.102009]  [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[   85.102009]  [<ffffffff815521ef>] ? sub_preempt_count+0x35/0x48
[   85.102009]  [<ffffffff8154e401>] ? _raw_spin_lock+0x36/0x45
[   85.102009]  [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[   85.102009]  [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[   85.102009]  [<ffffffff810579cd>] ? autoremove_wake_function+0x0/0x34
[   85.102009]  [<ffffffff811e1839>] ? selinux_inode_rmdir+0x15/0x17
[   85.102009]  [<ffffffff810fd4eb>] ? vfs_rmdir+0x79/0xbe
[   85.102009]  [<ffffffff810feaa0>] ? do_rmdir+0xd0/0x121
[   85.102009]  [<ffffffff8100256c>] ? sysret_check+0x27/0x62
[   85.102009]  [<ffffffff8106ac79>] ? trace_hardirqs_on_caller+0x117/0x13b
[   85.102009]  [<ffffffff8154e201>] ? trace_hardirqs_on_thunk+0x3a/0x3f
[   85.102009]  [<ffffffff8110040b>] ? sys_rmdir+0x11/0x13
[   85.102009]  [<ffffffff8100253b>] ? system_call_fastpath+0x16/0x1b
[   85.268272] systemd[1]: readahead-collect.service: main process exited, code=exited, status=1

Any ideas?

Nick Piggin

2011-01-14 03:35 UTC #3 lore raw

In reply to [email protected]

Quoted 52 lines · On Fri, Jan 14, 2011 at 2:34 AM, <[email protected]…

Seen booting yesterday's linux-next, was not present in 2.6.37-rc7-mmotm1202.
Not sure if it's an selinux or cgroup issue, so I'm throwing it at every address I can find for either. This is easily replicatable and happens at every boot, so I can test patches if needed. Am willing to bisect it down if nobody knows right off the bat what the problem is.
The 'W' taint is from the already-reported kernel/workqueue.c worker_enter_idle issue.
[ 85.100795] systemd[1]: readahead-replay.service: main process exited, code=exited, status=1 [ 85.101530]
[ 85.101531] =============================================
[ 85.101796] [ INFO: possible recursive locking detected ]
[ 85.102002] 2.6.37-next-20110111 #1
[ 85.102009] ---------------------------------------------
[ 85.102009] systemd/1 is trying to acquire lock:
[ 85.102009] (&(&dentry->d_lock)->rlock){+.+...}, at: [<ffffffff8107ca5c>] cgroup_rmdir+0x339/0x479 [ 85.102009]
[ 85.102009] but task is already holding lock:
[ 85.102009] (&(&dentry->d_lock)->rlock){+.+...}, at: [<ffffffff8107ca54>] cgroup_rmdir+0x331/0x479 [ 85.102009]
[ 85.102009] other info that might help us debug this:
[ 85.102009] 4 locks held by systemd/1:
[ 85.102009] #0: (&sb->s_type->i_mutex_key#14/1){+.+.+.}, at: [<ffffffff810fea4d>] do_rmdir+0x7d/0x121 [ 85.102009] #1: (&sb->s_type->i_mutex_key#14){+.+.+.}, at: [<ffffffff810fd4bc>] vfs_rmdir+0x4a/0xbe [ 85.102009] #2: (cgroup_mutex){+.+.+.}, at: [<ffffffff8107cb84>] cgroup_rmdir+0x461/0x479 [ 85.102009] #3: (&(&dentry->d_lock)->rlock){+.+...}, at: [<ffffffff8107ca54>] cgroup_rmdir+0x331/0x479 [ 85.102009]
[ 85.102009] stack backtrace:
[ 85.102009] Pid: 1, comm: systemd Tainted: G W 2.6.37-next-20110111 #1
[ 85.102009] Call Trace:
[ 85.102009] [<ffffffff81069f22>] ? __lock_acquire+0x929/0xd4e
[ 85.102009] [<ffffffff8107c6f1>] ? cgroup_clear_directory+0xff/0x131
[ 85.102009] [<ffffffff8107c6f1>] ? cgroup_clear_directory+0xff/0x131
[ 85.102009] [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[ 85.102009] [<ffffffff8106a859>] ? lock_acquire+0x100/0x126
[ 85.102009] [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[ 85.102009] [<ffffffff815521ef>] ? sub_preempt_count+0x35/0x48
[ 85.102009] [<ffffffff8154e401>] ? _raw_spin_lock+0x36/0x45
[ 85.102009] [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[ 85.102009] [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[ 85.102009] [<ffffffff810579cd>] ? autoremove_wake_function+0x0/0x34
[ 85.102009] [<ffffffff811e1839>] ? selinux_inode_rmdir+0x15/0x17
[ 85.102009] [<ffffffff810fd4eb>] ? vfs_rmdir+0x79/0xbe
[ 85.102009] [<ffffffff810feaa0>] ? do_rmdir+0xd0/0x121
[ 85.102009] [<ffffffff8100256c>] ? sysret_check+0x27/0x62
[ 85.102009] [<ffffffff8106ac79>] ? trace_hardirqs_on_caller+0x117/0x13b
[ 85.102009] [<ffffffff8154e201>] ? trace_hardirqs_on_thunk+0x3a/0x3f
[ 85.102009] [<ffffffff8110040b>] ? sys_rmdir+0x11/0x13
[ 85.102009] [<ffffffff8100253b>] ? system_call_fastpath+0x16/0x1b
[ 85.268272] systemd[1]: readahead-collect.service: main process exited, code=exited, status=1

Quoted 1 line

Any ideas?

It looks like it is just a missing parent->child lock order annotation, but mainline cgroupfs code looks to be OK there. What does cgroup_clear_directory() look like in mmotm?

Nick Piggin

2011-01-14 03:35 UTC #4 lore raw

In reply to [email protected]

Quoted 52 lines · On Fri, Jan 14, 2011 at 2:34 AM, <[email protected]> wrote:

Seen booting yesterday's linux-next, was not present in 2.6.37-rc7-mmotm1202.
Not sure if it's an selinux or cgroup issue, so I'm throwing it at every address I can find for either. This is easily replicatable and happens at every boot, so I can test patches if needed. Am willing to bisect it down if nobody knows right off the bat what the problem is.
The 'W' taint is from the already-reported kernel/workqueue.c worker_enter_idle issue.
[ 85.100795] systemd[1]: readahead-replay.service: main process exited, code=exited, status=1 [ 85.101530]
[ 85.101531] =============================================
[ 85.101796] [ INFO: possible recursive locking detected ]
[ 85.102002] 2.6.37-next-20110111 #1
[ 85.102009] ---------------------------------------------
[ 85.102009] systemd/1 is trying to acquire lock:
[ 85.102009] (&(&dentry->d_lock)->rlock){+.+...}, at: [<ffffffff8107ca5c>] cgroup_rmdir+0x339/0x479 [ 85.102009]
[ 85.102009] but task is already holding lock:
[ 85.102009] (&(&dentry->d_lock)->rlock){+.+...}, at: [<ffffffff8107ca54>] cgroup_rmdir+0x331/0x479 [ 85.102009]
[ 85.102009] other info that might help us debug this:
[ 85.102009] 4 locks held by systemd/1:
[ 85.102009] #0: (&sb->s_type->i_mutex_key#14/1){+.+.+.}, at: [<ffffffff810fea4d>] do_rmdir+0x7d/0x121 [ 85.102009] #1: (&sb->s_type->i_mutex_key#14){+.+.+.}, at: [<ffffffff810fd4bc>] vfs_rmdir+0x4a/0xbe [ 85.102009] #2: (cgroup_mutex){+.+.+.}, at: [<ffffffff8107cb84>] cgroup_rmdir+0x461/0x479 [ 85.102009] #3: (&(&dentry->d_lock)->rlock){+.+...}, at: [<ffffffff8107ca54>] cgroup_rmdir+0x331/0x479 [ 85.102009]
[ 85.102009] stack backtrace:
[ 85.102009] Pid: 1, comm: systemd Tainted: G W 2.6.37-next-20110111 #1
[ 85.102009] Call Trace:
[ 85.102009] [<ffffffff81069f22>] ? __lock_acquire+0x929/0xd4e
[ 85.102009] [<ffffffff8107c6f1>] ? cgroup_clear_directory+0xff/0x131
[ 85.102009] [<ffffffff8107c6f1>] ? cgroup_clear_directory+0xff/0x131
[ 85.102009] [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[ 85.102009] [<ffffffff8106a859>] ? lock_acquire+0x100/0x126
[ 85.102009] [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[ 85.102009] [<ffffffff815521ef>] ? sub_preempt_count+0x35/0x48
[ 85.102009] [<ffffffff8154e401>] ? _raw_spin_lock+0x36/0x45
[ 85.102009] [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[ 85.102009] [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[ 85.102009] [<ffffffff810579cd>] ? autoremove_wake_function+0x0/0x34
[ 85.102009] [<ffffffff811e1839>] ? selinux_inode_rmdir+0x15/0x17
[ 85.102009] [<ffffffff810fd4eb>] ? vfs_rmdir+0x79/0xbe
[ 85.102009] [<ffffffff810feaa0>] ? do_rmdir+0xd0/0x121
[ 85.102009] [<ffffffff8100256c>] ? sysret_check+0x27/0x62
[ 85.102009] [<ffffffff8106ac79>] ? trace_hardirqs_on_caller+0x117/0x13b
[ 85.102009] [<ffffffff8154e201>] ? trace_hardirqs_on_thunk+0x3a/0x3f
[ 85.102009] [<ffffffff8110040b>] ? sys_rmdir+0x11/0x13
[ 85.102009] [<ffffffff8100253b>] ? system_call_fastpath+0x16/0x1b
[ 85.268272] systemd[1]: readahead-collect.service: main process exited, code=exited, status=1

Quoted 1 line

Any ideas?

It looks like it is just a missing parent->child lock order annotation, but mainline cgroupfs code looks to be OK there. What does cgroup_clear_directory() look like in mmotm?

Li Zefan

2011-01-14 04:06 UTC #5 lore raw

Quoted 53 lines · Nick Piggin

Quoted 52 lines · On Fri, Jan 14, 2011 at 2:34 AM, <[email protected]> wrote:

Seen booting yesterday's linux-next, was not present in 2.6.37-rc7-mmotm1202.

The 'W' taint is from the already-reported kernel/workqueue.c worker_enter_idle issue.

[   85.100795] systemd[1]: readahead-replay.service: main process exited, code=exited, status=1
[   85.101530]
[   85.101531] =============================================
[   85.101796] [ INFO: possible recursive locking detected ]
[   85.102002] 2.6.37-next-20110111 #1
[   85.102009] ---------------------------------------------
[   85.102009] systemd/1 is trying to acquire lock:
[   85.102009]  (&(&dentry->d_lock)->rlock){+.+...}, at: [<ffffffff8107ca5c>] cgroup_rmdir+0x339/0x479
[   85.102009]
[   85.102009] but task is already holding lock:
[   85.102009]  (&(&dentry->d_lock)->rlock){+.+...}, at: [<ffffffff8107ca54>] cgroup_rmdir+0x331/0x479
[   85.102009]
[   85.102009] other info that might help us debug this:
[   85.102009] 4 locks held by systemd/1:
[   85.102009]  #0:  (&sb->s_type->i_mutex_key#14/1){+.+.+.}, at: [<ffffffff810fea4d>] do_rmdir+0x7d/0x121
[   85.102009]  #1:  (&sb->s_type->i_mutex_key#14){+.+.+.}, at: [<ffffffff810fd4bc>] vfs_rmdir+0x4a/0xbe
[   85.102009]  #2:  (cgroup_mutex){+.+.+.}, at: [<ffffffff8107cb84>] cgroup_rmdir+0x461/0x479
[   85.102009]  #3:  (&(&dentry->d_lock)->rlock){+.+...}, at: [<ffffffff8107ca54>] cgroup_rmdir+0x331/0x479
[   85.102009]
[   85.102009] stack backtrace:
[   85.102009] Pid: 1, comm: systemd Tainted: G        W   2.6.37-next-20110111 #1
[   85.102009] Call Trace:
[   85.102009]  [<ffffffff81069f22>] ? __lock_acquire+0x929/0xd4e
[   85.102009]  [<ffffffff8107c6f1>] ? cgroup_clear_directory+0xff/0x131
[   85.102009]  [<ffffffff8107c6f1>] ? cgroup_clear_directory+0xff/0x131
[   85.102009]  [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[   85.102009]  [<ffffffff8106a859>] ? lock_acquire+0x100/0x126
[   85.102009]  [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[   85.102009]  [<ffffffff815521ef>] ? sub_preempt_count+0x35/0x48
[   85.102009]  [<ffffffff8154e401>] ? _raw_spin_lock+0x36/0x45
[   85.102009]  [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[   85.102009]  [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[   85.102009]  [<ffffffff810579cd>] ? autoremove_wake_function+0x0/0x34
[   85.102009]  [<ffffffff811e1839>] ? selinux_inode_rmdir+0x15/0x17
[   85.102009]  [<ffffffff810fd4eb>] ? vfs_rmdir+0x79/0xbe
[   85.102009]  [<ffffffff810feaa0>] ? do_rmdir+0xd0/0x121
[   85.102009]  [<ffffffff8100256c>] ? sysret_check+0x27/0x62
[   85.102009]  [<ffffffff8106ac79>] ? trace_hardirqs_on_caller+0x117/0x13b
[   85.102009]  [<ffffffff8154e201>] ? trace_hardirqs_on_thunk+0x3a/0x3f
[   85.102009]  [<ffffffff8110040b>] ? sys_rmdir+0x11/0x13
[   85.102009]  [<ffffffff8100253b>] ? system_call_fastpath+0x16/0x1b
[   85.268272] systemd[1]: readahead-collect.service: main process exited, code=exited, status=1

Quoted 5 lines

Quoted 1 line
Any ideas?
It looks like it is just a missing parent->child lock order annotation, but mainline cgroupfs code looks to be OK there. What does cgroup_clear_directory() look like in mmotm?

It's not from cgroup_clear_directory()..

This should fix it:

=========================

From: Li Zefan <[email protected]>
Date: Fri, 14 Jan 2011 11:34:34 +0800
Subject: [PATCH] cgroups: Fix a lockdep warning at cgroup removal

Commit 2fd6b7f5 ("fs: dcache scale subdirs") forgot to annotate a dentry lock, which caused a lockdep warning.

Reported-by Valdis Kletnieks <[email protected]>

Signed-off-by Li Zefan <[email protected]>

---
 kernel/cgroup.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

kernel/cgroup.c +1 -1

index 5c5f4cc..db983e2 100644
@@ -910,7 +910,7 @@ static void cgroup_d_remove_dir(struct dentry *dentry)
 
        parent = dentry->d_parent;
        spin_lock(&parent->d_lock);
-       spin_lock(&dentry->d_lock);
+       spin_lock_nested(&dentry->d_lock, DENTRY_D_LOCK_NESTED);
        list_del_init(&dentry->d_u.d_child);
        spin_unlock(&dentry->d_lock);
        spin_unlock(&parent->d_lock);

Signature · 1 line

1.7.3.1

Li Zefan

2011-01-14 04:06 UTC #10 lore raw

In reply to a message not in the archive

Quoted 53 lines · Nick Piggin

Quoted 52 lines · On Fri, Jan 14, 2011 at 2:34 AM, <[email protected]…

Seen booting yesterday's linux-next, was not present in 2.6.37-rc7-mmotm1202.

The 'W' taint is from the already-reported kernel/workqueue.c worker_enter_idle issue.

[   85.100795] systemd[1]: readahead-replay.service: main process exited, code=exited, status=1
[   85.101530]
[   85.101531] =============================================
[   85.101796] [ INFO: possible recursive locking detected ]
[   85.102002] 2.6.37-next-20110111 #1
[   85.102009] ---------------------------------------------
[   85.102009] systemd/1 is trying to acquire lock:
[   85.102009]  (&(&dentry->d_lock)->rlock){+.+...}, at: [<ffffffff8107ca5c>] cgroup_rmdir+0x339/0x479
[   85.102009]
[   85.102009] but task is already holding lock:
[   85.102009]  (&(&dentry->d_lock)->rlock){+.+...}, at: [<ffffffff8107ca54>] cgroup_rmdir+0x331/0x479
[   85.102009]
[   85.102009] other info that might help us debug this:
[   85.102009] 4 locks held by systemd/1:
[   85.102009]  #0:  (&sb->s_type->i_mutex_key#14/1){+.+.+.}, at: [<ffffffff810fea4d>] do_rmdir+0x7d/0x121
[   85.102009]  #1:  (&sb->s_type->i_mutex_key#14){+.+.+.}, at: [<ffffffff810fd4bc>] vfs_rmdir+0x4a/0xbe
[   85.102009]  #2:  (cgroup_mutex){+.+.+.}, at: [<ffffffff8107cb84>] cgroup_rmdir+0x461/0x479
[   85.102009]  #3:  (&(&dentry->d_lock)->rlock){+.+...}, at: [<ffffffff8107ca54>] cgroup_rmdir+0x331/0x479
[   85.102009]
[   85.102009] stack backtrace:
[   85.102009] Pid: 1, comm: systemd Tainted: G        W   2.6.37-next-20110111 #1
[   85.102009] Call Trace:
[   85.102009]  [<ffffffff81069f22>] ? __lock_acquire+0x929/0xd4e
[   85.102009]  [<ffffffff8107c6f1>] ? cgroup_clear_directory+0xff/0x131
[   85.102009]  [<ffffffff8107c6f1>] ? cgroup_clear_directory+0xff/0x131
[   85.102009]  [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[   85.102009]  [<ffffffff8106a859>] ? lock_acquire+0x100/0x126
[   85.102009]  [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[   85.102009]  [<ffffffff815521ef>] ? sub_preempt_count+0x35/0x48
[   85.102009]  [<ffffffff8154e401>] ? _raw_spin_lock+0x36/0x45
[   85.102009]  [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[   85.102009]  [<ffffffff8107ca5c>] ? cgroup_rmdir+0x339/0x479
[   85.102009]  [<ffffffff810579cd>] ? autoremove_wake_function+0x0/0x34
[   85.102009]  [<ffffffff811e1839>] ? selinux_inode_rmdir+0x15/0x17
[   85.102009]  [<ffffffff810fd4eb>] ? vfs_rmdir+0x79/0xbe
[   85.102009]  [<ffffffff810feaa0>] ? do_rmdir+0xd0/0x121
[   85.102009]  [<ffffffff8100256c>] ? sysret_check+0x27/0x62
[   85.102009]  [<ffffffff8106ac79>] ? trace_hardirqs_on_caller+0x117/0x13b
[   85.102009]  [<ffffffff8154e201>] ? trace_hardirqs_on_thunk+0x3a/0x3f
[   85.102009]  [<ffffffff8110040b>] ? sys_rmdir+0x11/0x13
[   85.102009]  [<ffffffff8100253b>] ? system_call_fastpath+0x16/0x1b
[   85.268272] systemd[1]: readahead-collect.service: main process exited, code=exited, status=1

Quoted 5 lines

Quoted 1 line
Any ideas?
It looks like it is just a missing parent->child lock order annotation, but mainline cgroupfs code looks to be OK there. What does cgroup_clear_directory() look like in mmotm?

It's not from cgroup_clear_directory()..

This should fix it:

=========================

From: Li Zefan <[email protected]>
Date: Fri, 14 Jan 2011 11:34:34 +0800
Subject: [PATCH] cgroups: Fix a lockdep warning at cgroup removal

Commit 2fd6b7f5 ("fs: dcache scale subdirs") forgot to annotate a dentry lock, which caused a lockdep warning.

Reported-by Valdis Kletnieks <[email protected]>

Signed-off-by Li Zefan <[email protected]>

---
 kernel/cgroup.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

kernel/cgroup.c +1 -1

index 5c5f4cc..db983e2 100644
@@ -910,7 +910,7 @@ static void cgroup_d_remove_dir(struct dentry *dentry)
 
        parent = dentry->d_parent;
        spin_lock(&parent->d_lock);
-       spin_lock(&dentry->d_lock);
+       spin_lock_nested(&dentry->d_lock, DENTRY_D_LOCK_NESTED);
        list_del_init(&dentry->d_u.d_child);
        spin_unlock(&dentry->d_lock);
        spin_unlock(&parent->d_lock);

Signature · 1 line

1.7.3.1

Nick Piggin

2011-01-14 04:10 UTC #7 lore raw

In reply to a message not in the archive

Quoted 8 lines · Li Zefan

Quoted 3 lines · Nick Piggin
It looks like it is just a missing parent->child lock order annotation, but mainline cgroupfs code looks to be OK there. What does cgroup_clear_directory() look like in mmotm?
It's not from cgroup_clear_directory()..
This should fix it:

Yep, thanks!

Quoted 32 lines

=========================
From: Li Zefan <[email protected]>
Date: Fri, 14 Jan 2011 11:34:34 +0800
Subject: [PATCH] cgroups: Fix a lockdep warning at cgroup removal
Commit 2fd6b7f5 ("fs: dcache scale subdirs") forgot to annotate a dentry lock, which caused a lockdep warning.
Reported-by Valdis Kletnieks <[email protected]>
Signed-off-by Li Zefan <[email protected]>
---
kernel/cgroup.c | 2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
kernel/cgroup.c +1 -2
index 5c5f4cc..db983e2 100644
@@ -910,7 +910,7 @@ static void cgroup_d_remove_dir(struct dentry *dentry)
 
        parent = dentry->d_parent;
        spin_lock(&parent->d_lock);
-       spin_lock(&dentry->d_lock);
+       spin_lock_nested(&dentry->d_lock, DENTRY_D_LOCK_NESTED);
        list_del_init(&dentry->d_u.d_child);
        spin_unlock(&dentry->d_lock);
        spin_unlock(&parent->d_lock);
--
1.7.3.1

Nick Piggin

2011-01-14 04:10 UTC #8 lore raw

In reply to Li Zefan

Quoted 8 lines · Li Zefan

Quoted 3 lines · Nick Piggin
It looks like it is just a missing parent->child lock order annotation, but mainline cgroupfs code looks to be OK there. What does cgroup_clear_directory() look like in mmotm?
It's not from cgroup_clear_directory()..
This should fix it:

Yep, thanks!

Quoted 32 lines

=========================
From: Li Zefan <[email protected]>
Date: Fri, 14 Jan 2011 11:34:34 +0800
Subject: [PATCH] cgroups: Fix a lockdep warning at cgroup removal
Commit 2fd6b7f5 ("fs: dcache scale subdirs") forgot to annotate a dentry lock, which caused a lockdep warning.
Reported-by Valdis Kletnieks <[email protected]>
Signed-off-by Li Zefan <[email protected]>
---
kernel/cgroup.c | 2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
kernel/cgroup.c +1 -2
index 5c5f4cc..db983e2 100644
@@ -910,7 +910,7 @@ static void cgroup_d_remove_dir(struct dentry *dentry)
 
        parent = dentry->d_parent;
        spin_lock(&parent->d_lock);
-       spin_lock(&dentry->d_lock);
+       spin_lock_nested(&dentry->d_lock, DENTRY_D_LOCK_NESTED);
        list_del_init(&dentry->d_u.d_child);
        spin_unlock(&dentry->d_lock);
        spin_unlock(&parent->d_lock);
--
1.7.3.1