Skip to contentSkip to conversation

circular locking: /proc/sys/vm/drop_caches

DISCUSSION Ingo Molnar 3 participants 4 messages Feb 23, 2007

lore
Share & tools
lore
#0 lore raw

Andrew,

i just got the lockdep warning below when doing:

echo 3 > /proc/sys/vm/drop_caches

is this a known quirk in the drop_caches code?

Ingo

=======================================================
[ INFO: possible circular locking dependency detected ]
2.6.20-syslet #434
-------------------------------------------------------
bash/12166 is trying to acquire lock:
(&journal->j_list_lock){--..}, at: [<c01c82f5>] journal_try_to_free_buffers+0xd5/0x18f

but task is already holding lock:
(inode_lock){--..}, at: [<c01947a3>] drop_pagecache+0x45/0xc8

which lock already depends on the new lock.

the existing dependency chain (in reverse order) is:

-> #1 (inode_lock){--..}:

       [<c0144940>] __lock_acquire+0x984/0xa94
       [<c0144d6f>] lock_acquire+0x67/0x8c
       [<c0424a9f>] _spin_lock+0x35/0x42
       [<c0193ccb>] __mark_inode_dirty+0xe3/0x15e
       [<c016253b>] __set_page_dirty_nobuffers+0xb4/0xbf
       [<c0197361>] mark_buffer_dirty+0x1e/0x20
       [<c01c7af2>] __journal_temp_unlink_buffer+0x176/0x17d
       [<c01c7d23>] __journal_unfile_buffer+0xb/0x15
       [<c01c7d97>] __journal_refile_buffer+0x6a/0xe3
       [<c01cac34>] journal_commit_transaction+0xf46/0x11da
       [<c01cd32c>] kjournald+0xab/0x1e8
       [<c013b935>] kthread+0xb2/0xd7
       [<c0104e5f>] kernel_thread_helper+0x7/0x10
       [<ffffffff>] 0xffffffff

-> #0 (&journal->j_list_lock){--..}:

       [<c014483b>] __lock_acquire+0x87f/0xa94
       [<c0144d6f>] lock_acquire+0x67/0x8c
       [<c0424a9f>] _spin_lock+0x35/0x42
       [<c01c82f5>] journal_try_to_free_buffers+0xd5/0x18f
       [<c01b944e>] ext3_releasepage+0x68/0x74
       [<c015d027>] try_to_release_page+0x33/0x44
       [<c016406e>] invalidate_mapping_pages+0x69/0xca
       [<c01640dc>] invalidate_inode_pages+0xd/0x11
       [<c01947bf>] drop_pagecache+0x61/0xc8
       [<c019485c>] drop_caches_sysctl_handler+0x36/0x4e
       [<c012efdc>] do_rw_proc+0xaf/0xf1
       [<c012f078>] proc_writesys+0x20/0x25
       [<c017b3de>] vfs_write+0xb1/0x165
       [<c017bbc6>] sys_write+0x3d/0x61
       [<c01041a8>] syscall_call+0x7/0xb
       [<ffffffff>] 0xffffffff

other info that might help us debug this:

2 locks held by bash/12166:
#0: (&type->s_umount_key#17){----}, at: [<c0194793>] drop_pagecache+0x35/0xc8 #1: (inode_lock){--..}, at: [<c01947a3>] drop_pagecache+0x45/0xc8

stack backtrace:
 [<c010510d>] show_trace_log_lvl+0x19/0x2e
 [<c0105206>] show_trace+0x12/0x14
 [<c010521c>] dump_stack+0x14/0x16
 [<c01433ff>] print_circular_bug_tail+0x5f/0x68
 [<c014483b>] __lock_acquire+0x87f/0xa94
 [<c0144d6f>] lock_acquire+0x67/0x8c
 [<c0424a9f>] _spin_lock+0x35/0x42
 [<c01c82f5>] journal_try_to_free_buffers+0xd5/0x18f
 [<c01b944e>] ext3_releasepage+0x68/0x74
 [<c015d027>] try_to_release_page+0x33/0x44
 [<c016406e>] invalidate_mapping_pages+0x69/0xca
 [<c01640dc>] invalidate_inode_pages+0xd/0x11
 [<c01947bf>] drop_pagecache+0x61/0xc8
 [<c019485c>] drop_caches_sysctl_handler+0x36/0x4e
 [<c012efdc>] do_rw_proc+0xaf/0xf1
 [<c012f078>] proc_writesys+0x20/0x25
 [<c017b3de>] vfs_write+0xb1/0x165
 [<c017bbc6>] sys_write+0x3d/0x61
 [<c01041a8>] syscall_call+0x7/0xb
 =======================
#2 lore raw
In reply to Ingo Molnar
Quoted 6 lines · On Fri, 23 Feb 2007 07:35:38 +0100 Ingo Molnar <[email protected]> wrote:

i just got the lockdep warning below when doing:

echo 3 > /proc/sys/vm/drop_caches

is this a known quirk in the drop_caches code?

It is - we're taking inode_lock at just the wrong level there and I don't know how to fix it. The basic problem is that we're doing a list_head
walk: if we drop the lock we have to start again.

Well, I do know how to fix it: rip the various inode lists and use a tagged radix tree instead - make writeback's per-superblock inode walk work the same as writeback's per-inode page walk. That will also alow us to drop the lock and then resume the drop_caches thing at the point where we left off.

But it's a large change and has some quite tricky issues which need to be carried over from the existing code and I don't have time to do it.

The only (lame) thing I can say for now is that drop_caches is a debug-only, root-only thing :(

#3 lore raw
Quoted 2 lines · Andrew Morton

The only (lame) thing I can say for now is that drop_caches is a debug-only, root-only thing :(

ok, that's fair enough.

Ingo