INFO: possible circular locking dependency detected
Subsystems (2)
=======================================================
[ INFO: possible circular locking dependency detected ] 2.6.18-1.2189self #1
-------------------------------------------------------
kswapd0/186 is trying to acquire lock:
(&inode->i_mutex){--..}, at: [<c0326e32>] mutex_lock+0x21/0x24
but task is already holding lock:
(iprune_mutex){--..}, at: [<c0326e32>] mutex_lock+0x21/0x24
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #1 (iprune_mutex){--..}:
[<c012fe17>] lock_acquire+0x4b/0x6c
[<c0326cc4>] __mutex_lock_slowpath+0xb3/0x200
[<c0326e32>] mutex_lock+0x21/0x24
[<c017638a>] shrink_icache_memory+0x36/0x1b2
[<c014cf50>] shrink_slab+0xd0/0x123
[<c014d70e>] try_to_free_pages+0x114/0x1ba
[<c0149b78>] __alloc_pages+0x18b/0x279
[<c0146b10>] generic_file_buffered_write+0x167/0x546
[<c014722f>] __generic_file_aio_write_nolock+0x340/0x38a
[<c01472d5>] generic_file_aio_write+0x5c/0xaf
[<f9c9e1cf>] nfs_file_write+0x7b/0x97 [nfs]
[<c015f976>] do_sync_write+0xaf/0xe4
[<c0160229>] vfs_write+0xab/0x157
[<c0160760>] sys_write+0x3b/0x60
[<c0102df3>] syscall_call+0x7/0xb-> #0 (&inode->i_mutex){--..}:
[<c012fe17>] lock_acquire+0x4b/0x6c
[<c0326cc4>] __mutex_lock_slowpath+0xb3/0x200
[<c0326e32>] mutex_lock+0x21/0x24
[<f921706d>] ntfs_put_inode+0x3d/0x75 [ntfs]
[<c0175acd>] iput+0x33/0x6a
[<f9216eaa>] ntfs_clear_big_inode+0x99/0xb2 [ntfs]
[<c0175fdc>] clear_inode+0xd8/0x129
[<c01762cf>] dispose_list+0x4c/0xd1
[<c01764de>] shrink_icache_memory+0x18a/0x1b2
[<c014cf50>] shrink_slab+0xd0/0x123
[<c014d2a3>] kswapd+0x260/0x336
[<c012a4b7>] kthread+0xb0/0xdd
[<c0101005>] kernel_thread_helper+0x5/0xbother info that might help us debug this:
2 locks held by kswapd0/186:
#0: (shrinker_rwsem){----}, at: [<c014cea5>] shrink_slab+0x25/0x123 #1: (iprune_mutex){--..}, at: [<c0326e32>] mutex_lock+0x21/0x24
stack backtrace: [<c0103fe1>] show_trace+0xd/0x10 [<c010447d>] dump_stack+0x19/0x1b [<c012ef1e>] print_circular_bug_tail+0x59/0x64 [<c012f70b>] __lock_acquire+0x7e2/0x986 [<c012fe17>] lock_acquire+0x4b/0x6c [<c0326cc4>] __mutex_lock_slowpath+0xb3/0x200 [<c0326e32>] mutex_lock+0x21/0x24 [<f921706d>] ntfs_put_inode+0x3d/0x75 [ntfs] [<c0175acd>] iput+0x33/0x6a [<f9216eaa>] ntfs_clear_big_inode+0x99/0xb2 [ntfs] [<c0175fdc>] clear_inode+0xd8/0x129 [<c01762cf>] dispose_list+0x4c/0xd1 [<c01764de>] shrink_icache_memory+0x18a/0x1b2 [<c014cf50>] shrink_slab+0xd0/0x123 [<c014d2a3>] kswapd+0x260/0x336 [<c012a4b7>] kthread+0xb0/0xdd [<c0101005>] kernel_thread_helper+0x5/0xb
Quoted 11 lines · alpha @ steudten Engineering wrote:
=======================================================
[ INFO: possible circular locking dependency detected ] 2.6.18-1.2189self #1
-------------------------------------------------------
kswapd0/186 is trying to acquire lock:
(&inode->i_mutex){--..}, at: [<c0326e32>]mutex_lock+0x21/0x24but task is already holding lock:
(iprune_mutex){--..}, at: [<c0326e32>]mutex_lock+0x21/0x24which lock already depends on the new lock.
Thanks. __grab_cache_page wants to clear __GFP_FS, because it is holding the i_mutex so we don't want to reenter the filesystem in page reclaim.
This would be an easy two liner, except those funny page_cache_alloc routines which take a mapping rather than a gfp_t argument :P
Anyway, I'll get around to writing the real patch and queue it up with my other buffered write deadlock fixes. It should be fairly unlikely to cause you a deadlock. You could give this quick patch a try, though. Does it fix your problem?
Signature · 1 line
SUSE Labs, Novell Inc.
include/linux/pagemap.h +1 -1
@@ -57,7 +57,7 @@ extern struct page *page_cache_alloc_col#else static inline struct page *page_cache_alloc (struct address_space *x) { -return alloc_pages (mapping_gfp_mask (x),0 ); +return alloc_pages (mapping_gfp_mask (x)&~__GFP_FS,0 ); }static inline struct page *page_cache_alloc_cold (struct address_space *x)
mm/filemap.c +3 -3
@@ -471,9 +471,9 @@ struct page *page_cache_alloc(struct add {if (cpuset_do_page_mem_spread ()) {int n =cpuset_mem_spread_node (); -return alloc_pages_node (n,mapping_gfp_mask (x),0 ); +return alloc_pages_node (n,mapping_gfp_mask (x)&~__GFP_FS,0 ); } -return alloc_pages (mapping_gfp_mask (x),0 ); +return alloc_pages (mapping_gfp_mask (x)&~__GFP_FS,0 ); }EXPORT_SYMBOL (page_cache_alloc);
@@ -1864,7 +1864,7 @@ repeat:return NULL ; } err =add_to_page_cache (*cached_page, mapping, - index,GFP_KERNEL ); + index,GFP_KERNEL &~__GFP_FS);if (err == -EEXIST ) goto repeat;if (err ==0 ) {
Quoted 11 lines · On Tue, 17 Oct 2006 00:32:44 +1000 Nick Piggin wrote:
Quoted 10 lines · alpha @ steudten Engineering wrote:
=======================================================
[ INFO: possible circular locking dependency detected ] 2.6.18-1.2189self #1
-------------------------------------------------------
kswapd0/186 is trying to acquire lock:
(&inode->i_mutex){--..}, at: [<c0326e32>]mutex_lock+0x21/0x24but task is already holding lock:
(iprune_mutex){--..}, at: [<c0326e32>]mutex_lock+0x21/0x24
Quoted 8 lines
Quoted 1 line
which lock already depends on the new lock.
Thanks.
__grab_cache_pagewants to clear__GFP_FS, because it is holding thei_mutexso we don't want to reenter the filesystem in page reclaim.This would be an easy two liner, except those funny
page_cache_allocroutines which take a mapping rather than agfp_targument :P
and it would be only one email, but you forgot spaces there, so it's too ugly to use. ;) i.e., please add spaces around the '&'.
and it's an attachment :(
Quoted 4 lines
Anyway, I'll get around to writing the real patch and queue it up with my other buffered write deadlock fixes. It should be fairly unlikely to cause you a deadlock. You could give this quick patch a try, though. Does it fix your problem?
---
~Randy
Quoted 25 lines · Randy Dunlap
Quoted 12 lines · On Tue, 17 Oct 2006 00:32:44 +1000 Nick Piggin wrote:
Quoted 10 lines · alpha @ steudten Engineering wrote:
=======================================================
[ INFO: possible circular locking dependency detected ] 2.6.18-1.2189self #1
-------------------------------------------------------
kswapd0/186 is trying to acquire lock:
(&inode->i_mutex){--..}, at: [<c0326e32>]mutex_lock+0x21/0x24but task is already holding lock:
(iprune_mutex){--..}, at: [<c0326e32>]mutex_lock+0x21/0x24Quoted 8 lines
Quoted 1 line
which lock already depends on the new lock.
Thanks.
__grab_cache_pagewants to clear__GFP_FS, because it is holding thei_mutexso we don't want to reenter the filesystem in page reclaim.This would be an easy two liner, except those funny
page_cache_allocroutines which take a mapping rather than agfp_targument :P
Quoted 5 lines
and it would be only one email, but you forgot spaces there, so it's too ugly to use. ;) i.e., please add spaces around the '&'.
and it's an attachment :(
Oh yeah, it was an attachment because I didn't want anyone to see how ugly it is ;)
It's just a quick hack to see if it works. I'll send out a real patch when I get time to code and test it properly.
Signature · 2 lines
SUSE Labs, Novell Inc. Send instant messages to your online friends http://au.messenger.yahoo.com
Quoted 8 lines · On Tue, 17 Oct 2006 00:32:44 +1000 Nick Piggin <[email protected]> wrote:
Quoted 7 lines · alpha @ steudten Engineering wrote:
=======================================================
[ INFO: possible circular locking dependency detected ] 2.6.18-1.2189self #1
-------------------------------------------------------
kswapd0/186 is trying to acquire lock:
(&inode->i_mutex){--..}, at: [<c0326e32>]mutex_lock+0x21/0x24
Quoted 8 lines
Quoted 4 lines
but task is already holding lock:
(iprune_mutex){--..}, at: [<c0326e32>]mutex_lock+0x21/0x24which lock already depends on the new lock.
Thanks.
__grab_cache_pagewants to clear__GFP_FS, because it is holding thei_mutexso we don't want to reenter the filesystem in page reclaim.
We want to be able to enter page reclaim while holding i_mutex. Think what the effect of not doing this would be upon write() (!)
This warning is more fallout from ntfs's insistence on taking i_mutex in its clear_inode(). See lengthy and unproductive discussion at
http://lkml.org/lkml/2006/7/26/185 .
Quoted 23 lines · Andrew Morton
Quoted 9 lines · On Tue, 17 Oct 2006 00:32:44 +1000 Nick Piggin <[email protected]> wrote:
Quoted 7 lines · alpha @ steudten Engineering wrote:
=======================================================
[ INFO: possible circular locking dependency detected ] 2.6.18-1.2189self #1
-------------------------------------------------------
kswapd0/186 is trying to acquire lock:
(&inode->i_mutex){--..}, at: [<c0326e32>]mutex_lock+0x21/0x24Quoted 8 lines
Quoted 4 lines
but task is already holding lock:
(iprune_mutex){--..}, at: [<c0326e32>]mutex_lock+0x21/0x24which lock already depends on the new lock.
Thanks.
__grab_cache_pagewants to clear__GFP_FS, because it is holding thei_mutexso we don't want to reenter the filesystem in page reclaim.
Quoted 6 lines
We want to be able to enter page reclaim while holding
i_mutex. Think what the effect of not doing this would be uponwrite()(!)This warning is more fallout from ntfs's insistence on taking
i_mutexin itsclear_inode(). See lengthy and unproductive discussion at
http://lkml.org/lkml/2006/7/26/185 .
Yeah you're right. It will be a hot allocation + reclaim path for high bandwidth writes.
Signature · 2 lines
SUSE Labs, Novell Inc. Send instant messages to your online friends http://au.messenger.yahoo.com