Skip to contentSkip to conversation

fs: clear_inode failed with nrpages not zero!

DISCUSSION hitmoon 2 participants 2 messages Feb 26, 2014 VFS

lore
Share & tools
lore

hitmoon

author
#0 lore raw

Hi all:

I am running a redhat 2.6.32-279 offical kernel. Under heavy work
load and memory pressure, in my case, running ltp test for about 20 hours, kernel oops happened. Say concretely, a testcase process open a file, truncate to 128M, mmap, munmap and close the file, this circle repeatedly when kernel hangs. Through the vmcore, I also find it hangs at: BUG_ON(inode->i_data.nrpages) in function clear_inode, which means the truncate_inode_pages faild to decrase nrpages to 0. I have google this problem and find no clear solutions but make me confused. The comment of function truncate_inode_pages says that after it return, the nrpages may not be zero.

My understanding is: the page reclaime migth still in the process
of deletion of the page. Jan Kara once post a patch, which use spin_lock to sync the radix tree and nrpages. This kernel already contains this patch. Then problem come: When kernel hangs, the nrpages is not a small number like 1 or 2, but a bigger one, more than 500 or 700! So I think even we take some sync measures before clear inode, the function truncate_inode_pages together with other reclaim functions failed to set nrpages to zero. By dump the vmcore, I also find the radix tree is also not empty but with some slots left.

Then I think:
1. The fault might happen at pagevec_lookup, which return no page
even the radix tree is in fact not empty. Because lookup uses the rcu lock, is it possible a race condition
happened in the lookup process and lead the function return
unexpectedly? If possiable, how dose it happened ?
2. I find Johannes Weiner post a
patch(http://www.spinics.net/lists/linux-fsdevel/msg72395.html), which has following code:

+ if (nrpages || nrshadows) {
+ /*
+ * As truncation uses a lockless tree lookup, cycle
+ * the tree lock to make sure any ongoing tree
+ * modification that does not see AS_EXITING is
+ * completed before starting the final truncate.
+ */
+ spin_lock_irq(&mapping->tree_lock);
+ spin_unlock_irq(&mapping->tree_lock);
+
+ truncate_inode_pages(mapping, 0);
+ }

which wrapped the truncate_inode_pages in function
truncate_inode_pages_final. Does it make sence to my problem ?

Any suggestion will be appreciated!

reviewer
#1 lore raw

Hello,

Quoted 1 line · hitmoon

I am running a redhat 2.6.32-279 offical kernel.

Well, in that case you should consider contacting RH support instead of
general community forum... Also 2.6.32 is pretty old and RH (similarly as other enterprise distributors) has lots of stuff added on top of it. So it is hard to help you.

Quoted 45 lines

Under heavy work load and memory pressure, in my case, running ltp test for about 20 hours, kernel oops happened. Say concretely, a testcase process open a file, truncate to 128M, mmap, munmap and close the file, this circle repeatedly when kernel hangs. Through the vmcore, I also find it hangs at: BUG_ON(inode->i_data.nrpages) in function clear_inode, which means the truncate_inode_pages faild to decrase nrpages to 0. I have google this problem and find no clear solutions but make me confused. The comment of function truncate_inode_pages says that after it return, the nrpages may not be zero.

My understanding is: the page reclaime migth still in the
process of deletion of the page. Jan Kara once post a patch, which use spin_lock to sync the radix tree and nrpages. This kernel already contains this patch. Then problem come: When kernel hangs, the nrpages is not a small number like 1 or 2, but a bigger one, more than 500 or 700! So I think even we take some sync measures before clear inode, the function truncate_inode_pages together with other reclaim functions failed to set nrpages to zero. By dump the vmcore, I also find the radix tree is also not empty but with some slots left.

Then I think:
1. The fault might happen at pagevec_lookup, which return no
page even the radix tree is in fact not empty. Because lookup uses the rcu lock, is it possible a race condition
happened in the lookup process and lead the function return
unexpectedly? If possiable, how dose it happened ?
2. I find Johannes Weiner post a
patch(http://www.spinics.net/lists/linux-fsdevel/msg72395.html), which has following code:

(quoted diff) +12 -0
+	if (nrpages || nrshadows) {
+		/*
+		 * As truncation uses a lockless tree lookup, cycle
+		 * the tree lock to make sure any ongoing tree
+		 * modification that does not see AS_EXITING is
+		 * completed before starting the final truncate.
+		 */
+		spin_lock_irq(&mapping->tree_lock);
+		spin_unlock_irq(&mapping->tree_lock);
+
+		truncate_inode_pages(mapping, 0);
+	}
 
    which wrapped the truncate_inode_pages in function
Quoted 1 line

truncate_inode_pages_final. Does it make sence to my problem ?

This shouldn't be really related. That is specific to Johannes' patch set
adding new special radix tree entries.

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR