help with understanding evict inode functionality
Quoted 3 lines · Olga
[...]
In NFS development, weâve been seeing a race between reopening of the same file and evict inode code and unable figure out how to prevent such race. When file is opened for the first timed and then closed. As the last reference is dropped and
iput_final()callsevict()which will call filesystem specificevict_inode()code. As thatâs happening a new open for the same file is happening and causes problems.
Recently we encountered similar situation with NFS, that is, after evict_inode() & iget_locked() we were getting the same inode (same i->i_ino) and without I_NEW bit set.
Quoted 2 lines
Also can somebody suggest how to debug VFS code, as putting printks generates output for the local filesystem code as well.
You can call printk() only if current process is "nfsd" (at least, that is what we did).
Quoted 4 lines
Thank you.
[...]
Signature · 12 lines
Best regards, Leonid Fedorenchik Software Engineer Paragon Software Group Skype: leonid.fedorenchik http://www.paragon-software.com -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 6 lines · Leonid Fedorenchik
On Thu, 1 Oct 2015 22:54:08 +0000
"Kornievskaia, Olga" <[email protected]> wrote:
Quoted 6 lines
Quoted 3 lines
[...]
In NFS development, weâve been seeing a race between reopening of the same file and evict inode code and unable figure out how to prevent such race. When file is opened for the first timed and then closed. As the last reference is dropped and
iput_final()callsevict()which will call filesystem specificevict_inode()code. As thatâs happening a new open for the same file is happening and causes problems.Recently we encountered similar situation with NFS, that is, after
evict_inode()&iget_locked()we were getting the same inode (samei->i_ino) and withoutI_NEWbit set.
Thanks for the comment! When I tried to print inode state flags in the NFS code for opening a file I’ve never had “I_NEW” set. Not sure why.
Quoted 6 lines
Quoted 3 lines
Also can somebody suggest how to debug VFS code, as putting printks generates output for the local filesystem code as well.
You can call
printk()only if current process is "nfsd" (at least, that is what we did).
Thanks for the suggestion. I’ll try that.
Again a general question to the community about inodes and eviction. I was under the general impression that when the file is closed the inode sticks around just in case the file is reopened again. Then when resources are constrainted some background thread evicts unused inodes. Is that understanding correct? In that case what is that background process name?
Quoted 15 lines
Quoted 4 lines
Thank you.
[...]
Signature · 7 lines
Best regards, Leonid Fedorenchik Software Engineer Paragon Software Group Skype: leonid.fedorenchik http://www.paragon-software.com
Quoted 6 lines · Olga
Again a general question to the community about inodes and eviction. I was under the general impression that when the file is closed the inode sticks around just in case the file is reopened again. Then when resources are constrainted some background thread evicts unused inodes. Is that understanding correct?
I think you're confused about what evict_inode() does. The i_count field tracks how many in-memory references the inode has. Each iput() decrements i_count. Once i_count goes to zero, there are no more open file descriptors for that inode.
Then there is also i_nlink. This is the number of directory entries which point at the inode on disk. When you unlink() a file, this drops the i_nlink on the inode. When i_nlink drops to zero, then the inode is no longer referenced by any directory entry. But, this doesn't mean we can actually release the inode and the blocks associated with the file --- Unix semantics is that you can have an open file descriptor on an unlinked file, and when the last fd is closed, only then does the inode get released.
evict_inode() is what happens when i_nlink *and* i_icount hits zero. So it is only then that the local disk file system can actually release the inode and blocks associated with that inode.
So by the time the file system's evict_inode() is called, the inode is not coming back. With apologies to Monty Python, the inode is no more; it has ceased to be. It's expired and gone to meet its maker..... It is an ex-inode. :-)
Hence, there is no point trying to worry about what hapens if the file is reopened again, since the original inode is *gone*. You could create a new file with the same file name, but none of the resources associated with the old inode need to be preserved for the newly created file.
Cheers,
- Ted
P.S. Things do get a little complicated with non-native Unix file systems which don't obey Posix semantics, or if you are dealing with a pre-NFSv4 protocol which quaintly assumes that you can have a stateless networked file system protocol when a file system is inherently stateful. For more information, see:
Quoted 3 lines · Theodore Ts'o
evict_inode()is what happens wheni_nlink*and*i_icounthits zero. So it is only then that the local disk file system can actually release the inode and blocks associated with that inode.
That's not true. Evict is called when we want to remove an inode from memory. i_count needs to be zero for that, but i_nlink doesn't as we don't cache inodes forever. When we drop i_count and it reaches zero iput_final() is called, which decides depending on ->drop_inode to either evict the inode or keep it on the LRU.
Quoted 5 lines
Hence, there is no point trying to worry about what hapens if the file is reopened again, since the original inode is *gone*. You could create a new file with the same file name, but none of the resources associated with the old inode need to be preserved for the newly created file.
And this isn't quite true either - this particular struct inode instance might not come back, but we need to synchronize against a racing iget if i_nlink wasn't zero. This is done with the I_FREEING bit and the bit wake up on __I_NEW.
Quoted 6 lines · Christoph Hellwig
That's not true. Evict is called when we want to remove an inode from memory.
i_countneeds to be zero for that, buti_nlinkdoesn't as we don't cache inodes forever. When we dropi_countand it reaches zeroiput_final()is called, which decides depending on ->drop_inode to either evict the inode or keep it on the LRU.
Sorry, of course Cristoph is absolutely correct.
- Ted
Jan Kara
reviewerQuoted 6 lines · Olga
Quoted 5 lines
Quoted 3 lines
Also can somebody suggest how to debug VFS code, as putting printks generates output for the local filesystem code as well.
You can call
printk()only if current process is "nfsd" (at least, that is what we did).
Quoted 8 lines
Thanks for the suggestion. I’ll try that.
Again a general question to the community about inodes and eviction. I was under the general impression that when the file is closed the inode sticks around just in case the file is reopened again. Then when resources are constrainted some background thread evicts unused inodes. Is that understanding correct? In that case what is that background process name?
Yes, your understanding is correct. Generally, slab caches (and thus unused inodes) get reclaimed from kswapd processes. It can happen though, that they are reclaimed basically from any process doing memory allocation when there is shortage of free memory (this is called direct reclaim).
Honza
Signature · 6 lines
Jan Kara <[email protected]> SUSE Labs, CR -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html