What trigge fsync of file on last close of the open inode?
What triggers flush/fsync of dirty pages on last (file) close of inode? I was hunting through the sys_close code and did not see a call to fsync or filemap_write_and_wait there. Is it something done in libc above the vfs?
Someone had reported a problem with a writepages call coming in on with no open files (so presumably the file was closed, with dirty pages not written).
Quoted 1 line · Steve French
What triggers flush/fsync of dirty pages on last (file) close of inode?
Nothing. A close doesn't imply fsync. Dirty data is eventually written by pdflush, if something else (memory pressure, maybe) doesn't do it first.
Quoted 3 lines
I was hunting through the
sys_closecode and did not see a call to fsync orfilemap_write_and_waitthere. Is it something done in libc above the vfs?
No
Quoted 3 lines
Someone had reported a problem with a writepages call coming in on with no open files (so presumably the file was closed, with dirty pages not written).
This is normal behavior for most file systems. I thought cifs protected this by flushing dirty data in cifs_close. I don't think any data should be dirtied after cifs_close is called (on the last open file handle).
Signature · 2 lines
David Kleikamp IBM Linux Technology Center
Quoted 8 lines · Dave Kleikamp
Quoted 2 lines
Someone had reported a problem with a writepages call coming in on with no open files (so presumably the file was closed, with dirty pages not written).
This is normal behavior for most file systems. I thought cifs protected this by flushing dirty data in
cifs_close. I don't think any data should be dirtied aftercifs_closeis called (on the last open file handle).
I found it ...
cifs exports flush, filp_close calls flush (before calling close)
cifs_flush calls filemap_fdatawrite
May be a case in which filemap_fdatawrite returns before the write(s) is sent to the vfs and write races with close (although cifs will defer a file close if a write is pending on that handle)?
Quoted 4 lines · Steve French
Quoted 3 lines · Dave Kleikamp
Quoted 3 lines
Someone had reported a problem with a writepages call coming in on with no open files (so presumably the file was closed, with dirty pages not written).
So is the problem that you're getting a cifs_writepages() call after cifs_close() returns?
fwiw, 9 out of 10 brains would be less confused if cifs_close() was called cifs_release().
Quoted 3 lines
May be a case in which
filemap_fdatawritereturns before the write(s) is sent to the vfs and write races with close (although cifs will defer a file close if a write is pending on that handle)?
Are writes to mmap()ed regions involved at all? They lead to pages being dirtied at unmapping and eventually hitting ->writepage, potentially after ->flush and ->release have been called.
I imagine you could force writeback of dirty pages in ->release so that you don't wait for writeback to come around and hit them. Heck, it might be doing this already. I didn't look very hard :).
- z
Quoted 8 lines · Zach Brown
Quoted 4 lines · Steve French
Quoted 3 lines · Dave Kleikamp
Quoted 3 lines
Someone had reported a problem with a writepages call coming in on with no open files (so presumably the file was closed, with dirty pages not written).
So is the problem that you're getting a
cifs_writepages()call aftercifs_close()returns?
If I understand Steve, the problem may be that cifs_writepages() is called after cifs_flush(). cifs_flush() is called earlier than cifs_close().
Quoted 2 lines
fwiw, 9 out of 10 brains would be less confused if
cifs_close()was calledcifs_release().
That confused me a bit, but I still would have missed the cifs_flush bit.
Quoted 7 lines
Quoted 3 lines
May be a case in which
filemap_fdatawritereturns before the write(s) is sent to the vfs and write races with close (although cifs will defer a file close if a write is pending on that handle)?Are writes to mmap()ed regions involved at all? They lead to pages being dirtied at unmapping and eventually hitting ->writepage, potentially after ->flush and ->release have been called.
->flush does filemap_fdatawrite(), which should take care of any dirty pages (I believe).
Quoted 3 lines
I imagine you could force writeback of dirty pages in ->release so that you don't wait for writeback to come around and hit them. Heck, it might be doing this already. I didn't look very hard :).
->flush does. I'm not sure about ->release. I'm not sure if a write could squeeze in between ->flush and ->release.
Signature · 2 lines
David Kleikamp IBM Linux Technology Center
Quoted 13 lines · Steve French
Quoted 10 lines · Dave Kleikamp
Quoted 4 lines
Someone had reported a problem with a writepages call coming in on with no open files (so presumably the file was closed, with dirty pages not written).
This is normal behavior for most file systems. I thought cifs protected this by flushing dirty data in
cifs_close. I don't think any data should be dirtied aftercifs_closeis called (on the last open file handle).I found it ...
Quoted 7 lines
cifs exports flush,
filp_closecalls flush (before calling close)
cifs_flushcallsfilemap_fdatawriteMay be a case in which
filemap_fdatawritereturns before the write(s) is sent to the vfs and write races with close (although cifs will defer a file close if a write is pending on that handle)?
Steve,
Here's a comment I found in the NFSv4 code.... might be relevent.
Quoted 1 line
From
/usr/src/linux/fs/nfs/nfs4proc.c
/*
* It is possible for data to be read/written from a mem-mapped file
* after the sys_close call (which hits the vfs layer as a flush).
* This means that we can't safely call nfsv4 close on a file until
* the inode is cleared.
Quoted 7 lines
Quoted 4 lines
Are writes to mmap()ed regions involved at all? They lead to pages being dirtied at unmapping and eventually hitting ->writepage, potentially after ->flush and ->release have been called.
->flush does
filemap_fdatawrite(), which should take care of any dirty pages (I believe).
Not if they come from writes to mmap()ed pages. The page cache won't see them as dirty until they're unmapped and the page table dirty bits are transfered to the page cache page flags.
unmapping to transfer dirty tracking from page tables to the page cache is done in a few places. look for calls to unmap_mapping_range() like this one from the O_DIRECT write path:
/*
* If it's a write, unmap all mmappings of the file up-front. This
* will cause any pte dirty bits to be propagated into the pageframes
* for the subsequent filemap_write_and_wait().
*/
if (rw == WRITE) {
write_len = iov_length(iov, nr_segs);
if (mapping_mapped(mapping))
unmap_mapping_range(mapping, offset, write_len, 0);
}I'd hope that dirty tracking is transferred from the page tables to the page cache *before* ->release is called. (put_vma() makes me think this is the case.) So I think you could use unmap_mapping_range() and filemap_fdatawrite() to at least initiate writeback on all dirty pages in ->release.
That is, you should be able to stop ->writepages calls after ->release.
But it's a really bad idea to try and assert that ->writepages can't be
called after ->flush.
But I still don't know if mmap() is involved in his current problem.
- z
Quoted 4 lines · Zach Brown
Quoted 2 lines · Steve French
Dave Kleikamp wrote:
Quoted 10 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Someone had reported a problem with a writepages call coming in on with no open files (so presumably the file was closed, with dirty pages not written).
So is the problem that you're getting a
cifs_writepages()call aftercifs_close()returns?
Probably - I could not see any other path that could cause that cifs error to be logged to dmesg
Quoted 10 lines
Quoted 4 lines
May be a case in which
filemap_fdatawritereturns before the write(s) is sent to the vfs and write races with close (although cifs will defer a file close if a write is pending on that handle)?Are writes to mmap()ed regions involved at all? They lead to pages being dirtied at unmapping and eventually hitting ->writepage, potentially after ->flush and ->release have been called.
I don't think so - but the person reporting it did not give much information, and the error is not something that I have been seeing so I can't draw conclusions about that.
Quoted 4 lines
I imagine you could force writeback of dirty pages in ->release so that you don't wait for writeback to come around and hit them. Heck, it might be doing this already. I didn't look very hard :).
filemap_fdatawrite is called in cifs flush (which I verified is called just before close) and close will be delayed (not sent over the network) if a write is pending on that file handle but there may be two possibilities:
1) the page gets dirty while it is being released (between fput->->cifs_flush and fput->cifs_close)
2) filemap_fdatawrite did not flush the pages fast enough - and needs to be changed to the filemap write and wait variant of the call in this location.
Quoted 26 lines · Jeremy Allison
Quoted 13 lines · Steve French
Quoted 10 lines · Dave Kleikamp
Quoted 4 lines
Someone had reported a problem with a writepages call coming in on with no open files (so presumably the file was closed, with dirty pages not written).
This is normal behavior for most file systems. I thought cifs protected this by flushing dirty data in
cifs_close. I don't think any data should be dirtied aftercifs_closeis called (on the last open file handle).I found it ...
Quoted 7 lines
cifs exports flush,
filp_closecalls flush (before calling close)
cifs_flushcallsfilemap_fdatawriteMay be a case in which
filemap_fdatawritereturns before the write(s) is sent to the vfs and write races with close (although cifs will defer a file close if a write is pending on that handle)?Steve,
Here's a comment I found in the NFSv4 code.... might be relevent.
Quoted 7 lines
Quoted 1 line
From
/usr/src/linux/fs/nfs/nfs4proc.c/*
* It is possible for data to be read/written from a mem-mapped file
* after thesys_closecall (which hits the vfs layer as a flush).
* This means that we can't safely call nfsv4 close on a file until
* the inode is cleared.
That comment needs updating.
In a nutshell the rules are:
- The vm_area_struct (vma) that describes the mmapped area
holds a reference to the struct file that was used in the call to do_mmap().
- Once the the vma that referenced your struct file has been
destroyed (usually via a call to munmap() or via the call to mmput() when the task is destroyed), the reference to the struct file is released in the usual way, via a call to fput().
Note that once the last reference to the struct file disappears, the filesystem is notified by a call to filp->f_op->release().
Once all the struct file that refer to any given inode have been released, you should be able to assume that no-one is reading or writing to its pages (other than perhaps the CIFS client itself?).
Cheers,
Trond