mtime not updated in client cache?
Subsystems (2)
The following code snippet run on a 2.6.13.4 (+CITI patch) client shows that mtime doesn't get updated unless fclose() is called before stat, or O_DIRECT is used. In 2.4, the stat would return an updated mtime. Is this a bug, or expected cache behavior?
Thanks,
Vince
#include <stdio.h>
#include <time.h>
#include <sys/types.h>
#include <unistd.h>
#include <sys/stat.h>
int main(int argc, char **argv) {
time_t tm;
struct stat statinfo;
FILE *fp = fopen("testfile", "w");
sleep(2);
tm = time(NULL);
fwrite("X", 1, 1, fp);
fflush(fp);
stat("testfile", &statinfo);
if (statinfo.st_mtime < tm) {
printf("Bug: write time = %u, mtime = %u\n", tm, statinfo.st_mtime);
} fclose(fp);
return 0;
}
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 3 lines · Vince Busam
The following code snippet run on a 2.6.13.4 (+CITI patch) client shows that mtime doesn't get updated unless
fclose()is called before stat, orO_DIRECTis used. In 2.4, the stat would return an updated mtime. Is this a bug, or expected cache behavior?
That is a known issue that should be fixed in 2.6.15-rcX
Cheers,
Trond
Quoted 29 lines
Thanks,
Vince#include <
stdio.h>
#include <time.h>
#include <sys/types.h>
#include <unistd.h>
#include <sys/stat.h>int main(int argc, char **argv) { time_t tm; struct stat statinfo; FILE *fp = fopen("testfile", "w"); sleep(2); tm = time(NULL); fwrite("X", 1, 1, fp); fflush(fp); stat("testfile", &statinfo); if (statinfo.st_mtime < tm) { printf("Bug: write time = %u, mtime = %u\n", tm, statinfo.st_mtime); } fclose(fp); return 0; }
Quoted 8 lines
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 7 lines · Trond Myklebust
Quoted 1 line · Vince Busam
The following code snippet run on a 2.6.13.4 (+CITI patch) client shows that mtime doesn't get updated unless
fclose()is called before stat, orO_DIRECTis used. In 2.4, the stat would return an updated mtime. Is this a bug, or expected cache behavior?That is a known issue that should be fixed in 2.6.15-rcX
Awesome. Is there a patch I can help test out?
Vince
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 8 lines · Vince Busam
Quoted 7 lines · Trond Myklebust
Quoted 3 lines · Vince Busam
The following code snippet run on a 2.6.13.4 (+CITI patch) client shows that mtime doesn't get updated unless
fclose()is called before stat, orO_DIRECTis used. In 2.4, the stat would return an updated mtime. Is this a bug, or expected cache behavior?
Quoted 4 lines
Quoted 2 lines
That is a known issue that should be fixed in 2.6.15-rcX
Awesome. Is there a patch I can help test out?
The standard 2.6.15-rc3 patch from ftp.kernel.org should suffice. There is both an NFS_ALL and a CITI_ALL that you can apply on top of it if you like, but the straight 2.6.15-rc3 should already fix the mtime issue you reported.
Cheers,
Trond
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 13 lines · Trond Myklebust
Quoted 11 lines · Vince Busam
Quoted 8 lines · Trond Myklebust
Quoted 1 line · Vince Busam
The following code snippet run on a 2.6.13.4 (+CITI patch) client shows that mtime doesn't get updated unless
fclose()is called before stat, orO_DIRECTis used. In 2.4, the stat would return an updated mtime. Is this a bug, or expected cache behavior?That is a known issue that should be fixed in 2.6.15-rcX
Quoted 8 lines
Quoted 1 line
Awesome. Is there a patch I can help test out?
The standard 2.6.15-rc3 patch from ftp.kernel.org should suffice. There is both an
NFS_ALLand aCITI_ALLthat you can apply on top of it if you like, but the straight 2.6.15-rc3 should already fix the mtime issue you reported.
I tried 2.6.15-rc5 (withoug NFS_ALL and CITI_ALL), and it exhibits the same issue as demonstrated by the above referenced code.
Vince
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 2 lines · Vince Busam
I tried 2.6.15-rc5 (withoug
NFS_ALLandCITI_ALL), and it exhibits the same issue as demonstrated by the above referenced code.
<looks at code>
Yep. Your code contains a wrong assumption: fflush() is not guaranteed to immediately update mtime/ctime since it does not sync data to disk. The same is true of the posix "write()" function.
If you add the line fsync(fileno(fp)) after the call to fflush(), then the mtime will be correctly updated.
Cheers,
Trond
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 12 lines · Peter Staubach
Quoted 9 lines · Trond Myklebust
<looks at code>
Yep. Your code contains a wrong assumption:
fflush()is not guaranteed to immediately update mtime/ctime since it does not sync data to disk. The same is true of the posix "write()" function.If you add the line fsync(fileno(fp)) after the call to
fflush(), then the mtime will be correctly updated.
Quoted 5 lines
Perhaps I am missing something, but fsync(2) should not cause the mtime/ctime
to change. The write(2) call should. Sequences such as stat(&a)/write()/stat(&b)
should result b.st_mtime != a.st_mtime.
If you are caching the write on the client, then the server will not update the mtime on the file.
OTOH, I do not like the idea of updating mtime on the client, since there is no guarantee that the client and server are synchronised (you can easily end up with the mtime jumping forwards and backwards).
One alternative would be to flush dirty data to the server on every call to stat(), but that would slow it down considerably.
Cheers,
Trond
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 9 lines · Trond Myklebust
<looks at code>
Yep. Your code contains a wrong assumption:
fflush()is not guaranteed to immediately update mtime/ctime since it does not sync data to disk. The same is true of the posix "write()" function.If you add the line fsync(fileno(fp)) after the call to
fflush(), then the mtime will be correctly updated.
Perhaps I am missing something, but fsync(2) should not cause the mtime/ctime to change. The write(2) call should. Sequences such as stat(&a)/write()/stat(&b) should result b.st_mtime != a.st_mtime.
Thanx...
ps
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 27 lines · Trond Myklebust
Quoted 10 lines · Peter Staubach
Quoted 6 lines · Trond Myklebust
<looks at code>
Yep. Your code contains a wrong assumption:
fflush()is not guaranteed to immediately update mtime/ctime since it does not sync data to disk. The same is true of the posix "write()" function.Quoted 10 lines
Quoted 5 lines
If you add the line fsync(fileno(fp)) after the call to
fflush(), then the mtime will be correctly updated.Perhaps I am missing something, but fsync(2) should not cause the mtime/ctime to change. The write(2) call should. Sequences such as stat(&a)/write()/stat(&b) should result b.st_mtime != a.st_mtime.
If you are caching the write on the client, then the server will not update the mtime on the file.
Quoted 7 lines
OTOH, I do not like the idea of updating mtime on the client, since there is no guarantee that the client and server are synchronised (you can easily end up with the mtime jumping forwards and backwards).
One alternative would be to flush dirty data to the server on every call to
stat(), but that would slow it down considerably.
Yes, to this last, but this is how most NFS implementations that I have seen conform to the requirment that write(2) cause the mtime to be updated. The use of UNSTABLE NFSv3/NFSv4 writes can help as well as limiting the amount of dirty data that the client allows to be cached.
The client should _definitely_ not be setting the mtime itself. The file times are always managed by the server.
Thanx...
ps
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 8 lines · Peter Staubach
Quoted 3 lines
One alternative would be to flush dirty data to the server on every call to
stat(), but that would slow it down considerably.Yes, to this last, but this is how most NFS implementations that I have seen conform to the requirment that write(2) cause the mtime to be updated. The use of UNSTABLE NFSv3/NFSv4 writes can help as well as limiting the amount of dirty data that the client allows to be cached.
My main concern is that it will hit commands like 'ls -l' _hard_. The latter will, for instance, be forced to wait until all pending writes on all files in the directory have been flushed out. I'm not sure that strict POSIX compatibility is worth that kind of hit.
Anyhow, the attached patch implements it. Lets try it out, and see how it does. Definitely _not_ a 2.6.15 candidate, though.
Quoted 3 lines
The client should _definitely_ not be setting the mtime itself. The file times
are always managed by the server.
Agreed.
Cheers,
Trond
NFS: Make stat() return updated mtimes after a write()
The SuS states that a call to write() will cause mtime to be updated on the file. In order to satisfy that requirement, we need to flush out any cached writes in nfs_getattr().
Speed things up slightly by not committing the writes.
---
fs/nfs/inode.c | 2 ++ fs/nfs/write.c | 23 ++++++++++++----------- include/linux/nfs_fs.h | 1 + 3 files changed, 15 insertions(+), 11 deletions(-)
fs/nfs/inode.c +2 -0
@@ -952,6 +952,8 @@ int nfs_getattr(struct vfsmount *mnt, stint need_atime =NFS_I (inode)->cache_validity &NFS_INO_INVALID_ATIME ;int err; +/* Flush out writes to the server in order to update c/mtime */ +nfs_sync_inode (inode,0 ,0 ,FLUSH_WAIT |FLUSH_NOCOMMIT );if (__IS_FLG (inode,MS_NOATIME )) need_atime =0 ;else if (__IS_FLG (inode,MS_NODIRATIME ) &&S_ISDIR (inode->i_mode))
fs/nfs/write.c +12 -11
@@ -1377,22 +1377,23 @@ int nfs_commit_inode(struct inode *inodeint nfs_sync_inode (struct inode *inode,unsigned long idx_start,unsigned int npages,int how) { -int error, - wait; +int nocommit = how &FLUSH_NOCOMMIT ; +int wait = how &FLUSH_WAIT ; +int error; - wait = how &FLUSH_WAIT ; - how &= ~FLUSH_WAIT ; + how &= ~(FLUSH_WAIT |FLUSH_NOCOMMIT );do { - error =0 ; -if (wait) +if (wait) { error =nfs_wait_on_requests (inode, idx_start, npages); -if (error ==0 ) - error =nfs_flush_inode (inode, idx_start, npages, how); -#if defined(CONFIG_NFS_V3 ) || defined(CONFIG_NFS_V4 ) -if (error ==0 ) +if (error !=0 ) +continue ; + } + error =nfs_flush_inode (inode, idx_start, npages, how); +if (error !=0 ) +continue ; +if (!nocommit) error =nfs_commit_inode (inode, how); -#endif }while (error >0 );return error; }
include/linux/nfs_fs.h +1 -0
@@ -62,6 +62,7 @@#define FLUSH_STABLE 4/* commit to stable storage */ #define FLUSH_LOWPRI 8/* low priority background flush */ #define FLUSH_HIGHPRI 16/* high priority memory reclaim flush */ +#define FLUSH_NOCOMMIT 32/* Don't send the NFSv3/v4 COMMIT */ #ifdef __KERNEL__
Quoted 2 lines
Anyhow, the attached patch implements it. Lets try it out, and see how it does. Definitely _not_ a 2.6.15 candidate, though.
Without getting into the argument of whether this is the right thing to do or not, I will verify that this patch resolved the "problem" and that the example code acts as it does on 2.4 and Solaris.
Thanks,
Vince
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs