New nanosecond stat patch for 2.5.44
Quoted 12 lines · Andreas Dilger
Quoted 10 lines · Andi Kleen
Move
time_tmembers in struct stat to struct timespec and allow subsecond timestamps for files. Too big to post on the list, because it edits a lot of file systems and drivers in a straight forward way.This is required for reliable "make" on fast computers.
File systems that support nsec storage are currently: XFS, JFS, NFSv3 (if the filesystem on the server supports it), VFAT (not quite nanosecond), CIFS (unit in 100ns which is above what linux supports), SMBFS (for newer servers)
Quoted 7 lines
Two notes I might make about this:
1) It would be good if it were possible to select this with a config option (I don't care which way the default goes), so that people who don't need/care about the increased resolution don't need the extra space in their inodes and minor extra overhead. To make this a lot easier to code, having something akin to theinode_update_time()which does all of the i_[acm]time updates as appropriate.
You're joking right? That's twelve bytes of more state per struct inode and I bet even with the most insidious micro benchmark you won't be able to detect a difference in speed from the basic manipulation.
What could hurt a bit is that the "only flush atime once a second" optimization is gone currently. The right way to address that would be a mount option "atime_flush_interval", not a CONFIG.
Quoted 8 lines
2) Updating
i_atimebased on comparing the nsec timestamp is going to be a killer. I think AKPM saw dramatic performance improvements when he changed the code to only do the update once/second, and even though you are "only" updating the atime if the times are different, in practise this will be always. Even without the "per superblock interval" you suggest we should probably only update the atime once a second (I don't think anything is keyed off such high resolution atimes, unlike make and mtime/ctime).
Again I wrote about this in my original mail. Please see ftp://ftp.firstfloor.org/pub/ak/v2.5/nsec.notes
Basically I agree with you that it's a problem (although "killer" seems to be an exaggeration to me). The right solution IMHO would be to implement a new super block field / mount option that specifies that atime flush (basically generalized noatime). Then you can say you only want it flushed every 60s and the result will be much faster than what we have currently.
Some file systems already implement intelligent atime flushing (like XFS) and they don't need it.
But I didn't want to mix such a patch into the big patchkit. When the nsec patchkit is integrated and benchmarks show it is a problem I will submit a follow up patch instead.
Quoted 7 lines
3) The fields you are usurping in struct stat are actually there for the Y2038 problem (when
time_twraps). At least that's what Ted said when we were looking into nsec times for ext2/3. Granted, we may all be using 64-bit systems by 2038... I've always thought 64 bits is much to large fortime_t, so we could always use 20 or 30 bits for sub-second times, and the remaining bits for extendingtime_tat the high end, and mask those off for now, but that is a separate issue...
I wrote about this in my original notes (perhaps I should repost them, I think they are still on the ftp server) For year 2038 we will need lots of new syscalls: new time(2), new gettimeofday(2) and lots of others. When all these are added then a new stat isn't that big a problem. Also glibc currently doesn't know how to use these fields for y2038, so all user programs need to be relinked anyways. Again when that happens it's no big issue to add a new stat. I bet when y2038 comes there will be other reasons for a new stat too. So it's fine to reuse these fields.
The make problem is much more pressing.
-Andi
Quoted 9 lines · Chris Friesen
Quoted 6 lines · H. Peter Anvin
We probably need to revamp struct stat anyway, to support a larger
dev_t, and possibly a largerino_t(we should account for 64-bitino_tat least if we have to redesign the structure.) At that point I would really like to advocate forint64_tts_secanduint32_tts_nsecand quite possibly aint32_tts_taideltato deal with leap seconds... I'd personally like struct timespec to look like the above everywhere.
Quoted 3 lines
For filesystems can we get away with just the 64-bit nanoseconds? By my calculations that gives something like 584 years--do we need to worry about files older than that?
The current timestamps on 32bit systems are 32bit. 64bit nanoseconds would take the same room as 32bit second + 32bit nanosecond. And it would be incompatible with current glibc (which the additional nanosecond fields are perfectly compatible - they are zeroed currently). Also glibc would need to convert it to a timespec for Solaris compatbility anyways and need an unnecessary division for that.
The same thing applies to file system storage. Storing in nanoseconds (like e.g. NTFS or CIFS do - they store 64bit in 100ns units since 1601) would require slow divisions to convert from the user visible format, needs the same space and has no advantage as far as I can see.
-Andi
Quoted 11 lines · Andi Kleen
Quoted 6 lines · Andreas Dilger
1) It would be good if it were possible to select this with a config option (I don't care which way the default goes), so that people who don't need/care about the increased resolution don't need the extra space in their inodes and minor extra overhead. To make this a lot easier to code, having something akin to the
inode_update_time()which does all of the i_[acm]time updates as appropriate.You're joking right? That's twelve bytes of more state per struct inode and I bet even with the most insidious micro benchmark you won't be able to detect a difference in speed from the basic manipulation.
Except that people have a lot of inodes in their slab caches... It's not so much the processing overhead as the extra memory. struct inode is bloated enough without adding more into it that isn't necessarily useful for some people (people who don't have lots of RAM, or don't use any filesystems which support the higher resolution, or are slow enough that compiles don't have problems, or don't compile at all)...
Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/