nanosecond file timestamp resolution in filesystems, GNU make, etc.

Thanks for mentioning this, as I don't normally read linux-kernel. I'll add my comments below and CC: to linux-kernel. Please reply to me as well as to linux-kernel if you want me in on the thread.

I see this as a real flaw. Several programs (and not just GNU make) rely on file timestamps not being altered. For example, GNU diff uses file timestamps to decide whether two files might be the same file; if their timestamps differ, then they must not be the same file. If a file's timestamp might snap back to the previous second, a nanosecond-aware GNU diff will go astray. Similarly, GNU tar uses file timestamps to decide whether a file has changed and needs to be dumped; if the file timestamp jumps back, a nanosecond-aware GNU tar will erroneously decide that the file has changed and will dump it unnecessarily.

Rounding is even worse. GNU Make assumes truncation, i.e. it assumes that a timestamp is truncated (floored, actually) when it is stored on a non-nanosecond-aware filesystem.

Can't you truncate/floor to filesystem timestamp resolution immediately, i.e., before the inode is flushed? That would address the problems that I see.

Partly because it was there. But also because it avoids some bugs when "make" assumes that a file is up-to-date because its timestamp equals some other file's timestamp. E.g.:

cp foo1.c foo.c
make foo.o
cp foo2.c foo.c
make foo.o

This works only if timestamps have sufficient resolution. Admittedly this small example is strained, but I hope you get the idea.

We used this trick in one place in GNU make (look for the comment in remake.c that says "Avoid spurious rebuilds due to low resolution time stamps") but it is an application-specific hack; I'm dubious that it is a good idea in general-purpose filesystem code.

My personal experience is that it's hard to read and write code that futzes with timestamps of various resolutions, and there is a real advantage to sticking with a simple rule that always takes the floor when going to a lower-precision timestamp, even if that rule has suboptimal results in some cases.

I have to disagree. The whole point of accurate timestamps is that a program can reliably ask "are my assumptions about the contents of this file still valid?", "do I have to read the file to revalidate my assumptions?"

When you don't have accurate timestamps, or resolutions are mixed, then it's not possible to answer this question. The only correct behaviour for a program, such as a cacheing dynamic web server, a cacheing JIT compiler or something like Make, is to round the timestamp _up_.

Which is fine so long as you can write this in the application code:

    if (ts_nanoseconds == 0)
        ts_nanoseconds = 1e9-1;

That's a rare enough occurrence when timestamps have nanosecond accuracy that the the glitch is not a problem.

Unfortunately that application code breaks when the filesystem may have timestamps with resolution better than 1 second, but worse than 1 nanosecond. Then the application just can't do the right thing, unless it knows what rounding was applied by the kernel/filesystem, so it can change that rounding in a safe direction.

So for applications which actually _depend_ on accurate timestamps for reliability, I see only two valid things for the kernel to do:

1. Round timestamps _up_.

2. Or, round timestamps down _and_ store the rounding resolution in struct stat, in addition to the timestamp.

I'm in favour of 1.

(With Paul's suggestion of just rounding down without telling me when that happened, AFAICT my _reliable_ cacheing applications must simply ignore the nanoseconds field, which is a bit unfortunate isn't it?)

-- Jamie