gettimeofday running backwards on 2.4.20

[ Please cc [email protected] on any replies. Thank you! ]

A few months ago I noticed gettimeofday running backwards on dual Pentium II and dual Pentium Pro systems with 2.4.18. Based on postings made to linux-kernel in June of 2002, I upgraded to 2.4.20 and the problem seemed to go away:

http://groups.google.com/[email protected]

Just recently my NAS benchmarks and MPI latency tests showed bizarre results, so I pulled out my test program and am seeing the same problems again. It seems that roughly 50 in 1 million calls go backwards, even with 2.4.20.

Occasionally it is 4295 seconds (very consistently that value), but most of them are just a few microseconds backwards. No error code is returned from the system call either. No NTP or rdate type services are running.

My test program can be seen here:

http://www.swcp.com/~hudson/gettimeofday.c

Interestingly, it only happens on the compute nodes with NFS root. The service node has booted from a local SCSI disk and is serving roughly 140 compute nodes without any timing bugs.

service:~/tflop-linux: ssh node-181 uname -a \; cat /proc/cpuinfo Linux node-181 2.4.20 #73 SMP Mon Apr 21 14:06:49 MDT 2003 i686 unknown
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 3
model name : Pentium II (Klamath)
stepping : 2
cpu MHz : 333.356
cache size : 512 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr
bogomips : 665.19

processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 3
model name : Pentium II (Klamath)
stepping : 2
cpu MHz : 333.356
cache size : 512 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr
bogomips : 665.19

Trammell

  -----|----- [email protected]                   W 240-283-1700
*>=====[]L\   [email protected]                   M 505-463-1896
'     -'-`-   http://www.swcp.com/~hudson/                    KC5RNF

It's happening on my VAIO laptop as well, with a similar frequency. The size of the typical regression dropped by roughly an order of magnitude between 2.4.19 and 2.4.20, from several 10's of ms to a few ms.

There needs to be a final idiot check in there, as the last thing that happens in sys_gettimeofday, to catch and prevent such regressions in a brutally stupid way, in addition to the fancy algorithm that tries to prevent them, and has never fully succeeded in doing that.

A reasonable algorithm is: if time goes backwards by less than a second, it wasn't somebody changing the clock, so return the previous timeofday, otherwise, save the new value for the next test.

Applications like games (but not only games) can get pretty messed up by a timeofday that jumps backwards every couple of seconds.

Regards,

Daniel

It's a foolish game that doesn't implement your monotonicity algorithm itself...

-- Jamie

Maybe so, but on the other hand, game authors could, with some justification, accuse kernel developers of being the foolish ones.

There is already code in there that is supposed to fix up the timer: it doesn't work very well.

I wonder if SUS or whatnot has anything to say about gettimeofday being monotonic, so long as the user does not change the time.

Regards,

Daniel