Skip to content

x86_64: reliable TSC-based gettimeofday

[email protected] lkml 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest9 patches 52 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/9Fix HPET init racearch/x86_64/kernel/apic.c+6 -45 replies
2/9Remove the support for the VXTIME_PMTMR timer modearch/x86_64/kernel/apic.c arch/x86_64/kernel/pmtimer.c arch/x86_64/kernel/time.c +1 more+1 -1045 replies
3/9Remove the support for the VXTIME_HPET timer modearch/x86_64/kernel/time.c arch/x86_64/kernel/vsyscall.c include/asm-x86_64/vsyscall.h+0 -800 replies
4/9Remove the TSC synchronization on SMP machinesarch/x86_64/kernel/smpboot.c+0 -21919 replies
5/9Add all the necessary structures to the vsyscall pagearch/x86_64/kernel/vmlinux.lds.S arch/x86_64/kernel/vsyscall.c include/asm-x86_64/vsyscall.h+40 -141 reply
6/9Add the "Master Timer"arch/x86_64/kernel/time.c+143 -32 replies
7/9Adapt the time initialization codearch/x86_64/kernel/smpboot.c arch/x86_64/kernel/time.c include/linux/time.h+130 -312 replies
8/9Add time_update_mt_guess()arch/x86_64/kernel/apic.c arch/x86_64/kernel/time.c+183 -252 replies
9/9Make use of the Master Timerarch/x86_64/kernel/time.c arch/x86_64/kernel/vsyscall.c+166 -1345 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [patch 0/9] x86_64: reliable TSC-based gettimeofday

TSC-based x86_64 timekeeping implementation
===========================================
by Vojtech Pavlik and Jiri Bohac

This implementation allows the current time to be approximated by reading the CPU's TSC even on SMP machines with unsynchronised TSCs. This allows us to have a very fast gettimeofday() vsyscall on all SMP machines supporting the RDTSCP instruction (AMD) or having synchronised TSCs (Intel).

Inter-CPU monotonicity can not, however, be guaranteed in a vsyscall, so vsyscall is not used by default. Still, the syscall version of gettimeofday is a lot faster using the TSC approximation instead of other hardware timers.

At boot, either the PM timer or HPET (preferred) is chosen as the "Master Timer" (MT), from which all the time is calculated. As reading either of these is slow, we want to approximate it using the TSC.

Each CPU updates its idea of the real time in update_timer_caches() called from the LAPIC ISR. This function reads the real value of the MT and updates the per-CPU timekeeping variables accordingly. Each CPU maintains its own "tsc_slope" (a ratio of the MT and TSC frequencies) and a couple of offsets, allowing us to guess (using guess_mt()) the value of the MT at any time on any CPU. All this per-cpu data is kept in the vxtime structure.

The gettimeofday (both the syscall and vsyscall versions) use the approximated value of the MT to calculate the time elapsed since the last timer interrupt. For this purpose, vxtime.mt_wall holds the value of the MT at the last timer interrupt.

During a CPU frequency change, we cannot trust the TSCs. Therefore, when we get the pre-change notification, we switch to using the hardware Master Timer instead of the approximation by setting a flag in vxtime.tsc_invalid. After the post-change notification we keep using the hardware MT for a while, until the approximation becomes accurate again.

When strict inter-CPU monotonicity is not needed, the vsyscall version of gettimeofday may be forced using the "nomonotonic" command line parameter. gettimeofday()'s monotonicity is guaranteed on a single CPU even with the very fast vsyscall version. Across CPUs, the vsyscall version of gettimeofday is not guaranteed to be monotonic, but it should be pretty close. Currently, we get errors of tens/hundreds of microseconds.

We rely on neither the LAPIC timer nor the main timer interrupts being called in regular intervals (although a little modification would improve the MT approximation in this case), so we're basically ready for a tickless kernel.

A patch series follows. Comments welcome.

--
Jiri Bohac <[email protected]>
SUSE Labs, SUSE CZ