large, spurious[?] TSC skews on AMD 760MPX boards
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 4 lines · [email protected] (Monty) writes:
Ever since getting my first dual Athlon, the system timer was 'not quite right' when running at stock speed. Selects, alarms, etc, had a strange way of firing fractions of a second or several seconds 'too late'. I discovered that overclocking by about 10% made the problem
That points away from the TSC actually. select and alarm use the jiffies clock, which is managed by the PIT timer in the southbridge. AFAIK they never rely on the TSC.
Quoted 17 lines
magically go away. I've never been entirely comfortable doing that, but three dual athlons later (all 760MPX-B2 based boards of different makes), it was always the only way to make the problem disappear and I didn't think more about it.
Now that I'm on #3, it is not stable at the overclock I need to make the system timer problem disappear, so I finally started hunting for the cause. Whenever I run the system stock, I see:
Jul 20 21:48:26 Snotfish kernel: checking TSC synchronization across CPUs: Jul 20 21:48:26 Snotfish kernel: BIOS BUG: CPU#0 improperly initialized, has 6282588 usecs TSC skew! FIXED. Jul 20 21:48:26 Snotfish kernel: BIOS BUG: CPU#1 improperly initialized, has -6282588 usecs TSC skew! FIXED.When the system is running 'properly', that is to say, overclocked:
Jul 21 22:08:01 Snotfish kernel: checking TSC synchronization across CPUs: passed.
Quoted 8 lines
This behavior is reproducable on all three of my 760MPX systems (One Gigabyte GA-7DPXDW-P, and two MSI K7D Master-L). The amount of the reported skew varies in the stock case, but it's always large. Note that once in a blue moon, the system will come up with no TSC skew at stock timings, and the system timer issues seem to disappear.
What is the proper route to go about debugging this problem, as I have it bottled up here in a reproducability cage?
Assuming it is the TSC:
You could write a multithreaded program that polls the TSCs
on your both CPU for a long time and check out the drift rate.
The kernel will try to fix it at boot time, but it cannot do that when the TSCs are drifting later.
One way to work around it would be to boot with "notsc". This will make your gettimeofday() slower and more inaccurate though.
In theory you could resync the TSC in a regularly running timer, but this would probably be quite some overhead.
Assuming it is not:
Something is wrong with your PIT timer in the southbridge. Maybe just run ntpd ?
I know that later AMD chipsets - in particular the 8111 - are somewhat bad time keepers, which makes it a good idea to run NTP always.
-Andi
Quoted 10 lines · Andi Kleen
Quoted 4 lines · [email protected] (Monty) writes:
Ever since getting my first dual Athlon, the system timer was 'not quite right' when running at stock speed. Selects, alarms, etc, had a strange way of firing fractions of a second or several seconds 'too late'. I discovered that overclocking by about 10% made the problem
That points away from the TSC actually. select and alarm use the jiffies clock, which is managed by the PIT timer in the southbridge. AFAIK they never rely on the TSC.
Although I believe you, the timer problem exists only when boot reports the TSC skew.
Quoted 6 lines
Assuming it is the TSC:
You could write a multithreaded program that polls the TSCs
on your both CPU for a long time and check out the drift rate.
The kernel will try to fix it at boot time, but it cannot do that when the TSCs are drifting later.
Drift doesn't seem to be a problem; if the system boots without the 'skew' message, I have no timer difficulties even if the box is up for months. If the system boots with a skew message, not a single timer-based op on the machine seems to work ever; I can't watch movies, play games or anything. I'll get a few frames, a freeze for several seconds, a few seconds of frames, freeze for several seconds, a frame or two, more freeze, etc... This appears to be related to processor affinity (when the process gets bounced to the other CPU, the timers appear to just freeze for a while or stop entirely).
Quoted 2 lines
One way to work around it would be to boot with "notsc". This will make your
gettimeofday()slower and more inaccurate though.
I will try that and report back.
Quoted 4 lines
Assuming it is not:
Something is wrong with your PIT timer in the southbridge. Maybe just run ntpd ?
I do run ntpd. My problem and concern is primarily with sub-second timers having a granularity of several seconds.
Quoted 2 lines
I know that later AMD chipsets - in particular the 8111 - are somewhat bad time keepers, which makes it a good idea to run NTP always.
The bug is all or nothing. Without the bootup skew report, the machine runs flawlessly indefinitely.
Monty