AMD X2 unsynced TSC fix?
Someone recently pointed out to me that a Windows "CPU driver update" supplied by AMD fixes the unsynced TSC problem on dual core AMD64 systems.
http://www.amd.com/us-en/Processors/TechnicalResources/0,,30_182_871_13118,00.html
"The AMD Dual-Core Optimizer can help improve some PC gaming video performance by compensating for those applications that bypass the Windows API for timing by directly using the RDTSC (Read Time Stamp Counter) instruction. Applications that rely on RDTSC do not benefit from the logic in the operating system to properly account for the affect of power management mechanisms on the rate at which a processor core's Time Stamp Counter (TSC) is incremented. The AMD Dual-Core Optimizer helps to correct the resulting video performance effects or other incorrect timing effects that these applications may experience on dual-core processor systems, by periodically adjusting the core time-stamp-counters, so that they are synchronized."
What are the chances of Linux getting a similar fix?
Lee
Lee Revell <[email protected]> ha scritto:
Quoted 3 lines
Someone recently pointed out to me that a Windows "CPU driver update" supplied by AMD fixes the unsynced TSC problem on dual core AMD64 systems.
[...]
Quoted 5 lines
other incorrect timing effects that these applications may experience on dual-core processor systems, by periodically adjusting the core time-stamp-counters, so that they are synchronized."
What are the chances of Linux getting a similar fix?
Zero? ;)
There's always a window where the TSCs are not in sync (and userspace may see a non-monotonic counter); furthermore when C'n'Q is active TSCs aren't updated at a fixed frequency, userspace cannot use TSC for timing anyway.
Luca
Quoted 2 lines
While we're on all of this, are we going to change "tained" to some other less alarmist word?
"screwed" -- Alexander Viro
Quoted 4 lines · Luca Tettamanti
Lee Revell <[email protected]> ha scritto:
Quoted 3 lines
Someone recently pointed out to me that a Windows "CPU driver update" supplied by AMD fixes the unsynced TSC problem on dual core AMD64 systems.
Quoted 8 lines
[...]
Quoted 5 lines
other incorrect timing effects that these applications may experience on dual-core processor systems, by periodically adjusting the core time-stamp-counters, so that they are synchronized."
What are the chances of Linux getting a similar fix?
Zero? ;)
Wrong. We have a fix that has been under serious testing for a long time.
Quoted 4 lines
There's always a window where the TSCs are not in sync (and userspace may see a non-monotonic counter); furthermore when C'n'Q is active TSCs aren't updated at a fixed frequency, userspace cannot use TSC for timing anyway.
Wrong, too. We have a patch that will be coming SOON (trust me, I am pushing hard for the author to publish it). With this patch applied you should never see the TSC go backwards. Period. It should be monotonic (to userspace, kernel rdtsc calls can still be wrong). CPUs should stay very nearly in sync (again, to userspace). The overhead of this patch is pretty minimal and costs nothing unless you actually read the TSC.
The catch is that, while it is monotonic, it is not guaranteed to be perfectly linear. For many applications, this will be good enough. Time will always move forward, and you won't be subject to the weird HZ granularity gettimeofday that unsynced TSCs can show.
I'm BCCing the author to poke him more publicly.
Tim
Quoted 6 lines · On 10/28/06, [email protected] <[email protected]> wrote:
Quoted 4 lines · Luca Tettamanti
There's always a window where the TSCs are not in sync (and userspace may see a non-monotonic counter); furthermore when C'n'Q is active TSCs aren't updated at a fixed frequency, userspace cannot use TSC for timing anyway.
Quoted 6 lines
Wrong, too. We have a patch that will be coming SOON (trust me, I am pushing hard for the author to publish it). With this patch applied you should never see the TSC go backwards. Period. It should be monotonic (to userspace, kernel rdtsc calls can still be wrong). CPUs should stay very nearly in sync (again, to userspace). The overhead of this patch is pretty minimal and costs nothing unless you actually read the TSC.
I know that's it's possible to resync the TSCs, but:
Quoted 4 lines
The catch is that, while it is monotonic, it is not guaranteed to be perfectly linear. For many applications, this will be good enough. Time will always move forward, and you won't be subject to the weird HZ granularity gettimeofday that unsynced TSCs can show.
As you say you cannot use it to do timing unless you disable any power management on the CPU. Otherwise you can count the elapsed ticks but you cannot convert the number to anything meaningful.
You may be able to emulate rdtsc for userspace but then again the whole point of using rdtsc is that it should be uber-fast... if rdtsc is emulated then you can just use gettimeofday (which is also optimized to be *very* fast). No?
Luca
Quoted 8 lines · Luca Tettamanti
As you say you cannot use it to do timing unless you disable any power management on the CPU. Otherwise you can count the elapsed ticks but you cannot convert the number to anything meaningful.
You may be able to emulate rdtsc for userspace but then again the whole point of using rdtsc is that it should be uber-fast... if rdtsc is emulated then you can just use gettimeofday (which is also optimized to be *very* fast). No?
gettimeofday() cannot be fast if it has to use the ACPI PM timer. It's 50% slower on my shiny new "AMD Athlon(tm)64 X2 Dual Core Processor 3800+" than on my 600Mhz Via C3, which in general is about a 10x slower machine. That's a massive regression.
Lee
Quoted 10 lines · Luca Tettamanti
I know that's it's possible to resync the TSCs, but:
Quoted 4 lines
The catch is that, while it is monotonic, it is not guaranteed to be perfectly linear. For many applications, this will be good enough. Time will always move forward, and you won't be subject to the weird HZ granularity gettimeofday that unsynced TSCs can show.
As you say you cannot use it to do timing unless you disable any power management on the CPU. Otherwise you can count the elapsed ticks but you cannot convert the number to anything meaningful.
I fyou have a third-party clock you can get pretty darn close. Fortunately, we usually have an HPET, these days. You can definitely resync and get near-linear values of RDTSC.
Quoted 4 lines
You may be able to emulate rdtsc for userspace but then again the whole point of using rdtsc is that it should be uber-fast... if rdtsc is emulated then you can just use gettimeofday (which is also optimized to be *very* fast). No?
We're not emulating it at all. The vast vast vast majority of rdtsc calls are nothing more than the RDTSC instruction. RDTSC is faster than gettimeofday(), necessarily. If gettimeofday() uses RDTSC, then the gettimeofday() vsyscall will be pretty good.
But, if I recall, i386 does not support vsyscall? 32 bit binaries on x86_64 does not support vsyscall. There is still a need for very fast pure RDTSC.
There are few problems at hand. I'm not familiar with the patch Andi's talking about but it has to solve all these problems to be really useful:
* TSC skew across CPUs at bootup (Linux handles this already)
* TSC drift across CPUs at the "same" frequency (pretty constant, minimal)
* TSC drift because of PM states, such as C1 (hlt) (semi-random, severe)
Anyway, I hope that all solutions will be considered. And I hope this patch comes soon.
Tim
Quoted 7 lines · On Friday 27 October 2006 19:46, [email protected] wrote:
Quoted 6 lines · Luca Tettamanti
I know that's it's possible to resync the TSCs, but:
Quoted 4 lines
The catch is that, while it is monotonic, it is not guaranteed to be perfectly linear. For many applications, this will be good enough. Time will always move forward, and you won't be subject to the weird HZ granularity gettimeofday that unsynced TSCs can show.
Quoted 5 lines
Quoted 3 lines
As you say you cannot use it to do timing unless you disable any power management on the CPU. Otherwise you can count the elapsed ticks but you cannot convert the number to anything meaningful.
I fyou have a third-party clock you can get pretty darn close.
Not when powernow is involved on a multi socket system.
This means it could be probably gotten to work on a variety of systems, but it wouldn't work on other systems because of that and I don't think it makes sense to try to fix an interface that will never work everywhere.
Quoted 2 lines
Fortunately, we usually have an HPET, these days. You can definitely resync and get near-linear values of RDTSC.
No we don't -- most BIOS still don't give us the HPET table even when it is there in hardware. In the future this will change sure but people will still run a lot of older motherboards.
Quoted 9 lines
Quoted 4 lines
You may be able to emulate rdtsc for userspace but then again the whole point of using rdtsc is that it should be uber-fast... if rdtsc is emulated then you can just use gettimeofday (which is also optimized to be *very* fast). No?
We're not emulating it at all. The vast vast vast majority of rdtsc calls are nothing more than the RDTSC instruction.> RDTSC is faster than
gettimeofday(), necessarily. Ifgettimeofday()uses RDTSC, then thegettimeofday()vsyscall will be pretty good.
Yes.
Quoted 1 line
But, if I recall, i386 does not support vsyscall?
There are ways to make it work there.
Quoted 2 lines
32 bit binaries on
x86_64does not support vsyscall.
And here too.
Basically you have to test for the calls in the system call vDSO and jump off. It's a little ugly but possible. I think John had experimental patches for this once.
Quoted 2 lines
There are few problems at hand. I'm not familiar with the patch Andi's talking about but it has to solve all these problems to be really useful:
It's from Jiri and Vojtech. Basically it will allow to use RDTSC in gettimeofday even with unsynchronized TSCs by keeping the necessary offsets CPU local.
Drawback: for vsyscall you need RDTSCP, this means AMD F stepping at least. But even as a syscall it will be still faster than before.
Quoted 1 line
* TSC skew across CPUs at bootup (Linux handles this already)
Just not very good. There is still a significant error when it's done.
Quoted 1 line
* TSC drift across CPUs at the "same" frequency (pretty constant, minimal)
It just adds up over time.
Quoted 1 line
* TSC drift because of PM states, such as C1 (hlt) (semi-random, severe)
TSC drift with powernow -- CPUs run at different frequencies
-Andi
Quoted 3 lines · Andi Kleen
Quoted 1 line
I fyou have a third-party clock you can get pretty darn close.
Not when powernow is involved on a multi socket system.
When CPUs are in different P-States, any resync effort will become unsynced immediately. I agree with that. This is a further complication that I think our code does not handle perfectly, yet.
Quoted 6 lines
Quoted 2 lines
Fortunately, we usually have an HPET, these days. You can definitely resync and get near-linear values of RDTSC.
No we don't -- most BIOS still don't give us the HPET table even when it is there in hardware. In the future this will change sure but people will still run a lot of older motherboards.
If you know where the HPET base-address-register is, can't we program it ourselves? Even without HPET, we have PM Timer. As long as you don't need to resync the TSCs on most gtod(), you can still do better than not trying.
Quoted 6 lines
Quoted 2 lines
There are few problems at hand. I'm not familiar with the patch Andi's talking about but it has to solve all these problems to be really useful:
It's from Jiri and Vojtech. Basically it will allow to use RDTSC in gettimeofday even with unsynchronized TSCs by keeping the necessary offsets CPU local.
Offset from what? With automatic clock ramping in C1, the rate is cycling up and down a lot.
Quoted 3 lines
Quoted 1 line
* TSC drift because of PM states, such as C1 (hlt) (semi-random, severe)
TSC drift with powernow -- CPUs run at different frequencies
Yeah, C1 is workaround-able, because the clock returns to full frequency, and we never execute code in the reduced clock state. Powernow makes it more fun. Not only do you need some offset, but you need some scalar.
Assume you resync TSCs to a clock (PM, HPET, whatever) any time any CPU changes p-state. Then you can calculate the approximate TSC for now by:
tsc_now = tsc_at_last_resync + ((rdtsc - tsc_at_last_resync) * pstate_scalar)
Something like that. Not pretty, but still possible to get close. And close might be good enough. As long as you can guarantee monotonicity and approximate linearity, you can make most apps happy ENOUGH.
Tim
Quoted 14 lines · Andi Kleen
Quoted 2 lines
There are few problems at hand. I'm not familiar with the patch Andi's talking about but it has to solve all these problems to be really useful:
It's from Jiri and Vojtech. Basically it will allow to use RDTSC in gettimeofday even with unsynchronized TSCs by keeping the necessary offsets CPU local.
Drawback: for vsyscall you need RDTSCP, this means AMD F stepping at least. But even as a syscall it will be still faster than before.
Quoted 1 line
* TSC skew across CPUs at bootup (Linux handles this already)
Just not very good. There is still a significant error when it's done.
Quoted 7 lines
Quoted 1 line
* TSC drift across CPUs at the "same" frequency (pretty constant, minimal)
It just adds up over time.
Quoted 1 line
* TSC drift because of PM states, such as C1 (hlt) (semi-random, severe)
TSC drift with powernow -- CPUs run at different frequencies
And the patch does exactly that.
It doesn't assume much about TSCs, except that they're individually monotonic and that without a warning (cpufreq notifier, c1 state enter/leave) the frequency doesn't change quickly. Slow frequency drift (spread spectrum modulation, thermal effects on Xtal) is compensated for.
We still are testing the patch and fixing the issues we find, currently with our cpufreq handling, but I believe we're on a good way to have it working well.
Signature · 2 lines
Vojtech Pavlik Director SuSE Labs
Quoted 7 lines · Andi Kleen
Quoted 1 line
Fortunately, we usually have an HPET, these days. You can
definitely
Quoted 1 line
resync and get near-linear values of RDTSC.
No we don't -- most BIOS still don't give us the HPET table even when it is there in hardware. In the future this will change sure but people will still run a lot of older motherboards.
I have exactly such a system (see thread "x86-64 with nvidia MCP51
chipset: kernel does not find HPET"). Is there anything at all I can do to make the kernel see the HPET? Can I try to guess the address? BIOS upgrade?
Lee
Quoted 5 lines · Lee Revell
Quoted 4 lines · Andi Kleen
Quoted 1 line
Fortunately, we usually have an HPET, these days. You can
definitely
Quoted 1 line
resync and get near-linear values of RDTSC.
Quoted 8 lines
Quoted 3 lines
No we don't -- most BIOS still don't give us the HPET table even when it is there in hardware. In the future this will change sure but people will still run a lot of older motherboards.
I have exactly such a system (see thread "x86-64 with nvidia MCP51
chipset: kernel does not find HPET"). Is there anything at all I can do to make the kernel see the HPET? Can I try to guess the address? BIOS upgrade?
In most cases where the HPET is present but not reported, it's not configured. Usually, you need to write a chipset-specific register to configure the address.
Finding the register, finding some free MMIO space, writing the address to the register and telling the address to the kernel is enough.
Signature · 2 lines
Vojtech Pavlik Director SuSE Labs
Quoted 9 lines · Vojtech Pavlik
Quoted 8 lines
Quoted 3 lines
No we don't -- most BIOS still don't give us the HPET table even when it is there in hardware. In the future this will change sure but people will still run a lot of older motherboards.
I have exactly such a system (see thread "x86-64 with nvidia MCP51
chipset: kernel does not find HPET"). Is there anything at all I can do to make the kernel see the HPET? Can I try to guess the address? BIOS upgrade?
Quoted 6 lines
In most cases where the HPET is present but not reported, it's not configured. Usually, you need to write a chipset-specific register to configure the address.
Finding the register, finding some free MMIO space, writing the address to the register and telling the address to the kernel is enough.
Do we want to establish a precedent for chipsets that we can find the HPET and configure ourselves? Register them all as PCI quirks...
Quoted 7 lines
Wrong, too. We have a patch that will be coming SOON (trust me, I am pushing hard for the author to publish it). With this patch applied you should never see the TSC go backwards. Period. It should be monotonic (to userspace, kernel rdtsc calls can still be wrong). CPUs should stay very nearly in sync (again, to userspace). The Thoverhead of this patch is pretty minimal and costs nothing unless you actually read the TSC.
There is another patch in the pipeline to make gettimeofday use RDTSC in more cases by keeping the offsets per CPU
(this has nothing to do with syncing TSCs which is not possible in the general case on several platforms)
I don't think it makes too much sense to hack on pure RDTSC when gtod is fast enough -- RDTSC will be always icky and hard to use.
-Andi
Quoted 2 lines · Andi Kleen
I don't think it makes too much sense to hack on pure RDTSC when gtod is fast enough -- RDTSC will be always icky and hard to use.
I agree FWIW, our application would be happy to just use gtod if it wasn't so slow on these machines.
Lee
Quoted 6 lines · Lee Revell
Quoted 2 lines · Andi Kleen
I don't think it makes too much sense to hack on pure RDTSC when gtod is fast enough -- RDTSC will be always icky and hard to use.
I agree FWIW, our application would be happy to just use gtod if it wasn't so slow on these machines.
Agreed, I had to turn about 20 dual-core servers to single core because the only way to get a monotonic gtod made it so slow that it was not worth using a dual-core. I initially considered buying one dual-core AMD for my own use, but after seeing this, I'm definitely sure I won't ever buy one as long as this problem is not fixed, as it causes too many problems.
Willy
Quoted 8 lines · Willy Tarreau
Quoted 6 lines · Lee Revell
Quoted 2 lines · Andi Kleen
I don't think it makes too much sense to hack on pure RDTSC when gtod is fast enough -- RDTSC will be always icky and hard to use.
I agree FWIW, our application would be happy to just use gtod if it wasn't so slow on these machines.
Quoted 6 lines
Agreed, I had to turn about 20 dual-core servers to single core because the only way to get a monotonic gtod made it so slow that it was not worth using a dual-core. I initially considered buying one dual-core AMD for my own use, but after seeing this, I'm definitely sure I won't ever buy one as long as this problem is not fixed, as it causes too many problems.
Does anyone know if the problem will really be fixed in new CPUs, as AMD promised a year or so ago?
http://lkml.org/lkml/2005/11/4/173
Since that post, there has been Socket F and AM2 which apparently have the same issue.
Were the AMD guys just blowing smoke?
Lee
Quoted 9 lines · Lee Revell
Does anyone know if the problem will really be fixed in new CPUs, as AMD promised a year or so ago?
http://lkml.org/lkml/2005/11/4/173
Since that post, there has been Socket F and AM2 which apparently have the same issue.
Were the AMD guys just blowing smoke?
I think it is coming, but still not here yet.
Quoted 8 lines
Quoted 7 lines
Agreed, I had to turn about 20 dual-core servers to single core because the only way to get a monotonic gtod made it so slow that it was not worth using a dual-core. I initially considered buying one dual-core AMD for my own use, but after seeing this, I'm definitely sure I won't ever buy one as long as this problem is not fixed, as it causes too many problems.
Quoted 8 lines
Does anyone know if the problem will really be fixed in new CPUs, as AMD promised a year or so ago?
http://lkml.org/lkml/2005/11/4/173
Since that post, there has been Socket F and AM2 which apparently have the same issue.
Were the AMD guys just blowing smoke?
AMD was not blowing smoke. Future AMD processors will have pstate/cstate invariant TSCs detectable by a CPUID bit.
Unfortunately, those processors have not be released yet, and I can't comment on their release timeframe, other than to say they are on our roadmap.
-Mark Langsdorf
AMD, Inc.
Quoted 11 lines · Willy Tarreau
Quoted 6 lines · Lee Revell
Quoted 2 lines · Andi Kleen
I don't think it makes too much sense to hack on pure RDTSC when gtod is fast enough -- RDTSC will be always icky and hard to use.
I agree FWIW, our application would be happy to just use gtod if it wasn't so slow on these machines.
Agreed, I had to turn about 20 dual-core servers to single core because the only way to get a monotonic gtod made it so slow that it was not worth using a dual-core.
Curious - what workload was that?
While gtod is time critical and often appears high on profile lists it is normally not as time critical as you're claiming it is; especially not time critical enough to warrant such radical action.
Quoted 4 lines
I initially considered buying one dual-core
AMD for my own use, but after seeing this, I'm definitely sure I won't ever buy one as long as this problem is not fixed, as it causes too many problems.
It's somewhat slower, but I'm not sure what "too many problems" you're refering to.
-Andi
Quoted 6 lines · Andi Kleen
Quoted 5 lines · Willy Tarreau
Quoted 4 lines · Lee Revell
Quoted 2 lines · Andi Kleen
I don't think it makes too much sense to hack on pure RDTSC when gtod is fast enough -- RDTSC will be always icky and hard to use.
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
I agree FWIW, our application would be happy to just use gtod if it wasn't so slow on these machines.
Agreed, I had to turn about 20 dual-core servers to single core because the only way to get a monotonic gtod made it so slow that it was not worth using a dual-core.
Curious - what workload was that?
Two different but related workloads :
- load balancer doing between 10 and 100k gtod per second on a sun x2100 under RHEL 3. HPET was not available and the only way I found to get monotonic clock was to use the APIC timer IIRC (it was more than 6 months ago, so sorry if I don't remember about all the details).
- network sniffer that I tried to tune to get the highest possible packet rates on gigabit ethernet.
Quoted 3 lines
While gtod is time critical and often appears high on profile lists it is normally not as time critical as you're claiming it is; especially not time critical enough to warrant such radical action.
Yes it was, because the small gain of using a dual core with such a workload was clearly lost by that change. IIRC, I reached 25000 sessions/s on dual core with TSC if I didn't care about the clock, 20000 without TSC, and 18000 on single core+TSC. But with the sniffer, it was even worse : I had 500 kpps in dual-core+TSC, 70kpps without TSC and 300 kpps with single-core+TSC. Since I had to buy the same machines for both uses, this last argument was enough for me to stick to a single core.
Quoted 7 lines
Quoted 4 lines
I initially considered buying one dual-core
AMD for my own use, but after seeing this, I'm definitely sure I won't ever buy one as long as this problem is not fixed, as it causes too many problems.It's somewhat slower, but I'm not sure what "too many problems" you're refering to.
Anticipated or delayed timeouts on the proxy, time measurement errors (when the logs show that a session finishes before it begins, there's a real problem, particularly because we use those logs for troubleshooting). And for the sniffer, getting wrong times by about 2s was a real problem too. I would have preferred to get something monotonic with little accuracy than out of order packets !
This is definitely a design problem on those chips, probably because marketting targets gamers only. And that's very sad, because they are excellent processors !
Quoted 1 line
-Andi
Regards,
Willy
Quoted 4 lines · Willy Tarreau
Quoted 3 lines
While gtod is time critical and often appears high on profile lists it is normally not as time critical as you're claiming it is; especially not time critical enough to warrant such radical action.
Quoted 8 lines
Yes it was, because the small gain of using a dual core with such a workload was clearly lost by that change. IIRC, I reached 25000 sessions/s on dual core with TSC if I didn't care about the clock, 20000 without TSC, and 18000 on single core+TSC. But with the sniffer, it was even worse : I had 500 kpps in dual-core+TSC, 70kpps without TSC and 300 kpps with single-core+TSC. Since I had to buy the same machines for both uses, this last argument was enough for me to stick to a single core.
Was the problem that they were not synced at poweron or that they would drift due to power-states?
Did you try running with idle=poll, to avoid ever entering C1 state (hlt)?
Quoted 14 lines · On Sat, Oct 28, 2006 at 12:18:00PM -0700, [email protected] wrote:
Quoted 4 lines · Willy Tarreau
Quoted 3 lines
While gtod is time critical and often appears high on profile lists it is normally not as time critical as you're claiming it is; especially not time critical enough to warrant such radical action.
Quoted 8 lines
Yes it was, because the small gain of using a dual core with such a workload was clearly lost by that change. IIRC, I reached 25000 sessions/s on dual core with TSC if I didn't care about the clock, 20000 without TSC, and 18000 on single core+TSC. But with the sniffer, it was even worse : I had 500 kpps in dual-core+TSC, 70kpps without TSC and 300 kpps with single-core+TSC. Since I had to buy the same machines for both uses, this last argument was enough for me to stick to a single core.
Quoted 2 lines
Was the problem that they were not synced at poweron or that they would drift due to power-states?
They resynced at power up, but would constantly drift. I don't even know if it was caused by power states. When the machine was loaded, a single task moving across the cores could see its time jump back and forth several times a second by an offset sometimes close to +2/-2s.
Quoted 1 line
Did you try running with idle=poll, to avoid ever entering C1 state (hlt)?
Yes, I remember trying such things. I also tried 'nohlt', completely disabling power management, including ACPI, etc... I also tried vanilla kernels as well as severely patched ones, but the problem remained the same in all circumstances, that only 'notsc' could solve.
BTW, I've just found a remain of dmesg capture after boot in case you'd like to look for anything in it.
Regards,
Willy
Quoted 7 lines · Willy Tarreau
Quoted 2 lines
Was the problem that they were not synced at poweron or that they would drift due to power-states?
They resynced at power up, but would constantly drift. I don't even know if it was caused by power states. When the machine was loaded, a single task moving across the cores could see its time jump back and forth several times a second by an offset sometimes close to +2/-2s.
That sounds like C1, to me.
Quoted 6 lines
Quoted 1 line
Did you try running with idle=poll, to avoid ever entering C1 state (hlt)?
Yes, I remember trying such things. I also tried 'nohlt', completely disabling power management, including ACPI, etc... I also tried vanilla kernels as well as severely patched ones, but the problem remained the same in all circumstances, that only 'notsc' could solve.
That's exceedingly strange. On my dual-socket dual-core, I can get roughly synced TSCs (no appreciable drift) by just using idle=poll. If that did not work for you, I'd really want to poke at the system more.
Quoted 2 lines
BTW, I've just found a remain of dmesg capture after boot in case you'd like to look for anything in it.
A dmesg won't be that useful, I'd actually have to poke at the system.
Quoted 10 lines · On Sat, Oct 28, 2006 at 12:42:45PM -0700, [email protected] wrote:
Quoted 7 lines · Willy Tarreau
Quoted 2 lines
Was the problem that they were not synced at poweron or that they would drift due to power-states?
They resynced at power up, but would constantly drift. I don't even know if it was caused by power states. When the machine was loaded, a single task moving across the cores could see its time jump back and forth several times a second by an offset sometimes close to +2/-2s.
That sounds like C1, to me.
OK.
Quoted 9 lines
Quoted 6 lines
Quoted 1 line
Did you try running with idle=poll, to avoid ever entering C1 state (hlt)?
Yes, I remember trying such things. I also tried 'nohlt', completely disabling power management, including ACPI, etc... I also tried vanilla kernels as well as severely patched ones, but the problem remained the same in all circumstances, that only 'notsc' could solve.
That's exceedingly strange. On my dual-socket dual-core, I can get roughly synced TSCs (no appreciable drift) by just using idle=poll.
As I said in another mail, I thought I won by running several busy loops in parallel to the load, which prevented the system from either halting or slowing down. But it was OK for a few minutes only and started going mad again.
Quoted 1 line
If that did not work for you, I'd really want to poke at the system more.
The machine was returned to the supplier and for other reasons, we switched to a different maker for the about 20 machines (and all single-core). I've read somewhere that there's already a second version of the sun x2100, I don't know if it still exhibits the problem. Maybe at least they've fixed the BIOS to report the HPET.
Quoted 4 lines
Quoted 2 lines
BTW, I've just found a remain of dmesg capture after boot in case you'd like to look for anything in it.
A dmesg won't be that useful, I'd actually have to poke at the system.
OK. I don't know if anyone there has one at hand, as I don't have it anymore.
Regards,
Willy
Quoted 8 lines · Willy Tarreau
Yes it was, because the small gain of using a dual core with such a workload was clearly lost by that change. IIRC, I reached 25000 sessions/s on dual core with TSC if I didn't care about the clock, 20000 without TSC, and 18000 on single core+TSC. But with the sniffer, it was even worse : I had 500 kpps in dual-core+TSC, 70kpps without TSC and 300 kpps with single-core+TSC. Since I had to buy the same machines for both uses, this last argument was enough for me to stick to a single core.
Ok, but it is a very specialized situation not applicable to most others. I just say this for all the other people following the thread. Again most workloads are not that gtod intensive.
BTW if you don't use powernow and don't use blades with thermal clock ramping and use idle=poll then the TSCs should be synchronized on AMD dual core and TSC gtod can be used. But it will burn a lot of power and make the system run very hot.
Quoted 9 lines
Quoted 8 lines
Quoted 5 lines
I initially considered buying one dual-core
AMD for my own use, but after seeing this, I'm definitely sure I won't ever buy one as long as this problem is not fixed, as it causes too many problems.It's somewhat slower, but I'm not sure what "too many problems" you're refering to.
Quoted 6 lines
Anticipated or delayed timeouts on the proxy, time measurement errors (when the logs show that a session finishes before it begins, there's a real problem, particularly because we use those logs for troubleshooting). And for the sniffer, getting wrong times by about 2s was a real problem too. I would have preferred to get something monotonic with little accuracy than out of order packets !
Ah you mean you forced the kernel to use a unsynchronized TSC
for gtod during your tuning attempts and then discovered that it didn't work? Call me surprised.
In the default configuration there shouldn't be any problems like this, it will just run slower because the kernel falls back to a slower time source.
Quoted 2 lines
This is definitely a design problem on those chips, probably because marketting targets gamers only.
Last time I checked Dual core Opterons weren't marketed to gamers.
Quoted 2 lines
And that's very sad, because they are
excellent processors !
Lots of various parties are to blame here, not just AMD.
The BIOS vendors for not exposing HPET even when it is available in the hardware. While HPET is slower than TSC too it definitely isn't nearly as slow as pmtimer.
Possibly the Linux people for not getting per CPU TSC going quicker.
The writers of software who uses gtod too often or force the kernel to call it for each packet by carelessly using the timestamp ioctl.
-Andi
Quoted 11 lines · Andi Kleen
Quoted 8 lines · Willy Tarreau
Yes it was, because the small gain of using a dual core with such a workload was clearly lost by that change. IIRC, I reached 25000 sessions/s on dual core with TSC if I didn't care about the clock, 20000 without TSC, and 18000 on single core+TSC. But with the sniffer, it was even worse : I had 500 kpps in dual-core+TSC, 70kpps without TSC and 300 kpps with single-core+TSC. Since I had to buy the same machines for both uses, this last argument was enough for me to stick to a single core.
Quoted 3 lines
Ok, but it is a very specialized situation not applicable to most others. I just say this for all the other people following the thread. Again most workloads are not that gtod intensive.
100% agreed (fortunately !).
Quoted 4 lines
BTW if you don't use powernow and don't use blades with thermal clock ramping and use idle=poll then the TSCs should be synchronized on AMD dual core and TSC gtod can be used. But it will burn a lot of power and make the system run very hot.
I tried to make it run like this. I once was said that by racking pizza boxes, you get a pizza oven. I was prepared to accept it :-)
But I would not manage to keep them in sync. I even remember running background loops to ensure that there was no idle at all, and the clocks still managed to get out of sync ! I tried to disable a lot of devices, starting with everything susceptible to send interrupts with long processing time (eg: USB, SATA, ...), but with no success. I once thought that I succeeded by sticking all interrupts to one core and the tasks to the other one, but was proved wrong after several minutes.
I really think that the hardware was doing tricks far beyond my knowledge, because on another Sun (a V40Z), there were 4 dual cores which I never saw out of sync even after hours of testing. But the HPET was available in it, I don't remember if it's used by default when detected.
Quoted 15 lines
Quoted 8 lines
Quoted 7 lines
Quoted 4 lines
I initially considered buying one dual-core
AMD for my own use, but after seeing this, I'm definitely sure I won't ever buy one as long as this problem is not fixed, as it causes too many problems.It's somewhat slower, but I'm not sure what "too many problems" you're refering to.
Quoted 6 lines
Anticipated or delayed timeouts on the proxy, time measurement errors (when the logs show that a session finishes before it begins, there's a real problem, particularly because we use those logs for troubleshooting). And for the sniffer, getting wrong times by about 2s was a real problem too. I would have preferred to get something monotonic with little accuracy than out of order packets !
Quoted 3 lines
Ah you mean you forced the kernel to use a unsynchronized TSC
for gtod during your tuning attempts and then discovered that it didn't work? Call me surprised.
No I did not "force" anything at first. You take the RHEL3 CD, you install it, reboot and watch your logs report negative times, then scratch your head, first call red hat dumb ass, and after a few tests, apologize to the poor innocent red hat and call the box a total crap. To put it shortly (might be useful for people who Google for it) : Dual-core Sun x2100 is unreliable out of the box under Linux.
Quoted 3 lines
In the default configuration there shouldn't be any problems like this, it will just run slower because the kernel falls back to a slower time source.
You have to specify "notsc" for this. As an alternative, a NUMA kernel worked fine too (because TSC is disabled), but it's not obvious for anyone why a dual-core, single proc system should be considered "NUMA" !
Quoted 4 lines
Quoted 2 lines
This is definitely a design problem on those chips, probably because marketting targets gamers only.
Last time I checked Dual core Opterons weren't marketed to gamers.
Not "opterons" under this name, but AMD X2 yes. Ask google for "AMD X2" and click on the first non-AMD site (3rd link), then check how it's benchmarked... On the other hand, if you look for "opteron", you immediately find more serious usages.
Quoted 8 lines
Quoted 2 lines
And that's very sad, because they are
excellent processors !Lots of various parties are to blame here, not just AMD.
The BIOS vendors for not exposing HPET even when it is available in the hardware. While HPET is slower than TSC too it definitely isn't nearly as slow as pmtimer.
I'm sure that the BIOS is buggy there, because I too found it strange that there was no HPET reported in such a system. But I found no way to enable it by force either, as I did not know where to start looking at.
Quoted 4 lines
Possibly the Linux people for not getting per CPU TSC going quicker.
The writers of software who uses gtod too often or force the kernel to call it for each packet by carelessly using the timestamp ioctl.
You can't use gtod less than once in a poll() loop unfortunately. And believe me, I do count my syscalls because each one hits performance by a few percent. When it comes to getting time on each packet, the problem is the same : you're dependant on the frequency of external events. You need to get the time once for each event. But I agree that a per-CPU TSC could help a lot at getting monotonic clocks. I think that using the local TSC to measure non-accurate time and decide when to call an external source would be a great improvement.
Regards,
Willy
Quoted 4 lines · Willy Tarreau
I really think that the hardware was doing tricks far beyond my knowledge, because on another Sun (a V40Z), there were 4 dual cores which I never saw out of sync even after hours of testing. But the HPET was available in it, I don't remember if it's used by default when detected.
I think some system occasionally ramp the clock for thermal management, but that should be rare.
Quoted 4 lines
No I did not "force" anything at first. You take the RHEL3 CD, you install it, reboot and watch your logs report negative times, then scratch your head, first call red hat dumb ass, and after a few tests, apologize to the poor innocent red hat
Well they should have fixed the kernel to fall back to another clock by backporting the appropiate fixes from mainline. I assume they did actually.
Quoted 3 lines
and call the box a total crap. To put it shortly
(might be useful for people who Google for it) : Dual-core Sun x2100 is unreliable out of the box under Linux.
No that shouldn't be true with any modern kernel. It will just fallback to HPET or more likely PMtimer.
Quoted 6 lines
Quoted 4 lines
In the default configuration there shouldn't be any problems like this, it will just run slower because the kernel falls back to a slower time source.
You have to specify "notsc" for this.
No, the kernel should work out of the box. Some older kernels didn't at various points of time though.
-Andi
Quoted 9 lines · Andi Kleen
Quoted 4 lines · Willy Tarreau
I really think that the hardware was doing tricks far beyond my knowledge, because on another Sun (a V40Z), there were 4 dual cores which I never saw out of sync even after hours of testing. But the HPET was available in it, I don't remember if it's used by default when detected.
I think some system occasionally ramp the clock for thermal management, but that should be rare.
I should say that at one moment, I've been wondering whether they were or not performing sort of an automatic overclocking under load, because those machines were really faster even in single-core than other opterons I had tested. Since such boxes are often compared on workloads such as SSL, doing so might have favored them in comparative benchmarks.
Quoted 8 lines
Quoted 4 lines
No I did not "force" anything at first. You take the RHEL3 CD, you install it, reboot and watch your logs report negative times, then scratch your head, first call red hat dumb ass, and after a few tests, apologize to the poor innocent red hat
Well they should have fixed the kernel to fall back to another clock by backporting the appropiate fixes from mainline. I assume they did actually.
But upon what trigger should they apply the fallback ? I don't see what can be detected. I see no such thing in 2.4 mainline (except TSC resync at boot), and do not seem to find any such fallback either in 2.6 (though I might not have looked deep enough as the code is more complex there).
Quoted 6 lines
Quoted 3 lines
and call the box a total crap. To put it shortly
(might be useful for people who Google for it) : Dual-core Sun x2100 is unreliable out of the box under Linux.No that shouldn't be true with any modern kernel. It will just fallback to HPET or more likely PMtimer.
same comment as above :-)
Quoted 9 lines
Quoted 6 lines
Quoted 4 lines
In the default configuration there shouldn't be any problems like this, it will just run slower because the kernel falls back to a slower time source.
You have to specify "notsc" for this.
No, the kernel should work out of the box. Some older kernels didn't at various points of time though.
Anyway, if they started providing kernels which used TSC by default, I don't think they will change this afterwards, in order to avoid causing regressions.
Could you please check if the fallbacks you're talking about are hard to backport in 2.4 ? Depending on their complexity and risk, I would not be against a small backport. I think for instance that automatically disabling TSC on SMP when HPET is present would not be a terrible regression and might help in a number of occasions. The user would then have to force the use of TSC if needed.
Regards,
Willy
Quoted 11 lines · Andi Kleen
Quoted 8 lines · Willy Tarreau
Yes it was, because the small gain of using a dual core with such a workload was clearly lost by that change. IIRC, I reached 25000 sessions/s on dual core with TSC if I didn't care about the clock, 20000 without TSC, and 18000 on single core+TSC. But with the sniffer, it was even worse : I had 500 kpps in dual-core+TSC, 70kpps without TSC and 300 kpps with single-core+TSC. Since I had to buy the same machines for both uses, this last argument was enough for me to stick to a single core.
Quoted 3 lines
Ok, but it is a very specialized situation not applicable to most others. I just say this for all the other people following the thread. Again most workloads are not that gtod intensive.
Haven't benchmarked or anything, but isn't X11 also a very gtod intensive workload?
Lee
Quoted 3 lines · Willy Tarreau
This is definitely a design problem on those chips, probably because marketting targets gamers only. And that's very sad, because they are excellent processors !
Hmm, gamers seem to be the worst affected by this problem on other OS...
Lee
Quoted 11 lines · Willy Tarreau
Quoted 7 lines · Lee Revell
Quoted 2 lines · Andi Kleen
I don't think it makes too much sense to hack on pure RDTSC when gtod is fast enough -- RDTSC will be always icky and hard to use.
I agree FWIW, our application would be happy to just use gtod if it wasn't so slow on these machines.
Quoted 6 lines
Agreed, I had to turn about 20 dual-core servers to single core because the only way to get a monotonic gtod made it so slow that it was not worth using a dual-core. I initially considered buying one dual-core AMD for my own use, but after seeing this, I'm definitely sure I won't ever buy one as long as this problem is not fixed, as it causes too many problems.
For the record, in my previous job we were implementing
a very fast packet sniffer/timestamper using 2x3.2GHz P4 Xeons + linux 2.4.20 (with gtod) Very rarely we would see inter packet times jump by (2^32)/CPU_Hz seconds, when sniffing about 1.2 million packets per second on 2 e1000 links,
which suggested a wrap around of a 32 bit comparison somewhere.
This lead to the fix below which was never picked up
(I guessed because it was addressed elsewhere?).
Note we were only interested in millisecond resolution for the timestamps, but the approximation is very good in general as you know the TSCs are very close to each other when this condition happens.
Note power management was not used on our systems.
Pádraig.
arch/i386/kernel/time.c 2005-07-07 10:32:34.000000000 +0100 renamed from arch/i386/kernel/time.c 2002-11-28 23:53:09.000000000 +0000 +4 -0
@@ -94,6 +94,9 @@ /* .. relative to previous jiffy (32 bits is enough) */ eax -= last_tsc_low; /* tsc_low delta */ + if ((signed)eax < 0) { /* workaround for drifting TSCs */ + eax = 0; + printk(KERN_INFO "tsc wrap around applied\n"); /* rare */ + } /*
* Time offset = (tsc_low delta) * fast_gettimeoffset_quotient
Quoted 19 lines · Pádraig Brady
Quoted 11 lines · Willy Tarreau
Quoted 7 lines · Lee Revell
Quoted 2 lines · Andi Kleen
I don't think it makes too much sense to hack on pure RDTSC when gtod is fast enough -- RDTSC will be always icky and hard to use.
I agree FWIW, our application would be happy to just use gtod if it wasn't so slow on these machines.
Quoted 6 lines
Agreed, I had to turn about 20 dual-core servers to single core because the only way to get a monotonic gtod made it so slow that it was not worth using a dual-core. I initially considered buying one dual-core AMD for my own use, but after seeing this, I'm definitely sure I won't ever buy one as long as this problem is not fixed, as it causes too many problems.
Quoted 5 lines
For the record, in my previous job we were implementing
a very fast packet sniffer/timestamper using 2x3.2GHz P4 Xeons + linux 2.4.20 (with gtod) Very rarely we would see inter packet times jump by (2^32)/CPU_Hz seconds, when sniffing about 1.2 million packets per second on 2 e1000 links,
which suggested a wrap around of a 32 bit comparison somewhere.
Interesting, as in my case I was jumps of about +/- 2s on a 2.2 GHz box, which also suggests a wrap around.
Quoted 5 lines
This lead to the fix below which was never picked up
(I guessed because it was addressed elsewhere?).
Note we were only interested in millisecond resolution for the timestamps, but the approximation is very good in general as you know the TSCs are very close to each other when this condition happens.
100% agreed.
Quoted 8 lines
Note power management was not used on our systems.
Pádraig.
diff -Naru
linux-2.4.20/arch/i386/kernel/time.clinux-2.4.20-corvil/arch/i386/kernel/time.c
arch/i386/kernel/time.c 2005-07-07 10:32:34.000000000 +0100
Quoted 10 lines
arch/i386/kernel/time.c 2005-07-07 10:32:34.000000000 +0100 renamed from arch/i386/kernel/time.c 2002-11-28 23:53:09.000000000 +0000 +4
-0@@ -94,6 +94,9 @@ /* .. relative to previous jiffy (32 bits is enough) */ eax -= last_tsc_low; /* tsc_low delta */ + if ((signed)eax < 0) { /* workaround for drifting TSCs */ + eax = 0; + printk(KERN_INFO "tsc wrap around applied\n"); /* rare */ + } /* * Time offset = (tsc_low delta) * fast_gettimeoffset_quotient
Cheers,
Willy
Quoted 6 lines · On Fri, 27 Oct 2006, [email protected] wrote:
Wrong, too. We have a patch that will be coming SOON (trust me, I am pushing hard for the author to publish it). With this patch applied you should never see the TSC go backwards. Period. It should be monotonic (to userspace, kernel rdtsc calls can still be wrong). CPUs should stay very nearly in sync (again, to userspace). The overhead of this patch is pretty minimal and costs nothing unless you actually read the TSC.
Well why not use regular clock_gettime() instead? If you add code for TSC processing (intercepting RDTSC from user space???) then it may be comparable in performance to time retrieval via POSIX calls using vsyscalls. Look like you may start duplicating the time subsystem?
Quoted 2 lines
What are the chances of Linux getting a similar fix?
Fix isn't the right word i would use for this particular implementation.
-Andi
Quoted 3 lines · Andi Kleen
Quoted 1 line
What are the chances of Linux getting a similar fix?
Fix isn't the right word i would use for this particular implementation.
What exactly does that AMD patch do? Other OS users report that it makes TSC usable for timing again. Does it do something really heavy handed like disable power management features?
Lee
Quoted 1 line · Lee Revell
What exactly does that AMD patch do?
"...by periodically adjusting the core time-stamp-counters, so that they are synchronized."
It sounds like they just periodically write a new value to the TSC. Presumably they set the "slower" one equal to the "faster" one.
You'd likely still have windows where time might run backwards, but it would be better than nothing.
Chris
Quoted 4 lines · Chris Friesen
Quoted 1 line · Lee Revell
What exactly does that AMD patch do?
Quoted 8 lines
"...by periodically adjusting the core time-stamp-counters, so that they are synchronized."
It sounds like they just periodically write a new value to the TSC. Presumably they set the "slower" one equal to the "faster" one.
You'd likely still have windows where time might run backwards, but it would be better than nothing.
The patch also apparently changes boot params to make the OS use the ACPI PM timer, so it must not be a complete solution.
Lee
Quoted 8 lines · Lee Revell
Quoted 7 lines · Chris Friesen
Quoted 1 line · Lee Revell
What exactly does that AMD patch do?
"...by periodically adjusting the core time-stamp-counters, so that they are synchronized."
Quoted 8 lines
Quoted 5 lines
It sounds like they just periodically write a new value to the TSC. Presumably they set the "slower" one equal to the "faster" one.
You'd likely still have windows where time might run backwards, but it would be better than nothing.
The patch also apparently changes boot params to make the OS use the ACPI PM timer, so it must not be a complete solution.
Hi,
So far, has I can understand. Seems to me that my computer which have a Pentium D (Dual Core) on VIA chipset, also have unsynchronized TSC and with the patch of hrtimers on
( http://www.tglx.de/projects/hrtimers/2.6.18/ )
Kernel found and use a new clocksource, the acpi_pm. And works stable but I don't deny that could be a little slower.
Just to point out. This could be more a problem of chipsets than CPUs (AMD or Intel). AMD just begin first using x86_64 archs :)
Last Note:
I still have other minor problem, seems (to me) related with SATA drives. Kernel 2.4.19-rc3 have big changes on SATA and I like to test it but can't apply hrtimers patch (I don't understand half seems in kernel other half not).
In rc3 with jiffies clocksource even with boot parameter "notsc" I have unsynchronized issues and many "Lost timer tickets", but I can say that is a regression because computer never work well.
Thanks,
--
Sérgio M. B.
Quoted 4 lines
So far, has I can understand. Seems to me that my computer which have a Pentium D (Dual Core) on VIA chipset, also have unsynchronized TSC and with the patch of hrtimers on
Intel systems (except for some large highend systems) have synchronized TSCs. Only exception so far seems to be a few systems that are overclocked/overvolted and running outside their specification. When you do that you'e on your own and we're not interested in a bug report.
There was also one BIOS found that had this problem, but it was old and rare and got fixed with a upgrade.
Quoted 2 lines
Just to point out. This could be more a problem of chipsets than CPUs (AMD or Intel). AMD just begin first using
x86_64archs :)
No.
-Andi
Quoted 9 lines · Andi Kleen
Quoted 3 lines
So far, has I can understand. Seems to me that my computer which have a Pentium D (Dual Core) on VIA chipset, also have unsynchronized TSC and with the patch of hrtimers on
Intel systems (except for some large highend systems) have synchronized TSCs. Only exception so far seems to be a few systems that are overclocked/overvolted and running outside their specification. When you do that you'e on your own and we're not interested in a bug report.
and my computer :)
http://www.asrock.com/product/775Dual-880Pro.htm
http://www.asrock.com/support/CPU_Support/show.asp?Model=775Dual-880Pro
Monday I will checkout if my computer is under specs.
Seems that I like buy computers with many problems on Linux and fix :)
Quoted 9 lines
There was also one BIOS found that had this problem, but it was old and rare and got fixed with a upgrade.
Quoted 2 lines
Just to point out. This could be more a problem of chipsets than CPUs (AMD or Intel). AMD just begin first using
x86_64archs :)No.
-Andi
Quoted 11 lines · Sergio Monteiro Basto
Quoted 9 lines · Andi Kleen
Quoted 3 lines
So far, has I can understand. Seems to me that my computer which have a Pentium D (Dual Core) on VIA chipset, also have unsynchronized TSC and with the patch of hrtimers on
Intel systems (except for some large highend systems) have synchronized TSCs. Only exception so far seems to be a few systems that are overclocked/overvolted and running outside their specification. When you do that you'e on your own and we're not interested in a bug report.
Quoted 5 lines
and my computer :)
http://www.asrock.com/product/775Dual-880Pro.htm
http://www.asrock.com/support/CPU_Support/show.asp?Model=775Dual-880Pro
Monday I will checkout if my computer is under specs.
Seems that I like buy computers with many problems on Linux and fix :)
I bought this computer, on computers shop that have the best credits in Portugal. And I don't change anything.
cat /proc/cpuinfo
processor : 1vendor_id : GenuineIntel
cpu family : 15
model : 4
model name : Intel(R) Pentium(R) D CPU 2.80GHz
stepping : 4
cpu MHz : 2793.050
cache size : 1024 KB
with 2 x 1024 KB cache size just saw Pentium D 820 in
http://www.intel.com/products/processor_number/chart/pentium_d.htm
which is supported on
http://www.asrock.com/support/CPU_Support/show.asp?Model=775Dual-880Pro
775 Pentium D 820 2.80GHz 8O0MHz 2MB Smithfield All
Just see that don't have Enhanced Intel SpeedStep® Technology.
I attach here x86info which match with
http://processorfinder.intel.com/details.aspx?sSpec=SL88T
Other curiosity with kernel 2.6.18.1 and the hrtimers patch. Kernel boot oops and hang , if I don't give "notsc" option.
Quoted 3 lines
Quoted 3 lines
There was also one BIOS found that had this problem, but it was old and rare and got fixed with a upgrade.
I have last BIOS released
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
Just to point out. This could be more a problem of chipsets than CPUs (AMD or Intel). AMD just begin first using
x86_64archs :)No.
-Andi
Signature · 1 line
Sérgio M.B.
x86info v1.17. Dave Jones 2001-2005
Feedback to <[email protected]>.
Found 2 CPUs
--------------------------------------------------------------------------
CPU #1
Found unknown cache descriptors: 81 91 96
Family: 15 Model: 4 Stepping: 4 Type: 0 Brand: 0 CPU Model: Extreme Edition [A0]
Processor name string: Intel(R) Pentium(R) D CPU 2.80GHz
Feature flags:
fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflsh ds acpi mmx fxsr sse sse2 ss ht tm pbe sse3 monitor ds-cpl cntx-id cx16 xTPR
Extended feature flags:
SYSCALL em64t
L1 Data cache:
Size: 16KB Sectored, 8-way associative.
line size=64 bytes.
Instruction TLB: 4K, 2MB or 4MB pages, fully associative, 128 entries.
Found unknown cache descriptors: 81 91 96
Data TLB: 4KB or 4MB pages, fully associative, 64 entries.
Processor serial: 0000-0F44-0000-0000-0000-0000
The physical package supports 2 logical processors
--------------------------------------------------------------------------
CPU #2
Found unknown cache descriptors: 81 91 96
Family: 15 Model: 4 Stepping: 4 Type: 0 Brand: 0 CPU Model: Extreme Edition [A0]
Processor name string: Intel(R) Pentium(R) D CPU 2.80GHz
Feature flags:
fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflsh ds acpi mmx fxsr sse sse2 ss ht tm pbe sse3 monitor ds-cpl cntx-id cx16 xTPR
Extended feature flags:
SYSCALL em64t
L1 Data cache:
Size: 16KB Sectored, 8-way associative.
line size=64 bytes.
Instruction TLB: 4K, 2MB or 4MB pages, fully associative, 128 entries.
Found unknown cache descriptors: 81 91 96
Data TLB: 4KB or 4MB pages, fully associative, 64 entries.
Processor serial: 0000-0F44-0000-0000-0000-0000
The physical package supports 2 logical processors
--------------------------------------------------------------------------
Quoted 12 lines · Sergio Monteiro Basto
Quoted 11 lines · Sergio Monteiro Basto
Quoted 9 lines · Andi Kleen
Quoted 3 lines
So far, has I can understand. Seems to me that my computer which have a Pentium D (Dual Core) on VIA chipset, also have unsynchronized TSC and with the patch of hrtimers on
Intel systems (except for some large highend systems) have synchronized TSCs. Only exception so far seems to be a few systems that are overclocked/overvolted and running outside their specification. When you do that you'e on your own and we're not interested in a bug report.
Quoted 8 lines
Quoted 5 lines
and my computer :)
http://www.asrock.com/product/775Dual-880Pro.htm
http://www.asrock.com/support/CPU_Support/show.asp?Model=775Dual-880Pro
Monday I will checkout if my computer is under specs.
Seems that I like buy computers with many problems on Linux and fix :)I bought this computer, on computers shop that have the best credits in Portugal. And I don't change anything.
Can you give us a full dmesg without noapic or notsc please?
Adding Suresh to cc too because he spotted a similar problem last time.
-Andi
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 9 lines · Sergio Monteiro Basto
Quoted 2 lines · Andi Kleen
Can you give us a full dmesg without noapic or notsc please?
yes , I send an dmesg of 2.6.18-git20, dmesg27
and other dmesg of kernel 2.6.18.1, dmesg30
To vanilla kernel I just add this patch:
ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.19-rc1/2.6.19-rc1-mm1/broken-out/gregkh-pci-pci-via-irq-quirk-behaviour-change.patch
Quoted 4 lines
Quoted 2 lines
Adding Suresh to cc too because he spotted a similar problem last
time.Feel free to ask any test, test patches or even access to this machine.
Maybe I've been running -rt for too long but I don't see clocksource selection - does 2.6.18 not have John Stultz's GTOD rework?
How can it know not to use TSC on machines where it's unstable?
Lee
Quoted 10 lines · Lee Revell
Quoted 9 lines · Sergio Monteiro Basto
Quoted 2 lines · Andi Kleen
Can you give us a full dmesg without noapic or notsc please?
yes , I send an dmesg of 2.6.18-git20, dmesg27
and other dmesg of kernel 2.6.18.1, dmesg30
To vanilla kernel I just add this patch:
ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.19-rc1/2.6.19-rc1-mm1/broken-out/gregkh-pci-pci-via-irq-quirk-behaviour-change.patch
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Adding Suresh to cc too because he spotted a similar problem last
time.Feel free to ask any test, test patches or even access to this machine.
Maybe I've been running -rt for too long but I don't see clocksource selection - does 2.6.18 not have John Stultz's GTOD rework?
He's booting x86_64. I've not had the time yet to cleanup and push my x86_64 conversion to CONFIG_GENERIC_TIME. Soon hopefully.
thanks
-john
Quoted 4 lines · Sergio Monteiro Basto
time.c: Lost 300 timer tick(s)! ripmwait_idle+0x33/0x4f)time.c: Lost 300 timer tick(s)! ripmwait_idle+0x33/0x4f)time.c: Lost 300 timer tick(s)! ripmwait_idle+0x33/0x4f)time.c: Lost 300 timer tick(s)! ripmwait_idle+0x33/0x4f)
Is this the reason why you are saying your system has unsynchronized TSC? Some where in this thread, you mentioned that Lost ticks happen even when you use "notsc"
This sounds to me as a different problem. Can you send us the output of /proc/interrupts?
thanks,
suresh
Quoted 7 lines · Suresh B
Quoted 4 lines · Sergio Monteiro Basto
time.c: Lost 300 timer tick(s)! ripmwait_idle+0x33/0x4f)time.c: Lost 300 timer tick(s)! ripmwait_idle+0x33/0x4f)time.c: Lost 300 timer tick(s)! ripmwait_idle+0x33/0x4f)time.c: Lost 300 timer tick(s)! ripmwait_idle+0x33/0x4f)Is this the reason why you are saying your system has unsynchronized TSC?
yes
Quoted 2 lines
Some where in this thread, you mentioned that Lost ticks happen even when you use "notsc"
yes, with news kernels 2.6.19-rcx
Quoted 3 lines
This sounds to me as a different problem. Can you send us the output of
/proc/interrupts?
of which kernel ?
I am not at home ..
but I have here /proc/interrupts from one 2.6.16
http://bugzilla.kernel.org/attachment.cgi?id=7927&action=view
from my bug
http://bugzilla.kernel.org/show_bug.cgi?id=6419
Tonight I can attach on bugzilla bug#6419, /proc/interrupts from one kernel 2.6.18 and from one kernel 2.6.19-rc4
BTW: those kernels are for x86_64 arch, I haven't try, yet, i386, but maybe will be my next test.
Thanks,
--
Sérgio M. B.
Quoted 3 lines
thanks,
suresh
Quoted 6 lines · Suresh B
Quoted 4 lines · Sergio Monteiro Basto
time.c: Lost 300 timer tick(s)! ripmwait_idle+0x33/0x4f)time.c: Lost 300 timer tick(s)! ripmwait_idle+0x33/0x4f)time.c: Lost 300 timer tick(s)! ripmwait_idle+0x33/0x4f)time.c: Lost 300 timer tick(s)! ripmwait_idle+0x33/0x4f)
Quoted 6 lines
Is this the reason why you are saying your system has unsynchronized TSC? Some where in this thread, you mentioned that Lost ticks happen even when you use "notsc"
This sounds to me as a different problem. Can you send us the output of
/proc/interrupts?
/proc/interrupts on kernel 2.6.18
http://bugzilla.kernel.org/attachment.cgi?id=9384&action=view
dmesg w/o notsc kernel 2.6.19-rc4
http://bugzilla.kernel.org/attachment.cgi?id=9385&action=view/proc/interrupts kernel 2.6.19-rc4
http://bugzilla.kernel.org/attachment.cgi?id=9386&action=view
dmesg w/ notsc kernel 2.6.19-rc4
http://bugzilla.kernel.org/attachment.cgi?id=9387&action=view/proc/interrupts kernel 2.6.19-rc4
http://bugzilla.kernel.org/attachment.cgi?id=9388&action=view
list of interrupts give by windows XP
http://bugzilla.kernel.org/attachment.cgi?id=9389&action=view
Let me know, if I can help on something.
Thanks,
Signature · 1 line
Sérgio M.B.
Quoted 14 lines · Sergio Monteiro Basto
Quoted 6 lines · Suresh B
Quoted 4 lines · Sergio Monteiro Basto
time.c: Lost 300 timer tick(s)! ripmwait_idle+0x33/0x4f)time.c: Lost 300 timer tick(s)! ripmwait_idle+0x33/0x4f)time.c: Lost 300 timer tick(s)! ripmwait_idle+0x33/0x4f)time.c: Lost 300 timer tick(s)! ripmwait_idle+0x33/0x4f)Quoted 6 lines
Is this the reason why you are saying your system has unsynchronized TSC? Some where in this thread, you mentioned that Lost ticks happen even when you use "notsc"
This sounds to me as a different problem. Can you send us the output of
/proc/interrupts?
Quoted 12 lines
/proc/interruptson kernel 2.6.18
http://bugzilla.kernel.org/attachment.cgi?id=9384&action=view
dmesg w/o notsc kernel 2.6.19-rc4
http://bugzilla.kernel.org/attachment.cgi?id=9385&action=view/proc/interruptskernel 2.6.19-rc4
http://bugzilla.kernel.org/attachment.cgi?id=9386&action=view
dmesg w/ notsc kernel 2.6.19-rc4
http://bugzilla.kernel.org/attachment.cgi?id=9387&action=view/proc/interruptskernel 2.6.19-rc4
http://bugzilla.kernel.org/attachment.cgi?id=9388&action=view
list of interrupts give by windows XP
http://bugzilla.kernel.org/attachment.cgi?id=9389&action=view
First of all, from "lost timer ticks" messages and the fact that "notsc" decreases the number of ticks lost can't be concluded as a TSC sync issue.
Some device is hogging interrupts which results in lost timer ticks and from your 2.6.18 interrupts info, usb seems to be the culprit.. It is probably a side effect that "notsc" decreases the lost timer ticks..
copied Len who seems to be the owner of the bug for his thoughts..
(http://bugzilla.kernel.org/show_bug.cgi?id=6419)
thanks,
suresh
Quoted 2 lines · Suresh B
First of all, from "lost timer ticks" messages and the fact that "notsc" decreases the number of ticks lost can't be concluded as a TSC sync issue.
ok, but without notsc it is a nightmare
Quoted 4 lines
Some device is hogging interrupts which results in lost timer ticks and from your 2.6.18 interrupts info, usb seems to be the culprit.. It is probably a side effect that "notsc" decreases the lost timer ticks..
I begging use net with Ethernet instead usbnet and reduce a little the problems (I can have nvidia DRI working without problems or oops) but still appear the same lost tickets.
Quoted 2 lines
copied Len who seems to be the owner of the bug for his thoughts..
(http://bugzilla.kernel.org/show_bug.cgi?id=6419)
I had update bugzilla with dmesg from 2.6.19-RC4-mm2, which already came with the latest release of hrtimers, because for the first time I could boot without hang on boot, with hrtimers and without notsc boot option. But it have a long long oops that maybe could give you some clues.
http://bugzilla.kernel.org/show_bug.cgi?id=6419#c55
Thanks,
Signature · 1 line
Sérgio M.B.
Quoted 6 lines · Sergio Monteiro Basto
I had update bugzilla with dmesg from 2.6.19-RC4-mm2, which already came with the latest release of hrtimers, because for the first time I could boot without hang on boot, with hrtimers and without notsc boot option. But it have a long long oops that maybe could give you some clues.
This one is a lock dependency problem, which is fixed in -rc5-mm1
tglx
Quoted 1 line · Thomas Gleixner
This one is a lock dependency problem, which is fixed in -rc5-mm1
yes, oops fixed w/ and w/o notsc option.
Other question, hrtimer in 2.6.18 found acpi_pm clocksource and use it. With 2.6.19-rcx can't get acpi_pm clocksource even trying force at boot kernel with clocksource=acpi_pm, any idea ?
because with this clocksource my lost ticket disappears
Thanks,
Signature · 1 line
Sérgio M.B.
Quoted 8 lines · Sergio Monteiro Basto
Quoted 1 line · Thomas Gleixner
This one is a lock dependency problem, which is fixed in -rc5-mm1
yes, oops fixed w/ and w/o notsc option.
Other question, hrtimer in 2.6.18 foundacpi_pmclocksource and use it. With 2.6.19-rcx can't getacpi_pmclocksource even trying force at boot kernel with clocksource=acpi_pm, any idea ?
because with this clocksource my lost ticket disappears
Looking at the dmesg in the bugzilla:
http://bugzilla.kernel.org/show_bug.cgi?id=6419
I noticed you're using x86_64. x86_64 doesn't yet support clocksource overrides in mainline, as it is not converted to GENERIC_TIME. (Probably printing out such a warning if an override is used would be nice. I'll try to get to that soon.)
Now, the code to convert x86_64 is in tglx's hrtimer patch set, so I'm glad to hear its working for you, however I'm not sure if it really is solving the issue or just hiding it (as lost ticks won't affect timekeeping when you use continuous clocksources and GENERIC_TIME).
To use the ACPI PM w/ a 2.6.19-rcX kernel, use "notsc", and you'll see the line:
time.c: Using 3.579545 MHz WALL PM GTOD PM timer.
Using the "notsc" option, do you continue to see lost tick messages after bootup?
thanks
-john
Quoted 2 lines · john stultz
Using the "notsc" option, do you continue to see lost tick messages after bootup?
With notsc after boot up, lost ticket stops, the bigger exception was in last test kernel (2.6.19-RC5-mm1) which appear some few lost ticket but seems they just stop. I am waiting to see if appears a new one but don't.
Thanks,
Signature · 1 line
Sérgio M.B.
Quoted 10 lines · john stultz
Quoted 8 lines · Sergio Monteiro Basto
Quoted 1 line · Thomas Gleixner
This one is a lock dependency problem, which is fixed in -rc5-mm1
yes, oops fixed w/ and w/o notsc option.
Other question, hrtimer in 2.6.18 foundacpi_pmclocksource and use it. With 2.6.19-rcx can't getacpi_pmclocksource even trying force at boot kernel with clocksource=acpi_pm, any idea ?
because with this clocksource my lost ticket disappears
Quoted 4 lines
Looking at the dmesg in the bugzilla:
http://bugzilla.kernel.org/show_bug.cgi?id=6419I noticed you're using
x86_64.
yes, I _just_ use x86_64 never test it on i386.
Quoted 2 lines
x86_64doesn't yet support clocksource
overrides in mainline,
petty , can I have a experimental patch to test it?
Quoted 8 lines
as it is not converted to
GENERIC_TIME. (Probably
printing out such a warning if an override is used would be nice. I'll try to get to that soon.)Now, the code to convert
x86_64is in tglx's hrtimer patch set, so I'm glad to hear its working for you, however I'm not sure if it really is solving the issue or just hiding it (as lost ticks won't affect timekeeping when you use continuous clocksources andGENERIC_TIME).
Well, the only kernel where I can work (yes I use computer to work) is 2.6.18 + dyntick. I think don't hid neither solve the issue, is just use other resource (clocksource) that works better ! .
Quoted 7 lines
To use the ACPI PM w/ a 2.6.19-rcX kernel, use "notsc", and you'll see the line:
time.c: Using 3.579545 MHz WALL PM GTOD PM timer.Using the "notsc" option, do you continue to see lost tick messages after bootup?
I just test 2.6.19-RC5-mm2 and still very unstable even with notsc. And after bootup, yes appears some lost tick messages. Just trying rebuild other kernel and use command yum to update others things, at same time, have lock up my computer.
So I back to kernel 2.6.18 + dyntick
Thanks,
Quoted 5 lines
thanks
-john
Signature · 1 line
Sérgio M.B.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 7 lines · Andreas Arens
as I see from the dmesg on the Fedora bugzilla, your acpi tables don't provide an entry to the HPET timer.
As the VIA8237 happens to have a built-in HPET, I was able to force it on using the
attached patch (against 2.6.18) on an X2 system with the same problem, which greatly improved the system stability for me.
But I have one Intel(R) Pentium(R) D CPU 2.8 on a VIA8237
My latest suspect of the root of the problem of my computer is not in Processor but in those VIAs. As you find that "don't provide an entry to the HPET timer on acpi tables" it match, but how do you know that ? I don't send DSDT on bugzilla
Quoted 6 lines
The patch is hand-crafted from some older clock-tick kernel tree sources I found by googling.
The thing is hackish and not suitable for mainline inclusion, but may be useful nontheless.
If you find it useful, and it helps you please let me know.
I try your patch and it give me this differences on dmesg (file attach), detect a different timer.c but no improvement without notsc boot option and with notsc the computer got worst.
Quoted 1 line
Signature · 1 line
Sérgio M.B.
2c2
< Linux version 2.6.18-1.3_FC5 (root@monteirov) (gcc version 4.1.1 20060525 (Red Hat 4.1.1-1)) #1 SMP Wed Oct 18 23:54:57 WEST 2006
---
Quoted 1 line
Linux version 2.6.18-1.13_2.6.18.2+via_8237_force_hpet_FC6 (root@monteirov) (gcc version 4.1.1 20061011 (Red Hat 4.1.1-30)) #1 SMP Thu Nov 16 00:30:41 WET 2006
24,25c24,25
< On node 0 totalpages: 254172
< DMA zone: 1746 pages, LIFO batch:0
---
Quoted 2 lines
On node 0 totalpages: 254180
DMA zone: 1754 pages, LIFO batch:0
48c48
< Built 1 zonelists. Total pages: 254172
---
Quoted 1 line
Built 1 zonelists. Total pages: 254180
52,53c52,55
< time.c: Using 3.579545 MHz WALL PM GTOD PIT/TSC timer. < time.c: Detected 2793.051 MHz processor.
---
Quoted 4 lines
80000800
time.c: WARNING: Enabled VIA8237 HPET at0xfed00000.time.c: Using 14.318180 MHz WALL HPET GTOD HPET/TSC timer.time.c: Detected 2793.383 MHz processor.
55c57
< time.c: Lost 1 timer tick(s)! rip release_console_sem+0x1bc/0x232)
---
Quoted 1 line
time.c: Lost 11 timer tick(s)! riprelease_console_sem+0x1b3/0x229)
69,70c71,72
< Memory: 1013704k/1048256k available (2391k kernel code, 34164k reserved, 1964k data, 204k init) < Calibrating delay using timer specific routine.. 5595.08 BogoMIPS (lpj=11190172)
---
Quoted 2 lines
Memory: 1013332k/1048256k available (2382k kernel code, 34536k reserved, 1956k data, 204k init) Calibrating delay using timer specific routine.. 5594.01 BogoMIPS (lpj=2797007)
86,88c88,90
< result 12468969
< Detected 12.468 MHz APIC timer.
< time.c: Lost 9 timer tick(s)! rip setup_boot_APIC_clock+0x12c/0x12f)
---
Quoted 3 lines
result 12470375
Detected 12.470 MHz APIC timer.time.c: Lost 35 timer tick(s)! ripsetup_boot_APIC_clock+0x128/0x12f)
92c94
< Calibrating delay using timer specific routine.. 5586.53 BogoMIPS (lpj=11173068)
---
Quoted 1 line
Calibrating delay using timer specific routine.. 5586.07 BogoMIPS (lpj=2793035)
99a102
Quoted 1 line
time.c: Lost 2 timer tick(s)! rip__do_softirq+0x5c/0xf5)
101c104
< migration_cost=691
---
Quoted 1 line
migration_cost=681
103c106
< Freeing initrd memory: 1100k freed
---
Quoted 1 line
Freeing initrd memory: 1507k freed
169a173,174
Quoted 2 lines
hpet0: at MMIO
0xfed00000(virtual0xffffffffff5fe000), IRQs 2, 8, 0
hpet0: 3 32-bit timers, 14318180 Hz
191c196
< audit(1162252117.788:1): initialized
---
Quoted 1 line
audit(1163638320.760:1): initialized
210,211c215,216
< ACPI Error (psparse-0537): Method parse/execution failed [\_PR_.CPU1._PDC] (Node ffff8100027fb810), AE_BAD_HEADER < ACPI Error (psparse-0537): Method parse/execution failed [\_PR_.CPU2._PDC] (Node ffff8100027fb650), AE_BAD_HEADER
---
Quoted 2 lines
ACPI Error (psparse-0537): Method parse/execution failed [\_PR_.CPU1._PDC] (Node
ffff8100027e0810),AE_BAD_HEADERACPI Error (psparse-0537): Method parse/execution failed [\_PR_.CPU2._PDC] (Nodeffff8100027e0650),AE_BAD_HEADER
249d253
< PM: Adding info for serio:serio1
250a255
Quoted 1 line
PM: Adding info for serio:serio1
259c264
< Write protecting the kernel read-only data: 451k
---
Quoted 1 line
Write protecting the kernel read-only data: 444k
261,308d265
< SCSI subsystem initialized
< libata version 2.00 loaded.
< sata_via 0000:00:0f.0: version 2.0
< ACPI: PCI Interrupt 0000:00:0f.0[B] -> GSI 20 (level, low) -> IRQ 193 < sata_via 0000:00:0f.0: routed to hard irq line 3
< ata1: SATA max UDMA/133 cmd 0xD880 ctl 0xD802 bmdma 0xD080 irq 193 < ata2: SATA max UDMA/133 cmd 0xD480 ctl 0xD402 bmdma 0xD088 irq 193 < scsi0 : sata_via
< PM: Adding info for No Bus:host0
< ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
< input: ImPS/2 Generic Wheel Mouse as /class/input/input1
< ata1.00: ATA-7, max UDMA/100, 390721968 sectors: LBA48 NCQ (depth 0/32) < ata1.00: ata1: dev 0 multi count 16
< ata1.00: configured for UDMA/100
< scsi1 : sata_via
< PM: Adding info for No Bus:host1
< ata2: SATA link down 1.5 Gbps (SStatus 0 SControl 300)
< ATA: abnormal status 0x7F on port 0xD487
< time.c: Lost 1 timer tick(s)! rip __do_softirq+0x5c/0xf5)
< PM: Adding info for No Bus:target0:0:0
< Vendor: ATA Model: Maxtor 6L200M0 Rev: BANC < Type: Direct-Access ANSI SCSI revision: 05
< PM: Adding info for scsi:0:0:0:0
< SCSI device sda: 390721968 512-byte hdwr sectors (200050 MB)
< sda: Write Protect is off
< sda: Mode Sense: 00 3a 00 00
< SCSI device sda: drive cache: write back
< SCSI device sda: 390721968 512-byte hdwr sectors (200050 MB)
< sda: Write Protect is off
< sda: Mode Sense: 00 3a 00 00
< SCSI device sda: drive cache: write back
< sda: sda1 sda2 sda3 < sda5 sda6 sda7 sda8 >
< sd 0:0:0:0: Attached scsi disk sda
< kjournald starting. Commit interval 5 seconds
< EXT3-fs: mounted filesystem with ordered data mode.
< SELinux: Disabled at runtime.
< SELinux: Unregistering netfilter hooks
< audit(1162252122.040:2): selinux=0 auid=4294967295
< input: PC Speaker as /class/input/input2
< via-rhine.c:v1.10-LK1.4.1 July-24-2006 Written by Donald Becker < GSI 18 sharing vector 0xC9 and IRQ 18
< ACPI: PCI Interrupt 0000:00:12.0[A] -> GSI 23 (level, low) -> IRQ 201 < eth0: VIA Rhine II at 0xfbfffc00, 00:13:8f:6e:8f:c5, IRQ 201.
< eth0: MII PHY found at address 1, status 0x786d advertising 05e1 Link 0021. < hdc: ATAPI 48X DVD-ROM DVD-R-RAM CD-R/RW drive, 2048kB Cache, UDMA(33) < Uniform CD-ROM driver Revision: 3.20
< shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
< sd 0:0:0:0: Attached scsi generic sg0 type 0
310,311c267,268
< GSI 19 sharing vector 0xD1 and IRQ 19
< ACPI: PCI Interrupt 0000:00:10.0[A] -> GSI 21 (level, low) -> IRQ 209
---
Quoted 2 lines
GSI 18 sharing vector
0xC9and IRQ 18
ACPI: PCI Interrupt 0000:00:10.0[A] -> GSI 21 (level, low) -> IRQ 201
314c271
< uhci_hcd 0000:00:10.0: irq 209, io base 0x0000ec00
---
Quoted 1 line
uhci_hcd0000:00:10.0: irq 201, io base0x0000ec00
322c279
< ACPI: PCI Interrupt 0000:00:10.1[A] -> GSI 21 (level, low) -> IRQ 209
---
Quoted 1 line
ACPI: PCI Interrupt 0000:00:10.1[A] -> GSI 21 (level, low) -> IRQ 201
325c282
< uhci_hcd 0000:00:10.1: irq 209, io base 0x0000e080
---
Quoted 1 line
uhci_hcd0000:00:10.1: irq 201, io base0x0000e080
333c290
< ACPI: PCI Interrupt 0000:00:10.2[B] -> GSI 21 (level, low) -> IRQ 209
---
Quoted 1 line
ACPI: PCI Interrupt 0000:00:10.2[B] -> GSI 21 (level, low) -> IRQ 201
336c293
< uhci_hcd 0000:00:10.2: irq 209, io base 0x0000e000
---
Quoted 1 line
uhci_hcd0000:00:10.2: irq 201, io base0x0000e000
343,345d299
< Floppy drive(s): fd0 is 1.44M
< FDC 0 is a post-1991 82077
< PM: Adding info for platform:floppy.0
347c301
< ACPI: PCI Interrupt 0000:00:10.3[B] -> GSI 21 (level, low) -> IRQ 209
---
Quoted 1 line
ACPI: PCI Interrupt 0000:00:10.3[B] -> GSI 21 (level, low) -> IRQ 201
350c304
< uhci_hcd 0000:00:10.3: irq 209, io base 0x0000dc00
---
Quoted 1 line
uhci_hcd0000:00:10.3: irq 201, io base0x0000dc00
356a311
Quoted 1 line
input: ImPS/2 Generic Wheel Mouse as /class/input/input1
358,359c313,314
< PM: Adding info for No Bus:i2c-0
< ACPI: PCI Interrupt 0000:00:10.4[C] -> GSI 21 (level, low) -> IRQ 209
---
Quoted 2 lines
ohci_hcd: 2005 April 22 USB 1.1 'Open' Host Controller (OHCI) Driver (PCI)
ACPI: PCI Interrupt 0000:00:10.4[C] -> GSI 21 (level, low) -> IRQ 201
362c317
< ehci_hcd 0000:00:10.4: irq 209, io mem 0xfbfff800
---
Quoted 1 line
ehci_hcd0000:00:10.4: irq 201, io mem0xfbfff800
370a326,385
Quoted 45 lines
SCSI subsystem initialized
libata version 2.00 loaded.sata_via0000:00:0f.0: version 2.0
ACPI: PCI Interrupt 0000:00:0f.0[B] -> GSI 20 (level, low) -> IRQ 193sata_via0000:00:0f.0: routed to hard irq line 3
ata1: SATA max UDMA/133 cmd0xD880ctl0xD802bmdma0xD080irq 193
ata2: SATA max UDMA/133 cmd0xD480ctl0xD402bmdma0xD088irq 193 scsi0 :sata_via
PM: Adding info for No Bus:host0
ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 300) ata1.00: ATA-7, max UDMA/100, 390721968 sectors: LBA48 NCQ (depth 0/32) ata1.00: ata1: dev 0 multi count 16
ata1.00: configured for UDMA/100
scsi1 :sata_via
PM: Adding info for No Bus:host1
ata2: SATA link down 1.5 Gbps (SStatus 0 SControl 300)
ATA: abnormal status0x7Fon port0xD487time.c: Lost 10 timer tick(s)! rip__do_softirq+0x5c/0xf5) usb 5-6: new high speed USB device usingehci_hcdand address 2
PM: Adding info for No Bus:target0:0:0
Vendor: ATA Model: Maxtor 6L200M0 Rev: BANC
Type: Direct-Access ANSI SCSI revision: 05
PM: Adding info for scsi:0:0:0:0
SCSI device sda: 390721968 512-byte hdwr sectors (200050 MB)
sda: Write Protect is off
sda: Mode Sense: 00 3a 00 00
SCSI device sda: drive cache: write back
SCSI device sda: 390721968 512-byte hdwr sectors (200050 MB)
sda: Write Protect is off
sda: Mode Sense: 00 3a 00 00
SCSI device sda: drive cache: write back
sda: sda1 sda2 sda3 < sda5 sda6 sda7 sda8 >
sd 0:0:0:0: Attached scsi disk sda
PM: Adding info for usb:5-6
PM: Adding info for No Bus:usbdev5.2_ep00
usb 5-6: configuration #1 chosen from 1 choice
PM: Adding info for usb:5-6:1.0
PM: Adding info for No Bus:usbdev5.2_ep81
PM: Adding info for No Bus:usbdev5.2_ep02
PM: Adding info for No Bus:usbdev5.2_ep83
libusual: modprobe for usb-storage succeeded, but module is not present kjournald starting. Commit interval 5 secondsEXT3-fs: mounted filesystem with ordered data mode. SELinux: Disabled at runtime. SELinux: Unregistering netfilter hooks
Quoted 15 lines
audit(1163638326.450:2): selinux=0 auid=4294967295 via-rhine.c:v1.10-LK1.4.1 July-24-2006 Written by Donald Becker GSI 19 sharing vector
0xD1and IRQ 19
ACPI: PCI Interrupt 0000:00:12.0[A] -> GSI 23 (level, low) -> IRQ 209
eth0: VIA Rhine II at0xfbfffc00, 00:13:8f:6e:8f:c5, IRQ 209.
eth0: MII PHY found at address 1, status0x786dadvertising 05e1 Link 45e1.
shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
hdc: ATAPI 48X DVD-ROM DVD-R-RAM CD-R/RW drive, 2048kB Cache, UDMA(33) Uniform CD-ROM driver Revision: 3.20
sd 0:0:0:0: Attached scsi generic sg0 type 0
input: PC Speaker as /class/input/input2
PM: Adding info for No Bus:i2c-0
Floppy drive(s): fd0 is 1.44M
FDC 0 is a post-1991 82077
PM: Adding info for platform:floppy.0
375,388d389
< usb 5-6: new high speed USB device using ehci_hcd and address 3
< PM: Adding info for usb:5-6
< PM: Adding info for No Bus:usbdev5.3_ep00
< usb 5-6: configuration #1 chosen from 1 choice
< PM: Adding info for usb:5-6:1.0
< PM: Adding info for No Bus:usbdev5.3_ep81
< PM: Adding info for No Bus:usbdev5.3_ep02
< PM: Adding info for No Bus:usbdev5.3_ep83
< Initializing USB Mass Storage driver...
< usb 2-2: new full speed USB device using uhci_hcd and address 2
< PM: Adding info for usb:2-2
< PM: Adding info for No Bus:usbdev2.2_ep00
< usb 2-2: configuration #1 chosen from 1 choice
< PM: Adding info for usb:2-2:1.0
391,402d391
< PM: Adding info for No Bus:usbdev2.2_ep85
< PM: Adding info for usb:2-2:1.1
< scsi2 : SCSI emulation for USB Mass Storage devices
< PM: Adding info for No Bus:host2
< usb-storage: device found at 3
< usb-storage: waiting for device to settle before scanning
< usbcore: registered new driver usb-storage
< USB Mass Storage support registered.
< PM: Adding info for No Bus:usbdev2.2_ep81
< PM: Adding info for No Bus:usbdev2.2_ep02
< eth1: register 'cdc_ether' at usb-0000:00:10.1-2, CDC Ethernet Device, 00:90:64:fc:ce:2b < usbcore: registered new driver cdc_ether
407d395
< ibm_acpi: ec object not found
421,458c409,410
< PM: Adding info for No Bus:target2:0:0
< Vendor: OTi Model: CF CARD Reader Rev: 2.00 < Type: Direct-Access ANSI SCSI revision: 00 < PM: Adding info for scsi:2:0:0:0 < sd 2:0:0:0: Attached scsi removable disk sdb < sd 2:0:0:0: Attached scsi generic sg1 type 0 < Vendor: OTi Model: SM CARD Reader Rev: 2.00 < Type: Direct-Access ANSI SCSI revision: 00 < PM: Adding info for scsi:2:0:0:1 < sd 2:0:0:1: Attached scsi removable disk sdc < sd 2:0:0:1: Attached scsi generic sg2 type 0 < Vendor: OTi Model: SD CARD Reader Rev: 2.00 < Type: Direct-Access ANSI SCSI revision: 00 < PM: Adding info for scsi:2:0:0:2 < sd 2:0:0:2: Attached scsi removable disk sdd < sd 2:0:0:2: Attached scsi generic sg3 type 0 < Vendor: OTi Model: MS CARD Reader Rev: 2.00 < Type: Direct-Access ANSI SCSI revision: 00
< PM: Adding info for scsi:2:0:0:3
< sd 2:0:0:3: Attached scsi removable disk sde
< sd 2:0:0:3: Attached scsi generic sg4 type 0
< PM: Adding info for No Bus:target2:0:1
< PM: Removing info for No Bus:target2:0:1
< PM: Adding info for No Bus:target2:0:2
< PM: Removing info for No Bus:target2:0:2
< PM: Adding info for No Bus:target2:0:3
< PM: Removing info for No Bus:target2:0:3
< PM: Adding info for No Bus:target2:0:4
< PM: Removing info for No Bus:target2:0:4
< PM: Adding info for No Bus:target2:0:5
< PM: Removing info for No Bus:target2:0:5
< PM: Adding info for No Bus:target2:0:6
< PM: Removing info for No Bus:target2:0:6
< PM: Adding info for No Bus:target2:0:7
< PM: Removing info for No Bus:target2:0:7
< usb-storage: device scan complete
< eth0: link up, 10Mbps, half-duplex, lpa 0x0021
< audit(1162252141.385:3): audit_pid=2024 old=0 by auid=4294967295
---
Quoted 2 lines
eth0: link up, 100Mbps, full-duplex, lpa
0x45E1audit(1163638345.198:3): audit_pid=1980 old=0 by auid=4294967295
462d413
< time.c: Lost 300 timer tick(s)! rip mwait_idle+0x3f/0x54) 466c417
< eth1: no IPv6 routers present
---
Quoted 1 line
time.c: Lost 1201 timer tick(s)! ripmwait_idle+0x3f/0x54)
468,476c419,430
< time.c: Lost 300 timer tick(s)! rip mwait_idle+0x3f/0x54) < time.c: Lost 300 timer tick(s)! rip _spin_unlock_irq+0x2e/0x31) < time.c: Lost 300 timer tick(s)! rip mwait_idle+0x3f/0x54) < time.c: Lost 300 timer tick(s)! rip mwait_idle+0x3f/0x54) < time.c: Lost 300 timer tick(s)! rip mwait_idle+0x3f/0x54) < time.c: Lost 300 timer tick(s)! rip mwait_idle+0x3f/0x54) < time.c: Lost 300 timer tick(s)! rip mwait_idle+0x3f/0x54) < time.c: Lost 300 timer tick(s)! rip mwait_idle+0x3f/0x54) < time.c: Lost 300 timer tick(s)! rip mwait_idle+0x3f/0x54)
---
Quoted 12 lines
time.c: Lost 1201 timer tick(s)! ripmwait_idle+0x3f/0x54)time.c: Lost 1201 timer tick(s)! ripsha_transform+0x1c/0x1f4)time.c: Lost 1201 timer tick(s)! ripmwait_idle+0x3f/0x54)time.c: Lost 1201 timer tick(s)! rip0x2b5b93ffd59a)time.c: Lost 1201 timer tick(s)! ripmwait_idle+0x3f/0x54)time.c: Lost 1201 timer tick(s)! ripmwait_idle+0x3f/0x54)time.c: Lost 1201 timer tick(s)! ripmwait_idle+0x3f/0x54)time.c: Lost 1201 timer tick(s)! ripmwait_idle+0x3f/0x54)time.c: Lost 1201 timer tick(s)! ripmwait_idle+0x3f/0x54)time.c: Lost 1201 timer tick(s)! ripmwait_idle+0x3f/0x54)time.c: Lost 1201 timer tick(s)! ripmwait_idle+0x3f/0x54)time.c: Lost 1201 timer tick(s)! ripmwait_idle+0x3f/0x54)
yap Andreas Arens send the patch just for me, I am sending it to the maling lists.
Quoted 29 lines · Sergio Monteiro Basto
Quoted 7 lines · Andreas Arens
as I see from the dmesg on the Fedora bugzilla, your acpi tables don't provide an entry to the HPET timer.
As the VIA8237 happens to have a built-in HPET, I was able to force it on using the
attached patch (against 2.6.18) on an X2 system with the same problem, which greatly improved the system stability for me.But I have one Intel(R) Pentium(R) D CPU 2.8 on a VIA8237
My latest suspect of the root of the problem of my computer is not in Processor but in those VIAs. As you find that "don't provide an entry to the HPET timer on acpi tables" it match, but how do you know that ? I don't send DSDT on bugzillaQuoted 6 lines
The patch is hand-crafted from some older clock-tick kernel tree sources I found by googling.
The thing is hackish and not suitable for mainline inclusion, but may be useful nontheless.
If you find it useful, and it helps you please let me know.I try your patch and it give me this differences on dmesg (file attach), detect a different
timer.cbut no improvement without notsc boot option and with notsc the computer got worst.Quoted 1 line
Signature · 1 line
Sérgio M.B.
arch/x86_64/kernel/time.c renamed from arch/x86_64/kernel/time.c.unpatched +45 -0
@@ -42,6 +42,9 @@#ifdef CONFIG_X86_LOCAL_APIC #include <asm/apic.h> #endif +#if 1 +#include <linux/pci_ids.h> +#endif #ifdef CONFIG_CPU_FREQ static void cpufreq_delayed_get (void );
@@ -815,6 +818,48 @@static int hpet_init (void ) {unsigned int id; +#if 1 +union conf_address { +struct { +u8 reg; +u8 func:3 ; +u8 dev:5 ; +u8 bus; +u8 reserved:7 ; +u8 enable:1 ; + } bits; +u32 dword; + }; +union conf_address ca = { + .bits.reg =0 , + .bits.dev =17 , + .bits.enable =1 + }; +union { +struct { +u8 control; +u8 address[3 ]; + } hpet; +unsigned raw; + } hpet; +u32 vendor_id, control; + + control =inl (0xcf8 ); +printk ("%X\n" , control); +outl (ca.dword,0xcf8 ); + vendor_id =inl (0xcfc ); +if (vendor_id == (PCI_VENDOR_ID_VIA + (PCI_DEVICE_ID_VIA_8237 <<16 ))) { + hpet.raw =0xFED00000 ; + hpet.hpet.control =0x80 ; + ca.bits.reg =0x68 ; +outl (ca.dword,0xcf8 ); +outl (hpet.raw,0xcfc ); +outl (ca.dword,0xcf8 ); + vxtime.hpet_address = (inl (0xcfc ) &0xFFFFFF00 ); +printk (KERN_WARNING "time.c: WARNING: Enabled VIA8237 HPET " +"at %#lx.\n" , vxtime.hpet_address); + } +#endif if (!vxtime.hpet_address)return -1 ;
Quoted 6 lines · Andi Kleen
Quoted 4 lines
So far, has I can understand. Seems to me that my computer which have a Pentium D (Dual Core) on VIA chipset, also have unsynchronized TSC and with the patch of hrtimers on
Intel systems (except for some large highend systems) have synchronized TSCs.
Does Intel guarantee that, or is that just what we happen to see, so far.
Quoted 9 lines · On Fri, 27 Oct 2006 23:35:24 -0700 [email protected] wrote:
Quoted 6 lines · Andi Kleen
Quoted 4 lines
So far, has I can understand. Seems to me that my computer which have a Pentium D (Dual Core) on VIA chipset, also have unsynchronized TSC and with the patch of hrtimers on
Intel systems (except for some large highend systems) have synchronized TSCs.
Does Intel guarantee that, or is that just what we happen to see, so far.
Matthias has a Xeon machine on which the TSCs are unsynced, and which are unsyncable - write_tsc() just doesn't do anything. See thread at
http://lkml.org/lkml/2006/7/22/104
Quoted 9 lines · Andrew Morton
Quoted 6 lines · On Fri, 27 Oct 2006 23:35:24 -0700 [email protected] wrote:
Quoted 5 lines · Andi Kleen
Quoted 4 lines
So far, has I can understand. Seems to me that my computer which have a Pentium D (Dual Core) on VIA chipset, also have unsynchronized TSC and with the patch of hrtimers on
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
Intel systems (except for some large highend systems) have synchronized TSCs.
Does Intel guarantee that, or is that just what we happen to see, so far.
Matthias has a Xeon machine on which the TSCs are unsynced, and which are unsyncable -
write_tsc()just doesn't do anything. See thread at
http://lkml.org/lkml/2006/7/22/104
Nothing at all, or just the the low few bits are writeable? I had heard, but never seen that some Intel CPUs only allowed 16 bits of writable bits in the TSC MSR. I also heard of, but never saw, CPUs that cleared the TSC to 0 on a write!
Quoted 12 lines · On Fri, 27 Oct 2006 23:49:24 -0700 [email protected] wrote:
Quoted 11 lines · Andrew Morton
Quoted 8 lines · On Fri, 27 Oct 2006 23:35:24 -0700 [email protected] wrote:
Quoted 6 lines · Andi Kleen
Quoted 4 lines
So far, has I can understand. Seems to me that my computer which have a Pentium D (Dual Core) on VIA chipset, also have unsynchronized TSC and with the patch of hrtimers on
Intel systems (except for some large highend systems) have synchronized TSCs.
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
Does Intel guarantee that, or is that just what we happen to see, so far.
Matthias has a Xeon machine on which the TSCs are unsynced, and which are unsyncable -
write_tsc()just doesn't do anything. See thread at
http://lkml.org/lkml/2006/7/22/104Nothing at all, or just the the low few bits are writeable?
We don't know - the tsc sync code doesn't remeasure the errors after "correcting" them.
Quoted 6 lines · Andrew Morton
Quoted 3 lines
Quoted 1 line
Nothing at all, or just the the low few bits are writeable?
We don't know - the tsc sync code doesn't remeasure the errors after "correcting" them.
I read the thread. Just as a challenge, I'd love to poke at such a system, but I doubt very much I'll get the chance :)
Tim
Quoted 5 lines
Nothing at all, or just the the low few bits are writeable? I had heard, but never seen that some Intel CPUs only allowed 16 bits of writable bits in the TSC MSR. I also heard of, but never saw, CPUs that cleared the TSC to 0 on a write!
Normally on Intel you can only write the first 32bits
-Andi
Quoted 8 lines · Andrew Morton
On Fri, 27 Oct 2006 23:35:24 -0700
Quoted 5 lines · [email protected] wrote:
Quoted 4 lines · Andi Kleen
Quoted 3 lines
So far, has I can understand. Seems to me that my computer which have a Pentium D (Dual Core) on VIA chipset, also have unsynchronized TSC and with the patch of hrtimers on
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Intel systems (except for some large highend systems) have synchronized TSCs.
Does Intel guarantee that, or is that just what we happen to see, so far.
Matthias has a Xeon machine on which the TSCs are unsynced, and which are unsyncable -
write_tsc()just doesn't do anything. See thread at
http://lkml.org/lkml/2006/7/22/104
That is a clear BIOS bug (FSBs are programmed incorrectly) and doesn't seem to be common. In fact the BIOS bug is so bad that it's surprising the system works at all.
-Andi
Quoted 2 lines
Does Intel guarantee that, or is that just what we happen to see, so far.
I don't think it's architecturally guaranteed no.
-Andi
Quoted 4 lines · Lee Revell
Someone recently pointed out to me that a Windows "CPU driver update" supplied by AMD fixes the unsynced TSC problem on dual core AMD64 systems.
...
Quoted 4 lines
What are the chances of Linux getting a similar fix?
Lee
Hi,
This post earlier seems to suggest someone is indeed working on something similar, if I'm understanding this correctly:
http://lkml.org/lkml/2006/10/27/27
quote:
Quoted 3 lines
Jiri Bohac ([email protected]) is currently working on a new timekeeping code for x86-64 that takes a significantly different approach that allows for precise and fast gettimeofday even on CPUs with unsynchronized TSCs.
Cheers,
Friedrich Göpel