time() glitch on 2.4.18 at 177 days uptime?

The problem is with time(). Every second, for approximately 1.1ms, time() reports a value that is about 2^32 microseconds (4295 seconds, or about an hour and a quarter) in the future. The glitches always occur between a change of seconds.

After 180 days of uptime, it's not surprising that there would have been one read of the port that failed, triggering the problem, so I think the kernel should detect and fix this. We could just check for
it: if the returned count > LATCH, read an extra byte from port 0x40, as I did. Or, use the method in do_slow_gettimeoffset, which basically resets the 8253's counter if count > LATCH.

Any comments?

Quoted 1 line

Any comments?

Have a play with it, if your idea works when you deliberately disturb it then send in a patch

Quoted 5 lines

Quoted 2 lines

Any comments?

Have a play with it, if your idea works when you deliberately disturb it then send in a patch

This works well.

-jim

diff -urN linux-2.4.18/arch/i386/kernel/time.c linux-2.4.18-jim/arch/i386/kernel/time.c

arch/i386/kernel/time.c Tue Nov 5 11:22:02 2002 renamed from arch/i386/kernel/time.c Fri Mar 15 18:28:53 2002 +10 -1

@@ -501,6 +501,16 @@
 
                count = inb_p(0x40);    /* read the latched count */
                count |= inb(0x40) << 8;
+
+               /* Any unpaired read will cause the above to swap MSB/LSB
+                  forever.  Try to detect this and reset the counter. */
+               if (count > LATCH) {
+                       outb_p(0x34, 0x43);
+                       outb_p(LATCH & 0xff, 0x40);
+                       outb(LATCH >> 8, 0x40);
+                       count = LATCH - 1;
+               }
+
                spin_unlock(&i8253_lock);
 
                count = ((LATCH-1) - count) * TICK_SIZE;
-

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

arch/i386/kernel/time.c line 507

Quoted 1 line · Jim Paris

arch/i386/kernel/time.c +1 -0

+		if (count > LATCH) {

may be (count >= LATCH) would be even better ?

Quoted 3 lines

arch/i386/kernel/time.c line 507

Quoted 1 line

arch/i386/kernel/time.c +1 -0

+		if (count > LATCH) {

may be (count >= LATCH) would be even better ?

Some PIT clones seem to hold the LATCH value momentarily judging by other things that were triggered wrongly by >=

Quoted 4 lines

Quoted 3 lines

arch/i386/kernel/time.c line 507

Quoted 1 line

arch/i386/kernel/time.c +1 -0

+		if (count > LATCH) {

may be (count >= LATCH) would be even better ?

Quoted 2 lines

Some PIT clones seem to hold the LATCH value momentarily judging by other things that were triggered wrongly by >=

If so, then that's a separate problem: the later code

	count = ((LATCH-1) - count) * TICK_SIZE;
	delay_at_last_interrupt = (count + LATCH/2) / LATCH;

Quoted 7 lines · Jim Paris

Quoted 6 lines

Quoted 3 lines

arch/i386/kernel/time.c line 507

Quoted 1 line

arch/i386/kernel/time.c +1 -0

+		if (count > LATCH) {

may be (count >= LATCH) would be even better ?

Some PIT clones seem to hold the LATCH value momentarily judging by other things that were triggered wrongly by >=

Quoted 5 lines

If so, then that's a separate problem: the later code

	count = ((LATCH-1) - count) * TICK_SIZE;
	delay_at_last_interrupt = (count + LATCH/2) / LATCH;

It might be interesting to catch that case with a printk too and put both in 2.5 and see what comes out in the wash yes

could that be the reason a few people have experienced occasionnal jumps backwards in gettimeofday() a few months ago, which many others could never reproduce ? Just because of buggy hardware ?

This works well.

-jim

diff -urN linux-2.4.18/arch/i386/kernel/time.c linux-2.4.18-jim/arch/i386/kernel/time.c

arch/i386/kernel/time.c +3 -0

@@ -501,6 +501,16 @@
 
 		count = inb_p(0x40);    /* read the latched count */
 		count |= inb(0x40) << 8;
+
+		/* Any unpaired read will cause the above to swap MSB/LSB
+		   forever.  Try to detect this and reset the counter. */

arch/i386/kernel/time.c +10 -1

 		count = inb_p(0x40);    /* read the latched count */
 		count |= inb(0x40) << 8;
+
+		/* Any unpaired read will cause the above to swap MSB/LSB
+		   forever.  Try to detect this and reset the counter. */
+		if (count > LATCH) {
+			outb_p(0x34, 0x43);
+			outb_p(LATCH & 0xff, 0x40);
+			outb(LATCH >> 8, 0x40);
+			count = LATCH - 1;
+		}
+
 		spin_unlock(&i8253_lock);
 
 		count = ((LATCH-1) - count) * TICK_SIZE;
-

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

BTW, why not trying to resync, with something like :

if (count >= LATCH)

	count = (count >> 8) | inb(0x40) << 8;

Quoted 4 lines

BTW, why not trying to resync, with something like :

if (count >= LATCH)

	count = (count >> 8) | inb(0x40) << 8;

That's best if we really are out of sync, but it's hard to tell. It could be that the 8253's latch value got clobbered somehow, in which case we definitely want to fix that or our timer interrupts will come out at the wrong rate. We also still need to double-check that count < LATCH after your code.

Unless we assume that an unpaired read is more common than having the latch value clobbered in some other way, then I think just resetting the counter should be okay. Since none of this _should_ ever happen (but did on my system, grr), it's probably not worth making it too complicated just to try to figure out what went wrong.

Updated patch with the printk and corrected conditional is below.

-jim

diff -urN linux-2.4.18/arch/i386/kernel/time.c linux-2.4.18-jim/arch/i386/kernel/time.c

arch/i386/kernel/time.c +6 -0

@@ -501,6 +501,18 @@
 
 		count = inb_p(0x40);    /* read the latched count */
 		count |= inb(0x40) << 8;
+
+		/* Any unpaired read will cause the above to swap MSB/LSB
+		   forever.  Try to detect this and reset the counter. */
+		if (count >= LATCH) {
+			printk(KERN_WARNING 
+			       "i8253 count too high! resetting..\n");

No! You will break many machines. You cannot use out_p() when writing the latch it __must__ be out(). the "_p" puts a write to another port between the two writes. That will screw up the internal state-machine of most PITs including AMD-SC520.

Quoted 4 lines · Richard B. Johnson

No! You will break many machines. You cannot use out_p() when writing the latch it __must__ be out(). the "_p" puts a write to another port between the two writes. That will screw up the internal state-machine of most PITs including AMD-SC520.

You make an interesting point. Have you checked if an inb_p() on this hardware could unlock the latch when reading the first byte ? It may also be one cause of time jumps if the counter is just about to wrap when it's being read, and gets unlocked by an out to port 0x80 (the "_p").

arch/i386/kernel/time.c line 503

Quoted 2 lines

arch/i386/kernel/time.c +0 -0

 		count = inb_p(0x40);    /* read the latched count */
 		count |= inb(0x40) << 8;

Or perhaps it will be time to change port 0x80 to something else, that no chipset will use. I've seen 0xED in a bios somewhere, but I don't remember where.

Cheers,
Willy

The only hardware a modern PC needs to use "slow-down_io" on is the RTC CMOS device. Since we need to support older boards, you don't want to remove the _p options indiscriminately, but you do not want them ever between two consecutive writes to the same device- port.

Quoted 5 lines · Richard B. Johnson

The only hardware a modern PC needs to use "slow-down_io" on is the RTC CMOS device. Since we need to support older boards, you don't want to remove the _p options indiscriminately, but you do not want them ever between two consecutive writes to the same device- port.

I own at least one that needs the _p on the DMA controller and at one that needs _p on the PIT

Quoted 7 lines · Alan Cox

Quoted 5 lines · Richard B. Johnson

The only hardware a modern PC needs to use "slow-down_io" on is the RTC CMOS device. Since we need to support older boards, you don't want to remove the _p options indiscriminately, but you do not want them ever between two consecutive writes to the same device- port.

Quoted 2 lines

I own at least one that needs the _p on the DMA controller and at one that needs _p on the PIT

Well, in fact, Intel's 82C54 datasheet says that this chip needs at least 165 ns between two consecutive operations, either read or write. So with a 8 Mhz bus, you may effectively need to insert fake accesses, although most modern chipsets certainly have better specs.

But the spec clearly states that you can interleave accesses to other counters between the first and second bytes without problem, so good implementations should see no side effect.

Richard, if your PIT doesn't support accesses to port 80, could you try to use other ports ?

Cheers,
Willy

Quoted 8 lines · Willy Tarreau

Quoted 7 lines · Alan Cox

Quoted 5 lines · Richard B. Johnson

The only hardware a modern PC needs to use "slow-down_io" on is the RTC CMOS device. Since we need to support older boards, you don't want to remove the _p options indiscriminately, but you do not want them ever between two consecutive writes to the same device- port.

Quoted 8 lines

Quoted 2 lines

I own at least one that needs the _p on the DMA controller and at one that needs _p on the PIT

Well, in fact, Intel's 82C54 datasheet says that this chip needs at least 165 ns between two consecutive operations, either read or write. So with a 8 Mhz bus, you may effectively need to insert fake accesses, although most modern chipsets certainly have better specs.

Its not a clocked bus. The 8 MHz means nothing. It's an async bus and if the setup timing of the data/address-decode/chip-select sequence is not violated, you get ~200ns between accesses so chip-to-chip select time comes automatically. But, some older motherboards violated all kinds of specs. They did this deliberately so you could plug RAM boards into the ISA bus and get reasonable performance. If we still support them, you need some artifical waits.

It's not the port address. It's the bus state. The chip requires two consecutive writes when setting that latch. It's just like a 32-bit write to a 32-bit port (like PCI access), you can't write to the low-port, do something else, then write to the high port.

It needs to be done with no intervening bus states. In the AMD case, it's probably because the address is ignored on the second write.

Quoted 19 lines · Richard B. Johnson

Quoted 8 lines · Willy Tarreau

Quoted 7 lines · Alan Cox

Quoted 5 lines · Richard B. Johnson

The only hardware a modern PC needs to use "slow-down_io" on is the RTC CMOS device. Since we need to support older boards, you don't want to remove the _p options indiscriminately, but you do not want them ever between two consecutive writes to the same device- port.

Quoted 8 lines

Quoted 2 lines

I own at least one that needs the _p on the DMA controller and at one that needs _p on the PIT

Well, in fact, Intel's 82C54 datasheet says that this chip needs at least 165 ns between two consecutive operations, either read or write. So with a 8 Mhz bus, you may effectively need to insert fake accesses, although most modern chipsets certainly have better specs.

Quoted 7 lines

Its not a clocked bus. The 8 MHz means nothing. It's an async bus and if the setup timing of the data/address-decode/chip-select sequence is not violated, you get ~200ns between accesses so chip-to-chip select time comes automatically. But, some older motherboards violated all kinds of specs. They did this deliberately so you could plug RAM boards into the ISA bus and get reasonable performance. If we still support them, you need some artifical waits.

Well, some older (386) motherboards had the 8254 directly connected to the ISA bus. And since Linux is 386-compatible, I think we should still support those boards.

Quoted 4 lines

It's not the port address. It's the bus state. The chip requires two consecutive writes when setting that latch. It's just like a 32-bit write to a 32-bit port (like PCI access), you can't write to the low-port, do something else, then write to the high port.

Yes you can ! the chip uses a CS (chip select) from an address decoder, and sees the reads/writes only when both CS and either RD or WR are low. I understand that all this circuitry may be simplified a bit in recent chipsets (one unified decoder, fewer glue logic between chips), but there's no reason that this should not be respected if the chip makers claim a bit of compatibility.

Quoted 2 lines

It needs to be done with no intervening bus states. In the AMD case, it's probably because the address is ignored on the second write.

Then it's buggy. This chip was originaly designed to work for 8086/80186, which barely means work when directly connected to an ISA bus. If AMD make their chip rely on PCI-like back-to-back behaviour, I don't agree for the compatibility.

Now, since there's at least one buggy implementation, the solution is udelay(). But do you think that your chip may be confused by a memory access, when udelay() fetches the count variable from memory, for example ?

The last other solution may be to simply use jmp $+2, which will be slow enough on old sensible hardware, and imperceptible on more recent hardware which doesn't require this.

Since Alan and you have one piece of hardware which show opposite behaviour, we should get an ever working solution quickly :-)

udelay() would work memory/read/write is entirely different from I/O port read/write even though PWB traces are shared. But that might result in wasted CPU cycles.

This works in all cases in machines I have tested. I can't test everything but, depending upon whether or not the forces a cache-line refill, the delay can be from 200 ns to over 800 ns. I have a single instance of this, and call it, rather than doing in-line. This adds a further delay.

Quoted 5 lines · Richard B. Johnson

The only hardware a modern PC needs to use "slow-down_io" on is the RTC CMOS device. Since we need to support older boards, you don't want to remove the _p options indiscriminately, but you do not want them ever between two consecutive writes to the same device- port.

I own at least one that needs the _p on the DMA controller and at one that needs _p on the PIT

Hey, look. I can only warn. You do what you want. As far as I'm concerned support stopped at Linux 2.4.19 when poll got trashed. Nobody can use 2.4.19 or probably anything later unless they have powerful CPUs that can spin with 1000 SIGPOLL signals per second.

Quoted 4 lines · Richard B. Johnson

Hey, look. I can only warn. You do what you want. As far as I'm concerned support stopped at Linux 2.4.19 when poll got trashed. Nobody can use 2.4.19 or probably anything later unless they have powerful CPUs that can spin with 1000 SIGPOLL signals per second.

My 486SLC33 (running at 8MHz on battery mode) doesn't believe you.

Quoted 5 lines · Richard B. Johnson

Quoted 4 lines

Hey, look. I can only warn. You do what you want. As far as I'm concerned support stopped at Linux 2.4.19 when poll got trashed. Nobody can use 2.4.19 or probably anything later unless they have powerful CPUs that can spin with 1000 SIGPOLL signals per second.

Enable SIGPOLL on STDIN_FILENO and connect from the network as previously shown. You will get 100% CPU time on the task with this enabled.

Quoted 8 lines · Richard B. Johnson

Quoted 5 lines · Richard B. Johnson

Quoted 4 lines

Hey, look. I can only warn. You do what you want. As far as I'm concerned support stopped at Linux 2.4.19 when poll got trashed. Nobody can use 2.4.19 or probably anything later unless they have powerful CPUs that can spin with 1000 SIGPOLL signals per second.

Enable SIGPOLL on STDIN_FILENO and connect from the network as previously shown. You will get 100% CPU time on the task with this enabled.

Can you please stop harping about this? You have written essentially the same mail over and over again without adding any new information.

First of all, your problem has nothing to do with networking, the same thing will happen if you run your program from within screen, so it seems to have more to do with ptys and how both screen and telnetd sets them up.

Second, it seems that your program does exactly what is expected. Try applying this patch to your example program and see what happens:

diff -u ./polltest.c.orig ./polltest.c

polltest.c renamed from polltest.c.orig +1 -1

@@ -47,7 +47,7 @@
 {
     struct pollfd pf;
     pf.fd = STDIN_FILENO;
-    pf.events = POLLIN;
+    pf.events = POLLIN | POLLOUT | POLLERR;
     pf.revents = 0;
     (void)poll(&pf, 1, 0);
     if(pf.revents & POLLIN)

Each time you print the "POLLOUT = 3 POLLERR = 0" message, you fill up the write buffer, and each time the write queue empties you will get another SIGIO. Now, this may not be what you want, but it seems to be entirely within specs.

If you remove the fprintf(stderr, "POLLOUT... from your signal handler you will not see the same effect as you are complaining about. Your measurements are interfering with what you are trying to measure.

No! You will break many machines. You cannot use out_p() when writing the latch it __must__ be out(). the "_p" puts a write to another port between the two writes. That will screw up the internal state-machine of most PITs including AMD-SC520.