Skip to contentSkip to conversation

Can we kill f inb_p, outb_p and other random I/O on port 0x80, in 2.6?

DISCUSSION Eric W. Biederman 5 participants 18 messages Sep 22, 2003 – Sep 23, 2003

lore
Share & tools
lore
#0 lore raw

inb_p and outb_p issue outb's to port 0x80 to achieve a short delay. In a reasonable system there is nothing listening to port 0x80 or there is a post card, but there are no other devices there.

On a modern system with no post card the outb travels it's way down to the LPC bus, and the outb is terminated by an abort because nothing is listening.

So far so good. Except for the fact that recent high volume ROM chips get confused when they see an abort on the LPC bus. Making it problematic to update the ROM from under Linux.

I don't know if there are other buggy LPC devices or not. But I do know that it is generally bad form do I/O to a random port.

So can we gradually kill inb_p, outb_p in 2.6? An the other miscellaneous users of I/O port 0x80 for I/O delays?

Or possibly rewriting outb_p to look something like:
outb(); udelay(200); or whatever the appropriate delay is?

When debugging this I modified arch/i386/io.h to read:
#define __SLOW_DOWN_IO__ ""
Which totally removed the delay and the system ran fine.

Eric

#1 lore raw
Quoted 5 lines · Eric W. Biederman

So can we gradually kill inb_p, outb_p in 2.6? An the other miscellaneous users of I/O port 0x80 for I/O delays?

Or possibly rewriting outb_p to look something like:
outb(); udelay(200); or whatever the appropriate delay is?

The delay should be 8 ISA clocks. While you can easily fix inb_p and outb_p you also need to fix up the udelay() code since if you stick a BUG() check in udelay you'll find it gets used before the clock is initialized even now, let alone with inb_p relying on it. But that itself is quite fixable too.

(one part of the problem of course is you need inb_p/outb_p to drive the timer chip on some x86 boards in order to calibrate the udelay timer)

Quoted 3 lines

When debugging this I modified arch/i386/io.h to read:
#define __SLOW_DOWN_IO__ ""
Which totally removed the delay and the system ran fine.

Not all systems do - we had breakages from both the keyboard controller and the timer chips even on some modern boards when this got messed up.

#2 lore raw
Quoted 3 lines · Alan Cox

(one part of the problem of course is you need inb_p/outb_p to drive the timer chip on some x86 boards in order to calibrate the udelay timer)

What sort of timer chip problems do you see? Is it something that can be auto-detected, so that timer chip accesses can be made faster on boards where that is fine?

I'm sure I've seen timer chip code in DOS programs that didn't have the extra delay I/Os. Surely it cannot be a very widespread problem.

Quoted 6 lines
Quoted 3 lines

When debugging this I modified arch/i386/io.h to read:
#define __SLOW_DOWN_IO__ ""
Which totally removed the delay and the system ran fine.

Not all systems do - we had breakages from both the keyboard controller and the timer chips even on some modern boards when this got messed up.

I've also seen much DOS code that didn't have extra delays for keyboard I/Os. What sort of breakage did you observe with the keyboard?

Thanks,
-- Jamie

#3 lore raw
Quoted 3 lines · Jamie Lokier

What sort of timer chip problems do you see? Is it something that can be auto-detected, so that timer chip accesses can be made faster on boards where that is fine?

CS5520 is one example. Also VIA VP2 seems to care but only very very occasionally. On my 386 board its reliably borked without the delays (not sure what chipset and its ISA so harder to tell)

Quoted 3 lines

I've also seen much DOS code that didn't have extra delays for keyboard I/Os. What sort of breakage did you observe with the keyboard?

DEC laptops hang is the well known example of that one.

I'm *for* making this change to udelay, it just has to start up with a suitably pessimal udelay assumption until calibrated.

#4 lore raw

On Mon, 2003-09-22 at 18:33, Alan Cox wrote:
\

Quoted 8 lines
Quoted 3 lines

I've also seen much DOS code that didn't have extra delays for keyboard I/Os. What sort of breakage did you observe with the keyboard?

DEC laptops hang is the well known example of that one.

I'm *for* making this change to udelay, it just has to start up with a suitably pessimal udelay assumption until calibrated.

or we make udelay() do the port 80 access in the uncalibrated case....

The first person to complain about the extra branch miss in udelay for this will get laughed at by me ;)

#5 lore raw
Quoted 2 lines · Arjan van de Ven

The first person to complain about the extra branch miss in udelay for this will get laughed at by me ;)

udelay(1) is too slow on a 386 even without the branch miss.

If you think I/O operations are infinitely slower than other instructions, please explain why there is asm-optimised I/O code in asm-i386/floppy.h.

:)

-- Jamie

#6 lore raw
Quoted 5 lines · Jamie Lokier
Quoted 2 lines · Arjan van de Ven

The first person to complain about the extra branch miss in udelay for this will get laughed at by me ;)

udelay(1) is too slow on a 386 even without the branch miss.

Hmm. I will have to test that one.

Quoted 5 lines

If you think I/O operations are infinitely slower than other instructions, please explain why there is asm-optimised I/O code in asm-i386/floppy.h.

:)

Because the kernel was initially written in assembly and then fixed?

Eric

#7 lore raw
In reply to Jamie Lokier
Quoted 5 lines · Jamie Lokier
Quoted 2 lines · Arjan van de Ven

The first person to complain about the extra branch miss in udelay for this will get laughed at by me ;)

udelay(1) is too slow on a 386 even without the branch miss.

ok we have ndelay() now as well in 2.6

#8 lore raw
Quoted 6 lines · Arjan van de Ven
Quoted 4 lines
Quoted 2 lines

The first person to complain about the extra branch miss in udelay for this will get laughed at by me ;)

udelay(1) is too slow on a 386 even without the branch miss.

ok we have ndelay() now as well in 2.6

I was sort of joking, but the point is that a function call might be too slow on a 386.

On a 386, a function call for every byte transferred to/from the floppy disk might show up as a real overhead - this occurred to me because someone once thought it was worth rewriting the floppy data transfer code in assembly language, and presumably it did improve performance at the time.

Of course the answer is to not use udelay() on 386-optimised configurations.

-- Jamie

#9 lore raw
In reply to Jamie Lokier

In article <[email protected]>, Jamie Lokier <[email protected]> wrote:

| Arjan van de Ven wrote:
| > The first person to complain about the extra branch miss in udelay for
| > this will get laughed at by me ;)
| 
| udelay(1) is too slow on a 386 even without the branch miss.
| 
| If you think I/O operations are infinitely slower than other
| instructions, please explain why there is asm-optimised I/O code in
| asm-i386/floppy.h.
| 
| :)

The choices are:
1 - there really were some old crappy chips which were both slow and
timing sensitive
2 - someone thought that would optimize access
3 - gcc of the time generated bad code if you didn't

Signature · 3 lines
bill davidsen <[email protected]>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
#10 lore raw
In reply to Arjan van de Ven
Quoted 8 lines · Arjan van de Ven

On Mon, 2003-09-22 at 18:33, Alan Cox wrote:
\

Quoted 6 lines
Quoted 3 lines

I've also seen much DOS code that didn't have extra delays for keyboard I/Os. What sort of breakage did you observe with the keyboard?

DEC laptops hang is the well known example of that one.

Quoted 7 lines
Quoted 2 lines

I'm *for* making this change to udelay, it just has to start up with a suitably pessimal udelay assumption until calibrated.

or we make udelay() do the port 80 access in the uncalibrated case....

The first person to complain about the extra branch miss in udelay for this will get laughed at by me ;)

Sounds like a solution. I will see what I can do in that direction. Maintaining a suitably pessimistic udelay with multi gigahertz chips sounds like a challenge, so using outb to port 0x80 may be a reasonable solution there.

Alan, can you describe a little more what the original delay is needed for? I don't see it documented in my 8254 data sheet. The better I can understand the problem the better I can write the comments on this magic bit of code as I fix it.

The oldest machine I have is a 386 MCA system. Any chance of the bug showing up there? I'd love to have a test case.

Another reason for fixing this is we are killing who knows how much I/O bandwidth with this stream of failing writes to port 0x80.

Eric

#11 lore raw
Quoted 10 lines · Eric W. Biederman

Alan, can you describe a little more what the original delay is needed for? I don't see it documented in my 8254 data sheet. The better I can understand the problem the better I can write the comments on this magic bit of code as I fix it.

The oldest machine I have is a 386 MCA system. Any chance of the bug showing up there? I'd love to have a test case.

Another reason for fixing this is we are killing who knows how much I/O bandwidth with this stream of failing writes to port 0x80.

Unfortunately, a lot of drivers use the _p operators so if the delay is simply removed, it may take a while before the ill effects of that are discovered.

-- Jamie

#12 lore raw
In reply to Eric W. Biederman
Quoted 4 lines · Eric W. Biederman

Alan, can you describe a little more what the original delay is needed for? I don't see it documented in my 8254 data sheet. The better I can understand the problem the better I can write the comments on this magic bit of code as I fix it.

If I remember rightly its because it is a 2Mhz part on an 8Mhz bus.

Quoted 2 lines

The oldest machine I have is a 386 MCA system. Any chance of the bug showing up there? I'd love to have a test case.

No idea

Quoted 2 lines

Another reason for fixing this is we are killing who knows how much I/O bandwidth with this stream of failing writes to port 0x80.

Definitely - and if we can boot up with udelay() using some pessimal value then we don't even need the branch, just the udelay for the 8 ISA cycles.

Alan

#13 lore raw

In article <[email protected]>, Alan Cox <[email protected]> wrote:

| On Llu, 2003-09-22 at 19:58, Eric W. Biederman wrote:
| > Alan, can you describe a little more what the original delay is needed
| > for?  I don't see it documented in my 8254 data sheet.  The better I
| > can understand the problem the better I can write the comments on this
| > magic bit of code as I fix it.
| 
| If I remember rightly its because it is a 2Mhz part on an 8Mhz bus.

And I thought I was a hotshot overclocker ;-)

Signature · 3 lines
bill davidsen <[email protected]>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
#14 lore raw
In reply to Alan Cox

A few architectures define in[bwl]_p & out[bwl]_p differently from in[bwl] and out[bwl]. These are:

i386 (all machines)
- four methods are offered; the normal one is 1 write to port 0x80. The others are selected by manually editing a file.
- The port 0x80 is hard-coded in assembly language in <asm-i386/floppy.h>, although that code is no longer used.

m68k
- A delay is done for some machines. It is a write to port 0x80.

MIPS
- normally disabled, but can be enabled if CONF_SLOWDOWN_IO is manually edited in a file. It is a write to port 0x80.

PPC64
- the input operations don't have a delay
- the output operations have a udelay(1), _before_ the I/O.
- PPC (32 bit) doesn't have delays at all

SH (all machines except ec3104)
- only byte-size access has the delay method, except on the "overdrive" where all sizes have it.
- delay is done by a special I/O operation, ctrl_inw(0xa0000000)
- one SH board uses udelay(10) for the delay.
- another SH board has a funky single delay after insw/insl/outsw/outsl (but not insb/outsb), using a different special I/O, ctrl_inb(0xba000000).

x86_64
- four methods just like i386

Some of the architectures _call_ the _p operators even for some architecture-specific devices, even though those operators don't have a delay.

The m68k implementation is nice: it defines a function called isa_delay(), which contains the delay which is done after the I/O by _p operations.

IMHO, it would be nice if all architectures simply provided an appropriate isa_delay() function, and the _p I/O operators were simply removed. That would make the intent of drivers using those operators a little clearer, too.

Devices
=======

I think they're all ISA devices, or emulations of them.

Unfortunately, a _lot_ of drivers in the kernel use the _p I/O operators so it's not possible to look for a few special quirky devices. I suspect that in many cases, the _p operators are not required but have been used for safety. There is no harm in this of course.

Alan mentioned the PIT timer chip and keyboard which appear on x86 machines as being specifically quirky.

I find it interesting, but awfully difficult to know whether it's "correct", that the i386 PIT code uses inb/outb for some operations and inb_p/outb_p for some others.

Quoted 8 lines · Alan Cox
Quoted 3 lines · Jamie Lokier

What sort of timer chip problems do you see? Is it something that can be auto-detected, so that timer chip accesses can be made faster on boards where that is fine?

CS5520 is one example. Also VIA VP2 seems to care but only very very occasionally. On my 386 board its reliably borked without the delays (not sure what chipset and its ISA so harder to tell)

Yeah, but what's the problem and can it be detected? :)

Quoted 8 lines
Quoted 3 lines

I've also seen much DOS code that didn't have extra delays for keyboard I/Os. What sort of breakage did you observe with the keyboard?

DEC laptops hang is the well known example of that one.

I'm *for* making this change to udelay, it just has to start up with a suitably pessimal udelay assumption until calibrated.

I'm wondering if there's a way to detect how much udelay is needed on a particular board, and reduce or remove it on boards where it isn't needed.

udelay() is also unreliable nowadays, due to CPUs changing clock speeds according to the whims of the BIOS. On laptops, even the rdtsc rate varies. If the delay is critical to system reliability for unknown reasons, then switching to udelay() removes some of that "we always did this and it fixed the unknown problems" legacy driver safety

Unfortunately, there are a lot of drivers, and a lot of x86 arch-specific code, which use the delay operaters. There's no real way to verify that all the drivers are fine when the delay is reduced or removed.

The delay should only be effective for ISA devices. I wonder if it makes sense to separate the delay into an isa_delay() or io_delay() function, sprinkled into source where the _p operators currently appear, to make it clearer where the delays should appear.

In the i386 floppy driver, for example, it's clear why the delay is there in one part of the virtual DMA loop and not the other: to allow a device time to propagate a state change. That also explains the need for a delay after writing to port 0x42 of the PIT but not after writing port 0x43. I think calls to isa_delay() would make the intent there a little clearer.

-- Jamie

#15 lore raw
Quoted 5 lines · Jamie Lokier

A few architectures define in[bwl]_p & out[bwl]_p differently from in[bwl] and out[bwl]. These are:

	x86_64
		- four methods just like i386

At least until some new chipsets come out I can certify that x86_64 works just fine without the delay.

Quoted 4 lines

Some of the architectures _call_ the _p operators even for some architecture-specific devices, even though those operators don't have a delay.

Quoted 8 lines

The m68k implementation is nice: it defines a function called isa_delay(), which contains the delay which is done after the I/O by _p operations.

IMHO, it would be nice if all architectures simply provided an appropriate isa_delay() function, and the _p I/O operators were simply removed. That would make the intent of drivers using those operators a little clearer, too.

It certainly removes the magic coupling.

Quoted 14 lines

Devices
=======

I think they're all ISA devices, or emulations of them.

Unfortunately, a _lot_ of drivers in the kernel use the _p I/O operators so it's not possible to look for a few special quirky devices. I suspect that in many cases, the _p operators are not required but have been used for safety. There is no harm in this of course.

Alan mentioned the PIT timer chip and keyboard which appear on x86 machines as being specifically quirky.

Quoted 8 lines

I find it interesting, but awfully difficult to know whether it's "correct", that the i386 PIT code uses inb/outb for some operations and inb_p/outb_p for some others.

I'm wondering if there's a way to detect how much udelay is needed on a particular board, and reduce or remove it on boards where it isn't needed.

Until the problem is more clearly defined I don't think we can auto-detect. The best we can do is to have drivers that replace the legacy drivers when appropriate and don't do the delay. If the delay is done with udelay though it does not cause any bus traffic and another device can be using the bus.

Quoted 5 lines

udelay() is also unreliable nowadays, due to CPUs changing clock speeds according to the whims of the BIOS. On laptops, even the rdtsc rate varies. If the delay is critical to system reliability for unknown reasons, then switching to udelay() removes some of that "we always did this and it fixed the unknown problems" legacy driver safety

As long as udelay is calibrated at the fast system clock speed we are ok. When the clock slows down the delay just increases, which is fine.

Quoted 4 lines

Unfortunately, there are a lot of drivers, and a lot of x86 arch-specific code, which use the delay operaters. There's no real way to verify that all the drivers are fine when the delay is reduced or removed.

We just need something sufficiently good. If the delay is removed on a system that needs it someone will complain.

Quoted 11 lines

The delay should only be effective for ISA devices. I wonder if it makes sense to separate the delay into an isa_delay() or io_delay() function, sprinkled into source where the _p operators currently appear, to make it clearer where the delays should appear.

In the i386 floppy driver, for example, it's clear why the delay is there in one part of the virtual DMA loop and not the other: to allow a device time to propagate a state change. That also explains the need for a delay after writing to port 0x42 of the PIT but not after writing port 0x43. I think calls to isa_delay() would make the intent there a little clearer.

I would agree with that. Although I wonder if we are not mixing up various delays into one mechanism. In any event it is a small safe step forward to entangling this legacy confusion.

Eric

#16 lore raw

In article <[email protected]>, Eric W. Biederman <[email protected]> wrote:
| Jamie Lokier <[email protected]> writes:

| > Unfortunately, there are a lot of drivers, and a lot of x86
| > arch-specific code, which use the delay operaters.  There's no real
| > way to verify that all the drivers are fine when the delay is reduced
| > or removed.
| 
| We just need something sufficiently good.  If the delay is removed
| on a system that needs it someone will complain.

The only problem with that is that is that (a) a complaint and a dollar will get you a cheap beer, but this is Linux and no one *needs* to fix it, therefore not breaking it becomes more important. The top developers are not running legacy 386's, I bet. And (b) if the problem comes up months from now, will anyone think to try timing changes for "every once in a while" problems.

I really like the isa_delay() idea, or similar, which will be in a single place and probably get enough attention to make it work. It just sounds like a safer way to go with equal benefits.

Signature · 3 lines
bill davidsen <[email protected]>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
#17 lore raw
In reply to Jamie Lokier
Quoted 5 lines · Jamie Lokier
Quoted 3 lines

CS5520 is one example. Also VIA VP2 seems to care but only very very occasionally. On my 386 board its reliably borked without the delays (not sure what chipset and its ISA so harder to tell)

Yeah, but what's the problem and can it be detected? :)

You get bogus results

Quoted 3 lines

I'm wondering if there's a way to detect how much udelay is needed on a particular board, and reduce or remove it on boards where it isn't needed.

8 ISA cycles will be nice and safe - see the specification for the ISA bus. Its a nice easy known value and the way we moved various drivers to udelay that used isa delay cycles for timing loops

Quoted 5 lines

udelay() is also unreliable nowadays, due to CPUs changing clock speeds according to the whims of the BIOS. On laptops, even the rdtsc rate varies. If the delay is critical to system reliability for unknown reasons, then switching to udelay() removes some of that "we always did this and it fixed the unknown problems" legacy driver safety

Delaying too long is ok, delaying too little isnt good but as you say most modern hw seems not to care.