Memory mapped IO vs Port IO
A message that is not in the archive (sent off-list or not yet received): 20030911160116.GI21596@parcelfarce.linux.theplanet.co.uk.suse.lists.linux.kernel
Quoted 2 lines · Matthew Wilcox
There's a lot of drivers in the tree that allow you to access the device either via IO port space or IO mem space. Here's some examples:
My gut feeling is to just fix the drivers to make this runtime switchable and get rid of the compile time options.
This would help distributions (who normally want to build conservative by default, but still allow the users easy tuning without recompilation) For that it would be nice if a standard module parameter or maybe sysfs option existed.
The overhead of checking for PIO vs mmio at runtime in the drivers should be completely in the noise on any non ancient CPU (both MMIO and PIO typically take hundreds or thousands of CPU cycles for the bus access, having an dynamic function call or an if before that is makes no difference at all)
-Andi
Quoted 5 lines · Andi Kleen
The overhead of checking for PIO vs mmio at runtime in the drivers should be completely in the noise on any non ancient CPU (both MMIO and PIO typically take hundreds or thousands of CPU cycles for the bus access, having an dynamic function call or an if before that is makes no difference at all)
That's not true for MMIO writes which are posted. They should take no longer than a memory write. For MMIO reads and PIO reads & writes, you are, of course, correct.
Signature · 3 lines
"It's not Hollywood. War is real, war is primarily not about defeat or victory, it is about death. I've seen thousands and thousands of dead bodies. Do you think I want to have an academic debate on this subject?" -- Robert Fisk
Quoted 10 lines · On Thu, 11 Sep 2003 17:25:04 +0100 Matthew Wilcox <[email protected]> wrote:
Quoted 5 lines · Andi Kleen
The overhead of checking for PIO vs mmio at runtime in the drivers should be completely in the noise on any non ancient CPU (both MMIO and PIO typically take hundreds or thousands of CPU cycles for the bus access, having an dynamic function call or an if before that is makes no difference at all)
That's not true for MMIO writes which are posted. They should take no longer than a memory write. For MMIO reads and PIO reads & writes, you are, of course, correct.
Even a memory write is tens to hundres of cycles.
-Andi
Quoted 7 lines · Andi Kleen
Quoted 3 lines · On Thu, 11 Sep 2003 17:25:04 +0100 Matthew Wilcox <[email protected]> wrote:
That's not true for MMIO writes which are posted. They should take no longer than a memory write. For MMIO reads and PIO reads & writes, you are, of course, correct.
Even a memory write is tens to hundres of cycles.
So are mispredicted branches.
Signature · 3 lines
"It's not Hollywood. War is real, war is primarily not about defeat or victory, it is about death. I've seen thousands and thousands of dead bodies. Do you think I want to have an academic debate on this subject?" -- Robert Fisk
Quoted 1 line · Andi Kleen
Even a memory write is tens to hundres of cycles.
Not from the CPU's perspective. It is done in parallel with other instructions.
-- Jamie
Quoted 5 lines · On Thu, 11 Sep 2003 18:12:05 +0100 Jamie Lokier <[email protected]> wrote:
Quoted 1 line · Andi Kleen
Even a memory write is tens to hundres of cycles.
Not from the CPU's perspective. It is done in parallel with other instructions.
Only when there are more instructions to execute. But device driver code often does a following read e.g. to check if it can submit another request to the hardware.
My claim is basically:
Change everybody who currently does
#ifdef CONFIG_MMIO writel(... ) readl(...) #else outl( ... ) inl ( ...)
#endif
to
if (dev->mmio) {
writel();
real();
} else {
outl();
inl();
} and you will have a hard time to benchmark the difference on any non ancient system in actual driver operation.
-Andi
Quoted 34 lines · Andi Kleen
Quoted 5 lines · On Thu, 11 Sep 2003 18:12:05 +0100 Jamie Lokier <[email protected]> wrote:
Quoted 1 line · Andi Kleen
Even a memory write is tens to hundres of cycles.
Not from the CPU's perspective. It is done in parallel with other instructions.
Only when there are more instructions to execute. But device driver code often does a following read e.g. to check if it can submit another request to the hardware.
My claim is basically:
Change everybody who currently does
#ifdef CONFIG_MMIO writel(... ) readl(...) #else outl( ... ) inl ( ...)#endif
to
if (dev->mmio) { writel(); real(); } else { outl(); inl(); }
Quoted 2 lines
and you will have a hard time to benchmark the difference on any non ancient system in actual driver operation.
Shouldn't that be
dev->dev_ops->writel(...);
dev->dev_ops->readl(...);
with no conditionals?
Signature · 5 lines
________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: [email protected] Remember Cernan and Schmitt
Quoted 34 lines · Andi Kleen
Quoted 5 lines · On Thu, 11 Sep 2003 18:12:05 +0100 Jamie Lokier <[email protected]> wrote:
Quoted 1 line · Andi Kleen
Even a memory write is tens to hundres of cycles.
Not from the CPU's perspective. It is done in parallel with other instructions.
Only when there are more instructions to execute. But device driver code often does a following read e.g. to check if it can submit another request to the hardware.
My claim is basically:
Change everybody who currently does
#ifdef CONFIG_MMIO writel(... ) readl(...) #else outl( ... ) inl ( ...)#endif
to
if (dev->mmio) { writel(); real(); } else { outl(); inl(); }
Quoted 6 lines
and you will have a hard time to benchmark the difference on any non ancient system
in actual driver operation.-Andi
Hello,
Wouldn't it be better if we set the IN and OUT function pointers to the right functions during driver init. based on the setting of dev->mmio. And throughout the driver, we just call the IN and OUT functions by their pointers. Then we don't have to do if (dev->mmio) every time. It's similar to the concept of virtual member function in C++.
ADT
Quoted 5 lines · Anthony Dominic Truong
Wouldn't it be better if we set the IN and OUT function pointers to the right functions during driver init. based on the setting of
dev->mmio. And throughout the driver, we just call the IN and OUT functions by their pointers. Then we don't have to do if (dev->mmio) every time. It's similar to the concept of virtual member function in C++.
I think it would be faster to hide the "if (mmio)" part into an inline function which calls one or the other, given a resource cookie.
Branch prediction will remove most of the cost of a conditional test, which is always the same in these drivers after all, but I don't think it's quite so good at predicting indirect function calls.
-- Jamie
Quoted 22 lines · Anthony Dominic Truong
Quoted 21 lines · Andi Kleen
#ifdef CONFIG_MMIO writel(... ) readl(...) #else outl( ... ) inl ( ...)#endif
to
if (dev->mmio) { writel(); real(); } else { outl(); inl(); }and you will have a hard time to benchmark the difference on any non ancient system
in actual driver operation.
Quoted 8 lines
Quoted 2 lines
-Andi
Hello,
Wouldn't it be better if we set the IN and OUT function pointers to the right functions during driver init. based on the setting ofdev->mmio. And throughout the driver, we just call the IN and OUT functions by their pointers. Then we don't have to do if (dev->mmio) every time. It's similar to the concept of virtual member function in C++.
I'd rather not see any more pointer dereferences or even branches than absolutely necessary. Right now, readX/writeX and inX/outX are usually inlines, and outX and writeX can be very fast. So I'd prefer either a global CONFIG_MMIO option or consistent driver specific options that explain what the difference between MMIO and port I/O actually is.
Jesse
Ok, Andi asked for benchmarks, so I ran some. Let this should be a lesson on why you shouldn't use port I/O :) I ran these on an SGI Altix w/900 MHz McKinley processors.
Just straight calls to the routines (all of these are based on the average of 100 iterations):
writeq(val, reg) time: 64 cycles
outl(val, reg) time: 2126 cycles
A simple branch:
if (use_mmio)
writeq(val, reg) time: 132 cycles
else
outl(val, reg) time: 1990 cycles
Using a wrapper to point to one of the routines:
wrapper->write(val, reg) time: 215 cycles
wrapper->out(val, reg) time: 3931 cycles
Thanks,
Jesse
Quoted 7 lines · Andi Kleen
My gut feeling is to just fix the drivers to make this runtime switchable and get rid of the compile time options.
This would help distributions (who normally want to build conservative by default, but still allow the users easy tuning without recompilation) For that it would be nice if a standard module parameter or maybe sysfs option existed.
Another way to help distributions is to compile those drivers twice, once for each access type. There aren't all that many drivers that need it.
Quoted 5 lines
The overhead of checking for PIO vs mmio at runtime in the drivers should be completely in the noise on any non ancient CPU (both MMIO and PIO typically take hundreds or thousands of CPU cycles for the bus access, having an dynamic function call or an if before that is makes no difference at all)
Ah, but what about the ancient CPUs?
-- Jamie
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 4 lines · [email protected] (Jesse Barnes) writes:
I like this idea, esp. as a global option. It would help e.g. visws which can't reliably do MMIO. Most platforms could default MMIO to y also, to make things easier.
The problem AFAIK is usually that there are some buggy older chipset versions that do not work properly with MMIO, but others do. That's especially an issue on widely cloned cards like Tulip.
It's not a platform specific thing, more a device specific setting. Of course some platforms want to disable it completely too, but that's rare.
-Andi