{read,write}s{b,w,l} or iobarrier_*()
Hi !
Here I need people to agree so we decide once for all what we want and move further (this has been discussed several times already withotu agreement).
So, the "generic" problem is: reading/writing from/to a FIFO that is larger than 8 bits wide on a big endian machine requires more than just a loop of reads or writes. Those IO functions are doing byteswap (and barriers on every call) while the FIFO access wants typically to be not byteswapped nor have barriers on every access.
For "PIO", we already provide the "s" functions {in,out}s{b,w,l} that are implemented by every arch and provide drivers with a good abstraction to use (see ide_insw/ide_outsw in 2.5 which should just be a simple insw/outsw for normal interfaces for example).
However, we don't provide a similar abstraction for MMIO. Typical cases where we want this is MMIO versions of the IDE iops (we probably want to provide generic impl of both PIO and MMIO ones in ide-iops, not just PIO); some sound cards, etc...
A driver needing that today will have to either use a loop of {read,write}{b,w,l} and undo byteswapping (ugh), or try to re-implement it using raw_{read,write}{b,w,l} and adding proper iobarrier_* where needed. But the iobarrier functions aren't provided by all archs which means ugly #ifdef salad, and I hate having to put the burden of knowing everything about barriers one each single driver maintainer.
So we have 2 solutions here (one of which I prefer, but I still want the debate open here):
- Have all archs provide {read,write}s{b,w,l} functions.
Those will hide all of the details of bytewapping & barriers from the drivers and can be used as-is for things like IDE MMIO iops.
- Have all archs provide iobarrier_* functions. Here, drivers
would still have to re-implement the transfer loops with raw_{read,write}{b,w,l} and do proper use of iobarrier_*.
Ben.
Quoted 7 lines · Benjamin Herrenschmidt
So we have 2 solutions here (one of which I prefer, but I still want the debate open here):
- Have all archs provide {read,write}s{b,w,l} functions.
Those will hide all of the details of bytewapping & barriers from the drivers and can be used as-is for things like IDE MMIO iops.
I prefer this solution...
Quoted 3 lines
- Have all archs provide iobarrier_* functions. Here, drivers
would still have to re-implement the transfer loops with raw_{read,write}{b,w,l} and do proper use of iobarrier_*.
I have a tulip patch from Peter de Shivjer (sp?) that adds iobarrier_rw() and I think it looks ugly as sin. I would much prefer the first solution...
Jeff
From: Jeff Garzik <[email protected]>
Date: Thu, 26 Sep 2002 12:23:41 -0400
Quoted 4 lines · Benjamin Herrenschmidt
- Have all archs provide {read,write}s{b,w,l} functions.
Those will hide all of the details of bytewapping & barriers from the drivers and can be used as-is for things like IDE MMIO iops.
I prefer this solution...
I'm starting to think about taking back all the previous arguments I had against this idea. It's starting to sound like the preferred way to go.
Quoted 9 lines · David S. Miller
From: Jeff Garzik <[email protected]>
Date: Thu, 26 Sep 2002 12:23:41 -0400Quoted 4 lines · Benjamin Herrenschmidt
- Have all archs provide {read,write}s{b,w,l} functions.
Those will hide all of the details of bytewapping & barriers from the drivers and can be used as-is for things like IDE MMIO iops.
Quoted 5 lines
I prefer this solution...
I'm starting to think about taking back all the previous arguments I had against this idea. It's starting to sound like the preferred way to go.
Hi Dave,
One of the main reasons for changing the core iops of the ata/ide driver results from newer HBA's either supporting dual transports modes or in some case exclusive MMIO, similar to current IOMIO x86 HBA's today.
Additionally, I suspect the current dual path for execution may end up going N-ways. As there are a few PATA hosts and in the future all SATA hosts will convert to the FP-DMA (First Party DMA) messaging service protocol.
This will then make (3) different execution transports. The bad news is during transition period (just now starting) it will become more painful if we do not make the drive independent of iops regardless of arch/bus issues.
One of the key ideas hottly debated between Ben, Jeff, and myself were the impacts resulting from this change in direction. Obviously Ben lives in a world of MMIO and has suffered with a driver which was designed around IOMIO. Thus all the bastardized macros we all hate resulted from x86-centric lameness (industry driven for the most part, yeah me included).
Now if we expand the issue into Jeff's world of the net-stack drivers, he banged me over the head with the issue of "pci-posting delays". Ben got his shots in also about the issue, too. Thus the resulting io_barrier additions by Ben to the original ATA-driver transformation, can be extened to the Net-Drivers. Oh and the slope is increasing fast now.
So if it works for ATA and NET, could it not migrate to all hardware? If it makes it to all hardware, should it not be coupled to the BUS? If it is coupled to the BUS, are there going to be problems with exceptions?
Well once I leave ATA (storage in general), I am not really able to discuss those issues from a position of first hand experience. So here is my nickel spent.
Cheers,
Andre Hedrick
LAD Storage Consulting Group
Quoted 6 lines
Now if we expand the issue into Jeff's world of the net-stack drivers, he banged me over the head with the issue of "pci-posting delays". Ben got his shots in also about the issue, too. Thus the resulting
io_barrieradditions by Ben to the original ATA-driver transformation, can be extened to the Net-Drivers. Oh and the slope is increasing fast now.
Actually, the iobarrier is a slightly different issue, it's not related pci posting delays. (The only sane way to deal with them is to do a read() from the same bus path).
The iobarrier's are a CPU-specific things that basically tell the CPU not to re-order accesses around the barrier. This is necessary for reads & writes to IO locations to be done in the order they are written and not be speculated on CPUs that can do that sort of things (PPC is one). By default, IO macros like {in,out}* and {read,write}* do that implicitely (at least the PPC impl. of them does the barriers) in addition to byteswap. But the raw_* versions that we would need to avoid byteswap in implementing MMIO insw/outsw also don't do the barriers, thus the macro I had to add.
Anyway, looks like we finally agree on getting {read,write}s{b,w,l} in. I'll post a patch implementing them for PPC by tomorrow. That will make things easier for IDE and possibly other drivers as well.
BTW. Jeff: the tulip patch don't really need those explicit barriers from what I understand of it, you can probably merge the patch without them.
Ben.
Quoted 9 lines · Jeff Garzik
Quoted 7 lines · Benjamin Herrenschmidt
So we have 2 solutions here (one of which I prefer, but I still want the debate open here):
- Have all archs provide {read,write}s{b,w,l} functions.
Those will hide all of the details of bytewapping & barriers from the drivers and can be used as-is for things like IDE MMIO iops.
Quoted 8 lines
I prefer this solution...
Quoted 3 lines
- Have all archs provide iobarrier_* functions. Here, drivers
would still have to re-implement the transfer loops with raw_{read,write}{b,w,l} and do proper use of iobarrier_*.I have a tulip patch from Peter de Shivjer (sp?) that adds
Peter De Schrijver, I assume.
Quoted 2 lines
iobarrier_rw()and I think it looks ugly as sin. I would much prefer the first solution...
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]
In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds