Skip to contentSkip to conversation

PCI Bursting with PIO

DISCUSSION Dan Gora 4 participants 7 messages Feb 15, 2008

lore
Share & tools
lore

Dan Gora

author
#0 lore raw

Hi,

I am trying to optimize a driver for a slave only PCI device and am having a lot of trouble getting any kind of PCI burst transactions in either the read or the write direction. Using bcopy/memcpy or even a hand-crafted while (len) { *pdst++ = *psrc++} (with pdst and psrc unsigned long*) I can only get writes to burst and even in that case only for 2 data phases (8 bytes) and only on 64 bit machines. The best that I have managed is to use a hand crafted asm function which copies the data through mmx registers on i386 machines, but that still only bursts a maximum of 16 bytes in the write direction and not at all in the read direction. The source and destination pointers are both aligned to 8 byte boundaries, so I don't think that it's an alignment issue.

Is there any way to get PIO to burst over the PCI bus in the read and write direction? My device has 4 BAR registers, but the area where I am transferring data is marked 'prefetchable' (although the others are not). I read here: http://lkml.org/lkml/2004/9/23/393 that this was a prerequisite, but it is apparently not sufficient. He also mentioned that the area had to be marked as write-back, but it's not clear how you can tell (no /proc/mtrr doesn't tell you) or that it has anything to do with bursting reads.

Any ideas would be really appreciated,

thanks-
dan

#1 lore raw
Quoted 2 lines · Dan Gora

Is there any way to get PIO

I assume you really mean MMIO, not PIO. PIO would be port IO.

Quoted 2 lines

to burst over the PCI bus in the read and
write direction?

You should set the MMIO mapping to write combining using an MTRR You might need to add appropiate memory barriers if you rely on write ordering though.

-Andi

author
#2 lore raw
Quoted 5 lines · Andi Kleen
Quoted 2 lines · Dan Gora

Is there any way to get PIO

I assume you really mean MMIO, not PIO. PIO would be port IO.

Sorry, I always saw it referred to as "Programmed I/O" as opposed to DMA...

Quoted 1 line

You should set the MMIO mapping to write combining using an MTRR

Sorry to be thick here, but how would I go about doing that?

Quoted 2 lines

You might need to add appropiate memory barriers if you rely on write ordering though.

Ok, thanks for the info...

dan

#3 lore raw
In reply to Dan Gora
Quoted 2 lines

Is there any way to get PIO to burst over the PCI bus in the read and write direction? My device has 4 BAR registers, but the area where I

I think you are doign about as well as the X folks did when they spent time on trying to optimise pio transfers to and from graphics card RAM.

Quoted 1 line

Any ideas would be really appreciated,

Put a DMA controller on it ;)

Alan

author
#4 lore raw
Quoted 6 lines · Alan Cox
Quoted 2 lines

Is there any way to get PIO to burst over the PCI bus in the read and write direction? My device has 4 BAR registers, but the area where I

I think you are doign about as well as the X folks did when they spent time on trying to optimise pio transfers to and from graphics card RAM.

That's good to know. Do you have a link or anything to their discussion or some key words that I could hunt it down?

Quoted 4 lines
Quoted 2 lines

Any ideas would be really appreciated,

Put a DMA controller on it ;)

Ugh.. sadly that's what's coming. I really don't get why the northbridge cannot burst however. If the memory is mapped prefetchable and you have to do a PCI read through 3 PCIe bridges to finally get to your device it seems like it would _really_ behoove the bridge to do a Memory read multiple and get the whole cache line. I have searched around a lot and there doesn't seem to be any info at all on how you can persuade these bridges to do different PCI commands or burst. I don't know why....

thanks again for your help,

dan

#5 lore raw
Quoted 5 lines · Dan Gora
Quoted 2 lines

Put a DMA controller on it ;)

Ugh.. sadly that's what's coming. I really don't get why the northbridge cannot burst however.

Because the early Intel northbridges didn't, so noone else bothered either, since everyone designed their hardware to not require that capability.

-hpa

#6 lore raw
In reply to Dan Gora
Quoted 10 lines · On Fri, 15 Feb 2008 10:00:28 -0800 "Dan Gora" <[email protected]> wrote:
Quoted 6 lines · Alan Cox
Quoted 2 lines

Is there any way to get PIO to burst over the PCI bus in the read and write direction? My device has 4 BAR registers, but the area where I

I think you are doign about as well as the X folks did when they spent time on trying to optimise pio transfers to and from graphics card RAM.

That's good to know. Do you have a link or anything to their discussion or some key words that I could hunt it down?

It was some time ago but a look at the X tree will find you the code. It's basically the same as you did - using MMX.