pci: let devices flush DMA to host memory

On Altix, DMA may be reordered within the NUMA interconnect. This can be a problem with Infiniband, where DMA to Completion Queues can race with data DMA. This patchset allows a driver to associate a memory region with a "dmaflush" attribute, so that writes to the memory region flush in-flight DMA, preventing the CQ/data race.

There are three patches in this set:

	[1/3]: add pci_dma_flags_set_dmaflush() to pci interface
	[2/3]: redefine pci_dma_flags_set_dmaflush() for sn-ia64
	[3/3]: document pci_dma_flags_set_dmaflush()

And there would be additional patches to IB drivers to make use of the interface, of course.

Arthur

FYI to linux-pci folks
This patch had previous discussion on LKML:

    http://lkml.org/lkml/2007/8/17/336

James Bottomley at one point obliquely referred to my OLS2003 paper: "DMA Hints on ia64/PARISC"

After reading the thread, my take is we need a more elegant way for a device driver to handle registration of DMA regions allocated by user space. The API would "make this page/region act like dma_alloc_coherent()". That implies strong ordering between CPU and DMA to/from the device. Maybe the code is the right thing and I want a name that makes sense in the context of current DMA API.

On IRC, willy suggested an mmap() flag and that sounds reasonable too though I don't know if it's feasible.

hth,
grant

Need to think about this...

Yeah, we're doing something like this now as a band-aid solution. Not a flag to mmap(), but a magic offset value. But it wasn't acceptable to the maintainer of the mthca IB driver (Roland Dreier), hence the new proposal....

Arthur