Memory Barrier Definitions
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Quoted 6 lines
An example of where these primitives get us into trouble is the use of
wmb()to order two stores which are only to system memory (where a lwsync would do for ppc64) and for a store to system memory followed by a store to I/O (many examples in drivers).
2 questions:
1) Does that only affect memory barriers, or both memory barriers and spinlocks?
example (from drivers/net/natsemi.c)
cpu0:
spin_lock(&lock); writew(1, ioaddr+PGSEL); ... writew(0, ioaddr+PGSEL); spin_unlock(&lock);
cpu1:
spin_lock(&lock); readw(ioaddr+whatever); // assumes that the register window is 0.
writew(1, ioaddr+PGSEL) selects a register window of the NIC. Are writew and the spinlock synchonized on ppc64?
2) when you write "system memory", is that memory allocated with kmalloc/gfp, or also memory allocated with pci_alloc_consistent()?
I've always assumed that
pci_alloc_consistent_ptr->data=0; writew(0, ioaddr+TRIGGER);
is ordered, i.e. the memory write happens before the writew. Is that guaranteed?
--
Manfred
Quoted 25 lines · Manfred Spraul
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bitQuoted 6 lines
An example of where these primitives get us into trouble is the use of
wmb()to order two stores which are only to system memory (where a lwsync would do for ppc64) and for a store to system memory followed by a store to I/O (many examples in drivers).2 questions:
1) Does that only affect memory barriers, or both memory barriers and spinlocks?
example (from
drivers/net/natsemi.c)cpu0:
spin_lock(&lock); writew(1, ioaddr+PGSEL); ... writew(0, ioaddr+PGSEL); spin_unlock(&lock);
Quoted 6 lines
cpu1:
spin_lock(&lock);
readw(ioaddr+whatever); // assumes that the register window is 0.
writew(1, ioaddr+PGSEL)selects a register window of the NIC. Are writew and the spinlock synchonized on ppc64?
This is an interesting example. As the implementation stands today, for this specific example, we are ok because the spin_lock/unlock pair provides ordering within system memory access pairs OR i/o space pairs. Not across the types (we do not use the heavy weight sync). So if there are examples where the spin lock is meant to protect system memory access to i/o space, we are in trouble.
Quoted 10 lines
2) when you write "system memory", is that memory allocated with kmalloc/gfp, or also memory allocated with
pci_alloc_consistent()?I've always assumed that
pci_alloc_consistent_ptr->data=0; writew(0, ioaddr+TRIGGER);is ordered, i.e. the memory write happens before the writew. Is that guaranteed?
It is not guaranteed on all systems (PowerPC being an example). pci_alloc_consistent allocted storage is just normal system memory that happens to be mapped to a PCI bus for DMA access.
Your example would fail, and in fact is basically what has been observed to fail on Power4.
What is needed is:
pci_alloc_consistent_ptr->data = 0;wmb();writew(0, ioaddr+TRIGGER);
This code also was observed to fail, when wmb() = eieio, which does not order system memory accesses to I/O space accesses.
At present, we have worked around this by doing a heavy 'sync' before and after writew and its ilk. The point of my initial questions though is that this fix is not exactly optimal :(
Dave.