for UHCI driver (from kernel 2.6.6).
Quoted 11 lines · Sven Luther
Quoted 9 lines · Alan Cox
Quoted 2 lines · Nicolas DET
Also, we replace spin_lock_irqxxx/spin_unlock_irq by spin_lock/unlock (for SMP/PREEMPT kernel) + stop/start_interrupt.
This requires a lot of care to do right. Remember that on PC systems interrupts can be substantially posted. A "
stop_interrupt" may prevent IRQ issue but if an IRQ is already on the PC APIC bus it will kill you later on because the IRQ delivery and PCI bus access on the PC class machines are totally asynchronous
Quoted 4 lines
And what about the other part of the patch ? The one about the alignement of the buffer descriptors ? Does it seem sound, should we ward it with ppc specific stuff ? Is it fit to be included in the main kernel ?
Mmm, it seems that inversing the remove_list and the list also makes the problem go away, or at least be less important, in :
struct uhci_td {
/* Hardware fields */
__u32 link;
__u32 status;
__u32 token;
__u32 buffer;
/* Software fields */
dma_addr_t dma_handle;
struct usb_device *dev;
struct urb *urb;
struct list_head list; /* P: urb->lock */
struct list_head remove_list; /* P: uhci->td_remove_list_lock */
int frame; /* for iso: what frame? */
struct list_head fl_list; /* P: uhci->frame_list_lock */
} __attribute__((aligned(16)));Could it be possible that the warning in :
/*
* The documentation says "4 words for hardware, 4 words for software". *
* That's silly, the hardware doesn't care. The hardware only cares that
* the hardware words are 16-byte aligned, and we can have any amount of
* sw space after the TD entry as far as I can tell.
be still important, at least on G4 powerpc ? remove_list is the 5th software word here. Strange, because supposedly it works on x86 and other uhci using arches, and it works on the G3, but not on the G4.
Friendly,
Sven Luther
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
Quoted 4 lines
be still important, at least on G4 powerpc ?
remove_listis the 5th software word here. Strange, because supposedly it works on x86 and other uhci using arches, and it works on the G3, but not on the G4.
I think it's more likely we are dealing with an ordering issue of accesses to memory vs. mmio, those aren't order unless you use memory barriers. Actually, it's worse, you need a full mb() to order them, which is why lately, we made ppc64 writeX() do full sync's ... that sucks but it's near to impossible to get an abstract IO API that would cover our needs here and still make other archs happy it seems...
Ben.
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
Quoted 6 lines · Benjamin Herrenschmidt
I think it's more likely we are dealing with an ordering issue of accesses to memory vs. mmio, those aren't order unless you use memory barriers. Actually, it's worse, you need a full
mb()to order them, which is why lately, we made ppc64writeX()do full sync's ... that sucks but it's near to impossible to get an abstract IO API that would cover our needs here and still make other archs happy it seems...
I recently changed a few mb() calls to wmb(), because they only protected data the CPU was writing to be read by the device. Do you think changing all the wmb()'s back to mb()'s would make a difference?
(Actually it seems likely that this is _not_ directly related to the original problem, but it might be important anyway.)
Alan Stern
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
Quoted 9 lines · Alan Stern
Quoted 6 lines · Benjamin Herrenschmidt
I think it's more likely we are dealing with an ordering issue of accesses to memory vs. mmio, those aren't order unless you use memory barriers. Actually, it's worse, you need a full
mb()to order them, which is why lately, we made ppc64writeX()do full sync's ... that sucks but it's near to impossible to get an abstract IO API that would cover our needs here and still make other archs happy it seems...
Quoted 6 lines
I recently changed a few
mb()calls towmb(), because they only protected data the CPU was writing to be read by the device. Do you think changing all the wmb()'s back to mb()'s would make a difference?(Actually it seems likely that this is _not_ directly related to the original problem, but it might be important anyway.)
Well, the problem on ppc is that the eieio done by wmb() (or implicitely done by all writeX IO accessors) will only order stores in the same domain. That is cacheable aren't ordered vs. non cacheables.
For example, write to a descriptor in memory, then writel() to your device, that isn't guaranteed to happen in order.
In this case, you indeed need an mb(), but that's a ppc thing and the race on ppc32 CPUs is quite small (though ppc64, typically POWER4 and POWER5, will eat you for lunch with their multiple deep store queues).
Ben
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
Am Freitag, 18. Juni 2004 23:41 schrieb Alan Stern:
Quoted 8 lines
Quoted 4 lines
barriers. Actually, it's worse, you need a full
mb()to order them, which is why lately, we made ppc64writeX()do full sync's ... that sucks but it's near to impossible to get an abstract IO API that would cover our needs here and still make other archs happy it seems...I recently changed a few
mb()calls towmb(), because they only protected data the CPU was writing to be read by the device. Do you think changing all the wmb()'s back to mb()'s would make a difference?
Are these issues documented somewhere?
Regards
Oliver
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
Quoted 10 lines · Benjamin Herrenschmidt
Quoted 6 lines
I recently changed a few
mb()calls towmb(), because they only protected data the CPU was writing to be read by the device. Do you think changing all the wmb()'s back to mb()'s would make a difference?(Actually it seems likely that this is _not_ directly related to the original problem, but it might be important anyway.)
Well, the problem on ppc is that the eieio done by
wmb()(or implicitely done by all writeX IO accessors) will only order stores in the same domain. That is cacheable aren't ordered vs. non cacheables.
I'm not familiar with the term "eieio"; can you explain it?
Quoted 2 lines
For example, write to a descriptor in memory, then
writel()to your device, that isn't guaranteed to happen in order.
Then it shouldn't matter for what I'm doing, which involves multiple writes to the same region of "consistent" memory. Or maybe it's only "coherent" memory -- the important thing is that the first write must complete and be visible to the device's DMA before the second. wmb() will suffice for that, right?
Quoted 3 lines
In this case, you indeed need an
mb(), but that's a ppc thing and the race on ppc32 CPUs is quite small (though ppc64, typically POWER4 and POWER5, will eat you for lunch with their multiple deep store queues).
Thanks,
Alan Stern
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
Quoted 6 lines · Alan Stern
Quoted 4 lines · Benjamin Herrenschmidt
Quoted 4 lines
I recently changed a few
mb()calls towmb(), because they only protected data the CPU was writing to be read by the device. Do you think changing all the wmb()'s back to mb()'s would make a difference?
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
(Actually it seems likely that this is _not_ directly related to the original problem, but it might be important anyway.)
Well, the problem on ppc is that the eieio done by
wmb()(or implicitely done by all writeX IO accessors) will only order stores in the same domain. That is cacheable aren't ordered vs. non cacheables.I'm not familiar with the term "eieio"; can you explain it?
Enforced In-Order Execution of I/O. Page 8-61 of the Programing Environments for 32-Bit Microprocessors, by motorola, well, at least on my version.
Friendly,
Sven Luther
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
Quoted 4 lines
Enforced In-Order Execution of I/O. Page 8-61 of the Programing Environments for 32-Bit Microprocessors, by motorola, well, at least on my version.
Don't use that old stuff; use either the new PEM or the Book I/II/III:
PEM (64- and 32-bit) v2.0:
http://www-3.ibm.com/chips/techlib/techlib.nsf/techdocs/
F6153E213FDD912E87256D49006C6541/$file/pem._64bit.d20030611.pdf
PowerPC Book I/II/III v2.01:
www-106.ibm.com/developerworks/eserver/pdfs/archpub1.pdf www-106.ibm.com/developerworks/eserver/pdfs/archpub2.pdf www-106.ibm.com/developerworks/eserver/pdfs/archpub3.pdf
Most of the time the Books are better; sometimes the PEM helps as well.
Have fun,
Segher
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
Quoted 5 lines · Segher Boessenkool
Quoted 3 lines
Enforced In-Order Execution of I/O. Page 8-61 of the Programing Environments for 32-Bit Microprocessors, by motorola, well, at least on my version.
Don't use that old stuff; use either the new PEM or the Book I/II/III:
Well, the nice thing about the old stuff like you said, is that i got it in nice paper book format, and not a bunch of .pdfs, which Motorola used to ship for free back in the days. But sure, i will look at those new documents.
Quoted 11 lines
PEM (64- and 32-bit) v2.0:
http://www-3.ibm.com/chips/techlib/techlib.nsf/techdocs/
F6153E213FDD912E87256D49006C6541/$file/pem._64bit.d20030611.pdfPowerPC Book I/II/III v2.01:
www-106.ibm.com/developerworks/eserver/pdfs/archpub1.pdf www-106.ibm.com/developerworks/eserver/pdfs/archpub2.pdf www-106.ibm.com/developerworks/eserver/pdfs/archpub3.pdfMost of the time the Books are better; sometimes the PEM helps as well.
Friendly,
Sven Luther
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/