Inefficient PCI DMA usage (was: [experimental patch] UHCI updates)
Quoted 9 lines
TD's are around 32 bytes big (actually, they may be 48 or even 64 now, I haven't checked recently). That's a waste of space for an entire page.
However, having every driver implement it's own slab cache seems a complete waste of time when we already have the code to do so in
mm/slab.c. It would be nice if we could extend the generic slab code to understand the PCI DMA API for us.
I missed the beginning of the thread:
What are the exact requirements for TD's?
I have 3 tiny updates for mm/slab.c that I'll send to Linus as soon as 2.4 has stabilized a bit more, perhaps I can integrate the code for USB.
--
Manfred
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
Quoted 9 lines · Manfred Spraul
Quoted 9 lines
TD's are around 32 bytes big (actually, they may be 48 or even 64 now, I haven't checked recently). That's a waste of space for an entire page.
However, having every driver implement it's own slab cache seems a complete waste of time when we already have the code to do so in
mm/slab.c. It would be nice if we could extend the generic slab code to understand the PCI DMA API for us.
Quoted 5 lines
I missed the beginning of the thread:
What are the exact requirements for TD's?
I have 3 tiny updates formm/slab.cthat I'll send to Linus as soon as 2.4 has stabilized a bit more, perhaps I can integrate the code for USB.
They need to be visible via DMA. They need to be 16 byte aligned. We also have QH's which have similar requirements, but we don't use as many of them.
JE
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
Quoted 3 lines · Johannes Erdfelt
They need to be visible via DMA. They need to be 16 byte aligned. We also have QH's which have similar requirements, but we don't use as many of them.
Can we get away from the "16 byte aligned" and make it "n byte aligned"? I believe that slab already has support for this?
_____
|_____| ------------------------------------------------- ---+---+- | | Russell King [email protected] --- --- | | | | http://www.arm.linux.org.uk/personal/aboutme.html / / | | +-+-+ --- -+- / | THE developer of ARM Linux |+| /|\ / | | | --- | +-+-+ ------------------------------------------------- /\\\ |
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
Quoted 9 lines · Russell King
Quoted 3 lines · Johannes Erdfelt
They need to be visible via DMA. They need to be 16 byte aligned. We also have QH's which have similar requirements, but we don't use as many of them.
Can we get away from the "16 byte aligned" and make it "n byte aligned"? I believe that slab already has support for this?
Not yet, but that would be a 2 line patch (currently it's hardcoded to BYTES_PER_WORD align or L1_CACHE_BYTES, depending on the HWCACHE_ALIGN flag).
But there are 2 other problems:
* kmem_cache_alloc returns one pointer, pci_alloc_consistent 2 pointers:
one dma address, one virtual address.
* The code relies on the virt_to_page() macro.
The second problem is the difficult one, I don't see how I could remove that dependency without a major overhaul.
--
Manfred
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
Quoted 3 lines · Manfred Spraul
Not yet, but that would be a 2 line patch (currently it's hardcoded to
BYTES_PER_WORDalign orL1_CACHE_BYTES, depending on theHWCACHE_ALIGNflag).
I don't think there's a problem then. However, if slab can be told "I want 1024 bytes aligned to 1024 bytes" then I can get rid of arch/arm/mm/small_page.c (separate problem to the one we're discussing though) ;)
Quoted 4 lines
But there are 2 other problems:
*kmem_cache_allocreturns one pointer,pci_alloc_consistent2 pointers:
one dma address, one virtual address.
* The code relies on thevirt_to_page()macro.
What I'm wondering is what about a wrapper around the slab allocator, in a similar way to pci_alloc_consistent() is a wrapper around gfp. Since the slab allocator returns "pointers" in the same space as gfp returns page references, there shouldn't be a problem (Linus may complain here).
ie, we could make pci_alloc_consistent() a little more intelligent and allocate from the slab for small sizes, but use gfp for larger sizes?
Comments, anyone (DaveM, Linus, et al) ?
_____
|_____| ------------------------------------------------- ---+---+- | | Russell King [email protected] --- --- | | | | http://www.arm.linux.org.uk/personal/aboutme.html / / | | +-+-+ --- -+- / | THE developer of ARM Linux |+| /|\ / | | | --- | +-+-+ ------------------------------------------------- /\\\ |
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
Quoted 11 lines · Russell King
Quoted 3 lines · Manfred Spraul
Not yet, but that would be a 2 line patch (currently it's hardcoded to
BYTES_PER_WORDalign orL1_CACHE_BYTES, depending on theHWCACHE_ALIGNflag).I don't think there's a problem then. However, if slab can be told "I want 1024 bytes aligned to 1024 bytes" then I can get rid of
arch/arm/mm/small_page.c(separate problem to the one we're discussing though) ;)
That's easy, I'll include it in my next slab update.
--
Manfred
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
Quoted 7 lines · Russell King
Quoted 3 lines · Johannes Erdfelt
They need to be visible via DMA. They need to be 16 byte aligned. We also have QH's which have similar requirements, but we don't use as many of them.
Can we get away from the "16 byte aligned" and make it "n byte aligned"? I believe that slab already has support for this?
If you look at the part of the message that I quoted and you cut off, the requirements for UHCI are the data structures MUST be 16 byte aligned.
I don't mind if the API is more generalized, but those are the requirements that were asked about in this specific case.
JE
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
Hi,
You may have already found out that there's a problem using pci_alloc_consistent and friends in the USB layer which will only be obvious on CPUs where they need to do page table remapping
- that is that pci_alloc_consistent/pci_free_consistent aren't guaranteed to be interrupt-safe.
I'm not sure what the correct way around this is yet, but I do know its a major problem. ;(
Maybe we need to do a get_free_pages-type thing with this and keep a set amount of consistent area in reserve for atomic allocations (as per GFP_ATOMIC)? Yes, I know its not nice, but I don't see any other option at the moment with USB.
(yes, I'm hacking the 2.2.18 ohci driver for my own ends to get something up and running on one of my machines).
_____
|_____| ------------------------------------------------- ---+---+- | | Russell King [email protected] --- --- | | | | http://www.arm.linux.org.uk/personal/aboutme.html / / | | +-+-+ --- -+- / | THE developer of ARM Linux |+| /|\ / | | | --- | +-+-+ ------------------------------------------------- /\\\ |
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/