dma doc updates
PATCH 1/1 dma doc updates
This updates docs to clarify some points:
- dma_*map_sg() usage, mostly cloning text from the pci-specific writeup
- mention cpu (and pci bridge) write buffers as cases where having memory mapped as dma-coherent isn't sufficient for correctness;
- mention cacheline coherence issues for systems with dma-unaware caches.
Please merge ...
This updates the DMA API documentation to address a few issues:
- The dma_map_sg() call results are used like pci_map_sg() results:
using sg_dma_address() and sg_dma_len(). That's not wholly obvious to folk reading _only_ the "new" DMA-API.txt writeup.
- Buffers allocated by dma_alloc_coherent() may not be completely free of coherency concerns ... some CPUs also have write buffers that may need to be flushed.
- Cacheline coherence issues are now mentioned as being among issues which affect dma buffers, and complicate/prevent using of static and (especially) stack based buffers with the DMA calls.
I don't think many drivers currently need to worry about flushing write buffers, but I did hit it with one SOC using external SDRAM for DMA
descriptors: without explicit writebuffer flushing, the on-chip DMA controller accessed descriptors before the CPU completed the writes.
Documentation/DMA-API.txt +36 -13
@@ -33,7 +33,9 @@ pci_alloc_consistent(struct pci_dev *dev Consistent memory is memory for which a write by either the device or the processor can immediately be read by the processor or device -without having to worry about caching effects. +without having to worry about caching effects. (You may however need +to make sure to flush the processor's write buffers before telling +devices to read that memory.) This routine allocates a region of <size> bytes of consistent memory. it also returns a <dma_handle> which may be cast to an unsigned
@@ -304,12 +306,12 @@ dma address with dma_mapping_error(). A could not be created and the driver should take appropriate action (eg reduce current DMA mapping usage or delay and try again later). -int -dma_map_sg(struct device *dev, struct scatterlist *sg, int nents, - enum dma_data_direction direction) -int -pci_map_sg(struct pci_dev *hwdev, struct scatterlist *sg, - int nents, int direction) + int + dma_map_sg(struct device *dev, struct scatterlist *sg, + int nents, enum dma_data_direction direction) + int + pci_map_sg(struct pci_dev *hwdev, struct scatterlist *sg, + int nents, int direction) Maps a scatter gather list from the block layer.
@@ -327,12 +329,33 @@ critical that the driver do something, i aborting the request or even oopsing is better than doing nothing and corrupting the filesystem. -void -dma_unmap_sg(struct device *dev, struct scatterlist *sg, int nhwentries, - enum dma_data_direction direction) -void -pci_unmap_sg(struct pci_dev *hwdev, struct scatterlist *sg, - int nents, int direction) +With scatterlists, you use the resulting mapping like this: + + int i, count = dma_map_sg(dev, sglist, nents, direction); + struct scatterlist *sg; + + for (i = 0, sg = sglist; i < count; i++, sg++) { + hw_address[i] = sg_dma_address(sg); + hw_len[i] = sg_dma_len(sg); + } + +where nents is the number of entries in the sglist. + +The implementation is free to merge several consecutive sglist entries +into one (e.g. with an IOMMU, or if several pages just happen to be +physically contiguous) and returns the actual number of sg entries it +mapped them to. On failure 0, is returned. + +Then you should loop count times (note: this can be less than nents times) +and use sg_dma_address() and sg_dma_len() macros where you previously +accessed sg->address and sg->length as shown above. + + void + dma_unmap_sg(struct device *dev, struct scatterlist *sg, + int nhwentries, enum dma_data_direction direction) + void + pci_unmap_sg(struct pci_dev *hwdev, struct scatterlist *sg, + int nents, int direction) unmap the previously mapped scatter/gather list. All the parameters must be the same as those and passed in to the scatter/gather mapping
Documentation/DMA-mapping.txt +17 -5
@@ -58,11 +58,15 @@ translating each of those pages back to something like __va(). [ EDIT: Update this when we integrate Gerd Knorr's generic code which does this. ] -This rule also means that you may not use kernel image addresses -(ie. items in the kernel's data/text/bss segment, or your driver's) -nor may you use kernel stack addresses for DMA. Both of these items -might be mapped somewhere entirely different than the rest of physical -memory. +This rule also means that you may use neither kernel image addresses +(items in data/text/bss segments), nor module image addresses, nor +stack addresses for DMA. These could all be mapped somewhere entirely +different than the rest of physical memory. Even if those classes of +memory could physically work with DMA, you'd need to ensure the I/O +buffers were cacheline-aligned. Without that, you'd see cacheline +sharing problems (data corruption) on CPUs with DMA-incoherent caches. +(The CPU could write to one word, DMA would write to a different one +in the same cache line, and one of them could be overwritten.) Also, this means that you cannot take the return of a kmap() call and DMA to/from that. This is similar to vmalloc().
@@ -284,6 +288,11 @@ There are two types of DMA mappings: in order to get correct behavior on all platforms. + Also, on some platforms your driver may need to flush CPU write + buffers in much the same way as it needs to flush write buffers + found in PCI bridges (such as by reading a register's value + after writing it). + - Streaming DMA mappings which are usually mapped for one DMA transfer, unmapped right after it (unless you use pci_dma_sync_* below) and for which hardware can optimize for sequential accesses.
@@ -303,6 +312,9 @@ There are two types of DMA mappings: Neither type of DMA mapping has alignment restrictions that come from PCI, although some devices may have such restrictions. +Also, systems with caches that aren't DMA-coherent will work better +when the underlying buffers don't share cache lines with other data. + Using Consistent DMA mappings.
Documentation/DMA-API.txt dma address with dma_mapping_error(). A · line 311
Quoted 3 lines · David Brownell
Documentation/DMA-API.txt +3
-0+ int + dma_map_sg(struct device *dev, struct scatterlist *sg, + int nents, enum dma_data_direction direction)
While you're at it, care to s/enum dma_data_direction/int/? some archs use one and some use the other, and there was weak consensus that int is better.
Cheers,
Muli
Signature · 2 lines
Muli Ben-Yehuda http://www.mulix.org | http://mulix.livejournal.com/
Quoted 9 lines · Muli Ben-Yehuda
Documentation/DMA-API.txt dma address with dma_mapping_error(). A · line 311
Quoted 3 lines · David Brownell
Documentation/DMA-API.txt +3
-0+ int + dma_map_sg(struct device *dev, struct scatterlist *sg, + int nents, enum dma_data_direction direction)While you're at it, care to s/enum dma_data_direction/int/? some archs use one and some use the other, and there was weak consensus that int is better.
Nah. That patch was just to _clarify_ things not make API changes. That particular text was just being indented for readability.
Plus, I won't join a consensus to needlessly prevent compilers from reporting illegal parameters.
- Dave
[email protected]
Appliedpatch dma-doc-updates.patch added to gregkh-2.6 tree
In reply to David BrownellThis is a note to let you know that I've just added the patch titled
Subject: dma doc updates
to my gregkh-2.6 tree. Its filename is
dma-doc-updates.patch
This tree can be found at
http://www.kernel.org/pub/linux/kernel/people/gregkh/gregkh-2.6/patches/
Quoted 1 line
From [email protected] Sat Apr 1 10:43:27 2006
From: David Brownell <[email protected]>
To: Linux Kernel list <[email protected]>
Subject: dma doc updates
Date: Sat, 1 Apr 2006 10:21:52 -0800
Cc: Greg KH <[email protected]>
Message-Id: <[email protected]>
This updates the DMA API documentation to address a few issues:
- The dma_map_sg() call results are used like pci_map_sg() results:
using sg_dma_address() and sg_dma_len(). That's not wholly obvious to folk reading _only_ the "new" DMA-API.txt writeup.
- Buffers allocated by dma_alloc_coherent() may not be completely free of coherency concerns ... some CPUs also have write buffers that may need to be flushed.
- Cacheline coherence issues are now mentioned as being among issues which affect dma buffers, and complicate/prevent using of static and (especially) stack based buffers with the DMA calls.
I don't think many drivers currently need to worry about flushing write buffers, but I did hit it with one SOC using external SDRAM for DMA
descriptors: without explicit writebuffer flushing, the on-chip DMA controller accessed descriptors before the CPU completed the writes.
--- Documentation/DMA-API.txt | 49 ++++++++++++++++++++++++++++++------------ Documentation/DMA-mapping.txt | 22 ++++++++++++++---- 2 files changed, 53 insertions(+), 18 deletions(-)
Documentation/DMA-API.txt +36 -13
@@ -33,7 +33,9 @@ pci_alloc_consistent(struct pci_dev *dev Consistent memory is memory for which a write by either the device or the processor can immediately be read by the processor or device -without having to worry about caching effects. +without having to worry about caching effects. (You may however need +to make sure to flush the processor's write buffers before telling +devices to read that memory.) This routine allocates a region of <size> bytes of consistent memory. it also returns a <dma_handle> which may be cast to an unsigned
@@ -304,12 +306,12 @@ dma address with dma_mapping_error(). A could not be created and the driver should take appropriate action (eg reduce current DMA mapping usage or delay and try again later). -int -dma_map_sg(struct device *dev, struct scatterlist *sg, int nents, - enum dma_data_direction direction) -int -pci_map_sg(struct pci_dev *hwdev, struct scatterlist *sg, - int nents, int direction) + int + dma_map_sg(struct device *dev, struct scatterlist *sg, + int nents, enum dma_data_direction direction) + int + pci_map_sg(struct pci_dev *hwdev, struct scatterlist *sg, + int nents, int direction) Maps a scatter gather list from the block layer.
@@ -327,12 +329,33 @@ critical that the driver do something, i aborting the request or even oopsing is better than doing nothing and corrupting the filesystem. -void -dma_unmap_sg(struct device *dev, struct scatterlist *sg, int nhwentries, - enum dma_data_direction direction) -void -pci_unmap_sg(struct pci_dev *hwdev, struct scatterlist *sg, - int nents, int direction) +With scatterlists, you use the resulting mapping like this: + + int i, count = dma_map_sg(dev, sglist, nents, direction); + struct scatterlist *sg; + + for (i = 0, sg = sglist; i < count; i++, sg++) { + hw_address[i] = sg_dma_address(sg); + hw_len[i] = sg_dma_len(sg); + } + +where nents is the number of entries in the sglist. + +The implementation is free to merge several consecutive sglist entries +into one (e.g. with an IOMMU, or if several pages just happen to be +physically contiguous) and returns the actual number of sg entries it +mapped them to. On failure 0, is returned. + +Then you should loop count times (note: this can be less than nents times) +and use sg_dma_address() and sg_dma_len() macros where you previously +accessed sg->address and sg->length as shown above. + + void + dma_unmap_sg(struct device *dev, struct scatterlist *sg, + int nhwentries, enum dma_data_direction direction) + void + pci_unmap_sg(struct pci_dev *hwdev, struct scatterlist *sg, + int nents, int direction) unmap the previously mapped scatter/gather list. All the parameters must be the same as those and passed in to the scatter/gather mapping
Documentation/DMA-mapping.txt +17 -5
@@ -58,11 +58,15 @@ translating each of those pages back to something like __va(). [ EDIT: Update this when we integrate Gerd Knorr's generic code which does this. ] -This rule also means that you may not use kernel image addresses -(ie. items in the kernel's data/text/bss segment, or your driver's) -nor may you use kernel stack addresses for DMA. Both of these items -might be mapped somewhere entirely different than the rest of physical -memory. +This rule also means that you may use neither kernel image addresses +(items in data/text/bss segments), nor module image addresses, nor +stack addresses for DMA. These could all be mapped somewhere entirely +different than the rest of physical memory. Even if those classes of +memory could physically work with DMA, you'd need to ensure the I/O +buffers were cacheline-aligned. Without that, you'd see cacheline +sharing problems (data corruption) on CPUs with DMA-incoherent caches. +(The CPU could write to one word, DMA would write to a different one +in the same cache line, and one of them could be overwritten.) Also, this means that you cannot take the return of a kmap() call and DMA to/from that. This is similar to vmalloc().
@@ -284,6 +288,11 @@ There are two types of DMA mappings: in order to get correct behavior on all platforms. + Also, on some platforms your driver may need to flush CPU write + buffers in much the same way as it needs to flush write buffers + found in PCI bridges (such as by reading a register's value + after writing it). + - Streaming DMA mappings which are usually mapped for one DMA transfer, unmapped right after it (unless you use pci_dma_sync_* below) and for which hardware can optimize for sequential accesses.
@@ -303,6 +312,9 @@ There are two types of DMA mappings: Neither type of DMA mapping has alignment restrictions that come from PCI, although some devices may have such restrictions. +Also, systems with caches that aren't DMA-coherent will work better +when the underlying buffers don't share cache lines with other data. + Using Consistent DMA mappings.
Patches currently in gregkh-2.6 which might be from [email protected] are
driver/spi-add-pxa2xx-ssp-spi-driver.patch driver/spi-per-transfer-overrides-for-wordsize-and-clocking.patch pci/dma-doc-updates.patch