a saner API for allocating DMA addressable pages v2
Subsystems (34)
Hi all,
this series replaced the DMA_ATTR_NON_CONSISTENT flag to dma_alloc_attrs with a separate new dma_alloc_pages API, which is available on all platforms. In addition to cleaning up the convoluted code path, this ensures that other drivers that have asked for better support for non-coherent DMA to pages with incurring bounce buffering over can finally be properly supported.
I'm still a little unsure about the API naming, as alloc_pages sort of implies a struct page return value, but we return a kernel virtual address. The other alternative would be to name the API dma_alloc_noncoherent, but the whole non-coherent naming seems to put people off. As a follow up I plan to move the implementation of the DMA_ATTR_NO_KERNEL_MAPPING flag over to this framework as well, given that is also is a fundamentally non coherent allocation. The replacement for that flag would then return a struct page, as it is allowed to actually return pages without a kernel mapping as the name suggested (although most of the time they will actually have a kernel mapping..)
In addition to the conversions of the existing non-coherent DMA users, I've also added a patch to convert the firewire ohci driver to use the new dma_alloc_pages API.
Note that I haven't carried over any Tested-by: tags for the noncoherent allocation conversions as there was a bit of a patch reshuffle, but the result should be the same.
The first patch is queued up for 5.9 in the media tree, but included here for completeness.
A git tree is available here:
git://git.infradead.org/users/hch/misc.git dma_alloc_pages
Gitweb:
http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/dma_alloc_pages
Changes since v1:
- rebased on the latests dma-mapping tree, which merged many of the cleanups
- fix an argument passing typo in 53c700, caught by sparse
- rename a few macro arguments in 53c700
- pass the right device to the DMA API in the lib82596 drivers
- fix memory ownershiptransfers in sgiseeq
- better document what a page in the direct kernel mapping means
- split into dma_alloc_pages that returns a struct page and is in the direct mapping vs dma_alloc_noncoherent that can be vmapped
- conver the firewire ohci driver to dma_alloc_pages
Diffstat:
PATCH 1/17 media/v4l2: remove V4L2-FLAG-MEMORY-NON-CONSISTENT flag
From: Sergey Senozhatsky <[email protected]>
The patch partially reverts some of the UAPI bits of the buffer cache management hints. Namely, the queue consistency (memory coherency) user-space hint because, as it turned out, the kernel implementation of this feature was misusing DMA_ATTR_NON_CONSISTENT.
The patch revers both kernel and user space parts: removes the DMA consistency attr functions, rollbacks changes to v4l2_requestbuffers, v4l2_create_buffers structures and corresponding UAPI functions (plus compat32 layer) and cleanups the documentation.
--- .../userspace-api/media/v4l/buffer.rst | 17 ------- .../media/v4l/vidioc-create-bufs.rst | 6 +-- .../media/v4l/vidioc-reqbufs.rst | 12 +---- .../media/common/videobuf2/videobuf2-core.c | 46 +++---------------- .../common/videobuf2/videobuf2-dma-contig.c | 19 -------- .../media/common/videobuf2/videobuf2-dma-sg.c | 3 +- .../media/common/videobuf2/videobuf2-v4l2.c | 18 +------- drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 10 +--- drivers/media/v4l2-core/v4l2-ioctl.c | 5 +- include/media/videobuf2-core.h | 7 +-- include/uapi/linux/videodev2.h | 13 +----- 11 files changed, 22 insertions(+), 134 deletions(-)
Documentation/userspace-api/media/v4l/buffer.rst +0 -17
@@ -701,23 +701,6 @@ Memory Consistency Flags :stub-columns: 0 :widths: 3 1 4 - * .. _`V4L2-FLAG-MEMORY-NON-CONSISTENT`: - - - ``V4L2_FLAG_MEMORY_NON_CONSISTENT`` - - 0x00000001 - - A buffer is allocated either in consistent (it will be automatically - coherent between the CPU and the bus) or non-consistent memory. The - latter can provide performance gains, for instance the CPU cache - sync/flush operations can be avoided if the buffer is accessed by the - corresponding device only and the CPU does not read/write to/from that - buffer. However, this requires extra care from the driver -- it must - guarantee memory consistency by issuing a cache flush/sync when - consistency is needed. If this flag is set V4L2 will attempt to - allocate the buffer in non-consistent memory. The flag takes effect - only if the buffer is used for :ref:`memory mapping <mmap>` I/O and the - queue reports the :ref:`V4L2_BUF_CAP_SUPPORTS_MMAP_CACHE_HINTS - <V4L2-BUF-CAP-SUPPORTS-MMAP-CACHE-HINTS>` capability. - .. c:type:: v4l2_memory enum v4l2_memory
Documentation/userspace-api/media/v4l/vidioc-create-bufs.rst +1 -5
@@ -120,13 +120,9 @@ than the number requested. If you want to just query the capabilities without making any other changes, then set ``count`` to 0, ``memory`` to ``V4L2_MEMORY_MMAP`` and ``format.type`` to the buffer type. - * - __u32 - - ``flags`` - - Specifies additional buffer management attributes. - See :ref:`memory-flags`. * - __u32 - - ``reserved``\ [6] + - ``reserved``\ [7] - A place holder for future extensions. Drivers and applications must set the array to zero.
Documentation/userspace-api/media/v4l/vidioc-reqbufs.rst +2 -10
@@ -112,17 +112,10 @@ aborting or finishing any DMA in progress, an implicit ``V4L2_MEMORY_MMAP`` and ``type`` set to the buffer type. This will free any previously allocated buffers, so this is typically something that will be done at the start of the application. - * - union { - - (anonymous) - * - __u32 - - ``flags`` - - Specifies additional buffer management attributes. - See :ref:`memory-flags`. * - __u32 - ``reserved``\ [1] - - Kept for backwards compatibility. Use ``flags`` instead. - * - } - - + - A place holder for future extensions. Drivers and applications + must set the array to zero. .. tabularcolumns:: |p{6.1cm}|p{2.2cm}|p{8.7cm}|
@@ -169,7 +162,6 @@ aborting or finishing any DMA in progress, an implicit - This capability is set by the driver to indicate that the queue supports cache and memory management hints. However, it's only valid when the queue is used for :ref:`memory mapping <mmap>` streaming I/O. See - :ref:`V4L2_FLAG_MEMORY_NON_CONSISTENT <V4L2-FLAG-MEMORY-NON-CONSISTENT>`, :ref:`V4L2_BUF_FLAG_NO_CACHE_INVALIDATE <V4L2-BUF-FLAG-NO-CACHE-INVALIDATE>` and :ref:`V4L2_BUF_FLAG_NO_CACHE_CLEAN <V4L2-BUF-FLAG-NO-CACHE-CLEAN>`.
drivers/media/common/videobuf2/videobuf2-core.c +6 -40
114 lines not on this page · raw message
drivers/media/common/videobuf2/videobuf2-dma-contig.c +0 -19
@@ -42,11 +42,6 @@ struct vb2_dc_buf {struct dma_buf_attachment *db_attach; }; -static inline bool vb2_dc_buffer_consistent (unsigned long attr) -{ -return !(attr &DMA_ATTR_NON_CONSISTENT ); -} -/*********************************************/ /* scatterlist table functions */ /*********************************************/
@@ -341,13 +336,6 @@ static int vb2_dc_dmabuf_ops_begin_cpu_access(struct dma_buf *dbuf, enum dma_data_direction direction) { - struct vb2_dc_buf *buf = dbuf->priv; -struct sg_table *sgt = buf->dma_sgt; - -if (vb2_dc_buffer_consistent (buf->attrs)) -return 0 ; - -dma_sync_sg_for_cpu (buf->dev, sgt->sgl, sgt->nents, buf->dma_dir);return 0 ; }
@@ -355,13 +343,6 @@ static int vb2_dc_dmabuf_ops_end_cpu_access(struct dma_buf *dbuf, enum dma_data_direction direction) { - struct vb2_dc_buf *buf = dbuf->priv; -struct sg_table *sgt = buf->dma_sgt; - -if (vb2_dc_buffer_consistent (buf->attrs)) -return 0 ; - -dma_sync_sg_for_device (buf->dev, sgt->sgl, sgt->nents, buf->dma_dir);return 0 ; }
drivers/media/common/videobuf2/videobuf2-dma-sg.c +1 -2
@@ -123,8 +123,7 @@ static void *vb2_dma_sg_alloc(struct device *dev, unsigned long dma_attrs,/* * NOTE: dma-sg allocates memory using the page allocator directly, so * there is no memory consistency guarantee, hence dma-sg ignores DMA -* attributes passed from the upper layer. That means that -* V4L2_FLAG_MEMORY_NON_CONSISTENT has no effect on dma-sg buffers. +* attributes passed from the upper layer. */ buf->pages =kvmalloc_array (buf->num_pages,sizeof (struct page *),GFP_KERNEL | __GFP_ZERO);
drivers/media/common/videobuf2/videobuf2-v4l2.c +2 -16
@@ -722,22 +722,12 @@ static void fill_buf_caps(struct vb2_queue *q, u32 *caps)#endif } -static void clear_consistency_attr (struct vb2_queue *q, -int memory, -unsigned int *flags) -{ -if (!q->allow_cache_hints || memory !=V4L2_MEMORY_MMAP ) - *flags &= ~V4L2_FLAG_MEMORY_NON_CONSISTENT ; -} -int vb2_reqbufs (struct vb2_queue *q,struct v4l2_requestbuffers *req) {int ret =vb2_verify_memory_type (q, req->memory, req->type);fill_buf_caps (q, &req->capabilities); -clear_consistency_attr (q, req->memory, &req->flags); -return ret ? ret :vb2_core_reqbufs (q, req->memory, - req->flags, &req->count); +return ret ? ret :vb2_core_reqbufs (q, req->memory, &req->count); }EXPORT_SYMBOL_GPL (vb2_reqbufs);
@@ -769,7 +759,6 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)unsigned i;fill_buf_caps (q, &create->capabilities); -clear_consistency_attr (q, create->memory, &create->flags); create->index = q->num_buffers;if (create->count ==0 )return ret != -EBUSY ? ret :0 ;
@@ -813,7 +802,6 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)if (requested_sizes[i] ==0 )return -EINVAL ;return ret ? ret :vb2_core_create_bufs (q, create->memory, - create->flags, &create->count, requested_planes, requested_sizes);
@@ -998,12 +986,11 @@ int vb2_ioctl_reqbufs(struct file *file, void *priv,int res =vb2_verify_memory_type (vdev->queue, p->memory, p->type);fill_buf_caps (vdev->queue, &p->capabilities); -clear_consistency_attr (vdev->queue, p->memory, &p->flags);if (res)return res;if (vb2_queue_is_busy (vdev, file))return -EBUSY ; - res =vb2_core_reqbufs (vdev->queue, p->memory, p->flags, &p->count); + res =vb2_core_reqbufs (vdev->queue, p->memory, &p->count);/* If count == 0, then the owner has released all buffers and he is no longer owner of the queue. Otherwise we have a new owner. */ if (res ==0 )
@@ -1021,7 +1008,6 @@ int vb2_ioctl_create_bufs(struct file *file, void *priv, p->index = vdev->queue->num_buffers;fill_buf_caps (vdev->queue, &p->capabilities); -clear_consistency_attr (vdev->queue, p->memory, &p->flags);/* * If count == 0, then just check if memory and type are valid. * Any -EBUSY result from vb2_verify_memory_type can be mapped to 0.
drivers/media/v4l2-core/v4l2-compat-ioctl32.c +2 -8
@@ -246,9 +246,6 @@ struct v4l2_format32 {* @memory: buffer memory type * @format: frame format, for which buffers are requested * @capabilities: capabilities of this buffer type. -* @flags: additional buffer management attributes (ignored unless the -* queue has V4L2_BUF_CAP_SUPPORTS_MMAP_CACHE_HINTS capability and -* configured for MMAP streaming I/O). * @reserved: future extensions */ struct v4l2_create_buffers32 {
@@ -257,8 +254,7 @@ struct v4l2_create_buffers32 {__u32 memory;/* enum v4l2_memory */ struct v4l2_format32 format;__u32 capabilities; -__u32 flags; -__u32 reserved[6 ]; +__u32 reserved[7 ]; };static int __bufsize_v4l2_format (struct v4l2_format32 __user *p32,u32 *size)
@@ -359,8 +355,7 @@ static int get_v4l2_create32(struct v4l2_create_buffers __user *p64, {if (!access_ok (p32,sizeof (*p32)) ||copy_in_user (p64, p32, - offsetof(struct v4l2_create_buffers32 , format)) || -assign_in_user (&p64->flags, &p32->flags)) + offsetof(struct v4l2_create_buffers32 , format)))return -EFAULT ;return __get_v4l2_format32 (&p64->format, &p32->format, aux_buf, aux_space);
@@ -422,7 +417,6 @@ static int put_v4l2_create32(struct v4l2_create_buffers __user *p64,copy_in_user (p32, p64, offsetof(struct v4l2_create_buffers32 , format)) ||assign_in_user (&p32->capabilities, &p64->capabilities) || -assign_in_user (&p32->flags, &p64->flags) || copy_in_user(p32->reserved, p64->reserved, sizeof(p64->reserved))) return -EFAULT;return __put_v4l2_format32 (&p64->format, &p32->format);
drivers/media/v4l2-core/v4l2-ioctl.c +4 -1
@@ -2042,6 +2042,9 @@ static int v4l_reqbufs(const struct v4l2_ioctl_ops *ops,if (ret)return ret; + +CLEAR_AFTER_FIELD (p, capabilities); +return ops->vidioc_reqbufs (file, fh, p); }
@@ -2081,7 +2084,7 @@ static int v4l_create_bufs(const struct v4l2_ioctl_ops *ops,if (ret)return ret; -CLEAR_AFTER_FIELD (create, flags); +CLEAR_AFTER_FIELD (create, capabilities);v4l_sanitize_format (&create->format);
include/media/videobuf2-core.h +2 -5
@@ -744,8 +744,6 @@ void vb2_core_querybuf(struct vb2_queue *q, unsigned int index, void *pb);* vb2_core_reqbufs() - Initiate streaming. * @q: pointer to &struct vb2_queue with videobuf2 queue. * @memory: memory type, as defined by &enum vb2_memory. -* @flags: auxiliary queue/buffer management flags. Currently, the only -* used flag is %V4L2_FLAG_MEMORY_NON_CONSISTENT. * @count: requested buffer count. * * Videobuf2 core helper to implement VIDIOC_REQBUF() operation. It is called
@@ -770,13 +768,12 @@ void vb2_core_querybuf(struct vb2_queue *q, unsigned int index, void *pb);* Return: returns zero on success; an error code otherwise. */ int vb2_core_reqbufs (struct vb2_queue *q,enum vb2_memory memory, -unsigned int flags,unsigned int *count); +unsigned int *count);/** * vb2_core_create_bufs() - Allocate buffers and any required auxiliary structs * @q: pointer to &struct vb2_queue with videobuf2 queue. * @memory: memory type, as defined by &enum vb2_memory. -* @flags: auxiliary queue/buffer management flags. * @count: requested buffer count. * @requested_planes: number of planes requested. * @requested_sizes: array with the size of the planes.
@@ -794,7 +791,7 @@ int vb2_core_reqbufs(struct vb2_queue *q, enum vb2_memory memory,* Return: returns zero on success; an error code otherwise. */ int vb2_core_create_bufs (struct vb2_queue *q,enum vb2_memory memory, -unsigned int flags,unsigned int *count, +unsigned int *count,unsigned int requested_planes,const unsigned int requested_sizes[]);
include/uapi/linux/videodev2.h +2 -11
@@ -191,8 +191,6 @@ enum v4l2_memory { V4L2_MEMORY_DMABUF = 4, }; -#define V4L2_FLAG_MEMORY_NON_CONSISTENT (1 << 0) -/* see also http://vektor.theorem.ca/graphics/ycbcr/ */ enum v4l2_colorspace {/*
@@ -949,10 +947,7 @@ struct v4l2_requestbuffers {__u32 type;/* enum v4l2_buf_type */ __u32 memory;/* enum v4l2_memory */ __u32 capabilities; -union { -__u32 flags; -__u32 reserved[1 ]; - }; +__u32 reserved[1 ]; };/* capabilities for struct v4l2_requestbuffers and v4l2_create_buffers */
@@ -2456,9 +2451,6 @@ struct v4l2_dbg_chip_info {* @memory: enum v4l2_memory; buffer memory type * @format: frame format, for which buffers are requested * @capabilities: capabilities of this buffer type. -* @flags: additional buffer management attributes (ignored unless the -* queue has V4L2_BUF_CAP_SUPPORTS_MMAP_CACHE_HINTS capability -* and configured for MMAP streaming I/O). * @reserved: future extensions */ struct v4l2_create_buffers {
@@ -2467,8 +2459,7 @@ struct v4l2_create_buffers {__u32 memory;struct v4l2_format format;__u32 capabilities; -__u32 flags; -__u32 reserved[6 ]; +__u32 reserved[7 ]; };/*
Signature · 1 line
2.28.0
PATCH 2/17 mm: turn alloc_pages into an inline function
In reply to Christoph HellwigTo prevent a compiler error when a method call alloc_pages is added (which I plan to for the dma_map_ops).
--- include/linux/gfp.h | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
include/linux/gfp.h +4 -2
@@ -550,8 +550,10 @@ extern struct page *alloc_pages_vma(gfp_t gfp_mask, int order,#define alloc_hugepage_vma (gfp_mask, vma, addr, order) \ alloc_pages_vma(gfp_mask, order, vma, addr, numa_node_id(), true)#else -#define alloc_pages (gfp_mask, order) \ - alloc_pages_node(numa_node_id(), gfp_mask, order) +static inline struct page *alloc_pages (gfp_t gfp_mask,unsigned int order) +{ +return alloc_pages_node (numa_node_id (), gfp_mask, order); +}#define alloc_pages_vma (gfp_mask, order, vma, addr, node, false)\ alloc_pages(gfp_mask, order)#define alloc_hugepage_vma (gfp_mask, vma, addr, order) \
Signature · 1 line
2.28.0
PATCH 3/17 drm/exynos: stop setting DMA_ATTR_NON_CONSISTENT
In reply to Christoph HellwigDMA_ATTR_NON_CONSISTENT is a no-op except on PARISC and some mips configs, so don't set it in this ARM specific driver.
--- drivers/gpu/drm/exynos/exynos_drm_gem.c | 2 -- 1 file changed, 2 deletions(-)
drivers/gpu/drm/exynos/exynos_drm_gem.c +0 -2
@@ -42,8 +42,6 @@ static int exynos_drm_alloc_buf(struct exynos_drm_gem *exynos_gem, bool kvmap)if (exynos_gem->flags &EXYNOS_BO_WC || !(exynos_gem->flags &EXYNOS_BO_CACHABLE )) attr |=DMA_ATTR_WRITE_COMBINE ; -else - attr |=DMA_ATTR_NON_CONSISTENT ;/* FBDev emulation requires kernel mapping */ if (!kvmap)
Signature · 1 line
2.28.0
Re: [PATCH 03/17] drm/exynos: stop setting DMA_ATTR_NON_CONSISTENT
Quoted 2 lines · Christoph Hellwig
DMA_ATTR_NON_CONSISTENTis a no-op except on PARISC and some mips configs, so don't set it in this ARM specific driver.
Hm, PARICS and ARM capitalized but mips in lower case? :-)
Quoted 1 line
Signed-off-by Christoph Hellwig <[email protected]>
[...]
MBR, Sergei
Re: [PATCH 03/17] drm/exynos: stop setting DMA_ATTR_NON_CONSISTENT
Quoted 6 lines · Sergei Shtylyov
Quoted 2 lines · Christoph Hellwig
DMA_ATTR_NON_CONSISTENTis a no-op except on PARISC and some mips configs, so don't set it in this ARM specific driver.Hm, PARICS and ARM capitalized but mips in lower case? :-)
I guess it should also be PA-RISC if we start nitpicking..
PATCH 4/17 drm/nouveau/gk20a: stop setting DMA_ATTR_NON_CONSISTENT
In reply to Christoph HellwigDMA_ATTR_NON_CONSISTENT is a no-op except on PARISC and some mips configs, so don't set it in this ARM specific driver part.
--- drivers/gpu/drm/nouveau/nvkm/subdev/instmem/gk20a.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
drivers/gpu/drm/nouveau/nvkm/subdev/instmem/gk20a.c +1 -2
@@ -594,8 +594,7 @@ gk20a_instmem_new(struct nvkm_device *device, int index,nvkm_info (&imem->base.subdev,"using IOMMU\n" ); }else { - imem->attrs =DMA_ATTR_NON_CONSISTENT | -DMA_ATTR_WEAK_ORDERING | + imem->attrs =DMA_ATTR_WEAK_ORDERING |DMA_ATTR_WRITE_COMBINE ;nvkm_info (&imem->base.subdev,"using DMA API\n" );
Signature · 1 line
2.28.0
PATCH 5/17 net/au1000-eth: stop using DMA_ATTR_NON_CONSISTENT
In reply to Christoph HellwigThe au1000-eth driver contains none of the manual cache synchronization required for using DMA_ATTR_NON_CONSISTENT. From what I can tell it can be used on both dma coherent and non-coherent DMA platforms, but I suspect it has been buggy on the non-coherent platforms all along.
--- drivers/net/ethernet/amd/au1000_eth.c | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-)
drivers/net/ethernet/amd/au1000_eth.c +6 -9
@@ -1131,10 +1131,9 @@ static int au1000_probe(struct platform_device *pdev)/* Allocate the data buffers * Snooping works fine with eth on all au1xxx */ - aup->vaddr = (u32 )dma_alloc_attrs (&pdev->dev,MAX_BUF_SIZE * + aup->vaddr = (u32 )dma_alloc_coherent (&pdev->dev,MAX_BUF_SIZE * (NUM_TX_BUFFS +NUM_RX_BUFFS ), - &aup->dma_addr,0 , -DMA_ATTR_NON_CONSISTENT ); + &aup->dma_addr,0 );if (!aup->vaddr) {dev_err (&pdev->dev,"failed to allocate data buffers\n" ); err = -ENOMEM ;
@@ -1310,9 +1309,8 @@ static int au1000_probe(struct platform_device *pdev) err_remap2:iounmap (aup->mac); err_remap1: -dma_free_attrs (&pdev->dev,MAX_BUF_SIZE * (NUM_TX_BUFFS +NUM_RX_BUFFS ), - (void *)aup->vaddr, aup->dma_addr, -DMA_ATTR_NON_CONSISTENT ); +dma_free_coherent (&pdev->dev,MAX_BUF_SIZE * (NUM_TX_BUFFS +NUM_RX_BUFFS ), + (void *)aup->vaddr, aup->dma_addr); err_vaddr:free_netdev (dev); err_alloc:
@@ -1344,9 +1342,8 @@ static int au1000_remove(struct platform_device *pdev)if (aup->tx_db_inuse[i])au1000_ReleaseDB (aup, aup->tx_db_inuse[i]); -dma_free_attrs (&pdev->dev,MAX_BUF_SIZE * (NUM_TX_BUFFS +NUM_RX_BUFFS ), - (void *)aup->vaddr, aup->dma_addr, -DMA_ATTR_NON_CONSISTENT ); +dma_free_coherent (&pdev->dev,MAX_BUF_SIZE * (NUM_TX_BUFFS +NUM_RX_BUFFS ), + (void *)aup->vaddr, aup->dma_addr);iounmap (aup->macdma);iounmap (aup->mac);
Signature · 1 line
2.28.0
PATCH 6/17 lib82596: move DMA allocation into the callers of i82596_probe
In reply to Christoph HellwigThis allows us to get rid of the LIB82596_DMA_ATTR defined and prepare for untangling the coherent vs non-coherent DMA allocation API.
--- drivers/net/ethernet/i825xx/lasi_82596.c | 24 ++++++++++------ drivers/net/ethernet/i825xx/lib82596.c | 36 ++++++++---------------- drivers/net/ethernet/i825xx/sni_82596.c | 19 +++++++++---- 3 files changed, 40 insertions(+), 39 deletions(-)
drivers/net/ethernet/i825xx/lasi_82596.c +16 -8
@@ -96,8 +96,6 @@#define OPT_SWAP_PORT 0x0001/* Need to wordswp on the MPU port */ -#define LIB82596_DMA_ATTR DMA_ATTR_NON_CONSISTENT -#define DMA_WBACK (ndev, addr, len) \ do { dma_cache_sync((ndev)->dev.parent, (void *)addr, len, DMA_TO_DEVICE); } while (0)
@@ -155,7 +153,7 @@ lan_init_chip(struct parisc_device *dev) {struct net_device *netdevice;struct i596_private *lp; -int retval; +int retval = -ENOMEM ;int i;if (!dev->irq) {
@@ -186,12 +184,22 @@ lan_init_chip(struct parisc_device *dev) lp =netdev_priv (netdevice); lp->options = dev->id.sversion ==0x72 ?OPT_SWAP_PORT :0 ; + lp->dma =dma_alloc_attrs (&dev->dev,sizeof (struct i596_dma ), + &lp->dma_addr,GFP_KERNEL , +DMA_ATTR_NON_CONSISTENT ); +if (!lp->dma) + goto out_free_netdev; retval =i82596_probe (netdevice); -if (retval) { -free_netdev (netdevice); -return -ENODEV ; - } +if (retval) + goto out_free_dma; +return 0 ; + +out_free_dma: +dma_free_attrs (&dev->dev,sizeof (struct i596_dma ), lp->dma, + lp->dma_addr,DMA_ATTR_NON_CONSISTENT ); +out_free_netdev: +free_netdev (netdevice);return retval; }
@@ -202,7 +210,7 @@ static int __exit lan_remove_chip(struct parisc_device *pdev)unregister_netdev (dev);dma_free_attrs (&pdev->dev,sizeof (struct i596_private ), lp->dma, - lp->dma_addr,LIB82596_DMA_ATTR ); + lp->dma_addr,DMA_ATTR_NON_CONSISTENT );free_netdev (dev);return 0 ; }
drivers/net/ethernet/i825xx/lib82596.c +11 -25
@@ -1047,9 +1047,8 @@ static const struct net_device_ops i596_netdev_ops = {static int i82596_probe (struct net_device *dev) { -int i;struct i596_private *lp =netdev_priv (dev); -struct i596_dma *dma; +int ret;/* This lot is ensure things have been cache line aligned. */ BUILD_BUG_ON (sizeof (struct i596_rfd ) !=32 );
@@ -1063,41 +1062,28 @@ static int i82596_probe(struct net_device *dev)if (!dev->base_addr || !dev->irq)return -ENODEV ; - dma =dma_alloc_attrs (dev->dev.parent,sizeof (struct i596_dma ), - &lp->dma_addr,GFP_KERNEL , -LIB82596_DMA_ATTR ); -if (!dma) { -printk (KERN_ERR "%s: Couldn't get shared memory\n" , __FILE__); -return -ENOMEM ; - } - dev->netdev_ops = &i596_netdev_ops; dev->watchdog_timeo =TX_TIMEOUT ; -memset (dma,0 ,sizeof (struct i596_dma )); - lp->dma = dma; - - dma->scb.command =0 ; - dma->scb.cmd =I596_NULL ; - dma->scb.rfd =I596_NULL ; +memset (lp->dma,0 ,sizeof (struct i596_dma )); + lp->dma->scb.command =0 ; + lp->dma->scb.cmd =I596_NULL ; + lp->dma->scb.rfd =I596_NULL ;spin_lock_init (&lp->lock); -DMA_WBACK_INV (dev, dma,sizeof (struct i596_dma )); +DMA_WBACK_INV (dev, lp->dma,sizeof (struct i596_dma )); - i =register_netdev (dev); -if (i) { -dma_free_attrs (dev->dev.parent,sizeof (struct i596_dma ), - dma, lp->dma_addr,LIB82596_DMA_ATTR ); -return i; - } + ret =register_netdev (dev); +if (ret) +return ret;DEB (DEB_PROBE ,printk (KERN_INFO "%s: 82596 at %#3lx, %pM IRQ %d.\n" , dev->name, dev->base_addr, dev->dev_addr, dev->irq));DEB (DEB_INIT ,printk (KERN_INFO "%s: dma at 0x%p (%d bytes), lp->scb at 0x%p\n" , - dev->name, dma, (int )sizeof (struct i596_dma ), - &dma->scb)); + dev->name, lp->dma, (int )sizeof (struct i596_dma ), + &lp->dma->scb));return 0 ; }
drivers/net/ethernet/i825xx/sni_82596.c +13 -6
@@ -24,8 +24,6 @@static const char sni_82596_string[] ="snirm_82596" ; -#define LIB82596_DMA_ATTR 0 -#define DMA_WBACK (priv, addr, len) do { } while (0)#define DMA_INV (priv, addr, len) do { } while (0)#define DMA_WBACK_INV (priv, addr, len) do { } while (0)
@@ -134,10 +132,19 @@ static int sni_82596_probe(struct platform_device *dev) lp->ca = ca_addr; lp->mpu_port = mpu_addr; + lp->dma =dma_alloc_coherent (&dev->dev,sizeof (struct i596_dma ), + &lp->dma_addr,GFP_KERNEL ); +if (!lp->dma) + goto probe_failed; + retval =i82596_probe (netdevice); -if (retval ==0 ) -return 0 ; +if (retval) + goto probe_failed_free_dma; +return 0 ; +probe_failed_free_dma: +dma_free_coherent (&dev->dev,sizeof (struct i596_dma ), lp->dma, + lp->dma_addr); probe_failed:free_netdev (netdevice); probe_failed_free_ca:
@@ -153,8 +160,8 @@ static int sni_82596_driver_remove(struct platform_device *pdev)struct i596_private *lp =netdev_priv (dev);unregister_netdev (dev); -dma_free_attrs (dev->dev.parent,sizeof (struct i596_private ), lp->dma, - lp->dma_addr,LIB82596_DMA_ATTR ); +dma_free_coherent (&pdev->dev,sizeof (struct i596_private ), lp->dma, + lp->dma_addr);iounmap (lp->ca);iounmap (lp->mpu_port);free_netdev (dev);
Signature · 1 line
2.28.0
PATCH 7/17 53c700: improve non-coherent DMA handling
In reply to Christoph HellwigSwitch the 53c700 driver to only use non-coherent descriptor memory if it really has to because dma_alloc_coherent fails. This doesn't matter for any of the platforms it runs on currently, but that will change soon.
To help with this two new helpers to transfer ownership to and from the device are added that abstract the syncing of the non-coherent memory. The two current bidirectional cases are mapped to transfers to the device, as that appears to what they are used for. Note that for parisc, which is the only architecture this driver needs to use non-coherent memory on, the direction argument of dma_cache_sync is ignored, so this will not change behavior in any way.
--- drivers/scsi/53c700.c | 113 +++++++++++++++++++++++------------------- drivers/scsi/53c700.h | 9 ++-- 2 files changed, 68 insertions(+), 54 deletions(-)
drivers/scsi/53c700.c +63 -50
@@ -269,6 +269,20 @@ NCR_700_get_SXFER(struct scsi_device *SDp) spi_period(SDp->sdev_target)); } +static inline void dma_sync_to_dev (struct NCR_700_Host_Parameters *h, +void *addr,size_t size) +{ +if (h->noncoherent) +dma_cache_sync (h->dev, addr, size,DMA_TO_DEVICE ); +} + +static inline void dma_sync_from_dev (struct NCR_700_Host_Parameters *h, +void *addr,size_t size) +{ +if (h->noncoherent) +dma_cache_sync (h->dev, addr, size,DMA_FROM_DEVICE ); +} +struct Scsi_Host *NCR_700_detect (struct scsi_host_template *tpnt,struct NCR_700_Host_Parameters *hostdata,struct device *dev)
@@ -283,9 +297,13 @@ NCR_700_detect(struct scsi_host_template *tpnt,if (tpnt->sdev_attrs ==NULL ) tpnt->sdev_attrs = NCR_700_dev_attrs; - memory =dma_alloc_attrs (dev,TOTAL_MEM_SIZE , &pScript, -GFP_KERNEL ,DMA_ATTR_NON_CONSISTENT ); -if (memory ==NULL ) { + memory =dma_alloc_coherent (dev,TOTAL_MEM_SIZE , &pScript,GFP_KERNEL ); +if (!memory) { + hostdata->noncoherent =1 ; + memory =dma_alloc_attrs (dev,TOTAL_MEM_SIZE , &pScript, +GFP_KERNEL ,DMA_ATTR_NON_CONSISTENT ); + } +if (!memory) {printk (KERN_ERR "53c700: Failed to allocate memory for driver, detaching\n" );return NULL ; }
@@ -339,11 +357,11 @@ NCR_700_detect(struct scsi_host_template *tpnt,for (j =0 ; j <PATCHES ; j++) script[LABELPATCHES [j]] =bS_to_host (pScript +SCRIPT [LABELPATCHES [j]]);/* now patch up fixed addresses. */ -script_patch_32 (hostdata->dev, script, MessageLocation, +script_patch_32 (hostdata, script, MessageLocation, pScript +MSGOUT_OFFSET ); -script_patch_32 (hostdata->dev, script, StatusAddress, +script_patch_32 (hostdata, script, StatusAddress, pScript +STATUS_OFFSET ); -script_patch_32 (hostdata->dev, script, ReceiveMsgAddress, +script_patch_32 (hostdata, script, ReceiveMsgAddress, pScript +MSGIN_OFFSET ); hostdata->script = script;
@@ -395,8 +413,12 @@ NCR_700_release(struct Scsi_Host *host)struct NCR_700_Host_Parameters *hostdata = (struct NCR_700_Host_Parameters *)host->hostdata[0 ]; -dma_free_attrs (hostdata->dev,TOTAL_MEM_SIZE , hostdata->script, - hostdata->pScript,DMA_ATTR_NON_CONSISTENT ); +if (hostdata->noncoherent) +dma_free_attrs (hostdata->dev,TOTAL_MEM_SIZE , hostdata->script, + hostdata->pScript,DMA_ATTR_NON_CONSISTENT ); +else +dma_free_coherent (hostdata->dev,TOTAL_MEM_SIZE , + hostdata->script, hostdata->pScript);return 1 ; }
@@ -804,8 +826,8 @@ process_extended_message(struct Scsi_Host *host,shost_printk (KERN_WARNING , host,"Unexpected SDTR msg\n" ); hostdata->msgout[0 ] =A_REJECT_MSG ; -dma_cache_sync (hostdata->dev, hostdata->msgout,1 ,DMA_TO_DEVICE ); -script_patch_16 (hostdata->dev, hostdata->script, +dma_sync_to_dev (hostdata, hostdata->msgout,1 ); +script_patch_16 (hostdata, hostdata->script, MessageCount,1 );/* SendMsgOut returns, so set up the return * address */
@@ -817,9 +839,8 @@ process_extended_message(struct Scsi_Host *host,printk (KERN_INFO "scsi%d: (%d:%d), Unsolicited WDTR after CMD, Rejecting\n" , host->host_no, pun, lun); hostdata->msgout[0 ] =A_REJECT_MSG ; -dma_cache_sync (hostdata->dev, hostdata->msgout,1 ,DMA_TO_DEVICE ); -script_patch_16 (hostdata->dev, hostdata->script, MessageCount, -1 ); +dma_sync_to_dev (hostdata, hostdata->msgout,1 ); +script_patch_16 (hostdata, hostdata->script, MessageCount,1 ); resume_offset = hostdata->pScript + Ent_SendMessageWithATN;break ;
@@ -832,9 +853,8 @@ process_extended_message(struct Scsi_Host *host,printk ("\n" );/* just reject it */ hostdata->msgout[0 ] =A_REJECT_MSG ; -dma_cache_sync (hostdata->dev, hostdata->msgout,1 ,DMA_TO_DEVICE ); -script_patch_16 (hostdata->dev, hostdata->script, MessageCount, -1 ); +dma_sync_to_dev (hostdata, hostdata->msgout,1 ); +script_patch_16 (hostdata, hostdata->script, MessageCount,1 );/* SendMsgOut returns, so set up the return * address */ resume_offset = hostdata->pScript + Ent_SendMessageWithATN;
@@ -917,9 +937,8 @@ process_message(struct Scsi_Host *host, struct NCR_700_Host_Parameters *hostdataprintk ("\n" );/* just reject it */ hostdata->msgout[0 ] =A_REJECT_MSG ; -dma_cache_sync (hostdata->dev, hostdata->msgout,1 ,DMA_TO_DEVICE ); -script_patch_16 (hostdata->dev, hostdata->script, MessageCount, -1 ); +dma_sync_to_dev (hostdata, hostdata->msgout,1 ); +script_patch_16 (hostdata, hostdata->script, MessageCount,1 );/* SendMsgOut returns, so set up the return * address */ resume_offset = hostdata->pScript + Ent_SendMessageWithATN;
@@ -928,7 +947,7 @@ process_message(struct Scsi_Host *host, struct NCR_700_Host_Parameters *hostdata }NCR_700_writel (temp, host,TEMP_REG );/* set us up to receive another message */ -dma_cache_sync (hostdata->dev, hostdata->msgin,MSG_ARRAY_SIZE ,DMA_FROM_DEVICE ); +dma_sync_from_dev (hostdata, hostdata->msgin,MSG_ARRAY_SIZE );return resume_offset; }
@@ -1008,8 +1027,8 @@ process_script_interrupt(__u32 dsps, __u32 dsp, struct scsi_cmnd *SCp, slot->SG[1 ].ins =bS_to_host (SCRIPT_RETURN ); slot->SG[1 ].pAddr =0 ; slot->resume_offset = hostdata->pScript; -dma_cache_sync (hostdata->dev, slot->SG,sizeof (slot->SG[0 ])*2 ,DMA_TO_DEVICE ); -dma_cache_sync (hostdata->dev, SCp->sense_buffer,SCSI_SENSE_BUFFERSIZE ,DMA_FROM_DEVICE ); +dma_sync_to_dev (hostdata, slot->SG,sizeof (slot->SG[0 ])*2 ); +dma_sync_from_dev (hostdata, SCp->sense_buffer,SCSI_SENSE_BUFFERSIZE );/* queue the command for reissue */ slot->state =NCR_700_SLOT_QUEUED ;
@@ -1129,11 +1148,11 @@ process_script_interrupt(__u32 dsps, __u32 dsp, struct scsi_cmnd *SCp, hostdata->cmd = slot->cmnd;/* re-patch for this command */ -script_patch_32_abs (hostdata->dev, hostdata->script, +script_patch_32_abs (hostdata, hostdata->script, CommandAddress, slot->pCmd); -script_patch_16 (hostdata->dev, hostdata->script, +script_patch_16 (hostdata, hostdata->script, CommandCount, slot->cmnd->cmd_len); -script_patch_32_abs (hostdata->dev, hostdata->script, +script_patch_32_abs (hostdata, hostdata->script, SGScriptStartAddress,to32bit (&slot->pSG[0 ].ins));
@@ -1144,14 +1163,14 @@ process_script_interrupt(__u32 dsps, __u32 dsp, struct scsi_cmnd *SCp,* should therefore always clear ACK */ NCR_700_writeb (NCR_700_get_SXFER (hostdata->cmd->device), host,SXFER_REG ); -dma_cache_sync (hostdata->dev, hostdata->msgin, -MSG_ARRAY_SIZE ,DMA_FROM_DEVICE ); -dma_cache_sync (hostdata->dev, hostdata->msgout, -MSG_ARRAY_SIZE ,DMA_TO_DEVICE ); +dma_sync_from_dev (hostdata, hostdata->msgin, +MSG_ARRAY_SIZE ); +dma_sync_to_dev (hostdata, hostdata->msgout, +MSG_ARRAY_SIZE );/* I'm just being paranoid here, the command should * already have been flushed from the cache */ -dma_cache_sync (hostdata->dev, slot->cmnd->cmnd, - slot->cmnd->cmd_len,DMA_TO_DEVICE ); +dma_sync_to_dev (hostdata, slot->cmnd->cmnd, + slot->cmnd->cmd_len);
@@ -1214,8 +1233,7 @@ process_script_interrupt(__u32 dsps, __u32 dsp, struct scsi_cmnd *SCp, hostdata->reselection_id = reselection_id;/* just in case we have a stale simple tag message, clear it */ hostdata->msgin[1 ] =0 ; -dma_cache_sync (hostdata->dev, hostdata->msgin, -MSG_ARRAY_SIZE ,DMA_BIDIRECTIONAL ); +dma_sync_to_dev (hostdata, hostdata->msgin,MSG_ARRAY_SIZE );if (hostdata->tag_negotiated & (1 <<reselection_id)) { resume_offset = hostdata->pScript + Ent_GetReselectionWithTag; }else {
@@ -1329,8 +1347,7 @@ process_selection(struct Scsi_Host *host, __u32 dsp) hostdata->cmd =NULL ;/* clear any stale simple tag message */ hostdata->msgin[1 ] =0 ; -dma_cache_sync (hostdata->dev, hostdata->msgin,MSG_ARRAY_SIZE , -DMA_BIDIRECTIONAL ); +dma_sync_to_dev (hostdata, hostdata->msgin,MSG_ARRAY_SIZE );if (id ==0xff ) {/* Selected as target, Ignore */
@@ -1427,30 +1444,26 @@ NCR_700_start_command(struct scsi_cmnd *SCp)NCR_700_set_flag (SCp->device,NCR_700_DEV_BEGIN_SYNC_NEGOTIATION ); } -script_patch_16 (hostdata->dev, hostdata->script, MessageCount, count); - +script_patch_16 (hostdata, hostdata->script, MessageCount, count); -script_patch_ID (hostdata->dev, hostdata->script, - Device_ID,1 <<scmd_id (SCp)); +script_patch_ID (hostdata, hostdata->script, Device_ID,1 <<scmd_id (SCp)); -script_patch_32_abs (hostdata->dev, hostdata->script, CommandAddress, +script_patch_32_abs (hostdata, hostdata->script, CommandAddress, slot->pCmd); -script_patch_16 (hostdata->dev, hostdata->script, CommandCount, - SCp->cmd_len); +script_patch_16 (hostdata, hostdata->script, CommandCount, SCp->cmd_len);/* finally plumb the beginning of the SG list into the script * */ -script_patch_32_abs (hostdata->dev, hostdata->script, +script_patch_32_abs (hostdata, hostdata->script, SGScriptStartAddress,to32bit (&slot->pSG[0 ].ins));NCR_700_clear_fifo (SCp->device->host);if (slot->resume_offset ==0 ) slot->resume_offset = hostdata->pScript;/* now perform all the writebacks and invalidates */ -dma_cache_sync (hostdata->dev, hostdata->msgout, count,DMA_TO_DEVICE ); -dma_cache_sync (hostdata->dev, hostdata->msgin,MSG_ARRAY_SIZE , -DMA_FROM_DEVICE ); -dma_cache_sync (hostdata->dev, SCp->cmnd, SCp->cmd_len,DMA_TO_DEVICE ); -dma_cache_sync (hostdata->dev, hostdata->status,1 ,DMA_FROM_DEVICE ); +dma_sync_to_dev (hostdata, hostdata->msgout, count); +dma_sync_from_dev (hostdata, hostdata->msgin,MSG_ARRAY_SIZE ); +dma_sync_to_dev (hostdata, SCp->cmnd, SCp->cmd_len); +dma_sync_from_dev (hostdata, hostdata->status,1 );/* set the synchronous period/offset */ NCR_700_writeb(NCR_700_get_SXFER(SCp->device),
@@ -1626,7 +1639,7 @@ NCR_700_intr(int irq, void *dev_id) slot->SG[i].ins =bS_to_host (SCRIPT_NOP ); slot->SG[i].pAddr =0 ; } -dma_cache_sync (hostdata->dev, slot->SG,sizeof (slot->SG),DMA_TO_DEVICE ); +dma_sync_to_dev (hostdata, slot->SG,sizeof (slot->SG));/* and pretend we disconnected after * the command phase */ resume_offset = hostdata->pScript + Ent_MsgInDuringData;
@@ -1878,7 +1891,7 @@ NCR_700_queuecommand_lck(struct scsi_cmnd *SCp, void (*done)(struct scsi_cmnd *) } slot->SG[i].ins =bS_to_host (SCRIPT_RETURN ); slot->SG[i].pAddr =0 ; -dma_cache_sync (hostdata->dev, slot->SG,sizeof (slot->SG),DMA_TO_DEVICE ); +dma_sync_to_dev (hostdata, slot->SG,sizeof (slot->SG));DEBUG ((" SETTING %p to %x\n" , (&slot->pSG[i].ins), slot->SG[i].ins));
drivers/scsi/53c700.h +5 -4
@@ -209,6 +209,7 @@ struct NCR_700_Host_Parameters {#endif __u32 chip710:1; /* set if really a 710 not 700 */ __u32 burst_length:4; /* set to 0 to disable 710 bursting */ + __u32 noncoherent:1; /* needs to use non-coherent DMA *//* NOTHING BELOW HERE NEEDS ALTERING */ __u32 fast:1; /* if we can alter the SCSI bus clock
@@ -429,7 +430,7 @@ struct NCR_700_Host_Parameters { for(i=0; i< (sizeof(A_##symbol##_used) / sizeof(__u32)); i++) { \ __u32 val = bS_to_cpu((script)[A_##symbol##_used[i]]) + da; \ (script)[A_##symbol##_used[i]] = bS_to_host(val); \ - dma_cache_sync((dev), &(script)[A_##symbol##_used[i]], 4, DMA_TO_DEVICE); \ + dma_sync_to_dev((dev), &(script)[A_##symbol##_used[i]], 4); \ DEBUG((" script, patching %s at %d to %pad\n", \ #symbol, A_##symbol##_used[i], &da)); \ } \
@@ -441,7 +442,7 @@ struct NCR_700_Host_Parameters {dma_addr_t da = value; \ for(i=0; i< (sizeof(A_##symbol##_used) / sizeof(__u32)); i++) { \ (script)[A_##symbol##_used[i]] = bS_to_host(da); \ - dma_cache_sync((dev), &(script)[A_##symbol##_used[i]], 4, DMA_TO_DEVICE); \ + dma_sync_to_dev((dev), &(script)[A_##symbol##_used[i]], 4); \ DEBUG((" script, patching %s at %d to %pad\n", \ #symbol, A_##symbol##_used[i], &da)); \ } \
@@ -456,7 +457,7 @@ struct NCR_700_Host_Parameters { val &=0xff00ffff ; \ val |= ((value) &0xff ) <<16 ; \ (script)[A_##symbol##_used[i]] = bS_to_host(val); \ - dma_cache_sync((dev), &(script)[A_##symbol##_used[i]], 4, DMA_TO_DEVICE); \ + dma_sync_to_dev((dev), &(script)[A_##symbol##_used[i]], 4); \ DEBUG((" script, patching ID field %s at %d to 0x%x\n", \ #symbol, A_##symbol##_used[i], val)); \ } \
@@ -470,7 +471,7 @@ struct NCR_700_Host_Parameters { val &=0xffff0000 ; \ val |= ((value) &0xffff ); \ (script)[A_##symbol##_used[i]] = bS_to_host(val); \ - dma_cache_sync((dev), &(script)[A_##symbol##_used[i]], 4, DMA_TO_DEVICE); \ + dma_sync_to_dev((dev), &(script)[A_##symbol##_used[i]], 4); \ DEBUG((" script, patching short field %s at %d to 0x%x\n", \ #symbol, A_##symbol##_used[i], val)); \ } \
Signature · 1 line
2.28.0
James Bottomley
maintainerRe: [PATCH 07/17] 53c700: improve non-coherent DMA handling
Quoted 47 lines · Christoph Hellwig
(quoted diff) +1
-1@@ -429,7 +430,7 @@ struct NCR_700_Host_Parameters { for(i=0; i< (sizeof(A_##symbol##_used) / sizeof(__u32)); i++) { \ __u32 val = bS_to_cpu((script)[A_##symbol##_used[i]]) + da; \ (script)[A_##symbol##_used[i]] = bS_to_host(val); \ - dma_cache_sync((dev), &(script)[A_##symbol##_used[i]], 4, DMA_TO_DEVICE); \ + dma_sync_to_dev((dev),&(script)[A_##symbol##_used[i]], 4); \
drivers/scsi/53c700.h +1
-1DEBUG ((" script, patching %s at %d to %pad\n" , \ #symbol, A_##symbol##_used[i], &da)); \ } \@@ -441,7 +442,7 @@ struct NCR_700_Host_Parameters {dma_addr_t da = value; \ for(i=0; i< (sizeof(A_##symbol##_used) / sizeof(__u32)); i++) { \ (script)[A_##symbol##_used[i]] = bS_to_host(da); \ - dma_cache_sync((dev), &(script)[A_##symbol##_used[i]], 4, DMA_TO_DEVICE); \ + dma_sync_to_dev((dev), &(script)[A_##symbol##_used[i]], 4); \drivers/scsi/53c700.h +1
-1DEBUG ((" script, patching %s at %d to %pad\n" , \ #symbol, A_##symbol##_used[i], &da)); \ } \@@ -456,7 +457,7 @@ struct NCR_700_Host_Parameters { val &=0xff00ffff ; \ val |= ((value) &0xff ) <<16 ; \ (script)[A_##symbol##_used[i]] = bS_to_host(val); \ - dma_cache_sync((dev), &(script)[A_##symbol##_used[i]], 4, DMA_TO_DEVICE); \ + dma_sync_to_dev((dev), &(script)[A_##symbol##_used[i]], 4); \ DEBUG((" script, patching ID field %s at %d to0x%x\n", \
(quoted diff) +1
-1#symbol, A_##symbol##_used[i], val)); \ } \@@ -470,7 +471,7 @@ struct NCR_700_Host_Parameters { val &= 0xffff0000; \ val |= ((value) & 0xffff); \ (script)[A_##symbol##_used[i]] = bS_to_host(val); \ - dma_cache_sync((dev), &(script)[A_##symbol##_used[i]], 4, DMA_TO_DEVICE); \ + dma_sync_to_dev((dev), &(script)[A_##symbol##_used[i]], 4); \ DEBUG((" script, patching short field %s at %d to0x%x\n", \
Review
Quoted 2 lines
(quoted diff) +0
-0#symbol, A_##symbol##_used[i], val)); \ } \
If you're going to change the macros from taking a device to taking a hostdata structure then the descriptive argument name needs to change ... it can't be dev anymore. I'm happy with it simply becoming 'h' if hostdata is too long.
I already asked for this on the first go around:
https://lore.kernel.org/alsa-devel/1598971960.4238.5.camel-d9PhHud1JfjCXq6kfMZ53/[email protected]/
James
Re: [PATCH 07/17] 53c700: improve non-coherent DMA handling
Quoted 6 lines · James Bottomley
If you're going to change the macros from taking a device to taking a hostdata structure then the descriptive argument name needs to change ... it can't be dev anymore. I'm happy with it simply becoming 'h' if hostdata is too long.
I already asked for this on the first go around:
And I did rename them, those hunks just accidentally slipped into patch 12 instead of this one. Fixed for the next versions.
James Bottomley
maintainerRe: [PATCH 07/17] 53c700: improve non-coherent DMA handling
Quoted 10 lines · Christoph Hellwig
Quoted 6 lines · James Bottomley
If you're going to change the macros from taking a device to taking a hostdata structure then the descriptive argument name needs to change ... it can't be dev anymore. I'm happy with it simply becoming 'h' if hostdata is too long.
I already asked for this on the first go around:
And I did rename them, those hunks just accidentally slipped into patch 12 instead of this one. Fixed for the next versions.
Ah, yes, found it ... thanks for doing that!
James
PATCH 8/17 dma-mapping: add a new dma_alloc_noncoherent API
In reply to Christoph HellwigAdd a new API to allocate and free memory that is guaranteed to be addressable by a device, but which potentially is not cache coherent for DMA.
To transfer ownership to and from the device, the existing streaming DMA API calls dma_sync_single_for_device and dma_sync_single_for_cpu must be used.
For now the new calls are implemented on top of dma_alloc_attrs just like the old-noncoherent API, but once all drivers are switched to the new API it will be replaced with a better working implementation that is available on all architectures.
--- Documentation/core-api/dma-api.rst | 75 ++++++++++++++---------------- include/linux/dma-mapping.h | 12 +++++ 2 files changed, 48 insertions(+), 39 deletions(-)
Documentation/core-api/dma-api.rst +36 -39
@@ -516,48 +516,56 @@ routines, e.g.::: } -Part II - Advanced dma usage ----------------------------- +Part II - Non-coherent DMA allocations +-------------------------------------- -Warning: These pieces of the DMA API should not be used in the -majority of cases, since they cater for unlikely corner cases that -don't belong in usual drivers. +These APIs allow to allocate pages in the kernel direct mapping that are +guaranteed to be DMA addressable. This means that unlike dma_alloc_coherent, +virt_to_page can be called on the resulting address, and the resulting +struct page can be used for everything a struct page is suitable for. -If you don't understand how cache line coherency works between a -processor and an I/O device, you should not be using this part of the -API at all. +If you don't understand how cache line coherency works between a processor and +an I/O device, you should not be using this part of the API. :: void * - dma_alloc_attrs(struct device *dev, size_t size, dma_addr_t *dma_handle, - gfp_t flag, unsigned long attrs) + dma_alloc_noncoherent(struct device *dev, size_t size, + dma_addr_t *dma_handle, enum dma_data_direction dir, + gfp_t gfp) -Identical to dma_alloc_coherent() except that when the -DMA_ATTR_NON_CONSISTENT flags is passed in the attrs argument, the -platform will choose to return either consistent or non-consistent memory -as it sees fit. By using this API, you are guaranteeing to the platform -that you have all the correct and necessary sync points for this memory -in the driver should it choose to return non-consistent memory. +This routine allocates a region of <size> bytes of consistent memory. It +returns a pointer to the allocated region (in the processor's virtual address +space) or NULL if the allocation failed. The returned memory may or may not +be in the kernels direct mapping. Drivers must not call virt_to_page on +the returned memory region. -Note: where the platform can return consistent memory, it will -guarantee that the sync points become nops. +It also returns a <dma_handle> which may be cast to an unsigned integer the +same width as the bus and given to the device as the DMA address base of +the region. -Warning: Handling non-consistent memory is a real pain. You should -only use this API if you positively know your driver will be -required to work on one of the rare (usually non-PCI) architectures -that simply cannot make consistent memory. +The dir parameter specified if data is read and/or written by the device, +see dma_map_single() for details. + +The gfp parameter allows the caller to specify the ``GFP_`` flags (see +kmalloc()) for the allocation, but rejects flags used to specify a memory +zone such as GFP_DMA or GFP_HIGHMEM. + +Before giving the memory to the device, dma_sync_single_for_device() needs +to be called, and before reading memory written by the device, +dma_sync_single_for_cpu(), just like for streaming DMA mappings that are +reused. :: void - dma_free_attrs(struct device *dev, size_t size, void *cpu_addr, - dma_addr_t dma_handle, unsigned long attrs) + dma_free_noncoherent(struct device *dev, size_t size, void *cpu_addr, + dma_addr_t dma_handle, enum dma_data_direction dir) -Free memory allocated by the dma_alloc_attrs(). All common -parameters must be identical to those otherwise passed to dma_free_coherent, -and the attrs argument must be identical to the attrs passed to -dma_alloc_attrs(). +Free a region of memory previously allocated using dma_alloc_noncoherent(). +dev, size and dma_handle and dir must all be the same as those passed into +dma_alloc_noncoherent(). cpu_addr must be the virtual address returned by +the dma_alloc_noncoherent(). ::
@@ -575,17 +583,6 @@ memory or doing partial flushes. into the width returned by this call. It will also always be a power of two for easy alignment. -:: - - void - dma_cache_sync(struct device *dev, void *vaddr, size_t size, - enum dma_data_direction direction) - -Do a partial sync of memory that was allocated by dma_alloc_attrs() with -the DMA_ATTR_NON_CONSISTENT flag starting at virtual address vaddr and -continuing on for size. Again, you *must* observe the cache line -boundaries when doing this. - Part III - Debug drivers use of the DMA-API -------------------------------------------
include/linux/dma-mapping.h +12 -0
@@ -389,6 +389,18 @@ static inline unsigned long dma_get_merge_boundary(struct device *dev) }#endif /* CONFIG_HAS_DMA */ +static inline void *dma_alloc_noncoherent (struct device *dev,size_t size, +dma_addr_t *dma_handle,enum dma_data_direction dir,gfp_t gfp) +{ +return dma_alloc_attrs (dev, size, dma_handle, gfp, +DMA_ATTR_NON_CONSISTENT ); +} +static inline void dma_free_noncoherent (struct device *dev,size_t size, +void *vaddr,dma_addr_t dma_handle,enum dma_data_direction dir) +{ +dma_free_attrs (dev, size, vaddr, dma_handle,DMA_ATTR_NON_CONSISTENT ); +} +static inline dma_addr_t dma_map_single_attrs (struct device *dev,void *ptr,size_t size,enum dma_data_direction dir,unsigned long attrs) {
Signature · 1 line
2.28.0
PATCH 9/17 sgiwd93: convert to dma_alloc_noncoherent
In reply to Christoph HellwigUse the new non-coherent DMA API including proper ownership transfers. This also means we can allocate the memory as DMA_TO_DEVICE instead of bidirectional.
--- drivers/scsi/sgiwd93.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
drivers/scsi/sgiwd93.c +7 -7
@@ -95,7 +95,7 @@ void fill_hpc_entries(struct ip22_hostdata *hd, struct scsi_cmnd *cmd, int din)*/ hcp->desc.pbuf =0 ; hcp->desc.cntinfo =HPCDMA_EOX ; -dma_cache_sync (hd->dev, hd->cpu, +dma_sync_single_for_device (hd->dev, hd->dma, (unsigned long )(hcp +1 ) - (unsigned long )hd->cpu,DMA_TO_DEVICE ); }
@@ -234,8 +234,8 @@ static int sgiwd93_probe(struct platform_device *pdev) hdata =host_to_hostdata (host); hdata->dev = &pdev->dev; - hdata->cpu =dma_alloc_attrs (&pdev->dev,HPC_DMA_SIZE , &hdata->dma, -GFP_KERNEL ,DMA_ATTR_NON_CONSISTENT ); + hdata->cpu =dma_alloc_noncoherent (&pdev->dev,HPC_DMA_SIZE , + &hdata->dma,DMA_TO_DEVICE ,GFP_KERNEL );if (!hdata->cpu) {printk (KERN_WARNING "sgiwd93: Could not allocate memory for " "host %d buffer.\n" , unit);
@@ -274,8 +274,8 @@ static int sgiwd93_probe(struct platform_device *pdev) out_irq:free_irq (irq, host); out_free: -dma_free_attrs (&pdev->dev,HPC_DMA_SIZE , hdata->cpu, hdata->dma, -DMA_ATTR_NON_CONSISTENT ); +dma_free_noncoherent (&pdev->dev,HPC_DMA_SIZE , hdata->cpu, hdata->dma, +DMA_TO_DEVICE ); out_put:scsi_host_put (host); out:
@@ -291,8 +291,8 @@ static int sgiwd93_remove(struct platform_device *pdev)scsi_remove_host (host);free_irq (pd->irq, host); -dma_free_attrs (&pdev->dev,HPC_DMA_SIZE , hdata->cpu, hdata->dma, -DMA_ATTR_NON_CONSISTENT ); +dma_free_noncoherent (&pdev->dev,HPC_DMA_SIZE , hdata->cpu, hdata->dma, +DMA_TO_DEVICE );scsi_host_put (host);return 0 ; }
Signature · 1 line
2.28.0
PATCH 10/17 hal2: convert to dma_alloc_noncoherent
In reply to Christoph HellwigUse the new non-coherent DMA API including proper ownership transfers. This also means we can allocate the buffer memory with the proper direction instead of bidirectional.
--- sound/mips/hal2.c | 58 ++++++++++++++++++++++------------------------- 1 file changed, 27 insertions(+), 31 deletions(-)
sound/mips/hal2.c +27 -31
@@ -441,7 +441,8 @@ static inline void hal2_stop_adc(struct snd_hal2 *hal2) hal2->adc.pbus.pbus->pbdma_ctrl =HPC3_PDMACTRL_LD ; } -static int hal2_alloc_dmabuf (struct snd_hal2 *hal2,struct hal2_codec *codec) +static int hal2_alloc_dmabuf (struct snd_hal2 *hal2,struct hal2_codec *codec, +enum dma_data_direction buffer_dir) {struct device *dev = hal2->card->dev;struct hal2_desc *desc;
@@ -449,15 +450,15 @@ static int hal2_alloc_dmabuf(struct snd_hal2 *hal2, struct hal2_codec *codec)int count =H2_BUF_SIZE /H2_BLOCK_SIZE ;int i; - codec->buffer =dma_alloc_attrs (dev,H2_BUF_SIZE , &buffer_dma, -GFP_KERNEL ,DMA_ATTR_NON_CONSISTENT ); + codec->buffer =dma_alloc_noncoherent (dev,H2_BUF_SIZE , &buffer_dma, + buffer_dir,GFP_KERNEL );if (!codec->buffer)return -ENOMEM ; - desc =dma_alloc_attrs (dev, count *sizeof (struct hal2_desc ), - &desc_dma,GFP_KERNEL ,DMA_ATTR_NON_CONSISTENT ); + desc =dma_alloc_noncoherent (dev, count *sizeof (struct hal2_desc ), + &desc_dma,DMA_BIDIRECTIONAL ,GFP_KERNEL );if (!desc) { -dma_free_attrs (dev,H2_BUF_SIZE , codec->buffer, buffer_dma, -DMA_ATTR_NON_CONSISTENT ); +dma_free_noncoherent (dev,H2_BUF_SIZE , codec->buffer, buffer_dma, + buffer_dir);return -ENOMEM ; } codec->buffer_dma = buffer_dma;
@@ -470,20 +471,22 @@ static int hal2_alloc_dmabuf(struct snd_hal2 *hal2, struct hal2_codec *codec) desc_dma : desc_dma + (i +1 ) *sizeof (struct hal2_desc ); desc++; } -dma_cache_sync (dev, codec->desc, count *sizeof (struct hal2_desc ), -DMA_TO_DEVICE ); +dma_sync_single_for_device (dev, codec->desc_dma, + count *sizeof (struct hal2_desc ), +DMA_BIDIRECTIONAL ); codec->desc_count = count;return 0 ; } -static void hal2_free_dmabuf (struct snd_hal2 *hal2,struct hal2_codec *codec) +static void hal2_free_dmabuf (struct snd_hal2 *hal2,struct hal2_codec *codec, +enum dma_data_direction buffer_dir) {struct device *dev = hal2->card->dev; -dma_free_attrs (dev, codec->desc_count *sizeof (struct hal2_desc ), - codec->desc, codec->desc_dma,DMA_ATTR_NON_CONSISTENT ); -dma_free_attrs (dev,H2_BUF_SIZE , codec->buffer, codec->buffer_dma, -DMA_ATTR_NON_CONSISTENT ); +dma_free_noncoherent (dev, codec->desc_count *sizeof (struct hal2_desc ), + codec->desc, codec->desc_dma,DMA_BIDIRECTIONAL ); +dma_free_noncoherent (dev,H2_BUF_SIZE , codec->buffer, codec->buffer_dma, + buffer_dir); }static const struct snd_pcm_hardware hal2_pcm_hw = {
@@ -509,21 +512,16 @@ static int hal2_playback_open(struct snd_pcm_substream *substream) {struct snd_pcm_runtime *runtime = substream->runtime;struct snd_hal2 *hal2 =snd_pcm_substream_chip (substream); -int err; runtime->hw = hal2_pcm_hw; - - err =hal2_alloc_dmabuf (hal2, &hal2->dac); -if (err) -return err; -return 0 ; +return hal2_alloc_dmabuf (hal2, &hal2->dac,DMA_TO_DEVICE ); }static int hal2_playback_close (struct snd_pcm_substream *substream) {struct snd_hal2 *hal2 =snd_pcm_substream_chip (substream); -hal2_free_dmabuf (hal2, &hal2->dac); +hal2_free_dmabuf (hal2, &hal2->dac,DMA_TO_DEVICE );return 0 ; }
@@ -579,7 +577,9 @@ static void hal2_playback_transfer(struct snd_pcm_substream *substream,unsigned char *buf = hal2->dac.buffer + rec->hw_data;memcpy (buf, substream->runtime->dma_area + rec->sw_data, bytes); -dma_cache_sync (hal2->card->dev, buf, bytes,DMA_TO_DEVICE ); +dma_sync_single_for_device (hal2->card->dev, + hal2->dac.buffer_dma + rec->hw_data, bytes, +DMA_TO_DEVICE ); }
@@ -597,22 +597,16 @@ static int hal2_capture_open(struct snd_pcm_substream *substream) {struct snd_pcm_runtime *runtime = substream->runtime;struct snd_hal2 *hal2 =snd_pcm_substream_chip (substream); -struct hal2_codec *adc = &hal2->adc; -int err; runtime->hw = hal2_pcm_hw; - - err =hal2_alloc_dmabuf (hal2, adc); -if (err) -return err; -return 0 ; +return hal2_alloc_dmabuf (hal2, &hal2->adc,DMA_FROM_DEVICE ); }static int hal2_capture_close (struct snd_pcm_substream *substream) {struct snd_hal2 *hal2 =snd_pcm_substream_chip (substream); -hal2_free_dmabuf (hal2, &hal2->adc); +hal2_free_dmabuf (hal2, &hal2->adc,DMA_FROM_DEVICE );return 0 ; }
@@ -667,7 +661,9 @@ static void hal2_capture_transfer(struct snd_pcm_substream *substream,struct snd_hal2 *hal2 =snd_pcm_substream_chip (substream);unsigned char *buf = hal2->adc.buffer + rec->hw_data; -dma_cache_sync (hal2->card->dev, buf, bytes,DMA_FROM_DEVICE ); +dma_sync_single_for_cpu (hal2->card->dev, + hal2->adc.buffer_dma + rec->hw_data, bytes, +DMA_FROM_DEVICE );memcpy (substream->runtime->dma_area + rec->sw_data, buf, bytes); }
Signature · 1 line
2.28.0
PATCH 11/17 sgiseeq: convert to dma_alloc_noncoherent
In reply to Christoph HellwigUse the new non-coherent DMA API including proper ownership transfers. This includes adding additional calls to dma_sync_desc_dev as the old syncing was rather ad-hoc.
Thanks to Thomas Bogendoerfer for debugging the ownership transfer issues.
--- drivers/net/ethernet/i825xx/lasi_82596.c | 25 ++--- drivers/net/ethernet/i825xx/lib82596.c | 114 ++++++++++++++--------- drivers/net/ethernet/i825xx/sni_82596.c | 4 - drivers/net/ethernet/seeq/sgiseeq.c | 28 ++++-- drivers/scsi/53c700.c | 9 +- 5 files changed, 103 insertions(+), 77 deletions(-)
drivers/net/ethernet/i825xx/lasi_82596.c +9 -16
@@ -96,21 +96,14 @@#define OPT_SWAP_PORT 0x0001/* Need to wordswp on the MPU port */ -#define DMA_WBACK (ndev, addr, len) \ - do { dma_cache_sync((ndev)->dev.parent, (void *)addr, len, DMA_TO_DEVICE); } while (0) - -#define DMA_INV (ndev, addr, len) \ - do { dma_cache_sync((ndev)->dev.parent, (void *)addr, len, DMA_FROM_DEVICE); } while (0) - -#define DMA_WBACK_INV (ndev, addr, len) \ - do { dma_cache_sync((ndev)->dev.parent, (void *)addr, len, DMA_BIDIRECTIONAL); } while (0) -#define SYSBUS 0x0000006c/* big endian CPU, 82596 "big" endian mode */ #define SWAP32 (x) (((u32)(x)<<16) | ((((u32)(x)))>>16))#define SWAP16 (x) (x) +#define NONCOHERENT_DMA 1 +#include "lib82596.c" MODULE_AUTHOR ("Richard Hirst" );
@@ -184,9 +177,9 @@ lan_init_chip(struct parisc_device *dev) lp =netdev_priv (netdevice); lp->options = dev->id.sversion ==0x72 ?OPT_SWAP_PORT :0 ; - lp->dma =dma_alloc_attrs (&dev->dev,sizeof (struct i596_dma ), - &lp->dma_addr,GFP_KERNEL , -DMA_ATTR_NON_CONSISTENT ); + lp->dma =dma_alloc_noncoherent (&dev->dev, +sizeof (struct i596_dma ), &lp->dma_addr, +DMA_BIDIRECTIONAL ,GFP_KERNEL );if (!lp->dma) goto out_free_netdev;
@@ -196,8 +189,8 @@ lan_init_chip(struct parisc_device *dev)return 0 ; out_free_dma: -dma_free_attrs (&dev->dev,sizeof (struct i596_dma ), lp->dma, - lp->dma_addr,DMA_ATTR_NON_CONSISTENT ); +dma_free_noncoherent (&dev->dev,sizeof (struct i596_dma ), + lp->dma, lp->dma_addr,DMA_BIDIRECTIONAL ); out_free_netdev:free_netdev (netdevice);return retval;
@@ -209,8 +202,8 @@ static int __exit lan_remove_chip(struct parisc_device *pdev)struct i596_private *lp =netdev_priv (dev);unregister_netdev (dev); -dma_free_attrs (&pdev->dev,sizeof (struct i596_private ), lp->dma, - lp->dma_addr,DMA_ATTR_NON_CONSISTENT ); +dma_free_noncoherent (&pdev->dev,sizeof (struct i596_private ), lp->dma, + lp->dma_addr,DMA_BIDIRECTIONAL );free_netdev (dev);return 0 ; }
drivers/net/ethernet/i825xx/lib82596.c +71 -43
317 lines not on this page · raw message
drivers/net/ethernet/i825xx/sni_82596.c +0 -4
@@ -24,10 +24,6 @@static const char sni_82596_string[] ="snirm_82596" ; -#define DMA_WBACK (priv, addr, len) do { } while (0) -#define DMA_INV (priv, addr, len) do { } while (0) -#define DMA_WBACK_INV (priv, addr, len) do { } while (0) -#define SYSBUS 0x00004400/* big endian CPU, 82596 little endian */
drivers/net/ethernet/seeq/sgiseeq.c +18 -10
@@ -112,14 +112,18 @@ struct sgiseeq_private {static inline void dma_sync_desc_cpu (struct net_device *dev,void *addr) { -dma_cache_sync (dev->dev.parent, addr,sizeof (struct sgiseeq_rx_desc ), -DMA_FROM_DEVICE ); +struct sgiseeq_private *sp =netdev_priv (dev); + +dma_sync_single_for_cpu (dev->dev.parent,VIRT_TO_DMA (sp, addr), +sizeof (struct sgiseeq_rx_desc ),DMA_BIDIRECTIONAL ); }static inline void dma_sync_desc_dev (struct net_device *dev,void *addr) { -dma_cache_sync (dev->dev.parent, addr,sizeof (struct sgiseeq_rx_desc ), -DMA_TO_DEVICE ); +struct sgiseeq_private *sp =netdev_priv (dev); + +dma_sync_single_for_device (dev->dev.parent,VIRT_TO_DMA (sp, addr), +sizeof (struct sgiseeq_rx_desc ),DMA_BIDIRECTIONAL ); }static inline void hpc3_eth_reset (struct hpc3_ethregs *hregs)
@@ -403,6 +407,8 @@ static inline void sgiseeq_rx(struct net_device *dev, struct sgiseeq_private *sp rd = &sp->rx_desc[sp->rx_new];dma_sync_desc_cpu (dev, rd); } +dma_sync_desc_dev (dev, rd); +dma_sync_desc_cpu (dev, &sp->rx_desc[orig_end]); sp->rx_desc[orig_end].rdma.cntinfo &= ~(HPCDMA_EOR );dma_sync_desc_dev (dev, &sp->rx_desc[orig_end]);
@@ -443,6 +449,7 @@ static inline void kick_tx(struct net_device *dev,dma_sync_desc_cpu (dev, td); }if (td->tdma.cntinfo &HPCDMA_XIU ) { +dma_sync_desc_dev (dev, td); hregs->tx_ndptr =VIRT_TO_DMA (sp, td); hregs->tx_ctrl =HPC3_ETXCTRL_ACTIVE ; }
@@ -476,6 +483,7 @@ static inline void sgiseeq_tx(struct net_device *dev, struct sgiseeq_private *spif (!(td->tdma.cntinfo & (HPCDMA_XIU )))break ;if (!(td->tdma.cntinfo & (HPCDMA_ETXD ))) { +dma_sync_desc_dev (dev, td);if (!(status &HPC3_ETXCTRL_ACTIVE )) { hregs->tx_ndptr =VIRT_TO_DMA (sp, td); hregs->tx_ctrl =HPC3_ETXCTRL_ACTIVE ;
@@ -740,8 +748,8 @@ static int sgiseeq_probe(struct platform_device *pdev) sp =netdev_priv (dev);/* Make private data page aligned */ - sr =dma_alloc_attrs (&pdev->dev,sizeof (*sp->srings), &sp->srings_dma, -GFP_KERNEL ,DMA_ATTR_NON_CONSISTENT ); + sr =dma_alloc_noncoherent (&pdev->dev,sizeof (*sp->srings), + &sp->srings_dma,DMA_BIDIRECTIONAL ,GFP_KERNEL );if (!sr) {printk (KERN_ERR "Sgiseeq: Page alloc failed, aborting.\n" ); err = -ENOMEM ;
@@ -802,8 +810,8 @@ static int sgiseeq_probe(struct platform_device *pdev)return 0 ; err_out_free_attrs: -dma_free_attrs (&pdev->dev,sizeof (*sp->srings), sp->srings, - sp->srings_dma,DMA_ATTR_NON_CONSISTENT ); +dma_free_noncoherent (&pdev->dev,sizeof (*sp->srings), sp->srings, + sp->srings_dma,DMA_BIDIRECTIONAL ); err_out_free_dev:free_netdev (dev);
@@ -817,8 +825,8 @@ static int sgiseeq_remove(struct platform_device *pdev)struct sgiseeq_private *sp =netdev_priv (dev);unregister_netdev (dev); -dma_free_attrs (&pdev->dev,sizeof (*sp->srings), sp->srings, - sp->srings_dma,DMA_ATTR_NON_CONSISTENT ); +dma_free_noncoherent (&pdev->dev,sizeof (*sp->srings), sp->srings, + sp->srings_dma,DMA_BIDIRECTIONAL );free_netdev (dev);return 0 ;
drivers/scsi/53c700.c +5 -4
@@ -300,8 +300,8 @@ NCR_700_detect(struct scsi_host_template *tpnt, memory =dma_alloc_coherent (dev,TOTAL_MEM_SIZE , &pScript,GFP_KERNEL );if (!memory) { hostdata->noncoherent =1 ; - memory =dma_alloc_attrs (dev,TOTAL_MEM_SIZE , &pScript, -GFP_KERNEL ,DMA_ATTR_NON_CONSISTENT ); + memory =dma_alloc_noncoherent (dev,TOTAL_MEM_SIZE , &pScript, +DMA_BIDIRECTIONAL ,GFP_KERNEL ); }if (!memory) {printk (KERN_ERR "53c700: Failed to allocate memory for driver, detaching\n" );
@@ -414,8 +414,9 @@ NCR_700_release(struct Scsi_Host *host) (struct NCR_700_Host_Parameters *)host->hostdata[0 ];if (hostdata->noncoherent) -dma_free_attrs (hostdata->dev,TOTAL_MEM_SIZE , hostdata->script, - hostdata->pScript,DMA_ATTR_NON_CONSISTENT ); +dma_free_noncoherent (hostdata->dev,TOTAL_MEM_SIZE , + hostdata->script, hostdata->pScript, +DMA_BIDIRECTIONAL );else dma_free_coherent (hostdata->dev,TOTAL_MEM_SIZE , hostdata->script, hostdata->pScript);
Signature · 1 line
2.28.0
Re: [PATCH 11/17] sgiseeq: convert to dma_alloc_noncoherent
Quoted 6 lines · Christoph Hellwig
drivers/net/ethernet/i825xx/lasi_82596.c | 25 ++--- drivers/net/ethernet/i825xx/lib82596.c | 114 ++++++++++++++--------- drivers/net/ethernet/i825xx/sni_82596.c | 4 - drivers/net/ethernet/seeq/sgiseeq.c | 28 ++++-- drivers/scsi/53c700.c | 9 +- 5 files changed, 103 insertions(+), 77 deletions(-)
I think your patch slicing-and-dicing went wrong here ;-(
Re: [PATCH 11/17] sgiseeq: convert to dma_alloc_noncoherent
Quoted 9 lines · Matthew Wilcox
Quoted 6 lines · Christoph Hellwig
drivers/net/ethernet/i825xx/lasi_82596.c | 25 ++--- drivers/net/ethernet/i825xx/lib82596.c | 114 ++++++++++++++--------- drivers/net/ethernet/i825xx/sni_82596.c | 4 - drivers/net/ethernet/seeq/sgiseeq.c | 28 ++++-- drivers/scsi/53c700.c | 9 +- 5 files changed, 103 insertions(+), 77 deletions(-)I think your patch slicing-and-dicing went wrong here ;-(
Pretty much.. Fixed up for the next version.
PATCH 12/17 53c700: convert to dma_alloc_noncoherent
In reply to Christoph HellwigUse the new non-coherent DMA API including proper ownership transfers.
--- drivers/scsi/53c700.c | 11 +++++++++-- drivers/scsi/53c700.h | 16 ++++++++-------- 2 files changed, 17 insertions(+), 10 deletions(-)
drivers/scsi/53c700.c +9 -2
@@ -269,18 +269,25 @@ NCR_700_get_SXFER(struct scsi_device *SDp) spi_period(SDp->sdev_target)); } +static inline dma_addr_t virt_to_dma (struct NCR_700_Host_Parameters *h,void *p) +{ +return h->pScript + ((uintptr_t )p - (uintptr_t )h->script); +} +static inline void dma_sync_to_dev (struct NCR_700_Host_Parameters *h,void *addr,size_t size) {if (h->noncoherent) -dma_cache_sync (h->dev, addr, size,DMA_TO_DEVICE ); +dma_sync_single_for_device (h->dev,virt_to_dma (h, addr), + size,DMA_BIDIRECTIONAL ); }static inline void dma_sync_from_dev (struct NCR_700_Host_Parameters *h,void *addr,size_t size) {if (h->noncoherent) -dma_cache_sync (h->dev, addr, size,DMA_FROM_DEVICE ); +dma_sync_single_for_device (h->dev,virt_to_dma (h, addr), size, +DMA_BIDIRECTIONAL ); } struct Scsi_Host *
drivers/scsi/53c700.h +8 -8
@@ -423,33 +423,33 @@ struct NCR_700_Host_Parameters {#define NCR_710_MIN_XFERP 0#define NCR_700_MIN_PERIOD 25/* for SDTR message, 100ns */ -#define script_patch_32 (dev, script, symbol, value) \ +#define script_patch_32 (h, script, symbol, value) \ { \ int i; \ dma_addr_t da = value; \ for(i=0; i< (sizeof(A_##symbol##_used) / sizeof(__u32)); i++) { \ __u32 val = bS_to_cpu((script)[A_##symbol##_used[i]]) + da; \ (script)[A_##symbol##_used[i]] = bS_to_host(val); \ - dma_sync_to_dev((dev), &(script)[A_##symbol##_used[i]], 4); \ + dma_sync_to_dev((h), &(script)[A_##symbol##_used[i]], 4); \ DEBUG((" script, patching %s at %d to %pad\n", \ #symbol, A_##symbol##_used[i], &da)); \ } \ } -#define script_patch_32_abs (dev, script, symbol, value) \ +#define script_patch_32_abs (h, script, symbol, value) \ { \ int i; \ dma_addr_t da = value; \ for(i=0; i< (sizeof(A_##symbol##_used) / sizeof(__u32)); i++) { \ (script)[A_##symbol##_used[i]] = bS_to_host(da); \ - dma_sync_to_dev((dev), &(script)[A_##symbol##_used[i]], 4); \ + dma_sync_to_dev((h), &(script)[A_##symbol##_used[i]], 4); \ DEBUG((" script, patching %s at %d to %pad\n", \ #symbol, A_##symbol##_used[i], &da)); \ } \ }/* Used for patching the SCSI ID in the SELECT instruction */ -#define script_patch_ID (dev, script, symbol, value) \ +#define script_patch_ID (h, script, symbol, value) \ { \ int i; \ for(i=0; i< (sizeof(A_##symbol##_used) / sizeof(__u32)); i++) { \
@@ -457,13 +457,13 @@ struct NCR_700_Host_Parameters { val &=0xff00ffff ; \ val |= ((value) &0xff ) <<16 ; \ (script)[A_##symbol##_used[i]] = bS_to_host(val); \ - dma_sync_to_dev((dev), &(script)[A_##symbol##_used[i]], 4); \ + dma_sync_to_dev((h), &(script)[A_##symbol##_used[i]], 4); \ DEBUG((" script, patching ID field %s at %d to 0x%x\n", \ #symbol, A_##symbol##_used[i], val)); \ } \ } -#define script_patch_16 (dev, script, symbol, value) \ +#define script_patch_16 (h, script, symbol, value) \ { \ int i; \ for(i=0; i< (sizeof(A_##symbol##_used) / sizeof(__u32)); i++) { \
@@ -471,7 +471,7 @@ struct NCR_700_Host_Parameters { val &=0xffff0000 ; \ val |= ((value) &0xffff ); \ (script)[A_##symbol##_used[i]] = bS_to_host(val); \ - dma_sync_to_dev((dev), &(script)[A_##symbol##_used[i]], 4); \ + dma_sync_to_dev((h), &(script)[A_##symbol##_used[i]], 4); \ DEBUG((" script, patching short field %s at %d to 0x%x\n", \ #symbol, A_##symbol##_used[i], val)); \ } \
Signature · 1 line
2.28.0
PATCH 13/17 dma-mapping: remove dma_cache_sync
In reply to Christoph HellwigAll users are gone now, remove the API.
--- arch/mips/Kconfig | 1 - arch/mips/jazz/jazzdma.c | 1 - arch/mips/mm/dma-noncoherent.c | 6 ------ arch/parisc/Kconfig | 1 - arch/parisc/kernel/pci-dma.c | 6 ------ include/linux/dma-mapping.h | 8 -------- include/linux/dma-noncoherent.h | 10 ---------- kernel/dma/Kconfig | 3 --- kernel/dma/mapping.c | 14 -------------- 9 files changed, 50 deletions(-)
arch/mips/Kconfig +0 -1
@@ -1134,7 +1134,6 @@ config DMA_NONCOHERENT select ARCH_HAS_SYNC_DMA_FOR_DEVICE select ARCH_HAS_DMA_SET_UNCACHED select DMA_NONCOHERENT_MMAP - select DMA_NONCOHERENT_CACHE_SYNC select NEED_DMA_MAP_STATE config SYS_HAS_EARLY_PRINTK
arch/mips/jazz/jazzdma.c +0 -1
@@ -620,7 +620,6 @@ const struct dma_map_ops jazz_dma_ops = { .sync_single_for_device = jazz_dma_sync_single_for_device, .sync_sg_for_cpu = jazz_dma_sync_sg_for_cpu, .sync_sg_for_device = jazz_dma_sync_sg_for_device, - .cache_sync = arch_dma_cache_sync, .mmap = dma_common_mmap, .get_sgtable = dma_common_get_sgtable, };
arch/mips/mm/dma-noncoherent.c +0 -6
@@ -137,12 +137,6 @@ void arch_sync_dma_for_cpu(phys_addr_t paddr, size_t size, }#endif -void arch_dma_cache_sync (struct device *dev,void *vaddr,size_t size, -enum dma_data_direction direction) -{ -dma_sync_virt_for_device (vaddr, size, direction); -} -#ifdef CONFIG_DMA_PERDEV_COHERENT void arch_setup_dma_ops (struct device *dev,u64 dma_base,u64 size,const struct iommu_ops *iommu,bool coherent)
arch/parisc/Kconfig +0 -1
@@ -195,7 +195,6 @@ config PA11 depends on PA7000 || PA7100LC || PA7200 || PA7300LC select ARCH_HAS_SYNC_DMA_FOR_CPU select ARCH_HAS_SYNC_DMA_FOR_DEVICE - select DMA_NONCOHERENT_CACHE_SYNC config PREFETCH def_bool y
arch/parisc/kernel/pci-dma.c +0 -6
@@ -454,9 +454,3 @@ void arch_sync_dma_for_cpu(phys_addr_t paddr, size_t size, {flush_kernel_dcache_range ((unsigned long )phys_to_virt (paddr), size); } - -void arch_dma_cache_sync (struct device *dev,void *vaddr,size_t size, -enum dma_data_direction direction) -{ -flush_kernel_dcache_range ((unsigned long )vaddr, size); -}
include/linux/dma-mapping.h +0 -8
@@ -123,8 +123,6 @@ struct dma_map_ops {void (*sync_sg_for_device)(struct device *dev,struct scatterlist *sg,int nents,enum dma_data_direction dir); -void (*cache_sync)(struct device *dev,void *vaddr,size_t size, -enum dma_data_direction direction);int (*dma_supported)(struct device *dev,u64 mask);u64 (*get_required_mask)(struct device *dev);size_t (*max_mapping_size)(struct device *dev);
@@ -254,8 +252,6 @@ void *dmam_alloc_attrs(struct device *dev, size_t size, dma_addr_t *dma_handle, gfp_t gfp, unsigned long attrs);void dmam_free_coherent (struct device *dev,size_t size,void *vaddr,dma_addr_t dma_handle); -void dma_cache_sync (struct device *dev,void *vaddr,size_t size, -enum dma_data_direction dir);int dma_get_sgtable_attrs (struct device *dev,struct sg_table *sgt,void *cpu_addr,dma_addr_t dma_addr,size_t size,unsigned long attrs);
@@ -339,10 +335,6 @@ static inline void dmam_free_coherent(struct device *dev, size_t size, void *vaddr, dma_addr_t dma_handle) { } -static inline void dma_cache_sync (struct device *dev,void *vaddr,size_t size, -enum dma_data_direction dir) -{ -}static inline int dma_get_sgtable_attrs (struct device *dev,struct sg_table *sgt,void *cpu_addr,dma_addr_t dma_addr,size_t size,unsigned long attrs)
include/linux/dma-noncoherent.h +0 -10
@@ -62,16 +62,6 @@ static inline pgprot_t dma_pgprot(struct device *dev, pgprot_t prot, }#endif /* CONFIG_MMU */ -#ifdef CONFIG_DMA_NONCOHERENT_CACHE_SYNC -void arch_dma_cache_sync (struct device *dev,void *vaddr,size_t size, -enum dma_data_direction direction); -#else -static inline void arch_dma_cache_sync (struct device *dev,void *vaddr, -size_t size,enum dma_data_direction direction) -{ -} -#endif /* CONFIG_DMA_NONCOHERENT_CACHE_SYNC */ -#ifdef CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE void arch_sync_dma_for_device (phys_addr_t paddr,size_t size,enum dma_data_direction dir);
kernel/dma/Kconfig +0 -3
@@ -75,9 +75,6 @@ config ARCH_HAS_DMA_PREP_COHERENT config ARCH_HAS_FORCE_DMA_UNENCRYPTED bool -config DMA_NONCOHERENT_CACHE_SYNC - bool - config DMA_VIRT_OPS bool depends on HAS_DMA
kernel/dma/mapping.c +0 -14
@@ -518,20 +518,6 @@ int dma_set_coherent_mask(struct device *dev, u64 mask)EXPORT_SYMBOL (dma_set_coherent_mask);#endif -void dma_cache_sync (struct device *dev,void *vaddr,size_t size, -enum dma_data_direction dir) -{ -const struct dma_map_ops *ops =get_dma_ops (dev); - -BUG_ON (!valid_dma_direction (dir)); - -if (dma_alloc_direct (dev, ops)) -arch_dma_cache_sync (dev, vaddr, size, dir); -else if (ops->cache_sync) - ops->cache_sync (dev, vaddr, size, dir); -} -EXPORT_SYMBOL (dma_cache_sync); -size_t dma_max_mapping_size (struct device *dev) {const struct dma_map_ops *ops =get_dma_ops (dev);
Signature · 1 line
2.28.0
PATCH 14/17 dma-mapping: add a new dma_alloc_pages API
In reply to Christoph HellwigThis API is the equivalent of alloc_pages, except that the returned memory is guaranteed to be DMA addressable by the passed in device. The implementation will also be used to provide a more sensible replacement for DMA_ATTR_NON_CONSISTENT flag.
Additionally dma_alloc_noncoherent is switched over to use dma_alloc_pages as its backend.
--- Documentation/core-api/dma-attributes.rst | 8 --- arch/alpha/kernel/pci_iommu.c | 2 + arch/arm/mm/dma-mapping-nommu.c | 2 + arch/arm/mm/dma-mapping.c | 4 ++ arch/ia64/hp/common/sba_iommu.c | 2 + arch/mips/jazz/jazzdma.c | 7 +-- arch/powerpc/kernel/dma-iommu.c | 2 + arch/powerpc/platforms/ps3/system-bus.c | 4 ++ arch/powerpc/platforms/pseries/vio.c | 2 + arch/s390/pci/pci_dma.c | 2 + arch/x86/kernel/amd_gart_64.c | 2 + drivers/iommu/dma-iommu.c | 2 + drivers/iommu/intel/iommu.c | 4 ++ drivers/parisc/ccio-dma.c | 2 + drivers/parisc/sba_iommu.c | 2 + drivers/xen/swiotlb-xen.c | 2 + include/linux/dma-direct.h | 5 ++ include/linux/dma-mapping.h | 34 ++++++------ include/linux/dma-noncoherent.h | 3 -- kernel/dma/direct.c | 52 ++++++++++++++++++- kernel/dma/mapping.c | 63 +++++++++++++++++++++-- kernel/dma/ops_helpers.c | 35 +++++++++++++ kernel/dma/virt.c | 2 + 23 files changed, 206 insertions(+), 37 deletions(-)
Documentation/core-api/dma-attributes.rst +0 -8
@@ -25,14 +25,6 @@ Since it is optional for platforms to implement DMA_ATTR_WRITE_COMBINE, those that do not will simply ignore the attribute and exhibit default behavior. -DMA_ATTR_NON_CONSISTENT ------------------------ - -DMA_ATTR_NON_CONSISTENT lets the platform to choose to return either -consistent or non-consistent memory as it sees fit. By using this API, -you are guaranteeing to the platform that you have all the correct and -necessary sync points for this memory in the driver. - DMA_ATTR_NO_KERNEL_MAPPING --------------------------
arch/alpha/kernel/pci_iommu.c +2 -0
@@ -952,5 +952,7 @@ const struct dma_map_ops alpha_pci_ops = { .dma_supported = alpha_pci_supported, .mmap = dma_common_mmap, .get_sgtable = dma_common_get_sgtable, + .alloc_pages = dma_common_alloc_pages, + .free_pages = dma_common_free_pages, };EXPORT_SYMBOL (alpha_pci_ops);
arch/arm/mm/dma-mapping-nommu.c +2 -0
@@ -176,6 +176,8 @@ static void arm_nommu_dma_sync_sg_for_cpu(struct device *dev, struct scatterlistconst struct dma_map_ops arm_nommu_dma_ops = { .alloc = arm_nommu_dma_alloc, .free = arm_nommu_dma_free, + .alloc_pages = dma_direct_alloc_pages, + .free_pages = dma_direct_free_pages, .mmap = arm_nommu_dma_mmap, .map_page = arm_nommu_dma_map_page, .unmap_page = arm_nommu_dma_unmap_page,
arch/arm/mm/dma-mapping.c +4 -0
@@ -199,6 +199,8 @@ static int arm_dma_supported(struct device *dev, u64 mask)const struct dma_map_ops arm_dma_ops = { .alloc = arm_dma_alloc, .free = arm_dma_free, + .alloc_pages = dma_direct_alloc_pages, + .free_pages = dma_direct_free_pages, .mmap = arm_dma_mmap, .get_sgtable = arm_dma_get_sgtable, .map_page = arm_dma_map_page,
@@ -226,6 +228,8 @@ static int arm_coherent_dma_mmap(struct device *dev, struct vm_area_struct *vma,const struct dma_map_ops arm_coherent_dma_ops = { .alloc = arm_coherent_dma_alloc, .free = arm_coherent_dma_free, + .alloc_pages = dma_direct_alloc_pages, + .free_pages = dma_direct_free_pages, .mmap = arm_coherent_dma_mmap, .get_sgtable = arm_dma_get_sgtable, .map_page = arm_coherent_dma_map_page,
arch/ia64/hp/common/sba_iommu.c +2 -0
@@ -2070,6 +2070,8 @@ static const struct dma_map_ops sba_dma_ops = { .dma_supported = sba_dma_supported, .mmap = dma_common_mmap, .get_sgtable = dma_common_get_sgtable, + .alloc_pages = dma_common_alloc_pages, + .free_pages = dma_common_free_pages, }; static int __init
arch/mips/jazz/jazzdma.c +2 -5
@@ -506,9 +506,6 @@ static void *jazz_dma_alloc(struct device *dev, size_t size, *dma_handle =vdma_alloc (virt_to_phys (ret), size);if (*dma_handle ==DMA_MAPPING_ERROR ) goto out_free_pages; - -if (attrs &DMA_ATTR_NON_CONSISTENT ) -return ret;arch_dma_prep_coherent (page, size);return (void *)(UNCAC_BASE +__pa (ret));
@@ -521,8 +518,6 @@ static void jazz_dma_free(struct device *dev, size_t size, void *vaddr, dma_addr_t dma_handle, unsigned long attrs) {vdma_free (dma_handle ); -if (!(attrs &DMA_ATTR_NON_CONSISTENT )) - vaddr =__va (vaddr -UNCAC_BASE );__free_pages (virt_to_page (vaddr),get_order (size)); }
@@ -622,5 +617,7 @@ const struct dma_map_ops jazz_dma_ops = { .sync_sg_for_device = jazz_dma_sync_sg_for_device, .mmap = dma_common_mmap, .get_sgtable = dma_common_get_sgtable, + .alloc_pages = dma_common_alloc_pages, + .free_pages = dma_common_free_pages, };EXPORT_SYMBOL (jazz_dma_ops);
arch/powerpc/kernel/dma-iommu.c +2 -0
@@ -137,4 +137,6 @@ const struct dma_map_ops dma_iommu_ops = { .get_required_mask = dma_iommu_get_required_mask, .mmap = dma_common_mmap, .get_sgtable = dma_common_get_sgtable, + .alloc_pages = dma_common_alloc_pages, + .free_pages = dma_common_free_pages, };
arch/powerpc/platforms/ps3/system-bus.c +4 -0
@@ -696,6 +696,8 @@ static const struct dma_map_ops ps3_sb_dma_ops = { .unmap_page = ps3_unmap_page, .mmap = dma_common_mmap, .get_sgtable = dma_common_get_sgtable, + .alloc_pages = dma_common_alloc_pages, + .free_pages = dma_common_free_pages, };static const struct dma_map_ops ps3_ioc0_dma_ops = {
@@ -708,6 +710,8 @@ static const struct dma_map_ops ps3_ioc0_dma_ops = { .unmap_page = ps3_unmap_page, .mmap = dma_common_mmap, .get_sgtable = dma_common_get_sgtable, + .alloc_pages = dma_common_alloc_pages, + .free_pages = dma_common_free_pages, };/**
arch/powerpc/platforms/pseries/vio.c +2 -0
@@ -608,6 +608,8 @@ static const struct dma_map_ops vio_dma_mapping_ops = { .get_required_mask = dma_iommu_get_required_mask, .mmap = dma_common_mmap, .get_sgtable = dma_common_get_sgtable, + .alloc_pages = dma_common_alloc_pages, + .free_pages = dma_common_free_pages, };/**
arch/s390/pci/pci_dma.c +2 -0
@@ -668,6 +668,8 @@ const struct dma_map_ops s390_pci_dma_ops = { .unmap_page = s390_dma_unmap_pages, .mmap = dma_common_mmap, .get_sgtable = dma_common_get_sgtable, + .alloc_pages = dma_common_alloc_pages, + .free_pages = dma_common_free_pages,/* dma_supported is unconditionally true without a callback */ };EXPORT_SYMBOL_GPL (s390_pci_dma_ops);
arch/x86/kernel/amd_gart_64.c +2 -0
@@ -677,6 +677,8 @@ static const struct dma_map_ops gart_dma_ops = { .get_sgtable = dma_common_get_sgtable, .dma_supported = dma_direct_supported, .get_required_mask = dma_direct_get_required_mask, + .alloc_pages = dma_direct_alloc_pages, + .free_pages = dma_direct_free_pages, };static void gart_iommu_shutdown (void )
drivers/iommu/dma-iommu.c +2 -0
@@ -1120,6 +1120,8 @@ static unsigned long iommu_dma_get_merge_boundary(struct device *dev)static const struct dma_map_ops iommu_dma_ops = { .alloc = iommu_dma_alloc, .free = iommu_dma_free, + .alloc_pages = dma_common_alloc_pages, + .free_pages = dma_common_free_pages, .mmap = iommu_dma_mmap, .get_sgtable = iommu_dma_get_sgtable, .map_page = iommu_dma_map_page,
drivers/iommu/intel/iommu.c +4 -0
@@ -3669,6 +3669,8 @@ static const struct dma_map_ops intel_dma_ops = { .dma_supported = dma_direct_supported, .mmap = dma_common_mmap, .get_sgtable = dma_common_get_sgtable, + .alloc_pages = dma_common_alloc_pages, + .free_pages = dma_common_free_pages, .get_required_mask = intel_get_required_mask, };
@@ -3922,6 +3924,8 @@ static const struct dma_map_ops bounce_dma_ops = { .sync_sg_for_device = bounce_sync_sg_for_device, .map_resource = bounce_map_resource, .unmap_resource = bounce_unmap_resource, + .alloc_pages = dma_common_alloc_pages, + .free_pages = dma_common_free_pages, .dma_supported = dma_direct_supported, };
drivers/parisc/ccio-dma.c +2 -0
@@ -1024,6 +1024,8 @@ static const struct dma_map_ops ccio_ops = { .map_sg = ccio_map_sg, .unmap_sg = ccio_unmap_sg, .get_sgtable = dma_common_get_sgtable, + .alloc_pages = dma_common_alloc_pages, + .free_pages = dma_common_free_pages, };#ifdef CONFIG_PROC_FS
drivers/parisc/sba_iommu.c +2 -0
@@ -1076,6 +1076,8 @@ static const struct dma_map_ops sba_ops = { .map_sg = sba_map_sg, .unmap_sg = sba_unmap_sg, .get_sgtable = dma_common_get_sgtable, + .alloc_pages = dma_common_alloc_pages, + .free_pages = dma_common_free_pages, };
drivers/xen/swiotlb-xen.c +2 -0
@@ -578,4 +578,6 @@ const struct dma_map_ops xen_swiotlb_dma_ops = { .dma_supported = xen_swiotlb_dma_supported, .mmap = dma_common_mmap, .get_sgtable = dma_common_get_sgtable, + .alloc_pages = dma_common_alloc_pages, + .free_pages = dma_common_free_pages, };
include/linux/dma-direct.h +5 -0
@@ -77,6 +77,11 @@ void *dma_direct_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle, gfp_t gfp, unsigned long attrs);void dma_direct_free (struct device *dev,size_t size,void *cpu_addr,dma_addr_t dma_addr,unsigned long attrs); +struct page *dma_direct_alloc_pages (struct device *dev,size_t size, +dma_addr_t *dma_handle,enum dma_data_direction dir,gfp_t gfp); +void dma_direct_free_pages (struct device *dev,size_t size, +struct page *page,dma_addr_t dma_addr, +enum dma_data_direction dir);int dma_direct_get_sgtable (struct device *dev,struct sg_table *sgt,void *cpu_addr,dma_addr_t dma_addr,size_t size,unsigned long attrs);
include/linux/dma-mapping.h +17 -17
@@ -27,11 +27,6 @@* buffered to improve performance. */ #define DMA_ATTR_WRITE_COMBINE (1UL << 2) -/* -* DMA_ATTR_NON_CONSISTENT: Lets the platform to choose to return either -* consistent or non-consistent memory as it sees fit. -*/ -#define DMA_ATTR_NON_CONSISTENT (1UL << 3)/* * DMA_ATTR_NO_KERNEL_MAPPING: Lets the platform to avoid creating a kernel * virtual mapping for the allocated buffer.
@@ -80,6 +75,11 @@ struct dma_map_ops {void (*free)(struct device *dev,size_t size,void *vaddr,dma_addr_t dma_handle,unsigned long attrs); +struct page *(*alloc_pages)(struct device *dev,size_t size, +dma_addr_t *dma_handle,enum dma_data_direction dir, +gfp_t gfp); +void (*free_pages)(struct device *dev,size_t size,struct page *vaddr, +dma_addr_t dma_handle,enum dma_data_direction dir);int (*mmap)(struct device *,struct vm_area_struct *,void *,dma_addr_t ,size_t ,unsigned long attrs);
@@ -381,17 +381,14 @@ static inline unsigned long dma_get_merge_boundary(struct device *dev) }#endif /* CONFIG_HAS_DMA */ -static inline void *dma_alloc_noncoherent (struct device *dev,size_t size, -dma_addr_t *dma_handle,enum dma_data_direction dir,gfp_t gfp) -{ -return dma_alloc_attrs (dev, size, dma_handle, gfp, -DMA_ATTR_NON_CONSISTENT ); -} -static inline void dma_free_noncoherent (struct device *dev,size_t size, -void *vaddr,dma_addr_t dma_handle,enum dma_data_direction dir) -{ -dma_free_attrs (dev, size, vaddr, dma_handle,DMA_ATTR_NON_CONSISTENT ); -} +struct page *dma_alloc_pages (struct device *dev,size_t size, +dma_addr_t *dma_handle,enum dma_data_direction dir,gfp_t gfp); +void dma_free_pages (struct device *dev,size_t size,struct page *page, +dma_addr_t dma_handle,enum dma_data_direction dir); +void *dma_alloc_noncoherent (struct device *dev,size_t size, +dma_addr_t *dma_handle,enum dma_data_direction dir,gfp_t gfp); +void dma_free_noncoherent (struct device *dev,size_t size,void *vaddr, +dma_addr_t dma_handle,enum dma_data_direction dir);static inline dma_addr_t dma_map_single_attrs (struct device *dev,void *ptr,size_t size,enum dma_data_direction dir,unsigned long attrs)
@@ -517,7 +514,10 @@ static inline void dma_sync_sgtable_for_device(struct device *dev,extern int dma_common_mmap (struct device *dev,struct vm_area_struct *vma,void *cpu_addr,dma_addr_t dma_addr,size_t size,unsigned long attrs); - +struct page *dma_common_alloc_pages (struct device *dev,size_t size, +dma_addr_t *dma_handle,enum dma_data_direction dir,gfp_t gfp); +void dma_common_free_pages (struct device *dev,size_t size,struct page *vaddr, +dma_addr_t dma_handle,enum dma_data_direction dir);struct page **dma_common_find_pages (void *cpu_addr);void *dma_common_contiguous_remap (struct page *page,size_t size,pgprot_t prot,const void *caller);
include/linux/dma-noncoherent.h +0 -3
@@ -31,9 +31,6 @@ static __always_inline bool dma_alloc_need_uncached(struct device *dev,return false;if (attrs &DMA_ATTR_NO_KERNEL_MAPPING )return false; -if (IS_ENABLED (CONFIG_DMA_NONCOHERENT_CACHE_SYNC ) && - (attrs &DMA_ATTR_NON_CONSISTENT )) -return false;return true; }
kernel/dma/direct.c +51 -1
63 lines not on this page · raw message
kernel/dma/mapping.c +60 -3
75 lines not on this page · raw message
kernel/dma/ops_helpers.c +35 -0
@@ -3,6 +3,7 @@* Helpers for DMA ops implementations. These generally rely on the fact that * the allocated memory contains normal pages in the direct kernel mapping. */ +#include <linux/dma-contiguous.h> #include <linux/dma-noncoherent.h> /*
@@ -49,3 +50,37 @@ int dma_common_mmap(struct device *dev, struct vm_area_struct *vma,return -ENXIO ;#endif /* CONFIG_MMU */ } + +struct page *dma_common_alloc_pages (struct device *dev,size_t size, +dma_addr_t *dma_handle,enum dma_data_direction dir,gfp_t gfp) +{ +const struct dma_map_ops *ops =get_dma_ops (dev); +struct page *page; + + page =dma_alloc_contiguous (dev, size, gfp); +if (!page) + page =alloc_pages_node (dev_to_node (dev), gfp,get_order (size)); +if (!page) +return NULL ; + + *dma_handle = ops->map_page (dev, page,0 , size, dir, +DMA_ATTR_SKIP_CPU_SYNC ); +if (*dma_handle ==DMA_MAPPING_ERROR ) { +dma_free_contiguous (dev, page, size); +return NULL ; + } + +memset (page_address (page),0 , size); +return page; +} + +void dma_common_free_pages (struct device *dev,size_t size,struct page *page, +dma_addr_t dma_handle,enum dma_data_direction dir) +{ +const struct dma_map_ops *ops =get_dma_ops (dev); + +if (ops->unmap_page) + ops->unmap_page (dev, dma_handle, size, dir, +DMA_ATTR_SKIP_CPU_SYNC ); +dma_free_contiguous (dev, page, size); +}
kernel/dma/virt.c +2 -0
@@ -55,5 +55,7 @@ const struct dma_map_ops dma_virt_ops = { .free = dma_virt_free, .map_page = dma_virt_map_page, .map_sg = dma_virt_map_sg, + .alloc_pages = dma_common_alloc_pages, + .free_pages = dma_common_free_pages, };EXPORT_SYMBOL (dma_virt_ops);
Signature · 1 line
2.28.0
PATCH 15/17 dma-mapping: add new {alloc,free}_noncoherent dma_map_ops methods
In reply to Christoph HellwigThis will allow IOMMU drivers to allocate non-contigous memory and return a vmapped virtual address.
--- include/linux/dma-mapping.h | 5 +++++ kernel/dma/mapping.c | 33 +++++++++++++++++++++++++++------ 2 files changed, 32 insertions(+), 6 deletions(-)
include/linux/dma-mapping.h +5 -0
@@ -80,6 +80,11 @@ struct dma_map_ops { gfp_t gfp);void (*free_pages)(struct device *dev,size_t size,struct page *vaddr,dma_addr_t dma_handle,enum dma_data_direction dir); +void * (*alloc_noncoherent)(struct device *dev,size_t size, +dma_addr_t *dma_handle,enum dma_data_direction dir, +gfp_t gfp); +void (*free_noncoherent)(struct device *dev,size_t size,void *vaddr, +dma_addr_t dma_handle,enum dma_data_direction dir);int (*mmap)(struct device *,struct vm_area_struct *,void *,dma_addr_t ,size_t ,unsigned long attrs);
kernel/dma/mapping.c +27 -6
@@ -502,19 +502,40 @@ EXPORT_SYMBOL_GPL(dma_free_pages);void *dma_alloc_noncoherent (struct device *dev,size_t size,dma_addr_t *dma_handle,enum dma_data_direction dir,gfp_t gfp) { -struct page *page; +const struct dma_map_ops *ops =get_dma_ops (dev); +void *vaddr; - page =dma_alloc_pages (dev, size, dma_handle, dir, gfp); -if (!page) -return NULL ; -return page_address (page); +if (!ops || !ops->alloc_noncoherent) { +struct page *page; + + page =dma_alloc_pages (dev, size, dma_handle, dir, gfp); +if (!page) +return NULL ; +return page_address (page); + } + + size =PAGE_ALIGN (size); + vaddr = ops->alloc_noncoherent (dev, size, dma_handle, dir, gfp); +if (vaddr) +debug_dma_map_page (dev,virt_to_page (vaddr),0 , size, dir, + *dma_handle); +return vaddr; }EXPORT_SYMBOL_GPL (dma_alloc_noncoherent);void dma_free_noncoherent (struct device *dev,size_t size,void *vaddr,dma_addr_t dma_handle,enum dma_data_direction dir) { -dma_free_pages (dev, size,virt_to_page (vaddr), dma_handle, dir); +const struct dma_map_ops *ops =get_dma_ops (dev); + +if (!ops || !ops->free_noncoherent) { +dma_free_pages (dev, size,virt_to_page (vaddr), dma_handle, dir); +return ; + } + + size =PAGE_ALIGN (size); +debug_dma_unmap_page (dev, dma_handle, size, dir); + ops->free_noncoherent (dev, size, vaddr, dma_handle, dir); }EXPORT_SYMBOL_GPL (dma_free_noncoherent);
Signature · 1 line
2.28.0
PATCH 16/17 dma-iommu: implement ->alloc_noncoherent
In reply to Christoph HellwigImplement the alloc_noncoherent method to provide memory that is neither coherent not contiguous.
--- drivers/iommu/dma-iommu.c | 41 +++++++++++++++++++++++++++++++++++---- 1 file changed, 37 insertions(+), 4 deletions(-)
drivers/iommu/dma-iommu.c +37 -4
@@ -572,6 +572,7 @@ static struct page **__iommu_dma_alloc_pages(struct device *dev,* @size: Size of buffer in bytes * @dma_handle: Out argument for allocated DMA handle * @gfp: Allocation flags +* @prot: pgprot_t to use for the remapped mapping * @attrs: DMA attributes for this allocation * * If @size is less than PAGE_SIZE, then a full CPU page will be allocated,
@@ -580,14 +581,14 @@ static struct page **__iommu_dma_alloc_pages(struct device *dev,* Return: Mapped virtual address, or NULL on failure. */ static void *iommu_dma_alloc_remap (struct device *dev,size_t size, -dma_addr_t *dma_handle,gfp_t gfp,unsigned long attrs) +dma_addr_t *dma_handle,gfp_t gfp,pgprot_t prot, +unsigned long attrs) {struct iommu_domain *domain =iommu_get_dma_domain (dev);struct iommu_dma_cookie *cookie = domain->iova_cookie;struct iova_domain *iovad = &cookie->iovad;bool coherent =dev_is_dma_coherent (dev);int ioprot =dma_info_to_prot (DMA_BIDIRECTIONAL , coherent, attrs); -pgprot_t prot =dma_pgprot (dev,PAGE_KERNEL , attrs);unsigned int count, min_size, alloc_sizes = domain->pgsize_bitmap;struct page **pages;struct sg_table sgt;
@@ -1030,8 +1031,10 @@ static void *iommu_dma_alloc(struct device *dev, size_t size, gfp |= __GFP_ZERO;if (IS_ENABLED (CONFIG_DMA_REMAP ) &&gfpflags_allow_blocking (gfp) && - !(attrs &DMA_ATTR_FORCE_CONTIGUOUS )) -return iommu_dma_alloc_remap (dev, size, handle, gfp, attrs); + !(attrs &DMA_ATTR_FORCE_CONTIGUOUS )) { +return iommu_dma_alloc_remap (dev, size, handle, gfp, +dma_pgprot (dev,PAGE_KERNEL , attrs), attrs); + }if (IS_ENABLED (CONFIG_DMA_DIRECT_REMAP ) && !gfpflags_allow_blocking (gfp) && !coherent)
@@ -1052,6 +1055,34 @@ static void *iommu_dma_alloc(struct device *dev, size_t size,return cpu_addr; } +#ifdef CONFIG_DMA_REMAP +static void *iommu_dma_alloc_noncoherent (struct device *dev,size_t size, +dma_addr_t *handle,enum dma_data_direction dir,gfp_t gfp) +{ +if (!gfpflags_allow_blocking (gfp)) { +struct page *page; + + page =dma_common_alloc_pages (dev, size, handle, dir, gfp); +if (!page) +return NULL ; +return page_address (page); + } + +return iommu_dma_alloc_remap (dev, size, handle, gfp | __GFP_ZERO, +PAGE_KERNEL ,0 ); +} + +static void iommu_dma_free_noncoherent (struct device *dev,size_t size, +void *cpu_addr,dma_addr_t handle,enum dma_data_direction dir) +{ +__iommu_dma_unmap (dev, handle, size); +__iommu_dma_free (dev, size, cpu_addr); +} +#else +#define iommu_dma_alloc_noncoherent NULL +#define iommu_dma_free_noncoherent NULL +#endif /* CONFIG_DMA_REMAP */ +static int iommu_dma_mmap (struct device *dev,struct vm_area_struct *vma,void *cpu_addr,dma_addr_t dma_addr,size_t size,unsigned long attrs)
@@ -1122,6 +1153,8 @@ static const struct dma_map_ops iommu_dma_ops = { .free = iommu_dma_free, .alloc_pages = dma_common_alloc_pages, .free_pages = dma_common_free_pages, + .alloc_noncoherent = iommu_dma_alloc_noncoherent, + .free_noncoherent = iommu_dma_free_noncoherent, .mmap = iommu_dma_mmap, .get_sgtable = iommu_dma_get_sgtable, .map_page = iommu_dma_map_page,
Signature · 1 line
2.28.0
PATCH 17/17 firewire-ohci: use dma_alloc_pages
In reply to Christoph HellwigUse dma_alloc_pages to allocate DMAable pages instead of hoping that the architecture either has GFP_DMA32 or not more than 4G of memory.
--- drivers/firewire/ohci.c | 26 +++++++++++--------------- 1 file changed, 11 insertions(+), 15 deletions(-)
drivers/firewire/ohci.c +11 -15
@@ -674,17 +674,16 @@ static void ar_context_link_page(struct ar_context *ctx, unsigned int index)static void ar_context_release (struct ar_context *ctx) { +struct device *dev = ctx->ohci->card.device;unsigned int i;vunmap (ctx->buffer); -for (i =0 ; i <AR_BUFFERS ; i++) -if (ctx->pages[i]) { -dma_unmap_page (ctx->ohci->card.device, -ar_buffer_bus (ctx, i), -PAGE_SIZE ,DMA_FROM_DEVICE ); -__free_page (ctx->pages[i]); - } +for (i =0 ; i <AR_BUFFERS ; i++) { +if (ctx->pages[i]) +dma_free_pages (dev,PAGE_SIZE , ctx->pages[i], +ar_buffer_bus (ctx, i),DMA_FROM_DEVICE ); + } }static void ar_context_abort (struct ar_context *ctx,const char *error_msg)
@@ -970,6 +969,7 @@ static void ar_context_tasklet(unsigned long data)static int ar_context_init (struct ar_context *ctx,struct fw_ohci *ohci,unsigned int descriptors_offset,u32 regs) { +struct device *dev = ohci->card.device;unsigned int i;dma_addr_t dma_addr;struct page *pages[AR_BUFFERS +AR_WRAPAROUND_PAGES ];
@@ -980,17 +980,13 @@ static int ar_context_init(struct ar_context *ctx, struct fw_ohci *ohci,tasklet_init (&ctx->tasklet, ar_context_tasklet, (unsigned long )ctx);for (i =0 ; i <AR_BUFFERS ; i++) { - ctx->pages[i] =alloc_page (GFP_KERNEL |GFP_DMA32 ); + ctx->pages[i] =dma_alloc_pages (dev,PAGE_SIZE , &dma_addr, +DMA_FROM_DEVICE ,GFP_KERNEL );if (!ctx->pages[i]) goto out_of_memory; - dma_addr =dma_map_page (ohci->card.device, ctx->pages[i], -0 ,PAGE_SIZE ,DMA_FROM_DEVICE ); -if (dma_mapping_error (ohci->card.device, dma_addr)) { -__free_page (ctx->pages[i]); - ctx->pages[i] =NULL ; - goto out_of_memory; - }set_page_private (ctx->pages[i], dma_addr); +dma_sync_single_for_device (dev, dma_addr,PAGE_SIZE , +DMA_FROM_DEVICE ); }for (i =0 ; i <AR_BUFFERS ; i++)
Signature · 1 line
2.28.0
Quoted 3 lines · Christoph Hellwig
I'm still a little unsure about the API naming, as
alloc_pagessort of implies a struct page return value, but we return a kernel virtual address.
Erm ... dma_alloc_pages() returns a struct page, so is this sentence stale?
From patch 14:
+struct page *dma_alloc_pages(struct device *dev, size_t size,
+ dma_addr_t *dma_handle, enum dma_data_direction dir, gfp_t gfp);
Quoted 3 lines
The other alternative would be to name the API
dma_alloc_noncoherent, but the whole non-coherent naming seems to put people off.
You say that like it's a bad thing. I think the problem is more that people don't understand what non-coherent means and think they're supporting it when they're not.
dma_alloc_manual_flushing()?
Quoted 6 lines
As a follow up I plan to move the implementation of the
DMA_ATTR_NO_KERNEL_MAPPINGflag over to this framework as well, given that is also is a fundamentally non coherent allocation. The replacement for that flag would then return a struct page, as it is allowed to actually return pages without a kernel mapping as the name suggested (although most of the time they will actually have a kernel mapping..)
If the page doesn't have a kernel mapping, shouldn't it return a PFN or a phys_addr?
Quoted 7 lines · Matthew Wilcox
Quoted 3 lines · Christoph Hellwig
I'm still a little unsure about the API naming, as
alloc_pagessort of implies a struct page return value, but we return a kernel virtual address.Erm ...
dma_alloc_pages()returns a struct page, so is this sentence stale?
Yes.
Quoted 5 lines
You say that like it's a bad thing. I think the problem is more that people don't understand what non-coherent means and think they're supporting it when they're not.
dma_alloc_manual_flushing()?
That sounds pretty awkward..
Quoted 10 lines
Quoted 7 lines
As a follow up I plan to move the implementation of the
DMA_ATTR_NO_KERNEL_MAPPINGflag over to this framework as well, given that is also is a fundamentally non coherent allocation. The replacement for that flag would then return a struct page, as it is allowed to actually return pages without a kernel mapping as the name suggested (although most of the time they will actually have a kernel mapping..)If the page doesn't have a kernel mapping, shouldn't it return a PFN or a
phys_addr?
Most APIs we'll feed it into need a struct page. The difference is just that it can be a highmem page. And if we want to get fancy we could change the kernel mapping to PROT_NONE eventually.