DMA-BUF Heaps (destaging ION)
Subsystems (3)
v2 cover DMA-BUF Heaps (destaging ION)
Here is a initial RFC of the dma-buf heaps patchset Andrew and I have been working on which tries to destage a fair chunk of ION functionality.
The patchset implements per-heap devices which can be opened directly and then an ioctl is used to allocate a dmabuf from the heap.
The interface is similar, but much simpler then IONs, only providing an ALLOC ioctl.
Also, I've provided simple system and cma heaps. The system heap in particular is missing the page-pool optimizations ION had, but works well enough to validate the interface.
I've booted and tested these patches with AOSP on the HiKey960 using the kernel tree here:
https://git.linaro.org/people/john.stultz/android-dev.git/log/?h=dev/dma-buf-heap
And the userspace changes here:
https://android-review.googlesource.com/c/device/linaro/hikey/+/909436
Compared to ION, this patchset is missing the system-contig, carveout and chunk heaps, as I don't have a device that uses those, so I'm unable to do much useful validation there. Additionally we have no upstream users of chunk or carveout, and the system-contig has been deprecated in the common/andoid-* kernels, so this should be ok.
I've also removed the stats accounting for now, since it should be implemented by the heaps themselves.
Eventual TODOS:
* Reimplement page-pool for system heap (working on this)
* Add stats accounting to system/cma heaps
* Make the kselftest actually useful
* Add other heaps folks see as useful (would love to get some help from actual carveout/chunk users)!
That said, the main user-interface is shaping up and I wanted to get some input on the device model (particularly from GreKH) and any other API/ABI specific input.
thanks
-john
Andrew F. Davis (1):
dma-buf: Add dma-buf heaps framework
John Stultz (4):
dma-buf: heaps: Add heap helpers
dma-buf: heaps: Add system heap to dmabuf heaps
dma-buf: heaps: Add CMA heap to dmabuf heapss
kselftests: Add dma-heap test
MAINTAINERS | 16 + drivers/dma-buf/Kconfig | 10 + drivers/dma-buf/Makefile | 2 + drivers/dma-buf/dma-heap.c | 191 ++++++++++++ drivers/dma-buf/heaps/Kconfig | 14 + drivers/dma-buf/heaps/Makefile | 4 + drivers/dma-buf/heaps/cma_heap.c | 164 ++++++++++ drivers/dma-buf/heaps/heap-helpers.c | 335 +++++++++++++++++++++ drivers/dma-buf/heaps/heap-helpers.h | 48 +++ drivers/dma-buf/heaps/system_heap.c | 132 ++++++++ include/linux/dma-heap.h | 65 ++++ include/uapi/linux/dma-heap.h | 52 ++++ tools/testing/selftests/dmabuf-heaps/Makefile | 11 + tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c | 96 ++++++ 14 files changed, 1140 insertions(+) create mode 100644 drivers/dma-buf/dma-heap.c create mode 100644 drivers/dma-buf/heaps/Kconfig create mode 100644 drivers/dma-buf/heaps/Makefile create mode 100644 drivers/dma-buf/heaps/cma_heap.c create mode 100644 drivers/dma-buf/heaps/heap-helpers.c create mode 100644 drivers/dma-buf/heaps/heap-helpers.h create mode 100644 drivers/dma-buf/heaps/system_heap.c create mode 100644 include/linux/dma-heap.h create mode 100644 include/uapi/linux/dma-heap.h create mode 100644 tools/testing/selftests/dmabuf-heaps/Makefile create mode 100644 tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c
Signature · 6 lines
2.7.4 _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
RFC PATCH v2 1/5 dma-buf: Add dma-buf heaps framework
From: "Andrew F. Davis" <[email protected]>
This framework allows a unified userspace interface for dma-buf exporters, allowing userland to allocate specific types of memory for use in dma-buf sharing.
Each heap is given its own device node, which a user can allocate a dma-buf fd from using the DMA_HEAP_IOC_ALLOC.
This code is an evoluiton of the Android ION implementation, and a big thanks is due to its authors/maintainers over time for their effort:
Rebecca Schultz Zavin, Colin Cross, Benjamin Gaignard, Laura Abbott, and many other contributors!
---
v2:
* Folded down fixes I had previously shared in implementing heaps
* Make flags a u64 (Suggested by Laura)
* Add PAGE_ALIGN() fix to the core alloc funciton
* IOCTL fixups suggested by Brian
* Added fixes suggested by Benjamin
* Removed core stats mgmt, as that should be implemented by per-heap code
* Changed alloc to return a dma-buf fd, rather then a buffer (as it simplifies error handling)
--- MAINTAINERS | 16 ++++ drivers/dma-buf/Kconfig | 8 ++ drivers/dma-buf/Makefile | 1 + drivers/dma-buf/dma-heap.c | 191 ++++++++++++++++++++++++++++++++++++++++++ include/linux/dma-heap.h | 65 ++++++++++++++ include/uapi/linux/dma-heap.h | 52 ++++++++++++ 6 files changed, 333 insertions(+) create mode 100644 drivers/dma-buf/dma-heap.c create mode 100644 include/linux/dma-heap.h create mode 100644 include/uapi/linux/dma-heap.h
MAINTAINERS +16 -0
@@ -4621,6 +4621,22 @@ F: include/linux/*fence.h F: Documentation/driver-api/dma-buf.rst T: git git://anongit.freedesktop.org/drm/drm-misc +DMA-BUF HEAPS FRAMEWORK +M: Laura Abbott <[email protected]> +R: Liam Mark <[email protected]> +R: Brian Starkey <[email protected]> +R: "Andrew F. Davis" <[email protected]> +R: John Stultz <[email protected]> +S: Maintained +L: [email protected] +L: [email protected] +L: [email protected] (moderated for non-subscribers) +F: include/uapi/linux/dma-heap.h +F: include/linux/dma-heap.h +F: drivers/dma-buf/dma-heap.c +F: drivers/dma-buf/heaps/* +T: git git://anongit.freedesktop.org/drm/drm-misc + DMA GENERIC OFFLOAD ENGINE SUBSYSTEM M: Vinod Koul <[email protected]> L: [email protected]
drivers/dma-buf/Kconfig +8 -0
@@ -39,4 +39,12 @@ config UDMABUF A driver to let userspace turn memfd regions into dma-bufs. Qemu can use this to create host dmabufs for guest framebuffers. +menuconfig DMABUF_HEAPS + bool "DMA-BUF Userland Memory Heaps" + select DMA_SHARED_BUFFER + help + Choose this option to enable the DMA-BUF userland memory heaps, + this allows userspace to allocate dma-bufs that can be shared between + drivers. + endmenu
drivers/dma-buf/Makefile +1 -0
@@ -1,4 +1,5 @@ obj-y := dma-buf.o dma-fence.o dma-fence-array.o reservation.o seqno-fence.o +obj-$(CONFIG_DMABUF_HEAPS) += dma-heap.o obj-$(CONFIG_SYNC_FILE) += sync_file.o obj-$(CONFIG_SW_SYNC) += sw_sync.o sync_debug.o obj-$(CONFIG_UDMABUF) += udmabuf.o
drivers/dma-buf/dma-heap.c new file +191 -0
191 lines not on this page · raw message
include/linux/dma-heap.h new file +65 -0
65 lines not on this page · raw message
include/uapi/linux/dma-heap.h new file +52 -0
52 lines not on this page · raw message
Signature · 6 lines
2.7.4 _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [RFC][PATCH 1/5 v2] dma-buf: Add dma-buf heaps framework
Quoted 391 lines · John Stultz
From: "Andrew F. Davis" <[email protected]>
This framework allows a unified userspace interface for dma-buf exporters, allowing userland to allocate specific types of memory for use in dma-buf sharing.
Each heap is given its own device node, which a user can allocate a dma-buf fd from using the
DMA_HEAP_IOC_ALLOC.This code is an evoluiton of the Android ION implementation, and a big thanks is due to its authors/maintainers over time for their effort:
Rebecca Schultz Zavin, Colin Cross, Benjamin Gaignard, Laura Abbott, and many other contributors!Cc Laura Abbott <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Greg KH <[email protected]>Cc Sumit Semwal <[email protected]>Cc Liam Mark <[email protected]>Cc Brian Starkey <[email protected]>Cc Andrew F. Davis <[email protected]>Cc Chenbo Feng <[email protected]>Cc Alistair Strachan <[email protected]>Signed-off-by Andrew F. Davis <[email protected]>[jstultz: reworded commit message, and lots of cleanups]Signed-off-by John Stultz <[email protected]>---
v2:
* Folded down fixes I had previously shared in implementing heaps
* Make flags a u64 (Suggested by Laura)
* AddPAGE_ALIGN()fix to the core alloc funciton
* IOCTL fixups suggested by Brian
* Added fixes suggested by Benjamin
* Removed core stats mgmt, as that should be implemented by per-heap code
* Changed alloc to return a dma-buf fd, rather then a buffer (as it simplifies error handling)--- MAINTAINERS | 16 ++++ drivers/dma-buf/Kconfig | 8 ++ drivers/dma-buf/Makefile | 1 + drivers/dma-buf/dma-heap.c | 191 ++++++++++++++++++++++++++++++++++++++++++ include/linux/dma-heap.h | 65 ++++++++++++++ include/uapi/linux/dma-heap.h | 52 ++++++++++++ 6 files changed, 333 insertions(+) create mode 100644 drivers/dma-buf/dma-heap.c create mode 100644 include/linux/dma-heap.h create mode 100644 include/uapi/linux/dma-heap.hMAINTAINERS +16
-0@@ -4621,6 +4621,22 @@ F: include/linux/*fence.h F: Documentation/driver-api/dma-buf.rst T: git git://anongit.freedesktop.org/drm/drm-misc +DMA-BUF HEAPS FRAMEWORK +M: Laura Abbott <[email protected]> +R: Liam Mark <[email protected]> +R: Brian Starkey <[email protected]> +R: "Andrew F. Davis" <[email protected]> +R: John Stultz <[email protected]> +S: Maintained +L: [email protected] +L: [email protected] +L: [email protected] (moderated for non-subscribers) +F: include/uapi/linux/dma-heap.h +F: include/linux/dma-heap.h +F: drivers/dma-buf/dma-heap.c +F: drivers/dma-buf/heaps/* +T: git git://anongit.freedesktop.org/drm/drm-misc + DMA GENERIC OFFLOAD ENGINE SUBSYSTEM M: Vinod Koul <[email protected]> L: [email protected]drivers/dma-buf/Kconfig +8
-0@@ -39,4 +39,12 @@ config UDMABUF A driver to let userspace turn memfd regions into dma-bufs. Qemu can use this to create host dmabufs for guest framebuffers. +menuconfig DMABUF_HEAPS + bool "DMA-BUF Userland Memory Heaps" + select DMA_SHARED_BUFFER + help + Choose this option to enable the DMA-BUF userland memory heaps, + this allows userspace to allocate dma-bufs that can be shared between + drivers. + endmenudrivers/dma-buf/Makefile +1
-0@@ -1,4 +1,5 @@ obj-y := dma-buf.o dma-fence.o dma-fence-array.o reservation.o seqno-fence.o +obj-$(CONFIG_DMABUF_HEAPS) += dma-heap.o obj-$(CONFIG_SYNC_FILE) += sync_file.o obj-$(CONFIG_SW_SYNC) += sw_sync.o sync_debug.o obj-$(CONFIG_UDMABUF) += udmabuf.odrivers/dma-buf/dma-heap.c new file +191
-0@@ -0,0 +1,191 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Framework for userspace DMA-BUF allocations +* +* Copyright (C) 2011 Google, Inc. +* Copyright (C) 2019 Linaro Ltd. +*/ + +#include <linux/cdev.h> +#include <linux/debugfs.h> +#include <linux/device.h> +#include <linux/dma-buf.h> +#include <linux/err.h> +#include <linux/idr.h> +#include <linux/list.h> +#include <linux/slab.h> +#include <linux/uaccess.h> + +#include <linux/dma-heap.h> +#include <uapi/linux/dma-heap.h> + +#define DEVNAME "dma_heap" + +#define NUM_HEAP_MINORS 128 +static DEFINE_IDR (dma_heap_idr); +static DEFINE_MUTEX (minor_lock);/* Protect idr accesses */ + +dev_t dma_heap_devt; +struct class *dma_heap_class; +struct list_head dma_heap_list; +struct dentry *dma_heap_debug_root; + +static int dma_heap_buffer_alloc (struct dma_heap *heap,size_t len, +unsigned int flags) +{ + len =PAGE_ALIGN (len); +if (!len) +return -EINVAL ; + +return heap->ops->allocate (heap, len, flags); +} + +static int dma_heap_open (struct inode *inode,struct file *filp) +{ +struct dma_heap *heap; + +mutex_lock (&minor_lock); + heap =idr_find (&dma_heap_idr,iminor (inode)); +mutex_unlock (&minor_lock); +if (!heap) { +pr_err ("dma_heap: minor %d unknown.\n" ,iminor (inode)); +return -ENODEV ; + } + +/* instance data as context */ + filp->private_data = heap; +nonseekable_open (inode, filp); + +return 0 ; +} + +static int dma_heap_release (struct inode *inode,struct file *filp) +{ + filp->private_data =NULL ; + +return 0 ; +} + +static long dma_heap_ioctl (struct file *filp,unsigned int cmd, +unsigned long arg) +{ +switch (cmd) { +case DMA_HEAP_IOC_ALLOC : + { +struct dma_heap_allocation_data heap_allocation; +struct dma_heap *heap = filp->private_data; +int fd; + +if (copy_from_user (&heap_allocation, (void __user *)arg, +sizeof (heap_allocation))) +return -EFAULT ; + +if (heap_allocation.fd || + heap_allocation.reserved0 || + heap_allocation.reserved1 || + heap_allocation.reserved2) { +pr_warn_once ("dma_heap: ioctl data not valid\n" ); +return -EINVAL ; + } +if (heap_allocation.flags & ~DMA_HEAP_VALID_FLAGS ) { +pr_warn_once ("dma_heap: flags has invalid or unsupported flags set\n" ); +return -EINVAL ; + } + + fd =dma_heap_buffer_alloc (heap, heap_allocation.len, + heap_allocation.flags); +if (fd <0 ) +return fd; + + heap_allocation.fd = fd; + +if (copy_to_user ((void __user *)arg, &heap_allocation, +sizeof (heap_allocation))) +return -EFAULT ; + +break ; + } +default : +return -ENOTTY ; + } + +return 0 ; +} + +static const struct file_operations dma_heap_fops = { + .owner =THIS_MODULE , + .open = dma_heap_open, + .release = dma_heap_release, + .unlocked_ioctl = dma_heap_ioctl, +#ifdef CONFIG_COMPAT + .compat_ioctl = dma_heap_ioctl, +#endif +}; + +int dma_heap_add (struct dma_heap *heap) +{ +struct device *dev_ret; +int ret; + +if (!heap->name || !strcmp (heap->name,"" )) { +pr_err ("dma_heap: Cannot add heap without a name\n" ); +return -EINVAL ; + } + +if (!heap->ops || !heap->ops->allocate) { +pr_err ("dma_heap: Cannot add heap with invalid ops struct\n" ); +return -EINVAL ; + } + +/* Find unused minor number */ +mutex_lock (&minor_lock); + ret =idr_alloc (&dma_heap_idr, heap,0 ,NUM_HEAP_MINORS ,GFP_KERNEL ); +mutex_unlock (&minor_lock); +if (ret <0 ) { +pr_err ("dma_heap: Unable to get minor number for heap\n" ); +return ret; + } + heap->minor = ret; + +/* Create device */ + heap->heap_devt =MKDEV (MAJOR (dma_heap_devt), heap->minor); + dev_ret =device_create (dma_heap_class, +NULL , + heap->heap_devt, +NULL , + heap->name); +if (IS_ERR (dev_ret)) { +pr_err ("dma_heap: Unable to create char device\n" ); +return PTR_ERR (dev_ret); + } + +/* Add device */ +cdev_init (&heap->heap_cdev, &dma_heap_fops); + ret =cdev_add (&heap->heap_cdev, dma_heap_devt,NUM_HEAP_MINORS ); +if (ret <0 ) { +device_destroy (dma_heap_class, heap->heap_devt); +pr_err ("dma_heap: Unable to add char device\n" ); +return ret; + } + +return 0 ; +} +EXPORT_SYMBOL (dma_heap_add); + +static int dma_heap_init (void ) +{ +int ret; + + ret =alloc_chrdev_region (&dma_heap_devt,0 ,NUM_HEAP_MINORS ,DEVNAME ); +if (ret) +return ret; + + dma_heap_class =class_create (THIS_MODULE ,DEVNAME ); +if (IS_ERR (dma_heap_class)) { +unregister_chrdev_region (dma_heap_devt,NUM_HEAP_MINORS ); +return PTR_ERR (dma_heap_class); + } + +return 0 ; +} +subsys_initcall (dma_heap_init);include/linux/dma-heap.h new file +65
-0@@ -0,0 +1,65 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* DMABUF Heaps Allocation Infrastructure +* +* Copyright (C) 2011 Google, Inc. +* Copyright (C) 2019 Linaro Ltd. +*/ + +#ifndef _DMA_HEAPS_H +#define _DMA_HEAPS_H + +#include <linux/cdev.h> +#include <linux/types.h> + +/** +* struct dma_heap_buffer - metadata for a particular buffer +* @heap: back pointer to the heap the buffer came from +* @dmabuf: backing dma-buf for this buffer +* @size: size of the buffer +* @flags: buffer specific flags +*/ +struct dma_heap_buffer { +struct dma_heap *heap; +struct dma_buf *dmabuf; +size_t size; +unsigned long flags; +}; + +/** +* struct dma_heap - represents a dmabuf heap in the system +* @name: used for debugging/device-node name +* @ops: ops struct for this heap +* @minor minor number of this heap device +* @heap_devt heap device node +* @heap_cdev heap char device +* +* Represents a heap of memory from which buffers can be made. +*/ +struct dma_heap { +const char *name; +struct dma_heap_ops *ops; +unsigned int minor; +dev_t heap_devt; +struct cdev heap_cdev; +}; + +/** +* struct dma_heap_ops - ops to operate on a given heap +* @allocate: allocate dmabuf and return fd +* +* allocate returns dmabuf fd on success, -errno on error. +*/ +struct dma_heap_ops { +int (*allocate)(struct dma_heap *heap, +unsigned long len, +unsigned long flags); +}; + +/** +* dma_heap_add - adds a heap to dmabuf heaps +* @heap: the heap to add +*/ +int dma_heap_add (struct dma_heap *heap); + +#endif /* _DMA_HEAPS_H */ include/uapi/linux/dma-heap.h new file +10
-0@@ -0,0 +1,52 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* DMABUF Heaps Userspace API +* +* Copyright (C) 2011 Google, Inc. +* Copyright (C) 2019 Linaro Ltd. +*/ +#ifndef _UAPI_LINUX_DMABUF_POOL_H +#define _UAPI_LINUX_DMABUF_POOL_H +
include/uapi/linux/dma-heap.h line 20
Quoted 10 lines
include/uapi/linux/dma-heap.h new file +10
-0+#include <linux/ioctl.h> +#include <linux/types.h> + +/** +* DOC: DMABUF Heaps Userspace API +* +*/ + +/* Currently no flags */ +#define DMA_HEAP_VALID_FLAGS (0)
I think here you need to allow flags like O_RDWR or O_CLOEXEC else mmap will fail.
Benjamin
Quoted 35 lines
include/uapi/linux/dma-heap.h new file +32
-1+ +/** +* struct dma_heap_allocation_data - metadata passed from userspace for +* allocations +* @len: size of the allocation +* @flags: flags passed to pool +* @fd: will be populated with a fd which provdes the +* handle to the allocated dma-buf +* +* Provided by userspace as an argument to the ioctl +*/ +struct dma_heap_allocation_data { +__u64 len; +__u64 flags; +__u32 fd; +__u32 reserved0; +__u32 reserved1; +__u32 reserved2; +}; + +#define DMA_HEAP_IOC_MAGIC 'H' + +/** +* DOC: DMA_HEAP_IOC_ALLOC - allocate memory from pool +* +* Takes an dma_heap_allocation_data struct and returns it with the fd field +* populated with the dmabuf handle of the allocation. +*/ +#define DMA_HEAP_IOC_ALLOC _IOWR(DMA_HEAP_IOC_MAGIC, 0, \ + struct dma_heap_allocation_data) + +#endif /* _UAPI_LINUX_DMABUF_POOL_H */ --2.7.4
Signature · 7 lines
Benjamin Gaignard Graphic Study Group Linaro.org │ Open source software for ARM SoCs Follow Linaro: Facebook | Twitter | Blog
Re: [RFC][PATCH 1/5 v2] dma-buf: Add dma-buf heaps framework
Quoted 12 lines · Benjamin Gaignard
include/uapi/linux/dma-heap.h line 20
Quoted 7 lines · John Stultz
include/uapi/linux/dma-heap.h new file +7
-0+/** +* DOC: DMABUF Heaps Userspace API +* +*/ + +/* Currently no flags */ +#define DMA_HEAP_VALID_FLAGS (0)I think here you need to allow flags like
O_RDWRorO_CLOEXECelse mmap will fail.
Hm. So I meant for these to be just the bitmask of valid flags for the allocate IOCTL (used to make sure no one is passing junk flags accidentally), rather then valid flags for the heap open call. But this probably suggests I should call it something like DMA_HEAP_ALLOC_VALID_FLAGS instead?
thanks
-john
Re: [RFC][PATCH 1/5 v2] dma-buf: Add dma-buf heaps framework
In reply to Benjamin GaignardHi Benjamin,
please fix your mailer to avoid completely pointless full quotes.
Thank you!
Re: [RFC][PATCH 1/5 v2] dma-buf: Add dma-buf heaps framework
In reply to John StultzQuoted 57 lines · John Stultz
From: "Andrew F. Davis" <[email protected]>
This framework allows a unified userspace interface for dma-buf exporters, allowing userland to allocate specific types of memory for use in dma-buf sharing.
Each heap is given its own device node, which a user can allocate a dma-buf fd from using the
DMA_HEAP_IOC_ALLOC.This code is an evoluiton of the Android ION implementation, and a big thanks is due to its authors/maintainers over time for their effort:
Rebecca Schultz Zavin, Colin Cross, Benjamin Gaignard, Laura Abbott, and many other contributors!Cc Laura Abbott <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Greg KH <[email protected]>Cc Sumit Semwal <[email protected]>Cc Liam Mark <[email protected]>Cc Brian Starkey <[email protected]>Cc Andrew F. Davis <[email protected]>Cc Chenbo Feng <[email protected]>Cc Alistair Strachan <[email protected]>Signed-off-by Andrew F. Davis <[email protected]>[jstultz: reworded commit message, and lots of cleanups]Signed-off-by John Stultz <[email protected]>---
v2:
* Folded down fixes I had previously shared in implementing heaps
* Make flags a u64 (Suggested by Laura)
* AddPAGE_ALIGN()fix to the core alloc funciton
* IOCTL fixups suggested by Brian
* Added fixes suggested by Benjamin
* Removed core stats mgmt, as that should be implemented by per-heap code
* Changed alloc to return a dma-buf fd, rather then a buffer (as it simplifies error handling)--- MAINTAINERS | 16 ++++ drivers/dma-buf/Kconfig | 8 ++ drivers/dma-buf/Makefile | 1 + drivers/dma-buf/dma-heap.c | 191 ++++++++++++++++++++++++++++++++++++++++++ include/linux/dma-heap.h | 65 ++++++++++++++ include/uapi/linux/dma-heap.h | 52 ++++++++++++ 6 files changed, 333 insertions(+) create mode 100644 drivers/dma-buf/dma-heap.c create mode 100644 include/linux/dma-heap.h create mode 100644 include/uapi/linux/dma-heap.hMAINTAINERS +0
-0
MAINTAINERS F: include/linux/*fence.h · line 4628
Quoted 8 lines
MAINTAINERS +5
-0@@ -4621,6 +4621,22 @@ F: include/linux/*fence.h F: Documentation/driver-api/dma-buf.rst T: git git://anongit.freedesktop.org/drm/drm-misc +DMA-BUF HEAPS FRAMEWORK +M: Laura Abbott <[email protected]> +R: Liam Mark <[email protected]> +R: Brian Starkey <[email protected]> +R: "Andrew F. Davis" <[email protected]>
Quotes not needed in maintainers file.
Quoted 123 lines
MAINTAINERS +11
-0+R: John Stultz <[email protected]> +S: Maintained +L: [email protected] +L: [email protected] +L: [email protected] (moderated for non-subscribers) +F: include/uapi/linux/dma-heap.h +F: include/linux/dma-heap.h +F: drivers/dma-buf/dma-heap.c +F: drivers/dma-buf/heaps/* +T: git git://anongit.freedesktop.org/drm/drm-misc + DMA GENERIC OFFLOAD ENGINE SUBSYSTEM M: Vinod Koul <[email protected]> L: [email protected]drivers/dma-buf/Kconfig +8
-0@@ -39,4 +39,12 @@ config UDMABUF A driver to let userspace turn memfd regions into dma-bufs. Qemu can use this to create host dmabufs for guest framebuffers. +menuconfig DMABUF_HEAPS + bool "DMA-BUF Userland Memory Heaps" + select DMA_SHARED_BUFFER + help + Choose this option to enable the DMA-BUF userland memory heaps, + this allows userspace to allocate dma-bufs that can be shared between + drivers. + endmenudrivers/dma-buf/Makefile +1
-0@@ -1,4 +1,5 @@ obj-y := dma-buf.o dma-fence.o dma-fence-array.o reservation.o seqno-fence.o +obj-$(CONFIG_DMABUF_HEAPS) += dma-heap.o obj-$(CONFIG_SYNC_FILE) += sync_file.o obj-$(CONFIG_SW_SYNC) += sw_sync.o sync_debug.o obj-$(CONFIG_UDMABUF) += udmabuf.odrivers/dma-buf/dma-heap.c new file +76
-0@@ -0,0 +1,191 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Framework for userspace DMA-BUF allocations +* +* Copyright (C) 2011 Google, Inc. +* Copyright (C) 2019 Linaro Ltd. +*/ + +#include <linux/cdev.h> +#include <linux/debugfs.h> +#include <linux/device.h> +#include <linux/dma-buf.h> +#include <linux/err.h> +#include <linux/idr.h> +#include <linux/list.h> +#include <linux/slab.h> +#include <linux/uaccess.h> + +#include <linux/dma-heap.h> +#include <uapi/linux/dma-heap.h> + +#define DEVNAME "dma_heap" + +#define NUM_HEAP_MINORS 128 +static DEFINE_IDR (dma_heap_idr); +static DEFINE_MUTEX (minor_lock);/* Protect idr accesses */ + +dev_t dma_heap_devt; +struct class *dma_heap_class; +struct list_head dma_heap_list; +struct dentry *dma_heap_debug_root; + +static int dma_heap_buffer_alloc (struct dma_heap *heap,size_t len, +unsigned int flags) +{ + len =PAGE_ALIGN (len); +if (!len) +return -EINVAL ; + +return heap->ops->allocate (heap, len, flags); +} + +static int dma_heap_open (struct inode *inode,struct file *filp) +{ +struct dma_heap *heap; + +mutex_lock (&minor_lock); + heap =idr_find (&dma_heap_idr,iminor (inode)); +mutex_unlock (&minor_lock); +if (!heap) { +pr_err ("dma_heap: minor %d unknown.\n" ,iminor (inode)); +return -ENODEV ; + } + +/* instance data as context */ + filp->private_data = heap; +nonseekable_open (inode, filp); + +return 0 ; +} + +static int dma_heap_release (struct inode *inode,struct file *filp) +{ + filp->private_data =NULL ; + +return 0 ; +} + +static long dma_heap_ioctl (struct file *filp,unsigned int cmd, +unsigned long arg) +{ +switch (cmd) { +case DMA_HEAP_IOC_ALLOC : + { +struct dma_heap_allocation_data heap_allocation; +struct dma_heap *heap = filp->private_data;
drivers/dma-buf/dma-heap.c line 86
Quoted 10 lines
drivers/dma-buf/dma-heap.c new file +10
-0+int fd; + +if (copy_from_user (&heap_allocation, (void __user *)arg, +sizeof (heap_allocation))) +return -EFAULT ; + +if (heap_allocation.fd || + heap_allocation.reserved0 || + heap_allocation.reserved1 || + heap_allocation.reserved2) {
Seems too many reserved, I can understand one, but if we ever needed all of these we would be better off just adding another alloc ioctl.
Quoted 35 lines
drivers/dma-buf/dma-heap.c new file +35
-0+pr_warn_once ("dma_heap: ioctl data not valid\n" ); +return -EINVAL ; + } +if (heap_allocation.flags & ~DMA_HEAP_VALID_FLAGS ) { +pr_warn_once ("dma_heap: flags has invalid or unsupported flags set\n" ); +return -EINVAL ; + } + + fd =dma_heap_buffer_alloc (heap, heap_allocation.len, + heap_allocation.flags); +if (fd <0 ) +return fd; + + heap_allocation.fd = fd; + +if (copy_to_user ((void __user *)arg, &heap_allocation, +sizeof (heap_allocation))) +return -EFAULT ; + +break ; + } +default : +return -ENOTTY ; + } + +return 0 ; +} + +static const struct file_operations dma_heap_fops = { + .owner =THIS_MODULE , + .open = dma_heap_open, + .release = dma_heap_release, + .unlocked_ioctl = dma_heap_ioctl, +#ifdef CONFIG_COMPAT + .compat_ioctl = dma_heap_ioctl,
drivers/dma-buf/dma-heap.c line 131
Quoted 10 lines
drivers/dma-buf/dma-heap.c new file +10
-0+#endif +}; + +int dma_heap_add (struct dma_heap *heap) +{ +struct device *dev_ret; +int ret; + +if (!heap->name || !strcmp (heap->name,"" )) { +pr_err ("dma_heap: Cannot add heap without a name\n" );
As these names end up as the dev name in the file system we may want to check for invalid names, there is probably a helper for that somewhere.
Quoted 101 lines
drivers/dma-buf/dma-heap.c new file +60
-0+return -EINVAL ; + } + +if (!heap->ops || !heap->ops->allocate) { +pr_err ("dma_heap: Cannot add heap with invalid ops struct\n" ); +return -EINVAL ; + } + +/* Find unused minor number */ +mutex_lock (&minor_lock); + ret =idr_alloc (&dma_heap_idr, heap,0 ,NUM_HEAP_MINORS ,GFP_KERNEL ); +mutex_unlock (&minor_lock); +if (ret <0 ) { +pr_err ("dma_heap: Unable to get minor number for heap\n" ); +return ret; + } + heap->minor = ret; + +/* Create device */ + heap->heap_devt =MKDEV (MAJOR (dma_heap_devt), heap->minor); + dev_ret =device_create (dma_heap_class, +NULL , + heap->heap_devt, +NULL , + heap->name); +if (IS_ERR (dev_ret)) { +pr_err ("dma_heap: Unable to create char device\n" ); +return PTR_ERR (dev_ret); + } + +/* Add device */ +cdev_init (&heap->heap_cdev, &dma_heap_fops); + ret =cdev_add (&heap->heap_cdev, dma_heap_devt,NUM_HEAP_MINORS ); +if (ret <0 ) { +device_destroy (dma_heap_class, heap->heap_devt); +pr_err ("dma_heap: Unable to add char device\n" ); +return ret; + } + +return 0 ; +} +EXPORT_SYMBOL (dma_heap_add); + +static int dma_heap_init (void ) +{ +int ret; + + ret =alloc_chrdev_region (&dma_heap_devt,0 ,NUM_HEAP_MINORS ,DEVNAME ); +if (ret) +return ret; + + dma_heap_class =class_create (THIS_MODULE ,DEVNAME ); +if (IS_ERR (dma_heap_class)) { +unregister_chrdev_region (dma_heap_devt,NUM_HEAP_MINORS ); +return PTR_ERR (dma_heap_class); + } + +return 0 ; +} +subsys_initcall (dma_heap_init);include/linux/dma-heap.h new file +35
-0@@ -0,0 +1,65 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* DMABUF Heaps Allocation Infrastructure +* +* Copyright (C) 2011 Google, Inc. +* Copyright (C) 2019 Linaro Ltd. +*/ + +#ifndef _DMA_HEAPS_H +#define _DMA_HEAPS_H + +#include <linux/cdev.h> +#include <linux/types.h> + +/** +* struct dma_heap_buffer - metadata for a particular buffer +* @heap: back pointer to the heap the buffer came from +* @dmabuf: backing dma-buf for this buffer +* @size: size of the buffer +* @flags: buffer specific flags +*/ +struct dma_heap_buffer { +struct dma_heap *heap; +struct dma_buf *dmabuf; +size_t size; +unsigned long flags; +}; + +/** +* struct dma_heap - represents a dmabuf heap in the system +* @name: used for debugging/device-node name +* @ops: ops struct for this heap +* @minor minor number of this heap device +* @heap_devt heap device node +* @heap_cdev heap char device
include/linux/dma-heap.h line 45
Quoted 10 lines
include/linux/dma-heap.h new file +10
-0+* +* Represents a heap of memory from which buffers can be made. +*/ +struct dma_heap { +const char *name; +struct dma_heap_ops *ops; +unsigned int minor; +dev_t heap_devt; +struct cdev heap_cdev; +};
Still not sure about this, all of the members in this struct are strictly internally used by the framework. The users of this framework should not have access to them and only need to deal with an opaque pointer for tracking themselves (can store it in a private struct of their own then container_of to get back out their struct).
Anyway, not a big deal, and if it really bugs me enough I can always go fix it later, it's all kernel internal so not a blocker here. :)
Andrew
Quoted 69 lines
include/linux/dma-heap.h new file +20
-0+ +/** +* struct dma_heap_ops - ops to operate on a given heap +* @allocate: allocate dmabuf and return fd +* +* allocate returns dmabuf fd on success, -errno on error. +*/ +struct dma_heap_ops { +int (*allocate)(struct dma_heap *heap, +unsigned long len, +unsigned long flags); +}; + +/** +* dma_heap_add - adds a heap to dmabuf heaps +* @heap: the heap to add +*/ +int dma_heap_add (struct dma_heap *heap); + +#endif /* _DMA_HEAPS_H */ include/uapi/linux/dma-heap.h new file +42
-0@@ -0,0 +1,52 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* DMABUF Heaps Userspace API +* +* Copyright (C) 2011 Google, Inc. +* Copyright (C) 2019 Linaro Ltd. +*/ +#ifndef _UAPI_LINUX_DMABUF_POOL_H +#define _UAPI_LINUX_DMABUF_POOL_H + +#include <linux/ioctl.h> +#include <linux/types.h> + +/** +* DOC: DMABUF Heaps Userspace API +* +*/ + +/* Currently no flags */ +#define DMA_HEAP_VALID_FLAGS (0) + +/** +* struct dma_heap_allocation_data - metadata passed from userspace for +* allocations +* @len: size of the allocation +* @flags: flags passed to pool +* @fd: will be populated with a fd which provdes the +* handle to the allocated dma-buf +* +* Provided by userspace as an argument to the ioctl +*/ +struct dma_heap_allocation_data { +__u64 len; +__u64 flags; +__u32 fd; +__u32 reserved0; +__u32 reserved1; +__u32 reserved2; +}; + +#define DMA_HEAP_IOC_MAGIC 'H' +
include/uapi/linux/dma-heap.h line 52
Quoted 10 lines
include/uapi/linux/dma-heap.h new file +10
-0+/** +* DOC: DMA_HEAP_IOC_ALLOC - allocate memory from pool +* +* Takes an dma_heap_allocation_data struct and returns it with the fd field +* populated with the dmabuf handle of the allocation. +*/ +#define DMA_HEAP_IOC_ALLOC _IOWR(DMA_HEAP_IOC_MAGIC, 0, \ + struct dma_heap_allocation_data) + +#endif /* _UAPI_LINUX_DMABUF_POOL_H */
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [RFC][PATCH 1/5 v2] dma-buf: Add dma-buf heaps framework
Quoted 62 lines · Andrew F. Davis
Quoted 60 lines · John Stultz
From: "Andrew F. Davis" <[email protected]>
This framework allows a unified userspace interface for dma-buf exporters, allowing userland to allocate specific types of memory for use in dma-buf sharing.
Each heap is given its own device node, which a user can allocate a dma-buf fd from using the
DMA_HEAP_IOC_ALLOC.This code is an evoluiton of the Android ION implementation, and a big thanks is due to its authors/maintainers over time for their effort:
Rebecca Schultz Zavin, Colin Cross, Benjamin Gaignard, Laura Abbott, and many other contributors!Cc Laura Abbott <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Greg KH <[email protected]>Cc Sumit Semwal <[email protected]>Cc Liam Mark <[email protected]>Cc Brian Starkey <[email protected]>Cc Andrew F. Davis <[email protected]>Cc Chenbo Feng <[email protected]>Cc Alistair Strachan <[email protected]>Signed-off-by Andrew F. Davis <[email protected]>[jstultz: reworded commit message, and lots of cleanups]Signed-off-by John Stultz <[email protected]>---
v2:
* Folded down fixes I had previously shared in implementing heaps
* Make flags a u64 (Suggested by Laura)
* AddPAGE_ALIGN()fix to the core alloc funciton
* IOCTL fixups suggested by Brian
* Added fixes suggested by Benjamin
* Removed core stats mgmt, as that should be implemented by per-heap code
* Changed alloc to return a dma-buf fd, rather then a buffer (as it simplifies error handling)--- MAINTAINERS | 16 ++++ drivers/dma-buf/Kconfig | 8 ++ drivers/dma-buf/Makefile | 1 + drivers/dma-buf/dma-heap.c | 191 ++++++++++++++++++++++++++++++++++++++++++ include/linux/dma-heap.h | 65 ++++++++++++++ include/uapi/linux/dma-heap.h | 52 ++++++++++++ 6 files changed, 333 insertions(+) create mode 100644 drivers/dma-buf/dma-heap.c create mode 100644 include/linux/dma-heap.h create mode 100644 include/uapi/linux/dma-heap.hMAINTAINERS +0
-0@@ -4621,6 +4621,22 @@ F: include/linux/*fence.h F: Documentation/driver-api/dma-buf.rst T: git git://anongit.freedesktop.org/drm/drm-misc
Quoted 7 lines
MAINTAINERS F: include/linux/*fence.h · line 4628
Quoted 5 lines
MAINTAINERS +5
-0+DMA-BUF HEAPS FRAMEWORK +M: Laura Abbott <[email protected]> +R: Liam Mark <[email protected]> +R: Brian Starkey <[email protected]> +R: "Andrew F. Davis" <[email protected]>Quotes not needed in maintainers file.
Whatever you say, "Andrew F. Davis", or whomever you really are! ;)
Quoted 8 lines
MAINTAINERS F: include/linux/*fence.h · line 4633
Quoted 5 lines
MAINTAINERS +5
-0+ + if (heap_allocation.fd || + heap_allocation.reserved0 || + heap_allocation.reserved1 || + heap_allocation.reserved2) {Seems too many reserved, I can understand one, but if we ever needed all of these we would be better off just adding another alloc ioctl.
Well, we have to have one u32 for padding. And I figured if we needed anything more then a u32, then we're in for 2 more.
And I think the potential of the alignment and heap-private flags, I worry we might want to have something, but I guess we could just add a new ioctl and keep the support for the old one if folks prefer.
Quoted 10 lines
MAINTAINERS F: include/linux/*fence.h · line 4640
Quoted 7 lines
MAINTAINERS +7
-0+int dma_heap_add(struct dma_heap *heap) +{ + struct device *dev_ret; + int ret; + + if (!heap->name || !strcmp(heap->name, "")) { + pr_err("dma_heap: Cannot add heap without a name\n");As these names end up as the dev name in the file system we may want to check for invalid names, there is probably a helper for that somewhere.
Hrm. I'll have to look.
Quoted 14 lines
MAINTAINERS F: include/linux/*fence.h
Quoted 7 lines
MAINTAINERS +7
-0+struct dma_heap { + const char *name; + struct dma_heap_ops *ops; + unsigned int minor; + dev_t heap_devt; + struct cdev heap_cdev; +};Still not sure about this, all of the members in this struct are strictly internally used by the framework. The users of this framework should not have access to them and only need to deal with an opaque pointer for tracking themselves (can store it in a private struct of their own then
container_ofto get back out their struct).
Quoted 2 lines
Anyway, not a big deal, and if it really bugs me enough I can always go fix it later, it's all kernel internal so not a blocker here. :)
I guess I'd just move the include/linux/dma-heap.h to drivers/dma-buf/heaps/ and keep it localized there.
But whichever. Feel free to also send a patch and I can fold it down.
thanks
-john
Re: [RFC][PATCH 1/5 v2] dma-buf: Add dma-buf heaps framework
Quoted 71 lines · John Stultz
Quoted 62 lines · Andrew F. Davis
Quoted 60 lines · John Stultz
From: "Andrew F. Davis" <[email protected]>
This framework allows a unified userspace interface for dma-buf exporters, allowing userland to allocate specific types of memory for use in dma-buf sharing.
Each heap is given its own device node, which a user can allocate a dma-buf fd from using the
DMA_HEAP_IOC_ALLOC.This code is an evoluiton of the Android ION implementation, and a big thanks is due to its authors/maintainers over time for their effort:
Rebecca Schultz Zavin, Colin Cross, Benjamin Gaignard, Laura Abbott, and many other contributors!Cc Laura Abbott <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Greg KH <[email protected]>Cc Sumit Semwal <[email protected]>Cc Liam Mark <[email protected]>Cc Brian Starkey <[email protected]>Cc Andrew F. Davis <[email protected]>Cc Chenbo Feng <[email protected]>Cc Alistair Strachan <[email protected]>Signed-off-by Andrew F. Davis <[email protected]>[jstultz: reworded commit message, and lots of cleanups]Signed-off-by John Stultz <[email protected]>---
v2:
* Folded down fixes I had previously shared in implementing heaps
* Make flags a u64 (Suggested by Laura)
* AddPAGE_ALIGN()fix to the core alloc funciton
* IOCTL fixups suggested by Brian
* Added fixes suggested by Benjamin
* Removed core stats mgmt, as that should be implemented by per-heap code
* Changed alloc to return a dma-buf fd, rather then a buffer (as it simplifies error handling)--- MAINTAINERS | 16 ++++ drivers/dma-buf/Kconfig | 8 ++ drivers/dma-buf/Makefile | 1 + drivers/dma-buf/dma-heap.c | 191 ++++++++++++++++++++++++++++++++++++++++++ include/linux/dma-heap.h | 65 ++++++++++++++ include/uapi/linux/dma-heap.h | 52 ++++++++++++ 6 files changed, 333 insertions(+) create mode 100644 drivers/dma-buf/dma-heap.c create mode 100644 include/linux/dma-heap.h create mode 100644 include/uapi/linux/dma-heap.hMAINTAINERS +0
-0@@ -4621,6 +4621,22 @@ F: include/linux/*fence.h F: Documentation/driver-api/dma-buf.rst T: git git://anongit.freedesktop.org/drm/drm-miscQuoted 7 lines
MAINTAINERS F: include/linux/*fence.h · line 4628
Quoted 5 lines
MAINTAINERS +5
-0+DMA-BUF HEAPS FRAMEWORK +M: Laura Abbott <[email protected]> +R: Liam Mark <[email protected]> +R: Brian Starkey <[email protected]> +R: "Andrew F. Davis" <[email protected]>Quotes not needed in maintainers file.
Quoted 2 lines
Whatever you say, "Andrew F. Davis", or whomever you really are! ;)
<_<
Quoted 45 lines
_>
Quoted 8 lines
MAINTAINERS F: include/linux/*fence.h · line 4633
Quoted 5 lines
MAINTAINERS +5
-0+ + if (heap_allocation.fd || + heap_allocation.reserved0 || + heap_allocation.reserved1 || + heap_allocation.reserved2) {Seems too many reserved, I can understand one, but if we ever needed all of these we would be better off just adding another alloc ioctl.
Well, we have to have one u32 for padding. And I figured if we needed anything more then a u32, then we're in for 2 more.
And I think the potential of the alignment and heap-private flags, I worry we might want to have something, but I guess we could just add a new ioctl and keep the support for the old one if folks prefer.
Quoted 10 lines
MAINTAINERS F: include/linux/*fence.h · line 4640
Quoted 7 lines
MAINTAINERS +7
-0+int dma_heap_add(struct dma_heap *heap) +{ + struct device *dev_ret; + int ret; + + if (!heap->name || !strcmp(heap->name, "")) { + pr_err("dma_heap: Cannot add heap without a name\n");As these names end up as the dev name in the file system we may want to check for invalid names, there is probably a helper for that somewhere.
Hrm. I'll have to look.
Quoted 14 lines
MAINTAINERS F: include/linux/*fence.h
Quoted 7 lines
MAINTAINERS +7
-0+struct dma_heap { + const char *name; + struct dma_heap_ops *ops; + unsigned int minor; + dev_t heap_devt; + struct cdev heap_cdev; +};Still not sure about this, all of the members in this struct are strictly internally used by the framework. The users of this framework should not have access to them and only need to deal with an opaque pointer for tracking themselves (can store it in a private struct of their own then
container_ofto get back out their struct).
Quoted 7 lines
Quoted 2 lines
Anyway, not a big deal, and if it really bugs me enough I can always go fix it later, it's all kernel internal so not a blocker here. :)
I guess I'd just move the
include/linux/dma-heap.htodrivers/dma-buf/heaps/and keep it localized there.
But whichever. Feel free to also send a patch and I can fold it down.
The dma-heap.h needs to stay where it is, I was thinking just move struct dma_heap to inside drivers/dma-buf/dma-heap.c. I wouldn't worry about changing anything right now though, I'll post a patch you can squash in later one we confirm this whole dma-heap thing will get deemed acceptable in the first place.
Thanks,
Andrew
Quoted 3 lines
thanks
-john
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [RFC][PATCH 1/5 v2] dma-buf: Add dma-buf heaps framework
In reply to John Stultzdrivers/dma-buf/dma-heap.c line 67
Quoted 6 lines
drivers/dma-buf/dma-heap.c new file +6
-0+static int dma_heap_release (struct inode *inode,struct file *filp) +{ + filp->private_data =NULL ; + +return 0 ; +}
No point in clearing ->private_data, the file is about to be freed.
drivers/dma-buf/dma-heap.c line 70
Quoted 3 lines
drivers/dma-buf/dma-heap.c new file +3
-0+ +static long dma_heap_ioctl (struct file *filp,unsigned int cmd, +unsigned long arg)
Pleae don't use the weird legacy filp naming, file is a perfectly valid and readable default name for struct file pointers.
drivers/dma-buf/dma-heap.c line 77
Quoted 7 lines
drivers/dma-buf/dma-heap.c new file +7
-0+{ +switch (cmd) { +case DMA_HEAP_IOC_ALLOC : + { +struct dma_heap_allocation_data heap_allocation; +struct dma_heap *heap = filp->private_data; +int fd;
Please split each ioctl into a separate function from the very start, otherwise this will grow into a spaghetty mess sooner than you can see cheese.
drivers/dma-buf/dma-heap.c line 156
Quoted 5 lines
drivers/dma-buf/dma-heap.c new file +5
-0+ dev_ret =device_create (dma_heap_class, +NULL , + heap->heap_devt, +NULL , + heap->name);
No need this weird argument alignment.
Re: [RFC][PATCH 1/5 v2] dma-buf: Add dma-buf heaps framework
Quoted 9 lines · Christoph Hellwig
drivers/dma-buf/dma-heap.c line 67
Quoted 6 lines
drivers/dma-buf/dma-heap.c new file +6
-0+static int dma_heap_release (struct inode *inode,struct file *filp) +{ + filp->private_data =NULL ; + +return 0 ; +}No point in clearing ->private_data, the file is about to be freed.
This was leftover from when release had some memory to free, will remove.
Quoted 7 lines
drivers/dma-buf/dma-heap.c line 70
Quoted 3 lines
drivers/dma-buf/dma-heap.c new file +3
-0+ +static long dma_heap_ioctl (struct file *filp,unsigned int cmd, +unsigned long arg)Pleae don't use the weird legacy filp naming, file is a perfectly valid and readable default name for struct file pointers.
Thanks for info, I saw both used and this was used where I found the prototype so I used it too, will fix.
Quoted 12 lines
drivers/dma-buf/dma-heap.c line 77
Quoted 7 lines
drivers/dma-buf/dma-heap.c new file +7
-0+{ +switch (cmd) { +case DMA_HEAP_IOC_ALLOC : + { +struct dma_heap_allocation_data heap_allocation; +struct dma_heap *heap = filp->private_data; +int fd;Please split each ioctl into a separate function from the very start, otherwise this will grow into a spaghetty mess sooner than you can see cheese.
Good idea, will fix.
Quoted 8 lines
drivers/dma-buf/dma-heap.c line 156
Quoted 5 lines
drivers/dma-buf/dma-heap.c new file +5
-0+ dev_ret =device_create (dma_heap_class, +NULL , + heap->heap_devt, +NULL , + heap->name);No need this weird argument alignment.
I kinda like it this way, if everything cant fit on one line then everything gets its own line, seems more consistent. If there is strong objection I can fix.
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [RFC][PATCH 1/5 v2] dma-buf: Add dma-buf heaps framework
In reply to John StultzQuoted 5 lines · John Stultz
MAINTAINERS +5
-0+DMA-BUF HEAPS FRAMEWORK +M: Laura Abbott<[email protected]> +R: Liam Mark<[email protected]> +R: Brian Starkey<[email protected]> +R: "Andrew F. Davis"<[email protected]>
MAINTAINERS F: include/linux/*fence.h · line 4638
Quoted 10 lines
MAINTAINERS +10
-0+R: John Stultz<[email protected]> +S: Maintained +L: [email protected] +L: [email protected] +L: [email protected] (moderated for non-subscribers) +F: include/uapi/linux/dma-heap.h +F: include/linux/dma-heap.h +F: drivers/dma-buf/dma-heap.c +F: drivers/dma-buf/heaps/* +T: git git://anongit.freedesktop.org/drm/drm-misc
So I talked about this with Sumit privately but I think it might make sense to have me step down as maintainer when this goes out of staging. I mostly worked on Ion at my previous position and anything I do now is mostly a side project. I still want to see it succeed which is why I took on the maintainer role but I don't want to become blocking for people who have a stronger vision about where this needs to go (see also, I'm not working with this on a daily basis).
If you just want someone to help review or take patches to be pulled, I'm happy to do so but I'd hate to become the bottleneck on getting things done for people who are attempting to do real work.
Thanks,
Laura
Re: [RFC][PATCH 1/5 v2] dma-buf: Add dma-buf heaps framework
Quoted 26 lines · Laura Abbott
Quoted 5 lines · John Stultz
MAINTAINERS +5
-0+DMA-BUF HEAPS FRAMEWORK +M: Laura Abbott<[email protected]> +R: Liam Mark<[email protected]> +R: Brian Starkey<[email protected]> +R: "Andrew F. Davis"<[email protected]>MAINTAINERS F: include/linux/*fence.h · line 4638
Quoted 10 lines
MAINTAINERS +10
-0+R: John Stultz<[email protected]> +S: Maintained +L: [email protected] +L: [email protected] +L: [email protected] (moderated for non-subscribers) +F: include/uapi/linux/dma-heap.h +F: include/linux/dma-heap.h +F: drivers/dma-buf/dma-heap.c +F: drivers/dma-buf/heaps/* +T: git git://anongit.freedesktop.org/drm/drm-miscSo I talked about this with Sumit privately but I think it might make sense to have me step down as maintainer when this goes out of staging. I mostly worked on Ion at my previous position and anything I do now is mostly a side project. I still want to see it succeed which is why I took on the maintainer role but I don't want to become blocking for people who have a stronger vision about where this needs to go (see also, I'm not working with this on a daily basis).
Quoted 5 lines
If you just want someone to help review or take patches to be pulled, I'm happy to do so but I'd hate to become the bottleneck on getting things done for people who are attempting to do real work.
We could consider this as an "ION inspired" framework, and treat it like an extension of DMA-BUF. In which case Sumit could become the default Maintainer if he's up for it.
Andrew
Quoted 2 lines
Thanks,
Laura
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [RFC][PATCH 1/5 v2] dma-buf: Add dma-buf heaps framework
In reply to Laura AbbottQuoted 27 lines · Laura Abbott
Quoted 5 lines · John Stultz
MAINTAINERS +5
-0+DMA-BUF HEAPS FRAMEWORK +M: Laura Abbott<[email protected]> +R: Liam Mark<[email protected]> +R: Brian Starkey<[email protected]> +R: "Andrew F. Davis"<[email protected]>MAINTAINERS F: include/linux/*fence.h · line 4638
Quoted 10 lines
MAINTAINERS +10
-0+R: John Stultz<[email protected]> +S: Maintained +L: [email protected] +L: [email protected] +L: [email protected] (moderated for non-subscribers) +F: include/uapi/linux/dma-heap.h +F: include/linux/dma-heap.h +F: drivers/dma-buf/dma-heap.c +F: drivers/dma-buf/heaps/* +T: git git://anongit.freedesktop.org/drm/drm-miscSo I talked about this with Sumit privately but I think it might make sense to have me step down as maintainer when this goes out of staging. I mostly worked on Ion at my previous position and anything I do now is mostly a side project. I still want to see it succeed which is why I took on the maintainer role but I don't want to become blocking for people who have a stronger vision about where this needs to go (see also, I'm not working with this on a daily basis).
Quoted 4 lines
If you just want someone to help review or take patches to be pulled, I'm happy to do so but I'd hate to become the bottleneck on getting things done for people who are attempting to do real work.
I worry this will make everyone to touch the side of their nose and yell "NOT IT!" :)
First of all, thank you so much for your efforts maintaining ION along with your attempts to drag out requirements from interested parties and the numerous attempts to get collaborative discussion going at countless conferences! Your persistence and continual nudging in the face of apathetic private users of the code probably cannot be appreciated enough!
Your past practical experience with ION and active work with the upstream community made you a stand out pick for this, but I understand not wanting to be eternally stuck with a maintainership if your not active in the area. I'm happy to volunteer as a neutral party, but I worry my limited experience with some of the more complicated usage would make my opinions less informed then they probably need to be. Further, as a neutral party, Sumit would probably be a better pick since he's already maintaining the dmabuf core.
So I'd nominate Andrew, Liam or Benjamin (or all three?) as they all have more practical experience enabling past ION heaps on real devices and have demonstrated active interest in working in the community.
So, in other words... NOT IT! :)
-john
Re: [RFC][PATCH 1/5 v2] dma-buf: Add dma-buf heaps framework
Quoted 43 lines · John Stultz
Quoted 27 lines · Laura Abbott
Quoted 5 lines · John Stultz
MAINTAINERS +5
-0+DMA-BUF HEAPS FRAMEWORK +M: Laura Abbott<[email protected]> +R: Liam Mark<[email protected]> +R: Brian Starkey<[email protected]> +R: "Andrew F. Davis"<[email protected]>MAINTAINERS F: include/linux/*fence.h · line 4638
Quoted 10 lines
MAINTAINERS +10
-0+R: John Stultz<[email protected]> +S: Maintained +L: [email protected] +L: [email protected] +L: [email protected] (moderated for non-subscribers) +F: include/uapi/linux/dma-heap.h +F: include/linux/dma-heap.h +F: drivers/dma-buf/dma-heap.c +F: drivers/dma-buf/heaps/* +T: git git://anongit.freedesktop.org/drm/drm-miscSo I talked about this with Sumit privately but I think it might make sense to have me step down as maintainer when this goes out of staging. I mostly worked on Ion at my previous position and anything I do now is mostly a side project. I still want to see it succeed which is why I took on the maintainer role but I don't want to become blocking for people who have a stronger vision about where this needs to go (see also, I'm not working with this on a daily basis).
Quoted 4 lines
If you just want someone to help review or take patches to be pulled, I'm happy to do so but I'd hate to become the bottleneck on getting things done for people who are attempting to do real work.
I worry this will make everyone to touch the side of their nose and yell "NOT IT!" :)
First of all, thank you so much for your efforts maintaining ION along with your attempts to drag out requirements from interested parties and the numerous attempts to get collaborative discussion going at countless conferences! Your persistence and continual nudging in the face of apathetic private users of the code probably cannot be appreciated enough!
Quoted 10 lines
Your past practical experience with ION and active work with the upstream community made you a stand out pick for this, but I understand not wanting to be eternally stuck with a maintainership if your not active in the area. I'm happy to volunteer as a neutral party, but I worry my limited experience with some of the more complicated usage would make my opinions less informed then they probably need to be. Further, as a neutral party, Sumit would probably be a better pick since he's already maintaining the dmabuf core.
Honestly if you're doing the work to re-write everything, I think you're more than qualified to be the maintainer. I would also support Sumit as well.
Quoted 4 lines
So I'd nominate Andrew, Liam or Benjamin (or all three?) as they all have more practical experience enabling past ION heaps on real devices and have demonstrated active interest in working in the community.
I do think this would benefit both from multiple maintainers and from maintainers who are actively using the framework. Like I said, I can still be a maintainer but I think having some comaintainers would be very helpful (and I'd support any of the names you've suggested)
Quoted 1 line
So, in other words... NOT IT! :)
I think you have to shout "Noes goes" first. :)
Quoted 2 lines
-john
Thanks,
Laura
P.S. For the benefit of anyone who's confused,
https://en.wikipedia.org/wiki/Nose_goes
Sumit Semwal
maintainerRe: [RFC][PATCH 1/5 v2] dma-buf: Add dma-buf heaps framework
Quoted 37 lines · Laura Abbott
Quoted 36 lines · John Stultz
Quoted 27 lines · Laura Abbott
Quoted 5 lines · John Stultz
MAINTAINERS +5
-0+DMA-BUF HEAPS FRAMEWORK +M: Laura Abbott<[email protected]> +R: Liam Mark<[email protected]> +R: Brian Starkey<[email protected]> +R: "Andrew F. Davis"<[email protected]>MAINTAINERS F: include/linux/*fence.h · line 4638
Quoted 10 lines
MAINTAINERS +10
-0+R: John Stultz<[email protected]> +S: Maintained +L: [email protected] +L: [email protected] +L: [email protected] (moderated for non-subscribers) +F: include/uapi/linux/dma-heap.h +F: include/linux/dma-heap.h +F: drivers/dma-buf/dma-heap.c +F: drivers/dma-buf/heaps/* +T: git git://anongit.freedesktop.org/drm/drm-miscSo I talked about this with Sumit privately but I think it might make sense to have me step down as maintainer when this goes out of staging. I mostly worked on Ion at my previous position and anything I do now is mostly a side project. I still want to see it succeed which is why I took on the maintainer role but I don't want to become blocking for people who have a stronger vision about where this needs to go (see also, I'm not working with this on a daily basis).
Quoted 4 lines
If you just want someone to help review or take patches to be pulled, I'm happy to do so but I'd hate to become the bottleneck on getting things done for people who are attempting to do real work.
I worry this will make everyone to touch the side of their nose and yell "NOT IT!" :)
Quoted 7 lines
Quoted 6 lines
First of all, thank you so much for your efforts maintaining ION along with your attempts to drag out requirements from interested parties and the numerous attempts to get collaborative discussion going at countless conferences! Your persistence and continual nudging in the face of apathetic private users of the code probably cannot be appreciated enough!
I totally second John here - the persistence has been inspiring to me personally as well :)
Quoted 21 lines
Quoted 11 lines
Your past practical experience with ION and active work with the upstream community made you a stand out pick for this, but I understand not wanting to be eternally stuck with a maintainership if your not active in the area. I'm happy to volunteer as a neutral party, but I worry my limited experience with some of the more complicated usage would make my opinions less informed then they probably need to be. Further, as a neutral party, Sumit would probably be a better pick since he's already maintaining the dmabuf core.
Honestly if you're doing the work to re-write everything, I think you're more than qualified to be the maintainer. I would also support Sumit as well.
+ 1Quoted 4 lines
So I'd nominate Andrew, Liam or Benjamin (or all three?) as they all have more practical experience enabling past ION heaps on real devices and have demonstrated active interest in working in the community.
Quoted 6 lines
I do think this would benefit both from multiple maintainers and from maintainers who are actively using the framework. Like I said, I can still be a maintainer but I think having some comaintainers would be very helpful (and I'd support any of the names you've suggested)
If it's required, I am happy to co-maintain - we could even follow the drm-misc model of multiple co-maintainers. I will support any or all of the names above as well :)
Quoted 5 lines
Quoted 2 lines
So, in other words... NOT IT! :)
I think you have to shout "Noes goes" first. :)
Quoted 9 lines
Quoted 2 lines
-john
Thanks,
Laura
P.S. For the benefit of anyone who's confused,
https://en.wikipedia.org/wiki/Nose_goes
Best,
Sumit.
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Sumit Semwal
maintainerRe: [RFC][PATCH 1/5 v2] dma-buf: Add dma-buf heaps framework
In reply to Laura AbbottQuoted 38 lines · Laura Abbott
Quoted 36 lines · John Stultz
Quoted 27 lines · Laura Abbott
Quoted 5 lines · John Stultz
MAINTAINERS +5
-0+DMA-BUF HEAPS FRAMEWORK +M: Laura Abbott<[email protected]> +R: Liam Mark<[email protected]> +R: Brian Starkey<[email protected]> +R: "Andrew F. Davis"<[email protected]>MAINTAINERS F: include/linux/*fence.h · line 4638
Quoted 10 lines
MAINTAINERS +10
-0+R: John Stultz<[email protected]> +S: Maintained +L: [email protected] +L: [email protected] +L: [email protected] (moderated for non-subscribers) +F: include/uapi/linux/dma-heap.h +F: include/linux/dma-heap.h +F: drivers/dma-buf/dma-heap.c +F: drivers/dma-buf/heaps/* +T: git git://anongit.freedesktop.org/drm/drm-miscSo I talked about this with Sumit privately but I think it might make sense to have me step down as maintainer when this goes out of staging. I mostly worked on Ion at my previous position and anything I do now is mostly a side project. I still want to see it succeed which is why I took on the maintainer role but I don't want to become blocking for people who have a stronger vision about where this needs to go (see also, I'm not working with this on a daily basis).
Quoted 4 lines
If you just want someone to help review or take patches to be pulled, I'm happy to do so but I'd hate to become the bottleneck on getting things done for people who are attempting to do real work.
I worry this will make everyone to touch the side of their nose and yell "NOT IT!" :)
Quoted 6 lines
Quoted 6 lines
First of all, thank you so much for your efforts maintaining ION along with your attempts to drag out requirements from interested parties and the numerous attempts to get collaborative discussion going at countless conferences! Your persistence and continual nudging in the face of apathetic private users of the code probably cannot be appreciated enough!
I totally second John here - the persistence has been inspiring to me personally as well :)
Quoted 21 lines
Quoted 11 lines
Your past practical experience with ION and active work with the upstream community made you a stand out pick for this, but I understand not wanting to be eternally stuck with a maintainership if your not active in the area. I'm happy to volunteer as a neutral party, but I worry my limited experience with some of the more complicated usage would make my opinions less informed then they probably need to be. Further, as a neutral party, Sumit would probably be a better pick since he's already maintaining the dmabuf core.
Honestly if you're doing the work to re-write everything, I think you're more than qualified to be the maintainer. I would also support Sumit as well.
Quoted 4 lines
So I'd nominate Andrew, Liam or Benjamin (or all three?) as they all have more practical experience enabling past ION heaps on real devices and have demonstrated active interest in working in the community.
Quoted 5 lines
I do think this would benefit both from multiple maintainers and from maintainers who are actively using the framework. Like I said, I can still be a maintainer but I think having some comaintainers would be very helpful (and I'd support any of the names you've suggested)
If it's required, I am happy to co-maintain - we could even follow the drm-misc model of multiple co-maintainers. I will support any or all of the names above as well :)
Quoted 5 lines
Quoted 2 lines
So, in other words... NOT IT! :)
I think you have to shout "Noes goes" first. :)
Quoted 8 lines
Quoted 2 lines
-john
Thanks,
Laura
P.S. For the benefit of anyone who's confused,
https://en.wikipedia.org/wiki/Nose_goes
Best,
Sumit.
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Brian Starkey
reviewerRe: [RFC][PATCH 1/5 v2] dma-buf: Add dma-buf heaps framework
In reply to John StultzHi John,
Quoted 1 line · John Stultz
From: "Andrew F. Davis" <[email protected]>
[snip]
drivers/dma-buf/dma-heap.c line 26
Quoted 4 lines
drivers/dma-buf/dma-heap.c new file +4
-0+ +#define NUM_HEAP_MINORS 128 +static DEFINE_IDR (dma_heap_idr); +static DEFINE_MUTEX (minor_lock);/* Protect idr accesses */
I saw that Matthew Wilcox is trying to nuke idr:
https://patchwork.freedesktop.org/series/57073/
Perhaps a different data structure could be considered? (I don't have an informed opinion on which).
drivers/dma-buf/dma-heap.c line 38
Quoted 12 lines
drivers/dma-buf/dma-heap.c new file +12
-0+ +dev_t dma_heap_devt; +struct class *dma_heap_class; +struct list_head dma_heap_list; +struct dentry *dma_heap_debug_root; + +static int dma_heap_buffer_alloc (struct dma_heap *heap,size_t len, +unsigned int flags) +{ + len =PAGE_ALIGN (len); +if (!len) +return -EINVAL ;
I think aligning len to pages only makes sense if heaps are going to allocate aligned to pages too. Perhaps that's an implicit assumption? If so, lets document it.
Why not let the heaps take care of aligning len however they want though?
...
Quoted 31 lines
drivers/dma-buf/dma-heap.c new file +31
-0+ +int dma_heap_add (struct dma_heap *heap) +{ +struct device *dev_ret; +int ret; + +if (!heap->name || !strcmp (heap->name,"" )) { +pr_err ("dma_heap: Cannot add heap without a name\n" ); +return -EINVAL ; + } + +if (!heap->ops || !heap->ops->allocate) { +pr_err ("dma_heap: Cannot add heap with invalid ops struct\n" ); +return -EINVAL ; + } + +/* Find unused minor number */ +mutex_lock (&minor_lock); + ret =idr_alloc (&dma_heap_idr, heap,0 ,NUM_HEAP_MINORS ,GFP_KERNEL ); +mutex_unlock (&minor_lock); +if (ret <0 ) { +pr_err ("dma_heap: Unable to get minor number for heap\n" ); +return ret; + } + heap->minor = ret; + +/* Create device */ + heap->heap_devt =MKDEV (MAJOR (dma_heap_devt), heap->minor); + dev_ret = device_create(dma_heap_class, + NULL, + heap->heap_devt,
drivers/dma-buf/dma-heap.c line 164
Quoted 10 lines
drivers/dma-buf/dma-heap.c new file +10
-0+NULL , + heap->name); +if (IS_ERR (dev_ret)) { +pr_err ("dma_heap: Unable to create char device\n" ); +return PTR_ERR (dev_ret); + } + +/* Add device */ +cdev_init (&heap->heap_cdev, &dma_heap_fops); + ret =cdev_add (&heap->heap_cdev, dma_heap_devt,NUM_HEAP_MINORS );
Shouldn't this be s/dma_heap_devt/heap->heap_devt/ and a count of 1?
Also would it be better to have cdev_add/device_create the other way around? First create the char device, then once it's all set up register it with sysfs.
drivers/dma-buf/dma-heap.c line 173
Quoted 9 lines
drivers/dma-buf/dma-heap.c new file +9
-0+if (ret <0 ) { +device_destroy (dma_heap_class, heap->heap_devt); +pr_err ("dma_heap: Unable to add char device\n" ); +return ret; + } + +return 0 ; +} +EXPORT_SYMBOL (dma_heap_add);
Until we've figured out how modules are going to work, I still think it would be a good idea to not export this.
Cheers,
-Brian
Re: [RFC][PATCH 1/5 v2] dma-buf: Add dma-buf heaps framework
Quoted 15 lines · Brian Starkey
Hi John,
Quoted 1 line · John Stultz
From: "Andrew F. Davis" <[email protected]>
[snip]
drivers/dma-buf/dma-heap.c line 26
Quoted 4 lines
drivers/dma-buf/dma-heap.c new file +4
-0+ +#define NUM_HEAP_MINORS 128 +static DEFINE_IDR (dma_heap_idr); +static DEFINE_MUTEX (minor_lock);/* Protect idr accesses */ I saw that Matthew Wilcox is trying to nuke idr:
https://patchwork.freedesktop.org/series/57073/
Quoted 3 lines
Perhaps a different data structure could be considered? (I don't have an informed opinion on which).
Looks like XArray is the suggested replacement. Should be easy enough, the minor number would just index to our heap struct directly, I'll give it a shot and see.
Quoted 17 lines
drivers/dma-buf/dma-heap.c line 38
Quoted 12 lines
drivers/dma-buf/dma-heap.c new file +12
-0+ +dev_t dma_heap_devt; +struct class *dma_heap_class; +struct list_head dma_heap_list; +struct dentry *dma_heap_debug_root; + +static int dma_heap_buffer_alloc (struct dma_heap *heap,size_t len, +unsigned int flags) +{ + len =PAGE_ALIGN (len); +if (!len) +return -EINVAL ;I think aligning len to pages only makes sense if heaps are going to allocate aligned to pages too. Perhaps that's an implicit assumption? If so, lets document it.
Quoted 3 lines
Why not let the heaps take care of aligning len however they want though?
This is how I originally had it, but I think we couldn't find any case where you would want an the start or end of a buffer to not fall on a page boundary here. It would only lead to problems. As you say though, nothing keeping us from moving that into the heaps themselves.
Quoted 37 lines
...
Quoted 35 lines
drivers/dma-buf/dma-heap.c new file +35
-0+ +int dma_heap_add (struct dma_heap *heap) +{ +struct device *dev_ret; +int ret; + +if (!heap->name || !strcmp (heap->name,"" )) { +pr_err ("dma_heap: Cannot add heap without a name\n" ); +return -EINVAL ; + } + +if (!heap->ops || !heap->ops->allocate) { +pr_err ("dma_heap: Cannot add heap with invalid ops struct\n" ); +return -EINVAL ; + } + +/* Find unused minor number */ +mutex_lock (&minor_lock); + ret =idr_alloc (&dma_heap_idr, heap,0 ,NUM_HEAP_MINORS ,GFP_KERNEL ); +mutex_unlock (&minor_lock); +if (ret <0 ) { +pr_err ("dma_heap: Unable to get minor number for heap\n" ); +return ret; + } + heap->minor = ret; + +/* Create device */ + heap->heap_devt =MKDEV (MAJOR (dma_heap_devt), heap->minor); + dev_ret =device_create (dma_heap_class, +NULL , + heap->heap_devt, +NULL , + heap->name); +if (IS_ERR (dev_ret)) { +pr_err ("dma_heap: Unable to create char device\n" );
Quoted 9 lines
drivers/dma-buf/dma-heap.c line 164
Quoted 6 lines
drivers/dma-buf/dma-heap.c new file +6
-0+return PTR_ERR (dev_ret); + } + +/* Add device */ +cdev_init (&heap->heap_cdev, &dma_heap_fops); + ret =cdev_add (&heap->heap_cdev, dma_heap_devt,NUM_HEAP_MINORS );Shouldn't this be s/dma_heap_devt/heap->heap_devt/ and a count of 1?
Hmm, strange this ever worked before..
Quoted 4 lines
Also would it be better to have cdev_add/device_create the other way around? First create the char device, then once it's all set up register it with sysfs.
Yes that does seem to be more common, lets flip it.
Quoted 4 lines
Quoted 4 lines
drivers/dma-buf/dma-heap.c new file +4
-0+if (ret <0 ) { +device_destroy (dma_heap_class, heap->heap_devt); +pr_err ("dma_heap: Unable to add char device\n" ); +return ret;
Quoted 9 lines
drivers/dma-buf/dma-heap.c line 173
Quoted 5 lines
drivers/dma-buf/dma-heap.c new file +5
-0+ } + +return 0 ; +} +EXPORT_SYMBOL (dma_heap_add);Until we've figured out how modules are going to work, I still think it would be a good idea to not export this.
Agree.
Andrew
Quoted 3 lines
Cheers,
-Brian
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [RFC][PATCH 1/5 v2] dma-buf: Add dma-buf heaps framework
In reply to Brian StarkeyQuoted 16 lines · Brian Starkey
Hi John,
Quoted 1 line · John Stultz
From: "Andrew F. Davis" <[email protected]>
[snip]
drivers/dma-buf/dma-heap.c line 26
Quoted 4 lines
drivers/dma-buf/dma-heap.c new file +4
-0+ +#define NUM_HEAP_MINORS 128 +static DEFINE_IDR (dma_heap_idr); +static DEFINE_MUTEX (minor_lock);/* Protect idr accesses */ I saw that Matthew Wilcox is trying to nuke idr:
https://patchwork.freedesktop.org/series/57073/
Quoted 2 lines
Perhaps a different data structure could be considered? (I don't have an informed opinion on which).
Thanks for pointing this out! I've just switched to using the Xarray implementation in my tree.
Quoted 10 lines
drivers/dma-buf/dma-heap.c line 38
Quoted 6 lines
drivers/dma-buf/dma-heap.c new file +6
-0+static int dma_heap_buffer_alloc (struct dma_heap *heap,size_t len, +unsigned int flags) +{ + len =PAGE_ALIGN (len); +if (!len) +return -EINVAL ;I think aligning len to pages only makes sense if heaps are going to allocate aligned to pages too. Perhaps that's an implicit assumption? If so, lets document it.
I've added a comment as such (or do you have more thoughts on where it should be documented?), and for consistency removed the PAGE_ALIGN usage in the heap allocator hooks.
Quoted 2 lines
Why not let the heaps take care of aligning len however they want though?
As Andrew already said, It seems page granularity would have to be the finest allocation granularity for dmabufs. If heaps want to implement their own larger granularity alignment, I don't see any reason they would be limited there.
And for me, its mostly because I stubbed my toe implementing the heap code w/ the first patch that didn't have the page alignment in the generic code. :)
Quoted 16 lines
Quoted 5 lines
drivers/dma-buf/dma-heap.c new file +5
-0+/* Create device */ + heap->heap_devt =MKDEV (MAJOR (dma_heap_devt), heap->minor); + dev_ret = device_create(dma_heap_class, + NULL, + heap->heap_devt,drivers/dma-buf/dma-heap.c line 164
Quoted 10 lines
drivers/dma-buf/dma-heap.c new file +10
-0+NULL , + heap->name); +if (IS_ERR (dev_ret)) { +pr_err ("dma_heap: Unable to create char device\n" ); +return PTR_ERR (dev_ret); + } + +/* Add device */ +cdev_init (&heap->heap_cdev, &dma_heap_fops); + ret =cdev_add (&heap->heap_cdev, dma_heap_devt,NUM_HEAP_MINORS );
Quoted 5 lines
Shouldn't this be s/dma_heap_devt/heap->heap_devt/ and a count of 1?
Also would it be better to have cdev_add/device_create the other way around? First create the char device, then once it's all set up register it with sysfs.
Thanks for catching that! Much appreciated! Reworked as suggested.
Though I realized last week I have not figured out a consistent way to have the heaps show up in /dev/dma_heaps/<device> on both Android and classic Linux environments. I need to go stare at the /dev/input/ setup code some more.
Quoted 4 lines
Quoted 4 lines
drivers/dma-buf/dma-heap.c new file +4
-0+if (ret <0 ) { +device_destroy (dma_heap_class, heap->heap_devt); +pr_err ("dma_heap: Unable to add char device\n" ); +return ret;
Quoted 8 lines
drivers/dma-buf/dma-heap.c line 173
Quoted 5 lines
drivers/dma-buf/dma-heap.c new file +5
-0+ } + +return 0 ; +} +EXPORT_SYMBOL (dma_heap_add);Until we've figured out how modules are going to work, I still think it would be a good idea to not export this.
Done!
thanks
-john
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [RFC][PATCH 1/5 v2] dma-buf: Add dma-buf heaps framework
In reply to John StultzQuoted 6 lines · John Stultz
From: "Andrew F. Davis" <[email protected]>
This framework allows a unified userspace interface for dma-buf exporters, allowing userland to allocate specific types of memory for use in dma-buf sharing.
Quoted 8 lines
Each heap is given its own device node, which a user can allocate a dma-buf fd from using the
DMA_HEAP_IOC_ALLOC.This code is an evoluiton of the Android ION implementation, and a big thanks is due to its authors/maintainers over time for their effort:
Rebecca Schultz Zavin, Colin Cross, Benjamin Gaignard, Laura Abbott, and many other contributors!
Comments just on the user/kernel api and how it interacts with the driver model. Not on the "real" functionality of this code :)
drivers/dma-buf/dma-heap.c line 24
Quoted 3 lines
drivers/dma-buf/dma-heap.c new file +3
-0+#define DEVNAME "dma_heap" + +#define NUM_HEAP_MINORS 128
Why a max?
drivers/dma-buf/dma-heap.c line 26
Quoted 2 lines
drivers/dma-buf/dma-heap.c new file +2
-0+static DEFINE_IDR (dma_heap_idr); +static DEFINE_MUTEX (minor_lock);/* Protect idr accesses */
Move to use xarray now so that Matthew doesn't have to send patches converting this code later :)
It also allows you to drop the mutex.
drivers/dma-buf/dma-heap.c line 31
Quoted 5 lines
drivers/dma-buf/dma-heap.c new file +5
-0+ +dev_t dma_heap_devt; +struct class *dma_heap_class; +struct list_head dma_heap_list; +struct dentry *dma_heap_debug_root;
Global variables?
Quoted 23 lines
drivers/dma-buf/dma-heap.c new file +23
-0+ +static int dma_heap_buffer_alloc (struct dma_heap *heap,size_t len, +unsigned int flags) +{ + len =PAGE_ALIGN (len); +if (!len) +return -EINVAL ; + +return heap->ops->allocate (heap, len, flags); +} + +static int dma_heap_open (struct inode *inode,struct file *filp) +{ +struct dma_heap *heap; + +mutex_lock (&minor_lock); + heap =idr_find (&dma_heap_idr,iminor (inode)); +mutex_unlock (&minor_lock); +if (!heap) { +pr_err ("dma_heap: minor %d unknown.\n" ,iminor (inode)); +return -ENODEV ; + } +
drivers/dma-buf/dma-heap.c line 64
Quoted 10 lines
drivers/dma-buf/dma-heap.c new file +10
-0+/* instance data as context */ + filp->private_data = heap; +nonseekable_open (inode, filp); + +return 0 ; +} + +static int dma_heap_release (struct inode *inode,struct file *filp) +{ + filp->private_data =NULL ;
Why does this matter? release should only be called on the way out of here, no need to do anything as nothing else can be called, right?
release shouldn't be needed from what I can tell.
Quoted 15 lines
drivers/dma-buf/dma-heap.c new file +15
-0+ +return 0 ; +} + +static long dma_heap_ioctl (struct file *filp,unsigned int cmd, +unsigned long arg) +{ +switch (cmd) { +case DMA_HEAP_IOC_ALLOC : + { +struct dma_heap_allocation_data heap_allocation; +struct dma_heap *heap = filp->private_data; +int fd; + + if (copy_from_user(&heap_allocation, (void __user *)arg,
drivers/dma-buf/dma-heap.c line 89
Quoted 10 lines
drivers/dma-buf/dma-heap.c new file +10
-0+ sizeof(heap_allocation))) +return -EFAULT ; + +if (heap_allocation.fd || + heap_allocation.reserved0 || + heap_allocation.reserved1 || + heap_allocation.reserved2) { +pr_warn_once ("dma_heap: ioctl data not valid\n" ); +return -EINVAL ; + }
Good job in forcing the reserved fields to be 0!
drivers/dma-buf/dma-heap.c line 96
Quoted 7 lines
drivers/dma-buf/dma-heap.c new file +7
-0+if (heap_allocation.flags & ~DMA_HEAP_VALID_FLAGS ) { +pr_warn_once ("dma_heap: flags has invalid or unsupported flags set\n" ); +return -EINVAL ; + } + + fd =dma_heap_buffer_alloc (heap, heap_allocation.len, + heap_allocation.flags);
No max value checking for .len? Can you really ask for anything?
Quoted 16 lines
drivers/dma-buf/dma-heap.c new file +16
-0+if (fd <0 ) +return fd; + + heap_allocation.fd = fd; + +if (copy_to_user ((void __user *)arg, &heap_allocation, +sizeof (heap_allocation))) +return -EFAULT ; + +break ; + } +default : +return -ENOTTY ; + } + +return 0 ;
drivers/dma-buf/dma-heap.c line 122
Quoted 10 lines
drivers/dma-buf/dma-heap.c new file +10
-0+} + +static const struct file_operations dma_heap_fops = { + .owner =THIS_MODULE , + .open = dma_heap_open, + .release = dma_heap_release, + .unlocked_ioctl = dma_heap_ioctl, +#ifdef CONFIG_COMPAT + .compat_ioctl = dma_heap_ioctl, +#endif
Why is compat_ioctl even needed?
Quoted 11 lines
drivers/dma-buf/dma-heap.c new file +11
-0+}; + +int dma_heap_add (struct dma_heap *heap) +{ +struct device *dev_ret; +int ret; + +if (!heap->name || !strcmp (heap->name,"" )) { +pr_err ("dma_heap: Cannot add heap without a name\n" ); +return -EINVAL ; + }
drivers/dma-buf/dma-heap.c line 143
Quoted 10 lines
drivers/dma-buf/dma-heap.c new file +10
-0+ +if (!heap->ops || !heap->ops->allocate) { +pr_err ("dma_heap: Cannot add heap with invalid ops struct\n" ); +return -EINVAL ; + } + +/* Find unused minor number */ +mutex_lock (&minor_lock); + ret =idr_alloc (&dma_heap_idr, heap,0 ,NUM_HEAP_MINORS ,GFP_KERNEL ); +mutex_unlock (&minor_lock);
Again, xarray.
But I will ask you to back up, why need a major number at all? Why not just use the misc subsystem? How many of these are you going to have over time in a "normal" system? How about a "abnormal system"?
We have seen people running Android in "containers" such that they needed binderfs to handle huge numbers of individual android systems running at the same time. Will this api break those systems if you have a tiny maximum number you an allocate?
drivers/dma-buf/dma-heap.c line 156
Quoted 13 lines
drivers/dma-buf/dma-heap.c new file +13
-0+if (ret <0 ) { +pr_err ("dma_heap: Unable to get minor number for heap\n" ); +return ret; + } + heap->minor = ret; + +/* Create device */ + heap->heap_devt =MKDEV (MAJOR (dma_heap_devt), heap->minor); + dev_ret =device_create (dma_heap_class, +NULL , + heap->heap_devt, +NULL , + heap->name);
No parent? Can't hang this off of anything? Ok, having it show up in /sys/devices/virtual/ is probably good enough.
Quoted 7 lines
drivers/dma-buf/dma-heap.c new file +7
-0+if (IS_ERR (dev_ret)) { +pr_err ("dma_heap: Unable to create char device\n" ); +return PTR_ERR (dev_ret); + } + +/* Add device */ +cdev_init (&heap->heap_cdev, &dma_heap_fops);
drivers/dma-buf/dma-heap.c line 173
Quoted 10 lines
drivers/dma-buf/dma-heap.c new file +10
-0+ ret =cdev_add (&heap->heap_cdev, dma_heap_devt,NUM_HEAP_MINORS ); +if (ret <0 ) { +device_destroy (dma_heap_class, heap->heap_devt); +pr_err ("dma_heap: Unable to add char device\n" ); +return ret; + } + +return 0 ; +} +EXPORT_SYMBOL (dma_heap_add);
EXPORT_SYMBOL_GPL() please? For core stuff like this it's good.
Quoted 8 lines
drivers/dma-buf/dma-heap.c new file +8
-0+ +static int dma_heap_init (void ) +{ +int ret; + + ret =alloc_chrdev_region (&dma_heap_devt,0 ,NUM_HEAP_MINORS ,DEVNAME ); +if (ret) +return ret;
drivers/dma-buf/dma-heap.c line 191
Quoted 10 lines
drivers/dma-buf/dma-heap.c new file +10
-0+ + dma_heap_class =class_create (THIS_MODULE ,DEVNAME ); +if (IS_ERR (dma_heap_class)) { +unregister_chrdev_region (dma_heap_devt,NUM_HEAP_MINORS ); +return PTR_ERR (dma_heap_class); + } + +return 0 ; +} +subsys_initcall (dma_heap_init);
Overall, looks sane, the comments above are all really minor.
include/uapi/linux/dma-heap.h line 1
Quoted 4 lines
include/uapi/linux/dma-heap.h new file +1
-0@@ -0,0 +1,52 @@ +/* SPDX-License-Identifier: GPL-2.0 */
Wrong license for a uapi .h file :(
thanks,
greg k-h
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [RFC][PATCH 1/5 v2] dma-buf: Add dma-buf heaps framework
Quoted 11 lines · Greg KH
Quoted 9 lines · John Stultz
From: "Andrew F. Davis" <[email protected]>
This framework allows a unified userspace interface for dma-buf exporters, allowing userland to allocate specific types of memory for use in dma-buf sharing.
Each heap is given its own device node, which a user can allocate a dma-buf fd from using the
DMA_HEAP_IOC_ALLOC.
Quoted 8 lines
Quoted 5 lines
This code is an evoluiton of the Android ION implementation, and a big thanks is due to its authors/maintainers over time for their effort:
Rebecca Schultz Zavin, Colin Cross, Benjamin Gaignard, Laura Abbott, and many other contributors!Comments just on the user/kernel api and how it interacts with the driver model. Not on the "real" functionality of this code :)
Thanks so much for the feedback! In some cases Andrew and I have already made the changes you've suggested, and hopefully will have a new version to share soon.
Quoted 5 lines
drivers/dma-buf/dma-heap.c line 24
Quoted 3 lines
drivers/dma-buf/dma-heap.c new file +3
-0+#define DEVNAME "dma_heap" + +#define NUM_HEAP_MINORS 128Why a max?
Mostly because other drivers do. I'll see if this can be removed with the Xarray bits.
Quoted 7 lines
drivers/dma-buf/dma-heap.c line 26
Quoted 2 lines
drivers/dma-buf/dma-heap.c new file +2
-0+static DEFINE_IDR (dma_heap_idr); +static DEFINE_MUTEX (minor_lock);/* Protect idr accesses */ Move to use xarray now so that Matthew doesn't have to send patches converting this code later :)
It also allows you to drop the mutex.
Yep. Already converted to Xarray, it is nicer!
Quoted 6 lines
drivers/dma-buf/dma-heap.c line 31
Quoted 4 lines
drivers/dma-buf/dma-heap.c new file +4
-0+dev_t dma_heap_devt; +struct class *dma_heap_class; +struct list_head dma_heap_list; +struct dentry *dma_heap_debug_root;Global variables?
Oops. Will make those static. Thanks!
Quoted 37 lines
Quoted 23 lines
drivers/dma-buf/dma-heap.c new file +23
-0+ +static int dma_heap_buffer_alloc (struct dma_heap *heap,size_t len, +unsigned int flags) +{ + len =PAGE_ALIGN (len); +if (!len) +return -EINVAL ; + +return heap->ops->allocate (heap, len, flags); +} + +static int dma_heap_open (struct inode *inode,struct file *filp) +{ +struct dma_heap *heap; + +mutex_lock (&minor_lock); + heap =idr_find (&dma_heap_idr,iminor (inode)); +mutex_unlock (&minor_lock); +if (!heap) { +pr_err ("dma_heap: minor %d unknown.\n" ,iminor (inode)); +return -ENODEV ; + } +drivers/dma-buf/dma-heap.c line 64
Quoted 10 lines
drivers/dma-buf/dma-heap.c new file +10
-0+/* instance data as context */ + filp->private_data = heap; +nonseekable_open (inode, filp); + +return 0 ; +} + +static int dma_heap_release (struct inode *inode,struct file *filp) +{ + filp->private_data =NULL ;Why does this matter? release should only be called on the way out of here, no need to do anything as nothing else can be called, right?
Quoted 1 line
release shouldn't be needed from what I can tell.
Yep. Christoph suggested the same and its been removed already.
Quoted 9 lines
drivers/dma-buf/dma-heap.c line 96
Quoted 7 lines
drivers/dma-buf/dma-heap.c new file +7
-0+if (heap_allocation.flags & ~DMA_HEAP_VALID_FLAGS ) { +pr_warn_once ("dma_heap: flags has invalid or unsupported flags set\n" ); +return -EINVAL ; + } + + fd =dma_heap_buffer_alloc (heap, heap_allocation.len, + heap_allocation.flags);No max value checking for .len? Can you really ask for anything?
So I think any length caps would be heap specific, so we want to pass them on here.
Quoted 10 lines
drivers/dma-buf/dma-heap.c line 122
Quoted 8 lines
drivers/dma-buf/dma-heap.c new file +8
-0+static const struct file_operations dma_heap_fops = { + .owner =THIS_MODULE , + .open = dma_heap_open, + .release = dma_heap_release, + .unlocked_ioctl = dma_heap_ioctl, +#ifdef CONFIG_COMPAT + .compat_ioctl = dma_heap_ioctl, +#endifWhy is
compat_ioctleven needed?
Probably my mistake. I didn't realize if we're running 32bit on 64bit and there's no compat, the unlocked_ioctl gets called.
Quoted 6 lines
drivers/dma-buf/dma-heap.c line 143
Quoted 4 lines
drivers/dma-buf/dma-heap.c new file +4
-0+/* Find unused minor number */ +mutex_lock (&minor_lock); + ret =idr_alloc (&dma_heap_idr, heap,0 ,NUM_HEAP_MINORS ,GFP_KERNEL ); +mutex_unlock (&minor_lock);Again, xarray.
Ack.
Quoted 3 lines
But I will ask you to back up, why need a major number at all? Why not just use the misc subsystem? How many of these are you going to have over time in a "normal" system? How about a "abnormal system"?
So early implementations did use misc, but in order to get the /dev/heap/cma_heap style directories, in both Android and classic udev linux systems I had to create a class.
This v2 patch didn't get it quite right (got it working properly in android but not on classic systems), but the next version does get the subdir created properly (similar to how the input layer does it).
As for number of heaps, I wouldn't expect there to be a ton on any given system. Most likely less then 16, but possibly up to 32. 128 seemed like a safe "crazy out there" cap. But perspectives on crazy shift over time :)
Quoted 4 lines
We have seen people running Android in "containers" such that they needed binderfs to handle huge numbers of individual android systems running at the same time. Will this api break those systems if you have a tiny maximum number you an allocate?
I'd have to think some more on this. Right now I'd expect that you'd not be trying to virtualize the heaps in a container so you'd not have m heaps * n containers on the system. Instead the containers would mount/link in the devnode (I'm a bit fuzzy on how containers handle devnode creation/restrictions) as appropriate (the nice part with this over ION is we have per heap dev nodes, so the set shared can be limited). But I'd have to think more about the risks of how multiple containers might share things like cma heaps.
Quoted 28 lines
drivers/dma-buf/dma-heap.c line 156
Quoted 13 lines
drivers/dma-buf/dma-heap.c new file +13
-0+if (ret <0 ) { +pr_err ("dma_heap: Unable to get minor number for heap\n" ); +return ret; + } + heap->minor = ret; + +/* Create device */ + heap->heap_devt =MKDEV (MAJOR (dma_heap_devt), heap->minor); + dev_ret =device_create (dma_heap_class, +NULL , + heap->heap_devt, +NULL , + heap->name);No parent? Can't hang this off of anything? Ok, having it show up in
/sys/devices/virtual/is probably good enough.Quoted 11 lines
drivers/dma-buf/dma-heap.c new file +11
-0+if (IS_ERR (dev_ret)) { +pr_err ("dma_heap: Unable to create char device\n" ); +return PTR_ERR (dev_ret); + } + +/* Add device */ +cdev_init (&heap->heap_cdev, &dma_heap_fops); + ret =cdev_add (&heap->heap_cdev, dma_heap_devt,NUM_HEAP_MINORS ); +if (ret <0 ) { +device_destroy (dma_heap_class, heap->heap_devt); +pr_err ("dma_heap: Unable to add char device\n" );
Quoted 8 lines
drivers/dma-buf/dma-heap.c line 173
Quoted 6 lines
drivers/dma-buf/dma-heap.c new file +6
-0+return ret; + } + +return 0 ; +} +EXPORT_SYMBOL (dma_heap_add);
EXPORT_SYMBOL_GPL()please? For core stuff like this it's good.
Actually, removed the export completely for now since its probably not ready for modules yet. But will be sure to tag it GPL when we do re-add it.
Quoted 12 lines
Quoted 12 lines
drivers/dma-buf/dma-heap.c new file +12
-0+ +static int dma_heap_init (void ) +{ +int ret; + + ret =alloc_chrdev_region (&dma_heap_devt,0 ,NUM_HEAP_MINORS ,DEVNAME ); +if (ret) +return ret; + + dma_heap_class =class_create (THIS_MODULE ,DEVNAME ); +if (IS_ERR (dma_heap_class)) { +unregister_chrdev_region (dma_heap_devt,NUM_HEAP_MINORS );
Quoted 8 lines
drivers/dma-buf/dma-heap.c line 191
Quoted 6 lines
drivers/dma-buf/dma-heap.c new file +6
-0+return PTR_ERR (dma_heap_class); + } + +return 0 ; +} +subsys_initcall (dma_heap_init);Overall, looks sane, the comments above are all really minor.
Very much appreciate the review!
Quoted 7 lines
include/uapi/linux/dma-heap.h line 1
Quoted 5 lines
include/uapi/linux/dma-heap.h new file +1
-0@@ -0,0 +1,52 @@ +/* SPDX-License-Identifier: GPL-2.0 */ Wrong license for a uapi .h file :(
Ack. Fixed.
Thanks so much again!
-john
RFC PATCH v2 2/5 dma-buf: heaps: Add heap helpers
In reply to John StultzAdd generic helper dmabuf ops for dma heaps, so we can reduce the amount of duplicative code for the exported dmabufs.
This code is an evolution of the Android ION implementation, so thanks to its original authors and maintainters:
Rebecca Schultz Zavin, Colin Cross, Laura Abbott, and others!
---
v2:
* Removed cache management performance hack that I had accidentally folded in.
* Removed stats code that was in helpers
* Lots of checkpatch cleanups
--- drivers/dma-buf/Makefile | 1 + drivers/dma-buf/heaps/Makefile | 2 + drivers/dma-buf/heaps/heap-helpers.c | 335 +++++++++++++++++++++++++++++++++++ drivers/dma-buf/heaps/heap-helpers.h | 48 +++++ 4 files changed, 386 insertions(+) create mode 100644 drivers/dma-buf/heaps/Makefile create mode 100644 drivers/dma-buf/heaps/heap-helpers.c create mode 100644 drivers/dma-buf/heaps/heap-helpers.h
drivers/dma-buf/Makefile +1 -0
@@ -1,4 +1,5 @@ obj-y := dma-buf.o dma-fence.o dma-fence-array.o reservation.o seqno-fence.o +obj-$(CONFIG_DMABUF_HEAPS) += heaps/ obj-$(CONFIG_DMABUF_HEAPS) += dma-heap.o obj-$(CONFIG_SYNC_FILE) += sync_file.o obj-$(CONFIG_SW_SYNC) += sw_sync.o sync_debug.o
drivers/dma-buf/heaps/Makefile new file +2 -0
@@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-y += heap-helpers.o
drivers/dma-buf/heaps/heap-helpers.c new file +335 -0
335 lines not on this page · raw message
drivers/dma-buf/heaps/heap-helpers.h new file +48 -0
48 lines not on this page · raw message
Signature · 6 lines
2.7.4 _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [RFC][PATCH 2/5 v2] dma-buf: heaps: Add heap helpers
Quoted 271 lines · John Stultz
Add generic helper dmabuf ops for dma heaps, so we can reduce the amount of duplicative code for the exported dmabufs.
This code is an evolution of the Android ION implementation, so thanks to its original authors and maintainters:
Rebecca Schultz Zavin, Colin Cross, Laura Abbott, and others!Cc Laura Abbott <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Greg KH <[email protected]>Cc Sumit Semwal <[email protected]>Cc Liam Mark <[email protected]>Cc Brian Starkey <[email protected]>Cc Andrew F. Davis <[email protected]>Cc Chenbo Feng <[email protected]>Cc Alistair Strachan <[email protected]>Signed-off-by John Stultz <[email protected]>---
v2:
* Removed cache management performance hack that I had accidentally folded in.
* Removed stats code that was in helpers
* Lots of checkpatch cleanups--- drivers/dma-buf/Makefile | 1 + drivers/dma-buf/heaps/Makefile | 2 + drivers/dma-buf/heaps/heap-helpers.c | 335 +++++++++++++++++++++++++++++++++++ drivers/dma-buf/heaps/heap-helpers.h | 48 +++++ 4 files changed, 386 insertions(+) create mode 100644 drivers/dma-buf/heaps/Makefile create mode 100644 drivers/dma-buf/heaps/heap-helpers.c create mode 100644 drivers/dma-buf/heaps/heap-helpers.hdrivers/dma-buf/Makefile +1
-0@@ -1,4 +1,5 @@ obj-y := dma-buf.o dma-fence.o dma-fence-array.o reservation.o seqno-fence.o +obj-$(CONFIG_DMABUF_HEAPS) += heaps/ obj-$(CONFIG_DMABUF_HEAPS) += dma-heap.o obj-$(CONFIG_SYNC_FILE) += sync_file.o obj-$(CONFIG_SW_SYNC) += sw_sync.o sync_debug.odrivers/dma-buf/heaps/Makefile new file +2
-0@@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-y += heap-helpers.odrivers/dma-buf/heaps/heap-helpers.c new file +213
-0@@ -0,0 +1,335 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/device.h> +#include <linux/dma-buf.h> +#include <linux/err.h> +#include <linux/idr.h> +#include <linux/list.h> +#include <linux/slab.h> +#include <linux/uaccess.h> +#include <uapi/linux/dma-heap.h> + +#include "heap-helpers.h" + + +static void *dma_heap_map_kernel (struct heap_helper_buffer *buffer) +{ +struct scatterlist *sg; +int i, j; +void *vaddr; +pgprot_t pgprot; +struct sg_table *table = buffer->sg_table; +int npages =PAGE_ALIGN (buffer->heap_buffer.size) /PAGE_SIZE ; +struct page **pages =vmalloc (array_size (npages, +sizeof (struct page *))); +struct page **tmp = pages; + +if (!pages) +return ERR_PTR (-ENOMEM ); + + pgprot =PAGE_KERNEL ; + +for_each_sg (table->sgl, sg, table->nents, i) { +int npages_this_entry =PAGE_ALIGN (sg->length) /PAGE_SIZE ; +struct page *page =sg_page (sg); + +WARN_ON (i >= npages); +for (j =0 ; j < npages_this_entry; j++) + *(tmp++) = page++; + } + vaddr =vmap (pages, npages,VM_MAP , pgprot); +vfree (pages); + +if (!vaddr) +return ERR_PTR (-ENOMEM ); + +return vaddr; +} + +static int dma_heap_map_user (struct heap_helper_buffer *buffer, +struct vm_area_struct *vma) +{ +struct sg_table *table = buffer->sg_table; +unsigned long addr = vma->vm_start; +unsigned long offset = vma->vm_pgoff *PAGE_SIZE ; +struct scatterlist *sg; +int i; +int ret; + +for_each_sg (table->sgl, sg, table->nents, i) { +struct page *page =sg_page (sg); +unsigned long remainder = vma->vm_end - addr; +unsigned long len = sg->length; + +if (offset >= sg->length) { + offset -= sg->length; +continue ; + }else if (offset) { + page += offset /PAGE_SIZE ; + len = sg->length - offset; + offset =0 ; + } + len =min (len, remainder); + ret =remap_pfn_range (vma, addr,page_to_pfn (page), len, + vma->vm_page_prot); +if (ret) +return ret; + addr += len; +if (addr >= vma->vm_end) +return 0 ; + } + +return 0 ; +} + + +void dma_heap_buffer_destroy (struct dma_heap_buffer *heap_buffer) +{ +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + +if (buffer->kmap_cnt >0 ) { +pr_warn_once ("%s: buffer still mapped in the kernel\n" , + __func__); +vunmap (buffer->vaddr); + } + + buffer->free (buffer); +} + +static void *dma_heap_buffer_kmap_get (struct dma_heap_buffer *heap_buffer) +{ +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); +void *vaddr; + +if (buffer->kmap_cnt) { + buffer->kmap_cnt++; +return buffer->vaddr; + } + vaddr =dma_heap_map_kernel (buffer); +if (WARN_ONCE (!vaddr, +"heap->ops->map_kernel should return ERR_PTR on error" )) +return ERR_PTR (-EINVAL ); +if (IS_ERR (vaddr)) +return vaddr; + buffer->vaddr = vaddr; + buffer->kmap_cnt++; +return vaddr; +} + +static void dma_heap_buffer_kmap_put (struct dma_heap_buffer *heap_buffer) +{ +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + + buffer->kmap_cnt--; +if (!buffer->kmap_cnt) { +vunmap (buffer->vaddr); + buffer->vaddr =NULL ; + } +} + +static struct sg_table *dup_sg_table (struct sg_table *table) +{ +struct sg_table *new_table; +int ret, i; +struct scatterlist *sg, *new_sg; + + new_table =kzalloc (sizeof (*new_table),GFP_KERNEL ); +if (!new_table) +return ERR_PTR (-ENOMEM ); + + ret =sg_alloc_table (new_table, table->nents,GFP_KERNEL ); +if (ret) { +kfree (new_table); +return ERR_PTR (-ENOMEM ); + } + + new_sg = new_table->sgl; +for_each_sg (table->sgl, sg, table->nents, i) { +memcpy (new_sg, sg,sizeof (*sg)); + new_sg->dma_address =0 ; + new_sg =sg_next (new_sg); + } + +return new_table; +} + +static void free_duped_table (struct sg_table *table) +{ +sg_free_table (table); +kfree (table); +} + +struct dma_heaps_attachment { +struct device *dev; +struct sg_table *table; +struct list_head list; +}; + +static int dma_heap_attach (struct dma_buf *dmabuf, +struct dma_buf_attachment *attachment) +{ +struct dma_heaps_attachment *a; +struct sg_table *table; +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + + a =kzalloc (sizeof (*a),GFP_KERNEL ); +if (!a) +return -ENOMEM ; + + table =dup_sg_table (buffer->sg_table); +if (IS_ERR (table)) { +kfree (a); +return -ENOMEM ; + } + + a->table = table; + a->dev = attachment->dev; +INIT_LIST_HEAD (&a->list); + + attachment->priv = a; + +mutex_lock (&buffer->lock); +list_add (&a->list, &buffer->attachments); +mutex_unlock (&buffer->lock); + +return 0 ; +} + +static void dma_heap_detatch (struct dma_buf *dmabuf, +struct dma_buf_attachment *attachment) +{ +struct dma_heaps_attachment *a = attachment->priv; +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + +mutex_lock (&buffer->lock); +list_del (&a->list); +mutex_unlock (&buffer->lock); +free_duped_table (a->table); + +kfree (a); +} + +static struct sg_table *dma_heap_map_dma_buf(
drivers/dma-buf/heaps/heap-helpers.c line 223
Quoted 10 lines
drivers/dma-buf/heaps/heap-helpers.c new file +10
-0+struct dma_buf_attachment *attachment, + enum dma_data_direction direction) +{ + struct dma_heaps_attachment *a = attachment->priv; +struct sg_table *table; + + table = a->table; + +if (!dma_map_sg (attachment->dev, table->sgl, table->nents, + direction))
Since this code is used for system heap and and as the reference. In multimedia uses cases very large buffers can be allocated from system heap, and since system heap allocations have a cached kernel mapping it has been difficult to support uncached allocations so clients will likely allocate them as cached allocations.
Most access to these buffers will occur from non IO-coherent devices, however in frameworks such as Android these buffers will be dma mapped and dma unmapped frequently, for every frame and for each device in the "buffer pipeline", which leads to a lot of unnecessary cache maintenance in the dma map and dma unmap calls.
Quoted 1 line
From previous discussions it doesn't seem like this could be optimized by
only making changes to dma-buf heaps framework.
So I think it would be helpful to try and agree on what types of changes would be required to the Android framework and possibly the dma-buf heaps framework to resolve this.
Example
- Have Android framework keep buffers dma mapped for the whole use case
- Or perhaps have Android keep required devices attached to the buffer as they are "pipelined" so that cache maintenance can be skipped in dma map and dma umnap but reliably applied in begin/end_cpu_access.
Quoted 10 lines
drivers/dma-buf/heaps/heap-helpers.c new file +10
-0+ table =ERR_PTR (-ENOMEM ); +return table; +} + +static void dma_heap_unmap_dma_buf (struct dma_buf_attachment *attachment, +struct sg_table *table, +enum dma_data_direction direction) +{ +dma_unmap_sg (attachment->dev, table->sgl, table->nents, direction); +}
drivers/dma-buf/heaps/heap-helpers.c line 243
Quoted 10 lines
drivers/dma-buf/heaps/heap-helpers.c new file +10
-0+ +static int dma_heap_mmap (struct dma_buf *dmabuf,struct vm_area_struct *vma) +{ +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); +int ret =0 ; + +mutex_lock (&buffer->lock); +/* now map it to userspace */ + ret =dma_heap_map_user (buffer, vma);
Since this code is used for system heap and and as the reference. Currently in Android when a system heap buffer is moving down the "pipeline" CPU access may occur and there may be no device attached. In the past people have raised concerns that this should perhaps not be supported and that at least one device should be attached when CPU access occurs.
Can we decide, from a dma-buf contract perspective, on whether CPU access without a device attached should allowed and make it clear in the dma-buf documentation either way?
If it should not be allowed we can try to work with Android to see how they can change their framework to align with the dma-buf spec.
Quoted 32 lines
drivers/dma-buf/heaps/heap-helpers.c new file +32
-0+mutex_unlock (&buffer->lock); + +if (ret) +pr_err ("%s: failure mapping buffer to userspace\n" , + __func__); + +return ret; +} + +static void dma_heap_dma_buf_release (struct dma_buf *dmabuf) +{ +struct dma_heap_buffer *buffer = dmabuf->priv; + +dma_heap_buffer_destroy (buffer); +} + +static void *dma_heap_dma_buf_kmap (struct dma_buf *dmabuf, +unsigned long offset) +{ +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + +return buffer->vaddr + offset *PAGE_SIZE ; +} + +static void dma_heap_dma_buf_kunmap (struct dma_buf *dmabuf, +unsigned long offset, +void *ptr) +{ +} + +static int dma_heap_dma_buf_begin_cpu_access(struct dma_buf *dmabuf,
drivers/dma-buf/heaps/heap-helpers.c line 285
Quoted 10 lines
drivers/dma-buf/heaps/heap-helpers.c new file +10
-0+ enum dma_data_direction direction) +{ +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); +void *vaddr; +struct dma_heaps_attachment *a; +int ret =0 ; + +mutex_lock (&buffer->lock); + vaddr =dma_heap_buffer_kmap_get (heap_buffer);
Since this code is used for system heap and and as the reference. As has been discussed in the past there are several disadvantages to creating a kernel mapping on each call to begin_cpu_access.
The resulting call to alloc_vmap_area is expensive and can hurt client KPIs
Allowing userspace clients to create and destroy kernel mappings can provide opportunities to crash the kernel.
Can we look at removing the creation of a kernel mapping in begin_cpu_access and either introduce support for dma_buf_vmap and have clients use that instead or perahps change
the contract for dma_buf_kmap so that it doesn't always need to succeed?
drivers/dma-buf/heaps/heap-helpers.c line 295
Quoted 10 lines
drivers/dma-buf/heaps/heap-helpers.c new file +10
-0+if (IS_ERR (vaddr)) { + ret =PTR_ERR (vaddr); + goto unlock; + } +mutex_unlock (&buffer->lock); + +mutex_lock (&buffer->lock); +list_for_each_entry (a, &buffer->attachments, list) { +dma_sync_sg_for_cpu (a->dev, a->table->sgl, a->table->nents, + direction);
Since this code is used for system heap and and as the reference. Not a major issue for newer kernels but I still don't think it makes sense to apply cache maintenance when the buffer is not dma mapped, it doesn't makes sense to me from a logical perspective and from a performance perspective.
Quoted 12 lines
drivers/dma-buf/heaps/heap-helpers.c new file +12
-0+ } + +unlock: +mutex_unlock (&buffer->lock); +return ret; +} + +static int dma_heap_dma_buf_end_cpu_access (struct dma_buf *dmabuf, +enum dma_data_direction direction) +{ +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer);
drivers/dma-buf/heaps/heap-helpers.c line 317
Quoted 10 lines
drivers/dma-buf/heaps/heap-helpers.c new file +10
-0+struct dma_heaps_attachment *a; + +mutex_lock (&buffer->lock); +dma_heap_buffer_kmap_put (heap_buffer); +mutex_unlock (&buffer->lock); + +mutex_lock (&buffer->lock); +list_for_each_entry (a, &buffer->attachments, list) { +dma_sync_sg_for_device (a->dev, a->table->sgl, a->table->nents, + direction);
There are use cases in Android which result in only small parts of a large buffer are written do during of CPU access.
Applying cache maintenance to the complete buffer results in a lot of unnecessary cache maintenance that can affect KPIs.
I believe the Android team is wondering if there could be a way to support partial maintenance in where userspace could describe the buffer changes they have made.
I think it would be useful to make sure that there is a least a path forward with the current dma-buf heaps framework to solve this for system heap allocations.
I can get more details on the specific use cases if required.
Quoted 76 lines
drivers/dma-buf/heaps/heap-helpers.c new file +18
-0+ } +mutex_unlock (&buffer->lock); + +return 0 ; +} + +const struct dma_buf_ops heap_helper_ops = { + .map_dma_buf = dma_heap_map_dma_buf, + .unmap_dma_buf = dma_heap_unmap_dma_buf, + .mmap = dma_heap_mmap, + .release = dma_heap_dma_buf_release, + .attach = dma_heap_attach, + .detach = dma_heap_detatch, + .begin_cpu_access = dma_heap_dma_buf_begin_cpu_access, + .end_cpu_access = dma_heap_dma_buf_end_cpu_access, + .map = dma_heap_dma_buf_kmap, + .unmap = dma_heap_dma_buf_kunmap, +};drivers/dma-buf/heaps/heap-helpers.h new file +48
-0@@ -0,0 +1,48 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* DMABUF Heaps helper code +* +* Copyright (C) 2011 Google, Inc. +* Copyright (C) 2019 Linaro Ltd. +*/ + +#ifndef _HEAP_HELPERS_H +#define _HEAP_HELPERS_H + +#include <linux/dma-heap.h> +#include <linux/list.h> + +struct heap_helper_buffer { +struct dma_heap_buffer heap_buffer; + +unsigned long private_flags; +void *priv_virt; +struct mutex lock; +int kmap_cnt; +void *vaddr; +struct sg_table *sg_table; +struct list_head attachments; + +void (*free)(struct heap_helper_buffer *buffer); + +}; + +#define to_helper_buffer (x) \ + container_of(x, struct heap_helper_buffer, heap_buffer) + +static inline void INIT_HEAP_HELPER_BUFFER (struct heap_helper_buffer *buffer, +void (*free)(struct heap_helper_buffer *)) +{ + buffer->private_flags =0 ; + buffer->priv_virt =NULL ; +mutex_init (&buffer->lock); + buffer->kmap_cnt =0 ; + buffer->vaddr =NULL ; + buffer->sg_table =NULL ; +INIT_LIST_HEAD (&buffer->attachments); + buffer->free = free; +} + +extern const struct dma_buf_ops heap_helper_ops; + +#endif /* _HEAP_HELPERS_H */ Signature · 1 line
2.7.4
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
Re: [RFC][PATCH 2/5 v2] dma-buf: heaps: Add heap helpers
Quoted 302 lines · Liam Mark
Quoted 271 lines · John Stultz
Add generic helper dmabuf ops for dma heaps, so we can reduce the amount of duplicative code for the exported dmabufs.
This code is an evolution of the Android ION implementation, so thanks to its original authors and maintainters:
Rebecca Schultz Zavin, Colin Cross, Laura Abbott, and others!Cc Laura Abbott <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Greg KH <[email protected]>Cc Sumit Semwal <[email protected]>Cc Liam Mark <[email protected]>Cc Brian Starkey <[email protected]>Cc Andrew F. Davis <[email protected]>Cc Chenbo Feng <[email protected]>Cc Alistair Strachan <[email protected]>Signed-off-by John Stultz <[email protected]>---
v2:
* Removed cache management performance hack that I had accidentally folded in.
* Removed stats code that was in helpers
* Lots of checkpatch cleanups--- drivers/dma-buf/Makefile | 1 + drivers/dma-buf/heaps/Makefile | 2 + drivers/dma-buf/heaps/heap-helpers.c | 335 +++++++++++++++++++++++++++++++++++ drivers/dma-buf/heaps/heap-helpers.h | 48 +++++ 4 files changed, 386 insertions(+) create mode 100644 drivers/dma-buf/heaps/Makefile create mode 100644 drivers/dma-buf/heaps/heap-helpers.c create mode 100644 drivers/dma-buf/heaps/heap-helpers.hdrivers/dma-buf/Makefile +1
-0@@ -1,4 +1,5 @@ obj-y := dma-buf.o dma-fence.o dma-fence-array.o reservation.o seqno-fence.o +obj-$(CONFIG_DMABUF_HEAPS) += heaps/ obj-$(CONFIG_DMABUF_HEAPS) += dma-heap.o obj-$(CONFIG_SYNC_FILE) += sync_file.o obj-$(CONFIG_SW_SYNC) += sw_sync.o sync_debug.odrivers/dma-buf/heaps/Makefile new file +2
-0@@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-y += heap-helpers.odrivers/dma-buf/heaps/heap-helpers.c new file +213
-0@@ -0,0 +1,335 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/device.h> +#include <linux/dma-buf.h> +#include <linux/err.h> +#include <linux/idr.h> +#include <linux/list.h> +#include <linux/slab.h> +#include <linux/uaccess.h> +#include <uapi/linux/dma-heap.h> + +#include "heap-helpers.h" + + +static void *dma_heap_map_kernel (struct heap_helper_buffer *buffer) +{ +struct scatterlist *sg; +int i, j; +void *vaddr; +pgprot_t pgprot; +struct sg_table *table = buffer->sg_table; +int npages =PAGE_ALIGN (buffer->heap_buffer.size) /PAGE_SIZE ; +struct page **pages =vmalloc (array_size (npages, +sizeof (struct page *))); +struct page **tmp = pages; + +if (!pages) +return ERR_PTR (-ENOMEM ); + + pgprot =PAGE_KERNEL ; + +for_each_sg (table->sgl, sg, table->nents, i) { +int npages_this_entry =PAGE_ALIGN (sg->length) /PAGE_SIZE ; +struct page *page =sg_page (sg); + +WARN_ON (i >= npages); +for (j =0 ; j < npages_this_entry; j++) + *(tmp++) = page++; + } + vaddr =vmap (pages, npages,VM_MAP , pgprot); +vfree (pages); + +if (!vaddr) +return ERR_PTR (-ENOMEM ); + +return vaddr; +} + +static int dma_heap_map_user (struct heap_helper_buffer *buffer, +struct vm_area_struct *vma) +{ +struct sg_table *table = buffer->sg_table; +unsigned long addr = vma->vm_start; +unsigned long offset = vma->vm_pgoff *PAGE_SIZE ; +struct scatterlist *sg; +int i; +int ret; + +for_each_sg (table->sgl, sg, table->nents, i) { +struct page *page =sg_page (sg); +unsigned long remainder = vma->vm_end - addr; +unsigned long len = sg->length; + +if (offset >= sg->length) { + offset -= sg->length; +continue ; + }else if (offset) { + page += offset /PAGE_SIZE ; + len = sg->length - offset; + offset =0 ; + } + len =min (len, remainder); + ret =remap_pfn_range (vma, addr,page_to_pfn (page), len, + vma->vm_page_prot); +if (ret) +return ret; + addr += len; +if (addr >= vma->vm_end) +return 0 ; + } + +return 0 ; +} + + +void dma_heap_buffer_destroy (struct dma_heap_buffer *heap_buffer) +{ +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + +if (buffer->kmap_cnt >0 ) { +pr_warn_once ("%s: buffer still mapped in the kernel\n" , + __func__); +vunmap (buffer->vaddr); + } + + buffer->free (buffer); +} + +static void *dma_heap_buffer_kmap_get (struct dma_heap_buffer *heap_buffer) +{ +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); +void *vaddr; + +if (buffer->kmap_cnt) { + buffer->kmap_cnt++; +return buffer->vaddr; + } + vaddr =dma_heap_map_kernel (buffer); +if (WARN_ONCE (!vaddr, +"heap->ops->map_kernel should return ERR_PTR on error" )) +return ERR_PTR (-EINVAL ); +if (IS_ERR (vaddr)) +return vaddr; + buffer->vaddr = vaddr; + buffer->kmap_cnt++; +return vaddr; +} + +static void dma_heap_buffer_kmap_put (struct dma_heap_buffer *heap_buffer) +{ +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + + buffer->kmap_cnt--; +if (!buffer->kmap_cnt) { +vunmap (buffer->vaddr); + buffer->vaddr =NULL ; + } +} + +static struct sg_table *dup_sg_table (struct sg_table *table) +{ +struct sg_table *new_table; +int ret, i; +struct scatterlist *sg, *new_sg; + + new_table =kzalloc (sizeof (*new_table),GFP_KERNEL ); +if (!new_table) +return ERR_PTR (-ENOMEM ); + + ret =sg_alloc_table (new_table, table->nents,GFP_KERNEL ); +if (ret) { +kfree (new_table); +return ERR_PTR (-ENOMEM ); + } + + new_sg = new_table->sgl; +for_each_sg (table->sgl, sg, table->nents, i) { +memcpy (new_sg, sg,sizeof (*sg)); + new_sg->dma_address =0 ; + new_sg =sg_next (new_sg); + } + +return new_table; +} + +static void free_duped_table (struct sg_table *table) +{ +sg_free_table (table); +kfree (table); +} + +struct dma_heaps_attachment { +struct device *dev; +struct sg_table *table; +struct list_head list; +}; + +static int dma_heap_attach (struct dma_buf *dmabuf, +struct dma_buf_attachment *attachment) +{ +struct dma_heaps_attachment *a; +struct sg_table *table; +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + + a =kzalloc (sizeof (*a),GFP_KERNEL ); +if (!a) +return -ENOMEM ; + + table =dup_sg_table (buffer->sg_table); +if (IS_ERR (table)) { +kfree (a); +return -ENOMEM ; + } + + a->table = table; + a->dev = attachment->dev; +INIT_LIST_HEAD (&a->list); + + attachment->priv = a; + +mutex_lock (&buffer->lock); +list_add (&a->list, &buffer->attachments); +mutex_unlock (&buffer->lock); + +return 0 ; +} + +static void dma_heap_detatch (struct dma_buf *dmabuf, +struct dma_buf_attachment *attachment) +{ +struct dma_heaps_attachment *a = attachment->priv; +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + +mutex_lock (&buffer->lock); +list_del (&a->list); +mutex_unlock (&buffer->lock); +free_duped_table (a->table); + +kfree (a); +} + +static struct sg_table *dma_heap_map_dma_buf(drivers/dma-buf/heaps/heap-helpers.c line 223
Quoted 10 lines
drivers/dma-buf/heaps/heap-helpers.c new file +10
-0+struct dma_buf_attachment *attachment, + enum dma_data_direction direction) +{ + struct dma_heaps_attachment *a = attachment->priv; +struct sg_table *table; + + table = a->table; + +if (!dma_map_sg (attachment->dev, table->sgl, table->nents, + direction))Since this code is used for system heap and and as the reference. In multimedia uses cases very large buffers can be allocated from system heap, and since system heap allocations have a cached kernel mapping it has been difficult to support uncached allocations so clients will likely allocate them as cached allocations.
Most access to these buffers will occur from non IO-coherent devices, however in frameworks such as Android these buffers will be dma mapped and dma unmapped frequently, for every frame and for each device in the "buffer pipeline", which leads to a lot of unnecessary cache maintenance in the dma map and dma unmap calls.
From previous discussions it doesn't seem like this could be optimized by only making changes to dma-buf heaps framework.So I think it would be helpful to try and agree on what types of changes would be required to the Android framework and possibly the dma-buf heaps framework to resolve this.
Example
Quoted 6 lines
- Have Android framework keep buffers dma mapped for the whole use case
- Or perhaps have Android keep required devices attached to the buffer as they are "pipelined" so that cache maintenance can be skipped in dma map and dma umnap but reliably applied in begin/end_cpu_access.
I don't have a strong opinion on the solution here for Android, but from kernel/hardware side I can see we don't have a good ownership model for dma-buf and that looks to be causing your unneeded cache ops.
Let my give my vision for the model you request below.
Quoted 28 lines
Quoted 10 lines
drivers/dma-buf/heaps/heap-helpers.c new file +10
-0+ table =ERR_PTR (-ENOMEM ); +return table; +} + +static void dma_heap_unmap_dma_buf (struct dma_buf_attachment *attachment, +struct sg_table *table, +enum dma_data_direction direction) +{ +dma_unmap_sg (attachment->dev, table->sgl, table->nents, direction); +}drivers/dma-buf/heaps/heap-helpers.c line 243
Quoted 10 lines
drivers/dma-buf/heaps/heap-helpers.c new file +10
-0+ +static int dma_heap_mmap (struct dma_buf *dmabuf,struct vm_area_struct *vma) +{ +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); +int ret =0 ; + +mutex_lock (&buffer->lock); +/* now map it to userspace */ + ret =dma_heap_map_user (buffer, vma);Since this code is used for system heap and and as the reference. Currently in Android when a system heap buffer is moving down the "pipeline" CPU access may occur and there may be no device attached. In the past people have raised concerns that this should perhaps not be supported and that at least one device should be attached when CPU access occurs.
Quoted 7 lines
Can we decide, from a dma-buf contract perspective, on whether CPU access without a device attached should allowed and make it clear in the dma-buf documentation either way?
If it should not be allowed we can try to work with Android to see how they can change their framework to align with the dma-buf spec.
As you say, right now DMA-BUF does not really provide guidelines for "well behaved" heaps, and as such we all end up making different assumptions. Although I don't think they below should be enforced for all DMA-BUFs it should be used for our first set of reference heaps.
So from the start, lets looks at DMA-BUF buffer lifetimes. A buffer begins by being allocated, at this point the backing resources (usually some pages in physical CPU addressable RAM) have *not* been allocated this is a core feature of DMA-BUF allowing smart allocations at map time. At this point we should consider the buffer in a state "non-owned non-backed".
Next we have three valid actions:
* CPU use
* device attachments
* free
CPU use starts with a call to begin_cpu_access(), the buffer will now both become CPU owned and will pin it's backing buffer. This last part can cause begin_cpu_access() to fail for some types of buffers that only pick their backing resource based on the set of attached devices, which at this point we have none. For heaps like "system" heap this should not be a problem as the backing storage is already selected at allocation time.
Device attachments and mapping should behave as they do now. After mapping, the buffer is now owned by the attached devices and backed. The ownership can be transfered to the CPU and back as usual with {begin,end}_cpu_access().
Free right after allocation without anyone taking ownership is trivial.
Now here is where things may get interesting, we can now be in the following states:
CPU owned backed
Device owned backed
What should we do when CPU owned and end_cpu_access() is called or when Device owned and all devices detach. What state do we go into? My opinion would be to return to "non-owned non-backed" which means the backing resource can be freed. The other option is to leave it backed. I beleave this is what Android expects right now as it returns buffers to a "BufferQueue" to later be dequeued and used again in the same way without time consuming reallocations.
For "system" heaps the result is the same outside of the cache ops, if we maintain a buffer state like the above we can always know when a cache op or similar is needed (only when transitioning owners, not on every map/unmap). For more complex heaps we can do something similar when transitioning from backed to non-backed (and vice-versa) such as migrating the backing data to a more appropriate resource given the new set of attaching/mapping devices.
A more complete state transition map will probably be needed to fill out what should be allowed and what to do when, and I agree it will be good to have that for this first set of reference heaps. But all of this can be done now with DMA-heaps, so I don't want to tie up the core de-staging too much on getting every possible heap to behave just right either..
Quoted 50 lines
Quoted 32 lines
drivers/dma-buf/heaps/heap-helpers.c new file +32
-0+mutex_unlock (&buffer->lock); + +if (ret) +pr_err ("%s: failure mapping buffer to userspace\n" , + __func__); + +return ret; +} + +static void dma_heap_dma_buf_release (struct dma_buf *dmabuf) +{ +struct dma_heap_buffer *buffer = dmabuf->priv; + +dma_heap_buffer_destroy (buffer); +} + +static void *dma_heap_dma_buf_kmap (struct dma_buf *dmabuf, +unsigned long offset) +{ +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + +return buffer->vaddr + offset *PAGE_SIZE ; +} + +static void dma_heap_dma_buf_kunmap (struct dma_buf *dmabuf, +unsigned long offset, +void *ptr) +{ +} + +static int dma_heap_dma_buf_begin_cpu_access(struct dma_buf *dmabuf,drivers/dma-buf/heaps/heap-helpers.c line 285
Quoted 10 lines
drivers/dma-buf/heaps/heap-helpers.c new file +10
-0+ enum dma_data_direction direction) +{ +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); +void *vaddr; +struct dma_heaps_attachment *a; +int ret =0 ; + +mutex_lock (&buffer->lock); + vaddr =dma_heap_buffer_kmap_get (heap_buffer);Since this code is used for system heap and and as the reference. As has been discussed in the past there are several disadvantages to creating a kernel mapping on each call to
begin_cpu_access.The resulting call to
alloc_vmap_areais expensive and can hurt client KPIs
Quoted 8 lines
Allowing userspace clients to create and destroy kernel mappings can provide opportunities to crash the kernel.
Can we look at removing the creation of a kernel mapping in
begin_cpu_accessand either introduce support fordma_buf_vmapand have clients use that instead or perahps change
the contract fordma_buf_kmapso that it doesn't always need to succeed?
Agree, for this we should just fail (or succeed but perform no action?) if there are no active mappings (neither in kernel (kmap, vmap) nor in userspace (mmap)), forcing a new mapping just to keep dma_sync_sg_for_cpu() happy is not the correct thing to do here.
Quoted 8 lines
Quoted 8 lines
drivers/dma-buf/heaps/heap-helpers.c new file +8
-0+if (IS_ERR (vaddr)) { + ret =PTR_ERR (vaddr); + goto unlock; + } +mutex_unlock (&buffer->lock); + +mutex_lock (&buffer->lock); +list_for_each_entry (a, &buffer->attachments, list) {
Quoted 9 lines
drivers/dma-buf/heaps/heap-helpers.c line 295
Quoted 2 lines
drivers/dma-buf/heaps/heap-helpers.c new file +2
-0+dma_sync_sg_for_cpu (a->dev, a->table->sgl, a->table->nents, + direction);Since this code is used for system heap and and as the reference. Not a major issue for newer kernels but I still don't think it makes sense to apply cache maintenance when the buffer is not dma mapped, it doesn't makes sense to me from a logical perspective and from a performance perspective.
Agree again, this is only needed when we are transitioning out of state "device owned" which we will not be in if we have no active attached/mapped devices.
Quoted 18 lines
Quoted 18 lines
drivers/dma-buf/heaps/heap-helpers.c new file +17
-0+ } + +unlock: +mutex_unlock (&buffer->lock); +return ret; +} + +static int dma_heap_dma_buf_end_cpu_access (struct dma_buf *dmabuf, +enum dma_data_direction direction) +{ +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); +struct dma_heaps_attachment *a; + +mutex_lock (&buffer->lock); +dma_heap_buffer_kmap_put (heap_buffer); +mutex_unlock (&buffer->lock);
Quoted 9 lines
drivers/dma-buf/heaps/heap-helpers.c line 317
Quoted 5 lines
drivers/dma-buf/heaps/heap-helpers.c new file +5
-0+ +mutex_lock (&buffer->lock); +list_for_each_entry (a, &buffer->attachments, list) { +dma_sync_sg_for_device (a->dev, a->table->sgl, a->table->nents, + direction);There are use cases in Android which result in only small parts of a large buffer are written do during of CPU access.
We have the same requirements for some OpenCL use-cases (clCreateSubBuffer), which allows for chunks of a larger buffer to have different properties and therefor cache operations different from the larger parrent buffer.
Quoted 7 lines
Applying cache maintenance to the complete buffer results in a lot of unnecessary cache maintenance that can affect KPIs.
I believe the Android team is wondering if there could be a way to support partial maintenance in where userspace could describe the buffer changes they have made.
The issue I have with that is it would require userspace to know about the cache line size of all involved devices, otherwise you would have users writing to the first 64B and invalidate to read the second 64B written from a DMA device outside the cache, in a system with 128B cache-lines you will always lose some data here.
Quoted 4 lines
I think it would be useful to make sure that there is a least a path forward with the current dma-buf heaps framework to solve this for system heap allocations.
DMA-BUF sync operations always act on the full buffer, changing this would require changes to the core DMA-BUF framework (so outside the scope of this set) :)
Andrew
Quoted 73 lines
I can get more details on the specific use cases if required.
Quoted 71 lines
drivers/dma-buf/heaps/heap-helpers.c new file +18
-0+ } +mutex_unlock (&buffer->lock); + +return 0 ; +} + +const struct dma_buf_ops heap_helper_ops = { + .map_dma_buf = dma_heap_map_dma_buf, + .unmap_dma_buf = dma_heap_unmap_dma_buf, + .mmap = dma_heap_mmap, + .release = dma_heap_dma_buf_release, + .attach = dma_heap_attach, + .detach = dma_heap_detatch, + .begin_cpu_access = dma_heap_dma_buf_begin_cpu_access, + .end_cpu_access = dma_heap_dma_buf_end_cpu_access, + .map = dma_heap_dma_buf_kmap, + .unmap = dma_heap_dma_buf_kunmap, +};drivers/dma-buf/heaps/heap-helpers.h new file +47
-0@@ -0,0 +1,48 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* DMABUF Heaps helper code +* +* Copyright (C) 2011 Google, Inc. +* Copyright (C) 2019 Linaro Ltd. +*/ + +#ifndef _HEAP_HELPERS_H +#define _HEAP_HELPERS_H + +#include <linux/dma-heap.h> +#include <linux/list.h> + +struct heap_helper_buffer { +struct dma_heap_buffer heap_buffer; + +unsigned long private_flags; +void *priv_virt; +struct mutex lock; +int kmap_cnt; +void *vaddr; +struct sg_table *sg_table; +struct list_head attachments; + +void (*free)(struct heap_helper_buffer *buffer); + +}; + +#define to_helper_buffer (x) \ + container_of(x, struct heap_helper_buffer, heap_buffer) + +static inline void INIT_HEAP_HELPER_BUFFER (struct heap_helper_buffer *buffer, +void (*free)(struct heap_helper_buffer *)) +{ + buffer->private_flags =0 ; + buffer->priv_virt =NULL ; +mutex_init (&buffer->lock); + buffer->kmap_cnt =0 ; + buffer->vaddr =NULL ; + buffer->sg_table =NULL ; +INIT_LIST_HEAD (&buffer->attachments); + buffer->free = free; +} + +extern const struct dma_buf_ops heap_helper_ops; +
Quoted 9 lines
Quoted 5 lines
drivers/dma-buf/heaps/heap-helpers.h new file +1
-0+#endif /* _HEAP_HELPERS_H */ Signature · 1 line
2.7.4Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [RFC][PATCH 2/5 v2] dma-buf: heaps: Add heap helpers
Quoted 390 lines · Andrew F. Davis
Quoted 302 lines · Liam Mark
Quoted 271 lines · John Stultz
Add generic helper dmabuf ops for dma heaps, so we can reduce the amount of duplicative code for the exported dmabufs.
This code is an evolution of the Android ION implementation, so thanks to its original authors and maintainters:
Rebecca Schultz Zavin, Colin Cross, Laura Abbott, and others!Cc Laura Abbott <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Greg KH <[email protected]>Cc Sumit Semwal <[email protected]>Cc Liam Mark <[email protected]>Cc Brian Starkey <[email protected]>Cc Andrew F. Davis <[email protected]>Cc Chenbo Feng <[email protected]>Cc Alistair Strachan <[email protected]>Signed-off-by John Stultz <[email protected]>---
v2:
* Removed cache management performance hack that I had accidentally folded in.
* Removed stats code that was in helpers
* Lots of checkpatch cleanups--- drivers/dma-buf/Makefile | 1 + drivers/dma-buf/heaps/Makefile | 2 + drivers/dma-buf/heaps/heap-helpers.c | 335 +++++++++++++++++++++++++++++++++++ drivers/dma-buf/heaps/heap-helpers.h | 48 +++++ 4 files changed, 386 insertions(+) create mode 100644 drivers/dma-buf/heaps/Makefile create mode 100644 drivers/dma-buf/heaps/heap-helpers.c create mode 100644 drivers/dma-buf/heaps/heap-helpers.hdrivers/dma-buf/Makefile +1
-0@@ -1,4 +1,5 @@ obj-y := dma-buf.o dma-fence.o dma-fence-array.o reservation.o seqno-fence.o +obj-$(CONFIG_DMABUF_HEAPS) += heaps/ obj-$(CONFIG_DMABUF_HEAPS) += dma-heap.o obj-$(CONFIG_SYNC_FILE) += sync_file.o obj-$(CONFIG_SW_SYNC) += sw_sync.o sync_debug.odrivers/dma-buf/heaps/Makefile new file +2
-0@@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-y += heap-helpers.odrivers/dma-buf/heaps/heap-helpers.c new file +213
-0@@ -0,0 +1,335 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/device.h> +#include <linux/dma-buf.h> +#include <linux/err.h> +#include <linux/idr.h> +#include <linux/list.h> +#include <linux/slab.h> +#include <linux/uaccess.h> +#include <uapi/linux/dma-heap.h> + +#include "heap-helpers.h" + + +static void *dma_heap_map_kernel (struct heap_helper_buffer *buffer) +{ +struct scatterlist *sg; +int i, j; +void *vaddr; +pgprot_t pgprot; +struct sg_table *table = buffer->sg_table; +int npages =PAGE_ALIGN (buffer->heap_buffer.size) /PAGE_SIZE ; +struct page **pages =vmalloc (array_size (npages, +sizeof (struct page *))); +struct page **tmp = pages; + +if (!pages) +return ERR_PTR (-ENOMEM ); + + pgprot =PAGE_KERNEL ; + +for_each_sg (table->sgl, sg, table->nents, i) { +int npages_this_entry =PAGE_ALIGN (sg->length) /PAGE_SIZE ; +struct page *page =sg_page (sg); + +WARN_ON (i >= npages); +for (j =0 ; j < npages_this_entry; j++) + *(tmp++) = page++; + } + vaddr =vmap (pages, npages,VM_MAP , pgprot); +vfree (pages); + +if (!vaddr) +return ERR_PTR (-ENOMEM ); + +return vaddr; +} + +static int dma_heap_map_user (struct heap_helper_buffer *buffer, +struct vm_area_struct *vma) +{ +struct sg_table *table = buffer->sg_table; +unsigned long addr = vma->vm_start; +unsigned long offset = vma->vm_pgoff *PAGE_SIZE ; +struct scatterlist *sg; +int i; +int ret; + +for_each_sg (table->sgl, sg, table->nents, i) { +struct page *page =sg_page (sg); +unsigned long remainder = vma->vm_end - addr; +unsigned long len = sg->length; + +if (offset >= sg->length) { + offset -= sg->length; +continue ; + }else if (offset) { + page += offset /PAGE_SIZE ; + len = sg->length - offset; + offset =0 ; + } + len =min (len, remainder); + ret =remap_pfn_range (vma, addr,page_to_pfn (page), len, + vma->vm_page_prot); +if (ret) +return ret; + addr += len; +if (addr >= vma->vm_end) +return 0 ; + } + +return 0 ; +} + + +void dma_heap_buffer_destroy (struct dma_heap_buffer *heap_buffer) +{ +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + +if (buffer->kmap_cnt >0 ) { +pr_warn_once ("%s: buffer still mapped in the kernel\n" , + __func__); +vunmap (buffer->vaddr); + } + + buffer->free (buffer); +} + +static void *dma_heap_buffer_kmap_get (struct dma_heap_buffer *heap_buffer) +{ +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); +void *vaddr; + +if (buffer->kmap_cnt) { + buffer->kmap_cnt++; +return buffer->vaddr; + } + vaddr =dma_heap_map_kernel (buffer); +if (WARN_ONCE (!vaddr, +"heap->ops->map_kernel should return ERR_PTR on error" )) +return ERR_PTR (-EINVAL ); +if (IS_ERR (vaddr)) +return vaddr; + buffer->vaddr = vaddr; + buffer->kmap_cnt++; +return vaddr; +} + +static void dma_heap_buffer_kmap_put (struct dma_heap_buffer *heap_buffer) +{ +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + + buffer->kmap_cnt--; +if (!buffer->kmap_cnt) { +vunmap (buffer->vaddr); + buffer->vaddr =NULL ; + } +} + +static struct sg_table *dup_sg_table (struct sg_table *table) +{ +struct sg_table *new_table; +int ret, i; +struct scatterlist *sg, *new_sg; + + new_table =kzalloc (sizeof (*new_table),GFP_KERNEL ); +if (!new_table) +return ERR_PTR (-ENOMEM ); + + ret =sg_alloc_table (new_table, table->nents,GFP_KERNEL ); +if (ret) { +kfree (new_table); +return ERR_PTR (-ENOMEM ); + } + + new_sg = new_table->sgl; +for_each_sg (table->sgl, sg, table->nents, i) { +memcpy (new_sg, sg,sizeof (*sg)); + new_sg->dma_address =0 ; + new_sg =sg_next (new_sg); + } + +return new_table; +} + +static void free_duped_table (struct sg_table *table) +{ +sg_free_table (table); +kfree (table); +} + +struct dma_heaps_attachment { +struct device *dev; +struct sg_table *table; +struct list_head list; +}; + +static int dma_heap_attach (struct dma_buf *dmabuf, +struct dma_buf_attachment *attachment) +{ +struct dma_heaps_attachment *a; +struct sg_table *table; +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + + a =kzalloc (sizeof (*a),GFP_KERNEL ); +if (!a) +return -ENOMEM ; + + table =dup_sg_table (buffer->sg_table); +if (IS_ERR (table)) { +kfree (a); +return -ENOMEM ; + } + + a->table = table; + a->dev = attachment->dev; +INIT_LIST_HEAD (&a->list); + + attachment->priv = a; + +mutex_lock (&buffer->lock); +list_add (&a->list, &buffer->attachments); +mutex_unlock (&buffer->lock); + +return 0 ; +} + +static void dma_heap_detatch (struct dma_buf *dmabuf, +struct dma_buf_attachment *attachment) +{ +struct dma_heaps_attachment *a = attachment->priv; +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + +mutex_lock (&buffer->lock); +list_del (&a->list); +mutex_unlock (&buffer->lock); +free_duped_table (a->table); + +kfree (a); +} + +static struct sg_table *dma_heap_map_dma_buf(drivers/dma-buf/heaps/heap-helpers.c line 223
Quoted 10 lines
drivers/dma-buf/heaps/heap-helpers.c new file +10
-0+struct dma_buf_attachment *attachment, + enum dma_data_direction direction) +{ + struct dma_heaps_attachment *a = attachment->priv; +struct sg_table *table; + + table = a->table; + +if (!dma_map_sg (attachment->dev, table->sgl, table->nents, + direction))Since this code is used for system heap and and as the reference. In multimedia uses cases very large buffers can be allocated from system heap, and since system heap allocations have a cached kernel mapping it has been difficult to support uncached allocations so clients will likely allocate them as cached allocations.
Most access to these buffers will occur from non IO-coherent devices, however in frameworks such as Android these buffers will be dma mapped and dma unmapped frequently, for every frame and for each device in the "buffer pipeline", which leads to a lot of unnecessary cache maintenance in the dma map and dma unmap calls.
From previous discussions it doesn't seem like this could be optimized by only making changes to dma-buf heaps framework.So I think it would be helpful to try and agree on what types of changes would be required to the Android framework and possibly the dma-buf heaps framework to resolve this.
ExampleQuoted 6 lines
- Have Android framework keep buffers dma mapped for the whole use case
- Or perhaps have Android keep required devices attached to the buffer as they are "pipelined" so that cache maintenance can be skipped in dma map and dma umnap but reliably applied in begin/end_cpu_access.
I don't have a strong opinion on the solution here for Android, but from kernel/hardware side I can see we don't have a good ownership model for dma-buf and that looks to be causing your unneeded cache ops.
Let my give my vision for the model you request below.
Quoted 28 lines
Quoted 10 lines
drivers/dma-buf/heaps/heap-helpers.c new file +10
-0+ table =ERR_PTR (-ENOMEM ); +return table; +} + +static void dma_heap_unmap_dma_buf (struct dma_buf_attachment *attachment, +struct sg_table *table, +enum dma_data_direction direction) +{ +dma_unmap_sg (attachment->dev, table->sgl, table->nents, direction); +}drivers/dma-buf/heaps/heap-helpers.c line 243
Quoted 10 lines
drivers/dma-buf/heaps/heap-helpers.c new file +10
-0+ +static int dma_heap_mmap (struct dma_buf *dmabuf,struct vm_area_struct *vma) +{ +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); +int ret =0 ; + +mutex_lock (&buffer->lock); +/* now map it to userspace */ + ret =dma_heap_map_user (buffer, vma);Since this code is used for system heap and and as the reference. Currently in Android when a system heap buffer is moving down the "pipeline" CPU access may occur and there may be no device attached. In the past people have raised concerns that this should perhaps not be supported and that at least one device should be attached when CPU access occurs.
Quoted 7 lines
Can we decide, from a dma-buf contract perspective, on whether CPU access without a device attached should allowed and make it clear in the dma-buf documentation either way?
If it should not be allowed we can try to work with Android to see how they can change their framework to align with the dma-buf spec.
As you say, right now DMA-BUF does not really provide guidelines for "well behaved" heaps, and as such we all end up making different assumptions. Although I don't think they below should be enforced for all DMA-BUFs it should be used for our first set of reference heaps.
So from the start, lets looks at DMA-BUF buffer lifetimes. A buffer begins by being allocated, at this point the backing resources (usually some pages in physical CPU addressable RAM) have *not* been allocated this is a core feature of DMA-BUF allowing smart allocations at map time. At this point we should consider the buffer in a state "non-owned non-backed".
Next we have three valid actions:
* CPU use
* device attachments
* freeCPU use starts with a call to
begin_cpu_access(), the buffer will now both become CPU owned and will pin it's backing buffer. This last part can causebegin_cpu_access()to fail for some types of buffers that only pick their backing resource based on the set of attached devices, which at this point we have none. For heaps like "system" heap this should not be a problem as the backing storage is already selected at allocation time.Device attachments and mapping should behave as they do now. After mapping, the buffer is now owned by the attached devices and backed. The ownership can be transfered to the CPU and back as usual with {begin,end}_cpu_access().
Free right after allocation without anyone taking ownership is trivial.
Now here is where things may get interesting, we can now be in the following states:
CPU owned backed
Device owned backed
Quoted 8 lines
What should we do when CPU owned and
end_cpu_access()is called or when Device owned and all devices detach. What state do we go into? My opinion would be to return to "non-owned non-backed" which means the backing resource can be freed. The other option is to leave it backed. I beleave this is what Android expects right now as it returns buffers to a "BufferQueue" to later be dequeued and used again in the same way without time consuming reallocations.
I think the issue may be more serious with Android than allow improving the performance of buffer re-allocation.
In cases such as video playback:
#1 video device reads and transforms the video data #2 optionally there may be some software video post processing #3 surface flinger
#4 HAL
#5 display device eventually gets the buffer
My understanding is the last device attached to the buffer (the video device) can be detached before Android sends the buffer down the "pipeline" where some software module may do CPU access and where eventually a new device is attached when the buffer is going to be acceded by a device.
So Android is counting on the contents of the buffer being retained while it is "non-owned".
Quoted 4 lines
For "system" heaps the result is the same outside of the cache ops, if we maintain a buffer state like the above we can always know when a cache op or similar is needed (only when transitioning owners, not on every map/unmap).
Assuming I understood your comments correctly, I agree that we will know "when" to optimally apply the cache maintenance but the problem with allowing CPU access to buffers that are now "non-owned" is that we can't apply the cache maintenance when it is required (because there is no longer a device available).
Quoted 139 lines
For more complex heaps we can do something similar
when transitioning from backed to non-backed (and vice-versa) such as migrating the backing data to a more appropriate resource given the new set of attaching/mapping devices.A more complete state transition map will probably be needed to fill out what should be allowed and what to do when, and I agree it will be good to have that for this first set of reference heaps. But all of this can be done now with DMA-heaps, so I don't want to tie up the core de-staging too much on getting every possible heap to behave just right either..
Quoted 50 lines
Quoted 32 lines
drivers/dma-buf/heaps/heap-helpers.c new file +32
-0+mutex_unlock (&buffer->lock); + +if (ret) +pr_err ("%s: failure mapping buffer to userspace\n" , + __func__); + +return ret; +} + +static void dma_heap_dma_buf_release (struct dma_buf *dmabuf) +{ +struct dma_heap_buffer *buffer = dmabuf->priv; + +dma_heap_buffer_destroy (buffer); +} + +static void *dma_heap_dma_buf_kmap (struct dma_buf *dmabuf, +unsigned long offset) +{ +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + +return buffer->vaddr + offset *PAGE_SIZE ; +} + +static void dma_heap_dma_buf_kunmap (struct dma_buf *dmabuf, +unsigned long offset, +void *ptr) +{ +} + +static int dma_heap_dma_buf_begin_cpu_access(struct dma_buf *dmabuf,drivers/dma-buf/heaps/heap-helpers.c line 285
Quoted 10 lines
drivers/dma-buf/heaps/heap-helpers.c new file +10
-0+ enum dma_data_direction direction) +{ +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); +void *vaddr; +struct dma_heaps_attachment *a; +int ret =0 ; + +mutex_lock (&buffer->lock); + vaddr =dma_heap_buffer_kmap_get (heap_buffer);Since this code is used for system heap and and as the reference. As has been discussed in the past there are several disadvantages to creating a kernel mapping on each call to
begin_cpu_access.The resulting call to
alloc_vmap_areais expensive and can hurt client KPIsQuoted 8 lines
Allowing userspace clients to create and destroy kernel mappings can provide opportunities to crash the kernel.
Can we look at removing the creation of a kernel mapping in
begin_cpu_accessand either introduce support fordma_buf_vmapand have clients use that instead or perahps change
the contract fordma_buf_kmapso that it doesn't always need to succeed?Agree, for this we should just fail (or succeed but perform no action?) if there are no active mappings (neither in kernel (kmap, vmap) nor in userspace (mmap)), forcing a new mapping just to keep
dma_sync_sg_for_cpu()happy is not the correct thing to do here.Quoted 8 lines
Quoted 8 lines
drivers/dma-buf/heaps/heap-helpers.c new file +8
-0+if (IS_ERR (vaddr)) { + ret =PTR_ERR (vaddr); + goto unlock; + } +mutex_unlock (&buffer->lock); + +mutex_lock (&buffer->lock); +list_for_each_entry (a, &buffer->attachments, list) {Quoted 9 lines
drivers/dma-buf/heaps/heap-helpers.c line 295
Quoted 2 lines
drivers/dma-buf/heaps/heap-helpers.c new file +2
-0+dma_sync_sg_for_cpu (a->dev, a->table->sgl, a->table->nents, + direction);Since this code is used for system heap and and as the reference. Not a major issue for newer kernels but I still don't think it makes sense to apply cache maintenance when the buffer is not dma mapped, it doesn't makes sense to me from a logical perspective and from a performance perspective.
Agree again, this is only needed when we are transitioning out of state "device owned" which we will not be in if we have no active attached/mapped devices.
Quoted 18 lines
Quoted 18 lines
drivers/dma-buf/heaps/heap-helpers.c new file +17
-0+ } + +unlock: +mutex_unlock (&buffer->lock); +return ret; +} + +static int dma_heap_dma_buf_end_cpu_access (struct dma_buf *dmabuf, +enum dma_data_direction direction) +{ +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); +struct dma_heaps_attachment *a; + +mutex_lock (&buffer->lock); +dma_heap_buffer_kmap_put (heap_buffer); +mutex_unlock (&buffer->lock);Quoted 9 lines
drivers/dma-buf/heaps/heap-helpers.c line 317
Quoted 5 lines
drivers/dma-buf/heaps/heap-helpers.c new file +5
-0+ +mutex_lock (&buffer->lock); +list_for_each_entry (a, &buffer->attachments, list) { +dma_sync_sg_for_device (a->dev, a->table->sgl, a->table->nents, + direction);There are use cases in Android which result in only small parts of a large buffer are written do during of CPU access.
We have the same requirements for some OpenCL use-cases (clCreateSubBuffer), which allows for chunks of a larger buffer to have different properties and therefor cache operations different from the larger parrent buffer.
Quoted 7 lines
Applying cache maintenance to the complete buffer results in a lot of unnecessary cache maintenance that can affect KPIs.
I believe the Android team is wondering if there could be a way to support partial maintenance in where userspace could describe the buffer changes they have made.
Quoted 6 lines
The issue I have with that is it would require userspace to know about the cache line size of all involved devices, otherwise you would have users writing to the first 64B and invalidate to read the second 64B written from a DMA device outside the cache, in a system with 128B cache-lines you will always lose some data here.
Good point.
Quoted 5 lines
Quoted 4 lines
I think it would be useful to make sure that there is a least a path forward with the current dma-buf heaps framework to solve this for system heap allocations.
Quoted 89 lines
DMA-BUF sync operations always act on the full buffer, changing this would require changes to the core DMA-BUF framework (so outside the scope of this set) :)
Andrew
Quoted 82 lines
I can get more details on the specific use cases if required.
Quoted 76 lines
drivers/dma-buf/heaps/heap-helpers.c new file +18
-0+ } +mutex_unlock (&buffer->lock); + +return 0 ; +} + +const struct dma_buf_ops heap_helper_ops = { + .map_dma_buf = dma_heap_map_dma_buf, + .unmap_dma_buf = dma_heap_unmap_dma_buf, + .mmap = dma_heap_mmap, + .release = dma_heap_dma_buf_release, + .attach = dma_heap_attach, + .detach = dma_heap_detatch, + .begin_cpu_access = dma_heap_dma_buf_begin_cpu_access, + .end_cpu_access = dma_heap_dma_buf_end_cpu_access, + .map = dma_heap_dma_buf_kmap, + .unmap = dma_heap_dma_buf_kunmap, +};drivers/dma-buf/heaps/heap-helpers.h new file +48
-0@@ -0,0 +1,48 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* DMABUF Heaps helper code +* +* Copyright (C) 2011 Google, Inc. +* Copyright (C) 2019 Linaro Ltd. +*/ + +#ifndef _HEAP_HELPERS_H +#define _HEAP_HELPERS_H + +#include <linux/dma-heap.h> +#include <linux/list.h> + +struct heap_helper_buffer { +struct dma_heap_buffer heap_buffer; + +unsigned long private_flags; +void *priv_virt; +struct mutex lock; +int kmap_cnt; +void *vaddr; +struct sg_table *sg_table; +struct list_head attachments; + +void (*free)(struct heap_helper_buffer *buffer); + +}; + +#define to_helper_buffer (x) \ + container_of(x, struct heap_helper_buffer, heap_buffer) + +static inline void INIT_HEAP_HELPER_BUFFER (struct heap_helper_buffer *buffer, +void (*free)(struct heap_helper_buffer *)) +{ + buffer->private_flags =0 ; + buffer->priv_virt =NULL ; +mutex_init (&buffer->lock); + buffer->kmap_cnt =0 ; + buffer->vaddr =NULL ; + buffer->sg_table =NULL ; +INIT_LIST_HEAD (&buffer->attachments); + buffer->free = free; +} + +extern const struct dma_buf_ops heap_helper_ops; + +#endif /* _HEAP_HELPERS_H */ Signature · 1 line
2.7.4Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
Re: [RFC][PATCH 2/5 v2] dma-buf: heaps: Add heap helpers
Quoted 411 lines · Liam Mark
Quoted 390 lines · Andrew F. Davis
Quoted 302 lines · Liam Mark
Quoted 271 lines · John Stultz
Add generic helper dmabuf ops for dma heaps, so we can reduce the amount of duplicative code for the exported dmabufs.
This code is an evolution of the Android ION implementation, so thanks to its original authors and maintainters:
Rebecca Schultz Zavin, Colin Cross, Laura Abbott, and others!Cc Laura Abbott <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Greg KH <[email protected]>Cc Sumit Semwal <[email protected]>Cc Liam Mark <[email protected]>Cc Brian Starkey <[email protected]>Cc Andrew F. Davis <[email protected]>Cc Chenbo Feng <[email protected]>Cc Alistair Strachan <[email protected]>Signed-off-by John Stultz <[email protected]>---
v2:
* Removed cache management performance hack that I had accidentally folded in.
* Removed stats code that was in helpers
* Lots of checkpatch cleanups--- drivers/dma-buf/Makefile | 1 + drivers/dma-buf/heaps/Makefile | 2 + drivers/dma-buf/heaps/heap-helpers.c | 335 +++++++++++++++++++++++++++++++++++ drivers/dma-buf/heaps/heap-helpers.h | 48 +++++ 4 files changed, 386 insertions(+) create mode 100644 drivers/dma-buf/heaps/Makefile create mode 100644 drivers/dma-buf/heaps/heap-helpers.c create mode 100644 drivers/dma-buf/heaps/heap-helpers.hdrivers/dma-buf/Makefile +1
-0@@ -1,4 +1,5 @@ obj-y := dma-buf.o dma-fence.o dma-fence-array.o reservation.o seqno-fence.o +obj-$(CONFIG_DMABUF_HEAPS) += heaps/ obj-$(CONFIG_DMABUF_HEAPS) += dma-heap.o obj-$(CONFIG_SYNC_FILE) += sync_file.o obj-$(CONFIG_SW_SYNC) += sw_sync.o sync_debug.odrivers/dma-buf/heaps/Makefile new file +2
-0@@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-y += heap-helpers.odrivers/dma-buf/heaps/heap-helpers.c new file +213
-0@@ -0,0 +1,335 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/device.h> +#include <linux/dma-buf.h> +#include <linux/err.h> +#include <linux/idr.h> +#include <linux/list.h> +#include <linux/slab.h> +#include <linux/uaccess.h> +#include <uapi/linux/dma-heap.h> + +#include "heap-helpers.h" + + +static void *dma_heap_map_kernel (struct heap_helper_buffer *buffer) +{ +struct scatterlist *sg; +int i, j; +void *vaddr; +pgprot_t pgprot; +struct sg_table *table = buffer->sg_table; +int npages =PAGE_ALIGN (buffer->heap_buffer.size) /PAGE_SIZE ; +struct page **pages =vmalloc (array_size (npages, +sizeof (struct page *))); +struct page **tmp = pages; + +if (!pages) +return ERR_PTR (-ENOMEM ); + + pgprot =PAGE_KERNEL ; + +for_each_sg (table->sgl, sg, table->nents, i) { +int npages_this_entry =PAGE_ALIGN (sg->length) /PAGE_SIZE ; +struct page *page =sg_page (sg); + +WARN_ON (i >= npages); +for (j =0 ; j < npages_this_entry; j++) + *(tmp++) = page++; + } + vaddr =vmap (pages, npages,VM_MAP , pgprot); +vfree (pages); + +if (!vaddr) +return ERR_PTR (-ENOMEM ); + +return vaddr; +} + +static int dma_heap_map_user (struct heap_helper_buffer *buffer, +struct vm_area_struct *vma) +{ +struct sg_table *table = buffer->sg_table; +unsigned long addr = vma->vm_start; +unsigned long offset = vma->vm_pgoff *PAGE_SIZE ; +struct scatterlist *sg; +int i; +int ret; + +for_each_sg (table->sgl, sg, table->nents, i) { +struct page *page =sg_page (sg); +unsigned long remainder = vma->vm_end - addr; +unsigned long len = sg->length; + +if (offset >= sg->length) { + offset -= sg->length; +continue ; + }else if (offset) { + page += offset /PAGE_SIZE ; + len = sg->length - offset; + offset =0 ; + } + len =min (len, remainder); + ret =remap_pfn_range (vma, addr,page_to_pfn (page), len, + vma->vm_page_prot); +if (ret) +return ret; + addr += len; +if (addr >= vma->vm_end) +return 0 ; + } + +return 0 ; +} + + +void dma_heap_buffer_destroy (struct dma_heap_buffer *heap_buffer) +{ +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + +if (buffer->kmap_cnt >0 ) { +pr_warn_once ("%s: buffer still mapped in the kernel\n" , + __func__); +vunmap (buffer->vaddr); + } + + buffer->free (buffer); +} + +static void *dma_heap_buffer_kmap_get (struct dma_heap_buffer *heap_buffer) +{ +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); +void *vaddr; + +if (buffer->kmap_cnt) { + buffer->kmap_cnt++; +return buffer->vaddr; + } + vaddr =dma_heap_map_kernel (buffer); +if (WARN_ONCE (!vaddr, +"heap->ops->map_kernel should return ERR_PTR on error" )) +return ERR_PTR (-EINVAL ); +if (IS_ERR (vaddr)) +return vaddr; + buffer->vaddr = vaddr; + buffer->kmap_cnt++; +return vaddr; +} + +static void dma_heap_buffer_kmap_put (struct dma_heap_buffer *heap_buffer) +{ +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + + buffer->kmap_cnt--; +if (!buffer->kmap_cnt) { +vunmap (buffer->vaddr); + buffer->vaddr =NULL ; + } +} + +static struct sg_table *dup_sg_table (struct sg_table *table) +{ +struct sg_table *new_table; +int ret, i; +struct scatterlist *sg, *new_sg; + + new_table =kzalloc (sizeof (*new_table),GFP_KERNEL ); +if (!new_table) +return ERR_PTR (-ENOMEM ); + + ret =sg_alloc_table (new_table, table->nents,GFP_KERNEL ); +if (ret) { +kfree (new_table); +return ERR_PTR (-ENOMEM ); + } + + new_sg = new_table->sgl; +for_each_sg (table->sgl, sg, table->nents, i) { +memcpy (new_sg, sg,sizeof (*sg)); + new_sg->dma_address =0 ; + new_sg =sg_next (new_sg); + } + +return new_table; +} + +static void free_duped_table (struct sg_table *table) +{ +sg_free_table (table); +kfree (table); +} + +struct dma_heaps_attachment { +struct device *dev; +struct sg_table *table; +struct list_head list; +}; + +static int dma_heap_attach (struct dma_buf *dmabuf, +struct dma_buf_attachment *attachment) +{ +struct dma_heaps_attachment *a; +struct sg_table *table; +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + + a =kzalloc (sizeof (*a),GFP_KERNEL ); +if (!a) +return -ENOMEM ; + + table =dup_sg_table (buffer->sg_table); +if (IS_ERR (table)) { +kfree (a); +return -ENOMEM ; + } + + a->table = table; + a->dev = attachment->dev; +INIT_LIST_HEAD (&a->list); + + attachment->priv = a; + +mutex_lock (&buffer->lock); +list_add (&a->list, &buffer->attachments); +mutex_unlock (&buffer->lock); + +return 0 ; +} + +static void dma_heap_detatch (struct dma_buf *dmabuf, +struct dma_buf_attachment *attachment) +{ +struct dma_heaps_attachment *a = attachment->priv; +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + +mutex_lock (&buffer->lock); +list_del (&a->list); +mutex_unlock (&buffer->lock); +free_duped_table (a->table); + +kfree (a); +} + +static struct sg_table *dma_heap_map_dma_buf(drivers/dma-buf/heaps/heap-helpers.c line 223
Quoted 10 lines
drivers/dma-buf/heaps/heap-helpers.c new file +10
-0+struct dma_buf_attachment *attachment, + enum dma_data_direction direction) +{ + struct dma_heaps_attachment *a = attachment->priv; +struct sg_table *table; + + table = a->table; + +if (!dma_map_sg (attachment->dev, table->sgl, table->nents, + direction))Since this code is used for system heap and and as the reference. In multimedia uses cases very large buffers can be allocated from system heap, and since system heap allocations have a cached kernel mapping it has been difficult to support uncached allocations so clients will likely allocate them as cached allocations.
Most access to these buffers will occur from non IO-coherent devices, however in frameworks such as Android these buffers will be dma mapped and dma unmapped frequently, for every frame and for each device in the "buffer pipeline", which leads to a lot of unnecessary cache maintenance in the dma map and dma unmap calls.
From previous discussions it doesn't seem like this could be optimized by only making changes to dma-buf heaps framework.So I think it would be helpful to try and agree on what types of changes would be required to the Android framework and possibly the dma-buf heaps framework to resolve this.
ExampleQuoted 6 lines
- Have Android framework keep buffers dma mapped for the whole use case
- Or perhaps have Android keep required devices attached to the buffer as they are "pipelined" so that cache maintenance can be skipped in dma map and dma umnap but reliably applied in begin/end_cpu_access.
I don't have a strong opinion on the solution here for Android, but from kernel/hardware side I can see we don't have a good ownership model for dma-buf and that looks to be causing your unneeded cache ops.
Let my give my vision for the model you request below.
Quoted 28 lines
Quoted 10 lines
drivers/dma-buf/heaps/heap-helpers.c new file +10
-0+ table =ERR_PTR (-ENOMEM ); +return table; +} + +static void dma_heap_unmap_dma_buf (struct dma_buf_attachment *attachment, +struct sg_table *table, +enum dma_data_direction direction) +{ +dma_unmap_sg (attachment->dev, table->sgl, table->nents, direction); +}drivers/dma-buf/heaps/heap-helpers.c line 243
Quoted 10 lines
drivers/dma-buf/heaps/heap-helpers.c new file +10
-0+ +static int dma_heap_mmap (struct dma_buf *dmabuf,struct vm_area_struct *vma) +{ +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); +int ret =0 ; + +mutex_lock (&buffer->lock); +/* now map it to userspace */ + ret =dma_heap_map_user (buffer, vma);Since this code is used for system heap and and as the reference. Currently in Android when a system heap buffer is moving down the "pipeline" CPU access may occur and there may be no device attached. In the past people have raised concerns that this should perhaps not be supported and that at least one device should be attached when CPU access occurs.
Quoted 7 lines
Can we decide, from a dma-buf contract perspective, on whether CPU access without a device attached should allowed and make it clear in the dma-buf documentation either way?
If it should not be allowed we can try to work with Android to see how they can change their framework to align with the dma-buf spec.
As you say, right now DMA-BUF does not really provide guidelines for "well behaved" heaps, and as such we all end up making different assumptions. Although I don't think they below should be enforced for all DMA-BUFs it should be used for our first set of reference heaps.
So from the start, lets looks at DMA-BUF buffer lifetimes. A buffer begins by being allocated, at this point the backing resources (usually some pages in physical CPU addressable RAM) have *not* been allocated this is a core feature of DMA-BUF allowing smart allocations at map time. At this point we should consider the buffer in a state "non-owned non-backed".
Next we have three valid actions:
* CPU use
* device attachments
* freeCPU use starts with a call to
begin_cpu_access(), the buffer will now both become CPU owned and will pin it's backing buffer. This last part can causebegin_cpu_access()to fail for some types of buffers that only pick their backing resource based on the set of attached devices, which at this point we have none. For heaps like "system" heap this should not be a problem as the backing storage is already selected at allocation time.Device attachments and mapping should behave as they do now. After mapping, the buffer is now owned by the attached devices and backed. The ownership can be transfered to the CPU and back as usual with {begin,end}_cpu_access().
Free right after allocation without anyone taking ownership is trivial.
Now here is where things may get interesting, we can now be in the following states:
CPU owned backed
Device owned backedQuoted 8 lines
What should we do when CPU owned and
end_cpu_access()is called or when Device owned and all devices detach. What state do we go into? My opinion would be to return to "non-owned non-backed" which means the backing resource can be freed. The other option is to leave it backed. I beleave this is what Android expects right now as it returns buffers to a "BufferQueue" to later be dequeued and used again in the same way without time consuming reallocations.I think the issue may be more serious with Android than allow improving the performance of buffer re-allocation.
In cases such as video playback:
#1 video device reads and transforms the video data #2 optionally there may be some software video post processing #3 surface flinger
#4 HAL
#5 display device eventually gets the buffer
Quoted 9 lines
My understanding is the last device attached to the buffer (the video device) can be detached before Android sends the buffer down the "pipeline" where some software module may do CPU access and where eventually a new device is attached when the buffer is going to be acceded by a device.
So Android is counting on the contents of the buffer being retained while it is "non-owned".
It would not be owned, but still "backed", after it has been once attached/mapped it should it always keep the same data, even if that data has to be migrated to different backing resources based on new attachments. For instance if the video device and the display device have some memory that the other cannot access, after the video device maps it the heap may put the backing memory in a spot that cannot be reached by the display device.
This is why a core part of DMA-BUF is that all users should be attached before the first map so smart decisions can be made about the backing memory. What Android does here is to assume DMA-BUF is always like regular memory, it gets no use from the late-allocate features and so may suffer some performance penalties in some cases for some heaps.
Quoted 11 lines
Quoted 4 lines
For "system" heaps the result is the same outside of the cache ops, if we maintain a buffer state like the above we can always know when a cache op or similar is needed (only when transitioning owners, not on every map/unmap).
Assuming I understood your comments correctly, I agree that we will know "when" to optimally apply the cache maintenance but the problem with allowing CPU access to buffers that are now "non-owned" is that we can't apply the cache maintenance when it is required (because there is no longer a device available).
And without knowing what device was last attached we really cannot always do the right thing. For example, if our above video device was IO coherent the written data may have landed in L3 cache. So the standard cache maintenance (cache invalidate) we would normally do would actually wipe out the data we are looking for, but if it was not IO coherent it would be needed. This is why the DMA framework doesn't like doing maintenance without targeted device.
What you could do if you really want those savings (I'm still not sure they are worth anything, but it's your platform) you could track the set of last attached devices, then only when needed perform the ops. I don't think that will scale very well, to me the last valid time to do the maintenance is on device unmap, and so unless we can prove the CPU will not access the data between then and the next map (and that the next mapping device isn't IO coherent), then I really cant see a generic way to avoid those ops.
Andrew
Quoted 244 lines
Quoted 140 lines
For more complex heaps we can do something similar
when transitioning from backed to non-backed (and vice-versa) such as migrating the backing data to a more appropriate resource given the new set of attaching/mapping devices.A more complete state transition map will probably be needed to fill out what should be allowed and what to do when, and I agree it will be good to have that for this first set of reference heaps. But all of this can be done now with DMA-heaps, so I don't want to tie up the core de-staging too much on getting every possible heap to behave just right either..
Quoted 50 lines
Quoted 32 lines
drivers/dma-buf/heaps/heap-helpers.c new file +32
-0+mutex_unlock (&buffer->lock); + +if (ret) +pr_err ("%s: failure mapping buffer to userspace\n" , + __func__); + +return ret; +} + +static void dma_heap_dma_buf_release (struct dma_buf *dmabuf) +{ +struct dma_heap_buffer *buffer = dmabuf->priv; + +dma_heap_buffer_destroy (buffer); +} + +static void *dma_heap_dma_buf_kmap (struct dma_buf *dmabuf, +unsigned long offset) +{ +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + +return buffer->vaddr + offset *PAGE_SIZE ; +} + +static void dma_heap_dma_buf_kunmap (struct dma_buf *dmabuf, +unsigned long offset, +void *ptr) +{ +} + +static int dma_heap_dma_buf_begin_cpu_access(struct dma_buf *dmabuf,drivers/dma-buf/heaps/heap-helpers.c line 285
Quoted 10 lines
drivers/dma-buf/heaps/heap-helpers.c new file +10
-0+ enum dma_data_direction direction) +{ +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); +void *vaddr; +struct dma_heaps_attachment *a; +int ret =0 ; + +mutex_lock (&buffer->lock); + vaddr =dma_heap_buffer_kmap_get (heap_buffer);Since this code is used for system heap and and as the reference. As has been discussed in the past there are several disadvantages to creating a kernel mapping on each call to
begin_cpu_access.The resulting call to
alloc_vmap_areais expensive and can hurt client KPIsQuoted 8 lines
Allowing userspace clients to create and destroy kernel mappings can provide opportunities to crash the kernel.
Can we look at removing the creation of a kernel mapping in
begin_cpu_accessand either introduce support fordma_buf_vmapand have clients use that instead or perahps change
the contract fordma_buf_kmapso that it doesn't always need to succeed?Agree, for this we should just fail (or succeed but perform no action?) if there are no active mappings (neither in kernel (kmap, vmap) nor in userspace (mmap)), forcing a new mapping just to keep
dma_sync_sg_for_cpu()happy is not the correct thing to do here.Quoted 8 lines
Quoted 8 lines
drivers/dma-buf/heaps/heap-helpers.c new file +8
-0+if (IS_ERR (vaddr)) { + ret =PTR_ERR (vaddr); + goto unlock; + } +mutex_unlock (&buffer->lock); + +mutex_lock (&buffer->lock); +list_for_each_entry (a, &buffer->attachments, list) {Quoted 9 lines
drivers/dma-buf/heaps/heap-helpers.c line 295
Quoted 2 lines
drivers/dma-buf/heaps/heap-helpers.c new file +2
-0+dma_sync_sg_for_cpu (a->dev, a->table->sgl, a->table->nents, + direction);Since this code is used for system heap and and as the reference. Not a major issue for newer kernels but I still don't think it makes sense to apply cache maintenance when the buffer is not dma mapped, it doesn't makes sense to me from a logical perspective and from a performance perspective.
Agree again, this is only needed when we are transitioning out of state "device owned" which we will not be in if we have no active attached/mapped devices.
Quoted 18 lines
Quoted 18 lines
drivers/dma-buf/heaps/heap-helpers.c new file +17
-0+ } + +unlock: +mutex_unlock (&buffer->lock); +return ret; +} + +static int dma_heap_dma_buf_end_cpu_access (struct dma_buf *dmabuf, +enum dma_data_direction direction) +{ +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); +struct dma_heaps_attachment *a; + +mutex_lock (&buffer->lock); +dma_heap_buffer_kmap_put (heap_buffer); +mutex_unlock (&buffer->lock);Quoted 9 lines
drivers/dma-buf/heaps/heap-helpers.c line 317
Quoted 5 lines
drivers/dma-buf/heaps/heap-helpers.c new file +5
-0+ +mutex_lock (&buffer->lock); +list_for_each_entry (a, &buffer->attachments, list) { +dma_sync_sg_for_device (a->dev, a->table->sgl, a->table->nents, + direction);There are use cases in Android which result in only small parts of a large buffer are written do during of CPU access.
We have the same requirements for some OpenCL use-cases (clCreateSubBuffer), which allows for chunks of a larger buffer to have different properties and therefor cache operations different from the larger parrent buffer.
Quoted 7 lines
Applying cache maintenance to the complete buffer results in a lot of unnecessary cache maintenance that can affect KPIs.
I believe the Android team is wondering if there could be a way to support partial maintenance in where userspace could describe the buffer changes they have made.
Quoted 6 lines
The issue I have with that is it would require userspace to know about the cache line size of all involved devices, otherwise you would have users writing to the first 64B and invalidate to read the second 64B written from a DMA device outside the cache, in a system with 128B cache-lines you will always lose some data here.
Good point.
Quoted 5 lines
Quoted 4 lines
I think it would be useful to make sure that there is a least a path forward with the current dma-buf heaps framework to solve this for system heap allocations.
Quoted 89 lines
DMA-BUF sync operations always act on the full buffer, changing this would require changes to the core DMA-BUF framework (so outside the scope of this set) :)
Andrew
Quoted 82 lines
I can get more details on the specific use cases if required.
Quoted 76 lines
drivers/dma-buf/heaps/heap-helpers.c new file +18
-0+ } +mutex_unlock (&buffer->lock); + +return 0 ; +} + +const struct dma_buf_ops heap_helper_ops = { + .map_dma_buf = dma_heap_map_dma_buf, + .unmap_dma_buf = dma_heap_unmap_dma_buf, + .mmap = dma_heap_mmap, + .release = dma_heap_dma_buf_release, + .attach = dma_heap_attach, + .detach = dma_heap_detatch, + .begin_cpu_access = dma_heap_dma_buf_begin_cpu_access, + .end_cpu_access = dma_heap_dma_buf_end_cpu_access, + .map = dma_heap_dma_buf_kmap, + .unmap = dma_heap_dma_buf_kunmap, +};drivers/dma-buf/heaps/heap-helpers.h new file +48
-0@@ -0,0 +1,48 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* DMABUF Heaps helper code +* +* Copyright (C) 2011 Google, Inc. +* Copyright (C) 2019 Linaro Ltd. +*/ + +#ifndef _HEAP_HELPERS_H +#define _HEAP_HELPERS_H + +#include <linux/dma-heap.h> +#include <linux/list.h> + +struct heap_helper_buffer { +struct dma_heap_buffer heap_buffer; + +unsigned long private_flags; +void *priv_virt; +struct mutex lock; +int kmap_cnt; +void *vaddr; +struct sg_table *sg_table; +struct list_head attachments; + +void (*free)(struct heap_helper_buffer *buffer); + +}; + +#define to_helper_buffer (x) \ + container_of(x, struct heap_helper_buffer, heap_buffer) + +static inline void INIT_HEAP_HELPER_BUFFER (struct heap_helper_buffer *buffer, +void (*free)(struct heap_helper_buffer *)) +{ + buffer->private_flags =0 ; + buffer->priv_virt =NULL ; +mutex_init (&buffer->lock); + buffer->kmap_cnt =0 ; + buffer->vaddr =NULL ; + buffer->sg_table =NULL ; +INIT_LIST_HEAD (&buffer->attachments); + buffer->free = free; +} + +extern const struct dma_buf_ops heap_helper_ops; + +#endif /* _HEAP_HELPERS_H */ Signature · 1 line
2.7.4Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
Quoted 3 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [RFC][PATCH 2/5 v2] dma-buf: heaps: Add heap helpers
In reply to John StultzQuoted 14 lines
drivers/dma-buf/heaps/heap-helpers.c new file +14
-0+static void *dma_heap_map_kernel (struct heap_helper_buffer *buffer) +{ +struct scatterlist *sg; +int i, j; +void *vaddr; +pgprot_t pgprot; +struct sg_table *table = buffer->sg_table; +int npages =PAGE_ALIGN (buffer->heap_buffer.size) /PAGE_SIZE ; +struct page **pages =vmalloc (array_size (npages, +sizeof (struct page *))); +struct page **tmp = pages; + +if (!pages) +return ERR_PTR (-ENOMEM );
drivers/dma-buf/heaps/heap-helpers.c line 37
Quoted 10 lines
drivers/dma-buf/heaps/heap-helpers.c new file +10
-0+ + pgprot =PAGE_KERNEL ; + +for_each_sg (table->sgl, sg, table->nents, i) { +int npages_this_entry =PAGE_ALIGN (sg->length) /PAGE_SIZE ; +struct page *page =sg_page (sg); + +WARN_ON (i >= npages); +for (j =0 ; j < npages_this_entry; j++) + *(tmp++) = page++;
This should probably use nth_page.
That being said I really wish we could have a more iterative version of vmap, where the caller does a get_vm_area_caller and then adds each chunk using another call, including the possibility of mapping larger than PAGE_SIZE contigous ones. Any chance you could look into that?
drivers/dma-buf/heaps/heap-helpers.c line 73
Quoted 2 lines
drivers/dma-buf/heaps/heap-helpers.c new file +2
-0+ ret =remap_pfn_range (vma, addr,page_to_pfn (page), len, + vma->vm_page_prot);
So the same chunk could be mapped to userspace and vmap, and later on also DMA mapped. Who is going to take care of cache aliasing as I see nothing of that in this series?
drivers/dma-buf/heaps/heap-helpers.c line 114
Quoted 12 lines
drivers/dma-buf/heaps/heap-helpers.c new file +12
-0+if (buffer->kmap_cnt) { + buffer->kmap_cnt++; +return buffer->vaddr; + } + vaddr =dma_heap_map_kernel (buffer); +if (WARN_ONCE (!vaddr, +"heap->ops->map_kernel should return ERR_PTR on error" )) +return ERR_PTR (-EINVAL ); +if (IS_ERR (vaddr)) +return vaddr; + buffer->vaddr = vaddr; + buffer->kmap_cnt++;
The cnt manioulation is odd. The normal way to make this readable is to use a postfix op on the check, as that makes it clear to everyone, e.g.:
if (buffer->kmap_cnt++) return buffer->vaddr; ..
drivers/dma-buf/heaps/heap-helpers.c line 126
Quoted 5 lines
drivers/dma-buf/heaps/heap-helpers.c new file +5
-0+ buffer->kmap_cnt--; +if (!buffer->kmap_cnt) { +vunmap (buffer->vaddr); + buffer->vaddr =NULL ; + }
Same here, just with an infix.
drivers/dma-buf/heaps/heap-helpers.h line 44
Quoted 12 lines
drivers/dma-buf/heaps/heap-helpers.h new file +12
-0+static inline void INIT_HEAP_HELPER_BUFFER (struct heap_helper_buffer *buffer, +void (*free)(struct heap_helper_buffer *)) +{ + buffer->private_flags =0 ; + buffer->priv_virt =NULL ; +mutex_init (&buffer->lock); + buffer->kmap_cnt =0 ; + buffer->vaddr =NULL ; + buffer->sg_table =NULL ; +INIT_LIST_HEAD (&buffer->attachments); + buffer->free = free; +}
There is absolutely no reason to inlines this as far as I can tell.
Also it would seem much simpler to simply let the caller assign the free callback.
Re: [RFC][PATCH 2/5 v2] dma-buf: heaps: Add heap helpers
Quoted 7 lines · Christoph Hellwig
drivers/dma-buf/heaps/heap-helpers.c line 73
Quoted 2 lines
drivers/dma-buf/heaps/heap-helpers.c new file +2
-0+ ret =remap_pfn_range (vma, addr,page_to_pfn (page), len, + vma->vm_page_prot);So the same chunk could be mapped to userspace and vmap, and later on also DMA mapped. Who is going to take care of cache aliasing as I see nothing of that in this series?
We should only have one type of memory per heap so all mappings will have the same type. That should solve the ARM specific issues, but I'm guessing you are thinking of more tricky architectures where all mappings need to be tracked and cleaned/invalidated..
For that I think we will have to track each right? How do others handle that, we can't be the first to offer cached buffers to userspace.
Re: [RFC][PATCH 2/5 v2] dma-buf: heaps: Add heap helpers
In reply to Christoph HellwigQuoted 13 lines · Christoph Hellwig
drivers/dma-buf/heaps/heap-helpers.c line 114
Quoted 12 lines
drivers/dma-buf/heaps/heap-helpers.c new file +12
-0+if (buffer->kmap_cnt) { + buffer->kmap_cnt++; +return buffer->vaddr; + } + vaddr =dma_heap_map_kernel (buffer); +if (WARN_ONCE (!vaddr, +"heap->ops->map_kernel should return ERR_PTR on error" )) +return ERR_PTR (-EINVAL ); +if (IS_ERR (vaddr)) +return vaddr; + buffer->vaddr = vaddr; + buffer->kmap_cnt++;
Quoted 7 lines
The cnt manioulation is odd. The normal way to make this readable is to use a postfix op on the check, as that makes it clear to everyone, e.g.:
if (buffer->kmap_cnt++) return buffer->vaddr; ..
Thanks again on the feedback, I have had some other distractions recently, so just getting around to these details again now.
The trouble w/ the suggestion here, if we increment in the check, then if we trip on any of the other error paths, the cnt value will be wrong when we return (and doing the extra decrementing in the error paths feels as ugly as just doing the increment at the end of the success paths)
Quoted 8 lines
drivers/dma-buf/heaps/heap-helpers.c line 126
Quoted 5 lines
drivers/dma-buf/heaps/heap-helpers.c new file +5
-0+ buffer->kmap_cnt--; +if (!buffer->kmap_cnt) { +vunmap (buffer->vaddr); + buffer->vaddr =NULL ; + }Same here, just with an infix.
Quoted 14 lines
drivers/dma-buf/heaps/heap-helpers.h line 44
Quoted 12 lines
drivers/dma-buf/heaps/heap-helpers.h new file +12
-0+static inline void INIT_HEAP_HELPER_BUFFER (struct heap_helper_buffer *buffer, +void (*free)(struct heap_helper_buffer *)) +{ + buffer->private_flags =0 ; + buffer->priv_virt =NULL ; +mutex_init (&buffer->lock); + buffer->kmap_cnt =0 ; + buffer->vaddr =NULL ; + buffer->sg_table =NULL ; +INIT_LIST_HEAD (&buffer->attachments); + buffer->free = free; +}There is absolutely no reason to inlines this as far as I can tell.
Yea, I think I was mimicing some of the helpers like INIT_LIST_HEAD() But sounds good. I can uninline it.
Quoted 2 lines
Also it would seem much simpler to simply let the caller assign the free callback.
Yea, its a bit ugly but I worry the caller might forget?
Thanks again for the feedback!
-john
Brian Starkey
reviewerRe: [RFC][PATCH 2/5 v2] dma-buf: heaps: Add heap helpers
In reply to John StultzHi John,
On Tue, Mar 05, 2019 at 12:54:30PM -0800, John Stultz wrote:
...
drivers/dma-buf/heaps/heap-helpers.c line 91
Quoted 8 lines
drivers/dma-buf/heaps/heap-helpers.c new file +8
-0+ +void dma_heap_buffer_destroy (struct dma_heap_buffer *heap_buffer) +{ +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + +if (buffer->kmap_cnt >0 ) { +pr_warn_once ("%s: buffer still mapped in the kernel\n" , + __func__);
Could be worth something louder like a full WARN.
Quoted 6 lines
drivers/dma-buf/heaps/heap-helpers.c new file +6
-0+vunmap (buffer->vaddr); + } + + buffer->free (buffer); +} +
...
drivers/dma-buf/heaps/heap-helpers.c line 266
Quoted 8 lines
drivers/dma-buf/heaps/heap-helpers.c new file +8
-0+ +static void *dma_heap_dma_buf_kmap (struct dma_buf *dmabuf, +unsigned long offset) +{ +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + +return buffer->vaddr + offset *PAGE_SIZE ;
I think it'd be good to check for NULL vaddr and return NULL in that case. Less chance of an invalid pointer being accidentally used then.
Thanks,
-Brian
Re: [RFC][PATCH 2/5 v2] dma-buf: heaps: Add heap helpers
Quoted 27 lines · Brian Starkey
Hi John,
On Tue, Mar 05, 2019 at 12:54:30PM -0800, John Stultz wrote:
...
drivers/dma-buf/heaps/heap-helpers.c line 91
Quoted 8 lines
drivers/dma-buf/heaps/heap-helpers.c new file +8
-0+ +void dma_heap_buffer_destroy (struct dma_heap_buffer *heap_buffer) +{ +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + +if (buffer->kmap_cnt >0 ) { +pr_warn_once ("%s: buffer still mapped in the kernel\n" , + __func__);Could be worth something louder like a full WARN.
Quoted 6 lines
drivers/dma-buf/heaps/heap-helpers.c new file +6
-0+vunmap (buffer->vaddr); + } + + buffer->free (buffer); +} +...
Quoted 11 lines
drivers/dma-buf/heaps/heap-helpers.c line 266
Quoted 8 lines
drivers/dma-buf/heaps/heap-helpers.c new file +8
-0+ +static void *dma_heap_dma_buf_kmap (struct dma_buf *dmabuf, +unsigned long offset) +{ +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + +return buffer->vaddr + offset *PAGE_SIZE ;I think it'd be good to check for NULL vaddr and return NULL in that case. Less chance of an invalid pointer being accidentally used then.
Thanks so much for the feedback! I've added these two suggestions in! -john
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [RFC][PATCH 2/5 v2] dma-buf: heaps: Add heap helpers
In reply to Brian StarkeyQuoted 26 lines · Brian Starkey
Hi John,
On Tue, Mar 05, 2019 at 12:54:30PM -0800, John Stultz wrote:
...
drivers/dma-buf/heaps/heap-helpers.c line 91
Quoted 8 lines
drivers/dma-buf/heaps/heap-helpers.c new file +8
-0+ +void dma_heap_buffer_destroy (struct dma_heap_buffer *heap_buffer) +{ +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + +if (buffer->kmap_cnt >0 ) { +pr_warn_once ("%s: buffer still mapped in the kernel\n" , + __func__);Could be worth something louder like a full WARN.
Quoted 6 lines
drivers/dma-buf/heaps/heap-helpers.c new file +6
-0+vunmap (buffer->vaddr); + } + + buffer->free (buffer); +} +...
Quoted 12 lines
drivers/dma-buf/heaps/heap-helpers.c line 266
Quoted 8 lines
drivers/dma-buf/heaps/heap-helpers.c new file +8
-0+ +static void *dma_heap_dma_buf_kmap (struct dma_buf *dmabuf, +unsigned long offset) +{ +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + +return buffer->vaddr + offset *PAGE_SIZE ;I think it'd be good to check for NULL vaddr and return NULL in that case. Less chance of an invalid pointer being accidentally used then.
Why do we assume vaddr is set at all here? I'm guessing we expected dma_heap_map_kernel to have been called, that is not going to always be the case. kmap should perform it's own single page kmap here and not rely on the clunky full buffer vmap (which is probably broken on 32bit systems here when the buffers are large).
Andrew
Quoted 3 lines
Thanks,
-Brian
Re: [RFC][PATCH 2/5 v2] dma-buf: heaps: Add heap helpers
In reply to John StultzQuoted 155 lines · John Stultz
Add generic helper dmabuf ops for dma heaps, so we can reduce the amount of duplicative code for the exported dmabufs.
This code is an evolution of the Android ION implementation, so thanks to its original authors and maintainters:
Rebecca Schultz Zavin, Colin Cross, Laura Abbott, and others!Cc Laura Abbott <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Greg KH <[email protected]>Cc Sumit Semwal <[email protected]>Cc Liam Mark <[email protected]>Cc Brian Starkey <[email protected]>Cc Andrew F. Davis <[email protected]>Cc Chenbo Feng <[email protected]>Cc Alistair Strachan <[email protected]>Signed-off-by John Stultz <[email protected]>---
v2:
* Removed cache management performance hack that I had accidentally folded in.
* Removed stats code that was in helpers
* Lots of checkpatch cleanups--- drivers/dma-buf/Makefile | 1 + drivers/dma-buf/heaps/Makefile | 2 + drivers/dma-buf/heaps/heap-helpers.c | 335 +++++++++++++++++++++++++++++++++++ drivers/dma-buf/heaps/heap-helpers.h | 48 +++++ 4 files changed, 386 insertions(+) create mode 100644 drivers/dma-buf/heaps/Makefile create mode 100644 drivers/dma-buf/heaps/heap-helpers.c create mode 100644 drivers/dma-buf/heaps/heap-helpers.hdrivers/dma-buf/Makefile +1
-0@@ -1,4 +1,5 @@ obj-y := dma-buf.o dma-fence.o dma-fence-array.o reservation.o seqno-fence.o +obj-$(CONFIG_DMABUF_HEAPS) += heaps/ obj-$(CONFIG_DMABUF_HEAPS) += dma-heap.o obj-$(CONFIG_SYNC_FILE) += sync_file.o obj-$(CONFIG_SW_SYNC) += sw_sync.o sync_debug.odrivers/dma-buf/heaps/Makefile new file +2
-0@@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-y += heap-helpers.odrivers/dma-buf/heaps/heap-helpers.c new file +97
-0@@ -0,0 +1,335 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/device.h> +#include <linux/dma-buf.h> +#include <linux/err.h> +#include <linux/idr.h> +#include <linux/list.h> +#include <linux/slab.h> +#include <linux/uaccess.h> +#include <uapi/linux/dma-heap.h> + +#include "heap-helpers.h" + + +static void *dma_heap_map_kernel (struct heap_helper_buffer *buffer) +{ +struct scatterlist *sg; +int i, j; +void *vaddr; +pgprot_t pgprot; +struct sg_table *table = buffer->sg_table; +int npages =PAGE_ALIGN (buffer->heap_buffer.size) /PAGE_SIZE ; +struct page **pages =vmalloc (array_size (npages, +sizeof (struct page *))); +struct page **tmp = pages; + +if (!pages) +return ERR_PTR (-ENOMEM ); + + pgprot =PAGE_KERNEL ; + +for_each_sg (table->sgl, sg, table->nents, i) { +int npages_this_entry =PAGE_ALIGN (sg->length) /PAGE_SIZE ; +struct page *page =sg_page (sg); + +WARN_ON (i >= npages); +for (j =0 ; j < npages_this_entry; j++) + *(tmp++) = page++; + } + vaddr =vmap (pages, npages,VM_MAP , pgprot); +vfree (pages); + +if (!vaddr) +return ERR_PTR (-ENOMEM ); + +return vaddr; +} + +static int dma_heap_map_user (struct heap_helper_buffer *buffer, +struct vm_area_struct *vma) +{ +struct sg_table *table = buffer->sg_table; +unsigned long addr = vma->vm_start; +unsigned long offset = vma->vm_pgoff *PAGE_SIZE ; +struct scatterlist *sg; +int i; +int ret; + +for_each_sg (table->sgl, sg, table->nents, i) { +struct page *page =sg_page (sg); +unsigned long remainder = vma->vm_end - addr; +unsigned long len = sg->length; + +if (offset >= sg->length) { + offset -= sg->length; +continue ; + }else if (offset) { + page += offset /PAGE_SIZE ; + len = sg->length - offset; + offset =0 ; + } + len =min (len, remainder); + ret =remap_pfn_range (vma, addr,page_to_pfn (page), len, + vma->vm_page_prot); +if (ret) +return ret; + addr += len; +if (addr >= vma->vm_end) +return 0 ; + } + +return 0 ; +} + + +void dma_heap_buffer_destroy (struct dma_heap_buffer *heap_buffer) +{ +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + +if (buffer->kmap_cnt >0 ) { +pr_warn_once ("%s: buffer still mapped in the kernel\n" , + __func__); +vunmap (buffer->vaddr); + } + + buffer->free (buffer); +} +
drivers/dma-buf/heaps/heap-helpers.c line 107
Quoted 10 lines
drivers/dma-buf/heaps/heap-helpers.c new file +10
-0+static void *dma_heap_buffer_kmap_get (struct dma_heap_buffer *heap_buffer) +{ +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); +void *vaddr; + +if (buffer->kmap_cnt) { + buffer->kmap_cnt++; +return buffer->vaddr; + } + vaddr =dma_heap_map_kernel (buffer);
The function is called "kmap" but here we go and vmap the whole buffer. The function names are not consistent here.
Quoted 126 lines
drivers/dma-buf/heaps/heap-helpers.c new file +126
-0+if (WARN_ONCE (!vaddr, +"heap->ops->map_kernel should return ERR_PTR on error" )) +return ERR_PTR (-EINVAL ); +if (IS_ERR (vaddr)) +return vaddr; + buffer->vaddr = vaddr; + buffer->kmap_cnt++; +return vaddr; +} + +static void dma_heap_buffer_kmap_put (struct dma_heap_buffer *heap_buffer) +{ +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + + buffer->kmap_cnt--; +if (!buffer->kmap_cnt) { +vunmap (buffer->vaddr); + buffer->vaddr =NULL ; + } +} + +static struct sg_table *dup_sg_table (struct sg_table *table) +{ +struct sg_table *new_table; +int ret, i; +struct scatterlist *sg, *new_sg; + + new_table =kzalloc (sizeof (*new_table),GFP_KERNEL ); +if (!new_table) +return ERR_PTR (-ENOMEM ); + + ret =sg_alloc_table (new_table, table->nents,GFP_KERNEL ); +if (ret) { +kfree (new_table); +return ERR_PTR (-ENOMEM ); + } + + new_sg = new_table->sgl; +for_each_sg (table->sgl, sg, table->nents, i) { +memcpy (new_sg, sg,sizeof (*sg)); + new_sg->dma_address =0 ; + new_sg =sg_next (new_sg); + } + +return new_table; +} + +static void free_duped_table (struct sg_table *table) +{ +sg_free_table (table); +kfree (table); +} + +struct dma_heaps_attachment { +struct device *dev; +struct sg_table *table; +struct list_head list; +}; + +static int dma_heap_attach (struct dma_buf *dmabuf, +struct dma_buf_attachment *attachment) +{ +struct dma_heaps_attachment *a; +struct sg_table *table; +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + + a =kzalloc (sizeof (*a),GFP_KERNEL ); +if (!a) +return -ENOMEM ; + + table =dup_sg_table (buffer->sg_table); +if (IS_ERR (table)) { +kfree (a); +return -ENOMEM ; + } + + a->table = table; + a->dev = attachment->dev; +INIT_LIST_HEAD (&a->list); + + attachment->priv = a; + +mutex_lock (&buffer->lock); +list_add (&a->list, &buffer->attachments); +mutex_unlock (&buffer->lock); + +return 0 ; +} + +static void dma_heap_detatch (struct dma_buf *dmabuf, +struct dma_buf_attachment *attachment) +{ +struct dma_heaps_attachment *a = attachment->priv; +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + +mutex_lock (&buffer->lock); +list_del (&a->list); +mutex_unlock (&buffer->lock); +free_duped_table (a->table); + +kfree (a); +} + +static struct sg_table *dma_heap_map_dma_buf ( +struct dma_buf_attachment *attachment, +enum dma_data_direction direction) +{ +struct dma_heaps_attachment *a = attachment->priv; +struct sg_table *table; + + table = a->table; + +if (!dma_map_sg (attachment->dev, table->sgl, table->nents, + direction)) + table =ERR_PTR (-ENOMEM ); +return table; +} + +static void dma_heap_unmap_dma_buf (struct dma_buf_attachment *attachment, +struct sg_table *table, +enum dma_data_direction direction) +{ +dma_unmap_sg (attachment->dev, table->sgl, table->nents, direction); +}
drivers/dma-buf/heaps/heap-helpers.c line 243
Quoted 10 lines
drivers/dma-buf/heaps/heap-helpers.c new file +10
-0+ +static int dma_heap_mmap (struct dma_buf *dmabuf,struct vm_area_struct *vma) +{ +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); +int ret =0 ; + +mutex_lock (&buffer->lock); +/* now map it to userspace */ + ret =dma_heap_map_user (buffer, vma);
Only used here, we can just put is functions code here. Also do we need this locked? What can race here? Everything accessed is static for the buffers lifetime.
Quoted 32 lines
drivers/dma-buf/heaps/heap-helpers.c new file +32
-0+mutex_unlock (&buffer->lock); + +if (ret) +pr_err ("%s: failure mapping buffer to userspace\n" , + __func__); + +return ret; +} + +static void dma_heap_dma_buf_release (struct dma_buf *dmabuf) +{ +struct dma_heap_buffer *buffer = dmabuf->priv; + +dma_heap_buffer_destroy (buffer); +} + +static void *dma_heap_dma_buf_kmap (struct dma_buf *dmabuf, +unsigned long offset) +{ +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); + +return buffer->vaddr + offset *PAGE_SIZE ; +} + +static void dma_heap_dma_buf_kunmap (struct dma_buf *dmabuf, +unsigned long offset, +void *ptr) +{ +} + +static int dma_heap_dma_buf_begin_cpu_access(struct dma_buf *dmabuf,
drivers/dma-buf/heaps/heap-helpers.c line 285
Quoted 10 lines
drivers/dma-buf/heaps/heap-helpers.c new file +10
-0+ enum dma_data_direction direction) +{ +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); +void *vaddr; +struct dma_heaps_attachment *a; +int ret =0 ; + +mutex_lock (&buffer->lock); + vaddr =dma_heap_buffer_kmap_get (heap_buffer);
Not needed here, this can be dropped and so can the dma_heap_buffer_kmap_put() below.
Andrew
Quoted 95 lines
drivers/dma-buf/heaps/heap-helpers.c new file +50
-0+if (IS_ERR (vaddr)) { + ret =PTR_ERR (vaddr); + goto unlock; + } +mutex_unlock (&buffer->lock); + +mutex_lock (&buffer->lock); +list_for_each_entry (a, &buffer->attachments, list) { +dma_sync_sg_for_cpu (a->dev, a->table->sgl, a->table->nents, + direction); + } + +unlock: +mutex_unlock (&buffer->lock); +return ret; +} + +static int dma_heap_dma_buf_end_cpu_access (struct dma_buf *dmabuf, +enum dma_data_direction direction) +{ +struct dma_heap_buffer *heap_buffer = dmabuf->priv; +struct heap_helper_buffer *buffer =to_helper_buffer (heap_buffer); +struct dma_heaps_attachment *a; + +mutex_lock (&buffer->lock); +dma_heap_buffer_kmap_put (heap_buffer); +mutex_unlock (&buffer->lock); + +mutex_lock (&buffer->lock); +list_for_each_entry (a, &buffer->attachments, list) { +dma_sync_sg_for_device (a->dev, a->table->sgl, a->table->nents, + direction); + } +mutex_unlock (&buffer->lock); + +return 0 ; +} + +const struct dma_buf_ops heap_helper_ops = { + .map_dma_buf = dma_heap_map_dma_buf, + .unmap_dma_buf = dma_heap_unmap_dma_buf, + .mmap = dma_heap_mmap, + .release = dma_heap_dma_buf_release, + .attach = dma_heap_attach, + .detach = dma_heap_detatch, + .begin_cpu_access = dma_heap_dma_buf_begin_cpu_access, + .end_cpu_access = dma_heap_dma_buf_end_cpu_access, + .map = dma_heap_dma_buf_kmap, + .unmap = dma_heap_dma_buf_kunmap, +};drivers/dma-buf/heaps/heap-helpers.h new file +38
-0@@ -0,0 +1,48 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* DMABUF Heaps helper code +* +* Copyright (C) 2011 Google, Inc. +* Copyright (C) 2019 Linaro Ltd. +*/ + +#ifndef _HEAP_HELPERS_H +#define _HEAP_HELPERS_H + +#include <linux/dma-heap.h> +#include <linux/list.h> + +struct heap_helper_buffer { +struct dma_heap_buffer heap_buffer; + +unsigned long private_flags; +void *priv_virt; +struct mutex lock; +int kmap_cnt; +void *vaddr; +struct sg_table *sg_table; +struct list_head attachments; + +void (*free)(struct heap_helper_buffer *buffer); + +}; + +#define to_helper_buffer (x) \ + container_of(x, struct heap_helper_buffer, heap_buffer) + +static inline void INIT_HEAP_HELPER_BUFFER (struct heap_helper_buffer *buffer, +void (*free)(struct heap_helper_buffer *)) +{ + buffer->private_flags =0 ; + buffer->priv_virt =NULL ; +mutex_init (&buffer->lock);
drivers/dma-buf/heaps/heap-helpers.h line 48
Quoted 10 lines
drivers/dma-buf/heaps/heap-helpers.h new file +10
-0+ buffer->kmap_cnt =0 ; + buffer->vaddr =NULL ; + buffer->sg_table =NULL ; +INIT_LIST_HEAD (&buffer->attachments); + buffer->free = free; +} + +extern const struct dma_buf_ops heap_helper_ops; + +#endif /* _HEAP_HELPERS_H */
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
RFC PATCH v2 3/5 dma-buf: heaps: Add system heap to dmabuf heaps
In reply to John StultzThis patch adds system heap to the dma-buf heaps framework.
This allows applications to get a page-allocator backed dma-buf for non-contiguous memory.
This code is an evolution of the Android ION implementation, so thanks to its original authors and maintainters:
Rebecca Schultz Zavin, Colin Cross, Laura Abbott, and others!
---
v2:
* Switch allocate to return dmabuf fd
* Simplify init code
* Checkpatch fixups
* Droped dead system-contig code
--- drivers/dma-buf/Kconfig | 2 + drivers/dma-buf/heaps/Kconfig | 6 ++ drivers/dma-buf/heaps/Makefile | 1 + drivers/dma-buf/heaps/system_heap.c | 132 ++++++++++++++++++++++++++++++++++++ 4 files changed, 141 insertions(+) create mode 100644 drivers/dma-buf/heaps/Kconfig create mode 100644 drivers/dma-buf/heaps/system_heap.c
drivers/dma-buf/Kconfig +2 -0
@@ -47,4 +47,6 @@ menuconfig DMABUF_HEAPS this allows userspace to allocate dma-bufs that can be shared between drivers. +source "drivers/dma-buf/heaps/Kconfig" + endmenu
drivers/dma-buf/heaps/Kconfig new file +6 -0
@@ -0,0 +1,6 @@ +config DMABUF_HEAPS_SYSTEM + bool "DMA-BUF System Heap" + depends on DMABUF_HEAPS + help + Choose this option to enable the system dmabuf heap. The system heap + is backed by pages from the buddy allocator. If in doubt, say Y.
drivers/dma-buf/heaps/Makefile +1 -0
@@ -1,2 +1,3 @@ # SPDX-License-Identifier: GPL-2.0 obj-y += heap-helpers.o +obj-$(CONFIG_DMABUF_HEAPS_SYSTEM) += system_heap.o
drivers/dma-buf/heaps/system_heap.c new file +132 -0
132 lines not on this page · raw message
Signature · 1 line
2.7.4
Re: [RFC][PATCH 3/5 v2] dma-buf: heaps: Add system heap to dmabuf heaps
Quoted 21 lines · John Stultz
This patch adds system heap to the dma-buf heaps framework.
This allows applications to get a page-allocator backed dma-buf for non-contiguous memory.
This code is an evolution of the Android ION implementation, so thanks to its original authors and maintainters:
Rebecca Schultz Zavin, Colin Cross, Laura Abbott, and others!Cc Laura Abbott <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Greg KH <[email protected]>Cc Sumit Semwal <[email protected]>Cc Liam Mark <[email protected]>Cc Brian Starkey <[email protected]>Cc Andrew F. Davis <[email protected]>Cc Chenbo Feng <[email protected]>Cc Alistair Strachan <[email protected]>Signed-off-by John Stultz <[email protected]>
Quoted 6 lines
---
v2:
* Switch allocate to return dmabuf fd
* Simplify init code
* Checkpatch fixups
* Droped dead system-contig code
just few blank lines to remove.
Quoted 57 lines
--- drivers/dma-buf/Kconfig | 2 + drivers/dma-buf/heaps/Kconfig | 6 ++ drivers/dma-buf/heaps/Makefile | 1 + drivers/dma-buf/heaps/system_heap.c | 132 ++++++++++++++++++++++++++++++++++++ 4 files changed, 141 insertions(+) create mode 100644 drivers/dma-buf/heaps/Kconfig create mode 100644 drivers/dma-buf/heaps/system_heap.cdrivers/dma-buf/Kconfig +2
-0@@ -47,4 +47,6 @@ menuconfig DMABUF_HEAPS this allows userspace to allocate dma-bufs that can be shared between drivers. +source "drivers/dma-buf/heaps/Kconfig" + endmenudrivers/dma-buf/heaps/Kconfig new file +6
-0@@ -0,0 +1,6 @@ +config DMABUF_HEAPS_SYSTEM + bool "DMA-BUF System Heap" + depends on DMABUF_HEAPS + help + Choose this option to enable the system dmabuf heap. The system heap + is backed by pages from the buddy allocator. If in doubt, say Y.drivers/dma-buf/heaps/Makefile +1
-0@@ -1,2 +1,3 @@ # SPDX-License-Identifier: GPL-2.0 obj-y += heap-helpers.o +obj-$(CONFIG_DMABUF_HEAPS_SYSTEM) += system_heap.odrivers/dma-buf/heaps/system_heap.c new file +11
-0@@ -0,0 +1,132 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* DMABUF System heap exporter +* +* Copyright (C) 2011 Google, Inc. +* Copyright (C) 2019 Linaro Ltd. +*/ + +#include <asm/page.h> +#include <linux/dma-buf.h> +#include <linux/dma-mapping.h>
drivers/dma-buf/heaps/system_heap.c line 21
Quoted 10 lines
drivers/dma-buf/heaps/system_heap.c new file +10
-0+#include <linux/dma-heap.h> +#include <linux/err.h> +#include <linux/highmem.h> +#include <linux/mm.h> +#include <linux/scatterlist.h> +#include <linux/slab.h> + +#include "heap-helpers.h" + +
remove blank line
drivers/dma-buf/heaps/system_heap.c line 26
Quoted 5 lines
drivers/dma-buf/heaps/system_heap.c new file +5
-0+struct system_heap { +struct dma_heap heap; +}; + +
remove blank line
Quoted 77 lines
drivers/dma-buf/heaps/system_heap.c new file +77
-0+static void system_heap_free (struct heap_helper_buffer *buffer) +{ +int i; +struct scatterlist *sg; +struct sg_table *table = buffer->sg_table; + +for_each_sg (table->sgl, sg, table->nents, i) +__free_page (sg_page (sg)); + +sg_free_table (table); +kfree (table); +kfree (buffer); +} + +static int system_heap_allocate (struct dma_heap *heap, +unsigned long len, +unsigned long flags) +{ +struct heap_helper_buffer *helper_buffer; +struct sg_table *table; +struct scatterlist *sg; +int i, j; +int npages =PAGE_ALIGN (len) /PAGE_SIZE ; +DEFINE_DMA_BUF_EXPORT_INFO (exp_info); +struct dma_buf *dmabuf; +int ret = -ENOMEM ; + + helper_buffer =kzalloc (sizeof (*helper_buffer),GFP_KERNEL ); +if (!helper_buffer) +return -ENOMEM ; + +INIT_HEAP_HELPER_BUFFER (helper_buffer, system_heap_free); + helper_buffer->heap_buffer.flags = flags; + helper_buffer->heap_buffer.heap = heap; + helper_buffer->heap_buffer.size = len; + + table =kmalloc (sizeof (struct sg_table ),GFP_KERNEL ); +if (!table) + goto err0; + + i =sg_alloc_table (table, npages,GFP_KERNEL ); +if (i) + goto err1; +for_each_sg (table->sgl, sg, table->nents, i) { +struct page *page; + + page =alloc_page (GFP_KERNEL ); +if (!page) + goto err2; +sg_set_page (sg, page,PAGE_SIZE ,0 ); + } + +/* create the dmabuf */ + exp_info.ops = &heap_helper_ops; + exp_info.size = len; + exp_info.flags =O_RDWR ; + exp_info.priv = &helper_buffer->heap_buffer; + dmabuf =dma_buf_export (&exp_info); +if (IS_ERR (dmabuf)) { + ret =PTR_ERR (dmabuf); + goto err2; + } + + helper_buffer->heap_buffer.dmabuf = dmabuf; + helper_buffer->sg_table = table; + + ret =dma_buf_fd (dmabuf,O_CLOEXEC ); +if (ret <0 ) { +dma_buf_put (dmabuf); +/* just return, as put will call release and that will free */ +return ret; + } + +return ret; + +err2: +for_each_sg (table->sgl, sg, i, j)
drivers/dma-buf/heaps/system_heap.c line 113
Quoted 10 lines
drivers/dma-buf/heaps/system_heap.c new file +10
-0+__free_page (sg_page (sg)); +sg_free_table (table); +err1: +kfree (table); +err0: +kfree (helper_buffer); +return -ENOMEM ; +} + +
remove blank line
Quoted 22 lines
drivers/dma-buf/heaps/system_heap.c new file +19
-1+static struct dma_heap_ops system_heap_ops = { + .allocate = system_heap_allocate, +}; + +static int system_heap_create (void ) +{ +struct system_heap *sys_heap; + + sys_heap =kzalloc (sizeof (*sys_heap),GFP_KERNEL ); +if (!sys_heap) +return -ENOMEM ; + sys_heap->heap.name ="system_heap" ; + sys_heap->heap.ops = &system_heap_ops; + +dma_heap_add (&sys_heap->heap); + +return 0 ; +} +device_initcall (system_heap_create); --2.7.4
Signature · 7 lines
Benjamin Gaignard Graphic Study Group Linaro.org │ Open source software for ARM SoCs Follow Linaro: Facebook | Twitter | Blog
Re: [RFC][PATCH 3/5 v2] dma-buf: heaps: Add system heap to dmabuf heaps
Quoted 30 lines · Benjamin Gaignard
Quoted 21 lines · John Stultz
This patch adds system heap to the dma-buf heaps framework.
This allows applications to get a page-allocator backed dma-buf for non-contiguous memory.
This code is an evolution of the Android ION implementation, so thanks to its original authors and maintainters:
Rebecca Schultz Zavin, Colin Cross, Laura Abbott, and others!Cc Laura Abbott <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Greg KH <[email protected]>Cc Sumit Semwal <[email protected]>Cc Liam Mark <[email protected]>Cc Brian Starkey <[email protected]>Cc Andrew F. Davis <[email protected]>Cc Chenbo Feng <[email protected]>Cc Alistair Strachan <[email protected]>Signed-off-by John Stultz <[email protected]>Quoted 6 lines
---
v2:
* Switch allocate to return dmabuf fd
* Simplify init code
* Checkpatch fixups
* Droped dead system-contig code
Quoted 3 lines
just few blank lines to remove.
Reveiwed-by Benjamin Gaignard <[email protected]>
Done! Thanks so much for the review!
-john
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [RFC][PATCH 3/5 v2] dma-buf: heaps: Add system heap to dmabuf heaps
In reply to John StultzQuoted 135 lines · John Stultz
This patch adds system heap to the dma-buf heaps framework.
This allows applications to get a page-allocator backed dma-buf for non-contiguous memory.
This code is an evolution of the Android ION implementation, so thanks to its original authors and maintainters:
Rebecca Schultz Zavin, Colin Cross, Laura Abbott, and others!Cc Laura Abbott <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Greg KH <[email protected]>Cc Sumit Semwal <[email protected]>Cc Liam Mark <[email protected]>Cc Brian Starkey <[email protected]>Cc Andrew F. Davis <[email protected]>Cc Chenbo Feng <[email protected]>Cc Alistair Strachan <[email protected]>Signed-off-by John Stultz <[email protected]>---
v2:
* Switch allocate to return dmabuf fd
* Simplify init code
* Checkpatch fixups
* Droped dead system-contig code--- drivers/dma-buf/Kconfig | 2 + drivers/dma-buf/heaps/Kconfig | 6 ++ drivers/dma-buf/heaps/Makefile | 1 + drivers/dma-buf/heaps/system_heap.c | 132 ++++++++++++++++++++++++++++++++++++ 4 files changed, 141 insertions(+) create mode 100644 drivers/dma-buf/heaps/Kconfig create mode 100644 drivers/dma-buf/heaps/system_heap.cdrivers/dma-buf/Kconfig +2
-0@@ -47,4 +47,6 @@ menuconfig DMABUF_HEAPS this allows userspace to allocate dma-bufs that can be shared between drivers. +source "drivers/dma-buf/heaps/Kconfig" + endmenudrivers/dma-buf/heaps/Kconfig new file +6
-0@@ -0,0 +1,6 @@ +config DMABUF_HEAPS_SYSTEM + bool "DMA-BUF System Heap" + depends on DMABUF_HEAPS + help + Choose this option to enable the system dmabuf heap. The system heap + is backed by pages from the buddy allocator. If in doubt, say Y.drivers/dma-buf/heaps/Makefile +1
-0@@ -1,2 +1,3 @@ # SPDX-License-Identifier: GPL-2.0 obj-y += heap-helpers.o +obj-$(CONFIG_DMABUF_HEAPS_SYSTEM) += system_heap.odrivers/dma-buf/heaps/system_heap.c new file +63
-0@@ -0,0 +1,132 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* DMABUF System heap exporter +* +* Copyright (C) 2011 Google, Inc. +* Copyright (C) 2019 Linaro Ltd. +*/ + +#include <asm/page.h> +#include <linux/dma-buf.h> +#include <linux/dma-mapping.h> +#include <linux/dma-heap.h> +#include <linux/err.h> +#include <linux/highmem.h> +#include <linux/mm.h> +#include <linux/scatterlist.h> +#include <linux/slab.h> + +#include "heap-helpers.h" + + +struct system_heap { +struct dma_heap heap; +}; + + +static void system_heap_free (struct heap_helper_buffer *buffer) +{ +int i; +struct scatterlist *sg; +struct sg_table *table = buffer->sg_table; + +for_each_sg (table->sgl, sg, table->nents, i) +__free_page (sg_page (sg)); + +sg_free_table (table); +kfree (table); +kfree (buffer); +} + +static int system_heap_allocate (struct dma_heap *heap, +unsigned long len, +unsigned long flags) +{ +struct heap_helper_buffer *helper_buffer; +struct sg_table *table; +struct scatterlist *sg; +int i, j; +int npages =PAGE_ALIGN (len) /PAGE_SIZE ; +DEFINE_DMA_BUF_EXPORT_INFO (exp_info); +struct dma_buf *dmabuf; +int ret = -ENOMEM ; + + helper_buffer =kzalloc (sizeof (*helper_buffer),GFP_KERNEL ); +if (!helper_buffer) +return -ENOMEM ; + +INIT_HEAP_HELPER_BUFFER (helper_buffer, system_heap_free); + helper_buffer->heap_buffer.flags = flags; + helper_buffer->heap_buffer.heap = heap; + helper_buffer->heap_buffer.size = len; + + table =kmalloc (sizeof (struct sg_table ),GFP_KERNEL );
drivers/dma-buf/heaps/system_heap.c line 73
Quoted 10 lines
drivers/dma-buf/heaps/system_heap.c new file +10
-0+if (!table) + goto err0; + + i =sg_alloc_table (table, npages,GFP_KERNEL ); +if (i) + goto err1; +for_each_sg (table->sgl, sg, table->nents, i) { +struct page *page; + + page =alloc_page (GFP_KERNEL );
Need to zero the allocation (add __GFP_ZERO)
drivers/dma-buf/heaps/system_heap.c line 78
Quoted 5 lines
drivers/dma-buf/heaps/system_heap.c new file +5
-0+if (!page) + goto err2; +sg_set_page (sg, page,PAGE_SIZE ,0 ); + } +
Can always be done later, but it may be helpful to also move this common code from here (and from the cma heap) to the heap helpers file as it reduces code but will also make it easier to introduce future debug features such as making the dma buf names unique
to help make it easier to track down the source of memory leaks.
Quoted 58 lines
drivers/dma-buf/heaps/system_heap.c new file +54
-0+/* create the dmabuf */ + exp_info.ops = &heap_helper_ops; + exp_info.size = len; + exp_info.flags =O_RDWR ; + exp_info.priv = &helper_buffer->heap_buffer; + dmabuf =dma_buf_export (&exp_info); +if (IS_ERR (dmabuf)) { + ret =PTR_ERR (dmabuf); + goto err2; + } + + helper_buffer->heap_buffer.dmabuf = dmabuf; + helper_buffer->sg_table = table; + + ret =dma_buf_fd (dmabuf,O_CLOEXEC ); +if (ret <0 ) { +dma_buf_put (dmabuf); +/* just return, as put will call release and that will free */ +return ret; + } + +return ret; + +err2: +for_each_sg (table->sgl, sg, i, j) +__free_page (sg_page (sg)); +sg_free_table (table); +err1: +kfree (table); +err0: +kfree (helper_buffer); +return -ENOMEM ; +} + + +static struct dma_heap_ops system_heap_ops = { + .allocate = system_heap_allocate, +}; + +static int system_heap_create (void ) +{ +struct system_heap *sys_heap; + + sys_heap =kzalloc (sizeof (*sys_heap),GFP_KERNEL ); +if (!sys_heap) +return -ENOMEM ; + sys_heap->heap.name ="system_heap" ; + sys_heap->heap.ops = &system_heap_ops; + +dma_heap_add (&sys_heap->heap); + +return 0 ; +} +device_initcall (system_heap_create);Signature · 1 line
2.7.4
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
Re: [RFC][PATCH 3/5 v2] dma-buf: heaps: Add system heap to dmabuf heaps
Quoted 5 lines · Liam Mark
Review
Quoted 2 lines · John Stultz
(quoted diff) +2
-0+ + page = alloc_page(GFP_KERNEL);Need to zero the allocation (add
__GFP_ZERO)
Ah! Thanks! Fixed now.
Quoted 11 lines
drivers/dma-buf/heaps/system_heap.c line 78
Quoted 5 lines
drivers/dma-buf/heaps/system_heap.c new file +5
-0+if (!page) + goto err2; +sg_set_page (sg, page,PAGE_SIZE ,0 ); + } +Can always be done later, but it may be helpful to also move this common code from here (and from the cma heap) to the heap helpers file as it reduces code but will also make it easier to introduce future debug features such as making the dma buf names unique
to help make it easier to track down the source of memory leaks.
I think this is a good suggestion, but I do want to be careful to try to make sure we add debugging tools to the larger dmabuf infrastructure, rather then just the heap code (though having some heap specific usage info would still be good). I think there's a separate patchset to dmabufs originally from Greg Hackmann that provides names that is supposed to help with what your suggesting.
https://lists.freedesktop.org/archives/dri-devel/2019-February/208759.html
thanks!
-john
Re: [RFC][PATCH 3/5 v2] dma-buf: heaps: Add system heap to dmabuf heaps
In reply to John Stultzdrivers/dma-buf/heaps/system_heap.c line 77
Quoted 11 lines
drivers/dma-buf/heaps/system_heap.c new file +11
-0+ i =sg_alloc_table (table, npages,GFP_KERNEL ); +if (i) + goto err1; +for_each_sg (table->sgl, sg, table->nents, i) { +struct page *page; + + page =alloc_page (GFP_KERNEL ); +if (!page) + goto err2; +sg_set_page (sg, page,PAGE_SIZE ,0 ); + }
Given that one if not the primary intent here is to DMA map the memory this is a bad idea, as it might require bounce buffering.
What we really want here is something like this:
http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/dma-noncoherent-allocator.3
And I wonder if the S/G building should also move into common code instead of being duplicated everywhere.
drivers/dma-buf/heaps/system_heap.c line 128
Quoted 11 lines
drivers/dma-buf/heaps/system_heap.c new file +11
-0+static int system_heap_create (void ) +{ +struct system_heap *sys_heap; + + sys_heap =kzalloc (sizeof (*sys_heap),GFP_KERNEL ); +if (!sys_heap) +return -ENOMEM ; + sys_heap->heap.name ="system_heap" ; + sys_heap->heap.ops = &system_heap_ops; + +dma_heap_add (&sys_heap->heap);
Why is this dynamically allocated?
RFC PATCH v2 4/5 dma-buf: heaps: Add CMA heap to dmabuf heapss
In reply to John StultzThis adds a CMA heap, which allows userspace to allocate a dma-buf of contiguous memory out of a CMA region.
This code is an evolution of the Android ION implementation, so thanks to its original author and maintainters:
Benjamin Gaignard, Laura Abbott, and others!
---
v2:
* Switch allocate to return dmabuf fd
* Simplify init code
* Checkpatch fixups
--- drivers/dma-buf/heaps/Kconfig | 8 ++ drivers/dma-buf/heaps/Makefile | 1 + drivers/dma-buf/heaps/cma_heap.c | 164 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 173 insertions(+) create mode 100644 drivers/dma-buf/heaps/cma_heap.c
drivers/dma-buf/heaps/Kconfig +8 -0
@@ -4,3 +4,11 @@ config DMABUF_HEAPS_SYSTEM help Choose this option to enable the system dmabuf heap. The system heap is backed by pages from the buddy allocator. If in doubt, say Y. + +config DMABUF_HEAPS_CMA + bool "DMA-BUF CMA Heap" + depends on DMABUF_HEAPS && DMA_CMA + help + Choose this option to enable dma-buf CMA heap. This heap is backed + by the Contiguous Memory Allocator (CMA). If your system has these + regions, you should say Y here.
drivers/dma-buf/heaps/Makefile +1 -0
@@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0 obj-y += heap-helpers.o obj-$(CONFIG_DMABUF_HEAPS_SYSTEM) += system_heap.o +obj-$(CONFIG_DMABUF_HEAPS_CMA) += cma_heap.o
drivers/dma-buf/heaps/cma_heap.c new file +164 -0
164 lines not on this page · raw message
Signature · 1 line
2.7.4
Re: [RFC][PATCH 4/5 v2] dma-buf: heaps: Add CMA heap to dmabuf heapss
Quoted 79 lines · John Stultz
This adds a CMA heap, which allows userspace to allocate a dma-buf of contiguous memory out of a CMA region.
This code is an evolution of the Android ION implementation, so thanks to its original author and maintainters:
Benjamin Gaignard, Laura Abbott, and others!Cc Laura Abbott <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Greg KH <[email protected]>Cc Sumit Semwal <[email protected]>Cc Liam Mark <[email protected]>Cc Brian Starkey <[email protected]>Cc Andrew F. Davis <[email protected]>Cc Chenbo Feng <[email protected]>Cc Alistair Strachan <[email protected]>Signed-off-by John Stultz <[email protected]>---
v2:
* Switch allocate to return dmabuf fd
* Simplify init code
* Checkpatch fixups--- drivers/dma-buf/heaps/Kconfig | 8 ++ drivers/dma-buf/heaps/Makefile | 1 + drivers/dma-buf/heaps/cma_heap.c | 164 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 173 insertions(+) create mode 100644 drivers/dma-buf/heaps/cma_heap.cdrivers/dma-buf/heaps/Kconfig +8
-0@@ -4,3 +4,11 @@ config DMABUF_HEAPS_SYSTEM help Choose this option to enable the system dmabuf heap. The system heap is backed by pages from the buddy allocator. If in doubt, say Y. + +config DMABUF_HEAPS_CMA + bool "DMA-BUF CMA Heap" + depends on DMABUF_HEAPS && DMA_CMA + help + Choose this option to enable dma-buf CMA heap. This heap is backed + by the Contiguous Memory Allocator (CMA). If your system has these + regions, you should say Y here.drivers/dma-buf/heaps/Makefile +1
-0@@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0 obj-y += heap-helpers.o obj-$(CONFIG_DMABUF_HEAPS_SYSTEM) += system_heap.o +obj-$(CONFIG_DMABUF_HEAPS_CMA) += cma_heap.odrivers/dma-buf/heaps/cma_heap.c new file +17
-0@@ -0,0 +1,164 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* DMABUF CMA heap exporter +* +* Copyright (C) 2012, 2019 Linaro Ltd. +* Author: <[email protected]> for ST-Ericsson. +*/ + +#include <linux/device.h> +#include <linux/dma-buf.h> +#include <linux/dma-heap.h> +#include <linux/slab.h> +#include <linux/errno.h> +#include <linux/err.h> +#include <linux/cma.h> +#include <linux/scatterlist.h> +#include <linux/highmem.h>
drivers/dma-buf/heaps/cma_heap.c line 27
Quoted 10 lines
drivers/dma-buf/heaps/cma_heap.c new file +10
-0+ +#include "heap-helpers.h" + +struct cma_heap { +struct dma_heap heap; +struct cma *cma; +}; + + +#define to_cma_heap (x) container_of(x, struct cma_heap, heap)
Even if I had write this macro years ago, now I would prefer to have a static inline function
to be able to check the types.
with that:
Quoted 140 lines
drivers/dma-buf/heaps/cma_heap.c new file +137
-1+ + +static void cma_heap_free (struct heap_helper_buffer *buffer) +{ +struct cma_heap *cma_heap =to_cma_heap (buffer->heap_buffer.heap); +struct page *pages = buffer->priv_virt; +unsigned long nr_pages; + + nr_pages =PAGE_ALIGN (buffer->heap_buffer.size) >>PAGE_SHIFT ; + +/* release memory */ +cma_release (cma_heap->cma, pages, nr_pages); +/* release sg table */ +sg_free_table (buffer->sg_table); +kfree (buffer->sg_table); +kfree (buffer); +} + +/* dmabuf heap CMA operations functions */ +static int cma_heap_allocate (struct dma_heap *heap, +unsigned long len, +unsigned long flags) +{ +struct cma_heap *cma_heap =to_cma_heap (heap); +struct heap_helper_buffer *helper_buffer; +struct sg_table *table; +struct page *pages; +size_t size =PAGE_ALIGN (len); +unsigned long nr_pages = size >>PAGE_SHIFT ; +unsigned long align =get_order (size); +DEFINE_DMA_BUF_EXPORT_INFO (exp_info); +struct dma_buf *dmabuf; +int ret = -ENOMEM ; + +if (align >CONFIG_CMA_ALIGNMENT ) + align =CONFIG_CMA_ALIGNMENT ; + + helper_buffer =kzalloc (sizeof (*helper_buffer),GFP_KERNEL ); +if (!helper_buffer) +return -ENOMEM ; + +INIT_HEAP_HELPER_BUFFER (helper_buffer, cma_heap_free); + helper_buffer->heap_buffer.flags = flags; + helper_buffer->heap_buffer.heap = heap; + helper_buffer->heap_buffer.size = len; + + + pages =cma_alloc (cma_heap->cma, nr_pages, align, false); +if (!pages) + goto free_buf; + +if (PageHighMem (pages)) { +unsigned long nr_clear_pages = nr_pages; +struct page *page = pages; + +while (nr_clear_pages >0 ) { +void *vaddr =kmap_atomic (page); + +memset (vaddr,0 ,PAGE_SIZE ); +kunmap_atomic (vaddr); + page++; + nr_clear_pages--; + } + }else { +memset (page_address (pages),0 , size); + } + + table =kmalloc (sizeof (*table),GFP_KERNEL ); +if (!table) + goto free_cma; + + ret =sg_alloc_table (table,1 ,GFP_KERNEL ); +if (ret) + goto free_table; + +sg_set_page (table->sgl, pages, size,0 ); + +/* create the dmabuf */ + exp_info.ops = &heap_helper_ops; + exp_info.size = len; + exp_info.flags =O_RDWR ; + exp_info.priv = &helper_buffer->heap_buffer; + dmabuf =dma_buf_export (&exp_info); +if (IS_ERR (dmabuf)) { + ret =PTR_ERR (dmabuf); + goto free_table; + } + + helper_buffer->heap_buffer.dmabuf = dmabuf; + helper_buffer->priv_virt = pages; + helper_buffer->sg_table = table; + + ret =dma_buf_fd (dmabuf,O_CLOEXEC ); +if (ret <0 ) { +dma_buf_put (dmabuf); +/* just return, as put will call release and that will free */ +return ret; + } + +return ret; +free_table: +kfree (table); +free_cma: +cma_release (cma_heap->cma, pages, nr_pages); +free_buf: +kfree (helper_buffer); +return ret; +} + +static struct dma_heap_ops cma_heap_ops = { + .allocate = cma_heap_allocate, +}; + +static int __add_cma_heaps (struct cma *cma,void *data) +{ +struct cma_heap *cma_heap; + + cma_heap =kzalloc (sizeof (*cma_heap),GFP_KERNEL ); + +if (!cma_heap) +return -ENOMEM ; + + cma_heap->heap.name =cma_get_name (cma); + cma_heap->heap.ops = &cma_heap_ops; + cma_heap->cma = cma; + +dma_heap_add (&cma_heap->heap); + +return 0 ; +} + +static int add_cma_heaps (void ) +{ +cma_for_each_area (__add_cma_heaps,NULL ); +return 0 ; +} +device_initcall (add_cma_heaps); --2.7.4
Signature · 7 lines
Benjamin Gaignard Graphic Study Group Linaro.org │ Open source software for ARM SoCs Follow Linaro: Facebook | Twitter | Blog
Re: [RFC][PATCH 4/5 v2] dma-buf: heaps: Add CMA heap to dmabuf heapss
Quoted 9 lines · Benjamin Gaignard
Quoted 1 line · John Stultz
+#define
to_cma_heap(x)container_of(x, struct cma_heap, heap)Even if I had write this macro years ago, now I would prefer to have a static inline function
to be able to check the types.with that:
Reviewed-by Benjamin Gaignard <[email protected]>
Thanks for the suggestion! I've reworked that and the other container_of macro I had in the patch series to be inline functions.
thanks again!
-john
Re: [RFC][PATCH 4/5 v2] dma-buf: heaps: Add CMA heap to dmabuf heapss
In reply to John StultzQuoted 2 lines · John Stultz
This adds a CMA heap, which allows userspace to allocate a dma-buf of contiguous memory out of a CMA region.
With my previous suggestion of DMA API usage you'd get CMA support for free in the system one instead of all this duplicate code..
Re: [RFC][PATCH 4/5 v2] dma-buf: heaps: Add CMA heap to dmabuf heapss
Quoted 7 lines · Christoph Hellwig
Quoted 2 lines · John Stultz
This adds a CMA heap, which allows userspace to allocate a dma-buf of contiguous memory out of a CMA region.
With my previous suggestion of DMA API usage you'd get CMA support for free in the system one instead of all this duplicate code..
Hey Christoph! Thanks for the review here! I'm still digesting your comments, so apologies if I misunderstand.
On the point here, unless you're referring to some earlier suggestion on a previous discussion (and not the system heap feedback), part of the reason there are separate heaps is to allow Android to be able to optimize where the allocations are coming from to best match the use case. So they only want to allocate CMA backed dmabufs when the use case has devices that require it, or they may even want to have separate a CMA region reserved for a specific use case (like camera buffers). Similarly for any future heap for allocating secure dma-bufs. So while in the implementation we can consolidate the code more, but we'd still probably want to have separate heaps.
Does that make sense? Am I misinterpreting your feedback?
thanks
-john
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Brian Starkey
reviewerRe: [RFC][PATCH 4/5 v2] dma-buf: heaps: Add CMA heap to dmabuf heapss
In reply to John StultzQuoted 77 lines · John Stultz
This adds a CMA heap, which allows userspace to allocate a dma-buf of contiguous memory out of a CMA region.
This code is an evolution of the Android ION implementation, so thanks to its original author and maintainters:
Benjamin Gaignard, Laura Abbott, and others!Cc Laura Abbott <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Greg KH <[email protected]>Cc Sumit Semwal <[email protected]>Cc Liam Mark <[email protected]>Cc Brian Starkey <[email protected]>Cc Andrew F. Davis <[email protected]>Cc Chenbo Feng <[email protected]>Cc Alistair Strachan <[email protected]>Signed-off-by John Stultz <[email protected]>---
v2:
* Switch allocate to return dmabuf fd
* Simplify init code
* Checkpatch fixups--- drivers/dma-buf/heaps/Kconfig | 8 ++ drivers/dma-buf/heaps/Makefile | 1 + drivers/dma-buf/heaps/cma_heap.c | 164 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 173 insertions(+) create mode 100644 drivers/dma-buf/heaps/cma_heap.cdrivers/dma-buf/heaps/Kconfig +8
-0@@ -4,3 +4,11 @@ config DMABUF_HEAPS_SYSTEM help Choose this option to enable the system dmabuf heap. The system heap is backed by pages from the buddy allocator. If in doubt, say Y. + +config DMABUF_HEAPS_CMA + bool "DMA-BUF CMA Heap" + depends on DMABUF_HEAPS && DMA_CMA + help + Choose this option to enable dma-buf CMA heap. This heap is backed + by the Contiguous Memory Allocator (CMA). If your system has these + regions, you should say Y here.drivers/dma-buf/heaps/Makefile +1
-0@@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0 obj-y += heap-helpers.o obj-$(CONFIG_DMABUF_HEAPS_SYSTEM) += system_heap.o +obj-$(CONFIG_DMABUF_HEAPS_CMA) += cma_heap.odrivers/dma-buf/heaps/cma_heap.c new file +16
-0@@ -0,0 +1,164 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* DMABUF CMA heap exporter +* +* Copyright (C) 2012, 2019 Linaro Ltd. +* Author: <[email protected]> for ST-Ericsson. +*/ + +#include <linux/device.h> +#include <linux/dma-buf.h> +#include <linux/dma-heap.h> +#include <linux/slab.h> +#include <linux/errno.h> +#include <linux/err.h> +#include <linux/cma.h> +#include <linux/scatterlist.h>
drivers/dma-buf/heaps/cma_heap.c line 26
Quoted 10 lines
drivers/dma-buf/heaps/cma_heap.c new file +10
-0+#include <linux/highmem.h> + +#include "heap-helpers.h" + +struct cma_heap { +struct dma_heap heap; +struct cma *cma; +}; + +
extra line
drivers/dma-buf/heaps/cma_heap.c line 29
Quoted 3 lines
drivers/dma-buf/heaps/cma_heap.c new file +3
-0+#define to_cma_heap (x) container_of(x, struct cma_heap, heap) + +
extra line
drivers/dma-buf/heaps/cma_heap.c line 36
Quoted 7 lines
drivers/dma-buf/heaps/cma_heap.c new file +7
-0+static void cma_heap_free (struct heap_helper_buffer *buffer) +{ +struct cma_heap *cma_heap =to_cma_heap (buffer->heap_buffer.heap); +struct page *pages = buffer->priv_virt; +unsigned long nr_pages; + + nr_pages =PAGE_ALIGN (buffer->heap_buffer.size) >>PAGE_SHIFT ;
As you align at alloc time, I don't think the PAGE_ALIGN is really necessary here.
Quoted 28 lines
drivers/dma-buf/heaps/cma_heap.c new file +28
-0+ +/* release memory */ +cma_release (cma_heap->cma, pages, nr_pages); +/* release sg table */ +sg_free_table (buffer->sg_table); +kfree (buffer->sg_table); +kfree (buffer); +} + +/* dmabuf heap CMA operations functions */ +static int cma_heap_allocate (struct dma_heap *heap, +unsigned long len, +unsigned long flags) +{ +struct cma_heap *cma_heap =to_cma_heap (heap); +struct heap_helper_buffer *helper_buffer; +struct sg_table *table; +struct page *pages; +size_t size =PAGE_ALIGN (len); +unsigned long nr_pages = size >>PAGE_SHIFT ; +unsigned long align =get_order (size); +DEFINE_DMA_BUF_EXPORT_INFO (exp_info); +struct dma_buf *dmabuf; +int ret = -ENOMEM ; + +if (align >CONFIG_CMA_ALIGNMENT ) + align =CONFIG_CMA_ALIGNMENT ; +
drivers/dma-buf/heaps/cma_heap.c line 74
Quoted 10 lines
drivers/dma-buf/heaps/cma_heap.c new file +10
-0+ helper_buffer =kzalloc (sizeof (*helper_buffer),GFP_KERNEL ); +if (!helper_buffer) +return -ENOMEM ; + +INIT_HEAP_HELPER_BUFFER (helper_buffer, cma_heap_free); + helper_buffer->heap_buffer.flags = flags; + helper_buffer->heap_buffer.heap = heap; + helper_buffer->heap_buffer.size = len; + +
extra line
Quoted 57 lines
drivers/dma-buf/heaps/cma_heap.c new file +57
-0+ pages =cma_alloc (cma_heap->cma, nr_pages, align, false); +if (!pages) + goto free_buf; + +if (PageHighMem (pages)) { +unsigned long nr_clear_pages = nr_pages; +struct page *page = pages; + +while (nr_clear_pages >0 ) { +void *vaddr =kmap_atomic (page); + +memset (vaddr,0 ,PAGE_SIZE ); +kunmap_atomic (vaddr); + page++; + nr_clear_pages--; + } + }else { +memset (page_address (pages),0 , size); + } + + table =kmalloc (sizeof (*table),GFP_KERNEL ); +if (!table) + goto free_cma; + + ret =sg_alloc_table (table,1 ,GFP_KERNEL ); +if (ret) + goto free_table; + +sg_set_page (table->sgl, pages, size,0 ); + +/* create the dmabuf */ + exp_info.ops = &heap_helper_ops; + exp_info.size = len; + exp_info.flags =O_RDWR ; + exp_info.priv = &helper_buffer->heap_buffer; + dmabuf =dma_buf_export (&exp_info); +if (IS_ERR (dmabuf)) { + ret =PTR_ERR (dmabuf); + goto free_table; + } + + helper_buffer->heap_buffer.dmabuf = dmabuf; + helper_buffer->priv_virt = pages; + helper_buffer->sg_table = table; + + ret =dma_buf_fd (dmabuf,O_CLOEXEC ); +if (ret <0 ) { +dma_buf_put (dmabuf); +/* just return, as put will call release and that will free */ +return ret; + } + +return ret; +free_table: +kfree (table); +free_cma: +cma_release (cma_heap->cma, pages, nr_pages);
drivers/dma-buf/heaps/cma_heap.c line 141
Quoted 10 lines
drivers/dma-buf/heaps/cma_heap.c new file +10
-0+free_buf: +kfree (helper_buffer); +return ret; +} + +static struct dma_heap_ops cma_heap_ops = { + .allocate = cma_heap_allocate, +}; + +static int __add_cma_heaps (struct cma *cma,void *data)
nit: __add_cma_heap (not plural) seems more accurate.
Whatever you decide for the above, you can add my r-b.
Thanks,
-Brian
Quoted 26 lines
drivers/dma-buf/heaps/cma_heap.c new file +23
-0+{ +struct cma_heap *cma_heap; + + cma_heap =kzalloc (sizeof (*cma_heap),GFP_KERNEL ); + +if (!cma_heap) +return -ENOMEM ; + + cma_heap->heap.name =cma_get_name (cma); + cma_heap->heap.ops = &cma_heap_ops; + cma_heap->cma = cma; + +dma_heap_add (&cma_heap->heap); + +return 0 ; +} + +static int add_cma_heaps (void ) +{ +cma_for_each_area (__add_cma_heaps,NULL ); +return 0 ; +} +device_initcall (add_cma_heaps);Signature · 1 line
2.7.4
RFC PATCH v2 5/5 kselftests: Add dma-heap test
In reply to John StultzAdd very trivial allocation test for dma-heaps.
TODO: Need to actually do some validation on
the returned dma-buf.
---
v2: Switched to use reworked dma-heap apis
--- tools/testing/selftests/dmabuf-heaps/Makefile | 11 +++ tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c | 96 ++++++++++++++++++++++ 2 files changed, 107 insertions(+) create mode 100644 tools/testing/selftests/dmabuf-heaps/Makefile create mode 100644 tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c
tools/testing/selftests/dmabuf-heaps/Makefile new file +11 -0
@@ -0,0 +1,11 @@ +# SPDX-License-Identifier: GPL-2.0 +CFLAGS += -static -O3 -Wl,-no-as-needed -Wall +#LDLIBS += -lrt -lpthread -lm + +# these are all "safe" tests that don't modify +# system time or require escalated privileges +TEST_GEN_PROGS = dmabuf-heap + + +include ../lib.mk +
tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c new file +96 -0
96 lines not on this page · raw message
Signature · 6 lines
2.7.4 _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [RFC][PATCH 5/5 v2] kselftests: Add dma-heap test
Quoted 109 lines · John Stultz
Add very trivial allocation test for dma-heaps.
TODO: Need to actually do some validation on
the returned dma-buf.Cc Laura Abbott <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Greg KH <[email protected]>Cc Sumit Semwal <[email protected]>Cc Liam Mark <[email protected]>Cc Brian Starkey <[email protected]>Cc Andrew F. Davis <[email protected]>Cc Chenbo Feng <[email protected]>Cc Alistair Strachan <[email protected]>Signed-off-by John Stultz <[email protected]>---
v2: Switched to use reworked dma-heap apis--- tools/testing/selftests/dmabuf-heaps/Makefile | 11 +++ tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c | 96 ++++++++++++++++++++++ 2 files changed, 107 insertions(+) create mode 100644 tools/testing/selftests/dmabuf-heaps/Makefile create mode 100644 tools/testing/selftests/dmabuf-heaps/dmabuf-heap.ctools/testing/selftests/dmabuf-heaps/Makefile new file +11
-0@@ -0,0 +1,11 @@ +# SPDX-License-Identifier: GPL-2.0 +CFLAGS += -static -O3 -Wl,-no-as-needed -Wall +#LDLIBS += -lrt -lpthread -lm + +# these are all "safe" tests that don't modify +# system time or require escalated privileges +TEST_GEN_PROGS = dmabuf-heap + + +include ../lib.mk +tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c new file +60
-0@@ -0,0 +1,96 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <dirent.h> +#include <errno.h> +#include <fcntl.h> +#include <stdio.h> +#include <string.h> +#include <unistd.h> +#include <sys/ioctl.h> +#include <sys/mman.h> +#include <sys/types.h> + +#include "../../../../include/uapi/linux/dma-heap.h" + +#define DEVPATH "/dev/dma_heap" + +int dmabuf_heap_open (char *name) +{ +int ret, fd; +char buf[256 ]; + + ret =sprintf (buf,"%s/%s" ,DEVPATH , name); +if (ret <0 ) { +printf ("sprintf failed!\n" ); +return ret; + } + + fd =open (buf,O_RDWR ); +if (fd <0 ) +printf ("open %s failed!\n" , buf); +return fd; +} + +int dmabuf_heap_alloc (int fd,size_t len,unsigned int flags,int *dmabuf_fd) +{ +struct dma_heap_allocation_data data = { + .len = len, + .flags = flags, + }; +int ret; + +if (dmabuf_fd ==NULL ) +return -EINVAL ; + + ret =ioctl (fd,DMA_HEAP_IOC_ALLOC , &data); +if (ret <0 ) +return ret; + *dmabuf_fd = (int )data.fd; +return ret; +} + +#define ONE_MEG (1024*1024) + +void do_test (char *heap_name) +{ +int heap_fd =-1 , dmabuf_fd =-1 ; +int ret; + +printf ("Testing heap: %s\n" , heap_name); +
tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c line 70
Quoted 10 lines
tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c new file +10
-0+ heap_fd =dmabuf_heap_open (heap_name); +if (heap_fd <0 ) +return ; + +printf ("Allocating 1 MEG\n" ); + ret =dmabuf_heap_alloc (heap_fd,ONE_MEG ,0 , &dmabuf_fd); +if (ret) + goto out; + +/* DO SOMETHING WITH THE DMABUF HERE? */
You can do a call to mmap and write a pattern in the buffer.
Benjamin
Quoted 29 lines
tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c new file +26
-1+ +out: +if (dmabuf_fd >=0 ) +close (dmabuf_fd); +if (heap_fd >=0 ) +close (heap_fd); +} + + +int main (void ) +{ +DIR *d; +struct dirent *dir; + + d =opendir (DEVPATH ); +if (!d) { +printf ("No %s directory?\n" ,DEVPATH ); +return -1 ; + } + +while ((dir =readdir (d)) !=NULL ) +do_test (dir->d_name); + + +return 0 ; +} --2.7.4
Signature · 7 lines
Benjamin Gaignard Graphic Study Group Linaro.org │ Open source software for ARM SoCs Follow Linaro: Facebook | Twitter | Blog
Re: [RFC][PATCH 5/5 v2] kselftests: Add dma-heap test
Quoted 114 lines · Benjamin Gaignard
Quoted 113 lines · John Stultz
Add very trivial allocation test for dma-heaps.
TODO: Need to actually do some validation on
the returned dma-buf.Cc Laura Abbott <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Greg KH <[email protected]>Cc Sumit Semwal <[email protected]>Cc Liam Mark <[email protected]>Cc Brian Starkey <[email protected]>Cc Andrew F. Davis <[email protected]>Cc Chenbo Feng <[email protected]>Cc Alistair Strachan <[email protected]>Signed-off-by John Stultz <[email protected]>---
v2: Switched to use reworked dma-heap apis--- tools/testing/selftests/dmabuf-heaps/Makefile | 11 +++ tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c | 96 ++++++++++++++++++++++ 2 files changed, 107 insertions(+) create mode 100644 tools/testing/selftests/dmabuf-heaps/Makefile create mode 100644 tools/testing/selftests/dmabuf-heaps/dmabuf-heap.ctools/testing/selftests/dmabuf-heaps/Makefile new file +11
-0@@ -0,0 +1,11 @@ +# SPDX-License-Identifier: GPL-2.0 +CFLAGS += -static -O3 -Wl,-no-as-needed -Wall +#LDLIBS += -lrt -lpthread -lm + +# these are all "safe" tests that don't modify +# system time or require escalated privileges +TEST_GEN_PROGS = dmabuf-heap + + +include ../lib.mk +tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c new file +64
-0@@ -0,0 +1,96 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <dirent.h> +#include <errno.h> +#include <fcntl.h> +#include <stdio.h> +#include <string.h> +#include <unistd.h> +#include <sys/ioctl.h> +#include <sys/mman.h> +#include <sys/types.h> + +#include "../../../../include/uapi/linux/dma-heap.h" + +#define DEVPATH "/dev/dma_heap" + +int dmabuf_heap_open (char *name) +{ +int ret, fd; +char buf[256 ]; + + ret =sprintf (buf,"%s/%s" ,DEVPATH , name); +if (ret <0 ) { +printf ("sprintf failed!\n" ); +return ret; + } + + fd =open (buf,O_RDWR ); +if (fd <0 ) +printf ("open %s failed!\n" , buf); +return fd; +} + +int dmabuf_heap_alloc (int fd,size_t len,unsigned int flags,int *dmabuf_fd) +{ +struct dma_heap_allocation_data data = { + .len = len, + .flags = flags, + }; +int ret; + +if (dmabuf_fd ==NULL ) +return -EINVAL ; + + ret =ioctl (fd,DMA_HEAP_IOC_ALLOC , &data); +if (ret <0 ) +return ret; + *dmabuf_fd = (int )data.fd; +return ret; +} + +#define ONE_MEG (1024*1024) + +void do_test (char *heap_name) +{ +int heap_fd =-1 , dmabuf_fd =-1 ; +int ret; + +printf ("Testing heap: %s\n" , heap_name); + + heap_fd =dmabuf_heap_open (heap_name); +if (heap_fd <0 ) +return ; +
Quoted 9 lines
tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c line 70
Quoted 6 lines
tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c new file +6
-0+printf ("Allocating 1 MEG\n" ); + ret =dmabuf_heap_alloc (heap_fd,ONE_MEG ,0 , &dmabuf_fd); +if (ret) + goto out; + +/* DO SOMETHING WITH THE DMABUF HERE? */ You can do a call to mmap and write a pattern in the buffer.
mmap is optional for DMA-BUFs, only attach/map are required. To test those we would need a dummy device, so a test kernel module may be needed to really exercise this.
I have one I use for ION buffer testing, it consumes a DMA-BUF passed from userspace, attach/maps it to a dummy device then return the physical address of the first page of the buffer for validation. Might be a good test, but dummy devices don't always have the proper dma attributes set like a real device does, so it may also fail for some otherwise valid buffers.
Andrew
Quoted 22 lines
Benjamin
Quoted 21 lines
tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c new file +21
-0+ +out: +if (dmabuf_fd >=0 ) +close (dmabuf_fd); +if (heap_fd >=0 ) +close (heap_fd); +} + + +int main (void ) +{ +DIR *d; +struct dirent *dir; + + d =opendir (DEVPATH ); +if (!d) { +printf ("No %s directory?\n" ,DEVPATH ); +return -1 ; + } + +while ((dir =readdir (d)) !=NULL )
Quoted 10 lines
Quoted 8 lines
tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c new file +5
-1+do_test (dir->d_name); + + +return 0 ; +} --2.7.4
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [RFC][PATCH 5/5 v2] kselftests: Add dma-heap test
Quoted 130 lines · Andrew F. Davis
Quoted 114 lines · Benjamin Gaignard
Quoted 113 lines · John Stultz
Add very trivial allocation test for dma-heaps.
TODO: Need to actually do some validation on
the returned dma-buf.Cc Laura Abbott <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Greg KH <[email protected]>Cc Sumit Semwal <[email protected]>Cc Liam Mark <[email protected]>Cc Brian Starkey <[email protected]>Cc Andrew F. Davis <[email protected]>Cc Chenbo Feng <[email protected]>Cc Alistair Strachan <[email protected]>Signed-off-by John Stultz <[email protected]>---
v2: Switched to use reworked dma-heap apis--- tools/testing/selftests/dmabuf-heaps/Makefile | 11 +++ tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c | 96 ++++++++++++++++++++++ 2 files changed, 107 insertions(+) create mode 100644 tools/testing/selftests/dmabuf-heaps/Makefile create mode 100644 tools/testing/selftests/dmabuf-heaps/dmabuf-heap.ctools/testing/selftests/dmabuf-heaps/Makefile new file +11
-0@@ -0,0 +1,11 @@ +# SPDX-License-Identifier: GPL-2.0 +CFLAGS += -static -O3 -Wl,-no-as-needed -Wall +#LDLIBS += -lrt -lpthread -lm + +# these are all "safe" tests that don't modify +# system time or require escalated privileges +TEST_GEN_PROGS = dmabuf-heap + + +include ../lib.mk +tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c new file +64
-0@@ -0,0 +1,96 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <dirent.h> +#include <errno.h> +#include <fcntl.h> +#include <stdio.h> +#include <string.h> +#include <unistd.h> +#include <sys/ioctl.h> +#include <sys/mman.h> +#include <sys/types.h> + +#include "../../../../include/uapi/linux/dma-heap.h" + +#define DEVPATH "/dev/dma_heap" + +int dmabuf_heap_open (char *name) +{ +int ret, fd; +char buf[256 ]; + + ret =sprintf (buf,"%s/%s" ,DEVPATH , name); +if (ret <0 ) { +printf ("sprintf failed!\n" ); +return ret; + } + + fd =open (buf,O_RDWR ); +if (fd <0 ) +printf ("open %s failed!\n" , buf); +return fd; +} + +int dmabuf_heap_alloc (int fd,size_t len,unsigned int flags,int *dmabuf_fd) +{ +struct dma_heap_allocation_data data = { + .len = len, + .flags = flags, + }; +int ret; + +if (dmabuf_fd ==NULL ) +return -EINVAL ; + + ret =ioctl (fd,DMA_HEAP_IOC_ALLOC , &data); +if (ret <0 ) +return ret; + *dmabuf_fd = (int )data.fd; +return ret; +} + +#define ONE_MEG (1024*1024) + +void do_test (char *heap_name) +{ +int heap_fd =-1 , dmabuf_fd =-1 ; +int ret; + +printf ("Testing heap: %s\n" , heap_name); + + heap_fd =dmabuf_heap_open (heap_name); +if (heap_fd <0 ) +return ; +Quoted 9 lines
tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c line 70
Quoted 6 lines
tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c new file +6
-0+printf ("Allocating 1 MEG\n" ); + ret =dmabuf_heap_alloc (heap_fd,ONE_MEG ,0 , &dmabuf_fd); +if (ret) + goto out; + +/* DO SOMETHING WITH THE DMABUF HERE? */ You can do a call to mmap and write a pattern in the buffer.
mmap is optional for DMA-BUFs, only attach/map are required. To test those we would need a dummy device, so a test kernel module may be needed to really exercise this.
Quoted 6 lines
I have one I use for ION buffer testing, it consumes a DMA-BUF passed from userspace, attach/maps it to a dummy device then return the physical address of the first page of the buffer for validation. Might be a good test, but dummy devices don't always have the proper dma attributes set like a real device does, so it may also fail for some otherwise valid buffers.
Cool! Do you mind sharing that? I might try to rework and integrate it into this patchset?
thanks
-john
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [RFC][PATCH 5/5 v2] kselftests: Add dma-heap test
Quoted 138 lines · John Stultz
Quoted 130 lines · Andrew F. Davis
Quoted 114 lines · Benjamin Gaignard
Quoted 113 lines · John Stultz
Add very trivial allocation test for dma-heaps.
TODO: Need to actually do some validation on
the returned dma-buf.Cc Laura Abbott <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Greg KH <[email protected]>Cc Sumit Semwal <[email protected]>Cc Liam Mark <[email protected]>Cc Brian Starkey <[email protected]>Cc Andrew F. Davis <[email protected]>Cc Chenbo Feng <[email protected]>Cc Alistair Strachan <[email protected]>Signed-off-by John Stultz <[email protected]>---
v2: Switched to use reworked dma-heap apis--- tools/testing/selftests/dmabuf-heaps/Makefile | 11 +++ tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c | 96 ++++++++++++++++++++++ 2 files changed, 107 insertions(+) create mode 100644 tools/testing/selftests/dmabuf-heaps/Makefile create mode 100644 tools/testing/selftests/dmabuf-heaps/dmabuf-heap.ctools/testing/selftests/dmabuf-heaps/Makefile new file +11
-0@@ -0,0 +1,11 @@ +# SPDX-License-Identifier: GPL-2.0 +CFLAGS += -static -O3 -Wl,-no-as-needed -Wall +#LDLIBS += -lrt -lpthread -lm + +# these are all "safe" tests that don't modify +# system time or require escalated privileges +TEST_GEN_PROGS = dmabuf-heap + + +include ../lib.mk +tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c new file +64
-0@@ -0,0 +1,96 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <dirent.h> +#include <errno.h> +#include <fcntl.h> +#include <stdio.h> +#include <string.h> +#include <unistd.h> +#include <sys/ioctl.h> +#include <sys/mman.h> +#include <sys/types.h> + +#include "../../../../include/uapi/linux/dma-heap.h" + +#define DEVPATH "/dev/dma_heap" + +int dmabuf_heap_open (char *name) +{ +int ret, fd; +char buf[256 ]; + + ret =sprintf (buf,"%s/%s" ,DEVPATH , name); +if (ret <0 ) { +printf ("sprintf failed!\n" ); +return ret; + } + + fd =open (buf,O_RDWR ); +if (fd <0 ) +printf ("open %s failed!\n" , buf); +return fd; +} + +int dmabuf_heap_alloc (int fd,size_t len,unsigned int flags,int *dmabuf_fd) +{ +struct dma_heap_allocation_data data = { + .len = len, + .flags = flags, + }; +int ret; + +if (dmabuf_fd ==NULL ) +return -EINVAL ; + + ret =ioctl (fd,DMA_HEAP_IOC_ALLOC , &data); +if (ret <0 ) +return ret; + *dmabuf_fd = (int )data.fd; +return ret; +} + +#define ONE_MEG (1024*1024) + +void do_test (char *heap_name) +{ +int heap_fd =-1 , dmabuf_fd =-1 ; +int ret; + +printf ("Testing heap: %s\n" , heap_name); + + heap_fd =dmabuf_heap_open (heap_name); +if (heap_fd <0 ) +return ; +Quoted 9 lines
tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c line 70
Quoted 6 lines
tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c new file +6
-0+printf ("Allocating 1 MEG\n" ); + ret =dmabuf_heap_alloc (heap_fd,ONE_MEG ,0 , &dmabuf_fd); +if (ret) + goto out; + +/* DO SOMETHING WITH THE DMABUF HERE? */ You can do a call to mmap and write a pattern in the buffer.
mmap is optional for DMA-BUFs, only attach/map are required. To test those we would need a dummy device, so a test kernel module may be needed to really exercise this.
Quoted 6 lines
I have one I use for ION buffer testing, it consumes a DMA-BUF passed from userspace, attach/maps it to a dummy device then return the physical address of the first page of the buffer for validation. Might be a good test, but dummy devices don't always have the proper dma attributes set like a real device does, so it may also fail for some otherwise valid buffers.
Quoted 3 lines
Cool! Do you mind sharing that? I might try to rework and integrate it into this patchset?
Sure, top two patches here:
Andrew
Quoted 3 lines
thanks
-john
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [RFC][PATCH 5/5 v2] kselftests: Add dma-heap test
In reply to Benjamin GaignardQuoted 10 lines · Benjamin Gaignard
tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c line 70
Quoted 7 lines · John Stultz
tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c new file +7
-0+ +printf ("Allocating 1 MEG\n" ); + ret =dmabuf_heap_alloc (heap_fd,ONE_MEG ,0 , &dmabuf_fd); +if (ret) + goto out; + +/* DO SOMETHING WITH THE DMABUF HERE? */ You can do a call to mmap and write a pattern in the buffer.
Yea. I can also do some invalid allocations to make sure things fail properly.
But I was talking a bit w/ Sumit about the lack of any general dmabuf tests, and am curious if we need to have a importer device driver that can validate its a real dmabuf and exercise more of the dmabuf ops.
thanks
-john
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [RFC][PATCH 5/5 v2] kselftests: Add dma-heap test
Quoted 13 lines · John Stultz
Quoted 10 lines · Benjamin Gaignard
tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c line 70
Quoted 7 lines · John Stultz
tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c new file +7
-0+ +printf ("Allocating 1 MEG\n" ); + ret =dmabuf_heap_alloc (heap_fd,ONE_MEG ,0 , &dmabuf_fd); +if (ret) + goto out; + +/* DO SOMETHING WITH THE DMABUF HERE? */ You can do a call to mmap and write a pattern in the buffer.
Quoted 9 lines
Yea. I can also do some invalid allocations to make sure things fail properly.
But I was talking a bit w/ Sumit about the lack of any general dmabuf tests, and am curious if we need to have a importer device driver that can validate its a real dmabuf and exercise more of the dmabuf ops.
thanks
-john
There's the vgem driver in drm. I did some work to clean that up so it could take an import af33a9190d02 ("drm/vgem: Enable dmabuf import interfaces") . I mostly used it for private tests and never ended up upstreaming any of the tests.
Thanks,
Laura
Re: [RFC][PATCH 5/5 v2] kselftests: Add dma-heap test
Quoted 21 lines · Laura Abbott
Quoted 19 lines · John Stultz
Quoted 10 lines · Benjamin Gaignard
tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c line 70
Quoted 7 lines · John Stultz
tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c new file +7
-0+ +printf ("Allocating 1 MEG\n" ); + ret =dmabuf_heap_alloc (heap_fd,ONE_MEG ,0 , &dmabuf_fd); +if (ret) + goto out; + +/* DO SOMETHING WITH THE DMABUF HERE? */ You can do a call to mmap and write a pattern in the buffer.
Yea. I can also do some invalid allocations to make sure things fail properly.
But I was talking a bit w/ Sumit about the lack of any general dmabuf tests, and am curious if we need to have a importer device driver that can validate its a real dmabuf and exercise more of the dmabuf ops.
Quoted 8 lines
Quoted 3 lines
thanks
-johnThere's the vgem driver in drm. I did some work to clean that up so it could take an import
af33a9190d02("drm/vgem: Enable dmabuf import interfaces") . I mostly used it for private tests and never ended up upstreaming any of the tests.
Thanks for the poitner, I'll check that out as well! Also, if you still have them around, I'd be interested in checking out the tests to try to get something integrated into kselftest.
Talking with Brian yesterday, there was some thought that we should try to put together some sort of example dmabuf pipeline that isn't hardware dependent and can be used to demonstrate the usage model as well as validate the frameworks and maybe even benchmark some of the ideas floating around right now. So suggestions here would be welcome!
thanks
-john
Re: [RFC][PATCH 5/5 v2] kselftests: Add dma-heap test
Quoted 35 lines · John Stultz
Quoted 21 lines · Laura Abbott
Quoted 19 lines · John Stultz
Quoted 10 lines · Benjamin Gaignard
tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c line 70
Quoted 7 lines · John Stultz
tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c new file +7
-0+ +printf ("Allocating 1 MEG\n" ); + ret =dmabuf_heap_alloc (heap_fd,ONE_MEG ,0 , &dmabuf_fd); +if (ret) + goto out; + +/* DO SOMETHING WITH THE DMABUF HERE? */ You can do a call to mmap and write a pattern in the buffer.
Yea. I can also do some invalid allocations to make sure things fail properly.
But I was talking a bit w/ Sumit about the lack of any general dmabuf tests, and am curious if we need to have a importer device driver that can validate its a real dmabuf and exercise more of the dmabuf ops.
Quoted 8 lines
Quoted 3 lines
thanks
-johnThere's the vgem driver in drm. I did some work to clean that up so it could take an import
af33a9190d02("drm/vgem: Enable dmabuf import interfaces") . I mostly used it for private tests and never ended up upstreaming any of the tests.Thanks for the poitner, I'll check that out as well! Also, if you still have them around, I'd be interested in checking out the tests to try to get something integrated into kselftest.
Quoted 7 lines
Talking with Brian yesterday, there was some thought that we should try to put together some sort of example dmabuf pipeline that isn't hardware dependent and can be used to demonstrate the usage model as well as validate the frameworks and maybe even benchmark some of the ideas floating around right now. So suggestions here would be welcome!
So the existing ion selftest (tools/testing/selftests/android/ion) does make use of the import to do some very simple tests. I can't seem to find the more complex tests I had though they may have been lost during my last machine move :(
I do think building off of vgem would be a good first step for a testing pipeline, although I worry we wouldn't be able to measure caching effects without a real device since setting up coherency testing otherwise seems error prone.
Thanks,
Laura
Quoted 3 lines
thanks
-john
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [RFC][PATCH 5/5 v2] kselftests: Add dma-heap test
In reply to John StultzQuoted 104 lines · John Stultz
Add very trivial allocation test for dma-heaps.
TODO: Need to actually do some validation on
the returned dma-buf.Cc Laura Abbott <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Greg KH <[email protected]>Cc Sumit Semwal <[email protected]>Cc Liam Mark <[email protected]>Cc Brian Starkey <[email protected]>Cc Andrew F. Davis <[email protected]>Cc Chenbo Feng <[email protected]>Cc Alistair Strachan <[email protected]>Signed-off-by John Stultz <[email protected]>---
v2: Switched to use reworked dma-heap apis--- tools/testing/selftests/dmabuf-heaps/Makefile | 11 +++ tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c | 96 ++++++++++++++++++++++ 2 files changed, 107 insertions(+) create mode 100644 tools/testing/selftests/dmabuf-heaps/Makefile create mode 100644 tools/testing/selftests/dmabuf-heaps/dmabuf-heap.ctools/testing/selftests/dmabuf-heaps/Makefile new file +11
-0@@ -0,0 +1,11 @@ +# SPDX-License-Identifier: GPL-2.0 +CFLAGS += -static -O3 -Wl,-no-as-needed -Wall +#LDLIBS += -lrt -lpthread -lm + +# these are all "safe" tests that don't modify +# system time or require escalated privileges +TEST_GEN_PROGS = dmabuf-heap + + +include ../lib.mk +tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c new file +56
-0@@ -0,0 +1,96 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <dirent.h> +#include <errno.h> +#include <fcntl.h> +#include <stdio.h> +#include <string.h> +#include <unistd.h> +#include <sys/ioctl.h> +#include <sys/mman.h> +#include <sys/types.h> + +#include "../../../../include/uapi/linux/dma-heap.h" + +#define DEVPATH "/dev/dma_heap" + +int dmabuf_heap_open (char *name) +{ +int ret, fd; +char buf[256 ]; + + ret =sprintf (buf,"%s/%s" ,DEVPATH , name); +if (ret <0 ) { +printf ("sprintf failed!\n" ); +return ret; + } + + fd =open (buf,O_RDWR ); +if (fd <0 ) +printf ("open %s failed!\n" , buf); +return fd; +} + +int dmabuf_heap_alloc (int fd,size_t len,unsigned int flags,int *dmabuf_fd) +{ +struct dma_heap_allocation_data data = { + .len = len, + .flags = flags, + }; +int ret; + +if (dmabuf_fd ==NULL ) +return -EINVAL ; + + ret =ioctl (fd,DMA_HEAP_IOC_ALLOC , &data); +if (ret <0 ) +return ret; + *dmabuf_fd = (int )data.fd; +return ret; +} + +#define ONE_MEG (1024*1024) + +void do_test (char *heap_name) +{ +int heap_fd =-1 , dmabuf_fd =-1 ;
tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c line 66
Quoted 10 lines
tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c new file +10
-0+int ret; + +printf ("Testing heap: %s\n" , heap_name); + + heap_fd =dmabuf_heap_open (heap_name); +if (heap_fd <0 ) +return ; + +printf ("Allocating 1 MEG\n" ); + ret =dmabuf_heap_alloc (heap_fd,ONE_MEG ,0 , &dmabuf_fd);
Just be aware that some CMA heaps may already have all their memory allocated by the client, so you may see intermittent failures depending on when you run the test and failrues may
be more common when run on certain platform.
Quoted 34 lines
tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c new file +30
-0+if (ret) + goto out; + +/* DO SOMETHING WITH THE DMABUF HERE? */ + +out: +if (dmabuf_fd >=0 ) +close (dmabuf_fd); +if (heap_fd >=0 ) +close (heap_fd); +} + + +int main (void ) +{ +DIR *d; +struct dirent *dir; + + d =opendir (DEVPATH ); +if (!d) { +printf ("No %s directory?\n" ,DEVPATH ); +return -1 ; + } + +while ((dir =readdir (d)) !=NULL ) +do_test (dir->d_name); + + +return 0 ; +}Signature · 1 line
2.7.4
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
Quoted 33 lines · John Stultz
Here is a initial RFC of the dma-buf heaps patchset Andrew and I have been working on which tries to destage a fair chunk of ION functionality.
The patchset implements per-heap devices which can be opened directly and then an ioctl is used to allocate a dmabuf from the heap.
The interface is similar, but much simpler then IONs, only providing an ALLOC ioctl.
Also, I've provided simple system and cma heaps. The system heap in particular is missing the page-pool optimizations ION had, but works well enough to validate the interface.
I've booted and tested these patches with AOSP on the HiKey960 using the kernel tree here:
https://git.linaro.org/people/john.stultz/android-dev.git/log/?h=dev/dma-buf-heapAnd the userspace changes here:
https://android-review.googlesource.com/c/device/linaro/hikey/+/909436Compared to ION, this patchset is missing the system-contig, carveout and chunk heaps, as I don't have a device that uses those, so I'm unable to do much useful validation there. Additionally we have no upstream users of chunk or carveout, and the system-contig has been deprecated in the common/andoid-* kernels, so this should be ok.
I've also removed the stats accounting for now, since it should be implemented by the heaps themselves.
Quoted 6 lines
Eventual TODOS:
* Reimplement page-pool for system heap (working on this)
* Add stats accounting to system/cma heaps
* Make the kselftest actually useful
* Add other heaps folks see as useful (would love to get some help from actual carveout/chunk users)!
We use a modified carveout heap for certain secure use cases. Although there would probably be some benefit in discssing how the dma-buf heap framework may want to support
secure heaps in the future it is a large topic which I assume you don't want to tackle now.
We don't have any non-secure carveout heap use cases but the client use case I have seen usually revolve around
wanting large allocations to succeed very quickly.
For example I have seen camera use cases which do very large allocations on camera bootup from the carveout heap, these allocations would come from the carveout heap and fallback to the system heap when the carveout heap was full.
Actual non-secure carveout heap can perhaps provide more detail.
Quoted 7 lines
That said, the main user-interface is shaping up and I wanted to get some input on the device model (particularly from GreKH) and any other API/ABI specific input.
thanks
-john
Thanks John and Andrew, it's great to see this effort to get this functionality out of staging.
Since we are making some fundamental changes to how ION worked and since Android is likely also be the largest user of the dma-buf heaps framework I think it would be good
to have a path to resolve the issues which are currently preventing commercial Android releases from moving to the upstream version of ION.
I can understand if you don't necessarily want to put all/any of these changes into the dma-buf heaps framework as part of this series, but my hope is we can get
the upstream community and the Android framework team to agree on what upstreamable changes to dma-buf heaps framework, and/or the Android framework, would be required in order for Android to move to the upstream dma-buf heaps framework for commercial devices.
I don't mean to make this specific to Android, but my assumption is that many of the ION/dma-buf heaps issues which affect Android would likely affect other new large users of the dma-buf heaps framework, so if we resolve it for Android we would be helping these future users as well. And I do understand that some the issues facing Android may need to be resolved by making changes to Android framework.
I think it would be helpful to try and get as much of this agreed upon as possible before the dma-buf heaps framework moves out of staging.
As part of my review I will highlight some of the issues which would affect Android.
In my comments I will apply them to the system heap since that is what Android currently uses for a lot of its use cases.
I realize that this new framework provides more flexibility to heaps, so perhaps some of these issues can be solved by creating a new type of system heap which Android can use, but even if the solution involves creating a new system heap I would like to make sure that this "new" system heap is upstreamable.
Liam
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
Quoted 10 lines · Liam Mark
Quoted 7 lines · John Stultz
Eventual TODOS:
* Reimplement page-pool for system heap (working on this)
* Add stats accounting to system/cma heaps
* Make the kselftest actually useful
* Add other heaps folks see as useful (would love to get some help from actual carveout/chunk users)!We use a modified carveout heap for certain secure use cases.
Cool! It would be great to see if you have any concerns about adding such a secure-carveout heap to this framework. I suspect it would be fairly similar to how its integrated into ION, but particularly I'd be interested in issues around the lack of private flags and other allocation arguments like alignment.
Quoted 4 lines
Although there would probably be some benefit in discssing how the dma-buf heap framework may want to support
secure heaps in the future it is a large topic which I assume you don't want to tackle now.
So I suspect others (Benjamin?) would have a more informed opinion on the details, but the intent is to allow secure heap implementations. I'm not sure what areas of concern you have for this allocation framework in particular?
Quoted 8 lines
We don't have any non-secure carveout heap use cases but the client use case I have seen usually revolve around
wanting large allocations to succeed very quickly.
For example I have seen camera use cases which do very large allocations on camera bootup from the carveout heap, these allocations would come from the carveout heap and fallback to the system heap when the carveout heap was full.
Actual non-secure carveout heap can perhaps provide more detail.
Yea, I'm aware that folks still see carveout as preferable to CMA due to more consistent/predictable allocation latency. I think we still have the issue that we don't have bindings to establish/configure carveout regions w/ dts, and I'm not really wanting to hold up the allocation API on that issue.
Quoted 5 lines
Since we are making some fundamental changes to how ION worked and since Android is likely also be the largest user of the dma-buf heaps framework I think it would be good
to have a path to resolve the issues which are currently preventing commercial Android releases from moving to the upstream version of ION.
Yea, I do see solving the cache management efficiency issues as critical for the dmabuf heaps to be actually usable (my previous version of this patchset accidentally had my hacks to improve performance rolled in!). And there are discussions going on in various channels to try to figure out how to either change Android to use dma-bufs more in line with how upstream expects, or what more generic dma-buf changes we may need to allow Android to use dmabufs with the expected performance they need.
Quoted 7 lines
I can understand if you don't necessarily want to put all/any of these changes into the dma-buf heaps framework as part of this series, but my hope is we can get
the upstream community and the Android framework team to agree on what upstreamable changes to dma-buf heaps framework, and/or the Android framework, would be required in order for Android to move to the upstream dma-buf heaps framework for commercial devices.
Yes. Though I also don't want to get the bigger dma-buf usage discussion (which really affects all dmabuf exporters) too tied up with this patch sets attempt to provide a usable allocation interface. Part of the problem that I think we've seen with ION is that there is a nest of of related issues, and the entire thing is just too big to address at once, which I think is part of why ION has sat in staging for so long. This patchset just tries to provide an dmabuf allocation interface, and a few example exporter heap types.
Quoted 6 lines
I don't mean to make this specific to Android, but my assumption is that many of the ION/dma-buf heaps issues which affect Android would likely affect other new large users of the dma-buf heaps framework, so if we resolve it for Android we would be helping these future users as well. And I do understand that some the issues facing Android may need to be resolved by making changes to Android framework.
While true, I also think some of the assumptions in how the dma-bufs are used (pre-attachment of all devices, etc) are maybe not so realistic given how Android is using them. I do want to explore if Android can change how they use dma-bufs, but I also worry that we need to think about how we could loosen the expectations for dma-bufs, as well as trying to figure out how to support things folks have brought up like partial cache maintenance.
Quoted 12 lines
I think it would be helpful to try and get as much of this agreed upon as possible before the dma-buf heaps framework moves out of staging.
As part of my review I will highlight some of the issues which would affect Android.
In my comments I will apply them to the system heap since that is what Android currently uses for a lot of its use cases.
I realize that this new framework provides more flexibility to heaps, so perhaps some of these issues can be solved by creating a new type of system heap which Android can use, but even if the solution involves creating a new system heap I would like to make sure that this "new" system heap is upstreamable.
So yea, I do realize I'm dodging the hard problem here, but I think the cache-management/usage issue is far more generic.
You're right that this implementation give a lot of flexibility to the exporter heaps in how they implement the dmabuf ops (just like how other device drivers that are dmabuf exporters have the same flexibility), but I very much agree we don't want to add a system and then later a "system-android" heap. So yea, a reasonable amount of caution is warranted here.
Thanks so much for the review and feedback! I'll try to address things as I can as I'm traveling this week (so I may be a bit spotty).
thanks
-john
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 10 lines · John Stultz
Quoted 9 lines · Liam Mark
Quoted 7 lines · John Stultz
Eventual TODOS:
* Reimplement page-pool for system heap (working on this)
* Add stats accounting to system/cma heaps
* Make the kselftest actually useful
* Add other heaps folks see as useful (would love to get some help from actual carveout/chunk users)!
Quoted 8 lines
Quoted 1 line
We use a modified carveout heap for certain secure use cases.
Cool! It would be great to see if you have any concerns about adding such a secure-carveout heap to this framework. I suspect it would be fairly similar to how its integrated into ION, but particularly I'd be interested in issues around the lack of private flags and other allocation arguments like alignment.
We are actively working to drop our secure careveout heap in order to improve memory utilization so I don't think there would be a good case for upstreaming it.
Our other secure heaps are CMA based and system heap based. Because people have had difficulty designing a generic secure heap which would satisfy enough of everybody's use cases to be upstreamable we have been looking into moving away from having local secure heap changes and instead have been looking at how to instead have a separate driver be responsible for making an ION buffer memory secure. The idea was to do this in order to remove a lot of our local ION changes, but if a secure heap was upstreamed that supported our secure use cases I am sure we would be interested in using that.
The local change the ION API to support these heaps is the addition of all the VMID flags so that the client can specify where the client wants the memory assigned.
Quoted 10 lines
Quoted 4 lines
Although there would probably be some benefit in discssing how the dma-buf heap framework may want to support
secure heaps in the future it is a large topic which I assume you don't want to tackle now.So I suspect others (Benjamin?) would have a more informed opinion on the details, but the intent is to allow secure heap implementations. I'm not sure what areas of concern you have for this allocation framework in particular?
I don't have any areas of concern, my thought was just that flushing out a potential design for an upstreamable secure heap would allow us catch early on if there is anything fundamental that would need to change dma-buf heaps framework (such as the allocation API).
I don't think this is a necessary task at this point.
Liam
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
Quoted 24 lines · John Stultz
Quoted 10 lines · Liam Mark
Quoted 7 lines · John Stultz
Eventual TODOS:
* Reimplement page-pool for system heap (working on this)
* Add stats accounting to system/cma heaps
* Make the kselftest actually useful
* Add other heaps folks see as useful (would love to get some help from actual carveout/chunk users)!We use a modified carveout heap for certain secure use cases.
Cool! It would be great to see if you have any concerns about adding such a secure-carveout heap to this framework. I suspect it would be fairly similar to how its integrated into ION, but particularly I'd be interested in issues around the lack of private flags and other allocation arguments like alignment.
Quoted 4 lines
Although there would probably be some benefit in discssing how the dma-buf heap framework may want to support
secure heaps in the future it is a large topic which I assume you don't want to tackle now.
Quoted 4 lines
So I suspect others (Benjamin?) would have a more informed opinion on the details, but the intent is to allow secure heap implementations. I'm not sure what areas of concern you have for this allocation framework in particular?
yes I would be great to understand how you provide the information to tell that a dmabuf
is secure (or not) since we can't add flag in dmabuf structure itself. An option is manage
the access rights when a device attach itself to the dmabuf but in this case you need define
a list of allowed devices per heap...
If you have a good solution for secure heaps you are welcome :-)
Benjamin
Quoted 92 lines
Quoted 9 lines
We don't have any non-secure carveout heap use cases but the client use case I have seen usually revolve around
wanting large allocations to succeed very quickly.
For example I have seen camera use cases which do very large allocations on camera bootup from the carveout heap, these allocations would come from the carveout heap and fallback to the system heap when the carveout heap was full.
Actual non-secure carveout heap can perhaps provide more detail.Yea, I'm aware that folks still see carveout as preferable to CMA due to more consistent/predictable allocation latency. I think we still have the issue that we don't have bindings to establish/configure carveout regions w/ dts, and I'm not really wanting to hold up the allocation API on that issue.
Quoted 5 lines
Since we are making some fundamental changes to how ION worked and since Android is likely also be the largest user of the dma-buf heaps framework I think it would be good
to have a path to resolve the issues which are currently preventing commercial Android releases from moving to the upstream version of ION.Yea, I do see solving the cache management efficiency issues as critical for the dmabuf heaps to be actually usable (my previous version of this patchset accidentally had my hacks to improve performance rolled in!). And there are discussions going on in various channels to try to figure out how to either change Android to use dma-bufs more in line with how upstream expects, or what more generic dma-buf changes we may need to allow Android to use dmabufs with the expected performance they need.
Quoted 7 lines
I can understand if you don't necessarily want to put all/any of these changes into the dma-buf heaps framework as part of this series, but my hope is we can get
the upstream community and the Android framework team to agree on what upstreamable changes to dma-buf heaps framework, and/or the Android framework, would be required in order for Android to move to the upstream dma-buf heaps framework for commercial devices.Yes. Though I also don't want to get the bigger dma-buf usage discussion (which really affects all dmabuf exporters) too tied up with this patch sets attempt to provide a usable allocation interface. Part of the problem that I think we've seen with ION is that there is a nest of of related issues, and the entire thing is just too big to address at once, which I think is part of why ION has sat in staging for so long. This patchset just tries to provide an dmabuf allocation interface, and a few example exporter heap types.
Quoted 6 lines
I don't mean to make this specific to Android, but my assumption is that many of the ION/dma-buf heaps issues which affect Android would likely affect other new large users of the dma-buf heaps framework, so if we resolve it for Android we would be helping these future users as well. And I do understand that some the issues facing Android may need to be resolved by making changes to Android framework.
While true, I also think some of the assumptions in how the dma-bufs are used (pre-attachment of all devices, etc) are maybe not so realistic given how Android is using them. I do want to explore if Android can change how they use dma-bufs, but I also worry that we need to think about how we could loosen the expectations for dma-bufs, as well as trying to figure out how to support things folks have brought up like partial cache maintenance.
Quoted 12 lines
I think it would be helpful to try and get as much of this agreed upon as possible before the dma-buf heaps framework moves out of staging.
As part of my review I will highlight some of the issues which would affect Android.
In my comments I will apply them to the system heap since that is what Android currently uses for a lot of its use cases.
I realize that this new framework provides more flexibility to heaps, so perhaps some of these issues can be solved by creating a new type of system heap which Android can use, but even if the solution involves creating a new system heap I would like to make sure that this "new" system heap is upstreamable.So yea, I do realize I'm dodging the hard problem here, but I think the cache-management/usage issue is far more generic.
You're right that this implementation give a lot of flexibility to the exporter heaps in how they implement the dmabuf ops (just like how other device drivers that are dmabuf exporters have the same flexibility), but I very much agree we don't want to add a system and then later a "system-android" heap. So yea, a reasonable amount of caution is warranted here.
Thanks so much for the review and feedback! I'll try to address things as I can as I'm traveling this week (so I may be a bit spotty).
thanks
-john
Quoted 30 lines · Benjamin Gaignard
Quoted 24 lines · John Stultz
Quoted 10 lines · Liam Mark
Quoted 7 lines · John Stultz
Eventual TODOS:
* Reimplement page-pool for system heap (working on this)
* Add stats accounting to system/cma heaps
* Make the kselftest actually useful
* Add other heaps folks see as useful (would love to get some help from actual carveout/chunk users)!We use a modified carveout heap for certain secure use cases.
Cool! It would be great to see if you have any concerns about adding such a secure-carveout heap to this framework. I suspect it would be fairly similar to how its integrated into ION, but particularly I'd be interested in issues around the lack of private flags and other allocation arguments like alignment.
Quoted 4 lines
Although there would probably be some benefit in discssing how the dma-buf heap framework may want to support
secure heaps in the future it is a large topic which I assume you don't want to tackle now.Quoted 4 lines
So I suspect others (Benjamin?) would have a more informed opinion on the details, but the intent is to allow secure heap implementations. I'm not sure what areas of concern you have for this allocation framework in particular?
Quoted 9 lines
yes I would be great to understand how you provide the information to tell that a dmabuf
is secure (or not) since we can't add flag in dmabuf structure itself. An option is manage
the access rights when a device attach itself to the dmabuf but in this case you need define
a list of allowed devices per heap...
If you have a good solution for secure heaps you are welcome :-)
Do we really need any of that? A secure buffer is secured by the hardware firewalls that keep out certain IP (including often the processor running Linux). So the only thing we need to track internally is that we should not allow mmap/kmap on the buffer. That can be done in the per-heap layer, everything else stays the same as a standard carveout heap.
Andrew
Quoted 93 lines
Benjamin
Quoted 92 lines
Quoted 9 lines
We don't have any non-secure carveout heap use cases but the client use case I have seen usually revolve around
wanting large allocations to succeed very quickly.
For example I have seen camera use cases which do very large allocations on camera bootup from the carveout heap, these allocations would come from the carveout heap and fallback to the system heap when the carveout heap was full.
Actual non-secure carveout heap can perhaps provide more detail.Yea, I'm aware that folks still see carveout as preferable to CMA due to more consistent/predictable allocation latency. I think we still have the issue that we don't have bindings to establish/configure carveout regions w/ dts, and I'm not really wanting to hold up the allocation API on that issue.
Quoted 5 lines
Since we are making some fundamental changes to how ION worked and since Android is likely also be the largest user of the dma-buf heaps framework I think it would be good
to have a path to resolve the issues which are currently preventing commercial Android releases from moving to the upstream version of ION.Yea, I do see solving the cache management efficiency issues as critical for the dmabuf heaps to be actually usable (my previous version of this patchset accidentally had my hacks to improve performance rolled in!). And there are discussions going on in various channels to try to figure out how to either change Android to use dma-bufs more in line with how upstream expects, or what more generic dma-buf changes we may need to allow Android to use dmabufs with the expected performance they need.
Quoted 7 lines
I can understand if you don't necessarily want to put all/any of these changes into the dma-buf heaps framework as part of this series, but my hope is we can get
the upstream community and the Android framework team to agree on what upstreamable changes to dma-buf heaps framework, and/or the Android framework, would be required in order for Android to move to the upstream dma-buf heaps framework for commercial devices.Yes. Though I also don't want to get the bigger dma-buf usage discussion (which really affects all dmabuf exporters) too tied up with this patch sets attempt to provide a usable allocation interface. Part of the problem that I think we've seen with ION is that there is a nest of of related issues, and the entire thing is just too big to address at once, which I think is part of why ION has sat in staging for so long. This patchset just tries to provide an dmabuf allocation interface, and a few example exporter heap types.
Quoted 6 lines
I don't mean to make this specific to Android, but my assumption is that many of the ION/dma-buf heaps issues which affect Android would likely affect other new large users of the dma-buf heaps framework, so if we resolve it for Android we would be helping these future users as well. And I do understand that some the issues facing Android may need to be resolved by making changes to Android framework.
While true, I also think some of the assumptions in how the dma-bufs are used (pre-attachment of all devices, etc) are maybe not so realistic given how Android is using them. I do want to explore if Android can change how they use dma-bufs, but I also worry that we need to think about how we could loosen the expectations for dma-bufs, as well as trying to figure out how to support things folks have brought up like partial cache maintenance.
Quoted 12 lines
I think it would be helpful to try and get as much of this agreed upon as possible before the dma-buf heaps framework moves out of staging.
As part of my review I will highlight some of the issues which would affect Android.
In my comments I will apply them to the system heap since that is what Android currently uses for a lot of its use cases.
I realize that this new framework provides more flexibility to heaps, so perhaps some of these issues can be solved by creating a new type of system heap which Android can use, but even if the solution involves creating a new system heap I would like to make sure that this "new" system heap is upstreamable.So yea, I do realize I'm dodging the hard problem here, but I think the cache-management/usage issue is far more generic.
You're right that this implementation give a lot of flexibility to the exporter heaps in how they implement the dmabuf ops (just like how other device drivers that are dmabuf exporters have the same flexibility), but I very much agree we don't want to add a system and then later a "system-android" heap. So yea, a reasonable amount of caution is warranted here.
Thanks so much for the review and feedback! I'll try to address things as I can as I'm traveling this week (so I may be a bit spotty).
thanks
-john
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 42 lines · Andrew F. Davis
Quoted 30 lines · Benjamin Gaignard
Quoted 24 lines · John Stultz
Quoted 10 lines · Liam Mark
Quoted 7 lines · John Stultz
Eventual TODOS:
* Reimplement page-pool for system heap (working on this)
* Add stats accounting to system/cma heaps
* Make the kselftest actually useful
* Add other heaps folks see as useful (would love to get some help from actual carveout/chunk users)!We use a modified carveout heap for certain secure use cases.
Cool! It would be great to see if you have any concerns about adding such a secure-carveout heap to this framework. I suspect it would be fairly similar to how its integrated into ION, but particularly I'd be interested in issues around the lack of private flags and other allocation arguments like alignment.
Quoted 4 lines
Although there would probably be some benefit in discssing how the dma-buf heap framework may want to support
secure heaps in the future it is a large topic which I assume you don't want to tackle now.Quoted 4 lines
So I suspect others (Benjamin?) would have a more informed opinion on the details, but the intent is to allow secure heap implementations. I'm not sure what areas of concern you have for this allocation framework in particular?
Quoted 9 lines
yes I would be great to understand how you provide the information to tell that a dmabuf
is secure (or not) since we can't add flag in dmabuf structure itself. An option is manage
the access rights when a device attach itself to the dmabuf but in this case you need define
a list of allowed devices per heap...
If you have a good solution for secure heaps you are welcome :-)
Quoted 6 lines
Do we really need any of that? A secure buffer is secured by the hardware firewalls that keep out certain IP (including often the processor running Linux). So the only thing we need to track internally is that we should not allow mmap/kmap on the buffer. That can be done in the per-heap layer, everything else stays the same as a standard carveout heap.
For at least some hw the importing driver needs to configure things differently for secure buffers :-/
BR,
-R
Quoted 94 lines
Andrew
Quoted 91 lines
Benjamin
Quoted 90 lines
Quoted 9 lines
We don't have any non-secure carveout heap use cases but the client use case I have seen usually revolve around
wanting large allocations to succeed very quickly.
For example I have seen camera use cases which do very large allocations on camera bootup from the carveout heap, these allocations would come from the carveout heap and fallback to the system heap when the carveout heap was full.
Actual non-secure carveout heap can perhaps provide more detail.Yea, I'm aware that folks still see carveout as preferable to CMA due to more consistent/predictable allocation latency. I think we still have the issue that we don't have bindings to establish/configure carveout regions w/ dts, and I'm not really wanting to hold up the allocation API on that issue.
Quoted 5 lines
Since we are making some fundamental changes to how ION worked and since Android is likely also be the largest user of the dma-buf heaps framework I think it would be good
to have a path to resolve the issues which are currently preventing commercial Android releases from moving to the upstream version of ION.Yea, I do see solving the cache management efficiency issues as critical for the dmabuf heaps to be actually usable (my previous version of this patchset accidentally had my hacks to improve performance rolled in!). And there are discussions going on in various channels to try to figure out how to either change Android to use dma-bufs more in line with how upstream expects, or what more generic dma-buf changes we may need to allow Android to use dmabufs with the expected performance they need.
Quoted 7 lines
I can understand if you don't necessarily want to put all/any of these changes into the dma-buf heaps framework as part of this series, but my hope is we can get
the upstream community and the Android framework team to agree on what upstreamable changes to dma-buf heaps framework, and/or the Android framework, would be required in order for Android to move to the upstream dma-buf heaps framework for commercial devices.Yes. Though I also don't want to get the bigger dma-buf usage discussion (which really affects all dmabuf exporters) too tied up with this patch sets attempt to provide a usable allocation interface. Part of the problem that I think we've seen with ION is that there is a nest of of related issues, and the entire thing is just too big to address at once, which I think is part of why ION has sat in staging for so long. This patchset just tries to provide an dmabuf allocation interface, and a few example exporter heap types.
Quoted 6 lines
I don't mean to make this specific to Android, but my assumption is that many of the ION/dma-buf heaps issues which affect Android would likely affect other new large users of the dma-buf heaps framework, so if we resolve it for Android we would be helping these future users as well. And I do understand that some the issues facing Android may need to be resolved by making changes to Android framework.
While true, I also think some of the assumptions in how the dma-bufs are used (pre-attachment of all devices, etc) are maybe not so realistic given how Android is using them. I do want to explore if Android can change how they use dma-bufs, but I also worry that we need to think about how we could loosen the expectations for dma-bufs, as well as trying to figure out how to support things folks have brought up like partial cache maintenance.
Quoted 12 lines
I think it would be helpful to try and get as much of this agreed upon as possible before the dma-buf heaps framework moves out of staging.
As part of my review I will highlight some of the issues which would affect Android.
In my comments I will apply them to the system heap since that is what Android currently uses for a lot of its use cases.
I realize that this new framework provides more flexibility to heaps, so perhaps some of these issues can be solved by creating a new type of system heap which Android can use, but even if the solution involves creating a new system heap I would like to make sure that this "new" system heap is upstreamable.So yea, I do realize I'm dodging the hard problem here, but I think the cache-management/usage issue is far more generic.
You're right that this implementation give a lot of flexibility to the exporter heaps in how they implement the dmabuf ops (just like how other device drivers that are dmabuf exporters have the same flexibility), but I very much agree we don't want to add a system and then later a "system-android" heap. So yea, a reasonable amount of caution is warranted here.
Thanks so much for the review and feedback! I'll try to address things as I can as I'm traveling this week (so I may be a bit spotty).
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines
thanks
-john_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 51 lines · Rob Clark
Quoted 42 lines · Andrew F. Davis
Quoted 30 lines · Benjamin Gaignard
Quoted 24 lines · John Stultz
Quoted 10 lines · Liam Mark
Quoted 7 lines · John Stultz
Eventual TODOS:
* Reimplement page-pool for system heap (working on this)
* Add stats accounting to system/cma heaps
* Make the kselftest actually useful
* Add other heaps folks see as useful (would love to get some help from actual carveout/chunk users)!We use a modified carveout heap for certain secure use cases.
Cool! It would be great to see if you have any concerns about adding such a secure-carveout heap to this framework. I suspect it would be fairly similar to how its integrated into ION, but particularly I'd be interested in issues around the lack of private flags and other allocation arguments like alignment.
Quoted 4 lines
Although there would probably be some benefit in discssing how the dma-buf heap framework may want to support
secure heaps in the future it is a large topic which I assume you don't want to tackle now.Quoted 4 lines
So I suspect others (Benjamin?) would have a more informed opinion on the details, but the intent is to allow secure heap implementations. I'm not sure what areas of concern you have for this allocation framework in particular?
Quoted 9 lines
yes I would be great to understand how you provide the information to tell that a dmabuf
is secure (or not) since we can't add flag in dmabuf structure itself. An option is manage
the access rights when a device attach itself to the dmabuf but in this case you need define
a list of allowed devices per heap...
If you have a good solution for secure heaps you are welcome :-)Quoted 6 lines
Do we really need any of that? A secure buffer is secured by the hardware firewalls that keep out certain IP (including often the processor running Linux). So the only thing we need to track internally is that we should not allow mmap/kmap on the buffer. That can be done in the per-heap layer, everything else stays the same as a standard carveout heap.
Quoted 2 lines
For at least some hw the importing driver needs to configure things differently for secure buffers :-/
Does the import ioctl need/use a flag for that then? Userland already has to keep meta-data about dmabufs around.
thanks
-john
Quoted 53 lines · John Stultz
Quoted 51 lines · Rob Clark
Quoted 48 lines · Andrew F. Davis
Quoted 30 lines · Benjamin Gaignard
Quoted 24 lines · John Stultz
Quoted 10 lines · Liam Mark
Quoted 7 lines · John Stultz
Eventual TODOS:
* Reimplement page-pool for system heap (working on this)
* Add stats accounting to system/cma heaps
* Make the kselftest actually useful
* Add other heaps folks see as useful (would love to get some help from actual carveout/chunk users)!We use a modified carveout heap for certain secure use cases.
Cool! It would be great to see if you have any concerns about adding such a secure-carveout heap to this framework. I suspect it would be fairly similar to how its integrated into ION, but particularly I'd be interested in issues around the lack of private flags and other allocation arguments like alignment.
Quoted 4 lines
Although there would probably be some benefit in discssing how the dma-buf heap framework may want to support
secure heaps in the future it is a large topic which I assume you don't want to tackle now.Quoted 4 lines
So I suspect others (Benjamin?) would have a more informed opinion on the details, but the intent is to allow secure heap implementations. I'm not sure what areas of concern you have for this allocation framework in particular?
Quoted 9 lines
yes I would be great to understand how you provide the information to tell that a dmabuf
is secure (or not) since we can't add flag in dmabuf structure itself. An option is manage
the access rights when a device attach itself to the dmabuf but in this case you need define
a list of allowed devices per heap...
If you have a good solution for secure heaps you are welcome :-)Do we really need any of that? A secure buffer is secured by the hardware firewalls that keep out certain IP (including often the processor running Linux). So the only thing we need to track internally is that we should not allow mmap/kmap on the buffer. That can be done in the per-heap layer, everything else stays the same as a standard carveout heap.
Quoted 5 lines
Quoted 2 lines
For at least some hw the importing driver needs to configure things differently for secure buffers :-/
Does the import ioctl need/use a flag for that then? Userland already has to keep meta-data about dmabufs around.
To secure a buffer you need to know who is allowed to write/read it and hardware block involved in the dataflow may need to know that the buffer is secure to configure themself.
As example for a video decoding you allow hw video decoder to read in a buffer and display to read it. You can also allow cpu to write on the buffer to add subtitles. For that we need to be able to mmap/kmap the buffer. Using a carveout heap for secure buffer mean that you reserved a large memory region only for this purpose, that isn't possible on embedded device where we are always limited in memory so we use CMA.
In the past I have used dmabuf's attach function to know who write into the buffer and then configure who will be able to read it. It was working well but the issue was how to in generic way this behavior.
Quoted 3 lines
thanks
-john
Quoted 60 lines · Benjamin Gaignard
Quoted 53 lines · John Stultz
Quoted 51 lines · Rob Clark
Quoted 48 lines · Andrew F. Davis
Quoted 30 lines · Benjamin Gaignard
Quoted 24 lines · John Stultz
Quoted 10 lines · Liam Mark
Quoted 7 lines · John Stultz
Eventual TODOS:
* Reimplement page-pool for system heap (working on this)
* Add stats accounting to system/cma heaps
* Make the kselftest actually useful
* Add other heaps folks see as useful (would love to get some help from actual carveout/chunk users)!We use a modified carveout heap for certain secure use cases.
Cool! It would be great to see if you have any concerns about adding such a secure-carveout heap to this framework. I suspect it would be fairly similar to how its integrated into ION, but particularly I'd be interested in issues around the lack of private flags and other allocation arguments like alignment.
Quoted 4 lines
Although there would probably be some benefit in discssing how the dma-buf heap framework may want to support
secure heaps in the future it is a large topic which I assume you don't want to tackle now.Quoted 4 lines
So I suspect others (Benjamin?) would have a more informed opinion on the details, but the intent is to allow secure heap implementations. I'm not sure what areas of concern you have for this allocation framework in particular?
Quoted 9 lines
yes I would be great to understand how you provide the information to tell that a dmabuf
is secure (or not) since we can't add flag in dmabuf structure itself. An option is manage
the access rights when a device attach itself to the dmabuf but in this case you need define
a list of allowed devices per heap...
If you have a good solution for secure heaps you are welcome :-)Do we really need any of that? A secure buffer is secured by the hardware firewalls that keep out certain IP (including often the processor running Linux). So the only thing we need to track internally is that we should not allow mmap/kmap on the buffer. That can be done in the per-heap layer, everything else stays the same as a standard carveout heap.
Quoted 5 lines
Quoted 2 lines
For at least some hw the importing driver needs to configure things differently for secure buffers :-/
Does the import ioctl need/use a flag for that then? Userland already has to keep meta-data about dmabufs around.
Quoted 13 lines
To secure a buffer you need to know who is allowed to write/read it and hardware block involved in the dataflow may need to know that the buffer is secure to configure themself.
As example for a video decoding you allow hw video decoder to read in a buffer and display to read it. You can also allow cpu to write on the buffer to add subtitles. For that we need to be able to mmap/kmap the buffer. Using a carveout heap for secure buffer mean that you reserved a large memory region only for this purpose, that isn't possible on embedded device where we are always limited in memory so we use CMA.
In the past I have used dmabuf's attach function to know who write into the buffer and then configure who will be able to read it. It was working well but the issue was how to in generic way this behavior.
Okay, I think I see what you are saying now.
The way we handle secure playback is to firewall everything upfront and it is up to the application to inform the hardware about what it can and cannot do to the buffer, or simply not ask anything not allowed (E.g. writeback the decrypted stream) else it will get a firewall exception. The buffer itself doesn't have to carry any information.
It sounds like you want the hardware driver to be able to detect the use-case based on the buffer itself and configure itself accordingly? Or the exporter at attach time to check access permissions?
The first would need a change to DMA-BUF framework, maybe an added flag. The second would just need a heap exporter with the system wide smarts, but as you say that is not very generic..
Andrew
Quoted 3 lines
Quoted 3 lines
thanks
-john
Quoted 84 lines · Andrew F. Davis
Quoted 60 lines · Benjamin Gaignard
Quoted 53 lines · John Stultz
Quoted 51 lines · Rob Clark
Quoted 48 lines · Andrew F. Davis
Quoted 30 lines · Benjamin Gaignard
Quoted 24 lines · John Stultz
Quoted 10 lines · Liam Mark
Quoted 7 lines · John Stultz
Eventual TODOS:
* Reimplement page-pool for system heap (working on this)
* Add stats accounting to system/cma heaps
* Make the kselftest actually useful
* Add other heaps folks see as useful (would love to get some help from actual carveout/chunk users)!We use a modified carveout heap for certain secure use cases.
Cool! It would be great to see if you have any concerns about adding such a secure-carveout heap to this framework. I suspect it would be fairly similar to how its integrated into ION, but particularly I'd be interested in issues around the lack of private flags and other allocation arguments like alignment.
Quoted 4 lines
Although there would probably be some benefit in discssing how the dma-buf heap framework may want to support
secure heaps in the future it is a large topic which I assume you don't want to tackle now.Quoted 4 lines
So I suspect others (Benjamin?) would have a more informed opinion on the details, but the intent is to allow secure heap implementations. I'm not sure what areas of concern you have for this allocation framework in particular?
Quoted 9 lines
yes I would be great to understand how you provide the information to tell that a dmabuf
is secure (or not) since we can't add flag in dmabuf structure itself. An option is manage
the access rights when a device attach itself to the dmabuf but in this case you need define
a list of allowed devices per heap...
If you have a good solution for secure heaps you are welcome :-)Do we really need any of that? A secure buffer is secured by the hardware firewalls that keep out certain IP (including often the processor running Linux). So the only thing we need to track internally is that we should not allow mmap/kmap on the buffer. That can be done in the per-heap layer, everything else stays the same as a standard carveout heap.
Quoted 5 lines
Quoted 2 lines
For at least some hw the importing driver needs to configure things differently for secure buffers :-/
Does the import ioctl need/use a flag for that then? Userland already has to keep meta-data about dmabufs around.
Quoted 13 lines
To secure a buffer you need to know who is allowed to write/read it and hardware block involved in the dataflow may need to know that the buffer is secure to configure themself.
As example for a video decoding you allow hw video decoder to read in a buffer and display to read it. You can also allow cpu to write on the buffer to add subtitles. For that we need to be able to mmap/kmap the buffer. Using a carveout heap for secure buffer mean that you reserved a large memory region only for this purpose, that isn't possible on embedded device where we are always limited in memory so we use CMA.
In the past I have used dmabuf's attach function to know who write into the buffer and then configure who will be able to read it. It was working well but the issue was how to in generic way this behavior.Okay, I think I see what you are saying now.
The way we handle secure playback is to firewall everything upfront and it is up to the application to inform the hardware about what it can and cannot do to the buffer, or simply not ask anything not allowed (E.g. writeback the decrypted stream) else it will get a firewall exception. The buffer itself doesn't have to carry any information.
Quoted 3 lines
It sounds like you want the hardware driver to be able to detect the use-case based on the buffer itself and configure itself accordingly? Or the exporter at attach time to check access permissions?
Both are needed, the buffer client must know that it is a secure buffer and heap will have to configure the permissions.
Quoted 2 lines
The first would need a change to DMA-BUF framework, maybe an added flag.
Sumit will NACK that because dmabuf have to remain neutral and not embedded flags for every possible usage.
Quoted 2 lines
The second would just need a heap exporter with the system wide smarts, but as you say that is not very generic..
yes it is difficult to find a good solution for that.
Quoted 6 lines
Andrew
Quoted 3 lines
Quoted 3 lines
thanks
-john
Quoted 8 lines · Benjamin Gaignard
Quoted 6 lines · John Stultz
Quoted 2 lines · Rob Clark
For at least some hw the importing driver needs to configure things differently for secure buffers :-/
Does the import ioctl need/use a flag for that then? Userland already has to keep meta-data about dmabufs around.
Quoted 12 lines
To secure a buffer you need to know who is allowed to write/read it and hardware block involved in the dataflow may need to know that the buffer is secure to configure themself.
As example for a video decoding you allow hw video decoder to read in a buffer and display to read it. You can also allow cpu to write on the buffer to add subtitles. For that we need to be able to mmap/kmap the buffer. Using a carveout heap for secure buffer mean that you reserved a large memory region only for this purpose, that isn't possible on embedded device where we are always limited in memory so we use CMA.
In the past I have used dmabuf's attach function to know who write into the buffer and then configure who will be able to read it. It was working well but the issue was how to in generic way this behavior.
Given the complexity of the configuration needed when allocating the buffer, instead of trying to make a generic secure buffer allocator, would having per-usege heaps make sense? It just feels there are so many specifics to the secure buffer setup and configuration that maybe there can't be a generic configuration interface. So instead maybe we let the heap implementations provide set usage configs?
This doesn't necessarily require that you have separate pools of memory (they can share the same backing store), but by having multiple per-config heap devices, maybe this could avoid trying to fit all the options into one interface?
On the import side, I'm not sure how much the importing device needs to know about the specific rules here (out side of "secure buffer" or not), so maybe that's another catch.
thanks
-john
Quoted 82 lines · John Stultz
Here is a initial RFC of the dma-buf heaps patchset Andrew and I have been working on which tries to destage a fair chunk of ION functionality.
The patchset implements per-heap devices which can be opened directly and then an ioctl is used to allocate a dmabuf from the heap.
The interface is similar, but much simpler then IONs, only providing an ALLOC ioctl.
Also, I've provided simple system and cma heaps. The system heap in particular is missing the page-pool optimizations ION had, but works well enough to validate the interface.
I've booted and tested these patches with AOSP on the HiKey960 using the kernel tree here:
https://git.linaro.org/people/john.stultz/android-dev.git/log/?h=dev/dma-buf-heapAnd the userspace changes here:
https://android-review.googlesource.com/c/device/linaro/hikey/+/909436Compared to ION, this patchset is missing the system-contig, carveout and chunk heaps, as I don't have a device that uses those, so I'm unable to do much useful validation there. Additionally we have no upstream users of chunk or carveout, and the system-contig has been deprecated in the common/andoid-* kernels, so this should be ok.
I've also removed the stats accounting for now, since it should be implemented by the heaps themselves.
Eventual TODOS:
* Reimplement page-pool for system heap (working on this)
* Add stats accounting to system/cma heaps
* Make the kselftest actually useful
* Add other heaps folks see as useful (would love to get some help from actual carveout/chunk users)!That said, the main user-interface is shaping up and I wanted to get some input on the device model (particularly from GreKH) and any other API/ABI specific input.
thanks
-johnCc Laura Abbott <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Greg KH <[email protected]>Cc Sumit Semwal <[email protected]>Cc Liam Mark <[email protected]>Cc Brian Starkey <[email protected]>Cc Andrew F. Davis <[email protected]>Cc Chenbo Feng <[email protected]>Cc Alistair Strachan <[email protected]>Andrew F. Davis (1):
dma-buf: Add dma-buf heaps frameworkJohn Stultz (4):
dma-buf: heaps: Add heap helpers
dma-buf: heaps: Add system heap to dmabuf heaps
dma-buf: heaps: Add CMA heap to dmabuf heapss
kselftests: Add dma-heap testMAINTAINERS | 16 + drivers/dma-buf/Kconfig | 10 + drivers/dma-buf/Makefile | 2 + drivers/dma-buf/dma-heap.c | 191 ++++++++++++ drivers/dma-buf/heaps/Kconfig | 14 + drivers/dma-buf/heaps/Makefile | 4 + drivers/dma-buf/heaps/cma_heap.c | 164 ++++++++++ drivers/dma-buf/heaps/heap-helpers.c | 335 +++++++++++++++++++++ drivers/dma-buf/heaps/heap-helpers.h | 48 +++ drivers/dma-buf/heaps/system_heap.c | 132 ++++++++ include/linux/dma-heap.h | 65 ++++ include/uapi/linux/dma-heap.h | 52 ++++ tools/testing/selftests/dmabuf-heaps/Makefile | 11 + tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c | 96 ++++++ 14 files changed, 1140 insertions(+)
Quoted 12 lines
create mode 100644
drivers/dma-buf/dma-heap.c
create mode 100644drivers/dma-buf/heaps/Kconfig
create mode 100644drivers/dma-buf/heaps/Makefile
create mode 100644drivers/dma-buf/heaps/cma_heap.c
create mode 100644drivers/dma-buf/heaps/heap-helpers.ccreate mode 100644drivers/dma-buf/heaps/heap-helpers.hcreate mode 100644drivers/dma-buf/heaps/system_heap.ccreate mode 100644include/linux/dma-heap.h
create mode 100644include/uapi/linux/dma-heap.h
create mode 100644tools/testing/selftests/dmabuf-heaps/Makefilecreate mode 100644tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c
This is looking really great. Thanks for doing the work to push this forward. It seems like we're in general agreement about much of this. Which of the issues that have come up do you think are a "hard no" to keeping this from going in?
Thanks,
Laura
Quoted 15 lines · John Stultz
Here is a initial RFC of the dma-buf heaps patchset Andrew and I have been working on which tries to destage a fair chunk of ION functionality.
The patchset implements per-heap devices which can be opened directly and then an ioctl is used to allocate a dmabuf from the heap.
The interface is similar, but much simpler then IONs, only providing an ALLOC ioctl.
Also, I've provided simple system and cma heaps. The system heap in particular is missing the page-pool optimizations ION had, but works well enough to validate the interface.
Quoted 6 lines
I've booted and tested these patches with AOSP on the HiKey960 using the kernel tree here:
https://git.linaro.org/people/john.stultz/android-dev.git/log/?h=dev/dma-buf-heapAnd the userspace changes here:
https://android-review.googlesource.com/c/device/linaro/hikey/+/909436
What upstream driver will use this eventualy ? And why is it needed ?
Cheers,
Jérôme
Quoted 20 lines · Jerome Glisse
Quoted 19 lines · John Stultz
Here is a initial RFC of the dma-buf heaps patchset Andrew and I have been working on which tries to destage a fair chunk of ION functionality.
The patchset implements per-heap devices which can be opened directly and then an ioctl is used to allocate a dmabuf from the heap.
The interface is similar, but much simpler then IONs, only providing an ALLOC ioctl.
Also, I've provided simple system and cma heaps. The system heap in particular is missing the page-pool optimizations ION had, but works well enough to validate the interface.
I've booted and tested these patches with AOSP on the HiKey960 using the kernel tree here:
https://git.linaro.org/people/john.stultz/android-dev.git/log/?h=dev/dma-buf-heap
Quoted 5 lines
Quoted 2 lines
And the userspace changes here:
https://android-review.googlesource.com/c/device/linaro/hikey/+/909436What upstream driver will use this eventualy ? And why is it needed ?
So, its sort of a complicated answer, as we don't have a fully open pipeline just yet. The HiKey board's upstream kirin drm driver uses this, as it needs contiguous buffers for its framebuffers. So in Android the HiKey gralloc (opensource userspace) allocates the HW_FB buffers from the CMA heap. Other graphics buffers are then allocated by gralloc out of the system heap and SurfaceFlinger and drm_hwc (both also opensource userspace) coordinate squashing those other buffers down through the mali utgard gpu (proprietary GL blob) onto the target HW_FB buffer. (All of the above is the same for the HiKey960, but we're still working the drm driver into shape for upstreaming).
That said, I know the Lima driver is starting to shape up, and I'm hoping to give it a whirl to replace the proprietary utgard driver. Everything else would stay the same, which would give us a fully open pipeline.
I know for other dev boards like the db410c w/ freedreno, the Android pipeline gets away with using the gbmgralloc implementation, but my understanding in that case is the rendering/display pipeline doesn't require contiguous buffers, so the allocation logic can be simpler, and doesn't use ION heaps. But its possible a gralloc could be implemented to use the system heap for allocations on that device.
thanks
-john
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel