Skip to contentSkip to conversation

Direct userspace dma-buf mmap (v6)

PATCH Superseded Tiago Vignatti 5 participants 25 messages Dec 16, 2015 – Dec 22, 2015

Subsystems (9)

v6 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

Hi all,

The last version of this work was sent a while ago here:

http://lists.freedesktop.org/archives/dri-devel/2015-August/089263.html

So let's recap this series:

1. it adds a vendor-independent client interface for mapping gem objects through prime, IOW it implements userspace mmap() on dma-buf fd. This could be used for texturing from CPU rendered buffer, passing buffers among processes without performing copies in the userspace.
2. the series lets the client write on the mmap'ed memory, and
3. it deals with GPU and CPU caches synchronization.

Based on previous discussions seems that people are fine with 1. and 2. but not really with 3., given that caches coherency is a bit more boring to deal with.

It's easier to use this new infra on "coherent hardware" (systems with the memory cache that is shared by the GPU and CPU) because they rarely need to use that kind of synchronization. But would be much more convenient to have the very same interface exposed for clients no matter whether the underlying hardware is cache coherent or not.

One idea that came up was to force clients to call the sync ioctls after the dma-buf was mmaped. But apparently there's no easy, and performant, way to do so cause seems too costly to go over the page table entry and check the dirty bits. Also, depending on the instructions order sent for the devices, it might be needed a sync call after the mapped region gets accessed as well, to flush all cachelines and make sure for example the GPU domain won't read stale data. So that would make the things even more complicated, if we ever decide to go to this direction of forcing sync ioctls. The alternative therefore is to simply document it very well, strong wording the clients to use the sync ioctl regardless otherwise they will mis-behave. Do we have objections or maybe other wiser ways to circumvent this? I've made similar comments in August and no one has came up with better ideas.

Lastly, the diff of v6 series is that I've basically addressed concerns pointed in the igt tests, organized those changes better a bit (in smaller patches), documented the usage of sync ioctls and I have extensively tested this in different types of hardware.

https://github.com/tiagovignatti/drm-intel/commits/drm-intel-nightly_dma-buf-mmap-v6
https://github.com/tiagovignatti/intel-gpu-tools/commits/dma-buf-mmap-v6

Tiago

Daniel Thompson (1):
drm: prime: Honour O_RDWR during prime-handle-to-fd

Daniel Vetter (1):
dma-buf: Add ioctls to allow userspace to flush

Tiago Vignatti (3):
dma-buf: Remove range-based flush
drm/i915: Implement end_cpu_access
drm/i915: Use CPU mapping for userspace dma-buf mmap()

 Documentation/dma-buf-sharing.txt         | 41 +++++++++++++++-------
 drivers/dma-buf/dma-buf.c                 | 56 ++++++++++++++++++++++++++-----
 drivers/gpu/drm/drm_prime.c               | 10 ++----
 drivers/gpu/drm/i915/i915_gem_dmabuf.c    | 42 +++++++++++++++++++++--
 drivers/gpu/drm/omapdrm/omap_gem_dmabuf.c |  4 +--
 drivers/gpu/drm/udl/udl_fb.c              |  2 --
 drivers/staging/android/ion/ion.c         |  6 ++--
 drivers/staging/android/ion/ion_test.c    |  4 +--
 include/linux/dma-buf.h                   | 12 +++----
 include/uapi/drm/drm.h                    |  1 +
 include/uapi/linux/dma-buf.h              | 38 +++++++++++++++++++++
 11 files changed, 169 insertions(+), 47 deletions(-)
 create mode 100644 include/uapi/linux/dma-buf.h

And the igt changes:
Rob Bradford (1):
prime_mmap: Add new test for calling mmap() on dma-buf fds

Tiago Vignatti (5):
lib: Add gem_userptr and __gem_userptr helpers
prime_mmap: Add basic tests to write in a bo using CPU
lib: Add prime_sync_start and prime_sync_end helpers
tests: Add kms_mmap_write_crc for cache coherency tests
tests: Add prime_mmap_coherency for cache coherency tests

 benchmarks/gem_userptr_benchmark.c |  55 +----
 lib/ioctl_wrappers.c               |  92 +++++++
 lib/ioctl_wrappers.h               |  32 +++
 tests/Makefile.sources             |   3 +
 tests/gem_userptr_blits.c          | 104 ++------
 tests/kms_mmap_write_crc.c         | 281 +++++++++++++++++++++
 tests/prime_mmap.c                 | 494 +++++++++++++++++++++++++++++++++++++
 tests/prime_mmap_coherency.c       | 246 ++++++++++++++++++
 8 files changed, 1180 insertions(+), 127 deletions(-)
 create mode 100644 tests/kms_mmap_write_crc.c
 create mode 100644 tests/prime_mmap.c
 create mode 100644 tests/prime_mmap_coherency.c
Signature · 6 lines
2.1.4

_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel
#1 lore raw

PATCH v6 1/5 drm: prime: Honour O_RDWR during prime-handle-to-fd

From: Daniel Thompson <[email protected]>

Currently DRM_IOCTL_PRIME_HANDLE_TO_FD rejects all flags except (DRM|O)_CLOEXEC making it difficult (maybe impossible) for userspace to mmap() the resulting dma-buf even when this is supported by the DRM driver.

It is trivial to relax the restriction and permit read/write access. This is safe because the flags are seldom touched by drm; mostly they are passed verbatim to dma_buf calls.

v3 (Tiago): removed unused flags variable from drm_prime_handle_to_fd_ioctl.

---
 drivers/gpu/drm/drm_prime.c | 10 +++-------
 include/uapi/drm/drm.h      |  1 +
 2 files changed, 4 insertions(+), 7 deletions(-)
drivers/gpu/drm/drm_prime.c +3 -7
index 27aa718..df6cdc7 100644
@@ -329,7 +329,7 @@ static const struct dma_buf_ops drm_gem_prime_dmabuf_ops =  {
  * drm_gem_prime_export - helper library implementation of the export callback
  * @dev: drm_device to export from
  * @obj: GEM object to export
- * @flags: flags like DRM_CLOEXEC
+ * @flags: flags like DRM_CLOEXEC and DRM_RDWR
  *
  * This is the implementation of the gem_prime_export functions for GEM drivers
  * using the PRIME helpers.
@@ -628,7 +628,6 @@ int drm_prime_handle_to_fd_ioctl(struct drm_device *dev, void *data,
 				 struct drm_file *file_priv)
 {
 	struct drm_prime_handle *args = data;
-	uint32_t flags;
 
 	if (!drm_core_check_feature(dev, DRIVER_PRIME))
 		return -EINVAL;
@@ -637,14 +636,11 @@ int drm_prime_handle_to_fd_ioctl(struct drm_device *dev, void *data,
 		return -ENOSYS;
 
 	/* check flags are valid */
-	if (args->flags & ~DRM_CLOEXEC)
+	if (args->flags & ~(DRM_CLOEXEC | DRM_RDWR))
 		return -EINVAL;
 
-	/* we only want to pass DRM_CLOEXEC which is == O_CLOEXEC */
-	flags = args->flags & DRM_CLOEXEC;
-
 	return dev->driver->prime_handle_to_fd(dev, file_priv,
-			args->handle, flags, &args->fd);
+			args->handle, args->flags, &args->fd);
 }
 
 int drm_prime_fd_to_handle_ioctl(struct drm_device *dev, void *data,
include/uapi/drm/drm.h +1 -0
index b4e92eb..a0ebfe7 100644
@@ -669,6 +669,7 @@ struct drm_set_client_cap {
 	__u64 value;
 };
 
+#define DRM_RDWR O_RDWR
 #define DRM_CLOEXEC O_CLOEXEC
 struct drm_prime_handle {
 	__u32 handle;
Signature · 6 lines
2.1.4

_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel
#2 lore raw

PATCH v6 2/5 dma-buf: Remove range-based flush

In reply to Tiago Vignatti

This patch removes range-based information used for optimizations in begin_cpu_access and end_cpu_access.

We don't have any user nor implementation using range-based flush. It seems a consensus that if we ever want something like that again (or even more robust using 2D, 3D sub-range regions) we can use the upcoming dma-buf sync ioctl for such.

Cc Sumit Semwal <[email protected]>
Cc Daniel Vetter <[email protected]>
---
 Documentation/dma-buf-sharing.txt         | 19 ++++++++-----------
 drivers/dma-buf/dma-buf.c                 | 13 ++++---------
 drivers/gpu/drm/i915/i915_gem_dmabuf.c    |  2 +-
 drivers/gpu/drm/omapdrm/omap_gem_dmabuf.c |  4 ++--
 drivers/gpu/drm/udl/udl_fb.c              |  2 --
 drivers/staging/android/ion/ion.c         |  6 ++----
 drivers/staging/android/ion/ion_test.c    |  4 ++--
 include/linux/dma-buf.h                   | 12 +++++-------
 8 files changed, 24 insertions(+), 38 deletions(-)
Documentation/dma-buf-sharing.txt +8 -11
index 480c8de..4f4a84b 100644
@@ -257,17 +257,15 @@ Access to a dma_buf from the kernel context involves three steps:
 
    Interface:
       int dma_buf_begin_cpu_access(struct dma_buf *dmabuf,
-				   size_t start, size_t len,
 				   enum dma_data_direction direction)
 
    This allows the exporter to ensure that the memory is actually available for
    cpu access - the exporter might need to allocate or swap-in and pin the
    backing storage. The exporter also needs to ensure that cpu access is
-   coherent for the given range and access direction. The range and access
-   direction can be used by the exporter to optimize the cache flushing, i.e.
-   access outside of the range or with a different direction (read instead of
-   write) might return stale or even bogus data (e.g. when the exporter needs to
-   copy the data to temporary storage).
+   coherent for the access direction. The direction can be used by the exporter
+   to optimize the cache flushing, i.e. access with a different direction (read
+   instead of write) might return stale or even bogus data (e.g. when the
+   exporter needs to copy the data to temporary storage).
 
    This step might fail, e.g. in oom conditions.
 
@@ -322,14 +320,13 @@ Access to a dma_buf from the kernel context involves three steps:
 
 3. Finish access
 
-   When the importer is done accessing the range specified in begin_cpu_access,
-   it needs to announce this to the exporter (to facilitate cache flushing and
-   unpinning of any pinned resources). The result of any dma_buf kmap calls
-   after end_cpu_access is undefined.
+   When the importer is done accessing the CPU, it needs to announce this to
+   the exporter (to facilitate cache flushing and unpinning of any pinned
+   resources). The result of any dma_buf kmap calls after end_cpu_access is
+   undefined.
 
    Interface:
       void dma_buf_end_cpu_access(struct dma_buf *dma_buf,
-				  size_t start, size_t len,
 				  enum dma_data_direction dir);
 
 
drivers/dma-buf/dma-buf.c +4 -9
index 155c146..b2ac13b 100644
@@ -539,13 +539,11 @@ EXPORT_SYMBOL_GPL(dma_buf_unmap_attachment);
  * preparations. Coherency is only guaranteed in the specified range for the
  * specified access direction.
  * @dmabuf:	[in]	buffer to prepare cpu access for.
- * @start:	[in]	start of range for cpu access.
- * @len:	[in]	length of range for cpu access.
  * @direction:	[in]	length of range for cpu access.
  *
  * Can return negative error values, returns 0 on success.
  */
-int dma_buf_begin_cpu_access(struct dma_buf *dmabuf, size_t start, size_t len,
+int dma_buf_begin_cpu_access(struct dma_buf *dmabuf,
 			     enum dma_data_direction direction)
 {
 	int ret = 0;
@@ -554,8 +552,7 @@ int dma_buf_begin_cpu_access(struct dma_buf *dmabuf, size_t start, size_t len,
 		return -EINVAL;
 
 	if (dmabuf->ops->begin_cpu_access)
-		ret = dmabuf->ops->begin_cpu_access(dmabuf, start,
-							len, direction);
+		ret = dmabuf->ops->begin_cpu_access(dmabuf, direction);
 
 	return ret;
 }
@@ -567,19 +564,17 @@ EXPORT_SYMBOL_GPL(dma_buf_begin_cpu_access);
  * actions. Coherency is only guaranteed in the specified range for the
  * specified access direction.
  * @dmabuf:	[in]	buffer to complete cpu access for.
- * @start:	[in]	start of range for cpu access.
- * @len:	[in]	length of range for cpu access.
  * @direction:	[in]	length of range for cpu access.
  *
  * This call must always succeed.
  */
-void dma_buf_end_cpu_access(struct dma_buf *dmabuf, size_t start, size_t len,
+void dma_buf_end_cpu_access(struct dma_buf *dmabuf,
 			    enum dma_data_direction direction)
 {
 	WARN_ON(!dmabuf);
 
 	if (dmabuf->ops->end_cpu_access)
-		dmabuf->ops->end_cpu_access(dmabuf, start, len, direction);
+		dmabuf->ops->end_cpu_access(dmabuf, direction);
 }
 EXPORT_SYMBOL_GPL(dma_buf_end_cpu_access);
 
drivers/gpu/drm/i915/i915_gem_dmabuf.c +1 -1
index e9c2bfd..65ab2bd 100644
@@ -196,7 +196,7 @@ static int i915_gem_dmabuf_mmap(struct dma_buf *dma_buf, struct vm_area_struct *
 	return -EINVAL;
 }
 
-static int i915_gem_begin_cpu_access(struct dma_buf *dma_buf, size_t start, size_t length, enum dma_data_direction direction)
+static int i915_gem_begin_cpu_access(struct dma_buf *dma_buf, enum dma_data_direction direction)
 {
 	struct drm_i915_gem_object *obj = dma_buf_to_obj(dma_buf);
 	struct drm_device *dev = obj->base.dev;
drivers/gpu/drm/omapdrm/omap_gem_dmabuf.c +2 -2
index 27c2976..aebae1c 100644
@@ -79,7 +79,7 @@ static void omap_gem_dmabuf_release(struct dma_buf *buffer)
 
 
 static int omap_gem_dmabuf_begin_cpu_access(struct dma_buf *buffer,
-		size_t start, size_t len, enum dma_data_direction dir)
+		enum dma_data_direction dir)
 {
 	struct drm_gem_object *obj = buffer->priv;
 	struct page **pages;
@@ -94,7 +94,7 @@ static int omap_gem_dmabuf_begin_cpu_access(struct dma_buf *buffer,
 }
 
 static void omap_gem_dmabuf_end_cpu_access(struct dma_buf *buffer,
-		size_t start, size_t len, enum dma_data_direction dir)
+		enum dma_data_direction dir)
 {
 	struct drm_gem_object *obj = buffer->priv;
 	omap_gem_put_pages(obj);
drivers/gpu/drm/udl/udl_fb.c +0 -2
index 200419d..c427499 100644
@@ -409,7 +409,6 @@ static int udl_user_framebuffer_dirty(struct drm_framebuffer *fb,
 
 	if (ufb->obj->base.import_attach) {
 		ret = dma_buf_begin_cpu_access(ufb->obj->base.import_attach->dmabuf,
-					       0, ufb->obj->base.size,
 					       DMA_FROM_DEVICE);
 		if (ret)
 			goto unlock;
@@ -425,7 +424,6 @@ static int udl_user_framebuffer_dirty(struct drm_framebuffer *fb,
 
 	if (ufb->obj->base.import_attach) {
 		dma_buf_end_cpu_access(ufb->obj->base.import_attach->dmabuf,
-				       0, ufb->obj->base.size,
 				       DMA_FROM_DEVICE);
 	}
 
drivers/staging/android/ion/ion.c +2 -4
index e237e9f..0754a37 100644
@@ -1057,8 +1057,7 @@ static void ion_dma_buf_kunmap(struct dma_buf *dmabuf, unsigned long offset,
 {
 }
 
-static int ion_dma_buf_begin_cpu_access(struct dma_buf *dmabuf, size_t start,
-					size_t len,
+static int ion_dma_buf_begin_cpu_access(struct dma_buf *dmabuf,
 					enum dma_data_direction direction)
 {
 	struct ion_buffer *buffer = dmabuf->priv;
@@ -1076,8 +1075,7 @@ static int ion_dma_buf_begin_cpu_access(struct dma_buf *dmabuf, size_t start,
 	return PTR_ERR_OR_ZERO(vaddr);
 }
 
-static void ion_dma_buf_end_cpu_access(struct dma_buf *dmabuf, size_t start,
-				       size_t len,
+static void ion_dma_buf_end_cpu_access(struct dma_buf *dmabuf,
 				       enum dma_data_direction direction)
 {
 	struct ion_buffer *buffer = dmabuf->priv;
drivers/staging/android/ion/ion_test.c +2 -2
index b8dcf5a..da34bc12 100644
@@ -109,7 +109,7 @@ static int ion_handle_test_kernel(struct dma_buf *dma_buf, void __user *ptr,
 	if (offset > dma_buf->size || size > dma_buf->size - offset)
 		return -EINVAL;
 
-	ret = dma_buf_begin_cpu_access(dma_buf, offset, size, dir);
+	ret = dma_buf_begin_cpu_access(dma_buf, dir);
 	if (ret)
 		return ret;
 
@@ -139,7 +139,7 @@ static int ion_handle_test_kernel(struct dma_buf *dma_buf, void __user *ptr,
 		copy_offset = 0;
 	}
 err:
-	dma_buf_end_cpu_access(dma_buf, offset, size, dir);
+	dma_buf_end_cpu_access(dma_buf, dir);
 	return ret;
 }
 
include/linux/dma-buf.h +5 -7
index f98bd70..532108e 100644
@@ -54,7 +54,7 @@ struct dma_buf_attachment;
  * @release: release this buffer; to be called after the last dma_buf_put.
  * @begin_cpu_access: [optional] called before cpu access to invalidate cpu
  * 		      caches and allocate backing storage (if not yet done)
- * 		      respectively pin the objet into memory.
+ * 		      respectively pin the object into memory.
  * @end_cpu_access: [optional] called after cpu access to flush caches.
  * @kmap_atomic: maps a page from the buffer into kernel address
  * 		 space, users may not block until the subsequent unmap call.
@@ -93,10 +93,8 @@ struct dma_buf_ops {
 	/* after final dma_buf_put() */
 	void (*release)(struct dma_buf *);
 
-	int (*begin_cpu_access)(struct dma_buf *, size_t, size_t,
-				enum dma_data_direction);
-	void (*end_cpu_access)(struct dma_buf *, size_t, size_t,
-			       enum dma_data_direction);
+	int (*begin_cpu_access)(struct dma_buf *, enum dma_data_direction);
+	void (*end_cpu_access)(struct dma_buf *, enum dma_data_direction);
 	void *(*kmap_atomic)(struct dma_buf *, unsigned long);
 	void (*kunmap_atomic)(struct dma_buf *, unsigned long, void *);
 	void *(*kmap)(struct dma_buf *, unsigned long);
@@ -224,9 +222,9 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *,
 					enum dma_data_direction);
 void dma_buf_unmap_attachment(struct dma_buf_attachment *, struct sg_table *,
 				enum dma_data_direction);
-int dma_buf_begin_cpu_access(struct dma_buf *dma_buf, size_t start, size_t len,
+int dma_buf_begin_cpu_access(struct dma_buf *dma_buf,
 			     enum dma_data_direction dir);
-void dma_buf_end_cpu_access(struct dma_buf *dma_buf, size_t start, size_t len,
+void dma_buf_end_cpu_access(struct dma_buf *dma_buf,
 			    enum dma_data_direction dir);
 void *dma_buf_kmap_atomic(struct dma_buf *, unsigned long);
 void dma_buf_kunmap_atomic(struct dma_buf *, unsigned long, void *);
Signature · 6 lines
2.1.4

_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel
#3 lore raw

PATCH v6 3/5 dma-buf: Add ioctls to allow userspace to flush

In reply to Tiago Vignatti

From: Daniel Vetter <[email protected]>

The userspace might need some sort of cache coherency management e.g. when CPU and GPU domains are being accessed through dma-buf at the same time. To circumvent this problem there are begin/end coherency markers, that forward directly to existing dma-buf device drivers vfunc hooks. Userspace can make use of those markers through the DMA_BUF_IOCTL_SYNC ioctl. The sequence would be used like following:
- mmap dma-buf fd
- for each drawing/upload cycle in CPU 1. SYNC_START ioctl, 2. read/write to mmap area 3. SYNC_END ioctl. This can be repeated as often as you want (with the new data being consumed by the GPU or say scanout device)
- munmap once you don't need the buffer any more

v2 (Tiago): Fix header file type names (u64 -> __u64)
v3 (Tiago): Add documentation. Use enum dma_buf_sync_flags to the begin/end dma-buf functions. Check for overflows in start/length.
v4 (Tiago): use 2d regions for sync.
v5 (Tiago): forget about 2d regions (v4); use _IOW in DMA_BUF_IOCTL_SYNC and remove range information from struct dma_buf_sync.
v6 (Tiago): use __u64 structured padded flags instead enum. Adjust documentation about the recommendation on using sync ioctls.

---
 Documentation/dma-buf-sharing.txt | 22 +++++++++++++++++++-
 drivers/dma-buf/dma-buf.c         | 43 +++++++++++++++++++++++++++++++++++++++
 include/uapi/linux/dma-buf.h      | 38 ++++++++++++++++++++++++++++++++++
 3 files changed, 102 insertions(+), 1 deletion(-)
 create mode 100644 include/uapi/linux/dma-buf.h
Documentation/dma-buf-sharing.txt +21 -1
index 4f4a84b..2ddd4b2 100644
@@ -350,7 +350,27 @@ Being able to mmap an export dma-buf buffer object has 2 main use-cases:
    handles, too). So it's beneficial to support this in a similar fashion on
    dma-buf to have a good transition path for existing Android userspace.
 
-   No special interfaces, userspace simply calls mmap on the dma-buf fd.
+   No special interfaces, userspace simply calls mmap on the dma-buf fd. Very
+   important to note though is that, even if it is not mandatory, the userspace
+   is strongly recommended to always use the cache synchronization ioctl
+   (DMA_BUF_IOCTL_SYNC) discussed next.
+
+   Some systems might need some sort of cache coherency management e.g. when
+   CPU and GPU domains are being accessed through dma-buf at the same time. To
+   circumvent this problem there are begin/end coherency markers, that forward
+   directly to existing dma-buf device drivers vfunc hooks. Userspace can make
+   use of those markers through the DMA_BUF_IOCTL_SYNC ioctl. The sequence
+   would be used like following:
+     - mmap dma-buf fd
+     - for each drawing/upload cycle in CPU 1. SYNC_START ioctl, 2. read/write
+       to mmap area 3. SYNC_END ioctl. This can be repeated as often as you
+       want (with the new data being consumed by the GPU or say scanout device)
+     - munmap once you don't need the buffer any more
+
+    In principle systems with the memory cache shared by the GPU and CPU may
+    not need SYNC_START and SYNC_END but still, userspace is always encouraged
+    to use these ioctls before and after, respectively, when accessing the
+    mapped address.
 
 2. Supporting existing mmap interfaces in importers
 
drivers/dma-buf/dma-buf.c +43 -0

60 lines not on this page · raw message

include/uapi/linux/dma-buf.h new file +38 -0
new file mode 100644
index 0000000..bd195f2
@@ -0,0 +1,38 @@
+/*
+ * Framework for buffer objects that can be shared across devices/subsystems.
+ *
+ * Copyright(C) 2015 Intel Ltd
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef _DMA_BUF_UAPI_H_
+#define _DMA_BUF_UAPI_H_
+
+/* begin/end dma-buf functions used for userspace mmap. */
+struct dma_buf_sync {
+	__u64 flags;
+};
+
+#define DMA_BUF_SYNC_READ      (1 << 0)
+#define DMA_BUF_SYNC_WRITE     (2 << 0)
+#define DMA_BUF_SYNC_RW        (3 << 0)
+#define DMA_BUF_SYNC_START     (0 << 2)
+#define DMA_BUF_SYNC_END       (1 << 2)
+#define DMA_BUF_SYNC_VALID_FLAGS_MASK \
+	(DMA_BUF_SYNC_RW | DMA_BUF_SYNC_END)
+
+#define DMA_BUF_BASE		'b'
+#define DMA_BUF_IOCTL_SYNC	_IOW(DMA_BUF_BASE, 0, struct dma_buf_sync)
+
+#endif
Signature · 6 lines
2.1.4

_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel
#4 lore raw

Re: [PATCH v6 3/5] dma-buf: Add ioctls to allow userspace to flush

Quoted 157 lines · Tiago Vignatti

From: Daniel Vetter <[email protected]>

The userspace might need some sort of cache coherency management e.g. when CPU and GPU domains are being accessed through dma-buf at the same time. To circumvent this problem there are begin/end coherency markers, that forward directly to existing dma-buf device drivers vfunc hooks. Userspace can make use of those markers through the DMA_BUF_IOCTL_SYNC ioctl. The sequence would be used like following:
- mmap dma-buf fd
- for each drawing/upload cycle in CPU 1. SYNC_START ioctl, 2. read/write to mmap area 3. SYNC_END ioctl. This can be repeated as often as you want (with the new data being consumed by the GPU or say scanout device)
- munmap once you don't need the buffer any more

v2 (Tiago): Fix header file type names (u64 -> __u64)
v3 (Tiago): Add documentation. Use enum dma_buf_sync_flags to the begin/end dma-buf functions. Check for overflows in start/length.
v4 (Tiago): use 2d regions for sync.
v5 (Tiago): forget about 2d regions (v4); use _IOW in DMA_BUF_IOCTL_SYNC and remove range information from struct dma_buf_sync.
v6 (Tiago): use __u64 structured padded flags instead enum. Adjust documentation about the recommendation on using sync ioctls.

---
 Documentation/dma-buf-sharing.txt | 22 +++++++++++++++++++-
 drivers/dma-buf/dma-buf.c         | 43 +++++++++++++++++++++++++++++++++++++++
 include/uapi/linux/dma-buf.h      | 38 ++++++++++++++++++++++++++++++++++
 3 files changed, 102 insertions(+), 1 deletion(-)
 create mode 100644 include/uapi/linux/dma-buf.h
Documentation/dma-buf-sharing.txt +21 -1
index 4f4a84b..2ddd4b2 100644
@@ -350,7 +350,27 @@ Being able to mmap an export dma-buf buffer object has 2 main use-cases:
    handles, too). So it's beneficial to support this in a similar fashion on
    dma-buf to have a good transition path for existing Android userspace.
 
-   No special interfaces, userspace simply calls mmap on the dma-buf fd.
+   No special interfaces, userspace simply calls mmap on the dma-buf fd. Very
+   important to note though is that, even if it is not mandatory, the userspace
+   is strongly recommended to always use the cache synchronization ioctl
+   (DMA_BUF_IOCTL_SYNC) discussed next.
+
+   Some systems might need some sort of cache coherency management e.g. when
+   CPU and GPU domains are being accessed through dma-buf at the same time. To
+   circumvent this problem there are begin/end coherency markers, that forward
+   directly to existing dma-buf device drivers vfunc hooks. Userspace can make
+   use of those markers through the DMA_BUF_IOCTL_SYNC ioctl. The sequence
+   would be used like following:
+     - mmap dma-buf fd
+     - for each drawing/upload cycle in CPU 1. SYNC_START ioctl, 2. read/write
+       to mmap area 3. SYNC_END ioctl. This can be repeated as often as you
+       want (with the new data being consumed by the GPU or say scanout device)
+     - munmap once you don't need the buffer any more
+
+    In principle systems with the memory cache shared by the GPU and CPU may
+    not need SYNC_START and SYNC_END but still, userspace is always encouraged
+    to use these ioctls before and after, respectively, when accessing the
+    mapped address.
 
 2. Supporting existing mmap interfaces in importers
 
drivers/dma-buf/dma-buf.c +43 -0
index b2ac13b..9a298bd 100644
@@ -34,6 +34,8 @@
 #include <linux/poll.h>
 #include <linux/reservation.h>
 
+#include <uapi/linux/dma-buf.h>
+
 static inline int is_dma_buf_file(struct file *);
 
 struct dma_buf_list {
@@ -251,11 +253,52 @@ out:
        return events;
 }
 
+static long dma_buf_ioctl(struct file *file,
+                         unsigned int cmd, unsigned long arg)
+{
+       struct dma_buf *dmabuf;
+       struct dma_buf_sync sync;
+       enum dma_data_direction direction;
+
+       dmabuf = file->private_data;
+
+       if (!is_dma_buf_file(file))
+               return -EINVAL;
+
+       switch (cmd) {
+       case DMA_BUF_IOCTL_SYNC:
+               if (copy_from_user(&sync, (void __user *) arg, sizeof(sync)))
+                       return -EFAULT;
+
+               if (sync.flags & DMA_BUF_SYNC_RW)
+                       direction = DMA_BIDIRECTIONAL;
+               else if (sync.flags & DMA_BUF_SYNC_READ)
+                       direction = DMA_FROM_DEVICE;
+               else if (sync.flags & DMA_BUF_SYNC_WRITE)
+                       direction = DMA_TO_DEVICE;
+               else
+                       return -EINVAL;
+
+               if (sync.flags & ~DMA_BUF_SYNC_VALID_FLAGS_MASK)
+                       return -EINVAL;
+
+               if (sync.flags & DMA_BUF_SYNC_END)
+                       dma_buf_end_cpu_access(dmabuf, direction);
+               else
+                       dma_buf_begin_cpu_access(dmabuf, direction);
+
+               return 0;
+       default:
+               return -ENOTTY;
+       }
+}
+
 static const struct file_operations dma_buf_fops = {
        .release        = dma_buf_release,
        .mmap           = dma_buf_mmap_internal,
        .llseek         = dma_buf_llseek,
        .poll           = dma_buf_poll,
+       .unlocked_ioctl = dma_buf_ioctl,
 };
 
 /*
include/uapi/linux/dma-buf.h new file +19 -0
new file mode 100644
index 0000000..bd195f2
@@ -0,0 +1,38 @@
+/*
+ * Framework for buffer objects that can be shared across devices/subsystems.
+ *
+ * Copyright(C) 2015 Intel Ltd
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef _DMA_BUF_UAPI_H_

include/uapi/linux/dma-buf.h line 29

Quoted 10 lines
include/uapi/linux/dma-buf.h new file +10 -0
+#define _DMA_BUF_UAPI_H_
+
+/* begin/end dma-buf functions used for userspace mmap. */
+struct dma_buf_sync {
+       __u64 flags;
+};
+
+#define DMA_BUF_SYNC_READ      (1 << 0)
+#define DMA_BUF_SYNC_WRITE     (2 << 0)
+#define DMA_BUF_SYNC_RW        (3 << 0)

Maybe make this:

#define DMA_BUF_SYNC_RW (DMA_BUF_SYNC_READ | DMA_BUF_SYNC_WRITE)

or maybe:

#define DMA_BUF_SYNC_RW_MASK (3 << 0)

include/uapi/linux/dma-buf.h line 31

Quoted 2 lines
include/uapi/linux/dma-buf.h new file +2 -0
+#define DMA_BUF_SYNC_START     (0 << 2)
+#define DMA_BUF_SYNC_END       (1 << 2)

if you go with the mask above, maybe:
#define DMA_BUF_SYNC_MASK (1 << 2)

Quoted 10 lines
include/uapi/linux/dma-buf.h new file +7 -1
+#define DMA_BUF_SYNC_VALID_FLAGS_MASK \
+       (DMA_BUF_SYNC_RW | DMA_BUF_SYNC_END)
+
+#define DMA_BUF_BASE           'b'
+#define DMA_BUF_IOCTL_SYNC     _IOW(DMA_BUF_BASE, 0, struct dma_buf_sync)
+
+#endif
--

2.1.4

Quoted 4 lines

_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel

_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel

#5 lore raw

Re: [PATCH v6 3/5] dma-buf: Add ioctls to allow userspace to flush

Quoted 172 lines · Alex Deucher
Quoted 157 lines · Tiago Vignatti

From: Daniel Vetter <[email protected]>

The userspace might need some sort of cache coherency management e.g. when CPU and GPU domains are being accessed through dma-buf at the same time. To circumvent this problem there are begin/end coherency markers, that forward directly to existing dma-buf device drivers vfunc hooks. Userspace can make use of those markers through the DMA_BUF_IOCTL_SYNC ioctl. The sequence would be used like following:
- mmap dma-buf fd
- for each drawing/upload cycle in CPU 1. SYNC_START ioctl, 2. read/write to mmap area 3. SYNC_END ioctl. This can be repeated as often as you want (with the new data being consumed by the GPU or say scanout device)
- munmap once you don't need the buffer any more

v2 (Tiago): Fix header file type names (u64 -> __u64)
v3 (Tiago): Add documentation. Use enum dma_buf_sync_flags to the begin/end dma-buf functions. Check for overflows in start/length.
v4 (Tiago): use 2d regions for sync.
v5 (Tiago): forget about 2d regions (v4); use _IOW in DMA_BUF_IOCTL_SYNC and remove range information from struct dma_buf_sync.
v6 (Tiago): use __u64 structured padded flags instead enum. Adjust documentation about the recommendation on using sync ioctls.

---
  Documentation/dma-buf-sharing.txt | 22 +++++++++++++++++++-
  drivers/dma-buf/dma-buf.c         | 43 +++++++++++++++++++++++++++++++++++++++
  include/uapi/linux/dma-buf.h      | 38 ++++++++++++++++++++++++++++++++++
  3 files changed, 102 insertions(+), 1 deletion(-)

create mode 100644 include/uapi/linux/dma-buf.h

Documentation/dma-buf-sharing.txt +21 -1
index 4f4a84b..2ddd4b2 100644
@@ -350,7 +350,27 @@ Being able to mmap an export dma-buf buffer object has 2 main use-cases:
     handles, too). So it's beneficial to support this in a similar fashion on
     dma-buf to have a good transition path for existing Android userspace.
 
-   No special interfaces, userspace simply calls mmap on the dma-buf fd.
+   No special interfaces, userspace simply calls mmap on the dma-buf fd. Very
+   important to note though is that, even if it is not mandatory, the userspace
+   is strongly recommended to always use the cache synchronization ioctl
+   (DMA_BUF_IOCTL_SYNC) discussed next.
+
+   Some systems might need some sort of cache coherency management e.g. when
+   CPU and GPU domains are being accessed through dma-buf at the same time. To
+   circumvent this problem there are begin/end coherency markers, that forward
+   directly to existing dma-buf device drivers vfunc hooks. Userspace can make
+   use of those markers through the DMA_BUF_IOCTL_SYNC ioctl. The sequence
+   would be used like following:
+     - mmap dma-buf fd
+     - for each drawing/upload cycle in CPU 1. SYNC_START ioctl, 2. read/write
+       to mmap area 3. SYNC_END ioctl. This can be repeated as often as you
+       want (with the new data being consumed by the GPU or say scanout device)
+     - munmap once you don't need the buffer any more
+
+    In principle systems with the memory cache shared by the GPU and CPU may
+    not need SYNC_START and SYNC_END but still, userspace is always encouraged
+    to use these ioctls before and after, respectively, when accessing the
+    mapped address.
 
  2. Supporting existing mmap interfaces in importers
 
drivers/dma-buf/dma-buf.c +43 -0
index b2ac13b..9a298bd 100644
@@ -34,6 +34,8 @@
  #include <linux/poll.h>
  #include <linux/reservation.h>
 
+#include <uapi/linux/dma-buf.h>
+
  static inline int is_dma_buf_file(struct file *);
 
  struct dma_buf_list {
@@ -251,11 +253,52 @@ out:
         return events;
  }
 
+static long dma_buf_ioctl(struct file *file,
+                         unsigned int cmd, unsigned long arg)
+{
+       struct dma_buf *dmabuf;
+       struct dma_buf_sync sync;
+       enum dma_data_direction direction;
+
+       dmabuf = file->private_data;
+
+       if (!is_dma_buf_file(file))
+               return -EINVAL;
+
+       switch (cmd) {
+       case DMA_BUF_IOCTL_SYNC:
+               if (copy_from_user(&sync, (void __user *) arg, sizeof(sync)))
+                       return -EFAULT;
+
+               if (sync.flags & DMA_BUF_SYNC_RW)
+                       direction = DMA_BIDIRECTIONAL;
+               else if (sync.flags & DMA_BUF_SYNC_READ)
+                       direction = DMA_FROM_DEVICE;
+               else if (sync.flags & DMA_BUF_SYNC_WRITE)
+                       direction = DMA_TO_DEVICE;
+               else
+                       return -EINVAL;
+
+               if (sync.flags & ~DMA_BUF_SYNC_VALID_FLAGS_MASK)
+                       return -EINVAL;
+
+               if (sync.flags & DMA_BUF_SYNC_END)
+                       dma_buf_end_cpu_access(dmabuf, direction);
+               else
+                       dma_buf_begin_cpu_access(dmabuf, direction);
+
+               return 0;
+       default:
+               return -ENOTTY;
+       }
+}
+
  static const struct file_operations dma_buf_fops = {
         .release        = dma_buf_release,
         .mmap           = dma_buf_mmap_internal,
         .llseek         = dma_buf_llseek,
         .poll           = dma_buf_poll,
+       .unlocked_ioctl = dma_buf_ioctl,
  };
 
  /*
include/uapi/linux/dma-buf.h new file +19 -0
new file mode 100644
index 0000000..bd195f2
@@ -0,0 +1,38 @@
+/*
+ * Framework for buffer objects that can be shared across devices/subsystems.
+ *
+ * Copyright(C) 2015 Intel Ltd
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef _DMA_BUF_UAPI_H_

include/uapi/linux/dma-buf.h line 29

Quoted 10 lines
include/uapi/linux/dma-buf.h new file +10 -0
+#define _DMA_BUF_UAPI_H_
+
+/* begin/end dma-buf functions used for userspace mmap. */
+struct dma_buf_sync {
+       __u64 flags;
+};
+
+#define DMA_BUF_SYNC_READ      (1 << 0)
+#define DMA_BUF_SYNC_WRITE     (2 << 0)
+#define DMA_BUF_SYNC_RW        (3 << 0)

Maybe make this:

Quoted 5 lines

#define DMA_BUF_SYNC_RW (DMA_BUF_SYNC_READ | DMA_BUF_SYNC_WRITE)

or maybe:

#define DMA_BUF_SYNC_RW_MASK (3 << 0)

Thanks Alex. I think I like your first option for being a bit more suggestive than the other -- I'm changing for it now.

Tiago

_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel

#6 lore raw

Re: [PATCH v6 3/5] dma-buf: Add ioctls to allow userspace to flush

In reply to Tiago Vignatti
Quoted 54 lines · Tiago Vignatti

From: Daniel Vetter <[email protected]>

The userspace might need some sort of cache coherency management e.g. when CPU and GPU domains are being accessed through dma-buf at the same time. To circumvent this problem there are begin/end coherency markers, that forward directly to existing dma-buf device drivers vfunc hooks. Userspace can make use of those markers through the DMA_BUF_IOCTL_SYNC ioctl. The sequence would be used like following:
- mmap dma-buf fd
- for each drawing/upload cycle in CPU 1. SYNC_START ioctl, 2. read/write to mmap area 3. SYNC_END ioctl. This can be repeated as often as you want (with the new data being consumed by the GPU or say scanout device)
- munmap once you don't need the buffer any more

v2 (Tiago): Fix header file type names (u64 -> __u64)
v3 (Tiago): Add documentation. Use enum dma_buf_sync_flags to the begin/end dma-buf functions. Check for overflows in start/length.
v4 (Tiago): use 2d regions for sync.
v5 (Tiago): forget about 2d regions (v4); use _IOW in DMA_BUF_IOCTL_SYNC and remove range information from struct dma_buf_sync.
v6 (Tiago): use __u64 structured padded flags instead enum. Adjust documentation about the recommendation on using sync ioctls.

---
 Documentation/dma-buf-sharing.txt | 22 +++++++++++++++++++-
 drivers/dma-buf/dma-buf.c         | 43 +++++++++++++++++++++++++++++++++++++++
 include/uapi/linux/dma-buf.h      | 38 ++++++++++++++++++++++++++++++++++
 3 files changed, 102 insertions(+), 1 deletion(-)
 create mode 100644 include/uapi/linux/dma-buf.h
Documentation/dma-buf-sharing.txt +12 -1
index 4f4a84b..2ddd4b2 100644
@@ -350,7 +350,27 @@ Being able to mmap an export dma-buf buffer object has 2 main use-cases:
    handles, too). So it's beneficial to support this in a similar fashion on
    dma-buf to have a good transition path for existing Android userspace.
 
-   No special interfaces, userspace simply calls mmap on the dma-buf fd.
+   No special interfaces, userspace simply calls mmap on the dma-buf fd. Very
+   important to note though is that, even if it is not mandatory, the userspace
+   is strongly recommended to always use the cache synchronization ioctl
+   (DMA_BUF_IOCTL_SYNC) discussed next.
+
+   Some systems might need some sort of cache coherency management e.g. when
+   CPU and GPU domains are being accessed through dma-buf at the same time. To
+   circumvent this problem there are begin/end coherency markers, that forward
+   directly to existing dma-buf device drivers vfunc hooks. Userspace can make
+   use of those markers through the DMA_BUF_IOCTL_SYNC ioctl. The sequence
+   would be used like following:
+     - mmap dma-buf fd

Documentation/dma-buf-sharing.txt Being able to mmap an export dma-buf buffer object has 2 main use-cases: · line 374

Quoted 10 lines
Documentation/dma-buf-sharing.txt +9 -0
+     - for each drawing/upload cycle in CPU 1. SYNC_START ioctl, 2. read/write
+       to mmap area 3. SYNC_END ioctl. This can be repeated as often as you
+       want (with the new data being consumed by the GPU or say scanout device)
+     - munmap once you don't need the buffer any more
+
+    In principle systems with the memory cache shared by the GPU and CPU may
+    not need SYNC_START and SYNC_END but still, userspace is always encouraged
+    to use these ioctls before and after, respectively, when accessing the
+    mapped address.
 

I think the wording here is far too weak. If this is a generic user-space interface and syncing
is required for
a) Correctness: then syncing must be mandatory.
b) Optimal performance then an implementation must generate expected results also in the absence of SYNC ioctls, but is allowed to rely on correct pairing of SYNC_START and SYNC_END to render correctly.

Also recalling the discussion of multidimensional sync, which we said we would add when it was needed, my worst nightmare is when (not if) there are a number of important applications starting to abuse this interface for small writes or reads to large DMA buffers. It will work flawlessly on coherent architectures, and probably some incoherent architectures as well, but will probably be mostly useless on VMware. What is the plan for adding 2D sync to make it work? How do we pursuade app developers to think of damage regions, and can we count on support from the DRM community when this happens?

/Thomas

Quoted 102 lines
Documentation/dma-buf-sharing.txt +0 -0
 2. Supporting existing mmap interfaces in importers
 
drivers/dma-buf/dma-buf.c +43 -0
index b2ac13b..9a298bd 100644
@@ -34,6 +34,8 @@
 #include <linux/poll.h>
 #include <linux/reservation.h>
 
+#include <uapi/linux/dma-buf.h>
+
 static inline int is_dma_buf_file(struct file *);
 
 struct dma_buf_list {
@@ -251,11 +253,52 @@ out:
 	return events;
 }
 
+static long dma_buf_ioctl(struct file *file,
+			  unsigned int cmd, unsigned long arg)
+{
+	struct dma_buf *dmabuf;
+	struct dma_buf_sync sync;
+	enum dma_data_direction direction;
+
+	dmabuf = file->private_data;
+
+	if (!is_dma_buf_file(file))
+		return -EINVAL;
+
+	switch (cmd) {
+	case DMA_BUF_IOCTL_SYNC:
+		if (copy_from_user(&sync, (void __user *) arg, sizeof(sync)))
+			return -EFAULT;
+
+		if (sync.flags & DMA_BUF_SYNC_RW)
+			direction = DMA_BIDIRECTIONAL;
+		else if (sync.flags & DMA_BUF_SYNC_READ)
+			direction = DMA_FROM_DEVICE;
+		else if (sync.flags & DMA_BUF_SYNC_WRITE)
+			direction = DMA_TO_DEVICE;
+		else
+			return -EINVAL;
+
+		if (sync.flags & ~DMA_BUF_SYNC_VALID_FLAGS_MASK)
+			return -EINVAL;
+
+		if (sync.flags & DMA_BUF_SYNC_END)
+			dma_buf_end_cpu_access(dmabuf, direction);
+		else
+			dma_buf_begin_cpu_access(dmabuf, direction);
+
+		return 0;
+	default:
+		return -ENOTTY;
+	}
+}
+
 static const struct file_operations dma_buf_fops = {
 	.release	= dma_buf_release,
 	.mmap		= dma_buf_mmap_internal,
 	.llseek		= dma_buf_llseek,
 	.poll		= dma_buf_poll,
+	.unlocked_ioctl	= dma_buf_ioctl,
 };
 
 /*
include/uapi/linux/dma-buf.h new file +28 -0
new file mode 100644
index 0000000..bd195f2
@@ -0,0 +1,38 @@
+/*
+ * Framework for buffer objects that can be shared across devices/subsystems.
+ *
+ * Copyright(C) 2015 Intel Ltd
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef _DMA_BUF_UAPI_H_
+#define _DMA_BUF_UAPI_H_
+
+/* begin/end dma-buf functions used for userspace mmap. */
+struct dma_buf_sync {
+	__u64 flags;
+};
+
+#define DMA_BUF_SYNC_READ      (1 << 0)
+#define DMA_BUF_SYNC_WRITE     (2 << 0)

include/uapi/linux/dma-buf.h line 38

Quoted 10 lines
include/uapi/linux/dma-buf.h new file +10 -0
+#define DMA_BUF_SYNC_RW        (3 << 0)
+#define DMA_BUF_SYNC_START     (0 << 2)
+#define DMA_BUF_SYNC_END       (1 << 2)
+#define DMA_BUF_SYNC_VALID_FLAGS_MASK \
+	(DMA_BUF_SYNC_RW | DMA_BUF_SYNC_END)
+
+#define DMA_BUF_BASE		'b'
+#define DMA_BUF_IOCTL_SYNC	_IOW(DMA_BUF_BASE, 0, struct dma_buf_sync)
+
+#endif

_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel

#7 lore raw

Re: [PATCH v6 3/5] dma-buf: Add ioctls to allow userspace to flush

Quoted 66 lines · Thomas Hellstrom
Quoted 54 lines · Tiago Vignatti

From: Daniel Vetter <[email protected]>

The userspace might need some sort of cache coherency management e.g. when CPU and GPU domains are being accessed through dma-buf at the same time. To circumvent this problem there are begin/end coherency markers, that forward directly to existing dma-buf device drivers vfunc hooks. Userspace can make use of those markers through the DMA_BUF_IOCTL_SYNC ioctl. The sequence would be used like following:
- mmap dma-buf fd
- for each drawing/upload cycle in CPU 1. SYNC_START ioctl, 2. read/write to mmap area 3. SYNC_END ioctl. This can be repeated as often as you want (with the new data being consumed by the GPU or say scanout device)
- munmap once you don't need the buffer any more

v2 (Tiago): Fix header file type names (u64 -> __u64)
v3 (Tiago): Add documentation. Use enum dma_buf_sync_flags to the begin/end dma-buf functions. Check for overflows in start/length.
v4 (Tiago): use 2d regions for sync.
v5 (Tiago): forget about 2d regions (v4); use _IOW in DMA_BUF_IOCTL_SYNC and remove range information from struct dma_buf_sync.
v6 (Tiago): use __u64 structured padded flags instead enum. Adjust documentation about the recommendation on using sync ioctls.

---
 Documentation/dma-buf-sharing.txt | 22 +++++++++++++++++++-
 drivers/dma-buf/dma-buf.c         | 43 +++++++++++++++++++++++++++++++++++++++
 include/uapi/linux/dma-buf.h      | 38 ++++++++++++++++++++++++++++++++++
 3 files changed, 102 insertions(+), 1 deletion(-)
 create mode 100644 include/uapi/linux/dma-buf.h
Documentation/dma-buf-sharing.txt +12 -1
index 4f4a84b..2ddd4b2 100644
@@ -350,7 +350,27 @@ Being able to mmap an export dma-buf buffer object has 2 main use-cases:
    handles, too). So it's beneficial to support this in a similar fashion on
    dma-buf to have a good transition path for existing Android userspace.
 
-   No special interfaces, userspace simply calls mmap on the dma-buf fd.
+   No special interfaces, userspace simply calls mmap on the dma-buf fd. Very
+   important to note though is that, even if it is not mandatory, the userspace
+   is strongly recommended to always use the cache synchronization ioctl
+   (DMA_BUF_IOCTL_SYNC) discussed next.
+
+   Some systems might need some sort of cache coherency management e.g. when
+   CPU and GPU domains are being accessed through dma-buf at the same time. To
+   circumvent this problem there are begin/end coherency markers, that forward
+   directly to existing dma-buf device drivers vfunc hooks. Userspace can make
+   use of those markers through the DMA_BUF_IOCTL_SYNC ioctl. The sequence
+   would be used like following:
+     - mmap dma-buf fd

Documentation/dma-buf-sharing.txt Being able to mmap an export dma-buf buffer object has 2 main use-cases: · line 374

Quoted 10 lines
Documentation/dma-buf-sharing.txt +9 -0
+     - for each drawing/upload cycle in CPU 1. SYNC_START ioctl, 2. read/write
+       to mmap area 3. SYNC_END ioctl. This can be repeated as often as you
+       want (with the new data being consumed by the GPU or say scanout device)
+     - munmap once you don't need the buffer any more
+
+    In principle systems with the memory cache shared by the GPU and CPU may
+    not need SYNC_START and SYNC_END but still, userspace is always encouraged
+    to use these ioctls before and after, respectively, when accessing the
+    mapped address.
 
Quoted 7 lines

I think the wording here is far too weak. If this is a generic user-space interface and syncing
is required for
a) Correctness: then syncing must be mandatory.
b) Optimal performance then an implementation must generate expected results also in the absence of SYNC ioctls, but is allowed to rely on correct pairing of SYNC_START and SYNC_END to render correctly.

Yeah I guess the wroking should be stronger to make it clear you better call these, always.

Quoted 9 lines

Also recalling the discussion of multidimensional sync, which we said we would add when it was needed, my worst nightmare is when (not if) there are a number of important applications starting to abuse this interface for small writes or reads to large DMA buffers. It will work flawlessly on coherent architectures, and probably some incoherent architectures as well, but will probably be mostly useless on VMware. What is the plan for adding 2D sync to make it work? How do we pursuade app developers to think of damage regions, and can we count on support from the DRM community when this happens?

The current use-case in cros seems to be to do full-blown buffer uploads, nothing partial. I don't think there's anything we can do really (flushing for small uploads will make i915 crawl too, albeit you can still see a slideshow and it's not a complete stop), except maybe improve the docs with a statement that trying to use dma-buf mmap for small uploads will be result in horrible performance. I think inevitable reality is that some truly horrible software will always ship. I am pretty hopeful though that this won't be too common, since the main users for anything dma-buf are on mobile, and those systems (at least on i915) are all incoherent. So as long as it'll run acceptably on i915 I think it should run at least ok-ish on vmwgfx too.
-Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel
#8 lore raw

Re: [PATCH v6 3/5] dma-buf: Add ioctls to allow userspace to flush

In reply to Thomas Hellstrom
Quoted 66 lines · Thomas Hellstrom
Quoted 54 lines · Tiago Vignatti

From: Daniel Vetter <[email protected]>

The userspace might need some sort of cache coherency management e.g. when CPU and GPU domains are being accessed through dma-buf at the same time. To circumvent this problem there are begin/end coherency markers, that forward directly to existing dma-buf device drivers vfunc hooks. Userspace can make use of those markers through the DMA_BUF_IOCTL_SYNC ioctl. The sequence would be used like following:
- mmap dma-buf fd
- for each drawing/upload cycle in CPU 1. SYNC_START ioctl, 2. read/write to mmap area 3. SYNC_END ioctl. This can be repeated as often as you want (with the new data being consumed by the GPU or say scanout device)
- munmap once you don't need the buffer any more

v2 (Tiago): Fix header file type names (u64 -> __u64)
v3 (Tiago): Add documentation. Use enum dma_buf_sync_flags to the begin/end dma-buf functions. Check for overflows in start/length.
v4 (Tiago): use 2d regions for sync.
v5 (Tiago): forget about 2d regions (v4); use _IOW in DMA_BUF_IOCTL_SYNC and remove range information from struct dma_buf_sync.
v6 (Tiago): use __u64 structured padded flags instead enum. Adjust documentation about the recommendation on using sync ioctls.

---
  Documentation/dma-buf-sharing.txt | 22 +++++++++++++++++++-
  drivers/dma-buf/dma-buf.c         | 43 +++++++++++++++++++++++++++++++++++++++
  include/uapi/linux/dma-buf.h      | 38 ++++++++++++++++++++++++++++++++++
  3 files changed, 102 insertions(+), 1 deletion(-)

create mode 100644 include/uapi/linux/dma-buf.h

Documentation/dma-buf-sharing.txt +12 -1
index 4f4a84b..2ddd4b2 100644
@@ -350,7 +350,27 @@ Being able to mmap an export dma-buf buffer object has 2 main use-cases:
     handles, too). So it's beneficial to support this in a similar fashion on
     dma-buf to have a good transition path for existing Android userspace.
 
-   No special interfaces, userspace simply calls mmap on the dma-buf fd.
+   No special interfaces, userspace simply calls mmap on the dma-buf fd. Very
+   important to note though is that, even if it is not mandatory, the userspace
+   is strongly recommended to always use the cache synchronization ioctl
+   (DMA_BUF_IOCTL_SYNC) discussed next.
+
+   Some systems might need some sort of cache coherency management e.g. when
+   CPU and GPU domains are being accessed through dma-buf at the same time. To
+   circumvent this problem there are begin/end coherency markers, that forward
+   directly to existing dma-buf device drivers vfunc hooks. Userspace can make
+   use of those markers through the DMA_BUF_IOCTL_SYNC ioctl. The sequence
+   would be used like following:
+     - mmap dma-buf fd

Documentation/dma-buf-sharing.txt Being able to mmap an export dma-buf buffer object has 2 main use-cases: · line 374

Quoted 10 lines
Documentation/dma-buf-sharing.txt +9 -0
+     - for each drawing/upload cycle in CPU 1. SYNC_START ioctl, 2. read/write
+       to mmap area 3. SYNC_END ioctl. This can be repeated as often as you
+       want (with the new data being consumed by the GPU or say scanout device)
+     - munmap once you don't need the buffer any more
+
+    In principle systems with the memory cache shared by the GPU and CPU may
+    not need SYNC_START and SYNC_END but still, userspace is always encouraged
+    to use these ioctls before and after, respectively, when accessing the
+    mapped address.
 
Quoted 7 lines

I think the wording here is far too weak. If this is a generic user-space interface and syncing
is required for
a) Correctness: then syncing must be mandatory.
b) Optimal performance then an implementation must generate expected results also in the absence of SYNC ioctls, but is allowed to rely on correct pairing of SYNC_START and SYNC_END to render correctly.

Thomas, do you think the following write-up captures this?

-   No special interfaces, userspace simply calls mmap on the dma-buf fd.
+   No special interfaces, userspace simply calls mmap on the dma-buf fd, making
+   sure that the cache synchronization ioctl (DMA_BUF_IOCTL_SYNC) is *always*
+   used when the access happens. This is discussed next paragraphs.
+
+   Some systems might need some sort of cache coherency management e.g. when
+   CPU and GPU domains are being accessed through dma-buf at the same time. To
+   circumvent this problem there are begin/end coherency markers, that forward
+   directly to existing dma-buf device drivers vfunc hooks. Userspace can make
+   use of those markers through the DMA_BUF_IOCTL_SYNC ioctl. The sequence
+   would be used like following:

+ - mmap dma-buf fd
+ - for each drawing/upload cycle in CPU 1. SYNC_START ioctl, 2. read/write
+ to mmap area 3. SYNC_END ioctl. This can be repeated as often as you
+ want (with the new data being consumed by the GPU or say scanout device)
+ - munmap once you don't need the buffer any more
+

+    Therefore, for correctness and optimal performance, systems with the memory
+    cache shared by the GPU and CPU i.e. the "coherent" and also "incoherent"
+    systems are always required to use SYNC_START and SYNC_END before and
+    after, respectively, when accessing the mapped address.

Thank you,

Tiago
_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel

#9 lore raw

Re: [PATCH v6 3/5] dma-buf: Add ioctls to allow userspace to flush

Quoted 144 lines · Tiago Vignatti
Quoted 96 lines · Thomas Hellstrom
Quoted 87 lines · Tiago Vignatti

From: Daniel Vetter <[email protected]>

The userspace might need some sort of cache coherency management e.g. when CPU
and GPU domains are being accessed through dma-buf at the same time. To circumvent this problem there are begin/end coherency markers, that forward
directly to existing dma-buf device drivers vfunc hooks. Userspace can make use
of those markers through the DMA_BUF_IOCTL_SYNC ioctl. The sequence would be
used like following:
- mmap dma-buf fd
- for each drawing/upload cycle in CPU 1. SYNC_START ioctl, 2.
read/write
to mmap area 3. SYNC_END ioctl. This can be repeated as
often as you
want (with the new data being consumed by the GPU or say
scanout device)
- munmap once you don't need the buffer any more

v2 (Tiago): Fix header file type names (u64 -> __u64) v3 (Tiago): Add documentation. Use enum dma_buf_sync_flags to the begin/end
dma-buf functions. Check for overflows in start/length. v4 (Tiago): use 2d regions for sync.
v5 (Tiago): forget about 2d regions (v4); use _IOW in DMA_BUF_IOCTL_SYNC and
remove range information from struct dma_buf_sync.
v6 (Tiago): use __u64 structured padded flags instead enum. Adjust documentation about the recommendation on using sync ioctls.

---
  Documentation/dma-buf-sharing.txt | 22 +++++++++++++++++++-
  drivers/dma-buf/dma-buf.c         | 43
+++++++++++++++++++++++++++++++++++++++
  include/uapi/linux/dma-buf.h      | 38
++++++++++++++++++++++++++++++++++
  3 files changed, 102 insertions(+), 1 deletion(-)

create mode 100644 include/uapi/linux/dma-buf.h

Documentation/dma-buf-sharing.txt +0 -0
index 4f4a84b..2ddd4b2 100644
@@ -350,7 +350,27 @@ Being able to mmap an export dma-buf buffer

object has 2 main use-cases:
handles, too). So it's beneficial to support this in a similar
fashion on
dma-buf to have a good transition path for existing Android
userspace.

- No special interfaces, userspace simply calls mmap on the
dma-buf fd.
+ No special interfaces, userspace simply calls mmap on the
dma-buf fd. Very
+ important to note though is that, even if it is not mandatory,
the userspace
+ is strongly recommended to always use the cache synchronization
ioctl

Documentation/dma-buf-sharing.txt +3 -0
+   (DMA_BUF_IOCTL_SYNC) discussed next.
+
+   Some systems might need some sort of cache coherency management

e.g. when
+ CPU and GPU domains are being accessed through dma-buf at the
same time. To
+ circumvent this problem there are begin/end coherency markers,
that forward
+ directly to existing dma-buf device drivers vfunc hooks.
Userspace can make
+ use of those markers through the DMA_BUF_IOCTL_SYNC ioctl. The
sequence

Documentation/dma-buf-sharing.txt +3 -0
+   would be used like following:
+     - mmap dma-buf fd
+     - for each drawing/upload cycle in CPU 1. SYNC_START ioctl, 2.

read/write
+ to mmap area 3. SYNC_END ioctl. This can be repeated as
often as you
+ want (with the new data being consumed by the GPU or say
scanout device)

Documentation/dma-buf-sharing.txt +3 -0
+     - munmap once you don't need the buffer any more
+
+    In principle systems with the memory cache shared by the GPU
Quoted 7 lines

and CPU may
+ not need SYNC_START and SYNC_END but still, userspace is always
encouraged
+ to use these ioctls before and after, respectively, when
accessing the
+ mapped address.

Quoted 7 lines

I think the wording here is far too weak. If this is a generic user-space interface and syncing
is required for
a) Correctness: then syncing must be mandatory.
b) Optimal performance then an implementation must generate expected results also in the absence of SYNC ioctls, but is allowed to rely on correct pairing of SYNC_START and SYNC_END to render correctly.

Thomas, do you think the following write-up captures this?

(quoted diff) +1 -1
-   No special interfaces, userspace simply calls mmap on the dma-buf fd.
+   No special interfaces, userspace simply calls mmap on the dma-buf

fd, making
+ sure that the cache synchronization ioctl (DMA_BUF_IOCTL_SYNC) is
*always*

(quoted diff) +3 -0
+   used when the access happens. This is discussed next paragraphs.
+
+   Some systems might need some sort of cache coherency management

e.g. when
+ CPU and GPU domains are being accessed through dma-buf at the same
time. To
+ circumvent this problem there are begin/end coherency markers,
that forward
+ directly to existing dma-buf device drivers vfunc hooks. Userspace
can make
+ use of those markers through the DMA_BUF_IOCTL_SYNC ioctl. The
sequence

Documentation/dma-buf-sharing.txt +3 -0
+   would be used like following:
+     - mmap dma-buf fd
+     - for each drawing/upload cycle in CPU 1. SYNC_START ioctl, 2.

read/write
+ to mmap area 3. SYNC_END ioctl. This can be repeated as often
as you
+ want (with the new data being consumed by the GPU or say
scanout device)

Documentation/dma-buf-sharing.txt +3 -0
+     - munmap once you don't need the buffer any more
+
+    Therefore, for correctness and optimal performance, systems with

the memory
+ cache shared by the GPU and CPU i.e. the "coherent" and also
"incoherent"
+ systems are always required to use SYNC_START and SYNC_END before
and
+ after, respectively, when accessing the mapped address.

Quoted 3 lines

Thank you,

Tiago

Yes, that sounds better,

Thanks,
Thomas

_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel

#10 lore raw

PATCH v6 4/5 drm/i915: Implement end_cpu_access

In reply to Tiago Vignatti

This function is meant to be used with dma-buf mmap, when finishing the CPU access of the mapped pointer.

The error case should be rare to happen though, requiring the buffer become active during the sync period and for the end_cpu_access to be interrupted. So we use a uninterruptible mutex_lock to spit out when it ever happens.

v2: disable interruption to make sure errors are reported.
v3: update to the new end_cpu_access API.

---
 drivers/gpu/drm/i915/i915_gem_dmabuf.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
drivers/gpu/drm/i915/i915_gem_dmabuf.c +22 -0
index 65ab2bd..9dba876 100644
@@ -212,6 +212,27 @@ static int i915_gem_begin_cpu_access(struct dma_buf *dma_buf, enum dma_data_dire
 	return ret;
 }
 
+static void i915_gem_end_cpu_access(struct dma_buf *dma_buf, enum dma_data_direction direction)
+{
+	struct drm_i915_gem_object *obj = dma_buf_to_obj(dma_buf);
+	struct drm_device *dev = obj->base.dev;
+	struct drm_i915_private *dev_priv = to_i915(dev);
+	bool was_interruptible, write = (direction == DMA_BIDIRECTIONAL || direction == DMA_TO_DEVICE);
+	int ret;
+
+	mutex_lock(&dev->struct_mutex);
+	was_interruptible = dev_priv->mm.interruptible;
+	dev_priv->mm.interruptible = false;
+
+	ret = i915_gem_object_set_to_gtt_domain(obj, write);
+
+	dev_priv->mm.interruptible = was_interruptible;
+	mutex_unlock(&dev->struct_mutex);
+
+	if (unlikely(ret))
+		DRM_ERROR("unable to flush buffer following CPU access; rendering may be corrupt\n");
+}
+
 static const struct dma_buf_ops i915_dmabuf_ops =  {
 	.map_dma_buf = i915_gem_map_dma_buf,
 	.unmap_dma_buf = i915_gem_unmap_dma_buf,
@@ -224,6 +245,7 @@ static const struct dma_buf_ops i915_dmabuf_ops =  {
 	.vmap = i915_gem_dmabuf_vmap,
 	.vunmap = i915_gem_dmabuf_vunmap,
 	.begin_cpu_access = i915_gem_begin_cpu_access,
+	.end_cpu_access = i915_gem_end_cpu_access,
 };
 
 struct dma_buf *i915_gem_prime_export(struct drm_device *dev,
Signature · 6 lines
2.1.4

_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel
#11 lore raw

Re: [PATCH v6 4/5] drm/i915: Implement end_cpu_access

Quoted 6 lines · Tiago Vignatti

This function is meant to be used with dma-buf mmap, when finishing the CPU access of the mapped pointer.

drivers/gpu/drm/i915/i915_gem_dmabuf.c +3 -0
+static void i915_gem_end_cpu_access(struct dma_buf *dma_buf, enum dma_data_direction direction)
+{
+	struct drm_i915_gem_object *obj = dma_buf_to_obj(dma_buf);

drivers/gpu/drm/i915/i915_gem_dmabuf.c static int i915_gem_begin_cpu_access(struct dma_buf *dma_buf, enum dma_data_dire · line 227

Quoted 10 lines
drivers/gpu/drm/i915/i915_gem_dmabuf.c +10 -0
+	struct drm_device *dev = obj->base.dev;
+	struct drm_i915_private *dev_priv = to_i915(dev);
+	bool was_interruptible, write = (direction == DMA_BIDIRECTIONAL || direction == DMA_TO_DEVICE);
+	int ret;
+
+	mutex_lock(&dev->struct_mutex);
+	was_interruptible = dev_priv->mm.interruptible;
+	dev_priv->mm.interruptible = false;
+
+	ret = i915_gem_object_set_to_gtt_domain(obj, write);

This only needs to pass .write=false. The dma-buf direction is only for the period of the user access, and we are now flushing the caches. This is equivalent to the sw-finish ioctl and ideally we just want the i915_gem_object_flush_cpu_write_domain().
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel
#12 lore raw

Re: [PATCH v6 4/5] drm/i915: Implement end_cpu_access

Quoted 14 lines · Chris Wilson
Quoted 13 lines · Tiago Vignatti

This function is meant to be used with dma-buf mmap, when finishing the CPU access of the mapped pointer.

drivers/gpu/drm/i915/i915_gem_dmabuf.c +10 -0
+static void i915_gem_end_cpu_access(struct dma_buf *dma_buf, enum dma_data_direction direction)
+{
+	struct drm_i915_gem_object *obj = dma_buf_to_obj(dma_buf);
+	struct drm_device *dev = obj->base.dev;
+	struct drm_i915_private *dev_priv = to_i915(dev);
+	bool was_interruptible, write = (direction == DMA_BIDIRECTIONAL || direction == DMA_TO_DEVICE);
+	int ret;
+
+	mutex_lock(&dev->struct_mutex);
+	was_interruptible = dev_priv->mm.interruptible;
Quoted 8 lines

drivers/gpu/drm/i915/i915_gem_dmabuf.c static int i915_gem_begin_cpu_access(struct dma_buf *dma_buf, enum dma_data_dire · line 227

Quoted 3 lines
drivers/gpu/drm/i915/i915_gem_dmabuf.c +3 -0
+	dev_priv->mm.interruptible = false;
+
+	ret = i915_gem_object_set_to_gtt_domain(obj, write);

This only needs to pass .write=false. The dma-buf direction is only for the period of the user access, and we are now flushing the caches. This is equivalent to the sw-finish ioctl and ideally we just want the i915_gem_object_flush_cpu_write_domain().

in fact the only usage so far I found for end_cpu_access is when the pinned buffer is scanout out. Should I pretty much copy sw-finish in end_cpu_access then?

Thanks,

Tiago

_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel

#13 lore raw

Re: [PATCH v6 4/5] drm/i915: Implement end_cpu_access

Quoted 27 lines · Tiago Vignatti
Quoted 17 lines · Chris Wilson
Quoted 16 lines · Tiago Vignatti

This function is meant to be used with dma-buf mmap, when finishing the CPU
access of the mapped pointer.

+static void i915_gem_end_cpu_access(struct dma_buf *dma_buf, enum dma_data_direction direction)

drivers/gpu/drm/i915/i915_gem_dmabuf.c +5 -0
+{
+    struct drm_i915_gem_object *obj = dma_buf_to_obj(dma_buf);
+    struct drm_device *dev = obj->base.dev;
+    struct drm_i915_private *dev_priv = to_i915(dev);
+    bool was_interruptible, write = (direction == DMA_BIDIRECTIONAL

|| direction == DMA_TO_DEVICE);

drivers/gpu/drm/i915/i915_gem_dmabuf.c +4 -0
+    int ret;
+
+    mutex_lock(&dev->struct_mutex);
+    was_interruptible = dev_priv->mm.interruptible;
Quoted 8 lines

drivers/gpu/drm/i915/i915_gem_dmabuf.c static int i915_gem_begin_cpu_access(struct dma_buf *dma_buf, enum dma_data_dire · line 227

Quoted 3 lines
drivers/gpu/drm/i915/i915_gem_dmabuf.c +3 -0
+    dev_priv->mm.interruptible = false;
+
+    ret = i915_gem_object_set_to_gtt_domain(obj, write);

This only needs to pass .write=false. The dma-buf direction is only for the period of the user access, and we are now flushing the caches. This is equivalent to the sw-finish ioctl and ideally we just want the i915_gem_object_flush_cpu_write_domain().

Quoted 3 lines

in fact the only usage so far I found for end_cpu_access is when the pinned buffer is scanout out. Should I pretty much copy sw-finish in end_cpu_access then?

And do you think it's okay to declare i915_gem_object_flush_cpu_write_domain outside its file's only scope?

Tiago
_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel

#14 lore raw

Re: [PATCH v6 4/5] drm/i915: Implement end_cpu_access

Quoted 28 lines · Tiago Vignatti
Quoted 27 lines · Tiago Vignatti
Quoted 25 lines · Chris Wilson
Quoted 12 lines · Tiago Vignatti

This function is meant to be used with dma-buf mmap, when finishing the CPU
access of the mapped pointer.

+static void i915_gem_end_cpu_access(struct dma_buf *dma_buf, enum dma_data_direction direction)

drivers/gpu/drm/i915/i915_gem_dmabuf.c +5 -0
+{
+    struct drm_i915_gem_object *obj = dma_buf_to_obj(dma_buf);
+    struct drm_device *dev = obj->base.dev;
+    struct drm_i915_private *dev_priv = to_i915(dev);
+    bool was_interruptible, write = (direction == DMA_BIDIRECTIONAL

|| direction == DMA_TO_DEVICE);

drivers/gpu/drm/i915/i915_gem_dmabuf.c static int i915_gem_begin_cpu_access(struct dma_buf *dma_buf, enum dma_data_dire · line 227

Quoted 7 lines
drivers/gpu/drm/i915/i915_gem_dmabuf.c +7 -0
+    int ret;
+
+    mutex_lock(&dev->struct_mutex);
+    was_interruptible = dev_priv->mm.interruptible;
+    dev_priv->mm.interruptible = false;
+
+    ret = i915_gem_object_set_to_gtt_domain(obj, write);

This only needs to pass .write=false. The dma-buf direction is only for the period of the user access, and we are now flushing the caches. This is equivalent to the sw-finish ioctl and ideally we just want the i915_gem_object_flush_cpu_write_domain().

Quoted 7 lines
Quoted 3 lines

in fact the only usage so far I found for end_cpu_access is when the pinned buffer is scanout out. Should I pretty much copy sw-finish in end_cpu_access then?

And do you think it's okay to declare i915_gem_object_flush_cpu_write_domain outside its file's only scope?

Whilst the simplicity of just doing the flush appeals, calling set_gtt_domain(write=false) isn't that much heavier (the difference will be lost in the noise of any clflushing) and is going to be always correct. Whereas just flushing the CPU domain may be a hassle for us in future. -Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel
#15 lore raw

PATCH v6 5/5 drm/i915: Use CPU mapping for userspace dma-buf mmap()

In reply to Tiago Vignatti

Userspace is the one in charge of flush CPU by wrapping mmap with begin{,end}_cpu_access.

v2: Remove LLC check cause we have dma-buf sync providers now. Also, fix return before transferring ownership when mmap fails.
v3: Fix return values.
v4: !obj->base.filp is user triggerable, so removed the WARN_ON.

---
 drivers/gpu/drm/i915/i915_gem_dmabuf.c | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_gem_dmabuf.c +17 -1
index 9dba876..b7e7a90 100644
@@ -193,7 +193,23 @@ static void i915_gem_dmabuf_kunmap(struct dma_buf *dma_buf, unsigned long page_n
 
 static int i915_gem_dmabuf_mmap(struct dma_buf *dma_buf, struct vm_area_struct *vma)
 {
-	return -EINVAL;
+	struct drm_i915_gem_object *obj = dma_buf_to_obj(dma_buf);
+	int ret;
+
+	if (obj->base.size < vma->vm_end - vma->vm_start)
+		return -EINVAL;
+
+	if (!obj->base.filp)
+		return -ENODEV;
+
+	ret = obj->base.filp->f_op->mmap(obj->base.filp, vma);
+	if (ret)
+		return ret;
+
+	fput(vma->vm_file);
+	vma->vm_file = get_file(obj->base.filp);
+
+	return 0;
 }
 
 static int i915_gem_begin_cpu_access(struct dma_buf *dma_buf, enum dma_data_direction direction)
Signature · 6 lines
2.1.4

_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel
#16 lore raw

PATCH v6 1/6 lib: Add gem_userptr and __gem_userptr helpers

In reply to Tiago Vignatti

This patch moves userptr definitions and helpers implementation that were locally in gem_userptr_benchmark and gem_userptr_blits to the library, so other tests can make use of them as well. There's no functional changes.

v2: added __ function to differentiate when errors want to be handled back in the caller; bring gem_userptr_sync back to gem_userptr_blits; added gtkdoc.

---
 benchmarks/gem_userptr_benchmark.c |  55 +++-----------------
 lib/ioctl_wrappers.c               |  41 +++++++++++++++
 lib/ioctl_wrappers.h               |  13 +++++
 tests/gem_userptr_blits.c          | 104 ++++++++++---------------------------
 4 files changed, 86 insertions(+), 127 deletions(-)
benchmarks/gem_userptr_benchmark.c +6 -49
index 1eae7ff..f7716df 100644
@@ -58,17 +58,6 @@
   #define PAGE_SIZE 4096
 #endif
 
-#define LOCAL_I915_GEM_USERPTR       0x33
-#define LOCAL_IOCTL_I915_GEM_USERPTR DRM_IOWR (DRM_COMMAND_BASE + LOCAL_I915_GEM_USERPTR, struct local_i915_gem_userptr)
-struct local_i915_gem_userptr {
-	uint64_t user_ptr;
-	uint64_t user_size;
-	uint32_t flags;
-#define LOCAL_I915_USERPTR_READ_ONLY (1<<0)
-#define LOCAL_I915_USERPTR_UNSYNCHRONIZED (1<<31)
-	uint32_t handle;
-};
-
 static uint32_t userptr_flags = LOCAL_I915_USERPTR_UNSYNCHRONIZED;
 
 #define BO_SIZE (65536)
@@ -83,30 +72,6 @@ static void gem_userptr_test_synchronized(void)
 	userptr_flags = 0;
 }
 
-static int gem_userptr(int fd, void *ptr, int size, int read_only, uint32_t *handle)
-{
-	struct local_i915_gem_userptr userptr;
-	int ret;
-
-	userptr.user_ptr = (uintptr_t)ptr;
-	userptr.user_size = size;
-	userptr.flags = userptr_flags;
-	if (read_only)
-		userptr.flags |= LOCAL_I915_USERPTR_READ_ONLY;
-
-	ret = drmIoctl(fd, LOCAL_IOCTL_I915_GEM_USERPTR, &userptr);
-	if (ret)
-		ret = errno;
-	igt_skip_on_f(ret == ENODEV &&
-		      (userptr_flags & LOCAL_I915_USERPTR_UNSYNCHRONIZED) == 0 &&
-		      !read_only,
-		      "Skipping, synchronized mappings with no kernel CONFIG_MMU_NOTIFIER?");
-	if (ret == 0)
-		*handle = userptr.handle;
-
-	return ret;
-}
-
 static void **handle_ptr_map;
 static unsigned int num_handle_ptr_map;
 
@@ -144,8 +109,7 @@ static uint32_t create_userptr_bo(int fd, int size)
 	ret = posix_memalign(&ptr, PAGE_SIZE, size);
 	igt_assert(ret == 0);
 
-	ret = gem_userptr(fd, (uint32_t *)ptr, size, 0, &handle);
-	igt_assert(ret == 0);
+	gem_userptr(fd, (uint32_t *)ptr, size, 0, userptr_flags, &handle);
 	add_handle_ptr(handle, ptr);
 
 	return handle;
@@ -167,7 +131,7 @@ static int has_userptr(int fd)
 	assert(posix_memalign(&ptr, PAGE_SIZE, PAGE_SIZE) == 0);
 	oldflags = userptr_flags;
 	gem_userptr_test_unsynchronized();
-	ret = gem_userptr(fd, ptr, PAGE_SIZE, 0, &handle);
+	ret = __gem_userptr(fd, ptr, PAGE_SIZE, 0, userptr_flags, &handle);
 	userptr_flags = oldflags;
 	if (ret != 0) {
 		free(ptr);
@@ -379,9 +343,7 @@ static void test_impact_overlap(int fd, const char *prefix)
 
 			for (i = 0, p = block; i < nr_bos[subtest];
 			     i++, p += PAGE_SIZE)
-				ret = gem_userptr(fd, (uint32_t *)p, BO_SIZE, 0,
-						  &handles[i]);
-				igt_assert(ret == 0);
+				gem_userptr(fd, (uint32_t *)p, BO_SIZE, 0, userptr_flags, &handles[i]);
 		}
 
 		if (nr_bos[subtest] > 0)
@@ -427,7 +389,6 @@ static void test_single(int fd)
 	char *ptr, *bo_ptr;
 	uint32_t handle = 0;
 	unsigned long iter = 0;
-	int ret;
 	unsigned long map_size = BO_SIZE + PAGE_SIZE - 1;
 
 	ptr = mmap(NULL, map_size, PROT_READ | PROT_WRITE,
@@ -439,8 +400,7 @@ static void test_single(int fd)
 	start_test(test_duration_sec);
 
 	while (run_test) {
-		ret = gem_userptr(fd, bo_ptr, BO_SIZE, 0, &handle);
-		assert(ret == 0);
+		gem_userptr(fd, bo_ptr, BO_SIZE, 0, userptr_flags, &handle);
 		gem_close(fd, handle);
 		iter++;
 	}
@@ -456,7 +416,6 @@ static void test_multiple(int fd, unsigned int batch, int random)
 	uint32_t handles[10000];
 	int map[10000];
 	unsigned long iter = 0;
-	int ret;
 	int i;
 	unsigned long map_size = batch * BO_SIZE + PAGE_SIZE - 1;
 
@@ -478,10 +437,8 @@ static void test_multiple(int fd, unsigned int batch, int random)
 		if (random)
 			igt_permute_array(map, batch, igt_exchange_int);
 		for (i = 0; i < batch; i++) {
-			ret = gem_userptr(fd, bo_ptr + map[i] * BO_SIZE,
-						BO_SIZE,
-						0, &handles[i]);
-			assert(ret == 0);
+			gem_userptr(fd, bo_ptr + map[i] * BO_SIZE, BO_SIZE,
+						0, userptr_flags, &handles[i]);
 		}
 		if (random)
 			igt_permute_array(map, batch, igt_exchange_int);
lib/ioctl_wrappers.c +41 -0

47 lines not on this page · raw message

lib/ioctl_wrappers.h +13 -0
index fe2f687..bb8a858 100644
@@ -112,6 +112,19 @@ void gem_context_require_param(int fd, uint64_t param);
 void gem_context_get_param(int fd, struct local_i915_gem_context_param *p);
 void gem_context_set_param(int fd, struct local_i915_gem_context_param *p);
 
+#define LOCAL_I915_GEM_USERPTR       0x33
+#define LOCAL_IOCTL_I915_GEM_USERPTR DRM_IOWR (DRM_COMMAND_BASE + LOCAL_I915_GEM_USERPTR, struct local_i915_gem_userptr)
+struct local_i915_gem_userptr {
+  uint64_t user_ptr;
+  uint64_t user_size;
+  uint32_t flags;
+#define LOCAL_I915_USERPTR_READ_ONLY (1<<0)
+#define LOCAL_I915_USERPTR_UNSYNCHRONIZED (1<<31)
+  uint32_t handle;
+};
+void gem_userptr(int fd, void *ptr, int size, int read_only, uint32_t flags, uint32_t *handle);
+int __gem_userptr(int fd, void *ptr, int size, int read_only, uint32_t flags, uint32_t *handle);
+
 void gem_sw_finish(int fd, uint32_t handle);
 
 bool gem_bo_busy(int fd, uint32_t handle);
tests/gem_userptr_blits.c +26 -78

263 lines not on this page · raw message

Signature · 6 lines
2.1.4

_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel
#17 lore raw

PATCH v6 2/6 prime_mmap: Add new test for calling mmap() on dma-buf fds

In reply to Tiago Vignatti

From: Rob Bradford <[email protected]>

This test has the following subtests:
- test_correct for correctness of the data
- test_map_unmap checks for mapping idempotency
- test_reprime checks for dma-buf creation idempotency
- test_forked checks for multiprocess access
- test_refcounting checks for buffer reference counting
- test_dup checks that dup()ing the fd works
- test_userptr make sure it fails when mmaping due the lack of obj->base.filp in a userptr.
- test_errors checks the error return values for failures
- test_aperture_limit tests multiple buffer creation at the gtt aperture limit

v2 (Tiago): Removed pattern_check(), which was walking through a useless iterator. Removed superfluous PROT_WRITE from gem_mmap, in test_correct(). Added binary file to .gitignore
v3 (Tiago): squash patch "prime_mmap: Test for userptr mmap" into this one. v4 (Tiago): use synchronized userptr for testing. Add test for buffer overlapping.

---
 tests/Makefile.sources |   1 +
 tests/prime_mmap.c     | 417 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 418 insertions(+)
 create mode 100644 tests/prime_mmap.c
tests/Makefile.sources +1 -0
index d594038..75f3cb0 100644
@@ -96,6 +96,7 @@ TESTS_progs_M = \
 	pm_rps \
 	pm_rc6_residency \
 	pm_sseu \
+	prime_mmap \
 	prime_self_import \
 	template \
 	$(NULL)
tests/prime_mmap.c new file +417 -0

417 lines not on this page · raw message

Signature · 6 lines
2.1.4

_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel
#18 lore raw

PATCH v6 3/6 prime_mmap: Add basic tests to write in a bo using CPU

In reply to Tiago Vignatti

This patch adds test_correct_cpu_write, which maps the texture buffer through a prime fd and then writes directly to it using the CPU. It stresses the driver to guarantee cache synchronization among the different domains.

This test also adds test_forked_cpu_write, which creates the GEM bo in one process and pass the prime handle of the it to another process, which in turn uses the handle only to map and write. Roughly speaking this test simulates Chrome OS architecture, where the Web content ("unpriviledged process") maps and CPU-draws a buffer, which was previously allocated in the GPU process ("priviledged process").

This requires kernel modifications (Daniel Thompson's "drm: prime: Honour O_RDWR during prime-handle-to-fd") and therefore prime_handle_to_fd_for_mmap is added to fail in case these lack. Also, upcoming tests (e.g. next patch) are going to use it as well, so make it public and available in the lib.

v2: adds prime_handle_to_fd_with_mmap for skipping test in older kernels and test for invalid flags.

---
 lib/ioctl_wrappers.c | 25 +++++++++++++++
 lib/ioctl_wrappers.h |  4 +++
 tests/prime_mmap.c   | 89 ++++++++++++++++++++++++++++++++++++++++++++++++----
 3 files changed, 112 insertions(+), 6 deletions(-)
lib/ioctl_wrappers.c +25 -0
index 6cad8a2..86a61ba 100644
@@ -1329,6 +1329,31 @@ int prime_handle_to_fd(int fd, uint32_t handle)
 }
 
 /**
+ * prime_handle_to_fd_for_mmap:
+ * @fd: open i915 drm file descriptor
+ * @handle: file-private gem buffer object handle
+ *
+ * Same as prime_handle_to_fd above but with DRM_RDWR capabilities, which can
+ * be useful for writing into the mmap'ed dma-buf file-descriptor.
+ *
+ * Returns: The created dma-buf fd handle or -1 if the ioctl fails.
+ */
+int prime_handle_to_fd_for_mmap(int fd, uint32_t handle)
+{
+	struct drm_prime_handle args;
+
+	memset(&args, 0, sizeof(args));
+	args.handle = handle;
+	args.flags = DRM_CLOEXEC | DRM_RDWR;
+	args.fd = -1;
+
+	if (drmIoctl(fd, DRM_IOCTL_PRIME_HANDLE_TO_FD, &args) != 0)
+		return -1;
+
+	return args.fd;
+}
+
+/**
  * prime_fd_to_handle:
  * @fd: open i915 drm file descriptor
  * @dma_buf_fd: dma-buf fd handle
lib/ioctl_wrappers.h +4 -0
index bb8a858..d3ffba2 100644
@@ -149,6 +149,10 @@ void gem_require_ring(int fd, int ring_id);
 
 /* prime */
 int prime_handle_to_fd(int fd, uint32_t handle);
+#ifndef DRM_RDWR
+#define DRM_RDWR O_RDWR
+#endif
+int prime_handle_to_fd_for_mmap(int fd, uint32_t handle);
 uint32_t prime_fd_to_handle(int fd, int dma_buf_fd);
 off_t prime_get_size(int dma_buf_fd);
 
tests/prime_mmap.c +83 -6

144 lines not on this page · raw message

Signature · 6 lines
2.1.4

_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel
#19 lore raw

PATCH v6 4/6 lib: Add prime_sync_start and prime_sync_end helpers

In reply to Tiago Vignatti

This patch adds dma-buf mmap synchronization ioctls that can be used by tests for cache coherency management e.g. when CPU and GPU domains are being accessed through dma-buf at the same time.

---
 lib/ioctl_wrappers.c | 26 ++++++++++++++++++++++++++
 lib/ioctl_wrappers.h | 15 +++++++++++++++
 2 files changed, 41 insertions(+)
lib/ioctl_wrappers.c +26 -0
index 86a61ba..0d84d00 100644
@@ -1400,6 +1400,32 @@ off_t prime_get_size(int dma_buf_fd)
 }
 
 /**
+ * prime_sync_start
+ * @dma_buf_fd: dma-buf fd handle
+ */
+void prime_sync_start(int dma_buf_fd)
+{
+	struct local_dma_buf_sync sync_start;
+
+	memset(&sync_start, 0, sizeof(sync_start));
+	sync_start.flags = LOCAL_DMA_BUF_SYNC_START | LOCAL_DMA_BUF_SYNC_RW;
+	do_ioctl(dma_buf_fd, LOCAL_DMA_BUF_IOCTL_SYNC, &sync_start);
+}
+
+/**
+ * prime_sync_end
+ * @dma_buf_fd: dma-buf fd handle
+ */
+void prime_sync_end(int dma_buf_fd)
+{
+	struct local_dma_buf_sync sync_end;
+
+	memset(&sync_end, 0, sizeof(sync_end));
+	sync_end.flags = LOCAL_DMA_BUF_SYNC_END | LOCAL_DMA_BUF_SYNC_RW;
+	do_ioctl(dma_buf_fd, LOCAL_DMA_BUF_IOCTL_SYNC, &sync_end);
+}
+
+/**
  * igt_require_fb_modifiers:
  * @fd: Open DRM file descriptor.
  *
lib/ioctl_wrappers.h +15 -0
index d3ffba2..cbd7a73 100644
@@ -148,6 +148,19 @@ void gem_require_caching(int fd);
 void gem_require_ring(int fd, int ring_id);
 
 /* prime */
+struct local_dma_buf_sync {
+	uint64_t flags;
+};
+
+#define LOCAL_DMA_BUF_SYNC_RW        (3 << 0)
+#define LOCAL_DMA_BUF_SYNC_START     (0 << 2)
+#define LOCAL_DMA_BUF_SYNC_END       (1 << 2)
+#define DMA_BUF_SYNC_VALID_FLAGS_MASK \
+		(DMA_BUF_SYNC_RW | DMA_BUF_SYNC_END)
+
+#define LOCAL_DMA_BUF_BASE 'b'
+#define LOCAL_DMA_BUF_IOCTL_SYNC _IOW(LOCAL_DMA_BUF_BASE, 0, struct local_dma_buf_sync)
+
 int prime_handle_to_fd(int fd, uint32_t handle);
 #ifndef DRM_RDWR
 #define DRM_RDWR O_RDWR
@@ -155,6 +168,8 @@ int prime_handle_to_fd(int fd, uint32_t handle);
 int prime_handle_to_fd_for_mmap(int fd, uint32_t handle);
 uint32_t prime_fd_to_handle(int fd, int dma_buf_fd);
 off_t prime_get_size(int dma_buf_fd);
+void prime_sync_start(int dma_buf_fd);
+void prime_sync_end(int dma_buf_fd);
 
 /* addfb2 fb modifiers */
 struct local_drm_mode_fb_cmd2 {
Signature · 6 lines
2.1.4

_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel
#20 lore raw

Re: [PATCH igt v6 4/6] lib: Add prime_sync_start and prime_sync_end helpers

Quoted 54 lines · Tiago Vignatti

This patch adds dma-buf mmap synchronization ioctls that can be used by tests for cache coherency management e.g. when CPU and GPU domains are being accessed through dma-buf at the same time.

---
 lib/ioctl_wrappers.c | 26 ++++++++++++++++++++++++++
 lib/ioctl_wrappers.h | 15 +++++++++++++++
 2 files changed, 41 insertions(+)
lib/ioctl_wrappers.c +26 -0
index 86a61ba..0d84d00 100644
@@ -1400,6 +1400,32 @@ off_t prime_get_size(int dma_buf_fd)
 }
 
 /**
+ * prime_sync_start
+ * @dma_buf_fd: dma-buf fd handle
+ */
+void prime_sync_start(int dma_buf_fd)
+{
+	struct local_dma_buf_sync sync_start;
+
+	memset(&sync_start, 0, sizeof(sync_start));
+	sync_start.flags = LOCAL_DMA_BUF_SYNC_START | LOCAL_DMA_BUF_SYNC_RW;
+	do_ioctl(dma_buf_fd, LOCAL_DMA_BUF_IOCTL_SYNC, &sync_start);
+}
+
+/**
+ * prime_sync_end
+ * @dma_buf_fd: dma-buf fd handle
+ */
+void prime_sync_end(int dma_buf_fd)
+{
+	struct local_dma_buf_sync sync_end;
+
+	memset(&sync_end, 0, sizeof(sync_end));
+	sync_end.flags = LOCAL_DMA_BUF_SYNC_END | LOCAL_DMA_BUF_SYNC_RW;
+	do_ioctl(dma_buf_fd, LOCAL_DMA_BUF_IOCTL_SYNC, &sync_end);
+}
+
+/**
  * igt_require_fb_modifiers:
  * @fd: Open DRM file descriptor.
  *
lib/ioctl_wrappers.h +0 -0
index d3ffba2..cbd7a73 100644
@@ -148,6 +148,19 @@ void gem_require_caching(int fd);
 void gem_require_ring(int fd, int ring_id);
 

lib/ioctl_wrappers.h void gem_require_caching(int fd); · line 159

Quoted 10 lines
lib/ioctl_wrappers.h +9 -0
 /* prime */
+struct local_dma_buf_sync {
+	uint64_t flags;
+};
+
+#define LOCAL_DMA_BUF_SYNC_RW        (3 << 0)
+#define LOCAL_DMA_BUF_SYNC_START     (0 << 2)
+#define LOCAL_DMA_BUF_SYNC_END       (1 << 2)
+#define DMA_BUF_SYNC_VALID_FLAGS_MASK \
+		(DMA_BUF_SYNC_RW | DMA_BUF_SYNC_END)

What seems to be missing is negative ioclt tests, i.e. making sure invalid flags and stuff gets properly rejected.
-Daniel

Quoted 19 lines
lib/ioctl_wrappers.h +6 -0
+
+#define LOCAL_DMA_BUF_BASE 'b'
+#define LOCAL_DMA_BUF_IOCTL_SYNC _IOW(LOCAL_DMA_BUF_BASE, 0, struct local_dma_buf_sync)
+
 int prime_handle_to_fd(int fd, uint32_t handle);
 #ifndef DRM_RDWR
 #define DRM_RDWR O_RDWR
@@ -155,6 +168,8 @@ int prime_handle_to_fd(int fd, uint32_t handle);
 int prime_handle_to_fd_for_mmap(int fd, uint32_t handle);
 uint32_t prime_fd_to_handle(int fd, int dma_buf_fd);
 off_t prime_get_size(int dma_buf_fd);
+void prime_sync_start(int dma_buf_fd);
+void prime_sync_end(int dma_buf_fd);
 
 /* addfb2 fb modifiers */
 struct local_drm_mode_fb_cmd2 {
Signature · 1 line
2.1.4
Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel
#21 lore raw

PATCH v6 5/6 tests: Add kms_mmap_write_crc for cache coherency tests

In reply to Tiago Vignatti

This program can be used to detect when CPU writes in the dma-buf mapped object don't land in scanout due cache incoherency.

Although this seems a problem inherently of non-LCC machines ("Atom"), this particular test catches a cache dirt on scanout on LLC machines as well. It's inspired in Ville's kms_pwrite_crc.c and can be used also to test the correctness of the driver's begin_cpu_access and end_cpu_access (which requires i915 implementation.

To see the need for flush, one has to run this same binary a few times cause it's not 100% reproducible -- what I usually do is the following, using '-n' option to not call the sync ioctls:

    $ while ((1)) ; do ./kms_mmap_write_crc -n; done  # in terminal A
    $ find /                                          # in terminal B

That will most likely trashes the memory while the test will catch the coherency issue. If you now suppress '-n', then things should just work like expected.

I tested this with !llc and llc platforms, BTY and IVY respectively.

v2: use prime_handle_to_fd_for_mmap instead.
v3: merge end_cpu_access() patch with this and provide options to disable sync.
v4: use library's prime_sync_{start,end} instead.

---
 tests/Makefile.sources     |   1 +
 tests/kms_mmap_write_crc.c | 281 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 282 insertions(+)
 create mode 100644 tests/kms_mmap_write_crc.c
tests/Makefile.sources +1 -0
index 75f3cb0..ad2dd6a 100644
@@ -168,6 +168,7 @@ TESTS_progs = \
 	kms_3d \
 	kms_fence_pin_leak \
 	kms_force_connector_basic \
+	kms_mmap_write_crc \
 	kms_pwrite_crc \
 	kms_sink_crc_basic \
 	prime_udl \
tests/kms_mmap_write_crc.c new file +281 -0

281 lines not on this page · raw message

Signature · 6 lines
2.1.4

_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel
#22 lore raw

Re: [PATCH igt v6 5/6] tests: Add kms_mmap_write_crc for cache coherency tests

Quoted 9 lines · Tiago Vignatti

This program can be used to detect when CPU writes in the dma-buf mapped object don't land in scanout due cache incoherency.

Although this seems a problem inherently of non-LCC machines ("Atom"), this particular test catches a cache dirt on scanout on LLC machines as well. It's inspired in Ville's kms_pwrite_crc.c and can be used also to test the correctness of the driver's begin_cpu_access and end_cpu_access (which requires i915 implementation.

Quoted 6 lines

To see the need for flush, one has to run this same binary a few times cause it's not 100% reproducible -- what I usually do is the following, using '-n' option to not call the sync ioctls:

    $ while ((1)) ; do ./kms_mmap_write_crc -n; done  # in terminal A
    $ find /                                          # in terminal B

Sounds like we need a igt_fork_memhog_helper() and repeat the test for 20s? until faiure.
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel
#23 lore raw

PATCH v6 6/6 tests: Add prime_mmap_coherency for cache coherency tests

In reply to Tiago Vignatti

Different than kms_mmap_write_crc that captures the coherency issues within the scanout mapped buffer, this one is meant for test dma-buf mmap on !llc platforms mostly and provoke coherency bugs so we know where we need the sync ioctls.

I tested this with !llc and llc platforms, BTY and IVY respectively.

---
 tests/Makefile.sources       |   1 +
 tests/prime_mmap_coherency.c | 246 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 247 insertions(+)
 create mode 100644 tests/prime_mmap_coherency.c
tests/Makefile.sources +1 -0
index ad2dd6a..78605c6 100644
@@ -97,6 +97,7 @@ TESTS_progs_M = \
 	pm_rc6_residency \
 	pm_sseu \
 	prime_mmap \
+	prime_mmap_coherency \
 	prime_self_import \
 	template \
 	$(NULL)
tests/prime_mmap_coherency.c new file +246 -0

246 lines not on this page · raw message

Signature · 6 lines
2.1.4

_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel
#24 lore raw
In reply to Tiago Vignatti
Quoted 25 lines · Tiago Vignatti

Hi all,

The last version of this work was sent a while ago here:

http://lists.freedesktop.org/archives/dri-devel/2015-August/089263.html

So let's recap this series:

1. it adds a vendor-independent client interface for mapping gem objects through prime, IOW it implements userspace mmap() on dma-buf fd. This could be used for texturing from CPU rendered buffer, passing buffers among processes without performing copies in the userspace.
2. the series lets the client write on the mmap'ed memory, and
3. it deals with GPU and CPU caches synchronization.

Based on previous discussions seems that people are fine with 1. and 2. but not really with 3., given that caches coherency is a bit more boring to deal with.

It's easier to use this new infra on "coherent hardware" (systems with the memory cache that is shared by the GPU and CPU) because they rarely need to use that kind of synchronization. But would be much more convenient to have the very same interface exposed for clients no matter whether the underlying hardware is cache coherent or not.

Quoted 12 lines

One idea that came up was to force clients to call the sync ioctls after the dma-buf was mmaped. But apparently there's no easy, and performant, way to do so cause seems too costly to go over the page table entry and check the dirty bits. Also, depending on the instructions order sent for the devices, it might be needed a sync call after the mapped region gets accessed as well, to flush all cachelines and make sure for example the GPU domain won't read stale data. So that would make the things even more complicated, if we ever decide to go to this direction of forcing sync ioctls. The alternative therefore is to simply document it very well, strong wording the clients to use the sync ioctl regardless otherwise they will mis-behave. Do we have objections or maybe other wiser ways to circumvent this? I've made similar comments in August and no one has came up with better ideas.

I still think this is as good as it'll get. We can't force userspace to behave without a serious perf hit, and without enforcing it all the time there's not much use in it. Also there's the problem that mmap interfaces in the kernel don't really allow you (at least easily) to intercept mmap access.

It might make sense later on as a debug feature in case you do have a coherency bug and want to know who screwed up. Similar to what exists for the dma api.

Quickly looked through the patches and looks really nice. Especially the test coverage (including frontbuffer coherency checks for i915) is awesome. Imo as soon as we have an ack from chromium upstream that they're ok with this approach and your chromium patches, and after detailed code review is done this can go in. An ack from Thomas Hellstrom on the simplified coherency management interface would be good too.

Thanks, Daniel

Quoted 64 lines

Lastly, the diff of v6 series is that I've basically addressed concerns pointed in the igt tests, organized those changes better a bit (in smaller patches), documented the usage of sync ioctls and I have extensively tested this in different types of hardware.

https://github.com/tiagovignatti/drm-intel/commits/drm-intel-nightly_dma-buf-mmap-v6
https://github.com/tiagovignatti/intel-gpu-tools/commits/dma-buf-mmap-v6

Tiago

Daniel Thompson (1):
drm: prime: Honour O_RDWR during prime-handle-to-fd

Daniel Vetter (1):
dma-buf: Add ioctls to allow userspace to flush

Tiago Vignatti (3):
dma-buf: Remove range-based flush
drm/i915: Implement end_cpu_access
drm/i915: Use CPU mapping for userspace dma-buf mmap()

 Documentation/dma-buf-sharing.txt         | 41 +++++++++++++++-------
 drivers/dma-buf/dma-buf.c                 | 56 ++++++++++++++++++++++++++-----
 drivers/gpu/drm/drm_prime.c               | 10 ++----
 drivers/gpu/drm/i915/i915_gem_dmabuf.c    | 42 +++++++++++++++++++++--
 drivers/gpu/drm/omapdrm/omap_gem_dmabuf.c |  4 +--
 drivers/gpu/drm/udl/udl_fb.c              |  2 --
 drivers/staging/android/ion/ion.c         |  6 ++--
 drivers/staging/android/ion/ion_test.c    |  4 +--
 include/linux/dma-buf.h                   | 12 +++----
 include/uapi/drm/drm.h                    |  1 +
 include/uapi/linux/dma-buf.h              | 38 +++++++++++++++++++++
 11 files changed, 169 insertions(+), 47 deletions(-)
 create mode 100644 include/uapi/linux/dma-buf.h

And the igt changes:
Rob Bradford (1):
prime_mmap: Add new test for calling mmap() on dma-buf fds

Tiago Vignatti (5):
lib: Add gem_userptr and __gem_userptr helpers
prime_mmap: Add basic tests to write in a bo using CPU
lib: Add prime_sync_start and prime_sync_end helpers
tests: Add kms_mmap_write_crc for cache coherency tests
tests: Add prime_mmap_coherency for cache coherency tests

 benchmarks/gem_userptr_benchmark.c |  55 +----
 lib/ioctl_wrappers.c               |  92 +++++++
 lib/ioctl_wrappers.h               |  32 +++
 tests/Makefile.sources             |   3 +
 tests/gem_userptr_blits.c          | 104 ++------
 tests/kms_mmap_write_crc.c         | 281 +++++++++++++++++++++
 tests/prime_mmap.c                 | 494 +++++++++++++++++++++++++++++++++++++
 tests/prime_mmap_coherency.c       | 246 ++++++++++++++++++
 8 files changed, 1180 insertions(+), 127 deletions(-)
 create mode 100644 tests/kms_mmap_write_crc.c
 create mode 100644 tests/prime_mmap.c
 create mode 100644 tests/prime_mmap_coherency.c
Signature · 1 line
2.1.4
Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel