Skip to contentSkip to conversation

Reduce the time for which 'struct_mutex' is held

RFC Superseded [email protected] 3 participants 8 messages May 4, 2014 – Aug 6, 2015

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover Reduce the time for which 'struct_mutex' is held

From: Akash Goel <[email protected]>

We are trying to reduce the time for which the global 'struct_mutex' is locked. Execbuffer ioctl is one place where it is generally held for the longest time. And sometimes because of this occasional glitches/flickers are being observed in 60 fps playback (due to miss of V-blank intervals) as page flip call is getting delayed because the 'struct_mutex' is already locked.

Akash Goel (2):
drm/i915: Pre-allocation of shmem pages of a GEM object drm/i915: Moved the cache flush outside the 'struct_mutex' lock

 drivers/gpu/drm/i915/i915_debugfs.c        |  9 +++-
 drivers/gpu/drm/i915/i915_dma.c            |  1 +
 drivers/gpu/drm/i915/i915_drv.h            |  5 ++
 drivers/gpu/drm/i915/i915_gem.c            | 86 ++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 84 +++++++++++++++++++++++++----
 drivers/gpu/drm/i915/i915_trace.h          | 35 ++++++++++++
 6 files changed, 208 insertions(+), 12 deletions(-)
Signature · 1 line
1.9.2
#1 lore raw

RFC 1/2 drm/i915: Pre-allocation of shmem pages of a GEM object

From: Akash Goel <[email protected]>

This patch could help to reduce the time, 'struct_mutex' is kept locked during either the exec-buffer path or Page fault handling path as now the backing pages are requested from shmem layer without holding the 'struct_mutex'.

---
 drivers/gpu/drm/i915/i915_debugfs.c        |  9 +++-
 drivers/gpu/drm/i915/i915_dma.c            |  1 +
 drivers/gpu/drm/i915/i915_drv.h            |  5 ++
 drivers/gpu/drm/i915/i915_gem.c            | 75 ++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 84 ++++++++++++++++++++++++++----
 drivers/gpu/drm/i915/i915_trace.h          | 35 +++++++++++++
 6 files changed, 197 insertions(+), 12 deletions(-)
drivers/gpu/drm/i915/i915_debugfs.c +8 -1
index 18b3565..70c752b 100644
@@ -247,10 +247,16 @@ static int i915_gem_stolen_list_info(struct seq_file *m, void *data)
 	LIST_HEAD(stolen);
 	int count, ret;
 
-	ret = mutex_lock_interruptible(&dev->struct_mutex);
+	ret = mutex_lock_interruptible(&dev_priv->exec_lock);
 	if (ret)
 		return ret;
 
+	ret = mutex_lock_interruptible(&dev->struct_mutex);
+	if (ret) {
+		mutex_unlock(&dev_priv->exec_lock);
+		return ret;
+	}
+
 	total_obj_size = total_gtt_size = count = 0;
 	list_for_each_entry(obj, &dev_priv->mm.bound_list, global_list) {
 		if (obj->stolen == NULL)
@@ -281,6 +287,7 @@ static int i915_gem_stolen_list_info(struct seq_file *m, void *data)
 		list_del_init(&obj->obj_exec_link);
 	}
 	mutex_unlock(&dev->struct_mutex);
+	mutex_unlock(&dev_priv->exec_lock);
 
 	seq_printf(m, "Total %d objects, %zu bytes, %zu GTT size\n",
 		   count, total_obj_size, total_gtt_size);
drivers/gpu/drm/i915/i915_dma.c +1 -0
index e393a14..e4d1cb0 100644
@@ -1572,6 +1572,7 @@ int i915_driver_load(struct drm_device *dev, unsigned long flags)
 	dev_priv->ring_index = 0;
 	mutex_init(&dev_priv->dpio_lock);
 	mutex_init(&dev_priv->modeset_restore_lock);
+	mutex_init(&dev_priv->exec_lock);
 
 	intel_pm_setup(dev);
 
drivers/gpu/drm/i915/i915_drv.h +5 -0
index 5f4f631..6dc579a 100644
@@ -1475,6 +1475,8 @@ struct drm_i915_private {
 	struct i915_ums_state ums;
 	/* the indicator for dispatch video commands on two BSD rings */
 	int ring_index;
+	/* for concurrent execbuffer protection */
+	struct mutex exec_lock;
 };
 
 static inline struct drm_i915_private *to_i915(const struct drm_device *dev)
@@ -2116,6 +2118,9 @@ int i915_gem_dumb_create(struct drm_file *file_priv,
 			 struct drm_mode_create_dumb *args);
 int i915_gem_mmap_gtt(struct drm_file *file_priv, struct drm_device *dev,
 		      uint32_t handle, uint64_t *offset);
+void
+i915_gem_object_shmem_preallocate(struct drm_i915_gem_object *obj);
+
 /**
  * Returns true if seq1 is later than seq2.
  */
drivers/gpu/drm/i915/i915_gem.c +75 -0

87 lines not on this page · raw message

drivers/gpu/drm/i915/i915_gem_execbuffer.c +73 -11

179 lines not on this page · raw message

drivers/gpu/drm/i915/i915_trace.h +35 -0
index b29d7b1..21bf10d 100644
@@ -245,6 +245,41 @@ TRACE_EVENT(i915_gem_object_fault,
 		      __entry->write ? ", writable" : "")
 );
 
+TRACE_EVENT(i915_gem_obj_prealloc_start,
+	    TP_PROTO(struct drm_i915_gem_object *obj, u32 size),
+	    TP_ARGS(obj, size),
+
+	    TP_STRUCT__entry(
+			     __field(struct drm_i915_gem_object *, obj)
+			     __field(u32, size)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->obj = obj;
+			   __entry->size = size;
+			   ),
+
+	    TP_printk("obj=%p, size=%x",
+		      __entry->obj,
+		      __entry->size)
+);
+
+TRACE_EVENT(i915_gem_obj_prealloc_end,
+	    TP_PROTO(struct drm_i915_gem_object *obj),
+	    TP_ARGS(obj),
+
+	    TP_STRUCT__entry(
+			     __field(struct drm_i915_gem_object *, obj)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->obj = obj;
+			   ),
+
+	    TP_printk("obj=%p",
+		      __entry->obj)
+);
+
 DECLARE_EVENT_CLASS(i915_gem_object,
 	    TP_PROTO(struct drm_i915_gem_object *obj),
 	    TP_ARGS(obj),
Signature · 1 line
1.9.2
#2 lore raw

Re: [RFC 1/2] drm/i915: Pre-allocation of shmem pages of a GEM object

Quoted 6 lines · On Sun, May 04, 2014 at 04:48:24PM +0530, [email protected] wrote:

From: Akash Goel <[email protected]>

This patch could help to reduce the time, 'struct_mutex' is kept locked during either the exec-buffer path or Page fault handling path as now the backing pages are requested from shmem layer without holding the 'struct_mutex'.

I'm not keen on having an extra pass inside execbuffer for what should be relatively rare, but in discussion the idea of adding a flag to create2 (I915_CREATE_POPULATE) should cover the usecase nicely (having to pay the shmemfs allocation+zero price without holding struct_mutex).

drivers/gpu/drm/i915/i915_gem.c i915_gem_shrink_all(struct drm_i915_private *dev_priv) · line 1879

Quoted 2 lines
drivers/gpu/drm/i915/i915_gem.c +2 -0
+void
+i915_gem_object_shmem_preallocate(struct drm_i915_gem_object *obj)

This is unlocked. We have to be loud and clear about that

static void __i915_gem_object_shmemfs_populate(obj)

drivers/gpu/drm/i915/i915_gem.c i915_gem_shrink_all(struct drm_i915_private *dev_priv) · line 1887

Quoted 8 lines
drivers/gpu/drm/i915/i915_gem.c +8 -0
+{
+	int page_count, i;
+	struct address_space *mapping;
+	struct page *page;
+	gfp_t gfp;
+
+	if (obj->pages)
+		return;

if (READ_ONCE(obj->pages) return;

drivers/gpu/drm/i915/i915_gem.c i915_gem_shrink_all(struct drm_i915_private *dev_priv) · line 1890

Quoted 2 lines
drivers/gpu/drm/i915/i915_gem.c +2 -0
+	if (obj->madv != I915_MADV_WILLNEED) {
+		DRM_ERROR("Attempting to preallocate a purgeable object\n");

Ideally if (READ_ONCE(obj->madv)...

However, that requires a patch to move madv to unsigned obj->flags.

A user controllable *ERROR*? No, just make it debug.

drivers/gpu/drm/i915/i915_gem.c i915_gem_shrink_all(struct drm_i915_private *dev_priv) · line 1899

Quoted 9 lines
drivers/gpu/drm/i915/i915_gem.c +9 -0
+		return;
+	}
+
+	if (obj->base.filp) {
+		int ret;
+		struct inode *inode = file_inode(obj->base.filp);
+		struct shmem_inode_info *info = SHMEM_I(inode);
+		if (!inode)
+			return;

Assume it exists. If it doesn't it is an internal bug so we should just GPF out of here.

drivers/gpu/drm/i915/i915_gem.c i915_gem_shrink_all(struct drm_i915_private *dev_priv) · line 1910

Quoted 11 lines
drivers/gpu/drm/i915/i915_gem.c +11 -0
+		/* The alloced field stores how many data pages are allocated
+		 * to the file. If already shmem space has been allocated for
+		 * the object, then we can simply return */
+		spin_lock(&info->lock);
+		ret = info->alloced;
+		spin_unlock(&info->lock);
+		if (ret > 0) {
+			DRM_DEBUG_DRIVER("Already shmem space alloced for obj %p, %d pages\n",
+					obj, ret);
+			return;
+		}

I'm convinced this is of much use though. If the page is already allocated the shmem_read_mapping_page_gfp should be quick. Given the suggestion that we move this to create, it is clear that this won't be an issue.

drivers/gpu/drm/i915/i915_gem.c i915_gem_shrink_all(struct drm_i915_private *dev_priv) · line 1914

Quoted 4 lines
drivers/gpu/drm/i915/i915_gem.c +4 -0
+	} else
+		return;
+
+	BUG_ON(obj->pages_pin_count);

Requires struct_mutex; ignore.

drivers/gpu/drm/i915/i915_gem.c i915_gem_shrink_all(struct drm_i915_private *dev_priv) · line 1921

Quoted 7 lines
drivers/gpu/drm/i915/i915_gem.c +7 -0
+
+	/* Assert that the object is not currently in any GPU domain. As it
+	 * wasn't in the GTT, there shouldn't be any way it could have been in
+	 * a GPU cache
+	 */
+	BUG_ON(obj->base.read_domains & I915_GEM_GPU_DOMAINS);
+	BUG_ON(obj->base.write_domain & I915_GEM_GPU_DOMAINS);

Requires struct_mutex; ignore.

drivers/gpu/drm/i915/i915_gem.c i915_gem_shrink_all(struct drm_i915_private *dev_priv) · line 1923

Quoted 1 line
drivers/gpu/drm/i915/i915_gem.c +1 -0
+	trace_i915_gem_obj_prealloc_start(obj, obj->base.size);

You only need to pass obj to the tracer, it can work out the size itself.

drivers/gpu/drm/i915/i915_gem.c i915_gem_shrink_all(struct drm_i915_private *dev_priv) · line 1933

Quoted 9 lines
drivers/gpu/drm/i915/i915_gem.c +9 -0
+	page_count = obj->base.size / PAGE_SIZE;
+
+	/* Get the list of pages out of our struct file
+	 * Fail silently without starting the shrinker
+	 */
+	mapping = file_inode(obj->base.filp)->i_mapping;
+	gfp = mapping_gfp_mask(mapping);
+	gfp |= __GFP_NORETRY | __GFP_NOWARN | __GFP_NO_KSWAPD;
+	gfp &= ~(__GFP_IO | __GFP_WAIT);

Interesting. Disabling shrinker/oom for these pages - we will hit it later of course. But for the purposes of a quick preallocation, seems fine.

Quoted 6 lines
drivers/gpu/drm/i915/i915_gem.c +6 -0
+	for (i = 0; i < page_count; i++) {
+		page = shmem_read_mapping_page_gfp(mapping, i, gfp);
+		if (IS_ERR(page)) {
+			DRM_DEBUG_DRIVER("Failure for obj(%p), size(%x) at page(%d)\n",
+						obj, obj->base.size, i);
+			break;

drivers/gpu/drm/i915/i915_gem.c i915_gem_shrink_all(struct drm_i915_private *dev_priv) · line 1949

Quoted 10 lines
drivers/gpu/drm/i915/i915_gem.c +10 -0
+		}
+		/* Decrement the extra ref count on the returned page,
+		   otherwise when 'get_pages_gtt' will be called later on
+		   in the regular path, it will also increment the ref count,
+		   which will disturb the ref count management */
+		page_cache_release(page);
+	}
+
+	trace_i915_gem_obj_prealloc_end(obj);
+}

-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#3 lore raw

RFC 2/2 drm/i915: Moved the cache flush outside the 'struct_mutex' lock

In reply to [email protected]

From: Akash Goel <[email protected]>

Moved the cache flush of the preallocated shmem pages outside the span of 'struct_mutex' lock. This shall not lead to any redundancy as the cache flush of the newly allocated pages will be done anyways when same buffer is submitted to GPU or when the domain of the object is changed from CPU to GTT in Gem page fault handler.

---
 drivers/gpu/drm/i915/i915_gem.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_gem.c +12 -1
index b19ccb8..7ab2635 100644
@@ -1936,8 +1936,11 @@ i915_gem_object_shmem_preallocate(struct drm_i915_gem_object *obj)
 		if (IS_ERR(page)) {
 			DRM_DEBUG_DRIVER("Failure for obj(%p), size(%x) at page(%d)\n",
 						obj, obj->base.size, i);
-			break;
+			return;
 		}
+		/* Flush the cpu cache for the page now itself */
+		drm_clflush_pages(&page, 1);
+
 		/* Decrement the extra ref count on the returned page,
 		   otherwise when 'get_pages_gtt' will be called later on
 		   in the regular path, it will also increment the ref count,
@@ -1945,6 +1948,14 @@ i915_gem_object_shmem_preallocate(struct drm_i915_gem_object *obj)
 		page_cache_release(page);
 	}
 
+	/*
+	 * Reset the CPU domain now itself, so as to avoid the cache
+	 * flush later (under 'struct_mutex' lock), as the all pages
+	 * have been cache flushed.
+	 * Hope this is safe enough to be done here.
+	 */
+	obj->base.write_domain = 0;
+
 	trace_i915_gem_obj_prealloc_end(obj);
 }
 
Signature · 1 line
1.9.2
#4 lore raw

Re: [RFC 2/2] drm/i915: Moved the cache flush outside the 'struct_mutex' lock

Quoted 32 lines · On Sun, May 04, 2014 at 04:48:25PM +0530, [email protected] wrote:

From: Akash Goel <[email protected]>

Moved the cache flush of the preallocated shmem pages outside the span of 'struct_mutex' lock. This shall not lead to any redundancy as the cache flush of the newly allocated pages will be done anyways when same buffer is submitted to GPU or when the domain of the object is changed from CPU to GTT in Gem page fault handler.

---
 drivers/gpu/drm/i915/i915_gem.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_gem.c +4 -1
index b19ccb8..7ab2635 100644
@@ -1936,8 +1936,11 @@ i915_gem_object_shmem_preallocate(struct drm_i915_gem_object *obj)
 		if (IS_ERR(page)) {
 			DRM_DEBUG_DRIVER("Failure for obj(%p), size(%x) at page(%d)\n",
 						obj, obj->base.size, i);
-			break;
+			return;
 		}
+		/* Flush the cpu cache for the page now itself */
+		drm_clflush_pages(&page, 1);
+
 		/* Decrement the extra ref count on the returned page,
 		   otherwise when 'get_pages_gtt' will be called later on
 		   in the regular path, it will also increment the ref count,

drivers/gpu/drm/i915/i915_gem.c i915_gem_object_shmem_preallocate(struct drm_i915_gem_object *obj) · line 1957

Quoted 10 lines
drivers/gpu/drm/i915/i915_gem.c +7 -0
@@ -1945,6 +1948,14 @@ i915_gem_object_shmem_preallocate(struct drm_i915_gem_object *obj)
 		page_cache_release(page);
 	}
 
+	/*
+	 * Reset the CPU domain now itself, so as to avoid the cache
+	 * flush later (under 'struct_mutex' lock), as the all pages
+	 * have been cache flushed.
+	 * Hope this is safe enough to be done here.
+	 */
+	obj->base.write_domain = 0;

So cache-domain-management is under the struct_mutex, for good reason. However, if we follow the idea to move this to create2, then adding a flag to also do the clflush upon allocation seems fine. Note that I don't think doing it by default is a good match for everything, since my code and also the kernel already takes advantage of the fact that fresh buffers are in the CPU cache domain and uses that to do any initial transfers in the CPU domain.

So we have I915_CREATE_POPULATE, I915_CREATE_COHERENT ? -Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#5 lore raw
In reply to [email protected]
Quoted 8 lines · On Sun, May 04, 2014 at 04:48:23PM +0530, [email protected] wrote:

From: Akash Goel <[email protected]>

We are trying to reduce the time for which the global 'struct_mutex' is locked. Execbuffer ioctl is one place where it is generally held for the longest time. And sometimes because of this occasional glitches/flickers are being observed in 60 fps playback (due to miss of V-blank intervals) as page flip call is getting delayed because the 'struct_mutex' is already locked.

Once again, both of these make me wonder just what you are doing so badly wrong in userspace.
-Chris

Signature · 1 line
Chris Wilson, Intel Open Source Technology Centre
#6 lore raw

There could be something amiss on the User space side, but this is a very sporadic issue. It mainly comes to the fore, when some huge buffers (30 MB) are being processed.

Best Regards
Akash

-----Original Message-----
From: Chris Wilson [mailto:[email protected]]
Sent: Sunday, May 04, 2014 5:12 PM
To: Goel, Akash
Cc: [email protected]; Gupta, Sourab
Subject: Re: [Intel-gfx] [RFC 0/2] Reduce the time for which 'struct_mutex' is held

Quoted 8 lines · On Sun, May 04, 2014 at 04:48:23PM +0530, [email protected] wrote:

From: Akash Goel <[email protected]>

We are trying to reduce the time for which the global 'struct_mutex' is locked. Execbuffer ioctl is one place where it is generally held for the longest time. And sometimes because of this occasional glitches/flickers are being observed in 60 fps playback (due to miss of V-blank intervals) as page flip call is getting delayed because the 'struct_mutex' is already locked.

Once again, both of these make me wonder just what you are doing so badly wrong in userspace. -Chris

--
Chris Wilson, Intel Open Source Technology Centre

#7 lore raw
In reply to [email protected]
Quoted 8 lines · On Sun, May 04, 2014 at 04:48:23PM +0530, [email protected] wrote:

From: Akash Goel <[email protected]>

We are trying to reduce the time for which the global 'struct_mutex' is locked. Execbuffer ioctl is one place where it is generally held for the longest time. And sometimes because of this occasional glitches/flickers are being observed in 60 fps playback (due to miss of V-blank intervals) as page flip call is getting delayed because the 'struct_mutex' is already locked.

This is the point where working around our single-lock-to-rule-them-all scheme is no longer sustainable. The first patch might be safe but definitely scares the heck out of me, the 2nd one is definitely racy.

The right solution here is to work on per-object locking, as Chris suggest in another mail already. This is a major task. Major = overall effort likely exceeding man-years.

Also I need to block this on full ppgtt getting stable since we have a hard time building castles on sands, recursing that will be unmanageble.

Cheers, Daniel

Signature · 3 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch