Reduce the time for which 'struct_mutex' is held
Subsystems (2)
cover Reduce the time for which 'struct_mutex' is held
From: Akash Goel <[email protected]>
We are trying to reduce the time for which the global 'struct_mutex' is locked. Execbuffer ioctl is one place where it is generally held for the longest time. And sometimes because of this occasional glitches/flickers are being observed in 60 fps playback (due to miss of V-blank intervals) as page flip call is getting delayed because the 'struct_mutex' is already locked.
Akash Goel (2):
drm/i915: Pre-allocation of shmem pages of a GEM object drm/i915: Moved the cache flush outside the 'struct_mutex' lock
drivers/gpu/drm/i915/i915_debugfs.c | 9 +++- drivers/gpu/drm/i915/i915_dma.c | 1 + drivers/gpu/drm/i915/i915_drv.h | 5 ++ drivers/gpu/drm/i915/i915_gem.c | 86 ++++++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_gem_execbuffer.c | 84 +++++++++++++++++++++++++---- drivers/gpu/drm/i915/i915_trace.h | 35 ++++++++++++ 6 files changed, 208 insertions(+), 12 deletions(-)
Signature · 1 line
1.9.2
RFC 1/2 drm/i915: Pre-allocation of shmem pages of a GEM object
From: Akash Goel <[email protected]>
This patch could help to reduce the time, 'struct_mutex' is kept locked during either the exec-buffer path or Page fault handling path as now the backing pages are requested from shmem layer without holding the 'struct_mutex'.
--- drivers/gpu/drm/i915/i915_debugfs.c | 9 +++- drivers/gpu/drm/i915/i915_dma.c | 1 + drivers/gpu/drm/i915/i915_drv.h | 5 ++ drivers/gpu/drm/i915/i915_gem.c | 75 ++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_gem_execbuffer.c | 84 ++++++++++++++++++++++++++---- drivers/gpu/drm/i915/i915_trace.h | 35 +++++++++++++ 6 files changed, 197 insertions(+), 12 deletions(-)
drivers/gpu/drm/i915/i915_debugfs.c +8 -1
@@ -247,10 +247,16 @@ static int i915_gem_stolen_list_info(struct seq_file *m, void *data)LIST_HEAD (stolen);int count, ret; - ret =mutex_lock_interruptible (&dev->struct_mutex); + ret =mutex_lock_interruptible (&dev_priv->exec_lock);if (ret)return ret; + ret =mutex_lock_interruptible (&dev->struct_mutex); +if (ret) { +mutex_unlock (&dev_priv->exec_lock); +return ret; + } + total_obj_size = total_gtt_size = count =0 ;list_for_each_entry (obj, &dev_priv->mm.bound_list, global_list) {if (obj->stolen ==NULL )
@@ -281,6 +287,7 @@ static int i915_gem_stolen_list_info(struct seq_file *m, void *data)list_del_init (&obj->obj_exec_link); }mutex_unlock (&dev->struct_mutex); +mutex_unlock (&dev_priv->exec_lock);seq_printf (m,"Total %d objects, %zu bytes, %zu GTT size\n" , count, total_obj_size, total_gtt_size);
drivers/gpu/drm/i915/i915_dma.c +1 -0
@@ -1572,6 +1572,7 @@ int i915_driver_load(struct drm_device *dev, unsigned long flags) dev_priv->ring_index =0 ;mutex_init (&dev_priv->dpio_lock);mutex_init (&dev_priv->modeset_restore_lock); +mutex_init (&dev_priv->exec_lock);intel_pm_setup (dev);
drivers/gpu/drm/i915/i915_drv.h +5 -0
@@ -1475,6 +1475,8 @@ struct drm_i915_private {struct i915_ums_state ums;/* the indicator for dispatch video commands on two BSD rings */ int ring_index; +/* for concurrent execbuffer protection */ +struct mutex exec_lock; };static inline struct drm_i915_private *to_i915 (const struct drm_device *dev)
@@ -2116,6 +2118,9 @@ int i915_gem_dumb_create(struct drm_file *file_priv, struct drm_mode_create_dumb *args);int i915_gem_mmap_gtt (struct drm_file *file_priv,struct drm_device *dev,uint32_t handle,uint64_t *offset); +void +i915_gem_object_shmem_preallocate (struct drm_i915_gem_object *obj); +/** * Returns true if seq1 is later than seq2. */
drivers/gpu/drm/i915/i915_gem.c +75 -0
87 lines not on this page · raw message
drivers/gpu/drm/i915/i915_gem_execbuffer.c +73 -11
179 lines not on this page · raw message
drivers/gpu/drm/i915/i915_trace.h +35 -0
@@ -245,6 +245,41 @@ TRACE_EVENT(i915_gem_object_fault, __entry->write ? ", writable" : "") ); +TRACE_EVENT (i915_gem_obj_prealloc_start, + TP_PROTO(struct drm_i915_gem_object *obj, u32 size), +TP_ARGS (obj, size), + +TP_STRUCT__entry ( + __field(struct drm_i915_gem_object *, obj) +__field (u32, size) + ), + +TP_fast_assign ( + __entry->obj = obj; + __entry->size = size; + ), + +TP_printk ("obj=%p, size=%x" , + __entry->obj, + __entry->size) +); + +TRACE_EVENT (i915_gem_obj_prealloc_end, + TP_PROTO(struct drm_i915_gem_object *obj), +TP_ARGS (obj), + +TP_STRUCT__entry ( + __field(struct drm_i915_gem_object *, obj) + ), + +TP_fast_assign ( + __entry->obj = obj; + ), + +TP_printk ("obj=%p" , + __entry->obj) +); + DECLARE_EVENT_CLASS(i915_gem_object, TP_PROTO(struct drm_i915_gem_object *obj), TP_ARGS(obj),
Signature · 1 line
1.9.2
Re: [RFC 1/2] drm/i915: Pre-allocation of shmem pages of a GEM object
Quoted 6 lines · On Sun, May 04, 2014 at 04:48:24PM +0530, [email protected] wrote:
From: Akash Goel <[email protected]>
This patch could help to reduce the time, '
struct_mutex' is kept locked during either the exec-buffer path or Page fault handling path as now the backing pages are requested from shmem layer without holding the 'struct_mutex'.
I'm not keen on having an extra pass inside execbuffer for what should be relatively rare, but in discussion the idea of adding a flag to create2 (I915_CREATE_POPULATE) should cover the usecase nicely (having to pay the shmemfs allocation+zero price without holding struct_mutex).
drivers/gpu/drm/i915/i915_gem.c i915_gem_shrink_all(struct drm_i915_private *dev_priv) · line 1879
Quoted 2 lines
drivers/gpu/drm/i915/i915_gem.c +2
-0+void +i915_gem_object_shmem_preallocate (struct drm_i915_gem_object *obj)
This is unlocked. We have to be loud and clear about that
static void __i915_gem_object_shmemfs_populate(obj)
drivers/gpu/drm/i915/i915_gem.c i915_gem_shrink_all(struct drm_i915_private *dev_priv) · line 1887
Quoted 8 lines
drivers/gpu/drm/i915/i915_gem.c +8
-0+{ +int page_count, i; +struct address_space *mapping; +struct page *page; +gfp_t gfp; + +if (obj->pages) +return ;
if (READ_ONCE(obj->pages) return;
drivers/gpu/drm/i915/i915_gem.c i915_gem_shrink_all(struct drm_i915_private *dev_priv) · line 1890
Quoted 2 lines
drivers/gpu/drm/i915/i915_gem.c +2
-0+if (obj->madv !=I915_MADV_WILLNEED ) { +DRM_ERROR ("Attempting to preallocate a purgeable object\n" );
Ideally if (READ_ONCE(obj->madv)...
However, that requires a patch to move madv to unsigned obj->flags.
A user controllable *ERROR*? No, just make it debug.
drivers/gpu/drm/i915/i915_gem.c i915_gem_shrink_all(struct drm_i915_private *dev_priv) · line 1899
Quoted 9 lines
drivers/gpu/drm/i915/i915_gem.c +9
-0+return ; + } + +if (obj->base.filp) { +int ret; +struct inode *inode =file_inode (obj->base.filp); +struct shmem_inode_info *info =SHMEM_I (inode); +if (!inode) +return ;
Assume it exists. If it doesn't it is an internal bug so we should just GPF out of here.
drivers/gpu/drm/i915/i915_gem.c i915_gem_shrink_all(struct drm_i915_private *dev_priv) · line 1910
Quoted 11 lines
drivers/gpu/drm/i915/i915_gem.c +11
-0+/* The alloced field stores how many data pages are allocated +* to the file. If already shmem space has been allocated for +* the object, then we can simply return */ +spin_lock (&info->lock); + ret = info->alloced; +spin_unlock (&info->lock); +if (ret >0 ) { +DRM_DEBUG_DRIVER ("Already shmem space alloced for obj %p, %d pages\n" , + obj, ret); +return ; + }
I'm convinced this is of much use though. If the page is already allocated the shmem_read_mapping_page_gfp should be quick. Given the suggestion that we move this to create, it is clear that this won't be an issue.
drivers/gpu/drm/i915/i915_gem.c i915_gem_shrink_all(struct drm_i915_private *dev_priv) · line 1914
Quoted 4 lines
drivers/gpu/drm/i915/i915_gem.c +4
-0+ }else +return ; + +BUG_ON (obj->pages_pin_count);
Requires struct_mutex; ignore.
drivers/gpu/drm/i915/i915_gem.c i915_gem_shrink_all(struct drm_i915_private *dev_priv) · line 1921
Quoted 7 lines
drivers/gpu/drm/i915/i915_gem.c +7
-0+ +/* Assert that the object is not currently in any GPU domain. As it +* wasn't in the GTT, there shouldn't be any way it could have been in +* a GPU cache +*/ +BUG_ON (obj->base.read_domains &I915_GEM_GPU_DOMAINS ); +BUG_ON (obj->base.write_domain &I915_GEM_GPU_DOMAINS );
Requires struct_mutex; ignore.
drivers/gpu/drm/i915/i915_gem.c i915_gem_shrink_all(struct drm_i915_private *dev_priv) · line 1923
Quoted 1 line
drivers/gpu/drm/i915/i915_gem.c +1
-0+trace_i915_gem_obj_prealloc_start (obj, obj->base.size);
You only need to pass obj to the tracer, it can work out the size itself.
drivers/gpu/drm/i915/i915_gem.c i915_gem_shrink_all(struct drm_i915_private *dev_priv) · line 1933
Quoted 9 lines
drivers/gpu/drm/i915/i915_gem.c +9
-0+ page_count = obj->base.size /PAGE_SIZE ; + +/* Get the list of pages out of our struct file +* Fail silently without starting the shrinker +*/ + mapping =file_inode (obj->base.filp)->i_mapping; + gfp =mapping_gfp_mask (mapping); + gfp |= __GFP_NORETRY | __GFP_NOWARN | __GFP_NO_KSWAPD; + gfp &= ~(__GFP_IO | __GFP_WAIT);
Interesting. Disabling shrinker/oom for these pages - we will hit it later of course. But for the purposes of a quick preallocation, seems fine.
Quoted 6 lines
drivers/gpu/drm/i915/i915_gem.c +6
-0+for (i =0 ; i < page_count; i++) { + page =shmem_read_mapping_page_gfp (mapping, i, gfp); +if (IS_ERR (page)) { +DRM_DEBUG_DRIVER ("Failure for obj(%p), size(%x) at page(%d)\n" , + obj, obj->base.size, i); +break ;
drivers/gpu/drm/i915/i915_gem.c i915_gem_shrink_all(struct drm_i915_private *dev_priv) · line 1949
Quoted 10 lines
drivers/gpu/drm/i915/i915_gem.c +10
-0+ } +/* Decrement the extra ref count on the returned page, +otherwise when 'get_pages_gtt' will be called later on +in the regular path, it will also increment the ref count, +which will disturb the ref count management */ +page_cache_release (page); + } + +trace_i915_gem_obj_prealloc_end (obj); +}
-Chris
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/intel-gfx
RFC 2/2 drm/i915: Moved the cache flush outside the 'struct_mutex' lock
In reply to [email protected]From: Akash Goel <[email protected]>
Moved the cache flush of the preallocated shmem pages outside the span of 'struct_mutex' lock. This shall not lead to any redundancy as the cache flush of the newly allocated pages will be done anyways when same buffer is submitted to GPU or when the domain of the object is changed from CPU to GTT in Gem page fault handler.
--- drivers/gpu/drm/i915/i915_gem.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_gem.c +12 -1
@@ -1936,8 +1936,11 @@ i915_gem_object_shmem_preallocate(struct drm_i915_gem_object *obj)if (IS_ERR (page)) {DRM_DEBUG_DRIVER ("Failure for obj(%p), size(%x) at page(%d)\n" , obj, obj->base.size, i); -break ; +return ; } +/* Flush the cpu cache for the page now itself */ +drm_clflush_pages (&page,1 ); +/* Decrement the extra ref count on the returned page, otherwise when 'get_pages_gtt' will be called later on in the regular path, it will also increment the ref count,
@@ -1945,6 +1948,14 @@ i915_gem_object_shmem_preallocate(struct drm_i915_gem_object *obj)page_cache_release (page); } +/* +* Reset the CPU domain now itself, so as to avoid the cache +* flush later (under 'struct_mutex' lock), as the all pages +* have been cache flushed. +* Hope this is safe enough to be done here. +*/ + obj->base.write_domain =0 ; +trace_i915_gem_obj_prealloc_end (obj); }
Signature · 1 line
1.9.2
Re: [RFC 2/2] drm/i915: Moved the cache flush outside the 'struct_mutex' lock
Quoted 32 lines · On Sun, May 04, 2014 at 04:48:25PM +0530, [email protected] wrote:
From: Akash Goel <[email protected]>
Moved the cache flush of the preallocated shmem pages outside the span of '
struct_mutex' lock. This shall not lead to any redundancy as the cache flush of the newly allocated pages will be done anyways when same buffer is submitted to GPU or when the domain of the object is changed from CPU to GTT in Gem page fault handler.Signed-off-by Akash Goel <[email protected]>--- drivers/gpu/drm/i915/i915_gem.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)drivers/gpu/drm/i915/i915_gem.c +4
-1@@ -1936,8 +1936,11 @@ i915_gem_object_shmem_preallocate(struct drm_i915_gem_object *obj)if (IS_ERR (page)) {DRM_DEBUG_DRIVER ("Failure for obj(%p), size(%x) at page(%d)\n" , obj, obj->base.size, i); -break ; +return ; } +/* Flush the cpu cache for the page now itself */ +drm_clflush_pages (&page,1 ); +/* Decrement the extra ref count on the returned page, otherwise when 'get_pages_gtt' will be called later on in the regular path, it will also increment the ref count,
drivers/gpu/drm/i915/i915_gem.c i915_gem_object_shmem_preallocate(struct drm_i915_gem_object *obj) · line 1957
Quoted 10 lines
drivers/gpu/drm/i915/i915_gem.c +7
-0@@ -1945,6 +1948,14 @@ i915_gem_object_shmem_preallocate(struct drm_i915_gem_object *obj)page_cache_release (page); } +/* +* Reset the CPU domain now itself, so as to avoid the cache +* flush later (under 'struct_mutex' lock), as the all pages +* have been cache flushed. +* Hope this is safe enough to be done here. +*/ + obj->base.write_domain =0 ;
So cache-domain-management is under the struct_mutex, for good reason. However, if we follow the idea to move this to create2, then adding a flag to also do the clflush upon allocation seems fine. Note that I don't think doing it by default is a good match for everything, since my code and also the kernel already takes advantage of the fact that fresh buffers are in the CPU cache domain and uses that to do any initial transfers in the CPU domain.
So we have I915_CREATE_POPULATE, I915_CREATE_COHERENT ? -Chris
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 8 lines · On Sun, May 04, 2014 at 04:48:23PM +0530, [email protected] wrote:
From: Akash Goel <[email protected]>
We are trying to reduce the time for which the global '
struct_mutex' is locked. Execbuffer ioctl is one place where it is generally held for the longest time. And sometimes because of this occasional glitches/flickers are being observed in 60 fps playback (due to miss of V-blank intervals) as page flip call is getting delayed because the 'struct_mutex' is already locked.
Once again, both of these make me wonder just what you are doing so badly wrong in userspace.
-Chris
Signature · 1 line
Chris Wilson, Intel Open Source Technology Centre
There could be something amiss on the User space side, but this is a very sporadic issue. It mainly comes to the fore, when some huge buffers (30 MB) are being processed.
Best Regards
Akash
-----Original Message-----
From: Chris Wilson [mailto:[email protected]]
Sent: Sunday, May 04, 2014 5:12 PM
To: Goel, Akash
Cc: [email protected]; Gupta, Sourab
Subject: Re: [Intel-gfx] [RFC 0/2] Reduce the time for which 'struct_mutex' is held
Quoted 8 lines · On Sun, May 04, 2014 at 04:48:23PM +0530, [email protected] wrote:
From: Akash Goel <[email protected]>
We are trying to reduce the time for which the global '
struct_mutex' is locked. Execbuffer ioctl is one place where it is generally held for the longest time. And sometimes because of this occasional glitches/flickers are being observed in 60 fps playback (due to miss of V-blank intervals) as page flip call is getting delayed because the 'struct_mutex' is already locked.
Once again, both of these make me wonder just what you are doing so badly wrong in userspace. -Chris
--
Chris Wilson, Intel Open Source Technology Centre
Quoted 8 lines · On Sun, May 04, 2014 at 04:48:23PM +0530, [email protected] wrote:
From: Akash Goel <[email protected]>
We are trying to reduce the time for which the global '
struct_mutex' is locked. Execbuffer ioctl is one place where it is generally held for the longest time. And sometimes because of this occasional glitches/flickers are being observed in 60 fps playback (due to miss of V-blank intervals) as page flip call is getting delayed because the 'struct_mutex' is already locked.
This is the point where working around our single-lock-to-rule-them-all scheme is no longer sustainable. The first patch might be safe but definitely scares the heck out of me, the 2nd one is definitely racy.
The right solution here is to work on per-object locking, as Chris suggest in another mail already. This is a major task. Major = overall effort likely exceeding man-years.
Also I need to block this on full ppgtt getting stable since we have a hard time building castles on sands, recursing that will be unmanageble.
Cheers, Daniel
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch