GPU scheduler for i915 driver
Subsystems (4)
v5 cover GPU scheduler for i915 driver
From: John Harrison <[email protected]>
Implemented a batch buffer submission scheduler for the i915 DRM driver.
The general theory of operation is that when batch buffers are submitted to the driver, the execbuffer() code assigns a unique seqno value and then packages up all the information required to execute the batch buffer at a later time. This package is given over to the scheduler which adds it to an internal node list. The scheduler also scans the list of objects associated with the batch buffer and compares them against the objects already in use by other buffers in the node list. If matches are found then the new batch buffer node is marked as being dependent upon the matching node. The same is done for the context object. The scheduler also bumps up the priority of such matching nodes on the grounds that the more dependencies a given batch buffer has the more important it is likely to be.
The scheduler aims to have a given (tuneable) number of batch buffers in flight on the hardware at any given time. If fewer than this are currently executing when a new node is queued, then the node is passed straight through to the submit function. Otherwise it is simply added to the queue and the driver returns back to user land.
As each batch buffer completes, it raises an interrupt which wakes up the scheduler. Note that it is possible for multiple buffers to complete before the IRQ handler gets to run. Further, the seqno values of the individual buffers are not necessary incrementing as the scheduler may have re-ordered their submission. However, the scheduler keeps the list of executing buffers in order of hardware submission. Thus it can scan through the list until a matching seqno is found and then mark all in flight nodes from that point on as completed.
A deferred work queue is also poked by the interrupt handler. When this wakes up it can do more involved processing such as actually removing completed nodes from the queue and freeing up the resources associated with them (internal memory allocations, DRM object references, context reference, etc.). The work handler also checks the in flight count and calls the submission code if a new slot has appeared.
When the scheduler's submit code is called, it scans the queued node list for the highest priority node that has no unmet dependencies. Note that the dependency calculation is complex as it must take inter-ring dependencies and potential preemptions into account. Note also that in the future this will be extended to include external dependencies such as the Android Native Sync file descriptors and/or the linux dma-buff synchronisation scheme.
If a suitable node is found then it is sent to execbuff_final() for submission to the hardware. The in flight count is then re-checked and a new node popped from the list if appropriate.
The scheduler also allows high priority batch buffers (e.g. from a desktop compositor) to jump ahead of whatever is already running if the underlying hardware supports pre-emption. In this situation, any work that was pre-empted is returned to the queued list ready to be resubmitted when no more high priority work is outstanding.
Various IGT tests are in progress to test the scheduler's operation and will follow.
v2: Updated for changes in struct fence patch series and other changes to underlying tree (e.g. removal of cliprects). Also changed priority levels to be signed +/-1023 range and reduced mutex lock usage.
v3: More reuse of cached pointers rather than repeated dereferencing (David Gordon).
Moved the dependency generation code out to a seperate function for easier readability. Also added in support for the read-read optimisation.
Major simplification of the DRM file close handler.
Fixed up an overzealous WARN.
Removed unnecessary flushing of the scheduler queue when waiting for a request.
v4: Removed user land fence/sync integration as this is dependent upon the de-staging of the Android sync code. That de-staging is now being done by someone else. The sync support will be added back in to the scheduler in a separate patch series which must wait for the de-staging to be landed first.
Added support for killing batches from contexts that were banned after the batches were submitted to the scheduler.
Changed various comments to fix typos, update to reflect changes to the code, correct formatting and line wrapping, etc. Also wrapped various long lines and twiddled white space to keep the style checker happy.
Changed a bunch of BUG_ONs to WARN_ONs as apparently the latter are preferred.
Used the correct array memory allocator function (kmalloc_array instead of kmalloc).
Fixed a variable type (review comment by Joonas).
Fixed a WARN_ON firing incorrectly when removing killed nodes from the scheduler's queue.
Dropped register definition update patch from this series. The changes are all for pre-emption so it makes more sense for it to be part of that series instead.
v5: Reverted power management changes as they apparently conflict with mutex acquisition. Converted the override mask module parameter to a set of boolean enable flags (just one in this patch set, but others are added later for controlling pre-emption). [Chris Wilson]
Removed lots of whitespace from i915_scheduler.c and re-ordered it to remove all forward declarations. Squashed down the i915_scheduler.c sections of various patches into the initial 'start of scheduler' patch. Thus the later patches simply hook in existing code into various parts of the driver rather than adding the code as well. Added documentation to various functions. Re-worked the submit function in terms of mutex locking, error handling and exit paths. Split the delayed work handler function in half. Made use of the kernel 'clamp' macro. [Joonas Lahtinen]
Dropped the 'immediate submission' override option. This was a half way house between full scheduler and direct submission and was only really useful during early debug.
Added a re-install of the scheduler's interrupt hook around GPU reset. [Zhipeng Gong]
Used lighter weight spinlocks.
[Patches against drm-intel-nightly tree fetched 19/01/2016 with struct fence conversion patches applied]
Dave Gordon (2):
drm/i915: Cache request pointer in *_submission_final() drm/i915: Add scheduling priority to per-context parameters
John Harrison (33):
drm/i915: Add total count to context status debugfs output drm/i915: Prelude to splitting i915_gem_do_execbuffer in two drm/i915: Split i915_dem_do_execbuffer() in half
drm/i915: Re-instate request->uniq because it is extremely useful drm/i915: Start of GPU scheduler
drm/i915: Prepare retire_requests to handle out-of-order seqnos drm/i915: Disable hardware semaphores when GPU scheduler is enabled drm/i915: Force MMIO flips when scheduler enabled
drm/i915: Added scheduler hook when closing DRM file handles drm/i915: Added scheduler hook into i915_gem_request_notify() drm/i915: Added deferred work handler for scheduler
drm/i915: Redirect execbuffer_final() via scheduler
drm/i915: Keep the reserved space mechanism happy
drm/i915: Added tracking/locking of batch buffer objects drm/i915: Hook scheduler node clean up into retire requests drm/i915: Added scheduler support to __wait_request() calls drm/i915: Added scheduler support to page fault handler drm/i915: Added scheduler flush calls to ring throttle and idle functions drm/i915: Add scheduler hook to GPU reset
drm/i915: Added a module parameter to allow the scheduler to be disabled drm/i915: Support for 'unflushed' ring idle
drm/i915: Defer seqno allocation until actual hardware submission time drm/i915: Added trace points to scheduler
drm/i915: Added scheduler queue throttling by DRM file handle drm/i915: Added debugfs interface to scheduler tuning parameters drm/i915: Added debug state dump facilities to scheduler drm/i915: Add early exit to execbuff_final() if insufficient ring space drm/i915: Added scheduler statistic reporting to debugfs drm/i915: Add scheduler support functions for TDR
drm/i915: Scheduler state dump via debugfs
drm/i915: Enable GPU scheduler by default
drm/i915: Add support for retro-actively banning batch buffers drm/i915: Allow scheduler to manage inter-ring object synchronisation
drivers/gpu/drm/i915/Makefile | 1 + drivers/gpu/drm/i915/i915_debugfs.c | 280 ++++- drivers/gpu/drm/i915/i915_dma.c | 6 + drivers/gpu/drm/i915/i915_drv.c | 9 + drivers/gpu/drm/i915/i915_drv.h | 54 +- drivers/gpu/drm/i915/i915_gem.c | 179 ++- drivers/gpu/drm/i915/i915_gem_context.c | 24 + drivers/gpu/drm/i915/i915_gem_execbuffer.c | 286 ++++- drivers/gpu/drm/i915/i915_params.c | 4 + drivers/gpu/drm/i915/i915_params.h | 1 + drivers/gpu/drm/i915/i915_scheduler.c | 1663 ++++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_scheduler.h | 159 +++ drivers/gpu/drm/i915/i915_trace.h | 228 +++- drivers/gpu/drm/i915/intel_display.c | 10 +- drivers/gpu/drm/i915/intel_lrc.c | 161 ++- drivers/gpu/drm/i915/intel_lrc.h | 1 + drivers/gpu/drm/i915/intel_ringbuffer.c | 49 +- drivers/gpu/drm/i915/intel_ringbuffer.h | 3 +- include/uapi/drm/i915_drm.h | 1 + 19 files changed, 2976 insertions(+), 143 deletions(-) create mode 100644 drivers/gpu/drm/i915/i915_scheduler.c create mode 100644 drivers/gpu/drm/i915/i915_scheduler.h
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 1/35 drm/i915: Add total count to context status debugfs output
From: John Harrison <[email protected]>
When there are lots and lots and even more lots of contexts (e.g. when running with execlists) it is useful to be able to immediately see what the total context count is.
v4: Re-typed a variable (review feedback from Joonas)
For: VIZ-1587
--- drivers/gpu/drm/i915/i915_debugfs.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_debugfs.c +4 -1
@@ -1971,7 +1971,7 @@ static int i915_context_status(struct seq_file *m, void *unused)struct drm_i915_private *dev_priv = dev->dev_private;struct intel_engine_cs *ring;struct intel_context *ctx; -int ret, i; +int ret, i, count =0 ; ret =mutex_lock_interruptible (&dev->struct_mutex);if (ret)
@@ -1984,6 +1984,7 @@ static int i915_context_status(struct seq_file *m, void *unused)seq_puts (m,"HW context " );describe_ctx (m, ctx); + count++;for_each_ring (ring, dev_priv, i) {if (ring->default_context == ctx) seq_printf(m, "(default context %s) ",
@@ -2012,6 +2013,8 @@ static int i915_context_status(struct seq_file *m, void *unused)seq_putc (m,'\n' ); } +seq_printf (m,"Total: %d contexts\n" , count); +mutex_unlock (&dev->struct_mutex);return 0 ;
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 2/35 drm/i915: Prelude to splitting i915_gem_do_execbuffer in two
In reply to [email protected]From: John Harrison <[email protected]>
The scheduler decouples the submission of batch buffers to the driver with their submission to the hardware. This basically means splitting the execbuffer() function in half. This change rearranges some code ready for the split to occur.
v5: Dropped runtime PM calls as they conflict with the mutex lock. Instead of being done at the lowest submission level, they are now left right at the top driver entry level. [feedback from Chris Wilson]
For: VIZ-1587
--- drivers/gpu/drm/i915/i915_gem_execbuffer.c | 41 ++++++++++++++++++------------ drivers/gpu/drm/i915/intel_lrc.c | 18 +++++++++---- 2 files changed, 38 insertions(+), 21 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +25 -16
@@ -966,10 +966,7 @@ i915_gem_execbuffer_move_to_gpu(struct drm_i915_gem_request *req,if (flush_domains &I915_GEM_DOMAIN_GTT )wmb (); -/* Unconditionally invalidate gpu caches and ensure that we do flush -* any residual writes from the previous batch. -*/ -return intel_ring_invalidate_all_caches (req); +return 0 ; } static bool
@@ -1237,17 +1234,6 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,u32 instp_mask;int ret; - ret =i915_gem_execbuffer_move_to_gpu (params->request, vmas); -if (ret) -return ret; - - ret =i915_switch_context (params->request); -if (ret) -return ret; - -WARN (params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1 <<ring->id), -"%s didn't clear reload\n" , ring->name); - instp_mode = args->flags &I915_EXEC_CONSTANTS_MASK ; instp_mask =I915_EXEC_CONSTANTS_MASK ;switch (instp_mode) {
@@ -1281,6 +1267,30 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,return -EINVAL ; } + ret =i915_gem_execbuffer_move_to_gpu (params->request, vmas); +if (ret) +return ret; + +i915_gem_execbuffer_move_to_active (vmas, params->request); + +/* To be split into two functions here... */ + +/* +* Unconditionally invalidate gpu caches and ensure that we do flush +* any residual writes from the previous batch. +*/ + ret =intel_ring_invalidate_all_caches (params->request); +if (ret) +return ret; + +/* Switch to the correct context for the batch */ + ret =i915_switch_context (params->request); +if (ret) +return ret; + +WARN (params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1 <<ring->id), +"%s didn't clear reload\n" , ring->name); +if (ring == &dev_priv->ring[RCS ] && instp_mode != dev_priv->relative_constants_mode) { ret =intel_ring_begin (params->request,4 );
@@ -1317,7 +1327,6 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,trace_i915_gem_ring_dispatch (params->request, params->dispatch_flags); -i915_gem_execbuffer_move_to_active (vmas, params->request);i915_gem_execbuffer_retire_commands (params);return 0 ;
drivers/gpu/drm/i915/intel_lrc.c +13 -5
@@ -678,10 +678,7 @@ static int execlists_move_to_gpu(struct drm_i915_gem_request *req,if (flush_domains &I915_GEM_DOMAIN_GTT )wmb (); -/* Unconditionally invalidate gpu caches and ensure that we do flush -* any residual writes from the previous batch. -*/ -return logical_ring_invalidate_all_caches (req); +return 0 ; }int intel_logical_ring_alloc_request_extras (struct drm_i915_gem_request *request)
@@ -957,6 +954,18 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,if (ret)return ret; +i915_gem_execbuffer_move_to_active (vmas, params->request); + +/* To be split into two functions here... */ + +/* +* Unconditionally invalidate gpu caches and ensure that we do flush +* any residual writes from the previous batch. +*/ + ret =logical_ring_invalidate_all_caches (params->request); +if (ret) +return ret; +if (ring == &dev_priv->ring[RCS ] && instp_mode != dev_priv->relative_constants_mode) { ret =intel_logical_ring_begin (params->request,4 );
@@ -981,7 +990,6 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,trace_i915_gem_ring_dispatch (params->request, params->dispatch_flags); -i915_gem_execbuffer_move_to_active (vmas, params->request);i915_gem_execbuffer_retire_commands (params);return 0 ;
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 3/35 drm/i915: Split i915_dem_do_execbuffer() in half
In reply to [email protected]From: John Harrison <[email protected]>
Split the execbuffer() function in half. The first half collects and validates all the information required to process the batch buffer. It also does all the object pinning, relocations, active list management, etc - basically anything that must be done upfront before the IOCTL returns and allows the user land side to start changing/freeing things. The second half does the actual ring submission.
This change implements the split but leaves the back half being called directly from the end of the front half.
v2: Updated due to changes in underlying tree - addition of sync fence support and removal of cliprects.
v3: Moved local 'ringbuf' variable to make later patches in the series a bit neater.
v4: Corrected a typo in the commit message and downgraded a BUG_ON to a WARN_ON as the latter is preferred. Also removed all the external sync/fence support as that will now be a separate patch series.
v5: Updated for runtime PM changes.
For: VIZ-1587
--- drivers/gpu/drm/i915/i915_drv.h | 11 +++ drivers/gpu/drm/i915/i915_gem.c | 2 + drivers/gpu/drm/i915/i915_gem_execbuffer.c | 103 ++++++++++++++++++++++------- drivers/gpu/drm/i915/intel_lrc.c | 57 +++++++++++----- drivers/gpu/drm/i915/intel_lrc.h | 1 + 5 files changed, 132 insertions(+), 42 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +11 -0
@@ -1687,10 +1687,18 @@ struct i915_execbuffer_params {struct drm_device *dev;struct drm_file *file;uint32_t dispatch_flags; +uint32_t args_flags;uint32_t args_batch_start_offset; +uint32_t args_batch_len; +uint32_t args_num_cliprects; +uint32_t args_DR1; +uint32_t args_DR4;uint64_t batch_obj_vm_offset;struct intel_engine_cs *ring;struct drm_i915_gem_object *batch_obj; +struct drm_clip_rect *cliprects; +uint32_t instp_mask; +int instp_mode;struct intel_context *ctx;struct drm_i915_gem_request *request; };
@@ -1965,6 +1973,7 @@ struct drm_i915_private {int (*execbuf_submit)(struct i915_execbuffer_params *params,struct drm_i915_gem_execbuffer2 *args,struct list_head *vmas); +int (*execbuf_final)(struct i915_execbuffer_params *params);int (*init_rings)(struct drm_device *dev);void (*cleanup_ring)(struct intel_engine_cs *ring);void (*stop_ring)(struct intel_engine_cs *ring);
@@ -2835,9 +2844,11 @@ int i915_gem_sw_finish_ioctl(struct drm_device *dev, void *data,void i915_gem_execbuffer_move_to_active (struct list_head *vmas,struct drm_i915_gem_request *req);void i915_gem_execbuffer_retire_commands (struct i915_execbuffer_params *params); +void i915_gem_execbuff_release_batch_obj (struct drm_i915_gem_object *batch_obj);int i915_gem_ringbuffer_submission (struct i915_execbuffer_params *params,struct drm_i915_gem_execbuffer2 *args,struct list_head *vmas); +int i915_gem_ringbuffer_submission_final (struct i915_execbuffer_params *params);int i915_gem_execbuffer (struct drm_device *dev,void *data,struct drm_file *file_priv); int i915_gem_execbuffer2(struct drm_device *dev, void *data,
drivers/gpu/drm/i915/i915_gem.c +2 -0
@@ -5293,11 +5293,13 @@ int i915_gem_init(struct drm_device *dev)if (!i915.enable_execlists) { dev_priv->gt.execbuf_submit = i915_gem_ringbuffer_submission; + dev_priv->gt.execbuf_final = i915_gem_ringbuffer_submission_final; dev_priv->gt.init_rings = i915_gem_init_rings; dev_priv->gt.cleanup_ring = intel_cleanup_ring_buffer; dev_priv->gt.stop_ring = intel_stop_ring_buffer; }else { dev_priv->gt.execbuf_submit = intel_execlists_submission; + dev_priv->gt.execbuf_final = intel_execlists_submission_final; dev_priv->gt.init_rings = intel_logical_rings_init; dev_priv->gt.cleanup_ring = intel_logical_ring_cleanup; dev_priv->gt.stop_ring = intel_logical_ring_stop;
drivers/gpu/drm/i915/i915_gem_execbuffer.c +78 -25
181 lines not on this page · raw message
drivers/gpu/drm/i915/intel_lrc.c +40 -17
103 lines not on this page · raw message
drivers/gpu/drm/i915/intel_lrc.h +1 -0
@@ -116,6 +116,7 @@ struct i915_execbuffer_params;int intel_execlists_submission (struct i915_execbuffer_params *params,struct drm_i915_gem_execbuffer2 *args,struct list_head *vmas); +int intel_execlists_submission_final (struct i915_execbuffer_params *params);void intel_lrc_irq_handler (struct intel_engine_cs *ring);void intel_execlists_retire_requests (struct intel_engine_cs *ring);
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 4/35 drm/i915: Cache request pointer in *_submission_final()
In reply to [email protected]From: Dave Gordon <[email protected]>
Keep a local copy of the request pointer in the _final() functions rather than dereferencing the params block repeatedly.
v3: New patch in series.
For: VIZ-1587
--- drivers/gpu/drm/i915/i915_gem_execbuffer.c | 13 +++++++------ drivers/gpu/drm/i915/intel_lrc.c | 11 ++++++----- 2 files changed, 13 insertions(+), 11 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +7 -6
@@ -1291,6 +1291,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,int i915_gem_ringbuffer_submission_final (struct i915_execbuffer_params *params) {struct drm_i915_private *dev_priv = params->dev->dev_private; +struct drm_i915_gem_request *req = params->request;struct intel_engine_cs *ring = params->ring;u64 exec_start, exec_len;int ret;
@@ -1302,12 +1303,12 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)* Unconditionally invalidate gpu caches and ensure that we do flush * any residual writes from the previous batch. */ - ret =intel_ring_invalidate_all_caches (params->request); + ret =intel_ring_invalidate_all_caches (req);if (ret)return ret;/* Switch to the correct context for the batch */ - ret =i915_switch_context (params->request); + ret =i915_switch_context (req);if (ret)return ret;
@@ -1316,7 +1317,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)if (ring == &dev_priv->ring[RCS ] && params->instp_mode != dev_priv->relative_constants_mode) { - ret =intel_ring_begin (params->request,4 ); + ret =intel_ring_begin (req,4 );if (ret)return ret;
@@ -1330,7 +1331,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params) }if (params->args_flags &I915_EXEC_GEN7_SOL_RESET ) { - ret =i915_reset_gen7_sol_offsets (params->dev, params->request); + ret =i915_reset_gen7_sol_offsets (params->dev, req);if (ret)return ret; }
@@ -1342,13 +1343,13 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)if (exec_len ==0 ) exec_len = params->batch_obj->base.size; - ret = ring->dispatch_execbuffer (params->request, + ret = ring->dispatch_execbuffer (req, exec_start, exec_len, params->dispatch_flags);if (ret)return ret; -trace_i915_gem_ring_dispatch (params->request, params->dispatch_flags); +trace_i915_gem_ring_dispatch (req, params->dispatch_flags);i915_gem_execbuffer_retire_commands (params);
drivers/gpu/drm/i915/intel_lrc.c +6 -5
@@ -973,7 +973,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,int intel_execlists_submission_final (struct i915_execbuffer_params *params) {struct drm_i915_private *dev_priv = params->dev->dev_private; -struct intel_ringbuffer *ringbuf = params->request->ringbuf; +struct drm_i915_gem_request *req = params->request; +struct intel_ringbuffer *ringbuf = req->ringbuf;struct intel_engine_cs *ring = params->ring;u64 exec_start;int ret;
@@ -985,13 +986,13 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)* Unconditionally invalidate gpu caches and ensure that we do flush * any residual writes from the previous batch. */ - ret =logical_ring_invalidate_all_caches (params->request); + ret =logical_ring_invalidate_all_caches (req);if (ret)return ret;if (ring == &dev_priv->ring[RCS ] && params->instp_mode != dev_priv->relative_constants_mode) { - ret =intel_logical_ring_begin (params->request,4 ); + ret =intel_logical_ring_begin (req,4 );if (ret)return ret;
@@ -1007,11 +1008,11 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params) exec_start = params->batch_obj_vm_offset + params->args_batch_start_offset; - ret = ring->emit_bb_start (params->request, exec_start, params->dispatch_flags); + ret = ring->emit_bb_start (req, exec_start, params->dispatch_flags);if (ret)return ret; -trace_i915_gem_ring_dispatch (params->request, params->dispatch_flags); +trace_i915_gem_ring_dispatch (req, params->dispatch_flags);i915_gem_execbuffer_retire_commands (params);
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 5/35 drm/i915: Re-instate request->uniq because it is extremely useful
In reply to [email protected]From: John Harrison <[email protected]>
The seqno value cannot always be used when debugging issues via trace points. This is because it can be reset back to start, especially during TDR type tests. Also, when the scheduler arrives the seqno is only valid while a given request is executing on the hardware. While the request is simply queued waiting for submission, it's seqno value will be zero (meaning invalid).
v4: Wrapped a long line to keep the style checker happy.
v5: Added uniq to the dispatch trace point [Svetlana Kukanova]
For: VIZ-5115
--- drivers/gpu/drm/i915/i915_drv.h | 5 +++++ drivers/gpu/drm/i915/i915_gem.c | 4 +++- drivers/gpu/drm/i915/i915_trace.h | 32 ++++++++++++++++++++++---------- 3 files changed, 30 insertions(+), 11 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +5 -0
@@ -1986,6 +1986,8 @@ struct drm_i915_private {struct intel_encoder *dig_port_map[I915_MAX_PORTS ]; +uint32_t request_uniq; +/* * NOTE: This is the dri1/ums dungeon, don't add stuff here. Your patch * will be rejected. Instead look for a better place.
@@ -2242,6 +2244,9 @@ struct drm_i915_gem_request {*/ u32 seqno; +/* Unique identifier which can be used for trace points & debug */ +uint32_t uniq; +/** Position in the ringbuffer of the start of the request */ u32 head;
drivers/gpu/drm/i915/i915_gem.c +3 -1
@@ -2960,7 +2960,8 @@ static void i915_gem_request_fence_value_str(struct fence *req_fence, req =container_of (req_fence,typeof (*req), fence); -snprintf (str, size,"%d [%d]" , req->fence.seqno, req->seqno); +snprintf (str, size,"%d [%d:%d]" , req->fence.seqno, req->uniq, + req->seqno); }static const struct fence_ops i915_gem_request_fops = {
@@ -3036,6 +3037,7 @@ int i915_gem_request_alloc(struct intel_engine_cs *ring, req->i915 = dev_priv; req->ring = ring; + req->uniq = dev_priv->request_uniq++; req->ctx = ctx;i915_gem_context_reference (req->ctx);
drivers/gpu/drm/i915/i915_trace.h +22 -10
@@ -469,6 +469,7 @@ TRACE_EVENT(i915_gem_ring_sync_to, __field(u32, dev) __field(u32, sync_from) __field(u32, sync_to) + __field(u32, uniq_to)__field (u32, seqno) ),
@@ -476,13 +477,14 @@ TRACE_EVENT(i915_gem_ring_sync_to, __entry->dev = from->dev->primary->index; __entry->sync_from = from->id; __entry->sync_to = to_req->ring->id; + __entry->uniq_to = to_req->uniq; __entry->seqno =i915_gem_request_get_seqno (req); ), -TP_printk ("dev=%u, sync-from=%u, sync-to=%u, seqno=%u" , +TP_printk ("dev=%u, sync-from=%u, sync-to=%u, seqno=%u, to_uniq=%u" , __entry->dev, __entry->sync_from, __entry->sync_to, - __entry->seqno) + __entry->seqno, __entry->uniq_to) ); TRACE_EVENT(i915_gem_ring_dispatch,
@@ -492,6 +494,7 @@ TRACE_EVENT(i915_gem_ring_dispatch,TP_STRUCT__entry (__field (u32 ,dev )__field (u32, ring) +__field (u32, uniq)__field (u32, seqno)__field (u32, flags) ),
@@ -501,13 +504,15 @@ TRACE_EVENT(i915_gem_ring_dispatch,i915_gem_request_get_ring (req); __entry->dev = ring->dev->primary->index; __entry->ring = ring->id; + __entry->uniq = req->uniq; __entry->seqno =i915_gem_request_get_seqno (req); __entry->flags = flags;i915_trace_irq_get (ring, req); ), -TP_printk ("dev=%u, ring=%u, seqno=%u, flags=%x" , - __entry->dev, __entry->ring, __entry->seqno, __entry->flags) +TP_printk ("dev=%u, ring=%u, uniq=%u, seqno=%u, flags=%x" , + __entry->dev, __entry->ring, __entry->uniq, + __entry->seqno, __entry->flags) ); TRACE_EVENT(i915_gem_ring_flush,
@@ -517,6 +522,7 @@ TRACE_EVENT(i915_gem_ring_flush,TP_STRUCT__entry (__field (u32 ,dev )__field (u32, ring) +__field (u32, uniq)__field (u32, invalidate)__field (u32, flush) ),
@@ -524,12 +530,13 @@ TRACE_EVENT(i915_gem_ring_flush,TP_fast_assign ( __entry->dev = req->ring->dev->primary->index; __entry->ring = req->ring->id; + __entry->uniq = req->uniq; __entry->invalidate = invalidate; __entry->flush = flush; ), -TP_printk ("dev=%u, ring=%x, invalidate=%04x, flush=%04x" , - __entry->dev, __entry->ring, +TP_printk ("dev=%u, ring=%x, request=%u, invalidate=%04x, flush=%04x" , + __entry->dev, __entry->ring, __entry->uniq, __entry->invalidate, __entry->flush) );
@@ -540,6 +547,7 @@ DECLARE_EVENT_CLASS(i915_gem_request,TP_STRUCT__entry (__field (u32 ,dev )__field (u32, ring) +__field (u32, uniq)__field (u32, seqno) ),
@@ -548,11 +556,13 @@ DECLARE_EVENT_CLASS(i915_gem_request,i915_gem_request_get_ring (req); __entry->dev = ring->dev->primary->index; __entry->ring = ring->id; + __entry->uniq = req ? req->uniq :0 ; __entry->seqno =i915_gem_request_get_seqno (req); ), -TP_printk ("dev=%u, ring=%u, seqno=%u" , - __entry->dev, __entry->ring, __entry->seqno) +TP_printk ("dev=%u, ring=%u, uniq=%u, seqno=%u" , + __entry->dev, __entry->ring, __entry->uniq, + __entry->seqno) ); DEFINE_EVENT(i915_gem_request, i915_gem_request_add,
@@ -601,6 +611,7 @@ TRACE_EVENT(i915_gem_request_wait_begin,TP_STRUCT__entry (__field (u32 ,dev )__field (u32, ring) +__field (u32, uniq)__field (u32, seqno)__field (bool, blocking) ),
@@ -616,13 +627,14 @@ TRACE_EVENT(i915_gem_request_wait_begin,i915_gem_request_get_ring (req); __entry->dev = ring->dev->primary->index; __entry->ring = ring->id; + __entry->uniq = req ? req->uniq :0 ; __entry->seqno =i915_gem_request_get_seqno (req); __entry->blocking =mutex_is_locked (&ring->dev->struct_mutex); ), -TP_printk ("dev=%u, ring=%u, seqno=%u, blocking=%s" , - __entry->dev, __entry->ring, +TP_printk ("dev=%u, ring=%u, uniq=%u, seqno=%u, blocking=%s" , + __entry->dev, __entry->ring, __entry->uniq, __entry->seqno, __entry->blocking ?"yes (NB)" :"no" ) );
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 6/35 drm/i915: Start of GPU scheduler
In reply to [email protected]From: John Harrison <[email protected]>
Initial creation of scheduler source files. Note that this patch implements most of the scheduler functionality but does not hook it in to the driver yet. It also leaves the scheduler code in 'pass through' mode so that even when it is hooked in, it will not actually do very much. This allows the hooks to be added one at a time in bite size chunks and only when the scheduler is finally enabled at the end does anything start happening.
The general theory of operation is that when batch buffers are submitted to the driver, the execbuffer() code packages up all the information required to execute the batch buffer at a later time. This package is given over to the scheduler which adds it to an internal node list. The scheduler also scans the list of objects associated with the batch buffer and compares them against the objects already in use by other buffers in the node list. If matches are found then the new batch buffer node is marked as being dependent upon the matching node. The same is done for the context object. The scheduler also bumps up the priority of such matching nodes on the grounds that the more dependencies a given batch buffer has the more important it is likely to be.
The scheduler aims to have a given (tuneable) number of batch buffers in flight on the hardware at any given time. If fewer than this are currently executing when a new node is queued, then the node is passed straight through to the submit function. Otherwise it is simply added to the queue and the driver returns back to user land.
The scheduler is notified when each batch buffer completes and updates its internal tracking accordingly. At the end of the completion interrupt processing, if any scheduler tracked batches were processed, the scheduler's deferred worker thread is woken up. This can do more involved processing such as actually removing completed nodes from the queue and freeing up the resources associated with them (internal memory allocations, DRM object references, context reference, etc.). The work handler also checks the in flight count and calls the submission code if a new slot has appeared.
When the scheduler's submit code is called, it scans the queued node list for the highest priority node that has no unmet dependencies. Note that the dependency calculation is complex as it must take inter-ring dependencies and potential preemptions into account. Note also that in the future this will be extended to include external dependencies such as the Android Native Sync file descriptors and/or the linux dma-buff synchronisation scheme.
If a suitable node is found then it is sent to execbuff_final() for submission to the hardware. The in flight count is then re-checked and a new node popped from the list if appropriate. All nodes that are not submitted have their priority bumped. This ensures that low priority tasks do not get starved out by busy higher priority ones - everything will eventually get its turn to run.
Note that this patch does not implement pre-emptive scheduling. Only basic scheduling by re-ordering batch buffer submission is currently implemented. Pre-emption of actively executing batch buffers comes in the next patch series.
v2: Changed priority levels to +/-1023 due to feedback from Chris Wilson.
Removed redundant index from scheduler node.
Changed time stamps to use jiffies instead of raw monotonic. This provides lower resolution but improved compatibility with other i915 code.
Major re-write of completion tracking code due to struct fence conversion. The scheduler no longer has it's own private IRQ handler but just lets the existing request code handle completion events. Instead, the scheduler now hooks into the request notify code to be told when a request has completed.
Reduced driver mutex locking scope. Removal of scheduler nodes no longer grabs the mutex lock.
v3: Refactor of dependency generation to make the code more readable. Also added in read-read optimisation support - i.e., don't treat a shared read-only buffer as being a dependency.
Allowed the killing of queued nodes rather than only flying ones.
v4: Updated the commit message to better reflect the current state of the code. Downgraded some BUG_ONs to WARN_ONs. Used the correct array memory allocator function (kmalloc_array instead of kmalloc). Corrected the format of some comments. Wrapped some lines differently to keep the style checker happy.
Fixed a WARN_ON when killing nodes. The dependency removal code checks that nodes being destroyed do not have any oustanding dependencies (which would imply they should not have been executed yet). In the case of nodes being destroyed, e.g. due to context banning, then this might well be the case - they have not been executed and do indeed have outstanding dependencies.
Re-instated the code to disble interrupts when not in use. The underlying problem causing broken IRQ reference counts seems to have been fixed now.
v5: Shuffled various functions around to remove forward declarations as apparently these are frowned upon. Removed lots of white space as apparently having easy to read code is also frowned upon. Split the direct submission scheduler bypass code out into a separate function. Squashed down the i915_scheduler.c sections of various patches into this patch. Thus the later patches simply hook in existing code into various parts of the driver rather than adding the code as well. Added documentation to various functions. Re-worked the submit function in terms of mutex locking, error handling and exit paths. Split the delayed work handler function in half. Made use of the kernel 'clamp' macro. [Joonas Lahtinen]
Added runtime PM calls as these must be done at the top level before acquiring the driver mutex lock. [Chris Wilson]
Removed some obsolete debug code that had been forgotten about.
Moved more clean up code into the 'i915_gem_scheduler_clean_node()' function rather than replicating it in mutliple places.
Used lighter weight spinlocks.
For: VIZ-1587
--- drivers/gpu/drm/i915/Makefile | 1 + drivers/gpu/drm/i915/i915_drv.h | 6 + drivers/gpu/drm/i915/i915_gem.c | 5 + drivers/gpu/drm/i915/i915_scheduler.c | 874 ++++++++++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_scheduler.h | 95 ++++ 5 files changed, 981 insertions(+) create mode 100644 drivers/gpu/drm/i915/i915_scheduler.c create mode 100644 drivers/gpu/drm/i915/i915_scheduler.h
drivers/gpu/drm/i915/Makefile +1 -0
@@ -10,6 +10,7 @@ ccflags-y := -Werror i915-y := i915_drv.o \ i915_irq.o \ i915_params.o \ + i915_scheduler.o \ i915_suspend.o \ i915_sysfs.o \ intel_csr.o \
drivers/gpu/drm/i915/i915_drv.h +6 -0
@@ -1703,6 +1703,8 @@ struct i915_execbuffer_params {struct drm_i915_gem_request *request; }; +struct i915_scheduler ; +/* used in computing the new watermarks state */ struct intel_wm_config {unsigned int num_pipes_active;
@@ -1968,6 +1970,8 @@ struct drm_i915_private {struct i915_runtime_pm pm; +struct i915_scheduler *scheduler; +/* Abstract the submission mechanism (legacy ringbuffer or execlists) away */ struct { int (*execbuf_submit)(struct i915_execbuffer_params *params,
@@ -2290,6 +2294,8 @@ struct drm_i915_gem_request {/** process identifier submitting this request */ struct pid *pid; +struct i915_scheduler_queue_entry *scheduler_qe; +/** * The ELSP only accepts two elements at a time, so we queue * context/tail pairs on a given queue (ring->execlist_queue) until the
drivers/gpu/drm/i915/i915_gem.c +5 -0
@@ -32,6 +32,7 @@#include "i915_vgpu.h" #include "i915_trace.h" #include "intel_drv.h" +#include "i915_scheduler.h" #include <linux/shmem_fs.h> #include <linux/slab.h> #include <linux/swap.h>
@@ -5315,6 +5316,10 @@ int i915_gem_init(struct drm_device *dev)*/ intel_uncore_forcewake_get (dev_priv,FORCEWAKE_ALL ); + ret =i915_scheduler_init (dev); +if (ret) + goto out_unlock; + ret =i915_gem_init_userptr (dev);if (ret) goto out_unlock;
drivers/gpu/drm/i915/i915_scheduler.c new file +874 -0
874 lines not on this page · raw message
drivers/gpu/drm/i915/i915_scheduler.h new file +95 -0
95 lines not on this page · raw message
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Joonas Lahtinen
maintainerRe: [PATCH v5 06/35] drm/i915: Start of GPU scheduler
Hi,
Now the code is in reviewable chunks, excellent!
I've added my comments below. A few repeats from last round, but now with more questions about the logic itself.
Quoted 164 lines · On to, 2016-02-18 at 14:26 +0000, [email protected] wrote:
From: John Harrison <[email protected]>
Initial creation of scheduler source files. Note that this patch implements most of the scheduler functionality but does not hook it in to the driver yet. It also leaves the scheduler code in 'pass through' mode so that even when it is hooked in, it will not actually do very much. This allows the hooks to be added one at a time in bite size chunks and only when the scheduler is finally enabled at the end does anything start happening.
The general theory of operation is that when batch buffers are submitted to the driver, the
execbuffer()code packages up all the information required to execute the batch buffer at a later time. This package is given over to the scheduler which adds it to an internal node list. The scheduler also scans the list of objects associated with the batch buffer and compares them against the objects already in use by other buffers in the node list. If matches are found then the new batch buffer node is marked as being dependent upon the matching node. The same is done for the context object. The scheduler also bumps up the priority of such matching nodes on the grounds that the more dependencies a given batch buffer has the more important it is likely to be.The scheduler aims to have a given (tuneable) number of batch buffers in flight on the hardware at any given time. If fewer than this are currently executing when a new node is queued, then the node is passed straight through to the submit function. Otherwise it is simply added to the queue and the driver returns back to user land.
The scheduler is notified when each batch buffer completes and updates its internal tracking accordingly. At the end of the completion interrupt processing, if any scheduler tracked batches were processed, the scheduler's deferred worker thread is woken up. This can do more involved processing such as actually removing completed nodes from the queue and freeing up the resources associated with them (internal memory allocations, DRM object references, context reference, etc.). The work handler also checks the in flight count and calls the submission code if a new slot has appeared.
When the scheduler's submit code is called, it scans the queued node list for the highest priority node that has no unmet dependencies. Note that the dependency calculation is complex as it must take inter-ring dependencies and potential preemptions into account. Note also that in the future this will be extended to include external dependencies such as the Android Native Sync file descriptors and/or the linux dma-buff synchronisation scheme.
If a suitable node is found then it is sent to
execbuff_final()for submission to the hardware. The in flight count is then re-checked and a new node popped from the list if appropriate. All nodes that are not submitted have their priority bumped. This ensures that low priority tasks do not get starved out by busy higher priority ones - everything will eventually get its turn to run.Note that this patch does not implement pre-emptive scheduling. Only basic scheduling by re-ordering batch buffer submission is currently implemented. Pre-emption of actively executing batch buffers comes in the next patch series.
v2: Changed priority levels to +/-1023 due to feedback from Chris Wilson.
Removed redundant index from scheduler node.
Changed time stamps to use jiffies instead of raw monotonic. This provides lower resolution but improved compatibility with other i915 code.
Major re-write of completion tracking code due to struct fence conversion. The scheduler no longer has it's own private IRQ handler but just lets the existing request code handle completion events. Instead, the scheduler now hooks into the request notify code to be told when a request has completed.
Reduced driver mutex locking scope. Removal of scheduler nodes no longer grabs the mutex lock.
v3: Refactor of dependency generation to make the code more readable. Also added in read-read optimisation support - i.e., don't treat a shared read-only buffer as being a dependency.
Allowed the killing of queued nodes rather than only flying ones.
v4: Updated the commit message to better reflect the current state of the code. Downgraded some
BUG_ONstoWARN_ONs. Used the correct array memory allocator function (kmalloc_arrayinstead of kmalloc). Corrected the format of some comments. Wrapped some lines differently to keep the style checker happy.Fixed a
WARN_ONwhen killing nodes. The dependency removal code checks that nodes being destroyed do not have any oustanding dependencies (which would imply they should not have been executed yet). In the case of nodes being destroyed, e.g. due to context banning, then this might well be the case - they have not been executed and do indeed have outstanding dependencies.Re-instated the code to disble interrupts when not in use. The underlying problem causing broken IRQ reference counts seems to have been fixed now.
v5: Shuffled various functions around to remove forward declarations as apparently these are frowned upon. Removed lots of white space as apparently having easy to read code is also frowned upon. Split the direct submission scheduler bypass code out into a separate function. Squashed down the
i915_scheduler.csections of various patches into this patch. Thus the later patches simply hook in existing code into various parts of the driver rather than adding the code as well. Added documentation to various functions. Re-worked the submit function in terms of mutex locking, error handling and exit paths. Split the delayed work handler function in half. Made use of the kernel 'clamp' macro. [Joonas Lahtinen]Added runtime PM calls as these must be done at the top level before acquiring the driver mutex lock. [Chris Wilson]
Removed some obsolete debug code that had been forgotten about.
Moved more clean up code into the '
i915_gem_scheduler_clean_node()' function rather than replicating it in mutliple places.Used lighter weight spinlocks.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>Cc Joonas Lahtinen <[email protected]>---
drivers/gpu/drm/i915/Makefile | 1 + drivers/gpu/drm/i915/i915_drv.h | 6 + drivers/gpu/drm/i915/i915_gem.c | 5 +drivers/gpu/drm/i915/i915_scheduler.c| 874 ++++++++++++++++++++++++++++++++++drivers/gpu/drm/i915/i915_scheduler.h| 95 ++++5 files changed, 981 insertions(+)create mode 100644
drivers/gpu/drm/i915/i915_scheduler.ccreate mode 100644drivers/gpu/drm/i915/i915_scheduler.hdrivers/gpu/drm/i915/Makefile +1
-0@@ -10,6 +10,7 @@ ccflags-y := -Werror i915-y := i915_drv.o \ i915_irq.o \ i915_params.o \ + i915_scheduler.o \ i915_suspend.o \ i915_sysfs.o \ intel_csr.o \drivers/gpu/drm/i915/i915_drv.h +0
-0@@ -1703,6 +1703,8 @@ struct i915_execbuffer_params { struct drm_i915_gem_request *request; };drivers/gpu/drm/i915/i915_drv.h +2
-0+struct i915_scheduler ; +/* used in computing the new watermarks state */
structintel_wm_config{
unsigned intnum_pipes_active;drivers/gpu/drm/i915/i915_drv.h +0
-0@@ -1968,6 +1970,8 @@ struct drm_i915_private {
structi915_runtime_pmpm;
drivers/gpu/drm/i915/i915_drv.h struct drm_i915_private { · line 1974
Quoted 2 lines
drivers/gpu/drm/i915/i915_drv.h +2
-0+struct i915_scheduler *scheduler; +
I think we should have i915.enable_scheduler parameter (just like i915.enable_execlists) and make this a data member, not pointer.
Then we can go forward and have i915.enable_scheduler_preempt and so on to control things at boot time.
drivers/gpu/drm/i915/i915_drv.h struct drm_i915_gem_request { · line 2298
Quoted 9 lines
/* Abstract the submission mechanism (legacy ringbuffer or execlists) away */ struct {
int (*execbuf_submit)(structi915_execbuffer_params*params,drivers/gpu/drm/i915/i915_drv.h +0
-0@@ -2290,6 +2294,8 @@ struct drm_i915_gem_request { /** process identifier submitting this request */ struct pid *pid;drivers/gpu/drm/i915/i915_drv.h +2
-0+struct i915_scheduler_queue_entry *scheduler_qe; +
Funny whitespace.
Quoted 60 lines
/**
* The ELSP only accepts two elements at a time, so we queue
* context/tail pairs on a given queue (ring->execlist_queue) until thedrivers/gpu/drm/i915/i915_gem.c +1
-0@@ -32,6 +32,7 @@ #include "i915_vgpu.h" #include "i915_trace.h" #include "intel_drv.h" +#include "i915_scheduler.h" #include #include #include@@ -5315,6 +5316,10 @@ int i915_gem_init(struct drm_device *dev)*/ intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);drivers/gpu/drm/i915/i915_gem.c +4
-0+ ret =i915_scheduler_init (dev); +if (ret) + goto out_unlock; +ret =
i915_gem_init_userptr(dev);
if (ret)
gotoout_unlock;drivers/gpu/drm/i915/i915_scheduler.c new file +28
-0@@ -0,0 +1,874 @@ +/* +* Copyright (c) 2014 Intel Corporation +* +* Permission is hereby granted, free of charge, to any person obtaining a +* copy of this software and associated documentation files (the "Software"), +* to deal in the Software without restriction, including without limitation +* the rights to use, copy, modify, merge, publish, distribute, sublicense, +* and/or sell copies of the Software, and to permit persons to whom the +* Software is furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice (including the next +* paragraph) shall be included in all copies or substantial portions of the +* Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +* IN THE SOFTWARE. +* +*/ + +#include "i915_drv.h" +#include "intel_drv.h" +#include "i915_scheduler.h" +
drivers/gpu/drm/i915/i915_scheduler.c line 38
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+/** +* i915_scheduler_is_enabled - Returns true if the scheduler is enabled. +* @dev: DRM device +*/ +bool i915_scheduler_is_enabled (struct drm_device *dev) +{ +struct drm_i915_private *dev_priv = dev->dev_private; + +return dev_priv->scheduler !=NULL ; +}
i915.enable_scheduler would be used instead.
drivers/gpu/drm/i915/i915_scheduler.c line 50
Quoted 12 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +12
-0+ +/** +* i915_scheduler_init - Initialise the scheduler. +* @dev: DRM device +* Returns zero on success or -ENOMEM if memory allocations fail. +*/ +int i915_scheduler_init (struct drm_device *dev) +{ +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; +int r; +
This protection -->
drivers/gpu/drm/i915/i915_scheduler.c line 57
Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7
-0+if (scheduler) +return 0 ; + + scheduler =kzalloc (sizeof (*scheduler),GFP_KERNEL ); +if (!scheduler) +return -ENOMEM ; +
Is not needed if we have data member and i915.enable_scheduler.
Quoted 15 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +15
-0+spin_lock_init (&scheduler->lock); + +for (r =0 ; r <I915_NUM_RINGS ; r++) +INIT_LIST_HEAD (&scheduler->node_queue[r]); + +/* Default tuning values: */ + scheduler->priority_level_min = -1023; + scheduler->priority_level_max = 1023; + scheduler->priority_level_bump = 50; + scheduler->priority_level_preempt =900 ; + scheduler->min_flying = 2; + + dev_priv->scheduler = scheduler; + +return 0 ;
drivers/gpu/drm/i915/i915_scheduler.c line 82
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+} + +/* +* Add a popped node back in to the queue. For example, because the ring was +* hung when execfinal() was called and thus the ring submission needs to be +* retried later. +*/ +static void i915_scheduler_node_requeue (struct i915_scheduler_queue_entry *node) +{ +WARN_ON (!node);
Rather remove this completely as it is an internal function, or use BUG_ON because next line will straight lead to OOPS after warning. I'll pass mentioning rest of the obvious WARN_ON vs BUG_ON's.
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+WARN_ON (!I915_SQS_IS_FLYING (node)); + +/* Seqno will be reassigned on relaunch */ + node->params.request->seqno =0 ; + node->status = i915_sqs_queued; +} + +/* +* Give up on a node completely. For example, because it is causing the +* ring to hang or is using some resource that no longer exists.
drivers/gpu/drm/i915/i915_scheduler.c line 102
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+*/ +static void i915_scheduler_node_kill (struct i915_scheduler_queue_entry *node) +{ +WARN_ON (!node); +WARN_ON (I915_SQS_IS_COMPLETE (node)); + + node->status = i915_sqs_dead; +} + +/* Mark a node as in flight on the hardware. */
As for documentation, from below, I only assume scheduler lock must be held and node's can be only manipulated when scheduler lock is held? It's good to add a comment describing the expected locking. Or maybe add assert_scheduler_lock_held() helper and sprinkle it around the functions similar to assert_rpm_wakelock_held(), which is self- documenting?
drivers/gpu/drm/i915/i915_scheduler.c line 105
Quoted 3 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +3
-0+static int i915_scheduler_node_fly (struct i915_scheduler_queue_entry *node) +{ +struct drm_i915_private *dev_priv = node->params.dev->dev_private;
If node is NULL, this would already OOPS, I do not think it's unreasonable to expect node not to be NULL. The below WARN_ON check is never reached.
drivers/gpu/drm/i915/i915_scheduler.c line 114
Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +9
-0+struct i915_scheduler *scheduler = dev_priv->scheduler; +struct intel_engine_cs *ring; + +WARN_ON (!scheduler); +WARN_ON (!node); +WARN_ON (node->status != i915_sqs_popped); + + ring = node->params.ring; +
This and similar assignments can be squashed to declaration line, as this is basically an alias.
Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7
-0+/* +* Add the node (which should currently be in state popped) to the +* front of the queue. This ensure that flying nodes are always held +* in hardware submission order. +*/ +list_add (&node->link, &scheduler->node_queue[ring->id]); +
drivers/gpu/drm/i915/i915_scheduler.c line 131
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+ node->status = i915_sqs_flying; + +if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) { +bool success = true; + + success = ring->irq_get (ring); +if (success) + scheduler->flags[ring->id] |= i915_sf_interrupts_enabled; +else +return -EINVAL ;
Shouldn't the node be cleaned up from node_queue in case of an error?
Also, I think above could be written much more compact form (because it looks like something where more logic will be added later). It makes it easier to write and visualize the error paths when reading if there are no nested if's.
I won't mention about the error paths of functions below, I expect the following guidline to be adhered;
if (scheduler->flags[ring->id] & I915_SF_INT_ENABLED) goto out; if (!ring->irq_get(ring)) goto err_irq; if (!...) goto err_foo; scheduler->flags[ring->id] |= I915_SF_INT_ENABLED; out: return 0; err_foo: foobar(); err_irq: list_remove() return -EINVAL;
drivers/gpu/drm/i915/i915_scheduler.c line 143
Quoted 12 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +12
-0+ } + +return 0 ; +} + +static uint32_t i915_scheduler_count_flying (struct i915_scheduler *scheduler, + struct intel_engine_cs *ring) +{ +struct i915_scheduler_queue_entry *node; +uint32_t flying =0 ; + +list_for_each_entry (node, &scheduler->node_queue[ring->id], link)
+1 for the for_each_scheduler_node(...)
Quoted 13 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +13
-0+if (I915_SQS_IS_FLYING (node)) + flying++; + +return flying; +} + +static void i915_scheduler_priority_bump_clear (struct i915_scheduler *scheduler) +{ +struct i915_scheduler_queue_entry *node; +int i; + +/* +* Ensure circular dependencies don't cause problems and that a bump
drivers/gpu/drm/i915/i915_scheduler.c line 166
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+* by object usage only bumps each using buffer once: +*/ +for (i =0 ; i <I915_NUM_RINGS ; i++) { +list_for_each_entry (node, &scheduler->node_queue[i], link) + node->bumped = false; + } +} + +static int i915_scheduler_priority_bump(struct i915_scheduler *scheduler, + struct i915_scheduler_queue_entry *target,
Is there specific reason why struct i915_scheduler_queue_entry are not referred to just as "node" but "qe" and here something else, do "node" and "qe" have a special semantic meaning?
drivers/gpu/drm/i915/i915_scheduler.c line 173
Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7
-0+ uint32_t bump) +{ +uint32_t new_priority; +int i, count; + +if (target->priority >= scheduler->priority_level_max) +return 1 ;
So if one node reaches maximum priority the dependencies are expected to be maxed out already?
Quoted 110 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +110
-0+ +if (target->bumped) +return 0 ; + + new_priority = target->priority + bump; +if ((new_priority <= target->priority) || + (new_priority > scheduler->priority_level_max)) + target->priority = scheduler->priority_level_max; +else + target->priority = new_priority; + + count =1 ; + target->bumped = true; + +for (i =0 ; i < target->num_deps; i++) { +if (!target->dep_list[i]) +continue ; + +if (target->dep_list[i]->bumped) +continue ; + + count +=i915_scheduler_priority_bump (scheduler, + target->dep_list[i], + bump); + } + +return count; +} + +/* +* Nodes are considered valid dependencies if they are queued on any ring or +* if they are in flight on a different ring. In flight on the same ring is no +* longer interesting for non-premptive nodes as the ring serialises execution. +* For pre-empting nodes, all in flight dependencies are valid as they must not +* be jumped by the act of pre-empting. +* +* Anything that is neither queued nor flying is uninteresting. +*/ +static inline bool i915_scheduler_is_dependency_valid ( +struct i915_scheduler_queue_entry *node,uint32_t idx) +{ +struct i915_scheduler_queue_entry *dep; + + dep = node->dep_list[idx]; +if (!dep) +return false; + +if (I915_SQS_IS_QUEUED (dep)) +return true; + +if (I915_SQS_IS_FLYING (dep)) { +if (node->params.ring != dep->params.ring) +return true; + } + +return false; +} + +static int i915_scheduler_pop_from_queue_locked (struct intel_engine_cs *ring, +struct i915_scheduler_queue_entry **pop_node) +{ +struct drm_i915_private *dev_priv = ring->dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; +struct i915_scheduler_queue_entry *best =NULL ; +struct i915_scheduler_queue_entry *node; +int ret; +int i; +bool any_queued = false; +bool has_local, has_remote, only_remote = false; + + *pop_node =NULL ; + ret = -ENODATA ; + +list_for_each_entry (node, &scheduler->node_queue[ring->id], link) { +if (!I915_SQS_IS_QUEUED (node)) +continue ; + any_queued = true; + + has_local = false; + has_remote = false; +for (i =0 ; i < node->num_deps; i++) { +if (!i915_scheduler_is_dependency_valid (node, i)) +continue ; + +if (node->dep_list[i]->params.ring == node->params.ring) + has_local = true; +else + has_remote = true; + } + +if (has_remote && !has_local) + only_remote = true; + +if (!has_local && !has_remote) { +if (!best || + (node->priority > best->priority)) + best = node; + } + } + +if (best) { +list_del (&best->link); + +INIT_LIST_HEAD (&best->link); + best->status = i915_sqs_popped; + + ret =0 ; + }else { +/* Can only get here if: +* (a) there are no buffers in the queue
drivers/gpu/drm/i915/i915_scheduler.c line 293
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+* (b) all queued buffers are dependent on other buffers +* e.g. on a buffer that is in flight on a different ring +*/ +if (only_remote) { +/* The only dependent buffers are on another ring. */ + ret = -EAGAIN ; + }else if (any_queued) { +/* It seems that something has gone horribly wrong! */ +DRM_ERROR ("Broken dependency tracking on ring %d!\n" , + (int) ring->id);
Would this condition be worthy WARN_ONCE(), because it will keep repeating as the situation will persist?
Quoted 61 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +61
-0+ } + } + + *pop_node = best; +return ret; +} + +/* +* NB: The driver mutex lock must be held before calling this function. It is +* only really required during the actual back end submission call. However, +* attempting to acquire a mutex while holding a spin lock is a Bad Idea. +* And releasing the one before acquiring the other leads to other code +* being run and interfering. +*/ +static int i915_scheduler_submit (struct intel_engine_cs *ring) +{ +struct drm_device *dev = ring->dev; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; +struct i915_scheduler_queue_entry *node; +int ret, count =0 , flying; + +WARN_ON (!mutex_is_locked (&dev->struct_mutex)); + +spin_lock_irq (&scheduler->lock); + +WARN_ON (scheduler->flags[ring->id] & i915_sf_submitting); + scheduler->flags[ring->id] |= i915_sf_submitting; + +/* First time around, complain if anything unexpected occurs: */ + ret =i915_scheduler_pop_from_queue_locked (ring, &node); +if (ret) + goto error; + + do { + WARN_ON(!node); + WARN_ON(node->params.ring != ring); + WARN_ON(node->status != i915_sqs_popped); + count++; + + /* + * The call to pop above will have removed the node from the + * list. So add it back in and mark it as in flight. + */ + i915_scheduler_node_fly(node); + + spin_unlock_irq(&scheduler->lock); + ret = dev_priv->gt.execbuf_final(&node->params); + spin_lock_irq(&scheduler->lock); + + /* + * Handle failed submission but first check that the + * watchdog/reset code has not nuked the node while we + * weren't looking: + */ + if (ret && (node->status != i915_sqs_dead)) { + bool requeue = true; + + /* + * Oh dear! Either the node is broken or the ring is + * busy. So need to kill the node or requeue it and try
drivers/gpu/drm/i915/i915_scheduler.c line 364
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+* again later as appropriate. +*/ + +switch (-ret) { +case ENODEV : +case ENOENT : +/* Fatal errors. Kill the node. */ + requeue = false; +i915_scheduler_node_kill (node); +break ;
These break indents still.
Quoted 37 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +37
-0+ +case EAGAIN : +case EBUSY : +case EIO : +case ENOMEM : +case ERESTARTSYS : +case EINTR : +/* Supposedly recoverable errors. */ +break ; + +default : +/* +* Assume the error is recoverable and hope +* for the best. +*/ +MISSING_CASE (-ret); +break ; + } + +if (requeue) { +i915_scheduler_node_requeue (node); +/* +* No point spinning if the ring is currently +* unavailable so just give up and come back +* later. +*/ +break ; + } + } + +/* Keep launching until the sky is sufficiently full. */ + flying =i915_scheduler_count_flying (scheduler, ring); +if (flying >= scheduler->min_flying) +break ; + +/* Grab another node and go round again... */ + ret =i915_scheduler_pop_from_queue_locked (ring, &node);
drivers/gpu/drm/i915/i915_scheduler.c line 411
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+ }while (ret ==0 ); + +/* Don't complain about not being able to submit extra entries */ +if (ret == -ENODATA ) + ret =0 ; + +/* +* Bump the priority of everything that was not submitted to prevent +* starvation of low priority tasks by a spamming high priority task. +*/
This, this calls for an I-G-T test to make sure we're effective.
Quoted 35 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +35
-0+i915_scheduler_priority_bump_clear (scheduler); +list_for_each_entry (node, &scheduler->node_queue[ring->id], link) { +if (!I915_SQS_IS_QUEUED (node)) +continue ; + +i915_scheduler_priority_bump (scheduler, node, + scheduler->priority_level_bump); + } + +/* On success, return the number of buffers submitted. */ +if (ret ==0 ) + ret = count; + +error: + scheduler->flags[ring->id] &= ~i915_sf_submitting; +spin_unlock_irq (&scheduler->lock); +return ret; +} + +static void i915_generate_dependencies (struct i915_scheduler *scheduler, + struct i915_scheduler_queue_entry *node, + uint32_t ring) +{ +struct i915_scheduler_obj_entry *this, *that; +struct i915_scheduler_queue_entry *test; +int i, j; +bool found; + +list_for_each_entry (test, &scheduler->node_queue[ring], link) { +if (I915_SQS_IS_COMPLETE (test)) +continue ; + +/* +* Batches on the same ring for the same +* context must be kept in order.
drivers/gpu/drm/i915/i915_scheduler.c line 456
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+*/ + found = (node->params.ctx == test->params.ctx) && + (node->params.ring == test->params.ring); + +/* +* Batches working on the same objects must +* be kept in order. +*/ +for (i =0 ; (i < node->num_objs) && !found; i++) { + this = node->saved_objects + i;
node->objs and node->num_objs would be a better naming match. num_objs and saved_objects makes me think we have a bug here, indexing wrong array.
Quoted 23 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +23
-0+ +for (j =0 ; j < test->num_objs; j++) { + that = test->saved_objects + j; + +if (this->obj != that->obj) +continue ; + +/* Only need to worry about writes */ +if (this->read_only && that->read_only) +continue ; + + found = true; +break ; + } + } + +if (found) { + node->dep_list[node->num_deps] = test; + node->num_deps++; + } + } +} +
drivers/gpu/drm/i915/i915_scheduler.c line 489
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+static int i915_scheduler_queue_execbuffer_bypass (struct i915_scheduler_queue_entry *qe) +{ +struct drm_i915_private *dev_priv = qe->params.dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; +int ret; + + scheduler->flags[qe->params.ring->id] |= i915_sf_submitting; + ret = dev_priv->gt.execbuf_final (&qe->params); + scheduler->flags[qe->params.ring->id] &= ~i915_sf_submitting; +
The above code makes me think of locking again, maybe document at top of this function.
Quoted 32 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +32
-0+/* +* Don't do any clean up on failure because the caller will +* do it all anyway. +*/ +if (ret) +return ret; + +/* Need to release any resources held by the node: */ +i915_scheduler_clean_node (qe); + +return 0 ; +} + +/** +* i915_scheduler_queue_execbuffer - Submit a batch buffer request to the +* scheduler. +* @qe: The batch buffer request to be queued. +* The expectation is the qe passed in is a local stack variable. This +* function will copy its contents into a freshly allocated list node. The +* new node takes ownership of said contents so the original qe should simply +* be discarded and not cleaned up (i.e. don't free memory it points to or +* dereference objects it holds). The node is added to the scheduler's queue +* and the batch buffer will be submitted to the hardware at some future +* point in time (which may be immediately, before returning or may be quite +* a lot later). +*/ +int i915_scheduler_queue_execbuffer (struct i915_scheduler_queue_entry *qe) +{ +struct drm_i915_private *dev_priv = qe->params.dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; +struct intel_engine_cs *ring = qe->params.ring; +struct i915_scheduler_queue_entry *node;
drivers/gpu/drm/i915/i915_scheduler.c line 531
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+struct i915_scheduler_queue_entry *test; +bool not_flying; +int i, r; +int incomplete =0 ; + +WARN_ON (!scheduler); + +if (1 /*!i915.enable_scheduler*/ ) +return i915_scheduler_queue_execbuffer_bypass (qe); +
I'm thinking, should we branch already before calling a function named i915_scheduler_queue_execbuffer if scheduler is disabled?
drivers/gpu/drm/i915/i915_scheduler.c line 536
Quoted 5 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +5
-0+ node =kmalloc (sizeof (*node),GFP_KERNEL ); +if (!node) +return -ENOMEM ; + + *node = *qe;
Now I read the added documentation for the function, maybe we should at least zero out qe to avoid future problems?
Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +8
-0+INIT_LIST_HEAD (&node->link); + node->status = i915_sqs_queued; + node->stamp = jiffies; +i915_gem_request_reference (node->params.request); + +WARN_ON (node->params.request->scheduler_qe); + node->params.request->scheduler_qe = node; +
drivers/gpu/drm/i915/i915_scheduler.c line 554
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+/* Need to determine the number of incomplete entries in the list as +* that will be the maximum size of the dependency list. +* +* Note that the allocation must not be made with the spinlock acquired +* as kmalloc can sleep. However, the unlock/relock is safe because no +* new entries can be queued up during the unlock as the i915 driver +* mutex is still held. Entries could be removed from the list but that +* just means the dep_list will be over-allocated which is fine. +*/ +spin_lock_irq (&scheduler->lock);
-->
drivers/gpu/drm/i915/i915_scheduler.c line 563
Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +9
-0+for (r =0 ; r <I915_NUM_RINGS ; r++) { +list_for_each_entry (test, &scheduler->node_queue[r], link) { +if (I915_SQS_IS_COMPLETE (test)) +continue ; + + incomplete++; + } + } +
<-- This counting structure, I still think it would be good idea to make it a static helper.
Quoted 23 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +23
-0+/* Temporarily unlock to allocate memory: */ +spin_unlock_irq (&scheduler->lock); +if (incomplete) { + node->dep_list =kmalloc_array (incomplete, + sizeof(*node->dep_list), + GFP_KERNEL); +if (!node->dep_list) { +kfree (node); +return -ENOMEM ; + } + }else + node->dep_list =NULL ; + +spin_lock_irq (&scheduler->lock); + node->num_deps =0 ; + +if (node->dep_list) { +for (r =0 ; r <I915_NUM_RINGS ; r++) +i915_generate_dependencies (scheduler, node, r); + +WARN_ON (node->num_deps > incomplete); + } +
drivers/gpu/drm/i915/i915_scheduler.c line 596
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+ node->priority =clamp (node->priority, + scheduler->priority_level_min, + scheduler->priority_level_max); + +if ((node->priority >0 ) && node->num_deps) { +i915_scheduler_priority_bump_clear (scheduler); + +for (i =0 ; i < node->num_deps; i++) +i915_scheduler_priority_bump (scheduler, + node->dep_list[i], node->priority);
If node is posted with maximum priority to begin with, wouldn't the priority propagation stop after first level dependencies due to the check in the beginning of priority_bump function?
Quoted 122 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +122
-0+ +list_add_tail (&node->link, &scheduler->node_queue[ring->id]); + + not_flying =i915_scheduler_count_flying (scheduler, ring) < + scheduler->min_flying; + +spin_unlock_irq (&scheduler->lock); + +if (not_flying) +i915_scheduler_submit (ring); + +return 0 ; +} + +/** +* i915_scheduler_notify_request - Notify the scheduler that the given +* request has completed on the hardware. +* @req: Request structure which has completed +* @preempt: Did it complete pre-emptively? +* A sequence number has popped out of the hardware and the request handling +* code has mapped it back to a request and will mark that request complete. +* It also calls this function to notify the scheduler about the completion +* so the scheduler's node can be updated appropriately. +* Returns true if the request is scheduler managed, false if not. The return +* value is combined for all freshly completed requests and if any were true +* then i915_scheduler_wakeup() is called so the scheduler can do further +* processing (submit more work) at the end. +*/ +bool i915_scheduler_notify_request (struct drm_i915_gem_request *req) +{ +struct drm_i915_private *dev_priv =to_i915 (req->ring->dev); +struct i915_scheduler *scheduler = dev_priv->scheduler; +struct i915_scheduler_queue_entry *node = req->scheduler_qe; +unsigned long flags; + +if (!node) +return false; + +spin_lock_irqsave (&scheduler->lock, flags); + +WARN_ON (!I915_SQS_IS_FLYING (node)); + +/* Node was in flight so mark it as complete. */ +if (req->cancelled) + node->status = i915_sqs_dead; +else + node->status = i915_sqs_complete; + +spin_unlock_irqrestore (&scheduler->lock, flags); + +return true; +} + +static int i915_scheduler_remove_dependent (struct i915_scheduler *scheduler, +struct i915_scheduler_queue_entry *remove) +{ +struct i915_scheduler_queue_entry *node; +int i, r; +int count =0 ; + +/* +* Ensure that a node is not being removed which is still dependent +* upon other (not completed) work. If that happens, it implies +* something has gone very wrong with the dependency tracking! Note +* that there is no need to worry if this node has been explicitly +* killed for some reason - it might be being killed before it got +* sent to the hardware. +*/ +if (remove->status != i915_sqs_dead) { +for (i =0 ; i < remove->num_deps; i++) +if ((remove->dep_list[i]) && + (!I915_SQS_IS_COMPLETE(remove->dep_list[i]))) + count++; +WARN_ON (count); + } + +/* +* Remove this node from the dependency lists of any other node which +* might be waiting on it. +*/ +for (r =0 ; r <I915_NUM_RINGS ; r++) { +list_for_each_entry (node, &scheduler->node_queue[r], link) { +for (i =0 ; i < node->num_deps; i++) { +if (node->dep_list[i] != remove) +continue ; + + node->dep_list[i] =NULL ; + } + } + } + +return 0 ; +} + +/** +* i915_scheduler_wakeup - wake the scheduler's worker thread +* @dev: DRM device +* Called at the end of seqno interrupt processing if any request has +* completed that corresponds to a scheduler node. +*/ +void i915_scheduler_wakeup (struct drm_device *dev) +{ +/* XXX: Need to call i915_scheduler_remove() via work handler. */ +} + +/** +* i915_scheduler_clean_node - free up any allocations/references +* associated with the given scheduler queue entry. +* @node: Queue entry structure which is complete +* After a give batch buffer completes on the hardware, all the information +* required to resubmit it is no longer required. However, the node entry +* itself might still be required for tracking purposes for a while longer. +* This function should be called as soon as the node is known to be complete +* so that these resources may be freed even though the node itself might +* hang around. +*/ +void i915_scheduler_clean_node (struct i915_scheduler_queue_entry *node) +{ +if (!I915_SQS_IS_COMPLETE (node)) { +WARN (!node->params.request->cancelled, + "Cleaning active node: %d!\n", node->status); +return ;
drivers/gpu/drm/i915/i915_scheduler.c line 729
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+ } + +if (node->params.batch_obj) { +/* +* The batch buffer must be unpinned before it is unreferenced +* otherwise the unpin fails with a missing vma!? +*/ +if (node->params.dispatch_flags &I915_DISPATCH_SECURE ) +i915_gem_execbuff_release_batch_obj (node->params.batch_obj); +
This code seems little bit out of place.
drivers/gpu/drm/i915/i915_scheduler.c line 739
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+ node->params.batch_obj =NULL ; + } + +/* And anything else owned by the node: */ +if (node->params.cliprects) { +kfree (node->params.cliprects); + node->params.cliprects =NULL ; + } +} +
The below function is quite hacky, I understood this will be mitigated by per-ctx sequence numbering in the future?
Quoted 70 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +70
-0+static bool i915_scheduler_remove (struct i915_scheduler *scheduler, + struct intel_engine_cs *ring, + struct list_head *remove) +{ +struct i915_scheduler_queue_entry *node, *node_next; +int flying =0 , queued =0 ; +bool do_submit; +uint32_t min_seqno; + +spin_lock_irq (&scheduler->lock); + +/* +* In the case where the system is idle, starting 'min_seqno' from a big +* number will cause all nodes to be removed as they are now back to +* being in-order. However, this will be a problem if the last one to +* complete was actually out-of-order as the ring seqno value will be +* lower than one or more completed buffers. Thus code looking for the +* completion of said buffers will wait forever. +* Instead, use the hardware seqno as the starting point. This means +* that some buffers might be kept around even in a completely idle +* system but it should guarantee that no-one ever gets confused when +* waiting for buffer completion. +*/ + min_seqno = ring->get_seqno (ring, true); + +list_for_each_entry (node, &scheduler->node_queue[ring->id], link) { +if (I915_SQS_IS_QUEUED (node)) + queued++; +else if (I915_SQS_IS_FLYING (node)) + flying++; +else if (I915_SQS_IS_COMPLETE (node)) +continue ; + +if (node->params.request->seqno ==0 ) +continue ; + +if (!i915_seqno_passed (node->params.request->seqno, min_seqno)) + min_seqno = node->params.request->seqno; + } + +INIT_LIST_HEAD (remove); +list_for_each_entry_safe (node, node_next, &scheduler->node_queue[ring->id], link) { +/* +* Only remove completed nodes which have a lower seqno than +* all pending nodes. While there is the possibility of the +* ring's seqno counting backwards, all higher buffers must +* be remembered so that the 'i915_seqno_passed()' test can +* report that they have in fact passed. +* +* NB: This is not true for 'dead' nodes. The GPU reset causes +* the software seqno to restart from its initial value. Thus +* the dead nodes must be removed even though their seqno values +* are potentially vastly greater than the current ring seqno. +*/ +if (!I915_SQS_IS_COMPLETE (node)) +continue ; + +if (node->status != i915_sqs_dead) { +if (i915_seqno_passed (node->params.request->seqno, min_seqno) && + (node->params.request->seqno != min_seqno)) +continue ; + } + +list_del (&node->link); +list_add (&node->link, remove); + +/* Strip the dependency info while the mutex is still locked */ +i915_scheduler_remove_dependent (scheduler, node); + +continue ;
drivers/gpu/drm/i915/i915_scheduler.c line 819
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+ } + +/* +* Release the interrupt reference count if there are no longer any +* nodes to worry about. +*/ +if (!flying && !queued && + (scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) { + ring->irq_put (ring); + scheduler->flags[ring->id] &= ~i915_sf_interrupts_enabled;
Maybe have this piece of code as a helper? To enable interrupts and disable them, maybe even a reference count?
Quoted 38 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +38
-0+ } + +/* Launch more packets now? */ + do_submit = (queued >0 ) && (flying < scheduler->min_flying); + +spin_unlock_irq (&scheduler->lock); + +return do_submit; +} + +void i915_scheduler_process_work (struct intel_engine_cs *ring) +{ +struct drm_i915_private *dev_priv = ring->dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; +struct i915_scheduler_queue_entry *node; +bool do_submit; +struct list_head remove; + +if (list_empty (&scheduler->node_queue[ring->id])) +return ; + +/* Remove completed nodes. */ + do_submit =i915_scheduler_remove (scheduler, ring, &remove); + +if (!do_submit &&list_empty (&remove)) +return ; + +/* Need to grab the pm lock outside of the mutex lock */ +if (do_submit) +intel_runtime_pm_get (dev_priv); + +mutex_lock (&ring->dev->struct_mutex); + +if (do_submit) +i915_scheduler_submit (ring); + +while (!list_empty (&remove)) { + node =list_first_entry (&remove,typeof (*node), link);
drivers/gpu/drm/i915/i915_scheduler.c line 867
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+list_del (&node->link); + +/* Free up all the DRM references */ +i915_scheduler_clean_node (node); + +/* And anything else owned by the node: */ + node->params.request->scheduler_qe =NULL ; +i915_gem_request_unreference (node->params.request); +kfree (node->dep_list); +kfree (node);
Maybe have a separate helper for freeing a node, would make this much cleaner.
Quoted 47 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7
-0+ } + +mutex_unlock (&ring->dev->struct_mutex); + +if (do_submit) +intel_runtime_pm_put (dev_priv); +}drivers/gpu/drm/i915/i915_scheduler.h new file +34
-0@@ -0,0 +1,95 @@ +/* +* Copyright (c) 2014 Intel Corporation +* +* Permission is hereby granted, free of charge, to any person obtaining a +* copy of this software and associated documentation files (the "Software"), +* to deal in the Software without restriction, including without limitation +* the rights to use, copy, modify, merge, publish, distribute, sublicense, +* and/or sell copies of the Software, and to permit persons to whom the +* Software is furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice (including the next +* paragraph) shall be included in all copies or substantial portions of the +* Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +* IN THE SOFTWARE. +* +*/ + +#ifndef _I915_SCHEDULER_H_ +#define _I915_SCHEDULER_H_ + +enum i915_scheduler_queue_status { +/* Limbo: */ + i915_sqs_none =0 , +/* Not yet submitted to hardware: */ + i915_sqs_queued, +/* Popped from queue, ready to fly: */ + i915_sqs_popped,
drivers/gpu/drm/i915/i915_scheduler.h line 44
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +10
-0+/* Sent to hardware for processing: */ + i915_sqs_flying, +/* Finished processing on the hardware: */ + i915_sqs_complete, +/* Killed by watchdog or catastrophic submission failure: */ + i915_sqs_dead, +/* Limit value for use with arrays/loops */ + i915_sqs_MAX +}; +
To uppercase.
Quoted 12 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +12
-0+#define I915_SQS_IS_QUEUED (node) (((node)->status == i915_sqs_queued)) +#define I915_SQS_IS_FLYING (node) (((node)->status == i915_sqs_flying)) +#define I915_SQS_IS_COMPLETE (node) (((node)->status == i915_sqs_complete) || \ + ((node)->status == i915_sqs_dead)) + +struct i915_scheduler_obj_entry { + struct drm_i915_gem_object *obj; + bool read_only; +}; + +struct i915_scheduler_queue_entry { + struct i915_execbuffer_params params;
drivers/gpu/drm/i915/i915_scheduler.h line 66
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +10
-0+/* -1023 = lowest priority, 0 = default, 1023 = highest */ + int32_t priority; + struct i915_scheduler_obj_entry *saved_objects; + int num_objs; + bool bumped; + struct i915_scheduler_queue_entry **dep_list; + int num_deps; + enum i915_scheduler_queue_status status; + unsigned long stamp; + struct list_head link;
Above whitespace is still incorrect and I could really use comments.
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +10
-0+}; + +struct i915_scheduler { + struct list_head node_queue[I915_NUM_RINGS]; + uint32_t flags[I915_NUM_RINGS]; + spinlock_t lock; + +/* Tuning parameters: */ + int32_t priority_level_min; + int32_t priority_level_max;
drivers/gpu/drm/i915/i915_scheduler.h line 86
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +10
-0+ int32_t priority_level_bump; + int32_t priority_level_preempt; + uint32_t min_flying; +}; + +/* Flag bits for i915_scheduler::flags */ +enum { + i915_sf_interrupts_enabled = (1 << 0), + i915_sf_submitting = (1 << 1), +};
These must be I915_UPPERCASE_SOMETHING by convention.
Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +9
-0+ +bool i915_scheduler_is_enabled (struct drm_device *dev); +int i915_scheduler_init (struct drm_device *dev); +void i915_scheduler_clean_node (struct i915_scheduler_queue_entry *node); +int i915_scheduler_queue_execbuffer (struct i915_scheduler_queue_entry *qe); +bool i915_scheduler_notify_request (struct drm_i915_gem_request *req); +void i915_scheduler_wakeup (struct drm_device *dev); + +#endif /* _I915_SCHEDULER_H_ */
Signature · 7 lines
Joonas Lahtinen Open Source Technology Center Intel Corporation _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 06/35] drm/i915: Start of GPU scheduler
Quoted 172 lines · Joonas Lahtinen
Hi,
Now the code is in reviewable chunks, excellent!
I've added my comments below. A few repeats from last round, but now with more questions about the logic itself.
Quoted 164 lines · On to, 2016-02-18 at 14:26 +0000, [email protected] wrote:
From: John Harrison <[email protected]>
Initial creation of scheduler source files. Note that this patch implements most of the scheduler functionality but does not hook it in to the driver yet. It also leaves the scheduler code in 'pass through' mode so that even when it is hooked in, it will not actually do very much. This allows the hooks to be added one at a time in bite size chunks and only when the scheduler is finally enabled at the end does anything start happening.
The general theory of operation is that when batch buffers are submitted to the driver, the
execbuffer()code packages up all the information required to execute the batch buffer at a later time. This package is given over to the scheduler which adds it to an internal node list. The scheduler also scans the list of objects associated with the batch buffer and compares them against the objects already in use by other buffers in the node list. If matches are found then the new batch buffer node is marked as being dependent upon the matching node. The same is done for the context object. The scheduler also bumps up the priority of such matching nodes on the grounds that the more dependencies a given batch buffer has the more important it is likely to be.The scheduler aims to have a given (tuneable) number of batch buffers in flight on the hardware at any given time. If fewer than this are currently executing when a new node is queued, then the node is passed straight through to the submit function. Otherwise it is simply added to the queue and the driver returns back to user land.
The scheduler is notified when each batch buffer completes and updates its internal tracking accordingly. At the end of the completion interrupt processing, if any scheduler tracked batches were processed, the scheduler's deferred worker thread is woken up. This can do more involved processing such as actually removing completed nodes from the queue and freeing up the resources associated with them (internal memory allocations, DRM object references, context reference, etc.). The work handler also checks the in flight count and calls the submission code if a new slot has appeared.
When the scheduler's submit code is called, it scans the queued node list for the highest priority node that has no unmet dependencies. Note that the dependency calculation is complex as it must take inter-ring dependencies and potential preemptions into account. Note also that in the future this will be extended to include external dependencies such as the Android Native Sync file descriptors and/or the linux dma-buff synchronisation scheme.
If a suitable node is found then it is sent to
execbuff_final()for submission to the hardware. The in flight count is then re-checked and a new node popped from the list if appropriate. All nodes that are not submitted have their priority bumped. This ensures that low priority tasks do not get starved out by busy higher priority ones - everything will eventually get its turn to run.Note that this patch does not implement pre-emptive scheduling. Only basic scheduling by re-ordering batch buffer submission is currently implemented. Pre-emption of actively executing batch buffers comes in the next patch series.
v2: Changed priority levels to +/-1023 due to feedback from Chris Wilson.
Removed redundant index from scheduler node.
Changed time stamps to use jiffies instead of raw monotonic. This provides lower resolution but improved compatibility with other i915 code.
Major re-write of completion tracking code due to struct fence conversion. The scheduler no longer has it's own private IRQ handler but just lets the existing request code handle completion events. Instead, the scheduler now hooks into the request notify code to be told when a request has completed.
Reduced driver mutex locking scope. Removal of scheduler nodes no longer grabs the mutex lock.
v3: Refactor of dependency generation to make the code more readable. Also added in read-read optimisation support - i.e., don't treat a shared read-only buffer as being a dependency.
Allowed the killing of queued nodes rather than only flying ones.
v4: Updated the commit message to better reflect the current state of the code. Downgraded some
BUG_ONstoWARN_ONs. Used the correct array memory allocator function (kmalloc_arrayinstead of kmalloc). Corrected the format of some comments. Wrapped some lines differently to keep the style checker happy.Fixed a
WARN_ONwhen killing nodes. The dependency removal code checks that nodes being destroyed do not have any oustanding dependencies (which would imply they should not have been executed yet). In the case of nodes being destroyed, e.g. due to context banning, then this might well be the case - they have not been executed and do indeed have outstanding dependencies.Re-instated the code to disble interrupts when not in use. The underlying problem causing broken IRQ reference counts seems to have been fixed now.
v5: Shuffled various functions around to remove forward declarations as apparently these are frowned upon. Removed lots of white space as apparently having easy to read code is also frowned upon. Split the direct submission scheduler bypass code out into a separate function. Squashed down the
i915_scheduler.csections of various patches into this patch. Thus the later patches simply hook in existing code into various parts of the driver rather than adding the code as well. Added documentation to various functions. Re-worked the submit function in terms of mutex locking, error handling and exit paths. Split the delayed work handler function in half. Made use of the kernel 'clamp' macro. [Joonas Lahtinen]Added runtime PM calls as these must be done at the top level before acquiring the driver mutex lock. [Chris Wilson]
Removed some obsolete debug code that had been forgotten about.
Moved more clean up code into the '
i915_gem_scheduler_clean_node()' function rather than replicating it in mutliple places.Used lighter weight spinlocks.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>Cc Joonas Lahtinen <[email protected]>--- drivers/gpu/drm/i915/Makefile | 1 + drivers/gpu/drm/i915/i915_drv.h | 6 + drivers/gpu/drm/i915/i915_gem.c | 5 + drivers/gpu/drm/i915/i915_scheduler.c | 874 ++++++++++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_scheduler.h | 95 ++++ 5 files changed, 981 insertions(+)create mode 100644
drivers/gpu/drm/i915/i915_scheduler.ccreate mode 100644drivers/gpu/drm/i915/i915_scheduler.hdrivers/gpu/drm/i915/Makefile +1
-0@@ -10,6 +10,7 @@ ccflags-y := -Werror i915-y := i915_drv.o \ i915_irq.o \ i915_params.o \ + i915_scheduler.o \ i915_suspend.o \ i915_sysfs.o \ intel_csr.o \drivers/gpu/drm/i915/i915_drv.h +2
-0@@ -1703,6 +1703,8 @@ struct i915_execbuffer_params {struct drm_i915_gem_request *request; }; +struct i915_scheduler ; +/* used in computing the new watermarks state */ struct intel_wm_config {unsigned int num_pipes_active;@@ -1968,6 +1970,8 @@ struct drm_i915_private {struct i915_runtime_pm pm;
Quoted 7 lines
drivers/gpu/drm/i915/i915_drv.h struct drm_i915_private { · line 1974
Quoted 2 lines
drivers/gpu/drm/i915/i915_drv.h +2
-0+struct i915_scheduler *scheduler; +I think we should have i915.enable_scheduler parameter (just like i915.enable_execlists) and make this a data member, not pointer.
Then we can go forward and have i915.enable_scheduler_preempt and so on to control things at boot time.
There are. The enable_scheduler comes in as patch #21. Including it from the start means modifying i915_params.[ch] in this patch as well. And then you are getting into one big patch that contains everything and is unreviewable due to its sheer size. The enable_preemption parameter comes in with the pre-emption code. No point in adding that until there is something for it to control.
Quoted 75 lines
drivers/gpu/drm/i915/i915_drv.h struct drm_i915_gem_request { · line 2298
Quoted 10 lines
drivers/gpu/drm/i915/i915_drv.h +2
-0/* Abstract the submission mechanism (legacy ringbuffer or execlists) away */ struct { int (*execbuf_submit)(struct i915_execbuffer_params *params,@@ -2290,6 +2294,8 @@ struct drm_i915_gem_request {/** process identifier submitting this request */ struct pid *pid; +struct i915_scheduler_queue_entry *scheduler_qe; +Funny whitespace.
Quoted 63 lines
drivers/gpu/drm/i915/i915_drv.h +0
-0/** * The ELSP only accepts two elements at a time, so we queue * context/tail pairs on a given queue (ring->execlist_queue) until the drivers/gpu/drm/i915/i915_gem.c +5
-0@@ -32,6 +32,7 @@#include "i915_vgpu.h" #include "i915_trace.h" #include "intel_drv.h" +#include "i915_scheduler.h" #include #include #include@@ -5315,6 +5316,10 @@ int i915_gem_init(struct drm_device *dev)*/ intel_uncore_forcewake_get (dev_priv,FORCEWAKE_ALL ); + ret =i915_scheduler_init (dev); +if (ret) + goto out_unlock; + ret =i915_gem_init_userptr (dev);if (ret) goto out_unlock;drivers/gpu/drm/i915/i915_scheduler.c new file +31
-0@@ -0,0 +1,874 @@ +/* +* Copyright (c) 2014 Intel Corporation +* +* Permission is hereby granted, free of charge, to any person obtaining a +* copy of this software and associated documentation files (the "Software"), +* to deal in the Software without restriction, including without limitation +* the rights to use, copy, modify, merge, publish, distribute, sublicense, +* and/or sell copies of the Software, and to permit persons to whom the +* Software is furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice (including the next +* paragraph) shall be included in all copies or substantial portions of the +* Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +* IN THE SOFTWARE. +* +*/ + +#include "i915_drv.h" +#include "intel_drv.h" +#include "i915_scheduler.h" + +/** +* i915_scheduler_is_enabled - Returns true if the scheduler is enabled. +* @dev: DRM device
Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c line 38
Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7
-0+*/ +bool i915_scheduler_is_enabled (struct drm_device *dev) +{ +struct drm_i915_private *dev_priv = dev->dev_private; + +return dev_priv->scheduler !=NULL ; +}i915.enable_scheduler would be used instead.
It is when it arrives.
Quoted 15 lines
drivers/gpu/drm/i915/i915_scheduler.c line 50
Quoted 13 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +12
-0+ +/** +* i915_scheduler_init - Initialise the scheduler. +* @dev: DRM device +* Returns zero on success or -ENOMEM if memory allocations fail. +*/ +int i915_scheduler_init (struct drm_device *dev) +{ +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; +int r; +This protection -->
Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c line 57
Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7
-0+if (scheduler) +return 0 ; + + scheduler =kzalloc (sizeof (*scheduler),GFP_KERNEL ); +if (!scheduler) +return -ENOMEM ; +Is not needed if we have data member and i915.enable_scheduler.
Making the scheduler structure a object rather than a pointer basically means moving the whole of i915_scheduler.h into i915_drv.h. There is too much interconnectedness between the various structures. I would say that keeping the code nicely partitioned and readable is worth the one dynamic allocation at driver load time.
Quoted 21 lines
Quoted 21 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +20
-0+spin_lock_init (&scheduler->lock); + +for (r =0 ; r <I915_NUM_RINGS ; r++) +INIT_LIST_HEAD (&scheduler->node_queue[r]); + +/* Default tuning values: */ + scheduler->priority_level_min =-1023 ; + scheduler->priority_level_max =1023 ; + scheduler->priority_level_bump =50 ; + scheduler->priority_level_preempt =900 ; + scheduler->min_flying =2 ; + + dev_priv->scheduler = scheduler; + +return 0 ; +} + +/* +* Add a popped node back in to the queue. For example, because the ring was +* hung when execfinal() was called and thus the ring submission needs to be
Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c line 82
Quoted 5 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +5
-0+* retried later. +*/ +static void i915_scheduler_node_requeue (struct i915_scheduler_queue_entry *node) +{ +WARN_ON (!node);Rather remove this completely as it is an internal function, or use
BUG_ONbecause next line will straight lead to OOPS after warning. I'll pass mentioning rest of the obviousWARN_ONvs BUG_ON's.
Yeah, most of the WARN/BUG_ONs are probably unnecessary now. They were scattered liberally around during initial development when strange and unexpected things could happen. Now the the code paths are pretty well established they aren't really much use. I'll reduce them down to just the ones that still make sense.
Quoted 57 lines
Quoted 11 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+WARN_ON (!I915_SQS_IS_FLYING (node)); + +/* Seqno will be reassigned on relaunch */ + node->params.request->seqno =0 ; + node->status = i915_sqs_queued; +} + +/* +* Give up on a node completely. For example, because it is causing the +* ring to hang or is using some resource that no longer exists. drivers/gpu/drm/i915/i915_scheduler.c line 102
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+*/ +static void i915_scheduler_node_kill (struct i915_scheduler_queue_entry *node) +{ +WARN_ON (!node); +WARN_ON (I915_SQS_IS_COMPLETE (node)); + + node->status = i915_sqs_dead; +} + +/* Mark a node as in flight on the hardware. */ As for documentation, from below, I only assume scheduler lock must be held and node's can be only manipulated when scheduler lock is held? It's good to add a comment describing the expected locking. Or maybe add
assert_scheduler_lock_held()helper and sprinkle it around the functions similar toassert_rpm_wakelock_held(), which is self- documenting?drivers/gpu/drm/i915/i915_scheduler.c line 105
Quoted 3 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +3
-0+static int i915_scheduler_node_fly (struct i915_scheduler_queue_entry *node) +{ +struct drm_i915_private *dev_priv = node->params.dev->dev_private;If node is NULL, this would already OOPS, I do not think it's unreasonable to expect node not to be NULL. The below
WARN_ONcheck is never reached.drivers/gpu/drm/i915/i915_scheduler.c line 114
Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +9
-0+struct i915_scheduler *scheduler = dev_priv->scheduler; +struct intel_engine_cs *ring; + +WARN_ON (!scheduler); +WARN_ON (!node); +WARN_ON (node->status != i915_sqs_popped); + + ring = node->params.ring; +This and similar assignments can be squashed to declaration line, as this is basically an alias.
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+/* +* Add the node (which should currently be in state popped) to the +* front of the queue. This ensure that flying nodes are always held +* in hardware submission order. +*/ +list_add (&node->link, &scheduler->node_queue[ring->id]); + + node->status = i915_sqs_flying; + +if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c line 131
Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7
-0+bool success = true; + + success = ring->irq_get (ring); +if (success) + scheduler->flags[ring->id] |= i915_sf_interrupts_enabled; +else +return -EINVAL ;Shouldn't the node be cleaned up from
node_queuein case of an error?
Not really. The batch buffer will still be executed regardless of whether interrupts could be enabled or not. The return code is actually ignored so this should really be a void function, I guess. Without interrupts then completion might take a while to be noticed but stuff should still basically work.
Quoted 59 lines
Also, I think above could be written much more compact form (because it looks like something where more logic will be added later). It makes it easier to write and visualize the error paths when reading if there are no nested if's.
I won't mention about the error paths of functions below, I expect the following guidline to be adhered;
if (scheduler->flags[ring->id] & I915_SF_INT_ENABLED) goto out; if (!ring->irq_get(ring)) goto err_irq; if (!...) goto err_foo; scheduler->flags[ring->id] |= I915_SF_INT_ENABLED; out: return 0; err_foo: foobar(); err_irq: list_remove() return -EINVAL;drivers/gpu/drm/i915/i915_scheduler.c line 143
Quoted 12 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +12
-0+ } + +return 0 ; +} + +static uint32_t i915_scheduler_count_flying (struct i915_scheduler *scheduler, +struct intel_engine_cs *ring) +{ +struct i915_scheduler_queue_entry *node; +uint32_t flying =0 ; + +list_for_each_entry (node, &scheduler->node_queue[ring->id], link)+1 for the
for_each_scheduler_node(...)Quoted 18 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +18
-0+if (I915_SQS_IS_FLYING (node)) + flying++; + +return flying; +} + +static void i915_scheduler_priority_bump_clear (struct i915_scheduler *scheduler) +{ +struct i915_scheduler_queue_entry *node; +int i; + +/* +* Ensure circular dependencies don't cause problems and that a bump +* by object usage only bumps each using buffer once: +*/ +for (i =0 ; i <I915_NUM_RINGS ; i++) { +list_for_each_entry (node, &scheduler->node_queue[i], link) + node->bumped = false;
Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c line 166
Quoted 5 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +5
-0+ } +} + +static int i915_scheduler_priority_bump(struct i915_scheduler *scheduler, + struct i915_scheduler_queue_entry *target,Is there specific reason why struct
i915_scheduler_queue_entryare not referred to just as "node" but "qe" and here something else, do "node" and "qe" have a special semantic meaning?
Not sure what you mean. The use of 'node' is because almost everything an i915_scheduler_queue_entry object is being used it is as a node in a list that is being iterated over. The only 'qe' variable is in 'i915_scheduler_queue_execbuffer[_bypass]' and there it is because the qe is being passed in from outside and is copied to a local 'node' that is then added to the list. I think the use of 'target' here is largely historical. The subsequent array search was originally a list search and hence had a 'node' that was used as the iterator. Thus the qe passed in was called something else to avoid conflicts and target seemed appropriate.
Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.c line 173
Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7
-0+ uint32_t bump) +{ +uint32_t new_priority; +int i, count; + +if (target->priority >= scheduler->priority_level_max) +return 1 ;So if one node reaches maximum priority the dependencies are expected to be maxed out already?
Yes. Dependencies should always be of equal or higher priority to the dependee. When a new dependency is created, the priority of the dependee is added on to ensure that something of high priority can never be stuck waiting on something of low priority. Also, it means that a batch buffer that lots of other buffers are waiting on will be bumped to higher and higher priority the more waiters there are and thus is more likely to be executed sooner and free them all up.
Quoted 236 lines
Quoted 110 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +110
-0+ +if (target->bumped) +return 0 ; + + new_priority = target->priority + bump; +if ((new_priority <= target->priority) || + (new_priority > scheduler->priority_level_max)) + target->priority = scheduler->priority_level_max; +else + target->priority = new_priority; + + count =1 ; + target->bumped = true; + +for (i =0 ; i < target->num_deps; i++) { +if (!target->dep_list[i]) +continue ; + +if (target->dep_list[i]->bumped) +continue ; + + count +=i915_scheduler_priority_bump (scheduler, + target->dep_list[i], + bump); + } + +return count; +} + +/* +* Nodes are considered valid dependencies if they are queued on any ring or +* if they are in flight on a different ring. In flight on the same ring is no +* longer interesting for non-premptive nodes as the ring serialises execution. +* For pre-empting nodes, all in flight dependencies are valid as they must not +* be jumped by the act of pre-empting. +* +* Anything that is neither queued nor flying is uninteresting. +*/ +static inline bool i915_scheduler_is_dependency_valid ( +struct i915_scheduler_queue_entry *node,uint32_t idx) +{ +struct i915_scheduler_queue_entry *dep; + + dep = node->dep_list[idx]; +if (!dep) +return false; + +if (I915_SQS_IS_QUEUED (dep)) +return true; + +if (I915_SQS_IS_FLYING (dep)) { +if (node->params.ring != dep->params.ring) +return true; + } + +return false; +} + +static int i915_scheduler_pop_from_queue_locked (struct intel_engine_cs *ring, +struct i915_scheduler_queue_entry **pop_node) +{ +struct drm_i915_private *dev_priv = ring->dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; +struct i915_scheduler_queue_entry *best =NULL ; +struct i915_scheduler_queue_entry *node; +int ret; +int i; +bool any_queued = false; +bool has_local, has_remote, only_remote = false; + + *pop_node =NULL ; + ret = -ENODATA ; + +list_for_each_entry (node, &scheduler->node_queue[ring->id], link) { +if (!I915_SQS_IS_QUEUED (node)) +continue ; + any_queued = true; + + has_local = false; + has_remote = false; +for (i =0 ; i < node->num_deps; i++) { +if (!i915_scheduler_is_dependency_valid (node, i)) +continue ; + +if (node->dep_list[i]->params.ring == node->params.ring) + has_local = true; +else + has_remote = true; + } + +if (has_remote && !has_local) + only_remote = true; + +if (!has_local && !has_remote) { +if (!best || + (node->priority > best->priority)) + best = node; + } + } + +if (best) { +list_del (&best->link); + +INIT_LIST_HEAD (&best->link); + best->status = i915_sqs_popped; + + ret =0 ; + }else { +/* Can only get here if: +* (a) there are no buffers in the queue drivers/gpu/drm/i915/i915_scheduler.c line 293
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+* (b) all queued buffers are dependent on other buffers +* e.g. on a buffer that is in flight on a different ring +*/ +if (only_remote) { +/* The only dependent buffers are on another ring. */ + ret = -EAGAIN ; + }else if (any_queued) { +/* It seems that something has gone horribly wrong! */ +DRM_ERROR ("Broken dependency tracking on ring %d!\n" , + (int ) ring->id);Would this condition be worthy
WARN_ONCE(), because it will keep repeating as the situation will persist?Quoted 61 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +61
-0+ } + } + + *pop_node = best; +return ret; +} + +/* +* NB: The driver mutex lock must be held before calling this function. It is +* only really required during the actual back end submission call. However, +* attempting to acquire a mutex while holding a spin lock is a Bad Idea. +* And releasing the one before acquiring the other leads to other code +* being run and interfering. +*/ +static int i915_scheduler_submit (struct intel_engine_cs *ring) +{ +struct drm_device *dev = ring->dev; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; +struct i915_scheduler_queue_entry *node; +int ret, count =0 , flying; + +WARN_ON (!mutex_is_locked (&dev->struct_mutex)); + +spin_lock_irq (&scheduler->lock); + +WARN_ON (scheduler->flags[ring->id] & i915_sf_submitting); + scheduler->flags[ring->id] |= i915_sf_submitting; + +/* First time around, complain if anything unexpected occurs: */ + ret =i915_scheduler_pop_from_queue_locked (ring, &node); +if (ret) + goto error; + + do { + WARN_ON(!node); + WARN_ON(node->params.ring != ring); + WARN_ON(node->status != i915_sqs_popped); + count++; + + /* + * The call to pop above will have removed the node from the + * list. So add it back in and mark it as in flight. + */ + i915_scheduler_node_fly(node); + + spin_unlock_irq(&scheduler->lock); + ret = dev_priv->gt.execbuf_final(&node->params); + spin_lock_irq(&scheduler->lock); + + /* + * Handle failed submission but first check that the + * watchdog/reset code has not nuked the node while we + * weren't looking: + */ + if (ret && (node->status != i915_sqs_dead)) { + bool requeue = true; + + /* + * Oh dear! Either the node is broken or the ring is + * busy. So need to kill the node or requeue it and trydrivers/gpu/drm/i915/i915_scheduler.c line 364
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+* again later as appropriate. +*/ + +switch (-ret) { +case ENODEV : +case ENOENT : +/* Fatal errors. Kill the node. */ + requeue = false; +i915_scheduler_node_kill (node); +break ;These break indents still.
Quoted 40 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +40
-0+ +case EAGAIN : +case EBUSY : +case EIO : +case ENOMEM : +case ERESTARTSYS : +case EINTR : +/* Supposedly recoverable errors. */ +break ; + +default : +/* +* Assume the error is recoverable and hope +* for the best. +*/ +MISSING_CASE (-ret); +break ; + } + +if (requeue) { +i915_scheduler_node_requeue (node); +/* +* No point spinning if the ring is currently +* unavailable so just give up and come back +* later. +*/ +break ; + } + } + +/* Keep launching until the sky is sufficiently full. */ + flying =i915_scheduler_count_flying (scheduler, ring); +if (flying >= scheduler->min_flying) +break ; + +/* Grab another node and go round again... */ + ret =i915_scheduler_pop_from_queue_locked (ring, &node); + }while (ret ==0 ); + +/* Don't complain about not being able to submit extra entries */
Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c line 411
Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7
-0+if (ret == -ENODATA ) + ret =0 ; + +/* +* Bump the priority of everything that was not submitted to prevent +* starvation of low priority tasks by a spamming high priority task. +*/ This, this calls for an I-G-T test to make sure we're effective.
Yeah, IGTs are under development.
Quoted 76 lines
Quoted 35 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +35
-0+i915_scheduler_priority_bump_clear (scheduler); +list_for_each_entry (node, &scheduler->node_queue[ring->id], link) { +if (!I915_SQS_IS_QUEUED (node)) +continue ; + +i915_scheduler_priority_bump (scheduler, node, + scheduler->priority_level_bump); + } + +/* On success, return the number of buffers submitted. */ +if (ret ==0 ) + ret = count; + +error: + scheduler->flags[ring->id] &= ~i915_sf_submitting; +spin_unlock_irq (&scheduler->lock); +return ret; +} + +static void i915_generate_dependencies (struct i915_scheduler *scheduler, +struct i915_scheduler_queue_entry *node, +uint32_t ring) +{ +struct i915_scheduler_obj_entry *this, *that; +struct i915_scheduler_queue_entry *test; +int i, j; +bool found; + +list_for_each_entry (test, &scheduler->node_queue[ring], link) { +if (I915_SQS_IS_COMPLETE (test)) +continue ; + +/* +* Batches on the same ring for the same +* context must be kept in order. drivers/gpu/drm/i915/i915_scheduler.c line 456
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+*/ + found = (node->params.ctx == test->params.ctx) && + (node->params.ring == test->params.ring); + +/* +* Batches working on the same objects must +* be kept in order. +*/ +for (i =0 ; (i < node->num_objs) && !found; i++) { + this = node->saved_objects + i;
node->objsandnode->num_objswould be a better naming match.num_objsandsaved_objectsmakes me think we have a bug here, indexing wrong array.Quoted 27 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +27
-0+ +for (j =0 ; j < test->num_objs; j++) { + that = test->saved_objects + j; + +if (this->obj != that->obj) +continue ; + +/* Only need to worry about writes */ +if (this->read_only && that->read_only) +continue ; + + found = true; +break ; + } + } + +if (found) { + node->dep_list[node->num_deps] = test; + node->num_deps++; + } + } +} + +static int i915_scheduler_queue_execbuffer_bypass (struct i915_scheduler_queue_entry *qe) +{ +struct drm_i915_private *dev_priv = qe->params.dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler;
Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c line 489
Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +6
-0+int ret; + + scheduler->flags[qe->params.ring->id] |= i915_sf_submitting; + ret = dev_priv->gt.execbuf_final (&qe->params); + scheduler->flags[qe->params.ring->id] &= ~i915_sf_submitting; +The above code makes me think of locking again, maybe document at top of this function.
No need to acquire the spinlock at this point. If the scheduler is in bypass mode then there is no internal state to protect. The driver mutex lock will be held because there is only one code path to get here which is the execbuff IOCTL. The whole point of bypass mode is that it is a single contiguous execution path from IOCTL entry all the way through to hardware submission. A mutex check could be added but it seems unnecessary given the limited calling scope.
Quoted 37 lines
Quoted 37 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +36
-0+/* +* Don't do any clean up on failure because the caller will +* do it all anyway. +*/ +if (ret) +return ret; + +/* Need to release any resources held by the node: */ +i915_scheduler_clean_node (qe); + +return 0 ; +} + +/** +* i915_scheduler_queue_execbuffer - Submit a batch buffer request to the +* scheduler. +* @qe: The batch buffer request to be queued. +* The expectation is the qe passed in is a local stack variable. This +* function will copy its contents into a freshly allocated list node. The +* new node takes ownership of said contents so the original qe should simply +* be discarded and not cleaned up (i.e. don't free memory it points to or +* dereference objects it holds). The node is added to the scheduler's queue +* and the batch buffer will be submitted to the hardware at some future +* point in time (which may be immediately, before returning or may be quite +* a lot later). +*/ +int i915_scheduler_queue_execbuffer (struct i915_scheduler_queue_entry *qe) +{ +struct drm_i915_private *dev_priv = qe->params.dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; +struct intel_engine_cs *ring = qe->params.ring; +struct i915_scheduler_queue_entry *node; +struct i915_scheduler_queue_entry *test; +bool not_flying; +int i, r; +int incomplete =0 ;
Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c line 531
Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +6
-0+ +WARN_ON (!scheduler); + +if (1 /*!i915.enable_scheduler*/ ) +return i915_scheduler_queue_execbuffer_bypass (qe); +I'm thinking, should we branch already before calling a function named
i915_scheduler_queue_execbufferif scheduler is disabled?
That would require putting if(scheduler) logic in random points throughout the driver. Doing it this way, the driver as a whole has a single execution path which is via the scheduler. It is then up to the scheduler itself to decide whether to run in bypass mode, simple re-ordering mode, full pre-emption mode or any other mode we feel a need to add.
Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c line 536
Quoted 5 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +5
-0+ node =kmalloc (sizeof (*node),GFP_KERNEL ); +if (!node) +return -ENOMEM ; + + *node = *qe;Now I read the added documentation for the function, maybe we should at least zero out qe to avoid future problems?
The execbuff success path hands the qe contents over and pretty much immediately drops all the way back out of the IOCTL. The failure path does the full cleanup of de-allocating all the qe stuff it had previously allocated. IMO there doesn't seem to be much need to zero out the structure at this point.
Quoted 61 lines
Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +8
-0+INIT_LIST_HEAD (&node->link); + node->status = i915_sqs_queued; + node->stamp = jiffies; +i915_gem_request_reference (node->params.request); + +WARN_ON (node->params.request->scheduler_qe); + node->params.request->scheduler_qe = node; +drivers/gpu/drm/i915/i915_scheduler.c line 554
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+/* Need to determine the number of incomplete entries in the list as +* that will be the maximum size of the dependency list. +* +* Note that the allocation must not be made with the spinlock acquired +* as kmalloc can sleep. However, the unlock/relock is safe because no +* new entries can be queued up during the unlock as the i915 driver +* mutex is still held. Entries could be removed from the list but that +* just means the dep_list will be over-allocated which is fine. +*/ +spin_lock_irq (&scheduler->lock);-->
drivers/gpu/drm/i915/i915_scheduler.c line 563
Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +9
-0+for (r =0 ; r <I915_NUM_RINGS ; r++) { +list_for_each_entry (test, &scheduler->node_queue[r], link) { +if (I915_SQS_IS_COMPLETE (test)) +continue ; + + incomplete++; + } + } +<-- This counting structure, I still think it would be good idea to make it a static helper.
Quoted 28 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +28
-0+/* Temporarily unlock to allocate memory: */ +spin_unlock_irq (&scheduler->lock); +if (incomplete) { + node->dep_list =kmalloc_array (incomplete, +sizeof (*node->dep_list), +GFP_KERNEL ); +if (!node->dep_list) { +kfree (node); +return -ENOMEM ; + } + }else + node->dep_list =NULL ; + +spin_lock_irq (&scheduler->lock); + node->num_deps =0 ; + +if (node->dep_list) { +for (r =0 ; r <I915_NUM_RINGS ; r++) +i915_generate_dependencies (scheduler, node, r); + +WARN_ON (node->num_deps > incomplete); + } + + node->priority =clamp (node->priority, + scheduler->priority_level_min, + scheduler->priority_level_max); + +if ((node->priority >0 ) && node->num_deps) {
Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c line 596
Quoted 5 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +5
-0+i915_scheduler_priority_bump_clear (scheduler); + +for (i =0 ; i < node->num_deps; i++) +i915_scheduler_priority_bump (scheduler, + node->dep_list[i], node->priority);If node is posted with maximum priority to begin with, wouldn't the priority propagation stop after first level dependencies due to the check in the beginning of
priority_bumpfunction?
The recursion will stop whenever it encounters a node already at maximum priority. However, it is not starting from the newly submitted node but from that node's dependencies. If they are already at max priority then their dependencies must also be and so on down the chain and no further processing is required. On the other hand, if they are not then they will be bumped and their dependencies checked and so on down the chain.
Quoted 125 lines
Quoted 125 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +125
-0+ +list_add_tail (&node->link, &scheduler->node_queue[ring->id]); + + not_flying =i915_scheduler_count_flying (scheduler, ring) < + scheduler->min_flying; + +spin_unlock_irq (&scheduler->lock); + +if (not_flying) +i915_scheduler_submit (ring); + +return 0 ; +} + +/** +* i915_scheduler_notify_request - Notify the scheduler that the given +* request has completed on the hardware. +* @req: Request structure which has completed +* @preempt: Did it complete pre-emptively? +* A sequence number has popped out of the hardware and the request handling +* code has mapped it back to a request and will mark that request complete. +* It also calls this function to notify the scheduler about the completion +* so the scheduler's node can be updated appropriately. +* Returns true if the request is scheduler managed, false if not. The return +* value is combined for all freshly completed requests and if any were true +* then i915_scheduler_wakeup() is called so the scheduler can do further +* processing (submit more work) at the end. +*/ +bool i915_scheduler_notify_request (struct drm_i915_gem_request *req) +{ +struct drm_i915_private *dev_priv =to_i915 (req->ring->dev); +struct i915_scheduler *scheduler = dev_priv->scheduler; +struct i915_scheduler_queue_entry *node = req->scheduler_qe; +unsigned long flags; + +if (!node) +return false; + +spin_lock_irqsave (&scheduler->lock, flags); + +WARN_ON (!I915_SQS_IS_FLYING (node)); + +/* Node was in flight so mark it as complete. */ +if (req->cancelled) + node->status = i915_sqs_dead; +else + node->status = i915_sqs_complete; + +spin_unlock_irqrestore (&scheduler->lock, flags); + +return true; +} + +static int i915_scheduler_remove_dependent (struct i915_scheduler *scheduler, +struct i915_scheduler_queue_entry *remove) +{ +struct i915_scheduler_queue_entry *node; +int i, r; +int count =0 ; + +/* +* Ensure that a node is not being removed which is still dependent +* upon other (not completed) work. If that happens, it implies +* something has gone very wrong with the dependency tracking! Note +* that there is no need to worry if this node has been explicitly +* killed for some reason - it might be being killed before it got +* sent to the hardware. +*/ +if (remove->status != i915_sqs_dead) { +for (i =0 ; i < remove->num_deps; i++) +if ((remove->dep_list[i]) && + (!I915_SQS_IS_COMPLETE (remove->dep_list[i]))) + count++; +WARN_ON (count); + } + +/* +* Remove this node from the dependency lists of any other node which +* might be waiting on it. +*/ +for (r =0 ; r <I915_NUM_RINGS ; r++) { +list_for_each_entry (node, &scheduler->node_queue[r], link) { +for (i =0 ; i < node->num_deps; i++) { +if (node->dep_list[i] != remove) +continue ; + + node->dep_list[i] =NULL ; + } + } + } + +return 0 ; +} + +/** +* i915_scheduler_wakeup - wake the scheduler's worker thread +* @dev: DRM device +* Called at the end of seqno interrupt processing if any request has +* completed that corresponds to a scheduler node. +*/ +void i915_scheduler_wakeup (struct drm_device *dev) +{ +/* XXX: Need to call i915_scheduler_remove() via work handler. */ +} + +/** +* i915_scheduler_clean_node - free up any allocations/references +* associated with the given scheduler queue entry. +* @node: Queue entry structure which is complete +* After a give batch buffer completes on the hardware, all the information +* required to resubmit it is no longer required. However, the node entry +* itself might still be required for tracking purposes for a while longer. +* This function should be called as soon as the node is known to be complete +* so that these resources may be freed even though the node itself might +* hang around. +*/ +void i915_scheduler_clean_node (struct i915_scheduler_queue_entry *node) +{ +if (!I915_SQS_IS_COMPLETE (node)) { +WARN (!node->params.request->cancelled, +"Cleaning active node: %d!\n" , node->status); +return ; + } + +if (node->params.batch_obj) {
Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c line 729
Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7
-0+/* +* The batch buffer must be unpinned before it is unreferenced +* otherwise the unpin fails with a missing vma!? +*/ +if (node->params.dispatch_flags &I915_DISPATCH_SECURE ) +i915_gem_execbuff_release_batch_obj (node->params.batch_obj); +This code seems little bit out of place.
Not sure where else you could put it. The code itself is just a copy of the existing clean up code that was already in the execbuff code path. The cleanup can no longer be done within the execbuff call due to the batch buffer execution being arbitrarily delayed by the scheduler. Thus it is down to the scheduler to do all the necessary cleanup when it knows that the batch buffer is no longer required.
Quoted 13 lines
drivers/gpu/drm/i915/i915_scheduler.c line 739
Quoted 11 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+ node->params.batch_obj =NULL ; + } + +/* And anything else owned by the node: */ +if (node->params.cliprects) { +kfree (node->params.cliprects); + node->params.cliprects =NULL ; + } +} +The below function is quite hacky, I understood this will be mitigated by per-ctx sequence numbering in the future?
To be honest, I think it is already obsolete due to the previous work of converting from seqno tests everywhere to testing request structures. The point of the code below was to cope with seqno values popping out of the hardware in random order due to the re-ordering of batch buffers prior to execution. Whereas, the struct request conversion allowed the seqno to be late allocated and thus kept in order.
Unfortunately, pre-emption confuses matters again. Although the intention is that seqno values will still be kept in order. It's just that a given request might have multiple different seqno values assigned to it throughout its life. The pre-emption code hasn't quite settled down yet though, especially with interesting new hardware planned for future chips. Thus I don't really want to rip out all of this code quite yet just in case we do still need it.
As you say, per-ctx seqnos would definitely have an effect here as well. So yes, long term the intention is to remove all this nastyness. Short term, it seems best to keep it around until proven unnecessary.
Quoted 75 lines
Quoted 75 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +74
-0+static bool i915_scheduler_remove (struct i915_scheduler *scheduler, +struct intel_engine_cs *ring, +struct list_head *remove) +{ +struct i915_scheduler_queue_entry *node, *node_next; +int flying =0 , queued =0 ; +bool do_submit; +uint32_t min_seqno; + +spin_lock_irq (&scheduler->lock); + +/* +* In the case where the system is idle, starting 'min_seqno' from a big +* number will cause all nodes to be removed as they are now back to +* being in-order. However, this will be a problem if the last one to +* complete was actually out-of-order as the ring seqno value will be +* lower than one or more completed buffers. Thus code looking for the +* completion of said buffers will wait forever. +* Instead, use the hardware seqno as the starting point. This means +* that some buffers might be kept around even in a completely idle +* system but it should guarantee that no-one ever gets confused when +* waiting for buffer completion. +*/ + min_seqno = ring->get_seqno (ring, true); + +list_for_each_entry (node, &scheduler->node_queue[ring->id], link) { +if (I915_SQS_IS_QUEUED (node)) + queued++; +else if (I915_SQS_IS_FLYING (node)) + flying++; +else if (I915_SQS_IS_COMPLETE (node)) +continue ; + +if (node->params.request->seqno ==0 ) +continue ; + +if (!i915_seqno_passed (node->params.request->seqno, min_seqno)) + min_seqno = node->params.request->seqno; + } + +INIT_LIST_HEAD (remove); +list_for_each_entry_safe (node, node_next, &scheduler->node_queue[ring->id], link) { +/* +* Only remove completed nodes which have a lower seqno than +* all pending nodes. While there is the possibility of the +* ring's seqno counting backwards, all higher buffers must +* be remembered so that the 'i915_seqno_passed()' test can +* report that they have in fact passed. +* +* NB: This is not true for 'dead' nodes. The GPU reset causes +* the software seqno to restart from its initial value. Thus +* the dead nodes must be removed even though their seqno values +* are potentially vastly greater than the current ring seqno. +*/ +if (!I915_SQS_IS_COMPLETE (node)) +continue ; + +if (node->status != i915_sqs_dead) { +if (i915_seqno_passed (node->params.request->seqno, min_seqno) && + (node->params.request->seqno != min_seqno)) +continue ; + } + +list_del (&node->link); +list_add (&node->link, remove); + +/* Strip the dependency info while the mutex is still locked */ +i915_scheduler_remove_dependent (scheduler, node); + +continue ; + } + +/* +* Release the interrupt reference count if there are no longer any
Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c line 819
Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +6
-0+* nodes to worry about. +*/ +if (!flying && !queued && + (scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) { + ring->irq_put (ring); + scheduler->flags[ring->id] &= ~i915_sf_interrupts_enabled;Maybe have this piece of code as a helper? To enable interrupts and disable them, maybe even a reference count?
Not sure how much you could abstract out into a helper other than the two lines of irq_get|put + flag set|clear. The flying|queued test is particular to this one instance. Also, there is no need to reference count as the irq code does that already.
Quoted 43 lines
Quoted 43 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +42
-0+ } + +/* Launch more packets now? */ + do_submit = (queued >0 ) && (flying < scheduler->min_flying); + +spin_unlock_irq (&scheduler->lock); + +return do_submit; +} + +void i915_scheduler_process_work (struct intel_engine_cs *ring) +{ +struct drm_i915_private *dev_priv = ring->dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; +struct i915_scheduler_queue_entry *node; +bool do_submit; +struct list_head remove; + +if (list_empty (&scheduler->node_queue[ring->id])) +return ; + +/* Remove completed nodes. */ + do_submit =i915_scheduler_remove (scheduler, ring, &remove); + +if (!do_submit &&list_empty (&remove)) +return ; + +/* Need to grab the pm lock outside of the mutex lock */ +if (do_submit) +intel_runtime_pm_get (dev_priv); + +mutex_lock (&ring->dev->struct_mutex); + +if (do_submit) +i915_scheduler_submit (ring); + +while (!list_empty (&remove)) { + node =list_first_entry (&remove,typeof (*node), link); +list_del (&node->link); + +/* Free up all the DRM references */ +i915_scheduler_clean_node (node);
Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c line 867
Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +6
-0+ +/* And anything else owned by the node: */ + node->params.request->scheduler_qe =NULL ; +i915_gem_request_unreference (node->params.request); +kfree (node->dep_list); +kfree (node);Maybe have a separate helper for freeing a node, would make this much cleaner.
There is a helper for freeing up the contents of the node - see i915_scheduler_clean_node() above. However, that function is called from multiple places and this is the only instance where the node itself is actually freed. The other callers either need to keep the node around for a while longer (i.e. until this piece of code can run) or don't own the node because it is stack allocated (in the bypass case). So the 'anything else' section cannot be moved into the existing clean function and any secondary helper would be single usage and only four lines long. It doesn't seem worth it.
Quoted 51 lines
Quoted 51 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7
-0+ } + +mutex_unlock (&ring->dev->struct_mutex); + +if (do_submit) +intel_runtime_pm_put (dev_priv); +}drivers/gpu/drm/i915/i915_scheduler.h new file +37
-0@@ -0,0 +1,95 @@ +/* +* Copyright (c) 2014 Intel Corporation +* +* Permission is hereby granted, free of charge, to any person obtaining a +* copy of this software and associated documentation files (the "Software"), +* to deal in the Software without restriction, including without limitation +* the rights to use, copy, modify, merge, publish, distribute, sublicense, +* and/or sell copies of the Software, and to permit persons to whom the +* Software is furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice (including the next +* paragraph) shall be included in all copies or substantial portions of the +* Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +* IN THE SOFTWARE. +* +*/ + +#ifndef _I915_SCHEDULER_H_ +#define _I915_SCHEDULER_H_ + +enum i915_scheduler_queue_status { +/* Limbo: */ + i915_sqs_none =0 , +/* Not yet submitted to hardware: */ + i915_sqs_queued, +/* Popped from queue, ready to fly: */ + i915_sqs_popped, +/* Sent to hardware for processing: */ + i915_sqs_flying, + /* Finished processing on the hardware: */
Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.h line 44
Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +7
-0+ i915_sqs_complete, +/* Killed by watchdog or catastrophic submission failure: */ + i915_sqs_dead, +/* Limit value for use with arrays/loops */ + i915_sqs_MAX +}; +To uppercase.
Grrr. Linux kernel style guide is daft.
Quoted 47 lines
Quoted 13 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +12
-0+#define I915_SQS_IS_QUEUED (node) (((node)->status == i915_sqs_queued)) +#define I915_SQS_IS_FLYING (node) (((node)->status == i915_sqs_flying)) +#define I915_SQS_IS_COMPLETE (node) (((node)->status == i915_sqs_complete) || \ + ((node)->status == i915_sqs_dead)) + +struct i915_scheduler_obj_entry { +struct drm_i915_gem_object *obj; +bool read_only; +}; + +struct i915_scheduler_queue_entry { +struct i915_execbuffer_params params;drivers/gpu/drm/i915/i915_scheduler.h line 66
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +10
-0+/* -1023 = lowest priority, 0 = default, 1023 = highest */ +int32_t priority; +struct i915_scheduler_obj_entry *saved_objects; +int num_objs; +bool bumped; +struct i915_scheduler_queue_entry **dep_list; +int num_deps; +enum i915_scheduler_queue_status status; +unsigned long stamp; +struct list_head link;Above whitespace is still incorrect and I could really use comments.
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +10
-0+}; + +struct i915_scheduler { +struct list_head node_queue[I915_NUM_RINGS ]; +uint32_t flags[I915_NUM_RINGS ]; +spinlock_t lock; + +/* Tuning parameters: */ +int32_t priority_level_min; +int32_t priority_level_max;drivers/gpu/drm/i915/i915_scheduler.h line 86
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +10
-0+int32_t priority_level_bump; +int32_t priority_level_preempt; +uint32_t min_flying; +}; + +/* Flag bits for i915_scheduler::flags */ +enum { + i915_sf_interrupts_enabled = (1 <<0 ), + i915_sf_submitting = (1 <<1 ), +};These must be
I915_UPPERCASE_SOMETHINGby convention.
Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.h line 95
Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +9
-0+ +bool i915_scheduler_is_enabled (struct drm_device *dev); +int i915_scheduler_init (struct drm_device *dev); +void i915_scheduler_clean_node (struct i915_scheduler_queue_entry *node); +int i915_scheduler_queue_execbuffer (struct i915_scheduler_queue_entry *qe); +bool i915_scheduler_notify_request (struct drm_i915_gem_request *req); +void i915_scheduler_wakeup (struct drm_device *dev); + +#endif /* _I915_SCHEDULER_H_ */
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Joonas Lahtinen
maintainerRe: [PATCH v5 06/35] drm/i915: Start of GPU scheduler
Hi,
The below answers are reasonable. So v6 should be the version.
Regards, Joonas
Quoted 1,834 lines · John Harrison
Quoted 243 lines · Joonas Lahtinen
Hi,
Now the code is in reviewable chunks, excellent!
I've added my comments below. A few repeats from last round, but now
with more questions about the logic itself.Quoted 228 lines · On to, 2016-02-18 at 14:26 +0000, [email protected] wrote:
From: John Harrison <[email protected]>
Initial creation of scheduler source files. Note that this patch implements most of the scheduler functionality but does not hook it in
to the driver yet. It also leaves the scheduler code in 'pass through'
mode so that even when it is hooked in, it will not actually do very
much. This allows the hooks to be added one at a time in bite size
chunks and only when the scheduler is finally enabled at the end does
anything start happening.The general theory of operation is that when batch buffers are submitted to the driver, the
execbuffer()code packages up all the
information required to execute the batch buffer at a later time. This
package is given over to the scheduler which adds it to an internal
node list. The scheduler also scans the list of objects associated
with the batch buffer and compares them against the objects already in
use by other buffers in the node list. If matches are found then the
new batch buffer node is marked as being dependent upon the matching
node. The same is done for the context object. The scheduler also bumps up the priority of such matching nodes on the grounds that the
more dependencies a given batch buffer has the more important it is
likely to be.The scheduler aims to have a given (tuneable) number of batch buffers
in flight on the hardware at any given time. If fewer than this are
currently executing when a new node is queued, then the node is passed
straight through to the submit function. Otherwise it is simply added
to the queue and the driver returns back to user land.The scheduler is notified when each batch buffer completes and updates
its internal tracking accordingly. At the end of the completion interrupt processing, if any scheduler tracked batches were processed,
the scheduler's deferred worker thread is woken up. This can do more
involved processing such as actually removing completed nodes from the
queue and freeing up the resources associated with them (internal memory allocations, DRM object references, context reference, etc.).
The work handler also checks the in flight count and calls the submission code if a new slot has appeared.When the scheduler's submit code is called, it scans the queued node
list for the highest priority node that has no unmet dependencies.
Note that the dependency calculation is complex as it must take inter-ring dependencies and potential preemptions into account. Note
also that in the future this will be extended to include external dependencies such as the Android Native Sync file descriptors and/or
the linux dma-buff synchronisation scheme.If a suitable node is found then it is sent to
execbuff_final()for
submission to the hardware. The in flight count is then re- checked and
a new node popped from the list if appropriate. All nodes that are not
submitted have their priority bumped. This ensures that low priority
tasks do not get starved out by busy higher priority ones - everything
will eventually get its turn to run.Note that this patch does not implement pre-emptive scheduling. Only
basic scheduling by re-ordering batch buffer submission is currently
implemented. Pre-emption of actively executing batch buffers comes in
the next patch series.v2: Changed priority levels to +/-1023 due to feedback from Chris Wilson.
Removed redundant index from scheduler node.
Changed time stamps to use jiffies instead of raw monotonic. This provides lower resolution but improved compatibility with other i915
code.Major re-write of completion tracking code due to struct fence conversion. The scheduler no longer has it's own private IRQ handler
but just lets the existing request code handle completion events. Instead, the scheduler now hooks into the request notify code to be
told when a request has completed.Reduced driver mutex locking scope. Removal of scheduler nodes no longer grabs the mutex lock.
v3: Refactor of dependency generation to make the code more readable.
Also added in read-read optimisation support - i.e., don't treat a
shared read-only buffer as being a dependency.Allowed the killing of queued nodes rather than only flying ones.
v4: Updated the commit message to better reflect the current state of
the code. Downgraded someBUG_ONstoWARN_ONs. Used the correct array
memory allocator function (kmalloc_arrayinstead of kmalloc). Corrected the format of some comments. Wrapped some lines differently
to keep the style checker happy.Fixed a
WARN_ONwhen killing nodes. The dependency removal code checks
that nodes being destroyed do not have any oustanding dependencies
(which would imply they should not have been executed yet). In the
case of nodes being destroyed, e.g. due to context banning, then this
might well be the case - they have not been executed and do indeed
have outstanding dependencies.Re-instated the code to disble interrupts when not in use. The underlying problem causing broken IRQ reference counts seems to have
been fixed now.v5: Shuffled various functions around to remove forward declarations
as apparently these are frowned upon. Removed lots of white space as
apparently having easy to read code is also frowned upon. Split the
direct submission scheduler bypass code out into a separate function.
Squashed down thei915_scheduler.csections of various patches into
this patch. Thus the later patches simply hook in existing code into
various parts of the driver rather than adding the code as well. Added
documentation to various functions. Re-worked the submit function in
terms of mutex locking, error handling and exit paths. Split the delayed work handler function in half. Made use of the kernel 'clamp'
macro. [Joonas Lahtinen]Added runtime PM calls as these must be done at the top level before
acquiring the driver mutex lock. [Chris Wilson]Removed some obsolete debug code that had been forgotten about.
Moved more clean up code into the '
i915_gem_scheduler_clean_node()'
function rather than replicating it in mutliple places.Used lighter weight spinlocks.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>Cc Joonas Lahtinen <[email protected]>---
drivers/gpu/drm/i915/Makefile | 1 + drivers/gpu/drm/i915/i915_drv.h | 6 + drivers/gpu/drm/i915/i915_gem.c | 5 +drivers/gpu/drm/i915/i915_scheduler.c| 874
++++++++++++++++++++++++++++++++++
drivers/gpu/drm/i915/i915_scheduler.h| 95 ++++5 files changed, 981 insertions(+)create mode 100644
drivers/gpu/drm/i915/i915_scheduler.ccreate mode 100644drivers/gpu/drm/i915/i915_scheduler.hdrivers/gpu/drm/i915/Makefile +1
-0@@ -10,6 +10,7 @@ ccflags-y := -Werror i915-y := i915_drv.o \ i915_irq.o \ i915_params.o \ + i915_scheduler.o \ i915_suspend.o \ i915_sysfs.o \ intel_csr.o \drivers/gpu/drm/i915/i915_drv.h +0
-0@@ -1703,6 +1703,8 @@ struct i915_execbuffer_params { struct drm_i915_gem_request *request; };drivers/gpu/drm/i915/i915_drv.h +2
-0+struct i915_scheduler ; +/* used in computing the new watermarks state */ struct
intel_wm_config{
unsigned intnum_pipes_active;drivers/gpu/drm/i915/i915_drv.h +0
-0@@ -1968,6 +1970,8 @@ struct drm_i915_private {
structi915_runtime_pmpm;drivers/gpu/drm/i915/i915_drv.h struct drm_i915_private { · line 1974
Quoted 2 lines
drivers/gpu/drm/i915/i915_drv.h +2
-0+struct i915_scheduler *scheduler; +I think we should have i915.enable_scheduler parameter (just like i915.enable_execlists) and make this a data member, not pointer.
Quoted 3 lines
Then we can go forward and have i915.enable_scheduler_preempt and so on
to control things at boot time.There are. The
enable_schedulercomes in as patch #21. Including it from
the start means modifying i915_params.[ch] in this patch as well. And
then you are getting into one big patch that contains everything and is
unreviewable due to its sheer size. Theenable_preemptionparameter comes in with the pre-emption code. No point in adding that until there
is something for it to control.Quoted 99 lines
Quoted 12 lines
/* Abstract the submission mechanism (legacy ringbuffer or execlists) away */
struct {
int (*execbuf_submit)(structi915_execbuffer_params*params,drivers/gpu/drm/i915/i915_drv.h +0
-0@@ -2290,6 +2294,8 @@ struct drm_i915_gem_request { /** process identifier submitting this request */ struct pid *pid;drivers/gpu/drm/i915/i915_drv.h struct drm_i915_gem_request { · line 2298
Quoted 2 lines
drivers/gpu/drm/i915/i915_drv.h +2
-0+struct i915_scheduler_queue_entry *scheduler_qe; +Funny whitespace.
Quoted 83 lines
/**
* The ELSP only accepts two elements at a time, so we queue
* context/tail pairs on a given queue (ring-Quoted 1 line
execlist_queue) until thedrivers/gpu/drm/i915/i915_gem.c +1
-0@@ -32,6 +32,7 @@ #include "i915_vgpu.h" #include "i915_trace.h" #include "intel_drv.h" +#include "i915_scheduler.h" #include #include #include@@ -5315,6 +5316,10 @@ int i915_gem_init(struct drm_device *dev)*/ intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);drivers/gpu/drm/i915/i915_gem.c +4
-0+ ret =i915_scheduler_init (dev); +if (ret) + goto out_unlock; +ret =
i915_gem_init_userptr(dev);
if (ret)
gotoout_unlock;drivers/gpu/drm/i915/i915_scheduler.c new file +4
-0@@ -0,0 +1,874 @@ +/* +* Copyright (c) 2014 Intel Corporation +* +* Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */drivers/gpu/drm/i915/i915_scheduler.c new file +7
-0+ +#include "i915_drv.h" +#include "intel_drv.h" +#include "i915_scheduler.h" + +/** +* i915_scheduler_is_enabled - Returns true if the scheduler is enabled.
+ * @dev: DRM deviceQuoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c line 38
Quoted 7 lines
+ */
drivers/gpu/drm/i915/i915_scheduler.c new file +6
-0+bool i915_scheduler_is_enabled (struct drm_device *dev) +{ +struct drm_i915_private *dev_priv = dev->dev_private; + +return dev_priv->scheduler !=NULL ; +}i915.enable_scheduler would be used instead.
It is when it arrives.
Quoted 19 lines
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +5
-0+ +/** +* i915_scheduler_init - Initialise the scheduler. +* @dev: DRM device +* Returns zero on success or -ENOMEM if memory allocations fail.
+ */drivers/gpu/drm/i915/i915_scheduler.c line 50
Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +6
-0+int i915_scheduler_init (struct drm_device *dev) +{ +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; +int r; +This protection -->
Quoted 1 line
Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c line 57
Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7
-0+if (scheduler) +return 0 ; + + scheduler =kzalloc (sizeof (*scheduler),GFP_KERNEL ); +if (!scheduler) +return -ENOMEM ; +Is not needed if we have data member and i915.enable_scheduler.
Making the scheduler structure a object rather than a pointer basically
means moving the whole ofi915_scheduler.hintoi915_drv.h. There is too
much interconnectedness between the various structures. I would say that
keeping the code nicely partitioned and readable is worth the one dynamic allocation at driver load time.Quoted 27 lines
Quoted 27 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +19
-0+spin_lock_init (&scheduler->lock); + +for (r =0 ; r <I915_NUM_RINGS ; r++) +INIT_LIST_HEAD (&scheduler->node_queue[r]); + +/* Default tuning values: */ + scheduler->priority_level_min = -1023; + scheduler->priority_level_max = 1023; + scheduler->priority_level_bump = 50; + scheduler->priority_level_preempt =900 ; + scheduler->min_flying = 2; + + dev_priv->scheduler = scheduler; + +return 0 ; +} + +/* +* Add a popped node back in to the queue. For example, because the ring was
+ * hung whenexecfinal()was called and thus the ring submission needs to be
+ * retried later.
+ */Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c
Quoted 4 lines
+static void i915_scheduler_node_requeue(struct
i915_scheduler_queue_entry*node)drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+{ +WARN_ON (!node);Rather remove this completely as it is an internal function, or use
BUG_ONbecause next line will straight lead to OOPS after warning. I'll
pass mentioning rest of the obviousWARN_ONvs BUG_ON's.Yeah, most of the WARN/BUG_ONs are probably unnecessary now. They were
scattered liberally around during initial development when strange and
unexpected things could happen. Now the the code paths are pretty well
established they aren't really much use. I'll reduce them down to just
the ones that still make sense.Quoted 76 lines
Quoted 17 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +9
-0+WARN_ON (!I915_SQS_IS_FLYING (node)); + +/* Seqno will be reassigned on relaunch */ + node->params.request->seqno =0 ; + node->status = i915_sqs_queued; +} + +/* +* Give up on a node completely. For example, because it is causing the
+ * ring to hang or is using some resource that no longer exists.
+ */
+static voidi915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)drivers/gpu/drm/i915/i915_scheduler.c line 102
Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +8
-0+{ +WARN_ON (!node); +WARN_ON (I915_SQS_IS_COMPLETE (node)); + + node->status = i915_sqs_dead; +} + +/* Mark a node as in flight on the hardware. */ As for documentation, from below, I only assume scheduler lock must be
held and node's can be only manipulated when scheduler lock is held?
It's good to add a comment describing the expected locking. Or maybe
addassert_scheduler_lock_held()helper and sprinkle it around the functions similar toassert_rpm_wakelock_held(), which is self- documenting?Quoted 6 lines
+static int i915_scheduler_node_fly(struct
i915_scheduler_queue_entry*node)drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+{ + struct drm_i915_private *dev_priv = node->params.dev-Quoted 1 line
dev_private;If node is NULL, this would already OOPS, I do not think it's unreasonable to expect node not to be NULL. The below
WARN_ONcheck is
never reached.drivers/gpu/drm/i915/i915_scheduler.c line 114
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +9
-0+struct i915_scheduler *scheduler = dev_priv->scheduler; +struct intel_engine_cs *ring; + +WARN_ON (!scheduler); +WARN_ON (!node); +WARN_ON (node->status != i915_sqs_popped); + + ring = node->params.ring; +This and similar assignments can be squashed to declaration line, as
this is basically an alias.Quoted 16 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+/* +* Add the node (which should currently be in state popped) to the
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * front of the queue. This ensure that flying nodes arealways held
drivers/gpu/drm/i915/i915_scheduler.c new file +7
-0+* in hardware submission order. +*/ +list_add (&node->link, &scheduler->node_queue[ring->id]); + + node->status = i915_sqs_flying; + + if (!(scheduler->flags[ring->id] &
i915_sf_interrupts_enabled)) {drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+bool success = true; +Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c line 131
Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +3
-0+ success = ring->irq_get (ring); +if (success) + scheduler->flags[ring->id] |=
i915_sf_interrupts_enabled;drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+else +return -EINVAL ;Shouldn't the node be cleaned up from
node_queuein case of an error?Not really. The batch buffer will still be executed regardless of whether interrupts could be enabled or not. The return code is actually
ignored so this should really be a void function, I guess. Without interrupts then completion might take a while to be noticed but stuff
should still basically work.Quoted 71 lines
Also, I think above could be written much more compact form (because it
looks like something where more logic will be added later). It makes it
easier to write and visualize the error paths when reading if there are
no nested if's.I won't mention about the error paths of functions below, I expect the
following guidline to be adhered;if (scheduler->flags[ring->id] & I915_SF_INT_ENABLED) goto out; if (!ring->irq_get(ring)) goto err_irq; if (!...) goto err_foo; scheduler->flags[ring->id] |= I915_SF_INT_ENABLED; out: return 0; err_foo: foobar(); err_irq: list_remove() return -EINVAL;Quoted 15 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +6
-0+ } + +return 0 ; +} + +static uint32_t i915_scheduler_count_flying(struct
i915_scheduler*scheduler,drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ struct
intel_engine_cs*ring)drivers/gpu/drm/i915/i915_scheduler.c new file +5
-0+{ + struct i915_scheduler_queue_entry *node; + uint32_t flying = 0; + + list_for_each_entry(node, &scheduler->node_queue[ring-Quoted 1 line
Quoted 1 line
id], link)
+1 for the
for_each_scheduler_node(...)Quoted 21 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7
-0+if (I915_SQS_IS_FLYING (node)) + flying++; + +return flying; +} + +static void i915_scheduler_priority_bump_clear(struct
i915_scheduler*scheduler)drivers/gpu/drm/i915/i915_scheduler.c new file +6
-0+{ +struct i915_scheduler_queue_entry *node; +int i; + +/* +* Ensure circular dependencies don't cause problems and that a bump
drivers/gpu/drm/i915/i915_scheduler.c new file +4
-0+* by object usage only bumps each using buffer once: +*/ +for (i =0 ; i <I915_NUM_RINGS ; i++) { + list_for_each_entry(node, &scheduler-Quoted 1 line
node_queue[i], link)
Quoted 13 lines
Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +5
-0+ node->bumped = false; + } +} + +static int i915_scheduler_priority_bump(struct i915_scheduler*scheduler,
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ struct
i915_scheduler_queue_entry*target,Is there specific reason why struct
i915_scheduler_queue_entryare not
referred to just as "node" but "qe" and here something else, do "node"
and "qe" have a special semantic meaning?Not sure what you mean. The use of 'node' is because almost everything
ani915_scheduler_queue_entryobject is being used it is as a node in a
list that is being iterated over. The only 'qe' variable is in 'i915_scheduler_queue_execbuffer[_bypass]' and there it is because the
qe is being passed in from outside and is copied to a local 'node' that
is then added to the list. I think the use of 'target' here is largely
historical. The subsequent array search was originally a list search and
hence had a 'node' that was used as the iterator. Thus the qe passed in
was called something else to avoid conflicts and target seemed appropriate.Quoted 12 lines
drivers/gpu/drm/i915/i915_scheduler.c line 173
Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7
-0+ uint32_t bump) +{ +uint32_t new_priority; +int i, count; + +if (target->priority >= scheduler->priority_level_max) +return 1 ;So if one node reaches maximum priority the dependencies are expected
to be maxed out already?Yes. Dependencies should always be of equal or higher priority to the
dependee. When a new dependency is created, the priority of the dependee
is added on to ensure that something of high priority can never be stuck
waiting on something of low priority. Also, it means that a batch buffer
that lots of other buffers are waiting on will be bumped to higher and
higher priority the more waiters there are and thus is more likely to be
executed sooner and free them all up.Quoted 282 lines
Quoted 139 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +8
-0+ +if (target->bumped) +return 0 ; + + new_priority = target->priority + bump; +if ((new_priority <= target->priority) || + (new_priority > scheduler->priority_level_max)) + target->priority = scheduler-Quoted 1 line
priority_level_max;drivers/gpu/drm/i915/i915_scheduler.c new file +15
-0+else + target->priority = new_priority; + + count =1 ; + target->bumped = true; + +for (i =0 ; i < target->num_deps; i++) { +if (!target->dep_list[i]) +continue ; + +if (target->dep_list[i]->bumped) +continue ; + + count += i915_scheduler_priority_bump(scheduler, + target-Quoted 1 line
dep_list[i],
drivers/gpu/drm/i915/i915_scheduler.c new file +8
-0+ bump); + } + +return count; +} + +/* +* Nodes are considered valid dependencies if they are queued on any ring or
+ * if they are in flight on a different ring. In flight on the same ring is no
+ * longer interesting for non-premptive nodes as the ring serialises execution.
+ * For pre-empting nodes, all in flight dependencies are valid as they must not
+ * be jumped by the act of pre-empting.
+ *
+ * Anything that is neither queued nor flying is uninteresting.
+ */drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+static inline bool i915_scheduler_is_dependency_valid( + struct i915_scheduler_queue_entry *node,
uint32_tidx)drivers/gpu/drm/i915/i915_scheduler.c new file +19
-0+{ +struct i915_scheduler_queue_entry *dep; + + dep = node->dep_list[idx]; +if (!dep) +return false; + +if (I915_SQS_IS_QUEUED (dep)) +return true; + +if (I915_SQS_IS_FLYING (dep)) { +if (node->params.ring != dep->params.ring) +return true; + } + +return false; +} + +static int i915_scheduler_pop_from_queue_locked(struct
intel_engine_cs*ring,drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ struct
i915_scheduler_queue_entry**pop_node)drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+{ + struct drm_i915_private *dev_priv = ring->dev-Quoted 1 line
dev_private;drivers/gpu/drm/i915/i915_scheduler.c new file +12
-0+struct i915_scheduler *scheduler = dev_priv->scheduler; +struct i915_scheduler_queue_entry *best =NULL ; +struct i915_scheduler_queue_entry *node; +int ret; +int i; +bool any_queued = false; +bool has_local, has_remote, only_remote = false; + + *pop_node =NULL ; + ret = -ENODATA ; + + list_for_each_entry(node, &scheduler->node_queue[ring-Quoted 1 line
id], link) {
drivers/gpu/drm/i915/i915_scheduler.c new file +8
-0+if (!I915_SQS_IS_QUEUED (node)) +continue ; + any_queued = true; + + has_local = false; + has_remote = false; +for (i =0 ; i < node->num_deps; i++) { + if(!i915_scheduler_is_dependency_valid(node, i))
drivers/gpu/drm/i915/i915_scheduler.c new file +3
-0+continue ; + + if (node->dep_list[i]->params.ring ==
node->params.ring)drivers/gpu/drm/i915/i915_scheduler.c new file +26
-0+ has_local = true; + else + has_remote = true; + } + +if (has_remote && !has_local) + only_remote = true; + +if (!has_local && !has_remote) { +if (!best || + (node->priority > best->priority)) + best = node; + } + } + +if (best) { +list_del (&best->link); + +INIT_LIST_HEAD (&best->link); + best->status = i915_sqs_popped; + + ret =0 ; + }else { +/* Can only get here if: +* (a) there are no buffers in the queue +* (b) all queued buffers are dependent on other buffers
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * e.g. on a buffer that is in flight on adifferent ring
drivers/gpu/drm/i915/i915_scheduler.c new file +3
-0+*/ +if (only_remote) { +/* The only dependent buffers are on another ring. */
drivers/gpu/drm/i915/i915_scheduler.c new file +3
-0+ ret = -EAGAIN ; + }else if (any_queued) { +/* It seems that something has gone horribly wrong! */
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ DRM_ERROR("Broken dependency tracking onring %d!\n",
drivers/gpu/drm/i915/i915_scheduler.c line 293
Quoted 1 line
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ (int) ring->id);Would this condition be worthy
WARN_ONCE(), because it will keep repeating as the situation will persist?Quoted 82 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +9
-0+ } + } + + *pop_node = best; +return ret; +} + +/* +* NB: The driver mutex lock must be held before calling this function. It is
+ * only really required during the actual back end submission call. However,
+ * attempting to acquire a mutex while holding a spin lock is a Bad Idea.
+ * And releasing the one before acquiring the other leads to other code
+ * being run and interfering.
+ */drivers/gpu/drm/i915/i915_scheduler.c new file +13
-0+static int i915_scheduler_submit (struct intel_engine_cs *ring) +{ +struct drm_device *dev = ring->dev; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; +struct i915_scheduler_queue_entry *node; +int ret, count =0 , flying; + +WARN_ON (!mutex_is_locked (&dev->struct_mutex)); + +spin_lock_irq (&scheduler->lock); + + WARN_ON(scheduler->flags[ring->id] &
i915_sf_submitting);drivers/gpu/drm/i915/i915_scheduler.c new file +3
-0+ scheduler->flags[ring->id] |= i915_sf_submitting; + +/* First time around, complain if anything unexpected occurs: */
drivers/gpu/drm/i915/i915_scheduler.c new file +12
-0+ ret =i915_scheduler_pop_from_queue_locked (ring, &node); +if (ret) + goto error; + + do { + WARN_ON(!node); + WARN_ON(node->params.ring != ring); + WARN_ON(node->status != i915_sqs_popped); + count++; + + /* + * The call to pop above will have removed thenode from the
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * list. So add it back in and mark it as inflight.
drivers/gpu/drm/i915/i915_scheduler.c new file +9
-0+*/ +i915_scheduler_node_fly (node); + +spin_unlock_irq (&scheduler->lock); + ret = dev_priv->gt.execbuf_final (&node->params); +spin_lock_irq (&scheduler->lock); + +/* +* Handle failed submission but first check that the
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * watchdog/reset code has not nuked the nodewhile we
drivers/gpu/drm/i915/i915_scheduler.c new file +7
-0+* weren't looking: +*/ +if (ret && (node->status != i915_sqs_dead)) { +bool requeue = true; + +/* +* Oh dear! Either the node is broken or the ring is
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * busy. So need to kill the node orrequeue it and try
drivers/gpu/drm/i915/i915_scheduler.c new file +7
-0+* again later as appropriate. +*/ + +switch (-ret) { +case ENODEV : +case ENOENT : +/* Fatal errors. Kill the node. */
drivers/gpu/drm/i915/i915_scheduler.c line 364
Quoted 3 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +3
-0+ requeue = false; +i915_scheduler_node_kill (node); +break ;These break indents still.
Quoted 52 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +8
-0+ +case EAGAIN : +case EBUSY : +case EIO : +case ENOMEM : +case ERESTARTSYS : +case EINTR : +/* Supposedly recoverable errors. */
drivers/gpu/drm/i915/i915_scheduler.c new file +5
-0+break ; + +default : +/* +* Assume the error is recoverable and hope
drivers/gpu/drm/i915/i915_scheduler.c new file +8
-0+* for the best. +*/ +MISSING_CASE (-ret); +break ; + } + +if (requeue) { + i915_scheduler_node_requeue(node);
drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+/* +* No point spinning if the ring is currently
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * unavailable so just give upand come back
drivers/gpu/drm/i915/i915_scheduler.c new file +7
-0+* later. +*/ +break ; + } + } + +/* Keep launching until the sky is sufficiently full. */
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ flying = i915_scheduler_count_flying(scheduler,ring);
drivers/gpu/drm/i915/i915_scheduler.c new file +5
-0+if (flying >= scheduler->min_flying) +break ; + +/* Grab another node and go round again... */ + ret = i915_scheduler_pop_from_queue_locked(ring,&node);
drivers/gpu/drm/i915/i915_scheduler.c new file +3
-0+ }while (ret ==0 ); + +/* Don't complain about not being able to submit extra entries */
drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+if (ret == -ENODATA ) + ret =0 ;Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c
Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +3
-0+ +/* +* Bump the priority of everything that was not submitted to prevent
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * starvation of low priority tasks by a spamming highpriority task.
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+*/ This, this calls for an I-G-T test to make sure we're effective.
Yeah, IGTs are under development.
Quoted 92 lines
Quoted 55 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+i915_scheduler_priority_bump_clear (scheduler); + list_for_each_entry(node, &scheduler->node_queue[ring-Quoted 1 line
id], link) {
drivers/gpu/drm/i915/i915_scheduler.c new file +5
-0+if (!I915_SQS_IS_QUEUED (node)) +continue ; + + i915_scheduler_priority_bump(scheduler, node, + scheduler-Quoted 1 line
priority_level_bump);drivers/gpu/drm/i915/i915_scheduler.c new file +3
-0+ } + +/* On success, return the number of buffers submitted. */
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+if (ret ==0 ) + ret = count; + +error: + scheduler->flags[ring->id] &= ~i915_sf_submitting; +spin_unlock_irq (&scheduler->lock); +return ret; +} + +static void i915_generate_dependencies(struct i915_scheduler*scheduler,
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ struct
i915_scheduler_queue_entry*node,drivers/gpu/drm/i915/i915_scheduler.c new file +8
-0+ uint32_t ring) +{ +struct i915_scheduler_obj_entry *this, *that; +struct i915_scheduler_queue_entry *test; +int i, j; +bool found; + + list_for_each_entry(test, &scheduler->node_queue[ring],link) {
drivers/gpu/drm/i915/i915_scheduler.c new file +8
-0+if (I915_SQS_IS_COMPLETE (test)) +continue ; + +/* +* Batches on the same ring for the same +* context must be kept in order. +*/ + found = (node->params.ctx == test->params.ctx)&&
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ (node->params.ring == test-Quoted 1 line
params.ring);
drivers/gpu/drm/i915/i915_scheduler.c new file +6
-0+ +/* +* Batches working on the same objects must +* be kept in order. +*/ +for (i =0 ; (i < node->num_objs) && !found; i++){
drivers/gpu/drm/i915/i915_scheduler.c line 456
Quoted 1 line
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ this = node->saved_objects + i;
node->objsandnode->num_objswould be a better naming match.num_objs
andsaved_objectsmakes me think we have a bug here, indexing wrong array.Quoted 31 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +8
-0+ +for (j =0 ; j < test->num_objs; j++) { + that = test->saved_objects + j; + +if (this->obj != that->obj) +continue ; + +/* Only need to worry about writes */
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ if (this->read_only && that-Quoted 1 line
read_only)drivers/gpu/drm/i915/i915_scheduler.c new file +15
-0+continue ; + + found = true; +break ; + } + } + +if (found) { + node->dep_list[node->num_deps] = test; + node->num_deps++; + } + } +} + +static int i915_scheduler_queue_execbuffer_bypass(struct
i915_scheduler_queue_entry*qe)drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+{ + struct drm_i915_private *dev_priv = qe->params.dev-Quoted 1 line
dev_private;Quoted 12 lines
Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +4
-0+struct i915_scheduler *scheduler = dev_priv->scheduler; +int ret; + + scheduler->flags[qe->params.ring->id] |=
i915_sf_submitting;drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+ ret = dev_priv->gt.execbuf_final (&qe->params); + scheduler->flags[qe->params.ring->id] &=~i915_sf_submitting;
+The above code makes me think of locking again, maybe document at top
of this function.No need to acquire the spinlock at this point. If the scheduler is in
bypass mode then there is no internal state to protect. The driver mutex
lock will be held because there is only one code path to get here which
is the execbuff IOCTL. The whole point of bypass mode is that it is a
single contiguous execution path from IOCTL entry all the way through to
hardware submission. A mutex check could be added but it seems unnecessary given the limited calling scope.Quoted 52 lines
Quoted 52 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+/* +* Don't do any clean up on failure because the caller will
drivers/gpu/drm/i915/i915_scheduler.c new file +13
-0+* do it all anyway. +*/ +if (ret) +return ret; + +/* Need to release any resources held by the node: */ +i915_scheduler_clean_node (qe); + +return 0 ; +} + +/** +* i915_scheduler_queue_execbuffer - Submit a batch buffer request to the
+ * scheduler.
+ * @qe: The batch buffer request to be queued.
+ * The expectation is the qe passed in is a local stack variable. This
+ * function will copy its contents into a freshly allocated list node. The
+ * new node takes ownership of said contents so the original qe should simply
+ * be discarded and not cleaned up (i.e. don't free memory it points to or
+ * dereference objects it holds). The node is added to the scheduler's queue
+ * and the batch buffer will be submitted to the hardware at some future
+ * point in time (which may be immediately, before returning or may be quite
+ * a lot later).
+ */
+inti915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+{ + struct drm_i915_private *dev_priv = qe->params.dev-Quoted 1 line
dev_private;drivers/gpu/drm/i915/i915_scheduler.c new file +9
-0+struct i915_scheduler *scheduler = dev_priv->scheduler; +struct intel_engine_cs *ring = qe->params.ring; +struct i915_scheduler_queue_entry *node; +struct i915_scheduler_queue_entry *test; +bool not_flying; +int i, r; +int incomplete =0 ; + +WARN_ON (!scheduler);Quoted 8 lines
Quoted 5 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +3
-0+ +if (1 /*!i915.enable_scheduler*/ ) +return
i915_scheduler_queue_execbuffer_bypass(qe);
+I'm thinking, should we branch already before calling a function named
i915_scheduler_queue_execbufferif scheduler is disabled?That would require putting if(scheduler) logic in random points throughout the driver. Doing it this way, the driver as a whole has a
single execution path which is via the scheduler. It is then up to the
scheduler itself to decide whether to run in bypass mode, simple re-ordering mode, full pre-emption mode or any other mode we feel a need
to add.Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c line 536
Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +5
-0+ node =kmalloc (sizeof (*node),GFP_KERNEL ); +if (!node) +return -ENOMEM ; + + *node = *qe;Now I read the added documentation for the function, maybe we should at
least zero out qe to avoid future problems?The execbuff success path hands the qe contents over and pretty much immediately drops all the way back out of the IOCTL. The failure path
does the full cleanup of de-allocating all the qe stuff it had previously allocated. IMO there doesn't seem to be much need to zero out
the structure at this point.Quoted 79 lines
Quoted 25 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +9
-0+INIT_LIST_HEAD (&node->link); + node->status = i915_sqs_queued; + node->stamp = jiffies; +i915_gem_request_reference (node->params.request); + +WARN_ON (node->params.request->scheduler_qe); + node->params.request->scheduler_qe = node; + +/* Need to determine the number of incomplete entries in the list as
drivers/gpu/drm/i915/i915_scheduler.c new file +3
-0+ * that will be the maximum size of the dependency list. + * + * Note that the allocation must not be made with thespinlock acquired
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * as kmalloc can sleep. However, the unlock/relock issafe because no
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * new entries can be queued up during the unlock as thei915 driver
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * mutex is still held. Entries could be removed fromthe list but that
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * just means the dep_list will be over-allocated whichis fine.
drivers/gpu/drm/i915/i915_scheduler.c
Quoted 2 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+*/ +spin_lock_irq (&scheduler->lock);-->
drivers/gpu/drm/i915/i915_scheduler.c line 563
Quoted 11 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+for (r =0 ; r <I915_NUM_RINGS ; r++) { + list_for_each_entry(test, &scheduler-Quoted 1 line
node_queue[r], link) {
drivers/gpu/drm/i915/i915_scheduler.c new file +7
-0+if (I915_SQS_IS_COMPLETE (test)) +continue ; + + incomplete++; + } + } +<-- This counting structure, I still think it would be good idea to make it a static helper.
Quoted 36 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +5
-0+/* Temporarily unlock to allocate memory: */ +spin_unlock_irq (&scheduler->lock); +if (incomplete) { + node->dep_list = kmalloc_array(incomplete, + sizeof(*node-Quoted 1 line
dep_list),drivers/gpu/drm/i915/i915_scheduler.c new file +14
-0+ GFP_KERNEL); +if (!node->dep_list) { +kfree (node); +return -ENOMEM ; + } + }else + node->dep_list =NULL ; + +spin_lock_irq (&scheduler->lock); + node->num_deps =0 ; + +if (node->dep_list) { +for (r =0 ; r <I915_NUM_RINGS ; r++) + i915_generate_dependencies(scheduler,node, r);
drivers/gpu/drm/i915/i915_scheduler.c new file +14
-0+ +WARN_ON (node->num_deps > incomplete); + } + + node->priority =clamp (node->priority, + scheduler->priority_level_min, + scheduler->priority_level_max); + +if ((node->priority >0 ) && node->num_deps) { +i915_scheduler_priority_bump_clear (scheduler); + +for (i =0 ; i < node->num_deps; i++) + i915_scheduler_priority_bump(scheduler, + node->dep_list[i], node-Quoted 4 lines
Quoted 1 line
Quoted 1 line
priority);
If node is posted with maximum priority to begin with, wouldn't the priority propagation stop after first level dependencies due to the check in the beginning of
priority_bumpfunction?The recursion will stop whenever it encounters a node already at maximum
priority. However, it is not starting from the newly submitted node but
from that node's dependencies. If they are already at max priority then
their dependencies must also be and so on down the chain and no further
processing is required. On the other hand, if they are not then they will be bumped and their dependencies checked and so on down the chain.Quoted 156 lines
Quoted 156 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+ + list_add_tail(&node->link, &scheduler->node_queue[ring-Quoted 1 line
id]);
drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+ + not_flying = i915_scheduler_count_flying(scheduler,ring) <
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ scheduler-Quoted 1 line
min_flying;drivers/gpu/drm/i915/i915_scheduler.c new file +11
-0+ +spin_unlock_irq (&scheduler->lock); + +if (not_flying) +i915_scheduler_submit (ring); + +return 0 ; +} + +/** +* i915_scheduler_notify_request - Notify the scheduler that the given
+ * request has completed on the hardware.
+ * @req: Request structure which has completed
+ * @preempt: Did it complete pre-emptively?
+ * A sequence number has popped out of the hardware and the request handling
+ * code has mapped it back to a request and will mark that request complete.
+ * It also calls this function to notify the scheduler about the completion
+ * so the scheduler's node can be updated appropriately.
+ * Returns true if the request is scheduler managed, false if not. The return
+ * value is combined for all freshly completed requests and if any were true
+ * theni915_scheduler_wakeup()is called so the scheduler can do further
+ * processing (submit more work) at the end.
+ */
+booli915_scheduler_notify_request(struct drm_i915_gem_request *req)drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+{ + struct drm_i915_private *dev_priv = to_i915(req->ring-Quoted 1 line
dev);
drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+struct i915_scheduler *scheduler = dev_priv->scheduler; + struct i915_scheduler_queue_entry *node = req-Quoted 1 line
scheduler_qe;drivers/gpu/drm/i915/i915_scheduler.c new file +21
-0+unsigned long flags; + +if (!node) +return false; + +spin_lock_irqsave (&scheduler->lock, flags); + +WARN_ON (!I915_SQS_IS_FLYING (node)); + +/* Node was in flight so mark it as complete. */ +if (req->cancelled) + node->status = i915_sqs_dead; +else + node->status = i915_sqs_complete; + +spin_unlock_irqrestore (&scheduler->lock, flags); + +return true; +} + +static int i915_scheduler_remove_dependent(struct i915_scheduler*scheduler,
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ struct
i915_scheduler_queue_entry*remove)drivers/gpu/drm/i915/i915_scheduler.c new file +7
-0+{ +struct i915_scheduler_queue_entry *node; +int i, r; +int count =0 ; + +/* +* Ensure that a node is not being removed which is still dependent
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * upon other (not completed) work. If that happens, itimplies
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * something has gone very wrong with the dependencytracking! Note
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * that there is no need to worry if this node has beenexplicitly
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * killed for some reason - it might be being killedbefore it got
drivers/gpu/drm/i915/i915_scheduler.c new file +6
-0+ * sent to the hardware. + */ + if (remove->status != i915_sqs_dead) { + for (i = 0; i < remove->num_deps; i++) + if ((remove->dep_list[i]) && + (!I915_SQS_IS_COMPLETE(remove-Quoted 1 line
dep_list[i])))
drivers/gpu/drm/i915/i915_scheduler.c new file +6
-0+ count++; +WARN_ON (count); + } + +/* +* Remove this node from the dependency lists of any other node which
drivers/gpu/drm/i915/i915_scheduler.c new file +4
-0+* might be waiting on it. +*/ +for (r =0 ; r <I915_NUM_RINGS ; r++) { + list_for_each_entry(node, &scheduler-Quoted 1 line
node_queue[r], link) {
drivers/gpu/drm/i915/i915_scheduler.c new file +16
-0+for (i =0 ; i < node->num_deps; i++) { +if (node->dep_list[i] != remove) +continue ; + + node->dep_list[i] =NULL ; + } + } + } + +return 0 ; +} + +/** +* i915_scheduler_wakeup - wake the scheduler's worker thread +* @dev: DRM device +* Called at the end of seqno interrupt processing if any request has
+ * completed that corresponds to a scheduler node.
+ */drivers/gpu/drm/i915/i915_scheduler.c new file +3
-0+void i915_scheduler_wakeup (struct drm_device *dev) +{ +/* XXX: Need to call i915_scheduler_remove() via work handler. */
drivers/gpu/drm/i915/i915_scheduler.c new file +4
-0+} + +/** +* i915_scheduler_clean_node - free up any allocations/references
+ * associated with the given scheduler queue entry.
+ * @node: Queue entry structure which is complete
+ * After a give batch buffer completes on the hardware, all the information
+ * required to resubmit it is no longer required. However, the node entry
+ * itself might still be required for tracking purposes for a while longer.
+ * This function should be called as soon as the node is known to be complete
+ * so that these resources may be freed even though the node itself might
+ * hang around.
+ */
+voidi915_scheduler_clean_node(struct i915_scheduler_queue_entry *node)drivers/gpu/drm/i915/i915_scheduler.c new file +4
-0+{ +if (!I915_SQS_IS_COMPLETE (node)) { + WARN(!node->params.request->cancelled, + "Cleaning active node: %d!\n", node-Quoted 1 line
status);
Quoted 16 lines
Quoted 12 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +6
-0+return ; + } + +if (node->params.batch_obj) { +/* +* The batch buffer must be unpinned before it is unreferenced
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * otherwise the unpin fails with a missingvma!?
drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+*/ + if (node->params.dispatch_flags &
I915_DISPATCH_SECURE)drivers/gpu/drm/i915/i915_scheduler.c
Quoted 3 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +2
-1+ i915_gem_execbuff_release_batch_obj(node ->params.batch_obj); +This code seems little bit out of place.
Not sure where else you could put it. The code itself is just a copy of
the existing clean up code that was already in the execbuff code path.
The cleanup can no longer be done within the execbuff call due to the
batch buffer execution being arbitrarily delayed by the scheduler. Thus
it is down to the scheduler to do all the necessary cleanup when it knows that the batch buffer is no longer required.Quoted 16 lines
drivers/gpu/drm/i915/i915_scheduler.c line 739
Quoted 13 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+ node->params.batch_obj =NULL ; + } + +/* And anything else owned by the node: */ +if (node->params.cliprects) { +kfree (node->params.cliprects); + node->params.cliprects =NULL ; + } +} +The below function is quite hacky, I understood this will be mitigated
by per-ctx sequence numbering in the future?To be honest, I think it is already obsolete due to the previous work of
converting from seqno tests everywhere to testing request structures.
The point of the code below was to cope with seqno values popping out of
the hardware in random order due to the re-ordering of batch buffers prior to execution. Whereas, the struct request conversion allowed the
seqno to be late allocated and thus kept in order.Unfortunately, pre-emption confuses matters again. Although the intention is that seqno values will still be kept in order. It's just
that a given request might have multiple different seqno values assigned
to it throughout its life. The pre-emption code hasn't quite settled down yet though, especially with interesting new hardware planned for
future chips. Thus I don't really want to rip out all of this code quite
yet just in case we do still need it.As you say, per-ctx seqnos would definitely have an effect here as well.
So yes, long term the intention is to remove all this nastyness. Short
term, it seems best to keep it around until proven unnecessary.Quoted 104 lines
Quoted 104 lines
+static bool i915_scheduler_remove(struct
i915_scheduler*scheduler,drivers/gpu/drm/i915/i915_scheduler.c new file +12
-0+ struct intel_engine_cs *ring, + struct list_head *remove) +{ +struct i915_scheduler_queue_entry *node, *node_next; +int flying =0 , queued =0 ; +bool do_submit; +uint32_t min_seqno; + +spin_lock_irq (&scheduler->lock); + +/* +* In the case where the system is idle, starting '
min_seqno' from a bigdrivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * number will cause all nodes to be removed as they arenow back to
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * being in-order. However, this will be a problem ifthe last one to
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * complete was actually out-of-order as the ring seqnovalue will be
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * lower than one or more completed buffers. Thus codelooking for the
drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+ * completion of said buffers will wait forever. + * Instead, use the hardware seqno as the startingpoint. This means
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * that some buffers might be kept around even in acompletely idle
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * system but it should guarantee that no-one ever getsconfused when
drivers/gpu/drm/i915/i915_scheduler.c new file +5
-0+* waiting for buffer completion. +*/ + min_seqno = ring->get_seqno (ring, true); + + list_for_each_entry(node, &scheduler->node_queue[ring-Quoted 1 line
id], link) {
drivers/gpu/drm/i915/i915_scheduler.c new file +11
-0+if (I915_SQS_IS_QUEUED (node)) + queued++; +else if (I915_SQS_IS_FLYING (node)) + flying++; +else if (I915_SQS_IS_COMPLETE (node)) +continue ; + +if (node->params.request->seqno ==0 ) +continue ; + + if (!i915_seqno_passed(node->params.request-Quoted 1 line
seqno,
min_seqno))drivers/gpu/drm/i915/i915_scheduler.c new file +5
-0+ min_seqno = node->params.request->seqno; + } + +INIT_LIST_HEAD (remove); + list_for_each_entry_safe(node, node_next, &scheduler-Quoted 1 line
node_queue[ring->id], link) {
drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+/* +* Only remove completed nodes which have a lower seqno than
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * all pending nodes. While there is thepossibility of the
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * ring's seqno counting backwards, all higherbuffers must
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * be remembered so that the'
i915_seqno_passed()' test candrivers/gpu/drm/i915/i915_scheduler.c new file +3
-0+ * report that they have in fact passed. + * + * NB: This is not true for 'dead' nodes. TheGPU reset causes
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * the software seqno to restart from itsinitial value. Thus
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * the dead nodes must be removed even thoughtheir seqno values
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ * are potentially vastly greater than thecurrent ring seqno.
drivers/gpu/drm/i915/i915_scheduler.c new file +6
-0+*/ +if (!I915_SQS_IS_COMPLETE (node)) +continue ; + +if (node->status != i915_sqs_dead) { + if (i915_seqno_passed(node-Quoted 1 line
params.request->seqno,min_seqno) &&drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ (node->params.request->seqno !=
min_seqno))drivers/gpu/drm/i915/i915_scheduler.c new file +7
-0+continue ; + } + +list_del (&node->link); +list_add (&node->link, remove); + +/* Strip the dependency info while the mutex is still locked */
drivers/gpu/drm/i915/i915_scheduler.c new file +1
-0+ i915_scheduler_remove_dependent(scheduler,node);
drivers/gpu/drm/i915/i915_scheduler.c new file +6
-0+ +continue ; + } + +/* +* Release the interrupt reference count if there are no longer any
drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+* nodes to worry about. +*/ Quoted 8 lines
Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+if (!flying && !queued && + (scheduler->flags[ring->id] &
i915_sf_interrupts_enabled)) {drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+ ring->irq_put (ring); + scheduler->flags[ring->id] &=~i915_sf_interrupts_enabled;
Maybe have this piece of code as a helper? To enable interrupts and disable them, maybe even a reference count?
Not sure how much you could abstract out into a helper other than the
two lines of irq_get|put + flag set|clear. The flying|queued test is particular to this one instance. Also, there is no need to reference count as the irq code does that already.Quoted 53 lines
Quoted 53 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +4
-0+ } + +/* Launch more packets now? */ + do_submit = (queued > 0) && (flying < scheduler-Quoted 1 line
min_flying);drivers/gpu/drm/i915/i915_scheduler.c new file +9
-0+ +spin_unlock_irq (&scheduler->lock); + +return do_submit; +} + +void i915_scheduler_process_work (struct intel_engine_cs *ring) +{ + struct drm_i915_private *dev_priv = ring->dev-Quoted 1 line
dev_private;drivers/gpu/drm/i915/i915_scheduler.c new file +10
-0+struct i915_scheduler *scheduler = dev_priv->scheduler; +struct i915_scheduler_queue_entry *node; +bool do_submit; +struct list_head remove; + +if (list_empty (&scheduler->node_queue[ring->id])) +return ; + +/* Remove completed nodes. */ + do_submit = i915_scheduler_remove(scheduler, ring,&remove);
drivers/gpu/drm/i915/i915_scheduler.c new file +15
-0+ +if (!do_submit &&list_empty (&remove)) +return ; + +/* Need to grab the pm lock outside of the mutex lock */ +if (do_submit) +intel_runtime_pm_get (dev_priv); + +mutex_lock (&ring->dev->struct_mutex); + +if (do_submit) +i915_scheduler_submit (ring); + +while (!list_empty (&remove)) { + node = list_first_entry(&remove, typeof(*node),link);
drivers/gpu/drm/i915/i915_scheduler.c new file +8
-0+list_del (&node->link); + +/* Free up all the DRM references */ +i915_scheduler_clean_node (node); + +/* And anything else owned by the node: */ + node->params.request->scheduler_qe =NULL ; + i915_gem_request_unreference(node-Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.c line 867
Quoted 3 lines
Quoted 1 line
params.request);
drivers/gpu/drm/i915/i915_scheduler.c new file +2
-0+kfree (node->dep_list); +kfree (node);Maybe have a separate helper for freeing a node, would make this much
cleaner.There is a helper for freeing up the contents of the node - see
i915_scheduler_clean_node()above. However, that function is called from
multiple places and this is the only instance where the node itself is
actually freed. The other callers either need to keep the node around
for a while longer (i.e. until this piece of code can run) or don't own
the node because it is stack allocated (in the bypass case). So the 'anything else' section cannot be moved into the existing clean function
and any secondary helper would be single usage and only four lines long.
It doesn't seem worth it.Quoted 69 lines
Quoted 69 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7
-0+ } + +mutex_unlock (&ring->dev->struct_mutex); + +if (do_submit) +intel_runtime_pm_put (dev_priv); +}drivers/gpu/drm/i915/i915_scheduler.h new file +4
-0@@ -0,0 +1,95 @@ +/* +* Copyright (c) 2014 Intel Corporation +* +* Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */drivers/gpu/drm/i915/i915_scheduler.h new file +15
-0+ +#ifndef _I915_SCHEDULER_H_ +#define _I915_SCHEDULER_H_ + +enum i915_scheduler_queue_status { +/* Limbo: */ + i915_sqs_none =0 , +/* Not yet submitted to hardware: */ + i915_sqs_queued, +/* Popped from queue, ready to fly: */ + i915_sqs_popped, +/* Sent to hardware for processing: */ + i915_sqs_flying, +/* Finished processing on the hardware: */ + i915_sqs_complete,Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.h line 44
Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +1
-0+/* Killed by watchdog or catastrophic submission failure: */
drivers/gpu/drm/i915/i915_scheduler.h new file +5
-0+ i915_sqs_dead, +/* Limit value for use with arrays/loops */ + i915_sqs_MAX +}; +To uppercase.
Grrr. Linux kernel style guide is daft.
Quoted 69 lines
Quoted 21 lines
+#define
I915_SQS_IS_QUEUED(node)(((node)->status ==i915_sqs_queued))
+#defineI915_SQS_IS_FLYING(node)(((node)->status ==i915_sqs_flying))
+#defineI915_SQS_IS_COMPLETE(node)(((node)->status ==i915_sqs_complete) || \drivers/gpu/drm/i915/i915_scheduler.h new file +1
-0+ ((node)->status ==
i915_sqs_dead))drivers/gpu/drm/i915/i915_scheduler.h new file +9
-0+ +struct i915_scheduler_obj_entry { + struct drm_i915_gem_object *obj; + bool read_only; +}; + +struct i915_scheduler_queue_entry { + struct i915_execbuffer_params params; +/* -1023 = lowest priority, 0 = default, 1023 = highest */
drivers/gpu/drm/i915/i915_scheduler.h line 66
Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +9
-0+ int32_t priority; + struct i915_scheduler_obj_entry *saved_objects; + int num_objs; + bool bumped; + struct i915_scheduler_queue_entry **dep_list; + int num_deps; + enum i915_scheduler_queue_status status; + unsigned long stamp; + struct list_head link;Above whitespace is still incorrect and I could really use comments.
Quoted 11 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +10
-0+}; + +struct i915_scheduler { + struct list_head node_queue[I915_NUM_RINGS]; + uint32_t flags[I915_NUM_RINGS]; + spinlock_t lock; + +/* Tuning parameters: */ + int32_t priority_level_min; + int32_t priority_level_max;drivers/gpu/drm/i915/i915_scheduler.h line 86
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +10
-0+ int32_t priority_level_bump; + int32_t priority_level_preempt; + uint32_t min_flying; +}; + +/* Flag bits for i915_scheduler::flags */ +enum { + i915_sf_interrupts_enabled = (1 << 0), + i915_sf_submitting = (1 << 1), +};These must be
I915_UPPERCASE_SOMETHINGby convention.Quoted 13 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +4
-0+ +bool i915_scheduler_is_enabled (struct drm_device *dev); +int i915_scheduler_init (struct drm_device *dev); +void i915_scheduler_clean_node(struct i915_scheduler_queue_entry*node);
+inti915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
+booli915_scheduler_notify_request(struct drm_i915_gem_request *req);drivers/gpu/drm/i915/i915_scheduler.h new file +3
-0+void i915_scheduler_wakeup (struct drm_device *dev); + +#endif /* _I915_SCHEDULER_H_ */
Signature · 8 lines
Joonas Lahtinen Open Source Technology Center Intel Corporation _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 06/35] drm/i915: Start of GPU scheduler
Quoted 3 lines · Joonas Lahtinen
Hi,
The below answers are reasonable. So v6 should be the version.
Are you planning on looking at the other patches first or are you going to wait until v6 is posted?
Thanks,
John.
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 7/35 drm/i915: Prepare retire_requests to handle out-of-order seqnos
In reply to [email protected]From: John Harrison <[email protected]>
A major point of the GPU scheduler is that it re-orders batch buffers after they have been submitted to the driver. This leads to requests completing out of order. In turn, this means that the retire processing can no longer assume that all completed entries are at the front of the list. Rather than attempting to re-order the request list on a regular basis, it is better to simply scan the entire list.
v2: Removed deferred free code as no longer necessary due to request handling updates.
For: VIZ-1587
--- drivers/gpu/drm/i915/i915_gem.c | 31 +++++++++++++------------------ 1 file changed, 13 insertions(+), 18 deletions(-)
drivers/gpu/drm/i915/i915_gem.c +13 -18
@@ -3233,6 +3233,7 @@ void i915_gem_reset(struct drm_device *dev)void i915_gem_retire_requests_ring (struct intel_engine_cs *ring) { +struct drm_i915_gem_object *obj, *obj_next;struct drm_i915_gem_request *req, *req_next;LIST_HEAD (list_head);
@@ -3245,37 +3246,31 @@ i915_gem_retire_requests_ring(struct intel_engine_cs *ring)*/ i915_gem_request_notify (ring, false); +/* +* Note that request entries might be out of order due to rescheduling +* and pre-emption. Thus both lists must be processed in their entirety +* rather than stopping at the first non-complete entry. +*/ +/* Retire requests first as we use it above for the early return. * If we retire requests last, we may use a later seqno and so clear * the requests lists without clearing the active list, leading to * confusion. */ -while (!list_empty (&ring->request_list)) { -struct drm_i915_gem_request *request; - - request =list_first_entry (&ring->request_list, - struct drm_i915_gem_request, - list); - -if (!i915_gem_request_completed (request)) -break ; +list_for_each_entry_safe (req, req_next, &ring->request_list, list) { +if (!i915_gem_request_completed (req)) +continue ; -i915_gem_request_retire (request); +i915_gem_request_retire (req); }/* Move any buffers on the active list that are no longer referenced * by the ringbuffer to the flushing/inactive lists as appropriate, * before we free the context associated with the requests. */ -while (!list_empty (&ring->active_list)) { -struct drm_i915_gem_object *obj; - - obj =list_first_entry (&ring->active_list, - struct drm_i915_gem_object, - ring_list[ring->id]); - +list_for_each_entry_safe (obj, obj_next, &ring->active_list, ring_list[ring->id]) {if (!list_empty (&obj->last_read_req[ring->id]->list)) -break ; +continue ;i915_gem_object_retire__read (obj, ring->id); }
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 07/35] drm/i915: Prepare retire_requests to handle out-of-order seqnos
Quoted 71 lines · On 02/18/2016 06:26 AM, [email protected] wrote:
From: John Harrison <[email protected]>
A major point of the GPU scheduler is that it re-orders batch buffers after they have been submitted to the driver. This leads to requests completing out of order. In turn, this means that the retire processing can no longer assume that all completed entries are at the front of the list. Rather than attempting to re-order the request list on a regular basis, it is better to simply scan the entire list.
v2: Removed deferred free code as no longer necessary due to request handling updates.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>--- drivers/gpu/drm/i915/i915_gem.c | 31 +++++++++++++------------------ 1 file changed, 13 insertions(+), 18 deletions(-)drivers/gpu/drm/i915/i915_gem.c +11
-14@@ -3233,6 +3233,7 @@ void i915_gem_reset(struct drm_device *dev)void i915_gem_retire_requests_ring (struct intel_engine_cs *ring) { +struct drm_i915_gem_object *obj, *obj_next;struct drm_i915_gem_request *req, *req_next;LIST_HEAD (list_head);@@ -3245,37 +3246,31 @@ i915_gem_retire_requests_ring(struct intel_engine_cs *ring)*/ i915_gem_request_notify (ring, false); +/* +* Note that request entries might be out of order due to rescheduling +* and pre-emption. Thus both lists must be processed in their entirety +* rather than stopping at the first non-complete entry. +*/ +/* Retire requests first as we use it above for the early return. * If we retire requests last, we may use a later seqno and so clear * the requests lists without clearing the active list, leading to * confusion. */ -while (!list_empty (&ring->request_list)) { -struct drm_i915_gem_request *request; - - request =list_first_entry (&ring->request_list, - struct drm_i915_gem_request, - list); - -if (!i915_gem_request_completed (request)) -break ; +list_for_each_entry_safe (req, req_next, &ring->request_list, list) { +if (!i915_gem_request_completed (req)) +continue ; -i915_gem_request_retire (request); +i915_gem_request_retire (req); }/* Move any buffers on the active list that are no longer referenced * by the ringbuffer to the flushing/inactive lists as appropriate, * before we free the context associated with the requests. */ -while (!list_empty (&ring->active_list)) { -struct drm_i915_gem_object *obj; - - obj = list_first_entry(&ring->active_list,
drivers/gpu/drm/i915/i915_gem.c i915_gem_retire_requests_ring(struct intel_engine_cs *ring) · line 3276
Quoted 10 lines
drivers/gpu/drm/i915/i915_gem.c +2
-4- struct drm_i915_gem_object, - ring_list[ring->id]); - +list_for_each_entry_safe (obj, obj_next, &ring->active_list, ring_list[ring->id]) {if (!list_empty (&obj->last_read_req[ring->id]->list)) -break ; +continue ;i915_gem_object_retire__read (obj, ring->id); }
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 8/35 drm/i915: Disable hardware semaphores when GPU scheduler is enabled
In reply to [email protected]From: John Harrison <[email protected]>
Hardware sempahores require seqno values to be continuously incrementing. However, the scheduler's reordering of batch buffers means that the seqno values going through the hardware could be out of order. Thus semaphores can not be used.
On the other hand, the scheduler superceeds the need for hardware semaphores anyway. Having one ring stall waiting for something to complete on another ring is inefficient if that ring could be working on some other, independent task. This is what the scheduler is meant to do - keep the hardware as busy as possible by reordering batch buffers to avoid dependency stalls.
v4: Downgraded a BUG_ON to WARN_ON as the latter is preferred.
v5: Squashed the i915_scheduler.c portions down into the 'start of scheduler' patch. [Joonas Lahtinen]
For: VIZ-1587
--- drivers/gpu/drm/i915/i915_drv.c | 9 +++++++++ drivers/gpu/drm/i915/intel_ringbuffer.c | 4 ++++ 2 files changed, 13 insertions(+)
drivers/gpu/drm/i915/i915_drv.c +9 -0
@@ -34,6 +34,7 @@#include "i915_drv.h" #include "i915_trace.h" #include "intel_drv.h" +#include "i915_scheduler.h" #include <linux/console.h> #include <linux/module.h>
@@ -517,6 +518,14 @@ void intel_detect_pch(struct drm_device *dev)bool i915_semaphore_is_enabled (struct drm_device *dev) { +/* Hardware semaphores are not compatible with the scheduler due to the +* seqno values being potentially out of order. However, semaphores are +* also not required as the scheduler will handle interring dependencies +* and try do so in a way that does not cause dead time on the hardware. +*/ +if (i915_scheduler_is_enabled (dev)) +return false; +if (INTEL_INFO (dev)->gen <6 )return false;
drivers/gpu/drm/i915/intel_ringbuffer.c +4 -0
@@ -33,6 +33,7 @@#include <drm/i915_drm.h> #include "i915_trace.h" #include "intel_drv.h" +#include "i915_scheduler.h" int __intel_ring_space (int head,int tail,int size) {
@@ -1400,6 +1401,9 @@ gen6_ring_sync(struct drm_i915_gem_request *waiter_req,u32 wait_mbox = signaller->semaphore.mbox.wait[waiter->id];int ret; +/* Arithmetic on sequence numbers is unreliable with a scheduler. */ +WARN_ON (i915_scheduler_is_enabled (signaller->dev)); +/* Throughout all of the GEM code, seqno passed implies our current * seqno is >= the last seqno executed. However for hardware the * comparison is strictly greater than.
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 08/35] drm/i915: Disable hardware semaphores when GPU scheduler is enabled
Quoted 68 lines · On 02/18/2016 06:26 AM, [email protected] wrote:
From: John Harrison <[email protected]>
Hardware sempahores require seqno values to be continuously incrementing. However, the scheduler's reordering of batch buffers means that the seqno values going through the hardware could be out of order. Thus semaphores can not be used.
On the other hand, the scheduler superceeds the need for hardware semaphores anyway. Having one ring stall waiting for something to complete on another ring is inefficient if that ring could be working on some other, independent task. This is what the scheduler is meant to do - keep the hardware as busy as possible by reordering batch buffers to avoid dependency stalls.
v4: Downgraded a
BUG_ONtoWARN_ONas the latter is preferred.v5: Squashed the
i915_scheduler.cportions down into the 'start of scheduler' patch. [Joonas Lahtinen]For: VIZ-1587
Signed-off-by John Harrison <[email protected]>Cc Joonas Lahtinen <[email protected]>--- drivers/gpu/drm/i915/i915_drv.c | 9 +++++++++ drivers/gpu/drm/i915/intel_ringbuffer.c | 4 ++++ 2 files changed, 13 insertions(+)drivers/gpu/drm/i915/i915_drv.c +9
-0@@ -34,6 +34,7 @@#include "i915_drv.h" #include "i915_trace.h" #include "intel_drv.h" +#include "i915_scheduler.h" #include <linux/console.h> #include <linux/module.h> @@ -517,6 +518,14 @@ void intel_detect_pch(struct drm_device *dev)bool i915_semaphore_is_enabled (struct drm_device *dev) { +/* Hardware semaphores are not compatible with the scheduler due to the +* seqno values being potentially out of order. However, semaphores are +* also not required as the scheduler will handle interring dependencies +* and try do so in a way that does not cause dead time on the hardware. +*/ +if (i915_scheduler_is_enabled (dev)) +return false; +if (INTEL_INFO (dev)->gen <6 )return false;drivers/gpu/drm/i915/intel_ringbuffer.c +1
-0@@ -33,6 +33,7 @@#include <drm/i915_drm.h> #include "i915_trace.h" #include "intel_drv.h" +#include "i915_scheduler.h" int __intel_ring_space (int head,int tail,int size) {
drivers/gpu/drm/i915/intel_ringbuffer.c gen6_ring_sync(struct drm_i915_gem_request *waiter_req, · line 1409
Quoted 9 lines
drivers/gpu/drm/i915/intel_ringbuffer.c +3
-0@@ -1400,6 +1401,9 @@ gen6_ring_sync(struct drm_i915_gem_request *waiter_req,u32 wait_mbox = signaller->semaphore.mbox.wait[waiter->id];int ret; +/* Arithmetic on sequence numbers is unreliable with a scheduler. */ +WARN_ON (i915_scheduler_is_enabled (signaller->dev)); +/* Throughout all of the GEM code, seqno passed implies our current * seqno is >= the last seqno executed. However for hardware the * comparison is strictly greater than.
I'd rather get rid of this altogether, but I guess we'll need it for the older gens. Another option would be to make the sync_to hook NULL in the scheduler case, though I guess the failure mode is less desirable there.
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 9/35 drm/i915: Force MMIO flips when scheduler enabled
In reply to [email protected]From: John Harrison <[email protected]>
MMIO flips are the preferred mechanism now but more importantly, pipe based flips cause issues for the scheduler. Specifically, submitting work to the rings around the side of the scheduler could cause that work to be lost if the scheduler generates a pre-emption event on that ring.
For: VIZ-1587
--- drivers/gpu/drm/i915/intel_display.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/gpu/drm/i915/intel_display.c +3 -0
@@ -46,6 +46,7 @@#include <linux/dma_remapping.h> #include <linux/reservation.h> #include <linux/dma-buf.h> +#include "i915_scheduler.h" /* Primary plane formats for gen <= 3 */ static const uint32_t i8xx_primary_formats[] = {
@@ -11330,6 +11331,8 @@ static bool use_mmio_flip(struct intel_engine_cs *ring,return true; else if (i915.enable_execlists) return true; + else if (i915_scheduler_is_enabled(ring->dev)) + return true; else if (obj->base.dma_buf && !reservation_object_test_signaled_rcu(obj->base.dma_buf->resv,false ))
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled
Quoted 28 lines · On 02/18/2016 06:26 AM, [email protected] wrote:
From: John Harrison <[email protected]>
MMIO flips are the preferred mechanism now but more importantly, pipe based flips cause issues for the scheduler. Specifically, submitting work to the rings around the side of the scheduler could cause that work to be lost if the scheduler generates a pre-emption event on that ring.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>--- drivers/gpu/drm/i915/intel_display.c | 3 +++ 1 file changed, 3 insertions(+)drivers/gpu/drm/i915/intel_display.c +1
-0@@ -46,6 +46,7 @@#include <linux/dma_remapping.h> #include <linux/reservation.h> #include <linux/dma-buf.h> +#include "i915_scheduler.h" /* Primary plane formats for gen <= 3 */ static const uint32_t i8xx_primary_formats[] = {
drivers/gpu/drm/i915/intel_display.c static bool use_mmio_flip(struct intel_engine_cs *ring, · line 11338
Quoted 8 lines
drivers/gpu/drm/i915/intel_display.c +2
-0@@ -11330,6 +11331,8 @@ static bool use_mmio_flip(struct intel_engine_cs *ring,return true; else if (i915.enable_execlists) return true; + else if (i915_scheduler_is_enabled(ring->dev)) + return true; else if (obj->base.dma_buf && !reservation_object_test_signaled_rcu(obj->base.dma_buf->resv,false ))
Why can't we use mmio flips unconditionally? Maarten or Ville?
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled
Quoted 25 lines · Jesse Barnes
Quoted 24 lines · On 02/18/2016 06:26 AM, [email protected] wrote:
From: John Harrison <[email protected]>
MMIO flips are the preferred mechanism now but more importantly, pipe based flips cause issues for the scheduler. Specifically, submitting work to the rings around the side of the scheduler could cause that work to be lost if the scheduler generates a pre-emption event on that ring.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>--- drivers/gpu/drm/i915/intel_display.c | 3 +++ 1 file changed, 3 insertions(+)drivers/gpu/drm/i915/intel_display.c +1
-0@@ -46,6 +46,7 @@#include <linux/dma_remapping.h> #include <linux/reservation.h> #include <linux/dma-buf.h> +#include "i915_scheduler.h"
Quoted 14 lines
drivers/gpu/drm/i915/intel_display.c static bool use_mmio_flip(struct intel_engine_cs *ring, · line 11338
Quoted 12 lines
drivers/gpu/drm/i915/intel_display.c +2
-0/* Primary plane formats for gen <= 3 */ static const uint32_t i8xx_primary_formats[] = {@@ -11330,6 +11331,8 @@ static bool use_mmio_flip(struct intel_engine_cs *ring,return true; else if (i915.enable_execlists) return true; + else if (i915_scheduler_is_enabled(ring->dev)) + return true; else if (obj->base.dma_buf && !reservation_object_test_signaled_rcu(obj->base.dma_buf->resv,false ))Why can't we use mmio flips unconditionally? Maarten or Ville?
We do when execlists are used, which is always on gen9+. So I guess I'm missing the point of this patch. For gen5+ we could also do it trivially.
For older platforms it'd require a bit of work since we'd need to complete the flips from the vblank interrupt. Well, we actually do that already even with CS flips on those platforms, but we do look at the flip pending interrupt to figure out if CS already issued the flip or not. So that part would need changing.
I also think we should switch to using the vblank interrupt for this stuff on all platforms, mainly since the flip done interrupt is somewhat broken on at least BDW (no idea if it got fixed in SKL or later), and doing things in more than one way certainly won't decrease our bug count.
Signature · 6 lines
Ville Syrjälä Intel OTC _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled
Quoted 41 lines · Ville Syrjälä
Quoted 25 lines · Jesse Barnes
Quoted 24 lines · On 02/18/2016 06:26 AM, [email protected] wrote:
From: John Harrison <[email protected]>
MMIO flips are the preferred mechanism now but more importantly, pipe based flips cause issues for the scheduler. Specifically, submitting work to the rings around the side of the scheduler could cause that work to be lost if the scheduler generates a pre-emption event on that ring.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>--- drivers/gpu/drm/i915/intel_display.c | 3 +++ 1 file changed, 3 insertions(+)drivers/gpu/drm/i915/intel_display.c +1
-0@@ -46,6 +46,7 @@#include <linux/dma_remapping.h> #include <linux/reservation.h> #include <linux/dma-buf.h> +#include "i915_scheduler.h" Quoted 14 lines
drivers/gpu/drm/i915/intel_display.c static bool use_mmio_flip(struct intel_engine_cs *ring, · line 11338
Quoted 12 lines
drivers/gpu/drm/i915/intel_display.c +2
-0/* Primary plane formats for gen <= 3 */ static const uint32_t i8xx_primary_formats[] = {@@ -11330,6 +11331,8 @@ static bool use_mmio_flip(struct intel_engine_cs *ring,return true; else if (i915.enable_execlists) return true; + else if (i915_scheduler_is_enabled(ring->dev)) + return true; else if (obj->base.dma_buf && !reservation_object_test_signaled_rcu(obj->base.dma_buf->resv,false ))Why can't we use mmio flips unconditionally? Maarten or Ville?
Quoted 2 lines
We do when execlists are used, which is always on gen9+. So I guess I'm missing the point of this patch. For gen5+ we could also do it trivially.
didn't check if the scheduler is also enabled for gen8 (I guess it would be nice, that would cover BDW and BSW).
Quoted 11 lines
For older platforms it'd require a bit of work since we'd need to complete the flips from the vblank interrupt. Well, we actually do that already even with CS flips on those platforms, but we do look at the flip pending interrupt to figure out if CS already issued the flip or not. So that part would need changing.
I also think we should switch to using the vblank interrupt for this stuff on all platforms, mainly since the flip done interrupt is somewhat broken on at least BDW (no idea if it got fixed in SKL or later), and doing things in more than one way certainly won't decrease our bug count.
Yeah that's probably the way to go; I haven't checked the behavior on SKL either.
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled
Hey,
Jesse Barnes schreef op vr 19-02-2016 om 12:01 [-0800]:
Quoted 42 lines · Ville Syrjälä
Quoted 41 lines · Jesse Barnes
Quoted 39 lines · On 02/18/2016 06:26 AM, [email protected] wrote:
Quoted 28 lines
From: John Harrison <[email protected]>
MMIO flips are the preferred mechanism now but more importantly, pipe based flips cause issues for the scheduler. Specifically, submitting work to the rings around the side of the scheduler could cause that work to be lost if the scheduler generates a pre-emption event on that ring.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>--- drivers/gpu/drm/i915/intel_display.c | 3 +++ 1 file changed, 3 insertions(+)drivers/gpu/drm/i915/intel_display.c +1
-0@@ -46,6 +46,7 @@#include <linux/dma_remapping.h> #include <linux/reservation.h> #include <linux/dma-buf.h> +#include "i915_scheduler.h" /* Primary plane formats for gen <= 3 */ static const uint32_t i8xx_primary_formats[] = {drivers/gpu/drm/i915/intel_display.c static bool use_mmio_flip(struct intel_engine_cs *ring, · line 11338
Quoted 8 lines
drivers/gpu/drm/i915/intel_display.c +2
-0@@ -11330,6 +11331,8 @@ static bool use_mmio_flip(struct intel_engine_cs *ring,return true; else if (i915.enable_execlists) return true; + else if (i915_scheduler_is_enabled(ring->dev)) + return true; else if (obj->base.dma_buf && !reservation_object_test_signaled_rcu(obj->base.dma_buf->resv,false ))Why can't we use mmio flips unconditionally? Maarten or Ville?
Quoted 4 lines
Quoted 2 lines
We do when execlists are used, which is always on gen9+. So I guess I'm missing the point of this patch. For gen5+ we could also do it trivially.
didn't check if the scheduler is also enabled for gen8 (I guess it would be nice, that would cover BDW and BSW).
I have a patch that would enable support for mmio flips on all platforms, but because of Chris Wilson's objections I still didn't force mmio flips by default.
Quoted 6 lines
Quoted 6 lines
For older platforms it'd require a bit of work since we'd need to complete the flips from the vblank interrupt. Well, we actually do that already even with CS flips on those platforms, but we do look at the flip pending interrupt to figure out if CS already issued the flip or not. So that part would need changing.
It was easy to fix in a way similar to that.
Quoted 6 lines
Quoted 4 lines
I also think we should switch to using the vblank interrupt for this stuff on all platforms, mainly since the flip done interrupt is somewhat broken on at least BDW (no idea if it got fixed in SKL or later), and doing things in more than one way certainly won't decrease our bug count.
Yeah that's probably the way to go; I haven't checked the behavior on SKL either.
---------------------------------------------------------------------
Intel International B.V.
Registered in The Netherlands under number 34098535 Statutory seat: Haarlemmermeer
Registered address: Capronilaan 37, 1119NG Schiphol-Rijk
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled
So is the summary that currently MMIO flips do not work on some platforms?
That could be a problem because the scheduler is intended to be enabled for everything and thus will be forcing MMIO flips on everything. So should I reverse the logic here - only enable the scheduler if MMIO flips are already enabled?
Thanks,
John.
Quoted 51 lines · Maarten
Hey,
Jesse Barnes schreef op vr 19-02-2016 om 12:01 [-0800]:
Quoted 40 lines · Ville Syrjälä
Quoted 39 lines · Jesse Barnes
Quoted 38 lines · On 02/18/2016 06:26 AM, [email protected] wrote:
Quoted 28 lines
From: John Harrison <[email protected]>
MMIO flips are the preferred mechanism now but more importantly, pipe based flips cause issues for the scheduler. Specifically, submitting work to the rings around the side of the scheduler could cause that work to be lost if the scheduler generates a pre-emption event on that ring.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>--- drivers/gpu/drm/i915/intel_display.c | 3 +++ 1 file changed, 3 insertions(+)drivers/gpu/drm/i915/intel_display.c +1
-0@@ -46,6 +46,7 @@#include <linux/dma_remapping.h> #include <linux/reservation.h> #include <linux/dma-buf.h> +#include "i915_scheduler.h" /* Primary plane formats for gen <= 3 */ static const uint32_t i8xx_primary_formats[] = {drivers/gpu/drm/i915/intel_display.c static bool use_mmio_flip(struct intel_engine_cs *ring, · line 11338
Quoted 8 lines
drivers/gpu/drm/i915/intel_display.c +2
-0@@ -11330,6 +11331,8 @@ static bool use_mmio_flip(struct intel_engine_cs *ring,return true; else if (i915.enable_execlists) return true; + else if (i915_scheduler_is_enabled(ring->dev)) + return true; else if (obj->base.dma_buf && !reservation_object_test_signaled_rcu(obj->base.dma_buf->resv,false ))Why can't we use mmio flips unconditionally? Maarten or Ville?
Quoted 3 lines
Quoted 2 lines
We do when execlists are used, which is always on gen9+. So I guess I'm missing the point of this patch. For gen5+ we could also do it trivially.
didn't check if the scheduler is also enabled for gen8 (I guess it would be nice, that would cover BDW and BSW).
I have a patch that would enable support for mmio flips on all platforms, but because of Chris Wilson's objections I still didn't force mmio flips by default.
Quoted 12 lines
Quoted 5 lines
Quoted 5 lines
For older platforms it'd require a bit of work since we'd need to complete the flips from the vblank interrupt. Well, we actually do that already even with CS flips on those platforms, but we do look at the flip pending interrupt to figure out if CS already issued the flip or not. So that part would need changing.
It was easy to fix in a way similar to that.
Quoted 5 lines
Quoted 4 lines
I also think we should switch to using the vblank interrupt for this stuff on all platforms, mainly since the flip done interrupt is somewhat broken on at least BDW (no idea if it got fixed in SKL or later), and doing things in more than one way certainly won't decrease our bug count.
Yeah that's probably the way to go; I haven't checked the behavior on SKL either.
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled
In reply to Jesse BarnesQuoted 4 lines · Jesse Barnes
Quoted 3 lines · On 02/18/2016 06:26 AM, [email protected] wrote:
From: John Harrison <[email protected]>
MMIO flips are the preferred mechanism now
Because introducing variable latency in waking up a big core is a good idea?
Quoted 5 lines
but more importantly, pipe
Quoted 4 lines
based flips cause issues for the scheduler. Specifically, submitting work to the rings around the side of the scheduler could cause that work to be lost if the scheduler generates a pre-emption event on that ring.
No. That is just incredibily bad design.
Quoted 1 line
Why can't we use mmio flips unconditionally? Maarten or Ville?
Why would we want to? CS flips work just fine in execlists and no reason was ever given as to why they were not enabled, just laziness. -Chris
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled
Quoted 8 lines · Chris Wilson
Quoted 4 lines · Jesse Barnes
Quoted 3 lines · On 02/18/2016 06:26 AM, [email protected] wrote:
From: John Harrison <[email protected]>
MMIO flips are the preferred mechanism now
Because introducing variable latency in waking up a big core is a good idea?
Is the latency variability really that bad? I'm not too concerned about waking up the core either, I think it's going to be hit up quite a bit in these situations anyway, and vblanks should be driving the rendering as well.
Quoted 6 lines
Quoted 5 lines
but more importantly, pipe
Quoted 4 lines
based flips cause issues for the scheduler. Specifically, submitting work to the rings around the side of the scheduler could cause that work to be lost if the scheduler generates a pre-emption event on that ring.
Quoted 6 lines
No. That is just incredibily bad design.
Quoted 1 line
Why can't we use mmio flips unconditionally? Maarten or Ville?
Why would we want to? CS flips work just fine in execlists and no reason was ever given as to why they were not enabled, just laziness.
I'm not saying it can't be done, but I thought with atomic we decided to go with mmio because it makes things a lot simpler (iirc Ville's design did that way back?).
I'm fine with either, but it seems like we should settle on one rather than trying to maintain two different ways of flipping. We'll have work to do either way.
Jesse
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled
Quoted 11 lines · Jesse Barnes
Quoted 8 lines · Chris Wilson
Quoted 4 lines · Jesse Barnes
Quoted 3 lines · On 02/18/2016 06:26 AM, [email protected] wrote:
From: John Harrison <[email protected]>
MMIO flips are the preferred mechanism now
Because introducing variable latency in waking up a big core is a good idea?
Is the latency variability really that bad? I'm not too concerned about waking up the core either, I think it's going to be hit up quite a bit in these situations anyway, and vblanks should be driving the rendering as well.
The workqueue we use is subject to being preempted by the user, so the latency can be made arbitrarily large. Also, we don't control what tasks are ahead of the flip.
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
but more importantly, pipe
Quoted 4 lines
based flips cause issues for the scheduler. Specifically, submitting work to the rings around the side of the scheduler could cause that work to be lost if the scheduler generates a pre-emption event on that ring.
Quoted 8 lines
Quoted 6 lines
No. That is just incredibily bad design.
Quoted 1 line
Why can't we use mmio flips unconditionally? Maarten or Ville?
Why would we want to? CS flips work just fine in execlists and no reason was ever given as to why they were not enabled, just laziness.
I'm not saying it can't be done, but I thought with atomic we decided to go with mmio because it makes things a lot simpler (iirc Ville's design did that way back?).
My point is that the interface to the scheduler excludes even the possibility of doing so because it is not cleanly segrated into a tool for determining the next request to execute but instead subsumes half of GEM and half of the engine backend becoming another unwanted midlayer incorporating its own policy. The scheduler doesn't even deal in requests, which is what prevents us from constructing a request here with the known dependency and passing it to the scheduler for execution.
As I saw it, CS flips would be one of the best demonstrations for a scheduler - because to execute the flip within the deadline should cause a priority boost for all the prerequisites, and then ideally reduce the number of missed target msc under load. (Though we can equally promote the last_write_request through another scheduler interface.)
-Chris
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 10/35 drm/i915: Added scheduler hook when closing DRM file handles
In reply to [email protected]From: John Harrison <[email protected]>
The scheduler decouples the submission of batch buffers to the driver with submission of batch buffers to the hardware. Thus it is possible for an application to close its DRM file handle while there is still work outstanding. That means the scheduler needs to know about file close events so it can remove the file pointer from such orphaned batch buffers and not attempt to dereference it later.
v3: Updated to not wait for outstanding work to complete but merely remove the file handle reference. The wait was getting excessively complicated with inter-ring dependencies, pre-emption, and other such issues.
v4: Changed some white space to keep the style checker happy.
v5: Added function documentation and removed apparently objectionable white space. [Joonas Lahtinen]
Used lighter weight spinlocks.
For: VIZ-1587
--- drivers/gpu/drm/i915/i915_dma.c | 3 +++ drivers/gpu/drm/i915/i915_scheduler.c | 48 +++++++++++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_scheduler.h | 2 ++ 3 files changed, 53 insertions(+)
drivers/gpu/drm/i915/i915_dma.c +3 -0
@@ -46,6 +46,7 @@#include <linux/vga_switcheroo.h> #include <linux/slab.h> #include <acpi/video.h> +#include "i915_scheduler.h" #include <linux/pm.h> #include <linux/pm_runtime.h> #include <linux/oom.h>
@@ -1258,6 +1259,8 @@ void i915_driver_lastclose(struct drm_device *dev)void i915_driver_preclose (struct drm_device *dev,struct drm_file *file) { +i915_scheduler_closefile (dev, file); +mutex_lock (&dev->struct_mutex);i915_gem_context_close (dev, file);i915_gem_release (dev, file);
drivers/gpu/drm/i915/i915_scheduler.c +48 -0
51 lines not on this page · raw message
drivers/gpu/drm/i915/i915_scheduler.h +2 -0
@@ -87,6 +87,8 @@ enum {bool i915_scheduler_is_enabled (struct drm_device *dev);int i915_scheduler_init (struct drm_device *dev); +int i915_scheduler_closefile (struct drm_device *dev, +struct drm_file *file);void i915_scheduler_clean_node (struct i915_scheduler_queue_entry *node);int i915_scheduler_queue_execbuffer (struct i915_scheduler_queue_entry *qe);bool i915_scheduler_notify_request (struct drm_i915_gem_request *req);
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 10/35] drm/i915: Added scheduler hook when closing DRM file handles
Quoted 46 lines · On to, 2016-02-18 at 14:26 +0000, [email protected] wrote:
From: John Harrison <[email protected]>
The scheduler decouples the submission of batch buffers to the driver with submission of batch buffers to the hardware. Thus it is possible for an application to close its DRM file handle while there is still work outstanding. That means the scheduler needs to know about file close events so it can remove the file pointer from such orphaned batch buffers and not attempt to dereference it later.
v3: Updated to not wait for outstanding work to complete but merely remove the file handle reference. The wait was getting excessively complicated with inter-ring dependencies, pre-emption, and other such issues.
v4: Changed some white space to keep the style checker happy.
v5: Added function documentation and removed apparently objectionable white space. [Joonas Lahtinen]
Used lighter weight spinlocks.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>Cc Joonas Lahtinen <[email protected]>---
drivers/gpu/drm/i915/i915_dma.c | 3 +++drivers/gpu/drm/i915/i915_scheduler.c| 48 +++++++++++++++++++++++++++++++++++drivers/gpu/drm/i915/i915_scheduler.h| 2 ++3 files changed, 53 insertions(+)drivers/gpu/drm/i915/i915_dma.c +1
-0@@ -46,6 +46,7 @@ #include #include #include +#include "i915_scheduler.h" #include #include #include@@ -1258,6 +1259,8 @@ void i915_driver_lastclose(struct drm_device *dev)
voidi915_driver_preclose(struct drm_device *dev, struct drm_file *file){
drivers/gpu/drm/i915/i915_dma.c void i915_driver_lastclose(struct drm_device *dev) · line 1262
Quoted 1 line
drivers/gpu/drm/i915/i915_dma.c +1
-0+i915_scheduler_closefile (dev, file);
Any reason not to put this inside the struct_mutex lock?
Quoted 18 lines
+
mutex_lock(&dev->struct_mutex);
i915_gem_context_close(dev, file);
i915_gem_release(dev, file);drivers/gpu/drm/i915/i915_scheduler.c +6
-0@@ -872,3 +872,51 @@ void i915_scheduler_process_work(struct intel_engine_cs *ring) if (do_submit) intel_runtime_pm_put(dev_priv); } + +/** +* i915_scheduler_closefile - notify the scheduler that a DRM file handle +* has been closed. +* @dev: DRM device +* @file: file being closed
drivers/gpu/drm/i915/i915_scheduler.c void i915_scheduler_process_work(struct intel_engine_cs *ring) · line 890
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c +10
-0+* +* Goes through the scheduler's queues and removes all connections to the +* disappearing file handle that still exist. There is an argument to say +* that this should also flush such outstanding work through the hardware. +* However, with pre-emption, TDR and other such complications doing so +* becomes a locking nightmare. So instead, just warn with a debug message +* if the application is leaking uncompleted work and make sure a null +* pointer dereference will not follow. +*/ +int i915_scheduler_closefile (struct drm_device *dev,struct drm_file *file)
Return value not used, should be void if this can not fail.
Other than that,
Quoted 45 lines
drivers/gpu/drm/i915/i915_scheduler.c +32
-0+{ +struct i915_scheduler_queue_entry *node; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; +struct intel_engine_cs *ring; +int i; + +if (!scheduler) +return 0 ; + +spin_lock_irq (&scheduler->lock); + +for_each_ring (ring, dev_priv, i) { +list_for_each_entry (node, &scheduler->node_queue[ring->id], link) { +if (node->params.file != file) +continue ; + +if (!I915_SQS_IS_COMPLETE (node)) +DRM_DEBUG_DRIVER ("Closing file handle with outstanding work: %d:%d/%d on %s\n" , + node->params.request->uniq, + node->params.request->seqno, + node->status, + ring->name); + + node->params.file =NULL ; + } + } + +spin_unlock_irq (&scheduler->lock); + +return 0 ; +}drivers/gpu/drm/i915/i915_scheduler.h +0
-0@@ -87,6 +87,8 @@ enum {
booli915_scheduler_is_enabled(struct drm_device *dev); inti915_scheduler_init(struct drm_device *dev);drivers/gpu/drm/i915/i915_scheduler.h +2
-0+int i915_scheduler_closefile (struct drm_device *dev, + struct drm_file *file);void
i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node); inti915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe); booli915_scheduler_notify_request(struct drm_i915_gem_request *req);
Signature · 7 lines
Joonas Lahtinen Open Source Technology Center Intel Corporation _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 10/35] drm/i915: Added scheduler hook when closing DRM file handles
Quoted 45 lines · Joonas Lahtinen
Quoted 44 lines · On to, 2016-02-18 at 14:26 +0000, [email protected] wrote:
From: John Harrison <[email protected]>
The scheduler decouples the submission of batch buffers to the driver with submission of batch buffers to the hardware. Thus it is possible for an application to close its DRM file handle while there is still work outstanding. That means the scheduler needs to know about file close events so it can remove the file pointer from such orphaned batch buffers and not attempt to dereference it later.
v3: Updated to not wait for outstanding work to complete but merely remove the file handle reference. The wait was getting excessively complicated with inter-ring dependencies, pre-emption, and other such issues.
v4: Changed some white space to keep the style checker happy.
v5: Added function documentation and removed apparently objectionable white space. [Joonas Lahtinen]
Used lighter weight spinlocks.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>Cc Joonas Lahtinen <[email protected]>--- drivers/gpu/drm/i915/i915_dma.c | 3 +++ drivers/gpu/drm/i915/i915_scheduler.c | 48 +++++++++++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_scheduler.h | 2 ++ 3 files changed, 53 insertions(+)drivers/gpu/drm/i915/i915_dma.c +1
-0@@ -46,6 +46,7 @@#include #include #include +#include "i915_scheduler.h" #include #include #include@@ -1258,6 +1259,8 @@ void i915_driver_lastclose(struct drm_device *dev)
Quoted 4 lines
drivers/gpu/drm/i915/i915_dma.c void i915_driver_lastclose(struct drm_device *dev) · line 1262
Quoted 3 lines
drivers/gpu/drm/i915/i915_dma.c +1
-0void i915_driver_preclose (struct drm_device *dev,struct drm_file *file) { +i915_scheduler_closefile (dev, file);Any reason not to put this inside the
struct_mutexlock?
Any reason to put it inside? The smaller the processing done while holding any given lock the better. The implementation only works on scheduler internal data structures so while it does require the scheduler's lock, it does not require the driver lock.
Quoted 22 lines
Quoted 22 lines
drivers/gpu/drm/i915/i915_dma.c +1
-0+mutex_lock (&dev->struct_mutex);i915_gem_context_close (dev, file);i915_gem_release (dev, file);drivers/gpu/drm/i915/i915_scheduler.c +9
-0@@ -872,3 +872,51 @@ void i915_scheduler_process_work(struct intel_engine_cs *ring)if (do_submit)intel_runtime_pm_put (dev_priv); } + +/** +* i915_scheduler_closefile - notify the scheduler that a DRM file handle +* has been closed. +* @dev: DRM device +* @file: file being closed +* +* Goes through the scheduler's queues and removes all connections to the +* disappearing file handle that still exist. There is an argument to say
Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c void i915_scheduler_process_work(struct intel_engine_cs *ring) · line 890
Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c +7
-0+* that this should also flush such outstanding work through the hardware. +* However, with pre-emption, TDR and other such complications doing so +* becomes a locking nightmare. So instead, just warn with a debug message +* if the application is leaking uncompleted work and make sure a null +* pointer dereference will not follow. +*/ +int i915_scheduler_closefile (struct drm_device *dev,struct drm_file *file)Return value not used, should be void if this can not fail.
Left over from a previous implementation that was much more complicated.
Quoted 42 lines
Other than that,
Reviewed-by Joonas Lahtinen <[email protected]>Quoted 38 lines
drivers/gpu/drm/i915/i915_scheduler.c +32
-0+{ +struct i915_scheduler_queue_entry *node; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; +struct intel_engine_cs *ring; +int i; + +if (!scheduler) +return 0 ; + +spin_lock_irq (&scheduler->lock); + +for_each_ring (ring, dev_priv, i) { +list_for_each_entry (node, &scheduler->node_queue[ring->id], link) { +if (node->params.file != file) +continue ; + +if (!I915_SQS_IS_COMPLETE (node)) +DRM_DEBUG_DRIVER ("Closing file handle with outstanding work: %d:%d/%d on %s\n" , + node->params.request->uniq, + node->params.request->seqno, + node->status, + ring->name); + + node->params.file =NULL ; + } + } + +spin_unlock_irq (&scheduler->lock); + +return 0 ; +}drivers/gpu/drm/i915/i915_scheduler.h +0
-0@@ -87,6 +87,8 @@ enum {
Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.h enum { · line 94
Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.h +2
-0bool i915_scheduler_is_enabled (struct drm_device *dev);int i915_scheduler_init (struct drm_device *dev); +int i915_scheduler_closefile (struct drm_device *dev, +struct drm_file *file);void i915_scheduler_clean_node (struct i915_scheduler_queue_entry *node);int i915_scheduler_queue_execbuffer (struct i915_scheduler_queue_entry *qe);bool i915_scheduler_notify_request (struct drm_i915_gem_request *req);
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 11/35 drm/i915: Added scheduler hook into i915_gem_request_notify()
In reply to [email protected]From: John Harrison <[email protected]>
The scheduler needs to know when requests have completed so that it can keep its own internal state up to date and can submit new requests to the hardware from its queue.
v2: Updated due to changes in request handling. The operation is now reversed from before. Rather than the scheduler being in control of completion events, it is now the request code itself. The scheduler merely receives a notification event. It can then optionally request it's worker thread be woken up after all completion processing is complete.
v4: Downgraded a BUG_ON to a WARN_ON as the latter is preferred.
v5: Squashed the i915_scheduler.c portions down into the 'start of scheduler' patch. [Joonas Lahtinen]
For: VIZ-1587
--- drivers/gpu/drm/i915/i915_gem.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
drivers/gpu/drm/i915/i915_gem.c +16 -0
@@ -2872,6 +2872,7 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked) {struct drm_i915_gem_request *req, *req_next;unsigned long flags; +bool wake_sched = false;u32 seqno;if (list_empty (&ring->fence_signal_list)) {
@@ -2908,6 +2909,14 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)*/ list_del_init (&req->signal_link); +/* +* NB: Must notify the scheduler before signalling +* the node. Otherwise the node can get retired first +* and call scheduler_clean() while the scheduler +* thinks it is still active. +*/ + wake_sched |=i915_scheduler_notify_request (req); +if (!req->cancelled) {fence_signal_locked (&req->fence);trace_i915_gem_request_complete (req);
@@ -2924,6 +2933,13 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)if (!fence_locked)spin_unlock_irqrestore (&ring->fence_lock, flags); + +/* Necessary? Or does the fence_signal() call do an implicit wakeup? */ +wake_up_all (&ring->irq_queue); + +/* Final scheduler processing after all individual updates are done. */ +if (wake_sched) +i915_scheduler_wakeup (ring->dev); }static const char *i915_gem_request_get_driver_name (struct fence *req_fence)
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Joonas Lahtinen
maintainerRe: [PATCH v5 11/35] drm/i915: Added scheduler hook into i915_gem_request_notify()
Quoted 41 lines · On to, 2016-02-18 at 14:26 +0000, [email protected] wrote:
From: John Harrison <[email protected]>
The scheduler needs to know when requests have completed so that it can keep its own internal state up to date and can submit new requests to the hardware from its queue.
v2: Updated due to changes in request handling. The operation is now reversed from before. Rather than the scheduler being in control of completion events, it is now the request code itself. The scheduler merely receives a notification event. It can then optionally request it's worker thread be woken up after all completion processing is complete.
v4: Downgraded a
BUG_ONto aWARN_ONas the latter is preferred.v5: Squashed the
i915_scheduler.cportions down into the 'start of scheduler' patch. [Joonas Lahtinen]For: VIZ-1587
Signed-off-by John Harrison <[email protected]>Cc Joonas Lahtinen <[email protected]>---
drivers/gpu/drm/i915/i915_gem.c | 16 ++++++++++++++++1 file changed, 16 insertions(+)drivers/gpu/drm/i915/i915_gem.c +1
-0@@ -2872,6 +2872,7 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked) { struct drm_i915_gem_request *req, *req_next; unsigned long flags; +bool wake_sched = false; u32 seqno;
if (list_empty(&ring->fence_signal_list)) {drivers/gpu/drm/i915/i915_gem.c +0
-0@@ -2908,6 +2909,14 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)*/ list_del_init(&req->signal_link);
drivers/gpu/drm/i915/i915_gem.c void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked) · line 2918
Quoted 7 lines
drivers/gpu/drm/i915/i915_gem.c +7
-0+/* +* NB: Must notify the scheduler before signalling +* the node. Otherwise the node can get retired first +* and call scheduler_clean() while the scheduler +* thinks it is still active. +*/ + wake_sched |=i915_scheduler_notify_request (req);
if (i915_scheduler_notify_request(req)) wake_sched = true;
drivers/gpu/drm/i915/i915_gem.c void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked) · line 2938
Quoted 11 lines
+
if (!req->cancelled) {fence_signal_locked(&req->fence);trace_i915_gem_request_complete(req);drivers/gpu/drm/i915/i915_gem.c +0
-0@@ -2924,6 +2933,13 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)
if (!fence_locked)spin_unlock_irqrestore(&ring->fence_lock, flags);drivers/gpu/drm/i915/i915_gem.c +3
-0+ +/* Necessary? Or does the fence_signal() call do an implicit wakeup? */ +wake_up_all (&ring->irq_queue);
This should probably be figured out after the struct fence series is merged.
Regards, Joonas
Quoted 7 lines
drivers/gpu/drm/i915/i915_gem.c +4
-0+ +/* Final scheduler processing after all individual updates are done. */ +if (wake_sched) +i915_scheduler_wakeup (ring->dev);}
static const char *i915_gem_request_get_driver_name(struct fence*req_fence)
Signature · 7 lines
Joonas Lahtinen Open Source Technology Center Intel Corporation _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 12/35 drm/i915: Added deferred work handler for scheduler
In reply to [email protected]From: John Harrison <[email protected]>
The scheduler needs to do interrupt triggered work that is too complex to do in the interrupt handler. Thus it requires a deferred work handler to process such tasks asynchronously.
v2: Updated to reduce mutex lock usage. The lock is now only held for the minimum time within the remove function rather than for the whole of the worker thread's operation.
v5: Removed objectionable white space and added some documentation. [Joonas Lahtinen]
For: VIZ-1587
--- drivers/gpu/drm/i915/i915_dma.c | 3 +++ drivers/gpu/drm/i915/i915_drv.h | 10 ++++++++++ drivers/gpu/drm/i915/i915_gem.c | 2 ++ drivers/gpu/drm/i915/i915_scheduler.c | 29 +++++++++++++++++++++++++++-- drivers/gpu/drm/i915/i915_scheduler.h | 1 + 5 files changed, 43 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_dma.c +3 -0
@@ -1158,6 +1158,9 @@ int i915_driver_unload(struct drm_device *dev)WARN_ON (unregister_oom_notifier (&dev_priv->mm.oom_notifier));unregister_shrinker (&dev_priv->mm.shrinker); +/* Cancel the scheduler work handler, which should be idle now. */ +cancel_work_sync (&dev_priv->mm.scheduler_work); +io_mapping_free (dev_priv->gtt.mappable);arch_phys_wc_del (dev_priv->gtt.mtrr);
drivers/gpu/drm/i915/i915_drv.h +10 -0
@@ -1291,6 +1291,16 @@ struct i915_gem_mm {struct delayed_work retire_work;/** +* New scheme is to get an interrupt after every work packet +* in order to allow the low latency scheduling of pending +* packets. The idea behind adding new packets to a pending +* queue rather than directly into the hardware ring buffer +* is to allow high priority packets to over take low priority +* ones. +*/ +struct work_struct scheduler_work; + +/** * When we detect an idle GPU, we want to turn on * powersaving features. So once we see that there * are no more requests outstanding and no more
drivers/gpu/drm/i915/i915_gem.c +2 -0
@@ -5427,6 +5427,8 @@ i915_gem_load(struct drm_device *dev) i915_gem_retire_work_handler);INIT_DELAYED_WORK (&dev_priv->mm.idle_work, i915_gem_idle_work_handler); +INIT_WORK (&dev_priv->mm.scheduler_work, + i915_scheduler_work_handler);init_waitqueue_head (&dev_priv->gpu_error.reset_queue); dev_priv->relative_constants_mode =I915_EXEC_CONSTANTS_REL_GENERAL ;
drivers/gpu/drm/i915/i915_scheduler.c +27 -2
@@ -697,7 +697,9 @@ static int i915_scheduler_remove_dependent(struct i915_scheduler *scheduler,*/ void i915_scheduler_wakeup (struct drm_device *dev) { -/* XXX: Need to call i915_scheduler_remove() via work handler. */ +struct drm_i915_private *dev_priv =to_i915 (dev); + +queue_work (dev_priv->wq, &dev_priv->mm.scheduler_work); }/**
@@ -827,7 +829,7 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,return do_submit; } -void i915_scheduler_process_work (struct intel_engine_cs *ring) +static void i915_scheduler_process_work (struct intel_engine_cs *ring) {struct drm_i915_private *dev_priv = ring->dev->dev_private;struct i915_scheduler *scheduler = dev_priv->scheduler;
@@ -874,6 +876,29 @@ void i915_scheduler_process_work(struct intel_engine_cs *ring) }/** +* i915_scheduler_work_handler - scheduler's work handler callback. +* @work: Work structure +* A lot of the scheduler's work must be done asynchronously in response to +* an interrupt or other event. However, that work cannot be done at +* interrupt time or in the context of the event signaller (which might in +* fact be an interrupt). Thus a worker thread is required. This function +* will cause the thread to wake up and do its processing. +*/ +void i915_scheduler_work_handler (struct work_struct *work) +{ +struct intel_engine_cs *ring; +struct drm_i915_private *dev_priv; +struct drm_device *dev; +int i; + + dev_priv = container_of(work, struct drm_i915_private, mm.scheduler_work); + dev = dev_priv->dev; + +for_each_ring (ring, dev_priv, i) +i915_scheduler_process_work (ring); +} + +/** * i915_scheduler_closefile - notify the scheduler that a DRM file handle * has been closed. * @dev: DRM device
drivers/gpu/drm/i915/i915_scheduler.h +1 -0
@@ -93,5 +93,6 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);int i915_scheduler_queue_execbuffer (struct i915_scheduler_queue_entry *qe);bool i915_scheduler_notify_request (struct drm_i915_gem_request *req);void i915_scheduler_wakeup (struct drm_device *dev); +void i915_scheduler_work_handler (struct work_struct *work);#endif /* _I915_SCHEDULER_H_ */
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 12/35] drm/i915: Added deferred work handler for scheduler
Quoted 91 lines · On to, 2016-02-18 at 14:27 +0000, [email protected] wrote:
From: John Harrison <[email protected]>
The scheduler needs to do interrupt triggered work that is too complex to do in the interrupt handler. Thus it requires a deferred work handler to process such tasks asynchronously.
v2: Updated to reduce mutex lock usage. The lock is now only held for the minimum time within the remove function rather than for the whole of the worker thread's operation.
v5: Removed objectionable white space and added some documentation. [Joonas Lahtinen]
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>Cc Joonas Lahtinen <[email protected]>---
drivers/gpu/drm/i915/i915_dma.c | 3 +++ drivers/gpu/drm/i915/i915_drv.h | 10 ++++++++++ drivers/gpu/drm/i915/i915_gem.c | 2 ++ drivers/gpu/drm/i915/i915_scheduler.c | 29 +++++++++++++++++++++++++++-- drivers/gpu/drm/i915/i915_scheduler.h | 1 + 5 files changed, 43 insertions(+), 2 deletions(-)drivers/gpu/drm/i915/i915_dma.c +0
-0@@ -1158,6 +1158,9 @@ int i915_driver_unload(struct drm_device *dev) WARN_ON(unregister_oom_notifier(&dev_priv->mm.oom_notifier)); unregister_shrinker(&dev_priv->mm.shrinker);drivers/gpu/drm/i915/i915_dma.c +3
-0+/* Cancel the scheduler work handler, which should be idle now. */ +cancel_work_sync (&dev_priv->mm.scheduler_work); +
io_mapping_free(dev_priv->gtt.mappable);
arch_phys_wc_del(dev_priv->gtt.mtrr);drivers/gpu/drm/i915/i915_drv.h +0
-0@@ -1291,6 +1291,16 @@ struct i915_gem_mm { struct delayed_work retire_work;
/**drivers/gpu/drm/i915/i915_drv.h +10
-0+* New scheme is to get an interrupt after every work packet +* in order to allow the low latency scheduling of pending +* packets. The idea behind adding new packets to a pending +* queue rather than directly into the hardware ring buffer +* is to allow high priority packets to over take low priority +* ones. +*/ +struct work_struct scheduler_work; + +/** * When we detect an idle GPU, we want to turn on
* powersaving features. So once we see that there
* are no more requests outstanding and no moredrivers/gpu/drm/i915/i915_gem.c +2
-0@@ -5427,6 +5427,8 @@ i915_gem_load(struct drm_device *dev) i915_gem_retire_work_handler); INIT_DELAYED_WORK(&dev_priv->mm.idle_work, i915_gem_idle_work_handler); +INIT_WORK (&dev_priv->mm.scheduler_work, + i915_scheduler_work_handler); init_waitqueue_head(&dev_priv->gpu_error.reset_queue);
dev_priv->relative_constants_mode=I915_EXEC_CONSTANTS_REL_GENERAL;drivers/gpu/drm/i915/i915_scheduler.c +3
-1@@ -697,7 +697,9 @@ static int i915_scheduler_remove_dependent(struct i915_scheduler *scheduler,*/ void i915_scheduler_wakeup(struct drm_device *dev) { -/* XXX: Need to call i915_scheduler_remove() via work handler. */ +struct drm_i915_private *dev_priv =to_i915 (dev); + +queue_work (dev_priv->wq, &dev_priv->mm.scheduler_work); }
/**drivers/gpu/drm/i915/i915_scheduler.c +0
-0@@ -827,7 +829,7 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler, return do_submit; }
drivers/gpu/drm/i915/i915_scheduler.c static bool i915_scheduler_remove(struct i915_scheduler *scheduler, · line 832
Quoted 2 lines
drivers/gpu/drm/i915/i915_scheduler.c +1
-1-void i915_scheduler_process_work (struct intel_engine_cs *ring) +static void i915_scheduler_process_work (struct intel_engine_cs *ring)
This is an odd change, maybe should have been static void to begin with?
Quoted 15 lines
{
structdrm_i915_private*dev_priv=ring->dev->dev_private; structi915_scheduler*scheduler =dev_priv->scheduler;drivers/gpu/drm/i915/i915_scheduler.c +0
-0@@ -874,6 +876,29 @@ void i915_scheduler_process_work(struct intel_engine_cs *ring) }
/**
+ *i915_scheduler_work_handler- scheduler's work handler callback.
+ * @work: Work structure
+ * A lot of the scheduler's work must be done asynchronously in response to
+ * an interrupt or other event. However, that work cannot be done at
+ * interrupt time or in the context of the event signaller (which might in
+ * fact be an interrupt). Thus a worker thread is required. This function
+ * will cause the thread to wake up and do its processing.
+ */
drivers/gpu/drm/i915/i915_scheduler.c void i915_scheduler_process_work(struct intel_engine_cs *ring) · line 895
Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.c +9
-0+void i915_scheduler_work_handler (struct work_struct *work) +{ +struct intel_engine_cs *ring; +struct drm_i915_private *dev_priv; +struct drm_device *dev; +int i; + + dev_priv = container_of(work, struct drm_i915_private, mm.scheduler_work); + dev = dev_priv->dev;
dev is unused variable?
With those changes,
Quoted 20 lines
drivers/gpu/drm/i915/i915_scheduler.c +6
-0+ +for_each_ring (ring, dev_priv, i) +i915_scheduler_process_work (ring); +} + +/** *
i915_scheduler_closefile- notify the scheduler that a DRM file handle
* has been closed.
* @dev: DRM devicedrivers/gpu/drm/i915/i915_scheduler.h +1
-0@@ -93,5 +93,6 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node); int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe); bool i915_scheduler_notify_request(struct drm_i915_gem_request *req); void i915_scheduler_wakeup(struct drm_device *dev); +void i915_scheduler_work_handler (struct work_struct *work);
#endif /* _I915_SCHEDULER_H_ */
Signature · 7 lines
Joonas Lahtinen Open Source Technology Center Intel Corporation _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 12/35] drm/i915: Added deferred work handler for scheduler
Quoted 92 lines · Joonas Lahtinen
Quoted 91 lines · On to, 2016-02-18 at 14:27 +0000, [email protected] wrote:
From: John Harrison <[email protected]>
The scheduler needs to do interrupt triggered work that is too complex to do in the interrupt handler. Thus it requires a deferred work handler to process such tasks asynchronously.
v2: Updated to reduce mutex lock usage. The lock is now only held for the minimum time within the remove function rather than for the whole of the worker thread's operation.
v5: Removed objectionable white space and added some documentation. [Joonas Lahtinen]
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>Cc Joonas Lahtinen <[email protected]>--- drivers/gpu/drm/i915/i915_dma.c | 3 +++ drivers/gpu/drm/i915/i915_drv.h | 10 ++++++++++ drivers/gpu/drm/i915/i915_gem.c | 2 ++ drivers/gpu/drm/i915/i915_scheduler.c | 29 +++++++++++++++++++++++++++-- drivers/gpu/drm/i915/i915_scheduler.h | 1 + 5 files changed, 43 insertions(+), 2 deletions(-)drivers/gpu/drm/i915/i915_dma.c +3
-0@@ -1158,6 +1158,9 @@ int i915_driver_unload(struct drm_device *dev)WARN_ON (unregister_oom_notifier (&dev_priv->mm.oom_notifier));unregister_shrinker (&dev_priv->mm.shrinker); +/* Cancel the scheduler work handler, which should be idle now. */ +cancel_work_sync (&dev_priv->mm.scheduler_work); +io_mapping_free (dev_priv->gtt.mappable);arch_phys_wc_del (dev_priv->gtt.mtrr);drivers/gpu/drm/i915/i915_drv.h +10
-0@@ -1291,6 +1291,16 @@ struct i915_gem_mm {struct delayed_work retire_work;/** +* New scheme is to get an interrupt after every work packet +* in order to allow the low latency scheduling of pending +* packets. The idea behind adding new packets to a pending +* queue rather than directly into the hardware ring buffer +* is to allow high priority packets to over take low priority +* ones. +*/ +struct work_struct scheduler_work; + +/** * When we detect an idle GPU, we want to turn on * powersaving features. So once we see that there * are no more requests outstanding and no more drivers/gpu/drm/i915/i915_gem.c +2
-0@@ -5427,6 +5427,8 @@ i915_gem_load(struct drm_device *dev) i915_gem_retire_work_handler);INIT_DELAYED_WORK (&dev_priv->mm.idle_work, i915_gem_idle_work_handler); +INIT_WORK (&dev_priv->mm.scheduler_work, + i915_scheduler_work_handler);init_waitqueue_head (&dev_priv->gpu_error.reset_queue); dev_priv->relative_constants_mode =I915_EXEC_CONSTANTS_REL_GENERAL ;drivers/gpu/drm/i915/i915_scheduler.c +3
-1@@ -697,7 +697,9 @@ static int i915_scheduler_remove_dependent(struct i915_scheduler *scheduler,*/ void i915_scheduler_wakeup (struct drm_device *dev) { -/* XXX: Need to call i915_scheduler_remove() via work handler. */ +struct drm_i915_private *dev_priv =to_i915 (dev); + +queue_work (dev_priv->wq, &dev_priv->mm.scheduler_work); }/** @@ -827,7 +829,7 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,return do_submit; }
Quoted 4 lines
drivers/gpu/drm/i915/i915_scheduler.c static bool i915_scheduler_remove(struct i915_scheduler *scheduler, · line 832
Quoted 2 lines
drivers/gpu/drm/i915/i915_scheduler.c +1
-1-void i915_scheduler_process_work (struct intel_engine_cs *ring) +static void i915_scheduler_process_work (struct intel_engine_cs *ring)This is an odd change, maybe should have been static void to begin with?
The problems of introducing code in patch A and plumbing it to the driver in patch B. Making it static from the start gives an unused function compiler warning.
Quoted 49 lines
Quoted 14 lines
drivers/gpu/drm/i915/i915_scheduler.c +7
-0{struct drm_i915_private *dev_priv = ring->dev->dev_private;struct i915_scheduler *scheduler = dev_priv->scheduler;@@ -874,6 +876,29 @@ void i915_scheduler_process_work(struct intel_engine_cs *ring) }/** +* i915_scheduler_work_handler - scheduler's work handler callback. +* @work: Work structure +* A lot of the scheduler's work must be done asynchronously in response to +* an interrupt or other event. However, that work cannot be done at +* interrupt time or in the context of the event signaller (which might in +* fact be an interrupt). Thus a worker thread is required. This function +* will cause the thread to wake up and do its processing. drivers/gpu/drm/i915/i915_scheduler.c void i915_scheduler_process_work(struct intel_engine_cs *ring) · line 895
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c +10
-0+*/ +void i915_scheduler_work_handler (struct work_struct *work) +{ +struct intel_engine_cs *ring; +struct drm_i915_private *dev_priv; +struct drm_device *dev; +int i; + + dev_priv = container_of(work, struct drm_i915_private, mm.scheduler_work); + dev = dev_priv->dev;dev is unused variable?
With those changes,
Reviewed-by Joonas Lahtinen <[email protected]>Quoted 19 lines
drivers/gpu/drm/i915/i915_scheduler.c +6
-0+ +for_each_ring (ring, dev_priv, i) +i915_scheduler_process_work (ring); +} + +/** * i915_scheduler_closefile - notify the scheduler that a DRM file handle * has been closed. * @dev: DRM device drivers/gpu/drm/i915/i915_scheduler.h +1
-0@@ -93,5 +93,6 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);int i915_scheduler_queue_execbuffer (struct i915_scheduler_queue_entry *qe);bool i915_scheduler_notify_request (struct drm_i915_gem_request *req);void i915_scheduler_wakeup (struct drm_device *dev); +void i915_scheduler_work_handler (struct work_struct *work);
Quoted 1 line
drivers/gpu/drm/i915/i915_scheduler.h void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node); · line 98
Quoted 1 line
drivers/gpu/drm/i915/i915_scheduler.h +0
-0#endif /* _I915_SCHEDULER_H_ */
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 13/35 drm/i915: Redirect execbuffer_final() via scheduler
In reply to [email protected]From: John Harrison <[email protected]>
Updated the execbuffer() code to pass the packaged up batch buffer information to the scheduler rather than calling execbuffer_final() directly. The scheduler queue() code is currently a stub which simply chains on to _final() immediately.
For: VIZ-1587
--- drivers/gpu/drm/i915/i915_gem_execbuffer.c | 18 +++++++----------- drivers/gpu/drm/i915/intel_lrc.c | 12 ++++-------- 2 files changed, 11 insertions(+), 19 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +7 -11
@@ -33,6 +33,7 @@#include "intel_drv.h" #include <linux/dma_remapping.h> #include <linux/uaccess.h> +#include "i915_scheduler.h" #define __EXEC_OBJECT_HAS_PIN (1<<31)#define __EXEC_OBJECT_HAS_FENCE (1<<30)
@@ -1226,6 +1227,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,struct drm_i915_gem_execbuffer2 *args, struct list_head *vmas) { + struct i915_scheduler_queue_entry *qe;struct drm_device *dev = params->dev;struct intel_engine_cs *ring = params->ring;struct drm_i915_private *dev_priv = dev->dev_private;
@@ -1270,17 +1272,11 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,i915_gem_execbuffer_move_to_active (vmas, params->request); - ret = dev_priv->gt.execbuf_final (params); + qe =container_of (params,typeof (*qe), params); + ret =i915_scheduler_queue_execbuffer (qe);if (ret)return ret; -/* -* Free everything that was stored in the QE structure (until the -* scheduler arrives and does it instead): -*/ -if (params->dispatch_flags &I915_DISPATCH_SECURE ) -i915_gem_execbuff_release_batch_obj (params->batch_obj); -return 0 ; }
@@ -1420,8 +1416,8 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,struct intel_engine_cs *ring;struct intel_context *ctx;struct i915_address_space *vm; -struct i915_execbuffer_params params_master;/* XXX: will be removed later */ -struct i915_execbuffer_params *params = ¶ms_master; +struct i915_scheduler_queue_entry qe; +struct i915_execbuffer_params *params = &qe.params;const u32 ctx_id =i915_execbuffer2_get_context_id (*args);u32 dispatch_flags;int ret;
@@ -1529,7 +1525,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,else vm = &dev_priv->gtt.base; -memset (¶ms_master,0x00 ,sizeof (params_master)); +memset (&qe,0x00 ,sizeof (qe)); eb =eb_create (args);if (eb ==NULL ) {
drivers/gpu/drm/i915/intel_lrc.c +4 -8
@@ -136,6 +136,7 @@#include <drm/i915_drm.h> #include "i915_drv.h" #include "intel_mocs.h" +#include "i915_scheduler.h" #define GEN9_LR_CONTEXT_RENDER_SIZE (22 * PAGE_SIZE)#define GEN8_LR_CONTEXT_RENDER_SIZE (20 * PAGE_SIZE)
@@ -910,6 +911,7 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,struct drm_i915_gem_execbuffer2 *args, struct list_head *vmas) { + struct i915_scheduler_queue_entry *qe;struct drm_device *dev = params->dev;struct intel_engine_cs *ring = params->ring;struct drm_i915_private *dev_priv = dev->dev_private;
@@ -952,17 +954,11 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,i915_gem_execbuffer_move_to_active (vmas, params->request); - ret = dev_priv->gt.execbuf_final (params); + qe =container_of (params,typeof (*qe), params); + ret =i915_scheduler_queue_execbuffer (qe);if (ret)return ret; -/* -* Free everything that was stored in the QE structure (until the -* scheduler arrives and does it instead): -*/ -if (params->dispatch_flags &I915_DISPATCH_SECURE ) -i915_gem_execbuff_release_batch_obj (params->batch_obj); -return 0 ; }
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 13/35] drm/i915: Redirect execbuffer_final() via scheduler
Quoted 105 lines · On 02/18/2016 06:27 AM, [email protected] wrote:
From: John Harrison <[email protected]>
Updated the
execbuffer()code to pass the packaged up batch buffer information to the scheduler rather than callingexecbuffer_final()directly. The schedulerqueue()code is currently a stub which simply chains on to_final()immediately.For: VIZ-1587
Signed-off-by John Harrison <[email protected]>--- drivers/gpu/drm/i915/i915_gem_execbuffer.c | 18 +++++++----------- drivers/gpu/drm/i915/intel_lrc.c | 12 ++++-------- 2 files changed, 11 insertions(+), 19 deletions(-)drivers/gpu/drm/i915/i915_gem_execbuffer.c +7
-11@@ -33,6 +33,7 @@#include "intel_drv.h" #include <linux/dma_remapping.h> #include <linux/uaccess.h> +#include "i915_scheduler.h" #define __EXEC_OBJECT_HAS_PIN (1<<31)#define __EXEC_OBJECT_HAS_FENCE (1<<30)@@ -1226,6 +1227,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,struct drm_i915_gem_execbuffer2 *args, struct list_head *vmas) { + struct i915_scheduler_queue_entry *qe;struct drm_device *dev = params->dev;struct intel_engine_cs *ring = params->ring;struct drm_i915_private *dev_priv = dev->dev_private;@@ -1270,17 +1272,11 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,i915_gem_execbuffer_move_to_active (vmas, params->request); - ret = dev_priv->gt.execbuf_final (params); + qe =container_of (params,typeof (*qe), params); + ret =i915_scheduler_queue_execbuffer (qe);if (ret)return ret; -/* -* Free everything that was stored in the QE structure (until the -* scheduler arrives and does it instead): -*/ -if (params->dispatch_flags &I915_DISPATCH_SECURE ) -i915_gem_execbuff_release_batch_obj (params->batch_obj); -return 0 ; }@@ -1420,8 +1416,8 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,struct intel_engine_cs *ring;struct intel_context *ctx;struct i915_address_space *vm; -struct i915_execbuffer_params params_master;/* XXX: will be removed later */ -struct i915_execbuffer_params *params = ¶ms_master; +struct i915_scheduler_queue_entry qe; +struct i915_execbuffer_params *params = &qe.params;const u32 ctx_id =i915_execbuffer2_get_context_id (*args);u32 dispatch_flags;int ret;@@ -1529,7 +1525,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,else vm = &dev_priv->gtt.base; -memset (¶ms_master,0x00 ,sizeof (params_master)); +memset (&qe,0x00 ,sizeof (qe)); eb =eb_create (args);if (eb ==NULL ) {drivers/gpu/drm/i915/intel_lrc.c +4
-1@@ -136,6 +136,7 @@#include <drm/i915_drm.h> #include "i915_drv.h" #include "intel_mocs.h" +#include "i915_scheduler.h" #define GEN9_LR_CONTEXT_RENDER_SIZE (22 * PAGE_SIZE)#define GEN8_LR_CONTEXT_RENDER_SIZE (20 * PAGE_SIZE)@@ -910,6 +911,7 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,struct drm_i915_gem_execbuffer2 *args, struct list_head *vmas) { + struct i915_scheduler_queue_entry *qe;struct drm_device *dev = params->dev;struct intel_engine_cs *ring = params->ring;struct drm_i915_private *dev_priv = dev->dev_private;@@ -952,17 +954,11 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,i915_gem_execbuffer_move_to_active (vmas, params->request); - ret = dev_priv->gt.execbuf_final (params); + qe =container_of (params,typeof (*qe), params); + ret =i915_scheduler_queue_execbuffer (qe);if (ret)return ret;
drivers/gpu/drm/i915/intel_lrc.c int intel_execlists_submission(struct i915_execbuffer_params *params, · line 964
Quoted 10 lines
drivers/gpu/drm/i915/intel_lrc.c +0
-7-/* -* Free everything that was stored in the QE structure (until the -* scheduler arrives and does it instead): -*/ -if (params->dispatch_flags &I915_DISPATCH_SECURE ) -i915_gem_execbuff_release_batch_obj (params->batch_obj); -return 0 ; }
I think this is ok, but might need changes if some of the earlier patches see changes due to Joonas's reviews.
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 14/35 drm/i915: Keep the reserved space mechanism happy
In reply to [email protected]From: John Harrison <[email protected]>
Ring space is reserved when constructing a request to ensure that the subsequent 'add_request()' call cannot fail due to waiting for space on a busy or broken GPU. However, the scheduler jumps in to the middle of the execbuffer process between request creation and request submission. Thus it needs to cancel the reserved space when the request is simply added to the scheduler's queue and not yet submitted. Similarly, it needs to re-reserve the space when it finally does want to send the batch buffer to the hardware.
v3: Updated to use locally cached request pointer.
v5: Updated due to changes to earlier patches in series - for runtime PM calls and splitting bypass mode into a separate function.
For: VIZ-1587
--- drivers/gpu/drm/i915/i915_gem_execbuffer.c | 20 ++++++++++++++------ drivers/gpu/drm/i915/i915_scheduler.c | 4 ++++ drivers/gpu/drm/i915/intel_lrc.c | 13 +++++++++++-- 3 files changed, 29 insertions(+), 8 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +14 -6
@@ -1295,18 +1295,22 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)/* The mutex must be acquired before calling this function */ WARN_ON (!mutex_is_locked (¶ms->dev->struct_mutex)); + ret =intel_ring_reserve_space (req); +if (ret) + goto error; +/* * Unconditionally invalidate gpu caches and ensure that we do flush * any residual writes from the previous batch. */ ret =intel_ring_invalidate_all_caches (req);if (ret) -return ret; + goto error;/* Switch to the correct context for the batch */ ret =i915_switch_context (req);if (ret) -return ret; + goto error;WARN (params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1 <<ring->id),"%s didn't clear reload\n" , ring->name);
@@ -1315,7 +1319,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params) params->instp_mode != dev_priv->relative_constants_mode) { ret =intel_ring_begin (req,4 );if (ret) -return ret; + goto error;intel_ring_emit (ring,MI_NOOP );intel_ring_emit (ring,MI_LOAD_REGISTER_IMM (1 ));
@@ -1329,7 +1333,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)if (params->args_flags &I915_EXEC_GEN7_SOL_RESET ) { ret =i915_reset_gen7_sol_offsets (params->dev, req);if (ret) -return ret; + goto error; } exec_len = params->args_batch_len;
@@ -1343,13 +1347,17 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params) exec_start, exec_len, params->dispatch_flags);if (ret) -return ret; + goto error;trace_i915_gem_ring_dispatch (req, params->dispatch_flags);i915_gem_execbuffer_retire_commands (params); -return 0 ; +error: +if (ret) +intel_ring_reserved_space_cancel (req->ringbuf); + +return ret; }/**
drivers/gpu/drm/i915/i915_scheduler.c +4 -0
@@ -483,6 +483,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_enstruct i915_scheduler *scheduler = dev_priv->scheduler;int ret; +intel_ring_reserved_space_cancel (qe->params.request->ringbuf); + scheduler->flags[qe->params.ring->id] |= i915_sf_submitting; ret = dev_priv->gt.execbuf_final (&qe->params); scheduler->flags[qe->params.ring->id] &= ~i915_sf_submitting;
@@ -539,6 +541,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe) node->stamp = jiffies;i915_gem_request_reference (node->params.request); +intel_ring_reserved_space_cancel (node->params.request->ringbuf); +WARN_ON (node->params.request->scheduler_qe); node->params.request->scheduler_qe = node;
drivers/gpu/drm/i915/intel_lrc.c +11 -2
@@ -978,13 +978,17 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)/* The mutex must be acquired before calling this function */ WARN_ON (!mutex_is_locked (¶ms->dev->struct_mutex)); + ret =intel_logical_ring_reserve_space (req); +if (ret) + goto err; +/* * Unconditionally invalidate gpu caches and ensure that we do flush * any residual writes from the previous batch. */ ret =logical_ring_invalidate_all_caches (req);if (ret) -return ret; + goto err;if (ring == &dev_priv->ring[RCS ] && params->instp_mode != dev_priv->relative_constants_mode) {
@@ -1006,13 +1010,18 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params) ret = ring->emit_bb_start (req, exec_start, params->dispatch_flags);if (ret) -return ret; + goto err;trace_i915_gem_ring_dispatch (req, params->dispatch_flags);i915_gem_execbuffer_retire_commands (params);return 0 ; + +err: +intel_ring_reserved_space_cancel (params->request->ringbuf); + +return ret; }void intel_execlists_retire_requests (struct intel_engine_cs *ring)
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 14/35] drm/i915: Keep the reserved space mechanism happy
Quoted 147 lines · On 02/18/2016 06:27 AM, [email protected] wrote:
From: John Harrison <[email protected]>
Ring space is reserved when constructing a request to ensure that the subsequent '
add_request()' call cannot fail due to waiting for space on a busy or broken GPU. However, the scheduler jumps in to the middle of the execbuffer process between request creation and request submission. Thus it needs to cancel the reserved space when the request is simply added to the scheduler's queue and not yet submitted. Similarly, it needs to re-reserve the space when it finally does want to send the batch buffer to the hardware.v3: Updated to use locally cached request pointer.
v5: Updated due to changes to earlier patches in series - for runtime PM calls and splitting bypass mode into a separate function.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>--- drivers/gpu/drm/i915/i915_gem_execbuffer.c | 20 ++++++++++++++------ drivers/gpu/drm/i915/i915_scheduler.c | 4 ++++ drivers/gpu/drm/i915/intel_lrc.c | 13 +++++++++++-- 3 files changed, 29 insertions(+), 8 deletions(-)drivers/gpu/drm/i915/i915_gem_execbuffer.c +14
-6@@ -1295,18 +1295,22 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)/* The mutex must be acquired before calling this function */ WARN_ON (!mutex_is_locked (¶ms->dev->struct_mutex)); + ret =intel_ring_reserve_space (req); +if (ret) + goto error; +/* * Unconditionally invalidate gpu caches and ensure that we do flush * any residual writes from the previous batch. */ ret =intel_ring_invalidate_all_caches (req);if (ret) -return ret; + goto error;/* Switch to the correct context for the batch */ ret =i915_switch_context (req);if (ret) -return ret; + goto error;WARN (params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1 <<ring->id),"%s didn't clear reload\n" , ring->name);@@ -1315,7 +1319,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params) params->instp_mode != dev_priv->relative_constants_mode) { ret =intel_ring_begin (req,4 );if (ret) -return ret; + goto error;intel_ring_emit (ring,MI_NOOP );intel_ring_emit (ring,MI_LOAD_REGISTER_IMM (1 ));@@ -1329,7 +1333,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)if (params->args_flags &I915_EXEC_GEN7_SOL_RESET ) { ret =i915_reset_gen7_sol_offsets (params->dev, req);if (ret) -return ret; + goto error; } exec_len = params->args_batch_len;@@ -1343,13 +1347,17 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params) exec_start, exec_len, params->dispatch_flags);if (ret) -return ret; + goto error;trace_i915_gem_ring_dispatch (req, params->dispatch_flags);i915_gem_execbuffer_retire_commands (params); -return 0 ; +error: +if (ret) +intel_ring_reserved_space_cancel (req->ringbuf); + +return ret; }/** drivers/gpu/drm/i915/i915_scheduler.c +4
-0@@ -483,6 +483,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_enstruct i915_scheduler *scheduler = dev_priv->scheduler;int ret; +intel_ring_reserved_space_cancel (qe->params.request->ringbuf); + scheduler->flags[qe->params.ring->id] |= i915_sf_submitting; ret = dev_priv->gt.execbuf_final (&qe->params); scheduler->flags[qe->params.ring->id] &= ~i915_sf_submitting;@@ -539,6 +541,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe) node->stamp = jiffies;i915_gem_request_reference (node->params.request); +intel_ring_reserved_space_cancel (node->params.request->ringbuf); +WARN_ON (node->params.request->scheduler_qe); node->params.request->scheduler_qe = node;drivers/gpu/drm/i915/intel_lrc.c +6
-2@@ -978,13 +978,17 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)/* The mutex must be acquired before calling this function */ WARN_ON (!mutex_is_locked (¶ms->dev->struct_mutex)); + ret =intel_logical_ring_reserve_space (req); +if (ret) + goto err; +/* * Unconditionally invalidate gpu caches and ensure that we do flush * any residual writes from the previous batch. */ ret =logical_ring_invalidate_all_caches (req);if (ret) -return ret; + goto err;if (ring == &dev_priv->ring[RCS ] && params->instp_mode != dev_priv->relative_constants_mode) {@@ -1006,13 +1010,18 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params) ret = ring->emit_bb_start (req, exec_start, params->dispatch_flags);if (ret) -return ret; + goto err;trace_i915_gem_ring_dispatch (req, params->dispatch_flags);i915_gem_execbuffer_retire_commands (params);
drivers/gpu/drm/i915/intel_lrc.c int intel_execlists_submission_final(struct i915_execbuffer_params *params) · line 1027
Quoted 10 lines
drivers/gpu/drm/i915/intel_lrc.c +5
-0return 0 ; + +err: +intel_ring_reserved_space_cancel (params->request->ringbuf); + +return ret; }void intel_execlists_retire_requests (struct intel_engine_cs *ring)
I had to double check that it was ok to cancel space if the reserve failed (I guess it is), so seems ok.
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 15/35 drm/i915: Added tracking/locking of batch buffer objects
In reply to [email protected]From: John Harrison <[email protected]>
The scheduler needs to track interdependencies between batch buffers. These are calculated by analysing the object lists of the buffers and looking for commonality. The scheduler also needs to keep those buffers locked long after the initial IOCTL call has returned to user land.
v3: Updated to support read-read optimisation.
v5: Updated due to changes to earlier patches in series for splitting bypass mode into a separate function and consoliding the clean up code.
For: VIZ-1587
--- drivers/gpu/drm/i915/i915_gem_execbuffer.c | 48 ++++++++++++++++++++++++++++-- drivers/gpu/drm/i915/i915_scheduler.c | 15 ++++++++++ 2 files changed, 61 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +46 -2
@@ -1428,7 +1428,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,struct i915_execbuffer_params *params = &qe.params;const u32 ctx_id =i915_execbuffer2_get_context_id (*args);u32 dispatch_flags; -int ret; +int ret, i;bool need_relocs;if (!i915_gem_check_execbuffer (args))
@@ -1543,6 +1543,14 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data, goto pre_mutex_err; } + qe.saved_objects =kzalloc ( +sizeof (*qe.saved_objects) * args->buffer_count, +GFP_KERNEL ); +if (!qe.saved_objects) { + ret = -ENOMEM ; + goto err; + } +/* Look up object handles */ ret =eb_lookup_vmas (eb, exec, args, vm, file);if (ret)
@@ -1663,7 +1671,30 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data, params->args_DR1 = args->DR1; params->args_DR4 = args->DR4; params->batch_obj = batch_obj; - params->ctx = ctx; + +/* +* Save away the list of objects used by this batch buffer for the +* purpose of tracking inter-buffer dependencies. +*/ +for (i =0 ; i < args->buffer_count; i++) { +struct drm_i915_gem_object *obj; + +/* +* NB: 'drm_gem_object_lookup()' increments the object's +* reference count and so must be matched by a +* 'drm_gem_object_unreference' call. +*/ + obj =to_intel_bo (drm_gem_object_lookup (dev, file, + exec[i].handle)); + qe.saved_objects[i].obj = obj; + qe.saved_objects[i].read_only = obj->base.pending_write_domain ==0 ; + + } + qe.num_objs = i; + +/* Lock and save the context object as well. */ +i915_gem_context_reference (ctx); + params->ctx = ctx; ret = dev_priv->gt.execbuf_submit (params, args, &eb->vmas);if (ret)
@@ -1696,6 +1727,19 @@ err:i915_gem_context_unreference (ctx);eb_destroy (eb); +/* Need to release the objects: */ +if (qe.saved_objects) { +for (i =0 ; i < qe.num_objs; i++) +drm_gem_object_unreference ( + &qe.saved_objects[i].obj->base); + +kfree (qe.saved_objects); + } + +/* Context too */ +if (params->ctx) +i915_gem_context_unreference (params->ctx); +/* * If the request was created but not successfully submitted then it * must be freed again. If it was submitted then it is being tracked
drivers/gpu/drm/i915/i915_scheduler.c +15 -0
@@ -719,6 +719,8 @@ void i915_scheduler_wakeup(struct drm_device *dev)*/ void i915_scheduler_clean_node (struct i915_scheduler_queue_entry *node) { +int i; +if (!I915_SQS_IS_COMPLETE (node)) {WARN (!node->params.request->cancelled,"Cleaning active node: %d!\n" , node->status);
@@ -736,6 +738,19 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node) node->params.batch_obj =NULL ; } +/* Release the locked buffers: */ +for (i =0 ; i < node->num_objs; i++) +drm_gem_object_unreference (&node->saved_objects[i].obj->base); +kfree (node->saved_objects); + node->saved_objects =NULL ; + node->num_objs =0 ; + +/* Context too: */ +if (node->params.ctx) { +i915_gem_context_unreference (node->params.ctx); + node->params.ctx =NULL ; + } +/* And anything else owned by the node: */ if (node->params.cliprects) {kfree (node->params.cliprects);
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 15/35] drm/i915: Added tracking/locking of batch buffer objects
Quoted 124 lines · On 02/18/2016 06:27 AM, [email protected] wrote:
From: John Harrison <[email protected]>
The scheduler needs to track interdependencies between batch buffers. These are calculated by analysing the object lists of the buffers and looking for commonality. The scheduler also needs to keep those buffers locked long after the initial IOCTL call has returned to user land.
v3: Updated to support read-read optimisation.
v5: Updated due to changes to earlier patches in series for splitting bypass mode into a separate function and consoliding the clean up code.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>--- drivers/gpu/drm/i915/i915_gem_execbuffer.c | 48 ++++++++++++++++++++++++++++-- drivers/gpu/drm/i915/i915_scheduler.c | 15 ++++++++++ 2 files changed, 61 insertions(+), 2 deletions(-)drivers/gpu/drm/i915/i915_gem_execbuffer.c +46
-2@@ -1428,7 +1428,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,struct i915_execbuffer_params *params = &qe.params;const u32 ctx_id =i915_execbuffer2_get_context_id (*args);u32 dispatch_flags; -int ret; +int ret, i;bool need_relocs;if (!i915_gem_check_execbuffer (args))@@ -1543,6 +1543,14 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data, goto pre_mutex_err; } + qe.saved_objects =kzalloc ( +sizeof (*qe.saved_objects) * args->buffer_count, +GFP_KERNEL ); +if (!qe.saved_objects) { + ret = -ENOMEM ; + goto err; + } +/* Look up object handles */ ret =eb_lookup_vmas (eb, exec, args, vm, file);if (ret)@@ -1663,7 +1671,30 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data, params->args_DR1 = args->DR1; params->args_DR4 = args->DR4; params->batch_obj = batch_obj; - params->ctx = ctx; + +/* +* Save away the list of objects used by this batch buffer for the +* purpose of tracking inter-buffer dependencies. +*/ +for (i =0 ; i < args->buffer_count; i++) { +struct drm_i915_gem_object *obj; + +/* +* NB: 'drm_gem_object_lookup()' increments the object's +* reference count and so must be matched by a +* 'drm_gem_object_unreference' call. +*/ + obj =to_intel_bo (drm_gem_object_lookup (dev, file, + exec[i].handle)); + qe.saved_objects[i].obj = obj; + qe.saved_objects[i].read_only = obj->base.pending_write_domain ==0 ; + + } + qe.num_objs = i; + +/* Lock and save the context object as well. */ +i915_gem_context_reference (ctx); + params->ctx = ctx; ret = dev_priv->gt.execbuf_submit (params, args, &eb->vmas);if (ret)@@ -1696,6 +1727,19 @@ err:i915_gem_context_unreference (ctx);eb_destroy (eb); +/* Need to release the objects: */ +if (qe.saved_objects) { +for (i =0 ; i < qe.num_objs; i++) +drm_gem_object_unreference ( + &qe.saved_objects[i].obj->base); + +kfree (qe.saved_objects); + } + +/* Context too */ +if (params->ctx) +i915_gem_context_unreference (params->ctx); +/* * If the request was created but not successfully submitted then it * must be freed again. If it was submitted then it is being tracked drivers/gpu/drm/i915/i915_scheduler.c +8
-0@@ -719,6 +719,8 @@ void i915_scheduler_wakeup(struct drm_device *dev)*/ void i915_scheduler_clean_node (struct i915_scheduler_queue_entry *node) { +int i; +if (!I915_SQS_IS_COMPLETE (node)) {WARN (!node->params.request->cancelled,"Cleaning active node: %d!\n" , node->status);@@ -736,6 +738,19 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node) node->params.batch_obj =NULL ; } +/* Release the locked buffers: */ +for (i =0 ; i < node->num_objs; i++) +drm_gem_object_unreference (&node->saved_objects[i].obj->base); +kfree (node->saved_objects); + node->saved_objects =NULL ; + node->num_objs =0 ;
drivers/gpu/drm/i915/i915_scheduler.c void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node) · line 756
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c +7
-0+ +/* Context too: */ +if (node->params.ctx) { +i915_gem_context_unreference (node->params.ctx); + node->params.ctx =NULL ; + } +/* And anything else owned by the node: */ if (node->params.cliprects) {kfree (node->params.cliprects);
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 16/35 drm/i915: Hook scheduler node clean up into retire requests
In reply to [email protected]From: John Harrison <[email protected]>
The scheduler keeps its own lock on various DRM objects in order to guarantee safe access long after the original execbuff IOCTL has completed. This is especially important when pre-emption is enabled as the batch buffer might need to be submitted to the hardware multiple times. This patch hooks the clean up of these locks into the request retire function. The request can only be retired after it has completed on the hardware and thus is no longer eligible for re-submission. Thus there is no point holding on to the locks beyond that time.
v3: Updated to not WARN when cleaning a node that is being cancelled. The clean will happen later so skipping it at the point of cancellation is fine.
v5: Squashed the i915_scheduler.c portions down into the 'start of scheduler' patch. [Joonas Lahtinen]
For: VIZ-1587
--- drivers/gpu/drm/i915/i915_gem.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/gpu/drm/i915/i915_gem.c +3 -0
@@ -1489,6 +1489,9 @@ static void i915_gem_request_retire(struct drm_i915_gem_request *request)fence_signal_locked (&request->fence); } +if (request->scheduler_qe) +i915_scheduler_clean_node (request->scheduler_qe); +i915_gem_request_unreference (request); }
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 16/35] drm/i915: Hook scheduler node clean up into retire requests
Quoted 4 lines · On 02/18/2016 06:27 AM, [email protected] wrote:
From: John Harrison <[email protected]>
The scheduler keeps its own lock on various DRM objects in order to guarantee safe access long after the original execbuff IOCTL has
execbuf is getting bigger, but I'm not sure if it qualifies as "buff" yet. Intentional misspelling? :)
Quoted 28 lines
completed. This is especially important when pre-emption is enabled as the batch buffer might need to be submitted to the hardware multiple times. This patch hooks the clean up of these locks into the request retire function. The request can only be retired after it has completed on the hardware and thus is no longer eligible for re-submission. Thus there is no point holding on to the locks beyond that time.
v3: Updated to not WARN when cleaning a node that is being cancelled. The clean will happen later so skipping it at the point of cancellation is fine.
v5: Squashed the
i915_scheduler.cportions down into the 'start of scheduler' patch. [Joonas Lahtinen]For: VIZ-1587
Signed-off-by John Harrison <[email protected]>Cc Joonas Lahtinen <[email protected]>--- drivers/gpu/drm/i915/i915_gem.c | 3 +++ 1 file changed, 3 insertions(+)drivers/gpu/drm/i915/i915_gem.c +0
-0
drivers/gpu/drm/i915/i915_gem.c static void i915_gem_request_retire(struct drm_i915_gem_request *request) · line 1497
Quoted 9 lines
drivers/gpu/drm/i915/i915_gem.c +3
-0@@ -1489,6 +1489,9 @@ static void i915_gem_request_retire(struct drm_i915_gem_request *request)fence_signal_locked (&request->fence); } +if (request->scheduler_qe) +i915_scheduler_clean_node (request->scheduler_qe); +i915_gem_request_unreference (request); }
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 17/35 drm/i915: Added scheduler support to __wait_request() calls
In reply to [email protected]From: John Harrison <[email protected]>
The scheduler can cause batch buffers, and hence requests, to be submitted to the ring out of order and asynchronously to their submission to the driver. Thus at the point of waiting for the completion of a given request, it is not even guaranteed that the request has actually been sent to the hardware yet. Even it is has been sent, it is possible that it could be pre-empted and thus 'unsent'.
This means that it is necessary to be able to submit requests to the hardware during the wait call itself. Unfortunately, while some callers of __wait_request() release the mutex lock first, others do not (and apparently can not). Hence there is the ability to deadlock as the wait stalls for submission but the asynchronous submission is stalled for the mutex lock.
This change hooks the scheduler in to the __wait_request() code to ensure correct behaviour. That is, flush the target batch buffer through to the hardware and do not deadlock waiting for something that cannot currently be submitted. Instead, the wait call must return EAGAIN at least as far back as necessary to release the mutex lock and allow the scheduler's asynchronous processing to get in and handle the pre-emption operation and eventually (re-)submit the work.
v3: Removed the explicit scheduler flush from i915_wait_request(). This is no longer necessary and was causing unintended changes to the scheduler priority level which broke a validation team test.
v4: Corrected the format of some comments to keep the style checker happy.
v5: Added function description. [Joonas Lahtinen]
For: VIZ-1587
--- drivers/gpu/drm/i915/i915_drv.h | 3 ++- drivers/gpu/drm/i915/i915_gem.c | 37 ++++++++++++++++++++++++++------- drivers/gpu/drm/i915/i915_scheduler.c | 31 +++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_scheduler.h | 2 ++ drivers/gpu/drm/i915/intel_display.c | 5 +++-- drivers/gpu/drm/i915/intel_ringbuffer.c | 2 +- 6 files changed, 69 insertions(+), 11 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +2 -1
@@ -3071,7 +3071,8 @@ int __i915_wait_request(struct drm_i915_gem_request *req,unsigned reset_counter, bool interruptible, s64 *timeout, - struct intel_rps_client *rps); + struct intel_rps_client *rps, + bool is_locked);int __must_check i915_wait_request (struct drm_i915_gem_request *req);int i915_gem_fault (struct vm_area_struct *vma,struct vm_fault *vmf);int __must_check
drivers/gpu/drm/i915/i915_gem.c +30 -7
@@ -1258,7 +1258,8 @@ int __i915_wait_request(struct drm_i915_gem_request *req,unsigned reset_counter, bool interruptible, s64 *timeout, - struct intel_rps_client *rps) + struct intel_rps_client *rps, + bool is_locked) { struct intel_engine_cs *ring = i915_gem_request_get_ring(req);struct drm_device *dev = ring->dev;
@@ -1268,8 +1269,10 @@ int __i915_wait_request(struct drm_i915_gem_request *req,DEFINE_WAIT (wait);unsigned long timeout_expire;s64 before =0 ;/* Only to silence a compiler warning. */ -int ret; +int ret =0 ; +bool busy; +might_sleep ();WARN (!intel_irqs_enabled (dev_priv),"IRQs disabled" );if (i915_gem_request_completed (req))
@@ -1324,6 +1327,26 @@ int __i915_wait_request(struct drm_i915_gem_request *req,break ; } +if (is_locked) { +/* +* If this request is being processed by the scheduler +* then it is unsafe to sleep with the mutex lock held +* as the scheduler may require the lock in order to +* progress the request. +*/ +if (i915_scheduler_is_request_tracked (req,NULL , &busy)) { +if (busy) { + ret = -EAGAIN ; +break ; + } + } + +/* +* If the request is not tracked by the scheduler +* then the regular test can be done. +*/ + } +if (i915_gem_request_completed (req)) { ret =0 ;break ;
@@ -1542,7 +1565,7 @@ i915_wait_request(struct drm_i915_gem_request *req) ret =__i915_wait_request (req,atomic_read (&dev_priv->gpu_error.reset_counter), - interruptible,NULL ,NULL ); + interruptible,NULL ,NULL , true);if (ret)return ret;
@@ -1655,7 +1678,7 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,mutex_unlock (&dev->struct_mutex);for (i =0 ; ret ==0 && i < n; i++) ret =__i915_wait_request (requests[i], reset_counter, true, -NULL , rps); +NULL , rps, false);mutex_lock (&dev->struct_mutex);for (i =0 ; i < n; i++) {
@@ -3511,7 +3534,7 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)if (ret ==0 ) ret =__i915_wait_request (req[i], reset_counter, true, args->timeout_ns >0 ? &args->timeout_ns :NULL , -to_rps_client (file)); +to_rps_client (file), false);i915_gem_request_unreference (req[i]); }return ret;
@@ -3544,7 +3567,7 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,atomic_read (&i915->gpu_error.reset_counter), i915->mm.interruptible,NULL , - &i915->rps.semaphores); + &i915->rps.semaphores, true);if (ret)return ret;
@@ -4523,7 +4546,7 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)if (target ==NULL )return 0 ; - ret =__i915_wait_request (target, reset_counter, true,NULL ,NULL ); + ret =__i915_wait_request (target, reset_counter, true,NULL ,NULL , false);if (ret ==0 )queue_delayed_work (dev_priv->wq, &dev_priv->mm.retire_work,0 );
drivers/gpu/drm/i915/i915_scheduler.c +31 -0
@@ -918,6 +918,37 @@ void i915_scheduler_work_handler(struct work_struct *work) }/** +* i915_scheduler_is_request_tracked - return info to say what the scheduler's +* connection to this request is (if any). +* @req: request to be queried +* @compeleted: if non-null, set to completion status +* @busy: if non-null set to busy status +* +* Looks up the given request in the scheduler's internal queue and reports +* on whether the request has completed or is still pending. +* Returns true if the request was found or false if it was not. +*/ +bool i915_scheduler_is_request_tracked (struct drm_i915_gem_request *req, +bool *completed,bool *busy) +{ +struct drm_i915_private *dev_priv = req->ring->dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + +if (!scheduler) +return false; + +if (req->scheduler_qe ==NULL ) +return false; + +if (completed) + *completed =I915_SQS_IS_COMPLETE (req->scheduler_qe); +if (busy) + *busy =I915_SQS_IS_QUEUED (req->scheduler_qe); + +return true; +} + +/** * i915_scheduler_closefile - notify the scheduler that a DRM file handle * has been closed. * @dev: DRM device
drivers/gpu/drm/i915/i915_scheduler.h +2 -0
@@ -94,5 +94,7 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);bool i915_scheduler_notify_request (struct drm_i915_gem_request *req);void i915_scheduler_wakeup (struct drm_device *dev);void i915_scheduler_work_handler (struct work_struct *work); +bool i915_scheduler_is_request_tracked (struct drm_i915_gem_request *req, +bool *completed,bool *busy);#endif /* _I915_SCHEDULER_H_ */
drivers/gpu/drm/i915/intel_display.c +3 -2
@@ -11458,7 +11458,8 @@ static void intel_mmio_flip_work_func(struct work_struct *work)WARN_ON (__i915_wait_request (mmio_flip->req, mmio_flip->crtc->reset_counter, false,NULL , - &mmio_flip->i915->rps.mmioflips)); + &mmio_flip->i915->rps.mmioflips, + false));i915_gem_request_unreference (mmio_flip->req); }
@@ -13523,7 +13524,7 @@ static int intel_atomic_prepare_commit(struct drm_device *dev, ret =__i915_wait_request (intel_plane_state->wait_req, reset_counter, true, -NULL ,NULL ); +NULL ,NULL , false);/* Swallow -EIO errors to allow updates during hw lockup. */ if (ret == -EIO )
drivers/gpu/drm/i915/intel_ringbuffer.c +1 -1
@@ -2304,7 +2304,7 @@ int intel_ring_idle(struct intel_engine_cs *ring)return __i915_wait_request (req,atomic_read (&to_i915 (ring->dev)->gpu_error.reset_counter),to_i915 (ring->dev)->mm.interruptible, -NULL ,NULL ); +NULL ,NULL , true); }int intel_ring_alloc_request_extras (struct drm_i915_gem_request *request)
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Joonas Lahtinen
maintainerRe: [PATCH v5 17/35] drm/i915: Added scheduler support to __wait_request() calls
Quoted 51 lines · On to, 2016-02-18 at 14:27 +0000, [email protected] wrote:
From: John Harrison <[email protected]>
The scheduler can cause batch buffers, and hence requests, to be submitted to the ring out of order and asynchronously to their submission to the driver. Thus at the point of waiting for the completion of a given request, it is not even guaranteed that the request has actually been sent to the hardware yet. Even it is has been sent, it is possible that it could be pre-empted and thus 'unsent'.
This means that it is necessary to be able to submit requests to the hardware during the wait call itself. Unfortunately, while some callers of
__wait_request()release the mutex lock first, others do not (and apparently can not). Hence there is the ability to deadlock as the wait stalls for submission but the asynchronous submission is stalled for the mutex lock.This change hooks the scheduler in to the
__wait_request()code to ensure correct behaviour. That is, flush the target batch buffer through to the hardware and do not deadlock waiting for something that cannot currently be submitted. Instead, the wait call must return EAGAIN at least as far back as necessary to release the mutex lock and allow the scheduler's asynchronous processing to get in and handle the pre-emption operation and eventually (re-)submit the work.v3: Removed the explicit scheduler flush from
i915_wait_request(). This is no longer necessary and was causing unintended changes to the scheduler priority level which broke a validation team test.v4: Corrected the format of some comments to keep the style checker happy.
v5: Added function description. [Joonas Lahtinen]
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>Cc Joonas Lahtinen <[email protected]>---
drivers/gpu/drm/i915/i915_drv.h | 3 ++- drivers/gpu/drm/i915/i915_gem.c | 37 ++++++++++++++++++++++++++------- drivers/gpu/drm/i915/i915_scheduler.c | 31 +++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_scheduler.h | 2 ++ drivers/gpu/drm/i915/intel_display.c | 5 +++--drivers/gpu/drm/i915/intel_ringbuffer.c| 2 +-6 files changed, 69 insertions(+), 11 deletions(-)drivers/gpu/drm/i915/i915_drv.h +0
-0
drivers/gpu/drm/i915/i915_drv.h int __i915_wait_request(struct drm_i915_gem_request *req, · line 3075
Quoted 6 lines
drivers/gpu/drm/i915/i915_drv.h +2
-1@@ -3071,7 +3071,8 @@ int __i915_wait_request(struct drm_i915_gem_request *req, unsigned reset_counter, bool interruptible, s64 *timeout, - struct intel_rps_client *rps); + struct intel_rps_client *rps, + bool is_locked);
Multiple bool parameters are better converted to int flags, it makes the callsite code more readable if you have enum I915_WAIT_REQ_* flags.
Quoted 21 lines
drivers/gpu/drm/i915/i915_drv.h +0
-0int __must_check i915_wait_request(struct drm_i915_gem_request *req); int i915_gem_fault(struct vm_area_struct *vma, struct vm_fault *vmf);int
__must_checkdrivers/gpu/drm/i915/i915_gem.c +2
-1@@ -1258,7 +1258,8 @@ int __i915_wait_request(struct drm_i915_gem_request *req, unsigned reset_counter, bool interruptible, s64 *timeout, - struct intel_rps_client *rps) + struct intel_rps_client *rps, + bool is_locked) { struct intel_engine_cs *ring = i915_gem_request_get_ring(req); struct drm_device *dev = ring->dev;@@ -1268,8 +1269,10 @@ int __i915_wait_request(struct drm_i915_gem_request *req, DEFINE_WAIT(wait); unsigned long timeout_expire;s64 before = 0; /* Only to silence a compiler warning. */
drivers/gpu/drm/i915/i915_gem.c int __i915_wait_request(struct drm_i915_gem_request *req, · line 1273
Quoted 3 lines
drivers/gpu/drm/i915/i915_gem.c +2
-1-int ret; +int ret =0 ; + bool busy;
Strange whitespace.
drivers/gpu/drm/i915/i915_gem.c int __i915_wait_request(struct drm_i915_gem_request *req, · line 1275
Quoted 2 lines
drivers/gpu/drm/i915/i915_gem.c +1
-0+might_sleep ();
This will splat if is_locked is true? So maybe it should be protected by if (!is_locked)?
Quoted 7 lines
WARN(!intel_irqs_enabled(dev_priv), "IRQs disabled");
if (i915_gem_request_completed(req))drivers/gpu/drm/i915/i915_gem.c +0
-0@@ -1324,6 +1327,26 @@ int __i915_wait_request(struct drm_i915_gem_request *req, break; }
drivers/gpu/drm/i915/i915_gem.c int __i915_wait_request(struct drm_i915_gem_request *req, · line 1336
Quoted 7 lines
drivers/gpu/drm/i915/i915_gem.c +7
-0+if (is_locked) { +/* +* If this request is being processed by the scheduler +* then it is unsafe to sleep with the mutex lock held +* as the scheduler may require the lock in order to +* progress the request. +*/
This is becoming a monstrous function, I think it could be split down to two functions, __i915_wait_request and __i915_wait_request_locked/nonblocking, because if you take all the timeout/sleeping related stuff away, not much is left.
drivers/gpu/drm/i915/i915_gem.c int __i915_wait_request(struct drm_i915_gem_request *req, · line 1342
Quoted 6 lines
drivers/gpu/drm/i915/i915_gem.c +6
-0+if (i915_scheduler_is_request_tracked (req,NULL , &busy)) { +if (busy) { + ret = -EAGAIN ; +break ; + } + }
If this is kept in single function, would not it be enough to execute this before the loop?
Quoted 75 lines
drivers/gpu/drm/i915/i915_gem.c +7
-0+ +/* +* If the request is not tracked by the scheduler +* then the regular test can be done. +*/ + } +if (
i915_gem_request_completed(req)) { ret = 0;
break;drivers/gpu/drm/i915/i915_gem.c +0
-0@@ -1542,7 +1565,7 @@ i915_wait_request(struct drm_i915_gem_request *req)
ret = __i915_wait_request(req,atomic_read(&dev_priv->gpu_error.reset_counter),drivers/gpu/drm/i915/i915_gem.c +1
-1- interruptible, NULL, NULL); + interruptible, NULL, NULL, true);if (ret)
return ret;drivers/gpu/drm/i915/i915_gem.c +1
-1@@ -1655,7 +1678,7 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj, mutex_unlock(&dev->struct_mutex); for (i = 0; ret == 0 && i < n; i++) ret = __i915_wait_request(requests[i], reset_counter, true, - NULL, rps); + NULL, rps, false); mutex_lock(&dev->struct_mutex);
for (i = 0; i < n; i++) {drivers/gpu/drm/i915/i915_gem.c +2
-2@@ -3511,7 +3534,7 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file) if (ret == 0) ret = __i915_wait_request(req[i], reset_counter, true, args->timeout_ns > 0 ? &args->timeout_ns : NULL, - to_rps_client(file)); + to_rps_client(file), false); i915_gem_request_unreference(req[i]); } return ret;@@ -3544,7 +3567,7 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj, atomic_read(&i915->gpu_error.reset_counter), i915->mm.interruptible, NULL, - &i915->rps.semaphores); + &i915->rps.semaphores, true); if (ret) return ret;drivers/gpu/drm/i915/i915_gem.c +0
-0@@ -4523,7 +4546,7 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file) if (target == NULL) return 0;drivers/gpu/drm/i915/i915_gem.c +1
-1- ret =__i915_wait_request (target, reset_counter, true,NULL ,NULL ); + ret =__i915_wait_request (target, reset_counter, true,NULL ,NULL , false);if (ret == 0)
queue_delayed_work(dev_priv->wq, &dev_priv->mm.retire_work, 0);drivers/gpu/drm/i915/i915_scheduler.c +0
-0@@ -918,6 +918,37 @@ void i915_scheduler_work_handler(struct work_struct *work) }
/**
+ *i915_scheduler_is_request_tracked- return info to say what the scheduler's
+ * connection to this request is (if any).
+ * @req: request to be queried
+ * @compeleted: if non-null, set to completion status
+ * @busy: if non-null set to busy status
+ *
+ * Looks up the given request in the scheduler's internal queue and reports
+ * on whether the request has completed or is still pending.
+ * Returns true if the request was found or false if it was not.
+ */drivers/gpu/drm/i915/i915_scheduler.c +2
-0+bool i915_scheduler_is_request_tracked (struct drm_i915_gem_request *req, + bool *completed, bool *busy)
drivers/gpu/drm/i915/i915_scheduler.c void i915_scheduler_work_handler(struct work_struct *work) · line 942
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c +10
-0+{ +struct drm_i915_private *dev_priv = req->ring->dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + +if (!scheduler) +return false; + +if (req->scheduler_qe ==NULL ) +return false; +
These better be their own functions, making the code more readable.
_is_request_completed()_is_request_busy()
Quoted 60 lines
drivers/gpu/drm/i915/i915_scheduler.c +9
-0+if (completed) + *completed =I915_SQS_IS_COMPLETE (req->scheduler_qe); +if (busy) + *busy = I915_SQS_IS_QUEUED(req->scheduler_qe); + +return true; +} + +/** *
i915_scheduler_closefile- notify the scheduler that a DRM file handle
* has been closed.
* @dev: DRM devicedrivers/gpu/drm/i915/i915_scheduler.h +2
-0@@ -94,5 +94,7 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe); bool i915_scheduler_notify_request(struct drm_i915_gem_request *req); void i915_scheduler_wakeup(struct drm_device *dev); void i915_scheduler_work_handler(struct work_struct *work); +bool i915_scheduler_is_request_tracked (struct drm_i915_gem_request *req, + bool *completed, bool *busy);
#endif /* _I915_SCHEDULER_H_ */drivers/gpu/drm/i915/intel_display.c +2
-1@@ -11458,7 +11458,8 @@ static void intel_mmio_flip_work_func(struct work_struct *work) WARN_ON(__i915_wait_request(mmio_flip->req, mmio_flip->crtc->reset_counter, false, NULL, - &mmio_flip->i915->rps.mmioflips)); + &mmio_flip->i915->rps.mmioflips, + false)); i915_gem_request_unreference(mmio_flip->req); }drivers/gpu/drm/i915/intel_display.c +0
-0@@ -13523,7 +13524,7 @@ static int intel_atomic_prepare_commit(struct drm_device *dev,
ret = __i915_wait_request(intel_plane_state->wait_req,reset_counter, true,drivers/gpu/drm/i915/intel_display.c +1
-1- NULL, NULL); + NULL, NULL, false);
/* Swallow-EIOerrors to allow updates during hw lockup. */ if (ret ==-EIO)drivers/gpu/drm/i915/intel_ringbuffer.c +1
-1@@ -2304,7 +2304,7 @@ int intel_ring_idle(struct intel_engine_cs *ring) return __i915_wait_request(req, atomic_read(&to_i915(ring->dev)->gpu_error.reset_counter), to_i915(ring->dev)->mm.interruptible, - NULL, NULL); + NULL, NULL, true); }
intintel_ring_alloc_request_extras(struct drm_i915_gem_request *request)
Signature · 7 lines
Joonas Lahtinen Open Source Technology Center Intel Corporation _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 17/35] drm/i915: Added scheduler support to __wait_request() calls
Quoted 88 lines · Joonas Lahtinen
Quoted 51 lines · On to, 2016-02-18 at 14:27 +0000, [email protected] wrote:
From: John Harrison <[email protected]>
The scheduler can cause batch buffers, and hence requests, to be submitted to the ring out of order and asynchronously to their submission to the driver. Thus at the point of waiting for the completion of a given request, it is not even guaranteed that the request has actually been sent to the hardware yet. Even it is has been sent, it is possible that it could be pre-empted and thus 'unsent'.
This means that it is necessary to be able to submit requests to the hardware during the wait call itself. Unfortunately, while some callers of
__wait_request()release the mutex lock first, others do not (and apparently can not). Hence there is the ability to deadlock as the wait stalls for submission but the asynchronous submission is stalled for the mutex lock.This change hooks the scheduler in to the
__wait_request()code to ensure correct behaviour. That is, flush the target batch buffer through to the hardware and do not deadlock waiting for something that cannot currently be submitted. Instead, the wait call must return EAGAIN at least as far back as necessary to release the mutex lock and allow the scheduler's asynchronous processing to get in and handle the pre-emption operation and eventually (re-)submit the work.v3: Removed the explicit scheduler flush from
i915_wait_request(). This is no longer necessary and was causing unintended changes to the scheduler priority level which broke a validation team test.v4: Corrected the format of some comments to keep the style checker happy.
v5: Added function description. [Joonas Lahtinen]
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>Cc Joonas Lahtinen <[email protected]>--- drivers/gpu/drm/i915/i915_drv.h | 3 ++- drivers/gpu/drm/i915/i915_gem.c | 37 ++++++++++++++++++++++++++------- drivers/gpu/drm/i915/i915_scheduler.c | 31 +++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_scheduler.h | 2 ++ drivers/gpu/drm/i915/intel_display.c | 5 +++-- drivers/gpu/drm/i915/intel_ringbuffer.c | 2 +- 6 files changed, 69 insertions(+), 11 deletions(-)drivers/gpu/drm/i915/i915_drv.h +0
-0drivers/gpu/drm/i915/i915_drv.h int __i915_wait_request(struct drm_i915_gem_request *req, · line 3075
Quoted 6 lines
drivers/gpu/drm/i915/i915_drv.h +2
-1@@ -3071,7 +3071,8 @@ int __i915_wait_request(struct drm_i915_gem_request *req,unsigned reset_counter, bool interruptible, s64 *timeout, - struct intel_rps_client *rps); + struct intel_rps_client *rps, + bool is_locked);Multiple bool parameters are better converted to int flags, it makes the callsite code more readable if you have enum I915_WAIT_REQ_* flags.
Quoted 18 lines
drivers/gpu/drm/i915/i915_drv.h +0
-0int __must_check i915_wait_request (struct drm_i915_gem_request *req);int i915_gem_fault (struct vm_area_struct *vma,struct vm_fault *vmf);int __must_check drivers/gpu/drm/i915/i915_gem.c +2
-1@@ -1258,7 +1258,8 @@ int __i915_wait_request(struct drm_i915_gem_request *req,unsigned reset_counter, bool interruptible, s64 *timeout, - struct intel_rps_client *rps) + struct intel_rps_client *rps, + bool is_locked) { struct intel_engine_cs *ring = i915_gem_request_get_ring(req);struct drm_device *dev = ring->dev;drivers/gpu/drm/i915/i915_gem.c int __i915_wait_request(struct drm_i915_gem_request *req, · line 1273
Quoted 6 lines
drivers/gpu/drm/i915/i915_gem.c +2
-1@@ -1268,8 +1269,10 @@ int __i915_wait_request(struct drm_i915_gem_request *req,DEFINE_WAIT (wait);unsigned long timeout_expire;s64 before =0 ;/* Only to silence a compiler warning. */ -int ret; +int ret =0 ; +bool busy;Strange whitespace.
Quoted 1 line
Quoted 3 lines
drivers/gpu/drm/i915/i915_gem.c int __i915_wait_request(struct drm_i915_gem_request *req, · line 1275
Quoted 1 line
drivers/gpu/drm/i915/i915_gem.c +1
-0+might_sleep ();This will splat if
is_lockedis true? So maybe it should be protected by if (!is_locked)?
The locking here is just the driver mutex lock. Whereas the test will only go bang if in atomic context (spinlock, IRQ, etc.). So it should be safe. If it isn't then the driver is already broken because the whole point of _wait_request() is that it will sleep! Although one could argue it is already broken because it quite likes sleeping with the mutex lock held.
Quoted 7 lines
Quoted 7 lines
WARN(!intel_irqs_enabled(dev_priv), "IRQs disabled");if (
i915_gem_request_completed(req))drivers/gpu/drm/i915/i915_gem.c +0
-0@@ -1324,6 +1327,26 @@ int __i915_wait_request(struct drm_i915_gem_request *req,break ; }
Quoted 11 lines
drivers/gpu/drm/i915/i915_gem.c int __i915_wait_request(struct drm_i915_gem_request *req, · line 1336
Quoted 7 lines
drivers/gpu/drm/i915/i915_gem.c +7
-0+if (is_locked) { +/* +* If this request is being processed by the scheduler +* then it is unsafe to sleep with the mutex lock held +* as the scheduler may require the lock in order to +* progress the request. +*/ This is becoming a monstrous function, I think it could be split down to two functions,
__i915_wait_requestand __i915_wait_request_locked/nonblocking, because if you take all the timeout/sleeping related stuff away, not much is left.
I don't see what you mean. You can't take away all the timeout/sleeping related stuff, that is the whole point of the function! Everything that is inside the loop needs to be inside the loop. It is a hideously complicated wait function but as far as I can tell, it needs to be such. I recall at some point someone did a refactor to move the contents of the loop into a separate function but that doesn't seem to have made it upstream. Or maybe it did and it has since been reverted?
The scheduler addition is now much simpler due to the re-implementing of the _is_tracked() function - see comments below.
Quoted 8 lines
drivers/gpu/drm/i915/i915_gem.c int __i915_wait_request(struct drm_i915_gem_request *req, · line 1342
Quoted 6 lines
drivers/gpu/drm/i915/i915_gem.c +6
-0+if (i915_scheduler_is_request_tracked (req,NULL , &busy)) { +if (busy) { + ret = -EAGAIN ; +break ; + } + }If this is kept in single function, would not it be enough to execute this before the loop?
It might have changed. The sleep might have been aborted due to something that means the scheduler is now busy and sleeping with the mutex lock held will cause a deadlock.
Quoted 87 lines
Quoted 75 lines
drivers/gpu/drm/i915/i915_gem.c +12
-5+ +/* +* If the request is not tracked by the scheduler +* then the regular test can be done. +*/ + } +if (i915_gem_request_completed (req)) { ret =0 ;break ;@@ -1542,7 +1565,7 @@ i915_wait_request(struct drm_i915_gem_request *req) ret =__i915_wait_request (req,atomic_read (&dev_priv->gpu_error.reset_counter), - interruptible,NULL ,NULL ); + interruptible,NULL ,NULL , true);if (ret)return ret;@@ -1655,7 +1678,7 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,mutex_unlock (&dev->struct_mutex);for (i =0 ; ret ==0 && i < n; i++) ret =__i915_wait_request (requests[i], reset_counter, true, -NULL , rps); +NULL , rps, false);mutex_lock (&dev->struct_mutex);for (i =0 ; i < n; i++) {@@ -3511,7 +3534,7 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)if (ret ==0 ) ret =__i915_wait_request (req[i], reset_counter, true, args->timeout_ns >0 ? &args->timeout_ns :NULL , -to_rps_client (file)); +to_rps_client (file), false);i915_gem_request_unreference (req[i]); }return ret;@@ -3544,7 +3567,7 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,atomic_read (&i915->gpu_error.reset_counter), i915->mm.interruptible,NULL , - &i915->rps.semaphores); + &i915->rps.semaphores, true);if (ret)return ret;@@ -4523,7 +4546,7 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)if (target ==NULL )return 0 ; - ret =__i915_wait_request (target, reset_counter, true,NULL ,NULL ); + ret =__i915_wait_request (target, reset_counter, true,NULL ,NULL , false);if (ret ==0 )queue_delayed_work (dev_priv->wq, &dev_priv->mm.retire_work,0 );drivers/gpu/drm/i915/i915_scheduler.c +12
-0@@ -918,6 +918,37 @@ void i915_scheduler_work_handler(struct work_struct *work) }/** +* i915_scheduler_is_request_tracked - return info to say what the scheduler's +* connection to this request is (if any). +* @req: request to be queried +* @compeleted: if non-null, set to completion status +* @busy: if non-null set to busy status +* +* Looks up the given request in the scheduler's internal queue and reports +* on whether the request has completed or is still pending. +* Returns true if the request was found or false if it was not. +*/ +bool i915_scheduler_is_request_tracked (struct drm_i915_gem_request *req, +bool *completed,bool *busy)drivers/gpu/drm/i915/i915_scheduler.c void i915_scheduler_work_handler(struct work_struct *work) · line 942
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c +10
-0+{ +struct drm_i915_private *dev_priv = req->ring->dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + +if (!scheduler) +return false; + +if (req->scheduler_qe ==NULL ) +return false; +These better be their own functions, making the code more readable.
Quoted 2 lines
_is_request_completed()_is_request_busy()
Yeah, the original need has actually changed somewhat since this was written. There is no longer any code which requires all three items together. So I've split it into two simpler functions which are more appropriate to their use cases.
Quoted 59 lines
Quoted 59 lines
drivers/gpu/drm/i915/i915_scheduler.c +9
-0+if (completed) + *completed =I915_SQS_IS_COMPLETE (req->scheduler_qe); +if (busy) + *busy =I915_SQS_IS_QUEUED (req->scheduler_qe); + +return true; +} + +/** * i915_scheduler_closefile - notify the scheduler that a DRM file handle * has been closed. * @dev: DRM device drivers/gpu/drm/i915/i915_scheduler.h +2
-0@@ -94,5 +94,7 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);bool i915_scheduler_notify_request (struct drm_i915_gem_request *req);void i915_scheduler_wakeup (struct drm_device *dev);void i915_scheduler_work_handler (struct work_struct *work); +bool i915_scheduler_is_request_tracked (struct drm_i915_gem_request *req, +bool *completed,bool *busy);#endif /* _I915_SCHEDULER_H_ */ drivers/gpu/drm/i915/intel_display.c +3
-2@@ -11458,7 +11458,8 @@ static void intel_mmio_flip_work_func(struct work_struct *work)WARN_ON (__i915_wait_request (mmio_flip->req, mmio_flip->crtc->reset_counter, false,NULL , - &mmio_flip->i915->rps.mmioflips)); + &mmio_flip->i915->rps.mmioflips, + false));i915_gem_request_unreference (mmio_flip->req); }@@ -13523,7 +13524,7 @@ static int intel_atomic_prepare_commit(struct drm_device *dev, ret =__i915_wait_request (intel_plane_state->wait_req, reset_counter, true, -NULL ,NULL ); +NULL ,NULL , false);/* Swallow -EIO errors to allow updates during hw lockup. */ if (ret == -EIO )drivers/gpu/drm/i915/intel_ringbuffer.c +1
-1@@ -2304,7 +2304,7 @@ int intel_ring_idle(struct intel_engine_cs *ring)return __i915_wait_request (req,atomic_read (&to_i915 (ring->dev)->gpu_error.reset_counter),to_i915 (ring->dev)->mm.interruptible, -NULL ,NULL ); +NULL ,NULL , true); }
Quoted 1 line
drivers/gpu/drm/i915/intel_ringbuffer.c int intel_ring_idle(struct intel_engine_cs *ring) · line 2310
Quoted 1 line
drivers/gpu/drm/i915/intel_ringbuffer.c +0
-0int intel_ring_alloc_request_extras (struct drm_i915_gem_request *request)
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 18/35 drm/i915: Added scheduler support to page fault handler
In reply to [email protected]From: John Harrison <[email protected]>
GPU page faults can now require scheduler operation in order to complete. For example, in order to free up sufficient memory to handle the fault the handler must wait for a batch buffer to complete that has not even been sent to the hardware yet. Thus EAGAIN no longer means a GPU hang, it can occur under normal operation.
For: VIZ-1587
--- drivers/gpu/drm/i915/i915_gem.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_gem.c +7 -2
@@ -2003,10 +2003,15 @@ out: }case -EAGAIN :/* -* EAGAIN means the gpu is hung and we'll wait for the error -* handler to reset everything when re-faulting in +* EAGAIN can mean the gpu is hung and we'll have to wait for +* the error handler to reset everything when re-faulting in * i915_mutex_lock_interruptible. +* +* It can also indicate various other nonfatal errors for which +* the best response is to give other threads a chance to run, +* and then retry the failing operation in its entirety. */ +/*FALLTHRU*/ case 0 :case -ERESTARTSYS :case -EINTR :
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 18/35] drm/i915: Added scheduler support to page fault handler
Quoted 27 lines · On 02/18/2016 06:27 AM, [email protected] wrote:
From: John Harrison <[email protected]>
GPU page faults can now require scheduler operation in order to complete. For example, in order to free up sufficient memory to handle the fault the handler must wait for a batch buffer to complete that has not even been sent to the hardware yet. Thus EAGAIN no longer means a GPU hang, it can occur under normal operation.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>--- drivers/gpu/drm/i915/i915_gem.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)drivers/gpu/drm/i915/i915_gem.c +2
-2@@ -2003,10 +2003,15 @@ out: }case -EAGAIN :/* -* EAGAIN means the gpu is hung and we'll wait for the error -* handler to reset everything when re-faulting in +* EAGAIN can mean the gpu is hung and we'll have to wait for +* the error handler to reset everything when re-faulting in
drivers/gpu/drm/i915/i915_gem.c out: · line 2017
Quoted 10 lines
drivers/gpu/drm/i915/i915_gem.c +5
-0* i915_mutex_lock_interruptible. +* +* It can also indicate various other nonfatal errors for which +* the best response is to give other threads a chance to run, +* and then retry the failing operation in its entirety. */ +/*FALLTHRU*/ case 0 :case -ERESTARTSYS :case -EINTR :
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 19/35 drm/i915: Added scheduler flush calls to ring throttle and idle functions
In reply to [email protected]From: John Harrison <[email protected]>
When requesting that all GPU work is completed, it is now necessary to get the scheduler involved in order to flush out work that queued and not yet submitted.
v2: Updated to add support for flushing the scheduler queue by time stamp rather than just doing a blanket flush.
v3: Moved submit_max_priority() to this patch from an earlier patch is it is no longer required in the other.
v4: Corrected the format of a comment to keep the style checker happy. Downgraded a BUG_ON to a WARN_ON as the latter is preferred.
v5: Shuffled functions around to remove forward prototypes, removed similarly offensive white space and added documentation. Re-worked the mutex locking around the submit function. [Joonas Lahtinen]
Used lighter weight spinlocks.
For: VIZ-1587
--- drivers/gpu/drm/i915/i915_gem.c | 24 ++++- drivers/gpu/drm/i915/i915_scheduler.c | 178 ++++++++++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_scheduler.h | 3 + 3 files changed, 204 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_gem.c +23 -1
@@ -3786,6 +3786,10 @@ int i915_gpu_idle(struct drm_device *dev)/* Flush everything onto the inactive list. */ for_each_ring (ring, dev_priv, i) { + ret =i915_scheduler_flush (ring, true); +if (ret <0 ) +return ret; +if (!i915.enable_execlists) {struct drm_i915_gem_request *req;
@@ -4519,7 +4523,8 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)unsigned long recent_enough = jiffies -DRM_I915_THROTTLE_JIFFIES ;struct drm_i915_gem_request *request, *target =NULL ;unsigned reset_counter; -int ret; +int i, ret; +struct intel_engine_cs *ring; ret =i915_gem_wait_for_error (&dev_priv->gpu_error);if (ret)
@@ -4529,6 +4534,23 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)if (ret)return ret; +for_each_ring (ring, dev_priv, i) { +/* +* Flush out scheduler entries that are getting 'stale'. Note +* that the following recent_enough test will only check +* against the time at which the request was submitted to the +* hardware (i.e. when it left the scheduler) not the time it +* was submitted to the driver. +* +* Also, there is not much point worring about busy return +* codes from the scheduler flush call. Even if more work +* cannot be submitted right now for whatever reason, we +* still want to throttle against stale work that has already +* been submitted. +*/ +i915_scheduler_flush_stamp (ring, recent_enough, false); + } +spin_lock (&file_priv->mm.lock);list_for_each_entry (request, &file_priv->mm.request_list, client_list) {if (time_after_eq (request->emitted_jiffies, recent_enough))
drivers/gpu/drm/i915/i915_scheduler.c +178 -0
196 lines not on this page · raw message
drivers/gpu/drm/i915/i915_scheduler.h +3 -0
@@ -94,6 +94,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);bool i915_scheduler_notify_request (struct drm_i915_gem_request *req);void i915_scheduler_wakeup (struct drm_device *dev);void i915_scheduler_work_handler (struct work_struct *work); +int i915_scheduler_flush (struct intel_engine_cs *ring,bool is_locked); +int i915_scheduler_flush_stamp (struct intel_engine_cs *ring, +unsigned long stamp,bool is_locked);bool i915_scheduler_is_request_tracked (struct drm_i915_gem_request *req,bool *completed,bool *busy);
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Joonas Lahtinen
maintainerRe: [PATCH v5 19/35] drm/i915: Added scheduler flush calls to ring throttle and idle functions
Quoted 104 lines · On to, 2016-02-18 at 14:27 +0000, [email protected] wrote:
From: John Harrison <[email protected]>
When requesting that all GPU work is completed, it is now necessary to get the scheduler involved in order to flush out work that queued and not yet submitted.
v2: Updated to add support for flushing the scheduler queue by time stamp rather than just doing a blanket flush.
v3: Moved
submit_max_priority()to this patch from an earlier patch is it is no longer required in the other.v4: Corrected the format of a comment to keep the style checker happy. Downgraded a
BUG_ONto aWARN_ONas the latter is preferred.v5: Shuffled functions around to remove forward prototypes, removed similarly offensive white space and added documentation. Re-worked the mutex locking around the submit function. [Joonas Lahtinen]
Used lighter weight spinlocks.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>Cc Joonas Lahtinen <[email protected]>---
drivers/gpu/drm/i915/i915_gem.c | 24 ++++-drivers/gpu/drm/i915/i915_scheduler.c| 178 ++++++++++++++++++++++++++++++++++drivers/gpu/drm/i915/i915_scheduler.h| 3 +3 files changed, 204 insertions(+), 1 deletion(-)drivers/gpu/drm/i915/i915_gem.c +0
-0@@ -3786,6 +3786,10 @@ int i915_gpu_idle(struct drm_device *dev)
/* Flush everything onto the inactive list. */for_each_ring(ring, dev_priv, i){drivers/gpu/drm/i915/i915_gem.c +4
-0+ ret =i915_scheduler_flush (ring, true); +if (ret <0 ) +return ret; +if (!i915.enable_execlists) { struct
drm_i915_gem_request*req;drivers/gpu/drm/i915/i915_gem.c +2
-1@@ -4519,7 +4523,8 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file) unsigned long recent_enough = jiffies - DRM_I915_THROTTLE_JIFFIES; struct drm_i915_gem_request *request, *target = NULL; unsigned reset_counter; -int ret; +int i, ret; +struct intel_engine_cs *ring;
ret =i915_gem_wait_for_error(&dev_priv->gpu_error); if (ret)drivers/gpu/drm/i915/i915_gem.c +0
-0@@ -4529,6 +4534,23 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file) if (ret) return ret;drivers/gpu/drm/i915/i915_gem.c +17
-0+for_each_ring (ring, dev_priv, i) { +/* +* Flush out scheduler entries that are getting 'stale'. Note +* that the following recent_enough test will only check +* against the time at which the request was submitted to the +* hardware (i.e. when it left the scheduler) not the time it +* was submitted to the driver. +* +* Also, there is not much point worring about busy return +* codes from the scheduler flush call. Even if more work +* cannot be submitted right now for whatever reason, we +* still want to throttle against stale work that has already +* been submitted. +*/ +i915_scheduler_flush_stamp (ring, recent_enough, false); + } +
spin_lock(&file_priv->mm.lock);list_for_each_entry(request, &file_priv->mm.request_list, client_list){ if (time_after_eq(request->emitted_jiffies, recent_enough))drivers/gpu/drm/i915/i915_scheduler.c +4
-0@@ -304,6 +304,10 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,* attempting to acquire a mutex while holding a spin lock is a Bad Idea. * And releasing the one before acquiring the other leads to other code * being run and interfering. +* +* Hence any caller that does not already have the mutex lock for other +* reasons should call i915_scheduler_submit_unlocked() instead in order to +* obtain the lock first. */ static int i915_scheduler_submit(struct intel_engine_cs *ring) {@@ -428,6 +432,22 @@ error: return ret; }drivers/gpu/drm/i915/i915_scheduler.c +6
-0+static int i915_scheduler_submit_unlocked (struct intel_engine_cs *ring) +{ +struct drm_device *dev = ring->dev; +int ret; + + ret =i915_mutex_lock_interruptible (dev);
drivers/gpu/drm/i915/i915_scheduler.c error: · line 450
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c +10
-0+if (ret) +return ret; + + ret =i915_scheduler_submit (ring); + +mutex_unlock (&dev->struct_mutex); + +return ret; +} +
This kind of code should really be factored to upper level functions.
Quoted 28 lines
static void
i915_generate_dependencies(struct i915_scheduler *scheduler, struct i915_scheduler_queue_entry *node, uint32_t ring)drivers/gpu/drm/i915/i915_scheduler.c +0
-0@@ -917,6 +937,164 @@ void i915_scheduler_work_handler(struct work_struct *work) i915_scheduler_process_work(ring); }drivers/gpu/drm/i915/i915_scheduler.c +21
-0+static int i915_scheduler_submit_max_priority (struct intel_engine_cs *ring, + bool is_locked) +{ +struct i915_scheduler_queue_entry *node; +struct drm_i915_private *dev_priv = ring->dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; +int ret, count =0 ; +bool found; + + do { + found = false; + spin_lock_irq(&scheduler->lock); + list_for_each_entry(node, &scheduler->node_queue[ring->id], link) { + if (!I915_SQS_IS_QUEUED(node)) + continue; + + if (node->priority < scheduler->priority_level_max) + continue; + + found = true; + break;
drivers/gpu/drm/i915/i915_scheduler.c void i915_scheduler_work_handler(struct work_struct *work) · line 970
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c +10
-0+ } +spin_unlock_irq (&scheduler->lock); + +if (!found) +break ; + +if (is_locked) + ret =i915_scheduler_submit (ring); +else + ret =i915_scheduler_submit_unlocked (ring);
The widespread use of 'is_locked' notation is really bothering me, I think the difference in code path should be handled in the upper level, instead of being passed down, for example this will cause repetitive lock/unlock in loop.
Quoted 38 lines
drivers/gpu/drm/i915/i915_scheduler.c +38
-0+if (ret <0 ) +return ret; + + count += ret; + }while (found); + +return count; +} + +/** +* i915_scheduler_flush_stamp - force requests of a given age through the +* scheduler. +* @ring: Ring to be flushed +* @target: Jiffy based time stamp to flush up to +* @is_locked: Is the driver mutex lock held? +* DRM has a throttle by age of request facility. This requires waiting for +* outstanding work over a given age. This function helps that by forcing +* queued batch buffers over said age through the system. +* Returns zero on success or -EAGAIN if the scheduler is busy (e.g. waiting +* for a pre-emption event to complete) but the mutex lock is held which +* would prevent the scheduler's asynchronous processing from completing. +*/ +int i915_scheduler_flush_stamp (struct intel_engine_cs *ring, + unsigned long target, + bool is_locked) +{ +struct i915_scheduler_queue_entry *node; +struct drm_i915_private *dev_priv; +struct i915_scheduler *scheduler; +int flush_count =0 ; + +if (!ring) +return -EINVAL ; + + dev_priv = ring->dev->dev_private; + scheduler = dev_priv->scheduler; + +if (!scheduler)
drivers/gpu/drm/i915/i915_scheduler.c void i915_scheduler_work_handler(struct work_struct *work) · line 1018
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c +10
-0+return 0 ; + +if (is_locked && (scheduler->flags[ring->id] & i915_sf_submitting)) { +/* +* Scheduler is busy already submitting another batch, +* come back later rather than going recursive... +*/ +return -EAGAIN ; + } +
Again, this would be handled better at higher level.
So I would like to see those 'is_locked' and other similar constructs factored out from the series. It should make the code much more robust to modify at later point.
Regards, Joonas
Quoted 96 lines
drivers/gpu/drm/i915/i915_scheduler.c +79
-0+spin_lock_irq (&scheduler->lock); +i915_scheduler_priority_bump_clear (scheduler); +list_for_each_entry (node, &scheduler->node_queue[ring->id], link) { +if (!I915_SQS_IS_QUEUED (node)) +continue ; + +if (node->stamp > target) +continue ; + + flush_count =i915_scheduler_priority_bump (scheduler, + node, scheduler->priority_level_max); + } +spin_unlock_irq (&scheduler->lock); + +if (flush_count) { +DRM_DEBUG_DRIVER ("<%s> Bumped %d entries\n" , ring->name, flush_count); + flush_count =i915_scheduler_submit_max_priority (ring, is_locked); + } + +return flush_count; +} + +/** +* i915_scheduler_flush - force all requests through the scheduler. +* @ring: Ring to be flushed +* @is_locked: Is the driver mutex lock held? +* For various reasons it is sometimes necessary to the scheduler out, e.g. +* due to ring reset. +* Returns zero on success or -EAGAIN if the scheduler is busy (e.g. waiting +* for a pre-emption event to complete) but the mutex lock is held which +* would prevent the scheduler's asynchronous processing from completing. +*/ +int i915_scheduler_flush (struct intel_engine_cs *ring,bool is_locked) +{ +struct i915_scheduler_queue_entry *node; +struct drm_i915_private *dev_priv; +struct i915_scheduler *scheduler; +bool found; +int ret; +uint32_t count =0 ; + +if (!ring) +return -EINVAL ; + + dev_priv = ring->dev->dev_private; + scheduler = dev_priv->scheduler; + +if (!scheduler) +return 0 ; + +WARN_ON (is_locked && (scheduler->flags[ring->id] & i915_sf_submitting)); + +do { + found = false; +spin_lock_irq (&scheduler->lock); +list_for_each_entry (node, &scheduler->node_queue[ring->id], link) { +if (!I915_SQS_IS_QUEUED (node)) +continue ; + + found = true; +break ; + } +spin_unlock_irq (&scheduler->lock); + +if (found) { +if (is_locked) + ret =i915_scheduler_submit (ring); +else + ret =i915_scheduler_submit_unlocked (ring); +if (ret <0 ) +return ret; + + count += ret; + } + }while (found); + +return count; +} +/**
*i915_scheduler_is_request_tracked- return info to say what the scheduler's
* connection to this request is (if any).drivers/gpu/drm/i915/i915_scheduler.h +3
-0@@ -94,6 +94,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe); bool i915_scheduler_notify_request(struct drm_i915_gem_request *req); void i915_scheduler_wakeup(struct drm_device *dev); void i915_scheduler_work_handler(struct work_struct *work); +int i915_scheduler_flush (struct intel_engine_cs *ring,bool is_locked); +int i915_scheduler_flush_stamp (struct intel_engine_cs *ring, + unsigned long stamp, bool is_locked); bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req, bool *completed, bool *busy);
Signature · 7 lines
Joonas Lahtinen Open Source Technology Center Intel Corporation _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 19/35] drm/i915: Added scheduler flush calls to ring throttle and idle functions
Quoted 108 lines · Joonas Lahtinen
Quoted 107 lines · On to, 2016-02-18 at 14:27 +0000, [email protected] wrote:
From: John Harrison <[email protected]>
When requesting that all GPU work is completed, it is now necessary to get the scheduler involved in order to flush out work that queued and not yet submitted.
v2: Updated to add support for flushing the scheduler queue by time stamp rather than just doing a blanket flush.
v3: Moved
submit_max_priority()to this patch from an earlier patch is it is no longer required in the other.v4: Corrected the format of a comment to keep the style checker happy. Downgraded a
BUG_ONto aWARN_ONas the latter is preferred.v5: Shuffled functions around to remove forward prototypes, removed similarly offensive white space and added documentation. Re-worked the mutex locking around the submit function. [Joonas Lahtinen]
Used lighter weight spinlocks.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>Cc Joonas Lahtinen <[email protected]>--- drivers/gpu/drm/i915/i915_gem.c | 24 ++++- drivers/gpu/drm/i915/i915_scheduler.c | 178 ++++++++++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_scheduler.h | 3 + 3 files changed, 204 insertions(+), 1 deletion(-)drivers/gpu/drm/i915/i915_gem.c +23
-1@@ -3786,6 +3786,10 @@ int i915_gpu_idle(struct drm_device *dev)/* Flush everything onto the inactive list. */ for_each_ring (ring, dev_priv, i) { + ret =i915_scheduler_flush (ring, true); +if (ret <0 ) +return ret; +if (!i915.enable_execlists) {struct drm_i915_gem_request *req;@@ -4519,7 +4523,8 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)unsigned long recent_enough = jiffies -DRM_I915_THROTTLE_JIFFIES ;struct drm_i915_gem_request *request, *target =NULL ;unsigned reset_counter; -int ret; +int i, ret; +struct intel_engine_cs *ring; ret =i915_gem_wait_for_error (&dev_priv->gpu_error);if (ret)@@ -4529,6 +4534,23 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)if (ret)return ret; +for_each_ring (ring, dev_priv, i) { +/* +* Flush out scheduler entries that are getting 'stale'. Note +* that the following recent_enough test will only check +* against the time at which the request was submitted to the +* hardware (i.e. when it left the scheduler) not the time it +* was submitted to the driver. +* +* Also, there is not much point worring about busy return +* codes from the scheduler flush call. Even if more work +* cannot be submitted right now for whatever reason, we +* still want to throttle against stale work that has already +* been submitted. +*/ +i915_scheduler_flush_stamp (ring, recent_enough, false); + } +spin_lock (&file_priv->mm.lock);list_for_each_entry (request, &file_priv->mm.request_list, client_list) {if (time_after_eq (request->emitted_jiffies, recent_enough))drivers/gpu/drm/i915/i915_scheduler.c +13
-0@@ -304,6 +304,10 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,* attempting to acquire a mutex while holding a spin lock is a Bad Idea. * And releasing the one before acquiring the other leads to other code * being run and interfering. +* +* Hence any caller that does not already have the mutex lock for other +* reasons should call i915_scheduler_submit_unlocked() instead in order to +* obtain the lock first. */ static int i915_scheduler_submit (struct intel_engine_cs *ring) {@@ -428,6 +432,22 @@ error:return ret; } +static int i915_scheduler_submit_unlocked (struct intel_engine_cs *ring) +{ +struct drm_device *dev = ring->dev; +int ret; + + ret =i915_mutex_lock_interruptible (dev); +if (ret) +return ret; +
Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c error: · line 450
Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c +7
-0+ ret =i915_scheduler_submit (ring); + +mutex_unlock (&dev->struct_mutex); + +return ret; +} +This kind of code should really be factored to upper level functions.
See below.
Quoted 34 lines
Quoted 34 lines
drivers/gpu/drm/i915/i915_scheduler.c +27
-0static void i915_generate_dependencies (struct i915_scheduler *scheduler,struct i915_scheduler_queue_entry *node,uint32_t ring)@@ -917,6 +937,164 @@ void i915_scheduler_work_handler(struct work_struct *work)i915_scheduler_process_work (ring); } +static int i915_scheduler_submit_max_priority (struct intel_engine_cs *ring, +bool is_locked) +{ +struct i915_scheduler_queue_entry *node; +struct drm_i915_private *dev_priv = ring->dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; +int ret, count =0 ; +bool found; + + do { + found = false; + spin_lock_irq(&scheduler->lock); + list_for_each_entry(node, &scheduler->node_queue[ring->id], link) { + if (!I915_SQS_IS_QUEUED(node)) + continue; + + if (node->priority < scheduler->priority_level_max) + continue; + + found = true; + break; + } + spin_unlock_irq(&scheduler->lock); + + if (!found) + break; +
Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c void i915_scheduler_work_handler(struct work_struct *work) · line 970
Quoted 4 lines
drivers/gpu/drm/i915/i915_scheduler.c +4
-0+if (is_locked) + ret =i915_scheduler_submit (ring); +else + ret =i915_scheduler_submit_unlocked (ring);The widespread use of '
is_locked' notation is really bothering me, I think the difference in code path should be handled in the upper level, instead of being passed down, for example this will cause repetitive lock/unlock in loop.
My aim has been to try to keep the amount of code inside any given lock to the absolute minimum. One could just push the mutex lock acquire all the way up to the top level before any of the scheduler code is called. To me, that just seems an excessive and unnecessary hog of the global driver lock.
Quoted 51 lines
Quoted 39 lines
drivers/gpu/drm/i915/i915_scheduler.c +38
-0+if (ret <0 ) +return ret; + + count += ret; + }while (found); + +return count; +} + +/** +* i915_scheduler_flush_stamp - force requests of a given age through the +* scheduler. +* @ring: Ring to be flushed +* @target: Jiffy based time stamp to flush up to +* @is_locked: Is the driver mutex lock held? +* DRM has a throttle by age of request facility. This requires waiting for +* outstanding work over a given age. This function helps that by forcing +* queued batch buffers over said age through the system. +* Returns zero on success or -EAGAIN if the scheduler is busy (e.g. waiting +* for a pre-emption event to complete) but the mutex lock is held which +* would prevent the scheduler's asynchronous processing from completing. +*/ +int i915_scheduler_flush_stamp (struct intel_engine_cs *ring, +unsigned long target, +bool is_locked) +{ +struct i915_scheduler_queue_entry *node; +struct drm_i915_private *dev_priv; +struct i915_scheduler *scheduler; +int flush_count =0 ; + +if (!ring) +return -EINVAL ; + + dev_priv = ring->dev->dev_private; + scheduler = dev_priv->scheduler; + +if (!scheduler)drivers/gpu/drm/i915/i915_scheduler.c void i915_scheduler_work_handler(struct work_struct *work) · line 1018
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c +10
-0+return 0 ; + +if (is_locked && (scheduler->flags[ring->id] & i915_sf_submitting)) { +/* +* Scheduler is busy already submitting another batch, +* come back later rather than going recursive... +*/ +return -EAGAIN ; + } +Again, this would be handled better at higher level.
Quoted 3 lines
So I would like to see those '
is_locked' and other similar constructs factored out from the series. It should make the code much more robust to modify at later point.
As above. There is no need to lock the global driver mutex while doing internal scheduler processing. If you think the trade off of unnecessary locking for code simplification is best as just lock and be simple then sure, all the 'is_locked' stuff can disappear. Otherwise, I don't see any easy way to delay the mutex lock until it is actually required.
Quoted 91 lines
Regards, Joonas
Quoted 88 lines
drivers/gpu/drm/i915/i915_scheduler.c +79
-0+spin_lock_irq (&scheduler->lock); +i915_scheduler_priority_bump_clear (scheduler); +list_for_each_entry (node, &scheduler->node_queue[ring->id], link) { +if (!I915_SQS_IS_QUEUED (node)) +continue ; + +if (node->stamp > target) +continue ; + + flush_count =i915_scheduler_priority_bump (scheduler, + node, scheduler->priority_level_max); + } +spin_unlock_irq (&scheduler->lock); + +if (flush_count) { +DRM_DEBUG_DRIVER ("<%s> Bumped %d entries\n" , ring->name, flush_count); + flush_count =i915_scheduler_submit_max_priority (ring, is_locked); + } + +return flush_count; +} + +/** +* i915_scheduler_flush - force all requests through the scheduler. +* @ring: Ring to be flushed +* @is_locked: Is the driver mutex lock held? +* For various reasons it is sometimes necessary to the scheduler out, e.g. +* due to ring reset. +* Returns zero on success or -EAGAIN if the scheduler is busy (e.g. waiting +* for a pre-emption event to complete) but the mutex lock is held which +* would prevent the scheduler's asynchronous processing from completing. +*/ +int i915_scheduler_flush (struct intel_engine_cs *ring,bool is_locked) +{ +struct i915_scheduler_queue_entry *node; +struct drm_i915_private *dev_priv; +struct i915_scheduler *scheduler; +bool found; +int ret; +uint32_t count =0 ; + +if (!ring) +return -EINVAL ; + + dev_priv = ring->dev->dev_private; + scheduler = dev_priv->scheduler; + +if (!scheduler) +return 0 ; + +WARN_ON (is_locked && (scheduler->flags[ring->id] & i915_sf_submitting)); + +do { + found = false; +spin_lock_irq (&scheduler->lock); +list_for_each_entry (node, &scheduler->node_queue[ring->id], link) { +if (!I915_SQS_IS_QUEUED (node)) +continue ; + + found = true; +break ; + } +spin_unlock_irq (&scheduler->lock); + +if (found) { +if (is_locked) + ret =i915_scheduler_submit (ring); +else + ret =i915_scheduler_submit_unlocked (ring); +if (ret <0 ) +return ret; + + count += ret; + } + }while (found); + +return count; +} +/** * i915_scheduler_is_request_tracked - return info to say what the scheduler's * connection to this request is (if any). drivers/gpu/drm/i915/i915_scheduler.h +0
-0@@ -94,6 +94,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);bool i915_scheduler_notify_request (struct drm_i915_gem_request *req);
Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.h int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe); · line 102
Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.h +3
-0void i915_scheduler_wakeup (struct drm_device *dev);void i915_scheduler_work_handler (struct work_struct *work); +int i915_scheduler_flush (struct intel_engine_cs *ring,bool is_locked); +int i915_scheduler_flush_stamp (struct intel_engine_cs *ring, +unsigned long stamp,bool is_locked);bool i915_scheduler_is_request_tracked (struct drm_i915_gem_request *req,bool *completed,bool *busy);
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 20/35 drm/i915: Add scheduler hook to GPU reset
In reply to [email protected]From: John Harrison <[email protected]>
When the watchdog resets the GPU, all interrupts get disabled despite the reference count remaining. As the scheduler probably had interrupts enabled during the reset (it would have been waiting for the bad batch to complete), it must be poked to tell it that the interrupt has been disabled.
v5: New patch in series.
For: VIZ-1587
--- drivers/gpu/drm/i915/i915_gem.c | 2 ++ drivers/gpu/drm/i915/i915_scheduler.c | 11 +++++++++++ drivers/gpu/drm/i915/i915_scheduler.h | 1 + 3 files changed, 14 insertions(+)
drivers/gpu/drm/i915/i915_gem.c +2 -0
@@ -3248,6 +3248,8 @@ static void i915_gem_reset_ring_cleanup(struct drm_i915_private *dev_priv, buffer->last_retired_head = buffer->tail;intel_ring_update_space (buffer); } + +i915_scheduler_reset_cleanup (ring); }void i915_gem_reset (struct drm_device *dev)
drivers/gpu/drm/i915/i915_scheduler.c +11 -0
@@ -778,6 +778,17 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node) } } +void i915_scheduler_reset_cleanup (struct intel_engine_cs *ring) +{ +struct drm_i915_private *dev_priv = ring->dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + +if (scheduler->flags[ring->id] & i915_sf_interrupts_enabled) { + ring->irq_put (ring); + scheduler->flags[ring->id] &= ~i915_sf_interrupts_enabled; + } +} +static bool i915_scheduler_remove (struct i915_scheduler *scheduler,struct intel_engine_cs *ring,struct list_head *remove)
drivers/gpu/drm/i915/i915_scheduler.h +1 -0
@@ -89,6 +89,7 @@ bool i915_scheduler_is_enabled(struct drm_device *dev);int i915_scheduler_init (struct drm_device *dev);int i915_scheduler_closefile (struct drm_device *dev,struct drm_file *file); +void i915_scheduler_reset_cleanup (struct intel_engine_cs *ring);void i915_scheduler_clean_node (struct i915_scheduler_queue_entry *node);int i915_scheduler_queue_execbuffer (struct i915_scheduler_queue_entry *qe);bool i915_scheduler_notify_request (struct drm_i915_gem_request *req);
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 20/35] drm/i915: Add scheduler hook to GPU reset
Quoted 40 lines · On 02/18/2016 06:27 AM, [email protected] wrote:
From: John Harrison <[email protected]>
When the watchdog resets the GPU, all interrupts get disabled despite the reference count remaining. As the scheduler probably had interrupts enabled during the reset (it would have been waiting for the bad batch to complete), it must be poked to tell it that the interrupt has been disabled.
v5: New patch in series.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>--- drivers/gpu/drm/i915/i915_gem.c | 2 ++ drivers/gpu/drm/i915/i915_scheduler.c | 11 +++++++++++ drivers/gpu/drm/i915/i915_scheduler.h | 1 + 3 files changed, 14 insertions(+)drivers/gpu/drm/i915/i915_gem.c +2
-0@@ -3248,6 +3248,8 @@ static void i915_gem_reset_ring_cleanup(struct drm_i915_private *dev_priv, buffer->last_retired_head = buffer->tail;intel_ring_update_space (buffer); } + +i915_scheduler_reset_cleanup (ring); }void i915_gem_reset (struct drm_device *dev)drivers/gpu/drm/i915/i915_scheduler.c +1
-0@@ -778,6 +778,17 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node) } } +void i915_scheduler_reset_cleanup (struct intel_engine_cs *ring)
drivers/gpu/drm/i915/i915_scheduler.c void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node) · line 791
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c +10
-0+{ +struct drm_i915_private *dev_priv = ring->dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + +if (scheduler->flags[ring->id] & i915_sf_interrupts_enabled) { + ring->irq_put (ring); + scheduler->flags[ring->id] &= ~i915_sf_interrupts_enabled; + } +} +
So I guess these flags are also protected by the struct_mutex? If so, I guess it looks ok.
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 21/35 drm/i915: Added a module parameter to allow the scheduler to be disabled
In reply to [email protected]From: John Harrison <[email protected]>
It can be useful to be able to disable the GPU scheduler via a module parameter for debugging purposes.
v5: Converted from a multi-feature 'overrides' mask to a single 'enable' boolean. Further features (e.g. pre-emption) will now be separate 'enable' booleans added later. [Chris Wilson]
For: VIZ-1587
--- drivers/gpu/drm/i915/i915_params.c | 4 ++++ drivers/gpu/drm/i915/i915_params.h | 1 + drivers/gpu/drm/i915/i915_scheduler.c | 5 ++++- 3 files changed, 9 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_params.c +4 -0
@@ -57,6 +57,7 @@ struct i915_params i915 __read_mostly = { .edp_vswing = 0, .enable_guc_submission = true, .guc_log_level = -1, + .enable_scheduler = 0, };module_param_named (modeset, i915.modeset, int,0400 );
@@ -203,3 +204,6 @@ MODULE_PARM_DESC(enable_guc_submission, "Enable GuC submission (default:false)")module_param_named (guc_log_level, i915.guc_log_level, int,0400 );MODULE_PARM_DESC (guc_log_level,"GuC firmware logging level (-1:disabled (default), 0-3:enabled)" ); + +module_param_named_unsafe (enable_scheduler, i915.enable_scheduler, int,0600 ); +MODULE_PARM_DESC (enable_scheduler,"Enable scheduler (0 = disable [default], 1 = enable)" );
drivers/gpu/drm/i915/i915_params.h +1 -0
@@ -60,6 +60,7 @@ struct i915_params {bool enable_guc_submission;bool verbose_state_checks;bool nuclear_pageflip; +int enable_scheduler; };extern struct i915_params i915__read_mostly ;
drivers/gpu/drm/i915/i915_scheduler.c +4 -1
@@ -34,6 +34,9 @@ bool i915_scheduler_is_enabled(struct drm_device *dev) {struct drm_i915_private *dev_priv = dev->dev_private; +if (!i915.enable_scheduler) +return false; +return dev_priv->scheduler !=NULL ; }
@@ -548,7 +551,7 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)WARN_ON (!scheduler); -if (1 /*!i915.enable_scheduler*/ ) +if (!i915.enable_scheduler)return i915_scheduler_queue_execbuffer_bypass (qe); node =kmalloc (sizeof (*node),GFP_KERNEL );
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 21/35] drm/i915: Added a module parameter to allow the scheduler to be disabled
Quoted 64 lines · On 02/18/2016 06:27 AM, [email protected] wrote:
From: John Harrison <[email protected]>
It can be useful to be able to disable the GPU scheduler via a module parameter for debugging purposes.
v5: Converted from a multi-feature 'overrides' mask to a single 'enable' boolean. Further features (e.g. pre-emption) will now be separate 'enable' booleans added later. [Chris Wilson]
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>--- drivers/gpu/drm/i915/i915_params.c | 4 ++++ drivers/gpu/drm/i915/i915_params.h | 1 + drivers/gpu/drm/i915/i915_scheduler.c | 5 ++++- 3 files changed, 9 insertions(+), 1 deletion(-)drivers/gpu/drm/i915/i915_params.c +4
-0@@ -57,6 +57,7 @@ struct i915_params i915 __read_mostly = { .edp_vswing = 0, .enable_guc_submission = true, .guc_log_level = -1, + .enable_scheduler = 0, };module_param_named (modeset, i915.modeset, int,0400 );@@ -203,3 +204,6 @@ MODULE_PARM_DESC(enable_guc_submission, "Enable GuC submission (default:false)")module_param_named (guc_log_level, i915.guc_log_level, int,0400 );MODULE_PARM_DESC (guc_log_level,"GuC firmware logging level (-1:disabled (default), 0-3:enabled)" ); + +module_param_named_unsafe (enable_scheduler, i915.enable_scheduler, int,0600 ); +MODULE_PARM_DESC (enable_scheduler,"Enable scheduler (0 = disable [default], 1 = enable)" );drivers/gpu/drm/i915/i915_params.h +1
-0@@ -60,6 +60,7 @@ struct i915_params {bool enable_guc_submission;bool verbose_state_checks;bool nuclear_pageflip; +int enable_scheduler; };extern struct i915_params i915__read_mostly ;drivers/gpu/drm/i915/i915_scheduler.c +3
-0@@ -34,6 +34,9 @@ bool i915_scheduler_is_enabled(struct drm_device *dev) {struct drm_i915_private *dev_priv = dev->dev_private; +if (!i915.enable_scheduler) +return false; +return dev_priv->scheduler !=NULL ; }
drivers/gpu/drm/i915/i915_scheduler.c int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe) · line 557
Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c +1
-1@@ -548,7 +551,7 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)WARN_ON (!scheduler); -if (1 /*!i915.enable_scheduler*/ ) +if (!i915.enable_scheduler)return i915_scheduler_queue_execbuffer_bypass (qe); node =kmalloc (sizeof (*node),GFP_KERNEL );
I did a double take here; maybe a comment along the lines of "if the scheduler is disabled, queue the buffer immediately" would help, and something similar for where the if (1) is added temporarily.
Doesn't matter too much though.
Thanks,
Jesse
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 22/35 drm/i915: Support for 'unflushed' ring idle
In reply to [email protected]From: John Harrison <[email protected]>
When the seqno wraps around zero, the entire GPU is forced to be idle for some reason (possibly only to work around issues with hardware semaphores but no-one seems too sure!). This causes a problem if the force idle occurs at an inopportune moment such as in the middle of submitting a batch buffer. Specifically, it would lead to recursive submits - submitting work requires a new seqno, the new seqno requires idling the ring, idling the ring requires submitting work, submitting work requires a new seqno...
This change adds a 'flush' parameter to the idle function call which specifies whether the scheduler queues should be flushed out. I.e. is the call intended to just idle the ring as it is right now (no flush) or is it intended to force all outstanding work out of the system (with flush).
In the seqno wrap case, pending work is not an issue because the next operation will be to submit it. However, in other cases, the intention is to make sure everything that could be done has been done.
For: VIZ-1587
--- drivers/gpu/drm/i915/i915_gem.c | 4 ++-- drivers/gpu/drm/i915/intel_lrc.c | 2 +- drivers/gpu/drm/i915/intel_ringbuffer.c | 17 +++++++++++++++-- drivers/gpu/drm/i915/intel_ringbuffer.h | 2 +- 4 files changed, 19 insertions(+), 6 deletions(-)
drivers/gpu/drm/i915/i915_gem.c +2 -2
@@ -2564,7 +2564,7 @@ i915_gem_init_seqno(struct drm_device *dev, u32 seqno)/* Carefully retire all requests without writing to the rings */ for_each_ring (ring, dev_priv, i) { - ret =intel_ring_idle (ring); + ret =intel_ring_idle (ring, false);if (ret)return ret; }
@@ -3808,7 +3808,7 @@ int i915_gpu_idle(struct drm_device *dev)i915_add_request_no_flush (req); } - ret =intel_ring_idle (ring); + ret =intel_ring_idle (ring, true);if (ret)return ret; }
drivers/gpu/drm/i915/intel_lrc.c +1 -1
@@ -1058,7 +1058,7 @@ void intel_logical_ring_stop(struct intel_engine_cs *ring)if (!intel_ring_initialized (ring))return ; - ret =intel_ring_idle (ring); + ret =intel_ring_idle (ring, true);if (ret && !i915_reset_in_progress (&to_i915 (ring->dev)->gpu_error))DRM_ERROR ("failed to quiesce %s whilst cleaning up: %d\n" , ring->name, ret);
drivers/gpu/drm/i915/intel_ringbuffer.c +15 -2
@@ -2288,9 +2288,22 @@ static void __wrap_ring_buffer(struct intel_ringbuffer *ringbuf)intel_ring_update_space (ringbuf); } -int intel_ring_idle (struct intel_engine_cs *ring) +int intel_ring_idle (struct intel_engine_cs *ring,bool flush) {struct drm_i915_gem_request *req; +int ret; + +/* +* NB: Must not flush the scheduler if this idle request is from +* within an execbuff submission (i.e. due to 'get_seqno' calling +* 'wrap_seqno' calling 'idle'). As that would lead to recursive +* flushes! +*/ +if (flush) { + ret =i915_scheduler_flush (ring, true); +if (ret) +return ret; + }/* Wait upon the last request to be completed */ if (list_empty (&ring->request_list))
@@ -3095,7 +3108,7 @@ intel_stop_ring_buffer(struct intel_engine_cs *ring)if (!intel_ring_initialized (ring))return ; - ret =intel_ring_idle (ring); + ret =intel_ring_idle (ring, true);if (ret && !i915_reset_in_progress (&to_i915 (ring->dev)->gpu_error))DRM_ERROR ("failed to quiesce %s whilst cleaning up: %d\n" , ring->name, ret);
drivers/gpu/drm/i915/intel_ringbuffer.h +1 -1
@@ -478,7 +478,7 @@ void intel_ring_update_space(struct intel_ringbuffer *ringbuf);int intel_ring_space (struct intel_ringbuffer *ringbuf);bool intel_ring_stopped (struct intel_engine_cs *ring); -int __must_check intel_ring_idle (struct intel_engine_cs *ring); +int __must_check intel_ring_idle (struct intel_engine_cs *ring,bool flush);void intel_ring_init_seqno (struct intel_engine_cs *ring,u32 seqno);int intel_ring_flush_all_caches (struct drm_i915_gem_request *req);int intel_ring_invalidate_all_caches (struct drm_i915_gem_request *req);
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 22/35] drm/i915: Support for 'unflushed' ring idle
Quoted 108 lines · On 02/18/2016 06:27 AM, [email protected] wrote:
From: John Harrison <[email protected]>
When the seqno wraps around zero, the entire GPU is forced to be idle for some reason (possibly only to work around issues with hardware semaphores but no-one seems too sure!). This causes a problem if the force idle occurs at an inopportune moment such as in the middle of submitting a batch buffer. Specifically, it would lead to recursive submits - submitting work requires a new seqno, the new seqno requires idling the ring, idling the ring requires submitting work, submitting work requires a new seqno...
This change adds a 'flush' parameter to the idle function call which specifies whether the scheduler queues should be flushed out. I.e. is the call intended to just idle the ring as it is right now (no flush) or is it intended to force all outstanding work out of the system (with flush).
In the seqno wrap case, pending work is not an issue because the next operation will be to submit it. However, in other cases, the intention is to make sure everything that could be done has been done.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>--- drivers/gpu/drm/i915/i915_gem.c | 4 ++-- drivers/gpu/drm/i915/intel_lrc.c | 2 +- drivers/gpu/drm/i915/intel_ringbuffer.c | 17 +++++++++++++++-- drivers/gpu/drm/i915/intel_ringbuffer.h | 2 +- 4 files changed, 19 insertions(+), 6 deletions(-)drivers/gpu/drm/i915/i915_gem.c +2
-2@@ -2564,7 +2564,7 @@ i915_gem_init_seqno(struct drm_device *dev, u32 seqno)/* Carefully retire all requests without writing to the rings */ for_each_ring (ring, dev_priv, i) { - ret =intel_ring_idle (ring); + ret =intel_ring_idle (ring, false);if (ret)return ret; }@@ -3808,7 +3808,7 @@ int i915_gpu_idle(struct drm_device *dev)i915_add_request_no_flush (req); } - ret =intel_ring_idle (ring); + ret =intel_ring_idle (ring, true);if (ret)return ret; }drivers/gpu/drm/i915/intel_lrc.c +1
-1@@ -1058,7 +1058,7 @@ void intel_logical_ring_stop(struct intel_engine_cs *ring)if (!intel_ring_initialized (ring))return ; - ret =intel_ring_idle (ring); + ret =intel_ring_idle (ring, true);if (ret && !i915_reset_in_progress (&to_i915 (ring->dev)->gpu_error))DRM_ERROR ("failed to quiesce %s whilst cleaning up: %d\n" , ring->name, ret);drivers/gpu/drm/i915/intel_ringbuffer.c +15
-2@@ -2288,9 +2288,22 @@ static void __wrap_ring_buffer(struct intel_ringbuffer *ringbuf)intel_ring_update_space (ringbuf); } -int intel_ring_idle (struct intel_engine_cs *ring) +int intel_ring_idle (struct intel_engine_cs *ring,bool flush) {struct drm_i915_gem_request *req; +int ret; + +/* +* NB: Must not flush the scheduler if this idle request is from +* within an execbuff submission (i.e. due to 'get_seqno' calling +* 'wrap_seqno' calling 'idle'). As that would lead to recursive +* flushes! +*/ +if (flush) { + ret =i915_scheduler_flush (ring, true); +if (ret) +return ret; + }/* Wait upon the last request to be completed */ if (list_empty (&ring->request_list))@@ -3095,7 +3108,7 @@ intel_stop_ring_buffer(struct intel_engine_cs *ring)if (!intel_ring_initialized (ring))return ; - ret =intel_ring_idle (ring); + ret =intel_ring_idle (ring, true);if (ret && !i915_reset_in_progress (&to_i915 (ring->dev)->gpu_error))DRM_ERROR ("failed to quiesce %s whilst cleaning up: %d\n" , ring->name, ret);drivers/gpu/drm/i915/intel_ringbuffer.h +0
-0
drivers/gpu/drm/i915/intel_ringbuffer.h void intel_ring_update_space(struct intel_ringbuffer *ringbuf); · line 484
Quoted 8 lines
drivers/gpu/drm/i915/intel_ringbuffer.h +1
-1@@ -478,7 +478,7 @@ void intel_ring_update_space(struct intel_ringbuffer *ringbuf);int intel_ring_space (struct intel_ringbuffer *ringbuf);bool intel_ring_stopped (struct intel_engine_cs *ring); -int __must_check intel_ring_idle (struct intel_engine_cs *ring); +int __must_check intel_ring_idle (struct intel_engine_cs *ring,bool flush);void intel_ring_init_seqno (struct intel_engine_cs *ring,u32 seqno);int intel_ring_flush_all_caches (struct drm_i915_gem_request *req);int intel_ring_invalidate_all_caches (struct drm_i915_gem_request *req);
Maybe Chris remembers the history here; I think the wraparound idle goes all the way back to Eric's original work with wrapping (something we had a lot of trouble with early on iirc).
My only suggestion here is to add wrappers for a new __intel_ring_idle(ring, bool), one for the existing usage of intel_ring_idle(ring) (which would pass false) and a new intel_ring_flush(ring) that passes true, along with some kdoc explaining the difference. Otherwise everyone will have to look up the param all the time. :)
With that change (because I'm a bool param hater, at least in exposed APIs):
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 23/35 drm/i915: Defer seqno allocation until actual hardware submission time
In reply to [email protected]From: John Harrison <[email protected]>
The seqno value is now only used for the final test for completion of a request. It is no longer used to track the request through the software stack. Thus it is no longer necessary to allocate the seqno immediately with the request. Instead, it can be done lazily and left until the request is actually sent to the hardware. This is particular advantageous with a GPU scheduler as the requests can then be re-ordered between their creation and their hardware submission without having out of order seqnos.
v2: i915_add_request() can't fail!
Combine with 'drm/i915: Assign seqno at start of exec_final()' Various bits of code during the execbuf code path need a seqno value to be assigned to the request. This change makes this assignment explicit at the start of submission_final() rather than relying on an auto-generated seqno to have happened already. This is in preparation for a future patch which changes seqno values to be assigned lazily (during add_request).
v3: Updated to use locally cached request pointer.
v4: Changed some white space and comment formatting to keep the style checker happy.
For: VIZ-1587
--- drivers/gpu/drm/i915/i915_drv.h | 1 + drivers/gpu/drm/i915/i915_gem.c | 23 ++++++++++++++++++++++- drivers/gpu/drm/i915/i915_gem_execbuffer.c | 14 ++++++++++++++ drivers/gpu/drm/i915/intel_lrc.c | 14 ++++++++++++++ 4 files changed, 51 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_drv.h +1 -0
@@ -2257,6 +2257,7 @@ struct drm_i915_gem_request {* has finished processing this request. */ u32 seqno; +u32 reserved_seqno;/* Unique identifier which can be used for trace points & debug */ uint32_t uniq;
drivers/gpu/drm/i915/i915_gem.c +22 -1
@@ -2616,6 +2616,11 @@ i915_gem_get_seqno(struct drm_device *dev, u32 *seqno)/* reserve 0 for non-seqno */ if (dev_priv->next_seqno ==0 ) { +/* +* Why is the full re-initialisation required? Is it only for +* hardware semaphores? If so, could skip it in the case where +* semaphores are disabled? +*/ int ret =i915_gem_init_seqno (dev,0 );if (ret)return ret;
@@ -2673,6 +2678,12 @@ void __i915_add_request(struct drm_i915_gem_request *request,WARN (ret,"*_ring_flush_all_caches failed: %d!\n" , ret); } +/* Make the request's seqno 'live': */ +if (!request->seqno) { + request->seqno = request->reserved_seqno; +WARN_ON (request->seqno != dev_priv->last_seqno); + } +/* Record the position of the start of the request so that * should we detect the updated seqno part-way through the * GPU processing the request, we never over-estimate the
@@ -2930,6 +2941,9 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)list_for_each_entry_safe (req, req_next, &ring->fence_signal_list, signal_link) {if (!req->cancelled) { +/* How can this happen? */ +WARN_ON (req->seqno ==0 ); +if (!i915_seqno_passed (seqno, req->seqno))break ; }
@@ -3079,7 +3093,14 @@ int i915_gem_request_alloc(struct intel_engine_cs *ring,if (req ==NULL )return -ENOMEM ; - ret =i915_gem_get_seqno (ring->dev, &req->seqno); +/* +* Assign an identifier to track this request through the hardware +* but don't make it live yet. It could change in the future if this +* request gets overtaken. However, it still needs to be allocated +* in advance because the point of submission must not fail and seqno +* allocation can fail. +*/ + ret =i915_gem_get_seqno (ring->dev, &req->reserved_seqno);if (ret) goto err;
drivers/gpu/drm/i915/i915_gem_execbuffer.c +14 -0
@@ -1295,6 +1295,20 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)/* The mutex must be acquired before calling this function */ WARN_ON (!mutex_is_locked (¶ms->dev->struct_mutex)); +/* Make sure the request's seqno is the latest and greatest: */ +if (req->reserved_seqno != dev_priv->last_seqno) { + ret =i915_gem_get_seqno (ring->dev, &req->reserved_seqno); +if (ret) +return ret; + } +/* +* And make it live because some of the execbuff submission code +* requires the seqno to be available up front. +*/ +WARN_ON (req->seqno); + req->seqno = req->reserved_seqno; +WARN_ON (req->seqno != dev_priv->last_seqno); + ret =intel_ring_reserve_space (req);if (ret) goto error;
drivers/gpu/drm/i915/intel_lrc.c +14 -0
@@ -978,6 +978,20 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)/* The mutex must be acquired before calling this function */ WARN_ON (!mutex_is_locked (¶ms->dev->struct_mutex)); +/* Make sure the request's seqno is the latest and greatest: */ +if (req->reserved_seqno != dev_priv->last_seqno) { + ret =i915_gem_get_seqno (ring->dev, &req->reserved_seqno); +if (ret) +return ret; + } +/* +* And make it live because some of the execbuff submission code +* requires the seqno to be available up front. +*/ +WARN_ON (req->seqno); + req->seqno = req->reserved_seqno; +WARN_ON (req->seqno != dev_priv->last_seqno); + ret =intel_logical_ring_reserve_space (req);if (ret) goto err;
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Joonas Lahtinen
maintainerRe: [PATCH v5 23/35] drm/i915: Defer seqno allocation until actual hardware submission time
Quoted 110 lines · On to, 2016-02-18 at 14:27 +0000, [email protected] wrote:
From: John Harrison <[email protected]>
The seqno value is now only used for the final test for completion of a request. It is no longer used to track the request through the software stack. Thus it is no longer necessary to allocate the seqno immediately with the request. Instead, it can be done lazily and left until the request is actually sent to the hardware. This is particular advantageous with a GPU scheduler as the requests can then be re-ordered between their creation and their hardware submission without having out of order seqnos.
v2:
i915_add_request()can't fail!Combine with 'drm/i915: Assign seqno at start of
exec_final()' Various bits of code during the execbuf code path need a seqno value to be assigned to the request. This change makes this assignment explicit at the start ofsubmission_final()rather than relying on an auto-generated seqno to have happened already. This is in preparation for a future patch which changes seqno values to be assigned lazily (duringadd_request).v3: Updated to use locally cached request pointer.
v4: Changed some white space and comment formatting to keep the style checker happy.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>---
drivers/gpu/drm/i915/i915_drv.h | 1 + drivers/gpu/drm/i915/i915_gem.c | 23 ++++++++++++++++++++++-drivers/gpu/drm/i915/i915_gem_execbuffer.c| 14 ++++++++++++++ drivers/gpu/drm/i915/intel_lrc.c | 14 ++++++++++++++4 files changed, 51 insertions(+), 1 deletion(-)drivers/gpu/drm/i915/i915_drv.h +1
-0@@ -2257,6 +2257,7 @@ struct drm_i915_gem_request {* has finished processing this request. */ u32 seqno; +u32 reserved_seqno;
/* Unique identifier which can be used for trace points & debug */uint32_tuniq;drivers/gpu/drm/i915/i915_gem.c +0
-0@@ -2616,6 +2616,11 @@ i915_gem_get_seqno(struct drm_device *dev, u32 *seqno)
/* reserve 0 for non-seqno */
if (dev_priv->next_seqno== 0) {drivers/gpu/drm/i915/i915_gem.c +5
-0+/* +* Why is the full re-initialisation required? Is it only for +* hardware semaphores? If so, could skip it in the case where +* semaphores are disabled? +*/ int ret =
i915_gem_init_seqno(dev, 0); if (ret)
return ret;drivers/gpu/drm/i915/i915_gem.c +0
-0@@ -2673,6 +2678,12 @@ void __i915_add_request(struct drm_i915_gem_request *request, WARN(ret, "*_ring_flush_all_caches failed: %d!\n", ret); }drivers/gpu/drm/i915/i915_gem.c +6
-0+/* Make the request's seqno 'live': */ +if (!request->seqno) { + request->seqno = request->reserved_seqno; +WARN_ON (request->seqno != dev_priv->last_seqno); + } +/* Record the position of the start of the request so that
* should we detect the updated seqno part-way through the
* GPU processing the request, we never over-estimate thedrivers/gpu/drm/i915/i915_gem.c +0
-0@@ -2930,6 +2941,9 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)
list_for_each_entry_safe(req, req_next, &ring->fence_signal_list, signal_link){ if (!req->cancelled) {drivers/gpu/drm/i915/i915_gem.c +3
-0+/* How can this happen? */ +WARN_ON (req->seqno ==0 ); +if (!i915_seqno_passed(seqno,
req->seqno)) break;
}drivers/gpu/drm/i915/i915_gem.c +0
-0@@ -3079,7 +3093,14 @@ int i915_gem_request_alloc(struct intel_engine_cs *ring, if (req == NULL) return -ENOMEM;drivers/gpu/drm/i915/i915_gem.c +8
-1- ret =i915_gem_get_seqno (ring->dev, &req->seqno); +/* +* Assign an identifier to track this request through the hardware +* but don't make it live yet. It could change in the future if this +* request gets overtaken. However, it still needs to be allocated +* in advance because the point of submission must not fail and seqno +* allocation can fail. +*/ + ret =i915_gem_get_seqno (ring->dev, &req->reserved_seqno);if (ret)
goto err;drivers/gpu/drm/i915/i915_gem_execbuffer.c +0
-0@@ -1295,6 +1295,20 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params) /* The mutex must be acquired before calling this function */ WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
drivers/gpu/drm/i915/i915_gem_execbuffer.c int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params) · line 1303
Quoted 6 lines
drivers/gpu/drm/i915/i915_gem_execbuffer.c +6
-0+/* Make sure the request's seqno is the latest and greatest: */ +if (req->reserved_seqno != dev_priv->last_seqno) { + ret =i915_gem_get_seqno (ring->dev, &req->reserved_seqno); +if (ret) +return ret; + }
Would this situation be caused by reordering? Kinda strange to pregenerate and then re-generate if some request got prioritized over us. I'd assume that to happen constantly with compositor running.
What exactly stops from generating the seqno always at submission time, if we're able to re-generate it anyway?
Regards, Joonas
Quoted 36 lines
drivers/gpu/drm/i915/i915_gem_execbuffer.c +8
-0+/* +* And make it live because some of the execbuff submission code +* requires the seqno to be available up front. +*/ +WARN_ON (req->seqno); + req->seqno = req->reserved_seqno; +WARN_ON (req->seqno != dev_priv->last_seqno); +ret =
intel_ring_reserve_space(req);
if (ret)
goto error;drivers/gpu/drm/i915/intel_lrc.c +0
-0@@ -978,6 +978,20 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params) /* The mutex must be acquired before calling this function */ WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));drivers/gpu/drm/i915/intel_lrc.c +14
-0+/* Make sure the request's seqno is the latest and greatest: */ +if (req->reserved_seqno != dev_priv->last_seqno) { + ret =i915_gem_get_seqno (ring->dev, &req->reserved_seqno); +if (ret) +return ret; + } +/* +* And make it live because some of the execbuff submission code +* requires the seqno to be available up front. +*/ +WARN_ON (req->seqno); + req->seqno = req->reserved_seqno; +WARN_ON (req->seqno != dev_priv->last_seqno); +ret =
intel_logical_ring_reserve_space(req); if (ret)
goto err;
Signature · 7 lines
Joonas Lahtinen Open Source Technology Center Intel Corporation _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 24/35 drm/i915: Added trace points to scheduler
In reply to [email protected]From: John Harrison <[email protected]>
Added trace points to the scheduler to track all the various events, node state transitions and other interesting things that occur.
v2: Updated for new request completion tracking implementation.
v3: Updated for changes to node kill code.
v4: Wrapped some long lines to keep the style checker happy.
For: VIZ-1587
--- drivers/gpu/drm/i915/i915_gem_execbuffer.c | 2 + drivers/gpu/drm/i915/i915_scheduler.c | 26 ++++ drivers/gpu/drm/i915/i915_trace.h | 196 +++++++++++++++++++++++++++++ drivers/gpu/drm/i915/intel_lrc.c | 2 + 4 files changed, 226 insertions(+)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +2 -0
@@ -1272,6 +1272,8 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,i915_gem_execbuffer_move_to_active (vmas, params->request); +trace_i915_gem_ring_queue (ring, params); + qe =container_of (params,typeof (*qe), params); ret =i915_scheduler_queue_execbuffer (qe);if (ret)
drivers/gpu/drm/i915/i915_scheduler.c +26 -0
@@ -88,6 +88,8 @@ static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)/* Seqno will be reassigned on relaunch */ node->params.request->seqno =0 ; node->status = i915_sqs_queued; +trace_i915_scheduler_unfly (node->params.ring, node); +trace_i915_scheduler_node_state_change (node->params.ring, node); }/*
@@ -99,7 +101,11 @@ static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)WARN_ON (!node);WARN_ON (I915_SQS_IS_COMPLETE (node)); +if (I915_SQS_IS_FLYING (node)) +trace_i915_scheduler_unfly (node->params.ring, node); + node->status = i915_sqs_dead; +trace_i915_scheduler_node_state_change (node->params.ring, node); }/* Mark a node as in flight on the hardware. */
@@ -124,6 +130,9 @@ static int i915_scheduler_node_fly(struct i915_scheduler_queue_entry *node) node->status = i915_sqs_flying; +trace_i915_scheduler_fly (ring, node); +trace_i915_scheduler_node_state_change (ring, node); +if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {bool success = true;
@@ -280,6 +289,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,INIT_LIST_HEAD (&best->link); best->status = i915_sqs_popped; +trace_i915_scheduler_node_state_change (ring, best); + ret =0 ; }else {/* Can only get here if:
@@ -297,6 +308,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring, } } +trace_i915_scheduler_pop_from_queue (ring, best); + *pop_node = best;return ret; }
@@ -506,6 +519,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_enstruct i915_scheduler *scheduler = dev_priv->scheduler;int ret; +trace_i915_scheduler_queue (qe->params.ring, qe); +intel_ring_reserved_space_cancel (qe->params.request->ringbuf); scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
@@ -628,6 +643,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe) not_flying =i915_scheduler_count_flying (scheduler, ring) < scheduler->min_flying; +trace_i915_scheduler_queue (ring, node); +trace_i915_scheduler_node_state_change (ring, node); +spin_unlock_irq (&scheduler->lock);if (not_flying)
@@ -657,6 +675,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)struct i915_scheduler_queue_entry *node = req->scheduler_qe;unsigned long flags; +trace_i915_scheduler_landing (req); +if (!node)return false;
@@ -670,6 +690,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)else node->status = i915_sqs_complete; +trace_i915_scheduler_node_state_change (req->ring, node); +spin_unlock_irqrestore (&scheduler->lock, flags);return true;
@@ -877,6 +899,8 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,/* Launch more packets now? */ do_submit = (queued >0 ) && (flying < scheduler->min_flying); +trace_i915_scheduler_remove (ring, min_seqno, do_submit); +spin_unlock_irq (&scheduler->lock);return do_submit;
@@ -912,6 +936,8 @@ static void i915_scheduler_process_work(struct intel_engine_cs *ring) node =list_first_entry (&remove,typeof (*node), link);list_del (&node->link); +trace_i915_scheduler_destroy (ring, node); +/* Free up all the DRM references */ i915_scheduler_clean_node (node);
drivers/gpu/drm/i915/i915_trace.h +196 -0
208 lines not on this page · raw message
drivers/gpu/drm/i915/intel_lrc.c +2 -0
@@ -954,6 +954,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,i915_gem_execbuffer_move_to_active (vmas, params->request); +trace_i915_gem_ring_queue (ring, params); + qe =container_of (params,typeof (*qe), params); ret =i915_scheduler_queue_execbuffer (qe);if (ret)
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 24/35] drm/i915: Added trace points to scheduler
Quoted 361 lines · On 02/18/2016 06:27 AM, [email protected] wrote:
From: John Harrison <[email protected]>
Added trace points to the scheduler to track all the various events, node state transitions and other interesting things that occur.
v2: Updated for new request completion tracking implementation.
v3: Updated for changes to node kill code.
v4: Wrapped some long lines to keep the style checker happy.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>--- drivers/gpu/drm/i915/i915_gem_execbuffer.c | 2 + drivers/gpu/drm/i915/i915_scheduler.c | 26 ++++ drivers/gpu/drm/i915/i915_trace.h | 196 +++++++++++++++++++++++++++++ drivers/gpu/drm/i915/intel_lrc.c | 2 + 4 files changed, 226 insertions(+)drivers/gpu/drm/i915/i915_gem_execbuffer.c +2
-0@@ -1272,6 +1272,8 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,i915_gem_execbuffer_move_to_active (vmas, params->request); +trace_i915_gem_ring_queue (ring, params); + qe =container_of (params,typeof (*qe), params); ret =i915_scheduler_queue_execbuffer (qe);if (ret)drivers/gpu/drm/i915/i915_scheduler.c +26
-0@@ -88,6 +88,8 @@ static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)/* Seqno will be reassigned on relaunch */ node->params.request->seqno =0 ; node->status = i915_sqs_queued; +trace_i915_scheduler_unfly (node->params.ring, node); +trace_i915_scheduler_node_state_change (node->params.ring, node); }/* @@ -99,7 +101,11 @@ static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)WARN_ON (!node);WARN_ON (I915_SQS_IS_COMPLETE (node)); +if (I915_SQS_IS_FLYING (node)) +trace_i915_scheduler_unfly (node->params.ring, node); + node->status = i915_sqs_dead; +trace_i915_scheduler_node_state_change (node->params.ring, node); }/* Mark a node as in flight on the hardware. */ @@ -124,6 +130,9 @@ static int i915_scheduler_node_fly(struct i915_scheduler_queue_entry *node) node->status = i915_sqs_flying; +trace_i915_scheduler_fly (ring, node); +trace_i915_scheduler_node_state_change (ring, node); +if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {bool success = true;@@ -280,6 +289,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,INIT_LIST_HEAD (&best->link); best->status = i915_sqs_popped; +trace_i915_scheduler_node_state_change (ring, best); + ret =0 ; }else {/* Can only get here if: @@ -297,6 +308,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring, } } +trace_i915_scheduler_pop_from_queue (ring, best); + *pop_node = best;return ret; }@@ -506,6 +519,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_enstruct i915_scheduler *scheduler = dev_priv->scheduler;int ret; +trace_i915_scheduler_queue (qe->params.ring, qe); +intel_ring_reserved_space_cancel (qe->params.request->ringbuf); scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;@@ -628,6 +643,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe) not_flying =i915_scheduler_count_flying (scheduler, ring) < scheduler->min_flying; +trace_i915_scheduler_queue (ring, node); +trace_i915_scheduler_node_state_change (ring, node); +spin_unlock_irq (&scheduler->lock);if (not_flying)@@ -657,6 +675,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)struct i915_scheduler_queue_entry *node = req->scheduler_qe;unsigned long flags; +trace_i915_scheduler_landing (req); +if (!node)return false;@@ -670,6 +690,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)else node->status = i915_sqs_complete; +trace_i915_scheduler_node_state_change (req->ring, node); +spin_unlock_irqrestore (&scheduler->lock, flags);return true;@@ -877,6 +899,8 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,/* Launch more packets now? */ do_submit = (queued >0 ) && (flying < scheduler->min_flying); +trace_i915_scheduler_remove (ring, min_seqno, do_submit); +spin_unlock_irq (&scheduler->lock);return do_submit;@@ -912,6 +936,8 @@ static void i915_scheduler_process_work(struct intel_engine_cs *ring) node =list_first_entry (&remove,typeof (*node), link);list_del (&node->link); +trace_i915_scheduler_destroy (ring, node); +/* Free up all the DRM references */ i915_scheduler_clean_node (node);drivers/gpu/drm/i915/i915_trace.h +196
-0@@ -9,6 +9,7 @@#include "i915_drv.h" #include "intel_drv.h" #include "intel_ringbuffer.h" +#include "i915_scheduler.h" #undef TRACE_SYSTEM#define TRACE_SYSTEM i915@@ -826,6 +827,201 @@ TRACE_EVENT(switch_mm, __entry->dev, __entry->ring, __entry->to, __entry->vm) ); +TRACE_EVENT (i915_scheduler_queue, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_scheduler_queue_entry *node), +TP_ARGS (ring, node), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) +__field (u32, seqno) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = node ? node->params.request->uniq : 0; + __entry->seqno = node ? node->params.request->seqno : 0; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d" , + __entry->ring, __entry->uniq, __entry->seqno) +); + +TRACE_EVENT (i915_scheduler_fly, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_scheduler_queue_entry *node), +TP_ARGS (ring, node), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) +__field (u32, seqno) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = node ? node->params.request->uniq : 0; + __entry->seqno = node ? node->params.request->seqno : 0; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d" , + __entry->ring, __entry->uniq, __entry->seqno) +); + +TRACE_EVENT (i915_scheduler_unfly, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_scheduler_queue_entry *node), +TP_ARGS (ring, node), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) +__field (u32, seqno) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = node ? node->params.request->uniq : 0; + __entry->seqno = node ? node->params.request->seqno : 0; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d" , + __entry->ring, __entry->uniq, __entry->seqno) +); + +TRACE_EVENT (i915_scheduler_landing, + TP_PROTO(struct drm_i915_gem_request *req), +TP_ARGS (req), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) + __field(u32, seqno) +__field (u32, status) + ), + +TP_fast_assign ( + __entry->ring = req->ring->id; + __entry->uniq = req->uniq; + __entry->seqno = req->seqno; + __entry->status = req->scheduler_qe ? + req->scheduler_qe->status : ~0U; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d, status=%d" , + __entry->ring, __entry->uniq, __entry->seqno, + __entry->status) +); + +TRACE_EVENT (i915_scheduler_remove, + TP_PROTO(struct intel_engine_cs *ring, + u32 min_seqno, bool do_submit), +TP_ARGS (ring, min_seqno, do_submit), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, min_seqno) +__field (bool, do_submit) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->min_seqno = min_seqno; + __entry->do_submit = do_submit; + ), + +TP_printk ("ring=%d, min_seqno = %d, do_submit=%d" , + __entry->ring, __entry->min_seqno, __entry->do_submit) +); + +TRACE_EVENT (i915_scheduler_destroy, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_scheduler_queue_entry *node), +TP_ARGS (ring, node), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) +__field (u32, seqno) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = node ? node->params.request->uniq : 0; + __entry->seqno = node ? node->params.request->seqno : 0; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d" , + __entry->ring, __entry->uniq, __entry->seqno) +); + +TRACE_EVENT (i915_scheduler_pop_from_queue, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_scheduler_queue_entry *node), +TP_ARGS (ring, node), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) +__field (u32, seqno) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = node ? node->params.request->uniq : 0; + __entry->seqno = node ? node->params.request->seqno : 0; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d" , + __entry->ring, __entry->uniq, __entry->seqno) +); + +TRACE_EVENT (i915_scheduler_node_state_change, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_scheduler_queue_entry *node), +TP_ARGS (ring, node), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) + __field(u32, seqno) +__field (u32, status) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = node ? node->params.request->uniq : 0; + __entry->seqno = node->params.request->seqno; + __entry->status = node->status; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d, status=%d" , + __entry->ring, __entry->uniq, __entry->seqno, + __entry->status) +); + +TRACE_EVENT (i915_gem_ring_queue, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_execbuffer_params *params), +TP_ARGS (ring, params), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) +__field (u32, seqno) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = params->request->uniq; + __entry->seqno = params->request->seqno; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d" , __entry->ring, + __entry->uniq, __entry->seqno) +); +#endif /* _I915_TRACE_H_ */ /* This part must be outside protection */ drivers/gpu/drm/i915/intel_lrc.c +0
-0
drivers/gpu/drm/i915/intel_lrc.c int intel_execlists_submission(struct i915_execbuffer_params *params, · line 961
Quoted 8 lines
drivers/gpu/drm/i915/intel_lrc.c +2
-0@@ -954,6 +954,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,i915_gem_execbuffer_move_to_active (vmas, params->request); +trace_i915_gem_ring_queue (ring, params); + qe =container_of (params,typeof (*qe), params); ret =i915_scheduler_queue_execbuffer (qe);if (ret)
Looks fine.
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 24/35] drm/i915: Added trace points to scheduler
In reply to [email protected]Quoted 361 lines · On 02/18/2016 06:27 AM, [email protected] wrote:
From: John Harrison <[email protected]>
Added trace points to the scheduler to track all the various events, node state transitions and other interesting things that occur.
v2: Updated for new request completion tracking implementation.
v3: Updated for changes to node kill code.
v4: Wrapped some long lines to keep the style checker happy.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>--- drivers/gpu/drm/i915/i915_gem_execbuffer.c | 2 + drivers/gpu/drm/i915/i915_scheduler.c | 26 ++++ drivers/gpu/drm/i915/i915_trace.h | 196 +++++++++++++++++++++++++++++ drivers/gpu/drm/i915/intel_lrc.c | 2 + 4 files changed, 226 insertions(+)drivers/gpu/drm/i915/i915_gem_execbuffer.c +2
-0@@ -1272,6 +1272,8 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,i915_gem_execbuffer_move_to_active (vmas, params->request); +trace_i915_gem_ring_queue (ring, params); + qe =container_of (params,typeof (*qe), params); ret =i915_scheduler_queue_execbuffer (qe);if (ret)drivers/gpu/drm/i915/i915_scheduler.c +26
-0@@ -88,6 +88,8 @@ static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)/* Seqno will be reassigned on relaunch */ node->params.request->seqno =0 ; node->status = i915_sqs_queued; +trace_i915_scheduler_unfly (node->params.ring, node); +trace_i915_scheduler_node_state_change (node->params.ring, node); }/* @@ -99,7 +101,11 @@ static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)WARN_ON (!node);WARN_ON (I915_SQS_IS_COMPLETE (node)); +if (I915_SQS_IS_FLYING (node)) +trace_i915_scheduler_unfly (node->params.ring, node); + node->status = i915_sqs_dead; +trace_i915_scheduler_node_state_change (node->params.ring, node); }/* Mark a node as in flight on the hardware. */ @@ -124,6 +130,9 @@ static int i915_scheduler_node_fly(struct i915_scheduler_queue_entry *node) node->status = i915_sqs_flying; +trace_i915_scheduler_fly (ring, node); +trace_i915_scheduler_node_state_change (ring, node); +if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {bool success = true;@@ -280,6 +289,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,INIT_LIST_HEAD (&best->link); best->status = i915_sqs_popped; +trace_i915_scheduler_node_state_change (ring, best); + ret =0 ; }else {/* Can only get here if: @@ -297,6 +308,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring, } } +trace_i915_scheduler_pop_from_queue (ring, best); + *pop_node = best;return ret; }@@ -506,6 +519,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_enstruct i915_scheduler *scheduler = dev_priv->scheduler;int ret; +trace_i915_scheduler_queue (qe->params.ring, qe); +intel_ring_reserved_space_cancel (qe->params.request->ringbuf); scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;@@ -628,6 +643,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe) not_flying =i915_scheduler_count_flying (scheduler, ring) < scheduler->min_flying; +trace_i915_scheduler_queue (ring, node); +trace_i915_scheduler_node_state_change (ring, node); +spin_unlock_irq (&scheduler->lock);if (not_flying)@@ -657,6 +675,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)struct i915_scheduler_queue_entry *node = req->scheduler_qe;unsigned long flags; +trace_i915_scheduler_landing (req); +if (!node)return false;@@ -670,6 +690,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)else node->status = i915_sqs_complete; +trace_i915_scheduler_node_state_change (req->ring, node); +spin_unlock_irqrestore (&scheduler->lock, flags);return true;@@ -877,6 +899,8 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,/* Launch more packets now? */ do_submit = (queued >0 ) && (flying < scheduler->min_flying); +trace_i915_scheduler_remove (ring, min_seqno, do_submit); +spin_unlock_irq (&scheduler->lock);return do_submit;@@ -912,6 +936,8 @@ static void i915_scheduler_process_work(struct intel_engine_cs *ring) node =list_first_entry (&remove,typeof (*node), link);list_del (&node->link); +trace_i915_scheduler_destroy (ring, node); +/* Free up all the DRM references */ i915_scheduler_clean_node (node);drivers/gpu/drm/i915/i915_trace.h +196
-0@@ -9,6 +9,7 @@#include "i915_drv.h" #include "intel_drv.h" #include "intel_ringbuffer.h" +#include "i915_scheduler.h" #undef TRACE_SYSTEM#define TRACE_SYSTEM i915@@ -826,6 +827,201 @@ TRACE_EVENT(switch_mm, __entry->dev, __entry->ring, __entry->to, __entry->vm) ); +TRACE_EVENT (i915_scheduler_queue, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_scheduler_queue_entry *node), +TP_ARGS (ring, node), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) +__field (u32, seqno) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = node ? node->params.request->uniq : 0; + __entry->seqno = node ? node->params.request->seqno : 0; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d" , + __entry->ring, __entry->uniq, __entry->seqno) +); + +TRACE_EVENT (i915_scheduler_fly, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_scheduler_queue_entry *node), +TP_ARGS (ring, node), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) +__field (u32, seqno) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = node ? node->params.request->uniq : 0; + __entry->seqno = node ? node->params.request->seqno : 0; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d" , + __entry->ring, __entry->uniq, __entry->seqno) +); + +TRACE_EVENT (i915_scheduler_unfly, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_scheduler_queue_entry *node), +TP_ARGS (ring, node), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) +__field (u32, seqno) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = node ? node->params.request->uniq : 0; + __entry->seqno = node ? node->params.request->seqno : 0; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d" , + __entry->ring, __entry->uniq, __entry->seqno) +); + +TRACE_EVENT (i915_scheduler_landing, + TP_PROTO(struct drm_i915_gem_request *req), +TP_ARGS (req), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) + __field(u32, seqno) +__field (u32, status) + ), + +TP_fast_assign ( + __entry->ring = req->ring->id; + __entry->uniq = req->uniq; + __entry->seqno = req->seqno; + __entry->status = req->scheduler_qe ? + req->scheduler_qe->status : ~0U; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d, status=%d" , + __entry->ring, __entry->uniq, __entry->seqno, + __entry->status) +); + +TRACE_EVENT (i915_scheduler_remove, + TP_PROTO(struct intel_engine_cs *ring, + u32 min_seqno, bool do_submit), +TP_ARGS (ring, min_seqno, do_submit), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, min_seqno) +__field (bool, do_submit) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->min_seqno = min_seqno; + __entry->do_submit = do_submit; + ), + +TP_printk ("ring=%d, min_seqno = %d, do_submit=%d" , + __entry->ring, __entry->min_seqno, __entry->do_submit) +); + +TRACE_EVENT (i915_scheduler_destroy, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_scheduler_queue_entry *node), +TP_ARGS (ring, node), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) +__field (u32, seqno) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = node ? node->params.request->uniq : 0; + __entry->seqno = node ? node->params.request->seqno : 0; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d" , + __entry->ring, __entry->uniq, __entry->seqno) +); + +TRACE_EVENT (i915_scheduler_pop_from_queue, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_scheduler_queue_entry *node), +TP_ARGS (ring, node), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) +__field (u32, seqno) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = node ? node->params.request->uniq : 0; + __entry->seqno = node ? node->params.request->seqno : 0; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d" , + __entry->ring, __entry->uniq, __entry->seqno) +); + +TRACE_EVENT (i915_scheduler_node_state_change, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_scheduler_queue_entry *node), +TP_ARGS (ring, node), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) + __field(u32, seqno) +__field (u32, status) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = node ? node->params.request->uniq : 0; + __entry->seqno = node->params.request->seqno; + __entry->status = node->status; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d, status=%d" , + __entry->ring, __entry->uniq, __entry->seqno, + __entry->status) +); + +TRACE_EVENT (i915_gem_ring_queue, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_execbuffer_params *params), +TP_ARGS (ring, params), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) +__field (u32, seqno) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = params->request->uniq; + __entry->seqno = params->request->seqno; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d" , __entry->ring, + __entry->uniq, __entry->seqno) +); +#endif /* _I915_TRACE_H_ */ /* This part must be outside protection */ drivers/gpu/drm/i915/intel_lrc.c +0
-0
drivers/gpu/drm/i915/intel_lrc.c int intel_execlists_submission(struct i915_execbuffer_params *params, · line 961
Quoted 8 lines
drivers/gpu/drm/i915/intel_lrc.c +2
-0@@ -954,6 +954,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,i915_gem_execbuffer_move_to_active (vmas, params->request); +trace_i915_gem_ring_queue (ring, params); + qe =container_of (params,typeof (*qe), params); ret =i915_scheduler_queue_execbuffer (qe);if (ret)
Oops, forgot to ask if there are any tools for igt that make use of these, e.g. some kind of scheduler top util that looks for bubbles or starved batches or something.
Jesse
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 24/35] drm/i915: Added trace points to scheduler
Quoted 364 lines · Jesse Barnes
Quoted 363 lines · On 02/18/2016 06:27 AM, [email protected] wrote:
From: John Harrison <[email protected]>
Added trace points to the scheduler to track all the various events, node state transitions and other interesting things that occur.
v2: Updated for new request completion tracking implementation.
v3: Updated for changes to node kill code.
v4: Wrapped some long lines to keep the style checker happy.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>--- drivers/gpu/drm/i915/i915_gem_execbuffer.c | 2 + drivers/gpu/drm/i915/i915_scheduler.c | 26 ++++ drivers/gpu/drm/i915/i915_trace.h | 196 +++++++++++++++++++++++++++++ drivers/gpu/drm/i915/intel_lrc.c | 2 + 4 files changed, 226 insertions(+)drivers/gpu/drm/i915/i915_gem_execbuffer.c +2
-0@@ -1272,6 +1272,8 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,i915_gem_execbuffer_move_to_active (vmas, params->request); +trace_i915_gem_ring_queue (ring, params); + qe =container_of (params,typeof (*qe), params); ret =i915_scheduler_queue_execbuffer (qe);if (ret)drivers/gpu/drm/i915/i915_scheduler.c +26
-0@@ -88,6 +88,8 @@ static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)/* Seqno will be reassigned on relaunch */ node->params.request->seqno =0 ; node->status = i915_sqs_queued; +trace_i915_scheduler_unfly (node->params.ring, node); +trace_i915_scheduler_node_state_change (node->params.ring, node); }/* @@ -99,7 +101,11 @@ static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)WARN_ON (!node);WARN_ON (I915_SQS_IS_COMPLETE (node)); +if (I915_SQS_IS_FLYING (node)) +trace_i915_scheduler_unfly (node->params.ring, node); + node->status = i915_sqs_dead; +trace_i915_scheduler_node_state_change (node->params.ring, node); }/* Mark a node as in flight on the hardware. */ @@ -124,6 +130,9 @@ static int i915_scheduler_node_fly(struct i915_scheduler_queue_entry *node) node->status = i915_sqs_flying; +trace_i915_scheduler_fly (ring, node); +trace_i915_scheduler_node_state_change (ring, node); +if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {bool success = true;@@ -280,6 +289,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,INIT_LIST_HEAD (&best->link); best->status = i915_sqs_popped; +trace_i915_scheduler_node_state_change (ring, best); + ret =0 ; }else {/* Can only get here if: @@ -297,6 +308,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring, } } +trace_i915_scheduler_pop_from_queue (ring, best); + *pop_node = best;return ret; }@@ -506,6 +519,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_enstruct i915_scheduler *scheduler = dev_priv->scheduler;int ret; +trace_i915_scheduler_queue (qe->params.ring, qe); +intel_ring_reserved_space_cancel (qe->params.request->ringbuf); scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;@@ -628,6 +643,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe) not_flying =i915_scheduler_count_flying (scheduler, ring) < scheduler->min_flying; +trace_i915_scheduler_queue (ring, node); +trace_i915_scheduler_node_state_change (ring, node); +spin_unlock_irq (&scheduler->lock);if (not_flying)@@ -657,6 +675,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)struct i915_scheduler_queue_entry *node = req->scheduler_qe;unsigned long flags; +trace_i915_scheduler_landing (req); +if (!node)return false;@@ -670,6 +690,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)else node->status = i915_sqs_complete; +trace_i915_scheduler_node_state_change (req->ring, node); +spin_unlock_irqrestore (&scheduler->lock, flags);return true;@@ -877,6 +899,8 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,/* Launch more packets now? */ do_submit = (queued >0 ) && (flying < scheduler->min_flying); +trace_i915_scheduler_remove (ring, min_seqno, do_submit); +spin_unlock_irq (&scheduler->lock);return do_submit;@@ -912,6 +936,8 @@ static void i915_scheduler_process_work(struct intel_engine_cs *ring) node =list_first_entry (&remove,typeof (*node), link);list_del (&node->link); +trace_i915_scheduler_destroy (ring, node); +/* Free up all the DRM references */ i915_scheduler_clean_node (node);drivers/gpu/drm/i915/i915_trace.h +196
-0@@ -9,6 +9,7 @@#include "i915_drv.h" #include "intel_drv.h" #include "intel_ringbuffer.h" +#include "i915_scheduler.h" #undef TRACE_SYSTEM#define TRACE_SYSTEM i915@@ -826,6 +827,201 @@ TRACE_EVENT(switch_mm, __entry->dev, __entry->ring, __entry->to, __entry->vm) ); +TRACE_EVENT (i915_scheduler_queue, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_scheduler_queue_entry *node), +TP_ARGS (ring, node), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) +__field (u32, seqno) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = node ? node->params.request->uniq : 0; + __entry->seqno = node ? node->params.request->seqno : 0; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d" , + __entry->ring, __entry->uniq, __entry->seqno) +); + +TRACE_EVENT (i915_scheduler_fly, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_scheduler_queue_entry *node), +TP_ARGS (ring, node), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) +__field (u32, seqno) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = node ? node->params.request->uniq : 0; + __entry->seqno = node ? node->params.request->seqno : 0; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d" , + __entry->ring, __entry->uniq, __entry->seqno) +); + +TRACE_EVENT (i915_scheduler_unfly, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_scheduler_queue_entry *node), +TP_ARGS (ring, node), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) +__field (u32, seqno) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = node ? node->params.request->uniq : 0; + __entry->seqno = node ? node->params.request->seqno : 0; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d" , + __entry->ring, __entry->uniq, __entry->seqno) +); + +TRACE_EVENT (i915_scheduler_landing, + TP_PROTO(struct drm_i915_gem_request *req), +TP_ARGS (req), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) + __field(u32, seqno) +__field (u32, status) + ), + +TP_fast_assign ( + __entry->ring = req->ring->id; + __entry->uniq = req->uniq; + __entry->seqno = req->seqno; + __entry->status = req->scheduler_qe ? + req->scheduler_qe->status : ~0U; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d, status=%d" , + __entry->ring, __entry->uniq, __entry->seqno, + __entry->status) +); + +TRACE_EVENT (i915_scheduler_remove, + TP_PROTO(struct intel_engine_cs *ring, + u32 min_seqno, bool do_submit), +TP_ARGS (ring, min_seqno, do_submit), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, min_seqno) +__field (bool, do_submit) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->min_seqno = min_seqno; + __entry->do_submit = do_submit; + ), + +TP_printk ("ring=%d, min_seqno = %d, do_submit=%d" , + __entry->ring, __entry->min_seqno, __entry->do_submit) +); + +TRACE_EVENT (i915_scheduler_destroy, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_scheduler_queue_entry *node), +TP_ARGS (ring, node), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) +__field (u32, seqno) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = node ? node->params.request->uniq : 0; + __entry->seqno = node ? node->params.request->seqno : 0; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d" , + __entry->ring, __entry->uniq, __entry->seqno) +); + +TRACE_EVENT (i915_scheduler_pop_from_queue, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_scheduler_queue_entry *node), +TP_ARGS (ring, node), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) +__field (u32, seqno) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = node ? node->params.request->uniq : 0; + __entry->seqno = node ? node->params.request->seqno : 0; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d" , + __entry->ring, __entry->uniq, __entry->seqno) +); + +TRACE_EVENT (i915_scheduler_node_state_change, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_scheduler_queue_entry *node), +TP_ARGS (ring, node), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) + __field(u32, seqno) +__field (u32, status) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = node ? node->params.request->uniq : 0; + __entry->seqno = node->params.request->seqno; + __entry->status = node->status; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d, status=%d" , + __entry->ring, __entry->uniq, __entry->seqno, + __entry->status) +); + +TRACE_EVENT (i915_gem_ring_queue, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_execbuffer_params *params), +TP_ARGS (ring, params), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) +__field (u32, seqno) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = params->request->uniq; + __entry->seqno = params->request->seqno; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d" , __entry->ring, + __entry->uniq, __entry->seqno) +); +#endif /* _I915_TRACE_H_ */ /* This part must be outside protection */ drivers/gpu/drm/i915/intel_lrc.c +0
-0@@ -954,6 +954,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,i915_gem_execbuffer_move_to_active (vmas, params->request);
Quoted 7 lines
drivers/gpu/drm/i915/intel_lrc.c int intel_execlists_submission(struct i915_execbuffer_params *params, · line 961
Quoted 6 lines
drivers/gpu/drm/i915/intel_lrc.c +2
-0+trace_i915_gem_ring_queue (ring, params); + qe =container_of (params,typeof (*qe), params); ret =i915_scheduler_queue_execbuffer (qe);if (ret)Oops, forgot to ask if there are any tools for igt that make use of these, e.g. some kind of scheduler top util that looks for bubbles or starved batches or something.
Nothing at the moment. They have been used to debug issues by looking through trace logs. Also the validation team have tests that hook in to the various trace points to verify the internal operation of the scheduler and other parts of the i915 driver.
Quoted 2 lines
Jesse
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 24/35] drm/i915: Added trace points to scheduler
Quoted 345 lines · John Harrison
Quoted 336 lines · Jesse Barnes
Quoted 335 lines · On 02/18/2016 06:27 AM, [email protected] wrote:
From: John Harrison <[email protected]>
Added trace points to the scheduler to track all the various events, node state transitions and other interesting things that occur.
v2: Updated for new request completion tracking implementation.
v3: Updated for changes to node kill code.
v4: Wrapped some long lines to keep the style checker happy.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>--- drivers/gpu/drm/i915/i915_gem_execbuffer.c | 2 + drivers/gpu/drm/i915/i915_scheduler.c | 26 ++++ drivers/gpu/drm/i915/i915_trace.h | 196 +++++++++++++++++++++++++++++ drivers/gpu/drm/i915/intel_lrc.c | 2 + 4 files changed, 226 insertions(+)drivers/gpu/drm/i915/i915_gem_execbuffer.c +1
-0@@ -1272,6 +1272,8 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,i915_gem_execbuffer_move_to_active (vmas, params->request); +trace_i915_gem_ring_queue (ring, params); + qe =container_of (params,typeof (*qe), params); ret =i915_scheduler_queue_execbuffer (qe);if (ret)drivers/gpu/drm/i915/i915_scheduler.c +16
-0@@ -88,6 +88,8 @@ static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)/* Seqno will be reassigned on relaunch */ node->params.request->seqno =0 ; node->status = i915_sqs_queued; +trace_i915_scheduler_unfly (node->params.ring, node); +trace_i915_scheduler_node_state_change (node->params.ring, node); }/* @@ -99,7 +101,11 @@ static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)WARN_ON (!node);WARN_ON (I915_SQS_IS_COMPLETE (node)); + if (I915_SQS_IS_FLYING(node)) +trace_i915_scheduler_unfly (node->params.ring, node); + node->status = i915_sqs_dead; +trace_i915_scheduler_node_state_change (node->params.ring, node); }/* Mark a node as in flight on the hardware. */ @@ -124,6 +130,9 @@ static int i915_scheduler_node_fly(struct i915_scheduler_queue_entry *node) node->status = i915_sqs_flying; +trace_i915_scheduler_fly (ring, node); +trace_i915_scheduler_node_state_change (ring, node); +if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {bool success = true; @@ -280,6 +289,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring, INIT_LIST_HEAD(&best->link); best->status = i915_sqs_popped; +trace_i915_scheduler_node_state_change (ring, best); + ret =0 ; }else {/* Can only get here if: @@ -297,6 +308,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring, } } +trace_i915_scheduler_pop_from_queue (ring, best); + *pop_node = best;return ret; }@@ -506,6 +519,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_enstruct i915_scheduler *scheduler = dev_priv->scheduler;int ret; +trace_i915_scheduler_queue (qe->params.ring, qe); +intel_ring_reserved_space_cancel (qe->params.request->ringbuf); scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;@@ -628,6 +643,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe) not_flying =i915_scheduler_count_flying (scheduler, ring) < scheduler->min_flying; +trace_i915_scheduler_queue (ring, node); +trace_i915_scheduler_node_state_change (ring, node); +spin_unlock_irq (&scheduler->lock);if (not_flying)@@ -657,6 +675,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)struct i915_scheduler_queue_entry *node = req->scheduler_qe;unsigned long flags; +trace_i915_scheduler_landing (req); +if (!node)return false; @@ -670,6 +690,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req) else node->status = i915_sqs_complete; +trace_i915_scheduler_node_state_change (req->ring, node); +spin_unlock_irqrestore (&scheduler->lock, flags);return true;@@ -877,6 +899,8 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,/* Launch more packets now? */ do_submit = (queued >0 ) && (flying < scheduler->min_flying); +trace_i915_scheduler_remove (ring, min_seqno, do_submit); +spin_unlock_irq (&scheduler->lock);return do_submit;@@ -912,6 +936,8 @@ static void i915_scheduler_process_work(struct intel_engine_cs *ring) node =list_first_entry (&remove,typeof (*node), link);list_del (&node->link); +trace_i915_scheduler_destroy (ring, node); +/* Free up all the DRM references */ i915_scheduler_clean_node (node); diff --git a/drivers/gpu/drm/i915/i915_trace.h b/drivers/gpu/drm/i915/i915_trace.hindex 455c215..c3c4e58 100644
drivers/gpu/drm/i915/i915_trace.h +195
-0@@ -9,6 +9,7 @@#include "i915_drv.h" #include "intel_drv.h" #include "intel_ringbuffer.h" +#include "i915_scheduler.h" #undef TRACE_SYSTEM#define TRACE_SYSTEM i915@@ -826,6 +827,201 @@ TRACE_EVENT(switch_mm, __entry->dev, __entry->ring, __entry->to, __entry->vm) ); +TRACE_EVENT (i915_scheduler_queue, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_scheduler_queue_entry *node), +TP_ARGS (ring, node), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) +__field (u32, seqno) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = node ? node->params.request->uniq : 0; + __entry->seqno = node ? node->params.request->seqno : 0; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d" , + __entry->ring, __entry->uniq, __entry->seqno) +); + +TRACE_EVENT (i915_scheduler_fly, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_scheduler_queue_entry *node), +TP_ARGS (ring, node), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) +__field (u32, seqno) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = node ? node->params.request->uniq : 0; + __entry->seqno = node ? node->params.request->seqno : 0; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d" , + __entry->ring, __entry->uniq, __entry->seqno) +); + +TRACE_EVENT (i915_scheduler_unfly, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_scheduler_queue_entry *node), +TP_ARGS (ring, node), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) +__field (u32, seqno) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = node ? node->params.request->uniq : 0; + __entry->seqno = node ? node->params.request->seqno : 0; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d" , + __entry->ring, __entry->uniq, __entry->seqno) +); + +TRACE_EVENT (i915_scheduler_landing, + TP_PROTO(struct drm_i915_gem_request *req), +TP_ARGS (req), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) + __field(u32, seqno) +__field (u32, status) + ), + +TP_fast_assign ( + __entry->ring = req->ring->id; + __entry->uniq = req->uniq; + __entry->seqno = req->seqno; + __entry->status = req->scheduler_qe ? + req->scheduler_qe->status : ~0U; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d, status=%d" , + __entry->ring, __entry->uniq, __entry->seqno, + __entry->status) +); + +TRACE_EVENT (i915_scheduler_remove, + TP_PROTO(struct intel_engine_cs *ring, + u32 min_seqno, bool do_submit), +TP_ARGS (ring, min_seqno, do_submit), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, min_seqno) +__field (bool, do_submit) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->min_seqno = min_seqno; + __entry->do_submit = do_submit; + ), + +TP_printk ("ring=%d, min_seqno = %d, do_submit=%d" , + __entry->ring, __entry->min_seqno, __entry->do_submit) +); + +TRACE_EVENT (i915_scheduler_destroy, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_scheduler_queue_entry *node), +TP_ARGS (ring, node), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) +__field (u32, seqno) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = node ? node->params.request->uniq : 0; + __entry->seqno = node ? node->params.request->seqno : 0; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d" , + __entry->ring, __entry->uniq, __entry->seqno) +); + +TRACE_EVENT (i915_scheduler_pop_from_queue, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_scheduler_queue_entry *node), +TP_ARGS (ring, node), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) +__field (u32, seqno) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = node ? node->params.request->uniq : 0; + __entry->seqno = node ? node->params.request->seqno : 0; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d" , + __entry->ring, __entry->uniq, __entry->seqno) +); + +TRACE_EVENT (i915_scheduler_node_state_change, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_scheduler_queue_entry *node), +TP_ARGS (ring, node), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) + __field(u32, seqno) +__field (u32, status) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = node ? node->params.request->uniq : 0; + __entry->seqno = node->params.request->seqno; + __entry->status = node->status; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d, status=%d" , + __entry->ring, __entry->uniq, __entry->seqno, + __entry->status) +); + +TRACE_EVENT (i915_gem_ring_queue, + TP_PROTO(struct intel_engine_cs *ring, + struct i915_execbuffer_params *params), +TP_ARGS (ring, params), + +TP_STRUCT__entry ( + __field(u32, ring) + __field(u32, uniq) +__field (u32, seqno) + ), + +TP_fast_assign ( + __entry->ring = ring->id; + __entry->uniq = params->request->uniq; + __entry->seqno = params->request->seqno; + ), + +TP_printk ("ring=%d, uniq=%d, seqno=%d" , __entry->ring, + __entry->uniq, __entry->seqno) +); +#endif /* _I915_TRACE_H_ */ /* This part must be outside protection */ drivers/gpu/drm/i915/intel_lrc.c +0
-0@@ -954,6 +954,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,Quoted 8 lines
drivers/gpu/drm/i915/intel_lrc.c int intel_execlists_submission(struct i915_execbuffer_params *params, · line 960
Quoted 7 lines
drivers/gpu/drm/i915/intel_lrc.c +1
-0i915_gem_execbuffer_move_to_active (vmas, params->request); +trace_i915_gem_ring_queue (ring, params); + qe =container_of (params,typeof (*qe), params); ret =i915_scheduler_queue_execbuffer (qe);if (ret)Oops, forgot to ask if there are any tools for igt that make use of these, e.g. some kind of scheduler top util that looks for bubbles or starved batches or something.
Quoted 1 line
Nothing at the moment. They have been used to debug issues by looking through trace logs. Also the validation team have tests that hook in to the various trace points to verify the internal operation of the scheduler and other parts of the i915 driver.
Nice. I wasn't asking because it's a blocker or anything, just curious.
Thanks,
Jesse
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 25/35 drm/i915: Added scheduler queue throttling by DRM file handle
In reply to [email protected]From: John Harrison <[email protected]>
The scheduler decouples the submission of batch buffers to the driver from their subsequent submission to the hardware. This means that an application which is continuously submitting buffers as fast as it can could potentialy flood the driver. To prevent this, the driver now tracks how many buffers are in progress (queued in software or executing in hardware) and limits this to a given (tunable) number. If this number is exceeded then the queue to the driver will return EAGAIN and thus prevent the scheduler's queue becoming arbitrarily large.
v3: Added a missing decrement of the file queue counter.
v4: Updated a comment.
v5: Updated due to changes to earlier patches in series - removing forward declarations and white space. Also added some documentation. [Joonas Lahtinen]
For: VIZ-1587
--- drivers/gpu/drm/i915/i915_drv.h | 2 ++ drivers/gpu/drm/i915/i915_gem_execbuffer.c | 8 +++++ drivers/gpu/drm/i915/i915_scheduler.c | 48 ++++++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_scheduler.h | 2 ++ 4 files changed, 60 insertions(+)
drivers/gpu/drm/i915/i915_drv.h +2 -0
@@ -336,6 +336,8 @@ struct drm_i915_file_private { } rps;struct intel_engine_cs *bsd_ring; + +u32 scheduler_queue_length; };enum intel_dpll_id {
drivers/gpu/drm/i915/i915_gem_execbuffer.c +8 -0
@@ -1803,6 +1803,10 @@ i915_gem_execbuffer(struct drm_device *dev, void *data,return -EINVAL ; } +/* Throttle batch requests per device file */ +if (i915_scheduler_file_queue_is_full (file)) +return -EAGAIN ; +/* Copy in the exec list from userland */ exec_list =drm_malloc_ab (sizeof (*exec_list), args->buffer_count); exec2_list =drm_malloc_ab (sizeof (*exec2_list), args->buffer_count);
@@ -1893,6 +1897,10 @@ i915_gem_execbuffer2(struct drm_device *dev, void *data,return -EINVAL ; } +/* Throttle batch requests per device file */ +if (i915_scheduler_file_queue_is_full (file)) +return -EAGAIN ; + exec2_list =kmalloc (sizeof (*exec2_list)*args->buffer_count,GFP_TEMPORARY | __GFP_NOWARN | __GFP_NORETRY);if (exec2_list ==NULL )
drivers/gpu/drm/i915/i915_scheduler.c +48 -0
78 lines not on this page · raw message
drivers/gpu/drm/i915/i915_scheduler.h +2 -0
@@ -77,6 +77,7 @@ struct i915_scheduler {int32_t priority_level_bump;int32_t priority_level_preempt;uint32_t min_flying; +uint32_t file_queue_max; };/* Flag bits for i915_scheduler::flags */
@@ -100,5 +101,6 @@ int i915_scheduler_flush_stamp(struct intel_engine_cs *ring, unsigned long stamp, bool is_locked);bool i915_scheduler_is_request_tracked (struct drm_i915_gem_request *req,bool *completed,bool *busy); +bool i915_scheduler_file_queue_is_full (struct drm_file *file);#endif /* _I915_SCHEDULER_H_ */
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 25/35] drm/i915: Added scheduler queue throttling by DRM file handle
Quoted 169 lines · On 02/18/2016 06:27 AM, [email protected] wrote:
From: John Harrison <[email protected]>
The scheduler decouples the submission of batch buffers to the driver from their subsequent submission to the hardware. This means that an application which is continuously submitting buffers as fast as it can could potentialy flood the driver. To prevent this, the driver now tracks how many buffers are in progress (queued in software or executing in hardware) and limits this to a given (tunable) number. If this number is exceeded then the queue to the driver will return EAGAIN and thus prevent the scheduler's queue becoming arbitrarily large.
v3: Added a missing decrement of the file queue counter.
v4: Updated a comment.
v5: Updated due to changes to earlier patches in series - removing forward declarations and white space. Also added some documentation. [Joonas Lahtinen]
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>--- drivers/gpu/drm/i915/i915_drv.h | 2 ++ drivers/gpu/drm/i915/i915_gem_execbuffer.c | 8 +++++ drivers/gpu/drm/i915/i915_scheduler.c | 48 ++++++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_scheduler.h | 2 ++ 4 files changed, 60 insertions(+)drivers/gpu/drm/i915/i915_drv.h +2
-0@@ -336,6 +336,8 @@ struct drm_i915_file_private { } rps;struct intel_engine_cs *bsd_ring; + +u32 scheduler_queue_length; };enum intel_dpll_id {drivers/gpu/drm/i915/i915_gem_execbuffer.c +8
-0@@ -1803,6 +1803,10 @@ i915_gem_execbuffer(struct drm_device *dev, void *data,return -EINVAL ; } +/* Throttle batch requests per device file */ +if (i915_scheduler_file_queue_is_full (file)) +return -EAGAIN ; +/* Copy in the exec list from userland */ exec_list =drm_malloc_ab (sizeof (*exec_list), args->buffer_count); exec2_list =drm_malloc_ab (sizeof (*exec2_list), args->buffer_count);@@ -1893,6 +1897,10 @@ i915_gem_execbuffer2(struct drm_device *dev, void *data,return -EINVAL ; } +/* Throttle batch requests per device file */ +if (i915_scheduler_file_queue_is_full (file)) +return -EAGAIN ; + exec2_list =kmalloc (sizeof (*exec2_list)*args->buffer_count,GFP_TEMPORARY | __GFP_NOWARN | __GFP_NORETRY);if (exec2_list ==NULL )drivers/gpu/drm/i915/i915_scheduler.c +48
-0@@ -69,6 +69,7 @@ int i915_scheduler_init(struct drm_device *dev) scheduler->priority_level_bump =50 ; scheduler->priority_level_preempt =900 ; scheduler->min_flying =2 ; + scheduler->file_queue_max =64 ; dev_priv->scheduler = scheduler;@@ -464,6 +465,44 @@ static int i915_scheduler_submit_unlocked(struct intel_engine_cs *ring)return ret; } +/** +* i915_scheduler_file_queue_is_full - Returns true if the queue is full. +* @file: File object to query. +* This allows throttling of applications by limiting the total number of +* outstanding requests to a specified level. Once that limit is reached, +* this call will return true and no more requests should be accepted. +*/ +bool i915_scheduler_file_queue_is_full (struct drm_file *file) +{ +struct drm_i915_file_private *file_priv = file->driver_priv; +struct drm_i915_private *dev_priv = file_priv->dev_priv; +struct i915_scheduler *scheduler = dev_priv->scheduler; + +return file_priv->scheduler_queue_length >= scheduler->file_queue_max; +} + +/** +* i915_scheduler_file_queue_inc - Increment the file's request queue count. +* @file: File object to process. +*/ +static void i915_scheduler_file_queue_inc (struct drm_file *file) +{ +struct drm_i915_file_private *file_priv = file->driver_priv; + + file_priv->scheduler_queue_length++; +} + +/** +* i915_scheduler_file_queue_dec - Decrement the file's request queue count. +* @file: File object to process. +*/ +static void i915_scheduler_file_queue_dec (struct drm_file *file) +{ +struct drm_i915_file_private *file_priv = file->driver_priv; + + file_priv->scheduler_queue_length--; +} +static void i915_generate_dependencies (struct i915_scheduler *scheduler,struct i915_scheduler_queue_entry *node,uint32_t ring)@@ -640,6 +679,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)list_add_tail (&node->link, &scheduler->node_queue[ring->id]); +i915_scheduler_file_queue_inc (node->params.file); + not_flying =i915_scheduler_count_flying (scheduler, ring) < scheduler->min_flying;@@ -883,6 +924,12 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,/* Strip the dependency info while the mutex is still locked */ i915_scheduler_remove_dependent (scheduler, node); +/* Likewise clean up the file pointer. */ +if (node->params.file) { +i915_scheduler_file_queue_dec (node->params.file); + node->params.file =NULL ; + } +continue ; }@@ -1205,6 +1252,7 @@ int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file) node->status, ring->name); +i915_scheduler_file_queue_dec (node->params.file); node->params.file =NULL ; } }drivers/gpu/drm/i915/i915_scheduler.h +1
-0@@ -77,6 +77,7 @@ struct i915_scheduler {int32_t priority_level_bump;int32_t priority_level_preempt;uint32_t min_flying; +uint32_t file_queue_max; };/* Flag bits for i915_scheduler::flags */
drivers/gpu/drm/i915/i915_scheduler.h int i915_scheduler_flush_stamp(struct intel_engine_cs *ring, · line 106
Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.h +1
-0@@ -100,5 +101,6 @@ int i915_scheduler_flush_stamp(struct intel_engine_cs *ring, unsigned long stamp, bool is_locked);bool i915_scheduler_is_request_tracked (struct drm_i915_gem_request *req,bool *completed,bool *busy); +bool i915_scheduler_file_queue_is_full (struct drm_file *file);#endif /* _I915_SCHEDULER_H_ */
Just to clarify and make sure I understood the previous stuff: a queued execbuf that has not yet been dispatched does not reserve and pin pages right? That occurs at actual dispatch time? If so, I guess clients will hit this 64 queued item limit pretty regularly... How much metadata overhead does that involve? Has it been derived from some performance work with a bunch of workloads? It's fine if not, I can imagine that different mixes of workloads would be affected by lower or higher queue depths (e.g. small batch tests).
If this is tunable, I guess it should be clamped like a nice or rlimit value, with values outside that range requiring CAP_SYS_ADMIN.
Jesse
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 25/35] drm/i915: Added scheduler queue throttling by DRM file handle
Quoted 174 lines · Jesse Barnes
Quoted 173 lines · On 02/18/2016 06:27 AM, [email protected] wrote:
From: John Harrison <[email protected]>
The scheduler decouples the submission of batch buffers to the driver from their subsequent submission to the hardware. This means that an application which is continuously submitting buffers as fast as it can could potentialy flood the driver. To prevent this, the driver now tracks how many buffers are in progress (queued in software or executing in hardware) and limits this to a given (tunable) number. If this number is exceeded then the queue to the driver will return EAGAIN and thus prevent the scheduler's queue becoming arbitrarily large.
v3: Added a missing decrement of the file queue counter.
v4: Updated a comment.
v5: Updated due to changes to earlier patches in series - removing forward declarations and white space. Also added some documentation. [Joonas Lahtinen]
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>--- drivers/gpu/drm/i915/i915_drv.h | 2 ++ drivers/gpu/drm/i915/i915_gem_execbuffer.c | 8 +++++ drivers/gpu/drm/i915/i915_scheduler.c | 48 ++++++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_scheduler.h | 2 ++ 4 files changed, 60 insertions(+)drivers/gpu/drm/i915/i915_drv.h +2
-0@@ -336,6 +336,8 @@ struct drm_i915_file_private { } rps;struct intel_engine_cs *bsd_ring; + +u32 scheduler_queue_length; };enum intel_dpll_id {drivers/gpu/drm/i915/i915_gem_execbuffer.c +8
-0@@ -1803,6 +1803,10 @@ i915_gem_execbuffer(struct drm_device *dev, void *data,return -EINVAL ; } +/* Throttle batch requests per device file */ +if (i915_scheduler_file_queue_is_full (file)) +return -EAGAIN ; +/* Copy in the exec list from userland */ exec_list =drm_malloc_ab (sizeof (*exec_list), args->buffer_count); exec2_list =drm_malloc_ab (sizeof (*exec2_list), args->buffer_count);@@ -1893,6 +1897,10 @@ i915_gem_execbuffer2(struct drm_device *dev, void *data,return -EINVAL ; } +/* Throttle batch requests per device file */ +if (i915_scheduler_file_queue_is_full (file)) +return -EAGAIN ; + exec2_list =kmalloc (sizeof (*exec2_list)*args->buffer_count,GFP_TEMPORARY | __GFP_NOWARN | __GFP_NORETRY);if (exec2_list ==NULL )drivers/gpu/drm/i915/i915_scheduler.c +48
-0@@ -69,6 +69,7 @@ int i915_scheduler_init(struct drm_device *dev) scheduler->priority_level_bump =50 ; scheduler->priority_level_preempt =900 ; scheduler->min_flying =2 ; + scheduler->file_queue_max =64 ; dev_priv->scheduler = scheduler;@@ -464,6 +465,44 @@ static int i915_scheduler_submit_unlocked(struct intel_engine_cs *ring)return ret; } +/** +* i915_scheduler_file_queue_is_full - Returns true if the queue is full. +* @file: File object to query. +* This allows throttling of applications by limiting the total number of +* outstanding requests to a specified level. Once that limit is reached, +* this call will return true and no more requests should be accepted. +*/ +bool i915_scheduler_file_queue_is_full (struct drm_file *file) +{ +struct drm_i915_file_private *file_priv = file->driver_priv; +struct drm_i915_private *dev_priv = file_priv->dev_priv; +struct i915_scheduler *scheduler = dev_priv->scheduler; + +return file_priv->scheduler_queue_length >= scheduler->file_queue_max; +} + +/** +* i915_scheduler_file_queue_inc - Increment the file's request queue count. +* @file: File object to process. +*/ +static void i915_scheduler_file_queue_inc (struct drm_file *file) +{ +struct drm_i915_file_private *file_priv = file->driver_priv; + + file_priv->scheduler_queue_length++; +} + +/** +* i915_scheduler_file_queue_dec - Decrement the file's request queue count. +* @file: File object to process. +*/ +static void i915_scheduler_file_queue_dec (struct drm_file *file) +{ +struct drm_i915_file_private *file_priv = file->driver_priv; + + file_priv->scheduler_queue_length--; +} +static void i915_generate_dependencies (struct i915_scheduler *scheduler,struct i915_scheduler_queue_entry *node,uint32_t ring)@@ -640,6 +679,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)list_add_tail (&node->link, &scheduler->node_queue[ring->id]); +i915_scheduler_file_queue_inc (node->params.file); + not_flying =i915_scheduler_count_flying (scheduler, ring) < scheduler->min_flying;@@ -883,6 +924,12 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,/* Strip the dependency info while the mutex is still locked */ i915_scheduler_remove_dependent (scheduler, node); +/* Likewise clean up the file pointer. */ +if (node->params.file) { +i915_scheduler_file_queue_dec (node->params.file); + node->params.file =NULL ; + } +continue ; }@@ -1205,6 +1252,7 @@ int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file) node->status, ring->name); +i915_scheduler_file_queue_dec (node->params.file); node->params.file =NULL ; } }drivers/gpu/drm/i915/i915_scheduler.h +2
-0@@ -77,6 +77,7 @@ struct i915_scheduler {int32_t priority_level_bump;int32_t priority_level_preempt;uint32_t min_flying; +uint32_t file_queue_max; };/* Flag bits for i915_scheduler::flags */ @@ -100,5 +101,6 @@ int i915_scheduler_flush_stamp(struct intel_engine_cs *ring, unsigned long stamp, bool is_locked);bool i915_scheduler_is_request_tracked (struct drm_i915_gem_request *req,bool *completed,bool *busy); +bool i915_scheduler_file_queue_is_full (struct drm_file *file);
Quoted 3 lines
drivers/gpu/drm/i915/i915_scheduler.h int i915_scheduler_flush_stamp(struct intel_engine_cs *ring, · line 106
Quoted 2 lines
drivers/gpu/drm/i915/i915_scheduler.h +0
-0#endif /* _I915_SCHEDULER_H_ */ Just to clarify and make sure I understood the previous stuff: a queued execbuf that has not yet been dispatched does not reserve and pin pages right? That occurs at actual dispatch time? If so, I guess clients will hit this 64 queued item limit pretty regularly... How much metadata overhead does that involve? Has it been derived from some performance work with a bunch of workloads? It's fine if not, I can imagine that different mixes of workloads would be affected by lower or higher queue depths (e.g. small batch tests).
Whether a client will hit the limit is entirely down to how that client is written. A GL program that renders double buffered with one batch per frame will only ever have two batch buffer requests outstanding. Whereas, the gem_exec_nop IGT test submits a thousand batches before waiting for anything to complete.
Quoted 1 line
If this is tunable, I guess it should be clamped like a nice or rlimit value, with values outside that range requiring
CAP_SYS_ADMIN.
What is considered a valid range? Previously the limit would have been the size of the ring buffer. In times past, they were large and you could have thousands of batches in flight. With the GuC we are down to 4K words. At, for example, 32 words per batch buffer request that makes for 128 outstanding batches. So set the maximum permitted to 256? Larger? Smaller?
Quoted 4 lines
Reviewed-by Jesse Barnes <[email protected]>Jesse
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 26/35 drm/i915: Added debugfs interface to scheduler tuning parameters
In reply to [email protected]From: John Harrison <[email protected]>
There are various parameters within the scheduler which can be tuned to improve performance, reduce memory footprint, etc. This change adds support for altering these via debugfs.
v2: Updated for priorities now being signed values.
v5: Squashed priority bumping entries into this patch rather than a separate patch all of their own.
For: VIZ-1587
--- drivers/gpu/drm/i915/i915_debugfs.c | 169 ++++++++++++++++++++++++++++++++++++ 1 file changed, 169 insertions(+)
drivers/gpu/drm/i915/i915_debugfs.c +169 -0
@@ -39,6 +39,7 @@#include "intel_ringbuffer.h" #include <drm/i915_drm.h> #include "i915_drv.h" +#include "i915_scheduler.h" enum {ACTIVE_LIST ,
@@ -1122,6 +1123,168 @@ DEFINE_SIMPLE_ATTRIBUTE(i915_next_seqno_fops, i915_next_seqno_get, i915_next_seqno_set, "0x%llx\n"); +static int +i915_scheduler_priority_min_get (void *data,u64 *val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + *val = (u64 ) scheduler->priority_level_min; +return 0 ; +} + +static int +i915_scheduler_priority_min_set (void *data,u64 val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + scheduler->priority_level_min = (int32_t ) val; +return 0 ; +} + +DEFINE_SIMPLE_ATTRIBUTE (i915_scheduler_priority_min_fops, + i915_scheduler_priority_min_get, + i915_scheduler_priority_min_set, +"%lld\n" ); + +static int +i915_scheduler_priority_max_get (void *data,u64 *val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + *val = (u64 ) scheduler->priority_level_max; +return 0 ; +} + +static int +i915_scheduler_priority_max_set (void *data,u64 val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + scheduler->priority_level_max = (int32_t ) val; +return 0 ; +} + +DEFINE_SIMPLE_ATTRIBUTE (i915_scheduler_priority_max_fops, + i915_scheduler_priority_max_get, + i915_scheduler_priority_max_set, +"%lld\n" ); + +static int +i915_scheduler_priority_bump_get (void *data,u64 *val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + *val = (u64 ) scheduler->priority_level_bump; +return 0 ; +} + +static int +i915_scheduler_priority_bump_set (void *data,u64 val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + scheduler->priority_level_bump = (u32 ) val; +return 0 ; +} + +DEFINE_SIMPLE_ATTRIBUTE (i915_scheduler_priority_bump_fops, + i915_scheduler_priority_bump_get, + i915_scheduler_priority_bump_set, +"%lld\n" ); + +static int +i915_scheduler_priority_preempt_get (void *data,u64 *val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + *val = (u64 ) scheduler->priority_level_preempt; +return 0 ; +} + +static int +i915_scheduler_priority_preempt_set (void *data,u64 val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + scheduler->priority_level_preempt = (u32 ) val; +return 0 ; +} + +DEFINE_SIMPLE_ATTRIBUTE (i915_scheduler_priority_preempt_fops, + i915_scheduler_priority_preempt_get, + i915_scheduler_priority_preempt_set, +"%lld\n" ); + +static int +i915_scheduler_min_flying_get (void *data,u64 *val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + *val = (u64 ) scheduler->min_flying; +return 0 ; +} + +static int +i915_scheduler_min_flying_set (void *data,u64 val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + scheduler->min_flying = (u32 ) val; +return 0 ; +} + +DEFINE_SIMPLE_ATTRIBUTE (i915_scheduler_min_flying_fops, + i915_scheduler_min_flying_get, + i915_scheduler_min_flying_set, +"%llu\n" ); + +static int +i915_scheduler_file_queue_max_get (void *data,u64 *val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + *val = (u64 ) scheduler->file_queue_max; +return 0 ; +} + +static int +i915_scheduler_file_queue_max_set (void *data,u64 val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + scheduler->file_queue_max = (u32 ) val; +return 0 ; +} + +DEFINE_SIMPLE_ATTRIBUTE (i915_scheduler_file_queue_max_fops, + i915_scheduler_file_queue_max_get, + i915_scheduler_file_queue_max_set, +"%llu\n" ); +static int i915_frequency_info (struct seq_file *m,void *unused) {struct drm_info_node *node = m->private;
@@ -5424,6 +5587,12 @@ static const struct i915_debugfs_files { {"i915_gem_drop_caches", &i915_drop_caches_fops}, {"i915_error_state", &i915_error_state_fops}, {"i915_next_seqno", &i915_next_seqno_fops}, + {"i915_scheduler_priority_min", &i915_scheduler_priority_min_fops}, + {"i915_scheduler_priority_max", &i915_scheduler_priority_max_fops}, + {"i915_scheduler_priority_bump", &i915_scheduler_priority_bump_fops}, + {"i915_scheduler_priority_preempt", &i915_scheduler_priority_preempt_fops}, + {"i915_scheduler_min_flying", &i915_scheduler_min_flying_fops}, + {"i915_scheduler_file_queue_max", &i915_scheduler_file_queue_max_fops}, {"i915_display_crc_ctl", &i915_display_crc_ctl_fops}, {"i915_pri_wm_latency", &i915_pri_wm_latency_fops}, {"i915_spr_wm_latency", &i915_spr_wm_latency_fops},
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 26/35] drm/i915: Added debugfs interface to scheduler tuning parameters
Quoted 202 lines · On 02/18/2016 06:27 AM, [email protected] wrote:
From: John Harrison <[email protected]>
There are various parameters within the scheduler which can be tuned to improve performance, reduce memory footprint, etc. This change adds support for altering these via debugfs.
v2: Updated for priorities now being signed values.
v5: Squashed priority bumping entries into this patch rather than a separate patch all of their own.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>--- drivers/gpu/drm/i915/i915_debugfs.c | 169 ++++++++++++++++++++++++++++++++++++ 1 file changed, 169 insertions(+)drivers/gpu/drm/i915/i915_debugfs.c +163
-0@@ -39,6 +39,7 @@#include "intel_ringbuffer.h" #include <drm/i915_drm.h> #include "i915_drv.h" +#include "i915_scheduler.h" enum {ACTIVE_LIST ,@@ -1122,6 +1123,168 @@ DEFINE_SIMPLE_ATTRIBUTE(i915_next_seqno_fops, i915_next_seqno_get, i915_next_seqno_set, "0x%llx\n"); +static int +i915_scheduler_priority_min_get (void *data,u64 *val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + *val = (u64 ) scheduler->priority_level_min; +return 0 ; +} + +static int +i915_scheduler_priority_min_set (void *data,u64 val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + scheduler->priority_level_min = (int32_t ) val; +return 0 ; +} + +DEFINE_SIMPLE_ATTRIBUTE (i915_scheduler_priority_min_fops, + i915_scheduler_priority_min_get, + i915_scheduler_priority_min_set, +"%lld\n" ); + +static int +i915_scheduler_priority_max_get (void *data,u64 *val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + *val = (u64 ) scheduler->priority_level_max; +return 0 ; +} + +static int +i915_scheduler_priority_max_set (void *data,u64 val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + scheduler->priority_level_max = (int32_t ) val; +return 0 ; +} + +DEFINE_SIMPLE_ATTRIBUTE (i915_scheduler_priority_max_fops, + i915_scheduler_priority_max_get, + i915_scheduler_priority_max_set, +"%lld\n" ); + +static int +i915_scheduler_priority_bump_get (void *data,u64 *val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + *val = (u64 ) scheduler->priority_level_bump; +return 0 ; +} + +static int +i915_scheduler_priority_bump_set (void *data,u64 val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + scheduler->priority_level_bump = (u32 ) val; +return 0 ; +} + +DEFINE_SIMPLE_ATTRIBUTE (i915_scheduler_priority_bump_fops, + i915_scheduler_priority_bump_get, + i915_scheduler_priority_bump_set, +"%lld\n" ); + +static int +i915_scheduler_priority_preempt_get (void *data,u64 *val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + *val = (u64 ) scheduler->priority_level_preempt; +return 0 ; +} + +static int +i915_scheduler_priority_preempt_set (void *data,u64 val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + scheduler->priority_level_preempt = (u32 ) val; +return 0 ; +} + +DEFINE_SIMPLE_ATTRIBUTE (i915_scheduler_priority_preempt_fops, + i915_scheduler_priority_preempt_get, + i915_scheduler_priority_preempt_set, +"%lld\n" ); + +static int +i915_scheduler_min_flying_get (void *data,u64 *val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + *val = (u64 ) scheduler->min_flying; +return 0 ; +} + +static int +i915_scheduler_min_flying_set (void *data,u64 val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + scheduler->min_flying = (u32 ) val; +return 0 ; +} + +DEFINE_SIMPLE_ATTRIBUTE (i915_scheduler_min_flying_fops, + i915_scheduler_min_flying_get, + i915_scheduler_min_flying_set, +"%llu\n" ); + +static int +i915_scheduler_file_queue_max_get (void *data,u64 *val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + *val = (u64 ) scheduler->file_queue_max; +return 0 ; +} + +static int +i915_scheduler_file_queue_max_set (void *data,u64 val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + scheduler->file_queue_max = (u32 ) val; +return 0 ; +} + +DEFINE_SIMPLE_ATTRIBUTE (i915_scheduler_file_queue_max_fops, + i915_scheduler_file_queue_max_get, + i915_scheduler_file_queue_max_set, +"%llu\n" ); +static int i915_frequency_info (struct seq_file *m,void *unused) {struct drm_info_node *node = m->private;@@ -5424,6 +5587,12 @@ static const struct i915_debugfs_files { {"i915_gem_drop_caches", &i915_drop_caches_fops}, {"i915_error_state", &i915_error_state_fops},
drivers/gpu/drm/i915/i915_debugfs.c static const struct i915_debugfs_files { · line 5598
Quoted 10 lines
drivers/gpu/drm/i915/i915_debugfs.c +6
-0{"i915_next_seqno", &i915_next_seqno_fops}, + {"i915_scheduler_priority_min", &i915_scheduler_priority_min_fops}, + {"i915_scheduler_priority_max", &i915_scheduler_priority_max_fops}, + {"i915_scheduler_priority_bump", &i915_scheduler_priority_bump_fops}, + {"i915_scheduler_priority_preempt", &i915_scheduler_priority_preempt_fops}, + {"i915_scheduler_min_flying", &i915_scheduler_min_flying_fops}, + {"i915_scheduler_file_queue_max", &i915_scheduler_file_queue_max_fops}, {"i915_display_crc_ctl", &i915_display_crc_ctl_fops}, {"i915_pri_wm_latency", &i915_pri_wm_latency_fops}, {"i915_spr_wm_latency", &i915_spr_wm_latency_fops},
Do these need to be serialized at all? I guess some raciness doesn't hurt too much for these guys, unless somehow an inconsistent set of values would cause a livelock in the scheduler or something.
If not,
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 26/35] drm/i915: Added debugfs interface to scheduler tuning parameters
Quoted 206 lines · Jesse Barnes
Quoted 205 lines · On 02/18/2016 06:27 AM, [email protected] wrote:
From: John Harrison <[email protected]>
There are various parameters within the scheduler which can be tuned to improve performance, reduce memory footprint, etc. This change adds support for altering these via debugfs.
v2: Updated for priorities now being signed values.
v5: Squashed priority bumping entries into this patch rather than a separate patch all of their own.
For: VIZ-1587
Signed-off-by John Harrison <[email protected]>--- drivers/gpu/drm/i915/i915_debugfs.c | 169 ++++++++++++++++++++++++++++++++++++ 1 file changed, 169 insertions(+)drivers/gpu/drm/i915/i915_debugfs.c +166
-0@@ -39,6 +39,7 @@#include "intel_ringbuffer.h" #include <drm/i915_drm.h> #include "i915_drv.h" +#include "i915_scheduler.h" enum {ACTIVE_LIST ,@@ -1122,6 +1123,168 @@ DEFINE_SIMPLE_ATTRIBUTE(i915_next_seqno_fops, i915_next_seqno_get, i915_next_seqno_set, "0x%llx\n"); +static int +i915_scheduler_priority_min_get (void *data,u64 *val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + *val = (u64 ) scheduler->priority_level_min; +return 0 ; +} + +static int +i915_scheduler_priority_min_set (void *data,u64 val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + scheduler->priority_level_min = (int32_t ) val; +return 0 ; +} + +DEFINE_SIMPLE_ATTRIBUTE (i915_scheduler_priority_min_fops, + i915_scheduler_priority_min_get, + i915_scheduler_priority_min_set, +"%lld\n" ); + +static int +i915_scheduler_priority_max_get (void *data,u64 *val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + *val = (u64 ) scheduler->priority_level_max; +return 0 ; +} + +static int +i915_scheduler_priority_max_set (void *data,u64 val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + scheduler->priority_level_max = (int32_t ) val; +return 0 ; +} + +DEFINE_SIMPLE_ATTRIBUTE (i915_scheduler_priority_max_fops, + i915_scheduler_priority_max_get, + i915_scheduler_priority_max_set, +"%lld\n" ); + +static int +i915_scheduler_priority_bump_get (void *data,u64 *val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + *val = (u64 ) scheduler->priority_level_bump; +return 0 ; +} + +static int +i915_scheduler_priority_bump_set (void *data,u64 val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + scheduler->priority_level_bump = (u32 ) val; +return 0 ; +} + +DEFINE_SIMPLE_ATTRIBUTE (i915_scheduler_priority_bump_fops, + i915_scheduler_priority_bump_get, + i915_scheduler_priority_bump_set, +"%lld\n" ); + +static int +i915_scheduler_priority_preempt_get (void *data,u64 *val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + *val = (u64 ) scheduler->priority_level_preempt; +return 0 ; +} + +static int +i915_scheduler_priority_preempt_set (void *data,u64 val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + scheduler->priority_level_preempt = (u32 ) val; +return 0 ; +} + +DEFINE_SIMPLE_ATTRIBUTE (i915_scheduler_priority_preempt_fops, + i915_scheduler_priority_preempt_get, + i915_scheduler_priority_preempt_set, +"%lld\n" ); + +static int +i915_scheduler_min_flying_get (void *data,u64 *val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + *val = (u64 ) scheduler->min_flying; +return 0 ; +} + +static int +i915_scheduler_min_flying_set (void *data,u64 val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + scheduler->min_flying = (u32 ) val; +return 0 ; +} + +DEFINE_SIMPLE_ATTRIBUTE (i915_scheduler_min_flying_fops, + i915_scheduler_min_flying_get, + i915_scheduler_min_flying_set, +"%llu\n" ); + +static int +i915_scheduler_file_queue_max_get (void *data,u64 *val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + *val = (u64 ) scheduler->file_queue_max; +return 0 ; +} + +static int +i915_scheduler_file_queue_max_set (void *data,u64 val) +{ +struct drm_device *dev = data; +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; + + scheduler->file_queue_max = (u32 ) val; +return 0 ; +} + +DEFINE_SIMPLE_ATTRIBUTE (i915_scheduler_file_queue_max_fops, + i915_scheduler_file_queue_max_get, + i915_scheduler_file_queue_max_set, +"%llu\n" ); +static int i915_frequency_info (struct seq_file *m,void *unused) {struct drm_info_node *node = m->private;@@ -5424,6 +5587,12 @@ static const struct i915_debugfs_files { {"i915_gem_drop_caches", &i915_drop_caches_fops}, {"i915_error_state", &i915_error_state_fops}, {"i915_next_seqno", &i915_next_seqno_fops}, + {"i915_scheduler_priority_min", &i915_scheduler_priority_min_fops}, + {"i915_scheduler_priority_max", &i915_scheduler_priority_max_fops}, + {"i915_scheduler_priority_bump", &i915_scheduler_priority_bump_fops},
Quoted 8 lines
drivers/gpu/drm/i915/i915_debugfs.c static const struct i915_debugfs_files { · line 5598
Quoted 7 lines
drivers/gpu/drm/i915/i915_debugfs.c +3
-0+ {"i915_scheduler_priority_preempt", &i915_scheduler_priority_preempt_fops}, + {"i915_scheduler_min_flying", &i915_scheduler_min_flying_fops}, + {"i915_scheduler_file_queue_max", &i915_scheduler_file_queue_max_fops}, {"i915_display_crc_ctl", &i915_display_crc_ctl_fops}, {"i915_pri_wm_latency", &i915_pri_wm_latency_fops}, {"i915_spr_wm_latency", &i915_spr_wm_latency_fops},Do these need to be serialized at all? I guess some raciness doesn't hurt too much for these guys, unless somehow an inconsistent set of values would cause a livelock in the scheduler or something.
Serialised with what? Each other or the scheduler operation? Neither should be necessary. The scheduler will read the current values whenever it tests against one of these limits. If multiple are being changed while the system is busy, it doesn't really matter. They are just tuning values and best guesses type of numbers not array indices or other things that would cause kernel panics if you got them wrong. E.g. if you set the max file queue depth to smaller than the current queue contents that just means you won't be able to submit more stuff until the queue has drained - which is presumably the intended result of lowering the max queue value anyway. The queue won't leak the extra entries or get into an inconsistent state.
Quoted 3 lines
If not,
Reviewed-by Jesse Barnes <[email protected]>
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [PATCH v5 26/35] drm/i915: Added debugfs interface to scheduler tuning parameters
Quoted 24 lines · John Harrison
Quoted 15 lines · Jesse Barnes
Quoted 14 lines · On 02/18/2016 06:27 AM, [email protected] wrote:
From: John Harrison <[email protected]>
struct drm_info_node *node = m->private;drivers/gpu/drm/i915/i915_debugfs.c +6
-0@@ -5424,6 +5587,12 @@ static const struct i915_debugfs_files { {"i915_gem_drop_caches", &i915_drop_caches_fops}, {"i915_error_state", &i915_error_state_fops}, {"i915_next_seqno", &i915_next_seqno_fops}, + {"i915_scheduler_priority_min", &i915_scheduler_priority_min_fops}, + {"i915_scheduler_priority_max", &i915_scheduler_priority_max_fops}, + {"i915_scheduler_priority_bump", &i915_scheduler_priority_bump_fops}, + {"i915_scheduler_priority_preempt", &i915_scheduler_priority_preempt_fops}, + {"i915_scheduler_min_flying", &i915_scheduler_min_flying_fops}, + {"i915_scheduler_file_queue_max",Quoted 8 lines
Quoted 5 lines
(quoted diff) +0
-0&i915_scheduler_file_queue_max_fops},{"i915_display_crc_ctl", &i915_display_crc_ctl_fops}, {"i915_pri_wm_latency", &i915_pri_wm_latency_fops}, {"i915_spr_wm_latency", &i915_spr_wm_latency_fops},Do these need to be serialized at all? I guess some raciness doesn't hurt too much for these guys, unless somehow an inconsistent set of values would cause a livelock in the scheduler or something.
Quoted 11 lines
Serialised with what? Each other or the scheduler operation? Neither should be necessary. The scheduler will read the current values whenever it tests against one of these limits. If multiple are being changed while the system is busy, it doesn't really matter. They are just tuning values and best guesses type of numbers not array indices or other things that would cause kernel panics if you got them wrong. E.g. if you set the max file queue depth to smaller than the current queue contents that just means you won't be able to submit more stuff until the queue has drained - which is presumably the intended result of lowering the max queue value anyway. The queue won't leak the extra entries or get into an inconsistent state.
Yeah I meant both serialized against scheduler accesses/usage and atomicity with respect to one another. Sounds like it doesn't matter too much though, so it's fine with me.
Jesse
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v5 27/35 drm/i915: Added debug state dump facilities to scheduler
In reply to [email protected]From: John Harrison <[email protected]>
When debugging batch buffer submission issues, it is useful to be able to see what the current state of the scheduler is. This change adds functions for decoding the internal scheduler state and reporting it.
v3: Updated a debug message with the new state_str() function.
v4: Wrapped some long lines to keep the style checker happy. Removed the fence/sync code as that will now be part of a separate patch series.
v5: Removed forward declarations and white space. Added documentation. [Joonas Lahtinen]
Also squashed in later patch to add seqno information from the start. It was only being added in a separate patch due to historical reasons which have since gone away.
For: VIZ-1587
--- drivers/gpu/drm/i915/i915_scheduler.c | 302 +++++++++++++++++++++++++++++++++- drivers/gpu/drm/i915/i915_scheduler.h | 15 ++ 2 files changed, 315 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_scheduler.c +300 -2
@@ -40,6 +40,117 @@ bool i915_scheduler_is_enabled(struct drm_device *dev)return dev_priv->scheduler !=NULL ; } +const char *i915_qe_state_str (struct i915_scheduler_queue_entry *node) +{ +static char str[50 ]; +char *ptr = str; + + *(ptr++) = node->bumped ?'B' :'-' , + *(ptr++) =i915_gem_request_completed (node->params.request) ?'C' :'-' ; + + *ptr =0 ; + +return str; +} + +char i915_scheduler_queue_status_chr (enum i915_scheduler_queue_status status) +{ +switch (status) { +case i915_sqs_none: +return 'N' ; + +case i915_sqs_queued: +return 'Q' ; + +case i915_sqs_popped: +return 'X' ; + +case i915_sqs_flying: +return 'F' ; + +case i915_sqs_complete: +return 'C' ; + +case i915_sqs_dead: +return 'D' ; + +default : +break ; + } + +return '?' ; +} + +const char *i915_scheduler_queue_status_str ( +enum i915_scheduler_queue_status status) +{ +static char str[50 ]; + +switch (status) { +case i915_sqs_none: +return "None" ; + +case i915_sqs_queued: +return "Queued" ; + +case i915_sqs_popped: +return "Popped" ; + +case i915_sqs_flying: +return "Flying" ; + +case i915_sqs_complete: +return "Complete" ; + +case i915_sqs_dead: +return "Dead" ; + +default : +break ; + } + +sprintf (str,"[Unknown_%d!]" , status); +return str; +} + +const char *i915_scheduler_flag_str (uint32_t flags) +{ +static char str[100 ]; +char *ptr = str; + + *ptr =0 ; + +#define TEST_FLAG (flag, msg) \ + do { \ + if (flags & (flag)) { \ + strcpy(ptr, msg); \ + ptr += strlen(ptr); \ + flags &= ~(flag); \ + } \ + } while (0) + +TEST_FLAG (i915_sf_interrupts_enabled,"IntOn|" ); +TEST_FLAG (i915_sf_submitting,"Submitting|" ); +TEST_FLAG (i915_sf_dump_force,"DumpForce|" ); +TEST_FLAG (i915_sf_dump_details,"DumpDetails|" ); +TEST_FLAG (i915_sf_dump_dependencies,"DumpDeps|" ); +TEST_FLAG (i915_sf_dump_seqno,"DumpSeqno|" ); + +#undef TEST_FLAG + +if (flags) { +sprintf (ptr,"Unknown_0x%X!" , flags); + ptr +=strlen (ptr); + } + +if (ptr == str) +strcpy (str,"-" ); +else + ptr[-1 ] =0 ; + +return str; +}; +/** * i915_scheduler_init - Initialise the scheduler. * @dev: DRM device
@@ -1024,6 +1135,193 @@ void i915_scheduler_work_handler(struct work_struct *work)i915_scheduler_process_work (ring); } +static int i915_scheduler_dump_locked (struct intel_engine_cs *ring, +const char *msg) +{ +struct drm_i915_private *dev_priv = ring->dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; +struct i915_scheduler_queue_entry *node; +int flying =0 , queued =0 , complete =0 , other =0 ; +static int old_flying =-1 , old_queued =-1 , old_complete =-1 ; +bool b_dump; +char brkt[2 ] = {'<' ,'>' }; + +if (!ring) +return -EINVAL ; + +list_for_each_entry (node, &scheduler->node_queue[ring->id], link) { +if (I915_SQS_IS_QUEUED (node)) + queued++; +else if (I915_SQS_IS_FLYING (node)) + flying++; +else if (I915_SQS_IS_COMPLETE (node)) + complete++; +else + other++; + } + + b_dump = (flying != old_flying) || + (queued != old_queued) || + (complete != old_complete); +if (scheduler->flags[ring->id] & i915_sf_dump_force) { +if (!b_dump) { + b_dump = true; + brkt[0 ] ='{' ; + brkt[1 ] ='}' ; + } + + scheduler->flags[ring->id] &= ~i915_sf_dump_force; + } + +if (b_dump) { + old_flying = flying; + old_queued = queued; + old_complete = complete; +DRM_DEBUG_DRIVER ("<%s> Q:%02d, F:%02d, C:%02d, O:%02d, " +"Flags = %s, Next = %d:%d %c%s%c\n" , + ring->name, queued, flying, complete, other, +i915_scheduler_flag_str (scheduler->flags[ring->id]), + dev_priv->request_uniq, dev_priv->next_seqno, + brkt[0 ], msg, brkt[1 ]); + }else { +/*DRM_DEBUG_DRIVER("<%s> Q:%02d, F:%02d, C:%02d, O:%02d" +", Flags = %s, Next = %d:%d [%s]\n", +ring->name, +queued, flying, complete, other, +i915_scheduler_flag_str(scheduler->flags[ring->id]), +dev_priv->request_uniq, dev_priv->next_seqno, msg); */ + +return 0 ; + } + +if (scheduler->flags[ring->id] & i915_sf_dump_seqno) { +uint32_t seqno; + + seqno = ring->get_seqno (ring, true); + +DRM_DEBUG_DRIVER ("<%s> Seqno = %d\n" , ring->name, seqno); + } + +if (scheduler->flags[ring->id] & i915_sf_dump_details) { +int i, deps; +uint32_t count, counts[i915_sqs_MAX]; + +memset (counts,0x00 ,sizeof (counts)); + +list_for_each_entry (node, &scheduler->node_queue[ring->id], link) { +if (node->status < i915_sqs_MAX) { + count = counts[node->status]++; + }else { +DRM_DEBUG_DRIVER ("<%s> Unknown status: %d!\n" , + ring->name, node->status); + count =-1 ; + } + + deps =0 ; +for (i =0 ; i < node->num_deps; i++) +if (i915_scheduler_is_dependency_valid (node, i)) + deps++; + +DRM_DEBUG_DRIVER ("<%s> %c:%02d> uniq = %d, seqno" +" = %d/%s, deps = %d / %d, %s [pri = " +"%4d]\n" , ring->name, +i915_scheduler_queue_status_chr (node->status), + count, + node->params.request->uniq, + node->params.request->seqno, + node->params.ring->name, + deps, node->num_deps, +i915_qe_state_str (node), + node->priority); + +if ((scheduler->flags[ring->id] & i915_sf_dump_dependencies) + ==0 ) +continue ; + +for (i =0 ; i < node->num_deps; i++) +if (node->dep_list[i]) +DRM_DEBUG_DRIVER ("<%s> |-%c:" +"%02d%c uniq = %d, seqno = %d/%s, %s [pri = %4d]\n" , + ring->name, +i915_scheduler_queue_status_chr (node->dep_list[i]->status), + i, +i915_scheduler_is_dependency_valid (node, i) + ?'>' :'#' , + node->dep_list[i]->params.request->uniq, + node->dep_list[i]->params.request->seqno, + node->dep_list[i]->params.ring->name, +i915_qe_state_str (node->dep_list[i]), + node->dep_list[i]->priority); + } + } + +return 0 ; +} + +/** +* i915_scheduler_dump - dump the scheduler's internal state to the debug log. +* @ring: Ring to dump info for +* @msg: A reason why it is being dumped +* For debugging purposes, it can be very useful to see the internal state of +* the scheduler for a given ring. +*/ +int i915_scheduler_dump (struct intel_engine_cs *ring,const char *msg) +{ +struct drm_i915_private *dev_priv = ring->dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; +unsigned long flags; +int ret; + +spin_lock_irqsave (&scheduler->lock, flags); + ret =i915_scheduler_dump_locked (ring, msg); +spin_unlock_irqrestore (&scheduler->lock, flags); + +return ret; +} + +static int i915_scheduler_dump_all_locked (struct drm_device *dev, +const char *msg) +{ +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; +struct intel_engine_cs *ring; +int i, r, ret =0 ; + +for_each_ring (ring, dev_priv, i) { + scheduler->flags[ring->id] |= i915_sf_dump_force | + i915_sf_dump_details | + i915_sf_dump_seqno | + i915_sf_dump_dependencies; + r =i915_scheduler_dump_locked (ring, msg); +if (ret ==0 ) + ret = r; + } + +return ret; +} + +/** +* i915_scheduler_dump_all - dump the scheduler's internal state to the debug +* log. +* @dev: DRM device +* @msg: A reason why it is being dumped +* For debugging purposes, it can be very useful to see the internal state of +* the scheduler. +*/ +int i915_scheduler_dump_all (struct drm_device *dev,const char *msg) +{ +struct drm_i915_private *dev_priv = dev->dev_private; +struct i915_scheduler *scheduler = dev_priv->scheduler; +unsigned long flags; +int ret; + +spin_lock_irqsave (&scheduler->lock, flags); + ret =i915_scheduler_dump_all_locked (dev, msg); +spin_unlock_irqrestore (&scheduler->lock, flags); + +return ret; +} +static int i915_scheduler_submit_max_priority (struct intel_engine_cs *ring,bool is_locked) {
@@ -1246,10 +1544,10 @@ int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)continue ;if (!I915_SQS_IS_COMPLETE (node)) -DRM_DEBUG_DRIVER ("Closing file handle with outstanding work: %d:%d/%d on %s\n" , +DRM_DEBUG_DRIVER ("Closing file handle with outstanding work: %d:%d/%s on %s\n" , node->params.request->uniq, node->params.request->seqno, - node->status, +i915_qe_state_str (node), ring->name);i915_scheduler_file_queue_dec (node->params.file);
drivers/gpu/drm/i915/i915_scheduler.h +15 -0
@@ -41,6 +41,9 @@ enum i915_scheduler_queue_status {/* Limit value for use with arrays/loops */ i915_sqs_MAX }; +char i915_scheduler_queue_status_chr (enum i915_scheduler_queue_status status); +const char *i915_scheduler_queue_status_str ( +enum i915_scheduler_queue_status status);#define I915_SQS_IS_QUEUED (node) (((node)->status == i915_sqs_queued))#define I915_SQS_IS_FLYING (node) (((node)->status == i915_sqs_flying))
@@ -65,6 +68,7 @@ struct i915_scheduler_queue_entry {unsigned long stamp;struct list_head link; }; +const char *i915_qe_state_str (struct i915_scheduler_queue_entry *node);struct i915_scheduler {struct list_head node_queue[I915_NUM_RINGS ];
@@ -82,9 +86,17 @@ struct i915_scheduler {/* Flag bits for i915_scheduler::flags */ enum { +/* Internal state */ i915_sf_interrupts_enabled = (1 <<0 ), i915_sf_submitting = (1 <<1 ), + +/* Dump/debug flags */ + i915_sf_dump_force = (1 <<8 ), + i915_sf_dump_details = (1 <<9 ), + i915_sf_dump_dependencies = (1 <<10 ), + i915_sf_dump_seqno = (1 <<11 ), }; +const char *i915_scheduler_flag_str (uint32_t flags);bool i915_scheduler_is_enabled (struct drm_device *dev);int i915_scheduler_init (struct drm_device *dev);
@@ -99,6 +111,9 @@ void i915_scheduler_work_handler(struct work_struct *work);int i915_scheduler_flush (struct intel_engine_cs *ring,bool is_locked);int i915_scheduler_flush_stamp (struct intel_engine_cs *ring,unsigned long stamp,bool is_locked); +int i915_scheduler_dump (struct intel_engine_cs *ring, +const char *msg); +int i915_scheduler_dump_all (struct drm_device *dev,const char *msg);bool i915_scheduler_is_request_tracked (struct drm_i915_gem_request *req,bool *completed,bool *busy);bool i915_scheduler_file_queue_is_full (struct drm_file *file);
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx