Parallel submission aka multi-bb execbuf
Subsystems (4)
cover Parallel submission aka multi-bb execbuf
As discussed in [1] we are introducing a new parallel submission uAPI for the i915 which allows more than 1 BB to be submitted in an execbuf IOCTL. This is the implemenation for both GuC and execlists.
In addition to selftests in the series, an IGT is available implemented in the first 4 patches [2].
Media UMD changes to land soon.
This series is broken into 5 parts.
1. A series of GuC patches which introduces a state machine to deal with flow control conditions gracefully (e.g. don't punt them to the user). These are patches 1-12.
2. Update the GuC backend / connections to uAPI to configure it for parallel submission. These are patches 13-30.
3. Update execbuf IOCTL to accept more than 1 BB in a single IOCTL. These are patches 31-44.
4. A weak execlists implemenation for parallel submission. Patch 45.
5. Add a heuristic to enable issue schedule disables immediately after unpin. Not all that related but wanted to get this out on the list for review and based on the tip of all of these patches. Patch 46.
[1] https://patchwork.freedesktop.org/series/92028/
[2] https://patchwork.freedesktop.org/series/93071/
Matthew Brost (46):
drm/i915/guc: Allow flexible number of context ids drm/i915/guc: Connect the number of guc_ids to debugfs drm/i915/guc: Don't return -EAGAIN to user when guc_ids exhausted drm/i915/guc: Don't allow requests not ready to consume all guc_ids drm/i915/guc: Introduce guc_submit_engine object drm/i915/guc: Check return of __xa_store when registering a context drm/i915/guc: Non-static lrc descriptor registration buffer drm/i915/guc: Take GT PM ref when deregistering context drm/i915: Add GT PM unpark worker
drm/i915/guc: Take engine PM when a context is pinned with GuC
submission
drm/i915/guc: Don't call switch_to_kernel_context with GuC submission drm/i915/guc: Selftest for GuC flow control
drm/i915: Add logical engine mapping
drm/i915: Expose logical engine instance to user drm/i915/guc: Introduce context parent-child relationship drm/i915/guc: Implement GuC parent-child context pin / unpin functions drm/i915/guc: Add multi-lrc context registration drm/i915/guc: Ensure GuC schedule operations do not operate on child
contexts
drm/i915/guc: Assign contexts in parent-child relationship consecutive
guc_ids
drm/i915/guc: Add hang check to GuC submit engine drm/i915/guc: Add guc_child_context_destroy
drm/i915/guc: Implement multi-lrc submission
drm/i915/guc: Insert submit fences between requests in parent-child
relationship
drm/i915/guc: Implement multi-lrc reset
drm/i915/guc: Update debugfs for GuC multi-lrc
drm/i915: Connect UAPI to GuC multi-lrc interface drm/i915/doc: Update parallel submit doc to point to i915_drm.h drm/i915/guc: Add basic GuC multi-lrc selftest drm/i915/guc: Extend GuC flow control selftest for multi-lrc drm/i915/guc: Implement no mid batch preemption for multi-lrc drm/i915: Move secure execbuf check to execbuf2
drm/i915: Move input/exec fence handling to i915_gem_execbuffer2 drm/i915: Move output fence handling to i915_gem_execbuffer2 drm/i915: Return output fence from i915_gem_do_execbuffer drm/i915: Store batch index in struct i915_execbuffer drm/i915: Allow callers of i915_gem_do_execbuffer to override the
batch index
drm/i915: Teach execbuf there can be more than one batch in the
objects list
drm/i915: Only track object dependencies on first request drm/i915: Force parallel contexts to use copy engine for reloc drm/i915: Multi-batch execbuffer2
drm/i915: Eliminate unnecessary VMA calls for multi-BB submission drm/i915: Hold all parallel requests until last request, properly
handle error
drm/i915/guc: Handle errors in multi-lrc requests drm/i915: Enable multi-bb execbuf
drm/i915/execlists: Weak parallel submission support for execlists drm/i915/guc: Add delay before disabling scheduling on contexts
Documentation/gpu/rfc/i915_parallel_execbuf.h | 122 - Documentation/gpu/rfc/i915_scheduler.rst | 4 +- drivers/gpu/drm/i915/Makefile | 1 + drivers/gpu/drm/i915/gem/i915_gem_context.c | 159 +- .../gpu/drm/i915/gem/i915_gem_context_types.h | 6 + .../gpu/drm/i915/gem/i915_gem_execbuffer.c | 582 ++- .../i915/gem/selftests/i915_gem_coherency.c | 2 +- .../drm/i915/gem/selftests/i915_gem_dmabuf.c | 2 +- .../i915/gem/selftests/i915_gem_execbuffer.c | 14 +- .../drm/i915/gem/selftests/i915_gem_mman.c | 2 +- .../drm/i915/gem/selftests/i915_gem_object.c | 2 +- drivers/gpu/drm/i915/gt/intel_context.c | 236 +- drivers/gpu/drm/i915/gt/intel_context.h | 81 +- drivers/gpu/drm/i915/gt/intel_context_types.h | 62 +- drivers/gpu/drm/i915/gt/intel_engine.h | 12 +- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 66 +- drivers/gpu/drm/i915/gt/intel_engine_pm.c | 4 + drivers/gpu/drm/i915/gt/intel_engine_pm.h | 5 + drivers/gpu/drm/i915/gt/intel_engine_types.h | 1 + .../drm/i915/gt/intel_execlists_submission.c | 233 +- drivers/gpu/drm/i915/gt/intel_gt.c | 3 + drivers/gpu/drm/i915/gt/intel_gt_pm.c | 8 + drivers/gpu/drm/i915/gt/intel_gt_pm.h | 13 + .../gpu/drm/i915/gt/intel_gt_pm_unpark_work.c | 35 + .../gpu/drm/i915/gt/intel_gt_pm_unpark_work.h | 32 + drivers/gpu/drm/i915/gt/intel_gt_types.h | 3 + drivers/gpu/drm/i915/gt/intel_lrc.c | 31 +- drivers/gpu/drm/i915/gt/intel_lrc.h | 6 +- drivers/gpu/drm/i915/gt/intel_reset.c | 10 + .../gpu/drm/i915/gt/intel_ring_submission.c | 5 +- drivers/gpu/drm/i915/gt/mock_engine.c | 4 +- drivers/gpu/drm/i915/gt/selftest_execlists.c | 12 +- .../gpu/drm/i915/gt/uc/abi/guc_actions_abi.h | 1 + drivers/gpu/drm/i915/gt/uc/intel_guc.h | 46 +- drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c | 2 +- drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 43 +- drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h | 9 + .../gpu/drm/i915/gt/uc/intel_guc_debugfs.c | 59 + drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h | 10 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 3159 +++++++++++++++-- .../gpu/drm/i915/gt/uc/intel_guc_submission.h | 2 + .../i915/gt/uc/intel_guc_submission_types.h | 67 + .../i915/gt/uc/selftest_guc_flow_control.c | 891 +++++ .../drm/i915/gt/uc/selftest_guc_multi_lrc.c | 179 + drivers/gpu/drm/i915/i915_query.c | 2 + drivers/gpu/drm/i915/i915_request.c | 120 +- drivers/gpu/drm/i915/i915_request.h | 23 + drivers/gpu/drm/i915/i915_scheduler.c | 22 +- drivers/gpu/drm/i915/i915_scheduler.h | 3 + drivers/gpu/drm/i915/i915_selftest.h | 2 + drivers/gpu/drm/i915/i915_trace.h | 10 + drivers/gpu/drm/i915/i915_vma.c | 13 +- drivers/gpu/drm/i915/i915_vma.h | 16 +- drivers/gpu/drm/i915/intel_wakeref.c | 5 + drivers/gpu/drm/i915/intel_wakeref.h | 1 + drivers/gpu/drm/i915/selftests/i915_gem_gtt.c | 2 +- .../drm/i915/selftests/i915_live_selftests.h | 2 + drivers/gpu/drm/i915/selftests/i915_perf.c | 2 +- drivers/gpu/drm/i915/selftests/i915_request.c | 2 +- drivers/gpu/drm/i915/selftests/i915_vma.c | 2 +- .../i915/selftests/intel_scheduler_helpers.c | 12 + .../i915/selftests/intel_scheduler_helpers.h | 2 + include/uapi/drm/i915_drm.h | 136 +- 63 files changed, 5741 insertions(+), 862 deletions(-) delete mode 100644 Documentation/gpu/rfc/i915_parallel_execbuf.h create mode 100644 drivers/gpu/drm/i915/gt/intel_gt_pm_unpark_work.c create mode 100644 drivers/gpu/drm/i915/gt/intel_gt_pm_unpark_work.h create mode 100644 drivers/gpu/drm/i915/gt/uc/intel_guc_submission_types.h create mode 100644 drivers/gpu/drm/i915/gt/uc/selftest_guc_flow_control.c create mode 100644 drivers/gpu/drm/i915/gt/uc/selftest_guc_multi_lrc.c
Signature · 1 line
2.28.0
PATCH 1/46 drm/i915/guc: Allow flexible number of context ids
Number of available GuC contexts ids might be limited. Stop referring in code to macro and use variable instead.
--- drivers/gpu/drm/i915/gt/uc/intel_guc.h | 2 ++ .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 16 +++++++++------- 2 files changed, 11 insertions(+), 7 deletions(-)
drivers/gpu/drm/i915/gt/uc/intel_guc.h +2 -0
@@ -53,6 +53,8 @@ struct intel_guc {*/ spinlock_t contexts_lock;struct ida guc_ids; +u32 num_guc_ids; +u32 max_guc_ids;struct list_head guc_id_list;bool submission_supported;
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +9 -7
@@ -275,7 +275,7 @@ static struct guc_lrc_desc *__get_lrc_desc(struct intel_guc *guc, u32 index) {struct guc_lrc_desc *base = guc->lrc_desc_pool_vaddr; -GEM_BUG_ON (index >=GUC_MAX_LRC_DESCRIPTORS ); +GEM_BUG_ON (index >= guc->max_guc_ids);return &base[index]; }
@@ -284,7 +284,7 @@ static inline struct intel_context *__get_context(struct intel_guc *guc, u32 id) {struct intel_context *ce =xa_load (&guc->context_lookup, id); -GEM_BUG_ON (id >=GUC_MAX_LRC_DESCRIPTORS ); +GEM_BUG_ON (id >= guc->max_guc_ids);return ce; }
@@ -294,8 +294,7 @@ static int guc_lrc_desc_pool_create(struct intel_guc *guc)u32 size;int ret; - size =PAGE_ALIGN (sizeof (struct guc_lrc_desc ) * -GUC_MAX_LRC_DESCRIPTORS ); + size =PAGE_ALIGN (sizeof (struct guc_lrc_desc ) * guc->max_guc_ids); ret =intel_guc_allocate_and_map_vma (guc, size, &guc->lrc_desc_pool, (void **)&guc->lrc_desc_pool_vaddr);if (ret)
@@ -1070,7 +1069,7 @@ static void guc_submit_request(struct i915_request *rq)static int new_guc_id (struct intel_guc *guc) {return ida_simple_get (&guc->guc_ids,0 , -GUC_MAX_LRC_DESCRIPTORS ,GFP_KERNEL | + guc->num_guc_ids,GFP_KERNEL | __GFP_RETRY_MAYFAIL | __GFP_NOWARN); }
@@ -2562,6 +2561,8 @@ static bool __guc_submission_selected(struct intel_guc *guc)void intel_guc_submission_init_early (struct intel_guc *guc) { + guc->max_guc_ids =GUC_MAX_LRC_DESCRIPTORS ; + guc->num_guc_ids =GUC_MAX_LRC_DESCRIPTORS ; guc->submission_supported =__guc_submission_supported (guc); guc->submission_selected =__guc_submission_selected (guc); }
@@ -2571,7 +2572,7 @@ g2h_context_lookup(struct intel_guc *guc, u32 desc_idx) {struct intel_context *ce; -if (unlikely (desc_idx >=GUC_MAX_LRC_DESCRIPTORS )) { +if (unlikely (desc_idx >= guc->max_guc_ids)) {drm_err (&guc_to_gt (guc)->i915->drm,"Invalid desc_idx %u" , desc_idx);return NULL ;
@@ -2874,6 +2875,8 @@ void intel_guc_submission_print_info(struct intel_guc *guc,drm_printf (p,"GuC Number Outstanding Submission G2H: %u\n" ,atomic_read (&guc->outstanding_submission_g2h)); +drm_printf (p,"GuC Number GuC IDs: %u\n" , guc->num_guc_ids); +drm_printf (p,"GuC Max GuC IDs: %u\n" , guc->max_guc_ids);drm_printf (p,"GuC tasklet count: %u\n\n" ,atomic_read (&sched_engine->tasklet.count));
@@ -2913,7 +2916,6 @@ void intel_guc_submission_print_context_info(struct intel_guc *guc, {struct intel_context *ce;unsigned long index; -xa_for_each (&guc->context_lookup, index, ce) {drm_printf (p,"GuC lrc descriptor %u:\n" , ce->guc_id);drm_printf (p,"\tHW Context Desc: 0x%08x\n" , ce->lrc.lrca);
Signature · 1 line
2.28.0
PATCH 2/46 drm/i915/guc: Connect the number of guc_ids to debugfs
In reply to Matthew BrostFor testing purposes it may make sense to reduce the number of guc_ids available to be allocated. Add debugfs support for setting the number of guc_ids.
--- .../gpu/drm/i915/gt/uc/intel_guc_debugfs.c | 31 +++++++++++++++++++ .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 3 +- 2 files changed, 33 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/gt/uc/intel_guc_debugfs.c +31 -0
@@ -50,11 +50,42 @@ static int guc_registered_contexts_show(struct seq_file *m, void *data) }DEFINE_GT_DEBUGFS_ATTRIBUTE (guc_registered_contexts); +static int guc_num_id_get (void *data,u64 *val) +{ +struct intel_guc *guc = data; + +if (!intel_guc_submission_is_used (guc)) +return -ENODEV ; + + *val = guc->num_guc_ids; + +return 0 ; +} + +static int guc_num_id_set (void *data,u64 val) +{ +struct intel_guc *guc = data; + +if (!intel_guc_submission_is_used (guc)) +return -ENODEV ; + +if (val > guc->max_guc_ids) + val = guc->max_guc_ids; +else if (val <256 ) + val =256 ; + + guc->num_guc_ids = val; + +return 0 ; +} +DEFINE_SIMPLE_ATTRIBUTE (guc_num_id_fops, guc_num_id_get, guc_num_id_set,"%lld\n" ); +void intel_guc_debugfs_register (struct intel_guc *guc,struct dentry *root) {static const struct debugfs_gt_file files[] = { {"guc_info" , &guc_info_fops,NULL }, {"guc_registered_contexts" , &guc_registered_contexts_fops,NULL }, + {"guc_num_id" , &guc_num_id_fops,NULL }, };if (!intel_guc_is_supported (guc))
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +2 -1
@@ -2574,7 +2574,8 @@ g2h_context_lookup(struct intel_guc *guc, u32 desc_idx)if (unlikely (desc_idx >= guc->max_guc_ids)) {drm_err (&guc_to_gt (guc)->i915->drm, -"Invalid desc_idx %u" , desc_idx); +"Invalid desc_idx %u, max %u" , + desc_idx, guc->max_guc_ids);return NULL ; }
Signature · 1 line
2.28.0
PATCH 3/46 drm/i915/guc: Don't return -EAGAIN to user when guc_ids exhausted
In reply to Matthew BrostRather than returning -EAGAIN to the user when no guc_ids are available, implement a fair sharing algorithm in the kernel which blocks submissons until guc_ids become available. Submissions are released one at a time, based on priority, until the guc_id pressure is released to ensure fair sharing of the guc_ids. Once the pressure is fully released, the normal guc_id allocation (at request creation time in guc_request_alloc) can resume as this allocation path should be significantly faster and a fair sharing algorithm isn't needed when guc_ids are plentifully.
The fair sharing algorithm is implemented by forcing all submissions to the tasklet which serializes submissions, dequeuing one at a time.
If the submission doesn't have a guc_id and new guc_id can't be found, two lists are searched, one list with contexts that are not pinned but still registered with the guc (searched first) and another list with contexts that are pinned but do not have any submissions actively in inflight (scheduling enabled + registered, searched second). If no guc_ids can be found we kick a workqueue which will retire requests hopefully freeing a guc_id. The workqueue + tasklet ping / pong back and forth until a guc_id can be found.
Once a guc_id is found, we may have to disable context scheduling depending on which list the context is stolen from. When we disable scheduling, we block the tasklet from executing until the completion G2H returns. The disable scheduling must be issued from the workqueue because of the locking structure. When we deregister a context, we also do the same thing (waiting on the G2H) but we can safely issue the deregister H2G from the tasklet.
Once all the G2H have returned we can trigger a submission on the context.
--- drivers/gpu/drm/i915/gt/intel_context_types.h | 3 + drivers/gpu/drm/i915/gt/uc/intel_guc.h | 26 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 805 ++++++++++++++++-- drivers/gpu/drm/i915/i915_request.h | 6 + 4 files changed, 754 insertions(+), 86 deletions(-)
drivers/gpu/drm/i915/gt/intel_context_types.h +3 -0
@@ -185,6 +185,9 @@ struct intel_context {/* GuC LRC descriptor reference count */ atomic_t guc_id_ref; +/* Number of rq submitted without a guc_id */ +u16 guc_num_rq_submit_no_id; +/* * GuC ID link - in list when unpinned but guc_id still valid in GuC */
drivers/gpu/drm/i915/gt/uc/intel_guc.h +24 -2
@@ -33,7 +33,28 @@ struct intel_guc {/* Global engine used to submit requests to GuC */ struct i915_sched_engine *sched_engine; -struct i915_request *stalled_request; + +/* Global state related to submission tasklet */ +struct i915_request *stalled_rq; +struct intel_context *stalled_context; +struct work_struct retire_worker; +unsigned long flags; +int total_num_rq_with_no_guc_id; + +/* +* Submisson stall reason. See intel_guc_submission.c for detailed +* description. +*/ +enum { +STALL_NONE , +STALL_GUC_ID_WORKQUEUE , +STALL_GUC_ID_TASKLET , +STALL_SCHED_DISABLE , +STALL_REGISTER_CONTEXT , +STALL_DEREGISTER_CONTEXT , +STALL_MOVE_LRC_TAIL , +STALL_ADD_REQUEST , + } submission_stall_reason;/* intel_guc_recv interrupt related state */ spinlock_t irq_lock;
@@ -55,7 +76,8 @@ struct intel_guc {struct ida guc_ids;u32 num_guc_ids;u32 max_guc_ids; -struct list_head guc_id_list; +struct list_head guc_id_list_no_ref; +struct list_head guc_id_list_unpinned;bool submission_supported;bool submission_selected;
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +721 -84
1,153 lines not on this page · raw message
drivers/gpu/drm/i915/i915_request.h +6 -0
@@ -139,6 +139,12 @@ enum {* the GPU. Here we track such boost requests on a per-request basis. */ I915_FENCE_FLAG_BOOST , + +/* +* I915_FENCE_FLAG_GUC_ID_NOT_PINNED - Set to signal the GuC submission +* tasklet that the guc_id isn't pinned. +*/ + I915_FENCE_FLAG_GUC_ID_NOT_PINNED, };/**
Signature · 1 line
2.28.0
Re: [PATCH 03/46] drm/i915/guc: Don't return -EAGAIN to user when guc_ids exhausted
Quoted 65 lines · Matthew Brost
Rather than returning
-EAGAINto the user when noguc_idsare available, implement a fair sharing algorithm in the kernel which blocks submissons untilguc_idsbecome available. Submissions are released one at a time, based on priority, until theguc_idpressure is released to ensure fair sharing of theguc_ids. Once the pressure is fully released, the normalguc_idallocation (at request creation time inguc_request_alloc) can resume as this allocation path should be significantly faster and a fair sharing algorithm isn't needed whenguc_idsare plentifully.The fair sharing algorithm is implemented by forcing all submissions to the tasklet which serializes submissions, dequeuing one at a time.
If the submission doesn't have a
guc_idand newguc_idcan't be found, two lists are searched, one list with contexts that are not pinned but still registered with the guc (searched first) and another list with contexts that are pinned but do not have any submissions actively in inflight (scheduling enabled + registered, searched second). If noguc_idscan be found we kick a workqueue which will retire requests hopefully freeing aguc_id. The workqueue + tasklet ping / pong back and forth until aguc_idcan be found.Once a
guc_idis found, we may have to disable context scheduling depending on which list the context is stolen from. When we disable scheduling, we block the tasklet from executing until the completion G2H returns. The disable scheduling must be issued from the workqueue because of the locking structure. When we deregister a context, we also do the same thing (waiting on the G2H) but we can safely issue the deregister H2G from the tasklet.Once all the G2H have returned we can trigger a submission on the context.
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/gt/intel_context_types.h | 3 + drivers/gpu/drm/i915/gt/uc/intel_guc.h | 26 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 805 ++++++++++++++++-- drivers/gpu/drm/i915/i915_request.h | 6 + 4 files changed, 754 insertions(+), 86 deletions(-)drivers/gpu/drm/i915/gt/intel_context_types.h +3
-0@@ -185,6 +185,9 @@ struct intel_context {/* GuC LRC descriptor reference count */ atomic_t guc_id_ref; +/* Number of rq submitted without a guc_id */ +u16 guc_num_rq_submit_no_id; +/* * GuC ID link - in list when unpinned but guc_id still valid in GuC */ drivers/gpu/drm/i915/gt/uc/intel_guc.h +2
-1@@ -33,7 +33,28 @@ struct intel_guc {/* Global engine used to submit requests to GuC */ struct i915_sched_engine *sched_engine; -struct i915_request *stalled_request; + +/* Global state related to submission tasklet */
drivers/gpu/drm/i915/gt/uc/intel_guc.h struct intel_guc { · line 47
Quoted 10 lines
drivers/gpu/drm/i915/gt/uc/intel_guc.h +10
-0+struct i915_request *stalled_rq; +struct intel_context *stalled_context; +struct work_struct retire_worker; +unsigned long flags; +int total_num_rq_with_no_guc_id; + +/* +* Submisson stall reason. See intel_guc_submission.c for detailed +* description. +*/
I think documenting this kind of stuff inline as kerneldoc is neater, and closer to where it's generally needed. Source navigation tools point you to here, not the comment that's burried somewhere else.
Quoted 58 lines
drivers/gpu/drm/i915/gt/uc/intel_guc.h +12
-1+enum { +STALL_NONE , +STALL_GUC_ID_WORKQUEUE , +STALL_GUC_ID_TASKLET , +STALL_SCHED_DISABLE , +STALL_REGISTER_CONTEXT , +STALL_DEREGISTER_CONTEXT , +STALL_MOVE_LRC_TAIL , +STALL_ADD_REQUEST , + } submission_stall_reason;/* intel_guc_recv interrupt related state */ spinlock_t irq_lock;@@ -55,7 +76,8 @@ struct intel_guc {struct ida guc_ids;u32 num_guc_ids;u32 max_guc_ids; -struct list_head guc_id_list; +struct list_head guc_id_list_no_ref; +struct list_head guc_id_list_unpinned;bool submission_supported;bool submission_selected;drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +20
-0@@ -59,6 +59,25 @@* ELSP context descriptor dword into Work Item. * See guc_add_request() * +* GuC flow control state machine: +* The tasklet, workqueue (retire_worker), and the G2H handlers together more or +* less form a state machine which is used to submit requests + flow control +* requests, while waiting on resources / actions, if necessary. The enum, +* submission_stall_reason, controls the handoff of stalls between these +* entities with stalled_rq & stalled_context being the arguments. Each state +* described below. +* +* STALL_NONE No stall condition +* STALL_GUC_ID_WORKQUEUE Workqueue will try to free guc_ids +* STALL_GUC_ID_TASKLET Tasklet will try to find guc_id +* STALL_SCHED_DISABLE Workqueue will issue context schedule disable +* H2G +* STALL_REGISTER_CONTEXT Tasklet needs to register context +* STALL_DEREGISTER_CONTEXT G2H handler is waiting for context deregister, +* will register context upon receipt of G2H +* STALL_MOVE_LRC_TAIL Tasklet will try to move LRC tail +* STALL_ADD_REQUEST Tasklet will try to add the request (submit +* context) */ /* GuC Virtual Engine */ @@ -72,6 +91,83 @@ guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count);#define GUC_REQUEST_SIZE 64/* bytes */ +/*
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count); · line 104
Quoted 10 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +10
-0+* Global GuC flags helper functions +*/ +enum { +GUC_STATE_TASKLET_BLOCKED , +GUC_STATE_GUC_IDS_EXHAUSTED , +}; + +static bool tasklet_blocked (struct intel_guc *guc) +{ +return test_bit (GUC_STATE_TASKLET_BLOCKED , &guc->flags);
So I know the existing code absolutely loves it's atomic bitflags for a state machine in every corner, but that doesn't make it a good idea.
Roughly the justification needs to be:
- first try to get rid of your state machine as much as possible. Any transition or state you can remove is good
- next use big dumb locks for your state machine and state. Stuff like wrap your entire tasklet into a mutex.
- if you do smaller locks you already have to think way too hard about ordering and what happens if everything gets reorderd/delayed in unexpected ways and how things slip through. That needs substantial comments already if it's not some obvious pattern. Obvious here means you're just using established existing primitives like queue_work() and flush_work to manage your state machine (not sure what tasklets all provide here, but probably worth a look to make sure we're not reinventing wheels)
- if smaller locks are not good enough from a perf pov, then a) you must supply perf data to justify your b) the commit message really needs to explain why all this complexity is needed and your review requirements just went through the roof.
- anything that doesn't use locks and doesn't use estabslished well-understood primitives needs barriers. Those barriers each need a comment which explains where the counterparty is. If this doesn't feel like you're writing a small academic paper, you're not doing it right. Note that with dgpu we can't shrug this all off with "IA is a TSO, reordering never happens". Also on a TSO the compiler is still allowed ro reorder everything
Given that "guc is stalling" isn't really a perf critical thing I think this needs substantially dumbing down, or substantially more benchmark data proving that we need it.
And no there's simply no budget anymore in i915-gem for frivolous use of a bit of overengineered lockless stuff, we have way, way too much of that. We need to reduce this, not add more.
-Daniel
Quoted 1,174 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +691
-84+} + +static void set_tasklet_blocked (struct intel_guc *guc) +{ +lockdep_assert_held (&guc->sched_engine->lock); +set_bit (GUC_STATE_TASKLET_BLOCKED , &guc->flags); +} + +static void __clr_tasklet_blocked (struct intel_guc *guc) +{ +lockdep_assert_held (&guc->sched_engine->lock); +clear_bit (GUC_STATE_TASKLET_BLOCKED , &guc->flags); +} + +static void clr_tasklet_blocked (struct intel_guc *guc) +{ +unsigned long flags; + +spin_lock_irqsave (&guc->sched_engine->lock, flags); +__clr_tasklet_blocked (guc); +spin_unlock_irqrestore (&guc->sched_engine->lock, flags); +} + +static bool guc_ids_exhausted (struct intel_guc *guc) +{ +return test_bit (GUC_STATE_GUC_IDS_EXHAUSTED , &guc->flags); +} + +static bool test_and_update_guc_ids_exhausted (struct intel_guc *guc) +{ +unsigned long flags; +bool ret = false; + +/* +* Strict ordering on checking if guc_ids are exhausted isn't required, +* so let's avoid grabbing the submission lock if possible. +*/ +if (guc_ids_exhausted (guc)) { +spin_lock_irqsave (&guc->sched_engine->lock, flags); + ret =guc_ids_exhausted (guc); +if (ret) + ++guc->total_num_rq_with_no_guc_id; +spin_unlock_irqrestore (&guc->sched_engine->lock, flags); + } + +return ret; +} + +static void set_and_update_guc_ids_exhausted (struct intel_guc *guc) +{ +unsigned long flags; + +spin_lock_irqsave (&guc->sched_engine->lock, flags); + ++guc->total_num_rq_with_no_guc_id; +set_bit (GUC_STATE_GUC_IDS_EXHAUSTED , &guc->flags); +spin_unlock_irqrestore (&guc->sched_engine->lock, flags); +} + +static void clr_guc_ids_exhausted (struct intel_guc *guc) +{ +lockdep_assert_held (&guc->sched_engine->lock); +GEM_BUG_ON (guc->total_num_rq_with_no_guc_id); + +clear_bit (GUC_STATE_GUC_IDS_EXHAUSTED , &guc->flags); +} +/* * Below is a set of functions which control the GuC scheduling state which do * not require a lock as all state transitions are mutually exclusive. i.e. It @@ -82,6 +178,9 @@ guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count);#define SCHED_STATE_NO_LOCK_ENABLED BIT(0)#define SCHED_STATE_NO_LOCK_PENDING_ENABLE BIT(1)#define SCHED_STATE_NO_LOCK_REGISTERED BIT(2) +#define SCHED_STATE_NO_LOCK_BLOCK_TASKLET BIT(3) +#define SCHED_STATE_NO_LOCK_GUC_ID_STOLEN BIT(4) +#define SCHED_STATE_NO_LOCK_NEEDS_REGISTER BIT(5)static inline bool context_enabled (struct intel_context *ce) { return (atomic_read(&ce->guc_sched_state_no_lock) &@@ -135,6 +234,60 @@ static inline void clr_context_registered(struct intel_context *ce) &ce->guc_sched_state_no_lock); } +static inline bool context_block_tasklet (struct intel_context *ce) +{ +return (atomic_read (&ce->guc_sched_state_no_lock) & +SCHED_STATE_NO_LOCK_BLOCK_TASKLET ); +} + +static inline void set_context_block_tasklet (struct intel_context *ce) +{ +atomic_or (SCHED_STATE_NO_LOCK_BLOCK_TASKLET , + &ce->guc_sched_state_no_lock); +} + +static inline void clr_context_block_tasklet (struct intel_context *ce) +{ +atomic_and ((u32 )~SCHED_STATE_NO_LOCK_BLOCK_TASKLET , + &ce->guc_sched_state_no_lock); +} + +static inline bool context_guc_id_stolen (struct intel_context *ce) +{ +return (atomic_read (&ce->guc_sched_state_no_lock) & +SCHED_STATE_NO_LOCK_GUC_ID_STOLEN ); +} + +static inline void set_context_guc_id_stolen (struct intel_context *ce) +{ +atomic_or (SCHED_STATE_NO_LOCK_GUC_ID_STOLEN , + &ce->guc_sched_state_no_lock); +} + +static inline void clr_context_guc_id_stolen (struct intel_context *ce) +{ +atomic_and ((u32 )~SCHED_STATE_NO_LOCK_GUC_ID_STOLEN , + &ce->guc_sched_state_no_lock); +} + +static inline bool context_needs_register (struct intel_context *ce) +{ +return (atomic_read (&ce->guc_sched_state_no_lock) & +SCHED_STATE_NO_LOCK_NEEDS_REGISTER ); +} + +static inline void set_context_needs_register (struct intel_context *ce) +{ +atomic_or (SCHED_STATE_NO_LOCK_NEEDS_REGISTER , + &ce->guc_sched_state_no_lock); +} + +static inline void clr_context_needs_register (struct intel_context *ce) +{ +atomic_and ((u32 )~SCHED_STATE_NO_LOCK_NEEDS_REGISTER , + &ce->guc_sched_state_no_lock); +} +/* * Below is a set of functions which control the GuC scheduling state which * require a lock, aside from the special case where the functions are called @@ -418,9 +571,12 @@ int intel_guc_wait_for_idle(struct intel_guc *guc, long timeout) true, timeout); } -static int guc_lrc_desc_pin (struct intel_context *ce,bool loop); +static inline bool request_has_no_guc_id (struct i915_request *rq) +{ +return test_bit (I915_FENCE_FLAG_GUC_ID_NOT_PINNED , &rq->fence.flags); +} -static int guc_add_request (struct intel_guc *guc,struct i915_request *rq) +static int __guc_add_request (struct intel_guc *guc,struct i915_request *rq) {int err =0 ;struct intel_context *ce = rq->context;@@ -439,18 +595,15 @@ static int guc_add_request(struct intel_guc *guc, struct i915_request *rq) goto out; } +/* Ensure context is in correct state before a submission */ +GEM_BUG_ON (ce->guc_num_rq_submit_no_id); +GEM_BUG_ON (request_has_no_guc_id (rq));GEM_BUG_ON (!atomic_read (&ce->guc_id_ref)); +GEM_BUG_ON (context_needs_register (ce));GEM_BUG_ON (context_guc_id_invalid (ce)); - -/* -* Corner case where the GuC firmware was blown away and reloaded while -* this context was pinned. -*/ -if (unlikely (!lrc_desc_registered (guc, ce->guc_id))) { - err =guc_lrc_desc_pin (ce, false); -if (unlikely (err)) - goto out; - } +GEM_BUG_ON (context_pending_disable (ce)); +GEM_BUG_ON (context_wait_for_deregister_to_register (ce)); +GEM_BUG_ON (!lrc_desc_registered (guc, ce->guc_id));/* * The request / context will be run on the hardware when scheduling @@ -462,6 +615,8 @@ static int guc_add_request(struct intel_guc *guc, struct i915_request *rq) enabled =context_enabled (ce);if (!enabled) { +GEM_BUG_ON (context_pending_enable (ce)); + action[len++] =INTEL_GUC_ACTION_SCHED_CONTEXT_MODE_SET ; action[len++] = ce->guc_id; action[len++] =GUC_CONTEXT_ENABLE ;@@ -489,6 +644,67 @@ static int guc_add_request(struct intel_guc *guc, struct i915_request *rq)return err; } +static int guc_add_request (struct intel_guc *guc,struct i915_request *rq) +{ +int ret; + +lockdep_assert_held (&guc->sched_engine->lock); + + ret =__guc_add_request (guc, rq); +if (ret == -EBUSY ) { + guc->stalled_rq = rq; + guc->submission_stall_reason =STALL_ADD_REQUEST ; + }else { + guc->stalled_rq =NULL ; + guc->submission_stall_reason =STALL_NONE ; + } + +return ret; +} + +static int guc_lrc_desc_pin (struct intel_context *ce,bool loop); + +static int tasklet_register_context (struct intel_guc *guc, +struct i915_request *rq) +{ +struct intel_context *ce = rq->context; +int ret =0 ; + +/* Check state */ +lockdep_assert_held (&guc->sched_engine->lock); +GEM_BUG_ON (ce->guc_num_rq_submit_no_id); +GEM_BUG_ON (request_has_no_guc_id (rq)); +GEM_BUG_ON (context_guc_id_invalid (ce)); +GEM_BUG_ON (!atomic_read (&ce->guc_id_ref)); + +/* +* The guc_id is getting pinned during the tasklet and we need to +* register this context or a corner case where the GuC firmware was +* blown away and reloaded while this context was pinned +*/ +if (unlikely ((!lrc_desc_registered (guc, ce->guc_id) || +context_needs_register (ce)) && + !intel_context_is_banned (ce))) { +GEM_BUG_ON (context_pending_disable (ce)); +GEM_BUG_ON (context_wait_for_deregister_to_register (ce)); + + ret =guc_lrc_desc_pin (ce, false); + +if (likely (ret != -EBUSY )) +clr_context_needs_register (ce); + +if (unlikely (ret == -EBUSY )) { + guc->stalled_rq = rq; + guc->submission_stall_reason =STALL_REGISTER_CONTEXT ; + }else if (unlikely (ret == -EINPROGRESS )) { + guc->stalled_rq = rq; + guc->submission_stall_reason =STALL_DEREGISTER_CONTEXT ; + } + } + +return ret; +} +static inline void guc_set_lrc_tail (struct i915_request *rq) { rq->context->lrc_reg_state[CTX_RING_TAIL] =@@ -500,77 +716,142 @@ static inline int rq_prio(const struct i915_request *rq)return rq->sched.attr.priority; } +static void kick_retire_wq (struct intel_guc *guc) +{ +queue_work (system_unbound_wq, &guc->retire_worker); +} + +static int tasklet_pin_guc_id (struct intel_guc *guc,struct i915_request *rq); +static int guc_dequeue_one_context (struct intel_guc *guc) {struct i915_sched_engine *const sched_engine = guc->sched_engine; -struct i915_request *last =NULL ; -bool submit = false; +struct i915_request *last = guc->stalled_rq; +bool submit = !!last;struct rb_node *rb;int ret;lockdep_assert_held (&sched_engine->lock); +GEM_BUG_ON (guc->stalled_context); +GEM_BUG_ON (!submit && guc->submission_stall_reason); -if (guc->stalled_request) { - submit = true; - last = guc->stalled_request; - goto resubmit; - } +if (submit) { +/* Flow control conditions */ +switch (guc->submission_stall_reason) { +case STALL_GUC_ID_TASKLET : + goto done; +case STALL_REGISTER_CONTEXT : + goto register_context; +case STALL_MOVE_LRC_TAIL : + goto move_lrc_tail; +case STALL_ADD_REQUEST : + goto add_request; +default : +GEM_BUG_ON ("Invalid stall state" ); + } + }else { +GEM_BUG_ON (!guc->total_num_rq_with_no_guc_id && +guc_ids_exhausted (guc)); -while ((rb =rb_first_cached (&sched_engine->queue))) { -struct i915_priolist *p =to_priolist (rb); -struct i915_request *rq, *rn; +while ((rb =rb_first_cached (&sched_engine->queue))) { +struct i915_priolist *p =to_priolist (rb); +struct i915_request *rq, *rn; -priolist_for_each_request_consume (rq, rn, p) { -if (last && rq->context != last->context) - goto done; +priolist_for_each_request_consume (rq, rn, p) { +if (last && rq->context != last->context) + goto done; -list_del_init (&rq->sched.link); +list_del_init (&rq->sched.link); -__i915_request_submit (rq); +__i915_request_submit (rq); -trace_i915_request_in (rq,0 ); - last = rq; - submit = true; - } +trace_i915_request_in (rq,0 ); + last = rq; + submit = true; + } -rb_erase_cached (&p->node, &sched_engine->queue); -i915_priolist_free (p); +rb_erase_cached (&p->node, &sched_engine->queue); +i915_priolist_free (p); + } } + done:if (submit) { +struct intel_context *ce = last->context; + +if (ce->guc_num_rq_submit_no_id) { + ret =tasklet_pin_guc_id (guc, last); +if (ret) + goto blk_tasklet_kick; + } + +register_context: + ret =tasklet_register_context (guc, last); +if (unlikely (ret == -EINPROGRESS )) { + goto blk_tasklet; + }else if (unlikely (ret == -EPIPE )) { + goto deadlk; + }else if (ret == -EBUSY ) { + goto schedule_tasklet; + }else if (unlikely (ret !=0 )) { +GEM_WARN_ON (ret);/* Unexpected */ + goto deadlk; + } + +move_lrc_tail:guc_set_lrc_tail (last); -resubmit: + +add_request: ret =guc_add_request (guc, last); -if (unlikely (ret == -EPIPE )) +if (unlikely (ret == -EPIPE )) { + goto deadlk; + }else if (ret == -EBUSY ) { + goto schedule_tasklet; + }else if (unlikely (ret !=0 )) { +GEM_WARN_ON (ret);/* Unexpected */ goto deadlk; -else if (ret == -EBUSY ) { -tasklet_schedule (&sched_engine->tasklet); - guc->stalled_request = last; -return false; } } - guc->stalled_request =NULL ; +/* +* No requests without a guc_id, enable guc_id allocation at request +* creation time (guc_request_alloc). +*/ +if (!guc->total_num_rq_with_no_guc_id) +clr_guc_ids_exhausted (guc); +return submit; +schedule_tasklet: +tasklet_schedule (&sched_engine->tasklet); +return false; + deadlk: sched_engine->tasklet.callback =NULL ;tasklet_disable_nosync (&sched_engine->tasklet);return false; + +blk_tasklet_kick: +kick_retire_wq (guc); +blk_tasklet: +set_tasklet_blocked (guc); +return false; }static void guc_submission_tasklet (struct tasklet_struct *t) {struct i915_sched_engine *sched_engine =from_tasklet (sched_engine, t, tasklet); +struct intel_guc *guc = sched_engine->private_data;unsigned long flags;bool loop;spin_lock_irqsave (&sched_engine->lock, flags); -do { - loop =guc_dequeue_one_context (sched_engine->private_data); - }while (loop); +if (likely (!tasklet_blocked (guc))) +do { + loop =guc_dequeue_one_context (guc); + }while (loop);i915_sched_engine_reset_on_empty (sched_engine);@@ -653,6 +934,14 @@ submission_disabled(struct intel_guc *guc) !__tasklet_is_enabled(&sched_engine->tasklet)); } +static void kick_tasklet (struct intel_guc *guc) +{ +struct i915_sched_engine *const sched_engine = guc->sched_engine; + +if (likely (!tasklet_blocked (guc))) +tasklet_hi_schedule (&sched_engine->tasklet); +} +static void disable_submission (struct intel_guc *guc) {struct i915_sched_engine *const sched_engine = guc->sched_engine;@@ -676,8 +965,16 @@ static void enable_submission(struct intel_guc *guc) __tasklet_enable(&sched_engine->tasklet)) {GEM_BUG_ON (!guc->ct.enabled); +/* Reset tasklet state */ + guc->stalled_rq =NULL ; +if (guc->stalled_context) +intel_context_put (guc->stalled_context); + guc->stalled_context =NULL ; + guc->submission_stall_reason =STALL_NONE ; + guc->flags =0 ; +/* And kick in case we missed a new request submission. */ -tasklet_hi_schedule (&sched_engine->tasklet); +kick_tasklet (guc); }spin_unlock_irqrestore (&guc->sched_engine->lock, flags); }@@ -856,6 +1153,7 @@ static void __guc_reset_context(struct intel_context *ce, bool stalled) out_replay:guc_reset_state (ce, head, stalled);__unwind_incomplete_requests (ce); + ce->guc_num_rq_submit_no_id =0 ;intel_context_put (ce); }@@ -888,6 +1186,7 @@ static void guc_cancel_context_requests(struct intel_context *ce)spin_lock (&ce->guc_active.lock);list_for_each_entry (rq, &ce->guc_active.requests, sched.link)i915_request_put (i915_request_mark_eio (rq)); + ce->guc_num_rq_submit_no_id =0 ;spin_unlock (&ce->guc_active.lock);spin_unlock_irqrestore (&sched_engine->lock, flags); }@@ -924,11 +1223,15 @@ guc_cancel_sched_engine_requests(struct i915_sched_engine *sched_engine)struct i915_priolist *p =to_priolist (rb);priolist_for_each_request_consume (rq, rn, p) { +struct intel_context *ce = rq->context; +list_del_init (&rq->sched.link);__i915_request_submit (rq);i915_request_put (i915_request_mark_eio (rq)); + + ce->guc_num_rq_submit_no_id =0 ; }rb_erase_cached (&p->node, &sched_engine->queue);@@ -980,6 +1283,51 @@ void intel_guc_submission_reset_finish(struct intel_guc *guc)intel_gt_unpark_heartbeats (guc_to_gt (guc)); } +static void retire_worker_sched_disable (struct intel_guc *guc, +struct intel_context *ce); + +static void retire_worker_func (struct work_struct *w) +{ +struct intel_guc *guc = + container_of(w, struct intel_guc, retire_worker); + +/* +* It is possible that another thread issues the schedule disable + that +* G2H completes moving the state machine further along to a point +* where nothing needs to be done here. Let's be paranoid and kick the +* tasklet in that case. +*/ +if (guc->submission_stall_reason !=STALL_SCHED_DISABLE && + guc->submission_stall_reason !=STALL_GUC_ID_WORKQUEUE ) { +kick_tasklet (guc); +return ; + } + +if (guc->submission_stall_reason ==STALL_SCHED_DISABLE ) { +GEM_BUG_ON (!guc->stalled_context); +GEM_BUG_ON (context_guc_id_invalid (guc->stalled_context)); + +retire_worker_sched_disable (guc, guc->stalled_context); + } + +/* +* guc_id pressure, always try to release it regardless of state, +* albeit after possibly issuing a schedule disable as that is async +* operation. +*/ +intel_gt_retire_requests (guc_to_gt (guc)); + +if (guc->submission_stall_reason ==STALL_GUC_ID_WORKQUEUE ) { +GEM_BUG_ON (guc->stalled_context); + +/* Hopefully guc_ids are now available, kick tasklet */ + guc->submission_stall_reason =STALL_GUC_ID_TASKLET ; +clr_tasklet_blocked (guc); + +kick_tasklet (guc); + } +} +/* * Set up the memory resources to be shared with the GuC (via the GGTT) * at firmware loading time. @@ -1003,9 +1351,12 @@ int intel_guc_submission_init(struct intel_guc *guc)xa_init_flags (&guc->context_lookup,XA_FLAGS_LOCK_IRQ );spin_lock_init (&guc->contexts_lock); -INIT_LIST_HEAD (&guc->guc_id_list); +INIT_LIST_HEAD (&guc->guc_id_list_no_ref); +INIT_LIST_HEAD (&guc->guc_id_list_unpinned);ida_init (&guc->guc_ids); +INIT_WORK (&guc->retire_worker, retire_worker_func); +return 0 ; }@@ -1022,10 +1373,28 @@ static inline void queue_request(struct i915_sched_engine *sched_engine,struct i915_request *rq, int prio) { + bool empty = i915_sched_engine_is_empty(sched_engine); +GEM_BUG_ON (!list_empty (&rq->sched.link));list_add_tail (&rq->sched.link,i915_sched_lookup_priolist (sched_engine, prio));set_bit (I915_FENCE_FLAG_PQUEUE , &rq->fence.flags); + +if (empty) +kick_tasklet (&rq->engine->gt->uc.guc); +} + +static bool need_tasklet (struct intel_guc *guc,struct intel_context *ce) +{ +struct i915_sched_engine *const sched_engine = + ce->engine->sched_engine; + +lockdep_assert_held (&sched_engine->lock); + +return guc_ids_exhausted (guc) ||submission_disabled (guc) || + guc->stalled_rq || guc->stalled_context || + !lrc_desc_registered (guc, ce->guc_id) || + !i915_sched_engine_is_empty (sched_engine); } static int guc_bypass_tasklet_submit(struct intel_guc *guc,@@ -1039,8 +1408,6 @@ static int guc_bypass_tasklet_submit(struct intel_guc *guc,guc_set_lrc_tail (rq); ret =guc_add_request (guc, rq); -if (ret == -EBUSY ) - guc->stalled_request = rq;if (unlikely (ret == -EPIPE ))disable_submission (guc);@@ -1057,11 +1424,10 @@ static void guc_submit_request(struct i915_request *rq)/* Will be called from irq-context when using foreign fences. */ spin_lock_irqsave (&sched_engine->lock, flags); -if (submission_disabled (guc) || guc->stalled_request || - !i915_sched_engine_is_empty (sched_engine)) +if (need_tasklet (guc, rq->context))queue_request (sched_engine, rq,rq_prio (rq));else if (guc_bypass_tasklet_submit (guc, rq) == -EBUSY ) -tasklet_hi_schedule (&sched_engine->tasklet); +kick_tasklet (guc);spin_unlock_irqrestore (&sched_engine->lock, flags); }@@ -1093,32 +1459,71 @@ static void release_guc_id(struct intel_guc *guc, struct intel_context *ce)spin_unlock_irqrestore (&guc->contexts_lock, flags); } -static int steal_guc_id (struct intel_guc *guc) +/* +* We have two lists for guc_ids available to steal. One list is for contexts +* that to have a zero guc_id_ref but are still pinned (scheduling enabled, only +* available inside tasklet) and the other is for contexts that are not pinned +* but still registered (available both outside and inside tasklet). Stealing +* from the latter only requires a deregister H2G, while the former requires a +* schedule disable H2G + a deregister H2G. +*/ +static struct list_head *get_guc_id_list (struct intel_guc *guc, +bool unpinned) +{ +if (unpinned) +return &guc->guc_id_list_unpinned; +else +return &guc->guc_id_list_no_ref; +} + +static int steal_guc_id (struct intel_guc *guc,bool unpinned) {struct intel_context *ce;int guc_id; +struct list_head *guc_id_list =get_guc_id_list (guc, unpinned);lockdep_assert_held (&guc->contexts_lock); -if (!list_empty (&guc->guc_id_list)) { - ce =list_first_entry (&guc->guc_id_list, +if (!list_empty (guc_id_list)) { + ce =list_first_entry (guc_id_list, struct intel_context, guc_id_link); +/* Ensure context getting stolen in expected state */ GEM_BUG_ON (atomic_read (&ce->guc_id_ref));GEM_BUG_ON (context_guc_id_invalid (ce)); +GEM_BUG_ON (context_guc_id_stolen (ce));list_del_init (&ce->guc_id_link); guc_id = ce->guc_id;clr_context_registered (ce); -set_context_guc_id_invalid (ce); + +/* +* If stealing from the pinned list, defer invalidating +* the guc_id until the retire workqueue processes this +* context. +*/ +if (!unpinned) { +GEM_BUG_ON (guc->stalled_context); + guc->stalled_context =intel_context_get (ce); +set_context_guc_id_stolen (ce); + }else { +set_context_guc_id_invalid (ce); + } +return guc_id; }else {return -EAGAIN ; } } -static int assign_guc_id (struct intel_guc *guc,u16 *out) +enum {/* Return values for pin_guc_id / assign_guc_id */ +SAME_GUC_ID =0 , +NEW_GUC_ID_DISABLED =1 , +NEW_GUC_ID_ENABLED =2 , +}; + +static int assign_guc_id (struct intel_guc *guc,u16 *out,bool tasklet) {int ret;@@ -1126,17 +1531,33 @@ static int assign_guc_id(struct intel_guc *guc, u16 *out) ret =new_guc_id (guc);if (unlikely (ret <0 )) { - ret =steal_guc_id (guc); -if (ret <0 ) -return ret; + ret =steal_guc_id (guc, true); +if (ret >=0 ) { + *out = ret; + ret =NEW_GUC_ID_DISABLED ; + }else if (ret <0 && tasklet) { +/* +* We only steal a guc_id from a context with scheduling +* enabled if guc_ids are exhausted and we are submitting +* from the tasklet. +*/ + ret =steal_guc_id (guc, false); +if (ret >=0 ) { + *out = ret; + ret =NEW_GUC_ID_ENABLED ; + } + } + }else { + *out = ret; + ret =SAME_GUC_ID ; } - *out = ret; -return 0 ; +return ret; }#define PIN_GUC_ID_TRIES 4 -static int pin_guc_id (struct intel_guc *guc,struct intel_context *ce) +static int pin_guc_id (struct intel_guc *guc,struct intel_context *ce, +bool tasklet) {int ret =0 ;unsigned long flags, tries =PIN_GUC_ID_TRIES ;@@ -1146,11 +1567,15 @@ static int pin_guc_id(struct intel_guc *guc, struct intel_context *ce) try_again:spin_lock_irqsave (&guc->contexts_lock, flags); +if (!tasklet &&guc_ids_exhausted (guc)) { + ret = -EAGAIN ; + goto out_unlock; + } +if (context_guc_id_invalid (ce)) { - ret =assign_guc_id (guc, &ce->guc_id); -if (ret) + ret =assign_guc_id (guc, &ce->guc_id, tasklet); +if (unlikely (ret <0 )) goto out_unlock; - ret =1 ;/* Indidcates newly assigned guc_id */ }if (!list_empty (&ce->guc_id_link))list_del_init (&ce->guc_id_link);@@ -1166,8 +1591,11 @@ static int pin_guc_id(struct intel_guc *guc, struct intel_context *ce)* attempting to retire more requests. Double the sleep period each * subsequent pass before finally giving up. The sleep period has max of * 100ms and minimum of 1ms. +* +* We only try this if outside the tasklet, inside the tasklet we have a +* (slower, more complex, blocking) different flow control algorithm. */ -if (ret == -EAGAIN && --tries) { +if (ret == -EAGAIN && --tries && !tasklet) {if (PIN_GUC_ID_TRIES - tries >1 ) {unsigned int timeslice_shifted = ce->engine->props.timeslice_duration_ms <<@@ -1184,7 +1612,9 @@ static int pin_guc_id(struct intel_guc *guc, struct intel_context *ce)return ret; } -static void unpin_guc_id (struct intel_guc *guc,struct intel_context *ce) +static void unpin_guc_id (struct intel_guc *guc, +struct intel_context *ce, +bool unpinned) {unsigned long flags;@@ -1194,9 +1624,17 @@ static void unpin_guc_id(struct intel_guc *guc, struct intel_context *ce)return ;spin_lock_irqsave (&guc->contexts_lock, flags); -if (!context_guc_id_invalid (ce) &&list_empty (&ce->guc_id_link) && - !atomic_read (&ce->guc_id_ref)) -list_add_tail (&ce->guc_id_link, &guc->guc_id_list); + +if (!list_empty (&ce->guc_id_link)) +list_del_init (&ce->guc_id_link); + +if (!context_guc_id_invalid (ce) && !context_guc_id_stolen (ce) && + !atomic_read (&ce->guc_id_ref)) { +struct list_head *head =get_guc_id_list (guc, unpinned); + +list_add_tail (&ce->guc_id_link, head); + } +spin_unlock_irqrestore (&guc->contexts_lock, flags); }@@ -1300,6 +1738,7 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)int ret =0 ;GEM_BUG_ON (!engine->mask); +GEM_BUG_ON (context_guc_id_invalid (ce));/* * Ensure LRC + CT vmas are is same region as write barrier is done @@ -1342,6 +1781,7 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)trace_intel_context_steal_guc_id (ce);if (!loop) {set_context_wait_for_deregister_to_register (ce); +set_context_block_tasklet (ce);intel_context_get (ce); }else {bool disabled;@@ -1369,7 +1809,14 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop) ret =deregister_context (ce, ce->guc_id, loop);if (unlikely (ret == -EBUSY )) {clr_context_wait_for_deregister_to_register (ce); +clr_context_block_tasklet (ce);intel_context_put (ce); + }else if (!loop && !ret) { +/* +* A context de-registration has been issued from within +* the tasklet. Need to block until it complete. +*/ +return -EINPROGRESS ; }else if (unlikely (ret == -ENODEV )) { ret =0 ;/* Will get registered later */ }@@ -1425,7 +1872,9 @@ static void guc_context_unpin(struct intel_context *ce) {struct intel_guc *guc =ce_to_guc (ce); -unpin_guc_id (guc, ce); +GEM_BUG_ON (context_enabled (ce)); + +unpin_guc_id (guc, ce, true);lrc_unpin (ce); }@@ -1764,6 +2213,8 @@ static void guc_context_destroy(struct kref *kref)unsigned long flags;bool disabled; +GEM_BUG_ON (context_guc_id_stolen (ce)); +/* * If the guc_id is invalid this context has been stolen and we can free * it immediately. Also can be freed immediately if the context is not @@ -1925,6 +2376,9 @@ static void add_to_context(struct i915_request *rq)spin_lock (&ce->guc_active.lock);list_move_tail (&rq->sched.link, &ce->guc_active.requests); +if (unlikely (request_has_no_guc_id (rq))) + ++ce->guc_num_rq_submit_no_id; +if (rq->guc_prio ==GUC_PRIO_INIT ) { rq->guc_prio = new_guc_prio;add_context_inflight_prio (ce, rq->guc_prio);@@ -1966,7 +2420,12 @@ static void remove_from_context(struct i915_request *rq)spin_unlock_irq (&ce->guc_active.lock); -atomic_dec (&ce->guc_id_ref); +if (likely (!request_has_no_guc_id (rq))) +atomic_dec (&ce->guc_id_ref); +else + --ce_to_guc (rq->context)->total_num_rq_with_no_guc_id; +unpin_guc_id (ce_to_guc (ce), ce, false); +i915_request_notify_execute_cb_imm (rq); }@@ -2018,13 +2477,144 @@ static void guc_signal_context_fence(struct intel_context *ce)spin_unlock_irqrestore (&ce->guc_state.lock, flags); } -static bool context_needs_register (struct intel_context *ce,bool new_guc_id) +static void invalidate_guc_id_sched_disable (struct intel_context *ce) +{ +set_context_guc_id_invalid (ce); +wmb ();/* Make sure guc_id invalidation visible first */ +clr_context_guc_id_stolen (ce); +} + +static void retire_worker_sched_disable (struct intel_guc *guc, +struct intel_context *ce) +{ +unsigned long flags; +bool disabled; + + guc->stalled_context =NULL ; +spin_lock_irqsave (&ce->guc_state.lock, flags); + disabled =submission_disabled (guc); +if (!disabled && !context_pending_disable (ce) &&context_enabled (ce)) { +/* +* Still enabled, issue schedule disable + configure state so +* when G2H returns tasklet is kicked. +*/ + +struct intel_runtime_pm *runtime_pm = + &ce->engine->gt->i915->runtime_pm; +intel_wakeref_t wakeref; +u16 guc_id; + +/* +* We add +2 here as the schedule disable complete CTB handler +* calls intel_context_sched_disable_unpin (-2 to pin_count). +*/ +GEM_BUG_ON (!atomic_read (&ce->pin_count)); +atomic_add (2 , &ce->pin_count); + +set_context_block_tasklet (ce); + guc_id =prep_context_pending_disable (ce); +spin_unlock_irqrestore (&ce->guc_state.lock, flags); + + with_intel_runtime_pm(runtime_pm, wakeref) +__guc_context_sched_disable (guc, ce, guc_id); + +invalidate_guc_id_sched_disable (ce); + }else if (!disabled &&context_pending_disable (ce)) { +/* +* Schedule disable in flight, set bit to kick tasklet in G2H +* handler and call it a day. +*/ + +set_context_block_tasklet (ce); +spin_unlock_irqrestore (&ce->guc_state.lock, flags); + +invalidate_guc_id_sched_disable (ce); + }else { +/* Schedule disable is done, kick tasklet */ + +spin_unlock_irqrestore (&ce->guc_state.lock, flags); + +invalidate_guc_id_sched_disable (ce); + + guc->submission_stall_reason =STALL_REGISTER_CONTEXT ; +clr_tasklet_blocked (guc); + +kick_tasklet (ce_to_guc (ce)); + } + +intel_context_put (ce); +} + +static bool context_needs_lrc_desc_pin (struct intel_context *ce,bool new_guc_id) {return (new_guc_id ||test_bit (CONTEXT_LRCA_DIRTY , &ce->flags) || !lrc_desc_registered (ce_to_guc (ce), ce->guc_id)) && !submission_disabled (ce_to_guc (ce)); } +static int tasklet_pin_guc_id (struct intel_guc *guc,struct i915_request *rq) +{ +struct intel_context *ce = rq->context; +int ret =0 ; + +lockdep_assert_held (&guc->sched_engine->lock); +GEM_BUG_ON (!ce->guc_num_rq_submit_no_id); + +if (atomic_add_unless (&ce->guc_id_ref, ce->guc_num_rq_submit_no_id,0 )) + goto out; + + ret =pin_guc_id (guc, ce, true); +if (unlikely (ret <0 )) { +/* +* No guc_ids available, disable the tasklet and kick the retire +* workqueue hopefully freeing up some guc_ids. +*/ + guc->stalled_rq = rq; + guc->submission_stall_reason =STALL_GUC_ID_WORKQUEUE ; +return ret; + } + +if (ce->guc_num_rq_submit_no_id -1 >0 ) +atomic_add (ce->guc_num_rq_submit_no_id -1 , + &ce->guc_id_ref); + +if (context_needs_lrc_desc_pin (ce, !!ret)) +set_context_needs_register (ce); + +if (ret ==NEW_GUC_ID_ENABLED ) { + guc->stalled_rq = rq; + guc->submission_stall_reason =STALL_SCHED_DISABLE ; + } + +clear_bit (CONTEXT_LRCA_DIRTY , &ce->flags); +out: + guc->total_num_rq_with_no_guc_id -= ce->guc_num_rq_submit_no_id; +GEM_BUG_ON (guc->total_num_rq_with_no_guc_id <0 ); + +list_for_each_entry_reverse (rq, &ce->guc_active.requests, sched.link) +if (request_has_no_guc_id (rq)) { + --ce->guc_num_rq_submit_no_id; +clear_bit (I915_FENCE_FLAG_GUC_ID_NOT_PINNED , + &rq->fence.flags); + }else if (!ce->guc_num_rq_submit_no_id) { +break ; + } + +GEM_BUG_ON (ce->guc_num_rq_submit_no_id); + +/* +* When NEW_GUC_ID_ENABLED is returned it means we are stealing a guc_id +* from a context that has scheduling enabled. We have to disable +* scheduling before deregistering the context and it isn't safe to do +* in the tasklet because of lock inversion (ce->guc_state.lock must be +* acquired before guc->sched_engine->lock). To work around this +* we do the schedule disable in retire workqueue and block the tasklet +* until the schedule done G2H returns. Returning non-zero here kicks +* the workqueue. +*/ +return (ret ==NEW_GUC_ID_ENABLED ) ? ret :0 ; +} +static int guc_request_alloc (struct i915_request *rq) {struct intel_context *ce = rq->context;@@ -2056,6 +2646,15 @@ static int guc_request_alloc(struct i915_request *rq) rq->reserved_space -=GUC_REQUEST_SIZE ; +/* +* guc_ids are exhausted, don't allocate one here, defer to submission +* in the tasklet. +*/ +if (test_and_update_guc_ids_exhausted (guc)) { +set_bit (I915_FENCE_FLAG_GUC_ID_NOT_PINNED , &rq->fence.flags); + goto out; + } +/* * Call pin_guc_id here rather than in the pinning step as with * dma_resv, contexts can be repeatedly pinned / unpinned trashing the @@ -2063,9 +2662,7 @@ static int guc_request_alloc(struct i915_request *rq)* when guc_ids are being stolen due to over subscription. By the time * this function is reached, it is guaranteed that the guc_id will be * persistent until the generated request is retired. Thus, sealing these -* race conditions. It is still safe to fail here if guc_ids are -* exhausted and return -EAGAIN to the user indicating that they can try -* again in the future. +* race conditions. * * There is no need for a lock here as the timeline mutex ensures at * most one context can be executing this code path at once. The @@ -2076,10 +2673,26 @@ static int guc_request_alloc(struct i915_request *rq)if (atomic_add_unless (&ce->guc_id_ref,1 ,0 )) goto out; - ret =pin_guc_id (guc, ce);/* returns 1 if new guc_id assigned */ -if (unlikely (ret <0 )) + ret =pin_guc_id (guc, ce, false);/* > 0 indicates new guc_id */ +if (unlikely (ret == -EAGAIN )) { +/* +* No guc_ids available, so we force this submission and all +* future submissions to be serialized in the tasklet, sharing +* the guc_ids on a per submission basis to ensure (more) fair +* scheduling of submissions. Once the tasklet is flushed of +* submissions we return to allocating guc_ids in this function. +*/ +set_bit (I915_FENCE_FLAG_GUC_ID_NOT_PINNED , &rq->fence.flags); +set_and_update_guc_ids_exhausted (guc); + +return 0 ; + }else if (unlikely (ret <0 )) {return ret; -if (context_needs_register (ce, !!ret)) { + } + +GEM_BUG_ON (ret ==NEW_GUC_ID_ENABLED ); + +if (context_needs_lrc_desc_pin (ce, !!ret)) { ret =guc_lrc_desc_pin (ce, true);if (unlikely (ret)) {/* unwind */ if (ret == -EPIPE ) {@@ -2087,7 +2700,7 @@ static int guc_request_alloc(struct i915_request *rq) goto out;/* GPU will be reset */ }atomic_dec (&ce->guc_id_ref); -unpin_guc_id (guc, ce); +unpin_guc_id (guc, ce, true);return ret; } }@@ -2358,7 +2971,7 @@ static inline void guc_kernel_context_pin(struct intel_guc *guc, struct intel_context *ce) {if (context_guc_id_invalid (ce)) -pin_guc_id (guc, ce); +pin_guc_id (guc, ce, false);guc_lrc_desc_pin (ce, true); }@@ -2625,6 +3238,16 @@ int intel_guc_deregister_done_process_msg(struct intel_guc *guc, with_intel_runtime_pm(runtime_pm, wakeref)register_context (ce, true);guc_signal_context_fence (ce); +if (context_block_tasklet (ce)) { +GEM_BUG_ON (guc->submission_stall_reason != +STALL_DEREGISTER_CONTEXT ); + +clr_context_block_tasklet (ce); + guc->submission_stall_reason =STALL_MOVE_LRC_TAIL ; +clr_tasklet_blocked (guc); + +kick_tasklet (ce_to_guc (ce)); + }intel_context_put (ce); }else if (context_destroyed (ce)) {/* Context has been destroyed */ @@ -2688,6 +3311,14 @@ int intel_guc_sched_done_process_msg(struct intel_guc *guc,guc_blocked_fence_complete (ce);spin_unlock_irqrestore (&ce->guc_state.lock, flags); +if (context_block_tasklet (ce)) { +clr_context_block_tasklet (ce); + guc->submission_stall_reason =STALL_REGISTER_CONTEXT ; +clr_tasklet_blocked (guc); + +kick_tasklet (ce_to_guc (ce)); + } +if (banned) {guc_cancel_context_requests (ce);intel_engine_signal_breadcrumbs (ce->engine);@@ -2716,10 +3347,8 @@ static void capture_error_state(struct intel_guc *guc,static void guc_context_replay (struct intel_context *ce) { -struct i915_sched_engine *sched_engine = ce->engine->sched_engine; -__guc_reset_context (ce, true); -tasklet_hi_schedule (&sched_engine->tasklet); +kick_tasklet (ce_to_guc (ce)); } static void guc_handle_context_reset(struct intel_guc *guc,@@ -2878,8 +3507,16 @@ void intel_guc_submission_print_info(struct intel_guc *guc, atomic_read(&guc->outstanding_submission_g2h));drm_printf (p,"GuC Number GuC IDs: %u\n" , guc->num_guc_ids);drm_printf (p,"GuC Max GuC IDs: %u\n" , guc->max_guc_ids); -drm_printf (p,"GuC tasklet count: %u\n\n" , +drm_printf (p,"GuC tasklet count: %u\n" ,atomic_read (&sched_engine->tasklet.count)); +drm_printf (p,"GuC submit flags: 0x%04lx\n" , guc->flags); +drm_printf (p,"GuC total number request without guc_id: %d\n" , + guc->total_num_rq_with_no_guc_id); +drm_printf (p,"GuC stall reason: %d\n" , guc->submission_stall_reason); +drm_printf (p,"GuC stalled request: %s\n" , +yesno (guc->stalled_rq)); +drm_printf (p,"GuC stalled context: %s\n\n" , +yesno (guc->stalled_context));spin_lock_irqsave (&sched_engine->lock, flags);drm_printf (p,"Requests in GuC submit tasklet:\n" );drivers/gpu/drm/i915/i915_request.h +6
-0@@ -139,6 +139,12 @@ enum {* the GPU. Here we track such boost requests on a per-request basis. */ I915_FENCE_FLAG_BOOST , + +/* +* I915_FENCE_FLAG_GUC_ID_NOT_PINNED - Set to signal the GuC submission +* tasklet that the guc_id isn't pinned. +*/ + I915_FENCE_FLAG_GUC_ID_NOT_PINNED, };/** Signature · 1 line
2.28.0
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
PATCH 4/46 drm/i915/guc: Don't allow requests not ready to consume all guc_ids
In reply to Matthew BrostAdd a heuristic which checks if over half of the available guc_ids are currently consumed by requests not ready to be submitted. If this heuristic is true at request creation time (normal guc_id allocation location) force all submissions + guc_ids allocations to tasklet.
--- drivers/gpu/drm/i915/gt/intel_context_types.h | 3 ++ drivers/gpu/drm/i915/gt/intel_reset.c | 9 ++++ drivers/gpu/drm/i915/gt/uc/intel_guc.h | 1 + .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 53 +++++++++++++++++-- .../gpu/drm/i915/gt/uc/intel_guc_submission.h | 2 + 5 files changed, 65 insertions(+), 3 deletions(-)
drivers/gpu/drm/i915/gt/intel_context_types.h +3 -0
@@ -188,6 +188,9 @@ struct intel_context {/* Number of rq submitted without a guc_id */ u16 guc_num_rq_submit_no_id; +/* GuC number of requests not ready */ +atomic_t guc_num_rq_not_ready; +/* * GuC ID link - in list when unpinned but guc_id still valid in GuC */
drivers/gpu/drm/i915/gt/intel_reset.c +9 -0
@@ -22,6 +22,7 @@#include "intel_reset.h" #include "uc/intel_guc.h" +#include "uc/intel_guc_submission.h" #define RESET_MAX_RETRIES 3
@@ -850,6 +851,14 @@ static void nop_submit_request(struct i915_request *request) {RQ_TRACE (request,"-EIO\n" ); +/* +* XXX: Kinda ugly to check for GuC submission here but this function is +* going away once we switch to the DRM scheduler so we can live with +* this for now. +*/ +if (intel_engine_uses_guc (request->engine)) +intel_guc_decr_num_rq_not_ready (request->context); + request =i915_request_mark_eio (request);if (request) {i915_request_submit (request);
drivers/gpu/drm/i915/gt/uc/intel_guc.h +1 -0
@@ -76,6 +76,7 @@ struct intel_guc {struct ida guc_ids;u32 num_guc_ids;u32 max_guc_ids; +atomic_t num_guc_ids_not_ready;struct list_head guc_id_list_no_ref;struct list_head guc_id_list_unpinned;
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +50 -3
96 lines not on this page · raw message
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.h +2 -0
@@ -51,4 +51,6 @@ static inline bool intel_guc_submission_is_used(struct intel_guc *guc) return intel_guc_is_used(guc) && intel_guc_submission_is_wanted(guc); } +void intel_guc_decr_num_rq_not_ready (struct intel_context *ce); +#endif
Signature · 1 line
2.28.0
Re: [Intel-gfx] [PATCH 04/46] drm/i915/guc: Don't allow requests not ready to consume all guc_ids
Quoted 19 lines · Matthew Brost
Add a heuristic which checks if over half of the available
guc_idsare currently consumed by requests not ready to be submitted. If this heuristic is true at request creation time (normalguc_idallocation location) force all submissions +guc_idsallocations to tasklet.Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/gt/intel_context_types.h | 3 ++ drivers/gpu/drm/i915/gt/intel_reset.c | 9 ++++ drivers/gpu/drm/i915/gt/uc/intel_guc.h | 1 + .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 53 +++++++++++++++++-- .../gpu/drm/i915/gt/uc/intel_guc_submission.h | 2 + 5 files changed, 65 insertions(+), 3 deletions(-)drivers/gpu/drm/i915/gt/intel_context_types.h +0
-0
drivers/gpu/drm/i915/gt/intel_context_types.h struct intel_context { · line 192
Quoted 5 lines
drivers/gpu/drm/i915/gt/intel_context_types.h +2
-0@@ -188,6 +188,9 @@ struct intel_context {/* Number of rq submitted without a guc_id */ u16 guc_num_rq_submit_no_id; +/* GuC number of requests not ready */ +atomic_t guc_num_rq_not_ready;
atomic_t by default is unordered. This needs some giantic comments and explainers why this is totally ok and we don't need barriers here.
I think good excuse to convert all the docs here into kerneldoc. -Daniel
Quoted 164 lines
drivers/gpu/drm/i915/gt/intel_context_types.h +1
-0+/* * GuC ID link - in list when unpinned but guc_id still valid in GuC */ drivers/gpu/drm/i915/gt/intel_reset.c +9
-0@@ -22,6 +22,7 @@#include "intel_reset.h" #include "uc/intel_guc.h" +#include "uc/intel_guc_submission.h" #define RESET_MAX_RETRIES 3@@ -850,6 +851,14 @@ static void nop_submit_request(struct i915_request *request) {RQ_TRACE (request,"-EIO\n" ); +/* +* XXX: Kinda ugly to check for GuC submission here but this function is +* going away once we switch to the DRM scheduler so we can live with +* this for now. +*/ +if (intel_engine_uses_guc (request->engine)) +intel_guc_decr_num_rq_not_ready (request->context); + request =i915_request_mark_eio (request);if (request) {i915_request_submit (request);drivers/gpu/drm/i915/gt/uc/intel_guc.h +1
-0@@ -76,6 +76,7 @@ struct intel_guc {struct ida guc_ids;u32 num_guc_ids;u32 max_guc_ids; +atomic_t num_guc_ids_not_ready;struct list_head guc_id_list_no_ref;struct list_head guc_id_list_unpinned;drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +50
-3@@ -1384,6 +1384,41 @@ static inline void queue_request(struct i915_sched_engine *sched_engine,kick_tasklet (&rq->engine->gt->uc.guc); } +/* Macro to tweak heuristic, using a simple over 50% not ready for now */ +#define TOO_MANY_GUC_IDS_NOT_READY (avail, consumed) \ + ((consumed) > (avail) / 2) +static bool too_many_guc_ids_not_ready (struct intel_guc *guc, +struct intel_context *ce) +{ +u32 available_guc_ids, guc_ids_consumed; + + available_guc_ids = guc->num_guc_ids; + guc_ids_consumed =atomic_read (&guc->num_guc_ids_not_ready); + +if (TOO_MANY_GUC_IDS_NOT_READY (available_guc_ids, guc_ids_consumed)) { +set_and_update_guc_ids_exhausted (guc); +return true; + } + +return false; +} + +static void incr_num_rq_not_ready (struct intel_context *ce) +{ +struct intel_guc *guc =ce_to_guc (ce); + +if (!atomic_fetch_add (1 , &ce->guc_num_rq_not_ready)) +atomic_inc (&guc->num_guc_ids_not_ready); +} + +void intel_guc_decr_num_rq_not_ready (struct intel_context *ce) +{ +struct intel_guc *guc =ce_to_guc (ce); + +if (atomic_fetch_add (-1 , &ce->guc_num_rq_not_ready) ==1 ) +atomic_dec (&guc->num_guc_ids_not_ready); +} +static bool need_tasklet (struct intel_guc *guc,struct intel_context *ce) { struct i915_sched_engine * const sched_engine =@@ -1430,6 +1465,8 @@ static void guc_submit_request(struct i915_request *rq)kick_tasklet (guc);spin_unlock_irqrestore (&sched_engine->lock, flags); + +intel_guc_decr_num_rq_not_ready (rq->context); }static int new_guc_id (struct intel_guc *guc)@@ -2647,10 +2684,13 @@ static int guc_request_alloc(struct i915_request *rq) rq->reserved_space -=GUC_REQUEST_SIZE ;/* -* guc_ids are exhausted, don't allocate one here, defer to submission -* in the tasklet. +* guc_ids are exhausted or a heuristic is met indicating too many +* guc_ids are waiting on requests with submission dependencies (not +* ready to submit). Don't allocate one here, defer to submission in the +* tasklet. */ -if (test_and_update_guc_ids_exhausted (guc)) { +if (test_and_update_guc_ids_exhausted (guc) || +too_many_guc_ids_not_ready (guc, ce)) {set_bit (I915_FENCE_FLAG_GUC_ID_NOT_PINNED , &rq->fence.flags); goto out; }@@ -2684,6 +2724,7 @@ static int guc_request_alloc(struct i915_request *rq)*/ set_bit (I915_FENCE_FLAG_GUC_ID_NOT_PINNED , &rq->fence.flags);set_and_update_guc_ids_exhausted (guc); +incr_num_rq_not_ready (ce);return 0 ; }else if (unlikely (ret <0 )) {@@ -2708,6 +2749,8 @@ static int guc_request_alloc(struct i915_request *rq)clear_bit (CONTEXT_LRCA_DIRTY , &ce->flags); out: +incr_num_rq_not_ready (ce); +/* * We block all requests on this context if a G2H is pending for a * schedule disable or context deregistration as the GuC will fail a @@ -3512,6 +3555,8 @@ void intel_guc_submission_print_info(struct intel_guc *guc,drm_printf (p,"GuC submit flags: 0x%04lx\n" , guc->flags);drm_printf (p,"GuC total number request without guc_id: %d\n" , guc->total_num_rq_with_no_guc_id); +drm_printf (p,"GuC Number GuC IDs not ready: %d\n" , +atomic_read (&guc->num_guc_ids_not_ready));drm_printf (p,"GuC stall reason: %d\n" , guc->submission_stall_reason);drm_printf (p,"GuC stalled request: %s\n" ,yesno (guc->stalled_rq));@@ -3567,6 +3612,8 @@ void intel_guc_submission_print_context_info(struct intel_guc *guc, atomic_read(&ce->pin_count));drm_printf (p,"\t\tGuC ID Ref Count: %u\n" ,atomic_read (&ce->guc_id_ref)); +drm_printf (p,"\t\tNumber Requests Not Ready: %u\n" , +atomic_read (&ce->guc_num_rq_not_ready));drm_printf (p,"\t\tSchedule State: 0x%x, 0x%x\n\n" , ce->guc_state.sched_state,atomic_read (&ce->guc_sched_state_no_lock));drivers/gpu/drm/i915/gt/uc/intel_guc_submission.h +2
-0@@ -51,4 +51,6 @@ static inline bool intel_guc_submission_is_used(struct intel_guc *guc) return intel_guc_is_used(guc) && intel_guc_submission_is_wanted(guc); } +void intel_guc_decr_num_rq_not_ready (struct intel_context *ce); +#endif Signature · 1 line
2.28.0
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
PATCH 5/46 drm/i915/guc: Introduce guc_submit_engine object
In reply to Matthew BrostMove fields related to controlling the GuC submission state machine to a unique object (guc_submit_engine) rather than the global GuC state (intel_guc). This encapsulation allows multiple instances of submission objects to operate in parallel and a single instance can block if needed while another can make forward progress. This is analogous to how the execlist mode works assigning a schedule object per physical engine but rather in GuC mode we assign a schedule object based on the blocking dependencies.
The guc_submit_engine object also encapsulates the i915_sched_engine object as well.
Lots of find-replace.
Currently only 1 guc_submit_engine instantiated, future patches will instantiate more.
--- drivers/gpu/drm/i915/gt/uc/intel_guc.h | 33 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 557 +++++++++++------- .../i915/gt/uc/intel_guc_submission_types.h | 52 ++ drivers/gpu/drm/i915/i915_scheduler.c | 22 +- drivers/gpu/drm/i915/i915_scheduler.h | 3 + 5 files changed, 410 insertions(+), 257 deletions(-) create mode 100644 drivers/gpu/drm/i915/gt/uc/intel_guc_submission_types.h
drivers/gpu/drm/i915/gt/uc/intel_guc.h +7 -26
@@ -21,6 +21,11 @@struct __guc_ads_blob ; +enum { +GUC_SUBMIT_ENGINE_SINGLE_LRC , +GUC_SUBMIT_ENGINE_MAX +}; +/* * Top level structure of GuC. It handles firmware loading and manages client * pool. intel_guc owns a intel_guc_client to replace the legacy ExecList
@@ -31,31 +36,6 @@ struct intel_guc {struct intel_guc_log log;struct intel_guc_ct ct; -/* Global engine used to submit requests to GuC */ -struct i915_sched_engine *sched_engine; - -/* Global state related to submission tasklet */ -struct i915_request *stalled_rq; -struct intel_context *stalled_context; -struct work_struct retire_worker; -unsigned long flags; -int total_num_rq_with_no_guc_id; - -/* -* Submisson stall reason. See intel_guc_submission.c for detailed -* description. -*/ -enum { -STALL_NONE , -STALL_GUC_ID_WORKQUEUE , -STALL_GUC_ID_TASKLET , -STALL_SCHED_DISABLE , -STALL_REGISTER_CONTEXT , -STALL_DEREGISTER_CONTEXT , -STALL_MOVE_LRC_TAIL , -STALL_ADD_REQUEST , - } submission_stall_reason; -/* intel_guc_recv interrupt related state */ spinlock_t irq_lock;unsigned int msg_enabled_mask;
@@ -68,6 +48,8 @@ struct intel_guc {void (*disable)(struct intel_guc *guc); } interrupts; +struct guc_submit_engine *gse[GUC_SUBMIT_ENGINE_MAX ]; +/* * contexts_lock protects the pool of free guc ids and a linked list of * guc ids available to be stolen
@@ -76,7 +58,6 @@ struct intel_guc {struct ida guc_ids;u32 num_guc_ids;u32 max_guc_ids; -atomic_t num_guc_ids_not_ready;struct list_head guc_id_list_no_ref;struct list_head guc_id_list_unpinned;
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +334 -223
1,090 lines not on this page · raw message
drivers/gpu/drm/i915/gt/uc/intel_guc_submission_types.h new file +52 -0
52 lines not on this page · raw message
drivers/gpu/drm/i915/i915_scheduler.c +14 -8
@@ -448,15 +448,9 @@ static bool default_disabled(struct i915_sched_engine *sched_engine)return false; } -struct i915_sched_engine * -i915_sched_engine_create (unsigned int subclass) +void i915_sched_engine_init (struct i915_sched_engine *sched_engine, +unsigned int subclass) { -struct i915_sched_engine *sched_engine; - - sched_engine =kzalloc (sizeof (*sched_engine),GFP_KERNEL ); -if (!sched_engine) -return NULL ; -kref_init (&sched_engine->ref); sched_engine->queue =RB_ROOT_CACHED ;
@@ -481,6 +475,18 @@ i915_sched_engine_create(unsigned int subclass)lock_map_release (&sched_engine->lock.dep_map);local_irq_enable ();#endif +} + +struct i915_sched_engine * +i915_sched_engine_create (unsigned int subclass) +{ +struct i915_sched_engine *sched_engine; + + sched_engine =kzalloc (sizeof (*sched_engine),GFP_KERNEL ); +if (!sched_engine) +return NULL ; + +i915_sched_engine_init (sched_engine, subclass);return sched_engine; }
drivers/gpu/drm/i915/i915_scheduler.h +3 -0
@@ -48,6 +48,9 @@ static inline void i915_priolist_free(struct i915_priolist *p)__i915_priolist_free (p); } +void i915_sched_engine_init (struct i915_sched_engine *sched_engine, +unsigned int subclass); +struct i915_sched_engine *i915_sched_engine_create (unsigned int subclass);
Signature · 1 line
2.28.0
PATCH 6/46 drm/i915/guc: Check return of __xa_store when registering a context
In reply to Matthew BrostCheck return of __xa_store when registering a context as this can fail in a rare case if not memory can not be allocated. If this occurs fall back on the tasklet flow control and try again in the future.
--- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +12 -4
@@ -505,18 +505,24 @@ static inline bool lrc_desc_registered(struct intel_guc *guc, u32 id)return __get_context (guc, id); } -static inline void set_lrc_desc_registered (struct intel_guc *guc,u32 id, -struct intel_context *ce) +static inline int set_lrc_desc_registered (struct intel_guc *guc,u32 id, +struct intel_context *ce) {unsigned long flags; +void *ret;/* * xarray API doesn't have xa_save_irqsave wrapper, so calling the * lower level functions directly. */ xa_lock_irqsave (&guc->context_lookup, flags); -__xa_store (&guc->context_lookup, id, ce,GFP_ATOMIC ); + ret =__xa_store (&guc->context_lookup, id, ce,GFP_ATOMIC );xa_unlock_irqrestore (&guc->context_lookup, flags); + +if (unlikely (xa_is_err (ret))) +return -EBUSY ;/* Try again in future */ + +return 0 ; } static int guc_submission_send_busy_loop(struct intel_guc *guc,
@@ -1854,7 +1860,9 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)rcu_read_unlock ();reset_lrc_desc (guc, desc_idx); -set_lrc_desc_registered (guc, desc_idx, ce); + ret =set_lrc_desc_registered (guc, desc_idx, ce); +if (unlikely (ret)) +return ret; desc =__get_lrc_desc (guc, desc_idx); desc->engine_class =engine_class_to_guc_class (engine->class);
Signature · 1 line
2.28.0
PATCH 7/46 drm/i915/guc: Non-static lrc descriptor registration buffer
In reply to Matthew BrostDynamically allocate space for lrc descriptor registration with the GuC rather than using a large static buffer indexed by the guc_id. If no space is available to register a context, fall back to tasklet flow control mechanism. Only allow 1/2 of the space to be allocated outside the tasklet to prevent unready requests/contexts from consuming all registration space.
--- drivers/gpu/drm/i915/gt/intel_context_types.h | 3 + drivers/gpu/drm/i915/gt/uc/intel_guc.h | 9 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 202 ++++++++++++------ 3 files changed, 150 insertions(+), 64 deletions(-)
drivers/gpu/drm/i915/gt/intel_context_types.h +3 -0
@@ -179,6 +179,9 @@ struct intel_context {/* GuC scheduling state flags that do not require a lock. */ atomic_t guc_sched_state_no_lock; +/* GuC lrc descriptor registration buffer */ +unsigned int guc_lrcd_reg_idx; +/* GuC LRC descriptor ID */ u16 guc_id;
drivers/gpu/drm/i915/gt/uc/intel_guc.h +7 -2
@@ -69,8 +69,13 @@ struct intel_guc {u32 ads_regset_size;u32 ads_golden_ctxt_size; -struct i915_vma *lrc_desc_pool; -void *lrc_desc_pool_vaddr; +/* GuC LRC descriptor registration */ +struct { +struct i915_vma *vma; +void *vaddr; +struct ida ida; +unsigned int max_idx; + } lrcd_reg;/* guc_id to intel_context lookup */ struct xarray context_lookup;
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +140 -62
345 lines not on this page · raw message
Signature · 1 line
2.28.0
PATCH 8/46 drm/i915/guc: Take GT PM ref when deregistering context
In reply to Matthew BrostTaking a PM reference to prevent intel_gt_wait_for_idle from short circuiting while a deregister context H2G is in flight.
--- drivers/gpu/drm/i915/gt/intel_engine_pm.h | 5 + drivers/gpu/drm/i915/gt/intel_gt_pm.h | 13 +++ drivers/gpu/drm/i915/gt/uc/intel_guc.h | 4 + .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 99 +++++++++++++++---- 4 files changed, 102 insertions(+), 19 deletions(-)
drivers/gpu/drm/i915/gt/intel_engine_pm.h +5 -0
@@ -16,6 +16,11 @@ intel_engine_pm_is_awake(const struct intel_engine_cs *engine)return intel_wakeref_is_active (&engine->wakeref); } +static inline void __intel_engine_pm_get (struct intel_engine_cs *engine) +{ +__intel_wakeref_get (&engine->wakeref); +} +static inline void intel_engine_pm_get (struct intel_engine_cs *engine) {intel_wakeref_get (&engine->wakeref);
drivers/gpu/drm/i915/gt/intel_gt_pm.h +13 -0
@@ -41,6 +41,19 @@ static inline void intel_gt_pm_put_async(struct intel_gt *gt)intel_wakeref_put_async (>->wakeref); } +#define with_intel_gt_pm (gt, tmp) \ + for (tmp = 1, intel_gt_pm_get(gt); tmp; \ + intel_gt_pm_put(gt), tmp = 0) +#define with_intel_gt_pm_async (gt, tmp) \ + for (tmp = 1, intel_gt_pm_get(gt); tmp; \ + intel_gt_pm_put_async(gt), tmp = 0) +#define with_intel_gt_pm_if_awake (gt, tmp) \ + for (tmp = intel_gt_pm_get_if_awake(gt); tmp; \ + intel_gt_pm_put(gt), tmp = 0) +#define with_intel_gt_pm_if_awake_async (gt, tmp) \ + for (tmp = intel_gt_pm_get_if_awake(gt); tmp; \ + intel_gt_pm_put_async(gt), tmp = 0) +static inline int intel_gt_pm_wait_for_idle (struct intel_gt *gt) {return intel_wakeref_wait_for_idle (>->wakeref);
drivers/gpu/drm/i915/gt/uc/intel_guc.h +4 -0
@@ -61,6 +61,10 @@ struct intel_guc {struct list_head guc_id_list_no_ref;struct list_head guc_id_list_unpinned; +spinlock_t destroy_lock;/* protects list / worker */ +struct list_head destroyed_contexts; +struct work_struct destroy_worker; +bool submission_supported;bool submission_selected;
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +80 -19
177 lines not on this page · raw message
Signature · 1 line
2.28.0
PATCH 9/46 drm/i915: Add GT PM unpark worker
In reply to Matthew BrostSometimes it is desirable to queue work up for later if the GT PM isn't held and run that work on next GT PM unpark.
Implemented with a list in the GT of all pending work, workqueues in the list, a callback to add a workqueue to the list, and finally a wakeref post_get callback that iterates / drains the list + queues the workqueues.
First user of this is deregistration of GuC contexts.
--- drivers/gpu/drm/i915/gt/intel_gt.c | 3 ++ drivers/gpu/drm/i915/gt/intel_gt_pm.c | 8 +++++ .../gpu/drm/i915/gt/intel_gt_pm_unpark_work.c | 35 +++++++++++++++++++ .../gpu/drm/i915/gt/intel_gt_pm_unpark_work.h | 32 +++++++++++++++++ drivers/gpu/drm/i915/gt/intel_gt_types.h | 3 ++ drivers/gpu/drm/i915/gt/uc/intel_guc.h | 3 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 14 +++++--- drivers/gpu/drm/i915/intel_wakeref.c | 5 +++ drivers/gpu/drm/i915/intel_wakeref.h | 1 + 9 files changed, 99 insertions(+), 5 deletions(-) create mode 100644 drivers/gpu/drm/i915/gt/intel_gt_pm_unpark_work.c create mode 100644 drivers/gpu/drm/i915/gt/intel_gt_pm_unpark_work.h
drivers/gpu/drm/i915/gt/intel_gt.c +3 -0
@@ -29,6 +29,9 @@ void intel_gt_init_early(struct intel_gt *gt, struct drm_i915_private *i915)spin_lock_init (>->irq_lock); +spin_lock_init (>->pm_unpark_work_lock); +INIT_LIST_HEAD (>->pm_unpark_work_list); +INIT_LIST_HEAD (>->closed_vma);spin_lock_init (>->closed_lock);
drivers/gpu/drm/i915/gt/intel_gt_pm.c +8 -0
@@ -90,6 +90,13 @@ static int __gt_unpark(struct intel_wakeref *wf)return 0 ; } +static void __gt_unpark_work_queue (struct intel_wakeref *wf) +{ +struct intel_gt *gt =container_of (wf,typeof (*gt), wakeref); + +intel_gt_pm_unpark_work_queue (gt); +} +static int __gt_park (struct intel_wakeref *wf) {struct intel_gt *gt =container_of (wf,typeof (*gt), wakeref);
@@ -118,6 +125,7 @@ static int __gt_park(struct intel_wakeref *wf)static const struct intel_wakeref_ops wf_ops = { .get = __gt_unpark, + .post_get = __gt_unpark_work_queue, .put = __gt_park, };
drivers/gpu/drm/i915/gt/intel_gt_pm_unpark_work.c new file +35 -0
@@ -0,0 +1,35 @@ +// SPDX-License-Identifier: MIT +/* +* Copyright © 2021 Intel Corporation +*/ + +#include "i915_drv.h" +#include "intel_runtime_pm.h" +#include "intel_gt_pm.h" + +void intel_gt_pm_unpark_work_queue (struct intel_gt *gt) +{ +struct intel_gt_pm_unpark_work *work, *next; +unsigned long flags; + +spin_lock_irqsave (>->pm_unpark_work_lock, flags); +list_for_each_entry_safe (work, next, + >->pm_unpark_work_list, link) { +list_del_init (&work->link); +queue_work (system_unbound_wq, &work->worker); + } +spin_unlock_irqrestore (>->pm_unpark_work_lock, flags); +} + +void intel_gt_pm_unpark_work_add (struct intel_gt *gt, +struct intel_gt_pm_unpark_work *work) +{ +unsigned long flags; + +spin_lock_irqsave (>->pm_unpark_work_lock, flags); +if (intel_gt_pm_is_awake (gt)) +queue_work (system_unbound_wq, &work->worker); +else if (list_empty (&work->link)) +list_add_tail (&work->link, >->pm_unpark_work_list); +spin_unlock_irqrestore (>->pm_unpark_work_lock, flags); +}
drivers/gpu/drm/i915/gt/intel_gt_pm_unpark_work.h new file +32 -0
@@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: MIT */ +/* +* Copyright © 2021 Intel Corporation +*/ + +#ifndef INTEL_GT_PM_UNPARK_WORK_H +#define INTEL_GT_PM_UNPARK_WORK_H + +#include <linux/list.h> +#include <linux/workqueue.h> + +struct intel_gt ; + +struct intel_gt_pm_unpark_work { +struct list_head link; +struct work_struct worker; +}; + +void intel_gt_pm_unpark_work_queue (struct intel_gt *gt); + +void intel_gt_pm_unpark_work_add (struct intel_gt *gt, +struct intel_gt_pm_unpark_work *work); + +static inline void +intel_gt_pm_unpark_work_init (struct intel_gt_pm_unpark_work *work, +work_func_t fn) +{ +INIT_LIST_HEAD (&work->link); +INIT_WORK (&work->worker, fn); +} + +#endif /* INTEL_GT_PM_UNPARK_WORK_H */
drivers/gpu/drm/i915/gt/intel_gt_types.h +3 -0
@@ -91,6 +91,9 @@ struct intel_gt {struct intel_wakeref wakeref;atomic_t user_wakeref; +struct list_head pm_unpark_work_list; +spinlock_t pm_unpark_work_lock;/* protect list */ +struct list_head closed_vma;spinlock_t closed_lock;/* guards the list of closed_vma */
drivers/gpu/drm/i915/gt/uc/intel_guc.h +2 -1
@@ -18,6 +18,7 @@#include "intel_uc_fw.h" #include "i915_utils.h" #include "i915_vma.h" +#include "gt/intel_gt_pm_unpark_work.h" struct __guc_ads_blob ;
@@ -63,7 +64,7 @@ struct intel_guc {spinlock_t destroy_lock;/* protects list / worker */ struct list_head destroyed_contexts; -struct work_struct destroy_worker; +struct intel_gt_pm_unpark_work destroy_worker;bool submission_supported;bool submission_selected;
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +10 -4
@@ -1406,8 +1406,9 @@ int intel_guc_submission_init(struct intel_guc *guc)ida_init (&guc->guc_ids);spin_lock_init (&guc->destroy_lock); +INIT_LIST_HEAD (&guc->destroyed_contexts); -INIT_WORK (&guc->destroy_worker, destroy_worker_func); +intel_gt_pm_unpark_work_init (&guc->destroy_worker, destroy_worker_func);return 0 ; }
@@ -2446,13 +2447,18 @@ static void deregister_destroyed_contexts(struct intel_guc *guc)static void destroy_worker_func (struct work_struct *w) { +struct intel_gt_pm_unpark_work *destroy_worker = + container_of(w, struct intel_gt_pm_unpark_work, worker);struct intel_guc *guc = - container_of(w, struct intel_guc, destroy_worker); + container_of(destroy_worker, struct intel_guc, destroy_worker);struct intel_gt *gt =guc_to_gt (guc);int tmp; - with_intel_gt_pm(gt, tmp) + with_intel_gt_pm_if_awake(gt, tmp)deregister_destroyed_contexts (guc); + +if (!list_empty (&guc->destroyed_contexts)) +intel_gt_pm_unpark_work_add (gt, destroy_worker); }static void guc_context_destroy (struct kref *kref)
@@ -2515,7 +2521,7 @@ static void guc_context_destroy(struct kref *kref)* take the GT PM for the first time which isn't allowed from an atomic * context. */ -queue_work (system_unbound_wq, &guc->destroy_worker); +intel_gt_pm_unpark_work_add (guc_to_gt (guc), &guc->destroy_worker); }static int guc_context_alloc (struct intel_context *ce)
drivers/gpu/drm/i915/intel_wakeref.c +5 -0
@@ -24,6 +24,8 @@ static void rpm_put(struct intel_wakeref *wf)int __intel_wakeref_get_first (struct intel_wakeref *wf) { +bool do_post = false; +/* * Treat get/put as different subclasses, as we may need to run * the put callback from under the shrinker and do not want to
@@ -44,8 +46,11 @@ int __intel_wakeref_get_first(struct intel_wakeref *wf) }smp_mb__before_atomic ();/* release wf->count */ + do_post = true; }atomic_inc (&wf->count); +if (do_post && wf->ops->post_get) + wf->ops->post_get (wf);mutex_unlock (&wf->mutex);INTEL_WAKEREF_BUG_ON (atomic_read (&wf->count) <=0 );
drivers/gpu/drm/i915/intel_wakeref.h +1 -0
@@ -30,6 +30,7 @@ typedef depot_stack_handle_t intel_wakeref_t;struct intel_wakeref_ops {int (*get)(struct intel_wakeref *wf); +void (*post_get)(struct intel_wakeref *wf);int (*put)(struct intel_wakeref *wf); };
Signature · 1 line
2.28.0
PATCH 10/46 drm/i915/guc: Take engine PM when a context is pinned with GuC submission
In reply to Matthew BrostTaking a PM reference to prevent intel_gt_wait_for_idle from short circuiting while a scheduling of user context could be enabled.
--- drivers/gpu/drm/i915/Makefile | 1 + .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 36 +++++++++++++++++-- 2 files changed, 34 insertions(+), 3 deletions(-)
drivers/gpu/drm/i915/Makefile +1 -0
@@ -103,6 +103,7 @@ gt-y += \ gt/intel_gt_clock_utils.o \ gt/intel_gt_irq.o \ gt/intel_gt_pm.o \ + gt/intel_gt_pm_unpark_work.o \ gt/intel_gt_pm_irq.o \ gt/intel_gt_requests.o \ gt/intel_gtt.o \
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +33 -3
@@ -2056,7 +2056,12 @@ static int guc_context_pre_pin(struct intel_context *ce,static int guc_context_pin (struct intel_context *ce,void *vaddr) { -return __guc_context_pin (ce, ce->engine, vaddr); +int ret =__guc_context_pin (ce, ce->engine, vaddr); + +if (likely (!ret && !intel_context_is_barrier (ce))) +intel_engine_pm_get (ce->engine); + +return ret; }static void guc_context_unpin (struct intel_context *ce)
@@ -2067,6 +2072,9 @@ static void guc_context_unpin(struct intel_context *ce)unpin_guc_id (guc, ce, true);lrc_unpin (ce); + +if (likely (!intel_context_is_barrier (ce))) +intel_engine_pm_put (ce->engine); }static void guc_context_post_unpin (struct intel_context *ce)
@@ -3002,8 +3010,30 @@ static int guc_virtual_context_pre_pin(struct intel_context *ce,static int guc_virtual_context_pin (struct intel_context *ce,void *vaddr) {struct intel_engine_cs *engine =guc_virtual_get_sibling (ce->engine,0 ); +int ret =__guc_context_pin (ce, engine, vaddr); +intel_engine_mask_t tmp, mask = ce->engine->mask; + +if (likely (!ret)) +for_each_engine_masked (engine, ce->engine->gt, mask, tmp) +intel_engine_pm_get (engine); -return __guc_context_pin (ce, engine, vaddr); +return ret; +} + +static void guc_virtual_context_unpin (struct intel_context *ce) +{ +intel_engine_mask_t tmp, mask = ce->engine->mask; +struct intel_engine_cs *engine; +struct intel_guc *guc =ce_to_guc (ce); + +GEM_BUG_ON (context_enabled (ce)); +GEM_BUG_ON (intel_context_is_barrier (ce)); + +unpin_guc_id (guc, ce, true); +lrc_unpin (ce); + +for_each_engine_masked (engine, ce->engine->gt, mask, tmp) +intel_engine_pm_put (engine); }static void guc_virtual_context_enter (struct intel_context *ce)
@@ -3040,7 +3070,7 @@ static const struct intel_context_ops virtual_guc_context_ops = { .pre_pin = guc_virtual_context_pre_pin, .pin = guc_virtual_context_pin, - .unpin = guc_context_unpin, + .unpin = guc_virtual_context_unpin, .post_unpin = guc_context_post_unpin, .ban = guc_context_ban,
Signature · 1 line
2.28.0
Re: [PATCH 10/46] drm/i915/guc: Take engine PM when a context is pinned with GuC submission
Quoted 14 lines · Matthew Brost
Taking a PM reference to prevent
intel_gt_wait_for_idlefrom short circuiting while a scheduling of user context could be enabled.Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/Makefile | 1 + .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 36 +++++++++++++++++-- 2 files changed, 34 insertions(+), 3 deletions(-)drivers/gpu/drm/i915/Makefile +0
-0
drivers/gpu/drm/i915/Makefile gt-y += \ · line 106
Quoted 4 lines
drivers/gpu/drm/i915/Makefile +1
-0@@ -103,6 +103,7 @@ gt-y += \ gt/intel_gt_clock_utils.o \ gt/intel_gt_irq.o \ gt/intel_gt_pm.o \ + gt/intel_gt_pm_unpark_work.o \
This file isn't here?
Also pm stuff tends to have very nasty locking requirements, doing special stuff like this in the backend tends to lead to really big surprises. I think two options to make sure our locking design stays consistent:
- Lift this to generic code.
- expose some engine_pm_migt_get/put() calls which do have the right set of might_lock annoations, and call those in the generic code.
Imo the worst kernel abstractions are those where all implementations look&act the same, except for locking. Unfortunately i915-gem code is full of this stuff, and we need to stop this by enlisting lockdep to check the contracts for us.
-Daniel
Quoted 75 lines
drivers/gpu/drm/i915/Makefile +0
-0gt/intel_gt_pm_irq.o \ gt/intel_gt_requests.o \ gt/intel_gtt.o \drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +33
-3@@ -2056,7 +2056,12 @@ static int guc_context_pre_pin(struct intel_context *ce,static int guc_context_pin (struct intel_context *ce,void *vaddr) { -return __guc_context_pin (ce, ce->engine, vaddr); +int ret =__guc_context_pin (ce, ce->engine, vaddr); + +if (likely (!ret && !intel_context_is_barrier (ce))) +intel_engine_pm_get (ce->engine); + +return ret; }static void guc_context_unpin (struct intel_context *ce)@@ -2067,6 +2072,9 @@ static void guc_context_unpin(struct intel_context *ce)unpin_guc_id (guc, ce, true);lrc_unpin (ce); + +if (likely (!intel_context_is_barrier (ce))) +intel_engine_pm_put (ce->engine); }static void guc_context_post_unpin (struct intel_context *ce)@@ -3002,8 +3010,30 @@ static int guc_virtual_context_pre_pin(struct intel_context *ce,static int guc_virtual_context_pin (struct intel_context *ce,void *vaddr) {struct intel_engine_cs *engine =guc_virtual_get_sibling (ce->engine,0 ); +int ret =__guc_context_pin (ce, engine, vaddr); +intel_engine_mask_t tmp, mask = ce->engine->mask; + +if (likely (!ret)) +for_each_engine_masked (engine, ce->engine->gt, mask, tmp) +intel_engine_pm_get (engine); -return __guc_context_pin (ce, engine, vaddr); +return ret; +} + +static void guc_virtual_context_unpin (struct intel_context *ce) +{ +intel_engine_mask_t tmp, mask = ce->engine->mask; +struct intel_engine_cs *engine; +struct intel_guc *guc =ce_to_guc (ce); + +GEM_BUG_ON (context_enabled (ce)); +GEM_BUG_ON (intel_context_is_barrier (ce)); + +unpin_guc_id (guc, ce, true); +lrc_unpin (ce); + +for_each_engine_masked (engine, ce->engine->gt, mask, tmp) +intel_engine_pm_put (engine); }static void guc_virtual_context_enter (struct intel_context *ce)@@ -3040,7 +3070,7 @@ static const struct intel_context_ops virtual_guc_context_ops = { .pre_pin = guc_virtual_context_pre_pin, .pin = guc_virtual_context_pin, - .unpin = guc_context_unpin, + .unpin = guc_virtual_context_unpin, .post_unpin = guc_context_post_unpin, .ban = guc_context_ban,Signature · 1 line
2.28.0
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [PATCH 10/46] drm/i915/guc: Take engine PM when a context is pinned with GuC submission
Quoted 10 lines · Daniel Vetter
Quoted 9 lines · Matthew Brost
Taking a PM reference to prevent
intel_gt_wait_for_idlefrom short circuiting while a scheduling of user context could be enabled.Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/Makefile | 1 + .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 36 +++++++++++++++++-- 2 files changed, 34 insertions(+), 3 deletions(-)
Quoted 12 lines
drivers/gpu/drm/i915/Makefile gt-y += \ · line 106
Quoted 9 lines
drivers/gpu/drm/i915/Makefile +1
-0@@ -103,6 +103,7 @@ gt-y += \ gt/intel_gt_clock_utils.o \ gt/intel_gt_irq.o \ gt/intel_gt_pm.o \ + gt/intel_gt_pm_unpark_work.o \This file isn't here?
Yep, included this in the wrong patch. Should be in:
https://patchwork.freedesktop.org/patch/448462/?series=92789&rev=2
Quoted 4 lines
Also pm stuff tends to have very nasty locking requirements, doing special stuff like this in the backend tends to lead to really big surprises. I think two options to make sure our locking design stays consistent:
- Lift this to generic code.
Not sure I'm following this, intel_engine_pm_get/put are generic calls. Those calls should have all the correct annoations. If they don't we can add them.
Matt
Quoted 89 lines
- expose some engine_pm_migt_get/put() calls which do have the right set of
might_lockannoations, and call those in the generic code.Imo the worst kernel abstractions are those where all implementations look&act the same, except for locking. Unfortunately i915-gem code is full of this stuff, and we need to stop this by enlisting lockdep to check the contracts for us.
-DanielQuoted 75 lines
drivers/gpu/drm/i915/Makefile +0
-0gt/intel_gt_pm_irq.o \ gt/intel_gt_requests.o \ gt/intel_gtt.o \drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +33
-3@@ -2056,7 +2056,12 @@ static int guc_context_pre_pin(struct intel_context *ce,static int guc_context_pin (struct intel_context *ce,void *vaddr) { -return __guc_context_pin (ce, ce->engine, vaddr); +int ret =__guc_context_pin (ce, ce->engine, vaddr); + +if (likely (!ret && !intel_context_is_barrier (ce))) +intel_engine_pm_get (ce->engine); + +return ret; }static void guc_context_unpin (struct intel_context *ce)@@ -2067,6 +2072,9 @@ static void guc_context_unpin(struct intel_context *ce)unpin_guc_id (guc, ce, true);lrc_unpin (ce); + +if (likely (!intel_context_is_barrier (ce))) +intel_engine_pm_put (ce->engine); }static void guc_context_post_unpin (struct intel_context *ce)@@ -3002,8 +3010,30 @@ static int guc_virtual_context_pre_pin(struct intel_context *ce,static int guc_virtual_context_pin (struct intel_context *ce,void *vaddr) {struct intel_engine_cs *engine =guc_virtual_get_sibling (ce->engine,0 ); +int ret =__guc_context_pin (ce, engine, vaddr); +intel_engine_mask_t tmp, mask = ce->engine->mask; + +if (likely (!ret)) +for_each_engine_masked (engine, ce->engine->gt, mask, tmp) +intel_engine_pm_get (engine); -return __guc_context_pin (ce, engine, vaddr); +return ret; +} + +static void guc_virtual_context_unpin (struct intel_context *ce) +{ +intel_engine_mask_t tmp, mask = ce->engine->mask; +struct intel_engine_cs *engine; +struct intel_guc *guc =ce_to_guc (ce); + +GEM_BUG_ON (context_enabled (ce)); +GEM_BUG_ON (intel_context_is_barrier (ce)); + +unpin_guc_id (guc, ce, true); +lrc_unpin (ce); + +for_each_engine_masked (engine, ce->engine->gt, mask, tmp) +intel_engine_pm_put (engine); }static void guc_virtual_context_enter (struct intel_context *ce)@@ -3040,7 +3070,7 @@ static const struct intel_context_ops virtual_guc_context_ops = { .pre_pin = guc_virtual_context_pre_pin, .pin = guc_virtual_context_pin, - .unpin = guc_context_unpin, + .unpin = guc_virtual_context_unpin, .post_unpin = guc_context_post_unpin, .ban = guc_context_ban,Signature · 1 line
2.28.0Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [PATCH 10/46] drm/i915/guc: Take engine PM when a context is pinned with GuC submission
Quoted 27 lines · Matthew Brost
Quoted 10 lines · Daniel Vetter
Quoted 9 lines · Matthew Brost
Taking a PM reference to prevent
intel_gt_wait_for_idlefrom short circuiting while a scheduling of user context could be enabled.Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/Makefile | 1 + .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 36 +++++++++++++++++-- 2 files changed, 34 insertions(+), 3 deletions(-)Quoted 12 lines
drivers/gpu/drm/i915/Makefile gt-y += \ · line 106
Quoted 9 lines
drivers/gpu/drm/i915/Makefile +1
-0@@ -103,6 +103,7 @@ gt-y += \ gt/intel_gt_clock_utils.o \ gt/intel_gt_irq.o \ gt/intel_gt_pm.o \ + gt/intel_gt_pm_unpark_work.o \This file isn't here?
Yep, included this in the wrong patch. Should be in:
https://patchwork.freedesktop.org/patch/448462/?series=92789&rev=2
Quoted 8 lines
Quoted 4 lines
Also pm stuff tends to have very nasty locking requirements, doing special stuff like this in the backend tends to lead to really big surprises. I think two options to make sure our locking design stays consistent:
- Lift this to generic code.Not sure I'm following this, intel_engine_pm_get/put are generic calls. Those calls should have all the correct annoations. If they don't we can add them.
But you only call them in the GuC backend, not in all of them. Which is an inconsistency in locking, and unfortunately runtime pm is extremely nasty, so having potentially very divergent locking behind the same interface in the same driver is a recipe for an unmaintainable mess.
Iow, if the high-level code runs on execlist or the ringbuffer backend we still need to go through at least the lockdep motions of what you're adding here.
This is similar in spirit to all the might_sleep/might_lock calls we have all over the kernel where in many cases something doesn't happen, but we need to make sure it's allowed to have a consistent design.
So essentially in the intel_context_pin and all these functions put a intel_engine_pm_might_get (which compiles out without debugging enabled), unconditionally, across all platforms and sched backends.
In general I think backend specific locking (irrespective of what kind of backend or interface you implement) is a pretty bad idea in the kernel, and needs to be avoided if at all possible. Avoid here means "pull the might_lock/might_sleep/might_whatever checks into generic code". -Daniel
Quoted 91 lines
Matt
Quoted 89 lines
- expose some engine_pm_migt_get/put() calls which do have the right set of
might_lockannoations, and call those in the generic code.Imo the worst kernel abstractions are those where all implementations look&act the same, except for locking. Unfortunately i915-gem code is full of this stuff, and we need to stop this by enlisting lockdep to check the contracts for us.
-DanielQuoted 75 lines
drivers/gpu/drm/i915/Makefile +0
-0gt/intel_gt_pm_irq.o \ gt/intel_gt_requests.o \ gt/intel_gtt.o \drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +33
-3@@ -2056,7 +2056,12 @@ static int guc_context_pre_pin(struct intel_context *ce,static int guc_context_pin (struct intel_context *ce,void *vaddr) { -return __guc_context_pin (ce, ce->engine, vaddr); +int ret =__guc_context_pin (ce, ce->engine, vaddr); + +if (likely (!ret && !intel_context_is_barrier (ce))) +intel_engine_pm_get (ce->engine); + +return ret; }static void guc_context_unpin (struct intel_context *ce)@@ -2067,6 +2072,9 @@ static void guc_context_unpin(struct intel_context *ce)unpin_guc_id (guc, ce, true);lrc_unpin (ce); + +if (likely (!intel_context_is_barrier (ce))) +intel_engine_pm_put (ce->engine); }static void guc_context_post_unpin (struct intel_context *ce)@@ -3002,8 +3010,30 @@ static int guc_virtual_context_pre_pin(struct intel_context *ce,static int guc_virtual_context_pin (struct intel_context *ce,void *vaddr) {struct intel_engine_cs *engine =guc_virtual_get_sibling (ce->engine,0 ); +int ret =__guc_context_pin (ce, engine, vaddr); +intel_engine_mask_t tmp, mask = ce->engine->mask; + +if (likely (!ret)) +for_each_engine_masked (engine, ce->engine->gt, mask, tmp) +intel_engine_pm_get (engine); -return __guc_context_pin (ce, engine, vaddr); +return ret; +} + +static void guc_virtual_context_unpin (struct intel_context *ce) +{ +intel_engine_mask_t tmp, mask = ce->engine->mask; +struct intel_engine_cs *engine; +struct intel_guc *guc =ce_to_guc (ce); + +GEM_BUG_ON (context_enabled (ce)); +GEM_BUG_ON (intel_context_is_barrier (ce)); + +unpin_guc_id (guc, ce, true); +lrc_unpin (ce); + +for_each_engine_masked (engine, ce->engine->gt, mask, tmp) +intel_engine_pm_put (engine); }static void guc_virtual_context_enter (struct intel_context *ce)@@ -3040,7 +3070,7 @@ static const struct intel_context_ops virtual_guc_context_ops = { .pre_pin = guc_virtual_context_pre_pin, .pin = guc_virtual_context_pin, - .unpin = guc_context_unpin, + .unpin = guc_virtual_context_unpin, .post_unpin = guc_context_post_unpin, .ban = guc_context_ban,Signature · 1 line
2.28.0Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [PATCH 10/46] drm/i915/guc: Take engine PM when a context is pinned with GuC submission
Quoted 46 lines · Daniel Vetter
Quoted 27 lines · Matthew Brost
Quoted 10 lines · Daniel Vetter
Quoted 9 lines · Matthew Brost
Taking a PM reference to prevent
intel_gt_wait_for_idlefrom short circuiting while a scheduling of user context could be enabled.Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/Makefile | 1 + .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 36 +++++++++++++++++-- 2 files changed, 34 insertions(+), 3 deletions(-)Quoted 12 lines
drivers/gpu/drm/i915/Makefile gt-y += \ · line 106
Quoted 9 lines
drivers/gpu/drm/i915/Makefile +1
-0@@ -103,6 +103,7 @@ gt-y += \ gt/intel_gt_clock_utils.o \ gt/intel_gt_irq.o \ gt/intel_gt_pm.o \ + gt/intel_gt_pm_unpark_work.o \This file isn't here?
Yep, included this in the wrong patch. Should be in:
https://patchwork.freedesktop.org/patch/448462/?series=92789&rev=2Quoted 8 lines
Quoted 4 lines
Also pm stuff tends to have very nasty locking requirements, doing special stuff like this in the backend tends to lead to really big surprises. I think two options to make sure our locking design stays consistent:
- Lift this to generic code.Not sure I'm following this, intel_engine_pm_get/put are generic calls. Those calls should have all the correct annoations. If they don't we can add them.
But you only call them in the GuC backend, not in all of them. Which is an inconsistency in locking, and unfortunately runtime pm is extremely nasty, so having potentially very divergent locking behind the same interface in the same driver is a recipe for an unmaintainable mess.
Iow, if the high-level code runs on execlist or the ringbuffer backend we still need to go through at least the lockdep motions of what you're adding here.
Quoted 8 lines
This is similar in spirit to all the might_sleep/might_lock calls we have all over the kernel where in many cases something doesn't happen, but we need to make sure it's allowed to have a consistent design.
So essentially in the
intel_context_pinand all these functions put aintel_engine_pm_might_get(which compiles out without debugging enabled), unconditionally, across all platforms and sched backends.
Ok, I see your point here. We currently don't have a intel_engine_pm_might_get but I think this translates to roughly:
might_lock(engine_pm_wf_mutex)intel_gt_pm_might_get
Will dig in a big a more and add the annotations to the next rev.
Matt
Quoted 102 lines
In general I think backend specific locking (irrespective of what kind of backend or interface you implement) is a pretty bad idea in the kernel, and needs to be avoided if at all possible. Avoid here means "pull the might_lock/might_sleep/might_whatever checks into generic code". -Daniel
Quoted 91 lines
Matt
Quoted 89 lines
- expose some engine_pm_migt_get/put() calls which do have the right set of
might_lockannoations, and call those in the generic code.Imo the worst kernel abstractions are those where all implementations look&act the same, except for locking. Unfortunately i915-gem code is full of this stuff, and we need to stop this by enlisting lockdep to check the contracts for us.
-DanielQuoted 75 lines
drivers/gpu/drm/i915/Makefile +0
-0gt/intel_gt_pm_irq.o \ gt/intel_gt_requests.o \ gt/intel_gtt.o \drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +33
-3@@ -2056,7 +2056,12 @@ static int guc_context_pre_pin(struct intel_context *ce,static int guc_context_pin (struct intel_context *ce,void *vaddr) { -return __guc_context_pin (ce, ce->engine, vaddr); +int ret =__guc_context_pin (ce, ce->engine, vaddr); + +if (likely (!ret && !intel_context_is_barrier (ce))) +intel_engine_pm_get (ce->engine); + +return ret; }static void guc_context_unpin (struct intel_context *ce)@@ -2067,6 +2072,9 @@ static void guc_context_unpin(struct intel_context *ce)unpin_guc_id (guc, ce, true);lrc_unpin (ce); + +if (likely (!intel_context_is_barrier (ce))) +intel_engine_pm_put (ce->engine); }static void guc_context_post_unpin (struct intel_context *ce)@@ -3002,8 +3010,30 @@ static int guc_virtual_context_pre_pin(struct intel_context *ce,static int guc_virtual_context_pin (struct intel_context *ce,void *vaddr) {struct intel_engine_cs *engine =guc_virtual_get_sibling (ce->engine,0 ); +int ret =__guc_context_pin (ce, engine, vaddr); +intel_engine_mask_t tmp, mask = ce->engine->mask; + +if (likely (!ret)) +for_each_engine_masked (engine, ce->engine->gt, mask, tmp) +intel_engine_pm_get (engine); -return __guc_context_pin (ce, engine, vaddr); +return ret; +} + +static void guc_virtual_context_unpin (struct intel_context *ce) +{ +intel_engine_mask_t tmp, mask = ce->engine->mask; +struct intel_engine_cs *engine; +struct intel_guc *guc =ce_to_guc (ce); + +GEM_BUG_ON (context_enabled (ce)); +GEM_BUG_ON (intel_context_is_barrier (ce)); + +unpin_guc_id (guc, ce, true); +lrc_unpin (ce); + +for_each_engine_masked (engine, ce->engine->gt, mask, tmp) +intel_engine_pm_put (engine); }static void guc_virtual_context_enter (struct intel_context *ce)@@ -3040,7 +3070,7 @@ static const struct intel_context_ops virtual_guc_context_ops = { .pre_pin = guc_virtual_context_pre_pin, .pin = guc_virtual_context_pin, - .unpin = guc_context_unpin, + .unpin = guc_virtual_context_unpin, .post_unpin = guc_context_post_unpin, .ban = guc_context_ban,Signature · 1 line
2.28.0Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.chSignature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
PATCH 11/46 drm/i915/guc: Don't call switch_to_kernel_context with GuC submission
In reply to Matthew BrostCalling switch_to_kernel_context isn't needed if the engine PM reference is taken while all contexts are pinned. By not calling switch_to_kernel_context we save on issuing a request to the engine.
--- drivers/gpu/drm/i915/gt/intel_engine_pm.c | 4 ++++ 1 file changed, 4 insertions(+)
drivers/gpu/drm/i915/gt/intel_engine_pm.c +4 -0
@@ -162,6 +162,10 @@ static bool switch_to_kernel_context(struct intel_engine_cs *engine)unsigned long flags;bool result = true; +/* No need to switch_to_kernel_context if GuC submission */ +if (intel_engine_uses_guc (engine)) +return true; +/* GPU is pointing to the void, as good as in the kernel context. */ if (intel_gt_is_wedged (engine->gt))return true;
Signature · 1 line
2.28.0
Re: [Intel-gfx] [PATCH 11/46] drm/i915/guc: Don't call switch_to_kernel_context with GuC submission
Quoted 14 lines · Matthew Brost
Calling
switch_to_kernel_contextisn't needed if the engine PM reference is taken while all contexts are pinned. By not callingswitch_to_kernel_contextwe save on issuing a request to the engine.Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/gt/intel_engine_pm.c | 4 ++++ 1 file changed, 4 insertions(+)drivers/gpu/drm/i915/gt/intel_engine_pm.c +0
-0
drivers/gpu/drm/i915/gt/intel_engine_pm.c static bool switch_to_kernel_context(struct intel_engine_cs *engine) · line 165
Quoted 4 lines
drivers/gpu/drm/i915/gt/intel_engine_pm.c +1
-0@@ -162,6 +162,10 @@ static bool switch_to_kernel_context(struct intel_engine_cs *engine)unsigned long flags;bool result = true; +/* No need to switch_to_kernel_context if GuC submission */
Maybe whack a big FIXME on here that we should unravel this properly. Currently the execlist backend assumptions are leaked all over the place, leading to stuff like this. Which means extremely fragile code.
I currently don't have a great idea on how exactly we should do that, but oh well.
btw just in case we ever want to make guc lrc properly evictable (which as the og use-case for this function, way, way back), would we need to fully unregister them from guc? At least I'm assuming there's no other trick like the below one.
Another aside: How does the perf/OA patching work on GuC?
Anyway, patch looks legit:
Quoted 9 lines
drivers/gpu/drm/i915/gt/intel_engine_pm.c +3
-0+if (intel_engine_uses_guc (engine)) +return true; +/* GPU is pointing to the void, as good as in the kernel context. */ if (intel_gt_is_wedged (engine->gt))return true;Signature · 1 line
2.28.0
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [Intel-gfx] [PATCH 11/46] drm/i915/guc: Don't call switch_to_kernel_context with GuC submission
Quoted 18 lines · Daniel Vetter
Quoted 17 lines · Matthew Brost
Calling
switch_to_kernel_contextisn't needed if the engine PM reference is taken while all contexts are pinned. By not callingswitch_to_kernel_contextwe save on issuing a request to the engine.Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/gt/intel_engine_pm.c | 4 ++++ 1 file changed, 4 insertions(+)drivers/gpu/drm/i915/gt/intel_engine_pm.c +0
-0@@ -162,6 +162,10 @@ static bool switch_to_kernel_context(struct intel_engine_cs *engine)unsigned long flags;bool result = true;
Quoted 3 lines
drivers/gpu/drm/i915/gt/intel_engine_pm.c static bool switch_to_kernel_context(struct intel_engine_cs *engine) · line 165
Quoted 1 line
drivers/gpu/drm/i915/gt/intel_engine_pm.c +1
-0+/* No need to switch_to_kernel_context if GuC submission */ Maybe whack a big FIXME on here that we should unravel this properly.
Sure, can add a FIXME here.
Quoted 3 lines
Currently the execlist backend assumptions are leaked all over the place, leading to stuff like this. Which means extremely fragile code.
Yes, this something required for execlists implemented in what should be generic code.
Quoted 2 lines
I currently don't have a great idea on how exactly we should do that, but oh well.
Me either, it will be a process.
Quoted 3 lines
btw just in case we ever want to make guc lrc properly evictable (which as the og use-case for this function, way, way back), would we need to fully
Can you explain what you mean by fully evictable? Not getting what you mean in this context.
Quoted 1 line
unregister them from guc? At least I'm assuming there's no other trick
If scheduling is disabled on the context (currently done on unpin) you are free move anything around as the GuC is guaranteed not to touch the context state. If on re-pin something has moved (e.g. the LRC vaddr is different), you need to unregister and re-register the context with the GuC.
Quoted 4 lines
like the below one.
Another aside: How does the perf/OA patching work on GuC?
Not my area of expertise but perf somewhat a WIP. The plan is for the GuC to write out some stats to HWSP I think? John Harrison is working to get this fully implemented.
OA is working afaik, with Umesh Nerlige Ramappa being the expert here.
Matt
Quoted 19 lines
Anyway, patch looks legit:
Reviewed-by Daniel Vetter <[email protected]>Quoted 9 lines
drivers/gpu/drm/i915/gt/intel_engine_pm.c +3
-0+if (intel_engine_uses_guc (engine)) +return true; +/* GPU is pointing to the void, as good as in the kernel context. */ if (intel_gt_is_wedged (engine->gt))return true;Signature · 1 line
2.28.0Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [Intel-gfx] [PATCH 11/46] drm/i915/guc: Don't call switch_to_kernel_context with GuC submission
Quoted 44 lines · Matthew Brost
Quoted 18 lines · Daniel Vetter
Quoted 17 lines · Matthew Brost
Calling
switch_to_kernel_contextisn't needed if the engine PM reference is taken while all contexts are pinned. By not callingswitch_to_kernel_contextwe save on issuing a request to the engine.Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/gt/intel_engine_pm.c | 4 ++++ 1 file changed, 4 insertions(+)drivers/gpu/drm/i915/gt/intel_engine_pm.c +0
-0@@ -162,6 +162,10 @@ static bool switch_to_kernel_context(struct intel_engine_cs *engine)unsigned long flags;bool result = true;Quoted 3 lines
drivers/gpu/drm/i915/gt/intel_engine_pm.c static bool switch_to_kernel_context(struct intel_engine_cs *engine) · line 165
Quoted 1 line
drivers/gpu/drm/i915/gt/intel_engine_pm.c +1
-0+/* No need to switch_to_kernel_context if GuC submission */ Maybe whack a big FIXME on here that we should unravel this properly.
Sure, can add a FIXME here.
Quoted 3 lines
Currently the execlist backend assumptions are leaked all over the place, leading to stuff like this. Which means extremely fragile code.
Yes, this something required for execlists implemented in what should be generic code.
Quoted 2 lines
I currently don't have a great idea on how exactly we should do that, but oh well.
Me either, it will be a process.
Quoted 3 lines
btw just in case we ever want to make guc lrc properly evictable (which as the og use-case for this function, way, way back), would we need to fully
Can you explain what you mean by fully evictable? Not getting what you mean in this context.
Quoted 7 lines
Quoted 1 line
unregister them from guc? At least I'm assuming there's no other trick
If scheduling is disabled on the context (currently done on unpin) you are free move anything around as the GuC is guaranteed not to touch the context state. If on re-pin something has moved (e.g. the LRC vaddr is different), you need to unregister and re-register the context with the GuC.
So at that point GuC also guarantees that it's not left in the hw engine? Execlist has this barrier request to fully unload the ctx from the hw, and that's also why I cam on the topic of OA.
Quoted 10 lines
Quoted 4 lines
like the below one.
Another aside: How does the perf/OA patching work on GuC?
Not my area of expertise but perf somewhat a WIP. The plan is for the GuC to write out some stats to HWSP I think? John Harrison is working to get this fully implemented.
OA is working afaik, with Umesh Nerlige Ramappa being the expert here.
I think it's OA that I'm thinking of here: We have code in i915_perf.c to patch all the ctx currently in the system, so that they have a consistent OA config. That's also relying on this barrier stuff, and I was wondering how that will work with GuC.
-Daniel
Quoted 22 lines
Matt
Quoted 19 lines
Anyway, patch looks legit:
Reviewed-by Daniel Vetter <[email protected]>Quoted 9 lines
drivers/gpu/drm/i915/gt/intel_engine_pm.c +3
-0+if (intel_engine_uses_guc (engine)) +return true; +/* GPU is pointing to the void, as good as in the kernel context. */ if (intel_gt_is_wedged (engine->gt))return true;Signature · 1 line
2.28.0Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [Intel-gfx] [PATCH 11/46] drm/i915/guc: Don't call switch_to_kernel_context with GuC submission
Quoted 66 lines · Daniel Vetter
Quoted 44 lines · Matthew Brost
Quoted 18 lines · Daniel Vetter
Quoted 17 lines · Matthew Brost
Calling
switch_to_kernel_contextisn't needed if the engine PM reference is taken while all contexts are pinned. By not callingswitch_to_kernel_contextwe save on issuing a request to the engine.Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/gt/intel_engine_pm.c | 4 ++++ 1 file changed, 4 insertions(+)drivers/gpu/drm/i915/gt/intel_engine_pm.c +0
-0@@ -162,6 +162,10 @@ static bool switch_to_kernel_context(struct intel_engine_cs *engine)unsigned long flags;bool result = true;Quoted 3 lines
drivers/gpu/drm/i915/gt/intel_engine_pm.c static bool switch_to_kernel_context(struct intel_engine_cs *engine) · line 165
Quoted 1 line
drivers/gpu/drm/i915/gt/intel_engine_pm.c +1
-0+/* No need to switch_to_kernel_context if GuC submission */ Maybe whack a big FIXME on here that we should unravel this properly.
Sure, can add a FIXME here.
Quoted 3 lines
Currently the execlist backend assumptions are leaked all over the place, leading to stuff like this. Which means extremely fragile code.
Yes, this something required for execlists implemented in what should be generic code.
Quoted 2 lines
I currently don't have a great idea on how exactly we should do that, but oh well.
Me either, it will be a process.
Quoted 3 lines
btw just in case we ever want to make guc lrc properly evictable (which as the og use-case for this function, way, way back), would we need to fully
Can you explain what you mean by fully evictable? Not getting what you mean in this context.
Quoted 7 lines
Quoted 1 line
unregister them from guc? At least I'm assuming there's no other trick
If scheduling is disabled on the context (currently done on unpin) you are free move anything around as the GuC is guaranteed not to touch the context state. If on re-pin something has moved (e.g. the LRC vaddr is different), you need to unregister and re-register the context with the GuC.
So at that point GuC also guarantees that it's not left in the hw engine? Execlist has this barrier request to fully unload the ctx from the hw, and that's also why I cam on the topic of OA.
Quoted 9 lines
Quoted 4 lines
like the below one.
Another aside: How does the perf/OA patching work on GuC?
Not my area of expertise but perf somewhat a WIP. The plan is for the GuC to write out some stats to HWSP I think? John Harrison is working to get this fully implemented.
Quoted 8 lines
Quoted 1 line
OA is working afaik, with Umesh Nerlige Ramappa being the expert here.
I think it's OA that I'm thinking of here: We have code in
i915_perf.cto patch all the ctx currently in the system, so that they have a consistent OA config. That's also relying on this barrier stuff, and I was wondering how that will work with GuC.
-Daniel
Not an OA expert at all but glanced at the code I don't see anything in there that prevents it working with GuC submission. We certainly have this working internally. If you have questions about this I'd reach out to Umesh Nerlige Ramappa as he likely has the answers.
Matt
Quoted 27 lines
Quoted 22 lines
Matt
Quoted 19 lines
Anyway, patch looks legit:
Reviewed-by Daniel Vetter <[email protected]>Quoted 9 lines
drivers/gpu/drm/i915/gt/intel_engine_pm.c +3
-0+if (intel_engine_uses_guc (engine)) +return true; +/* GPU is pointing to the void, as good as in the kernel context. */ if (intel_gt_is_wedged (engine->gt))return true;Signature · 1 line
2.28.0Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.chSignature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
PATCH 12/46 drm/i915/guc: Selftest for GuC flow control
In reply to Matthew BrostAdd 5 selftests for hard (from user space) to recreate flow conditions. Test listed below:
1. A test to verify that the number of guc_ids can be exhausted and all submissions still complete.
2. A test to verify that the flow control state machine can recover from a full GPU reset.
3. A teset to verify that the lrcd registration slots can be exhausted and all submissions still complete.
4. A test to verify that the H2G channel can deadlock and a full GPU reset recovers the system.
5. A test to stress to CTB channel but submitting to lots of contexts and then immediately destroy the contexts.
Tests 1, 2, and 3 also ensure when the flow control is triggered by unready requests those unready requests do not DoS ready requests.
--- drivers/gpu/drm/i915/gt/uc/intel_guc.h | 6 + drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 43 +- drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h | 9 + .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 16 + .../i915/gt/uc/intel_guc_submission_types.h | 2 + .../i915/gt/uc/selftest_guc_flow_control.c | 581 ++++++++++++++++++ .../drm/i915/selftests/i915_live_selftests.h | 1 + .../i915/selftests/intel_scheduler_helpers.c | 12 + .../i915/selftests/intel_scheduler_helpers.h | 2 + 9 files changed, 662 insertions(+), 10 deletions(-) create mode 100644 drivers/gpu/drm/i915/gt/uc/selftest_guc_flow_control.c
drivers/gpu/drm/i915/gt/uc/intel_guc.h +6 -0
@@ -103,6 +103,12 @@ struct intel_guc {/* To serialize the intel_guc_send actions */ struct mutex send_mutex; + + I915_SELFTEST_DECLARE(bool gse_hang_expected;) + I915_SELFTEST_DECLARE(bool deadlock_expected;) + I915_SELFTEST_DECLARE(bool bad_desc_expected;) + I915_SELFTEST_DECLARE(bool inject_bad_sched_disable;) + I915_SELFTEST_DECLARE(bool inject_corrupt_h2g;) };static inline struct intel_guc *log_to_guc (struct intel_guc_log *log)
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +33 -10
79 lines not on this page · raw message
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +9 -0
@@ -6,6 +6,7 @@#ifndef _INTEL_GUC_CT_H_#define _INTEL_GUC_CT_H_ +#include <linux/circ_buf.h> #include <linux/interrupt.h> #include <linux/spinlock.h> #include <linux/workqueue.h>
@@ -117,4 +118,12 @@ void intel_guc_ct_event_handler(struct intel_guc_ct *ct);void intel_guc_ct_print_info (struct intel_guc_ct *ct,struct drm_printer *p); +static inline bool intel_guc_ct_is_recv_buffer_empty (struct intel_guc_ct *ct) +{ +struct intel_guc_ct_buffer *ctb = &ct->ctbs.recv; + +return atomic_read (&ctb->space) == + (CIRC_SPACE (0 ,0 , ctb->size) - ctb->resv_space); +} +#endif /* _INTEL_GUC_CT_H_ */
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +16 -0
@@ -824,6 +824,7 @@ static int gse_dequeue_one_context(struct guc_submit_engine *gse)GEM_WARN_ON (ret);/* Unexpected */ goto deadlk; } + I915_SELFTEST_DECLARE(++gse->tasklets_submit_count;) }/*
@@ -2107,7 +2108,15 @@ static void __guc_context_sched_disable(struct intel_guc *guc, GUC_CONTEXT_DISABLE }; +#if IS_ENABLED (CONFIG_DRM_I915_SELFTEST ) +if (guc->inject_bad_sched_disable && + guc_id ==GUC_INVALID_LRC_ID ) + guc->inject_bad_sched_disable = false; +else +GEM_BUG_ON (guc_id ==GUC_INVALID_LRC_ID ); +#else GEM_BUG_ON (guc_id ==GUC_INVALID_LRC_ID ); +#endif trace_intel_context_sched_disable (ce);
@@ -2770,6 +2779,9 @@ static void retire_worker_sched_disable(struct guc_submit_engine *gse, guc_id =prep_context_pending_disable (ce);spin_unlock_irqrestore (&ce->guc_state.lock, flags); +if (I915_SELFTEST_ONLY (guc->inject_bad_sched_disable)) + guc_id =GUC_INVALID_LRC_ID ; + with_intel_runtime_pm(runtime_pm, wakeref)__guc_context_sched_disable (guc, ce, guc_id);
@@ -4021,3 +4033,7 @@ bool intel_guc_virtual_engine_has_heartbeat(const struct intel_engine_cs *ve)return false; } + +#if IS_ENABLED (CONFIG_DRM_I915_SELFTEST ) +#include "selftest_guc_flow_control.c" +#endif
drivers/gpu/drm/i915/gt/uc/intel_guc_submission_types.h +2 -0
@@ -47,6 +47,8 @@ struct guc_submit_engine {STALL_MOVE_LRC_TAIL ,STALL_ADD_REQUEST , } submission_stall_reason; + + I915_SELFTEST_DECLARE(u64 tasklets_submit_count;) };#endif
drivers/gpu/drm/i915/gt/uc/selftest_guc_flow_control.c new file +581 -0
581 lines not on this page · raw message
drivers/gpu/drm/i915/selftests/i915_live_selftests.h +1 -0
@@ -47,5 +47,6 @@ selftest(hangcheck, intel_hangcheck_live_selftests) selftest(execlists, intel_execlists_live_selftests) selftest(ring_submission, intel_ring_submission_live_selftests) selftest(perf, i915_perf_live_selftests) +selftest (guc_flow_control, intel_guc_flow_control) /* Here be dragons: keep last to run last! */selftest (late_gt_pm, intel_gt_pm_late_selftests)
drivers/gpu/drm/i915/selftests/intel_scheduler_helpers.c +12 -0
@@ -14,6 +14,18 @@#define REDUCED_PREEMPT 10#define WAIT_FOR_RESET_TIME 10000 +struct intel_engine_cs *intel_selftest_find_any_engine (struct intel_gt *gt) +{ +struct intel_engine_cs *engine; +enum intel_engine_id id; + +for_each_engine (engine, gt, id) +return engine; + +pr_err ("No valid engine found!\n" ); +return NULL ; +} +int intel_selftest_modify_policy (struct intel_engine_cs *engine,struct intel_selftest_saved_policy *saved,u32 modify_type)
drivers/gpu/drm/i915/selftests/intel_scheduler_helpers.h +2 -0
@@ -10,6 +10,7 @@struct i915_request ;struct intel_engine_cs ; +struct intel_gt ;struct intel_selftest_saved_policy {u32 flags;
@@ -29,5 +30,6 @@ int intel_selftest_modify_policy(struct intel_engine_cs *engine,int intel_selftest_restore_policy (struct intel_engine_cs *engine,struct intel_selftest_saved_policy *saved);int intel_selftest_wait_for_rq (struct i915_request *rq); +struct intel_engine_cs *intel_selftest_find_any_engine (struct intel_gt *gt);#endif
Signature · 1 line
2.28.0
PATCH 13/46 drm/i915: Add logical engine mapping
In reply to Matthew BrostAdd logical engine mapping. This is required for split-frame, as workloads need to be placed on engines in a logically contiguous manner.
--- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 60 ++++++++++++++++--- drivers/gpu/drm/i915/gt/intel_engine_types.h | 1 + .../drm/i915/gt/intel_execlists_submission.c | 1 + drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c | 2 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 21 +------ 5 files changed, 56 insertions(+), 29 deletions(-)
drivers/gpu/drm/i915/gt/intel_engine_cs.c +51 -9
@@ -290,7 +290,8 @@ static void nop_irq_handler(struct intel_engine_cs *engine, u16 iir)GEM_DEBUG_WARN_ON (iir); } -static int intel_engine_setup (struct intel_gt *gt,enum intel_engine_id id) +static int intel_engine_setup (struct intel_gt *gt,enum intel_engine_id id, +u8 logical_instance) {const struct engine_info *info = &intel_engines[id];struct drm_i915_private *i915 = gt->i915;
@@ -334,6 +335,7 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id) engine->class = info->class; engine->instance = info->instance; + engine->logical_mask =BIT (logical_instance);__sprint_engine_name (engine); engine->props.heartbeat_interval_ms =
@@ -572,6 +574,37 @@ static intel_engine_mask_t init_engine_mask(struct intel_gt *gt)return info->engine_mask; } +static void populate_logical_ids (struct intel_gt *gt,u8 *logical_ids, +u8 class,const u8 *map,u8 num_instances) +{ +int i, j; +u8 current_logical_id =0 ; + +for (j =0 ; j < num_instances; ++j) { +for (i =0 ; i <ARRAY_SIZE (intel_engines); ++i) { +if (!HAS_ENGINE (gt, i) || + intel_engines[i].class != class) +continue ; + +if (intel_engines[i].instance == map[j]) { + logical_ids[intel_engines[i].instance] = + current_logical_id++; +break ; + } + } + } +} + +static void setup_logical_ids (struct intel_gt *gt,u8 *logical_ids,u8 class) +{ +int i; +u8 map[MAX_ENGINE_INSTANCE +1 ]; + +for (i =0 ; i <MAX_ENGINE_INSTANCE +1 ; ++i) + map[i] = i; +populate_logical_ids (gt, logical_ids, class, map,ARRAY_SIZE (map)); +} +/** * intel_engines_init_mmio() - allocate and prepare the Engine Command Streamers * @gt: pointer to struct intel_gt
@@ -583,7 +616,8 @@ int intel_engines_init_mmio(struct intel_gt *gt)struct drm_i915_private *i915 = gt->i915;const unsigned int engine_mask =init_engine_mask (gt);unsigned int mask =0 ; -unsigned int i; +unsigned int i, class; +u8 logical_ids[MAX_ENGINE_INSTANCE +1 ];int err;drm_WARN_ON (&i915->drm, engine_mask ==0 );
@@ -593,15 +627,23 @@ int intel_engines_init_mmio(struct intel_gt *gt)if (i915_inject_probe_failure (i915))return -ENODEV ; -for (i =0 ; i <ARRAY_SIZE (intel_engines); i++) { -if (!HAS_ENGINE (gt, i)) -continue ; +for (class =0 ; class <MAX_ENGINE_CLASS +1 ; ++class) { +setup_logical_ids (gt, logical_ids, class); - err =intel_engine_setup (gt, i); -if (err) - goto cleanup; +for (i =0 ; i <ARRAY_SIZE (intel_engines); ++i) { +u8 instance = intel_engines[i].instance; + +if (intel_engines[i].class != class || + !HAS_ENGINE (gt, i)) +continue ; - mask |=BIT (i); + err =intel_engine_setup (gt, i, + logical_ids[instance]); +if (err) + goto cleanup; + + mask |=BIT (i); + } }/*
drivers/gpu/drm/i915/gt/intel_engine_types.h +1 -0
@@ -266,6 +266,7 @@ struct intel_engine_cs {unsigned int guc_id;intel_engine_mask_t mask; +intel_engine_mask_t logical_mask;u8 class;u8 instance;
drivers/gpu/drm/i915/gt/intel_execlists_submission.c +1 -0
@@ -3879,6 +3879,7 @@ execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count) ve->siblings[ve->num_siblings++] = sibling; ve->base.mask |= sibling->mask; + ve->base.logical_mask |= sibling->logical_mask;/* * All physical engines must be compatible for their emission
drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c +1 -1
@@ -176,7 +176,7 @@ static void guc_mapping_table_init(struct intel_gt *gt,for_each_engine (engine, gt, id) {u8 guc_class =engine_class_to_guc_class (engine->class); - system_info->mapping_table[guc_class][engine->instance] = + system_info->mapping_table[guc_class][ilog2 (engine->logical_mask)] = engine->instance; } }
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +2 -19
@@ -1795,23 +1795,6 @@ static int deregister_context(struct intel_context *ce, u32 guc_id, bool loop)return __guc_action_deregister_context (guc, guc_id, loop); } -static intel_engine_mask_t adjust_engine_mask (u8 class,intel_engine_mask_t mask) -{ -switch (class) { -case RENDER_CLASS : -return mask >>RCS0 ; -case VIDEO_ENHANCEMENT_CLASS : -return mask >>VECS0 ; -case VIDEO_DECODE_CLASS : -return mask >>VCS0 ; -case COPY_ENGINE_CLASS : -return mask >>BCS0 ; -default : -MISSING_CASE (class); -return 0 ; - } -} -static void guc_context_policy_init (struct intel_engine_cs *engine,struct guc_lrc_desc *desc) {
@@ -1952,8 +1935,7 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop) desc =__get_lrc_desc (guc, ce->guc_lrcd_reg_idx); desc->engine_class =engine_class_to_guc_class (engine->class); - desc->engine_submit_mask =adjust_engine_mask (engine->class, - engine->mask); + desc->engine_submit_mask = engine->logical_mask; desc->hw_context_desc = ce->lrc.lrca; ce->guc_prio =map_i915_prio_to_guc_prio (prio); desc->priority = ce->guc_prio;
@@ -3978,6 +3960,7 @@ guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count) } ve->base.mask |= sibling->mask; + ve->base.logical_mask |= sibling->logical_mask;if (n !=0 && ve->base.class != sibling->class) { DRM_DEBUG("invalid mixing of engine class, sibling %d, already %d\n",
Signature · 1 line
2.28.0
Re: [PATCH 13/46] drm/i915: Add logical engine mapping
Quoted 118 lines · Matthew Brost
Add logical engine mapping. This is required for split-frame, as workloads need to be placed on engines in a logically contiguous manner.
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 60 ++++++++++++++++--- drivers/gpu/drm/i915/gt/intel_engine_types.h | 1 + .../drm/i915/gt/intel_execlists_submission.c | 1 + drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c | 2 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 21 +------ 5 files changed, 56 insertions(+), 29 deletions(-)drivers/gpu/drm/i915/gt/intel_engine_cs.c +51
-9@@ -290,7 +290,8 @@ static void nop_irq_handler(struct intel_engine_cs *engine, u16 iir)GEM_DEBUG_WARN_ON (iir); } -static int intel_engine_setup (struct intel_gt *gt,enum intel_engine_id id) +static int intel_engine_setup (struct intel_gt *gt,enum intel_engine_id id, +u8 logical_instance) {const struct engine_info *info = &intel_engines[id];struct drm_i915_private *i915 = gt->i915;@@ -334,6 +335,7 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id) engine->class = info->class; engine->instance = info->instance; + engine->logical_mask =BIT (logical_instance);__sprint_engine_name (engine); engine->props.heartbeat_interval_ms =@@ -572,6 +574,37 @@ static intel_engine_mask_t init_engine_mask(struct intel_gt *gt)return info->engine_mask; } +static void populate_logical_ids (struct intel_gt *gt,u8 *logical_ids, +u8 class,const u8 *map,u8 num_instances) +{ +int i, j; +u8 current_logical_id =0 ; + +for (j =0 ; j < num_instances; ++j) { +for (i =0 ; i <ARRAY_SIZE (intel_engines); ++i) { +if (!HAS_ENGINE (gt, i) || + intel_engines[i].class != class) +continue ; + +if (intel_engines[i].instance == map[j]) { + logical_ids[intel_engines[i].instance] = + current_logical_id++; +break ; + } + } + } +} + +static void setup_logical_ids (struct intel_gt *gt,u8 *logical_ids,u8 class) +{ +int i; +u8 map[MAX_ENGINE_INSTANCE +1 ]; + +for (i =0 ; i <MAX_ENGINE_INSTANCE +1 ; ++i) + map[i] = i; +populate_logical_ids (gt, logical_ids, class, map,ARRAY_SIZE (map)); +} +/** * intel_engines_init_mmio() - allocate and prepare the Engine Command Streamers * @gt: pointer to struct intel_gt @@ -583,7 +616,8 @@ int intel_engines_init_mmio(struct intel_gt *gt)struct drm_i915_private *i915 = gt->i915;const unsigned int engine_mask =init_engine_mask (gt);unsigned int mask =0 ; -unsigned int i; +unsigned int i, class; +u8 logical_ids[MAX_ENGINE_INSTANCE +1 ];int err;drm_WARN_ON (&i915->drm, engine_mask ==0 );@@ -593,15 +627,23 @@ int intel_engines_init_mmio(struct intel_gt *gt)if (i915_inject_probe_failure (i915))return -ENODEV ; -for (i =0 ; i <ARRAY_SIZE (intel_engines); i++) { -if (!HAS_ENGINE (gt, i)) -continue ; +for (class =0 ; class <MAX_ENGINE_CLASS +1 ; ++class) { +setup_logical_ids (gt, logical_ids, class); - err =intel_engine_setup (gt, i); -if (err) - goto cleanup; +for (i =0 ; i <ARRAY_SIZE (intel_engines); ++i) { +u8 instance = intel_engines[i].instance; + +if (intel_engines[i].class != class || + !HAS_ENGINE (gt, i)) +continue ; - mask |=BIT (i); + err =intel_engine_setup (gt, i, + logical_ids[instance]); +if (err) + goto cleanup; + + mask |=BIT (i); + } }/* drivers/gpu/drm/i915/gt/intel_engine_types.h +0
-0
drivers/gpu/drm/i915/gt/intel_engine_types.h struct intel_engine_cs { · line 269
Quoted 4 lines
drivers/gpu/drm/i915/gt/intel_engine_types.h +1
-0@@ -266,6 +266,7 @@ struct intel_engine_cs {unsigned int guc_id;intel_engine_mask_t mask; +intel_engine_mask_t logical_mask;
Kerneldoc at least for new stuff. Bonus points if you get the struct/header file up to speed (with dummy/fixme comments if need be) so we can include it into our overall html hierarchy).
-Daniel
Quoted 77 lines
drivers/gpu/drm/i915/gt/intel_engine_types.h +0
-0u8 class;u8 instance;drivers/gpu/drm/i915/gt/intel_execlists_submission.c +1
-0@@ -3879,6 +3879,7 @@ execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count) ve->siblings[ve->num_siblings++] = sibling; ve->base.mask |= sibling->mask; + ve->base.logical_mask |= sibling->logical_mask;/* * All physical engines must be compatible for their emission drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c +1
-1@@ -176,7 +176,7 @@ static void guc_mapping_table_init(struct intel_gt *gt,for_each_engine (engine, gt, id) {u8 guc_class =engine_class_to_guc_class (engine->class); - system_info->mapping_table[guc_class][engine->instance] = + system_info->mapping_table[guc_class][ilog2 (engine->logical_mask)] = engine->instance; } }drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +2
-19@@ -1795,23 +1795,6 @@ static int deregister_context(struct intel_context *ce, u32 guc_id, bool loop)return __guc_action_deregister_context (guc, guc_id, loop); } -static intel_engine_mask_t adjust_engine_mask (u8 class,intel_engine_mask_t mask) -{ -switch (class) { -case RENDER_CLASS : -return mask >>RCS0 ; -case VIDEO_ENHANCEMENT_CLASS : -return mask >>VECS0 ; -case VIDEO_DECODE_CLASS : -return mask >>VCS0 ; -case COPY_ENGINE_CLASS : -return mask >>BCS0 ; -default : -MISSING_CASE (class); -return 0 ; - } -} -static void guc_context_policy_init (struct intel_engine_cs *engine,struct guc_lrc_desc *desc) {@@ -1952,8 +1935,7 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop) desc =__get_lrc_desc (guc, ce->guc_lrcd_reg_idx); desc->engine_class =engine_class_to_guc_class (engine->class); - desc->engine_submit_mask =adjust_engine_mask (engine->class, - engine->mask); + desc->engine_submit_mask = engine->logical_mask; desc->hw_context_desc = ce->lrc.lrca; ce->guc_prio =map_i915_prio_to_guc_prio (prio); desc->priority = ce->guc_prio;@@ -3978,6 +3960,7 @@ guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count) } ve->base.mask |= sibling->mask; + ve->base.logical_mask |= sibling->logical_mask;if (n !=0 && ve->base.class != sibling->class) { DRM_DEBUG("invalid mixing of engine class, sibling %d, already %d\n",Signature · 1 line
2.28.0
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [PATCH 13/46] drm/i915: Add logical engine mapping
Quoted 121 lines · Daniel Vetter
Quoted 120 lines · Matthew Brost
Add logical engine mapping. This is required for split-frame, as workloads need to be placed on engines in a logically contiguous manner.
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 60 ++++++++++++++++--- drivers/gpu/drm/i915/gt/intel_engine_types.h | 1 + .../drm/i915/gt/intel_execlists_submission.c | 1 + drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c | 2 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 21 +------ 5 files changed, 56 insertions(+), 29 deletions(-)drivers/gpu/drm/i915/gt/intel_engine_cs.c +51
-9@@ -290,7 +290,8 @@ static void nop_irq_handler(struct intel_engine_cs *engine, u16 iir)GEM_DEBUG_WARN_ON (iir); } -static int intel_engine_setup (struct intel_gt *gt,enum intel_engine_id id) +static int intel_engine_setup (struct intel_gt *gt,enum intel_engine_id id, +u8 logical_instance) {const struct engine_info *info = &intel_engines[id];struct drm_i915_private *i915 = gt->i915;@@ -334,6 +335,7 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id) engine->class = info->class; engine->instance = info->instance; + engine->logical_mask =BIT (logical_instance);__sprint_engine_name (engine); engine->props.heartbeat_interval_ms =@@ -572,6 +574,37 @@ static intel_engine_mask_t init_engine_mask(struct intel_gt *gt)return info->engine_mask; } +static void populate_logical_ids (struct intel_gt *gt,u8 *logical_ids, +u8 class,const u8 *map,u8 num_instances) +{ +int i, j; +u8 current_logical_id =0 ; + +for (j =0 ; j < num_instances; ++j) { +for (i =0 ; i <ARRAY_SIZE (intel_engines); ++i) { +if (!HAS_ENGINE (gt, i) || + intel_engines[i].class != class) +continue ; + +if (intel_engines[i].instance == map[j]) { + logical_ids[intel_engines[i].instance] = + current_logical_id++; +break ; + } + } + } +} + +static void setup_logical_ids (struct intel_gt *gt,u8 *logical_ids,u8 class) +{ +int i; +u8 map[MAX_ENGINE_INSTANCE +1 ]; + +for (i =0 ; i <MAX_ENGINE_INSTANCE +1 ; ++i) + map[i] = i; +populate_logical_ids (gt, logical_ids, class, map,ARRAY_SIZE (map)); +} +/** * intel_engines_init_mmio() - allocate and prepare the Engine Command Streamers * @gt: pointer to struct intel_gt @@ -583,7 +616,8 @@ int intel_engines_init_mmio(struct intel_gt *gt)struct drm_i915_private *i915 = gt->i915;const unsigned int engine_mask =init_engine_mask (gt);unsigned int mask =0 ; -unsigned int i; +unsigned int i, class; +u8 logical_ids[MAX_ENGINE_INSTANCE +1 ];int err;drm_WARN_ON (&i915->drm, engine_mask ==0 );@@ -593,15 +627,23 @@ int intel_engines_init_mmio(struct intel_gt *gt)if (i915_inject_probe_failure (i915))return -ENODEV ; -for (i =0 ; i <ARRAY_SIZE (intel_engines); i++) { -if (!HAS_ENGINE (gt, i)) -continue ; +for (class =0 ; class <MAX_ENGINE_CLASS +1 ; ++class) { +setup_logical_ids (gt, logical_ids, class); - err =intel_engine_setup (gt, i); -if (err) - goto cleanup; +for (i =0 ; i <ARRAY_SIZE (intel_engines); ++i) { +u8 instance = intel_engines[i].instance; + +if (intel_engines[i].class != class || + !HAS_ENGINE (gt, i)) +continue ; - mask |=BIT (i); + err =intel_engine_setup (gt, i, + logical_ids[instance]); +if (err) + goto cleanup; + + mask |=BIT (i); + } }/* drivers/gpu/drm/i915/gt/intel_engine_types.h +0
-0@@ -266,6 +266,7 @@ struct intel_engine_cs {unsigned int guc_id;
Quoted 5 lines
drivers/gpu/drm/i915/gt/intel_engine_types.h struct intel_engine_cs { · line 269
Quoted 2 lines
drivers/gpu/drm/i915/gt/intel_engine_types.h +1
-0intel_engine_mask_t mask; +intel_engine_mask_t logical_mask;Kerneldoc at least for new stuff. Bonus points if you get the struct/header file up to speed (with dummy/fixme comments if need be) so
Sure can add Kerneldoc for new variables. Def don't have time to get all structs kerneldoc up to speed at moment as by backlog is about a mile long. Perhaps after we get all of GuC submission upstream I can take sometime to go through all the structures and update the DoC.
Matt
Quoted 85 lines
we can include it into our overall html hierarchy). -Daniel
Quoted 77 lines
drivers/gpu/drm/i915/gt/intel_engine_types.h +0
-0u8 class;u8 instance;drivers/gpu/drm/i915/gt/intel_execlists_submission.c +1
-0@@ -3879,6 +3879,7 @@ execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count) ve->siblings[ve->num_siblings++] = sibling; ve->base.mask |= sibling->mask; + ve->base.logical_mask |= sibling->logical_mask;/* * All physical engines must be compatible for their emission drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c +1
-1@@ -176,7 +176,7 @@ static void guc_mapping_table_init(struct intel_gt *gt,for_each_engine (engine, gt, id) {u8 guc_class =engine_class_to_guc_class (engine->class); - system_info->mapping_table[guc_class][engine->instance] = + system_info->mapping_table[guc_class][ilog2 (engine->logical_mask)] = engine->instance; } }drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +2
-19@@ -1795,23 +1795,6 @@ static int deregister_context(struct intel_context *ce, u32 guc_id, bool loop)return __guc_action_deregister_context (guc, guc_id, loop); } -static intel_engine_mask_t adjust_engine_mask (u8 class,intel_engine_mask_t mask) -{ -switch (class) { -case RENDER_CLASS : -return mask >>RCS0 ; -case VIDEO_ENHANCEMENT_CLASS : -return mask >>VECS0 ; -case VIDEO_DECODE_CLASS : -return mask >>VCS0 ; -case COPY_ENGINE_CLASS : -return mask >>BCS0 ; -default : -MISSING_CASE (class); -return 0 ; - } -} -static void guc_context_policy_init (struct intel_engine_cs *engine,struct guc_lrc_desc *desc) {@@ -1952,8 +1935,7 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop) desc =__get_lrc_desc (guc, ce->guc_lrcd_reg_idx); desc->engine_class =engine_class_to_guc_class (engine->class); - desc->engine_submit_mask =adjust_engine_mask (engine->class, - engine->mask); + desc->engine_submit_mask = engine->logical_mask; desc->hw_context_desc = ce->lrc.lrca; ce->guc_prio =map_i915_prio_to_guc_prio (prio); desc->priority = ce->guc_prio;@@ -3978,6 +3960,7 @@ guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count) } ve->base.mask |= sibling->mask; + ve->base.logical_mask |= sibling->logical_mask;if (n !=0 && ve->base.class != sibling->class) { DRM_DEBUG("invalid mixing of engine class, sibling %d, already %d\n",Signature · 1 line
2.28.0Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [PATCH 13/46] drm/i915: Add logical engine mapping
Quoted 128 lines · Matthew Brost
Quoted 121 lines · Daniel Vetter
Quoted 120 lines · Matthew Brost
Add logical engine mapping. This is required for split-frame, as workloads need to be placed on engines in a logically contiguous manner.
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 60 ++++++++++++++++--- drivers/gpu/drm/i915/gt/intel_engine_types.h | 1 + .../drm/i915/gt/intel_execlists_submission.c | 1 + drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c | 2 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 21 +------ 5 files changed, 56 insertions(+), 29 deletions(-)drivers/gpu/drm/i915/gt/intel_engine_cs.c +51
-9@@ -290,7 +290,8 @@ static void nop_irq_handler(struct intel_engine_cs *engine, u16 iir)GEM_DEBUG_WARN_ON (iir); } -static int intel_engine_setup (struct intel_gt *gt,enum intel_engine_id id) +static int intel_engine_setup (struct intel_gt *gt,enum intel_engine_id id, +u8 logical_instance) {const struct engine_info *info = &intel_engines[id];struct drm_i915_private *i915 = gt->i915;@@ -334,6 +335,7 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id) engine->class = info->class; engine->instance = info->instance; + engine->logical_mask =BIT (logical_instance);__sprint_engine_name (engine); engine->props.heartbeat_interval_ms =@@ -572,6 +574,37 @@ static intel_engine_mask_t init_engine_mask(struct intel_gt *gt)return info->engine_mask; } +static void populate_logical_ids (struct intel_gt *gt,u8 *logical_ids, +u8 class,const u8 *map,u8 num_instances) +{ +int i, j; +u8 current_logical_id =0 ; + +for (j =0 ; j < num_instances; ++j) { +for (i =0 ; i <ARRAY_SIZE (intel_engines); ++i) { +if (!HAS_ENGINE (gt, i) || + intel_engines[i].class != class) +continue ; + +if (intel_engines[i].instance == map[j]) { + logical_ids[intel_engines[i].instance] = + current_logical_id++; +break ; + } + } + } +} + +static void setup_logical_ids (struct intel_gt *gt,u8 *logical_ids,u8 class) +{ +int i; +u8 map[MAX_ENGINE_INSTANCE +1 ]; + +for (i =0 ; i <MAX_ENGINE_INSTANCE +1 ; ++i) + map[i] = i; +populate_logical_ids (gt, logical_ids, class, map,ARRAY_SIZE (map)); +} +/** * intel_engines_init_mmio() - allocate and prepare the Engine Command Streamers * @gt: pointer to struct intel_gt @@ -583,7 +616,8 @@ int intel_engines_init_mmio(struct intel_gt *gt)struct drm_i915_private *i915 = gt->i915;const unsigned int engine_mask =init_engine_mask (gt);unsigned int mask =0 ; -unsigned int i; +unsigned int i, class; +u8 logical_ids[MAX_ENGINE_INSTANCE +1 ];int err;drm_WARN_ON (&i915->drm, engine_mask ==0 );@@ -593,15 +627,23 @@ int intel_engines_init_mmio(struct intel_gt *gt)if (i915_inject_probe_failure (i915))return -ENODEV ; -for (i =0 ; i <ARRAY_SIZE (intel_engines); i++) { -if (!HAS_ENGINE (gt, i)) -continue ; +for (class =0 ; class <MAX_ENGINE_CLASS +1 ; ++class) { +setup_logical_ids (gt, logical_ids, class); - err =intel_engine_setup (gt, i); -if (err) - goto cleanup; +for (i =0 ; i <ARRAY_SIZE (intel_engines); ++i) { +u8 instance = intel_engines[i].instance; + +if (intel_engines[i].class != class || + !HAS_ENGINE (gt, i)) +continue ; - mask |=BIT (i); + err =intel_engine_setup (gt, i, + logical_ids[instance]); +if (err) + goto cleanup; + + mask |=BIT (i); + } }/* drivers/gpu/drm/i915/gt/intel_engine_types.h +0
-0@@ -266,6 +266,7 @@ struct intel_engine_cs {unsigned int guc_id;Quoted 5 lines
drivers/gpu/drm/i915/gt/intel_engine_types.h struct intel_engine_cs { · line 269
Quoted 2 lines
drivers/gpu/drm/i915/gt/intel_engine_types.h +1
-0intel_engine_mask_t mask; +intel_engine_mask_t logical_mask;Kerneldoc at least for new stuff. Bonus points if you get the struct/header file up to speed (with dummy/fixme comments if need be) so
Quoted 4 lines
Sure can add Kerneldoc for new variables. Def don't have time to get all structs kerneldoc up to speed at moment as by backlog is about a mile long. Perhaps after we get all of GuC submission upstream I can take sometime to go through all the structures and update the DoC.
The idea isn't to add comments that are actually meaningful to all of them, but just enough to be able to pull in the header without warnings. Once you have that then any new addition will cause a warning in the doc build, which CI iirc checks. And that's pretty good baseline to have, and hence why I think it'd be good to quickly go through the motions to add these.
Really fixing this is likely a few years of work across all the structs, because in many cases the locking/coherency design is somewhere between very tricky and outright broken. Doing that in one go makes no sense. -Daniel
Quoted 88 lines
Matt
Quoted 85 lines
we can include it into our overall html hierarchy). -Daniel
Quoted 77 lines
drivers/gpu/drm/i915/gt/intel_engine_types.h +0
-0u8 class;u8 instance;drivers/gpu/drm/i915/gt/intel_execlists_submission.c +1
-0@@ -3879,6 +3879,7 @@ execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count) ve->siblings[ve->num_siblings++] = sibling; ve->base.mask |= sibling->mask; + ve->base.logical_mask |= sibling->logical_mask;/* * All physical engines must be compatible for their emission drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c +1
-1@@ -176,7 +176,7 @@ static void guc_mapping_table_init(struct intel_gt *gt,for_each_engine (engine, gt, id) {u8 guc_class =engine_class_to_guc_class (engine->class); - system_info->mapping_table[guc_class][engine->instance] = + system_info->mapping_table[guc_class][ilog2 (engine->logical_mask)] = engine->instance; } }drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +2
-19@@ -1795,23 +1795,6 @@ static int deregister_context(struct intel_context *ce, u32 guc_id, bool loop)return __guc_action_deregister_context (guc, guc_id, loop); } -static intel_engine_mask_t adjust_engine_mask (u8 class,intel_engine_mask_t mask) -{ -switch (class) { -case RENDER_CLASS : -return mask >>RCS0 ; -case VIDEO_ENHANCEMENT_CLASS : -return mask >>VECS0 ; -case VIDEO_DECODE_CLASS : -return mask >>VCS0 ; -case COPY_ENGINE_CLASS : -return mask >>BCS0 ; -default : -MISSING_CASE (class); -return 0 ; - } -} -static void guc_context_policy_init (struct intel_engine_cs *engine,struct guc_lrc_desc *desc) {@@ -1952,8 +1935,7 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop) desc =__get_lrc_desc (guc, ce->guc_lrcd_reg_idx); desc->engine_class =engine_class_to_guc_class (engine->class); - desc->engine_submit_mask =adjust_engine_mask (engine->class, - engine->mask); + desc->engine_submit_mask = engine->logical_mask; desc->hw_context_desc = ce->lrc.lrca; ce->guc_prio =map_i915_prio_to_guc_prio (prio); desc->priority = ce->guc_prio;@@ -3978,6 +3960,7 @@ guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count) } ve->base.mask |= sibling->mask; + ve->base.logical_mask |= sibling->logical_mask;if (n !=0 && ve->base.class != sibling->class) { DRM_DEBUG("invalid mixing of engine class, sibling %d, already %d\n",Signature · 1 line
2.28.0Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
PATCH 14/46 drm/i915: Expose logical engine instance to user
In reply to Matthew BrostExpose logical engine instance to user via query engine info IOCTL. This is required for split-frame workloads as these needs to be placed on engines in a logically contiguous order. The logical mapping can change based on fusing. Rather than having user have knowledge of the fusing we simply just expose the logical mapping with the existing query engine info IOCTL.
--- drivers/gpu/drm/i915/i915_query.c | 2 ++ include/uapi/drm/i915_drm.h | 8 +++++++- 2 files changed, 9 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_query.c +2 -0
@@ -124,7 +124,9 @@ query_engine_info(struct drm_i915_private *i915,for_each_uabi_engine (engine, i915) { info.engine.engine_class = engine->uabi_class; info.engine.engine_instance = engine->uabi_instance; + info.flags =I915_ENGINE_INFO_HAS_LOGICAL_INSTANCE ; info.capabilities = engine->uabi_capabilities; + info.logical_instance =ilog2 (engine->logical_mask);if (copy_to_user (info_ptr, &info,sizeof (info)))return -EFAULT ;
include/uapi/drm/i915_drm.h +7 -1
@@ -2706,14 +2706,20 @@ struct drm_i915_engine_info {/** @flags: Engine flags. */ __u64 flags; +#define I915_ENGINE_INFO_HAS_LOGICAL_INSTANCE (1 << 0)/** @capabilities: Capabilities of this engine. */ __u64 capabilities;#define I915_VIDEO_CLASS_CAPABILITY_HEVC (1 << 0)#define I915_VIDEO_AND_ENHANCE_CLASS_CAPABILITY_SFC (1 << 1) +/** @logical_instance: Logical instance of engine */ +__u16 logical_instance; +/** @rsvd1: Reserved fields. */ -__u64 rsvd1[4 ]; +__u16 rsvd1[3 ]; +/** @rsvd2: Reserved fields. */ +__u64 rsvd2[3 ]; };/**
Signature · 1 line
2.28.0
Re: [PATCH 14/46] drm/i915: Expose logical engine instance to user
Quoted 9 lines · Matthew Brost
Expose logical engine instance to user via query engine info IOCTL. This is required for split-frame workloads as these needs to be placed on engines in a logically contiguous order. The logical mapping can change based on fusing. Rather than having user have knowledge of the fusing we simply just expose the logical mapping with the existing query engine info IOCTL.
Cc Tvrtko Ursulin <[email protected]>Signed-off-by Matthew Brost <[email protected]>
Uapi must have a link to the userspace MR/patch set using this, and to the igt patch set validating it.
Ideally in each patch, since it's way too hard to unfortunately find the cover letter late on.
Jason even went as far as making this a hard requirement because he wasted a bit too much time trying to find the userspace for new uapi:
https://lore.kernel.org/dri-devel/[email protected]/
Cheers, Daniel
Quoted 48 lines
--- drivers/gpu/drm/i915/i915_query.c | 2 ++ include/uapi/drm/i915_drm.h | 8 +++++++- 2 files changed, 9 insertions(+), 1 deletion(-)drivers/gpu/drm/i915/i915_query.c +2
-0@@ -124,7 +124,9 @@ query_engine_info(struct drm_i915_private *i915,for_each_uabi_engine (engine, i915) { info.engine.engine_class = engine->uabi_class; info.engine.engine_instance = engine->uabi_instance; + info.flags =I915_ENGINE_INFO_HAS_LOGICAL_INSTANCE ; info.capabilities = engine->uabi_capabilities; + info.logical_instance =ilog2 (engine->logical_mask);if (copy_to_user (info_ptr, &info,sizeof (info)))return -EFAULT ;include/uapi/drm/i915_drm.h +7
-1@@ -2706,14 +2706,20 @@ struct drm_i915_engine_info {/** @flags: Engine flags. */ __u64 flags; +#define I915_ENGINE_INFO_HAS_LOGICAL_INSTANCE (1 << 0)/** @capabilities: Capabilities of this engine. */ __u64 capabilities;#define I915_VIDEO_CLASS_CAPABILITY_HEVC (1 << 0)#define I915_VIDEO_AND_ENHANCE_CLASS_CAPABILITY_SFC (1 << 1) +/** @logical_instance: Logical instance of engine */ +__u16 logical_instance; +/** @rsvd1: Reserved fields. */ -__u64 rsvd1[4 ]; +__u16 rsvd1[3 ]; +/** @rsvd2: Reserved fields. */ +__u64 rsvd2[3 ]; };/** Signature · 1 line
2.28.0
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [PATCH 14/46] drm/i915: Expose logical engine instance to user
Quoted 8 lines · Daniel Vetter
Quoted 7 lines · Matthew Brost
Expose logical engine instance to user via query engine info IOCTL. This is required for split-frame workloads as these needs to be placed on engines in a logically contiguous order. The logical mapping can change based on fusing. Rather than having user have knowledge of the fusing we simply just expose the logical mapping with the existing query engine info IOCTL.
Quoted 6 lines
Quoted 2 lines
Cc Tvrtko Ursulin <[email protected]>Signed-off-by Matthew Brost <[email protected]>Uapi must have a link to the userspace MR/patch set using this, and to the igt patch set validating it.
Have an IGT:
https://patchwork.freedesktop.org/patch/447008/?series=93071&rev=1
Not sure when the media UMD is going to be updated upstream to use this. Does that mean I can't merge this until the media UMD is ready? Seems like it but isn't that a circular dependency? How can the media team develop for a new uAPI that isn't in the kernel yet?
For what it is worth the downstream release is already using this.
Matt
Quoted 63 lines
Ideally in each patch, since it's way too hard to unfortunately find the cover letter late on.
Jason even went as far as making this a hard requirement because he wasted a bit too much time trying to find the userspace for new uapi:
https://lore.kernel.org/dri-devel/[email protected]/
Cheers, Daniel
Quoted 48 lines
--- drivers/gpu/drm/i915/i915_query.c | 2 ++ include/uapi/drm/i915_drm.h | 8 +++++++- 2 files changed, 9 insertions(+), 1 deletion(-)drivers/gpu/drm/i915/i915_query.c +2
-0@@ -124,7 +124,9 @@ query_engine_info(struct drm_i915_private *i915,for_each_uabi_engine (engine, i915) { info.engine.engine_class = engine->uabi_class; info.engine.engine_instance = engine->uabi_instance; + info.flags =I915_ENGINE_INFO_HAS_LOGICAL_INSTANCE ; info.capabilities = engine->uabi_capabilities; + info.logical_instance =ilog2 (engine->logical_mask);if (copy_to_user (info_ptr, &info,sizeof (info)))return -EFAULT ;include/uapi/drm/i915_drm.h +7
-1@@ -2706,14 +2706,20 @@ struct drm_i915_engine_info {/** @flags: Engine flags. */ __u64 flags; +#define I915_ENGINE_INFO_HAS_LOGICAL_INSTANCE (1 << 0)/** @capabilities: Capabilities of this engine. */ __u64 capabilities;#define I915_VIDEO_CLASS_CAPABILITY_HEVC (1 << 0)#define I915_VIDEO_AND_ENHANCE_CLASS_CAPABILITY_SFC (1 << 1) +/** @logical_instance: Logical instance of engine */ +__u16 logical_instance; +/** @rsvd1: Reserved fields. */ -__u64 rsvd1[4 ]; +__u16 rsvd1[3 ]; +/** @rsvd2: Reserved fields. */ +__u64 rsvd2[3 ]; };/** Signature · 1 line
2.28.0Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [PATCH 14/46] drm/i915: Expose logical engine instance to user
Quoted 16 lines · Matthew Brost
Quoted 8 lines · Daniel Vetter
Quoted 7 lines · Matthew Brost
Expose logical engine instance to user via query engine info IOCTL. This is required for split-frame workloads as these needs to be placed on engines in a logically contiguous order. The logical mapping can change based on fusing. Rather than having user have knowledge of the fusing we simply just expose the logical mapping with the existing query engine info IOCTL.
Quoted 6 lines
Quoted 2 lines
Cc Tvrtko Ursulin <[email protected]>Signed-off-by Matthew Brost <[email protected]>Uapi must have a link to the userspace MR/patch set using this, and to the igt patch set validating it.
Quoted 7 lines
Have an IGT:
https://patchwork.freedesktop.org/patch/447008/?series=93071&rev=1Not sure when the media UMD is going to be updated upstream to use this. Does that mean I can't merge this until the media UMD is ready? Seems like it but isn't that a circular dependency? How can the media team develop for a new uAPI that isn't in the kernel yet?
Yes and no. Full explainer here:
https://dri.freedesktop.org/docs/drm/gpu/drm-uapi.html#open-source-userspace-requirements
In the drm subsystem this is pretty much the only rule where if you break it the book will be thrown at you with extreme prejudice.
Also wrt circular: If the umd aren't set up to test their branches against kernel branches they need to fix their stuff. I know that internally that's not been done, and its a disaster, but in upstream there's no room for excuses. Both kernel and userspace needs to be in branches until it's ready for merging.
Quoted 1 line
For what it is worth the downstream release is already using this.
Yeah which is another problem, shipping new uapi in downstream before it's in upstream is decidedly not great.
-Daniel
Quoted 66 lines
Matt
Quoted 63 lines
Ideally in each patch, since it's way too hard to unfortunately find the cover letter late on.
Jason even went as far as making this a hard requirement because he wasted a bit too much time trying to find the userspace for new uapi:
https://lore.kernel.org/dri-devel/[email protected]/
Cheers, Daniel
Quoted 48 lines
--- drivers/gpu/drm/i915/i915_query.c | 2 ++ include/uapi/drm/i915_drm.h | 8 +++++++- 2 files changed, 9 insertions(+), 1 deletion(-)drivers/gpu/drm/i915/i915_query.c +2
-0@@ -124,7 +124,9 @@ query_engine_info(struct drm_i915_private *i915,for_each_uabi_engine (engine, i915) { info.engine.engine_class = engine->uabi_class; info.engine.engine_instance = engine->uabi_instance; + info.flags =I915_ENGINE_INFO_HAS_LOGICAL_INSTANCE ; info.capabilities = engine->uabi_capabilities; + info.logical_instance =ilog2 (engine->logical_mask);if (copy_to_user (info_ptr, &info,sizeof (info)))return -EFAULT ;include/uapi/drm/i915_drm.h +7
-1@@ -2706,14 +2706,20 @@ struct drm_i915_engine_info {/** @flags: Engine flags. */ __u64 flags; +#define I915_ENGINE_INFO_HAS_LOGICAL_INSTANCE (1 << 0)/** @capabilities: Capabilities of this engine. */ __u64 capabilities;#define I915_VIDEO_CLASS_CAPABILITY_HEVC (1 << 0)#define I915_VIDEO_AND_ENHANCE_CLASS_CAPABILITY_SFC (1 << 1) +/** @logical_instance: Logical instance of engine */ +__u16 logical_instance; +/** @rsvd1: Reserved fields. */ -__u64 rsvd1[4 ]; +__u16 rsvd1[3 ]; +/** @rsvd2: Reserved fields. */ +__u64 rsvd2[3 ]; };/** Signature · 1 line
2.28.0Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [PATCH 14/46] drm/i915: Expose logical engine instance to user
Quoted 25 lines · Daniel Vetter
Quoted 16 lines · Matthew Brost
Quoted 8 lines · Daniel Vetter
Quoted 7 lines · Matthew Brost
Expose logical engine instance to user via query engine info IOCTL. This is required for split-frame workloads as these needs to be placed on engines in a logically contiguous order. The logical mapping can change based on fusing. Rather than having user have knowledge of the fusing we simply just expose the logical mapping with the existing query engine info IOCTL.
Quoted 6 lines
Quoted 2 lines
Cc Tvrtko Ursulin <[email protected]>Signed-off-by Matthew Brost <[email protected]>Uapi must have a link to the userspace MR/patch set using this, and to the igt patch set validating it.
Quoted 7 lines
Have an IGT:
https://patchwork.freedesktop.org/patch/447008/?series=93071&rev=1Not sure when the media UMD is going to be updated upstream to use this. Does that mean I can't merge this until the media UMD is ready? Seems like it but isn't that a circular dependency? How can the media team develop for a new uAPI that isn't in the kernel yet?
Quoted 7 lines
Yes and no. Full explainer here:
https://dri.freedesktop.org/docs/drm/gpu/drm-uapi.html#open-source-userspace-requirements
In the drm subsystem this is pretty much the only rule where if you break it the book will be thrown at you with extreme prejudice.
Well I don't want a book thrown at, new here and trying to play by the rules.
Quoted 6 lines
Also wrt circular: If the umd aren't set up to test their branches against kernel branches they need to fix their stuff. I know that internally that's not been done, and its a disaster, but in upstream there's no room for excuses. Both kernel and userspace needs to be in branches until it's ready for merging.
Ok, looks like a have a few things to learn. I'll coordinate with the media team on this. Likely won't have links to the UMD in the next spin but I'll have a branch for them to prep their patches on.
Matt
Quoted 77 lines
Quoted 1 line
For what it is worth the downstream release is already using this.
Yeah which is another problem, shipping new uapi in downstream before it's in upstream is decidedly not great.
-DanielQuoted 66 lines
Matt
Quoted 63 lines
Ideally in each patch, since it's way too hard to unfortunately find the cover letter late on.
Jason even went as far as making this a hard requirement because he wasted a bit too much time trying to find the userspace for new uapi:
https://lore.kernel.org/dri-devel/[email protected]/
Cheers, Daniel
Quoted 48 lines
--- drivers/gpu/drm/i915/i915_query.c | 2 ++ include/uapi/drm/i915_drm.h | 8 +++++++- 2 files changed, 9 insertions(+), 1 deletion(-)drivers/gpu/drm/i915/i915_query.c +2
-0@@ -124,7 +124,9 @@ query_engine_info(struct drm_i915_private *i915,for_each_uabi_engine (engine, i915) { info.engine.engine_class = engine->uabi_class; info.engine.engine_instance = engine->uabi_instance; + info.flags =I915_ENGINE_INFO_HAS_LOGICAL_INSTANCE ; info.capabilities = engine->uabi_capabilities; + info.logical_instance =ilog2 (engine->logical_mask);if (copy_to_user (info_ptr, &info,sizeof (info)))return -EFAULT ;include/uapi/drm/i915_drm.h +7
-1@@ -2706,14 +2706,20 @@ struct drm_i915_engine_info {/** @flags: Engine flags. */ __u64 flags; +#define I915_ENGINE_INFO_HAS_LOGICAL_INSTANCE (1 << 0)/** @capabilities: Capabilities of this engine. */ __u64 capabilities;#define I915_VIDEO_CLASS_CAPABILITY_HEVC (1 << 0)#define I915_VIDEO_AND_ENHANCE_CLASS_CAPABILITY_SFC (1 << 1) +/** @logical_instance: Logical instance of engine */ +__u16 logical_instance; +/** @rsvd1: Reserved fields. */ -__u64 rsvd1[4 ]; +__u16 rsvd1[3 ]; +/** @rsvd2: Reserved fields. */ +__u64 rsvd2[3 ]; };/** Signature · 1 line
2.28.0Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.chSignature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
PATCH 15/46 drm/i915/guc: Introduce context parent-child relationship
In reply to Matthew BrostIntroduce context parent-child relationship. Once this relationship is created all pinning / unpinning operations are directed to the parent context. The parent context is responsible for pinning all of its' children and itself.
This is a precursor to the full GuC multi-lrc implementation but aligns to how GuC mutli-lrc interface is defined - a single H2G is used register / deregister all of the contexts simultaneously.
Subsequent patches in the series will implement the pinning / unpinning operations for parent / child contexts.
--- drivers/gpu/drm/i915/gt/intel_context.c | 29 +++++++++++++++++++ drivers/gpu/drm/i915/gt/intel_context.h | 18 ++++++++++++ drivers/gpu/drm/i915/gt/intel_context_types.h | 12 ++++++++ 3 files changed, 59 insertions(+)
drivers/gpu/drm/i915/gt/intel_context.c +29 -0
@@ -395,6 +395,8 @@ intel_context_init(struct intel_context *ce, struct intel_engine_cs *engine)spin_lock_init (&ce->guc_state.lock);INIT_LIST_HEAD (&ce->guc_state.fences); +INIT_LIST_HEAD (&ce->guc_child_list); +spin_lock_init (&ce->guc_active.lock);INIT_LIST_HEAD (&ce->guc_active.requests);
@@ -414,10 +416,17 @@ intel_context_init(struct intel_context *ce, struct intel_engine_cs *engine)void intel_context_fini (struct intel_context *ce) { +struct intel_context *child, *next; +if (ce->timeline)intel_timeline_put (ce->timeline);i915_vm_put (ce->vm); +/* Need to put the creation ref for the children */ +if (intel_context_is_parent (ce)) +for_each_child_safe (ce, child, next) +intel_context_put (child); +mutex_destroy (&ce->pin_mutex);i915_active_fini (&ce->active); }
@@ -533,6 +542,26 @@ struct i915_request *intel_context_find_active_request(struct intel_context *ce)return active; } +void intel_context_bind_parent_child (struct intel_context *parent, +struct intel_context *child) +{ +/* +* Callers responsibility to validate that this function is used +* correctly but we use GEM_BUG_ON here ensure that they do. +*/ +GEM_BUG_ON (!intel_engine_uses_guc (parent->engine)); +GEM_BUG_ON (intel_context_is_pinned (parent)); +GEM_BUG_ON (intel_context_is_child (parent)); +GEM_BUG_ON (intel_context_is_pinned (child)); +GEM_BUG_ON (intel_context_is_child (child)); +GEM_BUG_ON (intel_context_is_parent (child)); + + parent->guc_number_children++; +list_add_tail (&child->guc_child_link, + &parent->guc_child_list); + child->parent = parent; +} +#if IS_ENABLED (CONFIG_DRM_I915_SELFTEST )#include "selftest_context.c" #endif
drivers/gpu/drm/i915/gt/intel_context.h +18 -0
@@ -44,6 +44,24 @@ void intel_context_free(struct intel_context *ce);int intel_context_reconfigure_sseu (struct intel_context *ce,const struct intel_sseu sseu); +static inline bool intel_context_is_child (struct intel_context *ce) +{ +return !!ce->parent; +} + +static inline bool intel_context_is_parent (struct intel_context *ce) +{ +return !!ce->guc_number_children; +} + +void intel_context_bind_parent_child (struct intel_context *parent, +struct intel_context *child); + +#define for_each_child (parent, ce)\ + list_for_each_entry(ce, &(parent)->guc_child_list, guc_child_link) +#define for_each_child_safe (parent, ce, cn)\ + list_for_each_entry_safe(ce, cn, &(parent)->guc_child_list, guc_child_link) +/** * intel_context_lock_pinned - Stablises the 'pinned' status of the HW context * @ce - the context
drivers/gpu/drm/i915/gt/intel_context_types.h +12 -0
@@ -202,6 +202,18 @@ struct intel_context {/* GuC context blocked fence */ struct i915_sw_fence guc_blocked; +/* Head of children list or link in parent's children list */ +union { +struct list_head guc_child_list;/* parent */ +struct list_head guc_child_link;/* child */ + }; + +/* Pointer to parent */ +struct intel_context *parent; + +/* Number of children if parent */ +u8 guc_number_children; +/* * GuC priority management */
Signature · 1 line
2.28.0
Re: [Intel-gfx] [PATCH 15/46] drm/i915/guc: Introduce context parent-child relationship
Quoted 111 lines · Matthew Brost
Introduce context parent-child relationship. Once this relationship is created all pinning / unpinning operations are directed to the parent context. The parent context is responsible for pinning all of its' children and itself.
This is a precursor to the full GuC multi-lrc implementation but aligns to how GuC mutli-lrc interface is defined - a single H2G is used register / deregister all of the contexts simultaneously.
Subsequent patches in the series will implement the pinning / unpinning operations for parent / child contexts.
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/gt/intel_context.c | 29 +++++++++++++++++++ drivers/gpu/drm/i915/gt/intel_context.h | 18 ++++++++++++ drivers/gpu/drm/i915/gt/intel_context_types.h | 12 ++++++++ 3 files changed, 59 insertions(+)drivers/gpu/drm/i915/gt/intel_context.c +29
-0@@ -395,6 +395,8 @@ intel_context_init(struct intel_context *ce, struct intel_engine_cs *engine)spin_lock_init (&ce->guc_state.lock);INIT_LIST_HEAD (&ce->guc_state.fences); +INIT_LIST_HEAD (&ce->guc_child_list); +spin_lock_init (&ce->guc_active.lock);INIT_LIST_HEAD (&ce->guc_active.requests);@@ -414,10 +416,17 @@ intel_context_init(struct intel_context *ce, struct intel_engine_cs *engine)void intel_context_fini (struct intel_context *ce) { +struct intel_context *child, *next; +if (ce->timeline)intel_timeline_put (ce->timeline);i915_vm_put (ce->vm); +/* Need to put the creation ref for the children */ +if (intel_context_is_parent (ce)) +for_each_child_safe (ce, child, next) +intel_context_put (child); +mutex_destroy (&ce->pin_mutex);i915_active_fini (&ce->active); }@@ -533,6 +542,26 @@ struct i915_request *intel_context_find_active_request(struct intel_context *ce)return active; } +void intel_context_bind_parent_child (struct intel_context *parent, +struct intel_context *child) +{ +/* +* Callers responsibility to validate that this function is used +* correctly but we use GEM_BUG_ON here ensure that they do. +*/ +GEM_BUG_ON (!intel_engine_uses_guc (parent->engine)); +GEM_BUG_ON (intel_context_is_pinned (parent)); +GEM_BUG_ON (intel_context_is_child (parent)); +GEM_BUG_ON (intel_context_is_pinned (child)); +GEM_BUG_ON (intel_context_is_child (child)); +GEM_BUG_ON (intel_context_is_parent (child)); + + parent->guc_number_children++; +list_add_tail (&child->guc_child_link, + &parent->guc_child_list); + child->parent = parent; +} +#if IS_ENABLED (CONFIG_DRM_I915_SELFTEST )#include "selftest_context.c" #endif drivers/gpu/drm/i915/gt/intel_context.h +18
-0@@ -44,6 +44,24 @@ void intel_context_free(struct intel_context *ce);int intel_context_reconfigure_sseu (struct intel_context *ce,const struct intel_sseu sseu); +static inline bool intel_context_is_child (struct intel_context *ce) +{ +return !!ce->parent; +} + +static inline bool intel_context_is_parent (struct intel_context *ce) +{ +return !!ce->guc_number_children; +} + +void intel_context_bind_parent_child (struct intel_context *parent, +struct intel_context *child); + +#define for_each_child (parent, ce)\ + list_for_each_entry(ce, &(parent)->guc_child_list, guc_child_link) +#define for_each_child_safe (parent, ce, cn)\ + list_for_each_entry_safe(ce, cn, &(parent)->guc_child_list, guc_child_link) +/** * intel_context_lock_pinned - Stablises the 'pinned' status of the HW context * @ce - the context drivers/gpu/drm/i915/gt/intel_context_types.h +0
-0
drivers/gpu/drm/i915/gt/intel_context_types.h struct intel_context { · line 205
Quoted 4 lines
drivers/gpu/drm/i915/gt/intel_context_types.h +1
-0@@ -202,6 +202,18 @@ struct intel_context {/* GuC context blocked fence */ struct i915_sw_fence guc_blocked; +/* Head of children list or link in parent's children list */
Kerneldoc layout would be nice, plus explaining when exactly this is set or the list empty (e.g. guch_child_list is empty if and only if guc_number_children > 0 and parent == NULL).
Also mentionting that these are invariant over the lifetime of the object would be nice.
Finally some words on refcounting (like who holds a reference on whom and how we guarantee that use-after-free doesn't go boom since you have links both ways). It looks like parent holds a reference on the child, so how do you make sure the child looking at the parent doesn't go boom? -Daniel
Quoted 17 lines
drivers/gpu/drm/i915/gt/intel_context_types.h +11
-0+union { +struct list_head guc_child_list;/* parent */ +struct list_head guc_child_link;/* child */ + }; + +/* Pointer to parent */ +struct intel_context *parent; + +/* Number of children if parent */ +u8 guc_number_children; +/* * GuC priority management */ Signature · 1 line
2.28.0
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [Intel-gfx] [PATCH 15/46] drm/i915/guc: Introduce context parent-child relationship
Quoted 130 lines · Daniel Vetter
Quoted 111 lines · Matthew Brost
Introduce context parent-child relationship. Once this relationship is created all pinning / unpinning operations are directed to the parent context. The parent context is responsible for pinning all of its' children and itself.
This is a precursor to the full GuC multi-lrc implementation but aligns to how GuC mutli-lrc interface is defined - a single H2G is used register / deregister all of the contexts simultaneously.
Subsequent patches in the series will implement the pinning / unpinning operations for parent / child contexts.
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/gt/intel_context.c | 29 +++++++++++++++++++ drivers/gpu/drm/i915/gt/intel_context.h | 18 ++++++++++++ drivers/gpu/drm/i915/gt/intel_context_types.h | 12 ++++++++ 3 files changed, 59 insertions(+)drivers/gpu/drm/i915/gt/intel_context.c +29
-0@@ -395,6 +395,8 @@ intel_context_init(struct intel_context *ce, struct intel_engine_cs *engine)spin_lock_init (&ce->guc_state.lock);INIT_LIST_HEAD (&ce->guc_state.fences); +INIT_LIST_HEAD (&ce->guc_child_list); +spin_lock_init (&ce->guc_active.lock);INIT_LIST_HEAD (&ce->guc_active.requests);@@ -414,10 +416,17 @@ intel_context_init(struct intel_context *ce, struct intel_engine_cs *engine)void intel_context_fini (struct intel_context *ce) { +struct intel_context *child, *next; +if (ce->timeline)intel_timeline_put (ce->timeline);i915_vm_put (ce->vm); +/* Need to put the creation ref for the children */ +if (intel_context_is_parent (ce)) +for_each_child_safe (ce, child, next) +intel_context_put (child); +mutex_destroy (&ce->pin_mutex);i915_active_fini (&ce->active); }@@ -533,6 +542,26 @@ struct i915_request *intel_context_find_active_request(struct intel_context *ce)return active; } +void intel_context_bind_parent_child (struct intel_context *parent, +struct intel_context *child) +{ +/* +* Callers responsibility to validate that this function is used +* correctly but we use GEM_BUG_ON here ensure that they do. +*/ +GEM_BUG_ON (!intel_engine_uses_guc (parent->engine)); +GEM_BUG_ON (intel_context_is_pinned (parent)); +GEM_BUG_ON (intel_context_is_child (parent)); +GEM_BUG_ON (intel_context_is_pinned (child)); +GEM_BUG_ON (intel_context_is_child (child)); +GEM_BUG_ON (intel_context_is_parent (child)); + + parent->guc_number_children++; +list_add_tail (&child->guc_child_link, + &parent->guc_child_list); + child->parent = parent; +} +#if IS_ENABLED (CONFIG_DRM_I915_SELFTEST )#include "selftest_context.c" #endif drivers/gpu/drm/i915/gt/intel_context.h +18
-0@@ -44,6 +44,24 @@ void intel_context_free(struct intel_context *ce);int intel_context_reconfigure_sseu (struct intel_context *ce,const struct intel_sseu sseu); +static inline bool intel_context_is_child (struct intel_context *ce) +{ +return !!ce->parent; +} + +static inline bool intel_context_is_parent (struct intel_context *ce) +{ +return !!ce->guc_number_children; +} + +void intel_context_bind_parent_child (struct intel_context *parent, +struct intel_context *child); + +#define for_each_child (parent, ce)\ + list_for_each_entry(ce, &(parent)->guc_child_list, guc_child_link) +#define for_each_child_safe (parent, ce, cn)\ + list_for_each_entry_safe(ce, cn, &(parent)->guc_child_list, guc_child_link) +/** * intel_context_lock_pinned - Stablises the 'pinned' status of the HW context * @ce - the context drivers/gpu/drm/i915/gt/intel_context_types.h +0
-0drivers/gpu/drm/i915/gt/intel_context_types.h struct intel_context { · line 205
Quoted 4 lines
drivers/gpu/drm/i915/gt/intel_context_types.h +1
-0@@ -202,6 +202,18 @@ struct intel_context {/* GuC context blocked fence */ struct i915_sw_fence guc_blocked; +/* Head of children list or link in parent's children list */ Kerneldoc layout would be nice, plus explaining when exactly this is set or the list empty (e.g.
guch_child_listis empty if and only ifguc_number_children> 0 and parent == NULL).Also mentionting that these are invariant over the lifetime of the object would be nice.
Finally some words on refcounting (like who holds a reference on whom and how we guarantee that use-after-free doesn't go boom since you have links both ways). It looks like parent holds a reference on the child, so how do you make sure the child looking at the parent doesn't go boom? -Daniel
Quoted 10 lines
drivers/gpu/drm/i915/gt/intel_context_types.h struct intel_context { · line 215
Quoted 10 lines
drivers/gpu/drm/i915/gt/intel_context_types.h +10
-0+union { +struct list_head guc_child_list;/* parent */ +struct list_head guc_child_link;/* child */ + }; + +/* Pointer to parent */ +struct intel_context *parent; + +/* Number of children if parent */ +u8 guc_number_children;
Another one: Can we really not afford a int here? The nasty thing about unsigned is that wrap-around is well-defined, which is why gcc won't ever complain about it. Which hides bugs. Same for next patch, which also micro-optimizes a few fields to be tiny.
We generally don't have thousands of contexts hanging around, unless there's a reason (which should be documented) this feels like it's squarely on the wrong side of "don't prematurely optimize". -Daniel
Quoted 12 lines
Quoted 7 lines
drivers/gpu/drm/i915/gt/intel_context_types.h +1
-0+/* * GuC priority management */ Signature · 1 line
2.28.0Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [Intel-gfx] [PATCH 15/46] drm/i915/guc: Introduce context parent-child relationship
Quoted 147 lines · Daniel Vetter
Quoted 130 lines · Daniel Vetter
Quoted 111 lines · Matthew Brost
Introduce context parent-child relationship. Once this relationship is created all pinning / unpinning operations are directed to the parent context. The parent context is responsible for pinning all of its' children and itself.
This is a precursor to the full GuC multi-lrc implementation but aligns to how GuC mutli-lrc interface is defined - a single H2G is used register / deregister all of the contexts simultaneously.
Subsequent patches in the series will implement the pinning / unpinning operations for parent / child contexts.
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/gt/intel_context.c | 29 +++++++++++++++++++ drivers/gpu/drm/i915/gt/intel_context.h | 18 ++++++++++++ drivers/gpu/drm/i915/gt/intel_context_types.h | 12 ++++++++ 3 files changed, 59 insertions(+)drivers/gpu/drm/i915/gt/intel_context.c +29
-0@@ -395,6 +395,8 @@ intel_context_init(struct intel_context *ce, struct intel_engine_cs *engine)spin_lock_init (&ce->guc_state.lock);INIT_LIST_HEAD (&ce->guc_state.fences); +INIT_LIST_HEAD (&ce->guc_child_list); +spin_lock_init (&ce->guc_active.lock);INIT_LIST_HEAD (&ce->guc_active.requests);@@ -414,10 +416,17 @@ intel_context_init(struct intel_context *ce, struct intel_engine_cs *engine)void intel_context_fini (struct intel_context *ce) { +struct intel_context *child, *next; +if (ce->timeline)intel_timeline_put (ce->timeline);i915_vm_put (ce->vm); +/* Need to put the creation ref for the children */ +if (intel_context_is_parent (ce)) +for_each_child_safe (ce, child, next) +intel_context_put (child); +mutex_destroy (&ce->pin_mutex);i915_active_fini (&ce->active); }@@ -533,6 +542,26 @@ struct i915_request *intel_context_find_active_request(struct intel_context *ce)return active; } +void intel_context_bind_parent_child (struct intel_context *parent, +struct intel_context *child) +{ +/* +* Callers responsibility to validate that this function is used +* correctly but we use GEM_BUG_ON here ensure that they do. +*/ +GEM_BUG_ON (!intel_engine_uses_guc (parent->engine)); +GEM_BUG_ON (intel_context_is_pinned (parent)); +GEM_BUG_ON (intel_context_is_child (parent)); +GEM_BUG_ON (intel_context_is_pinned (child)); +GEM_BUG_ON (intel_context_is_child (child)); +GEM_BUG_ON (intel_context_is_parent (child)); + + parent->guc_number_children++; +list_add_tail (&child->guc_child_link, + &parent->guc_child_list); + child->parent = parent; +} +#if IS_ENABLED (CONFIG_DRM_I915_SELFTEST )#include "selftest_context.c" #endif drivers/gpu/drm/i915/gt/intel_context.h +18
-0@@ -44,6 +44,24 @@ void intel_context_free(struct intel_context *ce);int intel_context_reconfigure_sseu (struct intel_context *ce,const struct intel_sseu sseu); +static inline bool intel_context_is_child (struct intel_context *ce) +{ +return !!ce->parent; +} + +static inline bool intel_context_is_parent (struct intel_context *ce) +{ +return !!ce->guc_number_children; +} + +void intel_context_bind_parent_child (struct intel_context *parent, +struct intel_context *child); + +#define for_each_child (parent, ce)\ + list_for_each_entry(ce, &(parent)->guc_child_list, guc_child_link) +#define for_each_child_safe (parent, ce, cn)\ + list_for_each_entry_safe(ce, cn, &(parent)->guc_child_list, guc_child_link) +/** * intel_context_lock_pinned - Stablises the 'pinned' status of the HW context * @ce - the context drivers/gpu/drm/i915/gt/intel_context_types.h +0
-0drivers/gpu/drm/i915/gt/intel_context_types.h struct intel_context { · line 205
Quoted 4 lines
drivers/gpu/drm/i915/gt/intel_context_types.h +1
-0@@ -202,6 +202,18 @@ struct intel_context {/* GuC context blocked fence */ struct i915_sw_fence guc_blocked; +/* Head of children list or link in parent's children list */ Kerneldoc layout would be nice, plus explaining when exactly this is set or the list empty (e.g.
guch_child_listis empty if and only ifguc_number_children> 0 and parent == NULL).Also mentionting that these are invariant over the lifetime of the object would be nice.
Finally some words on refcounting (like who holds a reference on whom and how we guarantee that use-after-free doesn't go boom since you have links both ways). It looks like parent holds a reference on the child, so how do you make sure the child looking at the parent doesn't go boom? -Daniel
Quoted 10 lines
drivers/gpu/drm/i915/gt/intel_context_types.h struct intel_context { · line 215
Quoted 10 lines
drivers/gpu/drm/i915/gt/intel_context_types.h +10
-0+union { +struct list_head guc_child_list;/* parent */ +struct list_head guc_child_link;/* child */ + }; + +/* Pointer to parent */ +struct intel_context *parent; + +/* Number of children if parent */ +u8 guc_number_children;Another one: Can we really not afford a int here? The nasty thing about unsigned is that wrap-around is well-defined, which is why gcc won't ever complain about it. Which hides bugs. Same for next patch, which also micro-optimizes a few fields to be tiny.
Quoted 3 lines
We generally don't have thousands of contexts hanging around, unless there's a reason (which should be documented) this feels like it's squarely on the wrong side of "don't prematurely optimize".
Ok, int it is.
Matt
Quoted 19 lines
-Daniel
Quoted 12 lines
Quoted 7 lines
drivers/gpu/drm/i915/gt/intel_context_types.h +1
-0+/* * GuC priority management */ Signature · 1 line
2.28.0Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.chSignature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [Intel-gfx] [PATCH 15/46] drm/i915/guc: Introduce context parent-child relationship
In reply to Daniel VetterQuoted 115 lines · Daniel Vetter
Quoted 114 lines · Matthew Brost
Introduce context parent-child relationship. Once this relationship is created all pinning / unpinning operations are directed to the parent context. The parent context is responsible for pinning all of its' children and itself.
This is a precursor to the full GuC multi-lrc implementation but aligns to how GuC mutli-lrc interface is defined - a single H2G is used register / deregister all of the contexts simultaneously.
Subsequent patches in the series will implement the pinning / unpinning operations for parent / child contexts.
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/gt/intel_context.c | 29 +++++++++++++++++++ drivers/gpu/drm/i915/gt/intel_context.h | 18 ++++++++++++ drivers/gpu/drm/i915/gt/intel_context_types.h | 12 ++++++++ 3 files changed, 59 insertions(+)drivers/gpu/drm/i915/gt/intel_context.c +29
-0@@ -395,6 +395,8 @@ intel_context_init(struct intel_context *ce, struct intel_engine_cs *engine)spin_lock_init (&ce->guc_state.lock);INIT_LIST_HEAD (&ce->guc_state.fences); +INIT_LIST_HEAD (&ce->guc_child_list); +spin_lock_init (&ce->guc_active.lock);INIT_LIST_HEAD (&ce->guc_active.requests);@@ -414,10 +416,17 @@ intel_context_init(struct intel_context *ce, struct intel_engine_cs *engine)void intel_context_fini (struct intel_context *ce) { +struct intel_context *child, *next; +if (ce->timeline)intel_timeline_put (ce->timeline);i915_vm_put (ce->vm); +/* Need to put the creation ref for the children */ +if (intel_context_is_parent (ce)) +for_each_child_safe (ce, child, next) +intel_context_put (child); +mutex_destroy (&ce->pin_mutex);i915_active_fini (&ce->active); }@@ -533,6 +542,26 @@ struct i915_request *intel_context_find_active_request(struct intel_context *ce)return active; } +void intel_context_bind_parent_child (struct intel_context *parent, +struct intel_context *child) +{ +/* +* Callers responsibility to validate that this function is used +* correctly but we use GEM_BUG_ON here ensure that they do. +*/ +GEM_BUG_ON (!intel_engine_uses_guc (parent->engine)); +GEM_BUG_ON (intel_context_is_pinned (parent)); +GEM_BUG_ON (intel_context_is_child (parent)); +GEM_BUG_ON (intel_context_is_pinned (child)); +GEM_BUG_ON (intel_context_is_child (child)); +GEM_BUG_ON (intel_context_is_parent (child)); + + parent->guc_number_children++; +list_add_tail (&child->guc_child_link, + &parent->guc_child_list); + child->parent = parent; +} +#if IS_ENABLED (CONFIG_DRM_I915_SELFTEST )#include "selftest_context.c" #endif drivers/gpu/drm/i915/gt/intel_context.h +18
-0@@ -44,6 +44,24 @@ void intel_context_free(struct intel_context *ce);int intel_context_reconfigure_sseu (struct intel_context *ce,const struct intel_sseu sseu); +static inline bool intel_context_is_child (struct intel_context *ce) +{ +return !!ce->parent; +} + +static inline bool intel_context_is_parent (struct intel_context *ce) +{ +return !!ce->guc_number_children; +} + +void intel_context_bind_parent_child (struct intel_context *parent, +struct intel_context *child); + +#define for_each_child (parent, ce)\ + list_for_each_entry(ce, &(parent)->guc_child_list, guc_child_link) +#define for_each_child_safe (parent, ce, cn)\ + list_for_each_entry_safe(ce, cn, &(parent)->guc_child_list, guc_child_link) +/** * intel_context_lock_pinned - Stablises the 'pinned' status of the HW context * @ce - the context drivers/gpu/drm/i915/gt/intel_context_types.h +0
-0@@ -202,6 +202,18 @@ struct intel_context {/* GuC context blocked fence */ struct i915_sw_fence guc_blocked;
Quoted 6 lines
drivers/gpu/drm/i915/gt/intel_context_types.h struct intel_context { · line 205
Quoted 1 line
drivers/gpu/drm/i915/gt/intel_context_types.h +1
-0+/* Head of children list or link in parent's children list */ Kerneldoc layout would be nice, plus explaining when exactly this is set or the list empty (e.g.
guch_child_listis empty if and only ifguc_number_children> 0 and parent == NULL).
Sure.
Quoted 3 lines
Also mentionting that these are invariant over the lifetime of the object would be nice.
Yes, this is a context creation setup step that is done exactly once and is invariant over the lifetime of these contexts.
Quoted 4 lines
Finally some words on refcounting (like who holds a reference on whom and how we guarantee that use-after-free doesn't go boom since you have links both ways). It looks like parent holds a reference on the child, so how do you make sure the child looking at the parent doesn't go boom?
I hadn't really thought about the child looking at the parent but I believe it is safe. The child only looks up the parent when submissions are in flight. We always have refs on the contexts when submissions are in flight so we should be good - e.g. the last ref to parent is dropped only after all submissions are done and the context is closed.
Matt
Quoted 24 lines
-Daniel
Quoted 17 lines
drivers/gpu/drm/i915/gt/intel_context_types.h +11
-0+union { +struct list_head guc_child_list;/* parent */ +struct list_head guc_child_link;/* child */ + }; + +/* Pointer to parent */ +struct intel_context *parent; + +/* Number of children if parent */ +u8 guc_number_children; +/* * GuC priority management */ Signature · 1 line
2.28.0Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [Intel-gfx] [PATCH 15/46] drm/i915/guc: Introduce context parent-child relationship
Quoted 137 lines · Matthew Brost
Quoted 115 lines · Daniel Vetter
Quoted 114 lines · Matthew Brost
Introduce context parent-child relationship. Once this relationship is created all pinning / unpinning operations are directed to the parent context. The parent context is responsible for pinning all of its' children and itself.
This is a precursor to the full GuC multi-lrc implementation but aligns to how GuC mutli-lrc interface is defined - a single H2G is used register / deregister all of the contexts simultaneously.
Subsequent patches in the series will implement the pinning / unpinning operations for parent / child contexts.
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/gt/intel_context.c | 29 +++++++++++++++++++ drivers/gpu/drm/i915/gt/intel_context.h | 18 ++++++++++++ drivers/gpu/drm/i915/gt/intel_context_types.h | 12 ++++++++ 3 files changed, 59 insertions(+)drivers/gpu/drm/i915/gt/intel_context.c +29
-0@@ -395,6 +395,8 @@ intel_context_init(struct intel_context *ce, struct intel_engine_cs *engine)spin_lock_init (&ce->guc_state.lock);INIT_LIST_HEAD (&ce->guc_state.fences); +INIT_LIST_HEAD (&ce->guc_child_list); +spin_lock_init (&ce->guc_active.lock);INIT_LIST_HEAD (&ce->guc_active.requests);@@ -414,10 +416,17 @@ intel_context_init(struct intel_context *ce, struct intel_engine_cs *engine)void intel_context_fini (struct intel_context *ce) { +struct intel_context *child, *next; +if (ce->timeline)intel_timeline_put (ce->timeline);i915_vm_put (ce->vm); +/* Need to put the creation ref for the children */ +if (intel_context_is_parent (ce)) +for_each_child_safe (ce, child, next) +intel_context_put (child); +mutex_destroy (&ce->pin_mutex);i915_active_fini (&ce->active); }@@ -533,6 +542,26 @@ struct i915_request *intel_context_find_active_request(struct intel_context *ce)return active; } +void intel_context_bind_parent_child (struct intel_context *parent, +struct intel_context *child) +{ +/* +* Callers responsibility to validate that this function is used +* correctly but we use GEM_BUG_ON here ensure that they do. +*/ +GEM_BUG_ON (!intel_engine_uses_guc (parent->engine)); +GEM_BUG_ON (intel_context_is_pinned (parent)); +GEM_BUG_ON (intel_context_is_child (parent)); +GEM_BUG_ON (intel_context_is_pinned (child)); +GEM_BUG_ON (intel_context_is_child (child)); +GEM_BUG_ON (intel_context_is_parent (child)); + + parent->guc_number_children++; +list_add_tail (&child->guc_child_link, + &parent->guc_child_list); + child->parent = parent; +} +#if IS_ENABLED (CONFIG_DRM_I915_SELFTEST )#include "selftest_context.c" #endif drivers/gpu/drm/i915/gt/intel_context.h +18
-0@@ -44,6 +44,24 @@ void intel_context_free(struct intel_context *ce);int intel_context_reconfigure_sseu (struct intel_context *ce,const struct intel_sseu sseu); +static inline bool intel_context_is_child (struct intel_context *ce) +{ +return !!ce->parent; +} + +static inline bool intel_context_is_parent (struct intel_context *ce) +{ +return !!ce->guc_number_children; +} + +void intel_context_bind_parent_child (struct intel_context *parent, +struct intel_context *child); + +#define for_each_child (parent, ce)\ + list_for_each_entry(ce, &(parent)->guc_child_list, guc_child_link) +#define for_each_child_safe (parent, ce, cn)\ + list_for_each_entry_safe(ce, cn, &(parent)->guc_child_list, guc_child_link) +/** * intel_context_lock_pinned - Stablises the 'pinned' status of the HW context * @ce - the context drivers/gpu/drm/i915/gt/intel_context_types.h +0
-0@@ -202,6 +202,18 @@ struct intel_context {/* GuC context blocked fence */ struct i915_sw_fence guc_blocked;Quoted 6 lines
drivers/gpu/drm/i915/gt/intel_context_types.h struct intel_context { · line 205
Quoted 1 line
drivers/gpu/drm/i915/gt/intel_context_types.h +1
-0+/* Head of children list or link in parent's children list */ Kerneldoc layout would be nice, plus explaining when exactly this is set or the list empty (e.g.
guch_child_listis empty if and only ifguc_number_children> 0 and parent == NULL).Sure.
Quoted 3 lines
Also mentionting that these are invariant over the lifetime of the object would be nice.
Yes, this is a context creation setup step that is done exactly once and is invariant over the lifetime of these contexts.
Quoted 4 lines
Finally some words on refcounting (like who holds a reference on whom and how we guarantee that use-after-free doesn't go boom since you have links both ways). It looks like parent holds a reference on the child, so how do you make sure the child looking at the parent doesn't go boom?
Quoted 5 lines
I hadn't really thought about the child looking at the parent but I believe it is safe. The child only looks up the parent when submissions are in flight. We always have refs on the contexts when submissions are in flight so we should be good - e.g. the last ref to parent is dropped only after all submissions are done and the context is closed.
Yeah that's pretty much the only safe option I could come up with too. Please
- document this
- enforce it with checks. I think a wrapper to get at the parent, which a) can fail and b) checks that the child request is not yet signalled should do. Something with try_get or whatever it the name to signal it can fail is best.
Then the rule is that the unsignalled child request has an implicit reference on the parent as long as it's unsignalled, but not afterwards. It might also be good to clear out the parent pointer before signalling the request. If that races in funny ways there's definitely more problems. -Daniel
Quoted 27 lines
Matt
Quoted 24 lines
-Daniel
Quoted 17 lines
drivers/gpu/drm/i915/gt/intel_context_types.h +11
-0+union { +struct list_head guc_child_list;/* parent */ +struct list_head guc_child_link;/* child */ + }; + +/* Pointer to parent */ +struct intel_context *parent; + +/* Number of children if parent */ +u8 guc_number_children; +/* * GuC priority management */ Signature · 1 line
2.28.0Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
PATCH 16/46 drm/i915/guc: Implement GuC parent-child context pin / unpin functions
In reply to Matthew BrostImplement GuC parent-child context pin / unpin functions in which in any contexts in the relationship are pinned all the contexts are pinned. The parent owns most of the pinning / unpinning process and the children direct any pins / unpins to the parent.
Patch implements a number of unused functions that will be connected later in the series.
--- drivers/gpu/drm/i915/gt/intel_context.c | 187 ++++++++++++++++-- drivers/gpu/drm/i915/gt/intel_context.h | 43 +--- drivers/gpu/drm/i915/gt/intel_context_types.h | 4 +- .../drm/i915/gt/intel_execlists_submission.c | 25 ++- drivers/gpu/drm/i915/gt/intel_lrc.c | 26 +-- drivers/gpu/drm/i915/gt/intel_lrc.h | 6 +- .../gpu/drm/i915/gt/intel_ring_submission.c | 5 +- drivers/gpu/drm/i915/gt/mock_engine.c | 4 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 183 +++++++++++++++-- 9 files changed, 371 insertions(+), 112 deletions(-)
drivers/gpu/drm/i915/gt/intel_context.c +169 -18
@@ -158,8 +158,8 @@ static void __ring_retire(struct intel_ring *ring)intel_ring_unpin (ring); } -static int intel_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) +static int __intel_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) {int err;
@@ -190,7 +190,7 @@ static int intel_context_pre_pin(struct intel_context *ce,return err; } -static void intel_context_post_unpin (struct intel_context *ce) +static void __intel_context_post_unpin (struct intel_context *ce) {if (ce->state)__context_unpin_state (ce->state);
@@ -199,13 +199,85 @@ static void intel_context_post_unpin(struct intel_context *ce)__ring_retire (ce->ring); } -int __intel_context_do_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) +static int intel_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) { -bool handoff = false; -void *vaddr; +struct intel_context *child; +int err, i =0 ; + +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) { + err =__intel_context_pre_pin (child, ww); +if (unlikely (err)) + goto unwind; + ++i; + } + + err =__intel_context_pre_pin (ce, ww); +if (unlikely (err)) + goto unwind; + +return 0 ; + +unwind: +for_each_child (ce, child) { +if (!i--) +break ; +__intel_context_post_unpin (ce); + } + +return err; +} + +static void intel_context_post_unpin (struct intel_context *ce) +{ +struct intel_context *child; + +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) +__intel_context_post_unpin (child); + +__intel_context_post_unpin (ce); +} + +static int __do_ww_lock (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +int err =i915_gem_object_lock (ce->timeline->hwsp_ggtt->obj, ww); + +if (!err && ce->ring->vma->obj) + err =i915_gem_object_lock (ce->ring->vma->obj, ww); +if (!err && ce->state) + err =i915_gem_object_lock (ce->state->obj, ww); + +return err; +} + +static int do_ww_lock (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +struct intel_context *child;int err =0 ; +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) { + err =__do_ww_lock (child, ww); +if (unlikely (err)) +return err; + } + +return __do_ww_lock (ce, ww); +} + +static int __intel_context_do_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +bool handoff = false; +int err; +if (unlikely (!test_bit (CONTEXT_ALLOC_BIT , &ce->flags))) { err =intel_context_alloc_state (ce);if (err)
@@ -217,14 +289,11 @@ int __intel_context_do_pin_ww(struct intel_context *ce,* refcount for __intel_context_active(), which prevent a lock * inversion of ce->pin_mutex vs dma_resv_lock(). */ + err =do_ww_lock (ce, ww); +if (err) +return err; - err =i915_gem_object_lock (ce->timeline->hwsp_ggtt->obj, ww); -if (!err && ce->ring->vma->obj) - err =i915_gem_object_lock (ce->ring->vma->obj, ww); -if (!err && ce->state) - err =i915_gem_object_lock (ce->state->obj, ww); -if (!err) - err =intel_context_pre_pin (ce, ww); + err =intel_context_pre_pin (ce, ww);if (err)return err;
@@ -232,7 +301,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,if (err) goto err_ctx_unpin; - err = ce->ops->pre_pin (ce, ww, &vaddr); + err = ce->ops->pre_pin (ce, ww);if (err) goto err_release;
@@ -250,7 +319,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,if (unlikely (err)) goto err_unlock; - err = ce->ops->pin (ce, vaddr); + err = ce->ops->pin (ce);if (err) {intel_context_active_release (ce); goto err_unlock;
@@ -290,7 +359,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,return err; } -int __intel_context_do_pin (struct intel_context *ce) +static int __intel_context_do_pin (struct intel_context *ce) {struct i915_gem_ww_ctx ww;int err;
@@ -337,7 +406,7 @@ static void __intel_context_retire(struct i915_active *active) intel_context_get_avg_runtime_ns(ce));set_bit (CONTEXT_VALID_BIT , &ce->flags); -intel_context_post_unpin (ce); +__intel_context_post_unpin (ce);intel_context_put (ce); }
@@ -562,6 +631,88 @@ void intel_context_bind_parent_child(struct intel_context *parent, child->parent = parent; } +static inline int ____intel_context_pin (struct intel_context *ce) +{ +if (likely (intel_context_pin_if_active (ce))) +return 0 ; + +return __intel_context_do_pin (ce); +} + +static inline int __intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +if (likely (intel_context_pin_if_active (ce))) +return 0 ; + +return __intel_context_do_pin_ww (ce, ww); +} + +static inline void __intel_context_unpin (struct intel_context *ce) +{ +if (!ce->ops->sched_disable) { +__intel_context_do_unpin (ce,1 ); + }else { +/* +* Move ownership of this pin to the scheduling disable which is +* an async operation. When that operation completes the above +* intel_context_sched_disable_unpin is called potentially +* unpinning the context. +*/ +while (!atomic_add_unless (&ce->pin_count,-1 ,1 )) { +if (atomic_cmpxchg (&ce->pin_count,1 ,2 ) ==1 ) { + ce->ops->sched_disable (ce); +break ; + } + } + } +} + +/* +* FIXME: This is ugly, these branches are only needed for parallel contexts in +* GuC submission. Basically the idea is if any of the contexts, that are +* configured for parallel submission, are pinned all the contexts need to be +* pinned in order to register these contexts with the GuC. We are adding the +* layer here while it should probably be pushed to the backend via a vfunc. But +* since we already have ce->pin + a layer atop it is confusing. Definitely +* needs a bit of rework how to properly layer / structure this code path. What +* is in place works but is not ideal. +*/ +int intel_context_pin (struct intel_context *ce) +{ +if (intel_context_is_child (ce)) { +if (!atomic_fetch_add (1 , &ce->pin_count)) +return ____intel_context_pin (ce->parent); +else +return 0 ; + }else { +return ____intel_context_pin (ce); + } +} + +int intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +if (intel_context_is_child (ce)) { +if (!atomic_fetch_add (1 , &ce->pin_count)) +return __intel_context_pin_ww (ce->parent, ww); +else +return 0 ; + }else { +return __intel_context_pin_ww (ce, ww); + } +} + +void intel_context_unpin (struct intel_context *ce) +{ +if (intel_context_is_child (ce)) { +if (atomic_fetch_add (-1 , &ce->pin_count) ==1 ) +__intel_context_unpin (ce->parent); + }else { +__intel_context_unpin (ce); + } +} +#if IS_ENABLED (CONFIG_DRM_I915_SELFTEST )#include "selftest_context.c" #endif
drivers/gpu/drm/i915/gt/intel_context.h +5 -38
64 lines not on this page · raw message
drivers/gpu/drm/i915/gt/intel_context_types.h +2 -2
@@ -39,8 +39,8 @@ struct intel_context_ops {void (*ban)(struct intel_context *ce,struct i915_request *rq); -int (*pre_pin)(struct intel_context *ce,struct i915_gem_ww_ctx *ww,void **vaddr); -int (*pin)(struct intel_context *ce,void *vaddr); +int (*pre_pin)(struct intel_context *ce,struct i915_gem_ww_ctx *ww); +int (*pin)(struct intel_context *ce);void (*unpin)(struct intel_context *ce);void (*post_unpin)(struct intel_context *ce);
drivers/gpu/drm/i915/gt/intel_execlists_submission.c +12 -13
@@ -2554,16 +2554,17 @@ static void execlists_submit_request(struct i915_request *request)static int __execlists_context_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww,void **vaddr) +struct i915_gem_ww_ctx *ww) {int err; - err =lrc_pre_pin (ce, engine, ww, vaddr); + err =lrc_pre_pin (ce, engine, ww);if (err)return err;if (!__test_and_set_bit (CONTEXT_INIT_BIT , &ce->flags)) { -lrc_init_state (ce, engine, *vaddr); +lrc_init_state (ce, engine, ce->lrc_reg_state - +LRC_STATE_OFFSET /sizeof (*ce->lrc_reg_state));__i915_gem_object_flush_map (ce->state->obj,0 , engine->context_size); }
@@ -2572,15 +2573,14 @@ __execlists_context_pre_pin(struct intel_context *ce, }static int execlists_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return __execlists_context_pre_pin (ce, ce->engine, ww, vaddr); +return __execlists_context_pre_pin (ce, ce->engine, ww); } -static int execlists_context_pin (struct intel_context *ce,void *vaddr) +static int execlists_context_pin (struct intel_context *ce) { -return lrc_pin (ce, ce->engine, vaddr); +return lrc_pin (ce, ce->engine); }static int execlists_context_alloc (struct intel_context *ce)
@@ -3570,20 +3570,19 @@ static int virtual_context_alloc(struct intel_context *ce) }static int virtual_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) {struct virtual_engine *ve =container_of (ce,typeof (*ve), context);/* Note: we must use a real engine class for setting up reg state */ -return __execlists_context_pre_pin (ce, ve->siblings[0 ], ww, vaddr); +return __execlists_context_pre_pin (ce, ve->siblings[0 ], ww); } -static int virtual_context_pin (struct intel_context *ce,void *vaddr) +static int virtual_context_pin (struct intel_context *ce) {struct virtual_engine *ve =container_of (ce,typeof (*ve), context); -return lrc_pin (ce, ve->siblings[0 ], vaddr); +return lrc_pin (ce, ve->siblings[0 ]); }static void virtual_context_enter (struct intel_context *ce)
drivers/gpu/drm/i915/gt/intel_lrc.c +13 -13
@@ -947,30 +947,30 @@ void lrc_reset(struct intel_context *ce)int lrc_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { +void *vaddr;GEM_BUG_ON (!ce->state);GEM_BUG_ON (!i915_vma_is_pinned (ce->state)); - *vaddr =i915_gem_object_pin_map (ce->state->obj, -i915_coherent_map_type (ce->engine->i915, - ce->state->obj, - false) | -I915_MAP_OVERRIDE ); + vaddr =i915_gem_object_pin_map (ce->state->obj, +i915_coherent_map_type (ce->engine->i915, + ce->state->obj, + false) | +I915_MAP_OVERRIDE ); -return PTR_ERR_OR_ZERO (*vaddr); + ce->lrc_reg_state = vaddr +LRC_STATE_OFFSET ; + +return PTR_ERR_OR_ZERO (vaddr); }int lrc_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr) +struct intel_engine_cs *engine) { - ce->lrc_reg_state = vaddr +LRC_STATE_OFFSET ; -if (!__test_and_set_bit (CONTEXT_INIT_BIT , &ce->flags)) -lrc_init_state (ce, engine, vaddr); +lrc_init_state (ce, engine, + (void *)ce->lrc_reg_state -LRC_STATE_OFFSET ); ce->lrc.lrca =lrc_update_regs (ce, engine, ce->ring->tail);return 0 ;
drivers/gpu/drm/i915/gt/intel_lrc.h +2 -4
@@ -38,12 +38,10 @@ void lrc_destroy(struct kref *kref);int lrc_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr); +struct i915_gem_ww_ctx *ww);int lrc_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr); +struct intel_engine_cs *engine);void lrc_unpin (struct intel_context *ce);void lrc_post_unpin (struct intel_context *ce);
drivers/gpu/drm/i915/gt/intel_ring_submission.c +2 -3
@@ -472,8 +472,7 @@ static int ring_context_init_default_state(struct intel_context *ce, }static int ring_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **unused) +struct i915_gem_ww_ctx *ww) {struct i915_address_space *vm;int err =0 ;
@@ -576,7 +575,7 @@ static int ring_context_alloc(struct intel_context *ce)return 0 ; } -static int ring_context_pin (struct intel_context *ce,void *unused) +static int ring_context_pin (struct intel_context *ce) {return 0 ; }
drivers/gpu/drm/i915/gt/mock_engine.c +2 -2
@@ -167,12 +167,12 @@ static int mock_context_alloc(struct intel_context *ce) }static int mock_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww,void **unused) +struct i915_gem_ww_ctx *ww) {return 0 ; } -static int mock_context_pin (struct intel_context *ce,void *unused) +static int mock_context_pin (struct intel_context *ce) {return 0 ; }
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +164 -19
244 lines not on this page · raw message
Signature · 1 line
2.28.0
Re: [Intel-gfx] [PATCH 16/46] drm/i915/guc: Implement GuC parent-child context pin / unpin functions
Quoted 214 lines · Matthew Brost
Implement GuC parent-child context pin / unpin functions in which in any contexts in the relationship are pinned all the contexts are pinned. The parent owns most of the pinning / unpinning process and the children direct any pins / unpins to the parent.
Patch implements a number of unused functions that will be connected later in the series.
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/gt/intel_context.c | 187 ++++++++++++++++-- drivers/gpu/drm/i915/gt/intel_context.h | 43 +--- drivers/gpu/drm/i915/gt/intel_context_types.h | 4 +- .../drm/i915/gt/intel_execlists_submission.c | 25 ++- drivers/gpu/drm/i915/gt/intel_lrc.c | 26 +-- drivers/gpu/drm/i915/gt/intel_lrc.h | 6 +- .../gpu/drm/i915/gt/intel_ring_submission.c | 5 +- drivers/gpu/drm/i915/gt/mock_engine.c | 4 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 183 +++++++++++++++-- 9 files changed, 371 insertions(+), 112 deletions(-)drivers/gpu/drm/i915/gt/intel_context.c +107
-18@@ -158,8 +158,8 @@ static void __ring_retire(struct intel_ring *ring)intel_ring_unpin (ring); } -static int intel_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) +static int __intel_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) {int err;@@ -190,7 +190,7 @@ static int intel_context_pre_pin(struct intel_context *ce,return err; } -static void intel_context_post_unpin (struct intel_context *ce) +static void __intel_context_post_unpin (struct intel_context *ce) {if (ce->state)__context_unpin_state (ce->state);@@ -199,13 +199,85 @@ static void intel_context_post_unpin(struct intel_context *ce)__ring_retire (ce->ring); } -int __intel_context_do_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) +static int intel_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) { -bool handoff = false; -void *vaddr; +struct intel_context *child; +int err, i =0 ; + +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) { + err =__intel_context_pre_pin (child, ww); +if (unlikely (err)) + goto unwind; + ++i; + } + + err =__intel_context_pre_pin (ce, ww); +if (unlikely (err)) + goto unwind; + +return 0 ; + +unwind: +for_each_child (ce, child) { +if (!i--) +break ; +__intel_context_post_unpin (ce); + } + +return err; +} + +static void intel_context_post_unpin (struct intel_context *ce) +{ +struct intel_context *child; + +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) +__intel_context_post_unpin (child); + +__intel_context_post_unpin (ce); +} + +static int __do_ww_lock (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +int err =i915_gem_object_lock (ce->timeline->hwsp_ggtt->obj, ww); + +if (!err && ce->ring->vma->obj) + err =i915_gem_object_lock (ce->ring->vma->obj, ww); +if (!err && ce->state) + err =i915_gem_object_lock (ce->state->obj, ww); + +return err; +} + +static int do_ww_lock (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +struct intel_context *child;int err =0 ; +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) { + err =__do_ww_lock (child, ww); +if (unlikely (err)) +return err; + } + +return __do_ww_lock (ce, ww); +} + +static int __intel_context_do_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +bool handoff = false; +int err; +if (unlikely (!test_bit (CONTEXT_ALLOC_BIT , &ce->flags))) { err =intel_context_alloc_state (ce);if (err)@@ -217,14 +289,11 @@ int __intel_context_do_pin_ww(struct intel_context *ce,* refcount for __intel_context_active(), which prevent a lock * inversion of ce->pin_mutex vs dma_resv_lock(). */ + err =do_ww_lock (ce, ww); +if (err) +return err; - err =i915_gem_object_lock (ce->timeline->hwsp_ggtt->obj, ww); -if (!err && ce->ring->vma->obj) - err =i915_gem_object_lock (ce->ring->vma->obj, ww); -if (!err && ce->state) - err =i915_gem_object_lock (ce->state->obj, ww); -if (!err) - err =intel_context_pre_pin (ce, ww); + err =intel_context_pre_pin (ce, ww);if (err)return err;@@ -232,7 +301,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,if (err) goto err_ctx_unpin; - err = ce->ops->pre_pin (ce, ww, &vaddr); + err = ce->ops->pre_pin (ce, ww);if (err) goto err_release;@@ -250,7 +319,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,if (unlikely (err)) goto err_unlock; - err = ce->ops->pin (ce, vaddr); + err = ce->ops->pin (ce);if (err) {intel_context_active_release (ce); goto err_unlock;@@ -290,7 +359,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,return err; } -int __intel_context_do_pin (struct intel_context *ce) +static int __intel_context_do_pin (struct intel_context *ce) {struct i915_gem_ww_ctx ww;int err;@@ -337,7 +406,7 @@ static void __intel_context_retire(struct i915_active *active) intel_context_get_avg_runtime_ns(ce));set_bit (CONTEXT_VALID_BIT , &ce->flags); -intel_context_post_unpin (ce); +__intel_context_post_unpin (ce);intel_context_put (ce); }@@ -562,6 +631,88 @@ void intel_context_bind_parent_child(struct intel_context *parent, child->parent = parent; } +static inline int ____intel_context_pin (struct intel_context *ce) +{ +if (likely (intel_context_pin_if_active (ce))) +return 0 ; + +return __intel_context_do_pin (ce); +} + +static inline int __intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +if (likely (intel_context_pin_if_active (ce))) +return 0 ; + +return __intel_context_do_pin_ww (ce, ww); +} + +static inline void __intel_context_unpin (struct intel_context *ce) +{ +if (!ce->ops->sched_disable) {
drivers/gpu/drm/i915/gt/intel_context.c void intel_context_bind_parent_child(struct intel_context *parent, · line 663
Quoted 10 lines
drivers/gpu/drm/i915/gt/intel_context.c +10
-0+__intel_context_do_unpin (ce,1 ); + }else { +/* +* Move ownership of this pin to the scheduling disable which is +* an async operation. When that operation completes the above +* intel_context_sched_disable_unpin is called potentially +* unpinning the context. +*/ +while (!atomic_add_unless (&ce->pin_count,-1 ,1 )) { +if (atomic_cmpxchg (&ce->pin_count,1 ,2 ) ==1 ) {
Uh man lockless algorithms.
Unless this comes:
- with essentially an academic looking paper that describes the abstract model of the lockless algorithm and proves it against the linux kernel meory model.
- lockless stuff generally needs barriers, and those barriers must be all documented. This means a) a comment next to each barrier in the code b) pointing to its counterparty c) with the overall design also explained in the kerneldoc for those datastructres.
If you don't know where your barriers are, see above point about "it should look more like an academic paper in the commit message"
- hard perf data about how this is absolutely required, based on a real-world use-case (which then sometimes justifies a microbenchmark metric for the details, but it always needs to be real-world based). And also a throughrough explainer how the perf issue isn't fixable through better design. If that's not doable, just protect the state machine with a big dumb lock and move on.
- Also, because the current code is in such bad shape wrt lockless algorithms and premature optimizations: Overall complexity should go down (it's way too high right now), so pay down your new lockless trick by removing one of the existing ones that we only have because we can.
Yes this is steep, but we're way out in the woods here and need to smoehow get back.
-Daniel
Quoted 577 lines
drivers/gpu/drm/i915/gt/intel_context.c +52
-0+ ce->ops->sched_disable (ce); +break ; + } + } + } +} + +/* +* FIXME: This is ugly, these branches are only needed for parallel contexts in +* GuC submission. Basically the idea is if any of the contexts, that are +* configured for parallel submission, are pinned all the contexts need to be +* pinned in order to register these contexts with the GuC. We are adding the +* layer here while it should probably be pushed to the backend via a vfunc. But +* since we already have ce->pin + a layer atop it is confusing. Definitely +* needs a bit of rework how to properly layer / structure this code path. What +* is in place works but is not ideal. +*/ +int intel_context_pin (struct intel_context *ce) +{ +if (intel_context_is_child (ce)) { +if (!atomic_fetch_add (1 , &ce->pin_count)) +return ____intel_context_pin (ce->parent); +else +return 0 ; + }else { +return ____intel_context_pin (ce); + } +} + +int intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +if (intel_context_is_child (ce)) { +if (!atomic_fetch_add (1 , &ce->pin_count)) +return __intel_context_pin_ww (ce->parent, ww); +else +return 0 ; + }else { +return __intel_context_pin_ww (ce, ww); + } +} + +void intel_context_unpin (struct intel_context *ce) +{ +if (intel_context_is_child (ce)) { +if (atomic_fetch_add (-1 , &ce->pin_count) ==1 ) +__intel_context_unpin (ce->parent); + }else { +__intel_context_unpin (ce); + } +} +#if IS_ENABLED (CONFIG_DRM_I915_SELFTEST )#include "selftest_context.c" #endif drivers/gpu/drm/i915/gt/intel_context.h +5
-38@@ -110,31 +110,15 @@ static inline void intel_context_unlock_pinned(struct intel_context *ce)mutex_unlock (&ce->pin_mutex); } -int __intel_context_do_pin (struct intel_context *ce); -int __intel_context_do_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww); -static inline bool intel_context_pin_if_active (struct intel_context *ce) {return atomic_inc_not_zero (&ce->pin_count); } -static inline int intel_context_pin (struct intel_context *ce) -{ -if (likely (intel_context_pin_if_active (ce))) -return 0 ; - -return __intel_context_do_pin (ce); -} - -static inline int intel_context_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) -{ -if (likely (intel_context_pin_if_active (ce))) -return 0 ; +int intel_context_pin (struct intel_context *ce); -return __intel_context_do_pin_ww (ce, ww); -} +int intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww);static inline void __intel_context_pin (struct intel_context *ce) {@@ -146,28 +130,11 @@ void __intel_context_do_unpin(struct intel_context *ce, int sub);static inline void intel_context_sched_disable_unpin (struct intel_context *ce) { +GEM_BUG_ON (intel_context_is_child (ce));__intel_context_do_unpin (ce,2 ); } -static inline void intel_context_unpin (struct intel_context *ce) -{ -if (!ce->ops->sched_disable) { -__intel_context_do_unpin (ce,1 ); - }else { -/* -* Move ownership of this pin to the scheduling disable which is -* an async operation. When that operation completes the above -* intel_context_sched_disable_unpin is called potentially -* unpinning the context. -*/ -while (!atomic_add_unless (&ce->pin_count,-1 ,1 )) { -if (atomic_cmpxchg (&ce->pin_count,1 ,2 ) ==1 ) { - ce->ops->sched_disable (ce); -break ; - } - } - } -} +void intel_context_unpin (struct intel_context *ce);void intel_context_enter_engine (struct intel_context *ce);void intel_context_exit_engine (struct intel_context *ce);drivers/gpu/drm/i915/gt/intel_context_types.h +2
-2@@ -39,8 +39,8 @@ struct intel_context_ops {void (*ban)(struct intel_context *ce,struct i915_request *rq); -int (*pre_pin)(struct intel_context *ce,struct i915_gem_ww_ctx *ww,void **vaddr); -int (*pin)(struct intel_context *ce,void *vaddr); +int (*pre_pin)(struct intel_context *ce,struct i915_gem_ww_ctx *ww); +int (*pin)(struct intel_context *ce);void (*unpin)(struct intel_context *ce);void (*post_unpin)(struct intel_context *ce);drivers/gpu/drm/i915/gt/intel_execlists_submission.c +12
-13@@ -2554,16 +2554,17 @@ static void execlists_submit_request(struct i915_request *request)static int __execlists_context_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww,void **vaddr) +struct i915_gem_ww_ctx *ww) {int err; - err =lrc_pre_pin (ce, engine, ww, vaddr); + err =lrc_pre_pin (ce, engine, ww);if (err)return err;if (!__test_and_set_bit (CONTEXT_INIT_BIT , &ce->flags)) { -lrc_init_state (ce, engine, *vaddr); +lrc_init_state (ce, engine, ce->lrc_reg_state - +LRC_STATE_OFFSET /sizeof (*ce->lrc_reg_state));__i915_gem_object_flush_map (ce->state->obj,0 , engine->context_size); }@@ -2572,15 +2573,14 @@ __execlists_context_pre_pin(struct intel_context *ce, }static int execlists_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return __execlists_context_pre_pin (ce, ce->engine, ww, vaddr); +return __execlists_context_pre_pin (ce, ce->engine, ww); } -static int execlists_context_pin (struct intel_context *ce,void *vaddr) +static int execlists_context_pin (struct intel_context *ce) { -return lrc_pin (ce, ce->engine, vaddr); +return lrc_pin (ce, ce->engine); }static int execlists_context_alloc (struct intel_context *ce)@@ -3570,20 +3570,19 @@ static int virtual_context_alloc(struct intel_context *ce) }static int virtual_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) {struct virtual_engine *ve =container_of (ce,typeof (*ve), context);/* Note: we must use a real engine class for setting up reg state */ -return __execlists_context_pre_pin (ce, ve->siblings[0 ], ww, vaddr); +return __execlists_context_pre_pin (ce, ve->siblings[0 ], ww); } -static int virtual_context_pin (struct intel_context *ce,void *vaddr) +static int virtual_context_pin (struct intel_context *ce) {struct virtual_engine *ve =container_of (ce,typeof (*ve), context); -return lrc_pin (ce, ve->siblings[0 ], vaddr); +return lrc_pin (ce, ve->siblings[0 ]); }static void virtual_context_enter (struct intel_context *ce)drivers/gpu/drm/i915/gt/intel_lrc.c +13
-13@@ -947,30 +947,30 @@ void lrc_reset(struct intel_context *ce)int lrc_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { +void *vaddr;GEM_BUG_ON (!ce->state);GEM_BUG_ON (!i915_vma_is_pinned (ce->state)); - *vaddr =i915_gem_object_pin_map (ce->state->obj, -i915_coherent_map_type (ce->engine->i915, - ce->state->obj, - false) | -I915_MAP_OVERRIDE ); + vaddr =i915_gem_object_pin_map (ce->state->obj, +i915_coherent_map_type (ce->engine->i915, + ce->state->obj, + false) | +I915_MAP_OVERRIDE ); -return PTR_ERR_OR_ZERO (*vaddr); + ce->lrc_reg_state = vaddr +LRC_STATE_OFFSET ; + +return PTR_ERR_OR_ZERO (vaddr); }int lrc_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr) +struct intel_engine_cs *engine) { - ce->lrc_reg_state = vaddr +LRC_STATE_OFFSET ; -if (!__test_and_set_bit (CONTEXT_INIT_BIT , &ce->flags)) -lrc_init_state (ce, engine, vaddr); +lrc_init_state (ce, engine, + (void *)ce->lrc_reg_state -LRC_STATE_OFFSET ); ce->lrc.lrca =lrc_update_regs (ce, engine, ce->ring->tail);return 0 ;drivers/gpu/drm/i915/gt/intel_lrc.h +2
-4@@ -38,12 +38,10 @@ void lrc_destroy(struct kref *kref);int lrc_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr); +struct i915_gem_ww_ctx *ww);int lrc_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr); +struct intel_engine_cs *engine);void lrc_unpin (struct intel_context *ce);void lrc_post_unpin (struct intel_context *ce);drivers/gpu/drm/i915/gt/intel_ring_submission.c +2
-3@@ -472,8 +472,7 @@ static int ring_context_init_default_state(struct intel_context *ce, }static int ring_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **unused) +struct i915_gem_ww_ctx *ww) {struct i915_address_space *vm;int err =0 ;@@ -576,7 +575,7 @@ static int ring_context_alloc(struct intel_context *ce)return 0 ; } -static int ring_context_pin (struct intel_context *ce,void *unused) +static int ring_context_pin (struct intel_context *ce) {return 0 ; }drivers/gpu/drm/i915/gt/mock_engine.c +2
-2@@ -167,12 +167,12 @@ static int mock_context_alloc(struct intel_context *ce) }static int mock_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww,void **unused) +struct i915_gem_ww_ctx *ww) {return 0 ; } -static int mock_context_pin (struct intel_context *ce,void *unused) +static int mock_context_pin (struct intel_context *ce) {return 0 ; }drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +164
-19@@ -1905,6 +1905,7 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)GEM_BUG_ON (!engine->mask);GEM_BUG_ON (context_guc_id_invalid (ce)); +GEM_BUG_ON (intel_context_is_child (ce));/* * Ensure LRC + CT vmas are is same region as write barrier is done @@ -2008,15 +2009,13 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)static int __guc_context_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return lrc_pre_pin (ce, engine, ww, vaddr); +return lrc_pre_pin (ce, engine, ww); }static int __guc_context_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr) +struct intel_engine_cs *engine) {if (i915_ggtt_offset (ce->state) != (ce->lrc.lrca &CTX_GTT_ADDRESS_MASK ))@@ -2027,20 +2026,33 @@ static int __guc_context_pin(struct intel_context *ce,* explaination of why. */ -return lrc_pin (ce, engine, vaddr); +return lrc_pin (ce, engine); +} + +static void __guc_context_unpin (struct intel_context *ce) +{ +lrc_unpin (ce); +} + +static void __guc_context_post_unpin (struct intel_context *ce) +{ +lrc_post_unpin (ce); }static int guc_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return __guc_context_pre_pin (ce, ce->engine, ww, vaddr); +return __guc_context_pre_pin (ce, ce->engine, ww); } -static int guc_context_pin (struct intel_context *ce,void *vaddr) +static int guc_context_pin (struct intel_context *ce) { -int ret =__guc_context_pin (ce, ce->engine, vaddr); +int ret; +GEM_BUG_ON (intel_context_is_parent (ce) || +intel_context_is_child (ce)); + + ret =__guc_context_pin (ce, ce->engine);if (likely (!ret && !intel_context_is_barrier (ce)))intel_engine_pm_get (ce->engine);@@ -2054,7 +2066,7 @@ static void guc_context_unpin(struct intel_context *ce)GEM_BUG_ON (context_enabled (ce));unpin_guc_id (guc, ce, true); -lrc_unpin (ce); +__guc_context_unpin (ce);if (likely (!intel_context_is_barrier (ce)))intel_engine_pm_put (ce->engine);@@ -2062,7 +2074,141 @@ static void guc_context_unpin(struct intel_context *ce)static void guc_context_post_unpin (struct intel_context *ce) { -lrc_post_unpin (ce); +__guc_context_post_unpin (ce); +} + +/* Future patches will use this function */ +__maybe_unused +static int guc_parent_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +struct intel_context *child; +int err, i =0 , j =0 ; + +for_each_child (ce, child) { + err =i915_active_acquire (&child->active); +if (unlikely (err)) + goto unwind_active; + ++i; + } + +for_each_child (ce, child) { + err =__guc_context_pre_pin (child, child->engine, ww); +if (unlikely (err)) + goto unwind_pre_pin; + ++j; + } + + err =__guc_context_pre_pin (ce, ce->engine, ww); +if (unlikely (err)) + goto unwind_pre_pin; + +return 0 ; + +unwind_pre_pin: +for_each_child (ce, child) { +if (!j--) +break ; +__guc_context_post_unpin (child); + } + +unwind_active: +for_each_child (ce, child) { +if (!i--) +break ; +i915_active_release (&child->active); + } + +return err; +} + +/* Future patches will use this function */ +__maybe_unused +static void guc_parent_context_post_unpin (struct intel_context *ce) +{ +struct intel_context *child; + +for_each_child (ce, child) +__guc_context_post_unpin (child); +__guc_context_post_unpin (ce); + +for_each_child (ce, child) { +intel_context_get (child); +i915_active_release (&child->active); +intel_context_put (child); + } +} + +/* Future patches will use this function */ +__maybe_unused +static int guc_parent_context_pin (struct intel_context *ce) +{ +int ret, i =0 , j =0 ; +struct intel_context *child; +struct intel_engine_cs *engine; +intel_engine_mask_t tmp; + +GEM_BUG_ON (!intel_context_is_parent (ce)); + +for_each_child (ce, child) { + ret =__guc_context_pin (child, child->engine); +if (unlikely (ret)) + goto unwind_pin; + ++i; + } + ret =__guc_context_pin (ce, ce->engine); +if (unlikely (ret)) + goto unwind_pin; + +for_each_child (ce, child) +if (test_bit (CONTEXT_LRCA_DIRTY , &child->flags)) { +set_bit (CONTEXT_LRCA_DIRTY , &ce->flags); +break ; + } + +for_each_engine_masked (engine, ce->engine->gt, + ce->engine->mask, tmp) +intel_engine_pm_get (engine); +for_each_child (ce, child) +for_each_engine_masked (engine, child->engine->gt, + child->engine->mask, tmp) +intel_engine_pm_get (engine); + +return 0 ; + +unwind_pin: +for_each_child (ce, child) { +if (++j > i) +break ; +__guc_context_unpin (child); + } + +return ret; +} + +/* Future patches will use this function */ +__maybe_unused +static void guc_parent_context_unpin (struct intel_context *ce) +{ +struct intel_context *child; +struct intel_engine_cs *engine; +intel_engine_mask_t tmp; + +GEM_BUG_ON (!intel_context_is_parent (ce)); +GEM_BUG_ON (context_enabled (ce)); + +unpin_guc_id (ce_to_guc (ce), ce, true); +for_each_child (ce, child) +__guc_context_unpin (child); +__guc_context_unpin (ce); + +for_each_engine_masked (engine, ce->engine->gt, + ce->engine->mask, tmp) +intel_engine_pm_put (engine); +for_each_child (ce, child) +for_each_engine_masked (engine, child->engine->gt, + child->engine->mask, tmp) +intel_engine_pm_put (engine); } static void __guc_context_sched_enable(struct intel_guc *guc,@@ -2993,18 +3139,17 @@ static int guc_request_alloc(struct i915_request *rq) }static int guc_virtual_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) {struct intel_engine_cs *engine =guc_virtual_get_sibling (ce->engine,0 ); -return __guc_context_pre_pin (ce, engine, ww, vaddr); +return __guc_context_pre_pin (ce, engine, ww); } -static int guc_virtual_context_pin (struct intel_context *ce,void *vaddr) +static int guc_virtual_context_pin (struct intel_context *ce) {struct intel_engine_cs *engine =guc_virtual_get_sibling (ce->engine,0 ); -int ret =__guc_context_pin (ce, engine, vaddr); +int ret =__guc_context_pin (ce, engine);intel_engine_mask_t tmp, mask = ce->engine->mask;if (likely (!ret))@@ -3024,7 +3169,7 @@ static void guc_virtual_context_unpin(struct intel_context *ce)GEM_BUG_ON (intel_context_is_barrier (ce));unpin_guc_id (guc, ce, true); -lrc_unpin (ce); +__guc_context_unpin (ce);for_each_engine_masked (engine, ce->engine->gt, mask, tmp)intel_engine_pm_put (engine);Signature · 1 line
2.28.0
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [Intel-gfx] [PATCH 16/46] drm/i915/guc: Implement GuC parent-child context pin / unpin functions
Quoted 248 lines · Daniel Vetter
Quoted 214 lines · Matthew Brost
Implement GuC parent-child context pin / unpin functions in which in any contexts in the relationship are pinned all the contexts are pinned. The parent owns most of the pinning / unpinning process and the children direct any pins / unpins to the parent.
Patch implements a number of unused functions that will be connected later in the series.
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/gt/intel_context.c | 187 ++++++++++++++++-- drivers/gpu/drm/i915/gt/intel_context.h | 43 +--- drivers/gpu/drm/i915/gt/intel_context_types.h | 4 +- .../drm/i915/gt/intel_execlists_submission.c | 25 ++- drivers/gpu/drm/i915/gt/intel_lrc.c | 26 +-- drivers/gpu/drm/i915/gt/intel_lrc.h | 6 +- .../gpu/drm/i915/gt/intel_ring_submission.c | 5 +- drivers/gpu/drm/i915/gt/mock_engine.c | 4 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 183 +++++++++++++++-- 9 files changed, 371 insertions(+), 112 deletions(-)drivers/gpu/drm/i915/gt/intel_context.c +107
-18@@ -158,8 +158,8 @@ static void __ring_retire(struct intel_ring *ring)intel_ring_unpin (ring); } -static int intel_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) +static int __intel_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) {int err;@@ -190,7 +190,7 @@ static int intel_context_pre_pin(struct intel_context *ce,return err; } -static void intel_context_post_unpin (struct intel_context *ce) +static void __intel_context_post_unpin (struct intel_context *ce) {if (ce->state)__context_unpin_state (ce->state);@@ -199,13 +199,85 @@ static void intel_context_post_unpin(struct intel_context *ce)__ring_retire (ce->ring); } -int __intel_context_do_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) +static int intel_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) { -bool handoff = false; -void *vaddr; +struct intel_context *child; +int err, i =0 ; + +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) { + err =__intel_context_pre_pin (child, ww); +if (unlikely (err)) + goto unwind; + ++i; + } + + err =__intel_context_pre_pin (ce, ww); +if (unlikely (err)) + goto unwind; + +return 0 ; + +unwind: +for_each_child (ce, child) { +if (!i--) +break ; +__intel_context_post_unpin (ce); + } + +return err; +} + +static void intel_context_post_unpin (struct intel_context *ce) +{ +struct intel_context *child; + +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) +__intel_context_post_unpin (child); + +__intel_context_post_unpin (ce); +} + +static int __do_ww_lock (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +int err =i915_gem_object_lock (ce->timeline->hwsp_ggtt->obj, ww); + +if (!err && ce->ring->vma->obj) + err =i915_gem_object_lock (ce->ring->vma->obj, ww); +if (!err && ce->state) + err =i915_gem_object_lock (ce->state->obj, ww); + +return err; +} + +static int do_ww_lock (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +struct intel_context *child;int err =0 ; +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) { + err =__do_ww_lock (child, ww); +if (unlikely (err)) +return err; + } + +return __do_ww_lock (ce, ww); +} + +static int __intel_context_do_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +bool handoff = false; +int err; +if (unlikely (!test_bit (CONTEXT_ALLOC_BIT , &ce->flags))) { err =intel_context_alloc_state (ce);if (err)@@ -217,14 +289,11 @@ int __intel_context_do_pin_ww(struct intel_context *ce,* refcount for __intel_context_active(), which prevent a lock * inversion of ce->pin_mutex vs dma_resv_lock(). */ + err =do_ww_lock (ce, ww); +if (err) +return err; - err =i915_gem_object_lock (ce->timeline->hwsp_ggtt->obj, ww); -if (!err && ce->ring->vma->obj) - err =i915_gem_object_lock (ce->ring->vma->obj, ww); -if (!err && ce->state) - err =i915_gem_object_lock (ce->state->obj, ww); -if (!err) - err =intel_context_pre_pin (ce, ww); + err =intel_context_pre_pin (ce, ww);if (err)return err;@@ -232,7 +301,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,if (err) goto err_ctx_unpin; - err = ce->ops->pre_pin (ce, ww, &vaddr); + err = ce->ops->pre_pin (ce, ww);if (err) goto err_release;@@ -250,7 +319,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,if (unlikely (err)) goto err_unlock; - err = ce->ops->pin (ce, vaddr); + err = ce->ops->pin (ce);if (err) {intel_context_active_release (ce); goto err_unlock;@@ -290,7 +359,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,return err; } -int __intel_context_do_pin (struct intel_context *ce) +static int __intel_context_do_pin (struct intel_context *ce) {struct i915_gem_ww_ctx ww;int err;@@ -337,7 +406,7 @@ static void __intel_context_retire(struct i915_active *active) intel_context_get_avg_runtime_ns(ce));set_bit (CONTEXT_VALID_BIT , &ce->flags); -intel_context_post_unpin (ce); +__intel_context_post_unpin (ce);intel_context_put (ce); }@@ -562,6 +631,88 @@ void intel_context_bind_parent_child(struct intel_context *parent, child->parent = parent; } +static inline int ____intel_context_pin (struct intel_context *ce) +{ +if (likely (intel_context_pin_if_active (ce))) +return 0 ; + +return __intel_context_do_pin (ce); +} + +static inline int __intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +if (likely (intel_context_pin_if_active (ce))) +return 0 ; + +return __intel_context_do_pin_ww (ce, ww); +} + +static inline void __intel_context_unpin (struct intel_context *ce) +{ +if (!ce->ops->sched_disable) {drivers/gpu/drm/i915/gt/intel_context.c void intel_context_bind_parent_child(struct intel_context *parent, · line 663
Quoted 10 lines
drivers/gpu/drm/i915/gt/intel_context.c +10
-0+__intel_context_do_unpin (ce,1 ); + }else { +/* +* Move ownership of this pin to the scheduling disable which is +* an async operation. When that operation completes the above +* intel_context_sched_disable_unpin is called potentially +* unpinning the context. +*/ +while (!atomic_add_unless (&ce->pin_count,-1 ,1 )) { +if (atomic_cmpxchg (&ce->pin_count,1 ,2 ) ==1 ) {Uh man lockless algorithms.
Unless this comes:
- with essentially an academic looking paper that describes the abstract model of the lockless algorithm and proves it against the linux kernel meory model.- lockless stuff generally needs barriers, and those barriers must be all documented. This means a) a comment next to each barrier in the code b) pointing to its counterparty c) with the overall design also explained in the kerneldoc for those datastructres.
If you don't know where your barriers are, see above point about "it should look more like an academic paper in the commit message"
- hard perf data about how this is absolutely required, based on a real-world use-case (which then sometimes justifies a microbenchmark metric for the details, but it always needs to be real-world based). And also a throughrough explainer how the perf issue isn't fixable through better design. If that's not doable, just protect the state machine with a big dumb lock and move on.
Quoted 7 lines
- Also, because the current code is in such bad shape wrt lockless algorithms and premature optimizations: Overall complexity should go down (it's way too high right now), so pay down your new lockless trick by removing one of the existing ones that we only have because we can.
Yes this is steep, but we're way out in the woods here and need to smoehow get back.
See below FIXME. At one point all of this was hidden in the backend but the dma-resv patches that landed upstream completely broke the layering, hence the need for the code here.
I guess I don't really understand what mean when you say lockless alg needs barriers, if the atomic functions are not really atomic wouldn't the world be broken?
Also here I don't think it is really as simple as grab big dump lock for a variety of reasons, at least with the current dynamic pin / unpin code in place. If we move a perma-pinned contexts this could be cleaned up then.
Matt
Quoted 584 lines
-Daniel
Quoted 577 lines
drivers/gpu/drm/i915/gt/intel_context.c +52
-0+ ce->ops->sched_disable (ce); +break ; + } + } + } +} + +/* +* FIXME: This is ugly, these branches are only needed for parallel contexts in +* GuC submission. Basically the idea is if any of the contexts, that are +* configured for parallel submission, are pinned all the contexts need to be +* pinned in order to register these contexts with the GuC. We are adding the +* layer here while it should probably be pushed to the backend via a vfunc. But +* since we already have ce->pin + a layer atop it is confusing. Definitely +* needs a bit of rework how to properly layer / structure this code path. What +* is in place works but is not ideal. +*/ +int intel_context_pin (struct intel_context *ce) +{ +if (intel_context_is_child (ce)) { +if (!atomic_fetch_add (1 , &ce->pin_count)) +return ____intel_context_pin (ce->parent); +else +return 0 ; + }else { +return ____intel_context_pin (ce); + } +} + +int intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +if (intel_context_is_child (ce)) { +if (!atomic_fetch_add (1 , &ce->pin_count)) +return __intel_context_pin_ww (ce->parent, ww); +else +return 0 ; + }else { +return __intel_context_pin_ww (ce, ww); + } +} + +void intel_context_unpin (struct intel_context *ce) +{ +if (intel_context_is_child (ce)) { +if (atomic_fetch_add (-1 , &ce->pin_count) ==1 ) +__intel_context_unpin (ce->parent); + }else { +__intel_context_unpin (ce); + } +} +#if IS_ENABLED (CONFIG_DRM_I915_SELFTEST )#include "selftest_context.c" #endif drivers/gpu/drm/i915/gt/intel_context.h +5
-38@@ -110,31 +110,15 @@ static inline void intel_context_unlock_pinned(struct intel_context *ce)mutex_unlock (&ce->pin_mutex); } -int __intel_context_do_pin (struct intel_context *ce); -int __intel_context_do_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww); -static inline bool intel_context_pin_if_active (struct intel_context *ce) {return atomic_inc_not_zero (&ce->pin_count); } -static inline int intel_context_pin (struct intel_context *ce) -{ -if (likely (intel_context_pin_if_active (ce))) -return 0 ; - -return __intel_context_do_pin (ce); -} - -static inline int intel_context_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) -{ -if (likely (intel_context_pin_if_active (ce))) -return 0 ; +int intel_context_pin (struct intel_context *ce); -return __intel_context_do_pin_ww (ce, ww); -} +int intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww);static inline void __intel_context_pin (struct intel_context *ce) {@@ -146,28 +130,11 @@ void __intel_context_do_unpin(struct intel_context *ce, int sub);static inline void intel_context_sched_disable_unpin (struct intel_context *ce) { +GEM_BUG_ON (intel_context_is_child (ce));__intel_context_do_unpin (ce,2 ); } -static inline void intel_context_unpin (struct intel_context *ce) -{ -if (!ce->ops->sched_disable) { -__intel_context_do_unpin (ce,1 ); - }else { -/* -* Move ownership of this pin to the scheduling disable which is -* an async operation. When that operation completes the above -* intel_context_sched_disable_unpin is called potentially -* unpinning the context. -*/ -while (!atomic_add_unless (&ce->pin_count,-1 ,1 )) { -if (atomic_cmpxchg (&ce->pin_count,1 ,2 ) ==1 ) { - ce->ops->sched_disable (ce); -break ; - } - } - } -} +void intel_context_unpin (struct intel_context *ce);void intel_context_enter_engine (struct intel_context *ce);void intel_context_exit_engine (struct intel_context *ce);drivers/gpu/drm/i915/gt/intel_context_types.h +2
-2@@ -39,8 +39,8 @@ struct intel_context_ops {void (*ban)(struct intel_context *ce,struct i915_request *rq); -int (*pre_pin)(struct intel_context *ce,struct i915_gem_ww_ctx *ww,void **vaddr); -int (*pin)(struct intel_context *ce,void *vaddr); +int (*pre_pin)(struct intel_context *ce,struct i915_gem_ww_ctx *ww); +int (*pin)(struct intel_context *ce);void (*unpin)(struct intel_context *ce);void (*post_unpin)(struct intel_context *ce);drivers/gpu/drm/i915/gt/intel_execlists_submission.c +12
-13@@ -2554,16 +2554,17 @@ static void execlists_submit_request(struct i915_request *request)static int __execlists_context_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww,void **vaddr) +struct i915_gem_ww_ctx *ww) {int err; - err =lrc_pre_pin (ce, engine, ww, vaddr); + err =lrc_pre_pin (ce, engine, ww);if (err)return err;if (!__test_and_set_bit (CONTEXT_INIT_BIT , &ce->flags)) { -lrc_init_state (ce, engine, *vaddr); +lrc_init_state (ce, engine, ce->lrc_reg_state - +LRC_STATE_OFFSET /sizeof (*ce->lrc_reg_state));__i915_gem_object_flush_map (ce->state->obj,0 , engine->context_size); }@@ -2572,15 +2573,14 @@ __execlists_context_pre_pin(struct intel_context *ce, }static int execlists_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return __execlists_context_pre_pin (ce, ce->engine, ww, vaddr); +return __execlists_context_pre_pin (ce, ce->engine, ww); } -static int execlists_context_pin (struct intel_context *ce,void *vaddr) +static int execlists_context_pin (struct intel_context *ce) { -return lrc_pin (ce, ce->engine, vaddr); +return lrc_pin (ce, ce->engine); }static int execlists_context_alloc (struct intel_context *ce)@@ -3570,20 +3570,19 @@ static int virtual_context_alloc(struct intel_context *ce) }static int virtual_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) {struct virtual_engine *ve =container_of (ce,typeof (*ve), context);/* Note: we must use a real engine class for setting up reg state */ -return __execlists_context_pre_pin (ce, ve->siblings[0 ], ww, vaddr); +return __execlists_context_pre_pin (ce, ve->siblings[0 ], ww); } -static int virtual_context_pin (struct intel_context *ce,void *vaddr) +static int virtual_context_pin (struct intel_context *ce) {struct virtual_engine *ve =container_of (ce,typeof (*ve), context); -return lrc_pin (ce, ve->siblings[0 ], vaddr); +return lrc_pin (ce, ve->siblings[0 ]); }static void virtual_context_enter (struct intel_context *ce)drivers/gpu/drm/i915/gt/intel_lrc.c +13
-13@@ -947,30 +947,30 @@ void lrc_reset(struct intel_context *ce)int lrc_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { +void *vaddr;GEM_BUG_ON (!ce->state);GEM_BUG_ON (!i915_vma_is_pinned (ce->state)); - *vaddr =i915_gem_object_pin_map (ce->state->obj, -i915_coherent_map_type (ce->engine->i915, - ce->state->obj, - false) | -I915_MAP_OVERRIDE ); + vaddr =i915_gem_object_pin_map (ce->state->obj, +i915_coherent_map_type (ce->engine->i915, + ce->state->obj, + false) | +I915_MAP_OVERRIDE ); -return PTR_ERR_OR_ZERO (*vaddr); + ce->lrc_reg_state = vaddr +LRC_STATE_OFFSET ; + +return PTR_ERR_OR_ZERO (vaddr); }int lrc_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr) +struct intel_engine_cs *engine) { - ce->lrc_reg_state = vaddr +LRC_STATE_OFFSET ; -if (!__test_and_set_bit (CONTEXT_INIT_BIT , &ce->flags)) -lrc_init_state (ce, engine, vaddr); +lrc_init_state (ce, engine, + (void *)ce->lrc_reg_state -LRC_STATE_OFFSET ); ce->lrc.lrca =lrc_update_regs (ce, engine, ce->ring->tail);return 0 ;drivers/gpu/drm/i915/gt/intel_lrc.h +2
-4@@ -38,12 +38,10 @@ void lrc_destroy(struct kref *kref);int lrc_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr); +struct i915_gem_ww_ctx *ww);int lrc_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr); +struct intel_engine_cs *engine);void lrc_unpin (struct intel_context *ce);void lrc_post_unpin (struct intel_context *ce);drivers/gpu/drm/i915/gt/intel_ring_submission.c +2
-3@@ -472,8 +472,7 @@ static int ring_context_init_default_state(struct intel_context *ce, }static int ring_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **unused) +struct i915_gem_ww_ctx *ww) {struct i915_address_space *vm;int err =0 ;@@ -576,7 +575,7 @@ static int ring_context_alloc(struct intel_context *ce)return 0 ; } -static int ring_context_pin (struct intel_context *ce,void *unused) +static int ring_context_pin (struct intel_context *ce) {return 0 ; }drivers/gpu/drm/i915/gt/mock_engine.c +2
-2@@ -167,12 +167,12 @@ static int mock_context_alloc(struct intel_context *ce) }static int mock_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww,void **unused) +struct i915_gem_ww_ctx *ww) {return 0 ; } -static int mock_context_pin (struct intel_context *ce,void *unused) +static int mock_context_pin (struct intel_context *ce) {return 0 ; }drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +164
-19@@ -1905,6 +1905,7 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)GEM_BUG_ON (!engine->mask);GEM_BUG_ON (context_guc_id_invalid (ce)); +GEM_BUG_ON (intel_context_is_child (ce));/* * Ensure LRC + CT vmas are is same region as write barrier is done @@ -2008,15 +2009,13 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)static int __guc_context_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return lrc_pre_pin (ce, engine, ww, vaddr); +return lrc_pre_pin (ce, engine, ww); }static int __guc_context_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr) +struct intel_engine_cs *engine) {if (i915_ggtt_offset (ce->state) != (ce->lrc.lrca &CTX_GTT_ADDRESS_MASK ))@@ -2027,20 +2026,33 @@ static int __guc_context_pin(struct intel_context *ce,* explaination of why. */ -return lrc_pin (ce, engine, vaddr); +return lrc_pin (ce, engine); +} + +static void __guc_context_unpin (struct intel_context *ce) +{ +lrc_unpin (ce); +} + +static void __guc_context_post_unpin (struct intel_context *ce) +{ +lrc_post_unpin (ce); }static int guc_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return __guc_context_pre_pin (ce, ce->engine, ww, vaddr); +return __guc_context_pre_pin (ce, ce->engine, ww); } -static int guc_context_pin (struct intel_context *ce,void *vaddr) +static int guc_context_pin (struct intel_context *ce) { -int ret =__guc_context_pin (ce, ce->engine, vaddr); +int ret; +GEM_BUG_ON (intel_context_is_parent (ce) || +intel_context_is_child (ce)); + + ret =__guc_context_pin (ce, ce->engine);if (likely (!ret && !intel_context_is_barrier (ce)))intel_engine_pm_get (ce->engine);@@ -2054,7 +2066,7 @@ static void guc_context_unpin(struct intel_context *ce)GEM_BUG_ON (context_enabled (ce));unpin_guc_id (guc, ce, true); -lrc_unpin (ce); +__guc_context_unpin (ce);if (likely (!intel_context_is_barrier (ce)))intel_engine_pm_put (ce->engine);@@ -2062,7 +2074,141 @@ static void guc_context_unpin(struct intel_context *ce)static void guc_context_post_unpin (struct intel_context *ce) { -lrc_post_unpin (ce); +__guc_context_post_unpin (ce); +} + +/* Future patches will use this function */ +__maybe_unused +static int guc_parent_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +struct intel_context *child; +int err, i =0 , j =0 ; + +for_each_child (ce, child) { + err =i915_active_acquire (&child->active); +if (unlikely (err)) + goto unwind_active; + ++i; + } + +for_each_child (ce, child) { + err =__guc_context_pre_pin (child, child->engine, ww); +if (unlikely (err)) + goto unwind_pre_pin; + ++j; + } + + err =__guc_context_pre_pin (ce, ce->engine, ww); +if (unlikely (err)) + goto unwind_pre_pin; + +return 0 ; + +unwind_pre_pin: +for_each_child (ce, child) { +if (!j--) +break ; +__guc_context_post_unpin (child); + } + +unwind_active: +for_each_child (ce, child) { +if (!i--) +break ; +i915_active_release (&child->active); + } + +return err; +} + +/* Future patches will use this function */ +__maybe_unused +static void guc_parent_context_post_unpin (struct intel_context *ce) +{ +struct intel_context *child; + +for_each_child (ce, child) +__guc_context_post_unpin (child); +__guc_context_post_unpin (ce); + +for_each_child (ce, child) { +intel_context_get (child); +i915_active_release (&child->active); +intel_context_put (child); + } +} + +/* Future patches will use this function */ +__maybe_unused +static int guc_parent_context_pin (struct intel_context *ce) +{ +int ret, i =0 , j =0 ; +struct intel_context *child; +struct intel_engine_cs *engine; +intel_engine_mask_t tmp; + +GEM_BUG_ON (!intel_context_is_parent (ce)); + +for_each_child (ce, child) { + ret =__guc_context_pin (child, child->engine); +if (unlikely (ret)) + goto unwind_pin; + ++i; + } + ret =__guc_context_pin (ce, ce->engine); +if (unlikely (ret)) + goto unwind_pin; + +for_each_child (ce, child) +if (test_bit (CONTEXT_LRCA_DIRTY , &child->flags)) { +set_bit (CONTEXT_LRCA_DIRTY , &ce->flags); +break ; + } + +for_each_engine_masked (engine, ce->engine->gt, + ce->engine->mask, tmp) +intel_engine_pm_get (engine); +for_each_child (ce, child) +for_each_engine_masked (engine, child->engine->gt, + child->engine->mask, tmp) +intel_engine_pm_get (engine); + +return 0 ; + +unwind_pin: +for_each_child (ce, child) { +if (++j > i) +break ; +__guc_context_unpin (child); + } + +return ret; +} + +/* Future patches will use this function */ +__maybe_unused +static void guc_parent_context_unpin (struct intel_context *ce) +{ +struct intel_context *child; +struct intel_engine_cs *engine; +intel_engine_mask_t tmp; + +GEM_BUG_ON (!intel_context_is_parent (ce)); +GEM_BUG_ON (context_enabled (ce)); + +unpin_guc_id (ce_to_guc (ce), ce, true); +for_each_child (ce, child) +__guc_context_unpin (child); +__guc_context_unpin (ce); + +for_each_engine_masked (engine, ce->engine->gt, + ce->engine->mask, tmp) +intel_engine_pm_put (engine); +for_each_child (ce, child) +for_each_engine_masked (engine, child->engine->gt, + child->engine->mask, tmp) +intel_engine_pm_put (engine); } static void __guc_context_sched_enable(struct intel_guc *guc,@@ -2993,18 +3139,17 @@ static int guc_request_alloc(struct i915_request *rq) }static int guc_virtual_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) {struct intel_engine_cs *engine =guc_virtual_get_sibling (ce->engine,0 ); -return __guc_context_pre_pin (ce, engine, ww, vaddr); +return __guc_context_pre_pin (ce, engine, ww); } -static int guc_virtual_context_pin (struct intel_context *ce,void *vaddr) +static int guc_virtual_context_pin (struct intel_context *ce) {struct intel_engine_cs *engine =guc_virtual_get_sibling (ce->engine,0 ); -int ret =__guc_context_pin (ce, engine, vaddr); +int ret =__guc_context_pin (ce, engine);intel_engine_mask_t tmp, mask = ce->engine->mask;if (likely (!ret))@@ -3024,7 +3169,7 @@ static void guc_virtual_context_unpin(struct intel_context *ce)GEM_BUG_ON (intel_context_is_barrier (ce));unpin_guc_id (guc, ce, true); -lrc_unpin (ce); +__guc_context_unpin (ce);for_each_engine_masked (engine, ce->engine->gt, mask, tmp)intel_engine_pm_put (engine);Signature · 1 line
2.28.0Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [Intel-gfx] [PATCH 16/46] drm/i915/guc: Implement GuC parent-child context pin / unpin functions
Quoted 257 lines · Matthew Brost
Quoted 248 lines · Daniel Vetter
Quoted 214 lines · Matthew Brost
Implement GuC parent-child context pin / unpin functions in which in any contexts in the relationship are pinned all the contexts are pinned. The parent owns most of the pinning / unpinning process and the children direct any pins / unpins to the parent.
Patch implements a number of unused functions that will be connected later in the series.
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/gt/intel_context.c | 187 ++++++++++++++++-- drivers/gpu/drm/i915/gt/intel_context.h | 43 +--- drivers/gpu/drm/i915/gt/intel_context_types.h | 4 +- .../drm/i915/gt/intel_execlists_submission.c | 25 ++- drivers/gpu/drm/i915/gt/intel_lrc.c | 26 +-- drivers/gpu/drm/i915/gt/intel_lrc.h | 6 +- .../gpu/drm/i915/gt/intel_ring_submission.c | 5 +- drivers/gpu/drm/i915/gt/mock_engine.c | 4 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 183 +++++++++++++++-- 9 files changed, 371 insertions(+), 112 deletions(-)drivers/gpu/drm/i915/gt/intel_context.c +107
-18@@ -158,8 +158,8 @@ static void __ring_retire(struct intel_ring *ring)intel_ring_unpin (ring); } -static int intel_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) +static int __intel_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) {int err;@@ -190,7 +190,7 @@ static int intel_context_pre_pin(struct intel_context *ce,return err; } -static void intel_context_post_unpin (struct intel_context *ce) +static void __intel_context_post_unpin (struct intel_context *ce) {if (ce->state)__context_unpin_state (ce->state);@@ -199,13 +199,85 @@ static void intel_context_post_unpin(struct intel_context *ce)__ring_retire (ce->ring); } -int __intel_context_do_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) +static int intel_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) { -bool handoff = false; -void *vaddr; +struct intel_context *child; +int err, i =0 ; + +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) { + err =__intel_context_pre_pin (child, ww); +if (unlikely (err)) + goto unwind; + ++i; + } + + err =__intel_context_pre_pin (ce, ww); +if (unlikely (err)) + goto unwind; + +return 0 ; + +unwind: +for_each_child (ce, child) { +if (!i--) +break ; +__intel_context_post_unpin (ce); + } + +return err; +} + +static void intel_context_post_unpin (struct intel_context *ce) +{ +struct intel_context *child; + +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) +__intel_context_post_unpin (child); + +__intel_context_post_unpin (ce); +} + +static int __do_ww_lock (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +int err =i915_gem_object_lock (ce->timeline->hwsp_ggtt->obj, ww); + +if (!err && ce->ring->vma->obj) + err =i915_gem_object_lock (ce->ring->vma->obj, ww); +if (!err && ce->state) + err =i915_gem_object_lock (ce->state->obj, ww); + +return err; +} + +static int do_ww_lock (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +struct intel_context *child;int err =0 ; +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) { + err =__do_ww_lock (child, ww); +if (unlikely (err)) +return err; + } + +return __do_ww_lock (ce, ww); +} + +static int __intel_context_do_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +bool handoff = false; +int err; +if (unlikely (!test_bit (CONTEXT_ALLOC_BIT , &ce->flags))) { err =intel_context_alloc_state (ce);if (err)@@ -217,14 +289,11 @@ int __intel_context_do_pin_ww(struct intel_context *ce,* refcount for __intel_context_active(), which prevent a lock * inversion of ce->pin_mutex vs dma_resv_lock(). */ + err =do_ww_lock (ce, ww); +if (err) +return err; - err =i915_gem_object_lock (ce->timeline->hwsp_ggtt->obj, ww); -if (!err && ce->ring->vma->obj) - err =i915_gem_object_lock (ce->ring->vma->obj, ww); -if (!err && ce->state) - err =i915_gem_object_lock (ce->state->obj, ww); -if (!err) - err =intel_context_pre_pin (ce, ww); + err =intel_context_pre_pin (ce, ww);if (err)return err;@@ -232,7 +301,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,if (err) goto err_ctx_unpin; - err = ce->ops->pre_pin (ce, ww, &vaddr); + err = ce->ops->pre_pin (ce, ww);if (err) goto err_release;@@ -250,7 +319,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,if (unlikely (err)) goto err_unlock; - err = ce->ops->pin (ce, vaddr); + err = ce->ops->pin (ce);if (err) {intel_context_active_release (ce); goto err_unlock;@@ -290,7 +359,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,return err; } -int __intel_context_do_pin (struct intel_context *ce) +static int __intel_context_do_pin (struct intel_context *ce) {struct i915_gem_ww_ctx ww;int err;@@ -337,7 +406,7 @@ static void __intel_context_retire(struct i915_active *active) intel_context_get_avg_runtime_ns(ce));set_bit (CONTEXT_VALID_BIT , &ce->flags); -intel_context_post_unpin (ce); +__intel_context_post_unpin (ce);intel_context_put (ce); }@@ -562,6 +631,88 @@ void intel_context_bind_parent_child(struct intel_context *parent, child->parent = parent; } +static inline int ____intel_context_pin (struct intel_context *ce) +{ +if (likely (intel_context_pin_if_active (ce))) +return 0 ; + +return __intel_context_do_pin (ce); +} + +static inline int __intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +if (likely (intel_context_pin_if_active (ce))) +return 0 ; + +return __intel_context_do_pin_ww (ce, ww); +} + +static inline void __intel_context_unpin (struct intel_context *ce) +{ +if (!ce->ops->sched_disable) {drivers/gpu/drm/i915/gt/intel_context.c void intel_context_bind_parent_child(struct intel_context *parent, · line 663
Quoted 10 lines
drivers/gpu/drm/i915/gt/intel_context.c +10
-0+__intel_context_do_unpin (ce,1 ); + }else { +/* +* Move ownership of this pin to the scheduling disable which is +* an async operation. When that operation completes the above +* intel_context_sched_disable_unpin is called potentially +* unpinning the context. +*/ +while (!atomic_add_unless (&ce->pin_count,-1 ,1 )) { +if (atomic_cmpxchg (&ce->pin_count,1 ,2 ) ==1 ) {Uh man lockless algorithms.
Unless this comes:
- with essentially an academic looking paper that describes the abstract model of the lockless algorithm and proves it against the linux kernel meory model.- lockless stuff generally needs barriers, and those barriers must be all documented. This means a) a comment next to each barrier in the code b) pointing to its counterparty c) with the overall design also explained in the kerneldoc for those datastructres.
If you don't know where your barriers are, see above point about "it should look more like an academic paper in the commit message"
- hard perf data about how this is absolutely required, based on a real-world use-case (which then sometimes justifies a microbenchmark metric for the details, but it always needs to be real-world based). And also a throughrough explainer how the perf issue isn't fixable through better design. If that's not doable, just protect the state machine with a big dumb lock and move on.
Quoted 7 lines
- Also, because the current code is in such bad shape wrt lockless algorithms and premature optimizations: Overall complexity should go down (it's way too high right now), so pay down your new lockless trick by removing one of the existing ones that we only have because we can.
Yes this is steep, but we're way out in the woods here and need to smoehow get back.
Quoted 7 lines
See below FIXME. At one point all of this was hidden in the backend but the dma-resv patches that landed upstream completely broke the layering, hence the need for the code here.
I guess I don't really understand what mean when you say lockless alg needs barriers, if the atomic functions are not really atomic wouldn't the world be broken?
They unordered atomics by default. Which means they're atomic itself, but entirely unordered with anything else that's going on. Except when you have one of the atomic ops which already guarantee a barrier, or you manually add the barriers yourself. And yes there's enormous amounts of bugs, and with our dgpu potentially running on non-IA cpus those bugs matter.
Note that in C++ atomics the default behaviour is strongly ordered atomics with full barriers by default, because those are much easier to program against. Kernel isn't like that and defaults to "you need to add all the barriers yourself".
I have a full lenght rant in the works and will work that through all channels, but essentially locking is really hard to get right. And lockless tricks practically need an academic paper with a formal correctness proof against the linux memory model, or you do have bugs.
And I know that the current code is choke full of this stuff, so it's tempting to just add more, but we really cant. The amount of locking trickery we have in the codebase must go down substantially. My take is that any code that adds anything trick needs to fully justify it against the above list, _and_ also clean up some of the existing nonsense so that overall complexity doesn't increase.
I'll share the full length rant with you internally, it's not yet ready for publishing (but that's planned too).
Quoted 4 lines
Also here I don't think it is really as simple as grab big dump lock for a variety of reasons, at least with the current dynamic pin / unpin code in place. If we move a perma-pinned contexts this could be cleaned up then.
Yes it's a disaster, but we need to stop the bleeding. If perma-pinned context can fix this I think we should do this asap. I'd say for parallel context we should just do it outright (special case them or whatever) so that we don't have to add even more very tricky code and tech debt.
Doable?
Cheers, Daniel
Quoted 587 lines
Matt
Quoted 584 lines
-Daniel
Quoted 577 lines
drivers/gpu/drm/i915/gt/intel_context.c +52
-0+ ce->ops->sched_disable (ce); +break ; + } + } + } +} + +/* +* FIXME: This is ugly, these branches are only needed for parallel contexts in +* GuC submission. Basically the idea is if any of the contexts, that are +* configured for parallel submission, are pinned all the contexts need to be +* pinned in order to register these contexts with the GuC. We are adding the +* layer here while it should probably be pushed to the backend via a vfunc. But +* since we already have ce->pin + a layer atop it is confusing. Definitely +* needs a bit of rework how to properly layer / structure this code path. What +* is in place works but is not ideal. +*/ +int intel_context_pin (struct intel_context *ce) +{ +if (intel_context_is_child (ce)) { +if (!atomic_fetch_add (1 , &ce->pin_count)) +return ____intel_context_pin (ce->parent); +else +return 0 ; + }else { +return ____intel_context_pin (ce); + } +} + +int intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +if (intel_context_is_child (ce)) { +if (!atomic_fetch_add (1 , &ce->pin_count)) +return __intel_context_pin_ww (ce->parent, ww); +else +return 0 ; + }else { +return __intel_context_pin_ww (ce, ww); + } +} + +void intel_context_unpin (struct intel_context *ce) +{ +if (intel_context_is_child (ce)) { +if (atomic_fetch_add (-1 , &ce->pin_count) ==1 ) +__intel_context_unpin (ce->parent); + }else { +__intel_context_unpin (ce); + } +} +#if IS_ENABLED (CONFIG_DRM_I915_SELFTEST )#include "selftest_context.c" #endif drivers/gpu/drm/i915/gt/intel_context.h +5
-38@@ -110,31 +110,15 @@ static inline void intel_context_unlock_pinned(struct intel_context *ce)mutex_unlock (&ce->pin_mutex); } -int __intel_context_do_pin (struct intel_context *ce); -int __intel_context_do_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww); -static inline bool intel_context_pin_if_active (struct intel_context *ce) {return atomic_inc_not_zero (&ce->pin_count); } -static inline int intel_context_pin (struct intel_context *ce) -{ -if (likely (intel_context_pin_if_active (ce))) -return 0 ; - -return __intel_context_do_pin (ce); -} - -static inline int intel_context_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) -{ -if (likely (intel_context_pin_if_active (ce))) -return 0 ; +int intel_context_pin (struct intel_context *ce); -return __intel_context_do_pin_ww (ce, ww); -} +int intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww);static inline void __intel_context_pin (struct intel_context *ce) {@@ -146,28 +130,11 @@ void __intel_context_do_unpin(struct intel_context *ce, int sub);static inline void intel_context_sched_disable_unpin (struct intel_context *ce) { +GEM_BUG_ON (intel_context_is_child (ce));__intel_context_do_unpin (ce,2 ); } -static inline void intel_context_unpin (struct intel_context *ce) -{ -if (!ce->ops->sched_disable) { -__intel_context_do_unpin (ce,1 ); - }else { -/* -* Move ownership of this pin to the scheduling disable which is -* an async operation. When that operation completes the above -* intel_context_sched_disable_unpin is called potentially -* unpinning the context. -*/ -while (!atomic_add_unless (&ce->pin_count,-1 ,1 )) { -if (atomic_cmpxchg (&ce->pin_count,1 ,2 ) ==1 ) { - ce->ops->sched_disable (ce); -break ; - } - } - } -} +void intel_context_unpin (struct intel_context *ce);void intel_context_enter_engine (struct intel_context *ce);void intel_context_exit_engine (struct intel_context *ce);drivers/gpu/drm/i915/gt/intel_context_types.h +2
-2@@ -39,8 +39,8 @@ struct intel_context_ops {void (*ban)(struct intel_context *ce,struct i915_request *rq); -int (*pre_pin)(struct intel_context *ce,struct i915_gem_ww_ctx *ww,void **vaddr); -int (*pin)(struct intel_context *ce,void *vaddr); +int (*pre_pin)(struct intel_context *ce,struct i915_gem_ww_ctx *ww); +int (*pin)(struct intel_context *ce);void (*unpin)(struct intel_context *ce);void (*post_unpin)(struct intel_context *ce);drivers/gpu/drm/i915/gt/intel_execlists_submission.c +12
-13@@ -2554,16 +2554,17 @@ static void execlists_submit_request(struct i915_request *request)static int __execlists_context_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww,void **vaddr) +struct i915_gem_ww_ctx *ww) {int err; - err =lrc_pre_pin (ce, engine, ww, vaddr); + err =lrc_pre_pin (ce, engine, ww);if (err)return err;if (!__test_and_set_bit (CONTEXT_INIT_BIT , &ce->flags)) { -lrc_init_state (ce, engine, *vaddr); +lrc_init_state (ce, engine, ce->lrc_reg_state - +LRC_STATE_OFFSET /sizeof (*ce->lrc_reg_state));__i915_gem_object_flush_map (ce->state->obj,0 , engine->context_size); }@@ -2572,15 +2573,14 @@ __execlists_context_pre_pin(struct intel_context *ce, }static int execlists_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return __execlists_context_pre_pin (ce, ce->engine, ww, vaddr); +return __execlists_context_pre_pin (ce, ce->engine, ww); } -static int execlists_context_pin (struct intel_context *ce,void *vaddr) +static int execlists_context_pin (struct intel_context *ce) { -return lrc_pin (ce, ce->engine, vaddr); +return lrc_pin (ce, ce->engine); }static int execlists_context_alloc (struct intel_context *ce)@@ -3570,20 +3570,19 @@ static int virtual_context_alloc(struct intel_context *ce) }static int virtual_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) {struct virtual_engine *ve =container_of (ce,typeof (*ve), context);/* Note: we must use a real engine class for setting up reg state */ -return __execlists_context_pre_pin (ce, ve->siblings[0 ], ww, vaddr); +return __execlists_context_pre_pin (ce, ve->siblings[0 ], ww); } -static int virtual_context_pin (struct intel_context *ce,void *vaddr) +static int virtual_context_pin (struct intel_context *ce) {struct virtual_engine *ve =container_of (ce,typeof (*ve), context); -return lrc_pin (ce, ve->siblings[0 ], vaddr); +return lrc_pin (ce, ve->siblings[0 ]); }static void virtual_context_enter (struct intel_context *ce)drivers/gpu/drm/i915/gt/intel_lrc.c +13
-13@@ -947,30 +947,30 @@ void lrc_reset(struct intel_context *ce)int lrc_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { +void *vaddr;GEM_BUG_ON (!ce->state);GEM_BUG_ON (!i915_vma_is_pinned (ce->state)); - *vaddr =i915_gem_object_pin_map (ce->state->obj, -i915_coherent_map_type (ce->engine->i915, - ce->state->obj, - false) | -I915_MAP_OVERRIDE ); + vaddr =i915_gem_object_pin_map (ce->state->obj, +i915_coherent_map_type (ce->engine->i915, + ce->state->obj, + false) | +I915_MAP_OVERRIDE ); -return PTR_ERR_OR_ZERO (*vaddr); + ce->lrc_reg_state = vaddr +LRC_STATE_OFFSET ; + +return PTR_ERR_OR_ZERO (vaddr); }int lrc_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr) +struct intel_engine_cs *engine) { - ce->lrc_reg_state = vaddr +LRC_STATE_OFFSET ; -if (!__test_and_set_bit (CONTEXT_INIT_BIT , &ce->flags)) -lrc_init_state (ce, engine, vaddr); +lrc_init_state (ce, engine, + (void *)ce->lrc_reg_state -LRC_STATE_OFFSET ); ce->lrc.lrca =lrc_update_regs (ce, engine, ce->ring->tail);return 0 ;drivers/gpu/drm/i915/gt/intel_lrc.h +2
-4@@ -38,12 +38,10 @@ void lrc_destroy(struct kref *kref);int lrc_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr); +struct i915_gem_ww_ctx *ww);int lrc_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr); +struct intel_engine_cs *engine);void lrc_unpin (struct intel_context *ce);void lrc_post_unpin (struct intel_context *ce);drivers/gpu/drm/i915/gt/intel_ring_submission.c +2
-3@@ -472,8 +472,7 @@ static int ring_context_init_default_state(struct intel_context *ce, }static int ring_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **unused) +struct i915_gem_ww_ctx *ww) {struct i915_address_space *vm;int err =0 ;@@ -576,7 +575,7 @@ static int ring_context_alloc(struct intel_context *ce)return 0 ; } -static int ring_context_pin (struct intel_context *ce,void *unused) +static int ring_context_pin (struct intel_context *ce) {return 0 ; }drivers/gpu/drm/i915/gt/mock_engine.c +2
-2@@ -167,12 +167,12 @@ static int mock_context_alloc(struct intel_context *ce) }static int mock_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww,void **unused) +struct i915_gem_ww_ctx *ww) {return 0 ; } -static int mock_context_pin (struct intel_context *ce,void *unused) +static int mock_context_pin (struct intel_context *ce) {return 0 ; }drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +164
-19@@ -1905,6 +1905,7 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)GEM_BUG_ON (!engine->mask);GEM_BUG_ON (context_guc_id_invalid (ce)); +GEM_BUG_ON (intel_context_is_child (ce));/* * Ensure LRC + CT vmas are is same region as write barrier is done @@ -2008,15 +2009,13 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)static int __guc_context_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return lrc_pre_pin (ce, engine, ww, vaddr); +return lrc_pre_pin (ce, engine, ww); }static int __guc_context_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr) +struct intel_engine_cs *engine) {if (i915_ggtt_offset (ce->state) != (ce->lrc.lrca &CTX_GTT_ADDRESS_MASK ))@@ -2027,20 +2026,33 @@ static int __guc_context_pin(struct intel_context *ce,* explaination of why. */ -return lrc_pin (ce, engine, vaddr); +return lrc_pin (ce, engine); +} + +static void __guc_context_unpin (struct intel_context *ce) +{ +lrc_unpin (ce); +} + +static void __guc_context_post_unpin (struct intel_context *ce) +{ +lrc_post_unpin (ce); }static int guc_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return __guc_context_pre_pin (ce, ce->engine, ww, vaddr); +return __guc_context_pre_pin (ce, ce->engine, ww); } -static int guc_context_pin (struct intel_context *ce,void *vaddr) +static int guc_context_pin (struct intel_context *ce) { -int ret =__guc_context_pin (ce, ce->engine, vaddr); +int ret; +GEM_BUG_ON (intel_context_is_parent (ce) || +intel_context_is_child (ce)); + + ret =__guc_context_pin (ce, ce->engine);if (likely (!ret && !intel_context_is_barrier (ce)))intel_engine_pm_get (ce->engine);@@ -2054,7 +2066,7 @@ static void guc_context_unpin(struct intel_context *ce)GEM_BUG_ON (context_enabled (ce));unpin_guc_id (guc, ce, true); -lrc_unpin (ce); +__guc_context_unpin (ce);if (likely (!intel_context_is_barrier (ce)))intel_engine_pm_put (ce->engine);@@ -2062,7 +2074,141 @@ static void guc_context_unpin(struct intel_context *ce)static void guc_context_post_unpin (struct intel_context *ce) { -lrc_post_unpin (ce); +__guc_context_post_unpin (ce); +} + +/* Future patches will use this function */ +__maybe_unused +static int guc_parent_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +struct intel_context *child; +int err, i =0 , j =0 ; + +for_each_child (ce, child) { + err =i915_active_acquire (&child->active); +if (unlikely (err)) + goto unwind_active; + ++i; + } + +for_each_child (ce, child) { + err =__guc_context_pre_pin (child, child->engine, ww); +if (unlikely (err)) + goto unwind_pre_pin; + ++j; + } + + err =__guc_context_pre_pin (ce, ce->engine, ww); +if (unlikely (err)) + goto unwind_pre_pin; + +return 0 ; + +unwind_pre_pin: +for_each_child (ce, child) { +if (!j--) +break ; +__guc_context_post_unpin (child); + } + +unwind_active: +for_each_child (ce, child) { +if (!i--) +break ; +i915_active_release (&child->active); + } + +return err; +} + +/* Future patches will use this function */ +__maybe_unused +static void guc_parent_context_post_unpin (struct intel_context *ce) +{ +struct intel_context *child; + +for_each_child (ce, child) +__guc_context_post_unpin (child); +__guc_context_post_unpin (ce); + +for_each_child (ce, child) { +intel_context_get (child); +i915_active_release (&child->active); +intel_context_put (child); + } +} + +/* Future patches will use this function */ +__maybe_unused +static int guc_parent_context_pin (struct intel_context *ce) +{ +int ret, i =0 , j =0 ; +struct intel_context *child; +struct intel_engine_cs *engine; +intel_engine_mask_t tmp; + +GEM_BUG_ON (!intel_context_is_parent (ce)); + +for_each_child (ce, child) { + ret =__guc_context_pin (child, child->engine); +if (unlikely (ret)) + goto unwind_pin; + ++i; + } + ret =__guc_context_pin (ce, ce->engine); +if (unlikely (ret)) + goto unwind_pin; + +for_each_child (ce, child) +if (test_bit (CONTEXT_LRCA_DIRTY , &child->flags)) { +set_bit (CONTEXT_LRCA_DIRTY , &ce->flags); +break ; + } + +for_each_engine_masked (engine, ce->engine->gt, + ce->engine->mask, tmp) +intel_engine_pm_get (engine); +for_each_child (ce, child) +for_each_engine_masked (engine, child->engine->gt, + child->engine->mask, tmp) +intel_engine_pm_get (engine); + +return 0 ; + +unwind_pin: +for_each_child (ce, child) { +if (++j > i) +break ; +__guc_context_unpin (child); + } + +return ret; +} + +/* Future patches will use this function */ +__maybe_unused +static void guc_parent_context_unpin (struct intel_context *ce) +{ +struct intel_context *child; +struct intel_engine_cs *engine; +intel_engine_mask_t tmp; + +GEM_BUG_ON (!intel_context_is_parent (ce)); +GEM_BUG_ON (context_enabled (ce)); + +unpin_guc_id (ce_to_guc (ce), ce, true); +for_each_child (ce, child) +__guc_context_unpin (child); +__guc_context_unpin (ce); + +for_each_engine_masked (engine, ce->engine->gt, + ce->engine->mask, tmp) +intel_engine_pm_put (engine); +for_each_child (ce, child) +for_each_engine_masked (engine, child->engine->gt, + child->engine->mask, tmp) +intel_engine_pm_put (engine); } static void __guc_context_sched_enable(struct intel_guc *guc,@@ -2993,18 +3139,17 @@ static int guc_request_alloc(struct i915_request *rq) }static int guc_virtual_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) {struct intel_engine_cs *engine =guc_virtual_get_sibling (ce->engine,0 ); -return __guc_context_pre_pin (ce, engine, ww, vaddr); +return __guc_context_pre_pin (ce, engine, ww); } -static int guc_virtual_context_pin (struct intel_context *ce,void *vaddr) +static int guc_virtual_context_pin (struct intel_context *ce) {struct intel_engine_cs *engine =guc_virtual_get_sibling (ce->engine,0 ); -int ret =__guc_context_pin (ce, engine, vaddr); +int ret =__guc_context_pin (ce, engine);intel_engine_mask_t tmp, mask = ce->engine->mask;if (likely (!ret))@@ -3024,7 +3169,7 @@ static void guc_virtual_context_unpin(struct intel_context *ce)GEM_BUG_ON (intel_context_is_barrier (ce));unpin_guc_id (guc, ce, true); -lrc_unpin (ce); +__guc_context_unpin (ce);for_each_engine_masked (engine, ce->engine->gt, mask, tmp)intel_engine_pm_put (engine);Signature · 1 line
2.28.0Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [Intel-gfx] [PATCH 16/46] drm/i915/guc: Implement GuC parent-child context pin / unpin functions
Quoted 219 lines · Daniel Vetter
Quoted 218 lines · Matthew Brost
Quoted 217 lines · Daniel Vetter
Quoted 216 lines · Matthew Brost
Implement GuC parent-child context pin / unpin functions in which in any contexts in the relationship are pinned all the contexts are pinned. The parent owns most of the pinning / unpinning process and the children direct any pins / unpins to the parent.
Patch implements a number of unused functions that will be connected later in the series.
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/gt/intel_context.c | 187 ++++++++++++++++-- drivers/gpu/drm/i915/gt/intel_context.h | 43 +--- drivers/gpu/drm/i915/gt/intel_context_types.h | 4 +- .../drm/i915/gt/intel_execlists_submission.c | 25 ++- drivers/gpu/drm/i915/gt/intel_lrc.c | 26 +-- drivers/gpu/drm/i915/gt/intel_lrc.h | 6 +- .../gpu/drm/i915/gt/intel_ring_submission.c | 5 +- drivers/gpu/drm/i915/gt/mock_engine.c | 4 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 183 +++++++++++++++-- 9 files changed, 371 insertions(+), 112 deletions(-)drivers/gpu/drm/i915/gt/intel_context.c +109
-18@@ -158,8 +158,8 @@ static void __ring_retire(struct intel_ring *ring)intel_ring_unpin (ring); } -static int intel_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) +static int __intel_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) {int err;@@ -190,7 +190,7 @@ static int intel_context_pre_pin(struct intel_context *ce,return err; } -static void intel_context_post_unpin (struct intel_context *ce) +static void __intel_context_post_unpin (struct intel_context *ce) {if (ce->state)__context_unpin_state (ce->state);@@ -199,13 +199,85 @@ static void intel_context_post_unpin(struct intel_context *ce)__ring_retire (ce->ring); } -int __intel_context_do_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) +static int intel_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) { -bool handoff = false; -void *vaddr; +struct intel_context *child; +int err, i =0 ; + +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) { + err =__intel_context_pre_pin (child, ww); +if (unlikely (err)) + goto unwind; + ++i; + } + + err =__intel_context_pre_pin (ce, ww); +if (unlikely (err)) + goto unwind; + +return 0 ; + +unwind: +for_each_child (ce, child) { +if (!i--) +break ; +__intel_context_post_unpin (ce); + } + +return err; +} + +static void intel_context_post_unpin (struct intel_context *ce) +{ +struct intel_context *child; + +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) +__intel_context_post_unpin (child); + +__intel_context_post_unpin (ce); +} + +static int __do_ww_lock (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +int err =i915_gem_object_lock (ce->timeline->hwsp_ggtt->obj, ww); + +if (!err && ce->ring->vma->obj) + err =i915_gem_object_lock (ce->ring->vma->obj, ww); +if (!err && ce->state) + err =i915_gem_object_lock (ce->state->obj, ww); + +return err; +} + +static int do_ww_lock (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +struct intel_context *child;int err =0 ; +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) { + err =__do_ww_lock (child, ww); +if (unlikely (err)) +return err; + } + +return __do_ww_lock (ce, ww); +} + +static int __intel_context_do_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +bool handoff = false; +int err; +if (unlikely (!test_bit (CONTEXT_ALLOC_BIT , &ce->flags))) { err =intel_context_alloc_state (ce);if (err)@@ -217,14 +289,11 @@ int __intel_context_do_pin_ww(struct intel_context *ce,* refcount for __intel_context_active(), which prevent a lock * inversion of ce->pin_mutex vs dma_resv_lock(). */ + err =do_ww_lock (ce, ww); +if (err) +return err; - err =i915_gem_object_lock (ce->timeline->hwsp_ggtt->obj, ww); -if (!err && ce->ring->vma->obj) - err =i915_gem_object_lock (ce->ring->vma->obj, ww); -if (!err && ce->state) - err =i915_gem_object_lock (ce->state->obj, ww); -if (!err) - err =intel_context_pre_pin (ce, ww); + err =intel_context_pre_pin (ce, ww);if (err)return err;@@ -232,7 +301,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,if (err) goto err_ctx_unpin; - err = ce->ops->pre_pin (ce, ww, &vaddr); + err = ce->ops->pre_pin (ce, ww);if (err) goto err_release;@@ -250,7 +319,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,if (unlikely (err)) goto err_unlock; - err = ce->ops->pin (ce, vaddr); + err = ce->ops->pin (ce);if (err) {intel_context_active_release (ce); goto err_unlock;@@ -290,7 +359,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,return err; } -int __intel_context_do_pin (struct intel_context *ce) +static int __intel_context_do_pin (struct intel_context *ce) {struct i915_gem_ww_ctx ww;int err;@@ -337,7 +406,7 @@ static void __intel_context_retire(struct i915_active *active) intel_context_get_avg_runtime_ns(ce));set_bit (CONTEXT_VALID_BIT , &ce->flags); -intel_context_post_unpin (ce); +__intel_context_post_unpin (ce);intel_context_put (ce); }@@ -562,6 +631,88 @@ void intel_context_bind_parent_child(struct intel_context *parent, child->parent = parent; } +static inline int ____intel_context_pin (struct intel_context *ce) +{ +if (likely (intel_context_pin_if_active (ce))) +return 0 ; + +return __intel_context_do_pin (ce); +} + +static inline int __intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +if (likely (intel_context_pin_if_active (ce))) +return 0 ; + +return __intel_context_do_pin_ww (ce, ww); +} + +static inline void __intel_context_unpin (struct intel_context *ce) +{ +if (!ce->ops->sched_disable) { +__intel_context_do_unpin (ce,1 ); + }else {
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
drivers/gpu/drm/i915/gt/intel_context.c void intel_context_bind_parent_child(struct intel_context *parent, · line 663
Quoted 8 lines
drivers/gpu/drm/i915/gt/intel_context.c +8
-0+/* +* Move ownership of this pin to the scheduling disable which is +* an async operation. When that operation completes the above +* intel_context_sched_disable_unpin is called potentially +* unpinning the context. +*/ +while (!atomic_add_unless (&ce->pin_count,-1 ,1 )) { +if (atomic_cmpxchg (&ce->pin_count,1 ,2 ) ==1 ) {
Just as an example of what I mean here on the code review side. This is an endless loop, and you need to prove that there's no livelock or starvation issues. Or explain how else you handle that if there is one.
Because unlike hand-rolled stuff linux kernel spinlocks are not dumb spinlocks, but ticketed/queued locks and therefor starvation proof. But this stuff actually matters on todays multi-core and not-so-uniform (even without fully NUMA) architectures.
Also I've just found another lockless retry loop which does actually degenerate into a full endless loop (if you're sufficiently unlucky in your races), so this really isn't academic at all.
-Daniel
Quoted 674 lines
Quoted 31 lines
Quoted 23 lines
Uh man lockless algorithms.
Unless this comes:
- with essentially an academic looking paper that describes the abstract model of the lockless algorithm and proves it against the linux kernel meory model.- lockless stuff generally needs barriers, and those barriers must be all documented. This means a) a comment next to each barrier in the code b) pointing to its counterparty c) with the overall design also explained in the kerneldoc for those datastructres.
If you don't know where your barriers are, see above point about "it should look more like an academic paper in the commit message"
- hard perf data about how this is absolutely required, based on a real-world use-case (which then sometimes justifies a microbenchmark metric for the details, but it always needs to be real-world based). And also a throughrough explainer how the perf issue isn't fixable through better design. If that's not doable, just protect the state machine with a big dumb lock and move on.
Quoted 7 lines
- Also, because the current code is in such bad shape wrt lockless algorithms and premature optimizations: Overall complexity should go down (it's way too high right now), so pay down your new lockless trick by removing one of the existing ones that we only have because we can.
Yes this is steep, but we're way out in the woods here and need to smoehow get back.
Quoted 7 lines
See below FIXME. At one point all of this was hidden in the backend but the dma-resv patches that landed upstream completely broke the layering, hence the need for the code here.
I guess I don't really understand what mean when you say lockless alg needs barriers, if the atomic functions are not really atomic wouldn't the world be broken?
They unordered atomics by default. Which means they're atomic itself, but entirely unordered with anything else that's going on. Except when you have one of the atomic ops which already guarantee a barrier, or you manually add the barriers yourself. And yes there's enormous amounts of bugs, and with our dgpu potentially running on non-IA cpus those bugs matter.
Note that in C++ atomics the default behaviour is strongly ordered atomics with full barriers by default, because those are much easier to program against. Kernel isn't like that and defaults to "you need to add all the barriers yourself".
I have a full lenght rant in the works and will work that through all channels, but essentially locking is really hard to get right. And lockless tricks practically need an academic paper with a formal correctness proof against the linux memory model, or you do have bugs.
And I know that the current code is choke full of this stuff, so it's tempting to just add more, but we really cant. The amount of locking trickery we have in the codebase must go down substantially. My take is that any code that adds anything trick needs to fully justify it against the above list, _and_ also clean up some of the existing nonsense so that overall complexity doesn't increase.
I'll share the full length rant with you internally, it's not yet ready for publishing (but that's planned too).
Quoted 4 lines
Also here I don't think it is really as simple as grab big dump lock for a variety of reasons, at least with the current dynamic pin / unpin code in place. If we move a perma-pinned contexts this could be cleaned up then.
Yes it's a disaster, but we need to stop the bleeding. If perma-pinned context can fix this I think we should do this asap. I'd say for parallel context we should just do it outright (special case them or whatever) so that we don't have to add even more very tricky code and tech debt.
Doable?
Cheers, Daniel
Quoted 587 lines
Matt
Quoted 584 lines
-Daniel
Quoted 577 lines
drivers/gpu/drm/i915/gt/intel_context.c +52
-0+ ce->ops->sched_disable (ce); +break ; + } + } + } +} + +/* +* FIXME: This is ugly, these branches are only needed for parallel contexts in +* GuC submission. Basically the idea is if any of the contexts, that are +* configured for parallel submission, are pinned all the contexts need to be +* pinned in order to register these contexts with the GuC. We are adding the +* layer here while it should probably be pushed to the backend via a vfunc. But +* since we already have ce->pin + a layer atop it is confusing. Definitely +* needs a bit of rework how to properly layer / structure this code path. What +* is in place works but is not ideal. +*/ +int intel_context_pin (struct intel_context *ce) +{ +if (intel_context_is_child (ce)) { +if (!atomic_fetch_add (1 , &ce->pin_count)) +return ____intel_context_pin (ce->parent); +else +return 0 ; + }else { +return ____intel_context_pin (ce); + } +} + +int intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +if (intel_context_is_child (ce)) { +if (!atomic_fetch_add (1 , &ce->pin_count)) +return __intel_context_pin_ww (ce->parent, ww); +else +return 0 ; + }else { +return __intel_context_pin_ww (ce, ww); + } +} + +void intel_context_unpin (struct intel_context *ce) +{ +if (intel_context_is_child (ce)) { +if (atomic_fetch_add (-1 , &ce->pin_count) ==1 ) +__intel_context_unpin (ce->parent); + }else { +__intel_context_unpin (ce); + } +} +#if IS_ENABLED (CONFIG_DRM_I915_SELFTEST )#include "selftest_context.c" #endif drivers/gpu/drm/i915/gt/intel_context.h +5
-38@@ -110,31 +110,15 @@ static inline void intel_context_unlock_pinned(struct intel_context *ce)mutex_unlock (&ce->pin_mutex); } -int __intel_context_do_pin (struct intel_context *ce); -int __intel_context_do_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww); -static inline bool intel_context_pin_if_active (struct intel_context *ce) {return atomic_inc_not_zero (&ce->pin_count); } -static inline int intel_context_pin (struct intel_context *ce) -{ -if (likely (intel_context_pin_if_active (ce))) -return 0 ; - -return __intel_context_do_pin (ce); -} - -static inline int intel_context_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) -{ -if (likely (intel_context_pin_if_active (ce))) -return 0 ; +int intel_context_pin (struct intel_context *ce); -return __intel_context_do_pin_ww (ce, ww); -} +int intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww);static inline void __intel_context_pin (struct intel_context *ce) {@@ -146,28 +130,11 @@ void __intel_context_do_unpin(struct intel_context *ce, int sub);static inline void intel_context_sched_disable_unpin (struct intel_context *ce) { +GEM_BUG_ON (intel_context_is_child (ce));__intel_context_do_unpin (ce,2 ); } -static inline void intel_context_unpin (struct intel_context *ce) -{ -if (!ce->ops->sched_disable) { -__intel_context_do_unpin (ce,1 ); - }else { -/* -* Move ownership of this pin to the scheduling disable which is -* an async operation. When that operation completes the above -* intel_context_sched_disable_unpin is called potentially -* unpinning the context. -*/ -while (!atomic_add_unless (&ce->pin_count,-1 ,1 )) { -if (atomic_cmpxchg (&ce->pin_count,1 ,2 ) ==1 ) { - ce->ops->sched_disable (ce); -break ; - } - } - } -} +void intel_context_unpin (struct intel_context *ce);void intel_context_enter_engine (struct intel_context *ce);void intel_context_exit_engine (struct intel_context *ce);drivers/gpu/drm/i915/gt/intel_context_types.h +2
-2@@ -39,8 +39,8 @@ struct intel_context_ops {void (*ban)(struct intel_context *ce,struct i915_request *rq); -int (*pre_pin)(struct intel_context *ce,struct i915_gem_ww_ctx *ww,void **vaddr); -int (*pin)(struct intel_context *ce,void *vaddr); +int (*pre_pin)(struct intel_context *ce,struct i915_gem_ww_ctx *ww); +int (*pin)(struct intel_context *ce);void (*unpin)(struct intel_context *ce);void (*post_unpin)(struct intel_context *ce);drivers/gpu/drm/i915/gt/intel_execlists_submission.c +12
-13@@ -2554,16 +2554,17 @@ static void execlists_submit_request(struct i915_request *request)static int __execlists_context_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww,void **vaddr) +struct i915_gem_ww_ctx *ww) {int err; - err =lrc_pre_pin (ce, engine, ww, vaddr); + err =lrc_pre_pin (ce, engine, ww);if (err)return err;if (!__test_and_set_bit (CONTEXT_INIT_BIT , &ce->flags)) { -lrc_init_state (ce, engine, *vaddr); +lrc_init_state (ce, engine, ce->lrc_reg_state - +LRC_STATE_OFFSET /sizeof (*ce->lrc_reg_state));__i915_gem_object_flush_map (ce->state->obj,0 , engine->context_size); }@@ -2572,15 +2573,14 @@ __execlists_context_pre_pin(struct intel_context *ce, }static int execlists_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return __execlists_context_pre_pin (ce, ce->engine, ww, vaddr); +return __execlists_context_pre_pin (ce, ce->engine, ww); } -static int execlists_context_pin (struct intel_context *ce,void *vaddr) +static int execlists_context_pin (struct intel_context *ce) { -return lrc_pin (ce, ce->engine, vaddr); +return lrc_pin (ce, ce->engine); }static int execlists_context_alloc (struct intel_context *ce)@@ -3570,20 +3570,19 @@ static int virtual_context_alloc(struct intel_context *ce) }static int virtual_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) {struct virtual_engine *ve =container_of (ce,typeof (*ve), context);/* Note: we must use a real engine class for setting up reg state */ -return __execlists_context_pre_pin (ce, ve->siblings[0 ], ww, vaddr); +return __execlists_context_pre_pin (ce, ve->siblings[0 ], ww); } -static int virtual_context_pin (struct intel_context *ce,void *vaddr) +static int virtual_context_pin (struct intel_context *ce) {struct virtual_engine *ve =container_of (ce,typeof (*ve), context); -return lrc_pin (ce, ve->siblings[0 ], vaddr); +return lrc_pin (ce, ve->siblings[0 ]); }static void virtual_context_enter (struct intel_context *ce)drivers/gpu/drm/i915/gt/intel_lrc.c +13
-13@@ -947,30 +947,30 @@ void lrc_reset(struct intel_context *ce)int lrc_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { +void *vaddr;GEM_BUG_ON (!ce->state);GEM_BUG_ON (!i915_vma_is_pinned (ce->state)); - *vaddr =i915_gem_object_pin_map (ce->state->obj, -i915_coherent_map_type (ce->engine->i915, - ce->state->obj, - false) | -I915_MAP_OVERRIDE ); + vaddr =i915_gem_object_pin_map (ce->state->obj, +i915_coherent_map_type (ce->engine->i915, + ce->state->obj, + false) | +I915_MAP_OVERRIDE ); -return PTR_ERR_OR_ZERO (*vaddr); + ce->lrc_reg_state = vaddr +LRC_STATE_OFFSET ; + +return PTR_ERR_OR_ZERO (vaddr); }int lrc_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr) +struct intel_engine_cs *engine) { - ce->lrc_reg_state = vaddr +LRC_STATE_OFFSET ; -if (!__test_and_set_bit (CONTEXT_INIT_BIT , &ce->flags)) -lrc_init_state (ce, engine, vaddr); +lrc_init_state (ce, engine, + (void *)ce->lrc_reg_state -LRC_STATE_OFFSET ); ce->lrc.lrca =lrc_update_regs (ce, engine, ce->ring->tail);return 0 ;drivers/gpu/drm/i915/gt/intel_lrc.h +2
-4@@ -38,12 +38,10 @@ void lrc_destroy(struct kref *kref);int lrc_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr); +struct i915_gem_ww_ctx *ww);int lrc_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr); +struct intel_engine_cs *engine);void lrc_unpin (struct intel_context *ce);void lrc_post_unpin (struct intel_context *ce);drivers/gpu/drm/i915/gt/intel_ring_submission.c +2
-3@@ -472,8 +472,7 @@ static int ring_context_init_default_state(struct intel_context *ce, }static int ring_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **unused) +struct i915_gem_ww_ctx *ww) {struct i915_address_space *vm;int err =0 ;@@ -576,7 +575,7 @@ static int ring_context_alloc(struct intel_context *ce)return 0 ; } -static int ring_context_pin (struct intel_context *ce,void *unused) +static int ring_context_pin (struct intel_context *ce) {return 0 ; }drivers/gpu/drm/i915/gt/mock_engine.c +2
-2@@ -167,12 +167,12 @@ static int mock_context_alloc(struct intel_context *ce) }static int mock_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww,void **unused) +struct i915_gem_ww_ctx *ww) {return 0 ; } -static int mock_context_pin (struct intel_context *ce,void *unused) +static int mock_context_pin (struct intel_context *ce) {return 0 ; }drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +164
-19@@ -1905,6 +1905,7 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)GEM_BUG_ON (!engine->mask);GEM_BUG_ON (context_guc_id_invalid (ce)); +GEM_BUG_ON (intel_context_is_child (ce));/* * Ensure LRC + CT vmas are is same region as write barrier is done @@ -2008,15 +2009,13 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)static int __guc_context_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return lrc_pre_pin (ce, engine, ww, vaddr); +return lrc_pre_pin (ce, engine, ww); }static int __guc_context_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr) +struct intel_engine_cs *engine) {if (i915_ggtt_offset (ce->state) != (ce->lrc.lrca &CTX_GTT_ADDRESS_MASK ))@@ -2027,20 +2026,33 @@ static int __guc_context_pin(struct intel_context *ce,* explaination of why. */ -return lrc_pin (ce, engine, vaddr); +return lrc_pin (ce, engine); +} + +static void __guc_context_unpin (struct intel_context *ce) +{ +lrc_unpin (ce); +} + +static void __guc_context_post_unpin (struct intel_context *ce) +{ +lrc_post_unpin (ce); }static int guc_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return __guc_context_pre_pin (ce, ce->engine, ww, vaddr); +return __guc_context_pre_pin (ce, ce->engine, ww); } -static int guc_context_pin (struct intel_context *ce,void *vaddr) +static int guc_context_pin (struct intel_context *ce) { -int ret =__guc_context_pin (ce, ce->engine, vaddr); +int ret; +GEM_BUG_ON (intel_context_is_parent (ce) || +intel_context_is_child (ce)); + + ret =__guc_context_pin (ce, ce->engine);if (likely (!ret && !intel_context_is_barrier (ce)))intel_engine_pm_get (ce->engine);@@ -2054,7 +2066,7 @@ static void guc_context_unpin(struct intel_context *ce)GEM_BUG_ON (context_enabled (ce));unpin_guc_id (guc, ce, true); -lrc_unpin (ce); +__guc_context_unpin (ce);if (likely (!intel_context_is_barrier (ce)))intel_engine_pm_put (ce->engine);@@ -2062,7 +2074,141 @@ static void guc_context_unpin(struct intel_context *ce)static void guc_context_post_unpin (struct intel_context *ce) { -lrc_post_unpin (ce); +__guc_context_post_unpin (ce); +} + +/* Future patches will use this function */ +__maybe_unused +static int guc_parent_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +struct intel_context *child; +int err, i =0 , j =0 ; + +for_each_child (ce, child) { + err =i915_active_acquire (&child->active); +if (unlikely (err)) + goto unwind_active; + ++i; + } + +for_each_child (ce, child) { + err =__guc_context_pre_pin (child, child->engine, ww); +if (unlikely (err)) + goto unwind_pre_pin; + ++j; + } + + err =__guc_context_pre_pin (ce, ce->engine, ww); +if (unlikely (err)) + goto unwind_pre_pin; + +return 0 ; + +unwind_pre_pin: +for_each_child (ce, child) { +if (!j--) +break ; +__guc_context_post_unpin (child); + } + +unwind_active: +for_each_child (ce, child) { +if (!i--) +break ; +i915_active_release (&child->active); + } + +return err; +} + +/* Future patches will use this function */ +__maybe_unused +static void guc_parent_context_post_unpin (struct intel_context *ce) +{ +struct intel_context *child; + +for_each_child (ce, child) +__guc_context_post_unpin (child); +__guc_context_post_unpin (ce); + +for_each_child (ce, child) { +intel_context_get (child); +i915_active_release (&child->active); +intel_context_put (child); + } +} + +/* Future patches will use this function */ +__maybe_unused +static int guc_parent_context_pin (struct intel_context *ce) +{ +int ret, i =0 , j =0 ; +struct intel_context *child; +struct intel_engine_cs *engine; +intel_engine_mask_t tmp; + +GEM_BUG_ON (!intel_context_is_parent (ce)); + +for_each_child (ce, child) { + ret =__guc_context_pin (child, child->engine); +if (unlikely (ret)) + goto unwind_pin; + ++i; + } + ret =__guc_context_pin (ce, ce->engine); +if (unlikely (ret)) + goto unwind_pin; + +for_each_child (ce, child) +if (test_bit (CONTEXT_LRCA_DIRTY , &child->flags)) { +set_bit (CONTEXT_LRCA_DIRTY , &ce->flags); +break ; + } + +for_each_engine_masked (engine, ce->engine->gt, + ce->engine->mask, tmp) +intel_engine_pm_get (engine); +for_each_child (ce, child) +for_each_engine_masked (engine, child->engine->gt, + child->engine->mask, tmp) +intel_engine_pm_get (engine); + +return 0 ; + +unwind_pin: +for_each_child (ce, child) { +if (++j > i) +break ; +__guc_context_unpin (child); + } + +return ret; +} + +/* Future patches will use this function */ +__maybe_unused +static void guc_parent_context_unpin (struct intel_context *ce) +{ +struct intel_context *child; +struct intel_engine_cs *engine; +intel_engine_mask_t tmp; + +GEM_BUG_ON (!intel_context_is_parent (ce)); +GEM_BUG_ON (context_enabled (ce)); + +unpin_guc_id (ce_to_guc (ce), ce, true); +for_each_child (ce, child) +__guc_context_unpin (child); +__guc_context_unpin (ce); + +for_each_engine_masked (engine, ce->engine->gt, + ce->engine->mask, tmp) +intel_engine_pm_put (engine); +for_each_child (ce, child) +for_each_engine_masked (engine, child->engine->gt, + child->engine->mask, tmp) +intel_engine_pm_put (engine); } static void __guc_context_sched_enable(struct intel_guc *guc,@@ -2993,18 +3139,17 @@ static int guc_request_alloc(struct i915_request *rq) }static int guc_virtual_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) {struct intel_engine_cs *engine =guc_virtual_get_sibling (ce->engine,0 ); -return __guc_context_pre_pin (ce, engine, ww, vaddr); +return __guc_context_pre_pin (ce, engine, ww); } -static int guc_virtual_context_pin (struct intel_context *ce,void *vaddr) +static int guc_virtual_context_pin (struct intel_context *ce) {struct intel_engine_cs *engine =guc_virtual_get_sibling (ce->engine,0 ); -int ret =__guc_context_pin (ce, engine, vaddr); +int ret =__guc_context_pin (ce, engine);intel_engine_mask_t tmp, mask = ce->engine->mask;if (likely (!ret))@@ -3024,7 +3169,7 @@ static void guc_virtual_context_unpin(struct intel_context *ce)GEM_BUG_ON (intel_context_is_barrier (ce));unpin_guc_id (guc, ce, true); -lrc_unpin (ce); +__guc_context_unpin (ce);for_each_engine_masked (engine, ce->engine->gt, mask, tmp)intel_engine_pm_put (engine);Signature · 1 line
2.28.0Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.chSignature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [Intel-gfx] [PATCH 16/46] drm/i915/guc: Implement GuC parent-child context pin / unpin functions
Quoted 224 lines · Daniel Vetter
Quoted 223 lines · Daniel Vetter
Quoted 222 lines · Matthew Brost
Quoted 221 lines · Daniel Vetter
Quoted 220 lines · Matthew Brost
Implement GuC parent-child context pin / unpin functions in which in any contexts in the relationship are pinned all the contexts are pinned. The parent owns most of the pinning / unpinning process and the children direct any pins / unpins to the parent.
Patch implements a number of unused functions that will be connected later in the series.
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/gt/intel_context.c | 187 ++++++++++++++++-- drivers/gpu/drm/i915/gt/intel_context.h | 43 +--- drivers/gpu/drm/i915/gt/intel_context_types.h | 4 +- .../drm/i915/gt/intel_execlists_submission.c | 25 ++- drivers/gpu/drm/i915/gt/intel_lrc.c | 26 +-- drivers/gpu/drm/i915/gt/intel_lrc.h | 6 +- .../gpu/drm/i915/gt/intel_ring_submission.c | 5 +- drivers/gpu/drm/i915/gt/mock_engine.c | 4 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 183 +++++++++++++++-- 9 files changed, 371 insertions(+), 112 deletions(-)drivers/gpu/drm/i915/gt/intel_context.c +113
-18@@ -158,8 +158,8 @@ static void __ring_retire(struct intel_ring *ring)intel_ring_unpin (ring); } -static int intel_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) +static int __intel_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) {int err;@@ -190,7 +190,7 @@ static int intel_context_pre_pin(struct intel_context *ce,return err; } -static void intel_context_post_unpin (struct intel_context *ce) +static void __intel_context_post_unpin (struct intel_context *ce) {if (ce->state)__context_unpin_state (ce->state);@@ -199,13 +199,85 @@ static void intel_context_post_unpin(struct intel_context *ce)__ring_retire (ce->ring); } -int __intel_context_do_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) +static int intel_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) { -bool handoff = false; -void *vaddr; +struct intel_context *child; +int err, i =0 ; + +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) { + err =__intel_context_pre_pin (child, ww); +if (unlikely (err)) + goto unwind; + ++i; + } + + err =__intel_context_pre_pin (ce, ww); +if (unlikely (err)) + goto unwind; + +return 0 ; + +unwind: +for_each_child (ce, child) { +if (!i--) +break ; +__intel_context_post_unpin (ce); + } + +return err; +} + +static void intel_context_post_unpin (struct intel_context *ce) +{ +struct intel_context *child; + +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) +__intel_context_post_unpin (child); + +__intel_context_post_unpin (ce); +} + +static int __do_ww_lock (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +int err =i915_gem_object_lock (ce->timeline->hwsp_ggtt->obj, ww); + +if (!err && ce->ring->vma->obj) + err =i915_gem_object_lock (ce->ring->vma->obj, ww); +if (!err && ce->state) + err =i915_gem_object_lock (ce->state->obj, ww); + +return err; +} + +static int do_ww_lock (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +struct intel_context *child;int err =0 ; +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) { + err =__do_ww_lock (child, ww); +if (unlikely (err)) +return err; + } + +return __do_ww_lock (ce, ww); +} + +static int __intel_context_do_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +bool handoff = false; +int err; +if (unlikely (!test_bit (CONTEXT_ALLOC_BIT , &ce->flags))) { err =intel_context_alloc_state (ce);if (err)@@ -217,14 +289,11 @@ int __intel_context_do_pin_ww(struct intel_context *ce,* refcount for __intel_context_active(), which prevent a lock * inversion of ce->pin_mutex vs dma_resv_lock(). */ + err =do_ww_lock (ce, ww); +if (err) +return err; - err =i915_gem_object_lock (ce->timeline->hwsp_ggtt->obj, ww); -if (!err && ce->ring->vma->obj) - err =i915_gem_object_lock (ce->ring->vma->obj, ww); -if (!err && ce->state) - err =i915_gem_object_lock (ce->state->obj, ww); -if (!err) - err =intel_context_pre_pin (ce, ww); + err =intel_context_pre_pin (ce, ww);if (err)return err;@@ -232,7 +301,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,if (err) goto err_ctx_unpin; - err = ce->ops->pre_pin (ce, ww, &vaddr); + err = ce->ops->pre_pin (ce, ww);if (err) goto err_release;@@ -250,7 +319,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,if (unlikely (err)) goto err_unlock; - err = ce->ops->pin (ce, vaddr); + err = ce->ops->pin (ce);if (err) {intel_context_active_release (ce); goto err_unlock;@@ -290,7 +359,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,return err; } -int __intel_context_do_pin (struct intel_context *ce) +static int __intel_context_do_pin (struct intel_context *ce) {struct i915_gem_ww_ctx ww;int err;@@ -337,7 +406,7 @@ static void __intel_context_retire(struct i915_active *active) intel_context_get_avg_runtime_ns(ce));set_bit (CONTEXT_VALID_BIT , &ce->flags); -intel_context_post_unpin (ce); +__intel_context_post_unpin (ce);intel_context_put (ce); }@@ -562,6 +631,88 @@ void intel_context_bind_parent_child(struct intel_context *parent, child->parent = parent; } +static inline int ____intel_context_pin (struct intel_context *ce) +{ +if (likely (intel_context_pin_if_active (ce))) +return 0 ; + +return __intel_context_do_pin (ce); +} + +static inline int __intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +if (likely (intel_context_pin_if_active (ce))) +return 0 ; + +return __intel_context_do_pin_ww (ce, ww); +} + +static inline void __intel_context_unpin (struct intel_context *ce) +{ +if (!ce->ops->sched_disable) { +__intel_context_do_unpin (ce,1 ); + }else { +/* +* Move ownership of this pin to the scheduling disable which is +* an async operation. When that operation completes the above +* intel_context_sched_disable_unpin is called potentially
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
drivers/gpu/drm/i915/gt/intel_context.c void intel_context_bind_parent_child(struct intel_context *parent, · line 663
Quoted 4 lines
drivers/gpu/drm/i915/gt/intel_context.c +4
-0+* unpinning the context. +*/ +while (!atomic_add_unless (&ce->pin_count,-1 ,1 )) { +if (atomic_cmpxchg (&ce->pin_count,1 ,2 ) ==1 ) {Just as an example of what I mean here on the code review side. This is an endless loop, and you need to prove that there's no livelock or starvation issues. Or explain how else you handle that if there is one.
If we pop into the while loop the pin_count = 1, so in all likelihood the follow if evaluates to true and the loop is broken. Only way it evaluates to false is the context gets pinned between the while & if, so on the next pass the while statement should evaluate to false breaking the looping unless of course the contexts gets unpinned again... In pratice this should be at most 3 atomic operations unless the loop is broken.
Matt
Quoted 689 lines
Because unlike hand-rolled stuff linux kernel spinlocks are not dumb spinlocks, but ticketed/queued locks and therefor starvation proof. But this stuff actually matters on todays multi-core and not-so-uniform (even without fully NUMA) architectures.
Also I've just found another lockless retry loop which does actually degenerate into a full endless loop (if you're sufficiently unlucky in your races), so this really isn't academic at all.
-DanielQuoted 674 lines
Quoted 31 lines
Quoted 23 lines
Uh man lockless algorithms.
Unless this comes:
- with essentially an academic looking paper that describes the abstract model of the lockless algorithm and proves it against the linux kernel meory model.- lockless stuff generally needs barriers, and those barriers must be all documented. This means a) a comment next to each barrier in the code b) pointing to its counterparty c) with the overall design also explained in the kerneldoc for those datastructres.
If you don't know where your barriers are, see above point about "it should look more like an academic paper in the commit message"
- hard perf data about how this is absolutely required, based on a real-world use-case (which then sometimes justifies a microbenchmark metric for the details, but it always needs to be real-world based). And also a throughrough explainer how the perf issue isn't fixable through better design. If that's not doable, just protect the state machine with a big dumb lock and move on.
Quoted 7 lines
- Also, because the current code is in such bad shape wrt lockless algorithms and premature optimizations: Overall complexity should go down (it's way too high right now), so pay down your new lockless trick by removing one of the existing ones that we only have because we can.
Yes this is steep, but we're way out in the woods here and need to smoehow get back.
Quoted 7 lines
See below FIXME. At one point all of this was hidden in the backend but the dma-resv patches that landed upstream completely broke the layering, hence the need for the code here.
I guess I don't really understand what mean when you say lockless alg needs barriers, if the atomic functions are not really atomic wouldn't the world be broken?
They unordered atomics by default. Which means they're atomic itself, but entirely unordered with anything else that's going on. Except when you have one of the atomic ops which already guarantee a barrier, or you manually add the barriers yourself. And yes there's enormous amounts of bugs, and with our dgpu potentially running on non-IA cpus those bugs matter.
Note that in C++ atomics the default behaviour is strongly ordered atomics with full barriers by default, because those are much easier to program against. Kernel isn't like that and defaults to "you need to add all the barriers yourself".
I have a full lenght rant in the works and will work that through all channels, but essentially locking is really hard to get right. And lockless tricks practically need an academic paper with a formal correctness proof against the linux memory model, or you do have bugs.
And I know that the current code is choke full of this stuff, so it's tempting to just add more, but we really cant. The amount of locking trickery we have in the codebase must go down substantially. My take is that any code that adds anything trick needs to fully justify it against the above list, _and_ also clean up some of the existing nonsense so that overall complexity doesn't increase.
I'll share the full length rant with you internally, it's not yet ready for publishing (but that's planned too).
Quoted 4 lines
Also here I don't think it is really as simple as grab big dump lock for a variety of reasons, at least with the current dynamic pin / unpin code in place. If we move a perma-pinned contexts this could be cleaned up then.
Yes it's a disaster, but we need to stop the bleeding. If perma-pinned context can fix this I think we should do this asap. I'd say for parallel context we should just do it outright (special case them or whatever) so that we don't have to add even more very tricky code and tech debt.
Doable?
Cheers, Daniel
Quoted 587 lines
Matt
Quoted 584 lines
-Daniel
Quoted 577 lines
drivers/gpu/drm/i915/gt/intel_context.c +52
-0+ ce->ops->sched_disable (ce); +break ; + } + } + } +} + +/* +* FIXME: This is ugly, these branches are only needed for parallel contexts in +* GuC submission. Basically the idea is if any of the contexts, that are +* configured for parallel submission, are pinned all the contexts need to be +* pinned in order to register these contexts with the GuC. We are adding the +* layer here while it should probably be pushed to the backend via a vfunc. But +* since we already have ce->pin + a layer atop it is confusing. Definitely +* needs a bit of rework how to properly layer / structure this code path. What +* is in place works but is not ideal. +*/ +int intel_context_pin (struct intel_context *ce) +{ +if (intel_context_is_child (ce)) { +if (!atomic_fetch_add (1 , &ce->pin_count)) +return ____intel_context_pin (ce->parent); +else +return 0 ; + }else { +return ____intel_context_pin (ce); + } +} + +int intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +if (intel_context_is_child (ce)) { +if (!atomic_fetch_add (1 , &ce->pin_count)) +return __intel_context_pin_ww (ce->parent, ww); +else +return 0 ; + }else { +return __intel_context_pin_ww (ce, ww); + } +} + +void intel_context_unpin (struct intel_context *ce) +{ +if (intel_context_is_child (ce)) { +if (atomic_fetch_add (-1 , &ce->pin_count) ==1 ) +__intel_context_unpin (ce->parent); + }else { +__intel_context_unpin (ce); + } +} +#if IS_ENABLED (CONFIG_DRM_I915_SELFTEST )#include "selftest_context.c" #endif drivers/gpu/drm/i915/gt/intel_context.h +5
-38@@ -110,31 +110,15 @@ static inline void intel_context_unlock_pinned(struct intel_context *ce)mutex_unlock (&ce->pin_mutex); } -int __intel_context_do_pin (struct intel_context *ce); -int __intel_context_do_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww); -static inline bool intel_context_pin_if_active (struct intel_context *ce) {return atomic_inc_not_zero (&ce->pin_count); } -static inline int intel_context_pin (struct intel_context *ce) -{ -if (likely (intel_context_pin_if_active (ce))) -return 0 ; - -return __intel_context_do_pin (ce); -} - -static inline int intel_context_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) -{ -if (likely (intel_context_pin_if_active (ce))) -return 0 ; +int intel_context_pin (struct intel_context *ce); -return __intel_context_do_pin_ww (ce, ww); -} +int intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww);static inline void __intel_context_pin (struct intel_context *ce) {@@ -146,28 +130,11 @@ void __intel_context_do_unpin(struct intel_context *ce, int sub);static inline void intel_context_sched_disable_unpin (struct intel_context *ce) { +GEM_BUG_ON (intel_context_is_child (ce));__intel_context_do_unpin (ce,2 ); } -static inline void intel_context_unpin (struct intel_context *ce) -{ -if (!ce->ops->sched_disable) { -__intel_context_do_unpin (ce,1 ); - }else { -/* -* Move ownership of this pin to the scheduling disable which is -* an async operation. When that operation completes the above -* intel_context_sched_disable_unpin is called potentially -* unpinning the context. -*/ -while (!atomic_add_unless (&ce->pin_count,-1 ,1 )) { -if (atomic_cmpxchg (&ce->pin_count,1 ,2 ) ==1 ) { - ce->ops->sched_disable (ce); -break ; - } - } - } -} +void intel_context_unpin (struct intel_context *ce);void intel_context_enter_engine (struct intel_context *ce);void intel_context_exit_engine (struct intel_context *ce);drivers/gpu/drm/i915/gt/intel_context_types.h +2
-2@@ -39,8 +39,8 @@ struct intel_context_ops {void (*ban)(struct intel_context *ce,struct i915_request *rq); -int (*pre_pin)(struct intel_context *ce,struct i915_gem_ww_ctx *ww,void **vaddr); -int (*pin)(struct intel_context *ce,void *vaddr); +int (*pre_pin)(struct intel_context *ce,struct i915_gem_ww_ctx *ww); +int (*pin)(struct intel_context *ce);void (*unpin)(struct intel_context *ce);void (*post_unpin)(struct intel_context *ce);drivers/gpu/drm/i915/gt/intel_execlists_submission.c +12
-13@@ -2554,16 +2554,17 @@ static void execlists_submit_request(struct i915_request *request)static int __execlists_context_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww,void **vaddr) +struct i915_gem_ww_ctx *ww) {int err; - err =lrc_pre_pin (ce, engine, ww, vaddr); + err =lrc_pre_pin (ce, engine, ww);if (err)return err;if (!__test_and_set_bit (CONTEXT_INIT_BIT , &ce->flags)) { -lrc_init_state (ce, engine, *vaddr); +lrc_init_state (ce, engine, ce->lrc_reg_state - +LRC_STATE_OFFSET /sizeof (*ce->lrc_reg_state));__i915_gem_object_flush_map (ce->state->obj,0 , engine->context_size); }@@ -2572,15 +2573,14 @@ __execlists_context_pre_pin(struct intel_context *ce, }static int execlists_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return __execlists_context_pre_pin (ce, ce->engine, ww, vaddr); +return __execlists_context_pre_pin (ce, ce->engine, ww); } -static int execlists_context_pin (struct intel_context *ce,void *vaddr) +static int execlists_context_pin (struct intel_context *ce) { -return lrc_pin (ce, ce->engine, vaddr); +return lrc_pin (ce, ce->engine); }static int execlists_context_alloc (struct intel_context *ce)@@ -3570,20 +3570,19 @@ static int virtual_context_alloc(struct intel_context *ce) }static int virtual_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) {struct virtual_engine *ve =container_of (ce,typeof (*ve), context);/* Note: we must use a real engine class for setting up reg state */ -return __execlists_context_pre_pin (ce, ve->siblings[0 ], ww, vaddr); +return __execlists_context_pre_pin (ce, ve->siblings[0 ], ww); } -static int virtual_context_pin (struct intel_context *ce,void *vaddr) +static int virtual_context_pin (struct intel_context *ce) {struct virtual_engine *ve =container_of (ce,typeof (*ve), context); -return lrc_pin (ce, ve->siblings[0 ], vaddr); +return lrc_pin (ce, ve->siblings[0 ]); }static void virtual_context_enter (struct intel_context *ce)drivers/gpu/drm/i915/gt/intel_lrc.c +13
-13@@ -947,30 +947,30 @@ void lrc_reset(struct intel_context *ce)int lrc_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { +void *vaddr;GEM_BUG_ON (!ce->state);GEM_BUG_ON (!i915_vma_is_pinned (ce->state)); - *vaddr =i915_gem_object_pin_map (ce->state->obj, -i915_coherent_map_type (ce->engine->i915, - ce->state->obj, - false) | -I915_MAP_OVERRIDE ); + vaddr =i915_gem_object_pin_map (ce->state->obj, +i915_coherent_map_type (ce->engine->i915, + ce->state->obj, + false) | +I915_MAP_OVERRIDE ); -return PTR_ERR_OR_ZERO (*vaddr); + ce->lrc_reg_state = vaddr +LRC_STATE_OFFSET ; + +return PTR_ERR_OR_ZERO (vaddr); }int lrc_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr) +struct intel_engine_cs *engine) { - ce->lrc_reg_state = vaddr +LRC_STATE_OFFSET ; -if (!__test_and_set_bit (CONTEXT_INIT_BIT , &ce->flags)) -lrc_init_state (ce, engine, vaddr); +lrc_init_state (ce, engine, + (void *)ce->lrc_reg_state -LRC_STATE_OFFSET ); ce->lrc.lrca =lrc_update_regs (ce, engine, ce->ring->tail);return 0 ;drivers/gpu/drm/i915/gt/intel_lrc.h +2
-4@@ -38,12 +38,10 @@ void lrc_destroy(struct kref *kref);int lrc_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr); +struct i915_gem_ww_ctx *ww);int lrc_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr); +struct intel_engine_cs *engine);void lrc_unpin (struct intel_context *ce);void lrc_post_unpin (struct intel_context *ce);drivers/gpu/drm/i915/gt/intel_ring_submission.c +2
-3@@ -472,8 +472,7 @@ static int ring_context_init_default_state(struct intel_context *ce, }static int ring_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **unused) +struct i915_gem_ww_ctx *ww) {struct i915_address_space *vm;int err =0 ;@@ -576,7 +575,7 @@ static int ring_context_alloc(struct intel_context *ce)return 0 ; } -static int ring_context_pin (struct intel_context *ce,void *unused) +static int ring_context_pin (struct intel_context *ce) {return 0 ; }drivers/gpu/drm/i915/gt/mock_engine.c +2
-2@@ -167,12 +167,12 @@ static int mock_context_alloc(struct intel_context *ce) }static int mock_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww,void **unused) +struct i915_gem_ww_ctx *ww) {return 0 ; } -static int mock_context_pin (struct intel_context *ce,void *unused) +static int mock_context_pin (struct intel_context *ce) {return 0 ; }drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +164
-19@@ -1905,6 +1905,7 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)GEM_BUG_ON (!engine->mask);GEM_BUG_ON (context_guc_id_invalid (ce)); +GEM_BUG_ON (intel_context_is_child (ce));/* * Ensure LRC + CT vmas are is same region as write barrier is done @@ -2008,15 +2009,13 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)static int __guc_context_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return lrc_pre_pin (ce, engine, ww, vaddr); +return lrc_pre_pin (ce, engine, ww); }static int __guc_context_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr) +struct intel_engine_cs *engine) {if (i915_ggtt_offset (ce->state) != (ce->lrc.lrca &CTX_GTT_ADDRESS_MASK ))@@ -2027,20 +2026,33 @@ static int __guc_context_pin(struct intel_context *ce,* explaination of why. */ -return lrc_pin (ce, engine, vaddr); +return lrc_pin (ce, engine); +} + +static void __guc_context_unpin (struct intel_context *ce) +{ +lrc_unpin (ce); +} + +static void __guc_context_post_unpin (struct intel_context *ce) +{ +lrc_post_unpin (ce); }static int guc_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return __guc_context_pre_pin (ce, ce->engine, ww, vaddr); +return __guc_context_pre_pin (ce, ce->engine, ww); } -static int guc_context_pin (struct intel_context *ce,void *vaddr) +static int guc_context_pin (struct intel_context *ce) { -int ret =__guc_context_pin (ce, ce->engine, vaddr); +int ret; +GEM_BUG_ON (intel_context_is_parent (ce) || +intel_context_is_child (ce)); + + ret =__guc_context_pin (ce, ce->engine);if (likely (!ret && !intel_context_is_barrier (ce)))intel_engine_pm_get (ce->engine);@@ -2054,7 +2066,7 @@ static void guc_context_unpin(struct intel_context *ce)GEM_BUG_ON (context_enabled (ce));unpin_guc_id (guc, ce, true); -lrc_unpin (ce); +__guc_context_unpin (ce);if (likely (!intel_context_is_barrier (ce)))intel_engine_pm_put (ce->engine);@@ -2062,7 +2074,141 @@ static void guc_context_unpin(struct intel_context *ce)static void guc_context_post_unpin (struct intel_context *ce) { -lrc_post_unpin (ce); +__guc_context_post_unpin (ce); +} + +/* Future patches will use this function */ +__maybe_unused +static int guc_parent_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +struct intel_context *child; +int err, i =0 , j =0 ; + +for_each_child (ce, child) { + err =i915_active_acquire (&child->active); +if (unlikely (err)) + goto unwind_active; + ++i; + } + +for_each_child (ce, child) { + err =__guc_context_pre_pin (child, child->engine, ww); +if (unlikely (err)) + goto unwind_pre_pin; + ++j; + } + + err =__guc_context_pre_pin (ce, ce->engine, ww); +if (unlikely (err)) + goto unwind_pre_pin; + +return 0 ; + +unwind_pre_pin: +for_each_child (ce, child) { +if (!j--) +break ; +__guc_context_post_unpin (child); + } + +unwind_active: +for_each_child (ce, child) { +if (!i--) +break ; +i915_active_release (&child->active); + } + +return err; +} + +/* Future patches will use this function */ +__maybe_unused +static void guc_parent_context_post_unpin (struct intel_context *ce) +{ +struct intel_context *child; + +for_each_child (ce, child) +__guc_context_post_unpin (child); +__guc_context_post_unpin (ce); + +for_each_child (ce, child) { +intel_context_get (child); +i915_active_release (&child->active); +intel_context_put (child); + } +} + +/* Future patches will use this function */ +__maybe_unused +static int guc_parent_context_pin (struct intel_context *ce) +{ +int ret, i =0 , j =0 ; +struct intel_context *child; +struct intel_engine_cs *engine; +intel_engine_mask_t tmp; + +GEM_BUG_ON (!intel_context_is_parent (ce)); + +for_each_child (ce, child) { + ret =__guc_context_pin (child, child->engine); +if (unlikely (ret)) + goto unwind_pin; + ++i; + } + ret =__guc_context_pin (ce, ce->engine); +if (unlikely (ret)) + goto unwind_pin; + +for_each_child (ce, child) +if (test_bit (CONTEXT_LRCA_DIRTY , &child->flags)) { +set_bit (CONTEXT_LRCA_DIRTY , &ce->flags); +break ; + } + +for_each_engine_masked (engine, ce->engine->gt, + ce->engine->mask, tmp) +intel_engine_pm_get (engine); +for_each_child (ce, child) +for_each_engine_masked (engine, child->engine->gt, + child->engine->mask, tmp) +intel_engine_pm_get (engine); + +return 0 ; + +unwind_pin: +for_each_child (ce, child) { +if (++j > i) +break ; +__guc_context_unpin (child); + } + +return ret; +} + +/* Future patches will use this function */ +__maybe_unused +static void guc_parent_context_unpin (struct intel_context *ce) +{ +struct intel_context *child; +struct intel_engine_cs *engine; +intel_engine_mask_t tmp; + +GEM_BUG_ON (!intel_context_is_parent (ce)); +GEM_BUG_ON (context_enabled (ce)); + +unpin_guc_id (ce_to_guc (ce), ce, true); +for_each_child (ce, child) +__guc_context_unpin (child); +__guc_context_unpin (ce); + +for_each_engine_masked (engine, ce->engine->gt, + ce->engine->mask, tmp) +intel_engine_pm_put (engine); +for_each_child (ce, child) +for_each_engine_masked (engine, child->engine->gt, + child->engine->mask, tmp) +intel_engine_pm_put (engine); } static void __guc_context_sched_enable(struct intel_guc *guc,@@ -2993,18 +3139,17 @@ static int guc_request_alloc(struct i915_request *rq) }static int guc_virtual_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) {struct intel_engine_cs *engine =guc_virtual_get_sibling (ce->engine,0 ); -return __guc_context_pre_pin (ce, engine, ww, vaddr); +return __guc_context_pre_pin (ce, engine, ww); } -static int guc_virtual_context_pin (struct intel_context *ce,void *vaddr) +static int guc_virtual_context_pin (struct intel_context *ce) {struct intel_engine_cs *engine =guc_virtual_get_sibling (ce->engine,0 ); -int ret =__guc_context_pin (ce, engine, vaddr); +int ret =__guc_context_pin (ce, engine);intel_engine_mask_t tmp, mask = ce->engine->mask;if (likely (!ret))@@ -3024,7 +3169,7 @@ static void guc_virtual_context_unpin(struct intel_context *ce)GEM_BUG_ON (intel_context_is_barrier (ce));unpin_guc_id (guc, ce, true); -lrc_unpin (ce); +__guc_context_unpin (ce);for_each_engine_masked (engine, ce->engine->gt, mask, tmp)intel_engine_pm_put (engine);Signature · 1 line
2.28.0Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.chSignature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.chSignature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [Intel-gfx] [PATCH 16/46] drm/i915/guc: Implement GuC parent-child context pin / unpin functions
Quoted 235 lines · Matthew Brost
Quoted 224 lines · Daniel Vetter
Quoted 223 lines · Daniel Vetter
Quoted 222 lines · Matthew Brost
Quoted 221 lines · Daniel Vetter
Quoted 220 lines · Matthew Brost
Implement GuC parent-child context pin / unpin functions in which in any contexts in the relationship are pinned all the contexts are pinned. The parent owns most of the pinning / unpinning process and the children direct any pins / unpins to the parent.
Patch implements a number of unused functions that will be connected later in the series.
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/gt/intel_context.c | 187 ++++++++++++++++-- drivers/gpu/drm/i915/gt/intel_context.h | 43 +--- drivers/gpu/drm/i915/gt/intel_context_types.h | 4 +- .../drm/i915/gt/intel_execlists_submission.c | 25 ++- drivers/gpu/drm/i915/gt/intel_lrc.c | 26 +-- drivers/gpu/drm/i915/gt/intel_lrc.h | 6 +- .../gpu/drm/i915/gt/intel_ring_submission.c | 5 +- drivers/gpu/drm/i915/gt/mock_engine.c | 4 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 183 +++++++++++++++-- 9 files changed, 371 insertions(+), 112 deletions(-)drivers/gpu/drm/i915/gt/intel_context.c +113
-18@@ -158,8 +158,8 @@ static void __ring_retire(struct intel_ring *ring)intel_ring_unpin (ring); } -static int intel_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) +static int __intel_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) {int err;@@ -190,7 +190,7 @@ static int intel_context_pre_pin(struct intel_context *ce,return err; } -static void intel_context_post_unpin (struct intel_context *ce) +static void __intel_context_post_unpin (struct intel_context *ce) {if (ce->state)__context_unpin_state (ce->state);@@ -199,13 +199,85 @@ static void intel_context_post_unpin(struct intel_context *ce)__ring_retire (ce->ring); } -int __intel_context_do_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) +static int intel_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) { -bool handoff = false; -void *vaddr; +struct intel_context *child; +int err, i =0 ; + +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) { + err =__intel_context_pre_pin (child, ww); +if (unlikely (err)) + goto unwind; + ++i; + } + + err =__intel_context_pre_pin (ce, ww); +if (unlikely (err)) + goto unwind; + +return 0 ; + +unwind: +for_each_child (ce, child) { +if (!i--) +break ; +__intel_context_post_unpin (ce); + } + +return err; +} + +static void intel_context_post_unpin (struct intel_context *ce) +{ +struct intel_context *child; + +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) +__intel_context_post_unpin (child); + +__intel_context_post_unpin (ce); +} + +static int __do_ww_lock (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +int err =i915_gem_object_lock (ce->timeline->hwsp_ggtt->obj, ww); + +if (!err && ce->ring->vma->obj) + err =i915_gem_object_lock (ce->ring->vma->obj, ww); +if (!err && ce->state) + err =i915_gem_object_lock (ce->state->obj, ww); + +return err; +} + +static int do_ww_lock (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +struct intel_context *child;int err =0 ; +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) { + err =__do_ww_lock (child, ww); +if (unlikely (err)) +return err; + } + +return __do_ww_lock (ce, ww); +} + +static int __intel_context_do_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +bool handoff = false; +int err; +if (unlikely (!test_bit (CONTEXT_ALLOC_BIT , &ce->flags))) { err =intel_context_alloc_state (ce);if (err)@@ -217,14 +289,11 @@ int __intel_context_do_pin_ww(struct intel_context *ce,* refcount for __intel_context_active(), which prevent a lock * inversion of ce->pin_mutex vs dma_resv_lock(). */ + err =do_ww_lock (ce, ww); +if (err) +return err; - err =i915_gem_object_lock (ce->timeline->hwsp_ggtt->obj, ww); -if (!err && ce->ring->vma->obj) - err =i915_gem_object_lock (ce->ring->vma->obj, ww); -if (!err && ce->state) - err =i915_gem_object_lock (ce->state->obj, ww); -if (!err) - err =intel_context_pre_pin (ce, ww); + err =intel_context_pre_pin (ce, ww);if (err)return err;@@ -232,7 +301,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,if (err) goto err_ctx_unpin; - err = ce->ops->pre_pin (ce, ww, &vaddr); + err = ce->ops->pre_pin (ce, ww);if (err) goto err_release;@@ -250,7 +319,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,if (unlikely (err)) goto err_unlock; - err = ce->ops->pin (ce, vaddr); + err = ce->ops->pin (ce);if (err) {intel_context_active_release (ce); goto err_unlock;@@ -290,7 +359,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,return err; } -int __intel_context_do_pin (struct intel_context *ce) +static int __intel_context_do_pin (struct intel_context *ce) {struct i915_gem_ww_ctx ww;int err;@@ -337,7 +406,7 @@ static void __intel_context_retire(struct i915_active *active) intel_context_get_avg_runtime_ns(ce));set_bit (CONTEXT_VALID_BIT , &ce->flags); -intel_context_post_unpin (ce); +__intel_context_post_unpin (ce);intel_context_put (ce); }@@ -562,6 +631,88 @@ void intel_context_bind_parent_child(struct intel_context *parent, child->parent = parent; } +static inline int ____intel_context_pin (struct intel_context *ce) +{ +if (likely (intel_context_pin_if_active (ce))) +return 0 ; + +return __intel_context_do_pin (ce); +} + +static inline int __intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +if (likely (intel_context_pin_if_active (ce))) +return 0 ; + +return __intel_context_do_pin_ww (ce, ww); +} + +static inline void __intel_context_unpin (struct intel_context *ce) +{ +if (!ce->ops->sched_disable) { +__intel_context_do_unpin (ce,1 ); + }else { +/* +* Move ownership of this pin to the scheduling disable which is +* an async operation. When that operation completes the above +* intel_context_sched_disable_unpin is called potentially Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
drivers/gpu/drm/i915/gt/intel_context.c void intel_context_bind_parent_child(struct intel_context *parent, · line 663
Quoted 4 lines
drivers/gpu/drm/i915/gt/intel_context.c +4
-0+* unpinning the context. +*/ +while (!atomic_add_unless (&ce->pin_count,-1 ,1 )) { +if (atomic_cmpxchg (&ce->pin_count,1 ,2 ) ==1 ) {Just as an example of what I mean here on the code review side. This is an endless loop, and you need to prove that there's no livelock or starvation issues. Or explain how else you handle that if there is one.
Quoted 7 lines
If we pop into the while loop the
pin_count= 1, so in all likelihood the follow if evaluates to true and the loop is broken. Only way it evaluates to false is the context gets pinned between the while & if, so on the next pass the while statement should evaluate to false breaking the looping unless of course the contexts gets unpinned again... In pratice this should be at most 3 atomic operations unless the loop is broken.
That's not really how this works ... E.g. the linux spinlocks are ticketed/queued locks, exactly because the "this likely doesn't happen" argument is not a very good one.
Can't we at least convert intel_context->pin_count into a normal counter with a spinlock around it and then just make all this stuff a lot more reasonable?
Yes this doesn't help the situation overall much, but at least we're not spreading dubious hand-roll locking patterns all over the place for not much good reasons.
Hand-crafted artisanal locking is never a quality sign. -Daniel
Quoted 692 lines
Matt
Quoted 689 lines
Because unlike hand-rolled stuff linux kernel spinlocks are not dumb spinlocks, but ticketed/queued locks and therefor starvation proof. But this stuff actually matters on todays multi-core and not-so-uniform (even without fully NUMA) architectures.
Also I've just found another lockless retry loop which does actually degenerate into a full endless loop (if you're sufficiently unlucky in your races), so this really isn't academic at all.
-DanielQuoted 674 lines
Quoted 31 lines
Quoted 23 lines
Uh man lockless algorithms.
Unless this comes:
- with essentially an academic looking paper that describes the abstract model of the lockless algorithm and proves it against the linux kernel meory model.- lockless stuff generally needs barriers, and those barriers must be all documented. This means a) a comment next to each barrier in the code b) pointing to its counterparty c) with the overall design also explained in the kerneldoc for those datastructres.
If you don't know where your barriers are, see above point about "it should look more like an academic paper in the commit message"
- hard perf data about how this is absolutely required, based on a real-world use-case (which then sometimes justifies a microbenchmark metric for the details, but it always needs to be real-world based). And also a throughrough explainer how the perf issue isn't fixable through better design. If that's not doable, just protect the state machine with a big dumb lock and move on.
Quoted 7 lines
- Also, because the current code is in such bad shape wrt lockless algorithms and premature optimizations: Overall complexity should go down (it's way too high right now), so pay down your new lockless trick by removing one of the existing ones that we only have because we can.
Yes this is steep, but we're way out in the woods here and need to smoehow get back.
Quoted 7 lines
See below FIXME. At one point all of this was hidden in the backend but the dma-resv patches that landed upstream completely broke the layering, hence the need for the code here.
I guess I don't really understand what mean when you say lockless alg needs barriers, if the atomic functions are not really atomic wouldn't the world be broken?
They unordered atomics by default. Which means they're atomic itself, but entirely unordered with anything else that's going on. Except when you have one of the atomic ops which already guarantee a barrier, or you manually add the barriers yourself. And yes there's enormous amounts of bugs, and with our dgpu potentially running on non-IA cpus those bugs matter.
Note that in C++ atomics the default behaviour is strongly ordered atomics with full barriers by default, because those are much easier to program against. Kernel isn't like that and defaults to "you need to add all the barriers yourself".
I have a full lenght rant in the works and will work that through all channels, but essentially locking is really hard to get right. And lockless tricks practically need an academic paper with a formal correctness proof against the linux memory model, or you do have bugs.
And I know that the current code is choke full of this stuff, so it's tempting to just add more, but we really cant. The amount of locking trickery we have in the codebase must go down substantially. My take is that any code that adds anything trick needs to fully justify it against the above list, _and_ also clean up some of the existing nonsense so that overall complexity doesn't increase.
I'll share the full length rant with you internally, it's not yet ready for publishing (but that's planned too).
Quoted 4 lines
Also here I don't think it is really as simple as grab big dump lock for a variety of reasons, at least with the current dynamic pin / unpin code in place. If we move a perma-pinned contexts this could be cleaned up then.
Yes it's a disaster, but we need to stop the bleeding. If perma-pinned context can fix this I think we should do this asap. I'd say for parallel context we should just do it outright (special case them or whatever) so that we don't have to add even more very tricky code and tech debt.
Doable?
Cheers, Daniel
Quoted 587 lines
Matt
Quoted 584 lines
-Daniel
Quoted 577 lines
drivers/gpu/drm/i915/gt/intel_context.c +52
-0+ ce->ops->sched_disable (ce); +break ; + } + } + } +} + +/* +* FIXME: This is ugly, these branches are only needed for parallel contexts in +* GuC submission. Basically the idea is if any of the contexts, that are +* configured for parallel submission, are pinned all the contexts need to be +* pinned in order to register these contexts with the GuC. We are adding the +* layer here while it should probably be pushed to the backend via a vfunc. But +* since we already have ce->pin + a layer atop it is confusing. Definitely +* needs a bit of rework how to properly layer / structure this code path. What +* is in place works but is not ideal. +*/ +int intel_context_pin (struct intel_context *ce) +{ +if (intel_context_is_child (ce)) { +if (!atomic_fetch_add (1 , &ce->pin_count)) +return ____intel_context_pin (ce->parent); +else +return 0 ; + }else { +return ____intel_context_pin (ce); + } +} + +int intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +if (intel_context_is_child (ce)) { +if (!atomic_fetch_add (1 , &ce->pin_count)) +return __intel_context_pin_ww (ce->parent, ww); +else +return 0 ; + }else { +return __intel_context_pin_ww (ce, ww); + } +} + +void intel_context_unpin (struct intel_context *ce) +{ +if (intel_context_is_child (ce)) { +if (atomic_fetch_add (-1 , &ce->pin_count) ==1 ) +__intel_context_unpin (ce->parent); + }else { +__intel_context_unpin (ce); + } +} +#if IS_ENABLED (CONFIG_DRM_I915_SELFTEST )#include "selftest_context.c" #endif drivers/gpu/drm/i915/gt/intel_context.h +5
-38@@ -110,31 +110,15 @@ static inline void intel_context_unlock_pinned(struct intel_context *ce)mutex_unlock (&ce->pin_mutex); } -int __intel_context_do_pin (struct intel_context *ce); -int __intel_context_do_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww); -static inline bool intel_context_pin_if_active (struct intel_context *ce) {return atomic_inc_not_zero (&ce->pin_count); } -static inline int intel_context_pin (struct intel_context *ce) -{ -if (likely (intel_context_pin_if_active (ce))) -return 0 ; - -return __intel_context_do_pin (ce); -} - -static inline int intel_context_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) -{ -if (likely (intel_context_pin_if_active (ce))) -return 0 ; +int intel_context_pin (struct intel_context *ce); -return __intel_context_do_pin_ww (ce, ww); -} +int intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww);static inline void __intel_context_pin (struct intel_context *ce) {@@ -146,28 +130,11 @@ void __intel_context_do_unpin(struct intel_context *ce, int sub);static inline void intel_context_sched_disable_unpin (struct intel_context *ce) { +GEM_BUG_ON (intel_context_is_child (ce));__intel_context_do_unpin (ce,2 ); } -static inline void intel_context_unpin (struct intel_context *ce) -{ -if (!ce->ops->sched_disable) { -__intel_context_do_unpin (ce,1 ); - }else { -/* -* Move ownership of this pin to the scheduling disable which is -* an async operation. When that operation completes the above -* intel_context_sched_disable_unpin is called potentially -* unpinning the context. -*/ -while (!atomic_add_unless (&ce->pin_count,-1 ,1 )) { -if (atomic_cmpxchg (&ce->pin_count,1 ,2 ) ==1 ) { - ce->ops->sched_disable (ce); -break ; - } - } - } -} +void intel_context_unpin (struct intel_context *ce);void intel_context_enter_engine (struct intel_context *ce);void intel_context_exit_engine (struct intel_context *ce);drivers/gpu/drm/i915/gt/intel_context_types.h +2
-2@@ -39,8 +39,8 @@ struct intel_context_ops {void (*ban)(struct intel_context *ce,struct i915_request *rq); -int (*pre_pin)(struct intel_context *ce,struct i915_gem_ww_ctx *ww,void **vaddr); -int (*pin)(struct intel_context *ce,void *vaddr); +int (*pre_pin)(struct intel_context *ce,struct i915_gem_ww_ctx *ww); +int (*pin)(struct intel_context *ce);void (*unpin)(struct intel_context *ce);void (*post_unpin)(struct intel_context *ce);drivers/gpu/drm/i915/gt/intel_execlists_submission.c +12
-13@@ -2554,16 +2554,17 @@ static void execlists_submit_request(struct i915_request *request)static int __execlists_context_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww,void **vaddr) +struct i915_gem_ww_ctx *ww) {int err; - err =lrc_pre_pin (ce, engine, ww, vaddr); + err =lrc_pre_pin (ce, engine, ww);if (err)return err;if (!__test_and_set_bit (CONTEXT_INIT_BIT , &ce->flags)) { -lrc_init_state (ce, engine, *vaddr); +lrc_init_state (ce, engine, ce->lrc_reg_state - +LRC_STATE_OFFSET /sizeof (*ce->lrc_reg_state));__i915_gem_object_flush_map (ce->state->obj,0 , engine->context_size); }@@ -2572,15 +2573,14 @@ __execlists_context_pre_pin(struct intel_context *ce, }static int execlists_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return __execlists_context_pre_pin (ce, ce->engine, ww, vaddr); +return __execlists_context_pre_pin (ce, ce->engine, ww); } -static int execlists_context_pin (struct intel_context *ce,void *vaddr) +static int execlists_context_pin (struct intel_context *ce) { -return lrc_pin (ce, ce->engine, vaddr); +return lrc_pin (ce, ce->engine); }static int execlists_context_alloc (struct intel_context *ce)@@ -3570,20 +3570,19 @@ static int virtual_context_alloc(struct intel_context *ce) }static int virtual_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) {struct virtual_engine *ve =container_of (ce,typeof (*ve), context);/* Note: we must use a real engine class for setting up reg state */ -return __execlists_context_pre_pin (ce, ve->siblings[0 ], ww, vaddr); +return __execlists_context_pre_pin (ce, ve->siblings[0 ], ww); } -static int virtual_context_pin (struct intel_context *ce,void *vaddr) +static int virtual_context_pin (struct intel_context *ce) {struct virtual_engine *ve =container_of (ce,typeof (*ve), context); -return lrc_pin (ce, ve->siblings[0 ], vaddr); +return lrc_pin (ce, ve->siblings[0 ]); }static void virtual_context_enter (struct intel_context *ce)drivers/gpu/drm/i915/gt/intel_lrc.c +13
-13@@ -947,30 +947,30 @@ void lrc_reset(struct intel_context *ce)int lrc_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { +void *vaddr;GEM_BUG_ON (!ce->state);GEM_BUG_ON (!i915_vma_is_pinned (ce->state)); - *vaddr =i915_gem_object_pin_map (ce->state->obj, -i915_coherent_map_type (ce->engine->i915, - ce->state->obj, - false) | -I915_MAP_OVERRIDE ); + vaddr =i915_gem_object_pin_map (ce->state->obj, +i915_coherent_map_type (ce->engine->i915, + ce->state->obj, + false) | +I915_MAP_OVERRIDE ); -return PTR_ERR_OR_ZERO (*vaddr); + ce->lrc_reg_state = vaddr +LRC_STATE_OFFSET ; + +return PTR_ERR_OR_ZERO (vaddr); }int lrc_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr) +struct intel_engine_cs *engine) { - ce->lrc_reg_state = vaddr +LRC_STATE_OFFSET ; -if (!__test_and_set_bit (CONTEXT_INIT_BIT , &ce->flags)) -lrc_init_state (ce, engine, vaddr); +lrc_init_state (ce, engine, + (void *)ce->lrc_reg_state -LRC_STATE_OFFSET ); ce->lrc.lrca =lrc_update_regs (ce, engine, ce->ring->tail);return 0 ;drivers/gpu/drm/i915/gt/intel_lrc.h +2
-4@@ -38,12 +38,10 @@ void lrc_destroy(struct kref *kref);int lrc_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr); +struct i915_gem_ww_ctx *ww);int lrc_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr); +struct intel_engine_cs *engine);void lrc_unpin (struct intel_context *ce);void lrc_post_unpin (struct intel_context *ce);drivers/gpu/drm/i915/gt/intel_ring_submission.c +2
-3@@ -472,8 +472,7 @@ static int ring_context_init_default_state(struct intel_context *ce, }static int ring_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **unused) +struct i915_gem_ww_ctx *ww) {struct i915_address_space *vm;int err =0 ;@@ -576,7 +575,7 @@ static int ring_context_alloc(struct intel_context *ce)return 0 ; } -static int ring_context_pin (struct intel_context *ce,void *unused) +static int ring_context_pin (struct intel_context *ce) {return 0 ; }drivers/gpu/drm/i915/gt/mock_engine.c +2
-2@@ -167,12 +167,12 @@ static int mock_context_alloc(struct intel_context *ce) }static int mock_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww,void **unused) +struct i915_gem_ww_ctx *ww) {return 0 ; } -static int mock_context_pin (struct intel_context *ce,void *unused) +static int mock_context_pin (struct intel_context *ce) {return 0 ; }drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +164
-19@@ -1905,6 +1905,7 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)GEM_BUG_ON (!engine->mask);GEM_BUG_ON (context_guc_id_invalid (ce)); +GEM_BUG_ON (intel_context_is_child (ce));/* * Ensure LRC + CT vmas are is same region as write barrier is done @@ -2008,15 +2009,13 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)static int __guc_context_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return lrc_pre_pin (ce, engine, ww, vaddr); +return lrc_pre_pin (ce, engine, ww); }static int __guc_context_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr) +struct intel_engine_cs *engine) {if (i915_ggtt_offset (ce->state) != (ce->lrc.lrca &CTX_GTT_ADDRESS_MASK ))@@ -2027,20 +2026,33 @@ static int __guc_context_pin(struct intel_context *ce,* explaination of why. */ -return lrc_pin (ce, engine, vaddr); +return lrc_pin (ce, engine); +} + +static void __guc_context_unpin (struct intel_context *ce) +{ +lrc_unpin (ce); +} + +static void __guc_context_post_unpin (struct intel_context *ce) +{ +lrc_post_unpin (ce); }static int guc_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return __guc_context_pre_pin (ce, ce->engine, ww, vaddr); +return __guc_context_pre_pin (ce, ce->engine, ww); } -static int guc_context_pin (struct intel_context *ce,void *vaddr) +static int guc_context_pin (struct intel_context *ce) { -int ret =__guc_context_pin (ce, ce->engine, vaddr); +int ret; +GEM_BUG_ON (intel_context_is_parent (ce) || +intel_context_is_child (ce)); + + ret =__guc_context_pin (ce, ce->engine);if (likely (!ret && !intel_context_is_barrier (ce)))intel_engine_pm_get (ce->engine);@@ -2054,7 +2066,7 @@ static void guc_context_unpin(struct intel_context *ce)GEM_BUG_ON (context_enabled (ce));unpin_guc_id (guc, ce, true); -lrc_unpin (ce); +__guc_context_unpin (ce);if (likely (!intel_context_is_barrier (ce)))intel_engine_pm_put (ce->engine);@@ -2062,7 +2074,141 @@ static void guc_context_unpin(struct intel_context *ce)static void guc_context_post_unpin (struct intel_context *ce) { -lrc_post_unpin (ce); +__guc_context_post_unpin (ce); +} + +/* Future patches will use this function */ +__maybe_unused +static int guc_parent_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +struct intel_context *child; +int err, i =0 , j =0 ; + +for_each_child (ce, child) { + err =i915_active_acquire (&child->active); +if (unlikely (err)) + goto unwind_active; + ++i; + } + +for_each_child (ce, child) { + err =__guc_context_pre_pin (child, child->engine, ww); +if (unlikely (err)) + goto unwind_pre_pin; + ++j; + } + + err =__guc_context_pre_pin (ce, ce->engine, ww); +if (unlikely (err)) + goto unwind_pre_pin; + +return 0 ; + +unwind_pre_pin: +for_each_child (ce, child) { +if (!j--) +break ; +__guc_context_post_unpin (child); + } + +unwind_active: +for_each_child (ce, child) { +if (!i--) +break ; +i915_active_release (&child->active); + } + +return err; +} + +/* Future patches will use this function */ +__maybe_unused +static void guc_parent_context_post_unpin (struct intel_context *ce) +{ +struct intel_context *child; + +for_each_child (ce, child) +__guc_context_post_unpin (child); +__guc_context_post_unpin (ce); + +for_each_child (ce, child) { +intel_context_get (child); +i915_active_release (&child->active); +intel_context_put (child); + } +} + +/* Future patches will use this function */ +__maybe_unused +static int guc_parent_context_pin (struct intel_context *ce) +{ +int ret, i =0 , j =0 ; +struct intel_context *child; +struct intel_engine_cs *engine; +intel_engine_mask_t tmp; + +GEM_BUG_ON (!intel_context_is_parent (ce)); + +for_each_child (ce, child) { + ret =__guc_context_pin (child, child->engine); +if (unlikely (ret)) + goto unwind_pin; + ++i; + } + ret =__guc_context_pin (ce, ce->engine); +if (unlikely (ret)) + goto unwind_pin; + +for_each_child (ce, child) +if (test_bit (CONTEXT_LRCA_DIRTY , &child->flags)) { +set_bit (CONTEXT_LRCA_DIRTY , &ce->flags); +break ; + } + +for_each_engine_masked (engine, ce->engine->gt, + ce->engine->mask, tmp) +intel_engine_pm_get (engine); +for_each_child (ce, child) +for_each_engine_masked (engine, child->engine->gt, + child->engine->mask, tmp) +intel_engine_pm_get (engine); + +return 0 ; + +unwind_pin: +for_each_child (ce, child) { +if (++j > i) +break ; +__guc_context_unpin (child); + } + +return ret; +} + +/* Future patches will use this function */ +__maybe_unused +static void guc_parent_context_unpin (struct intel_context *ce) +{ +struct intel_context *child; +struct intel_engine_cs *engine; +intel_engine_mask_t tmp; + +GEM_BUG_ON (!intel_context_is_parent (ce)); +GEM_BUG_ON (context_enabled (ce)); + +unpin_guc_id (ce_to_guc (ce), ce, true); +for_each_child (ce, child) +__guc_context_unpin (child); +__guc_context_unpin (ce); + +for_each_engine_masked (engine, ce->engine->gt, + ce->engine->mask, tmp) +intel_engine_pm_put (engine); +for_each_child (ce, child) +for_each_engine_masked (engine, child->engine->gt, + child->engine->mask, tmp) +intel_engine_pm_put (engine); } static void __guc_context_sched_enable(struct intel_guc *guc,@@ -2993,18 +3139,17 @@ static int guc_request_alloc(struct i915_request *rq) }static int guc_virtual_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) {struct intel_engine_cs *engine =guc_virtual_get_sibling (ce->engine,0 ); -return __guc_context_pre_pin (ce, engine, ww, vaddr); +return __guc_context_pre_pin (ce, engine, ww); } -static int guc_virtual_context_pin (struct intel_context *ce,void *vaddr) +static int guc_virtual_context_pin (struct intel_context *ce) {struct intel_engine_cs *engine =guc_virtual_get_sibling (ce->engine,0 ); -int ret =__guc_context_pin (ce, engine, vaddr); +int ret =__guc_context_pin (ce, engine);intel_engine_mask_t tmp, mask = ce->engine->mask;if (likely (!ret))@@ -3024,7 +3169,7 @@ static void guc_virtual_context_unpin(struct intel_context *ce)GEM_BUG_ON (intel_context_is_barrier (ce));unpin_guc_id (guc, ce, true); -lrc_unpin (ce); +__guc_context_unpin (ce);for_each_engine_masked (engine, ce->engine->gt, mask, tmp)intel_engine_pm_put (engine);Signature · 1 line
2.28.0Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.chSignature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.chSignature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [Intel-gfx] [PATCH 16/46] drm/i915/guc: Implement GuC parent-child context pin / unpin functions
Quoted 253 lines · Daniel Vetter
Quoted 235 lines · Matthew Brost
Quoted 224 lines · Daniel Vetter
Quoted 223 lines · Daniel Vetter
Quoted 222 lines · Matthew Brost
Quoted 221 lines · Daniel Vetter
Quoted 220 lines · Matthew Brost
Implement GuC parent-child context pin / unpin functions in which in any contexts in the relationship are pinned all the contexts are pinned. The parent owns most of the pinning / unpinning process and the children direct any pins / unpins to the parent.
Patch implements a number of unused functions that will be connected later in the series.
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/gt/intel_context.c | 187 ++++++++++++++++-- drivers/gpu/drm/i915/gt/intel_context.h | 43 +--- drivers/gpu/drm/i915/gt/intel_context_types.h | 4 +- .../drm/i915/gt/intel_execlists_submission.c | 25 ++- drivers/gpu/drm/i915/gt/intel_lrc.c | 26 +-- drivers/gpu/drm/i915/gt/intel_lrc.h | 6 +- .../gpu/drm/i915/gt/intel_ring_submission.c | 5 +- drivers/gpu/drm/i915/gt/mock_engine.c | 4 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 183 +++++++++++++++-- 9 files changed, 371 insertions(+), 112 deletions(-)drivers/gpu/drm/i915/gt/intel_context.c +113
-18@@ -158,8 +158,8 @@ static void __ring_retire(struct intel_ring *ring)intel_ring_unpin (ring); } -static int intel_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) +static int __intel_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) {int err;@@ -190,7 +190,7 @@ static int intel_context_pre_pin(struct intel_context *ce,return err; } -static void intel_context_post_unpin (struct intel_context *ce) +static void __intel_context_post_unpin (struct intel_context *ce) {if (ce->state)__context_unpin_state (ce->state);@@ -199,13 +199,85 @@ static void intel_context_post_unpin(struct intel_context *ce)__ring_retire (ce->ring); } -int __intel_context_do_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) +static int intel_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) { -bool handoff = false; -void *vaddr; +struct intel_context *child; +int err, i =0 ; + +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) { + err =__intel_context_pre_pin (child, ww); +if (unlikely (err)) + goto unwind; + ++i; + } + + err =__intel_context_pre_pin (ce, ww); +if (unlikely (err)) + goto unwind; + +return 0 ; + +unwind: +for_each_child (ce, child) { +if (!i--) +break ; +__intel_context_post_unpin (ce); + } + +return err; +} + +static void intel_context_post_unpin (struct intel_context *ce) +{ +struct intel_context *child; + +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) +__intel_context_post_unpin (child); + +__intel_context_post_unpin (ce); +} + +static int __do_ww_lock (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +int err =i915_gem_object_lock (ce->timeline->hwsp_ggtt->obj, ww); + +if (!err && ce->ring->vma->obj) + err =i915_gem_object_lock (ce->ring->vma->obj, ww); +if (!err && ce->state) + err =i915_gem_object_lock (ce->state->obj, ww); + +return err; +} + +static int do_ww_lock (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +struct intel_context *child;int err =0 ; +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) { + err =__do_ww_lock (child, ww); +if (unlikely (err)) +return err; + } + +return __do_ww_lock (ce, ww); +} + +static int __intel_context_do_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +bool handoff = false; +int err; +if (unlikely (!test_bit (CONTEXT_ALLOC_BIT , &ce->flags))) { err =intel_context_alloc_state (ce);if (err)@@ -217,14 +289,11 @@ int __intel_context_do_pin_ww(struct intel_context *ce,* refcount for __intel_context_active(), which prevent a lock * inversion of ce->pin_mutex vs dma_resv_lock(). */ + err =do_ww_lock (ce, ww); +if (err) +return err; - err =i915_gem_object_lock (ce->timeline->hwsp_ggtt->obj, ww); -if (!err && ce->ring->vma->obj) - err =i915_gem_object_lock (ce->ring->vma->obj, ww); -if (!err && ce->state) - err =i915_gem_object_lock (ce->state->obj, ww); -if (!err) - err =intel_context_pre_pin (ce, ww); + err =intel_context_pre_pin (ce, ww);if (err)return err;@@ -232,7 +301,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,if (err) goto err_ctx_unpin; - err = ce->ops->pre_pin (ce, ww, &vaddr); + err = ce->ops->pre_pin (ce, ww);if (err) goto err_release;@@ -250,7 +319,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,if (unlikely (err)) goto err_unlock; - err = ce->ops->pin (ce, vaddr); + err = ce->ops->pin (ce);if (err) {intel_context_active_release (ce); goto err_unlock;@@ -290,7 +359,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,return err; } -int __intel_context_do_pin (struct intel_context *ce) +static int __intel_context_do_pin (struct intel_context *ce) {struct i915_gem_ww_ctx ww;int err;@@ -337,7 +406,7 @@ static void __intel_context_retire(struct i915_active *active) intel_context_get_avg_runtime_ns(ce));set_bit (CONTEXT_VALID_BIT , &ce->flags); -intel_context_post_unpin (ce); +__intel_context_post_unpin (ce);intel_context_put (ce); }@@ -562,6 +631,88 @@ void intel_context_bind_parent_child(struct intel_context *parent, child->parent = parent; } +static inline int ____intel_context_pin (struct intel_context *ce) +{ +if (likely (intel_context_pin_if_active (ce))) +return 0 ; + +return __intel_context_do_pin (ce); +} + +static inline int __intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +if (likely (intel_context_pin_if_active (ce))) +return 0 ; + +return __intel_context_do_pin_ww (ce, ww); +} + +static inline void __intel_context_unpin (struct intel_context *ce) +{ +if (!ce->ops->sched_disable) { +__intel_context_do_unpin (ce,1 ); + }else { +/* +* Move ownership of this pin to the scheduling disable which is +* an async operation. When that operation completes the above +* intel_context_sched_disable_unpin is called potentially Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
drivers/gpu/drm/i915/gt/intel_context.c void intel_context_bind_parent_child(struct intel_context *parent, · line 663
Quoted 4 lines
drivers/gpu/drm/i915/gt/intel_context.c +4
-0+* unpinning the context. +*/ +while (!atomic_add_unless (&ce->pin_count,-1 ,1 )) { +if (atomic_cmpxchg (&ce->pin_count,1 ,2 ) ==1 ) {Just as an example of what I mean here on the code review side. This is an endless loop, and you need to prove that there's no livelock or starvation issues. Or explain how else you handle that if there is one.
Quoted 7 lines
If we pop into the while loop the
pin_count= 1, so in all likelihood the follow if evaluates to true and the loop is broken. Only way it evaluates to false is the context gets pinned between the while & if, so on the next pass the while statement should evaluate to false breaking the looping unless of course the contexts gets unpinned again... In pratice this should be at most 3 atomic operations unless the loop is broken.That's not really how this works ... E.g. the linux spinlocks are ticketed/queued locks, exactly because the "this likely doesn't happen" argument is not a very good one.
Can't we at least convert
intel_context->pin_countinto a normal counter with a spinlock around it and then just make all this stuff a lot more reasonable?
Quoted 5 lines
Yes this doesn't help the situation overall much, but at least we're not spreading dubious hand-roll locking patterns all over the place for not much good reasons.
Hand-crafted artisanal locking is never a quality sign.
Or the simple plan, just pin the contexts always. That still leaves you with some fun on final unpin, because that's called in an awkward place, but that can be fixed a work_struct.
-Daniel
Quoted 699 lines
-Daniel
Quoted 692 lines
Matt
Quoted 689 lines
Because unlike hand-rolled stuff linux kernel spinlocks are not dumb spinlocks, but ticketed/queued locks and therefor starvation proof. But this stuff actually matters on todays multi-core and not-so-uniform (even without fully NUMA) architectures.
Also I've just found another lockless retry loop which does actually degenerate into a full endless loop (if you're sufficiently unlucky in your races), so this really isn't academic at all.
-DanielQuoted 670 lines
Quoted 31 lines
Quoted 23 lines
Uh man lockless algorithms.
Unless this comes:
- with essentially an academic looking paper that describes the abstract model of the lockless algorithm and proves it against the linux kernel meory model.- lockless stuff generally needs barriers, and those barriers must be all documented. This means a) a comment next to each barrier in the code b) pointing to its counterparty c) with the overall design also explained in the kerneldoc for those datastructres.
If you don't know where your barriers are, see above point about "it should look more like an academic paper in the commit message"
- hard perf data about how this is absolutely required, based on a real-world use-case (which then sometimes justifies a microbenchmark metric for the details, but it always needs to be real-world based). And also a throughrough explainer how the perf issue isn't fixable through better design. If that's not doable, just protect the state machine with a big dumb lock and move on.
Quoted 7 lines
- Also, because the current code is in such bad shape wrt lockless algorithms and premature optimizations: Overall complexity should go down (it's way too high right now), so pay down your new lockless trick by removing one of the existing ones that we only have because we can.
Yes this is steep, but we're way out in the woods here and need to smoehow get back.
Quoted 7 lines
See below FIXME. At one point all of this was hidden in the backend but the dma-resv patches that landed upstream completely broke the layering, hence the need for the code here.
I guess I don't really understand what mean when you say lockless alg needs barriers, if the atomic functions are not really atomic wouldn't the world be broken?
They unordered atomics by default. Which means they're atomic itself, but entirely unordered with anything else that's going on. Except when you have one of the atomic ops which already guarantee a barrier, or you manually add the barriers yourself. And yes there's enormous amounts of bugs, and with our dgpu potentially running on non-IA cpus those bugs matter.
Note that in C++ atomics the default behaviour is strongly ordered atomics with full barriers by default, because those are much easier to program against. Kernel isn't like that and defaults to "you need to add all the barriers yourself".
I have a full lenght rant in the works and will work that through all channels, but essentially locking is really hard to get right. And lockless tricks practically need an academic paper with a formal correctness proof against the linux memory model, or you do have bugs.
And I know that the current code is choke full of this stuff, so it's tempting to just add more, but we really cant. The amount of locking trickery we have in the codebase must go down substantially. My take is that any code that adds anything trick needs to fully justify it against the above list, _and_ also clean up some of the existing nonsense so that overall complexity doesn't increase.
I'll share the full length rant with you internally, it's not yet ready for publishing (but that's planned too).
Quoted 4 lines
Also here I don't think it is really as simple as grab big dump lock for a variety of reasons, at least with the current dynamic pin / unpin code in place. If we move a perma-pinned contexts this could be cleaned up then.
Yes it's a disaster, but we need to stop the bleeding. If perma-pinned context can fix this I think we should do this asap. I'd say for parallel context we should just do it outright (special case them or whatever) so that we don't have to add even more very tricky code and tech debt.
Doable?
Cheers, Daniel
Quoted 587 lines
Matt
Quoted 584 lines
-Daniel
Quoted 575 lines
drivers/gpu/drm/i915/gt/intel_context.c +52
-0+ ce->ops->sched_disable (ce); +break ; + } + } + } +} + +/* +* FIXME: This is ugly, these branches are only needed for parallel contexts in +* GuC submission. Basically the idea is if any of the contexts, that are +* configured for parallel submission, are pinned all the contexts need to be +* pinned in order to register these contexts with the GuC. We are adding the +* layer here while it should probably be pushed to the backend via a vfunc. But +* since we already have ce->pin + a layer atop it is confusing. Definitely +* needs a bit of rework how to properly layer / structure this code path. What +* is in place works but is not ideal. +*/ +int intel_context_pin (struct intel_context *ce) +{ +if (intel_context_is_child (ce)) { +if (!atomic_fetch_add (1 , &ce->pin_count)) +return ____intel_context_pin (ce->parent); +else +return 0 ; + }else { +return ____intel_context_pin (ce); + } +} + +int intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +if (intel_context_is_child (ce)) { +if (!atomic_fetch_add (1 , &ce->pin_count)) +return __intel_context_pin_ww (ce->parent, ww); +else +return 0 ; + }else { +return __intel_context_pin_ww (ce, ww); + } +} + +void intel_context_unpin (struct intel_context *ce) +{ +if (intel_context_is_child (ce)) { +if (atomic_fetch_add (-1 , &ce->pin_count) ==1 ) +__intel_context_unpin (ce->parent); + }else { +__intel_context_unpin (ce); + } +} +#if IS_ENABLED (CONFIG_DRM_I915_SELFTEST )#include "selftest_context.c" #endif drivers/gpu/drm/i915/gt/intel_context.h +5
-38@@ -110,31 +110,15 @@ static inline void intel_context_unlock_pinned(struct intel_context *ce)mutex_unlock (&ce->pin_mutex); } -int __intel_context_do_pin (struct intel_context *ce); -int __intel_context_do_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww); -static inline bool intel_context_pin_if_active (struct intel_context *ce) {return atomic_inc_not_zero (&ce->pin_count); } -static inline int intel_context_pin (struct intel_context *ce) -{ -if (likely (intel_context_pin_if_active (ce))) -return 0 ; - -return __intel_context_do_pin (ce); -} - -static inline int intel_context_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) -{ -if (likely (intel_context_pin_if_active (ce))) -return 0 ; +int intel_context_pin (struct intel_context *ce); -return __intel_context_do_pin_ww (ce, ww); -} +int intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww);static inline void __intel_context_pin (struct intel_context *ce) {@@ -146,28 +130,11 @@ void __intel_context_do_unpin(struct intel_context *ce, int sub);static inline void intel_context_sched_disable_unpin (struct intel_context *ce) { +GEM_BUG_ON (intel_context_is_child (ce));__intel_context_do_unpin (ce,2 ); } -static inline void intel_context_unpin (struct intel_context *ce) -{ -if (!ce->ops->sched_disable) { -__intel_context_do_unpin (ce,1 ); - }else { -/* -* Move ownership of this pin to the scheduling disable which is -* an async operation. When that operation completes the above -* intel_context_sched_disable_unpin is called potentially -* unpinning the context. -*/ -while (!atomic_add_unless (&ce->pin_count,-1 ,1 )) { -if (atomic_cmpxchg (&ce->pin_count,1 ,2 ) ==1 ) { - ce->ops->sched_disable (ce); -break ; - } - } - } -} +void intel_context_unpin (struct intel_context *ce);void intel_context_enter_engine (struct intel_context *ce);void intel_context_exit_engine (struct intel_context *ce);drivers/gpu/drm/i915/gt/intel_context_types.h +2
-2@@ -39,8 +39,8 @@ struct intel_context_ops {void (*ban)(struct intel_context *ce,struct i915_request *rq); -int (*pre_pin)(struct intel_context *ce,struct i915_gem_ww_ctx *ww,void **vaddr); -int (*pin)(struct intel_context *ce,void *vaddr); +int (*pre_pin)(struct intel_context *ce,struct i915_gem_ww_ctx *ww); +int (*pin)(struct intel_context *ce);void (*unpin)(struct intel_context *ce);void (*post_unpin)(struct intel_context *ce);drivers/gpu/drm/i915/gt/intel_execlists_submission.c +12
-13@@ -2554,16 +2554,17 @@ static void execlists_submit_request(struct i915_request *request)static int __execlists_context_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww,void **vaddr) +struct i915_gem_ww_ctx *ww) {int err; - err =lrc_pre_pin (ce, engine, ww, vaddr); + err =lrc_pre_pin (ce, engine, ww);if (err)return err;if (!__test_and_set_bit (CONTEXT_INIT_BIT , &ce->flags)) { -lrc_init_state (ce, engine, *vaddr); +lrc_init_state (ce, engine, ce->lrc_reg_state - +LRC_STATE_OFFSET /sizeof (*ce->lrc_reg_state));__i915_gem_object_flush_map (ce->state->obj,0 , engine->context_size); }@@ -2572,15 +2573,14 @@ __execlists_context_pre_pin(struct intel_context *ce, }static int execlists_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return __execlists_context_pre_pin (ce, ce->engine, ww, vaddr); +return __execlists_context_pre_pin (ce, ce->engine, ww); } -static int execlists_context_pin (struct intel_context *ce,void *vaddr) +static int execlists_context_pin (struct intel_context *ce) { -return lrc_pin (ce, ce->engine, vaddr); +return lrc_pin (ce, ce->engine); }static int execlists_context_alloc (struct intel_context *ce)@@ -3570,20 +3570,19 @@ static int virtual_context_alloc(struct intel_context *ce) }static int virtual_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) {struct virtual_engine *ve =container_of (ce,typeof (*ve), context);/* Note: we must use a real engine class for setting up reg state */ -return __execlists_context_pre_pin (ce, ve->siblings[0 ], ww, vaddr); +return __execlists_context_pre_pin (ce, ve->siblings[0 ], ww); } -static int virtual_context_pin (struct intel_context *ce,void *vaddr) +static int virtual_context_pin (struct intel_context *ce) {struct virtual_engine *ve =container_of (ce,typeof (*ve), context); -return lrc_pin (ce, ve->siblings[0 ], vaddr); +return lrc_pin (ce, ve->siblings[0 ]); }static void virtual_context_enter (struct intel_context *ce)drivers/gpu/drm/i915/gt/intel_lrc.c +13
-13@@ -947,30 +947,30 @@ void lrc_reset(struct intel_context *ce)int lrc_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { +void *vaddr;GEM_BUG_ON (!ce->state);GEM_BUG_ON (!i915_vma_is_pinned (ce->state)); - *vaddr =i915_gem_object_pin_map (ce->state->obj, -i915_coherent_map_type (ce->engine->i915, - ce->state->obj, - false) | -I915_MAP_OVERRIDE ); + vaddr =i915_gem_object_pin_map (ce->state->obj, +i915_coherent_map_type (ce->engine->i915, + ce->state->obj, + false) | +I915_MAP_OVERRIDE ); -return PTR_ERR_OR_ZERO (*vaddr); + ce->lrc_reg_state = vaddr +LRC_STATE_OFFSET ; + +return PTR_ERR_OR_ZERO (vaddr); }int lrc_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr) +struct intel_engine_cs *engine) { - ce->lrc_reg_state = vaddr +LRC_STATE_OFFSET ; -if (!__test_and_set_bit (CONTEXT_INIT_BIT , &ce->flags)) -lrc_init_state (ce, engine, vaddr); +lrc_init_state (ce, engine, + (void *)ce->lrc_reg_state -LRC_STATE_OFFSET ); ce->lrc.lrca =lrc_update_regs (ce, engine, ce->ring->tail);return 0 ;drivers/gpu/drm/i915/gt/intel_lrc.h +2
-4@@ -38,12 +38,10 @@ void lrc_destroy(struct kref *kref);int lrc_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr); +struct i915_gem_ww_ctx *ww);int lrc_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr); +struct intel_engine_cs *engine);void lrc_unpin (struct intel_context *ce);void lrc_post_unpin (struct intel_context *ce);drivers/gpu/drm/i915/gt/intel_ring_submission.c +2
-3@@ -472,8 +472,7 @@ static int ring_context_init_default_state(struct intel_context *ce, }static int ring_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **unused) +struct i915_gem_ww_ctx *ww) {struct i915_address_space *vm;int err =0 ;@@ -576,7 +575,7 @@ static int ring_context_alloc(struct intel_context *ce)return 0 ; } -static int ring_context_pin (struct intel_context *ce,void *unused) +static int ring_context_pin (struct intel_context *ce) {return 0 ; }drivers/gpu/drm/i915/gt/mock_engine.c +2
-2@@ -167,12 +167,12 @@ static int mock_context_alloc(struct intel_context *ce) }static int mock_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww,void **unused) +struct i915_gem_ww_ctx *ww) {return 0 ; } -static int mock_context_pin (struct intel_context *ce,void *unused) +static int mock_context_pin (struct intel_context *ce) {return 0 ; }drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +164
-20@@ -1905,6 +1905,7 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)GEM_BUG_ON (!engine->mask);GEM_BUG_ON (context_guc_id_invalid (ce)); +GEM_BUG_ON (intel_context_is_child (ce));/* * Ensure LRC + CT vmas are is same region as write barrier is done @@ -2008,15 +2009,13 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)static int __guc_context_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return lrc_pre_pin (ce, engine, ww, vaddr); +return lrc_pre_pin (ce, engine, ww); }static int __guc_context_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr) +struct intel_engine_cs *engine) {if (i915_ggtt_offset (ce->state) != (ce->lrc.lrca &CTX_GTT_ADDRESS_MASK ))@@ -2027,20 +2026,33 @@ static int __guc_context_pin(struct intel_context *ce,* explaination of why. */ -return lrc_pin (ce, engine, vaddr); +return lrc_pin (ce, engine); +} + +static void __guc_context_unpin (struct intel_context *ce) +{ +lrc_unpin (ce); +} + +static void __guc_context_post_unpin (struct intel_context *ce) +{ +lrc_post_unpin (ce); }static int guc_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return __guc_context_pre_pin (ce, ce->engine, ww, vaddr); +return __guc_context_pre_pin (ce, ce->engine, ww); } -static int guc_context_pin (struct intel_context *ce,void *vaddr) +static int guc_context_pin (struct intel_context *ce) { -int ret =__guc_context_pin (ce, ce->engine, vaddr); +int ret; +GEM_BUG_ON (intel_context_is_parent (ce) || +intel_context_is_child (ce)); + + ret =__guc_context_pin (ce, ce->engine);if (likely (!ret && !intel_context_is_barrier (ce)))intel_engine_pm_get (ce->engine);@@ -2054,7 +2066,7 @@ static void guc_context_unpin(struct intel_context *ce)GEM_BUG_ON (context_enabled (ce));unpin_guc_id (guc, ce, true); -lrc_unpin (ce); +__guc_context_unpin (ce);if (likely (!intel_context_is_barrier (ce)))intel_engine_pm_put (ce->engine);@@ -2062,7 +2074,141 @@ static void guc_context_unpin(struct intel_context *ce)static void guc_context_post_unpin (struct intel_context *ce) { -lrc_post_unpin (ce); +__guc_context_post_unpin (ce); +} + +/* Future patches will use this function */ +__maybe_unused +static int guc_parent_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +struct intel_context *child; +int err, i =0 , j =0 ; + +for_each_child (ce, child) { + err =i915_active_acquire (&child->active); +if (unlikely (err)) + goto unwind_active; + ++i; + } + +for_each_child (ce, child) { + err =__guc_context_pre_pin (child, child->engine, ww); +if (unlikely (err)) + goto unwind_pre_pin; + ++j; + } + + err =__guc_context_pre_pin (ce, ce->engine, ww); +if (unlikely (err)) + goto unwind_pre_pin; + +return 0 ; + +unwind_pre_pin: +for_each_child (ce, child) { +if (!j--) +break ; +__guc_context_post_unpin (child); + } + +unwind_active: +for_each_child (ce, child) { +if (!i--) +break ; +i915_active_release (&child->active); + } + +return err; +} + +/* Future patches will use this function */ +__maybe_unused +static void guc_parent_context_post_unpin (struct intel_context *ce) +{ +struct intel_context *child; + +for_each_child (ce, child) +__guc_context_post_unpin (child); +__guc_context_post_unpin (ce); + +for_each_child (ce, child) { +intel_context_get (child); +i915_active_release (&child->active); +intel_context_put (child); + } +} + +/* Future patches will use this function */ +__maybe_unused +static int guc_parent_context_pin (struct intel_context *ce) +{ +int ret, i =0 , j =0 ; +struct intel_context *child; +struct intel_engine_cs *engine; +intel_engine_mask_t tmp; + +GEM_BUG_ON (!intel_context_is_parent (ce)); + +for_each_child (ce, child) { + ret =__guc_context_pin (child, child->engine); +if (unlikely (ret)) + goto unwind_pin; + ++i; + } + ret =__guc_context_pin (ce, ce->engine); +if (unlikely (ret)) + goto unwind_pin; + +for_each_child (ce, child) +if (test_bit (CONTEXT_LRCA_DIRTY , &child->flags)) { +set_bit (CONTEXT_LRCA_DIRTY , &ce->flags); +break ; + } + +for_each_engine_masked (engine, ce->engine->gt, + ce->engine->mask, tmp) +intel_engine_pm_get (engine); +for_each_child (ce, child) +for_each_engine_masked (engine, child->engine->gt, + child->engine->mask, tmp) +intel_engine_pm_get (engine); + +return 0 ; + +unwind_pin: +for_each_child (ce, child) { +if (++j > i) +break ; +__guc_context_unpin (child); + } + +return ret; +} + +/* Future patches will use this function */ +__maybe_unused +static void guc_parent_context_unpin (struct intel_context *ce) +{ +struct intel_context *child; +struct intel_engine_cs *engine; +intel_engine_mask_t tmp; + +GEM_BUG_ON (!intel_context_is_parent (ce)); +GEM_BUG_ON (context_enabled (ce)); + +unpin_guc_id (ce_to_guc (ce), ce, true); +for_each_child (ce, child) +__guc_context_unpin (child); +__guc_context_unpin (ce); + +for_each_engine_masked (engine, ce->engine->gt, + ce->engine->mask, tmp) +intel_engine_pm_put (engine); +for_each_child (ce, child) +for_each_engine_masked (engine, child->engine->gt, + child->engine->mask, tmp) +intel_engine_pm_put (engine); } static void __guc_context_sched_enable(struct intel_guc *guc,@@ -2993,18 +3139,17 @@ static int guc_request_alloc(struct i915_request *rq) }static int guc_virtual_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) {struct intel_engine_cs *engine =guc_virtual_get_sibling (ce->engine,0 ); -return __guc_context_pre_pin (ce, engine, ww, vaddr); +return __guc_context_pre_pin (ce, engine, ww); } -static int guc_virtual_context_pin (struct intel_context *ce,void *vaddr) +static int guc_virtual_context_pin (struct intel_context *ce) {struct intel_engine_cs *engine =guc_virtual_get_sibling (ce->engine,0 ); -int ret =__guc_context_pin (ce, engine, vaddr); +int ret =__guc_context_pin (ce, engine);intel_engine_mask_t tmp, mask = ce->engine->mask;if (likely (!ret))@@ -3024,7 +3169,7 @@ static void guc_virtual_context_unpin(struct intel_context *ce)GEM_BUG_ON (intel_context_is_barrier (ce));unpin_guc_id (guc, ce, true); -lrc_unpin (ce); +__guc_context_unpin (ce);for_each_engine_masked (engine, ce->engine->gt, mask, tmp)intel_engine_pm_put (engine); --Quoted 2 lines
2.28.0
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.chQuoted 4 lines
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [Intel-gfx] [PATCH 16/46] drm/i915/guc: Implement GuC parent-child context pin / unpin functions
In reply to Daniel VetterQuoted 266 lines · Daniel Vetter
Quoted 257 lines · Matthew Brost
Quoted 248 lines · Daniel Vetter
Quoted 214 lines · Matthew Brost
Implement GuC parent-child context pin / unpin functions in which in any contexts in the relationship are pinned all the contexts are pinned. The parent owns most of the pinning / unpinning process and the children direct any pins / unpins to the parent.
Patch implements a number of unused functions that will be connected later in the series.
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/gt/intel_context.c | 187 ++++++++++++++++-- drivers/gpu/drm/i915/gt/intel_context.h | 43 +--- drivers/gpu/drm/i915/gt/intel_context_types.h | 4 +- .../drm/i915/gt/intel_execlists_submission.c | 25 ++- drivers/gpu/drm/i915/gt/intel_lrc.c | 26 +-- drivers/gpu/drm/i915/gt/intel_lrc.h | 6 +- .../gpu/drm/i915/gt/intel_ring_submission.c | 5 +- drivers/gpu/drm/i915/gt/mock_engine.c | 4 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 183 +++++++++++++++-- 9 files changed, 371 insertions(+), 112 deletions(-)drivers/gpu/drm/i915/gt/intel_context.c +107
-18@@ -158,8 +158,8 @@ static void __ring_retire(struct intel_ring *ring)intel_ring_unpin (ring); } -static int intel_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) +static int __intel_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) {int err;@@ -190,7 +190,7 @@ static int intel_context_pre_pin(struct intel_context *ce,return err; } -static void intel_context_post_unpin (struct intel_context *ce) +static void __intel_context_post_unpin (struct intel_context *ce) {if (ce->state)__context_unpin_state (ce->state);@@ -199,13 +199,85 @@ static void intel_context_post_unpin(struct intel_context *ce)__ring_retire (ce->ring); } -int __intel_context_do_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) +static int intel_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) { -bool handoff = false; -void *vaddr; +struct intel_context *child; +int err, i =0 ; + +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) { + err =__intel_context_pre_pin (child, ww); +if (unlikely (err)) + goto unwind; + ++i; + } + + err =__intel_context_pre_pin (ce, ww); +if (unlikely (err)) + goto unwind; + +return 0 ; + +unwind: +for_each_child (ce, child) { +if (!i--) +break ; +__intel_context_post_unpin (ce); + } + +return err; +} + +static void intel_context_post_unpin (struct intel_context *ce) +{ +struct intel_context *child; + +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) +__intel_context_post_unpin (child); + +__intel_context_post_unpin (ce); +} + +static int __do_ww_lock (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +int err =i915_gem_object_lock (ce->timeline->hwsp_ggtt->obj, ww); + +if (!err && ce->ring->vma->obj) + err =i915_gem_object_lock (ce->ring->vma->obj, ww); +if (!err && ce->state) + err =i915_gem_object_lock (ce->state->obj, ww); + +return err; +} + +static int do_ww_lock (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +struct intel_context *child;int err =0 ; +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) { + err =__do_ww_lock (child, ww); +if (unlikely (err)) +return err; + } + +return __do_ww_lock (ce, ww); +} + +static int __intel_context_do_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +bool handoff = false; +int err; +if (unlikely (!test_bit (CONTEXT_ALLOC_BIT , &ce->flags))) { err =intel_context_alloc_state (ce);if (err)@@ -217,14 +289,11 @@ int __intel_context_do_pin_ww(struct intel_context *ce,* refcount for __intel_context_active(), which prevent a lock * inversion of ce->pin_mutex vs dma_resv_lock(). */ + err =do_ww_lock (ce, ww); +if (err) +return err; - err =i915_gem_object_lock (ce->timeline->hwsp_ggtt->obj, ww); -if (!err && ce->ring->vma->obj) - err =i915_gem_object_lock (ce->ring->vma->obj, ww); -if (!err && ce->state) - err =i915_gem_object_lock (ce->state->obj, ww); -if (!err) - err =intel_context_pre_pin (ce, ww); + err =intel_context_pre_pin (ce, ww);if (err)return err;@@ -232,7 +301,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,if (err) goto err_ctx_unpin; - err = ce->ops->pre_pin (ce, ww, &vaddr); + err = ce->ops->pre_pin (ce, ww);if (err) goto err_release;@@ -250,7 +319,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,if (unlikely (err)) goto err_unlock; - err = ce->ops->pin (ce, vaddr); + err = ce->ops->pin (ce);if (err) {intel_context_active_release (ce); goto err_unlock;@@ -290,7 +359,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,return err; } -int __intel_context_do_pin (struct intel_context *ce) +static int __intel_context_do_pin (struct intel_context *ce) {struct i915_gem_ww_ctx ww;int err;@@ -337,7 +406,7 @@ static void __intel_context_retire(struct i915_active *active) intel_context_get_avg_runtime_ns(ce));set_bit (CONTEXT_VALID_BIT , &ce->flags); -intel_context_post_unpin (ce); +__intel_context_post_unpin (ce);intel_context_put (ce); }@@ -562,6 +631,88 @@ void intel_context_bind_parent_child(struct intel_context *parent, child->parent = parent; } +static inline int ____intel_context_pin (struct intel_context *ce) +{ +if (likely (intel_context_pin_if_active (ce))) +return 0 ; + +return __intel_context_do_pin (ce); +} + +static inline int __intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +if (likely (intel_context_pin_if_active (ce))) +return 0 ; + +return __intel_context_do_pin_ww (ce, ww); +} + +static inline void __intel_context_unpin (struct intel_context *ce) +{ +if (!ce->ops->sched_disable) {drivers/gpu/drm/i915/gt/intel_context.c void intel_context_bind_parent_child(struct intel_context *parent, · line 663
Quoted 10 lines
drivers/gpu/drm/i915/gt/intel_context.c +10
-0+__intel_context_do_unpin (ce,1 ); + }else { +/* +* Move ownership of this pin to the scheduling disable which is +* an async operation. When that operation completes the above +* intel_context_sched_disable_unpin is called potentially +* unpinning the context. +*/ +while (!atomic_add_unless (&ce->pin_count,-1 ,1 )) { +if (atomic_cmpxchg (&ce->pin_count,1 ,2 ) ==1 ) {Uh man lockless algorithms.
Unless this comes:
- with essentially an academic looking paper that describes the abstract model of the lockless algorithm and proves it against the linux kernel meory model.- lockless stuff generally needs barriers, and those barriers must be all documented. This means a) a comment next to each barrier in the code b) pointing to its counterparty c) with the overall design also explained in the kerneldoc for those datastructres.
If you don't know where your barriers are, see above point about "it should look more like an academic paper in the commit message"
- hard perf data about how this is absolutely required, based on a real-world use-case (which then sometimes justifies a microbenchmark metric for the details, but it always needs to be real-world based). And also a throughrough explainer how the perf issue isn't fixable through better design. If that's not doable, just protect the state machine with a big dumb lock and move on.
Quoted 7 lines
- Also, because the current code is in such bad shape wrt lockless algorithms and premature optimizations: Overall complexity should go down (it's way too high right now), so pay down your new lockless trick by removing one of the existing ones that we only have because we can.
Yes this is steep, but we're way out in the woods here and need to smoehow get back.
Quoted 7 lines
See below FIXME. At one point all of this was hidden in the backend but the dma-resv patches that landed upstream completely broke the layering, hence the need for the code here.
I guess I don't really understand what mean when you say lockless alg needs barriers, if the atomic functions are not really atomic wouldn't the world be broken?
Quoted 7 lines
They unordered atomics by default. Which means they're atomic itself, but entirely unordered with anything else that's going on. Except when you have one of the atomic ops which already guarantee a barrier, or you manually add the barriers yourself. And yes there's enormous amounts of bugs, and with our dgpu potentially running on non-IA cpus those bugs matter.
Here I don't think it matters if they are unorderd relative to anything else. We really only care about the pin_count and resulting value of the atomic operation.
Quoted 17 lines
Note that in C++ atomics the default behaviour is strongly ordered atomics with full barriers by default, because those are much easier to program against. Kernel isn't like that and defaults to "you need to add all the barriers yourself".
I have a full lenght rant in the works and will work that through all channels, but essentially locking is really hard to get right. And lockless tricks practically need an academic paper with a formal correctness proof against the linux memory model, or you do have bugs.
And I know that the current code is choke full of this stuff, so it's tempting to just add more, but we really cant. The amount of locking trickery we have in the codebase must go down substantially. My take is that any code that adds anything trick needs to fully justify it against the above list, _and_ also clean up some of the existing nonsense so that overall complexity doesn't increase.
Quoted 3 lines
I'll share the full length rant with you internally, it's not yet ready for publishing (but that's planned too).
Sure we can chat about this. I am new here and basically taught myself to be a kernel developer by looking at the i915 which probably wasn't the best way to learn.
Quoted 6 lines
Quoted 5 lines
Also here I don't think it is really as simple as grab big dump lock for a variety of reasons, at least with the current dynamic pin / unpin code in place. If we move a perma-pinned contexts this could be cleaned up then.
Quoted 6 lines
Yes it's a disaster, but we need to stop the bleeding. If perma-pinned context can fix this I think we should do this asap. I'd say for parallel context we should just do it outright (special case them or whatever) so that we don't have to add even more very tricky code and tech debt.
Doable?
I think it is doable to perma-pin parallel contexts, regular contexts not so doable as that is a much large rework. I actually like this as I can drop a few other things in the parallel submission code if we move to perm-pinned contexts.
The only potential issue I see is running on a system with a limited number of guc_ids, lots of engine instances, and media UMD creating more parallel contexts that it really needs. This isn't a blocker as this really isn't a concern for upstream yet and is a workable problem one way or another.
Matt
Quoted 596 lines
Cheers, Daniel
Quoted 587 lines
Matt
Quoted 584 lines
-Daniel
Quoted 577 lines
drivers/gpu/drm/i915/gt/intel_context.c +52
-0+ ce->ops->sched_disable (ce); +break ; + } + } + } +} + +/* +* FIXME: This is ugly, these branches are only needed for parallel contexts in +* GuC submission. Basically the idea is if any of the contexts, that are +* configured for parallel submission, are pinned all the contexts need to be +* pinned in order to register these contexts with the GuC. We are adding the +* layer here while it should probably be pushed to the backend via a vfunc. But +* since we already have ce->pin + a layer atop it is confusing. Definitely +* needs a bit of rework how to properly layer / structure this code path. What +* is in place works but is not ideal. +*/ +int intel_context_pin (struct intel_context *ce) +{ +if (intel_context_is_child (ce)) { +if (!atomic_fetch_add (1 , &ce->pin_count)) +return ____intel_context_pin (ce->parent); +else +return 0 ; + }else { +return ____intel_context_pin (ce); + } +} + +int intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +if (intel_context_is_child (ce)) { +if (!atomic_fetch_add (1 , &ce->pin_count)) +return __intel_context_pin_ww (ce->parent, ww); +else +return 0 ; + }else { +return __intel_context_pin_ww (ce, ww); + } +} + +void intel_context_unpin (struct intel_context *ce) +{ +if (intel_context_is_child (ce)) { +if (atomic_fetch_add (-1 , &ce->pin_count) ==1 ) +__intel_context_unpin (ce->parent); + }else { +__intel_context_unpin (ce); + } +} +#if IS_ENABLED (CONFIG_DRM_I915_SELFTEST )#include "selftest_context.c" #endif drivers/gpu/drm/i915/gt/intel_context.h +5
-38@@ -110,31 +110,15 @@ static inline void intel_context_unlock_pinned(struct intel_context *ce)mutex_unlock (&ce->pin_mutex); } -int __intel_context_do_pin (struct intel_context *ce); -int __intel_context_do_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww); -static inline bool intel_context_pin_if_active (struct intel_context *ce) {return atomic_inc_not_zero (&ce->pin_count); } -static inline int intel_context_pin (struct intel_context *ce) -{ -if (likely (intel_context_pin_if_active (ce))) -return 0 ; - -return __intel_context_do_pin (ce); -} - -static inline int intel_context_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) -{ -if (likely (intel_context_pin_if_active (ce))) -return 0 ; +int intel_context_pin (struct intel_context *ce); -return __intel_context_do_pin_ww (ce, ww); -} +int intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww);static inline void __intel_context_pin (struct intel_context *ce) {@@ -146,28 +130,11 @@ void __intel_context_do_unpin(struct intel_context *ce, int sub);static inline void intel_context_sched_disable_unpin (struct intel_context *ce) { +GEM_BUG_ON (intel_context_is_child (ce));__intel_context_do_unpin (ce,2 ); } -static inline void intel_context_unpin (struct intel_context *ce) -{ -if (!ce->ops->sched_disable) { -__intel_context_do_unpin (ce,1 ); - }else { -/* -* Move ownership of this pin to the scheduling disable which is -* an async operation. When that operation completes the above -* intel_context_sched_disable_unpin is called potentially -* unpinning the context. -*/ -while (!atomic_add_unless (&ce->pin_count,-1 ,1 )) { -if (atomic_cmpxchg (&ce->pin_count,1 ,2 ) ==1 ) { - ce->ops->sched_disable (ce); -break ; - } - } - } -} +void intel_context_unpin (struct intel_context *ce);void intel_context_enter_engine (struct intel_context *ce);void intel_context_exit_engine (struct intel_context *ce);drivers/gpu/drm/i915/gt/intel_context_types.h +2
-2@@ -39,8 +39,8 @@ struct intel_context_ops {void (*ban)(struct intel_context *ce,struct i915_request *rq); -int (*pre_pin)(struct intel_context *ce,struct i915_gem_ww_ctx *ww,void **vaddr); -int (*pin)(struct intel_context *ce,void *vaddr); +int (*pre_pin)(struct intel_context *ce,struct i915_gem_ww_ctx *ww); +int (*pin)(struct intel_context *ce);void (*unpin)(struct intel_context *ce);void (*post_unpin)(struct intel_context *ce);drivers/gpu/drm/i915/gt/intel_execlists_submission.c +12
-13@@ -2554,16 +2554,17 @@ static void execlists_submit_request(struct i915_request *request)static int __execlists_context_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww,void **vaddr) +struct i915_gem_ww_ctx *ww) {int err; - err =lrc_pre_pin (ce, engine, ww, vaddr); + err =lrc_pre_pin (ce, engine, ww);if (err)return err;if (!__test_and_set_bit (CONTEXT_INIT_BIT , &ce->flags)) { -lrc_init_state (ce, engine, *vaddr); +lrc_init_state (ce, engine, ce->lrc_reg_state - +LRC_STATE_OFFSET /sizeof (*ce->lrc_reg_state));__i915_gem_object_flush_map (ce->state->obj,0 , engine->context_size); }@@ -2572,15 +2573,14 @@ __execlists_context_pre_pin(struct intel_context *ce, }static int execlists_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return __execlists_context_pre_pin (ce, ce->engine, ww, vaddr); +return __execlists_context_pre_pin (ce, ce->engine, ww); } -static int execlists_context_pin (struct intel_context *ce,void *vaddr) +static int execlists_context_pin (struct intel_context *ce) { -return lrc_pin (ce, ce->engine, vaddr); +return lrc_pin (ce, ce->engine); }static int execlists_context_alloc (struct intel_context *ce)@@ -3570,20 +3570,19 @@ static int virtual_context_alloc(struct intel_context *ce) }static int virtual_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) {struct virtual_engine *ve =container_of (ce,typeof (*ve), context);/* Note: we must use a real engine class for setting up reg state */ -return __execlists_context_pre_pin (ce, ve->siblings[0 ], ww, vaddr); +return __execlists_context_pre_pin (ce, ve->siblings[0 ], ww); } -static int virtual_context_pin (struct intel_context *ce,void *vaddr) +static int virtual_context_pin (struct intel_context *ce) {struct virtual_engine *ve =container_of (ce,typeof (*ve), context); -return lrc_pin (ce, ve->siblings[0 ], vaddr); +return lrc_pin (ce, ve->siblings[0 ]); }static void virtual_context_enter (struct intel_context *ce)drivers/gpu/drm/i915/gt/intel_lrc.c +13
-13@@ -947,30 +947,30 @@ void lrc_reset(struct intel_context *ce)int lrc_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { +void *vaddr;GEM_BUG_ON (!ce->state);GEM_BUG_ON (!i915_vma_is_pinned (ce->state)); - *vaddr =i915_gem_object_pin_map (ce->state->obj, -i915_coherent_map_type (ce->engine->i915, - ce->state->obj, - false) | -I915_MAP_OVERRIDE ); + vaddr =i915_gem_object_pin_map (ce->state->obj, +i915_coherent_map_type (ce->engine->i915, + ce->state->obj, + false) | +I915_MAP_OVERRIDE ); -return PTR_ERR_OR_ZERO (*vaddr); + ce->lrc_reg_state = vaddr +LRC_STATE_OFFSET ; + +return PTR_ERR_OR_ZERO (vaddr); }int lrc_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr) +struct intel_engine_cs *engine) { - ce->lrc_reg_state = vaddr +LRC_STATE_OFFSET ; -if (!__test_and_set_bit (CONTEXT_INIT_BIT , &ce->flags)) -lrc_init_state (ce, engine, vaddr); +lrc_init_state (ce, engine, + (void *)ce->lrc_reg_state -LRC_STATE_OFFSET ); ce->lrc.lrca =lrc_update_regs (ce, engine, ce->ring->tail);return 0 ;drivers/gpu/drm/i915/gt/intel_lrc.h +2
-4@@ -38,12 +38,10 @@ void lrc_destroy(struct kref *kref);int lrc_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr); +struct i915_gem_ww_ctx *ww);int lrc_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr); +struct intel_engine_cs *engine);void lrc_unpin (struct intel_context *ce);void lrc_post_unpin (struct intel_context *ce);drivers/gpu/drm/i915/gt/intel_ring_submission.c +2
-3@@ -472,8 +472,7 @@ static int ring_context_init_default_state(struct intel_context *ce, }static int ring_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **unused) +struct i915_gem_ww_ctx *ww) {struct i915_address_space *vm;int err =0 ;@@ -576,7 +575,7 @@ static int ring_context_alloc(struct intel_context *ce)return 0 ; } -static int ring_context_pin (struct intel_context *ce,void *unused) +static int ring_context_pin (struct intel_context *ce) {return 0 ; }drivers/gpu/drm/i915/gt/mock_engine.c +2
-2@@ -167,12 +167,12 @@ static int mock_context_alloc(struct intel_context *ce) }static int mock_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww,void **unused) +struct i915_gem_ww_ctx *ww) {return 0 ; } -static int mock_context_pin (struct intel_context *ce,void *unused) +static int mock_context_pin (struct intel_context *ce) {return 0 ; }drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +164
-19@@ -1905,6 +1905,7 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)GEM_BUG_ON (!engine->mask);GEM_BUG_ON (context_guc_id_invalid (ce)); +GEM_BUG_ON (intel_context_is_child (ce));/* * Ensure LRC + CT vmas are is same region as write barrier is done @@ -2008,15 +2009,13 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)static int __guc_context_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return lrc_pre_pin (ce, engine, ww, vaddr); +return lrc_pre_pin (ce, engine, ww); }static int __guc_context_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr) +struct intel_engine_cs *engine) {if (i915_ggtt_offset (ce->state) != (ce->lrc.lrca &CTX_GTT_ADDRESS_MASK ))@@ -2027,20 +2026,33 @@ static int __guc_context_pin(struct intel_context *ce,* explaination of why. */ -return lrc_pin (ce, engine, vaddr); +return lrc_pin (ce, engine); +} + +static void __guc_context_unpin (struct intel_context *ce) +{ +lrc_unpin (ce); +} + +static void __guc_context_post_unpin (struct intel_context *ce) +{ +lrc_post_unpin (ce); }static int guc_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return __guc_context_pre_pin (ce, ce->engine, ww, vaddr); +return __guc_context_pre_pin (ce, ce->engine, ww); } -static int guc_context_pin (struct intel_context *ce,void *vaddr) +static int guc_context_pin (struct intel_context *ce) { -int ret =__guc_context_pin (ce, ce->engine, vaddr); +int ret; +GEM_BUG_ON (intel_context_is_parent (ce) || +intel_context_is_child (ce)); + + ret =__guc_context_pin (ce, ce->engine);if (likely (!ret && !intel_context_is_barrier (ce)))intel_engine_pm_get (ce->engine);@@ -2054,7 +2066,7 @@ static void guc_context_unpin(struct intel_context *ce)GEM_BUG_ON (context_enabled (ce));unpin_guc_id (guc, ce, true); -lrc_unpin (ce); +__guc_context_unpin (ce);if (likely (!intel_context_is_barrier (ce)))intel_engine_pm_put (ce->engine);@@ -2062,7 +2074,141 @@ static void guc_context_unpin(struct intel_context *ce)static void guc_context_post_unpin (struct intel_context *ce) { -lrc_post_unpin (ce); +__guc_context_post_unpin (ce); +} + +/* Future patches will use this function */ +__maybe_unused +static int guc_parent_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +struct intel_context *child; +int err, i =0 , j =0 ; + +for_each_child (ce, child) { + err =i915_active_acquire (&child->active); +if (unlikely (err)) + goto unwind_active; + ++i; + } + +for_each_child (ce, child) { + err =__guc_context_pre_pin (child, child->engine, ww); +if (unlikely (err)) + goto unwind_pre_pin; + ++j; + } + + err =__guc_context_pre_pin (ce, ce->engine, ww); +if (unlikely (err)) + goto unwind_pre_pin; + +return 0 ; + +unwind_pre_pin: +for_each_child (ce, child) { +if (!j--) +break ; +__guc_context_post_unpin (child); + } + +unwind_active: +for_each_child (ce, child) { +if (!i--) +break ; +i915_active_release (&child->active); + } + +return err; +} + +/* Future patches will use this function */ +__maybe_unused +static void guc_parent_context_post_unpin (struct intel_context *ce) +{ +struct intel_context *child; + +for_each_child (ce, child) +__guc_context_post_unpin (child); +__guc_context_post_unpin (ce); + +for_each_child (ce, child) { +intel_context_get (child); +i915_active_release (&child->active); +intel_context_put (child); + } +} + +/* Future patches will use this function */ +__maybe_unused +static int guc_parent_context_pin (struct intel_context *ce) +{ +int ret, i =0 , j =0 ; +struct intel_context *child; +struct intel_engine_cs *engine; +intel_engine_mask_t tmp; + +GEM_BUG_ON (!intel_context_is_parent (ce)); + +for_each_child (ce, child) { + ret =__guc_context_pin (child, child->engine); +if (unlikely (ret)) + goto unwind_pin; + ++i; + } + ret =__guc_context_pin (ce, ce->engine); +if (unlikely (ret)) + goto unwind_pin; + +for_each_child (ce, child) +if (test_bit (CONTEXT_LRCA_DIRTY , &child->flags)) { +set_bit (CONTEXT_LRCA_DIRTY , &ce->flags); +break ; + } + +for_each_engine_masked (engine, ce->engine->gt, + ce->engine->mask, tmp) +intel_engine_pm_get (engine); +for_each_child (ce, child) +for_each_engine_masked (engine, child->engine->gt, + child->engine->mask, tmp) +intel_engine_pm_get (engine); + +return 0 ; + +unwind_pin: +for_each_child (ce, child) { +if (++j > i) +break ; +__guc_context_unpin (child); + } + +return ret; +} + +/* Future patches will use this function */ +__maybe_unused +static void guc_parent_context_unpin (struct intel_context *ce) +{ +struct intel_context *child; +struct intel_engine_cs *engine; +intel_engine_mask_t tmp; + +GEM_BUG_ON (!intel_context_is_parent (ce)); +GEM_BUG_ON (context_enabled (ce)); + +unpin_guc_id (ce_to_guc (ce), ce, true); +for_each_child (ce, child) +__guc_context_unpin (child); +__guc_context_unpin (ce); + +for_each_engine_masked (engine, ce->engine->gt, + ce->engine->mask, tmp) +intel_engine_pm_put (engine); +for_each_child (ce, child) +for_each_engine_masked (engine, child->engine->gt, + child->engine->mask, tmp) +intel_engine_pm_put (engine); } static void __guc_context_sched_enable(struct intel_guc *guc,@@ -2993,18 +3139,17 @@ static int guc_request_alloc(struct i915_request *rq) }static int guc_virtual_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) {struct intel_engine_cs *engine =guc_virtual_get_sibling (ce->engine,0 ); -return __guc_context_pre_pin (ce, engine, ww, vaddr); +return __guc_context_pre_pin (ce, engine, ww); } -static int guc_virtual_context_pin (struct intel_context *ce,void *vaddr) +static int guc_virtual_context_pin (struct intel_context *ce) {struct intel_engine_cs *engine =guc_virtual_get_sibling (ce->engine,0 ); -int ret =__guc_context_pin (ce, engine, vaddr); +int ret =__guc_context_pin (ce, engine);intel_engine_mask_t tmp, mask = ce->engine->mask;if (likely (!ret))@@ -3024,7 +3169,7 @@ static void guc_virtual_context_unpin(struct intel_context *ce)GEM_BUG_ON (intel_context_is_barrier (ce));unpin_guc_id (guc, ce, true); -lrc_unpin (ce); +__guc_context_unpin (ce);for_each_engine_masked (engine, ce->engine->gt, mask, tmp)intel_engine_pm_put (engine);Signature · 1 line
2.28.0Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.chSignature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
PATCH 17/46 drm/i915/guc: Add multi-lrc context registration
In reply to Matthew BrostAdd multi-lrc context registration H2G. In addition a workqueue and process descriptor are setup during multi-lrc context registration as these data structures are needed for multi-lrc submission.
--- drivers/gpu/drm/i915/gt/intel_context_types.h | 6 + drivers/gpu/drm/i915/gt/intel_lrc.c | 5 + .../gpu/drm/i915/gt/uc/abi/guc_actions_abi.h | 1 + drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h | 2 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 117 +++++++++++++++++- 5 files changed, 129 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/gt/intel_context_types.h +6 -0
@@ -211,9 +211,15 @@ struct intel_context {/* Pointer to parent */ struct intel_context *parent; +/* GuC workqueue head / tail - only applies to parent */ +u16 guc_wqi_tail; +u16 guc_wqi_head; +/* Number of children if parent */ u8 guc_number_children; +u8 parent_page;/* if set, page num reserved for parent context */ +/* * GuC priority management */
drivers/gpu/drm/i915/gt/intel_lrc.c +5 -0
@@ -861,6 +861,11 @@ __lrc_alloc_state(struct intel_context *ce, struct intel_engine_cs *engine) context_size +=PAGE_SIZE ; } +if (intel_context_is_parent (ce)) { + ce->parent_page = context_size /PAGE_SIZE ; + context_size +=PAGE_SIZE ; + } + obj =i915_gem_object_create_lmem (engine->i915, context_size,0 );if (IS_ERR (obj)) obj =i915_gem_object_create_shmem (engine->i915, context_size);
drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h +1 -0
@@ -142,6 +142,7 @@ enum intel_guc_action {INTEL_GUC_ACTION_REGISTER_COMMAND_TRANSPORT_BUFFER =0x4505 ,INTEL_GUC_ACTION_DEREGISTER_COMMAND_TRANSPORT_BUFFER =0x4506 ,INTEL_GUC_ACTION_DEREGISTER_CONTEXT_DONE =0x4600 , +INTEL_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC =0x4601 ,INTEL_GUC_ACTION_RESET_CLIENT =0x5507 , INTEL_GUC_ACTION_LIMIT };
drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h +1 -1
@@ -51,7 +51,7 @@#define GUC_DOORBELL_INVALID 256 -#define GUC_WQ_SIZE (PAGE_SIZE * 2) +#define GUC_WQ_SIZE (PAGE_SIZE / 2)/* Work queue item header definitions */ #define WQ_STATUS_ACTIVE 1
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +116 -1
174 lines not on this page · raw message
Signature · 1 line
2.28.0
PATCH 18/46 drm/i915/guc: Ensure GuC schedule operations do not operate on child contexts
In reply to Matthew BrostIn GuC parent-child contexts the parent context controls the scheduling, ensure only the parent does the scheduling operations.
--- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 52 +++++++++++++++---- 1 file changed, 41 insertions(+), 11 deletions(-)
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +41 -11
@@ -405,6 +405,18 @@ static inline void decr_context_blocked(struct intel_context *ce) ce->guc_state.sched_state -=SCHED_STATE_BLOCKED ; } +static inline struct intel_context * +to_parent (struct intel_context *ce) +{ +return intel_context_is_child (ce) ? ce->parent : ce; +} + +static inline struct intel_context * +request_to_scheduling_context (struct i915_request *rq) +{ +return to_parent (rq->context); +} +static inline bool context_guc_id_invalid (struct intel_context *ce) {return ce->guc_id ==GUC_INVALID_LRC_ID ;
@@ -711,7 +723,7 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop);static int tasklet_register_context (struct guc_submit_engine *gse,struct i915_request *rq) { -struct intel_context *ce = rq->context; +struct intel_context *ce =request_to_scheduling_context (rq);struct intel_guc *guc = gse->sched_engine.private_data;int ret =0 ;
@@ -720,6 +732,7 @@ static int tasklet_register_context(struct guc_submit_engine *gse,GEM_BUG_ON (ce->guc_num_rq_submit_no_id);GEM_BUG_ON (request_has_no_guc_id (rq));GEM_BUG_ON (context_guc_id_invalid (ce)); +GEM_BUG_ON (intel_context_is_child (ce));GEM_BUG_ON (!atomic_read (&ce->guc_id_ref));/*
@@ -2355,6 +2368,7 @@ static void __guc_context_sched_disable(struct intel_guc *guc,GEM_BUG_ON (guc_id ==GUC_INVALID_LRC_ID );#endif +GEM_BUG_ON (intel_context_is_child (ce));trace_intel_context_sched_disable (ce); guc_submission_send_busy_loop(guc, action, ARRAY_SIZE(action),
@@ -2570,6 +2584,8 @@ static void guc_context_sched_disable(struct intel_context *ce)u16 guc_id;bool enabled; +GEM_BUG_ON (intel_context_is_child (ce)); +if (submission_disabled (guc) ||context_guc_id_invalid (ce) || !lrc_desc_registered (guc, ce->guc_id)) {clr_context_enabled (ce);
@@ -2971,6 +2987,8 @@ static void guc_signal_context_fence(struct intel_context *ce) {unsigned long flags; +GEM_BUG_ON (intel_context_is_child (ce)); +spin_lock_irqsave (&ce->guc_state.lock, flags);clr_context_wait_for_deregister_to_register (ce);__guc_signal_context_fence (ce);
@@ -3056,14 +3074,26 @@ static bool context_needs_lrc_desc_pin(struct intel_context *ce, bool new_guc_id !submission_disabled (ce_to_guc (ce)); } +static void clear_lrca_dirty (struct intel_context *ce) +{ +struct intel_context *child; + +GEM_BUG_ON (intel_context_is_child (ce)); + +clear_bit (CONTEXT_LRCA_DIRTY , &ce->flags); +for_each_child (ce, child) +clear_bit (CONTEXT_LRCA_DIRTY , &child->flags); +} +static int tasklet_pin_guc_id (struct guc_submit_engine *gse,struct i915_request *rq) { -struct intel_context *ce = rq->context; +struct intel_context *ce =request_to_scheduling_context (rq);int ret =0 ;lockdep_assert_held (&gse->sched_engine.lock);GEM_BUG_ON (!ce->guc_num_rq_submit_no_id); +GEM_BUG_ON (intel_context_is_child (ce));if (atomic_add_unless (&ce->guc_id_ref, ce->guc_num_rq_submit_no_id,0 )) goto out;
@@ -3091,7 +3121,7 @@ static int tasklet_pin_guc_id(struct guc_submit_engine *gse, gse->submission_stall_reason =STALL_SCHED_DISABLE ; } -clear_bit (CONTEXT_LRCA_DIRTY , &ce->flags); +clear_lrca_dirty (ce); out: gse->total_num_rq_with_no_guc_id -= ce->guc_num_rq_submit_no_id;GEM_BUG_ON (gse->total_num_rq_with_no_guc_id <0 );
@@ -3122,7 +3152,7 @@ static int tasklet_pin_guc_id(struct guc_submit_engine *gse,static int guc_request_alloc (struct i915_request *rq) { -struct intel_context *ce = rq->context; +struct intel_context *ce =request_to_scheduling_context (rq);struct intel_guc *guc =ce_to_guc (ce);struct guc_submit_engine *gse =ce_to_gse (ce);unsigned long flags;
@@ -3173,11 +3203,12 @@ static int guc_request_alloc(struct i915_request *rq)* persistent until the generated request is retired. Thus, sealing these * race conditions. * -* There is no need for a lock here as the timeline mutex ensures at -* most one context can be executing this code path at once. The -* guc_id_ref is incremented once for every request in flight and -* decremented on each retire. When it is zero, a lock around the -* increment (in pin_guc_id) is needed to seal a race with unpin_guc_id. +* There is no need for a lock here as the timeline mutex (or +* parallel_submit mutex in the case of multi-lrc) ensures at most one +* context can be executing this code path at once. The guc_id_ref is +* incremented once for every request in flight and decremented on each +* retire. When it is zero, a lock around the increment (in pin_guc_id) +* is needed to seal a race with unpin_guc_id. */ if (atomic_add_unless (&ce->guc_id_ref,1 ,0 )) goto out;
@@ -3215,8 +3246,7 @@ static int guc_request_alloc(struct i915_request *rq) } } -clear_bit (CONTEXT_LRCA_DIRTY , &ce->flags); - +clear_lrca_dirty (ce); out:incr_num_rq_not_ready (ce);
Signature · 1 line
2.28.0
PATCH 19/46 drm/i915/guc: Assign contexts in parent-child relationship consecutive guc_ids
In reply to Matthew BrostAssign contexts in parent-child relationship consecutive guc_ids. This is accomplished by partitioning guc_id space between ones that need to be consecutive (1/16 available guc_ids) and ones that do not (15/16 of available guc_ids). The consecutive search is implemented via the bitmap API.
This is a precursor to the full GuC multi-lrc implementation but aligns to how GuC mutli-lrc interface is defined - guc_ids must be consecutive when using the GuC multi-lrc interface.
--- drivers/gpu/drm/i915/gt/intel_context.h | 6 + drivers/gpu/drm/i915/gt/intel_reset.c | 3 +- drivers/gpu/drm/i915/gt/uc/intel_guc.h | 7 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 222 ++++++++++++------ .../i915/gt/uc/intel_guc_submission_types.h | 10 + 5 files changed, 179 insertions(+), 69 deletions(-)
drivers/gpu/drm/i915/gt/intel_context.h +6 -0
@@ -54,6 +54,12 @@ static inline bool intel_context_is_parent(struct intel_context *ce)return !!ce->guc_number_children; } +static inline struct intel_context * +intel_context_to_parent (struct intel_context *ce) +{ +return intel_context_is_child (ce) ? ce->parent : ce; +} +void intel_context_bind_parent_child (struct intel_context *parent,struct intel_context *child);
drivers/gpu/drm/i915/gt/intel_reset.c +2 -1
@@ -849,6 +849,7 @@ static void reset_finish(struct intel_gt *gt, intel_engine_mask_t awake)static void nop_submit_request (struct i915_request *request) { +struct intel_context *ce =intel_context_to_parent (request->context);RQ_TRACE (request,"-EIO\n" );/*
@@ -857,7 +858,7 @@ static void nop_submit_request(struct i915_request *request)* this for now. */ if (intel_engine_uses_guc (request->engine)) -intel_guc_decr_num_rq_not_ready (request->context); +intel_guc_decr_num_rq_not_ready (ce); request =i915_request_mark_eio (request);if (request) {
drivers/gpu/drm/i915/gt/uc/intel_guc.h +5 -2
@@ -24,6 +24,7 @@ struct __guc_ads_blob;enum {GUC_SUBMIT_ENGINE_SINGLE_LRC , +GUC_SUBMIT_ENGINE_MULTI_LRC ,GUC_SUBMIT_ENGINE_MAX };
@@ -59,8 +60,10 @@ struct intel_guc {struct ida guc_ids;u32 num_guc_ids;u32 max_guc_ids; -struct list_head guc_id_list_no_ref; -struct list_head guc_id_list_unpinned; +unsigned long *guc_ids_bitmap; +#define MAX_GUC_ID_ORDER (order_base_2(MAX_ENGINE_INSTANCE + 1)) +struct list_head guc_id_list_no_ref[MAX_GUC_ID_ORDER +1 ]; +struct list_head guc_id_list_unpinned[MAX_GUC_ID_ORDER +1 ];spinlock_t destroy_lock;/* protects list / worker */ struct list_head destroyed_contexts;
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +156 -66
366 lines not on this page · raw message
drivers/gpu/drm/i915/gt/uc/intel_guc_submission_types.h +10 -0
@@ -22,6 +22,16 @@ struct guc_virtual_engine {/* * Object which encapsulates the globally operated on i915_sched_engine + * the GuC submission state machine described in intel_guc_submission.c. +* +* Currently we have two instances of these per GuC. One for single-lrc and one +* for multi-lrc submission. We split these into two submission engines as they +* can operate in parallel allowing a blocking condition on one not to affect +* the other. i.e. guc_ids are statically allocated between these two submission +* modes. One mode may have guc_ids exhausted which requires blocking while the +* other has plenty of guc_ids and can make forward progres. +* +* In the future if different submission use cases arise we can simply +* instantiate another of these objects and assign it to the context. */ struct guc_submit_engine {struct i915_sched_engine sched_engine;
Signature · 1 line
2.28.0
Re: [PATCH 19/46] drm/i915/guc: Assign contexts in parent-child relationship consecutive guc_ids
Quoted 70 lines · Matthew Brost
Assign contexts in parent-child relationship consecutive
guc_ids. This is accomplished by partitioningguc_idspace between ones that need to be consecutive (1/16 availableguc_ids) and ones that do not (15/16 of availableguc_ids). The consecutive search is implemented via the bitmap API.This is a precursor to the full GuC multi-lrc implementation but aligns to how GuC mutli-lrc interface is defined -
guc_idsmust be consecutive when using the GuC multi-lrc interface.Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/i915/gt/intel_context.h | 6 + drivers/gpu/drm/i915/gt/intel_reset.c | 3 +- drivers/gpu/drm/i915/gt/uc/intel_guc.h | 7 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 222 ++++++++++++------ .../i915/gt/uc/intel_guc_submission_types.h | 10 + 5 files changed, 179 insertions(+), 69 deletions(-)drivers/gpu/drm/i915/gt/intel_context.h +6
-0@@ -54,6 +54,12 @@ static inline bool intel_context_is_parent(struct intel_context *ce)return !!ce->guc_number_children; } +static inline struct intel_context * +intel_context_to_parent (struct intel_context *ce) +{ +return intel_context_is_child (ce) ? ce->parent : ce; +} +void intel_context_bind_parent_child (struct intel_context *parent,struct intel_context *child);drivers/gpu/drm/i915/gt/intel_reset.c +2
-1@@ -849,6 +849,7 @@ static void reset_finish(struct intel_gt *gt, intel_engine_mask_t awake)static void nop_submit_request (struct i915_request *request) { +struct intel_context *ce =intel_context_to_parent (request->context);RQ_TRACE (request,"-EIO\n" );/* @@ -857,7 +858,7 @@ static void nop_submit_request(struct i915_request *request)* this for now. */ if (intel_engine_uses_guc (request->engine)) -intel_guc_decr_num_rq_not_ready (request->context); +intel_guc_decr_num_rq_not_ready (ce); request =i915_request_mark_eio (request);if (request) {drivers/gpu/drm/i915/gt/uc/intel_guc.h +1
-0@@ -24,6 +24,7 @@ struct __guc_ads_blob;enum {GUC_SUBMIT_ENGINE_SINGLE_LRC , +GUC_SUBMIT_ENGINE_MULTI_LRC ,GUC_SUBMIT_ENGINE_MAX };
drivers/gpu/drm/i915/gt/uc/intel_guc.h struct intel_guc { · line 66
Quoted 9 lines
drivers/gpu/drm/i915/gt/uc/intel_guc.h +4
-2@@ -59,8 +60,10 @@ struct intel_guc {struct ida guc_ids;u32 num_guc_ids;u32 max_guc_ids; -struct list_head guc_id_list_no_ref; -struct list_head guc_id_list_unpinned; +unsigned long *guc_ids_bitmap; +#define MAX_GUC_ID_ORDER (order_base_2(MAX_ENGINE_INSTANCE + 1)) +struct list_head guc_id_list_no_ref[MAX_GUC_ID_ORDER +1 ]; +struct list_head guc_id_list_unpinned[MAX_GUC_ID_ORDER +1 ];
Random new global lists definitely need kerneldoc about what is on them, how they're linked, what their lifetime rules are and what locks we're holding.
Leaving this all to reviews to figure out, and worse, future readers of your code, is not kind.
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c static void clr_guc_ids_exhausted(struct guc_submit_engine *gse) · line 173
Quoted 12 lines
drivers/gpu/drm/i915/gt/uc/intel_guc.h +0
-0spinlock_t destroy_lock;/* protects list / worker */ struct list_head destroyed_contexts;drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +2
-0@@ -169,6 +169,15 @@ static void clr_guc_ids_exhausted(struct guc_submit_engine *gse)clear_bit (GSE_STATE_GUC_IDS_EXHAUSTED , &gse->flags); } +/* +* We reserve 1/16 of the guc_ids for multi-lrc as these need to be contiguous
I think it'd be good to put down the reason here for why. Is this a requirement of the guc interface, or just an artifact of our current implementation? In the latter case also explain what exactly the contstraint is (but honestly I can't think of much reasons for that) -Daniel
Quoted 400 lines
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +154
-66+* and a different allocation algorithm is used (bitmap vs. ida). We believe the +* number of multi-lrc contexts in use should be low and 1/16 should be +* sufficient. Minimum of 32 ids for multi-lrc. +*/ +#define NUMBER_MULTI_LRC_GUC_ID (guc) \ + ((guc)->num_guc_ids / 16 > 32 ? (guc)->num_guc_ids / 16 : 32) +/* * Below is a set of functions which control the GuC scheduling state which do * not require a lock as all state transitions are mutually exclusive. i.e. It @@ -405,16 +414,10 @@ static inline void decr_context_blocked(struct intel_context *ce) ce->guc_state.sched_state -=SCHED_STATE_BLOCKED ; } -static inline struct intel_context * -to_parent (struct intel_context *ce) -{ -return intel_context_is_child (ce) ? ce->parent : ce; -} -static inline struct intel_context *request_to_scheduling_context (struct i915_request *rq) { -return to_parent (rq->context); +return intel_context_to_parent (rq->context); }static inline bool context_guc_id_invalid (struct intel_context *ce)@@ -1436,7 +1439,7 @@ static void destroy_worker_func(struct work_struct *w);*/ int intel_guc_submission_init (struct intel_guc *guc) { -int ret; +int ret, i;if (guc_submission_initialized (guc))return 0 ;@@ -1448,9 +1451,13 @@ int intel_guc_submission_init(struct intel_guc *guc)xa_init_flags (&guc->context_lookup,XA_FLAGS_LOCK_IRQ );spin_lock_init (&guc->contexts_lock); -INIT_LIST_HEAD (&guc->guc_id_list_no_ref); -INIT_LIST_HEAD (&guc->guc_id_list_unpinned); +for (i =0 ; i <MAX_GUC_ID_ORDER +1 ; ++i) { +INIT_LIST_HEAD (&guc->guc_id_list_no_ref[i]); +INIT_LIST_HEAD (&guc->guc_id_list_unpinned[i]); + }ida_init (&guc->guc_ids); + guc->guc_ids_bitmap = +bitmap_zalloc (NUMBER_MULTI_LRC_GUC_ID (guc),GFP_KERNEL );spin_lock_init (&guc->destroy_lock);@@ -1476,6 +1483,8 @@ void intel_guc_submission_fini(struct intel_guc *guc)i915_sched_engine_put (sched_engine); } + +bitmap_free (guc->guc_ids_bitmap); } static inline void queue_request(struct i915_sched_engine *sched_engine,@@ -1499,11 +1508,13 @@ static inline void queue_request(struct i915_sched_engine *sched_engine,static bool too_many_guc_ids_not_ready (struct guc_submit_engine *gse,struct intel_context *ce) { -u32 available_guc_ids, guc_ids_consumed;struct intel_guc *guc = gse->sched_engine.private_data; +u32 available_guc_ids =intel_context_is_parent (ce) ? +NUMBER_MULTI_LRC_GUC_ID (guc) : + guc->num_guc_ids -NUMBER_MULTI_LRC_GUC_ID (guc); +u32 guc_ids_consumed =atomic_read (&gse->num_guc_ids_not_ready); - available_guc_ids = guc->num_guc_ids; - guc_ids_consumed =atomic_read (&gse->num_guc_ids_not_ready); +GEM_BUG_ON (intel_context_is_child (ce));if (TOO_MANY_GUC_IDS_NOT_READY (available_guc_ids, guc_ids_consumed)) {set_and_update_guc_ids_exhausted (gse);@@ -1517,17 +1528,26 @@ static void incr_num_rq_not_ready(struct intel_context *ce) {struct guc_submit_engine *gse =ce_to_gse (ce); +GEM_BUG_ON (intel_context_is_child (ce)); +GEM_BUG_ON (!intel_context_is_parent (ce) && + ce->guc_number_children); +if (!atomic_fetch_add (1 , &ce->guc_num_rq_not_ready)) -atomic_inc (&gse->num_guc_ids_not_ready); +atomic_add (ce->guc_number_children +1 , + &gse->num_guc_ids_not_ready); }void intel_guc_decr_num_rq_not_ready (struct intel_context *ce) {struct guc_submit_engine *gse =ce_to_gse (ce); +GEM_BUG_ON (intel_context_is_child (ce)); +if (atomic_fetch_add (-1 , &ce->guc_num_rq_not_ready) ==1 ) {GEM_BUG_ON (!atomic_read (&gse->num_guc_ids_not_ready)); -atomic_dec (&gse->num_guc_ids_not_ready); + +atomic_sub (ce->guc_number_children +1 , + &gse->num_guc_ids_not_ready); } }@@ -1579,20 +1599,42 @@ static void guc_submit_request(struct i915_request *rq)spin_unlock_irqrestore (&sched_engine->lock, flags); -intel_guc_decr_num_rq_not_ready (rq->context); +intel_guc_decr_num_rq_not_ready (request_to_scheduling_context (rq)); } -static int new_guc_id (struct intel_guc *guc) +static int new_guc_id (struct intel_guc *guc,struct intel_context *ce) { -return ida_simple_get (&guc->guc_ids,0 , - guc->num_guc_ids,GFP_KERNEL | - __GFP_RETRY_MAYFAIL | __GFP_NOWARN); +int ret; + +GEM_BUG_ON (intel_context_is_child (ce)); + +if (intel_context_is_parent (ce)) + ret =bitmap_find_free_region (guc->guc_ids_bitmap, +NUMBER_MULTI_LRC_GUC_ID (guc), +order_base_2 (ce->guc_number_children + +1 )); +else + ret =ida_simple_get (&guc->guc_ids, +NUMBER_MULTI_LRC_GUC_ID (guc), + guc->num_guc_ids,GFP_KERNEL | + __GFP_RETRY_MAYFAIL | __GFP_NOWARN); +if (unlikely (ret <0 )) +return ret; + + ce->guc_id = ret; +return 0 ; }static void __release_guc_id (struct intel_guc *guc,struct intel_context *ce) { +GEM_BUG_ON (intel_context_is_child (ce));if (!context_guc_id_invalid (ce)) { -ida_simple_remove (&guc->guc_ids, ce->guc_id); +if (intel_context_is_parent (ce)) +bitmap_release_region (guc->guc_ids_bitmap, ce->guc_id, +order_base_2 (ce->guc_number_children + +1 )); +else +ida_simple_remove (&guc->guc_ids, ce->guc_id);clr_lrc_desc_registered (guc, ce->guc_id);set_context_guc_id_invalid (ce); }@@ -1604,6 +1646,8 @@ static void release_guc_id(struct intel_guc *guc, struct intel_context *ce) {unsigned long flags; +GEM_BUG_ON (intel_context_is_child (ce)); +spin_lock_irqsave (&guc->contexts_lock, flags);__release_guc_id (guc, ce);spin_unlock_irqrestore (&guc->contexts_lock, flags);@@ -1618,54 +1662,93 @@ static void release_guc_id(struct intel_guc *guc, struct intel_context *ce)* schedule disable H2G + a deregister H2G. */ static struct list_head *get_guc_id_list (struct intel_guc *guc, +u8 number_children,bool unpinned) { +GEM_BUG_ON (order_base_2 (number_children +1 ) >MAX_GUC_ID_ORDER ); +if (unpinned) -return &guc->guc_id_list_unpinned; +return &guc->guc_id_list_unpinned[order_base_2 (number_children +1 )];else -return &guc->guc_id_list_no_ref; +return &guc->guc_id_list_no_ref[order_base_2 (number_children +1 )]; } -static int steal_guc_id (struct intel_guc *guc,bool unpinned) +static int steal_guc_id (struct intel_guc *guc,struct intel_context *ce, +bool unpinned) { -struct intel_context *ce; -int guc_id; -struct list_head *guc_id_list =get_guc_id_list (guc, unpinned); +struct intel_context *cn; +u8 number_children = ce->guc_number_children;lockdep_assert_held (&guc->contexts_lock); +GEM_BUG_ON (intel_context_is_child (ce)); -if (!list_empty (guc_id_list)) { - ce =list_first_entry (guc_id_list, - struct intel_context, - guc_id_link); +do { +struct list_head *guc_id_list = +get_guc_id_list (guc, number_children, unpinned); -/* Ensure context getting stolen in expected state */ -GEM_BUG_ON (atomic_read (&ce->guc_id_ref)); -GEM_BUG_ON (context_guc_id_invalid (ce)); -GEM_BUG_ON (context_guc_id_stolen (ce)); +if (!list_empty (guc_id_list)) { +u8 cn_o2, ce_o2 = +order_base_2 (ce->guc_number_children +1 ); -list_del_init (&ce->guc_id_link); - guc_id = ce->guc_id; -clr_context_registered (ce); + cn =list_first_entry (guc_id_list, + struct intel_context, + guc_id_link); + cn_o2 =order_base_2 (cn->guc_number_children +1 ); + +/* +* Corner case where a multi-lrc context steals a guc_id +* from another context that has more guc_id that itself. +*/ +if (cn_o2 != ce_o2) { +bitmap_release_region (guc->guc_ids_bitmap, + cn->guc_id, + cn_o2); +bitmap_allocate_region (guc->guc_ids_bitmap, + ce->guc_id, + ce_o2); + } + +/* Ensure context getting stolen in expected state */ +GEM_BUG_ON (atomic_read (&cn->guc_id_ref)); +GEM_BUG_ON (context_guc_id_invalid (cn)); +GEM_BUG_ON (context_guc_id_stolen (cn)); +GEM_BUG_ON (ce_to_gse (ce) !=ce_to_gse (cn)); + +list_del_init (&cn->guc_id_link); + ce->guc_id = cn->guc_id; + +/* +* If stealing from the pinned list, defer invalidating +* the guc_id until the retire workqueue processes this +* context. +*/ +clr_context_registered (cn); +if (!unpinned) { +GEM_BUG_ON (ce_to_gse (cn)->stalled_context); +ce_to_gse (cn)->stalled_context = +intel_context_get (cn); +set_context_guc_id_stolen (cn); + }else { +set_context_guc_id_invalid (cn); + } + +return 0 ; + }/* -* If stealing from the pinned list, defer invalidating -* the guc_id until the retire workqueue processes this -* context. +* When using multi-lrc we search the guc_id_lists with the +* least amount of guc_ids required first but will consume a +* block larger of guc_ids if necessary. 2x the children always +* moves you two the next list. */ -if (!unpinned) { -GEM_BUG_ON (ce_to_gse (ce)->stalled_context); +if (!number_children || +order_base_2 (number_children +1 ) ==MAX_GUC_ID_ORDER ) +break ; -ce_to_gse (ce)->stalled_context =intel_context_get (ce); -set_context_guc_id_stolen (ce); - }else { -set_context_guc_id_invalid (ce); - } + number_children *=2 ; + }while (true); -return guc_id; - }else { -return -EAGAIN ; - } +return -EAGAIN ; }enum {/* Return values for pin_guc_id / assign_guc_id */ @@ -1674,17 +1757,18 @@ enum { /* Return values for pin_guc_id / assign_guc_id */ NEW_GUC_ID_ENABLED = 2, }; -static int assign_guc_id (struct intel_guc *guc,u16 *out,bool tasklet) +static int assign_guc_id (struct intel_guc *guc,struct intel_context *ce, +bool tasklet) {int ret;lockdep_assert_held (&guc->contexts_lock); +GEM_BUG_ON (intel_context_is_child (ce)); - ret =new_guc_id (guc); + ret =new_guc_id (guc, ce);if (unlikely (ret <0 )) { - ret =steal_guc_id (guc, true); -if (ret >=0 ) { - *out = ret; + ret =steal_guc_id (guc, ce, true); +if (!ret) { ret =NEW_GUC_ID_DISABLED ; }else if (ret <0 && tasklet) {/* @@ -1692,15 +1776,18 @@ static int assign_guc_id(struct intel_guc *guc, u16 *out, bool tasklet)* enabled if guc_ids are exhausted and we are submitting * from the tasklet. */ - ret =steal_guc_id (guc, false); -if (ret >=0 ) { - *out = ret; + ret =steal_guc_id (guc, ce, false); +if (!ret) ret =NEW_GUC_ID_ENABLED ; - } } - }else { - *out = ret; - ret =SAME_GUC_ID ; + } + +if (!(ret <0 ) &&intel_context_is_parent (ce)) { +struct intel_context *child; +int i =1 ; + +for_each_child (ce, child) + child->guc_id = ce->guc_id + i++; }return ret;@@ -1713,6 +1800,7 @@ static int pin_guc_id(struct intel_guc *guc, struct intel_context *ce,int ret =0 ;unsigned long flags, tries =PIN_GUC_ID_TRIES ; +GEM_BUG_ON (intel_context_is_child (ce));GEM_BUG_ON (atomic_read (&ce->guc_id_ref)); try_again:@@ -1724,7 +1812,7 @@ static int pin_guc_id(struct intel_guc *guc, struct intel_context *ce, }if (context_guc_id_invalid (ce)) { - ret =assign_guc_id (guc, &ce->guc_id, tasklet); + ret =assign_guc_id (guc, ce, tasklet);if (unlikely (ret <0 )) goto out_unlock; }@@ -1770,6 +1858,7 @@ static void unpin_guc_id(struct intel_guc *guc,unsigned long flags;GEM_BUG_ON (atomic_read (&ce->guc_id_ref) <0 ); +GEM_BUG_ON (intel_context_is_child (ce));if (unlikely (context_guc_id_invalid (ce)))return ;@@ -1781,7 +1870,8 @@ static void unpin_guc_id(struct intel_guc *guc,if (!context_guc_id_invalid (ce) && !context_guc_id_stolen (ce) && !atomic_read (&ce->guc_id_ref)) { -struct list_head *head =get_guc_id_list (guc, unpinned); +struct list_head *head = +get_guc_id_list (guc, ce->guc_number_children, unpinned);list_add_tail (&ce->guc_id_link, head); }drivers/gpu/drm/i915/gt/uc/intel_guc_submission_types.h +10
-0@@ -22,6 +22,16 @@ struct guc_virtual_engine {/* * Object which encapsulates the globally operated on i915_sched_engine + * the GuC submission state machine described in intel_guc_submission.c. +* +* Currently we have two instances of these per GuC. One for single-lrc and one +* for multi-lrc submission. We split these into two submission engines as they +* can operate in parallel allowing a blocking condition on one not to affect +* the other. i.e. guc_ids are statically allocated between these two submission +* modes. One mode may have guc_ids exhausted which requires blocking while the +* other has plenty of guc_ids and can make forward progres. +* +* In the future if different submission use cases arise we can simply +* instantiate another of these objects and assign it to the context. */ struct guc_submit_engine {struct i915_sched_engine sched_engine;Signature · 1 line
2.28.0
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch