Parallel submission aka multi-bb execbuf
Subsystems (3)
cover Parallel submission aka multi-bb execbuf
As discussed in [1] we are introducing a new parallel submission uAPI for the i915 which allows more than 1 BB to be submitted in an execbuf IOCTL. This is the implemenation for both GuC and execlists.
This series is broken into 5 parts.
1. Basic GuC submission. Patch 1 which the squashed version of series [2].
2. A series of GuC patches which introduces a state machine to deal with flow control conditions gracefully (e.g. don't punt them to the user). These are patches 2-13.
3. Update the GuC backend / connections to uAPI to configure it for parallel submission. These are patches 14-29.
4. Update execbuf IOCTL to accept more than 1 BB in a single IOCTL. These are patches 30-41.
5. A weak execlists implemenation for parallel submission. Patch 42.
Looking for initial feedback all parts except #1.
[1] https://patchwork.freedesktop.org/series/92028/
[2] https://patchwork.freedesktop.org/series/91840/
Matthew Brost (42):
drm/i915/guc: GuC submission squashed into single patch drm/i915/guc: Allow flexible number of context ids drm/i915/guc: Connect the number of guc_ids to debugfs drm/i915/guc: Don't return -EAGAIN to user when guc_ids exhausted drm/i915/guc: Don't allow requests not ready to consume all guc_ids drm/i915/guc: Introduce guc_submit_engine object drm/i915/guc: Check return of __xa_store when registering a context drm/i915/guc: Non-static lrc descriptor registration buffer drm/i915/guc: Take GT PM ref when deregistering context drm/i915: Add GT PM unpark worker
drm/i915/guc: Take engine PM when a context is pinned with GuC
submission
drm/i915/guc: Don't call switch_to_kernel_context with GuC submission drm/i915/guc: Selftest for GuC flow control
drm/i915: Add logical engine mapping
drm/i915: Expose logical engine instance to user drm/i915/guc: Introduce context parent-child relationship drm/i915/guc: Implement GuC parent-child context pin / unpin functions drm/i915/guc: Add multi-lrc context registration drm/i915/guc: Ensure GuC schedule operations do not operate on child
contexts
drm/i915/guc: Assign contexts in parent-child relationship consecutive
guc_ids
drm/i915/guc: Add hang check to GuC submit engine drm/i915/guc: Add guc_child_context_destroy
drm/i915/guc: Implement multi-lrc submission
drm/i915/guc: Insert submit fences between requests in parent-child
relationship
drm/i915/guc: Implement multi-lrc reset
drm/i915/guc: Update debugfs for GuC multi-lrc
drm/i915: Connect UAPI to GuC multi-lrc interface drm/i915/guc: Add basic GuC multi-lrc selftest drm/i915/guc: Implement BB boundary preemption for multi-lrc i915/drm: Move secure execbuf check to execbuf2
drm/i915: Move input/exec fence handling to i915_gem_execbuffer2 drm/i915: Move output fence handling to i915_gem_execbuffer2 drm/i915: Return output fence from i915_gem_do_execbuffer drm/i915: Store batch index in struct i915_execbuffer drm/i915: Allow callers of i915_gem_do_execbuffer to override the
batch index
drm/i915: Teach execbuf there can be more than one batch in the
objects list
drm/i915: Only track object dependencies on first request drm/i915: Force parallel contexts to use copy engine for reloc drm/i915: Multi-batch execbuffer2
drm/i915: Eliminate unnecessary VMA calls for multi-BB submission drm/i915: Enable multi-bb execbuf
drm/i915/execlists: Parallel submission support for execlists
drivers/gpu/drm/i915/Makefile | 2 + drivers/gpu/drm/i915/gem/i915_gem_context.c | 176 +- .../gpu/drm/i915/gem/i915_gem_context_types.h | 6 + .../gpu/drm/i915/gem/i915_gem_execbuffer.c | 541 +- drivers/gpu/drm/i915/gem/i915_gem_mman.c | 3 +- .../i915/gem/selftests/i915_gem_execbuffer.c | 14 +- drivers/gpu/drm/i915/gt/gen8_engine_cs.c | 6 +- drivers/gpu/drm/i915/gt/intel_breadcrumbs.c | 44 +- drivers/gpu/drm/i915/gt/intel_breadcrumbs.h | 16 +- .../gpu/drm/i915/gt/intel_breadcrumbs_types.h | 7 + drivers/gpu/drm/i915/gt/intel_context.c | 275 +- drivers/gpu/drm/i915/gt/intel_context.h | 82 +- drivers/gpu/drm/i915/gt/intel_context_types.h | 115 +- drivers/gpu/drm/i915/gt/intel_engine.h | 67 +- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 242 +- .../gpu/drm/i915/gt/intel_engine_heartbeat.c | 71 +- .../gpu/drm/i915/gt/intel_engine_heartbeat.h | 4 + drivers/gpu/drm/i915/gt/intel_engine_pm.c | 4 + drivers/gpu/drm/i915/gt/intel_engine_pm.h | 5 + drivers/gpu/drm/i915/gt/intel_engine_types.h | 14 +- drivers/gpu/drm/i915/gt/intel_engine_user.c | 4 + .../drm/i915/gt/intel_execlists_submission.c | 313 +- .../drm/i915/gt/intel_execlists_submission.h | 4 - drivers/gpu/drm/i915/gt/intel_gt.c | 24 + drivers/gpu/drm/i915/gt/intel_gt.h | 2 + drivers/gpu/drm/i915/gt/intel_gt_pm.c | 14 +- drivers/gpu/drm/i915/gt/intel_gt_pm.h | 13 + .../gpu/drm/i915/gt/intel_gt_pm_unpark_work.c | 35 + .../gpu/drm/i915/gt/intel_gt_pm_unpark_work.h | 32 + drivers/gpu/drm/i915/gt/intel_gt_requests.c | 21 +- drivers/gpu/drm/i915/gt/intel_gt_requests.h | 9 +- drivers/gpu/drm/i915/gt/intel_gt_types.h | 3 + drivers/gpu/drm/i915/gt/intel_lrc.c | 23 +- drivers/gpu/drm/i915/gt/intel_lrc.h | 6 +- drivers/gpu/drm/i915/gt/intel_lrc_reg.h | 1 - drivers/gpu/drm/i915/gt/intel_reset.c | 58 +- .../gpu/drm/i915/gt/intel_ring_submission.c | 47 +- drivers/gpu/drm/i915/gt/intel_rps.c | 4 + drivers/gpu/drm/i915/gt/intel_workarounds.c | 46 +- .../gpu/drm/i915/gt/intel_workarounds_types.h | 1 + drivers/gpu/drm/i915/gt/mock_engine.c | 38 +- drivers/gpu/drm/i915/gt/selftest_context.c | 10 + .../drm/i915/gt/selftest_engine_heartbeat.c | 22 + .../drm/i915/gt/selftest_engine_heartbeat.h | 2 + drivers/gpu/drm/i915/gt/selftest_execlists.c | 12 +- drivers/gpu/drm/i915/gt/selftest_hangcheck.c | 314 +- drivers/gpu/drm/i915/gt/selftest_mocs.c | 50 +- .../gpu/drm/i915/gt/selftest_workarounds.c | 132 +- .../gpu/drm/i915/gt/uc/abi/guc_actions_abi.h | 16 + drivers/gpu/drm/i915/gt/uc/intel_guc.c | 82 +- drivers/gpu/drm/i915/gt/uc/intel_guc.h | 126 +- drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c | 465 +- drivers/gpu/drm/i915/gt/uc/intel_guc_ads.h | 4 + drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 175 +- drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h | 25 +- .../gpu/drm/i915/gt/uc/intel_guc_debugfs.c | 56 +- drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h | 98 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 4961 +++++++++++++++-- .../gpu/drm/i915/gt/uc/intel_guc_submission.h | 20 +- .../i915/gt/uc/intel_guc_submission_types.h | 67 + drivers/gpu/drm/i915/gt/uc/intel_uc.c | 101 +- drivers/gpu/drm/i915/gt/uc/intel_uc.h | 11 + .../i915/gt/uc/selftest_guc_flow_control.c | 581 ++ .../drm/i915/gt/uc/selftest_guc_multi_lrc.c | 168 + drivers/gpu/drm/i915/i915_debugfs_params.c | 31 + drivers/gpu/drm/i915/i915_gem_evict.c | 1 + drivers/gpu/drm/i915/i915_gpu_error.c | 25 +- drivers/gpu/drm/i915/i915_query.c | 2 + drivers/gpu/drm/i915/i915_reg.h | 2 + drivers/gpu/drm/i915/i915_request.c | 288 +- drivers/gpu/drm/i915/i915_request.h | 43 + drivers/gpu/drm/i915/i915_scheduler.c | 36 +- drivers/gpu/drm/i915/i915_scheduler.h | 13 +- drivers/gpu/drm/i915/i915_scheduler_types.h | 22 + drivers/gpu/drm/i915/i915_trace.h | 199 +- drivers/gpu/drm/i915/i915_vma.c | 13 +- drivers/gpu/drm/i915/i915_vma.h | 16 +- drivers/gpu/drm/i915/intel_wakeref.c | 5 + drivers/gpu/drm/i915/intel_wakeref.h | 1 + .../drm/i915/selftests/i915_live_selftests.h | 2 + drivers/gpu/drm/i915/selftests/i915_request.c | 4 +- .../gpu/drm/i915/selftests/igt_flush_test.c | 2 +- .../gpu/drm/i915/selftests/igt_live_test.c | 2 +- .../i915/selftests/intel_scheduler_helpers.c | 101 + .../i915/selftests/intel_scheduler_helpers.h | 36 + .../gpu/drm/i915/selftests/mock_gem_device.c | 3 +- include/uapi/drm/i915_drm.h | 135 +- 87 files changed, 9648 insertions(+), 1224 deletions(-) create mode 100644 drivers/gpu/drm/i915/gt/intel_gt_pm_unpark_work.c create mode 100644 drivers/gpu/drm/i915/gt/intel_gt_pm_unpark_work.h create mode 100644 drivers/gpu/drm/i915/gt/uc/intel_guc_submission_types.h create mode 100644 drivers/gpu/drm/i915/gt/uc/selftest_guc_flow_control.c create mode 100644 drivers/gpu/drm/i915/gt/uc/selftest_guc_multi_lrc.c create mode 100644 drivers/gpu/drm/i915/selftests/intel_scheduler_helpers.c create mode 100644 drivers/gpu/drm/i915/selftests/intel_scheduler_helpers.h
Signature · 1 line
2.28.0
RFC PATCH 1/42 drm/i915/guc: GuC submission squashed into single patch
https://patchwork.freedesktop.org/series/91840/
--- drivers/gpu/drm/i915/Makefile | 1 + drivers/gpu/drm/i915/gem/i915_gem_context.c | 21 +- drivers/gpu/drm/i915/gem/i915_gem_mman.c | 3 +- drivers/gpu/drm/i915/gt/gen8_engine_cs.c | 6 +- drivers/gpu/drm/i915/gt/intel_breadcrumbs.c | 44 +- drivers/gpu/drm/i915/gt/intel_breadcrumbs.h | 16 +- .../gpu/drm/i915/gt/intel_breadcrumbs_types.h | 7 + drivers/gpu/drm/i915/gt/intel_context.c | 50 +- drivers/gpu/drm/i915/gt/intel_context.h | 50 +- drivers/gpu/drm/i915/gt/intel_context_types.h | 63 +- drivers/gpu/drm/i915/gt/intel_engine.h | 57 +- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 182 +- .../gpu/drm/i915/gt/intel_engine_heartbeat.c | 71 +- .../gpu/drm/i915/gt/intel_engine_heartbeat.h | 4 + drivers/gpu/drm/i915/gt/intel_engine_types.h | 13 +- drivers/gpu/drm/i915/gt/intel_engine_user.c | 4 + .../drm/i915/gt/intel_execlists_submission.c | 89 +- .../drm/i915/gt/intel_execlists_submission.h | 4 - drivers/gpu/drm/i915/gt/intel_gt.c | 21 + drivers/gpu/drm/i915/gt/intel_gt.h | 2 + drivers/gpu/drm/i915/gt/intel_gt_pm.c | 6 +- drivers/gpu/drm/i915/gt/intel_gt_requests.c | 21 +- drivers/gpu/drm/i915/gt/intel_gt_requests.h | 9 +- drivers/gpu/drm/i915/gt/intel_lrc_reg.h | 1 - drivers/gpu/drm/i915/gt/intel_reset.c | 50 +- .../gpu/drm/i915/gt/intel_ring_submission.c | 42 + drivers/gpu/drm/i915/gt/intel_rps.c | 4 + drivers/gpu/drm/i915/gt/intel_workarounds.c | 46 +- .../gpu/drm/i915/gt/intel_workarounds_types.h | 1 + drivers/gpu/drm/i915/gt/mock_engine.c | 34 +- drivers/gpu/drm/i915/gt/selftest_context.c | 10 + .../drm/i915/gt/selftest_engine_heartbeat.c | 22 + .../drm/i915/gt/selftest_engine_heartbeat.h | 2 + drivers/gpu/drm/i915/gt/selftest_execlists.c | 12 +- drivers/gpu/drm/i915/gt/selftest_hangcheck.c | 314 +- drivers/gpu/drm/i915/gt/selftest_mocs.c | 50 +- .../gpu/drm/i915/gt/selftest_workarounds.c | 132 +- .../gpu/drm/i915/gt/uc/abi/guc_actions_abi.h | 15 + drivers/gpu/drm/i915/gt/uc/intel_guc.c | 82 +- drivers/gpu/drm/i915/gt/uc/intel_guc.h | 101 +- drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c | 463 ++- drivers/gpu/drm/i915/gt/uc/intel_guc_ads.h | 4 + drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 132 +- drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h | 16 +- .../gpu/drm/i915/gt/uc/intel_guc_debugfs.c | 25 +- drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h | 88 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2848 +++++++++++++++-- .../gpu/drm/i915/gt/uc/intel_guc_submission.h | 18 +- drivers/gpu/drm/i915/gt/uc/intel_uc.c | 101 +- drivers/gpu/drm/i915/gt/uc/intel_uc.h | 11 + drivers/gpu/drm/i915/i915_debugfs_params.c | 31 + drivers/gpu/drm/i915/i915_gem_evict.c | 1 + drivers/gpu/drm/i915/i915_gpu_error.c | 25 +- drivers/gpu/drm/i915/i915_reg.h | 2 + drivers/gpu/drm/i915/i915_request.c | 173 +- drivers/gpu/drm/i915/i915_request.h | 29 + drivers/gpu/drm/i915/i915_scheduler.c | 16 +- drivers/gpu/drm/i915/i915_scheduler.h | 10 +- drivers/gpu/drm/i915/i915_scheduler_types.h | 22 + drivers/gpu/drm/i915/i915_trace.h | 199 +- drivers/gpu/drm/i915/selftests/i915_request.c | 4 +- .../gpu/drm/i915/selftests/igt_flush_test.c | 2 +- .../gpu/drm/i915/selftests/igt_live_test.c | 2 +- .../i915/selftests/intel_scheduler_helpers.c | 89 + .../i915/selftests/intel_scheduler_helpers.h | 35 + .../gpu/drm/i915/selftests/mock_gem_device.c | 3 +- include/uapi/drm/i915_drm.h | 9 + 67 files changed, 5122 insertions(+), 898 deletions(-) create mode 100644 drivers/gpu/drm/i915/selftests/intel_scheduler_helpers.c create mode 100644 drivers/gpu/drm/i915/selftests/intel_scheduler_helpers.h
drivers/gpu/drm/i915/Makefile +1 -0
@@ -280,6 +280,7 @@ i915-$(CONFIG_DRM_I915_CAPTURE_ERROR) += i915_gpu_error.o i915-$(CONFIG_DRM_I915_SELFTEST) += \ gem/selftests/i915_gem_client_blt.o \ gem/selftests/igt_gem_utils.o \ + selftests/intel_scheduler_helpers.o \ selftests/i915_random.o \ selftests/i915_selftest.o \ selftests/igt_atomic.o \
drivers/gpu/drm/i915/gem/i915_gem_context.c +12 -9
@@ -74,7 +74,6 @@#include "gt/intel_context_param.h" #include "gt/intel_engine_heartbeat.h" #include "gt/intel_engine_user.h" -#include "gt/intel_execlists_submission.h" /* virtual_engine */ #include "gt/intel_gpu_commands.h" #include "gt/intel_ring.h"
@@ -363,9 +362,6 @@ set_proto_ctx_engines_balance(struct i915_user_extension __user *base,if (!HAS_EXECLISTS (i915))return -ENODEV ; -if (intel_uc_uses_guc_submission (&i915->gt.uc)) -return -ENODEV ;/* not implement yet */ -if (get_user (idx, &ext->engine_index))return -EFAULT ;
@@ -495,6 +491,11 @@ set_proto_ctx_engines_bond(struct i915_user_extension __user *base, void *data)return -EINVAL ; } +if (intel_engine_uses_guc (master)) { +DRM_DEBUG ("bonding extension not supported with GuC submission" ); +return -ENODEV ; + } +if (get_user (num_bonds, &ext->num_bonds))return -EFAULT ;
@@ -799,7 +800,8 @@ static int intel_context_set_gem(struct intel_context *ce, }if (ctx->sched.priority >=I915_PRIORITY_NORMAL && -intel_engine_has_timeslices (ce->engine)) +intel_engine_has_timeslices (ce->engine) && +intel_engine_has_semaphores (ce->engine))__set_bit (CONTEXT_USE_SEMAPHORES , &ce->flags); if (IS_ACTIVE(CONFIG_DRM_I915_REQUEST_TIMEOUT) &&
@@ -949,8 +951,8 @@ static struct i915_gem_engines *user_engines(struct i915_gem_context *ctx,break ;case I915_GEM_ENGINE_TYPE_BALANCED : - ce =intel_execlists_create_virtual (pe[n].siblings, - pe[n].num_siblings); + ce =intel_engine_create_virtual (pe[n].siblings, + pe[n].num_siblings);break ;case I915_GEM_ENGINE_TYPE_INVALID :
@@ -1082,7 +1084,7 @@ static void kill_engines(struct i915_gem_engines *engines, bool ban)for_each_gem_engine (ce, engines, it) {struct intel_engine_cs *engine; -if (ban &&intel_context_set_banned (ce)) +if (ban &&intel_context_ban (ce,NULL ))continue ;/*
@@ -1778,7 +1780,8 @@ static void __apply_priority(struct intel_context *ce, void *arg)if (!intel_engine_has_timeslices (ce->engine))return ; -if (ctx->sched.priority >=I915_PRIORITY_NORMAL ) +if (ctx->sched.priority >=I915_PRIORITY_NORMAL && +intel_engine_has_semaphores (ce->engine))intel_context_set_use_semaphores (ce);else intel_context_clear_use_semaphores (ce);
drivers/gpu/drm/i915/gem/i915_gem_mman.c +2 -1
@@ -645,7 +645,8 @@ mmap_offset_attach(struct drm_i915_gem_object *obj, goto insert;/* Attempt to reap some mmap space from dead objects */ - err =intel_gt_retire_requests_timeout (&i915->gt,MAX_SCHEDULE_TIMEOUT ); + err =intel_gt_retire_requests_timeout (&i915->gt,MAX_SCHEDULE_TIMEOUT , +NULL );if (err) goto err;
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +4 -2
@@ -506,7 +506,8 @@ gen8_emit_fini_breadcrumb_tail(struct i915_request *rq, u32 *cs) *cs++ =MI_USER_INTERRUPT ; *cs++ =MI_ARB_ON_OFF |MI_ARB_ENABLE ; -if (intel_engine_has_semaphores (rq->engine)) +if (intel_engine_has_semaphores (rq->engine) && + !intel_uc_uses_guc_submission (&rq->engine->gt->uc)) cs =emit_preempt_busywait (rq, cs); rq->tail =intel_ring_offset (rq, cs);
@@ -598,7 +599,8 @@ gen12_emit_fini_breadcrumb_tail(struct i915_request *rq, u32 *cs) *cs++ =MI_USER_INTERRUPT ; *cs++ =MI_ARB_ON_OFF |MI_ARB_ENABLE ; -if (intel_engine_has_semaphores (rq->engine)) +if (intel_engine_has_semaphores (rq->engine) && + !intel_uc_uses_guc_submission (&rq->engine->gt->uc)) cs =gen12_emit_preempt_busywait (rq, cs); rq->tail =intel_ring_offset (rq, cs);
drivers/gpu/drm/i915/gt/intel_breadcrumbs.c +20 -24
101 lines not on this page · raw message
drivers/gpu/drm/i915/gt/intel_breadcrumbs.h +14 -2
@@ -9,7 +9,7 @@#include <linux/atomic.h> #include <linux/irq_work.h> -#include "intel_engine_types.h" +#include "intel_breadcrumbs_types.h" struct drm_printer ;struct i915_request ;
@@ -17,7 +17,7 @@ struct intel_breadcrumbs;struct intel_breadcrumbs *intel_breadcrumbs_create (struct intel_engine_cs *irq_engine); -void intel_breadcrumbs_free (struct intel_breadcrumbs *b); +void intel_breadcrumbs_free (struct kref *kref);void intel_breadcrumbs_reset (struct intel_breadcrumbs *b);void __intel_breadcrumbs_park (struct intel_breadcrumbs *b);
@@ -48,4 +48,16 @@ void i915_request_cancel_breadcrumb(struct i915_request *request);void intel_context_remove_breadcrumbs (struct intel_context *ce,struct intel_breadcrumbs *b); +static inline struct intel_breadcrumbs * +intel_breadcrumbs_get (struct intel_breadcrumbs *b) +{ +kref_get (&b->ref); +return b; +} + +static inline void intel_breadcrumbs_put (struct intel_breadcrumbs *b) +{ +kref_put (&b->ref, intel_breadcrumbs_free); +} +#endif /* __INTEL_BREADCRUMBS__ */
drivers/gpu/drm/i915/gt/intel_breadcrumbs_types.h +7 -0
@@ -7,10 +7,13 @@#define __INTEL_BREADCRUMBS_TYPES__#include <linux/irq_work.h> +#include <linux/kref.h> #include <linux/list.h> #include <linux/spinlock.h> #include <linux/types.h> +#include "intel_engine_types.h" +/* * Rather than have every client wait upon all user interrupts, * with the herd waking after every interrupt and each doing the
@@ -29,6 +32,7 @@* the overhead of waking that client is much preferred. */ struct intel_breadcrumbs { +struct kref ref;atomic_t active;spinlock_t signalers_lock;/* protects the list of signalers */
@@ -42,7 +46,10 @@ struct intel_breadcrumbs {bool irq_armed;/* Not all breadcrumbs are attached to physical HW */ +intel_engine_mask_t engine_mask;struct intel_engine_cs *irq_engine; +bool (*irq_enable)(struct intel_breadcrumbs *b); + void (*irq_disable)(struct intel_breadcrumbs *b); };#endif /* __INTEL_BREADCRUMBS_TYPES__ */
drivers/gpu/drm/i915/gt/intel_context.c +47 -3
111 lines not on this page · raw message
drivers/gpu/drm/i915/gt/intel_context.h +49 -1
80 lines not on this page · raw message
drivers/gpu/drm/i915/gt/intel_context_types.h +62 -1
102 lines not on this page · raw message
drivers/gpu/drm/i915/gt/intel_engine.h +55 -2
86 lines not on this page · raw message
drivers/gpu/drm/i915/gt/intel_engine_cs.c +137 -45
234 lines not on this page · raw message
drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c +58 -13
95 lines not on this page · raw message
drivers/gpu/drm/i915/gt/intel_engine_heartbeat.h +4 -0
@@ -7,6 +7,7 @@#define INTEL_ENGINE_HEARTBEAT_H struct intel_engine_cs ; +struct intel_gt ;void intel_engine_init_heartbeat (struct intel_engine_cs *engine);
@@ -16,6 +17,9 @@ int intel_engine_set_heartbeat(struct intel_engine_cs *engine,void intel_engine_park_heartbeat (struct intel_engine_cs *engine);void intel_engine_unpark_heartbeat (struct intel_engine_cs *engine); +void intel_gt_park_heartbeats (struct intel_gt *gt); +void intel_gt_unpark_heartbeats (struct intel_gt *gt); +int intel_engine_pulse (struct intel_engine_cs *engine);int intel_engine_flush_barriers (struct intel_engine_cs *engine);
drivers/gpu/drm/i915/gt/intel_engine_types.h +12 -1
@@ -21,7 +21,6 @@#include "i915_pmu.h" #include "i915_priolist_types.h" #include "i915_selftest.h" -#include "intel_breadcrumbs_types.h" #include "intel_sseu.h" #include "intel_timeline_types.h" #include "intel_uncore.h"
@@ -63,6 +62,7 @@ struct i915_sched_engine;struct intel_gt ;struct intel_ring ;struct intel_uncore ; +struct intel_breadcrumbs ;typedef u8 intel_engine_mask_t ;#define ALL_ENGINES ((intel_engine_mask_t)~0ul)
@@ -304,6 +304,8 @@ struct intel_engine_cs {/* keep a request in reserve for a [pm] barrier under oom */ struct i915_request *request_pool; +struct intel_context *hung_ce; +struct llist_head barrier_tasks;struct intel_context *kernel_context;/* pinned */
@@ -388,6 +390,8 @@ struct intel_engine_cs {void (*park)(struct intel_engine_cs *engine);void (*unpark)(struct intel_engine_cs *engine); +void (*bump_serial)(struct intel_engine_cs *engine); +void (*set_default_submission)(struct intel_engine_cs *engine);const struct intel_context_ops *cops;
@@ -418,6 +422,12 @@ struct intel_engine_cs {void (*release)(struct intel_engine_cs *engine); +/* +* Add / remove request from engine active tracking +*/ +void (*add_active_request)(struct i915_request *rq); +void (*remove_active_request)(struct i915_request *rq); +struct intel_engine_execlists execlists;/*
@@ -439,6 +449,7 @@ struct intel_engine_cs {#define I915_ENGINE_IS_VIRTUAL BIT(5)#define I915_ENGINE_HAS_RELATIVE_MMIO BIT(6)#define I915_ENGINE_REQUIRES_CMD_PARSER BIT(7) +#define I915_ENGINE_WANT_FORCED_PREEMPTION BIT(8)unsigned int flags;/*
drivers/gpu/drm/i915/gt/intel_engine_user.c +4 -0
@@ -11,6 +11,7 @@#include "intel_engine.h" #include "intel_engine_user.h" #include "intel_gt.h" +#include "uc/intel_guc_submission.h" struct intel_engine_cs *intel_engine_lookup_user (struct drm_i915_private *i915,u8 class,u8 instance)
@@ -115,6 +116,9 @@ static void set_scheduler_caps(struct drm_i915_private *i915) disabled |= (I915_SCHEDULER_CAP_ENABLED |I915_SCHEDULER_CAP_PRIORITY ); +if (intel_uc_uses_guc_submission (&i915->gt.uc)) + enabled |=I915_SCHEDULER_CAP_STATIC_PRIORITY_MAP ; +for (i =0 ; i <ARRAY_SIZE (map); i++) {if (engine->flags &BIT (map[i].engine)) enabled |=BIT (map[i].sched);
drivers/gpu/drm/i915/gt/intel_execlists_submission.c +79 -10
170 lines not on this page · raw message
drivers/gpu/drm/i915/gt/intel_execlists_submission.h +0 -4
@@ -32,10 +32,6 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine, int indent), unsigned int max); -struct intel_context * -intel_execlists_create_virtual (struct intel_engine_cs **siblings, -unsigned int count); -bool intel_engine_in_execlists_submission_mode (const struct intel_engine_cs *engine);
drivers/gpu/drm/i915/gt/intel_gt.c +21 -0
@@ -585,6 +585,25 @@ static void __intel_gt_disable(struct intel_gt *gt)GEM_BUG_ON (intel_gt_pm_is_awake (gt)); } +int intel_gt_wait_for_idle (struct intel_gt *gt,long timeout) +{ +long remaining_timeout; + +/* If the device is asleep, we have no requests outstanding */ +if (!intel_gt_pm_is_awake (gt)) +return 0 ; + +while ((timeout =intel_gt_retire_requests_timeout (gt, timeout, + &remaining_timeout)) >0 ) { +cond_resched (); +if (signal_pending (current)) +return -EINTR ; + } + +return timeout ? timeout :intel_uc_wait_for_idle (>->uc, + remaining_timeout); +} +int intel_gt_init (struct intel_gt *gt) {int err;
@@ -635,6 +654,8 @@ int intel_gt_init(struct intel_gt *gt)if (err) goto err_gt; +intel_uc_init_late (>->uc); + err =i915_inject_probe_error (gt->i915, -EIO );if (err) goto err_gt;
drivers/gpu/drm/i915/gt/intel_gt.h +2 -0
@@ -48,6 +48,8 @@ void intel_gt_driver_release(struct intel_gt *gt);void intel_gt_driver_late_release (struct intel_gt *gt); +int intel_gt_wait_for_idle (struct intel_gt *gt,long timeout); +void intel_gt_check_and_clear_faults (struct intel_gt *gt);void intel_gt_clear_error_registers (struct intel_gt *gt,intel_engine_mask_t engine_mask);
drivers/gpu/drm/i915/gt/intel_gt_pm.c +4 -2
@@ -174,8 +174,6 @@ static void gt_sanitize(struct intel_gt *gt, bool force)if (intel_gt_is_wedged (gt))intel_gt_unset_wedged (gt); -intel_uc_sanitize (>->uc); -for_each_engine (engine, gt, id)if (engine->reset.prepare) engine->reset.prepare (engine);
@@ -191,6 +189,8 @@ static void gt_sanitize(struct intel_gt *gt, bool force)__intel_engine_reset (engine, false); } +intel_uc_reset (>->uc, false); +for_each_engine (engine, gt, id)if (engine->reset.finish) engine->reset.finish (engine);
@@ -243,6 +243,8 @@ int intel_gt_resume(struct intel_gt *gt) goto err_wedged; } +intel_uc_reset_finish (>->uc); +intel_rps_enable (>->rps);intel_llc_enable (>->llc);
drivers/gpu/drm/i915/gt/intel_gt_requests.c +5 -16
@@ -130,7 +130,8 @@ void intel_engine_fini_retire(struct intel_engine_cs *engine)GEM_BUG_ON (engine->retire); } -long intel_gt_retire_requests_timeout (struct intel_gt *gt,long timeout) +long intel_gt_retire_requests_timeout (struct intel_gt *gt,long timeout, +long *remaining_timeout) {struct intel_gt_timelines *timelines = >->timelines;struct intel_timeline *tl, *tn;
@@ -195,22 +196,10 @@ out_active: spin_lock(&timelines->lock);if (flush_submission (gt, timeout))/* Wait, there's more! */ active_count++; -return active_count ? timeout :0 ; -} - -int intel_gt_wait_for_idle (struct intel_gt *gt,long timeout) -{ -/* If the device is asleep, we have no requests outstanding */ -if (!intel_gt_pm_is_awake (gt)) -return 0 ; - -while ((timeout =intel_gt_retire_requests_timeout (gt, timeout)) >0 ) { -cond_resched (); -if (signal_pending (current)) -return -EINTR ; - } +if (remaining_timeout) + *remaining_timeout = timeout; -return timeout; +return active_count ? timeout :0 ; }static void retire_work_handler (struct work_struct *work)
drivers/gpu/drm/i915/gt/intel_gt_requests.h +5 -4
@@ -6,14 +6,17 @@#ifndef INTEL_GT_REQUESTS_H #define INTEL_GT_REQUESTS_H +#include <stddef.h> +struct intel_engine_cs ;struct intel_gt ;struct intel_timeline ; -long intel_gt_retire_requests_timeout (struct intel_gt *gt,long timeout); +long intel_gt_retire_requests_timeout (struct intel_gt *gt,long timeout, +long *remaining_timeout);static inline void intel_gt_retire_requests (struct intel_gt *gt) { -intel_gt_retire_requests_timeout (gt,0 ); +intel_gt_retire_requests_timeout (gt,0 ,NULL ); }void intel_engine_init_retire (struct intel_engine_cs *engine);
@@ -21,8 +24,6 @@ void intel_engine_add_retire(struct intel_engine_cs *engine, struct intel_timeline *tl);void intel_engine_fini_retire (struct intel_engine_cs *engine); -int intel_gt_wait_for_idle (struct intel_gt *gt,long timeout); -void intel_gt_init_requests (struct intel_gt *gt);void intel_gt_park_requests (struct intel_gt *gt);void intel_gt_unpark_requests (struct intel_gt *gt);
drivers/gpu/drm/i915/gt/intel_lrc_reg.h +0 -1
@@ -87,7 +87,6 @@#define GEN11_CSB_WRITE_PTR_MASK (GEN11_CSB_PTR_MASK << 0)#define MAX_CONTEXT_HW_ID (1 << 21)/* exclusive */ -#define MAX_GUC_CONTEXT_HW_ID (1 << 20)/* exclusive */ #define GEN11_MAX_CONTEXT_HW_ID (1 << 11)/* exclusive */ /* in Gen12 ID 0x7FF is reserved to indicate idle */ #define GEN12_MAX_CONTEXT_HW_ID (GEN11_MAX_CONTEXT_HW_ID - 1)
drivers/gpu/drm/i915/gt/intel_reset.c +20 -30
132 lines not on this page · raw message
drivers/gpu/drm/i915/gt/intel_ring_submission.c +42 -0
63 lines not on this page · raw message
drivers/gpu/drm/i915/gt/intel_rps.c +4 -0
@@ -1877,6 +1877,10 @@ void intel_rps_init(struct intel_rps *rps)if (GRAPHICS_VER (i915) >=8 &&GRAPHICS_VER (i915) <11 ) rps->pm_intrmsk_mbz |=GEN8_PMINTR_DISABLE_REDIRECT_TO_GUC ; + +/* GuC needs ARAT expired interrupt unmasked */ +if (intel_uc_uses_guc_submission (&rps_to_gt (rps)->uc)) + rps->pm_intrmsk_mbz |=ARAT_EXPIRED_INTRMSK ; }void intel_rps_sanitize (struct intel_rps *rps)
drivers/gpu/drm/i915/gt/intel_workarounds.c +24 -22
129 lines not on this page · raw message
drivers/gpu/drm/i915/gt/intel_workarounds_types.h +1 -0
@@ -15,6 +15,7 @@ struct i915_wa {u32 clr;u32 set;u32 read; +bool masked_reg; };struct i915_wa_list {
drivers/gpu/drm/i915/gt/mock_engine.c +32 -2
@@ -235,6 +235,34 @@ static void mock_submit_request(struct i915_request *request)spin_unlock_irqrestore (&engine->hw_lock, flags); } +static void mock_add_to_engine (struct i915_request *rq) +{ +lockdep_assert_held (&rq->engine->sched_engine->lock); +list_move_tail (&rq->sched.link, &rq->engine->sched_engine->requests); +} + +static void mock_remove_from_engine (struct i915_request *rq) +{ +struct intel_engine_cs *engine, *locked; + +/* +* Virtual engines complicate acquiring the engine timeline lock, +* as their rq->engine pointer is not stable until under that +* engine lock. The simple ploy we use is to take the lock then +* check that the rq still belongs to the newly locked engine. +*/ + + locked =READ_ONCE (rq->engine); +spin_lock_irq (&locked->sched_engine->lock); +while (unlikely (locked != (engine =READ_ONCE (rq->engine)))) { +spin_unlock (&locked->sched_engine->lock); +spin_lock (&engine->sched_engine->lock); + locked = engine; + } +list_del_init (&rq->sched.link); +spin_unlock_irq (&locked->sched_engine->lock); +} +static void mock_reset_prepare (struct intel_engine_cs *engine) { }
@@ -284,7 +312,7 @@ static void mock_engine_release(struct intel_engine_cs *engine)GEM_BUG_ON (timer_pending (&mock->hw_delay));i915_sched_engine_put (engine->sched_engine); -intel_breadcrumbs_free (engine->breadcrumbs); +intel_breadcrumbs_put (engine->breadcrumbs);intel_context_unpin (engine->kernel_context);intel_context_put (engine->kernel_context);
@@ -321,6 +349,8 @@ struct intel_engine_cs *mock_engine(struct drm_i915_private *i915, engine->base.emit_flush = mock_emit_flush; engine->base.emit_fini_breadcrumb = mock_emit_breadcrumb; engine->base.submit_request = mock_submit_request; + engine->base.add_active_request = mock_add_to_engine; + engine->base.remove_active_request = mock_remove_from_engine; engine->base.reset.prepare = mock_reset_prepare; engine->base.reset.rewind = mock_reset_rewind;
@@ -370,7 +400,7 @@ int mock_engine_init(struct intel_engine_cs *engine)return 0 ; err_breadcrumbs: -intel_breadcrumbs_free (engine->breadcrumbs); +intel_breadcrumbs_put (engine->breadcrumbs); err_schedule:i915_sched_engine_put (engine->sched_engine);return -ENOMEM ;
drivers/gpu/drm/i915/gt/selftest_context.c +10 -0
@@ -209,7 +209,13 @@ static int __live_active_context(struct intel_engine_cs *engine)* This test makes sure that the context is kept alive until a * subsequent idle-barrier (emitted when the engine wakeref hits 0 * with no more outstanding requests). +* +* In GuC submission mode we don't use idle barriers and we instead +* get a message from the GuC to signal that it is safe to unpin the +* context from memory. */ +if (intel_engine_uses_guc (engine)) +return 0 ;if (intel_engine_pm_is_awake (engine)) { pr_err("%s is awake before starting %s!\n",
@@ -357,7 +363,11 @@ static int __live_remote_context(struct intel_engine_cs *engine)* on the context image remotely (intel_context_prepare_remote_request), * which inserts foreign fences into intel_context.active, does not * clobber the idle-barrier. +* +* In GuC submission mode we don't use idle barriers. */ +if (intel_engine_uses_guc (engine)) +return 0 ;if (intel_engine_pm_is_awake (engine)) { pr_err("%s is awake before starting %s!\n",
drivers/gpu/drm/i915/gt/selftest_engine_heartbeat.c +22 -0
@@ -405,3 +405,25 @@ void st_engine_heartbeat_enable(struct intel_engine_cs *engine) engine->props.heartbeat_interval_ms = engine->defaults.heartbeat_interval_ms; } + +void st_engine_heartbeat_disable_no_pm (struct intel_engine_cs *engine) +{ + engine->props.heartbeat_interval_ms =0 ; + +/* +* Park the heartbeat but without holding the PM lock as that +* makes the engines appear not-idle. Note that if/when unpark +* is called due to the PM lock being acquired later the +* heartbeat still won't be enabled because of the above = 0. +*/ +if (intel_engine_pm_get_if_awake (engine)) { +intel_engine_park_heartbeat (engine); +intel_engine_pm_put (engine); + } +} + +void st_engine_heartbeat_enable_no_pm (struct intel_engine_cs *engine) +{ + engine->props.heartbeat_interval_ms = + engine->defaults.heartbeat_interval_ms; +}
drivers/gpu/drm/i915/gt/selftest_engine_heartbeat.h +2 -0
@@ -9,6 +9,8 @@struct intel_engine_cs ;void st_engine_heartbeat_disable (struct intel_engine_cs *engine); +void st_engine_heartbeat_disable_no_pm (struct intel_engine_cs *engine);void st_engine_heartbeat_enable (struct intel_engine_cs *engine); +void st_engine_heartbeat_enable_no_pm (struct intel_engine_cs *engine);#endif /* SELFTEST_ENGINE_HEARTBEAT_H */
drivers/gpu/drm/i915/gt/selftest_execlists.c +6 -6
@@ -3727,7 +3727,7 @@ static int nop_virtual_engine(struct intel_gt *gt,GEM_BUG_ON (!nctx || nctx >ARRAY_SIZE (ve));for (n =0 ; n < nctx; n++) { - ve[n] =intel_execlists_create_virtual (siblings, nsibling); + ve[n] =intel_engine_create_virtual (siblings, nsibling);if (IS_ERR (ve[n])) { err =PTR_ERR (ve[n]); nctx = n;
@@ -3923,7 +3923,7 @@ static int mask_virtual_engine(struct intel_gt *gt,* restrict it to our desired engine within the virtual engine. */ - ve =intel_execlists_create_virtual (siblings, nsibling); + ve =intel_engine_create_virtual (siblings, nsibling);if (IS_ERR (ve)) { err =PTR_ERR (ve); goto out_close;
@@ -4054,7 +4054,7 @@ static int slicein_virtual_engine(struct intel_gt *gt,i915_request_add (rq); } - ce =intel_execlists_create_virtual (siblings, nsibling); + ce =intel_engine_create_virtual (siblings, nsibling);if (IS_ERR (ce)) { err =PTR_ERR (ce); goto out;
@@ -4106,7 +4106,7 @@ static int sliceout_virtual_engine(struct intel_gt *gt,/* XXX We do not handle oversubscription and fairness with normal rq */ for (n =0 ; n < nsibling; n++) { - ce =intel_execlists_create_virtual (siblings, nsibling); + ce =intel_engine_create_virtual (siblings, nsibling);if (IS_ERR (ce)) { err =PTR_ERR (ce); goto out;
@@ -4208,7 +4208,7 @@ static int preserved_virtual_engine(struct intel_gt *gt,if (err) goto out_scratch; - ve =intel_execlists_create_virtual (siblings, nsibling); + ve =intel_engine_create_virtual (siblings, nsibling);if (IS_ERR (ve)) { err =PTR_ERR (ve); goto out_scratch;
@@ -4348,7 +4348,7 @@ static int reset_virtual_engine(struct intel_gt *gt,if (igt_spinner_init (&spin, gt))return -ENOMEM ; - ve =intel_execlists_create_virtual (siblings, nsibling); + ve =intel_engine_create_virtual (siblings, nsibling);if (IS_ERR (ve)) { err =PTR_ERR (ve); goto out_spin;
drivers/gpu/drm/i915/gt/selftest_hangcheck.c +242 -72
670 lines not on this page · raw message
drivers/gpu/drm/i915/gt/selftest_mocs.c +36 -14
98 lines not on this page · raw message
drivers/gpu/drm/i915/gt/selftest_workarounds.c +98 -34
216 lines not on this page · raw message
drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h +15 -0
@@ -124,10 +124,25 @@ enum intel_guc_action {INTEL_GUC_ACTION_FORCE_LOG_BUFFER_FLUSH =0x302 ,INTEL_GUC_ACTION_ENTER_S_STATE =0x501 ,INTEL_GUC_ACTION_EXIT_S_STATE =0x502 , +INTEL_GUC_ACTION_GLOBAL_SCHED_POLICY_CHANGE =0x506 , +INTEL_GUC_ACTION_SCHED_CONTEXT =0x1000 , +INTEL_GUC_ACTION_SCHED_CONTEXT_MODE_SET =0x1001 , +INTEL_GUC_ACTION_SCHED_CONTEXT_MODE_DONE =0x1002 , +INTEL_GUC_ACTION_SCHED_ENGINE_MODE_SET =0x1003 , +INTEL_GUC_ACTION_SCHED_ENGINE_MODE_DONE =0x1004 , +INTEL_GUC_ACTION_SET_CONTEXT_PRIORITY =0x1005 , +INTEL_GUC_ACTION_SET_CONTEXT_EXECUTION_QUANTUM =0x1006 , +INTEL_GUC_ACTION_SET_CONTEXT_PREEMPTION_TIMEOUT =0x1007 , +INTEL_GUC_ACTION_CONTEXT_RESET_NOTIFICATION =0x1008 , +INTEL_GUC_ACTION_ENGINE_FAILURE_NOTIFICATION =0x1009 ,INTEL_GUC_ACTION_SLPC_REQUEST =0x3003 ,INTEL_GUC_ACTION_AUTHENTICATE_HUC =0x4000 , +INTEL_GUC_ACTION_REGISTER_CONTEXT =0x4502 , +INTEL_GUC_ACTION_DEREGISTER_CONTEXT =0x4503 ,INTEL_GUC_ACTION_REGISTER_COMMAND_TRANSPORT_BUFFER =0x4505 ,INTEL_GUC_ACTION_DEREGISTER_COMMAND_TRANSPORT_BUFFER =0x4506 , +INTEL_GUC_ACTION_DEREGISTER_CONTEXT_DONE =0x4600 , +INTEL_GUC_ACTION_RESET_CLIENT =0x5B01 ,INTEL_GUC_ACTION_LIMIT };
drivers/gpu/drm/i915/gt/uc/intel_guc.c +31 -51
109 lines not on this page · raw message
drivers/gpu/drm/i915/gt/uc/intel_guc.h +95 -6
16 lines not on this page · raw message
drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c +431 -32
463 lines not on this page · raw message
drivers/gpu/drm/i915/gt/uc/intel_guc_ads.h +4 -0
Lines of this file are not shown. View the raw message.
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +114 -18
132 lines not on this page · raw message
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +15 -1
Lines of this file are not shown. View the raw message.
drivers/gpu/drm/i915/gt/uc/intel_guc_debugfs.c +24 -1
Lines of this file are not shown. View the raw message.
drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h +34 -54
88 lines not on this page · raw message
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +2567 -281
2,848 lines not on this page · raw message
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.h +16 -2
Lines of this file are not shown. View the raw message.
drivers/gpu/drm/i915/gt/uc/intel_uc.c +71 -30
101 lines not on this page · raw message
drivers/gpu/drm/i915/i915_debugfs_params.c +31 -0
Lines of this file are not shown. View the raw message.
drivers/gpu/drm/i915/i915_request.c +89 -84
173 lines not on this page · raw message
drivers/gpu/drm/i915/i915_scheduler_types.h +22 -0
Lines of this file are not shown. View the raw message.
drivers/gpu/drm/i915/i915_trace.h +195 -4
199 lines not on this page · raw message
drivers/gpu/drm/i915/selftests/i915_request.c +2 -2
Lines of this file are not shown. View the raw message.
drivers/gpu/drm/i915/selftests/igt_flush_test.c +1 -1
Lines of this file are not shown. View the raw message.
drivers/gpu/drm/i915/selftests/igt_live_test.c +1 -1
Lines of this file are not shown. View the raw message.
drivers/gpu/drm/i915/selftests/intel_scheduler_helpers.c new file +89 -0
89 lines not on this page · raw message
drivers/gpu/drm/i915/selftests/intel_scheduler_helpers.h new file +35 -0
Lines of this file are not shown. View the raw message.
drivers/gpu/drm/i915/selftests/mock_gem_device.c +2 -1
Lines of this file are not shown. View the raw message.
The rest of this diff is not shown. View the raw message.
Signature · 6 lines
2.28.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Part of the diff is not shown here. View the raw message.
kernel test robot
build botRe: [Intel-gfx] [RFC PATCH 01/42] drm/i915/guc: GuC submission squashed into single patch
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]
Hi Matthew,
[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on drm-tip/drm-tip]
[cannot apply to drm-intel/for-linux-next drm-exynos/exynos-drm-next tegra-drm/drm/tegra/for-next drm/drm-next v5.14-rc3 next-20210727] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Matthew-Brost/Parallel-submission-aka-multi-bb-execbuf/20210721-044431 base: git://anongit.freedesktop.org/drm/drm-tip drm-tip config: x86_64-randconfig-a004-20210721 (attached as .config) compiler: clang version 13.0.0 (https://github.com/llvm/llvm-project c781eb153bfbd1b52b03efe34f56bbeccbb8aba6)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# install x86_64 cross compiling tool for clang build
# apt-get install binutils-x86-64-linux-gnu
# https://github.com/0day-ci/linux/commit/b0216b025067b85590916d67e9ea1311f6f068c1
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Matthew-Brost/Parallel-submission-aka-multi-bb-execbuf/20210721-044431 git checkout b0216b025067b85590916d67e9ea1311f6f068c1
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=x86_64
If you fix the issue, kindly add following tag as appropriate
All errors (new ones prefixed by >>):
In file included from drivers/gpu/drm/i915/gt/intel_reset.c:1508:
>> drivers/gpu/drm/i915/gt/selftest_hangcheck.c:464:62: error: variable 'err' is uninitialized when used here [-Werror,-Wuninitialized]
pr_err("[%s] Create context failed: %d!\n", engine->name, err);
^~~
include/linux/printk.h:390:33: note: expanded from macro 'pr_err'
printk(KERN_ERR pr_fmt(fmt), ##__VA_ARGS__)
^~~~~~~~~~~
drivers/gpu/drm/i915/gt/selftest_hangcheck.c:452:10: note: initialize the variable 'err' to silence this warning
int err;
^
= 0
drivers/gpu/drm/i915/gt/selftest_hangcheck.c:579:62: error: variable 'err' is uninitialized when used here [-Werror,-Wuninitialized]
pr_err("[%s] Create context failed: %d!\n", engine->name, err);
^~~
include/linux/printk.h:390:33: note: expanded from macro 'pr_err'
printk(KERN_ERR pr_fmt(fmt), ##__VA_ARGS__)
^~~~~~~~~~~
drivers/gpu/drm/i915/gt/selftest_hangcheck.c:571:10: note: initialize the variable 'err' to silence this warning
int err;
^
= 02 errors generated.
vim +/err +464 drivers/gpu/drm/i915/gt/selftest_hangcheck.c
435
436 static int igt_reset_nop_engine(void *arg)
437 {
438 struct intel_gt *gt = arg;
439 struct i915_gpu_error *global = >->i915->gpu_error;
440 struct intel_engine_cs *engine;
441 enum intel_engine_id id;
442
443 /* Check that we can engine-reset during non-user portions */
444
445 if (!intel_has_reset_engine(gt))
446 return 0;
447
448 for_each_engine(engine, gt, id) {
449 unsigned int reset_count, reset_engine_count, count;
450 struct intel_context *ce;
451 IGT_TIMEOUT(end_time);
452 int err;
453
454 if (intel_engine_uses_guc(engine)) {
455 /* Engine level resets are triggered by GuC when a hang
456 * is detected. They can't be triggered by the KMD any
457 * more. Thus a nop batch cannot be used as a reset test
458 */
459 continue;
460 }
461
462 ce = intel_context_create(engine);
463 if (IS_ERR(ce)) {
> 464 pr_err("[%s] Create context failed: %d!\n", engine->name, err);
465 return PTR_ERR(ce);
466 }
467
468 reset_count = i915_reset_count(global);
469 reset_engine_count = i915_reset_engine_count(global, engine);
470 count = 0;
471
472 st_engine_heartbeat_disable(engine);
473 set_bit(I915_RESET_ENGINE + id, >->reset.flags);
474 do {
475 int i;
476
477 if (!wait_for_idle(engine)) {
478 pr_err("%s failed to idle before reset\n",
479 engine->name);
480 err = -EIO;
481 break;
482 }
483
484 for (i = 0; i < 16; i++) {
485 struct i915_request *rq;
486
487 rq = intel_context_create_request(ce);
488 if (IS_ERR(rq)) {
489 struct drm_printer p =
490 drm_info_printer(gt->i915->drm.dev);
491 intel_engine_dump(engine, &p,
492 "%s(%s): failed to submit request\n",
493 __func__,
494 engine->name); 495
496 GEM_TRACE("%s(%s): failed to submit request\n",
497 __func__,
498 engine->name);
499 GEM_TRACE_DUMP();
500
501 intel_gt_set_wedged(gt);
502
503 err = PTR_ERR(rq);
504 break;
505 }
506
507 i915_request_add(rq);
508 }
509 err = intel_engine_reset(engine, NULL);
510 if (err) {
511 pr_err("intel_engine_reset(%s) failed, err:%d\n",
512 engine->name, err);
513 break;
514 }
515
516 if (i915_reset_count(global) != reset_count) {
517 pr_err("Full GPU reset recorded! (engine reset expected)\n");
518 err = -EINVAL;
519 break;
520 }
521
522 if (i915_reset_engine_count(global, engine) !=
523 reset_engine_count + ++count) {
524 pr_err("%s engine reset not recorded!\n",
525 engine->name);
526 err = -EINVAL;
527 break;
528 }
529 } while (time_before(jiffies, end_time));
530 clear_bit(I915_RESET_ENGINE + id, >->reset.flags);
531 st_engine_heartbeat_enable(engine);
532
533 pr_info("%s(%s): %d resets\n", __func__, engine->name, count);
534
535 intel_context_put(ce);
536 if (igt_flush_test(gt->i915))
537 err = -EIO;
538 if (err)
539 return err;
540 }
541
542 return 0;
543 }
544
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
RFC PATCH 2/42 drm/i915/guc: Allow flexible number of context ids
In reply to Matthew BrostNumber of available GuC contexts ids might be limited. Stop refering in code to macro and use variable instead.
--- drivers/gpu/drm/i915/gt/uc/intel_guc.h | 2 ++ drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c | 15 ++++++++------- 2 files changed, 10 insertions(+), 7 deletions(-)
drivers/gpu/drm/i915/gt/uc/intel_guc.h +2 -0
@@ -53,6 +53,8 @@ struct intel_guc {*/ spinlock_t contexts_lock;struct ida guc_ids; +u32 num_guc_ids; +u32 max_guc_ids;struct list_head guc_id_list;bool submission_supported;
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +8 -7
@@ -272,7 +272,7 @@ static struct guc_lrc_desc *__get_lrc_desc(struct intel_guc *guc, u32 index) {struct guc_lrc_desc *base = guc->lrc_desc_pool_vaddr; -GEM_BUG_ON (index >=GUC_MAX_LRC_DESCRIPTORS ); +GEM_BUG_ON (index >= guc->max_guc_ids);return &base[index]; }
@@ -281,7 +281,7 @@ static inline struct intel_context *__get_context(struct intel_guc *guc, u32 id) {struct intel_context *ce =xa_load (&guc->context_lookup, id); -GEM_BUG_ON (id >=GUC_MAX_LRC_DESCRIPTORS ); +GEM_BUG_ON (id >= guc->max_guc_ids);return ce; }
@@ -291,8 +291,7 @@ static int guc_lrc_desc_pool_create(struct intel_guc *guc)u32 size;int ret; - size =PAGE_ALIGN (sizeof (struct guc_lrc_desc ) * -GUC_MAX_LRC_DESCRIPTORS ); + size =PAGE_ALIGN (sizeof (struct guc_lrc_desc ) * guc->max_guc_ids); ret =intel_guc_allocate_and_map_vma (guc, size, &guc->lrc_desc_pool, (void **)&guc->lrc_desc_pool_vaddr);if (ret)
@@ -1060,7 +1059,7 @@ static void guc_submit_request(struct i915_request *rq)static int new_guc_id (struct intel_guc *guc) {return ida_simple_get (&guc->guc_ids,0 , -GUC_MAX_LRC_DESCRIPTORS ,GFP_KERNEL | + guc->num_guc_ids,GFP_KERNEL | __GFP_RETRY_MAYFAIL | __GFP_NOWARN); }
@@ -2529,6 +2528,7 @@ static bool __guc_submission_selected(struct intel_guc *guc)void intel_guc_submission_init_early (struct intel_guc *guc) { + guc->max_guc_ids = guc->num_guc_ids =GUC_MAX_LRC_DESCRIPTORS ; guc->submission_supported =__guc_submission_supported (guc); guc->submission_selected =__guc_submission_selected (guc); }
@@ -2538,7 +2538,7 @@ g2h_context_lookup(struct intel_guc *guc, u32 desc_idx) {struct intel_context *ce; -if (unlikely (desc_idx >=GUC_MAX_LRC_DESCRIPTORS )) { +if (unlikely (desc_idx >= guc->max_guc_ids)) {drm_err (&guc_to_gt (guc)->i915->drm,"Invalid desc_idx %u" , desc_idx);return NULL ;
@@ -2841,6 +2841,8 @@ void intel_guc_submission_print_info(struct intel_guc *guc,drm_printf (p,"GuC Number Outstanding Submission G2H: %u\n" ,atomic_read (&guc->outstanding_submission_g2h)); +drm_printf (p,"GuC Number GuC IDs: %u\n" , guc->num_guc_ids); +drm_printf (p,"GuC Max GuC IDs: %u\n" , guc->max_guc_ids);drm_printf (p,"GuC tasklet count: %u\n\n" ,atomic_read (&sched_engine->tasklet.count));
@@ -2880,7 +2882,6 @@ void intel_guc_submission_print_context_info(struct intel_guc *guc, {struct intel_context *ce;unsigned long index; -xa_for_each (&guc->context_lookup, index, ce) {drm_printf (p,"GuC lrc descriptor %u:\n" , ce->guc_id);drm_printf (p,"\tHW Context Desc: 0x%08x\n" , ce->lrc.lrca);
Signature · 1 line
2.28.0
RFC PATCH 3/42 drm/i915/guc: Connect the number of guc_ids to debugfs
In reply to Matthew BrostFor testing purposes it may make sense to reduce the number of guc_ids available to be allocated. Add debugfs support for setting the number of guc_ids.
--- .../gpu/drm/i915/gt/uc/intel_guc_debugfs.c | 31 +++++++++++++++++++ .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 3 +- 2 files changed, 33 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/gt/uc/intel_guc_debugfs.c +31 -0
@@ -50,11 +50,42 @@ static int guc_registered_contexts_show(struct seq_file *m, void *data) }DEFINE_GT_DEBUGFS_ATTRIBUTE (guc_registered_contexts); +static int guc_num_id_get (void *data,u64 *val) +{ +struct intel_guc *guc = data; + +if (!intel_guc_submission_is_used (guc)) +return -ENODEV ; + + *val = guc->num_guc_ids; + +return 0 ; +} + +static int guc_num_id_set (void *data,u64 val) +{ +struct intel_guc *guc = data; + +if (!intel_guc_submission_is_used (guc)) +return -ENODEV ; + +if (val > guc->max_guc_ids) + val = guc->max_guc_ids; +else if (val <256 ) + val =256 ; + + guc->num_guc_ids = val; + +return 0 ; +} +DEFINE_SIMPLE_ATTRIBUTE (guc_num_id_fops, guc_num_id_get, guc_num_id_set,"%lld\n" ); +void intel_guc_debugfs_register (struct intel_guc *guc,struct dentry *root) {static const struct debugfs_gt_file files[] = { {"guc_info" , &guc_info_fops,NULL }, {"guc_registered_contexts" , &guc_registered_contexts_fops,NULL }, + {"guc_num_id" , &guc_num_id_fops,NULL }, };if (!intel_guc_is_supported (guc))
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +2 -1
@@ -2540,7 +2540,8 @@ g2h_context_lookup(struct intel_guc *guc, u32 desc_idx)if (unlikely (desc_idx >= guc->max_guc_ids)) {drm_err (&guc_to_gt (guc)->i915->drm, -"Invalid desc_idx %u" , desc_idx); +"Invalid desc_idx %u, max %u" , + desc_idx, guc->max_guc_ids);return NULL ; }
Signature · 6 lines
2.28.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
RFC PATCH 4/42 drm/i915/guc: Don't return -EAGAIN to user when guc_ids exhausted
In reply to Matthew BrostRather than returning -EAGAIN to the user when no guc_ids are available, implement a fair sharing algorithm in the kernel which blocks submissons until guc_ids become available. Submissions are released one at a time, based on priority, until the guc_id pressure is released to ensure fair sharing of the guc_ids. Once the pressure is fully released, the normal guc_id allocation (at request creation time in guc_request_alloc) can resume as this allocation path should be significantly faster and a fair sharing algorithm isn't needed when guc_ids are plentifully.
The fair sharing algorithm is implemented by forcing all submissions to the tasklet which serializes submissions, dequeuing one at a time.
If the submission doesn't have a guc_id and new guc_id can't be found, two lists are searched, one list with contexts that are not pinned but still registered with the guc (searched first) and another list with contexts that are pinned but do not have any submissions actively in inflight (scheduling enabled + registered, searched second). If no guc_ids can be found we kick a workqueue which will retire requests hopefully freeing a guc_id. The workqueue + tasklet ping / pong back and forth until a guc_id can be found.
Once a guc_id is found, we may have to disable context scheduling depending on which list the context is stolen from. When we disable scheduling, we block the tasklet from executing until the completion G2H returns. The disable scheduling must be issued from the workqueue because of the locking structure. When we deregister a context, we also do the same thing (waiting on the G2H) but we can safely issue the deregister H2G from the tasklet.
Once all the G2H have returned we can trigger a submission on the context.
--- drivers/gpu/drm/i915/gt/intel_context_types.h | 3 + drivers/gpu/drm/i915/gt/uc/intel_guc.h | 26 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 806 ++++++++++++++++-- drivers/gpu/drm/i915/i915_request.h | 6 + 4 files changed, 755 insertions(+), 86 deletions(-)
drivers/gpu/drm/i915/gt/intel_context_types.h +3 -0
@@ -185,6 +185,9 @@ struct intel_context {/* GuC LRC descriptor reference count */ atomic_t guc_id_ref; +/* Number of rq submitted without a guc_id */ +u16 guc_num_rq_submit_no_id; +/* * GuC ID link - in list when unpinned but guc_id still valid in GuC */
drivers/gpu/drm/i915/gt/uc/intel_guc.h +24 -2
@@ -33,7 +33,28 @@ struct intel_guc {/* Global engine used to submit requests to GuC */ struct i915_sched_engine *sched_engine; -struct i915_request *stalled_request; + +/* Global state related to submission tasklet */ +struct i915_request *stalled_rq; +struct intel_context *stalled_context; +struct work_struct retire_worker; +unsigned long flags; +int total_num_rq_with_no_guc_id; + +/* +* Submisson stall reason. See intel_guc_submission.c for detailed +* description. +*/ +enum { +STALL_NONE , +STALL_GUC_ID_WORKQUEUE , +STALL_GUC_ID_TASKLET , +STALL_SCHED_DISABLE , +STALL_REGISTER_CONTEXT , +STALL_DEREGISTER_CONTEXT , +STALL_MOVE_LRC_TAIL , +STALL_ADD_REQUEST , + } submission_stall_reason;/* intel_guc_recv interrupt related state */ spinlock_t irq_lock;
@@ -55,7 +76,8 @@ struct intel_guc {struct ida guc_ids;u32 num_guc_ids;u32 max_guc_ids; -struct list_head guc_id_list; +struct list_head guc_id_list_no_ref; +struct list_head guc_id_list_unpinned;bool submission_supported;bool submission_selected;
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +722 -84
1,155 lines not on this page · raw message
drivers/gpu/drm/i915/i915_request.h +6 -0
@@ -139,6 +139,12 @@ enum {* the GPU. Here we track such boost requests on a per-request basis. */ I915_FENCE_FLAG_BOOST , + +/* +* I915_FENCE_FLAG_GUC_ID_NOT_PINNED - Set to signal the GuC submission +* tasklet that the guc_id isn't pinned. +*/ + I915_FENCE_FLAG_GUC_ID_NOT_PINNED, };/**
Signature · 6 lines
2.28.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
RFC PATCH 5/42 drm/i915/guc: Don't allow requests not ready to consume all guc_ids
In reply to Matthew BrostAdd a heuristic which checks if over half of the available guc_ids are currently consumed by requests not ready to be submitted. If this heuristic is true at request creation time (normal guc_id allocation location) force all submissions + guc_ids allocations to tasklet.
--- drivers/gpu/drm/i915/gt/intel_context_types.h | 3 ++ drivers/gpu/drm/i915/gt/intel_reset.c | 9 ++++ drivers/gpu/drm/i915/gt/uc/intel_guc.h | 1 + .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 53 +++++++++++++++++-- .../gpu/drm/i915/gt/uc/intel_guc_submission.h | 2 + 5 files changed, 65 insertions(+), 3 deletions(-)
drivers/gpu/drm/i915/gt/intel_context_types.h +3 -0
@@ -188,6 +188,9 @@ struct intel_context {/* Number of rq submitted without a guc_id */ u16 guc_num_rq_submit_no_id; +/* GuC number of requests not ready */ +atomic_t guc_num_rq_not_ready; +/* * GuC ID link - in list when unpinned but guc_id still valid in GuC */
drivers/gpu/drm/i915/gt/intel_reset.c +9 -0
@@ -22,6 +22,7 @@#include "intel_reset.h" #include "uc/intel_guc.h" +#include "uc/intel_guc_submission.h" #define RESET_MAX_RETRIES 3
@@ -844,6 +845,14 @@ static void nop_submit_request(struct i915_request *request) {RQ_TRACE (request,"-EIO\n" ); +/* +* XXX: Kinda ugly to check for GuC submission here but this function is +* going away once we switch to the DRM scheduler so we can live with +* this for now. +*/ +if (intel_engine_uses_guc (request->engine)) +intel_guc_decr_num_rq_not_ready (request->context); + request =i915_request_mark_eio (request);if (request) {i915_request_submit (request);
drivers/gpu/drm/i915/gt/uc/intel_guc.h +1 -0
@@ -76,6 +76,7 @@ struct intel_guc {struct ida guc_ids;u32 num_guc_ids;u32 max_guc_ids; +atomic_t num_guc_ids_not_ready;struct list_head guc_id_list_no_ref;struct list_head guc_id_list_unpinned;
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +50 -3
96 lines not on this page · raw message
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.h +2 -0
@@ -51,4 +51,6 @@ static inline bool intel_guc_submission_is_used(struct intel_guc *guc) return intel_guc_is_used(guc) && intel_guc_submission_is_wanted(guc); } +void intel_guc_decr_num_rq_not_ready (struct intel_context *ce); +#endif
Signature · 1 line
2.28.0
RFC PATCH 6/42 drm/i915/guc: Introduce guc_submit_engine object
In reply to Matthew BrostMove fields related to controlling the GuC submission state machine to a unique object (guc_submit_engine) rather than the global GuC state (intel_guc). This encapsulation allows multiple instances of submission objects to operate in parallel and a single instance can block if needed while another can make forward progress. This is analogous to how the execlist mode works assigning a schedule object per physical engine but rather in GuC mode we assign a schedule object based on the blocking dependencies.
The guc_submit_engine object also encapsulates the i915_sched_engine object as well.
Lots of find-replace.
Currently only 1 guc_submit_engine instantiated, future patches will instantiate more.
--- drivers/gpu/drm/i915/gt/uc/intel_guc.h | 33 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 555 +++++++++++------- .../i915/gt/uc/intel_guc_submission_types.h | 52 ++ drivers/gpu/drm/i915/i915_scheduler.c | 22 +- drivers/gpu/drm/i915/i915_scheduler.h | 3 + 5 files changed, 409 insertions(+), 256 deletions(-) create mode 100644 drivers/gpu/drm/i915/gt/uc/intel_guc_submission_types.h
drivers/gpu/drm/i915/gt/uc/intel_guc.h +7 -26
@@ -21,6 +21,11 @@struct __guc_ads_blob ; +enum { +GUC_SUBMIT_ENGINE_SINGLE_LRC , +GUC_SUBMIT_ENGINE_MAX +}; +/* * Top level structure of GuC. It handles firmware loading and manages client * pool. intel_guc owns a intel_guc_client to replace the legacy ExecList
@@ -31,31 +36,6 @@ struct intel_guc {struct intel_guc_log log;struct intel_guc_ct ct; -/* Global engine used to submit requests to GuC */ -struct i915_sched_engine *sched_engine; - -/* Global state related to submission tasklet */ -struct i915_request *stalled_rq; -struct intel_context *stalled_context; -struct work_struct retire_worker; -unsigned long flags; -int total_num_rq_with_no_guc_id; - -/* -* Submisson stall reason. See intel_guc_submission.c for detailed -* description. -*/ -enum { -STALL_NONE , -STALL_GUC_ID_WORKQUEUE , -STALL_GUC_ID_TASKLET , -STALL_SCHED_DISABLE , -STALL_REGISTER_CONTEXT , -STALL_DEREGISTER_CONTEXT , -STALL_MOVE_LRC_TAIL , -STALL_ADD_REQUEST , - } submission_stall_reason; -/* intel_guc_recv interrupt related state */ spinlock_t irq_lock;unsigned int msg_enabled_mask;
@@ -68,6 +48,8 @@ struct intel_guc {void (*disable)(struct intel_guc *guc); } interrupts; +struct guc_submit_engine *gse[GUC_SUBMIT_ENGINE_MAX ]; +/* * contexts_lock protects the pool of free guc ids and a linked list of * guc ids available to be stolen
@@ -76,7 +58,6 @@ struct intel_guc {struct ida guc_ids;u32 num_guc_ids;u32 max_guc_ids; -atomic_t num_guc_ids_not_ready;struct list_head guc_id_list_no_ref;struct list_head guc_id_list_unpinned;
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +333 -222
1,089 lines not on this page · raw message
drivers/gpu/drm/i915/gt/uc/intel_guc_submission_types.h new file +52 -0
52 lines not on this page · raw message
drivers/gpu/drm/i915/i915_scheduler.c +14 -8
@@ -452,15 +452,9 @@ static bool default_disabled(struct i915_sched_engine *sched_engine)return false; } -struct i915_sched_engine * -i915_sched_engine_create (unsigned int subclass) +void i915_sched_engine_init (struct i915_sched_engine *sched_engine, +unsigned int subclass) { -struct i915_sched_engine *sched_engine; - - sched_engine =kzalloc (sizeof (*sched_engine),GFP_KERNEL ); -if (!sched_engine) -return NULL ; -kref_init (&sched_engine->ref); sched_engine->queue =RB_ROOT_CACHED ;
@@ -485,6 +479,18 @@ i915_sched_engine_create(unsigned int subclass)lock_map_release (&sched_engine->lock.dep_map);local_irq_enable ();#endif +} + +struct i915_sched_engine * +i915_sched_engine_create (unsigned int subclass) +{ +struct i915_sched_engine *sched_engine; + + sched_engine =kzalloc (sizeof (*sched_engine),GFP_KERNEL ); +if (!sched_engine) +return NULL ; + +i915_sched_engine_init (sched_engine, subclass);return sched_engine; }
drivers/gpu/drm/i915/i915_scheduler.h +3 -0
@@ -48,6 +48,9 @@ static inline void i915_priolist_free(struct i915_priolist *p)__i915_priolist_free (p); } +void i915_sched_engine_init (struct i915_sched_engine *sched_engine, +unsigned int subclass); +struct i915_sched_engine *i915_sched_engine_create (unsigned int subclass);
Signature · 1 line
2.28.0
RFC PATCH 7/42 drm/i915/guc: Check return of __xa_store when registering a context
In reply to Matthew BrostCheck return of __xa_store when registering a context as this can fail in a rare case if not memory can not be allocated. If this occurs fall back on the tasklet flow control and try again in the future.
--- drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-)
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +11 -3
@@ -502,18 +502,24 @@ static inline bool lrc_desc_registered(struct intel_guc *guc, u32 id)return __get_context (guc, id); } -static inline void set_lrc_desc_registered (struct intel_guc *guc,u32 id, +static inline int set_lrc_desc_registered (struct intel_guc *guc,u32 id,struct intel_context *ce) {unsigned long flags; +void *ret;/* * xarray API doesn't have xa_save_irqsave wrapper, so calling the * lower level functions directly. */ xa_lock_irqsave (&guc->context_lookup, flags); -__xa_store (&guc->context_lookup, id, ce,GFP_ATOMIC ); + ret =__xa_store (&guc->context_lookup, id, ce,GFP_ATOMIC );xa_unlock_irqrestore (&guc->context_lookup, flags); + +if (unlikely (xa_is_err (ret))) +return -EBUSY ;/* Try again in future */ + +return 0 ; } static int guc_submission_send_busy_loop(struct intel_guc* guc,
@@ -1840,7 +1846,9 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)rcu_read_unlock ();reset_lrc_desc (guc, desc_idx); -set_lrc_desc_registered (guc, desc_idx, ce); + ret =set_lrc_desc_registered (guc, desc_idx, ce); +if (unlikely (ret)) +return ret; desc =__get_lrc_desc (guc, desc_idx); desc->engine_class =engine_class_to_guc_class (engine->class);
Signature · 1 line
2.28.0
RFC PATCH 8/42 drm/i915/guc: Non-static lrc descriptor registration buffer
In reply to Matthew BrostDynamically allocate space for lrc descriptor registration with the GuC rather than using a large static buffer indexed by the guc_id. If no space is available to register a context, fall back to tasklet flow control mechanism. Only allow 1/2 of the space to be allocated outside the tasklet to prevent unready requests/contexts from consuming all registration space.
--- drivers/gpu/drm/i915/gt/intel_context_types.h | 3 + drivers/gpu/drm/i915/gt/uc/intel_guc.h | 9 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 204 ++++++++++++------ 3 files changed, 152 insertions(+), 64 deletions(-)
drivers/gpu/drm/i915/gt/intel_context_types.h +3 -0
@@ -179,6 +179,9 @@ struct intel_context {/* GuC scheduling state flags that do not require a lock. */ atomic_t guc_sched_state_no_lock; +/* GuC lrc descriptor registration buffer */ +unsigned int guc_lrcd_reg_idx; +/* GuC LRC descriptor ID */ u16 guc_id;
drivers/gpu/drm/i915/gt/uc/intel_guc.h +7 -2
@@ -69,8 +69,13 @@ struct intel_guc {u32 ads_regset_size;u32 ads_golden_ctxt_size; -struct i915_vma *lrc_desc_pool; -void *lrc_desc_pool_vaddr; +/* GuC LRC descriptor registration */ +struct { +struct i915_vma *vma; +void *vaddr; +struct ida ida; +unsigned int max_idx; + } lrcd_reg;/* guc_id to intel_context lookup */ struct xarray context_lookup;
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +142 -62
353 lines not on this page · raw message
Signature · 1 line
2.28.0
RFC PATCH 9/42 drm/i915/guc: Take GT PM ref when deregistering context
In reply to Matthew BrostTaking a PM reference to prevent intel_gt_wait_for_idle from short circuiting while a deregister context H2G is in flight.
--- drivers/gpu/drm/i915/gt/intel_engine_pm.h | 5 + drivers/gpu/drm/i915/gt/intel_gt_pm.h | 13 +++ drivers/gpu/drm/i915/gt/uc/intel_guc.h | 4 + .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 98 +++++++++++++++---- 4 files changed, 101 insertions(+), 19 deletions(-)
drivers/gpu/drm/i915/gt/intel_engine_pm.h +5 -0
@@ -16,6 +16,11 @@ intel_engine_pm_is_awake(const struct intel_engine_cs *engine)return intel_wakeref_is_active (&engine->wakeref); } +static inline void __intel_engine_pm_get (struct intel_engine_cs *engine) +{ +__intel_wakeref_get (&engine->wakeref); +} +static inline void intel_engine_pm_get (struct intel_engine_cs *engine) {intel_wakeref_get (&engine->wakeref);
drivers/gpu/drm/i915/gt/intel_gt_pm.h +13 -0
@@ -41,6 +41,19 @@ static inline void intel_gt_pm_put_async(struct intel_gt *gt)intel_wakeref_put_async (>->wakeref); } +#define with_intel_gt_pm (gt, tmp) \ + for (tmp = 1, intel_gt_pm_get(gt); tmp; \ + intel_gt_pm_put(gt), tmp = 0) +#define with_intel_gt_pm_async (gt, tmp) \ + for (tmp = 1, intel_gt_pm_get(gt); tmp; \ + intel_gt_pm_put_async(gt), tmp = 0) +#define with_intel_gt_pm_if_awake (gt, tmp) \ + for (tmp = intel_gt_pm_get_if_awake(gt); tmp; \ + intel_gt_pm_put(gt), tmp = 0) +#define with_intel_gt_pm_if_awake_async (gt, tmp) \ + for (tmp = intel_gt_pm_get_if_awake(gt); tmp; \ + intel_gt_pm_put_async(gt), tmp = 0) +static inline int intel_gt_pm_wait_for_idle (struct intel_gt *gt) {return intel_wakeref_wait_for_idle (>->wakeref);
drivers/gpu/drm/i915/gt/uc/intel_guc.h +4 -0
@@ -61,6 +61,10 @@ struct intel_guc {struct list_head guc_id_list_no_ref;struct list_head guc_id_list_unpinned; +spinlock_t destroy_lock; +struct list_head destroyed_contexts; +struct work_struct destroy_worker; +bool submission_supported;bool submission_selected;
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +79 -19
176 lines not on this page · raw message
Signature · 1 line
2.28.0
RFC PATCH 10/42 drm/i915: Add GT PM unpark worker
In reply to Matthew BrostSometimes it is desirable to queue work up for later if the GT PM isn't held and run that work on next GT PM unpark.
Implemented with a list in the GT of all pending work, workqueues in the list, a callback to add a workqueue to the list, and finally a wakeref post_get callback that iterates / drains the list + queues the workqueues.
First user of this is deregistration of GuC contexts.
--- drivers/gpu/drm/i915/gt/intel_gt.c | 3 ++ drivers/gpu/drm/i915/gt/intel_gt_pm.c | 8 +++++ .../gpu/drm/i915/gt/intel_gt_pm_unpark_work.c | 35 +++++++++++++++++++ .../gpu/drm/i915/gt/intel_gt_pm_unpark_work.h | 32 +++++++++++++++++ drivers/gpu/drm/i915/gt/intel_gt_types.h | 3 ++ drivers/gpu/drm/i915/gt/uc/intel_guc.h | 3 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 14 +++++--- drivers/gpu/drm/i915/intel_wakeref.c | 5 +++ drivers/gpu/drm/i915/intel_wakeref.h | 1 + 9 files changed, 99 insertions(+), 5 deletions(-) create mode 100644 drivers/gpu/drm/i915/gt/intel_gt_pm_unpark_work.c create mode 100644 drivers/gpu/drm/i915/gt/intel_gt_pm_unpark_work.h
drivers/gpu/drm/i915/gt/intel_gt.c +3 -0
@@ -29,6 +29,9 @@ void intel_gt_init_early(struct intel_gt *gt, struct drm_i915_private *i915)spin_lock_init (>->irq_lock); +spin_lock_init (>->pm_unpark_work_lock); +INIT_LIST_HEAD (>->pm_unpark_work_list); +INIT_LIST_HEAD (>->closed_vma);spin_lock_init (>->closed_lock);
drivers/gpu/drm/i915/gt/intel_gt_pm.c +8 -0
@@ -93,6 +93,13 @@ static int __gt_unpark(struct intel_wakeref *wf)return 0 ; } +static void __gt_unpark_work_queue (struct intel_wakeref *wf) +{ +struct intel_gt *gt =container_of (wf,typeof (*gt), wakeref); + +intel_gt_pm_unpark_work_queue (gt); +} +static int __gt_park (struct intel_wakeref *wf) {struct intel_gt *gt =container_of (wf,typeof (*gt), wakeref);
@@ -123,6 +130,7 @@ static int __gt_park(struct intel_wakeref *wf)static const struct intel_wakeref_ops wf_ops = { .get = __gt_unpark, + .post_get = __gt_unpark_work_queue, .put = __gt_park, };
drivers/gpu/drm/i915/gt/intel_gt_pm_unpark_work.c new file +35 -0
@@ -0,0 +1,35 @@ +// SPDX-License-Identifier: MIT +/* +* Copyright © 2021 Intel Corporation +*/ + +#include "i915_drv.h" +#include "intel_runtime_pm.h" +#include "intel_gt_pm.h" + +void intel_gt_pm_unpark_work_queue (struct intel_gt *gt) +{ +struct intel_gt_pm_unpark_work *work, *next; +unsigned long flags; + +spin_lock_irqsave (>->pm_unpark_work_lock, flags); +list_for_each_entry_safe (work, next, + >->pm_unpark_work_list, link) { +list_del_init (&work->link); +queue_work (system_unbound_wq, &work->worker); + } +spin_unlock_irqrestore (>->pm_unpark_work_lock, flags); +} + +void intel_gt_pm_unpark_work_add (struct intel_gt *gt, +struct intel_gt_pm_unpark_work *work) +{ +unsigned long flags; + +spin_lock_irqsave (>->pm_unpark_work_lock, flags); +if (intel_gt_pm_is_awake (gt)) +queue_work (system_unbound_wq, &work->worker); +else if (list_empty (&work->link)) +list_add_tail (&work->link, >->pm_unpark_work_list); +spin_unlock_irqrestore (>->pm_unpark_work_lock, flags); +}
drivers/gpu/drm/i915/gt/intel_gt_pm_unpark_work.h new file +32 -0
@@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: MIT */ +/* +* Copyright © 2021 Intel Corporation +*/ + +#ifndef INTEL_GT_PM_UNPARK_WORK_H +#define INTEL_GT_PM_UNPARK_WORK_H + +#include <linux/list.h> +#include <linux/workqueue.h> + +struct intel_gt ; + +struct intel_gt_pm_unpark_work { +struct list_head link; +struct work_struct worker; +}; + +void intel_gt_pm_unpark_work_queue (struct intel_gt *gt); + +void intel_gt_pm_unpark_work_add (struct intel_gt *gt, +struct intel_gt_pm_unpark_work *work); + +static inline void +intel_gt_pm_unpark_work_init (struct intel_gt_pm_unpark_work *work, +work_func_t fn) +{ +INIT_LIST_HEAD (&work->link); +INIT_WORK (&work->worker, fn); +} + +#endif /* INTEL_GT_PM_UNPARK_WORK_H */
drivers/gpu/drm/i915/gt/intel_gt_types.h +3 -0
@@ -91,6 +91,9 @@ struct intel_gt {struct intel_wakeref wakeref;atomic_t user_wakeref; +struct list_head pm_unpark_work_list; +spinlock_t pm_unpark_work_lock; +struct list_head closed_vma;spinlock_t closed_lock;/* guards the list of closed_vma */
drivers/gpu/drm/i915/gt/uc/intel_guc.h +2 -1
@@ -18,6 +18,7 @@#include "intel_uc_fw.h" #include "i915_utils.h" #include "i915_vma.h" +#include "gt/intel_gt_pm_unpark_work.h" struct __guc_ads_blob ;
@@ -63,7 +64,7 @@ struct intel_guc {spinlock_t destroy_lock;struct list_head destroyed_contexts; -struct work_struct destroy_worker; +struct intel_gt_pm_unpark_work destroy_worker;bool submission_supported;bool submission_selected;
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +10 -4
@@ -1397,8 +1397,9 @@ int intel_guc_submission_init(struct intel_guc *guc)ida_init (&guc->guc_ids);spin_lock_init (&guc->destroy_lock); +INIT_LIST_HEAD (&guc->destroyed_contexts); -INIT_WORK (&guc->destroy_worker, destroy_worker_func); +intel_gt_pm_unpark_work_init (&guc->destroy_worker, destroy_worker_func);return 0 ; }
@@ -2420,13 +2421,18 @@ static void deregister_destroyed_contexts(struct intel_guc *guc)static void destroy_worker_func (struct work_struct *w) { +struct intel_gt_pm_unpark_work *destroy_worker = + container_of(w, struct intel_gt_pm_unpark_work, worker);struct intel_guc *guc = - container_of(w, struct intel_guc, destroy_worker); + container_of(destroy_worker, struct intel_guc, destroy_worker);struct intel_gt *gt =guc_to_gt (guc);int tmp; - with_intel_gt_pm(gt, tmp) + with_intel_gt_pm_if_awake(gt, tmp)deregister_destroyed_contexts (guc); + +if (!list_empty (&guc->destroyed_contexts)) +intel_gt_pm_unpark_work_add (gt, destroy_worker); }static void guc_context_destroy (struct kref *kref)
@@ -2489,7 +2495,7 @@ static void guc_context_destroy(struct kref *kref)* take the GT PM for the first time which isn't allowed from an atomic * context. */ -queue_work (system_unbound_wq, &guc->destroy_worker); +intel_gt_pm_unpark_work_add (guc_to_gt (guc), &guc->destroy_worker); }static int guc_context_alloc (struct intel_context *ce)
drivers/gpu/drm/i915/intel_wakeref.c +5 -0
@@ -24,6 +24,8 @@ static void rpm_put(struct intel_wakeref *wf)int __intel_wakeref_get_first (struct intel_wakeref *wf) { +bool do_post = false; +/* * Treat get/put as different subclasses, as we may need to run * the put callback from under the shrinker and do not want to
@@ -44,8 +46,11 @@ int __intel_wakeref_get_first(struct intel_wakeref *wf) }smp_mb__before_atomic ();/* release wf->count */ + do_post = true; }atomic_inc (&wf->count); +if (do_post && wf->ops->post_get) + wf->ops->post_get (wf);mutex_unlock (&wf->mutex);INTEL_WAKEREF_BUG_ON (atomic_read (&wf->count) <=0 );
drivers/gpu/drm/i915/intel_wakeref.h +1 -0
@@ -30,6 +30,7 @@ typedef depot_stack_handle_t intel_wakeref_t;struct intel_wakeref_ops {int (*get)(struct intel_wakeref *wf); +void (*post_get)(struct intel_wakeref *wf);int (*put)(struct intel_wakeref *wf); };
Signature · 1 line
2.28.0
RFC PATCH 11/42 drm/i915/guc: Take engine PM when a context is pinned with GuC submission
In reply to Matthew BrostTaking a PM reference to prevent intel_gt_wait_for_idle from short circuiting while a scheduling of user context could be enabled.
--- drivers/gpu/drm/i915/Makefile | 1 + .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 36 +++++++++++++++++-- 2 files changed, 34 insertions(+), 3 deletions(-)
drivers/gpu/drm/i915/Makefile +1 -0
@@ -102,6 +102,7 @@ gt-y += \ gt/intel_gt_clock_utils.o \ gt/intel_gt_irq.o \ gt/intel_gt_pm.o \ + gt/intel_gt_pm_unpark_work.o \ gt/intel_gt_pm_irq.o \ gt/intel_gt_requests.o \ gt/intel_gtt.o \
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +33 -3
@@ -2041,7 +2041,12 @@ static int guc_context_pre_pin(struct intel_context *ce,static int guc_context_pin (struct intel_context *ce,void *vaddr) { -return __guc_context_pin (ce, ce->engine, vaddr); +int ret =__guc_context_pin (ce, ce->engine, vaddr); + +if (likely (!ret && !intel_context_is_barrier (ce))) +intel_engine_pm_get (ce->engine); + +return ret; }static void guc_context_unpin (struct intel_context *ce)
@@ -2052,6 +2057,9 @@ static void guc_context_unpin(struct intel_context *ce)unpin_guc_id (guc, ce, true);lrc_unpin (ce); + +if (likely (!intel_context_is_barrier (ce))) +intel_engine_pm_put (ce->engine); }static void guc_context_post_unpin (struct intel_context *ce)
@@ -2971,8 +2979,30 @@ static int guc_virtual_context_pre_pin(struct intel_context *ce,static int guc_virtual_context_pin (struct intel_context *ce,void *vaddr) {struct intel_engine_cs *engine =guc_virtual_get_sibling (ce->engine,0 ); +int ret =__guc_context_pin (ce, engine, vaddr); +intel_engine_mask_t tmp, mask = ce->engine->mask; + +if (likely (!ret)) +for_each_engine_masked (engine, ce->engine->gt, mask, tmp) +intel_engine_pm_get (engine); + +return ret; +} + +static void guc_virtual_context_unpin (struct intel_context *ce) +{ +intel_engine_mask_t tmp, mask = ce->engine->mask; +struct intel_engine_cs *engine; +struct intel_guc *guc =ce_to_guc (ce); -return __guc_context_pin (ce, engine, vaddr); +GEM_BUG_ON (context_enabled (ce)); +GEM_BUG_ON (intel_context_is_barrier (ce)); + +unpin_guc_id (guc, ce, true); +lrc_unpin (ce); + +for_each_engine_masked (engine, ce->engine->gt, mask, tmp) +intel_engine_pm_put (engine); }static void guc_virtual_context_enter (struct intel_context *ce)
@@ -3009,7 +3039,7 @@ static const struct intel_context_ops virtual_guc_context_ops = { .pre_pin = guc_virtual_context_pre_pin, .pin = guc_virtual_context_pin, - .unpin = guc_context_unpin, + .unpin = guc_virtual_context_unpin, .post_unpin = guc_context_post_unpin, .ban = guc_context_ban,
Signature · 6 lines
2.28.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
RFC PATCH 12/42 drm/i915/guc: Don't call switch_to_kernel_context with GuC submission
In reply to Matthew BrostCalling switch_to_kernel_context isn't needed if the engine PM reference is taken while all contexts are pinned. By not calling switch_to_kernel_context we save on issuing a request to the engine.
--- drivers/gpu/drm/i915/gt/intel_engine_pm.c | 4 ++++ 1 file changed, 4 insertions(+)
drivers/gpu/drm/i915/gt/intel_engine_pm.c +4 -0
@@ -162,6 +162,10 @@ static bool switch_to_kernel_context(struct intel_engine_cs *engine)unsigned long flags;bool result = true; +/* No need to switch_to_kernel_context if GuC submission */ +if (intel_engine_uses_guc (engine)) +return true; +/* GPU is pointing to the void, as good as in the kernel context. */ if (intel_gt_is_wedged (engine->gt))return true;
Signature · 1 line
2.28.0
RFC PATCH 13/42 drm/i915/guc: Selftest for GuC flow control
In reply to Matthew BrostAdd 5 selftests for hard (from user space) to recreate flow conditions. Test listed below:
1. A test to verify that the number of guc_ids can be exhausted and all submissions still complete.
2. A test to verify that the flow control state machine can recover from a full GPU reset.
3. A teset to verify that the lrcd registration slots can be exhausted and all submissions still complete.
4. A test to verify that the H2G channel can deadlock and a full GPU reset recovers the system.
5. A test to stress to CTB channel but submitting to lots of contexts and then immediately destroy the contexts.
Tests 1, 2, and 3 also ensure when the flow control is triggered by unready requests those unready requests do not DoS ready requests.
--- drivers/gpu/drm/i915/gt/uc/intel_guc.h | 6 + drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 43 +- drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h | 9 + .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 16 + .../i915/gt/uc/intel_guc_submission_types.h | 2 + .../i915/gt/uc/selftest_guc_flow_control.c | 581 ++++++++++++++++++ .../drm/i915/selftests/i915_live_selftests.h | 1 + .../i915/selftests/intel_scheduler_helpers.c | 12 + .../i915/selftests/intel_scheduler_helpers.h | 1 + 9 files changed, 661 insertions(+), 10 deletions(-) create mode 100644 drivers/gpu/drm/i915/gt/uc/selftest_guc_flow_control.c
drivers/gpu/drm/i915/gt/uc/intel_guc.h +6 -0
@@ -103,6 +103,12 @@ struct intel_guc {/* To serialize the intel_guc_send actions */ struct mutex send_mutex; + + I915_SELFTEST_DECLARE(bool gse_hang_expected;) + I915_SELFTEST_DECLARE(bool deadlock_expected;) + I915_SELFTEST_DECLARE(bool bad_desc_expected;) + I915_SELFTEST_DECLARE(bool inject_bad_sched_disable;) + I915_SELFTEST_DECLARE(bool inject_corrupt_h2g;) };static inline struct intel_guc *log_to_guc (struct intel_guc_log *log)
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +33 -10
79 lines not on this page · raw message
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +9 -0
@@ -6,6 +6,7 @@#ifndef _INTEL_GUC_CT_H_#define _INTEL_GUC_CT_H_ +#include <linux/circ_buf.h> #include <linux/interrupt.h> #include <linux/spinlock.h> #include <linux/workqueue.h>
@@ -115,4 +116,12 @@ void intel_guc_ct_event_handler(struct intel_guc_ct *ct);void intel_guc_ct_print_info (struct intel_guc_ct *ct,struct drm_printer *p); +static inline bool intel_guc_ct_is_recv_buffer_empty (struct intel_guc_ct *ct) +{ +struct intel_guc_ct_buffer *ctb = &ct->ctbs.recv; + +return atomic_read (&ctb->space) == + (CIRC_SPACE (0 ,0 , ctb->size) - ctb->resv_space); +} +#endif /* _INTEL_GUC_CT_H_ */
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +16 -0
@@ -818,6 +818,7 @@ static int gse_dequeue_one_context(struct guc_submit_engine *gse)GEM_WARN_ON (ret);/* Unexpected */ goto deadlk; } + I915_SELFTEST_DECLARE(++gse->tasklets_submit_count;) }/*
@@ -2092,7 +2093,15 @@ static void __guc_context_sched_disable(struct intel_guc *guc, GUC_CONTEXT_DISABLE }; +#if IS_ENABLED (CONFIG_DRM_I915_SELFTEST ) +if (guc->inject_bad_sched_disable && + guc_id ==GUC_INVALID_LRC_ID ) + guc->inject_bad_sched_disable = false; +else +GEM_BUG_ON (guc_id ==GUC_INVALID_LRC_ID ); +#else GEM_BUG_ON (guc_id ==GUC_INVALID_LRC_ID ); +#endif trace_intel_context_sched_disable (ce);
@@ -2739,6 +2748,9 @@ static void retire_worker_sched_disable(struct guc_submit_engine *gse, guc_id =prep_context_pending_disable (ce);spin_unlock_irqrestore (&ce->guc_state.lock, flags); +if (I915_SELFTEST_ONLY (guc->inject_bad_sched_disable)) + guc_id =GUC_INVALID_LRC_ID ; + with_intel_runtime_pm(runtime_pm, wakeref)__guc_context_sched_disable (guc, ce, guc_id);
@@ -3991,3 +4003,7 @@ bool intel_guc_virtual_engine_has_heartbeat(const struct intel_engine_cs *ve)return false; } + +#if IS_ENABLED (CONFIG_DRM_I915_SELFTEST ) +#include "selftest_guc_flow_control.c" +#endif
drivers/gpu/drm/i915/gt/uc/intel_guc_submission_types.h +2 -0
@@ -47,6 +47,8 @@ struct guc_submit_engine {STALL_MOVE_LRC_TAIL ,STALL_ADD_REQUEST , } submission_stall_reason; + + I915_SELFTEST_DECLARE(u64 tasklets_submit_count;) };#endif
drivers/gpu/drm/i915/gt/uc/selftest_guc_flow_control.c new file +581 -0
581 lines not on this page · raw message
drivers/gpu/drm/i915/selftests/i915_live_selftests.h +1 -0
@@ -47,5 +47,6 @@ selftest(hangcheck, intel_hangcheck_live_selftests) selftest(execlists, intel_execlists_live_selftests) selftest(ring_submission, intel_ring_submission_live_selftests) selftest(perf, i915_perf_live_selftests) +selftest (guc_flow_control, intel_guc_flow_control) /* Here be dragons: keep last to run last! */selftest (late_gt_pm, intel_gt_pm_late_selftests)
drivers/gpu/drm/i915/selftests/intel_scheduler_helpers.c +12 -0
@@ -15,6 +15,18 @@#define REDUCED_PREEMPT 10#define WAIT_FOR_RESET_TIME 10000 +struct intel_engine_cs *intel_selftest_find_any_engine (struct intel_gt *gt) +{ +struct intel_engine_cs *engine; +enum intel_engine_id id; + +for_each_engine (engine, gt, id) +return engine; + +pr_err ("No valid engine found!\n" ); +return NULL ; +} +int intel_selftest_modify_policy (struct intel_engine_cs *engine,struct intel_selftest_saved_policy *saved,u32 modify_type)
drivers/gpu/drm/i915/selftests/intel_scheduler_helpers.h +1 -0
@@ -31,5 +31,6 @@ int intel_selftest_modify_policy(struct intel_engine_cs *engine,int intel_selftest_restore_policy (struct intel_engine_cs *engine,struct intel_selftest_saved_policy *saved);int intel_selftest_wait_for_rq (struct i915_request *rq); +struct intel_engine_cs *intel_selftest_find_any_engine (struct intel_gt *gt);#endif
Signature · 6 lines
2.28.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
RFC PATCH 14/42 drm/i915: Add logical engine mapping
In reply to Matthew BrostAdd logical engine mapping. This is required for split-frame, as workloads need to be placed on engines in a logically contiguous manner.
--- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 60 ++++++++++++++++--- drivers/gpu/drm/i915/gt/intel_engine_types.h | 1 + .../drm/i915/gt/intel_execlists_submission.c | 1 + drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c | 2 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 21 +------ 5 files changed, 56 insertions(+), 29 deletions(-)
drivers/gpu/drm/i915/gt/intel_engine_cs.c +51 -9
@@ -260,7 +260,8 @@ static void nop_irq_handler(struct intel_engine_cs *engine, u16 iir)GEM_DEBUG_WARN_ON (iir); } -static int intel_engine_setup (struct intel_gt *gt,enum intel_engine_id id) +static int intel_engine_setup (struct intel_gt *gt,enum intel_engine_id id, +u8 logical_instance) {const struct engine_info *info = &intel_engines[id];struct drm_i915_private *i915 = gt->i915;
@@ -303,6 +304,7 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id) engine->class = info->class; engine->instance = info->instance; + engine->logical_mask =BIT (logical_instance);__sprint_engine_name (engine); engine->props.heartbeat_interval_ms =
@@ -516,6 +518,37 @@ static intel_engine_mask_t init_engine_mask(struct intel_gt *gt)return info->engine_mask; } +static void populate_logical_ids (struct intel_gt *gt,u8 *logical_ids, +u8 class,const u8 *map,u8 num_instances) +{ +int i, j; +u8 current_logical_id =0 ; + +for (j =0 ; j < num_instances; ++j) { +for (i =0 ; i <ARRAY_SIZE (intel_engines); ++i) { +if (!HAS_ENGINE (gt, i) || + intel_engines[i].class != class) +continue ; + +if (intel_engines[i].instance == map[j]) { + logical_ids[intel_engines[i].instance] = + current_logical_id++; +break ; + } + } + } +} + +static void setup_logical_ids (struct intel_gt *gt,u8 *logical_ids,u8 class) +{ +int i; +u8 map[MAX_ENGINE_INSTANCE +1 ]; + +for (i =0 ; i <MAX_ENGINE_INSTANCE +1 ; ++i) + map[i] = i; +populate_logical_ids (gt, logical_ids, class, map,ARRAY_SIZE (map)); +} +/** * intel_engines_init_mmio() - allocate and prepare the Engine Command Streamers * @gt: pointer to struct intel_gt
@@ -527,7 +560,8 @@ int intel_engines_init_mmio(struct intel_gt *gt)struct drm_i915_private *i915 = gt->i915;const unsigned int engine_mask =init_engine_mask (gt);unsigned int mask =0 ; -unsigned int i; +unsigned int i, class; +u8 logical_ids[MAX_ENGINE_INSTANCE +1 ];int err;drm_WARN_ON (&i915->drm, engine_mask ==0 );
@@ -537,15 +571,23 @@ int intel_engines_init_mmio(struct intel_gt *gt)if (i915_inject_probe_failure (i915))return -ENODEV ; -for (i =0 ; i <ARRAY_SIZE (intel_engines); i++) { -if (!HAS_ENGINE (gt, i)) -continue ; +for (class =0 ; class <MAX_ENGINE_CLASS +1 ; ++class) { +setup_logical_ids (gt, logical_ids, class); - err =intel_engine_setup (gt, i); -if (err) - goto cleanup; +for (i =0 ; i <ARRAY_SIZE (intel_engines); ++i) { +u8 instance = intel_engines[i].instance; + +if (intel_engines[i].class != class || + !HAS_ENGINE (gt, i)) +continue ; - mask |=BIT (i); + err =intel_engine_setup (gt, i, + logical_ids[instance]); +if (err) + goto cleanup; + + mask |=BIT (i); + } }/*
drivers/gpu/drm/i915/gt/intel_engine_types.h +1 -0
@@ -272,6 +272,7 @@ struct intel_engine_cs {unsigned int guc_id;intel_engine_mask_t mask; +intel_engine_mask_t logical_mask;u8 class;u8 instance;
drivers/gpu/drm/i915/gt/intel_execlists_submission.c +1 -0
@@ -3823,6 +3823,7 @@ execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count) ve->siblings[ve->num_siblings++] = sibling; ve->base.mask |= sibling->mask; + ve->base.logical_mask |= sibling->logical_mask;/* * All physical engines must be compatible for their emission
drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c +1 -1
@@ -176,7 +176,7 @@ static void guc_mapping_table_init(struct intel_gt *gt,for_each_engine (engine, gt, id) {u8 guc_class =engine_class_to_guc_class (engine->class); - system_info->mapping_table[guc_class][engine->instance] = + system_info->mapping_table[guc_class][ilog2 (engine->logical_mask)] = engine->instance; } }
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +2 -19
@@ -1781,23 +1781,6 @@ static int deregister_context(struct intel_context *ce, u32 guc_id, bool loop)return __guc_action_deregister_context (guc, guc_id, loop); } -static intel_engine_mask_t adjust_engine_mask (u8 class,intel_engine_mask_t mask) -{ -switch (class) { -case RENDER_CLASS : -return mask >>RCS0 ; -case VIDEO_ENHANCEMENT_CLASS : -return mask >>VECS0 ; -case VIDEO_DECODE_CLASS : -return mask >>VCS0 ; -case COPY_ENGINE_CLASS : -return mask >>BCS0 ; -default : -MISSING_CASE (class); -return 0 ; - } -} -static void guc_context_policy_init (struct intel_engine_cs *engine,struct guc_lrc_desc *desc) {
@@ -1938,8 +1921,7 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop) desc =__get_lrc_desc (guc, ce->guc_lrcd_reg_idx); desc->engine_class =engine_class_to_guc_class (engine->class); - desc->engine_submit_mask =adjust_engine_mask (engine->class, - engine->mask); + desc->engine_submit_mask = engine->logical_mask; desc->hw_context_desc = ce->lrc.lrca; ce->guc_prio =map_i915_prio_to_guc_prio (prio); desc->priority = ce->guc_prio;
@@ -3946,6 +3928,7 @@ guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count) } ve->base.mask |= sibling->mask; + ve->base.logical_mask |= sibling->logical_mask;if (n !=0 && ve->base.class != sibling->class) { DRM_DEBUG("invalid mixing of engine class, sibling %d, already %d\n",
Signature · 1 line
2.28.0
RFC PATCH 15/42 drm/i915: Expose logical engine instance to user
In reply to Matthew BrostExpose logical engine instance to user via query engine info IOCTL. This is required for split-frame workloads as these needs to be placed on engines in a logically contiguous order. The logical mapping can change based on fusing. Rather than having user have knowledge of the fusing we simply just expose the logical mapping with the existing query engine info IOCTL.
--- drivers/gpu/drm/i915/i915_query.c | 2 ++ include/uapi/drm/i915_drm.h | 8 +++++++- 2 files changed, 9 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_query.c +2 -0
@@ -124,7 +124,9 @@ query_engine_info(struct drm_i915_private *i915,for_each_uabi_engine (engine, i915) { info.engine.engine_class = engine->uabi_class; info.engine.engine_instance = engine->uabi_instance; + info.flags =I915_ENGINE_INFO_HAS_LOGICAL_INSTANCE ; info.capabilities = engine->uabi_capabilities; + info.logical_instance =ilog2 (engine->logical_mask);if (copy_to_user (info_ptr, &info,sizeof (info)))return -EFAULT ;
include/uapi/drm/i915_drm.h +7 -1
@@ -2662,14 +2662,20 @@ struct drm_i915_engine_info {/** @flags: Engine flags. */ __u64 flags; +#define I915_ENGINE_INFO_HAS_LOGICAL_INSTANCE (1 << 0)/** @capabilities: Capabilities of this engine. */ __u64 capabilities;#define I915_VIDEO_CLASS_CAPABILITY_HEVC (1 << 0)#define I915_VIDEO_AND_ENHANCE_CLASS_CAPABILITY_SFC (1 << 1) +/** @logical_instance: Logical instance of engine */ +__u16 logical_instance; +/** @rsvd1: Reserved fields. */ -__u64 rsvd1[4 ]; +__u16 rsvd1[3 ]; +/** @rsvd2: Reserved fields. */ +__u64 rsvd2[2 ]; };/**
Signature · 1 line
2.28.0
RFC PATCH 16/42 drm/i915/guc: Introduce context parent-child relationship
In reply to Matthew BrostIntroduce context parent-child relationship. Once this relationship is created all pinning / unpinning operations are directed to the parent context. The parent context is responsible for pinning all of its' children and itself.
This is a precursor to the full GuC multi-lrc implementation but aligns to how GuC mutli-lrc interface is defined - a single H2G is used register / deregister all of the contexts simultaneously.
Subsequent patches in the series will implement the pinning / unpinning operations for parent / child contexts.
--- drivers/gpu/drm/i915/gt/intel_context.c | 29 +++++++++++++++++++ drivers/gpu/drm/i915/gt/intel_context.h | 18 ++++++++++++ drivers/gpu/drm/i915/gt/intel_context_types.h | 12 ++++++++ 3 files changed, 59 insertions(+)
drivers/gpu/drm/i915/gt/intel_context.c +29 -0
@@ -399,6 +399,8 @@ intel_context_init(struct intel_context *ce, struct intel_engine_cs *engine)spin_lock_init (&ce->guc_state.lock);INIT_LIST_HEAD (&ce->guc_state.fences); +INIT_LIST_HEAD (&ce->guc_child_list); +spin_lock_init (&ce->guc_active.lock);INIT_LIST_HEAD (&ce->guc_active.requests);
@@ -414,10 +416,17 @@ intel_context_init(struct intel_context *ce, struct intel_engine_cs *engine)void intel_context_fini (struct intel_context *ce) { +struct intel_context *child, *next; +if (ce->timeline)intel_timeline_put (ce->timeline);i915_vm_put (ce->vm); +/* Need to put the creation ref for the children */ +if (intel_context_is_parent (ce)) +for_each_child_safe (ce, child, next) +intel_context_put (child); +mutex_destroy (&ce->pin_mutex);i915_active_fini (&ce->active); }
@@ -544,6 +553,26 @@ struct i915_request *intel_context_find_active_request(struct intel_context *ce)return active; } +void intel_context_bind_parent_child (struct intel_context *parent, +struct intel_context *child) +{ +/* +* Callers responsibility to validate that this function is used +* correctly but we use GEM_BUG_ON here ensure that they do. +*/ +GEM_BUG_ON (!intel_engine_uses_guc (parent->engine)); +GEM_BUG_ON (intel_context_is_pinned (parent)); +GEM_BUG_ON (intel_context_is_child (parent)); +GEM_BUG_ON (intel_context_is_pinned (child)); +GEM_BUG_ON (intel_context_is_child (child)); +GEM_BUG_ON (intel_context_is_parent (child)); + + parent->guc_number_children++; +list_add_tail (&child->guc_child_link, + &parent->guc_child_list); + child->parent = parent; +} +#if IS_ENABLED (CONFIG_DRM_I915_SELFTEST )#include "selftest_context.c" #endif
drivers/gpu/drm/i915/gt/intel_context.h +18 -0
@@ -41,6 +41,24 @@ void intel_context_free(struct intel_context *ce);int intel_context_reconfigure_sseu (struct intel_context *ce,const struct intel_sseu sseu); +static inline bool intel_context_is_child (struct intel_context *ce) +{ +return !!ce->parent; +} + +static inline bool intel_context_is_parent (struct intel_context *ce) +{ +return !!ce->guc_number_children; +} + +void intel_context_bind_parent_child (struct intel_context *parent, +struct intel_context *child); + +#define for_each_child (parent, ce)\ + list_for_each_entry(ce, &parent->guc_child_list, guc_child_link) +#define for_each_child_safe (parent, ce, cn)\ + list_for_each_entry_safe(ce, cn, &parent->guc_child_list, guc_child_link) +/** * intel_context_lock_pinned - Stablises the 'pinned' status of the HW context * @ce - the context
drivers/gpu/drm/i915/gt/intel_context_types.h +12 -0
@@ -202,6 +202,18 @@ struct intel_context {/* GuC context blocked fence */ struct i915_sw_fence guc_blocked; +/* Head of children list or link in parent's children list */ +union { +struct list_head guc_child_list;/* parent */ +struct list_head guc_child_link;/* child */ + }; + +/* Pointer to parent */ +struct intel_context *parent; + +/* Number of children if parent */ +u8 guc_number_children; +/* * GuC priority management */
Signature · 6 lines
2.28.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
RFC PATCH 17/42 drm/i915/guc: Implement GuC parent-child context pin / unpin functions
In reply to Matthew BrostImplement GuC parent-child context pin / unpin functions in which in any contexts in the relationship are pinned all the contexts are pinned. The parent owns most of the pinning / unpinning process and the children direct any pins / unpins to the parent.
Patch implements a number of unused functions that will be connected later in the series.
--- drivers/gpu/drm/i915/gt/intel_context.c | 188 ++++++++++++++++-- drivers/gpu/drm/i915/gt/intel_context.h | 43 +--- drivers/gpu/drm/i915/gt/intel_context_types.h | 4 +- .../drm/i915/gt/intel_execlists_submission.c | 25 ++- drivers/gpu/drm/i915/gt/intel_lrc.c | 18 +- drivers/gpu/drm/i915/gt/intel_lrc.h | 6 +- .../gpu/drm/i915/gt/intel_ring_submission.c | 5 +- drivers/gpu/drm/i915/gt/mock_engine.c | 4 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 183 +++++++++++++++-- 9 files changed, 368 insertions(+), 108 deletions(-)
drivers/gpu/drm/i915/gt/intel_context.c +170 -18
@@ -162,8 +162,8 @@ static void __ring_retire(struct intel_ring *ring)intel_ring_unpin (ring); } -static int intel_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) +static int __intel_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) {int err;
@@ -194,7 +194,7 @@ static int intel_context_pre_pin(struct intel_context *ce,return err; } -static void intel_context_post_unpin (struct intel_context *ce) +static void __intel_context_post_unpin (struct intel_context *ce) {if (ce->state)__context_unpin_state (ce->state);
@@ -203,13 +203,84 @@ static void intel_context_post_unpin(struct intel_context *ce)__ring_retire (ce->ring); } -int __intel_context_do_pin_ww (struct intel_context *ce, -struct i915_gem_ww_ctx *ww) +static int intel_context_pre_pin (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) { -bool handoff = false; -void *vaddr; +struct intel_context *child; +int err, i =0 ; + +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) { + err =__intel_context_pre_pin (child, ww); +if (unlikely (err)) + goto unwind; + ++i; + } + + err =__intel_context_pre_pin (ce, ww); +if (unlikely (err)) + goto unwind; + +return 0 ; + +unwind: +for_each_child (ce, child) { +if (!i--) +break ; +__intel_context_post_unpin (ce); + } + +return err; +} + +static void intel_context_post_unpin (struct intel_context *ce) +{ +struct intel_context *child; + +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) +__intel_context_post_unpin (child); + +__intel_context_post_unpin (ce); +} + +static int __do_ww_lock (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +int err =i915_gem_object_lock (ce->timeline->hwsp_ggtt->obj, ww); +if (!err && ce->ring->vma->obj) + err =i915_gem_object_lock (ce->ring->vma->obj, ww); +if (!err && ce->state) + err =i915_gem_object_lock (ce->state->obj, ww); + +return err; +} + +static int do_ww_lock (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +struct intel_context *child;int err =0 ; +GEM_BUG_ON (intel_context_is_child (ce)); + +for_each_child (ce, child) { + err =__do_ww_lock (child, ww); +if (unlikely (err)) +return err; + } + +return __do_ww_lock (ce, ww); +} + +static int __intel_context_do_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +bool handoff = false; +int err; +if (unlikely (!test_bit (CONTEXT_ALLOC_BIT , &ce->flags))) { err =intel_context_alloc_state (ce);if (err)
@@ -221,14 +292,11 @@ int __intel_context_do_pin_ww(struct intel_context *ce,* refcount for __intel_context_active(), which prevent a lock * inversion of ce->pin_mutex vs dma_resv_lock(). */ + err =do_ww_lock (ce, ww); +if (err) +return err; - err =i915_gem_object_lock (ce->timeline->hwsp_ggtt->obj, ww); -if (!err && ce->ring->vma->obj) - err =i915_gem_object_lock (ce->ring->vma->obj, ww); -if (!err && ce->state) - err =i915_gem_object_lock (ce->state->obj, ww); -if (!err) - err =intel_context_pre_pin (ce, ww); + err =intel_context_pre_pin (ce, ww);if (err)return err;
@@ -236,7 +304,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,if (err) goto err_ctx_unpin; - err = ce->ops->pre_pin (ce, ww, &vaddr); + err = ce->ops->pre_pin (ce, ww);if (err) goto err_release;
@@ -254,7 +322,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,if (unlikely (err)) goto err_unlock; - err = ce->ops->pin (ce, vaddr); + err = ce->ops->pin (ce);if (err) {intel_context_active_release (ce); goto err_unlock;
@@ -294,7 +362,7 @@ int __intel_context_do_pin_ww(struct intel_context *ce,return err; } -int __intel_context_do_pin (struct intel_context *ce) +static int __intel_context_do_pin (struct intel_context *ce) {struct i915_gem_ww_ctx ww;int err;
@@ -341,7 +409,7 @@ static void __intel_context_retire(struct i915_active *active) intel_context_get_avg_runtime_ns(ce));set_bit (CONTEXT_VALID_BIT , &ce->flags); -intel_context_post_unpin (ce); +__intel_context_post_unpin (ce);intel_context_put (ce); }
@@ -573,6 +641,90 @@ void intel_context_bind_parent_child(struct intel_context *parent, child->parent = parent; } +static inline int ____intel_context_pin (struct intel_context *ce) +{ +if (likely (intel_context_pin_if_active (ce))) +return 0 ; + +return __intel_context_do_pin (ce); +} + +static inline int __intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +if (likely (intel_context_pin_if_active (ce))) +return 0 ; + +return __intel_context_do_pin_ww (ce, ww); +} + +static inline void __intel_context_unpin (struct intel_context *ce) +{ +if (!ce->ops->sched_disable) { +__intel_context_do_unpin (ce,1 ); + }else { +/* +* Move ownership of this pin to the scheduling disable which is +* an async operation. When that operation completes the above +* intel_context_sched_disable_unpin is called potentially +* unpinning the context. +*/ +while (!atomic_add_unless (&ce->pin_count,-1 ,1 )) { +if (atomic_cmpxchg (&ce->pin_count,1 ,2 ) ==1 ) { + ce->ops->sched_disable (ce); +break ; + } + } + } +} + +/* +* FIXME: This is ugly, these branches are only needed for parallel contexts in +* GuC submission. Basically the idea is if any of the contexts, that are +* configured for parallel submission, are pinned all the contexts need to be +* pinned in order to register these contexts with the GuC. We are adding the +* layer here while it should probably be pushed to the backend via a vfunc. But +* since we already have ce->pin + a layer atop it is confusing. Definitely +* needs a bit of rework how to properly layer / structure this code path. What +* is in place works but is not ideal. +*/ +int intel_context_pin (struct intel_context *ce) +{ +if (intel_context_is_child (ce)) { +if (!atomic_fetch_add (1 , &ce->pin_count)) { +return ____intel_context_pin (ce->parent); + }else { +return 0 ; + } + }else { +return ____intel_context_pin (ce); + } +} + +int intel_context_pin_ww (struct intel_context *ce, +struct i915_gem_ww_ctx *ww) +{ +if (intel_context_is_child (ce)) { +if (!atomic_fetch_add (1 , &ce->pin_count)) { +return __intel_context_pin_ww (ce->parent, ww); + }else { +return 0 ; + } + }else { +return __intel_context_pin_ww (ce, ww); + } +} + +void intel_context_unpin (struct intel_context *ce) +{ +if (intel_context_is_child (ce)) { +if (atomic_fetch_add (-1 , &ce->pin_count) ==1 ) +__intel_context_unpin (ce->parent); + }else { +__intel_context_unpin (ce); + } +} +#if IS_ENABLED (CONFIG_DRM_I915_SELFTEST )#include "selftest_context.c" #endif
drivers/gpu/drm/i915/gt/intel_context.h +5 -38
64 lines not on this page · raw message
drivers/gpu/drm/i915/gt/intel_context_types.h +2 -2
@@ -39,8 +39,8 @@ struct intel_context_ops {void (*ban)(struct intel_context *ce,struct i915_request *rq); -int (*pre_pin)(struct intel_context *ce,struct i915_gem_ww_ctx *ww,void **vaddr); -int (*pin)(struct intel_context *ce,void *vaddr); +int (*pre_pin)(struct intel_context *ce,struct i915_gem_ww_ctx *ww); +int (*pin)(struct intel_context *ce);void (*unpin)(struct intel_context *ce);void (*post_unpin)(struct intel_context *ce);
drivers/gpu/drm/i915/gt/intel_execlists_submission.c +12 -13
@@ -2503,16 +2503,17 @@ static void execlists_submit_request(struct i915_request *request)static int __execlists_context_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww,void **vaddr) +struct i915_gem_ww_ctx *ww) {int err; - err =lrc_pre_pin (ce, engine, ww, vaddr); + err =lrc_pre_pin (ce, engine, ww);if (err)return err;if (!__test_and_set_bit (CONTEXT_INIT_BIT , &ce->flags)) { -lrc_init_state (ce, engine, *vaddr); +lrc_init_state (ce, engine, ce->lrc_reg_state - +LRC_STATE_OFFSET /sizeof (*ce->lrc_reg_state));__i915_gem_object_flush_map (ce->state->obj,0 , engine->context_size); }
@@ -2521,15 +2522,14 @@ __execlists_context_pre_pin(struct intel_context *ce, }static int execlists_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { -return __execlists_context_pre_pin (ce, ce->engine, ww, vaddr); +return __execlists_context_pre_pin (ce, ce->engine, ww); } -static int execlists_context_pin (struct intel_context *ce,void *vaddr) +static int execlists_context_pin (struct intel_context *ce) { -return lrc_pin (ce, ce->engine, vaddr); +return lrc_pin (ce, ce->engine); }static int execlists_context_alloc (struct intel_context *ce)
@@ -3514,20 +3514,19 @@ static int virtual_context_alloc(struct intel_context *ce) }static int virtual_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) {struct virtual_engine *ve =container_of (ce,typeof (*ve), context);/* Note: we must use a real engine class for setting up reg state */ -return __execlists_context_pre_pin (ce, ve->siblings[0 ], ww, vaddr); +return __execlists_context_pre_pin (ce, ve->siblings[0 ], ww); } -static int virtual_context_pin (struct intel_context *ce,void *vaddr) +static int virtual_context_pin (struct intel_context *ce) {struct virtual_engine *ve =container_of (ce,typeof (*ve), context); -return lrc_pin (ce, ve->siblings[0 ], vaddr); +return lrc_pin (ce, ve->siblings[0 ]); }static void virtual_context_enter (struct intel_context *ce)
drivers/gpu/drm/i915/gt/intel_lrc.c +9 -9
@@ -896,30 +896,30 @@ void lrc_reset(struct intel_context *ce)int lrc_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr) +struct i915_gem_ww_ctx *ww) { +void *vaddr;GEM_BUG_ON (!ce->state);GEM_BUG_ON (!i915_vma_is_pinned (ce->state)); - *vaddr =i915_gem_object_pin_map (ce->state->obj, + vaddr =i915_gem_object_pin_map (ce->state->obj,i915_coherent_map_type (ce->engine->i915, ce->state->obj, false) |I915_MAP_OVERRIDE ); -return PTR_ERR_OR_ZERO (*vaddr); + ce->lrc_reg_state = vaddr +LRC_STATE_OFFSET ; + +return PTR_ERR_OR_ZERO (vaddr); }int lrc_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr) +struct intel_engine_cs *engine) { - ce->lrc_reg_state = vaddr +LRC_STATE_OFFSET ; -if (!__test_and_set_bit (CONTEXT_INIT_BIT , &ce->flags)) -lrc_init_state (ce, engine, vaddr); +lrc_init_state (ce, engine, + (void *)ce->lrc_reg_state -LRC_STATE_OFFSET ); ce->lrc.lrca =lrc_update_regs (ce, engine, ce->ring->tail);return 0 ;
drivers/gpu/drm/i915/gt/intel_lrc.h +2 -4
@@ -38,12 +38,10 @@ void lrc_destroy(struct kref *kref);int lrc_pre_pin (struct intel_context *ce,struct intel_engine_cs *engine, -struct i915_gem_ww_ctx *ww, -void **vaddr); +struct i915_gem_ww_ctx *ww);int lrc_pin (struct intel_context *ce, -struct intel_engine_cs *engine, -void *vaddr); +struct intel_engine_cs *engine);void lrc_unpin (struct intel_context *ce);void lrc_post_unpin (struct intel_context *ce);
drivers/gpu/drm/i915/gt/intel_ring_submission.c +2 -3
@@ -471,8 +471,7 @@ static int ring_context_init_default_state(struct intel_context *ce, }static int ring_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww, -void **unused) +struct i915_gem_ww_ctx *ww) {struct i915_address_space *vm;int err =0 ;
@@ -575,7 +574,7 @@ static int ring_context_alloc(struct intel_context *ce)return 0 ; } -static int ring_context_pin (struct intel_context *ce,void *unused) +static int ring_context_pin (struct intel_context *ce) {return 0 ; }
drivers/gpu/drm/i915/gt/mock_engine.c +2 -2
@@ -167,12 +167,12 @@ static int mock_context_alloc(struct intel_context *ce) }static int mock_context_pre_pin (struct intel_context *ce, -struct i915_gem_ww_ctx *ww,void **unused) +struct i915_gem_ww_ctx *ww) {return 0 ; } -static int mock_context_pin (struct intel_context *ce,void *unused) +static int mock_context_pin (struct intel_context *ce) {return 0 ; }
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +164 -19
244 lines not on this page · raw message
Signature · 1 line
2.28.0
RFC PATCH 18/42 drm/i915/guc: Add multi-lrc context registration
In reply to Matthew BrostAdd multi-lrc context registration H2G. In addition a workqueue and process descriptor are setup during multi-lrc context registration as these data structures are needed for multi-lrc submission.
--- drivers/gpu/drm/i915/gt/intel_context_types.h | 6 + drivers/gpu/drm/i915/gt/intel_lrc.c | 5 + .../gpu/drm/i915/gt/uc/abi/guc_actions_abi.h | 1 + drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h | 2 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 118 +++++++++++++++++- 5 files changed, 130 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/gt/intel_context_types.h +6 -0
@@ -211,9 +211,15 @@ struct intel_context {/* Pointer to parent */ struct intel_context *parent; +/* GuC workqueue head / tail - only applies to parent */ +u16 guc_wqi_tail; +u16 guc_wqi_head; +/* Number of children if parent */ u8 guc_number_children; +u8 parent_page;/* if set, page num reserved for parent context */ +/* * GuC priority management */
drivers/gpu/drm/i915/gt/intel_lrc.c +5 -0
@@ -810,6 +810,11 @@ __lrc_alloc_state(struct intel_context *ce, struct intel_engine_cs *engine) context_size +=PAGE_SIZE ; } +if (intel_context_is_parent (ce)) { + ce->parent_page = context_size /PAGE_SIZE ; + context_size +=PAGE_SIZE ; + } + obj =i915_gem_object_create_lmem (engine->i915, context_size,0 );if (IS_ERR (obj)) obj =i915_gem_object_create_shmem (engine->i915, context_size);
drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h +1 -0
@@ -142,6 +142,7 @@ enum intel_guc_action {INTEL_GUC_ACTION_REGISTER_COMMAND_TRANSPORT_BUFFER =0x4505 ,INTEL_GUC_ACTION_DEREGISTER_COMMAND_TRANSPORT_BUFFER =0x4506 ,INTEL_GUC_ACTION_DEREGISTER_CONTEXT_DONE =0x4600 , +INTEL_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC =0x4601 ,INTEL_GUC_ACTION_RESET_CLIENT =0x5B01 , INTEL_GUC_ACTION_LIMIT };
drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h +1 -1
@@ -51,7 +51,7 @@#define GUC_DOORBELL_INVALID 256 -#define GUC_WQ_SIZE (PAGE_SIZE * 2) +#define GUC_WQ_SIZE (PAGE_SIZE / 2)/* Work queue item header definitions */ #define WQ_STATUS_ACTIVE 1
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +117 -1
175 lines not on this page · raw message
Signature · 1 line
2.28.0
RFC PATCH 19/42 drm/i915/guc: Ensure GuC schedule operations do not operate on child contexts
In reply to Matthew BrostIn GuC parent-child contexts the parent context controls the scheduling, ensure only the parent does the scheduling operations.
--- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 48 +++++++++++++++---- 1 file changed, 38 insertions(+), 10 deletions(-)
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +38 -10
@@ -402,6 +402,18 @@ static inline void clr_context_banned(struct intel_context *ce) ce->guc_state.sched_state &= ~SCHED_STATE_BANNED ; } +static inline struct intel_context * +to_parent (struct intel_context *ce) +{ +return intel_context_is_child (ce) ? ce->parent : ce; +} + +static inline struct intel_context * +request_to_scheduling_context (struct i915_request *rq) +{ +return to_parent (rq->context); +} +static inline bool context_guc_id_invalid (struct intel_context *ce) {return (ce->guc_id ==GUC_INVALID_LRC_ID );
@@ -2341,6 +2353,7 @@ static void __guc_context_sched_disable(struct intel_guc *guc,GEM_BUG_ON (guc_id ==GUC_INVALID_LRC_ID );#endif +GEM_BUG_ON (intel_context_is_child (ce));trace_intel_context_sched_disable (ce); guc_submission_send_busy_loop(guc, action, ARRAY_SIZE(action),
@@ -2546,6 +2559,8 @@ static void guc_context_sched_disable(struct intel_context *ce)u16 guc_id;bool enabled; +GEM_BUG_ON (intel_context_is_child (ce)); +if (submission_disabled (guc) ||context_guc_id_invalid (ce) || !lrc_desc_registered (guc, ce->guc_id)) {clr_context_enabled (ce);
@@ -2941,6 +2956,8 @@ static void guc_signal_context_fence(struct intel_context *ce) {unsigned long flags; +GEM_BUG_ON (intel_context_is_child (ce)); +spin_lock_irqsave (&ce->guc_state.lock, flags);clr_context_wait_for_deregister_to_register (ce);__guc_signal_context_fence (ce);
@@ -3026,10 +3043,21 @@ static bool context_needs_lrc_desc_pin(struct intel_context *ce, bool new_guc_id !submission_disabled (ce_to_guc (ce)); } +static void clear_lrca_dirty (struct intel_context *ce) +{ +struct intel_context *child; + +GEM_BUG_ON (intel_context_is_child (ce)); + +clear_bit (CONTEXT_LRCA_DIRTY , &ce->flags); +for_each_child (ce, child) +clear_bit (CONTEXT_LRCA_DIRTY , &child->flags); +} +static int tasklet_pin_guc_id (struct guc_submit_engine *gse,struct i915_request *rq) { -struct intel_context *ce = rq->context; +struct intel_context *ce =request_to_scheduling_context (rq);int ret =0 ;lockdep_assert_held (&gse->sched_engine.lock);
@@ -3061,7 +3089,7 @@ static int tasklet_pin_guc_id(struct guc_submit_engine *gse, gse->submission_stall_reason =STALL_SCHED_DISABLE ; } -clear_bit (CONTEXT_LRCA_DIRTY , &ce->flags); +clear_lrca_dirty (ce); out: gse->total_num_rq_with_no_guc_id -= ce->guc_num_rq_submit_no_id;GEM_BUG_ON (gse->total_num_rq_with_no_guc_id <0 );
@@ -3092,7 +3120,7 @@ static int tasklet_pin_guc_id(struct guc_submit_engine *gse,static int guc_request_alloc (struct i915_request *rq) { -struct intel_context *ce = rq->context; +struct intel_context *ce =request_to_scheduling_context (rq);struct intel_guc *guc =ce_to_guc (ce);struct guc_submit_engine *gse =ce_to_gse (ce);unsigned long flags;
@@ -3143,11 +3171,12 @@ static int guc_request_alloc(struct i915_request *rq)* persistent until the generated request is retired. Thus, sealing these * race conditions. * -* There is no need for a lock here as the timeline mutex ensures at -* most one context can be executing this code path at once. The -* guc_id_ref is incremented once for every request in flight and -* decremented on each retire. When it is zero, a lock around the -* increment (in pin_guc_id) is needed to seal a race with unpin_guc_id. +* There is no need for a lock here as the timeline mutex (or +* parallel_submit mutex in the case of multi-lrc) ensures at most one +* context can be executing this code path at once. The guc_id_ref is +* incremented once for every request in flight and decremented on each +* retire. When it is zero, a lock around the increment (in pin_guc_id) +* is needed to seal a race with unpin_guc_id. */ if (atomic_add_unless (&ce->guc_id_ref,1 ,0 )) goto out;
@@ -3185,8 +3214,7 @@ static int guc_request_alloc(struct i915_request *rq) } } -clear_bit (CONTEXT_LRCA_DIRTY , &ce->flags); - +clear_lrca_dirty (ce); out:incr_num_rq_not_ready (ce);
Signature · 6 lines
2.28.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
RFC PATCH 20/42 drm/i915/guc: Assign contexts in parent-child relationship consecutive guc_ids
In reply to Matthew BrostAssign contexts in parent-child relationship consecutive guc_ids. This is accomplished by partitioning guc_id space between ones that need to be consecutive (1/16 available guc_ids) and ones that do not (15/16 of available guc_ids). The consecutive search is implemented via the bitmap API.
This is a precursor to the full GuC multi-lrc implementation but aligns to how GuC mutli-lrc interface is defined - guc_ids must be consecutive when using the GuC multi-lrc interface.
--- drivers/gpu/drm/i915/gt/intel_context.h | 6 + drivers/gpu/drm/i915/gt/intel_reset.c | 3 +- drivers/gpu/drm/i915/gt/uc/intel_guc.h | 7 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 222 ++++++++++++------ .../i915/gt/uc/intel_guc_submission_types.h | 10 + 5 files changed, 178 insertions(+), 70 deletions(-)
drivers/gpu/drm/i915/gt/intel_context.h +6 -0
@@ -51,6 +51,12 @@ static inline bool intel_context_is_parent(struct intel_context *ce)return !!ce->guc_number_children; } +static inline struct intel_context * +intel_context_to_parent (struct intel_context *ce) +{ +return intel_context_is_child (ce) ? ce->parent : ce; +} +void intel_context_bind_parent_child (struct intel_context *parent,struct intel_context *child);
drivers/gpu/drm/i915/gt/intel_reset.c +2 -1
@@ -843,6 +843,7 @@ static void reset_finish(struct intel_gt *gt, intel_engine_mask_t awake)static void nop_submit_request (struct i915_request *request) { +struct intel_context *ce =intel_context_to_parent (request->context);RQ_TRACE (request,"-EIO\n" );/*
@@ -851,7 +852,7 @@ static void nop_submit_request(struct i915_request *request)* this for now. */ if (intel_engine_uses_guc (request->engine)) -intel_guc_decr_num_rq_not_ready (request->context); +intel_guc_decr_num_rq_not_ready (ce); request =i915_request_mark_eio (request);if (request) {
drivers/gpu/drm/i915/gt/uc/intel_guc.h +5 -2
@@ -24,6 +24,7 @@ struct __guc_ads_blob;enum {GUC_SUBMIT_ENGINE_SINGLE_LRC , +GUC_SUBMIT_ENGINE_MULTI_LRC ,GUC_SUBMIT_ENGINE_MAX };
@@ -59,8 +60,10 @@ struct intel_guc {struct ida guc_ids;u32 num_guc_ids;u32 max_guc_ids; -struct list_head guc_id_list_no_ref; -struct list_head guc_id_list_unpinned; +unsigned long *guc_ids_bitmap; +#define MAX_GUC_ID_ORDER 3 +struct list_head guc_id_list_no_ref[MAX_GUC_ID_ORDER +1 ]; +struct list_head guc_id_list_unpinned[MAX_GUC_ID_ORDER +1 ];spinlock_t destroy_lock;struct list_head destroyed_contexts;
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +155 -67
365 lines not on this page · raw message
drivers/gpu/drm/i915/gt/uc/intel_guc_submission_types.h +10 -0
@@ -22,6 +22,16 @@ struct guc_virtual_engine {/* * Object which encapsulates the globally operated on i915_sched_engine + * the GuC submission state machine described in intel_guc_submission.c. +* +* Currently we have two instances of these per GuC. One for single-lrc and one +* for multi-lrc submission. We split these into two submission engines as they +* can operate in parallel allowing a blocking condition on one not to affect +* the other. i.e. guc_ids are statically allocated between these two submission +* modes. One mode may have guc_ids exhausted which requires blocking while the +* other has plenty of guc_ids and can make forward progres. +* +* In the future if different submission use cases arise we can simply +* instantiate another of these objects and assign it to the context. */ struct guc_submit_engine {struct i915_sched_engine sched_engine;
Signature · 6 lines
2.28.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
RFC PATCH 21/42 drm/i915/guc: Add hang check to GuC submit engine
In reply to Matthew BrostThe heartbeat uses a single instance of a GuC submit engine (GSE) to do the hang check. As such if a different GSE's state machine hangs, the heartbeat cannot detect this hang. Add timer to each GSE which in turn can disable all submissions if it is hung.
--- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 36 +++++++++++++++++++ .../i915/gt/uc/intel_guc_submission_types.h | 3 ++ 2 files changed, 39 insertions(+)
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +36 -0
@@ -105,15 +105,21 @@ static bool tasklet_blocked(struct guc_submit_engine *gse)return test_bit (GSE_STATE_TASKLET_BLOCKED , &gse->flags); } +/* 2 seconds seems like a reasonable timeout waiting for a G2H */ +#define MAX_TASKLET_BLOCKED_NS 2000000000static void set_tasklet_blocked (struct guc_submit_engine *gse) {lockdep_assert_held (&gse->sched_engine.lock); +hrtimer_start_range_ns (&gse->hang_timer, +ns_to_ktime (MAX_TASKLET_BLOCKED_NS ),0 , +HRTIMER_MODE_REL_PINNED );set_bit (GSE_STATE_TASKLET_BLOCKED , &gse->flags); }static void __clr_tasklet_blocked (struct guc_submit_engine *gse) {lockdep_assert_held (&gse->sched_engine.lock); +hrtimer_cancel (&gse->hang_timer);clear_bit (GSE_STATE_TASKLET_BLOCKED , &gse->flags); }
@@ -1021,6 +1027,7 @@ static void disable_submission(struct intel_guc *guc)if (__tasklet_is_enabled (&sched_engine->tasklet)) {GEM_BUG_ON (!guc->ct.enabled);__tasklet_disable_sync_once (&sched_engine->tasklet); +hrtimer_try_to_cancel (&guc->gse[i]->hang_timer); sched_engine->tasklet.callback =NULL ; } }
@@ -3716,6 +3723,33 @@ static void guc_sched_engine_destroy(struct kref *kref)kfree (gse); } +static enum hrtimer_restart gse_hang (struct hrtimer *hrtimer) +{ +struct guc_submit_engine *gse = + container_of(hrtimer, struct guc_submit_engine, hang_timer); +struct intel_guc *guc = gse->sched_engine.private_data; + +#if IS_ENABLED (CONFIG_DRM_I915_SELFTEST ) +if (guc->gse_hang_expected) +drm_dbg (&guc_to_gt (guc)->i915->drm, +"GSE[%i] hung, disabling submission" , gse->id); +else +drm_err (&guc_to_gt (guc)->i915->drm, +"GSE[%i] hung, disabling submission" , gse->id); +#else +drm_err (&guc_to_gt (guc)->i915->drm, +"GSE[%i] hung, disabling submission" , gse->id); +#endif + +/* +* Tasklet not making forward progress, disable submission which in turn +* will kick in the heartbeat to do a full GPU reset. +*/ +disable_submission (guc); + +return HRTIMER_NORESTART ; +} +static void guc_submit_engine_init (struct intel_guc *guc,struct guc_submit_engine *gse,int id)
@@ -3733,6 +3767,8 @@ static void guc_submit_engine_init(struct intel_guc *guc, sched_engine->retire_inflight_request_prio = guc_retire_inflight_request_prio; sched_engine->private_data = guc; +hrtimer_init (&gse->hang_timer,CLOCK_MONOTONIC ,HRTIMER_MODE_REL ); + gse->hang_timer.function = gse_hang; gse->id = id; }
drivers/gpu/drm/i915/gt/uc/intel_guc_submission_types.h +3 -0
@@ -6,6 +6,8 @@#ifndef _INTEL_GUC_SUBMISSION_TYPES_H_#define _INTEL_GUC_SUBMISSION_TYPES_H_ +#include <linux/xarray.h> +#include "gt/intel_engine_types.h" #include "gt/intel_context_types.h" #include "i915_scheduler_types.h"
@@ -41,6 +43,7 @@ struct guc_submit_engine {unsigned long flags;int total_num_rq_with_no_guc_id;atomic_t num_guc_ids_not_ready; +struct hrtimer hang_timer;int id;/*
Signature · 1 line
2.28.0
RFC PATCH 22/42 drm/i915/guc: Add guc_child_context_destroy
In reply to Matthew BrostSince child contexts do not own the guc_ids or GuC context registration, child contexts can simply be freed on destroy. Add guc_child_context_destroy context operation to do this.
--- drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c | 7 +++++++ 1 file changed, 7 insertions(+)
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +7 -0
@@ -2800,6 +2800,13 @@ static void destroy_worker_func(struct work_struct *w)intel_gt_pm_unpark_work_add (gt, destroy_worker); } +/* Future patches will use this function */ +__attribute__ ((unused)) +static void guc_child_context_destroy (struct kref *kref) +{ +__guc_context_destroy (container_of(kref ,struct intel_context ,ref )); +} +static void guc_context_destroy (struct kref *kref) {struct intel_context *ce =container_of (kref,typeof (*ce), ref);
Signature · 1 line
2.28.0
RFC PATCH 23/42 drm/i915/guc: Implement multi-lrc submission
In reply to Matthew BrostImplement multi-lrc submission via a single workqueue entry and single H2G. The workqueue entry contains an updated tail value for each request, of all the contexts in the multi-lrc submission, and updates these values simultaneously. As such, the tasklet and bypass path have been updated to coalesce requests into a single submission.
--- drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h | 6 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 224 ++++++++++++++++-- drivers/gpu/drm/i915/i915_request.h | 8 + 3 files changed, 222 insertions(+), 16 deletions(-)
drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h +4 -2
@@ -64,12 +64,14 @@#define WQ_TYPE_PSEUDO (0x2 << WQ_TYPE_SHIFT)#define WQ_TYPE_INORDER (0x3 << WQ_TYPE_SHIFT)#define WQ_TYPE_NOOP (0x4 << WQ_TYPE_SHIFT) -#define WQ_TARGET_SHIFT 10 +#define WQ_TYPE_MULTI_LRC (0x5 << WQ_TYPE_SHIFT) +#define WQ_TARGET_SHIFT 8#define WQ_LEN_SHIFT 16#define WQ_NO_WCFLUSH_WAIT (1 << 27)#define WQ_PRESENT_WORKLOAD (1 << 28) -#define WQ_RING_TAIL_SHIFT 20 +#define WQ_GUC_ID_SHIFT 0 +#define WQ_RING_TAIL_SHIFT 18#define WQ_RING_TAIL_MAX 0x7FF/* 2^11 QWords */ #define WQ_RING_TAIL_MASK (WQ_RING_TAIL_MAX << WQ_RING_TAIL_SHIFT)
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +210 -14
321 lines not on this page · raw message
drivers/gpu/drm/i915/i915_request.h +8 -0
@@ -145,6 +145,14 @@ enum {* tasklet that the guc_id isn't pinned. */ I915_FENCE_FLAG_GUC_ID_NOT_PINNED , + +/* +* I915_FENCE_FLAG_SUBMIT_PARALLEL - request with a context in a +* parent-child relationship (parallel submission, multi-lrc) should +* trigger a submission to the GuC rather than just moving the context +* tail. +*/ + I915_FENCE_FLAG_SUBMIT_PARALLEL, };/**
Signature · 6 lines
2.28.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
RFC PATCH 24/42 drm/i915/guc: Insert submit fences between requests in parent-child relationship
In reply to Matthew BrostThe GuC must receive requests in the order submitted for contexts in a parent-child relationship to function correctly. To ensure this, insert a submit fence between the current request and last request submitted for requests / contexts in a parent child relationship. This is conceptually similar to a single timeline.
--- drivers/gpu/drm/i915/gt/intel_context.c | 2 + drivers/gpu/drm/i915/gt/intel_context.h | 5 + drivers/gpu/drm/i915/gt/intel_context_types.h | 3 + .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 3 +- drivers/gpu/drm/i915/i915_request.c | 119 ++++++++++++++---- 5 files changed, 105 insertions(+), 27 deletions(-)
drivers/gpu/drm/i915/gt/intel_context.c +2 -0
@@ -486,6 +486,8 @@ void intel_context_fini(struct intel_context *ce) {struct intel_context *child, *next; +if (ce->last_rq) +i915_request_put (ce->last_rq);if (ce->timeline)intel_timeline_put (ce->timeline);i915_vm_put (ce->vm);
drivers/gpu/drm/i915/gt/intel_context.h +5 -0
@@ -57,6 +57,11 @@ intel_context_to_parent(struct intel_context *ce)return intel_context_is_child (ce) ? ce->parent : ce; } +static inline bool intel_context_is_parallel (struct intel_context *ce) +{ +return intel_context_is_child (ce) ||intel_context_is_parent (ce); +} +void intel_context_bind_parent_child (struct intel_context *parent,struct intel_context *child);
drivers/gpu/drm/i915/gt/intel_context_types.h +3 -0
@@ -225,6 +225,9 @@ struct intel_context {*/ u8 guc_prio;u32 guc_prio_count[GUC_CLIENT_PRIORITY_NUM ]; + +/* Last request submitted on a parent */ + struct i915_request *last_rq; };#endif /* __INTEL_CONTEXT_TYPES__ */
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +1 -2
@@ -812,8 +812,7 @@ static inline int rq_prio(const struct i915_request *rq)static inline bool is_multi_lrc_rq (struct i915_request *rq) { -return intel_context_is_child (rq->context) || -intel_context_is_parent (rq->context); +return intel_context_is_parallel (rq->context); }/*
drivers/gpu/drm/i915/i915_request.c +94 -25
148 lines not on this page · raw message
Signature · 1 line
2.28.0
RFC PATCH 25/42 drm/i915/guc: Implement multi-lrc reset
In reply to Matthew BrostUpdate context and full GPU reset to work with multi-lrc. The idea is parent context tracks all the active requests inflight for itself and its' children. The parent context owns the reset replaying / canceling requests as needed.
--- drivers/gpu/drm/i915/gt/intel_context.c | 12 +++-- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 54 ++++++++++++++----- 2 files changed, 47 insertions(+), 19 deletions(-)
drivers/gpu/drm/i915/gt/intel_context.c +7 -5
@@ -605,20 +605,22 @@ struct i915_request *intel_context_create_request(struct intel_context *ce)struct i915_request *intel_context_find_active_request (struct intel_context *ce) { +struct intel_context *parent =intel_context_is_child (ce) ? + ce->parent : ce;struct i915_request *rq, *active =NULL ;unsigned long flags;GEM_BUG_ON (!intel_engine_uses_guc (ce->engine)); -spin_lock_irqsave (&ce->guc_active.lock, flags); -list_for_each_entry_reverse (rq, &ce->guc_active.requests, +spin_lock_irqsave (&parent->guc_active.lock, flags); +list_for_each_entry_reverse (rq, &parent->guc_active.requests, sched.link) { -if (i915_request_completed (rq)) +if (i915_request_completed (rq) && rq->context == ce)break ; - active = rq; + active = (rq->context == ce) ? rq : active; } -spin_unlock_irqrestore (&ce->guc_active.lock, flags); +spin_unlock_irqrestore (&parent->guc_active.lock, flags);return active; }
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +40 -14
104 lines not on this page · raw message
Signature · 6 lines
2.28.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
RFC PATCH 26/42 drm/i915/guc: Update debugfs for GuC multi-lrc
In reply to Matthew BrostDisplay the workqueue status in debugfs for GuC contexts that are in parent-child relationship.
--- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 56 +++++++++++++------ 1 file changed, 39 insertions(+), 17 deletions(-)
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +39 -17
@@ -4489,31 +4489,53 @@ void intel_guc_submission_print_info(struct intel_guc *guc,gse_log_submission_info (guc->gse[i], p, i); } +static inline void guc_log_context (struct drm_printer *p, +struct intel_context *ce) +{ +drm_printf (p,"GuC lrc descriptor %u:\n" , ce->guc_id); +drm_printf (p,"\tHW Context Desc: 0x%08x\n" , ce->lrc.lrca); +drm_printf (p,"\t\tLRC Head: Internal %u, Memory %u\n" , + ce->ring->head, + ce->lrc_reg_state[CTX_RING_HEAD ]); +drm_printf (p,"\t\tLRC Tail: Internal %u, Memory %u\n" , + ce->ring->tail, + ce->lrc_reg_state[CTX_RING_TAIL ]); +drm_printf (p,"\t\tContext Pin Count: %u\n" , +atomic_read (&ce->pin_count)); +drm_printf (p,"\t\tGuC ID Ref Count: %u\n" , +atomic_read (&ce->guc_id_ref)); +drm_printf (p,"\t\tNumber Requests Not Ready: %u\n" , +atomic_read (&ce->guc_num_rq_not_ready)); +drm_printf (p,"\t\tSchedule State: 0x%x, 0x%x\n\n" , + ce->guc_state.sched_state, +atomic_read (&ce->guc_sched_state_no_lock)); +} +void intel_guc_submission_print_context_info (struct intel_guc *guc,struct drm_printer *p) {struct intel_context *ce;unsigned long index;xa_for_each (&guc->context_lookup, index, ce) { -drm_printf (p,"GuC lrc descriptor %u:\n" , ce->guc_id); -drm_printf (p,"\tHW Context Desc: 0x%08x\n" , ce->lrc.lrca); -drm_printf (p,"\t\tLRC Head: Internal %u, Memory %u\n" , - ce->ring->head, - ce->lrc_reg_state[CTX_RING_HEAD ]); -drm_printf (p,"\t\tLRC Tail: Internal %u, Memory %u\n" , - ce->ring->tail, - ce->lrc_reg_state[CTX_RING_TAIL ]); -drm_printf (p,"\t\tContext Pin Count: %u\n" , -atomic_read (&ce->pin_count)); -drm_printf (p,"\t\tGuC ID Ref Count: %u\n" , -atomic_read (&ce->guc_id_ref)); -drm_printf (p,"\t\tNumber Requests Not Ready: %u\n" , -atomic_read (&ce->guc_num_rq_not_ready)); -drm_printf (p,"\t\tSchedule State: 0x%x, 0x%x\n\n" , - ce->guc_state.sched_state, -atomic_read (&ce->guc_sched_state_no_lock)); +GEM_BUG_ON (intel_context_is_child (ce)); +guc_log_context (p, ce);guc_log_context_priority (p, ce); + +if (intel_context_is_parent (ce)) { +struct guc_process_desc *desc =__get_process_desc (ce); +struct intel_context *child; + +drm_printf (p,"\t\tWQI Head: %u\n" , +READ_ONCE (desc->head)); +drm_printf (p,"\t\tWQI Tail: %u\n" , +READ_ONCE (desc->tail)); +drm_printf (p,"\t\tWQI Status: %u\n\n" , +READ_ONCE (desc->wq_status)); + +for_each_child (ce, child) +guc_log_context (p, child); + } } }
Signature · 6 lines
2.28.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
RFC PATCH 27/42 drm/i915: Connect UAPI to GuC multi-lrc interface
In reply to Matthew BrostIntroduce 'set parallel submit' extension to connect UAPI to GuC multi-lrc interface. Kernel doc in new uAPI should explain it all.
--- drivers/gpu/drm/i915/gem/i915_gem_context.c | 157 +++++++++++++++++- .../gpu/drm/i915/gem/i915_gem_context_types.h | 6 + drivers/gpu/drm/i915/gt/intel_context_types.h | 8 +- drivers/gpu/drm/i915/gt/intel_engine.h | 12 +- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 6 +- .../drm/i915/gt/intel_execlists_submission.c | 6 +- drivers/gpu/drm/i915/gt/selftest_execlists.c | 12 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 111 +++++++++++-- include/uapi/drm/i915_drm.h | 118 +++++++++++++ 9 files changed, 407 insertions(+), 29 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_context.c +155 -2
@@ -519,9 +519,149 @@ set_proto_ctx_engines_bond(struct i915_user_extension __user *base, void *data)return 0 ; } +static int +set_proto_ctx_engines_parallel_submit (struct i915_user_extension __user *base, +void *data) +{ +struct i915_context_engines_parallel_submit __user *ext = +container_of_user (base,typeof (*ext), base); +const struct set_proto_ctx_engines *set = data; +struct drm_i915_private *i915 = set->i915; +u64 flags; +int err =0 , n, i, j; +u16 slot, width, num_siblings; +struct intel_engine_cs **siblings =NULL ; +intel_engine_mask_t prev_mask; + +/* Disabling for now */ +return -ENODEV ; + +if (!(intel_uc_uses_guc_submission (&i915->gt.uc))) +return -ENODEV ; + +if (get_user (slot, &ext->engine_index)) +return -EFAULT ; + +if (get_user (width, &ext->width)) +return -EFAULT ; + +if (get_user (num_siblings, &ext->num_siblings)) +return -EFAULT ; + +if (slot >= set->num_engines) { +drm_dbg (&i915->drm,"Invalid placement value, %d >= %d\n" , + slot, set->num_engines); +return -EINVAL ; + } + +if (set->engines[slot].type !=I915_GEM_ENGINE_TYPE_INVALID ) { +drm_dbg (&i915->drm, +"Invalid placement[%d], already occupied\n" , slot); +return -EINVAL ; + } + +if (get_user (flags, &ext->flags)) +return -EFAULT ; + +if (flags) { +drm_dbg (&i915->drm,"Unknown flags 0x%02llx" , flags); +return -EINVAL ; + } + +for (n =0 ; n <ARRAY_SIZE (ext->mbz64); n++) { + err =check_user_mbz (&ext->mbz64[n]); +if (err) +return err; + } + +if (width <2 ) { +drm_dbg (&i915->drm,"Width (%d) < 2 \n" , width); +return -EINVAL ; + } + +if (num_siblings <1 ) { +drm_dbg (&i915->drm,"Number siblings (%d) < 1 \n" , + num_siblings); +return -EINVAL ; + } + + siblings =kmalloc_array (num_siblings * width, +sizeof (*siblings), +GFP_KERNEL ); +if (!siblings) +return -ENOMEM ; + +/* Create contexts / engines */ +for (i =0 ; i < width; ++i) { +intel_engine_mask_t current_mask =0 ; +struct i915_engine_class_instance prev_engine; + +for (j =0 ; j < num_siblings; ++j) { +struct i915_engine_class_instance ci; + n = i * num_siblings + j; + +if (copy_from_user (&ci, &ext->engines[n],sizeof (ci))) { + err = -EFAULT ; + goto out_err; + } + + siblings[n] = +intel_engine_lookup_user (i915, ci.engine_class, + ci.engine_instance); +if (!siblings[n]) { +drm_dbg (&i915->drm, +"Invalid sibling[%d]: { class:%d, inst:%d }\n" , + n, ci.engine_class, ci.engine_instance); + err = -EINVAL ; + goto out_err; + } + +if (n) { +if (prev_engine.engine_class != + ci.engine_class) { +drm_dbg (&i915->drm, +"Mismatched class %d, %d\n" , + prev_engine.engine_class, + ci.engine_class); + err = -EINVAL ; + goto out_err; + } + } + + prev_engine = ci; + current_mask |= siblings[n]->logical_mask; + } + +if (i >0 ) { +if (current_mask != prev_mask <<1 ) { +drm_dbg (&i915->drm, +"Non contiguous logical mask 0x%x, 0x%x\n" , + prev_mask, current_mask); + err = -EINVAL ; + goto out_err; + } + } + prev_mask = current_mask; + } + + set->engines[slot].type =I915_GEM_ENGINE_TYPE_PARALLEL ; + set->engines[slot].num_siblings = num_siblings; + set->engines[slot].width = width; + set->engines[slot].siblings = siblings; + +return 0 ; + +out_err: +kfree (siblings); + +return err; +} +static const i915_user_extension_fn set_proto_ctx_engines_extensions[] = { [I915_CONTEXT_ENGINES_EXT_LOAD_BALANCE ] = set_proto_ctx_engines_balance, [I915_CONTEXT_ENGINES_EXT_BOND ] = set_proto_ctx_engines_bond, + [I915_CONTEXT_ENGINES_EXT_PARALLEL_SUBMIT ] = + set_proto_ctx_engines_parallel_submit, }; static int set_proto_ctx_engines(struct drm_i915_file_private *fpriv,
@@ -942,7 +1082,7 @@ static struct i915_gem_engines *user_engines(struct i915_gem_context *ctx, e =alloc_engines (num_engines);for (n =0 ; n < num_engines; n++) { -struct intel_context *ce; +struct intel_context *ce, *child;int ret;switch (pe[n].type) {
@@ -952,7 +1092,13 @@ static struct i915_gem_engines *user_engines(struct i915_gem_context *ctx,case I915_GEM_ENGINE_TYPE_BALANCED : ce =intel_engine_create_virtual (pe[n].siblings, - pe[n].num_siblings); + pe[n].num_siblings,0 ); +break ; + +case I915_GEM_ENGINE_TYPE_PARALLEL : + ce =intel_engine_create_parallel (pe[n].siblings, + pe[n].num_siblings, + pe[n].width);break ;case I915_GEM_ENGINE_TYPE_INVALID :
@@ -973,6 +1119,13 @@ static struct i915_gem_engines *user_engines(struct i915_gem_context *ctx, err =ERR_PTR (ret); goto free_engines; } +for_each_child (ce, child) { + ret =intel_context_set_gem (child, ctx, pe->sseu); +if (ret) { + err =ERR_PTR (ret); + goto free_engines; + } + } } e->num_engines = num_engines;
drivers/gpu/drm/i915/gem/i915_gem_context_types.h +6 -0
@@ -78,6 +78,9 @@ enum i915_gem_engine_type {/** @I915_GEM_ENGINE_TYPE_BALANCED: A load-balanced engine set */ I915_GEM_ENGINE_TYPE_BALANCED , + +/** @I915_GEM_ENGINE_TYPE_PARALLEL: A parallel engine set */ + I915_GEM_ENGINE_TYPE_PARALLEL, };/**
@@ -108,6 +111,9 @@ struct i915_gem_proto_engine {/** @num_siblings: Number of balanced siblings */ unsigned int num_siblings; +/** @width: Width of each sibling */ +unsigned int width; +/** @siblings: Balanced siblings */ struct intel_engine_cs **siblings;
drivers/gpu/drm/i915/gt/intel_context_types.h +6 -2
@@ -55,9 +55,13 @@ struct intel_context_ops {void (*reset)(struct intel_context *ce);void (*destroy)(struct kref *kref); -/* virtual engine/context interface */ +/* virtual/parallel engine/context interface */ struct intel_context *(*create_virtual)(struct intel_engine_cs **engine, -unsigned int count); +unsigned int count, +unsigned long flags); +struct intel_context *(*create_parallel)(struct intel_engine_cs **engines, +unsigned int num_siblings, +unsigned int width);struct intel_engine_cs *(*get_sibling)(struct intel_engine_cs *engine,unsigned int sibling); };
drivers/gpu/drm/i915/gt/intel_engine.h +11 -1
@@ -279,9 +279,19 @@ intel_engine_has_preempt_reset(const struct intel_engine_cs *engine)return intel_engine_has_preemption (engine); } +#define FORCE_VIRTUAL BIT(0)struct intel_context *intel_engine_create_virtual (struct intel_engine_cs **siblings, -unsigned int count); +unsigned int count,unsigned long flags); + +static inline struct intel_context * +intel_engine_create_parallel (struct intel_engine_cs **engines, +unsigned int num_engines, +unsigned int width) +{ +GEM_BUG_ON (!engines[0 ]->cops->create_parallel); +return engines[0 ]->cops->create_parallel (engines, num_engines, width); +}static inline bool intel_virtual_engine_has_heartbeat (const struct intel_engine_cs *engine)
drivers/gpu/drm/i915/gt/intel_engine_cs.c +3 -3
@@ -1866,16 +1866,16 @@ ktime_t intel_engine_get_busy_time(struct intel_engine_cs *engine, ktime_t *now)struct intel_context *intel_engine_create_virtual (struct intel_engine_cs **siblings, -unsigned int count) +unsigned int count,unsigned long flags) {if (count ==0 )return ERR_PTR (-EINVAL ); -if (count ==1 ) +if (count ==1 && !(flags &FORCE_VIRTUAL ))return intel_context_create (siblings[0 ]);GEM_BUG_ON (!siblings[0 ]->cops->create_virtual); -return siblings[0 ]->cops->create_virtual (siblings, count); +return siblings[0 ]->cops->create_virtual (siblings, count, flags); } struct i915_request *
drivers/gpu/drm/i915/gt/intel_execlists_submission.c +4 -2
@@ -195,7 +195,8 @@ static struct virtual_engine *to_virtual_engine(struct intel_engine_cs *engine) }static struct intel_context * -execlists_create_virtual (struct intel_engine_cs **siblings,unsigned int count); +execlists_create_virtual (struct intel_engine_cs **siblings,unsigned int count, +unsigned long flags);static struct i915_request * __active_request(const struct intel_timeline * const tl,
@@ -3729,7 +3730,8 @@ static void virtual_submit_request(struct i915_request *rq) }static struct intel_context * -execlists_create_virtual (struct intel_engine_cs **siblings,unsigned int count) +execlists_create_virtual (struct intel_engine_cs **siblings,unsigned int count, +unsigned long flags) {struct virtual_engine *ve;unsigned int n;
drivers/gpu/drm/i915/gt/selftest_execlists.c +6 -6
@@ -3727,7 +3727,7 @@ static int nop_virtual_engine(struct intel_gt *gt,GEM_BUG_ON (!nctx || nctx >ARRAY_SIZE (ve));for (n =0 ; n < nctx; n++) { - ve[n] =intel_engine_create_virtual (siblings, nsibling); + ve[n] =intel_engine_create_virtual (siblings, nsibling,0 );if (IS_ERR (ve[n])) { err =PTR_ERR (ve[n]); nctx = n;
@@ -3923,7 +3923,7 @@ static int mask_virtual_engine(struct intel_gt *gt,* restrict it to our desired engine within the virtual engine. */ - ve =intel_engine_create_virtual (siblings, nsibling); + ve =intel_engine_create_virtual (siblings, nsibling,0 );if (IS_ERR (ve)) { err =PTR_ERR (ve); goto out_close;
@@ -4054,7 +4054,7 @@ static int slicein_virtual_engine(struct intel_gt *gt,i915_request_add (rq); } - ce =intel_engine_create_virtual (siblings, nsibling); + ce =intel_engine_create_virtual (siblings, nsibling,0 );if (IS_ERR (ce)) { err =PTR_ERR (ce); goto out;
@@ -4106,7 +4106,7 @@ static int sliceout_virtual_engine(struct intel_gt *gt,/* XXX We do not handle oversubscription and fairness with normal rq */ for (n =0 ; n < nsibling; n++) { - ce =intel_engine_create_virtual (siblings, nsibling); + ce =intel_engine_create_virtual (siblings, nsibling,0 );if (IS_ERR (ce)) { err =PTR_ERR (ce); goto out;
@@ -4208,7 +4208,7 @@ static int preserved_virtual_engine(struct intel_gt *gt,if (err) goto out_scratch; - ve =intel_engine_create_virtual (siblings, nsibling); + ve =intel_engine_create_virtual (siblings, nsibling,0 );if (IS_ERR (ve)) { err =PTR_ERR (ve); goto out_scratch;
@@ -4348,7 +4348,7 @@ static int reset_virtual_engine(struct intel_gt *gt,if (igt_spinner_init (&spin, gt))return -ENOMEM ; - ve =intel_engine_create_virtual (siblings, nsibling); + ve =intel_engine_create_virtual (siblings, nsibling,0 );if (IS_ERR (ve)) { err =PTR_ERR (ve); goto out_spin;
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +98 -13
177 lines not on this page · raw message
include/uapi/drm/i915_drm.h +118 -0
136 lines not on this page · raw message
Signature · 6 lines
2.28.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
RFC PATCH 28/42 drm/i915/guc: Add basic GuC multi-lrc selftest
In reply to Matthew BrostAdd very basic (single submission) multi-lrc selftest.
--- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 1 + .../drm/i915/gt/uc/selftest_guc_multi_lrc.c | 168 ++++++++++++++++++ .../drm/i915/selftests/i915_live_selftests.h | 1 + 3 files changed, 170 insertions(+) create mode 100644 drivers/gpu/drm/i915/gt/uc/selftest_guc_multi_lrc.c
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +1 -0
@@ -4737,4 +4737,5 @@ bool intel_guc_virtual_engine_has_heartbeat(const struct intel_engine_cs *ve)#if IS_ENABLED (CONFIG_DRM_I915_SELFTEST )#include "selftest_guc_flow_control.c" +#include "selftest_guc_multi_lrc.c" #endif
drivers/gpu/drm/i915/gt/uc/selftest_guc_multi_lrc.c new file +168 -0
168 lines not on this page · raw message
drivers/gpu/drm/i915/selftests/i915_live_selftests.h +1 -0
@@ -48,5 +48,6 @@ selftest(execlists, intel_execlists_live_selftests) selftest(ring_submission, intel_ring_submission_live_selftests) selftest(perf, i915_perf_live_selftests) selftest(guc_flow_control, intel_guc_flow_control) +selftest (guc_multi_lrc, intel_guc_multi_lrc) /* Here be dragons: keep last to run last! */selftest (late_gt_pm, intel_gt_pm_late_selftests)
Signature · 6 lines
2.28.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
RFC PATCH 29/42 drm/i915/guc: Implement BB boundary preemption for multi-lrc
In reply to Matthew BrostFor some users of multi-lrc, e.g. split frame, it isn't safe to preempt mid BB. To safely enable preemption at the BB boundary, a handshake between to parent and child is needed. This is implemented via custom emit_bb_start & emit_fini_breadcrumb functions and enabled via by default if a context is configured by set parallel extension.
--- drivers/gpu/drm/i915/gt/intel_context.c | 2 +- drivers/gpu/drm/i915/gt/intel_context_types.h | 3 + drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h | 2 +- .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 285 +++++++++++++++++- 4 files changed, 289 insertions(+), 3 deletions(-)
drivers/gpu/drm/i915/gt/intel_context.c +1 -1
@@ -639,7 +639,7 @@ void intel_context_bind_parent_child(struct intel_context *parent,GEM_BUG_ON (intel_context_is_child (child));GEM_BUG_ON (intel_context_is_parent (child)); - parent->guc_number_children++; + child->guc_child_index = parent->guc_number_children++;list_add_tail (&child->guc_child_link, &parent->guc_child_list); child->parent = parent;
drivers/gpu/drm/i915/gt/intel_context_types.h +3 -0
@@ -222,6 +222,9 @@ struct intel_context {/* Number of children if parent */ u8 guc_number_children; +/* Child index if child */ +u8 guc_child_index; +u8 parent_page;/* if set, page num reserved for parent context */ /*
drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h +1 -1
@@ -181,7 +181,7 @@ struct guc_process_desc {u32 wq_status;u32 engine_presence;u32 priority; -u32 reserved[30 ]; +u32 reserved[36 ]; } __packed;#define CONTEXT_REGISTRATION_FLAG_KMD BIT(0)
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +284 -1
336 lines not on this page · raw message
Signature · 1 line
2.28.0
RFC PATCH 30/42 i915/drm: Move secure execbuf check to execbuf2
In reply to Matthew BrostGoal is to remove all input sanity checks from the core submission.
--- .../gpu/drm/i915/gem/i915_gem_execbuffer.c | 35 +++++++++++-------- 1 file changed, 21 insertions(+), 14 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +21 -14
@@ -3184,19 +3184,8 @@ i915_gem_do_execbuffer(struct drm_device *dev, eb.num_fences =0 ; eb.batch_flags =0 ; -if (args->flags &I915_EXEC_SECURE ) { -if (GRAPHICS_VER (i915) >=11 ) -return -ENODEV ; - -/* Return -EPERM to trigger fallback code on old binaries. */ -if (!HAS_SECURE_BATCHES (i915)) -return -EPERM ; - -if (!drm_is_current_master (file) || !capable (CAP_SYS_ADMIN )) -return -EPERM ; - +if (args->flags &I915_EXEC_SECURE ) eb.batch_flags |=I915_DISPATCH_SECURE ; - }if (args->flags &I915_EXEC_IS_PINNED ) eb.batch_flags |=I915_DISPATCH_PINNED ;
@@ -3414,6 +3403,18 @@ i915_gem_execbuffer2_ioctl(struct drm_device *dev, void *data,return -EINVAL ; } +if (args->flags &I915_EXEC_SECURE ) { +if (GRAPHICS_VER (i915) >=11 ) +return -ENODEV ; + +/* Return -EPERM to trigger fallback code on old binaries. */ +if (!HAS_SECURE_BATCHES (i915)) +return -EPERM ; + +if (!drm_is_current_master (file) || !capable (CAP_SYS_ADMIN )) +return -EPERM ; + } + err =i915_gem_check_execbuffer (args);if (err)return err;
@@ -3430,8 +3431,8 @@ i915_gem_execbuffer2_ioctl(struct drm_device *dev, void *data, u64_to_user_ptr(args->buffers_ptr), sizeof(*exec2_list) * count)) {drm_dbg (&i915->drm,"copy %zd exec entries failed\n" , count); -kvfree (exec2_list); -return -EFAULT ; + err = -EFAULT ; + goto err_copy; } err =i915_gem_do_execbuffer (dev, file, args, exec2_list);
@@ -3476,6 +3477,12 @@ end:; args->flags &= ~__I915_EXEC_UNKNOWN_FLAGS;kvfree (exec2_list); + +return err; + +err_copy: +kvfree (exec2_list); +return err; }
Signature · 6 lines
2.28.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
RFC PATCH 31/42 drm/i915: Move input/exec fence handling to i915_gem_execbuffer2
In reply to Matthew BrostMove the job of creating an input/exec fences (from a file descriptor) out of i915_gem_do_execbuffer.
--- .../gpu/drm/i915/gem/i915_gem_execbuffer.c | 75 +++++++++++-------- 1 file changed, 43 insertions(+), 32 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +43 -32
@@ -3146,11 +3146,12 @@ static int i915_gem_do_execbuffer(struct drm_device *dev, struct drm_file *file, struct drm_i915_gem_execbuffer2 *args, - struct drm_i915_gem_exec_object2 *exec) + struct drm_i915_gem_exec_object2 *exec, + struct dma_fence *in_fence, + struct dma_fence *exec_fence) { struct drm_i915_private *i915 = to_i915(dev);struct i915_execbuffer eb; -struct dma_fence *in_fence =NULL ;struct sync_file *out_fence =NULL ;struct i915_vma *batch;int out_fence_fd =-1 ;
@@ -3197,25 +3198,10 @@ i915_gem_do_execbuffer(struct drm_device *dev,if (err) goto err_ext; -#define IN_FENCES (I915_EXEC_FENCE_IN | I915_EXEC_FENCE_SUBMIT) -if (args->flags &IN_FENCES ) { -if ((args->flags &IN_FENCES ) ==IN_FENCES ) -return -EINVAL ; - - in_fence =sync_file_get_fence (lower_32_bits (args->rsvd2)); -if (!in_fence) { - err = -EINVAL ; - goto err_ext; - } - } -#undef IN_FENCES -if (args->flags &I915_EXEC_FENCE_OUT ) { out_fence_fd =get_unused_fd_flags (O_CLOEXEC ); -if (out_fence_fd <0 ) { - err = out_fence_fd; - goto err_in_fence; - } +if (out_fence_fd <0 ) + goto err_ext; } err =eb_create (&eb);
@@ -3277,13 +3263,16 @@ i915_gem_do_execbuffer(struct drm_device *dev, goto err_ext; } +if (exec_fence) { + err =i915_request_await_execution (eb.request, + exec_fence); +if (err <0 ) + goto err_request; + } +if (in_fence) { -if (args->flags &I915_EXEC_FENCE_SUBMIT ) - err =i915_request_await_execution (eb.request, - in_fence); -else - err =i915_request_await_dma_fence (eb.request, - in_fence); + err =i915_request_await_dma_fence (eb.request, + in_fence);if (err <0 ) goto err_request; }
@@ -3363,8 +3352,6 @@ i915_gem_do_execbuffer(struct drm_device *dev, err_out_fence:if (out_fence_fd !=-1 )put_unused_fd (out_fence_fd); -err_in_fence: -dma_fence_put (in_fence); err_ext:put_fence_array (eb.fences, eb.num_fences);return err;
@@ -3395,6 +3382,8 @@ i915_gem_execbuffer2_ioctl(struct drm_device *dev, void *data,struct drm_i915_private *i915 =to_i915 (dev);struct drm_i915_gem_execbuffer2 *args = data;struct drm_i915_gem_exec_object2 *exec2_list; +struct dma_fence *in_fence =NULL ; +struct dma_fence *exec_fence =NULL ;const size_t count = args->buffer_count;int err;
@@ -3419,13 +3408,33 @@ i915_gem_execbuffer2_ioctl(struct drm_device *dev, void *data,if (err)return err; +if (args->flags &I915_EXEC_FENCE_IN ) { + in_fence =sync_file_get_fence (lower_32_bits (args->rsvd2)); +if (!in_fence) +return -EINVAL ; + } + +if (args->flags &I915_EXEC_FENCE_SUBMIT ) { +if (in_fence) { + err = -EINVAL ; + goto err_exec_fence; + } + + exec_fence =sync_file_get_fence (lower_32_bits (args->rsvd2)); +if (!exec_fence) { + err = -EINVAL ; + goto err_exec_fence; + } + } +/* Allocate extra slots for use by the command parser */ exec2_list =kvmalloc_array (count +2 ,eb_element_size (), __GFP_NOWARN |GFP_KERNEL );if (exec2_list ==NULL ) {drm_dbg (&i915->drm,"Failed to allocate exec list for %zd buffers\n" , count); -return -ENOMEM ; + err = -ENOMEM ; + goto err_alloc; } if (copy_from_user(exec2_list, u64_to_user_ptr(args->buffers_ptr),
@@ -3435,7 +3444,8 @@ i915_gem_execbuffer2_ioctl(struct drm_device *dev, void *data, goto err_copy; } - err =i915_gem_do_execbuffer (dev, file, args, exec2_list); + err =i915_gem_do_execbuffer (dev, file, args, exec2_list, in_fence, + exec_fence);/* * Now that we have begun execution of the batchbuffer, we ignore
@@ -3476,12 +3486,13 @@ end:; } args->flags &= ~__I915_EXEC_UNKNOWN_FLAGS; -kvfree (exec2_list); - -return err; err_copy:kvfree (exec2_list); +err_alloc: +dma_fence_put (exec_fence); +err_exec_fence: +dma_fence_put (in_fence);return err; }
Signature · 6 lines
2.28.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
RFC PATCH 32/42 drm/i915: Move output fence handling to i915_gem_execbuffer2
In reply to Matthew BrostMove the job of creating a new file descriptor and passing it back to userspace to i915_gem_execbuffer2.
--- .../gpu/drm/i915/gem/i915_gem_execbuffer.c | 45 ++++++++++--------- 1 file changed, 25 insertions(+), 20 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +25 -20
@@ -3148,13 +3148,13 @@ i915_gem_do_execbuffer(struct drm_device *dev,struct drm_i915_gem_execbuffer2 *args, struct drm_i915_gem_exec_object2 *exec, struct dma_fence *in_fence, - struct dma_fence *exec_fence) + struct dma_fence *exec_fence, + int out_fence_fd) { struct drm_i915_private *i915 = to_i915(dev);struct i915_execbuffer eb;struct sync_file *out_fence =NULL ;struct i915_vma *batch; -int out_fence_fd =-1 ;int err;BUILD_BUG_ON (__EXEC_INTERNAL_FLAGS & ~__I915_EXEC_ILLEGAL_FLAGS);
@@ -3198,15 +3198,9 @@ i915_gem_do_execbuffer(struct drm_device *dev,if (err) goto err_ext; -if (args->flags &I915_EXEC_FENCE_OUT ) { - out_fence_fd =get_unused_fd_flags (O_CLOEXEC ); -if (out_fence_fd <0 ) - goto err_ext; - } - err =eb_create (&eb);if (err) - goto err_out_fence; + goto err_ext;GEM_BUG_ON (!eb.lut_size);
@@ -3283,7 +3277,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, goto err_request; } -if (out_fence_fd !=-1 ) { +if (out_fence_fd >=0 ) { out_fence =sync_file_create (&eb.request->fence);if (!out_fence) { err = -ENOMEM ;
@@ -3313,14 +3307,10 @@ i915_gem_do_execbuffer(struct drm_device *dev,signal_fence_array (&eb);if (out_fence) { -if (err ==0 ) { +if (err ==0 )fd_install (out_fence_fd, out_fence->file); - args->rsvd2 &=GENMASK_ULL (31 ,0 );/* keep in-fence */ - args->rsvd2 |= (u64 )out_fence_fd <<32 ; - out_fence_fd =-1 ; - }else { +else fput (out_fence->file); - } }if (unlikely (eb.gem_context->syncobj)) {
@@ -3349,9 +3339,6 @@ i915_gem_do_execbuffer(struct drm_device *dev,i915_gem_context_put (eb.gem_context); err_destroy:eb_destroy (&eb); -err_out_fence: -if (out_fence_fd !=-1 ) -put_unused_fd (out_fence_fd); err_ext:put_fence_array (eb.fences, eb.num_fences);return err;
@@ -3384,6 +3371,7 @@ i915_gem_execbuffer2_ioctl(struct drm_device *dev, void *data,struct drm_i915_gem_exec_object2 *exec2_list;struct dma_fence *in_fence =NULL ;struct dma_fence *exec_fence =NULL ; +int out_fence_fd =-1 ;const size_t count = args->buffer_count;int err;
@@ -3427,6 +3415,14 @@ i915_gem_execbuffer2_ioctl(struct drm_device *dev, void *data, } } +if (args->flags &I915_EXEC_FENCE_OUT ) { + out_fence_fd =get_unused_fd_flags (O_CLOEXEC ); +if (out_fence_fd <0 ) { + err = out_fence_fd; + goto err_out_fence; + } + } +/* Allocate extra slots for use by the command parser */ exec2_list =kvmalloc_array (count +2 ,eb_element_size (), __GFP_NOWARN |GFP_KERNEL );
@@ -3445,7 +3441,7 @@ i915_gem_execbuffer2_ioctl(struct drm_device *dev, void *data, } err =i915_gem_do_execbuffer (dev, file, args, exec2_list, in_fence, - exec_fence); + exec_fence, out_fence_fd);/* * Now that we have begun execution of the batchbuffer, we ignore
@@ -3485,11 +3481,20 @@ i915_gem_execbuffer2_ioctl(struct drm_device *dev, void *data, end:; } +if (!err && out_fence_fd >=0 ) { + args->rsvd2 &=GENMASK_ULL (31 ,0 );/* keep in-fence */ + args->rsvd2 |= (u64 )out_fence_fd <<32 ; + out_fence_fd =-1 ; + } + args->flags &= ~__I915_EXEC_UNKNOWN_FLAGS; err_copy:kvfree (exec2_list); err_alloc: +if (out_fence_fd >=0 ) +put_unused_fd (out_fence_fd); +err_out_fence:dma_fence_put (exec_fence); err_exec_fence:dma_fence_put (in_fence);
Signature · 6 lines
2.28.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
RFC PATCH 33/42 drm/i915: Return output fence from i915_gem_do_execbuffer
In reply to Matthew BrostMove the job of creating a new sync fence and installing it onto a file descriptor to i915_gem_execbuffer2.
--- .../gpu/drm/i915/gem/i915_gem_execbuffer.c | 39 +++++++++---------- 1 file changed, 19 insertions(+), 20 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +19 -20
@@ -3149,11 +3149,10 @@ i915_gem_do_execbuffer(struct drm_device *dev,struct drm_i915_gem_exec_object2 *exec, struct dma_fence *in_fence, struct dma_fence *exec_fence, - int out_fence_fd) + struct dma_fence **out_fence) { struct drm_i915_private *i915 = to_i915(dev);struct i915_execbuffer eb; -struct sync_file *out_fence =NULL ;struct i915_vma *batch;int err;
@@ -3277,14 +3276,6 @@ i915_gem_do_execbuffer(struct drm_device *dev, goto err_request; } -if (out_fence_fd >=0 ) { - out_fence =sync_file_create (&eb.request->fence); -if (!out_fence) { - err = -ENOMEM ; - goto err_request; - } - } -/* * Whilst this request exists, batch_obj will be on the * active_list, and so will hold the active reference. Only when this
@@ -3306,12 +3297,8 @@ i915_gem_do_execbuffer(struct drm_device *dev,if (eb.fences)signal_fence_array (&eb); -if (out_fence) { -if (err ==0 ) -fd_install (out_fence_fd, out_fence->file); -else -fput (out_fence->file); - } +if (!err && out_fence) + *out_fence =dma_fence_get (&eb.request->fence);if (unlikely (eb.gem_context->syncobj)) { drm_syncobj_replace_fence(eb.gem_context->syncobj,
@@ -3369,6 +3356,8 @@ i915_gem_execbuffer2_ioctl(struct drm_device *dev, void *data,struct drm_i915_private *i915 =to_i915 (dev);struct drm_i915_gem_execbuffer2 *args = data;struct drm_i915_gem_exec_object2 *exec2_list; +struct dma_fence **out_fence_p =NULL ; +struct dma_fence *out_fence =NULL ;struct dma_fence *in_fence =NULL ;struct dma_fence *exec_fence =NULL ;int out_fence_fd =-1 ;
@@ -3421,6 +3410,7 @@ i915_gem_execbuffer2_ioctl(struct drm_device *dev, void *data, err = out_fence_fd; goto err_out_fence; } + out_fence_p = &out_fence; }/* Allocate extra slots for use by the command parser */
@@ -3441,7 +3431,7 @@ i915_gem_execbuffer2_ioctl(struct drm_device *dev, void *data, } err =i915_gem_do_execbuffer (dev, file, args, exec2_list, in_fence, - exec_fence, out_fence_fd); + exec_fence, out_fence_p);/* * Now that we have begun execution of the batchbuffer, we ignore
@@ -3482,9 +3472,18 @@ end:; }if (!err && out_fence_fd >=0 ) { - args->rsvd2 &=GENMASK_ULL (31 ,0 );/* keep in-fence */ - args->rsvd2 |= (u64 )out_fence_fd <<32 ; - out_fence_fd =-1 ; +struct sync_file *sync_fence; + + sync_fence =sync_file_create (out_fence); +if (sync_fence) { +fd_install (out_fence_fd, sync_fence->file); + args->rsvd2 &=GENMASK_ULL (31 ,0 );/* keep in-fence */ + args->rsvd2 |= (u64 )out_fence_fd <<32 ; + out_fence_fd =-1 ; + } +dma_fence_put (out_fence); + }else if (out_fence) { +dma_fence_put (out_fence); } args->flags &= ~__I915_EXEC_UNKNOWN_FLAGS;
Signature · 6 lines
2.28.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
RFC PATCH 34/42 drm/i915: Store batch index in struct i915_execbuffer
In reply to Matthew BrostThis will help with upcoming extensions where more than 1 batch can be submitted in a single execbuf IOCTL.
--- .../gpu/drm/i915/gem/i915_gem_execbuffer.c | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +9 -10
@@ -252,6 +252,9 @@ struct i915_execbuffer {struct eb_vma *batch;/** identity of the batch obj/vma */ struct i915_vma *trampoline;/** trampoline used for chaining */ +/* batch_index in vma list */ +unsigned int batch_index; +/** actual size of execobj[] as we may extend it for the cmdparser */ unsigned int buffer_count;
@@ -361,6 +364,11 @@ static int eb_create(struct i915_execbuffer *eb) eb->lut_size = -eb->buffer_count; } +if (eb->args->flags &I915_EXEC_BATCH_FIRST ) + eb->batch_index =0 ; +else + eb->batch_index = eb->args->buffer_count -1 ; +return 0 ; }
@@ -735,14 +743,6 @@ static int eb_reserve(struct i915_execbuffer *eb) }while (1 ); } -static unsigned int eb_batch_index (const struct i915_execbuffer *eb) -{ -if (eb->args->flags &I915_EXEC_BATCH_FIRST ) -return 0 ; -else -return eb->buffer_count -1 ; -} -static int eb_select_context (struct i915_execbuffer *eb) {struct i915_gem_context *ctx;
@@ -852,7 +852,6 @@ static struct i915_vma *eb_lookup_vma(struct i915_execbuffer *eb, u32 handle)static int eb_lookup_vmas (struct i915_execbuffer *eb) {struct drm_i915_private *i915 = eb->i915; -unsigned int batch =eb_batch_index (eb);unsigned int i;int err =0 ;
@@ -873,7 +872,7 @@ static int eb_lookup_vmas(struct i915_execbuffer *eb) goto err; } -eb_add_vma (eb, i, batch, vma); +eb_add_vma (eb, i, eb->batch_index, vma);if (i915_gem_object_is_userptr (vma->obj)) { err =i915_gem_object_userptr_submit_init (vma->obj);
Signature · 6 lines
2.28.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
RFC PATCH 35/42 drm/i915: Allow callers of i915_gem_do_execbuffer to override the batch index
In reply to Matthew BrostAllow specifying the batch directly over what is inferred from passed in execbuf flags.
--- drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +5 -1
@@ -3146,6 +3146,7 @@ i915_gem_do_execbuffer(struct drm_device *dev,struct drm_file *file, struct drm_i915_gem_execbuffer2 *args, struct drm_i915_gem_exec_object2 *exec, + int batch_index, struct dma_fence *in_fence, struct dma_fence *exec_fence, struct dma_fence **out_fence)
@@ -3202,6 +3203,9 @@ i915_gem_do_execbuffer(struct drm_device *dev,GEM_BUG_ON (!eb.lut_size); +if (batch_index >=0 ) + eb.batch_index = batch_index; + err =eb_select_context (&eb);if (unlikely (err)) goto err_destroy;
@@ -3429,7 +3433,7 @@ i915_gem_execbuffer2_ioctl(struct drm_device *dev, void *data, goto err_copy; } - err =i915_gem_do_execbuffer (dev, file, args, exec2_list, in_fence, + err =i915_gem_do_execbuffer (dev, file, args, exec2_list,-1 , in_fence, exec_fence, out_fence_p);/*
Signature · 6 lines
2.28.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
RFC PATCH 36/42 drm/i915: Teach execbuf there can be more than one batch in the objects list
In reply to Matthew BrostIn case of multiple batches all batches will be at the beginning on the exec objects array or at the end based on the existing execbuffer2 flag.
Batches not executed in the current execbuf call will not be processed for relocations or but will be pinned in same manner as the current batch.
This will enable multiple do_execbuf calls with a single exec object array in a later patch.
--- .../gpu/drm/i915/gem/i915_gem_execbuffer.c | 31 +++++++++++++------ 1 file changed, 22 insertions(+), 9 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +22 -9
@@ -255,6 +255,9 @@ struct i915_execbuffer {/* batch_index in vma list */ unsigned int batch_index; +/* number of batches in execbuf IOCTL */ +unsigned int num_batches; +/** actual size of execobj[] as we may extend it for the cmdparser */ unsigned int buffer_count;
@@ -511,6 +514,14 @@ static bool platform_has_relocs_enabled(const struct i915_execbuffer *eb)return false; } +static inline bool +is_batch_buffer (struct i915_execbuffer *eb,unsigned buffer_idx) +{ +return eb->args->flags &I915_EXEC_BATCH_FIRST ? + buffer_idx <= eb->num_batches : + buffer_idx >= eb->args->buffer_count - eb->num_batches; +} + static int eb_validate_vma(struct i915_execbuffer *eb, struct drm_i915_gem_exec_object2 *entry,
@@ -562,11 +573,10 @@ eb_validate_vma(struct i915_execbuffer *eb,static void eb_add_vma (struct i915_execbuffer *eb, -unsigned int i,unsigned batch_idx, -struct i915_vma *vma) +unsigned int buffer_idx,struct i915_vma *vma) { -struct drm_i915_gem_exec_object2 *entry = &eb->exec[i]; -struct eb_vma *ev = &eb->vma[i]; +struct drm_i915_gem_exec_object2 *entry = &eb->exec[buffer_idx]; +struct eb_vma *ev = &eb->vma[buffer_idx]; ev->vma = vma; ev->exec = entry;
@@ -591,14 +601,15 @@ eb_add_vma(struct i915_execbuffer *eb,* Note that actual hangs have only been observed on gen7, but for * paranoia do it everywhere. */ -if (i == batch_idx) { +if (is_batch_buffer (eb, buffer_idx)) {if (entry->relocation_count && !(ev->flags &EXEC_OBJECT_PINNED )) ev->flags |= __EXEC_OBJECT_NEEDS_BIAS;if (eb->reloc_cache.has_fence) ev->flags |=EXEC_OBJECT_NEEDS_FENCE ; - eb->batch = ev; +if (buffer_idx == eb->batch_index) + eb->batch = ev; } }
@@ -872,7 +883,7 @@ static int eb_lookup_vmas(struct i915_execbuffer *eb) goto err; } -eb_add_vma (eb, i, eb->batch_index, vma); +eb_add_vma (eb, i, vma);if (i915_gem_object_is_userptr (vma->obj)) { err =i915_gem_object_userptr_submit_init (vma->obj);
@@ -3147,6 +3158,7 @@ i915_gem_do_execbuffer(struct drm_device *dev,struct drm_i915_gem_execbuffer2 *args, struct drm_i915_gem_exec_object2 *exec, int batch_index, + unsigned int num_batches, struct dma_fence *in_fence, struct dma_fence *exec_fence, struct dma_fence **out_fence)
@@ -3203,6 +3215,7 @@ i915_gem_do_execbuffer(struct drm_device *dev,GEM_BUG_ON (!eb.lut_size); + eb.num_batches = num_batches;if (batch_index >=0 ) eb.batch_index = batch_index;
@@ -3433,8 +3446,8 @@ i915_gem_execbuffer2_ioctl(struct drm_device *dev, void *data, goto err_copy; } - err =i915_gem_do_execbuffer (dev, file, args, exec2_list,-1 , in_fence, - exec_fence, out_fence_p); + err =i915_gem_do_execbuffer (dev, file, args, exec2_list,-1 ,1 , + in_fence, exec_fence, out_fence_p);/* * Now that we have begun execution of the batchbuffer, we ignore
Signature · 1 line
2.28.0
RFC PATCH 37/42 drm/i915: Only track object dependencies on first request
In reply to Matthew BrostOnly track object dependencies on the first request generated from the execbuf, this help with the upcoming multi-bb execbuf extension.
--- drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +6 -5
@@ -2239,7 +2239,7 @@ static int eb_relocate_parse(struct i915_execbuffer *eb)return err; } -static int eb_move_to_gpu (struct i915_execbuffer *eb) +static int eb_move_to_gpu (struct i915_execbuffer *eb,bool first) {const unsigned int count = eb->buffer_count;unsigned int i = count;
@@ -2281,7 +2281,7 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb) flags &= ~EXEC_OBJECT_ASYNC ; } -if (err ==0 && !(flags &EXEC_OBJECT_ASYNC )) { +if (err ==0 && first && !(flags &EXEC_OBJECT_ASYNC )) { err =i915_request_await_object (eb->request, obj, flags &EXEC_OBJECT_WRITE ); }
@@ -2525,14 +2525,15 @@ static int eb_parse(struct i915_execbuffer *eb)return err; } -static int eb_submit (struct i915_execbuffer *eb,struct i915_vma *batch) +static int eb_submit (struct i915_execbuffer *eb,struct i915_vma *batch, +bool first) {int err;if (intel_context_nopreempt (eb->context))__set_bit (I915_FENCE_FLAG_NOPREEMPT , &eb->request->fence.flags); - err =eb_move_to_gpu (eb); + err =eb_move_to_gpu (eb, first);if (err)return err;
@@ -3304,7 +3305,7 @@ i915_gem_do_execbuffer(struct drm_device *dev,intel_gt_buffer_pool_mark_active (eb.batch_pool, eb.request);trace_i915_request_queue (eb.request, eb.batch_flags); - err =eb_submit (&eb, batch); + err =eb_submit (&eb, batch, true); err_request:i915_request_get (eb.request);
Signature · 6 lines
2.28.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
RFC PATCH 38/42 drm/i915: Force parallel contexts to use copy engine for reloc
In reply to Matthew BrostSubmitting to a subset of hardware contexts is not allowed, so use the copy engine for GPU relocations when using a parallel context.
--- drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +4 -2
@@ -1386,7 +1386,8 @@ static int __reloc_gpu_alloc(struct i915_execbuffer *eb,if (err) goto err_unmap; -if (engine == eb->context->engine) { +if (engine == eb->context->engine && + !intel_context_is_parallel (eb->context)) { rq =i915_request_create (eb->context); }else {struct intel_context *ce = eb->reloc_context;
@@ -1483,7 +1484,8 @@ static u32 *reloc_gpu(struct i915_execbuffer *eb,if (eb_use_cmdparser (eb))return ERR_PTR (-EWOULDBLOCK ); -if (!reloc_can_use_engine (engine)) { +if (!reloc_can_use_engine (engine) || +intel_context_is_parallel (eb->context)) { engine = engine->gt->engine_class[COPY_ENGINE_CLASS ][0 ];if (!engine)return ERR_PTR (-ENODEV );
Signature · 1 line
2.28.0
RFC PATCH 39/42 drm/i915: Multi-batch execbuffer2
In reply to Matthew BrostFor contexts with width set to two or more, we add a mode to execbuf2 which implies there are N batch buffers in the buffer list, each of which will be sent to one of the engines from the engine map array (I915_CONTEXT_PARAM_ENGINES, I915_CONTEXT_ENGINES_EXT_PARALLEL_SUBMIT).
Those N batches can either be first N, or last N objects in the list as controlled by the existing execbuffer2 flag.
The N batches will be submitted to consecutive engines from the previously configured allowed engine array starting at index 0.
Input and output fences are fully supported, with the latter getting signalled when all batch buffers have completed.
Last, it isn't safe for subsequent batches to touch any objects written to by a multi-BB submission until all the batches in that submission complete. As such all batches in a multi-BB submission must be combined into a single composite fence and put into the dma reseveration excl fence slot.
--- .../gpu/drm/i915/gem/i915_gem_execbuffer.c | 263 +++++++++++++++--- drivers/gpu/drm/i915/gt/intel_context.c | 5 + drivers/gpu/drm/i915/gt/intel_context_types.h | 9 + drivers/gpu/drm/i915/i915_vma.c | 13 +- drivers/gpu/drm/i915/i915_vma.h | 16 +- 5 files changed, 267 insertions(+), 39 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +232 -31
@@ -252,6 +252,9 @@ struct i915_execbuffer {struct eb_vma *batch;/** identity of the batch obj/vma */ struct i915_vma *trampoline;/** trampoline used for chaining */ +/** used for excl fence in dma_resv objects when > 1 BB submitted */ +struct dma_fence *composite_fence; +/* batch_index in vma list */ unsigned int batch_index;
@@ -367,11 +370,6 @@ static int eb_create(struct i915_execbuffer *eb) eb->lut_size = -eb->buffer_count; } -if (eb->args->flags &I915_EXEC_BATCH_FIRST ) - eb->batch_index =0 ; -else - eb->batch_index = eb->args->buffer_count -1 ; -return 0 ; }
@@ -2241,7 +2239,7 @@ static int eb_relocate_parse(struct i915_execbuffer *eb)return err; } -static int eb_move_to_gpu (struct i915_execbuffer *eb,bool first) +static int eb_move_to_gpu (struct i915_execbuffer *eb,bool first,bool last) {const unsigned int count = eb->buffer_count;unsigned int i = count;
@@ -2289,8 +2287,16 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb, bool first) }if (err ==0 ) - err =i915_vma_move_to_active (vma, eb->request, - flags | __EXEC_OBJECT_NO_RESERVE); + err =_i915_vma_move_to_active (vma, eb->request, + flags | __EXEC_OBJECT_NO_RESERVE, + !last ? +NULL : + eb->composite_fence ? + eb->composite_fence : + &eb->request->fence, + eb->composite_fence ? + eb->composite_fence : + &eb->request->fence); }#ifdef CONFIG_MMU_NOTIFIER
@@ -2528,14 +2534,14 @@ static int eb_parse(struct i915_execbuffer *eb) }static int eb_submit (struct i915_execbuffer *eb,struct i915_vma *batch, -bool first) +bool first,bool last) {int err;if (intel_context_nopreempt (eb->context))__set_bit (I915_FENCE_FLAG_NOPREEMPT , &eb->request->fence.flags); - err =eb_move_to_gpu (eb, first); + err =eb_move_to_gpu (eb, first, last);if (err)return err;
@@ -2748,7 +2754,7 @@ eb_select_legacy_ring(struct i915_execbuffer *eb) }static int -eb_select_engine (struct i915_execbuffer *eb) +eb_select_engine (struct i915_execbuffer *eb,unsigned int batch_number) {struct intel_context *ce;unsigned int idx;
@@ -2763,6 +2769,18 @@ eb_select_engine(struct i915_execbuffer *eb)if (IS_ERR (ce))return PTR_ERR (ce); +if (batch_number >0 ) { +struct intel_context *parent = ce; + +GEM_BUG_ON (!intel_context_is_parent (parent)); + +for_each_child (parent, ce) +if (!--batch_number) +break ; +intel_context_put (parent); +intel_context_get (ce); + } +intel_gt_pm_get (ce->engine->gt);if (!test_bit (CONTEXT_ALLOC_BIT , &ce->flags)) {
@@ -3155,13 +3173,49 @@ parse_execbuf2_extensions(struct drm_i915_gem_execbuffer2 *args, eb); } +static int setup_composite_fence (struct i915_execbuffer *eb, +struct dma_fence **out_fence, +unsigned int num_batches) +{ +struct dma_fence_array *fence_array; +struct dma_fence **fences =kmalloc (num_batches *sizeof (*fences), +GFP_KERNEL ); +struct intel_context *parent =intel_context_to_parent (eb->context); +int i; + +if (!fences) +return -ENOMEM ; + +for (i =0 ; i < num_batches; ++i) + fences[i] = out_fence[i]; + + fence_array =dma_fence_array_create (num_batches, + fences, + parent->fence_context, + ++parent->seqno, + false); +if (!fence_array) { +kfree (fences); +return -ENOMEM ; + } + +/* Move ownership to the dma_fence_array created above */ +for (i =0 ; i < num_batches; ++i) +dma_fence_get (fences[i]); + + eb->composite_fence = &fence_array->base; + +return 0 ; +} +static int i915_gem_do_execbuffer (struct drm_device *dev,struct drm_file *file,struct drm_i915_gem_execbuffer2 *args,struct drm_i915_gem_exec_object2 *exec, -int batch_index, +unsigned int batch_index,unsigned int num_batches, +unsigned int batch_number,struct dma_fence *in_fence,struct dma_fence *exec_fence,struct dma_fence **out_fence)
@@ -3170,6 +3224,8 @@ i915_gem_do_execbuffer(struct drm_device *dev,struct i915_execbuffer eb;struct i915_vma *batch;int err; +bool first = batch_number ==0 ; +bool last = batch_number +1 == num_batches;BUILD_BUG_ON (__EXEC_INTERNAL_FLAGS & ~__I915_EXEC_ILLEGAL_FLAGS); BUILD_BUG_ON(__EXEC_OBJECT_INTERNAL_FLAGS &
@@ -3194,6 +3250,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, eb.batch_start_offset = args->batch_start_offset; eb.batch_len = args->batch_len; eb.trampoline =NULL ; + eb.composite_fence =NULL ; eb.fences =NULL ; eb.num_fences =0 ;
@@ -3219,14 +3276,13 @@ i915_gem_do_execbuffer(struct drm_device *dev,GEM_BUG_ON (!eb.lut_size); eb.num_batches = num_batches; -if (batch_index >=0 ) - eb.batch_index = batch_index; + eb.batch_index = batch_index; err =eb_select_context (&eb);if (unlikely (err)) goto err_destroy; - err =eb_select_engine (&eb); + err =eb_select_engine (&eb, batch_number);if (unlikely (err)) goto err_context;
@@ -3282,6 +3338,10 @@ i915_gem_do_execbuffer(struct drm_device *dev, goto err_request; } +if (last) +set_bit (I915_FENCE_FLAG_SUBMIT_PARALLEL , + &eb.request->fence.flags); +if (in_fence) { err =i915_request_await_dma_fence (eb.request, in_fence);
@@ -3295,6 +3355,23 @@ i915_gem_do_execbuffer(struct drm_device *dev, goto err_request; } +if (out_fence) { +/* Move ownership to caller (i915_gem_execbuffer2_ioctl) */ + out_fence[batch_number] =dma_fence_get (&eb.request->fence); + +/* +* Need to create a composite fence (dma_fence_array, +* eb.composite_fence) for the excl fence of the dma_resv +* objects as each BB can write to the object. Since we create +*/ +if (num_batches >1 && last) { + err =setup_composite_fence (&eb, out_fence, + num_batches); +if (err <0 ) + goto err_request; + } + } +/* * Whilst this request exists, batch_obj will be on the * active_list, and so will hold the active reference. Only when this
@@ -3307,7 +3384,7 @@ i915_gem_do_execbuffer(struct drm_device *dev,intel_gt_buffer_pool_mark_active (eb.batch_pool, eb.request);trace_i915_request_queue (eb.request, eb.batch_flags); - err =eb_submit (&eb, batch, true); + err =eb_submit (&eb, batch, first, last); err_request:i915_request_get (eb.request);
@@ -3316,8 +3393,14 @@ i915_gem_do_execbuffer(struct drm_device *dev,if (eb.fences)signal_fence_array (&eb); -if (!err && out_fence) - *out_fence =dma_fence_get (&eb.request->fence); +/* +* Ownership of the composite fence (dma_fence_array, +* eb.composite_fence) has been moved to the dma_resv objects these BB +* write to in i915_vma_move_to_active. It is ok to release the creation +* reference of this fence now. +*/ +if (eb.composite_fence) +dma_fence_put (eb.composite_fence);if (unlikely (eb.gem_context->syncobj)) { drm_syncobj_replace_fence(eb.gem_context->syncobj,
@@ -3368,6 +3451,17 @@ static bool check_buffer_count(size_t count)return !(count <1 || count >INT_MAX || count >SIZE_MAX / sz -1 ); } +/* Release fences from the dma_fence_get in i915_gem_do_execbuffer. */ +static inline void put_out_fences (struct dma_fence **out_fences, +unsigned int num_batches) +{ +int i; + +for (i =0 ; i < num_batches; ++i) +if (out_fences[i]) +dma_fence_put (out_fences[i]); +} +int i915_gem_execbuffer2_ioctl (struct drm_device *dev,void *data,struct drm_file *file)
@@ -3375,13 +3469,16 @@ i915_gem_execbuffer2_ioctl(struct drm_device *dev, void *data,struct drm_i915_private *i915 =to_i915 (dev);struct drm_i915_gem_execbuffer2 *args = data;struct drm_i915_gem_exec_object2 *exec2_list; -struct dma_fence **out_fence_p =NULL ; -struct dma_fence *out_fence =NULL ; +struct dma_fence **out_fences =NULL ;struct dma_fence *in_fence =NULL ;struct dma_fence *exec_fence =NULL ;int out_fence_fd =-1 ;const size_t count = args->buffer_count;int err; +struct i915_gem_context *ctx; +struct intel_context *parent =NULL ; +unsigned int num_batches =1 , i; +bool is_parallel = false;if (!check_buffer_count (count)) {drm_dbg (&i915->drm,"execbuf2 with %zd buffers\n" , count);
@@ -3404,10 +3501,39 @@ i915_gem_execbuffer2_ioctl(struct drm_device *dev, void *data,if (err)return err; + ctx =i915_gem_context_lookup (file->driver_priv, args->rsvd1); +if (unlikely (IS_ERR (ctx))) +return PTR_ERR (ctx); + +if (i915_gem_context_user_engines (ctx)) { + parent =i915_gem_context_get_engine (ctx, args->flags & +I915_EXEC_RING_MASK ); +if (IS_ERR (parent)) { + err =PTR_ERR (parent); + goto err_context; + } + +if (intel_context_is_parent (parent)) { +if (args->batch_len) { + err = -EINVAL ; + goto err_context; + } + + num_batches = parent->guc_number_children +1 ; +if (num_batches > count) { +i915_gem_context_put (ctx); + goto err_parent; + } + is_parallel = true; + } + } +if (args->flags &I915_EXEC_FENCE_IN ) { in_fence =sync_file_get_fence (lower_32_bits (args->rsvd2)); -if (!in_fence) -return -EINVAL ; +if (!in_fence) { + err = -EINVAL ; + goto err_parent; + } }if (args->flags &I915_EXEC_FENCE_SUBMIT ) {
@@ -3423,15 +3549,28 @@ i915_gem_execbuffer2_ioctl(struct drm_device *dev, void *data, } } -if (args->flags &I915_EXEC_FENCE_OUT ) { - out_fence_fd =get_unused_fd_flags (O_CLOEXEC ); -if (out_fence_fd <0 ) { - err = out_fence_fd; +/* +* We always allocate out fences when doing multi-BB submission as +* this is required to create an excl fence for any dma buf objects +* these BBs touch. +*/ +if (args->flags &I915_EXEC_FENCE_OUT || is_parallel) { + out_fences =kcalloc (num_batches,sizeof (*out_fences), +GFP_KERNEL ); +if (!out_fences) { + err = -ENOMEM ; goto err_out_fence; } - out_fence_p = &out_fence; +if (args->flags &I915_EXEC_FENCE_OUT ) { + out_fence_fd =get_unused_fd_flags (O_CLOEXEC ); +if (out_fence_fd <0 ) { + err = out_fence_fd; + goto err_out_fence; + } + } } +/* Allocate extra slots for use by the command parser */ exec2_list =kvmalloc_array (count +2 ,eb_element_size (), __GFP_NOWARN |GFP_KERNEL );
@@ -3449,8 +3588,35 @@ i915_gem_execbuffer2_ioctl(struct drm_device *dev, void *data, goto err_copy; } - err =i915_gem_do_execbuffer (dev, file, args, exec2_list,-1 ,1 , - in_fence, exec_fence, out_fence_p); +/* +* Downstream submission code expects all parallel submissions to occur +* in intel_context sequence, thus only 1 submission can happen at a +* time. +*/ +if (is_parallel) +mutex_lock (&parent->parallel_submit); + + err =i915_gem_do_execbuffer (dev, file, args, exec2_list, + args->flags &I915_EXEC_BATCH_FIRST ? +0 : count - num_batches, + num_batches, +0 , + in_fence, + exec_fence, + out_fences); + +for (i =1 ; err ==0 && i < num_batches; i++) + err =i915_gem_do_execbuffer (dev, file, args, exec2_list, + args->flags &I915_EXEC_BATCH_FIRST ? + i : count - num_batches + i, + num_batches, + i, +NULL , +NULL , + out_fences); + +if (is_parallel) +mutex_unlock (&parent->parallel_submit);/* * Now that we have begun execution of the batchbuffer, we ignore
@@ -3491,8 +3657,31 @@ end:; }if (!err && out_fence_fd >=0 ) { +struct dma_fence *out_fence =NULL ;struct sync_file *sync_fence; +if (is_parallel) { +struct dma_fence_array *fence_array; + +/* +* The dma_fence_array now owns out_fences (from +* dma_fence_get in i915_gem_do_execbuffer) assuming +* successful creation of dma_fence_array. +*/ + fence_array =dma_fence_array_create (num_batches, + out_fences, + parent->fence_context, + ++parent->seqno, + false); +if (!fence_array) + goto put_out_fences; + + out_fence = &fence_array->base; + out_fences =NULL ; + }else { + out_fence = out_fences[0 ]; + } + sync_fence =sync_file_create (out_fence);if (sync_fence) {fd_install (out_fence_fd, sync_fence->file);
@@ -3500,9 +3689,15 @@ end:; args->rsvd2 |= (u64 )out_fence_fd <<32 ; out_fence_fd =-1 ; } + +/* +* The sync_file now owns out_fence, drop the creation +* reference. +*/ dma_fence_put (out_fence); - }else if (out_fence) { -dma_fence_put (out_fence); + }else if (out_fences) { +put_out_fences: +put_out_fences (out_fences, num_batches); } args->flags &= ~__I915_EXEC_UNKNOWN_FLAGS;
@@ -3513,9 +3708,15 @@ end:;if (out_fence_fd >=0 )put_unused_fd (out_fence_fd); err_out_fence: +kfree (out_fences);dma_fence_put (exec_fence); err_exec_fence:dma_fence_put (in_fence); +err_parent: +if (parent) +intel_context_put (parent); +err_context: +i915_gem_context_put (ctx);return err; }
drivers/gpu/drm/i915/gt/intel_context.c +5 -0
@@ -475,6 +475,9 @@ intel_context_init(struct intel_context *ce, struct intel_engine_cs *engine) ce->guc_id =GUC_INVALID_LRC_ID ;INIT_LIST_HEAD (&ce->guc_id_link); +mutex_init (&ce->parallel_submit); + ce->fence_context =dma_fence_context_alloc (1 ); +i915_sw_fence_init (&ce->guc_blocked, sw_fence_dummy_notify);i915_sw_fence_commit (&ce->guc_blocked);
@@ -497,6 +500,8 @@ void intel_context_fini(struct intel_context *ce)for_each_child_safe (ce, child, next)intel_context_put (child); +mutex_destroy (&ce->parallel_submit); +mutex_destroy (&ce->pin_mutex);i915_active_fini (&ce->active); }
drivers/gpu/drm/i915/gt/intel_context_types.h +9 -0
@@ -235,6 +235,15 @@ struct intel_context {/* Last request submitted on a parent */ struct i915_request *last_rq; + +/* Parallel submission mutex */ +struct mutex parallel_submit; + +/* Fence context for parallel submission */ +u64 fence_context; + +/* Seqno for parallel submission */ + u32 seqno; };#endif /* __INTEL_CONTEXT_TYPES__ */
drivers/gpu/drm/i915/i915_vma.c +8 -5
@@ -1238,9 +1238,11 @@ int __i915_vma_move_to_active(struct i915_vma *vma, struct i915_request *rq)return i915_active_add_request (&vma->active, rq); } -int i915_vma_move_to_active (struct i915_vma *vma, -struct i915_request *rq, -unsigned int flags) +int _i915_vma_move_to_active (struct i915_vma *vma, +struct i915_request *rq, +unsigned int flags, +struct dma_fence *shared_fence, +struct dma_fence *excl_fence) {struct drm_i915_gem_object *obj = vma->obj;int err;
@@ -1261,7 +1263,7 @@ int i915_vma_move_to_active(struct i915_vma *vma,intel_frontbuffer_put (front); } -dma_resv_add_excl_fence (vma->resv, &rq->fence); +dma_resv_add_excl_fence (vma->resv, excl_fence); obj->write_domain =I915_GEM_DOMAIN_RENDER ; obj->read_domains =0 ; }else {
@@ -1271,7 +1273,8 @@ int i915_vma_move_to_active(struct i915_vma *vma,return err; } -dma_resv_add_shared_fence (vma->resv, &rq->fence); +if (shared_fence) +dma_resv_add_shared_fence (vma->resv, shared_fence); obj->write_domain =0 ; }
drivers/gpu/drm/i915/i915_vma.h +13 -3
@@ -57,9 +57,19 @@ static inline bool i915_vma_is_active(const struct i915_vma *vma)int __must_check __i915_vma_move_to_active (struct i915_vma *vma,struct i915_request *rq); -int __must_check i915_vma_move_to_active (struct i915_vma *vma, -struct i915_request *rq, -unsigned int flags); + +int __must_check _i915_vma_move_to_active (struct i915_vma *vma, +struct i915_request *rq, +unsigned int flags, +struct dma_fence *shared_fence, +struct dma_fence *excl_fence); +static inline int __must_check +i915_vma_move_to_active (struct i915_vma *vma, +struct i915_request *rq, +unsigned int flags) +{ +return _i915_vma_move_to_active (vma, rq, flags, &rq->fence, &rq->fence); +}#define __i915_vma_flags (v) ((unsigned long *)&(v)->flags.counter)
Signature · 1 line
2.28.0
RFC PATCH 40/42 drm/i915: Eliminate unnecessary VMA calls for multi-BB submission
In reply to Matthew BrostCertain VMA functions in the execbuf IOCTL only need to be called on first or last BB of a multi-BB submission. eb_relocate() on the first and eb_release_vmas() on the last. Doing so will save CPU / GPU cycles.
--- .../gpu/drm/i915/gem/i915_gem_execbuffer.c | 123 +++++++++++------- .../i915/gem/selftests/i915_gem_execbuffer.c | 14 +- 2 files changed, 81 insertions(+), 56 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +73 -50
@@ -270,7 +270,7 @@ struct i915_execbuffer {/** list of vma that have execobj.relocation_count */ struct list_head relocs; -struct i915_gem_ww_ctx ww; +struct i915_gem_ww_ctx *ww;/** * Track the most recently used object for relocations, as we
@@ -448,7 +448,7 @@ eb_pin_vma(struct i915_execbuffer *eb, pin_flags |=PIN_GLOBAL ;/* Attempt to reuse the current location if available */ - err =i915_vma_pin_ww (vma, &eb->ww,0 ,0 , pin_flags); + err =i915_vma_pin_ww (vma, eb->ww,0 ,0 , pin_flags);if (err == -EDEADLK )return err;
@@ -457,11 +457,11 @@ eb_pin_vma(struct i915_execbuffer *eb,return err;/* Failing that pick any _free_ space if suitable */ - err =i915_vma_pin_ww (vma, &eb->ww, - entry->pad_to_size, - entry->alignment, -eb_pin_flags (entry, ev->flags) | -PIN_USER |PIN_NOEVICT ); + err =i915_vma_pin_ww (vma, eb->ww, + entry->pad_to_size, + entry->alignment, +eb_pin_flags (entry, ev->flags) | +PIN_USER |PIN_NOEVICT );if (unlikely (err))return err; }
@@ -643,7 +643,7 @@ static int eb_reserve_vma(struct i915_execbuffer *eb,return err; } - err =i915_vma_pin_ww (vma, &eb->ww, + err =i915_vma_pin_ww (vma, eb->ww, entry->pad_to_size, entry->alignment,eb_pin_flags (entry, ev->flags) | pin_flags);if (err)
@@ -940,7 +940,7 @@ static int eb_lock_vmas(struct i915_execbuffer *eb)struct eb_vma *ev = &eb->vma[i];struct i915_vma *vma = ev->vma; - err =i915_gem_object_lock (vma->obj, &eb->ww); + err =i915_gem_object_lock (vma->obj, eb->ww);if (err)return err; }
@@ -1020,12 +1020,13 @@ eb_get_vma(const struct i915_execbuffer *eb, unsigned long handle) } } -static void eb_release_vmas (struct i915_execbuffer *eb,bool final) +static void eb_release_vmas (struct i915_execbuffer *eb,bool final, +bool unreserve) {const unsigned int count = eb->buffer_count;unsigned int i; -for (i =0 ; i < count; i++) { +for (i =0 ; unreserve && i < count; i++) {struct eb_vma *ev = &eb->vma[i];struct i915_vma *vma = ev->vma;
@@ -1237,7 +1238,7 @@ static void *reloc_iomap(struct drm_i915_gem_object *obj,if (err)return ERR_PTR (err); - vma =i915_gem_object_ggtt_pin_ww (obj, &eb->ww,NULL ,0 ,0 , + vma =i915_gem_object_ggtt_pin_ww (obj, eb->ww,NULL ,0 ,0 ,PIN_MAPPABLE |PIN_NONBLOCK /* NOWARN */ |PIN_NOEVICT );
@@ -1361,7 +1362,7 @@ static int __reloc_gpu_alloc(struct i915_execbuffer *eb, } eb->reloc_pool =NULL ; - err =i915_gem_object_lock (pool->obj, &eb->ww); + err =i915_gem_object_lock (pool->obj, eb->ww);if (err) goto err_pool;
@@ -1380,7 +1381,7 @@ static int __reloc_gpu_alloc(struct i915_execbuffer *eb, goto err_unmap; } - err =i915_vma_pin_ww (batch, &eb->ww,0 ,0 ,PIN_USER |PIN_NONBLOCK ); + err =i915_vma_pin_ww (batch, eb->ww,0 ,0 ,PIN_USER |PIN_NONBLOCK );if (err) goto err_unmap;
@@ -1402,7 +1403,7 @@ static int __reloc_gpu_alloc(struct i915_execbuffer *eb, eb->reloc_context = ce; } - err =intel_context_pin_ww (ce, &eb->ww); + err =intel_context_pin_ww (ce, eb->ww);if (err) goto err_unpin;
@@ -2017,8 +2018,8 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb, }/* We may process another execbuffer during the unlock... */ -eb_release_vmas (eb, false); -i915_gem_ww_ctx_fini (&eb->ww); +eb_release_vmas (eb, false, true); +i915_gem_ww_ctx_fini (eb->ww);if (rq) {/* nonblocking is always false */
@@ -2062,7 +2063,7 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb, err =eb_reinit_userptr (eb); err_relock: -i915_gem_ww_ctx_init (&eb->ww, true); +i915_gem_ww_ctx_init (eb->ww, true);if (err) goto out;
@@ -2119,8 +2120,8 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb, err:if (err == -EDEADLK ) { -eb_release_vmas (eb, false); - err =i915_gem_ww_ctx_backoff (&eb->ww); +eb_release_vmas (eb, false, true); + err =i915_gem_ww_ctx_backoff (eb->ww);if (!err) goto repeat_validate; }
@@ -2152,7 +2153,7 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb,return err; } -static int eb_relocate_parse (struct i915_execbuffer *eb) +static int eb_relocate_parse (struct i915_execbuffer *eb,bool first) {int err;struct i915_request *rq =NULL ;
@@ -2189,14 +2190,16 @@ static int eb_relocate_parse(struct i915_execbuffer *eb)/* only throttle once, even if we didn't need to throttle */ throttle = false; - err =eb_validate_vmas (eb); -if (err == -EAGAIN ) - goto slow; -else if (err) - goto err; +if (first) { + err =eb_validate_vmas (eb); +if (err == -EAGAIN ) + goto slow; +else if (err) + goto err; + }/* The objects are in their final locations, apply the relocations. */ -if (eb->args->flags & __EXEC_HAS_RELOC) { +if (eb->args->flags & __EXEC_HAS_RELOC && first) {struct eb_vma *ev;list_for_each_entry (ev, &eb->relocs, reloc_link) {
@@ -2211,13 +2214,13 @@ static int eb_relocate_parse(struct i915_execbuffer *eb) goto slow; } -if (!err) +if (!err && first) err =eb_parse (eb); err:if (err == -EDEADLK ) { -eb_release_vmas (eb, false); - err =i915_gem_ww_ctx_backoff (&eb->ww); +eb_release_vmas (eb, false, true); + err =i915_gem_ww_ctx_backoff (eb->ww);if (!err) goto retry; }
@@ -2398,7 +2401,7 @@ shadow_batch_pin(struct i915_execbuffer *eb,if (IS_ERR (vma))return vma; - err =i915_vma_pin_ww (vma, &eb->ww,0 ,0 , flags); + err =i915_vma_pin_ww (vma, eb->ww,0 ,0 , flags);if (err)return ERR_PTR (err);
@@ -2412,7 +2415,7 @@ static struct i915_vma *eb_dispatch_secure(struct i915_execbuffer *eb, struct i9* batch" bit. Hence we need to pin secure batches into the global gtt. * hsw should have this fixed, but bdw mucks it up again. */ if (eb->batch_flags &I915_DISPATCH_SECURE ) -return i915_gem_object_ggtt_pin_ww (vma->obj, &eb->ww,NULL ,0 ,0 ,0 ); +return i915_gem_object_ggtt_pin_ww (vma->obj, eb->ww,NULL ,0 ,0 ,0 );return NULL ; }
@@ -2458,7 +2461,7 @@ static int eb_parse(struct i915_execbuffer *eb) eb->batch_pool = pool; } - err =i915_gem_object_lock (pool->obj, &eb->ww); + err =i915_gem_object_lock (pool->obj, eb->ww);if (err) goto err;
@@ -2666,7 +2669,7 @@ static struct i915_request *eb_pin_engine(struct i915_execbuffer *eb, bool throt* GGTT space, so do this first before we reserve a seqno for * ourselves. */ - err =intel_context_pin_ww (ce, &eb->ww); + err =intel_context_pin_ww (ce, eb->ww);if (err)return ERR_PTR (err);
@@ -3218,7 +3221,8 @@ i915_gem_do_execbuffer(struct drm_device *dev,unsigned int batch_number, struct dma_fence *in_fence, struct dma_fence *exec_fence, - struct dma_fence **out_fence) + struct dma_fence **out_fence, + struct i915_gem_ww_ctx *ww) { struct drm_i915_private *i915 = to_i915(dev);struct i915_execbuffer eb;
@@ -3239,7 +3243,8 @@ i915_gem_do_execbuffer(struct drm_device *dev, eb.exec = exec; eb.vma = (struct eb_vma *)(exec + args->buffer_count +1 ); - eb.vma[0 ].vma =NULL ; +if (first) + eb.vma[0 ].vma =NULL ; eb.reloc_pool = eb.batch_pool =NULL ; eb.reloc_context =NULL ;
@@ -3251,6 +3256,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, eb.batch_len = args->batch_len; eb.trampoline =NULL ; eb.composite_fence =NULL ; + eb.ww = ww; eb.fences =NULL ; eb.num_fences =0 ;
@@ -3269,9 +3275,14 @@ i915_gem_do_execbuffer(struct drm_device *dev,if (err) goto err_ext; - err =eb_create (&eb); -if (err) - goto err_ext; +if (first) { + err =eb_create (&eb); +if (err) + goto err_ext; + }else { + eb.lut_size = -eb.buffer_count; + } +GEM_BUG_ON (!eb.lut_size);
@@ -3286,15 +3297,22 @@ i915_gem_do_execbuffer(struct drm_device *dev,if (unlikely (err)) goto err_context; - err =eb_lookup_vmas (&eb); -if (err) { -eb_release_vmas (&eb, true); - goto err_engine; +if (first) { + err =eb_lookup_vmas (&eb); +if (err) { +eb_release_vmas (&eb, true, true); + goto err_engine; + } + + }else { + eb.batch = &eb.vma[eb.batch_index]; } -i915_gem_ww_ctx_init (&eb.ww, true); - err =eb_relocate_parse (&eb); +if (first) +i915_gem_ww_ctx_init (eb.ww, true); + + err =eb_relocate_parse (&eb, first);if (err) {/* * If the user expects the execobject.offset and
@@ -3307,7 +3325,8 @@ i915_gem_do_execbuffer(struct drm_device *dev, goto err_vma; } -ww_acquire_done (&eb.ww.ctx); +if (first) +ww_acquire_done (&eb.ww->ctx); batch = eb.batch->vma;
@@ -3410,11 +3429,12 @@ i915_gem_do_execbuffer(struct drm_device *dev,i915_request_put (eb.request); err_vma: -eb_release_vmas (&eb, true); +eb_release_vmas (&eb, true, err || last);if (eb.trampoline)i915_vma_unpin (eb.trampoline);WARN_ON (err == -EDEADLK ); -i915_gem_ww_ctx_fini (&eb.ww); +if (err || last) +i915_gem_ww_ctx_fini (eb.ww);if (eb.batch_pool)intel_gt_buffer_pool_put (eb.batch_pool);
@@ -3476,6 +3496,7 @@ i915_gem_execbuffer2_ioctl(struct drm_device *dev, void *data,const size_t count = args->buffer_count;int err;struct i915_gem_context *ctx; +struct i915_gem_ww_ctx ww;struct intel_context *parent =NULL ;unsigned int num_batches =1 , i;bool is_parallel = false;
@@ -3603,7 +3624,8 @@ i915_gem_execbuffer2_ioctl(struct drm_device *dev, void *data,0 , in_fence, exec_fence, - out_fences); + out_fences, + &ww);for (i =1 ; err ==0 && i < num_batches; i++) err = i915_gem_do_execbuffer(dev, file, args, exec2_list,
@@ -3613,7 +3635,8 @@ i915_gem_execbuffer2_ioctl(struct drm_device *dev, void *data, i,NULL ,NULL , - out_fences); + out_fences, + &ww);if (is_parallel)mutex_unlock (&parent->parallel_submit);
drivers/gpu/drm/i915/gem/selftests/i915_gem_execbuffer.c +8 -6
@@ -32,11 +32,11 @@ static int __igt_gpu_reloc(struct i915_execbuffer *eb,if (IS_ERR (vma))return PTR_ERR (vma); - err =i915_gem_object_lock (obj, &eb->ww); + err =i915_gem_object_lock (obj, eb->ww);if (err)return err; - err =i915_vma_pin_ww (vma, &eb->ww,0 ,0 ,PIN_USER |PIN_HIGH ); + err =i915_vma_pin_ww (vma, eb->ww,0 ,0 ,PIN_USER |PIN_HIGH );if (err)return err;
@@ -106,10 +106,12 @@ static int __igt_gpu_reloc(struct i915_execbuffer *eb,static int igt_gpu_reloc (void *arg) {struct i915_execbuffer eb; +struct i915_gem_ww_ctx ww;struct drm_i915_gem_object *scratch;int err =0 ;u32 *map; + eb.ww = &ww; eb.i915 = arg; scratch =i915_gem_object_create_internal (eb.i915,4096 );
@@ -141,20 +143,20 @@ static int igt_gpu_reloc(void *arg) eb.reloc_pool =NULL ; eb.reloc_context =NULL ; -i915_gem_ww_ctx_init (&eb.ww, false); +i915_gem_ww_ctx_init (eb.ww, false); retry: - err =intel_context_pin_ww (eb.context, &eb.ww); + err =intel_context_pin_ww (eb.context, eb.ww);if (!err) { err =__igt_gpu_reloc (&eb, scratch);intel_context_unpin (eb.context); }if (err == -EDEADLK ) { - err =i915_gem_ww_ctx_backoff (&eb.ww); + err =i915_gem_ww_ctx_backoff (eb.ww);if (!err) goto retry; } -i915_gem_ww_ctx_fini (&eb.ww); +i915_gem_ww_ctx_fini (eb.ww);if (eb.reloc_pool)intel_gt_buffer_pool_put (eb.reloc_pool);
Signature · 6 lines
2.28.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
RFC PATCH 41/42 drm/i915: Enable multi-bb execbuf
In reply to Matthew BrostEnable multi-bb execbuf by enabling the set_parallel extension.
--- drivers/gpu/drm/i915/gem/i915_gem_context.c | 3 --- 1 file changed, 3 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_context.c +0 -3
@@ -533,9 +533,6 @@ set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,struct intel_engine_cs **siblings =NULL ;intel_engine_mask_t prev_mask; -/* Disabling for now */ -return -ENODEV ; -if (!(intel_uc_uses_guc_submission (&i915->gt.uc)))return -ENODEV ;
Signature · 1 line
2.28.0
RFC PATCH 42/42 drm/i915/execlists: Parallel submission support for execlists
In reply to Matthew BrostA weak implmentation of parallel submission (multi-bb execbuf IOCTL) for execlists. Basically doing as little as possible to support this interface for execlists - basically just passing submit fences between each request generated and virtual engines are not allowed. This is on par with what is there for the existing (hopefully soon deprecated) bonding interface.
As with the GuC implementation the pinning interface laying is broken. This will get cleaned up once the pre_pin / post_unpin laying violations get fixed. Rather than try to fix it here, just do what the GuC is doing in the meantime.
--- drivers/gpu/drm/i915/gem/i915_gem_context.c | 9 +- drivers/gpu/drm/i915/gt/intel_context.c | 1 - drivers/gpu/drm/i915/gt/intel_context_types.h | 2 + .../drm/i915/gt/intel_execlists_submission.c | 196 ++++++++++++++++++ 4 files changed, 204 insertions(+), 4 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_context.c +6 -3
@@ -533,9 +533,6 @@ set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,struct intel_engine_cs **siblings =NULL ;intel_engine_mask_t prev_mask; -if (!(intel_uc_uses_guc_submission (&i915->gt.uc))) -return -ENODEV ; -if (get_user (slot, &ext->engine_index))return -EFAULT ;
@@ -545,6 +542,12 @@ set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,if (get_user (num_siblings, &ext->num_siblings))return -EFAULT ; +if (!intel_uc_uses_guc_submission (&i915->gt.uc) && num_siblings !=1 ) { +drm_dbg (&i915->drm,"Only 1 sibling (%d) supported in non-GuC mode\n" , + num_siblings); +return -EINVAL ; + } +if (slot >= set->num_engines) {drm_dbg (&i915->drm,"Invalid placement value, %d >= %d\n" , slot, set->num_engines);
drivers/gpu/drm/i915/gt/intel_context.c +0 -1
@@ -637,7 +637,6 @@ void intel_context_bind_parent_child(struct intel_context *parent,* Callers responsibility to validate that this function is used * correctly but we use GEM_BUG_ON here ensure that they do. */ -GEM_BUG_ON (!intel_engine_uses_guc (parent->engine));GEM_BUG_ON (intel_context_is_pinned (parent));GEM_BUG_ON (intel_context_is_child (parent));GEM_BUG_ON (intel_context_is_pinned (child));
drivers/gpu/drm/i915/gt/intel_context_types.h +2 -0
@@ -116,6 +116,8 @@ struct intel_context {#define CONTEXT_FORCE_SINGLE_SUBMISSION 7#define CONTEXT_NOPREEMPT 8#define CONTEXT_LRCA_DIRTY 9 +#define CONTEXT_PIN_TO_PARENT 10 +#define CONTEXT_PIN_CHILDREN 11struct {u64 timeout_us;
drivers/gpu/drm/i915/gt/intel_execlists_submission.c +196 -0
208 lines not on this page · raw message
Signature · 6 lines
2.28.0 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Patchwork
CI[Intel-gfx] ✗ Fi.CI.BUILD: failure for Parallel submission aka multi-bb execbuf
In reply to Matthew Brost== Series Details ==
== Series Details ==
Series: Parallel submission aka multi-bb execbuf
URL : https://patchwork.freedesktop.org/series/92789/
State : failure
== Summary ==
CALL scripts/checksyscalls.sh
CALL scripts/atomic/check-atomics.sh
DESCEND objtool
CHK include/generated/compile.h
LD [M] drivers/gpu/drm/i915/i915.o
HDRTEST drivers/gpu/drm/i915/selftests/intel_scheduler_helpers.h
In file included from <command-line>:./drivers/gpu/drm/i915/selftests/intel_scheduler_helpers.h:34:63: error: ‘struct intel_gt’ declared inside parameter list will not be visible outside of this definition or declaration [-Werror]
struct intel_engine_cs *intel_selftest_find_any_engine(struct intel_gt *gt);
^~~~~~~~cc1: all warnings being treated as errorsdrivers/gpu/drm/i915/Makefile:320: recipe for target 'drivers/gpu/drm/i915/selftests/intel_scheduler_helpers.hdrtest' failed make[4]: *** [drivers/gpu/drm/i915/selftests/intel_scheduler_helpers.hdrtest] Error 1 scripts/Makefile.build:514: recipe for target 'drivers/gpu/drm/i915' failed
make[3]: *** [drivers/gpu/drm/i915] Error 2scripts/Makefile.build:514: recipe for target 'drivers/gpu/drm' failed
make[2]: *** [drivers/gpu/drm] Error 2scripts/Makefile.build:514: recipe for target 'drivers/gpu' failed
make[1]: *** [drivers/gpu] Error 2
Makefile:1842: recipe for target 'drivers' failed
make: *** [drivers] Error 2
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx