Skip to contentSkip to conversation

Upstreaming prep / all of mbrosts patches

PATCH Superseded Matthew Brost 3 participants 34 messages May 1, 2023

Subsystems (16)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Matthew Brost

author maintainer
#0 lore raw

cover Upstreaming prep / all of mbrosts patches

Series includes:

- DRM scheduler changes for firmware backends (1 to 1 entity to scheduler)
- LR workload story
- VM LRU handling
- GuC doorbell submission
- Basic GPUVA
- Sparse binding support
- GPUVA + extobj + drm exec (collaboration with dakr + Francois Dugast)
- GPUVA + userptr (minimal, more can be once Nouveua has userptr)
- Fix fencing rules for compute / fault mode
- Remove async worker for VM + error handling updates

Series is not fully ready for upstream and some of these things need to get merged upstream first but overall it is largely correct and certainly step in the right direction. Based on its size and the fact it took me 8 hours to rebase this today I'd say let's get this tree and fixup everything else in place.

Minor uAPI breakage, IGT series will be posted early tomorrow.

gitlab link:
https://gitlab.freedesktop.org/drm/xe/kernel/-/merge_requests/344

Christian König (1):
drm: execution context for GEM buffers v3

Danilo Krummrich (2):
maple_tree: split up MA_STATE() macro
drm: manager to keep track of GPUs VA mappings

Matthew Brost (27):
drm/sched: Add run_wq argument to drm_sched_init drm/sched: Move schedule policy to scheduler drm/sched: Add DRM_SCHED_POLICY_SINGLE_ENTITY scheduling policy drm/xe: Use DRM_SCHED_POLICY_SINGLE_ENTITY mode drm/xe: Long running job update
drm/xe: Ensure LR engines are not persistent
drm/xe: Only try to lock external BOs in VM bind drm/xe: VM LRU bulk move
drm/xe/guc: Read HXG fields from DW1 of G2H response drm/xe/guc: Return the lower part of blocking H2G message drm/xe/guc: Use doorbells for submission if possible drm/xe/guc: Print doorbell ID in GuC engine debugfs entry
maple_tree: Export mas_preallocate
drm/xe: Port Xe to GPUVA
drm/xe: NULL binding implementation
drm/xe: Avoid doing rebinds
drm/xe: Reduce the number list links in xe_vma drm/xe: Optimize size of xe_vma allocation
drm/gpuva: Add drm device to GPUVA manager
drm/gpuva: Move dma-resv to GPUVA manager
drm/gpuva: Add support for extobj
drm/xe: Userptr refactor
drm/exec: Always compile drm_exec
drm/xe: Use drm_exec for locking rather than TTM exec helpers drm/xe: Allow dma-fences as in-syncs for compute / faulting VM drm/xe: Allow compute VMs to output dma-fences on binds drm/xe: remove async worker, sync binds, new error handling

 Documentation/gpu/drm-mm.rst                 |   43 +
 drivers/gpu/drm/Kconfig                      |    6 +
 drivers/gpu/drm/Makefile                     |    4 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c   |    3 +-
 drivers/gpu/drm/drm_debugfs.c                |   41 +
 drivers/gpu/drm/drm_exec.c                   |  248 ++
 drivers/gpu/drm/drm_gem.c                    |    3 +
 drivers/gpu/drm/drm_gpuva_mgr.c              | 1779 ++++++++++++
 drivers/gpu/drm/etnaviv/etnaviv_sched.c      |    5 +-
 drivers/gpu/drm/i915/display/intel_display.c |    6 +-
 drivers/gpu/drm/lima/lima_sched.c            |    5 +-
 drivers/gpu/drm/msm/msm_ringbuffer.c         |    5 +-
 drivers/gpu/drm/panfrost/panfrost_job.c      |    5 +-
 drivers/gpu/drm/scheduler/sched_entity.c     |   84 +-
 drivers/gpu/drm/scheduler/sched_fence.c      |    2 +-
 drivers/gpu/drm/scheduler/sched_main.c       |   88 +-
 drivers/gpu/drm/v3d/v3d_sched.c              |   25 +-
 drivers/gpu/drm/xe/Kconfig                   |    1 +
 drivers/gpu/drm/xe/regs/xe_guc_regs.h        |    1 +
 drivers/gpu/drm/xe/xe_bo.c                   |  100 +-
 drivers/gpu/drm/xe/xe_bo.h                   |   13 +-
 drivers/gpu/drm/xe/xe_bo_evict.c             |   24 +-
 drivers/gpu/drm/xe/xe_bo_types.h             |    1 -
 drivers/gpu/drm/xe/xe_device.c               |    2 +-
 drivers/gpu/drm/xe/xe_dma_buf.c              |    2 +-
 drivers/gpu/drm/xe/xe_engine.c               |   50 +-
 drivers/gpu/drm/xe/xe_engine.h               |    4 +
 drivers/gpu/drm/xe/xe_engine_types.h         |    1 +
 drivers/gpu/drm/xe/xe_exec.c                 |  117 +-
 drivers/gpu/drm/xe/xe_execlist.c             |    3 +-
 drivers/gpu/drm/xe/xe_gt_pagefault.c         |   84 +-
 drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c  |   14 +-
 drivers/gpu/drm/xe/xe_guc.c                  |    6 +
 drivers/gpu/drm/xe/xe_guc_ct.c               |   12 +-
 drivers/gpu/drm/xe/xe_guc_engine_types.h     |    9 +
 drivers/gpu/drm/xe/xe_guc_pc.c               |    6 +-
 drivers/gpu/drm/xe/xe_guc_submit.c           |  398 ++-
 drivers/gpu/drm/xe/xe_guc_submit.h           |    1 +
 drivers/gpu/drm/xe/xe_guc_types.h            |    4 +
 drivers/gpu/drm/xe/xe_huc.c                  |    2 +-
 drivers/gpu/drm/xe/xe_lrc.c                  |    8 +-
 drivers/gpu/drm/xe/xe_migrate.c              |   31 +-
 drivers/gpu/drm/xe/xe_pt.c                   |  198 +-
 drivers/gpu/drm/xe/xe_sync.c                 |   26 +-
 drivers/gpu/drm/xe/xe_sync.h                 |    2 +-
 drivers/gpu/drm/xe/xe_trace.h                |   20 +-
 drivers/gpu/drm/xe/xe_vm.c                   | 2565 +++++++-----------
 drivers/gpu/drm/xe/xe_vm.h                   |  135 +-
 drivers/gpu/drm/xe/xe_vm_madvise.c           |  125 +-
 drivers/gpu/drm/xe/xe_vm_types.h             |  324 ++-
 drivers/gpu/drm/xe/xe_wait_user_fence.c      |   43 +-
 include/drm/drm_debugfs.h                    |   24 +
 include/drm/drm_drv.h                        |    7 +
 include/drm/drm_exec.h                       |  115 +
 include/drm/drm_gem.h                        |   75 +
 include/drm/drm_gpuva_mgr.h                  |  759 ++++++
 include/drm/gpu_scheduler.h                  |   29 +-
 include/linux/maple_tree.h                   |    7 +-
 include/uapi/drm/xe_drm.h                    |   87 +-
 lib/maple_tree.c                             |    1 +
 60 files changed, 5484 insertions(+), 2304 deletions(-)
 create mode 100644 drivers/gpu/drm/drm_exec.c
 create mode 100644 drivers/gpu/drm/drm_gpuva_mgr.c
 create mode 100644 include/drm/drm_exec.h
 create mode 100644 include/drm/drm_gpuva_mgr.h
Signature · 1 line
2.34.1
author maintainer
#1 lore raw

PATCH 1/30 drm/sched: Add run_wq argument to drm_sched_init

We will have this argument upstream, lets pull into the Xe repo.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c |  2 +-
 drivers/gpu/drm/etnaviv/etnaviv_sched.c    |  2 +-
 drivers/gpu/drm/lima/lima_sched.c          |  2 +-
 drivers/gpu/drm/msm/msm_ringbuffer.c       |  2 +-
 drivers/gpu/drm/panfrost/panfrost_job.c    |  2 +-
 drivers/gpu/drm/scheduler/sched_main.c     |  4 +++-
 drivers/gpu/drm/v3d/v3d_sched.c            | 10 +++++-----
 drivers/gpu/drm/xe/xe_execlist.c           |  2 +-
 drivers/gpu/drm/xe/xe_guc_submit.c         |  2 +-
 include/drm/gpu_scheduler.h                |  1 +
 10 files changed, 16 insertions(+), 13 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1 -1
index 902f9b5ff82c..fe28f6b71fe3 100644
@@ -2364,7 +2364,7 @@ static int amdgpu_device_init_schedulers(struct amdgpu_device *adev)
 			break;
 		}
 
-		r = drm_sched_init(&ring->sched, &amdgpu_sched_ops,
+		r = drm_sched_init(&ring->sched, &amdgpu_sched_ops, NULL,
 				   ring->num_hw_submission, amdgpu_job_hang_limit,
 				   timeout, adev->reset_domain->wq,
 				   ring->sched_score, ring->name,
drivers/gpu/drm/etnaviv/etnaviv_sched.c +1 -1
index 1ae87dfd19c4..8486a2923f1b 100644
@@ -133,7 +133,7 @@ int etnaviv_sched_init(struct etnaviv_gpu *gpu)
 {
 	int ret;
 
-	ret = drm_sched_init(&gpu->sched, &etnaviv_sched_ops,
+	ret = drm_sched_init(&gpu->sched, &etnaviv_sched_ops, NULL,
 			     etnaviv_hw_jobs_limit, etnaviv_job_hang_limit,
 			     msecs_to_jiffies(500), NULL, NULL,
 			     dev_name(gpu->dev), gpu->dev);
drivers/gpu/drm/lima/lima_sched.c +1 -1
index ff003403fbbc..54f53bece27c 100644
@@ -488,7 +488,7 @@ int lima_sched_pipe_init(struct lima_sched_pipe *pipe, const char *name)
 
 	INIT_WORK(&pipe->recover_work, lima_sched_recover_work);
 
-	return drm_sched_init(&pipe->base, &lima_sched_ops, 1,
+	return drm_sched_init(&pipe->base, &lima_sched_ops, NULL, 1,
 			      lima_job_hang_limit,
 			      msecs_to_jiffies(timeout), NULL,
 			      NULL, name, pipe->ldev->dev);
drivers/gpu/drm/msm/msm_ringbuffer.c +1 -1
index 57a8e9564540..5879fc262047 100644
@@ -95,7 +95,7 @@ struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int id,
 	 /* currently managing hangcheck ourselves: */
 	sched_timeout = MAX_SCHEDULE_TIMEOUT;
 
-	ret = drm_sched_init(&ring->sched, &msm_sched_ops,
+	ret = drm_sched_init(&ring->sched, &msm_sched_ops, NULL,
 			num_hw_submissions, 0, sched_timeout,
 			NULL, NULL, to_msm_bo(ring->bo)->name, gpu->dev->dev);
 	if (ret) {
drivers/gpu/drm/panfrost/panfrost_job.c +1 -1
index dbc597ab46fb..f48b07056a16 100644
@@ -815,7 +815,7 @@ int panfrost_job_init(struct panfrost_device *pfdev)
 		js->queue[j].fence_context = dma_fence_context_alloc(1);
 
 		ret = drm_sched_init(&js->queue[j].sched,
-				     &panfrost_sched_ops,
+				     &panfrost_sched_ops, NULL,
 				     nentries, 0,
 				     msecs_to_jiffies(JOB_TIMEOUT_MS),
 				     pfdev->reset.wq,
drivers/gpu/drm/scheduler/sched_main.c +3 -1
index cfd8a838e283..e79b9c760efe 100644
@@ -1182,6 +1182,7 @@ static void drm_sched_main(struct work_struct *w)
  *
  * @sched: scheduler instance
  * @ops: backend operations for this scheduler
+ * @run_wq: workqueue to use for run work. If NULL, the system_wq is used
  * @hw_submission: number of hw submissions that can be in flight
  * @hang_limit: number of times to allow a job to hang before dropping it
  * @timeout: timeout value in jiffies for the scheduler
@@ -1195,6 +1196,7 @@ static void drm_sched_main(struct work_struct *w)
  */
 int drm_sched_init(struct drm_gpu_scheduler *sched,
 		   const struct drm_sched_backend_ops *ops,
+		   struct workqueue_struct *run_wq,
 		   unsigned hw_submission, unsigned hang_limit,
 		   long timeout, struct workqueue_struct *timeout_wq,
 		   atomic_t *score, const char *name, struct device *dev)
@@ -1203,9 +1205,9 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,
 	sched->ops = ops;
 	sched->hw_submission_limit = hw_submission;
 	sched->name = name;
+	sched->run_wq = run_wq ? : system_wq;
 	sched->timeout = timeout;
 	sched->timeout_wq = timeout_wq ? : system_wq;
-	sched->run_wq = system_wq;	/* FIXME: Let user pass this in */
 	sched->hang_limit = hang_limit;
 	sched->score = score ? score : &sched->_score;
 	sched->dev = dev;
drivers/gpu/drm/v3d/v3d_sched.c +5 -5
index 06238e6d7f5c..38e092ea41e6 100644
@@ -388,7 +388,7 @@ v3d_sched_init(struct v3d_dev *v3d)
 	int ret;
 
 	ret = drm_sched_init(&v3d->queue[V3D_BIN].sched,
-			     &v3d_bin_sched_ops,
+			     &v3d_bin_sched_ops, NULL,
 			     hw_jobs_limit, job_hang_limit,
 			     msecs_to_jiffies(hang_limit_ms), NULL,
 			     NULL, "v3d_bin", v3d->drm.dev);
@@ -396,7 +396,7 @@ v3d_sched_init(struct v3d_dev *v3d)
 		return ret;
 
 	ret = drm_sched_init(&v3d->queue[V3D_RENDER].sched,
-			     &v3d_render_sched_ops,
+			     &v3d_render_sched_ops, NULL,
 			     hw_jobs_limit, job_hang_limit,
 			     msecs_to_jiffies(hang_limit_ms), NULL,
 			     NULL, "v3d_render", v3d->drm.dev);
@@ -404,7 +404,7 @@ v3d_sched_init(struct v3d_dev *v3d)
 		goto fail;
 
 	ret = drm_sched_init(&v3d->queue[V3D_TFU].sched,
-			     &v3d_tfu_sched_ops,
+			     &v3d_tfu_sched_ops, NULL,
 			     hw_jobs_limit, job_hang_limit,
 			     msecs_to_jiffies(hang_limit_ms), NULL,
 			     NULL, "v3d_tfu", v3d->drm.dev);
@@ -413,7 +413,7 @@ v3d_sched_init(struct v3d_dev *v3d)
 
 	if (v3d_has_csd(v3d)) {
 		ret = drm_sched_init(&v3d->queue[V3D_CSD].sched,
-				     &v3d_csd_sched_ops,
+				     &v3d_csd_sched_ops, NULL,
 				     hw_jobs_limit, job_hang_limit,
 				     msecs_to_jiffies(hang_limit_ms), NULL,
 				     NULL, "v3d_csd", v3d->drm.dev);
@@ -421,7 +421,7 @@ v3d_sched_init(struct v3d_dev *v3d)
 			goto fail;
 
 		ret = drm_sched_init(&v3d->queue[V3D_CACHE_CLEAN].sched,
-				     &v3d_cache_clean_sched_ops,
+				     &v3d_cache_clean_sched_ops, NULL,
 				     hw_jobs_limit, job_hang_limit,
 				     msecs_to_jiffies(hang_limit_ms), NULL,
 				     NULL, "v3d_cache_clean", v3d->drm.dev);
drivers/gpu/drm/xe/xe_execlist.c +1 -1
index de4f0044b211..d6d60ebf3d5f 100644
@@ -336,7 +336,7 @@ static int execlist_engine_init(struct xe_engine *e)
 
 	exl->engine = e;
 
-	err = drm_sched_init(&exl->sched, &drm_sched_ops,
+	err = drm_sched_init(&exl->sched, &drm_sched_ops, NULL,
 			     e->lrc[0].ring.size / MAX_JOB_SIZE_BYTES,
 			     XE_SCHED_HANG_LIMIT, XE_SCHED_JOB_TIMEOUT,
 			     NULL, NULL, e->hwe->name,
drivers/gpu/drm/xe/xe_guc_submit.c +1 -1
index e857013070b9..735f31257f3a 100644
@@ -1081,7 +1081,7 @@ static int guc_engine_init(struct xe_engine *e)
 	init_waitqueue_head(&ge->suspend_wait);
 
 	timeout = xe_vm_no_dma_fences(e->vm) ? MAX_SCHEDULE_TIMEOUT : HZ * 5;
-	err = drm_sched_init(&ge->sched, &drm_sched_ops,
+	err = drm_sched_init(&ge->sched, &drm_sched_ops, NULL,
 			     e->lrc[0].ring.size / MAX_JOB_SIZE_BYTES,
 			     64, timeout, guc_to_gt(guc)->ordered_wq, NULL,
 			     e->name, gt_to_xe(e->gt)->drm.dev);
include/drm/gpu_scheduler.h +1 -0
index cf85f93218fc..09bc39840dc8 100644
@@ -539,6 +539,7 @@ struct drm_gpu_scheduler {
 
 int drm_sched_init(struct drm_gpu_scheduler *sched,
 		   const struct drm_sched_backend_ops *ops,
+		   struct workqueue_struct *run_wq,
 		   uint32_t hw_submission, unsigned hang_limit,
 		   long timeout, struct workqueue_struct *timeout_wq,
 		   atomic_t *score, const char *name, struct device *dev);
Signature · 1 line
2.34.1
author maintainer
#2 lore raw

PATCH 2/30 drm/sched: Move schedule policy to scheduler

In reply to Matthew Brost

Rather than a global modparam for scheduling policy, move the scheduling policy to scheduler so user can control each scheduler policy.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c |  1 +
 drivers/gpu/drm/etnaviv/etnaviv_sched.c    |  3 ++-
 drivers/gpu/drm/lima/lima_sched.c          |  3 ++-
 drivers/gpu/drm/msm/msm_ringbuffer.c       |  3 ++-
 drivers/gpu/drm/panfrost/panfrost_job.c    |  3 ++-
 drivers/gpu/drm/scheduler/sched_entity.c   | 24 ++++++++++++++++++----
 drivers/gpu/drm/scheduler/sched_main.c     | 21 ++++++++++++++-----
 drivers/gpu/drm/v3d/v3d_sched.c            | 15 +++++++++-----
 drivers/gpu/drm/xe/xe_execlist.c           |  2 +-
 drivers/gpu/drm/xe/xe_guc_submit.c         |  3 ++-
 include/drm/gpu_scheduler.h                | 20 ++++++++++++------
 11 files changed, 72 insertions(+), 26 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1 -0
index fe28f6b71fe3..577ea5b98cd5 100644
@@ -2368,6 +2368,7 @@ static int amdgpu_device_init_schedulers(struct amdgpu_device *adev)
 				   ring->num_hw_submission, amdgpu_job_hang_limit,
 				   timeout, adev->reset_domain->wq,
 				   ring->sched_score, ring->name,
+				   DRM_SCHED_POLICY_DEFAULT,
 				   adev->dev);
 		if (r) {
 			DRM_ERROR("Failed to create scheduler on ring %s.\n",
drivers/gpu/drm/etnaviv/etnaviv_sched.c +2 -1
index 8486a2923f1b..61204a3f8b0b 100644
@@ -136,7 +136,8 @@ int etnaviv_sched_init(struct etnaviv_gpu *gpu)
 	ret = drm_sched_init(&gpu->sched, &etnaviv_sched_ops, NULL,
 			     etnaviv_hw_jobs_limit, etnaviv_job_hang_limit,
 			     msecs_to_jiffies(500), NULL, NULL,
-			     dev_name(gpu->dev), gpu->dev);
+			     dev_name(gpu->dev), DRM_SCHED_POLICY_DEFAULT,
+			     gpu->dev);
 	if (ret)
 		return ret;
 
drivers/gpu/drm/lima/lima_sched.c +2 -1
index 54f53bece27c..33042ba6ae93 100644
@@ -491,7 +491,8 @@ int lima_sched_pipe_init(struct lima_sched_pipe *pipe, const char *name)
 	return drm_sched_init(&pipe->base, &lima_sched_ops, NULL, 1,
 			      lima_job_hang_limit,
 			      msecs_to_jiffies(timeout), NULL,
-			      NULL, name, pipe->ldev->dev);
+			      NULL, name, DRM_SCHED_POLICY_DEFAULT,
+			      pipe->ldev->dev);
 }
 
 void lima_sched_pipe_fini(struct lima_sched_pipe *pipe)
drivers/gpu/drm/msm/msm_ringbuffer.c +2 -1
index 5879fc262047..f408a9097315 100644
@@ -97,7 +97,8 @@ struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int id,
 
 	ret = drm_sched_init(&ring->sched, &msm_sched_ops, NULL,
 			num_hw_submissions, 0, sched_timeout,
-			NULL, NULL, to_msm_bo(ring->bo)->name, gpu->dev->dev);
+			NULL, NULL, to_msm_bo(ring->bo)->name,
+			DRM_SCHED_POLICY_DEFAULT, gpu->dev->dev);
 	if (ret) {
 		goto fail;
 	}
drivers/gpu/drm/panfrost/panfrost_job.c +2 -1
index f48b07056a16..effa48b33dce 100644
@@ -819,7 +819,8 @@ int panfrost_job_init(struct panfrost_device *pfdev)
 				     nentries, 0,
 				     msecs_to_jiffies(JOB_TIMEOUT_MS),
 				     pfdev->reset.wq,
-				     NULL, "pan_js", pfdev->dev);
+				     NULL, "pan_js", DRM_SCHED_POLICY_DEFAULT,
+				     pfdev->dev);
 		if (ret) {
 			dev_err(pfdev->dev, "Failed to create scheduler: %d.", ret);
 			goto err_sched;
drivers/gpu/drm/scheduler/sched_entity.c +20 -4
index 15d04a0ec623..2300b2fc06ab 100644
@@ -33,6 +33,20 @@
 #define to_drm_sched_job(sched_job)		\
 		container_of((sched_job), struct drm_sched_job, queue_node)
 
+static bool bad_policies(struct drm_gpu_scheduler **sched_list,
+			 unsigned int num_sched_list)
+{
+	enum drm_sched_policy sched_policy = sched_list[0]->sched_policy;
+	unsigned int i;
+
+	/* All scdedule policies must match */
+	for (i = 1; i < num_sched_list; ++i)
+		if (sched_policy != sched_list[i]->sched_policy)
+			return true;
+
+	return false;
+}
+
 /**
  * drm_sched_entity_init - Init a context entity used by scheduler when
  * submit to HW ring.
@@ -62,7 +76,8 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,
 			  unsigned int num_sched_list,
 			  atomic_t *guilty)
 {
-	if (!(entity && sched_list && (num_sched_list == 0 || sched_list[0])))
+	if (!(entity && sched_list && (num_sched_list == 0 || sched_list[0])) ||
+	    bad_policies(sched_list, num_sched_list))
 		return -EINVAL;
 
 	memset(entity, 0, sizeof(struct drm_sched_entity));
@@ -75,8 +90,9 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,
 	entity->last_scheduled = NULL;
 	RB_CLEAR_NODE(&entity->rb_tree_node);
 
-	if(num_sched_list)
+	if(num_sched_list) {
 		entity->rq = &sched_list[0]->sched_rq[entity->priority];
+	}
 
 	init_completion(&entity->entity_idle);
 
@@ -440,7 +456,7 @@ struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)
 	 * Update the entity's location in the min heap according to
 	 * the timestamp of the next job, if any.
 	 */
-	if (drm_sched_policy == DRM_SCHED_POLICY_FIFO) {
+	if (entity->rq->sched->sched_policy == DRM_SCHED_POLICY_FIFO) {
 		struct drm_sched_job *next;
 
 		next = to_drm_sched_job(spsc_queue_peek(&entity->job_queue));
@@ -528,7 +544,7 @@ void drm_sched_entity_push_job(struct drm_sched_job *sched_job)
 		drm_sched_rq_add_entity(entity->rq, entity);
 		spin_unlock(&entity->rq_lock);
 
-		if (drm_sched_policy == DRM_SCHED_POLICY_FIFO)
+		if (entity->rq->sched->sched_policy == DRM_SCHED_POLICY_FIFO)
 			drm_sched_rq_update_fifo(entity, sched_job->submit_ts);
 
 		drm_sched_wakeup(entity->rq->sched);
drivers/gpu/drm/scheduler/sched_main.c +16 -5
index e79b9c760efe..6777a2db554f 100644
@@ -62,14 +62,14 @@
 #define to_drm_sched_job(sched_job)		\
 		container_of((sched_job), struct drm_sched_job, queue_node)
 
-int drm_sched_policy = DRM_SCHED_POLICY_FIFO;
+int default_drm_sched_policy = DRM_SCHED_POLICY_FIFO;
 
 /**
  * DOC: sched_policy (int)
  * Used to override default entities scheduling policy in a run queue.
  */
 MODULE_PARM_DESC(sched_policy, "Specify the scheduling policy for entities on a run-queue, " __stringify(DRM_SCHED_POLICY_RR) " = Round Robin, " __stringify(DRM_SCHED_POLICY_FIFO) " = FIFO (default).");
-module_param_named(sched_policy, drm_sched_policy, int, 0444);
+module_param_named(sched_policy, default_drm_sched_policy, int, 0444);
 
 static __always_inline bool drm_sched_entity_compare_before(struct rb_node *a,
 							    const struct rb_node *b)
@@ -173,7 +173,7 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,
 	if (rq->current_entity == entity)
 		rq->current_entity = NULL;
 
-	if (drm_sched_policy == DRM_SCHED_POLICY_FIFO)
+	if (rq->sched->sched_policy == DRM_SCHED_POLICY_FIFO)
 		drm_sched_rq_remove_fifo_locked(entity);
 
 	spin_unlock(&rq->lock);
@@ -956,7 +956,7 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)
 
 	/* Kernel run queue has higher priority than normal run queue*/
 	for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >= DRM_SCHED_PRIORITY_MIN; i--) {
-		entity = drm_sched_policy == DRM_SCHED_POLICY_FIFO ?
+		entity = sched->sched_policy == DRM_SCHED_POLICY_FIFO ?
 			drm_sched_rq_select_entity_fifo(&sched->sched_rq[i]) :
 			drm_sched_rq_select_entity_rr(&sched->sched_rq[i]);
 		if (entity)
@@ -1190,6 +1190,7 @@ static void drm_sched_main(struct work_struct *w)
  *		used
  * @score: optional score atomic shared with other schedulers
  * @name: name used for debugging
+ * @sched_policy: schedule policy
  * @dev: target &struct device
  *
  * Return 0 on success, otherwise error code.
@@ -1199,9 +1200,15 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,
 		   struct workqueue_struct *run_wq,
 		   unsigned hw_submission, unsigned hang_limit,
 		   long timeout, struct workqueue_struct *timeout_wq,
-		   atomic_t *score, const char *name, struct device *dev)
+		   atomic_t *score, const char *name,
+		   enum drm_sched_policy sched_policy,
+		   struct device *dev)
 {
 	int i;
+
+	if (sched_policy >= DRM_SCHED_POLICY_COUNT)
+		return -EINVAL;
+
 	sched->ops = ops;
 	sched->hw_submission_limit = hw_submission;
 	sched->name = name;
@@ -1211,6 +1218,10 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,
 	sched->hang_limit = hang_limit;
 	sched->score = score ? score : &sched->_score;
 	sched->dev = dev;
+	if (sched_policy == DRM_SCHED_POLICY_DEFAULT)
+		sched->sched_policy = default_drm_sched_policy;
+	else
+		sched->sched_policy = sched_policy;
 	for (i = DRM_SCHED_PRIORITY_MIN; i < DRM_SCHED_PRIORITY_COUNT; i++)
 		drm_sched_rq_init(sched, &sched->sched_rq[i]);
 
drivers/gpu/drm/v3d/v3d_sched.c +10 -5
index 38e092ea41e6..5e3fe77fa991 100644
@@ -391,7 +391,8 @@ v3d_sched_init(struct v3d_dev *v3d)
 			     &v3d_bin_sched_ops, NULL,
 			     hw_jobs_limit, job_hang_limit,
 			     msecs_to_jiffies(hang_limit_ms), NULL,
-			     NULL, "v3d_bin", v3d->drm.dev);
+			     NULL, "v3d_bin", DRM_SCHED_POLICY_DEFAULT,
+			     v3d->drm.dev);
 	if (ret)
 		return ret;
 
@@ -399,7 +400,8 @@ v3d_sched_init(struct v3d_dev *v3d)
 			     &v3d_render_sched_ops, NULL,
 			     hw_jobs_limit, job_hang_limit,
 			     msecs_to_jiffies(hang_limit_ms), NULL,
-			     NULL, "v3d_render", v3d->drm.dev);
+			     ULL, "v3d_render", DRM_SCHED_POLICY_DEFAULT,
+			     v3d->drm.dev);
 	if (ret)
 		goto fail;
 
@@ -407,7 +409,8 @@ v3d_sched_init(struct v3d_dev *v3d)
 			     &v3d_tfu_sched_ops, NULL,
 			     hw_jobs_limit, job_hang_limit,
 			     msecs_to_jiffies(hang_limit_ms), NULL,
-			     NULL, "v3d_tfu", v3d->drm.dev);
+			     NULL, "v3d_tfu", DRM_SCHED_POLICY_DEFAULT,
+			     v3d->drm.dev);
 	if (ret)
 		goto fail;
 
@@ -416,7 +419,8 @@ v3d_sched_init(struct v3d_dev *v3d)
 				     &v3d_csd_sched_ops, NULL,
 				     hw_jobs_limit, job_hang_limit,
 				     msecs_to_jiffies(hang_limit_ms), NULL,
-				     NULL, "v3d_csd", v3d->drm.dev);
+				     NULL, "v3d_csd", DRM_SCHED_POLICY_DEFAULT,
+				     v3d->drm.dev);
 		if (ret)
 			goto fail;
 
@@ -424,7 +428,8 @@ v3d_sched_init(struct v3d_dev *v3d)
 				     &v3d_cache_clean_sched_ops, NULL,
 				     hw_jobs_limit, job_hang_limit,
 				     msecs_to_jiffies(hang_limit_ms), NULL,
-				     NULL, "v3d_cache_clean", v3d->drm.dev);
+				     NULL, "v3d_cache_clean",
+				     DRM_SCHED_POLICY_DEFAULT, v3d->drm.dev);
 		if (ret)
 			goto fail;
 	}
drivers/gpu/drm/xe/xe_execlist.c +1 -1
index d6d60ebf3d5f..48060d14547a 100644
@@ -339,7 +339,7 @@ static int execlist_engine_init(struct xe_engine *e)
 	err = drm_sched_init(&exl->sched, &drm_sched_ops, NULL,
 			     e->lrc[0].ring.size / MAX_JOB_SIZE_BYTES,
 			     XE_SCHED_HANG_LIMIT, XE_SCHED_JOB_TIMEOUT,
-			     NULL, NULL, e->hwe->name,
+			     NULL, NULL, e->hwe->name, DRM_SCHED_POLICY_DEFAULT,
 			     gt_to_xe(e->gt)->drm.dev);
 	if (err)
 		goto err_free;
drivers/gpu/drm/xe/xe_guc_submit.c +2 -1
index 735f31257f3a..9d3fadca43be 100644
@@ -1084,7 +1084,8 @@ static int guc_engine_init(struct xe_engine *e)
 	err = drm_sched_init(&ge->sched, &drm_sched_ops, NULL,
 			     e->lrc[0].ring.size / MAX_JOB_SIZE_BYTES,
 			     64, timeout, guc_to_gt(guc)->ordered_wq, NULL,
-			     e->name, gt_to_xe(e->gt)->drm.dev);
+			     e->name, DRM_SCHED_POLICY_DEFAULT,
+			     gt_to_xe(e->gt)->drm.dev);
 	if (err)
 		goto err_free;
 
include/drm/gpu_scheduler.h +14 -6
index 09bc39840dc8..3df801401028 100644
@@ -63,11 +63,15 @@ enum drm_sched_priority {
 	DRM_SCHED_PRIORITY_UNSET = -2
 };
 
-/* Used to chose between FIFO and RR jobs scheduling */
-extern int drm_sched_policy;
-
-#define DRM_SCHED_POLICY_RR    0
-#define DRM_SCHED_POLICY_FIFO  1
+/* Used to chose default scheduling policy*/
+extern int default_drm_sched_policy;
+
+enum drm_sched_policy {
+	DRM_SCHED_POLICY_DEFAULT,
+	DRM_SCHED_POLICY_RR,
+	DRM_SCHED_POLICY_FIFO,
+	DRM_SCHED_POLICY_COUNT,
+};
 
 /**
  * struct drm_sched_entity - A wrapper around a job queue (typically
@@ -505,6 +509,7 @@ struct drm_sched_backend_ops {
  *              guilty and it will no longer be considered for scheduling.
  * @score: score to help loadbalancer pick a idle sched
  * @_score: score used when the driver doesn't provide one
+ * @sched_policy: Schedule policy for scheduler
  * @ready: marks if the underlying HW is ready to work
  * @free_guilty: A hit to time out handler to free the guilty job.
  * @pause_run_wq: pause queuing of @work_run on @run_wq
@@ -531,6 +536,7 @@ struct drm_gpu_scheduler {
 	int				hang_limit;
 	atomic_t                        *score;
 	atomic_t                        _score;
+	enum drm_sched_policy		sched_policy;
 	bool				ready;
 	bool				free_guilty;
 	bool				pause_run_wq;
@@ -542,7 +548,9 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,
 		   struct workqueue_struct *run_wq,
 		   uint32_t hw_submission, unsigned hang_limit,
 		   long timeout, struct workqueue_struct *timeout_wq,
-		   atomic_t *score, const char *name, struct device *dev);
+		   atomic_t *score, const char *name,
+		   enum drm_sched_policy sched_policy,
+		   struct device *dev);
 
 void drm_sched_fini(struct drm_gpu_scheduler *sched);
 int drm_sched_job_init(struct drm_sched_job *job,
Signature · 1 line
2.34.1
author maintainer
#3 lore raw

PATCH 3/30 drm/sched: Add DRM_SCHED_POLICY_SINGLE_ENTITY scheduling policy

In reply to Matthew Brost

DRM_SCHED_POLICY_SINGLE_ENTITY creates a 1 to 1 relationship between scheduler and entity. No priorities or run queue used in this mode. Intended for devices with firmware schedulers.

---
 drivers/gpu/drm/scheduler/sched_entity.c | 64 +++++++++++++++++++-----
 drivers/gpu/drm/scheduler/sched_fence.c  |  2 +-
 drivers/gpu/drm/scheduler/sched_main.c   | 63 ++++++++++++++++++++---
 include/drm/gpu_scheduler.h              |  8 +++
 4 files changed, 115 insertions(+), 22 deletions(-)
drivers/gpu/drm/scheduler/sched_entity.c +51 -13
index 2300b2fc06ab..8b70900c54cc 100644
@@ -83,6 +83,7 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,
 	memset(entity, 0, sizeof(struct drm_sched_entity));
 	INIT_LIST_HEAD(&entity->list);
 	entity->rq = NULL;
+	entity->single_sched = NULL;
 	entity->guilty = guilty;
 	entity->num_sched_list = num_sched_list;
 	entity->priority = priority;
@@ -91,7 +92,15 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,
 	RB_CLEAR_NODE(&entity->rb_tree_node);
 
 	if(num_sched_list) {
-		entity->rq = &sched_list[0]->sched_rq[entity->priority];
+		if (sched_list[0]->sched_policy !=
+		    DRM_SCHED_POLICY_SINGLE_ENTITY) {
+			entity->rq = &sched_list[0]->sched_rq[entity->priority];
+		} else {
+			if (num_sched_list != 1 || sched_list[0]->single_entity)
+				return -EINVAL;
+			sched_list[0]->single_entity = entity;
+			entity->single_sched = sched_list[0];
+		}
 	}
 
 	init_completion(&entity->entity_idle);
@@ -125,7 +134,8 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,
 				    struct drm_gpu_scheduler **sched_list,
 				    unsigned int num_sched_list)
 {
-	WARN_ON(!num_sched_list || !sched_list);
+	WARN_ON(!num_sched_list || !sched_list ||
+		!!entity->single_sched);
 
 	entity->sched_list = sched_list;
 	entity->num_sched_list = num_sched_list;
@@ -195,13 +205,15 @@ static void drm_sched_entity_kill(struct drm_sched_entity *entity)
 {
 	struct drm_sched_job *job;
 	struct dma_fence *prev;
+	bool single_entity = !!entity->single_sched;
 
-	if (!entity->rq)
+	if (!entity->rq && !single_entity)
 		return;
 
 	spin_lock(&entity->rq_lock);
 	entity->stopped = true;
-	drm_sched_rq_remove_entity(entity->rq, entity);
+	if (!single_entity)
+		drm_sched_rq_remove_entity(entity->rq, entity);
 	spin_unlock(&entity->rq_lock);
 
 	/* Make sure this entity is not used by the scheduler at the moment */
@@ -223,6 +235,20 @@ static void drm_sched_entity_kill(struct drm_sched_entity *entity)
 	dma_fence_put(prev);
 }
 
+/**
+ * drm_sched_entity_to_scheduler - Schedule entity to GPU scheduler
+ * @entity: scheduler entity
+ *
+ * Returns GPU scheduler for the entity
+ */
+struct drm_gpu_scheduler *
+drm_sched_entity_to_scheduler(struct drm_sched_entity *entity)
+{
+	bool single_entity = !!entity->single_sched;
+
+	return single_entity ? entity->single_sched : entity->rq->sched;
+}
+
 /**
  * drm_sched_entity_flush - Flush a context entity
  *
@@ -240,11 +266,12 @@ long drm_sched_entity_flush(struct drm_sched_entity *entity, long timeout)
 	struct drm_gpu_scheduler *sched;
 	struct task_struct *last_user;
 	long ret = timeout;
+	bool single_entity = !!entity->single_sched;
 
-	if (!entity->rq)
+	if (!entity->rq && !single_entity)
 		return 0;
 
-	sched = entity->rq->sched;
+	sched = drm_sched_entity_to_scheduler(entity);
 	/**
 	 * The client will not queue more IBs during this fini, consume existing
 	 * queued IBs or discard them on SIGKILL
@@ -337,7 +364,7 @@ static void drm_sched_entity_wakeup(struct dma_fence *f,
 		container_of(cb, struct drm_sched_entity, cb);
 
 	drm_sched_entity_clear_dep(f, cb);
-	drm_sched_wakeup(entity->rq->sched);
+	drm_sched_wakeup(drm_sched_entity_to_scheduler(entity));
 }
 
 /**
@@ -351,6 +378,8 @@ static void drm_sched_entity_wakeup(struct dma_fence *f,
 void drm_sched_entity_set_priority(struct drm_sched_entity *entity,
 				   enum drm_sched_priority priority)
 {
+	WARN_ON(!!entity->single_sched);
+
 	spin_lock(&entity->rq_lock);
 	entity->priority = priority;
 	spin_unlock(&entity->rq_lock);
@@ -363,7 +392,7 @@ EXPORT_SYMBOL(drm_sched_entity_set_priority);
  */
 static bool drm_sched_entity_add_dependency_cb(struct drm_sched_entity *entity)
 {
-	struct drm_gpu_scheduler *sched = entity->rq->sched;
+	struct drm_gpu_scheduler *sched = drm_sched_entity_to_scheduler(entity);
 	struct dma_fence *fence = entity->dependency;
 	struct drm_sched_fence *s_fence;
 
@@ -456,7 +485,8 @@ struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)
 	 * Update the entity's location in the min heap according to
 	 * the timestamp of the next job, if any.
 	 */
-	if (entity->rq->sched->sched_policy == DRM_SCHED_POLICY_FIFO) {
+	if (drm_sched_entity_to_scheduler(entity)->sched_policy ==
+	    DRM_SCHED_POLICY_FIFO) {
 		struct drm_sched_job *next;
 
 		next = to_drm_sched_job(spsc_queue_peek(&entity->job_queue));
@@ -473,6 +503,8 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
 	struct drm_gpu_scheduler *sched;
 	struct drm_sched_rq *rq;
 
+	WARN_ON(!!entity->single_sched);
+
 	/* single possible engine and already selected */
 	if (!entity->sched_list)
 		return;
@@ -522,16 +554,21 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
 void drm_sched_entity_push_job(struct drm_sched_job *sched_job)
 {
 	struct drm_sched_entity *entity = sched_job->entity;
+	bool single_entity = !!entity->single_sched;
 	bool first;
 
 	trace_drm_sched_job(sched_job, entity);
-	atomic_inc(entity->rq->sched->score);
+	if (!single_entity)
+		atomic_inc(entity->rq->sched->score);
 	WRITE_ONCE(entity->last_user, current->group_leader);
 	first = spsc_queue_push(&entity->job_queue, &sched_job->queue_node);
 	sched_job->submit_ts = ktime_get();
 
 	/* first job wakes up scheduler */
 	if (first) {
+		struct drm_gpu_scheduler *sched =
+			drm_sched_entity_to_scheduler(entity);
+
 		/* Add the entity to the run queue */
 		spin_lock(&entity->rq_lock);
 		if (entity->stopped) {
@@ -541,13 +578,14 @@ void drm_sched_entity_push_job(struct drm_sched_job *sched_job)
 			return;
 		}
 
-		drm_sched_rq_add_entity(entity->rq, entity);
+		if (!single_entity)
+			drm_sched_rq_add_entity(entity->rq, entity);
 		spin_unlock(&entity->rq_lock);
 
-		if (entity->rq->sched->sched_policy == DRM_SCHED_POLICY_FIFO)
+		if (sched->sched_policy == DRM_SCHED_POLICY_FIFO)
 			drm_sched_rq_update_fifo(entity, sched_job->submit_ts);
 
-		drm_sched_wakeup(entity->rq->sched);
+		drm_sched_wakeup(sched);
 	}
 }
 EXPORT_SYMBOL(drm_sched_entity_push_job);
drivers/gpu/drm/scheduler/sched_fence.c +1 -1
index 7fd869520ef2..1ba5056851dd 100644
@@ -167,7 +167,7 @@ void drm_sched_fence_init(struct drm_sched_fence *fence,
 {
 	unsigned seq;
 
-	fence->sched = entity->rq->sched;
+	fence->sched = drm_sched_entity_to_scheduler(entity);
 	seq = atomic_inc_return(&entity->fence_seq);
 	dma_fence_init(&fence->scheduled, &drm_sched_fence_ops_scheduled,
 		       &fence->lock, entity->fence_context, seq);
drivers/gpu/drm/scheduler/sched_main.c +55 -8

145 lines not on this page · raw message

include/drm/gpu_scheduler.h +8 -0
index 3df801401028..669d6520cd3a 100644
@@ -70,6 +70,7 @@ enum drm_sched_policy {
 	DRM_SCHED_POLICY_DEFAULT,
 	DRM_SCHED_POLICY_RR,
 	DRM_SCHED_POLICY_FIFO,
+	DRM_SCHED_POLICY_SINGLE_ENTITY,
 	DRM_SCHED_POLICY_COUNT,
 };
 
@@ -103,6 +104,9 @@ struct drm_sched_entity {
 	 */
 	struct drm_sched_rq		*rq;
 
+	/** @single_sched: Single scheduler */
+	struct drm_gpu_scheduler	*single_sched;
+
 	/**
 	 * @sched_list:
 	 *
@@ -488,6 +492,7 @@ struct drm_sched_backend_ops {
  * struct drm_gpu_scheduler - scheduler instance-specific data
  *
  * @ops: backend operations provided by the driver.
+ * @single_entity: Single entity for the scheduler
  * @hw_submission_limit: the max size of the hardware queue.
  * @timeout: the time after which a job is removed from the scheduler.
  * @name: name of the ring for which this scheduler is being used.
@@ -519,6 +524,7 @@ struct drm_sched_backend_ops {
  */
 struct drm_gpu_scheduler {
 	const struct drm_sched_backend_ops	*ops;
+	struct drm_sched_entity		*single_entity;
 	uint32_t			hw_submission_limit;
 	long				timeout;
 	const char			*name;
@@ -604,6 +610,8 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,
 			  struct drm_gpu_scheduler **sched_list,
 			  unsigned int num_sched_list,
 			  atomic_t *guilty);
+struct drm_gpu_scheduler *
+drm_sched_entity_to_scheduler(struct drm_sched_entity *entity);
 long drm_sched_entity_flush(struct drm_sched_entity *entity, long timeout);
 void drm_sched_entity_fini(struct drm_sched_entity *entity);
 void drm_sched_entity_destroy(struct drm_sched_entity *entity);
Signature · 1 line
2.34.1
author maintainer
#4 lore raw

PATCH 4/30 drm/xe: Use DRM_SCHED_POLICY_SINGLE_ENTITY mode

In reply to Matthew Brost

We create 1 GPU scheduler per entity in Xe, use DRM_SCHED_POLICY_SINGLE_ENTITY scheduling which is designed for that paradigm.

---
 drivers/gpu/drm/xe/xe_execlist.c   | 3 ++-
 drivers/gpu/drm/xe/xe_guc_submit.c | 3 +--
 2 files changed, 3 insertions(+), 3 deletions(-)
drivers/gpu/drm/xe/xe_execlist.c +2 -1
index 48060d14547a..79fb951c2965 100644
@@ -339,7 +339,8 @@ static int execlist_engine_init(struct xe_engine *e)
 	err = drm_sched_init(&exl->sched, &drm_sched_ops, NULL,
 			     e->lrc[0].ring.size / MAX_JOB_SIZE_BYTES,
 			     XE_SCHED_HANG_LIMIT, XE_SCHED_JOB_TIMEOUT,
-			     NULL, NULL, e->hwe->name, DRM_SCHED_POLICY_DEFAULT,
+			     NULL, NULL, e->hwe->name,
+			     DRM_SCHED_POLICY_SINGLE_ENTITY,
 			     gt_to_xe(e->gt)->drm.dev);
 	if (err)
 		goto err_free;
drivers/gpu/drm/xe/xe_guc_submit.c +1 -2
index 9d3fadca43be..68d09e7a4cc0 100644
@@ -1084,7 +1084,7 @@ static int guc_engine_init(struct xe_engine *e)
 	err = drm_sched_init(&ge->sched, &drm_sched_ops, NULL,
 			     e->lrc[0].ring.size / MAX_JOB_SIZE_BYTES,
 			     64, timeout, guc_to_gt(guc)->ordered_wq, NULL,
-			     e->name, DRM_SCHED_POLICY_DEFAULT,
+			     e->name, DRM_SCHED_POLICY_SINGLE_ENTITY,
 			     gt_to_xe(e->gt)->drm.dev);
 	if (err)
 		goto err_free;
@@ -1185,7 +1185,6 @@ static int guc_engine_set_priority(struct xe_engine *e,
 	if (!msg)
 		return -ENOMEM;
 
-	drm_sched_entity_set_priority(e->entity, priority);
 	guc_engine_add_msg(e, msg, SET_SCHED_PROPS);
 
 	return 0;
Signature · 1 line
2.34.1
author maintainer
#5 lore raw

PATCH 5/30 drm/xe: Long running job update

In reply to Matthew Brost

Flow control + write ring in exec, return NULL in run_job, siganl xe_hw_fence immediately, and override TDR for LR jobs.

---
 drivers/gpu/drm/xe/xe_engine.c           | 32 ++++++++
 drivers/gpu/drm/xe/xe_engine.h           |  4 +
 drivers/gpu/drm/xe/xe_exec.c             |  8 ++
 drivers/gpu/drm/xe/xe_guc_engine_types.h |  2 +
 drivers/gpu/drm/xe/xe_guc_submit.c       | 95 +++++++++++++++++++++---
 drivers/gpu/drm/xe/xe_trace.h            |  5 ++
 6 files changed, 137 insertions(+), 9 deletions(-)
drivers/gpu/drm/xe/xe_engine.c +32 -0
index 094ec17d3004..d1e84d7adbd4 100644
@@ -18,6 +18,7 @@
 #include "xe_macros.h"
 #include "xe_migrate.h"
 #include "xe_pm.h"
+#include "xe_ring_ops_types.h"
 #include "xe_trace.h"
 #include "xe_vm.h"
 
@@ -673,6 +674,37 @@ static void engine_kill_compute(struct xe_engine *e)
 	up_write(&e->vm->lock);
 }
 
+/**
+ * xe_engine_is_lr() - Whether an engine is long-running
+ * @e: The engine
+ *
+ * Return: True if the engine is long-running, false otherwise.
+ */
+bool xe_engine_is_lr(struct xe_engine *e)
+{
+	return e->vm && xe_vm_no_dma_fences(e->vm) &&
+		!(e->flags & ENGINE_FLAG_VM);
+}
+
+static s32 xe_engine_num_job_inflight(struct xe_engine *e)
+{
+	return e->lrc->fence_ctx.next_seqno - xe_lrc_seqno(e->lrc) - 1;
+}
+
+/**
+ * xe_engine_ring_full() - Whether an engine's ring is full
+ * @e: The engine
+ *
+ * Return: True if the engine's ring is full, false otherwise.
+ */
+bool xe_engine_ring_full(struct xe_engine *e)
+{
+	struct xe_lrc *lrc = e->lrc;
+	s32 max_job = lrc->ring.size / MAX_JOB_SIZE_BYTES;
+
+	return xe_engine_num_job_inflight(e) >= max_job;
+}
+
 /**
  * xe_engine_is_idle() - Whether an engine is idle.
  * @engine: The engine
drivers/gpu/drm/xe/xe_engine.h +4 -0
index a49cf2ab405e..2e60f6d90226 100644
@@ -42,6 +42,10 @@ static inline bool xe_engine_is_parallel(struct xe_engine *engine)
 	return engine->width > 1;
 }
 
+bool xe_engine_is_lr(struct xe_engine *e);
+
+bool xe_engine_ring_full(struct xe_engine *e);
+
 bool xe_engine_is_idle(struct xe_engine *engine);
 
 void xe_engine_kill(struct xe_engine *e);
drivers/gpu/drm/xe/xe_exec.c +8 -0
index ea869f2452ef..44ea9bcd0066 100644
@@ -13,6 +13,7 @@
 #include "xe_device.h"
 #include "xe_engine.h"
 #include "xe_macros.h"
+#include "xe_ring_ops_types.h"
 #include "xe_sched_job.h"
 #include "xe_sync.h"
 #include "xe_vm.h"
@@ -277,6 +278,11 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
 		goto err_engine_end;
 	}
 
+	if (xe_engine_is_lr(engine) && xe_engine_ring_full(engine)) {
+		err = -EWOULDBLOCK;
+		goto err_engine_end;
+	}
+
 	job = xe_sched_job_create(engine, xe_engine_is_parallel(engine) ?
 				  addresses : &args->address);
 	if (IS_ERR(job)) {
@@ -363,6 +369,8 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
 		xe_sync_entry_signal(&syncs[i], job,
 				     &job->drm.s_fence->finished);
 
+	if (xe_engine_is_lr(engine))
+		engine->ring_ops->emit_job(job);
 	xe_sched_job_push(job);
 	xe_vm_reactivate_rebind(vm);
 
drivers/gpu/drm/xe/xe_guc_engine_types.h +2 -0
index cbfb13026ec1..5d83132034a6 100644
@@ -31,6 +31,8 @@ struct xe_guc_engine {
 	 */
 #define MAX_STATIC_MSG_TYPE	3
 	struct drm_sched_msg static_msgs[MAX_STATIC_MSG_TYPE];
+	/** @lr_tdr: long running TDR worker */
+	struct work_struct lr_tdr;
 	/** @fini_async: do final fini async from this worker */
 	struct work_struct fini_async;
 	/** @resume_time: time of last resume */
drivers/gpu/drm/xe/xe_guc_submit.c +86 -9

190 lines not on this page · raw message

drivers/gpu/drm/xe/xe_trace.h +5 -0
index 2f8eb7ebe9a7..02861c26e145 100644
@@ -219,6 +219,11 @@ DEFINE_EVENT(xe_engine, xe_engine_resubmit,
 	     TP_ARGS(e)
 );
 
+DEFINE_EVENT(xe_engine, xe_engine_lr_cleanup,
+	     TP_PROTO(struct xe_engine *e),
+	     TP_ARGS(e)
+);
+
 DECLARE_EVENT_CLASS(xe_sched_job,
 		    TP_PROTO(struct xe_sched_job *job),
 		    TP_ARGS(job),
Signature · 1 line
2.34.1
author maintainer
#6 lore raw

PATCH 6/30 drm/xe: Ensure LR engines are not persistent

In reply to Matthew Brost

With our ref counting scheme LR engines only close properly if not persistent, ensure that LR engines are non-persistent.

---
 drivers/gpu/drm/xe/xe_engine.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_engine.c +3 -1
index d1e84d7adbd4..91600b1e8249 100644
@@ -596,7 +596,9 @@ int xe_engine_create_ioctl(struct drm_device *dev, void *data,
 			return -ENOENT;
 
 		e = xe_engine_create(xe, vm, logical_mask,
-				     args->width, hwe, ENGINE_FLAG_PERSISTENT);
+				     args->width, hwe,
+				     xe_vm_no_dma_fences(vm) ? 0 :
+				     ENGINE_FLAG_PERSISTENT);
 		xe_vm_put(vm);
 		if (IS_ERR(e))
 			return PTR_ERR(e);
Signature · 1 line
2.34.1
author maintainer
#7 lore raw

PATCH 7/30 drm/xe: Only try to lock external BOs in VM bind

In reply to Matthew Brost

Not needed and causes some issues with bulk LRU moves.

---
 drivers/gpu/drm/xe/xe_vm.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
drivers/gpu/drm/xe/xe_vm.c +5 -3
index 272f0f7f24fe..6c427ff92c44 100644
@@ -2064,9 +2064,11 @@ static int vm_bind_ioctl(struct xe_vm *vm, struct xe_vma *vma,
 		 */
 		xe_bo_get(vbo);
 
-		tv_bo.bo = &vbo->ttm;
-		tv_bo.num_shared = 1;
-		list_add(&tv_bo.head, &objs);
+		if (!vbo->vm) {
+			tv_bo.bo = &vbo->ttm;
+			tv_bo.num_shared = 1;
+			list_add(&tv_bo.head, &objs);
+		}
 	}
 
 again:
Signature · 1 line
2.34.1
author maintainer
#8 lore raw

PATCH 8/30 drm/xe: VM LRU bulk move

In reply to Matthew Brost

Use the TTM LRU bulk move for BOs tied to a VM. Update the bulk moves LRU position on every exec.

---
 drivers/gpu/drm/xe/xe_bo.c       | 32 ++++++++++++++++++++++++++++----
 drivers/gpu/drm/xe/xe_bo.h       |  4 ++--
 drivers/gpu/drm/xe/xe_dma_buf.c  |  2 +-
 drivers/gpu/drm/xe/xe_exec.c     |  6 ++++++
 drivers/gpu/drm/xe/xe_vm_types.h |  3 +++
 5 files changed, 40 insertions(+), 7 deletions(-)
drivers/gpu/drm/xe/xe_bo.c +28 -4
index 3ab404e33fae..da99ee53e7d7 100644
@@ -985,6 +985,23 @@ static void xe_gem_object_free(struct drm_gem_object *obj)
 	ttm_bo_put(container_of(obj, struct ttm_buffer_object, base));
 }
 
+static void xe_gem_object_close(struct drm_gem_object *obj,
+				struct drm_file *file_priv)
+{
+	struct xe_bo *bo = gem_to_xe_bo(obj);
+
+	if (bo->vm && !xe_vm_no_dma_fences(bo->vm)) {
+		struct ww_acquire_ctx ww;
+
+		XE_BUG_ON(!xe_bo_is_user(bo));
+
+		xe_bo_lock(bo, &ww, 0, false);
+		ttm_bo_set_bulk_move(&bo->ttm, NULL);
+		xe_bo_unlock(bo, &ww);
+	}
+}
+
+
 static bool should_migrate_to_system(struct xe_bo *bo)
 {
 	struct xe_device *xe = xe_bo_device(bo);
@@ -1040,6 +1057,7 @@ static const struct vm_operations_struct xe_gem_vm_ops = {
 
 static const struct drm_gem_object_funcs xe_gem_object_funcs = {
 	.free = xe_gem_object_free,
+	.close = xe_gem_object_close,
 	.mmap = drm_gem_ttm_mmap,
 	.export = xe_gem_prime_export,
 	.vm_ops = &xe_gem_vm_ops,
@@ -1081,8 +1099,8 @@ void xe_bo_free(struct xe_bo *bo)
 
 struct xe_bo *__xe_bo_create_locked(struct xe_device *xe, struct xe_bo *bo,
 				    struct xe_gt *gt, struct dma_resv *resv,
-				    size_t size, enum ttm_bo_type type,
-				    u32 flags)
+				    struct ttm_lru_bulk_move *bulk, size_t size,
+				    enum ttm_bo_type type, u32 flags)
 {
 	struct ttm_operation_ctx ctx = {
 		.interruptible = true,
@@ -1149,7 +1167,10 @@ struct xe_bo *__xe_bo_create_locked(struct xe_device *xe, struct xe_bo *bo,
 		return ERR_PTR(err);
 
 	bo->created = true;
-	ttm_bo_move_to_lru_tail_unlocked(&bo->ttm);
+	if (bulk)
+		ttm_bo_set_bulk_move(&bo->ttm, bulk);
+	else
+		ttm_bo_move_to_lru_tail_unlocked(&bo->ttm);
 
 	return bo;
 }
@@ -1219,7 +1240,10 @@ xe_bo_create_locked_range(struct xe_device *xe,
 		}
 	}
 
-	bo = __xe_bo_create_locked(xe, bo, gt, vm ? &vm->resv : NULL, size,
+	bo = __xe_bo_create_locked(xe, bo, gt, vm ? &vm->resv : NULL,
+				   vm && !xe_vm_no_dma_fences(vm) &&
+				   flags & XE_BO_CREATE_USER_BIT ?
+				   &vm->lru_bulk_move : NULL, size,
 				   type, flags);
 	if (IS_ERR(bo))
 		return bo;
drivers/gpu/drm/xe/xe_bo.h +2 -2
index 8354d05ccdf3..25457b3c757b 100644
@@ -81,8 +81,8 @@ void xe_bo_free(struct xe_bo *bo);
 
 struct xe_bo *__xe_bo_create_locked(struct xe_device *xe, struct xe_bo *bo,
 				    struct xe_gt *gt, struct dma_resv *resv,
-				    size_t size, enum ttm_bo_type type,
-				    u32 flags);
+				    struct ttm_lru_bulk_move *bulk, size_t size,
+				    enum ttm_bo_type type, u32 flags);
 struct xe_bo *
 xe_bo_create_locked_range(struct xe_device *xe,
 			  struct xe_gt *gt, struct xe_vm *vm,
drivers/gpu/drm/xe/xe_dma_buf.c +1 -1
index 9b252cc782b7..975dee1f770f 100644
@@ -199,7 +199,7 @@ xe_dma_buf_init_obj(struct drm_device *dev, struct xe_bo *storage,
 	int ret;
 
 	dma_resv_lock(resv, NULL);
-	bo = __xe_bo_create_locked(xe, storage, NULL, resv, dma_buf->size,
+	bo = __xe_bo_create_locked(xe, storage, NULL, resv, NULL, dma_buf->size,
 				   ttm_bo_type_sg, XE_BO_CREATE_SYSTEM_BIT);
 	if (IS_ERR(bo)) {
 		ret = PTR_ERR(bo);
drivers/gpu/drm/xe/xe_exec.c +6 -0
index 44ea9bcd0066..21a9c2fddf86 100644
@@ -374,6 +374,12 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
 	xe_sched_job_push(job);
 	xe_vm_reactivate_rebind(vm);
 
+	if (!err && !xe_vm_no_dma_fences(vm)) {
+		spin_lock(&xe->ttm.lru_lock);
+		ttm_lru_bulk_move_tail(&vm->lru_bulk_move);
+		spin_unlock(&xe->ttm.lru_lock);
+	}
+
 err_repin:
 	if (!xe_vm_no_dma_fences(vm))
 		up_read(&vm->userptr.notifier_lock);
drivers/gpu/drm/xe/xe_vm_types.h +3 -0
index fada7896867f..d3e99f22510d 100644
@@ -164,6 +164,9 @@ struct xe_vm {
 	/** Protects @rebind_list and the page-table structures */
 	struct dma_resv resv;
 
+	/** @lru_bulk_move: Bulk LRU move list for this VM's BOs */
+	struct ttm_lru_bulk_move lru_bulk_move;
+
 	u64 size;
 	struct rb_root vmas;
 
Signature · 1 line
2.34.1
author maintainer
#9 lore raw

PATCH 9/30 drm/xe/guc: Read HXG fields from DW1 of G2H response

In reply to Matthew Brost

The HXG fields are DW1 not DW0, fix this.

---
 drivers/gpu/drm/xe/xe_guc_ct.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/gpu/drm/xe/xe_guc_ct.c +3 -3
index 9055ff133a7c..6abf1dee95af 100644
@@ -782,13 +782,13 @@ static int parse_g2h_response(struct xe_guc_ct *ct, u32 *msg, u32 len)
 	if (type == GUC_HXG_TYPE_RESPONSE_FAILURE) {
 		g2h_fence->fail = true;
 		g2h_fence->error =
-			FIELD_GET(GUC_HXG_FAILURE_MSG_0_ERROR, msg[0]);
+			FIELD_GET(GUC_HXG_FAILURE_MSG_0_ERROR, msg[1]);
 		g2h_fence->hint =
-			FIELD_GET(GUC_HXG_FAILURE_MSG_0_HINT, msg[0]);
+			FIELD_GET(GUC_HXG_FAILURE_MSG_0_HINT, msg[1]);
 	} else if (type == GUC_HXG_TYPE_NO_RESPONSE_RETRY) {
 		g2h_fence->retry = true;
 		g2h_fence->reason =
-			FIELD_GET(GUC_HXG_RETRY_MSG_0_REASON, msg[0]);
+			FIELD_GET(GUC_HXG_RETRY_MSG_0_REASON, msg[1]);
 	} else if (g2h_fence->response_buffer) {
 		g2h_fence->response_len = response_len;
 		memcpy(g2h_fence->response_buffer, msg + GUC_CTB_MSG_MIN_LEN,
Signature · 1 line
2.34.1
author maintainer
#10 lore raw

PATCH 10/30 drm/xe/guc: Return the lower part of blocking H2G message

In reply to Matthew Brost

The upper layers may need this data, an example of this is allocating DIST doorbell.

---
 drivers/gpu/drm/xe/xe_guc_ct.c | 6 +++++-
 drivers/gpu/drm/xe/xe_guc_pc.c | 6 ++++--
 drivers/gpu/drm/xe/xe_huc.c    | 2 +-
 3 files changed, 10 insertions(+), 4 deletions(-)
drivers/gpu/drm/xe/xe_guc_ct.c +5 -1
index 6abf1dee95af..60b69fcfac9f 100644
@@ -25,6 +25,7 @@
 struct g2h_fence {
 	u32 *response_buffer;
 	u32 seqno;
+	u32 status;
 	u16 response_len;
 	u16 error;
 	u16 hint;
@@ -727,7 +728,7 @@ static int guc_ct_send_recv(struct xe_guc_ct *ct, const u32 *action, u32 len,
 		ret = -EIO;
 	}
 
-	return ret > 0 ? 0 : ret;
+	return ret > 0 ? g2h_fence.status : ret;
 }
 
 int xe_guc_ct_send_recv(struct xe_guc_ct *ct, const u32 *action, u32 len,
@@ -793,6 +794,9 @@ static int parse_g2h_response(struct xe_guc_ct *ct, u32 *msg, u32 len)
 		g2h_fence->response_len = response_len;
 		memcpy(g2h_fence->response_buffer, msg + GUC_CTB_MSG_MIN_LEN,
 		       response_len * sizeof(u32));
+	} else {
+		g2h_fence->status =
+			FIELD_GET(GUC_HXG_RESPONSE_MSG_0_DATA0, msg[1]);
 	}
 
 	g2h_release_space(ct, GUC_CTB_HXG_MSG_MAX_LEN);
drivers/gpu/drm/xe/xe_guc_pc.c +4 -2
index 72d460d5323b..3d2ea723a4a7 100644
@@ -204,11 +204,13 @@ static int pc_action_query_task_state(struct xe_guc_pc *pc)
 
 	/* Blocking here to ensure the results are ready before reading them */
 	ret = xe_guc_ct_send_block(ct, action, ARRAY_SIZE(action));
-	if (ret)
+	if (ret < 0) {
 		drm_err(&pc_to_xe(pc)->drm,
 			"GuC PC query task state failed: %pe", ERR_PTR(ret));
+		return ret;
+	}
 
-	return ret;
+	return 0;
 }
 
 static int pc_action_set_param(struct xe_guc_pc *pc, u8 id, u32 value)
drivers/gpu/drm/xe/xe_huc.c +1 -1
index 55dcaab34ea4..9c48c3075410 100644
@@ -39,7 +39,7 @@ int xe_huc_init(struct xe_huc *huc)
 
 	huc->fw.type = XE_UC_FW_TYPE_HUC;
 	ret = xe_uc_fw_init(&huc->fw);
-	if (ret)
+	if (ret < 0)
 		goto out;
 
 	xe_uc_fw_change_status(&huc->fw, XE_UC_FIRMWARE_LOADABLE);
Signature · 1 line
2.34.1
author maintainer
#11 lore raw

PATCH 11/30 drm/xe/guc: Use doorbells for submission if possible

In reply to Matthew Brost

We have 256 doorbells (on most platforms) that we can allocate to bypass using the H2G channel for submission. This will avoid contention on the CT mutex.

---
 drivers/gpu/drm/xe/regs/xe_guc_regs.h    |   1 +
 drivers/gpu/drm/xe/xe_guc.c              |   6 +
 drivers/gpu/drm/xe/xe_guc_engine_types.h |   7 +
 drivers/gpu/drm/xe/xe_guc_submit.c       | 295 ++++++++++++++++++++++-
 drivers/gpu/drm/xe/xe_guc_submit.h       |   1 +
 drivers/gpu/drm/xe/xe_guc_types.h        |   4 +
 drivers/gpu/drm/xe/xe_trace.h            |   5 +
 7 files changed, 315 insertions(+), 4 deletions(-)
drivers/gpu/drm/xe/regs/xe_guc_regs.h +1 -0
index 37e0ac550931..11b117293a62 100644
@@ -109,6 +109,7 @@ struct guc_doorbell_info {
 
 #define DIST_DBS_POPULATED			XE_REG(0xd08)
 #define   DOORBELLS_PER_SQIDI_MASK		REG_GENMASK(23, 16)
+#define	  DOORBELLS_PER_SQIDI_SHIFT		16
 #define   SQIDIS_DOORBELL_EXIST_MASK		REG_GENMASK(15, 0)
 
 #define GUC_BCS_RCS_IER				XE_REG(0xC550)
drivers/gpu/drm/xe/xe_guc.c +6 -0
index 89d20faced19..0c87f78a868b 100644
@@ -297,6 +297,12 @@ int xe_guc_init(struct xe_guc *guc)
  */
 int xe_guc_init_post_hwconfig(struct xe_guc *guc)
 {
+	int ret;
+
+	ret = xe_guc_submit_init_post_hwconfig(guc);
+	if (ret)
+		return ret;
+
 	return xe_guc_ads_init_post_hwconfig(&guc->ads);
 }
 
drivers/gpu/drm/xe/xe_guc_engine_types.h +7 -0
index 5d83132034a6..420b7f53e649 100644
@@ -12,6 +12,7 @@
 #include <drm/gpu_scheduler.h>
 
 struct dma_fence;
+struct xe_bo;
 struct xe_engine;
 
 /**
@@ -37,6 +38,10 @@ struct xe_guc_engine {
 	struct work_struct fini_async;
 	/** @resume_time: time of last resume */
 	u64 resume_time;
+	/** @doorbell_bo: BO for memory doorbell */
+	struct xe_bo *doorbell_bo;
+	/** @doorbell_offset: MMIO doorbell offset */
+	u32 doorbell_offset;
 	/** @state: GuC specific state for this xe_engine */
 	atomic_t state;
 	/** @wqi_head: work queue item tail */
@@ -45,6 +50,8 @@ struct xe_guc_engine {
 	u32 wqi_tail;
 	/** @id: GuC id for this xe_engine */
 	u16 id;
+	/** @doorbell_id: doorbell id */
+	u16 doorbell_id;
 	/** @suspend_wait: wait queue used to wait on pending suspends */
 	wait_queue_head_t suspend_wait;
 	/** @suspend_pending: a suspend of the engine is pending */
drivers/gpu/drm/xe/xe_guc_submit.c +291 -4

400 lines not on this page · raw message

drivers/gpu/drm/xe/xe_guc_submit.h +1 -0
index 8002734d6f24..bada6c02d6aa 100644
@@ -13,6 +13,7 @@ struct xe_engine;
 struct xe_guc;
 
 int xe_guc_submit_init(struct xe_guc *guc);
+int xe_guc_submit_init_post_hwconfig(struct xe_guc *guc);
 void xe_guc_submit_print(struct xe_guc *guc, struct drm_printer *p);
 
 int xe_guc_submit_reset_prepare(struct xe_guc *guc);
drivers/gpu/drm/xe/xe_guc_types.h +4 -0
index ac7eec28934d..9ee4d572f4e0 100644
@@ -36,10 +36,14 @@ struct xe_guc {
 		struct xarray engine_lookup;
 		/** @guc_ids: used to allocate new guc_ids, single-lrc */
 		struct ida guc_ids;
+		/** @doorbell_ids: use to allocate new doorbells */
+		struct ida doorbell_ids;
 		/** @guc_ids_bitmap: used to allocate new guc_ids, multi-lrc */
 		unsigned long *guc_ids_bitmap;
 		/** @stopped: submissions are stopped */
 		atomic_t stopped;
+		/** @num_doorbells: number of doorbels */
+		int num_doorbells;
 		/** @lock: protects submission state */
 		struct mutex lock;
 		/** @suspend: suspend fence state */
drivers/gpu/drm/xe/xe_trace.h +5 -0
index 02861c26e145..38e9d7c6197b 100644
@@ -149,6 +149,11 @@ DEFINE_EVENT(xe_engine, xe_engine_submit,
 	     TP_ARGS(e)
 );
 
+DEFINE_EVENT(xe_engine, xe_engine_ring_db,
+	     TP_PROTO(struct xe_engine *e),
+	     TP_ARGS(e)
+);
+
 DEFINE_EVENT(xe_engine, xe_engine_scheduling_enable,
 	     TP_PROTO(struct xe_engine *e),
 	     TP_ARGS(e)
Signature · 1 line
2.34.1
author maintainer
#12 lore raw

PATCH 12/30 drm/xe/guc: Print doorbell ID in GuC engine debugfs entry

In reply to Matthew Brost

This information is helpful so print it.

---
 drivers/gpu/drm/xe/xe_guc_submit.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/gpu/drm/xe/xe_guc_submit.c +2 -0
index 1b6f36b04cd1..880f480c6d5f 100644
@@ -2016,6 +2016,8 @@ static void guc_engine_print(struct xe_engine *e, struct drm_printer *p)
 	drm_printf(p, "\tTimeslice: %u (us)\n", e->sched_props.timeslice_us);
 	drm_printf(p, "\tPreempt timeout: %u (us)\n",
 		   e->sched_props.preempt_timeout_us);
+	drm_printf(p, "\tDoorbell ID: %u\n",
+		   e->guc->doorbell_id);
 	for (i = 0; i < e->width; ++i ) {
 		struct xe_lrc *lrc = e->lrc + i;
 
Signature · 1 line
2.34.1
author maintainer
#13 lore raw

PATCH 13/30 maple_tree: split up MA_STATE() macro

In reply to Matthew Brost

From: Danilo Krummrich <[email protected]>

Split up the MA_STATE() macro such that components using the maple tree can easily inherit from struct ma_state and build custom tree walk macros to hide their internals from users.

Example:

struct sample_iterator {
        struct ma_state mas;
        struct sample_mgr *mgr;
};
\#define SAMPLE_ITERATOR(name, __mgr, start)                    \
        struct sample_iterator name = {                         \
                .mas = MA_STATE_INIT(&(__mgr)->mt, start, 0),   \
                .mgr = __mgr,                                   \
        }
\#define sample_iter_for_each_range(it__, entry__, end__) \
        mas_for_each(&(it__).mas, entry__, end__)

--

struct sample *sample;
SAMPLE_ITERATOR(si, min);

sample_iter_for_each_range(&si, sample, max) {
        frob(mgr, sample);
}
---
 include/linux/maple_tree.h | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
include/linux/maple_tree.h +5 -2
index 1fadb5f5978b..87d55334f1c2 100644
@@ -423,8 +423,8 @@ struct ma_wr_state {
 #define MA_ERROR(err) \
 		((struct maple_enode *)(((unsigned long)err << 2) | 2UL))
 
-#define MA_STATE(name, mt, first, end)					\
-	struct ma_state name = {					\
+#define MA_STATE_INIT(mt, first, end)					\
+	{								\
 		.tree = mt,						\
 		.index = first,						\
 		.last = end,						\
@@ -435,6 +435,9 @@ struct ma_wr_state {
 		.mas_flags = 0,						\
 	}
 
+#define MA_STATE(name, mt, first, end)					\
+	struct ma_state name = MA_STATE_INIT(mt, first, end)
+
 #define MA_WR_STATE(name, ma_state, wr_entry)				\
 	struct ma_wr_state name = {					\
 		.mas = ma_state,					\
Signature · 1 line
2.34.1
author maintainer
#14 lore raw

PATCH 14/30 maple_tree: Export mas_preallocate

In reply to Matthew Brost

The DRM GPUVA implementation needs this function.

---
 lib/maple_tree.c | 1 +
 1 file changed, 1 insertion(+)
lib/maple_tree.c +1 -0
index 9e2735cbc2b4..ae37a167e25d 100644
@@ -5726,6 +5726,7 @@ int mas_preallocate(struct ma_state *mas, gfp_t gfp)
 	mas_reset(mas);
 	return ret;
 }
+EXPORT_SYMBOL_GPL(mas_preallocate);
 
 /*
  * mas_destroy() - destroy a maple state.
Signature · 1 line
2.34.1
author maintainer
#15 lore raw

PATCH 15/30 drm: manager to keep track of GPUs VA mappings

In reply to Matthew Brost

From: Danilo Krummrich <[email protected]>

Add infrastructure to keep track of GPU virtual address (VA) mappings with a decicated VA space manager implementation.

New UAPIs, motivated by Vulkan sparse memory bindings graphics drivers start implementing, allow userspace applications to request multiple and arbitrary GPU VA mappings of buffer objects. The DRM GPU VA manager is intended to serve the following purposes in this context.

1) Provide infrastructure to track GPU VA allocations and mappings, making use of the maple_tree.

2) Generically connect GPU VA mappings to their backing buffers, in particular DRM GEM objects.

3) Provide a common implementation to perform more complex mapping operations on the GPU VA space. In particular splitting and merging of GPU VA mappings, e.g. for intersecting mapping requests or partial unmap requests.

---
 Documentation/gpu/drm-mm.rst    |   31 +
 drivers/gpu/drm/Makefile        |    1 +
 drivers/gpu/drm/drm_debugfs.c   |   41 +
 drivers/gpu/drm/drm_gem.c       |    3 +
 drivers/gpu/drm/drm_gpuva_mgr.c | 1686 +++++++++++++++++++++++++++++++
 include/drm/drm_debugfs.h       |   24 +
 include/drm/drm_drv.h           |    7 +
 include/drm/drm_gem.h           |   75 ++
 include/drm/drm_gpuva_mgr.h     |  681 +++++++++++++
 9 files changed, 2549 insertions(+)
 create mode 100644 drivers/gpu/drm/drm_gpuva_mgr.c
 create mode 100644 include/drm/drm_gpuva_mgr.h
Documentation/gpu/drm-mm.rst +31 -0
index a79fd3549ff8..fe40ee686f6e 100644
@@ -466,6 +466,37 @@ DRM MM Range Allocator Function References
 .. kernel-doc:: drivers/gpu/drm/drm_mm.c
    :export:
 
+DRM GPU VA Manager
+==================
+
+Overview
+--------
+
+.. kernel-doc:: drivers/gpu/drm/drm_gpuva_mgr.c
+   :doc: Overview
+
+Split and Merge
+---------------
+
+.. kernel-doc:: drivers/gpu/drm/drm_gpuva_mgr.c
+   :doc: Split and Merge
+
+Locking
+-------
+
+.. kernel-doc:: drivers/gpu/drm/drm_gpuva_mgr.c
+   :doc: Locking
+
+
+DRM GPU VA Manager Function References
+--------------------------------------
+
+.. kernel-doc:: include/drm/drm_gpuva_mgr.h
+   :internal:
+
+.. kernel-doc:: drivers/gpu/drm/drm_gpuva_mgr.c
+   :export:
+
 DRM Buddy Allocator
 ===================
 
drivers/gpu/drm/Makefile +1 -0
index 66dd2c48944a..ad6267273503 100644
@@ -46,6 +46,7 @@ drm-y := \
 	drm_vblank.o \
 	drm_vblank_work.o \
 	drm_vma_manager.o \
+	drm_gpuva_mgr.o \
 	drm_writeback.o
 drm-$(CONFIG_DRM_LEGACY) += \
 	drm_agpsupport.o \
drivers/gpu/drm/drm_debugfs.c +41 -0

53 lines not on this page · raw message

drivers/gpu/drm/drm_gem.c +3 -0
index a6208e2c089b..15fe61856190 100644
@@ -164,6 +164,9 @@ void drm_gem_private_object_init(struct drm_device *dev,
 	if (!obj->resv)
 		obj->resv = &obj->_resv;
 
+	if (drm_core_check_feature(dev, DRIVER_GEM_GPUVA))
+		drm_gem_gpuva_init(obj);
+
 	drm_vma_node_reset(&obj->vma_node);
 	INIT_LIST_HEAD(&obj->lru_node);
 }
drivers/gpu/drm/drm_gpuva_mgr.c new file +1686 -0

1,686 lines not on this page · raw message

include/drm/drm_debugfs.h +24 -0
index 7616f457ce70..3031fcb96b39 100644
@@ -34,6 +34,22 @@
 
 #include <linux/types.h>
 #include <linux/seq_file.h>
+
+#include <drm/drm_gpuva_mgr.h>
+
+/**
+ * DRM_DEBUGFS_GPUVA_INFO - &drm_info_list entry to dump a GPU VA space
+ * @show: the &drm_info_list's show callback
+ * @data: driver private data
+ *
+ * Drivers should use this macro to define a &drm_info_list entry to provide a
+ * debugfs file for dumping the GPU VA space regions and mappings.
+ *
+ * For each DRM GPU VA space drivers should call drm_debugfs_gpuva_info() from
+ * their @show callback.
+ */
+#define DRM_DEBUGFS_GPUVA_INFO(show, data) {"gpuvas", show, DRIVER_GEM_GPUVA, data}
+
 /**
  * struct drm_info_list - debugfs info list entry
  *
@@ -134,6 +150,8 @@ void drm_debugfs_add_file(struct drm_device *dev, const char *name,
 
 void drm_debugfs_add_files(struct drm_device *dev,
 			   const struct drm_debugfs_info *files, int count);
+int drm_debugfs_gpuva_info(struct seq_file *m,
+			   struct drm_gpuva_manager *mgr);
 #else
 static inline void drm_debugfs_create_files(const struct drm_info_list *files,
 					    int count, struct dentry *root,
@@ -155,6 +173,12 @@ static inline void drm_debugfs_add_files(struct drm_device *dev,
 					 const struct drm_debugfs_info *files,
 					 int count)
 {}
+
+static inline int drm_debugfs_gpuva_info(struct seq_file *m,
+					 struct drm_gpuva_manager *mgr)
+{
+	return 0;
+}
 #endif
 
 #endif /* _DRM_DEBUGFS_H_ */
include/drm/drm_drv.h +7 -0
index b419c59c4bef..9e2ec2da0685 100644
@@ -105,6 +105,13 @@ enum drm_driver_feature {
 	 */
 	DRIVER_COMPUTE_ACCEL            = BIT(7),
 
+	/**
+	 * @DRIVER_GEM_GPUVA:
+	 *
+	 * Driver supports user defined GPU VA bindings for GEM objects.
+	 */
+	DRIVER_GEM_GPUVA		= BIT(8),
+
 	/* IMPORTANT: Below are all the legacy flags, add new ones above. */
 
 	/**
include/drm/drm_gem.h +75 -0

91 lines not on this page · raw message

include/drm/drm_gpuva_mgr.h new file +681 -0

681 lines not on this page · raw message

Signature · 1 line
2.34.1
author maintainer
#16 lore raw

PATCH 16/30 drm/xe: Port Xe to GPUVA

In reply to Matthew Brost

Rather than open coding VM binds and VMA tracking, use the GPUVA library. GPUVA provides a common infrastructure for VM binds to use mmap / munmap semantics and support for VK sparse bindings.

The concepts are:

1) xe_vm inherits from drm_gpuva_manager
2) xe_vma inherits from drm_gpuva
3) xe_vma_op inherits from drm_gpuva_op
4) VM bind operations (MAP, UNMAP, PREFETCH, UNMAP_ALL) call into the GPUVA code to generate an VMA operations list which is parsed, commited, and executed.

v2 (CI): Add break after default in case statement.
v3: Rebase

---
 drivers/gpu/drm/xe/xe_bo.c                  |   10 +-
 drivers/gpu/drm/xe/xe_device.c              |    2 +-
 drivers/gpu/drm/xe/xe_exec.c                |    2 +-
 drivers/gpu/drm/xe/xe_gt_pagefault.c        |   23 +-
 drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c |   14 +-
 drivers/gpu/drm/xe/xe_migrate.c             |    8 +-
 drivers/gpu/drm/xe/xe_pt.c                  |  106 +-
 drivers/gpu/drm/xe/xe_trace.h               |   10 +-
 drivers/gpu/drm/xe/xe_vm.c                  | 1847 +++++++++----------
 drivers/gpu/drm/xe/xe_vm.h                  |   66 +-
 drivers/gpu/drm/xe/xe_vm_madvise.c          |   88 +-
 drivers/gpu/drm/xe/xe_vm_types.h            |  167 +-
 12 files changed, 1168 insertions(+), 1175 deletions(-)
drivers/gpu/drm/xe/xe_bo.c +5 -5
index da99ee53e7d7..a475d0584916 100644
@@ -391,7 +391,8 @@ static int xe_bo_trigger_rebind(struct xe_device *xe, struct xe_bo *bo,
 {
 	struct dma_resv_iter cursor;
 	struct dma_fence *fence;
-	struct xe_vma *vma;
+	struct drm_gpuva *gpuva;
+	struct drm_gem_object *obj = &bo->ttm.base;
 	int ret = 0;
 
 	dma_resv_assert_held(bo->ttm.base.resv);
@@ -404,8 +405,9 @@ static int xe_bo_trigger_rebind(struct xe_device *xe, struct xe_bo *bo,
 		dma_resv_iter_end(&cursor);
 	}
 
-	list_for_each_entry(vma, &bo->vmas, bo_link) {
-		struct xe_vm *vm = vma->vm;
+	drm_gem_for_each_gpuva(gpuva, obj) {
+		struct xe_vma *vma = gpuva_to_vma(gpuva);
+		struct xe_vm *vm = xe_vma_vm(vma);
 
 		trace_xe_vma_evict(vma);
 
@@ -430,10 +432,8 @@ static int xe_bo_trigger_rebind(struct xe_device *xe, struct xe_bo *bo,
 			} else {
 				ret = timeout;
 			}
-
 		} else {
 			bool vm_resv_locked = false;
-			struct xe_vm *vm = vma->vm;
 
 			/*
 			 * We need to put the vma on the vm's rebind_list,
drivers/gpu/drm/xe/xe_device.c +1 -1
index 45d6e5ff47fd..ab2ecd208f97 100644
@@ -131,7 +131,7 @@ static struct drm_driver driver = {
 	.driver_features =
 	    DRIVER_GEM |
 	    DRIVER_RENDER | DRIVER_SYNCOBJ |
-	    DRIVER_SYNCOBJ_TIMELINE,
+	    DRIVER_SYNCOBJ_TIMELINE | DRIVER_GEM_GPUVA,
 	.open = xe_file_open,
 	.postclose = xe_file_close,
 
drivers/gpu/drm/xe/xe_exec.c +1 -1
index 21a9c2fddf86..90c46d092737 100644
@@ -119,7 +119,7 @@ static int xe_exec_begin(struct xe_engine *e, struct ww_acquire_ctx *ww,
 		if (xe_vma_is_userptr(vma))
 			continue;
 
-		err = xe_bo_validate(vma->bo, vm, false);
+		err = xe_bo_validate(xe_vma_bo(vma), vm, false);
 		if (err) {
 			xe_vm_unlock_dma_resv(vm, tv_onstack, *tv, ww, objs);
 			*tv = NULL;
drivers/gpu/drm/xe/xe_gt_pagefault.c +9 -14
index 1677640e1075..f7a066090a13 100644
@@ -75,9 +75,10 @@ static bool vma_is_valid(struct xe_gt *gt, struct xe_vma *vma)
 		!(BIT(gt->info.id) & vma->usm.gt_invalidated);
 }
 
-static bool vma_matches(struct xe_vma *vma, struct xe_vma *lookup)
+static bool vma_matches(struct xe_vma *vma, u64 page_addr)
 {
-	if (lookup->start > vma->end || lookup->end < vma->start)
+	if (page_addr > xe_vma_end(vma) - 1 ||
+	    page_addr + SZ_4K < xe_vma_start(vma))
 		return false;
 
 	return true;
@@ -90,16 +91,14 @@ static bool only_needs_bo_lock(struct xe_bo *bo)
 
 static struct xe_vma *lookup_vma(struct xe_vm *vm, u64 page_addr)
 {
-	struct xe_vma *vma = NULL, lookup;
+	struct xe_vma *vma = NULL;
 
-	lookup.start = page_addr;
-	lookup.end = lookup.start + SZ_4K - 1;
 	if (vm->usm.last_fault_vma) {   /* Fast lookup */
-		if (vma_matches(vm->usm.last_fault_vma, &lookup))
+		if (vma_matches(vm->usm.last_fault_vma, page_addr))
 			vma = vm->usm.last_fault_vma;
 	}
 	if (!vma)
-		vma = xe_vm_find_overlapping_vma(vm, &lookup);
+		vma = xe_vm_find_overlapping_vma(vm, page_addr, SZ_4K);
 
 	return vma;
 }
@@ -170,7 +169,7 @@ static int handle_pagefault(struct xe_gt *gt, struct pagefault *pf)
 	}
 
 	/* Lock VM and BOs dma-resv */
-	bo = vma->bo;
+	bo = xe_vma_bo(vma);
 	if (only_needs_bo_lock(bo)) {
 		/* This path ensures the BO's LRU is updated */
 		ret = xe_bo_lock(bo, &ww, xe->info.tile_count, false);
@@ -487,12 +486,8 @@ static struct xe_vma *get_acc_vma(struct xe_vm *vm, struct acc *acc)
 {
 	u64 page_va = acc->va_range_base + (ffs(acc->sub_granularity) - 1) *
 		sub_granularity_in_byte(acc->granularity);
-	struct xe_vma lookup;
-
-	lookup.start = page_va;
-	lookup.end = lookup.start + SZ_4K - 1;
 
-	return xe_vm_find_overlapping_vma(vm, &lookup);
+	return xe_vm_find_overlapping_vma(vm, page_va, SZ_4K);
 }
 
 static int handle_acc(struct xe_gt *gt, struct acc *acc)
@@ -536,7 +531,7 @@ static int handle_acc(struct xe_gt *gt, struct acc *acc)
 		goto unlock_vm;
 
 	/* Lock VM and BOs dma-resv */
-	bo = vma->bo;
+	bo = xe_vma_bo(vma);
 	if (only_needs_bo_lock(bo)) {
 		/* This path ensures the BO's LRU is updated */
 		ret = xe_bo_lock(bo, &ww, xe->info.tile_count, false);
drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c +7 -7
index 604f189dbd70..66fd67ffa8a9 100644
@@ -201,8 +201,8 @@ int xe_gt_tlb_invalidation_vma(struct xe_gt *gt,
 	if (!xe->info.has_range_tlb_invalidation) {
 		action[len++] = MAKE_INVAL_OP(XE_GUC_TLB_INVAL_FULL);
 	} else {
-		u64 start = vma->start;
-		u64 length = vma->end - vma->start + 1;
+		u64 start = xe_vma_start(vma);
+		u64 length = xe_vma_size(vma);
 		u64 align, end;
 
 		if (length < SZ_4K)
@@ -215,12 +215,12 @@ int xe_gt_tlb_invalidation_vma(struct xe_gt *gt,
 		 * address mask covering the required range.
 		 */
 		align = roundup_pow_of_two(length);
-		start = ALIGN_DOWN(vma->start, align);
-		end = ALIGN(vma->start + length, align);
+		start = ALIGN_DOWN(xe_vma_start(vma), align);
+		end = ALIGN(xe_vma_start(vma) + length, align);
 		length = align;
 		while (start + length < end) {
 			length <<= 1;
-			start = ALIGN_DOWN(vma->start, length);
+			start = ALIGN_DOWN(xe_vma_start(vma), length);
 		}
 
 		/*
@@ -229,7 +229,7 @@ int xe_gt_tlb_invalidation_vma(struct xe_gt *gt,
 		 */
 		if (length >= SZ_2M) {
 			length = max_t(u64, SZ_16M, length);
-			start = ALIGN_DOWN(vma->start, length);
+			start = ALIGN_DOWN(xe_vma_start(vma), length);
 		}
 
 		XE_BUG_ON(length < SZ_4K);
@@ -238,7 +238,7 @@ int xe_gt_tlb_invalidation_vma(struct xe_gt *gt,
 		XE_BUG_ON(!IS_ALIGNED(start, length));
 
 		action[len++] = MAKE_INVAL_OP(XE_GUC_TLB_INVAL_PAGE_SELECTIVE);
-		action[len++] = vma->vm->usm.asid;
+		action[len++] = xe_vma_vm(vma)->usm.asid;
 		action[len++] = lower_32_bits(start);
 		action[len++] = upper_32_bits(start);
 		action[len++] = ilog2(length) - ilog2(SZ_4K);
drivers/gpu/drm/xe/xe_migrate.c +6 -2
index f40f47ccb76f..b44aa094a466 100644
@@ -1050,8 +1050,10 @@ xe_migrate_update_pgtables_cpu(struct xe_migrate *m,
 		return ERR_PTR(-ETIME);
 
 	if (wait_vm && !dma_resv_test_signaled(&vm->resv,
-					       DMA_RESV_USAGE_BOOKKEEP))
+					       DMA_RESV_USAGE_BOOKKEEP)) {
+		vm_dbg(&vm->xe->drm, "wait on VM for munmap");
 		return ERR_PTR(-ETIME);
+	}
 
 	if (ops->pre_commit) {
 		err = ops->pre_commit(pt_update);
@@ -1139,7 +1141,8 @@ xe_migrate_update_pgtables(struct xe_migrate *m,
 	u64 addr;
 	int err = 0;
 	bool usm = !eng && xe->info.supports_usm;
-	bool first_munmap_rebind = vma && vma->first_munmap_rebind;
+	bool first_munmap_rebind = vma &&
+		vma->gpuva.flags & XE_VMA_FIRST_REBIND;
 
 	/* Use the CPU if no in syncs and engine is idle */
 	if (no_in_syncs(syncs, num_syncs) && (!eng || xe_engine_is_idle(eng))) {
@@ -1260,6 +1263,7 @@ xe_migrate_update_pgtables(struct xe_migrate *m,
 	 * trigger preempts before moving forward
 	 */
 	if (first_munmap_rebind) {
+		vm_dbg(&vm->xe->drm, "wait on first_munmap_rebind");
 		err = job_add_deps(job, &vm->resv,
 				   DMA_RESV_USAGE_BOOKKEEP);
 		if (err)
drivers/gpu/drm/xe/xe_pt.c +55 -51

288 lines not on this page · raw message

drivers/gpu/drm/xe/xe_trace.h +5 -5
index 38e9d7c6197b..5c7515296345 100644
@@ -18,7 +18,7 @@
 #include "xe_gt_types.h"
 #include "xe_guc_engine_types.h"
 #include "xe_sched_job.h"
-#include "xe_vm_types.h"
+#include "xe_vm.h"
 
 DECLARE_EVENT_CLASS(xe_gt_tlb_invalidation_fence,
 		    TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence),
@@ -378,10 +378,10 @@ DECLARE_EVENT_CLASS(xe_vma,
 
 		    TP_fast_assign(
 			   __entry->vma = (unsigned long)vma;
-			   __entry->asid = vma->vm->usm.asid;
-			   __entry->start = vma->start;
-			   __entry->end = vma->end;
-			   __entry->ptr = (u64)vma->userptr.ptr;
+			   __entry->asid = xe_vma_vm(vma)->usm.asid;
+			   __entry->start = xe_vma_start(vma);
+			   __entry->end = xe_vma_end(vma) - 1;
+			   __entry->ptr = xe_vma_userptr(vma);
 			   ),
 
 		    TP_printk("vma=0x%016llx, asid=0x%05x, start=0x%012llx, end=0x%012llx, ptr=0x%012llx,",
drivers/gpu/drm/xe/xe_vm.c +882 -965

2,471 lines not on this page · raw message

drivers/gpu/drm/xe/xe_vm.h +63 -3

45 lines not on this page · raw message

drivers/gpu/drm/xe/xe_vm_madvise.c +21 -67

88 lines not on this page · raw message

drivers/gpu/drm/xe/xe_vm_types.h +113 -54

167 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.34.1

Part of the diff is not shown here. View the raw message.

author maintainer
#17 lore raw

PATCH 17/30 drm/xe: NULL binding implementation

In reply to Matthew Brost

Add uAPI and implementation for NULL bindings. A NULL binding is defined as writes dropped and read zero. A single bit in the uAPI has been added which results in a single bit in the PTEs being set.

NULL bindings are indended to be used to implement VK sparse bindings.

---
 drivers/gpu/drm/xe/xe_bo.h           |  1 +
 drivers/gpu/drm/xe/xe_exec.c         |  2 +
 drivers/gpu/drm/xe/xe_gt_pagefault.c |  4 +-
 drivers/gpu/drm/xe/xe_pt.c           | 77 ++++++++++++++++-------
 drivers/gpu/drm/xe/xe_vm.c           | 92 ++++++++++++++++++----------
 drivers/gpu/drm/xe/xe_vm.h           | 10 +++
 drivers/gpu/drm/xe/xe_vm_madvise.c   |  2 +-
 drivers/gpu/drm/xe/xe_vm_types.h     |  3 +
 include/uapi/drm/xe_drm.h            |  8 +++
 9 files changed, 144 insertions(+), 55 deletions(-)
drivers/gpu/drm/xe/xe_bo.h +1 -0
index 25457b3c757b..81051f456874 100644
@@ -56,6 +56,7 @@
 #define XE_PDE_IPS_64K			BIT_ULL(11)
 
 #define XE_GGTT_PTE_LM			BIT_ULL(1)
+#define XE_PTE_NULL			BIT_ULL(9)
 #define XE_USM_PPGTT_PTE_AE		BIT_ULL(10)
 #define XE_PPGTT_PTE_LM			BIT_ULL(11)
 #define XE_PDE_64K			BIT_ULL(6)
drivers/gpu/drm/xe/xe_exec.c +2 -0
index 90c46d092737..68f876afd13c 100644
@@ -116,6 +116,8 @@ static int xe_exec_begin(struct xe_engine *e, struct ww_acquire_ctx *ww,
 	 * to a location where the GPU can access it).
 	 */
 	list_for_each_entry(vma, &vm->rebind_list, rebind_link) {
+		XE_BUG_ON(xe_vma_is_null(vma));
+
 		if (xe_vma_is_userptr(vma))
 			continue;
 
drivers/gpu/drm/xe/xe_gt_pagefault.c +2 -2
index f7a066090a13..cfffe3398fe4 100644
@@ -526,8 +526,8 @@ static int handle_acc(struct xe_gt *gt, struct acc *acc)
 
 	trace_xe_vma_acc(vma);
 
-	/* Userptr can't be migrated, nothing to do */
-	if (xe_vma_is_userptr(vma))
+	/* Userptr or null can't be migrated, nothing to do */
+	if (xe_vma_has_no_bo(vma))
 		goto unlock_vm;
 
 	/* Lock VM and BOs dma-resv */
drivers/gpu/drm/xe/xe_pt.c +56 -21

157 lines not on this page · raw message

drivers/gpu/drm/xe/xe_vm.c +61 -31

241 lines not on this page · raw message

drivers/gpu/drm/xe/xe_vm.h +10 -0
index 21b1054949c4..96e2c6b07bf8 100644
@@ -175,7 +175,17 @@ static inline void xe_vm_reactivate_rebind(struct xe_vm *vm)
 	}
 }
 
+static inline bool xe_vma_is_null(struct xe_vma *vma)
+{
+	return vma->gpuva.flags & XE_VMA_NULL;
+}
+
 static inline bool xe_vma_is_userptr(struct xe_vma *vma)
+{
+	return !xe_vma_bo(vma) && !xe_vma_is_null(vma);
+}
+
+static inline bool xe_vma_has_no_bo(struct xe_vma *vma)
 {
 	return !xe_vma_bo(vma);
 }
drivers/gpu/drm/xe/xe_vm_madvise.c +1 -1
index 02d27a354b36..03508645fa08 100644
@@ -227,7 +227,7 @@ get_vmas(struct xe_vm *vm, int *num_vmas, u64 addr, u64 range)
 	drm_gpuva_iter_for_each_range(gpuva, it, addr + range) {
 		struct xe_vma *vma = gpuva_to_vma(gpuva);
 
-		if (xe_vma_is_userptr(vma))
+		if (xe_vma_has_no_bo(vma))
 			continue;
 
 		if (*num_vmas == max_vmas) {
drivers/gpu/drm/xe/xe_vm_types.h +3 -0
index 243dc91a61b0..b61007b70502 100644
@@ -29,6 +29,7 @@ struct xe_vm;
 #define XE_VMA_ATOMIC_PTE_BIT	(DRM_GPUVA_USERBITS << 2)
 #define XE_VMA_FIRST_REBIND	(DRM_GPUVA_USERBITS << 3)
 #define XE_VMA_LAST_REBIND	(DRM_GPUVA_USERBITS << 4)
+#define XE_VMA_NULL		(DRM_GPUVA_USERBITS << 5)
 
 struct xe_vma {
 	/** @gpuva: Base GPUVA object */
@@ -315,6 +316,8 @@ struct xe_vma_op_map {
 	bool immediate;
 	/** @read_only: Read only */
 	bool read_only;
+	/** @null: NULL (writes dropped, read zero) */
+	bool null;
 };
 
 /** struct xe_vma_op_unmap - VMA unmap operation */
include/uapi/drm/xe_drm.h +8 -0
index b0b80aae3ee8..27c51946fadd 100644
@@ -447,6 +447,14 @@ struct drm_xe_vm_bind_op {
 	 * than differing the MAP to the page fault handler.
 	 */
 #define XE_VM_BIND_FLAG_IMMEDIATE	(0x1 << 18)
+	/*
+	 * When the NULL flag is set, the page tables are setup with a special
+	 * bit which indicates writes are dropped and all reads return zero. The
+	 * NULL flags is only valid for XE_VM_BIND_OP_MAP operations, the BO
+	 * handle MBZ, and the BO offset MBZ. This flag is intended to implement
+	 * VK sparse bindings.
+	 */
+#define XE_VM_BIND_FLAG_NULL		(0x1 << 19)
 
 	/** @reserved: Reserved */
 	__u64 reserved[2];
Signature · 1 line
2.34.1
author maintainer
#18 lore raw

PATCH 18/30 drm/xe: Avoid doing rebinds

In reply to Matthew Brost

If we dont change page sizes we can avoid doing rebinds rather just do a partial unbind. The algorithm to determine is page size is greedy as we assume all pages in the removed VMA are the largest page used in the VMA.

---
 drivers/gpu/drm/xe/xe_pt.c       |  4 ++
 drivers/gpu/drm/xe/xe_vm.c       | 71 +++++++++++++++++++++++++-------
 drivers/gpu/drm/xe/xe_vm_types.h | 17 ++++----
 3 files changed, 67 insertions(+), 25 deletions(-)
drivers/gpu/drm/xe/xe_pt.c +4 -0
index b4edb751bfbb..010f44260cda 100644
@@ -412,6 +412,8 @@ struct xe_pt_stage_bind_walk {
 	/* Input parameters for the walk */
 	/** @vm: The vm we're building for. */
 	struct xe_vm *vm;
+	/** @vma: The vma we are binding for. */
+	struct xe_vma *vma;
 	/** @gt: The gt we're building for. */
 	struct xe_gt *gt;
 	/** @cache: Desired cache level for the ptes */
@@ -688,6 +690,7 @@ xe_pt_stage_bind_entry(struct drm_pt *parent, pgoff_t offset,
 		if (!null)
 			xe_res_next(curs, next - addr);
 		xe_walk->va_curs_start = next;
+		xe_walk->vma->gpuva.flags |= (XE_VMA_PTE_4K << level);
 		*action = ACTION_CONTINUE;
 
 		return ret;
@@ -776,6 +779,7 @@ xe_pt_stage_bind(struct xe_gt *gt, struct xe_vma *vma,
 			.max_level = XE_PT_HIGHEST_LEVEL,
 		},
 		.vm = xe_vma_vm(vma),
+		.vma = vma,
 		.gt = gt,
 		.curs = &curs,
 		.va_curs_start = xe_vma_start(vma),
drivers/gpu/drm/xe/xe_vm.c +56 -15

113 lines not on this page · raw message

drivers/gpu/drm/xe/xe_vm_types.h +7 -10
index b61007b70502..d55ec8156caa 100644
@@ -30,6 +30,9 @@ struct xe_vm;
 #define XE_VMA_FIRST_REBIND	(DRM_GPUVA_USERBITS << 3)
 #define XE_VMA_LAST_REBIND	(DRM_GPUVA_USERBITS << 4)
 #define XE_VMA_NULL		(DRM_GPUVA_USERBITS << 5)
+#define XE_VMA_PTE_4K		(DRM_GPUVA_USERBITS << 6)
+#define XE_VMA_PTE_2M		(DRM_GPUVA_USERBITS << 7)
+#define XE_VMA_PTE_1G		(DRM_GPUVA_USERBITS << 8)
 
 struct xe_vma {
 	/** @gpuva: Base GPUVA object */
@@ -320,14 +323,6 @@ struct xe_vma_op_map {
 	bool null;
 };
 
-/** struct xe_vma_op_unmap - VMA unmap operation */
-struct xe_vma_op_unmap {
-	/** @start: start of the VMA unmap */
-	u64 start;
-	/** @range: range of the VMA unmap */
-	u64 range;
-};
-
 /** struct xe_vma_op_remap - VMA remap operation */
 struct xe_vma_op_remap {
 	/** @prev: VMA preceding part of a split mapping */
@@ -338,6 +333,10 @@ struct xe_vma_op_remap {
 	u64 start;
 	/** @range: range of the VMA unmap */
 	u64 range;
+	/** @skip_prev: skip prev rebind */
+	bool skip_prev;
+	/** @skip_next: skip next rebind */
+	bool skip_next;
 	/** @unmap_done: unmap operation in done */
 	bool unmap_done;
 };
@@ -395,8 +394,6 @@ struct xe_vma_op {
 	union {
 		/** @map: VMA map operation specific data */
 		struct xe_vma_op_map map;
-		/** @unmap: VMA unmap operation specific data */
-		struct xe_vma_op_unmap unmap;
 		/** @remap: VMA remap operation specific data */
 		struct xe_vma_op_remap remap;
 		/** @prefetch: VMA prefetch operation specific data */
Signature · 1 line
2.34.1
author maintainer
#19 lore raw

PATCH 19/30 drm/xe: Reduce the number list links in xe_vma

In reply to Matthew Brost

5 list links in can be squashed into a union in xe_vma as being on the various list is mutually exclusive.

---
 drivers/gpu/drm/xe/xe_gt_pagefault.c |  2 +-
 drivers/gpu/drm/xe/xe_pt.c           |  5 +-
 drivers/gpu/drm/xe/xe_vm.c           | 29 ++++++------
 drivers/gpu/drm/xe/xe_vm_types.h     | 71 +++++++++++++++-------------
 4 files changed, 55 insertions(+), 52 deletions(-)
drivers/gpu/drm/xe/xe_gt_pagefault.c +1 -1
index cfffe3398fe4..d7bf6b0a0697 100644
@@ -157,7 +157,7 @@ static int handle_pagefault(struct xe_gt *gt, struct pagefault *pf)
 
 	if (xe_vma_is_userptr(vma) && write_locked) {
 		spin_lock(&vm->userptr.invalidated_lock);
-		list_del_init(&vma->userptr.invalidate_link);
+		list_del_init(&vma->invalidate_link);
 		spin_unlock(&vm->userptr.invalidated_lock);
 
 		ret = xe_vma_userptr_pin_pages(vma);
drivers/gpu/drm/xe/xe_pt.c +2 -3
index 010f44260cda..8eab8e1bbaf0 100644
@@ -1116,8 +1116,7 @@ static int xe_pt_userptr_inject_eagain(struct xe_vma *vma)
 
 		vma->userptr.divisor = divisor << 1;
 		spin_lock(&vm->userptr.invalidated_lock);
-		list_move_tail(&vma->userptr.invalidate_link,
-			       &vm->userptr.invalidated);
+		list_move_tail(&vma->invalidate_link, &vm->userptr.invalidated);
 		spin_unlock(&vm->userptr.invalidated_lock);
 		return true;
 	}
@@ -1724,7 +1723,7 @@ __xe_pt_unbind_vma(struct xe_gt *gt, struct xe_vma *vma, struct xe_engine *e,
 
 		if (!vma->gt_present) {
 			spin_lock(&vm->userptr.invalidated_lock);
-			list_del_init(&vma->userptr.invalidate_link);
+			list_del_init(&vma->invalidate_link);
 			spin_unlock(&vm->userptr.invalidated_lock);
 		}
 		up_read(&vm->userptr.notifier_lock);
drivers/gpu/drm/xe/xe_vm.c +14 -15
index 0712cf9d9894..0224ef916794 100644
@@ -677,8 +677,7 @@ static bool vma_userptr_invalidate(struct mmu_interval_notifier *mni,
 	if (!xe_vm_in_fault_mode(vm) &&
 	    !(vma->gpuva.flags & XE_VMA_DESTROYED) && vma->gt_present) {
 		spin_lock(&vm->userptr.invalidated_lock);
-		list_move_tail(&vma->userptr.invalidate_link,
-			       &vm->userptr.invalidated);
+		list_move_tail(&vma->invalidate_link, &vm->userptr.invalidated);
 		spin_unlock(&vm->userptr.invalidated_lock);
 	}
 
@@ -726,8 +725,8 @@ int xe_vm_userptr_pin(struct xe_vm *vm)
 	/* Collect invalidated userptrs */
 	spin_lock(&vm->userptr.invalidated_lock);
 	list_for_each_entry_safe(vma, next, &vm->userptr.invalidated,
-				 userptr.invalidate_link) {
-		list_del_init(&vma->userptr.invalidate_link);
+				 invalidate_link) {
+		list_del_init(&vma->invalidate_link);
 		list_move_tail(&vma->userptr_link, &vm->userptr.repin_list);
 	}
 	spin_unlock(&vm->userptr.invalidated_lock);
@@ -830,12 +829,11 @@ static struct xe_vma *xe_vma_create(struct xe_vm *vm,
 		return vma;
 	}
 
-	/* FIXME: Way to many lists, should be able to reduce this */
+	/*
+	 * userptr_link, destroy_link, notifier.rebind_link,
+	 * invalidate_link
+	 */
 	INIT_LIST_HEAD(&vma->rebind_link);
-	INIT_LIST_HEAD(&vma->unbind_link);
-	INIT_LIST_HEAD(&vma->userptr_link);
-	INIT_LIST_HEAD(&vma->userptr.invalidate_link);
-	INIT_LIST_HEAD(&vma->notifier.rebind_link);
 	INIT_LIST_HEAD(&vma->extobj.link);
 
 	INIT_LIST_HEAD(&vma->gpuva.gem.entry);
@@ -953,15 +951,14 @@ static void xe_vma_destroy(struct xe_vma *vma, struct dma_fence *fence)
 	struct xe_vm *vm = xe_vma_vm(vma);
 
 	lockdep_assert_held_write(&vm->lock);
-	XE_BUG_ON(!list_empty(&vma->unbind_link));
 
 	if (xe_vma_is_userptr(vma)) {
 		XE_WARN_ON(!(vma->gpuva.flags & XE_VMA_DESTROYED));
 
 		spin_lock(&vm->userptr.invalidated_lock);
-		list_del_init(&vma->userptr.invalidate_link);
+		if (!list_empty(&vma->invalidate_link))
+			list_del_init(&vma->invalidate_link);
 		spin_unlock(&vm->userptr.invalidated_lock);
-		list_del(&vma->userptr_link);
 	} else if (!xe_vma_is_null(vma)) {
 		xe_bo_assert_held(xe_vma_bo(vma));
 		drm_gpuva_unlink(&vma->gpuva);
@@ -1328,7 +1325,9 @@ void xe_vm_close_and_put(struct xe_vm *vm)
 			continue;
 		}
 
-		list_add_tail(&vma->unbind_link, &contested);
+		if (!list_empty(&vma->destroy_link))
+			list_del_init(&vma->destroy_link);
+		list_add_tail(&vma->destroy_link, &contested);
 	}
 
 	/*
@@ -1356,8 +1355,8 @@ void xe_vm_close_and_put(struct xe_vm *vm)
 	 * Since we hold a refcount to the bo, we can remove and free
 	 * the members safely without locking.
 	 */
-	list_for_each_entry_safe(vma, next_vma, &contested, unbind_link) {
-		list_del_init(&vma->unbind_link);
+	list_for_each_entry_safe(vma, next_vma, &contested, destroy_link) {
+		list_del_init(&vma->destroy_link);
 		xe_vma_destroy_unlocked(vma);
 	}
 
drivers/gpu/drm/xe/xe_vm_types.h +38 -33

93 lines not on this page · raw message

Signature · 1 line
2.34.1
author maintainer
#20 lore raw

PATCH 20/30 drm/xe: Optimize size of xe_vma allocation

In reply to Matthew Brost

Reduce gt_mask to a u8 from a u64, only allocate userptr state if VMA is a userptr, and union of destroy callback and worker.

---
 drivers/gpu/drm/xe/xe_vm.c       | 14 +++--
 drivers/gpu/drm/xe/xe_vm_types.h | 88 +++++++++++++++++---------------
 2 files changed, 57 insertions(+), 45 deletions(-)
drivers/gpu/drm/xe/xe_vm.c +9 -5
index 0224ef916794..3eecf1d0958f 100644
@@ -814,7 +814,7 @@ static struct xe_vma *xe_vma_create(struct xe_vm *vm,
 				    u64 bo_offset_or_userptr,
 				    u64 start, u64 end,
 				    bool read_only, bool null,
-				    u64 gt_mask)
+				    u8 gt_mask)
 {
 	struct xe_vma *vma;
 	struct xe_gt *gt;
@@ -823,7 +823,11 @@ static struct xe_vma *xe_vma_create(struct xe_vm *vm,
 	XE_BUG_ON(start >= end);
 	XE_BUG_ON(end >= vm->size);
 
-	vma = kzalloc(sizeof(*vma), GFP_KERNEL);
+	if (!bo && !null)	/* userptr */
+		vma = kzalloc(sizeof(*vma), GFP_KERNEL);
+	else
+		vma = kzalloc(sizeof(*vma) - sizeof(struct xe_userptr),
+			      GFP_KERNEL);
 	if (!vma) {
 		vma = ERR_PTR(-ENOMEM);
 		return vma;
@@ -2149,7 +2153,7 @@ static void print_op(struct xe_device *xe, struct drm_gpuva_op *op)
 static struct drm_gpuva_ops *
 vm_bind_ioctl_ops_create(struct xe_vm *vm, struct xe_bo *bo,
 			 u64 bo_offset_or_userptr, u64 addr, u64 range,
-			 u32 operation, u64 gt_mask, u32 region)
+			 u32 operation, u8 gt_mask, u32 region)
 {
 	struct drm_gem_object *obj = bo ? &bo->ttm.base : NULL;
 	struct ww_acquire_ctx ww;
@@ -2234,7 +2238,7 @@ vm_bind_ioctl_ops_create(struct xe_vm *vm, struct xe_bo *bo,
 }
 
 static struct xe_vma *new_vma(struct xe_vm *vm, struct drm_gpuva_op_map *op,
-			      u64 gt_mask, bool read_only, bool null)
+			      u8 gt_mask, bool read_only, bool null)
 {
 	struct xe_bo *bo = op->gem.obj ? gem_to_xe_bo(op->gem.obj) : NULL;
 	struct xe_vma *vma;
@@ -3213,8 +3217,8 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
 		u64 addr = bind_ops[i].addr;
 		u32 op = bind_ops[i].op;
 		u64 obj_offset = bind_ops[i].obj_offset;
-		u64 gt_mask = bind_ops[i].gt_mask;
 		u32 region = bind_ops[i].region;
+		u8 gt_mask = bind_ops[i].gt_mask;
 
 		ops[i] = vm_bind_ioctl_ops_create(vm, bos[i], obj_offset,
 						  addr, range, op, gt_mask,
drivers/gpu/drm/xe/xe_vm_types.h +48 -40

121 lines not on this page · raw message

Signature · 1 line
2.34.1
author maintainer
#21 lore raw

PATCH 21/30 drm/gpuva: Add drm device to GPUVA manager

In reply to Matthew Brost

This is the logical place for this, will help with upcoming changes too.

---
 drivers/gpu/drm/drm_gpuva_mgr.c  |  3 +++
 drivers/gpu/drm/xe/xe_migrate.c  | 10 +++++-----
 drivers/gpu/drm/xe/xe_pt.c       | 18 +++++++++---------
 drivers/gpu/drm/xe/xe_vm.c       | 31 +++++++++++++++----------------
 drivers/gpu/drm/xe/xe_vm.h       | 10 ++++++++++
 drivers/gpu/drm/xe/xe_vm_types.h |  2 --
 include/drm/drm_gpuva_mgr.h      |  4 ++++
 7 files changed, 46 insertions(+), 32 deletions(-)
drivers/gpu/drm/drm_gpuva_mgr.c +3 -0
index bd7d27ee44bb..137322945e91 100644
@@ -413,6 +413,7 @@ static void __drm_gpuva_remove(struct drm_gpuva *va);
 /**
  * drm_gpuva_manager_init - initialize a &drm_gpuva_manager
  * @mgr: pointer to the &drm_gpuva_manager to initialize
+ * @drm: drm device
  * @name: the name of the GPU VA space
  * @start_offset: the start offset of the GPU VA space
  * @range: the size of the GPU VA space
@@ -427,6 +428,7 @@ static void __drm_gpuva_remove(struct drm_gpuva *va);
  */
 void
 drm_gpuva_manager_init(struct drm_gpuva_manager *mgr,
+		       struct drm_device *drm,
 		       const char *name,
 		       u64 start_offset, u64 range,
 		       u64 reserve_offset, u64 reserve_range,
@@ -437,6 +439,7 @@ drm_gpuva_manager_init(struct drm_gpuva_manager *mgr,
 	mgr->mm_start = start_offset;
 	mgr->mm_range = range;
 
+	mgr->drm = drm;
 	mgr->name = name ? name : "unknown";
 	mgr->ops = ops;
 
drivers/gpu/drm/xe/xe_migrate.c +5 -5
index b44aa094a466..0a393c5772e5 100644
@@ -129,7 +129,7 @@ static u64 xe_migrate_vram_ofs(u64 addr)
 static int xe_migrate_create_cleared_bo(struct xe_migrate *m, struct xe_vm *vm)
 {
 	struct xe_gt *gt = m->gt;
-	struct xe_device *xe = vm->xe;
+	struct xe_device *xe = xe_vm_device(vm);
 	size_t cleared_size;
 	u64 vram_addr;
 	bool is_vram;
@@ -175,7 +175,7 @@ static int xe_migrate_prepare_vm(struct xe_gt *gt, struct xe_migrate *m,
 	/* Need to be sure everything fits in the first PT, or create more */
 	XE_BUG_ON(m->batch_base_ofs + batch->size >= SZ_2M);
 
-	bo = xe_bo_create_pin_map(vm->xe, m->gt, vm,
+	bo = xe_bo_create_pin_map(xe_vm_device(vm), m->gt, vm,
 				  num_entries * XE_PAGE_SIZE,
 				  ttm_bo_type_kernel,
 				  XE_BO_CREATE_VRAM_IF_DGFX(m->gt) |
@@ -1051,7 +1051,7 @@ xe_migrate_update_pgtables_cpu(struct xe_migrate *m,
 
 	if (wait_vm && !dma_resv_test_signaled(&vm->resv,
 					       DMA_RESV_USAGE_BOOKKEEP)) {
-		vm_dbg(&vm->xe->drm, "wait on VM for munmap");
+		vm_dbg(&xe_vm_device(vm)->drm, "wait on VM for munmap");
 		return ERR_PTR(-ETIME);
 	}
 
@@ -1069,7 +1069,7 @@ xe_migrate_update_pgtables_cpu(struct xe_migrate *m,
 
 	if (vm) {
 		trace_xe_vm_cpu_bind(vm);
-		xe_device_wmb(vm->xe);
+		xe_device_wmb(xe_vm_device(vm));
 	}
 
 	fence = dma_fence_get_stub();
@@ -1263,7 +1263,7 @@ xe_migrate_update_pgtables(struct xe_migrate *m,
 	 * trigger preempts before moving forward
 	 */
 	if (first_munmap_rebind) {
-		vm_dbg(&vm->xe->drm, "wait on first_munmap_rebind");
+		vm_dbg(&xe_vm_device(vm)->drm, "wait on first_munmap_rebind");
 		err = job_add_deps(job, &vm->resv,
 				   DMA_RESV_USAGE_BOOKKEEP);
 		if (err)
drivers/gpu/drm/xe/xe_pt.c +9 -9
index 8eab8e1bbaf0..4167f666d98d 100644
@@ -218,7 +218,7 @@ struct xe_pt *xe_pt_create(struct xe_vm *vm, struct xe_gt *gt,
 	if (!pt)
 		return ERR_PTR(-ENOMEM);
 
-	bo = xe_bo_create_pin_map(vm->xe, gt, vm, SZ_4K,
+	bo = xe_bo_create_pin_map(xe_vm_device(vm), gt, vm, SZ_4K,
 				  ttm_bo_type_kernel,
 				  XE_BO_CREATE_VRAM_IF_DGFX(gt) |
 				  XE_BO_CREATE_IGNORE_MIN_PAGE_SIZE_BIT |
@@ -264,11 +264,11 @@ void xe_pt_populate_empty(struct xe_gt *gt, struct xe_vm *vm,
 		 * FIXME: Some memory is allocated already allocated to zero?
 		 * Find out which memory that is and avoid this memset...
 		 */
-		xe_map_memset(vm->xe, map, 0, 0, SZ_4K);
+		xe_map_memset(xe_vm_device(vm), map, 0, 0, SZ_4K);
 	} else {
 		empty = __xe_pt_empty_pte(gt, vm, pt->level);
 		for (i = 0; i < XE_PDES; i++)
-			xe_pt_write(vm->xe, map, i, empty);
+			xe_pt_write(xe_vm_device(vm), map, i, empty);
 	}
 }
 
@@ -355,7 +355,7 @@ int xe_pt_create_scratch(struct xe_device *xe, struct xe_gt *gt,
 	if (IS_ERR(vm->scratch_bo[id]))
 		return PTR_ERR(vm->scratch_bo[id]);
 
-	xe_map_memset(vm->xe, &vm->scratch_bo[id]->vmap, 0, 0,
+	xe_map_memset(xe_vm_device(vm), &vm->scratch_bo[id]->vmap, 0, 0,
 		      vm->scratch_bo[id]->size);
 
 	for (i = 0; i < vm->pt_root[id]->level; i++) {
@@ -538,7 +538,7 @@ xe_pt_insert_entry(struct xe_pt_stage_bind_walk *xe_walk, struct xe_pt *parent,
 		if (unlikely(xe_child))
 			parent->drm.dir->entries[offset] = &xe_child->drm;
 
-		xe_pt_write(xe_walk->vm->xe, map, offset, pte);
+		xe_pt_write(xe_vm_device(xe_walk->vm), map, offset, pte);
 		parent->num_live++;
 	} else {
 		/* Shared pt. Stage update. */
@@ -1337,7 +1337,7 @@ __xe_pt_bind_vma(struct xe_gt *gt, struct xe_vma *vma, struct xe_engine *e,
 	xe_vm_assert_held(vm);
 	XE_BUG_ON(xe_gt_is_media_type(gt));
 
-	vm_dbg(&xe_vma_vm(vma)->xe->drm,
+	vm_dbg(&xe_vma_device(vma)->drm,
 	       "Preparing bind, with range [%llx...%llx) engine %p.\n",
 	       xe_vma_start(vma), xe_vma_end(vma) - 1, e);
 
@@ -1366,7 +1366,7 @@ __xe_pt_bind_vma(struct xe_gt *gt, struct xe_vma *vma, struct xe_engine *e,
 
 
 		if (last_munmap_rebind)
-			vm_dbg(&vm->xe->drm, "last_munmap_rebind");
+			vm_dbg(&xe_vm_device(vm)->drm, "last_munmap_rebind");
 
 		/* TLB invalidation must be done before signaling rebind */
 		if (rebind && !xe_vm_no_dma_fences(xe_vma_vm(vma))) {
@@ -1401,7 +1401,7 @@ __xe_pt_bind_vma(struct xe_gt *gt, struct xe_vma *vma, struct xe_engine *e,
 			xe_bo_put_commit(&deferred);
 		}
 		if (!rebind && last_munmap_rebind && xe_vm_in_compute_mode(vm))
-			queue_work(vm->xe->ordered_wq,
+			queue_work(xe_vm_device(vm)->ordered_wq,
 				   &vm->preempt.rebind_work);
 	} else {
 		kfree(ifence);
@@ -1664,7 +1664,7 @@ __xe_pt_unbind_vma(struct xe_gt *gt, struct xe_vma *vma, struct xe_engine *e,
 	xe_vm_assert_held(vm);
 	XE_BUG_ON(xe_gt_is_media_type(gt));
 
-	vm_dbg(&xe_vma_vm(vma)->xe->drm,
+	vm_dbg(&xe_vma_device(vma)->drm,
 	       "Preparing unbind, with range [%llx...%llx) engine %p.\n",
 	       xe_vma_start(vma), xe_vma_end(vma) - 1, e);
 
drivers/gpu/drm/xe/xe_vm.c +15 -16
index 3eecf1d0958f..77220a17d29c 100644
@@ -50,7 +50,7 @@ int xe_vma_userptr_check_repin(struct xe_vma *vma)
 int xe_vma_userptr_pin_pages(struct xe_vma *vma)
 {
 	struct xe_vm *vm = xe_vma_vm(vma);
-	struct xe_device *xe = vm->xe;
+	struct xe_device *xe = xe_vm_device(vm);
 	const unsigned long num_pages = xe_vma_size(vma) >> PAGE_SHIFT;
 	struct page **pages;
 	bool in_kthread = !current->mm;
@@ -852,12 +852,12 @@ static struct xe_vma *xe_vma_create(struct xe_vm *vm,
 	if (gt_mask) {
 		vma->gt_mask = gt_mask;
 	} else {
-		for_each_gt(gt, vm->xe, id)
+		for_each_gt(gt, xe_vm_device(vm), id)
 			if (!xe_gt_is_media_type(gt))
 				vma->gt_mask |= 0x1 << id;
 	}
 
-	if (vm->xe->info.platform == XE_PVC)
+	if (xe_vm_device(vm)->info.platform == XE_PVC)
 		vma->gpuva.flags |= XE_VMA_ATOMIC_PTE_BIT;
 
 	if (bo) {
@@ -904,7 +904,7 @@ static void vm_remove_extobj(struct xe_vma *vma)
 static void xe_vma_destroy_late(struct xe_vma *vma)
 {
 	struct xe_vm *vm = xe_vma_vm(vma);
-	struct xe_device *xe = vm->xe;
+	struct xe_device *xe = xe_vm_device(vm);
 	bool read_only = xe_vma_read_only(vma);
 
 	if (xe_vma_is_userptr(vma)) {
@@ -1084,7 +1084,6 @@ struct xe_vm *xe_vm_create(struct xe_device *xe, u32 flags)
 	if (!vm)
 		return ERR_PTR(-ENOMEM);
 
-	vm->xe = xe;
 	kref_init(&vm->refcount);
 	dma_resv_init(&vm->resv);
 
@@ -1125,7 +1124,7 @@ struct xe_vm *xe_vm_create(struct xe_device *xe, u32 flags)
 	if (err)
 		goto err_put;
 
-	drm_gpuva_manager_init(&vm->mgr, "Xe VM", 0, vm->size, 0, 0,
+	drm_gpuva_manager_init(&vm->mgr, &xe->drm, "Xe VM", 0, vm->size, 0, 0,
 			       &gpuva_ops);
 	if (IS_DGFX(xe) && xe->info.vram_flags & XE_VRAM_FLAGS_NEED64K)
 		vm->flags |= XE_VM_FLAGS_64K;
@@ -1284,7 +1283,7 @@ void xe_vm_close_and_put(struct xe_vm *vm)
 {
 	struct list_head contested;
 	struct ww_acquire_ctx ww;
-	struct xe_device *xe = vm->xe;
+	struct xe_device *xe = xe_vm_device(vm);
 	struct xe_gt *gt;
 	struct xe_vma *vma, *next_vma;
 	struct drm_gpuva *gpuva;
@@ -1387,7 +1386,7 @@ static void vm_destroy_work_func(struct work_struct *w)
 	struct xe_vm *vm =
 		container_of(w, struct xe_vm, destroy_work);
 	struct ww_acquire_ctx ww;
-	struct xe_device *xe = vm->xe;
+	struct xe_device *xe = xe_vm_device(vm);
 	struct xe_gt *gt;
 	u8 id;
 	void *lookup;
@@ -1481,7 +1480,7 @@ xe_vm_unbind_vma(struct xe_vma *vma, struct xe_engine *e,
 			return ERR_PTR(-ENOMEM);
 	}
 
-	for_each_gt(gt, vm->xe, id) {
+	for_each_gt(gt, xe_vm_device(vm), id) {
 		if (!(vma->gt_present & BIT(id)))
 			goto next;
 
@@ -1555,7 +1554,7 @@ xe_vm_bind_vma(struct xe_vma *vma, struct xe_engine *e,
 			return ERR_PTR(-ENOMEM);
 	}
 
-	for_each_gt(gt, vm->xe, id) {
+	for_each_gt(gt, xe_vm_device(vm), id) {
 		if (!(vma->gt_mask & BIT(id)))
 			goto next;
 
@@ -2061,7 +2060,7 @@ static int vm_insert_extobj(struct xe_vm *vm, struct xe_vma *vma)
 static int vm_bind_ioctl_lookup_vma(struct xe_vm *vm, struct xe_bo *bo,
 				    u64 addr, u64 range, u32 op)
 {
-	struct xe_device *xe = vm->xe;
+	struct xe_device *xe = xe_vm_device(vm);
 	struct xe_vma *vma;
 	bool async = !!(op & XE_VM_BIND_FLAG_ASYNC);
 
@@ -2164,7 +2163,7 @@ vm_bind_ioctl_ops_create(struct xe_vm *vm, struct xe_bo *bo,
 
 	lockdep_assert_held_write(&vm->lock);
 
-	vm_dbg(&vm->xe->drm,
+	vm_dbg(&xe_vm_device(vm)->drm,
 	       "op=%d, addr=0x%016llx, range=0x%016llx, bo_offset_or_userptr=0x%016llx",
 	       VM_BIND_OP(operation), addr, range, bo_offset_or_userptr);
 
@@ -2232,7 +2231,7 @@ vm_bind_ioctl_ops_create(struct xe_vm *vm, struct xe_bo *bo,
 
 	if (!IS_ERR(ops))
 		drm_gpuva_for_each_op(__op, ops)
-			print_op(vm->xe, __op);
+			print_op(xe_vm_device(vm), __op);
 
 	return ops;
 }
@@ -2779,7 +2778,7 @@ static void xe_vma_op_work_func(struct work_struct *w)
 			down_write(&vm->lock);
 			err = xe_vma_op_execute(vm, op);
 			if (err) {
-				drm_warn(&vm->xe->drm,
+				drm_warn(&xe_vm_device(vm)->drm,
 					 "Async VM op(%d) failed with %d",
 					 op->base.op, err);
 				vm_set_async_error(vm, err);
@@ -3099,7 +3098,7 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
 
 			/* Rebinds may have been blocked, give worker a kick */
 			if (xe_vm_in_compute_mode(vm))
-				queue_work(vm->xe->ordered_wq,
+				queue_work(xe_vm_device(vm)->ordered_wq,
 					   &vm->preempt.rebind_work);
 		}
 
@@ -3311,7 +3310,7 @@ void xe_vm_unlock(struct xe_vm *vm, struct ww_acquire_ctx *ww)
  */
 int xe_vm_invalidate_vma(struct xe_vma *vma)
 {
-	struct xe_device *xe = xe_vma_vm(vma)->xe;
+	struct xe_device *xe = xe_vm_device(xe_vma_vm(vma));
 	struct xe_gt *gt;
 	u32 gt_needs_invalidate = 0;
 	int seqno[XE_MAX_GT];
drivers/gpu/drm/xe/xe_vm.h +10 -0
index 96e2c6b07bf8..cbbe95d6291f 100644
@@ -52,6 +52,11 @@ static inline bool xe_vm_is_closed(struct xe_vm *vm)
 struct xe_vma *
 xe_vm_find_overlapping_vma(struct xe_vm *vm, u64 start, u64 range);
 
+static inline struct xe_device *xe_vm_device(struct xe_vm *vm)
+{
+	return container_of(vm->mgr.drm, struct xe_device, drm);
+}
+
 static inline struct xe_vm *gpuva_to_vm(struct drm_gpuva *gpuva)
 {
 	return container_of(gpuva->mgr, struct xe_vm, mgr);
@@ -102,6 +107,11 @@ static inline struct xe_vm *xe_vma_vm(struct xe_vma *vma)
 	return container_of(vma->gpuva.mgr, struct xe_vm, mgr);
 }
 
+static inline struct xe_device *xe_vma_device(struct xe_vma *vma)
+{
+	return xe_vm_device(xe_vma_vm(vma));
+}
+
 static inline bool xe_vma_read_only(struct xe_vma *vma)
 {
 	return vma->gpuva.flags & XE_VMA_READ_ONLY;
drivers/gpu/drm/xe/xe_vm_types.h +0 -2
index df4797ec4d7f..fca42910dcae 100644
@@ -137,8 +137,6 @@ struct xe_vm {
 	/** @mgr: base GPUVA used to track VMAs */
 	struct drm_gpuva_manager mgr;
 
-	struct xe_device *xe;
-
 	struct kref refcount;
 
 	/* engine used for (un)binding vma's */
include/drm/drm_gpuva_mgr.h +4 -0
index 62169d850098..55b0acfdcc44 100644
@@ -169,6 +169,9 @@ static inline bool drm_gpuva_evicted(struct drm_gpuva *va)
  * There should be one manager instance per GPU virtual address space.
  */
 struct drm_gpuva_manager {
+	/** @drm: drm device */
+	struct drm_device *drm;
+
 	/**
 	 * @name: the name of the DRM GPU VA space
 	 */
@@ -204,6 +207,7 @@ struct drm_gpuva_manager {
 };
 
 void drm_gpuva_manager_init(struct drm_gpuva_manager *mgr,
+			    struct drm_device *drm,
 			    const char *name,
 			    u64 start_offset, u64 range,
 			    u64 reserve_offset, u64 reserve_range,
Signature · 1 line
2.34.1
author maintainer
#22 lore raw

PATCH 22/30 drm/gpuva: Move dma-resv to GPUVA manager

In reply to Matthew Brost

Logical place for this, will help with upcoming patches.

---
 drivers/gpu/drm/drm_gpuva_mgr.c  |  2 ++
 drivers/gpu/drm/xe/xe_bo.c       | 10 +++++-----
 drivers/gpu/drm/xe/xe_bo.h       |  2 +-
 drivers/gpu/drm/xe/xe_exec.c     |  4 ++--
 drivers/gpu/drm/xe/xe_migrate.c  |  4 ++--
 drivers/gpu/drm/xe/xe_pt.c       |  6 +++---
 drivers/gpu/drm/xe/xe_vm.c       | 34 ++++++++++++++++----------------
 drivers/gpu/drm/xe/xe_vm.h       | 12 ++++++++++-
 drivers/gpu/drm/xe/xe_vm_types.h |  6 +-----
 include/drm/drm_gpuva_mgr.h      |  6 ++++++
 10 files changed, 50 insertions(+), 36 deletions(-)
drivers/gpu/drm/drm_gpuva_mgr.c +2 -0
index 137322945e91..6d2d0f4d5018 100644
@@ -443,6 +443,8 @@ drm_gpuva_manager_init(struct drm_gpuva_manager *mgr,
 	mgr->name = name ? name : "unknown";
 	mgr->ops = ops;
 
+	dma_resv_init(&mgr->resv);
+
 	memset(&mgr->kernel_alloc_node, 0, sizeof(struct drm_gpuva));
 
 	if (reserve_range) {
drivers/gpu/drm/xe/xe_bo.c +5 -5
index a475d0584916..e0422ffb6327 100644
@@ -441,9 +441,9 @@ static int xe_bo_trigger_rebind(struct xe_device *xe, struct xe_bo *bo,
 			 * that we indeed have it locked, put the vma an the
 			 * vm's notifier.rebind_list instead and scoop later.
 			 */
-			if (dma_resv_trylock(&vm->resv))
+			if (dma_resv_trylock(xe_vm_resv(vm)))
 				vm_resv_locked = true;
-			else if (ctx->resv != &vm->resv) {
+			else if (ctx->resv != xe_vm_resv(vm)) {
 				spin_lock(&vm->notifier.list_lock);
 				list_move_tail(&vma->notifier.rebind_link,
 					       &vm->notifier.rebind_list);
@@ -456,7 +456,7 @@ static int xe_bo_trigger_rebind(struct xe_device *xe, struct xe_bo *bo,
 				list_add_tail(&vma->rebind_link, &vm->rebind_list);
 
 			if (vm_resv_locked)
-				dma_resv_unlock(&vm->resv);
+				dma_resv_unlock(xe_vm_resv(vm));
 		}
 	}
 
@@ -1240,7 +1240,7 @@ xe_bo_create_locked_range(struct xe_device *xe,
 		}
 	}
 
-	bo = __xe_bo_create_locked(xe, bo, gt, vm ? &vm->resv : NULL,
+	bo = __xe_bo_create_locked(xe, bo, gt, vm ? xe_vm_resv(vm) : NULL,
 				   vm && !xe_vm_no_dma_fences(vm) &&
 				   flags & XE_BO_CREATE_USER_BIT ?
 				   &vm->lru_bulk_move : NULL, size,
@@ -1555,7 +1555,7 @@ int xe_bo_validate(struct xe_bo *bo, struct xe_vm *vm, bool allow_res_evict)
 		xe_vm_assert_held(vm);
 
 		ctx.allow_res_evict = allow_res_evict;
-		ctx.resv = &vm->resv;
+		ctx.resv = xe_vm_resv(vm);
 	}
 
 	return ttm_bo_validate(&bo->ttm, &bo->placement, &ctx);
drivers/gpu/drm/xe/xe_bo.h +1 -1
index 81051f456874..9b401d30a130 100644
@@ -150,7 +150,7 @@ void xe_bo_unlock(struct xe_bo *bo, struct ww_acquire_ctx *ww);
 static inline void xe_bo_unlock_vm_held(struct xe_bo *bo)
 {
 	if (bo) {
-		XE_BUG_ON(bo->vm && bo->ttm.base.resv != &bo->vm->resv);
+		XE_BUG_ON(bo->vm && bo->ttm.base.resv != &bo->vm->mgr.resv);
 		if (bo->vm)
 			xe_vm_assert_held(bo->vm);
 		else
drivers/gpu/drm/xe/xe_exec.c +2 -2
index 68f876afd13c..b352fd6e1f4d 100644
@@ -327,7 +327,7 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
 	/* Wait behind munmap style rebinds */
 	if (!xe_vm_no_dma_fences(vm)) {
 		err = drm_sched_job_add_resv_dependencies(&job->drm,
-							  &vm->resv,
+							  xe_vm_resv(vm),
 							  DMA_RESV_USAGE_KERNEL);
 		if (err)
 			goto err_put_job;
@@ -355,7 +355,7 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
 	xe_sched_job_arm(job);
 	if (!xe_vm_no_dma_fences(vm)) {
 		/* Block userptr invalidations / BO eviction */
-		dma_resv_add_fence(&vm->resv,
+		dma_resv_add_fence(xe_vm_resv(vm),
 				   &job->drm.s_fence->finished,
 				   DMA_RESV_USAGE_BOOKKEEP);
 
drivers/gpu/drm/xe/xe_migrate.c +2 -2
index 0a393c5772e5..91a06c925a1e 100644
@@ -1049,7 +1049,7 @@ xe_migrate_update_pgtables_cpu(struct xe_migrate *m,
 					  DMA_RESV_USAGE_KERNEL))
 		return ERR_PTR(-ETIME);
 
-	if (wait_vm && !dma_resv_test_signaled(&vm->resv,
+	if (wait_vm && !dma_resv_test_signaled(xe_vm_resv(vm),
 					       DMA_RESV_USAGE_BOOKKEEP)) {
 		vm_dbg(&xe_vm_device(vm)->drm, "wait on VM for munmap");
 		return ERR_PTR(-ETIME);
@@ -1264,7 +1264,7 @@ xe_migrate_update_pgtables(struct xe_migrate *m,
 	 */
 	if (first_munmap_rebind) {
 		vm_dbg(&xe_vm_device(vm)->drm, "wait on first_munmap_rebind");
-		err = job_add_deps(job, &vm->resv,
+		err = job_add_deps(job, xe_vm_resv(vm),
 				   DMA_RESV_USAGE_BOOKKEEP);
 		if (err)
 			goto err_job;
drivers/gpu/drm/xe/xe_pt.c +3 -3
index 4167f666d98d..0f40f1950686 100644
@@ -1020,7 +1020,7 @@ static void xe_pt_commit_locks_assert(struct xe_vma *vma)
 	else if (!xe_vma_is_null(vma))
 		dma_resv_assert_held(xe_vma_bo(vma)->ttm.base.resv);
 
-	dma_resv_assert_held(&vm->resv);
+	dma_resv_assert_held(xe_vm_resv(vm));
 }
 
 static void xe_pt_commit_bind(struct xe_vma *vma,
@@ -1381,7 +1381,7 @@ __xe_pt_bind_vma(struct xe_gt *gt, struct xe_vma *vma, struct xe_engine *e,
 		}
 
 		/* add shared fence now for pagetable delayed destroy */
-		dma_resv_add_fence(&vm->resv, fence, !rebind &&
+		dma_resv_add_fence(xe_vm_resv(vm), fence, !rebind &&
 				   last_munmap_rebind ?
 				   DMA_RESV_USAGE_KERNEL :
 				   DMA_RESV_USAGE_BOOKKEEP);
@@ -1701,7 +1701,7 @@ __xe_pt_unbind_vma(struct xe_gt *gt, struct xe_vma *vma, struct xe_engine *e,
 		fence = &ifence->base.base;
 
 		/* add shared fence now for pagetable delayed destroy */
-		dma_resv_add_fence(&vm->resv, fence,
+		dma_resv_add_fence(xe_vm_resv(vm), fence,
 				   DMA_RESV_USAGE_BOOKKEEP);
 
 		/* This fence will be installed by caller when doing eviction */
drivers/gpu/drm/xe/xe_vm.c +17 -17
index 77220a17d29c..ab840f74b7a0 100644
@@ -307,7 +307,7 @@ static void resume_and_reinstall_preempt_fences(struct xe_vm *vm)
 	list_for_each_entry(e, &vm->preempt.engines, compute.link) {
 		e->ops->resume(e);
 
-		dma_resv_add_fence(&vm->resv, e->compute.pfence,
+		dma_resv_add_fence(xe_vm_resv(vm), e->compute.pfence,
 				   DMA_RESV_USAGE_BOOKKEEP);
 		xe_vm_fence_all_extobjs(vm, e->compute.pfence,
 					DMA_RESV_USAGE_BOOKKEEP);
@@ -345,7 +345,7 @@ int xe_vm_add_compute_engine(struct xe_vm *vm, struct xe_engine *e)
 
 	down_read(&vm->userptr.notifier_lock);
 
-	dma_resv_add_fence(&vm->resv, pfence,
+	dma_resv_add_fence(xe_vm_resv(vm), pfence,
 			   DMA_RESV_USAGE_BOOKKEEP);
 
 	xe_vm_fence_all_extobjs(vm, pfence, DMA_RESV_USAGE_BOOKKEEP);
@@ -603,7 +603,7 @@ static void preempt_rebind_work_func(struct work_struct *w)
 	}
 
 	/* Wait on munmap style VM unbinds */
-	wait = dma_resv_wait_timeout(&vm->resv,
+	wait = dma_resv_wait_timeout(xe_vm_resv(vm),
 				     DMA_RESV_USAGE_KERNEL,
 				     false, MAX_SCHEDULE_TIMEOUT);
 	if (wait <= 0) {
@@ -689,13 +689,13 @@ static bool vma_userptr_invalidate(struct mmu_interval_notifier *mni,
 	 * unbinds to complete, and those are attached as BOOKMARK fences
 	 * to the vm.
 	 */
-	dma_resv_iter_begin(&cursor, &vm->resv,
+	dma_resv_iter_begin(&cursor, xe_vm_resv(vm),
 			    DMA_RESV_USAGE_BOOKKEEP);
 	dma_resv_for_each_fence_unlocked(&cursor, fence)
 		dma_fence_enable_sw_signaling(fence);
 	dma_resv_iter_end(&cursor);
 
-	err = dma_resv_wait_timeout(&vm->resv,
+	err = dma_resv_wait_timeout(xe_vm_resv(vm),
 				    DMA_RESV_USAGE_BOOKKEEP,
 				    false, MAX_SCHEDULE_TIMEOUT);
 	XE_WARN_ON(err <= 0);
@@ -742,12 +742,12 @@ int xe_vm_userptr_pin(struct xe_vm *vm)
 	}
 
 	/* Take lock and move to rebind_list for rebinding. */
-	err = dma_resv_lock_interruptible(&vm->resv, NULL);
+	err = dma_resv_lock_interruptible(xe_vm_resv(vm), NULL);
 	if (err)
 		goto out_err;
 
 	list_splice_tail(&tmp_evict, &vm->rebind_list);
-	dma_resv_unlock(&vm->resv);
+	dma_resv_unlock(xe_vm_resv(vm));
 
 	return 0;
 
@@ -1085,7 +1085,6 @@ struct xe_vm *xe_vm_create(struct xe_device *xe, u32 flags)
 		return ERR_PTR(-ENOMEM);
 
 	kref_init(&vm->refcount);
-	dma_resv_init(&vm->resv);
 
 	vm->size = 1ull << xe_pt_shift(xe->info.vm_max_level + 1);
 
@@ -1120,12 +1119,13 @@ struct xe_vm *xe_vm_create(struct xe_device *xe, u32 flags)
 		xe_device_mem_access_get(xe);
 	}
 
-	err = dma_resv_lock_interruptible(&vm->resv, NULL);
+	drm_gpuva_manager_init(&vm->mgr, &xe->drm, "Xe VM", 0, vm->size, 0, 0,
+			       &gpuva_ops);
+
+	err = dma_resv_lock_interruptible(xe_vm_resv(vm), NULL);
 	if (err)
 		goto err_put;
 
-	drm_gpuva_manager_init(&vm->mgr, &xe->drm, "Xe VM", 0, vm->size, 0, 0,
-			       &gpuva_ops);
 	if (IS_DGFX(xe) && xe->info.vram_flags & XE_VRAM_FLAGS_NEED64K)
 		vm->flags |= XE_VM_FLAGS_64K;
 
@@ -1173,7 +1173,7 @@ struct xe_vm *xe_vm_create(struct xe_device *xe, u32 flags)
 
 		xe_pt_populate_empty(gt, vm, vm->pt_root[id]);
 	}
-	dma_resv_unlock(&vm->resv);
+	dma_resv_unlock(xe_vm_resv(vm));
 
 	/* Kernel migration VM shouldn't have a circular loop.. */
 	if (!(flags & XE_VM_FLAG_MIGRATION)) {
@@ -1230,10 +1230,10 @@ struct xe_vm *xe_vm_create(struct xe_device *xe, u32 flags)
 		if (vm->pt_root[id])
 			xe_pt_destroy(vm->pt_root[id], vm->flags, NULL);
 	}
-	dma_resv_unlock(&vm->resv);
+	dma_resv_unlock(xe_vm_resv(vm));
 	drm_gpuva_manager_destroy(&vm->mgr);
 err_put:
-	dma_resv_fini(&vm->resv);
+	dma_resv_fini(xe_vm_resv(vm));
 	kfree(vm);
 	if (!(flags & XE_VM_FLAG_MIGRATION)) {
 		xe_device_mem_access_put(xe);
@@ -1422,7 +1422,7 @@ static void vm_destroy_work_func(struct work_struct *w)
 
 	trace_xe_vm_free(vm);
 	dma_fence_put(vm->rebind_fence);
-	dma_resv_fini(&vm->resv);
+	dma_resv_fini(xe_vm_resv(vm));
 	kfree(vm);
 }
 
@@ -3294,7 +3294,7 @@ int xe_vm_lock(struct xe_vm *vm, struct ww_acquire_ctx *ww,
 
 void xe_vm_unlock(struct xe_vm *vm, struct ww_acquire_ctx *ww)
 {
-	dma_resv_unlock(&vm->resv);
+	dma_resv_unlock(xe_vm_resv(vm));
 	ww_acquire_fini(ww);
 }
 
@@ -3327,7 +3327,7 @@ int xe_vm_invalidate_vma(struct xe_vma *vma)
 			WARN_ON_ONCE(!mmu_interval_check_retry
 				     (&vma->userptr.notifier,
 				      vma->userptr.notifier_seq));
-			WARN_ON_ONCE(!dma_resv_test_signaled(&xe_vma_vm(vma)->resv,
+			WARN_ON_ONCE(!dma_resv_test_signaled(xe_vma_resv(vma),
 							     DMA_RESV_USAGE_BOOKKEEP));
 
 		} else {
drivers/gpu/drm/xe/xe_vm.h +11 -1
index cbbe95d6291f..81a9271be728 100644
@@ -57,6 +57,11 @@ static inline struct xe_device *xe_vm_device(struct xe_vm *vm)
 	return container_of(vm->mgr.drm, struct xe_device, drm);
 }
 
+static inline struct dma_resv *xe_vm_resv(struct xe_vm *vm)
+{
+	return &vm->mgr.resv;
+}
+
 static inline struct xe_vm *gpuva_to_vm(struct drm_gpuva *gpuva)
 {
 	return container_of(gpuva->mgr, struct xe_vm, mgr);
@@ -112,6 +117,11 @@ static inline struct xe_device *xe_vma_device(struct xe_vma *vma)
 	return xe_vm_device(xe_vma_vm(vma));
 }
 
+static inline struct dma_resv *xe_vma_resv(struct xe_vma *vma)
+{
+	return xe_vm_resv(xe_vma_vm(vma));
+}
+
 static inline bool xe_vma_read_only(struct xe_vma *vma)
 {
 	return vma->gpuva.flags & XE_VMA_READ_ONLY;
@@ -122,7 +132,7 @@ static inline u64 xe_vma_userptr(struct xe_vma *vma)
 	return vma->gpuva.gem.offset;
 }
 
-#define xe_vm_assert_held(vm) dma_resv_assert_held(&(vm)->resv)
+#define xe_vm_assert_held(vm) dma_resv_assert_held(&(vm)->mgr.resv)
 
 u64 xe_vm_pdp4_descriptor(struct xe_vm *vm, struct xe_gt *full_gt);
 
drivers/gpu/drm/xe/xe_vm_types.h +1 -5
index fca42910dcae..26571d171a43 100644
@@ -8,7 +8,6 @@
 
 #include <drm/drm_gpuva_mgr.h>
 
-#include <linux/dma-resv.h>
 #include <linux/kref.h>
 #include <linux/mmu_notifier.h>
 #include <linux/scatterlist.h>
@@ -131,7 +130,7 @@ struct xe_vma {
 
 struct xe_device;
 
-#define xe_vm_assert_held(vm) dma_resv_assert_held(&(vm)->resv)
+#define xe_vm_assert_held(vm) dma_resv_assert_held(&(vm)->mgr.resv)
 
 struct xe_vm {
 	/** @mgr: base GPUVA used to track VMAs */
@@ -142,9 +141,6 @@ struct xe_vm {
 	/* engine used for (un)binding vma's */
 	struct xe_engine *eng[XE_MAX_GT];
 
-	/** Protects @rebind_list and the page-table structures */
-	struct dma_resv resv;
-
 	/** @lru_bulk_move: Bulk LRU move list for this VM's BOs */
 	struct ttm_lru_bulk_move lru_bulk_move;
 
include/drm/drm_gpuva_mgr.h +6 -0
index 55b0acfdcc44..010b649e363f 100644
@@ -25,6 +25,7 @@
  * OTHER DEALINGS IN THE SOFTWARE.
  */
 
+#include <linux/dma-resv.h>
 #include <linux/maple_tree.h>
 #include <linux/mm.h>
 #include <linux/rbtree.h>
@@ -177,6 +178,11 @@ struct drm_gpuva_manager {
 	 */
 	const char *name;
 
+	/**
+	 * @resv: dma-resv for all private GEMs mapped in this address space
+	 */
+	struct dma_resv resv;
+
 	/**
 	 * @mm_start: start of the VA space
 	 */
Signature · 1 line
2.34.1
author maintainer
#23 lore raw

PATCH 23/30 drm/gpuva: Add support for extobj

In reply to Matthew Brost

Manager maintains lists of GPUVA with extobjs.

---
 drivers/gpu/drm/drm_gpuva_mgr.c  | 45 +++++++++++++--
 drivers/gpu/drm/xe/xe_exec.c     | 24 ++++----
 drivers/gpu/drm/xe/xe_vm.c       | 99 +++++---------------------------
 drivers/gpu/drm/xe/xe_vm.h       |  3 -
 drivers/gpu/drm/xe/xe_vm_types.h | 16 ------
 include/drm/drm_gpuva_mgr.h      | 39 ++++++++++++-
 6 files changed, 105 insertions(+), 121 deletions(-)
drivers/gpu/drm/drm_gpuva_mgr.c +41 -4
index 6d2d0f4d5018..e8cd6e154336 100644
@@ -447,6 +447,9 @@ drm_gpuva_manager_init(struct drm_gpuva_manager *mgr,
 
 	memset(&mgr->kernel_alloc_node, 0, sizeof(struct drm_gpuva));
 
+	mgr->extobj.entries = 0;
+	INIT_LIST_HEAD(&mgr->extobj.list);
+
 	if (reserve_range) {
 		mgr->kernel_alloc_node.va.addr = reserve_offset;
 		mgr->kernel_alloc_node.va.range = reserve_range;
@@ -706,7 +709,8 @@ EXPORT_SYMBOL(drm_gpuva_remove);
  * @va: the &drm_gpuva to link
  *
  * This adds the given &va to the GPU VA list of the &drm_gem_object it is
- * associated with.
+ * associated with and to &drm_gpuva_manager.extobj.list if GPUVA maps an
+ * extobj.
  *
  * This function expects the caller to protect the GEM's GPUVA list against
  * concurrent access.
@@ -714,8 +718,14 @@ EXPORT_SYMBOL(drm_gpuva_remove);
 void
 drm_gpuva_link(struct drm_gpuva *va)
 {
-	if (likely(va->gem.obj))
+	if (likely(va->gem.obj)) {
 		list_add_tail(&va->gem.entry, &va->gem.obj->gpuva.list);
+		if (va->flags & DRM_GPUVA_EXTOBJ) {
+			list_add_tail(&va->gem.extobj_link,
+				      &va->mgr->extobj.list);
+			++va->mgr->extobj.entries;
+		}
+	}
 }
 EXPORT_SYMBOL(drm_gpuva_link);
 
@@ -724,7 +734,8 @@ EXPORT_SYMBOL(drm_gpuva_link);
  * @va: the &drm_gpuva to unlink
  *
  * This removes the given &va from the GPU VA list of the &drm_gem_object it is
- * associated with.
+ * associated with and from &drm_gpuva_manager.extobj.list if GPUVA maps an
+ * extobj.
  *
  * This function expects the caller to protect the GEM's GPUVA list against
  * concurrent access.
@@ -732,8 +743,13 @@ EXPORT_SYMBOL(drm_gpuva_link);
 void
 drm_gpuva_unlink(struct drm_gpuva *va)
 {
-	if (likely(va->gem.obj))
+	if (likely(va->gem.obj)) {
 		list_del_init(&va->gem.entry);
+		if (va->flags & DRM_GPUVA_EXTOBJ) {
+			list_del(&va->gem.extobj_link);
+			--va->mgr->extobj.entries;
+		}
+	}
 }
 EXPORT_SYMBOL(drm_gpuva_unlink);
 
@@ -871,6 +887,27 @@ drm_gpuva_interval_empty(struct drm_gpuva_manager *mgr, u64 addr, u64 range)
 }
 EXPORT_SYMBOL(drm_gpuva_interval_empty);
 
+/**
+ * drm_gpuva_add_fence - add fence to private and all extobj dma-resv
+ * @mgr: the &drm_gpuva_manager to add a fence to
+ * @fence: fence to add
+ * @private_usage: private dma-resv usage
+ * @extobj_usage: extobj dma-resv usage
+ *
+ * Returns: true if the interval is empty, false otherwise
+ */
+void drm_gpuva_add_fence(struct drm_gpuva_manager *mgr, struct dma_fence *fence,
+			 enum dma_resv_usage private_usage,
+			 enum dma_resv_usage extobj_usage)
+{
+	struct drm_gpuva *gpuva;
+
+	dma_resv_add_fence(&mgr->resv, fence, private_usage);
+	drm_gpuva_for_each_extobj(gpuva, mgr)
+		dma_resv_add_fence(gpuva->gem.obj->resv, fence, extobj_usage);
+}
+EXPORT_SYMBOL(drm_gpuva_add_fence);
+
 /**
  * drm_gpuva_map - helper to insert a &drm_gpuva from &drm_gpuva_fn_ops
  * callbacks
drivers/gpu/drm/xe/xe_exec.c +11 -13
index b352fd6e1f4d..2ae02f1500d5 100644
@@ -353,19 +353,17 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
 	 * the job and let the DRM scheduler / backend clean up the job.
 	 */
 	xe_sched_job_arm(job);
-	if (!xe_vm_no_dma_fences(vm)) {
-		/* Block userptr invalidations / BO eviction */
-		dma_resv_add_fence(xe_vm_resv(vm),
-				   &job->drm.s_fence->finished,
-				   DMA_RESV_USAGE_BOOKKEEP);
-
-		/*
-		 * Make implicit sync work across drivers, assuming all external
-		 * BOs are written as we don't pass in a read / write list.
-		 */
-		xe_vm_fence_all_extobjs(vm, &job->drm.s_fence->finished,
-					DMA_RESV_USAGE_WRITE);
-	}
+
+	/*
+	 * Block userptr invalidations / BO eviction
+	 *
+	 * Make implicit sync work across drivers, assuming all external BOss
+	 * are written as we don't pass in a read / write list.
+	 */
+	if (!xe_vm_no_dma_fences(vm))
+		drm_gpuva_add_fence(&vm->mgr, &job->drm.s_fence->finished,
+				    DMA_RESV_USAGE_BOOKKEEP,
+				    DMA_RESV_USAGE_WRITE);
 
 	for (i = 0; i < num_syncs; i++)
 		xe_sync_entry_signal(&syncs[i], job,
drivers/gpu/drm/xe/xe_vm.c +15 -84

187 lines not on this page · raw message

drivers/gpu/drm/xe/xe_vm.h +0 -3
index 81a9271be728..12de652d8d1c 100644
@@ -232,9 +232,6 @@ void xe_vm_unlock_dma_resv(struct xe_vm *vm,
 			   struct ww_acquire_ctx *ww,
 			   struct list_head *objs);
 
-void xe_vm_fence_all_extobjs(struct xe_vm *vm, struct dma_fence *fence,
-			     enum dma_resv_usage usage);
-
 int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id);
 
 #if IS_ENABLED(CONFIG_DRM_XE_DEBUG_VM)
drivers/gpu/drm/xe/xe_vm_types.h +0 -16
index 26571d171a43..0b59bde3bc4e 100644
@@ -113,14 +113,6 @@ struct xe_vma {
 		u8 gt_invalidated;
 	} usm;
 
-	struct {
-		/**
-		 * @extobj.link: Link into vm's external object list.
-		 * protected by the vm lock.
-		 */
-		struct list_head link;
-	} extobj;
-
 	/**
 	 * @userptr: user pointer state, only allocated for VMAs that are
 	 * user pointers
@@ -189,14 +181,6 @@ struct xe_vm {
 	 */
 	struct work_struct destroy_work;
 
-	/** @extobj: bookkeeping for external objects. Protected by the vm lock */
-	struct {
-		/** @enties: number of external BOs attached this VM */
-		u32 entries;
-		/** @list: list of vmas with external bos attached */
-		struct list_head list;
-	} extobj;
-
 	/** @async_ops: async VM operations (bind / unbinds) */
 	struct {
 		/** @list: list of pending async VM ops */
include/drm/drm_gpuva_mgr.h +38 -1
index 010b649e363f..57861a7ed504 100644
@@ -54,10 +54,18 @@ enum drm_gpuva_flags {
 	 */
 	DRM_GPUVA_SPARSE = (1 << 1),
 
+	/**
+	 * @DRM_GPUVA_EXTOBJ:
+	 *
+	 * Flag indicating that the &drm_gpuva is a mapping of an extobj (GEN
+	 * not tied to a single address space).
+	 */
+	DRM_GPUVA_EXTOBJ = (1 << 2),
+
 	/**
 	 * @DRM_GPUVA_USERBITS: user defined bits
 	 */
-	DRM_GPUVA_USERBITS = (1 << 2),
+	DRM_GPUVA_USERBITS = (1 << 3),
 };
 
 /**
@@ -112,6 +120,12 @@ struct drm_gpuva {
 		 * @entry: the &list_head to attach this object to a &drm_gem_object
 		 */
 		struct list_head entry;
+
+		/**
+		 * @extobj_link: the &list_head to attach this object to a
+		 * @drm_gpuva_manager.extobj.list
+		 */
+		struct list_head extobj_link;
 	} gem;
 };
 
@@ -134,6 +148,10 @@ struct drm_gpuva *drm_gpuva_find_next(struct drm_gpuva_manager *mgr, u64 end);
 
 bool drm_gpuva_interval_empty(struct drm_gpuva_manager *mgr, u64 addr, u64 range);
 
+void drm_gpuva_add_fence(struct drm_gpuva_manager *mgr, struct dma_fence *fence,
+			 enum dma_resv_usage private_usage,
+			 enum dma_resv_usage extobj_usage);
+
 /**
  * drm_gpuva_evict - sets whether the backing GEM of this &drm_gpuva is evicted
  * @va: the &drm_gpuva to set the evict flag for
@@ -206,6 +224,17 @@ struct drm_gpuva_manager {
 	 */
 	struct drm_gpuva kernel_alloc_node;
 
+	/** @extobj: bookkeeping for external GEMs */
+	struct {
+		/**
+		 * @entries: number of external GEMs attached this address
+		 * space
+		 */
+		u32 entries;
+		/** @list: list of GPUVAs with external GEMs attached */
+		struct list_head list;
+	} extobj;
+
 	/**
 	 * @ops: &drm_gpuva_fn_ops providing the split/merge steps to drivers
 	 */
@@ -509,6 +538,14 @@ struct drm_gpuva_ops {
 	struct list_head list;
 };
 
+/**
+ * drm_gpuva_for_each_op - iterator to walk over &drm_gpuva of extobjs
+ * @va: &drm_gpuva to assign in each iteration step
+ * @mgr: &drm_gpuva_manager to walk extobj lisy
+ */
+#define drm_gpuva_for_each_extobj(va, mgr) \
+	list_for_each_entry(va, &(mgr)->extobj.list, gem.extobj_link)
+
 /**
  * drm_gpuva_for_each_op - iterator to walk over &drm_gpuva_ops
  * @op: &drm_gpuva_op to assign in each iteration step
Signature · 1 line
2.34.1
author maintainer
#24 lore raw

PATCH 24/30 drm/xe: Userptr refactor

In reply to Matthew Brost

Add GPUVA userptr flag, add GPUVA userptr sub-struct, and drop sg pointer. A larger follow on cleanup may push more of userptr implementation to GPUVA.

---
 drivers/gpu/drm/xe/xe_pt.c       |  6 +--
 drivers/gpu/drm/xe/xe_vm.c       | 41 +++++++++++----------
 drivers/gpu/drm/xe/xe_vm.h       | 23 +++++++-----
 drivers/gpu/drm/xe/xe_vm_types.h | 20 +++++-----
 include/drm/drm_gpuva_mgr.h      | 63 +++++++++++++++++++++-----------
 5 files changed, 89 insertions(+), 64 deletions(-)
drivers/gpu/drm/xe/xe_pt.c +3 -3
index 0f40f1950686..964baa24eba3 100644
@@ -92,8 +92,8 @@ static dma_addr_t vma_addr(struct xe_vma *vma, u64 offset,
 		page = offset >> PAGE_SHIFT;
 		offset &= (PAGE_SIZE - 1);
 
-		xe_res_first_sg(vma->userptr.sg, page << PAGE_SHIFT, page_size,
-				&cur);
+		xe_res_first_sg(&vma->userptr.sgt, page << PAGE_SHIFT,
+				page_size, &cur);
 		return xe_res_dma(&cur) + offset;
 	} else {
 		return xe_bo_addr(xe_vma_bo(vma), offset, page_size, is_vram);
@@ -813,7 +813,7 @@ xe_pt_stage_bind(struct xe_gt *gt, struct xe_vma *vma,
 	xe_bo_assert_held(bo);
 	if (!xe_vma_is_null(vma)) {
 		if (xe_vma_is_userptr(vma))
-			xe_res_first_sg(vma->userptr.sg, 0, xe_vma_size(vma),
+			xe_res_first_sg(&vma->userptr.sgt, 0, xe_vma_size(vma),
 					&curs);
 		else if (xe_bo_is_vram(bo) || xe_bo_is_stolen(bo))
 			xe_res_first(bo->ttm.resource, xe_vma_bo_offset(vma),
drivers/gpu/drm/xe/xe_vm.c +21 -20

115 lines not on this page · raw message

drivers/gpu/drm/xe/xe_vm.h +14 -9
index 12de652d8d1c..f279fa622260 100644
@@ -101,12 +101,6 @@ static inline u64 xe_vma_bo_offset(struct xe_vma *vma)
 	return vma->gpuva.gem.offset;
 }
 
-static inline struct xe_bo *xe_vma_bo(struct xe_vma *vma)
-{
-	return !vma->gpuva.gem.obj ? NULL :
-		container_of(vma->gpuva.gem.obj, struct xe_bo, ttm.base);
-}
-
 static inline struct xe_vm *xe_vma_vm(struct xe_vma *vma)
 {
 	return container_of(vma->gpuva.mgr, struct xe_vm, mgr);
@@ -129,7 +123,7 @@ static inline bool xe_vma_read_only(struct xe_vma *vma)
 
 static inline u64 xe_vma_userptr(struct xe_vma *vma)
 {
-	return vma->gpuva.gem.offset;
+	return vma->gpuva.userptr.address;
 }
 
 #define xe_vm_assert_held(vm) dma_resv_assert_held(&(vm)->mgr.resv)
@@ -197,12 +191,18 @@ static inline void xe_vm_reactivate_rebind(struct xe_vm *vm)
 
 static inline bool xe_vma_is_null(struct xe_vma *vma)
 {
-	return vma->gpuva.flags & XE_VMA_NULL;
+	return vma->gpuva.flags & DRM_GPUVA_SPARSE;
 }
 
 static inline bool xe_vma_is_userptr(struct xe_vma *vma)
 {
-	return !xe_vma_bo(vma) && !xe_vma_is_null(vma);
+	return vma->gpuva.flags & DRM_GPUVA_USERPTR;
+}
+
+static inline struct xe_bo *xe_vma_bo(struct xe_vma *vma)
+{
+	return xe_vma_is_null(vma) || xe_vma_is_userptr(vma) ? NULL :
+		container_of(vma->gpuva.gem.obj, struct xe_bo, ttm.base);
 }
 
 static inline bool xe_vma_has_no_bo(struct xe_vma *vma)
@@ -210,6 +210,11 @@ static inline bool xe_vma_has_no_bo(struct xe_vma *vma)
 	return !xe_vma_bo(vma);
 }
 
+static inline bool xe_vma_userptr_sg_mapped(struct xe_vma *vma)
+{
+	return vma->gpuva.flags & XE_VMA_USERPTR_SG_MAPPED;
+}
+
 int xe_vma_userptr_pin_pages(struct xe_vma *vma);
 
 int xe_vma_userptr_check_repin(struct xe_vma *vma);
drivers/gpu/drm/xe/xe_vm_types.h +9 -11
index 0b59bde3bc4e..ce1260b8d3ef 100644
@@ -23,15 +23,15 @@ struct xe_vm;
 #define TEST_VM_ASYNC_OPS_ERROR
 #define FORCE_ASYNC_OP_ERROR	BIT(31)
 
-#define XE_VMA_READ_ONLY	DRM_GPUVA_USERBITS
-#define XE_VMA_DESTROYED	(DRM_GPUVA_USERBITS << 1)
-#define XE_VMA_ATOMIC_PTE_BIT	(DRM_GPUVA_USERBITS << 2)
-#define XE_VMA_FIRST_REBIND	(DRM_GPUVA_USERBITS << 3)
-#define XE_VMA_LAST_REBIND	(DRM_GPUVA_USERBITS << 4)
-#define XE_VMA_NULL		(DRM_GPUVA_USERBITS << 5)
-#define XE_VMA_PTE_4K		(DRM_GPUVA_USERBITS << 6)
-#define XE_VMA_PTE_2M		(DRM_GPUVA_USERBITS << 7)
-#define XE_VMA_PTE_1G		(DRM_GPUVA_USERBITS << 8)
+#define XE_VMA_READ_ONLY		DRM_GPUVA_USERBITS
+#define XE_VMA_DESTROYED		(DRM_GPUVA_USERBITS << 1)
+#define XE_VMA_ATOMIC_PTE_BIT		(DRM_GPUVA_USERBITS << 2)
+#define XE_VMA_FIRST_REBIND		(DRM_GPUVA_USERBITS << 3)
+#define XE_VMA_LAST_REBIND		(DRM_GPUVA_USERBITS << 4)
+#define XE_VMA_USERPTR_SG_MAPPED	(DRM_GPUVA_USERBITS << 5)
+#define XE_VMA_PTE_4K			(DRM_GPUVA_USERBITS << 6)
+#define XE_VMA_PTE_2M			(DRM_GPUVA_USERBITS << 7)
+#define XE_VMA_PTE_1G			(DRM_GPUVA_USERBITS << 8)
 
 /** struct xe_userptr - User pointer */
 struct xe_userptr {
@@ -41,8 +41,6 @@ struct xe_userptr {
 	struct mmu_interval_notifier notifier;
 	/** @sgt: storage for a scatter gather table */
 	struct sg_table sgt;
-	/** @sg: allocated scatter gather table */
-	struct sg_table *sg;
 	/** @notifier_seq: notifier sequence number */
 	unsigned long notifier_seq;
 	/**
include/drm/drm_gpuva_mgr.h +42 -21

83 lines not on this page · raw message

Signature · 1 line
2.34.1
author maintainer
#25 lore raw

PATCH 25/30 drm: execution context for GEM buffers v3

In reply to Matthew Brost

From: Christian König <[email protected]>

This adds the infrastructure for an execution context for GEM buffers which is similar to the existinc TTMs execbuf util and intended to replace it in the long term.

The basic functionality is that we abstracts the necessary loop to lock many different GEM buffers with automated deadlock and duplicate handling.

v2: drop xarray and use dynamic resized array instead, the locking
overhead is unecessary and measureable.
v3: drop duplicate tracking, radeon is really the only one needing that.

---
 Documentation/gpu/drm-mm.rst |  12 ++
 drivers/gpu/drm/Kconfig      |   6 +
 drivers/gpu/drm/Makefile     |   2 +
 drivers/gpu/drm/drm_exec.c   | 248 +++++++++++++++++++++++++++++++++++
 include/drm/drm_exec.h       | 115 ++++++++++++++++
 5 files changed, 383 insertions(+)
 create mode 100644 drivers/gpu/drm/drm_exec.c
 create mode 100644 include/drm/drm_exec.h
Documentation/gpu/drm-mm.rst +12 -0
index fe40ee686f6e..c9f120cfe730 100644
@@ -524,6 +524,18 @@ DRM Sync Objects
 .. kernel-doc:: drivers/gpu/drm/drm_syncobj.c
    :export:
 
+DRM Execution context
+=====================
+
+.. kernel-doc:: drivers/gpu/drm/drm_exec.c
+   :doc: Overview
+
+.. kernel-doc:: include/drm/drm_exec.h
+   :internal:
+
+.. kernel-doc:: drivers/gpu/drm/drm_exec.c
+   :export:
+
 GPU Scheduler
 =============
 
drivers/gpu/drm/Kconfig +6 -0
index e928284b4357..39c9d079d52a 100644
@@ -201,6 +201,12 @@ config DRM_TTM
 	  GPU memory types. Will be enabled automatically if a device driver
 	  uses it.
 
+config DRM_EXEC
+	tristate
+	depends on DRM
+	help
+	  Execution context for command submissions
+
 config DRM_BUDDY
 	tristate
 	depends on DRM
drivers/gpu/drm/Makefile +2 -0
index ad6267273503..ab728632d8a2 100644
@@ -80,6 +80,8 @@ obj-$(CONFIG_DRM_PANEL_ORIENTATION_QUIRKS) += drm_panel_orientation_quirks.o
 #
 # Memory-management helpers
 #
+#
+obj-$(CONFIG_DRM_EXEC) += drm_exec.o
 
 obj-$(CONFIG_DRM_BUDDY) += drm_buddy.o
 
drivers/gpu/drm/drm_exec.c new file +248 -0

248 lines not on this page · raw message

include/drm/drm_exec.h new file +115 -0

115 lines not on this page · raw message

Signature · 1 line
2.34.1
maintainer
#26 lore raw

Re: [Intel-xe] [PATCH 25/30] drm: execution context for GEM buffers v3

Just FYI I found time to work a bit on this last week and will probably send out the next version this week.

Christian.

Am 01.05.23 um 09:51 schrieb Matthew Brost:

Quoted 452 lines

From: Christian König <[email protected]>

This adds the infrastructure for an execution context for GEM buffers which is similar to the existinc TTMs execbuf util and intended to replace it in the long term.

The basic functionality is that we abstracts the necessary loop to lock many different GEM buffers with automated deadlock and duplicate handling.

v2: drop xarray and use dynamic resized array instead, the locking
overhead is unecessary and measureable.
v3: drop duplicate tracking, radeon is really the only one needing that.

---
  Documentation/gpu/drm-mm.rst |  12 ++
  drivers/gpu/drm/Kconfig      |   6 +
  drivers/gpu/drm/Makefile     |   2 +
  drivers/gpu/drm/drm_exec.c   | 248 +++++++++++++++++++++++++++++++++++
  include/drm/drm_exec.h       | 115 ++++++++++++++++
  5 files changed, 383 insertions(+)

create mode 100644 drivers/gpu/drm/drm_exec.c
create mode 100644 include/drm/drm_exec.h

Documentation/gpu/drm-mm.rst +12 -0
index fe40ee686f6e..c9f120cfe730 100644
@@ -524,6 +524,18 @@ DRM Sync Objects
  .. kernel-doc:: drivers/gpu/drm/drm_syncobj.c
     :export:
 
+DRM Execution context
+=====================
+
+.. kernel-doc:: drivers/gpu/drm/drm_exec.c
+   :doc: Overview
+
+.. kernel-doc:: include/drm/drm_exec.h
+   :internal:
+
+.. kernel-doc:: drivers/gpu/drm/drm_exec.c
+   :export:
+
  GPU Scheduler
  =============
 
drivers/gpu/drm/Kconfig +6 -0
index e928284b4357..39c9d079d52a 100644
@@ -201,6 +201,12 @@ config DRM_TTM
  	  GPU memory types. Will be enabled automatically if a device driver
  	  uses it.
 
+config DRM_EXEC
+	tristate
+	depends on DRM
+	help
+	  Execution context for command submissions
+
  config DRM_BUDDY
  	tristate
  	depends on DRM
drivers/gpu/drm/Makefile +2 -0
index ad6267273503..ab728632d8a2 100644
@@ -80,6 +80,8 @@ obj-$(CONFIG_DRM_PANEL_ORIENTATION_QUIRKS) += drm_panel_orientation_quirks.o
  #
  # Memory-management helpers
  #
+#
+obj-$(CONFIG_DRM_EXEC) += drm_exec.o
 
  obj-$(CONFIG_DRM_BUDDY) += drm_buddy.o
 
drivers/gpu/drm/drm_exec.c new file +248 -0
new file mode 100644
index 000000000000..f645d22a0863
@@ -0,0 +1,248 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+
+#include <drm/drm_exec.h>
+#include <drm/drm_gem.h>
+#include <linux/dma-resv.h>
+
+/**
+ * DOC: Overview
+ *
+ * This component mainly abstracts the retry loop necessary for locking
+ * multiple GEM objects while preparing hardware operations (e.g. command
+ * submissions, page table updates etc..).
+ *
+ * If a contention is detected while locking a GEM object the cleanup procedure
+ * unlocks all previously locked GEM objects and locks the contended one first
+ * before locking any further objects.
+ *
+ * After an object is locked fences slots can optionally be reserved on the
+ * dma_resv object inside the GEM object.
+ *
+ * A typical usage pattern should look like this::
+ *
+ *	struct drm_gem_object *obj;
+ *	struct drm_exec exec;
+ *	unsigned long index;
+ *	int ret;
+ *
+ *	drm_exec_init(&exec, true);
+ *	drm_exec_while_not_all_locked(&exec) {
+ *		ret = drm_exec_prepare_obj(&exec, boA, 1);
+ *		drm_exec_continue_on_contention(&exec);
+ *		if (ret)
+ *			goto error;
+ *
+ *		ret = drm_exec_lock(&exec, boB, 1);
+ *		drm_exec_continue_on_contention(&exec);
+ *		if (ret)
+ *			goto error;
+ *	}
+ *
+ *	drm_exec_for_each_locked_object(&exec, index, obj) {
+ *		dma_resv_add_fence(obj->resv, fence, DMA_RESV_USAGE_READ);
+ *		...
+ *	}
+ *	drm_exec_fini(&exec);
+ *
+ * See struct dma_exec for more details.
+ */
+
+/* Dummy value used to initially enter the retry loop */
+#define DRM_EXEC_DUMMY (void*)~0
+
+/* Unlock all objects and drop references */
+static void drm_exec_unlock_all(struct drm_exec *exec)
+{
+	struct drm_gem_object *obj;
+	unsigned long index;
+
+	drm_exec_for_each_locked_object(exec, index, obj) {
+		dma_resv_unlock(obj->resv);
+		drm_gem_object_put(obj);
+	}
+
+	if (exec->prelocked) {
+		drm_gem_object_put(exec->prelocked);
+		exec->prelocked = NULL;
+	}
+}
+
+/**
+ * drm_exec_init - initialize a drm_exec object
+ * @exec: the drm_exec object to initialize
+ * @interruptible: if locks should be acquired interruptible
+ *
+ * Initialize the object and make sure that we can track locked and duplicate
+ * objects.
+ */
+void drm_exec_init(struct drm_exec *exec, bool interruptible)
+{
+	exec->interruptible = interruptible;
+	exec->objects = kmalloc(PAGE_SIZE, GFP_KERNEL);
+
+	/* If allocation here fails, just delay that till the first use */
+	exec->max_objects = exec->objects ? PAGE_SIZE / sizeof(void *) : 0;
+	exec->num_objects = 0;
+	exec->contended = DRM_EXEC_DUMMY;
+	exec->prelocked = NULL;
+}
+EXPORT_SYMBOL(drm_exec_init);
+
+/**
+ * drm_exec_fini - finalize a drm_exec object
+ * @exec: the drm_exec object to finilize
+ *
+ * Unlock all locked objects, drop the references to objects and free all memory
+ * used for tracking the state.
+ */
+void drm_exec_fini(struct drm_exec *exec)
+{
+	drm_exec_unlock_all(exec);
+	kvfree(exec->objects);
+	if (exec->contended != DRM_EXEC_DUMMY) {
+		drm_gem_object_put(exec->contended);
+		ww_acquire_fini(&exec->ticket);
+	}
+}
+EXPORT_SYMBOL(drm_exec_fini);
+
+/**
+ * drm_exec_cleanup - cleanup when contention is detected
+ * @exec: the drm_exec object to cleanup
+ *
+ * Cleanup the current state and return true if we should stay inside the retry
+ * loop, false if there wasn't any contention detected and we can keep the
+ * objects locked.
+ */
+bool drm_exec_cleanup(struct drm_exec *exec)
+{
+	if (likely(!exec->contended)) {
+		ww_acquire_done(&exec->ticket);
+		return false;
+	}
+
+	if (likely(exec->contended == DRM_EXEC_DUMMY)) {
+		exec->contended = NULL;
+		ww_acquire_init(&exec->ticket, &reservation_ww_class);
+		return true;
+	}
+
+	drm_exec_unlock_all(exec);
+	exec->num_objects = 0;
+	return true;
+}
+EXPORT_SYMBOL(drm_exec_cleanup);
+
+/* Track the locked object in the xa and reserve fences */
+static int drm_exec_obj_locked(struct drm_exec *exec,
+			       struct drm_gem_object *obj)
+{
+	if (unlikely(exec->num_objects == exec->max_objects)) {
+		size_t size = exec->max_objects * sizeof(void *);
+		void *tmp;
+
+		tmp = kvrealloc(exec->objects, size, size + PAGE_SIZE,
+				GFP_KERNEL);
+		if (!tmp)
+			return -ENOMEM;
+
+		exec->objects = tmp;
+		exec->max_objects += PAGE_SIZE / sizeof(void *);
+	}
+	drm_gem_object_get(obj);
+	exec->objects[exec->num_objects++] = obj;
+
+	return 0;
+}
+
+/* Make sure the contended object is locked first */
+static int drm_exec_lock_contended(struct drm_exec *exec)
+{
+	struct drm_gem_object *obj = exec->contended;
+	int ret;
+
+	if (likely(!obj))
+		return 0;
+
+	if (exec->interruptible) {
+		ret = dma_resv_lock_slow_interruptible(obj->resv,
+						       &exec->ticket);
+		if (unlikely(ret))
+			goto error_dropref;
+	} else {
+		dma_resv_lock_slow(obj->resv, &exec->ticket);
+	}
+
+	ret = drm_exec_obj_locked(exec, obj);
+	if (unlikely(ret)) {
+		dma_resv_unlock(obj->resv);
+		goto error_dropref;
+	}
+
+	swap(exec->prelocked, obj);
+
+error_dropref:
+	/* Always cleanup the contention so that error handling can kick in */
+	drm_gem_object_put(obj);
+	exec->contended = NULL;
+	return ret;
+}
+
+/**
+ * drm_exec_prepare_obj - prepare a GEM object for use
+ * @exec: the drm_exec object with the state
+ * @obj: the GEM object to prepare
+ * @num_fences: how many fences to reserve
+ *
+ * Prepare a GEM object for use by locking it and reserving fence slots. All
+ * succesfully locked objects are put into the locked container. Duplicates
+ * detected as well and automatically moved into the duplicates container.
+ *
+ * Returns: -EDEADLK if a contention is detected, -ENOMEM when memory
+ * allocation failed and zero for success.
+ */
+int drm_exec_prepare_obj(struct drm_exec *exec, struct drm_gem_object *obj,
+			 unsigned int num_fences)
+{
+	int ret;
+
+	ret = drm_exec_lock_contended(exec);
+	if (unlikely(ret))
+		return ret;
+
+	if (exec->prelocked == obj) {
+		drm_gem_object_put(exec->prelocked);
+		exec->prelocked = NULL;
+
+		return dma_resv_reserve_fences(obj->resv, num_fences);
+	}
+
+	if (exec->interruptible)
+		ret = dma_resv_lock_interruptible(obj->resv, &exec->ticket);
+	else
+		ret = dma_resv_lock(obj->resv, &exec->ticket);
+
+	if (unlikely(ret == -EDEADLK)) {
+		drm_gem_object_get(obj);
+		exec->contended = obj;
+		return -EDEADLK;
+	}
+
+	if (unlikely(ret))
+		return ret;
+
+	ret = drm_exec_obj_locked(exec, obj);
+	if (ret)
+		goto error_unlock;
+
+	/* Keep locked when reserving fences fails */
+	return dma_resv_reserve_fences(obj->resv, num_fences);
+
+error_unlock:
+	dma_resv_unlock(obj->resv);
+	return ret;
+}
+EXPORT_SYMBOL(drm_exec_prepare_obj);
+
+MODULE_DESCRIPTION("DRM execution context");
+MODULE_LICENSE("Dual MIT/GPL");
include/drm/drm_exec.h new file +115 -0
new file mode 100644
index 000000000000..65e518c01db3
@@ -0,0 +1,115 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+
+#ifndef __DRM_EXEC_H__
+#define __DRM_EXEC_H__
+
+#include <linux/ww_mutex.h>
+
+struct drm_gem_object;
+
+/**
+ * struct drm_exec - Execution context
+ */
+struct drm_exec {
+	/**
+	 * @interruptible: If locks should be taken interruptible
+	 */
+	bool			interruptible;
+
+	/**
+	 * @ticket: WW ticket used for acquiring locks
+	 */
+	struct ww_acquire_ctx	ticket;
+
+	/**
+	 * @num_objects: number of objects locked
+	 */
+	unsigned int		num_objects;
+
+	/**
+	 * @max_objects: maximum objects in array
+	 */
+	unsigned int		max_objects;
+
+	/**
+	 * @objects: array of the locked objects
+	 */
+	struct drm_gem_object	**objects;
+
+	/**
+	 * @contended: contended GEM object we backet of for
+	 */
+	struct drm_gem_object	*contended;
+
+	/**
+	 * @prelocked: already locked GEM object because of contention
+	 */
+	struct drm_gem_object *prelocked;
+};
+
+/**
+ * drm_exec_for_each_locked_object - iterate over all the locked objects
+ * @exec: drm_exec object
+ * @index: unsigned long index for the iteration
+ * @obj: the current GEM object
+ *
+ * Iterate over all the locked GEM objects inside the drm_exec object.
+ */
+#define drm_exec_for_each_locked_object(exec, index, obj)	\
+	for (index = 0, obj = (exec)->objects[0];		\
+	     index < (exec)->num_objects;			\
+	     ++index, obj = (exec)->objects[index])
+
+/**
+ * drm_exec_while_not_all_locked - loop until all GEM objects are prepared
+ * @exec: drm_exec object
+ *
+ * Core functionality of the drm_exec object. Loops until all GEM objects are
+ * prepared and no more contention exists.
+ *
+ * At the beginning of the loop it is guaranteed that no GEM object is locked.
+ */
+#define drm_exec_while_not_all_locked(exec)	\
+	while (drm_exec_cleanup(exec))
+
+/**
+ * drm_exec_continue_on_contention - continue the loop when we need to cleanup
+ * @exec: drm_exec object
+ *
+ * Control flow helper to continue when a contention was detected and we need to
+ * clean up and re-start the loop to prepare all GEM objects.
+ */
+#define drm_exec_continue_on_contention(exec)		\
+	if (unlikely(drm_exec_is_contended(exec)))	\
+		continue
+
+/**
+ * drm_exec_break_on_contention - break a subordinal loop on contention
+ * @exec: drm_exec object
+ *
+ * Control flow helper to break a subordinal loop when a contention was detected
+ * and we need to clean up and re-start the loop to prepare all GEM objects.
+ */
+#define drm_exec_break_on_contention(exec)		\
+	if (unlikely(drm_exec_is_contended(exec)))	\
+		break
+
+/**
+ * drm_exec_is_contended - check for contention
+ * @exec: drm_exec object
+ *
+ * Returns true if the drm_exec object has run into some contention while
+ * locking a GEM object and needs to clean up.
+ */
+static inline bool drm_exec_is_contended(struct drm_exec *exec)
+{
+	return !!exec->contended;
+}
+
+void drm_exec_init(struct drm_exec *exec, bool interruptible);
+void drm_exec_fini(struct drm_exec *exec);
+bool drm_exec_cleanup(struct drm_exec *exec);
+int drm_exec_prepare_obj(struct drm_exec *exec, struct drm_gem_object *obj,
+			 unsigned int num_fences);
+
+#endif
author maintainer
#27 lore raw

PATCH 26/30 drm/exec: Always compile drm_exec

In reply to Matthew Brost

We want some helpers for DRM exec in gpuva, alway compile this.

---
 drivers/gpu/drm/Makefile | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
drivers/gpu/drm/Makefile +1 -2
index ab728632d8a2..40067970af04 100644
@@ -23,6 +23,7 @@ drm-y := \
 	drm_dumb_buffers.o \
 	drm_edid.o \
 	drm_encoder.o \
+	drm_exec.o \
 	drm_file.o \
 	drm_fourcc.o \
 	drm_framebuffer.o \
@@ -81,8 +82,6 @@ obj-$(CONFIG_DRM_PANEL_ORIENTATION_QUIRKS) += drm_panel_orientation_quirks.o
 # Memory-management helpers
 #
 #
-obj-$(CONFIG_DRM_EXEC) += drm_exec.o
-
 obj-$(CONFIG_DRM_BUDDY) += drm_buddy.o
 
 drm_dma_helper-y := drm_gem_dma_helper.o
Signature · 1 line
2.34.1
author maintainer
#28 lore raw

PATCH 27/30 drm/xe: Use drm_exec for locking rather than TTM exec helpers

In reply to Matthew Brost

drm_exec is intended to replace TTM exec helpers, use drm_exec.

---
 drivers/gpu/drm/drm_gpuva_mgr.c              |  67 ++++-
 drivers/gpu/drm/i915/display/intel_display.c |   6 +-
 drivers/gpu/drm/xe/Kconfig                   |   1 +
 drivers/gpu/drm/xe/xe_bo.c                   |  56 ++--
 drivers/gpu/drm/xe/xe_bo.h                   |   6 +-
 drivers/gpu/drm/xe/xe_bo_evict.c             |  24 +-
 drivers/gpu/drm/xe/xe_bo_types.h             |   1 -
 drivers/gpu/drm/xe/xe_engine.c               |   7 +-
 drivers/gpu/drm/xe/xe_exec.c                 |  37 +--
 drivers/gpu/drm/xe/xe_gt_pagefault.c         |  55 +---
 drivers/gpu/drm/xe/xe_lrc.c                  |   8 +-
 drivers/gpu/drm/xe/xe_migrate.c              |  13 +-
 drivers/gpu/drm/xe/xe_vm.c                   | 283 ++++++++-----------
 drivers/gpu/drm/xe/xe_vm.h                   |  27 +-
 drivers/gpu/drm/xe/xe_vm_madvise.c           |  37 +--
 include/drm/drm_gpuva_mgr.h                  |  16 +-
 16 files changed, 299 insertions(+), 345 deletions(-)
drivers/gpu/drm/drm_gpuva_mgr.c +59 -8
index e8cd6e154336..93c912c34211 100644
@@ -483,6 +483,50 @@ drm_gpuva_manager_destroy(struct drm_gpuva_manager *mgr)
 }
 EXPORT_SYMBOL(drm_gpuva_manager_destroy);
 
+/**
+ * TODO
+ */
+int drm_gpuva_manager_lock(struct drm_gpuva_manager *mgr, struct drm_exec *exec,
+			   struct drm_gem_object *mgr_obj, bool intr,
+			   unsigned int num_fences)
+{
+	struct drm_gpuva *gpuva;
+	int ret;
+
+	drm_exec_init(exec, intr);
+	drm_exec_while_not_all_locked(exec) {
+		ret = drm_exec_prepare_obj(exec, mgr_obj, num_fences);
+		drm_exec_continue_on_contention(exec);
+		if (ret && ret != -EALREADY)
+			goto err_exec;
+
+		drm_gpuva_for_each_extobj(gpuva, mgr) {
+			ret = drm_exec_prepare_obj(exec, gpuva->gem.obj,
+						   num_fences);
+			drm_exec_break_on_contention(exec);
+			if (ret && ret != -EALREADY)
+				goto err_exec;
+		}
+	}
+
+	return 0;
+
+err_exec:
+	drm_exec_fini(exec);
+	return ret;
+}
+EXPORT_SYMBOL(drm_gpuva_manager_lock);
+
+/**
+ * TODO
+ */
+void drm_gpuva_manager_unlock(struct drm_gpuva_manager *mgr,
+			      struct drm_exec *exec)
+{
+	drm_exec_fini(exec);
+}
+EXPORT_SYMBOL(drm_gpuva_manager_unlock);
+
 static inline bool
 drm_gpuva_in_mm_range(struct drm_gpuva_manager *mgr, u64 addr, u64 range)
 {
@@ -888,7 +932,7 @@ drm_gpuva_interval_empty(struct drm_gpuva_manager *mgr, u64 addr, u64 range)
 EXPORT_SYMBOL(drm_gpuva_interval_empty);
 
 /**
- * drm_gpuva_add_fence - add fence to private and all extobj dma-resv
+ * drm_gpuva_manager_add_fence - add fence to private and all extobj dma-resv
  * @mgr: the &drm_gpuva_manager to add a fence to
  * @fence: fence to add
  * @private_usage: private dma-resv usage
@@ -896,17 +940,24 @@ EXPORT_SYMBOL(drm_gpuva_interval_empty);
  *
  * Returns: true if the interval is empty, false otherwise
  */
-void drm_gpuva_add_fence(struct drm_gpuva_manager *mgr, struct dma_fence *fence,
-			 enum dma_resv_usage private_usage,
-			 enum dma_resv_usage extobj_usage)
+void drm_gpuva_manager_add_fence(struct drm_gpuva_manager *mgr,
+				 struct drm_exec *exec,
+				 struct dma_fence *fence,
+				 enum dma_resv_usage private_usage,
+				 enum dma_resv_usage extobj_usage)
 {
-	struct drm_gpuva *gpuva;
+	struct drm_gem_object *obj;
+	unsigned long index;
+
+	dma_resv_assert_held(&mgr->resv);
 
 	dma_resv_add_fence(&mgr->resv, fence, private_usage);
-	drm_gpuva_for_each_extobj(gpuva, mgr)
-		dma_resv_add_fence(gpuva->gem.obj->resv, fence, extobj_usage);
+	drm_exec_for_each_locked_object(exec, index, obj)
+		if (likely(&mgr->resv != obj->resv))
+			dma_resv_add_fence(obj->resv, fence, extobj_usage);
 }
-EXPORT_SYMBOL(drm_gpuva_add_fence);
+EXPORT_SYMBOL(drm_gpuva_manager_add_fence);
+
 
 /**
  * drm_gpuva_map - helper to insert a &drm_gpuva from &drm_gpuva_fn_ops
drivers/gpu/drm/i915/display/intel_display.c +3 -3
index 28a227450329..aab1a3a0f06d 100644
@@ -7340,11 +7340,11 @@ static int i915_gem_object_read_from_page(struct xe_bo *bo,
 	void *virtual;
 	bool is_iomem;
 	int ret;
-	struct ww_acquire_ctx ww;
+	struct drm_exec exec;
 
 	XE_BUG_ON(size != 8);
 
-	ret = xe_bo_lock(bo, &ww, 0, true);
+	ret = xe_bo_lock(bo, &exec, 0, true);
 	if (ret)
 		return ret;
 
@@ -7361,7 +7361,7 @@ static int i915_gem_object_read_from_page(struct xe_bo *bo,
 
 	ttm_bo_kunmap(&map);
 out_unlock:
-	xe_bo_unlock(bo, &ww);
+	xe_bo_unlock(bo, &exec);
 	return ret;
 }
 #endif
drivers/gpu/drm/xe/Kconfig +1 -0
index f6f3b491d162..bbcc9b64b776 100644
@@ -8,6 +8,7 @@ config DRM_XE
 	select SHMEM
 	select TMPFS
 	select DRM_BUDDY
+	select DRM_EXEC
 	select DRM_KMS_HELPER
 	select DRM_PANEL
 	select DRM_SUBALLOC_HELPER
drivers/gpu/drm/xe/xe_bo.c +23 -33

122 lines not on this page · raw message

drivers/gpu/drm/xe/xe_bo.h +3 -3
index 9b401d30a130..5a80ebf72d10 100644
@@ -75,6 +75,7 @@
 
 #define XE_BO_PROPS_INVALID	(-1)
 
+struct drm_exec;
 struct sg_table;
 
 struct xe_bo *xe_bo_alloc(void);
@@ -142,10 +143,9 @@ static inline void xe_bo_assert_held(struct xe_bo *bo)
 		dma_resv_assert_held((bo)->ttm.base.resv);
 }
 
-int xe_bo_lock(struct xe_bo *bo, struct ww_acquire_ctx *ww,
+int xe_bo_lock(struct xe_bo *bo, struct drm_exec *exec,
 	       int num_resv, bool intr);
-
-void xe_bo_unlock(struct xe_bo *bo, struct ww_acquire_ctx *ww);
+void xe_bo_unlock(struct xe_bo *bo, struct drm_exec *exec);
 
 static inline void xe_bo_unlock_vm_held(struct xe_bo *bo)
 {
drivers/gpu/drm/xe/xe_bo_evict.c +13 -11
index 6642c5f52009..46d9d9eb110c 100644
@@ -3,6 +3,8 @@
  * Copyright © 2022 Intel Corporation
  */
 
+#include <drm/drm_exec.h>
+
 #include "xe_bo_evict.h"
 
 #include "xe_bo.h"
@@ -27,7 +29,7 @@
 int xe_bo_evict_all(struct xe_device *xe)
 {
 	struct ttm_device *bdev = &xe->ttm;
-	struct ww_acquire_ctx ww;
+	struct drm_exec exec;
 	struct xe_bo *bo;
 	struct xe_gt *gt;
 	struct list_head still_in_list;
@@ -62,9 +64,9 @@ int xe_bo_evict_all(struct xe_device *xe)
 		list_move_tail(&bo->pinned_link, &still_in_list);
 		spin_unlock(&xe->pinned.lock);
 
-		xe_bo_lock(bo, &ww, 0, false);
+		xe_bo_lock(bo, &exec, 0, false);
 		ret = xe_bo_evict_pinned(bo);
-		xe_bo_unlock(bo, &ww);
+		xe_bo_unlock(bo, &exec);
 		xe_bo_put(bo);
 		if (ret) {
 			spin_lock(&xe->pinned.lock);
@@ -96,9 +98,9 @@ int xe_bo_evict_all(struct xe_device *xe)
 		list_move_tail(&bo->pinned_link, &xe->pinned.evicted);
 		spin_unlock(&xe->pinned.lock);
 
-		xe_bo_lock(bo, &ww, 0, false);
+		xe_bo_lock(bo, &exec, 0, false);
 		ret = xe_bo_evict_pinned(bo);
-		xe_bo_unlock(bo, &ww);
+		xe_bo_unlock(bo, &exec);
 		xe_bo_put(bo);
 		if (ret)
 			return ret;
@@ -123,7 +125,7 @@ int xe_bo_evict_all(struct xe_device *xe)
  */
 int xe_bo_restore_kernel(struct xe_device *xe)
 {
-	struct ww_acquire_ctx ww;
+	struct drm_exec exec;
 	struct xe_bo *bo;
 	int ret;
 
@@ -140,9 +142,9 @@ int xe_bo_restore_kernel(struct xe_device *xe)
 		list_move_tail(&bo->pinned_link, &xe->pinned.kernel_bo_present);
 		spin_unlock(&xe->pinned.lock);
 
-		xe_bo_lock(bo, &ww, 0, false);
+		xe_bo_lock(bo, &exec, 0, false);
 		ret = xe_bo_restore_pinned(bo);
-		xe_bo_unlock(bo, &ww);
+		xe_bo_unlock(bo, &exec);
 		if (ret) {
 			xe_bo_put(bo);
 			return ret;
@@ -182,7 +184,7 @@ int xe_bo_restore_kernel(struct xe_device *xe)
  */
 int xe_bo_restore_user(struct xe_device *xe)
 {
-	struct ww_acquire_ctx ww;
+	struct drm_exec exec;
 	struct xe_bo *bo;
 	struct xe_gt *gt;
 	struct list_head still_in_list;
@@ -204,9 +206,9 @@ int xe_bo_restore_user(struct xe_device *xe)
 		xe_bo_get(bo);
 		spin_unlock(&xe->pinned.lock);
 
-		xe_bo_lock(bo, &ww, 0, false);
+		xe_bo_lock(bo, &exec, 0, false);
 		ret = xe_bo_restore_pinned(bo);
-		xe_bo_unlock(bo, &ww);
+		xe_bo_unlock(bo, &exec);
 		xe_bo_put(bo);
 		if (ret) {
 			spin_lock(&xe->pinned.lock);
drivers/gpu/drm/xe/xe_bo_types.h +0 -1
index 06de3330211d..2ba34a8c9b66 100644
@@ -11,7 +11,6 @@
 #include <drm/drm_mm.h>
 #include <drm/ttm/ttm_bo.h>
 #include <drm/ttm/ttm_device.h>
-#include <drm/ttm/ttm_execbuf_util.h>
 #include <drm/ttm/ttm_placement.h>
 
 struct xe_device;
drivers/gpu/drm/xe/xe_engine.c +4 -3
index 91600b1e8249..8b425b777259 100644
@@ -8,6 +8,7 @@
 #include <linux/nospec.h>
 
 #include <drm/drm_device.h>
+#include <drm/drm_exec.h>
 #include <drm/drm_file.h>
 #include <drm/xe_drm.h>
 
@@ -89,18 +90,18 @@ struct xe_engine *xe_engine_create(struct xe_device *xe, struct xe_vm *vm,
 				   u32 logical_mask, u16 width,
 				   struct xe_hw_engine *hwe, u32 flags)
 {
-	struct ww_acquire_ctx ww;
+	struct drm_exec exec;
 	struct xe_engine *e;
 	int err;
 
 	if (vm) {
-		err = xe_vm_lock(vm, &ww, 0, true);
+		err = xe_vm_lock(vm, &exec, 0, true);
 		if (err)
 			return ERR_PTR(err);
 	}
 	e = __xe_engine_create(xe, vm, logical_mask, width, hwe, flags);
 	if (vm)
-		xe_vm_unlock(vm, &ww);
+		xe_vm_unlock(vm, &exec);
 
 	return e;
 }
drivers/gpu/drm/xe/xe_exec.c +13 -24
index 2ae02f1500d5..9f7f1088c403 100644
@@ -6,6 +6,7 @@
 #include "xe_exec.h"
 
 #include <drm/drm_device.h>
+#include <drm/drm_exec.h>
 #include <drm/drm_file.h>
 #include <drm/xe_drm.h>
 
@@ -92,21 +93,16 @@
  *	Unlock all
  */
 
-static int xe_exec_begin(struct xe_engine *e, struct ww_acquire_ctx *ww,
-			 struct ttm_validate_buffer tv_onstack[],
-			 struct ttm_validate_buffer **tv,
-			 struct list_head *objs)
+static int xe_exec_begin(struct xe_engine *e, struct drm_exec *exec)
 {
 	struct xe_vm *vm = e->vm;
 	struct xe_vma *vma;
-	LIST_HEAD(dups);
 	int err;
 
-	*tv = NULL;
 	if (xe_vm_no_dma_fences(e->vm))
 		return 0;
 
-	err = xe_vm_lock_dma_resv(vm, ww, tv_onstack, tv, objs, true, 1);
+	err = xe_vm_lock_dma_resv(vm, exec, true, 1);
 	if (err)
 		return err;
 
@@ -123,8 +119,7 @@ static int xe_exec_begin(struct xe_engine *e, struct ww_acquire_ctx *ww,
 
 		err = xe_bo_validate(xe_vma_bo(vma), vm, false);
 		if (err) {
-			xe_vm_unlock_dma_resv(vm, tv_onstack, *tv, ww, objs);
-			*tv = NULL;
+			xe_vm_unlock_dma_resv(vm, exec);
 			return err;
 		}
 	}
@@ -132,14 +127,10 @@ static int xe_exec_begin(struct xe_engine *e, struct ww_acquire_ctx *ww,
 	return 0;
 }
 
-static void xe_exec_end(struct xe_engine *e,
-			struct ttm_validate_buffer *tv_onstack,
-			struct ttm_validate_buffer *tv,
-			struct ww_acquire_ctx *ww,
-			struct list_head *objs)
+static void xe_exec_end(struct xe_engine *e, struct drm_exec *exec)
 {
 	if (!xe_vm_no_dma_fences(e->vm))
-		xe_vm_unlock_dma_resv(e->vm, tv_onstack, tv, ww, objs);
+		xe_vm_unlock_dma_resv(e->vm, exec);
 }
 
 int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
@@ -149,17 +140,14 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
 	struct drm_xe_exec *args = data;
 	struct drm_xe_sync __user *syncs_user = u64_to_user_ptr(args->syncs);
 	u64 __user *addresses_user = u64_to_user_ptr(args->address);
+	struct drm_exec exec;
 	struct xe_engine *engine;
 	struct xe_sync_entry *syncs = NULL;
 	u64 addresses[XE_HW_ENGINE_MAX_INSTANCE];
-	struct ttm_validate_buffer tv_onstack[XE_ONSTACK_TV];
-	struct ttm_validate_buffer *tv = NULL;
 	u32 i, num_syncs = 0;
 	struct xe_sched_job *job;
 	struct dma_fence *rebind_fence;
 	struct xe_vm *vm;
-	struct ww_acquire_ctx ww;
-	struct list_head objs;
 	bool write_locked;
 	int err = 0;
 
@@ -270,7 +258,7 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
 			goto err_unlock_list;
 	}
 
-	err = xe_exec_begin(engine, &ww, tv_onstack, &tv, &objs);
+	err = xe_exec_begin(engine, &exec);
 	if (err)
 		goto err_unlock_list;
 
@@ -361,9 +349,10 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
 	 * are written as we don't pass in a read / write list.
 	 */
 	if (!xe_vm_no_dma_fences(vm))
-		drm_gpuva_add_fence(&vm->mgr, &job->drm.s_fence->finished,
-				    DMA_RESV_USAGE_BOOKKEEP,
-				    DMA_RESV_USAGE_WRITE);
+		drm_gpuva_manager_add_fence(&vm->mgr, &exec,
+					    &job->drm.s_fence->finished,
+					    DMA_RESV_USAGE_BOOKKEEP,
+					    DMA_RESV_USAGE_WRITE);
 
 	for (i = 0; i < num_syncs; i++)
 		xe_sync_entry_signal(&syncs[i], job,
@@ -387,7 +376,7 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
 	if (err)
 		xe_sched_job_put(job);
 err_engine_end:
-	xe_exec_end(engine, tv_onstack, tv, &ww, &objs);
+	xe_exec_end(engine, &exec);
 err_unlock_list:
 	if (write_locked)
 		up_write(&vm->lock);
drivers/gpu/drm/xe/xe_gt_pagefault.c +7 -48

103 lines not on this page · raw message

drivers/gpu/drm/xe/xe_lrc.c +5 -3
index ae605e7805de..3cc34efe8dd8 100644
@@ -3,6 +3,8 @@
  * Copyright © 2021 Intel Corporation
  */
 
+#include <drm/drm_exec.h>
+
 #include "xe_lrc.h"
 
 #include "regs/xe_engine_regs.h"
@@ -712,16 +714,16 @@ int xe_lrc_init(struct xe_lrc *lrc, struct xe_hw_engine *hwe,
 
 void xe_lrc_finish(struct xe_lrc *lrc)
 {
-	struct ww_acquire_ctx ww;
+	struct drm_exec exec;
 
 	xe_hw_fence_ctx_finish(&lrc->fence_ctx);
 	if (lrc->bo->vm)
-		xe_vm_lock(lrc->bo->vm, &ww, 0, false);
+		xe_vm_lock(lrc->bo->vm, &exec, 0, false);
 	else
 		xe_bo_lock_no_vm(lrc->bo, NULL);
 	xe_bo_unpin(lrc->bo);
 	if (lrc->bo->vm)
-		xe_vm_unlock(lrc->bo->vm, &ww);
+		xe_vm_unlock(lrc->bo->vm, &exec);
 	else
 		xe_bo_unlock_no_vm(lrc->bo);
 	xe_bo_put(lrc->bo);
drivers/gpu/drm/xe/xe_migrate.c +7 -6
index 91a06c925a1e..1dd497252640 100644
@@ -9,6 +9,7 @@
 #include <linux/sizes.h>
 
 #include <drm/drm_managed.h>
+#include <drm/drm_exec.h>
 #include <drm/ttm/ttm_tt.h>
 #include <drm/xe_drm.h>
 
@@ -86,13 +87,13 @@ struct xe_engine *xe_gt_migrate_engine(struct xe_gt *gt)
 static void xe_migrate_fini(struct drm_device *dev, void *arg)
 {
 	struct xe_migrate *m = arg;
-	struct ww_acquire_ctx ww;
+	struct drm_exec exec;
 
-	xe_vm_lock(m->eng->vm, &ww, 0, false);
+	xe_vm_lock(m->eng->vm, &exec, 0, false);
 	xe_bo_unpin(m->pt_bo);
 	if (m->cleared_bo)
 		xe_bo_unpin(m->cleared_bo);
-	xe_vm_unlock(m->eng->vm, &ww);
+	xe_vm_unlock(m->eng->vm, &exec);
 
 	dma_fence_put(m->fence);
 	if (m->cleared_bo)
@@ -315,7 +316,7 @@ struct xe_migrate *xe_migrate_init(struct xe_gt *gt)
 	struct xe_device *xe = gt_to_xe(gt);
 	struct xe_migrate *m;
 	struct xe_vm *vm;
-	struct ww_acquire_ctx ww;
+	struct drm_exec exec;
 	int err;
 
 	XE_BUG_ON(xe_gt_is_media_type(gt));
@@ -332,9 +333,9 @@ struct xe_migrate *xe_migrate_init(struct xe_gt *gt)
 	if (IS_ERR(vm))
 		return ERR_CAST(vm);
 
-	xe_vm_lock(vm, &ww, 0, false);
+	xe_vm_lock(vm, &exec, 0, false);
 	err = xe_migrate_prepare_vm(gt, m, vm);
-	xe_vm_unlock(vm, &ww);
+	xe_vm_unlock(vm, &exec);
 	if (err) {
 		xe_vm_close_and_put(vm);
 		return ERR_PTR(err);
drivers/gpu/drm/xe/xe_vm.c +120 -163

538 lines not on this page · raw message

drivers/gpu/drm/xe/xe_vm.h +9 -18
index f279fa622260..47b981d9fc04 100644
@@ -12,6 +12,7 @@
 #include "xe_vm_types.h"
 
 struct drm_device;
+struct drm_exec;
 struct drm_printer;
 struct drm_file;
 
@@ -38,10 +39,14 @@ static inline void xe_vm_put(struct xe_vm *vm)
 	kref_put(&vm->refcount, xe_vm_free);
 }
 
-int xe_vm_lock(struct xe_vm *vm, struct ww_acquire_ctx *ww,
+int xe_vm_lock(struct xe_vm *vm, struct drm_exec *exec,
 	       int num_resv, bool intr);
+void xe_vm_unlock(struct xe_vm *vm, struct drm_exec *exec);
 
-void xe_vm_unlock(struct xe_vm *vm, struct ww_acquire_ctx *ww);
+int xe_vm_bo_lock(struct xe_vm *vm, struct xe_bo *bo, struct drm_exec *exec,
+		  int num_resv, bool intr);
+void xe_vm_bo_unlock(struct xe_vm *vm, struct xe_bo *bo, struct drm_exec *exec,
+		     bool lru_update);
 
 static inline bool xe_vm_is_closed(struct xe_vm *vm)
 {
@@ -219,23 +224,9 @@ int xe_vma_userptr_pin_pages(struct xe_vma *vma);
 
 int xe_vma_userptr_check_repin(struct xe_vma *vma);
 
-/*
- * XE_ONSTACK_TV is used to size the tv_onstack array that is input
- * to xe_vm_lock_dma_resv() and xe_vm_unlock_dma_resv().
- */
-#define XE_ONSTACK_TV 20
-int xe_vm_lock_dma_resv(struct xe_vm *vm, struct ww_acquire_ctx *ww,
-			struct ttm_validate_buffer *tv_onstack,
-			struct ttm_validate_buffer **tv,
-			struct list_head *objs,
-			bool intr,
+int xe_vm_lock_dma_resv(struct xe_vm *vm, struct drm_exec *exec, bool intr,
 			unsigned int num_shared);
-
-void xe_vm_unlock_dma_resv(struct xe_vm *vm,
-			   struct ttm_validate_buffer *tv_onstack,
-			   struct ttm_validate_buffer *tv,
-			   struct ww_acquire_ctx *ww,
-			   struct list_head *objs);
+void xe_vm_unlock_dma_resv(struct xe_vm *vm, struct drm_exec *exec);
 
 int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id);
 
drivers/gpu/drm/xe/xe_vm_madvise.c +19 -18
index 03508645fa08..a68bc6fec1de 100644
@@ -7,6 +7,7 @@
 
 #include <linux/nospec.h>
 
+#include <drm/drm_exec.h>
 #include <drm/ttm/ttm_tt.h>
 #include <drm/xe_drm.h>
 
@@ -28,16 +29,16 @@ static int madvise_preferred_mem_class(struct xe_device *xe, struct xe_vm *vm,
 
 	for (i = 0; i < num_vmas; ++i) {
 		struct xe_bo *bo;
-		struct ww_acquire_ctx ww;
+		struct drm_exec exec;
 
 		bo = xe_vma_bo(vmas[i]);
 
-		err = xe_bo_lock(bo, &ww, 0, true);
+		err = xe_bo_lock(bo, &exec, 0, true);
 		if (err)
 			return err;
 		bo->props.preferred_mem_class = value;
 		xe_bo_placement_for_flags(xe, bo, bo->flags);
-		xe_bo_unlock(bo, &ww);
+		xe_bo_unlock(bo, &exec);
 	}
 
 	return 0;
@@ -53,16 +54,16 @@ static int madvise_preferred_gt(struct xe_device *xe, struct xe_vm *vm,
 
 	for (i = 0; i < num_vmas; ++i) {
 		struct xe_bo *bo;
-		struct ww_acquire_ctx ww;
+		struct drm_exec exec;
 
 		bo = xe_vma_bo(vmas[i]);
 
-		err = xe_bo_lock(bo, &ww, 0, true);
+		err = xe_bo_lock(bo, &exec, 0, true);
 		if (err)
 			return err;
 		bo->props.preferred_gt = value;
 		xe_bo_placement_for_flags(xe, bo, bo->flags);
-		xe_bo_unlock(bo, &ww);
+		xe_bo_unlock(bo, &exec);
 	}
 
 	return 0;
@@ -89,17 +90,17 @@ static int madvise_preferred_mem_class_gt(struct xe_device *xe,
 
 	for (i = 0; i < num_vmas; ++i) {
 		struct xe_bo *bo;
-		struct ww_acquire_ctx ww;
+		struct drm_exec exec;
 
 		bo = xe_vma_bo(vmas[i]);
 
-		err = xe_bo_lock(bo, &ww, 0, true);
+		err = xe_bo_lock(bo, &exec, 0, true);
 		if (err)
 			return err;
 		bo->props.preferred_mem_class = mem_class;
 		bo->props.preferred_gt = gt_id;
 		xe_bo_placement_for_flags(xe, bo, bo->flags);
-		xe_bo_unlock(bo, &ww);
+		xe_bo_unlock(bo, &exec);
 	}
 
 	return 0;
@@ -112,13 +113,13 @@ static int madvise_cpu_atomic(struct xe_device *xe, struct xe_vm *vm,
 
 	for (i = 0; i < num_vmas; ++i) {
 		struct xe_bo *bo;
-		struct ww_acquire_ctx ww;
+		struct drm_exec exec;
 
 		bo = xe_vma_bo(vmas[i]);
 		if (XE_IOCTL_ERR(xe, !(bo->flags & XE_BO_CREATE_SYSTEM_BIT)))
 			return -EINVAL;
 
-		err = xe_bo_lock(bo, &ww, 0, true);
+		err = xe_bo_lock(bo, &exec, 0, true);
 		if (err)
 			return err;
 		bo->props.cpu_atomic = !!value;
@@ -130,7 +131,7 @@ static int madvise_cpu_atomic(struct xe_device *xe, struct xe_vm *vm,
 		 */
 		if (bo->props.cpu_atomic)
 			ttm_bo_unmap_virtual(&bo->ttm);
-		xe_bo_unlock(bo, &ww);
+		xe_bo_unlock(bo, &exec);
 	}
 
 	return 0;
@@ -143,18 +144,18 @@ static int madvise_device_atomic(struct xe_device *xe, struct xe_vm *vm,
 
 	for (i = 0; i < num_vmas; ++i) {
 		struct xe_bo *bo;
-		struct ww_acquire_ctx ww;
+		struct drm_exec exec;
 
 		bo = xe_vma_bo(vmas[i]);
 		if (XE_IOCTL_ERR(xe, !(bo->flags & XE_BO_CREATE_VRAM0_BIT) &&
 				 !(bo->flags & XE_BO_CREATE_VRAM1_BIT)))
 			return -EINVAL;
 
-		err = xe_bo_lock(bo, &ww, 0, true);
+		err = xe_bo_lock(bo, &exec, 0, true);
 		if (err)
 			return err;
 		bo->props.device_atomic = !!value;
-		xe_bo_unlock(bo, &ww);
+		xe_bo_unlock(bo, &exec);
 	}
 
 	return 0;
@@ -174,16 +175,16 @@ static int madvise_priority(struct xe_device *xe, struct xe_vm *vm,
 
 	for (i = 0; i < num_vmas; ++i) {
 		struct xe_bo *bo;
-		struct ww_acquire_ctx ww;
+		struct drm_exec exec;
 
 		bo = xe_vma_bo(vmas[i]);
 
-		err = xe_bo_lock(bo, &ww, 0, true);
+		err = xe_bo_lock(bo, &exec, 0, true);
 		if (err)
 			return err;
 		bo->ttm.priority = value;
 		ttm_bo_move_to_lru_tail(&bo->ttm);
-		xe_bo_unlock(bo, &ww);
+		xe_bo_unlock(bo, &exec);
 	}
 
 	return 0;
include/drm/drm_gpuva_mgr.h +13 -3
index 943c8fcda533..a2f6d90ac899 100644
@@ -32,6 +32,8 @@
 #include <linux/spinlock.h>
 #include <linux/types.h>
 
+#include <drm/drm_exec.h>
+
 struct drm_gpuva_manager;
 struct drm_gpuva_fn_ops;
 struct drm_gpuva_prealloc;
@@ -169,9 +171,17 @@ struct drm_gpuva *drm_gpuva_find_next(struct drm_gpuva_manager *mgr, u64 end);
 
 bool drm_gpuva_interval_empty(struct drm_gpuva_manager *mgr, u64 addr, u64 range);
 
-void drm_gpuva_add_fence(struct drm_gpuva_manager *mgr, struct dma_fence *fence,
-			 enum dma_resv_usage private_usage,
-			 enum dma_resv_usage extobj_usage);
+int drm_gpuva_manager_lock(struct drm_gpuva_manager *mgr, struct drm_exec *exec,
+			   struct drm_gem_object *mgr_obj, bool intr,
+			   unsigned int num_fences);
+void drm_gpuva_manager_unlock(struct drm_gpuva_manager *mgr,
+			      struct drm_exec *exec);
+
+void drm_gpuva_manager_add_fence(struct drm_gpuva_manager *mgr,
+				 struct drm_exec *exec,
+				 struct dma_fence *fence,
+				 enum dma_resv_usage private_usage,
+				 enum dma_resv_usage extobj_usage);
 
 /**
  * drm_gpuva_evict - sets whether the backing GEM of this &drm_gpuva is evicted
Signature · 1 line
2.34.1
author maintainer
#29 lore raw

PATCH 28/30 drm/xe: Allow dma-fences as in-syncs for compute / faulting VM

In reply to Matthew Brost

This is allowed per the dma-fencing rules.

---
 drivers/gpu/drm/xe/xe_sync.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)
drivers/gpu/drm/xe/xe_sync.c +7 -5
index 99f1ed87196d..1e4e4acb2c4a 100644
@@ -105,6 +105,7 @@ int xe_sync_entry_parse(struct xe_device *xe, struct xe_file *xef,
 {
 	struct drm_xe_sync sync_in;
 	int err;
+	bool signal;
 
 	if (copy_from_user(&sync_in, sync_user, sizeof(*sync_user)))
 		return -EFAULT;
@@ -113,9 +114,10 @@ int xe_sync_entry_parse(struct xe_device *xe, struct xe_file *xef,
 			 ~(SYNC_FLAGS_TYPE_MASK | DRM_XE_SYNC_SIGNAL)))
 		return -EINVAL;
 
+	signal = sync_in.flags & DRM_XE_SYNC_SIGNAL;
 	switch (sync_in.flags & SYNC_FLAGS_TYPE_MASK) {
 	case DRM_XE_SYNC_SYNCOBJ:
-		if (XE_IOCTL_ERR(xe, no_dma_fences))
+		if (XE_IOCTL_ERR(xe, no_dma_fences && signal))
 			return -ENOTSUPP;
 
 		if (XE_IOCTL_ERR(xe, upper_32_bits(sync_in.addr)))
@@ -125,7 +127,7 @@ int xe_sync_entry_parse(struct xe_device *xe, struct xe_file *xef,
 		if (XE_IOCTL_ERR(xe, !sync->syncobj))
 			return -ENOENT;
 
-		if (!(sync_in.flags & DRM_XE_SYNC_SIGNAL)) {
+		if (!signal) {
 			sync->fence = drm_syncobj_fence_get(sync->syncobj);
 			if (XE_IOCTL_ERR(xe, !sync->fence))
 				return -EINVAL;
@@ -133,7 +135,7 @@ int xe_sync_entry_parse(struct xe_device *xe, struct xe_file *xef,
 		break;
 
 	case DRM_XE_SYNC_TIMELINE_SYNCOBJ:
-		if (XE_IOCTL_ERR(xe, no_dma_fences))
+		if (XE_IOCTL_ERR(xe, no_dma_fences && signal))
 			return -ENOTSUPP;
 
 		if (XE_IOCTL_ERR(xe, upper_32_bits(sync_in.addr)))
@@ -146,7 +148,7 @@ int xe_sync_entry_parse(struct xe_device *xe, struct xe_file *xef,
 		if (XE_IOCTL_ERR(xe, !sync->syncobj))
 			return -ENOENT;
 
-		if (sync_in.flags & DRM_XE_SYNC_SIGNAL) {
+		if (signal) {
 			sync->chain_fence = dma_fence_chain_alloc();
 			if (!sync->chain_fence)
 				return -ENOMEM;
@@ -168,7 +170,7 @@ int xe_sync_entry_parse(struct xe_device *xe, struct xe_file *xef,
 		break;
 
 	case DRM_XE_SYNC_USER_FENCE:
-		if (XE_IOCTL_ERR(xe, !(sync_in.flags & DRM_XE_SYNC_SIGNAL)))
+		if (XE_IOCTL_ERR(xe, !signal))
 			return -ENOTSUPP;
 
 		if (XE_IOCTL_ERR(xe, sync_in.addr & 0x7))
Signature · 1 line
2.34.1
author maintainer
#30 lore raw

PATCH 29/30 drm/xe: Allow compute VMs to output dma-fences on binds

In reply to Matthew Brost

Binds are not long running jobs thus we can export dma-fences even if a VM is in compute mode.

---
 drivers/gpu/drm/xe/xe_vm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_vm.c +1 -1
index 52726541692a..2bf773e1a49e 100644
@@ -3043,7 +3043,7 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
 	for (num_syncs = 0; num_syncs < args->num_syncs; num_syncs++) {
 		err = xe_sync_entry_parse(xe, xef, &syncs[num_syncs],
 					  &syncs_user[num_syncs], false,
-					  xe_vm_no_dma_fences(vm));
+					  xe_vm_in_fault_mode(vm));
 		if (err)
 			goto free_syncs;
 	}
Signature · 1 line
2.34.1
author maintainer
#31 lore raw

PATCH 30/30 drm/xe: remove async worker, sync binds, new error handling

In reply to Matthew Brost

Async worker is gone, all jobs and memory allocations done in IOCTL.

Async vs. sync now means when do bind operations complete relative to the IOCTL. Async completes when out-syncs signal while sync completes when the IOCTL returns. In-syncs and out-syncs are only allowed in async mode.

The error handling is similar to before, on memory allocation errors binds are pause, VM is put in an error state, and the bind IOCTL returns -ENOSPC. The user is allowed to issue sync unbinds, with the reclaim bit set, while in an error state. Bind operations without the reclaim bit set are rejected with -EALREADY until the exits the error state. To exit the error issue a restart bind operation which will pick up where the original failure left off.

TODO: Update kernel doc

---
 drivers/gpu/drm/xe/xe_engine.c          |   7 +-
 drivers/gpu/drm/xe/xe_engine_types.h    |   1 +
 drivers/gpu/drm/xe/xe_exec.c            |  42 --
 drivers/gpu/drm/xe/xe_sync.c            |  14 +-
 drivers/gpu/drm/xe/xe_sync.h            |   2 +-
 drivers/gpu/drm/xe/xe_vm.c              | 712 ++++++------------------
 drivers/gpu/drm/xe/xe_vm.h              |   2 -
 drivers/gpu/drm/xe/xe_vm_types.h        |  37 +-
 drivers/gpu/drm/xe/xe_wait_user_fence.c |  43 +-
 include/uapi/drm/xe_drm.h               |  79 +--
 10 files changed, 213 insertions(+), 726 deletions(-)
drivers/gpu/drm/xe/xe_engine.c +6 -1
index 8b425b777259..d26aadb5e727 100644
@@ -541,7 +541,10 @@ int xe_engine_create_ioctl(struct drm_device *dev, void *data,
 	if (XE_IOCTL_ERR(xe, eci[0].gt_id >= xe->info.tile_count))
 	       return -EINVAL;
 
-	if (eci[0].engine_class == DRM_XE_ENGINE_CLASS_VM_BIND) {
+	if (eci[0].engine_class >= DRM_XE_ENGINE_CLASS_VM_BIND_ASYNC) {
+		bool sync = eci[0].engine_class ==
+			DRM_XE_ENGINE_CLASS_VM_BIND_SYNC;
+	
 		for_each_gt(gt, xe, id) {
 			struct xe_engine *new;
 
@@ -564,6 +567,8 @@ int xe_engine_create_ioctl(struct drm_device *dev, void *data,
 					       args->width, hwe,
 					       ENGINE_FLAG_PERSISTENT |
 					       ENGINE_FLAG_VM |
+					       (sync ? 0 :
+						ENGINE_FLAG_VM_ASYNC) |
 					       (id ?
 					       ENGINE_FLAG_BIND_ENGINE_CHILD :
 					       0));
drivers/gpu/drm/xe/xe_engine_types.h +1 -0
index 36bfaeec23f4..4949edfa0980 100644
@@ -59,6 +59,7 @@ struct xe_engine {
 #define ENGINE_FLAG_VM			BIT(4)
 #define ENGINE_FLAG_BIND_ENGINE_CHILD	BIT(5)
 #define ENGINE_FLAG_WA			BIT(6)
+#define ENGINE_FLAG_VM_ASYNC		BIT(7)
 
 	/**
 	 * @flags: flags for this engine, should statically setup aside from ban
drivers/gpu/drm/xe/xe_exec.c +0 -42

54 lines not on this page · raw message

drivers/gpu/drm/xe/xe_sync.c +3 -11
index 1e4e4acb2c4a..c05142f9780a 100644
@@ -18,7 +18,6 @@
 #include "xe_sched_job_types.h"
 
 #define SYNC_FLAGS_TYPE_MASK 0x3
-#define SYNC_FLAGS_FENCE_INSTALLED	0x10000
 
 struct user_fence {
 	struct xe_device *xe;
@@ -221,12 +220,11 @@ int xe_sync_entry_add_deps(struct xe_sync_entry *sync, struct xe_sched_job *job)
 	return 0;
 }
 
-bool xe_sync_entry_signal(struct xe_sync_entry *sync, struct xe_sched_job *job,
+void xe_sync_entry_signal(struct xe_sync_entry *sync, struct xe_sched_job *job,
 			  struct dma_fence *fence)
 {
-	if (!(sync->flags & DRM_XE_SYNC_SIGNAL) ||
-	    sync->flags & SYNC_FLAGS_FENCE_INSTALLED)
-		return false;
+	if (!(sync->flags & DRM_XE_SYNC_SIGNAL))
+		return;
 
 	if (sync->chain_fence) {
 		drm_syncobj_add_point(sync->syncobj, sync->chain_fence,
@@ -258,12 +256,6 @@ bool xe_sync_entry_signal(struct xe_sync_entry *sync, struct xe_sched_job *job,
 		job->user_fence.addr = sync->addr;
 		job->user_fence.value = sync->timeline_value;
 	}
-
-	/* TODO: external BO? */
-
-	sync->flags |= SYNC_FLAGS_FENCE_INSTALLED;
-
-	return true;
 }
 
 void xe_sync_entry_cleanup(struct xe_sync_entry *sync)
drivers/gpu/drm/xe/xe_sync.h +1 -1
index 4cbcf7a19911..30958ddc4cdc 100644
@@ -19,7 +19,7 @@ int xe_sync_entry_parse(struct xe_device *xe, struct xe_file *xef,
 int xe_sync_entry_wait(struct xe_sync_entry *sync);
 int xe_sync_entry_add_deps(struct xe_sync_entry *sync,
 			   struct xe_sched_job *job);
-bool xe_sync_entry_signal(struct xe_sync_entry *sync,
+void xe_sync_entry_signal(struct xe_sync_entry *sync,
 			  struct xe_sched_job *job,
 			  struct dma_fence *fence);
 void xe_sync_entry_cleanup(struct xe_sync_entry *sync);
drivers/gpu/drm/xe/xe_vm.c +183 -529

1,084 lines not on this page · raw message

drivers/gpu/drm/xe/xe_vm.h +0 -2
index 47b981d9fc04..908e0c764968 100644
@@ -171,8 +171,6 @@ struct dma_fence *xe_vm_rebind(struct xe_vm *vm, bool rebind_worker);
 
 int xe_vm_invalidate_vma(struct xe_vma *vma);
 
-int xe_vm_async_fence_wait_start(struct dma_fence *fence);
-
 extern struct ttm_device_funcs xe_ttm_funcs;
 
 struct ttm_buffer_object *xe_vm_ttm_bo(struct xe_vm *vm);
drivers/gpu/drm/xe/xe_vm_types.h +5 -32
index ce1260b8d3ef..b48d0eaa8939 100644
@@ -15,7 +15,6 @@
 #include "xe_device_types.h"
 #include "xe_pt_types.h"
 
-struct async_op_fence;
 struct xe_bo;
 struct xe_sync_entry;
 struct xe_vm;
@@ -143,7 +142,7 @@ struct xe_vm {
 	/** @flags: flags for this VM, statically setup a creation time */
 #define XE_VM_FLAGS_64K			BIT(0)
 #define XE_VM_FLAG_COMPUTE_MODE		BIT(1)
-#define XE_VM_FLAG_ASYNC_BIND_OPS	BIT(2)
+#define XE_VM_FLAG_ASYNC_DEFAULT	BIT(2)
 #define XE_VM_FLAG_MIGRATION		BIT(3)
 #define XE_VM_FLAG_SCRATCH_PAGE		BIT(4)
 #define XE_VM_FLAG_FAULT_MODE		BIT(5)
@@ -179,40 +178,18 @@ struct xe_vm {
 	 */
 	struct work_struct destroy_work;
 
-	/** @async_ops: async VM operations (bind / unbinds) */
+	/** @ops_state: VM operations (bind / unbinds) state*/
 	struct {
-		/** @list: list of pending async VM ops */
+		/** @list: list of pending VM ops */
 		struct list_head pending;
-		/** @work: worker to execute async VM ops */
-		struct work_struct work;
-		/** @lock: protects list of pending async VM ops and fences */
-		spinlock_t lock;
-		/** @error_capture: error capture state */
-		struct {
-			/** @mm: user MM */
-			struct mm_struct *mm;
-			/**
-			 * @addr: user pointer to copy error capture state too
-			 */
-			u64 addr;
-			/** @wq: user fence wait queue for VM errors */
-			wait_queue_head_t wq;
-		} error_capture;
-		/** @fence: fence state */
-		struct {
-			/** @context: context of async fence */
-			u64 context;
-			/** @seqno: seqno of async fence */
-			u32 seqno;
-		} fence;
-		/** @error: error state for async VM ops */
+		/** @error: error state for VM ops */
 		int error;
 		/**
 		 * @munmap_rebind_inflight: an munmap style VM bind is in the
 		 * middle of a set of ops which requires a rebind at the end.
 		 */
 		bool munmap_rebind_inflight;
-	} async_ops;
+	} ops_state;
 
 	/** @userptr: user pointer state */
 	struct {
@@ -366,10 +343,6 @@ struct xe_vma_op {
 	u32 num_syncs;
 	/** @link: async operation link */
 	struct list_head link;
-	/**
-	 * @fence: async operation fence, signaled on last operation complete
-	 */
-	struct async_op_fence *fence;
 	/** @gt_mask: gt mask for this operation */
 	u8 gt_mask;
 	/** @flags: operation flags */
drivers/gpu/drm/xe/xe_wait_user_fence.c +5 -38

88 lines not on this page · raw message

include/uapi/drm/xe_drm.h +9 -70

125 lines not on this page · raw message

Signature · 1 line
2.34.1
#32 lore raw

[Intel-xe] ✓ CI.Patch_applied: success for Upstreaming prep / all of mbrosts patches

In reply to Matthew Brost
== Series Details ==

== Series Details ==

Series: Upstreaming prep / all of mbrosts patches
URL : https://patchwork.freedesktop.org/series/117156/
State : success

== Summary ==

=== Applying kernel patches on branch 'drm-xe-next' with base: === Base commit: f7b51a4d5 drm/xe: Fix media detection for pre-GMD_ID platforms === git am output follows ===
.git/rebase-apply/patch:26: trailing whitespace.

warning: 1 line adds whitespace errors.
Applying: drm/sched: Add run_wq argument to drm_sched_init
Applying: drm/sched: Move schedule policy to scheduler
Applying: drm/sched: Add DRM_SCHED_POLICY_SINGLE_ENTITY scheduling policy
Applying: drm/xe: Use DRM_SCHED_POLICY_SINGLE_ENTITY mode
Applying: drm/xe: Long running job update
Applying: drm/xe: Ensure LR engines are not persistent
Applying: drm/xe: Only try to lock external BOs in VM bind
Applying: drm/xe: VM LRU bulk move
Applying: drm/xe/guc: Read HXG fields from DW1 of G2H response
Applying: drm/xe/guc: Return the lower part of blocking H2G message
Applying: drm/xe/guc: Use doorbells for submission if possible
Applying: drm/xe/guc: Print doorbell ID in GuC engine debugfs entry
Applying: maple_tree: split up MA_STATE() macro
Applying: maple_tree: Export mas_preallocate
Applying: drm: manager to keep track of GPUs VA mappings
Applying: drm/xe: Port Xe to GPUVA
Applying: drm/xe: NULL binding implementation
Applying: drm/xe: Avoid doing rebinds
Applying: drm/xe: Reduce the number list links in xe_vma
Applying: drm/xe: Optimize size of xe_vma allocation
Applying: drm/gpuva: Add drm device to GPUVA manager
Applying: drm/gpuva: Move dma-resv to GPUVA manager
Applying: drm/gpuva: Add support for extobj
Applying: drm/xe: Userptr refactor
Applying: drm: execution context for GEM buffers v3
Applying: drm/exec: Always compile drm_exec
Applying: drm/xe: Use drm_exec for locking rather than TTM exec helpers
Applying: drm/xe: Allow dma-fences as in-syncs for compute / faulting VM
Applying: drm/xe: Allow compute VMs to output dma-fences on binds
Applying: drm/xe: remove async worker, sync binds, new error handling

#33 lore raw

[Intel-xe] ✗ CI.KUnit: failure for Upstreaming prep / all of mbrosts patches

In reply to Matthew Brost
== Series Details ==

== Series Details ==

Series: Upstreaming prep / all of mbrosts patches
URL : https://patchwork.freedesktop.org/series/117156/
State : failure

== Summary ==

+ trap cleanup EXIT
+ /kernel/tools/testing/kunit/kunit.py run --kunitconfig /kernel/drivers/gpu/drm/xe/.kunitconfig
ERROR:root:In file included from ../drivers/gpu/drm/xe/xe_bo.c:1969:
../drivers/gpu/drm/xe/tests/xe_bo.c: In function ‘evict_test_run_gt’:
../drivers/gpu/drm/xe/tests/xe_bo.c:185:18: error: passing argument 2 of ‘xe_vm_lock’ from incompatible pointer type [-Werror=incompatible-pointer-types]

  185 |   xe_vm_lock(vm, &ww, 0, false);
      |                  ^~~
      |                  |
      |                  struct ww_acquire_ctx *
In file included from ../drivers/gpu/drm/xe/xe_trace.h:21,
                 from ../drivers/gpu/drm/xe/xe_bo.c:26:
../drivers/gpu/drm/xe/xe_vm.h:42:51: note: expected ‘struct drm_exec *’ but argument is of type ‘struct ww_acquire_ctx *’
   42 | int xe_vm_lock(struct xe_vm *vm, struct drm_exec *exec,
      |                                  ~~~~~~~~~~~~~~~~~^~~~
In file included from ../drivers/gpu/drm/xe/xe_bo.c:1969:
../drivers/gpu/drm/xe/tests/xe_bo.c:188:20: error: passing argument 2 of ‘xe_vm_unlock’ from incompatible pointer type [-Werror=incompatible-pointer-types]
  188 |   xe_vm_unlock(vm, &ww);
      |                    ^~~
      |                    |
      |                    struct ww_acquire_ctx *
In file included from ../drivers/gpu/drm/xe/xe_trace.h:21,
                 from ../drivers/gpu/drm/xe/xe_bo.c:26:
../drivers/gpu/drm/xe/xe_vm.h:44:54: note: expected ‘struct drm_exec *’ but argument is of type ‘struct ww_acquire_ctx *’
   44 | void xe_vm_unlock(struct xe_vm *vm, struct drm_exec *exec);
      |                                     ~~~~~~~~~~~~~~~~~^~~~
In file included from ../drivers/gpu/drm/xe/xe_bo.c:1969:
../drivers/gpu/drm/xe/tests/xe_bo.c:201:24: error: passing argument 2 of ‘xe_bo_lock’ from incompatible pointer type [-Werror=incompatible-pointer-types]
  201 |   xe_bo_lock(external, &ww, 0, false);
      |                        ^~~
      |                        |
      |                        struct ww_acquire_ctx *
../drivers/gpu/drm/xe/xe_bo.c:1733:51: note: expected ‘struct drm_exec *’ but argument is of type ‘struct ww_acquire_ctx *’
 1733 | int xe_bo_lock(struct xe_bo *bo, struct drm_exec *exec,
      |                                  ~~~~~~~~~~~~~~~~~^~~~
In file included from ../drivers/gpu/drm/xe/xe_bo.c:1969:
../drivers/gpu/drm/xe/tests/xe_bo.c:203:26: error: passing argument 2 of ‘xe_bo_unlock’ from incompatible pointer type [-Werror=incompatible-pointer-types]
  203 |   xe_bo_unlock(external, &ww);
      |                          ^~~
      |                          |
      |                          struct ww_acquire_ctx *
../drivers/gpu/drm/xe/xe_bo.c:1754:54: note: expected ‘struct drm_exec *’ but argument is of type ‘struct ww_acquire_ctx *’
 1754 | void xe_bo_unlock(struct xe_bo *bo, struct drm_exec *exec)
      |                                     ~~~~~~~~~~~~~~~~~^~~~
In file included from ../drivers/gpu/drm/xe/xe_bo.c:1969:
../drivers/gpu/drm/xe/tests/xe_bo.c:243:19: error: passing argument 2 of ‘xe_vm_lock’ from incompatible pointer type [-Werror=incompatible-pointer-types]
  243 |    xe_vm_lock(vm, &ww, 0, false);
      |                   ^~~
      |                   |
      |                   struct ww_acquire_ctx *
In file included from ../drivers/gpu/drm/xe/xe_trace.h:21,
                 from ../drivers/gpu/drm/xe/xe_bo.c:26:
../drivers/gpu/drm/xe/xe_vm.h:42:51: note: expected ‘struct drm_exec *’ but argument is of type ‘struct ww_acquire_ctx *’
   42 | int xe_vm_lock(struct xe_vm *vm, struct drm_exec *exec,
      |                                  ~~~~~~~~~~~~~~~~~^~~~
In file included from ../drivers/gpu/drm/xe/xe_bo.c:1969:
../drivers/gpu/drm/xe/tests/xe_bo.c:245:21: error: passing argument 2 of ‘xe_vm_unlock’ from incompatible pointer type [-Werror=incompatible-pointer-types]
  245 |    xe_vm_unlock(vm, &ww);
      |                     ^~~
      |                     |
      |                     struct ww_acquire_ctx *
In file included from ../drivers/gpu/drm/xe/xe_trace.h:21,
                 from ../drivers/gpu/drm/xe/xe_bo.c:26:
../drivers/gpu/drm/xe/xe_vm.h:44:54: note: expected ‘struct drm_exec *’ but argument is of type ‘struct ww_acquire_ctx *’
   44 | void xe_vm_unlock(struct xe_vm *vm, struct drm_exec *exec);
      |                                     ~~~~~~~~~~~~~~~~~^~~~
In file included from ../drivers/gpu/drm/xe/xe_bo.c:1969:
../drivers/gpu/drm/xe/tests/xe_bo.c:252:25: error: passing argument 2 of ‘xe_bo_lock’ from incompatible pointer type [-Werror=incompatible-pointer-types]
  252 |    xe_bo_lock(external, &ww, 0, false);
      |                         ^~~
      |                         |
      |                         struct ww_acquire_ctx *
../drivers/gpu/drm/xe/xe_bo.c:1733:51: note: expected ‘struct drm_exec *’ but argument is of type ‘struct ww_acquire_ctx *’
 1733 | int xe_bo_lock(struct xe_bo *bo, struct drm_exec *exec,
      |                                  ~~~~~~~~~~~~~~~~~^~~~
In file included from ../drivers/gpu/drm/xe/xe_bo.c:1969:
../drivers/gpu/drm/xe/tests/xe_bo.c:254:27: error: passing argument 2 of ‘xe_bo_unlock’ from incompatible pointer type [-Werror=incompatible-pointer-types]
  254 |    xe_bo_unlock(external, &ww);
      |                           ^~~
      |                           |
      |                           struct ww_acquire_ctx *
../drivers/gpu/drm/xe/xe_bo.c:1754:54: note: expected ‘struct drm_exec *’ but argument is of type ‘struct ww_acquire_ctx *’
 1754 | void xe_bo_unlock(struct xe_bo *bo, struct drm_exec *exec)
      |                                     ~~~~~~~~~~~~~~~~~^~~~
In file included from ../drivers/gpu/drm/xe/xe_bo.c:1969:
../drivers/gpu/drm/xe/tests/xe_bo.c:262:24: error: passing argument 2 of ‘xe_bo_lock’ from incompatible pointer type [-Werror=incompatible-pointer-types]
  262 |   xe_bo_lock(external, &ww, 0, false);
      |                        ^~~
      |                        |
      |                        struct ww_acquire_ctx *
../drivers/gpu/drm/xe/xe_bo.c:1733:51: note: expected ‘struct drm_exec *’ but argument is of type ‘struct ww_acquire_ctx *’
 1733 | int xe_bo_lock(struct xe_bo *bo, struct drm_exec *exec,
      |                                  ~~~~~~~~~~~~~~~~~^~~~
In file included from ../drivers/gpu/drm/xe/xe_bo.c:1969:
../drivers/gpu/drm/xe/tests/xe_bo.c:264:26: error: passing argument 2 of ‘xe_bo_unlock’ from incompatible pointer type [-Werror=incompatible-pointer-types]
  264 |   xe_bo_unlock(external, &ww);
      |                          ^~~
      |                          |
      |                          struct ww_acquire_ctx *
../drivers/gpu/drm/xe/xe_bo.c:1754:54: note: expected ‘struct drm_exec *’ but argument is of type ‘struct ww_acquire_ctx *’
 1754 | void xe_bo_unlock(struct xe_bo *bo, struct drm_exec *exec)
      |                                     ~~~~~~~~~~~~~~~~~^~~~
In file included from ../drivers/gpu/drm/xe/xe_bo.c:1969:
../drivers/gpu/drm/xe/tests/xe_bo.c:271:24: error: passing argument 2 of ‘xe_bo_lock’ from incompatible pointer type [-Werror=incompatible-pointer-types]
  271 |   xe_bo_lock(external, &ww, 0, false);
      |                        ^~~
      |                        |
      |                        struct ww_acquire_ctx *
../drivers/gpu/drm/xe/xe_bo.c:1733:51: note: expected ‘struct drm_exec *’ but argument is of type ‘struct ww_acquire_ctx *’
 1733 | int xe_bo_lock(struct xe_bo *bo, struct drm_exec *exec,
      |                                  ~~~~~~~~~~~~~~~~~^~~~
In file included from ../drivers/gpu/drm/xe/xe_bo.c:1969:
../drivers/gpu/drm/xe/tests/xe_bo.c:273:26: error: passing argument 2 of ‘xe_bo_unlock’ from incompatible pointer type [-Werror=incompatible-pointer-types]
  273 |   xe_bo_unlock(external, &ww);
      |                          ^~~
      |                          |
      |                          struct ww_acquire_ctx *
../drivers/gpu/drm/xe/xe_bo.c:1754:54: note: expected ‘struct drm_exec *’ but argument is of type ‘struct ww_acquire_ctx *’
 1754 | void xe_bo_unlock(struct xe_bo *bo, struct drm_exec *exec)
      |                                     ~~~~~~~~~~~~~~~~~^~~~

cc1: some warnings being treated as errors
make[6]: *** [../scripts/Makefile.build:252: drivers/gpu/drm/xe/xe_bo.o] Error 1 make[6]: *** Waiting for unfinished jobs....
make[5]: *** [../scripts/Makefile.build:494: drivers/gpu/drm/xe] Error 2 make[5]: *** Waiting for unfinished jobs....
make[4]: *** [../scripts/Makefile.build:494: drivers/gpu/drm] Error 2 make[3]: *** [../scripts/Makefile.build:494: drivers/gpu] Error 2 make[3]: *** Waiting for unfinished jobs....
make[2]: *** [../scripts/Makefile.build:494: drivers] Error 2 make[2]: *** Waiting for unfinished jobs....
make[1]: *** [/kernel/Makefile:2025: .] Error 2
make: *** [Makefile:226: __sub-make] Error 2

[07:54:32] Configuring KUnit Kernel ...
Generating .config ...
Populating config with:

$ make ARCH=um O=.kunit olddefconfig
[07:54:36] Building KUnit Kernel ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
Building with:
$ make ARCH=um O=.kunit --jobs=48
+ cleanup
++ stat -c %u:%g /kernel
+ chown -R 1003:1003 /kernel