Skip to contentSkip to conversation

Adreno code for 4.13

PATCH In review Jordan Crouse 6 participants 21 messages May 8, 2017 – May 31, 2017 3 messages not in the archive

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

cover Adreno code for 4.13

Hey Rob, here is a newly refreshed chunk of my active stack starting with some bug fixes and cleanups and culminating in A5XX preemption. Not everything here is 100% production ready - I would definately like a comment on the drawqueue idea (like for example, is drawqueue a stupid name?). There are probably tons of cleanups to be had too so the sooner it gets torn apart, the sooner I can have it fixed and ready for msm-next to be ready for drm-next.

Thanks!
Jordan

Jordan Crouse (13):
drm/msm: Take the mutex before calling msm_gem_new_impl drm/msm: Fix the check for the command size
drm/msm: Remove DRM_MSM_NUM_IOCTLS
drm/msm: Remove idle function hook
drm/msm: Add hint to DRM_IOCTL_MSM_GEM_INFO to return an object IOVA drm/msm: get an iova from the address space instead of an id drm/msm: Add a struct to pass configuration to msm_gpu_init() drm/msm: Remove memptrs->wptr
drm/msm: Add drawqueues
drm/msm: Support multiple ringbuffers
drm/msm: Shadow current pointer in the ring until command is complete drm/msm: Make the value of RB_CNTL (almost) generic drm/msm: Implement preemption for A5XX targets

 drivers/gpu/drm/msm/Makefile              |   1 +
 drivers/gpu/drm/msm/adreno/a3xx_gpu.c     |  13 +-
 drivers/gpu/drm/msm/adreno/a4xx_gpu.c     |  13 +-
 drivers/gpu/drm/msm/adreno/a5xx_gpu.c     | 222 +++++++++++++++++---
 drivers/gpu/drm/msm/adreno/a5xx_gpu.h     | 102 ++++++++-
 drivers/gpu/drm/msm/adreno/a5xx_power.c   |  11 +-
 drivers/gpu/drm/msm/adreno/a5xx_preempt.c | 332 ++++++++++++++++++++++++++++++
 drivers/gpu/drm/msm/adreno/adreno_gpu.c   | 183 ++++++++++------
 drivers/gpu/drm/msm/adreno/adreno_gpu.h   |  44 ++--
 drivers/gpu/drm/msm/dsi/dsi_host.c        |  15 +-
 drivers/gpu/drm/msm/mdp/mdp4/mdp4_crtc.c  |   8 +-
 drivers/gpu/drm/msm/mdp/mdp4/mdp4_kms.c   |  16 +-
 drivers/gpu/drm/msm/mdp/mdp4/mdp4_kms.h   |   4 -
 drivers/gpu/drm/msm/mdp/mdp4/mdp4_plane.c |  13 +-
 drivers/gpu/drm/msm/mdp/mdp5/mdp5_crtc.c  |   5 +-
 drivers/gpu/drm/msm/mdp/mdp5/mdp5_kms.c   |  11 +-
 drivers/gpu/drm/msm/mdp/mdp5/mdp5_kms.h   |   4 -
 drivers/gpu/drm/msm/mdp/mdp5/mdp5_plane.c |  13 +-
 drivers/gpu/drm/msm/msm_drv.c             | 137 ++++++++++--
 drivers/gpu/drm/msm/msm_drv.h             |  35 ++--
 drivers/gpu/drm/msm/msm_fb.c              |  15 +-
 drivers/gpu/drm/msm/msm_fbdev.c           |  10 +-
 drivers/gpu/drm/msm/msm_fence.c           |  92 +++++++--
 drivers/gpu/drm/msm/msm_fence.h           |  15 +-
 drivers/gpu/drm/msm/msm_gem.c             | 138 +++++++++----
 drivers/gpu/drm/msm/msm_gem.h             |   7 +-
 drivers/gpu/drm/msm/msm_gem_submit.c      |  51 ++++-
 drivers/gpu/drm/msm/msm_gpu.c             | 153 +++++++++-----
 drivers/gpu/drm/msm/msm_gpu.h             |  51 ++++-
 drivers/gpu/drm/msm/msm_kms.h             |   3 +
 drivers/gpu/drm/msm/msm_ringbuffer.c      |  21 +-
 drivers/gpu/drm/msm/msm_ringbuffer.h      |  22 +-
 include/uapi/drm/msm_drm.h                |  29 ++-
 33 files changed, 1418 insertions(+), 371 deletions(-)
 create mode 100644 drivers/gpu/drm/msm/adreno/a5xx_preempt.c
Signature · 1 line
1.9.1
#1 lore raw

PATCH 1/13 drm/msm: Take the mutex before calling msm_gem_new_impl

Amongst its other duties, msm_gem_new_impl adds the newly created GEM object to the shared inactive list which may also be actively modifiying the list during submission. All the paths to modify the list are protected by the mutex except for the one through msm_gem_import which can end up causing list corruption.

---
 drivers/gpu/drm/msm/msm_gem.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/gpu/drm/msm/msm_gem.c +4 -0
index 68e509b..7a82eaa 100644
@@ -853,7 +853,11 @@ struct drm_gem_object *msm_gem_import(struct drm_device *dev,
 
 	size = PAGE_ALIGN(dmabuf->size);
 
+	/* Take mutex so we can modify the inactive list in msm_gem_new_impl */
+	mutex_lock(&dev->struct_mutex);
 	ret = msm_gem_new_impl(dev, size, MSM_BO_WC, dmabuf->resv, &obj);
+	mutex_unlock(&dev->struct_mutex);
+
 	if (ret)
 		goto fail;
 
Signature · 1 line
1.9.1
#2 lore raw

Re: [Freedreno] [PATCH 01/13] drm/msm: Take the mutex before calling msm_gem_new_impl

Quoted 5 lines · Jordan Crouse

Amongst its other duties, msm_gem_new_impl adds the newly created GEM object to the shared inactive list which may also be actively modifiying the list during submission. All the paths to modify the list are protected by the mutex except for the one through msm_gem_import which can end up causing list corruption.

hmm, I guess it would be a good idea to also throw one of these into msm_gem_new_impl():

    WARN_ON(!mutex_is_locked(&dev->struct_mutex));

BR,
-R

Quoted 28 lines
---
 drivers/gpu/drm/msm/msm_gem.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/gpu/drm/msm/msm_gem.c +4 -1
index 68e509b..7a82eaa 100644
@@ -853,7 +853,11 @@ struct drm_gem_object *msm_gem_import(struct drm_device *dev,
 
        size = PAGE_ALIGN(dmabuf->size);
 
+       /* Take mutex so we can modify the inactive list in msm_gem_new_impl */
+       mutex_lock(&dev->struct_mutex);
        ret = msm_gem_new_impl(dev, size, MSM_BO_WC, dmabuf->resv, &obj);
+       mutex_unlock(&dev->struct_mutex);
+
        if (ret)
                goto fail;
 
--

1.9.1

_______________________________________________
Freedreno mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/freedreno

#3 lore raw

PATCH 2/13 drm/msm: Fix the check for the command size

In reply to Jordan Crouse

The overrun check for the size of submitted commands is off by one. It should allow the offset plus the size to be equal to the size of the memory object when the command stream is very tightly constructed.

---
 drivers/gpu/drm/msm/msm_gem_submit.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
drivers/gpu/drm/msm/msm_gem_submit.c +3 -2
index 1c545eb..c51cf03 100644
@@ -496,8 +496,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
 			goto out;
 		}
 
-		if ((submit_cmd.size + submit_cmd.submit_offset) >=
-				msm_obj->base.size) {
+		if (!(submit_cmd.size) ||
+			((submit_cmd.size + submit_cmd.submit_offset) >
+				msm_obj->base.size)) {
 			DRM_ERROR("invalid cmdstream size: %u\n", submit_cmd.size);
 			ret = -EINVAL;
 			goto out;
Signature · 1 line
1.9.1
#4 lore raw

PATCH 3/13 drm/msm: Remove DRM_MSM_NUM_IOCTLS

In reply to Jordan Crouse

The ioctl array is sparsely populated but the compiler will make sure that it is sufficiently sized for all the values that we have so we can safely use ARRAY_SIZE() instead of having a constantly changing #define in the uapi header.

---
 drivers/gpu/drm/msm/msm_drv.c | 2 +-
 include/uapi/drm/msm_drm.h    | 1 -
 2 files changed, 1 insertion(+), 2 deletions(-)
drivers/gpu/drm/msm/msm_drv.c +1 -1
index 87b5695..66a3a05 100644
@@ -841,7 +841,7 @@ static int msm_ioctl_gem_madvise(struct drm_device *dev, void *data,
 	.debugfs_init       = msm_debugfs_init,
 #endif
 	.ioctls             = msm_ioctls,
-	.num_ioctls         = DRM_MSM_NUM_IOCTLS,
+	.num_ioctls         = ARRAY_SIZE(msm_ioctls),
 	.fops               = &fops,
 	.name               = "msm",
 	.desc               = "MSM Snapdragon DRM",
include/uapi/drm/msm_drm.h +0 -1
index a4a189a..a9985fe 100644
@@ -261,7 +261,6 @@ struct drm_msm_gem_madvise {
 #define DRM_MSM_GEM_SUBMIT             0x06
 #define DRM_MSM_WAIT_FENCE             0x07
 #define DRM_MSM_GEM_MADVISE            0x08
-#define DRM_MSM_NUM_IOCTLS             0x09
 
 #define DRM_IOCTL_MSM_GET_PARAM        DRM_IOWR(DRM_COMMAND_BASE + DRM_MSM_GET_PARAM, struct drm_msm_param)
 #define DRM_IOCTL_MSM_GEM_NEW          DRM_IOWR(DRM_COMMAND_BASE + DRM_MSM_GEM_NEW, struct drm_msm_gem_new)
Signature · 1 line
1.9.1
#5 lore raw

PATCH 4/13 drm/msm: Remove idle function hook

In reply to Jordan Crouse

There isn't any generic code that uses ->idle so remove it.

---
 drivers/gpu/drm/msm/adreno/a3xx_gpu.c   | 4 ++--
 drivers/gpu/drm/msm/adreno/a4xx_gpu.c   | 4 ++--
 drivers/gpu/drm/msm/adreno/a5xx_gpu.c   | 9 ++++-----
 drivers/gpu/drm/msm/adreno/a5xx_gpu.h   | 1 +
 drivers/gpu/drm/msm/adreno/a5xx_power.c | 2 +-
 drivers/gpu/drm/msm/msm_gpu.h           | 1 -
 6 files changed, 10 insertions(+), 11 deletions(-)
drivers/gpu/drm/msm/adreno/a3xx_gpu.c +2 -2
index 7fd7795..0e3828ed 100644
@@ -40,6 +40,7 @@
 extern bool hang_debug;
 
 static void a3xx_dump(struct msm_gpu *gpu);
+static bool a3xx_idle(struct msm_gpu *gpu);
 
 static bool a3xx_me_init(struct msm_gpu *gpu)
 {
@@ -65,7 +66,7 @@ static bool a3xx_me_init(struct msm_gpu *gpu)
 	OUT_RING(ring, 0x00000000);
 
 	gpu->funcs->flush(gpu);
-	return gpu->funcs->idle(gpu);
+	return a3xx_idle(gpu);
 }
 
 static int a3xx_hw_init(struct msm_gpu *gpu)
@@ -446,7 +447,6 @@ static void a3xx_dump(struct msm_gpu *gpu)
 		.last_fence = adreno_last_fence,
 		.submit = adreno_submit,
 		.flush = adreno_flush,
-		.idle = a3xx_idle,
 		.irq = a3xx_irq,
 		.destroy = a3xx_destroy,
 #ifdef CONFIG_DEBUG_FS
drivers/gpu/drm/msm/adreno/a4xx_gpu.c +2 -2
index dfe0ece..19abf22 100644
@@ -31,6 +31,7 @@
 
 extern bool hang_debug;
 static void a4xx_dump(struct msm_gpu *gpu);
+static bool a4xx_idle(struct msm_gpu *gpu);
 
 /*
  * a4xx_enable_hwcg() - Program the clock control registers
@@ -137,7 +138,7 @@ static bool a4xx_me_init(struct msm_gpu *gpu)
 	OUT_RING(ring, 0x00000000);
 
 	gpu->funcs->flush(gpu);
-	return gpu->funcs->idle(gpu);
+	return a4xx_idle(gpu);
 }
 
 static int a4xx_hw_init(struct msm_gpu *gpu)
@@ -534,7 +535,6 @@ static int a4xx_get_timestamp(struct msm_gpu *gpu, uint64_t *value)
 		.last_fence = adreno_last_fence,
 		.submit = adreno_submit,
 		.flush = adreno_flush,
-		.idle = a4xx_idle,
 		.irq = a4xx_irq,
 		.destroy = a4xx_destroy,
 #ifdef CONFIG_DEBUG_FS
drivers/gpu/drm/msm/adreno/a5xx_gpu.c +4 -5
index 4545064..2d3af90 100644
@@ -282,7 +282,7 @@ static int a5xx_me_init(struct msm_gpu *gpu)
 
 	gpu->funcs->flush(gpu);
 
-	return gpu->funcs->idle(gpu) ? 0 : -EINVAL;
+	return a5xx_idle(gpu) ? 0 : -EINVAL;
 }
 
 static struct drm_gem_object *a5xx_ucode_load_bo(struct msm_gpu *gpu,
@@ -632,7 +632,7 @@ static int a5xx_hw_init(struct msm_gpu *gpu)
 		OUT_RING(gpu->rb, 0x0F);
 
 		gpu->funcs->flush(gpu);
-		if (!gpu->funcs->idle(gpu))
+		if (!a5xx_idle(gpu))
 			return -EINVAL;
 	}
 
@@ -649,7 +649,7 @@ static int a5xx_hw_init(struct msm_gpu *gpu)
 		OUT_RING(gpu->rb, 0x00000000);
 
 		gpu->funcs->flush(gpu);
-		if (!gpu->funcs->idle(gpu))
+		if (!a5xx_idle(gpu))
 			return -EINVAL;
 	} else {
 		/* Print a warning so if we die, we know why */
@@ -726,7 +726,7 @@ static inline bool _a5xx_check_idle(struct msm_gpu *gpu)
 		A5XX_RBBM_INT_0_MASK_MISC_HANG_DETECT);
 }
 
-static bool a5xx_idle(struct msm_gpu *gpu)
+bool a5xx_idle(struct msm_gpu *gpu)
 {
 	/* wait for CP to drain ringbuffer: */
 	if (!adreno_idle(gpu))
@@ -1031,7 +1031,6 @@ static void a5xx_show(struct msm_gpu *gpu, struct seq_file *m)
 		.last_fence = adreno_last_fence,
 		.submit = a5xx_submit,
 		.flush = adreno_flush,
-		.idle = a5xx_idle,
 		.irq = a5xx_irq,
 		.destroy = a5xx_destroy,
 #ifdef CONFIG_DEBUG_FS
drivers/gpu/drm/msm/adreno/a5xx_gpu.h +1 -0
index 78408f5..6638bc8 100644
@@ -58,5 +58,6 @@ static inline int spin_usecs(struct msm_gpu *gpu, uint32_t usecs,
 	return -ETIMEDOUT;
 }
 
+bool a5xx_idle(struct msm_gpu *gpu);
 
 #endif /* __A5XX_GPU_H__ */
drivers/gpu/drm/msm/adreno/a5xx_power.c +1 -1
index 72d52c7..ed0802e 100644
@@ -194,7 +194,7 @@ static int a5xx_gpmu_init(struct msm_gpu *gpu)
 
 	gpu->funcs->flush(gpu);
 
-	if (!gpu->funcs->idle(gpu)) {
+	if (!a5xx_idle(gpu)) {
 		DRM_ERROR("%s: Unable to load GPMU firmware. GPMU will not be active\n",
 			gpu->name);
 		return -EINVAL;
drivers/gpu/drm/msm/msm_gpu.h +0 -1
index aa32410..6364502 100644
@@ -50,7 +50,6 @@ struct msm_gpu_funcs {
 	void (*submit)(struct msm_gpu *gpu, struct msm_gem_submit *submit,
 			struct msm_file_private *ctx);
 	void (*flush)(struct msm_gpu *gpu);
-	bool (*idle)(struct msm_gpu *gpu);
 	irqreturn_t (*irq)(struct msm_gpu *irq);
 	uint32_t (*last_fence)(struct msm_gpu *gpu);
 	void (*recover)(struct msm_gpu *gpu);
Signature · 1 line
1.9.1
#6 lore raw

PATCH 5/13 drm/msm: Add hint to DRM_IOCTL_MSM_GEM_INFO to return an object IOVA

In reply to Jordan Crouse

Modify the 'pad' member of struct drm_msm_gem_info to 'hint'. If the user sets 'hint' to non-zero it means that they want a IOVA for the GEM object instead of a mmap() offset. Return the iova in the 'offset' member.

---
 drivers/gpu/drm/msm/msm_drv.c | 23 +++++++++++++++++++++--
 include/uapi/drm/msm_drm.h    |  8 ++++++--
 2 files changed, 27 insertions(+), 4 deletions(-)
drivers/gpu/drm/msm/msm_drv.c +21 -2
index 66a3a05..257ee81 100644
@@ -699,6 +699,17 @@ static int msm_ioctl_gem_cpu_fini(struct drm_device *dev, void *data,
 	return ret;
 }
 
+static int msm_ioctl_gem_info_iova(struct drm_device *dev,
+		struct drm_gem_object *obj, uint64_t *iova)
+{
+	struct msm_drm_private *priv = dev->dev_private;
+
+	if (!priv->gpu)
+		return -EINVAL;
+
+	return msm_gem_get_iova(obj, priv->gpu->id, iova);
+}
+
 static int msm_ioctl_gem_info(struct drm_device *dev, void *data,
 		struct drm_file *file)
 {
@@ -706,14 +717,22 @@ static int msm_ioctl_gem_info(struct drm_device *dev, void *data,
 	struct drm_gem_object *obj;
 	int ret = 0;
 
-	if (args->pad)
+	if (args->flags & ~MSM_INFO_FLAGS)
 		return -EINVAL;
 
 	obj = drm_gem_object_lookup(file, args->handle);
 	if (!obj)
 		return -ENOENT;
 
-	args->offset = msm_gem_mmap_offset(obj);
+	if (args->flags & MSM_INFO_IOVA) {
+		uint64_t iova;
+
+		ret = msm_ioctl_gem_info_iova(dev, obj, &iova);
+		if (!ret)
+			args->offset = iova;
+	} else {
+		args->offset = msm_gem_mmap_offset(obj);
+	}
 
 	drm_gem_object_unreference_unlocked(obj);
 
include/uapi/drm/msm_drm.h +6 -2
index a9985fe..26c54f6 100644
@@ -104,10 +104,14 @@ struct drm_msm_gem_new {
 	__u32 handle;         /* out */
 };
 
+#define MSM_INFO_IOVA	0x01
+
+#define MSM_INFO_FLAGS (MSM_INFO_IOVA)
+
 struct drm_msm_gem_info {
 	__u32 handle;         /* in */
-	__u32 pad;
-	__u64 offset;         /* out, offset to pass to mmap() */
+	__u32 flags;	      /* in - combination of MSM_INFO_* flags */
+	__u64 offset;         /* out, mmap() offset or iova */
 };
 
 #define MSM_PREP_READ        0x01
Signature · 1 line
1.9.1
#7 lore raw

PATCH 6/13 drm/msm: get an iova from the address space instead of an id

In reply to Jordan Crouse

In the future we won't have a fixed set of addresses spaces. Instead of going through the effort of assigning a ID for each address space just use the address space itself as a token for getting / putting an iova.

This forces a few changes in the gem object however: instead of using a simple index into a list of domains, we need to maintain a list of them. Luckily the list will be pretty small; even with dynamic address spaces we wouldn't ever see more than two or three.

---
 drivers/gpu/drm/msm/adreno/a5xx_gpu.c     |   8 +-
 drivers/gpu/drm/msm/adreno/a5xx_power.c   |   5 +-
 drivers/gpu/drm/msm/adreno/adreno_gpu.c   |   6 +-
 drivers/gpu/drm/msm/dsi/dsi_host.c        |  15 +++-
 drivers/gpu/drm/msm/mdp/mdp4/mdp4_crtc.c  |   8 +-
 drivers/gpu/drm/msm/mdp/mdp4/mdp4_kms.c   |  16 ++--
 drivers/gpu/drm/msm/mdp/mdp4/mdp4_kms.h   |   4 -
 drivers/gpu/drm/msm/mdp/mdp4/mdp4_plane.c |  13 +--
 drivers/gpu/drm/msm/mdp/mdp5/mdp5_crtc.c  |   5 +-
 drivers/gpu/drm/msm/mdp/mdp5/mdp5_kms.c   |  11 +--
 drivers/gpu/drm/msm/mdp/mdp5/mdp5_kms.h   |   4 -
 drivers/gpu/drm/msm/mdp/mdp5/mdp5_plane.c |  13 +--
 drivers/gpu/drm/msm/msm_drv.c             |  15 +---
 drivers/gpu/drm/msm/msm_drv.h             |  25 +++---
 drivers/gpu/drm/msm/msm_fb.c              |  15 ++--
 drivers/gpu/drm/msm/msm_fbdev.c           |  10 ++-
 drivers/gpu/drm/msm/msm_gem.c             | 134 +++++++++++++++++++++---------
 drivers/gpu/drm/msm/msm_gem.h             |   4 +-
 drivers/gpu/drm/msm/msm_gem_submit.c      |   4 +-
 drivers/gpu/drm/msm/msm_gpu.c             |   8 +-
 drivers/gpu/drm/msm/msm_gpu.h             |   1 -
 drivers/gpu/drm/msm/msm_kms.h             |   3 +
 22 files changed, 193 insertions(+), 134 deletions(-)
drivers/gpu/drm/msm/adreno/a5xx_gpu.c +4 -4
index 2d3af90..fd54cc7 100644
@@ -306,7 +306,7 @@ static struct drm_gem_object *a5xx_ucode_load_bo(struct msm_gpu *gpu,
 	}
 
 	if (iova) {
-		int ret = msm_gem_get_iova(bo, gpu->id, iova);
+		int ret = msm_gem_get_iova(bo, gpu->aspace, iova);
 
 		if (ret) {
 			drm_gem_object_unreference_unlocked(bo);
@@ -693,19 +693,19 @@ static void a5xx_destroy(struct msm_gpu *gpu)
 
 	if (a5xx_gpu->pm4_bo) {
 		if (a5xx_gpu->pm4_iova)
-			msm_gem_put_iova(a5xx_gpu->pm4_bo, gpu->id);
+			msm_gem_put_iova(a5xx_gpu->pm4_bo, gpu->aspace);
 		drm_gem_object_unreference_unlocked(a5xx_gpu->pm4_bo);
 	}
 
 	if (a5xx_gpu->pfp_bo) {
 		if (a5xx_gpu->pfp_iova)
-			msm_gem_put_iova(a5xx_gpu->pfp_bo, gpu->id);
+			msm_gem_put_iova(a5xx_gpu->pfp_bo, gpu->aspace);
 		drm_gem_object_unreference_unlocked(a5xx_gpu->pfp_bo);
 	}
 
 	if (a5xx_gpu->gpmu_bo) {
 		if (a5xx_gpu->gpmu_iova)
-			msm_gem_put_iova(a5xx_gpu->gpmu_bo, gpu->id);
+			msm_gem_put_iova(a5xx_gpu->gpmu_bo, gpu->aspace);
 		drm_gem_object_unreference_unlocked(a5xx_gpu->gpmu_bo);
 	}
 
drivers/gpu/drm/msm/adreno/a5xx_power.c +3 -2
index ed0802e..2fdee44 100644
@@ -301,7 +301,8 @@ void a5xx_gpmu_ucode_init(struct msm_gpu *gpu)
 	if (IS_ERR(a5xx_gpu->gpmu_bo))
 		goto err;
 
-	if (msm_gem_get_iova(a5xx_gpu->gpmu_bo, gpu->id, &a5xx_gpu->gpmu_iova))
+	if (msm_gem_get_iova(a5xx_gpu->gpmu_bo, gpu->aspace,
+		&a5xx_gpu->gpmu_iova))
 		goto err;
 
 	ptr = msm_gem_get_vaddr(a5xx_gpu->gpmu_bo);
@@ -330,7 +331,7 @@ void a5xx_gpmu_ucode_init(struct msm_gpu *gpu)
 
 err:
 	if (a5xx_gpu->gpmu_iova)
-		msm_gem_put_iova(a5xx_gpu->gpmu_bo, gpu->id);
+		msm_gem_put_iova(a5xx_gpu->gpmu_bo, gpu->aspace);
 	if (a5xx_gpu->gpmu_bo)
 		drm_gem_object_unreference_unlocked(a5xx_gpu->gpmu_bo);
 
drivers/gpu/drm/msm/adreno/adreno_gpu.c +3 -3
index 5b63fc6..868a969 100644
@@ -64,7 +64,7 @@ int adreno_hw_init(struct msm_gpu *gpu)
 
 	DBG("%s", gpu->name);
 
-	ret = msm_gem_get_iova(gpu->rb->bo, gpu->id, &gpu->rb_iova);
+	ret = msm_gem_get_iova(gpu->rb->bo, gpu->aspace, &gpu->rb_iova);
 	if (ret) {
 		gpu->rb_iova = 0;
 		dev_err(gpu->dev->dev, "could not map ringbuffer: %d\n", ret);
@@ -409,7 +409,7 @@ int adreno_gpu_init(struct drm_device *drm, struct platform_device *pdev,
 		return -ENOMEM;
 	}
 
-	ret = msm_gem_get_iova(adreno_gpu->memptrs_bo, gpu->id,
+	ret = msm_gem_get_iova(adreno_gpu->memptrs_bo, gpu->aspace,
 			&adreno_gpu->memptrs_iova);
 	if (ret) {
 		dev_err(drm->dev, "could not map memptrs: %d\n", ret);
@@ -428,7 +428,7 @@ void adreno_gpu_cleanup(struct adreno_gpu *adreno_gpu)
 			msm_gem_put_vaddr(adreno_gpu->memptrs_bo);
 
 		if (adreno_gpu->memptrs_iova)
-			msm_gem_put_iova(adreno_gpu->memptrs_bo, gpu->id);
+			msm_gem_put_iova(adreno_gpu->memptrs_bo, gpu->aspace);
 
 		drm_gem_object_unreference_unlocked(adreno_gpu->memptrs_bo);
 	}
drivers/gpu/drm/msm/dsi/dsi_host.c +13 -2
index 4f79b10..4a58653 100644
@@ -28,6 +28,7 @@
 #include <linux/regmap.h>
 #include <video/mipi_display.h>
 
+#include "msm_kms.h"
 #include "dsi.h"
 #include "dsi.xml.h"
 #include "sfpb.xml.h"
@@ -975,6 +976,7 @@ static void dsi_wait4video_eng_busy(struct msm_dsi_host *msm_host)
 static int dsi_tx_buf_alloc(struct msm_dsi_host *msm_host, int size)
 {
 	struct drm_device *dev = msm_host->dev;
+	struct msm_drm_private *priv = dev->dev_private;
 	const struct msm_dsi_cfg_handler *cfg_hnd = msm_host->cfg_hnd;
 	int ret;
 	uint64_t iova;
@@ -991,7 +993,13 @@ static int dsi_tx_buf_alloc(struct msm_dsi_host *msm_host, int size)
 			return ret;
 		}
 
-		ret = msm_gem_get_iova_locked(msm_host->tx_gem_obj, 0, &iova);
+		if (!priv->kms) {
+			pr_err("%s: No KMS is initialized\n", __func__);
+			return -ENODEV;
+		}
+
+		ret = msm_gem_get_iova_locked(msm_host->tx_gem_obj,
+			priv->kms->aspace, &iova);
 		mutex_unlock(&dev->struct_mutex);
 		if (ret) {
 			pr_err("%s: failed to get iova, %d\n", __func__, ret);
@@ -1023,9 +1031,12 @@ static int dsi_tx_buf_alloc(struct msm_dsi_host *msm_host, int size)
 static void dsi_tx_buf_free(struct msm_dsi_host *msm_host)
 {
 	struct drm_device *dev = msm_host->dev;
+	struct msm_drm_private *priv = dev->dev_private;
 
 	if (msm_host->tx_gem_obj) {
-		msm_gem_put_iova(msm_host->tx_gem_obj, 0);
+		if (priv->kms)
+			msm_gem_put_iova(msm_host->tx_gem_obj,
+				priv->kms->aspace);
 		mutex_lock(&dev->struct_mutex);
 		msm_gem_free_object(msm_host->tx_gem_obj);
 		msm_host->tx_gem_obj = NULL;
drivers/gpu/drm/msm/mdp/mdp4/mdp4_crtc.c +5 -3
index f29194a..ca45f47 100644
@@ -127,7 +127,7 @@ static void unref_cursor_worker(struct drm_flip_work *work, void *val)
 		container_of(work, struct mdp4_crtc, unref_cursor_work);
 	struct mdp4_kms *mdp4_kms = get_kms(&mdp4_crtc->base);
 
-	msm_gem_put_iova(val, mdp4_kms->id);
+	msm_gem_put_iova(val, mdp4_kms->base.base.aspace);
 	drm_gem_object_unreference_unlocked(val);
 }
 
@@ -372,7 +372,8 @@ static void update_cursor(struct drm_crtc *crtc)
 		if (next_bo) {
 			/* take a obj ref + iova ref when we start scanning out: */
 			drm_gem_object_reference(next_bo);
-			msm_gem_get_iova_locked(next_bo, mdp4_kms->id, &iova);
+			msm_gem_get_iova_locked(next_bo,
+				mdp4_kms->base.base.aspace, &iova);
 
 			/* enable cursor: */
 			mdp4_write(mdp4_kms, REG_MDP4_DMA_CURSOR_SIZE(dma),
@@ -429,7 +430,8 @@ static int mdp4_crtc_cursor_set(struct drm_crtc *crtc,
 	}
 
 	if (cursor_bo) {
-		ret = msm_gem_get_iova(cursor_bo, mdp4_kms->id, &iova);
+		ret = msm_gem_get_iova(cursor_bo, mdp4_kms->base.base.aspace,
+			&iova);
 		if (ret)
 			goto fail;
 	} else {
drivers/gpu/drm/msm/mdp/mdp4/mdp4_kms.c +5 -11
index 6295204..34425b16 100644
@@ -160,10 +160,11 @@ static void mdp4_destroy(struct msm_kms *kms)
 {
 	struct mdp4_kms *mdp4_kms = to_mdp4_kms(to_mdp_kms(kms));
 	struct device *dev = mdp4_kms->dev->dev;
-	struct msm_gem_address_space *aspace = mdp4_kms->aspace;
+	struct msm_gem_address_space *aspace = kms->aspace;
 
 	if (mdp4_kms->blank_cursor_iova)
-		msm_gem_put_iova(mdp4_kms->blank_cursor_bo, mdp4_kms->id);
+		msm_gem_put_iova(mdp4_kms->blank_cursor_bo, aspace);
+
 	drm_gem_object_unreference_unlocked(mdp4_kms->blank_cursor_bo);
 
 	if (aspace) {
@@ -536,7 +537,7 @@ struct msm_kms *mdp4_kms_init(struct drm_device *dev)
 			goto fail;
 		}
 
-		mdp4_kms->aspace = aspace;
+		kms->aspace = aspace;
 
 		ret = aspace->mmu->funcs->attach(aspace->mmu, iommu_ports,
 				ARRAY_SIZE(iommu_ports));
@@ -548,13 +549,6 @@ struct msm_kms *mdp4_kms_init(struct drm_device *dev)
 		aspace = NULL;
 	}
 
-	mdp4_kms->id = msm_register_address_space(dev, aspace);
-	if (mdp4_kms->id < 0) {
-		ret = mdp4_kms->id;
-		dev_err(dev->dev, "failed to register mdp4 iommu: %d\n", ret);
-		goto fail;
-	}
-
 	ret = modeset_init(mdp4_kms);
 	if (ret) {
 		dev_err(dev->dev, "modeset_init failed: %d\n", ret);
@@ -571,7 +565,7 @@ struct msm_kms *mdp4_kms_init(struct drm_device *dev)
 		goto fail;
 	}
 
-	ret = msm_gem_get_iova(mdp4_kms->blank_cursor_bo, mdp4_kms->id,
+	ret = msm_gem_get_iova(mdp4_kms->blank_cursor_bo, kms->aspace,
 			&mdp4_kms->blank_cursor_iova);
 	if (ret) {
 		dev_err(dev->dev, "could not pin blank-cursor bo: %d\n", ret);
drivers/gpu/drm/msm/mdp/mdp4/mdp4_kms.h +0 -4
index 62712ca..0eacaf0 100644
@@ -32,9 +32,6 @@ struct mdp4_kms {
 
 	int rev;
 
-	/* mapper-id used to request GEM buffer mapped for scanout: */
-	int id;
-
 	void __iomem *mmio;
 
 	struct regulator *vdd;
@@ -43,7 +40,6 @@ struct mdp4_kms {
 	struct clk *pclk;
 	struct clk *lut_clk;
 	struct clk *axi_clk;
-	struct msm_gem_address_space *aspace;
 
 	struct mdp_irq error_handler;
 
drivers/gpu/drm/msm/mdp/mdp4/mdp4_plane.c +7 -6
index 53619d0..c48984d 100644
@@ -109,7 +109,7 @@ static int mdp4_plane_prepare_fb(struct drm_plane *plane,
 		return 0;
 
 	DBG("%s: prepare: FB[%u]", mdp4_plane->name, fb->base.id);
-	return msm_framebuffer_prepare(fb, mdp4_kms->id);
+	return msm_framebuffer_prepare(fb, mdp4_kms->base.base.aspace);
 }
 
 static void mdp4_plane_cleanup_fb(struct drm_plane *plane,
@@ -123,7 +123,7 @@ static void mdp4_plane_cleanup_fb(struct drm_plane *plane,
 		return;
 
 	DBG("%s: cleanup: FB[%u]", mdp4_plane->name, fb->base.id);
-	msm_framebuffer_cleanup(fb, mdp4_kms->id);
+	msm_framebuffer_cleanup(fb, mdp4_kms->base.base.aspace);
 }
 
 
@@ -161,6 +161,7 @@ static void mdp4_plane_set_scanout(struct drm_plane *plane,
 {
 	struct mdp4_plane *mdp4_plane = to_mdp4_plane(plane);
 	struct mdp4_kms *mdp4_kms = get_kms(plane);
+	struct msm_kms *kms = &mdp4_kms->base.base;
 	enum mdp4_pipe pipe = mdp4_plane->pipe;
 
 	mdp4_write(mdp4_kms, REG_MDP4_PIPE_SRC_STRIDE_A(pipe),
@@ -172,13 +173,13 @@ static void mdp4_plane_set_scanout(struct drm_plane *plane,
 			MDP4_PIPE_SRC_STRIDE_B_P3(fb->pitches[3]));
 
 	mdp4_write(mdp4_kms, REG_MDP4_PIPE_SRCP0_BASE(pipe),
-			msm_framebuffer_iova(fb, mdp4_kms->id, 0));
+			msm_framebuffer_iova(fb, kms->aspace, 0));
 	mdp4_write(mdp4_kms, REG_MDP4_PIPE_SRCP1_BASE(pipe),
-			msm_framebuffer_iova(fb, mdp4_kms->id, 1));
+			msm_framebuffer_iova(fb, kms->aspace, 1));
 	mdp4_write(mdp4_kms, REG_MDP4_PIPE_SRCP2_BASE(pipe),
-			msm_framebuffer_iova(fb, mdp4_kms->id, 2));
+			msm_framebuffer_iova(fb, kms->aspace, 2));
 	mdp4_write(mdp4_kms, REG_MDP4_PIPE_SRCP3_BASE(pipe),
-			msm_framebuffer_iova(fb, mdp4_kms->id, 3));
+			msm_framebuffer_iova(fb, kms->aspace, 3));
 
 	plane->fb = fb;
 }
drivers/gpu/drm/msm/mdp/mdp5/mdp5_crtc.c +3 -2
index 9217e0d..dfa3c14 100644
@@ -161,7 +161,7 @@ static void unref_cursor_worker(struct drm_flip_work *work, void *val)
 		container_of(work, struct mdp5_crtc, unref_cursor_work);
 	struct mdp5_kms *mdp5_kms = get_kms(&mdp5_crtc->base);
 
-	msm_gem_put_iova(val, mdp5_kms->id);
+	msm_gem_put_iova(val, mdp5_kms->base.base.aspace);
 	drm_gem_object_unreference_unlocked(val);
 }
 
@@ -758,7 +758,8 @@ static int mdp5_crtc_cursor_set(struct drm_crtc *crtc,
 	if (!cursor_bo)
 		return -ENOENT;
 
-	ret = msm_gem_get_iova(cursor_bo, mdp5_kms->id, &cursor_addr);
+	ret = msm_gem_get_iova(cursor_bo, mdp5_kms->base.base.aspace,
+		&cursor_addr);
 	if (ret)
 		return -EINVAL;
 
drivers/gpu/drm/msm/mdp/mdp5/mdp5_kms.c +2 -9
index d3d6b4c..b3db53a 100644
@@ -163,7 +163,7 @@ static void mdp5_set_encoder_mode(struct msm_kms *kms,
 static void mdp5_kms_destroy(struct msm_kms *kms)
 {
 	struct mdp5_kms *mdp5_kms = to_mdp5_kms(to_mdp_kms(kms));
-	struct msm_gem_address_space *aspace = mdp5_kms->aspace;
+	struct msm_gem_address_space *aspace = kms->aspace;
 	int i;
 
 	for (i = 0; i < mdp5_kms->num_hwmixers; i++)
@@ -692,7 +692,7 @@ struct msm_kms *mdp5_kms_init(struct drm_device *dev)
 			goto fail;
 		}
 
-		mdp5_kms->aspace = aspace;
+		kms->aspace = aspace;
 
 		ret = aspace->mmu->funcs->attach(aspace->mmu, iommu_ports,
 				ARRAY_SIZE(iommu_ports));
@@ -707,13 +707,6 @@ struct msm_kms *mdp5_kms_init(struct drm_device *dev)
 		aspace = NULL;;
 	}
 
-	mdp5_kms->id = msm_register_address_space(dev, aspace);
-	if (mdp5_kms->id < 0) {
-		ret = mdp5_kms->id;
-		dev_err(&pdev->dev, "failed to register mdp5 iommu: %d\n", ret);
-		goto fail;
-	}
-
 	ret = modeset_init(mdp5_kms);
 	if (ret) {
 		dev_err(&pdev->dev, "modeset_init failed: %d\n", ret);
drivers/gpu/drm/msm/mdp/mdp5/mdp5_kms.h +0 -4
index 8bdb7ee..17caa0e 100644
@@ -55,10 +55,6 @@ struct mdp5_kms {
 	struct mdp5_state *state;
 	struct drm_modeset_lock state_lock;
 
-	/* mapper-id used to request GEM buffer mapped for scanout: */
-	int id;
-	struct msm_gem_address_space *aspace;
-
 	struct mdp5_smp *smp;
 	struct mdp5_ctl_manager *ctlm;
 
drivers/gpu/drm/msm/mdp/mdp5/mdp5_plane.c +7 -6
index a38c5fe..4c0e1ae 100644
@@ -279,7 +279,7 @@ static int mdp5_plane_prepare_fb(struct drm_plane *plane,
 		return 0;
 
 	DBG("%s: prepare: FB[%u]", plane->name, fb->base.id);
-	return msm_framebuffer_prepare(fb, mdp5_kms->id);
+	return msm_framebuffer_prepare(fb, mdp5_kms->base.base.aspace);
 }
 
 static void mdp5_plane_cleanup_fb(struct drm_plane *plane,
@@ -292,7 +292,7 @@ static void mdp5_plane_cleanup_fb(struct drm_plane *plane,
 		return;
 
 	DBG("%s: cleanup: FB[%u]", plane->name, fb->base.id);
-	msm_framebuffer_cleanup(fb, mdp5_kms->id);
+	msm_framebuffer_cleanup(fb, mdp5_kms->base.base.aspace);
 }
 
 #define FRAC_16_16(mult, div)    (((mult) << 16) / (div))
@@ -495,6 +495,7 @@ static void set_scanout_locked(struct mdp5_kms *mdp5_kms,
 			       enum mdp5_pipe pipe,
 			       struct drm_framebuffer *fb)
 {
+	struct msm_kms *kms = &mdp5_kms->base.base;
 	mdp5_write(mdp5_kms, REG_MDP5_PIPE_SRC_STRIDE_A(pipe),
 			MDP5_PIPE_SRC_STRIDE_A_P0(fb->pitches[0]) |
 			MDP5_PIPE_SRC_STRIDE_A_P1(fb->pitches[1]));
@@ -504,13 +505,13 @@ static void set_scanout_locked(struct mdp5_kms *mdp5_kms,
 			MDP5_PIPE_SRC_STRIDE_B_P3(fb->pitches[3]));
 
 	mdp5_write(mdp5_kms, REG_MDP5_PIPE_SRC0_ADDR(pipe),
-			msm_framebuffer_iova(fb, mdp5_kms->id, 0));
+			msm_framebuffer_iova(fb, kms->aspace, 0));
 	mdp5_write(mdp5_kms, REG_MDP5_PIPE_SRC1_ADDR(pipe),
-			msm_framebuffer_iova(fb, mdp5_kms->id, 1));
+			msm_framebuffer_iova(fb, kms->aspace, 1));
 	mdp5_write(mdp5_kms, REG_MDP5_PIPE_SRC2_ADDR(pipe),
-			msm_framebuffer_iova(fb, mdp5_kms->id, 2));
+			msm_framebuffer_iova(fb, kms->aspace, 2));
 	mdp5_write(mdp5_kms, REG_MDP5_PIPE_SRC3_ADDR(pipe),
-			msm_framebuffer_iova(fb, mdp5_kms->id, 3));
+			msm_framebuffer_iova(fb, kms->aspace, 3));
 }
 
 /* Note: mdp5_plane->pipe_lock must be locked */
drivers/gpu/drm/msm/msm_drv.c +1 -14
index 257ee81..005ae06 100644
@@ -51,19 +51,6 @@ static void msm_fb_output_poll_changed(struct drm_device *dev)
 	.atomic_state_free = msm_atomic_state_free,
 };
 
-int msm_register_address_space(struct drm_device *dev,
-		struct msm_gem_address_space *aspace)
-{
-	struct msm_drm_private *priv = dev->dev_private;
-
-	if (WARN_ON(priv->num_aspaces >= ARRAY_SIZE(priv->aspace)))
-		return -EINVAL;
-
-	priv->aspace[priv->num_aspaces] = aspace;
-
-	return priv->num_aspaces++;
-}
-
 #ifdef CONFIG_DRM_MSM_REGISTER_LOGGING
 static bool reglog = false;
 MODULE_PARM_DESC(reglog, "Enable register read/write logging");
@@ -707,7 +694,7 @@ static int msm_ioctl_gem_info_iova(struct drm_device *dev,
 	if (!priv->gpu)
 		return -EINVAL;
 
-	return msm_gem_get_iova(obj, priv->gpu->id, iova);
+	return msm_gem_get_iova(obj, priv->gpu->aspace, iova);
 }
 
 static int msm_ioctl_gem_info(struct drm_device *dev, void *data,
drivers/gpu/drm/msm/msm_drv.h +14 -11
index 28b6f9b..69e839c 100644
@@ -183,9 +183,6 @@ int msm_atomic_commit(struct drm_device *dev,
 void msm_atomic_state_clear(struct drm_atomic_state *state);
 void msm_atomic_state_free(struct drm_atomic_state *state);
 
-int msm_register_address_space(struct drm_device *dev,
-		struct msm_gem_address_space *aspace);
-
 void msm_gem_unmap_vma(struct msm_gem_address_space *aspace,
 		struct msm_gem_vma *vma, struct sg_table *sgt);
 int msm_gem_map_vma(struct msm_gem_address_space *aspace,
@@ -209,13 +206,16 @@ int msm_gem_mmap_obj(struct drm_gem_object *obj,
 int msm_gem_mmap(struct file *filp, struct vm_area_struct *vma);
 int msm_gem_fault(struct vm_fault *vmf);
 uint64_t msm_gem_mmap_offset(struct drm_gem_object *obj);
-int msm_gem_get_iova_locked(struct drm_gem_object *obj, int id,
-		uint64_t *iova);
-int msm_gem_get_iova(struct drm_gem_object *obj, int id, uint64_t *iova);
-uint64_t msm_gem_iova(struct drm_gem_object *obj, int id);
+int msm_gem_get_iova_locked(struct drm_gem_object *obj,
+		struct msm_gem_address_space *aspace, uint64_t *iova);
+int msm_gem_get_iova(struct drm_gem_object *obj,
+		struct msm_gem_address_space *aspace, uint64_t *iova);
+uint64_t msm_gem_iova(struct drm_gem_object *obj,
+		struct msm_gem_address_space *aspace);
 struct page **msm_gem_get_pages(struct drm_gem_object *obj);
 void msm_gem_put_pages(struct drm_gem_object *obj);
-void msm_gem_put_iova(struct drm_gem_object *obj, int id);
+void msm_gem_put_iova(struct drm_gem_object *obj,
+		struct msm_gem_address_space *aspace);
 int msm_gem_dumb_create(struct drm_file *file, struct drm_device *dev,
 		struct drm_mode_create_dumb *args);
 int msm_gem_dumb_map_offset(struct drm_file *file, struct drm_device *dev,
@@ -250,9 +250,12 @@ struct drm_gem_object *msm_gem_new(struct drm_device *dev,
 struct drm_gem_object *msm_gem_import(struct drm_device *dev,
 		struct dma_buf *dmabuf, struct sg_table *sgt);
 
-int msm_framebuffer_prepare(struct drm_framebuffer *fb, int id);
-void msm_framebuffer_cleanup(struct drm_framebuffer *fb, int id);
-uint32_t msm_framebuffer_iova(struct drm_framebuffer *fb, int id, int plane);
+int msm_framebuffer_prepare(struct drm_framebuffer *fb,
+		struct msm_gem_address_space *aspace);
+void msm_framebuffer_cleanup(struct drm_framebuffer *fb,
+		struct msm_gem_address_space *aspace);
+uint32_t msm_framebuffer_iova(struct drm_framebuffer *fb,
+		struct msm_gem_address_space *aspace, int plane);
 struct drm_gem_object *msm_framebuffer_bo(struct drm_framebuffer *fb, int plane);
 const struct msm_format *msm_framebuffer_format(struct drm_framebuffer *fb);
 struct drm_framebuffer *msm_framebuffer_init(struct drm_device *dev,
drivers/gpu/drm/msm/msm_fb.c +9 -6
index 5cf165c..36505f3 100644
@@ -84,14 +84,15 @@ void msm_framebuffer_describe(struct drm_framebuffer *fb, struct seq_file *m)
  * should be fine, since only the scanout (mdpN) side of things needs
  * this, the gpu doesn't care about fb's.
  */
-int msm_framebuffer_prepare(struct drm_framebuffer *fb, int id)
+int msm_framebuffer_prepare(struct drm_framebuffer *fb,
+		struct msm_gem_address_space *aspace)
 {
 	struct msm_framebuffer *msm_fb = to_msm_framebuffer(fb);
 	int ret, i, n = fb->format->num_planes;
 	uint64_t iova;
 
 	for (i = 0; i < n; i++) {
-		ret = msm_gem_get_iova(msm_fb->planes[i], id, &iova);
+		ret = msm_gem_get_iova(msm_fb->planes[i], aspace, &iova);
 		DBG("FB[%u]: iova[%d]: %08llx (%d)", fb->base.id, i, iova, ret);
 		if (ret)
 			return ret;
@@ -100,21 +101,23 @@ int msm_framebuffer_prepare(struct drm_framebuffer *fb, int id)
 	return 0;
 }
 
-void msm_framebuffer_cleanup(struct drm_framebuffer *fb, int id)
+void msm_framebuffer_cleanup(struct drm_framebuffer *fb,
+		struct msm_gem_address_space *aspace)
 {
 	struct msm_framebuffer *msm_fb = to_msm_framebuffer(fb);
 	int i, n = fb->format->num_planes;
 
 	for (i = 0; i < n; i++)
-		msm_gem_put_iova(msm_fb->planes[i], id);
+		msm_gem_put_iova(msm_fb->planes[i], aspace);
 }
 
-uint32_t msm_framebuffer_iova(struct drm_framebuffer *fb, int id, int plane)
+uint32_t msm_framebuffer_iova(struct drm_framebuffer *fb,
+		struct msm_gem_address_space *aspace, int plane)
 {
 	struct msm_framebuffer *msm_fb = to_msm_framebuffer(fb);
 	if (!msm_fb->planes[plane])
 		return 0;
-	return msm_gem_iova(msm_fb->planes[plane], id) + fb->offsets[plane];
+	return msm_gem_iova(msm_fb->planes[plane], aspace) + fb->offsets[plane];
 }
 
 struct drm_gem_object *msm_framebuffer_bo(struct drm_framebuffer *fb, int plane)
drivers/gpu/drm/msm/msm_fbdev.c +9 -1
index 951e40f..e071828 100644
@@ -20,6 +20,7 @@
 #include "drm_crtc.h"
 #include "drm_fb_helper.h"
 #include "msm_gem.h"
+#include "msm_kms.h"
 
 extern int msm_gem_mmap_obj(struct drm_gem_object *obj,
 					struct vm_area_struct *vma);
@@ -73,6 +74,7 @@ static int msm_fbdev_create(struct drm_fb_helper *helper,
 {
 	struct msm_fbdev *fbdev = to_msm_fbdev(helper);
 	struct drm_device *dev = helper->dev;
+	struct msm_drm_private *priv = dev->dev_private;
 	struct drm_framebuffer *fb = NULL;
 	struct fb_info *fbi = NULL;
 	struct drm_mode_fb_cmd2 mode_cmd = {0};
@@ -124,7 +126,13 @@ static int msm_fbdev_create(struct drm_fb_helper *helper,
 	 * in panic (ie. lock-safe, etc) we could avoid pinning the
 	 * buffer now:
 	 */
-	ret = msm_gem_get_iova_locked(fbdev->bo, 0, &paddr);
+
+	if (!priv->kms) {
+		ret = -ENODEV;
+		goto fail_unlock;
+	}
+
+	ret = msm_gem_get_iova_locked(fbdev->bo, priv->kms->aspace, &paddr);
 	if (ret) {
 		dev_err(dev->dev, "failed to get buffer obj iova: %d\n", ret);
 		goto fail_unlock;
drivers/gpu/drm/msm/msm_gem.c +97 -37

215 lines not on this page · raw message

drivers/gpu/drm/msm/msm_gem.h +3 -1
index 1b4cf20..2767014 100644
@@ -37,7 +37,9 @@ struct msm_gem_address_space {
 
 struct msm_gem_vma {
 	struct drm_mm_node node;
+	struct msm_gem_address_space *aspace;
 	uint64_t iova;
+	struct list_head list;
 };
 
 struct msm_gem_object {
@@ -77,7 +79,7 @@ struct msm_gem_object {
 	struct sg_table *sgt;
 	void *vaddr;
 
-	struct msm_gem_vma domain[NUM_DOMAINS];
+	struct list_head domains;
 
 	/* normally (resv == &_resv) except for imported bo's */
 	struct reservation_object *resv;
drivers/gpu/drm/msm/msm_gem_submit.c +2 -2
index c51cf03..4c2525e 100644
@@ -158,7 +158,7 @@ static void submit_unlock_unpin_bo(struct msm_gem_submit *submit, int i)
 	struct msm_gem_object *msm_obj = submit->bos[i].obj;
 
 	if (submit->bos[i].flags & BO_PINNED)
-		msm_gem_put_iova(&msm_obj->base, submit->gpu->id);
+		msm_gem_put_iova(&msm_obj->base, submit->gpu->aspace);
 
 	if (submit->bos[i].flags & BO_LOCKED)
 		ww_mutex_unlock(&msm_obj->resv->lock);
@@ -246,7 +246,7 @@ static int submit_pin_objects(struct msm_gem_submit *submit)
 
 		/* if locking succeeded, pin bo: */
 		ret = msm_gem_get_iova_locked(&msm_obj->base,
-				submit->gpu->id, &iova);
+				submit->gpu->aspace, &iova);
 
 		if (ret)
 			break;
drivers/gpu/drm/msm/msm_gpu.c +3 -5
index 97b9c38..8e5174fa 100644
@@ -414,7 +414,7 @@ static void retire_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit)
 		struct msm_gem_object *msm_obj = submit->bos[i].obj;
 		/* move to inactive: */
 		msm_gem_move_to_inactive(&msm_obj->base);
-		msm_gem_put_iova(&msm_obj->base, gpu->id);
+		msm_gem_put_iova(&msm_obj->base, gpu->aspace);
 		drm_gem_object_unreference(&msm_obj->base);
 	}
 
@@ -496,7 +496,7 @@ void msm_gpu_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit,
 		/* submit takes a reference to the bo and iova until retired: */
 		drm_gem_object_reference(&msm_obj->base);
 		msm_gem_get_iova_locked(&msm_obj->base,
-				submit->gpu->id, &iova);
+				submit->gpu->aspace, &iova);
 
 		if (submit->bos[i].flags & MSM_SUBMIT_BO_WRITE)
 			msm_gem_move_to_active(&msm_obj->base, gpu, true, submit->fence);
@@ -658,8 +658,6 @@ int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev,
 	} else {
 		dev_info(drm->dev, "%s: no IOMMU, fallback to VRAM carveout!\n", name);
 	}
-	gpu->id = msm_register_address_space(drm, gpu->aspace);
-
 
 	/* Create ringbuffer: */
 	mutex_lock(&drm->struct_mutex);
@@ -693,7 +691,7 @@ void msm_gpu_cleanup(struct msm_gpu *gpu)
 
 	if (gpu->rb) {
 		if (gpu->rb_iova)
-			msm_gem_put_iova(gpu->rb->bo, gpu->id);
+			msm_gem_put_iova(gpu->rb->bo, gpu->aspace);
 		msm_ringbuffer_destroy(gpu->rb);
 	}
 
drivers/gpu/drm/msm/msm_gpu.h +0 -1
index 6364502..4892e18 100644
@@ -98,7 +98,6 @@ struct msm_gpu {
 	int irq;
 
 	struct msm_gem_address_space *aspace;
-	int id;
 
 	/* Power Control: */
 	struct regulator *gpu_reg, *gpu_cx;
drivers/gpu/drm/msm/msm_kms.h +3 -0
index faa22c7..a8f2ba5 100644
@@ -72,6 +72,9 @@ struct msm_kms {
 
 	/* irq number to be passed on to drm_irq_install */
 	int irq;
+
+	/* mapper-id used to request GEM buffer mapped for scanout: */
+	struct msm_gem_address_space *aspace;
 };
 
 /**
Signature · 1 line
1.9.1
#8 lore raw

PATCH 7/13 drm/msm: Add a struct to pass configuration to msm_gpu_init()

In reply to Jordan Crouse

The amount of information that we need to pass into msm_gpu_init() is steadily increasing, so add a new struct to stabilize the function call and make it easier to add new configuration down the line.

---
 drivers/gpu/drm/msm/adreno/adreno_gpu.c | 12 ++++++++++--
 drivers/gpu/drm/msm/msm_gpu.c           | 13 ++++++-------
 drivers/gpu/drm/msm/msm_gpu.h           | 11 ++++++++++-
 3 files changed, 26 insertions(+), 10 deletions(-)
drivers/gpu/drm/msm/adreno/adreno_gpu.c +10 -2
index 868a969..107f91b 100644
@@ -342,6 +342,7 @@ int adreno_gpu_init(struct drm_device *drm, struct platform_device *pdev,
 		struct adreno_gpu *adreno_gpu, const struct adreno_gpu_funcs *funcs)
 {
 	struct adreno_platform_config *config = pdev->dev.platform_data;
+	struct msm_gpu_config adreno_gpu_config  = { 0 };
 	struct msm_gpu *gpu = &adreno_gpu->base;
 	int ret;
 
@@ -360,9 +361,16 @@ int adreno_gpu_init(struct drm_device *drm, struct platform_device *pdev,
 	DBG("fast_rate=%u, slow_rate=27000000, bus_freq=%u",
 			gpu->fast_rate, gpu->bus_freq);
 
+	adreno_gpu_config.ioname = "kgsl_3d0_reg_memory";
+	adreno_gpu_config.irqname = "kgsl_3d0_irq";
+
+	adreno_gpu_config.va_start = SZ_16M;
+	adreno_gpu_config.va_end = 0xffffffff;
+
+	adreno_gpu_config.ringsz = RB_SIZE;
+
 	ret = msm_gpu_init(drm, pdev, &adreno_gpu->base, &funcs->base,
-			adreno_gpu->info->name, "kgsl_3d0_reg_memory", "kgsl_3d0_irq",
-			RB_SIZE);
+			adreno_gpu->info->name, &adreno_gpu_config);
 	if (ret)
 		return ret;
 
drivers/gpu/drm/msm/msm_gpu.c +6 -7
index 8e5174fa..1f753f0 100644
@@ -562,7 +562,7 @@ static int get_clocks(struct platform_device *pdev, struct msm_gpu *gpu)
 
 int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev,
 		struct msm_gpu *gpu, const struct msm_gpu_funcs *funcs,
-		const char *name, const char *ioname, const char *irqname, int ringsz)
+		const char *name, struct msm_gpu_config *config)
 {
 	struct iommu_domain *iommu;
 	int ret;
@@ -593,14 +593,14 @@ int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev,
 
 
 	/* Map registers: */
-	gpu->mmio = msm_ioremap(pdev, ioname, name);
+	gpu->mmio = msm_ioremap(pdev, config->ioname, name);
 	if (IS_ERR(gpu->mmio)) {
 		ret = PTR_ERR(gpu->mmio);
 		goto fail;
 	}
 
 	/* Get Interrupt: */
-	gpu->irq = platform_get_irq_byname(pdev, irqname);
+	gpu->irq = platform_get_irq_byname(pdev, config->irqname);
 	if (gpu->irq < 0) {
 		ret = gpu->irq;
 		dev_err(drm->dev, "failed to get irq: %d\n", ret);
@@ -640,9 +640,8 @@ int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev,
 	 */
 	iommu = iommu_domain_alloc(&platform_bus_type);
 	if (iommu) {
-		/* TODO 32b vs 64b address space.. */
-		iommu->geometry.aperture_start = SZ_16M;
-		iommu->geometry.aperture_end = 0xffffffff;
+		iommu->geometry.aperture_start = config->va_start;
+		iommu->geometry.aperture_end = config->va_end;
 
 		dev_info(drm->dev, "%s: using IOMMU\n", name);
 		gpu->aspace = msm_gem_address_space_create(&pdev->dev,
@@ -661,7 +660,7 @@ int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev,
 
 	/* Create ringbuffer: */
 	mutex_lock(&drm->struct_mutex);
-	gpu->rb = msm_ringbuffer_new(gpu, ringsz);
+	gpu->rb = msm_ringbuffer_new(gpu, config->ringsz);
 	mutex_unlock(&drm->struct_mutex);
 	if (IS_ERR(gpu->rb)) {
 		ret = PTR_ERR(gpu->rb);
drivers/gpu/drm/msm/msm_gpu.h +10 -1
index 4892e18..df4e277 100644
@@ -28,6 +28,14 @@
 struct msm_gem_submit;
 struct msm_gpu_perfcntr;
 
+struct msm_gpu_config {
+	const char *ioname;
+	const char *irqname;
+	uint64_t va_start;
+	uint64_t va_end;
+	unsigned int ringsz;
+};
+
 /* So far, with hardware that I've seen to date, we can have:
  *  + zero, one, or two z180 2d cores
  *  + a3xx or a2xx 3d core, which share a common CP (the firmware
@@ -207,7 +215,8 @@ void msm_gpu_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit,
 
 int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev,
 		struct msm_gpu *gpu, const struct msm_gpu_funcs *funcs,
-		const char *name, const char *ioname, const char *irqname, int ringsz);
+		const char *name, struct msm_gpu_config *config);
+
 void msm_gpu_cleanup(struct msm_gpu *gpu);
 
 struct msm_gpu *adreno_load_gpu(struct drm_device *dev);
Signature · 1 line
1.9.1
#9 lore raw

PATCH 8/13 drm/msm: Remove memptrs->wptr

In reply to Jordan Crouse

memptrs->wptr seems to be unused. Remove it to avoid confusing the upcoming preemption code.

---
 drivers/gpu/drm/msm/adreno/adreno_gpu.c | 3 ---
 drivers/gpu/drm/msm/adreno/adreno_gpu.h | 1 -
 2 files changed, 4 deletions(-)
drivers/gpu/drm/msm/adreno/adreno_gpu.c +0 -3
index 107f91b..4a24506 100644
@@ -77,7 +77,6 @@ int adreno_hw_init(struct msm_gpu *gpu)
 	/* reset completed fence seqno: */
 	adreno_gpu->memptrs->fence = gpu->fctx->completed_fence;
 	adreno_gpu->memptrs->rptr  = 0;
-	adreno_gpu->memptrs->wptr  = 0;
 
 	/* Setup REG_CP_RB_CNTL: */
 	adreno_gpu_write(adreno_gpu, REG_ADRENO_CP_RB_CNTL,
@@ -258,7 +257,6 @@ void adreno_show(struct msm_gpu *gpu, struct seq_file *m)
 	seq_printf(m, "fence:    %d/%d\n", adreno_gpu->memptrs->fence,
 			gpu->fctx->last_fence);
 	seq_printf(m, "rptr:     %d\n", get_rptr(adreno_gpu));
-	seq_printf(m, "wptr:     %d\n", adreno_gpu->memptrs->wptr);
 	seq_printf(m, "rb wptr:  %d\n", get_wptr(gpu->rb));
 
 	/* dump these out in a form that can be parsed by demsm: */
@@ -294,7 +292,6 @@ void adreno_dump_info(struct msm_gpu *gpu)
 	printk("fence:    %d/%d\n", adreno_gpu->memptrs->fence,
 			gpu->fctx->last_fence);
 	printk("rptr:     %d\n", get_rptr(adreno_gpu));
-	printk("wptr:     %d\n", adreno_gpu->memptrs->wptr);
 	printk("rb wptr:  %d\n", get_wptr(gpu->rb));
 }
 
drivers/gpu/drm/msm/adreno/adreno_gpu.h +0 -1
index 12b1483..4d9165f 100644
@@ -87,7 +87,6 @@ struct adreno_info {
 
 struct adreno_rbmemptrs {
 	volatile uint32_t rptr;
-	volatile uint32_t wptr;
 	volatile uint32_t fence;
 };
 
Signature · 1 line
1.9.1
#10 lore raw

PATCH 9/13 drm/msm: Add drawqueues

In reply to Jordan Crouse

Currently the priority and other behavior of a command stream is provided by the user application during submission and the application is expected to internally maintain the settings for each 'context' or 'rendering queue' and specify the correct ones.

This works okay for simple cases but as applications become more complex we might want to start doing permissions checks for some operations. For example we may want to restrict access to the highest priority ringbuffer only to processes identified as the master (usually the compositor). This would be easiest if we did this once and then reused the settings for each submission.

Add kernel-side draw queues to be analogous to 'contexts' or 'rendering queues' on the application side. Each file descriptor instance will maintain its own list of draw queues indexed starting at 1. Draw queues cannot be shared between file descriptors.

For backwards compatibility context id '0' is defined as a default context specifying middle priority and no special flags. This is intended to be the usual configuration for 99% of applications so that a garden variety application can function correctly without creating a draw queue. Only those applications intending to use different priority levels or submission flags need to create and use a "new" draw queue.

Missing from this patch is the 'queue' part of a 'drawqueue'. The code isn't quite to the point where we need individual render queues for a file descriptor instance but if we did this is where it would go so there is no harm in laying down the basis for the future.

---
 drivers/gpu/drm/msm/msm_drv.c        | 99 +++++++++++++++++++++++++++++++++---
 drivers/gpu/drm/msm/msm_drv.h        |  8 ++-
 drivers/gpu/drm/msm/msm_gem_submit.c | 32 ++++++++++++
 drivers/gpu/drm/msm/msm_gpu.h        |  7 +++
 include/uapi/drm/msm_drm.h           | 20 ++++++++
 5 files changed, 155 insertions(+), 11 deletions(-)
drivers/gpu/drm/msm/msm_drv.c +93 -6
index 005ae06..3e97123 100644
@@ -509,22 +509,51 @@ static void load_gpu(struct drm_device *dev)
 	mutex_unlock(&init_lock);
 }
 
-static int msm_open(struct drm_device *dev, struct drm_file *file)
+static int context_init(struct drm_file *file)
 {
 	struct msm_file_private *ctx;
 
+	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+	if (!ctx)
+		return -ENOMEM;
+
+	INIT_LIST_HEAD(&ctx->drawqueue_list);
+	rwlock_init(&ctx->lock);
+
+	ctx->drawqueueid = 1;
+
+	file->driver_priv = ctx;
+
+	return 0;
+}
+
+static int msm_open(struct drm_device *dev, struct drm_file *file)
+{
 	/* For now, load gpu on open.. to avoid the requirement of having
 	 * firmware in the initrd.
 	 */
 	load_gpu(dev);
 
-	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+	return context_init(file);
+}
+
+static void context_close(struct msm_file_private *ctx)
+{
+	struct msm_gpu_drawqueue *entry, *tmp;
+
 	if (!ctx)
-		return -ENOMEM;
+		return;
 
-	file->driver_priv = ctx;
+	/*
+	 * A lock isn't needed here because we are in postclose and we
+	 * don't have to worry about any ioctls coming along
+	 */
+	list_for_each_entry_safe(entry, tmp, &ctx->drawqueue_list, node) {
+		list_del(&entry->node);
+		kfree(entry);
+	}
 
-	return 0;
+	kfree(ctx);
 }
 
 static void msm_postclose(struct drm_device *dev, struct drm_file *file)
@@ -537,7 +566,7 @@ static void msm_postclose(struct drm_device *dev, struct drm_file *file)
 		priv->lastctx = NULL;
 	mutex_unlock(&dev->struct_mutex);
 
-	kfree(ctx);
+	context_close(ctx);
 }
 
 static void msm_lastclose(struct drm_device *dev)
@@ -782,6 +811,62 @@ static int msm_ioctl_gem_madvise(struct drm_device *dev, void *data,
 	return ret;
 }
 
+static int msm_ioctl_drawqueue_new(struct drm_device *dev, void *data,
+		struct drm_file *file)
+{
+	struct drm_msm_drawqueue *args = data;
+	struct msm_file_private *ctx = file->driver_priv;
+	struct msm_gpu_drawqueue *drawqueue;
+
+	if (args->flags & ~MSM_DRAWQUEUE_FLAGS)
+		return -EINVAL;
+
+	drawqueue = kzalloc(sizeof(*drawqueue), GFP_KERNEL);
+	if (!drawqueue)
+		return -ENOMEM;
+
+	drawqueue->flags = args->flags;
+	drawqueue->prio = args->prio;
+
+	write_lock(&ctx->lock);
+
+	/* per-instance IDs are allocated starting at 1 */
+	drawqueue->id = ctx->drawqueueid++;
+	list_add_tail(&drawqueue->node, &ctx->drawqueue_list);
+	args->id = drawqueue->id;
+
+	write_unlock(&ctx->lock);
+
+	return 0;
+}
+
+static int msm_ioctl_drawqueue_close(struct drm_device *dev, void *data,
+		struct drm_file *file)
+{
+	struct drm_msm_drawqueue *args = data;
+	struct msm_file_private *ctx = file->driver_priv;
+	struct msm_gpu_drawqueue *entry;
+	int ret = -EINVAL;
+
+	if (!args->id)
+		return -EINVAL;
+
+	write_lock(&ctx->lock);
+
+	list_for_each_entry(entry, &ctx->drawqueue_list, node) {
+		if (entry->id == args->id) {
+			list_del(&entry->node);
+			kfree(entry);
+			ret = 0;
+			break;
+		}
+	}
+
+	write_unlock(&ctx->lock);
+
+	return ret;
+}
+
 static const struct drm_ioctl_desc msm_ioctls[] = {
 	DRM_IOCTL_DEF_DRV(MSM_GET_PARAM,    msm_ioctl_get_param,    DRM_AUTH|DRM_RENDER_ALLOW),
 	DRM_IOCTL_DEF_DRV(MSM_GEM_NEW,      msm_ioctl_gem_new,      DRM_AUTH|DRM_RENDER_ALLOW),
@@ -791,6 +876,8 @@ static int msm_ioctl_gem_madvise(struct drm_device *dev, void *data,
 	DRM_IOCTL_DEF_DRV(MSM_GEM_SUBMIT,   msm_ioctl_gem_submit,   DRM_AUTH|DRM_RENDER_ALLOW),
 	DRM_IOCTL_DEF_DRV(MSM_WAIT_FENCE,   msm_ioctl_wait_fence,   DRM_AUTH|DRM_RENDER_ALLOW),
 	DRM_IOCTL_DEF_DRV(MSM_GEM_MADVISE,  msm_ioctl_gem_madvise,  DRM_AUTH|DRM_RENDER_ALLOW),
+	DRM_IOCTL_DEF_DRV(MSM_DRAWQUEUE_NEW,   msm_ioctl_drawqueue_new,   DRM_AUTH|DRM_RENDER_ALLOW),
+	DRM_IOCTL_DEF_DRV(MSM_DRAWQUEUE_CLOSE, msm_ioctl_drawqueue_close, DRM_AUTH|DRM_RENDER_ALLOW),
 };
 
 static const struct vm_operations_struct vm_ops = {
drivers/gpu/drm/msm/msm_drv.h +3 -5
index 69e839c..192147c 100644
@@ -58,11 +58,9 @@
 #define NUM_DOMAINS 2    /* one for KMS, then one per gpu core (?) */
 
 struct msm_file_private {
-	/* currently we don't do anything useful with this.. but when
-	 * per-context address spaces are supported we'd keep track of
-	 * the context's page-tables here.
-	 */
-	int dummy;
+	rwlock_t lock;
+	struct list_head drawqueue_list;
+	int drawqueueid;
 };
 
 enum msm_mdp_plane_property {
drivers/gpu/drm/msm/msm_gem_submit.c +32 -0
index 4c2525e..0129ca2 100644
@@ -378,6 +378,33 @@ static void submit_cleanup(struct msm_gem_submit *submit)
 	ww_acquire_fini(&submit->ticket);
 }
 
+static int drawqueue_get_info(struct msm_file_private *ctx,
+		u32 id, u32 *prio)
+{
+	struct msm_gpu_drawqueue *entry;
+	int ret = -EINVAL;
+
+	if (!id) {
+		*prio = 0;
+		return 0;
+	}
+
+	read_lock(&ctx->lock);
+
+	list_for_each_entry(entry, &ctx->drawqueue_list, node) {
+		if (entry->id != id)
+			continue;
+
+		*prio = entry->prio;
+
+		ret = 0;
+		break;
+	}
+
+	read_unlock(&ctx->lock);
+	return ret;
+}
+
 int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
 		struct drm_file *file)
 {
@@ -390,6 +417,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
 	struct sync_file *sync_file = NULL;
 	int out_fence_fd = -1;
 	unsigned i;
+	u32 prio = 0;
 	int ret;
 
 	if (!gpu)
@@ -404,6 +432,10 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
 	if (MSM_PIPE_FLAGS(args->flags) & ~MSM_SUBMIT_FLAGS)
 		return -EINVAL;
 
+	ret = drawqueue_get_info(ctx, args->drawid, &prio);
+	if (ret)
+		return ret;
+
 	if (args->flags & MSM_SUBMIT_FENCE_FD_IN) {
 		in_fence = sync_file_get_fence(args->fence_fd);
 
drivers/gpu/drm/msm/msm_gpu.h +7 -0
index df4e277..ca07a21 100644
@@ -132,6 +132,13 @@ struct msm_gpu {
 	struct list_head submit_list;
 };
 
+struct msm_gpu_drawqueue {
+	int id;
+	u32 flags;
+	u32 prio;
+	struct list_head node;
+};
+
 static inline bool msm_gpu_active(struct msm_gpu *gpu)
 {
 	return gpu->fctx->last_fence > gpu->funcs->last_fence(gpu);
include/uapi/drm/msm_drm.h +20 -0
index 26c54f6..cad0133 100644
@@ -218,6 +218,7 @@ struct drm_msm_gem_submit {
 	__u64 __user bos;     /* in, ptr to array of submit_bo's */
 	__u64 __user cmds;    /* in, ptr to array of submit_cmd's */
 	__s32 fence_fd;       /* in/out fence fd (see MSM_SUBMIT_FENCE_FD_IN/OUT) */
+	__u32 drawid;         /* in, drawqueue id */
 };
 
 /* The normal way to synchronize with the GPU is just to CPU_PREP on
@@ -254,6 +255,21 @@ struct drm_msm_gem_madvise {
 	__u32 retained;       /* out, whether backing store still exists */
 };
 
+/*
+ * Draw queues allow the user to set specific submission parameter. Command
+ * submissions specify a specific drawqueue to use.  ID 0 is reserved for
+ * backwards compatibility as a "default" drawqueue with medium priority and no
+ * special flags
+ */
+
+#define MSM_DRAWQUEUE_FLAGS (0)
+
+struct drm_msm_drawqueue {
+	__u32 flags;   /* in, MSM_DRAWQUEUE_x */
+	__u32 prio;    /* in, Priority level */
+	__u32 id;      /* out, identifier */
+};
+
 #define DRM_MSM_GET_PARAM              0x00
 /* placeholder:
 #define DRM_MSM_SET_PARAM              0x01
@@ -265,6 +281,8 @@ struct drm_msm_gem_madvise {
 #define DRM_MSM_GEM_SUBMIT             0x06
 #define DRM_MSM_WAIT_FENCE             0x07
 #define DRM_MSM_GEM_MADVISE            0x08
+#define DRM_MSM_DRAWQUEUE_NEW          0x09
+#define DRM_MSM_DRAWQUEUE_CLOSE        0x0A
 
 #define DRM_IOCTL_MSM_GET_PARAM        DRM_IOWR(DRM_COMMAND_BASE + DRM_MSM_GET_PARAM, struct drm_msm_param)
 #define DRM_IOCTL_MSM_GEM_NEW          DRM_IOWR(DRM_COMMAND_BASE + DRM_MSM_GEM_NEW, struct drm_msm_gem_new)
@@ -274,6 +292,8 @@ struct drm_msm_gem_madvise {
 #define DRM_IOCTL_MSM_GEM_SUBMIT       DRM_IOWR(DRM_COMMAND_BASE + DRM_MSM_GEM_SUBMIT, struct drm_msm_gem_submit)
 #define DRM_IOCTL_MSM_WAIT_FENCE       DRM_IOW (DRM_COMMAND_BASE + DRM_MSM_WAIT_FENCE, struct drm_msm_wait_fence)
 #define DRM_IOCTL_MSM_GEM_MADVISE      DRM_IOWR(DRM_COMMAND_BASE + DRM_MSM_GEM_MADVISE, struct drm_msm_gem_madvise)
+#define DRM_IOCTL_MSM_DRAWQUEUE_NEW    DRM_IOWR(DRM_COMMAND_BASE + DRM_MSM_DRAWQUEUE_NEW, struct drm_msm_drawqueue)
+#define DRM_IOCTL_MSM_DRAWQUEUE_CLOSE  DRM_IOW (DRM_COMMAND_BASE + DRM_MSM_DRAWQUEUE_CLOSE, struct drm_msm_drawqueue)
 
 #if defined(__cplusplus)
 }
Signature · 1 line
1.9.1
#11 lore raw

PATCH 10/13 drm/msm: Support multiple ringbuffers

In reply to Jordan Crouse

Add the infrastructure to support the idea of multiple ringbuffers. Assign each ringbuffer an id and use that as an index for the various ring specific operations.

The biggest delta is to support legacy fences. Each fence gets its own sequence number but the legacy functions expect to use a unique integer. To handle this we return a unique identifer for each submission but map it to a specific ring/sequence under the covers. Newer users use a dma_fence pointer anyway so they don't care about the actual sequence ID or ring.

The actual mechanics for multiple ringbuffers are very target specific so this code just allows for the possibility but still only defines one ringbuffer for each target family.

---
 drivers/gpu/drm/msm/adreno/a3xx_gpu.c   |   9 +-
 drivers/gpu/drm/msm/adreno/a4xx_gpu.c   |   9 +-
 drivers/gpu/drm/msm/adreno/a5xx_gpu.c   |  45 +++++-----
 drivers/gpu/drm/msm/adreno/a5xx_gpu.h   |   2 +-
 drivers/gpu/drm/msm/adreno/a5xx_power.c |   6 +-
 drivers/gpu/drm/msm/adreno/adreno_gpu.c | 149 ++++++++++++++++++++------------
 drivers/gpu/drm/msm/adreno/adreno_gpu.h |  36 +++++---
 drivers/gpu/drm/msm/msm_drv.h           |   2 +
 drivers/gpu/drm/msm/msm_fence.c         |  92 +++++++++++++++-----
 drivers/gpu/drm/msm/msm_fence.h         |  15 ++--
 drivers/gpu/drm/msm/msm_gem.h           |   3 +-
 drivers/gpu/drm/msm/msm_gem_submit.c    |  10 ++-
 drivers/gpu/drm/msm/msm_gpu.c           | 138 ++++++++++++++++++++---------
 drivers/gpu/drm/msm/msm_gpu.h           |  30 ++++---
 drivers/gpu/drm/msm/msm_ringbuffer.c    |  19 ++--
 drivers/gpu/drm/msm/msm_ringbuffer.h    |   9 +-
 16 files changed, 381 insertions(+), 193 deletions(-)
drivers/gpu/drm/msm/adreno/a3xx_gpu.c +5 -4
index 0e3828ed..10d0234 100644
@@ -44,7 +44,7 @@
 
 static bool a3xx_me_init(struct msm_gpu *gpu)
 {
-	struct msm_ringbuffer *ring = gpu->rb;
+	struct msm_ringbuffer *ring = gpu->rb[0];
 
 	OUT_PKT3(ring, CP_ME_INIT, 17);
 	OUT_RING(ring, 0x000003f7);
@@ -65,7 +65,7 @@ static bool a3xx_me_init(struct msm_gpu *gpu)
 	OUT_RING(ring, 0x00000000);
 	OUT_RING(ring, 0x00000000);
 
-	gpu->funcs->flush(gpu);
+	gpu->funcs->flush(gpu, ring);
 	return a3xx_idle(gpu);
 }
 
@@ -339,7 +339,7 @@ static void a3xx_destroy(struct msm_gpu *gpu)
 static bool a3xx_idle(struct msm_gpu *gpu)
 {
 	/* wait for ringbuffer to drain: */
-	if (!adreno_idle(gpu))
+	if (!adreno_idle(gpu, gpu->rb[0]))
 		return false;
 
 	/* then wait for GPU to finish: */
@@ -447,6 +447,7 @@ static void a3xx_dump(struct msm_gpu *gpu)
 		.last_fence = adreno_last_fence,
 		.submit = adreno_submit,
 		.flush = adreno_flush,
+		.active_ring = adreno_active_ring,
 		.irq = a3xx_irq,
 		.destroy = a3xx_destroy,
 #ifdef CONFIG_DEBUG_FS
@@ -494,7 +495,7 @@ struct msm_gpu *a3xx_gpu_init(struct drm_device *dev)
 	adreno_gpu->registers = a3xx_registers;
 	adreno_gpu->reg_offsets = a3xx_register_offsets;
 
-	ret = adreno_gpu_init(dev, pdev, adreno_gpu, &funcs);
+	ret = adreno_gpu_init(dev, pdev, adreno_gpu, &funcs, 1);
 	if (ret)
 		goto fail;
 
drivers/gpu/drm/msm/adreno/a4xx_gpu.c +5 -4
index 19abf22..35fbf18 100644
@@ -116,7 +116,7 @@ static void a4xx_enable_hwcg(struct msm_gpu *gpu)
 
 static bool a4xx_me_init(struct msm_gpu *gpu)
 {
-	struct msm_ringbuffer *ring = gpu->rb;
+	struct msm_ringbuffer *ring = gpu->rb[0];
 
 	OUT_PKT3(ring, CP_ME_INIT, 17);
 	OUT_RING(ring, 0x000003f7);
@@ -137,7 +137,7 @@ static bool a4xx_me_init(struct msm_gpu *gpu)
 	OUT_RING(ring, 0x00000000);
 	OUT_RING(ring, 0x00000000);
 
-	gpu->funcs->flush(gpu);
+	gpu->funcs->flush(gpu, ring);
 	return a4xx_idle(gpu);
 }
 
@@ -337,7 +337,7 @@ static void a4xx_destroy(struct msm_gpu *gpu)
 static bool a4xx_idle(struct msm_gpu *gpu)
 {
 	/* wait for ringbuffer to drain: */
-	if (!adreno_idle(gpu))
+	if (!adreno_idle(gpu, gpu->rb[0]))
 		return false;
 
 	/* then wait for GPU to finish: */
@@ -535,6 +535,7 @@ static int a4xx_get_timestamp(struct msm_gpu *gpu, uint64_t *value)
 		.last_fence = adreno_last_fence,
 		.submit = adreno_submit,
 		.flush = adreno_flush,
+		.active_ring = adreno_active_ring,
 		.irq = a4xx_irq,
 		.destroy = a4xx_destroy,
 #ifdef CONFIG_DEBUG_FS
@@ -576,7 +577,7 @@ struct msm_gpu *a4xx_gpu_init(struct drm_device *dev)
 	adreno_gpu->registers = a4xx_registers;
 	adreno_gpu->reg_offsets = a4xx_register_offsets;
 
-	ret = adreno_gpu_init(dev, pdev, adreno_gpu, &funcs);
+	ret = adreno_gpu_init(dev, pdev, adreno_gpu, &funcs, 1);
 	if (ret)
 		goto fail;
 
drivers/gpu/drm/msm/adreno/a5xx_gpu.c +24 -21

109 lines not on this page · raw message

drivers/gpu/drm/msm/adreno/a5xx_gpu.h +1 -1
index 6638bc8..aba6faf 100644
@@ -58,6 +58,6 @@ static inline int spin_usecs(struct msm_gpu *gpu, uint32_t usecs,
 	return -ETIMEDOUT;
 }
 
-bool a5xx_idle(struct msm_gpu *gpu);
+bool a5xx_idle(struct msm_gpu *gpu, struct msm_ringbuffer *ring);
 
 #endif /* __A5XX_GPU_H__ */
drivers/gpu/drm/msm/adreno/a5xx_power.c +3 -3
index 2fdee44..a7d91ac 100644
@@ -173,7 +173,7 @@ static int a5xx_gpmu_init(struct msm_gpu *gpu)
 {
 	struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
 	struct a5xx_gpu *a5xx_gpu = to_a5xx_gpu(adreno_gpu);
-	struct msm_ringbuffer *ring = gpu->rb;
+	struct msm_ringbuffer *ring = gpu->rb[0];
 
 	if (!a5xx_gpu->gpmu_dwords)
 		return 0;
@@ -192,9 +192,9 @@ static int a5xx_gpmu_init(struct msm_gpu *gpu)
 	OUT_PKT7(ring, CP_SET_PROTECTED_MODE, 1);
 	OUT_RING(ring, 1);
 
-	gpu->funcs->flush(gpu);
+	gpu->funcs->flush(gpu, ring);
 
-	if (!a5xx_idle(gpu)) {
+	if (!a5xx_idle(gpu, ring)) {
 		DRM_ERROR("%s: Unable to load GPMU firmware. GPMU will not be active\n",
 			gpu->name);
 		return -EINVAL;
drivers/gpu/drm/msm/adreno/adreno_gpu.c +95 -54

267 lines not on this page · raw message

drivers/gpu/drm/msm/adreno/adreno_gpu.h +23 -13
index 4d9165f..9e78e49 100644
@@ -82,12 +82,18 @@ struct adreno_info {
 
 const struct adreno_info *adreno_info(struct adreno_rev rev);
 
-#define rbmemptr(adreno_gpu, member)  \
+#define _sizeof(member) \
+	sizeof(((struct adreno_rbmemptrs *) 0)->member[0])
+
+#define _base(adreno_gpu, member)  \
 	((adreno_gpu)->memptrs_iova + offsetof(struct adreno_rbmemptrs, member))
 
+#define rbmemptr(adreno_gpu, index, member) \
+	(_base((adreno_gpu), member) + ((index) * _sizeof(member)))
+
 struct adreno_rbmemptrs {
-	volatile uint32_t rptr;
-	volatile uint32_t fence;
+	volatile uint32_t rptr[MSM_GPU_MAX_RINGS];
+	volatile uint32_t fence[MSM_GPU_MAX_RINGS];
 };
 
 struct adreno_gpu {
@@ -197,21 +203,25 @@ static inline int adreno_is_a530(struct adreno_gpu *gpu)
 
 int adreno_get_param(struct msm_gpu *gpu, uint32_t param, uint64_t *value);
 int adreno_hw_init(struct msm_gpu *gpu);
-uint32_t adreno_last_fence(struct msm_gpu *gpu);
+uint32_t adreno_last_fence(struct msm_gpu *gpu, struct msm_ringbuffer *ring);
+uint32_t adreno_submitted_fence(struct msm_gpu *gpu,
+		struct msm_ringbuffer *ring);
 void adreno_recover(struct msm_gpu *gpu);
 void adreno_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit,
 		struct msm_file_private *ctx);
-void adreno_flush(struct msm_gpu *gpu);
-bool adreno_idle(struct msm_gpu *gpu);
+void adreno_flush(struct msm_gpu *gpu, struct msm_ringbuffer *ring);
+bool adreno_idle(struct msm_gpu *gpu, struct msm_ringbuffer *ring);
 #ifdef CONFIG_DEBUG_FS
 void adreno_show(struct msm_gpu *gpu, struct seq_file *m);
 #endif
 void adreno_dump_info(struct msm_gpu *gpu);
 void adreno_dump(struct msm_gpu *gpu);
-void adreno_wait_ring(struct msm_gpu *gpu, uint32_t ndwords);
+void adreno_wait_ring(struct msm_ringbuffer *ring, uint32_t ndwords);
+struct msm_ringbuffer *adreno_active_ring(struct msm_gpu *gpu);
 
 int adreno_gpu_init(struct drm_device *drm, struct platform_device *pdev,
-		struct adreno_gpu *gpu, const struct adreno_gpu_funcs *funcs);
+		struct adreno_gpu *gpu, const struct adreno_gpu_funcs *funcs,
+		int nr_rings);
 void adreno_gpu_cleanup(struct adreno_gpu *gpu);
 
 
@@ -220,7 +230,7 @@ int adreno_gpu_init(struct drm_device *drm, struct platform_device *pdev,
 static inline void
 OUT_PKT0(struct msm_ringbuffer *ring, uint16_t regindx, uint16_t cnt)
 {
-	adreno_wait_ring(ring->gpu, cnt+1);
+	adreno_wait_ring(ring, cnt+1);
 	OUT_RING(ring, CP_TYPE0_PKT | ((cnt-1) << 16) | (regindx & 0x7FFF));
 }
 
@@ -228,14 +238,14 @@ int adreno_gpu_init(struct drm_device *drm, struct platform_device *pdev,
 static inline void
 OUT_PKT2(struct msm_ringbuffer *ring)
 {
-	adreno_wait_ring(ring->gpu, 1);
+	adreno_wait_ring(ring, 1);
 	OUT_RING(ring, CP_TYPE2_PKT);
 }
 
 static inline void
 OUT_PKT3(struct msm_ringbuffer *ring, uint8_t opcode, uint16_t cnt)
 {
-	adreno_wait_ring(ring->gpu, cnt+1);
+	adreno_wait_ring(ring, cnt+1);
 	OUT_RING(ring, CP_TYPE3_PKT | ((cnt-1) << 16) | ((opcode & 0xFF) << 8));
 }
 
@@ -257,14 +267,14 @@ static inline u32 PM4_PARITY(u32 val)
 static inline void
 OUT_PKT4(struct msm_ringbuffer *ring, uint16_t regindx, uint16_t cnt)
 {
-	adreno_wait_ring(ring->gpu, cnt + 1);
+	adreno_wait_ring(ring, cnt + 1);
 	OUT_RING(ring, PKT4(regindx, cnt));
 }
 
 static inline void
 OUT_PKT7(struct msm_ringbuffer *ring, uint8_t opcode, uint16_t cnt)
 {
-	adreno_wait_ring(ring->gpu, cnt + 1);
+	adreno_wait_ring(ring, cnt + 1);
 	OUT_RING(ring, CP_TYPE7_PKT | (cnt << 0) | (PM4_PARITY(cnt) << 15) |
 		((opcode & 0x7F) << 16) | (PM4_PARITY(opcode) << 23));
 }
drivers/gpu/drm/msm/msm_drv.h +2 -0
index 192147c..bbf7d3d 100644
@@ -76,6 +76,8 @@ struct msm_vblank_ctrl {
 	spinlock_t lock;
 };
 
+#define MSM_GPU_MAX_RINGS 1
+
 struct msm_drm_private {
 
 	struct drm_device *dev;
drivers/gpu/drm/msm/msm_fence.c +71 -21

180 lines not on this page · raw message

drivers/gpu/drm/msm/msm_fence.h +10 -5
index 56061aa..b5c6830 100644
@@ -18,17 +18,18 @@
 #ifndef __MSM_FENCE_H__
 #define __MSM_FENCE_H__
 
+#include <linux/hashtable.h>
 #include "msm_drv.h"
+#include "msm_ringbuffer.h"
 
 struct msm_fence_context {
 	struct drm_device *dev;
 	const char *name;
 	unsigned context;
-	/* last_fence == completed_fence --> no pending work */
-	uint32_t last_fence;          /* last assigned fence */
-	uint32_t completed_fence;     /* last completed fence */
+	u32 fence_id;
 	wait_queue_head_t event;
 	spinlock_t spinlock;
+	DECLARE_HASHTABLE(hash, 4);
 };
 
 struct msm_fence_context * msm_fence_context_alloc(struct drm_device *dev,
@@ -39,8 +40,12 @@ int msm_wait_fence(struct msm_fence_context *fctx, uint32_t fence,
 		ktime_t *timeout, bool interruptible);
 int msm_queue_fence_cb(struct msm_fence_context *fctx,
 		struct msm_fence_cb *cb, uint32_t fence);
-void msm_update_fence(struct msm_fence_context *fctx, uint32_t fence);
+void msm_update_fence(struct msm_fence_context *fctx,
+		struct msm_ringbuffer *ring, uint32_t fence);
 
-struct dma_fence * msm_fence_alloc(struct msm_fence_context *fctx);
+struct dma_fence *msm_fence_alloc(struct msm_fence_context *fctx,
+		struct msm_ringbuffer *ring);
+
+uint32_t msm_fence_id(struct dma_fence *fence);
 
 #endif
drivers/gpu/drm/msm/msm_gem.h +2 -1
index 2767014..ddae0a9 100644
@@ -116,12 +116,13 @@ static inline bool is_vunmapable(struct msm_gem_object *msm_obj)
 struct msm_gem_submit {
 	struct drm_device *dev;
 	struct msm_gpu *gpu;
-	struct list_head node;   /* node in gpu submit_list */
+	struct list_head node;   /* node in ring submit list */
 	struct list_head bo_list;
 	struct ww_acquire_ctx ticket;
 	struct dma_fence *fence;
 	struct pid *pid;    /* submitting process */
 	bool valid;         /* true if no cmdstream patching needed */
+	struct msm_ringbuffer *ring;
 	unsigned int nr_cmds;
 	unsigned int nr_bos;
 	struct {
drivers/gpu/drm/msm/msm_gem_submit.c +7 -3
index 0129ca2..4f483c0 100644
@@ -418,7 +418,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
 	int out_fence_fd = -1;
 	unsigned i;
 	u32 prio = 0;
-	int ret;
+	int ret, ring;
 
 	if (!gpu)
 		return -ENXIO;
@@ -552,7 +552,11 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
 
 	submit->nr_cmds = i;
 
-	submit->fence = msm_fence_alloc(gpu->fctx);
+	ring = clamp_t(uint32_t, prio, 0, gpu->nr_rings - 1);
+
+	submit->ring = gpu->rb[ring];
+
+	submit->fence = msm_fence_alloc(gpu->fctx, submit->ring);
 	if (IS_ERR(submit->fence)) {
 		ret = PTR_ERR(submit->fence);
 		submit->fence = NULL;
@@ -569,7 +573,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
 
 	msm_gpu_submit(gpu, submit, ctx);
 
-	args->fence = submit->fence->seqno;
+	args->fence = msm_fence_id(submit->fence);
 
 	if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
 		fd_install(out_fence_fd, sync_file->file);
drivers/gpu/drm/msm/msm_gpu.c +95 -43

239 lines not on this page · raw message

drivers/gpu/drm/msm/msm_gpu.h +20 -10
index ca07a21..c0e7c84 100644
@@ -33,7 +33,7 @@ struct msm_gpu_config {
 	const char *irqname;
 	uint64_t va_start;
 	uint64_t va_end;
-	unsigned int ringsz;
+	unsigned int nr_rings;
 };
 
 /* So far, with hardware that I've seen to date, we can have:
@@ -57,9 +57,11 @@ struct msm_gpu_funcs {
 	int (*pm_resume)(struct msm_gpu *gpu);
 	void (*submit)(struct msm_gpu *gpu, struct msm_gem_submit *submit,
 			struct msm_file_private *ctx);
-	void (*flush)(struct msm_gpu *gpu);
+	void (*flush)(struct msm_gpu *gpu, struct msm_ringbuffer *ring);
 	irqreturn_t (*irq)(struct msm_gpu *irq);
-	uint32_t (*last_fence)(struct msm_gpu *gpu);
+	uint32_t (*last_fence)(struct msm_gpu *gpu,
+			struct msm_ringbuffer *ring);
+	struct msm_ringbuffer *(*active_ring)(struct msm_gpu *gpu);
 	void (*recover)(struct msm_gpu *gpu);
 	void (*destroy)(struct msm_gpu *gpu);
 #ifdef CONFIG_DEBUG_FS
@@ -86,9 +88,8 @@ struct msm_gpu {
 	const struct msm_gpu_perfcntr *perfcntrs;
 	uint32_t num_perfcntrs;
 
-	/* ringbuffer: */
-	struct msm_ringbuffer *rb;
-	uint64_t rb_iova;
+	struct msm_ringbuffer *rb[MSM_GPU_MAX_RINGS];
+	int nr_rings;
 
 	/* list of GEM active objects: */
 	struct list_head active_list;
@@ -126,10 +127,8 @@ struct msm_gpu {
 #define DRM_MSM_HANGCHECK_PERIOD 500 /* in ms */
 #define DRM_MSM_HANGCHECK_JIFFIES msecs_to_jiffies(DRM_MSM_HANGCHECK_PERIOD)
 	struct timer_list hangcheck_timer;
-	uint32_t hangcheck_fence;
+	uint32_t hangcheck_fence[MSM_GPU_MAX_RINGS];
 	struct work_struct recover_work;
-
-	struct list_head submit_list;
 };
 
 struct msm_gpu_drawqueue {
@@ -139,9 +138,20 @@ struct msm_gpu_drawqueue {
 	struct list_head node;
 };
 
+/* It turns out that all targets use the same ringbuffer size */
+#define MSM_GPU_RINGBUFFER_SZ SZ_32K
+
 static inline bool msm_gpu_active(struct msm_gpu *gpu)
 {
-	return gpu->fctx->last_fence > gpu->funcs->last_fence(gpu);
+	int i;
+
+	for (i = 0; i < gpu->nr_rings; i++) {
+		if (gpu->rb[i]->last_fence >
+			gpu->funcs->last_fence(gpu, gpu->rb[i]))
+			return true;
+	}
+
+	return false;
 }
 
 /* Perf-Counters:
drivers/gpu/drm/msm/msm_ringbuffer.c +11 -8
index 67b34e0..10f1d948 100644
@@ -18,13 +18,13 @@
 #include "msm_ringbuffer.h"
 #include "msm_gpu.h"
 
-struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int size)
+struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int id)
 {
 	struct msm_ringbuffer *ring;
 	int ret;
 
-	if (WARN_ON(!is_power_of_2(size)))
-		return ERR_PTR(-EINVAL);
+	/* We assume everwhere that MSM_GPU_RINGBUFFER_SZ is a power of 2 */
+	BUILD_BUG_ON(!is_power_of_2(MSM_GPU_RINGBUFFER_SZ));
 
 	ring = kzalloc(sizeof(*ring), GFP_KERNEL);
 	if (!ring) {
@@ -33,7 +33,8 @@ struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int size)
 	}
 
 	ring->gpu = gpu;
-	ring->bo = msm_gem_new(gpu->dev, size, MSM_BO_WC);
+	ring->id = id;
+	ring->bo = msm_gem_new(gpu->dev, MSM_GPU_RINGBUFFER_SZ, MSM_BO_WC);
 	if (IS_ERR(ring->bo)) {
 		ret = PTR_ERR(ring->bo);
 		ring->bo = NULL;
@@ -45,21 +46,23 @@ struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int size)
 		ret = PTR_ERR(ring->start);
 		goto fail;
 	}
-	ring->end   = ring->start + (size / 4);
+	ring->end   = ring->start + (MSM_GPU_RINGBUFFER_SZ >> 2);
 	ring->cur   = ring->start;
 
-	ring->size = size;
+	INIT_LIST_HEAD(&ring->submits);
 
 	return ring;
 
 fail:
-	if (ring)
-		msm_ringbuffer_destroy(ring);
+	msm_ringbuffer_destroy(ring);
 	return ERR_PTR(ret);
 }
 
 void msm_ringbuffer_destroy(struct msm_ringbuffer *ring)
 {
+	if (IS_ERR_OR_NULL(ring))
+		return;
+
 	if (ring->bo) {
 		msm_gem_put_vaddr(ring->bo);
 		drm_gem_object_unreference_unlocked(ring->bo);
drivers/gpu/drm/msm/msm_ringbuffer.h +7 -2
index 6e0e104..c803364 100644
@@ -22,12 +22,17 @@
 
 struct msm_ringbuffer {
 	struct msm_gpu *gpu;
-	int size;
+	int id;
 	struct drm_gem_object *bo;
 	uint32_t *start, *end, *cur;
+	uint64_t iova;
+	/* last_fence == completed_fence --> no pending work */
+	uint32_t last_fence;
+	uint32_t completed_fence;
+	struct list_head submits;
 };
 
-struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int size);
+struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int id);
 void msm_ringbuffer_destroy(struct msm_ringbuffer *ring);
 
 /* ringbuffer helpers (the parts that are same for a3xx/a2xx/z180..) */
Signature · 1 line
1.9.1
#12 lore raw

Re: [PATCH 10/13] drm/msm: Support multiple ringbuffers

Quoted 6 lines · Jordan Crouse
drivers/gpu/drm/msm/adreno/adreno_gpu.h +4 -1
-#define rbmemptr(adreno_gpu, member)  \
+#define _sizeof(member) \
+	sizeof(((struct adreno_rbmemptrs *) 0)->member[0])
+
+#define _base(adreno_gpu, member)  \
 	((adreno_gpu)->memptrs_iova + offsetof(struct adreno_rbmemptrs, member))

drivers/gpu/drm/msm/adreno/adreno_gpu.h struct adreno_info { · line 97

Quoted 10 lines
drivers/gpu/drm/msm/adreno/adreno_gpu.h +5 -2
 
+#define rbmemptr(adreno_gpu, index, member) \
+	(_base((adreno_gpu), member) + ((index) * _sizeof(member)))
+
 struct adreno_rbmemptrs {
-	volatile uint32_t rptr;
-	volatile uint32_t fence;
+	volatile uint32_t rptr[MSM_GPU_MAX_RINGS];
+	volatile uint32_t fence[MSM_GPU_MAX_RINGS];
 };

I'm looking for opinions on moving this to a per-ring buffer object. It would be a lot simpler to understand but it would cost us a page per ring as opposed to the 1 page we use now.

Looking ahead we are going to want to start using trace messages in conjunction with tools like systrace:

(https://developer.android.com/studio/profile/systrace-commandline.html)

This will involve tracking the always on counter value at start/retire for each outstanding submit on each ring. I *think* we could fit those values into the existing rmemptrs buffer if we wanted to but I can't imagine these would be the last runtime statistics we would gather.

I guess I'm leaning toward the per-ring solution but I'll listen to anybody argue that the memory usage isn't worth it.

Jordan

Signature · 6 lines
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#13 lore raw

Re: [Freedreno] [PATCH 10/13] drm/msm: Support multiple ringbuffers

Quoted 8 lines · Jordan Crouse
Quoted 7 lines · Jordan Crouse
drivers/gpu/drm/msm/adreno/adreno_gpu.h +4 -1
-#define rbmemptr(adreno_gpu, member)  \
+#define _sizeof(member) \
+     sizeof(((struct adreno_rbmemptrs *) 0)->member[0])
+
+#define _base(adreno_gpu, member)  \
      ((adreno_gpu)->memptrs_iova + offsetof(struct adreno_rbmemptrs, member))
Quoted 13 lines

drivers/gpu/drm/msm/adreno/adreno_gpu.h struct adreno_info { · line 97

Quoted 9 lines
drivers/gpu/drm/msm/adreno/adreno_gpu.h +5 -2
+#define rbmemptr(adreno_gpu, index, member) \
+     (_base((adreno_gpu), member) + ((index) * _sizeof(member)))
+
 struct adreno_rbmemptrs {
-     volatile uint32_t rptr;
-     volatile uint32_t fence;
+     volatile uint32_t rptr[MSM_GPU_MAX_RINGS];
+     volatile uint32_t fence[MSM_GPU_MAX_RINGS];
 };

I'm looking for opinions on moving this to a per-ring buffer object. It would be a lot simpler to understand but it would cost us a page per ring as opposed to the 1 page we use now.

Well, I guess sub-allocation is an option.. we don't *have* to do a page-per memptrs struct just to have separate struct per pring.

Just turn the single rbmemptrs allocation into an memptrs[MAX_RINGS]..

BR,
-R

Quoted 22 lines

Looking ahead we are going to want to start using trace messages in conjunction with tools like systrace:

(https://developer.android.com/studio/profile/systrace-commandline.html)

This will involve tracking the always on counter value at start/retire for each outstanding submit on each ring. I *think* we could fit those values into the existing rmemptrs buffer if we wanted to but I can't imagine these would be the last runtime statistics we would gather.

I guess I'm leaning toward the per-ring solution but I'll listen to anybody argue that the memory usage isn't worth it.

Jordan

--
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
_______________________________________________
Freedreno mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/freedreno

A message that is not in the archive (sent off-list or not yet received): 1494275709-25782-11-git-send-email-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org

#15 lore raw

Re: [PATCH 10/13] drm/msm: Support multiple ringbuffers

Quoted 10 lines · Jordan Crouse

Add the infrastructure to support the idea of multiple ringbuffers. Assign each ringbuffer an id and use that as an index for the various ring specific operations.

The biggest delta is to support legacy fences. Each fence gets its own sequence number but the legacy functions expect to use a unique integer. To handle this we return a unique identifer for each submission but map it to a specific ring/sequence under the covers. Newer users use a dma_fence pointer anyway so they don't care about the actual sequence ID or ring.

So, WAIT_FENCE is alive and well, and useful since it avoids the overhead of creating a 'struct file', but it is only used within a single pipe_context (or at least situations where we know which ctx the seqno fence applies to). It seems like it would be simpler if we just introduced a ctx-id in all the ioctls (SUBMIT and WAIT_FENCE) that take a uint fence. Then I think we don't need hashtable fancyness.

Also, one thing I was thinking of is that some-day we might want to make SUBMIT non-blocking when there is a dependency on a fence from a different ring. (Ie. queue it up but don't write cmds into rb yet.) Which means we'd need multiple fence timelines per priority-level rb. Which brings me back to wanting a CREATE_CTX type of ioctl. (And I guess DESTROY_CTX.) We could make these simple stubs for now, ie. CREATE_CTX just returns the priority level back, and not really have any separate "context" object on the kernel side for now. This wouldn't change the implementation much from what you have, but I think that gives us some flexibility to later on actually let us have multiple contexts at a given priority level which don't block each other for submits that are still pending on some fence, without another UABI change.

BR,
-R

Quoted 1,373 lines

The actual mechanics for multiple ringbuffers are very target specific so this code just allows for the possibility but still only defines one ringbuffer for each target family.

---
 drivers/gpu/drm/msm/adreno/a3xx_gpu.c   |   9 +-
 drivers/gpu/drm/msm/adreno/a4xx_gpu.c   |   9 +-
 drivers/gpu/drm/msm/adreno/a5xx_gpu.c   |  45 +++++-----
 drivers/gpu/drm/msm/adreno/a5xx_gpu.h   |   2 +-
 drivers/gpu/drm/msm/adreno/a5xx_power.c |   6 +-
 drivers/gpu/drm/msm/adreno/adreno_gpu.c | 149 ++++++++++++++++++++------------
 drivers/gpu/drm/msm/adreno/adreno_gpu.h |  36 +++++---
 drivers/gpu/drm/msm/msm_drv.h           |   2 +
 drivers/gpu/drm/msm/msm_fence.c         |  92 +++++++++++++++-----
 drivers/gpu/drm/msm/msm_fence.h         |  15 ++--
 drivers/gpu/drm/msm/msm_gem.h           |   3 +-
 drivers/gpu/drm/msm/msm_gem_submit.c    |  10 ++-
 drivers/gpu/drm/msm/msm_gpu.c           | 138 ++++++++++++++++++++---------
 drivers/gpu/drm/msm/msm_gpu.h           |  30 ++++---
 drivers/gpu/drm/msm/msm_ringbuffer.c    |  19 ++--
 drivers/gpu/drm/msm/msm_ringbuffer.h    |   9 +-
 16 files changed, 381 insertions(+), 193 deletions(-)
drivers/gpu/drm/msm/adreno/a3xx_gpu.c +5 -4
index 0e3828ed..10d0234 100644
@@ -44,7 +44,7 @@
 
 static bool a3xx_me_init(struct msm_gpu *gpu)
 {
-       struct msm_ringbuffer *ring = gpu->rb;
+       struct msm_ringbuffer *ring = gpu->rb[0];
 
        OUT_PKT3(ring, CP_ME_INIT, 17);
        OUT_RING(ring, 0x000003f7);
@@ -65,7 +65,7 @@ static bool a3xx_me_init(struct msm_gpu *gpu)
        OUT_RING(ring, 0x00000000);
        OUT_RING(ring, 0x00000000);
 
-       gpu->funcs->flush(gpu);
+       gpu->funcs->flush(gpu, ring);
        return a3xx_idle(gpu);
 }
 
@@ -339,7 +339,7 @@ static void a3xx_destroy(struct msm_gpu *gpu)
 static bool a3xx_idle(struct msm_gpu *gpu)
 {
        /* wait for ringbuffer to drain: */
-       if (!adreno_idle(gpu))
+       if (!adreno_idle(gpu, gpu->rb[0]))
                return false;
 
        /* then wait for GPU to finish: */
@@ -447,6 +447,7 @@ static void a3xx_dump(struct msm_gpu *gpu)
                .last_fence = adreno_last_fence,
                .submit = adreno_submit,
                .flush = adreno_flush,
+               .active_ring = adreno_active_ring,
                .irq = a3xx_irq,
                .destroy = a3xx_destroy,
 #ifdef CONFIG_DEBUG_FS
@@ -494,7 +495,7 @@ struct msm_gpu *a3xx_gpu_init(struct drm_device *dev)
        adreno_gpu->registers = a3xx_registers;
        adreno_gpu->reg_offsets = a3xx_register_offsets;
 
-       ret = adreno_gpu_init(dev, pdev, adreno_gpu, &funcs);
+       ret = adreno_gpu_init(dev, pdev, adreno_gpu, &funcs, 1);
        if (ret)
                goto fail;
 
drivers/gpu/drm/msm/adreno/a4xx_gpu.c +5 -4
index 19abf22..35fbf18 100644
@@ -116,7 +116,7 @@ static void a4xx_enable_hwcg(struct msm_gpu *gpu)
 
 static bool a4xx_me_init(struct msm_gpu *gpu)
 {
-       struct msm_ringbuffer *ring = gpu->rb;
+       struct msm_ringbuffer *ring = gpu->rb[0];
 
        OUT_PKT3(ring, CP_ME_INIT, 17);
        OUT_RING(ring, 0x000003f7);
@@ -137,7 +137,7 @@ static bool a4xx_me_init(struct msm_gpu *gpu)
        OUT_RING(ring, 0x00000000);
        OUT_RING(ring, 0x00000000);
 
-       gpu->funcs->flush(gpu);
+       gpu->funcs->flush(gpu, ring);
        return a4xx_idle(gpu);
 }
 
@@ -337,7 +337,7 @@ static void a4xx_destroy(struct msm_gpu *gpu)
 static bool a4xx_idle(struct msm_gpu *gpu)
 {
        /* wait for ringbuffer to drain: */
-       if (!adreno_idle(gpu))
+       if (!adreno_idle(gpu, gpu->rb[0]))
                return false;
 
        /* then wait for GPU to finish: */
@@ -535,6 +535,7 @@ static int a4xx_get_timestamp(struct msm_gpu *gpu, uint64_t *value)
                .last_fence = adreno_last_fence,
                .submit = adreno_submit,
                .flush = adreno_flush,
+               .active_ring = adreno_active_ring,
                .irq = a4xx_irq,
                .destroy = a4xx_destroy,
 #ifdef CONFIG_DEBUG_FS
@@ -576,7 +577,7 @@ struct msm_gpu *a4xx_gpu_init(struct drm_device *dev)
        adreno_gpu->registers = a4xx_registers;
        adreno_gpu->reg_offsets = a4xx_register_offsets;
 
-       ret = adreno_gpu_init(dev, pdev, adreno_gpu, &funcs);
+       ret = adreno_gpu_init(dev, pdev, adreno_gpu, &funcs, 1);
        if (ret)
                goto fail;
 
drivers/gpu/drm/msm/adreno/a5xx_gpu.c +24 -21
index fd54cc7..aaa941e 100644
@@ -80,7 +80,7 @@ static void a5xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit,
 {
        struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
        struct msm_drm_private *priv = gpu->dev->dev_private;
-       struct msm_ringbuffer *ring = gpu->rb;
+       struct msm_ringbuffer *ring = submit->ring;
        unsigned int i, ibs = 0;
 
        for (i = 0; i < submit->nr_cmds; i++) {
@@ -105,11 +105,11 @@ static void a5xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit,
 
        OUT_PKT7(ring, CP_EVENT_WRITE, 4);
        OUT_RING(ring, CACHE_FLUSH_TS | (1 << 31));
-       OUT_RING(ring, lower_32_bits(rbmemptr(adreno_gpu, fence)));
-       OUT_RING(ring, upper_32_bits(rbmemptr(adreno_gpu, fence)));
+       OUT_RING(ring, lower_32_bits(rbmemptr(adreno_gpu, ring->id, fence)));
+       OUT_RING(ring, upper_32_bits(rbmemptr(adreno_gpu, ring->id, fence)));
        OUT_RING(ring, submit->fence->seqno);
 
-       gpu->funcs->flush(gpu);
+       gpu->funcs->flush(gpu, ring);
 }
 
 struct a5xx_hwcg {
@@ -249,7 +249,7 @@ static void a5xx_enable_hwcg(struct msm_gpu *gpu)
 static int a5xx_me_init(struct msm_gpu *gpu)
 {
        struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
-       struct msm_ringbuffer *ring = gpu->rb;
+       struct msm_ringbuffer *ring = gpu->rb[0];
 
        OUT_PKT7(ring, CP_ME_INIT, 8);
 
@@ -280,9 +280,8 @@ static int a5xx_me_init(struct msm_gpu *gpu)
        OUT_RING(ring, 0x00000000);
        OUT_RING(ring, 0x00000000);
 
-       gpu->funcs->flush(gpu);
-
-       return a5xx_idle(gpu) ? 0 : -EINVAL;
+       gpu->funcs->flush(gpu, ring);
+       return a5xx_idle(gpu, ring) ? 0 : -EINVAL;
 }
 
 static struct drm_gem_object *a5xx_ucode_load_bo(struct msm_gpu *gpu,
@@ -628,11 +627,11 @@ static int a5xx_hw_init(struct msm_gpu *gpu)
         * ticking correctly
         */
        if (adreno_is_a530(adreno_gpu)) {
-               OUT_PKT7(gpu->rb, CP_EVENT_WRITE, 1);
-               OUT_RING(gpu->rb, 0x0F);
+               OUT_PKT7(gpu->rb[0], CP_EVENT_WRITE, 1);
+               OUT_RING(gpu->rb[0], 0x0F);
 
-               gpu->funcs->flush(gpu);
-               if (!a5xx_idle(gpu))
+               gpu->funcs->flush(gpu, gpu->rb[0]);
+               if (!a5xx_idle(gpu, gpu->rb[0]))
                        return -EINVAL;
        }
 
@@ -645,11 +644,11 @@ static int a5xx_hw_init(struct msm_gpu *gpu)
         */
        ret = a5xx_zap_shader_init(gpu);
        if (!ret) {
-               OUT_PKT7(gpu->rb, CP_SET_SECURE_MODE, 1);
-               OUT_RING(gpu->rb, 0x00000000);
+               OUT_PKT7(gpu->rb[0], CP_SET_SECURE_MODE, 1);
+               OUT_RING(gpu->rb[0], 0x00000000);
 
-               gpu->funcs->flush(gpu);
-               if (!a5xx_idle(gpu))
+               gpu->funcs->flush(gpu, gpu->rb[0]);
+               if (!a5xx_idle(gpu, gpu->rb[0]))
                        return -EINVAL;
        } else {
                /* Print a warning so if we die, we know why */
@@ -726,16 +725,19 @@ static inline bool _a5xx_check_idle(struct msm_gpu *gpu)
                A5XX_RBBM_INT_0_MASK_MISC_HANG_DETECT);
 }
 
-bool a5xx_idle(struct msm_gpu *gpu)
+bool a5xx_idle(struct msm_gpu *gpu, struct msm_ringbuffer *ring)
 {
        /* wait for CP to drain ringbuffer: */
-       if (!adreno_idle(gpu))
+       if (!adreno_idle(gpu, ring))
                return false;
 
        if (spin_until(_a5xx_check_idle(gpu))) {
-               DRM_ERROR("%s: %ps: timeout waiting for GPU to idle: status %8.8X irq %8.8X\n",
-                       gpu->name, __builtin_return_address(0),
+               DRM_DEV_ERROR(gpu->dev->dev,
+                       "timeout waiting for GPU RB %d to idle: status %8.8X rptr/wptr: %4.4X/%4.4X irq %8.8X\n",
+                       ring->id,
                        gpu_read(gpu, REG_A5XX_RBBM_STATUS),
+                       gpu_read(gpu, REG_A5XX_CP_RB_RPTR),
+                       gpu_read(gpu, REG_A5XX_CP_RB_WPTR),
                        gpu_read(gpu, REG_A5XX_RBBM_INT_0_STATUS));
 
                return false;
@@ -1031,6 +1033,7 @@ static void a5xx_show(struct msm_gpu *gpu, struct seq_file *m)
                .last_fence = adreno_last_fence,
                .submit = a5xx_submit,
                .flush = adreno_flush,
+               .active_ring = adreno_active_ring,
                .irq = a5xx_irq,
                .destroy = a5xx_destroy,
 #ifdef CONFIG_DEBUG_FS
@@ -1067,7 +1070,7 @@ struct msm_gpu *a5xx_gpu_init(struct drm_device *dev)
 
        a5xx_gpu->lm_leakage = 0x4E001A;
 
-       ret = adreno_gpu_init(dev, pdev, adreno_gpu, &funcs);
+       ret = adreno_gpu_init(dev, pdev, adreno_gpu, &funcs, 1);
        if (ret) {
                a5xx_destroy(&(a5xx_gpu->base.base));
                return ERR_PTR(ret);
drivers/gpu/drm/msm/adreno/a5xx_gpu.h +1 -1
index 6638bc8..aba6faf 100644
@@ -58,6 +58,6 @@ static inline int spin_usecs(struct msm_gpu *gpu, uint32_t usecs,
        return -ETIMEDOUT;
 }
 
-bool a5xx_idle(struct msm_gpu *gpu);
+bool a5xx_idle(struct msm_gpu *gpu, struct msm_ringbuffer *ring);
 
 #endif /* __A5XX_GPU_H__ */
drivers/gpu/drm/msm/adreno/a5xx_power.c +3 -3
index 2fdee44..a7d91ac 100644
@@ -173,7 +173,7 @@ static int a5xx_gpmu_init(struct msm_gpu *gpu)
 {
        struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
        struct a5xx_gpu *a5xx_gpu = to_a5xx_gpu(adreno_gpu);
-       struct msm_ringbuffer *ring = gpu->rb;
+       struct msm_ringbuffer *ring = gpu->rb[0];
 
        if (!a5xx_gpu->gpmu_dwords)
                return 0;
@@ -192,9 +192,9 @@ static int a5xx_gpmu_init(struct msm_gpu *gpu)
        OUT_PKT7(ring, CP_SET_PROTECTED_MODE, 1);
        OUT_RING(ring, 1);
 
-       gpu->funcs->flush(gpu);
+       gpu->funcs->flush(gpu, ring);
 
-       if (!a5xx_idle(gpu)) {
+       if (!a5xx_idle(gpu, ring)) {
                DRM_ERROR("%s: Unable to load GPMU firmware. GPMU will not be active\n",
                        gpu->name);
                return -EINVAL;
drivers/gpu/drm/msm/adreno/adreno_gpu.c +95 -54
index 4a24506..6b7114d 100644
@@ -21,7 +21,6 @@
 #include "msm_gem.h"
 #include "msm_mmu.h"
 
-#define RB_SIZE    SZ_32K
 #define RB_BLKSIZE 32
 
 int adreno_get_param(struct msm_gpu *gpu, uint32_t param, uint64_t *value)
@@ -60,39 +59,47 @@ int adreno_get_param(struct msm_gpu *gpu, uint32_t param, uint64_t *value)
 int adreno_hw_init(struct msm_gpu *gpu)
 {
        struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
-       int ret;
+       int i;
 
        DBG("%s", gpu->name);
 
-       ret = msm_gem_get_iova(gpu->rb->bo, gpu->aspace, &gpu->rb_iova);
-       if (ret) {
-               gpu->rb_iova = 0;
-               dev_err(gpu->dev->dev, "could not map ringbuffer: %d\n", ret);
-               return ret;
-       }
+       for (i = 0; i < gpu->nr_rings; i++) {
+               struct msm_ringbuffer *ring = gpu->rb[i];
+               int ret;
 
-       /* reset ringbuffer: */
-       gpu->rb->cur = gpu->rb->start;
+               if (!ring)
+                       continue;
 
-       /* reset completed fence seqno: */
-       adreno_gpu->memptrs->fence = gpu->fctx->completed_fence;
-       adreno_gpu->memptrs->rptr  = 0;
+               ret = msm_gem_get_iova(ring->bo, gpu->aspace, &ring->iova);
+               if (ret) {
+                       ring->iova = 0;
+                       dev_err(gpu->dev->dev,
+                               "could not map ringbuffer %d: %d\n", i, ret);
+                       return ret;
+               }
+
+               ring->cur = ring->start;
+
+               /* reset completed fence seqno: */
+               adreno_gpu->memptrs->fence[ring->id] = ring->completed_fence;
+               adreno_gpu->memptrs->rptr[ring->id]  = 0;
+       }
 
        /* Setup REG_CP_RB_CNTL: */
        adreno_gpu_write(adreno_gpu, REG_ADRENO_CP_RB_CNTL,
-                       /* size is log2(quad-words): */
-                       AXXX_CP_RB_CNTL_BUFSZ(ilog2(gpu->rb->size / 8)) |
-                       AXXX_CP_RB_CNTL_BLKSZ(ilog2(RB_BLKSIZE / 8)) |
-                       (adreno_is_a430(adreno_gpu) ? AXXX_CP_RB_CNTL_NO_UPDATE : 0));
+               /* size is log2(quad-words): */
+               AXXX_CP_RB_CNTL_BUFSZ(ilog2(MSM_GPU_RINGBUFFER_SZ / 8)) |
+               AXXX_CP_RB_CNTL_BLKSZ(ilog2(RB_BLKSIZE / 8)) |
+               (adreno_is_a430(adreno_gpu) ? AXXX_CP_RB_CNTL_NO_UPDATE : 0));
 
-       /* Setup ringbuffer address: */
+       /* Setup ringbuffer address - use ringbuffer[0] for GPU init */
        adreno_gpu_write64(adreno_gpu, REG_ADRENO_CP_RB_BASE,
-               REG_ADRENO_CP_RB_BASE_HI, gpu->rb_iova);
+               REG_ADRENO_CP_RB_BASE_HI, gpu->rb[0]->iova);
 
        if (!adreno_is_a430(adreno_gpu)) {
                adreno_gpu_write64(adreno_gpu, REG_ADRENO_CP_RB_RPTR_ADDR,
                        REG_ADRENO_CP_RB_RPTR_ADDR_HI,
-                       rbmemptr(adreno_gpu, rptr));
+                       rbmemptr(adreno_gpu, 0, rptr));
        }
 
        return 0;
@@ -104,19 +111,35 @@ static uint32_t get_wptr(struct msm_ringbuffer *ring)
 }
 
 /* Use this helper to read rptr, since a430 doesn't update rptr in memory */
-static uint32_t get_rptr(struct adreno_gpu *adreno_gpu)
+static uint32_t get_rptr(struct adreno_gpu *adreno_gpu,
+               struct msm_ringbuffer *ring)
 {
-       if (adreno_is_a430(adreno_gpu))
-               return adreno_gpu->memptrs->rptr = adreno_gpu_read(
+       if (adreno_is_a430(adreno_gpu)) {
+               /*
+                * If index is anything but 0 this will probably break horribly,
+                * but I think that we have enough infrastructure in place to
+                * ensure that it won't be. If not then this is why your
+                * a430 stopped working.
+                */
+               return adreno_gpu->memptrs->rptr[ring->id] = adreno_gpu_read(
                        adreno_gpu, REG_ADRENO_CP_RB_RPTR);
-       else
-               return adreno_gpu->memptrs->rptr;
+       } else
+               return adreno_gpu->memptrs->rptr[ring->id];
 }
 
-uint32_t adreno_last_fence(struct msm_gpu *gpu)
+struct msm_ringbuffer *adreno_active_ring(struct msm_gpu *gpu)
+{
+       return gpu->rb[0];
+}
+
+uint32_t adreno_last_fence(struct msm_gpu *gpu, struct msm_ringbuffer *ring)
 {
        struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
-       return adreno_gpu->memptrs->fence;
+
+       if (!ring)
+               return 0;
+
+       return adreno_gpu->memptrs->fence[ring->id];
 }
 
 void adreno_recover(struct msm_gpu *gpu)
@@ -142,7 +165,7 @@ void adreno_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit,
 {
        struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
        struct msm_drm_private *priv = gpu->dev->dev_private;
-       struct msm_ringbuffer *ring = gpu->rb;
+       struct msm_ringbuffer *ring = submit->ring;
        unsigned i;
 
        for (i = 0; i < submit->nr_cmds; i++) {
@@ -181,7 +204,7 @@ void adreno_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit,
 
        OUT_PKT3(ring, CP_EVENT_WRITE, 3);
        OUT_RING(ring, CACHE_FLUSH_TS);
-       OUT_RING(ring, rbmemptr(adreno_gpu, fence));
+       OUT_RING(ring, rbmemptr(adreno_gpu, ring->id, fence));
        OUT_RING(ring, submit->fence->seqno);
 
        /* we could maybe be clever and only CP_COND_EXEC the interrupt: */
@@ -208,10 +231,10 @@ void adreno_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit,
        }
 #endif
 
-       gpu->funcs->flush(gpu);
+       gpu->funcs->flush(gpu, ring);
 }
 
-void adreno_flush(struct msm_gpu *gpu)
+void adreno_flush(struct msm_gpu *gpu, struct msm_ringbuffer *ring)
 {
        struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
        uint32_t wptr;
@@ -221,7 +244,7 @@ void adreno_flush(struct msm_gpu *gpu)
         * to account for the possibility that the last command fit exactly into
         * the ringbuffer and rb->next hasn't wrapped to zero yet
         */
-       wptr = get_wptr(gpu->rb) & ((gpu->rb->size / 4) - 1);
+       wptr = get_wptr(ring) % (MSM_GPU_RINGBUFFER_SZ >> 2);
 
        /* ensure writes to ringbuffer have hit system memory: */
        mb();
@@ -229,17 +252,18 @@ void adreno_flush(struct msm_gpu *gpu)
        adreno_gpu_write(adreno_gpu, REG_ADRENO_CP_RB_WPTR, wptr);
 }
 
-bool adreno_idle(struct msm_gpu *gpu)
+bool adreno_idle(struct msm_gpu *gpu, struct msm_ringbuffer *ring)
 {
        struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
-       uint32_t wptr = get_wptr(gpu->rb);
+       uint32_t wptr = get_wptr(ring);
 
        /* wait for CP to drain ringbuffer: */
-       if (!spin_until(get_rptr(adreno_gpu) == wptr))
+       if (!spin_until(get_rptr(adreno_gpu, ring) == wptr))
                return true;
 
        /* TODO maybe we need to reset GPU here to recover from hang? */
-       DRM_ERROR("%s: timeout waiting to drain ringbuffer!\n", gpu->name);
+       DRM_ERROR("%s: timeout waiting to drain ringbuffer %d!\n", gpu->name,
+               ring->id);
        return false;
 }
 
@@ -254,10 +278,17 @@ void adreno_show(struct msm_gpu *gpu, struct seq_file *m)
                        adreno_gpu->rev.major, adreno_gpu->rev.minor,
                        adreno_gpu->rev.patchid);
 
-       seq_printf(m, "fence:    %d/%d\n", adreno_gpu->memptrs->fence,
-                       gpu->fctx->last_fence);
-       seq_printf(m, "rptr:     %d\n", get_rptr(adreno_gpu));
-       seq_printf(m, "rb wptr:  %d\n", get_wptr(gpu->rb));
+       for (i = 0; i < gpu->nr_rings; i++) {
+               struct msm_ringbuffer *ring = gpu->rb[i];
+
+               seq_printf(m, "rb %d: fence:    %d/%d\n", i,
+                       adreno_last_fence(gpu, ring),
+                       ring->completed_fence);
+
+               seq_printf(m, "      rptr:     %d\n",
+                       get_rptr(adreno_gpu, ring));
+               seq_printf(m, "rb wptr:  %d\n", get_wptr(ring));
+       }
 
        /* dump these out in a form that can be parsed by demsm: */
        seq_printf(m, "IO:region %s 00000000 00020000\n", gpu->name);
@@ -283,16 +314,23 @@ void adreno_show(struct msm_gpu *gpu, struct seq_file *m)
 void adreno_dump_info(struct msm_gpu *gpu)
 {
        struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
+       int i;
 
        printk("revision: %d (%d.%d.%d.%d)\n",
                        adreno_gpu->info->revn, adreno_gpu->rev.core,
                        adreno_gpu->rev.major, adreno_gpu->rev.minor,
                        adreno_gpu->rev.patchid);
 
-       printk("fence:    %d/%d\n", adreno_gpu->memptrs->fence,
-                       gpu->fctx->last_fence);
-       printk("rptr:     %d\n", get_rptr(adreno_gpu));
-       printk("rb wptr:  %d\n", get_wptr(gpu->rb));
+       for (i = 0; i < gpu->nr_rings; i++) {
+               struct msm_ringbuffer *ring = gpu->rb[i];
+
+               printk("rb %d: fence:    %d/%d\n", i,
+                       adreno_last_fence(gpu, ring),
+                       ring->completed_fence);
+
+               printk("rptr:     %d\n", get_rptr(adreno_gpu, ring));
+               printk("rb wptr:  %d\n", get_wptr(ring));
+       }
 }
 
 /* would be nice to not have to duplicate the _show() stuff with printk(): */
@@ -315,19 +353,21 @@ void adreno_dump(struct msm_gpu *gpu)
        }
 }
 
-static uint32_t ring_freewords(struct msm_gpu *gpu)
+static uint32_t ring_freewords(struct msm_ringbuffer *ring)
 {
-       struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
-       uint32_t size = gpu->rb->size / 4;
-       uint32_t wptr = get_wptr(gpu->rb);
-       uint32_t rptr = get_rptr(adreno_gpu);
+       struct adreno_gpu *adreno_gpu = to_adreno_gpu(ring->gpu);
+       uint32_t size = MSM_GPU_RINGBUFFER_SZ >> 2;
+       uint32_t wptr = get_wptr(ring);
+       uint32_t rptr = get_rptr(adreno_gpu, ring);
        return (rptr + (size - 1) - wptr) % size;
 }
 
-void adreno_wait_ring(struct msm_gpu *gpu, uint32_t ndwords)
+void adreno_wait_ring(struct msm_ringbuffer *ring, uint32_t ndwords)
 {
-       if (spin_until(ring_freewords(gpu) >= ndwords))
-               DRM_ERROR("%s: timeout waiting for ringbuffer space\n", gpu->name);
+       if (spin_until(ring_freewords(ring) >= ndwords))
+               DRM_DEV_ERROR(ring->gpu->dev->dev,
+                       "timeout waiting for space in ringubffer %d\n",
+                       ring->id);
 }
 
 static const char *iommu_ports[] = {
@@ -336,7 +376,8 @@ void adreno_wait_ring(struct msm_gpu *gpu, uint32_t ndwords)
 };
 
 int adreno_gpu_init(struct drm_device *drm, struct platform_device *pdev,
-               struct adreno_gpu *adreno_gpu, const struct adreno_gpu_funcs *funcs)
+               struct adreno_gpu *adreno_gpu,
+               const struct adreno_gpu_funcs *funcs, int nr_rings)
 {
        struct adreno_platform_config *config = pdev->dev.platform_data;
        struct msm_gpu_config adreno_gpu_config  = { 0 };
@@ -364,7 +405,7 @@ int adreno_gpu_init(struct drm_device *drm, struct platform_device *pdev,
        adreno_gpu_config.va_start = SZ_16M;
        adreno_gpu_config.va_end = 0xffffffff;
 
-       adreno_gpu_config.ringsz = RB_SIZE;
+       adreno_gpu_config.nr_rings = nr_rings;
 
        ret = msm_gpu_init(drm, pdev, &adreno_gpu->base, &funcs->base,
                        adreno_gpu->info->name, &adreno_gpu_config);
drivers/gpu/drm/msm/adreno/adreno_gpu.h +23 -13
index 4d9165f..9e78e49 100644
@@ -82,12 +82,18 @@ struct adreno_info {
 
 const struct adreno_info *adreno_info(struct adreno_rev rev);
 
-#define rbmemptr(adreno_gpu, member)  \
+#define _sizeof(member) \
+       sizeof(((struct adreno_rbmemptrs *) 0)->member[0])
+
+#define _base(adreno_gpu, member)  \
        ((adreno_gpu)->memptrs_iova + offsetof(struct adreno_rbmemptrs, member))
 
+#define rbmemptr(adreno_gpu, index, member) \
+       (_base((adreno_gpu), member) + ((index) * _sizeof(member)))
+
 struct adreno_rbmemptrs {
-       volatile uint32_t rptr;
-       volatile uint32_t fence;
+       volatile uint32_t rptr[MSM_GPU_MAX_RINGS];
+       volatile uint32_t fence[MSM_GPU_MAX_RINGS];
 };
 
 struct adreno_gpu {
@@ -197,21 +203,25 @@ static inline int adreno_is_a530(struct adreno_gpu *gpu)
 
 int adreno_get_param(struct msm_gpu *gpu, uint32_t param, uint64_t *value);
 int adreno_hw_init(struct msm_gpu *gpu);
-uint32_t adreno_last_fence(struct msm_gpu *gpu);
+uint32_t adreno_last_fence(struct msm_gpu *gpu, struct msm_ringbuffer *ring);
+uint32_t adreno_submitted_fence(struct msm_gpu *gpu,
+               struct msm_ringbuffer *ring);
 void adreno_recover(struct msm_gpu *gpu);
 void adreno_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit,
                struct msm_file_private *ctx);
-void adreno_flush(struct msm_gpu *gpu);
-bool adreno_idle(struct msm_gpu *gpu);
+void adreno_flush(struct msm_gpu *gpu, struct msm_ringbuffer *ring);
+bool adreno_idle(struct msm_gpu *gpu, struct msm_ringbuffer *ring);
 #ifdef CONFIG_DEBUG_FS
 void adreno_show(struct msm_gpu *gpu, struct seq_file *m);
 #endif
 void adreno_dump_info(struct msm_gpu *gpu);
 void adreno_dump(struct msm_gpu *gpu);
-void adreno_wait_ring(struct msm_gpu *gpu, uint32_t ndwords);
+void adreno_wait_ring(struct msm_ringbuffer *ring, uint32_t ndwords);
+struct msm_ringbuffer *adreno_active_ring(struct msm_gpu *gpu);
 
 int adreno_gpu_init(struct drm_device *drm, struct platform_device *pdev,
-               struct adreno_gpu *gpu, const struct adreno_gpu_funcs *funcs);
+               struct adreno_gpu *gpu, const struct adreno_gpu_funcs *funcs,
+               int nr_rings);
 void adreno_gpu_cleanup(struct adreno_gpu *gpu);
 
 
@@ -220,7 +230,7 @@ int adreno_gpu_init(struct drm_device *drm, struct platform_device *pdev,
 static inline void
 OUT_PKT0(struct msm_ringbuffer *ring, uint16_t regindx, uint16_t cnt)
 {
-       adreno_wait_ring(ring->gpu, cnt+1);
+       adreno_wait_ring(ring, cnt+1);
        OUT_RING(ring, CP_TYPE0_PKT | ((cnt-1) << 16) | (regindx & 0x7FFF));
 }
 
@@ -228,14 +238,14 @@ int adreno_gpu_init(struct drm_device *drm, struct platform_device *pdev,
 static inline void
 OUT_PKT2(struct msm_ringbuffer *ring)
 {
-       adreno_wait_ring(ring->gpu, 1);
+       adreno_wait_ring(ring, 1);
        OUT_RING(ring, CP_TYPE2_PKT);
 }
 
 static inline void
 OUT_PKT3(struct msm_ringbuffer *ring, uint8_t opcode, uint16_t cnt)
 {
-       adreno_wait_ring(ring->gpu, cnt+1);
+       adreno_wait_ring(ring, cnt+1);
        OUT_RING(ring, CP_TYPE3_PKT | ((cnt-1) << 16) | ((opcode & 0xFF) << 8));
 }
 
@@ -257,14 +267,14 @@ static inline u32 PM4_PARITY(u32 val)
 static inline void
 OUT_PKT4(struct msm_ringbuffer *ring, uint16_t regindx, uint16_t cnt)
 {
-       adreno_wait_ring(ring->gpu, cnt + 1);
+       adreno_wait_ring(ring, cnt + 1);
        OUT_RING(ring, PKT4(regindx, cnt));
 }
 
 static inline void
 OUT_PKT7(struct msm_ringbuffer *ring, uint8_t opcode, uint16_t cnt)
 {
-       adreno_wait_ring(ring->gpu, cnt + 1);
+       adreno_wait_ring(ring, cnt + 1);
        OUT_RING(ring, CP_TYPE7_PKT | (cnt << 0) | (PM4_PARITY(cnt) << 15) |
                ((opcode & 0x7F) << 16) | (PM4_PARITY(opcode) << 23));
 }
drivers/gpu/drm/msm/msm_drv.h +2 -0
index 192147c..bbf7d3d 100644
@@ -76,6 +76,8 @@ struct msm_vblank_ctrl {
        spinlock_t lock;
 };
 
+#define MSM_GPU_MAX_RINGS 1
+
 struct msm_drm_private {
 
        struct drm_device *dev;
drivers/gpu/drm/msm/msm_fence.c +71 -21
index 3f299c5..8cf029f 100644
@@ -20,7 +20,6 @@
 #include "msm_drv.h"
 #include "msm_fence.h"
 
-
 struct msm_fence_context *
 msm_fence_context_alloc(struct drm_device *dev, const char *name)
 {
@@ -32,9 +31,10 @@ struct msm_fence_context *
 
        fctx->dev = dev;
        fctx->name = name;
-       fctx->context = dma_fence_context_alloc(1);
+       fctx->context = dma_fence_context_alloc(MSM_GPU_MAX_RINGS);
        init_waitqueue_head(&fctx->event);
        spin_lock_init(&fctx->spinlock);
+       hash_init(fctx->hash);
 
        return fctx;
 }
@@ -44,64 +44,94 @@ void msm_fence_context_free(struct msm_fence_context *fctx)
        kfree(fctx);
 }
 
-static inline bool fence_completed(struct msm_fence_context *fctx, uint32_t fence)
+static inline bool fence_completed(struct msm_ringbuffer *ring, uint32_t fence)
+{
+       return (int32_t)(ring->completed_fence - fence) >= 0;
+}
+
+struct msm_fence {
+       struct msm_fence_context *fctx;
+       struct msm_ringbuffer *ring;
+       struct dma_fence base;
+       struct hlist_node node;
+       u32 fence_id;
+};
+
+static struct msm_fence *fence_from_id(struct msm_fence_context *fctx,
+               uint32_t id)
 {
-       return (int32_t)(fctx->completed_fence - fence) >= 0;
+       struct msm_fence *f;
+
+       hash_for_each_possible_rcu(fctx->hash, f, node, id) {
+               if (f->fence_id == id) {
+                       if (dma_fence_get_rcu(&f->base))
+                               return f;
+               }
+       }
+
+       return NULL;
 }
 
 /* legacy path for WAIT_FENCE ioctl: */
 int msm_wait_fence(struct msm_fence_context *fctx, uint32_t fence,
                ktime_t *timeout, bool interruptible)
 {
+       struct msm_fence *f = fence_from_id(fctx, fence);
        int ret;
 
-       if (fence > fctx->last_fence) {
-               DRM_ERROR("%s: waiting on invalid fence: %u (of %u)\n",
-                               fctx->name, fence, fctx->last_fence);
-               return -EINVAL;
+       /* If no active fence was found, there are two possibilities */
+       if (!f) {
+               /* The requested ID is newer than last issued - return error */
+               if (fence > fctx->fence_id) {
+                       DRM_ERROR("%s: waiting on invalid fence: %u (of %u)\n",
+                               fctx->name, fence, fctx->fence_id);
+                       return -EINVAL;
+               }
+
+               /* If the id has been issued assume fence has been retired */
+               return 0;
        }
 
        if (!timeout) {
                /* no-wait: */
-               ret = fence_completed(fctx, fence) ? 0 : -EBUSY;
+               ret = fence_completed(f->ring, f->base.seqno) ? 0 : -EBUSY;
        } else {
                unsigned long remaining_jiffies = timeout_to_jiffies(timeout);
 
                if (interruptible)
                        ret = wait_event_interruptible_timeout(fctx->event,
-                               fence_completed(fctx, fence),
+                               fence_completed(f->ring, f->base.seqno),
                                remaining_jiffies);
                else
                        ret = wait_event_timeout(fctx->event,
-                               fence_completed(fctx, fence),
+                               fence_completed(f->ring, f->base.seqno),
                                remaining_jiffies);
 
                if (ret == 0) {
                        DBG("timeout waiting for fence: %u (completed: %u)",
-                                       fence, fctx->completed_fence);
+                               f->base.seqno, f->ring->completed_fence);
                        ret = -ETIMEDOUT;
                } else if (ret != -ERESTARTSYS) {
                        ret = 0;
                }
        }
 
+       dma_fence_put(&f->base);
+
        return ret;
 }
 
 /* called from workqueue */
-void msm_update_fence(struct msm_fence_context *fctx, uint32_t fence)
+void msm_update_fence(struct msm_fence_context *fctx,
+               struct msm_ringbuffer *ring, uint32_t fence)
 {
        spin_lock(&fctx->spinlock);
-       fctx->completed_fence = max(fence, fctx->completed_fence);
+       ring->completed_fence = max(fence, ring->completed_fence);
        spin_unlock(&fctx->spinlock);
 
        wake_up_all(&fctx->event);
 }
 
-struct msm_fence {
-       struct msm_fence_context *fctx;
-       struct dma_fence base;
-};
 
 static inline struct msm_fence *to_msm_fence(struct dma_fence *fence)
 {
@@ -127,12 +157,17 @@ static bool msm_fence_enable_signaling(struct dma_fence *fence)
 static bool msm_fence_signaled(struct dma_fence *fence)
 {
        struct msm_fence *f = to_msm_fence(fence);
-       return fence_completed(f->fctx, f->base.seqno);
+       return fence_completed(f->ring, f->base.seqno);
 }
 
 static void msm_fence_release(struct dma_fence *fence)
 {
        struct msm_fence *f = to_msm_fence(fence);
+
+       spin_lock(&f->fctx->spinlock);
+       hash_del_rcu(&f->node);
+       spin_unlock(&f->fctx->spinlock);
+
        kfree_rcu(f, base.rcu);
 }
 
@@ -145,8 +180,15 @@ static void msm_fence_release(struct dma_fence *fence)
        .release = msm_fence_release,
 };
 
+uint32_t msm_fence_id(struct dma_fence *fence)
+{
+       struct msm_fence *f = to_msm_fence(fence);
+
+       return f->fence_id;
+}
+
 struct dma_fence *
-msm_fence_alloc(struct msm_fence_context *fctx)
+msm_fence_alloc(struct msm_fence_context *fctx, struct msm_ringbuffer *ring)
 {
        struct msm_fence *f;
 
@@ -155,9 +197,17 @@ struct dma_fence *
                return ERR_PTR(-ENOMEM);
 
        f->fctx = fctx;
+       f->ring = ring;
+
+       /* Make a user fence ID to pass back for the legacy functions */
+       f->fence_id = ++fctx->fence_id;
+
+       spin_lock(&fctx->spinlock);
+       hash_add(fctx->hash, &f->node, f->fence_id);
+       spin_unlock(&fctx->spinlock);
 
        dma_fence_init(&f->base, &msm_fence_ops, &fctx->spinlock,
-                      fctx->context, ++fctx->last_fence);
+                      fctx->context + ring->id, ++ring->last_fence);
 
        return &f->base;
 }
drivers/gpu/drm/msm/msm_fence.h +10 -5
index 56061aa..b5c6830 100644
@@ -18,17 +18,18 @@
 #ifndef __MSM_FENCE_H__
 #define __MSM_FENCE_H__
 
+#include <linux/hashtable.h>
 #include "msm_drv.h"
+#include "msm_ringbuffer.h"
 
 struct msm_fence_context {
        struct drm_device *dev;
        const char *name;
        unsigned context;
-       /* last_fence == completed_fence --> no pending work */
-       uint32_t last_fence;          /* last assigned fence */
-       uint32_t completed_fence;     /* last completed fence */
+       u32 fence_id;
        wait_queue_head_t event;
        spinlock_t spinlock;
+       DECLARE_HASHTABLE(hash, 4);
 };
 
 struct msm_fence_context * msm_fence_context_alloc(struct drm_device *dev,
@@ -39,8 +40,12 @@ int msm_wait_fence(struct msm_fence_context *fctx, uint32_t fence,
                ktime_t *timeout, bool interruptible);
 int msm_queue_fence_cb(struct msm_fence_context *fctx,
                struct msm_fence_cb *cb, uint32_t fence);
-void msm_update_fence(struct msm_fence_context *fctx, uint32_t fence);
+void msm_update_fence(struct msm_fence_context *fctx,
+               struct msm_ringbuffer *ring, uint32_t fence);
 
-struct dma_fence * msm_fence_alloc(struct msm_fence_context *fctx);
+struct dma_fence *msm_fence_alloc(struct msm_fence_context *fctx,
+               struct msm_ringbuffer *ring);
+
+uint32_t msm_fence_id(struct dma_fence *fence);
 
 #endif
drivers/gpu/drm/msm/msm_gem.h +2 -1
index 2767014..ddae0a9 100644
@@ -116,12 +116,13 @@ static inline bool is_vunmapable(struct msm_gem_object *msm_obj)
 struct msm_gem_submit {
        struct drm_device *dev;
        struct msm_gpu *gpu;
-       struct list_head node;   /* node in gpu submit_list */
+       struct list_head node;   /* node in ring submit list */
        struct list_head bo_list;
        struct ww_acquire_ctx ticket;
        struct dma_fence *fence;
        struct pid *pid;    /* submitting process */
        bool valid;         /* true if no cmdstream patching needed */
+       struct msm_ringbuffer *ring;
        unsigned int nr_cmds;
        unsigned int nr_bos;
        struct {
drivers/gpu/drm/msm/msm_gem_submit.c +7 -3
index 0129ca2..4f483c0 100644
@@ -418,7 +418,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
        int out_fence_fd = -1;
        unsigned i;
        u32 prio = 0;
-       int ret;
+       int ret, ring;
 
        if (!gpu)
                return -ENXIO;
@@ -552,7 +552,11 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
 
        submit->nr_cmds = i;
 
-       submit->fence = msm_fence_alloc(gpu->fctx);
+       ring = clamp_t(uint32_t, prio, 0, gpu->nr_rings - 1);
+
+       submit->ring = gpu->rb[ring];
+
+       submit->fence = msm_fence_alloc(gpu->fctx, submit->ring);
        if (IS_ERR(submit->fence)) {
                ret = PTR_ERR(submit->fence);
                submit->fence = NULL;
@@ -569,7 +573,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
 
        msm_gpu_submit(gpu, submit, ctx);
 
-       args->fence = submit->fence->seqno;
+       args->fence = msm_fence_id(submit->fence);
 
        if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
                fd_install(out_fence_fd, sync_file->file);
drivers/gpu/drm/msm/msm_gpu.c +95 -43
index 1f753f0..a1bb3db 100644
@@ -226,15 +226,35 @@ static void recover_worker(struct work_struct *work)
        struct msm_gpu *gpu = container_of(work, struct msm_gpu, recover_work);
        struct drm_device *dev = gpu->dev;
        struct msm_gem_submit *submit;
-       uint32_t fence = gpu->funcs->last_fence(gpu);
+       struct msm_ringbuffer *cur_ring = gpu->funcs->active_ring(gpu);
+       uint32_t fence;
+       int i;
+
+       /* Update all the rings with the latest and greatest fence */
+       for (i = 0; i < ARRAY_SIZE(gpu->rb); i++) {
+               struct msm_ringbuffer *ring = gpu->rb[i];
+               uint32_t fence = gpu->funcs->last_fence(gpu, ring);
+
+               /*
+                * For the current (faulting?) ring/submit advance the fence by
+                * one more to clear the faulting submit
+                */
+               if (ring == cur_ring)
+                       fence = fence + 1;
 
-       msm_update_fence(gpu->fctx, fence + 1);
+               msm_update_fence(gpu->fctx, cur_ring, fence);
+       }
 
        mutex_lock(&dev->struct_mutex);
 
+
        dev_err(dev->dev, "%s: hangcheck recover!\n", gpu->name);
-       list_for_each_entry(submit, &gpu->submit_list, node) {
-               if (submit->fence->seqno == (fence + 1)) {
+
+       fence = gpu->funcs->last_fence(gpu, cur_ring) + 1;
+
+       list_for_each_entry(submit, &cur_ring->submits, node) {
+
+               if (submit->fence->seqno == fence) {
                        struct task_struct *task;
 
                        rcu_read_lock();
@@ -256,9 +276,16 @@ static void recover_worker(struct work_struct *work)
                gpu->funcs->recover(gpu);
                pm_runtime_put_sync(&gpu->pdev->dev);
 
-               /* replay the remaining submits after the one that hung: */
-               list_for_each_entry(submit, &gpu->submit_list, node) {
-                       gpu->funcs->submit(gpu, submit, NULL);
+               /*
+                * Replay all remaining submits starting with highest priority
+                * ring
+                */
+
+               for (i = gpu->nr_rings - 1; i >= 0; i--) {
+                       struct msm_ringbuffer *ring = gpu->rb[i];
+
+                       list_for_each_entry(submit, &ring->submits, node)
+                               gpu->funcs->submit(gpu, submit, NULL);
                }
        }
 
@@ -279,25 +306,27 @@ static void hangcheck_handler(unsigned long data)
        struct msm_gpu *gpu = (struct msm_gpu *)data;
        struct drm_device *dev = gpu->dev;
        struct msm_drm_private *priv = dev->dev_private;
-       uint32_t fence = gpu->funcs->last_fence(gpu);
+       struct msm_ringbuffer *ring = gpu->funcs->active_ring(gpu);
+       uint32_t fence = gpu->funcs->last_fence(gpu, ring);
 
-       if (fence != gpu->hangcheck_fence) {
+       if (fence != gpu->hangcheck_fence[ring->id]) {
                /* some progress has been made.. ya! */
-               gpu->hangcheck_fence = fence;
-       } else if (fence < gpu->fctx->last_fence) {
+               gpu->hangcheck_fence[ring->id] = fence;
+       } else if (fence < ring->last_fence) {
                /* no progress and not done.. hung! */
-               gpu->hangcheck_fence = fence;
-               dev_err(dev->dev, "%s: hangcheck detected gpu lockup!\n",
-                               gpu->name);
+               gpu->hangcheck_fence[ring->id] = fence;
+               dev_err(dev->dev, "%s: hangcheck detected gpu lockup rb %d!\n",
+                               gpu->name, ring->id);
                dev_err(dev->dev, "%s:     completed fence: %u\n",
                                gpu->name, fence);
                dev_err(dev->dev, "%s:     submitted fence: %u\n",
-                               gpu->name, gpu->fctx->last_fence);
+                               gpu->name, ring->last_fence);
+
                queue_work(priv->wq, &gpu->recover_work);
        }
 
        /* if still more pending work, reset the hangcheck timer: */
-       if (gpu->fctx->last_fence > gpu->hangcheck_fence)
+       if (ring->last_fence > gpu->hangcheck_fence[ring->id])
                hangcheck_timer_reset(gpu);
 
        /* workaround for missing irq: */
@@ -426,19 +455,18 @@ static void retire_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit)
 static void retire_submits(struct msm_gpu *gpu)
 {
        struct drm_device *dev = gpu->dev;
+       struct msm_gem_submit *submit, *tmp;
+       int i;
 
        WARN_ON(!mutex_is_locked(&dev->struct_mutex));
 
-       while (!list_empty(&gpu->submit_list)) {
-               struct msm_gem_submit *submit;
+       /* Retire the commits starting with highest priority */
+       for (i = gpu->nr_rings - 1; i >= 0; i--) {
+               struct msm_ringbuffer *ring = gpu->rb[i];
 
-               submit = list_first_entry(&gpu->submit_list,
-                               struct msm_gem_submit, node);
-
-               if (dma_fence_is_signaled(submit->fence)) {
-                       retire_submit(gpu, submit);
-               } else {
-                       break;
+               list_for_each_entry_safe(submit, tmp, &ring->submits, node) {
+                       if (dma_fence_is_signaled(submit->fence))
+                               retire_submit(gpu, submit);
                }
        }
 }
@@ -447,9 +475,12 @@ static void retire_worker(struct work_struct *work)
 {
        struct msm_gpu *gpu = container_of(work, struct msm_gpu, retire_work);
        struct drm_device *dev = gpu->dev;
-       uint32_t fence = gpu->funcs->last_fence(gpu);
+       int i;
+
+       for (i = 0; i < gpu->nr_rings; i++)
+               msm_update_fence(gpu->fctx, gpu->rb[i],
+                       gpu->funcs->last_fence(gpu, gpu->rb[i]));
 
-       msm_update_fence(gpu->fctx, fence);
 
        mutex_lock(&dev->struct_mutex);
        retire_submits(gpu);
@@ -470,6 +501,7 @@ void msm_gpu_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit,
 {
        struct drm_device *dev = gpu->dev;
        struct msm_drm_private *priv = dev->dev_private;
+       struct msm_ringbuffer *ring = submit->ring;
        int i;
 
        WARN_ON(!mutex_is_locked(&dev->struct_mutex));
@@ -478,7 +510,7 @@ void msm_gpu_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit,
 
        msm_gpu_hw_init(gpu);
 
-       list_add_tail(&submit->node, &gpu->submit_list);
+       list_add_tail(&submit->node, &ring->submits);
 
        msm_rd_dump_submit(submit);
 
@@ -565,7 +597,7 @@ int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev,
                const char *name, struct msm_gpu_config *config)
 {
        struct iommu_domain *iommu;
-       int ret;
+       int i, ret, nr_rings = config->nr_rings;
 
        if (WARN_ON(gpu->num_perfcntrs > ARRAY_SIZE(gpu->last_cntrs)))
                gpu->num_perfcntrs = ARRAY_SIZE(gpu->last_cntrs);
@@ -584,7 +616,6 @@ int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev,
        INIT_WORK(&gpu->retire_work, retire_worker);
        INIT_WORK(&gpu->recover_work, recover_worker);
 
-       INIT_LIST_HEAD(&gpu->submit_list);
 
        setup_timer(&gpu->hangcheck_timer, hangcheck_handler,
                        (unsigned long)gpu);
@@ -658,40 +689,61 @@ int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev,
                dev_info(drm->dev, "%s: no IOMMU, fallback to VRAM carveout!\n", name);
        }
 
-       /* Create ringbuffer: */
-       mutex_lock(&drm->struct_mutex);
-       gpu->rb = msm_ringbuffer_new(gpu, config->ringsz);
-       mutex_unlock(&drm->struct_mutex);
-       if (IS_ERR(gpu->rb)) {
-               ret = PTR_ERR(gpu->rb);
-               gpu->rb = NULL;
-               dev_err(drm->dev, "could not create ringbuffer: %d\n", ret);
-               goto fail;
+       if (nr_rings > ARRAY_SIZE(gpu->rb)) {
+               DRM_DEV_INFO_ONCE(drm->dev, "Only creating %lu ringbuffers\n",
+                       ARRAY_SIZE(gpu->rb));
+               nr_rings = ARRAY_SIZE(gpu->rb);
+       }
+
+       /* Create ringbuffer(s): */
+       for (i = 0; i < nr_rings; i++) {
+               mutex_lock(&drm->struct_mutex);
+               gpu->rb[i] = msm_ringbuffer_new(gpu, i);
+               mutex_unlock(&drm->struct_mutex);
+
+               if (IS_ERR(gpu->rb[i])) {
+                       ret = PTR_ERR(gpu->rb[i]);
+                       dev_err(drm->dev,
+                               "could not create ringbuffer %d: %d\n", i, ret);
+                       goto fail;
+               }
        }
 
+       gpu->nr_rings = nr_rings;
+
        gpu->pdev = pdev;
        platform_set_drvdata(pdev, gpu);
 
+
        bs_init(gpu);
 
        return 0;
 
 fail:
+       for (i = 0; i < nr_rings; i++) {
+               msm_ringbuffer_destroy(gpu->rb[i]);
+               gpu->rb[i] = NULL;
+       }
+
        return ret;
 }
 
 void msm_gpu_cleanup(struct msm_gpu *gpu)
 {
+       int i;
+
        DBG("%s", gpu->name);
 
        WARN_ON(!list_empty(&gpu->active_list));
 
        bs_fini(gpu);
 
-       if (gpu->rb) {
-               if (gpu->rb_iova)
-                       msm_gem_put_iova(gpu->rb->bo, gpu->aspace);
-               msm_ringbuffer_destroy(gpu->rb);
+       for (i = 0; i < gpu->nr_rings; i++) {
+               if (gpu->rb[i]->iova)
+                       msm_gem_put_iova(gpu->rb[i]->bo, gpu->aspace);
+
+               msm_ringbuffer_destroy(gpu->rb[i]);
+               gpu->rb[i] = NULL;
        }
 
        if (gpu->fctx)
drivers/gpu/drm/msm/msm_gpu.h +20 -10
index ca07a21..c0e7c84 100644
@@ -33,7 +33,7 @@ struct msm_gpu_config {
        const char *irqname;
        uint64_t va_start;
        uint64_t va_end;
-       unsigned int ringsz;
+       unsigned int nr_rings;
 };
 
 /* So far, with hardware that I've seen to date, we can have:
@@ -57,9 +57,11 @@ struct msm_gpu_funcs {
        int (*pm_resume)(struct msm_gpu *gpu);
        void (*submit)(struct msm_gpu *gpu, struct msm_gem_submit *submit,
                        struct msm_file_private *ctx);
-       void (*flush)(struct msm_gpu *gpu);
+       void (*flush)(struct msm_gpu *gpu, struct msm_ringbuffer *ring);
        irqreturn_t (*irq)(struct msm_gpu *irq);
-       uint32_t (*last_fence)(struct msm_gpu *gpu);
+       uint32_t (*last_fence)(struct msm_gpu *gpu,
+                       struct msm_ringbuffer *ring);
+       struct msm_ringbuffer *(*active_ring)(struct msm_gpu *gpu);
        void (*recover)(struct msm_gpu *gpu);
        void (*destroy)(struct msm_gpu *gpu);
 #ifdef CONFIG_DEBUG_FS
@@ -86,9 +88,8 @@ struct msm_gpu {
        const struct msm_gpu_perfcntr *perfcntrs;
        uint32_t num_perfcntrs;
 
-       /* ringbuffer: */
-       struct msm_ringbuffer *rb;
-       uint64_t rb_iova;
+       struct msm_ringbuffer *rb[MSM_GPU_MAX_RINGS];
+       int nr_rings;
 
        /* list of GEM active objects: */
        struct list_head active_list;
@@ -126,10 +127,8 @@ struct msm_gpu {
 #define DRM_MSM_HANGCHECK_PERIOD 500 /* in ms */
 #define DRM_MSM_HANGCHECK_JIFFIES msecs_to_jiffies(DRM_MSM_HANGCHECK_PERIOD)
        struct timer_list hangcheck_timer;
-       uint32_t hangcheck_fence;
+       uint32_t hangcheck_fence[MSM_GPU_MAX_RINGS];
        struct work_struct recover_work;
-
-       struct list_head submit_list;
 };
 
 struct msm_gpu_drawqueue {
@@ -139,9 +138,20 @@ struct msm_gpu_drawqueue {
        struct list_head node;
 };
 
+/* It turns out that all targets use the same ringbuffer size */
+#define MSM_GPU_RINGBUFFER_SZ SZ_32K
+
 static inline bool msm_gpu_active(struct msm_gpu *gpu)
 {
-       return gpu->fctx->last_fence > gpu->funcs->last_fence(gpu);
+       int i;
+
+       for (i = 0; i < gpu->nr_rings; i++) {
+               if (gpu->rb[i]->last_fence >
+                       gpu->funcs->last_fence(gpu, gpu->rb[i]))
+                       return true;
+       }
+
+       return false;
 }
 
 /* Perf-Counters:
drivers/gpu/drm/msm/msm_ringbuffer.c +11 -8
index 67b34e0..10f1d948 100644
@@ -18,13 +18,13 @@
 #include "msm_ringbuffer.h"
 #include "msm_gpu.h"
 
-struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int size)
+struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int id)
 {
        struct msm_ringbuffer *ring;
        int ret;
 
-       if (WARN_ON(!is_power_of_2(size)))
-               return ERR_PTR(-EINVAL);
+       /* We assume everwhere that MSM_GPU_RINGBUFFER_SZ is a power of 2 */
+       BUILD_BUG_ON(!is_power_of_2(MSM_GPU_RINGBUFFER_SZ));
 
        ring = kzalloc(sizeof(*ring), GFP_KERNEL);
        if (!ring) {
@@ -33,7 +33,8 @@ struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int size)
        }
 
        ring->gpu = gpu;
-       ring->bo = msm_gem_new(gpu->dev, size, MSM_BO_WC);
+       ring->id = id;
+       ring->bo = msm_gem_new(gpu->dev, MSM_GPU_RINGBUFFER_SZ, MSM_BO_WC);
        if (IS_ERR(ring->bo)) {
                ret = PTR_ERR(ring->bo);
                ring->bo = NULL;
@@ -45,21 +46,23 @@ struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int size)
                ret = PTR_ERR(ring->start);
                goto fail;
        }
-       ring->end   = ring->start + (size / 4);
+       ring->end   = ring->start + (MSM_GPU_RINGBUFFER_SZ >> 2);
        ring->cur   = ring->start;
 
-       ring->size = size;
+       INIT_LIST_HEAD(&ring->submits);
 
        return ring;
 
 fail:
-       if (ring)
-               msm_ringbuffer_destroy(ring);
+       msm_ringbuffer_destroy(ring);
        return ERR_PTR(ret);
 }
 
 void msm_ringbuffer_destroy(struct msm_ringbuffer *ring)
 {
+       if (IS_ERR_OR_NULL(ring))
+               return;
+
        if (ring->bo) {
                msm_gem_put_vaddr(ring->bo);
                drm_gem_object_unreference_unlocked(ring->bo);
drivers/gpu/drm/msm/msm_ringbuffer.h +7 -3
index 6e0e104..c803364 100644
@@ -22,12 +22,17 @@
 
 struct msm_ringbuffer {
        struct msm_gpu *gpu;
-       int size;
+       int id;
        struct drm_gem_object *bo;
        uint32_t *start, *end, *cur;
+       uint64_t iova;
+       /* last_fence == completed_fence --> no pending work */
+       uint32_t last_fence;
+       uint32_t completed_fence;
+       struct list_head submits;
 };
 
-struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int size);
+struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int id);
 void msm_ringbuffer_destroy(struct msm_ringbuffer *ring);
 
 /* ringbuffer helpers (the parts that are same for a3xx/a2xx/z180..) */
--

1.9.1

Quoted 4 lines

_______________________________________________
Freedreno mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/freedreno

_______________________________________________
Freedreno mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/freedreno

A message that is not in the archive (sent off-list or not yet received): CAF6AEGtiw4yBEZyDhLrQhCFPHVPi9ukD4MkKfkT0_AvTtCeCfg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#17 lore raw

Re: [PATCH 10/13] drm/msm: Support multiple ringbuffers

Quoted 20 lines · Rob Clark
Quoted 10 lines · Jordan Crouse

Add the infrastructure to support the idea of multiple ringbuffers. Assign each ringbuffer an id and use that as an index for the various ring specific operations.

The biggest delta is to support legacy fences. Each fence gets its own sequence number but the legacy functions expect to use a unique integer. To handle this we return a unique identifer for each submission but map it to a specific ring/sequence under the covers. Newer users use a dma_fence pointer anyway so they don't care about the actual sequence ID or ring.

So, WAIT_FENCE is alive and well, and useful since it avoids the overhead of creating a 'struct file', but it is only used within a single pipe_context (or at least situations where we know which ctx the seqno fence applies to). It seems like it would be simpler if we just introduced a ctx-id in all the ioctls (SUBMIT and WAIT_FENCE) that take a uint fence. Then I think we don't need hashtable fancyness.

Quoted 13 lines

Also, one thing I was thinking of is that some-day we might want to make SUBMIT non-blocking when there is a dependency on a fence from a different ring. (Ie. queue it up but don't write cmds into rb yet.) Which means we'd need multiple fence timelines per priority-level rb. Which brings me back to wanting a CREATE_CTX type of ioctl. (And I guess DESTROY_CTX.) We could make these simple stubs for now, ie. CREATE_CTX just returns the priority level back, and not really have any separate "context" object on the kernel side for now. This wouldn't change the implementation much from what you have, but I think that gives us some flexibility to later on actually let us have multiple contexts at a given priority level which don't block each other for submits that are still pending on some fence, without another UABI change.

Sure. My motivation here was to mostly avoid making that decision because I know from experience once we start going down that path we end up using the context ID for everything and we end up re-spinning a bunch of APIs.

But I agree that the context concept is our inevitable future - I've already posted one set of patches for "draw queues" (which will soon be bravely renamed as submit queues). I think thats the way we want to go because as you said, there is a 100% chance we'll go for asynchronous submissions in the very near future.

That said, there is a bit of added complexity for per-queue fences - namely, we need to move the per-ring fence value in the memptrs to a per-queue value. This probably isn't a huge deal (an extra page of memory would give us up to 1024 queues to work with globally) but I get itchy every time an arbitrary limit is introduced no matter how reasonable it might be.

Jordan

Signature · 6 lines
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project
_______________________________________________
Freedreno mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/freedreno
#18 lore raw

Re: [Freedreno] [PATCH 10/13] drm/msm: Support multiple ringbuffers

Quoted 45 lines · Jordan Crouse
Quoted 20 lines · Rob Clark
Quoted 10 lines · Jordan Crouse

Add the infrastructure to support the idea of multiple ringbuffers. Assign each ringbuffer an id and use that as an index for the various ring specific operations.

The biggest delta is to support legacy fences. Each fence gets its own sequence number but the legacy functions expect to use a unique integer. To handle this we return a unique identifer for each submission but map it to a specific ring/sequence under the covers. Newer users use a dma_fence pointer anyway so they don't care about the actual sequence ID or ring.

So, WAIT_FENCE is alive and well, and useful since it avoids the overhead of creating a 'struct file', but it is only used within a single pipe_context (or at least situations where we know which ctx the seqno fence applies to). It seems like it would be simpler if we just introduced a ctx-id in all the ioctls (SUBMIT and WAIT_FENCE) that take a uint fence. Then I think we don't need hashtable fancyness.

Quoted 13 lines

Also, one thing I was thinking of is that some-day we might want to make SUBMIT non-blocking when there is a dependency on a fence from a different ring. (Ie. queue it up but don't write cmds into rb yet.) Which means we'd need multiple fence timelines per priority-level rb. Which brings me back to wanting a CREATE_CTX type of ioctl. (And I guess DESTROY_CTX.) We could make these simple stubs for now, ie. CREATE_CTX just returns the priority level back, and not really have any separate "context" object on the kernel side for now. This wouldn't change the implementation much from what you have, but I think that gives us some flexibility to later on actually let us have multiple contexts at a given priority level which don't block each other for submits that are still pending on some fence, without another UABI change.

Sure. My motivation here was to mostly avoid making that decision because I know from experience once we start going down that path we end up using the context ID for everything and we end up re-spinning a bunch of APIs.

But I agree that the context concept is our inevitable future - I've already posted one set of patches for "draw queues" (which will soon be bravely renamed as submit queues). I think thats the way we want to go because as you said, there is a 100% chance we'll go for asynchronous submissions in the very near future.

Quoted 6 lines

That said, there is a bit of added complexity for per-queue fences - namely, we need to move the per-ring fence value in the memptrs to a per-queue value. This probably isn't a huge deal (an extra page of memory would give us up to 1024 queues to work with globally) but I get itchy every time an arbitrary limit is introduced no matter how reasonable it might be.

FWIW, we have contexts in amdgpu and it makes a lot of things easier when dealing with dependencies. Feel free to browse our implementation for ideas.

Alex
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#19 lore raw

Re: [Freedreno] [PATCH 10/13] drm/msm: Support multiple ringbuffers

Quoted 53 lines · Alex Deucher
Quoted 45 lines · Jordan Crouse
Quoted 20 lines · Rob Clark
Quoted 10 lines · Jordan Crouse

Add the infrastructure to support the idea of multiple ringbuffers. Assign each ringbuffer an id and use that as an index for the various ring specific operations.

The biggest delta is to support legacy fences. Each fence gets its own sequence number but the legacy functions expect to use a unique integer. To handle this we return a unique identifer for each submission but map it to a specific ring/sequence under the covers. Newer users use a dma_fence pointer anyway so they don't care about the actual sequence ID or ring.

So, WAIT_FENCE is alive and well, and useful since it avoids the overhead of creating a 'struct file', but it is only used within a single pipe_context (or at least situations where we know which ctx the seqno fence applies to). It seems like it would be simpler if we just introduced a ctx-id in all the ioctls (SUBMIT and WAIT_FENCE) that take a uint fence. Then I think we don't need hashtable fancyness.

Quoted 13 lines

Also, one thing I was thinking of is that some-day we might want to make SUBMIT non-blocking when there is a dependency on a fence from a different ring. (Ie. queue it up but don't write cmds into rb yet.) Which means we'd need multiple fence timelines per priority-level rb. Which brings me back to wanting a CREATE_CTX type of ioctl. (And I guess DESTROY_CTX.) We could make these simple stubs for now, ie. CREATE_CTX just returns the priority level back, and not really have any separate "context" object on the kernel side for now. This wouldn't change the implementation much from what you have, but I think that gives us some flexibility to later on actually let us have multiple contexts at a given priority level which don't block each other for submits that are still pending on some fence, without another UABI change.

Sure. My motivation here was to mostly avoid making that decision because I know from experience once we start going down that path we end up using the context ID for everything and we end up re-spinning a bunch of APIs.

But I agree that the context concept is our inevitable future - I've already posted one set of patches for "draw queues" (which will soon be bravely renamed as submit queues). I think thats the way we want to go because as you said, there is a 100% chance we'll go for asynchronous submissions in the very near future.

Quoted 6 lines

That said, there is a bit of added complexity for per-queue fences - namely, we need to move the per-ring fence value in the memptrs to a per-queue value. This probably isn't a huge deal (an extra page of memory would give us up to 1024 queues to work with globally) but I get itchy every time an arbitrary limit is introduced no matter how reasonable it might be.

Quoted 3 lines

FWIW, we have contexts in amdgpu and it makes a lot of things easier when dealing with dependencies. Feel free to browse our implementation for ideas.

Same on i915, we use contexts (not batches) as the scheduling entity. Think of them like threads on a cpu, at least in our case. And we can dynamically allocate as many as we need (well until we run out of memory of course), we can even swap them in/out :-)
-Daniel

Signature · 3 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
#20 lore raw

PATCH 11/13 drm/msm: Shadow current pointer in the ring until command is complete

In reply to Jordan Crouse

Add a shadow pointer to track the current command being written into the ring. Don't commit it as 'cur' until the command is submitted. Because 'cur' is used to construct the software copy of the wptr this ensures that somebody peeking in on the ring doesn't assume that a command is inflight while it is being written. This isn't a huge deal with a single ring (though technically the hangcheck could assume the system is prematurely busy when it isn't) but it will be rather important for preemption where the decision to preempt is based on a non-empty ringbuffer. Without a shadow an aggressive preemption scheme could assume that the ringbuffer is non empty and switch to it before the CPU is done writing the command and boom.

Even though preemption won't be supported for all targets because of the way the code is organized it is simpler to make this generic for all targets. The extra load for non-preemption targets should be minimal.

---
 drivers/gpu/drm/msm/adreno/adreno_gpu.c |  9 +++++++--
 drivers/gpu/drm/msm/msm_ringbuffer.c    |  1 +
 drivers/gpu/drm/msm/msm_ringbuffer.h    | 12 ++++++++----
 3 files changed, 16 insertions(+), 6 deletions(-)
drivers/gpu/drm/msm/adreno/adreno_gpu.c +7 -2
index 6b7114d..00f8436 100644
@@ -79,6 +79,7 @@ int adreno_hw_init(struct msm_gpu *gpu)
 		}
 
 		ring->cur = ring->start;
+		ring->next = ring->start;
 
 		/* reset completed fence seqno: */
 		adreno_gpu->memptrs->fence[ring->id] = ring->completed_fence;
@@ -239,12 +240,15 @@ void adreno_flush(struct msm_gpu *gpu, struct msm_ringbuffer *ring)
 	struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
 	uint32_t wptr;
 
+	/* Copy the shadow to the actual register */
+	ring->cur = ring->next;
+
 	/*
 	 * Mask wptr value that we calculate to fit in the HW range. This is
 	 * to account for the possibility that the last command fit exactly into
 	 * the ringbuffer and rb->next hasn't wrapped to zero yet
 	 */
-	wptr = get_wptr(ring) % (MSM_GPU_RINGBUFFER_SZ >> 2);
+	wptr = (ring->cur - ring->start) % (MSM_GPU_RINGBUFFER_SZ >> 2);
 
 	/* ensure writes to ringbuffer have hit system memory: */
 	mb();
@@ -357,7 +361,8 @@ static uint32_t ring_freewords(struct msm_ringbuffer *ring)
 {
 	struct adreno_gpu *adreno_gpu = to_adreno_gpu(ring->gpu);
 	uint32_t size = MSM_GPU_RINGBUFFER_SZ >> 2;
-	uint32_t wptr = get_wptr(ring);
+	/* Use ring->next to calculate free size */
+	uint32_t wptr = ring->next - ring->start;
 	uint32_t rptr = get_rptr(adreno_gpu, ring);
 	return (rptr + (size - 1) - wptr) % size;
 }
drivers/gpu/drm/msm/msm_ringbuffer.c +1 -0
index 10f1d948..c5b21cf 100644
@@ -47,6 +47,7 @@ struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int id)
 		goto fail;
 	}
 	ring->end   = ring->start + (MSM_GPU_RINGBUFFER_SZ >> 2);
+	ring->next  = ring->start;
 	ring->cur   = ring->start;
 
 	INIT_LIST_HEAD(&ring->submits);
drivers/gpu/drm/msm/msm_ringbuffer.h +8 -4
index c803364..0dafe00 100644
@@ -24,7 +24,7 @@ struct msm_ringbuffer {
 	struct msm_gpu *gpu;
 	int id;
 	struct drm_gem_object *bo;
-	uint32_t *start, *end, *cur;
+	uint32_t *start, *end, *cur, *next;
 	uint64_t iova;
 	/* last_fence == completed_fence --> no pending work */
 	uint32_t last_fence;
@@ -40,9 +40,13 @@ struct msm_ringbuffer {
 static inline void
 OUT_RING(struct msm_ringbuffer *ring, uint32_t data)
 {
-	if (ring->cur == ring->end)
-		ring->cur = ring->start;
-	*(ring->cur++) = data;
+	/*
+	 * ring->next points to the current command being written - it won't be
+	 * committed as ring->cur until the flush
+	 */
+	if (ring->next == ring->end)
+		ring->next = ring->start;
+	*(ring->next++) = data;
 }
 
 #endif /* __MSM_RINGBUFFER_H__ */
Signature · 1 line
1.9.1

A message that is not in the archive (sent off-list or not yet received): 1494275709-25782-1-git-send-email-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org

#22 lore raw

PATCH 12/13 drm/msm: Make the value of RB_CNTL (almost) generic

We use a global ringbuffer size and block size for all targets and at least for 5XX preemption we need to know the value the RB_CNTL in several locations so it makes sense to caculate it once and use it everywhere.

The only monkey wrench is that we need to disable the RPTR shadow for A430 targets but that only needs to be done once and doesn't affect A5XX so we can or in the value at init time.

---
 drivers/gpu/drm/msm/adreno/adreno_gpu.c | 12 +++++++-----
 drivers/gpu/drm/msm/msm_gpu.h           |  5 +++++
 2 files changed, 12 insertions(+), 5 deletions(-)
drivers/gpu/drm/msm/adreno/adreno_gpu.c +7 -5
index 00f8436..c106e98 100644
@@ -21,7 +21,6 @@
 #include "msm_gem.h"
 #include "msm_mmu.h"
 
-#define RB_BLKSIZE 32
 
 int adreno_get_param(struct msm_gpu *gpu, uint32_t param, uint64_t *value)
 {
@@ -86,11 +85,14 @@ int adreno_hw_init(struct msm_gpu *gpu)
 		adreno_gpu->memptrs->rptr[ring->id]  = 0;
 	}
 
-	/* Setup REG_CP_RB_CNTL: */
+	/*
+	 * Setup REG_CP_RB_CNTL.  The same value is used across targets (with
+	 * the excpetion of A430 that disables the RPTR shadow) - the cacluation
+	 * for the ringbuffer size and block size is moved to msm_gpu.h for the
+	 * pre-processor to deal with and the A430 variant is ORed in here
+	 */
 	adreno_gpu_write(adreno_gpu, REG_ADRENO_CP_RB_CNTL,
-		/* size is log2(quad-words): */
-		AXXX_CP_RB_CNTL_BUFSZ(ilog2(MSM_GPU_RINGBUFFER_SZ / 8)) |
-		AXXX_CP_RB_CNTL_BLKSZ(ilog2(RB_BLKSIZE / 8)) |
+		MSM_GPU_RB_CNTL_DEFAULT |
 		(adreno_is_a430(adreno_gpu) ? AXXX_CP_RB_CNTL_NO_UPDATE : 0));
 
 	/* Setup ringbuffer address - use ringbuffer[0] for GPU init */
drivers/gpu/drm/msm/msm_gpu.h +5 -0
index c0e7c84..85b4602 100644
@@ -140,6 +140,11 @@ struct msm_gpu_drawqueue {
 
 /* It turns out that all targets use the same ringbuffer size */
 #define MSM_GPU_RINGBUFFER_SZ SZ_32K
+#define MSM_GPU_RINGBUFFER_BLKSIZE 32
+
+#define MSM_GPU_RB_CNTL_DEFAULT \
+		(AXXX_CP_RB_CNTL_BUFSZ(ilog2(MSM_GPU_RINGBUFFER_SZ / 8)) | \
+		AXXX_CP_RB_CNTL_BLKSZ(ilog2(MSM_GPU_RINGBUFFER_BLKSIZE / 8)))
 
 static inline bool msm_gpu_active(struct msm_gpu *gpu)
 {
Signature · 6 lines
1.9.1

_______________________________________________
Freedreno mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/freedreno
#23 lore raw

PATCH 13/13 drm/msm: Implement preemption for A5XX targets

In reply to Jordan Crouse

Implement preemption for A5XX targets - this allows multiple ringbuffers for different priorities with automatic preemption of a lower priority ringbuffer if a higher one is ready.

---
 drivers/gpu/drm/msm/Makefile              |   1 +
 drivers/gpu/drm/msm/adreno/a5xx_gpu.c     | 176 +++++++++++++++-
 drivers/gpu/drm/msm/adreno/a5xx_gpu.h     | 101 ++++++++-
 drivers/gpu/drm/msm/adreno/a5xx_preempt.c | 332 ++++++++++++++++++++++++++++++
 drivers/gpu/drm/msm/adreno/adreno_gpu.c   |  14 +-
 drivers/gpu/drm/msm/adreno/adreno_gpu.h   |   7 +-
 drivers/gpu/drm/msm/msm_drv.h             |   2 +-
 drivers/gpu/drm/msm/msm_ringbuffer.c      |   1 +
 drivers/gpu/drm/msm/msm_ringbuffer.h      |   1 +
 9 files changed, 618 insertions(+), 17 deletions(-)
 create mode 100644 drivers/gpu/drm/msm/adreno/a5xx_preempt.c
drivers/gpu/drm/msm/Makefile +1 -0
index 5241ac8..a6945c8 100644
@@ -8,6 +8,7 @@ msm-y := \
 	adreno/a4xx_gpu.o \
 	adreno/a5xx_gpu.o \
 	adreno/a5xx_power.o \
+	adreno/a5xx_preempt.o \
 	hdmi/hdmi.o \
 	hdmi/hdmi_audio.o \
 	hdmi/hdmi_bridge.o \
drivers/gpu/drm/msm/adreno/a5xx_gpu.c +171 -5

271 lines not on this page · raw message

drivers/gpu/drm/msm/adreno/a5xx_gpu.h +100 -1

118 lines not on this page · raw message

drivers/gpu/drm/msm/adreno/a5xx_preempt.c new file +332 -0

332 lines not on this page · raw message

drivers/gpu/drm/msm/adreno/adreno_gpu.c +5 -9
index c106e98..d868879 100644
@@ -108,11 +108,6 @@ int adreno_hw_init(struct msm_gpu *gpu)
 	return 0;
 }
 
-static uint32_t get_wptr(struct msm_ringbuffer *ring)
-{
-	return ring->cur - ring->start;
-}
-
 /* Use this helper to read rptr, since a430 doesn't update rptr in memory */
 static uint32_t get_rptr(struct adreno_gpu *adreno_gpu,
 		struct msm_ringbuffer *ring)
@@ -183,7 +178,7 @@ void adreno_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit,
 		case MSM_SUBMIT_CMD_BUF:
 			OUT_PKT3(ring, adreno_is_a430(adreno_gpu) ?
 				CP_INDIRECT_BUFFER_PFE : CP_INDIRECT_BUFFER_PFD, 2);
-			OUT_RING(ring, submit->cmd[i].iova);
+			OUT_RING(ring, lower_32_bits(submit->cmd[i].iova));
 			OUT_RING(ring, submit->cmd[i].size);
 			OUT_PKT2(ring);
 			break;
@@ -250,7 +245,7 @@ void adreno_flush(struct msm_gpu *gpu, struct msm_ringbuffer *ring)
 	 * to account for the possibility that the last command fit exactly into
 	 * the ringbuffer and rb->next hasn't wrapped to zero yet
 	 */
-	wptr = (ring->cur - ring->start) % (MSM_GPU_RINGBUFFER_SZ >> 2);
+	wptr = get_wptr(ring);
 
 	/* ensure writes to ringbuffer have hit system memory: */
 	mb();
@@ -268,8 +263,9 @@ bool adreno_idle(struct msm_gpu *gpu, struct msm_ringbuffer *ring)
 		return true;
 
 	/* TODO maybe we need to reset GPU here to recover from hang? */
-	DRM_ERROR("%s: timeout waiting to drain ringbuffer %d!\n", gpu->name,
-		ring->id);
+	DRM_ERROR("%s: timeout waiting to drain ringbuffer %d rptr/wptr = %X/%X\n",
+		gpu->name, ring->id, get_rptr(adreno_gpu, ring), wptr);
+
 	return false;
 }
 
drivers/gpu/drm/msm/adreno/adreno_gpu.h +6 -1
index 9e78e49..885360f 100644
@@ -2,7 +2,7 @@
  * Copyright (C) 2013 Red Hat
  * Author: Rob Clark <[email protected]>
  *
- * Copyright (c) 2014 The Linux Foundation. All rights reserved.
+ * Copyright (c) 2014,2017 The Linux Foundation. All rights reserved.
  *
  * This program is free software; you can redistribute it and/or modify it
  * under the terms of the GNU General Public License version 2 as published by
@@ -333,6 +333,11 @@ static inline void adreno_gpu_write64(struct adreno_gpu *gpu,
 	adreno_gpu_write(gpu, hi, upper_32_bits(data));
 }
 
+static inline uint32_t get_wptr(struct msm_ringbuffer *ring)
+{
+	return (ring->cur - ring->start) % (MSM_GPU_RINGBUFFER_SZ >> 2);
+}
+
 /*
  * Given a register and a count, return a value to program into
  * REG_CP_PROTECT_REG(n) - this will block both reads and writes for _len
drivers/gpu/drm/msm/msm_drv.h +1 -1
index bbf7d3d..b003898 100644
@@ -76,7 +76,7 @@ struct msm_vblank_ctrl {
 	spinlock_t lock;
 };
 
-#define MSM_GPU_MAX_RINGS 1
+#define MSM_GPU_MAX_RINGS 4
 
 struct msm_drm_private {
 
drivers/gpu/drm/msm/msm_ringbuffer.c +1 -0
index c5b21cf..40206ef 100644
@@ -51,6 +51,7 @@ struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int id)
 	ring->cur   = ring->start;
 
 	INIT_LIST_HEAD(&ring->submits);
+	spin_lock_init(&ring->lock);
 
 	return ring;
 
drivers/gpu/drm/msm/msm_ringbuffer.h +1 -0
index 0dafe00..edea3d42 100644
@@ -29,6 +29,7 @@ struct msm_ringbuffer {
 	/* last_fence == completed_fence --> no pending work */
 	uint32_t last_fence;
 	uint32_t completed_fence;
+	spinlock_t lock;
 	struct list_head submits;
 };
 
Signature · 1 line
1.9.1