Skip to contentSkip to conversation

Add support for context priorities

PATCH In review Andres Rodriguez 2 participants 5 messages Jul 7, 2017 2 messages not in the archive DRM drivers

lore
Share & tools
lore
#0 lore raw

cover Add support for context priorities

As requested, for validation of the kernel series.

Andres Rodriguez (2):
headers: sync amdgpu_drm.h with the kernel
amdgpu: implement context priority for amdgpu_cs_ctx_create2 v3

 amdgpu/amdgpu-symbol-check |  1 +
 amdgpu/amdgpu.h            | 17 +++++++++++++++--
 amdgpu/amdgpu_cs.c         | 17 +++++++++++++----
 include/drm/amdgpu_drm.h   | 31 ++++++++++++++++++++++++++++++-
 4 files changed, 59 insertions(+), 7 deletions(-)
Signature · 6 lines
2.9.3

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 20170707192839.17582-1-andresx7-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#2 lore raw

PATCH 1/2 headers: sync amdgpu_drm.h with the kernel

Generated using make headers_install from:
[TODO: Pending kernel patches]

---
 include/drm/amdgpu_drm.h | 31 ++++++++++++++++++++++++++++++-
 1 file changed, 30 insertions(+), 1 deletion(-)
include/drm/amdgpu_drm.h +30 -1
index d9aa4a3..da2ade6 100644
@@ -52,6 +52,7 @@ extern "C" {
 #define DRM_AMDGPU_GEM_USERPTR		0x11
 #define DRM_AMDGPU_WAIT_FENCES		0x12
 #define DRM_AMDGPU_VM			0x13
+#define DRM_AMDGPU_SCHED		0x14
 
 #define DRM_IOCTL_AMDGPU_GEM_CREATE	DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_GEM_CREATE, union drm_amdgpu_gem_create)
 #define DRM_IOCTL_AMDGPU_GEM_MMAP	DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_GEM_MMAP, union drm_amdgpu_gem_mmap)
@@ -67,6 +68,7 @@ extern "C" {
 #define DRM_IOCTL_AMDGPU_GEM_USERPTR	DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_GEM_USERPTR, struct drm_amdgpu_gem_userptr)
 #define DRM_IOCTL_AMDGPU_WAIT_FENCES	DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_WAIT_FENCES, union drm_amdgpu_wait_fences)
 #define DRM_IOCTL_AMDGPU_VM		DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_VM, union drm_amdgpu_vm)
+#define DRM_IOCTL_AMDGPU_SCHED		DRM_IOW(DRM_COMMAND_BASE + DRM_AMDGPU_SCHED, union drm_amdgpu_sched)
 
 #define AMDGPU_GEM_DOMAIN_CPU		0x1
 #define AMDGPU_GEM_DOMAIN_GTT		0x2
@@ -162,13 +164,22 @@ union drm_amdgpu_bo_list {
 /* unknown cause */
 #define AMDGPU_CTX_UNKNOWN_RESET	3
 
+/* Context priority level */
+#define AMDGPU_CTX_PRIORITY_UNSET       -2048
+#define AMDGPU_CTX_PRIORITY_LOW_HW      -1023
+#define AMDGPU_CTX_PRIORITY_LOW_SW      -512
+#define AMDGPU_CTX_PRIORITY_NORMAL      0
+/* Selecting a priority above NORMAL requires CAP_SYS_NICE or DRM_MASTER */
+#define AMDGPU_CTX_PRIORITY_HIGH_SW     512
+#define AMDGPU_CTX_PRIORITY_HIGH_HW     1023
+
 struct drm_amdgpu_ctx_in {
 	/** AMDGPU_CTX_OP_* */
 	__u32	op;
 	/** For future use, no flags defined so far */
 	__u32	flags;
 	__u32	ctx_id;
-	__u32	_pad;
+	__s32	priority;
 };
 
 union drm_amdgpu_ctx_out {
@@ -212,6 +223,21 @@ union drm_amdgpu_vm {
 	struct drm_amdgpu_vm_out out;
 };
 
+/* sched ioctl */
+#define AMDGPU_SCHED_OP_PROCESS_PRIORITY_OVERRIDE	1
+
+struct drm_amdgpu_sched_in {
+	/* AMDGPU_SCHED_OP_* */
+	__u32	op;
+	__u32	fd;
+	__s32	priority;
+	__u32	flags;
+};
+
+union drm_amdgpu_sched {
+	struct drm_amdgpu_sched_in in;
+};
+
 /*
  * This is not a reliable API and you should expect it to fail for any
  * number of reasons and have fallback path that do not use userptr to
@@ -764,6 +790,7 @@ struct drm_amdgpu_info_device {
 	__u64 max_memory_clock;
 	/* cu information */
 	__u32 cu_active_number;
+	/* NOTE: cu_ao_mask is INVALID, DON'T use it */
 	__u32 cu_ao_mask;
 	__u32 cu_bitmap[4][4];
 	/** Render backend pipe mask. One render backend is CB+DB. */
@@ -818,6 +845,8 @@ struct drm_amdgpu_info_device {
 	/* max gs wavefront per vgt*/
 	__u32 max_gs_waves_per_vgt;
 	__u32 _pad1;
+	/* always on cu bitmap */
+	__u32 cu_ao_bitmap[4][4];
 };
 
 struct drm_amdgpu_info_hw_ip {
Signature · 6 lines
2.9.3

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
#3 lore raw

PATCH 2/2 amdgpu: implement context priority for amdgpu_cs_ctx_create2 v3

In reply to a message not in the archive

Add a new context creation function that allows specifying the context priority.

A high priority context has the potential of starving lower priority contexts. The current kernel driver implementation only allows only the root user to allocate high priority contexts.

v2: corresponding changes for kernel patch v2
v3: Swap enum values of _NORMAL/_HIGH for backwards compatibility
Fixed 'make check' symbol error

---
 amdgpu/amdgpu-symbol-check |  1 +
 amdgpu/amdgpu.h            | 17 +++++++++++++++--
 amdgpu/amdgpu_cs.c         | 17 +++++++++++++----
 3 files changed, 29 insertions(+), 6 deletions(-)
amdgpu/amdgpu-symbol-check +1 -0
index 81ef9b4..64251aa 100755
@@ -27,6 +27,7 @@ amdgpu_bo_wait_for_idle
 amdgpu_create_bo_from_user_mem
 amdgpu_cs_create_semaphore
 amdgpu_cs_ctx_create
+amdgpu_cs_ctx_create2
 amdgpu_cs_ctx_free
 amdgpu_cs_destroy_semaphore
 amdgpu_cs_query_fence_status
amdgpu/amdgpu.h +15 -2
index 1901fa8..8ac866e 100644
@@ -798,8 +798,9 @@ int amdgpu_bo_list_update(amdgpu_bo_list_handle handle,
  * context will always be executed in order (first come, first serve).
  *
  *
- * \param   dev	    - \c [in] Device handle. See #amdgpu_device_initialize()
- * \param   context - \c [out] GPU Context handle
+ * \param   dev      - \c [in] Device handle. See #amdgpu_device_initialize()
+ * \param   priority - \c [in] Context creation flags. See AMDGPU_CTX_PRIORITY_*
+ * \param   context  - \c [out] GPU Context handle
  *
  * \return   0 on success\n
  *          <0 - Negative POSIX Error code
@@ -807,6 +808,18 @@ int amdgpu_bo_list_update(amdgpu_bo_list_handle handle,
  * \sa amdgpu_cs_ctx_free()
  *
 */
+int amdgpu_cs_ctx_create2(amdgpu_device_handle dev,
+			 uint32_t priority,
+			 amdgpu_context_handle *context);
+/**
+ * Create GPU execution Context
+ *
+ * Refer to amdgpu_cs_ctx_create2 for full documentation. This call
+ * is missing the priority parameter.
+ *
+ * \sa amdgpu_cs_ctx_create2()
+ *
+*/
 int amdgpu_cs_ctx_create(amdgpu_device_handle dev,
 			 amdgpu_context_handle *context);
 
amdgpu/amdgpu_cs.c +13 -4
index 868eb7b..1af5e59 100644
@@ -46,13 +46,14 @@ static int amdgpu_cs_reset_sem(amdgpu_semaphore_handle sem);
 /**
  * Create command submission context
  *
- * \param   dev - \c [in] amdgpu device handle
- * \param   context - \c [out] amdgpu context handle
+ * \param   dev      - \c [in] Device handle. See #amdgpu_device_initialize()
+ * \param   priority - \c [in] Context creation flags. See AMDGPU_CTX_PRIORITY_*
+ * \param   context  - \c [out] GPU Context handle
  *
  * \return  0 on success otherwise POSIX Error code
 */
-int amdgpu_cs_ctx_create(amdgpu_device_handle dev,
-			 amdgpu_context_handle *context)
+int amdgpu_cs_ctx_create2(amdgpu_device_handle dev, uint32_t priority,
+							amdgpu_context_handle *context)
 {
 	struct amdgpu_context *gpu_context;
 	union drm_amdgpu_ctx args;
@@ -75,6 +76,8 @@ int amdgpu_cs_ctx_create(amdgpu_device_handle dev,
 	/* Create the context */
 	memset(&args, 0, sizeof(args));
 	args.in.op = AMDGPU_CTX_OP_ALLOC_CTX;
+	args.in.priority = priority;
+
 	r = drmCommandWriteRead(dev->fd, DRM_AMDGPU_CTX, &args, sizeof(args));
 	if (r)
 		goto error;
@@ -94,6 +97,12 @@ error:
 	return r;
 }
 
+int amdgpu_cs_ctx_create(amdgpu_device_handle dev,
+			 amdgpu_context_handle *context)
+{
+	return amdgpu_cs_ctx_create2(dev, AMDGPU_CTX_PRIORITY_NORMAL, context);
+}
+
 /**
  * Release command submission context
  *
Signature · 6 lines
2.9.3

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
#4 lore raw
In reply to a message not in the archive
Quoted 5 lines · Andres Rodriguez

As requested, for validation of the kernel series.

Andres Rodriguez (2):
headers: sync amdgpu_drm.h with the kernel
amdgpu: implement context priority for amdgpu_cs_ctx_create2 v3

Do you have a test app or mesa patches that actually use this that you could share?

Alex

Quoted 7 lines
 amdgpu/amdgpu-symbol-check |  1 +
 amdgpu/amdgpu.h            | 17 +++++++++++++++--
 amdgpu/amdgpu_cs.c         | 17 +++++++++++++----
 include/drm/amdgpu_drm.h   | 31 ++++++++++++++++++++++++++++++-
 4 files changed, 59 insertions(+), 7 deletions(-)
Quoted 7 lines

--
2.9.3

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): CADnq5_NDpnJbVVo+FL=9=edjpBe4NgJfXBH0zER4agFPPJ-vBA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#6 lore raw
Quoted 10 lines · Alex Deucher
Quoted 5 lines · Andres Rodriguez

As requested, for validation of the kernel series.

Andres Rodriguez (2):
headers: sync amdgpu_drm.h with the kernel
amdgpu: implement context priority for amdgpu_cs_ctx_create2 v3

Do you have a test app or mesa patches that actually use this that you could share?

Yeah.

The mesa portion is a little messy since the extension isn't registered yet. But it can be found on the wip-high-priority branch here:
https://github.com/lostgoat/mesa.git

For a test app you can download this other project (wip-high-priority branch as well):
https://github.com/lostgoat/Vulkan.git

Once everything is setup I run one session of the SSAO sample:
QUEUE_NUM=0 ./ssao

Then you can simultaneously run the computeparticles with or without high priority:
sudo QUEUE_NUM=0 nice -n 20 ./computeparticles -high-priority
Or
sudo QUEUE_NUM=0 nice -n 20 ./computeparticles

Note: QUEUE_NUM is required.

If you have any issues building/running these let me know.

For reference, following are the output of these apps on my system on an RX480:

ssao:
Max: 2.917ms Min: 2.583ms MaxH: 2.917ms MinH: 2.583ms Med: 2.806ms Avg: 2.797ms Frames: 58 Cur: 2.745ms

computeparticles with regular priority:
Max: 2.021ms Min: 0.058ms MaxH: 1.899ms MinH: 0.932ms Med: 1.187ms Avg: 1.236ms Frames: 3017 Cur: 1.339ms

computeparticles with high priority:
Max: 0.298ms Min: 0.054ms MaxH: 0.077ms MinH: 0.054ms Med: 0.062ms Avg: 0.061ms Frames: 5999 Cur: 0.063ms

Regards,
Andres

Quoted 17 lines

Alex

Quoted 14 lines
  amdgpu/amdgpu-symbol-check |  1 +
  amdgpu/amdgpu.h            | 17 +++++++++++++++--
  amdgpu/amdgpu_cs.c         | 17 +++++++++++++----
  include/drm/amdgpu_drm.h   | 31 ++++++++++++++++++++++++++++++-
  4 files changed, 59 insertions(+), 7 deletions(-)

--
2.9.3

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx