Skip to contentSkip to conversation

High priority usermode contexts

PATCH Superseded Andres Rodriguez 6 participants 14 messages Jan 3, 2017 – Jan 5, 2017 10 messages not in the archive

Subsystems (3)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 High priority usermode contexts

This patch series provides the initial APIs for high priority contexts.

The current implementation is based on top of the SW scheduler, there are no HW priorities set yet.

This doesn't provide the quality of service we need for VR. Further changes to enable HW priorities will be required.

For some performance data collected with multiple apps see:
https://lists.freedesktop.org/archives/amd-gfx/2016-December/004257.html

Test apps available here:
https://github.com/lostgoat/Vulkan

Related libdrm/radv patches to follow
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1483484076-10424-1-git-send-email-andresx7-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#2 lore raw

PATCH 1/1 drm/amdgpu: add flag for high priority contexts

Add a new context creation flag, AMDGPU_CTX_FLAG_HIGHPRIORITY. This flag results in the allocated context receiving a higher scheduler priority that other contexts system-wide.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c       | 24 ++++++++++++++++++------
 drivers/gpu/drm/amd/scheduler/gpu_scheduler.h |  1 +
 include/uapi/drm/amdgpu_drm.h                 |  3 ++-
 3 files changed, 21 insertions(+), 7 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +18 -6
index 400c66b..ce470e2 100644
@@ -25,11 +25,17 @@
 #include <drm/drmP.h>
 #include "amdgpu.h"
 
-static int amdgpu_ctx_init(struct amdgpu_device *adev, struct amdgpu_ctx *ctx)
+static int amdgpu_ctx_init(struct amdgpu_device *adev, int priority, struct amdgpu_ctx *ctx)
 {
 	unsigned i, j;
 	int r;
 
+	if (priority < 0 || priority >= AMD_SCHED_MAX_PRIORITY)
+		return -EINVAL;
+
+	if (priority == AMD_SCHED_PRIORITY_HIGH && !capable(CAP_SYS_ADMIN))
+		return -EACCES;
+
 	memset(ctx, 0, sizeof(*ctx));
 	ctx->adev = adev;
 	kref_init(&ctx->refcount);
@@ -51,7 +57,7 @@ static int amdgpu_ctx_init(struct amdgpu_device *adev, struct amdgpu_ctx *ctx)
 		struct amdgpu_ring *ring = adev->rings[i];
 		struct amd_sched_rq *rq;
 
-		rq = &ring->sched.sched_rq[AMD_SCHED_PRIORITY_NORMAL];
+		rq = &ring->sched.sched_rq[priority];
 		r = amd_sched_entity_init(&ring->sched, &ctx->rings[i].entity,
 					  rq, amdgpu_sched_jobs);
 		if (r)
@@ -90,11 +96,15 @@ static void amdgpu_ctx_fini(struct amdgpu_ctx *ctx)
 
 static int amdgpu_ctx_alloc(struct amdgpu_device *adev,
 			    struct amdgpu_fpriv *fpriv,
+			    int flags,
 			    uint32_t *id)
 {
 	struct amdgpu_ctx_mgr *mgr = &fpriv->ctx_mgr;
 	struct amdgpu_ctx *ctx;
-	int r;
+	int r, priority;
+
+	priority = flags & AMDGPU_CTX_FLAG_HIGHPRIORITY ?
+		AMD_SCHED_PRIORITY_HIGH : AMD_SCHED_PRIORITY_NORMAL;
 
 	ctx = kmalloc(sizeof(*ctx), GFP_KERNEL);
 	if (!ctx)
@@ -107,8 +117,9 @@ static int amdgpu_ctx_alloc(struct amdgpu_device *adev,
 		kfree(ctx);
 		return r;
 	}
+
 	*id = (uint32_t)r;
-	r = amdgpu_ctx_init(adev, ctx);
+	r = amdgpu_ctx_init(adev, priority, ctx);
 	if (r) {
 		idr_remove(&mgr->ctx_handles, *id);
 		*id = 0;
@@ -186,7 +197,7 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void *data,
 		     struct drm_file *filp)
 {
 	int r;
-	uint32_t id;
+	uint32_t id, flags;
 
 	union drm_amdgpu_ctx *args = data;
 	struct amdgpu_device *adev = dev->dev_private;
@@ -194,10 +205,11 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void *data,
 
 	r = 0;
 	id = args->in.ctx_id;
+	flags = args->in.flags;
 
 	switch (args->in.op) {
 	case AMDGPU_CTX_OP_ALLOC_CTX:
-		r = amdgpu_ctx_alloc(adev, fpriv, &id);
+		r = amdgpu_ctx_alloc(adev, fpriv, flags, &id);
 		args->out.alloc.ctx_id = id;
 		break;
 	case AMDGPU_CTX_OP_FREE_CTX:
drivers/gpu/drm/amd/scheduler/gpu_scheduler.h +1 -0
index d8dc681..2e458de 100644
@@ -108,6 +108,7 @@ struct amd_sched_backend_ops {
 
 enum amd_sched_priority {
 	AMD_SCHED_PRIORITY_KERNEL = 0,
+	AMD_SCHED_PRIORITY_HIGH,
 	AMD_SCHED_PRIORITY_NORMAL,
 	AMD_SCHED_MAX_PRIORITY
 };
include/uapi/drm/amdgpu_drm.h +2 -1
index 3961836..702332e 100644
@@ -160,10 +160,11 @@ union drm_amdgpu_bo_list {
 /* unknown cause */
 #define AMDGPU_CTX_UNKNOWN_RESET	3
 
+#define AMDGPU_CTX_FLAG_HIGHPRIORITY	(1 << 0)
 struct drm_amdgpu_ctx_in {
 	/** AMDGPU_CTX_OP_* */
 	__u32	op;
-	/** For future use, no flags defined so far */
+	/** AMDGPU_CTX_FLAG_* */
 	__u32	flags;
 	__u32	ctx_id;
 	__u32	_pad;
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1483484076-10424-2-git-send-email-andresx7-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#4 lore raw

Re: [PATCH] drm/amdgpu: add flag for high priority contexts

Quoted 110 lines · Andres Rodriguez

Add a new context creation flag, AMDGPU_CTX_FLAG_HIGHPRIORITY. This flag results in the allocated context receiving a higher scheduler priority that other contexts system-wide.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c       | 24 ++++++++++++++++++------
 drivers/gpu/drm/amd/scheduler/gpu_scheduler.h |  1 +
 include/uapi/drm/amdgpu_drm.h                 |  3 ++-
 3 files changed, 21 insertions(+), 7 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +18 -6
index 400c66b..ce470e2 100644
@@ -25,11 +25,17 @@
 #include <drm/drmP.h>
 #include "amdgpu.h"
 
-static int amdgpu_ctx_init(struct amdgpu_device *adev, struct amdgpu_ctx *ctx)
+static int amdgpu_ctx_init(struct amdgpu_device *adev, int priority, struct amdgpu_ctx *ctx)
 {
        unsigned i, j;
        int r;
 
+       if (priority < 0 || priority >= AMD_SCHED_MAX_PRIORITY)
+               return -EINVAL;
+
+       if (priority == AMD_SCHED_PRIORITY_HIGH && !capable(CAP_SYS_ADMIN))
+               return -EACCES;
+
        memset(ctx, 0, sizeof(*ctx));
        ctx->adev = adev;
        kref_init(&ctx->refcount);
@@ -51,7 +57,7 @@ static int amdgpu_ctx_init(struct amdgpu_device *adev, struct amdgpu_ctx *ctx)
                struct amdgpu_ring *ring = adev->rings[i];
                struct amd_sched_rq *rq;
 
-               rq = &ring->sched.sched_rq[AMD_SCHED_PRIORITY_NORMAL];
+               rq = &ring->sched.sched_rq[priority];
                r = amd_sched_entity_init(&ring->sched, &ctx->rings[i].entity,
                                          rq, amdgpu_sched_jobs);
                if (r)
@@ -90,11 +96,15 @@ static void amdgpu_ctx_fini(struct amdgpu_ctx *ctx)
 
 static int amdgpu_ctx_alloc(struct amdgpu_device *adev,
                            struct amdgpu_fpriv *fpriv,
+                           int flags,
                            uint32_t *id)
 {
        struct amdgpu_ctx_mgr *mgr = &fpriv->ctx_mgr;
        struct amdgpu_ctx *ctx;
-       int r;
+       int r, priority;
+
+       priority = flags & AMDGPU_CTX_FLAG_HIGHPRIORITY ?
+               AMD_SCHED_PRIORITY_HIGH : AMD_SCHED_PRIORITY_NORMAL;
 
        ctx = kmalloc(sizeof(*ctx), GFP_KERNEL);
        if (!ctx)
@@ -107,8 +117,9 @@ static int amdgpu_ctx_alloc(struct amdgpu_device *adev,
                kfree(ctx);
                return r;
        }
+
        *id = (uint32_t)r;
-       r = amdgpu_ctx_init(adev, ctx);
+       r = amdgpu_ctx_init(adev, priority, ctx);
        if (r) {
                idr_remove(&mgr->ctx_handles, *id);
                *id = 0;
@@ -186,7 +197,7 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void *data,
                     struct drm_file *filp)
 {
        int r;
-       uint32_t id;
+       uint32_t id, flags;
 
        union drm_amdgpu_ctx *args = data;
        struct amdgpu_device *adev = dev->dev_private;
@@ -194,10 +205,11 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void *data,
 
        r = 0;
        id = args->in.ctx_id;
+       flags = args->in.flags;
 
        switch (args->in.op) {
        case AMDGPU_CTX_OP_ALLOC_CTX:
-               r = amdgpu_ctx_alloc(adev, fpriv, &id);
+               r = amdgpu_ctx_alloc(adev, fpriv, flags, &id);
                args->out.alloc.ctx_id = id;
                break;
        case AMDGPU_CTX_OP_FREE_CTX:
drivers/gpu/drm/amd/scheduler/gpu_scheduler.h +1 -0
index d8dc681..2e458de 100644
@@ -108,6 +108,7 @@ struct amd_sched_backend_ops {
 
 enum amd_sched_priority {
        AMD_SCHED_PRIORITY_KERNEL = 0,
+       AMD_SCHED_PRIORITY_HIGH,
        AMD_SCHED_PRIORITY_NORMAL,
        AMD_SCHED_MAX_PRIORITY
 };
include/uapi/drm/amdgpu_drm.h +0 -0
index 3961836..702332e 100644

include/uapi/drm/amdgpu_drm.h union drm_amdgpu_bo_list { · line 163

Quoted 4 lines
include/uapi/drm/amdgpu_drm.h +1 -0
@@ -160,10 +160,11 @@ union drm_amdgpu_bo_list {
 /* unknown cause */
 #define AMDGPU_CTX_UNKNOWN_RESET       3
 
+#define AMDGPU_CTX_FLAG_HIGHPRIORITY   (1 << 0)

Would it be better to expose a priority level rather than a single flag? If we want to expose more than just normal/high in the future for example.

Alex

Quoted 11 lines
include/uapi/drm/amdgpu_drm.h +1 -2
 struct drm_amdgpu_ctx_in {
        /** AMDGPU_CTX_OP_* */
        __u32   op;
-       /** For future use, no flags defined so far */
+       /** AMDGPU_CTX_FLAG_* */
        __u32   flags;
        __u32   ctx_id;
        __u32   _pad;
--

2.7.4

Quoted 4 lines

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): CADnq5_Mt_-61h6DG77kqepp5psS2JGHJOA-mek513s5HgvtTDA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#6 lore raw

Re: [PATCH] drm/amdgpu: add flag for high priority contexts

I was thinking of that originally, but the allocation context already has a flags variable which allows us to preserve the IOCTL ABI.

We could reserve a few bits of the flags for a priority level instead if that sounds good?

Regards,
Andres

Quoted 127 lines · Alex Deucher
Quoted 7 lines · Andres Rodriguez

Add a new context creation flag, AMDGPU_CTX_FLAG_HIGHPRIORITY. This flag results in the allocated context receiving a higher scheduler priority that other contexts system-wide.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c       | 24

++++++++++++++++++------

Review

Quoted 5 lines
 drivers/gpu/drm/amd/scheduler/gpu_scheduler.h |  1 +
 include/uapi/drm/amdgpu_drm.h                 |  3 ++-
 3 files changed, 21 insertions(+), 7 deletions(-)
(quoted diff) +0 -0

b/drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c

drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c line 28

Quoted 8 lines

index 400c66b..ce470e2 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +0 -1
@@ -25,11 +25,17 @@
 #include <drm/drmP.h>
 #include "amdgpu.h"
 
-static int amdgpu_ctx_init(struct amdgpu_device *adev, struct

amdgpu_ctx *ctx)

drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c line 28

Quoted 1 line
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +1 -0
+static int amdgpu_ctx_init(struct amdgpu_device *adev, int priority,

struct amdgpu_ctx *ctx)

drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +4 -0
 {
        unsigned i, j;
        int r;
 
+       if (priority < 0 || priority >= AMD_SCHED_MAX_PRIORITY)
+               return -EINVAL;
+
+       if (priority == AMD_SCHED_PRIORITY_HIGH &&

!capable(CAP_SYS_ADMIN))

drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c static int amdgpu_ctx_init(struct amdgpu_device *adev,

Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +2 -0
+               return -EACCES;
+
        memset(ctx, 0, sizeof(*ctx));
        ctx->adev = adev;
        kref_init(&ctx->refcount);
@@ -51,7 +57,7 @@ static int amdgpu_ctx_init(struct amdgpu_device *adev,

struct amdgpu_ctx *ctx)

drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c static int amdgpu_ctx_init(struct amdgpu_device *adev, struct amdgpu_ctx *ctx)

Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +1 -1
                struct amdgpu_ring *ring = adev->rings[i];
                struct amd_sched_rq *rq;
 
-               rq = &ring->sched.sched_rq[AMD_SCHED_PRIORITY_NORMAL];
+               rq = &ring->sched.sched_rq[priority];
                r = amd_sched_entity_init(&ring->sched,

&ctx->rings[i].entity,

drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c static int amdgpu_ctx_alloc(struct amdgpu_device

Quoted 20 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +5 -1
                                          rq, amdgpu_sched_jobs);
                if (r)
@@ -90,11 +96,15 @@ static void amdgpu_ctx_fini(struct amdgpu_ctx *ctx)
 
 static int amdgpu_ctx_alloc(struct amdgpu_device *adev,
                            struct amdgpu_fpriv *fpriv,
+                           int flags,
                            uint32_t *id)
 {
        struct amdgpu_ctx_mgr *mgr = &fpriv->ctx_mgr;
        struct amdgpu_ctx *ctx;
-       int r;
+       int r, priority;
+
+       priority = flags & AMDGPU_CTX_FLAG_HIGHPRIORITY ?
+               AMD_SCHED_PRIORITY_HIGH : AMD_SCHED_PRIORITY_NORMAL;
 
        ctx = kmalloc(sizeof(*ctx), GFP_KERNEL);
        if (!ctx)
@@ -107,8 +117,9 @@ static int amdgpu_ctx_alloc(struct amdgpu_device

*adev,

drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c int amdgpu_ctx_ioctl(struct drm_device *dev, void

Quoted 11 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +2 -1
                kfree(ctx);
                return r;
        }
+
        *id = (uint32_t)r;
-       r = amdgpu_ctx_init(adev, ctx);
+       r = amdgpu_ctx_init(adev, priority, ctx);
        if (r) {
                idr_remove(&mgr->ctx_handles, *id);
                *id = 0;
@@ -186,7 +197,7 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void

*data,

drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c int amdgpu_ctx_ioctl(struct drm_device *dev, void

Quoted 9 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +1 -1
                     struct drm_file *filp)
 {
        int r;
-       uint32_t id;
+       uint32_t id, flags;
 
        union drm_amdgpu_ctx *args = data;
        struct amdgpu_device *adev = dev->dev_private;
@@ -194,10 +205,11 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void

*data,

Review

Quoted 13 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +2 -1
        r = 0;
        id = args->in.ctx_id;
+       flags = args->in.flags;
 
        switch (args->in.op) {
        case AMDGPU_CTX_OP_ALLOC_CTX:
-               r = amdgpu_ctx_alloc(adev, fpriv, &id);
+               r = amdgpu_ctx_alloc(adev, fpriv, flags, &id);
                args->out.alloc.ctx_id = id;
                break;
        case AMDGPU_CTX_OP_FREE_CTX:
(quoted diff) +0 -0

b/drivers/gpu/drm/amd/scheduler/gpu_scheduler.h

Review

Quoted 12 lines

index d8dc681..2e458de 100644

drivers/gpu/drm/amd/scheduler/gpu_scheduler.h +1 -0
@@ -108,6 +108,7 @@ struct amd_sched_backend_ops {
 
 enum amd_sched_priority {
        AMD_SCHED_PRIORITY_KERNEL = 0,
+       AMD_SCHED_PRIORITY_HIGH,
        AMD_SCHED_PRIORITY_NORMAL,
        AMD_SCHED_MAX_PRIORITY
 };
(quoted diff) +0 -0

b/include/uapi/drm/amdgpu_drm.h

Quoted 7 lines

index 3961836..702332e 100644

include/uapi/drm/amdgpu_drm.h +0 -0
@@ -160,10 +160,11 @@ union drm_amdgpu_bo_list {
 /* unknown cause */
 #define AMDGPU_CTX_UNKNOWN_RESET       3
 
Quoted 24 lines

include/uapi/drm/amdgpu_drm.h union drm_amdgpu_bo_list { · line 163

Quoted 1 line
include/uapi/drm/amdgpu_drm.h +1 -0
+#define AMDGPU_CTX_FLAG_HIGHPRIORITY   (1 << 0)

Would it be better to expose a priority level rather than a single flag? If we want to expose more than just normal/high in the future for example.

Alex

Quoted 15 lines
include/uapi/drm/amdgpu_drm.h +1 -2
 struct drm_amdgpu_ctx_in {
        /** AMDGPU_CTX_OP_* */
        __u32   op;
-       /** For future use, no flags defined so far */
+       /** AMDGPU_CTX_FLAG_* */
        __u32   flags;
        __u32   ctx_id;
        __u32   _pad;
--

2.7.4

_______________________________________________
amd-gfx mailing list [email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#7 lore raw

Re: [PATCH] drm/amdgpu: add flag for high priority contexts

Quoted 5 lines · Andres Rodriguez

I was thinking of that originally, but the allocation context already has a flags variable which allows us to preserve the IOCTL ABI.

We could reserve a few bits of the flags for a priority level instead if that sounds good?

We can also use _pad for something else.

Alex

Quoted 158 lines

Regards,
Andres

Quoted 151 lines · Alex Deucher
Quoted 122 lines · Andres Rodriguez

Add a new context creation flag, AMDGPU_CTX_FLAG_HIGHPRIORITY. This flag results in the allocated context receiving a higher scheduler priority that other contexts system-wide.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c       | 24
++++++++++++++++++------
 drivers/gpu/drm/amd/scheduler/gpu_scheduler.h |  1 +
 include/uapi/drm/amdgpu_drm.h                 |  3 ++-
 3 files changed, 21 insertions(+), 7 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +7 -1
index 400c66b..ce470e2 100644
@@ -25,11 +25,17 @@
 #include <drm/drmP.h>
 #include "amdgpu.h"
 
-static int amdgpu_ctx_init(struct amdgpu_device *adev, struct
 amdgpu_ctx *ctx)
+static int amdgpu_ctx_init(struct amdgpu_device *adev, int priority,
 struct amdgpu_ctx *ctx)
 {
        unsigned i, j;
        int r;
 
+       if (priority < 0 || priority >= AMD_SCHED_MAX_PRIORITY)
+               return -EINVAL;
+
+       if (priority == AMD_SCHED_PRIORITY_HIGH &&
 !capable(CAP_SYS_ADMIN))
+               return -EACCES;
+
        memset(ctx, 0, sizeof(*ctx));
        ctx->adev = adev;
        kref_init(&ctx->refcount);
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +1 -1
@@ -51,7 +57,7 @@ static int amdgpu_ctx_init(struct amdgpu_device *adev,
 struct amdgpu_ctx *ctx)
                struct amdgpu_ring *ring = adev->rings[i];
                struct amd_sched_rq *rq;
 
-               rq = &ring->sched.sched_rq[AMD_SCHED_PRIORITY_NORMAL];
+               rq = &ring->sched.sched_rq[priority];
                r = amd_sched_entity_init(&ring->sched,
 &ctx->rings[i].entity,
                                          rq, amdgpu_sched_jobs);
                if (r)
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +7 -2
@@ -90,11 +96,15 @@ static void amdgpu_ctx_fini(struct amdgpu_ctx *ctx)
 
 static int amdgpu_ctx_alloc(struct amdgpu_device *adev,
                            struct amdgpu_fpriv *fpriv,
+                           int flags,
                            uint32_t *id)
 {
        struct amdgpu_ctx_mgr *mgr = &fpriv->ctx_mgr;
        struct amdgpu_ctx *ctx;
-       int r;
+       int r, priority;
+
+       priority = flags & AMDGPU_CTX_FLAG_HIGHPRIORITY ?
+               AMD_SCHED_PRIORITY_HIGH : AMD_SCHED_PRIORITY_NORMAL;
 
        ctx = kmalloc(sizeof(*ctx), GFP_KERNEL);
        if (!ctx)
@@ -107,8 +117,9 @@ static int amdgpu_ctx_alloc(struct amdgpu_device
 *adev,
                kfree(ctx);
                return r;
        }
+
        *id = (uint32_t)r;
-       r = amdgpu_ctx_init(adev, ctx);
+       r = amdgpu_ctx_init(adev, priority, ctx);
        if (r) {
                idr_remove(&mgr->ctx_handles, *id);
                *id = 0;
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +1 -1
@@ -186,7 +197,7 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void
 *data,
                     struct drm_file *filp)
 {
        int r;
-       uint32_t id;
+       uint32_t id, flags;
 
        union drm_amdgpu_ctx *args = data;
        struct amdgpu_device *adev = dev->dev_private;
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +2 -1
@@ -194,10 +205,11 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void
 *data,
 
        r = 0;
        id = args->in.ctx_id;
+       flags = args->in.flags;
 
        switch (args->in.op) {
        case AMDGPU_CTX_OP_ALLOC_CTX:
-               r = amdgpu_ctx_alloc(adev, fpriv, &id);
+               r = amdgpu_ctx_alloc(adev, fpriv, flags, &id);
                args->out.alloc.ctx_id = id;
                break;

case AMDGPU_CTX_OP_FREE_CTX:

drivers/gpu/drm/amd/scheduler/gpu_scheduler.h +1 -0
index d8dc681..2e458de 100644
@@ -108,6 +108,7 @@ struct amd_sched_backend_ops {
 
 enum amd_sched_priority {
        AMD_SCHED_PRIORITY_KERNEL = 0,
+       AMD_SCHED_PRIORITY_HIGH,
        AMD_SCHED_PRIORITY_NORMAL,
        AMD_SCHED_MAX_PRIORITY
 };
include/uapi/drm/amdgpu_drm.h +0 -0
index 3961836..702332e 100644

include/uapi/drm/amdgpu_drm.h union drm_amdgpu_bo_list { · line 163

Quoted 4 lines
include/uapi/drm/amdgpu_drm.h +1 -0
@@ -160,10 +160,11 @@ union drm_amdgpu_bo_list {
 /* unknown cause */
 #define AMDGPU_CTX_UNKNOWN_RESET       3
 
+#define AMDGPU_CTX_FLAG_HIGHPRIORITY   (1 << 0)

Would it be better to expose a priority level rather than a single flag? If we want to expose more than just normal/high in the future for example.

Alex

Quoted 15 lines
include/uapi/drm/amdgpu_drm.h +1 -2
 struct drm_amdgpu_ctx_in {
        /** AMDGPU_CTX_OP_* */
        __u32   op;
-       /** For future use, no flags defined so far */
+       /** AMDGPU_CTX_FLAG_* */
        __u32   flags;
        __u32   ctx_id;
        __u32   _pad;
--

2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): CADnq5_Mu4t4i9prGhZakMdd3cFB5NRWE6zh0zFp05T1Z_WONmQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#9 lore raw

Re: [PATCH] drm/amdgpu: add flag for high priority contexts

Cool. Follow up question then:

We can either:
1. Expose a priority number
2. Expose a priority enum, e.g. PRIORITY_HIGH, PRIORITY_NORMAL (to start)

The HW can do numerical priorities, but the SW scheduler would require a bit of rework to consume arbitrary priorities.

At least for our use case, I don't really see a need for super fine-grained priorities. Something like HIGH/NORMAL is sufficient, with possibility of expanding to add REALTIME/LOW. So I think going for an enum is simpler/better.

yay or nay on enum?

Quoted 113 lines · Alex Deucher
Quoted 1 line · Andres Rodriguez

I was thinking of that originally, but the allocation context already

has a

Quoted 4 lines

flags variable which allows us to preserve the IOCTL ABI.

We could reserve a few bits of the flags for a priority level instead if that sounds good?

We can also use _pad for something else.

Alex

Quoted 5 lines

Regards,
Andres

On Tue, Jan 3, 2017 at 5:59 PM, Alex Deucher <[email protected]>

Quoted 4 lines · wrote:
Quoted 4 lines
Quoted 1 line · Andres Rodriguez

Add a new context creation flag, AMDGPU_CTX_FLAG_HIGHPRIORITY. This

flag

Quoted 28 lines
Quoted 28 lines
Quoted 28 lines

results in the allocated context receiving a higher scheduler priority that other contexts system-wide.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c       | 24
++++++++++++++++++------
 drivers/gpu/drm/amd/scheduler/gpu_scheduler.h |  1 +
 include/uapi/drm/amdgpu_drm.h                 |  3 ++-
 3 files changed, 21 insertions(+), 7 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +1 -1
index 400c66b..ce470e2 100644
@@ -25,11 +25,17 @@
 #include <drm/drmP.h>
 #include "amdgpu.h"
 
-static int amdgpu_ctx_init(struct amdgpu_device *adev, struct
 amdgpu_ctx *ctx)
+static int amdgpu_ctx_init(struct amdgpu_device *adev, int priority,
 struct amdgpu_ctx *ctx)
 {
        unsigned i, j;
        int r;
 
Quoted 11 lines
Quoted 11 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c static int amdgpu_ctx_init(struct amdgpu_device

Quoted 11 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +6 -0
+       if (priority < 0 || priority >= AMD_SCHED_MAX_PRIORITY)
+               return -EINVAL;
+
+       if (priority == AMD_SCHED_PRIORITY_HIGH &&
 !capable(CAP_SYS_ADMIN))
+               return -EACCES;
+
        memset(ctx, 0, sizeof(*ctx));
        ctx->adev = adev;
        kref_init(&ctx->refcount);
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +0 -0
@@ -51,7 +57,7 @@ static int amdgpu_ctx_init(struct amdgpu_device

*adev,

Quoted 5 lines
Quoted 5 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c static int amdgpu_ctx_init(struct amdgpu_device · line 55

Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +0 -1
 struct amdgpu_ctx *ctx)
                struct amdgpu_ring *ring = adev->rings[i];
                struct amd_sched_rq *rq;
 
-               rq = &ring->sched.sched_rq[AMD_

SCHED_PRIORITY_NORMAL];

Quoted 6 lines
Quoted 6 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c static void amdgpu_ctx_fini(struct amdgpu_ctx

Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +1 -0
+               rq = &ring->sched.sched_rq[priority];
                r = amd_sched_entity_init(&ring->sched,
 &ctx->rings[i].entity,
                                          rq, amdgpu_sched_jobs);
                if (r)
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +0 -0
@@ -90,11 +96,15 @@ static void amdgpu_ctx_fini(struct amdgpu_ctx

*ctx)

Quoted 36 lines
Quoted 36 lines
Quoted 36 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +7 -2
 static int amdgpu_ctx_alloc(struct amdgpu_device *adev,
                            struct amdgpu_fpriv *fpriv,
+                           int flags,
                            uint32_t *id)
 {
        struct amdgpu_ctx_mgr *mgr = &fpriv->ctx_mgr;
        struct amdgpu_ctx *ctx;
-       int r;
+       int r, priority;
+
+       priority = flags & AMDGPU_CTX_FLAG_HIGHPRIORITY ?
+               AMD_SCHED_PRIORITY_HIGH : AMD_SCHED_PRIORITY_NORMAL;
 
        ctx = kmalloc(sizeof(*ctx), GFP_KERNEL);
        if (!ctx)
@@ -107,8 +117,9 @@ static int amdgpu_ctx_alloc(struct amdgpu_device
 *adev,
                kfree(ctx);
                return r;
        }
+
        *id = (uint32_t)r;
-       r = amdgpu_ctx_init(adev, ctx);
+       r = amdgpu_ctx_init(adev, priority, ctx);
        if (r) {
                idr_remove(&mgr->ctx_handles, *id);
                *id = 0;
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +1 -1
@@ -186,7 +197,7 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void
 *data,
                     struct drm_file *filp)
 {
        int r;
-       uint32_t id;
+       uint32_t id, flags;
 
Quoted 65 lines
Quoted 3 lines
Quoted 3 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c int amdgpu_ctx_ioctl(struct drm_device *dev,

Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +0 -0
        union drm_amdgpu_ctx *args = data;
        struct amdgpu_device *adev = dev->dev_private;
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +0 -0
@@ -194,10 +205,11 @@ int amdgpu_ctx_ioctl(struct drm_device *dev,

void

Quoted 60 lines
Quoted 58 lines
Quoted 32 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +2 -1
 *data,
 
        r = 0;
        id = args->in.ctx_id;
+       flags = args->in.flags;
 
        switch (args->in.op) {
        case AMDGPU_CTX_OP_ALLOC_CTX:
-               r = amdgpu_ctx_alloc(adev, fpriv, &id);
+               r = amdgpu_ctx_alloc(adev, fpriv, flags, &id);
                args->out.alloc.ctx_id = id;
                break;

case AMDGPU_CTX_OP_FREE_CTX:

drivers/gpu/drm/amd/scheduler/gpu_scheduler.h +1 -0
index d8dc681..2e458de 100644
@@ -108,6 +108,7 @@ struct amd_sched_backend_ops {
 
 enum amd_sched_priority {
        AMD_SCHED_PRIORITY_KERNEL = 0,
+       AMD_SCHED_PRIORITY_HIGH,
        AMD_SCHED_PRIORITY_NORMAL,
        AMD_SCHED_MAX_PRIORITY
 };
include/uapi/drm/amdgpu_drm.h +0 -0
index 3961836..702332e 100644

include/uapi/drm/amdgpu_drm.h union drm_amdgpu_bo_list { · line 163

Quoted 4 lines
include/uapi/drm/amdgpu_drm.h +1 -0
@@ -160,10 +160,11 @@ union drm_amdgpu_bo_list {
 /* unknown cause */
 #define AMDGPU_CTX_UNKNOWN_RESET       3
 
+#define AMDGPU_CTX_FLAG_HIGHPRIORITY   (1 << 0)

Would it be better to expose a priority level rather than a single flag? If we want to expose more than just normal/high in the future for example.

Alex

Quoted 15 lines
include/uapi/drm/amdgpu_drm.h +1 -2
 struct drm_amdgpu_ctx_in {
        /** AMDGPU_CTX_OP_* */
        __u32   op;
-       /** For future use, no flags defined so far */
+       /** AMDGPU_CTX_FLAG_* */
        __u32   flags;
        __u32   ctx_id;
        __u32   _pad;
--

2.7.4

_______________________________________________
amd-gfx mailing list [email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#10 lore raw

Re: [PATCH] drm/amdgpu: add flag for high priority contexts

In reply to a message not in the archive

Andres,

I have on rather generic design question:

Why we want to restrict it to CAP_SYS_ADMIN?

+       if (priority == AMD_SCHED_PRIORITY_HIGH && !capable(CAP_SYS_ADMIN)) +               return -EACCES;

Should we make it generic? My understanding is that If we follow "nice" semantic then it will not require such privilege.

Sincerely yours,
Serguei Sagalovitch

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): SN1PR12MB07031776F52022AF4D659AA6FE610-z7L1TMIYDg6P/[email protected]

#12 lore raw

Re: [PATCH] drm/amdgpu: add flag for high priority contexts

Am 04.01.2017 um 12:03 schrieb Sagalovitch, Serguei:

Quoted 11 lines

Andres,

I have on rather generic design question:

Why we want to restrict it to CAP_SYS_ADMIN?

drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +2 -0
+       if (priority == AMD_SCHED_PRIORITY_HIGH && !capable(CAP_SYS_ADMIN))
+               return -EACCES;

Should we make it generic? My understanding is that If we follow "nice" semantic then it will not require such privilege.

Well it follows the "nice" semantic, the the documentation of the nice system call:

nice() adds inc to the nice value for the calling process. (A
higher nice value means a low priority.) Only the superuser may specify a negative increment, or priority increase. The range
for nice values is described in getpriority(2).

Of course the nice limit is more fine grained these days. IIRC it was a soft resource limit now the last time I looked.

We would essentially need something similar for the GPU if we want to allow a regular process to get a higher priority.

Regards,
Christian.

Quoted 9 lines

Sincerely yours,
Serguei Sagalovitch

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 19988121-36b8-00e4-afc9-bb9f8a0621e4-ANTagKRnAhcb1SvskN2V4Q@public.gmane.org

#14 lore raw

Re: [PATCH] drm/amdgpu: add flag for high priority contexts

Hey Serguei

Quoted 16 lines · Christian König

Am 04.01.2017 um 12:03 schrieb Sagalovitch, Serguei:

Quoted 4 lines

Andres,

I have on rather generic design question:

Quoted 8 lines

Why we want to restrict it to CAP_SYS_ADMIN?

+ if (priority == AMD_SCHED_PRIORITY_HIGH &&
!capable(CAP_SYS_ADMIN))
+ return -EACCES;

Should we make it generic? My understanding is that If we follow "nice" semantic then it will not require such privilege.

Well it follows the "nice" semantic, the the documentation of the nice system call:

Quoted 7 lines

nice() adds inc to the nice value for the calling process. (A
higher nice value means a low priority.) Only the superuser may specify a negative increment, or priority increase. The range
for nice values is described in getpriority(2).

Of course the nice limit is more fine grained these days. IIRC it was a soft resource limit now the last time I looked.

We would essentially need something similar for the GPU if we want to allow a regular process to get a higher priority.

As Christian mentioned this is based on the same restrictions as nice.

Also thinking along the lines of, it would be easier in the future to relax requirements than to increase them.

If we end up in a scenario where we want looser restrictions, that can be implemented as a new feature.

However, the opposite is not true. If we require to tighten restrictions that would result in an backwards compatibility breakage, and the patch would not be allowed. Apps that used to have access no longer have access.

Regards,
Andres

Quoted 14 lines

Regards,
Christian.

Quoted 8 lines

Sincerely yours,
Serguei Sagalovitch

     _______________________________________________

amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Quoted 4 lines

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): b4f3b2df-5b32-b7c4-9289-acf3111f6fdf-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#16 lore raw

Re: [PATCH] drm/amdgpu: add flag for high priority contexts

Am 04.01.2017 um 21:56 schrieb Andres Rodriguez:

Quoted 24 lines

Hey Serguei

Quoted 20 lines · Christian König

Am 04.01.2017 um 12:03 schrieb Sagalovitch, Serguei:

Quoted 4 lines

Andres,

I have on rather generic design question:

Quoted 8 lines

Why we want to restrict it to CAP_SYS_ADMIN?

+ if (priority == AMD_SCHED_PRIORITY_HIGH &&
!capable(CAP_SYS_ADMIN))
+ return -EACCES;

Should we make it generic? My understanding is that If we follow "nice" semantic then it will not require such privilege.

Well it follows the "nice" semantic, the the documentation of the nice system call:

nice() adds inc to the nice value for the calling process. (A
higher nice value means a low priority.) Only the superuser may specify a negative increment, or priority increase. The range
for nice values is described in getpriority(2).

Quoted 7 lines
Quoted 3 lines

Of course the nice limit is more fine grained these days. IIRC it was a soft resource limit now the last time I looked.

We would essentially need something similar for the GPU if we want to allow a regular process to get a higher priority.

As Christian mentioned this is based on the same restrictions as nice.

Also thinking along the lines of, it would be easier in the future to relax requirements than to increase them.

That is a rather interesting idea. Instead of increasing the priority for the compositor we could decrease it for everybody else. We could do this by setting an environment variable for the loged in user for example.

This would be similar to how nice() handles it and could avoid some of the security problems.

Quoted 4 lines

If we end up in a scenario where we want looser restrictions, that can be implemented as a new feature.

However, the opposite is not true. If we require to tighten restrictions that would result in an backwards compatibility breakage, and the patch would not be allowed. Apps that used to have access no longer have access.

Yes exactly. As soon as an interface gets upstream it is nailed in stone.

This means that we can't break existing userspace by adding more restrictions, but we can easily lower restrictions.

Regards,
Christian.

Quoted 23 lines

Regards,
Andres

Quoted 18 lines

Regards,
Christian.

Quoted 8 lines

Sincerely yours,
Serguei Sagalovitch

     _______________________________________________

amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#17 lore raw

RE: [PATCH] drm/amdgpu: add flag for high priority contexts

In reply to a message not in the archive

Hi Andres,
I did not follow the previous discussion, so please remind me if my questions have been covered already~
- The priority should be the queue properties, and do we really want to expose high priority on none-compute queue?
- Another question is we may need to do per submission priority tweak if we don't reserve one compute queue ahead of time. From the patch, it seems you only tweak the GPU scheduler's priority, but I think it is still insufficient given we don't have OS preemption supported,

Thanks.
Best Regards,
David

-----Original Message-----
From: amd-gfx [mailto:[email protected]] On Behalf Of Andres Rodriguez
Sent: Wednesday, January 4, 2017 6:55 AM
To: [email protected]
Cc: Andres Rodriguez <[email protected]>
Subject: [PATCH] drm/amdgpu: add flag for high priority contexts

Add a new context creation flag, AMDGPU_CTX_FLAG_HIGHPRIORITY. This flag results in the allocated context receiving a higher scheduler priority that other contexts system-wide.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c       | 24 ++++++++++++++++++------
 drivers/gpu/drm/amd/scheduler/gpu_scheduler.h |  1 +
 include/uapi/drm/amdgpu_drm.h                 |  3 ++-
 3 files changed, 21 insertions(+), 7 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +8 -1
index 400c66b..ce470e2 100644
@@ -25,11 +25,17 @@
 #include <drm/drmP.h>
 #include "amdgpu.h"
 
-static int amdgpu_ctx_init(struct amdgpu_device *adev, struct amdgpu_ctx *ctx)
+static int amdgpu_ctx_init(struct amdgpu_device *adev, int priority, 
+struct amdgpu_ctx *ctx)
 {
 	unsigned i, j;
 	int r;
 
+	if (priority < 0 || priority >= AMD_SCHED_MAX_PRIORITY)
+		return -EINVAL;
+
+	if (priority == AMD_SCHED_PRIORITY_HIGH && !capable(CAP_SYS_ADMIN))
+		return -EACCES;
+
 	memset(ctx, 0, sizeof(*ctx));
 	ctx->adev = adev;

kref_init(&ctx->refcount);
@@ -51,7 +57,7 @@ static int amdgpu_ctx_init(struct amdgpu_device *adev, struct amdgpu_ctx *ctx)
struct amdgpu_ring *ring = adev->rings[i];
struct amd_sched_rq *rq;

- rq = &ring->sched.sched_rq[AMD_SCHED_PRIORITY_NORMAL];
+ rq = &ring->sched.sched_rq[priority];
r = amd_sched_entity_init(&ring->sched, &ctx->rings[i].entity,
rq, amdgpu_sched_jobs);
if (r)
@@ -90,11 +96,15 @@ static void amdgpu_ctx_fini(struct amdgpu_ctx *ctx)

static int amdgpu_ctx_alloc(struct amdgpu_device *adev,
struct amdgpu_fpriv *fpriv,
+ int flags,
uint32_t *id)
{
struct amdgpu_ctx_mgr *mgr = &fpriv->ctx_mgr; struct amdgpu_ctx *ctx;
- int r;
+ int r, priority;
+
+ priority = flags & AMDGPU_CTX_FLAG_HIGHPRIORITY ?
+ AMD_SCHED_PRIORITY_HIGH : AMD_SCHED_PRIORITY_NORMAL;

ctx = kmalloc(sizeof(*ctx), GFP_KERNEL);
if (!ctx)
@@ -107,8 +117,9 @@ static int amdgpu_ctx_alloc(struct amdgpu_device *adev,
kfree(ctx);
return r;
}
+
*id = (uint32_t)r;
- r = amdgpu_ctx_init(adev, ctx);
+ r = amdgpu_ctx_init(adev, priority, ctx);
if (r) {
idr_remove(&mgr->ctx_handles, *id);
*id = 0;
@@ -186,7 +197,7 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void *data,
struct drm_file *filp)
{
int r;
- uint32_t id;
+ uint32_t id, flags;

union drm_amdgpu_ctx *args = data;
struct amdgpu_device *adev = dev->dev_private; @@ -194,10 +205,11 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void *data,

r = 0;
id = args->in.ctx_id;
+ flags = args->in.flags;

switch (args->in.op) {
case AMDGPU_CTX_OP_ALLOC_CTX:
- r = amdgpu_ctx_alloc(adev, fpriv, &id);
+ r = amdgpu_ctx_alloc(adev, fpriv, flags, &id);
args->out.alloc.ctx_id = id;
break;
case AMDGPU_CTX_OP_FREE_CTX:

drivers/gpu/drm/amd/scheduler/gpu_scheduler.h +1 -0
index d8dc681..2e458de 100644
@@ -108,6 +108,7 @@ struct amd_sched_backend_ops {
 
 enum amd_sched_priority {
 	AMD_SCHED_PRIORITY_KERNEL = 0,
+	AMD_SCHED_PRIORITY_HIGH,
 	AMD_SCHED_PRIORITY_NORMAL,
 	AMD_SCHED_MAX_PRIORITY
 };
include/uapi/drm/amdgpu_drm.h +2 -1
@@ -160,10 +160,11 @@ union drm_amdgpu_bo_list {
 /* unknown cause */
 #define AMDGPU_CTX_UNKNOWN_RESET	3
 
+#define AMDGPU_CTX_FLAG_HIGHPRIORITY	(1 << 0)
 struct drm_amdgpu_ctx_in {
 	/** AMDGPU_CTX_OP_* */
 	__u32	op;
-	/** For future use, no flags defined so far */
+	/** AMDGPU_CTX_FLAG_* */
 	__u32	flags;
 	__u32	ctx_id;
 	__u32	_pad;

--
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): BN4PR12MB07876FAB038319F8E263B0CCEE610-aH9FTdWx9BancvD3hK8fMAdYzm3356FpvxpqHgZTriW3zl9H0oFU5g@public.gmane.org

#19 lore raw

Re: [PATCH] drm/amdgpu: add flag for high priority contexts

Hi David,

Quoted 3 lines · David

Hi Andres,
I did not follow the previous discussion, so please remind me if my questions have been covered already~
- The priority should be the queue properties, and do we really want to expose high priority on none-compute queue?

I exposing the concept across all queues is good for consistency. It will probably end up staying as a SW-scheduler feature for all queues except for compute.

However, if we do end up getting some HW features that enable priority support on other engines, then the API will be ready, and we'll only need a kernel side change to enable the support.

Quoted 2 lines

- Another question is we may need to do per submission priority tweak if we don't reserve one compute queue ahead of time.
From the patch, it seems you only tweak the GPU scheduler's priority, but I think it is still insufficient given we don't have OS preemption supported,

Correct, the current patch is just to get an environment in place to start doing measurements. With this framework in place we can start adding HW support to increase the priorities.

Dynamic tweaking might be necessary depending on how CU reservation is going to work.

Quoted 134 lines

Thanks.
Best Regards,
David

-----Original Message-----
From: amd-gfx [mailto:[email protected]] On Behalf Of Andres Rodriguez
Sent: Wednesday, January 4, 2017 6:55 AM
To: [email protected]
Cc: Andres Rodriguez <[email protected]>
Subject: [PATCH] drm/amdgpu: add flag for high priority contexts

Add a new context creation flag, AMDGPU_CTX_FLAG_HIGHPRIORITY. This flag results in the allocated context receiving a higher scheduler priority that other contexts system-wide.

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c       | 24 ++++++++++++++++++------
  drivers/gpu/drm/amd/scheduler/gpu_scheduler.h |  1 +
  include/uapi/drm/amdgpu_drm.h                 |  3 ++-
  3 files changed, 21 insertions(+), 7 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +17 -5
index 400c66b..ce470e2 100644
@@ -25,11 +25,17 @@
  #include <drm/drmP.h>
  #include "amdgpu.h"
 
-static int amdgpu_ctx_init(struct amdgpu_device *adev, struct amdgpu_ctx *ctx)
+static int amdgpu_ctx_init(struct amdgpu_device *adev, int priority,
+struct amdgpu_ctx *ctx)
  {
  	unsigned i, j;
  	int r;
 
+	if (priority < 0 || priority >= AMD_SCHED_MAX_PRIORITY)
+		return -EINVAL;
+
+	if (priority == AMD_SCHED_PRIORITY_HIGH && !capable(CAP_SYS_ADMIN))
+		return -EACCES;
+
  	memset(ctx, 0, sizeof(*ctx));
  	ctx->adev = adev;
  	kref_init(&ctx->refcount);
@@ -51,7 +57,7 @@ static int amdgpu_ctx_init(struct amdgpu_device *adev, struct amdgpu_ctx *ctx)
  		struct amdgpu_ring *ring = adev->rings[i];
  		struct amd_sched_rq *rq;
 
-		rq = &ring->sched.sched_rq[AMD_SCHED_PRIORITY_NORMAL];
+		rq = &ring->sched.sched_rq[priority];
  		r = amd_sched_entity_init(&ring->sched, &ctx->rings[i].entity,
  					  rq, amdgpu_sched_jobs);
  		if (r)
@@ -90,11 +96,15 @@ static void amdgpu_ctx_fini(struct amdgpu_ctx *ctx)
 
  static int amdgpu_ctx_alloc(struct amdgpu_device *adev,
  			    struct amdgpu_fpriv *fpriv,
+			    int flags,
  			    uint32_t *id)
  {
  	struct amdgpu_ctx_mgr *mgr = &fpriv->ctx_mgr;
  	struct amdgpu_ctx *ctx;
-	int r;
+	int r, priority;
+
+	priority = flags & AMDGPU_CTX_FLAG_HIGHPRIORITY ?
+		AMD_SCHED_PRIORITY_HIGH : AMD_SCHED_PRIORITY_NORMAL;
 
  	ctx = kmalloc(sizeof(*ctx), GFP_KERNEL);
  	if (!ctx)
@@ -107,8 +117,9 @@ static int amdgpu_ctx_alloc(struct amdgpu_device *adev,
  		kfree(ctx);
  		return r;
  	}
+
  	*id = (uint32_t)r;
-	r = amdgpu_ctx_init(adev, ctx);
+	r = amdgpu_ctx_init(adev, priority, ctx);
  	if (r) {
  		idr_remove(&mgr->ctx_handles, *id);
  		*id = 0;
@@ -186,7 +197,7 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void *data,
  		     struct drm_file *filp)
  {
  	int r;
-	uint32_t id;
+	uint32_t id, flags;
 
  	union drm_amdgpu_ctx *args = data;
  	struct amdgpu_device *adev = dev->dev_private; @@ -194,10 +205,11 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void *data,
  
  	r = 0;
  	id = args->in.ctx_id;
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +2 -1
+	flags = args->in.flags;
 
  	switch (args->in.op) {
  	case AMDGPU_CTX_OP_ALLOC_CTX:
-		r = amdgpu_ctx_alloc(adev, fpriv, &id);
+		r = amdgpu_ctx_alloc(adev, fpriv, flags, &id);
  		args->out.alloc.ctx_id = id;
  		break;
  	case AMDGPU_CTX_OP_FREE_CTX:
drivers/gpu/drm/amd/scheduler/gpu_scheduler.h +1 -0
index d8dc681..2e458de 100644
@@ -108,6 +108,7 @@ struct amd_sched_backend_ops {
 
  enum amd_sched_priority {
  	AMD_SCHED_PRIORITY_KERNEL = 0,
+	AMD_SCHED_PRIORITY_HIGH,
  	AMD_SCHED_PRIORITY_NORMAL,
  	AMD_SCHED_MAX_PRIORITY
  };
include/uapi/drm/amdgpu_drm.h +2 -2
@@ -160,10 +160,11 @@ union drm_amdgpu_bo_list {
  /* unknown cause */
  #define AMDGPU_CTX_UNKNOWN_RESET	3
 
+#define AMDGPU_CTX_FLAG_HIGHPRIORITY	(1 << 0)
  struct drm_amdgpu_ctx_in {
  	/** AMDGPU_CTX_OP_* */
  	__u32	op;
-	/** For future use, no flags defined so far */
+	/** AMDGPU_CTX_FLAG_* */
  	__u32	flags;
  	__u32	ctx_id;
  	__u32	_pad;
--

2.7.4

Quoted 4 lines

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): edb8b730-35ce-1df2-4919-39c868224b8d-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#21 lore raw

Re: [PATCH] drm/amdgpu: add flag for high priority contexts

Quoted 19 lines · Andres Rodriguez
Quoted 5 lines · David

Hi Andres,
I did not follow the previous discussion, so please remind me if my questions have been covered already~
- The priority should be the queue properties, and do we really want to expose high priority on none-compute queue?

I exposing the concept across all queues is good for consistency. It will probably end up staying as a SW-scheduler feature for all queues except for compute.

However, if we do end up getting some HW features that enable priority support on other engines, then the API will be ready, and we'll only need a kernel side change to enable the support.

Quoted 5 lines

- Another question is we may need to do per submission priority tweak if we don't reserve one compute queue ahead of time.
From the patch, it seems you only tweak the GPU scheduler's priority,
but I think it is still insufficient given we don't have OS preemption supported,

Quoted 6 lines

Correct, the current patch is just to get an environment in place to start doing measurements. With this framework in place we can start adding HW support to increase the priorities.

Dynamic tweaking might be necessary depending on how CU reservation is going to work.

It sounds like it's still too early to tell if this simple priority mechanism will end up being useful. It's probably better to hold off on merging this upstream until you have at least a proof of concept showing that it's actually useful in practice.

Signature · 6 lines
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 296af036-da31-72c6-4b4e-718ce0cf27bc-otUistvHUpPR7s880joybQ@public.gmane.org

#23 lore raw

Re: [PATCH] drm/amdgpu: add flag for high priority contexts

Hey Michel,

Sounds fair. I don't think we are in a rush to get this merged until it actually can provide the guarantees that we need.

I've gotten some good feedback on code change improvements, and that should be enough to get me to the next stage with HW support.

I'll send and updated patchset when I get new data on that.

Regards,
Andres

Quoted 24 lines · Michel Dänzer
Quoted 23 lines · Andres Rodriguez
Quoted 5 lines · David

Hi Andres,
I did not follow the previous discussion, so please remind me if my questions have been covered already~
- The priority should be the queue properties, and do we really want to expose high priority on none-compute queue?

I exposing the concept across all queues is good for consistency. It will probably end up staying as a SW-scheduler feature for all queues except for compute.

However, if we do end up getting some HW features that enable priority support on other engines, then the API will be ready, and we'll only need a kernel side change to enable the support.

Quoted 5 lines

- Another question is we may need to do per submission priority tweak if we don't reserve one compute queue ahead of time.
From the patch, it seems you only tweak the GPU scheduler's priority,
but I think it is still insufficient given we don't have OS preemption supported,

Correct, the current patch is just to get an environment in place to start doing measurements. With this framework in place we can start adding HW support to increase the priorities.

Quoted 8 lines
Quoted 2 lines

Dynamic tweaking might be necessary depending on how CU reservation is going to work.

It sounds like it's still too early to tell if this simple priority mechanism will end up being useful. It's probably better to hold off on merging this upstream until you have at least a proof of concept showing that it's actually useful in practice.

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx