High priority usermode contexts
Subsystems (3)
PATCH 1/1 High priority usermode contexts
This patch series provides the initial APIs for high priority contexts.
The current implementation is based on top of the SW scheduler, there are no HW priorities set yet.
This doesn't provide the quality of service we need for VR. Further changes to enable HW priorities will be required.
For some performance data collected with multiple apps see:
https://lists.freedesktop.org/archives/amd-gfx/2016-December/004257.html
Test apps available here:
https://github.com/lostgoat/Vulkan
Related libdrm/radv patches to follow
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 1483484076-10424-1-git-send-email-andresx7-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
PATCH 1/1 drm/amdgpu: add flag for high priority contexts
Add a new context creation flag, AMDGPU_CTX_FLAG_HIGHPRIORITY. This flag results in the allocated context receiving a higher scheduler priority that other contexts system-wide.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c | 24 ++++++++++++++++++------ drivers/gpu/drm/amd/scheduler/gpu_scheduler.h | 1 + include/uapi/drm/amdgpu_drm.h | 3 ++- 3 files changed, 21 insertions(+), 7 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +18 -6
@@ -25,11 +25,17 @@#include <drm/drmP.h> #include "amdgpu.h" -static int amdgpu_ctx_init (struct amdgpu_device *adev,struct amdgpu_ctx *ctx) +static int amdgpu_ctx_init (struct amdgpu_device *adev,int priority,struct amdgpu_ctx *ctx) {unsigned i, j;int r; +if (priority <0 || priority >=AMD_SCHED_MAX_PRIORITY ) +return -EINVAL ; + +if (priority ==AMD_SCHED_PRIORITY_HIGH && !capable (CAP_SYS_ADMIN )) +return -EACCES ; +memset (ctx,0 ,sizeof (*ctx)); ctx->adev = adev;kref_init (&ctx->refcount);
@@ -51,7 +57,7 @@ static int amdgpu_ctx_init(struct amdgpu_device *adev, struct amdgpu_ctx *ctx)struct amdgpu_ring *ring = adev->rings[i];struct amd_sched_rq *rq; - rq = &ring->sched.sched_rq[AMD_SCHED_PRIORITY_NORMAL ]; + rq = &ring->sched.sched_rq[priority]; r =amd_sched_entity_init (&ring->sched, &ctx->rings[i].entity, rq, amdgpu_sched_jobs);if (r)
@@ -90,11 +96,15 @@ static void amdgpu_ctx_fini(struct amdgpu_ctx *ctx)static int amdgpu_ctx_alloc (struct amdgpu_device *adev,struct amdgpu_fpriv *fpriv, +int flags,uint32_t *id) {struct amdgpu_ctx_mgr *mgr = &fpriv->ctx_mgr;struct amdgpu_ctx *ctx; -int r; +int r, priority; + + priority = flags &AMDGPU_CTX_FLAG_HIGHPRIORITY ? +AMD_SCHED_PRIORITY_HIGH :AMD_SCHED_PRIORITY_NORMAL ; ctx =kmalloc (sizeof (*ctx),GFP_KERNEL );if (!ctx)
@@ -107,8 +117,9 @@ static int amdgpu_ctx_alloc(struct amdgpu_device *adev,kfree (ctx);return r; } + *id = (uint32_t )r; - r =amdgpu_ctx_init (adev, ctx); + r =amdgpu_ctx_init (adev, priority, ctx);if (r) {idr_remove (&mgr->ctx_handles, *id); *id =0 ;
@@ -186,7 +197,7 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) {int r; -uint32_t id; +uint32_t id, flags;union drm_amdgpu_ctx *args = data;struct amdgpu_device *adev = dev->dev_private;
@@ -194,10 +205,11 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void *data, r =0 ; id = args->in.ctx_id; + flags = args->in.flags;switch (args->in.op) {case AMDGPU_CTX_OP_ALLOC_CTX : - r =amdgpu_ctx_alloc (adev, fpriv, &id); + r =amdgpu_ctx_alloc (adev, fpriv, flags, &id); args->out.alloc.ctx_id = id;break ;case AMDGPU_CTX_OP_FREE_CTX :
drivers/gpu/drm/amd/scheduler/gpu_scheduler.h +1 -0
@@ -108,6 +108,7 @@ struct amd_sched_backend_ops {enum amd_sched_priority {AMD_SCHED_PRIORITY_KERNEL =0 , +AMD_SCHED_PRIORITY_HIGH ,AMD_SCHED_PRIORITY_NORMAL ,AMD_SCHED_MAX_PRIORITY };
include/uapi/drm/amdgpu_drm.h +2 -1
@@ -160,10 +160,11 @@ union drm_amdgpu_bo_list {/* unknown cause */ #define AMDGPU_CTX_UNKNOWN_RESET 3 +#define AMDGPU_CTX_FLAG_HIGHPRIORITY (1 << 0)struct drm_amdgpu_ctx_in {/** AMDGPU_CTX_OP_* */ __u32 op; -/** For future use, no flags defined so far */ +/** AMDGPU_CTX_FLAG_* */ __u32 flags;__u32 ctx_id;__u32 _pad;
Signature · 6 lines
2.7.4 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 1483484076-10424-2-git-send-email-andresx7-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Re: [PATCH] drm/amdgpu: add flag for high priority contexts
Quoted 110 lines · Andres Rodriguez
Add a new context creation flag,
AMDGPU_CTX_FLAG_HIGHPRIORITY. This flag results in the allocated context receiving a higher scheduler priority that other contexts system-wide.Signed-off-by Andres Rodriguez <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c | 24 ++++++++++++++++++------ drivers/gpu/drm/amd/scheduler/gpu_scheduler.h | 1 + include/uapi/drm/amdgpu_drm.h | 3 ++- 3 files changed, 21 insertions(+), 7 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +18
-6@@ -25,11 +25,17 @@#include <drm/drmP.h> #include "amdgpu.h" -static int amdgpu_ctx_init (struct amdgpu_device *adev,struct amdgpu_ctx *ctx) +static int amdgpu_ctx_init (struct amdgpu_device *adev,int priority,struct amdgpu_ctx *ctx) {unsigned i, j;int r; +if (priority <0 || priority >=AMD_SCHED_MAX_PRIORITY ) +return -EINVAL ; + +if (priority ==AMD_SCHED_PRIORITY_HIGH && !capable (CAP_SYS_ADMIN )) +return -EACCES ; +memset (ctx,0 ,sizeof (*ctx)); ctx->adev = adev;kref_init (&ctx->refcount);@@ -51,7 +57,7 @@ static int amdgpu_ctx_init(struct amdgpu_device *adev, struct amdgpu_ctx *ctx)struct amdgpu_ring *ring = adev->rings[i];struct amd_sched_rq *rq; - rq = &ring->sched.sched_rq[AMD_SCHED_PRIORITY_NORMAL ]; + rq = &ring->sched.sched_rq[priority]; r =amd_sched_entity_init (&ring->sched, &ctx->rings[i].entity, rq, amdgpu_sched_jobs);if (r)@@ -90,11 +96,15 @@ static void amdgpu_ctx_fini(struct amdgpu_ctx *ctx)static int amdgpu_ctx_alloc (struct amdgpu_device *adev,struct amdgpu_fpriv *fpriv, +int flags,uint32_t *id) {struct amdgpu_ctx_mgr *mgr = &fpriv->ctx_mgr;struct amdgpu_ctx *ctx; -int r; +int r, priority; + + priority = flags &AMDGPU_CTX_FLAG_HIGHPRIORITY ? +AMD_SCHED_PRIORITY_HIGH :AMD_SCHED_PRIORITY_NORMAL ; ctx =kmalloc (sizeof (*ctx),GFP_KERNEL );if (!ctx)@@ -107,8 +117,9 @@ static int amdgpu_ctx_alloc(struct amdgpu_device *adev,kfree (ctx);return r; } + *id = (uint32_t )r; - r =amdgpu_ctx_init (adev, ctx); + r =amdgpu_ctx_init (adev, priority, ctx);if (r) {idr_remove (&mgr->ctx_handles, *id); *id =0 ;@@ -186,7 +197,7 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) {int r; -uint32_t id; +uint32_t id, flags;union drm_amdgpu_ctx *args = data;struct amdgpu_device *adev = dev->dev_private;@@ -194,10 +205,11 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void *data, r =0 ; id = args->in.ctx_id; + flags = args->in.flags;switch (args->in.op) {case AMDGPU_CTX_OP_ALLOC_CTX : - r =amdgpu_ctx_alloc (adev, fpriv, &id); + r =amdgpu_ctx_alloc (adev, fpriv, flags, &id); args->out.alloc.ctx_id = id;break ;case AMDGPU_CTX_OP_FREE_CTX :drivers/gpu/drm/amd/scheduler/gpu_scheduler.h +1
-0@@ -108,6 +108,7 @@ struct amd_sched_backend_ops {enum amd_sched_priority {AMD_SCHED_PRIORITY_KERNEL =0 , +AMD_SCHED_PRIORITY_HIGH ,AMD_SCHED_PRIORITY_NORMAL ,AMD_SCHED_MAX_PRIORITY };include/uapi/drm/amdgpu_drm.h +0
-0
include/uapi/drm/amdgpu_drm.h union drm_amdgpu_bo_list { · line 163
Quoted 4 lines
include/uapi/drm/amdgpu_drm.h +1
-0@@ -160,10 +160,11 @@ union drm_amdgpu_bo_list {/* unknown cause */ #define AMDGPU_CTX_UNKNOWN_RESET 3 +#define AMDGPU_CTX_FLAG_HIGHPRIORITY (1 << 0)
Would it be better to expose a priority level rather than a single flag? If we want to expose more than just normal/high in the future for example.
Alex
Quoted 11 lines
include/uapi/drm/amdgpu_drm.h +1
-2struct drm_amdgpu_ctx_in {/** AMDGPU_CTX_OP_* */ __u32 op; -/** For future use, no flags defined so far */ +/** AMDGPU_CTX_FLAG_* */ __u32 flags;__u32 ctx_id;__u32 _pad; --2.7.4
Quoted 4 lines
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): CADnq5_Mt_-61h6DG77kqepp5psS2JGHJOA-mek513s5HgvtTDA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Re: [PATCH] drm/amdgpu: add flag for high priority contexts
I was thinking of that originally, but the allocation context already has a flags variable which allows us to preserve the IOCTL ABI.
We could reserve a few bits of the flags for a priority level instead if that sounds good?
Regards,
Andres
Quoted 127 lines · Alex Deucher
Quoted 7 lines · Andres Rodriguez
Add a new context creation flag,
AMDGPU_CTX_FLAG_HIGHPRIORITY. This flag results in the allocated context receiving a higher scheduler priority that other contexts system-wide.Signed-off-by Andres Rodriguez <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c | 24++++++++++++++++++------
Review
Quoted 5 lines
drivers/gpu/drm/amd/scheduler/gpu_scheduler.h | 1 + include/uapi/drm/amdgpu_drm.h | 3 ++- 3 files changed, 21 insertions(+), 7 deletions(-)(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.cdrivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c line 28
Quoted 8 lines
index 400c66b..ce470e2 100644
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +0
-1@@ -25,11 +25,17 @@#include <drm/drmP.h> #include "amdgpu.h" -static int amdgpu_ctx_init(struct amdgpu_device *adev, struct
amdgpu_ctx*ctx)drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c line 28
Quoted 1 line
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +1
-0+static int amdgpu_ctx_init(struct amdgpu_device *adev, int priority,struct
amdgpu_ctx*ctx)drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c
Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +4
-0{unsigned i, j;int r; +if (priority <0 || priority >=AMD_SCHED_MAX_PRIORITY ) +return -EINVAL ; + + if (priority == AMD_SCHED_PRIORITY_HIGH &&!capable(CAP_SYS_ADMIN))
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c static int amdgpu_ctx_init(struct amdgpu_device *adev,
Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +2
-0+return -EACCES ; +memset (ctx,0 ,sizeof (*ctx)); ctx->adev = adev;kref_init (&ctx->refcount);@@ -51,7 +57,7 @@ static int amdgpu_ctx_init(struct amdgpu_device *adev,struct
amdgpu_ctx*ctx)drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c static int amdgpu_ctx_init(struct amdgpu_device *adev, struct amdgpu_ctx *ctx)
Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +1
-1struct amdgpu_ring *ring = adev->rings[i];struct amd_sched_rq *rq; - rq = &ring->sched.sched_rq[AMD_SCHED_PRIORITY_NORMAL ]; + rq = &ring->sched.sched_rq[priority]; r = amd_sched_entity_init(&ring->sched,&ctx->rings[i].entity,
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c static int amdgpu_ctx_alloc(struct amdgpu_device
Quoted 20 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +5
-1rq, amdgpu_sched_jobs);if (r)@@ -90,11 +96,15 @@ static void amdgpu_ctx_fini(struct amdgpu_ctx *ctx)static int amdgpu_ctx_alloc (struct amdgpu_device *adev,struct amdgpu_fpriv *fpriv, +int flags,uint32_t *id) {struct amdgpu_ctx_mgr *mgr = &fpriv->ctx_mgr;struct amdgpu_ctx *ctx; -int r; +int r, priority; + + priority = flags &AMDGPU_CTX_FLAG_HIGHPRIORITY ? +AMD_SCHED_PRIORITY_HIGH :AMD_SCHED_PRIORITY_NORMAL ; ctx =kmalloc (sizeof (*ctx),GFP_KERNEL );if (!ctx)@@ -107,8 +117,9 @@ static int amdgpu_ctx_alloc(struct amdgpu_device*adev,
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c int amdgpu_ctx_ioctl(struct drm_device *dev, void
Quoted 11 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +2
-1kfree (ctx);return r; } + *id = (uint32_t )r; - r =amdgpu_ctx_init (adev, ctx); + r =amdgpu_ctx_init (adev, priority, ctx);if (r) {idr_remove (&mgr->ctx_handles, *id); *id =0 ;@@ -186,7 +197,7 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void*data,
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c int amdgpu_ctx_ioctl(struct drm_device *dev, void
Quoted 9 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +1
-1struct drm_file *filp) {int r; -uint32_t id; +uint32_t id, flags;union drm_amdgpu_ctx *args = data;struct amdgpu_device *adev = dev->dev_private;@@ -194,10 +205,11 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void*data,
Review
Quoted 13 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +2
-1r =0 ; id = args->in.ctx_id; + flags = args->in.flags;switch (args->in.op) {case AMDGPU_CTX_OP_ALLOC_CTX : - r =amdgpu_ctx_alloc (adev, fpriv, &id); + r =amdgpu_ctx_alloc (adev, fpriv, flags, &id); args->out.alloc.ctx_id = id;break ;case AMDGPU_CTX_OP_FREE_CTX :(quoted diff) +0
-0
b/drivers/gpu/drm/amd/scheduler/gpu_scheduler.hReview
Quoted 12 lines
index d8dc681..2e458de 100644
drivers/gpu/drm/amd/scheduler/gpu_scheduler.h +1
-0@@ -108,6 +108,7 @@ struct amd_sched_backend_ops {enum amd_sched_priority {AMD_SCHED_PRIORITY_KERNEL =0 , +AMD_SCHED_PRIORITY_HIGH ,AMD_SCHED_PRIORITY_NORMAL ,AMD_SCHED_MAX_PRIORITY };(quoted diff) +0
-0
b/include/uapi/drm/amdgpu_drm.hQuoted 7 lines
index 3961836..702332e 100644
include/uapi/drm/amdgpu_drm.h +0
-0@@ -160,10 +160,11 @@ union drm_amdgpu_bo_list {/* unknown cause */ #define AMDGPU_CTX_UNKNOWN_RESET 3
Quoted 24 lines
include/uapi/drm/amdgpu_drm.h union drm_amdgpu_bo_list { · line 163
Quoted 1 line
include/uapi/drm/amdgpu_drm.h +1
-0+#define AMDGPU_CTX_FLAG_HIGHPRIORITY (1 << 0)Would it be better to expose a priority level rather than a single flag? If we want to expose more than just normal/high in the future for example.
Alex
Quoted 15 lines
include/uapi/drm/amdgpu_drm.h +1
-2struct drm_amdgpu_ctx_in {/** AMDGPU_CTX_OP_* */ __u32 op; -/** For future use, no flags defined so far */ +/** AMDGPU_CTX_FLAG_* */ __u32 flags;__u32 ctx_id;__u32 _pad; --2.7.4
_______________________________________________
amd-gfx mailing list [email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Re: [PATCH] drm/amdgpu: add flag for high priority contexts
Quoted 5 lines · Andres Rodriguez
I was thinking of that originally, but the allocation context already has a flags variable which allows us to preserve the IOCTL ABI.
We could reserve a few bits of the flags for a priority level instead if that sounds good?
We can also use _pad for something else.
Alex
Quoted 158 lines
Regards,
AndresQuoted 151 lines · Alex Deucher
Quoted 122 lines · Andres Rodriguez
Add a new context creation flag,
AMDGPU_CTX_FLAG_HIGHPRIORITY. This flag results in the allocated context receiving a higher scheduler priority that other contexts system-wide.Signed-off-by Andres Rodriguez <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c | 24 ++++++++++++++++++------ drivers/gpu/drm/amd/scheduler/gpu_scheduler.h | 1 + include/uapi/drm/amdgpu_drm.h | 3 ++- 3 files changed, 21 insertions(+), 7 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +7
-1@@ -25,11 +25,17 @@#include <drm/drmP.h> #include "amdgpu.h" -static int amdgpu_ctx_init (struct amdgpu_device *adev,struct amdgpu_ctx *ctx) +static int amdgpu_ctx_init (struct amdgpu_device *adev,int priority,struct amdgpu_ctx *ctx) {unsigned i, j;int r; +if (priority <0 || priority >=AMD_SCHED_MAX_PRIORITY ) +return -EINVAL ; + +if (priority ==AMD_SCHED_PRIORITY_HIGH && !capable (CAP_SYS_ADMIN )) +return -EACCES ; +memset(ctx, 0, sizeof(*ctx)); ctx->adev = adev; kref_init(&ctx->refcount);drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +1
-1@@ -51,7 +57,7 @@ static int amdgpu_ctx_init(struct amdgpu_device *adev, struct amdgpu_ctx *ctx)struct amdgpu_ring *ring = adev->rings[i];struct amd_sched_rq *rq; - rq = &ring->sched.sched_rq[AMD_SCHED_PRIORITY_NORMAL ]; + rq = &ring->sched.sched_rq[priority]; r = amd_sched_entity_init(&ring->sched, &ctx->rings[i].entity,rq, amdgpu_sched_jobs); if (r)drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +7
-2@@ -90,11 +96,15 @@ static void amdgpu_ctx_fini(struct amdgpu_ctx *ctx)static int amdgpu_ctx_alloc (struct amdgpu_device *adev,struct amdgpu_fpriv *fpriv, +int flags,uint32_t *id) {struct amdgpu_ctx_mgr *mgr = &fpriv->ctx_mgr;struct amdgpu_ctx *ctx; -int r; +int r, priority; + + priority = flags &AMDGPU_CTX_FLAG_HIGHPRIORITY ? +AMD_SCHED_PRIORITY_HIGH :AMD_SCHED_PRIORITY_NORMAL ; ctx =kmalloc (sizeof (*ctx),GFP_KERNEL );if (!ctx)@@ -107,8 +117,9 @@ static int amdgpu_ctx_alloc(struct amdgpu_device *adev,kfree (ctx);return r; } + *id = (uint32_t )r; - r =amdgpu_ctx_init (adev, ctx); + r =amdgpu_ctx_init (adev, priority, ctx);if (r) {idr_remove (&mgr->ctx_handles, *id);*id = 0;drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +1
-1@@ -186,7 +197,7 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) {int r; -uint32_t id; +uint32_t id, flags;union drm_amdgpu_ctx *args = data;struct amdgpu_device *adev = dev->dev_private;drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +2
-1@@ -194,10 +205,11 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void *data, r =0 ; id = args->in.ctx_id; + flags = args->in.flags;switch (args->in.op) {case AMDGPU_CTX_OP_ALLOC_CTX : - r =amdgpu_ctx_alloc (adev, fpriv, &id); + r =amdgpu_ctx_alloc (adev, fpriv, flags, &id); args->out.alloc.ctx_id = id;break ;case
AMDGPU_CTX_OP_FREE_CTX:drivers/gpu/drm/amd/scheduler/gpu_scheduler.h +1
-0@@ -108,6 +108,7 @@ struct amd_sched_backend_ops {enum amd_sched_priority {AMD_SCHED_PRIORITY_KERNEL =0 , +AMD_SCHED_PRIORITY_HIGH ,AMD_SCHED_PRIORITY_NORMAL ,AMD_SCHED_MAX_PRIORITY };include/uapi/drm/amdgpu_drm.h +0
-0include/uapi/drm/amdgpu_drm.h union drm_amdgpu_bo_list { · line 163
Quoted 4 lines
include/uapi/drm/amdgpu_drm.h +1
-0@@ -160,10 +160,11 @@ union drm_amdgpu_bo_list {/* unknown cause */ #define AMDGPU_CTX_UNKNOWN_RESET 3 +#define AMDGPU_CTX_FLAG_HIGHPRIORITY (1 << 0)Would it be better to expose a priority level rather than a single flag? If we want to expose more than just normal/high in the future for example.
Alex
Quoted 15 lines
include/uapi/drm/amdgpu_drm.h +1
-2struct drm_amdgpu_ctx_in {/** AMDGPU_CTX_OP_* */ __u32 op; -/** For future use, no flags defined so far */ +/** AMDGPU_CTX_FLAG_* */ __u32 flags;__u32 ctx_id;__u32 _pad; --2.7.4
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): CADnq5_Mu4t4i9prGhZakMdd3cFB5NRWE6zh0zFp05T1Z_WONmQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Re: [PATCH] drm/amdgpu: add flag for high priority contexts
Cool. Follow up question then:
We can either:
1. Expose a priority number
2. Expose a priority enum, e.g. PRIORITY_HIGH, PRIORITY_NORMAL (to start)
The HW can do numerical priorities, but the SW scheduler would require a bit of rework to consume arbitrary priorities.
At least for our use case, I don't really see a need for super fine-grained priorities. Something like HIGH/NORMAL is sufficient, with possibility of expanding to add REALTIME/LOW. So I think going for an enum is simpler/better.
yay or nay on enum?
Quoted 113 lines · Alex Deucher
Quoted 1 line · Andres Rodriguez
I was thinking of that originally, but the allocation context already
has a
Quoted 4 lines
flags variable which allows us to preserve the IOCTL ABI.
We could reserve a few bits of the flags for a priority level instead if that sounds good?
We can also use
_padfor something else.Alex
Quoted 5 lines
Regards,
AndresOn Tue, Jan 3, 2017 at 5:59 PM, Alex Deucher <[email protected]>
Quoted 4 lines · wrote:
Quoted 4 lines
Quoted 1 line · Andres Rodriguez
Add a new context creation flag,
AMDGPU_CTX_FLAG_HIGHPRIORITY. Thisflag
Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
results in the allocated context receiving a higher scheduler priority that other contexts system-wide.
Signed-off-by Andres Rodriguez <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c | 24 ++++++++++++++++++------ drivers/gpu/drm/amd/scheduler/gpu_scheduler.h | 1 + include/uapi/drm/amdgpu_drm.h | 3 ++- 3 files changed, 21 insertions(+), 7 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +1
-1@@ -25,11 +25,17 @@#include <drm/drmP.h> #include "amdgpu.h" -static int amdgpu_ctx_init (struct amdgpu_device *adev,struct amdgpu_ctx *ctx) +static int amdgpu_ctx_init (struct amdgpu_device *adev,int priority,struct amdgpu_ctx *ctx) {unsigned i, j;int r;Quoted 11 lines
Quoted 11 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c static int amdgpu_ctx_init(struct amdgpu_device
Quoted 11 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +6
-0+if (priority <0 || priority >=AMD_SCHED_MAX_PRIORITY ) +return -EINVAL ; + +if (priority ==AMD_SCHED_PRIORITY_HIGH && !capable (CAP_SYS_ADMIN )) +return -EACCES ; +memset(ctx, 0, sizeof(*ctx)); ctx->adev = adev; kref_init(&ctx->refcount);drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +0
-0@@ -51,7 +57,7 @@ static int amdgpu_ctx_init(struct amdgpu_device*adev,
Quoted 5 lines
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c static int amdgpu_ctx_init(struct amdgpu_device · line 55
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +0
-1struct amdgpu_ctx *ctx)struct amdgpu_ring *ring = adev->rings[i];struct amd_sched_rq *rq; - rq = &ring->sched.sched_rq[AMD_
SCHED_PRIORITY_NORMAL];Quoted 6 lines
Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c static void amdgpu_ctx_fini(struct amdgpu_ctx
Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +1
-0+ rq = &ring->sched.sched_rq[priority]; r = amd_sched_entity_init(&ring->sched, &ctx->rings[i].entity,rq, amdgpu_sched_jobs); if (r)drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +0
-0@@ -90,11 +96,15 @@ static void amdgpu_ctx_fini(struct amdgpu_ctx*ctx)
Quoted 36 lines
Quoted 36 lines
Quoted 36 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +7
-2static int amdgpu_ctx_alloc (struct amdgpu_device *adev,struct amdgpu_fpriv *fpriv, +int flags,uint32_t *id) {struct amdgpu_ctx_mgr *mgr = &fpriv->ctx_mgr;struct amdgpu_ctx *ctx; -int r; +int r, priority; + + priority = flags &AMDGPU_CTX_FLAG_HIGHPRIORITY ? +AMD_SCHED_PRIORITY_HIGH :AMD_SCHED_PRIORITY_NORMAL ; ctx =kmalloc (sizeof (*ctx),GFP_KERNEL );if (!ctx)@@ -107,8 +117,9 @@ static int amdgpu_ctx_alloc(struct amdgpu_device *adev,kfree (ctx);return r; } + *id = (uint32_t )r; - r =amdgpu_ctx_init (adev, ctx); + r =amdgpu_ctx_init (adev, priority, ctx);if (r) {idr_remove (&mgr->ctx_handles, *id);*id = 0;drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +1
-1@@ -186,7 +197,7 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) {int r; -uint32_t id; +uint32_t id, flags;
Quoted 65 lines
Quoted 3 lines
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c int amdgpu_ctx_ioctl(struct drm_device *dev,
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +0
-0union drm_amdgpu_ctx *args = data;struct amdgpu_device *adev = dev->dev_private;drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +0
-0@@ -194,10 +205,11 @@ int amdgpu_ctx_ioctl(struct drm_device *dev,void
Quoted 60 lines
Quoted 58 lines
Quoted 32 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +2
-1*data, r =0 ; id = args->in.ctx_id; + flags = args->in.flags;switch (args->in.op) {case AMDGPU_CTX_OP_ALLOC_CTX : - r =amdgpu_ctx_alloc (adev, fpriv, &id); + r =amdgpu_ctx_alloc (adev, fpriv, flags, &id); args->out.alloc.ctx_id = id;break ;case
AMDGPU_CTX_OP_FREE_CTX:drivers/gpu/drm/amd/scheduler/gpu_scheduler.h +1
-0@@ -108,6 +108,7 @@ struct amd_sched_backend_ops {enum amd_sched_priority {AMD_SCHED_PRIORITY_KERNEL =0 , +AMD_SCHED_PRIORITY_HIGH ,AMD_SCHED_PRIORITY_NORMAL ,AMD_SCHED_MAX_PRIORITY };include/uapi/drm/amdgpu_drm.h +0
-0include/uapi/drm/amdgpu_drm.h union drm_amdgpu_bo_list { · line 163
Quoted 4 lines
include/uapi/drm/amdgpu_drm.h +1
-0@@ -160,10 +160,11 @@ union drm_amdgpu_bo_list {/* unknown cause */ #define AMDGPU_CTX_UNKNOWN_RESET 3 +#define AMDGPU_CTX_FLAG_HIGHPRIORITY (1 << 0)Would it be better to expose a priority level rather than a single flag? If we want to expose more than just normal/high in the future for example.
Alex
Quoted 15 lines
include/uapi/drm/amdgpu_drm.h +1
-2struct drm_amdgpu_ctx_in {/** AMDGPU_CTX_OP_* */ __u32 op; -/** For future use, no flags defined so far */ +/** AMDGPU_CTX_FLAG_* */ __u32 flags;__u32 ctx_id;__u32 _pad; --2.7.4
_______________________________________________
amd-gfx mailing list [email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Re: [PATCH] drm/amdgpu: add flag for high priority contexts
In reply to a message not in the archiveAndres,
I have on rather generic design question:
Why we want to restrict it to CAP_SYS_ADMIN?
+ if (priority == AMD_SCHED_PRIORITY_HIGH && !capable(CAP_SYS_ADMIN)) + return -EACCES;
Should we make it generic? My understanding is that If we follow "nice" semantic then it will not require such privilege.
Sincerely yours,
Serguei Sagalovitch
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): SN1PR12MB07031776F52022AF4D659AA6FE610-z7L1TMIYDg6P/[email protected]
Re: [PATCH] drm/amdgpu: add flag for high priority contexts
Am 04.01.2017 um 12:03 schrieb Sagalovitch, Serguei:
Quoted 11 lines
Andres,
I have on rather generic design question:
Why we want to restrict it to
CAP_SYS_ADMIN?drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +2
-0+if (priority ==AMD_SCHED_PRIORITY_HIGH && !capable (CAP_SYS_ADMIN )) +return -EACCES ;Should we make it generic? My understanding is that If we follow "nice" semantic then it will not require such privilege.
Well it follows the "nice" semantic, the the documentation of the nice system call:
nice() adds inc to the nice value for the calling process. (A
higher nice value means a low priority.) Only the superuser may specify a negative increment, or priority increase. The range
for nice values is described in getpriority(2).
Of course the nice limit is more fine grained these days. IIRC it was a soft resource limit now the last time I looked.
We would essentially need something similar for the GPU if we want to allow a regular process to get a higher priority.
Regards,
Christian.
Quoted 9 lines
Sincerely yours,
Serguei Sagalovitch_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 19988121-36b8-00e4-afc9-bb9f8a0621e4-ANTagKRnAhcb1SvskN2V4Q@public.gmane.org
Re: [PATCH] drm/amdgpu: add flag for high priority contexts
Hey Serguei
Quoted 16 lines · Christian König
Am 04.01.2017 um 12:03 schrieb Sagalovitch, Serguei:
Quoted 4 lines
Andres,
I have on rather generic design question:
Quoted 8 lines
Why we want to restrict it to
CAP_SYS_ADMIN?+ if (priority ==
AMD_SCHED_PRIORITY_HIGH&&
!capable(CAP_SYS_ADMIN))
+ return-EACCES;Should we make it generic? My understanding is that If we follow "nice" semantic then it will not require such privilege.
Well it follows the "nice" semantic, the the documentation of the nice system call:
Quoted 7 lines
nice()adds inc to the nice value for the calling process. (A
higher nice value means a low priority.) Only the superuser may specify a negative increment, or priority increase. The range
for nice values is described in getpriority(2).Of course the nice limit is more fine grained these days. IIRC it was a soft resource limit now the last time I looked.
We would essentially need something similar for the GPU if we want to allow a regular process to get a higher priority.
As Christian mentioned this is based on the same restrictions as nice.
Also thinking along the lines of, it would be easier in the future to relax requirements than to increase them.
If we end up in a scenario where we want looser restrictions, that can be implemented as a new feature.
However, the opposite is not true. If we require to tighten restrictions that would result in an backwards compatibility breakage, and the patch would not be allowed. Apps that used to have access no longer have access.
Regards,
Andres
Quoted 14 lines
Regards,
Christian.Quoted 8 lines
Sincerely yours,
Serguei Sagalovitch_______________________________________________amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Quoted 4 lines
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): b4f3b2df-5b32-b7c4-9289-acf3111f6fdf-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Re: [PATCH] drm/amdgpu: add flag for high priority contexts
Am 04.01.2017 um 21:56 schrieb Andres Rodriguez:
Quoted 24 lines
Hey Serguei
Quoted 20 lines · Christian König
Am 04.01.2017 um 12:03 schrieb Sagalovitch, Serguei:
Quoted 4 lines
Andres,
I have on rather generic design question:
Quoted 8 lines
Why we want to restrict it to
CAP_SYS_ADMIN?+ if (priority ==
AMD_SCHED_PRIORITY_HIGH&&
!capable(CAP_SYS_ADMIN))
+ return-EACCES;Should we make it generic? My understanding is that If we follow "nice" semantic then it will not require such privilege.
Well it follows the "nice" semantic, the the documentation of the nice system call:
nice()adds inc to the nice value for the calling process. (A
higher nice value means a low priority.) Only the superuser may specify a negative increment, or priority increase. The range
for nice values is described in getpriority(2).
Quoted 7 lines
Quoted 3 lines
Of course the nice limit is more fine grained these days. IIRC it was a soft resource limit now the last time I looked.
We would essentially need something similar for the GPU if we want to allow a regular process to get a higher priority.
As Christian mentioned this is based on the same restrictions as nice.
Also thinking along the lines of, it would be easier in the future to relax requirements than to increase them.
That is a rather interesting idea. Instead of increasing the priority for the compositor we could decrease it for everybody else. We could do this by setting an environment variable for the loged in user for example.
This would be similar to how nice() handles it and could avoid some of the security problems.
Quoted 4 lines
If we end up in a scenario where we want looser restrictions, that can be implemented as a new feature.
However, the opposite is not true. If we require to tighten restrictions that would result in an backwards compatibility breakage, and the patch would not be allowed. Apps that used to have access no longer have access.
Yes exactly. As soon as an interface gets upstream it is nailed in stone.
This means that we can't break existing userspace by adding more restrictions, but we can easily lower restrictions.
Regards,
Christian.
Quoted 23 lines
Regards,
AndresQuoted 18 lines
Regards,
Christian.Quoted 8 lines
Sincerely yours,
Serguei Sagalovitch_______________________________________________amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
RE: [PATCH] drm/amdgpu: add flag for high priority contexts
In reply to a message not in the archiveHi Andres,
I did not follow the previous discussion, so please remind me if my questions have been covered already~
- The priority should be the queue properties, and do we really want to expose high priority on none-compute queue?
- Another question is we may need to do per submission priority tweak if we don't reserve one compute queue ahead of time. From the patch, it seems you only tweak the GPU scheduler's priority, but I think it is still insufficient given we don't have OS preemption supported,
Thanks.
Best Regards,
David
-----Original Message-----
From: amd-gfx [mailto:[email protected]] On Behalf Of Andres Rodriguez
Sent: Wednesday, January 4, 2017 6:55 AM
To: [email protected]
Cc: Andres Rodriguez <[email protected]>
Subject: [PATCH] drm/amdgpu: add flag for high priority contexts
Add a new context creation flag, AMDGPU_CTX_FLAG_HIGHPRIORITY. This flag results in the allocated context receiving a higher scheduler priority that other contexts system-wide.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c | 24 ++++++++++++++++++------ drivers/gpu/drm/amd/scheduler/gpu_scheduler.h | 1 + include/uapi/drm/amdgpu_drm.h | 3 ++- 3 files changed, 21 insertions(+), 7 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +8 -1
@@ -25,11 +25,17 @@#include <drm/drmP.h> #include "amdgpu.h" -static int amdgpu_ctx_init (struct amdgpu_device *adev,struct amdgpu_ctx *ctx) +static int amdgpu_ctx_init (struct amdgpu_device *adev,int priority, +struct amdgpu_ctx *ctx) {unsigned i, j;int r; +if (priority <0 || priority >=AMD_SCHED_MAX_PRIORITY ) +return -EINVAL ; + +if (priority ==AMD_SCHED_PRIORITY_HIGH && !capable (CAP_SYS_ADMIN )) +return -EACCES ; +memset (ctx,0 ,sizeof (*ctx)); ctx->adev = adev;
kref_init(&ctx->refcount);
@@ -51,7 +57,7 @@ static int amdgpu_ctx_init(struct amdgpu_device *adev, struct amdgpu_ctx *ctx)
struct amdgpu_ring *ring = adev->rings[i];
struct amd_sched_rq *rq;
- rq = &ring->sched.sched_rq[AMD_SCHED_PRIORITY_NORMAL];
+ rq = &ring->sched.sched_rq[priority];
r = amd_sched_entity_init(&ring->sched, &ctx->rings[i].entity,
rq, amdgpu_sched_jobs);
if (r)
@@ -90,11 +96,15 @@ static void amdgpu_ctx_fini(struct amdgpu_ctx *ctx)
static int amdgpu_ctx_alloc(struct amdgpu_device *adev,
struct amdgpu_fpriv *fpriv,
+ int flags,
uint32_t *id)
{
struct amdgpu_ctx_mgr *mgr = &fpriv->ctx_mgr; struct amdgpu_ctx *ctx;
- int r;
+ int r, priority;
+
+ priority = flags & AMDGPU_CTX_FLAG_HIGHPRIORITY ?
+ AMD_SCHED_PRIORITY_HIGH : AMD_SCHED_PRIORITY_NORMAL;
ctx = kmalloc(sizeof(*ctx), GFP_KERNEL);
if (!ctx)
@@ -107,8 +117,9 @@ static int amdgpu_ctx_alloc(struct amdgpu_device *adev,
kfree(ctx);
return r;
}
+
*id = (uint32_t)r;
- r = amdgpu_ctx_init(adev, ctx);
+ r = amdgpu_ctx_init(adev, priority, ctx);
if (r) {
idr_remove(&mgr->ctx_handles, *id);
*id = 0;
@@ -186,7 +197,7 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void *data,
struct drm_file *filp)
{
int r;
- uint32_t id;
+ uint32_t id, flags;
union drm_amdgpu_ctx *args = data;
struct amdgpu_device *adev = dev->dev_private; @@ -194,10 +205,11 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void *data,
r = 0;
id = args->in.ctx_id;
+ flags = args->in.flags;
switch (args->in.op) {
case AMDGPU_CTX_OP_ALLOC_CTX:
- r = amdgpu_ctx_alloc(adev, fpriv, &id);
+ r = amdgpu_ctx_alloc(adev, fpriv, flags, &id);
args->out.alloc.ctx_id = id;
break;
case AMDGPU_CTX_OP_FREE_CTX:
drivers/gpu/drm/amd/scheduler/gpu_scheduler.h +1 -0
@@ -108,6 +108,7 @@ struct amd_sched_backend_ops {enum amd_sched_priority {AMD_SCHED_PRIORITY_KERNEL =0 , +AMD_SCHED_PRIORITY_HIGH ,AMD_SCHED_PRIORITY_NORMAL ,AMD_SCHED_MAX_PRIORITY };
include/uapi/drm/amdgpu_drm.h +2 -1
@@ -160,10 +160,11 @@ union drm_amdgpu_bo_list {/* unknown cause */ #define AMDGPU_CTX_UNKNOWN_RESET 3 +#define AMDGPU_CTX_FLAG_HIGHPRIORITY (1 << 0)struct drm_amdgpu_ctx_in {/** AMDGPU_CTX_OP_* */ __u32 op; -/** For future use, no flags defined so far */ +/** AMDGPU_CTX_FLAG_* */ __u32 flags;__u32 ctx_id;__u32 _pad;
--
2.7.4
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): BN4PR12MB07876FAB038319F8E263B0CCEE610-aH9FTdWx9BancvD3hK8fMAdYzm3356FpvxpqHgZTriW3zl9H0oFU5g@public.gmane.org
Re: [PATCH] drm/amdgpu: add flag for high priority contexts
Hi David,
Quoted 3 lines · David
Hi Andres,
I did not follow the previous discussion, so please remind me if my questions have been covered already~
- The priority should be the queue properties, and do we really want to expose high priority on none-compute queue?
I exposing the concept across all queues is good for consistency. It will probably end up staying as a SW-scheduler feature for all queues except for compute.
However, if we do end up getting some HW features that enable priority support on other engines, then the API will be ready, and we'll only need a kernel side change to enable the support.
Quoted 2 lines
- Another question is we may need to do per submission priority tweak if we don't reserve one compute queue ahead of time.
From the patch, it seems you only tweak the GPU scheduler's priority, but I think it is still insufficient given we don't have OS preemption supported,
Correct, the current patch is just to get an environment in place to start doing measurements. With this framework in place we can start adding HW support to increase the priorities.
Dynamic tweaking might be necessary depending on how CU reservation is going to work.
Quoted 134 lines
Thanks.
Best Regards,
David-----Original Message-----
From: amd-gfx [mailto:[email protected]] On Behalf Of Andres Rodriguez
Sent: Wednesday, January 4, 2017 6:55 AM
To: [email protected]
Cc: Andres Rodriguez <[email protected]>
Subject: [PATCH] drm/amdgpu: add flag for high priority contextsAdd a new context creation flag,
AMDGPU_CTX_FLAG_HIGHPRIORITY. This flag results in the allocated context receiving a higher scheduler priority that other contexts system-wide.Signed-off-by Andres Rodriguez <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c | 24 ++++++++++++++++++------ drivers/gpu/drm/amd/scheduler/gpu_scheduler.h | 1 + include/uapi/drm/amdgpu_drm.h | 3 ++- 3 files changed, 21 insertions(+), 7 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +17
-5@@ -25,11 +25,17 @@#include <drm/drmP.h> #include "amdgpu.h" -static int amdgpu_ctx_init (struct amdgpu_device *adev,struct amdgpu_ctx *ctx) +static int amdgpu_ctx_init (struct amdgpu_device *adev,int priority, +struct amdgpu_ctx *ctx) {unsigned i, j;int r; +if (priority <0 || priority >=AMD_SCHED_MAX_PRIORITY ) +return -EINVAL ; + +if (priority ==AMD_SCHED_PRIORITY_HIGH && !capable (CAP_SYS_ADMIN )) +return -EACCES ; +memset (ctx,0 ,sizeof (*ctx)); ctx->adev = adev;kref_init (&ctx->refcount);@@ -51,7 +57,7 @@ static int amdgpu_ctx_init(struct amdgpu_device *adev, struct amdgpu_ctx *ctx)struct amdgpu_ring *ring = adev->rings[i];struct amd_sched_rq *rq; - rq = &ring->sched.sched_rq[AMD_SCHED_PRIORITY_NORMAL ]; + rq = &ring->sched.sched_rq[priority]; r =amd_sched_entity_init (&ring->sched, &ctx->rings[i].entity, rq, amdgpu_sched_jobs);if (r)@@ -90,11 +96,15 @@ static void amdgpu_ctx_fini(struct amdgpu_ctx *ctx)static int amdgpu_ctx_alloc (struct amdgpu_device *adev,struct amdgpu_fpriv *fpriv, +int flags,uint32_t *id) {struct amdgpu_ctx_mgr *mgr = &fpriv->ctx_mgr;struct amdgpu_ctx *ctx; -int r; +int r, priority; + + priority = flags &AMDGPU_CTX_FLAG_HIGHPRIORITY ? +AMD_SCHED_PRIORITY_HIGH :AMD_SCHED_PRIORITY_NORMAL ; ctx =kmalloc (sizeof (*ctx),GFP_KERNEL );if (!ctx)@@ -107,8 +117,9 @@ static int amdgpu_ctx_alloc(struct amdgpu_device *adev,kfree (ctx);return r; } + *id = (uint32_t )r; - r =amdgpu_ctx_init (adev, ctx); + r =amdgpu_ctx_init (adev, priority, ctx);if (r) {idr_remove (&mgr->ctx_handles, *id); *id =0 ;@@ -186,7 +197,7 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) {int r; -uint32_t id; +uint32_t id, flags;union drm_amdgpu_ctx *args = data; struct amdgpu_device *adev = dev->dev_private; @@ -194,10 +205,11 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void *data,r = 0; id = args->in.ctx_id;drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +2
-1+ flags = args->in.flags;switch (args->in.op) {case AMDGPU_CTX_OP_ALLOC_CTX : - r =amdgpu_ctx_alloc (adev, fpriv, &id); + r =amdgpu_ctx_alloc (adev, fpriv, flags, &id); args->out.alloc.ctx_id = id;break ;case AMDGPU_CTX_OP_FREE_CTX :drivers/gpu/drm/amd/scheduler/gpu_scheduler.h +1
-0@@ -108,6 +108,7 @@ struct amd_sched_backend_ops {enum amd_sched_priority {AMD_SCHED_PRIORITY_KERNEL =0 , +AMD_SCHED_PRIORITY_HIGH ,AMD_SCHED_PRIORITY_NORMAL ,AMD_SCHED_MAX_PRIORITY };include/uapi/drm/amdgpu_drm.h +2
-2@@ -160,10 +160,11 @@ union drm_amdgpu_bo_list {/* unknown cause */ #define AMDGPU_CTX_UNKNOWN_RESET 3 +#define AMDGPU_CTX_FLAG_HIGHPRIORITY (1 << 0)struct drm_amdgpu_ctx_in {/** AMDGPU_CTX_OP_* */ __u32 op; -/** For future use, no flags defined so far */ +/** AMDGPU_CTX_FLAG_* */ __u32 flags;__u32 ctx_id;__u32 _pad; --2.7.4
Quoted 4 lines
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): edb8b730-35ce-1df2-4919-39c868224b8d-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Re: [PATCH] drm/amdgpu: add flag for high priority contexts
Quoted 19 lines · Andres Rodriguez
Quoted 5 lines · David
Hi Andres,
I did not follow the previous discussion, so please remind me if my questions have been covered already~
- The priority should be the queue properties, and do we really want to expose high priority on none-compute queue?I exposing the concept across all queues is good for consistency. It will probably end up staying as a SW-scheduler feature for all queues except for compute.
However, if we do end up getting some HW features that enable priority support on other engines, then the API will be ready, and we'll only need a kernel side change to enable the support.
Quoted 5 lines
- Another question is we may need to do per submission priority tweak if we don't reserve one compute queue ahead of time.
From the patch, it seems you only tweak the GPU scheduler's priority,
but I think it is still insufficient given we don't have OS preemption supported,
Quoted 6 lines
Correct, the current patch is just to get an environment in place to start doing measurements. With this framework in place we can start adding HW support to increase the priorities.
Dynamic tweaking might be necessary depending on how CU reservation is going to work.
It sounds like it's still too early to tell if this simple priority mechanism will end up being useful. It's probably better to hold off on merging this upstream until you have at least a proof of concept showing that it's actually useful in practice.
Signature · 6 lines
Earthling Michel Dänzer | http://www.amd.com Libre software enthusiast | Mesa and X developer _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 296af036-da31-72c6-4b4e-718ce0cf27bc-otUistvHUpPR7s880joybQ@public.gmane.org
Re: [PATCH] drm/amdgpu: add flag for high priority contexts
Hey Michel,
Sounds fair. I don't think we are in a rush to get this merged until it actually can provide the guarantees that we need.
I've gotten some good feedback on code change improvements, and that should be enough to get me to the next stage with HW support.
I'll send and updated patchset when I get new data on that.
Regards,
Andres
Quoted 24 lines · Michel Dänzer
Quoted 23 lines · Andres Rodriguez
Quoted 5 lines · David
Hi Andres,
I did not follow the previous discussion, so please remind me if my questions have been covered already~
- The priority should be the queue properties, and do we really want to expose high priority on none-compute queue?I exposing the concept across all queues is good for consistency. It will probably end up staying as a SW-scheduler feature for all queues except for compute.
However, if we do end up getting some HW features that enable priority support on other engines, then the API will be ready, and we'll only need a kernel side change to enable the support.
Quoted 5 lines
- Another question is we may need to do per submission priority tweak if we don't reserve one compute queue ahead of time.
From the patch, it seems you only tweak the GPU scheduler's priority,
but I think it is still insufficient given we don't have OS preemption supported,Correct, the current patch is just to get an environment in place to start doing measurements. With this framework in place we can start adding HW support to increase the priorities.
Quoted 8 lines
Quoted 2 lines
Dynamic tweaking might be necessary depending on how CU reservation is going to work.
It sounds like it's still too early to tell if this simple priority mechanism will end up being useful. It's probably better to hold off on merging this upstream until you have at least a proof of concept showing that it's actually useful in practice.
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx