Add support for high priority scheduling in amdgpu v9
Subsystems (4)
PATCH 3/3 Add support for high priority scheduling in amdgpu v9
Third part of the split of the series:
Add support for high priority scheduling in amdgpu v8
This is the part of the series that is in a bit more murky water than the rest.
Sending out this patch series mostly for completion. And maybe for discussion purposes as well. There are still 2 issues open with this series:
1) Is the spinlock patch still okay? Should we pursue this differently?
I'd rather not use a mutex here. That would mean that to program srbm registers from an interrupt we'd need to dispatch a worker thread. That could mean extra time that the CU reservation is in place which can impact performance.
So my preferred (biased) alternative is to still move to a spinlock.
Another alternative I'm not sure of: Can we take advantage of the KIQ FIFO semantics to perform srbm writes atomically?
Something like:
ib_append(ib, PKT_WRITE_REG(SRBM_SELECT(...))) ib_append(ib, PKT_WRITE_REG(SOME_REG, VAL) ib_append(ib, PKT_WRITE_REG(SRBM_SELECT(0, 0, 0))) ib_sumbit(kiq_ring, ib)
Something that makes this immediately feel wrong though is the possibility of a race condition between an srbm operation on the KIQ and one through MMIO.
2) Alex suggested changing some MMIO writes to happen on the KIQ instead. I
still haven't addressed that.
I'm not sure the full criteria for patches landing on -wip. But if these are good enough to fix with some followup work, I wouldn't be oppossed to that idea.
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 20170413223024.21868-1-andresx7-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
PATCH 1/4 drm/amdgpu: add parameter to allocate high priority contexts v7
Add a new context creation parameter to express a global context priority.
Contexts allocated with AMDGPU_CTX_PRIORITY_HIGH will receive higher priority to schedule their work than AMDGPU_CTX_PRIORITY_NORMAL (default) contexts.
v2: Instead of using flags, repurpose __pad
v3: Swap enum values of _NORMAL _HIGH for backwards compatibility
v4: Validate usermode priority and store it
v5: Move priority validation into amdgpu_ctx_ioctl(), headline reword
v6: add UAPI note regarding priorities requiring CAP_SYS_ADMIN
v7: remove ctx->priority
--- drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c | 36 ++++++++++++++++++++++++--- drivers/gpu/drm/amd/scheduler/gpu_scheduler.h | 1 + include/uapi/drm/amdgpu_drm.h | 8 +++++- 3 files changed, 40 insertions(+), 5 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +32 -4
@@ -8,67 +8,75 @@* and/or sell copies of the Software, and to permit persons to whom the * Software is furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in * all copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR * OTHER DEALINGS IN THE SOFTWARE. * * Authors: monk liu <[email protected]> */ #include <drm/drmP.h> #include "amdgpu.h" -static int amdgpu_ctx_init (struct amdgpu_device *adev,struct amdgpu_ctx *ctx) +static int amdgpu_ctx_init (struct amdgpu_device *adev, +enum amd_sched_priority priority, +struct amdgpu_ctx *ctx) {unsigned i, j;int r; +if (priority <0 || priority >=AMD_SCHED_PRIORITY_MAX ) +return -EINVAL ; + +if (priority ==AMD_SCHED_PRIORITY_HIGH && !capable (CAP_SYS_ADMIN )) +return -EACCES ; +memset (ctx,0 ,sizeof (*ctx)); ctx->adev = adev;kref_init (&ctx->refcount);spin_lock_init (&ctx->ring_lock); ctx->fences =kcalloc (amdgpu_sched_jobs *AMDGPU_MAX_RINGS ,sizeof (struct dma_fence *),GFP_KERNEL );if (!ctx->fences)return -ENOMEM ;for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) { ctx->rings[i].sequence =1 ; ctx->rings[i].fences = &ctx->fences[amdgpu_sched_jobs * i]; } ctx->reset_counter =atomic_read (&adev->gpu_reset_counter);/* create context entity for each ring */ for (i =0 ; i < adev->num_rings; i++) {struct amdgpu_ring *ring = adev->rings[i];struct amd_sched_rq *rq; - rq = &ring->sched.sched_rq[AMD_SCHED_PRIORITY_NORMAL ]; + rq = &ring->sched.sched_rq[priority]; r =amd_sched_entity_init (&ring->sched, &ctx->rings[i].entity, rq, amdgpu_sched_jobs);if (r) goto failed; } r =amdgpu_queue_mgr_init (adev, &ctx->queue_mgr);if (r) goto failed;return 0 ; failed:for (j =0 ; j < i; j++)amd_sched_entity_fini (&adev->rings[j]->sched, &ctx->rings[j].entity);kfree (ctx->fences); ctx->fences =NULL ;return r; }
@@ -79,59 +87,61 @@ static void amdgpu_ctx_fini(struct amdgpu_ctx *ctx)unsigned i, j;if (!adev)return ;for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i)for (j =0 ; j < amdgpu_sched_jobs; ++j)dma_fence_put (ctx->rings[i].fences[j]);kfree (ctx->fences); ctx->fences =NULL ;for (i =0 ; i < adev->num_rings; i++)amd_sched_entity_fini (&adev->rings[i]->sched, &ctx->rings[i].entity);amdgpu_queue_mgr_fini (adev, &ctx->queue_mgr); }static int amdgpu_ctx_alloc (struct amdgpu_device *adev,struct amdgpu_fpriv *fpriv, +enum amd_sched_priority priority,uint32_t *id) {struct amdgpu_ctx_mgr *mgr = &fpriv->ctx_mgr;struct amdgpu_ctx *ctx;int r; ctx =kmalloc (sizeof (*ctx),GFP_KERNEL );if (!ctx)return -ENOMEM ;mutex_lock (&mgr->lock); r =idr_alloc (&mgr->ctx_handles, ctx,1 ,0 ,GFP_KERNEL );if (r <0 ) {mutex_unlock (&mgr->lock);kfree (ctx);return r; } + *id = (uint32_t )r; - r =amdgpu_ctx_init (adev, ctx); + r =amdgpu_ctx_init (adev, priority, ctx);if (r) {idr_remove (&mgr->ctx_handles, *id); *id =0 ;kfree (ctx); }mutex_unlock (&mgr->lock);return r; }static void amdgpu_ctx_do_release (struct kref *ref) {struct amdgpu_ctx *ctx; ctx = container_of(ref, struct amdgpu_ctx, refcount);amdgpu_ctx_fini (ctx);kfree (ctx); }
@@ -167,56 +177,74 @@ static int amdgpu_ctx_query(struct amdgpu_device *adev,return -EINVAL ; }/* TODO: these two are always zero */ out->state.flags =0x0 ; out->state.hangs =0x0 ;/* determine if a GPU reset has occured since the last call */ reset_counter =atomic_read (&adev->gpu_reset_counter);/* TODO: this should ideally return NO, GUILTY, or INNOCENT. */ if (ctx->reset_counter == reset_counter) out->state.reset_status =AMDGPU_CTX_NO_RESET ;else out->state.reset_status =AMDGPU_CTX_UNKNOWN_RESET ; ctx->reset_counter = reset_counter;mutex_unlock (&mgr->lock);return 0 ; } +static enum amd_sched_priority amdgpu_to_sched_priority (int amdgpu_priority) +{ +switch (amdgpu_priority) { +case AMDGPU_CTX_PRIORITY_HIGH : +return AMD_SCHED_PRIORITY_HIGH ; +case AMDGPU_CTX_PRIORITY_NORMAL : +return AMD_SCHED_PRIORITY_NORMAL ; +default : +WARN (1 ,"Invalid context priority %d\n" , amdgpu_priority); +return AMD_SCHED_PRIORITY_NORMAL ; + } +} +int amdgpu_ctx_ioctl (struct drm_device *dev,void *data,struct drm_file *filp) {int r;uint32_t id; +enum amd_sched_priority priority;union drm_amdgpu_ctx *args = data;struct amdgpu_device *adev = dev->dev_private;struct amdgpu_fpriv *fpriv = filp->driver_priv; r =0 ; id = args->in.ctx_id; + priority =amdgpu_to_sched_priority (args->in.priority); + +if (priority >=AMD_SCHED_PRIORITY_MAX ) +return -EINVAL ;switch (args->in.op) {case AMDGPU_CTX_OP_ALLOC_CTX : - r =amdgpu_ctx_alloc (adev, fpriv, &id); + r =amdgpu_ctx_alloc (adev, fpriv, priority, &id); args->out.alloc.ctx_id = id;break ;case AMDGPU_CTX_OP_FREE_CTX : r =amdgpu_ctx_free (fpriv, id);break ;case AMDGPU_CTX_OP_QUERY_STATE : r =amdgpu_ctx_query (adev, fpriv, id, &args->out);break ;default :return -EINVAL ; }return r; }struct amdgpu_ctx *amdgpu_ctx_get (struct amdgpu_fpriv *fpriv,uint32_t id) {struct amdgpu_ctx *ctx;struct amdgpu_ctx_mgr *mgr;
drivers/gpu/drm/amd/scheduler/gpu_scheduler.h +1 -0
@@ -93,40 +93,41 @@ static inline struct amd_sched_fence *to_amd_sched_fence(struct dma_fence *f)if (f->ops == &amd_sched_fence_ops_finished) return container_of(f, struct amd_sched_fence, finished);return NULL ; }/** * Define the backend operations called by the scheduler, * these functions should be implemented in driver side */ struct amd_sched_backend_ops {struct dma_fence *(*dependency)(struct amd_sched_job *sched_job);struct dma_fence *(*run_job)(struct amd_sched_job *sched_job);void (*timedout_job)(struct amd_sched_job *sched_job);void (*free_job)(struct amd_sched_job *sched_job); };enum amd_sched_priority {AMD_SCHED_PRIORITY_MIN ,AMD_SCHED_PRIORITY_NORMAL =AMD_SCHED_PRIORITY_MIN , +AMD_SCHED_PRIORITY_HIGH ,AMD_SCHED_PRIORITY_KERNEL ,AMD_SCHED_PRIORITY_MAX };/** * One scheduler is implemented for each hardware ring */ struct amd_gpu_scheduler {const struct amd_sched_backend_ops *ops;uint32_t hw_submission_limit;long timeout;const char *name;struct amd_sched_rq sched_rq[AMD_SCHED_PRIORITY_MAX ];wait_queue_head_t wake_up_worker;wait_queue_head_t job_scheduled;atomic_t hw_rq_count;atomic64_t job_id_count;struct task_struct *thread;struct list_head ring_mirror_list;spinlock_t job_list_lock;
include/uapi/drm/amdgpu_drm.h +7 -1
@@ -143,47 +143,53 @@ struct drm_amdgpu_bo_list_out {union drm_amdgpu_bo_list {struct drm_amdgpu_bo_list_in in;struct drm_amdgpu_bo_list_out out; };/* context related */ #define AMDGPU_CTX_OP_ALLOC_CTX 1#define AMDGPU_CTX_OP_FREE_CTX 2#define AMDGPU_CTX_OP_QUERY_STATE 3/* GPU reset status */ #define AMDGPU_CTX_NO_RESET 0/* this the context caused it */ #define AMDGPU_CTX_GUILTY_RESET 1/* some other context caused it */ #define AMDGPU_CTX_INNOCENT_RESET 2/* unknown cause */ #define AMDGPU_CTX_UNKNOWN_RESET 3 +/* Context priority level */ +#define AMDGPU_CTX_PRIORITY_NORMAL 0 +/* Selecting a priority above NORMAL requires CAP_SYS_ADMIN */ +#define AMDGPU_CTX_PRIORITY_HIGH 1 +#define AMDGPU_CTX_PRIORITY_NUM 2 +struct drm_amdgpu_ctx_in {/** AMDGPU_CTX_OP_* */ __u32 op;/** For future use, no flags defined so far */ __u32 flags;__u32 ctx_id; -__u32 _pad; +__u32 priority; };union drm_amdgpu_ctx_out {struct {__u32 ctx_id;__u32 _pad; } alloc;struct {/** For future use, no flags defined so far */ __u64 flags;/** Number of resets caused by this context so far. */ __u32 hangs;/** Reset status since the last call of the ioctl. */ __u32 reset_status; } state; };union drm_amdgpu_ctx {struct drm_amdgpu_ctx_in in;
Signature · 6 lines
2.9.3 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 20170413223024.21868-2-andresx7-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Re: [PATCH 1/4] drm/amdgpu: add parameter to allocate high priority contexts v7
MrCooper just pointed me to the following EGL extension last night:
https://www.khronos.org/registry/EGL/extensions/IMG/EGL_IMG_context_priority.txt
I think it might be a good idea to add an AMDGPU_CTX_PRIORITY_LOW to the kernel ABI in this patch in order to accommodate that extension.
Regards,
Andres
Quoted 339 lines · Andres Rodriguez
Add a new context creation parameter to express a global context priority.
Contexts allocated with
AMDGPU_CTX_PRIORITY_HIGHwill receive higher priority to schedule their work thanAMDGPU_CTX_PRIORITY_NORMAL(default) contexts.v2: Instead of using flags, repurpose
__pad
v3: Swap enum values of_NORMAL_HIGHfor backwards compatibility
v4: Validate usermode priority and store it
v5: Move priority validation intoamdgpu_ctx_ioctl(), headline reword
v6: add UAPI note regarding priorities requiringCAP_SYS_ADMIN
v7: removectx->priorityReviewed-by Emil Velikov <[email protected]>Reviewed-by Christian König <[email protected]>Signed-off-by Andres Rodriguez <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c | 36 ++++++++++++++++++++++++--- drivers/gpu/drm/amd/scheduler/gpu_scheduler.h | 1 + include/uapi/drm/amdgpu_drm.h | 8 +++++- 3 files changed, 40 insertions(+), 5 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +32
-4@@ -8,67 +8,75 @@* and/or sell copies of the Software, and to permit persons to whom the * Software is furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in * all copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR * OTHER DEALINGS IN THE SOFTWARE. * * Authors: monk liu <[email protected]> */ #include <drm/drmP.h> #include "amdgpu.h" -static int amdgpu_ctx_init (struct amdgpu_device *adev,struct amdgpu_ctx *ctx) +static int amdgpu_ctx_init (struct amdgpu_device *adev, +enum amd_sched_priority priority, +struct amdgpu_ctx *ctx) {unsigned i, j;int r; +if (priority <0 || priority >=AMD_SCHED_PRIORITY_MAX ) +return -EINVAL ; + +if (priority ==AMD_SCHED_PRIORITY_HIGH && !capable (CAP_SYS_ADMIN )) +return -EACCES ; +memset (ctx,0 ,sizeof (*ctx)); ctx->adev = adev;kref_init (&ctx->refcount);spin_lock_init (&ctx->ring_lock); ctx->fences =kcalloc (amdgpu_sched_jobs *AMDGPU_MAX_RINGS ,sizeof (struct dma_fence *),GFP_KERNEL );if (!ctx->fences)return -ENOMEM ;for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) { ctx->rings[i].sequence =1 ; ctx->rings[i].fences = &ctx->fences[amdgpu_sched_jobs * i]; } ctx->reset_counter =atomic_read (&adev->gpu_reset_counter);/* create context entity for each ring */ for (i =0 ; i < adev->num_rings; i++) {struct amdgpu_ring *ring = adev->rings[i];struct amd_sched_rq *rq; - rq = &ring->sched.sched_rq[AMD_SCHED_PRIORITY_NORMAL ]; + rq = &ring->sched.sched_rq[priority]; r =amd_sched_entity_init (&ring->sched, &ctx->rings[i].entity, rq, amdgpu_sched_jobs);if (r) goto failed; } r =amdgpu_queue_mgr_init (adev, &ctx->queue_mgr);if (r) goto failed;return 0 ; failed:for (j =0 ; j < i; j++)amd_sched_entity_fini (&adev->rings[j]->sched, &ctx->rings[j].entity);kfree (ctx->fences); ctx->fences =NULL ;return r; }@@ -79,59 +87,61 @@ static void amdgpu_ctx_fini(struct amdgpu_ctx *ctx)unsigned i, j;if (!adev)return ;for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i)for (j =0 ; j < amdgpu_sched_jobs; ++j)dma_fence_put (ctx->rings[i].fences[j]);kfree (ctx->fences); ctx->fences =NULL ;for (i =0 ; i < adev->num_rings; i++)amd_sched_entity_fini (&adev->rings[i]->sched, &ctx->rings[i].entity);amdgpu_queue_mgr_fini (adev, &ctx->queue_mgr); }static int amdgpu_ctx_alloc (struct amdgpu_device *adev,struct amdgpu_fpriv *fpriv, +enum amd_sched_priority priority,uint32_t *id) {struct amdgpu_ctx_mgr *mgr = &fpriv->ctx_mgr;struct amdgpu_ctx *ctx;int r; ctx =kmalloc (sizeof (*ctx),GFP_KERNEL );if (!ctx)return -ENOMEM ;mutex_lock (&mgr->lock); r =idr_alloc (&mgr->ctx_handles, ctx,1 ,0 ,GFP_KERNEL );if (r <0 ) {mutex_unlock (&mgr->lock);kfree (ctx);return r; } + *id = (uint32_t )r; - r =amdgpu_ctx_init (adev, ctx); + r =amdgpu_ctx_init (adev, priority, ctx);if (r) {idr_remove (&mgr->ctx_handles, *id); *id =0 ;kfree (ctx); }mutex_unlock (&mgr->lock);return r; }static void amdgpu_ctx_do_release (struct kref *ref) {struct amdgpu_ctx *ctx; ctx = container_of(ref, struct amdgpu_ctx, refcount);amdgpu_ctx_fini (ctx);kfree (ctx); }@@ -167,56 +177,74 @@ static int amdgpu_ctx_query(struct amdgpu_device *adev,return -EINVAL ; }/* TODO: these two are always zero */ out->state.flags =0x0 ; out->state.hangs =0x0 ;/* determine if a GPU reset has occured since the last call */ reset_counter =atomic_read (&adev->gpu_reset_counter);/* TODO: this should ideally return NO, GUILTY, or INNOCENT. */ if (ctx->reset_counter == reset_counter) out->state.reset_status =AMDGPU_CTX_NO_RESET ;else out->state.reset_status =AMDGPU_CTX_UNKNOWN_RESET ; ctx->reset_counter = reset_counter;mutex_unlock (&mgr->lock);return 0 ; } +static enum amd_sched_priority amdgpu_to_sched_priority (int amdgpu_priority) +{ +switch (amdgpu_priority) { +case AMDGPU_CTX_PRIORITY_HIGH : +return AMD_SCHED_PRIORITY_HIGH ; +case AMDGPU_CTX_PRIORITY_NORMAL : +return AMD_SCHED_PRIORITY_NORMAL ; +default : +WARN (1 ,"Invalid context priority %d\n" , amdgpu_priority); +return AMD_SCHED_PRIORITY_NORMAL ; + } +} +int amdgpu_ctx_ioctl (struct drm_device *dev,void *data,struct drm_file *filp) {int r;uint32_t id; +enum amd_sched_priority priority;union drm_amdgpu_ctx *args = data;struct amdgpu_device *adev = dev->dev_private;struct amdgpu_fpriv *fpriv = filp->driver_priv; r =0 ; id = args->in.ctx_id; + priority =amdgpu_to_sched_priority (args->in.priority); + +if (priority >=AMD_SCHED_PRIORITY_MAX ) +return -EINVAL ;switch (args->in.op) {case AMDGPU_CTX_OP_ALLOC_CTX : - r =amdgpu_ctx_alloc (adev, fpriv, &id); + r =amdgpu_ctx_alloc (adev, fpriv, priority, &id); args->out.alloc.ctx_id = id;break ;case AMDGPU_CTX_OP_FREE_CTX : r =amdgpu_ctx_free (fpriv, id);break ;case AMDGPU_CTX_OP_QUERY_STATE : r =amdgpu_ctx_query (adev, fpriv, id, &args->out);break ;default :return -EINVAL ; }return r; }struct amdgpu_ctx *amdgpu_ctx_get (struct amdgpu_fpriv *fpriv,uint32_t id) {struct amdgpu_ctx *ctx;struct amdgpu_ctx_mgr *mgr;drivers/gpu/drm/amd/scheduler/gpu_scheduler.h +1
-0@@ -93,40 +93,41 @@ static inline struct amd_sched_fence *to_amd_sched_fence(struct dma_fence *f)if (f->ops == &amd_sched_fence_ops_finished) return container_of(f, struct amd_sched_fence, finished);return NULL ; }/** * Define the backend operations called by the scheduler, * these functions should be implemented in driver side */ struct amd_sched_backend_ops {struct dma_fence *(*dependency)(struct amd_sched_job *sched_job);struct dma_fence *(*run_job)(struct amd_sched_job *sched_job);void (*timedout_job)(struct amd_sched_job *sched_job);void (*free_job)(struct amd_sched_job *sched_job); };enum amd_sched_priority {AMD_SCHED_PRIORITY_MIN ,AMD_SCHED_PRIORITY_NORMAL =AMD_SCHED_PRIORITY_MIN , +AMD_SCHED_PRIORITY_HIGH ,AMD_SCHED_PRIORITY_KERNEL ,AMD_SCHED_PRIORITY_MAX };/** * One scheduler is implemented for each hardware ring */ struct amd_gpu_scheduler {const struct amd_sched_backend_ops *ops;uint32_t hw_submission_limit;long timeout;const char *name;struct amd_sched_rq sched_rq[AMD_SCHED_PRIORITY_MAX ];wait_queue_head_t wake_up_worker;wait_queue_head_t job_scheduled;atomic_t hw_rq_count;atomic64_t job_id_count;struct task_struct *thread;struct list_head ring_mirror_list;spinlock_t job_list_lock;include/uapi/drm/amdgpu_drm.h +7
-1@@ -143,47 +143,53 @@ struct drm_amdgpu_bo_list_out {union drm_amdgpu_bo_list {struct drm_amdgpu_bo_list_in in;struct drm_amdgpu_bo_list_out out; };/* context related */ #define AMDGPU_CTX_OP_ALLOC_CTX 1#define AMDGPU_CTX_OP_FREE_CTX 2#define AMDGPU_CTX_OP_QUERY_STATE 3/* GPU reset status */ #define AMDGPU_CTX_NO_RESET 0/* this the context caused it */ #define AMDGPU_CTX_GUILTY_RESET 1/* some other context caused it */ #define AMDGPU_CTX_INNOCENT_RESET 2/* unknown cause */ #define AMDGPU_CTX_UNKNOWN_RESET 3 +/* Context priority level */ +#define AMDGPU_CTX_PRIORITY_NORMAL 0 +/* Selecting a priority above NORMAL requires CAP_SYS_ADMIN */ +#define AMDGPU_CTX_PRIORITY_HIGH 1 +#define AMDGPU_CTX_PRIORITY_NUM 2 +struct drm_amdgpu_ctx_in {/** AMDGPU_CTX_OP_* */ __u32 op;/** For future use, no flags defined so far */ __u32 flags;__u32 ctx_id; -__u32 _pad; +__u32 priority; };union drm_amdgpu_ctx_out {struct {__u32 ctx_id;__u32 _pad; } alloc;struct {/** For future use, no flags defined so far */
include/uapi/drm/amdgpu_drm.h struct drm_amdgpu_bo_list_out { · line 195
Quoted 10 lines
include/uapi/drm/amdgpu_drm.h +0
-0__u64 flags;/** Number of resets caused by this context so far. */ __u32 hangs;/** Reset status since the last call of the ioctl. */ __u32 reset_status; } state; };union drm_amdgpu_ctx {struct drm_amdgpu_ctx_in in;
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): d872c6f6-e285-d9f3-b562-a11b62e5b28f-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Re: [PATCH 1/4] drm/amdgpu: add parameter to allocate high priority contexts v7
Quoted 7 lines · Andres Rodriguez
MrCooper just pointed me to the following EGL extension last night:
https://www.khronos.org/registry/EGL/extensions/IMG/EGL_IMG_context_priority.txtI think it might be a good idea to add an
AMDGPU_CTX_PRIORITY_LOWto the kernel ABI in this patch in order to accommodate that extension.
Also, the query mechanism can be implemented fully in usermode. I don't think we would require any API in the kernel to return the real priority.
If a user requests a priority that is unsupported we can just return an error and usermode can retry creating the context with a different priority.
Quoted 365 lines
Regards,
AndresQuoted 361 lines · Andres Rodriguez
Add a new context creation parameter to express a global context priority.
Contexts allocated with
AMDGPU_CTX_PRIORITY_HIGHwill receive higher priority to schedule their work thanAMDGPU_CTX_PRIORITY_NORMAL(default) contexts.v2: Instead of using flags, repurpose
__pad
v3: Swap enum values of_NORMAL_HIGHfor backwards compatibility
v4: Validate usermode priority and store it
v5: Move priority validation intoamdgpu_ctx_ioctl(), headline reword
v6: add UAPI note regarding priorities requiringCAP_SYS_ADMIN
v7: removectx->priorityReviewed-by Emil Velikov <[email protected]>Reviewed-by Christian König <[email protected]>Signed-off-by Andres Rodriguez <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c | 36 ++++++++++++++++++++++++--- drivers/gpu/drm/amd/scheduler/gpu_scheduler.h | 1 + include/uapi/drm/amdgpu_drm.h | 8 +++++- 3 files changed, 40 insertions(+), 5 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +0
-0@@ -8,67 +8,75 @@* and/or sell copies of the Software, and to permit persons to whom the * Software is furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL
* THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM,
DAMAGES OR
* OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*
* Authors: monk liu <[email protected]>
*/#include <
drm/drmP.h>
#include "amdgpu.h"-static int
amdgpu_ctx_init(struct amdgpu_device *adev, struct amdgpu_ctx *ctx)drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +32
-3+static int amdgpu_ctx_init (struct amdgpu_device *adev, +enum amd_sched_priority priority, +struct amdgpu_ctx *ctx) {unsigned i, j;int r; +if (priority <0 || priority >=AMD_SCHED_PRIORITY_MAX ) +return -EINVAL ; + +if (priority ==AMD_SCHED_PRIORITY_HIGH && !capable (CAP_SYS_ADMIN )) +return -EACCES ; +memset (ctx,0 ,sizeof (*ctx)); ctx->adev = adev;kref_init (&ctx->refcount);spin_lock_init (&ctx->ring_lock); ctx->fences =kcalloc (amdgpu_sched_jobs *AMDGPU_MAX_RINGS ,sizeof (struct dma_fence *),GFP_KERNEL );if (!ctx->fences)return -ENOMEM ;for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) { ctx->rings[i].sequence =1 ; ctx->rings[i].fences = &ctx->fences[amdgpu_sched_jobs * i]; } ctx->reset_counter =atomic_read (&adev->gpu_reset_counter);/* create context entity for each ring */ for (i =0 ; i < adev->num_rings; i++) {struct amdgpu_ring *ring = adev->rings[i];struct amd_sched_rq *rq; - rq = &ring->sched.sched_rq[AMD_SCHED_PRIORITY_NORMAL ]; + rq = &ring->sched.sched_rq[priority]; r =amd_sched_entity_init (&ring->sched, &ctx->rings[i].entity, rq, amdgpu_sched_jobs);if (r) goto failed; } r =amdgpu_queue_mgr_init (adev, &ctx->queue_mgr);if (r) goto failed;return 0 ; failed:for (j =0 ; j < i; j++)amd_sched_entity_fini (&adev->rings[j]->sched, &ctx->rings[j].entity);kfree (ctx->fences); ctx->fences =NULL ;return r; }@@ -79,59 +87,61 @@ static void amdgpu_ctx_fini(struct amdgpu_ctx *ctx)unsigned i, j;if (!adev)return ;for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i)for (j =0 ; j < amdgpu_sched_jobs; ++j)dma_fence_put (ctx->rings[i].fences[j]);kfree (ctx->fences); ctx->fences =NULL ;for (i =0 ; i < adev->num_rings; i++)amd_sched_entity_fini (&adev->rings[i]->sched, &ctx->rings[i].entity);amdgpu_queue_mgr_fini (adev, &ctx->queue_mgr); }static int amdgpu_ctx_alloc (struct amdgpu_device *adev,struct amdgpu_fpriv *fpriv, +enum amd_sched_priority priority,uint32_t *id) {struct amdgpu_ctx_mgr *mgr = &fpriv->ctx_mgr;struct amdgpu_ctx *ctx;int r; ctx =kmalloc (sizeof (*ctx),GFP_KERNEL );if (!ctx)return -ENOMEM ;mutex_lock (&mgr->lock); r =idr_alloc (&mgr->ctx_handles, ctx,1 ,0 ,GFP_KERNEL );if (r <0 ) {mutex_unlock (&mgr->lock);kfree (ctx);return r; } + *id = (uint32_t )r; - r =amdgpu_ctx_init (adev, ctx); + r =amdgpu_ctx_init (adev, priority, ctx);if (r) {idr_remove (&mgr->ctx_handles, *id); *id =0 ;kfree (ctx); }mutex_unlock (&mgr->lock);return r; }static void amdgpu_ctx_do_release (struct kref *ref) {struct amdgpu_ctx *ctx; ctx = container_of(ref, struct amdgpu_ctx, refcount);amdgpu_ctx_fini (ctx);kfree (ctx); }@@ -167,56 +177,74 @@ static int amdgpu_ctx_query(struct amdgpu_device *adev, return -EINVAL ; }/* TODO: these two are always zero */ out->state.flags =0x0 ; out->state.hangs =0x0 ;/* determine if a GPU reset has occured since the last call */ reset_counter =atomic_read (&adev->gpu_reset_counter);/* TODO: this should ideally return NO, GUILTY, or INNOCENT. */ if (ctx->reset_counter == reset_counter) out->state.reset_status =AMDGPU_CTX_NO_RESET ;else out->state.reset_status =AMDGPU_CTX_UNKNOWN_RESET ; ctx->reset_counter = reset_counter;mutex_unlock (&mgr->lock);return 0 ; } +static enum amd_sched_priority amdgpu_to_sched_priority (int amdgpu_priority) +{ +switch (amdgpu_priority) { +case AMDGPU_CTX_PRIORITY_HIGH : +return AMD_SCHED_PRIORITY_HIGH ; +case AMDGPU_CTX_PRIORITY_NORMAL : +return AMD_SCHED_PRIORITY_NORMAL ; +default : +WARN (1 ,"Invalid context priority %d\n" , amdgpu_priority); +return AMD_SCHED_PRIORITY_NORMAL ; + } +} +int amdgpu_ctx_ioctl (struct drm_device *dev,void *data,struct drm_file *filp) {int r;uint32_t id; +enum amd_sched_priority priority;union drm_amdgpu_ctx *args = data;struct amdgpu_device *adev = dev->dev_private;struct amdgpu_fpriv *fpriv = filp->driver_priv; r =0 ; id = args->in.ctx_id; + priority =amdgpu_to_sched_priority (args->in.priority); + +if (priority >=AMD_SCHED_PRIORITY_MAX ) +return -EINVAL ;switch (args->in.op) {case AMDGPU_CTX_OP_ALLOC_CTX : - r =amdgpu_ctx_alloc (adev, fpriv, &id); + r =amdgpu_ctx_alloc (adev, fpriv, priority, &id); args->out.alloc.ctx_id = id;break ;case AMDGPU_CTX_OP_FREE_CTX : r =amdgpu_ctx_free (fpriv, id);break ;case AMDGPU_CTX_OP_QUERY_STATE : r =amdgpu_ctx_query (adev, fpriv, id, &args->out);break ;default :return -EINVAL ; }return r; }struct amdgpu_ctx *amdgpu_ctx_get (struct amdgpu_fpriv *fpriv,uint32_t id) {struct amdgpu_ctx *ctx; struct amdgpu_ctx_mgr *mgr;drivers/gpu/drm/amd/scheduler/gpu_scheduler.h +1
-0@@ -93,40 +93,41 @@ static inline struct amd_sched_fence *to_amd_sched_fence(struct dma_fence *f)if (f->ops == &amd_sched_fence_ops_finished) return container_of(f, struct amd_sched_fence, finished);return NULL ; }/** * Define the backend operations called by the scheduler, * these functions should be implemented in driver side */ struct amd_sched_backend_ops {struct dma_fence *(*dependency)(struct amd_sched_job *sched_job);struct dma_fence *(*run_job)(struct amd_sched_job *sched_job);void (*timedout_job)(struct amd_sched_job *sched_job);void (*free_job)(struct amd_sched_job *sched_job); };enum amd_sched_priority {AMD_SCHED_PRIORITY_MIN ,AMD_SCHED_PRIORITY_NORMAL =AMD_SCHED_PRIORITY_MIN , +AMD_SCHED_PRIORITY_HIGH ,AMD_SCHED_PRIORITY_KERNEL ,AMD_SCHED_PRIORITY_MAX };/** * One scheduler is implemented for each hardware ring */ struct amd_gpu_scheduler {const struct amd_sched_backend_ops *ops;uint32_t hw_submission_limit;long timeout;const char *name;struct amd_sched_rq sched_rq[AMD_SCHED_PRIORITY_MAX ];wait_queue_head_t wake_up_worker;wait_queue_head_t job_scheduled;atomic_t hw_rq_count;atomic64_t job_id_count;struct task_struct *thread;struct list_head ring_mirror_list;spinlock_t job_list_lock;include/uapi/drm/amdgpu_drm.h +7
-1@@ -143,47 +143,53 @@ struct drm_amdgpu_bo_list_out {union drm_amdgpu_bo_list {struct drm_amdgpu_bo_list_in in;struct drm_amdgpu_bo_list_out out; };/* context related */ #define AMDGPU_CTX_OP_ALLOC_CTX 1#define AMDGPU_CTX_OP_FREE_CTX 2#define AMDGPU_CTX_OP_QUERY_STATE 3/* GPU reset status */ #define AMDGPU_CTX_NO_RESET 0/* this the context caused it */ #define AMDGPU_CTX_GUILTY_RESET 1/* some other context caused it */ #define AMDGPU_CTX_INNOCENT_RESET 2/* unknown cause */ #define AMDGPU_CTX_UNKNOWN_RESET 3 +/* Context priority level */ +#define AMDGPU_CTX_PRIORITY_NORMAL 0 +/* Selecting a priority above NORMAL requires CAP_SYS_ADMIN */ +#define AMDGPU_CTX_PRIORITY_HIGH 1 +#define AMDGPU_CTX_PRIORITY_NUM 2 +struct drm_amdgpu_ctx_in {/** AMDGPU_CTX_OP_* */ __u32 op;/** For future use, no flags defined so far */ __u32 flags;__u32 ctx_id; -__u32 _pad; +__u32 priority; };union drm_amdgpu_ctx_out {struct {__u32 ctx_id;__u32 _pad; } alloc;struct {/** For future use, no flags defined so far */ __u64 flags;/** Number of resets caused by this context so far. */ __u32 hangs;/** Reset status since the last call of the ioctl. */ __u32 reset_status; } state; };
Quoted 3 lines
include/uapi/drm/amdgpu_drm.h struct drm_amdgpu_bo_list_out { · line 195
Quoted 3 lines
include/uapi/drm/amdgpu_drm.h +0
-0union drm_amdgpu_ctx {struct drm_amdgpu_ctx_in in;
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 2/4 drm/amdgpu: add framework for HW specific priority settings v6
In reply to a message not in the archiveAdd an initial framework for changing the HW priorities of rings. The framework allows requesting priority changes for the lifetime of an amdgpu_job. After the job completes the priority will decay to the next lowest priority for which a request is still valid.
A new ring function set_priority() can now be populated to take care of the HW specific programming sequence for priority changes.
v2: set priority before emitting IB, and take a ref on amdgpu_job
v3: use AMD_SCHED_PRIORITY_* instead of AMDGPU_CTX_PRIORITY_*
v4: plug amdgpu_ring_restore_priority_cb into amdgpu_job_free_cb
v5: use atomic for tracking job priorities instead of last_job
v6: rename amdgpu_ring_priority_[get/put]() and align parameters
--- drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 7 +++ drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c | 78 ++++++++++++++++++++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 15 ++++++ drivers/gpu/drm/amd/scheduler/gpu_scheduler.h | 7 +++ 4 files changed, 106 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +7 -0
@@ -78,40 +78,41 @@ int amdgpu_job_alloc_with_ib(struct amdgpu_device *adev, unsigned size,return r; }void amdgpu_job_free_resources (struct amdgpu_job *job) {struct dma_fence *f;unsigned i;/* use sched fence if available */ f = job->base.s_fence ? &job->base.s_fence->finished : job->fence;for (i =0 ; i < job->num_ibs; ++i)amdgpu_ib_free (job->adev, &job->ibs[i], f); }static void amdgpu_job_free_cb (struct amd_sched_job *s_job) { struct amdgpu_job *job = container_of(s_job, struct amdgpu_job, base); +amdgpu_ring_priority_put (job->ring,amd_sched_get_job_priority (s_job));dma_fence_put (job->fence);amdgpu_sync_free (&job->sync);kfree (job); }void amdgpu_job_free (struct amdgpu_job *job) {amdgpu_job_free_resources (job);dma_fence_put (job->fence);amdgpu_sync_free (&job->sync);kfree (job); }int amdgpu_job_submit (struct amdgpu_job *job,struct amdgpu_ring *ring,struct amd_sched_entity *entity,void *owner,struct dma_fence **f) {int r; job->ring = ring;
@@ -152,38 +153,44 @@ static struct dma_fence *amdgpu_job_dependency(struct amd_sched_job *sched_job) fence =amdgpu_sync_get_fence (&job->sync); }return fence; }static struct dma_fence *amdgpu_job_run (struct amd_sched_job *sched_job) {struct dma_fence *fence =NULL ;struct amdgpu_job *job;int r;if (!sched_job) {DRM_ERROR ("job is null\n" );return NULL ; } job =to_amdgpu_job (sched_job);BUG_ON (amdgpu_sync_peek_fence (&job->sync,NULL )); + r =amdgpu_ring_priority_get (job->ring, +amd_sched_get_job_priority (&job->base)); +if (r) +DRM_ERROR ("Failed to set job priority (%d)\n" , r); +trace_amdgpu_sched_run_job (job); r =amdgpu_ib_schedule (job->ring, job->num_ibs, job->ibs, job, &fence);if (r)DRM_ERROR ("Error scheduling IBs (%d)\n" , r);/* if gpu reset, hw fence will be replaced here */ dma_fence_put (job->fence); job->fence =dma_fence_get (fence); +amdgpu_job_free_resources (job);return fence; }const struct amd_sched_backend_ops amdgpu_sched_ops = { .dependency = amdgpu_job_dependency, .run_job = amdgpu_job_run, .timedout_job = amdgpu_job_timedout, .free_job = amdgpu_job_free_cb };
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c +77 -1
175 lines not on this page · raw message
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +15 -0
@@ -7,70 +7,72 @@* the rights to use, copy, modify, merge, publish, distribute, sublicense, * and/or sell copies of the Software, and to permit persons to whom the * Software is furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in * all copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR * OTHER DEALINGS IN THE SOFTWARE. * * Authors: Christian König */ #ifndef __AMDGPU_RING_H__#define __AMDGPU_RING_H__ +#include <drm/amdgpu_drm.h> #include "gpu_scheduler.h" /* max number of rings */ #define AMDGPU_MAX_RINGS 18#define AMDGPU_MAX_GFX_RINGS 1#define AMDGPU_MAX_COMPUTE_RINGS 8#define AMDGPU_MAX_VCE_RINGS 3#define AMDGPU_MAX_UVD_ENC_RINGS 2/* some special values for the owner field */ #define AMDGPU_FENCE_OWNER_UNDEFINED ((void*)0ul)#define AMDGPU_FENCE_OWNER_VM ((void*)1ul)#define AMDGPU_FENCE_FLAG_64BIT (1 << 0)#define AMDGPU_FENCE_FLAG_INT (1 << 1)enum amdgpu_ring_type {AMDGPU_RING_TYPE_GFX ,AMDGPU_RING_TYPE_COMPUTE ,AMDGPU_RING_TYPE_SDMA ,AMDGPU_RING_TYPE_UVD ,AMDGPU_RING_TYPE_VCE ,AMDGPU_RING_TYPE_KIQ ,AMDGPU_RING_TYPE_UVD_ENC };struct amdgpu_device ;struct amdgpu_ring ;struct amdgpu_ib ;struct amdgpu_cs_parser ; +struct amdgpu_job ;/* * Fences. */ struct amdgpu_fence_driver {uint64_t gpu_addr;volatile uint32_t *cpu_addr;/* sync_seq is protected by ring emission lock */ uint32_t sync_seq;atomic_t last_seq;bool initialized;struct amdgpu_irq_src *irq_src;unsigned irq_type;struct timer_list fallback_timer;unsigned num_fences_mask;spinlock_t lock;struct dma_fence **fences; };int amdgpu_fence_driver_init (struct amdgpu_device *adev);
@@ -125,87 +127,100 @@ struct amdgpu_ring_funcs { uint32_t gds_base, uint32_t gds_size, uint32_t gws_base, uint32_t gws_size, uint32_t oa_base, uint32_t oa_size);/* testing functions */ int (*test_ring)(struct amdgpu_ring *ring);int (*test_ib)(struct amdgpu_ring *ring,long timeout);/* insert NOP packets */ void (*insert_nop)(struct amdgpu_ring *ring,uint32_t count);void (*insert_end)(struct amdgpu_ring *ring);/* pad the indirect buffer to the necessary number of dw */ void (*pad_ib)(struct amdgpu_ring *ring,struct amdgpu_ib *ib);unsigned (*init_cond_exec)(struct amdgpu_ring *ring);void (*patch_cond_exec)(struct amdgpu_ring *ring,unsigned offset);/* note usage for clock and power gating */ void (*begin_use)(struct amdgpu_ring *ring);void (*end_use)(struct amdgpu_ring *ring);void (*emit_switch_buffer) (struct amdgpu_ring *ring);void (*emit_cntxcntl) (struct amdgpu_ring *ring,uint32_t flags);void (*emit_rreg)(struct amdgpu_ring *ring,uint32_t reg);void (*emit_wreg)(struct amdgpu_ring *ring,uint32_t reg,uint32_t val); +/* priority functions */ +void (*set_priority) (struct amdgpu_ring *ring, +enum amd_sched_priority priority); };struct amdgpu_ring {struct amdgpu_device *adev;const struct amdgpu_ring_funcs *funcs;struct amdgpu_fence_driver fence_drv;struct amd_gpu_scheduler sched;struct list_head lru_list;struct amdgpu_bo *ring_obj;volatile uint32_t *ring;unsigned rptr_offs;u64 wptr;u64 wptr_old;unsigned ring_size;unsigned max_dw;int count_dw;uint64_t gpu_addr;uint64_t ptr_mask;uint32_t buf_mask;bool ready;u32 idx;u32 me;u32 pipe;u32 queue;struct amdgpu_bo *mqd_obj;uint64_t mqd_gpu_addr;void *mqd_ptr;uint64_t eop_gpu_addr;u32 doorbell_index;bool use_doorbell;unsigned wptr_offs;unsigned fence_offs;uint64_t current_ctx;char name[16 ];unsigned cond_exe_offs;u64 cond_exe_gpu_addr;volatile u32 *cond_exe_cpu_addr;unsigned vm_inv_eng; + +atomic_t num_jobs[AMD_SCHED_PRIORITY_MAX ]; +spinlock_t priority_lock; +/* protected by priority_lock */ +int priority; +#if defined(CONFIG_DEBUG_FS )struct dentry *ent;#endif };int amdgpu_ring_is_valid_index (struct amdgpu_device *adev,int hw_ip,int ring);int amdgpu_ring_alloc (struct amdgpu_ring *ring,unsigned ndw);void amdgpu_ring_insert_nop (struct amdgpu_ring *ring,uint32_t count);void amdgpu_ring_generic_pad_ib (struct amdgpu_ring *ring,struct amdgpu_ib *ib);void amdgpu_ring_commit (struct amdgpu_ring *ring);void amdgpu_ring_undo (struct amdgpu_ring *ring); +int amdgpu_ring_priority_get (struct amdgpu_ring *ring, +enum amd_sched_priority priority); +void amdgpu_ring_priority_put (struct amdgpu_ring *ring, +enum amd_sched_priority priority);int amdgpu_ring_init (struct amdgpu_device *adev,struct amdgpu_ring *ring,unsigned ring_size,struct amdgpu_irq_src *irq_src,unsigned irq_type);void amdgpu_ring_fini (struct amdgpu_ring *ring);int amdgpu_ring_lru_get (struct amdgpu_device *adev,int type,int *blacklist,int num_blacklist,struct amdgpu_ring **ring);void amdgpu_ring_lru_touch (struct amdgpu_device *adev,struct amdgpu_ring *ring);static inline void amdgpu_ring_clear_ring (struct amdgpu_ring *ring) {int i =0 ;while (i <= ring->buf_mask) ring->ring[i++] = ring->funcs->nop; }#endif
drivers/gpu/drm/amd/scheduler/gpu_scheduler.h +7 -0
@@ -142,21 +142,28 @@ int amd_sched_entity_init(struct amd_gpu_scheduler *sched,struct amd_sched_entity *entity, struct amd_sched_rq *rq, uint32_t jobs);void amd_sched_entity_fini (struct amd_gpu_scheduler *sched,struct amd_sched_entity *entity);void amd_sched_entity_push_job (struct amd_sched_job *sched_job);int amd_sched_fence_slab_init (void );void amd_sched_fence_slab_fini (void );struct amd_sched_fence *amd_sched_fence_create (struct amd_sched_entity *s_entity,void *owner);void amd_sched_fence_scheduled (struct amd_sched_fence *fence);void amd_sched_fence_finished (struct amd_sched_fence *fence);int amd_sched_job_init (struct amd_sched_job *job,struct amd_gpu_scheduler *sched,struct amd_sched_entity *entity,void *owner);void amd_sched_hw_job_reset (struct amd_gpu_scheduler *sched);void amd_sched_job_recovery (struct amd_gpu_scheduler *sched); + +static inline enum amd_sched_priority +amd_sched_get_job_priority (struct amd_sched_job *job) +{ +return (job->s_entity->rq - job->sched->sched_rq); +} +#endif
Signature · 6 lines
2.9.3 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 3/4 drm/amdgpu: convert srbm lock to a spinlock v2
In reply to a message not in the archiveReplace adev->srbm_mutex with a spinlock adev->srbm_lock
v2: rebased on 4.12 and included gfx9
--- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v7.c | 4 +-- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v8.c | 4 +-- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 2 +- drivers/gpu/drm/amd/amdgpu/cik_sdma.c | 4 +-- drivers/gpu/drm/amd/amdgpu/gfx_v7_0.c | 20 ++++++------- drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c | 34 +++++++++++------------ drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c | 24 ++++++++-------- drivers/gpu/drm/amd/amdgpu/sdma_v2_4.c | 4 +-- drivers/gpu/drm/amd/amdgpu/sdma_v3_0.c | 4 +-- 10 files changed, 51 insertions(+), 51 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +1 -1
@@ -1465,41 +1465,41 @@ struct amdgpu_device {enum amd_asic_type asic_type;uint32_t family;uint32_t rev_id;uint32_t external_rev_id;unsigned long flags;int usec_timeout;const struct amdgpu_asic_funcs *asic_funcs;bool shutdown;bool need_dma32;bool accel_working;struct work_struct reset_work;struct notifier_block acpi_nb;struct amdgpu_i2c_chan *i2c_bus[AMDGPU_MAX_I2C_BUS ];struct amdgpu_debugfs debugfs[AMDGPU_DEBUGFS_MAX_COMPONENTS ];unsigned debugfs_count;#if defined(CONFIG_DEBUG_FS )struct dentry *debugfs_regs[AMDGPU_DEBUGFS_MAX_COMPONENTS ];#endif struct amdgpu_atif atif;struct amdgpu_atcs atcs; -struct mutex srbm_mutex; +spinlock_t srbm_lock;/* GRBM index mutex. Protects concurrent access to GRBM index */ struct mutex grbm_idx_mutex;struct dev_pm_domain vga_pm_domain;bool have_disp_power_ref;/* BIOS */ bool is_atom_fw;uint8_t *bios;uint32_t bios_size;struct amdgpu_bo *stollen_vga_memory;uint32_t bios_scratch_reg_offset;uint32_t bios_scratch[AMDGPU_BIOS_NUM_SCRATCH ];/* Register/doorbell mmio */ resource_size_t rmmio_base;resource_size_t rmmio_size;void __iomem *rmmio;/* protects concurrent MM_INDEX/DATA based register access */ spinlock_t mmio_idx_lock;/* protects concurrent SMC based register access */
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v7.c +2 -2
@@ -152,50 +152,50 @@ static const struct kfd2kgd_calls kfd2kgd = { .write_vmid_invalidate_request = write_vmid_invalidate_request, .get_fw_version = get_fw_version };struct kfd2kgd_calls *amdgpu_amdkfd_gfx_7_get_functions (void ) {return (struct kfd2kgd_calls *)&kfd2kgd; }static inline struct amdgpu_device *get_amdgpu_device (struct kgd_dev *kgd) {return (struct amdgpu_device *)kgd; }static void lock_srbm (struct kgd_dev *kgd,uint32_t mec,uint32_t pipe,uint32_t queue,uint32_t vmid) {struct amdgpu_device *adev =get_amdgpu_device (kgd);uint32_t value =PIPEID (pipe) |MEID (mec) |VMID (vmid) |QUEUEID (queue); -mutex_lock (&adev->srbm_mutex); +spin_lock (&adev->srbm_lock);WREG32 (mmSRBM_GFX_CNTL, value); }static void unlock_srbm (struct kgd_dev *kgd) {struct amdgpu_device *adev =get_amdgpu_device (kgd);WREG32 (mmSRBM_GFX_CNTL,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock); }static void acquire_queue (struct kgd_dev *kgd,uint32_t pipe_id,uint32_t queue_id) {struct amdgpu_device *adev =get_amdgpu_device (kgd);uint32_t mec = (++pipe_id / adev->gfx.mec.num_pipe_per_mec) +1 ;uint32_t pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec);lock_srbm (kgd, mec, pipe, queue_id,0 ); }static void release_queue (struct kgd_dev *kgd) {unlock_srbm (kgd); } static void kgd_program_sh_mem_settings(struct kgd_dev *kgd, uint32_t vmid, uint32_t sh_mem_config,
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v8.c +2 -2
@@ -113,50 +113,50 @@ static const struct kfd2kgd_calls kfd2kgd = { .write_vmid_invalidate_request = write_vmid_invalidate_request, .get_fw_version = get_fw_version };struct kfd2kgd_calls *amdgpu_amdkfd_gfx_8_0_get_functions (void ) {return (struct kfd2kgd_calls *)&kfd2kgd; }static inline struct amdgpu_device *get_amdgpu_device (struct kgd_dev *kgd) {return (struct amdgpu_device *)kgd; }static void lock_srbm (struct kgd_dev *kgd,uint32_t mec,uint32_t pipe,uint32_t queue,uint32_t vmid) {struct amdgpu_device *adev =get_amdgpu_device (kgd);uint32_t value =PIPEID (pipe) |MEID (mec) |VMID (vmid) |QUEUEID (queue); -mutex_lock (&adev->srbm_mutex); +spin_lock (&adev->srbm_lock);WREG32 (mmSRBM_GFX_CNTL, value); }static void unlock_srbm (struct kgd_dev *kgd) {struct amdgpu_device *adev =get_amdgpu_device (kgd);WREG32 (mmSRBM_GFX_CNTL,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock); }static void acquire_queue (struct kgd_dev *kgd,uint32_t pipe_id,uint32_t queue_id) {struct amdgpu_device *adev =get_amdgpu_device (kgd);uint32_t mec = (++pipe_id / adev->gfx.mec.num_pipe_per_mec) +1 ;uint32_t pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec);lock_srbm (kgd, mec, pipe, queue_id,0 ); }static void release_queue (struct kgd_dev *kgd) {unlock_srbm (kgd); } static void kgd_program_sh_mem_settings(struct kgd_dev *kgd, uint32_t vmid, uint32_t sh_mem_config,
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1 -1
@@ -1841,41 +1841,41 @@ int amdgpu_device_init(struct amdgpu_device *adev, adev->uvd_ctx_rreg = &amdgpu_invalid_rreg; adev->uvd_ctx_wreg = &amdgpu_invalid_wreg; adev->didt_rreg = &amdgpu_invalid_rreg; adev->didt_wreg = &amdgpu_invalid_wreg; adev->gc_cac_rreg = &amdgpu_invalid_rreg; adev->gc_cac_wreg = &amdgpu_invalid_wreg; adev->audio_endpt_rreg = &amdgpu_block_invalid_rreg; adev->audio_endpt_wreg = &amdgpu_block_invalid_wreg;DRM_INFO ("initializing kernel modesetting (%s 0x%04X:0x%04X 0x%04X:0x%04X 0x%02X).\n" , amdgpu_asic_name[adev->asic_type], pdev->vendor, pdev->device, pdev->subsystem_vendor, pdev->subsystem_device, pdev->revision);/* mutex initialization are all done here so we * can recall function without having locking issues */ atomic_set (&adev->irq.ih.lock,0 );mutex_init (&adev->firmware.mutex);mutex_init (&adev->pm.mutex);mutex_init (&adev->gfx.gpu_clock_mutex); -mutex_init (&adev->srbm_mutex); +spin_lock_init (&adev->srbm_lock);mutex_init (&adev->grbm_idx_mutex);mutex_init (&adev->mn_lock);hash_init (adev->mn_hash);amdgpu_check_arguments (adev);/* Registers mapping */ /* TODO: block userspace mapping of io register */ spin_lock_init (&adev->mmio_idx_lock);spin_lock_init (&adev->smc_idx_lock);spin_lock_init (&adev->pcie_idx_lock);spin_lock_init (&adev->uvd_ctx_idx_lock);spin_lock_init (&adev->didt_idx_lock);spin_lock_init (&adev->gc_cac_idx_lock);spin_lock_init (&adev->audio_endpt_idx_lock);spin_lock_init (&adev->mm_stats.lock);INIT_LIST_HEAD (&adev->shadow_list);mutex_init (&adev->shadow_list_lock);
drivers/gpu/drm/amd/amdgpu/cik_sdma.c +2 -2
@@ -372,50 +372,50 @@ static void cik_sdma_enable(struct amdgpu_device *adev, bool enable)/** * cik_sdma_gfx_resume - setup and start the async dma engines * * @adev: amdgpu_device pointer * * Set up the gfx DMA ring buffers and enable them (CIK). * Returns 0 for success, error for failure. */ static int cik_sdma_gfx_resume (struct amdgpu_device *adev) {struct amdgpu_ring *ring;u32 rb_cntl, ib_cntl;u32 rb_bufsz;u32 wb_offset;int i, j, r;for (i =0 ; i < adev->sdma.num_instances; i++) { ring = &adev->sdma.instance[i].ring; wb_offset = (ring->rptr_offs *4 ); -mutex_lock (&adev->srbm_mutex); +spin_lock (&adev->srbm_lock);for (j =0 ; j <16 ; j++) {cik_srbm_select (adev,0 ,0 ,0 , j);/* SDMA GFX */ WREG32 (mmSDMA0_GFX_VIRTUAL_ADDR + sdma_offsets[i],0 );WREG32 (mmSDMA0_GFX_APE1_CNTL + sdma_offsets[i],0 );/* XXX SDMA RLC - todo */ }cik_srbm_select (adev,0 ,0 ,0 ,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock);WREG32 (mmSDMA0_TILING_CONFIG + sdma_offsets[i], adev->gfx.config.gb_addr_config &0x70 );WREG32 (mmSDMA0_SEM_INCOMPLETE_TIMER_CNTL + sdma_offsets[i],0 );WREG32 (mmSDMA0_SEM_WAIT_FAIL_TIMER_CNTL + sdma_offsets[i],0 );/* Set ring buffer size in dwords */ rb_bufsz =order_base_2 (ring->ring_size /4 ); rb_cntl = rb_bufsz <<1 ;#ifdef __BIG_ENDIAN rb_cntl |=SDMA0_GFX_RB_CNTL__RB_SWAP_ENABLE_MASK |SDMA0_GFX_RB_CNTL__RPTR_WRITEBACK_SWAP_ENABLE_MASK ;#endif WREG32 (mmSDMA0_GFX_RB_CNTL + sdma_offsets[i], rb_cntl);/* Initialize the ring buffer's read and write pointers */ WREG32 (mmSDMA0_GFX_RB_RPTR + sdma_offsets[i],0 );WREG32 (mmSDMA0_GFX_RB_WPTR + sdma_offsets[i],0 );WREG32 (mmSDMA0_GFX_IB_RPTR + sdma_offsets[i],0 );
drivers/gpu/drm/amd/amdgpu/gfx_v7_0.c +10 -10
@@ -1845,51 +1845,51 @@ static void gfx_v7_0_setup_rb(struct amdgpu_device *adev)*/ #define DEFAULT_SH_MEM_BASES (0x6000)#define FIRST_COMPUTE_VMID (8)#define LAST_COMPUTE_VMID (16)static void gmc_v7_0_init_compute_vmid (struct amdgpu_device *adev) {int i;uint32_t sh_mem_config;uint32_t sh_mem_bases;/* * Configure apertures: * LDS: 0x60000000'00000000 - 0x60000001'00000000 (4GB) * Scratch: 0x60000001'00000000 - 0x60000002'00000000 (4GB) * GPUVM: 0x60010000'00000000 - 0x60020000'00000000 (1TB) */ sh_mem_bases =DEFAULT_SH_MEM_BASES | (DEFAULT_SH_MEM_BASES <<16 ); sh_mem_config =SH_MEM_ALIGNMENT_MODE_UNALIGNED <<SH_MEM_CONFIG__ALIGNMENT_MODE__SHIFT ; sh_mem_config |=MTYPE_NONCACHED <<SH_MEM_CONFIG__DEFAULT_MTYPE__SHIFT ; -mutex_lock (&adev->srbm_mutex); +spin_lock (&adev->srbm_lock);for (i =FIRST_COMPUTE_VMID ; i <LAST_COMPUTE_VMID ; i++) {cik_srbm_select (adev,0 ,0 ,0 , i);/* CP and shaders */ WREG32 (mmSH_MEM_CONFIG, sh_mem_config);WREG32 (mmSH_MEM_APE1_BASE,1 );WREG32 (mmSH_MEM_APE1_LIMIT,0 );WREG32 (mmSH_MEM_BASES, sh_mem_bases); }cik_srbm_select (adev,0 ,0 ,0 ,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock); }static void gfx_v7_0_config_init (struct amdgpu_device *adev) { adev->gfx.config.double_offchip_lds_buf =1 ; }/** * gfx_v7_0_gpu_init - setup the 3D engine * * @adev: amdgpu_device pointer * * Configures the 3D engine and tiling configuration * registers so that the 3D engine is usable. */ static void gfx_v7_0_gpu_init (struct amdgpu_device *adev) {u32 sh_mem_cfg, sh_static_mem_cfg, sh_mem_base;u32 tmp;int i;
@@ -1918,56 +1918,56 @@ static void gfx_v7_0_gpu_init(struct amdgpu_device *adev)*/ gfx_v7_0_select_se_sh (adev,0xffffffff ,0xffffffff ,0xffffffff );/* XXX SH_MEM regs */ /* where to put LDS, scratch, GPUVM in FSA64 space */ sh_mem_cfg =REG_SET_FIELD (0 ,SH_MEM_CONFIG ,ALIGNMENT_MODE ,SH_MEM_ALIGNMENT_MODE_UNALIGNED ); sh_mem_cfg =REG_SET_FIELD (sh_mem_cfg,SH_MEM_CONFIG ,DEFAULT_MTYPE ,MTYPE_NC ); sh_mem_cfg =REG_SET_FIELD (sh_mem_cfg,SH_MEM_CONFIG ,APE1_MTYPE ,MTYPE_UC ); sh_mem_cfg =REG_SET_FIELD (sh_mem_cfg,SH_MEM_CONFIG ,PRIVATE_ATC ,0 ); sh_static_mem_cfg =REG_SET_FIELD (0 ,SH_STATIC_MEM_CONFIG ,SWIZZLE_ENABLE ,1 ); sh_static_mem_cfg =REG_SET_FIELD (sh_static_mem_cfg,SH_STATIC_MEM_CONFIG ,ELEMENT_SIZE ,1 ); sh_static_mem_cfg =REG_SET_FIELD (sh_static_mem_cfg,SH_STATIC_MEM_CONFIG ,INDEX_STRIDE ,3 ); -mutex_lock (&adev->srbm_mutex); +spin_lock (&adev->srbm_lock);for (i =0 ; i < adev->vm_manager.id_mgr[0 ].num_ids; i++) {if (i ==0 ) sh_mem_base =0 ;else sh_mem_base = adev->mc.shared_aperture_start >>48 ;cik_srbm_select (adev,0 ,0 ,0 , i);/* CP and shaders */ WREG32 (mmSH_MEM_CONFIG, sh_mem_cfg);WREG32 (mmSH_MEM_APE1_BASE,1 );WREG32 (mmSH_MEM_APE1_LIMIT,0 );WREG32 (mmSH_MEM_BASES, sh_mem_base);WREG32 (mmSH_STATIC_MEM_CONFIG, sh_static_mem_cfg); }cik_srbm_select (adev,0 ,0 ,0 ,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock);gmc_v7_0_init_compute_vmid (adev);WREG32 (mmSX_DEBUG_1,0x20 );WREG32 (mmTA_CNTL_AUX,0x00010000 ); tmp =RREG32 (mmSPI_CONFIG_CNTL); tmp |=0x03000000 ;WREG32 (mmSPI_CONFIG_CNTL, tmp);WREG32 (mmSQ_CONFIG,1 );WREG32 (mmDB_DEBUG,0 ); tmp =RREG32 (mmDB_DEBUG2) & ~0xf00fffff ; tmp |=0x00000400 ;WREG32 (mmDB_DEBUG2, tmp); tmp =RREG32 (mmDB_DEBUG3) & ~0x0002021c ;
@@ -2950,60 +2950,60 @@ struct hqd_registersu32 cp_hqd_iq_rptr;u32 cp_hqd_dequeue_request;u32 cp_hqd_dma_offload;u32 cp_hqd_sema_cmd;u32 cp_hqd_msg_type;u32 cp_hqd_atomic0_preop_lo;u32 cp_hqd_atomic0_preop_hi;u32 cp_hqd_atomic1_preop_lo;u32 cp_hqd_atomic1_preop_hi;u32 cp_hqd_hq_scheduler0;u32 cp_hqd_hq_scheduler1;u32 cp_mqd_control; };static void gfx_v7_0_compute_pipe_init (struct amdgpu_device *adev,int me,int pipe) {u64 eop_gpu_addr;u32 tmp;size_t eop_offset = me * pipe *GFX7_MEC_HPD_SIZE *2 ; -mutex_lock (&adev->srbm_mutex); +spin_lock (&adev->srbm_lock); eop_gpu_addr = adev->gfx.mec.hpd_eop_gpu_addr + eop_offset;cik_srbm_select (adev, me, pipe,0 ,0 );/* write the EOP addr */ WREG32 (mmCP_HPD_EOP_BASE_ADDR, eop_gpu_addr >>8 );WREG32 (mmCP_HPD_EOP_BASE_ADDR_HI,upper_32_bits (eop_gpu_addr) >>8 );/* set the VMID assigned */ WREG32 (mmCP_HPD_EOP_VMID,0 );/* set the EOP size, register value is 2^(EOP_SIZE+1) dwords */ tmp =RREG32 (mmCP_HPD_EOP_CONTROL); tmp &= ~CP_HPD_EOP_CONTROL__EOP_SIZE_MASK ; tmp |=order_base_2 (GFX7_MEC_HPD_SIZE /8 );WREG32 (mmCP_HPD_EOP_CONTROL, tmp);cik_srbm_select (adev,0 ,0 ,0 ,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock); }static int gfx_v7_0_mqd_deactivate (struct amdgpu_device *adev) {int i;/* disable the queue if it's active */ if (RREG32 (mmCP_HQD_ACTIVE) &1 ) {WREG32 (mmCP_HQD_DEQUEUE_REQUEST,1 );for (i =0 ; i < adev->usec_timeout; i++) {if (!(RREG32 (mmCP_HQD_ACTIVE) &1 ))break ;udelay (1 ); }if (i == adev->usec_timeout)return -ETIMEDOUT ;WREG32 (mmCP_HQD_DEQUEUE_REQUEST,0 );WREG32 (mmCP_HQD_PQ_RPTR,0 );
@@ -3176,49 +3176,49 @@ static int gfx_v7_0_compute_queue_init(struct amdgpu_device *adev, int ring_id)return r; } } r =amdgpu_bo_reserve (ring->mqd_obj, false);if (unlikely (r !=0 )) goto out; r =amdgpu_bo_pin (ring->mqd_obj,AMDGPU_GEM_DOMAIN_GTT , &mqd_gpu_addr);if (r) {dev_warn (adev->dev,"(%d) pin MQD bo failed\n" , r); goto out_unreserve; } r =amdgpu_bo_kmap (ring->mqd_obj, (void **)&mqd);if (r) {dev_warn (adev->dev,"(%d) map MQD bo failed\n" , r); goto out_unreserve; } -mutex_lock (&adev->srbm_mutex); +spin_lock (&adev->srbm_lock);cik_srbm_select (adev, ring->me, ring->pipe, ring->queue,0 );gfx_v7_0_mqd_init (adev, mqd, mqd_gpu_addr, ring);gfx_v7_0_mqd_deactivate (adev);gfx_v7_0_mqd_commit (adev, mqd);cik_srbm_select (adev,0 ,0 ,0 ,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock);amdgpu_bo_kunmap (ring->mqd_obj); out_unreserve:amdgpu_bo_unreserve (ring->mqd_obj); out:return 0 ; }/** * gfx_v7_0_cp_compute_resume - setup the compute queue registers * * @adev: amdgpu_device pointer * * Program the compute queues and test them to make sure they * are working. * Returns 0 for success, error for failure. */ static int gfx_v7_0_cp_compute_resume (struct amdgpu_device *adev) {int r, i, j;
@@ -5054,48 +5054,48 @@ static void gfx_v7_0_set_gfx_eop_interrupt_state(struct amdgpu_device *adev, }static void gfx_v7_0_set_compute_eop_interrupt_state (struct amdgpu_device *adev,int me,int pipe,enum amdgpu_interrupt_state state) {/* Me 0 is for graphics and Me 2 is reserved for HW scheduling * So we should only really be configuring ME 1 i.e. MEC0 */ if (me !=1 ) {DRM_ERROR ("Ignoring request to enable interrupts for invalid me:%d\n" , me);return ; }if (pipe >= adev->gfx.mec.num_pipe_per_mec) {DRM_ERROR ("Ignoring request to enable interrupts for invalid " "me:%d pipe:%d\n" , pipe, me);return ; } -mutex_lock (&adev->srbm_mutex); +spin_lock (&adev->srbm_lock);cik_srbm_select (adev, me, pipe,0 ,0 );WREG32_FIELD (CPC_INT_CNTL ,TIME_STAMP_INT_ENABLE , state ==AMDGPU_IRQ_STATE_DISABLE ?0 :1 );cik_srbm_select (adev,0 ,0 ,0 ,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock); }static int gfx_v7_0_set_priv_reg_fault_state (struct amdgpu_device *adev,struct amdgpu_irq_src *src,unsigned type,enum amdgpu_interrupt_state state) {u32 cp_int_cntl;switch (state) {case AMDGPU_IRQ_STATE_DISABLE : cp_int_cntl =RREG32 (mmCP_INT_CNTL_RING0); cp_int_cntl &= ~CP_INT_CNTL_RING0__PRIV_REG_INT_ENABLE_MASK ;WREG32 (mmCP_INT_CNTL_RING0, cp_int_cntl);break ;case AMDGPU_IRQ_STATE_ENABLE : cp_int_cntl =RREG32 (mmCP_INT_CNTL_RING0); cp_int_cntl |=CP_INT_CNTL_RING0__PRIV_REG_INT_ENABLE_MASK ;WREG32 (mmCP_INT_CNTL_RING0, cp_int_cntl);break ;
drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c +17 -17
@@ -3923,116 +3923,116 @@ static void gfx_v8_0_init_compute_vmid(struct amdgpu_device *adev) {int i;uint32_t sh_mem_config;uint32_t sh_mem_bases;/* * Configure apertures: * LDS: 0x60000000'00000000 - 0x60000001'00000000 (4GB) * Scratch: 0x60000001'00000000 - 0x60000002'00000000 (4GB) * GPUVM: 0x60010000'00000000 - 0x60020000'00000000 (1TB) */ sh_mem_bases =DEFAULT_SH_MEM_BASES | (DEFAULT_SH_MEM_BASES <<16 ); sh_mem_config =SH_MEM_ADDRESS_MODE_HSA64 <<SH_MEM_CONFIG__ADDRESS_MODE__SHIFT |SH_MEM_ALIGNMENT_MODE_UNALIGNED <<SH_MEM_CONFIG__ALIGNMENT_MODE__SHIFT |MTYPE_CC <<SH_MEM_CONFIG__DEFAULT_MTYPE__SHIFT |SH_MEM_CONFIG__PRIVATE_ATC_MASK ; -mutex_lock (&adev->srbm_mutex); +spin_lock (&adev->srbm_lock);for (i =FIRST_COMPUTE_VMID ; i <LAST_COMPUTE_VMID ; i++) {vi_srbm_select (adev,0 ,0 ,0 , i);/* CP and shaders */ WREG32 (mmSH_MEM_CONFIG, sh_mem_config);WREG32 (mmSH_MEM_APE1_BASE,1 );WREG32 (mmSH_MEM_APE1_LIMIT,0 );WREG32 (mmSH_MEM_BASES, sh_mem_bases); }vi_srbm_select (adev,0 ,0 ,0 ,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock); }static void gfx_v8_0_config_init (struct amdgpu_device *adev) {switch (adev->asic_type) {default : adev->gfx.config.double_offchip_lds_buf =1 ;break ;case CHIP_CARRIZO :case CHIP_STONEY : adev->gfx.config.double_offchip_lds_buf =0 ;break ; } }static void gfx_v8_0_gpu_init (struct amdgpu_device *adev) {u32 tmp, sh_static_mem_cfg;int i;WREG32_FIELD (GRBM_CNTL ,READ_TIMEOUT ,0xFF );WREG32 (mmGB_ADDR_CONFIG, adev->gfx.config.gb_addr_config);WREG32 (mmHDP_ADDR_CONFIG, adev->gfx.config.gb_addr_config);WREG32 (mmDMIF_ADDR_CALC, adev->gfx.config.gb_addr_config);gfx_v8_0_tiling_mode_table_init (adev);gfx_v8_0_setup_rb (adev);gfx_v8_0_get_cu_info (adev);gfx_v8_0_config_init (adev);/* XXX SH_MEM regs */ /* where to put LDS, scratch, GPUVM in FSA64 space */ sh_static_mem_cfg =REG_SET_FIELD (0 ,SH_STATIC_MEM_CONFIG ,SWIZZLE_ENABLE ,1 ); sh_static_mem_cfg =REG_SET_FIELD (sh_static_mem_cfg,SH_STATIC_MEM_CONFIG ,ELEMENT_SIZE ,1 ); sh_static_mem_cfg =REG_SET_FIELD (sh_static_mem_cfg,SH_STATIC_MEM_CONFIG ,INDEX_STRIDE ,3 ); -mutex_lock (&adev->srbm_mutex); +spin_lock (&adev->srbm_lock);for (i =0 ; i < adev->vm_manager.id_mgr[0 ].num_ids; i++) {vi_srbm_select (adev,0 ,0 ,0 , i);/* CP and shaders */ if (i ==0 ) { tmp =REG_SET_FIELD (0 ,SH_MEM_CONFIG ,DEFAULT_MTYPE ,MTYPE_UC ); tmp =REG_SET_FIELD (tmp,SH_MEM_CONFIG ,APE1_MTYPE ,MTYPE_UC ); tmp =REG_SET_FIELD (tmp,SH_MEM_CONFIG ,ALIGNMENT_MODE ,SH_MEM_ALIGNMENT_MODE_UNALIGNED );WREG32 (mmSH_MEM_CONFIG, tmp);WREG32 (mmSH_MEM_BASES,0 ); }else { tmp =REG_SET_FIELD (0 ,SH_MEM_CONFIG ,DEFAULT_MTYPE ,MTYPE_NC ); tmp =REG_SET_FIELD (tmp,SH_MEM_CONFIG ,APE1_MTYPE ,MTYPE_UC ); tmp =REG_SET_FIELD (tmp,SH_MEM_CONFIG ,ALIGNMENT_MODE ,SH_MEM_ALIGNMENT_MODE_UNALIGNED );WREG32 (mmSH_MEM_CONFIG, tmp); tmp = adev->mc.shared_aperture_start >>48 ;WREG32 (mmSH_MEM_BASES, tmp); }WREG32 (mmSH_MEM_APE1_BASE,1 );WREG32 (mmSH_MEM_APE1_LIMIT,0 );WREG32 (mmSH_STATIC_MEM_CONFIG, sh_static_mem_cfg); }vi_srbm_select (adev,0 ,0 ,0 ,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock);gfx_v8_0_init_compute_vmid (adev);mutex_lock (&adev->grbm_idx_mutex);/* * making sure that the following register writes will be broadcasted * to all the shaders */ gfx_v8_0_select_se_sh (adev,0xffffffff ,0xffffffff ,0xffffffff );WREG32 (mmPA_SC_FIFO_SIZE, (adev->gfx.config.sc_prim_fifo_size_frontend <<PA_SC_FIFO_SIZE__SC_FRONTEND_PRIM_FIFO_SIZE__SHIFT ) | (adev->gfx.config.sc_prim_fifo_size_backend <<PA_SC_FIFO_SIZE__SC_BACKEND_PRIM_FIFO_SIZE__SHIFT ) | (adev->gfx.config.sc_hiz_tile_fifo_size <<PA_SC_FIFO_SIZE__SC_HIZ_TILE_FIFO_SIZE__SHIFT ) | (adev->gfx.config.sc_earlyz_tile_fifo_size <<PA_SC_FIFO_SIZE__SC_EARLYZ_TILE_FIFO_SIZE__SHIFT ));
@@ -5075,87 +5075,87 @@ int gfx_v8_0_mqd_commit(struct amdgpu_device *adev, }static int gfx_v8_0_kiq_init_queue (struct amdgpu_ring *ring) {int r =0 ;struct amdgpu_device *adev = ring->adev;struct vi_mqd *mqd = ring->mqd_ptr;int mqd_idx =AMDGPU_MAX_COMPUTE_RINGS ;gfx_v8_0_kiq_setting (ring);if (adev->gfx.in_reset) {/* for GPU_RESET case */ /* reset MQD to a clean status */ if (adev->gfx.mec.mqd_backup[mqd_idx])memcpy (mqd, adev->gfx.mec.mqd_backup[mqd_idx],sizeof (*mqd));/* reset ring buffer */ ring->wptr =0 ;amdgpu_ring_clear_ring (ring); -mutex_lock (&adev->srbm_mutex); +spin_lock (&adev->srbm_lock);vi_srbm_select (adev, ring->me, ring->pipe, ring->queue,0 ); r =gfx_v8_0_deactivate_hqd (adev,1 );if (r) {dev_err (adev->dev,"failed to deactivate ring %s\n" , ring->name); goto out_unlock; }gfx_v8_0_mqd_commit (adev, mqd);vi_srbm_select (adev,0 ,0 ,0 ,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock); }else { -mutex_lock (&adev->srbm_mutex); +spin_lock (&adev->srbm_lock);vi_srbm_select (adev, ring->me, ring->pipe, ring->queue,0 );gfx_v8_0_mqd_init (ring); r =gfx_v8_0_deactivate_hqd (adev,1 );if (r) {dev_err (adev->dev,"failed to deactivate ring %s\n" , ring->name); goto out_unlock; }gfx_v8_0_mqd_commit (adev, mqd);vi_srbm_select (adev,0 ,0 ,0 ,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock);if (adev->gfx.mec.mqd_backup[mqd_idx])memcpy (adev->gfx.mec.mqd_backup[mqd_idx], mqd,sizeof (*mqd)); }return r; out_unlock:vi_srbm_select (adev,0 ,0 ,0 ,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock);return r; }static int gfx_v8_0_kcq_init_queue (struct amdgpu_ring *ring) {struct amdgpu_device *adev = ring->adev;struct vi_mqd *mqd = ring->mqd_ptr;int mqd_idx = ring - &adev->gfx.compute_ring[0 ];if (!adev->gfx.in_reset && !adev->gfx.in_suspend) { -mutex_lock (&adev->srbm_mutex); +spin_lock (&adev->srbm_lock);vi_srbm_select (adev, ring->me, ring->pipe, ring->queue,0 );gfx_v8_0_mqd_init (ring);vi_srbm_select (adev,0 ,0 ,0 ,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock);if (adev->gfx.mec.mqd_backup[mqd_idx])memcpy (adev->gfx.mec.mqd_backup[mqd_idx], mqd,sizeof (*mqd)); }else if (adev->gfx.in_reset) {/* for GPU_RESET case */ /* reset MQD to a clean status */ if (adev->gfx.mec.mqd_backup[mqd_idx])memcpy (mqd, adev->gfx.mec.mqd_backup[mqd_idx],sizeof (*mqd));/* reset ring buffer */ ring->wptr =0 ;amdgpu_ring_clear_ring (ring); }return 0 ; }static void gfx_v8_0_set_mec_doorbell_range (struct amdgpu_device *adev) {if (adev->asic_type >CHIP_TONGA ) {WREG32 (mmCP_MEC_DOORBELL_RANGE_LOWER,AMDGPU_DOORBELL_KIQ <<2 );
@@ -5450,45 +5450,45 @@ static int gfx_v8_0_pre_soft_reset(void *handle) grbm_soft_reset = adev->gfx.grbm_soft_reset; srbm_soft_reset = adev->gfx.srbm_soft_reset;/* stop the rlc */ gfx_v8_0_rlc_stop (adev);if (REG_GET_FIELD (grbm_soft_reset,GRBM_SOFT_RESET ,SOFT_RESET_CP ) ||REG_GET_FIELD (grbm_soft_reset,GRBM_SOFT_RESET ,SOFT_RESET_GFX ))/* Disable GFX parsing/prefetching */ gfx_v8_0_cp_gfx_enable (adev, false);if (REG_GET_FIELD (grbm_soft_reset,GRBM_SOFT_RESET ,SOFT_RESET_CP ) ||REG_GET_FIELD (grbm_soft_reset,GRBM_SOFT_RESET ,SOFT_RESET_CPF ) ||REG_GET_FIELD (grbm_soft_reset,GRBM_SOFT_RESET ,SOFT_RESET_CPC ) ||REG_GET_FIELD (grbm_soft_reset,GRBM_SOFT_RESET ,SOFT_RESET_CPG )) {int i;for (i =0 ; i < adev->gfx.num_compute_rings; i++) {struct amdgpu_ring *ring = &adev->gfx.compute_ring[i]; -mutex_lock (&adev->srbm_mutex); +spin_lock (&adev->srbm_lock);vi_srbm_select (adev, ring->me, ring->pipe, ring->queue,0 );gfx_v8_0_deactivate_hqd (adev,2 );vi_srbm_select (adev,0 ,0 ,0 ,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock); }/* Disable MEC parsing/prefetching */ gfx_v8_0_cp_compute_enable (adev, false); }return 0 ; }static int gfx_v8_0_soft_reset (void *handle) {struct amdgpu_device *adev = (struct amdgpu_device *)handle;u32 grbm_soft_reset =0 , srbm_soft_reset =0 ;u32 tmp;if ((!adev->gfx.grbm_soft_reset) && (!adev->gfx.srbm_soft_reset))return 0 ; grbm_soft_reset = adev->gfx.grbm_soft_reset; srbm_soft_reset = adev->gfx.srbm_soft_reset;
@@ -5550,45 +5550,45 @@ static int gfx_v8_0_post_soft_reset(void *handle)if ((!adev->gfx.grbm_soft_reset) && (!adev->gfx.srbm_soft_reset))return 0 ; grbm_soft_reset = adev->gfx.grbm_soft_reset; srbm_soft_reset = adev->gfx.srbm_soft_reset;if (REG_GET_FIELD (grbm_soft_reset,GRBM_SOFT_RESET ,SOFT_RESET_CP ) ||REG_GET_FIELD (grbm_soft_reset,GRBM_SOFT_RESET ,SOFT_RESET_GFX ))gfx_v8_0_cp_gfx_resume (adev);if (REG_GET_FIELD (grbm_soft_reset,GRBM_SOFT_RESET ,SOFT_RESET_CP ) ||REG_GET_FIELD (grbm_soft_reset,GRBM_SOFT_RESET ,SOFT_RESET_CPF ) ||REG_GET_FIELD (grbm_soft_reset,GRBM_SOFT_RESET ,SOFT_RESET_CPC ) ||REG_GET_FIELD (grbm_soft_reset,GRBM_SOFT_RESET ,SOFT_RESET_CPG )) {int i;for (i =0 ; i < adev->gfx.num_compute_rings; i++) {struct amdgpu_ring *ring = &adev->gfx.compute_ring[i]; -mutex_lock (&adev->srbm_mutex); +spin_lock (&adev->srbm_lock);vi_srbm_select (adev, ring->me, ring->pipe, ring->queue,0 );gfx_v8_0_deactivate_hqd (adev,2 );vi_srbm_select (adev,0 ,0 ,0 ,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock); }gfx_v8_0_kiq_resume (adev); }gfx_v8_0_rlc_start (adev);return 0 ; }/** * gfx_v8_0_get_gpu_clock_counter - return GPU clock counter snapshot * * @adev: amdgpu_device pointer * * Fetches a GPU clock counter snapshot. * Returns the 64 bit clock counter snapshot. */ static uint64_t gfx_v8_0_get_gpu_clock_counter (struct amdgpu_device *adev) {uint64_t clock;
@@ -6794,48 +6794,48 @@ static void gfx_v8_0_set_gfx_eop_interrupt_state(struct amdgpu_device *adev,WREG32_FIELD (CP_INT_CNTL_RING0 ,TIME_STAMP_INT_ENABLE , state ==AMDGPU_IRQ_STATE_DISABLE ?0 :1 ); }static void gfx_v8_0_set_compute_eop_interrupt_state (struct amdgpu_device *adev,int me,int pipe,enum amdgpu_interrupt_state state) {/* Me 0 is reserved for graphics */ if (me <1 || me > adev->gfx.mec.num_mec) {DRM_ERROR ("Ignoring request to enable interrupts for invalid me:%d\n" , me);return ; }if (pipe >= adev->gfx.mec.num_pipe_per_mec) {DRM_ERROR ("Ignoring request to enable interrupts for invalid " "me:%d pipe:%d\n" , pipe, me);return ; } -mutex_lock (&adev->srbm_mutex); +spin_lock (&adev->srbm_lock);vi_srbm_select (adev, me, pipe,0 ,0 );WREG32_FIELD (CPC_INT_CNTL ,TIME_STAMP_INT_ENABLE , state ==AMDGPU_IRQ_STATE_DISABLE ?0 :1 );vi_srbm_select (adev,0 ,0 ,0 ,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock); }static int gfx_v8_0_set_priv_reg_fault_state (struct amdgpu_device *adev,struct amdgpu_irq_src *source,unsigned type,enum amdgpu_interrupt_state state) {WREG32_FIELD (CP_INT_CNTL_RING0 ,PRIV_REG_INT_ENABLE , state ==AMDGPU_IRQ_STATE_DISABLE ?0 :1 );return 0 ; }static int gfx_v8_0_set_priv_inst_fault_state (struct amdgpu_device *adev,struct amdgpu_irq_src *source,unsigned type,enum amdgpu_interrupt_state state) {WREG32_FIELD (CP_INT_CNTL_RING0 ,PRIV_INSTR_INT_ENABLE , state ==AMDGPU_IRQ_STATE_DISABLE ?0 :1 );
drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c +12 -12
@@ -1362,78 +1362,78 @@ static void gfx_v9_0_setup_rb(struct amdgpu_device *adev)#define FIRST_COMPUTE_VMID (8)#define LAST_COMPUTE_VMID (16)static void gfx_v9_0_init_compute_vmid (struct amdgpu_device *adev) {int i;uint32_t sh_mem_config;uint32_t sh_mem_bases;/* * Configure apertures: * LDS: 0x60000000'00000000 - 0x60000001'00000000 (4GB) * Scratch: 0x60000001'00000000 - 0x60000002'00000000 (4GB) * GPUVM: 0x60010000'00000000 - 0x60020000'00000000 (1TB) */ sh_mem_bases =DEFAULT_SH_MEM_BASES | (DEFAULT_SH_MEM_BASES <<16 ); sh_mem_config =SH_MEM_ADDRESS_MODE_64 |SH_MEM_ALIGNMENT_MODE_UNALIGNED <<SH_MEM_CONFIG__ALIGNMENT_MODE__SHIFT ; -mutex_lock (&adev->srbm_mutex); +spin_lock (&adev->srbm_lock);for (i =FIRST_COMPUTE_VMID ; i <LAST_COMPUTE_VMID ; i++) {soc15_grbm_select (adev,0 ,0 ,0 , i);/* CP and shaders */ WREG32_SOC15 (GC ,0 , mmSH_MEM_CONFIG, sh_mem_config);WREG32_SOC15 (GC ,0 , mmSH_MEM_BASES, sh_mem_bases); }soc15_grbm_select (adev,0 ,0 ,0 ,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock); }static void gfx_v9_0_gpu_init (struct amdgpu_device *adev) {u32 tmp;int i;WREG32_FIELD15 (GC ,0 ,GRBM_CNTL ,READ_TIMEOUT ,0xff );gfx_v9_0_tiling_mode_table_init (adev);gfx_v9_0_setup_rb (adev);gfx_v9_0_get_cu_info (adev, &adev->gfx.cu_info);/* XXX SH_MEM regs */ /* where to put LDS, scratch, GPUVM in FSA64 space */ -mutex_lock (&adev->srbm_mutex); +spin_lock (&adev->srbm_lock);for (i =0 ; i <16 ; i++) {soc15_grbm_select (adev,0 ,0 ,0 , i);/* CP and shaders */ tmp =0 ; tmp =REG_SET_FIELD (tmp,SH_MEM_CONFIG ,ALIGNMENT_MODE ,SH_MEM_ALIGNMENT_MODE_UNALIGNED );WREG32_SOC15 (GC ,0 , mmSH_MEM_CONFIG, tmp);WREG32_SOC15 (GC ,0 , mmSH_MEM_BASES,0 ); }soc15_grbm_select (adev,0 ,0 ,0 ,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock);gfx_v9_0_init_compute_vmid (adev);mutex_lock (&adev->grbm_idx_mutex);/* * making sure that the following register writes will be broadcasted * to all the shaders */ gfx_v9_0_select_se_sh (adev,0xffffffff ,0xffffffff ,0xffffffff );WREG32_SOC15 (GC ,0 , mmPA_SC_FIFO_SIZE, (adev->gfx.config.sc_prim_fifo_size_frontend <<PA_SC_FIFO_SIZE__SC_FRONTEND_PRIM_FIFO_SIZE__SHIFT ) | (adev->gfx.config.sc_prim_fifo_size_backend <<PA_SC_FIFO_SIZE__SC_BACKEND_PRIM_FIFO_SIZE__SHIFT ) | (adev->gfx.config.sc_hiz_tile_fifo_size <<PA_SC_FIFO_SIZE__SC_HIZ_TILE_FIFO_SIZE__SHIFT ) | (adev->gfx.config.sc_earlyz_tile_fifo_size <<PA_SC_FIFO_SIZE__SC_EARLYZ_TILE_FIFO_SIZE__SHIFT ));mutex_unlock (&adev->grbm_idx_mutex);
@@ -2234,60 +2234,60 @@ static int gfx_v9_0_kiq_init_register(struct amdgpu_ring *ring)return 0 ; }static int gfx_v9_0_kiq_init_queue (struct amdgpu_ring *ring) {struct amdgpu_device *adev = ring->adev;struct amdgpu_kiq *kiq = &adev->gfx.kiq;struct v9_mqd *mqd = ring->mqd_ptr;bool is_kiq = (ring->funcs->type ==AMDGPU_RING_TYPE_KIQ );int mqd_idx =AMDGPU_MAX_COMPUTE_RINGS ;if (is_kiq) {gfx_v9_0_kiq_setting (&kiq->ring); }else { mqd_idx = ring - &adev->gfx.compute_ring[0 ]; }if (!adev->gfx.in_reset) {memset ((void *)mqd,0 ,sizeof (*mqd)); -mutex_lock (&adev->srbm_mutex); +spin_lock (&adev->srbm_lock);soc15_grbm_select (adev, ring->me, ring->pipe, ring->queue,0 );gfx_v9_0_mqd_init (ring);if (is_kiq)gfx_v9_0_kiq_init_register (ring);soc15_grbm_select (adev,0 ,0 ,0 ,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock); }else {/* for GPU_RESET case */ /* reset MQD to a clean status */ /* reset ring buffer */ ring->wptr =0 ;if (is_kiq) { -mutex_lock (&adev->srbm_mutex); +spin_lock (&adev->srbm_lock);soc15_grbm_select (adev, ring->me, ring->pipe, ring->queue,0 );gfx_v9_0_kiq_init_register (ring);soc15_grbm_select (adev,0 ,0 ,0 ,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock); } }if (is_kiq)gfx_v9_0_kiq_enable (ring);else gfx_v9_0_map_queue_enable (&kiq->ring, ring);return 0 ; }static int gfx_v9_0_kiq_resume (struct amdgpu_device *adev) {struct amdgpu_ring *ring =NULL ;int r =0 , i;gfx_v9_0_cp_compute_enable (adev, true); ring = &adev->gfx.kiq.ring;
@@ -3316,48 +3316,48 @@ static void gfx_v9_0_set_gfx_eop_interrupt_state(struct amdgpu_device *adev,break ; } }static void gfx_v9_0_set_compute_eop_interrupt_state (struct amdgpu_device *adev,int me,int pipe,enum amdgpu_interrupt_state state) {/* Me 0 is reserved for graphics */ if (me <1 || me > adev->gfx.mec.num_mec) {DRM_ERROR ("Ignoring request to enable interrupts for invalid me:%d\n" , me);return ; }if (pipe >= adev->gfx.mec.num_pipe_per_mec) {DRM_ERROR ("Ignoring request to enable interrupts for invalid " "me:%d pipe:%d\n" , pipe, me);return ; } -mutex_lock (&adev->srbm_mutex); +spin_lock (&adev->srbm_lock);soc15_grbm_select (adev, me, pipe,0 ,0 );WREG32_FIELD (CPC_INT_CNTL ,TIME_STAMP_INT_ENABLE , state ==AMDGPU_IRQ_STATE_DISABLE ?0 :1 );soc15_grbm_select (adev,0 ,0 ,0 ,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock); }static int gfx_v9_0_set_priv_reg_fault_state (struct amdgpu_device *adev,struct amdgpu_irq_src *source,unsigned type,enum amdgpu_interrupt_state state) {switch (state) {case AMDGPU_IRQ_STATE_DISABLE :case AMDGPU_IRQ_STATE_ENABLE :WREG32_FIELD15 (GC ,0 ,CP_INT_CNTL_RING0 ,PRIV_REG_INT_ENABLE , state ==AMDGPU_IRQ_STATE_ENABLE ?1 :0 );break ;default :break ; }return 0 ; }
@@ -3839,41 +3839,41 @@ static int gfx_v9_0_init_queue(struct amdgpu_ring *ring)return r; } r =amdgpu_bo_kmap (ring->mqd_obj, (void **)&buf);if (r) {dev_warn (adev->dev,"(%d) map MQD bo failed\n" , r);gfx_v9_0_cp_compute_fini (adev);return r; }/* init the mqd struct */ memset (buf,0 ,sizeof (struct v9_mqd )); mqd = (struct v9_mqd *)buf; mqd->header =0xC0310800 ; mqd->compute_pipelinestat_enable =0x00000001 ; mqd->compute_static_thread_mgmt_se0 =0xffffffff ; mqd->compute_static_thread_mgmt_se1 =0xffffffff ; mqd->compute_static_thread_mgmt_se2 =0xffffffff ; mqd->compute_static_thread_mgmt_se3 =0xffffffff ; mqd->compute_misc_reserved =0x00000003 ; -mutex_lock (&adev->srbm_mutex); +spin_lock (&adev->srbm_lock);soc15_grbm_select (adev, ring->me, ring->pipe, ring->queue,0 );/* disable wptr polling */ WREG32_FIELD15 (GC ,0 ,CP_PQ_WPTR_POLL_CNTL ,EN ,0 );/* write the EOP addr */ BUG_ON (ring->me !=1 || ring->pipe !=0 );/* can't handle other cases eop address */ eop_gpu_addr = adev->gfx.mec.hpd_eop_gpu_addr + (ring->queue *GFX9_MEC_HPD_SIZE ); eop_gpu_addr >>=8 ;WREG32_SOC15 (GC ,0 , mmCP_HQD_EOP_BASE_ADDR,lower_32_bits (eop_gpu_addr));WREG32_SOC15 (GC ,0 , mmCP_HQD_EOP_BASE_ADDR_HI,upper_32_bits (eop_gpu_addr)); mqd->cp_hqd_eop_base_addr_lo =lower_32_bits (eop_gpu_addr); mqd->cp_hqd_eop_base_addr_hi =upper_32_bits (eop_gpu_addr);/* set the EOP size, register value is 2^(EOP_SIZE+1) dwords */ tmp =RREG32_SOC15 (GC ,0 , mmCP_HQD_EOP_CONTROL); tmp =REG_SET_FIELD (tmp,CP_HQD_EOP_CONTROL ,EOP_SIZE , (order_base_2 (GFX9_MEC_HPD_SIZE /4 ) -1 ));
@@ -3983,39 +3983,39 @@ static int gfx_v9_0_init_queue(struct amdgpu_ring *ring) mqd->cp_hqd_pq_doorbell_control);/* reset read and write pointers, similar to CP_RB0_WPTR/_RPTR */ WREG32_SOC15 (GC ,0 , mmCP_HQD_PQ_WPTR_LO, mqd->cp_hqd_pq_wptr_lo);WREG32_SOC15 (GC ,0 , mmCP_HQD_PQ_WPTR_HI, mqd->cp_hqd_pq_wptr_hi);/* set the vmid for the queue */ mqd->cp_hqd_vmid =0 ;WREG32_SOC15 (GC ,0 , mmCP_HQD_VMID, mqd->cp_hqd_vmid); tmp =RREG32_SOC15 (GC ,0 , mmCP_HQD_PERSISTENT_STATE); tmp =REG_SET_FIELD (tmp,CP_HQD_PERSISTENT_STATE ,PRELOAD_SIZE ,0x53 );WREG32_SOC15 (GC ,0 , mmCP_HQD_PERSISTENT_STATE, tmp); mqd->cp_hqd_persistent_state = tmp;/* activate the queue */ mqd->cp_hqd_active =1 ;WREG32_SOC15 (GC ,0 , mmCP_HQD_ACTIVE, mqd->cp_hqd_active);soc15_grbm_select (adev,0 ,0 ,0 ,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock);amdgpu_bo_kunmap (ring->mqd_obj);amdgpu_bo_unreserve (ring->mqd_obj);if (use_doorbell)WREG32_FIELD15 (GC ,0 ,CP_PQ_STATUS ,DOORBELL_ENABLE ,1 );return 0 ; }const struct amdgpu_ip_block_version gfx_v9_0_ip_block = { .type =AMD_IP_BLOCK_TYPE_GFX , .major =9 , .minor =0 , .rev =0 , .funcs = &gfx_v9_0_ip_funcs, };
drivers/gpu/drm/amd/amdgpu/sdma_v2_4.c +2 -2
@@ -405,49 +405,49 @@ static void sdma_v2_4_enable(struct amdgpu_device *adev, bool enable)/** * sdma_v2_4_gfx_resume - setup and start the async dma engines * * @adev: amdgpu_device pointer * * Set up the gfx DMA ring buffers and enable them (VI). * Returns 0 for success, error for failure. */ static int sdma_v2_4_gfx_resume (struct amdgpu_device *adev) {struct amdgpu_ring *ring;u32 rb_cntl, ib_cntl;u32 rb_bufsz;u32 wb_offset;int i, j, r;for (i =0 ; i < adev->sdma.num_instances; i++) { ring = &adev->sdma.instance[i].ring; wb_offset = (ring->rptr_offs *4 ); -mutex_lock (&adev->srbm_mutex); +spin_lock (&adev->srbm_lock);for (j =0 ; j <16 ; j++) {vi_srbm_select (adev,0 ,0 ,0 , j);/* SDMA GFX */ WREG32 (mmSDMA0_GFX_VIRTUAL_ADDR + sdma_offsets[i],0 );WREG32 (mmSDMA0_GFX_APE1_CNTL + sdma_offsets[i],0 ); }vi_srbm_select (adev,0 ,0 ,0 ,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock);WREG32 (mmSDMA0_TILING_CONFIG + sdma_offsets[i], adev->gfx.config.gb_addr_config &0x70 );WREG32 (mmSDMA0_SEM_WAIT_FAIL_TIMER_CNTL + sdma_offsets[i],0 );/* Set ring buffer size in dwords */ rb_bufsz =order_base_2 (ring->ring_size /4 ); rb_cntl =RREG32 (mmSDMA0_GFX_RB_CNTL + sdma_offsets[i]); rb_cntl =REG_SET_FIELD (rb_cntl,SDMA0_GFX_RB_CNTL ,RB_SIZE , rb_bufsz);#ifdef __BIG_ENDIAN rb_cntl =REG_SET_FIELD (rb_cntl,SDMA0_GFX_RB_CNTL ,RB_SWAP_ENABLE ,1 ); rb_cntl =REG_SET_FIELD (rb_cntl,SDMA0_GFX_RB_CNTL ,RPTR_WRITEBACK_SWAP_ENABLE ,1 );#endif WREG32 (mmSDMA0_GFX_RB_CNTL + sdma_offsets[i], rb_cntl);/* Initialize the ring buffer's read and write pointers */ WREG32 (mmSDMA0_GFX_RB_RPTR + sdma_offsets[i],0 );WREG32 (mmSDMA0_GFX_RB_WPTR + sdma_offsets[i],0 );
drivers/gpu/drm/amd/amdgpu/sdma_v3_0.c +2 -2
@@ -599,49 +599,49 @@ static void sdma_v3_0_enable(struct amdgpu_device *adev, bool enable)* * @adev: amdgpu_device pointer * * Set up the gfx DMA ring buffers and enable them (VI). * Returns 0 for success, error for failure. */ static int sdma_v3_0_gfx_resume (struct amdgpu_device *adev) {struct amdgpu_ring *ring;u32 rb_cntl, ib_cntl;u32 rb_bufsz;u32 wb_offset;u32 doorbell;int i, j, r;for (i =0 ; i < adev->sdma.num_instances; i++) { ring = &adev->sdma.instance[i].ring;amdgpu_ring_clear_ring (ring); wb_offset = (ring->rptr_offs *4 ); -mutex_lock (&adev->srbm_mutex); +spin_lock (&adev->srbm_lock);for (j =0 ; j <16 ; j++) {vi_srbm_select (adev,0 ,0 ,0 , j);/* SDMA GFX */ WREG32 (mmSDMA0_GFX_VIRTUAL_ADDR + sdma_offsets[i],0 );WREG32 (mmSDMA0_GFX_APE1_CNTL + sdma_offsets[i],0 ); }vi_srbm_select (adev,0 ,0 ,0 ,0 ); -mutex_unlock (&adev->srbm_mutex); +spin_unlock (&adev->srbm_lock);WREG32 (mmSDMA0_TILING_CONFIG + sdma_offsets[i], adev->gfx.config.gb_addr_config &0x70 );WREG32 (mmSDMA0_SEM_WAIT_FAIL_TIMER_CNTL + sdma_offsets[i],0 );/* Set ring buffer size in dwords */ rb_bufsz =order_base_2 (ring->ring_size /4 ); rb_cntl =RREG32 (mmSDMA0_GFX_RB_CNTL + sdma_offsets[i]); rb_cntl =REG_SET_FIELD (rb_cntl,SDMA0_GFX_RB_CNTL ,RB_SIZE , rb_bufsz);#ifdef __BIG_ENDIAN rb_cntl =REG_SET_FIELD (rb_cntl,SDMA0_GFX_RB_CNTL ,RB_SWAP_ENABLE ,1 ); rb_cntl =REG_SET_FIELD (rb_cntl,SDMA0_GFX_RB_CNTL ,RPTR_WRITEBACK_SWAP_ENABLE ,1 );#endif WREG32 (mmSDMA0_GFX_RB_CNTL + sdma_offsets[i], rb_cntl);/* Initialize the ring buffer's read and write pointers */ WREG32 (mmSDMA0_GFX_RB_RPTR + sdma_offsets[i],0 );WREG32 (mmSDMA0_GFX_RB_WPTR + sdma_offsets[i],0 );
Signature · 6 lines
2.9.3 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 4/4 drm/amdgpu: implement ring set_priority for gfx_v8 compute v5
In reply to a message not in the archiveProgramming CP_HQD_QUEUE_PRIORITY enables a queue to take priority over other queues on the same pipe. Multiple queues on a pipe are timesliced so this gives us full precedence over other queues.
Programming CP_HQD_PIPE_PRIORITY changes the SPI_ARB_PRIORITY of the wave as follows:
0x2: CS_H
0x1: CS_M
0x0: CS_LThe SPI block will then dispatch work according to the policy set by SPI_ARB_PRIORITY. In the current policy CS_H is higher priority than gfx.
In order to prevent getting stuck in loops of CUs bouncing between GFX and high priority compute and introducing further latency, we reserve CUs 2+ for high priority compute on-demand.
v2: fix srbm_select to ring->queue and use ring->funcs->type
v3: use AMD_SCHED_PRIORITY_* instead of AMDGPU_CTX_PRIORITY_*
v4: switch int to enum amd_sched_priority
v5: corresponding changes for srbm_lock
--- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 3 + drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 1 + drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c | 95 ++++++++++++++++++++++++++++++ 3 files changed, 99 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +3 -0
@@ -1057,40 +1057,43 @@ struct amdgpu_gfx {const struct firmware *pfp_fw;/* PFP firmware */ uint32_t pfp_fw_version;const struct firmware *ce_fw;/* CE firmware */ uint32_t ce_fw_version;const struct firmware *rlc_fw;/* RLC firmware */ uint32_t rlc_fw_version;const struct firmware *mec_fw;/* MEC firmware */ uint32_t mec_fw_version;const struct firmware *mec2_fw;/* MEC2 firmware */ uint32_t mec2_fw_version;uint32_t me_feature_version;uint32_t ce_feature_version;uint32_t pfp_feature_version;uint32_t rlc_feature_version;uint32_t mec_feature_version;uint32_t mec2_feature_version;struct amdgpu_ring gfx_ring[AMDGPU_MAX_GFX_RINGS ];unsigned num_gfx_rings;struct amdgpu_ring compute_ring[AMDGPU_MAX_COMPUTE_RINGS ];unsigned num_compute_rings; +spinlock_t cu_reserve_lock; +uint32_t cu_reserve_pipe_mask; +uint32_t cu_reserve_queue_mask[AMDGPU_MAX_COMPUTE_RINGS ];struct amdgpu_irq_src eop_irq;struct amdgpu_irq_src priv_reg_irq;struct amdgpu_irq_src priv_inst_irq;/* gfx status */ uint32_t gfx_current_status;/* ce ram size*/ unsigned ce_ram_size;struct amdgpu_cu_info cu_info;const struct amdgpu_gfx_funcs *funcs;/* reset mask */ uint32_t grbm_soft_reset;uint32_t srbm_soft_reset;bool in_reset;/* s3/s4 mask */ bool in_suspend;/* NGG */ struct amdgpu_ngg ngg; };
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1 -0
@@ -1858,40 +1858,41 @@ int amdgpu_device_init(struct amdgpu_device *adev,mutex_init (&adev->firmware.mutex);mutex_init (&adev->pm.mutex);mutex_init (&adev->gfx.gpu_clock_mutex);spin_lock_init (&adev->srbm_lock);mutex_init (&adev->grbm_idx_mutex);mutex_init (&adev->mn_lock);hash_init (adev->mn_hash);amdgpu_check_arguments (adev);/* Registers mapping */ /* TODO: block userspace mapping of io register */ spin_lock_init (&adev->mmio_idx_lock);spin_lock_init (&adev->smc_idx_lock);spin_lock_init (&adev->pcie_idx_lock);spin_lock_init (&adev->uvd_ctx_idx_lock);spin_lock_init (&adev->didt_idx_lock);spin_lock_init (&adev->gc_cac_idx_lock);spin_lock_init (&adev->audio_endpt_idx_lock);spin_lock_init (&adev->mm_stats.lock); +spin_lock_init (&adev->gfx.cu_reserve_lock);INIT_LIST_HEAD (&adev->shadow_list);mutex_init (&adev->shadow_list_lock);INIT_LIST_HEAD (&adev->gtt_list);spin_lock_init (&adev->gtt_list_lock);INIT_LIST_HEAD (&adev->ring_lru_list);spin_lock_init (&adev->ring_lru_list_lock);if (adev->asic_type >=CHIP_BONAIRE ) { adev->rmmio_base =pci_resource_start (adev->pdev,5 ); adev->rmmio_size =pci_resource_len (adev->pdev,5 ); }else { adev->rmmio_base =pci_resource_start (adev->pdev,2 ); adev->rmmio_size =pci_resource_len (adev->pdev,2 ); } adev->rmmio =ioremap (adev->rmmio_base, adev->rmmio_size);if (adev->rmmio ==NULL ) {
drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c +95 -0
215 lines not on this page · raw message
Signature · 6 lines
2.9.3 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Quoted 19 lines · Andres Rodriguez
Third part of the split of the series:
Add support for high priority scheduling in amdgpu v8This is the part of the series that is in a bit more murky water than the rest.
Sending out this patch series mostly for completion. And maybe for discussion purposes as well. There are still 2 issues open with this series:
1) Is the spinlock patch still okay? Should we pursue this differently?
I'd rather not use a mutex here. That would mean that to program srbm registers from an interrupt we'd need to dispatch a worker thread. That could mean extra time that the CU reservation is in place which can impact performance.
So my preferred (biased) alternative is to still move to a spinlock.
Another alternative I'm not sure of: Can we take advantage of the KIQ FIFO semantics to perform srbm writes atomically?
Quoted 8 lines
Something like:
ib_append(ib, PKT_WRITE_REG(SRBM_SELECT(...))) ib_append(ib, PKT_WRITE_REG(SOME_REG, VAL) ib_append(ib, PKT_WRITE_REG(SRBM_SELECT(0, 0, 0))) ib_sumbit(kiq_ring, ib)Something that makes this immediately feel wrong though is the possibility of a race condition between an srbm operation on the KIQ and one through MMIO.
GFX has its own interface to srbm so it should never conflict with MMIO for access to things like indexed registers.
Another possibility would be to switch from using static queues to using runlists (similar to what KFD does, but still kernel based). Then everything could be packet driven. If you need to change the priority on a queue, you'd just adjust the value in the mqd before scheduling the runlist on the KIQ. I think there is even a special packet for setting the CU reservation.
Alex
Quoted 11 lines
2) Alex suggested changing some MMIO writes to happen on the KIQ instead. I
still haven't addressed that.I'm not sure the full criteria for patches landing on -wip. But if these are good enough to fix with some followup work, I wouldn't be oppossed to that idea.
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): CADnq5_Njj9RatxFSsy+D0xJpUGUAaN=wJcM=2GC4k=HJ8DO=xw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Quoted 26 lines · Alex Deucher
Quoted 25 lines · Andres Rodriguez
Third part of the split of the series:
Add support for high priority scheduling in amdgpu v8This is the part of the series that is in a bit more murky water than the rest.
Sending out this patch series mostly for completion. And maybe for discussion purposes as well. There are still 2 issues open with this series:
1) Is the spinlock patch still okay? Should we pursue this differently?
I'd rather not use a mutex here. That would mean that to program srbm registers from an interrupt we'd need to dispatch a worker thread. That could mean extra time that the CU reservation is in place which can impact performance.
So my preferred (biased) alternative is to still move to a spinlock.
Another alternative I'm not sure of: Can we take advantage of the KIQ FIFO semantics to perform srbm writes atomically?
Something like:
ib_append(ib, PKT_WRITE_REG(SRBM_SELECT(...))) ib_append(ib, PKT_WRITE_REG(SOME_REG, VAL) ib_append(ib, PKT_WRITE_REG(SRBM_SELECT(0, 0, 0))) ib_sumbit(kiq_ring, ib)
Quoted 5 lines
Quoted 2 lines
Something that makes this immediately feel wrong though is the possibility of a race condition between an srbm operation on the KIQ and one through MMIO.
GFX has its own interface to srbm so it should never conflict with MMIO for access to things like indexed registers.
Silly question. I'm assuming GFX here includes compute queues (i.e. your sentence referred to the GFX HW block which includes the KIQ). Correct?
Would I have to worry about submitting writes on two different pipes simultaneously? I think that working on the assumption that register programming would only happen on the KIQ should be enought
Quoted 7 lines
Another possibility would be to switch from using static queues to using runlists (similar to what KFD does, but still kernel based). Then everything could be packet driven. If you need to change the priority on a queue, you'd just adjust the value in the mqd before scheduling the runlist on the KIQ. I think there is even a special packet for setting the CU reservation.
I took a quick look last time through the docs and couldn't find anything. Also a quick grep through the register headers and I didn't stumble into anything. If you do find it I would be interested.
Quoted 14 lines
Alex
Quoted 11 lines
2) Alex suggested changing some MMIO writes to happen on the KIQ instead. I
still haven't addressed that.I'm not sure the full criteria for patches landing on -wip. But if these are good enough to fix with some followup work, I wouldn't be oppossed to that idea.
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx