drm/amdgpu: fix deadlock of reservation between cs and gpu reset v2
Subsystems (2)
PATCH 1/1 drm/amdgpu: fix deadlock of reservation between cs and gpu reset v2
the case could happen when gpu reset:
1. when gpu reset, cs can be continue until sw queue is full, then push job will wait with holding pd reservation.
2. gpu_reset routine will also need pd reservation to restore page table from their shadow.
3. cs is waiting for gpu_reset complete, but gpu reset is waiting for cs releases reservation.
v2: handle amdgpu_cs_submit error path.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 4 ++++ 1 file changed, 4 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +4 -0
@@ -1074,6 +1074,7 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p, cs->out.handle =amdgpu_ctx_add_fence (p->ctx, ring, p->fence); job->uf_sequence = cs->out.handle;amdgpu_job_free_resources (job); +amdgpu_cs_parser_fini (p,0 , true);trace_amdgpu_cs_ioctl (job);amd_sched_entity_push_job (&job->base);
@@ -1129,7 +1130,10 @@ int amdgpu_cs_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) goto out; r =amdgpu_cs_submit (&parser, cs); +if (r) + goto out; +return 0 ; out:amdgpu_cs_parser_fini (&parser, r, reserved_buffers);return r;
Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 1493347534-8201-1-git-send-email-David1.Zhou-5C7GfCeVMHo@public.gmane.org
Zhang, Jerry
Reviewed-by Junwei ZhangNice catch!
Regards,
Jerry (Junwei Zhang)
Linux Base Graphics
SRDC Software Development
_____________________________________
Quoted 57 lines
-----Original Message-----
From: amd-gfx [mailto:[email protected]] On Behalf Of Chunming Zhou
Sent: Friday, April 28, 2017 10:46
To: [email protected]
Cc: Zhou, David(ChunMing)
Subject: [PATCH] drm/amdgpu: fix deadlock of reservation between cs and gpu reset v2the case could happen when gpu reset:
1. when gpu reset, cs can be continue until sw queue is full, then push job will wait with holding pd reservation.
2.gpu_resetroutine will also need pd reservation to restore page table from their shadow.
3. cs is waiting forgpu_resetcomplete, but gpu reset is waiting for cs releases reservation.v2: handle
amdgpu_cs_submiterror path.Change-Id I0f66d04b2bef3433035109623c8a5c5992c84202Signed-off-by Chunming Zhou <[email protected]>Reviewed-by Christian König <[email protected]>Reviewed-by Junwei Zhang <[email protected]>Reviewed-by Monk Liu <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 4 ++++ 1 file changed, 4 insertions(+)drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +1
-0@@ -1074,6 +1074,7 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p, cs->out.handle =amdgpu_ctx_add_fence (p->ctx, ring, p->fence); job->uf_sequence = cs->out.handle;amdgpu_job_free_resources (job); +amdgpu_cs_parser_fini (p,0 , true);trace_amdgpu_cs_ioctl (job);drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +3
-0amd_sched_entity_push_job (&job->base);@@ -1129,7 +1130,10 @@ int amdgpu_cs_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) goto out; r =amdgpu_cs_submit (&parser, cs); +if (r) + goto out; +return 0 ; out:amdgpu_cs_parser_fini (&parser, r, reserved_buffers);return r;
--
1.9.1
Quoted 4 lines
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): DM5PR12MB181832DC8E8A41338E5748F4FF130-2J9CzHegvk+QhrfEZJlvtAdYzm3356FpvxpqHgZTriW3zl9H0oFU5g@public.gmane.org
Indeed, but after a bit of thinking I've found another problem with that patch.
When two threads are pushing jobs into the same scheduler context we don't guarantee correct execution order any more!
Before that patch it was handled by the exclusiveness we had because of reserving the VM page tables, but now nothing prevents us from calling amd_sched_entity_push_job() in nondeterministic order.
In other words we need an additional lock in amdgpu_ctx_ring or something like that.
Regards,
Christian.
Am 28.04.2017 um 04:51 schrieb Zhang, Jerry:
Quoted 68 lines
Nice catch!
Reviewed-by Junwei Zhang <[email protected]>Regards,
Jerry (Junwei Zhang)Linux Base Graphics
SRDC Software Development
_____________________________________Quoted 57 lines
-----Original Message-----
From: amd-gfx [mailto:[email protected]] On Behalf Of Chunming Zhou
Sent: Friday, April 28, 2017 10:46
To: [email protected]
Cc: Zhou, David(ChunMing)
Subject: [PATCH] drm/amdgpu: fix deadlock of reservation between cs and gpu reset v2the case could happen when gpu reset:
1. when gpu reset, cs can be continue until sw queue is full, then push job will wait with holding pd reservation.
2.gpu_resetroutine will also need pd reservation to restore page table from their shadow.
3. cs is waiting forgpu_resetcomplete, but gpu reset is waiting for cs releases reservation.v2: handle
amdgpu_cs_submiterror path.Change-Id I0f66d04b2bef3433035109623c8a5c5992c84202Signed-off-by Chunming Zhou <[email protected]>Reviewed-by Christian König <[email protected]>Reviewed-by Junwei Zhang <[email protected]>Reviewed-by Monk Liu <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 4 ++++ 1 file changed, 4 insertions(+)drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +1
-0@@ -1074,6 +1074,7 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p, cs->out.handle =amdgpu_ctx_add_fence (p->ctx, ring, p->fence); job->uf_sequence = cs->out.handle;amdgpu_job_free_resources (job); +amdgpu_cs_parser_fini (p,0 , true);trace_amdgpu_cs_ioctl (job);amd_sched_entity_push_job(&job->base);drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +3
-0@@ -1129,7 +1130,10 @@ int amdgpu_cs_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) goto out; r =amdgpu_cs_submit (&parser, cs); +if (r) + goto out; +return 0 ; out:amdgpu_cs_parser_fini (&parser, r, reserved_buffers);return r;
--
1.9.1
Quoted 8 lines
Quoted 4 lines
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): a8fe5428-756b-9fb8-a7ba-52ea9058f90c-ANTagKRnAhcb1SvskN2V4Q@public.gmane.org
Agree, but libdrm doesn't allow concurrent submissions from same context, like protection 'pthread_mutex_lock(&context->sequence_mutex);' in amdgpu_cs_submit_one.
Regards,
David Zhou
Quoted 4 lines · Christian König
Indeed, but after a bit of thinking I've found another problem with that patch.
When two threads are pushing jobs into the same scheduler context we don't guarantee correct execution order any more!
Quoted 87 lines
Before that patch it was handled by the exclusiveness we had because of reserving the VM page tables, but now nothing prevents us from calling
amd_sched_entity_push_job()in nondeterministic order.In other words we need an additional lock in
amdgpu_ctx_ringor something like that.Regards,
Christian.Am 28.04.2017 um 04:51 schrieb Zhang, Jerry:
Quoted 77 lines
Nice catch!
Reviewed-by Junwei Zhang <[email protected]>Regards,
Jerry (Junwei Zhang)Linux Base Graphics
SRDC Software Development
_____________________________________Quoted 54 lines
-----Original Message-----
From: amd-gfx [mailto:[email protected]] On Behalf Of Chunming Zhou
Sent: Friday, April 28, 2017 10:46
To: [email protected]
Cc: Zhou, David(ChunMing)
Subject: [PATCH] drm/amdgpu: fix deadlock of reservation between cs and gpu reset v2the case could happen when gpu reset:
1. when gpu reset, cs can be continue until sw queue is full, then push job will wait with holding pd reservation.
2.gpu_resetroutine will also need pd reservation to restore page table from their shadow.
3. cs is waiting forgpu_resetcomplete, but gpu reset is waiting for cs releases reservation.v2: handle
amdgpu_cs_submiterror path.Change-Id I0f66d04b2bef3433035109623c8a5c5992c84202Signed-off-by Chunming Zhou <[email protected]>Reviewed-by Christian König <[email protected]>Reviewed-by Junwei Zhang <[email protected]>Reviewed-by Monk Liu <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 4 ++++ 1 file changed, 4 insertions(+)drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +0
-0@@ -1074,6 +1074,7 @@ static int amdgpu_cs_submit(struct
amdgpu_cs_parser
*p,cs->out.handle = amdgpu_ctx_add_fence(p->ctx, ring, p->fence); job->uf_sequence = cs->out.handle; amdgpu_job_free_resources(job);drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +4
-0+amdgpu_cs_parser_fini (p,0 , true);trace_amdgpu_cs_ioctl (job);amd_sched_entity_push_job (&job->base);@@ -1129,7 +1130,10 @@ int amdgpu_cs_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) goto out; r =amdgpu_cs_submit (&parser, cs); +if (r) + goto out; +return 0 ; out:amdgpu_cs_parser_fini (&parser, r, reserved_buffers);Quoted 8 lines
return r;Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx