drm/amd/amdgpu: locking fixes
Subsystems (2)
cover drm/amd/amdgpu: locking fixes
Hi all,
two fixes for locking issues that I noticed.
The first one is something that I actually encountered live; it probably only matters when lock debugging is enabled, but obviously needs to be fixed anyway.
The second one I only noticed upon reading the code -- I haven't seen it fail live yet, at least not with lock debugging enabled (who knows what its effects could be without lock debugging).
Please review!
I noticed two more locking-related problems, but since I'm not sure whether I'll get to those this week, I'm sending out these two patches already. The other two problems are (both of them probably mostly benign, but I did encounter them in live tests):
1. amdgpu_bo_create_restricted(resv == NULL) --> ttm_bo_init will free a reservation object while it is locked. Fixing this soundly probably requires mild changes to the ttm_bo_init API (perhaps a separate ttm_bo_init_locked?).
2. ttm_bo_pipeline_move will ttm_bo_unref(&ghost_obj) leading to mutex_destroy(&ghost_obj->wu_mutex) while that mutex "looks" locked (due to the dumb copy).
Thanks,
Nicolai
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 1484119865-11390-1-git-send-email-nhaehnle-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
PATCH 1/2 drm/amd/amdgpu: lock reservation object while creating shadow bo
From: Nicolai Hähnle <[email protected]>
ttm_bo_init checks that the reservation object is locked. This is the caller's responsibility when resv != NULL. Otherwise, the inline reservation object of the newly allocated buffer is used and must explicitly be locked. Using a trylock is fine, since nobody else has a reference to the reservation lock.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +11 -0
@@ -465,21 +465,32 @@ int amdgpu_bo_create(struct amdgpu_device *adev,amdgpu_ttm_placement_init (adev, &placement, placements, domain, flags); r =amdgpu_bo_create_restricted (adev, size, byte_align, kernel, domain, flags, sg, &placement, resv, bo_ptr);if (r)return r;if (amdgpu_need_backup (adev) && (flags &AMDGPU_GEM_CREATE_SHADOW )) { +if (!resv) { +bool locked; + + locked =ww_mutex_trylock (&(*bo_ptr)->tbo.resv->lock); +WARN_ON (!locked); + } + r =amdgpu_bo_create_shadow (adev, size, byte_align, (*bo_ptr)); + +if (!resv) +ww_mutex_unlock (&(*bo_ptr)->tbo.resv->lock); +if (r)amdgpu_bo_unref (bo_ptr); }return r; } int amdgpu_bo_backup_to_shadow(struct amdgpu_device *adev, struct amdgpu_ring *ring, struct amdgpu_bo *bo,
Signature · 6 lines
2.7.4 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 1484119865-11390-2-git-send-email-nhaehnle-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Re: [PATCH 1/2] drm/amd/amdgpu: lock reservation object while creating shadow bo
Am 11.01.2017 um 08:31 schrieb Nicolai Hähnle:
Quoted 7 lines
From: Nicolai Hähnle <[email protected]>
ttm_bo_initchecks that the reservation object is locked. This is the caller's responsibility when resv != NULL. Otherwise, the inline reservation object of the newly allocated buffer is used and must explicitly be locked. Using a trylock is fine, since nobody else has a reference to the reservation lock.
Good, catch. But while using trylock is fine, why do you need to use it in the first place?
Regards,
Christian.
Quoted 33 lines
Signed-off-by Nicolai Hähnle <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 11 +++++++++++ 1 file changed, 11 insertions(+)drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +11
-0@@ -465,21 +465,32 @@ int amdgpu_bo_create(struct amdgpu_device *adev,amdgpu_ttm_placement_init (adev, &placement, placements, domain, flags); r =amdgpu_bo_create_restricted (adev, size, byte_align, kernel, domain, flags, sg, &placement, resv, bo_ptr);if (r)return r;if (amdgpu_need_backup (adev) && (flags &AMDGPU_GEM_CREATE_SHADOW )) { +if (!resv) { +bool locked; + + locked =ww_mutex_trylock (&(*bo_ptr)->tbo.resv->lock); +WARN_ON (!locked); + } + r =amdgpu_bo_create_shadow (adev, size, byte_align, (*bo_ptr)); + +if (!resv) +ww_mutex_unlock (&(*bo_ptr)->tbo.resv->lock); +
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c int amdgpu_bo_create(struct amdgpu_device *adev, · line 496
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +0
-0if (r)amdgpu_bo_unref (bo_ptr); }return r; } int amdgpu_bo_backup_to_shadow(struct amdgpu_device *adev, struct amdgpu_ring *ring, struct amdgpu_bo *bo,
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 621bf6ab-ad3b-7655-c633-d18c051f4667-ANTagKRnAhcb1SvskN2V4Q@public.gmane.org
Re: [PATCH 1/2] drm/amd/amdgpu: lock reservation object while creating shadow bo
Quoted 11 lines · Christian König
Am 11.01.2017 um 08:31 schrieb Nicolai Hähnle:
Quoted 7 lines
From: Nicolai Hähnle <[email protected]>
ttm_bo_initchecks that the reservation object is locked. This is the caller's responsibility when resv != NULL. Otherwise, the inline reservation object of the newly allocated buffer is used and must explicitly be locked. Using a trylock is fine, since nobody else has a reference to the reservation lock.Good, catch. But while using trylock is fine, why do you need to use it in the first place?
(*bo_ptr)->tbo.resv is passed as the reservation object to ttm_bo_init via amdgpu_bo_create_shadow, and ttm_bo_init has a lockdep check for it.
I haven't dug deeper whether that's _really_ necessary, but since among other things ttm_bo_validate is called by ttm_bo_init, I'd assume that it is (ttm_bo_validate also has a lockdep check). If it isn't, then the internal locking of bo->resv->lock done by ttm_bo_init in the !resv case would also be unnecessary...
Cheers,
Nicolai
Quoted 37 lines
Regards,
Christian.Quoted 33 lines
Signed-off-by Nicolai Hähnle <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 11 +++++++++++ 1 file changed, 11 insertions(+)drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +11
-0@@ -465,21 +465,32 @@ int amdgpu_bo_create(struct amdgpu_device *adev,amdgpu_ttm_placement_init (adev, &placement, placements, domain, flags); r =amdgpu_bo_create_restricted (adev, size, byte_align, kernel, domain, flags, sg, &placement, resv, bo_ptr);if (r)return r;if (amdgpu_need_backup (adev) && (flags &AMDGPU_GEM_CREATE_SHADOW )) { +if (!resv) { +bool locked; + + locked =ww_mutex_trylock (&(*bo_ptr)->tbo.resv->lock); +WARN_ON (!locked); + } + r =amdgpu_bo_create_shadow (adev, size, byte_align, (*bo_ptr)); + +if (!resv) +ww_mutex_unlock (&(*bo_ptr)->tbo.resv->lock); +
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c int amdgpu_bo_create(struct amdgpu_device *adev, · line 496
Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +0
-0if (r)amdgpu_bo_unref (bo_ptr); }return r; } int amdgpu_bo_backup_to_shadow(struct amdgpu_device *adev, struct amdgpu_ring *ring, struct amdgpu_bo *bo,
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Re: [PATCH 1/2] drm/amd/amdgpu: lock reservation object while creating shadow bo
In reply to a message not in the archiveQuoted 11 lines · Christian König
Am 11.01.2017 um 08:31 schrieb Nicolai Hähnle:
Quoted 7 lines
From: Nicolai Hähnle <[email protected]>
ttm_bo_initchecks that the reservation object is locked. This is the caller's responsibility when resv != NULL. Otherwise, the inline reservation object of the newly allocated buffer is used and must explicitly be locked. Using a trylock is fine, since nobody else has a reference to the reservation lock.Good, catch. But while using trylock is fine, why do you need to use it in the first place?
Or is this a question about trylock vs. lock?
Actually, now that I think about this again, perhaps the following sequence would be possible:
1. Create the main bo in amdgpu_bo_create.
2. Other thread, for whatever reason, tries to make space and evict the bo, and probably locks the reservation for that?
3. amdgpu_bo_create continues, calls the trylock which now fails.
Nicolai
Quoted 37 lines
Regards,
Christian.Quoted 33 lines
Signed-off-by Nicolai Hähnle <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 11 +++++++++++ 1 file changed, 11 insertions(+)drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +11
-0@@ -465,21 +465,32 @@ int amdgpu_bo_create(struct amdgpu_device *adev,amdgpu_ttm_placement_init (adev, &placement, placements, domain, flags); r =amdgpu_bo_create_restricted (adev, size, byte_align, kernel, domain, flags, sg, &placement, resv, bo_ptr);if (r)return r;if (amdgpu_need_backup (adev) && (flags &AMDGPU_GEM_CREATE_SHADOW )) { +if (!resv) { +bool locked; + + locked =ww_mutex_trylock (&(*bo_ptr)->tbo.resv->lock); +WARN_ON (!locked); + } + r =amdgpu_bo_create_shadow (adev, size, byte_align, (*bo_ptr)); + +if (!resv) +ww_mutex_unlock (&(*bo_ptr)->tbo.resv->lock); +
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c int amdgpu_bo_create(struct amdgpu_device *adev, · line 496
Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +0
-0if (r)amdgpu_bo_unref (bo_ptr); }return r; } int amdgpu_bo_backup_to_shadow(struct amdgpu_device *adev, struct amdgpu_ring *ring, struct amdgpu_bo *bo,
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 56e05f07-2b21-5fc9-7d69-9acfe6d0b519-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Re: [PATCH 1/2] drm/amd/amdgpu: lock reservation object while creating shadow bo
Am 11.01.2017 um 15:09 schrieb Nicolai Hähnle:
Quoted 13 lines · Christian König
Quoted 11 lines
Am 11.01.2017 um 08:31 schrieb Nicolai Hähnle:
Quoted 7 lines
From: Nicolai Hähnle <[email protected]>
ttm_bo_initchecks that the reservation object is locked. This is the caller's responsibility when resv != NULL. Otherwise, the inline reservation object of the newly allocated buffer is used and must explicitly be locked. Using a trylock is fine, since nobody else has a reference to the reservation lock.Good, catch. But while using trylock is fine, why do you need to use it in the first place?
Quoted 7 lines
Or is this a question about trylock vs. lock?
Actually, now that I think about this again, perhaps the following sequence would be possible:
1. Create the main bo in
amdgpu_bo_create.
2. Other thread, for whatever reason, tries to make space and evict the bo, and probably locks the reservation for that?
3.amdgpu_bo_createcontinues, calls the trylock which now fails.
Yes, exactly that was my thinking as well.
Please just use the normal uninterruptible locking variant if there isn't a good reason for using trylock.
Christian.
Quoted 51 lines
Nicolai
Quoted 38 lines
Regards,
Christian.Quoted 34 lines
Signed-off-by Nicolai Hähnle <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 11 +++++++++++ 1 file changed, 11 insertions(+)drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +11
-0@@ -465,21 +465,32 @@ int amdgpu_bo_create(struct amdgpu_device *adev,amdgpu_ttm_placement_init (adev, &placement, placements, domain, flags); r =amdgpu_bo_create_restricted (adev, size, byte_align, kernel, domain, flags, sg, &placement, resv, bo_ptr);if (r)return r;if (amdgpu_need_backup (adev) && (flags &AMDGPU_GEM_CREATE_SHADOW )) { +if (!resv) { +bool locked; + + locked =ww_mutex_trylock (&(*bo_ptr)->tbo.resv->lock); +WARN_ON (!locked); + } + r =amdgpu_bo_create_shadow (adev, size, byte_align, (*bo_ptr)); + +if (!resv) +ww_mutex_unlock (&(*bo_ptr)->tbo.resv->lock); +Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c int amdgpu_bo_create(struct amdgpu_device *adev, · line 496
Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +0
-0if (r)amdgpu_bo_unref (bo_ptr); }return r; } int amdgpu_bo_backup_to_shadow(struct amdgpu_device *adev, struct amdgpu_ring *ring, struct amdgpu_bo *bo,
Quoted 4 lines
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 2/2 drm/amd/amdgpu: fix locking in bo creation error path
In reply to a message not in the archiveFrom: Nicolai Hähnle <[email protected]>
Unlock the resv lock only if we were the ones to lock it in the first place.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +2 -1
@@ -401,21 +401,22 @@ int amdgpu_bo_create_restricted(struct amdgpu_device *adev, }if (!resv)ww_mutex_unlock (&bo->tbo.resv->lock); *bo_ptr = bo;trace_amdgpu_bo_create (bo);return 0 ; fail_unreserve: -ww_mutex_unlock (&bo->tbo.resv->lock); +if (!resv) +ww_mutex_unlock (&bo->tbo.resv->lock);amdgpu_bo_unref (&bo);return r; }static int amdgpu_bo_create_shadow (struct amdgpu_device *adev,unsigned long size,int byte_align,struct amdgpu_bo *bo) {struct ttm_placement placement = {0 };struct ttm_place placements[AMDGPU_GEM_DOMAIN_MAX +1 ];
Signature · 6 lines
2.7.4 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 1484119865-11390-3-git-send-email-nhaehnle-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Christian König
Reviewed-byRe: [PATCH 2/2] drm/amd/amdgpu: fix locking in bo creation error path
Am 11.01.2017 um 08:31 schrieb Nicolai Hähnle:
Quoted 6 lines
From: Nicolai Hähnle <[email protected]>
Unlock the resv lock only if we were the ones to lock it in the first place.
Signed-off-by Nicolai Hähnle <[email protected]>
Quoted 22 lines
--- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +2
-1@@ -401,21 +401,22 @@ int amdgpu_bo_create_restricted(struct amdgpu_device *adev, }if (!resv)ww_mutex_unlock (&bo->tbo.resv->lock); *bo_ptr = bo;trace_amdgpu_bo_create (bo);return 0 ; fail_unreserve: -ww_mutex_unlock (&bo->tbo.resv->lock); +if (!resv) +ww_mutex_unlock (&bo->tbo.resv->lock);
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c int amdgpu_bo_create_restricted(struct amdgpu_device *adev, · line 422
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +0
-0amdgpu_bo_unref (&bo);return r; }static int amdgpu_bo_create_shadow (struct amdgpu_device *adev,unsigned long size,int byte_align,struct amdgpu_bo *bo) {struct ttm_placement placement = {0 };struct ttm_place placements[AMDGPU_GEM_DOMAIN_MAX +1 ];
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Edward O'Callaghan
Reviewed-bythis series is,
Quoted 28 lines · Nicolai Hähnle
Hi all,
two fixes for locking issues that I noticed.
The first one is something that I actually encountered live; it probably only matters when lock debugging is enabled, but obviously needs to be fixed anyway.
The second one I only noticed upon reading the code -- I haven't seen it fail live yet, at least not with lock debugging enabled (who knows what its effects could be without lock debugging).
Please review!
I noticed two more locking-related problems, but since I'm not sure whether I'll get to those this week, I'm sending out these two patches already. The other two problems are (both of them probably mostly benign, but I did encounter them in live tests):
1.
amdgpu_bo_create_restricted(resv == NULL)-->ttm_bo_initwill free a reservation object while it is locked. Fixing this soundly probably requires mild changes to thettm_bo_initAPI (perhaps a separatettm_bo_init_locked?).2.
ttm_bo_pipeline_movewillttm_bo_unref(&ghost_obj)leading tomutex_destroy(&ghost_obj->wu_mutex)while that mutex "looks" locked (due to the dumb copy).
Quoted 8 lines
Thanks,
Nicolai_______________________________________________
amd-gfx mailing list [email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx