drm/amdgpu: Reserve shared fence slot in amdgpu_vm_update_directories
Subsystems (2)
PATCH 1/1 drm/amdgpu: Reserve shared fence slot in amdgpu_vm_update_directories
From: Michel Dänzer <[email protected]>
Without this, there could not be enough slots, which could trigger the BUG_ON in reservation_object_add_shared_fence.
Cc: [email protected]
Bugzilla: https://bugs.freedesktop.org/106418
--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 4 ++++ 1 file changed, 4 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +4 -0
@@ -1111,6 +1111,10 @@ int amdgpu_vm_update_directories(struct amdgpu_device *adev,struct amdgpu_ring *ring;struct dma_fence *fence; + r =reservation_object_reserve_shared (root->tbo.resv); +if (r) +return r; + ring = container_of(vm->entity.sched, struct amdgpu_ring, sched);
Signature · 6 lines
2.17.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 20180622164235.30770-1-michel-otUistvHUpPR7s880joybQ@public.gmane.org
Quoted 18 lines · Michel Dänzer
From: Michel Dänzer <[email protected]>
Without this, there could not be enough slots, which could trigger the
BUG_ONinreservation_object_add_shared_fence.Cc: [email protected]
Bugzilla: https://bugs.freedesktop.org/106418Reported-by [email protected]Signed-off-by Michel Dänzer <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 4 ++++ 1 file changed, 4 insertions(+)drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0
-0
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c int amdgpu_vm_update_directories(struct amdgpu_device *adev, · line 1116
Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +3
-0@@ -1111,6 +1111,10 @@ int amdgpu_vm_update_directories(struct amdgpu_device *adev,struct amdgpu_ring *ring;struct dma_fence *fence; + r =reservation_object_reserve_shared (root->tbo.resv); +if (r) +return r;
Good catch.
But we may "goto error" here to do some cleanup.
Jerry
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c int amdgpu_vm_update_directories(struct amdgpu_device *adev, · line 1120
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +1
-0+ ring = container_of(vm->entity.sched, struct amdgpu_ring, sched);
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 24 lines · Jerry (Junwei)
Quoted 23 lines · Michel Dänzer
From: Michel Dänzer <[email protected]>
Without this, there could not be enough slots, which could trigger the
BUG_ONinreservation_object_add_shared_fence.Cc: [email protected]
Bugzilla: https://bugs.freedesktop.org/106418Reported-by [email protected]Signed-off-by Michel Dänzer <[email protected]>---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c| 4 ++++1 file changed, 4 insertions(+)drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0
-0@@ -1111,6 +1111,10 @@ int amdgpu_vm_update_directories(struct amdgpu_device *adev, struct amdgpu_ring *ring; struct dma_fence *fence;
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c int amdgpu_vm_update_directories(struct amdgpu_device *adev, · line 1116
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +3
-0+ r = reservation_object_reserve_shared(root->tbo.resv); + if (r) + return r;Good catch.
But we may "goto error" here to do some cleanup.
Indeed, thanks, fixed in v2.
Signature · 6 lines
Earthling Michel Dänzer | http://www.amd.com Libre software enthusiast | Mesa and X developer _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH v2 1/1 drm/amdgpu: Reserve shared fence slot in amdgpu_vm_update_directories
In reply to a message not in the archiveFrom: Michel Dänzer <[email protected]>
Without this, there could not be enough slots, which could trigger the BUG_ON in reservation_object_add_shared_fence.
v2:
* Jump to the error label instead of returning directly (Jerry Zhang)
Cc: [email protected]
Bugzilla: https://bugs.freedesktop.org/106418
--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 4 ++++ 1 file changed, 4 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +4 -0
@@ -1111,6 +1111,10 @@ int amdgpu_vm_update_directories(struct amdgpu_device *adev,struct amdgpu_ring *ring;struct dma_fence *fence; + r =reservation_object_reserve_shared (root->tbo.resv); +if (r) + goto error; + ring = container_of(vm->entity.sched, struct amdgpu_ring, sched);
Signature · 6 lines
2.17.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 20180625090717.28522-1-michel-otUistvHUpPR7s880joybQ@public.gmane.org
Zhang, Jerry (Junwei)
Reviewed-by Junwei ZhangQuoted 5 lines · Michel Dänzer
From: Michel Dänzer <[email protected]>
Without this, there could not be enough slots, which could trigger the
BUG_ONinreservation_object_add_shared_fence.
Quoted 7 lines
v2:
* Jump to the error label instead of returning directly (Jerry Zhang)Cc: [email protected]
Bugzilla: https://bugs.freedesktop.org/106418Reported-by [email protected]Signed-off-by Michel Dänzer <[email protected]>
Quoted 10 lines
--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 4 ++++ 1 file changed, 4 insertions(+)drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0
-0
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c int amdgpu_vm_update_directories(struct amdgpu_device *adev, · line 1120
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +4
-0@@ -1111,6 +1111,10 @@ int amdgpu_vm_update_directories(struct amdgpu_device *adev,struct amdgpu_ring *ring;struct dma_fence *fence; + r =reservation_object_reserve_shared (root->tbo.resv); +if (r) + goto error; + ring = container_of(vm->entity.sched, struct amdgpu_ring, sched);
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Am 25.06.2018 um 11:07 schrieb Michel Dänzer:
Quoted 7 lines
From: Michel Dänzer <[email protected]>
Without this, there could not be enough slots, which could trigger the
BUG_ONinreservation_object_add_shared_fence.v2:
* Jump to the error label instead of returning directly (Jerry Zhang)
Well good catch, but NAK that is not the right solution to the issue.
amdgpu_vm_update_directories() is only called by two places, the first one is amdgpu_bo_vm_update_pte() and the second amdgpu_gem_va_update_vm().
In both cases the directory is reserved using ttm_eu_reserve_buffers() with "entry->tv.shared = true;".
What most likely needs to be done instead is to call reservation_object_reserve_shared() after all the VM operations in amdgpu_bo_vm_update_pte() to make sure that we still have one additional slot for the command submission.
Regards,
Christian.
Quoted 14 lines
Cc: [email protected]
Bugzilla: https://bugs.freedesktop.org/106418Reported-by [email protected]Signed-off-by Michel Dänzer <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 4 ++++ 1 file changed, 4 insertions(+)drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0
-0
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c int amdgpu_vm_update_directories(struct amdgpu_device *adev, · line 1120
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +4
-0@@ -1111,6 +1111,10 @@ int amdgpu_vm_update_directories(struct amdgpu_device *adev,struct amdgpu_ring *ring;struct dma_fence *fence; + r =reservation_object_reserve_shared (root->tbo.resv); +if (r) + goto error; + ring = container_of(vm->entity.sched, struct amdgpu_ring, sched);
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 15c3905c-8e69-9d03-e506-929e4d9176e3-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Quoted 9 lines · Christian König
Am 25.06.2018 um 11:07 schrieb Michel Dänzer:
Quoted 7 lines
From: Michel Dänzer <[email protected]>
Without this, there could not be enough slots, which could trigger the
BUG_ONinreservation_object_add_shared_fence.v2:
* Jump to the error label instead of returning directly (Jerry Zhang)
Quoted 8 lines
Well good catch, but NAK that is not the right solution to the issue.
amdgpu_vm_update_directories()is only called by two places, the first one isamdgpu_bo_vm_update_pte()and the secondamdgpu_gem_va_update_vm().In both cases the directory is reserved using
ttm_eu_reserve_buffers()with "entry->tv.shared= true;".What most likely needs to be done instead is to call
reservation_object_reserve_shared()after all the VM operations inamdgpu_bo_vm_update_pte()to make sure that we still have one additional slot for the command submission.
In this case, it looks like amdgpu_vm_bo_update_mapping() handling, inside ttm_eu_reserve_buffers() as well.
Is there any specific reason to call it at the end of amdgpu_bo_vm_update_pte()?
AFAIW, that has to be called right before where to add shared slot.
Jerry
Quoted 23 lines
Regards,
Christian.Quoted 19 lines
Cc: [email protected]
Bugzilla: https://bugs.freedesktop.org/106418Reported-by [email protected]Signed-off-by Michel Dänzer <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 4 ++++ 1 file changed, 4 insertions(+)drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +3
-0@@ -1111,6 +1111,10 @@ int amdgpu_vm_update_directories(struct amdgpu_device *adev,struct amdgpu_ring *ring;struct dma_fence *fence; + r =reservation_object_reserve_shared (root->tbo.resv); +if (r) + goto error;
Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c int amdgpu_vm_update_directories(struct amdgpu_device *adev, · line 1119
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +1
-0+ ring = container_of(vm->entity.sched, struct amdgpu_ring, sched);_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): [email protected]
Am 03.07.2018 um 03:59 schrieb Zhang, Jerry (Junwei):
Quoted 16 lines · Christian König
Quoted 15 lines
Am 25.06.2018 um 11:07 schrieb Michel Dänzer:
Quoted 7 lines
From: Michel Dänzer <[email protected]>
Without this, there could not be enough slots, which could trigger the
BUG_ONinreservation_object_add_shared_fence.v2:
* Jump to the error label instead of returning directly (Jerry Zhang)Well good catch, but NAK that is not the right solution to the issue.
amdgpu_vm_update_directories()is only called by two places, the first one isamdgpu_bo_vm_update_pte()and the secondamdgpu_gem_va_update_vm().In both cases the directory is reserved using
ttm_eu_reserve_buffers()with "entry->tv.shared= true;".
Quoted 8 lines
Quoted 2 lines
What most likely needs to be done instead is to call
reservation_object_reserve_shared()after all the VM operations inamdgpu_bo_vm_update_pte()to make sure that we still have one additional slot for the command submission.In this case, it looks like
amdgpu_vm_bo_update_mapping()handling, insidettm_eu_reserve_buffers()as well.Is there any specific reason to call it at the end of
amdgpu_bo_vm_update_pte()?AFAIW, that has to be called right before where to add shared slot.
The reservation_object_reserve_shared() function needs to be called before you make submissions to the hardware to make sure that you have a free slot after you made the submission.
The problem is now that we always reserve one free slot during command submission, but for the page directory reservation object it can happen that this free slot is used by VM updates.
So the safest approach is to make sure that we have another free slot after the VM updates are done.
Michel's patch nearly did that, because we rarely have only one submission in amdgpu_vm_update_directories(). But it isn't 100% correct.
Christian.
Quoted 35 lines
Jerry
Quoted 32 lines
Regards,
Christian.Quoted 14 lines
Cc: [email protected]
Bugzilla: https://bugs.freedesktop.org/106418Reported-by [email protected]Signed-off-by Michel Dänzer <[email protected]>---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c| 4 ++++1 file changed, 4 insertions(+)drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0
-0drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c int amdgpu_vm_update_directories(struct · line 1119
Quoted 9 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +4
-0@@ -1111,6 +1111,10 @@ int amdgpu_vm_update_directories(struct amdgpu_device *adev, struct amdgpu_ring *ring; struct dma_fence *fence; + r = reservation_object_reserve_shared(root->tbo.resv); + if (r) + goto error; + ring = container_of(vm->entity.sched, struct amdgpu_ring, sched);_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): a95367bc-1181-649d-eca0-64ec0380c6c2-5C7GfCeVMHo@public.gmane.org
Quoted 26 lines · Christian König
Am 03.07.2018 um 03:59 schrieb Zhang, Jerry (Junwei):
Quoted 16 lines · Christian König
Quoted 15 lines
Am 25.06.2018 um 11:07 schrieb Michel Dänzer:
Quoted 7 lines
From: Michel Dänzer <[email protected]>
Without this, there could not be enough slots, which could trigger the
BUG_ONinreservation_object_add_shared_fence.v2:
* Jump to the error label instead of returning directly (Jerry Zhang)Well good catch, but NAK that is not the right solution to the issue.
amdgpu_vm_update_directories()is only called by two places, the first one isamdgpu_bo_vm_update_pte()and the secondamdgpu_gem_va_update_vm().In both cases the directory is reserved using
ttm_eu_reserve_buffers()with "entry->tv.shared= true;".Quoted 8 lines
Quoted 2 lines
What most likely needs to be done instead is to call
reservation_object_reserve_shared()after all the VM operations inamdgpu_bo_vm_update_pte()to make sure that we still have one additional slot for the command submission.In this case, it looks like
amdgpu_vm_bo_update_mapping()handling, insidettm_eu_reserve_buffers()as well.Is there any specific reason to call it at the end of
amdgpu_bo_vm_update_pte()?AFAIW, that has to be called right before where to add shared slot.
Quoted 7 lines
The
reservation_object_reserve_shared()function needs to be called before you make submissions to the hardware to make sure that you have a free slot after you made the submission.The problem is now that we always reserve one free slot during command submission, but for the page directory reservation object it can happen that this free slot is used by VM updates.
So the safest approach is to make sure that we have another free slot after the VM updates are done.
Michel's patch nearly did that, because we rarely have only one submission in
amdgpu_vm_update_directories(). But it isn't 100% correct.
Thanks to explain that, I think got your meaning that we should reserve slot for command submission after VM update.
if so, shall we add reservation_object_reserve_shared() for both of them, since any of place may run out of slot.
While the gap from me is I fail to find reserving slot during command submission, the root bo reservation seems to be used by VM update only(if not correct me). command submission is likely to use bo's reservation from bo list rather than root bo reservation.
BTW, reservation_object_reserve_shared() will extend the max num as 2 times if it's not enough. (default value is 4)
Jerry
Quoted 38 lines
Christian.
Quoted 34 lines
Jerry
Quoted 31 lines
Regards,
Christian.Quoted 14 lines
Cc: [email protected]
Bugzilla: https://bugs.freedesktop.org/106418Reported-by [email protected]Signed-off-by Michel Dänzer <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 4 ++++ 1 file changed, 4 insertions(+)drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0
-0drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c int amdgpu_vm_update_directories(struct amdgpu_device *adev, · line 1118
Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +4
-0@@ -1111,6 +1111,10 @@ int amdgpu_vm_update_directories(struct amdgpu_device *adev,struct amdgpu_ring *ring;struct dma_fence *fence; + r =reservation_object_reserve_shared (root->tbo.resv); +if (r) + goto error; + ring = container_of(vm->entity.sched, struct amdgpu_ring, sched);_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 4 lines · Jerry (Junwei)
BTW,
reservation_object_reserve_shared()will extend the max num as 2 times if it's not enough.
(default value is 4)
That's an implementation detail and depends on the circumstances. The only guarantee is that there's space for at least one fence after calling reservation_object_reserve_shared, until reservation_object_add_shared_fence is called.
Signature · 6 lines
Earthling Michel Dänzer | http://www.amd.com Libre software enthusiast | Mesa and X developer _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 78dcd991-2406-59f3-199a-721797a32c2c-otUistvHUpPR7s880joybQ@public.gmane.org
Quoted 10 lines · Michel Dänzer
Quoted 4 lines · Jerry (Junwei)
BTW,
reservation_object_reserve_shared()will extend the max num as 2 times if it's not enough.
(default value is 4)That's an implementation detail and depends on the circumstances. The only guarantee is that there's space for at least one fence after calling
reservation_object_reserve_shared, untilreservation_object_add_shared_fenceis called.
yes, it's a bit more detail discussion about that. Anyway, it should reserve at least one for shared reservation. Let's back to the upper level discussion.
Jerry
Quoted 2 lines
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Am 03.07.2018 um 10:12 schrieb Zhang, Jerry (Junwei):
Quoted 10 lines · Christian König
Quoted 9 lines
Am 03.07.2018 um 03:59 schrieb Zhang, Jerry (Junwei):
Quoted 2 lines · Christian König
Quoted 1 line
[SNIP]
The
reservation_object_reserve_shared()function needs to be called before you make submissions to the hardware to make sure that you have a free slot after you made the submission.The problem is now that we always reserve one free slot during command submission, but for the page directory reservation object it can happen that this free slot is used by VM updates.
So the safest approach is to make sure that we have another free slot after the VM updates are done.
Quoted 7 lines
Quoted 1 line
Michel's patch nearly did that, because we rarely have only one submission in
amdgpu_vm_update_directories(). But it isn't 100% correct.Thanks to explain that, I think got your meaning that we should reserve slot for command submission after VM update.
if so, shall we add
reservation_object_reserve_shared()for both of them, since any of place may run out of slot.While the gap from me is I fail to find reserving slot during command submission,
That's done by ttm_eu_reserve_buffers().
Quoted 1 line
the root bo reservation seems to be used by VM update only(if not correct me).
No, that is not correct. The root BO is also part of the list of BOs for the command submission used by ttm_eu_reserve_buffers() and ttm_eu_fence_buffer_objects().
All VM updates will use the same slot, so we just need one extra reservation_object_reserve_shared() after all the VM updates are done.
Regards,
Christian.
Quoted 6 lines
command submission is likely to use bo's reservation from bo list rather than root bo reservation.
BTW,
reservation_object_reserve_shared()will extend the max num as 2 times if it's not enough. (default value is 4)Jerry
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): f6dc6a5b-f6c4-5693-6bad-b2ce29ea54bc-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Quoted 19 lines · Christian König
Am 03.07.2018 um 10:12 schrieb Zhang, Jerry (Junwei):
Quoted 10 lines · Christian König
Quoted 9 lines
Am 03.07.2018 um 03:59 schrieb Zhang, Jerry (Junwei):
Quoted 2 lines · Christian König
Quoted 1 line
[SNIP]
The
reservation_object_reserve_shared()function needs to be called before you make submissions to the hardware to make sure that you have a free slot after you made the submission.The problem is now that we always reserve one free slot during command submission, but for the page directory reservation object it can happen that this free slot is used by VM updates.
So the safest approach is to make sure that we have another free slot after the VM updates are done.
Quoted 7 lines
Quoted 1 line
Michel's patch nearly did that, because we rarely have only one submission in
amdgpu_vm_update_directories(). But it isn't 100% correct.Thanks to explain that, I think got your meaning that we should reserve slot for command submission after VM update.
if so, shall we add
reservation_object_reserve_shared()for both of them, since any of place may run out of slot.While the gap from me is I fail to find reserving slot during command submission,
Quoted 7 lines
That's done by
ttm_eu_reserve_buffers().Quoted 1 line
the root bo reservation seems to be used by VM update only(if not correct me).
No, that is not correct. The root BO is also part of the list of BOs for the command submission used by
ttm_eu_reserve_buffers()andttm_eu_fence_buffer_objects().All VM updates will use the same slot, so we just need one extra
reservation_object_reserve_shared()after all the VM updates are done.
Yes, thanks to point it out.
the root bo indeed needs at least one extra shared slot for command submission.
Jerry
Quoted 11 lines
Regards,
Christian.Quoted 6 lines
command submission is likely to use bo's reservation from bo list rather than root bo reservation.
BTW,
reservation_object_reserve_shared()will extend the max num as 2 times if it's not enough. (default value is 4)Jerry
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx