drm/amdgpu: Attach exclusive fence to prime exported bo's. (v3)
Subsystems (2)
PATCH 1/1 drm/amdgpu: Attach exclusive fence to prime exported bo's. (v3)
External clients which import our bo's wait only for exclusive dmabuf-fences, not on shared ones, so attach fences on exported buffers as exclusive ones, if the buffers get imported by an external client.
See discussion in thread:
https://lists.freedesktop.org/archives/dri-devel/2016-October/122370.html
Tested on Intel iGPU + AMD Tonga dGPU as DRI3/Present Prime render offload, and with the Tonga standalone as primary gpu.
v2: Add a wait for all shared fences before prime export,
as suggested by Christian Koenig.
v3: - Mark buffer prime_exported in amdgpu_gem_prime_pin,
so we only use the exclusive fence when exporting a bo to external clients like a separate iGPU, but not when exporting/importing from/to ourselves as part of regular DRI3 fd passing.
- Propagate failure of reservation_object_wait_rcu back to caller.
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=95472
(v1) Tested-by: Mike Lothian <[email protected]>
--- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_prime.c | 14 ++++++++++++++ 3 files changed, 16 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +1 -0
@@ -459,6 +459,7 @@ struct amdgpu_bo {u64 metadata_flags;void *metadata;u32 metadata_size; +bool prime_exported;/* list of all virtual address to which this bo * is associated to */
drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.c +1 -1
@@ -132,7 +132,7 @@ static int amdgpu_bo_list_set(struct amdgpu_device *adev, entry->priority =min (info[i].bo_priority,AMDGPU_BO_LIST_MAX_PRIORITY ); entry->tv.bo = &entry->robj->tbo; - entry->tv.shared = true; + entry->tv.shared = !entry->robj->prime_exported;if (entry->robj->prefered_domains ==AMDGPU_GEM_DOMAIN_GDS ) gds_obj = entry->robj;
drivers/gpu/drm/amd/amdgpu/amdgpu_prime.c +14 -0
@@ -81,6 +81,20 @@ int amdgpu_gem_prime_pin(struct drm_gem_object *obj) {struct amdgpu_bo *bo =gem_to_amdgpu_bo (obj);int ret =0 ; +long lret; + +/* +* Wait for all shared fences to complete before we switch to future +* use of exclusive fence on this prime_exported bo. +*/ + lret =reservation_object_wait_timeout_rcu (bo->tbo.resv, true, false, +MAX_SCHEDULE_TIMEOUT ); +if (unlikely (lret <0 )) { +DRM_DEBUG_PRIME ("Fence wait failed: %li\n" , lret); +return lret; + } + + bo->prime_exported = true; ret =amdgpu_bo_reserve (bo, false);if (unlikely (ret !=0 ))
Signature · 6 lines
2.7.4 _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 22 lines · Mario Kleiner
External clients which import our bo's wait only for exclusive dmabuf-fences, not on shared ones, so attach fences on exported buffers as exclusive ones, if the buffers get imported by an external client.
See discussion in thread:
https://lists.freedesktop.org/archives/dri-devel/2016-October/122370.htmlTested on Intel iGPU + AMD Tonga dGPU as DRI3/Present Prime render offload, and with the Tonga standalone as primary gpu.
v2: Add a wait for all shared fences before prime export,
as suggested by Christian Koenig.v3: - Mark buffer
prime_exportedinamdgpu_gem_prime_pin,
so we only use the exclusive fence when exporting a bo to external clients like a separate iGPU, but not when exporting/importing from/to ourselves as part of regular DRI3 fd passing.
Quoted 5 lines
- Propagate failure of
reservation_object_wait_rcuback to caller.Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=95472
(v1) Tested-by: Mike Lothian <[email protected]>
FWIW, I think the (v1) is usually added at the end of the line, not at the beginning.
[...]
Quoted 12 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_prime.c +4
-0@@ -81,6 +81,20 @@ int amdgpu_gem_prime_pin(struct drm_gem_object *obj) {struct amdgpu_bo *bo =gem_to_amdgpu_bo (obj);int ret =0 ; +long lret; + +/* +* Wait for all shared fences to complete before we switch to future
drivers/gpu/drm/amd/amdgpu/amdgpu_prime.c int amdgpu_gem_prime_pin(struct drm_gem_object *obj) · line 97
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_prime.c +10
-0+* use of exclusive fence on this prime_exported bo. +*/ + lret =reservation_object_wait_timeout_rcu (bo->tbo.resv, true, false, +MAX_SCHEDULE_TIMEOUT ); +if (unlikely (lret <0 )) { +DRM_DEBUG_PRIME ("Fence wait failed: %li\n" , lret); +return lret; + } + + bo->prime_exported = true;
We should probably clear bo->prime_exported in amdgpu_gem_prime_unpin.
Also, I think we should set bo->prime_exported (prime_shared?) in amdgpu_gem_prime_import_sg_table as well, so that we'll also set exclusive fences on BOs imported from other GPUs.
Signature · 6 lines
Earthling Michel Dänzer | http://www.amd.com Libre software enthusiast | Mesa and X developer _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
A message that is not in the archive (sent off-list or not yet received): aec496f1-9a5d-d7c5-1f6f-bfce991b510e-otUistvHUpPR7s880joybQ@public.gmane.org
Am 07.11.2016 um 04:29 schrieb Michel Dänzer:
Quoted 57 lines · Mario Kleiner
Quoted 22 lines
External clients which import our bo's wait only for exclusive dmabuf-fences, not on shared ones, so attach fences on exported buffers as exclusive ones, if the buffers get imported by an external client.
See discussion in thread:
https://lists.freedesktop.org/archives/dri-devel/2016-October/122370.htmlTested on Intel iGPU + AMD Tonga dGPU as DRI3/Present Prime render offload, and with the Tonga standalone as primary gpu.
v2: Add a wait for all shared fences before prime export,
as suggested by Christian Koenig.v3: - Mark buffer
prime_exportedinamdgpu_gem_prime_pin,
so we only use the exclusive fence when exporting a bo to external clients like a separate iGPU, but not when exporting/importing from/to ourselves as part of regular DRI3 fd passing.Quoted 5 lines
- Propagate failure of
reservation_object_wait_rcuback to caller.Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=95472
(v1) Tested-by: Mike Lothian <[email protected]>FWIW, I think the (v1) is usually added at the end of the line, not at the beginning.
[...]
Quoted 12 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_prime.c +4
-0@@ -81,6 +81,20 @@ int amdgpu_gem_prime_pin(struct drm_gem_object *obj) {struct amdgpu_bo *bo =gem_to_amdgpu_bo (obj);int ret =0 ; +long lret; + +/* +* Wait for all shared fences to complete before we switch to future drivers/gpu/drm/amd/amdgpu/amdgpu_prime.c int amdgpu_gem_prime_pin(struct drm_gem_object *obj) · line 97
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_prime.c +10
-0+* use of exclusive fence on this prime_exported bo. +*/ + lret =reservation_object_wait_timeout_rcu (bo->tbo.resv, true, false, +MAX_SCHEDULE_TIMEOUT ); +if (unlikely (lret <0 )) { +DRM_DEBUG_PRIME ("Fence wait failed: %li\n" , lret); +return lret; + } + + bo->prime_exported = true;We should probably clear
bo->prime_exportedinamdgpu_gem_prime_unpin.
Quoted 3 lines
Also, I think we should set
bo->prime_exported(prime_shared?) inamdgpu_gem_prime_import_sg_tableas well, so that we'll also set exclusive fences on BOs imported from other GPUs.
Yes, really good idea.
Additional to that are we sure that amdgpu_gem_prime_pin() is only called once for each GEM object? Could in theory be that somebody exports a BO to another GFX device as well as a V4L device for example.
In this case we would need a counter, but I need to double check that as well.
In general I would protected the flag/counter by the BO being reserved to avoid any races. In other word move those lines a bit down after the amdgpu_bo_reserve().
Regards,
Christian.
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Quoted 32 lines · Christian König
Am 07.11.2016 um 04:29 schrieb Michel Dänzer:
Quoted 31 lines · Mario Kleiner
Quoted 23 lines
External clients which import our bo's wait only for exclusive dmabuf-fences, not on shared ones, so attach fences on exported buffers as exclusive ones, if the buffers get imported by an external client.
See discussion in thread:
https://lists.freedesktop.org/archives/dri-devel/2016-October/122370.htmlTested on Intel iGPU + AMD Tonga dGPU as DRI3/Present Prime render offload, and with the Tonga standalone as primary gpu.
v2: Add a wait for all shared fences before prime export,
as suggested by Christian Koenig.v3: - Mark buffer
prime_exportedinamdgpu_gem_prime_pin,
so we only use the exclusive fence when exporting a bo to external clients like a separate iGPU, but not when exporting/importing from/to ourselves as part of regular DRI3 fd passing.Quoted 5 lines
- Propagate failure of
reservation_object_wait_rcuback to caller.Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=95472
(v1) Tested-by: Mike Lothian <[email protected]>FWIW, I think the (v1) is usually added at the end of the line, not at the beginning.
Ok.
Quoted 30 lines
Quoted 30 lines
[...]
Quoted 18 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_prime.c +4
-0@@ -81,6 +81,20 @@ int amdgpu_gem_prime_pin(struct drm_gem_object *obj) {struct amdgpu_bo *bo =gem_to_amdgpu_bo (obj);int ret =0 ; +long lret; + +/* +* Wait for all shared fences to complete before we switch to future
drivers/gpu/drm/amd/amdgpu/amdgpu_prime.c +3
-0+* use of exclusive fence on this prime_exported bo. +*/ + lret = reservation_object_wait_timeout_rcu(bo->tbo.resv, true,false,
drivers/gpu/drm/amd/amdgpu/amdgpu_prime.c int amdgpu_gem_prime_pin(struct drm_gem_object *obj) · line 97
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_prime.c +7
-0+ MAX_SCHEDULE_TIMEOUT); +if (unlikely (lret <0 )) { +DRM_DEBUG_PRIME ("Fence wait failed: %li\n" , lret); +return lret; + } + + bo->prime_exported = true;We should probably clear
bo->prime_exportedinamdgpu_gem_prime_unpin.
Quoted 5 lines
Quoted 3 lines
Also, I think we should set
bo->prime_exported(prime_shared?) inamdgpu_gem_prime_import_sg_tableas well, so that we'll also set exclusive fences on BOs imported from other GPUs.Yes, really good idea.
Ok. I guess we don't need to wait for fences there before setting the flag/counter, as we can't have done anything yet with the bo just created from the imported sg_table?
Quoted 7 lines
Additional to that are we sure that
amdgpu_gem_prime_pin()is only called once for each GEM object? Could in theory be that somebody exports a BO to another GFX device as well as a V4L device for example.In this case we would need a counter, but I need to double check that as well.
If we want to clear the prime_exported flag in amdgpu_gem_prime_unpin() as an optimization if all clients detach from the buffer, then we need a prime_shared_counter instead of a prime_exported flag afaics. The dmabuf api allows multiple clients to import and attach to an exported dmabuf, each one calling dma_buf_attach which will call amdgpu_gem_prime_pin.
Quoted 3 lines
In general I would protected the flag/counter by the BO being reserved to avoid any races. In other word move those lines a bit down after the
amdgpu_bo_reserve().
Ok.
-mario
Quoted 3 lines
Regards,
Christian.
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
A message that is not in the archive (sent off-list or not yet received): 2969084b-e63c-a852-c17c-fe7ac9110153-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Am 07.11.2016 um 20:20 schrieb Mario Kleiner:
Quoted 9 lines · Christian König
Quoted 6 lines
Am 07.11.2016 um 04:29 schrieb Michel Dänzer:
Also, I think we should setbo->prime_exported(prime_shared?) inamdgpu_gem_prime_import_sg_tableas well, so that we'll also set exclusive fences on BOs imported from other GPUs.
Yes, really good idea.Ok. I guess we don't need to wait for fences there before setting the flag/counter, as we can't have done anything yet with the bo just created from the imported
sg_table?
Correct, waiting at this point shouldn't be necessary.
Regards,
Christian.
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx