drm/msm: Add MSM_SUBMIT_BO_NO_IMPLICIT
Subsystems (5)
PATCH 1/1 drm/msm: Add MSM_SUBMIT_BO_NO_IMPLICIT
From: Rob Clark <[email protected]>
In cases where implicit sync is used, it is still useful (for things like sub-allocation, etc) to allow userspace to opt-out of implicit sync on per-BO basis.
--- drivers/gpu/drm/msm/msm_drv.c | 3 ++- drivers/gpu/drm/msm/msm_gem_submit.c | 11 +++++++++++ include/uapi/drm/msm_drm.h | 4 +++- 3 files changed, 16 insertions(+), 2 deletions(-)
drivers/gpu/drm/msm/msm_drv.c +2 -1
@@ -45,9 +45,10 @@* - 1.7.0 - Add MSM_PARAM_SUSPENDS to access suspend count * - 1.8.0 - Add MSM_BO_CACHED_COHERENT for supported GPUs (a6xx) * - 1.9.0 - Add MSM_SUBMIT_FENCE_SN_IN +* - 1.10.0 - Add MSM_SUBMIT_BO_NO_IMPLICIT */ #define MSM_VERSION_MAJOR 1 -#define MSM_VERSION_MINOR 9 +#define MSM_VERSION_MINOR 10#define MSM_VERSION_PATCHLEVEL 0static const struct drm_mode_config_funcs mode_config_funcs = {
drivers/gpu/drm/msm/msm_gem_submit.c +11 -0
@@ -334,9 +334,20 @@ static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit)if (ret)return ret; +/* If userspace has determined that explicit fencing is +* used, it can disable implicit sync on the entire +* submit: +*/ if (no_implicit)continue ; +/* Otherwise userspace can ask for implicit sync to be +* disabled on specific buffers. This is useful for internal +* usermode driver managed buffers, suballocation, etc. +*/ +if (submit->bos[i].flags &MSM_SUBMIT_BO_NO_IMPLICIT ) +continue ; + ret =drm_sched_job_add_implicit_dependencies (&submit->base, obj, write);
include/uapi/drm/msm_drm.h +3 -1
@@ -222,10 +222,12 @@ struct drm_msm_gem_submit_cmd {#define MSM_SUBMIT_BO_READ 0x0001#define MSM_SUBMIT_BO_WRITE 0x0002#define MSM_SUBMIT_BO_DUMP 0x0004 +#define MSM_SUBMIT_BO_NO_IMPLICIT 0x0008#define MSM_SUBMIT_BO_FLAGS (MSM_SUBMIT_BO_READ | \ MSM_SUBMIT_BO_WRITE | \ - MSM_SUBMIT_BO_DUMP) + MSM_SUBMIT_BO_DUMP | \ + MSM_SUBMIT_BO_NO_IMPLICIT)struct drm_msm_gem_submit_bo {__u32 flags;/* in, mask of MSM_SUBMIT_BO_x */
Signature · 1 line
2.38.1
Hi Rob,
Am Dienstag, dem 06.12.2022 um 11:21 -0800 schrieb Rob Clark:
Quoted 6 lines
From: Rob Clark <[email protected]>
In cases where implicit sync is used, it is still useful (for things like sub-allocation, etc) to allow userspace to opt-out of implicit sync on per-BO basis.
Out of curiosity and because I have been thinking about something like that for etnaviv for while: do you only use this for immutable buffers or do you have some kind of userspace fencing in place for the suballocated buffers?
Regards,
Lucas
Quoted 66 lines
Signed-off-by Rob Clark <[email protected]>--- drivers/gpu/drm/msm/msm_drv.c | 3 ++- drivers/gpu/drm/msm/msm_gem_submit.c | 11 +++++++++++ include/uapi/drm/msm_drm.h | 4 +++- 3 files changed, 16 insertions(+), 2 deletions(-)drivers/gpu/drm/msm/msm_drv.c +2
-1@@ -45,9 +45,10 @@* - 1.7.0 - Add MSM_PARAM_SUSPENDS to access suspend count * - 1.8.0 - Add MSM_BO_CACHED_COHERENT for supported GPUs (a6xx) * - 1.9.0 - Add MSM_SUBMIT_FENCE_SN_IN +* - 1.10.0 - Add MSM_SUBMIT_BO_NO_IMPLICIT */ #define MSM_VERSION_MAJOR 1 -#define MSM_VERSION_MINOR 9 +#define MSM_VERSION_MINOR 10#define MSM_VERSION_PATCHLEVEL 0static const struct drm_mode_config_funcs mode_config_funcs = {drivers/gpu/drm/msm/msm_gem_submit.c +11
-0@@ -334,9 +334,20 @@ static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit)if (ret)return ret; +/* If userspace has determined that explicit fencing is +* used, it can disable implicit sync on the entire +* submit: +*/ if (no_implicit)continue ; +/* Otherwise userspace can ask for implicit sync to be +* disabled on specific buffers. This is useful for internal +* usermode driver managed buffers, suballocation, etc. +*/ +if (submit->bos[i].flags &MSM_SUBMIT_BO_NO_IMPLICIT ) +continue ; + ret =drm_sched_job_add_implicit_dependencies (&submit->base, obj, write);include/uapi/drm/msm_drm.h +3
-1@@ -222,10 +222,12 @@ struct drm_msm_gem_submit_cmd {#define MSM_SUBMIT_BO_READ 0x0001#define MSM_SUBMIT_BO_WRITE 0x0002#define MSM_SUBMIT_BO_DUMP 0x0004 +#define MSM_SUBMIT_BO_NO_IMPLICIT 0x0008#define MSM_SUBMIT_BO_FLAGS (MSM_SUBMIT_BO_READ | \ MSM_SUBMIT_BO_WRITE | \ - MSM_SUBMIT_BO_DUMP) + MSM_SUBMIT_BO_DUMP | \ + MSM_SUBMIT_BO_NO_IMPLICIT)struct drm_msm_gem_submit_bo {__u32 flags;/* in, mask of MSM_SUBMIT_BO_x */
Quoted 6 lines · Lucas Stach
Hi Rob,
Am Dienstag, dem 06.12.2022 um 11:21 -0800 schrieb Rob Clark:
Quoted 2 lines
From: Rob Clark <[email protected]>
Quoted 8 lines
Quoted 4 lines
In cases where implicit sync is used, it is still useful (for things like sub-allocation, etc) to allow userspace to opt-out of implicit sync on per-BO basis.
Out of curiosity and because I have been thinking about something like that for etnaviv for while: do you only use this for immutable buffers or do you have some kind of userspace fencing in place for the suballocated buffers?
yup, userspace fences.. which is a thing you'd really want for doing suballocation
BR,
-R
Quoted 72 lines
Regards,
LucasQuoted 66 lines
Signed-off-by Rob Clark <[email protected]>--- drivers/gpu/drm/msm/msm_drv.c | 3 ++- drivers/gpu/drm/msm/msm_gem_submit.c | 11 +++++++++++ include/uapi/drm/msm_drm.h | 4 +++- 3 files changed, 16 insertions(+), 2 deletions(-)drivers/gpu/drm/msm/msm_drv.c +2
-1@@ -45,9 +45,10 @@* - 1.7.0 - Add MSM_PARAM_SUSPENDS to access suspend count * - 1.8.0 - Add MSM_BO_CACHED_COHERENT for supported GPUs (a6xx) * - 1.9.0 - Add MSM_SUBMIT_FENCE_SN_IN +* - 1.10.0 - Add MSM_SUBMIT_BO_NO_IMPLICIT */ #define MSM_VERSION_MAJOR 1 -#define MSM_VERSION_MINOR 9 +#define MSM_VERSION_MINOR 10#define MSM_VERSION_PATCHLEVEL 0static const struct drm_mode_config_funcs mode_config_funcs = {drivers/gpu/drm/msm/msm_gem_submit.c +11
-0@@ -334,9 +334,20 @@ static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit)if (ret)return ret; +/* If userspace has determined that explicit fencing is +* used, it can disable implicit sync on the entire +* submit: +*/ if (no_implicit)continue ; +/* Otherwise userspace can ask for implicit sync to be +* disabled on specific buffers. This is useful for internal +* usermode driver managed buffers, suballocation, etc. +*/ +if (submit->bos[i].flags &MSM_SUBMIT_BO_NO_IMPLICIT ) +continue ; + ret =drm_sched_job_add_implicit_dependencies (&submit->base, obj, write);include/uapi/drm/msm_drm.h +3
-1@@ -222,10 +222,12 @@ struct drm_msm_gem_submit_cmd {#define MSM_SUBMIT_BO_READ 0x0001#define MSM_SUBMIT_BO_WRITE 0x0002#define MSM_SUBMIT_BO_DUMP 0x0004 +#define MSM_SUBMIT_BO_NO_IMPLICIT 0x0008#define MSM_SUBMIT_BO_FLAGS (MSM_SUBMIT_BO_READ | \ MSM_SUBMIT_BO_WRITE | \ - MSM_SUBMIT_BO_DUMP) + MSM_SUBMIT_BO_DUMP | \ + MSM_SUBMIT_BO_NO_IMPLICIT)struct drm_msm_gem_submit_bo {__u32 flags;/* in, mask of MSM_SUBMIT_BO_x */
Quoted 42 lines · Rob Clark
From: Rob Clark <[email protected]>
In cases where implicit sync is used, it is still useful (for things like sub-allocation, etc) to allow userspace to opt-out of implicit sync on per-BO basis.
Signed-off-by Rob Clark <[email protected]>--- drivers/gpu/drm/msm/msm_drv.c | 3 ++- drivers/gpu/drm/msm/msm_gem_submit.c | 11 +++++++++++ include/uapi/drm/msm_drm.h | 4 +++- 3 files changed, 16 insertions(+), 2 deletions(-)drivers/gpu/drm/msm/msm_drv.c +2
-1@@ -45,9 +45,10 @@* - 1.7.0 - Add MSM_PARAM_SUSPENDS to access suspend count * - 1.8.0 - Add MSM_BO_CACHED_COHERENT for supported GPUs (a6xx) * - 1.9.0 - Add MSM_SUBMIT_FENCE_SN_IN +* - 1.10.0 - Add MSM_SUBMIT_BO_NO_IMPLICIT */ #define MSM_VERSION_MAJOR 1 -#define MSM_VERSION_MINOR 9 +#define MSM_VERSION_MINOR 10#define MSM_VERSION_PATCHLEVEL 0static const struct drm_mode_config_funcs mode_config_funcs = {drivers/gpu/drm/msm/msm_gem_submit.c +4
-0@@ -334,9 +334,20 @@ static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit)if (ret)return ret; +/* If userspace has determined that explicit fencing is +* used, it can disable implicit sync on the entire +* submit: +*/ if (no_implicit)
drivers/gpu/drm/msm/msm_gem_submit.c static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit) · line 351
Quoted 10 lines
drivers/gpu/drm/msm/msm_gem_submit.c +7
-0continue ; +/* Otherwise userspace can ask for implicit sync to be +* disabled on specific buffers. This is useful for internal +* usermode driver managed buffers, suballocation, etc. +*/ +if (submit->bos[i].flags &MSM_SUBMIT_BO_NO_IMPLICIT ) +continue ; + ret = drm_sched_job_add_implicit_dependencies(&submit->base,
Won't this break shrinkers and fun stuff like that? It's why we added the new USAGE_OTHER fence slot at least, and also why I wonder whether we shouldn't push this into the helper to make the right call. Every driver kinda needs the same wheel.
-Daniel
Quoted 23 lines
drivers/gpu/drm/msm/msm_gem_submit.c +0
-0obj, write);include/uapi/drm/msm_drm.h +3
-1@@ -222,10 +222,12 @@ struct drm_msm_gem_submit_cmd {#define MSM_SUBMIT_BO_READ 0x0001#define MSM_SUBMIT_BO_WRITE 0x0002#define MSM_SUBMIT_BO_DUMP 0x0004 +#define MSM_SUBMIT_BO_NO_IMPLICIT 0x0008#define MSM_SUBMIT_BO_FLAGS (MSM_SUBMIT_BO_READ | \ MSM_SUBMIT_BO_WRITE | \ - MSM_SUBMIT_BO_DUMP) + MSM_SUBMIT_BO_DUMP | \ + MSM_SUBMIT_BO_NO_IMPLICIT)struct drm_msm_gem_submit_bo {__u32 flags;/* in, mask of MSM_SUBMIT_BO_x */ Signature · 1 line
2.38.1
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Quoted 46 lines · Daniel Vetter
Quoted 44 lines · Rob Clark
From: Rob Clark <[email protected]>
In cases where implicit sync is used, it is still useful (for things like sub-allocation, etc) to allow userspace to opt-out of implicit sync on per-BO basis.
Signed-off-by Rob Clark <[email protected]>--- drivers/gpu/drm/msm/msm_drv.c | 3 ++- drivers/gpu/drm/msm/msm_gem_submit.c | 11 +++++++++++ include/uapi/drm/msm_drm.h | 4 +++- 3 files changed, 16 insertions(+), 2 deletions(-)drivers/gpu/drm/msm/msm_drv.c +2
-1@@ -45,9 +45,10 @@* - 1.7.0 - Add MSM_PARAM_SUSPENDS to access suspend count * - 1.8.0 - Add MSM_BO_CACHED_COHERENT for supported GPUs (a6xx) * - 1.9.0 - Add MSM_SUBMIT_FENCE_SN_IN +* - 1.10.0 - Add MSM_SUBMIT_BO_NO_IMPLICIT */ #define MSM_VERSION_MAJOR 1 -#define MSM_VERSION_MINOR 9 +#define MSM_VERSION_MINOR 10#define MSM_VERSION_PATCHLEVEL 0static const struct drm_mode_config_funcs mode_config_funcs = {drivers/gpu/drm/msm/msm_gem_submit.c +4
-0@@ -334,9 +334,20 @@ static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit)if (ret)return ret; +/* If userspace has determined that explicit fencing is +* used, it can disable implicit sync on the entire +* submit: +*/ if (no_implicit)continue ;
Quoted 11 lines
drivers/gpu/drm/msm/msm_gem_submit.c static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit) · line 351
Quoted 8 lines
drivers/gpu/drm/msm/msm_gem_submit.c +7
-0+/* Otherwise userspace can ask for implicit sync to be +* disabled on specific buffers. This is useful for internal +* usermode driver managed buffers, suballocation, etc. +*/ +if (submit->bos[i].flags &MSM_SUBMIT_BO_NO_IMPLICIT ) +continue ; + ret = drm_sched_job_add_implicit_dependencies(&submit->base,Won't this break shrinkers and fun stuff like that? It's why we added the new
USAGE_OTHERfence slot at least, and also why I wonder whether we
Only if the entire explicit sync path was busted.. My daily driver for email/docs/meet/chat/corpstuff is a 4G device and CrOS is all explicit sync.. I would have found out rapidly and dramatically if it was busted :-P
But seriously, this doesn't change what fences we attach to buffers, only what the sched job waits on
Quoted 2 lines
shouldn't push this into the helper to make the right call. Every driver kinda needs the same wheel.
We kinda already have moved everything we can (with the current driver-specific-uabi model) to helpers, what is left is driver specific ioctl parsing. We absolutely should take a step back and re-evaluate this before anyone else adds a new submit/execbuf ioctl. For example, the driver specific ioctl could just have a pointer to a drm_gem_submit_bo_table type structure, and then we could move the whole thing to a helper. Short of breaking the submit ioctl up (which a uring type uabi would let us do), I think the next best thing is to split out common cross-driver structs for common parts of submit/execbuf.
BR,
-R
Quoted 30 lines
-Daniel
Quoted 21 lines
drivers/gpu/drm/msm/msm_gem_submit.c +0
-0obj, write);include/uapi/drm/msm_drm.h +3
-2@@ -222,10 +222,12 @@ struct drm_msm_gem_submit_cmd {#define MSM_SUBMIT_BO_READ 0x0001#define MSM_SUBMIT_BO_WRITE 0x0002#define MSM_SUBMIT_BO_DUMP 0x0004 +#define MSM_SUBMIT_BO_NO_IMPLICIT 0x0008#define MSM_SUBMIT_BO_FLAGS (MSM_SUBMIT_BO_READ | \ MSM_SUBMIT_BO_WRITE | \ - MSM_SUBMIT_BO_DUMP) + MSM_SUBMIT_BO_DUMP | \ + MSM_SUBMIT_BO_NO_IMPLICIT)struct drm_msm_gem_submit_bo {__u32 flags;/* in, mask of MSM_SUBMIT_BO_x */ --Quoted 2 lines
2.38.1
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
Quoted 60 lines · Rob Clark
Quoted 46 lines · Daniel Vetter
Quoted 44 lines · Rob Clark
From: Rob Clark <[email protected]>
In cases where implicit sync is used, it is still useful (for things like sub-allocation, etc) to allow userspace to opt-out of implicit sync on per-BO basis.
Signed-off-by Rob Clark <[email protected]>--- drivers/gpu/drm/msm/msm_drv.c | 3 ++- drivers/gpu/drm/msm/msm_gem_submit.c | 11 +++++++++++ include/uapi/drm/msm_drm.h | 4 +++- 3 files changed, 16 insertions(+), 2 deletions(-)drivers/gpu/drm/msm/msm_drv.c +2
-1@@ -45,9 +45,10 @@* - 1.7.0 - Add MSM_PARAM_SUSPENDS to access suspend count * - 1.8.0 - Add MSM_BO_CACHED_COHERENT for supported GPUs (a6xx) * - 1.9.0 - Add MSM_SUBMIT_FENCE_SN_IN +* - 1.10.0 - Add MSM_SUBMIT_BO_NO_IMPLICIT */ #define MSM_VERSION_MAJOR 1 -#define MSM_VERSION_MINOR 9 +#define MSM_VERSION_MINOR 10#define MSM_VERSION_PATCHLEVEL 0static const struct drm_mode_config_funcs mode_config_funcs = {drivers/gpu/drm/msm/msm_gem_submit.c +4
-0@@ -334,9 +334,20 @@ static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit)if (ret)return ret; +/* If userspace has determined that explicit fencing is +* used, it can disable implicit sync on the entire +* submit: +*/ if (no_implicit)continue ;Quoted 11 lines
drivers/gpu/drm/msm/msm_gem_submit.c static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit) · line 351
Quoted 8 lines
drivers/gpu/drm/msm/msm_gem_submit.c +7
-0+/* Otherwise userspace can ask for implicit sync to be +* disabled on specific buffers. This is useful for internal +* usermode driver managed buffers, suballocation, etc. +*/ +if (submit->bos[i].flags &MSM_SUBMIT_BO_NO_IMPLICIT ) +continue ; + ret = drm_sched_job_add_implicit_dependencies(&submit->base,Won't this break shrinkers and fun stuff like that? It's why we added the new
USAGE_OTHERfence slot at least, and also why I wonder whether we
Quoted 7 lines
Only if the entire explicit sync path was busted.. My daily driver for email/docs/meet/chat/corpstuff is a 4G device and CrOS is all explicit sync.. I would have found out rapidly and dramatically if it was busted :-P
But seriously, this doesn't change what fences we attach to buffers, only what the sched job waits on
Oh I'm a silly one :-)
Quoted 13 lines
Quoted 2 lines
shouldn't push this into the helper to make the right call. Every driver kinda needs the same wheel.
We kinda already have moved everything we can (with the current driver-specific-uabi model) to helpers, what is left is driver specific ioctl parsing. We absolutely should take a step back and re-evaluate this before anyone else adds a new submit/execbuf ioctl. For example, the driver specific ioctl could just have a pointer to a
drm_gem_submit_bo_tabletype structure, and then we could move the whole thing to a helper. Short of breaking the submit ioctl up (which a uring type uabi would let us do), I think the next best thing is to split out common cross-driver structs for common parts of submit/execbuf.
Yeah I thought this was the "attach fence to dma_resv" side of things. This here is I think fine as a pattern, unless we build some kind of outright submit ioctl helpers that take care of everything. -Daniel
Quoted 34 lines
BR,
-RQuoted 30 lines
-Daniel
Quoted 21 lines
drivers/gpu/drm/msm/msm_gem_submit.c +0
-0obj, write);include/uapi/drm/msm_drm.h +3
-2@@ -222,10 +222,12 @@ struct drm_msm_gem_submit_cmd {#define MSM_SUBMIT_BO_READ 0x0001#define MSM_SUBMIT_BO_WRITE 0x0002#define MSM_SUBMIT_BO_DUMP 0x0004 +#define MSM_SUBMIT_BO_NO_IMPLICIT 0x0008#define MSM_SUBMIT_BO_FLAGS (MSM_SUBMIT_BO_READ | \ MSM_SUBMIT_BO_WRITE | \ - MSM_SUBMIT_BO_DUMP) + MSM_SUBMIT_BO_DUMP | \ + MSM_SUBMIT_BO_NO_IMPLICIT)struct drm_msm_gem_submit_bo {__u32 flags;/* in, mask of MSM_SUBMIT_BO_x */ --Quoted 2 lines
2.38.1
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch