drm/msm: Add the MSM_WAIT_IOVA ioctl
Subsystems (5)
cover drm/msm: Add the MSM_WAIT_IOVA ioctl
This patch set implements the MSM_WAIT_IOVA ioctl which lets userspace sleep until the value at a given iova reaches a certain condition. This is needed in turnip to implement the VK_QUERY_RESULT_WAIT_BIT flag for vkGetQueryPoolResults.
First, we add a GPU-wide wait queue that is signaled on all IRQs. We can then wait on this wait queue inside MSM_WAIT_IOVA until the condition is met.
The corresponding merge request in mesa can be found at:
https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3279
Brian Ho (2):
drm/msm: Add a GPU-wide wait queue
drm/msm: Add MSM_WAIT_IOVA ioctl
drivers/gpu/drm/msm/msm_drv.c | 63 +++++++++++++++++++++++++++++++++-- drivers/gpu/drm/msm/msm_gpu.c | 4 +++ drivers/gpu/drm/msm/msm_gpu.h | 3 ++ include/uapi/drm/msm_drm.h | 13 ++++++++ 4 files changed, 81 insertions(+), 2 deletions(-)
Signature · 1 line
2.25.0.rc1.283.g88dfdc4193-goog
PATCH 1/2 drm/msm: Add a GPU-wide wait queue
This wait queue is signaled on all IRQs for a given GPU and will be used as part of the new MSM_WAIT_IOVA ioctl so userspace can sleep until the value at a given iova reaches a certain condition.
--- drivers/gpu/drm/msm/msm_gpu.c | 4 ++++ drivers/gpu/drm/msm/msm_gpu.h | 3 +++ 2 files changed, 7 insertions(+)
drivers/gpu/drm/msm/msm_gpu.c +4 -0
@@ -779,6 +779,8 @@ void msm_gpu_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit,static irqreturn_t irq_handler (int irq,void *data) {struct msm_gpu *gpu = data; +wake_up_all (&gpu->event); +return gpu->funcs->irq (gpu); }
@@ -871,6 +873,8 @@ int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev,spin_lock_init (&gpu->perf_lock); +init_waitqueue_head (&gpu->event); +/* Map registers: */ gpu->mmio =msm_ioremap (pdev, config->ioname, name);
drivers/gpu/drm/msm/msm_gpu.h +3 -0
@@ -104,6 +104,9 @@ struct msm_gpu {struct msm_gem_address_space *aspace; +/* GPU-wide wait queue that is signaled on all IRQs */ +wait_queue_head_t event; +/* Power Control: */ struct regulator *gpu_reg, *gpu_cx;struct clk_bulk_data *grp_clks;
Signature · 1 line
2.25.0.rc1.283.g88dfdc4193-goog
Re: [Freedreno] [PATCH 1/2] drm/msm: Add a GPU-wide wait queue
Quoted 15 lines · Brian Ho
This wait queue is signaled on all IRQs for a given GPU and will be used as part of the new
MSM_WAIT_IOVAioctl so userspace can sleep until the value at a given iova reaches a certain condition.Signed-off-by Brian Ho <[email protected]>--- drivers/gpu/drm/msm/msm_gpu.c | 4 ++++ drivers/gpu/drm/msm/msm_gpu.h | 3 +++ 2 files changed, 7 insertions(+)drivers/gpu/drm/msm/msm_gpu.c +0
-0
drivers/gpu/drm/msm/msm_gpu.c void msm_gpu_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit, · line 783
Quoted 5 lines
drivers/gpu/drm/msm/msm_gpu.c +2
-0@@ -779,6 +779,8 @@ void msm_gpu_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit,static irqreturn_t irq_handler (int irq,void *data) {struct msm_gpu *gpu = data; +wake_up_all (&gpu->event); +
I suppose it is intentional to have this happen on *all* interrupts because you might be using the CP interrupts for fun and profit and you don't want to plumb in callbacks? I suppose it is okay to do this for all interrupts (including errors) but if we're spending a lot of time here we might want to only trigger on certain IRQs.
Quoted 33 lines
drivers/gpu/drm/msm/msm_gpu.c +2
-0return gpu->funcs->irq (gpu); }@@ -871,6 +873,8 @@ int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev,spin_lock_init (&gpu->perf_lock); +init_waitqueue_head (&gpu->event); +/* Map registers: */ gpu->mmio =msm_ioremap (pdev, config->ioname, name);drivers/gpu/drm/msm/msm_gpu.h +3
-0@@ -104,6 +104,9 @@ struct msm_gpu {struct msm_gem_address_space *aspace; +/* GPU-wide wait queue that is signaled on all IRQs */ +wait_queue_head_t event; +/* Power Control: */ struct regulator *gpu_reg, *gpu_cx;struct clk_bulk_data *grp_clks;Signature · 6 lines
2.25.0.rc1.283.g88dfdc4193-goog _______________________________________________ Freedreno mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/freedreno
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
Re: [Freedreno] [PATCH 1/2] drm/msm: Add a GPU-wide wait queue
Quoted 12 lines · Jordan Crouse
Quoted 10 lines · Brian Ho
This wait queue is signaled on all IRQs for a given GPU and will be used as part of the new
MSM_WAIT_IOVAioctl so userspace can sleep until the value at a given iova reaches a certain condition.Signed-off-by Brian Ho <[email protected]>--- drivers/gpu/drm/msm/msm_gpu.c | 4 ++++ drivers/gpu/drm/msm/msm_gpu.h | 3 +++ 2 files changed, 7 insertions(+)
Quoted 16 lines
drivers/gpu/drm/msm/msm_gpu.c void msm_gpu_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit, · line 783
Quoted 10 lines
drivers/gpu/drm/msm/msm_gpu.c +2
-0@@ -779,6 +779,8 @@ void msm_gpu_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit,static irqreturn_t irq_handler (int irq,void *data) {struct msm_gpu *gpu = data; +wake_up_all (&gpu->event); +I suppose it is intentional to have this happen on *all* interrupts because you might be using the CP interrupts for fun and profit and you don't want to plumb in callbacks? I suppose it is okay to do this for all interrupts (including errors) but if we're spending a lot of time here we might want to only trigger on certain IRQs.
Was just talking to Kristian about GPU hangs.. and I suspect we might want the ioctl to return an error if there is a gpu reset (so that userspace can use the robustness uapi to test if the gpu reset was something it cares about, etc)
Which is as good as a reason as I can think of the wake_up_all() on all irqs..
BR,
-R
Quoted 38 lines
Quoted 30 lines
drivers/gpu/drm/msm/msm_gpu.c +2
-0return gpu->funcs->irq (gpu); }@@ -871,6 +873,8 @@ int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev,spin_lock_init (&gpu->perf_lock); +init_waitqueue_head (&gpu->event); +/* Map registers: */ gpu->mmio =msm_ioremap (pdev, config->ioname, name);drivers/gpu/drm/msm/msm_gpu.h +3
-1@@ -104,6 +104,9 @@ struct msm_gpu {struct msm_gem_address_space *aspace; +/* GPU-wide wait queue that is signaled on all IRQs */ +wait_queue_head_t event; +/* Power Control: */ struct regulator *gpu_reg, *gpu_cx;struct clk_bulk_data *grp_clks; --2.25.0.rc1.283.g88dfdc4193-goog
Quoted 4 lines
_______________________________________________
Freedreno mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/freedreno--
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
PATCH 2/2 drm/msm: Add MSM_WAIT_IOVA ioctl
In reply to Brian HoImplements an ioctl to wait until a value at a given iova is greater than or equal to a supplied value.
This will initially be used by turnip (open-source Vulkan driver for QC in mesa) for occlusion queries where the userspace driver can block on a query becoming available before continuing via vkGetQueryPoolResults.
--- drivers/gpu/drm/msm/msm_drv.c | 63 +++++++++++++++++++++++++++++++++-- include/uapi/drm/msm_drm.h | 13 ++++++++ 2 files changed, 74 insertions(+), 2 deletions(-)
drivers/gpu/drm/msm/msm_drv.c +61 -2
@@ -36,10 +36,11 @@* MSM_GEM_INFO ioctl. * - 1.4.0 - softpin, MSM_RELOC_BO_DUMP, and GEM_INFO support to set/get * GEM object's debug name -* - 1.5.0 - Add SUBMITQUERY_QUERY ioctl +* - 1.5.0 - Add SUBMITQUEUE_QUERY ioctl +* - 1.6.0 - Add WAIT_IOVA ioctl */ #define MSM_VERSION_MAJOR 1 -#define MSM_VERSION_MINOR 5 +#define MSM_VERSION_MINOR 6#define MSM_VERSION_PATCHLEVEL 0static const struct drm_mode_config_funcs mode_config_funcs = {
@@ -952,6 +953,63 @@ static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data,return msm_submitqueue_remove (file->driver_priv, id); } +static int msm_ioctl_wait_iova (struct drm_device *dev,void *data, +struct drm_file *file) +{ +struct msm_drm_private *priv = dev->dev_private; +struct drm_gem_object *obj; +struct drm_msm_wait_iova *args = data; +ktime_t timeout =to_ktime (args->timeout); +unsigned long remaining_jiffies =timeout_to_jiffies (&timeout); +struct msm_gpu *gpu = priv->gpu; +void *base_vaddr; +uint64_t *vaddr; +int ret; + +if (args->pad) +return -EINVAL ; + +if (!gpu) +return 0 ; + + obj =drm_gem_object_lookup (file, args->handle); +if (!obj) +return -ENOENT ; + + base_vaddr =msm_gem_get_vaddr (obj); +if (IS_ERR (base_vaddr)) { + ret =PTR_ERR (base_vaddr); + goto err_put_gem_object; + } +if (args->offset +sizeof (*vaddr) > obj->size) { + ret = -EINVAL ; + goto err_put_vaddr; + } + + vaddr = base_vaddr + args->offset; + +/* Assumes WC mapping */ + ret =wait_event_interruptible_timeout ( + gpu->event, *vaddr >= args->value, remaining_jiffies); + +if (ret ==0 ) { + ret = -ETIMEDOUT ; + goto err_put_vaddr; + }else if (ret == -ERESTARTSYS ) { + goto err_put_vaddr; + } + +msm_gem_put_vaddr (obj); +drm_gem_object_put_unlocked (obj); +return 0 ; + +err_put_vaddr: +msm_gem_put_vaddr (obj); +err_put_gem_object: +drm_gem_object_put_unlocked (obj); +return ret; +} +static const struct drm_ioctl_desc msm_ioctls[] = {DRM_IOCTL_DEF_DRV (MSM_GET_PARAM , msm_ioctl_get_param,DRM_RENDER_ALLOW ),DRM_IOCTL_DEF_DRV (MSM_GEM_NEW , msm_ioctl_gem_new,DRM_RENDER_ALLOW ),
@@ -964,6 +1022,7 @@ static const struct drm_ioctl_desc msm_ioctls[] = {DRM_IOCTL_DEF_DRV (MSM_SUBMITQUEUE_NEW , msm_ioctl_submitqueue_new,DRM_RENDER_ALLOW ),DRM_IOCTL_DEF_DRV (MSM_SUBMITQUEUE_CLOSE , msm_ioctl_submitqueue_close,DRM_RENDER_ALLOW ),DRM_IOCTL_DEF_DRV (MSM_SUBMITQUEUE_QUERY , msm_ioctl_submitqueue_query,DRM_RENDER_ALLOW ), + DRM_IOCTL_DEF_DRV(MSM_WAIT_IOVA, msm_ioctl_wait_iova, DRM_RENDER_ALLOW), }; static const struct vm_operations_struct vm_ops = {
include/uapi/drm/msm_drm.h +13 -0
@@ -298,6 +298,17 @@ struct drm_msm_submitqueue_query {__u32 pad; }; +/* This ioctl blocks until the u64 value at bo + offset is greater than or +* equal to the reference value. +*/ +struct drm_msm_wait_iova { +__u32 handle;/* in, GEM handle */ +__u32 pad; +struct drm_msm_timespec timeout;/* in */ +__u64 offset;/* offset into bo */ +__u64 value;/* reference value */ +}; +#define DRM_MSM_GET_PARAM 0x00/* placeholder: #define DRM_MSM_SET_PARAM 0x01
@@ -315,6 +326,7 @@ struct drm_msm_submitqueue_query {#define DRM_MSM_SUBMITQUEUE_NEW 0x0A#define DRM_MSM_SUBMITQUEUE_CLOSE 0x0B#define DRM_MSM_SUBMITQUEUE_QUERY 0x0C +#define DRM_MSM_WAIT_IOVA 0x0D#define DRM_IOCTL_MSM_GET_PARAM DRM_IOWR(DRM_COMMAND_BASE + DRM_MSM_GET_PARAM, struct drm_msm_param)#define DRM_IOCTL_MSM_GEM_NEW DRM_IOWR(DRM_COMMAND_BASE + DRM_MSM_GEM_NEW, struct drm_msm_gem_new)
@@ -327,6 +339,7 @@ struct drm_msm_submitqueue_query {#define DRM_IOCTL_MSM_SUBMITQUEUE_NEW DRM_IOWR(DRM_COMMAND_BASE + DRM_MSM_SUBMITQUEUE_NEW, struct drm_msm_submitqueue)#define DRM_IOCTL_MSM_SUBMITQUEUE_CLOSE DRM_IOW (DRM_COMMAND_BASE + DRM_MSM_SUBMITQUEUE_CLOSE, __u32)#define DRM_IOCTL_MSM_SUBMITQUEUE_QUERY DRM_IOW (DRM_COMMAND_BASE + DRM_MSM_SUBMITQUEUE_QUERY, struct drm_msm_submitqueue_query) +#define DRM_IOCTL_MSM_WAIT_IOVA DRM_IOW (DRM_COMMAND_BASE + DRM_MSM_WAIT_IOVA, struct drm_msm_wait_iova)#if defined(__cplusplus) }
Signature · 1 line
2.25.0.rc1.283.g88dfdc4193-goog
Re: [PATCH 2/2] drm/msm: Add MSM_WAIT_IOVA ioctl
Quoted 45 lines · Brian Ho
Implements an ioctl to wait until a value at a given iova is greater than or equal to a supplied value.
This will initially be used by turnip (open-source Vulkan driver for QC in mesa) for occlusion queries where the userspace driver can block on a query becoming available before continuing via vkGetQueryPoolResults.
Signed-off-by Brian Ho <[email protected]>--- drivers/gpu/drm/msm/msm_drv.c | 63 +++++++++++++++++++++++++++++++++-- include/uapi/drm/msm_drm.h | 13 ++++++++ 2 files changed, 74 insertions(+), 2 deletions(-)drivers/gpu/drm/msm/msm_drv.c +11
-2@@ -36,10 +36,11 @@* MSM_GEM_INFO ioctl. * - 1.4.0 - softpin, MSM_RELOC_BO_DUMP, and GEM_INFO support to set/get * GEM object's debug name -* - 1.5.0 - Add SUBMITQUERY_QUERY ioctl +* - 1.5.0 - Add SUBMITQUEUE_QUERY ioctl +* - 1.6.0 - Add WAIT_IOVA ioctl */ #define MSM_VERSION_MAJOR 1 -#define MSM_VERSION_MINOR 5 +#define MSM_VERSION_MINOR 6#define MSM_VERSION_PATCHLEVEL 0static const struct drm_mode_config_funcs mode_config_funcs = {@@ -952,6 +953,63 @@ static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data,return msm_submitqueue_remove (file->driver_priv, id); } +static int msm_ioctl_wait_iova (struct drm_device *dev,void *data, +struct drm_file *file) +{ +struct msm_drm_private *priv = dev->dev_private; +struct drm_gem_object *obj; +struct drm_msm_wait_iova *args = data; +ktime_t timeout =to_ktime (args->timeout); +unsigned long remaining_jiffies =timeout_to_jiffies (&timeout);
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 973
Quoted 10 lines
drivers/gpu/drm/msm/msm_drv.c +10
-0+struct msm_gpu *gpu = priv->gpu; +void *base_vaddr; +uint64_t *vaddr; +int ret; + +if (args->pad) +return -EINVAL ; + +if (!gpu) +return 0 ;
hmm, I'm not sure we should return zero in this case.. maybe -ENODEV?
Quoted 17 lines
drivers/gpu/drm/msm/msm_drv.c +17
-0+ + obj =drm_gem_object_lookup (file, args->handle); +if (!obj) +return -ENOENT ; + + base_vaddr =msm_gem_get_vaddr (obj); +if (IS_ERR (base_vaddr)) { + ret =PTR_ERR (base_vaddr); + goto err_put_gem_object; + } +if (args->offset +sizeof (*vaddr) > obj->size) { + ret = -EINVAL ; + goto err_put_vaddr; + } + + vaddr = base_vaddr + args->offset; +
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 1000
Quoted 10 lines
drivers/gpu/drm/msm/msm_drv.c +10
-0+/* Assumes WC mapping */ + ret =wait_event_interruptible_timeout ( + gpu->event, *vaddr >= args->value, remaining_jiffies); + +if (ret ==0 ) { + ret = -ETIMEDOUT ; + goto err_put_vaddr; + }else if (ret == -ERESTARTSYS ) { + goto err_put_vaddr; + }
maybe:
} else {
ret = 0;
}
and then drop the next three lines?
Quoted 30 lines
drivers/gpu/drm/msm/msm_drv.c +13
-0+ +msm_gem_put_vaddr (obj); +drm_gem_object_put_unlocked (obj); +return 0 ; + +err_put_vaddr: +msm_gem_put_vaddr (obj); +err_put_gem_object: +drm_gem_object_put_unlocked (obj); +return ret; +} +static const struct drm_ioctl_desc msm_ioctls[] = {DRM_IOCTL_DEF_DRV (MSM_GET_PARAM , msm_ioctl_get_param,DRM_RENDER_ALLOW ),DRM_IOCTL_DEF_DRV (MSM_GEM_NEW , msm_ioctl_gem_new,DRM_RENDER_ALLOW ),@@ -964,6 +1022,7 @@ static const struct drm_ioctl_desc msm_ioctls[] = {DRM_IOCTL_DEF_DRV (MSM_SUBMITQUEUE_NEW , msm_ioctl_submitqueue_new,DRM_RENDER_ALLOW ),DRM_IOCTL_DEF_DRV (MSM_SUBMITQUEUE_CLOSE , msm_ioctl_submitqueue_close,DRM_RENDER_ALLOW ),DRM_IOCTL_DEF_DRV (MSM_SUBMITQUEUE_QUERY , msm_ioctl_submitqueue_query,DRM_RENDER_ALLOW ), + DRM_IOCTL_DEF_DRV(MSM_WAIT_IOVA, msm_ioctl_wait_iova, DRM_RENDER_ALLOW), }; static const struct vm_operations_struct vm_ops = {include/uapi/drm/msm_drm.h +0
-0@@ -298,6 +298,17 @@ struct drm_msm_submitqueue_query {__u32 pad; };
include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 309
Quoted 10 lines
include/uapi/drm/msm_drm.h +9
-0+/* This ioctl blocks until the u64 value at bo + offset is greater than or +* equal to the reference value. +*/ +struct drm_msm_wait_iova { +__u32 handle;/* in, GEM handle */ +__u32 pad; +struct drm_msm_timespec timeout;/* in */ +__u64 offset;/* offset into bo */ +__u64 value;/* reference value */
Maybe we should go ahead and add a __u64 mask;
that would let us wait for 32b values as well, and wait for bits in a bitmask
Other than those minor comments, it looks pretty good to me
BR,
-R
Quoted 24 lines
include/uapi/drm/msm_drm.h +4
-1+}; +#define DRM_MSM_GET_PARAM 0x00/* placeholder: #define DRM_MSM_SET_PARAM 0x01 @@ -315,6 +326,7 @@ struct drm_msm_submitqueue_query {#define DRM_MSM_SUBMITQUEUE_NEW 0x0A#define DRM_MSM_SUBMITQUEUE_CLOSE 0x0B#define DRM_MSM_SUBMITQUEUE_QUERY 0x0C +#define DRM_MSM_WAIT_IOVA 0x0D#define DRM_IOCTL_MSM_GET_PARAM DRM_IOWR(DRM_COMMAND_BASE + DRM_MSM_GET_PARAM, struct drm_msm_param)#define DRM_IOCTL_MSM_GEM_NEW DRM_IOWR(DRM_COMMAND_BASE + DRM_MSM_GEM_NEW, struct drm_msm_gem_new)@@ -327,6 +339,7 @@ struct drm_msm_submitqueue_query {#define DRM_IOCTL_MSM_SUBMITQUEUE_NEW DRM_IOWR(DRM_COMMAND_BASE + DRM_MSM_SUBMITQUEUE_NEW, struct drm_msm_submitqueue)#define DRM_IOCTL_MSM_SUBMITQUEUE_CLOSE DRM_IOW (DRM_COMMAND_BASE + DRM_MSM_SUBMITQUEUE_CLOSE, __u32)#define DRM_IOCTL_MSM_SUBMITQUEUE_QUERY DRM_IOW (DRM_COMMAND_BASE + DRM_MSM_SUBMITQUEUE_QUERY, struct drm_msm_submitqueue_query) +#define DRM_IOCTL_MSM_WAIT_IOVA DRM_IOW (DRM_COMMAND_BASE + DRM_MSM_WAIT_IOVA, struct drm_msm_wait_iova)#if defined(__cplusplus) } --2.25.0.rc1.283.g88dfdc4193-goog
Re: [PATCH 2/2] drm/msm: Add MSM_WAIT_IOVA ioctl
Quoted 74 lines · Rob Clark
Quoted 9 lines · Brian Ho
Implements an ioctl to wait until a value at a given iova is greater than or equal to a supplied value.
This will initially be used by turnip (open-source Vulkan driver for QC in mesa) for occlusion queries where the userspace driver can block on a query becoming available before continuing via vkGetQueryPoolResults.
Review
Quoted 7 lines
Signed-off-by Brian Ho <[email protected]>--- drivers/gpu/drm/msm/msm_drv.c | 63 +++++++++++++++++++++++++++++++++-- include/uapi/drm/msm_drm.h | 13 ++++++++ 2 files changed, 74 insertions(+), 2 deletions(-)(quoted diff) +0
-0
b/drivers/gpu/drm/msm/msm_drv.cdrivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct
Quoted 18 lines
index
c84f0a8b3f2c..dcc46874a5a2100644drivers/gpu/drm/msm/msm_drv.c +3
-2@@ -36,10 +36,11 @@* MSM_GEM_INFO ioctl. * - 1.4.0 - softpin, MSM_RELOC_BO_DUMP, and GEM_INFO support to set/get * GEM object's debug name -* - 1.5.0 - Add SUBMITQUERY_QUERY ioctl +* - 1.5.0 - Add SUBMITQUEUE_QUERY ioctl +* - 1.6.0 - Add WAIT_IOVA ioctl */ #define MSM_VERSION_MAJOR 1 -#define MSM_VERSION_MINOR 5 +#define MSM_VERSION_MINOR 6#define MSM_VERSION_PATCHLEVEL 0static const struct drm_mode_config_funcs mode_config_funcs = {@@ -952,6 +953,63 @@ static int msm_ioctl_submitqueue_close(struct
drm_device*dev, void *data,Quoted 11 lines
drivers/gpu/drm/msm/msm_drv.c +8
-0return msm_submitqueue_remove (file->driver_priv, id); } +static int msm_ioctl_wait_iova (struct drm_device *dev,void *data, +struct drm_file *file) +{ +struct msm_drm_private *priv = dev->dev_private; +struct drm_gem_object *obj; +struct drm_msm_wait_iova *args = data; +ktime_t timeout =to_ktime (args->timeout); +unsigned long remaining_jiffies =timeout_to_jiffies (&timeout);drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 973
Quoted 10 lines
drivers/gpu/drm/msm/msm_drv.c +10
-0+struct msm_gpu *gpu = priv->gpu; +void *base_vaddr; +uint64_t *vaddr; +int ret; + +if (args->pad) +return -EINVAL ; + +if (!gpu) +return 0 ;hmm, I'm not sure we should return zero in this case.. maybe
-ENODEV?Quoted 13 lines
drivers/gpu/drm/msm/msm_drv.c +13
-0+ + obj =drm_gem_object_lookup (file, args->handle); +if (!obj) +return -ENOENT ; + + base_vaddr =msm_gem_get_vaddr (obj); +if (IS_ERR (base_vaddr)) { + ret =PTR_ERR (base_vaddr); + goto err_put_gem_object; + } +if (args->offset +sizeof (*vaddr) > obj->size) { + ret = -EINVAL ; + goto err_put_vaddr;
Quoted 9 lines
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data,
Quoted 7 lines
drivers/gpu/drm/msm/msm_drv.c +7
-0+ } + + vaddr = base_vaddr + args->offset; + +/* Assumes WC mapping */ + ret = wait_event_interruptible_timeout( + gpu->event, *vaddr >= args->value,
remaining_jiffies);
This needs to do the awkward looking
(int64_t)(*data - value) >= 0
to properly handle the wraparound case.
Quoted 64 lines
+
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 1000
Quoted 6 lines
drivers/gpu/drm/msm/msm_drv.c +6
-0+if (ret ==0 ) { + ret = -ETIMEDOUT ; + goto err_put_vaddr; + }else if (ret == -ERESTARTSYS ) { + goto err_put_vaddr; + }maybe:
} else { ret = 0;}
and then drop the next three lines?
Quoted 4 lines
drivers/gpu/drm/msm/msm_drv.c +4
-0+ +msm_gem_put_vaddr (obj); +drm_gem_object_put_unlocked (obj); +return 0 ;drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data,
Quoted 10 lines
drivers/gpu/drm/msm/msm_drv.c +8
-0+ +err_put_vaddr: +msm_gem_put_vaddr (obj); +err_put_gem_object: +drm_gem_object_put_unlocked (obj); +return ret; +} +static const struct drm_ioctl_desc msm_ioctls[] = { DRM_IOCTL_DEF_DRV(MSM_GET_PARAM, msm_ioctl_get_param,
DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct · line 1014
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +0
-0DRM_IOCTL_DEF_DRV(MSM_GEM_NEW, msm_ioctl_gem_new,
DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1022
Quoted 2 lines
drivers/gpu/drm/msm/msm_drv.c +0
-0@@ -964,6 +1022,7 @@ static const struct drm_ioctl_desc msm_ioctls[] = { DRM_IOCTL_DEF_DRV(MSM_SUBMITQUEUE_NEW,
msm_ioctl_submitqueue_new,DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1023
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +0
-0DRM_IOCTL_DEF_DRV(MSM_SUBMITQUEUE_CLOSE,
msm_ioctl_submitqueue_close,DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1024
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +0
-0DRM_IOCTL_DEF_DRV(MSM_SUBMITQUEUE_QUERY,
msm_ioctl_submitqueue_query,DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1025
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +1
-0+ DRM_IOCTL_DEF_DRV(MSM_WAIT_IOVA, msm_ioctl_wait_iova,
DRM_RENDER_ALLOW),include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 301
Quoted 12 lines
drivers/gpu/drm/msm/msm_drv.c +0
-0};static const struct vm_operations_struct vm_ops = {include/uapi/drm/msm_drm.h +1
-0@@ -298,6 +298,17 @@ struct drm_msm_submitqueue_query {__u32 pad; }; +/* This ioctl blocks until the u64 value at bo + offset is greater than or
include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 309
Quoted 8 lines
include/uapi/drm/msm_drm.h +8
-0+* equal to the reference value. +*/ +struct drm_msm_wait_iova { +__u32 handle;/* in, GEM handle */ +__u32 pad; +struct drm_msm_timespec timeout;/* in */ +__u64 offset;/* offset into bo */ +__u64 value;/* reference value */
Quoted 5 lines
Maybe we should go ahead and add a
__u64mask;that would let us wait for 32b values as well, and wait for bits in a bitmask
I think we'd be OK to just default to 32 bit values instead, since most of the CP commands that this is intended to work with (CP_EVENT_WRITE, CP_WAIT_MEM_GTE etc) operate on 32 bit values. We could move 'value' to the slot right after 'handle' but then we'd not have any pad/reserved fields. Maybe we keep 'value' 64 bit but restrict it to 32 bits, with an option to add a 64 bit flag in 'pad' later on?
Quoted 24 lines
Other than those minor comments, it looks pretty good to me
BR,
-Rinclude/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 331
Quoted 12 lines
include/uapi/drm/msm_drm.h +3
-0+}; +#define DRM_MSM_GET_PARAM 0x00/* placeholder: #define DRM_MSM_SET_PARAM 0x01 @@ -315,6 +326,7 @@ struct drm_msm_submitqueue_query {#define DRM_MSM_SUBMITQUEUE_NEW 0x0A#define DRM_MSM_SUBMITQUEUE_CLOSE 0x0B#define DRM_MSM_SUBMITQUEUE_QUERY 0x0C +#define DRM_MSM_WAIT_IOVA 0x0D#define DRM_IOCTL_MSM_GET_PARAM DRM_IOWR(DRM_COMMAND_BASE +
DRM_MSM_GET_PARAM, structdrm_msm_param)include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 332
Quoted 1 line
include/uapi/drm/msm_drm.h +0
-0#define DRM_IOCTL_MSM_GEM_NEW DRM_IOWR(DRM_COMMAND_BASE +
DRM_MSM_GEM_NEW, structdrm_msm_gem_new)include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 339
Quoted 2 lines
include/uapi/drm/msm_drm.h +0
-0@@ -327,6 +339,7 @@ struct drm_msm_submitqueue_query {#define DRM_IOCTL_MSM_SUBMITQUEUE_NEW DRM_IOWR(DRM_COMMAND_BASE +
DRM_MSM_SUBMITQUEUE_NEW, structdrm_msm_submitqueue)
Quoted 13 lines
include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 340
Quoted 1 line
include/uapi/drm/msm_drm.h +0
-0#define DRM_IOCTL_MSM_SUBMITQUEUE_CLOSE DRM_IOW (DRM_COMMAND_BASE +
DRM_MSM_SUBMITQUEUE_CLOSE,__u32)include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 341
Quoted 1 line
include/uapi/drm/msm_drm.h +0
-0#define DRM_IOCTL_MSM_SUBMITQUEUE_QUERY DRM_IOW (DRM_COMMAND_BASE +
DRM_MSM_SUBMITQUEUE_QUERY, structdrm_msm_submitqueue_query)include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 342
Quoted 1 line
include/uapi/drm/msm_drm.h +1
-0+#define DRM_IOCTL_MSM_WAIT_IOVA DRM_IOW (DRM_COMMAND_BASE +
DRM_MSM_WAIT_IOVA, structdrm_msm_wait_iova)Quoted 6 lines
include/uapi/drm/msm_drm.h +0
-1#if defined(__cplusplus) } --2.25.0.rc1.283.g88dfdc4193-goog
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [PATCH 2/2] drm/msm: Add MSM_WAIT_IOVA ioctl
Quoted 86 lines · Kristian Kristensen
Quoted 74 lines · Rob Clark
Quoted 9 lines · Brian Ho
Implements an ioctl to wait until a value at a given iova is greater than or equal to a supplied value.
This will initially be used by turnip (open-source Vulkan driver for QC in mesa) for occlusion queries where the userspace driver can block on a query becoming available before continuing via vkGetQueryPoolResults.
Review
Quoted 7 lines
Signed-off-by Brian Ho <[email protected]>--- drivers/gpu/drm/msm/msm_drv.c | 63 +++++++++++++++++++++++++++++++++-- include/uapi/drm/msm_drm.h | 13 ++++++++ 2 files changed, 74 insertions(+), 2 deletions(-)(quoted diff) +0
-0
b/drivers/gpu/drm/msm/msm_drv.cdrivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct
Quoted 18 lines
index
c84f0a8b3f2c..dcc46874a5a2100644drivers/gpu/drm/msm/msm_drv.c +3
-2@@ -36,10 +36,11 @@* MSM_GEM_INFO ioctl. * - 1.4.0 - softpin, MSM_RELOC_BO_DUMP, and GEM_INFO support to set/get * GEM object's debug name -* - 1.5.0 - Add SUBMITQUERY_QUERY ioctl +* - 1.5.0 - Add SUBMITQUEUE_QUERY ioctl +* - 1.6.0 - Add WAIT_IOVA ioctl */ #define MSM_VERSION_MAJOR 1 -#define MSM_VERSION_MINOR 5 +#define MSM_VERSION_MINOR 6#define MSM_VERSION_PATCHLEVEL 0static const struct drm_mode_config_funcs mode_config_funcs = {@@ -952,6 +953,63 @@ static int msm_ioctl_submitqueue_close(struct
drm_device*dev, void *data,Quoted 11 lines
drivers/gpu/drm/msm/msm_drv.c +8
-0return msm_submitqueue_remove (file->driver_priv, id); } +static int msm_ioctl_wait_iova (struct drm_device *dev,void *data, +struct drm_file *file) +{ +struct msm_drm_private *priv = dev->dev_private; +struct drm_gem_object *obj; +struct drm_msm_wait_iova *args = data; +ktime_t timeout =to_ktime (args->timeout); +unsigned long remaining_jiffies =timeout_to_jiffies (&timeout);drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 973
Quoted 10 lines
drivers/gpu/drm/msm/msm_drv.c +10
-0+struct msm_gpu *gpu = priv->gpu; +void *base_vaddr; +uint64_t *vaddr; +int ret; + +if (args->pad) +return -EINVAL ; + +if (!gpu) +return 0 ;hmm, I'm not sure we should return zero in this case.. maybe
-ENODEV?Quoted 13 lines
drivers/gpu/drm/msm/msm_drv.c +13
-0+ + obj =drm_gem_object_lookup (file, args->handle); +if (!obj) +return -ENOENT ; + + base_vaddr =msm_gem_get_vaddr (obj); +if (IS_ERR (base_vaddr)) { + ret =PTR_ERR (base_vaddr); + goto err_put_gem_object; + } +if (args->offset +sizeof (*vaddr) > obj->size) { + ret = -EINVAL ; + goto err_put_vaddr;Quoted 9 lines
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data,
Quoted 7 lines
drivers/gpu/drm/msm/msm_drv.c +7
-0+ } + + vaddr = base_vaddr + args->offset; + +/* Assumes WC mapping */ + ret = wait_event_interruptible_timeout( + gpu->event, *vaddr >= args->value,
remaining_jiffies);
Quoted 6 lines
This needs to do the awkward looking
(int64_t)(*data - value) >= 0
to properly handle the wraparound case.
I think this comparison will run into issues if we allow for 64-bit reference values. For example, if value is ULLONG_MAX, and *data starts at 0 on the first comparison, we'll immediately return.
It's not too much of an issue in fence_completed (msm_fence.c), but in this ioctl, *data can grow at an arbitrary rate. Are we concerned about this?
Quoted 70 lines
Quoted 64 lines
+
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 1000
Quoted 6 lines
drivers/gpu/drm/msm/msm_drv.c +6
-0+if (ret ==0 ) { + ret = -ETIMEDOUT ; + goto err_put_vaddr; + }else if (ret == -ERESTARTSYS ) { + goto err_put_vaddr; + }maybe:
} else { ret = 0;}
and then drop the next three lines?
Quoted 4 lines
drivers/gpu/drm/msm/msm_drv.c +4
-0+ +msm_gem_put_vaddr (obj); +drm_gem_object_put_unlocked (obj); +return 0 ;drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data,
Quoted 10 lines
drivers/gpu/drm/msm/msm_drv.c +8
-0+ +err_put_vaddr: +msm_gem_put_vaddr (obj); +err_put_gem_object: +drm_gem_object_put_unlocked (obj); +return ret; +} +static const struct drm_ioctl_desc msm_ioctls[] = { DRM_IOCTL_DEF_DRV(MSM_GET_PARAM, msm_ioctl_get_param,
DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct · line 1014
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +0
-0DRM_IOCTL_DEF_DRV(MSM_GEM_NEW, msm_ioctl_gem_new,
DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1022
Quoted 2 lines
drivers/gpu/drm/msm/msm_drv.c +0
-0@@ -964,6 +1022,7 @@ static const struct drm_ioctl_desc msm_ioctls[] = { DRM_IOCTL_DEF_DRV(MSM_SUBMITQUEUE_NEW,
msm_ioctl_submitqueue_new,DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1023
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +0
-0DRM_IOCTL_DEF_DRV(MSM_SUBMITQUEUE_CLOSE,
msm_ioctl_submitqueue_close,DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1024
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +0
-0DRM_IOCTL_DEF_DRV(MSM_SUBMITQUEUE_QUERY,
msm_ioctl_submitqueue_query,DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1025
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +1
-0+ DRM_IOCTL_DEF_DRV(MSM_WAIT_IOVA, msm_ioctl_wait_iova,
DRM_RENDER_ALLOW),include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 301
Quoted 12 lines
drivers/gpu/drm/msm/msm_drv.c +0
-0};static const struct vm_operations_struct vm_ops = {include/uapi/drm/msm_drm.h +1
-0@@ -298,6 +298,17 @@ struct drm_msm_submitqueue_query {__u32 pad; }; +/* This ioctl blocks until the u64 value at bo + offset is greater than or
include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 309
Quoted 8 lines
include/uapi/drm/msm_drm.h +8
-0+* equal to the reference value. +*/ +struct drm_msm_wait_iova { +__u32 handle;/* in, GEM handle */ +__u32 pad; +struct drm_msm_timespec timeout;/* in */ +__u64 offset;/* offset into bo */ +__u64 value;/* reference value */ Quoted 5 lines
Maybe we should go ahead and add a
__u64mask;that would let us wait for 32b values as well, and wait for bits in a bitmask
Quoted 7 lines
I think we'd be OK to just default to 32 bit values instead, since most of the CP commands that this is intended to work with (
CP_EVENT_WRITE,CP_WAIT_MEM_GTEetc) operate on 32 bit values. We could move 'value' to the slot right after 'handle' but then we'd not have any pad/reserved fields. Maybe we keep 'value' 64 bit but restrict it to 32 bits, with an option to add a 64 bit flag in 'pad' later on?
FWIW, the current usage of this in my mesa MR uses a 64 bit value. There's no super great reason that the available bit is 64 bits and not 32 bits (I think it made the addressing math a bit simpler), but I'm fine with whatever you all decide on here.
Quoted 37 lines
Quoted 37 lines
Other than those minor comments, it looks pretty good to me
BR,
-Rinclude/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 331
Quoted 12 lines
include/uapi/drm/msm_drm.h +3
-0+}; +#define DRM_MSM_GET_PARAM 0x00/* placeholder: #define DRM_MSM_SET_PARAM 0x01 @@ -315,6 +326,7 @@ struct drm_msm_submitqueue_query {#define DRM_MSM_SUBMITQUEUE_NEW 0x0A#define DRM_MSM_SUBMITQUEUE_CLOSE 0x0B#define DRM_MSM_SUBMITQUEUE_QUERY 0x0C +#define DRM_MSM_WAIT_IOVA 0x0D#define DRM_IOCTL_MSM_GET_PARAM DRM_IOWR(DRM_COMMAND_BASE +
DRM_MSM_GET_PARAM, structdrm_msm_param)include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 332
Quoted 1 line
include/uapi/drm/msm_drm.h +0
-0#define DRM_IOCTL_MSM_GEM_NEW DRM_IOWR(DRM_COMMAND_BASE +
DRM_MSM_GEM_NEW, structdrm_msm_gem_new)include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 339
Quoted 2 lines
include/uapi/drm/msm_drm.h +0
-0@@ -327,6 +339,7 @@ struct drm_msm_submitqueue_query {#define DRM_IOCTL_MSM_SUBMITQUEUE_NEW DRM_IOWR(DRM_COMMAND_BASE +
DRM_MSM_SUBMITQUEUE_NEW, structdrm_msm_submitqueue)include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 340
Quoted 1 line
include/uapi/drm/msm_drm.h +0
-0#define DRM_IOCTL_MSM_SUBMITQUEUE_CLOSE DRM_IOW (DRM_COMMAND_BASE +
DRM_MSM_SUBMITQUEUE_CLOSE,__u32)include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 341
Quoted 1 line
include/uapi/drm/msm_drm.h +0
-0#define DRM_IOCTL_MSM_SUBMITQUEUE_QUERY DRM_IOW (DRM_COMMAND_BASE +
DRM_MSM_SUBMITQUEUE_QUERY, structdrm_msm_submitqueue_query)include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 342
Quoted 1 line
include/uapi/drm/msm_drm.h +1
-0+#define DRM_IOCTL_MSM_WAIT_IOVA DRM_IOW (DRM_COMMAND_BASE +
DRM_MSM_WAIT_IOVA, structdrm_msm_wait_iova)Quoted 6 lines
include/uapi/drm/msm_drm.h +0
-1#if defined(__cplusplus) } --2.25.0.rc1.283.g88dfdc4193-goog
Re: [PATCH 2/2] drm/msm: Add MSM_WAIT_IOVA ioctl
Quoted 181 lines · Brian Ho
Quoted 86 lines · Kristian Kristensen
Quoted 74 lines · Rob Clark
Quoted 9 lines · Brian Ho
Implements an ioctl to wait until a value at a given iova is greater than or equal to a supplied value.
This will initially be used by turnip (open-source Vulkan driver for QC in mesa) for occlusion queries where the userspace driver can block on a query becoming available before continuing via vkGetQueryPoolResults.
Review
Quoted 7 lines
Signed-off-by Brian Ho <[email protected]>--- drivers/gpu/drm/msm/msm_drv.c | 63 +++++++++++++++++++++++++++++++++-- include/uapi/drm/msm_drm.h | 13 ++++++++ 2 files changed, 74 insertions(+), 2 deletions(-)(quoted diff) +0
-0
b/drivers/gpu/drm/msm/msm_drv.cdrivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct
Quoted 18 lines
index
c84f0a8b3f2c..dcc46874a5a2100644drivers/gpu/drm/msm/msm_drv.c +3
-2@@ -36,10 +36,11 @@* MSM_GEM_INFO ioctl. * - 1.4.0 - softpin, MSM_RELOC_BO_DUMP, and GEM_INFO support to set/get * GEM object's debug name -* - 1.5.0 - Add SUBMITQUERY_QUERY ioctl +* - 1.5.0 - Add SUBMITQUEUE_QUERY ioctl +* - 1.6.0 - Add WAIT_IOVA ioctl */ #define MSM_VERSION_MAJOR 1 -#define MSM_VERSION_MINOR 5 +#define MSM_VERSION_MINOR 6#define MSM_VERSION_PATCHLEVEL 0static const struct drm_mode_config_funcs mode_config_funcs = {@@ -952,6 +953,63 @@ static int msm_ioctl_submitqueue_close(struct
drm_device*dev, void *data,Quoted 11 lines
drivers/gpu/drm/msm/msm_drv.c +8
-0return msm_submitqueue_remove (file->driver_priv, id); } +static int msm_ioctl_wait_iova (struct drm_device *dev,void *data, +struct drm_file *file) +{ +struct msm_drm_private *priv = dev->dev_private; +struct drm_gem_object *obj; +struct drm_msm_wait_iova *args = data; +ktime_t timeout =to_ktime (args->timeout); +unsigned long remaining_jiffies =timeout_to_jiffies (&timeout);drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 973
Quoted 10 lines
drivers/gpu/drm/msm/msm_drv.c +10
-0+struct msm_gpu *gpu = priv->gpu; +void *base_vaddr; +uint64_t *vaddr; +int ret; + +if (args->pad) +return -EINVAL ; + +if (!gpu) +return 0 ;hmm, I'm not sure we should return zero in this case.. maybe
-ENODEV?Quoted 13 lines
drivers/gpu/drm/msm/msm_drv.c +13
-0+ + obj =drm_gem_object_lookup (file, args->handle); +if (!obj) +return -ENOENT ; + + base_vaddr =msm_gem_get_vaddr (obj); +if (IS_ERR (base_vaddr)) { + ret =PTR_ERR (base_vaddr); + goto err_put_gem_object; + } +if (args->offset +sizeof (*vaddr) > obj->size) { + ret = -EINVAL ; + goto err_put_vaddr;Quoted 9 lines
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data,
Quoted 7 lines
drivers/gpu/drm/msm/msm_drv.c +7
-0+ } + + vaddr = base_vaddr + args->offset; + +/* Assumes WC mapping */ + ret = wait_event_interruptible_timeout( + gpu->event, *vaddr >= args->value,
remaining_jiffies);Quoted 6 lines
This needs to do the awkward looking
(int64_t)(*data - value) >= 0
to properly handle the wraparound case.
I think this comparison will run into issues if we allow for 64-bit reference values. For example, if value is
ULLONG_MAX, and *data starts at 0 on the first comparison, we'll immediately return.It's not too much of an issue in
fence_completed(msm_fence.c), but in this ioctl, *data can grow at an arbitrary rate. Are we concerned about this?Quoted 70 lines
Quoted 64 lines
+
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 1000
Quoted 6 lines
drivers/gpu/drm/msm/msm_drv.c +6
-0+if (ret ==0 ) { + ret = -ETIMEDOUT ; + goto err_put_vaddr; + }else if (ret == -ERESTARTSYS ) { + goto err_put_vaddr; + }maybe:
} else { ret = 0;}
and then drop the next three lines?
Quoted 4 lines
drivers/gpu/drm/msm/msm_drv.c +4
-0+ +msm_gem_put_vaddr (obj); +drm_gem_object_put_unlocked (obj); +return 0 ;drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data,
Quoted 10 lines
drivers/gpu/drm/msm/msm_drv.c +8
-0+ +err_put_vaddr: +msm_gem_put_vaddr (obj); +err_put_gem_object: +drm_gem_object_put_unlocked (obj); +return ret; +} +static const struct drm_ioctl_desc msm_ioctls[] = { DRM_IOCTL_DEF_DRV(MSM_GET_PARAM, msm_ioctl_get_param,
DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct · line 1014
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +0
-0DRM_IOCTL_DEF_DRV(MSM_GEM_NEW, msm_ioctl_gem_new,
DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1022
Quoted 2 lines
drivers/gpu/drm/msm/msm_drv.c +0
-0@@ -964,6 +1022,7 @@ static const struct drm_ioctl_desc msm_ioctls[] = { DRM_IOCTL_DEF_DRV(MSM_SUBMITQUEUE_NEW,
msm_ioctl_submitqueue_new,DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1023
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +0
-0DRM_IOCTL_DEF_DRV(MSM_SUBMITQUEUE_CLOSE,
msm_ioctl_submitqueue_close,DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1024
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +0
-0DRM_IOCTL_DEF_DRV(MSM_SUBMITQUEUE_QUERY,
msm_ioctl_submitqueue_query,DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1025
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +1
-0+ DRM_IOCTL_DEF_DRV(MSM_WAIT_IOVA, msm_ioctl_wait_iova,
DRM_RENDER_ALLOW),include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 301
Quoted 12 lines
drivers/gpu/drm/msm/msm_drv.c +0
-0};static const struct vm_operations_struct vm_ops = {include/uapi/drm/msm_drm.h +1
-0@@ -298,6 +298,17 @@ struct drm_msm_submitqueue_query {__u32 pad; }; +/* This ioctl blocks until the u64 value at bo + offset is greater than or
include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 309
Quoted 8 lines
include/uapi/drm/msm_drm.h +8
-0+* equal to the reference value. +*/ +struct drm_msm_wait_iova { +__u32 handle;/* in, GEM handle */ +__u32 pad; +struct drm_msm_timespec timeout;/* in */ +__u64 offset;/* offset into bo */ +__u64 value;/* reference value */ Quoted 5 lines
Maybe we should go ahead and add a
__u64mask;that would let us wait for 32b values as well, and wait for bits in a bitmask
Quoted 7 lines
I think we'd be OK to just default to 32 bit values instead, since most of the CP commands that this is intended to work with (
CP_EVENT_WRITE,CP_WAIT_MEM_GTEetc) operate on 32 bit values. We could move 'value' to the slot right after 'handle' but then we'd not have any pad/reserved fields. Maybe we keep 'value' 64 bit but restrict it to 32 bits, with an option to add a 64 bit flag in 'pad' later on?
Quoted 5 lines
FWIW, the current usage of this in my mesa MR uses a 64 bit value. There's no super great reason that the available bit is 64 bits and not 32 bits (I think it made the addressing math a bit simpler), but I'm fine with whatever you all decide on here.
I assume you are waiting for a fence value written w/ CP_EVENT_WRITE? Or at least that is what I'd recommend. That would be 32b
BR,
-R
Quoted 37 lines
Quoted 37 lines
Quoted 37 lines
Other than those minor comments, it looks pretty good to me
BR,
-Rinclude/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 331
Quoted 12 lines
include/uapi/drm/msm_drm.h +3
-0+}; +#define DRM_MSM_GET_PARAM 0x00/* placeholder: #define DRM_MSM_SET_PARAM 0x01 @@ -315,6 +326,7 @@ struct drm_msm_submitqueue_query {#define DRM_MSM_SUBMITQUEUE_NEW 0x0A#define DRM_MSM_SUBMITQUEUE_CLOSE 0x0B#define DRM_MSM_SUBMITQUEUE_QUERY 0x0C +#define DRM_MSM_WAIT_IOVA 0x0D#define DRM_IOCTL_MSM_GET_PARAM DRM_IOWR(DRM_COMMAND_BASE +
DRM_MSM_GET_PARAM, structdrm_msm_param)include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 332
Quoted 1 line
include/uapi/drm/msm_drm.h +0
-0#define DRM_IOCTL_MSM_GEM_NEW DRM_IOWR(DRM_COMMAND_BASE +
DRM_MSM_GEM_NEW, structdrm_msm_gem_new)include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 339
Quoted 2 lines
include/uapi/drm/msm_drm.h +0
-0@@ -327,6 +339,7 @@ struct drm_msm_submitqueue_query {#define DRM_IOCTL_MSM_SUBMITQUEUE_NEW DRM_IOWR(DRM_COMMAND_BASE +
DRM_MSM_SUBMITQUEUE_NEW, structdrm_msm_submitqueue)include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 340
Quoted 1 line
include/uapi/drm/msm_drm.h +0
-0#define DRM_IOCTL_MSM_SUBMITQUEUE_CLOSE DRM_IOW (DRM_COMMAND_BASE +
DRM_MSM_SUBMITQUEUE_CLOSE,__u32)include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 341
Quoted 1 line
include/uapi/drm/msm_drm.h +0
-0#define DRM_IOCTL_MSM_SUBMITQUEUE_QUERY DRM_IOW (DRM_COMMAND_BASE +
DRM_MSM_SUBMITQUEUE_QUERY, structdrm_msm_submitqueue_query)include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 342
Quoted 1 line
include/uapi/drm/msm_drm.h +1
-0+#define DRM_IOCTL_MSM_WAIT_IOVA DRM_IOW (DRM_COMMAND_BASE +
DRM_MSM_WAIT_IOVA, structdrm_msm_wait_iova)Quoted 6 lines
include/uapi/drm/msm_drm.h +0
-1#if defined(__cplusplus) } --2.25.0.rc1.283.g88dfdc4193-goog
Re: [Freedreno] [PATCH 2/2] drm/msm: Add MSM_WAIT_IOVA ioctl
Quoted 92 lines · Rob Clark
Quoted 90 lines · Brian Ho
Quoted 88 lines · Kristian Kristensen
Quoted 74 lines · Rob Clark
Quoted 9 lines · Brian Ho
Implements an ioctl to wait until a value at a given iova is greater than or equal to a supplied value.
This will initially be used by turnip (open-source Vulkan driver for QC in mesa) for occlusion queries where the userspace driver can block on a query becoming available before continuing via vkGetQueryPoolResults.
Review
Quoted 7 lines
Signed-off-by Brian Ho <[email protected]>--- drivers/gpu/drm/msm/msm_drv.c | 63 +++++++++++++++++++++++++++++++++-- include/uapi/drm/msm_drm.h | 13 ++++++++ 2 files changed, 74 insertions(+), 2 deletions(-)(quoted diff) +0
-0
b/drivers/gpu/drm/msm/msm_drv.cdrivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct
Quoted 18 lines
index
c84f0a8b3f2c..dcc46874a5a2100644drivers/gpu/drm/msm/msm_drv.c +3
-2@@ -36,10 +36,11 @@* MSM_GEM_INFO ioctl. * - 1.4.0 - softpin, MSM_RELOC_BO_DUMP, and GEM_INFO support to set/get * GEM object's debug name -* - 1.5.0 - Add SUBMITQUERY_QUERY ioctl +* - 1.5.0 - Add SUBMITQUEUE_QUERY ioctl +* - 1.6.0 - Add WAIT_IOVA ioctl */ #define MSM_VERSION_MAJOR 1 -#define MSM_VERSION_MINOR 5 +#define MSM_VERSION_MINOR 6#define MSM_VERSION_PATCHLEVEL 0static const struct drm_mode_config_funcs mode_config_funcs = {@@ -952,6 +953,63 @@ static int msm_ioctl_submitqueue_close(struct
drm_device*dev, void *data,Quoted 11 lines
drivers/gpu/drm/msm/msm_drv.c +8
-0return msm_submitqueue_remove (file->driver_priv, id); } +static int msm_ioctl_wait_iova (struct drm_device *dev,void *data, +struct drm_file *file) +{ +struct msm_drm_private *priv = dev->dev_private; +struct drm_gem_object *obj; +struct drm_msm_wait_iova *args = data; +ktime_t timeout =to_ktime (args->timeout); +unsigned long remaining_jiffies =timeout_to_jiffies (&timeout);drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 973
Quoted 10 lines
drivers/gpu/drm/msm/msm_drv.c +10
-0+struct msm_gpu *gpu = priv->gpu; +void *base_vaddr; +uint64_t *vaddr; +int ret; + +if (args->pad) +return -EINVAL ; + +if (!gpu) +return 0 ;hmm, I'm not sure we should return zero in this case.. maybe
-ENODEV?Quoted 13 lines
drivers/gpu/drm/msm/msm_drv.c +13
-0+ + obj =drm_gem_object_lookup (file, args->handle); +if (!obj) +return -ENOENT ; + + base_vaddr =msm_gem_get_vaddr (obj); +if (IS_ERR (base_vaddr)) { + ret =PTR_ERR (base_vaddr); + goto err_put_gem_object; + } +if (args->offset +sizeof (*vaddr) > obj->size) { + ret = -EINVAL ; + goto err_put_vaddr;Quoted 9 lines
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data,
Quoted 7 lines
drivers/gpu/drm/msm/msm_drv.c +7
-0+ } + + vaddr = base_vaddr + args->offset; + +/* Assumes WC mapping */ + ret = wait_event_interruptible_timeout( + gpu->event, *vaddr >= args->value,
remaining_jiffies);This needs to do the awkward looking
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
(int64_t)(*data - value) >= 0
to properly handle the wraparound case.
I think this comparison will run into issues if we allow for 64-bit reference values. For example, if value is
ULLONG_MAX, and *data starts at 0 on the first comparison, we'll immediately return.
The comparison will have to account of the number of bits in the serial number. The (int64_t) (*data - value) works for 64 bit unsigned serial numbers, but for 32 bit serials as suggested below, we need to cast to int32_t. It does work though, in the case where value is ULLONG_MAX and and *data is 0, 0 - ULLONG_MAX is one, which is correct. The serial numbers wrap around and the distance is computed modulo 2^64. See
https://en.wikipedia.org/wiki/Serial_number_arithmetic.
Quoted 136 lines
Quoted 83 lines
It's not too much of an issue in
fence_completed(msm_fence.c), but in this ioctl, *data can grow at an arbitrary rate. Are we concerned about this?Quoted 70 lines
Quoted 64 lines
+
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 1000
Quoted 6 lines
drivers/gpu/drm/msm/msm_drv.c +6
-0+if (ret ==0 ) { + ret = -ETIMEDOUT ; + goto err_put_vaddr; + }else if (ret == -ERESTARTSYS ) { + goto err_put_vaddr; + }maybe:
} else { ret = 0;}
and then drop the next three lines?
Quoted 4 lines
drivers/gpu/drm/msm/msm_drv.c +4
-0+ +msm_gem_put_vaddr (obj); +drm_gem_object_put_unlocked (obj); +return 0 ;drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data,
Quoted 10 lines
drivers/gpu/drm/msm/msm_drv.c +8
-0+ +err_put_vaddr: +msm_gem_put_vaddr (obj); +err_put_gem_object: +drm_gem_object_put_unlocked (obj); +return ret; +} +static const struct drm_ioctl_desc msm_ioctls[] = { DRM_IOCTL_DEF_DRV(MSM_GET_PARAM, msm_ioctl_get_param,
DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct · line 1014
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +0
-0DRM_IOCTL_DEF_DRV(MSM_GEM_NEW, msm_ioctl_gem_new,
DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1022
Quoted 2 lines
drivers/gpu/drm/msm/msm_drv.c +0
-0@@ -964,6 +1022,7 @@ static const struct drm_ioctl_desc msm_ioctls[] = { DRM_IOCTL_DEF_DRV(MSM_SUBMITQUEUE_NEW,
msm_ioctl_submitqueue_new,DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1023
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +0
-0DRM_IOCTL_DEF_DRV(MSM_SUBMITQUEUE_CLOSE,
msm_ioctl_submitqueue_close,DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1024
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +0
-0DRM_IOCTL_DEF_DRV(MSM_SUBMITQUEUE_QUERY,
msm_ioctl_submitqueue_query,DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1025
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +1
-0+ DRM_IOCTL_DEF_DRV(MSM_WAIT_IOVA, msm_ioctl_wait_iova,
DRM_RENDER_ALLOW),include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 301
Quoted 12 lines
drivers/gpu/drm/msm/msm_drv.c +0
-0};static const struct vm_operations_struct vm_ops = {include/uapi/drm/msm_drm.h +1
-0@@ -298,6 +298,17 @@ struct drm_msm_submitqueue_query {__u32 pad; }; +/* This ioctl blocks until the u64 value at bo + offset is greater than or
include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 309
Quoted 8 lines
include/uapi/drm/msm_drm.h +8
-0+* equal to the reference value. +*/ +struct drm_msm_wait_iova { +__u32 handle;/* in, GEM handle */ +__u32 pad; +struct drm_msm_timespec timeout;/* in */ +__u64 offset;/* offset into bo */ +__u64 value;/* reference value */ Quoted 5 lines
Maybe we should go ahead and add a
__u64mask;that would let us wait for 32b values as well, and wait for bits in a bitmask
Quoted 7 lines
I think we'd be OK to just default to 32 bit values instead, since most of the CP commands that this is intended to work with (
CP_EVENT_WRITE,CP_WAIT_MEM_GTEetc) operate on 32 bit values. We could move 'value' to the slot right after 'handle' but then we'd not have any pad/reserved fields. Maybe we keep 'value' 64 bit but restrict it to 32 bits, with an option to add a 64 bit flag in 'pad' later on?Quoted 5 lines
FWIW, the current usage of this in my mesa MR uses a 64 bit value. There's no super great reason that the available bit is 64 bits and not 32 bits (I think it made the addressing math a bit simpler), but I'm fine with whatever you all decide on here.
I assume you are waiting for a fence value written w/
CP_EVENT_WRITE? Or at least that is what I'd recommend. That would be 32bBR,
-RQuoted 29 lines
Quoted 29 lines
Quoted 29 lines
Other than those minor comments, it looks pretty good to me
BR,
-Rinclude/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 331
Quoted 12 lines
include/uapi/drm/msm_drm.h +3
-0+}; +#define DRM_MSM_GET_PARAM 0x00/* placeholder: #define DRM_MSM_SET_PARAM 0x01 @@ -315,6 +326,7 @@ struct drm_msm_submitqueue_query {#define DRM_MSM_SUBMITQUEUE_NEW 0x0A#define DRM_MSM_SUBMITQUEUE_CLOSE 0x0B#define DRM_MSM_SUBMITQUEUE_QUERY 0x0C +#define DRM_MSM_WAIT_IOVA 0x0D#define DRM_IOCTL_MSM_GET_PARAM DRM_IOWR(DRM_COMMAND_BASE +
DRM_MSM_GET_PARAM, structdrm_msm_param)include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 332
Quoted 1 line
include/uapi/drm/msm_drm.h +0
-0#define DRM_IOCTL_MSM_GEM_NEW DRM_IOWR(DRM_COMMAND_BASE +
DRM_MSM_GEM_NEW, structdrm_msm_gem_new)include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 339
Quoted 2 lines
include/uapi/drm/msm_drm.h +0
-0@@ -327,6 +339,7 @@ struct drm_msm_submitqueue_query {#define DRM_IOCTL_MSM_SUBMITQUEUE_NEW DRM_IOWR(DRM_COMMAND_BASE +
DRM_MSM_SUBMITQUEUE_NEW, structdrm_msm_submitqueue)include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 340
Quoted 1 line
include/uapi/drm/msm_drm.h +0
-0#define DRM_IOCTL_MSM_SUBMITQUEUE_CLOSE DRM_IOW (DRM_COMMAND_BASE +
DRM_MSM_SUBMITQUEUE_CLOSE,__u32)include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 341
Quoted 1 line
include/uapi/drm/msm_drm.h +0
-0#define DRM_IOCTL_MSM_SUBMITQUEUE_QUERY DRM_IOW (DRM_COMMAND_BASE +
DRM_MSM_SUBMITQUEUE_QUERY, structdrm_msm_submitqueue_query)Quoted 1 line
include/uapi/drm/msm_drm.h +1
-0+#define DRM_IOCTL_MSM_WAIT_IOVA DRM_IOW (DRM_COMMAND_BASE +Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
DRM_MSM_WAIT_IOVA, structdrm_msm_wait_iova)Quoted 6 lines
include/uapi/drm/msm_drm.h +0
-1#if defined(__cplusplus) } --2.25.0.rc1.283.g88dfdc4193-goog
_______________________________________________
Freedreno mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/freedreno
Re: [PATCH 2/2] drm/msm: Add MSM_WAIT_IOVA ioctl
In reply to Rob ClarkQuoted 188 lines · Rob Clark
Quoted 181 lines · Brian Ho
Quoted 86 lines · Kristian Kristensen
Quoted 74 lines · Rob Clark
Quoted 9 lines · Brian Ho
Implements an ioctl to wait until a value at a given iova is greater than or equal to a supplied value.
This will initially be used by turnip (open-source Vulkan driver for QC in mesa) for occlusion queries where the userspace driver can block on a query becoming available before continuing via vkGetQueryPoolResults.
Review
Quoted 7 lines
Signed-off-by Brian Ho <[email protected]>--- drivers/gpu/drm/msm/msm_drv.c | 63 +++++++++++++++++++++++++++++++++-- include/uapi/drm/msm_drm.h | 13 ++++++++ 2 files changed, 74 insertions(+), 2 deletions(-)(quoted diff) +0
-0
b/drivers/gpu/drm/msm/msm_drv.cdrivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct
Quoted 18 lines
index
c84f0a8b3f2c..dcc46874a5a2100644drivers/gpu/drm/msm/msm_drv.c +3
-2@@ -36,10 +36,11 @@* MSM_GEM_INFO ioctl. * - 1.4.0 - softpin, MSM_RELOC_BO_DUMP, and GEM_INFO support to set/get * GEM object's debug name -* - 1.5.0 - Add SUBMITQUERY_QUERY ioctl +* - 1.5.0 - Add SUBMITQUEUE_QUERY ioctl +* - 1.6.0 - Add WAIT_IOVA ioctl */ #define MSM_VERSION_MAJOR 1 -#define MSM_VERSION_MINOR 5 +#define MSM_VERSION_MINOR 6#define MSM_VERSION_PATCHLEVEL 0static const struct drm_mode_config_funcs mode_config_funcs = {@@ -952,6 +953,63 @@ static int msm_ioctl_submitqueue_close(struct
drm_device*dev, void *data,Quoted 11 lines
drivers/gpu/drm/msm/msm_drv.c +8
-0return msm_submitqueue_remove (file->driver_priv, id); } +static int msm_ioctl_wait_iova (struct drm_device *dev,void *data, +struct drm_file *file) +{ +struct msm_drm_private *priv = dev->dev_private; +struct drm_gem_object *obj; +struct drm_msm_wait_iova *args = data; +ktime_t timeout =to_ktime (args->timeout); +unsigned long remaining_jiffies =timeout_to_jiffies (&timeout);drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 973
Quoted 10 lines
drivers/gpu/drm/msm/msm_drv.c +10
-0+struct msm_gpu *gpu = priv->gpu; +void *base_vaddr; +uint64_t *vaddr; +int ret; + +if (args->pad) +return -EINVAL ; + +if (!gpu) +return 0 ;hmm, I'm not sure we should return zero in this case.. maybe
-ENODEV?Quoted 13 lines
drivers/gpu/drm/msm/msm_drv.c +13
-0+ + obj =drm_gem_object_lookup (file, args->handle); +if (!obj) +return -ENOENT ; + + base_vaddr =msm_gem_get_vaddr (obj); +if (IS_ERR (base_vaddr)) { + ret =PTR_ERR (base_vaddr); + goto err_put_gem_object; + } +if (args->offset +sizeof (*vaddr) > obj->size) { + ret = -EINVAL ; + goto err_put_vaddr;Quoted 9 lines
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data,
Quoted 7 lines
drivers/gpu/drm/msm/msm_drv.c +7
-0+ } + + vaddr = base_vaddr + args->offset; + +/* Assumes WC mapping */ + ret = wait_event_interruptible_timeout( + gpu->event, *vaddr >= args->value,
remaining_jiffies);Quoted 6 lines
This needs to do the awkward looking
(int64_t)(*data - value) >= 0
to properly handle the wraparound case.
I think this comparison will run into issues if we allow for 64-bit reference values. For example, if value is
ULLONG_MAX, and *data starts at 0 on the first comparison, we'll immediately return.It's not too much of an issue in
fence_completed(msm_fence.c), but in this ioctl, *data can grow at an arbitrary rate. Are we concerned about this?Quoted 70 lines
Quoted 64 lines
+
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 1000
Quoted 6 lines
drivers/gpu/drm/msm/msm_drv.c +6
-0+if (ret ==0 ) { + ret = -ETIMEDOUT ; + goto err_put_vaddr; + }else if (ret == -ERESTARTSYS ) { + goto err_put_vaddr; + }maybe:
} else { ret = 0;}
and then drop the next three lines?
Quoted 4 lines
drivers/gpu/drm/msm/msm_drv.c +4
-0+ +msm_gem_put_vaddr (obj); +drm_gem_object_put_unlocked (obj); +return 0 ;drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data,
Quoted 10 lines
drivers/gpu/drm/msm/msm_drv.c +8
-0+ +err_put_vaddr: +msm_gem_put_vaddr (obj); +err_put_gem_object: +drm_gem_object_put_unlocked (obj); +return ret; +} +static const struct drm_ioctl_desc msm_ioctls[] = { DRM_IOCTL_DEF_DRV(MSM_GET_PARAM, msm_ioctl_get_param,
DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct · line 1014
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +0
-0DRM_IOCTL_DEF_DRV(MSM_GEM_NEW, msm_ioctl_gem_new,
DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1022
Quoted 2 lines
drivers/gpu/drm/msm/msm_drv.c +0
-0@@ -964,6 +1022,7 @@ static const struct drm_ioctl_desc msm_ioctls[] = { DRM_IOCTL_DEF_DRV(MSM_SUBMITQUEUE_NEW,
msm_ioctl_submitqueue_new,DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1023
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +0
-0DRM_IOCTL_DEF_DRV(MSM_SUBMITQUEUE_CLOSE,
msm_ioctl_submitqueue_close,DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1024
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +0
-0DRM_IOCTL_DEF_DRV(MSM_SUBMITQUEUE_QUERY,
msm_ioctl_submitqueue_query,DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1025
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +1
-0+ DRM_IOCTL_DEF_DRV(MSM_WAIT_IOVA, msm_ioctl_wait_iova,
DRM_RENDER_ALLOW),include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 301
Quoted 12 lines
drivers/gpu/drm/msm/msm_drv.c +0
-0};static const struct vm_operations_struct vm_ops = {include/uapi/drm/msm_drm.h +1
-0@@ -298,6 +298,17 @@ struct drm_msm_submitqueue_query {__u32 pad; }; +/* This ioctl blocks until the u64 value at bo + offset is greater than or
include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 309
Quoted 8 lines
include/uapi/drm/msm_drm.h +8
-0+* equal to the reference value. +*/ +struct drm_msm_wait_iova { +__u32 handle;/* in, GEM handle */ +__u32 pad; +struct drm_msm_timespec timeout;/* in */ +__u64 offset;/* offset into bo */ +__u64 value;/* reference value */ Quoted 5 lines
Maybe we should go ahead and add a
__u64mask;that would let us wait for 32b values as well, and wait for bits in a bitmask
Quoted 7 lines
I think we'd be OK to just default to 32 bit values instead, since most of the CP commands that this is intended to work with (
CP_EVENT_WRITE,CP_WAIT_MEM_GTEetc) operate on 32 bit values. We could move 'value' to the slot right after 'handle' but then we'd not have any pad/reserved fields. Maybe we keep 'value' 64 bit but restrict it to 32 bits, with an option to add a 64 bit flag in 'pad' later on?Quoted 5 lines
FWIW, the current usage of this in my mesa MR uses a 64 bit value. There's no super great reason that the available bit is 64 bits and not 32 bits (I think it made the addressing math a bit simpler), but I'm fine with whatever you all decide on here.
Quoted 6 lines
I assume you are waiting for a fence value written w/
CP_EVENT_WRITE? Or at least that is what I'd recommend. That would be 32bBR,
-R
So it's actually a little bit different than that. I allocate a bo for the occlusion query which has an "availability" bit (0 for unavailable, 1 for available). When the occlusion query ends, we write the fragments passed value to the bo via CP_EVENT_WRITE and then wait for that write to complete before setting the available bit to 1 via a simple CP_MEM_WRITE [1].
It's that CP_MEM_WRITE that I plan on waiting on with this new iova ioctl.
Quoted 37 lines
Quoted 37 lines
Quoted 37 lines
Quoted 37 lines
Other than those minor comments, it looks pretty good to me
BR,
-Rinclude/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 331
Quoted 12 lines
include/uapi/drm/msm_drm.h +3
-0+}; +#define DRM_MSM_GET_PARAM 0x00/* placeholder: #define DRM_MSM_SET_PARAM 0x01 @@ -315,6 +326,7 @@ struct drm_msm_submitqueue_query {#define DRM_MSM_SUBMITQUEUE_NEW 0x0A#define DRM_MSM_SUBMITQUEUE_CLOSE 0x0B#define DRM_MSM_SUBMITQUEUE_QUERY 0x0C +#define DRM_MSM_WAIT_IOVA 0x0D#define DRM_IOCTL_MSM_GET_PARAM DRM_IOWR(DRM_COMMAND_BASE +
DRM_MSM_GET_PARAM, structdrm_msm_param)include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 332
Quoted 1 line
include/uapi/drm/msm_drm.h +0
-0#define DRM_IOCTL_MSM_GEM_NEW DRM_IOWR(DRM_COMMAND_BASE +
DRM_MSM_GEM_NEW, structdrm_msm_gem_new)include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 339
Quoted 2 lines
include/uapi/drm/msm_drm.h +0
-0@@ -327,6 +339,7 @@ struct drm_msm_submitqueue_query {#define DRM_IOCTL_MSM_SUBMITQUEUE_NEW DRM_IOWR(DRM_COMMAND_BASE +
DRM_MSM_SUBMITQUEUE_NEW, structdrm_msm_submitqueue)include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 340
Quoted 1 line
include/uapi/drm/msm_drm.h +0
-0#define DRM_IOCTL_MSM_SUBMITQUEUE_CLOSE DRM_IOW (DRM_COMMAND_BASE +
DRM_MSM_SUBMITQUEUE_CLOSE,__u32)include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 341
Quoted 1 line
include/uapi/drm/msm_drm.h +0
-0#define DRM_IOCTL_MSM_SUBMITQUEUE_QUERY DRM_IOW (DRM_COMMAND_BASE +
DRM_MSM_SUBMITQUEUE_QUERY, structdrm_msm_submitqueue_query)include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 342
Quoted 1 line
include/uapi/drm/msm_drm.h +1
-0+#define DRM_IOCTL_MSM_WAIT_IOVA DRM_IOW (DRM_COMMAND_BASE +
DRM_MSM_WAIT_IOVA, structdrm_msm_wait_iova)Quoted 6 lines
include/uapi/drm/msm_drm.h +0
-1#if defined(__cplusplus) } --2.25.0.rc1.283.g88dfdc4193-goog
Re: [PATCH 2/2] drm/msm: Add MSM_WAIT_IOVA ioctl
Quoted 204 lines · Brian Ho
Quoted 188 lines · Rob Clark
Quoted 181 lines · Brian Ho
Quoted 86 lines · Kristian Kristensen
Quoted 74 lines · Rob Clark
Quoted 9 lines · Brian Ho
Implements an ioctl to wait until a value at a given iova is greater than or equal to a supplied value.
This will initially be used by turnip (open-source Vulkan driver for QC in mesa) for occlusion queries where the userspace driver can block on a query becoming available before continuing via vkGetQueryPoolResults.
Review
Quoted 7 lines
Signed-off-by Brian Ho <[email protected]>--- drivers/gpu/drm/msm/msm_drv.c | 63 +++++++++++++++++++++++++++++++++-- include/uapi/drm/msm_drm.h | 13 ++++++++ 2 files changed, 74 insertions(+), 2 deletions(-)(quoted diff) +0
-0
b/drivers/gpu/drm/msm/msm_drv.cdrivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct
Quoted 18 lines
index
c84f0a8b3f2c..dcc46874a5a2100644drivers/gpu/drm/msm/msm_drv.c +3
-2@@ -36,10 +36,11 @@* MSM_GEM_INFO ioctl. * - 1.4.0 - softpin, MSM_RELOC_BO_DUMP, and GEM_INFO support to set/get * GEM object's debug name -* - 1.5.0 - Add SUBMITQUERY_QUERY ioctl +* - 1.5.0 - Add SUBMITQUEUE_QUERY ioctl +* - 1.6.0 - Add WAIT_IOVA ioctl */ #define MSM_VERSION_MAJOR 1 -#define MSM_VERSION_MINOR 5 +#define MSM_VERSION_MINOR 6#define MSM_VERSION_PATCHLEVEL 0static const struct drm_mode_config_funcs mode_config_funcs = {@@ -952,6 +953,63 @@ static int msm_ioctl_submitqueue_close(struct
drm_device*dev, void *data,Quoted 11 lines
drivers/gpu/drm/msm/msm_drv.c +8
-0return msm_submitqueue_remove (file->driver_priv, id); } +static int msm_ioctl_wait_iova (struct drm_device *dev,void *data, +struct drm_file *file) +{ +struct msm_drm_private *priv = dev->dev_private; +struct drm_gem_object *obj; +struct drm_msm_wait_iova *args = data; +ktime_t timeout =to_ktime (args->timeout); +unsigned long remaining_jiffies =timeout_to_jiffies (&timeout);drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 973
Quoted 10 lines
drivers/gpu/drm/msm/msm_drv.c +10
-0+struct msm_gpu *gpu = priv->gpu; +void *base_vaddr; +uint64_t *vaddr; +int ret; + +if (args->pad) +return -EINVAL ; + +if (!gpu) +return 0 ;hmm, I'm not sure we should return zero in this case.. maybe
-ENODEV?Quoted 13 lines
drivers/gpu/drm/msm/msm_drv.c +13
-0+ + obj =drm_gem_object_lookup (file, args->handle); +if (!obj) +return -ENOENT ; + + base_vaddr =msm_gem_get_vaddr (obj); +if (IS_ERR (base_vaddr)) { + ret =PTR_ERR (base_vaddr); + goto err_put_gem_object; + } +if (args->offset +sizeof (*vaddr) > obj->size) { + ret = -EINVAL ; + goto err_put_vaddr;Quoted 9 lines
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data,
Quoted 7 lines
drivers/gpu/drm/msm/msm_drv.c +7
-0+ } + + vaddr = base_vaddr + args->offset; + +/* Assumes WC mapping */ + ret = wait_event_interruptible_timeout( + gpu->event, *vaddr >= args->value,
remaining_jiffies);Quoted 6 lines
This needs to do the awkward looking
(int64_t)(*data - value) >= 0
to properly handle the wraparound case.
I think this comparison will run into issues if we allow for 64-bit reference values. For example, if value is
ULLONG_MAX, and *data starts at 0 on the first comparison, we'll immediately return.It's not too much of an issue in
fence_completed(msm_fence.c), but in this ioctl, *data can grow at an arbitrary rate. Are we concerned about this?Quoted 70 lines
Quoted 64 lines
+
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 1000
Quoted 6 lines
drivers/gpu/drm/msm/msm_drv.c +6
-0+if (ret ==0 ) { + ret = -ETIMEDOUT ; + goto err_put_vaddr; + }else if (ret == -ERESTARTSYS ) { + goto err_put_vaddr; + }maybe:
} else { ret = 0;}
and then drop the next three lines?
Quoted 4 lines
drivers/gpu/drm/msm/msm_drv.c +4
-0+ +msm_gem_put_vaddr (obj); +drm_gem_object_put_unlocked (obj); +return 0 ;drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data,
Quoted 10 lines
drivers/gpu/drm/msm/msm_drv.c +8
-0+ +err_put_vaddr: +msm_gem_put_vaddr (obj); +err_put_gem_object: +drm_gem_object_put_unlocked (obj); +return ret; +} +static const struct drm_ioctl_desc msm_ioctls[] = { DRM_IOCTL_DEF_DRV(MSM_GET_PARAM, msm_ioctl_get_param,
DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct · line 1014
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +0
-0DRM_IOCTL_DEF_DRV(MSM_GEM_NEW, msm_ioctl_gem_new,
DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1022
Quoted 2 lines
drivers/gpu/drm/msm/msm_drv.c +0
-0@@ -964,6 +1022,7 @@ static const struct drm_ioctl_desc msm_ioctls[] = { DRM_IOCTL_DEF_DRV(MSM_SUBMITQUEUE_NEW,
msm_ioctl_submitqueue_new,DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1023
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +0
-0DRM_IOCTL_DEF_DRV(MSM_SUBMITQUEUE_CLOSE,
msm_ioctl_submitqueue_close,DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1024
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +0
-0DRM_IOCTL_DEF_DRV(MSM_SUBMITQUEUE_QUERY,
msm_ioctl_submitqueue_query,DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1025
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +1
-0+ DRM_IOCTL_DEF_DRV(MSM_WAIT_IOVA, msm_ioctl_wait_iova,
DRM_RENDER_ALLOW),include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 301
Quoted 12 lines
drivers/gpu/drm/msm/msm_drv.c +0
-0};static const struct vm_operations_struct vm_ops = {include/uapi/drm/msm_drm.h +1
-0@@ -298,6 +298,17 @@ struct drm_msm_submitqueue_query {__u32 pad; }; +/* This ioctl blocks until the u64 value at bo + offset is greater than or
include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 309
Quoted 8 lines
include/uapi/drm/msm_drm.h +8
-0+* equal to the reference value. +*/ +struct drm_msm_wait_iova { +__u32 handle;/* in, GEM handle */ +__u32 pad; +struct drm_msm_timespec timeout;/* in */ +__u64 offset;/* offset into bo */ +__u64 value;/* reference value */ Quoted 5 lines
Maybe we should go ahead and add a
__u64mask;that would let us wait for 32b values as well, and wait for bits in a bitmask
Quoted 7 lines
I think we'd be OK to just default to 32 bit values instead, since most of the CP commands that this is intended to work with (
CP_EVENT_WRITE,CP_WAIT_MEM_GTEetc) operate on 32 bit values. We could move 'value' to the slot right after 'handle' but then we'd not have any pad/reserved fields. Maybe we keep 'value' 64 bit but restrict it to 32 bits, with an option to add a 64 bit flag in 'pad' later on?Quoted 5 lines
FWIW, the current usage of this in my mesa MR uses a 64 bit value. There's no super great reason that the available bit is 64 bits and not 32 bits (I think it made the addressing math a bit simpler), but I'm fine with whatever you all decide on here.
Quoted 6 lines
I assume you are waiting for a fence value written w/
CP_EVENT_WRITE? Or at least that is what I'd recommend. That would be 32bBR,
-RSo it's actually a little bit different than that. I allocate a bo for the occlusion query which has an "availability" bit (0 for unavailable, 1 for available). When the occlusion query ends, we write the fragments passed value to the bo via
CP_EVENT_WRITEand then wait for that write to complete before setting the available bit to 1 via a simpleCP_MEM_WRITE[1].
Quoted 5 lines
It's that
CP_MEM_WRITEthat I plan on waiting on with this new iova ioctl.
hmm, interesting.. I had in mind something more like:
https://gitlab.freedesktop.org/drm/msm/blob/msm-next/drivers/gpu/drm/msm/adreno/a6xx_gpu.c#L137
The high bit in the first dword of the packet (which we probably shouldn't open-code) is the "give me an irq after the value in last dword is written to memory"..
(I haven't checked yet whether we can use the "gimme an irq" bit from userspace)
BR,
-R
Re: [PATCH 2/2] drm/msm: Add MSM_WAIT_IOVA ioctl
Quoted 215 lines · Rob Clark
Quoted 204 lines · Brian Ho
Quoted 188 lines · Rob Clark
Quoted 181 lines · Brian Ho
Quoted 86 lines · Kristian Kristensen
Quoted 74 lines · Rob Clark
Quoted 9 lines · Brian Ho
Implements an ioctl to wait until a value at a given iova is greater than or equal to a supplied value.
This will initially be used by turnip (open-source Vulkan driver for QC in mesa) for occlusion queries where the userspace driver can block on a query becoming available before continuing via vkGetQueryPoolResults.
Review
Quoted 7 lines
Signed-off-by Brian Ho <[email protected]>--- drivers/gpu/drm/msm/msm_drv.c | 63 +++++++++++++++++++++++++++++++++-- include/uapi/drm/msm_drm.h | 13 ++++++++ 2 files changed, 74 insertions(+), 2 deletions(-)(quoted diff) +0
-0
b/drivers/gpu/drm/msm/msm_drv.cdrivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct
Quoted 18 lines
index
c84f0a8b3f2c..dcc46874a5a2100644drivers/gpu/drm/msm/msm_drv.c +3
-2@@ -36,10 +36,11 @@* MSM_GEM_INFO ioctl. * - 1.4.0 - softpin, MSM_RELOC_BO_DUMP, and GEM_INFO support to set/get * GEM object's debug name -* - 1.5.0 - Add SUBMITQUERY_QUERY ioctl +* - 1.5.0 - Add SUBMITQUEUE_QUERY ioctl +* - 1.6.0 - Add WAIT_IOVA ioctl */ #define MSM_VERSION_MAJOR 1 -#define MSM_VERSION_MINOR 5 +#define MSM_VERSION_MINOR 6#define MSM_VERSION_PATCHLEVEL 0static const struct drm_mode_config_funcs mode_config_funcs = {@@ -952,6 +953,63 @@ static int msm_ioctl_submitqueue_close(struct
drm_device*dev, void *data,Quoted 11 lines
drivers/gpu/drm/msm/msm_drv.c +8
-0return msm_submitqueue_remove (file->driver_priv, id); } +static int msm_ioctl_wait_iova (struct drm_device *dev,void *data, +struct drm_file *file) +{ +struct msm_drm_private *priv = dev->dev_private; +struct drm_gem_object *obj; +struct drm_msm_wait_iova *args = data; +ktime_t timeout =to_ktime (args->timeout); +unsigned long remaining_jiffies =timeout_to_jiffies (&timeout);drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct · line 973
Quoted 10 lines
drivers/gpu/drm/msm/msm_drv.c +10
-0+struct msm_gpu *gpu = priv->gpu; +void *base_vaddr; +uint64_t *vaddr; +int ret; + +if (args->pad) +return -EINVAL ; + +if (!gpu) +return 0 ;hmm, I'm not sure we should return zero in this case.. maybe
-ENODEV?Quoted 13 lines
drivers/gpu/drm/msm/msm_drv.c +13
-0+ + obj =drm_gem_object_lookup (file, args->handle); +if (!obj) +return -ENOENT ; + + base_vaddr =msm_gem_get_vaddr (obj); +if (IS_ERR (base_vaddr)) { + ret =PTR_ERR (base_vaddr); + goto err_put_gem_object; + } +if (args->offset +sizeof (*vaddr) > obj->size) { + ret = -EINVAL ; + goto err_put_vaddr;Quoted 9 lines
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct · line 993
Quoted 7 lines
drivers/gpu/drm/msm/msm_drv.c +7
-0+ } + + vaddr = base_vaddr + args->offset; + +/* Assumes WC mapping */ + ret = wait_event_interruptible_timeout( + gpu->event, *vaddr >= args->value,
remaining_jiffies);Quoted 6 lines
This needs to do the awkward looking
(int64_t)(*data - value) >= 0
to properly handle the wraparound case.
I think this comparison will run into issues if we allow for 64-bit reference values. For example, if value is
ULLONG_MAX, and *data starts at 0 on the first comparison, we'll immediately return.It's not too much of an issue in
fence_completed(msm_fence.c), but in this ioctl, *data can grow at an arbitrary rate. Are we concerned about this?Quoted 70 lines
Quoted 64 lines
+
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct · line 999
Quoted 6 lines
drivers/gpu/drm/msm/msm_drv.c +6
-0+if (ret ==0 ) { + ret = -ETIMEDOUT ; + goto err_put_vaddr; + }else if (ret == -ERESTARTSYS ) { + goto err_put_vaddr; + }maybe:
} else { ret = 0;}
and then drop the next three lines?
Quoted 4 lines
drivers/gpu/drm/msm/msm_drv.c +4
-0+ +msm_gem_put_vaddr (obj); +drm_gem_object_put_unlocked (obj); +return 0 ;drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct · line 1013
Quoted 10 lines
drivers/gpu/drm/msm/msm_drv.c +8
-0+ +err_put_vaddr: +msm_gem_put_vaddr (obj); +err_put_gem_object: +drm_gem_object_put_unlocked (obj); +return ret; +} +static const struct drm_ioctl_desc msm_ioctls[] = { DRM_IOCTL_DEF_DRV(MSM_GET_PARAM, msm_ioctl_get_param,
DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct · line 1014
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +0
-0DRM_IOCTL_DEF_DRV(MSM_GEM_NEW, msm_ioctl_gem_new,
DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1022
Quoted 2 lines
drivers/gpu/drm/msm/msm_drv.c +0
-0@@ -964,6 +1022,7 @@ static const struct drm_ioctl_desc msm_ioctls[] = { DRM_IOCTL_DEF_DRV(MSM_SUBMITQUEUE_NEW,
msm_ioctl_submitqueue_new,DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1023
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +0
-0DRM_IOCTL_DEF_DRV(MSM_SUBMITQUEUE_CLOSE,
msm_ioctl_submitqueue_close,DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1024
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +0
-0DRM_IOCTL_DEF_DRV(MSM_SUBMITQUEUE_QUERY,
msm_ioctl_submitqueue_query,DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c static const struct drm_ioctl_desc msm_ioctls[] = { · line 1025
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +1
-0+ DRM_IOCTL_DEF_DRV(MSM_WAIT_IOVA, msm_ioctl_wait_iova,
DRM_RENDER_ALLOW),include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 301
Quoted 12 lines
drivers/gpu/drm/msm/msm_drv.c +0
-0};static const struct vm_operations_struct vm_ops = {include/uapi/drm/msm_drm.h +1
-0@@ -298,6 +298,17 @@ struct drm_msm_submitqueue_query {__u32 pad; }; +/* This ioctl blocks until the u64 value at bo + offset is greater than or
include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 309
Quoted 8 lines
include/uapi/drm/msm_drm.h +8
-0+* equal to the reference value. +*/ +struct drm_msm_wait_iova { +__u32 handle;/* in, GEM handle */ +__u32 pad; +struct drm_msm_timespec timeout;/* in */ +__u64 offset;/* offset into bo */ +__u64 value;/* reference value */ Quoted 5 lines
Maybe we should go ahead and add a
__u64mask;that would let us wait for 32b values as well, and wait for bits in a bitmask
Quoted 7 lines
I think we'd be OK to just default to 32 bit values instead, since most of the CP commands that this is intended to work with (
CP_EVENT_WRITE,CP_WAIT_MEM_GTEetc) operate on 32 bit values. We could move 'value' to the slot right after 'handle' but then we'd not have any pad/reserved fields. Maybe we keep 'value' 64 bit but restrict it to 32 bits, with an option to add a 64 bit flag in 'pad' later on?Quoted 5 lines
FWIW, the current usage of this in my mesa MR uses a 64 bit value. There's no super great reason that the available bit is 64 bits and not 32 bits (I think it made the addressing math a bit simpler), but I'm fine with whatever you all decide on here.
Quoted 6 lines
I assume you are waiting for a fence value written w/
CP_EVENT_WRITE? Or at least that is what I'd recommend. That would be 32bBR,
-RSo it's actually a little bit different than that. I allocate a bo for the occlusion query which has an "availability" bit (0 for unavailable, 1 for available). When the occlusion query ends, we write the fragments passed value to the bo via
CP_EVENT_WRITEand then wait for that write to complete before setting the available bit to 1 via a simpleCP_MEM_WRITE[1].Quoted 5 lines
It's that
CP_MEM_WRITEthat I plan on waiting on with this new iova ioctl.hmm, interesting.. I had in mind something more like:
https://gitlab.freedesktop.org/drm/msm/blob/msm-next/drivers/gpu/drm/msm/adreno/a6xx_gpu.c#L137
Quoted 8 lines
The high bit in the first dword of the packet (which we probably shouldn't open-code) is the "give me an irq after the value in last dword is written to memory"..
(I haven't checked yet whether we can use the "gimme an irq" bit from userspace)
BR,
-R
I see. Let's continue discussing this on the mesa MR because there's more context there.
Regardless of how we end up implementing vkCmdEndQuery, I think it will be safe to default to 32 bit values for this ioctl and have a flag to use 64 bit instead like Kristian suggested. If that sounds good to you, I'll go ahead and make the change for the v2 patch series.
Re: [Freedreno] [PATCH 2/2] drm/msm: Add MSM_WAIT_IOVA ioctl
In reply to Brian HoQuoted 44 lines · Brian Ho
Implements an ioctl to wait until a value at a given iova is greater than or equal to a supplied value.
This will initially be used by turnip (open-source Vulkan driver for QC in mesa) for occlusion queries where the userspace driver can block on a query becoming available before continuing via vkGetQueryPoolResults.
Signed-off-by Brian Ho <[email protected]>--- drivers/gpu/drm/msm/msm_drv.c | 63 +++++++++++++++++++++++++++++++++-- include/uapi/drm/msm_drm.h | 13 ++++++++ 2 files changed, 74 insertions(+), 2 deletions(-)drivers/gpu/drm/msm/msm_drv.c +11
-2@@ -36,10 +36,11 @@* MSM_GEM_INFO ioctl. * - 1.4.0 - softpin, MSM_RELOC_BO_DUMP, and GEM_INFO support to set/get * GEM object's debug name -* - 1.5.0 - Add SUBMITQUERY_QUERY ioctl +* - 1.5.0 - Add SUBMITQUEUE_QUERY ioctl +* - 1.6.0 - Add WAIT_IOVA ioctl */ #define MSM_VERSION_MAJOR 1 -#define MSM_VERSION_MINOR 5 +#define MSM_VERSION_MINOR 6#define MSM_VERSION_PATCHLEVEL 0static const struct drm_mode_config_funcs mode_config_funcs = {@@ -952,6 +953,63 @@ static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data,return msm_submitqueue_remove (file->driver_priv, id); } +static int msm_ioctl_wait_iova (struct drm_device *dev,void *data, +struct drm_file *file) +{ +struct msm_drm_private *priv = dev->dev_private; +struct drm_gem_object *obj; +struct drm_msm_wait_iova *args = data; +ktime_t timeout =to_ktime (args->timeout); +unsigned long remaining_jiffies =timeout_to_jiffies (&timeout);
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 973
Quoted 10 lines
drivers/gpu/drm/msm/msm_drv.c +10
-0+struct msm_gpu *gpu = priv->gpu; +void *base_vaddr; +uint64_t *vaddr; +int ret; + +if (args->pad) +return -EINVAL ; + +if (!gpu) +return 0 ;
If the GPU isn't up, it should be an error since this macro is specifically designed for just the GPU (though, I suppose the display *COULD* use it to watch a memory mapped register or something).
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 984
Quoted 11 lines
drivers/gpu/drm/msm/msm_drv.c +11
-0+ + obj =drm_gem_object_lookup (file, args->handle); +if (!obj) +return -ENOENT ; + + base_vaddr =msm_gem_get_vaddr (obj); +if (IS_ERR (base_vaddr)) { + ret =PTR_ERR (base_vaddr); + goto err_put_gem_object; + } +if (args->offset +sizeof (*vaddr) > obj->size) {
There is a chance to trigger a u64 overflow here resulting in an arbitrary (ish) vaddr two lines below.
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 987
Quoted 3 lines
drivers/gpu/drm/msm/msm_drv.c +3
-0+ ret = -EINVAL ; + goto err_put_vaddr; + }
You can check this before getting the vaddr which would save you a clean up step.
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 993
Quoted 6 lines
drivers/gpu/drm/msm/msm_drv.c +6
-0+ + vaddr = base_vaddr + args->offset; + +/* Assumes WC mapping */ + ret =wait_event_interruptible_timeout ( + gpu->event, *vaddr >= args->value, remaining_jiffies);
I feel like a barrier might be needed before checking *vaddr just in case you get the interrupt and wake up the queue before the write posts from the hardware.
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 1000
Quoted 8 lines
drivers/gpu/drm/msm/msm_drv.c +7
-0+ +if (ret ==0 ) { + ret = -ETIMEDOUT ; + goto err_put_vaddr; + }else if (ret == -ERESTARTSYS ) { + goto err_put_vaddr; + }
You don't need either goto here because both paths execute the following cleanup steps. I'm also not sure you need to worry about explicitly checking the ERESTARTSYS value, I think that this would be sufficient:
if (ret == 0)
ret = -ETIMEDOUT;
else if (ret > 0)
ret = 0;drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 1002
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +1
-0+
Put your err_put_vaddr: label here, but looking up, if you move the bounds check before the msm_gem_get_vaddr, I don't think you need a label.
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 1003
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +1
-0+msm_gem_put_vaddr (obj);
Put the err_put_gem_object: label here.
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 1004
Quoted 2 lines
drivers/gpu/drm/msm/msm_drv.c +2
-0+drm_gem_object_put_unlocked (obj); +return 0 ;
return ret;
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 1011
Quoted 7 lines
drivers/gpu/drm/msm/msm_drv.c +7
-0+ +err_put_vaddr: +msm_gem_put_vaddr (obj); +err_put_gem_object: +drm_gem_object_put_unlocked (obj); +return ret; +}
And then these guys aren't needed.
Quoted 19 lines
drivers/gpu/drm/msm/msm_drv.c +1
-0+static const struct drm_ioctl_desc msm_ioctls[] = {DRM_IOCTL_DEF_DRV (MSM_GET_PARAM , msm_ioctl_get_param,DRM_RENDER_ALLOW ),drivers/gpu/drm/msm/msm_drv.c +1
-0DRM_IOCTL_DEF_DRV(MSM_GEM_NEW, msm_ioctl_gem_new, DRM_RENDER_ALLOW),@@ -964,6 +1022,7 @@ static const struct drm_ioctl_desc msm_ioctls[] = {DRM_IOCTL_DEF_DRV (MSM_SUBMITQUEUE_NEW , msm_ioctl_submitqueue_new,DRM_RENDER_ALLOW ),DRM_IOCTL_DEF_DRV (MSM_SUBMITQUEUE_CLOSE , msm_ioctl_submitqueue_close,DRM_RENDER_ALLOW ),DRM_IOCTL_DEF_DRV (MSM_SUBMITQUEUE_QUERY , msm_ioctl_submitqueue_query,DRM_RENDER_ALLOW ), + DRM_IOCTL_DEF_DRV(MSM_WAIT_IOVA, msm_ioctl_wait_iova, DRM_RENDER_ALLOW), }; static const struct vm_operations_struct vm_ops = {include/uapi/drm/msm_drm.h +0
-0@@ -298,6 +298,17 @@ struct drm_msm_submitqueue_query {__u32 pad; };
include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 309
Quoted 10 lines
include/uapi/drm/msm_drm.h +9
-0+/* This ioctl blocks until the u64 value at bo + offset is greater than or +* equal to the reference value. +*/ +struct drm_msm_wait_iova { +__u32 handle;/* in, GEM handle */ +__u32 pad; +struct drm_msm_timespec timeout;/* in */ +__u64 offset;/* offset into bo */ +__u64 value;/* reference value */
Any specific reason why we wouldn't just put the offset and value first and save ourselves the padding?
Quoted 28 lines
include/uapi/drm/msm_drm.h +4
-0+}; +#define DRM_MSM_GET_PARAM 0x00/* placeholder: #define DRM_MSM_SET_PARAM 0x01 @@ -315,6 +326,7 @@ struct drm_msm_submitqueue_query {#define DRM_MSM_SUBMITQUEUE_NEW 0x0A#define DRM_MSM_SUBMITQUEUE_CLOSE 0x0B#define DRM_MSM_SUBMITQUEUE_QUERY 0x0C +#define DRM_MSM_WAIT_IOVA 0x0D#define DRM_IOCTL_MSM_GET_PARAM DRM_IOWR(DRM_COMMAND_BASE + DRM_MSM_GET_PARAM, struct drm_msm_param)#define DRM_IOCTL_MSM_GEM_NEW DRM_IOWR(DRM_COMMAND_BASE + DRM_MSM_GEM_NEW, struct drm_msm_gem_new)@@ -327,6 +339,7 @@ struct drm_msm_submitqueue_query {#define DRM_IOCTL_MSM_SUBMITQUEUE_NEW DRM_IOWR(DRM_COMMAND_BASE + DRM_MSM_SUBMITQUEUE_NEW, struct drm_msm_submitqueue)#define DRM_IOCTL_MSM_SUBMITQUEUE_CLOSE DRM_IOW (DRM_COMMAND_BASE + DRM_MSM_SUBMITQUEUE_CLOSE, __u32)#define DRM_IOCTL_MSM_SUBMITQUEUE_QUERY DRM_IOW (DRM_COMMAND_BASE + DRM_MSM_SUBMITQUEUE_QUERY, struct drm_msm_submitqueue_query) +#define DRM_IOCTL_MSM_WAIT_IOVA DRM_IOW (DRM_COMMAND_BASE + DRM_MSM_WAIT_IOVA, struct drm_msm_wait_iova)#if defined(__cplusplus) }Signature · 6 lines
2.25.0.rc1.283.g88dfdc4193-goog _______________________________________________ Freedreno mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/freedreno
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
Re: [Freedreno] [PATCH 2/2] drm/msm: Add MSM_WAIT_IOVA ioctl
Quoted 157 lines · Jordan Crouse
Quoted 44 lines · Brian Ho
Implements an ioctl to wait until a value at a given iova is greater than or equal to a supplied value.
This will initially be used by turnip (open-source Vulkan driver for QC in mesa) for occlusion queries where the userspace driver can block on a query becoming available before continuing via vkGetQueryPoolResults.
Signed-off-by Brian Ho <[email protected]>--- drivers/gpu/drm/msm/msm_drv.c | 63 +++++++++++++++++++++++++++++++++-- include/uapi/drm/msm_drm.h | 13 ++++++++ 2 files changed, 74 insertions(+), 2 deletions(-)drivers/gpu/drm/msm/msm_drv.c +11
-2@@ -36,10 +36,11 @@* MSM_GEM_INFO ioctl. * - 1.4.0 - softpin, MSM_RELOC_BO_DUMP, and GEM_INFO support to set/get * GEM object's debug name -* - 1.5.0 - Add SUBMITQUERY_QUERY ioctl +* - 1.5.0 - Add SUBMITQUEUE_QUERY ioctl +* - 1.6.0 - Add WAIT_IOVA ioctl */ #define MSM_VERSION_MAJOR 1 -#define MSM_VERSION_MINOR 5 +#define MSM_VERSION_MINOR 6#define MSM_VERSION_PATCHLEVEL 0static const struct drm_mode_config_funcs mode_config_funcs = {@@ -952,6 +953,63 @@ static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data,return msm_submitqueue_remove (file->driver_priv, id); } +static int msm_ioctl_wait_iova (struct drm_device *dev,void *data, +struct drm_file *file) +{ +struct msm_drm_private *priv = dev->dev_private; +struct drm_gem_object *obj; +struct drm_msm_wait_iova *args = data; +ktime_t timeout =to_ktime (args->timeout); +unsigned long remaining_jiffies =timeout_to_jiffies (&timeout);drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 973
Quoted 10 lines
drivers/gpu/drm/msm/msm_drv.c +10
-0+struct msm_gpu *gpu = priv->gpu; +void *base_vaddr; +uint64_t *vaddr; +int ret; + +if (args->pad) +return -EINVAL ; + +if (!gpu) +return 0 ;If the GPU isn't up, it should be an error since this macro is specifically designed for just the GPU (though, I suppose the display *COULD* use it to watch a memory mapped register or something).
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 984
Quoted 11 lines
drivers/gpu/drm/msm/msm_drv.c +11
-0+ + obj =drm_gem_object_lookup (file, args->handle); +if (!obj) +return -ENOENT ; + + base_vaddr =msm_gem_get_vaddr (obj); +if (IS_ERR (base_vaddr)) { + ret =PTR_ERR (base_vaddr); + goto err_put_gem_object; + } +if (args->offset +sizeof (*vaddr) > obj->size) {There is a chance to trigger a u64 overflow here resulting in an arbitrary (ish) vaddr two lines below.
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 987
Quoted 3 lines
drivers/gpu/drm/msm/msm_drv.c +3
-0+ ret = -EINVAL ; + goto err_put_vaddr; + }You can check this before getting the vaddr which would save you a clean up step.
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 993
Quoted 6 lines
drivers/gpu/drm/msm/msm_drv.c +6
-0+ + vaddr = base_vaddr + args->offset; + +/* Assumes WC mapping */ + ret =wait_event_interruptible_timeout ( + gpu->event, *vaddr >= args->value, remaining_jiffies);I feel like a barrier might be needed before checking *vaddr just in case you get the interrupt and wake up the queue before the write posts from the hardware.
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 1000
Quoted 8 lines
drivers/gpu/drm/msm/msm_drv.c +7
-0+ +if (ret ==0 ) { + ret = -ETIMEDOUT ; + goto err_put_vaddr; + }else if (ret == -ERESTARTSYS ) { + goto err_put_vaddr; + }You don't need either goto here because both paths execute the following cleanup steps. I'm also not sure you need to worry about explicitly checking the ERESTARTSYS value, I think that this would be sufficient:
if (ret == 0)
ret = -ETIMEDOUT; else if (ret > 0) ret = 0;drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 1002
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +1
-0+Put your
err_put_vaddr: label here, but looking up, if you move the bounds check before themsm_gem_get_vaddr, I don't think you need a label.drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 1003
Quoted 1 line
drivers/gpu/drm/msm/msm_drv.c +1
-0+msm_gem_put_vaddr (obj);Put the
err_put_gem_object: label here.drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 1004
Quoted 2 lines
drivers/gpu/drm/msm/msm_drv.c +2
-0+drm_gem_object_put_unlocked (obj); +return 0 ;return ret;
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 1011
Quoted 7 lines
drivers/gpu/drm/msm/msm_drv.c +7
-0+ +err_put_vaddr: +msm_gem_put_vaddr (obj); +err_put_gem_object: +drm_gem_object_put_unlocked (obj); +return ret; +}And then these guys aren't needed.
Quoted 20 lines
drivers/gpu/drm/msm/msm_drv.c +1
-0+static const struct drm_ioctl_desc msm_ioctls[] = {DRM_IOCTL_DEF_DRV (MSM_GET_PARAM , msm_ioctl_get_param,DRM_RENDER_ALLOW ),
DRM_IOCTL_DEF_DRV(MSM_GEM_NEW, msm_ioctl_gem_new, DRM_RENDER_ALLOW),drivers/gpu/drm/msm/msm_drv.c +1
-0@@ -964,6 +1022,7 @@ static const struct drm_ioctl_desc msm_ioctls[] = {DRM_IOCTL_DEF_DRV (MSM_SUBMITQUEUE_NEW , msm_ioctl_submitqueue_new,DRM_RENDER_ALLOW ),DRM_IOCTL_DEF_DRV (MSM_SUBMITQUEUE_CLOSE , msm_ioctl_submitqueue_close,DRM_RENDER_ALLOW ),DRM_IOCTL_DEF_DRV (MSM_SUBMITQUEUE_QUERY , msm_ioctl_submitqueue_query,DRM_RENDER_ALLOW ), + DRM_IOCTL_DEF_DRV(MSM_WAIT_IOVA, msm_ioctl_wait_iova, DRM_RENDER_ALLOW), }; static const struct vm_operations_struct vm_ops = {include/uapi/drm/msm_drm.h +0
-0@@ -298,6 +298,17 @@ struct drm_msm_submitqueue_query {__u32 pad; };
Quoted 12 lines
include/uapi/drm/msm_drm.h struct drm_msm_submitqueue_query { · line 309
Quoted 9 lines
include/uapi/drm/msm_drm.h +9
-0+/* This ioctl blocks until the u64 value at bo + offset is greater than or +* equal to the reference value. +*/ +struct drm_msm_wait_iova { +__u32 handle;/* in, GEM handle */ +__u32 pad; +struct drm_msm_timespec timeout;/* in */ +__u64 offset;/* offset into bo */ +__u64 value;/* reference value */ Any specific reason why we wouldn't just put the offset and value first and save ourselves the padding?
I'd actually like to have a pad must-be-zero flag to make this easier to extend in the future, if needed.
(Maybe there is a use-case for a wait_iova that is scoped to a specific submit-queue, or we want to add other types of tests beyond GTE, etc)
BR,
-R
Quoted 32 lines
Quoted 24 lines
include/uapi/drm/msm_drm.h +4
-1+}; +#define DRM_MSM_GET_PARAM 0x00/* placeholder: #define DRM_MSM_SET_PARAM 0x01 @@ -315,6 +326,7 @@ struct drm_msm_submitqueue_query {#define DRM_MSM_SUBMITQUEUE_NEW 0x0A#define DRM_MSM_SUBMITQUEUE_CLOSE 0x0B#define DRM_MSM_SUBMITQUEUE_QUERY 0x0C +#define DRM_MSM_WAIT_IOVA 0x0D#define DRM_IOCTL_MSM_GET_PARAM DRM_IOWR(DRM_COMMAND_BASE + DRM_MSM_GET_PARAM, struct drm_msm_param)#define DRM_IOCTL_MSM_GEM_NEW DRM_IOWR(DRM_COMMAND_BASE + DRM_MSM_GEM_NEW, struct drm_msm_gem_new)@@ -327,6 +339,7 @@ struct drm_msm_submitqueue_query {#define DRM_IOCTL_MSM_SUBMITQUEUE_NEW DRM_IOWR(DRM_COMMAND_BASE + DRM_MSM_SUBMITQUEUE_NEW, struct drm_msm_submitqueue)#define DRM_IOCTL_MSM_SUBMITQUEUE_CLOSE DRM_IOW (DRM_COMMAND_BASE + DRM_MSM_SUBMITQUEUE_CLOSE, __u32)#define DRM_IOCTL_MSM_SUBMITQUEUE_QUERY DRM_IOW (DRM_COMMAND_BASE + DRM_MSM_SUBMITQUEUE_QUERY, struct drm_msm_submitqueue_query) +#define DRM_IOCTL_MSM_WAIT_IOVA DRM_IOW (DRM_COMMAND_BASE + DRM_MSM_WAIT_IOVA, struct drm_msm_wait_iova)#if defined(__cplusplus) } --2.25.0.rc1.283.g88dfdc4193-goog
Quoted 4 lines
_______________________________________________
Freedreno mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/freedreno--
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
Re: [Freedreno] [PATCH 2/2] drm/msm: Add MSM_WAIT_IOVA ioctl
In reply to Jordan CrouseQuoted 13 lines · Jordan Crouse
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 993
Quoted 6 lines · Brian Ho
drivers/gpu/drm/msm/msm_drv.c +6
-0+ + vaddr = base_vaddr + args->offset; + +/* Assumes WC mapping */ + ret =wait_event_interruptible_timeout ( + gpu->event, *vaddr >= args->value, remaining_jiffies);I feel like a barrier might be needed before checking *vaddr just in case you get the interrupt and wake up the queue before the write posts from the hardware.
if the gpu is doing posted (or cached) writes, I don't think there is even a CPU side barrier primitive that could wait for that? I think we rely on the GPU not interrupting the CPU until the write is posted
BR,
-R
Re: [Freedreno] [PATCH 2/2] drm/msm: Add MSM_WAIT_IOVA ioctl
Quoted 15 lines · Rob Clark
Quoted 13 lines · Jordan Crouse
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 993
Quoted 6 lines · Brian Ho
drivers/gpu/drm/msm/msm_drv.c +6
-0+ + vaddr = base_vaddr + args->offset; + +/* Assumes WC mapping */ + ret =wait_event_interruptible_timeout ( + gpu->event, *vaddr >= args->value, remaining_jiffies);I feel like a barrier might be needed before checking *vaddr just in case you get the interrupt and wake up the queue before the write posts from the hardware.
Quoted 3 lines
if the gpu is doing posted (or cached) writes, I don't think there is even a CPU side barrier primitive that could wait for that? I think we rely on the GPU not interrupting the CPU until the write is posted
Once the GPU puts the write on the bus then it is up to the whims of the CPU architecture. If the writes are being done out of order you run a chance of firing the interrupt and making it all the way to your handler before the writes catch up.
Since you are scheduling and doing a bunch of things in between you probably don't need to worry but if you start missing events and you don't know why then this could be why. A rmb() would give you piece of mind at the cost of being Yet Another Barrier (TM).
Jordan
Quoted 6 lines
BR,
-R
_______________________________________________
Freedreno mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/freedreno
Signature · 6 lines
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [Freedreno] [PATCH 2/2] drm/msm: Add MSM_WAIT_IOVA ioctl
Quoted 26 lines · Jordan Crouse
Quoted 15 lines · Rob Clark
Quoted 13 lines · Jordan Crouse
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 993
Quoted 6 lines · Brian Ho
drivers/gpu/drm/msm/msm_drv.c +6
-0+ + vaddr = base_vaddr + args->offset; + +/* Assumes WC mapping */ + ret =wait_event_interruptible_timeout ( + gpu->event, *vaddr >= args->value, remaining_jiffies);I feel like a barrier might be needed before checking *vaddr just in case you get the interrupt and wake up the queue before the write posts from the hardware.
Quoted 3 lines
if the gpu is doing posted (or cached) writes, I don't think there is even a CPU side barrier primitive that could wait for that? I think we rely on the GPU not interrupting the CPU until the write is posted
Once the GPU puts the write on the bus then it is up to the whims of the CPU architecture. If the writes are being done out of order you run a chance of firing the interrupt and making it all the way to your handler before the writes catch up.
Quoted 4 lines
Since you are scheduling and doing a bunch of things in between you probably don't need to worry but if you start missing events and you don't know why then this could be why. A
rmb()would give you piece of mind at the cost of being Yet Another Barrier (TM).
Doesn't the fence case do the same thing without a barrier?
Quoted 12 lines
Jordan
Quoted 6 lines
BR,
-R
_______________________________________________
Freedreno mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/freedreno--
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
Re: [Freedreno] [PATCH 2/2] drm/msm: Add MSM_WAIT_IOVA ioctl
Quoted 27 lines · Kristian Kristensen
Quoted 26 lines · Jordan Crouse
Quoted 15 lines · Rob Clark
Quoted 13 lines · Jordan Crouse
drivers/gpu/drm/msm/msm_drv.c static int msm_ioctl_submitqueue_close(struct drm_device *dev, void *data, · line 993
Quoted 6 lines · Brian Ho
drivers/gpu/drm/msm/msm_drv.c +6
-0+ + vaddr = base_vaddr + args->offset; + +/* Assumes WC mapping */ + ret =wait_event_interruptible_timeout ( + gpu->event, *vaddr >= args->value, remaining_jiffies);I feel like a barrier might be needed before checking *vaddr just in case you get the interrupt and wake up the queue before the write posts from the hardware.
Quoted 3 lines
if the gpu is doing posted (or cached) writes, I don't think there is even a CPU side barrier primitive that could wait for that? I think we rely on the GPU not interrupting the CPU until the write is posted
Once the GPU puts the write on the bus then it is up to the whims of the CPU architecture. If the writes are being done out of order you run a chance of firing the interrupt and making it all the way to your handler before the writes catch up.
Quoted 6 lines
Quoted 4 lines
Since you are scheduling and doing a bunch of things in between you probably don't need to worry but if you start missing events and you don't know why then this could be why. A
rmb()would give you piece of mind at the cost of being Yet Another Barrier (TM).Doesn't the fence case do the same thing without a barrier?
We get a free __iormb() and dma_rmb() from every gpu_read() so I think that is enough to catch everything up when the interrupt handler reads the status register and in most cases we don't check the fence until after that. I'm not sure that covers all the possible error cases. Maybe something to look into?
Also that field is marked as volatile in the struct, but I'm not sure that does anything useful.
Jordan
Quoted 14 lines
Quoted 10 lines
Quoted 6 lines
BR,
-R
_______________________________________________
Freedreno mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/freedreno--
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project_______________________________________________
Freedreno mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/freedreno
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project