accel: ethosu: Runtime PM refcounting and cmd stream validation fixes
Rob Herring (Arm)
maintainercover accel: ethosu: Runtime PM refcounting and cmd stream validation fixes
This is series of fixes for issues I found in testing additional models and adding more supported operations in mesa.
---
Rob Herring (Arm) (3):
accel: ethosu: Fix job submit error clean-up refcount underflows
accel: ethosu: Fix NPU_OP_ELEMENTWISE validation with scalar
accel: ethosu: Handle possible underflow in IFM size calculations
drivers/accel/ethosu/ethosu_gem.c | 12 +++++++++--- drivers/accel/ethosu/ethosu_job.c | 26 ++++++++++++++++++-------- 2 files changed, 27 insertions(+), 11 deletions(-)
---
base-commit: 05f7e89ab9731565d8a62e3b5d1ec206485eeb0b
change-id: 20260218-ethos-fixes-e3508a579582
Best regards,
Signature · 1 line
Rob Herring (Arm) <[email protected]>
Rob Herring (Arm)
maintainerPATCH 1/3 accel: ethosu: Fix job submit error clean-up refcount underflows
If the job submit fails before adding the job to the scheduler queue such as when the GEM buffer bounds checks fail, then doing a ethosu_job_put() results in a pm_runtime_put_autosuspend() without the corresponding pm_runtime_resume_and_get(). The dma_fence_put()'s are also unnecessary, but seem to be harmless.
Split the ethosu_job_cleanup() function into 2 parts for the before and after the job is queued.
--- drivers/accel/ethosu/ethosu_job.c | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-)
drivers/accel/ethosu/ethosu_job.c +18 -8
@@ -143,23 +143,29 @@ static int ethosu_job_push(struct ethosu_job *job)return ret; } +static void ethosu_job_err_cleanup (struct ethosu_job *job) +{ +unsigned int i; + +for (i =0 ; i < job->region_cnt; i++) +drm_gem_object_put (job->region_bo[i]); + +drm_gem_object_put (job->cmd_bo); + +kfree (job); +} +static void ethosu_job_cleanup (struct kref *ref) { struct ethosu_job *job = container_of(ref, struct ethosu_job, refcount); -unsigned int i;pm_runtime_put_autosuspend (job->dev->base.dev);dma_fence_put (job->done_fence);dma_fence_put (job->inference_done_fence); -for (i =0 ; i < job->region_cnt; i++) -drm_gem_object_put (job->region_bo[i]); - -drm_gem_object_put (job->cmd_bo); - -kfree (job); +ethosu_job_err_cleanup (job); }static void ethosu_job_put (struct ethosu_job *job)
@@ -454,12 +460,16 @@ static int ethosu_ioctl_submit_job(struct drm_device *dev, struct drm_file *file } } ret =ethosu_job_push (ejob); +if (!ret) { +ethosu_job_put (ejob); +return 0 ; + } out_cleanup_job:if (ret)drm_sched_job_cleanup (&ejob->base); out_put_job: -ethosu_job_put (ejob); +ethosu_job_err_cleanup (ejob);return ret; }
Signature · 1 line
2.51.0
Rob Herring (Arm)
maintainerPATCH 2/3 accel: ethosu: Fix NPU_OP_ELEMENTWISE validation with scalar
In reply to Rob Herring (Arm)The NPU_OP_ELEMENTWISE instruction uses a scalar value for IFM2 if the IFM2_BROADCAST "scalar" mode is set. It is a bit (7) on the u65 and part of a field (bits 3:0) on the u85. The driver was hardcoded to the u85.
--- drivers/accel/ethosu/ethosu_gem.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/accel/ethosu/ethosu_gem.c +4 -1
@@ -417,7 +417,10 @@ static int ethosu_gem_cmdstream_copy_and_validate(struct drm_device *ddev,return ret;break ;case NPU_OP_ELEMENTWISE : - use_ifm2 = !((st.ifm2.broadcast ==8 ) || (param ==5 ) || + use_scale =ethosu_is_u65 (edev) ? + (st.ifm2.broadcast &0x80 ) : + (st.ifm2.broadcast ==8 ); + use_ifm2 = !(use_scale || (param ==5 ) || (param ==6 ) || (param ==7 ) || (param ==0x24 )); use_ifm = st.ifm.broadcast !=8 ; ret =calc_sizes_elemwise (ddev, info, cmd, &st, use_ifm, use_ifm2);
Signature · 1 line
2.51.0
Rob Herring (Arm)
maintainerPATCH 3/3 accel: ethosu: Handle possible underflow in IFM size calculations
In reply to Rob Herring (Arm)If the command stream has larger padding sizes than the IFM and OFM diminsions, then the calculations will underflow to a negative value. The result is a very large region bounds which is caught on submit, but it's better to catch it earlier.
Current mesa ethosu driver has a signedness bug which resulted in padding of 127 (the max) and triggers this issue.
--- drivers/accel/ethosu/ethosu_gem.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
drivers/accel/ethosu/ethosu_gem.c +5 -2
@@ -245,11 +245,14 @@ static int calc_sizes(struct drm_device *ddev, ((st->ifm.stride_kernel >>1 ) &0x1 ) +1 ;u32 stride_x = ((st->ifm.stride_kernel >>5 ) &0x2 ) + (st->ifm.stride_kernel &0x1 ) +1 ; -u32 ifm_height = st->ofm.height[2 ] * stride_y + +s32 ifm_height = st->ofm.height[2 ] * stride_y + st->ifm.height[2 ] - (st->ifm.pad_top + st->ifm.pad_bottom); -u32 ifm_width = st->ofm.width * stride_x + +s32 ifm_width = st->ofm.width * stride_x + st->ifm.width - (st->ifm.pad_left + st->ifm.pad_right); +if (ifm_height <0 || ifm_width <0 ) +return -EINVAL ; + len =feat_matrix_length (info, &st->ifm, ifm_width, ifm_height, st->ifm.depth); dev_dbg(ddev->dev, "op %d: IFM:%d:0x%llx-0x%llx\n",
Signature · 1 line
2.51.0
Re: [PATCH 1/3] accel: ethosu: Fix job submit error clean-up refcount underflows
In reply to Rob Herring (Arm)Quoted 7 lines · Rob Herring (Arm)
If the job submit fails before adding the job to the scheduler queue such as when the GEM buffer bounds checks fail, then doing a
ethosu_job_put()results in apm_runtime_put_autosuspend()without the correspondingpm_runtime_resume_and_get(). The dma_fence_put()'s are also unnecessary, but seem to be harmless.
Quoted 5 lines
Split the
ethosu_job_cleanup()function into 2 parts for the before and after the job is queued.Fixes 5a5e9c0228e6 ("accel: Add Arm Ethos-U NPU driver")Signed-off-by Rob Herring (Arm) <[email protected]>
Quoted 67 lines
--- drivers/accel/ethosu/ethosu_job.c | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-)drivers/accel/ethosu/ethosu_job.c +18
-9@@ -143,23 +143,29 @@ static int ethosu_job_push(struct ethosu_job *job)return ret; } +static void ethosu_job_err_cleanup (struct ethosu_job *job) +{ +unsigned int i; + +for (i =0 ; i < job->region_cnt; i++) +drm_gem_object_put (job->region_bo[i]); + +drm_gem_object_put (job->cmd_bo); + +kfree (job); +} +static void ethosu_job_cleanup (struct kref *ref) { struct ethosu_job *job = container_of(ref, struct ethosu_job, refcount); -unsigned int i;pm_runtime_put_autosuspend (job->dev->base.dev);dma_fence_put (job->done_fence);dma_fence_put (job->inference_done_fence); -for (i =0 ; i < job->region_cnt; i++) -drm_gem_object_put (job->region_bo[i]); - -drm_gem_object_put (job->cmd_bo); - -kfree (job); +ethosu_job_err_cleanup (job); }static void ethosu_job_put (struct ethosu_job *job)@@ -454,12 +460,16 @@ static int ethosu_ioctl_submit_job(struct drm_device *dev, struct drm_file *file } } ret =ethosu_job_push (ejob); +if (!ret) { +ethosu_job_put (ejob); +return 0 ; + } out_cleanup_job:if (ret)drm_sched_job_cleanup (&ejob->base); out_put_job: -ethosu_job_put (ejob); +ethosu_job_err_cleanup (ejob);return ret; } --2.51.0
Re: [PATCH 2/3] accel: ethosu: Fix NPU_OP_ELEMENTWISE validation with scalar
In reply to Rob Herring (Arm)Quoted 8 lines · Rob Herring (Arm)
The
NPU_OP_ELEMENTWISEinstruction uses a scalar value for IFM2 if theIFM2_BROADCAST"scalar" mode is set. It is a bit (7) on the u65 and part of a field (bits 3:0) on the u85. The driver was hardcoded to the u85.Fixes 5a5e9c0228e6 ("accel: Add Arm Ethos-U NPU driver")Signed-off-by Rob Herring (Arm) <[email protected]>
Quoted 24 lines
--- drivers/accel/ethosu/ethosu_gem.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)drivers/accel/ethosu/ethosu_gem.c +4
-2@@ -417,7 +417,10 @@ static int ethosu_gem_cmdstream_copy_and_validate(struct drm_device *ddev,return ret;break ;case NPU_OP_ELEMENTWISE : - use_ifm2 = !((st.ifm2.broadcast ==8 ) || (param ==5 ) || + use_scale =ethosu_is_u65 (edev) ? + (st.ifm2.broadcast &0x80 ) : + (st.ifm2.broadcast ==8 ); + use_ifm2 = !(use_scale || (param ==5 ) || (param ==6 ) || (param ==7 ) || (param ==0x24 )); use_ifm = st.ifm.broadcast !=8 ; ret =calc_sizes_elemwise (ddev, info, cmd, &st, use_ifm, use_ifm2); --2.51.0
Re: [PATCH 3/3] accel: ethosu: Handle possible underflow in IFM size calculations
In reply to Rob Herring (Arm)Quoted 3 lines · Rob Herring (Arm)
If the command stream has larger padding sizes than the IFM and OFM diminsions, then the calculations will underflow to a negative value.
Nit is "diminsions" -> "dimensions"
Quoted 7 lines
The result is a very large region bounds which is caught on submit, but it's better to catch it earlier.
Current mesa ethosu driver has a signedness bug which resulted in padding of 127 (the max) and triggers this issue.
Signed-off-by Rob Herring (Arm) <[email protected]>
Quoted 29 lines
--- drivers/accel/ethosu/ethosu_gem.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)drivers/accel/ethosu/ethosu_gem.c +5
-3@@ -245,11 +245,14 @@ static int calc_sizes(struct drm_device *ddev, ((st->ifm.stride_kernel >>1 ) &0x1 ) +1 ;u32 stride_x = ((st->ifm.stride_kernel >>5 ) &0x2 ) + (st->ifm.stride_kernel &0x1 ) +1 ; -u32 ifm_height = st->ofm.height[2 ] * stride_y + +s32 ifm_height = st->ofm.height[2 ] * stride_y + st->ifm.height[2 ] - (st->ifm.pad_top + st->ifm.pad_bottom); -u32 ifm_width = st->ofm.width * stride_x + +s32 ifm_width = st->ofm.width * stride_x + st->ifm.width - (st->ifm.pad_left + st->ifm.pad_right); +if (ifm_height <0 || ifm_width <0 ) +return -EINVAL ; + len =feat_matrix_length (info, &st->ifm, ifm_width, ifm_height, st->ifm.depth); dev_dbg(ddev->dev, "op %d: IFM:%d:0x%llx-0x%llx\n", --2.51.0