media: iris: check decoder format allocations
Subsystems (4)
PATCH 1/1 media: iris: check decoder format allocations
iris_vdec_inst_init() allocates the source and destination v4l2_format structures and then immediately writes fields through inst->fmt_src and inst->fmt_dst. Either allocation can fail, leading to a NULL pointer dereference during instance initialization.
Check both allocations before initializing the formats. Free any partial allocation, clear the instance pointers so later cleanup does not see dangling values, and return -ENOMEM so the open path can unwind the instance.
--- drivers/media/platform/qcom/iris/iris_vdec.c | 7 +++++++ 1 file changed, 7 insertions(+)
drivers/media/platform/qcom/iris/iris_vdec.c +7 -0
@@ -23,6 +23,13 @@ int iris_vdec_inst_init(struct iris_inst *inst) inst->fmt_src =kzalloc_obj (*inst->fmt_src); inst->fmt_dst =kzalloc_obj (*inst->fmt_dst); +if (!inst->fmt_src || !inst->fmt_dst) { +kfree (inst->fmt_src); +kfree (inst->fmt_dst); + inst->fmt_src =NULL ; + inst->fmt_dst =NULL ; +return -ENOMEM ; + } inst->fw_min_count =MIN_BUFFERS ;
Signature · 1 line
2.34.1
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] When max_session_count is reached, iris_add_session() silently drops the new instance, but iris_open() successfully returns, creating an untracked session. --
commit 0059fb169f9cd49b702675622f4a0ad32fd2392e
Author: Ruoyu Wang <[email protected]>
media: iris: check decoder format allocations
This commit adds checks for v4l2_format allocations in iris_vdec_inst_init() to prevent NULL pointer dereferences. It properly frees any partial allocations and returns -ENOMEM to allow the open path to unwind.
Quoted 4 lines
drivers/media/platform/qcom/iris/iris_vdec.c +0
-0
[ ... ]
drivers/media/platform/qcom/iris/iris_vdec.c int iris_vdec_inst_init(struct iris_inst *inst) · line 34
Quoted 13 lines
drivers/media/platform/qcom/iris/iris_vdec.c +7
-0@@ -23,6 +23,13 @@ int iris_vdec_inst_init(struct iris_inst *inst) inst->fmt_src =kzalloc_obj (*inst->fmt_src); inst->fmt_dst =kzalloc_obj (*inst->fmt_dst); +if (!inst->fmt_src || !inst->fmt_dst) { +kfree (inst->fmt_src); +kfree (inst->fmt_dst); + inst->fmt_src =NULL ; + inst->fmt_dst =NULL ; +return -ENOMEM ; + } inst->fw_min_count =MIN_BUFFERS ;
While reviewing the unwinding logic in the open path, I noticed a separate issue in drivers/media/platform/qcom/iris/iris_vidc.c:
drivers/media/platform/qcom/iris/iris_vidc.c:iris_add_session() {
...
count++;
if (count < core->iris_platform_data->max_session_count)
list_add_tail(&inst->list, &core->instances);
...
}drivers/media/platform/qcom/iris/iris_vidc.c:iris_open() {
...
iris_add_session(inst);
inst->fh.m2m_ctx = inst->m2m_ctx;
return 0;
}[Severity: High]
This isn't a bug introduced by this patch, but does this logic allow userspace to bypass the session limit and create an untracked session?
When the current session count reaches max_session_count, iris_add_session() silently drops the new instance without returning an error. Since iris_open() does not verify if the instance was successfully added to the core instances list, it returns 0 and provides userspace with a valid file descriptor.
Could this cause problems during streaming when resource management functions like iris_scale_power() and iris_check_core_mbps() iterate over the instances list to calculate total bandwidth and clock requirements? Since the untracked instance is missing from the list, its requirements are ignored, which might leave the hardware under-clocked or starved of bandwidth, potentially leading to hardware timeouts or watchdog resets.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Quoted 20 lines · Ruoyu Wang
iris_vdec_inst_init()allocates the source and destinationv4l2_formatstructures and then immediately writes fields throughinst->fmt_srcandinst->fmt_dst. Either allocation can fail, leading to a NULL pointer dereference during instance initialization.Check both allocations before initializing the formats. Free any partial allocation, clear the instance pointers so later cleanup does not see dangling values, and return
-ENOMEMso the open path can unwind the instance.Signed-off-by Ruoyu Wang <[email protected]>--- drivers/media/platform/qcom/iris/iris_vdec.c | 7 +++++++ 1 file changed, 7 insertions(+)drivers/media/platform/qcom/iris/iris_vdec.c +0
-0
drivers/media/platform/qcom/iris/iris_vdec.c int iris_vdec_inst_init(struct iris_inst *inst) · line 32
Quoted 10 lines
drivers/media/platform/qcom/iris/iris_vdec.c +7
-0@@ -23,6 +23,13 @@ int iris_vdec_inst_init(struct iris_inst *inst) inst->fmt_src =kzalloc_obj (*inst->fmt_src); inst->fmt_dst =kzalloc_obj (*inst->fmt_dst); +if (!inst->fmt_src || !inst->fmt_dst) { +kfree (inst->fmt_src); +kfree (inst->fmt_dst); + inst->fmt_src =NULL ; + inst->fmt_dst =NULL ; +return -ENOMEM ; + }
I'd rather see the check for the allocated objects before they are assigned to the fields in the instance.
Quoted 6 lines
drivers/media/platform/qcom/iris/iris_vdec.c +0
-0inst->fw_min_count =MIN_BUFFERS ;Signature · 1 line
2.34.1
Signature · 2 lines
With best wishes Dmitry
PATCH v2 1/1 media: iris: check decoder format allocations
In reply to Ruoyu Wangiris_vdec_inst_init() allocates source and destination v4l2_format structures before initializing their fields. Allocation failures would leave the function dereferencing NULL pointers during instance initialization.
Allocate the formats into local variables and check each allocation before assigning them to the instance. If the second allocation fails, free the first allocation and return -ENOMEM. Store the pointers in the instance only after both allocations have succeeded so the open path can unwind cleanly.
---
Changes in v2:
- Allocate the formats into local variables and assign them to the instance only after both allocations succeed, as requested in review.
drivers/media/platform/qcom/iris/iris_vdec.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-)
drivers/media/platform/qcom/iris/iris_vdec.c +13 -2
@@ -19,10 +19,21 @@int iris_vdec_inst_init (struct iris_inst *inst) {struct iris_core *core = inst->core; +struct v4l2_format *fmt_src, *fmt_dst;struct v4l2_format *f; - inst->fmt_src =kzalloc_obj (*inst->fmt_src); - inst->fmt_dst =kzalloc_obj (*inst->fmt_dst); + fmt_src =kzalloc_obj (*fmt_src); +if (!fmt_src) +return -ENOMEM ; + + fmt_dst =kzalloc_obj (*fmt_dst); +if (!fmt_dst) { +kfree (fmt_src); +return -ENOMEM ; + } + + inst->fmt_src = fmt_src; + inst->fmt_dst = fmt_dst; inst->fw_min_count =MIN_BUFFERS ;
Signature · 1 line
2.51.0
Quoted 32 lines · Ruoyu Wang
iris_vdec_inst_init()allocates source and destinationv4l2_formatstructures before initializing their fields. Allocation failures would leave the function dereferencing NULL pointers during instance initialization.Allocate the formats into local variables and check each allocation before assigning them to the instance. If the second allocation fails, free the first allocation and return
-ENOMEM. Store the pointers in the instance only after both allocations have succeeded so the open path can unwind cleanly.Signed-off-by Ruoyu Wang <[email protected]>---
Changes in v2:
- Allocate the formats into local variables and assign them to the instance only after both allocations succeed, as requested in review.drivers/media/platform/qcom/iris/iris_vdec.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-)drivers/media/platform/qcom/iris/iris_vdec.c +1
-1@@ -19,10 +19,21 @@int iris_vdec_inst_init (struct iris_inst *inst) {struct iris_core *core = inst->core; +struct v4l2_format *fmt_src, *fmt_dst;struct v4l2_format *f; - inst->fmt_src =kzalloc_obj (*inst->fmt_src);
drivers/media/platform/qcom/iris/iris_vdec.c line 33
Quoted 10 lines
drivers/media/platform/qcom/iris/iris_vdec.c +9
-1- inst->fmt_dst =kzalloc_obj (*inst->fmt_dst); + fmt_src =kzalloc_obj (*fmt_src); +if (!fmt_src) +return -ENOMEM ; + + fmt_dst =kzalloc_obj (*fmt_dst); +if (!fmt_dst) { +kfree (fmt_src); +return -ENOMEM ; + }
This is not the style of the rollback that is used in Linux kernel. Also if iris_ctrls_init() fails, then the allocate memory will not be unallocated. Further iris_open() will happily overwrite those pointers, resulting in a memory leak.
Should we replace the pointers with the instances of v4l2_format instead?
BTW: please don't send patch iterations as a reply to a previous thread. Always start a new thread for the new iteration.
Quoted 9 lines
drivers/media/platform/qcom/iris/iris_vdec.c +3
-0+ + inst->fmt_src = fmt_src; + inst->fmt_dst = fmt_dst; inst->fw_min_count =MIN_BUFFERS ;Signature · 1 line
2.51.0
Signature · 2 lines
With best wishes Dmitry