media: iris: optimize COMV buffer allocation for VPU3x and VPU4x
Subsystems (4)
PATCH v4 1/1 media: iris: optimize COMV buffer allocation for VPU3x and VPU4x
The existing iris_vpu_dec_comv_size() used VIDEO_MAX_FRAME (32) as num_comv count unconditionally when calculating the co-located motion vector (COMV) buffer size. This resulted in an oversized COMV buffer allocation throughout decode session, wasting memory regardless of actual number of buffers required.
For VPU3x and VPU4x platforms, introduce iris_vpu3x_4x_dec_comv_size() to replace iris_vpu_dec_comv_size(). These derive num_comv dynamically, it uses inst->fw_min_count once the firmware has reported its buffer requirements, and fallback to output count during initialization before firmware has communicated its requirements. This aligns the COMV buffer size to the actual count needed rather than always allocating with fixed VIDEO_MAX_FRAME value.
Additionally, during iris_vdec_inst_init(), fw_min_count was initialized to MIN_BUFFERS instead of 0. This masked the fallback logic and caused the COMV size calculation to use MIN_BUFFERS even before firmware had reported its actual requirements. Fix this by initializing fw_min_count to 0.
During testing of 1080p AVC, it reduces the COMV buffer size from 32.89MB to 6.16MB per decode session, significantly reducing memory consumption.
---
Changes in v4:
- Rebase on linux-media/users/bodonoghue next+fixes
- Resolve merge conflicts due to merged patch "media: qcom: iris: don't
use function indirection in gen2-specific code"
- Link to v3: https://lore.kernel.org/all/[email protected]/
Changes in v3:
- Update num_comv from instance data instead of using caps num_comv
in set_num_comv function to avoid wrong value update during concurrency.
- Link to v2: https://lore.kernel.org/r/[email protected]
Changes in v2:
- Update commit description (Bryan)
- Update hfi comv buffer count value to use actual num_comv count which
used for buffer calculation to avoid any overhead or fixed values (Vikash)
- Link to v1: https://lore.kernel.org/r/[email protected]
--- .../platform/qcom/iris/iris_hfi_gen2_command.c | 15 +++------------ .../platform/qcom/iris/iris_platform_common.h | 1 - .../platform/qcom/iris/iris_platform_qcs8300.h | 1 - .../platform/qcom/iris/iris_platform_sm8550.h | 1 - drivers/media/platform/qcom/iris/iris_vdec.c | 3 ++- drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 22 ++++++++++++++++++++-- 6 files changed, 25 insertions(+), 18 deletions(-)
drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c +3 -12
@@ -10,7 +10,6 @@#define UNSPECIFIED_COLOR_FORMAT 5#define NUM_SYS_INIT_PACKETS 8 -#define NUM_COMV_AV1 18#define SYS_INIT_PKT_SIZE (sizeof(struct iris_hfi_header) + \ NUM_SYS_INIT_PACKETS * (sizeof(struct iris_hfi_packet) + sizeof(u32)))
@@ -1207,18 +1206,10 @@ static u32 iris_hfi_gen2_buf_type_from_driver(u32 domain, enum iris_buffer_typestatic int iris_hfi_gen2_set_num_comv (struct iris_inst *inst) { -struct platform_inst_caps *caps; -struct iris_core *core = inst->core; -u32 num_comv; - - caps = core->iris_platform_data->inst_caps; +u32 num_comv = inst->buffers[BUF_OUTPUT ].min_count; -/* -* AV1 needs more comv buffers than other codecs. -* Update accordingly. -*/ - num_comv = (inst->codec ==V4L2_PIX_FMT_AV1 ) ? -NUM_COMV_AV1 : caps->num_comv; +if (inst->fw_min_count) + num_comv = inst->fw_min_count; return iris_hfi_gen2_session_set_property(inst, HFI_PROP_COMV_BUFFER_COUNT,
drivers/media/platform/qcom/iris/iris_platform_common.h +0 -1
@@ -87,7 +87,6 @@ struct platform_inst_caps {u32 mb_cycles_vpp;u32 mb_cycles_fw;u32 mb_cycles_fw_vpp; -u32 num_comv;u32 max_frame_rate;u32 max_operating_rate; };
drivers/media/platform/qcom/iris/iris_platform_qcs8300.h +0 -1
@@ -15,7 +15,6 @@ static struct platform_inst_caps platform_inst_cap_qcs8300 = { .mb_cycles_vpp = 200, .mb_cycles_fw = 326389, .mb_cycles_fw_vpp = 44156, - .num_comv = 0, .max_frame_rate = MAXIMUM_FPS, .max_operating_rate = MAXIMUM_FPS, };
drivers/media/platform/qcom/iris/iris_platform_sm8550.h +0 -1
@@ -23,7 +23,6 @@ static struct platform_inst_caps platform_inst_cap_sm8550 = { .mb_cycles_vpp = 200, .mb_cycles_fw = 489583, .mb_cycles_fw_vpp = 66234, - .num_comv = 0, .max_frame_rate = MAXIMUM_FPS, .max_operating_rate = MAXIMUM_FPS, };
drivers/media/platform/qcom/iris/iris_vdec.c +2 -1
@@ -24,7 +24,7 @@ int iris_vdec_inst_init(struct iris_inst *inst) inst->fmt_src =kzalloc_obj (*inst->fmt_src); inst->fmt_dst =kzalloc_obj (*inst->fmt_dst); - inst->fw_min_count =MIN_BUFFERS ; + inst->fw_min_count =0 ; f = inst->fmt_src; f->type =V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE ;
@@ -252,6 +252,7 @@ int iris_vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f)/* Update capture format based on new ip w/h */ output_fmt->fmt.pix_mp.width =ALIGN (f->fmt.pix_mp.width,128 ); output_fmt->fmt.pix_mp.height =ALIGN (f->fmt.pix_mp.height,32 ); + inst->buffers[BUF_OUTPUT ].min_count =iris_vpu_buf_count (inst,BUF_OUTPUT ); inst->buffers[BUF_OUTPUT ].size =iris_get_buffer_size (inst,BUF_OUTPUT ); inst->crop.left =0 ;
drivers/media/platform/qcom/iris/iris_vpu_buffer.c +20 -2
@@ -731,6 +731,24 @@ static u32 iris_vpu_dec_comv_size(struct iris_inst *inst)u32 height = f->fmt.pix_mp.height;u32 width = f->fmt.pix_mp.width; +if (inst->codec ==V4L2_PIX_FMT_H264 ) +return hfi_buffer_comv_h264d (width, height, num_comv); +else if (inst->codec ==V4L2_PIX_FMT_HEVC ) +return hfi_buffer_comv_h265d (width, height, num_comv); + +return 0 ; +} + +static u32 iris_vpu3x_4x_dec_comv_size (struct iris_inst *inst) +{ +u32 num_comv = inst->buffers[BUF_OUTPUT ].min_count; +struct v4l2_format *f = inst->fmt_src; +u32 height = f->fmt.pix_mp.height; +u32 width = f->fmt.pix_mp.width; + +if (inst->fw_min_count) + num_comv = inst->fw_min_count; +if (inst->codec ==V4L2_PIX_FMT_H264 )return hfi_buffer_comv_h264d (width, height, num_comv);else if (inst->codec ==V4L2_PIX_FMT_HEVC )
@@ -2025,7 +2043,7 @@ u32 iris_vpu_buf_size(struct iris_inst *inst, enum iris_buffer_type buffer_type)static const struct iris_vpu_buf_type_handle dec_internal_buf_type_handle[] = { {BUF_BIN , iris_vpu_dec_bin_size }, - {BUF_COMV , iris_vpu_dec_comv_size }, + {BUF_COMV , iris_vpu3x_4x_dec_comv_size }, {BUF_NON_COMV , iris_vpu_dec_non_comv_size }, {BUF_LINE , iris_vpu_dec_line_size }, {BUF_PERSIST , iris_vpu_dec_persist_size },
@@ -2098,7 +2116,7 @@ u32 iris_vpu4x_buf_size(struct iris_inst *inst, enum iris_buffer_type buffer_typstatic const struct iris_vpu_buf_type_handle dec_internal_buf_type_handle[] = { {BUF_BIN , iris_vpu_dec_bin_size }, - {BUF_COMV , iris_vpu_dec_comv_size }, + {BUF_COMV , iris_vpu3x_4x_dec_comv_size }, {BUF_NON_COMV , iris_vpu_dec_non_comv_size }, {BUF_LINE , iris_vpu4x_dec_line_size }, {BUF_PERSIST , iris_vpu4x_dec_persist_size },
---
base-commit: 82e452f73dc86f42a2df8bb1f5f18908c6144d6c
change-id: 20260513-optimize_comv_buffer-45f0c121b306
Best regards,
Signature · 1 line
Vishnu Reddy <[email protected]>
[email protected]
maintainerQuoted 36 lines · Vishnu Reddy
The existing
iris_vpu_dec_comv_size()usedVIDEO_MAX_FRAME(32) asnum_comvcount unconditionally when calculating the co-located motion vector (COMV) buffer size. This resulted in an oversized COMV buffer allocation throughout decode session, wasting memory regardless of actual number of buffers required.For VPU3x and VPU4x platforms, introduce
iris_vpu3x_4x_dec_comv_size()to replaceiris_vpu_dec_comv_size(). These derivenum_comvdynamically, it usesinst->fw_min_countonce the firmware has reported its buffer requirements, and fallback to output count during initialization before firmware has communicated its requirements. This aligns the COMV buffer size to the actual count needed rather than always allocating with fixedVIDEO_MAX_FRAMEvalue.Additionally, during
iris_vdec_inst_init(),fw_min_countwas initialized toMIN_BUFFERSinstead of 0. This masked the fallback logic and caused the COMV size calculation to useMIN_BUFFERSeven before firmware had reported its actual requirements. Fix this by initializingfw_min_countto 0.During testing of 1080p AVC, it reduces the COMV buffer size from 32.89MB to 6.16MB per decode session, significantly reducing memory consumption.
Reviewed-by Vikash Garodia <[email protected]>Signed-off-by Vishnu Reddy <[email protected]>---
Changes in v4:
- Rebase on linux-media/users/bodonoghue next+fixes
- Resolve merge conflicts due to merged patch "media: qcom: iris: don't
use function indirection in gen2-specific code"
- Link to v3: https://lore.kernel.org/all/[email protected]/Changes in v3:
- Updatenum_comvfrom instance data instead of using capsnum_comv
inset_num_comvfunction to avoid wrong value update during concurrency.
- Link to v2: https://lore.kernel.org/r/[email protected]
Quoted 5 lines
Changes in v2:
- Update commit description (Bryan)
- Update hfi comv buffer count value to use actualnum_comvcount which
used for buffer calculation to avoid any overhead or fixed values (Vikash)
- Link to v1: https://lore.kernel.org/r/[email protected]
Please rebase.
<<< deckard@inspiron14p-linux~/Development/linux-worktrees/b4/bod-media-committers-next-plaform-qcom
Quoted 1 line
Quoted 1 line
Quoted 1 line
b4 shazam [email protected] ‹git:next-smoketest ✘› 01:14.01 Sat May 30 2026 >>>
Looking up https://lore.kernel.org/all/[email protected]/
Grabbing thread from lore.kernel.org/all/[email protected]/t.mbox.gz Checking for newer revisions
Grabbing search results from lore.kernel.org
Nothing matching that query.
Analyzing 1 messages in the thread
Looking for additional code-review trailers on lore.kernel.org
Analyzing 0 code-review messages
Checking attestation on all messages, may take a moment...
---
✗ [PATCH v4] media: iris: optimize COMV buffer allocation for VPU3x and VPU4x
---
✗ No key: ed25519/[email protected]
---
NOTE: install dkimpy for DKIM signature verification
---
Total patches: 1
---
Applying: media: iris: optimize COMV buffer allocation for VPU3x and VPU4x
Patch failed at 0001 media: iris: optimize COMV buffer allocation for VPU3x and VPU4x
error: patch failed: drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c:10
error: drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c: patch does not apply
error: patch failed: drivers/media/platform/qcom/iris/iris_platform_common.h:87
error: drivers/media/platform/qcom/iris/iris_platform_common.h: patch does not apply
error: patch failed: drivers/media/platform/qcom/iris/iris_platform_qcs8300.h:15
error: drivers/media/platform/qcom/iris/iris_platform_qcs8300.h: patch does not apply
error: patch failed: drivers/media/platform/qcom/iris/iris_platform_sm8550.h:23
error: drivers/media/platform/qcom/iris/iris_platform_sm8550.h: patch does not apply
error: patch failed: drivers/media/platform/qcom/iris/iris_vdec.c:24
error: drivers/media/platform/qcom/iris/iris_vdec.c: patch does not apply
error: patch failed: drivers/media/platform/qcom/iris/iris_vpu_buffer.c:2025
error: drivers/media/platform/qcom/iris/iris_vpu_buffer.c: patch does not apply
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
---
bod
Quoted 37 lines · On 5/30/2026 5:55 AM, [email protected] wrote:
Quoted 36 lines · Vishnu Reddy
The existing
iris_vpu_dec_comv_size()usedVIDEO_MAX_FRAME(32) asnum_comvcount unconditionally when calculating the co-located motion vector (COMV) buffer size. This resulted in an oversized COMV buffer allocation throughout decode session, wasting memory regardless of actual number of buffers required.For VPU3x and VPU4x platforms, introduce
iris_vpu3x_4x_dec_comv_size()to replaceiris_vpu_dec_comv_size(). These derivenum_comvdynamically, it usesinst->fw_min_countonce the firmware has reported its buffer requirements, and fallback to output count during initialization before firmware has communicated its requirements. This aligns the COMV buffer size to the actual count needed rather than always allocating with fixedVIDEO_MAX_FRAMEvalue.Additionally, during
iris_vdec_inst_init(),fw_min_countwas initialized toMIN_BUFFERSinstead of 0. This masked the fallback logic and caused the COMV size calculation to useMIN_BUFFERSeven before firmware had reported its actual requirements. Fix this by initializingfw_min_countto 0.During testing of 1080p AVC, it reduces the COMV buffer size from 32.89MB to 6.16MB per decode session, significantly reducing memory consumption.
Reviewed-by Vikash Garodia <[email protected]>Signed-off-by Vishnu Reddy <[email protected]>---
Changes in v4:
- Rebase on linux-media/users/bodonoghue next+fixes
- Resolve merge conflicts due to merged patch "media: qcom: iris: don't
use function indirection in gen2-specific code"
- Link to v3: https://lore.kernel.org/all/[email protected]/Changes in v3:
- Updatenum_comvfrom instance data instead of using capsnum_comv
inset_num_comvfunction to avoid wrong value update during concurrency.
- Link to v2: https://lore.kernel.org/r/[email protected]
Quoted 6 lines
Quoted 5 lines
Changes in v2:
- Update commit description (Bryan)
- Update hfi comv buffer count value to use actualnum_comvcount which
used for buffer calculation to avoid any overhead or fixed values (Vikash)
- Link to v1: https://lore.kernel.org/r/[email protected]Please rebase.
Hi Bryan,
I checked that the patch is already present in linux-media/users/bodonoghue → venus-iris-next.
It also applies cleanly on the following branches:
linux-next → master
media tree → next
linux-media/users/bodonoghue → next
linux-media/users/bodonoghue → next+fixes linux-media/users/bodonoghue → next-smoketest
Could you please let me know on which branch the patch is not applying cleanly and where exactly you'd like me to rebase?
Thanks,
Vishnu Reddy.
Quoted 43 lines
<<< deckard@inspiron14p-linux~/Development/linux-worktrees/b4/bod-media-committers-next-plaform-qcom
Quoted 1 line
Quoted 1 line
Quoted 1 line
b4 shazam [email protected] ‹git:next-smoketest ✘› 01:14.01 Sat May 30 2026 >>>
Looking up https://lore.kernel.org/all/[email protected]/
Grabbing thread from lore.kernel.org/all/[email protected]/t.mbox.gz Checking for newer revisions
Grabbing search results from lore.kernel.org
Nothing matching that query.
Analyzing 1 messages in the thread
Looking for additional code-review trailers on lore.kernel.org
Analyzing 0 code-review messages
Checking attestation on all messages, may take a moment...
---
✗ [PATCH v4] media: iris: optimize COMV buffer allocation for VPU3x and VPU4x
---
✗ No key: ed25519/[email protected]
---
NOTE: install dkimpy for DKIM signature verification
---
Total patches: 1
---
Applying: media: iris: optimize COMV buffer allocation for VPU3x and VPU4x
Patch failed at 0001 media: iris: optimize COMV buffer allocation for VPU3x and VPU4x
error: patch failed:drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c:10
error:drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c: patch does not apply
error: patch failed:drivers/media/platform/qcom/iris/iris_platform_common.h:87
error:drivers/media/platform/qcom/iris/iris_platform_common.h: patch does not apply
error: patch failed:drivers/media/platform/qcom/iris/iris_platform_qcs8300.h:15
error:drivers/media/platform/qcom/iris/iris_platform_qcs8300.h: patch does not apply
error: patch failed:drivers/media/platform/qcom/iris/iris_platform_sm8550.h:23
error:drivers/media/platform/qcom/iris/iris_platform_sm8550.h: patch does not apply
error: patch failed:drivers/media/platform/qcom/iris/iris_vdec.c:24
error:drivers/media/platform/qcom/iris/iris_vdec.c: patch does not apply
error: patch failed:drivers/media/platform/qcom/iris/iris_vpu_buffer.c:2025
error:drivers/media/platform/qcom/iris/iris_vpu_buffer.c: patch does not apply
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"---
bod
Bryan O'Donoghue
maintainerQuoted 11 lines · Vishnu Reddy
Hi Bryan,
I checked that the patch is already present in linux-media/users/bodonoghue → venus-iris-next.
It also applies cleanly on the following branches:
linux-next → master
media tree → next
linux-media/users/bodonoghue → next
linux-media/users/bodonoghue → next+fixes linux-media/users/bodonoghue → next-smoketest
Quoted 5 lines
Could you please let me know on which branch the patch is not applying cleanly and where exactly you'd like me to rebase?
Thanks,
Vishnu Reddy.
As you can see I'm doing "b4 shazam" at 1:24 am when I should probably be in bed with my teddy bear.
Ignore.
---
bod