Skip to contentSkip to conversation

media: iris: optimize COMV buffer allocation for VPU3x and VPU4x

PATCH Superseded Vishnu Reddy 3 participants 5 messages Apr 21 – Apr 24

Subsystems (4)

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 media: iris: optimize COMV buffer allocation for VPU3x and VPU4x

The existing iris_vpu_dec_comv_size() used VIDEO_MAX_FRAME (32) as num_comv count unconditionally when calculating the COMV buffer size. This resulted in an oversized COMV buffer allocation throughout decode session, wasting memory regardless of actual number of buffers required.

For VPU3x and VPU4x platforms, introduce iris_vpu3x_4x_dec_comv_size() to replace iris_vpu_dec_comv_size(). It derives num_comv dynamically, it uses inst->fw_min_count once the firmware has reported its minimum buffer requirements, and fallback to inst->buffers[BUF_OUTPUT].min_count during initialization before firmware has communicated its requirements. This aligns the COMV buffer size to the actual count needed rather than always allocating with fixed VIDEO_MAX_FRAME value.

Additionally, during iris_vdec_inst_init(), fw_min_count was initialized to MIN_BUFFERS instead of 0. This masked the fallback logic and caused the COMV size calculation to use MIN_BUFFERS even before firmware had reported its actual requirements. Fix this by initializing fw_min_count to 0.

During testing of 1080p AVC, it reduces the COMV buffer size from 32.89MB to 6.16MB per decode session, significantly reducing memory consumption.

---
 drivers/media/platform/qcom/iris/iris_vdec.c       |  2 +-
 drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 24 +++++++++++++++++++---
 drivers/media/platform/qcom/iris/iris_vpu_buffer.h |  1 +
 3 files changed, 23 insertions(+), 4 deletions(-)
drivers/media/platform/qcom/iris/iris_vdec.c +1 -1
index 719217399a30..f433065e08b2 100644
@@ -24,7 +24,7 @@ int iris_vdec_inst_init(struct iris_inst *inst)
 	inst->fmt_src = kzalloc_obj(*inst->fmt_src);
 	inst->fmt_dst = kzalloc_obj(*inst->fmt_dst);
 
-	inst->fw_min_count = MIN_BUFFERS;
+	inst->fw_min_count = 0;
 
 	f = inst->fmt_src;
 	f->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
drivers/media/platform/qcom/iris/iris_vpu_buffer.c +21 -3
index 9270422c1601..57237543b229 100644
@@ -731,6 +731,23 @@ static u32 iris_vpu_dec_comv_size(struct iris_inst *inst)
 	u32 height = f->fmt.pix_mp.height;
 	u32 width = f->fmt.pix_mp.width;
 
+	if (inst->codec == V4L2_PIX_FMT_H264)
+		return hfi_buffer_comv_h264d(width, height, num_comv);
+	else if (inst->codec == V4L2_PIX_FMT_HEVC)
+		return hfi_buffer_comv_h265d(width, height, num_comv);
+
+	return 0;
+}
+
+static u32 iris_vpu3x_4x_dec_comv_size(struct iris_inst *inst)
+{
+	struct v4l2_format *f = inst->fmt_src;
+	u32 height = f->fmt.pix_mp.height;
+	u32 width = f->fmt.pix_mp.width;
+	u32 num_comv;
+
+	num_comv = inst->fw_min_count ? inst->fw_min_count : inst->buffers[BUF_OUTPUT].min_count;
+
 	if (inst->codec == V4L2_PIX_FMT_H264)
 		return hfi_buffer_comv_h264d(width, height, num_comv);
 	else if (inst->codec == V4L2_PIX_FMT_HEVC)
@@ -739,7 +756,8 @@ static u32 iris_vpu_dec_comv_size(struct iris_inst *inst)
 		if (inst->fw_caps[DRAP].value)
 			return 0;
 		else
-			return hfi_buffer_comv_av1d(width, height, num_comv);
+			return hfi_buffer_comv_av1d(width, height,
+						    num_comv + AV1D_COMV_BUFFER_OVERHEAD);
 	}
 
 	return 0;
@@ -2025,7 +2043,7 @@ u32 iris_vpu_buf_size(struct iris_inst *inst, enum iris_buffer_type buffer_type)
 
 	static const struct iris_vpu_buf_type_handle dec_internal_buf_type_handle[] = {
 		{BUF_BIN,         iris_vpu_dec_bin_size             },
-		{BUF_COMV,        iris_vpu_dec_comv_size            },
+		{BUF_COMV,        iris_vpu3x_4x_dec_comv_size       },
 		{BUF_NON_COMV,    iris_vpu_dec_non_comv_size        },
 		{BUF_LINE,        iris_vpu_dec_line_size            },
 		{BUF_PERSIST,     iris_vpu_dec_persist_size         },
@@ -2098,7 +2116,7 @@ u32 iris_vpu4x_buf_size(struct iris_inst *inst, enum iris_buffer_type buffer_typ
 
 	static const struct iris_vpu_buf_type_handle dec_internal_buf_type_handle[] = {
 		{BUF_BIN,         iris_vpu_dec_bin_size         },
-		{BUF_COMV,        iris_vpu_dec_comv_size        },
+		{BUF_COMV,        iris_vpu3x_4x_dec_comv_size   },
 		{BUF_NON_COMV,    iris_vpu_dec_non_comv_size    },
 		{BUF_LINE,        iris_vpu4x_dec_line_size      },
 		{BUF_PERSIST,     iris_vpu4x_dec_persist_size   },
drivers/media/platform/qcom/iris/iris_vpu_buffer.h +1 -0
index 12640eb5ed8c..7a9cc1c92da3 100644
@@ -110,6 +110,7 @@ struct iris_inst;
 #define MAX_PE_NBR_DATA_LCU16_LINE_BUFFER_SIZE 96
 #define AV1D_NUM_HW_PIC_BUF    16
 #define AV1D_NUM_FRAME_HEADERS 16
+#define AV1D_COMV_BUFFER_OVERHEAD 7
 #define SIZE_AV1D_SEQUENCE_HEADER 768
 #define SIZE_AV1D_METADATA        512
 #define SIZE_AV1D_FRAME_HEADER    1280

---
base-commit: 4fbeef21f5387234111b5d52924e77757626faa5
change-id: 20260421-optimize_comv_buffer-ae7107673609

Best regards,

Signature · 1 line
Vishnu Reddy <[email protected]>
maintainer
#1 lore raw
Quoted 4 lines · Vishnu Reddy

The existing iris_vpu_dec_comv_size() used VIDEO_MAX_FRAME (32) as num_comv count unconditionally when calculating the COMV buffer size. This resulted in an oversized COMV buffer allocation throughout decode session, wasting memory regardless of actual number of buffers required.

You should define what a COMV buffer is before talking about how you are changing it, i.e. define the term Co-located Motion Vector (CMOV) and then use the abbreviation CMOV liberally as you wish.

Quoted 2 lines

For VPU3x and VPU4x platforms, introduce iris_vpu3x_4x_dec_comv_size() to replace iris_vpu_dec_comv_size(). It derives num_comv dynamically, it

"These derive num_cmove dynamically"

Quoted 6 lines

uses inst->fw_min_count once the firmware has reported its minimum buffer requirements, and fallback to inst->buffers[BUF_OUTPUT].min_count during initialization before firmware has communicated its requirements. This aligns the COMV buffer size to the actual count needed rather than always allocating with fixed VIDEO_MAX_FRAME value.

Quoted 7 lines

Additionally, during iris_vdec_inst_init(), fw_min_count was initialized to MIN_BUFFERS instead of 0. This masked the fallback logic and caused the COMV size calculation to use MIN_BUFFERS even before firmware had reported its actual requirements. Fix this by initializing fw_min_count to 0.

During testing of 1080p AVC, it reduces the COMV buffer size from 32.89MB to 6.16MB per decode session, significantly reducing memory consumption.

Cool nice fix.

Quoted 35 lines
---
  drivers/media/platform/qcom/iris/iris_vdec.c       |  2 +-
  drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 24 +++++++++++++++++++---
  drivers/media/platform/qcom/iris/iris_vpu_buffer.h |  1 +
  3 files changed, 23 insertions(+), 4 deletions(-)
drivers/media/platform/qcom/iris/iris_vdec.c +1 -1
index 719217399a30..f433065e08b2 100644
@@ -24,7 +24,7 @@ int iris_vdec_inst_init(struct iris_inst *inst)
  	inst->fmt_src = kzalloc_obj(*inst->fmt_src);
  	inst->fmt_dst = kzalloc_obj(*inst->fmt_dst);
 
-	inst->fw_min_count = MIN_BUFFERS;
+	inst->fw_min_count = 0;
 
  	f = inst->fmt_src;
  	f->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
drivers/media/platform/qcom/iris/iris_vpu_buffer.c +6 -0
index 9270422c1601..57237543b229 100644
@@ -731,6 +731,23 @@ static u32 iris_vpu_dec_comv_size(struct iris_inst *inst)
  	u32 height = f->fmt.pix_mp.height;
  	u32 width = f->fmt.pix_mp.width;
 
+	if (inst->codec == V4L2_PIX_FMT_H264)
+		return hfi_buffer_comv_h264d(width, height, num_comv);
+	else if (inst->codec == V4L2_PIX_FMT_HEVC)
+		return hfi_buffer_comv_h265d(width, height, num_comv);
+
+	return 0;

drivers/media/platform/qcom/iris/iris_vpu_buffer.c static u32 iris_vpu_dec_comv_size(struct iris_inst *inst) · line 749

Quoted 10 lines
drivers/media/platform/qcom/iris/iris_vpu_buffer.c +10 -0
+}
+
+static u32 iris_vpu3x_4x_dec_comv_size(struct iris_inst *inst)
+{
+	struct v4l2_format *f = inst->fmt_src;
+	u32 height = f->fmt.pix_mp.height;
+	u32 width = f->fmt.pix_mp.width;
+	u32 num_comv;
+
+	num_comv = inst->fw_min_count ? inst->fw_min_count : inst->buffers[BUF_OUTPUT].min_count;

Please just if/else this though its far easier to read/understand that way.

Quoted 37 lines
drivers/media/platform/qcom/iris/iris_vpu_buffer.c +5 -3
+
  	if (inst->codec == V4L2_PIX_FMT_H264)
  		return hfi_buffer_comv_h264d(width, height, num_comv);
  	else if (inst->codec == V4L2_PIX_FMT_HEVC)
@@ -739,7 +756,8 @@ static u32 iris_vpu_dec_comv_size(struct iris_inst *inst)
  		if (inst->fw_caps[DRAP].value)
  			return 0;
  		else
-			return hfi_buffer_comv_av1d(width, height, num_comv);
+			return hfi_buffer_comv_av1d(width, height,
+						    num_comv + AV1D_COMV_BUFFER_OVERHEAD);
  	}
 
  	return 0;
@@ -2025,7 +2043,7 @@ u32 iris_vpu_buf_size(struct iris_inst *inst, enum iris_buffer_type buffer_type)
 
  	static const struct iris_vpu_buf_type_handle dec_internal_buf_type_handle[] = {
  		{BUF_BIN,         iris_vpu_dec_bin_size             },
-		{BUF_COMV,        iris_vpu_dec_comv_size            },
+		{BUF_COMV,        iris_vpu3x_4x_dec_comv_size       },
  		{BUF_NON_COMV,    iris_vpu_dec_non_comv_size        },
  		{BUF_LINE,        iris_vpu_dec_line_size            },
  		{BUF_PERSIST,     iris_vpu_dec_persist_size         },
@@ -2098,7 +2116,7 @@ u32 iris_vpu4x_buf_size(struct iris_inst *inst, enum iris_buffer_type buffer_typ
 
  	static const struct iris_vpu_buf_type_handle dec_internal_buf_type_handle[] = {
  		{BUF_BIN,         iris_vpu_dec_bin_size         },
-		{BUF_COMV,        iris_vpu_dec_comv_size        },
+		{BUF_COMV,        iris_vpu3x_4x_dec_comv_size   },
  		{BUF_NON_COMV,    iris_vpu_dec_non_comv_size    },
  		{BUF_LINE,        iris_vpu4x_dec_line_size      },
  		{BUF_PERSIST,     iris_vpu4x_dec_persist_size   },
drivers/media/platform/qcom/iris/iris_vpu_buffer.h +0 -0
index 12640eb5ed8c..7a9cc1c92da3 100644

drivers/media/platform/qcom/iris/iris_vpu_buffer.h struct iris_inst; · line 113

Quoted 4 lines
drivers/media/platform/qcom/iris/iris_vpu_buffer.h +1 -0
@@ -110,6 +110,7 @@ struct iris_inst;
  #define MAX_PE_NBR_DATA_LCU16_LINE_BUFFER_SIZE 96
  #define AV1D_NUM_HW_PIC_BUF    16
  #define AV1D_NUM_FRAME_HEADERS 16
+#define AV1D_COMV_BUFFER_OVERHEAD 7

Whats this ? Why is there a new seven byte overhead ? Does it represent a header, an alignment ?

An overhead can mean anything.

Quoted 12 lines
drivers/media/platform/qcom/iris/iris_vpu_buffer.h +0 -1
  #define SIZE_AV1D_SEQUENCE_HEADER 768
  #define SIZE_AV1D_METADATA        512
  #define SIZE_AV1D_FRAME_HEADER    1280
 
---

base-commit: 4fbeef21f5387234111b5d52924e77757626faa5
change-id: 20260421-optimize_comv_buffer-ae7107673609

Best regards,
--
Vishnu Reddy <[email protected]>

maintainer
#2 lore raw
Quoted 24 lines · Bryan O'Donoghue
Quoted 4 lines · Vishnu Reddy

The existing iris_vpu_dec_comv_size() used VIDEO_MAX_FRAME (32) as num_comv count unconditionally when calculating the COMV buffer size. This resulted in an oversized COMV buffer allocation throughout decode session, wasting memory regardless of actual number of buffers required.

You should define what a COMV buffer is before talking about how you are changing it, i.e. define the term Co-located Motion Vector (CMOV) and then use the abbreviation CMOV liberally as you wish.

Quoted 2 lines

For VPU3x and VPU4x platforms, introduce iris_vpu3x_4x_dec_comv_size() to replace iris_vpu_dec_comv_size(). It derives num_comv dynamically, it

"These derive num_cmove dynamically"

Quoted 11 lines

uses inst->fw_min_count once the firmware has reported its minimum buffer requirements, and fallback to inst->buffers[BUF_OUTPUT].min_count during initialization before firmware has communicated its requirements. This aligns the COMV buffer size to the actual count needed rather than always allocating with fixed VIDEO_MAX_FRAME value.

Additionally, during iris_vdec_inst_init(), fw_min_count was initialized to MIN_BUFFERS instead of 0. This masked the fallback logic and caused the COMV size calculation to use MIN_BUFFERS even before firmware had reported its actual requirements. Fix this by initializing fw_min_count to 0.

Quoted 5 lines
Quoted 2 lines

During testing of 1080p AVC, it reduces the COMV buffer size from 32.89MB to 6.16MB per decode session, significantly reducing memory consumption.

Cool nice fix.

Indeed, a good saving.

Quoted 89 lines
Quoted 38 lines

---
  drivers/media/platform/qcom/iris/iris_vdec.c       |  2 +- drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 24 ++++++++++++
+++++++---
  drivers/media/platform/qcom/iris/iris_vpu_buffer.h |  1 +

  3 files changed, 23 insertions(+), 4 deletions(-)
drivers/media/platform/qcom/iris/iris_vdec.c +1 -1
index 719217399a30..f433065e08b2 100644
@@ -24,7 +24,7 @@ int iris_vdec_inst_init(struct iris_inst *inst)
       inst->fmt_src = kzalloc_obj(*inst->fmt_src);
       inst->fmt_dst = kzalloc_obj(*inst->fmt_dst);
 
-    inst->fw_min_count = MIN_BUFFERS;
+    inst->fw_min_count = 0;
 
       f = inst->fmt_src;
       f->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
drivers/media/platform/qcom/iris/iris_vpu_buffer.c +7 -0
index 9270422c1601..57237543b229 100644
@@ -731,6 +731,23 @@ static u32 iris_vpu_dec_comv_size(struct 
 iris_inst *inst)
       u32 height = f->fmt.pix_mp.height;
       u32 width = f->fmt.pix_mp.width;
 
+    if (inst->codec == V4L2_PIX_FMT_H264)
+        return hfi_buffer_comv_h264d(width, height, num_comv);
+    else if (inst->codec == V4L2_PIX_FMT_HEVC)
+        return hfi_buffer_comv_h265d(width, height, num_comv);
+
+    return 0;
+}

drivers/media/platform/qcom/iris/iris_vpu_buffer.c static u32 iris_vpu_dec_comv_size(struct · line 751

Quoted 10 lines
drivers/media/platform/qcom/iris/iris_vpu_buffer.c +9 -0
+
+static u32 iris_vpu3x_4x_dec_comv_size(struct iris_inst *inst)
+{
+    struct v4l2_format *f = inst->fmt_src;
+    u32 height = f->fmt.pix_mp.height;
+    u32 width = f->fmt.pix_mp.width;
+    u32 num_comv;
+
+    num_comv = inst->fw_min_count ? inst->fw_min_count : inst- 
 >buffers[BUF_OUTPUT].min_count;

Please just if/else this though its far easier to read/understand that way.

Quoted 38 lines
drivers/media/platform/qcom/iris/iris_vpu_buffer.c +1 -0
+
       if (inst->codec == V4L2_PIX_FMT_H264)

          return hfi_buffer_comv_h264d(width, height, num_comv); else if (inst->codec == V4L2_PIX_FMT_HEVC)

drivers/media/platform/qcom/iris/iris_vpu_buffer.c +0 -0
@@ -739,7 +756,8 @@ static u32 iris_vpu_dec_comv_size(struct iris_inst 
 *inst)
           if (inst->fw_caps[DRAP].value)
               return 0;

          else
-            return hfi_buffer_comv_av1d(width, height, num_comv); +            return hfi_buffer_comv_av1d(width, height, +                            num_comv + AV1D_COMV_BUFFER_OVERHEAD); }

      return 0;

drivers/media/platform/qcom/iris/iris_vpu_buffer.c +1 -1
@@ -2025,7 +2043,7 @@ u32 iris_vpu_buf_size(struct iris_inst *inst, 
 enum iris_buffer_type buffer_type)
 
       static const struct iris_vpu_buf_type_handle dec_internal_buf_type_handle[] = {
           {BUF_BIN,         iris_vpu_dec_bin_size             },
-        {BUF_COMV,        iris_vpu_dec_comv_size            },
+        {BUF_COMV,        iris_vpu3x_4x_dec_comv_size       },
           {BUF_NON_COMV,    iris_vpu_dec_non_comv_size        },
           {BUF_LINE,        iris_vpu_dec_line_size            },

          {BUF_PERSIST,     iris_vpu_dec_persist_size         },

drivers/media/platform/qcom/iris/iris_vpu_buffer.c +0 -0
@@ -2098,7 +2116,7 @@ u32 iris_vpu4x_buf_size(struct iris_inst *inst, 

enum iris_buffer_type buffer_typ

      static const struct iris_vpu_buf_type_handle dec_internal_buf_type_handle[] = { {BUF_BIN,         iris_vpu_dec_bin_size         },
-        {BUF_COMV,        iris_vpu_dec_comv_size        },
+        {BUF_COMV,        iris_vpu3x_4x_dec_comv_size   }, {BUF_NON_COMV,    iris_vpu_dec_non_comv_size    },
          {BUF_LINE,        iris_vpu4x_dec_line_size      }, {BUF_PERSIST,     iris_vpu4x_dec_persist_size   },

drivers/media/platform/qcom/iris/iris_vpu_buffer.h renamed from drivers/media/platform/qcom/iris/iris_vpu_buffer.h +0 -1
index 12640eb5ed8c..7a9cc1c92da3 100644
--- a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h
Quoted 8 lines

drivers/media/platform/qcom/iris/iris_vpu_buffer.h struct iris_inst; · line 113

Quoted 6 lines

+++ b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h

drivers/media/platform/qcom/iris/iris_vpu_buffer.h renamed from drivers/media/platform/qcom/iris/iris_vpu_buffer.h +1 -0
@@ -110,6 +110,7 @@ struct iris_inst;
   #define MAX_PE_NBR_DATA_LCU16_LINE_BUFFER_SIZE 96
   #define AV1D_NUM_HW_PIC_BUF    16
   #define AV1D_NUM_FRAME_HEADERS 16
+#define AV1D_COMV_BUFFER_OVERHEAD 7

Whats this ? Why is there a new seven byte overhead ? Does it represent a header, an alignment ?

Vishnu, pls check if we need to add this as initial count was 18 [1] ? What if initial count was 11 [2], and post DRC, fw_min_count would be 11 too for AV1d, so the overhead can be avoided.

[1] https://elixir.bootlin.com/linux/v7.0-rc7/source/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c#L1220

[2] https://elixir.bootlin.com/linux/v7.0-rc7/source/drivers/media/platform/qcom/iris/iris_vpu_buffer.c#L2157

Regards,
Vikash

Quoted 17 lines

An overhead can mean anything.

Quoted 12 lines
drivers/media/platform/qcom/iris/iris_vpu_buffer.h renamed from drivers/media/platform/qcom/iris/iris_vpu_buffer.h +0 -1
   #define SIZE_AV1D_SEQUENCE_HEADER 768
   #define SIZE_AV1D_METADATA        512
   #define SIZE_AV1D_FRAME_HEADER    1280
 
---

base-commit: 4fbeef21f5387234111b5d52924e77757626faa5
change-id: 20260421-optimize_comv_buffer-ae7107673609

Best regards,

Signature · 1 line
Vishnu Reddy <[email protected]>
#3 lore raw
Quoted 143 lines · Vikash Garodia
Quoted 26 lines · Bryan O'Donoghue
Quoted 4 lines · Vishnu Reddy

The existing iris_vpu_dec_comv_size() used VIDEO_MAX_FRAME (32) as num_comv count unconditionally when calculating the COMV buffer size. This resulted in an oversized COMV buffer allocation throughout decode session, wasting memory regardless of actual number of buffers required.

You should define what a COMV buffer is before talking about how you are changing it, i.e. define the term Co-located Motion Vector (CMOV) and then use the abbreviation CMOV liberally as you wish.

Quoted 2 lines

For VPU3x and VPU4x platforms, introduce iris_vpu3x_4x_dec_comv_size() to replace iris_vpu_dec_comv_size(). It derives num_comv dynamically, it

"These derive num_cmove dynamically"

Quoted 11 lines

uses inst->fw_min_count once the firmware has reported its minimum buffer requirements, and fallback to inst->buffers[BUF_OUTPUT].min_count during initialization before firmware has communicated its requirements. This aligns the COMV buffer size to the actual count needed rather than always allocating with fixed VIDEO_MAX_FRAME value.

Additionally, during iris_vdec_inst_init(), fw_min_count was initialized to MIN_BUFFERS instead of 0. This masked the fallback logic and caused the COMV size calculation to use MIN_BUFFERS even before firmware had reported its actual requirements. Fix this by initializing fw_min_count to 0.

Quoted 5 lines
Quoted 2 lines

During testing of 1080p AVC, it reduces the COMV buffer size from 32.89MB to 6.16MB per decode session, significantly reducing memory consumption.

Cool nice fix.

Indeed, a good saving.

Quoted 93 lines
Quoted 39 lines

---
  drivers/media/platform/qcom/iris/iris_vdec.c       |  2 +- drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 24 ++++++++++++
+++++++---
  drivers/media/platform/qcom/iris/iris_vpu_buffer.h |  1 +

  3 files changed, 23 insertions(+), 4 deletions(-)
drivers/ renamed from drivers/media/platform/qcom/iris/iris_vdec.c +0 -0

media/platform/qcom/iris/iris_vdec.c
index 719217399a30..f433065e08b2 100644

drivers/media/platform/qcom/iris/iris_vdec.c +1 -1
@@ -24,7 +24,7 @@ int iris_vdec_inst_init(struct iris_inst *inst)
       inst->fmt_src = kzalloc_obj(*inst->fmt_src);
       inst->fmt_dst = kzalloc_obj(*inst->fmt_dst);
 
-    inst->fw_min_count = MIN_BUFFERS;
+    inst->fw_min_count = 0;
 
       f = inst->fmt_src;
       f->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
(quoted diff) renamed from drivers/media/platform/qcom/iris/iris_vpu_buffer.c +0 -0

drivers/media/platform/qcom/iris/iris_vpu_buffer.c index 9270422c1601..57237543b229 100644

drivers/media/platform/qcom/iris/iris_vpu_buffer.c +7 -0
@@ -731,6 +731,23 @@ static u32 iris_vpu_dec_comv_size(struct iris_inst *inst)
       u32 height = f->fmt.pix_mp.height;
       u32 width = f->fmt.pix_mp.width;
 
+    if (inst->codec == V4L2_PIX_FMT_H264)
+        return hfi_buffer_comv_h264d(width, height, num_comv);
+    else if (inst->codec == V4L2_PIX_FMT_HEVC)
+        return hfi_buffer_comv_h265d(width, height, num_comv);
+
+    return 0;
+}

drivers/media/platform/qcom/iris/iris_vpu_buffer.c static u32 iris_vpu_dec_comv_size(struct iris_inst *inst) · line 750

Quoted 10 lines
drivers/media/platform/qcom/iris/iris_vpu_buffer.c +9 -0
+
+static u32 iris_vpu3x_4x_dec_comv_size(struct iris_inst *inst)
+{
+    struct v4l2_format *f = inst->fmt_src;
+    u32 height = f->fmt.pix_mp.height;
+    u32 width = f->fmt.pix_mp.width;
+    u32 num_comv;
+
+    num_comv = inst->fw_min_count ? inst->fw_min_count : inst-
 >buffers[BUF_OUTPUT].min_count;

Please just if/else this though its far easier to read/understand that way.

Quoted 41 lines
drivers/media/platform/qcom/iris/iris_vpu_buffer.c +1 -0
+
       if (inst->codec == V4L2_PIX_FMT_H264)
           return hfi_buffer_comv_h264d(width, height, num_comv);

      else if (inst->codec == V4L2_PIX_FMT_HEVC)

drivers/media/platform/qcom/iris/iris_vpu_buffer.c +0 -0
@@ -739,7 +756,8 @@ static u32 iris_vpu_dec_comv_size(struct iris_inst *inst)
           if (inst->fw_caps[DRAP].value)
               return 0;

          else
-            return hfi_buffer_comv_av1d(width, height, num_comv); +            return hfi_buffer_comv_av1d(width, height, +                            num_comv + AV1D_COMV_BUFFER_OVERHEAD); }

      return 0;

drivers/media/platform/qcom/iris/iris_vpu_buffer.c +0 -0
@@ -2025,7 +2043,7 @@ u32 iris_vpu_buf_size(struct iris_inst *inst, enum
 iris_buffer_type buffer_type)
 

      static const struct iris_vpu_buf_type_handle dec_internal_buf_type_handle[] = {
          {BUF_BIN,         iris_vpu_dec_bin_size             }, -        {BUF_COMV,        iris_vpu_dec_comv_size            }, +        {BUF_COMV,        iris_vpu3x_4x_dec_comv_size       }, {BUF_NON_COMV,    iris_vpu_dec_non_comv_size        }, {BUF_LINE,        iris_vpu_dec_line_size            }, {BUF_PERSIST,     iris_vpu_dec_persist_size         },

drivers/media/platform/qcom/iris/iris_vpu_buffer.c +0 -0
@@ -2098,7 +2116,7 @@ u32 iris_vpu4x_buf_size(struct iris_inst *inst, enum

iris_buffer_type buffer_typ

      static const struct iris_vpu_buf_type_handle dec_internal_buf_type_handle[] = {
          {BUF_BIN,         iris_vpu_dec_bin_size         }, -        {BUF_COMV,        iris_vpu_dec_comv_size        }, +        {BUF_COMV,        iris_vpu3x_4x_dec_comv_size   }, {BUF_NON_COMV,    iris_vpu_dec_non_comv_size    }, {BUF_LINE,        iris_vpu4x_dec_line_size      }, {BUF_PERSIST,     iris_vpu4x_dec_persist_size   },

(quoted diff) renamed from drivers/media/platform/qcom/iris/iris_vpu_buffer.h +0 -0

drivers/media/platform/qcom/iris/iris_vpu_buffer.h index 12640eb5ed8c..7a9cc1c92da3 100644

drivers/media/platform/qcom/iris/iris_vpu_buffer.h +0 -0
Quoted 8 lines

drivers/media/platform/qcom/iris/iris_vpu_buffer.h struct iris_inst; · line 113

Quoted 5 lines
drivers/media/platform/qcom/iris/iris_vpu_buffer.h +1 -0
@@ -110,6 +110,7 @@ struct iris_inst;
   #define MAX_PE_NBR_DATA_LCU16_LINE_BUFFER_SIZE 96
   #define AV1D_NUM_HW_PIC_BUF    16
   #define AV1D_NUM_FRAME_HEADERS 16
+#define AV1D_COMV_BUFFER_OVERHEAD 7

Whats this ? Why is there a new seven byte overhead ? Does it represent a header, an alignment ?

Vishnu, pls check if we need to add this as initial count was 18 [1] ? What if initial count was 11 [2], and post DRC, fw_min_count would be 11 too for AV1d, so the overhead can be avoided.

Quoted 6 lines

[1]
https://elixir.bootlin.com/linux/v7.0-rc7/source/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c#L1220

[2]
https://elixir.bootlin.com/linux/v7.0-rc7/source/drivers/media/platform/qcom/iris/iris_vpu_buffer.c#L2157

Thanks for the information, will check and update in next revision.

Quoted 20 lines

Regards,
Vikash

Quoted 17 lines

An overhead can mean anything.

Quoted 12 lines
drivers/media/platform/qcom/iris/iris_vpu_buffer.h +0 -1
   #define SIZE_AV1D_SEQUENCE_HEADER 768
   #define SIZE_AV1D_METADATA        512
   #define SIZE_AV1D_FRAME_HEADER    1280
 
---

base-commit: 4fbeef21f5387234111b5d52924e77757626faa5
change-id: 20260421-optimize_comv_buffer-ae7107673609

Best regards,
--
Vishnu Reddy <[email protected]>

#4 lore raw
In reply to Bryan O'Donoghue
Quoted 9 lines · Bryan O'Donoghue
Quoted 4 lines · Vishnu Reddy

The existing iris_vpu_dec_comv_size() used VIDEO_MAX_FRAME (32) as num_comv count unconditionally when calculating the COMV buffer size. This resulted in an oversized COMV buffer allocation throughout decode session, wasting memory regardless of actual number of buffers required.

You should define what a COMV buffer is before talking about how you are changing it, i.e. define the term Co-located Motion Vector (CMOV) and then use the abbreviation CMOV liberally as you wish.

Ack, will update accordingly.

Quoted 5 lines
Quoted 3 lines

For VPU3x and VPU4x platforms, introduce iris_vpu3x_4x_dec_comv_size() to replace iris_vpu_dec_comv_size(). It derives num_comv dynamically, it

"These derive num_cmove dynamically"

Ack.

Quoted 12 lines
Quoted 12 lines

uses inst->fw_min_count once the firmware has reported its minimum buffer requirements, and fallback to inst->buffers[BUF_OUTPUT].min_count during initialization before firmware has communicated its requirements. This aligns the COMV buffer size to the actual count needed rather than always allocating with fixed VIDEO_MAX_FRAME value.

Additionally, during iris_vdec_inst_init(), fw_min_count was initialized to MIN_BUFFERS instead of 0. This masked the fallback logic and caused the COMV size calculation to use MIN_BUFFERS even before firmware had reported its actual requirements. Fix this by initializing fw_min_count to 0.

Quoted 5 lines
Quoted 2 lines

During testing of 1080p AVC, it reduces the COMV buffer size from 32.89MB to 6.16MB per decode session, significantly reducing memory consumption.

Cool nice fix.

Thanks!

Quoted 42 lines
Quoted 42 lines

---
  drivers/media/platform/qcom/iris/iris_vdec.c       |  2 +- drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 24 +++++++++++++++++++--- drivers/media/platform/qcom/iris/iris_vpu_buffer.h |  1 +

  3 files changed, 23 insertions(+), 4 deletions(-)
drivers/media/platform/qcom/iris/iris_vdec.c +1 -1
index 719217399a30..f433065e08b2 100644
@@ -24,7 +24,7 @@ int iris_vdec_inst_init(struct iris_inst *inst)
       inst->fmt_src = kzalloc_obj(*inst->fmt_src);
       inst->fmt_dst = kzalloc_obj(*inst->fmt_dst);
 
-    inst->fw_min_count = MIN_BUFFERS;
+    inst->fw_min_count = 0;
 
       f = inst->fmt_src;
       f->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
drivers/media/platform/qcom/iris/iris_vpu_buffer.c +11 -0
index 9270422c1601..57237543b229 100644
@@ -731,6 +731,23 @@ static u32 iris_vpu_dec_comv_size(struct iris_inst *inst)
       u32 height = f->fmt.pix_mp.height;
       u32 width = f->fmt.pix_mp.width;
 
+    if (inst->codec == V4L2_PIX_FMT_H264)
+        return hfi_buffer_comv_h264d(width, height, num_comv);
+    else if (inst->codec == V4L2_PIX_FMT_HEVC)
+        return hfi_buffer_comv_h265d(width, height, num_comv);
+
+    return 0;
+}
+
+static u32 iris_vpu3x_4x_dec_comv_size(struct iris_inst *inst)
+{
+    struct v4l2_format *f = inst->fmt_src;
Quoted 9 lines

drivers/media/platform/qcom/iris/iris_vpu_buffer.c static u32 iris_vpu_dec_comv_size(struct iris_inst *inst)

Quoted 6 lines
drivers/media/platform/qcom/iris/iris_vpu_buffer.c +5 -0
+    u32 height = f->fmt.pix_mp.height;
+    u32 width = f->fmt.pix_mp.width;
+    u32 num_comv;
+
+    num_comv = inst->fw_min_count ? inst->fw_min_count :
 inst->buffers[BUF_OUTPUT].min_count;

Please just if/else this though its far easier to read/understand that way.

Ack.

Thank you
Vishnu Reddy

Quoted 66 lines
Quoted 42 lines
drivers/media/platform/qcom/iris/iris_vpu_buffer.c +1 -0
+
       if (inst->codec == V4L2_PIX_FMT_H264)
           return hfi_buffer_comv_h264d(width, height, num_comv);

      else if (inst->codec == V4L2_PIX_FMT_HEVC)

drivers/media/platform/qcom/iris/iris_vpu_buffer.c +0 -0
@@ -739,7 +756,8 @@ static u32 iris_vpu_dec_comv_size(struct iris_inst *inst)
           if (inst->fw_caps[DRAP].value)
               return 0;

          else
-            return hfi_buffer_comv_av1d(width, height, num_comv); +            return hfi_buffer_comv_av1d(width, height, +                            num_comv + AV1D_COMV_BUFFER_OVERHEAD); }

      return 0;

drivers/media/platform/qcom/iris/iris_vpu_buffer.c +0 -0
@@ -2025,7 +2043,7 @@ u32 iris_vpu_buf_size(struct iris_inst *inst, enum
 iris_buffer_type buffer_type)
 

      static const struct iris_vpu_buf_type_handle dec_internal_buf_type_handle[] = {
          {BUF_BIN,         iris_vpu_dec_bin_size             }, -        {BUF_COMV,        iris_vpu_dec_comv_size            }, +        {BUF_COMV,        iris_vpu3x_4x_dec_comv_size       }, {BUF_NON_COMV,    iris_vpu_dec_non_comv_size        }, {BUF_LINE,        iris_vpu_dec_line_size            }, {BUF_PERSIST,     iris_vpu_dec_persist_size         },

drivers/media/platform/qcom/iris/iris_vpu_buffer.c +0 -0
@@ -2098,7 +2116,7 @@ u32 iris_vpu4x_buf_size(struct iris_inst *inst, enum

iris_buffer_type buffer_typ

      static const struct iris_vpu_buf_type_handle dec_internal_buf_type_handle[] = {
          {BUF_BIN,         iris_vpu_dec_bin_size         }, -        {BUF_COMV,        iris_vpu_dec_comv_size        }, +        {BUF_COMV,        iris_vpu3x_4x_dec_comv_size   }, {BUF_NON_COMV,    iris_vpu_dec_non_comv_size    }, {BUF_LINE,        iris_vpu4x_dec_line_size      }, {BUF_PERSIST,     iris_vpu4x_dec_persist_size   },

drivers/media/platform/qcom/iris/iris_vpu_buffer.h +0 -0
index 12640eb5ed8c..7a9cc1c92da3 100644

drivers/media/platform/qcom/iris/iris_vpu_buffer.h struct iris_inst; · line 113

Quoted 4 lines
drivers/media/platform/qcom/iris/iris_vpu_buffer.h +1 -0
@@ -110,6 +110,7 @@ struct iris_inst;
   #define MAX_PE_NBR_DATA_LCU16_LINE_BUFFER_SIZE 96
   #define AV1D_NUM_HW_PIC_BUF    16
   #define AV1D_NUM_FRAME_HEADERS 16
+#define AV1D_COMV_BUFFER_OVERHEAD 7

Whats this ? Why is there a new seven byte overhead ? Does it represent a header, an alignment ?

An overhead can mean anything.

Quoted 12 lines
drivers/media/platform/qcom/iris/iris_vpu_buffer.h +0 -1
   #define SIZE_AV1D_SEQUENCE_HEADER 768
   #define SIZE_AV1D_METADATA        512
   #define SIZE_AV1D_FRAME_HEADER    1280
 
---

base-commit: 4fbeef21f5387234111b5d52924e77757626faa5
change-id: 20260421-optimize_comv_buffer-ae7107673609

Best regards,

Signature · 1 line
Vishnu Reddy <[email protected]>