Skip to contentSkip to conversation

media: qcom: iris: add dynamic encoder properties

PATCH Superseded Wangao Wang 3 participants 8 messages Jun 4 – Jun 23

Subsystems (4)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v2 cover media: qcom: iris: add dynamic encoder properties

This patch series introduces several enhancements to the Qualcomm Iris encoder driver, improving support for V4L2 controls and enabling more video encoding features.

All patches have been tested with iris_test_app on gen1:QCS6490, gen2:QCS9100, X1E80100.

---
Changes in v2:
- Rebased due to a conflict with commit 'media: qcom: iris: split HFI session ops from core ops' (35da0884068226ca3a53371dbf685db6e0d74658).
- Link to v1: https://lore.kernel.org/r/[email protected]

---
Wangao Wang (2):
media: qcom: iris: Add gop size support for gen1 encoder
media: qcom: iris: Add request key frame support for encoder

 drivers/media/platform/qcom/iris/iris_ctrls.c      | 24 +++++++++++++++++++-
 drivers/media/platform/qcom/iris/iris_ctrls.h      |  1 +
 drivers/media/platform/qcom/iris/iris_hfi_gen1.c   | 26 ++++++++++++----------
 .../platform/qcom/iris/iris_hfi_gen1_command.c     |  3 +++
 .../platform/qcom/iris/iris_hfi_gen1_defines.h     |  1 +
 drivers/media/platform/qcom/iris/iris_hfi_gen2.c   | 12 +++++++++-
 .../platform/qcom/iris/iris_hfi_gen2_defines.h     |  7 ++++++
 .../platform/qcom/iris/iris_platform_common.h      |  2 +-
 8 files changed, 61 insertions(+), 15 deletions(-)

---
base-commit: a225caacc36546a09586e3ece36c0313146e7da9
change-id: 20260417-dynamic_encode-3751a6360ed4

Best regards,

Signature · 1 line
Wangao Wang <[email protected]>
#1 lore raw

PATCH v2 1/2 media: qcom: iris: Add gop size support for gen1 encoder

Add support for gop size configuration on gen1 encoder by enabling

V4L2_CID_MPEG_VIDEO_GOP_SIZE.

---
 drivers/media/platform/qcom/iris/iris_ctrls.c           |  2 +-
 drivers/media/platform/qcom/iris/iris_hfi_gen1.c        | 16 ++++------------
 drivers/media/platform/qcom/iris/iris_platform_common.h |  1 -
 3 files changed, 5 insertions(+), 14 deletions(-)
drivers/media/platform/qcom/iris/iris_ctrls.c +1 -1
index 10e33b8a73f60759c4f1cb17b5c95897f0e1468f..391e1fc5f6e2ec8e9cf5ba4e0f76b2d1da3e2a35 100644
@@ -1293,7 +1293,7 @@ int iris_set_use_and_mark_ltr(struct iris_inst *inst, enum platform_inst_fw_cap_
 int iris_set_intra_period(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_id)
 {
 	const struct iris_hfi_session_ops *hfi_ops = inst->hfi_session_ops;
-	u32 gop_size = inst->fw_caps[GOP_SIZE].value;
+	u32 gop_size = inst->fw_caps[cap_id].value;
 	u32 b_frame = inst->fw_caps[B_FRAME].value;
 	u32 hfi_id = inst->fw_caps[cap_id].hfi_id;
 	struct hfi_intra_period intra_period;
drivers/media/platform/qcom/iris/iris_hfi_gen1.c +4 -12
index ca1545d28b5310d8c45a905287dcb69f6184d9cc..eff9216bb50143c0d752051a6400605bbaf3803e 100644
@@ -171,7 +171,9 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = {
 		.max = (1 << 16) - 1,
 		.step_or_mask = 1,
 		.value = 30,
-		.set = iris_set_u32
+		.hfi_id = HFI_PROPERTY_CONFIG_VENC_INTRA_PERIOD,
+		.flags = CAP_FLAG_OUTPUT_PORT,
+		.set = iris_set_intra_period,
 	},
 	{
 		.cap_id = ENTROPY_MODE,
@@ -240,7 +242,7 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = {
 		.step_or_mask = 1,
 		.value = 0,
 		.hfi_id = HFI_PROPERTY_PARAM_VENC_INTRA_REFRESH,
-		.flags = CAP_FLAG_OUTPUT_PORT,
+		.flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED,
 		.set = iris_set_ir_period_gen1,
 	},
 	{
@@ -281,16 +283,6 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = {
 		.value = 0,
 		.flags = CAP_FLAG_OUTPUT_PORT,
 	},
-	{
-		.cap_id = INTRA_PERIOD,
-		.min = 0,
-		.max = 1,
-		.step_or_mask = 1,
-		.value = 0,
-		.hfi_id = HFI_PROPERTY_CONFIG_VENC_INTRA_PERIOD,
-		.flags = CAP_FLAG_OUTPUT_PORT,
-		.set = iris_set_intra_period,
-	},
 	{
 		.cap_id = LAYER_ENABLE,
 		.min = 0,
drivers/media/platform/qcom/iris/iris_platform_common.h +0 -1
index c9256f2323dc4521f9eacaeffb0fc08a180de3ff..ff48333ad089894c3393e8ad45903c2603288eb9 100644
@@ -165,7 +165,6 @@ enum platform_inst_fw_cap_type {
 	USE_LTR,
 	MARK_LTR,
 	B_FRAME,
-	INTRA_PERIOD,
 	LAYER_ENABLE,
 	LAYER_TYPE_H264,
 	LAYER_TYPE_HEVC,
Signature · 1 line
2.43.0
#2 lore raw

Re: [PATCH v2 1/2] media: qcom: iris: Add gop size support for gen1 encoder

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

New issues:
- [Medium] Undocumented addition of dynamic allowed flag to IR_PERIOD capability.

Pre-existing issues:
- [High] The driver lacks a .try_ctrl implementation to validate dependencies between controls, resulting in silent hardware configuration failures for valid V4L2 inputs like gop_size == 0. --

commit 0331fb3580ba43928b379274c9bd300bf884fc44
Author: Wangao Wang <[email protected]>

media: qcom: iris: Add gop size support for gen1 encoder

This commit adds support for GOP size configuration on gen1 encoders by enabling V4L2_CID_MPEG_VIDEO_GOP_SIZE and routing it to the hardware handler.

drivers/media/platform/qcom/iris/iris_hfi_gen1.c static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { · line 177

Quoted 13 lines
drivers/media/platform/qcom/iris/iris_hfi_gen1.c +3 -1
index ca1545d28b531..eff9216bb5014 100644
@@ -171,7 +171,9 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = {
 		.max = (1 << 16) - 1,
 		.step_or_mask = 1,
 		.value = 30,
-		.set = iris_set_u32
+		.hfi_id = HFI_PROPERTY_CONFIG_VENC_INTRA_PERIOD,
+		.flags = CAP_FLAG_OUTPUT_PORT,
+		.set = iris_set_intra_period,
 	},

[Severity: High]
This isn't a bug introduced by this patch, but this patch explicitly wires up the GOP_SIZE control to the hardware for Gen1, bringing a pre-existing silent failure path into active use.

The V4L2 specification defines V4L2_CID_MPEG_VIDEO_GOP_SIZE = 0 as meaning "only I-frames are used". The driver exposes .min = 0 for this control, allowing userspace to request it.

However, the hardware handler explicitly rejects a 0 gop_size:

iris_set_intra_period() {
    ...
    if (!gop_size || b_frame >= gop_size)
        return -EINVAL;
    ...
}

During streamon, the driver iterates over capabilities and applies them to the firmware, but ignores any returned errors:

iris_set_properties() {
    ...
    if (cap->cap_id && cap->set)
        cap->set(inst, i);
    ...
}

Because the return value is ignored, the error is swallowed and user space receives a success response, but the property is never dispatched to the firmware. Could this cause the encoder hardware to operate with out-of-sync default GOP settings instead of the requested all-I-frames configuration?

[ ... ]

drivers/media/platform/qcom/iris/iris_hfi_gen1.c static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { · line 247

Quoted 8 lines
drivers/media/platform/qcom/iris/iris_hfi_gen1.c +1 -1
@@ -240,7 +242,7 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = {
 		.step_or_mask = 1,
 		.value = 0,
 		.hfi_id = HFI_PROPERTY_PARAM_VENC_INTRA_REFRESH,
-		.flags = CAP_FLAG_OUTPUT_PORT,
+		.flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED,
 		.set = iris_set_ir_period_gen1,
 	},

[Severity: Medium]
Does this modification belong in this patch?

The commit message strictly states that it adds support for GOP size configuration on gen1 encoders. However, this line modifies the IR_PERIOD (Intra Refresh Period) capability to include the CAP_FLAG_DYNAMIC_ALLOWED flag.

Modifying intra-refresh dynamic behavior appears to be a distinct functional change that isn't mentioned in the commit message. Could this be split into a separate commit or mentioned in the commit message?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#3 lore raw

Re: [PATCH v2 1/2] media: qcom: iris: Add gop size support for gen1 encoder

In reply to Wangao Wang
Quoted 3 lines · Wangao Wang

Add support for gop size configuration on gen1 encoder by enabling

V4L2_CID_MPEG_VIDEO_GOP_SIZE.

Extra empty line within a sentence. Instead please explain, why INTRA_PERIOD is being merged into GOP_SIZE cap.

Quoted 13 lines
---
 drivers/media/platform/qcom/iris/iris_ctrls.c           |  2 +-
 drivers/media/platform/qcom/iris/iris_hfi_gen1.c        | 16 ++++------------
 drivers/media/platform/qcom/iris/iris_platform_common.h |  1 -
 3 files changed, 5 insertions(+), 14 deletions(-)
drivers/media/platform/qcom/iris/iris_ctrls.c +0 -0
index 10e33b8a73f60759c4f1cb17b5c95897f0e1468f..391e1fc5f6e2ec8e9cf5ba4e0f76b2d1da3e2a35 100644

drivers/media/platform/qcom/iris/iris_ctrls.c int iris_set_use_and_mark_ltr(struct iris_inst *inst, enum platform_inst_fw_cap_ · line 1296

Quoted 5 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +1 -1
@@ -1293,7 +1293,7 @@ int iris_set_use_and_mark_ltr(struct iris_inst *inst, enum platform_inst_fw_cap_
 int iris_set_intra_period(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_id)
 {
 	const struct iris_hfi_session_ops *hfi_ops = inst->hfi_session_ops;
-	u32 gop_size = inst->fw_caps[GOP_SIZE].value;
+	u32 gop_size = inst->fw_caps[cap_id].value;

Why? I think it was good as is.

Quoted 8 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +0 -0
 	u32 b_frame = inst->fw_caps[B_FRAME].value;
 	u32 hfi_id = inst->fw_caps[cap_id].hfi_id;
 	struct hfi_intra_period intra_period;
drivers/media/platform/qcom/iris/iris_hfi_gen1.c +0 -0
index ca1545d28b5310d8c45a905287dcb69f6184d9cc..eff9216bb50143c0d752051a6400605bbaf3803e 100644

drivers/media/platform/qcom/iris/iris_hfi_gen1.c static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { · line 176

Quoted 7 lines
drivers/media/platform/qcom/iris/iris_hfi_gen1.c +3 -1
@@ -171,7 +171,9 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = {
 		.max = (1 << 16) - 1,
 		.step_or_mask = 1,
 		.value = 30,
-		.set = iris_set_u32
+		.hfi_id = HFI_PROPERTY_CONFIG_VENC_INTRA_PERIOD,
+		.flags = CAP_FLAG_OUTPUT_PORT,
+		.set = iris_set_intra_period,

This is the GOP_SIZE capability, so you will get it as cap_id.

Quoted 3 lines
drivers/media/platform/qcom/iris/iris_hfi_gen1.c +0 -0
 	},
 	{
 		.cap_id = ENTROPY_MODE,
Signature · 2 lines
With best wishes
Dmitry
#4 lore raw

PATCH v2 2/2 media: qcom: iris: Add request key frame support for encoder

In reply to Wangao Wang

Add request key frame support for both gen1 and gen2 encoders by enabling

V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME.

---
 drivers/media/platform/qcom/iris/iris_ctrls.c      | 22 ++++++++++++++++++++++
 drivers/media/platform/qcom/iris/iris_ctrls.h      |  1 +
 drivers/media/platform/qcom/iris/iris_hfi_gen1.c   | 10 ++++++++++
 .../platform/qcom/iris/iris_hfi_gen1_command.c     |  3 +++
 .../platform/qcom/iris/iris_hfi_gen1_defines.h     |  1 +
 drivers/media/platform/qcom/iris/iris_hfi_gen2.c   | 12 +++++++++++-
 .../platform/qcom/iris/iris_hfi_gen2_defines.h     |  7 +++++++
 .../platform/qcom/iris/iris_platform_common.h      |  1 +
 8 files changed, 56 insertions(+), 1 deletion(-)
drivers/media/platform/qcom/iris/iris_ctrls.c +22 -0
index 391e1fc5f6e2ec8e9cf5ba4e0f76b2d1da3e2a35..2918a7df0b403a886648e90ec00968f51c4660a0 100644
@@ -154,6 +154,8 @@ static enum platform_inst_fw_cap_type iris_get_cap_id(u32 id)
 		return LAYER4_BITRATE_HEVC;
 	case V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L5_BR:
 		return LAYER5_BITRATE_HEVC;
+	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:
+		return REQUEST_SYNC_FRAME;
 	default:
 		return INST_FW_CAP_MAX;
 	}
@@ -297,6 +299,8 @@ static u32 iris_get_v4l2_id(enum platform_inst_fw_cap_type cap_id)
 		return V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L4_BR;
 	case LAYER5_BITRATE_HEVC:
 		return V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L5_BR;
+	case REQUEST_SYNC_FRAME:
+		return V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME;
 	default:
 		return 0;
 	}
@@ -1477,6 +1481,24 @@ int iris_set_layer_bitrate(struct iris_inst *inst, enum platform_inst_fw_cap_typ
 					     &bitrate, sizeof(u32));
 }
 
+int iris_set_req_sync_frame(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_id)
+{
+	const struct iris_hfi_session_ops *hfi_ops = inst->hfi_session_ops;
+	u32 hfi_id = inst->fw_caps[cap_id].hfi_id;
+	u32 hfi_val = 0;
+
+	if (inst->fw_caps[PREPEND_SPSPPS_TO_IDR].value)
+		hfi_val = HFI_SYNC_FRAME_REQUEST_WITH_PREFIX_SEQ_HDR;
+	else
+		hfi_val = HFI_SYNC_FRAME_REQUEST_WITHOUT_SEQ_HDR;
+
+	return hfi_ops->session_set_property(inst, hfi_id,
+					     HFI_HOST_FLAGS_NONE,
+					     iris_get_port_info(inst, cap_id),
+					     HFI_PAYLOAD_U32_ENUM,
+					     &hfi_val, sizeof(u32));
+}
+
 int iris_set_properties(struct iris_inst *inst, u32 plane)
 {
 	const struct iris_hfi_session_ops *hfi_ops = inst->hfi_session_ops;
drivers/media/platform/qcom/iris/iris_ctrls.h +1 -0
index 3c462ec9190be8935176b290588f224fe4f144a4..8d8bbef960076c08e5af0f65920dde187f24a313 100644
@@ -47,6 +47,7 @@ int iris_set_layer_type(struct iris_inst *inst, enum platform_inst_fw_cap_type c
 int iris_set_layer_count_gen1(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_id);
 int iris_set_layer_count_gen2(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_id);
 int iris_set_layer_bitrate(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_id);
+int iris_set_req_sync_frame(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_id);
 int iris_set_properties(struct iris_inst *inst, u32 plane);
 
 #endif
drivers/media/platform/qcom/iris/iris_hfi_gen1.c +10 -0
index eff9216bb50143c0d752051a6400605bbaf3803e..04c78417dbae33dff1cc7099e6f105caa606048e 100644
@@ -375,6 +375,16 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = {
 			CAP_FLAG_DYNAMIC_ALLOWED,
 		.set = iris_set_bitrate_gen1,
 	},
+	{
+		.cap_id = REQUEST_SYNC_FRAME,
+		.min = 0,
+		.max = 0,
+		.step_or_mask = 0,
+		.value = 0,
+		.hfi_id = HFI_PROPERTY_CONFIG_VENC_REQUEST_SYNC_FRAME,
+		.flags = CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED,
+		.set = iris_set_req_sync_frame,
+	},
 };
 
 static const u32 sm8250_vdec_input_config_param_default[] = {
drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +3 -0
index 7674b47ad6c49adfaf16b444c8ec7295b230a529..d7fd867cf6c98523123e9205fcfe4b469708da72 100644
@@ -741,6 +741,9 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p
 		packet->shdr.hdr.size += sizeof(u32);
 		break;
 	}
+	case HFI_PROPERTY_CONFIG_VENC_REQUEST_SYNC_FRAME: {
+		break;
+	}
 	default:
 		return -EINVAL;
 	}
drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h +1 -0
index 0e4dee19238464a9671a94eaab8eeda2d7f7ca9f..275f3fea3d2aabe26a2a0498c78c09ec28765b24 100644
@@ -157,6 +157,7 @@
 #define HFI_PROPERTY_PARAM_VENC_HIER_P_MAX_NUM_ENH_LAYER	0x2005026
 #define HFI_PROPERTY_CONFIG_VENC_TARGET_BITRATE			0x2006001
 #define HFI_PROPERTY_CONFIG_VENC_INTRA_PERIOD			0x2006003
+#define HFI_PROPERTY_CONFIG_VENC_REQUEST_SYNC_FRAME		0x2006004
 #define HFI_PROPERTY_CONFIG_VENC_MARKLTRFRAME			0x2006009
 #define HFI_PROPERTY_CONFIG_VENC_USELTRFRAME			0x200600a
 #define HFI_PROPERTY_CONFIG_VENC_SYNC_FRAME_SEQUENCE_HEADER	0x2006008
drivers/media/platform/qcom/iris/iris_hfi_gen2.c +11 -1
index acc0ed8adda1ae726c94ab6346b0b4b2b3db1d7b..df8e70b133a12e513aab04427ea96fca22055fcf 100644
@@ -937,7 +937,17 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8550_enc[] = {
 		.flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
 			CAP_FLAG_DYNAMIC_ALLOWED,
 		.set = iris_set_layer_bitrate,
-	}
+	},
+	{
+		.cap_id = REQUEST_SYNC_FRAME,
+		.min = 0,
+		.max = 0,
+		.step_or_mask = 0,
+		.value = 0,
+		.hfi_id = HFI_PROP_REQUEST_SYNC_FRAME,
+		.flags = CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED,
+		.set = iris_set_req_sync_frame,
+	},
 };
 
 static const u32 sm8550_vdec_input_config_params_default[] = {
drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h +7 -0
index 776b21cd11b2cd5555cbced8e438cb32e87a539c..a8b47e4ceba0fc868125f841a5635476571aab85 100644
@@ -90,6 +90,13 @@ enum hfi_layer_encoding_type {
 #define HFI_PROP_BITRATE_LAYER4			0x0300013f
 #define HFI_PROP_BITRATE_LAYER5			0x03000140
 #define HFI_PROP_BITRATE_LAYER6			0x03000141
+
+enum hfi_syncframe_request_mode {
+	HFI_SYNC_FRAME_REQUEST_WITHOUT_SEQ_HDR		= 0x00000001,
+	HFI_SYNC_FRAME_REQUEST_WITH_PREFIX_SEQ_HDR	= 0x00000002,
+};
+
+#define HFI_PROP_REQUEST_SYNC_FRAME		0x03000145
 #define HFI_PROP_MAX_GOP_FRAMES			0x03000146
 #define HFI_PROP_MAX_B_FRAMES			0x03000147
 #define HFI_PROP_QUALITY_MODE			0x03000148
drivers/media/platform/qcom/iris/iris_platform_common.h +1 -0
index ff48333ad089894c3393e8ad45903c2603288eb9..23b6e7676d05bd01725f8e2cfca47b06ef112c27 100644
@@ -182,6 +182,7 @@ enum platform_inst_fw_cap_type {
 	LAYER3_BITRATE_HEVC,
 	LAYER4_BITRATE_HEVC,
 	LAYER5_BITRATE_HEVC,
+	REQUEST_SYNC_FRAME,
 	INST_FW_CAP_MAX,
 };
 
Signature · 1 line
2.43.0
#5 lore raw

Re: [PATCH v2 2/2] media: qcom: iris: Add request key frame support for encoder

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

New issues:
- [High] In Gen1, the payload for HFI_PROPERTY_CONFIG_VENC_REQUEST_SYNC_FRAME is discarded and the packet size is not incremented, resulting in a truncated firmware command packet.
- [Medium] The driver unconditionally requests a sync frame on stream initialization, bypassing V4L2 button control semantics.

Pre-existing issues:
- [High] Concurrent modification of the shared inst_hfi_gen2->packet buffer causes HFI command packet corruption. --

commit 69576d77651c69239afd97ac72082948d7e40043
Author: Wangao Wang <[email protected]>

media: qcom: iris: Add request key frame support for encoder

This commit adds support for requesting key frames for both gen1 and gen2 encoders by enabling the V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME control.

Quoted 4 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +0 -0
index 391e1fc5f6e2e..2918a7df0b403 100644

[ ... ]

Quoted 11 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +7 -0
@@ -1477,6 +1481,24 @@ int iris_set_layer_bitrate(struct iris_inst *inst, enum platform_inst_fw_cap_typ
 					     &bitrate, sizeof(u32));
 }
 
+int iris_set_req_sync_frame(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_id)
+{
+	const struct iris_hfi_session_ops *hfi_ops = inst->hfi_session_ops;
+	u32 hfi_id = inst->fw_caps[cap_id].hfi_id;
+	u32 hfi_val = 0;
+
+	if (inst->fw_caps[PREPEND_SPSPPS_TO_IDR].value)

drivers/media/platform/qcom/iris/iris_ctrls.c int iris_set_layer_bitrate(struct iris_inst *inst, enum platform_inst_fw_cap_typ · line 1500

Quoted 10 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +10 -0
+		hfi_val = HFI_SYNC_FRAME_REQUEST_WITH_PREFIX_SEQ_HDR;
+	else
+		hfi_val = HFI_SYNC_FRAME_REQUEST_WITHOUT_SEQ_HDR;
+
+	return hfi_ops->session_set_property(inst, hfi_id,
+					     HFI_HOST_FLAGS_NONE,
+					     iris_get_port_info(inst, cap_id),
+					     HFI_PAYLOAD_U32_ENUM,
+					     &hfi_val, sizeof(u32));
+}

[Severity: Medium]
Will this unconditionally request a sync frame on stream initialization?

During stream initialization, iris_set_properties() iterates over all fw_caps and invokes their .set callbacks. Because this function lacks a check for inst->fw_caps[cap_id].flags & CAP_FLAG_CLIENT_SET (like bitrate does), it seems it will unconditionally issue a REQUEST_SYNC_FRAME command to the firmware on every stream start, bypassing V4L2 button control semantics.

Quoted 2 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +1 -0
+
 int iris_set_properties(struct iris_inst *inst, u32 plane)

[ ... ]

drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p · line 746

Quoted 11 lines
drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +3 -0
index 7674b47ad6c49..d7fd867cf6c98 100644
@@ -741,6 +741,9 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p
 		packet->shdr.hdr.size += sizeof(u32);
 		break;
 	}
+	case HFI_PROPERTY_CONFIG_VENC_REQUEST_SYNC_FRAME: {
+		break;
+	}

[Severity: High]
Does this result in a truncated HFI packet?

In iris_hfi_gen1_packet_session_set_property(), the switch statement for HFI_PROPERTY_CONFIG_VENC_REQUEST_SYNC_FRAME simply breaks without writing the 4-byte payload or incrementing packet->shdr.hdr.size by sizeof(u32). This omits the payload entirely, which may cause the firmware to reject the request with a parameter error.

Quoted 3 lines
drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +0 -0
 	default:
 		return -EINVAL;
 	}

[ ... ]

Quoted 10 lines
drivers/media/platform/qcom/iris/iris_hfi_gen2.c +1 -1
index acc0ed8adda1a..df8e70b133a12 100644
@@ -937,7 +937,17 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8550_enc[] = {
 		.flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
 			CAP_FLAG_DYNAMIC_ALLOWED,
 		.set = iris_set_layer_bitrate,
-	}
+	},

drivers/media/platform/qcom/iris/iris_hfi_gen2.c static const struct platform_inst_fw_cap inst_fw_cap_sm8550_enc[] = { · line 950

Quoted 10 lines
drivers/media/platform/qcom/iris/iris_hfi_gen2.c +10 -0
+	{
+		.cap_id = REQUEST_SYNC_FRAME,
+		.min = 0,
+		.max = 0,
+		.step_or_mask = 0,
+		.value = 0,
+		.hfi_id = HFI_PROP_REQUEST_SYNC_FRAME,
+		.flags = CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED,
+		.set = iris_set_req_sync_frame,
+	},

[Severity: High]
This is a pre-existing issue, but does adding this dynamic control expose a race condition where the shared packet buffer could be corrupted?

In Gen2, properties and commands are built using a single, per-instance dynamically allocated buffer inst_hfi_gen2->packet. The V4L2 control framework allows dynamic controls, such as the newly added V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME, to be set during streaming via VIDIOC_S_CTRL.

The s_ctrl handler is protected only by the control handler's internal lock (ctrl_handler.lock), while queue-related ioctls like VIDIOC_QBUF and VIDIOC_STREAMON are protected by the queue lock (inst->ctx_q_lock).

When s_ctrl and qbuf are executed concurrently, they can both invoke Gen2 HFI functions which write headers and payload data directly into the shared inst_hfi_gen2->packet. Since there is no common lock protecting inst_hfi_gen2->packet prior to its submission in iris_hfi_queue_cmd_write(), this could result in data races and corrupted HFI packets sent to the firmware.

Quoted 1 line
drivers/media/platform/qcom/iris/iris_hfi_gen2.c +0 -0
 };
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
#6 lore raw

Re: [PATCH v2 2/2] media: qcom: iris: Add request key frame support for encoder

In reply to Wangao Wang
Quoted 58 lines · Wangao Wang

Add request key frame support for both gen1 and gen2 encoders by enabling

V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME.

---
 drivers/media/platform/qcom/iris/iris_ctrls.c      | 22 ++++++++++++++++++++++
 drivers/media/platform/qcom/iris/iris_ctrls.h      |  1 +
 drivers/media/platform/qcom/iris/iris_hfi_gen1.c   | 10 ++++++++++
 .../platform/qcom/iris/iris_hfi_gen1_command.c     |  3 +++
 .../platform/qcom/iris/iris_hfi_gen1_defines.h     |  1 +
 drivers/media/platform/qcom/iris/iris_hfi_gen2.c   | 12 +++++++++++-
 .../platform/qcom/iris/iris_hfi_gen2_defines.h     |  7 +++++++
 .../platform/qcom/iris/iris_platform_common.h      |  1 +
 8 files changed, 56 insertions(+), 1 deletion(-)
drivers/media/platform/qcom/iris/iris_ctrls.c +18 -0
@@ -1477,6 +1481,24 @@ int iris_set_layer_bitrate(struct iris_inst *inst, enum platform_inst_fw_cap_typ
 					     &bitrate, sizeof(u32));
 }
 
+int iris_set_req_sync_frame(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_id)
+{
+	const struct iris_hfi_session_ops *hfi_ops = inst->hfi_session_ops;
+	u32 hfi_id = inst->fw_caps[cap_id].hfi_id;
+	u32 hfi_val = 0;
+
+	if (inst->fw_caps[PREPEND_SPSPPS_TO_IDR].value)
+		hfi_val = HFI_SYNC_FRAME_REQUEST_WITH_PREFIX_SEQ_HDR;
+	else
+		hfi_val = HFI_SYNC_FRAME_REQUEST_WITHOUT_SEQ_HDR;
+
+	return hfi_ops->session_set_property(inst, hfi_id,
+					     HFI_HOST_FLAGS_NONE,
+					     iris_get_port_info(inst, cap_id),
+					     HFI_PAYLOAD_U32_ENUM,
+					     &hfi_val, sizeof(u32));
+}
+
 int iris_set_properties(struct iris_inst *inst, u32 plane)
 {
 	const struct iris_hfi_session_ops *hfi_ops = inst->hfi_session_ops;
drivers/media/platform/qcom/iris/iris_ctrls.h +1 -0
index 3c462ec9190be8935176b290588f224fe4f144a4..8d8bbef960076c08e5af0f65920dde187f24a313 100644
@@ -47,6 +47,7 @@ int iris_set_layer_type(struct iris_inst *inst, enum platform_inst_fw_cap_type c
 int iris_set_layer_count_gen1(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_id);
 int iris_set_layer_count_gen2(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_id);
 int iris_set_layer_bitrate(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_id);
+int iris_set_req_sync_frame(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_id);
 int iris_set_properties(struct iris_inst *inst, u32 plane);
 
 #endif
drivers/media/platform/qcom/iris/iris_hfi_gen1.c +0 -0
index eff9216bb50143c0d752051a6400605bbaf3803e..04c78417dbae33dff1cc7099e6f105caa606048e 100644

drivers/media/platform/qcom/iris/iris_hfi_gen1.c static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { · line 383

Quoted 9 lines
drivers/media/platform/qcom/iris/iris_hfi_gen1.c +6 -0
@@ -375,6 +375,16 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = {
 			CAP_FLAG_DYNAMIC_ALLOWED,
 		.set = iris_set_bitrate_gen1,
 	},
+	{
+		.cap_id = REQUEST_SYNC_FRAME,
+		.min = 0,
+		.max = 0,
+		.step_or_mask = 0,
+		.value = 0,

How does it work if min = max = 0?

Quoted 6 lines
drivers/media/platform/qcom/iris/iris_hfi_gen1.c +4 -0
+		.hfi_id = HFI_PROPERTY_CONFIG_VENC_REQUEST_SYNC_FRAME,
+		.flags = CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED,
+		.set = iris_set_req_sync_frame,
+	},
 };
 
Signature · 2 lines
With best wishes
Dmitry
#7 lore raw

Re: [PATCH v2 2/2] media: qcom: iris: Add request key frame support for encoder

Quoted 9 lines · Dmitry Baryshkov

drivers/media/platform/qcom/iris/iris_hfi_gen1.c static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { · line 383

Quoted 6 lines
drivers/media/platform/qcom/iris/iris_hfi_gen1.c +6 -0
+	{
+		.cap_id = REQUEST_SYNC_FRAME,
+		.min = 0,
+		.max = 0,
+		.step_or_mask = 0,
+		.value = 0,

How does it work if min = max = 0?

Quoted 7 lines

drivers/media/platform/qcom/iris/iris_hfi_gen1.c static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { · line 388

Quoted 6 lines
drivers/media/platform/qcom/iris/iris_hfi_gen1.c +4 -0
+		.hfi_id = HFI_PROPERTY_CONFIG_VENC_REQUEST_SYNC_FRAME,
+		.flags = CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED,
+		.set = iris_set_req_sync_frame,
+	},
  };

When using this cap, the value here is not actually read, so it runs fine. But I think you're right — max should be set to 1. Will be fixed in the next version.

Signature · 2 lines
Best Regards,
Wangao