media: qcom: iris: add dynamic encoder properties
Subsystems (4)
v2 cover media: qcom: iris: add dynamic encoder properties
This patch series introduces several enhancements to the Qualcomm Iris encoder driver, improving support for V4L2 controls and enabling more video encoding features.
All patches have been tested with iris_test_app on gen1:QCS6490, gen2:QCS9100, X1E80100.
---
Changes in v2:
- Rebased due to a conflict with commit 'media: qcom: iris: split HFI session ops from core ops' (35da0884068226ca3a53371dbf685db6e0d74658).
- Link to v1: https://lore.kernel.org/r/[email protected]
---
Wangao Wang (2):
media: qcom: iris: Add gop size support for gen1 encoder
media: qcom: iris: Add request key frame support for encoder
drivers/media/platform/qcom/iris/iris_ctrls.c | 24 +++++++++++++++++++- drivers/media/platform/qcom/iris/iris_ctrls.h | 1 + drivers/media/platform/qcom/iris/iris_hfi_gen1.c | 26 ++++++++++++---------- .../platform/qcom/iris/iris_hfi_gen1_command.c | 3 +++ .../platform/qcom/iris/iris_hfi_gen1_defines.h | 1 + drivers/media/platform/qcom/iris/iris_hfi_gen2.c | 12 +++++++++- .../platform/qcom/iris/iris_hfi_gen2_defines.h | 7 ++++++ .../platform/qcom/iris/iris_platform_common.h | 2 +- 8 files changed, 61 insertions(+), 15 deletions(-)
---
base-commit: a225caacc36546a09586e3ece36c0313146e7da9
change-id: 20260417-dynamic_encode-3751a6360ed4
Best regards,
Signature · 1 line
Wangao Wang <[email protected]>
PATCH v2 1/2 media: qcom: iris: Add gop size support for gen1 encoder
Add support for gop size configuration on gen1 encoder by enabling
V4L2_CID_MPEG_VIDEO_GOP_SIZE.
--- drivers/media/platform/qcom/iris/iris_ctrls.c | 2 +- drivers/media/platform/qcom/iris/iris_hfi_gen1.c | 16 ++++------------ drivers/media/platform/qcom/iris/iris_platform_common.h | 1 - 3 files changed, 5 insertions(+), 14 deletions(-)
drivers/media/platform/qcom/iris/iris_ctrls.c +1 -1
@@ -1293,7 +1293,7 @@ int iris_set_use_and_mark_ltr(struct iris_inst *inst, enum platform_inst_fw_cap_int iris_set_intra_period (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) {const struct iris_hfi_session_ops *hfi_ops = inst->hfi_session_ops; -u32 gop_size = inst->fw_caps[GOP_SIZE ].value; +u32 gop_size = inst->fw_caps[cap_id].value;u32 b_frame = inst->fw_caps[B_FRAME ].value;u32 hfi_id = inst->fw_caps[cap_id].hfi_id;struct hfi_intra_period intra_period;
drivers/media/platform/qcom/iris/iris_hfi_gen1.c +4 -12
@@ -171,7 +171,9 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { .max = (1 << 16) - 1, .step_or_mask = 1, .value = 30, - .set = iris_set_u32 + .hfi_id = HFI_PROPERTY_CONFIG_VENC_INTRA_PERIOD, + .flags = CAP_FLAG_OUTPUT_PORT, + .set = iris_set_intra_period, }, { .cap_id = ENTROPY_MODE,
@@ -240,7 +242,7 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { .step_or_mask = 1, .value = 0, .hfi_id = HFI_PROPERTY_PARAM_VENC_INTRA_REFRESH, - .flags = CAP_FLAG_OUTPUT_PORT, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, .set = iris_set_ir_period_gen1, }, {
@@ -281,16 +283,6 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { .value = 0, .flags = CAP_FLAG_OUTPUT_PORT, }, - { - .cap_id = INTRA_PERIOD, - .min = 0, - .max = 1, - .step_or_mask = 1, - .value = 0, - .hfi_id = HFI_PROPERTY_CONFIG_VENC_INTRA_PERIOD, - .flags = CAP_FLAG_OUTPUT_PORT, - .set = iris_set_intra_period, - }, { .cap_id = LAYER_ENABLE, .min = 0,
drivers/media/platform/qcom/iris/iris_platform_common.h +0 -1
@@ -165,7 +165,6 @@ enum platform_inst_fw_cap_type {USE_LTR ,MARK_LTR ,B_FRAME , -INTRA_PERIOD ,LAYER_ENABLE ,LAYER_TYPE_H264 , LAYER_TYPE_HEVC,
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v2 1/2] media: qcom: iris: Add gop size support for gen1 encoder
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
New issues:
- [Medium] Undocumented addition of dynamic allowed flag to IR_PERIOD capability.
Pre-existing issues:
- [High] The driver lacks a .try_ctrl implementation to validate dependencies between controls, resulting in silent hardware configuration failures for valid V4L2 inputs like gop_size == 0. --
commit 0331fb3580ba43928b379274c9bd300bf884fc44
Author: Wangao Wang <[email protected]>
media: qcom: iris: Add gop size support for gen1 encoder
This commit adds support for GOP size configuration on gen1 encoders by enabling V4L2_CID_MPEG_VIDEO_GOP_SIZE and routing it to the hardware handler.
drivers/media/platform/qcom/iris/iris_hfi_gen1.c static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { · line 177
Quoted 13 lines
drivers/media/platform/qcom/iris/iris_hfi_gen1.c +3
-1@@ -171,7 +171,9 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { .max = (1 << 16) - 1, .step_or_mask = 1, .value = 30, - .set = iris_set_u32 + .hfi_id = HFI_PROPERTY_CONFIG_VENC_INTRA_PERIOD, + .flags = CAP_FLAG_OUTPUT_PORT, + .set = iris_set_intra_period, },
[Severity: High]
This isn't a bug introduced by this patch, but this patch explicitly wires up the GOP_SIZE control to the hardware for Gen1, bringing a pre-existing silent failure path into active use.
The V4L2 specification defines V4L2_CID_MPEG_VIDEO_GOP_SIZE = 0 as meaning "only I-frames are used". The driver exposes .min = 0 for this control, allowing userspace to request it.
However, the hardware handler explicitly rejects a 0 gop_size:
iris_set_intra_period() {
...
if (!gop_size || b_frame >= gop_size)
return -EINVAL;
...
}During streamon, the driver iterates over capabilities and applies them to the firmware, but ignores any returned errors:
iris_set_properties() {
...
if (cap->cap_id && cap->set)
cap->set(inst, i);
...
}Because the return value is ignored, the error is swallowed and user space receives a success response, but the property is never dispatched to the firmware. Could this cause the encoder hardware to operate with out-of-sync default GOP settings instead of the requested all-I-frames configuration?
[ ... ]
drivers/media/platform/qcom/iris/iris_hfi_gen1.c static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { · line 247
Quoted 8 lines
drivers/media/platform/qcom/iris/iris_hfi_gen1.c +1
-1@@ -240,7 +242,7 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { .step_or_mask = 1, .value = 0, .hfi_id = HFI_PROPERTY_PARAM_VENC_INTRA_REFRESH, - .flags = CAP_FLAG_OUTPUT_PORT, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, .set = iris_set_ir_period_gen1, },
[Severity: Medium]
Does this modification belong in this patch?
The commit message strictly states that it adds support for GOP size configuration on gen1 encoders. However, this line modifies the IR_PERIOD (Intra Refresh Period) capability to include the CAP_FLAG_DYNAMIC_ALLOWED flag.
Modifying intra-refresh dynamic behavior appears to be a distinct functional change that isn't mentioned in the commit message. Could this be split into a separate commit or mentioned in the commit message?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Re: [PATCH v2 1/2] media: qcom: iris: Add gop size support for gen1 encoder
In reply to Wangao WangQuoted 3 lines · Wangao Wang
Add support for gop size configuration on gen1 encoder by enabling
V4L2_CID_MPEG_VIDEO_GOP_SIZE.
Extra empty line within a sentence. Instead please explain, why INTRA_PERIOD is being merged into GOP_SIZE cap.
Quoted 13 lines
Signed-off-by Wangao Wang <[email protected]>--- drivers/media/platform/qcom/iris/iris_ctrls.c | 2 +- drivers/media/platform/qcom/iris/iris_hfi_gen1.c | 16 ++++------------ drivers/media/platform/qcom/iris/iris_platform_common.h | 1 - 3 files changed, 5 insertions(+), 14 deletions(-)drivers/media/platform/qcom/iris/iris_ctrls.c +0
-0
drivers/media/platform/qcom/iris/iris_ctrls.c int iris_set_use_and_mark_ltr(struct iris_inst *inst, enum platform_inst_fw_cap_ · line 1296
Quoted 5 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +1
-1@@ -1293,7 +1293,7 @@ int iris_set_use_and_mark_ltr(struct iris_inst *inst, enum platform_inst_fw_cap_int iris_set_intra_period (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) {const struct iris_hfi_session_ops *hfi_ops = inst->hfi_session_ops; -u32 gop_size = inst->fw_caps[GOP_SIZE ].value; +u32 gop_size = inst->fw_caps[cap_id].value;
Why? I think it was good as is.
Quoted 8 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +0
-0u32 b_frame = inst->fw_caps[B_FRAME ].value;u32 hfi_id = inst->fw_caps[cap_id].hfi_id;struct hfi_intra_period intra_period;drivers/media/platform/qcom/iris/iris_hfi_gen1.c +0
-0
drivers/media/platform/qcom/iris/iris_hfi_gen1.c static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { · line 176
Quoted 7 lines
drivers/media/platform/qcom/iris/iris_hfi_gen1.c +3
-1@@ -171,7 +171,9 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { .max = (1 << 16) - 1, .step_or_mask = 1, .value = 30, - .set = iris_set_u32 + .hfi_id = HFI_PROPERTY_CONFIG_VENC_INTRA_PERIOD, + .flags = CAP_FLAG_OUTPUT_PORT, + .set = iris_set_intra_period,
This is the GOP_SIZE capability, so you will get it as cap_id.
Quoted 3 lines
drivers/media/platform/qcom/iris/iris_hfi_gen1.c +0
-0}, { .cap_id = ENTROPY_MODE,
Signature · 2 lines
With best wishes Dmitry
PATCH v2 2/2 media: qcom: iris: Add request key frame support for encoder
In reply to Wangao WangAdd request key frame support for both gen1 and gen2 encoders by enabling
V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME.
--- drivers/media/platform/qcom/iris/iris_ctrls.c | 22 ++++++++++++++++++++++ drivers/media/platform/qcom/iris/iris_ctrls.h | 1 + drivers/media/platform/qcom/iris/iris_hfi_gen1.c | 10 ++++++++++ .../platform/qcom/iris/iris_hfi_gen1_command.c | 3 +++ .../platform/qcom/iris/iris_hfi_gen1_defines.h | 1 + drivers/media/platform/qcom/iris/iris_hfi_gen2.c | 12 +++++++++++- .../platform/qcom/iris/iris_hfi_gen2_defines.h | 7 +++++++ .../platform/qcom/iris/iris_platform_common.h | 1 + 8 files changed, 56 insertions(+), 1 deletion(-)
drivers/media/platform/qcom/iris/iris_ctrls.c +22 -0
@@ -154,6 +154,8 @@ static enum platform_inst_fw_cap_type iris_get_cap_id(u32 id)return LAYER4_BITRATE_HEVC ;case V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L5_BR :return LAYER5_BITRATE_HEVC ; +case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME : +return REQUEST_SYNC_FRAME ;default :return INST_FW_CAP_MAX ; }
@@ -297,6 +299,8 @@ static u32 iris_get_v4l2_id(enum platform_inst_fw_cap_type cap_id)return V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L4_BR ;case LAYER5_BITRATE_HEVC :return V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L5_BR ; +case REQUEST_SYNC_FRAME : +return V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME ;default :return 0 ; }
@@ -1477,6 +1481,24 @@ int iris_set_layer_bitrate(struct iris_inst *inst, enum platform_inst_fw_cap_typ &bitrate, sizeof(u32)); } +int iris_set_req_sync_frame (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +{ +const struct iris_hfi_session_ops *hfi_ops = inst->hfi_session_ops; +u32 hfi_id = inst->fw_caps[cap_id].hfi_id; +u32 hfi_val =0 ; + +if (inst->fw_caps[PREPEND_SPSPPS_TO_IDR ].value) + hfi_val =HFI_SYNC_FRAME_REQUEST_WITH_PREFIX_SEQ_HDR ; +else + hfi_val =HFI_SYNC_FRAME_REQUEST_WITHOUT_SEQ_HDR ; + +return hfi_ops->session_set_property (inst, hfi_id, +HFI_HOST_FLAGS_NONE , +iris_get_port_info (inst, cap_id), +HFI_PAYLOAD_U32_ENUM , + &hfi_val,sizeof (u32)); +} +int iris_set_properties (struct iris_inst *inst,u32 plane) {const struct iris_hfi_session_ops *hfi_ops = inst->hfi_session_ops;
drivers/media/platform/qcom/iris/iris_ctrls.h +1 -0
@@ -47,6 +47,7 @@ int iris_set_layer_type(struct iris_inst *inst, enum platform_inst_fw_cap_type cint iris_set_layer_count_gen1 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_layer_count_gen2 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_layer_bitrate (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); +int iris_set_req_sync_frame (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_properties (struct iris_inst *inst,u32 plane);#endif
drivers/media/platform/qcom/iris/iris_hfi_gen1.c +10 -0
@@ -375,6 +375,16 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { CAP_FLAG_DYNAMIC_ALLOWED, .set = iris_set_bitrate_gen1, }, + { + .cap_id = REQUEST_SYNC_FRAME, + .min = 0, + .max = 0, + .step_or_mask = 0, + .value = 0, + .hfi_id = HFI_PROPERTY_CONFIG_VENC_REQUEST_SYNC_FRAME, + .flags = CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_req_sync_frame, + }, };static const u32 sm8250_vdec_input_config_param_default[] = {
drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +3 -0
@@ -741,6 +741,9 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p packet->shdr.hdr.size +=sizeof (u32);break ; } +case HFI_PROPERTY_CONFIG_VENC_REQUEST_SYNC_FRAME : { +break ; + }default :return -EINVAL ; }
drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h +1 -0
@@ -157,6 +157,7 @@#define HFI_PROPERTY_PARAM_VENC_HIER_P_MAX_NUM_ENH_LAYER 0x2005026#define HFI_PROPERTY_CONFIG_VENC_TARGET_BITRATE 0x2006001#define HFI_PROPERTY_CONFIG_VENC_INTRA_PERIOD 0x2006003 +#define HFI_PROPERTY_CONFIG_VENC_REQUEST_SYNC_FRAME 0x2006004#define HFI_PROPERTY_CONFIG_VENC_MARKLTRFRAME 0x2006009#define HFI_PROPERTY_CONFIG_VENC_USELTRFRAME 0x200600a#define HFI_PROPERTY_CONFIG_VENC_SYNC_FRAME_SEQUENCE_HEADER 0x2006008
drivers/media/platform/qcom/iris/iris_hfi_gen2.c +11 -1
@@ -937,7 +937,17 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8550_enc[] = { .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, .set = iris_set_layer_bitrate, - } + }, + { + .cap_id = REQUEST_SYNC_FRAME, + .min = 0, + .max = 0, + .step_or_mask = 0, + .value = 0, + .hfi_id = HFI_PROP_REQUEST_SYNC_FRAME, + .flags = CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_req_sync_frame, + }, };static const u32 sm8550_vdec_input_config_params_default[] = {
drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h +7 -0
@@ -90,6 +90,13 @@ enum hfi_layer_encoding_type {#define HFI_PROP_BITRATE_LAYER4 0x0300013f#define HFI_PROP_BITRATE_LAYER5 0x03000140#define HFI_PROP_BITRATE_LAYER6 0x03000141 + +enum hfi_syncframe_request_mode { +HFI_SYNC_FRAME_REQUEST_WITHOUT_SEQ_HDR =0x00000001 , +HFI_SYNC_FRAME_REQUEST_WITH_PREFIX_SEQ_HDR =0x00000002 , +}; + +#define HFI_PROP_REQUEST_SYNC_FRAME 0x03000145#define HFI_PROP_MAX_GOP_FRAMES 0x03000146#define HFI_PROP_MAX_B_FRAMES 0x03000147#define HFI_PROP_QUALITY_MODE 0x03000148
drivers/media/platform/qcom/iris/iris_platform_common.h +1 -0
@@ -182,6 +182,7 @@ enum platform_inst_fw_cap_type {LAYER3_BITRATE_HEVC ,LAYER4_BITRATE_HEVC ,LAYER5_BITRATE_HEVC , +REQUEST_SYNC_FRAME , INST_FW_CAP_MAX, };
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v2 2/2] media: qcom: iris: Add request key frame support for encoder
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
New issues:
- [High] In Gen1, the payload for HFI_PROPERTY_CONFIG_VENC_REQUEST_SYNC_FRAME is discarded and the packet size is not incremented, resulting in a truncated firmware command packet.
- [Medium] The driver unconditionally requests a sync frame on stream initialization, bypassing V4L2 button control semantics.
Pre-existing issues:
- [High] Concurrent modification of the shared inst_hfi_gen2->packet buffer causes HFI command packet corruption. --
commit 69576d77651c69239afd97ac72082948d7e40043
Author: Wangao Wang <[email protected]>
media: qcom: iris: Add request key frame support for encoder
This commit adds support for requesting key frames for both gen1 and gen2 encoders by enabling the V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME control.
Quoted 4 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +0
-0
[ ... ]
Quoted 11 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +7
-0@@ -1477,6 +1481,24 @@ int iris_set_layer_bitrate(struct iris_inst *inst, enum platform_inst_fw_cap_typ &bitrate, sizeof(u32)); } +int iris_set_req_sync_frame (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +{ +const struct iris_hfi_session_ops *hfi_ops = inst->hfi_session_ops; +u32 hfi_id = inst->fw_caps[cap_id].hfi_id; +u32 hfi_val =0 ; + +if (inst->fw_caps[PREPEND_SPSPPS_TO_IDR ].value)
drivers/media/platform/qcom/iris/iris_ctrls.c int iris_set_layer_bitrate(struct iris_inst *inst, enum platform_inst_fw_cap_typ · line 1500
Quoted 10 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +10
-0+ hfi_val =HFI_SYNC_FRAME_REQUEST_WITH_PREFIX_SEQ_HDR ; + else + hfi_val =HFI_SYNC_FRAME_REQUEST_WITHOUT_SEQ_HDR ; + +return hfi_ops->session_set_property (inst, hfi_id, +HFI_HOST_FLAGS_NONE , +iris_get_port_info (inst, cap_id), +HFI_PAYLOAD_U32_ENUM , + &hfi_val,sizeof (u32)); +}
[Severity: Medium]
Will this unconditionally request a sync frame on stream initialization?
During stream initialization, iris_set_properties() iterates over all fw_caps and invokes their .set callbacks. Because this function lacks a check for inst->fw_caps[cap_id].flags & CAP_FLAG_CLIENT_SET (like bitrate does), it seems it will unconditionally issue a REQUEST_SYNC_FRAME command to the firmware on every stream start, bypassing V4L2 button control semantics.
Quoted 2 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +1
-0+int iris_set_properties (struct iris_inst *inst,u32 plane)
[ ... ]
drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p · line 746
Quoted 11 lines
drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +3
-0@@ -741,6 +741,9 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p packet->shdr.hdr.size +=sizeof (u32);break ; } +case HFI_PROPERTY_CONFIG_VENC_REQUEST_SYNC_FRAME : { +break ; + }
[Severity: High]
Does this result in a truncated HFI packet?
In iris_hfi_gen1_packet_session_set_property(), the switch statement for HFI_PROPERTY_CONFIG_VENC_REQUEST_SYNC_FRAME simply breaks without writing the 4-byte payload or incrementing packet->shdr.hdr.size by sizeof(u32). This omits the payload entirely, which may cause the firmware to reject the request with a parameter error.
Quoted 3 lines
drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +0
-0default :return -EINVAL ; }
[ ... ]
Quoted 10 lines
drivers/media/platform/qcom/iris/iris_hfi_gen2.c +1
-1@@ -937,7 +937,17 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8550_enc[] = { .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, .set = iris_set_layer_bitrate, - } + },
drivers/media/platform/qcom/iris/iris_hfi_gen2.c static const struct platform_inst_fw_cap inst_fw_cap_sm8550_enc[] = { · line 950
Quoted 10 lines
drivers/media/platform/qcom/iris/iris_hfi_gen2.c +10
-0+ { + .cap_id = REQUEST_SYNC_FRAME, + .min = 0, + .max = 0, + .step_or_mask = 0, + .value = 0, + .hfi_id = HFI_PROP_REQUEST_SYNC_FRAME, + .flags = CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_req_sync_frame, + },
[Severity: High]
This is a pre-existing issue, but does adding this dynamic control expose a race condition where the shared packet buffer could be corrupted?
In Gen2, properties and commands are built using a single, per-instance dynamically allocated buffer inst_hfi_gen2->packet. The V4L2 control framework allows dynamic controls, such as the newly added V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME, to be set during streaming via VIDIOC_S_CTRL.
The s_ctrl handler is protected only by the control handler's internal lock (ctrl_handler.lock), while queue-related ioctls like VIDIOC_QBUF and VIDIOC_STREAMON are protected by the queue lock (inst->ctx_q_lock).
When s_ctrl and qbuf are executed concurrently, they can both invoke Gen2 HFI functions which write headers and payload data directly into the shared inst_hfi_gen2->packet. Since there is no common lock protecting inst_hfi_gen2->packet prior to its submission in iris_hfi_queue_cmd_write(), this could result in data races and corrupted HFI packets sent to the firmware.
Quoted 1 line
drivers/media/platform/qcom/iris/iris_hfi_gen2.c +0
-0};
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Re: [PATCH v2 2/2] media: qcom: iris: Add request key frame support for encoder
In reply to Wangao WangQuoted 58 lines · Wangao Wang
Add request key frame support for both gen1 and gen2 encoders by enabling
V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME.Signed-off-by Wangao Wang <[email protected]>--- drivers/media/platform/qcom/iris/iris_ctrls.c | 22 ++++++++++++++++++++++ drivers/media/platform/qcom/iris/iris_ctrls.h | 1 + drivers/media/platform/qcom/iris/iris_hfi_gen1.c | 10 ++++++++++ .../platform/qcom/iris/iris_hfi_gen1_command.c | 3 +++ .../platform/qcom/iris/iris_hfi_gen1_defines.h | 1 + drivers/media/platform/qcom/iris/iris_hfi_gen2.c | 12 +++++++++++- .../platform/qcom/iris/iris_hfi_gen2_defines.h | 7 +++++++ .../platform/qcom/iris/iris_platform_common.h | 1 + 8 files changed, 56 insertions(+), 1 deletion(-)drivers/media/platform/qcom/iris/iris_ctrls.c +18
-0@@ -1477,6 +1481,24 @@ int iris_set_layer_bitrate(struct iris_inst *inst, enum platform_inst_fw_cap_typ &bitrate, sizeof(u32)); } +int iris_set_req_sync_frame (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +{ +const struct iris_hfi_session_ops *hfi_ops = inst->hfi_session_ops; +u32 hfi_id = inst->fw_caps[cap_id].hfi_id; +u32 hfi_val =0 ; + +if (inst->fw_caps[PREPEND_SPSPPS_TO_IDR ].value) + hfi_val =HFI_SYNC_FRAME_REQUEST_WITH_PREFIX_SEQ_HDR ; +else + hfi_val =HFI_SYNC_FRAME_REQUEST_WITHOUT_SEQ_HDR ; + +return hfi_ops->session_set_property (inst, hfi_id, +HFI_HOST_FLAGS_NONE , +iris_get_port_info (inst, cap_id), +HFI_PAYLOAD_U32_ENUM , + &hfi_val,sizeof (u32)); +} +int iris_set_properties (struct iris_inst *inst,u32 plane) {const struct iris_hfi_session_ops *hfi_ops = inst->hfi_session_ops;drivers/media/platform/qcom/iris/iris_ctrls.h +1
-0@@ -47,6 +47,7 @@ int iris_set_layer_type(struct iris_inst *inst, enum platform_inst_fw_cap_type cint iris_set_layer_count_gen1 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_layer_count_gen2 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_layer_bitrate (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); +int iris_set_req_sync_frame (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_properties (struct iris_inst *inst,u32 plane);#endif drivers/media/platform/qcom/iris/iris_hfi_gen1.c +0
-0
drivers/media/platform/qcom/iris/iris_hfi_gen1.c static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { · line 383
Quoted 9 lines
drivers/media/platform/qcom/iris/iris_hfi_gen1.c +6
-0@@ -375,6 +375,16 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { CAP_FLAG_DYNAMIC_ALLOWED, .set = iris_set_bitrate_gen1, }, + { + .cap_id = REQUEST_SYNC_FRAME, + .min = 0, + .max = 0, + .step_or_mask = 0, + .value = 0,
How does it work if min = max = 0?
Quoted 6 lines
drivers/media/platform/qcom/iris/iris_hfi_gen1.c +4
-0+ .hfi_id = HFI_PROPERTY_CONFIG_VENC_REQUEST_SYNC_FRAME, + .flags = CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_req_sync_frame, + }, };
Signature · 2 lines
With best wishes Dmitry
Re: [PATCH v2 2/2] media: qcom: iris: Add request key frame support for encoder
Quoted 9 lines · Dmitry Baryshkov
drivers/media/platform/qcom/iris/iris_hfi_gen1.c static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { · line 383
Quoted 6 lines
drivers/media/platform/qcom/iris/iris_hfi_gen1.c +6
-0+ { + .cap_id = REQUEST_SYNC_FRAME, + .min = 0, + .max = 0, + .step_or_mask = 0, + .value = 0,How does it work if min = max = 0?
Quoted 7 lines
drivers/media/platform/qcom/iris/iris_hfi_gen1.c static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { · line 388
Quoted 6 lines
drivers/media/platform/qcom/iris/iris_hfi_gen1.c +4
-0+ .hfi_id = HFI_PROPERTY_CONFIG_VENC_REQUEST_SYNC_FRAME, + .flags = CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_req_sync_frame, + }, };
When using this cap, the value here is not actually read, so it runs fine. But I think you're right — max should be set to 1. Will be fixed in the next version.
Signature · 2 lines
Best Regards, Wangao