media: qcom: iris: encoder feature enhancements batch2
Subsystems (4)
v3 cover media: qcom: iris: encoder feature enhancements batch2
Hi All,
This patch series introduces several enhancements to the Qualcomm Iris encoder driver, improving support for V4L2 controls and enabling more
video encoding features.
All patches have been tested with v4l2-compliance and v4l2-ctl on gen1:SM8250, QCS6490, gen2:QCS8300, QCS8550, QCS9100, X1E-80100.
Commands used for V4l2-ctl validation:
Intra Refresh:
./v4l2-ctl --verbose -d /dev/video1 \
--set-fmt-video-out=width=1920,height=1080,pixelformat=NV12 \
--set-selection-output target=crop,width=1920,height=1080 \
--set-fmt-video=pixelformat=H264 --stream-mmap --stream-out-mmap \
--stream-from=input_nv12_1080p.yuv \
--stream-to=output/ir_random.h264 \
--set-ctrl intra_refresh_period_type=0,intra_refresh_period=30
./v4l2-ctl --verbose -d /dev/video1 \
--set-fmt-video-out=width=1920,height=1080,pixelformat=NV12 \
--set-selection-output target=crop,width=1920,height=1080 \
--set-fmt-video=pixelformat=H264 --stream-mmap --stream-out-mmap \
--stream-from=input_nv12_1080p.yuv \
--stream-to=output/ir_cyclic.h264 \
--set-ctrl intra_refresh_period_type=1,intra_refresh_period=30
./v4l2-ctl --verbose -d /dev/video1 \
--set-fmt-video-out=width=1920,height=1080,pixelformat=NV12 \
--set-selection-output target=crop,width=1920,height=1080 \
--set-fmt-video=pixelformat=HEVC --stream-mmap --stream-out-mmap \
--stream-from=input_nv12_1080p.yuv \
--stream-to=output/ir_random.h265 \
--set-ctrl intra_refresh_period_type=0,intra_refresh_period=30
./v4l2-ctl --verbose -d /dev/video1 \
--set-fmt-video-out=width=1920,height=1080,pixelformat=NV12 \
--set-selection-output target=crop,width=1920,height=1080 \
--set-fmt-video=pixelformat=HEVC --stream-mmap --stream-out-mmap \
--stream-from=input_nv12_1080p.yuv \
--stream-to=output/ir_cyclic.h265 \
--set-ctrl intra_refresh_period_type=1,intra_refresh_period=30
B frames:
./v4l2-ctl --verbose -d /dev/video1 \
--set-fmt-video-out=width=1920,height=1080,pixelformat=NV12 \
--set-selection-output target=crop,width=1920,height=1080 \
--set-fmt-video=pixelformat=H264 --stream-mmap --stream-out-mmap \
--stream-from=input_nv12_1080p.yuv \
--stream-to=output/b_frames.h264 \
--set-ctrl video_b_frames=1
./v4l2-ctl --verbose -d /dev/video1 \
--set-fmt-video-out=width=1920,height=1080,pixelformat=NV12 \
--set-selection-output target=crop,width=1920,height=1080 \
--set-fmt-video=pixelformat=HEVC --stream-mmap --stream-out-mmap \
--stream-from=input_nv12_1080p.yuv \
--stream-to=output/b_frames.h265 \
--set-ctrl video_b_frames=1
LTR:
./v4l2-ctl --verbose -d /dev/video1 \
--set-fmt-video-out=width=1920,height=1080,pixelformat=NV12 \
--set-selection-output target=crop,width=1920,height=1080 \
--set-fmt-video=pixelformat=H264 --stream-mmap --stream-out-mmap \
--stream-from=input_nv12_1080p.yuv \
--stream-to=output/ltr.h264 \
--set-ctrl ltr_count=2,frame_ltr_index=1
./v4l2-ctl --verbose -d /dev/video1 \
--set-fmt-video-out=width=1920,height=1080,pixelformat=NV12 \
--set-selection-output target=crop,width=1920,height=1080 \
--set-fmt-video=pixelformat=HEVC --stream-mmap --stream-out-mmap \
--stream-from=input_nv12_1080p.yuv \
--stream-to=output/ltr.h265 \
--set-ctrl ltr_count=2,frame_ltr_index=1
Hierarchical Coding:
./v4l2-ctl --verbose -d /dev/video1 \
--set-fmt-video-out=width=1920,height=1080,pixelformat=NV12 \
--set-selection-output target=crop,width=1920,height=1080 \
--set-fmt-video=pixelformat=H264 --stream-mmap --stream-out-mmap \
--stream-from=input_nv12_1080p.yuv \
--stream-to=output/hier_coding_layer1_3M.h264 \
--set-ctrl video_bitrate_mode=1,enable_h264_hierarchical_coding=1,h264_hierarchical_coding_type=1,h264_number_of_hc_layers=2,h264_hierarchical_lay_1_bitrate=3000000
./v4l2-ctl --verbose -d /dev/video1 \
--set-fmt-video-out=width=1920,height=1080,pixelformat=NV12 \
--set-selection-output target=crop,width=1920,height=1080 \
--set-fmt-video=pixelformat=H264 --stream-mmap --stream-out-mmap \
--stream-from=input_nv12_1080p.yuv \
--stream-to=output/hier_coding_layer0_6M.h264 \
--set-ctrl video_bitrate_mode=1,enable_h264_hierarchical_coding=1,h264_hierarchical_coding_type=1,h264_number_of_hc_layers=2,h264_hierarchical_lay_0_bitrate=6000000
./v4l2-ctl --verbose -d /dev/video1 \
--set-fmt-video-out=width=1920,height=1080,pixelformat=NV12 \
--set-selection-output target=crop,width=1920,height=1080 \
--set-fmt-video=pixelformat=HEVC --stream-mmap --stream-out-mmap \
--stream-from=input_nv12_1080p.yuv \
--stream-to=output/hier_coding.h265 \
--set-ctrl hevc_hierarchical_coding_type=1,hevc_hierarchical_coding_layer=5
The result of v4l2-compliance on QCS8300:
v4l2-compliance 1.31.0-5379, 64 bits, 64-bit time_t v4l2-compliance SHA: 14c988631ad4 2025-11-11 11:19:35
Compliance test for iris_driver device /dev/video1:
Driver Info:
Driver name : iris_driver
Card type : Iris Encoder
Bus info : platform:aa00000.video-codec
Driver version : 6.18.0
Capabilities : 0x84204000 Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Detected Stateful Encoder
Required ioctls:
test VIDIOC_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/video1 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 52 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK
test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK
test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test CREATE_BUFS maximum buffers: OK test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported) test blocking wait: OK
Test input 0:
Streaming ioctls:
test read/write: OK (Not Supported) Video Capture Multiplanar: Captured 61 buffers test MMAP (select, REQBUFS): OK
Video Capture Multiplanar: Captured 61 buffers test MMAP (epoll, REQBUFS): OK
Video Capture Multiplanar: Captured 61 buffers test MMAP (select, CREATE_BUFS): OK Video Capture Multiplanar: Captured 61 buffers test MMAP (epoll, CREATE_BUFS): OK
test USERPTR (select): OK (Not Supported) test DMABUF: Cannot test, specify --expbuf-device
Total for iris_driver device /dev/video1: 54, Succeeded: 54, Failed: 0, Warnings: 0
The result of v4l2-compliance on QCS6490:
v4l2-compliance 1.31.0-5379, 64 bits, 64-bit time_t v4l2-compliance SHA: 14c988631ad4 2025-11-11 11:19:35
Compliance test for iris_driver device /dev/video1:
Driver Info:
Driver name : iris_driver
Card type : Iris Encoder
Bus info : platform:aa00000.video-codec
Driver version : 6.18.0
Capabilities : 0x84204000 Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Detected Stateful Encoder
Required ioctls:
test VIDIOC_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/video1 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 33 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK
test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK
test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test CREATE_BUFS maximum buffers: OK test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported) test blocking wait: OK
Test input 0:
Streaming ioctls:
test read/write: OK (Not Supported) Video Capture Multiplanar: Captured 61 buffers test MMAP (select, REQBUFS): OK
Video Capture Multiplanar: Captured 61 buffers test MMAP (epoll, REQBUFS): OK
Video Capture Multiplanar: Captured 61 buffers test MMAP (select, CREATE_BUFS): OK Video Capture Multiplanar: Captured 61 buffers test MMAP (epoll, CREATE_BUFS): OK
test USERPTR (select): OK (Not Supported) test DMABUF: Cannot test, specify --expbuf-device
Total for iris_driver device /dev/video1: 54, Succeeded: 54, Failed: 0, Warnings: 0
Dependencies:
https://lore.kernel.org/all/20251114-iris_encoder_enhancements-v6-0-d6d35a92635d@oss.qualcomm.com/
---
Changes in v3:
- Correct the maximum value of IR_PERIOD.(Dikshita)
- Use 'codec' instead of 'standard' in the calculation of COMV size.
(Dikshita)
- Verified these patches on SM8250.(Dikshita)
- Link to v2: https://lore.kernel.org/r/20251219-batch2_iris_encoder_enhancements-v2-0-371f7fe24801@oss.qualcomm.com
Changes in v2:
- Add comment for ir_period calculation.(bod,Dikshita)
- Correct the maximum value of IR_PERIOD.(Dikshita)
- Add a patch to optimize the calculation of hdr size.(bod)
- Correct the num_recon calculation.(Dikshita)
- Add a patch to simplify the calculation of COMV size.(Dikshita)
- Catch the result code of session_set_property.(bod)
- Add comment for shift operation in hfi_buffer_get_recon_count.(bod)
- Add support for layer bitrate setting.(Dikshita)
- Link to v1: https://lore.kernel.org/r/20251127-batch2_iris_encoder_enhancements-v1-0-5ea78e2de2ae@oss.qualcomm.com
---
Wangao Wang (6):
media: qcom: iris: Add intra refresh support for gen1 encoder
media: qcom: iris: Add Long-Term Reference support for encoder
media: qcom: iris: Add B frames support for encoder
media: qcom: iris: Add hierarchical coding support for encoder
media: qcom: iris: Optimize iris_hfi_gen1_packet_session_set_property
media: qcom: iris: Simplify COMV size calculation
drivers/media/platform/qcom/iris/iris_ctrls.c | 476 ++++++++++++++++++++- drivers/media/platform/qcom/iris/iris_ctrls.h | 16 +- .../platform/qcom/iris/iris_hfi_gen1_command.c | 98 ++++- .../platform/qcom/iris/iris_hfi_gen1_defines.h | 49 +++ .../platform/qcom/iris/iris_hfi_gen2_defines.h | 18 + drivers/media/platform/qcom/iris/iris_instance.h | 4 + .../platform/qcom/iris/iris_platform_common.h | 31 ++ .../media/platform/qcom/iris/iris_platform_gen1.c | 161 ++++++- .../media/platform/qcom/iris/iris_platform_gen2.c | 224 +++++++++- drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 65 ++- 10 files changed, 1101 insertions(+), 41 deletions(-)
---
base-commit: cc3aa43b44bdb43dfbac0fcb51c56594a11338a8
change-id: 20251125-batch2_iris_encoder_enhancements-3e5c42b93a25
prerequisite-message-id: 20251114-iris_encoder_enhancements-v6-0-d6d35a92635d@oss.qualcomm.com
prerequisite-patch-id: 2b77a7b9bfb436751da0bf69488dbff567a38479
prerequisite-patch-id: 190f29ef211b046f0d031e3a4b52f2c3bccd0cd9
prerequisite-patch-id: 92ff1a1958eeacd53d70e155bb8214f9987ddf85
prerequisite-patch-id: 0dae6aaa8db7df0729436cf9268d29ea363f0eed
prerequisite-patch-id: 6382693f5a5a7438660f2e592b30163551d92791
prerequisite-patch-id: 9aa16211c36540a61d219eb25f09b7625b9bdbde
Best regards,
Signature · 1 line
Wangao Wang <[email protected]>
PATCH v3 1/6 media: qcom: iris: Add intra refresh support for gen1 encoder
Add support for intra refresh configuration on gen1 encoder by enabling V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD and V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE controls.
--- drivers/media/platform/qcom/iris/iris_ctrls.c | 39 +++++++++++++++++++++- drivers/media/platform/qcom/iris/iris_ctrls.h | 3 +- .../platform/qcom/iris/iris_hfi_gen1_command.c | 8 +++++ .../platform/qcom/iris/iris_hfi_gen1_defines.h | 13 ++++++++ .../media/platform/qcom/iris/iris_platform_gen1.c | 19 +++++++++++ .../media/platform/qcom/iris/iris_platform_gen2.c | 2 +- 6 files changed, 81 insertions(+), 3 deletions(-)
drivers/media/platform/qcom/iris/iris_ctrls.c +38 -1
@@ -962,7 +962,44 @@ int iris_set_flip(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_id) &hfi_val, sizeof(u32)); } -int iris_set_ir_period (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +int iris_set_ir_period_gen1 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +{ +const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; +struct v4l2_pix_format_mplane *fmt = &inst->fmt_dst->fmt.pix_mp; +u32 codec_align = inst->codec ==V4L2_PIX_FMT_HEVC ?32 :16 ; +u32 ir_period = inst->fw_caps[cap_id].value; +u32 hfi_id = inst->fw_caps[cap_id].hfi_id; +struct hfi_intra_refresh hfi_val; + +if (!ir_period) +return -EINVAL ; + +if (inst->fw_caps[IR_TYPE ].value == +V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_RANDOM ) { + hfi_val.mode =HFI_INTRA_REFRESH_RANDOM ; + }else if (inst->fw_caps[IR_TYPE ].value == +V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_CYCLIC ) { + hfi_val.mode =HFI_INTRA_REFRESH_CYCLIC ; + }else { +return -EINVAL ; + } + +/* +* Calculate the number of macroblocks in a frame, +* then determine how many macroblocks need to be +* refreshed within one ir_period. +*/ + hfi_val.mbs = (fmt->width / codec_align) * (fmt->height / codec_align); + hfi_val.mbs /= ir_period; + +return hfi_ops->session_set_property (inst, hfi_id, +HFI_HOST_FLAGS_NONE , +iris_get_port_info (inst, cap_id), +HFI_PAYLOAD_STRUCTURE , + &hfi_val,sizeof (hfi_val)); +} + +int iris_set_ir_period_gen2 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) {const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops;struct vb2_queue *q =v4l2_m2m_get_dst_vq (inst->m2m_ctx);
drivers/media/platform/qcom/iris/iris_ctrls.h +2 -1
@@ -34,7 +34,8 @@ int iris_set_frame_qp(struct iris_inst *inst, enum platform_inst_fw_cap_type capint iris_set_qp_range (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_rotation (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_flip (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); -int iris_set_ir_period (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); +int iris_set_ir_period_gen1 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); +int iris_set_ir_period_gen2 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_properties (struct iris_inst *inst,u32 plane);#endif
drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +8 -0
@@ -685,6 +685,14 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p packet->shdr.hdr.size +=sizeof (u32) +sizeof (*plane_actual_info);break ; } +case HFI_PROPERTY_PARAM_VENC_INTRA_REFRESH : { +struct hfi_intra_refresh *in = pdata, *intra_refresh = prop_data; + + intra_refresh->mode = in->mode; + intra_refresh->mbs = in->mbs; + packet->shdr.hdr.size +=sizeof (u32) +sizeof (*intra_refresh); +break ; + }default :return -EINVAL ; }
drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h +13 -0
@@ -139,6 +139,14 @@#define HFI_PROPERTY_PARAM_VENC_H264_DEBLOCK_CONTROL 0x2005003#define HFI_PROPERTY_PARAM_VENC_RATE_CONTROL 0x2005004#define HFI_PROPERTY_PARAM_VENC_SESSION_QP_RANGE_V2 0x2005009 + +#define HFI_INTRA_REFRESH_NONE 0x1 +#define HFI_INTRA_REFRESH_CYCLIC 0x2 +#define HFI_INTRA_REFRESH_ADAPTIVE 0x3 +#define HFI_INTRA_REFRESH_CYCLIC_ADAPTIVE 0x4 +#define HFI_INTRA_REFRESH_RANDOM 0x5 + +#define HFI_PROPERTY_PARAM_VENC_INTRA_REFRESH 0x200500d#define HFI_PROPERTY_PARAM_VENC_MAX_NUM_B_FRAMES 0x2005020#define HFI_PROPERTY_CONFIG_VENC_TARGET_BITRATE 0x2006001#define HFI_PROPERTY_CONFIG_VENC_SYNC_FRAME_SEQUENCE_HEADER 0x2006008
@@ -447,6 +455,11 @@ struct hfi_framerate {u32 framerate; }; +struct hfi_intra_refresh { +u32 mode; +u32 mbs; +}; +struct hfi_event_data {u32 error;u32 height;
drivers/media/platform/qcom/iris/iris_platform_gen1.c +19 -0
@@ -230,6 +230,25 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { .flags = CAP_FLAG_OUTPUT_PORT, .set = iris_set_qp_range, }, + { + .cap_id = IR_TYPE, + .min = V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_RANDOM, + .max = V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_CYCLIC, + .step_or_mask =BIT (V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_RANDOM ) | + BIT(V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_CYCLIC), + .value = V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_RANDOM, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + }, + { + .cap_id = IR_PERIOD, + .min = 0, + .max = ((4096 * 2304) >> 8), + .step_or_mask = 1, + .value = 0, + .hfi_id = HFI_PROPERTY_PARAM_VENC_INTRA_REFRESH, + .flags = CAP_FLAG_OUTPUT_PORT, + .set = iris_set_ir_period_gen1, + }, }; static struct platform_inst_caps platform_inst_cap_sm8250 = {
drivers/media/platform/qcom/iris/iris_platform_gen2.c +1 -1
@@ -635,7 +635,7 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8550_enc[] = { .value = 0, .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, - .set = iris_set_ir_period, + .set = iris_set_ir_period_gen2, }, };
Signature · 1 line
2.43.0
Re: [PATCH v3 1/6] media: qcom: iris: Add intra refresh support for gen1 encoder
Quoted 5 lines · Wangao Wang
Add support for intra refresh configuration on gen1 encoder by enabling
V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIODandV4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPEcontrols.Signed-off-by Wangao Wang <[email protected]>
Quoted 9 lines
--- drivers/media/platform/qcom/iris/iris_ctrls.c | 39 +++++++++++++++++++++- drivers/media/platform/qcom/iris/iris_ctrls.h | 3 +- .../platform/qcom/iris/iris_hfi_gen1_command.c | 8 +++++ .../platform/qcom/iris/iris_hfi_gen1_defines.h | 13 ++++++++ .../media/platform/qcom/iris/iris_platform_gen1.c | 19 +++++++++++ .../media/platform/qcom/iris/iris_platform_gen2.c | 2 +- 6 files changed, 81 insertions(+), 3 deletions(-)
Thanks,
Dikshita
PATCH v3 2/6 media: qcom: iris: Add Long-Term Reference support for encoder
In reply to Wangao WangAdd Long-Term Reference(LTR) frame support for both gen1 and gen2 encoders by enabling the following V4L2 controls:V4L2_CID_MPEG_VIDEO_LTR_COUNT V4L2_CID_MPEG_VIDEO_USE_LTR_FRAMES V4L2_CID_MPEG_VIDEO_FRAME_LTR_INDEX
--- drivers/media/platform/qcom/iris/iris_ctrls.c | 128 +++++++++++++++++++++ drivers/media/platform/qcom/iris/iris_ctrls.h | 5 + .../platform/qcom/iris/iris_hfi_gen1_command.c | 25 ++++ .../platform/qcom/iris/iris_hfi_gen1_defines.h | 24 ++++ .../platform/qcom/iris/iris_hfi_gen2_defines.h | 3 + .../platform/qcom/iris/iris_platform_common.h | 6 + .../media/platform/qcom/iris/iris_platform_gen1.c | 30 +++++ .../media/platform/qcom/iris/iris_platform_gen2.c | 30 +++++ drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 20 +++- 9 files changed, 267 insertions(+), 4 deletions(-)
drivers/media/platform/qcom/iris/iris_ctrls.c +128 -0
@@ -108,6 +108,12 @@ static enum platform_inst_fw_cap_type iris_get_cap_id(u32 id)return IR_TYPE ;case V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD :return IR_PERIOD ; +case V4L2_CID_MPEG_VIDEO_LTR_COUNT : +return LTR_COUNT ; +case V4L2_CID_MPEG_VIDEO_USE_LTR_FRAMES : +return USE_LTR ; +case V4L2_CID_MPEG_VIDEO_FRAME_LTR_INDEX : +return MARK_LTR ;default :return INST_FW_CAP_MAX ; }
@@ -205,6 +211,12 @@ static u32 iris_get_v4l2_id(enum platform_inst_fw_cap_type cap_id)return V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE ;case IR_PERIOD :return V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD ; +case LTR_COUNT : +return V4L2_CID_MPEG_VIDEO_LTR_COUNT ; +case USE_LTR : +return V4L2_CID_MPEG_VIDEO_USE_LTR_FRAMES ; +case MARK_LTR : +return V4L2_CID_MPEG_VIDEO_FRAME_LTR_INDEX ;default :return 0 ; }
@@ -1025,6 +1037,122 @@ int iris_set_ir_period_gen2(struct iris_inst *inst, enum platform_inst_fw_cap_ty &ir_period, sizeof(u32)); } +int iris_set_ltr_count_gen1 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +{ +const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; +u32 ltr_count = inst->fw_caps[cap_id].value; +u32 hfi_id = inst->fw_caps[cap_id].hfi_id; +struct hfi_ltr_mode ltr_mode; + +if (!ltr_count) +return -EINVAL ; + + ltr_mode.count = ltr_count; + ltr_mode.mode =HFI_LTR_MODE_MANUAL ; + ltr_mode.trust_mode =1 ; + +return hfi_ops->session_set_property (inst, hfi_id, +HFI_HOST_FLAGS_NONE , +iris_get_port_info (inst, cap_id), +HFI_PAYLOAD_STRUCTURE , + <r_mode,sizeof (ltr_mode)); +} + +int iris_set_use_ltr (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +{ +const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; +struct vb2_queue *sq =v4l2_m2m_get_src_vq (inst->m2m_ctx); +struct vb2_queue *dq =v4l2_m2m_get_dst_vq (inst->m2m_ctx); +u32 ltr_count = inst->fw_caps[LTR_COUNT ].value; +u32 hfi_id = inst->fw_caps[cap_id].hfi_id; +struct hfi_ltr_use ltr_use; + +if (!vb2_is_streaming (sq) && !vb2_is_streaming (dq)) +return -EINVAL ; + +if (!ltr_count) +return -EINVAL ; + + ltr_use.ref_ltr = inst->fw_caps[cap_id].value; + ltr_use.use_constrnt = true; + ltr_use.frames =0 ; + +return hfi_ops->session_set_property (inst, hfi_id, +HFI_HOST_FLAGS_NONE , +iris_get_port_info (inst, cap_id), +HFI_PAYLOAD_STRUCTURE , + <r_use,sizeof (ltr_use)); +} + +int iris_set_mark_ltr (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +{ +const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; +struct vb2_queue *sq =v4l2_m2m_get_src_vq (inst->m2m_ctx); +struct vb2_queue *dq =v4l2_m2m_get_dst_vq (inst->m2m_ctx); +u32 ltr_count = inst->fw_caps[LTR_COUNT ].value; +u32 hfi_id = inst->fw_caps[cap_id].hfi_id; +struct hfi_ltr_mark ltr_mark; + +if (!vb2_is_streaming (sq) && !vb2_is_streaming (dq)) +return -EINVAL ; + +if (!ltr_count) +return -EINVAL ; + + ltr_mark.mark_frame = inst->fw_caps[cap_id].value; + +return hfi_ops->session_set_property (inst, hfi_id, +HFI_HOST_FLAGS_NONE , +iris_get_port_info (inst, cap_id), +HFI_PAYLOAD_STRUCTURE , + <r_mark,sizeof (ltr_mark)); +} + +int iris_set_ltr_count_gen2 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +{ +const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; +u32 ltr_count = inst->fw_caps[cap_id].value; +u32 hfi_id = inst->fw_caps[cap_id].hfi_id; + +if (!ltr_count) +return -EINVAL ; + +if (inst->hfi_rc_type ==HFI_RC_CBR_VFR || + inst->hfi_rc_type ==HFI_RC_CBR_CFR || + inst->hfi_rc_type ==HFI_RC_OFF ) { + inst->fw_caps[LTR_COUNT ].value =0 ; +return -EINVAL ; + } + +return hfi_ops->session_set_property (inst, hfi_id, +HFI_HOST_FLAGS_NONE , +iris_get_port_info (inst, cap_id), +HFI_PAYLOAD_U32 , + <r_count,sizeof (u32)); +} + +int iris_set_use_and_mark_ltr (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +{ +const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; +struct vb2_queue *sq =v4l2_m2m_get_src_vq (inst->m2m_ctx); +struct vb2_queue *dq =v4l2_m2m_get_dst_vq (inst->m2m_ctx); +u32 ltr_count = inst->fw_caps[LTR_COUNT ].value; +u32 hfi_val = inst->fw_caps[cap_id].value; +u32 hfi_id = inst->fw_caps[cap_id].hfi_id; + +if (!vb2_is_streaming (sq) && !vb2_is_streaming (dq)) +return -EINVAL ; + +if (!ltr_count || hfi_val ==INVALID_DEFAULT_MARK_OR_USE_LTR ) +return -EINVAL ; + +return hfi_ops->session_set_property (inst, hfi_id, +HFI_HOST_FLAGS_NONE , +iris_get_port_info (inst, cap_id), +HFI_PAYLOAD_U32 , + &hfi_val,sizeof (u32)); +} +int iris_set_properties (struct iris_inst *inst,u32 plane) {const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops;
drivers/media/platform/qcom/iris/iris_ctrls.h +5 -0
@@ -36,6 +36,11 @@ int iris_set_rotation(struct iris_inst *inst, enum platform_inst_fw_cap_type capint iris_set_flip (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_ir_period_gen1 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_ir_period_gen2 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); +int iris_set_ltr_count_gen1 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); +int iris_set_ltr_count_gen2 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); +int iris_set_use_ltr (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); +int iris_set_mark_ltr (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); +int iris_set_use_and_mark_ltr (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_properties (struct iris_inst *inst,u32 plane);#endif
drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +25 -0
@@ -693,6 +693,31 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p packet->shdr.hdr.size +=sizeof (u32) +sizeof (*intra_refresh);break ; } +case HFI_PROPERTY_PARAM_VENC_LTRMODE : { +struct hfi_ltr_mode *in = pdata, *ltr_mode = prop_data; + + ltr_mode->mode = in->mode; + ltr_mode->count = in->count; + ltr_mode->trust_mode = in->trust_mode; + packet->shdr.hdr.size +=sizeof (u32) +sizeof (*ltr_mode); +break ; + } +case HFI_PROPERTY_CONFIG_VENC_USELTRFRAME : { +struct hfi_ltr_use *in = pdata, *ltr_use = prop_data; + + ltr_use->frames = in->frames; + ltr_use->ref_ltr = in->ref_ltr; + ltr_use->use_constrnt = in->use_constrnt; + packet->shdr.hdr.size +=sizeof (u32) +sizeof (*ltr_use); +break ; + } +case HFI_PROPERTY_CONFIG_VENC_MARKLTRFRAME : { +struct hfi_ltr_mark *in = pdata, *ltr_mark = prop_data; + + ltr_mark->mark_frame = in->mark_frame; + packet->shdr.hdr.size +=sizeof (u32) +sizeof (*ltr_mark); +break ; + }default :return -EINVAL ; }
drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h +24 -0
@@ -147,8 +147,16 @@#define HFI_INTRA_REFRESH_RANDOM 0x5#define HFI_PROPERTY_PARAM_VENC_INTRA_REFRESH 0x200500d + +#define HFI_LTR_MODE_DISABLE 0x0 +#define HFI_LTR_MODE_MANUAL 0x1 +#define HFI_LTR_MODE_PERIODIC 0x2 + +#define HFI_PROPERTY_PARAM_VENC_LTRMODE 0x200501c#define HFI_PROPERTY_PARAM_VENC_MAX_NUM_B_FRAMES 0x2005020#define HFI_PROPERTY_CONFIG_VENC_TARGET_BITRATE 0x2006001 +#define HFI_PROPERTY_CONFIG_VENC_MARKLTRFRAME 0x2006009 +#define HFI_PROPERTY_CONFIG_VENC_USELTRFRAME 0x200600a#define HFI_PROPERTY_CONFIG_VENC_SYNC_FRAME_SEQUENCE_HEADER 0x2006008struct hfi_pkt_hdr {
@@ -460,6 +468,22 @@ struct hfi_intra_refresh {u32 mbs; }; +struct hfi_ltr_mode { +u32 mode; +u32 count; +u32 trust_mode; +}; + +struct hfi_ltr_use { +u32 ref_ltr; +u32 use_constrnt; +u32 frames; +}; + +struct hfi_ltr_mark { +u32 mark_frame; +}; +struct hfi_event_data {u32 error;u32 height;
drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h +3 -0
@@ -71,6 +71,9 @@ enum hfi_rate_control {#define HFI_PROP_MIN_QP_PACKED 0x0300012f#define HFI_PROP_MAX_QP_PACKED 0x03000130#define HFI_PROP_IR_RANDOM_PERIOD 0x03000131 +#define HFI_PROP_LTR_COUNT 0x03000134 +#define HFI_PROP_LTR_MARK 0x03000135 +#define HFI_PROP_LTR_USE 0x03000136#define HFI_PROP_TOTAL_BITRATE 0x0300013b#define HFI_PROP_MAX_GOP_FRAMES 0x03000146#define HFI_PROP_MAX_B_FRAMES 0x03000147
drivers/media/platform/qcom/iris/iris_platform_common.h +6 -0
@@ -29,6 +29,9 @@ struct iris_inst;#define MAX_QP_HEVC 63#define DEFAULT_QP 20#define BITRATE_DEFAULT 20000000 +#define INVALID_DEFAULT_MARK_OR_USE_LTR -1 +#define MAX_LTR_FRAME_COUNT_GEN1 4 +#define MAX_LTR_FRAME_COUNT_GEN2 2enum stage_type {STAGE_1 =1 ,
@@ -148,6 +151,9 @@ enum platform_inst_fw_cap_type {VFLIP ,IR_TYPE ,IR_PERIOD , +LTR_COUNT , +USE_LTR , +MARK_LTR , INST_FW_CAP_MAX, };
drivers/media/platform/qcom/iris/iris_platform_gen1.c +30 -0
@@ -249,6 +249,36 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { .flags = CAP_FLAG_OUTPUT_PORT, .set = iris_set_ir_period_gen1, }, + { + .cap_id = LTR_COUNT, + .min = 0, + .max = MAX_LTR_FRAME_COUNT_GEN1, + .step_or_mask = 1, + .value = 0, + .hfi_id = HFI_PROPERTY_PARAM_VENC_LTRMODE, + .flags = CAP_FLAG_OUTPUT_PORT, + .set = iris_set_ltr_count_gen1, + }, + { + .cap_id = USE_LTR, + .min = 0, + .max = ((1 << MAX_LTR_FRAME_COUNT_GEN1) - 1), + .step_or_mask = 0, + .value = 0, + .hfi_id = HFI_PROPERTY_CONFIG_VENC_USELTRFRAME, + .flags = CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_use_ltr, + }, + { + .cap_id = MARK_LTR, + .min = 0, + .max = (MAX_LTR_FRAME_COUNT_GEN1 - 1), + .step_or_mask = 1, + .value = 0, + .hfi_id = HFI_PROPERTY_CONFIG_VENC_MARKLTRFRAME, + .flags = CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_mark_ltr, + }, }; static struct platform_inst_caps platform_inst_cap_sm8250 = {
drivers/media/platform/qcom/iris/iris_platform_gen2.c +30 -0
@@ -637,6 +637,36 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8550_enc[] = { CAP_FLAG_DYNAMIC_ALLOWED, .set = iris_set_ir_period_gen2, }, + { + .cap_id = LTR_COUNT, + .min = 0, + .max = MAX_LTR_FRAME_COUNT_GEN2, + .step_or_mask = 1, + .value = 0, + .hfi_id = HFI_PROP_LTR_COUNT, + .flags = CAP_FLAG_OUTPUT_PORT, + .set = iris_set_ltr_count_gen2, + }, + { + .cap_id = USE_LTR, + .min = 0, + .max = ((1 << MAX_LTR_FRAME_COUNT_GEN2) - 1), + .step_or_mask = 0, + .value = 0, + .hfi_id = HFI_PROP_LTR_USE, + .flags = CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_use_and_mark_ltr, + }, + { + .cap_id = MARK_LTR, + .min = INVALID_DEFAULT_MARK_OR_USE_LTR, + .max = (MAX_LTR_FRAME_COUNT_GEN2 - 1), + .step_or_mask = 1, + .value = INVALID_DEFAULT_MARK_OR_USE_LTR, + .hfi_id = HFI_PROP_LTR_MARK, + .flags = CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_use_and_mark_ltr, + }, };static struct platform_inst_caps platform_inst_cap_sm8550 = {
drivers/media/platform/qcom/iris/iris_vpu_buffer.c +16 -4
@@ -668,6 +668,19 @@ static u32 iris_vpu_enc_bin_size(struct iris_inst *inst) num_vpp_pipes, inst->hfi_rc_type); } +static inline u32 hfi_buffer_get_recon_count (struct iris_inst *inst) +{ +u32 num_ref =1 ; +u32 ltr_count; + + ltr_count = inst->fw_caps[LTR_COUNT ].value; + +if (ltr_count) + num_ref = num_ref + ltr_count; + +return num_ref; +} +static inline u32 hfi_buffer_comv_enc (u32 frame_width,u32 frame_height,u32 lcu_size,u32 num_recon,u32 standard)
@@ -693,7 +706,7 @@ static u32 iris_vpu_enc_comv_size(struct iris_inst *inst) {u32 height =iris_vpu_enc_get_bitstream_height (inst);u32 width =iris_vpu_enc_get_bitstream_width (inst); -u32 num_recon =1 ; +u32 num_recon =hfi_buffer_get_recon_count (inst);u32 lcu_size =16 ;if (inst->codec ==V4L2_PIX_FMT_HEVC ) {
@@ -1402,10 +1415,9 @@ static u32 iris_vpu_enc_scratch2_size(struct iris_inst *inst) {u32 frame_height =iris_vpu_enc_get_bitstream_height (inst);u32 frame_width =iris_vpu_enc_get_bitstream_width (inst); -u32 num_ref =1 ; +u32 num_ref =hfi_buffer_get_recon_count (inst); -return hfi_buffer_scratch2_enc (frame_width, frame_height, num_ref, - false); +return hfi_buffer_scratch2_enc (frame_width, frame_height, num_ref, false); }static u32 iris_vpu_enc_vpss_size (struct iris_inst *inst)
Signature · 1 line
2.43.0
PATCH v3 3/6 media: qcom: iris: Add B frames support for encoder
In reply to Wangao WangAdd support for B-frame configuration on both gen1 and gen2 encoders by enabling V4L2_CID_MPEG_VIDEO_B_FRAMES control.
--- drivers/media/platform/qcom/iris/iris_ctrls.c | 30 ++++++++++++++++++++++ drivers/media/platform/qcom/iris/iris_ctrls.h | 1 + .../platform/qcom/iris/iris_hfi_gen1_command.c | 8 ++++++ .../platform/qcom/iris/iris_hfi_gen1_defines.h | 10 ++++++++ .../platform/qcom/iris/iris_platform_common.h | 2 ++ .../media/platform/qcom/iris/iris_platform_gen1.c | 18 +++++++++++++ .../media/platform/qcom/iris/iris_platform_gen2.c | 10 ++++++++ drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 6 ++++- 8 files changed, 84 insertions(+), 1 deletion(-)
drivers/media/platform/qcom/iris/iris_ctrls.c +30 -0
@@ -114,6 +114,8 @@ static enum platform_inst_fw_cap_type iris_get_cap_id(u32 id)return USE_LTR ;case V4L2_CID_MPEG_VIDEO_FRAME_LTR_INDEX :return MARK_LTR ; +case V4L2_CID_MPEG_VIDEO_B_FRAMES : +return B_FRAME ;default :return INST_FW_CAP_MAX ; }
@@ -217,6 +219,8 @@ static u32 iris_get_v4l2_id(enum platform_inst_fw_cap_type cap_id)return V4L2_CID_MPEG_VIDEO_USE_LTR_FRAMES ;case MARK_LTR :return V4L2_CID_MPEG_VIDEO_FRAME_LTR_INDEX ; +case B_FRAME : +return V4L2_CID_MPEG_VIDEO_B_FRAMES ;default :return 0 ; }
@@ -1153,6 +1157,32 @@ int iris_set_use_and_mark_ltr(struct iris_inst *inst, enum platform_inst_fw_cap_ &hfi_val, sizeof(u32)); } +int iris_set_intra_period (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +{ +const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; +u32 gop_size = inst->fw_caps[GOP_SIZE ].value; +u32 b_frame = inst->fw_caps[B_FRAME ].value; +u32 hfi_id = inst->fw_caps[cap_id].hfi_id; +struct hfi_intra_period intra_period; + +if (!gop_size || b_frame >= gop_size) +return -EINVAL ; + +/* +* intra_period represents the length of a GOP, which includes both P-frames +* and B-frames. The counts of P-frames and B-frames within a GOP must be +* communicated to the firmware. +*/ + intra_period.pframes = (gop_size -1 ) / (b_frame +1 ); + intra_period.bframes = b_frame; + +return hfi_ops->session_set_property (inst, hfi_id, +HFI_HOST_FLAGS_NONE , +iris_get_port_info (inst, cap_id), +HFI_PAYLOAD_STRUCTURE , + &intra_period,sizeof (intra_period)); +} +int iris_set_properties (struct iris_inst *inst,u32 plane) {const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops;
drivers/media/platform/qcom/iris/iris_ctrls.h +1 -0
@@ -41,6 +41,7 @@ int iris_set_ltr_count_gen2(struct iris_inst *inst, enum platform_inst_fw_cap_tyint iris_set_use_ltr (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_mark_ltr (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_use_and_mark_ltr (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); +int iris_set_intra_period (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_properties (struct iris_inst *inst,u32 plane);#endif
drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +8 -0
@@ -718,6 +718,14 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p packet->shdr.hdr.size +=sizeof (u32) +sizeof (*ltr_mark);break ; } +case HFI_PROPERTY_CONFIG_VENC_INTRA_PERIOD : { +struct hfi_intra_period *in = pdata, *intra_period = prop_data; + + intra_period->pframes = in->pframes; + intra_period->bframes = in->bframes; + packet->shdr.hdr.size +=sizeof (u32) +sizeof (*intra_period); +break ; + }default :return -EINVAL ; }
drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h +10 -0
@@ -155,6 +155,7 @@#define HFI_PROPERTY_PARAM_VENC_LTRMODE 0x200501c#define HFI_PROPERTY_PARAM_VENC_MAX_NUM_B_FRAMES 0x2005020#define HFI_PROPERTY_CONFIG_VENC_TARGET_BITRATE 0x2006001 +#define HFI_PROPERTY_CONFIG_VENC_INTRA_PERIOD 0x2006003#define HFI_PROPERTY_CONFIG_VENC_MARKLTRFRAME 0x2006009#define HFI_PROPERTY_CONFIG_VENC_USELTRFRAME 0x200600a#define HFI_PROPERTY_CONFIG_VENC_SYNC_FRAME_SEQUENCE_HEADER 0x2006008
@@ -484,6 +485,15 @@ struct hfi_ltr_mark {u32 mark_frame; }; +struct hfi_max_num_b_frames { +u32 max_num_b_frames; +}; + +struct hfi_intra_period { +u32 pframes; +u32 bframes; +}; +struct hfi_event_data {u32 error;u32 height;
drivers/media/platform/qcom/iris/iris_platform_common.h +2 -0
@@ -154,6 +154,8 @@ enum platform_inst_fw_cap_type {LTR_COUNT ,USE_LTR ,MARK_LTR , +B_FRAME , +INTRA_PERIOD , INST_FW_CAP_MAX, };
drivers/media/platform/qcom/iris/iris_platform_gen1.c +18 -0
@@ -279,6 +279,24 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { .flags = CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, .set = iris_set_mark_ltr, }, + { + .cap_id = B_FRAME, + .min = 0, + .max = 3, + .step_or_mask = 1, + .value = 0, + .flags = CAP_FLAG_OUTPUT_PORT, + }, + { + .cap_id = INTRA_PERIOD, + .min = 0, + .max = 1, + .step_or_mask = 1, + .value = 0, + .hfi_id = HFI_PROPERTY_CONFIG_VENC_INTRA_PERIOD, + .flags = CAP_FLAG_OUTPUT_PORT, + .set = iris_set_intra_period, + }, }; static struct platform_inst_caps platform_inst_cap_sm8250 = {
drivers/media/platform/qcom/iris/iris_platform_gen2.c +10 -0
@@ -667,6 +667,16 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8550_enc[] = { .flags = CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, .set = iris_set_use_and_mark_ltr, }, + { + .cap_id = B_FRAME, + .min = 0, + .max = 1, + .step_or_mask = 1, + .value = 0, + .hfi_id = HFI_PROP_MAX_B_FRAMES, + .flags = CAP_FLAG_OUTPUT_PORT, + .set = iris_set_u32, + }, };static struct platform_inst_caps platform_inst_cap_sm8550 = {
drivers/media/platform/qcom/iris/iris_vpu_buffer.c +5 -1
@@ -670,11 +670,15 @@ static u32 iris_vpu_enc_bin_size(struct iris_inst *inst)static inline u32 hfi_buffer_get_recon_count (struct iris_inst *inst) { +u32 bframe_count, ltr_count;u32 num_ref =1 ; -u32 ltr_count; + bframe_count = inst->fw_caps[B_FRAME ].value; ltr_count = inst->fw_caps[LTR_COUNT ].value; +if (bframe_count) + num_ref =2 ; +if (ltr_count) num_ref = num_ref + ltr_count;
Signature · 1 line
2.43.0
Re: [PATCH v3 3/6] media: qcom: iris: Add B frames support for encoder
Quoted 4 lines · Wangao Wang
Add support for B-frame configuration on both gen1 and gen2 encoders by enabling
V4L2_CID_MPEG_VIDEO_B_FRAMEScontrol.Signed-off-by Wangao Wang <[email protected]>
Quoted 11 lines
--- drivers/media/platform/qcom/iris/iris_ctrls.c | 30 ++++++++++++++++++++++ drivers/media/platform/qcom/iris/iris_ctrls.h | 1 + .../platform/qcom/iris/iris_hfi_gen1_command.c | 8 ++++++ .../platform/qcom/iris/iris_hfi_gen1_defines.h | 10 ++++++++ .../platform/qcom/iris/iris_platform_common.h | 2 ++ .../media/platform/qcom/iris/iris_platform_gen1.c | 18 +++++++++++++ .../media/platform/qcom/iris/iris_platform_gen2.c | 10 ++++++++ drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 6 ++++- 8 files changed, 84 insertions(+), 1 deletion(-)
Thanks,
Dikshita
PATCH v3 4/6 media: qcom: iris: Add hierarchical coding support for encoder
In reply to Wangao WangAdd hierarchical coding support for both gen1 and gen2 encoders by enabling
the following V4L2 controls:
H264:V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING, V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_TYPE, V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER
HEVC(gen2 only):V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_TYPE, V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_LAYER
--- drivers/media/platform/qcom/iris/iris_ctrls.c | 279 ++++++++++++++++++++- drivers/media/platform/qcom/iris/iris_ctrls.h | 7 +- .../platform/qcom/iris/iris_hfi_gen1_command.c | 21 +- .../platform/qcom/iris/iris_hfi_gen1_defines.h | 2 + .../platform/qcom/iris/iris_hfi_gen2_defines.h | 15 ++ drivers/media/platform/qcom/iris/iris_instance.h | 4 + .../platform/qcom/iris/iris_platform_common.h | 23 ++ .../media/platform/qcom/iris/iris_platform_gen1.c | 94 ++++++- .../media/platform/qcom/iris/iris_platform_gen2.c | 182 +++++++++++++- drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 28 +++ 10 files changed, 647 insertions(+), 8 deletions(-)
drivers/media/platform/qcom/iris/iris_ctrls.c +278 -1
@@ -116,6 +116,40 @@ static enum platform_inst_fw_cap_type iris_get_cap_id(u32 id)return MARK_LTR ;case V4L2_CID_MPEG_VIDEO_B_FRAMES :return B_FRAME ; +case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING : +return LAYER_ENABLE ; +case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_TYPE : +return LAYER_TYPE_H264 ; +case V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_TYPE : +return LAYER_TYPE_HEVC ; +case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER : +return LAYER_COUNT_H264 ; +case V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_LAYER : +return LAYER_COUNT_HEVC ; +case V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L0_BR : +return LAYER0_BITRATE_H264 ; +case V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L1_BR : +return LAYER1_BITRATE_H264 ; +case V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L2_BR : +return LAYER2_BITRATE_H264 ; +case V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L3_BR : +return LAYER3_BITRATE_H264 ; +case V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L4_BR : +return LAYER4_BITRATE_H264 ; +case V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L5_BR : +return LAYER5_BITRATE_H264 ; +case V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L0_BR : +return LAYER0_BITRATE_HEVC ; +case V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L1_BR : +return LAYER1_BITRATE_HEVC ; +case V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L2_BR : +return LAYER2_BITRATE_HEVC ; +case V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L3_BR : +return LAYER3_BITRATE_HEVC ; +case V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L4_BR : +return LAYER4_BITRATE_HEVC ; +case V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L5_BR : +return LAYER5_BITRATE_HEVC ;default :return INST_FW_CAP_MAX ; }
@@ -221,6 +255,40 @@ static u32 iris_get_v4l2_id(enum platform_inst_fw_cap_type cap_id)return V4L2_CID_MPEG_VIDEO_FRAME_LTR_INDEX ;case B_FRAME :return V4L2_CID_MPEG_VIDEO_B_FRAMES ; +case LAYER_ENABLE : +return V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING ; +case LAYER_TYPE_H264 : +return V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_TYPE ; +case LAYER_TYPE_HEVC : +return V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_TYPE ; +case LAYER_COUNT_H264 : +return V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER ; +case LAYER_COUNT_HEVC : +return V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_LAYER ; +case LAYER0_BITRATE_H264 : +return V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L0_BR ; +case LAYER1_BITRATE_H264 : +return V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L1_BR ; +case LAYER2_BITRATE_H264 : +return V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L2_BR ; +case LAYER3_BITRATE_H264 : +return V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L3_BR ; +case LAYER4_BITRATE_H264 : +return V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L4_BR ; +case LAYER5_BITRATE_H264 : +return V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L5_BR ; +case LAYER0_BITRATE_HEVC : +return V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L0_BR ; +case LAYER1_BITRATE_HEVC : +return V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L1_BR ; +case LAYER2_BITRATE_HEVC : +return V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L2_BR ; +case LAYER3_BITRATE_HEVC : +return V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L3_BR ; +case LAYER4_BITRATE_HEVC : +return V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L4_BR ; +case LAYER5_BITRATE_HEVC : +return V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L5_BR ;default :return 0 ; }
@@ -567,7 +635,60 @@ int iris_set_header_mode_gen2(struct iris_inst *inst, enum platform_inst_fw_cap_ &hfi_val, sizeof(u32)); } -int iris_set_bitrate (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +int iris_set_bitrate_gen1 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +{ +const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; +u32 entropy_mode = inst->fw_caps[ENTROPY_MODE ].value; +u32 bitrate = inst->fw_caps[cap_id].value; +u32 hfi_id = inst->fw_caps[cap_id].hfi_id; +struct hfi_bitrate hfi_val; +u32 max_bitrate; + +if (!(inst->fw_caps[cap_id].flags &CAP_FLAG_CLIENT_SET ) && cap_id !=BITRATE ) +return -EINVAL ; + +if (inst->codec ==V4L2_PIX_FMT_HEVC ) + max_bitrate =CABAC_MAX_BITRATE ; + +if (entropy_mode ==V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC ) + max_bitrate =CABAC_MAX_BITRATE ; +else + max_bitrate =CAVLC_MAX_BITRATE ; + + hfi_val.bitrate =min (bitrate, max_bitrate); + +switch (cap_id) { +case BITRATE : +case LAYER0_BITRATE_H264 : + hfi_val.layer_id =0 ; +break ; +case LAYER1_BITRATE_H264 : + hfi_val.layer_id =1 ; +break ; +case LAYER2_BITRATE_H264 : + hfi_val.layer_id =2 ; +break ; +case LAYER3_BITRATE_H264 : + hfi_val.layer_id =3 ; +break ; +case LAYER4_BITRATE_H264 : + hfi_val.layer_id =4 ; +break ; +case LAYER5_BITRATE_H264 : + hfi_val.layer_id =5 ; +break ; +default : +return -EINVAL ; + } + +return hfi_ops->session_set_property (inst, hfi_id, +HFI_HOST_FLAGS_NONE , +iris_get_port_info (inst, cap_id), +HFI_PAYLOAD_STRUCTURE , + &hfi_val,sizeof (hfi_val)); +} + +int iris_set_bitrate_gen2 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) {const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops;u32 entropy_mode = inst->fw_caps[ENTROPY_MODE ].value;
@@ -1183,6 +1304,162 @@ int iris_set_intra_period(struct iris_inst *inst, enum platform_inst_fw_cap_type &intra_period, sizeof(intra_period)); } +int iris_set_layer_type (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +{ +const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; +u32 layer_enable = inst->fw_caps[LAYER_ENABLE ].value; +u32 hfi_id = inst->fw_caps[cap_id].hfi_id; +u32 layer_type; + +if (inst->hfi_rc_type ==HFI_RATE_CONTROL_CQ || + inst->hfi_rc_type ==HFI_RATE_CONTROL_OFF ) +return -EINVAL ; + +if (inst->codec ==V4L2_PIX_FMT_H264 ) { +if (!layer_enable || !inst->fw_caps[LAYER_COUNT_H264 ].value) +return -EINVAL ; + +if (inst->fw_caps[LAYER_TYPE_H264 ].value == +V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_P ) { +if (inst->hfi_rc_type ==HFI_RC_VBR_CFR ) + layer_type =HFI_HIER_P_HYBRID_LTR ; +else + layer_type =HFI_HIER_P_SLIDING_WINDOW ; + }else if (inst->fw_caps[LAYER_TYPE_HEVC ].value == +V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_B ) { +if (inst->hfi_rc_type ==HFI_RC_VBR_CFR ) + layer_type =HFI_HIER_B ; +else +return -EINVAL ; + }else { +return -EINVAL ; + } + }else if (inst->codec ==V4L2_PIX_FMT_HEVC ) { +if (!inst->fw_caps[LAYER_COUNT_HEVC ].value) +return -EINVAL ; + +if (inst->fw_caps[LAYER_TYPE_HEVC ].value == +V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_P ) { + layer_type =HFI_HIER_P_SLIDING_WINDOW ; + }else if (inst->fw_caps[LAYER_TYPE_HEVC ].value == +V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_B ) { +if (inst->hfi_rc_type ==HFI_RC_VBR_CFR ) + layer_type =HFI_HIER_B ; +else +return -EINVAL ; + }else { +return -EINVAL ; + } + }else { +return -EINVAL ; + } + + inst->hfi_layer_type = layer_type; + +return hfi_ops->session_set_property (inst, hfi_id, +HFI_HOST_FLAGS_NONE , +iris_get_port_info (inst, cap_id), +HFI_PAYLOAD_U32_ENUM , + &layer_type,sizeof (u32)); +} + +int iris_set_layer_count_gen1 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +{ +const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; +struct vb2_queue *sq =v4l2_m2m_get_src_vq (inst->m2m_ctx); +struct vb2_queue *dq =v4l2_m2m_get_dst_vq (inst->m2m_ctx); +u32 layer_enable = inst->fw_caps[LAYER_ENABLE ].value; +u32 layer_count = inst->fw_caps[cap_id].value; +u32 hfi_id, ret; + +if (!layer_enable || !layer_count) +return -EINVAL ; + + inst->hfi_layer_count = layer_count; + +if (!vb2_is_streaming (sq) && !vb2_is_streaming (dq)) { + hfi_id =HFI_PROPERTY_PARAM_VENC_HIER_P_MAX_NUM_ENH_LAYER ; + ret = hfi_ops->session_set_property (inst, hfi_id, +HFI_HOST_FLAGS_NONE , +iris_get_port_info (inst, cap_id), +HFI_PAYLOAD_U32 , + &layer_count,sizeof (u32)); +if (ret) +return ret; + } + + hfi_id = inst->fw_caps[cap_id].hfi_id; +return hfi_ops->session_set_property (inst, hfi_id, +HFI_HOST_FLAGS_NONE , +iris_get_port_info (inst, cap_id), +HFI_PAYLOAD_U32 , + &layer_count,sizeof (u32)); +} + +int iris_set_layer_count_gen2 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +{ +const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; +u32 hfi_id = inst->fw_caps[cap_id].hfi_id; +u32 layer_type = inst->hfi_layer_type; +u32 layer_count, layer_count_max; + + layer_count = (inst->codec ==V4L2_PIX_FMT_H264 ) ? + inst->fw_caps[LAYER_COUNT_H264 ].value : + inst->fw_caps[LAYER_COUNT_HEVC ].value; + +if (!layer_count) +return -EINVAL ; + +if (layer_type ==HFI_HIER_B ) { + layer_count_max =MAX_LAYER_HB ; + }else if (layer_type ==HFI_HIER_P_HYBRID_LTR ) { + layer_count_max =MAX_AVC_LAYER_HP_HYBRID_LTR ; + }else if (layer_type ==HFI_HIER_P_SLIDING_WINDOW ) { +if (inst->codec ==V4L2_PIX_FMT_H264 ) { + layer_count_max =MAX_AVC_LAYER_HP_SLIDING_WINDOW ; + }else { +if (inst->hfi_rc_type ==HFI_RC_VBR_CFR ) + layer_count_max =MAX_HEVC_VBR_LAYER_HP_SLIDING_WINDOW ; +else + layer_count_max =MAX_HEVC_LAYER_HP_SLIDING_WINDOW ; + } + }else { +return -EINVAL ; + } + +if (layer_count > layer_count_max) + layer_count = layer_count_max; + + layer_count +=1 ;/* base layer */ + inst->hfi_layer_count = layer_count; + +return hfi_ops->session_set_property (inst, hfi_id, +HFI_HOST_FLAGS_NONE , +iris_get_port_info (inst, cap_id), +HFI_PAYLOAD_U32 , + &layer_count,sizeof (u32)); +} + +int iris_set_layer_bitrate (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +{ +const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; +u32 hfi_id = inst->fw_caps[cap_id].hfi_id; +u32 bitrate = inst->fw_caps[cap_id].value; + +/* ignore layer bitrate when total bitrate is set */ +if (inst->fw_caps[BITRATE ].flags &CAP_FLAG_CLIENT_SET ) +return 0 ; + +if (!(inst->fw_caps[cap_id].flags &CAP_FLAG_CLIENT_SET )) +return -EINVAL ; + +return hfi_ops->session_set_property (inst, hfi_id, +HFI_HOST_FLAGS_NONE , +iris_get_port_info (inst, cap_id), +HFI_PAYLOAD_U32 , + &bitrate,sizeof (u32)); +} +int iris_set_properties (struct iris_inst *inst,u32 plane) {const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops;
drivers/media/platform/qcom/iris/iris_ctrls.h +6 -1
@@ -22,7 +22,8 @@ int iris_set_level(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_idint iris_set_profile_level_gen1 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_header_mode_gen1 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_header_mode_gen2 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); -int iris_set_bitrate (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); +int iris_set_bitrate_gen1 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); +int iris_set_bitrate_gen2 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_peak_bitrate (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_bitrate_mode_gen1 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_bitrate_mode_gen2 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);
@@ -42,6 +43,10 @@ int iris_set_use_ltr(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_int iris_set_mark_ltr (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_use_and_mark_ltr (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_intra_period (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); +int iris_set_layer_type (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); +int iris_set_layer_count_gen1 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); +int iris_set_layer_count_gen2 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); +int iris_set_layer_bitrate (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_properties (struct iris_inst *inst,u32 plane);#endif
drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +17 -4
@@ -602,11 +602,10 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *pbreak ; }case HFI_PROPERTY_CONFIG_VENC_TARGET_BITRATE : { -struct hfi_bitrate *brate = prop_data; -u32 *in = pdata; +struct hfi_bitrate *in = pdata, *brate = prop_data; - brate->bitrate = *in; - brate->layer_id =0 ; + brate->bitrate = in->bitrate; + brate->layer_id = in->layer_id; packet->shdr.hdr.size +=sizeof (u32) +sizeof (*brate);break ; }
@@ -726,6 +725,20 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p packet->shdr.hdr.size +=sizeof (u32) +sizeof (*intra_period);break ; } +case HFI_PROPERTY_PARAM_VENC_HIER_P_MAX_NUM_ENH_LAYER : { +u32 *in = pdata; + + packet->data[1 ] = *in; + packet->shdr.hdr.size +=sizeof (u32) +sizeof (u32); +break ; + } +case HFI_PROPERTY_CONFIG_VENC_HIER_P_ENH_LAYER : { +u32 *in = pdata; + + packet->data[1 ] = *in; + packet->shdr.hdr.size +=sizeof (u32) +sizeof (u32); +break ; + }default :return -EINVAL ; }
drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h +2 -0
@@ -154,11 +154,13 @@#define HFI_PROPERTY_PARAM_VENC_LTRMODE 0x200501c#define HFI_PROPERTY_PARAM_VENC_MAX_NUM_B_FRAMES 0x2005020 +#define HFI_PROPERTY_PARAM_VENC_HIER_P_MAX_NUM_ENH_LAYER 0x2005026#define HFI_PROPERTY_CONFIG_VENC_TARGET_BITRATE 0x2006001#define HFI_PROPERTY_CONFIG_VENC_INTRA_PERIOD 0x2006003#define HFI_PROPERTY_CONFIG_VENC_MARKLTRFRAME 0x2006009#define HFI_PROPERTY_CONFIG_VENC_USELTRFRAME 0x200600a#define HFI_PROPERTY_CONFIG_VENC_SYNC_FRAME_SEQUENCE_HEADER 0x2006008 +#define HFI_PROPERTY_CONFIG_VENC_HIER_P_ENH_LAYER 0x200600bstruct hfi_pkt_hdr {u32 size;
drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h +15 -0
@@ -74,7 +74,22 @@ enum hfi_rate_control {#define HFI_PROP_LTR_COUNT 0x03000134#define HFI_PROP_LTR_MARK 0x03000135#define HFI_PROP_LTR_USE 0x03000136 + +enum hfi_layer_encoding_type { +HFI_HIER_P_SLIDING_WINDOW =0x1 , +HFI_HIER_P_HYBRID_LTR =0x2 , +HFI_HIER_B =0x3 , +}; + +#define HFI_PROP_LAYER_ENCODING_TYPE 0x03000138 +#define HFI_PROP_LAYER_COUNT 0x03000139#define HFI_PROP_TOTAL_BITRATE 0x0300013b +#define HFI_PROP_BITRATE_LAYER1 0x0300013c +#define HFI_PROP_BITRATE_LAYER2 0x0300013d +#define HFI_PROP_BITRATE_LAYER3 0x0300013e +#define HFI_PROP_BITRATE_LAYER4 0x0300013f +#define HFI_PROP_BITRATE_LAYER5 0x03000140 +#define HFI_PROP_BITRATE_LAYER6 0x03000141#define HFI_PROP_MAX_GOP_FRAMES 0x03000146#define HFI_PROP_MAX_B_FRAMES 0x03000147#define HFI_PROP_QUALITY_MODE 0x03000148
drivers/media/platform/qcom/iris/iris_instance.h +4 -0
@@ -73,6 +73,8 @@ struct iris_fmt {* @enc_raw_height: source image height for encoder instance * @enc_scale_width: scale width for encoder instance * @enc_scale_height: scale height for encoder instance +* @hfi_layer_type: hierarchical coding layer type +* @hfi_layer_count: hierarchical coding layer count */ struct iris_inst {
@@ -115,6 +117,8 @@ struct iris_inst {u32 enc_raw_height;u32 enc_scale_width;u32 enc_scale_height; +u32 hfi_layer_type; + u32 hfi_layer_count; };#endif
drivers/media/platform/qcom/iris/iris_platform_common.h +23 -0
@@ -32,6 +32,12 @@ struct iris_inst;#define INVALID_DEFAULT_MARK_OR_USE_LTR -1#define MAX_LTR_FRAME_COUNT_GEN1 4#define MAX_LTR_FRAME_COUNT_GEN2 2 +#define MAX_LAYER_HB 3 +#define MAX_AVC_LAYER_HP_HYBRID_LTR 5 +#define MAX_AVC_LAYER_HP_SLIDING_WINDOW 3 +#define MAX_HEVC_LAYER_HP_SLIDING_WINDOW 3 +#define MAX_HEVC_VBR_LAYER_HP_SLIDING_WINDOW 5 +#define MAX_HIER_CODING_LAYER_GEN1 6enum stage_type {STAGE_1 =1 ,
@@ -156,6 +162,23 @@ enum platform_inst_fw_cap_type {MARK_LTR ,B_FRAME ,INTRA_PERIOD , +LAYER_ENABLE , +LAYER_TYPE_H264 , +LAYER_TYPE_HEVC , +LAYER_COUNT_H264 , +LAYER_COUNT_HEVC , +LAYER0_BITRATE_H264 , +LAYER1_BITRATE_H264 , +LAYER2_BITRATE_H264 , +LAYER3_BITRATE_H264 , +LAYER4_BITRATE_H264 , +LAYER5_BITRATE_H264 , +LAYER0_BITRATE_HEVC , +LAYER1_BITRATE_HEVC , +LAYER2_BITRATE_HEVC , +LAYER3_BITRATE_HEVC , +LAYER4_BITRATE_HEVC , +LAYER5_BITRATE_HEVC , INST_FW_CAP_MAX, };
drivers/media/platform/qcom/iris/iris_platform_gen1.c +93 -1
106 lines not on this page · raw message
drivers/media/platform/qcom/iris/iris_platform_gen2.c +181 -1
194 lines not on this page · raw message
drivers/media/platform/qcom/iris/iris_vpu_buffer.c +28 -0
@@ -670,6 +670,8 @@ static u32 iris_vpu_enc_bin_size(struct iris_inst *inst)static inline u32 hfi_buffer_get_recon_count (struct iris_inst *inst) { +u32 layer_count = inst->hfi_layer_count; +u32 layer_type = inst->hfi_layer_type;u32 bframe_count, ltr_count;u32 num_ref =1 ;
@@ -679,9 +681,35 @@ static inline u32 hfi_buffer_get_recon_count(struct iris_inst *inst)if (bframe_count) num_ref =2 ; +/* The shift operation here is rounding logic, similar to [(x+1)/2]. */ +if (layer_type ==HFI_HIER_P_HYBRID_LTR ) + num_ref = (layer_count +1 ) >>1 ; + +if (layer_type ==HFI_HIER_P_SLIDING_WINDOW ) { +if (inst->codec ==V4L2_PIX_FMT_HEVC ) + num_ref = (layer_count +1 ) >>1 ; +else if (inst->codec ==V4L2_PIX_FMT_H264 && layer_count <4 ) + num_ref = (layer_count -1 ); +else + num_ref = layer_count; + } +if (ltr_count) num_ref = num_ref + ltr_count; +/* +* The expression (1 << layers - 2) + 1 accounts for the number of reference +* frames in the Adaptive Hierarchical B-frame encoding case. In this scheme, +* the number of frames in a sub-GOP is related to (2^(number of layers) - 1), +* hence the use of the shift operation. +*/ +if (layer_type ==HFI_HIER_B ) { +if (inst->codec ==V4L2_PIX_FMT_HEVC ) + num_ref = layer_count; +else + num_ref = (1 << (layer_count -2 )) +1 ; + } +return num_ref; }
Signature · 1 line
2.43.0
Dikshita Agarwal
maintainerRe: [PATCH v3 4/6] media: qcom: iris: Add hierarchical coding support for encoder
Quoted 29 lines · Wangao Wang
Add hierarchical coding support for both gen1 and gen2 encoders by enabling
the following V4L2 controls:
H264:V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING,V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_TYPE,V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER
HEVC(gen2 only):V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_TYPE,V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_LAYERSigned-off-by Wangao Wang <[email protected]>--- drivers/media/platform/qcom/iris/iris_ctrls.c | 279 ++++++++++++++++++++- drivers/media/platform/qcom/iris/iris_ctrls.h | 7 +- .../platform/qcom/iris/iris_hfi_gen1_command.c | 21 +- .../platform/qcom/iris/iris_hfi_gen1_defines.h | 2 + .../platform/qcom/iris/iris_hfi_gen2_defines.h | 15 ++ drivers/media/platform/qcom/iris/iris_instance.h | 4 + .../platform/qcom/iris/iris_platform_common.h | 23 ++ .../media/platform/qcom/iris/iris_platform_gen1.c | 94 ++++++- .../media/platform/qcom/iris/iris_platform_gen2.c | 182 +++++++++++++- drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 28 +++ 10 files changed, 647 insertions(+), 8 deletions(-)drivers/media/platform/qcom/iris/iris_ctrls.c +0
-0
drivers/media/platform/qcom/iris/iris_ctrls.c static enum platform_inst_fw_cap_type iris_get_cap_id(u32 id) · line 120
Quoted 5 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +2
-0@@ -116,6 +116,40 @@ static enum platform_inst_fw_cap_type iris_get_cap_id(u32 id)return MARK_LTR ;case V4L2_CID_MPEG_VIDEO_B_FRAMES :return B_FRAME ; +case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING : +return LAYER_ENABLE ;
Will the same control be used for HEVC as well? I think this is applicable for only H264 encoders.
Quoted 82 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +67
-1+case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_TYPE : +return LAYER_TYPE_H264 ; +case V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_TYPE : +return LAYER_TYPE_HEVC ; +case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER : +return LAYER_COUNT_H264 ; +case V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_LAYER : +return LAYER_COUNT_HEVC ; +case V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L0_BR : +return LAYER0_BITRATE_H264 ; +case V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L1_BR : +return LAYER1_BITRATE_H264 ; +case V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L2_BR : +return LAYER2_BITRATE_H264 ; +case V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L3_BR : +return LAYER3_BITRATE_H264 ; +case V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L4_BR : +return LAYER4_BITRATE_H264 ; +case V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L5_BR : +return LAYER5_BITRATE_H264 ; +case V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L0_BR : +return LAYER0_BITRATE_HEVC ; +case V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L1_BR : +return LAYER1_BITRATE_HEVC ; +case V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L2_BR : +return LAYER2_BITRATE_HEVC ; +case V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L3_BR : +return LAYER3_BITRATE_HEVC ; +case V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L4_BR : +return LAYER4_BITRATE_HEVC ; +case V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L5_BR : +return LAYER5_BITRATE_HEVC ;default :return INST_FW_CAP_MAX ; }@@ -221,6 +255,40 @@ static u32 iris_get_v4l2_id(enum platform_inst_fw_cap_type cap_id)return V4L2_CID_MPEG_VIDEO_FRAME_LTR_INDEX ;case B_FRAME :return V4L2_CID_MPEG_VIDEO_B_FRAMES ; +case LAYER_ENABLE : +return V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING ; +case LAYER_TYPE_H264 : +return V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_TYPE ; +case LAYER_TYPE_HEVC : +return V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_TYPE ; +case LAYER_COUNT_H264 : +return V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER ; +case LAYER_COUNT_HEVC : +return V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_LAYER ; +case LAYER0_BITRATE_H264 : +return V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L0_BR ; +case LAYER1_BITRATE_H264 : +return V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L1_BR ; +case LAYER2_BITRATE_H264 : +return V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L2_BR ; +case LAYER3_BITRATE_H264 : +return V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L3_BR ; +case LAYER4_BITRATE_H264 : +return V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L4_BR ; +case LAYER5_BITRATE_H264 : +return V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L5_BR ; +case LAYER0_BITRATE_HEVC : +return V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L0_BR ; +case LAYER1_BITRATE_HEVC : +return V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L1_BR ; +case LAYER2_BITRATE_HEVC : +return V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L2_BR ; +case LAYER3_BITRATE_HEVC : +return V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L3_BR ; +case LAYER4_BITRATE_HEVC : +return V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L4_BR ; +case LAYER5_BITRATE_HEVC : +return V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L5_BR ;default :return 0 ; }@@ -567,7 +635,60 @@ int iris_set_header_mode_gen2(struct iris_inst *inst, enum platform_inst_fw_cap_ &hfi_val, sizeof(u32)); } -int iris_set_bitrate (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +int iris_set_bitrate_gen1 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id)
drivers/media/platform/qcom/iris/iris_ctrls.c int iris_set_header_mode_gen2(struct iris_inst *inst, enum platform_inst_fw_cap_ · line 648
Quoted 10 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +10
-0+{ +const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; +u32 entropy_mode = inst->fw_caps[ENTROPY_MODE ].value; +u32 bitrate = inst->fw_caps[cap_id].value; +u32 hfi_id = inst->fw_caps[cap_id].hfi_id; +struct hfi_bitrate hfi_val; +u32 max_bitrate; + +if (!(inst->fw_caps[cap_id].flags &CAP_FLAG_CLIENT_SET ) && cap_id !=BITRATE ) +return -EINVAL ;
Can you pls explain what is this check for?
Quoted 24 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +24
-0+ +if (inst->codec ==V4L2_PIX_FMT_HEVC ) + max_bitrate =CABAC_MAX_BITRATE ; + +if (entropy_mode ==V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC ) + max_bitrate =CABAC_MAX_BITRATE ; +else + max_bitrate =CAVLC_MAX_BITRATE ; + + hfi_val.bitrate =min (bitrate, max_bitrate); + +switch (cap_id) { +case BITRATE : +case LAYER0_BITRATE_H264 : + hfi_val.layer_id =0 ; +break ; +case LAYER1_BITRATE_H264 : + hfi_val.layer_id =1 ; +break ; +case LAYER2_BITRATE_H264 : + hfi_val.layer_id =2 ; +break ; +case LAYER3_BITRATE_H264 : + hfi_val.layer_id =3 ;
drivers/media/platform/qcom/iris/iris_ctrls.c int iris_set_header_mode_gen2(struct iris_inst *inst, enum platform_inst_fw_cap_ · line 682
Quoted 10 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +10
-0+break ; +case LAYER4_BITRATE_H264 : + hfi_val.layer_id =4 ; +break ; +case LAYER5_BITRATE_H264 : + hfi_val.layer_id =5 ; +break ; +default : +return -EINVAL ; + }
The layer bitrate should only be set if layer encoding is enabled, isn't it?
Quoted 29 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +22
-0+ +return hfi_ops->session_set_property (inst, hfi_id, +HFI_HOST_FLAGS_NONE , +iris_get_port_info (inst, cap_id), +HFI_PAYLOAD_STRUCTURE , + &hfi_val,sizeof (hfi_val)); +} + +int iris_set_bitrate_gen2 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) {const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops;u32 entropy_mode = inst->fw_caps[ENTROPY_MODE ].value;@@ -1183,6 +1304,162 @@ int iris_set_intra_period(struct iris_inst *inst, enum platform_inst_fw_cap_type &intra_period, sizeof(intra_period)); } +int iris_set_layer_type (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +{ +const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; +u32 layer_enable = inst->fw_caps[LAYER_ENABLE ].value; +u32 hfi_id = inst->fw_caps[cap_id].hfi_id; +u32 layer_type; + +if (inst->hfi_rc_type ==HFI_RATE_CONTROL_CQ || + inst->hfi_rc_type ==HFI_RATE_CONTROL_OFF ) +return -EINVAL ; + +if (inst->codec ==V4L2_PIX_FMT_H264 ) { +if (!layer_enable || !inst->fw_caps[LAYER_COUNT_H264 ].value)
drivers/media/platform/qcom/iris/iris_ctrls.c int iris_set_intra_period(struct iris_inst *inst, enum platform_inst_fw_cap_type · line 1329
Quoted 10 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +10
-0+return -EINVAL ; + +if (inst->fw_caps[LAYER_TYPE_H264 ].value == +V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_P ) { +if (inst->hfi_rc_type ==HFI_RC_VBR_CFR ) + layer_type =HFI_HIER_P_HYBRID_LTR ; +else + layer_type =HFI_HIER_P_SLIDING_WINDOW ; + }else if (inst->fw_caps[LAYER_TYPE_HEVC ].value == +V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_B ) {
why are you checking HEVC layer type for H264 codec? seems like a bug.
Quoted 42 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +42
-0+if (inst->hfi_rc_type ==HFI_RC_VBR_CFR ) + layer_type =HFI_HIER_B ; +else +return -EINVAL ; + }else { +return -EINVAL ; + } + }else if (inst->codec ==V4L2_PIX_FMT_HEVC ) { +if (!inst->fw_caps[LAYER_COUNT_HEVC ].value) +return -EINVAL ; + +if (inst->fw_caps[LAYER_TYPE_HEVC ].value == +V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_P ) { + layer_type =HFI_HIER_P_SLIDING_WINDOW ; + }else if (inst->fw_caps[LAYER_TYPE_HEVC ].value == +V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_B ) { +if (inst->hfi_rc_type ==HFI_RC_VBR_CFR ) + layer_type =HFI_HIER_B ; +else +return -EINVAL ; + }else { +return -EINVAL ; + } + }else { +return -EINVAL ; + } + + inst->hfi_layer_type = layer_type; + +return hfi_ops->session_set_property (inst, hfi_id, +HFI_HOST_FLAGS_NONE , +iris_get_port_info (inst, cap_id), +HFI_PAYLOAD_U32_ENUM , + &layer_type,sizeof (u32)); +} + +int iris_set_layer_count_gen1 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +{ +const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; +struct vb2_queue *sq =v4l2_m2m_get_src_vq (inst->m2m_ctx); +struct vb2_queue *dq =v4l2_m2m_get_dst_vq (inst->m2m_ctx); +u32 layer_enable = inst->fw_caps[LAYER_ENABLE ].value;
drivers/media/platform/qcom/iris/iris_ctrls.c int iris_set_intra_period(struct iris_inst *inst, enum platform_inst_fw_cap_type · line 1381
Quoted 10 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +10
-0+u32 layer_count = inst->fw_caps[cap_id].value; +u32 hfi_id, ret; + +if (!layer_enable || !layer_count) +return -EINVAL ; + + inst->hfi_layer_count = layer_count; + +if (!vb2_is_streaming (sq) && !vb2_is_streaming (dq)) { + hfi_id =HFI_PROPERTY_PARAM_VENC_HIER_P_MAX_NUM_ENH_LAYER ;
why the streaming check? and what's the significance of this setting? why this prop is set under streaming check?
Quoted 61 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +61
-0+ ret = hfi_ops->session_set_property (inst, hfi_id, +HFI_HOST_FLAGS_NONE , +iris_get_port_info (inst, cap_id), +HFI_PAYLOAD_U32 , + &layer_count,sizeof (u32)); +if (ret) +return ret; + } + + hfi_id = inst->fw_caps[cap_id].hfi_id; +return hfi_ops->session_set_property (inst, hfi_id, +HFI_HOST_FLAGS_NONE , +iris_get_port_info (inst, cap_id), +HFI_PAYLOAD_U32 , + &layer_count,sizeof (u32)); +} + +int iris_set_layer_count_gen2 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +{ +const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; +u32 hfi_id = inst->fw_caps[cap_id].hfi_id; +u32 layer_type = inst->hfi_layer_type; +u32 layer_count, layer_count_max; + + layer_count = (inst->codec ==V4L2_PIX_FMT_H264 ) ? + inst->fw_caps[LAYER_COUNT_H264 ].value : + inst->fw_caps[LAYER_COUNT_HEVC ].value; + +if (!layer_count) +return -EINVAL ; + +if (layer_type ==HFI_HIER_B ) { + layer_count_max =MAX_LAYER_HB ; + }else if (layer_type ==HFI_HIER_P_HYBRID_LTR ) { + layer_count_max =MAX_AVC_LAYER_HP_HYBRID_LTR ; + }else if (layer_type ==HFI_HIER_P_SLIDING_WINDOW ) { +if (inst->codec ==V4L2_PIX_FMT_H264 ) { + layer_count_max =MAX_AVC_LAYER_HP_SLIDING_WINDOW ; + }else { +if (inst->hfi_rc_type ==HFI_RC_VBR_CFR ) + layer_count_max =MAX_HEVC_VBR_LAYER_HP_SLIDING_WINDOW ; +else + layer_count_max =MAX_HEVC_LAYER_HP_SLIDING_WINDOW ; + } + }else { +return -EINVAL ; + } + +if (layer_count > layer_count_max) + layer_count = layer_count_max; + + layer_count +=1 ;/* base layer */ + inst->hfi_layer_count = layer_count; + +return hfi_ops->session_set_property (inst, hfi_id, +HFI_HOST_FLAGS_NONE , +iris_get_port_info (inst, cap_id), +HFI_PAYLOAD_U32 , + &layer_count,sizeof (u32)); +} +
drivers/media/platform/qcom/iris/iris_ctrls.c int iris_set_intra_period(struct iris_inst *inst, enum platform_inst_fw_cap_type · line 1452
Quoted 10 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +10
-0+int iris_set_layer_bitrate (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +{ +const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; +u32 hfi_id = inst->fw_caps[cap_id].hfi_id; +u32 bitrate = inst->fw_caps[cap_id].value; + +/* ignore layer bitrate when total bitrate is set */ +if (inst->fw_caps[BITRATE ].flags &CAP_FLAG_CLIENT_SET ) +return 0 ; +
any streaming check required here?
Quoted 333 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +10
-0+if (!(inst->fw_caps[cap_id].flags &CAP_FLAG_CLIENT_SET )) +return -EINVAL ; + +return hfi_ops->session_set_property (inst, hfi_id, +HFI_HOST_FLAGS_NONE , +iris_get_port_info (inst, cap_id), +HFI_PAYLOAD_U32 , + &bitrate,sizeof (u32)); +} +int iris_set_properties (struct iris_inst *inst,u32 plane) {const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops;drivers/media/platform/qcom/iris/iris_ctrls.h +6
-1@@ -22,7 +22,8 @@ int iris_set_level(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_idint iris_set_profile_level_gen1 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_header_mode_gen1 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_header_mode_gen2 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); -int iris_set_bitrate (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); +int iris_set_bitrate_gen1 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); +int iris_set_bitrate_gen2 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_peak_bitrate (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_bitrate_mode_gen1 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_bitrate_mode_gen2 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);@@ -42,6 +43,10 @@ int iris_set_use_ltr(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_int iris_set_mark_ltr (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_use_and_mark_ltr (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_intra_period (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); +int iris_set_layer_type (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); +int iris_set_layer_count_gen1 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); +int iris_set_layer_count_gen2 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id); +int iris_set_layer_bitrate (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id);int iris_set_properties (struct iris_inst *inst,u32 plane);#endif drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +17
-4@@ -602,11 +602,10 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *pbreak ; }case HFI_PROPERTY_CONFIG_VENC_TARGET_BITRATE : { -struct hfi_bitrate *brate = prop_data; -u32 *in = pdata; +struct hfi_bitrate *in = pdata, *brate = prop_data; - brate->bitrate = *in; - brate->layer_id =0 ; + brate->bitrate = in->bitrate; + brate->layer_id = in->layer_id; packet->shdr.hdr.size +=sizeof (u32) +sizeof (*brate);break ; }@@ -726,6 +725,20 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p packet->shdr.hdr.size +=sizeof (u32) +sizeof (*intra_period);break ; } +case HFI_PROPERTY_PARAM_VENC_HIER_P_MAX_NUM_ENH_LAYER : { +u32 *in = pdata; + + packet->data[1 ] = *in; + packet->shdr.hdr.size +=sizeof (u32) +sizeof (u32); +break ; + } +case HFI_PROPERTY_CONFIG_VENC_HIER_P_ENH_LAYER : { +u32 *in = pdata; + + packet->data[1 ] = *in; + packet->shdr.hdr.size +=sizeof (u32) +sizeof (u32); +break ; + }default :return -EINVAL ; }drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h +2
-0@@ -154,11 +154,13 @@#define HFI_PROPERTY_PARAM_VENC_LTRMODE 0x200501c#define HFI_PROPERTY_PARAM_VENC_MAX_NUM_B_FRAMES 0x2005020 +#define HFI_PROPERTY_PARAM_VENC_HIER_P_MAX_NUM_ENH_LAYER 0x2005026#define HFI_PROPERTY_CONFIG_VENC_TARGET_BITRATE 0x2006001#define HFI_PROPERTY_CONFIG_VENC_INTRA_PERIOD 0x2006003#define HFI_PROPERTY_CONFIG_VENC_MARKLTRFRAME 0x2006009#define HFI_PROPERTY_CONFIG_VENC_USELTRFRAME 0x200600a#define HFI_PROPERTY_CONFIG_VENC_SYNC_FRAME_SEQUENCE_HEADER 0x2006008 +#define HFI_PROPERTY_CONFIG_VENC_HIER_P_ENH_LAYER 0x200600bstruct hfi_pkt_hdr {u32 size;drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h +15
-0@@ -74,7 +74,22 @@ enum hfi_rate_control {#define HFI_PROP_LTR_COUNT 0x03000134#define HFI_PROP_LTR_MARK 0x03000135#define HFI_PROP_LTR_USE 0x03000136 + +enum hfi_layer_encoding_type { +HFI_HIER_P_SLIDING_WINDOW =0x1 , +HFI_HIER_P_HYBRID_LTR =0x2 , +HFI_HIER_B =0x3 , +}; + +#define HFI_PROP_LAYER_ENCODING_TYPE 0x03000138 +#define HFI_PROP_LAYER_COUNT 0x03000139#define HFI_PROP_TOTAL_BITRATE 0x0300013b +#define HFI_PROP_BITRATE_LAYER1 0x0300013c +#define HFI_PROP_BITRATE_LAYER2 0x0300013d +#define HFI_PROP_BITRATE_LAYER3 0x0300013e +#define HFI_PROP_BITRATE_LAYER4 0x0300013f +#define HFI_PROP_BITRATE_LAYER5 0x03000140 +#define HFI_PROP_BITRATE_LAYER6 0x03000141#define HFI_PROP_MAX_GOP_FRAMES 0x03000146#define HFI_PROP_MAX_B_FRAMES 0x03000147#define HFI_PROP_QUALITY_MODE 0x03000148drivers/media/platform/qcom/iris/iris_instance.h +4
-0@@ -73,6 +73,8 @@ struct iris_fmt {* @enc_raw_height: source image height for encoder instance * @enc_scale_width: scale width for encoder instance * @enc_scale_height: scale height for encoder instance +* @hfi_layer_type: hierarchical coding layer type +* @hfi_layer_count: hierarchical coding layer count */ struct iris_inst {@@ -115,6 +117,8 @@ struct iris_inst {u32 enc_raw_height;u32 enc_scale_width;u32 enc_scale_height; +u32 hfi_layer_type; + u32 hfi_layer_count; };#endif drivers/media/platform/qcom/iris/iris_platform_common.h +23
-0@@ -32,6 +32,12 @@ struct iris_inst;#define INVALID_DEFAULT_MARK_OR_USE_LTR -1#define MAX_LTR_FRAME_COUNT_GEN1 4#define MAX_LTR_FRAME_COUNT_GEN2 2 +#define MAX_LAYER_HB 3 +#define MAX_AVC_LAYER_HP_HYBRID_LTR 5 +#define MAX_AVC_LAYER_HP_SLIDING_WINDOW 3 +#define MAX_HEVC_LAYER_HP_SLIDING_WINDOW 3 +#define MAX_HEVC_VBR_LAYER_HP_SLIDING_WINDOW 5 +#define MAX_HIER_CODING_LAYER_GEN1 6enum stage_type {STAGE_1 =1 ,@@ -156,6 +162,23 @@ enum platform_inst_fw_cap_type {MARK_LTR ,B_FRAME ,INTRA_PERIOD , +LAYER_ENABLE , +LAYER_TYPE_H264 , +LAYER_TYPE_HEVC , +LAYER_COUNT_H264 , +LAYER_COUNT_HEVC , +LAYER0_BITRATE_H264 , +LAYER1_BITRATE_H264 , +LAYER2_BITRATE_H264 , +LAYER3_BITRATE_H264 , +LAYER4_BITRATE_H264 , +LAYER5_BITRATE_H264 , +LAYER0_BITRATE_HEVC , +LAYER1_BITRATE_HEVC , +LAYER2_BITRATE_HEVC , +LAYER3_BITRATE_HEVC , +LAYER4_BITRATE_HEVC , +LAYER5_BITRATE_HEVC , INST_FW_CAP_MAX, };drivers/media/platform/qcom/iris/iris_platform_gen1.c +93
-1@@ -142,7 +142,7 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { .hfi_id = HFI_PROPERTY_CONFIG_VENC_TARGET_BITRATE, .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, - .set = iris_set_bitrate, + .set = iris_set_bitrate_gen1, }, { .cap_id = BITRATE_MODE,@@ -297,6 +297,98 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8250_enc[] = { .flags = CAP_FLAG_OUTPUT_PORT, .set = iris_set_intra_period, }, + { + .cap_id = LAYER_ENABLE, + .min = 0, + .max = 1, + .step_or_mask = 1, + .value = 0, + .flags = CAP_FLAG_OUTPUT_PORT, + }, + { + .cap_id = LAYER_TYPE_H264, + .min = V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_P, + .max = V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_P, + .step_or_mask = BIT(V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_P), + .value = V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_P, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + }, + { + .cap_id = LAYER_COUNT_H264, + .min = 0, + .max = MAX_HIER_CODING_LAYER_GEN1, + .step_or_mask = 1, + .value = 0, + .hfi_id = HFI_PROPERTY_CONFIG_VENC_HIER_P_ENH_LAYER, + .flags = CAP_FLAG_OUTPUT_PORT, + .set = iris_set_layer_count_gen1, + }, + { + .cap_id = LAYER0_BITRATE_H264, + .min = 1, + .max = BITRATE_MAX, + .step_or_mask = 1, + .value = BITRATE_DEFAULT, + .hfi_id = HFI_PROPERTY_CONFIG_VENC_TARGET_BITRATE, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_bitrate_gen1, + }, + { + .cap_id = LAYER1_BITRATE_H264, + .min = 1, + .max = BITRATE_MAX, + .step_or_mask = 1, + .value = BITRATE_DEFAULT, + .hfi_id = HFI_PROPERTY_CONFIG_VENC_TARGET_BITRATE, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_bitrate_gen1, + }, + { + .cap_id = LAYER2_BITRATE_H264, + .min = 1, + .max = BITRATE_MAX, + .step_or_mask = 1, + .value = BITRATE_DEFAULT, + .hfi_id = HFI_PROPERTY_CONFIG_VENC_TARGET_BITRATE, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_bitrate_gen1, + }, + { + .cap_id = LAYER3_BITRATE_H264, + .min = 1, + .max = BITRATE_MAX, + .step_or_mask = 1, + .value = BITRATE_DEFAULT, + .hfi_id = HFI_PROPERTY_CONFIG_VENC_TARGET_BITRATE, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_bitrate_gen1, + }, + { + .cap_id = LAYER4_BITRATE_H264, + .min = 1, + .max = BITRATE_MAX, + .step_or_mask = 1, + .value = BITRATE_DEFAULT, + .hfi_id = HFI_PROPERTY_CONFIG_VENC_TARGET_BITRATE, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_bitrate_gen1, + }, + { + .cap_id = LAYER5_BITRATE_H264, + .min = 1, + .max = BITRATE_MAX, + .step_or_mask = 1, + .value = BITRATE_DEFAULT, + .hfi_id = HFI_PROPERTY_CONFIG_VENC_TARGET_BITRATE, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_bitrate_gen1, + }, }; static struct platform_inst_caps platform_inst_cap_sm8250 = {drivers/media/platform/qcom/iris/iris_platform_gen2.c +19
-1@@ -313,7 +313,7 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8550_enc[] = { .hfi_id = HFI_PROP_TOTAL_BITRATE, .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, - .set = iris_set_bitrate, + .set = iris_set_bitrate_gen2, }, { .cap_id = BITRATE_PEAK,@@ -677,6 +677,186 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8550_enc[] = { .flags = CAP_FLAG_OUTPUT_PORT, .set = iris_set_u32, }, + { + .cap_id = LAYER_ENABLE, + .min = 0, + .max = 1, + .step_or_mask = 1, + .value = 0, + .flags = CAP_FLAG_OUTPUT_PORT, + }, + { + .cap_id = LAYER_TYPE_H264, + .min = V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_B, + .max = V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_P, + .step_or_mask = BIT(V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_B) | + BIT(V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_P), + .value = V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_P, + .hfi_id = HFI_PROP_LAYER_ENCODING_TYPE, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + },
drivers/media/platform/qcom/iris/iris_platform_gen2.c static const struct platform_inst_fw_cap inst_fw_cap_sm8550_enc[] = { · line 707
Quoted 10 lines
drivers/media/platform/qcom/iris/iris_platform_gen2.c +10
-0+ { + .cap_id = LAYER_TYPE_HEVC, + .min = V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_B, + .max = V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_P, + .step_or_mask =BIT (V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_B ) | + BIT(V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_P), + .value = V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_P, + .hfi_id = HFI_PROP_LAYER_ENCODING_TYPE, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + .set = iris_set_layer_type,
layer type depends on layer count so shouldn't you have layer count before layer type in caps? or handle both in same set API?
Quoted 190 lines
drivers/media/platform/qcom/iris/iris_platform_gen2.c +152
-0+ }, + { + .cap_id = LAYER_COUNT_H264, + .min = 0, + .max = 5, + .step_or_mask = 1, + .value = 0, + .hfi_id = HFI_PROP_LAYER_COUNT, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, + }, + { + .cap_id = LAYER_COUNT_HEVC, + .min = 0, + .max = 5, + .step_or_mask = 1, + .value = 0, + .hfi_id = HFI_PROP_LAYER_COUNT, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_layer_count_gen2, + }, + { + .cap_id = LAYER0_BITRATE_H264, + .min = 1, + .max = BITRATE_MAX, + .step_or_mask = 1, + .value = BITRATE_DEFAULT, + .hfi_id = HFI_PROP_BITRATE_LAYER1, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_layer_bitrate, + }, + { + .cap_id = LAYER1_BITRATE_H264, + .min = 1, + .max = BITRATE_MAX, + .step_or_mask = 1, + .value = BITRATE_DEFAULT, + .hfi_id = HFI_PROP_BITRATE_LAYER2, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_layer_bitrate, + }, + { + .cap_id = LAYER2_BITRATE_H264, + .min = 1, + .max = BITRATE_MAX, + .step_or_mask = 1, + .value = BITRATE_DEFAULT, + .hfi_id = HFI_PROP_BITRATE_LAYER3, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_layer_bitrate, + }, + { + .cap_id = LAYER3_BITRATE_H264, + .min = 1, + .max = BITRATE_MAX, + .step_or_mask = 1, + .value = BITRATE_DEFAULT, + .hfi_id = HFI_PROP_BITRATE_LAYER4, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_layer_bitrate, + }, + { + .cap_id = LAYER4_BITRATE_H264, + .min = 1, + .max = BITRATE_MAX, + .step_or_mask = 1, + .value = BITRATE_DEFAULT, + .hfi_id = HFI_PROP_BITRATE_LAYER5, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_layer_bitrate, + }, + { + .cap_id = LAYER5_BITRATE_H264, + .min = 1, + .max = BITRATE_MAX, + .step_or_mask = 1, + .value = BITRATE_DEFAULT, + .hfi_id = HFI_PROP_BITRATE_LAYER6, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_layer_bitrate, + }, + { + .cap_id = LAYER0_BITRATE_HEVC, + .min = 1, + .max = BITRATE_MAX, + .step_or_mask = 1, + .value = BITRATE_DEFAULT, + .hfi_id = HFI_PROP_BITRATE_LAYER1, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_layer_bitrate, + }, + { + .cap_id = LAYER1_BITRATE_HEVC, + .min = 1, + .max = BITRATE_MAX, + .step_or_mask = 1, + .value = BITRATE_DEFAULT, + .hfi_id = HFI_PROP_BITRATE_LAYER2, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_layer_bitrate, + }, + { + .cap_id = LAYER2_BITRATE_HEVC, + .min = 1, + .max = BITRATE_MAX, + .step_or_mask = 1, + .value = BITRATE_DEFAULT, + .hfi_id = HFI_PROP_BITRATE_LAYER3, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_layer_bitrate, + }, + { + .cap_id = LAYER3_BITRATE_HEVC, + .min = 1, + .max = BITRATE_MAX, + .step_or_mask = 1, + .value = BITRATE_DEFAULT, + .hfi_id = HFI_PROP_BITRATE_LAYER4, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_layer_bitrate, + }, + { + .cap_id = LAYER4_BITRATE_HEVC, + .min = 1, + .max = BITRATE_MAX, + .step_or_mask = 1, + .value = BITRATE_DEFAULT, + .hfi_id = HFI_PROP_BITRATE_LAYER5, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_layer_bitrate, + }, + { + .cap_id = LAYER5_BITRATE_HEVC, + .min = 1, + .max = BITRATE_MAX, + .step_or_mask = 1, + .value = BITRATE_DEFAULT, + .hfi_id = HFI_PROP_BITRATE_LAYER5, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + .set = iris_set_layer_bitrate, + } }; static struct platform_inst_caps platform_inst_cap_sm8550 = {drivers/media/platform/qcom/iris/iris_vpu_buffer.c +17
-0@@ -670,6 +670,8 @@ static u32 iris_vpu_enc_bin_size(struct iris_inst *inst)static inline u32 hfi_buffer_get_recon_count (struct iris_inst *inst) { +u32 layer_count = inst->hfi_layer_count; +u32 layer_type = inst->hfi_layer_type;u32 bframe_count, ltr_count;u32 num_ref =1 ;@@ -679,9 +681,35 @@ static inline u32 hfi_buffer_get_recon_count(struct iris_inst *inst)if (bframe_count) num_ref =2 ; +/* The shift operation here is rounding logic, similar to [(x+1)/2]. */ +if (layer_type ==HFI_HIER_P_HYBRID_LTR ) + num_ref = (layer_count +1 ) >>1 ; + +if (layer_type ==HFI_HIER_P_SLIDING_WINDOW ) { +if (inst->codec ==V4L2_PIX_FMT_HEVC ) + num_ref = (layer_count +1 ) >>1 ; +else if (inst->codec ==V4L2_PIX_FMT_H264 && layer_count <4 ) + num_ref = (layer_count -1 ); +else + num_ref = layer_count; + } +if (ltr_count) num_ref = num_ref + ltr_count; +/* +* The expression (1 << layers - 2) + 1 accounts for the number of reference
drivers/media/platform/qcom/iris/iris_vpu_buffer.c static inline u32 hfi_buffer_get_recon_count(struct iris_inst *inst) · line 711
Quoted 10 lines
drivers/media/platform/qcom/iris/iris_vpu_buffer.c +10
-0+* frames in the Adaptive Hierarchical B-frame encoding case. In this scheme, +* the number of frames in a sub-GOP is related to (2^(number of layers) - 1), +* hence the use of the shift operation. +*/ +if (layer_type ==HFI_HIER_B ) { +if (inst->codec ==V4L2_PIX_FMT_HEVC ) + num_ref = layer_count; +else + num_ref = (1 << (layer_count -2 )) +1 ; + }
were you able to test these different scenarios?
Thanks,
Dikshita
Quoted 5 lines
drivers/media/platform/qcom/iris/iris_vpu_buffer.c +1
-0+return num_ref; }
Re: [PATCH v3 4/6] media: qcom: iris: Add hierarchical coding support for encoder
Quoted 10 lines · Dikshita Agarwal
drivers/media/platform/qcom/iris/iris_ctrls.c static enum platform_inst_fw_cap_type iris_get_cap_id(u32 id) · line 120
Quoted 6 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +2
-0@@ -116,6 +116,40 @@ static enum platform_inst_fw_cap_type iris_get_cap_id(u32 id)return MARK_LTR ;case V4L2_CID_MPEG_VIDEO_B_FRAMES :return B_FRAME ; +case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING : +return LAYER_ENABLE ;Will the same control be used for HEVC as well? I think this is applicable for only H264 encoders.
H264 flow:V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING, V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_TYPE, V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER HEVC flow:V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_TYPE, V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_LAYER
LAYER_ENABLE is used for H.264. In the HEVC flow, this flag is considered redundant—once the type is set, layer encoding is implicitly enabled.
Quoted 14 lines
drivers/media/platform/qcom/iris/iris_ctrls.c int iris_set_header_mode_gen2(struct iris_inst *inst, enum platform_inst_fw_cap_ · line 648
Quoted 11 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +11
-0+int iris_set_bitrate_gen1 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +{ +const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; +u32 entropy_mode = inst->fw_caps[ENTROPY_MODE ].value; +u32 bitrate = inst->fw_caps[cap_id].value; +u32 hfi_id = inst->fw_caps[cap_id].hfi_id; +struct hfi_bitrate hfi_val; +u32 max_bitrate; + +if (!(inst->fw_caps[cap_id].flags &CAP_FLAG_CLIENT_SET ) && cap_id !=BITRATE ) +return -EINVAL ;Can you pls explain what is this check for?
Quoted 2 lines
The layer bitrate should only be set if layer encoding is enabled, isn't it?
This flag is used to confirm whether the corresponding ctrl has been invoked. A check should also be added to determine whether layer encoding is enabled, while excluding bitrate configuration in non–layer‑encoding scenarios.
Quoted 6 lines
Quoted 6 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +6
-0+if (inst->codec ==V4L2_PIX_FMT_H264 ) { +if (!layer_enable || !inst->fw_caps[LAYER_COUNT_H264 ].value) +return -EINVAL ; + +if (inst->fw_caps[LAYER_TYPE_H264 ].value == +V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_P ) {
Quoted 9 lines
drivers/media/platform/qcom/iris/iris_ctrls.c int iris_set_intra_period(struct iris_inst *inst, enum platform_inst_fw_cap_type · line 1329
Quoted 6 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +6
-0+if (inst->hfi_rc_type ==HFI_RC_VBR_CFR ) + layer_type =HFI_HIER_P_HYBRID_LTR ; +else + layer_type =HFI_HIER_P_SLIDING_WINDOW ; + }else if (inst->fw_caps[LAYER_TYPE_HEVC ].value == +V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_B ) {why are you checking HEVC layer type for H264 codec? seems like a bug.
This is a bug and will be fixed in v4.
Quoted 11 lines
Quoted 11 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +11
-0+int iris_set_layer_count_gen1 (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +{ +const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; +struct vb2_queue *sq =v4l2_m2m_get_src_vq (inst->m2m_ctx); +struct vb2_queue *dq =v4l2_m2m_get_dst_vq (inst->m2m_ctx); +u32 layer_enable = inst->fw_caps[LAYER_ENABLE ].value; +u32 layer_count = inst->fw_caps[cap_id].value; +u32 hfi_id, ret; + +if (!layer_enable || !layer_count) +return -EINVAL ;
Quoted 9 lines
drivers/media/platform/qcom/iris/iris_ctrls.c int iris_set_intra_period(struct iris_inst *inst, enum platform_inst_fw_cap_type · line 1381
Quoted 5 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +5
-0+ + inst->hfi_layer_count = layer_count; + +if (!vb2_is_streaming (sq) && !vb2_is_streaming (dq)) { + hfi_id =HFI_PROPERTY_PARAM_VENC_HIER_P_MAX_NUM_ENH_LAYER ;why the streaming check? and what's the significance of this setting? why this prop is set under streaming check?
This property needs to be set to the firmware before streaming. It represents the maximum layer count and is static; any dynamically configured layer count later must not exceed this maximum.
Quoted 5 lines
Quoted 5 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +5
-0+ +int iris_set_layer_bitrate (struct iris_inst *inst,enum platform_inst_fw_cap_type cap_id) +{ +const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; +u32 hfi_id = inst->fw_caps[cap_id].hfi_id;
Quoted 9 lines
drivers/media/platform/qcom/iris/iris_ctrls.c int iris_set_intra_period(struct iris_inst *inst, enum platform_inst_fw_cap_type · line 1452
Quoted 6 lines
drivers/media/platform/qcom/iris/iris_ctrls.c +6
-0+u32 bitrate = inst->fw_caps[cap_id].value; + +/* ignore layer bitrate when total bitrate is set */ +if (inst->fw_caps[BITRATE ].flags &CAP_FLAG_CLIENT_SET ) +return 0 ; +any streaming check required here?
A streaming check will be added here.
Quoted 5 lines
Quoted 5 lines
drivers/media/platform/qcom/iris/iris_platform_gen2.c +5
-0+ { + .cap_id = LAYER_TYPE_HEVC, + .min = V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_B, + .max = V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_P, + .step_or_mask = BIT(V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_B) |
Quoted 9 lines
drivers/media/platform/qcom/iris/iris_platform_gen2.c static const struct platform_inst_fw_cap inst_fw_cap_sm8550_enc[] = { · line 707
Quoted 5 lines
drivers/media/platform/qcom/iris/iris_platform_gen2.c +5
-0+BIT (V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_P ), + .value = V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_P, + .hfi_id = HFI_PROP_LAYER_ENCODING_TYPE, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + .set = iris_set_layer_type,layer type depends on layer count so shouldn't you have layer count before layer type in caps? or handle both in same set API?
In the iris_set_layer_type API, there is a check for the layer count. If the count is 0, layer type will not be set to the firmware.
Quoted 6 lines
Quoted 6 lines
drivers/media/platform/qcom/iris/iris_vpu_buffer.c +6
-0+/* +* The expression (1 << layers - 2) + 1 accounts for the number of reference +* frames in the Adaptive Hierarchical B-frame encoding case. In this scheme, +* the number of frames in a sub-GOP is related to (2^(number of layers) - 1), +* hence the use of the shift operation. +*/
Quoted 9 lines
drivers/media/platform/qcom/iris/iris_vpu_buffer.c static inline u32 hfi_buffer_get_recon_count(struct iris_inst *inst) · line 711
Quoted 6 lines
drivers/media/platform/qcom/iris/iris_vpu_buffer.c +6
-0+if (layer_type ==HFI_HIER_B ) { +if (inst->codec ==V4L2_PIX_FMT_HEVC ) + num_ref = layer_count; +else + num_ref = (1 << (layer_count -2 )) +1 ; + }were you able to test these different scenarios?
Okay, I will test the other scenarios.
Signature · 2 lines
Best Regards, Wangao
PATCH v3 5/6 media: qcom: iris: Optimize iris_hfi_gen1_packet_session_set_property
In reply to Wangao WangModify iris_hfi_gen1_packet_session_set_property to simplify size calculations and remove redundant code patterns.
Previously, packet->shdr.hdr.size was incremented by sizeof(u32) in every switch case, resulting in repetitive and less maintainable logic.
--- .../platform/qcom/iris/iris_hfi_gen1_command.c | 50 +++++++++++----------- 1 file changed, 25 insertions(+), 25 deletions(-)
drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +25 -25
@@ -483,7 +483,7 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p {void *prop_data = &packet->data[1 ]; - packet->shdr.hdr.size =sizeof (*packet); + packet->shdr.hdr.size =sizeof (*packet) +sizeof (ptype); packet->shdr.hdr.pkt_type =HFI_CMD_SESSION_SET_PROPERTY ; packet->shdr.session_id = inst->session_id; packet->num_properties =1 ;
@@ -496,14 +496,14 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p fsize->buffer_type = in->buffer_type; fsize->height = in->height; fsize->width = in->width; - packet->shdr.hdr.size +=sizeof (u32) +sizeof (*fsize); + packet->shdr.hdr.size +=sizeof (*fsize);break ; }case HFI_PROPERTY_CONFIG_VIDEOCORES_USAGE : {struct hfi_videocores_usage_type *in = pdata, *cu = prop_data; cu->video_core_enable_mask = in->video_core_enable_mask; - packet->shdr.hdr.size +=sizeof (u32) +sizeof (*cu); + packet->shdr.hdr.size +=sizeof (*cu);break ; }case HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SELECT : {
@@ -512,7 +512,7 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p hfi->buffer_type = in->buffer_type; hfi->format = in->format; - packet->shdr.hdr.size +=sizeof (u32) +sizeof (*hfi); + packet->shdr.hdr.size +=sizeof (*hfi);break ; }case HFI_PROPERTY_PARAM_UNCOMPRESSED_PLANE_ACTUAL_CONSTRAINTS_INFO : {
@@ -531,7 +531,7 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p info->plane_format[1 ].buffer_alignment =256 ; } - packet->shdr.hdr.size +=sizeof (u32) +sizeof (*info); + packet->shdr.hdr.size +=sizeof (*info);break ; }case HFI_PROPERTY_PARAM_BUFFER_COUNT_ACTUAL : {
@@ -541,7 +541,7 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p count->type = in->type; count->count_actual = in->count_actual; count->count_min_host = in->count_min_host; - packet->shdr.hdr.size +=sizeof (u32) +sizeof (*count); + packet->shdr.hdr.size +=sizeof (*count);break ; }case HFI_PROPERTY_PARAM_VDEC_MULTI_STREAM : {
@@ -550,7 +550,7 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p multi->buffer_type = in->buffer_type; multi->enable = in->enable; - packet->shdr.hdr.size +=sizeof (u32) +sizeof (*multi); + packet->shdr.hdr.size +=sizeof (*multi);break ; }case HFI_PROPERTY_PARAM_BUFFER_SIZE_ACTUAL : {
@@ -558,7 +558,7 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p sz->size = in->size; sz->type = in->type; - packet->shdr.hdr.size +=sizeof (u32) +sizeof (*sz); + packet->shdr.hdr.size +=sizeof (*sz);break ; }case HFI_PROPERTY_PARAM_WORK_ROUTE : {
@@ -566,7 +566,7 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *pu32 *in = pdata; wr->video_work_route = *in; - packet->shdr.hdr.size +=sizeof (u32) +sizeof (*wr); + packet->shdr.hdr.size +=sizeof (*wr);break ; }case HFI_PROPERTY_PARAM_WORK_MODE : {
@@ -574,7 +574,7 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *pu32 *in = pdata; wm->video_work_mode = *in; - packet->shdr.hdr.size +=sizeof (u32) +sizeof (*wm); + packet->shdr.hdr.size +=sizeof (*wm);break ; }case HFI_PROPERTY_PARAM_PROFILE_LEVEL_CURRENT : {
@@ -590,7 +590,7 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p/* Level not supported, falling back to 1 */ pl->level =1 ; - packet->shdr.hdr.size +=sizeof (u32) +sizeof (*pl); + packet->shdr.hdr.size +=sizeof (*pl);break ; }case HFI_PROPERTY_CONFIG_VENC_SYNC_FRAME_SEQUENCE_HEADER : {
@@ -598,7 +598,7 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *pu32 *in = pdata; en->enable = *in; - packet->shdr.hdr.size +=sizeof (u32) +sizeof (*en); + packet->shdr.hdr.size +=sizeof (*en);break ; }case HFI_PROPERTY_CONFIG_VENC_TARGET_BITRATE : {
@@ -606,7 +606,7 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p brate->bitrate = in->bitrate; brate->layer_id = in->layer_id; - packet->shdr.hdr.size +=sizeof (u32) +sizeof (*brate); + packet->shdr.hdr.size +=sizeof (*brate);break ; }case HFI_PROPERTY_PARAM_VENC_RATE_CONTROL : {
@@ -625,7 +625,7 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p } packet->data[1 ] = *in; - packet->shdr.hdr.size +=sizeof (u32) *2 ; + packet->shdr.hdr.size +=sizeof (u32);break ; }case HFI_PROPERTY_PARAM_VENC_H264_ENTROPY_CONTROL : {
@@ -635,7 +635,7 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p entropy->entropy_mode = *in;if (entropy->entropy_mode ==HFI_H264_ENTROPY_CABAC ) entropy->cabac_model =HFI_H264_CABAC_MODEL_0 ; - packet->shdr.hdr.size +=sizeof (u32) +sizeof (*entropy); + packet->shdr.hdr.size +=sizeof (*entropy);break ; }case HFI_PROPERTY_PARAM_VENC_SESSION_QP_RANGE_V2 : {
@@ -660,7 +660,7 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p ((max_qp &0xFF ) <<16 ); range->min_qp.enable =7 ; range->max_qp.enable =7 ; - packet->shdr.hdr.size +=sizeof (u32) +sizeof (*range); + packet->shdr.hdr.size +=sizeof (*range);break ; }case HFI_PROPERTY_CONFIG_FRAME_RATE : {
@@ -669,7 +669,7 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p frate->buffer_type = in->buffer_type; frate->framerate = in->framerate; - packet->shdr.hdr.size +=sizeof (u32) +sizeof (*frate); + packet->shdr.hdr.size +=sizeof (*frate);break ; }case HFI_PROPERTY_PARAM_UNCOMPRESSED_PLANE_ACTUAL_INFO : {
@@ -681,7 +681,7 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p plane_actual_info->plane_format[0 ] = in->plane_format[0 ];if (in->num_planes >1 ) plane_actual_info->plane_format[1 ] = in->plane_format[1 ]; - packet->shdr.hdr.size +=sizeof (u32) +sizeof (*plane_actual_info); + packet->shdr.hdr.size +=sizeof (*plane_actual_info);break ; }case HFI_PROPERTY_PARAM_VENC_INTRA_REFRESH : {
@@ -689,7 +689,7 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p intra_refresh->mode = in->mode; intra_refresh->mbs = in->mbs; - packet->shdr.hdr.size +=sizeof (u32) +sizeof (*intra_refresh); + packet->shdr.hdr.size +=sizeof (*intra_refresh);break ; }case HFI_PROPERTY_PARAM_VENC_LTRMODE : {
@@ -698,7 +698,7 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p ltr_mode->mode = in->mode; ltr_mode->count = in->count; ltr_mode->trust_mode = in->trust_mode; - packet->shdr.hdr.size +=sizeof (u32) +sizeof (*ltr_mode); + packet->shdr.hdr.size +=sizeof (*ltr_mode);break ; }case HFI_PROPERTY_CONFIG_VENC_USELTRFRAME : {
@@ -707,14 +707,14 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p ltr_use->frames = in->frames; ltr_use->ref_ltr = in->ref_ltr; ltr_use->use_constrnt = in->use_constrnt; - packet->shdr.hdr.size +=sizeof (u32) +sizeof (*ltr_use); + packet->shdr.hdr.size +=sizeof (*ltr_use);break ; }case HFI_PROPERTY_CONFIG_VENC_MARKLTRFRAME : {struct hfi_ltr_mark *in = pdata, *ltr_mark = prop_data; ltr_mark->mark_frame = in->mark_frame; - packet->shdr.hdr.size +=sizeof (u32) +sizeof (*ltr_mark); + packet->shdr.hdr.size +=sizeof (*ltr_mark);break ; }case HFI_PROPERTY_CONFIG_VENC_INTRA_PERIOD : {
@@ -722,21 +722,21 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p intra_period->pframes = in->pframes; intra_period->bframes = in->bframes; - packet->shdr.hdr.size +=sizeof (u32) +sizeof (*intra_period); + packet->shdr.hdr.size +=sizeof (*intra_period);break ; }case HFI_PROPERTY_PARAM_VENC_HIER_P_MAX_NUM_ENH_LAYER : {u32 *in = pdata; packet->data[1 ] = *in; - packet->shdr.hdr.size +=sizeof (u32) +sizeof (u32); + packet->shdr.hdr.size +=sizeof (u32);break ; }case HFI_PROPERTY_CONFIG_VENC_HIER_P_ENH_LAYER : {u32 *in = pdata; packet->data[1 ] = *in; - packet->shdr.hdr.size +=sizeof (u32) +sizeof (u32); + packet->shdr.hdr.size +=sizeof (u32);break ; }default :
Signature · 1 line
2.43.0
Re: [PATCH v3 5/6] media: qcom: iris: Optimize iris_hfi_gen1_packet_session_set_property
Quoted 12 lines · Wangao Wang
Modify
iris_hfi_gen1_packet_session_set_propertyto simplify size calculations and remove redundant code patterns.Previously,
packet->shdr.hdr.sizewas incremented by sizeof(u32) in every switch case, resulting in repetitive and less maintainable logic.Signed-off-by Wangao Wang <[email protected]>--- .../platform/qcom/iris/iris_hfi_gen1_command.c | 50 +++++++++++----------- 1 file changed, 25 insertions(+), 25 deletions(-)
Thanks,
Dikshita
PATCH v3 6/6 media: qcom: iris: Simplify COMV size calculation
In reply to Wangao WangUnify AVC/HEVC handling by computing codec and lcu_size upfront.
--- drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-)
drivers/media/platform/qcom/iris/iris_vpu_buffer.c +5 -8
@@ -739,16 +739,13 @@ static u32 iris_vpu_enc_comv_size(struct iris_inst *inst)u32 height =iris_vpu_enc_get_bitstream_height (inst);u32 width =iris_vpu_enc_get_bitstream_width (inst);u32 num_recon =hfi_buffer_get_recon_count (inst); -u32 lcu_size =16 ; +u32 codec, lcu_size; -if (inst->codec ==V4L2_PIX_FMT_HEVC ) { - lcu_size =32 ; -return hfi_buffer_comv_enc (width, height, lcu_size, - num_recon +1 ,HFI_CODEC_ENCODE_HEVC ); - } + codec = (inst->codec ==V4L2_PIX_FMT_HEVC ) ? +HFI_CODEC_ENCODE_HEVC :HFI_CODEC_ENCODE_AVC ; + lcu_size = (inst->codec ==V4L2_PIX_FMT_HEVC ) ?32 :16 ; -return hfi_buffer_comv_enc (width, height, lcu_size, - num_recon +1 ,HFI_CODEC_ENCODE_AVC ); +return hfi_buffer_comv_enc (width, height, lcu_size, num_recon +1 , codec); } static inline
Signature · 1 line
2.43.0
Re: [PATCH v3 6/6] media: qcom: iris: Simplify COMV size calculation
Quoted 24 lines · Wangao Wang
Unify AVC/HEVC handling by computing codec and
lcu_sizeupfront.Signed-off-by Wangao Wang <[email protected]>--- drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-)drivers/media/platform/qcom/iris/iris_vpu_buffer.c +1
-6@@ -739,16 +739,13 @@ static u32 iris_vpu_enc_comv_size(struct iris_inst *inst)u32 height =iris_vpu_enc_get_bitstream_height (inst);u32 width =iris_vpu_enc_get_bitstream_width (inst);u32 num_recon =hfi_buffer_get_recon_count (inst); -u32 lcu_size =16 ; +u32 codec, lcu_size; -if (inst->codec ==V4L2_PIX_FMT_HEVC ) { - lcu_size =32 ; -return hfi_buffer_comv_enc (width, height, lcu_size, - num_recon +1 ,HFI_CODEC_ENCODE_HEVC ); - }
drivers/media/platform/qcom/iris/iris_vpu_buffer.c static u32 iris_vpu_enc_comv_size(struct iris_inst *inst) · line 751
Quoted 10 lines
drivers/media/platform/qcom/iris/iris_vpu_buffer.c +4
-2+ codec = (inst->codec ==V4L2_PIX_FMT_HEVC ) ? +HFI_CODEC_ENCODE_HEVC :HFI_CODEC_ENCODE_AVC ; + lcu_size = (inst->codec ==V4L2_PIX_FMT_HEVC ) ?32 :16 ; -return hfi_buffer_comv_enc (width, height, lcu_size, - num_recon +1 ,HFI_CODEC_ENCODE_AVC ); +return hfi_buffer_comv_enc (width, height, lcu_size, num_recon +1 , codec); } static inline
Thanks,
Dikshita
Quoted 377 lines · Wangao Wang
Hi All,
This patch series introduces several enhancements to the Qualcomm Iris encoder driver, improving support for V4L2 controls and enabling more
video encoding features.All patches have been tested with v4l2-compliance and v4l2-ctl on gen1:SM8250, QCS6490, gen2:QCS8300, QCS8550, QCS9100, X1E-80100.
Commands used for V4l2-ctl validation:
Intra Refresh:
./v4l2-ctl --verbose -d/dev/video1\
--set-fmt-video-out=width=1920,height=1080,pixelformat=NV12 \
--set-selection-output target=crop,width=1920,height=1080 \
--set-fmt-video=pixelformat=H264 --stream-mmap --stream-out-mmap \
--stream-from=input_nv12_1080p.yuv \
--stream-to=output/ir_random.h264 \
--set-ctrl intra_refresh_period_type=0,intra_refresh_period=30./v4l2-ctl --verbose -d
/dev/video1\
--set-fmt-video-out=width=1920,height=1080,pixelformat=NV12 \
--set-selection-output target=crop,width=1920,height=1080 \
--set-fmt-video=pixelformat=H264 --stream-mmap --stream-out-mmap \
--stream-from=input_nv12_1080p.yuv \
--stream-to=output/ir_cyclic.h264 \
--set-ctrl intra_refresh_period_type=1,intra_refresh_period=30./v4l2-ctl --verbose -d
/dev/video1\
--set-fmt-video-out=width=1920,height=1080,pixelformat=NV12 \
--set-selection-output target=crop,width=1920,height=1080 \
--set-fmt-video=pixelformat=HEVC --stream-mmap --stream-out-mmap \
--stream-from=input_nv12_1080p.yuv \
--stream-to=output/ir_random.h265 \
--set-ctrl intra_refresh_period_type=0,intra_refresh_period=30./v4l2-ctl --verbose -d
/dev/video1\
--set-fmt-video-out=width=1920,height=1080,pixelformat=NV12 \
--set-selection-output target=crop,width=1920,height=1080 \
--set-fmt-video=pixelformat=HEVC --stream-mmap --stream-out-mmap \
--stream-from=input_nv12_1080p.yuv \
--stream-to=output/ir_cyclic.h265 \
--set-ctrl intra_refresh_period_type=1,intra_refresh_period=30B frames:
./v4l2-ctl --verbose -d/dev/video1\
--set-fmt-video-out=width=1920,height=1080,pixelformat=NV12 \
--set-selection-output target=crop,width=1920,height=1080 \
--set-fmt-video=pixelformat=H264 --stream-mmap --stream-out-mmap \
--stream-from=input_nv12_1080p.yuv \
--stream-to=output/b_frames.h264 \
--set-ctrl video_b_frames=1./v4l2-ctl --verbose -d
/dev/video1\
--set-fmt-video-out=width=1920,height=1080,pixelformat=NV12 \
--set-selection-output target=crop,width=1920,height=1080 \
--set-fmt-video=pixelformat=HEVC --stream-mmap --stream-out-mmap \
--stream-from=input_nv12_1080p.yuv \
--stream-to=output/b_frames.h265 \
--set-ctrl video_b_frames=1LTR:
./v4l2-ctl --verbose -d/dev/video1\
--set-fmt-video-out=width=1920,height=1080,pixelformat=NV12 \
--set-selection-output target=crop,width=1920,height=1080 \
--set-fmt-video=pixelformat=H264 --stream-mmap --stream-out-mmap \
--stream-from=input_nv12_1080p.yuv \
--stream-to=output/ltr.h264 \
--set-ctrl ltr_count=2,frame_ltr_index=1./v4l2-ctl --verbose -d
/dev/video1\
--set-fmt-video-out=width=1920,height=1080,pixelformat=NV12 \
--set-selection-output target=crop,width=1920,height=1080 \
--set-fmt-video=pixelformat=HEVC --stream-mmap --stream-out-mmap \
--stream-from=input_nv12_1080p.yuv \
--stream-to=output/ltr.h265 \
--set-ctrl ltr_count=2,frame_ltr_index=1Hierarchical Coding:
./v4l2-ctl --verbose -d/dev/video1\
--set-fmt-video-out=width=1920,height=1080,pixelformat=NV12 \
--set-selection-output target=crop,width=1920,height=1080 \
--set-fmt-video=pixelformat=H264 --stream-mmap --stream-out-mmap \
--stream-from=input_nv12_1080p.yuv \
--stream-to=output/hier_coding_layer1_3M.h264 \
--set-ctrl video_bitrate_mode=1,enable_h264_hierarchical_coding=1,h264_hierarchical_coding_type=1,h264_number_of_hc_layers=2,h264_hierarchical_lay_1_bitrate=3000000./v4l2-ctl --verbose -d
/dev/video1\
--set-fmt-video-out=width=1920,height=1080,pixelformat=NV12 \
--set-selection-output target=crop,width=1920,height=1080 \
--set-fmt-video=pixelformat=H264 --stream-mmap --stream-out-mmap \
--stream-from=input_nv12_1080p.yuv \
--stream-to=output/hier_coding_layer0_6M.h264 \
--set-ctrl video_bitrate_mode=1,enable_h264_hierarchical_coding=1,h264_hierarchical_coding_type=1,h264_number_of_hc_layers=2,h264_hierarchical_lay_0_bitrate=6000000./v4l2-ctl --verbose -d
/dev/video1\
--set-fmt-video-out=width=1920,height=1080,pixelformat=NV12 \
--set-selection-output target=crop,width=1920,height=1080 \
--set-fmt-video=pixelformat=HEVC --stream-mmap --stream-out-mmap \
--stream-from=input_nv12_1080p.yuv \
--stream-to=output/hier_coding.h265 \
--set-ctrl hevc_hierarchical_coding_type=1,hevc_hierarchical_coding_layer=5The result of v4l2-compliance on QCS8300:
v4l2-compliance 1.31.0-5379, 64 bits, 64-bittime_tv4l2-compliance SHA:14c988631ad42025-11-11 11:19:35Compliance test for
iris_driverdevice/dev/video1:Driver Info:
Driver name : iris_driver Card type : Iris Encoder Bus info : platform:aa00000.video-codec Driver version : 6.18.0 Capabilities : 0x84204000Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Device Capabilities
Device Caps :0x04204000
Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Detected Stateful EncoderRequired ioctls:
testVIDIOC_QUERYCAP: OK
test invalid ioctls: OKAllow for multiple opens:
test second/dev/video1open: OK
testVIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OKDebug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) testVIDIOC_LOG_STATUS: OK (Not Supported)Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) testVIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) testVIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) testVIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) testVIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported)Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK testVIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 52 Private Controls: 0Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK
testVIDIOC_G_FBUF: OK (Not Supported) testVIDIOC_G_FMT: OK
testVIDIOC_TRY_FMT: OK
testVIDIOC_S_FMT: OK
testVIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK
testVIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK testCREATE_BUFSmaximum buffers: OK testVIDIOC_REMOVE_BUFS: OK
testVIDIOC_EXPBUF: OK
test Requests: OK (Not Supported) test blocking wait: OKTest input 0:
Streaming ioctls:
test read/write: OK (Not Supported) Video Capture Multiplanar: Captured 61 buffers test MMAP (select, REQBUFS): OK
Video Capture Multiplanar: Captured 61 buffers test MMAP (epoll, REQBUFS): OK
Video Capture Multiplanar: Captured 61 buffers test MMAP (select,CREATE_BUFS): OK Video Capture Multiplanar: Captured 61 buffers test MMAP (epoll,CREATE_BUFS): OK
test USERPTR (select): OK (Not Supported) test DMABUF: Cannot test, specify --expbuf-deviceTotal for
iris_driverdevice/dev/video1: 54, Succeeded: 54, Failed: 0, Warnings: 0The result of v4l2-compliance on QCS6490:
v4l2-compliance 1.31.0-5379, 64 bits, 64-bittime_tv4l2-compliance SHA:14c988631ad42025-11-11 11:19:35Compliance test for
iris_driverdevice/dev/video1:Driver Info:
Driver name : iris_driver Card type : Iris Encoder Bus info : platform:aa00000.video-codec Driver version : 6.18.0 Capabilities : 0x84204000Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Device Capabilities
Device Caps :0x04204000
Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Detected Stateful EncoderRequired ioctls:
testVIDIOC_QUERYCAP: OK
test invalid ioctls: OKAllow for multiple opens:
test second/dev/video1open: OK
testVIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OKDebug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) testVIDIOC_LOG_STATUS: OK (Not Supported)Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) testVIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) testVIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) testVIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) testVIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported)Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK testVIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 33 Private Controls: 0Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK
testVIDIOC_G_FBUF: OK (Not Supported) testVIDIOC_G_FMT: OK
testVIDIOC_TRY_FMT: OK
testVIDIOC_S_FMT: OK
testVIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK
testVIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK testCREATE_BUFSmaximum buffers: OK testVIDIOC_REMOVE_BUFS: OK
testVIDIOC_EXPBUF: OK
test Requests: OK (Not Supported) test blocking wait: OKTest input 0:
Streaming ioctls:
test read/write: OK (Not Supported) Video Capture Multiplanar: Captured 61 buffers test MMAP (select, REQBUFS): OK
Video Capture Multiplanar: Captured 61 buffers test MMAP (epoll, REQBUFS): OK
Video Capture Multiplanar: Captured 61 buffers test MMAP (select,CREATE_BUFS): OK Video Capture Multiplanar: Captured 61 buffers test MMAP (epoll,CREATE_BUFS): OK
test USERPTR (select): OK (Not Supported) test DMABUF: Cannot test, specify --expbuf-deviceTotal for
iris_driverdevice/dev/video1: 54, Succeeded: 54, Failed: 0, Warnings: 0Dependencies:
https://lore.kernel.org/all/20251114-iris_encoder_enhancements-v6-0-d6d35a92635d@oss.qualcomm.com/Signed-off-by Wangao Wang <[email protected]>---
Changes in v3:
- Correct the maximum value of IR_PERIOD.(Dikshita)
- Use 'codec' instead of 'standard' in the calculation of COMV size.
(Dikshita)
- Verified these patches on SM8250.(Dikshita)
- Link to v2: https://lore.kernel.org/r/20251219-batch2_iris_encoder_enhancements-v2-0-371f7fe24801@oss.qualcomm.comChanges in v2:
- Add comment forir_periodcalculation.(bod,Dikshita)
- Correct the maximum value of IR_PERIOD.(Dikshita)
- Add a patch to optimize the calculation of hdr size.(bod)
- Correct thenum_reconcalculation.(Dikshita)
- Add a patch to simplify the calculation of COMV size.(Dikshita)
- Catch the result code of session_set_property.(bod)
- Add comment for shift operation in hfi_buffer_get_recon_count.(bod)
- Add support for layer bitrate setting.(Dikshita)
- Link to v1: https://lore.kernel.org/r/20251127-batch2_iris_encoder_enhancements-v1-0-5ea78e2de2ae@oss.qualcomm.com---
Wangao Wang (6):
media: qcom: iris: Add intra refresh support for gen1 encoder
media: qcom: iris: Add Long-Term Reference support for encoder
media: qcom: iris: Add B frames support for encoder
media: qcom: iris: Add hierarchical coding support for encoder
media: qcom: iris: Optimizeiris_hfi_gen1_packet_session_set_property
media: qcom: iris: Simplify COMV size calculationdrivers/media/platform/qcom/iris/iris_ctrls.c | 476 ++++++++++++++++++++- drivers/media/platform/qcom/iris/iris_ctrls.h | 16 +- .../platform/qcom/iris/iris_hfi_gen1_command.c | 98 ++++- .../platform/qcom/iris/iris_hfi_gen1_defines.h | 49 +++ .../platform/qcom/iris/iris_hfi_gen2_defines.h | 18 + drivers/media/platform/qcom/iris/iris_instance.h | 4 + .../platform/qcom/iris/iris_platform_common.h | 31 ++ .../media/platform/qcom/iris/iris_platform_gen1.c | 161 ++++++- .../media/platform/qcom/iris/iris_platform_gen2.c | 224 +++++++++- drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 65 ++- 10 files changed, 1101 insertions(+), 41 deletions(-)---
base-commit:cc3aa43b44bdb43dfbac0fcb51c56594a11338a8
change-id: 20251125-batch2_iris_encoder_enhancements-3e5c42b93a25
prerequisite-message-id: 20251114-iris_encoder_enhancements-v6-0-d6d35a92635d@oss.qualcomm.com
prerequisite-patch-id:2b77a7b9bfb436751da0bf69488dbff567a38479
prerequisite-patch-id:190f29ef211b046f0d031e3a4b52f2c3bccd0cd9
prerequisite-patch-id:92ff1a1958eeacd53d70e155bb8214f9987ddf85
prerequisite-patch-id:0dae6aaa8db7df0729436cf9268d29ea363f0eed
prerequisite-patch-id:6382693f5a5a7438660f2e592b30163551d92791
prerequisite-patch-id:9aa16211c36540a61d219eb25f09b7625b9bdbde
Quoted 1 line
Best regards,
With the following reverted:456c99da41cb ("media: iris: Add support for QC08C format for encoder") causing this: https://pasteboard.co/t8HmeNaIFMDR.png
Nevertheless, the changes are functional and output are visually coherent with the parameters, so:
Thanks,
Neil
Quoted 10 lines · Neil Armstrong
With the following reverted:
456c99da41cb("media: iris: Add support for QC08C format for encoder") causing this: https://pasteboard.co/t8HmeNaIFMDR.pngNevertheless, the changes are functional and output are visually coherent with the parameters, so:
Thanks,
Neil
Thank you for helping with testing on SM8650. Did you revert commit 456c99da41cb because v4l2-ctl couldn’t run properly? There’s a change in v4l2-ctl that needs to be applied, please check.
https://lore.kernel.org/linux-media/[email protected]/T/#u
Signature · 2 lines
Best Regards, Wangao
Quoted 12 lines · Wangao Wang
Quoted 9 lines · Neil Armstrong
With the following reverted:
456c99da41cb("media: iris: Add support for QC08C format for encoder") causing this: https://pasteboard.co/t8HmeNaIFMDR.pngNevertheless, the changes are functional and output are visually coherent with the parameters, so:
Quoted 8 lines
Quoted 2 lines
Thanks,
NeilThank you for helping with testing on SM8650. Did you revert commit
456c99da41cbbecause v4l2-ctl couldn’t run properly? There’s a change in v4l2-ctl that needs to be applied, please check.
If the kernel changes break userspace, then it is a regression on the kernel side, which needs to be fixed there. You can't demand all users to update userspace apps when performing kernel changes.
Signature · 2 lines
With best wishes Dmitry
Quoted 11 lines · Wangao Wang
Quoted 8 lines · Neil Armstrong
With the following reverted:
456c99da41cb("media: iris: Add support for QC08C format for encoder") causing this: https://pasteboard.co/t8HmeNaIFMDR.pngNevertheless, the changes are functional and output are visually coherent with the parameters, so:
Quoted 6 lines
Quoted 2 lines
Thanks,
NeilThank you for helping with testing on SM8650. Did you revert commit
456c99da41cbbecause v4l2-ctl couldn’t run properly? There’s a change in v4l2-ctl that needs to be applied, please check.
Yes I applied the v4l2-ctl fix and it works fine. Please retain my tested-by.
Neil