media: rockchip: rkvdec: add support for the VDPU346 variant
Subsystems (4)
v2 cover media: rockchip: rkvdec: add support for the VDPU346 variant
This series depends upon Detlev Casanova's current v7 series for VDPU381 and VDPU383 support [0]. It adds support for the VDPU346 IP block used for H264, HEVC and (in active work) VP9 on the RK356X boards. VDPU346 appears to be a close relation to VDPU381 used with RK3588, except with a single core, output limited to 4K, and minor feature differences, e.g. HEVC level 5.1 on VDPU346 vs 6.1 on VDPU381. To handle differences we declare a new compatible.
The device-tree changes are derived from prior-art in Detlev's patches for RK3576 and RK3588, the vendor kernel [1], and some earlier patch attempts from Piotr Oniszczuk and Diederik de Haas.
The register differences between VDPU346 and VDPU381 appear to be minimal so we currently and intentionally reuse the VDPU381 h264, hevc, and regs files to avoid duplicating code. There is some evidence of buffer size changes in testing which still need to be explored. If further register analysis needs separate files for VDPU346 the refactoring will be beyond my current n00b coding skills and I will either need some coaching or would be happy to pass the series over to a more experienced developer.
NB: Testing with the v1 series showed lower mbps bitrate performance. This appears to be resolved though it's unclear to me whether this results from kernel changes or the ongoing reworking of ffmpeg v4l2_request support [2]. However with my current Linux 6.19-rc2 test branch [3] I'm now able to play Jellyfish H264 and HEVC test media over 100mbps.
Changes since v1:
- Drop DO-NOT-MERGE patch as changes included in Detvlev v7 series
- Adapt to variant/capability changes in Detlev's v7 series
- Add bindings ack from Rob
- Add rkvdec ack from Nicolas
- Rebase against Linux 6.19-rc2
[0] https://patchwork.kernel.org/project/linux-rockchip/list/?series=1034794
[1] https://github.com/rockchip-linux/kernel/blob/develop-6.6/arch/arm64/boot/dts/rockchip/rk356x.dtsi#L1539
[2] https://code.ffmpeg.org/Kwiboo/FFmpeg/commits/branch/v4l2request-v3
[3] https://github.com/chewitt/linux/commits/rockchip-6.19.y
Christian Hewitt (3):
media: dt-bindings: rockchip: Add RK3568 Video Decoder bindings
media: rkvdec: Add support for the VDPU346 variant
arm64: dts: rockchip: Add the vdpu346 Video Decoders on RK356X
.../bindings/media/rockchip,vdec.yaml | 2 + arch/arm64/boot/dts/rockchip/rk356x-base.dtsi | 49 +++++++++ .../media/platform/rockchip/rkvdec/rkvdec.c | 103 ++++++++++++++++++ 3 files changed, 154 insertions(+)
Signature · 1 line
2.34.1
PATCH v2 1/3 media: dt-bindings: rockchip: Add RK3568 Video Decoder bindings
The video decoder in RK356X (vdpu346) is described in the same way as the one in RK3588 (vdpu381). A new compatible is added as the decoder capabilities are a subset of the vdpu381 capabilities.
--- Documentation/devicetree/bindings/media/rockchip,vdec.yaml | 2 ++ 1 file changed, 2 insertions(+)
Documentation/devicetree/bindings/media/rockchip,vdec.yaml +2 -0
@@ -18,6 +18,7 @@ properties: oneOf: - const: rockchip,rk3288-vdec - const: rockchip,rk3399-vdec + - const: rockchip,rk3568-vdec - const: rockchip,rk3576-vdec - const: rockchip,rk3588-vdec - items:
@@ -107,6 +108,7 @@ allOf: compatible: contains: enum: + - rockchip,rk3568-vdec - rockchip,rk3576-vdec - rockchip,rk3588-vdec then:
Signature · 1 line
2.34.1
PATCH v2 2/3 media: rkvdec: Add support for the VDPU346 variant
In reply to Christian HewittVDPU346 is similar to VDPU381 but with a single core and limited to 4K60 media. It is also limited to H264 L5.1 and omits AV1 and AVS2 capabilities. VDPU346 is used with RK3566 and RK3568.
--- .../media/platform/rockchip/rkvdec/rkvdec.c | 103 ++++++++++++++++++ 1 file changed, 103 insertions(+)
drivers/media/platform/rockchip/rkvdec/rkvdec.c +103 -0
@@ -236,6 +236,62 @@ static const struct rkvdec_ctrls rkvdec_hevc_ctrls = { .num_ctrls = ARRAY_SIZE(rkvdec_hevc_ctrl_descs), }; +static const struct rkvdec_ctrl_desc vdpu346_hevc_ctrl_descs[] = { + { + .cfg.id =V4L2_CID_STATELESS_HEVC_DECODE_PARAMS , + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_SPS , + .cfg.ops = &rkvdec_ctrl_ops, + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_PPS , + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_SCALING_MATRIX , + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_DECODE_MODE , + .cfg.min =V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED , + .cfg.max =V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED , + .cfg.def =V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED , + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_START_CODE , + .cfg.min =V4L2_STATELESS_HEVC_START_CODE_ANNEX_B , + .cfg.def =V4L2_STATELESS_HEVC_START_CODE_ANNEX_B , + .cfg.max =V4L2_STATELESS_HEVC_START_CODE_ANNEX_B , + }, + { + .cfg.id =V4L2_CID_MPEG_VIDEO_HEVC_PROFILE , + .cfg.min =V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN , + .cfg.max =V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10 , + .cfg.menu_skip_mask = +BIT (V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE ), + .cfg.def =V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN , + }, + { + .cfg.id =V4L2_CID_MPEG_VIDEO_HEVC_LEVEL , + .cfg.min =V4L2_MPEG_VIDEO_HEVC_LEVEL_1 , + .cfg.max =V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1 , + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_EXT_SPS_ST_RPS , + .cfg.ops = &rkvdec_ctrl_ops, + .cfg.dims = {65 }, + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_EXT_SPS_LT_RPS , + .cfg.ops = &rkvdec_ctrl_ops, + .cfg.dims = {65 }, + }, +}; + +static const struct rkvdec_ctrls vdpu346_hevc_ctrls = { + .ctrls = vdpu346_hevc_ctrl_descs, + .num_ctrls =ARRAY_SIZE (vdpu346_hevc_ctrl_descs), +}; +static const struct rkvdec_ctrl_desc vdpu38x_hevc_ctrl_descs[] = { { .cfg.id =V4L2_CID_STATELESS_HEVC_DECODE_PARAMS ,
@@ -463,6 +519,41 @@ static const struct rkvdec_coded_fmt_desc rk3288_coded_fmts[] = { } }; +static const struct rkvdec_coded_fmt_desc vdpu346_coded_fmts[] = { + { + .fourcc =V4L2_PIX_FMT_HEVC_SLICE , + .frmsize = { + .min_width =64 , + .max_width =65472 , + .step_width =64 , + .min_height =64 , + .max_height =65472 , + .step_height =16 , + }, + .ctrls = &vdpu346_hevc_ctrls, + .ops = &rkvdec_vdpu381_hevc_fmt_ops, + .num_decoded_fmts =ARRAY_SIZE (rkvdec_hevc_decoded_fmts), + .decoded_fmts = rkvdec_hevc_decoded_fmts, + .subsystem_flags =VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF , + }, + { + .fourcc =V4L2_PIX_FMT_H264_SLICE , + .frmsize = { + .min_width =64 , + .max_width =65520 , + .step_width =64 , + .min_height =64 , + .max_height =65520 , + .step_height =16 , + }, + .ctrls = &rkvdec_h264_ctrls, + .ops = &rkvdec_vdpu381_h264_fmt_ops, + .num_decoded_fmts =ARRAY_SIZE (rkvdec_h264_decoded_fmts), + .decoded_fmts = rkvdec_h264_decoded_fmts, + .subsystem_flags =VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF , + }, +}; +static const struct rkvdec_coded_fmt_desc vdpu381_coded_fmts[] = { { .fourcc =V4L2_PIX_FMT_HEVC_SLICE ,
@@ -1643,6 +1734,14 @@ static const struct rkvdec_variant_ops vdpu381_variant_ops = { .flatten_matrices = transpose_and_flatten_matrices, }; +static const struct rkvdec_variant vdpu346_variant = { + .coded_fmts = vdpu346_coded_fmts, + .num_coded_fmts =ARRAY_SIZE (vdpu346_coded_fmts), + .rcb_sizes = vdpu381_rcb_sizes, + .num_rcb_sizes =ARRAY_SIZE (vdpu381_rcb_sizes), + .ops = &vdpu381_variant_ops, +}; +static const struct rkvdec_variant vdpu381_variant = { .coded_fmts = vdpu381_coded_fmts, .num_coded_fmts =ARRAY_SIZE (vdpu381_coded_fmts),
@@ -1691,6 +1790,10 @@ static const struct of_device_id of_rkvdec_match[] = { .compatible = "rockchip,rk3399-vdec", .data = &rk3399_rkvdec_variant, }, + { + .compatible = "rockchip,rk3568-vdec", + .data = &vdpu346_variant, + }, { .compatible = "rockchip,rk3588-vdec", .data = &vdpu381_variant,
Signature · 1 line
2.34.1
Re: [PATCH v2 2/3] media: rkvdec: Add support for the VDPU346 variant
Hi Christian,
Quoted 78 lines · Christian Hewitt
VDPU346 is similar to VDPU381 but with a single core and limited to 4K60 media. It is also limited to H264 L5.1 and omits AV1 and AVS2 capabilities. VDPU346 is used with RK3566 and RK3568.
Signed-off-by Christian Hewitt <[email protected]>Reviewed-by Nicolas Dufresne <[email protected]>--- .../media/platform/rockchip/rkvdec/rkvdec.c | 103 ++++++++++++++++++ 1 file changed, 103 insertions(+)drivers/media/platform/rockchip/rkvdec/rkvdec.c +56
-0@@ -236,6 +236,62 @@ static const struct rkvdec_ctrls rkvdec_hevc_ctrls = { .num_ctrls = ARRAY_SIZE(rkvdec_hevc_ctrl_descs), }; +static const struct rkvdec_ctrl_desc vdpu346_hevc_ctrl_descs[] = { + { + .cfg.id =V4L2_CID_STATELESS_HEVC_DECODE_PARAMS , + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_SPS , + .cfg.ops = &rkvdec_ctrl_ops, + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_PPS , + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_SCALING_MATRIX , + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_DECODE_MODE , + .cfg.min =V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED , + .cfg.max =V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED , + .cfg.def =V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED , + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_START_CODE , + .cfg.min =V4L2_STATELESS_HEVC_START_CODE_ANNEX_B , + .cfg.def =V4L2_STATELESS_HEVC_START_CODE_ANNEX_B , + .cfg.max =V4L2_STATELESS_HEVC_START_CODE_ANNEX_B , + }, + { + .cfg.id =V4L2_CID_MPEG_VIDEO_HEVC_PROFILE , + .cfg.min =V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN , + .cfg.max =V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10 , + .cfg.menu_skip_mask = +BIT (V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE ), + .cfg.def =V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN , + }, + { + .cfg.id =V4L2_CID_MPEG_VIDEO_HEVC_LEVEL , + .cfg.min =V4L2_MPEG_VIDEO_HEVC_LEVEL_1 , + .cfg.max =V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1 , + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_EXT_SPS_ST_RPS , + .cfg.ops = &rkvdec_ctrl_ops, + .cfg.dims = {65 }, + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_EXT_SPS_LT_RPS , + .cfg.ops = &rkvdec_ctrl_ops, + .cfg.dims = {65 }, + }, +}; + +static const struct rkvdec_ctrls vdpu346_hevc_ctrls = { + .ctrls = vdpu346_hevc_ctrl_descs, + .num_ctrls =ARRAY_SIZE (vdpu346_hevc_ctrl_descs), +}; +static const struct rkvdec_ctrl_desc vdpu38x_hevc_ctrl_descs[] = { { .cfg.id =V4L2_CID_STATELESS_HEVC_DECODE_PARAMS ,
drivers/media/platform/rockchip/rkvdec/rkvdec.c static const struct rkvdec_coded_fmt_desc rk3288_coded_fmts[] = { · line 527
Quoted 9 lines
drivers/media/platform/rockchip/rkvdec/rkvdec.c +6
-0@@ -463,6 +519,41 @@ static const struct rkvdec_coded_fmt_desc rk3288_coded_fmts[] = { } }; +static const struct rkvdec_coded_fmt_desc vdpu346_coded_fmts[] = { + { + .fourcc =V4L2_PIX_FMT_HEVC_SLICE , + .frmsize = { + .min_width =64 , + .max_width =65472 ,
This should be 4096 according to page 469 of RK3568 TRM Part 2 ...
drivers/media/platform/rockchip/rkvdec/rkvdec.c static const struct rkvdec_coded_fmt_desc rk3288_coded_fmts[] = { · line 530
Quoted 3 lines
drivers/media/platform/rockchip/rkvdec/rkvdec.c +3
-0+ .step_width = 64, + .min_height = 64, + .max_height = 65472,
... and this 2304.
drivers/media/platform/rockchip/rkvdec/rkvdec.c static const struct rkvdec_coded_fmt_desc rk3288_coded_fmts[] = { · line 543
Quoted 13 lines
drivers/media/platform/rockchip/rkvdec/rkvdec.c +13
-0+ .step_height = 16, + }, + .ctrls = &vdpu346_hevc_ctrls, + .ops = &rkvdec_vdpu381_hevc_fmt_ops, + .num_decoded_fmts = ARRAY_SIZE(rkvdec_hevc_decoded_fmts), + .decoded_fmts = rkvdec_hevc_decoded_fmts, + .subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF, + }, + { + .fourcc = V4L2_PIX_FMT_H264_SLICE, + .frmsize = { + .min_width = 64, + .max_width = 65520,
This too should be 4096 according to page 469 of RK3568 TRM Part 2 ...
drivers/media/platform/rockchip/rkvdec/rkvdec.c static const struct rkvdec_coded_fmt_desc rk3288_coded_fmts[] = { · line 546
Quoted 3 lines
drivers/media/platform/rockchip/rkvdec/rkvdec.c +3
-0+ .step_width = 64, + .min_height = 64, + .max_height = 65520,
... and this 2304.
I guess this also explains the 'green images' Nicolas noticed.
drivers/media/platform/rockchip/rkvdec/rkvdec.c static const struct rkvdec_coded_fmt_desc rk3288_coded_fmts[] = { · line 554
Quoted 8 lines
drivers/media/platform/rockchip/rkvdec/rkvdec.c +8
-0+ .step_height = 16, + }, + .ctrls = &rkvdec_h264_ctrls, + .ops = &rkvdec_vdpu381_h264_fmt_ops, + .num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_decoded_fmts), + .decoded_fmts = rkvdec_h264_decoded_fmts, + .subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF, + },
I see you've reversed the order of the blocks so that HEVC now comes before the H264 block. While that makes it consistent with what Detlev has in their v7 and with the existing code in the driver ... I actually prefer having H264 before HEVC as the alphabetical sorting order is H264 before HEVC.
In the existing code the VP9 'stuff' is listed below H264 and HEVC.
But then Detlev should do that too in their patch set ... and 'ideally' the order of the existing code be updated to be alphabetically too.
OTOH, a consistent order works for me too.
Cheers,
Diederik
Quoted 31 lines
drivers/media/platform/rockchip/rkvdec/rkvdec.c +14
-0+}; +static const struct rkvdec_coded_fmt_desc vdpu381_coded_fmts[] = { { .fourcc =V4L2_PIX_FMT_HEVC_SLICE ,@@ -1643,6 +1734,14 @@ static const struct rkvdec_variant_ops vdpu381_variant_ops = { .flatten_matrices = transpose_and_flatten_matrices, }; +static const struct rkvdec_variant vdpu346_variant = { + .coded_fmts = vdpu346_coded_fmts, + .num_coded_fmts =ARRAY_SIZE (vdpu346_coded_fmts), + .rcb_sizes = vdpu381_rcb_sizes, + .num_rcb_sizes =ARRAY_SIZE (vdpu381_rcb_sizes), + .ops = &vdpu381_variant_ops, +}; +static const struct rkvdec_variant vdpu381_variant = { .coded_fmts = vdpu381_coded_fmts, .num_coded_fmts =ARRAY_SIZE (vdpu381_coded_fmts),@@ -1691,6 +1790,10 @@ static const struct of_device_id of_rkvdec_match[] = { .compatible = "rockchip,rk3399-vdec", .data = &rk3399_rkvdec_variant, }, + { + .compatible = "rockchip,rk3568-vdec", + .data = &vdpu346_variant, + }, { .compatible = "rockchip,rk3588-vdec", .data = &vdpu381_variant,
Re: [PATCH v2 2/3] media: rkvdec: Add support for the VDPU346 variant
Quoted 117 lines · Diederik de Haas
Hi Christian,
Quoted 78 lines · Christian Hewitt
VDPU346 is similar to VDPU381 but with a single core and limited to 4K60 media. It is also limited to H264 L5.1 and omits AV1 and AVS2 capabilities. VDPU346 is used with RK3566 and RK3568.
Signed-off-by Christian Hewitt <[email protected]>Reviewed-by Nicolas Dufresne <[email protected]>---
.../media/platform/rockchip/rkvdec/rkvdec.c| 103 ++++++++++++++++++1 file changed, 103 insertions(+)drivers/media/platform/rockchip/rkvdec/rkvdec.c +56
-0@@ -236,6 +236,62 @@ static const struct rkvdec_ctrls rkvdec_hevc_ctrls = { .num_ctrls = ARRAY_SIZE(rkvdec_hevc_ctrl_descs), }; +static const struct rkvdec_ctrl_desc vdpu346_hevc_ctrl_descs[] = { + { + .cfg.id =V4L2_CID_STATELESS_HEVC_DECODE_PARAMS , + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_SPS , + .cfg.ops = &rkvdec_ctrl_ops, + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_PPS , + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_SCALING_MATRIX , + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_DECODE_MODE , + .cfg.min =V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED , + .cfg.max =V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED , + .cfg.def =V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED , + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_START_CODE , + .cfg.min =V4L2_STATELESS_HEVC_START_CODE_ANNEX_B , + .cfg.def =V4L2_STATELESS_HEVC_START_CODE_ANNEX_B , + .cfg.max =V4L2_STATELESS_HEVC_START_CODE_ANNEX_B , + }, + { + .cfg.id =V4L2_CID_MPEG_VIDEO_HEVC_PROFILE , + .cfg.min =V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN , + .cfg.max =V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10 , + .cfg.menu_skip_mask = +BIT (V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE ), + .cfg.def =V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN , + }, + { + .cfg.id =V4L2_CID_MPEG_VIDEO_HEVC_LEVEL , + .cfg.min =V4L2_MPEG_VIDEO_HEVC_LEVEL_1 , + .cfg.max =V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1 , + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_EXT_SPS_ST_RPS , + .cfg.ops = &rkvdec_ctrl_ops, + .cfg.dims = {65 }, + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_EXT_SPS_LT_RPS , + .cfg.ops = &rkvdec_ctrl_ops, + .cfg.dims = {65 }, + }, +}; + +static const struct rkvdec_ctrls vdpu346_hevc_ctrls = { + .ctrls = vdpu346_hevc_ctrl_descs, + .num_ctrls =ARRAY_SIZE (vdpu346_hevc_ctrl_descs), +}; +static const struct rkvdec_ctrl_desc vdpu38x_hevc_ctrl_descs[] = { { .cfg.id =V4L2_CID_STATELESS_HEVC_DECODE_PARAMS ,drivers/media/platform/rockchip/rkvdec/rkvdec.c static const struct rkvdec_coded_fmt_desc rk3288_coded_fmts[] = { · line 527
Quoted 9 lines
drivers/media/platform/rockchip/rkvdec/rkvdec.c +6
-0@@ -463,6 +519,41 @@ static const struct rkvdec_coded_fmt_desc rk3288_coded_fmts[] = { } }; +static const struct rkvdec_coded_fmt_desc vdpu346_coded_fmts[] = { + { + .fourcc =V4L2_PIX_FMT_HEVC_SLICE , + .frmsize = { + .min_width =64 , + .max_width =65472 ,This should be 4096 according to page 469 of RK3568 TRM Part 2 ...
drivers/media/platform/rockchip/rkvdec/rkvdec.c static const struct rkvdec_coded_fmt_desc rk3288_coded_fmts[] = { · line 530
Quoted 3 lines
drivers/media/platform/rockchip/rkvdec/rkvdec.c +3
-0+ .step_width = 64, + .min_height = 64, + .max_height = 65472,... and this 2304.
drivers/media/platform/rockchip/rkvdec/rkvdec.c static const struct rkvdec_coded_fmt_desc rk3288_coded_fmts[] = { · line 543
Quoted 13 lines
drivers/media/platform/rockchip/rkvdec/rkvdec.c +13
-0+ .step_height = 16, + }, + .ctrls = &vdpu346_hevc_ctrls, + .ops = &rkvdec_vdpu381_hevc_fmt_ops, + .num_decoded_fmts = ARRAY_SIZE(rkvdec_hevc_decoded_fmts), + .decoded_fmts = rkvdec_hevc_decoded_fmts, + .subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF, + }, + { + .fourcc = V4L2_PIX_FMT_H264_SLICE, + .frmsize = { + .min_width = 64, + .max_width = 65520,This too should be 4096 according to page 469 of RK3568 TRM Part 2 ...
Quoted 7 lines
drivers/media/platform/rockchip/rkvdec/rkvdec.c static const struct rkvdec_coded_fmt_desc rk3288_coded_fmts[] = { · line 546
Quoted 3 lines
drivers/media/platform/rockchip/rkvdec/rkvdec.c +3
-0+ .step_width = 64, + .min_height = 64, + .max_height = 65520,... and this 2304.
I guess this also explains the 'green images' Nicolas noticed.
Quite probably. I’ve picked the above changes into my working tree (for those following it) and will send a v3 series in response to the next revision of Detlev’s patches.
Quoted 16 lines
+ .step_height = 16,
drivers/media/platform/rockchip/rkvdec/rkvdec.c static const struct rkvdec_coded_fmt_desc rk3288_coded_fmts[] = { · line 554
Quoted 7 lines
drivers/media/platform/rockchip/rkvdec/rkvdec.c +7
-0+ }, + .ctrls = &rkvdec_h264_ctrls, + .ops = &rkvdec_vdpu381_h264_fmt_ops, + .num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_decoded_fmts), + .decoded_fmts = rkvdec_h264_decoded_fmts, + .subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF, + },I see you've reversed the order of the blocks so that HEVC now comes before the H264 block. While that makes it consistent with what Detlev has in their v7 and with the existing code in the driver ... I actually prefer having H264 before HEVC as the alphabetical sorting order is H264 before HEVC.
In the existing code the VP9 'stuff' is listed below H264 and HEVC.
Quoted 4 lines
But then Detlev should do that too in their patch set ... and 'ideally' the order of the existing code be updated to be alphabetically too.
OTOH, a consistent order works for me too.
I believe the reorder was requested by Nic (offline from the list) so there’s probably a reason behind it. I’ll keep things aligned to the order in Detlev’s series (whatever that is).
Christian
Quoted 34 lines
Cheers,
DiederikQuoted 31 lines
drivers/media/platform/rockchip/rkvdec/rkvdec.c +14
-0+}; +static const struct rkvdec_coded_fmt_desc vdpu381_coded_fmts[] = { { .fourcc =V4L2_PIX_FMT_HEVC_SLICE ,@@ -1643,6 +1734,14 @@ static const struct rkvdec_variant_ops vdpu381_variant_ops = { .flatten_matrices = transpose_and_flatten_matrices, }; +static const struct rkvdec_variant vdpu346_variant = { + .coded_fmts = vdpu346_coded_fmts, + .num_coded_fmts =ARRAY_SIZE (vdpu346_coded_fmts), + .rcb_sizes = vdpu381_rcb_sizes, + .num_rcb_sizes =ARRAY_SIZE (vdpu381_rcb_sizes), + .ops = &vdpu381_variant_ops, +}; +static const struct rkvdec_variant vdpu381_variant = { .coded_fmts = vdpu381_coded_fmts, .num_coded_fmts =ARRAY_SIZE (vdpu381_coded_fmts),@@ -1691,6 +1790,10 @@ static const struct of_device_id of_rkvdec_match[] = { .compatible = "rockchip,rk3399-vdec", .data = &rk3399_rkvdec_variant, }, + { + .compatible = "rockchip,rk3568-vdec", + .data = &vdpu346_variant, + }, { .compatible = "rockchip,rk3588-vdec", .data = &vdpu381_variant,
Re: [PATCH v2 2/3] media: rkvdec: Add support for the VDPU346 variant
Le vendredi 26 décembre 2025 à 16:25 +0400, Christian Hewitt a écrit :
Quoted 145 lines
Quoted 117 lines · Diederik de Haas
Hi Christian,
Quoted 78 lines · Christian Hewitt
VDPU346 is similar to VDPU381 but with a single core and limited to 4K60 media. It is also limited to H264 L5.1 and omits AV1 and AVS2 capabilities. VDPU346 is used with RK3566 and RK3568.
Signed-off-by Christian Hewitt <[email protected]>Reviewed-by Nicolas Dufresne <[email protected]>---
.../media/platform/rockchip/rkvdec/rkvdec.c | 103 ++++++++++++++++++1 file changed, 103 insertions(+)drivers/media/platform/rockchip/rkvdec/rkvdec.c +56
-0@@ -236,6 +236,62 @@ static const struct rkvdec_ctrls rkvdec_hevc_ctrls = { .num_ctrls = ARRAY_SIZE(rkvdec_hevc_ctrl_descs), }; +static const struct rkvdec_ctrl_desc vdpu346_hevc_ctrl_descs[] = { + { + .cfg.id =V4L2_CID_STATELESS_HEVC_DECODE_PARAMS , + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_SPS , + .cfg.ops = &rkvdec_ctrl_ops, + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_PPS , + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_SCALING_MATRIX , + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_DECODE_MODE , + .cfg.min =V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED , + .cfg.max =V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED , + .cfg.def =V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED , + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_START_CODE , + .cfg.min =V4L2_STATELESS_HEVC_START_CODE_ANNEX_B , + .cfg.def =V4L2_STATELESS_HEVC_START_CODE_ANNEX_B , + .cfg.max =V4L2_STATELESS_HEVC_START_CODE_ANNEX_B , + }, + { + .cfg.id =V4L2_CID_MPEG_VIDEO_HEVC_PROFILE , + .cfg.min =V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN , + .cfg.max =V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10 , + .cfg.menu_skip_mask = +BIT (V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE ), + .cfg.def =V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN , + }, + { + .cfg.id =V4L2_CID_MPEG_VIDEO_HEVC_LEVEL , + .cfg.min =V4L2_MPEG_VIDEO_HEVC_LEVEL_1 , + .cfg.max =V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1 , + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_EXT_SPS_ST_RPS , + .cfg.ops = &rkvdec_ctrl_ops, + .cfg.dims = {65 }, + }, + { + .cfg.id =V4L2_CID_STATELESS_HEVC_EXT_SPS_LT_RPS , + .cfg.ops = &rkvdec_ctrl_ops, + .cfg.dims = {65 }, + }, +}; + +static const struct rkvdec_ctrls vdpu346_hevc_ctrls = { + .ctrls = vdpu346_hevc_ctrl_descs, + .num_ctrls =ARRAY_SIZE (vdpu346_hevc_ctrl_descs), +}; +static const struct rkvdec_ctrl_desc vdpu38x_hevc_ctrl_descs[] = { { .cfg.id =V4L2_CID_STATELESS_HEVC_DECODE_PARAMS ,drivers/media/platform/rockchip/rkvdec/rkvdec.c static const struct rkvdec_coded_fmt_desc rk3288_coded_fmts[] = { · line 527
Quoted 9 lines
drivers/media/platform/rockchip/rkvdec/rkvdec.c +6
-0@@ -463,6 +519,41 @@ static const struct rkvdec_coded_fmt_desc rk3288_coded_fmts[] = { } }; +static const struct rkvdec_coded_fmt_desc vdpu346_coded_fmts[] = { + { + .fourcc =V4L2_PIX_FMT_HEVC_SLICE , + .frmsize = { + .min_width =64 , + .max_width =65472 ,This should be 4096 according to page 469 of RK3568 TRM Part 2 ...
drivers/media/platform/rockchip/rkvdec/rkvdec.c static const struct rkvdec_coded_fmt_desc rk3288_coded_fmts[] = { · line 530
Quoted 3 lines
drivers/media/platform/rockchip/rkvdec/rkvdec.c +3
-0+ .step_width = 64, + .min_height = 64, + .max_height = 65472,... and this 2304.
drivers/media/platform/rockchip/rkvdec/rkvdec.c static const struct rkvdec_coded_fmt_desc rk3288_coded_fmts[] = { · line 543
Quoted 13 lines
drivers/media/platform/rockchip/rkvdec/rkvdec.c +13
-0+ .step_height = 16, + }, + .ctrls = &vdpu346_hevc_ctrls, + .ops = &rkvdec_vdpu381_hevc_fmt_ops, + .num_decoded_fmts = ARRAY_SIZE(rkvdec_hevc_decoded_fmts), + .decoded_fmts = rkvdec_hevc_decoded_fmts, + .subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF, + }, + { + .fourcc = V4L2_PIX_FMT_H264_SLICE, + .frmsize = { + .min_width = 64, + .max_width = 65520,This too should be 4096 according to page 469 of RK3568 TRM Part 2 ...
Quoted 7 lines
drivers/media/platform/rockchip/rkvdec/rkvdec.c static const struct rkvdec_coded_fmt_desc rk3288_coded_fmts[] = { · line 546
Quoted 3 lines
drivers/media/platform/rockchip/rkvdec/rkvdec.c +3
-0+ .step_width = 64, + .min_height = 64, + .max_height = 65520,... and this 2304.
I guess this also explains the 'green images' Nicolas noticed.
Quite probably. I’ve picked the above changes into my working tree (for those following it) and will send a v3 series in response to the next revision of Detlev’s patches.
Quoted 16 lines
+ .step_height = 16,
drivers/media/platform/rockchip/rkvdec/rkvdec.c static const struct rkvdec_coded_fmt_desc rk3288_coded_fmts[] = { · line 554
Quoted 7 lines
drivers/media/platform/rockchip/rkvdec/rkvdec.c +7
-0+ }, + .ctrls = &rkvdec_h264_ctrls, + .ops = &rkvdec_vdpu381_h264_fmt_ops, + .num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_decoded_fmts), + .decoded_fmts = rkvdec_h264_decoded_fmts, + .subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF, + },I see you've reversed the order of the blocks so that HEVC now comes before the H264 block. While that makes it consistent with what Detlev has in their v7 and with the existing code in the driver ... I actually prefer having H264 before HEVC as the alphabetical sorting order is H264 before HEVC.
In the existing code the VP9 'stuff' is listed below H264 and HEVC.
Quoted 8 lines
Quoted 4 lines
But then Detlev should do that too in their patch set ... and 'ideally' the order of the existing code be updated to be alphabetically too.
OTOH, a consistent order works for me too.
I believe the reorder was requested by Nic (offline from the list) so there’s probably a reason behind it. I’ll keep things aligned to the order in Detlev’s series (whatever that is).
I'm taking note of this feedback, and instead of multiplying the revisions, I will probably fix it on top once we have the series ready.
regards,
Nicolas
Quoted 40 lines
Christian
Quoted 34 lines
Cheers,
DiederikQuoted 31 lines
drivers/media/platform/rockchip/rkvdec/rkvdec.c +14
-0+}; +static const struct rkvdec_coded_fmt_desc vdpu381_coded_fmts[] = { { .fourcc =V4L2_PIX_FMT_HEVC_SLICE ,@@ -1643,6 +1734,14 @@ static const struct rkvdec_variant_ops vdpu381_variant_ops = { .flatten_matrices = transpose_and_flatten_matrices, }; +static const struct rkvdec_variant vdpu346_variant = { + .coded_fmts = vdpu346_coded_fmts, + .num_coded_fmts =ARRAY_SIZE (vdpu346_coded_fmts), + .rcb_sizes = vdpu381_rcb_sizes, + .num_rcb_sizes =ARRAY_SIZE (vdpu381_rcb_sizes), + .ops = &vdpu381_variant_ops, +}; +static const struct rkvdec_variant vdpu381_variant = { .coded_fmts = vdpu381_coded_fmts, .num_coded_fmts =ARRAY_SIZE (vdpu381_coded_fmts),@@ -1691,6 +1790,10 @@ static const struct of_device_id of_rkvdec_match[] = { .compatible = "rockchip,rk3399-vdec", .data = &rk3399_rkvdec_variant, }, + { + .compatible = "rockchip,rk3568-vdec", + .data = &vdpu346_variant, + }, { .compatible = "rockchip,rk3588-vdec", .data = &vdpu381_variant,
PATCH v2 3/3 arm64: dts: rockchip: Add the vdpu346 Video Decoders on RK356X
In reply to Christian HewittAdd the vdpu346 Video Decoders to the rk356x-base devicetree to enable support on RK3566 and RK3568 boards. Also add the needed sram and vdec_mmu nodes.
--- arch/arm64/boot/dts/rockchip/rk356x-base.dtsi | 49 +++++++++++++++++++ 1 file changed, 49 insertions(+)
arch/arm64/boot/dts/rockchip/rk356x-base.dtsi +49 -0
@@ -383,6 +383,19 @@ usb2phy1_grf: syscon@fdca8000 { reg = <0x0 0xfdca8000 0x0 0x8000>; }; + sram@fdcc0000 { + compatible = "mmio-sram"; + reg = <0x0 0xfdcc0000 0x0 0xb000>; + ranges = <0x0 0x0 0xfdcc0000 0xb000>; + #address-cells = <1>; + #size-cells = <1>; + + vdec_sram: rkvdec-sram@0 { + reg = <0x0 0xb000>; + pool; + }; + }; + pmucru: clock-controller@fdd00000 { compatible = "rockchip,rk3568-pmucru"; reg = <0x0 0xfdd00000 0x0 0x1000>;
@@ -619,6 +632,42 @@ vepu_mmu: iommu@fdee0800 { #iommu-cells = <0>; }; + vdec: video-codec@fdf80100 { + compatible = "rockchip,rk3568-vdec"; + reg = <0x0 0xfdf80200 0x0 0x500>, + <0x0 0xfdf80100 0x0 0x100>, + <0x0 0xfdf80700 0x0 0x100>; + reg-names = "function", "link", "cache"; + interrupts = <GIC_SPI 91 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&cru ACLK_RKVDEC>, <&cru HCLK_RKVDEC>, + <&cru CLK_RKVDEC_CA>, <&cru CLK_RKVDEC_CORE>, + <&cru CLK_RKVDEC_HEVC_CA>; + clock-names = "axi", "ahb", "cabac", "core", "hevc_cabac"; + assigned-clocks = <&cru ACLK_RKVDEC>, + <&cru CLK_RKVDEC_CORE>, + <&cru CLK_RKVDEC_CA>, + <&cru CLK_RKVDEC_HEVC_CA>; + assigned-clock-rates = <297000000>, <297000000>, + <297000000>, <600000000>; + iommus = <&vdec_mmu>; + power-domains = <&power RK3568_PD_RKVDEC>; + resets = <&cru SRST_A_RKVDEC>, <&cru SRST_H_RKVDEC>, + <&cru SRST_RKVDEC_CA>, <&cru SRST_RKVDEC_CORE>, + <&cru SRST_RKVDEC_HEVC_CA>; + reset-names = "axi", "ahb", "cabac", "core", "hevc_cabac"; + sram = <&vdec_sram>; + }; + + vdec_mmu: iommu@fdf80800 { + compatible = "rockchip,rk3568-iommu"; + reg = <0x0 0xfdf80800 0x0 0x40>, <0x0 0xfdf80840 0x0 0x40>; + interrupts = <GIC_SPI 92 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&cru ACLK_RKVDEC>, <&cru HCLK_RKVDEC>; + clock-names = "aclk", "iface"; + power-domains = <&power RK3568_PD_RKVDEC>; + #iommu-cells = <0>; + }; + vicap: video-capture@fdfe0000 { compatible = "rockchip,rk3568-vicap"; reg = <0x0 0xfdfe0000 0x0 0x200>;
Signature · 1 line
2.34.1
Hi Christian,
Quoted 45 lines · Coordinated Universal Time Christian Hewitt
This series depends upon Detlev Casanova's current v7 series for VDPU381 and VDPU383 support [0]. It adds support for the VDPU346 IP block used for H264, HEVC and (in active work) VP9 on the RK356X boards. VDPU346 appears to be a close relation to VDPU381 used with RK3588, except with a single core, output limited to 4K, and minor feature differences, e.g. HEVC level 5.1 on VDPU346 vs 6.1 on VDPU381. To handle differences we declare a new compatible.
The device-tree changes are derived from prior-art in Detlev's patches for RK3576 and RK3588, the vendor kernel [1], and some earlier patch attempts from Piotr Oniszczuk and Diederik de Haas.
The register differences between VDPU346 and VDPU381 appear to be minimal so we currently and intentionally reuse the VDPU381 h264, hevc, and regs files to avoid duplicating code. There is some evidence of buffer size changes in testing which still need to be explored. If further register analysis needs separate files for VDPU346 the refactoring will be beyond my current n00b coding skills and I will either need some coaching or would be happy to pass the series over to a more experienced developer.
NB: Testing with the v1 series showed lower mbps bitrate performance. This appears to be resolved though it's unclear to me whether this results from kernel changes or the ongoing reworking of ffmpeg
v4l2_requestsupport [2]. However with my current Linux 6.19-rc2 test branch [3] I'm now able to play Jellyfish H264 and HEVC test media over 100mbps.Changes since v1:
- Drop DO-NOT-MERGE patch as changes included in Detvlev v7 series
- Adapt to variant/capability changes in Detlev's v7 series
- Add bindings ack from Rob
- Add rkvdec ack from Nicolas
- Rebase against Linux 6.19-rc2[0] https://patchwork.kernel.org/project/linux-rockchip/list/?series=1034794
[1]
https://github.com/rockchip-linux/kernel/blob/develop-6.6/arch/arm64/boot/d
ts/rockchip/rk356x.dtsi#L1539 [2]
https://code.ffmpeg.org/Kwiboo/FFmpeg/commits/branch/v4l2request-v3 [3]
https://github.com/chewitt/linux/commits/rockchip-6.19.yChristian Hewitt (3):
media: dt-bindings: rockchip: Add RK3568 Video Decoder bindings
media: rkvdec: Add support for the VDPU346 variant
arm64: dts: rockchip: Add the vdpu346 Video Decoders on RK356X
Quoted 4 lines
.../bindings/media/rockchip,vdec.yaml | 2 + arch/arm64/boot/dts/rockchip/rk356x-base.dtsi | 49 +++++++++ .../media/platform/rockchip/rkvdec/rkvdec.c | 103 ++++++++++++++++++ 3 files changed, 154 insertions(+)
I tested this patch series on my RK3566 board with my own test videos and BBB (1080p60 and 4k30) using mpv with [1] patchset.
The command I used was:mpv --vo=dmabuf-wayland --hwdec=v4l2request input_file.mp4
Both H264 and HEVC plays with minor framedrops (which to me isn't noticeable).
You can add my Tested-by:
[1]: https://github.com/mpv-player/mpv/pull/14690
Best regards,
Dang
Hi Christian,
Le vendredi 26 décembre 2025 à 11:31 +0000, Christian Hewitt a écrit :
Quoted 19 lines
This series depends upon Detlev Casanova's current v7 series for VDPU381 and VDPU383 support [0]. It adds support for the VDPU346 IP block used for H264, HEVC and (in active work) VP9 on the RK356X boards. VDPU346 appears to be a close relation to VDPU381 used with RK3588, except with a single core, output limited to 4K, and minor feature differences, e.g. HEVC level 5.1 on VDPU346 vs 6.1 on VDPU381. To handle differences we declare a new compatible.
The device-tree changes are derived from prior-art in Detlev's patches for RK3576 and RK3588, the vendor kernel [1], and some earlier patch attempts from Piotr Oniszczuk and Diederik de Haas.
The register differences between VDPU346 and VDPU381 appear to be minimal so we currently and intentionally reuse the VDPU381 h264, hevc, and regs files to avoid duplicating code. There is some evidence of buffer size changes in testing which still need to be explored. If further register analysis needs separate files for VDPU346 the refactoring will be beyond my current n00b coding skills and I will either need some coaching or would be happy to pass the series over to a more experienced developer.
Quoted 5 lines
NB: Testing with the v1 series showed lower mbps bitrate performance. This appears to be resolved though it's unclear to me whether this results from kernel changes or the ongoing reworking of ffmpeg
v4l2_requestsupport [2]. However with my current Linux 6.19-rc2 test branch [3] I'm now able to play Jellyfish H264 and HEVC test media over 100mbps.
A fluster score and explained error report is missing your this cover letter. This will probably highlight some of the instability issues reported in this series. I'll mark the series are "Change Request", as you have few review comments and now this test.
Overall, I'd like to see probably investigations of the problem before going ahead.
regards,
Nicolas
Quoted 23 lines
Changes since v1:
- Drop DO-NOT-MERGE patch as changes included in Detvlev v7 series
- Adapt to variant/capability changes in Detlev's v7 series
- Add bindings ack from Rob
- Add rkvdec ack from Nicolas
- Rebase against Linux 6.19-rc2[0] https://patchwork.kernel.org/project/linux-rockchip/list/?series=1034794
[1]
https://github.com/rockchip-linux/kernel/blob/develop-6.6/arch/arm64/boot/dts/rockchip/rk356x.dtsi#L1539
[2] https://code.ffmpeg.org/Kwiboo/FFmpeg/commits/branch/v4l2request-v3
[3] https://github.com/chewitt/linux/commits/rockchip-6.19.yChristian Hewitt (3):
media: dt-bindings: rockchip: Add RK3568 Video Decoder bindings
media: rkvdec: Add support for the VDPU346 variant
arm64: dts: rockchip: Add the vdpu346 Video Decoders on RK356X.../bindings/media/rockchip,vdec.yaml | 2 +
arch/arm64/boot/dts/rockchip/rk356x-base.dtsi| 49 +++++++++ .../media/platform/rockchip/rkvdec/rkvdec.c | 103 ++++++++++++++++++3 files changed, 154 insertions(+)
Hi Nicolas,
Quoted 20 lines · Nicolas Dufresne
Le vendredi 26 décembre 2025 à 11:31 +0000, Christian Hewitt a écrit :
Quoted 11 lines
This series depends upon Detlev Casanova's current v7 series for VDPU381 and VDPU383 support [0]. It adds support for the VDPU346 IP block used for H264, HEVC and (in active work) VP9 on the RK356X boards. VDPU346 appears to be a close relation to VDPU381 used with RK3588, except with a single core, output limited to 4K, and minor feature differences, e.g. HEVC level 5.1 on VDPU346 vs 6.1 on VDPU381. To handle differences we declare a new compatible.
The device-tree changes are derived from prior-art in Detlev's patches for RK3576 and RK3588, the vendor kernel [1], and some earlier patch attempts from Piotr Oniszczuk and Diederik de Haas.
Quoted 7 lines
The register differences between VDPU346 and VDPU381 appear to be minimal so we currently and intentionally reuse the VDPU381 h264, hevc, and regs files to avoid duplicating code. There is some evidence of buffer size changes in testing which still need to be explored. If further register analysis needs separate files for VDPU346 the refactoring will be beyond my current n00b coding skills and I will either need some coaching or would be happy to pass the series over to a more experienced developer.
Quoted 7 lines
A fluster score and explained error report is missing your this cover letter. This will probably highlight some of the instability issues reported in this series. I'll mark the series are "Change Request", as you have few review comments and now this test.
Overall, I'd like to see probably investigations of the problem before going ahead.
FTR: You replied to v2 while there is a v3 here:
https://patch.msgid.link/[email protected]/
It (still) doesn't have a fluster score, but it is the currently most recent version posted on the MLs.
Cheers,
Diederik