Skip to content

media: rockchip: rkvdec: add support for the VDPU346 variant

Daniel Golle linux-arm-kernel linux-devicetree linux-media linux-rockchip lkml 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest7 patches 2 repliesIn review

Changes since v3 from the cover letter

- Add "media: rkvdec: vdpu346: Limit HEVC EXT SPS RPS dimensions"
- Add "media: rkvdec: vdpu381: Program the read caches"
- Add "arm64: dts: rockchip: rk356x: Order vdec regs by address"
- Add "arm64: dts: rockchip: rk356x: Clock the vdec at 400 MHz"
- Add fluster scores and failure analysis
- Add Tested-by from Diederik de Haas

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/7media: dt-bindings: rockchip: Add RK3568 Video Decoder bindingsDocumentation/devicetree/bindings/media/rockchip,vdec.yaml+2 -02 replies
2/7media: rkvdec: Add support for the VDPU346 variantdrivers/media/platform/rockchip/rkvdec/rkvdec.c+103 -00 replies
3/7media: rkvdec: vdpu346: Limit HEVC EXT SPS RPS dimensionsdrivers/media/platform/rockchip/rkvdec/rkvdec.c+2 -20 replies
4/7media: rkvdec: vdpu381: Program the read cachesdrivers/media/platform/rockchip/rkvdec/rkvdec-vdpu381-h264.c drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu381-hevc.c drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu381-regs.h +2 more+33 -00 replies
5/7arm64: dts: rockchip: Add the vdpu346 Video Decoders on RK356Xarch/arm64/boot/dts/rockchip/rk356x-base.dtsi+49 -00 replies
6/7arm64: dts: rockchip: rk356x: Order vdec regs by addressarch/arm64/boot/dts/rockchip/rk356x-base.dtsi+3 -30 replies
7/7arm64: dts: rockchip: rk356x: Clock the vdec at 400 MHzarch/arm64/boot/dts/rockchip/rk356x-base.dtsi+2 -20 replies

Reviews trailers per patch

2 of 7 patches have a Reviewed-by or Acked-by.

PatchRob Herring (Arm)Diederik de HaasNicolas DufresneDang Huynh
1/7A-b T-b
2/7T-b R-b T-b
3/7
4/7
5/7T-b
6/7
7/7

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 4 reviewers.

Read the v1 thread

Cover letter [PATCH 0/7] media: rockchip: rkvdec: add support for the VDPU346 variant

This series adds the VDPU346 decoder of the RK3566 and RK3568 to the rkvdec driver, for H.264 and HEVC. It continues Christian Hewitt's v3 [1], whose three patches are carried unchanged; in the v2 cover letter he offered to pass the series on. Patches 3, 4, 6 and 7 are new and come out of a comparison of the driver with the vendor kernel driver (rockchip-linux/kernel develop-6.6, drivers/video/rockchip/mpp) and the Rockchip MPP library, followed by conformance runs on hardware.

Fluster on a NanoPi R5C (RK3568B2), GStreamer 1.28.7 v4l2sl decoders:

JCT-VC-HEVC_V1 142/147
JVT-AVC_V1 129/135
JVT-FR-EXT 65/69

HEVC failures: PICSIZE_A/B/C/D_Bossen_1 are coded at up to 8440 luma samples in one dimension, beyond the 4096x2304 limit of the decoder. TSUNEQBD_A_MAIN10_Technicolor_2 has unequal luma and chroma bit depth, which the driver rejects for every variant.

H.264 failures: CVFC1_Sony_C, FM1_BT_B, FM1_FT_E, SP1_BT_A, sp2_bt_b and FM2_SVA_C fail in JVT-AVC_V1; the Hantro G1 of the same board fails the same six with the same GStreamer. In JVT-FR-EXT Hi422FREXT17_SONY_A, Hi422FREXT19_SONY_A, HPCAMOLQ_BRCM_B and HPCVMOLQ_BRCM_B fail; the two BRCM vectors fail on the Hantro G1 as well, the two Hi422 intra profile vectors are not understood yet.

SAODBLK_A/B_MainConcept_4 and FREH10-1/2 are counted as passes: they decode correctly and exceed the default 60 s fluster timeout only because the checksum is computed from an uncached mapping of the capture buffers.

Patch 4 is what makes the score deterministic. With the read caches left in their reset state, RAP_B_Bossen_2, WPP_D_ericsson_MAIN_2, WPP_D_ericsson_MAIN10_2, WPP_E_ericsson_MAIN_2 (HEVC) and CAPM3_Sony_D, MR8_BT_B, MR9_BT_B, NRF_MW_E (H.264) produce wrong output in three runs out of three; with the patch they pass in three out of three. This is likely the source of the artefacts reported against v3. The function is shared with the VDPU381, where the cache block is mapped by the binding already; the patch has been run on the RK3568 only.

Decode rates with the series: HEVC Main 2160p 92 fps, Main10 2160p 77 fps, Main 1080p 380 fps, H.264 High 1080p 197 fps.

On the row and column buffer sizes raised in the v3 review [2]: the MPP library uses one vdpu34x HAL with one set of size formulas for the RK356x and the RK3588, so the variant keeps vdpu381_rcb_sizes.

Known limits, none of them introduced here:

- FFmpeg's v4l2-request hwaccel does not set the HEVC EXT_SPS_ST_RPS and EXT_SPS_LT_RPS controls, so HEVC through FFmpeg decodes wrongly on this decoder as on the VDPU381 ("Long and short term RPS not set").
- A decode error on such a stream ends in the IOMMU page faults and the oops reported against v3 [3]. The cause is in the error path shared by all variants: the interrupt handler restores the IOMMU while the watchdog is still armed, and the watchdog starts the next job while the IOMMU is detached. "media: rkvdec: Handle interrupts without a current context" [4] covers the oops; the remainder will be addressed separately.

Patches 3, 4, 6 and 7 were written with the help of an LLM coding assistant, which also performed the comparison with the vendor sources. It was asked to review the v3 series against the vendor driver and to complete it; the conformance and A/B runs above were done on the board with builds of this series on Linux 6.18.

Changes since v3:
- Add "media: rkvdec: vdpu346: Limit HEVC EXT SPS RPS dimensions"
- Add "media: rkvdec: vdpu381: Program the read caches"
- Add "arm64: dts: rockchip: rk356x: Order vdec regs by address"
- Add "arm64: dts: rockchip: rk356x: Clock the vdec at 400 MHz"
- Add fluster scores and failure analysis
- Add Tested-by from Diederik de Haas

v3: https://lore.kernel.org/r/[email protected]
v2: https://lore.kernel.org/r/[email protected]

[1] https://lore.kernel.org/r/[email protected]
[2] https://lore.kernel.org/r/[email protected]
[3] https://lore.kernel.org/r/[email protected]
[4] https://lore.kernel.org/r/[email protected]

Christian Hewitt (3):
media: dt-bindings: rockchip: Add RK3568 Video Decoder bindings
media: rkvdec: Add support for the VDPU346 variant
arm64: dts: rockchip: Add the vdpu346 Video Decoders on RK356X

Daniel Golle (4):
media: rkvdec: vdpu346: Limit HEVC EXT SPS RPS dimensions
media: rkvdec: vdpu381: Program the read caches
arm64: dts: rockchip: rk356x: Order vdec regs by address
arm64: dts: rockchip: rk356x: Clock the vdec at 400 MHz

 .../bindings/media/rockchip,vdec.yaml         |   2 +
 arch/arm64/boot/dts/rockchip/rk356x-base.dtsi |  49 +++++++
 .../rockchip/rkvdec/rkvdec-vdpu381-h264.c     |   2 +
 .../rockchip/rkvdec/rkvdec-vdpu381-hevc.c     |   2 +
 .../rockchip/rkvdec/rkvdec-vdpu381-regs.h     |   9 ++
 .../media/platform/rockchip/rkvdec/rkvdec.c   | 121 ++++++++++++++++++
 .../media/platform/rockchip/rkvdec/rkvdec.h   |   2 +
 7 files changed, 187 insertions(+)

base-commit: 7475ea8b58616b33c04afb976cf0fbaad9dd63fb

Signature · 1 line
2.56.0