Skip to contentSkip to conversation

media: rockchip: rkvdec: add support for the VDPU346 variant

PATCH In review Daniel Golle 3 participants 11 messages Nov 14

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover media: rockchip: rkvdec: add support for the VDPU346 variant

This series adds the VDPU346 decoder of the RK3566 and RK3568 to the rkvdec driver, for H.264 and HEVC. It continues Christian Hewitt's v3 [1], whose three patches are carried unchanged; in the v2 cover letter he offered to pass the series on. Patches 3, 4, 6 and 7 are new and come out of a comparison of the driver with the vendor kernel driver (rockchip-linux/kernel develop-6.6, drivers/video/rockchip/mpp) and the Rockchip MPP library, followed by conformance runs on hardware.

Fluster on a NanoPi R5C (RK3568B2), GStreamer 1.28.7 v4l2sl decoders:

JCT-VC-HEVC_V1 142/147
JVT-AVC_V1 129/135
JVT-FR-EXT 65/69

HEVC failures: PICSIZE_A/B/C/D_Bossen_1 are coded at up to 8440 luma samples in one dimension, beyond the 4096x2304 limit of the decoder. TSUNEQBD_A_MAIN10_Technicolor_2 has unequal luma and chroma bit depth, which the driver rejects for every variant.

H.264 failures: CVFC1_Sony_C, FM1_BT_B, FM1_FT_E, SP1_BT_A, sp2_bt_b and FM2_SVA_C fail in JVT-AVC_V1; the Hantro G1 of the same board fails the same six with the same GStreamer. In JVT-FR-EXT Hi422FREXT17_SONY_A, Hi422FREXT19_SONY_A, HPCAMOLQ_BRCM_B and HPCVMOLQ_BRCM_B fail; the two BRCM vectors fail on the Hantro G1 as well, the two Hi422 intra profile vectors are not understood yet.

SAODBLK_A/B_MainConcept_4 and FREH10-1/2 are counted as passes: they decode correctly and exceed the default 60 s fluster timeout only because the checksum is computed from an uncached mapping of the capture buffers.

Patch 4 is what makes the score deterministic. With the read caches left in their reset state, RAP_B_Bossen_2, WPP_D_ericsson_MAIN_2, WPP_D_ericsson_MAIN10_2, WPP_E_ericsson_MAIN_2 (HEVC) and CAPM3_Sony_D, MR8_BT_B, MR9_BT_B, NRF_MW_E (H.264) produce wrong output in three runs out of three; with the patch they pass in three out of three. This is likely the source of the artefacts reported against v3. The function is shared with the VDPU381, where the cache block is mapped by the binding already; the patch has been run on the RK3568 only.

Decode rates with the series: HEVC Main 2160p 92 fps, Main10 2160p 77 fps, Main 1080p 380 fps, H.264 High 1080p 197 fps.

On the row and column buffer sizes raised in the v3 review [2]: the MPP library uses one vdpu34x HAL with one set of size formulas for the RK356x and the RK3588, so the variant keeps vdpu381_rcb_sizes.

Known limits, none of them introduced here:

- FFmpeg's v4l2-request hwaccel does not set the HEVC EXT_SPS_ST_RPS and EXT_SPS_LT_RPS controls, so HEVC through FFmpeg decodes wrongly on this decoder as on the VDPU381 ("Long and short term RPS not set").
- A decode error on such a stream ends in the IOMMU page faults and the oops reported against v3 [3]. The cause is in the error path shared by all variants: the interrupt handler restores the IOMMU while the watchdog is still armed, and the watchdog starts the next job while the IOMMU is detached. "media: rkvdec: Handle interrupts without a current context" [4] covers the oops; the remainder will be addressed separately.

Patches 3, 4, 6 and 7 were written with the help of an LLM coding assistant, which also performed the comparison with the vendor sources. It was asked to review the v3 series against the vendor driver and to complete it; the conformance and A/B runs above were done on the board with builds of this series on Linux 6.18.

Changes since v3:
- Add "media: rkvdec: vdpu346: Limit HEVC EXT SPS RPS dimensions"
- Add "media: rkvdec: vdpu381: Program the read caches"
- Add "arm64: dts: rockchip: rk356x: Order vdec regs by address"
- Add "arm64: dts: rockchip: rk356x: Clock the vdec at 400 MHz"
- Add fluster scores and failure analysis
- Add Tested-by from Diederik de Haas

v3: https://lore.kernel.org/r/[email protected]
v2: https://lore.kernel.org/r/[email protected]

[1] https://lore.kernel.org/r/[email protected]
[2] https://lore.kernel.org/r/[email protected]
[3] https://lore.kernel.org/r/[email protected]
[4] https://lore.kernel.org/r/[email protected]

Christian Hewitt (3):
media: dt-bindings: rockchip: Add RK3568 Video Decoder bindings
media: rkvdec: Add support for the VDPU346 variant
arm64: dts: rockchip: Add the vdpu346 Video Decoders on RK356X

Daniel Golle (4):
media: rkvdec: vdpu346: Limit HEVC EXT SPS RPS dimensions
media: rkvdec: vdpu381: Program the read caches
arm64: dts: rockchip: rk356x: Order vdec regs by address
arm64: dts: rockchip: rk356x: Clock the vdec at 400 MHz

 .../bindings/media/rockchip,vdec.yaml         |   2 +
 arch/arm64/boot/dts/rockchip/rk356x-base.dtsi |  49 +++++++
 .../rockchip/rkvdec/rkvdec-vdpu381-h264.c     |   2 +
 .../rockchip/rkvdec/rkvdec-vdpu381-hevc.c     |   2 +
 .../rockchip/rkvdec/rkvdec-vdpu381-regs.h     |   9 ++
 .../media/platform/rockchip/rkvdec/rkvdec.c   | 121 ++++++++++++++++++
 .../media/platform/rockchip/rkvdec/rkvdec.h   |   2 +
 7 files changed, 187 insertions(+)

base-commit: 7475ea8b58616b33c04afb976cf0fbaad9dd63fb

Signature · 1 line
2.56.0
#1 lore raw

PATCH 1/7 media: dt-bindings: rockchip: Add RK3568 Video Decoder bindings

From: Christian Hewitt <[email protected]>

The video decoder in RK356X (vdpu346) is described in the same way as the one in RK3588 (vdpu381). A new compatible is added as the decoder capabilities are a subset of the vdpu381 capabilities.

---
 Documentation/devicetree/bindings/media/rockchip,vdec.yaml | 2 ++
 1 file changed, 2 insertions(+)
Documentation/devicetree/bindings/media/rockchip,vdec.yaml +2 -0
index 4f38a0ef29d84..ea5386e84f17e 100644
@@ -18,6 +18,7 @@ properties:
     oneOf:
       - const: rockchip,rk3288-vdec
       - const: rockchip,rk3399-vdec
+      - const: rockchip,rk3568-vdec
       - const: rockchip,rk3576-vdec
       - const: rockchip,rk3588-vdec
       - items:
@@ -110,6 +111,7 @@ allOf:
         compatible:
           contains:
             enum:
+              - rockchip,rk3568-vdec
               - rockchip,rk3576-vdec
               - rockchip,rk3588-vdec
     then:
Signature · 1 line
2.56.0
#2 lore raw

Re: [PATCH 1/7] media: dt-bindings: rockchip: Add RK3568 Video Decoder bindings

Quoted 10 lines · Daniel Golle

From: Christian Hewitt <[email protected]>

The video decoder in RK356X (vdpu346) is described in the same way as the one in RK3588 (vdpu381). A new compatible is added as the decoder capabilities are a subset of the vdpu381 capabilities.

How v1 could get it?

Anyway, drop, not applicable here. You cannot test a binding on PineTab2.

Best regards,
Krzysztof

#3 lore raw

Re: [PATCH 1/7] media: dt-bindings: rockchip: Add RK3568 Video Decoder bindings

Quoted 7 lines · Krzysztof Kozlowski
Quoted 6 lines · Daniel Golle

From: Christian Hewitt <[email protected]>

The video decoder in RK356X (vdpu346) is described in the same way as the one in RK3588 (vdpu381). A new compatible is added as the decoder capabilities are a subset of the vdpu381 capabilities.

Quoted 8 lines
Quoted 4 lines

How v1 could get it?

Anyway, drop, not applicable here. You cannot test a binding on PineTab2.

This patch was previously posted (see cover letter) and received a whoole-series Tested-by. It's true that for DT bindings that doesn't make much sense, I just mechanically copied the tag to the patches which have previosly been submitted and weren't changed.

#4 lore raw

PATCH 2/7 media: rkvdec: Add support for the VDPU346 variant

In reply to Daniel Golle

From: Christian Hewitt <[email protected]>

VDPU346 is similar to VDPU381 but with a single core and limited to 4K60 media. It is also limited to H264 L5.1 and omits AV1 and AVS2 capabilities. VDPU346 is used with RK3566 and RK3568.

---
 .../media/platform/rockchip/rkvdec/rkvdec.c   | 103 ++++++++++++++++++
 1 file changed, 103 insertions(+)
drivers/media/platform/rockchip/rkvdec/rkvdec.c +103 -0
index 5250dd59df2ec..119564e33877d 100644
@@ -236,6 +236,62 @@ static const struct rkvdec_ctrls rkvdec_hevc_ctrls = {
 	.num_ctrls = ARRAY_SIZE(rkvdec_hevc_ctrl_descs),
 };
 
+static const struct rkvdec_ctrl_desc vdpu346_hevc_ctrl_descs[] = {
+	{
+		.cfg.id = V4L2_CID_STATELESS_HEVC_DECODE_PARAMS,
+	},
+	{
+		.cfg.id = V4L2_CID_STATELESS_HEVC_SPS,
+		.cfg.ops = &rkvdec_ctrl_ops,
+	},
+	{
+		.cfg.id = V4L2_CID_STATELESS_HEVC_PPS,
+	},
+	{
+		.cfg.id = V4L2_CID_STATELESS_HEVC_SCALING_MATRIX,
+	},
+	{
+		.cfg.id = V4L2_CID_STATELESS_HEVC_DECODE_MODE,
+		.cfg.min = V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
+		.cfg.max = V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
+		.cfg.def = V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
+	},
+	{
+		.cfg.id = V4L2_CID_STATELESS_HEVC_START_CODE,
+		.cfg.min = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
+		.cfg.def = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
+		.cfg.max = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
+	},
+	{
+		.cfg.id = V4L2_CID_MPEG_VIDEO_HEVC_PROFILE,
+		.cfg.min = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
+		.cfg.max = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10,
+		.cfg.menu_skip_mask =
+			BIT(V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE),
+		.cfg.def = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
+	},
+	{
+		.cfg.id = V4L2_CID_MPEG_VIDEO_HEVC_LEVEL,
+		.cfg.min = V4L2_MPEG_VIDEO_HEVC_LEVEL_1,
+		.cfg.max = V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1,
+	},
+	{
+		.cfg.id = V4L2_CID_STATELESS_HEVC_EXT_SPS_ST_RPS,
+		.cfg.ops = &rkvdec_ctrl_ops,
+		.cfg.dims = { 65 },
+	},
+	{
+		.cfg.id = V4L2_CID_STATELESS_HEVC_EXT_SPS_LT_RPS,
+		.cfg.ops = &rkvdec_ctrl_ops,
+		.cfg.dims = { 65 },
+	},
+};
+
+static const struct rkvdec_ctrls vdpu346_hevc_ctrls = {
+	.ctrls = vdpu346_hevc_ctrl_descs,
+	.num_ctrls = ARRAY_SIZE(vdpu346_hevc_ctrl_descs),
+};
+
 static const struct rkvdec_ctrl_desc vdpu38x_hevc_ctrl_descs[] = {
 	{
 		.cfg.id = V4L2_CID_STATELESS_HEVC_DECODE_PARAMS,
@@ -510,6 +566,41 @@ static const struct rkvdec_coded_fmt_desc rk3288_coded_fmts[] = {
 	}
 };
 
+static const struct rkvdec_coded_fmt_desc vdpu346_coded_fmts[] = {
+	{
+		.fourcc = V4L2_PIX_FMT_HEVC_SLICE,
+		.frmsize = {
+			.min_width = 64,
+			.max_width = 4096,
+			.step_width = 64,
+			.min_height = 64,
+			.max_height = 2304,
+			.step_height = 16,
+		},
+		.ctrls = &vdpu346_hevc_ctrls,
+		.ops = &rkvdec_vdpu381_hevc_fmt_ops,
+		.num_decoded_fmts = ARRAY_SIZE(rkvdec_hevc_decoded_fmts),
+		.decoded_fmts = rkvdec_hevc_decoded_fmts,
+		.subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF,
+	},
+	{
+		.fourcc = V4L2_PIX_FMT_H264_SLICE,
+		.frmsize = {
+			.min_width = 64,
+			.max_width =  4096,
+			.step_width = 64,
+			.min_height = 64,
+			.max_height =  2304,
+			.step_height = 16,
+		},
+		.ctrls = &rkvdec_h264_ctrls,
+		.ops = &rkvdec_vdpu381_h264_fmt_ops,
+		.num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_decoded_fmts),
+		.decoded_fmts = rkvdec_h264_decoded_fmts,
+		.subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF,
+	},
+};
+
 static const struct rkvdec_coded_fmt_desc vdpu381_coded_fmts[] = {
 	{
 		.fourcc = V4L2_PIX_FMT_HEVC_SLICE,
@@ -1704,6 +1795,14 @@ static const struct rkvdec_variant_ops vdpu381_variant_ops = {
 	.flatten_matrices = transpose_and_flatten_matrices,
 };
 
+static const struct rkvdec_variant vdpu346_variant = {
+	.coded_fmts = vdpu346_coded_fmts,
+	.num_coded_fmts = ARRAY_SIZE(vdpu346_coded_fmts),
+	.rcb_sizes = vdpu381_rcb_sizes,
+	.num_rcb_sizes = ARRAY_SIZE(vdpu381_rcb_sizes),
+	.ops = &vdpu381_variant_ops,
+};
+
 static const struct rkvdec_variant vdpu381_variant = {
 	.coded_fmts = vdpu381_coded_fmts,
 	.num_coded_fmts = ARRAY_SIZE(vdpu381_coded_fmts),
@@ -1752,6 +1851,10 @@ static const struct of_device_id of_rkvdec_match[] = {
 		.compatible = "rockchip,rk3399-vdec",
 		.data = &rk3399_rkvdec_variant,
 	},
+	{
+		.compatible = "rockchip,rk3568-vdec",
+		.data = &vdpu346_variant,
+	},
 	{
 		.compatible = "rockchip,rk3588-vdec",
 		.data = &vdpu381_variant,
Signature · 1 line
2.56.0
#5 lore raw

PATCH 3/7 media: rkvdec: vdpu346: Limit HEVC EXT SPS RPS dimensions

In reply to Daniel Golle

The VDPU346 control table registers the HEVC EXT_SPS_ST_RPS and EXT_SPS_LT_RPS controls with 65 elements each. struct rkvdec_rps holds 64 short term sets and 32 long term references, the limits of ITU-T H.265 7.4.8 and 7.4.3.2.1, and the VDPU346 shares that table with the VDPU381.

Register the controls with 64 and 32 elements, the dimensions the VDPU381 and VDPU383 use since commit f0b9d7e5be06 ("media: rkvdec:
hevc: tighten EXT SPS RPS control dimensions").

---
v4: new patch

---
 drivers/media/platform/rockchip/rkvdec/rkvdec.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/media/platform/rockchip/rkvdec/rkvdec.c +2 -2
index 119564e33877d..ba61e373b4342 100644
@@ -278,12 +278,12 @@ static const struct rkvdec_ctrl_desc vdpu346_hevc_ctrl_descs[] = {
 	{
 		.cfg.id = V4L2_CID_STATELESS_HEVC_EXT_SPS_ST_RPS,
 		.cfg.ops = &rkvdec_ctrl_ops,
-		.cfg.dims = { 65 },
+		.cfg.dims = { 64 },
 	},
 	{
 		.cfg.id = V4L2_CID_STATELESS_HEVC_EXT_SPS_LT_RPS,
 		.cfg.ops = &rkvdec_ctrl_ops,
-		.cfg.dims = { 65 },
+		.cfg.dims = { 32 },
 	},
 };
 
Signature · 1 line
2.56.0
#6 lore raw

PATCH 4/7 media: rkvdec: vdpu381: Program the read caches

In reply to Daniel Golle

The "cache" register block of the VDPU381 and VDPU346 holds the configuration of three read caches. The driver leaves the block unmapped and the caches in their reset state.

The vendor kernel driver, in rkvdec2_run() and in its link mode, permits cacheable access and read allocation with 64 byte lines and clears all three caches before every decode. Map the block and write the same configuration before each H.264 and HEVC run.

On the RK3568 the HEVC conformance vectors RAP_B_Bossen_2, WPP_D_ericsson_MAIN_2, WPP_D_ericsson_MAIN10_2 and WPP_E_ericsson_MAIN_2 and the H.264 vectors CAPM3_Sony_D, MR8_BT_B, MR9_BT_B and NRF_MW_E decode with wrong output while the caches are in their reset state, and pass once they are configured.

---
v4: new patch

---
 .../rockchip/rkvdec/rkvdec-vdpu381-h264.c      |  2 ++
 .../rockchip/rkvdec/rkvdec-vdpu381-hevc.c      |  2 ++
 .../rockchip/rkvdec/rkvdec-vdpu381-regs.h      |  9 +++++++++
 .../media/platform/rockchip/rkvdec/rkvdec.c    | 18 ++++++++++++++++++
 .../media/platform/rockchip/rkvdec/rkvdec.h    |  2 ++
 5 files changed, 33 insertions(+)
drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu381-h264.c +2 -0
index b961fddc8583a..40db8ba3856a7 100644
@@ -445,6 +445,8 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
 
 	rkvdec_schedule_watchdog(rkvdec, h264_ctx->regs.common.reg032_timeout_threshold);
 
+	rkvdec_vdpu381_config_cache(rkvdec);
+
 	/* Start decoding! */
 	writel(VDPU381_DEC_E_BIT, rkvdec->regs + VDPU381_REG_DEC_E);
 
drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu381-hevc.c +2 -0
index e1936e87f45be..8afb13866d0bb 100644
@@ -617,6 +617,8 @@ static int rkvdec_hevc_run(struct rkvdec_ctx *ctx)
 
 	rkvdec_schedule_watchdog(rkvdec, hevc_ctx->regs.common.reg032_timeout_threshold);
 
+	rkvdec_vdpu381_config_cache(rkvdec);
+
 	/* Start decoding! */
 	writel(VDPU381_DEC_E_BIT, rkvdec->regs + VDPU381_REG_DEC_E);
 
drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu381-regs.h +9 -0
index 6da36031df2dc..b23b47217d43b 100644
@@ -41,6 +41,15 @@
 #define VDPU381_STA_INT_TIMEOUT		BIT(5)
 #define VDPU381_STA_INT_SOFTRESET_RDY	BIT(9)
 
+/* base: "cache" register block */
+#define VDPU381_CACHE_COUNT		3
+#define VDPU381_REG_CACHE_CLR(i)	(0x010 + (i) * 0x40)
+#define VDPU381_CACHE_CLR		BIT(0)
+#define VDPU381_REG_CACHE_SIZE(i)	(0x01c + (i) * 0x40)
+#define VDPU381_CACHE_PERMIT_CACHEABLE_ACCESS	BIT(0)
+#define VDPU381_CACHE_PERMIT_READ_ALLOCATE	BIT(1)
+#define VDPU381_CACHE_LINE_SIZE_64_BYTES	BIT(4)
+
 /* base: OFFSET_COMMON_REGS */
 struct rkvdec_vdpu381_regs_common {
 	struct {
drivers/media/platform/rockchip/rkvdec/rkvdec.c +18 -0
index ba61e373b4342..5673c5205bfcd 100644
@@ -1249,6 +1249,20 @@ void rkvdec_schedule_watchdog(struct rkvdec_dev *rkvdec, u32 timeout_threshold)
 	schedule_delayed_work(&rkvdec->watchdog_work, msecs_to_jiffies(watchdog_time));
 }
 
+void rkvdec_vdpu381_config_cache(struct rkvdec_dev *rkvdec)
+{
+	u32 cfg = VDPU381_CACHE_PERMIT_CACHEABLE_ACCESS |
+		  VDPU381_CACHE_PERMIT_READ_ALLOCATE |
+		  VDPU381_CACHE_LINE_SIZE_64_BYTES;
+	unsigned int i;
+
+	for (i = 0; i < VDPU381_CACHE_COUNT; i++)
+		writel(cfg, rkvdec->cache + VDPU381_REG_CACHE_SIZE(i));
+
+	for (i = 0; i < VDPU381_CACHE_COUNT; i++)
+		writel(VDPU381_CACHE_CLR, rkvdec->cache + VDPU381_REG_CACHE_CLR(i));
+}
+
 static void rkvdec_device_run(void *priv)
 {
 	struct rkvdec_ctx *ctx = priv;
@@ -1910,6 +1924,10 @@ static int rkvdec_probe(struct platform_device *pdev)
 		rkvdec->link = devm_platform_ioremap_resource_byname(pdev, "link");
 		if (IS_ERR(rkvdec->link))
 			return PTR_ERR(rkvdec->link);
+
+		rkvdec->cache = devm_platform_ioremap_resource_byname(pdev, "cache");
+		if (IS_ERR(rkvdec->cache))
+			return PTR_ERR(rkvdec->cache);
 	}
 
 	ret = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
drivers/media/platform/rockchip/rkvdec/rkvdec.h +2 -0
index a24be6638b6b7..54e6f50c0ff9f 100644
@@ -136,6 +136,7 @@ struct rkvdec_dev {
 	struct clk *axi_clk;
 	void __iomem *regs;
 	void __iomem *link;
+	void __iomem *cache;
 	struct mutex vdev_lock; /* serializes ioctls */
 	struct delayed_work watchdog_work;
 	struct gen_pool *sram_pool;
@@ -180,6 +181,7 @@ void rkvdec_run_preamble(struct rkvdec_ctx *ctx, struct rkvdec_run *run);
 void rkvdec_run_postamble(struct rkvdec_ctx *ctx, struct rkvdec_run *run);
 void rkvdec_memcpy_toio(void __iomem *dst, void *src, size_t len);
 void rkvdec_schedule_watchdog(struct rkvdec_dev *rkvdec, u32 timeout_threshold);
+void rkvdec_vdpu381_config_cache(struct rkvdec_dev *rkvdec);
 
 void rkvdec_quirks_disable_qos(struct rkvdec_ctx *ctx);
 
Signature · 1 line
2.56.0
#7 lore raw

Re: [PATCH 4/7] media: rkvdec: vdpu381: Program the read caches

Thank you for your contribution! Sashiko AI review found no new issues in this patch, but noticed 1 related pre-existing issue it does not introduce: - [Critical] media: rkvdec: missing device link…

Thank you for your contribution! Sashiko AI review found no new issues in this patch, but noticed 1 related pre-existing issue it does not introduce:
- [Critical] media: rkvdec: missing device link for SRAM pool in rkvdec_probe() [1]
--

commit 99da5f7b908ec0166f8a92ee5d45147870aca1c8
Author: Daniel Golle <[email protected]>

media: rkvdec: vdpu381: Program the read caches

This commit maps the VDPU381 "cache" register block in rkvdec_probe() and configures and clears the three read caches before starting each H.264 and HEVC decode run.

Quoted 4 lines
drivers/media/platform/rockchip/rkvdec/rkvdec.c +0 -0
index ba61e373b4342..5673c5205bfcd 100644

[ ... ]

drivers/media/platform/rockchip/rkvdec/rkvdec.c static int rkvdec_probe(struct platform_device *pdev) · line 1933

Quoted 11 lines
drivers/media/platform/rockchip/rkvdec/rkvdec.c +4 -0
@@ -1910,6 +1924,10 @@ static int rkvdec_probe(struct platform_device *pdev)
 		rkvdec->link = devm_platform_ioremap_resource_byname(pdev, "link");
 		if (IS_ERR(rkvdec->link))
 			return PTR_ERR(rkvdec->link);
+
+		rkvdec->cache = devm_platform_ioremap_resource_byname(pdev, "cache");
+		if (IS_ERR(rkvdec->cache))
+			return PTR_ERR(rkvdec->cache);
 	}
 
 	ret = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));

[Severity: Critical]
This problem wasn't introduced by this patch, but can unbinding the SRAM provider cause a use-after-free on rkvdec->sram_pool [1]?

Later in rkvdec_probe(), rkvdec->sram_pool is obtained via of_gen_pool_get(pdev->dev.of_node, "sram", 0) without creating a device link to the SRAM provider device. of_gen_pool_get() drops its device reference before returning, so if the SRAM provider is unbound, devres frees the gen_pool while rkvdec retains a dangling pointer that rkvdec_allocate_rcb() and rkvdec_free_rcb() pass to gen_pool_dma_zalloc_align() and gen_pool_free().

[1] https://sashiko.dev/#/bug/linux-052dff48-14cb-4d3e-8996-c834344fcb16

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
#8 lore raw

PATCH 5/7 arm64: dts: rockchip: Add the vdpu346 Video Decoders on RK356X

In reply to Daniel Golle

From: Christian Hewitt <[email protected]>

Add the vdpu346 Video Decoders to the rk356x-base devicetree to enable support on RK3566 and RK3568 boards. Also add the needed sram and vdec_mmu nodes.

---
 arch/arm64/boot/dts/rockchip/rk356x-base.dtsi | 49 +++++++++++++++++++
 1 file changed, 49 insertions(+)
arch/arm64/boot/dts/rockchip/rk356x-base.dtsi +49 -0
index 1a4562df20ee5..fac41e34005f2 100644
@@ -402,6 +402,19 @@ usb2phy1_grf: syscon@fdca8000 {
 		reg = <0x0 0xfdca8000 0x0 0x8000>;
 	};
 
+	sram@fdcc0000 {
+		compatible = "mmio-sram";
+		reg = <0x0 0xfdcc0000 0x0 0xb000>;
+		ranges = <0x0 0x0 0xfdcc0000 0xb000>;
+		#address-cells = <1>;
+		#size-cells = <1>;
+
+		vdec_sram: rkvdec-sram@0 {
+			reg = <0x0 0xb000>;
+			pool;
+		};
+	};
+
 	pmucru: clock-controller@fdd00000 {
 		compatible = "rockchip,rk3568-pmucru";
 		reg = <0x0 0xfdd00000 0x0 0x1000>;
@@ -638,6 +651,42 @@ vepu_mmu: iommu@fdee0800 {
 		#iommu-cells = <0>;
 	};
 
+	vdec: video-codec@fdf80100 {
+		compatible = "rockchip,rk3568-vdec";
+		reg = <0x0 0xfdf80200 0x0 0x500>,
+		      <0x0 0xfdf80100 0x0 0x100>,
+		      <0x0 0xfdf80700 0x0 0x100>;
+		reg-names = "function", "link", "cache";
+		interrupts = <GIC_SPI 91 IRQ_TYPE_LEVEL_HIGH>;
+		clocks = <&cru ACLK_RKVDEC>, <&cru HCLK_RKVDEC>,
+			 <&cru CLK_RKVDEC_CA>, <&cru CLK_RKVDEC_CORE>,
+			 <&cru CLK_RKVDEC_HEVC_CA>;
+		clock-names = "axi", "ahb", "cabac", "core", "hevc_cabac";
+		assigned-clocks = <&cru ACLK_RKVDEC>,
+				  <&cru CLK_RKVDEC_CORE>,
+				  <&cru CLK_RKVDEC_CA>,
+				  <&cru CLK_RKVDEC_HEVC_CA>;
+		assigned-clock-rates = <297000000>, <297000000>,
+				       <297000000>, <600000000>;
+		iommus = <&vdec_mmu>;
+		power-domains = <&power RK3568_PD_RKVDEC>;
+		resets = <&cru SRST_A_RKVDEC>, <&cru SRST_H_RKVDEC>,
+			 <&cru SRST_RKVDEC_CA>, <&cru SRST_RKVDEC_CORE>,
+			 <&cru SRST_RKVDEC_HEVC_CA>;
+		reset-names = "axi", "ahb", "cabac", "core", "hevc_cabac";
+		sram = <&vdec_sram>;
+	};
+
+	vdec_mmu: iommu@fdf80800 {
+		compatible = "rockchip,rk3568-iommu";
+		reg = <0x0 0xfdf80800 0x0 0x40>, <0x0 0xfdf80840 0x0 0x40>;
+		interrupts = <GIC_SPI 92 IRQ_TYPE_LEVEL_HIGH>;
+		clocks = <&cru ACLK_RKVDEC>, <&cru HCLK_RKVDEC>;
+		clock-names = "aclk", "iface";
+		power-domains = <&power RK3568_PD_RKVDEC>;
+		#iommu-cells = <0>;
+	};
+
 	vicap: video-capture@fdfe0000 {
 		compatible = "rockchip,rk3568-vicap";
 		reg = <0x0 0xfdfe0000 0x0 0x200>;
Signature · 1 line
2.56.0
#9 lore raw

PATCH 6/7 arm64: dts: rockchip: rk356x: Order vdec regs by address

In reply to Daniel Golle

The rockchip,vdec binding lists the register blocks as link, function, cache since commit 35c8178ed2bd ("media: dt-bindings: rockchip,vdec:
Add alternative reg-names order for RK35{76,88}") and marks the function, link, cache order deprecated. The RK356x node uses the deprecated order, which also leaves its unit address 0xfdf80100 on the second reg entry.

List the blocks as link, function, cache. The driver looks them up by name.

---
v4: new patch

---
 arch/arm64/boot/dts/rockchip/rk356x-base.dtsi | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
arch/arm64/boot/dts/rockchip/rk356x-base.dtsi +3 -3
index fac41e34005f2..eefcaa2c26ffe 100644
@@ -653,10 +653,10 @@ vepu_mmu: iommu@fdee0800 {
 
 	vdec: video-codec@fdf80100 {
 		compatible = "rockchip,rk3568-vdec";
-		reg = <0x0 0xfdf80200 0x0 0x500>,
-		      <0x0 0xfdf80100 0x0 0x100>,
+		reg = <0x0 0xfdf80100 0x0 0x100>,
+		      <0x0 0xfdf80200 0x0 0x500>,
 		      <0x0 0xfdf80700 0x0 0x100>;
-		reg-names = "function", "link", "cache";
+		reg-names = "link", "function", "cache";
 		interrupts = <GIC_SPI 91 IRQ_TYPE_LEVEL_HIGH>;
 		clocks = <&cru ACLK_RKVDEC>, <&cru HCLK_RKVDEC>,
 			 <&cru CLK_RKVDEC_CA>, <&cru CLK_RKVDEC_CORE>,
Signature · 1 line
2.56.0
#10 lore raw

PATCH 7/7 arm64: dts: rockchip: rk356x: Clock the vdec at 400 MHz

In reply to Daniel Golle

The decoder AXI, core and CABAC clocks are assigned 297 MHz, the rate the vendor kernel derives as a quarter of its 1188 MHz GPLL. This dtsi runs the GPLL at 1200 MHz, from which the closest rate below 297 MHz is 240 MHz.

Assign 400 MHz, a third of the GPLL. The vendor kernel clocks the decoder at a third of its GPLL for streams above 1920x1088 and lists 400 MHz at 900 mV as the highest decoder operating point.

---
v4: new patch

---
 arch/arm64/boot/dts/rockchip/rk356x-base.dtsi | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm64/boot/dts/rockchip/rk356x-base.dtsi +2 -2
index eefcaa2c26ffe..bb30c2939a269 100644
@@ -666,8 +666,8 @@ vdec: video-codec@fdf80100 {
 				  <&cru CLK_RKVDEC_CORE>,
 				  <&cru CLK_RKVDEC_CA>,
 				  <&cru CLK_RKVDEC_HEVC_CA>;
-		assigned-clock-rates = <297000000>, <297000000>,
-				       <297000000>, <600000000>;
+		assigned-clock-rates = <400000000>, <400000000>,
+				       <400000000>, <600000000>;
 		iommus = <&vdec_mmu>;
 		power-domains = <&power RK3568_PD_RKVDEC>;
 		resets = <&cru SRST_A_RKVDEC>, <&cru SRST_H_RKVDEC>,
Signature · 1 line
2.56.0