Prepare for new rkvdec variants
Subsystems (5)
Detlev Casanova
maintainercover Prepare for new rkvdec variants
This patch set introduces the necessary bits for new variants implementations that do not modify the driver.
The driver changes will come in another patch set.
The first 4 patches document the device nodes and then add the nodes for rk3588 and rk3576. I kept the R-b tag from Conor Dooley as the patch didn't change much from [1]. I can remove it if needed.
Then a new v4l2 control for HEVC is added to be able to send the missing RPS information to the driver.
Finally, the driver is unstaged as the TODO points are removed. I added myself as maintainer for now as I'll be adding support for several variants.
Those 3 parts can be applied separately and do not depend on each other. Based on v6.16-rc3.
[1]: https://lore.kernel.org/all/20240620-unsmooth-surfer-b62ed94b4a5e@spud/
Detlev Casanova (8):
media: dt-bindings: rockchip: Document RK3588 Video Decoder bindings
media: dt-bindings: rockchip: Add RK3576 Video Decoder bindings
arm64: dts: rockchip: Add the vdpu381 Video Decoders on RK3588
arm64: dts: rockchip: Add the vdpu383 Video Decoder on rk3576
media: uapi: HEVC: Add v4l2_ctrl_hevc_ext_sps_rps control
media: v4l2-ctrls: Add hevc_ext_sps_rps control
media: rkvdec: Remove TODO file
media: rkvdec: Unstage the driver
.../bindings/media/rockchip,vdec.yaml | 80 ++++++++++++++++++-
.../media/v4l/ext-ctrls-codec-stateless.rst | 73 +++++++++++++++++
.../media/v4l/vidioc-queryctrl.rst | 6 ++
MAINTAINERS | 8 ++
arch/arm64/boot/dts/rockchip/rk3576.dtsi | 36 +++++++++
arch/arm64/boot/dts/rockchip/rk3588-base.dtsi | 74 +++++++++++++++++
drivers/media/platform/rockchip/Kconfig | 1 +
drivers/media/platform/rockchip/Makefile | 1 +
.../platform/rockchip}/rkvdec/Kconfig | 0
.../platform/rockchip}/rkvdec/Makefile | 0
.../platform/rockchip}/rkvdec/rkvdec-h264.c | 0
.../platform/rockchip}/rkvdec/rkvdec-regs.h | 0
.../platform/rockchip}/rkvdec/rkvdec-vp9.c | 0
.../platform/rockchip}/rkvdec/rkvdec.c | 0
.../platform/rockchip}/rkvdec/rkvdec.h | 0
drivers/media/v4l2-core/v4l2-ctrls-core.c | 9 +++
drivers/media/v4l2-core/v4l2-ctrls-defs.c | 5 ++
drivers/staging/media/Kconfig | 2 -
drivers/staging/media/Makefile | 1 -
drivers/staging/media/rkvdec/TODO | 11 ---
include/uapi/linux/v4l2-controls.h | 48 +++++++++++
include/uapi/linux/videodev2.h | 1 +
22 files changed, 339 insertions(+), 17 deletions(-)
rename drivers/{staging/media => media/platform/rockchip}/rkvdec/Kconfig (100%)
rename drivers/{staging/media => media/platform/rockchip}/rkvdec/Makefile (100%)
rename drivers/{staging/media => media/platform/rockchip}/rkvdec/rkvdec-h264.c (100%)
rename drivers/{staging/media => media/platform/rockchip}/rkvdec/rkvdec-regs.h (100%)
rename drivers/{staging/media => media/platform/rockchip}/rkvdec/rkvdec-vp9.c (100%)
rename drivers/{staging/media => media/platform/rockchip}/rkvdec/rkvdec.c (100%)
rename drivers/{staging/media => media/platform/rockchip}/rkvdec/rkvdec.h (100%)
delete mode 100644 drivers/staging/media/rkvdec/TODOSignature · 1 line
2.50.0
Detlev Casanova
maintainerPATCH 1/8 media: dt-bindings: rockchip: Document RK3588 Video Decoder bindings
Document the Rockchip RK3588 Video Decoder bindings.
--- .../bindings/media/rockchip,vdec.yaml | 77 ++++++++++++++++++- 1 file changed, 74 insertions(+), 3 deletions(-)
Documentation/devicetree/bindings/media/rockchip,vdec.yaml +74 -3
@@ -10,13 +10,14 @@ maintainers: - Heiko Stuebner <[email protected]> description: |- - The Rockchip rk3399 has a stateless Video Decoder that can decodes H.264, - HEVC an VP9 streams. + Rockchip SoCs have variants of the same stateless Video Decoder that can + decodes H.264, HEVC, VP9 and AV1 streams, depending on the variant. properties: compatible: oneOf: - const: rockchip,rk3399-vdec + - const: rockchip,rk3588-vdec - items: - enum: - rockchip,rk3228-vdec
@@ -24,35 +25,72 @@ properties: - const: rockchip,rk3399-vdec reg: - maxItems: 1 + minItems: 1 + items: + - description: The function configuration registers base + - description: The link table configuration registers base + - description: The cache configuration registers base + + reg-names: + items: + - const: function + - const: link + - const: cache interrupts: maxItems: 1 clocks: + minItems: 4 items: - description: The Video Decoder AXI interface clock - description: The Video Decoder AHB interface clock - description: The Video Decoded CABAC clock - description: The Video Decoder core clock + - description: The Video decoder HEVC CABAC clock clock-names: + minItems: 4 items: - const: axi - const: ahb - const: cabac - const: core + - const: hevc_cabac assigned-clocks: true assigned-clock-rates: true + resets: + items: + - description: The Video Decoder AXI interface reset + - description: The Video Decoder AHB interface reset + - description: The Video Decoded CABAC reset + - description: The Video Decoder core reset + - description: The Video decoder HEVC CABAC reset + + reset-names: + items: + - const: axi + - const: ahb + - const: cabac + - const: core + - const: hevc_cabac + power-domains: maxItems: 1 iommus: maxItems: 1 + sram: + $ref: /schemas/types.yaml#/definitions/phandle + description: | + phandle to a reserved on-chip SRAM regions. + Some SoCs, like rk3588 provide on-chip SRAM to store temporary + buffers during decoding. + required: - compatible - reg
@@ -61,6 +99,39 @@ required: - clock-names - power-domains +allOf: + - if: + properties: + compatible: + contains: + const: rockchip,rk3588-vdec + then: + properties: + reg: + minItems: 3 + reg-names: + minItems: 3 + clocks: + minItems: 5 + clock-names: + minItems: 5 + resets: + minItems: 5 + reset-names: + minItems: 5 + else: + properties: + reg: + maxItems: 1 + reg-names: false + clocks: + maxItems: 4 + clock-names: + maxItems: 4 + resets: false + reset-names: false + sram: false + additionalProperties: false examples:
Signature · 1 line
2.50.0
Detlev Casanova
maintainerPATCH 2/8 media: dt-bindings: rockchip: Add RK3576 Video Decoder bindings
In reply to Detlev CasanovaThe video decoder in RK3576 (vdpu383) is described the same way as the one in RK3588 (vdpu381). A new compatible is added as the driver implementation will be different.
--- Documentation/devicetree/bindings/media/rockchip,vdec.yaml | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
Documentation/devicetree/bindings/media/rockchip,vdec.yaml +4 -1
@@ -17,6 +17,7 @@ properties: compatible: oneOf: - const: rockchip,rk3399-vdec + - const: rockchip,rk3576-vdec - const: rockchip,rk3588-vdec - items: - enum:
@@ -104,7 +105,9 @@ allOf: properties: compatible: contains: - const: rockchip,rk3588-vdec + enum: + - rockchip,rk3576-vdec + - rockchip,rk3588-vdec then: properties: reg:
Signature · 1 line
2.50.0
Re: [PATCH 2/8] media: dt-bindings: rockchip: Add RK3576 Video Decoder bindings
Quoted 9 lines · Detlev Casanova
The video decoder in RK3576 (vdpu383) is described the same way as the one in RK3588 (vdpu381). A new compatible is added as the driver implementation will be different.
Signed-off-by Detlev Casanova <[email protected]>--- Documentation/devicetree/bindings/media/rockchip,vdec.yaml | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
Detlev Casanova
maintainerPATCH 3/8 arm64: dts: rockchip: Add the vdpu381 Video Decoders on RK3588
In reply to Detlev CasanovaAdd the vdpu381 Video Decoders to the rk3588-base devicetree.
The RK3588 based SoCs all embed 2 vdpu381 decoders. This also adds the dedicated IOMMU controllers.
--- arch/arm64/boot/dts/rockchip/rk3588-base.dtsi | 74 +++++++++++++++++++ 1 file changed, 74 insertions(+)
arch/arm64/boot/dts/rockchip/rk3588-base.dtsi +74 -0
@@ -1252,6 +1252,70 @@ vepu121_3_mmu: iommu@fdbac800 { #iommu-cells = <0>; }; + vdec0: video-codec@fdc38000 { + compatible = "rockchip,rk3588-vdec"; + reg = <0x0 0xfdc38100 0x0 0x500>, + <0x0 0xfdc38000 0x0 0x100>, + <0x0 0xfdc38600 0x0 0x100>; + reg-names = "function", "link", "cache"; + interrupts = <GIC_SPI 95 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&cru ACLK_RKVDEC0>, <&cru HCLK_RKVDEC0>, <&cru CLK_RKVDEC0_CA>, + <&cru CLK_RKVDEC0_CORE>, <&cru CLK_RKVDEC0_HEVC_CA>; + clock-names = "axi", "ahb", "cabac", "core", "hevc_cabac"; + assigned-clocks = <&cru ACLK_RKVDEC0>, <&cru CLK_RKVDEC0_CORE>, + <&cru CLK_RKVDEC0_CA>, <&cru CLK_RKVDEC0_HEVC_CA>; + assigned-clock-rates = <800000000>, <600000000>, + <600000000>, <1000000000>; + iommus = <&vdec0_mmu>; + power-domains = <&power RK3588_PD_RKVDEC0>; + resets = <&cru SRST_A_RKVDEC0>, <&cru SRST_H_RKVDEC0>, <&cru SRST_RKVDEC0_CA>, + <&cru SRST_RKVDEC0_CORE>, <&cru SRST_RKVDEC0_HEVC_CA>; + reset-names = "axi", "ahb", "cabac", "core", "hevc_cabac"; + sram = <&vdec0_sram>; + }; + + vdec0_mmu: iommu@fdc38700 { + compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu"; + reg = <0x0 0xfdc38700 0x0 0x40>, <0x0 0xfdc38740 0x0 0x40>; + interrupts = <GIC_SPI 96 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&cru ACLK_RKVDEC0>, <&cru HCLK_RKVDEC0>; + clock-names = "aclk", "iface"; + power-domains = <&power RK3588_PD_RKVDEC0>; + #iommu-cells = <0>; + }; + + vdec1: video-codec@fdc40000 { + compatible = "rockchip,rk3588-vdec"; + reg = <0x0 0xfdc40100 0x0 0x500>, + <0x0 0xfdc40000 0x0 0x100>, + <0x0 0xfdc40600 0x0 0x100>; + reg-names = "function", "link", "cache"; + interrupts = <GIC_SPI 97 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&cru ACLK_RKVDEC1>, <&cru HCLK_RKVDEC1>, <&cru CLK_RKVDEC1_CA>, + <&cru CLK_RKVDEC1_CORE>, <&cru CLK_RKVDEC1_HEVC_CA>; + clock-names = "axi", "ahb", "cabac", "core", "hevc_cabac"; + assigned-clocks = <&cru ACLK_RKVDEC1>, <&cru CLK_RKVDEC1_CORE>, + <&cru CLK_RKVDEC1_CA>, <&cru CLK_RKVDEC1_HEVC_CA>; + assigned-clock-rates = <800000000>, <600000000>, + <600000000>, <1000000000>; + iommus = <&vdec1_mmu>; + power-domains = <&power RK3588_PD_RKVDEC1>; + resets = <&cru SRST_A_RKVDEC1>, <&cru SRST_H_RKVDEC1>, <&cru SRST_RKVDEC1_CA>, + <&cru SRST_RKVDEC1_CORE>, <&cru SRST_RKVDEC1_HEVC_CA>; + reset-names = "axi", "ahb", "cabac", "core", "hevc_cabac"; + sram = <&vdec1_sram>; + }; + + vdec1_mmu: iommu@fdc40700 { + compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu"; + reg = <0x0 0xfdc40700 0x0 0x40>, <0x0 0xfdc40740 0x0 0x40>; + interrupts = <GIC_SPI 96 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&cru ACLK_RKVDEC1>, <&cru HCLK_RKVDEC1>; + clock-names = "aclk", "iface"; + power-domains = <&power RK3588_PD_RKVDEC1>; + #iommu-cells = <0>; + }; + av1d: video-codec@fdc70000 { compatible = "rockchip,rk3588-av1-vpu"; reg = <0x0 0xfdc70000 0x0 0x800>;
@@ -3093,6 +3157,16 @@ system_sram2: sram@ff001000 { ranges = <0x0 0x0 0xff001000 0xef000>; #address-cells = <1>; #size-cells = <1>; + + vdec0_sram: codec-sram@0 { + reg = <0x0 0x78000>; + pool; + }; + + vdec1_sram: codec-sram@78000 { + reg = <0x78000 0x77000>; + pool; + }; }; pinctrl: pinctrl {
Signature · 1 line
2.50.0
Detlev Casanova
maintainerPATCH 4/8 arm64: dts: rockchip: Add the vdpu383 Video Decoder on rk3576
In reply to Detlev CasanovaAdd the vdpu383 Video Decoder variant to the RK3576 device tree.
Also allow using the dedicated SRAM as a pool.
--- arch/arm64/boot/dts/rockchip/rk3576.dtsi | 36 ++++++++++++++++++++++++ 1 file changed, 36 insertions(+)
arch/arm64/boot/dts/rockchip/rk3576.dtsi +36 -0
@@ -1139,6 +1139,41 @@ gpu: gpu@27800000 { status = "disabled"; }; + vdec: video-codec@27b00000 { + compatible = "rockchip,rk3576-vdec"; + reg = <0x0 0x27b00100 0x0 0x500>, + <0x0 0x27b00000 0x0 0x100>, + <0x0 0x27b00600 0x0 0x100>; + reg-names = "function", "link", "cache"; + interrupts = <GIC_SPI 308 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&cru ACLK_RKVDEC_ROOT>, <&cru HCLK_RKVDEC>, + <&cru ACLK_RKVDEC_ROOT_BAK>, <&cru CLK_RKVDEC_CORE>, + <&cru CLK_RKVDEC_HEVC_CA>; + clock-names = "axi", "ahb", "cabac", "core", "hevc_cabac"; + assigned-clocks = <&cru ACLK_RKVDEC_ROOT>, <&cru CLK_RKVDEC_CORE>, + <&cru ACLK_RKVDEC_ROOT_BAK>, <&cru CLK_RKVDEC_HEVC_CA>; + assigned-clock-rates = <600000000>, <600000000>, + <500000000>, <1000000000>; + iommus = <&vdec_mmu>; + power-domains = <&power RK3576_PD_VDEC>; + resets = <&cru SRST_A_RKVDEC_BIU>, <&cru SRST_H_RKVDEC_BIU>, + <&cru SRST_H_RKVDEC>, <&cru SRST_RKVDEC_CORE>, + <&cru SRST_RKVDEC_HEVC_CA>; + reset-names = "axi", "ahb", "cabac", "core", "hevc_cabac"; + sram = <&rkvdec_sram>; + }; + + vdec_mmu: iommu@27b00800 { + compatible = "rockchip,rk3576-iommu", "rockchip,rk3568-iommu"; + reg = <0x0 0x27b00800 0x0 0x40>, <0x0 0x27b00900 0x0 0x40>; + interrupts = <GIC_SPI 309 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&cru CLK_RKVDEC_CORE>, <&cru HCLK_RKVDEC>; + clock-names = "aclk", "iface"; + power-domains = <&power RK3576_PD_VDEC>; + rockchip,disable-mmu-reset; + #iommu-cells = <0>; + }; + vop: vop@27d00000 { compatible = "rockchip,rk3576-vop"; reg = <0x0 0x27d00000 0x0 0x3000>, <0x0 0x27d05000 0x0 0x1000>;
@@ -2412,6 +2447,7 @@ sram: sram@3ff88000 { /* start address and size should be 4k align */ rkvdec_sram: rkvdec-sram@0 { reg = <0x0 0x78000>; + pool; }; };
Signature · 1 line
2.50.0
Detlev Casanova
maintainerPATCH 5/8 media: uapi: HEVC: Add v4l2_ctrl_hevc_ext_sps_rps control
In reply to Detlev CasanovaSome hardware (e.g.: Rockchip's rk3588 hevc decoder) need the parsed long and short term reference information for HEVC decoding.
--- .../media/v4l/ext-ctrls-codec-stateless.rst | 73 +++++++++++++++++++ .../media/v4l/vidioc-queryctrl.rst | 6 ++ 2 files changed, 79 insertions(+)
Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst +73 -0
@@ -2958,6 +2958,79 @@ This structure contains all loop filter related parameters. See sections - 0x00000004 - +.. c:type:: v4l2_ctrl_hevc_ext_sps_rps + +.. cssclass:: longtable + +.. flat-table:: struct v4l2_ctrl_hevc_ext_sps_rps + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __u16 + - ``lt_ref_pic_poc_lsb_sps`` + - Long term reference picture order count as described in section 7.4.3.2.1 + "General sequence parameter set RBSP semantics" of the specification. + * - __u8 + - ``delta_idx_minus1`` + - Specifies the delta compare to the index. See details in section 7.4.8 "Short-term + reference picture set semantics" of the specification. + * - __u8 + - ``delta_rps_sign`` + - Sign of the delta as specified in section 7.4.8 "Short-term reference picture set + semantics" of the specification. + * - __u16 + - ``abs_delta_rps_minus1`` + - Absolute delta RPS as specified in section 7.4.8 "Short-term reference picture set + semantics" of the specification. + * - __u8 + - ``num_negative_pics`` + - Number of short-term RPS entries that have picture order count values less than the + picture order count value of the current picture. + * - __u8 + - ``num_positive_pics`` + - Number of short-term RPS entries that have picture order count values greater than the + picture order count value of the current picture. + * - __u8 + - ``used_by_curr_pic_s0[16]`` + - Specifies if short-term RPS i (that has a negative POC) is used by the current picture. + * - __u8 + - ``used_by_curr_pic_s1[16]`` + - Specifies if short-term RPS i (that has a positive POC) is used by the current picture. + * - __s32 + - ``delta_poc_s0[16]`` + - Specifies the negative picture order count delta for the i-th entry in the short-term RPS. + See details in section 7.4.8 "Short-term reference picture set semantics" of the + specification. + * - __s32 + - ``delta_poc_s1[16]`` + - Specifies the positive picture order count delta for the i-th entry in the short-term RPS. + See details in section 7.4.8 "Short-term reference picture set semantics" of the + specification. + * - __u8 + - ``flags`` + - See :ref:`Extended RPS Flags <hevc_ext_sps_rps_flags>` + +.. _hevc_ext_sps_rps_flags: + +``Extended SPS RPS Flags`` + +.. cssclass:: longtable + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - ``V4L2_HEVC_EXT_SPS_RPS_FLAG_USED_LT`` + - 0x00000001 + - Specifies if the long-term reference picture is used 7.4.3.2.1 "General sequence parameter + set RBSP semantics" of the specification. + * - ``V4L2_HEVC_EXT_SPS_RPS_FLAG_INTER_REF_PIC_SET_PRED`` + - 0x00000002 + - Specifies if the short-term RPS is predicted from another short term RPS. See details in + section 7.4.8 "Short-term reference picture set semantics" of the specification. + .. _v4l2-codec-stateless-av1: ``V4L2_CID_STATELESS_AV1_SEQUENCE (struct)``
Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst +6 -0
@@ -523,6 +523,12 @@ See also the examples in :ref:`control`. - n/a - A struct :c:type:`v4l2_ctrl_hevc_decode_params`, containing HEVC decoding parameters for stateless video decoders. + * - ``V4L2_CTRL_TYPE_HEVC_EXT_SPS_RPS`` + - n/a + - n/a + - n/a + - A struct :c:type:`v4l2_ctrl_hevc_ext_sps_rps`, containing HEVC + extended RPS for stateless video decoders. * - ``V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR`` - n/a - n/a
Signature · 1 line
2.50.0
Re: [PATCH 5/8] media: uapi: HEVC: Add v4l2_ctrl_hevc_ext_sps_rps control
Hi,
Le lundi 23 juin 2025 à 12:07 -0400, Detlev Casanova a écrit :
Quoted 18 lines
Some hardware (e.g.: Rockchip's rk3588 hevc decoder) need the parsed long and short term reference information for HEVC decoding.
Signed-off-by Detlev Casanova <[email protected]>---
.../media/v4l/ext-ctrls-codec-stateless.rst | 73 +++++++++++++++++++ .../media/v4l/vidioc-queryctrl.rst | 6 ++2 files changed, 79 insertions(+)Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst +0
-0@@ -2958,6 +2958,79 @@ This structure contains all loop filter related parameters. See sections-
0x00000004
-
+.. c:type::v4l2_ctrl_hevc_ext_sps_rps
Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst This structure contains all loop filter related parameters. See sections · line 2968
Quoted 7 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst +7
-0+ +.. cssclass:: longtable + +.. flat-table:: struct v4l2_ctrl_hevc_ext_sps_rps + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2
Something, somewhere should say this is an array.
Quoted 5 lines
+
+ * -__u16
+ - ``lt_ref_pic_poc_lsb_sps``
+ - Long term reference picture order count as described in section 7.4.3.2.1 + "General sequence parameter set RBSP semantics" of the specification.
Each LONG term reference have one of these.
Quoted 4 lines
+ * -
__u8
+ - ``delta_idx_minus1``
+ - Specifies the delta compare to the index. See details in section 7.4.8 "Short-term + reference picture set semantics" of the specification.
And each SHORT term have one of these. How can both be int he same control ? For me this could make a bit more sense with two structures:
v4l2_ctrl_hevc_ext_sps_st_rps[] v4l2_ctrl_hevc_ext_sps_lt_rps[]
Did I miss something ?
Quoted 11 lines
+ * -
__u8
+ - ``delta_rps_sign``
+ - Sign of the delta as specified in section 7.4.8 "Short-term reference picture set + semantics" of the specification.
+ * -__u16
+ - ``abs_delta_rps_minus1``
+ - Absolute delta RPS as specified in section 7.4.8 "Short-term reference picture set + semantics" of the specification.
+ * -__u8
+ - ``num_negative_pics``
+ - Number of short-term RPS entries that have picture order count values less than the
Quoted 8 lines
+ picture order count value of the current picture.
+ * -__u8
+ - ``num_positive_pics``
+ - Number of short-term RPS entries that have picture order count values greater than the + picture order count value of the current picture.
+ * -__u8
+ - ``used_by_curr_pic_s0[16]``
+ - Specifies if short-term RPS i (that has a negative POC) is used by the current picture.
Use bit field for that, a __u16 works, and one per ST.
Quoted 5 lines
+ * -
__u8
+ - ``used_by_curr_pic_s1[16]``
+ - Specifies if short-term RPS i (that has a positive POC) is used by the current picture.
Same.
Quoted 5 lines
+ * -
__s32
+ - ``delta_poc_s0[16]``
+ - Specifies the negative picture order count delta for the i-th entry in the short-term RPS. + See details in section 7.4.8 "Short-term reference picture set semantics" of the + specification.
Keep it delta_poc_s0_minus1 like int he spec, and then you can go back so u16.
Quoted 5 lines
+ * -
__s32
+ - ``delta_poc_s1[16]``
+ - Specifies the positive picture order count delta for the i-th entry in the short-term RPS. + See details in section 7.4.8 "Short-term reference picture set semantics" of the + specification.
Same.
regards,
Nicolas
Quoted 44 lines
+ * -
__u8
+ - ``flags``
+ - See :ref:Extended RPS Flags <hevc_ext_sps_rps_flags>Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst +21
-0+ +.. _hevc_ext_sps_rps_flags: + +``Extended SPS RPS Flags`` + +.. cssclass:: longtable + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - ``V4L2_HEVC_EXT_SPS_RPS_FLAG_USED_LT`` + - 0x00000001 + - Specifies if the long-term reference picture is used 7.4.3.2.1 "General sequence parameter + set RBSP semantics" of the specification. + * - ``V4L2_HEVC_EXT_SPS_RPS_FLAG_INTER_REF_PIC_SET_PRED`` + - 0x00000002 + - Specifies if the short-term RPS is predicted from another short term RPS. See details in + section 7.4.8 "Short-term reference picture set semantics" of the specification. +.. _v4l2-codec-stateless-av1:
``V4L2_CID_STATELESS_AV1_SEQUENCE (struct)``Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst +0
-0@@ -523,6 +523,12 @@ See also the examples in :ref:`control`.- n/a
- A struct :c:type:v4l2_ctrl_hevc_decode_params, containing HEVC decoding parameters for stateless video decoders.
+ * - ``V4L2_CTRL_TYPE_HEVC_EXT_SPS_RPS``
+ - n/a
+ - n/a
+ - n/a
+ - A struct :c:type:v4l2_ctrl_hevc_ext_sps_rps, containing HEVCDocumentation/userspace-api/media/v4l/vidioc-queryctrl.rst +1
-0+ extended RPS for stateless video decoders.* - ``V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR``
- n/a
- n/a
Detlev Casanova
maintainerPATCH 6/8 media: v4l2-ctrls: Add hevc_ext_sps_rps control
In reply to Detlev CasanovaThe vdpu381 decoder found on newer Rockchip SoC need the information from the long term and short term ref pic sets from the SPS.
So far, it wasn't included in the v4l2 API, so add it with a new dynamic sized control.
Each element of the array contains the long term and short term ref pic set at that index.
--- drivers/media/v4l2-core/v4l2-ctrls-core.c | 9 +++++ drivers/media/v4l2-core/v4l2-ctrls-defs.c | 5 +++ include/uapi/linux/v4l2-controls.h | 48 +++++++++++++++++++++++ include/uapi/linux/videodev2.h | 1 + 4 files changed, 63 insertions(+)
drivers/media/v4l2-core/v4l2-ctrls-core.c +9 -0
@@ -418,6 +418,9 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl)case V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS :pr_cont ("HEVC_SLICE_PARAMS" );break ; +case V4L2_CTRL_TYPE_HEVC_EXT_SPS_RPS : +pr_cont ("HEVC_EXT_SPS_RPS" ); +break ;case V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX :pr_cont ("HEVC_SCALING_MATRIX" );break ;
@@ -1173,6 +1176,9 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx,case V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS :break ; +case V4L2_CTRL_TYPE_HEVC_EXT_SPS_RPS : +break ; +case V4L2_CTRL_TYPE_HDR10_CLL_INFO :break ;
@@ -1921,6 +1927,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,case V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS : elem_size =sizeof (struct v4l2_ctrl_hevc_slice_params );break ; +case V4L2_CTRL_TYPE_HEVC_EXT_SPS_RPS : + elem_size =sizeof (struct v4l2_ctrl_hevc_ext_sps_rps ); +break ;case V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX : elem_size =sizeof (struct v4l2_ctrl_hevc_scaling_matrix );break ;
drivers/media/v4l2-core/v4l2-ctrls-defs.c +5 -0
@@ -1233,6 +1233,7 @@ const char *v4l2_ctrl_get_name(u32 id)case V4L2_CID_STATELESS_HEVC_DECODE_MODE :return "HEVC Decode Mode" ;case V4L2_CID_STATELESS_HEVC_START_CODE :return "HEVC Start Code" ;case V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS :return "HEVC Entry Point Offsets" ; +case V4L2_CID_STATELESS_HEVC_EXT_SPS_RPS :return "HEVC Extended RPS data" ;case V4L2_CID_STATELESS_AV1_SEQUENCE :return "AV1 Sequence Parameters" ;case V4L2_CID_STATELESS_AV1_TILE_GROUP_ENTRY :return "AV1 Tile Group Entry" ;case V4L2_CID_STATELESS_AV1_FRAME :return "AV1 Frame Parameters" ;
@@ -1578,6 +1579,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type, *type =V4L2_CTRL_TYPE_U32 ; *flags |=V4L2_CTRL_FLAG_DYNAMIC_ARRAY ;break ; +case V4L2_CID_STATELESS_HEVC_EXT_SPS_RPS : + *type =V4L2_CTRL_TYPE_HEVC_EXT_SPS_RPS ; + *flags |=V4L2_CTRL_FLAG_DYNAMIC_ARRAY ; +break ;case V4L2_CID_STATELESS_VP9_COMPRESSED_HDR : *type =V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR ;break ;
include/uapi/linux/v4l2-controls.h +48 -0
60 lines not on this page · raw message
include/uapi/linux/videodev2.h +1 -0
@@ -1974,6 +1974,7 @@ enum v4l2_ctrl_type {V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS =0x0272 ,V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX =0x0273 ,V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS =0x0274 , +V4L2_CTRL_TYPE_HEVC_EXT_SPS_RPS =0x0275 ,V4L2_CTRL_TYPE_AV1_SEQUENCE =0x280 , V4L2_CTRL_TYPE_AV1_TILE_GROUP_ENTRY = 0x281,
Signature · 1 line
2.50.0
Detlev Casanova
maintainerPATCH 7/8 media: rkvdec: Remove TODO file
In reply to Detlev Casanova2 items are present in the TODO file:
- HEVC support
- Evaluate adding helper for rkvdec_request_validate
Missing HEVC support is not a reason for a driver to be in staging, support for different features of the hardware can be added in drivers in the main tree.
The rkvdec_request_validate function was simplified in
commit 54676d5f5630 ("media: rkvdec: Do not require all controls to be present in every request") by not setting controls that have not changed.
As it now basically just calls vb2_request_validate(), there is no need
for a helper.
--- drivers/staging/media/rkvdec/TODO | 11 ----------- 1 file changed, 11 deletions(-) delete mode 100644 drivers/staging/media/rkvdec/TODO
drivers/staging/media/rkvdec/TODO deleted +0 -11
@@ -1,11 +0,0 @@ -* Support for HEVC is planned for this driver. - - Given the V4L controls for that CODEC will be part of - the uABI, it will be required to have the driver in staging. - - For this reason, we are keeping this driver in staging for now. - -* Evaluate introducing a helper to consolidate duplicated - code in rkvdec_request_validate and cedrus_request_validate. - The helper needs to the driver private data associated with - the videobuf2 queue, from a media request.
Signature · 1 line
2.50.0
Re: [PATCH 7/8] media: rkvdec: Remove TODO file
Le lundi 23 juin 2025 à 12:07 -0400, Detlev Casanova a écrit :
Quoted 8 lines
2 items are present in the TODO file:
- HEVC support
- Evaluate adding helper forrkvdec_request_validateMissing HEVC support is not a reason for a driver to be in staging, support for different features of the hardware can be added in drivers in the main tree.
Quoted 8 lines
The
rkvdec_request_validatefunction was simplified in
commit54676d5f5630("media: rkvdec: Do not require all controls to be present in every request")
by not setting controls that have not changed.
As it now basically just callsvb2_request_validate(), there is no need for a helper.Signed-off-by Detlev Casanova <[email protected]>
Quoted 23 lines
---
drivers/staging/media/rkvdec/TODO | 11 -----------1 file changed, 11 deletions(-)delete mode 100644
drivers/staging/media/rkvdec/TODOdrivers/staging/media/rkvdec/TODO deleted +0
-11@@ -1,11 +0,0 @@ -* Support for HEVC is planned for this driver. - - Given the V4L controls for that CODEC will be part of - the uABI, it will be required to have the driver in staging. - - For this reason, we are keeping this driver in staging for now. - -* Evaluate introducing a helper to consolidate duplicated - code in rkvdec_request_validate and cedrus_request_validate. - The helper needs to the driver private data associated with - the videobuf2 queue, from a media request.
Detlev Casanova
maintainerPATCH 8/8 media: rkvdec: Unstage the driver
In reply to Detlev CasanovaThe TODO list for unstaging being empty, the driver can now be moved to the main media folder.
Also add myself as maintainer.
---
MAINTAINERS | 8 ++++++++
drivers/media/platform/rockchip/Kconfig | 1 +
drivers/media/platform/rockchip/Makefile | 1 +
.../media => media/platform/rockchip}/rkvdec/Kconfig | 0
.../media => media/platform/rockchip}/rkvdec/Makefile | 0
.../platform/rockchip}/rkvdec/rkvdec-h264.c | 0
.../platform/rockchip}/rkvdec/rkvdec-regs.h | 0
.../media => media/platform/rockchip}/rkvdec/rkvdec-vp9.c | 0
.../media => media/platform/rockchip}/rkvdec/rkvdec.c | 0
.../media => media/platform/rockchip}/rkvdec/rkvdec.h | 0
drivers/staging/media/Kconfig | 2 --
drivers/staging/media/Makefile | 1 -
12 files changed, 10 insertions(+), 3 deletions(-)
rename drivers/{staging/media => media/platform/rockchip}/rkvdec/Kconfig (100%)
rename drivers/{staging/media => media/platform/rockchip}/rkvdec/Makefile (100%)
rename drivers/{staging/media => media/platform/rockchip}/rkvdec/rkvdec-h264.c (100%)
rename drivers/{staging/media => media/platform/rockchip}/rkvdec/rkvdec-regs.h (100%)
rename drivers/{staging/media => media/platform/rockchip}/rkvdec/rkvdec-vp9.c (100%)
rename drivers/{staging/media => media/platform/rockchip}/rkvdec/rkvdec.c (100%)
rename drivers/{staging/media => media/platform/rockchip}/rkvdec/rkvdec.h (100%)MAINTAINERS +8 -0
@@ -21466,6 +21466,14 @@ S: Maintained F: Documentation/devicetree/bindings/media/rockchip-rga.yaml F: drivers/media/platform/rockchip/rga/ +ROCKCHIP RKVDEC VIDEO DECODER DRIVER +M: Detlev Casanova <[email protected]> +L: [email protected] +L: [email protected] +S: Maintained +F: Documentation/devicetree/bindings/media/rockchip,vdec.yaml +F: drivers/media/platform/rockchip/rkvdec/ + ROCKCHIP RK3308 INTERNAL AUDIO CODEC M: Luca Ceresoli <[email protected]> S: Maintained
drivers/media/platform/rockchip/Kconfig +1 -0
@@ -4,3 +4,4 @@ comment "Rockchip media platform drivers" source "drivers/media/platform/rockchip/rga/Kconfig" source "drivers/media/platform/rockchip/rkisp1/Kconfig" +source "drivers/media/platform/rockchip/rkvdec/Kconfig"
drivers/media/platform/rockchip/Makefile +1 -0
@@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only obj-y += rga/ obj-y += rkisp1/ +obj-y += rkvdec/
drivers/media/platform/rockchip/rkvdec/Kconfig renamed from drivers/staging/media/rkvdec/Kconfig +0 -0
drivers/media/platform/rockchip/rkvdec/Makefile renamed from drivers/staging/media/rkvdec/Makefile +0 -0
drivers/media/platform/rockchip/rkvdec/rkvdec-h264.c renamed from drivers/staging/media/rkvdec/rkvdec-h264.c +0 -0
drivers/media/platform/rockchip/rkvdec/rkvdec-regs.h renamed from drivers/staging/media/rkvdec/rkvdec-regs.h +0 -0
drivers/media/platform/rockchip/rkvdec/rkvdec-vp9.c renamed from drivers/staging/media/rkvdec/rkvdec-vp9.c +0 -0
drivers/media/platform/rockchip/rkvdec/rkvdec.c renamed from drivers/staging/media/rkvdec/rkvdec.c +0 -0
drivers/media/platform/rockchip/rkvdec/rkvdec.h renamed from drivers/staging/media/rkvdec/rkvdec.h +0 -0
drivers/staging/media/Kconfig +0 -2
@@ -32,8 +32,6 @@ source "drivers/staging/media/max96712/Kconfig" source "drivers/staging/media/meson/vdec/Kconfig" -source "drivers/staging/media/rkvdec/Kconfig" - source "drivers/staging/media/starfive/Kconfig" source "drivers/staging/media/sunxi/Kconfig"
drivers/staging/media/Makefile +0 -1
@@ -4,7 +4,6 @@ obj-$(CONFIG_INTEL_ATOMISP) += atomisp/ obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx/ obj-$(CONFIG_VIDEO_MAX96712) += max96712/ obj-$(CONFIG_VIDEO_MESON_VDEC) += meson/vdec/ -obj-$(CONFIG_VIDEO_ROCKCHIP_VDEC) += rkvdec/ obj-$(CONFIG_VIDEO_STARFIVE_CAMSS) += starfive/ obj-$(CONFIG_VIDEO_SUNXI) += sunxi/ obj-$(CONFIG_VIDEO_TEGRA) += tegra-video/
Signature · 1 line
2.50.0
Re: [PATCH 8/8] media: rkvdec: Unstage the driver
Le lundi 23 juin 2025 à 12:07 -0400, Detlev Casanova a écrit :
Quoted 6 lines
The TODO list for unstaging being empty, the driver can now be moved to the main media folder.
Also add myself as maintainer.
Signed-off-by Detlev Casanova <[email protected]>
Quoted 128 lines
---
MAINTAINERS | 8 ++++++++ drivers/media/platform/rockchip/Kconfig | 1 + drivers/media/platform/rockchip/Makefile | 1 + .../media => media/platform/rockchip}/rkvdec/Kconfig | 0 .../media => media/platform/rockchip}/rkvdec/Makefile | 0 .../platform/rockchip}/rkvdec/rkvdec-h264.c | 0 .../platform/rockchip}/rkvdec/rkvdec-regs.h | 0 .../media =>media/platform/rockchip}/rkvdec/rkvdec-vp9.c| 0 .../media => media/platform/rockchip}/rkvdec/rkvdec.c | 0 .../media => media/platform/rockchip}/rkvdec/rkvdec.h | 0 drivers/staging/media/Kconfig | 2 -- drivers/staging/media/Makefile | 1 -12 files changed, 10 insertions(+), 3 deletions(-)rename
drivers/{staging/media=> media/platform/rockchip}/rkvdec/Kconfig (100%)
renamedrivers/{staging/media=> media/platform/rockchip}/rkvdec/Makefile (100%)
renamedrivers/{staging/media=> media/platform/rockchip}/rkvdec/rkvdec-h264.c(100%)
renamedrivers/{staging/media=> media/platform/rockchip}/rkvdec/rkvdec-regs.h(100%)
renamedrivers/{staging/media=>media/platform/rockchip}/rkvdec/rkvdec-vp9.c(100%)
renamedrivers/{staging/media=>media/platform/rockchip}/rkvdec/rkvdec.c(100%)
renamedrivers/{staging/media=>media/platform/rockchip}/rkvdec/rkvdec.h(100%)MAINTAINERS +0
-0@@ -21466,6 +21466,14 @@ S: MaintainedF:
Documentation/devicetree/bindings/media/rockchip-rga.yaml
F:drivers/media/platform/rockchip/rga/MAINTAINERS +8
-0+ROCKCHIP RKVDEC VIDEO DECODER DRIVER +M: Detlev Casanova <[email protected]> +L: [email protected] +L: [email protected] +S: Maintained +F: Documentation/devicetree/bindings/media/rockchip,vdec.yaml +F: drivers/media/platform/rockchip/rkvdec/ +ROCKCHIP RK3308 INTERNAL AUDIO CODEC
M: Luca Ceresoli <[email protected]>
S: Maintaineddrivers/media/platform/rockchip/Kconfig +0
-0@@ -4,3 +4,4 @@ comment "Rockchip media platform drivers"
source "drivers/media/platform/rockchip/rga/Kconfig" source "drivers/media/platform/rockchip/rkisp1/Kconfig"drivers/media/platform/rockchip/Kconfig +1
-0+source "drivers/media/platform/rockchip/rkvdec/Kconfig"drivers/media/platform/rockchip/Makefile +0
-0@@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-onlyobj-y += rga/
obj-y += rkisp1/drivers/media/platform/rockchip/Makefile +1
-0+obj-y += rkvdec/drivers/media/platform/rockchip/rkvdec/Kconfig renamed from drivers/staging/media/rkvdec/Kconfig +0
-0drivers/media/platform/rockchip/rkvdec/Makefile renamed from drivers/staging/media/rkvdec/Makefile +0
-0drivers/media/platform/rockchip/rkvdec/rkvdec-h264.c renamed from drivers/staging/media/rkvdec/rkvdec-h264.c +0
-0drivers/media/platform/rockchip/rkvdec/rkvdec-regs.h renamed from drivers/staging/media/rkvdec/rkvdec-regs.h +0
-0drivers/media/platform/rockchip/rkvdec/rkvdec-vp9.c renamed from drivers/staging/media/rkvdec/rkvdec-vp9.c +0
-0drivers/media/platform/rockchip/rkvdec/rkvdec.c renamed from drivers/staging/media/rkvdec/rkvdec.c +0
-0drivers/media/platform/rockchip/rkvdec/rkvdec.h renamed from drivers/staging/media/rkvdec/rkvdec.h +0
-0drivers/staging/media/Kconfig +0
-0@@ -32,8 +32,6 @@ source "drivers/staging/media/max96712/Kconfig"
source "drivers/staging/media/meson/vdec/Kconfig"drivers/staging/media/Kconfig +0
-2-source "drivers/staging/media/rkvdec/Kconfig" -source "
drivers/staging/media/starfive/Kconfig"
source "drivers/staging/media/sunxi/Kconfig"drivers/staging/media/Makefile +0
-0@@ -4,7 +4,6 @@ obj-$(CONFIG_INTEL_ATOMISP) += atomisp/obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx/
obj-$(CONFIG_VIDEO_MAX96712) += max96712/
obj-$(CONFIG_VIDEO_MESON_VDEC) += meson/vdec/
-obj-$(CONFIG_VIDEO_ROCKCHIP_VDEC) += rkvdec/
obj-$(CONFIG_VIDEO_STARFIVE_CAMSS) += starfive/
obj-$(CONFIG_VIDEO_SUNXI) += sunxi/
obj-$(CONFIG_VIDEO_TEGRA) += tegra-video/
Re: [PATCH 8/8] media: rkvdec: Unstage the driver
In reply to Detlev CasanovaHi,
Quoted 4 lines · Detlev Casanova
The TODO list for unstaging being empty, the driver can now be moved to the main media folder.
Also add myself as maintainer.
I've tested this patch on a Rock64 (rk3328) and RockPro64 (rk3399) and did not notice any regressions. And I no longer see this warning:
rockchip_vdec: module is from the staging directory, the quality is unknown, you have been warned.
Cheers,
Diederik
Quoted 22 lines
Signed-off-by Detlev Casanova <[email protected]>--- MAINTAINERS | 8 ++++++++ drivers/media/platform/rockchip/Kconfig | 1 + drivers/media/platform/rockchip/Makefile | 1 + .../media => media/platform/rockchip}/rkvdec/Kconfig | 0 .../media => media/platform/rockchip}/rkvdec/Makefile | 0 .../platform/rockchip}/rkvdec/rkvdec-h264.c | 0 .../platform/rockchip}/rkvdec/rkvdec-regs.h | 0 .../media => media/platform/rockchip}/rkvdec/rkvdec-vp9.c | 0 .../media => media/platform/rockchip}/rkvdec/rkvdec.c | 0 .../media => media/platform/rockchip}/rkvdec/rkvdec.h | 0 drivers/staging/media/Kconfig | 2 -- drivers/staging/media/Makefile | 1 - 12 files changed, 10 insertions(+), 3 deletions(-) rename drivers/{staging/media => media/platform/rockchip}/rkvdec/Kconfig (100%) rename drivers/{staging/media => media/platform/rockchip}/rkvdec/Makefile (100%) rename drivers/{staging/media => media/platform/rockchip}/rkvdec/rkvdec-h264.c (100%) rename drivers/{staging/media => media/platform/rockchip}/rkvdec/rkvdec-regs.h (100%) rename drivers/{staging/media => media/platform/rockchip}/rkvdec/rkvdec-vp9.c (100%) rename drivers/{staging/media => media/platform/rockchip}/rkvdec/rkvdec.c (100%) rename drivers/{staging/media => media/platform/rockchip}/rkvdec/rkvdec.h (100%)