media: rkvdec: Add a VP9 backend
Subsystems (3)
v3 cover media: rkvdec: Add a VP9 backend
Third iteration of the VP9 stateless codec uAPI, plus support for Rockchip VDEC.
This series addresses an issue I found while testing, not sure how it slipped before! See patch 1.
Keep in mind the same issue should likely be fixed on Hantro and Cedrus. We can create a helper for it as a follow-up.
Other than that, minor documentation and smatch-detected warnings are fixed.
Thanks!
Ezequiel
Changelog
---------
v3:
* Fix documentation issues found by Hans.
* Fix smatch detected issues as pointed out by Hans.
* Added patch to fix wrong bytesused set on .buf_prepare.
v2:
* Documentation style issues pointed out by Nicolas internally.
* s/VP9_PROFILE_MAX/V4L2_VP9_PROFILE_MAX/
* Fix wrong kfree(ctx).
* constify a couple structs on rkvdec-vp9.c
Boris Brezillon (2):
media: uapi: Add VP9 stateless decoder controls
media: rkvdec: Add the VP9 backend
Ezequiel Garcia (1):
media: rkvdec: Fix .buf_prepare
.../userspace-api/media/v4l/biblio.rst | 10 + .../media/v4l/ext-ctrls-codec.rst | 581 ++++++ drivers/media/v4l2-core/v4l2-ctrls.c | 242 +++ drivers/media/v4l2-core/v4l2-ioctl.c | 1 + drivers/staging/media/rkvdec/Makefile | 2 +- drivers/staging/media/rkvdec/rkvdec-vp9.c | 1577 +++++++++++++++++ drivers/staging/media/rkvdec/rkvdec.c | 66 +- drivers/staging/media/rkvdec/rkvdec.h | 6 + include/media/v4l2-ctrls.h | 1 + include/media/vp9-ctrls.h | 510 ++++++ 10 files changed, 2991 insertions(+), 5 deletions(-) create mode 100644 drivers/staging/media/rkvdec/rkvdec-vp9.c create mode 100644 include/media/vp9-ctrls.h
Signature · 1 line
2.26.0.rc2
PATCH v3 1/3 media: rkvdec: Fix .buf_prepare
The driver should only set the payload on .buf_prepare if the buffer is CAPTURE type, or if an OUTPUT buffer has a zeroed payload.
Fix it.
--- drivers/staging/media/rkvdec/rkvdec.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
drivers/staging/media/rkvdec/rkvdec.c +9 -1
@@ -456,7 +456,15 @@ static int rkvdec_buf_prepare(struct vb2_buffer *vb)if (vb2_plane_size (vb, i) < sizeimage)return -EINVAL ; } -vb2_set_plane_payload (vb,0 , f->fmt.pix_mp.plane_fmt[0 ].sizeimage); + +/* +* Buffer's bytesused is written by the driver for CAPTURE buffers, +* or if the application passed zero bytesused on an OUTPUT buffer. +*/ +if (!V4L2_TYPE_IS_OUTPUT (vq->type) || + (V4L2_TYPE_IS_OUTPUT (vq->type) && !vb2_get_plane_payload (vb,0 ))) +vb2_set_plane_payload (vb,0 , + f->fmt.pix_mp.plane_fmt[0 ].sizeimage);return 0 ; }
Signature · 1 line
2.26.0.rc2
Re: [PATCH v3 1/3] media: rkvdec: Fix .buf_prepare
Hi Ezequiel,
Quoted 4 lines · Ezequiel Garcia
The driver should only set the payload on .buf_prepare if the buffer is CAPTURE type, or if an OUTPUT buffer has a zeroed payload.
Thanks for the patch. Just one question below.
Where does the requirement to set OUTPUT buffer bytesused to sizeimage if the original bytesused is 0 come from?
Best regards,
Tomasz
Quoted 33 lines
Fix it.
Fixes cd33c830448ba ("media: rkvdec: Add the rkvdec driver")Signed-off-by Ezequiel Garcia <[email protected]>--- drivers/staging/media/rkvdec/rkvdec.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)drivers/staging/media/rkvdec/rkvdec.c +9
-2@@ -456,7 +456,15 @@ static int rkvdec_buf_prepare(struct vb2_buffer *vb)if (vb2_plane_size (vb, i) < sizeimage)return -EINVAL ; } -vb2_set_plane_payload (vb,0 , f->fmt.pix_mp.plane_fmt[0 ].sizeimage); + +/* +* Buffer's bytesused is written by the driver for CAPTURE buffers, +* or if the application passed zero bytesused on an OUTPUT buffer. +*/ +if (!V4L2_TYPE_IS_OUTPUT (vq->type) || + (V4L2_TYPE_IS_OUTPUT (vq->type) && !vb2_get_plane_payload (vb,0 ))) +vb2_set_plane_payload (vb,0 , + f->fmt.pix_mp.plane_fmt[0 ].sizeimage);return 0 ; } --2.26.0.rc2
Re: [PATCH v3 1/3] media: rkvdec: Fix .buf_prepare
Quoted 12 lines · Tomasz Figa
Hi Ezequiel,
Quoted 3 lines · Ezequiel Garcia
The driver should only set the payload on .buf_prepare if the buffer is CAPTURE type, or if an OUTPUT buffer has a zeroed payload.
Thanks for the patch. Just one question below.
Where does the requirement to set OUTPUT buffer bytesused to sizeimage if the original bytesused is 0 come from?
If I'm reading english correctly, it's here:
https://www.kernel.org/doc/html/latest/media/uapi/v4l/buffer.html
"""
The number of bytes occupied by the data in the buffer. It depends on the negotiated data format and may change with each buffer for compressed variable size data like JPEG images. Drivers must set this field when type refers to a capture stream, applications when it refers to an output stream. If the application sets this to 0 for an output stream, then bytesused will be set to the size of the buffer (see the length field of this struct) by the driver. For multiplanar formats this field is ignored and the planes pointer is used instead.
"""
Thanks!
Ezequiel
Quoted 35 lines
Best regards,
TomaszQuoted 32 lines
Fix it.
Fixes cd33c830448ba ("media: rkvdec: Add the rkvdec driver")Signed-off-by Ezequiel Garcia <[email protected]>--- drivers/staging/media/rkvdec/rkvdec.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)drivers/staging/media/rkvdec/rkvdec.c +9
-2@@ -456,7 +456,15 @@ static int rkvdec_buf_prepare(struct vb2_buffer *vb)if (vb2_plane_size (vb, i) < sizeimage)return -EINVAL ; } -vb2_set_plane_payload (vb,0 , f->fmt.pix_mp.plane_fmt[0 ].sizeimage); + +/* +* Buffer's bytesused is written by the driver for CAPTURE buffers, +* or if the application passed zero bytesused on an OUTPUT buffer. +*/ +if (!V4L2_TYPE_IS_OUTPUT (vq->type) || + (V4L2_TYPE_IS_OUTPUT (vq->type) && !vb2_get_plane_payload (vb,0 ))) +vb2_set_plane_payload (vb,0 , + f->fmt.pix_mp.plane_fmt[0 ].sizeimage);return 0 ; } --2.26.0.rc2
Re: [PATCH v3 1/3] media: rkvdec: Fix .buf_prepare
Quoted 17 lines · Ezequiel Garcia
Quoted 12 lines · Tomasz Figa
Hi Ezequiel,
Quoted 3 lines · Ezequiel Garcia
The driver should only set the payload on .buf_prepare if the buffer is CAPTURE type, or if an OUTPUT buffer has a zeroed payload.
Thanks for the patch. Just one question below.
Where does the requirement to set OUTPUT buffer bytesused to sizeimage if the original bytesused is 0 come from?
If I'm reading english correctly, it's here:
Quoted 8 lines
https://www.kernel.org/doc/html/latest/media/uapi/v4l/buffer.html
"""
The number of bytes occupied by the data in the buffer. It depends on the negotiated data format and may change with each buffer for compressed variable size data like JPEG images. Drivers must set this field when type refers to a capture stream, applications when it refers to an output stream. If the application sets this to 0 for an output stream, then bytesused will be set to the size of the buffer (see the length field of this struct) by the driver. For multiplanar formats this field is ignored and the planes pointer is used instead.
"""
Okay, thanks. I wonder if this shouldn't be handled by the core, though. Especially given that the document refers to the length field specifically and not the sizeimage set in the format.
Best regards,
Tomasz
Re: [PATCH v3 1/3] media: rkvdec: Fix .buf_prepare
Quoted 26 lines · Tomasz Figa
Quoted 16 lines · Ezequiel Garcia
Quoted 12 lines · Tomasz Figa
Hi Ezequiel,
Quoted 3 lines · Ezequiel Garcia
The driver should only set the payload on .buf_prepare if the buffer is CAPTURE type, or if an OUTPUT buffer has a zeroed payload.
Thanks for the patch. Just one question below.
Where does the requirement to set OUTPUT buffer bytesused to sizeimage if the original bytesused is 0 come from?
If I'm reading english correctly, it's here:
Quoted 8 lines
https://www.kernel.org/doc/html/latest/media/uapi/v4l/buffer.html
"""
The number of bytes occupied by the data in the buffer. It depends on the negotiated data format and may change with each buffer for compressed variable size data like JPEG images. Drivers must set this field when type refers to a capture stream, applications when it refers to an output stream. If the application sets this to 0 for an output stream, then bytesused will be set to the size of the buffer (see the length field of this struct) by the driver. For multiplanar formats this field is ignored and the planes pointer is used instead.
"""
Quoted 4 lines
Okay, thanks. I wonder if this shouldn't be handled by the core, though. Especially given that the document refers to the length field specifically and not the sizeimage set in the format.
Yes, either core or helper, this definitely calls for a generic solution.
Ezequiel
Re: [PATCH v3 1/3] media: rkvdec: Fix .buf_prepare
Quoted 27 lines · Ezequiel Garcia
Quoted 26 lines · Tomasz Figa
Quoted 24 lines · Ezequiel Garcia
Quoted 12 lines · Tomasz Figa
Hi Ezequiel,
Quoted 3 lines · Ezequiel Garcia
The driver should only set the payload on .buf_prepare if the buffer is CAPTURE type, or if an OUTPUT buffer has a zeroed payload.
Thanks for the patch. Just one question below.
Where does the requirement to set OUTPUT buffer bytesused to sizeimage if the original bytesused is 0 come from?
If I'm reading english correctly, it's here:
https://www.kernel.org/doc/html/latest/media/uapi/v4l/buffer.html
"""
The number of bytes occupied by the data in the buffer. It depends on the negotiated data format and may change with each buffer for compressed variable size data like JPEG images. Drivers must set this field when type refers to a capture stream, applications when it refers to an output stream. If the application sets this to 0 for an output stream, then bytesused will be set to the size of the buffer (see the length field of this struct) by the driver. For multiplanar formats this field is ignored and the planes pointer is used instead.
"""
Quoted 6 lines
Quoted 4 lines
Okay, thanks. I wonder if this shouldn't be handled by the core, though. Especially given that the document refers to the length field specifically and not the sizeimage set in the format.
Yes, either core or helper, this definitely calls for a generic solution.
For the context, this is for backward compatibility. I'm not certain it make sense for new driver interface like RKVDEC. Specially that if the user did pass an empty buffer by accident, this will push garbage into the driver. That being said, it seems to match the spec.
Quoted 3 lines
Ezequiel
Re: [PATCH v3 1/3] media: rkvdec: Fix .buf_prepare
In reply to Ezequiel GarciaQuoted 21 lines · Ezequiel Garcia
The driver should only set the payload on .buf_prepare if the buffer is CAPTURE type, or if an OUTPUT buffer has a zeroed payload.
Fix it.
Fixes cd33c830448ba ("media: rkvdec: Add the rkvdec driver")Signed-off-by Ezequiel Garcia <[email protected]>--- drivers/staging/media/rkvdec/rkvdec.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)drivers/staging/media/rkvdec/rkvdec.c +0
-0@@ -456,7 +456,15 @@ static int rkvdec_buf_prepare(struct vb2_buffer *vb)if (vb2_plane_size (vb, i) < sizeimage)return -EINVAL ; }
drivers/staging/media/rkvdec/rkvdec.c static int rkvdec_buf_prepare(struct vb2_buffer *vb) · line 467
Quoted 10 lines
drivers/staging/media/rkvdec/rkvdec.c +9
-1-vb2_set_plane_payload (vb,0 , f->fmt.pix_mp.plane_fmt[0 ].sizeimage); + +/* +* Buffer's bytesused is written by the driver for CAPTURE buffers, +* or if the application passed zero bytesused on an OUTPUT buffer. +*/ +if (!V4L2_TYPE_IS_OUTPUT (vq->type) || + (V4L2_TYPE_IS_OUTPUT (vq->type) && !vb2_get_plane_payload (vb,0 ))) +vb2_set_plane_payload (vb,0 , + f->fmt.pix_mp.plane_fmt[0 ].sizeimage);
I believe the spec lacks a bit of a clarification. Converting from 0 to sizeimage should only be allowed for RAW images. So I would like to suggest to change this fix into:
- vb2_set_plane_payload(vb, 0, f->fmt.pix_mp.plane_fmt[0].sizeimage);
+
+ /* Buffer's bytesused is written by the driver for CAPTURE buffers */
+ if (!V4L2_TYPE_IS_OUTPUT(vq->type))
+ vb2_set_plane_payload(vb, 0,
+ f->fmt.pix_mp.plane_fmt[0].sizeimage);
And then we can fix the spec accordingly. Note that neighter FFMPEG or GStreamer will pass empty (zero sized) payload at the moment, and if it did, it would be a bug, and the payload should instead be ignored.
Quoted 7 lines
drivers/staging/media/rkvdec/rkvdec.c +0
-0return 0 ; }Signature · 1 line
2.26.0.rc2
PATCH v3 2/3 media: uapi: Add VP9 stateless decoder controls
In reply to Ezequiel GarciaFrom: Boris Brezillon <[email protected]>
Add the VP9 stateless decoder controls plus the documentation that goes with it.
--- .../userspace-api/media/v4l/biblio.rst | 10 + .../media/v4l/ext-ctrls-codec.rst | 581 ++++++++++++++++++ drivers/media/v4l2-core/v4l2-ctrls.c | 242 ++++++++ drivers/media/v4l2-core/v4l2-ioctl.c | 1 + include/media/v4l2-ctrls.h | 1 + include/media/vp9-ctrls.h | 510 +++++++++++++++ 6 files changed, 1345 insertions(+) create mode 100644 include/media/vp9-ctrls.h
Documentation/userspace-api/media/v4l/biblio.rst +10 -0
@@ -414,3 +414,13 @@ VP8 :title: RFC 6386: "VP8 Data Format and Decoding Guide" :author: J. Bankoski et al. + +.. _vp9: + +VP9 +=== + + +:title: VP9 Bitstream & Decoding Process Specification + +:author: Adrian Grange (Google), Peter de Rivaz (Argon Design), Jonathan Hunt (Argon Design)
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +581 -0
587 lines not on this page · raw message
drivers/media/v4l2-core/v4l2-ctrls.c +242 -0
272 lines not on this page · raw message
drivers/media/v4l2-core/v4l2-ioctl.c +1 -0
@@ -1371,6 +1371,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)case V4L2_PIX_FMT_VP8 : descr ="VP8" ;break ;case V4L2_PIX_FMT_VP8_FRAME : descr ="VP8 Frame" ;break ;case V4L2_PIX_FMT_VP9 : descr ="VP9" ;break ; +case V4L2_PIX_FMT_VP9_FRAME : descr ="VP9 Frame" ;break ;case V4L2_PIX_FMT_HEVC : descr ="HEVC" ;break ;/* aka H.265 */ case V4L2_PIX_FMT_HEVC_SLICE : descr ="HEVC Parsed Slice Data" ;break ;case V4L2_PIX_FMT_FWHT : descr ="FWHT" ;break ;/* used in vicodec */
include/media/v4l2-ctrls.h +1 -0
@@ -21,6 +21,7 @@#include <media/fwht-ctrls.h> #include <media/h264-ctrls.h> #include <media/vp8-ctrls.h> +#include <media/vp9-ctrls.h> #include <media/hevc-ctrls.h> /* forward references */
include/media/vp9-ctrls.h new file +510 -0
510 lines not on this page · raw message
Signature · 1 line
2.26.0.rc2
Re: [PATCH v3 2/3] media: uapi: Add VP9 stateless decoder controls
Quoted 46 lines · Ezequiel Garcia
From: Boris Brezillon <[email protected]>
Add the VP9 stateless decoder controls plus the documentation that goes with it.
Signed-off-by Boris Brezillon <[email protected]>Signed-off-by Ezequiel Garcia <[email protected]>--- .../userspace-api/media/v4l/biblio.rst | 10 + .../media/v4l/ext-ctrls-codec.rst | 581 ++++++++++++++++++ drivers/media/v4l2-core/v4l2-ctrls.c | 242 ++++++++ drivers/media/v4l2-core/v4l2-ioctl.c | 1 + include/media/v4l2-ctrls.h | 1 + include/media/vp9-ctrls.h | 510 +++++++++++++++ 6 files changed, 1345 insertions(+) create mode 100644 include/media/vp9-ctrls.hDocumentation/userspace-api/media/v4l/biblio.rst +10
-0@@ -414,3 +414,13 @@ VP8 :title: RFC 6386: "VP8 Data Format and Decoding Guide" :author: J. Bankoski et al. + +.. _vp9: + +VP9 +=== + + +:title: VP9 Bitstream & Decoding Process Specification + +:author: Adrian Grange (Google), Peter de Rivaz (Argon Design), Jonathan Hunt (Argon Design)Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +3
-0@@ -2668,6 +2668,587 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - - ``padding[3]`` - Applications and drivers must set this to zero. +.. _v4l2-mpeg-vp9: + +``V4L2_CID_MPEG_VIDEO_VP9_FRAME_CONTEXT(0..3) (struct)``
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst enum v4l2_mpeg_video_h264_hierarchical_coding_type - · line 2683
Quoted 10 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +10
-0+ Stores VP9 probabilities attached to a specific frame context. The VP9 + specification allows using a maximum of 4 contexts. Each frame being + decoded refers to one of those context. See section '7.1.2 Refresh + probs semantics' section of :ref:`vp9` for more details about these + contexts. + + This control is bi-directional: + + * all 4 contexts must be initialized by userspace just after the + stream is started and before the first decoding request is submitted
Missing . at the end.
Quoted 146 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +146
-0+ * the referenced context might be read by the kernel when a decoding + request is submitted, and will be updated after the decoder is done + decoding the frame if the `V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX` flag + is set. + * contexts will be read back by user space before each decoding request + to retrieve the updated probabilities. + * userspace will re-initialize the context to their default values when + a reset context is required. + + .. note:: + + This compound control is not yet part of the public kernel API and + it is expected to change. + +.. c:type:: v4l2_ctrl_vp9_frame_ctx + +.. cssclass:: longtable + +.. tabularcolumns:: |p{5.8cm}|p{4.8cm}|p{6.6cm}| + +.. flat-table:: struct v4l2_ctrl_vp9_frame_ctx + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - struct :c:type:`v4l2_vp9_probabilities` + - ``probs`` + - Structure with VP9 probabilities attached to the context. + +.. c:type:: v4l2_vp9_probabilities + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: struct v4l2_vp9_probabilities + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __u8 + - ``tx8[2][1]`` + - TX 8x8 probabilities. + * - __u8 + - ``tx16[2][2]`` + - TX 16x16 probabilities. + * - __u8 + - ``tx32[2][3]`` + - TX 32x32 probabilities. + * - __u8 + - ``coef[4][2][2][6][6][3]`` + - Coefficient probabilities. + * - __u8 + - ``skip[3]`` + - Skip probabilities. + * - __u8 + - ``inter_mode[7][3]`` + - Inter prediction mode probabilities. + * - __u8 + - ``interp_filter[4][2]`` + - Interpolation filter probabilities. + * - __u8 + - ``is_inter[4]`` + - Is inter-block probabilities. + * - __u8 + - ``comp_mode[5]`` + - Compound prediction mode probabilities. + * - __u8 + - ``single_ref[5][2]`` + - Single reference probabilities. + * - __u8 + - ``comp_mode[5]`` + - Compound reference probabilities. + * - __u8 + - ``y_mode[4][9]`` + - Y prediction mode probabilities. + * - __u8 + - ``uv_mode[10][9]`` + - UV prediction mode probabilities. + * - __u8 + - ``partition[16][3]`` + - Partition probabilities. + * - __u8 + - ``mv.joint[3]`` + - Motion vector joint probabilities. + * - __u8 + - ``mv.sign[2]`` + - Motion vector sign probabilities. + * - __u8 + - ``mv.class[2][10]`` + - Motion vector class probabilities. + * - __u8 + - ``mv.class0_bit[2]`` + - Motion vector class0 bit probabilities. + * - __u8 + - ``mv.bits[2][10]`` + - Motion vector bits probabilities. + * - __u8 + - ``mv.class0_fr[2][2][3]`` + - Motion vector class0 fractional bit probabilities. + * - __u8 + - ``mv.fr[2][3]`` + - Motion vector fractional bit probabilities. + * - __u8 + - ``mv.class0_hp[2]`` + - Motion vector class0 high precision fractional bit probabilities. + * - __u8 + - ``mv.hp[2]`` + - Motion vector high precision fractional bit probabilities. + +``V4L2_CID_MPEG_VIDEO_VP9_FRAME_DECODE_PARAMS (struct)`` + Specifies the frame parameters for the associated VP9 frame decode request. + This includes the necessary parameters for configuring a stateless hardware + decoding pipeline for VP9. The bitstream parameters are defined according + to :ref:`vp9`. + + .. note:: + + This compound control is not yet part of the public kernel API and + it is expected to change. + +.. c:type:: v4l2_ctrl_vp9_frame_decode_params + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: struct v4l2_ctrl_vp9_frame_decode_params + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __u32 + - ``flags`` + - Combination of V4L2_VP9_FRAME_FLAG_* flags. See + :c:type:`v4l2_vp9_frame_flags`. + * - __u16 + - ``compressed_header_size`` + - Compressed header size in bytes. + * - __u16 + - ``uncompressed_header_size`` + - Uncompressed header size in bytes. + * - __u8 + - ``profile`` + - VP9 profile. Can be 0, 1, 2 or 3. + * - __u8
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst enum v4l2_mpeg_video_h264_hierarchical_coding_type - · line 2839
Quoted 10 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +10
-0+ - ``reset_frame_context`` + - Frame context that should be used/updated when decoding the frame. + * - __u8 + - ``bit_depth`` + - Component depth in bits. Must be 8 for profile 0 and 1. Must 10 or 12 + for profile 2 and 3. + * - __u8 + - ``color_space`` + - Specifies the color space of the stream. See V4L2_VP9_COLOR_SPACE_* + values. See :c:type:`v4l2_vp9_color_space`
Ditto.
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst enum v4l2_mpeg_video_h264_hierarchical_coding_type - · line 2852
Quoted 13 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +13
-0+ * - __u8 + - ``interpolation_filter`` + - Specifies the filter selection used for performing inter prediction. See + :c:type:`v4l2_vp9_interpolation_filter`. + * - __u8 + - ``tile_cols_log2`` + - Specifies the base 2 logarithm of the width of each tile (where the + width is measured in units of 8x8 blocks). Shall be less than or equal + to 6. + * - __u8 + - ``tile_rows_log2`` + - Specifies the base 2 logarithm of the height of each tile (where the + height is measured in units of 8x8 blocks)
Ditto
Quoted 177 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +177
-0+ * - __u8 + - ``tx_mode`` + - Specifies the TX mode. See :c:type:`v4l2_vp9_tx_mode`. + * - __u8 + - ``reference_mode`` + - Specifies the type of inter prediction to be used. See + :c:type:`v4l2_vp9_reference_mode`. + * - __u8 + - ``padding`` + - Needed to make this struct 64 bit aligned. Shall be filled with zeros. + * - __u16 + - ``frame_width_minus_1`` + - Add 1 to get the frame width expressed in pixels. + * - __u16 + - ``frame_height_minus_1`` + - Add 1 to to get the frame height expressed in pixels. + * - __u16 + - ``frame_width_minus_1`` + - Add 1 to to get the expected render width expressed in pixels. This is + not used during the decoding process but might be used by HW scalers to + prepare a frame that's ready for scanout. + * - __u16 + - frame_height_minus_1 + - Add 1 to get the expected render height expressed in pixels. This is + not used during the decoding process but might be used by HW scalers to + prepare a frame that's ready for scanout. + * - __u64 + - ``refs[3]`` + - Array of reference frame timestamps. + * - struct :c:type:`v4l2_vp9_loop_filter` + - ``lf`` + - Loop filter parameters. See struct :c:type:`v4l2_vp9_loop_filter`. + * - struct :c:type:`v4l2_vp9_quantization` + - ``quant`` + - Quantization parameters. See :c:type:`v4l2_vp9_quantization`. + * - struct :c:type:`v4l2_vp9_segmentation` + - ``seg`` + - Segmentation parameters. See :c:type:`v4l2_vp9_segmentation`. + * - struct :c:type:`v4l2_vp9_probabilities` + - ``probs`` + - Probabilities. See :c:type:`v4l2_vp9_probabilities`. + +.. c:type:: v4l2_vp9_frame_flags + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_frame_flags + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_VP9_FRAME_FLAG_KEY_FRAME`` + - The frame is a key frame. + * - ``V4L2_VP9_FRAME_FLAG_SHOW_FRAME`` + - The frame should be displayed. + * - ``V4L2_VP9_FRAME_FLAG_ERROR_RESILIENT`` + - The decoding should be error resilient. + * - ``V4L2_VP9_FRAME_FLAG_INTRA_ONLY`` + - The frame does not reference other frames. + * - ``V4L2_VP9_FRAME_FLAG_ALLOW_HIGH_PREC_MV`` + - the frame might can high precision motion vectors. + * - ``V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX`` + - Frame context should be updated after decoding. + * - ``V4L2_VP9_FRAME_FLAG_PARALLEL_DEC_MODE`` + - Parallel decoding is used. + * - ``V4L2_VP9_FRAME_FLAG_X_SUBSAMPLING`` + - Vertical subsampling is enabled. + * - ``V4L2_VP9_FRAME_FLAG_Y_SUBSAMPLING`` + - Horizontal subsampling is enabled. + * - ``V4L2_VP9_FRAME_FLAG_COLOR_RANGE_FULL_SWING`` + - The full UV range is used. + +.. c:type:: v4l2_vp9_ref_id + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_ref_id + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_REF_ID_LAST`` + - Last reference frame. + * - ``V4L2_REF_ID_GOLDEN`` + - Golden reference frame. + * - ``V4L2_REF_ID_ALTREF`` + - Alternative reference frame. + * - ``V4L2_REF_ID_CNT`` + - Number of reference frames. + +.. c:type:: v4l2_vp9_tx_mode + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_tx_mode + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_VP9_TX_MODE_ONLY_4X4`` + - Transform size is 4x4. + * - ``V4L2_VP9_TX_MODE_ALLOW_8X8`` + - Transform size can be up to 8x8. + * - ``V4L2_VP9_TX_MODE_ALLOW_16X16`` + - Transform size can be up to 16x16. + * - ``V4L2_VP9_TX_MODE_ALLOW_32X32`` + - transform size can be up to 32x32. + * - ``V4L2_VP9_TX_MODE_SELECT`` + - Bitstream contains transform size for each block. + +.. c:type:: v4l2_vp9_reference_mode + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_reference_mode + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_VP9_REF_MODE_SINGLE`` + - Indicates that all the inter blocks use only a single reference frame + to generate motion compensated prediction. + * - ``V4L2_VP9_REF_MODE_COMPOUND`` + - Requires all the inter blocks to use compound mode. Single reference + frame prediction is not allowed. + * - ``V4L2_VP9_REF_MODE_SELECT`` + - Allows each individual inter block to select between single and + compound prediction modes. + +.. c:type:: v4l2_vp9_interpolation_filter + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_interpolation_filter + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_VP9_INTERP_FILTER_8TAP`` + - Height tap filter. + * - ``V4L2_VP9_INTERP_FILTER_8TAP_SMOOTH`` + - Height tap smooth filter. + * - ``V4L2_VP9_INTERP_FILTER_8TAP_SHARP`` + - Height tap sharp filter. + * - ``V4L2_VP9_INTERP_FILTER_BILINEAR`` + - Bilinear filter. + * - ``V4L2_VP9_INTERP_FILTER_SWITCHABLE`` + - Filter selection is signaled at the block level. + +.. c:type:: v4l2_vp9_color_space + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_color_space + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_VP9_COLOR_SPACE_UNKNOWN`` + - Unknown color space. In this case the color space must be signaled + outside the VP9 bitstream. + * - ``V4L2_VP9_COLOR_SPACE_BT_601`` + - Rec. ITU-R BT.601-7 color space. + * - ``V4L2_VP9_COLOR_SPACE_BT_709`` + - Rec. ITU-R BT.709-6 color space.
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst enum v4l2_mpeg_video_h264_hierarchical_coding_type - · line 3039
Quoted 10 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +10
-0+ * - ``V4L2_VP9_COLOR_SPACE_SMPTE_170`` + - SMPTE-170 color space. + * - ``V4L2_VP9_COLOR_SPACE_SMPTE_240`` + - SMPTE-240 color space. + * - ``V4L2_VP9_COLOR_SPACE_BT_2020`` + - Rec. ITU-R BT.2020-2 color space. + * - ``V4L2_VP9_COLOR_SPACE_RESERVED`` + - Reserved. This value shall never be passed. + * - ``V4L2_VP9_COLOR_SPACE_SRGB`` + - sRGB (IEC 61966-2-1) color space.
Should this be here? We already have colorspace information in the pix format structs.
It is also unclear from reading just this bit whether e.g. V4L2_VP9_COLOR_SPACE_SRGB just describes the sRGB color primaries or also the transfer function and quantization range.
I'd like to see some more about how this is used. None of the other codec headers have this.
Quoted 60 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +60
-0+ +.. c:type:: v4l2_vp9_reset_frame_context + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_reset_frame_context + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_VP9_RESET_FRAME_CTX_NONE`` + - Do not reset any frame context. + * - ``V4L2_VP9_RESET_FRAME_CTX_SPEC`` + - Reset the frame context pointed by + :c:type:`v4l2_ctrl_vp9_frame_decode_params`.frame_context_idx. + * - ``V4L2_VP9_RESET_FRAME_CTX_ALL`` + - Reset all frame contexts. + +.. c:type:: v4l2_vp9_intra_prediction_mode + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_intra_prediction_mode + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_VP9_INTRA_PRED_DC`` + - DC intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_V`` + - Vertical intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_H`` + - Horizontal intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_D45`` + - D45 intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_D135`` + - D135 intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_D117`` + - D117 intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_D153`` + - D153 intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_D207`` + - D207 intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_D63`` + - D63 intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_TM`` + - True motion intra prediction. + +.. c:type:: v4l2_vp9_segmentation + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: struct v4l2_vp9_segmentation + :header-rows: 0
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst enum v4l2_mpeg_video_h264_hierarchical_coding_type - · line 3109
Quoted 10 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +10
-0+ :stub-columns: 0 + :widths: 1 1 2 + + * - __u8 + - ``flags`` + - Combination of V4L2_VP9_SEGMENTATION_FLAG_* flags. See + :c:type:`v4l2_vp9_segmentation_flags`. + * - __u8 + - ``tree_probs[7]`` + - Specifies the probability values to be used when decoding segment_id.
segment_id? Does this refer to a term defined in the VP9 spec?
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst enum v4l2_mpeg_video_h264_hierarchical_coding_type - · line 3113
Quoted 4 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +4
-0+ * - __u8 + - ``pred_prob[3]`` + - Specifies the probability values to be used when decoding + seg_id_predicte.
seg_id_predicte: typo? I don't see any field called seg_id_predicte.
It looks like segment_id and seg_id_predicte are C field names, but I suspect they are not. I think this should be clarified.
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst enum v4l2_mpeg_video_h264_hierarchical_coding_type - · line 3116
Quoted 3 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +3
-0+ * - __u8 + - ``padding[5]`` + - Used to align this struct on 64 bit. Shall be filled with zeros.
zeroes.
Quoted 102 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +102
-0+ * - __u8 + - ``feature_enabled[8]`` + - Bitmask defining which features are enabled in each segment. + * - __u8 + - ``feature_data[8][4]`` + - Data attached to each feature. Data entry is only valid if the feature + is enabled. + +.. c:type:: v4l2_vp9_segment_feature + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_segment_feature + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_VP9_SEGMENT_FEATURE_QP_DELTA`` + - QP delta segment feature. + * - ``V4L2_VP9_SEGMENT_FEATURE_LF`` + - Loop filter segment feature. + * - ``V4L2_VP9_SEGMENT_FEATURE_REF_FRAME`` + - Reference frame segment feature. + * - ``V4L2_VP9_SEGMENT_FEATURE_SKIP`` + - Skip segment feature. + * - ``V4L2_VP9_SEGMENT_FEATURE_CNT`` + - Number of segment features. + +.. c:type:: v4l2_vp9_segmentation_flags + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_segmentation_flags + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_VP9_SEGMENTATION_FLAG_ENABLED`` + - Indicates that this frame makes use of the segmentation tool. + * - ``V4L2_VP9_SEGMENTATION_FLAG_UPDATE_MAP`` + - Indicates that the segmentation map should be updated during the + decoding of this frame. + * - ``V4L2_VP9_SEGMENTATION_FLAG_TEMPORAL_UPDATE`` + - Indicates that the updates to the segmentation map are coded + relative to the existing segmentation map. + * - ``V4L2_VP9_SEGMENTATION_FLAG_UPDATE_DATA`` + - Indicates that new parameters are about to be specified for each + segment. + * - ``V4L2_VP9_SEGMENTATION_FLAG_ABS_OR_DELTA_UPDATE`` + - Indicates that the segmentation parameters represent the actual values + to be used. + +.. c:type:: v4l2_vp9_quantization + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: struct v4l2_vp9_quantization + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __u8 + - ``base_q_idx`` + - Indicates the base frame qindex. + * - __s8 + - ``delta_q_y_dc`` + - Indicates the Y DC quantizer relative to base_q_idx. + * - __s8 + - ``delta_q_uv_dc`` + - Indicates the UV DC quantizer relative to base_q_idx. + * - __s8 + - ``delta_q_uv_ac`` + - Indicates the UV AC quantizer relative to base_q_idx. + * - __u8 + - ``padding[4]`` + - Padding bytes used to align this struct on 64 bit. Must be set to 0. + +.. c:type:: v4l2_vp9_loop_filter + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: struct v4l2_vp9_loop_filter + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __u8 + - ``flags`` + - Combination of V4L2_VP9_LOOP_FILTER_FLAG_* flags. + See :c:type:`v4l2_vp9_loop_filter_flags`. + * - __u8 + - ``level`` + - Indicates the loop filter strength. + * - __u8
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst enum v4l2_mpeg_video_h264_hierarchical_coding_type - · line 3228
Quoted 10 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +10
-0+ - ``sharpness`` + - Indicates the sharpness level. + * - __s8 + - ``ref_deltas[4]`` + - Contains the adjustment needed for the filter level based on the chosen + reference frame. + * - __s8 + - ``mode_deltas[2]`` + - Contains the adjustment needed for the filter level based on the chosen + mode
Missing period at the end.
Quoted 71 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +23
-0+ * - __u8 + - ``lvl_lookup[8][4][2]`` + - Level lookup table. + + +.. c:type:: v4l2_vp9_loop_filter_flags + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_loop_filter_flags + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_VP9_LOOP_FILTER_FLAG_DELTA_ENABLED`` + - When set, the filter level depends on the mode and reference frame used + to predict a block. + * - ``V4L2_VP9_LOOP_FILTER_FLAG_DELTA_UPDATE`` + - When set, the bitstream contains additional syntax elements that + specify which mode and reference frame deltas are to be updated. + .. raw:: latex \normalsizedrivers/media/v4l2-core/v4l2-ctrls.c +23
-0@@ -930,6 +930,11 @@ const char *v4l2_ctrl_get_name(u32 id)case V4L2_CID_MPEG_VIDEO_VP8_PROFILE :return "VP8 Profile" ;case V4L2_CID_MPEG_VIDEO_VP9_PROFILE :return "VP9 Profile" ;case V4L2_CID_MPEG_VIDEO_VP8_FRAME_HEADER :return "VP8 Frame Header" ; +case V4L2_CID_MPEG_VIDEO_VP9_FRAME_DECODE_PARAMS :return "VP9 Frame Decode Parameters" ; +case V4L2_CID_MPEG_VIDEO_VP9_FRAME_CONTEXT (0 ):return "VP9 Frame Context 0" ; +case V4L2_CID_MPEG_VIDEO_VP9_FRAME_CONTEXT (1 ):return "VP9 Frame Context 1" ; +case V4L2_CID_MPEG_VIDEO_VP9_FRAME_CONTEXT (2 ):return "VP9 Frame Context 2" ; +case V4L2_CID_MPEG_VIDEO_VP9_FRAME_CONTEXT (3 ):return "VP9 Frame Context 3" ;/* HEVC controls */ case V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_QP :return "HEVC I-Frame QP Value" ;@@ -1403,6 +1408,15 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,case V4L2_CID_MPEG_VIDEO_VP8_FRAME_HEADER : *type =V4L2_CTRL_TYPE_VP8_FRAME_HEADER ;break ; +case V4L2_CID_MPEG_VIDEO_VP9_FRAME_DECODE_PARAMS : + *type =V4L2_CTRL_TYPE_VP9_FRAME_DECODE_PARAMS ; +break ; +case V4L2_CID_MPEG_VIDEO_VP9_FRAME_CONTEXT (0 ): +case V4L2_CID_MPEG_VIDEO_VP9_FRAME_CONTEXT (1 ): +case V4L2_CID_MPEG_VIDEO_VP9_FRAME_CONTEXT (2 ): +case V4L2_CID_MPEG_VIDEO_VP9_FRAME_CONTEXT (3 ): + *type =V4L2_CTRL_TYPE_VP9_FRAME_CONTEXT ; +break ;case V4L2_CID_MPEG_VIDEO_HEVC_SPS : *type =V4L2_CTRL_TYPE_HEVC_SPS ;break ;@@ -1703,6 +1717,222 @@ static void std_log(const struct v4l2_ctrl *ctrl)0 ; \ }) +static int +validate_vp9_lf_params (struct v4l2_vp9_loop_filter *lf) +{ +unsigned int i, j, k; + +if (lf->flags & + ~(V4L2_VP9_LOOP_FILTER_FLAG_DELTA_ENABLED | +V4L2_VP9_LOOP_FILTER_FLAG_DELTA_UPDATE )) +return -EINVAL ;
drivers/media/v4l2-core/v4l2-ctrls.c static void std_log(const struct v4l2_ctrl *ctrl) · line 1738
Quoted 10 lines
drivers/media/v4l2-core/v4l2-ctrls.c +10
-0+ +/* +* V4L2_VP9_LOOP_FILTER_FLAG_DELTA_ENABLED implies +* V4L2_VP9_LOOP_FILTER_FLAG_DELTA_UPDATE. +*/ +if (lf->flags &V4L2_VP9_LOOP_FILTER_FLAG_DELTA_UPDATE && + !(lf->flags &V4L2_VP9_LOOP_FILTER_FLAG_DELTA_ENABLED )) +return -EINVAL ; + +/* That all values are in the accepted range. */
I think that should be: /* Check that
Quoted 691 lines
drivers/media/v4l2-core/v4l2-ctrls.c +209
-0+if (lf->level >GENMASK (5 ,0 )) +return -EINVAL ; + +if (lf->sharpness >GENMASK (2 ,0 )) +return -EINVAL ; + +for (i =0 ; i <ARRAY_SIZE (lf->ref_deltas); i++) { +if (lf->ref_deltas[i] <-63 || lf->ref_deltas[i] >63 ) +return -EINVAL ; + } + +for (i =0 ; i <ARRAY_SIZE (lf->mode_deltas); i++) { +if (lf->mode_deltas[i] <-63 || lf->mode_deltas[i] >63 ) +return -EINVAL ; + } + +for (i =0 ; i <ARRAY_SIZE (lf->lvl_lookup); i++) { +for (j =0 ; j <ARRAY_SIZE (lf->lvl_lookup[0 ]); j++) { +for (k =0 ; k <ARRAY_SIZE (lf->lvl_lookup[0 ][0 ]); k++) { +if (lf->lvl_lookup[i][j][k] >63 ) +return -EINVAL ; + } + } + } + +return 0 ; +} + +static int +validate_vp9_quant_params (struct v4l2_vp9_quantization *quant) +{ +if (quant->delta_q_y_dc <-15 || quant->delta_q_y_dc >15 || + quant->delta_q_uv_dc <-15 || quant->delta_q_uv_dc >15 || + quant->delta_q_uv_ac <-15 || quant->delta_q_uv_ac >15 ) +return -EINVAL ; + +memset (quant->padding,0 ,sizeof (quant->padding)); +return 0 ; +} + +static int +validate_vp9_seg_params (struct v4l2_vp9_segmentation *seg) +{ +unsigned int i, j; + +if (seg->flags & + ~(V4L2_VP9_SEGMENTATION_FLAG_ENABLED | +V4L2_VP9_SEGMENTATION_FLAG_UPDATE_MAP | +V4L2_VP9_SEGMENTATION_FLAG_TEMPORAL_UPDATE | +V4L2_VP9_SEGMENTATION_FLAG_UPDATE_DATA | +V4L2_VP9_SEGMENTATION_FLAG_ABS_OR_DELTA_UPDATE )) +return -EINVAL ; + +/* +* V4L2_VP9_SEGMENTATION_FLAG_UPDATE_MAP and +* V4L2_VP9_SEGMENTATION_FLAG_UPDATE_DATA imply +* V4L2_VP9_SEGMENTATION_FLAG_ENABLED. +*/ +if ((seg->flags & + (V4L2_VP9_SEGMENTATION_FLAG_UPDATE_MAP | +V4L2_VP9_SEGMENTATION_FLAG_UPDATE_DATA )) && + !(seg->flags &V4L2_VP9_SEGMENTATION_FLAG_ENABLED )) +return -EINVAL ; + +/* +* V4L2_VP9_SEGMENTATION_FLAG_TEMPORAL_UPDATE implies +* V4L2_VP9_SEGMENTATION_FLAG_UPDATE_MAP. +*/ +if (seg->flags &V4L2_VP9_SEGMENTATION_FLAG_TEMPORAL_UPDATE && + !(seg->flags &V4L2_VP9_SEGMENTATION_FLAG_UPDATE_MAP )) +return -EINVAL ; + +/* +* V4L2_VP9_SEGMENTATION_FLAG_ABS_OR_DELTA_UPDATE implies +* V4L2_VP9_SEGMENTATION_FLAG_UPDATE_DATA. +*/ +if (seg->flags &V4L2_VP9_SEGMENTATION_FLAG_ABS_OR_DELTA_UPDATE && + !(seg->flags &V4L2_VP9_SEGMENTATION_FLAG_UPDATE_DATA )) +return -EINVAL ; + +for (i =0 ; i <ARRAY_SIZE (seg->feature_enabled); i++) { +if (seg->feature_enabled[i] & + ~(V4L2_VP9_SEGMENT_FEATURE_QP_DELTA | +V4L2_VP9_SEGMENT_FEATURE_LF | +V4L2_VP9_SEGMENT_FEATURE_REF_FRAME | +V4L2_VP9_SEGMENT_FEATURE_SKIP )) +return -EINVAL ; + } + +for (i =0 ; i <ARRAY_SIZE (seg->feature_data); i++) { +const int range[] = {255 ,63 ,3 ,0 }; + +for (j =0 ; j <ARRAY_SIZE (seg->feature_data[j]); j++) { +if (seg->feature_data[i][j] < -range[j] || + seg->feature_data[i][j] > range[j]) +return -EINVAL ; + } + } + +memset (seg->padding,0 ,sizeof (seg->padding)); +return 0 ; +} + +static int +validate_vp9_frame_decode_params (struct v4l2_ctrl_vp9_frame_decode_params *dec_params) +{ +int ret; + +/* Make sure we're not passed invalid flags. */ +if (dec_params->flags & + ~(V4L2_VP9_FRAME_FLAG_KEY_FRAME | +V4L2_VP9_FRAME_FLAG_SHOW_FRAME | +V4L2_VP9_FRAME_FLAG_ERROR_RESILIENT | +V4L2_VP9_FRAME_FLAG_INTRA_ONLY | +V4L2_VP9_FRAME_FLAG_ALLOW_HIGH_PREC_MV | +V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX | +V4L2_VP9_FRAME_FLAG_PARALLEL_DEC_MODE | +V4L2_VP9_FRAME_FLAG_X_SUBSAMPLING | +V4L2_VP9_FRAME_FLAG_Y_SUBSAMPLING | +V4L2_VP9_FRAME_FLAG_COLOR_RANGE_FULL_SWING )) +return -EINVAL ; + +/* +* The refresh context and error resilient flags are mutually exclusive. +* Same goes for parallel decoding and error resilient modes. +*/ +if (dec_params->flags &V4L2_VP9_FRAME_FLAG_ERROR_RESILIENT && + dec_params->flags & + (V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX | +V4L2_VP9_FRAME_FLAG_PARALLEL_DEC_MODE )) +return -EINVAL ; + +if (dec_params->profile >V4L2_VP9_PROFILE_MAX ) +return -EINVAL ; + +if (dec_params->reset_frame_context >V4L2_VP9_RESET_FRAME_CTX_ALL ) +return -EINVAL ; + +if (dec_params->frame_context_idx >=V4L2_VP9_NUM_FRAME_CTX ) +return -EINVAL ; + +/* +* Profiles 0 and 1 only support 8-bit depth, profiles 2 and 3 only 10 +* and 12 bit depths. +*/ +if ((dec_params->profile <2 && dec_params->bit_depth !=8 ) || + (dec_params->profile >=2 && + (dec_params->bit_depth !=10 && dec_params->bit_depth !=12 ))) +return -EINVAL ; + +/* Profile 0 and 2 only accept YUV 4:2:0. */ +if ((dec_params->profile ==0 || dec_params->profile ==2 ) && + (!(dec_params->flags &V4L2_VP9_FRAME_FLAG_X_SUBSAMPLING ) || + !(dec_params->flags &V4L2_VP9_FRAME_FLAG_Y_SUBSAMPLING ))) +return -EINVAL ; + +/* Profile 1 and 3 only accept YUV 4:2:2, 4:4:0 and 4:4:4. */ +if ((dec_params->profile ==1 || dec_params->profile ==3 ) && + ((dec_params->flags &V4L2_VP9_FRAME_FLAG_X_SUBSAMPLING ) && + (dec_params->flags &V4L2_VP9_FRAME_FLAG_Y_SUBSAMPLING ))) +return -EINVAL ; + +if (dec_params->color_space >V4L2_VP9_COLOR_SPACE_SRGB ) +return -EINVAL ; + +if (dec_params->interpolation_filter >V4L2_VP9_INTERP_FILTER_SWITCHABLE ) +return -EINVAL ; + +/* +* According to the spec, tile_cols_log2 shall be less than or equal +* to 6. +*/ +if (dec_params->tile_cols_log2 >6 ) +return -EINVAL ; + +if (dec_params->tx_mode >V4L2_VP9_TX_MODE_SELECT ) +return -EINVAL ; + +if (dec_params->reference_mode >V4L2_VP9_REF_MODE_SELECT ) +return -EINVAL ; + + ret =validate_vp9_lf_params (&dec_params->lf); +if (ret) +return ret; + + ret =validate_vp9_quant_params (&dec_params->quant); +if (ret) +return ret; + + ret =validate_vp9_seg_params (&dec_params->seg); +if (ret) +return ret; + +memset (dec_params->padding,0 ,sizeof (dec_params->padding)); +return 0 ; +} +/* Validate a new control */ #define zero_padding (s) \@@ -1799,6 +2029,12 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx,zero_padding (p_vp8_frame_header->coder_state);break ; +case V4L2_CTRL_TYPE_VP9_FRAME_DECODE_PARAMS : +return validate_vp9_frame_decode_params (p); + +case V4L2_CTRL_TYPE_VP9_FRAME_CONTEXT : +break ; +case V4L2_CTRL_TYPE_HEVC_SPS : p_hevc_sps = p;@@ -2542,6 +2778,12 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,case V4L2_CTRL_TYPE_VP8_FRAME_HEADER : elem_size =sizeof (struct v4l2_ctrl_vp8_frame_header );break ; +case V4L2_CTRL_TYPE_VP9_FRAME_CONTEXT : + elem_size =sizeof (struct v4l2_ctrl_vp9_frame_ctx ); +break ; +case V4L2_CTRL_TYPE_VP9_FRAME_DECODE_PARAMS : + elem_size =sizeof (struct v4l2_ctrl_vp9_frame_decode_params ); +break ;case V4L2_CTRL_TYPE_HEVC_SPS : elem_size =sizeof (struct v4l2_ctrl_hevc_sps );break ;drivers/media/v4l2-core/v4l2-ioctl.c +1
-0@@ -1371,6 +1371,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)case V4L2_PIX_FMT_VP8 : descr ="VP8" ;break ;case V4L2_PIX_FMT_VP8_FRAME : descr ="VP8 Frame" ;break ;case V4L2_PIX_FMT_VP9 : descr ="VP9" ;break ; +case V4L2_PIX_FMT_VP9_FRAME : descr ="VP9 Frame" ;break ;case V4L2_PIX_FMT_HEVC : descr ="HEVC" ;break ;/* aka H.265 */ case V4L2_PIX_FMT_HEVC_SLICE : descr ="HEVC Parsed Slice Data" ;break ;case V4L2_PIX_FMT_FWHT : descr ="FWHT" ;break ;/* used in vicodec */ include/media/v4l2-ctrls.h +1
-0@@ -21,6 +21,7 @@#include <media/fwht-ctrls.h> #include <media/h264-ctrls.h> #include <media/vp8-ctrls.h> +#include <media/vp9-ctrls.h> #include <media/hevc-ctrls.h> /* forward references */ include/media/vp9-ctrls.h new file +435
-0@@ -0,0 +1,510 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* These are the VP9 state controls for use with stateless VP9 +* codec drivers. +* +* It turns out that these structs are not stable yet and will undergo +* more changes. So keep them private until they are stable and ready to +* become part of the official public API. +*/ + +#ifndef _VP9_CTRLS_H_ +#define _VP9_CTRLS_H_ + +#include <linux/types.h> + +#define V4L2_PIX_FMT_VP9_FRAME v4l2_fourcc('V', 'P', '9', 'F') + +#define V4L2_CID_MPEG_VIDEO_VP9_FRAME_CONTEXT (i) (V4L2_CID_MPEG_BASE + 4000 + (i)) +#define V4L2_CID_MPEG_VIDEO_VP9_FRAME_DECODE_PARAMS (V4L2_CID_MPEG_BASE + 4004) +#define V4L2_CTRL_TYPE_VP9_FRAME_CONTEXT 0x400 +#define V4L2_CTRL_TYPE_VP9_FRAME_DECODE_PARAMS 0x404 + +/** +* enum v4l2_vp9_loop_filter_flags - VP9 loop filter flags +* +* @V4L2_VP9_LOOP_FILTER_FLAG_DELTA_ENABLED: the filter level depends on +* the mode and reference frame used +* to predict a block +* @V4L2_VP9_LOOP_FILTER_FLAG_DELTA_UPDATE: the bitstream contains additional +* syntax elements that specify which +* mode and reference frame deltas +* are to be updated +* +* Those are the flags you should pass to &v4l2_vp9_loop_filter.flags. See +* section '7.2.8 Loop filter semantics' of the VP9 specification for more +* details. +*/ +enum v4l2_vp9_loop_filter_flags { +V4L2_VP9_LOOP_FILTER_FLAG_DELTA_ENABLED =1 <<0 , +V4L2_VP9_LOOP_FILTER_FLAG_DELTA_UPDATE =1 <<1 , +}; + +/** +* struct v4l2_vp9_loop_filter - VP9 loop filter parameters +* +* @flags: combination of V4L2_VP9_LOOP_FILTER_FLAG_* flags +* @level: indicates the loop filter strength +* @sharpness: indicates the sharpness level +* @ref_deltas: contains the adjustment needed for the filter level based on +* the chosen reference frame +* @mode_deltas: contains the adjustment needed for the filter level based on +* the chosen mode +* @lvl_lookup: level lookup table +* +* This structure contains all loop filter related parameters. See sections +* '7.2.8 Loop filter semantics' and '8.8.1 Loop filter frame init process' +* of the VP9 specification for more details. +*/ +struct v4l2_vp9_loop_filter { +__u8 flags; +__u8 level; +__u8 sharpness; +__s8 ref_deltas[4 ]; +__s8 mode_deltas[2 ]; +__u8 lvl_lookup[8 ][4 ][2 ]; +}; + +/** +* struct v4l2_vp9_quantization - VP9 quantization parameters +* +* @base_q_idx: indicates the base frame qindex +* @delta_q_y_dc: indicates the Y DC quantizer relative to base_q_idx +* @delta_q_uv_dc: indicates the UV DC quantizer relative to base_q_idx +* @delta_q_uv_ac indicates the UV AC quantizer relative to base_q_idx +* @padding: padding bytes to align things on 64 bits. Must be set to 0 +* +* Encodes the quantization parameters. See section '7.2.9 Quantization params +* syntax' of the VP9 specification for more details. +*/ +struct v4l2_vp9_quantization { +__u8 base_q_idx; +__s8 delta_q_y_dc; +__s8 delta_q_uv_dc; +__s8 delta_q_uv_ac; +__u8 padding[4 ]; +}; + +/** +* enum v4l2_vp9_segmentation_flags - VP9 segmentation flags +* +* @V4L2_VP9_SEGMENTATION_FLAG_ENABLED: indicates that this frame makes use of +* the segmentation tool +* @V4L2_VP9_SEGMENTATION_FLAG_UPDATE_MAP: indicates that the segmentation map +* should be updated during the +* decoding of this frame +* @V4L2_VP9_SEGMENTATION_FLAG_TEMPORAL_UPDATE: indicates that the updates to +* the segmentation map are coded +* relative to the existing +* segmentation map +* @V4L2_VP9_SEGMENTATION_FLAG_UPDATE_DATA: indicates that new parameters are +* about to be specified for each +* segment +* @V4L2_VP9_SEGMENTATION_FLAG_ABS_OR_DELTA_UPDATE: indicates that the +* segmentation parameters +* represent the actual values +* to be used +* +* Those are the flags you should pass to &v4l2_vp9_segmentation.flags. See +* section '7.2.10 Segmentation params syntax' of the VP9 specification for +* more details. +*/ +enum v4l2_vp9_segmentation_flags { +V4L2_VP9_SEGMENTATION_FLAG_ENABLED =1 <<0 , +V4L2_VP9_SEGMENTATION_FLAG_UPDATE_MAP =1 <<1 , +V4L2_VP9_SEGMENTATION_FLAG_TEMPORAL_UPDATE =1 <<2 , +V4L2_VP9_SEGMENTATION_FLAG_UPDATE_DATA =1 <<3 , +V4L2_VP9_SEGMENTATION_FLAG_ABS_OR_DELTA_UPDATE =1 <<4 , +}; + +#define V4L2_VP9_SEGMENT_FEATURE_ENABLED (id) (1 << (id)) +#define V4L2_VP9_SEGMENT_FEATURE_ENABLED_MASK 0xf + +/** +* enum v4l2_vp9_segment_feature - VP9 segment feature IDs +* +* @V4L2_VP9_SEGMENT_FEATURE_QP_DELTA: QP delta segment feature +* @V4L2_VP9_SEGMENT_FEATURE_LF: loop filter segment feature +* @V4L2_VP9_SEGMENT_FEATURE_REF_FRAME: reference frame segment feature +* @V4L2_VP9_SEGMENT_FEATURE_SKIP: skip segment feature +* @V4L2_VP9_SEGMENT_FEATURE_CNT: number of segment features +* +* Segment feature IDs. See section '7.2.10 Segmentation params syntax' of the +* VP9 specification for more details. +*/ +enum v4l2_vp9_segment_feature { +V4L2_VP9_SEGMENT_FEATURE_QP_DELTA , +V4L2_VP9_SEGMENT_FEATURE_LF , +V4L2_VP9_SEGMENT_FEATURE_REF_FRAME , +V4L2_VP9_SEGMENT_FEATURE_SKIP , +V4L2_VP9_SEGMENT_FEATURE_CNT , +}; + +/** +* struct v4l2_vp9_segmentation - VP9 segmentation parameters +* +* @flags: combination of V4L2_VP9_SEGMENTATION_FLAG_* flags +* @tree_probs: specify the probability values to be used when decoding +* segment_id +* @pred_prob: specify the probability values to be used when decoding +* seg_id_predicte +* @padding: padding used to make things aligned on 64 bits. Shall be zero +* filled +* @feature_enabled: bitmask defining which features are enabled in each +* segment +* @feature_data: data attached to each feature. Data entry is only valid if +* the feature is enabled +* +* Encodes the quantization parameters. See section '7.2.10 Segmentation +* params syntax' of the VP9 specification for more details. +*/ +struct v4l2_vp9_segmentation { +__u8 flags; +__u8 tree_probs[7 ]; +__u8 pred_probs[3 ]; +__u8 padding[5 ]; +__u8 feature_enabled[8 ]; +__s16 feature_data[8 ][4 ]; +}; + +/** +* enum v4l2_vp9_intra_prediction_mode - VP9 Intra prediction modes +* +* @V4L2_VP9_INTRA_PRED_DC: DC intra prediction +* @V4L2_VP9_INTRA_PRED_MODE_V: vertical intra prediction +* @V4L2_VP9_INTRA_PRED_MODE_H: horizontal intra prediction +* @V4L2_VP9_INTRA_PRED_MODE_D45: D45 intra prediction +* @V4L2_VP9_INTRA_PRED_MODE_D135: D135 intra prediction +* @V4L2_VP9_INTRA_PRED_MODE_D117: D117 intra prediction +* @V4L2_VP9_INTRA_PRED_MODE_D153: D153 intra prediction +* @V4L2_VP9_INTRA_PRED_MODE_D207: D207 intra prediction +* @V4L2_VP9_INTRA_PRED_MODE_D63: D63 intra prediction +* @V4L2_VP9_INTRA_PRED_MODE_TM: True Motion intra prediction +* +* See section '7.4.5 Intra frame mode info semantics' for more details. +*/ +enum v4l2_vp9_intra_prediction_mode { +V4L2_VP9_INTRA_PRED_MODE_DC , +V4L2_VP9_INTRA_PRED_MODE_V , +V4L2_VP9_INTRA_PRED_MODE_H , +V4L2_VP9_INTRA_PRED_MODE_D45 , +V4L2_VP9_INTRA_PRED_MODE_D135 , +V4L2_VP9_INTRA_PRED_MODE_D117 , +V4L2_VP9_INTRA_PRED_MODE_D153 , +V4L2_VP9_INTRA_PRED_MODE_D207 , +V4L2_VP9_INTRA_PRED_MODE_D63 , +V4L2_VP9_INTRA_PRED_MODE_TM , +}; + +/** +* struct v4l2_vp9_probabilities - VP9 Probabilities +* +* @tx8: TX 8x8 probabilities +* @tx16: TX 16x16 probabilities +* @tx32: TX 32x32 probabilities +* @coef: coefficient probabilities +* @skip: skip probabilities +* @inter_mode: inter mode probabilities +* @interp_filter: interpolation filter probabilities +* @is_inter: is inter-block probabilities +* @comp_mode: compound prediction mode probabilities +* @single_ref: single ref probabilities +* @comp_ref: compound ref probabilities +* @y_mode: Y prediction mode probabilities +* @uv_mode: UV prediction mode probabilities +* @partition: partition probabilities +* @mv.joint: motion vector joint probabilities +* @mv.sign: motion vector sign probabilities +* @mv.class: motion vector class probabilities +* @mv.class0_bit: motion vector class0 bit probabilities +* @mv.bits: motion vector bits probabilities +* @mv.class0_fr: motion vector class0 fractional bit probabilities +* @mv.fr: motion vector fractional bit probabilities +* @mv.class0_hp: motion vector class0 high precision fractional bit +* probabilities +* @mv.hp: motion vector high precision fractional bit probabilities +* @mv: motion vector probabilities +* +* Structure containing most VP9 probabilities. See the VP9 specification +* for more details. +*/ +struct v4l2_vp9_probabilities { +__u8 tx8[2 ][1 ]; +__u8 tx16[2 ][2 ]; +__u8 tx32[2 ][3 ]; +__u8 coef[4 ][2 ][2 ][6 ][6 ][3 ]; +__u8 skip[3 ]; +__u8 inter_mode[7 ][3 ]; +__u8 interp_filter[4 ][2 ]; +__u8 is_inter[4 ]; +__u8 comp_mode[5 ]; +__u8 single_ref[5 ][2 ]; +__u8 comp_ref[5 ]; +__u8 y_mode[4 ][9 ]; +__u8 uv_mode[10 ][9 ]; +__u8 partition[16 ][3 ]; +struct { +__u8 joint[3 ]; +__u8 sign[2 ]; +__u8 class[2 ][10 ]; +__u8 class0_bit[2 ]; +__u8 bits[2 ][10 ]; +__u8 class0_fr[2 ][2 ][3 ]; +__u8 fr[2 ][3 ]; +__u8 class0_hp[2 ]; +__u8 hp[2 ]; + } mv; +}; + +/** +* enum v4l2_vp9_reset_frame_context - Valid values for +* &v4l2_ctrl_vp9_frame_decode_params->reset_frame_context +* +* @V4L2_VP9_RESET_FRAME_CTX_NONE: don't reset any frame context +* @V4L2_VP9_RESET_FRAME_CTX_SPEC: reset the frame context pointed by +* &v4l2_ctrl_vp9_frame_decode_params.frame_context_idx +* @V4L2_VP9_RESET_FRAME_CTX_ALL: reset all frame contexts +* +* See section '7.2 Uncompressed header semantics' of the VP9 specification +* for more details. +*/ +enum v4l2_vp9_reset_frame_context { +V4L2_VP9_RESET_FRAME_CTX_NONE , +V4L2_VP9_RESET_FRAME_CTX_SPEC , +V4L2_VP9_RESET_FRAME_CTX_ALL , +}; + +/** +* enum v4l2_vp9_color_space - Valid values for +* &v4l2_ctrl_vp9_frame_decode_params->color_space +* +* @V4L2_VP9_COLOR_SPACE_UNKNOWN: unknown color space. In this case the color +* space must be signaled outside the VP9 +* bitstream +* @V4L2_VP9_COLOR_SPACE_BT_601: Rec. ITU-R BT.601-7 +* @V4L2_VP9_COLOR_SPACE_BT_709: Rec. ITU-R BT.709-6 +* @V4L2_VP9_COLOR_SPACE_SMPTE_170: SMPTE-170 +* @V4L2_VP9_COLOR_SPACE_SMPTE_240: SMPTE-240 +* @V4L2_VP9_COLOR_SPACE_BT_2020: Rec. ITU-R BT.2020-2 +* @V4L2_VP9_COLOR_SPACE_RESERVED: reserved. This value should never be passed +* @V4L2_VP9_COLOR_SPACE_SRGB: sRGB (IEC 61966-2-1) +* +* See section '7.2.2 Color config semantics' of the VP9 specification for more +* details. +*/ +enum v4l2_vp9_color_space { +V4L2_VP9_COLOR_SPACE_UNKNOWN , +V4L2_VP9_COLOR_SPACE_BT_601 , +V4L2_VP9_COLOR_SPACE_BT_709 , +V4L2_VP9_COLOR_SPACE_SMPTE_170 , +V4L2_VP9_COLOR_SPACE_SMPTE_240 , +V4L2_VP9_COLOR_SPACE_BT_2020 , +V4L2_VP9_COLOR_SPACE_RESERVED , +V4L2_VP9_COLOR_SPACE_SRGB , +}; + +/** +* enum v4l2_vp9_interpolation_filter - VP9 interpolation filter types +* +* @V4L2_VP9_INTERP_FILTER_8TAP: height tap filter +* @V4L2_VP9_INTERP_FILTER_8TAP_SMOOTH: height tap smooth filter +* @V4L2_VP9_INTERP_FILTER_8TAP_SHARP: height tap sharp filter +* @V4L2_VP9_INTERP_FILTER_BILINEAR: bilinear filter +* @V4L2_VP9_INTERP_FILTER_SWITCHABLE: filter selection is signaled at the +* block level +* +* See section '7.2.7 Interpolation filter semantics' of the VP9 specification +* for more details. +*/ +enum v4l2_vp9_interpolation_filter { +V4L2_VP9_INTERP_FILTER_8TAP , +V4L2_VP9_INTERP_FILTER_8TAP_SMOOTH , +V4L2_VP9_INTERP_FILTER_8TAP_SHARP , +V4L2_VP9_INTERP_FILTER_BILINEAR , +V4L2_VP9_INTERP_FILTER_SWITCHABLE , +}; + +/** +* enum v4l2_vp9_reference_mode - VP9 reference modes +* +* @V4L2_VP9_REF_MODE_SINGLE: indicates that all the inter blocks use only a +* single reference frame to generate motion +* compensated prediction +* @V4L2_VP9_REF_MODE_COMPOUND: requires all the inter blocks to use compound +* mode. Single reference frame prediction is not +* allowed +* @V4L2_VP9_REF_MODE_SELECT: allows each individual inter block to select +* between single and compound prediction modes +* +* See section '7.3.6 Frame reference mode semantics' of the VP9 specification +* for more details. +*/ +enum v4l2_vp9_reference_mode { +V4L2_VP9_REF_MODE_SINGLE , +V4L2_VP9_REF_MODE_COMPOUND , +V4L2_VP9_REF_MODE_SELECT , +}; + +/** +* enum v4l2_vp9_tx_mode - VP9 TX modes +* +* @V4L2_VP9_TX_MODE_ONLY_4X4: transform size is 4x4 +* @V4L2_VP9_TX_MODE_ALLOW_8X8: transform size can be up to 8x8 +* @V4L2_VP9_TX_MODE_ALLOW_16X16: transform size can be up to 16x16 +* @V4L2_VP9_TX_MODE_ALLOW_32X32: transform size can be up to 32x32 +* @V4L2_VP9_TX_MODE_SELECT: bitstream contains transform size for each block +* +* See section '7.3.1 Tx mode semantics' of the VP9 specification for more +* details. +*/ +enum v4l2_vp9_tx_mode { +V4L2_VP9_TX_MODE_ONLY_4X4 , +V4L2_VP9_TX_MODE_ALLOW_8X8 , +V4L2_VP9_TX_MODE_ALLOW_16X16 , +V4L2_VP9_TX_MODE_ALLOW_32X32 , +V4L2_VP9_TX_MODE_SELECT , +}; + +/** +* enum v4l2_vp9_ref_id - VP9 Reference frame IDs +* +* @V4L2_REF_ID_LAST: last reference frame +* @V4L2_REF_ID_GOLDEN: golden reference frame +* @V4L2_REF_ID_ALTREF: alternative reference frame +* @V4L2_REF_ID_CNT: number of reference frames +* +* See section '7.4.12 Ref frames semantics' of the VP9 specification for more +* details. +*/ +enum v4l2_vp9_ref_id { +V4L2_REF_ID_LAST , +V4L2_REF_ID_GOLDEN , +V4L2_REF_ID_ALTREF , +V4L2_REF_ID_CNT , +}; + +/** +* enum v4l2_vp9_frame_flags - VP9 frame flags +* @V4L2_VP9_FRAME_FLAG_KEY_FRAME: the frame is a key frame +* @V4L2_VP9_FRAME_FLAG_SHOW_FRAME: the frame should be displayed +* @V4L2_VP9_FRAME_FLAG_ERROR_RESILIENT: the decoding should be error resilient +* @V4L2_VP9_FRAME_FLAG_INTRA_ONLY: the frame does not reference other frames +* @V4L2_VP9_FRAME_FLAG_ALLOW_HIGH_PREC_MV: the frame might can high precision +* motion vectors +* @V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX: frame context should be updated +* after decoding +* @V4L2_VP9_FRAME_FLAG_PARALLEL_DEC_MODE: parallel decoding is used +* @V4L2_VP9_FRAME_FLAG_X_SUBSAMPLING: vertical subsampling is enabled +* @V4L2_VP9_FRAME_FLAG_Y_SUBSAMPLING: horizontal subsampling is enabled +* @V4L2_VP9_FRAME_FLAG_COLOR_RANGE_FULL_SWING: full UV range is used +* +* Check the VP9 specification for more details. +*/ +enum v4l2_vp9_frame_flags { +V4L2_VP9_FRAME_FLAG_KEY_FRAME =1 <<0 , +V4L2_VP9_FRAME_FLAG_SHOW_FRAME =1 <<1 , +V4L2_VP9_FRAME_FLAG_ERROR_RESILIENT =1 <<2 , +V4L2_VP9_FRAME_FLAG_INTRA_ONLY =1 <<3 , +V4L2_VP9_FRAME_FLAG_ALLOW_HIGH_PREC_MV =1 <<4 , +V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX =1 <<5 , +V4L2_VP9_FRAME_FLAG_PARALLEL_DEC_MODE =1 <<6 , +V4L2_VP9_FRAME_FLAG_X_SUBSAMPLING =1 <<7 , +V4L2_VP9_FRAME_FLAG_Y_SUBSAMPLING =1 <<8 , +V4L2_VP9_FRAME_FLAG_COLOR_RANGE_FULL_SWING =1 <<9 , +}; + +#define V4L2_VP9_PROFILE_MAX 3 + +/** +* struct v4l2_ctrl_vp9_frame_decode_params - VP9 frame decoding control +* +* @flags: combination of V4L2_VP9_FRAME_FLAG_* flags +* @compressed_header_size: compressed header size in bytes +* @uncompressed_header_size: uncompressed header size in bytes +* @profile: VP9 profile. Can be 0, 1, 2 or 3 +* @reset_frame_context: specifies whether the frame context should be reset +* to default values. See &v4l2_vp9_reset_frame_context +* for more details +* @frame_context_idx: frame context that should be used/updated +* @bit_depth: bits per components. Can be 8, 10 or 12. Note that not all +* profiles support 10 and/or 12 bits depths +* @color_space: specifies the color space of the stream. See +* &v4l2_vp9_color_space for more details +* @interpolation_filter: specifies the filter selection used for performing +* inter prediction. See &v4l2_vp9_interpolation_filter +* for more details
include/media/vp9-ctrls.h line 445
Quoted 10 lines
include/media/vp9-ctrls.h new file +10
-0+* @tile_cols_log2: specifies the base 2 logarithm of the width of each tile +* (where the width is measured in units of 8x8 blocks). +* Shall be less than or equal to 6 +* @tile_rows_log2: specifies the base 2 logarithm of the height of each tile +* (where the height is measured in units of 8x8 blocks) +* @tx_mode: specifies the TX mode. See &v4l2_vp9_tx_mode for more details +* @reference_mode: specifies the type of inter prediction to be used. See +* &v4l2_vp9_reference_mode for more details +* @padding: needed to make this struct 64 bit aligned. Shall be filled with +* zeros
zeros -> zeroes
Quoted 56 lines
include/media/vp9-ctrls.h new file +55
-0+* @frame_width_minus_1: add 1 to it and you'll get the frame width expressed +* in pixels +* @frame_height_minus_1: add 1 to it and you'll get the frame height expressed +* in pixels +* @frame_width_minus_1: add 1 to it and you'll get the expected render width +* expressed in pixels. This is not used during the +* decoding process but might be used by HW scalers to +* prepare a frame that's ready for scanout +* @frame_height_minus_1: add 1 to it and you'll get the expected render height +* expressed in pixels. This is not used during the +* decoding process but might be used by HW scalers to +* prepare a frame that's ready for scanout +* @refs: array of reference frames. See &v4l2_vp9_ref_id for more details +* @lf: loop filter parameters. See &v4l2_vp9_loop_filter for more details +* @quant: quantization parameters. See &v4l2_vp9_quantization for more details +* @seg: segmentation parameters. See &v4l2_vp9_segmentation for more details +* @probs: probabilities. See &v4l2_vp9_probabilities for more details +*/ +struct v4l2_ctrl_vp9_frame_decode_params { +__u32 flags; +__u16 compressed_header_size; +__u16 uncompressed_header_size; +__u8 profile; +__u8 reset_frame_context; +__u8 frame_context_idx; +__u8 bit_depth; +__u8 color_space; +__u8 interpolation_filter; +__u8 tile_cols_log2; +__u8 tile_rows_log2; +__u8 tx_mode; +__u8 reference_mode; +__u8 padding[6 ]; +__u16 frame_width_minus_1; +__u16 frame_height_minus_1; +__u16 render_width_minus_1; +__u16 render_height_minus_1; +__u64 refs[V4L2_REF_ID_CNT ]; +struct v4l2_vp9_loop_filter lf; +struct v4l2_vp9_quantization quant; +struct v4l2_vp9_segmentation seg; +struct v4l2_vp9_probabilities probs; +}; + +#define V4L2_VP9_NUM_FRAME_CTX 4 + +/** +* struct v4l2_ctrl_vp9_frame_ctx - VP9 frame context control +* +* @probs: VP9 probabilities +* +* This control is accessed in both direction. The user should initialize the +* 4 contexts with default values just after starting the stream. Then before +* decoding a frame it should query the current frame context (the one passed +* through &v4l2_ctrl_vp9_frame_decode_params.frame_context_idx) to initialize
include/media/vp9-ctrls.h line 510
Quoted 10 lines
include/media/vp9-ctrls.h new file +10
-0+* &v4l2_ctrl_vp9_frame_decode_params.probs. The probs are then adjusted based +* on the bitstream info and passed to the kernel. The codec should update +* the frame context after the frame has been decoded, so that next time +* userspace query this context it contains the updated probabilities. +*/ +struct v4l2_ctrl_vp9_frame_ctx { +struct v4l2_vp9_probabilities probs; +}; + +#endif /* _VP9_CTRLS_H_ */
Regards,
Hans
Re: [PATCH v3 2/3] media: uapi: Add VP9 stateless decoder controls
Hi Hans,
Quoted 44 lines · Hans Verkuil
Quoted 43 lines · Ezequiel Garcia
From: Boris Brezillon <[email protected]>
Add the VP9 stateless decoder controls plus the documentation that goes with it.
Signed-off-by Boris Brezillon <[email protected]>Signed-off-by Ezequiel Garcia <[email protected]>--- .../userspace-api/media/v4l/biblio.rst | 10 + .../media/v4l/ext-ctrls-codec.rst | 581 ++++++++++++++++++ drivers/media/v4l2-core/v4l2-ctrls.c | 242 ++++++++ drivers/media/v4l2-core/v4l2-ioctl.c | 1 + include/media/v4l2-ctrls.h | 1 + include/media/vp9-ctrls.h | 510 +++++++++++++++ 6 files changed, 1345 insertions(+) create mode 100644 include/media/vp9-ctrls.hDocumentation/userspace-api/media/v4l/biblio.rst +10
-0@@ -414,3 +414,13 @@ VP8 :title: RFC 6386: "VP8 Data Format and Decoding Guide" :author: J. Bankoski et al. + +.. _vp9: + +VP9 +=== + + +:title: VP9 Bitstream & Decoding Process Specification + +:author: Adrian Grange (Google), Peter de Rivaz (Argon Design), Jonathan Hunt (Argon Design)Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +0
-0@@ -2668,6 +2668,587 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - - ``padding[3]`` - Applications and drivers must set this to zero.
Quoted 16 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst enum v4l2_mpeg_video_h264_hierarchical_coding_type - · line 2683
Quoted 13 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +13
-0+.. _v4l2-mpeg-vp9: + +``V4L2_CID_MPEG_VIDEO_VP9_FRAME_CONTEXT(0..3) (struct)`` + Stores VP9 probabilities attached to a specific frame context. The VP9 + specification allows using a maximum of 4 contexts. Each frame being + decoded refers to one of those context. See section '7.1.2 Refresh + probs semantics' section of :ref:`vp9` for more details about these + contexts. + + This control is bi-directional: + + * all 4 contexts must be initialized by userspace just after the + stream is started and before the first decoding request is submittedMissing . at the end.
Oops, thanks for catching this.
Quoted 365 lines
Quoted 146 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +146
-0+ * the referenced context might be read by the kernel when a decoding + request is submitted, and will be updated after the decoder is done + decoding the frame if the `V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX` flag + is set. + * contexts will be read back by user space before each decoding request + to retrieve the updated probabilities. + * userspace will re-initialize the context to their default values when + a reset context is required. + + .. note:: + + This compound control is not yet part of the public kernel API and + it is expected to change. + +.. c:type:: v4l2_ctrl_vp9_frame_ctx + +.. cssclass:: longtable + +.. tabularcolumns:: |p{5.8cm}|p{4.8cm}|p{6.6cm}| + +.. flat-table:: struct v4l2_ctrl_vp9_frame_ctx + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - struct :c:type:`v4l2_vp9_probabilities` + - ``probs`` + - Structure with VP9 probabilities attached to the context. + +.. c:type:: v4l2_vp9_probabilities + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: struct v4l2_vp9_probabilities + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __u8 + - ``tx8[2][1]`` + - TX 8x8 probabilities. + * - __u8 + - ``tx16[2][2]`` + - TX 16x16 probabilities. + * - __u8 + - ``tx32[2][3]`` + - TX 32x32 probabilities. + * - __u8 + - ``coef[4][2][2][6][6][3]`` + - Coefficient probabilities. + * - __u8 + - ``skip[3]`` + - Skip probabilities. + * - __u8 + - ``inter_mode[7][3]`` + - Inter prediction mode probabilities. + * - __u8 + - ``interp_filter[4][2]`` + - Interpolation filter probabilities. + * - __u8 + - ``is_inter[4]`` + - Is inter-block probabilities. + * - __u8 + - ``comp_mode[5]`` + - Compound prediction mode probabilities. + * - __u8 + - ``single_ref[5][2]`` + - Single reference probabilities. + * - __u8 + - ``comp_mode[5]`` + - Compound reference probabilities. + * - __u8 + - ``y_mode[4][9]`` + - Y prediction mode probabilities. + * - __u8 + - ``uv_mode[10][9]`` + - UV prediction mode probabilities. + * - __u8 + - ``partition[16][3]`` + - Partition probabilities. + * - __u8 + - ``mv.joint[3]`` + - Motion vector joint probabilities. + * - __u8 + - ``mv.sign[2]`` + - Motion vector sign probabilities. + * - __u8 + - ``mv.class[2][10]`` + - Motion vector class probabilities. + * - __u8 + - ``mv.class0_bit[2]`` + - Motion vector class0 bit probabilities. + * - __u8 + - ``mv.bits[2][10]`` + - Motion vector bits probabilities. + * - __u8 + - ``mv.class0_fr[2][2][3]`` + - Motion vector class0 fractional bit probabilities. + * - __u8 + - ``mv.fr[2][3]`` + - Motion vector fractional bit probabilities. + * - __u8 + - ``mv.class0_hp[2]`` + - Motion vector class0 high precision fractional bit probabilities. + * - __u8 + - ``mv.hp[2]`` + - Motion vector high precision fractional bit probabilities. + +``V4L2_CID_MPEG_VIDEO_VP9_FRAME_DECODE_PARAMS (struct)`` + Specifies the frame parameters for the associated VP9 frame decode request. + This includes the necessary parameters for configuring a stateless hardware + decoding pipeline for VP9. The bitstream parameters are defined according + to :ref:`vp9`. + + .. note:: + + This compound control is not yet part of the public kernel API and + it is expected to change. + +.. c:type:: v4l2_ctrl_vp9_frame_decode_params + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: struct v4l2_ctrl_vp9_frame_decode_params + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __u32 + - ``flags`` + - Combination of V4L2_VP9_FRAME_FLAG_* flags. See + :c:type:`v4l2_vp9_frame_flags`. + * - __u16 + - ``compressed_header_size`` + - Compressed header size in bytes. + * - __u16 + - ``uncompressed_header_size`` + - Uncompressed header size in bytes. + * - __u8 + - ``profile`` + - VP9 profile. Can be 0, 1, 2 or 3. + * - __u8Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst enum v4l2_mpeg_video_h264_hierarchical_coding_type - · line 2839
Quoted 10 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +10
-0+ - ``reset_frame_context`` + - Frame context that should be used/updated when decoding the frame. + * - __u8 + - ``bit_depth`` + - Component depth in bits. Must be 8 for profile 0 and 1. Must 10 or 12 + for profile 2 and 3. + * - __u8 + - ``color_space`` + - Specifies the color space of the stream. See V4L2_VP9_COLOR_SPACE_* + values. See :c:type:`v4l2_vp9_color_space`Ditto.
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst enum v4l2_mpeg_video_h264_hierarchical_coding_type - · line 2852
Quoted 13 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +13
-0+ * - __u8 + - ``interpolation_filter`` + - Specifies the filter selection used for performing inter prediction. See + :c:type:`v4l2_vp9_interpolation_filter`. + * - __u8 + - ``tile_cols_log2`` + - Specifies the base 2 logarithm of the width of each tile (where the + width is measured in units of 8x8 blocks). Shall be less than or equal + to 6. + * - __u8 + - ``tile_rows_log2`` + - Specifies the base 2 logarithm of the height of each tile (where the + height is measured in units of 8x8 blocks)Ditto
Quoted 177 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +177
-0+ * - __u8 + - ``tx_mode`` + - Specifies the TX mode. See :c:type:`v4l2_vp9_tx_mode`. + * - __u8 + - ``reference_mode`` + - Specifies the type of inter prediction to be used. See + :c:type:`v4l2_vp9_reference_mode`. + * - __u8 + - ``padding`` + - Needed to make this struct 64 bit aligned. Shall be filled with zeros. + * - __u16 + - ``frame_width_minus_1`` + - Add 1 to get the frame width expressed in pixels. + * - __u16 + - ``frame_height_minus_1`` + - Add 1 to to get the frame height expressed in pixels. + * - __u16 + - ``frame_width_minus_1`` + - Add 1 to to get the expected render width expressed in pixels. This is + not used during the decoding process but might be used by HW scalers to + prepare a frame that's ready for scanout. + * - __u16 + - frame_height_minus_1 + - Add 1 to get the expected render height expressed in pixels. This is + not used during the decoding process but might be used by HW scalers to + prepare a frame that's ready for scanout. + * - __u64 + - ``refs[3]`` + - Array of reference frame timestamps. + * - struct :c:type:`v4l2_vp9_loop_filter` + - ``lf`` + - Loop filter parameters. See struct :c:type:`v4l2_vp9_loop_filter`. + * - struct :c:type:`v4l2_vp9_quantization` + - ``quant`` + - Quantization parameters. See :c:type:`v4l2_vp9_quantization`. + * - struct :c:type:`v4l2_vp9_segmentation` + - ``seg`` + - Segmentation parameters. See :c:type:`v4l2_vp9_segmentation`. + * - struct :c:type:`v4l2_vp9_probabilities` + - ``probs`` + - Probabilities. See :c:type:`v4l2_vp9_probabilities`. + +.. c:type:: v4l2_vp9_frame_flags + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_frame_flags + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_VP9_FRAME_FLAG_KEY_FRAME`` + - The frame is a key frame. + * - ``V4L2_VP9_FRAME_FLAG_SHOW_FRAME`` + - The frame should be displayed. + * - ``V4L2_VP9_FRAME_FLAG_ERROR_RESILIENT`` + - The decoding should be error resilient. + * - ``V4L2_VP9_FRAME_FLAG_INTRA_ONLY`` + - The frame does not reference other frames. + * - ``V4L2_VP9_FRAME_FLAG_ALLOW_HIGH_PREC_MV`` + - the frame might can high precision motion vectors. + * - ``V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX`` + - Frame context should be updated after decoding. + * - ``V4L2_VP9_FRAME_FLAG_PARALLEL_DEC_MODE`` + - Parallel decoding is used. + * - ``V4L2_VP9_FRAME_FLAG_X_SUBSAMPLING`` + - Vertical subsampling is enabled. + * - ``V4L2_VP9_FRAME_FLAG_Y_SUBSAMPLING`` + - Horizontal subsampling is enabled. + * - ``V4L2_VP9_FRAME_FLAG_COLOR_RANGE_FULL_SWING`` + - The full UV range is used. + +.. c:type:: v4l2_vp9_ref_id + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_ref_id + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_REF_ID_LAST`` + - Last reference frame. + * - ``V4L2_REF_ID_GOLDEN`` + - Golden reference frame. + * - ``V4L2_REF_ID_ALTREF`` + - Alternative reference frame. + * - ``V4L2_REF_ID_CNT`` + - Number of reference frames. + +.. c:type:: v4l2_vp9_tx_mode + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_tx_mode + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_VP9_TX_MODE_ONLY_4X4`` + - Transform size is 4x4. + * - ``V4L2_VP9_TX_MODE_ALLOW_8X8`` + - Transform size can be up to 8x8. + * - ``V4L2_VP9_TX_MODE_ALLOW_16X16`` + - Transform size can be up to 16x16. + * - ``V4L2_VP9_TX_MODE_ALLOW_32X32`` + - transform size can be up to 32x32. + * - ``V4L2_VP9_TX_MODE_SELECT`` + - Bitstream contains transform size for each block. + +.. c:type:: v4l2_vp9_reference_mode + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_reference_mode + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_VP9_REF_MODE_SINGLE`` + - Indicates that all the inter blocks use only a single reference frame + to generate motion compensated prediction. + * - ``V4L2_VP9_REF_MODE_COMPOUND`` + - Requires all the inter blocks to use compound mode. Single reference + frame prediction is not allowed. + * - ``V4L2_VP9_REF_MODE_SELECT`` + - Allows each individual inter block to select between single and + compound prediction modes. + +.. c:type:: v4l2_vp9_interpolation_filter + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_interpolation_filter + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_VP9_INTERP_FILTER_8TAP`` + - Height tap filter. + * - ``V4L2_VP9_INTERP_FILTER_8TAP_SMOOTH`` + - Height tap smooth filter. + * - ``V4L2_VP9_INTERP_FILTER_8TAP_SHARP`` + - Height tap sharp filter. + * - ``V4L2_VP9_INTERP_FILTER_BILINEAR`` + - Bilinear filter. + * - ``V4L2_VP9_INTERP_FILTER_SWITCHABLE`` + - Filter selection is signaled at the block level. + +.. c:type:: v4l2_vp9_color_space + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_color_space + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_VP9_COLOR_SPACE_UNKNOWN`` + - Unknown color space. In this case the color space must be signaled + outside the VP9 bitstream. + * - ``V4L2_VP9_COLOR_SPACE_BT_601`` + - Rec. ITU-R BT.601-7 color space. + * - ``V4L2_VP9_COLOR_SPACE_BT_709`` + - Rec. ITU-R BT.709-6 color space.Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst enum v4l2_mpeg_video_h264_hierarchical_coding_type - · line 3039
Quoted 10 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +10
-0+ * - ``V4L2_VP9_COLOR_SPACE_SMPTE_170`` + - SMPTE-170 color space. + * - ``V4L2_VP9_COLOR_SPACE_SMPTE_240`` + - SMPTE-240 color space. + * - ``V4L2_VP9_COLOR_SPACE_BT_2020`` + - Rec. ITU-R BT.2020-2 color space. + * - ``V4L2_VP9_COLOR_SPACE_RESERVED`` + - Reserved. This value shall never be passed. + * - ``V4L2_VP9_COLOR_SPACE_SRGB`` + - sRGB (IEC 61966-2-1) color space.Should this be here? We already have colorspace information in the pix format structs.
Quoted 7 lines
It is also unclear from reading just this bit whether e.g.
V4L2_VP9_COLOR_SPACE_SRGBjust describes the sRGB color primaries or also the transfer function and quantization range.I'd like to see some more about how this is used. None of the other codec headers have this.
I think this comes directly from the spec. The Rockchip VDEC doesn't seem to use it.
Do you think we can drop it from here, and rely on the V4L2 colorspace passed in the format negotiation?
Quoted 64 lines
Quoted 64 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +64
-0+ +.. c:type:: v4l2_vp9_reset_frame_context + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_reset_frame_context + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_VP9_RESET_FRAME_CTX_NONE`` + - Do not reset any frame context. + * - ``V4L2_VP9_RESET_FRAME_CTX_SPEC`` + - Reset the frame context pointed by + :c:type:`v4l2_ctrl_vp9_frame_decode_params`.frame_context_idx. + * - ``V4L2_VP9_RESET_FRAME_CTX_ALL`` + - Reset all frame contexts. + +.. c:type:: v4l2_vp9_intra_prediction_mode + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_intra_prediction_mode + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_VP9_INTRA_PRED_DC`` + - DC intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_V`` + - Vertical intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_H`` + - Horizontal intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_D45`` + - D45 intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_D135`` + - D135 intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_D117`` + - D117 intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_D153`` + - D153 intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_D207`` + - D207 intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_D63`` + - D63 intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_TM`` + - True motion intra prediction. + +.. c:type:: v4l2_vp9_segmentation + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: struct v4l2_vp9_segmentation + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __u8
Quoted 9 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst enum v4l2_mpeg_video_h264_hierarchical_coding_type - · line 3109
Quoted 6 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +6
-0+ - ``flags`` + - Combination of V4L2_VP9_SEGMENTATION_FLAG_* flags. See + :c:type:`v4l2_vp9_segmentation_flags`. + * - __u8 + - ``tree_probs[7]`` + - Specifies the probability values to be used when decoding segment_id.
segment_id? Does this refer to a term defined in the VP9 spec?
Yes, this is all strictly spec compliant. segment_id and seg_id_predicted are VP9 syntax elements.
In particular that description is copy-pasted from the spec, but I agree it may seem confusing in this context. How about:
"to be used when decoding a Segment-ID"
Quoted 10 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst enum v4l2_mpeg_video_h264_hierarchical_coding_type - · line 3113
Quoted 4 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +4
-0+ * - __u8 + - ``pred_prob[3]`` + - Specifies the probability values to be used when decoding + seg_id_predicte.
seg_id_predicte: typo? I don't see any field calledseg_id_predicte.It looks like
segment_idandseg_id_predicteare C field names, but I suspect they are not. I think this should be clarified.
Ditto,
"to be used when decoding a Predicted-Segment-ID"
Maybe this will be clearer ?
Quoted 6 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst enum v4l2_mpeg_video_h264_hierarchical_coding_type - · line 3116
Quoted 3 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +3
-0+ * - __u8 + - ``padding[5]`` + - Used to align this struct on 64 bit. Shall be filled with zeros.zeroes.
Oops, more typos :-)
Thanks for reviewing,
Ezequiel
Re: [PATCH v3 2/3] media: uapi: Add VP9 stateless decoder controls
Quoted 439 lines · Ezequiel Garcia
Hi Hans,
Quoted 44 lines · Hans Verkuil
Quoted 43 lines · Ezequiel Garcia
From: Boris Brezillon <[email protected]>
Add the VP9 stateless decoder controls plus the documentation that goes with it.
Signed-off-by Boris Brezillon <[email protected]>Signed-off-by Ezequiel Garcia <[email protected]>--- .../userspace-api/media/v4l/biblio.rst | 10 + .../media/v4l/ext-ctrls-codec.rst | 581 ++++++++++++++++++ drivers/media/v4l2-core/v4l2-ctrls.c | 242 ++++++++ drivers/media/v4l2-core/v4l2-ioctl.c | 1 + include/media/v4l2-ctrls.h | 1 + include/media/vp9-ctrls.h | 510 +++++++++++++++ 6 files changed, 1345 insertions(+) create mode 100644 include/media/vp9-ctrls.hDocumentation/userspace-api/media/v4l/biblio.rst +10
-0@@ -414,3 +414,13 @@ VP8 :title: RFC 6386: "VP8 Data Format and Decoding Guide" :author: J. Bankoski et al. + +.. _vp9: + +VP9 +=== + + +:title: VP9 Bitstream & Decoding Process Specification + +:author: Adrian Grange (Google), Peter de Rivaz (Argon Design), Jonathan Hunt (Argon Design)Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +0
-0@@ -2668,6 +2668,587 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - - ``padding[3]`` - Applications and drivers must set this to zero.Quoted 16 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst enum v4l2_mpeg_video_h264_hierarchical_coding_type - · line 2683
Quoted 13 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +13
-0+.. _v4l2-mpeg-vp9: + +``V4L2_CID_MPEG_VIDEO_VP9_FRAME_CONTEXT(0..3) (struct)`` + Stores VP9 probabilities attached to a specific frame context. The VP9 + specification allows using a maximum of 4 contexts. Each frame being + decoded refers to one of those context. See section '7.1.2 Refresh + probs semantics' section of :ref:`vp9` for more details about these + contexts. + + This control is bi-directional: + + * all 4 contexts must be initialized by userspace just after the + stream is started and before the first decoding request is submittedMissing . at the end.
Oops, thanks for catching this.
Quoted 365 lines
Quoted 146 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +146
-0+ * the referenced context might be read by the kernel when a decoding + request is submitted, and will be updated after the decoder is done + decoding the frame if the `V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX` flag + is set. + * contexts will be read back by user space before each decoding request + to retrieve the updated probabilities. + * userspace will re-initialize the context to their default values when + a reset context is required. + + .. note:: + + This compound control is not yet part of the public kernel API and + it is expected to change. + +.. c:type:: v4l2_ctrl_vp9_frame_ctx + +.. cssclass:: longtable + +.. tabularcolumns:: |p{5.8cm}|p{4.8cm}|p{6.6cm}| + +.. flat-table:: struct v4l2_ctrl_vp9_frame_ctx + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - struct :c:type:`v4l2_vp9_probabilities` + - ``probs`` + - Structure with VP9 probabilities attached to the context. + +.. c:type:: v4l2_vp9_probabilities + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: struct v4l2_vp9_probabilities + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __u8 + - ``tx8[2][1]`` + - TX 8x8 probabilities. + * - __u8 + - ``tx16[2][2]`` + - TX 16x16 probabilities. + * - __u8 + - ``tx32[2][3]`` + - TX 32x32 probabilities. + * - __u8 + - ``coef[4][2][2][6][6][3]`` + - Coefficient probabilities. + * - __u8 + - ``skip[3]`` + - Skip probabilities. + * - __u8 + - ``inter_mode[7][3]`` + - Inter prediction mode probabilities. + * - __u8 + - ``interp_filter[4][2]`` + - Interpolation filter probabilities. + * - __u8 + - ``is_inter[4]`` + - Is inter-block probabilities. + * - __u8 + - ``comp_mode[5]`` + - Compound prediction mode probabilities. + * - __u8 + - ``single_ref[5][2]`` + - Single reference probabilities. + * - __u8 + - ``comp_mode[5]`` + - Compound reference probabilities. + * - __u8 + - ``y_mode[4][9]`` + - Y prediction mode probabilities. + * - __u8 + - ``uv_mode[10][9]`` + - UV prediction mode probabilities. + * - __u8 + - ``partition[16][3]`` + - Partition probabilities. + * - __u8 + - ``mv.joint[3]`` + - Motion vector joint probabilities. + * - __u8 + - ``mv.sign[2]`` + - Motion vector sign probabilities. + * - __u8 + - ``mv.class[2][10]`` + - Motion vector class probabilities. + * - __u8 + - ``mv.class0_bit[2]`` + - Motion vector class0 bit probabilities. + * - __u8 + - ``mv.bits[2][10]`` + - Motion vector bits probabilities. + * - __u8 + - ``mv.class0_fr[2][2][3]`` + - Motion vector class0 fractional bit probabilities. + * - __u8 + - ``mv.fr[2][3]`` + - Motion vector fractional bit probabilities. + * - __u8 + - ``mv.class0_hp[2]`` + - Motion vector class0 high precision fractional bit probabilities. + * - __u8 + - ``mv.hp[2]`` + - Motion vector high precision fractional bit probabilities. + +``V4L2_CID_MPEG_VIDEO_VP9_FRAME_DECODE_PARAMS (struct)`` + Specifies the frame parameters for the associated VP9 frame decode request. + This includes the necessary parameters for configuring a stateless hardware + decoding pipeline for VP9. The bitstream parameters are defined according + to :ref:`vp9`. + + .. note:: + + This compound control is not yet part of the public kernel API and + it is expected to change. + +.. c:type:: v4l2_ctrl_vp9_frame_decode_params + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: struct v4l2_ctrl_vp9_frame_decode_params + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __u32 + - ``flags`` + - Combination of V4L2_VP9_FRAME_FLAG_* flags. See + :c:type:`v4l2_vp9_frame_flags`. + * - __u16 + - ``compressed_header_size`` + - Compressed header size in bytes. + * - __u16 + - ``uncompressed_header_size`` + - Uncompressed header size in bytes. + * - __u8 + - ``profile`` + - VP9 profile. Can be 0, 1, 2 or 3. + * - __u8Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst enum v4l2_mpeg_video_h264_hierarchical_coding_type - · line 2839
Quoted 10 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +10
-0+ - ``reset_frame_context`` + - Frame context that should be used/updated when decoding the frame. + * - __u8 + - ``bit_depth`` + - Component depth in bits. Must be 8 for profile 0 and 1. Must 10 or 12 + for profile 2 and 3. + * - __u8 + - ``color_space`` + - Specifies the color space of the stream. See V4L2_VP9_COLOR_SPACE_* + values. See :c:type:`v4l2_vp9_color_space`Ditto.
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst enum v4l2_mpeg_video_h264_hierarchical_coding_type - · line 2852
Quoted 13 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +13
-0+ * - __u8 + - ``interpolation_filter`` + - Specifies the filter selection used for performing inter prediction. See + :c:type:`v4l2_vp9_interpolation_filter`. + * - __u8 + - ``tile_cols_log2`` + - Specifies the base 2 logarithm of the width of each tile (where the + width is measured in units of 8x8 blocks). Shall be less than or equal + to 6. + * - __u8 + - ``tile_rows_log2`` + - Specifies the base 2 logarithm of the height of each tile (where the + height is measured in units of 8x8 blocks)Ditto
Quoted 177 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +177
-0+ * - __u8 + - ``tx_mode`` + - Specifies the TX mode. See :c:type:`v4l2_vp9_tx_mode`. + * - __u8 + - ``reference_mode`` + - Specifies the type of inter prediction to be used. See + :c:type:`v4l2_vp9_reference_mode`. + * - __u8 + - ``padding`` + - Needed to make this struct 64 bit aligned. Shall be filled with zeros. + * - __u16 + - ``frame_width_minus_1`` + - Add 1 to get the frame width expressed in pixels. + * - __u16 + - ``frame_height_minus_1`` + - Add 1 to to get the frame height expressed in pixels. + * - __u16 + - ``frame_width_minus_1`` + - Add 1 to to get the expected render width expressed in pixels. This is + not used during the decoding process but might be used by HW scalers to + prepare a frame that's ready for scanout. + * - __u16 + - frame_height_minus_1 + - Add 1 to get the expected render height expressed in pixels. This is + not used during the decoding process but might be used by HW scalers to + prepare a frame that's ready for scanout. + * - __u64 + - ``refs[3]`` + - Array of reference frame timestamps. + * - struct :c:type:`v4l2_vp9_loop_filter` + - ``lf`` + - Loop filter parameters. See struct :c:type:`v4l2_vp9_loop_filter`. + * - struct :c:type:`v4l2_vp9_quantization` + - ``quant`` + - Quantization parameters. See :c:type:`v4l2_vp9_quantization`. + * - struct :c:type:`v4l2_vp9_segmentation` + - ``seg`` + - Segmentation parameters. See :c:type:`v4l2_vp9_segmentation`. + * - struct :c:type:`v4l2_vp9_probabilities` + - ``probs`` + - Probabilities. See :c:type:`v4l2_vp9_probabilities`. + +.. c:type:: v4l2_vp9_frame_flags + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_frame_flags + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_VP9_FRAME_FLAG_KEY_FRAME`` + - The frame is a key frame. + * - ``V4L2_VP9_FRAME_FLAG_SHOW_FRAME`` + - The frame should be displayed. + * - ``V4L2_VP9_FRAME_FLAG_ERROR_RESILIENT`` + - The decoding should be error resilient. + * - ``V4L2_VP9_FRAME_FLAG_INTRA_ONLY`` + - The frame does not reference other frames. + * - ``V4L2_VP9_FRAME_FLAG_ALLOW_HIGH_PREC_MV`` + - the frame might can high precision motion vectors. + * - ``V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX`` + - Frame context should be updated after decoding. + * - ``V4L2_VP9_FRAME_FLAG_PARALLEL_DEC_MODE`` + - Parallel decoding is used. + * - ``V4L2_VP9_FRAME_FLAG_X_SUBSAMPLING`` + - Vertical subsampling is enabled. + * - ``V4L2_VP9_FRAME_FLAG_Y_SUBSAMPLING`` + - Horizontal subsampling is enabled. + * - ``V4L2_VP9_FRAME_FLAG_COLOR_RANGE_FULL_SWING`` + - The full UV range is used. + +.. c:type:: v4l2_vp9_ref_id + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_ref_id + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_REF_ID_LAST`` + - Last reference frame. + * - ``V4L2_REF_ID_GOLDEN`` + - Golden reference frame. + * - ``V4L2_REF_ID_ALTREF`` + - Alternative reference frame. + * - ``V4L2_REF_ID_CNT`` + - Number of reference frames. + +.. c:type:: v4l2_vp9_tx_mode + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_tx_mode + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_VP9_TX_MODE_ONLY_4X4`` + - Transform size is 4x4. + * - ``V4L2_VP9_TX_MODE_ALLOW_8X8`` + - Transform size can be up to 8x8. + * - ``V4L2_VP9_TX_MODE_ALLOW_16X16`` + - Transform size can be up to 16x16. + * - ``V4L2_VP9_TX_MODE_ALLOW_32X32`` + - transform size can be up to 32x32. + * - ``V4L2_VP9_TX_MODE_SELECT`` + - Bitstream contains transform size for each block. + +.. c:type:: v4l2_vp9_reference_mode + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_reference_mode + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_VP9_REF_MODE_SINGLE`` + - Indicates that all the inter blocks use only a single reference frame + to generate motion compensated prediction. + * - ``V4L2_VP9_REF_MODE_COMPOUND`` + - Requires all the inter blocks to use compound mode. Single reference + frame prediction is not allowed. + * - ``V4L2_VP9_REF_MODE_SELECT`` + - Allows each individual inter block to select between single and + compound prediction modes. + +.. c:type:: v4l2_vp9_interpolation_filter + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_interpolation_filter + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_VP9_INTERP_FILTER_8TAP`` + - Height tap filter. + * - ``V4L2_VP9_INTERP_FILTER_8TAP_SMOOTH`` + - Height tap smooth filter. + * - ``V4L2_VP9_INTERP_FILTER_8TAP_SHARP`` + - Height tap sharp filter. + * - ``V4L2_VP9_INTERP_FILTER_BILINEAR`` + - Bilinear filter. + * - ``V4L2_VP9_INTERP_FILTER_SWITCHABLE`` + - Filter selection is signaled at the block level. + +.. c:type:: v4l2_vp9_color_space + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_color_space + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_VP9_COLOR_SPACE_UNKNOWN`` + - Unknown color space. In this case the color space must be signaled + outside the VP9 bitstream. + * - ``V4L2_VP9_COLOR_SPACE_BT_601`` + - Rec. ITU-R BT.601-7 color space. + * - ``V4L2_VP9_COLOR_SPACE_BT_709`` + - Rec. ITU-R BT.709-6 color space.Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst enum v4l2_mpeg_video_h264_hierarchical_coding_type - · line 3039
Quoted 10 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +10
-0+ * - ``V4L2_VP9_COLOR_SPACE_SMPTE_170`` + - SMPTE-170 color space. + * - ``V4L2_VP9_COLOR_SPACE_SMPTE_240`` + - SMPTE-240 color space. + * - ``V4L2_VP9_COLOR_SPACE_BT_2020`` + - Rec. ITU-R BT.2020-2 color space. + * - ``V4L2_VP9_COLOR_SPACE_RESERVED`` + - Reserved. This value shall never be passed. + * - ``V4L2_VP9_COLOR_SPACE_SRGB`` + - sRGB (IEC 61966-2-1) color space.Should this be here? We already have colorspace information in the pix format structs.
Quoted 7 lines
It is also unclear from reading just this bit whether e.g.
V4L2_VP9_COLOR_SPACE_SRGBjust describes the sRGB color primaries or also the transfer function and quantization range.I'd like to see some more about how this is used. None of the other codec headers have this.
Quoted 5 lines
I think this comes directly from the spec. The Rockchip VDEC doesn't seem to use it.
Do you think we can drop it from here, and rely on the V4L2 colorspace passed in the format negotiation?
That would be preferred, yes. Otherwise you would have two places where you can define colorspaces, and that's confusing.
Quoted 75 lines
Quoted 65 lines
Quoted 65 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +64
-0+ +.. c:type:: v4l2_vp9_reset_frame_context + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_reset_frame_context + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_VP9_RESET_FRAME_CTX_NONE`` + - Do not reset any frame context. + * - ``V4L2_VP9_RESET_FRAME_CTX_SPEC`` + - Reset the frame context pointed by + :c:type:`v4l2_ctrl_vp9_frame_decode_params`.frame_context_idx. + * - ``V4L2_VP9_RESET_FRAME_CTX_ALL`` + - Reset all frame contexts. + +.. c:type:: v4l2_vp9_intra_prediction_mode + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: enum v4l2_vp9_intra_prediction_mode + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 + + * - ``V4L2_VP9_INTRA_PRED_DC`` + - DC intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_V`` + - Vertical intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_H`` + - Horizontal intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_D45`` + - D45 intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_D135`` + - D135 intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_D117`` + - D117 intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_D153`` + - D153 intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_D207`` + - D207 intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_D63`` + - D63 intra prediction. + * - ``V4L2_VP9_INTRA_PRED_MODE_TM`` + - True motion intra prediction. + +.. c:type:: v4l2_vp9_segmentation + +.. cssclass:: longtable + +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}| + +.. flat-table:: struct v4l2_vp9_segmentation + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __u8Quoted 9 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst enum v4l2_mpeg_video_h264_hierarchical_coding_type - · line 3109
Quoted 6 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +6
-0+ - ``flags`` + - Combination of V4L2_VP9_SEGMENTATION_FLAG_* flags. See + :c:type:`v4l2_vp9_segmentation_flags`. + * - __u8 + - ``tree_probs[7]`` + - Specifies the probability values to be used when decoding segment_id.
segment_id? Does this refer to a term defined in the VP9 spec?
Quoted 7 lines
Yes, this is all strictly spec compliant.
segment_idandseg_id_predictedare VP9 syntax elements.In particular that description is copy-pasted from the spec, but I agree it may seem confusing in this context. How about:
"to be used when decoding a Segment-ID"
That would be better, perhaps with a reference to the spec where this is described.
Quoted 12 lines
Quoted 11 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst enum v4l2_mpeg_video_h264_hierarchical_coding_type - · line 3113
Quoted 5 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +4
-0+ * - __u8 + - ``pred_prob[3]`` + - Specifies the probability values to be used when decoding + seg_id_predicte.
seg_id_predicte: typo? I don't see any field calledseg_id_predicte.It looks like
segment_idandseg_id_predicteare C field names, but I suspect they are not. I think this should be clarified.
Quoted 5 lines
Ditto,
"to be used when decoding a Predicted-Segment-ID"
Maybe this will be clearer ?
Yes, and also with a reference if possible.
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +3
-0+ * - __u8 + - ``padding[5]`` + - Used to align this struct on 64 bit. Shall be filled with zeros.
Quoted 8 lines
Quoted 2 lines
zeroes.
Oops, more typos :-)
Thanks for reviewing,
Ezequiel
Regards,
Hans
Re: [PATCH v3 2/3] media: uapi: Add VP9 stateless decoder controls
Quoted 8 lines · Hans Verkuil
Quoted 4 lines
I think this comes directly from the spec. The Rockchip VDEC doesn't seem to use it.
Do you think we can drop it from here, and rely on the V4L2 colorspace passed in the format negotiation?That would be preferred, yes. Otherwise you would have two places where you can define colorspaces, and that's confusing.
This is indeed redundant with the color information in the format. In VP8/9 there is only 1 header for everything, so it's harder to figure- out what to filter. While in H.264/HEVC this information is usually set an an extension header.
Though this do re-open the discussion about SPS in H.264/HEVC. In there you will find the coded size and the crop window and the sub-sampling. I don't remember exactly what was the conclusion, but I think it was kept to allow allow bitstream reconstruction. But it will effectively overlap directly (or indirectly) with some V4L2 generic API.
regards,
Nicolas
PATCH v3 3/3 media: rkvdec: Add the VP9 backend
In reply to Ezequiel GarciaFrom: Boris Brezillon <[email protected]>
The Rockchip VDEC supports VP9 profile 0 up to 4096x2304@30fps. Add a backend for this new format.
--- drivers/staging/media/rkvdec/Makefile | 2 +- drivers/staging/media/rkvdec/rkvdec-vp9.c | 1577 +++++++++++++++++++++ drivers/staging/media/rkvdec/rkvdec.c | 56 +- drivers/staging/media/rkvdec/rkvdec.h | 6 + 4 files changed, 1637 insertions(+), 4 deletions(-) create mode 100644 drivers/staging/media/rkvdec/rkvdec-vp9.c
drivers/staging/media/rkvdec/Makefile +1 -1
@@ -1,3 +1,3 @@ obj-$(CONFIG_VIDEO_ROCKCHIP_VDEC) += rockchip-vdec.o -rockchip-vdec-y += rkvdec.o rkvdec-h264.o +rockchip-vdec-y += rkvdec.o rkvdec-h264.o rkvdec-vp9.o
drivers/staging/media/rkvdec/rkvdec-vp9.c new file +1577 -0
1,577 lines not on this page · raw message
drivers/staging/media/rkvdec/rkvdec.c +53 -3
71 lines not on this page · raw message
drivers/staging/media/rkvdec/rkvdec.h +6 -0
@@ -52,6 +52,10 @@ struct rkvdec_vp9_decoded_buffer_info {struct rkvdec_decoded_buffer {/* Must be the first field in this struct. */ struct v4l2_m2m_buffer base; + +union { +struct rkvdec_vp9_decoded_buffer_info vp9; + }; }; static inline struct rkvdec_decoded_buffer *
@@ -118,4 +122,6 @@ void rkvdec_run_preamble(struct rkvdec_ctx *ctx, struct rkvdec_run *run);void rkvdec_run_postamble (struct rkvdec_ctx *ctx,struct rkvdec_run *run);extern const struct rkvdec_coded_fmt_ops rkvdec_h264_fmt_ops; +extern const struct rkvdec_coded_fmt_ops rkvdec_vp9_fmt_ops; +#endif /* RKVDEC_H_ */
Signature · 1 line
2.26.0.rc2
Re: [PATCH v3 3/3] media: rkvdec: Add the VP9 backend
Quoted 30 lines · Ezequiel Garcia
From: Boris Brezillon <[email protected]>
The Rockchip VDEC supports VP9 profile 0 up to 4096x2304@30fps. Add a backend for this new format.
Signed-off-by Boris Brezillon <[email protected]>Signed-off-by Ezequiel Garcia <[email protected]>--- drivers/staging/media/rkvdec/Makefile | 2 +- drivers/staging/media/rkvdec/rkvdec-vp9.c | 1577 +++++++++++++++++++++ drivers/staging/media/rkvdec/rkvdec.c | 56 +- drivers/staging/media/rkvdec/rkvdec.h | 6 + 4 files changed, 1637 insertions(+), 4 deletions(-) create mode 100644 drivers/staging/media/rkvdec/rkvdec-vp9.cdrivers/staging/media/rkvdec/Makefile +1
-1@@ -1,3 +1,3 @@ obj-$(CONFIG_VIDEO_ROCKCHIP_VDEC) += rockchip-vdec.o -rockchip-vdec-y += rkvdec.o rkvdec-h264.o +rockchip-vdec-y += rkvdec.o rkvdec-h264.o rkvdec-vp9.odrivers/staging/media/rkvdec/rkvdec-vp9.c new file +0
-0@@ -0,0 +1,1577 @@
<snip>
Quoted 61 lines
drivers/staging/media/rkvdec/rkvdec-vp9.c new file +61
-0+static void init_inter_probs (struct rkvdec_ctx *ctx, +const struct rkvdec_vp9_run *run) +{ +const struct v4l2_ctrl_vp9_frame_decode_params *dec_params; +struct rkvdec_vp9_ctx *vp9_ctx = ctx->priv; +struct rkvdec_vp9_priv_tbl *tbl = vp9_ctx->priv_tbl.cpu; +struct rkvdec_vp9_inter_frame_probs *rkprobs; +const struct v4l2_vp9_probabilities *probs; +unsigned int i, j, k; + + rkprobs = &tbl->probs.inter; + dec_params = run->decode_params; + probs = &dec_params->probs; + +/* +* inter probs +* 151 x 128 bits, aligned to 152 x 128 bits +* inter only +* intra_y_mode & inter_block info 6 x 128 bits +*/ + +memcpy (rkprobs->y_mode, probs->y_mode,sizeof (rkprobs->y_mode)); +memcpy (rkprobs->comp_mode, probs->comp_mode, +sizeof (rkprobs->comp_mode)); +memcpy (rkprobs->comp_ref, probs->comp_ref, +sizeof (rkprobs->comp_ref)); +memcpy (rkprobs->single_ref, probs->single_ref, +sizeof (rkprobs->single_ref)); +memcpy (rkprobs->inter_mode, probs->inter_mode, +sizeof (rkprobs->inter_mode)); +memcpy (rkprobs->interp_filter, probs->interp_filter, +sizeof (rkprobs->interp_filter)); + +/* 128 x 128 bits coeff related */ +for (i =0 ; i <ARRAY_SIZE (probs->coef); i++) { +for (j =0 ; j <ARRAY_SIZE (probs->coef[0 ]); j++) { +for (k =0 ; k <ARRAY_SIZE (probs->coef[0 ][0 ]); k++) +write_coeff_plane (probs->coef[i][j][k], + rkprobs->coef[k][i][j]); + } + } + +/* intra uv mode 6 x 128 */ +memcpy (rkprobs->uv_mode_0_2, &probs->uv_mode[0 ], +sizeof (rkprobs->uv_mode_0_2)); +memcpy (rkprobs->uv_mode_3_5, &probs->uv_mode[3 ], +sizeof (rkprobs->uv_mode_3_5)); +memcpy (rkprobs->uv_mode_6_8, &probs->uv_mode[6 ], +sizeof (rkprobs->uv_mode_6_8)); +memcpy (rkprobs->uv_mode_9, &probs->uv_mode[9 ], +sizeof (rkprobs->uv_mode_9)); + +/* mv related 6 x 128 */ +memcpy (rkprobs->mv.joint, probs->mv.joint, +sizeof (rkprobs->mv.joint)); +memcpy (rkprobs->mv.sign, probs->mv.sign, +sizeof (rkprobs->mv.sign)); +memcpy (rkprobs->mv.class, probs->mv.class, +sizeof (rkprobs->mv.class)); +memcpy (rkprobs->mv.class0_bit, probs->mv.class0_bit, +sizeof (rkprobs->mv.class0_bit));
drivers/staging/media/rkvdec/rkvdec-vp9.c line 502
Quoted 10 lines
drivers/staging/media/rkvdec/rkvdec-vp9.c new file +10
-0+memcpy (rkprobs->mv.bits, probs->mv.bits, +sizeof (rkprobs->mv.bits)); +memcpy (rkprobs->mv.class0_fr, probs->mv.class0_fr, +sizeof (rkprobs->mv.class0_fr)); +memcpy (rkprobs->mv.fr, probs->mv.fr, +sizeof (rkprobs->mv.fr)); +memcpy (rkprobs->mv.class0_hp, probs->mv.class0_hp, +sizeof (rkprobs->mv.class0_hp)); +memcpy (rkprobs->mv.hp, probs->mv.hp, +sizeof (rkprobs->mv.hp));
Can't you just do: 'rkprobs->mv = probs->mv'?
Quoted 1 line
+}
Regards,
Hans
Re: [PATCH v3 3/3] media: rkvdec: Add the VP9 backend
Quoted 99 lines · Hans Verkuil
Quoted 30 lines · Ezequiel Garcia
From: Boris Brezillon <[email protected]>
The Rockchip VDEC supports VP9 profile 0 up to 4096x2304@30fps. Add a backend for this new format.
Signed-off-by Boris Brezillon <[email protected]>Signed-off-by Ezequiel Garcia <[email protected]>--- drivers/staging/media/rkvdec/Makefile | 2 +- drivers/staging/media/rkvdec/rkvdec-vp9.c | 1577 +++++++++++++++++++++ drivers/staging/media/rkvdec/rkvdec.c | 56 +- drivers/staging/media/rkvdec/rkvdec.h | 6 + 4 files changed, 1637 insertions(+), 4 deletions(-) create mode 100644 drivers/staging/media/rkvdec/rkvdec-vp9.cdrivers/staging/media/rkvdec/Makefile +1
-1@@ -1,3 +1,3 @@ obj-$(CONFIG_VIDEO_ROCKCHIP_VDEC) += rockchip-vdec.o -rockchip-vdec-y += rkvdec.o rkvdec-h264.o +rockchip-vdec-y += rkvdec.o rkvdec-h264.o rkvdec-vp9.odrivers/staging/media/rkvdec/rkvdec-vp9.c new file +0
-0@@ -0,0 +1,1577 @@<snip>
Quoted 65 lines
drivers/staging/media/rkvdec/rkvdec-vp9.c new file +65
-0+static void init_inter_probs (struct rkvdec_ctx *ctx, +const struct rkvdec_vp9_run *run) +{ +const struct v4l2_ctrl_vp9_frame_decode_params *dec_params; +struct rkvdec_vp9_ctx *vp9_ctx = ctx->priv; +struct rkvdec_vp9_priv_tbl *tbl = vp9_ctx->priv_tbl.cpu; +struct rkvdec_vp9_inter_frame_probs *rkprobs; +const struct v4l2_vp9_probabilities *probs; +unsigned int i, j, k; + + rkprobs = &tbl->probs.inter; + dec_params = run->decode_params; + probs = &dec_params->probs; + +/* +* inter probs +* 151 x 128 bits, aligned to 152 x 128 bits +* inter only +* intra_y_mode & inter_block info 6 x 128 bits +*/ + +memcpy (rkprobs->y_mode, probs->y_mode,sizeof (rkprobs->y_mode)); +memcpy (rkprobs->comp_mode, probs->comp_mode, +sizeof (rkprobs->comp_mode)); +memcpy (rkprobs->comp_ref, probs->comp_ref, +sizeof (rkprobs->comp_ref)); +memcpy (rkprobs->single_ref, probs->single_ref, +sizeof (rkprobs->single_ref)); +memcpy (rkprobs->inter_mode, probs->inter_mode, +sizeof (rkprobs->inter_mode)); +memcpy (rkprobs->interp_filter, probs->interp_filter, +sizeof (rkprobs->interp_filter)); + +/* 128 x 128 bits coeff related */ +for (i =0 ; i <ARRAY_SIZE (probs->coef); i++) { +for (j =0 ; j <ARRAY_SIZE (probs->coef[0 ]); j++) { +for (k =0 ; k <ARRAY_SIZE (probs->coef[0 ][0 ]); k++) +write_coeff_plane (probs->coef[i][j][k], + rkprobs->coef[k][i][j]); + } + } + +/* intra uv mode 6 x 128 */ +memcpy (rkprobs->uv_mode_0_2, &probs->uv_mode[0 ], +sizeof (rkprobs->uv_mode_0_2)); +memcpy (rkprobs->uv_mode_3_5, &probs->uv_mode[3 ], +sizeof (rkprobs->uv_mode_3_5)); +memcpy (rkprobs->uv_mode_6_8, &probs->uv_mode[6 ], +sizeof (rkprobs->uv_mode_6_8)); +memcpy (rkprobs->uv_mode_9, &probs->uv_mode[9 ], +sizeof (rkprobs->uv_mode_9)); + +/* mv related 6 x 128 */ +memcpy (rkprobs->mv.joint, probs->mv.joint, +sizeof (rkprobs->mv.joint)); +memcpy (rkprobs->mv.sign, probs->mv.sign, +sizeof (rkprobs->mv.sign)); +memcpy (rkprobs->mv.class, probs->mv.class, +sizeof (rkprobs->mv.class)); +memcpy (rkprobs->mv.class0_bit, probs->mv.class0_bit, +sizeof (rkprobs->mv.class0_bit)); +memcpy (rkprobs->mv.bits, probs->mv.bits, +sizeof (rkprobs->mv.bits)); +memcpy (rkprobs->mv.class0_fr, probs->mv.class0_fr, +sizeof (rkprobs->mv.class0_fr));
Quoted 9 lines
drivers/staging/media/rkvdec/rkvdec-vp9.c line 502
Quoted 6 lines
drivers/staging/media/rkvdec/rkvdec-vp9.c new file +6
-0+memcpy (rkprobs->mv.fr, probs->mv.fr, +sizeof (rkprobs->mv.fr)); +memcpy (rkprobs->mv.class0_hp, probs->mv.class0_hp, +sizeof (rkprobs->mv.class0_hp)); +memcpy (rkprobs->mv.hp, probs->mv.hp, +sizeof (rkprobs->mv.hp));Can't you just do: '
rkprobs->mv=probs->mv'?
I think I'd like to keep this as-is.
Having the memcpy makes it explicit that we are copying these structs around. While the assignment would bring type checking, it can be misleading for readers.
Thanks,
Ezequiel
Re: [PATCH v3 3/3] media: rkvdec: Add the VP9 backend
Quoted 111 lines · Ezequiel Garcia
Quoted 99 lines · Hans Verkuil
Quoted 30 lines · Ezequiel Garcia
From: Boris Brezillon <[email protected]>
The Rockchip VDEC supports VP9 profile 0 up to 4096x2304@30fps. Add a backend for this new format.
Signed-off-by Boris Brezillon <[email protected]>Signed-off-by Ezequiel Garcia <[email protected]>--- drivers/staging/media/rkvdec/Makefile | 2 +- drivers/staging/media/rkvdec/rkvdec-vp9.c | 1577 +++++++++++++++++++++ drivers/staging/media/rkvdec/rkvdec.c | 56 +- drivers/staging/media/rkvdec/rkvdec.h | 6 + 4 files changed, 1637 insertions(+), 4 deletions(-) create mode 100644 drivers/staging/media/rkvdec/rkvdec-vp9.cdrivers/staging/media/rkvdec/Makefile +1
-1@@ -1,3 +1,3 @@ obj-$(CONFIG_VIDEO_ROCKCHIP_VDEC) += rockchip-vdec.o -rockchip-vdec-y += rkvdec.o rkvdec-h264.o +rockchip-vdec-y += rkvdec.o rkvdec-h264.o rkvdec-vp9.odrivers/staging/media/rkvdec/rkvdec-vp9.c new file +0
-0@@ -0,0 +1,1577 @@<snip>
Quoted 65 lines
drivers/staging/media/rkvdec/rkvdec-vp9.c new file +65
-0+static void init_inter_probs (struct rkvdec_ctx *ctx, +const struct rkvdec_vp9_run *run) +{ +const struct v4l2_ctrl_vp9_frame_decode_params *dec_params; +struct rkvdec_vp9_ctx *vp9_ctx = ctx->priv; +struct rkvdec_vp9_priv_tbl *tbl = vp9_ctx->priv_tbl.cpu; +struct rkvdec_vp9_inter_frame_probs *rkprobs; +const struct v4l2_vp9_probabilities *probs; +unsigned int i, j, k; + + rkprobs = &tbl->probs.inter; + dec_params = run->decode_params; + probs = &dec_params->probs; + +/* +* inter probs +* 151 x 128 bits, aligned to 152 x 128 bits +* inter only +* intra_y_mode & inter_block info 6 x 128 bits +*/ + +memcpy (rkprobs->y_mode, probs->y_mode,sizeof (rkprobs->y_mode)); +memcpy (rkprobs->comp_mode, probs->comp_mode, +sizeof (rkprobs->comp_mode)); +memcpy (rkprobs->comp_ref, probs->comp_ref, +sizeof (rkprobs->comp_ref)); +memcpy (rkprobs->single_ref, probs->single_ref, +sizeof (rkprobs->single_ref)); +memcpy (rkprobs->inter_mode, probs->inter_mode, +sizeof (rkprobs->inter_mode)); +memcpy (rkprobs->interp_filter, probs->interp_filter, +sizeof (rkprobs->interp_filter)); + +/* 128 x 128 bits coeff related */ +for (i =0 ; i <ARRAY_SIZE (probs->coef); i++) { +for (j =0 ; j <ARRAY_SIZE (probs->coef[0 ]); j++) { +for (k =0 ; k <ARRAY_SIZE (probs->coef[0 ][0 ]); k++) +write_coeff_plane (probs->coef[i][j][k], + rkprobs->coef[k][i][j]); + } + } + +/* intra uv mode 6 x 128 */ +memcpy (rkprobs->uv_mode_0_2, &probs->uv_mode[0 ], +sizeof (rkprobs->uv_mode_0_2)); +memcpy (rkprobs->uv_mode_3_5, &probs->uv_mode[3 ], +sizeof (rkprobs->uv_mode_3_5)); +memcpy (rkprobs->uv_mode_6_8, &probs->uv_mode[6 ], +sizeof (rkprobs->uv_mode_6_8)); +memcpy (rkprobs->uv_mode_9, &probs->uv_mode[9 ], +sizeof (rkprobs->uv_mode_9)); + +/* mv related 6 x 128 */ +memcpy (rkprobs->mv.joint, probs->mv.joint, +sizeof (rkprobs->mv.joint)); +memcpy (rkprobs->mv.sign, probs->mv.sign, +sizeof (rkprobs->mv.sign)); +memcpy (rkprobs->mv.class, probs->mv.class, +sizeof (rkprobs->mv.class)); +memcpy (rkprobs->mv.class0_bit, probs->mv.class0_bit, +sizeof (rkprobs->mv.class0_bit)); +memcpy (rkprobs->mv.bits, probs->mv.bits, +sizeof (rkprobs->mv.bits)); +memcpy (rkprobs->mv.class0_fr, probs->mv.class0_fr, +sizeof (rkprobs->mv.class0_fr));Quoted 9 lines
drivers/staging/media/rkvdec/rkvdec-vp9.c line 502
Quoted 6 lines
drivers/staging/media/rkvdec/rkvdec-vp9.c new file +6
-0+memcpy (rkprobs->mv.fr, probs->mv.fr, +sizeof (rkprobs->mv.fr)); +memcpy (rkprobs->mv.class0_hp, probs->mv.class0_hp, +sizeof (rkprobs->mv.class0_hp)); +memcpy (rkprobs->mv.hp, probs->mv.hp, +sizeof (rkprobs->mv.hp));Can't you just do: '
rkprobs->mv=probs->mv'?
Quoted 5 lines
I think I'd like to keep this as-is.
Having the memcpy makes it explicit that we are copying these structs around. While the assignment would bring type checking, it can be misleading for readers.
On the other hand, it's not obvious from the code that all fields of the structure are copied. Perhaps memcpy(&rkprobs->mv, &probs->mv, sizeof(rkprobs->mv)) would be a good compromise?
Re: [PATCH v3 3/3] media: rkvdec: Add the VP9 backend
Quoted 113 lines · Tomasz Figa
Quoted 112 lines · Ezequiel Garcia
Quoted 99 lines · Hans Verkuil
Quoted 30 lines · Ezequiel Garcia
From: Boris Brezillon <[email protected]>
The Rockchip VDEC supports VP9 profile 0 up to 4096x2304@30fps. Add a backend for this new format.
Signed-off-by Boris Brezillon <[email protected]>Signed-off-by Ezequiel Garcia <[email protected]>--- drivers/staging/media/rkvdec/Makefile | 2 +- drivers/staging/media/rkvdec/rkvdec-vp9.c | 1577 +++++++++++++++++++++ drivers/staging/media/rkvdec/rkvdec.c | 56 +- drivers/staging/media/rkvdec/rkvdec.h | 6 + 4 files changed, 1637 insertions(+), 4 deletions(-) create mode 100644 drivers/staging/media/rkvdec/rkvdec-vp9.cdrivers/staging/media/rkvdec/Makefile +1
-1@@ -1,3 +1,3 @@ obj-$(CONFIG_VIDEO_ROCKCHIP_VDEC) += rockchip-vdec.o -rockchip-vdec-y += rkvdec.o rkvdec-h264.o +rockchip-vdec-y += rkvdec.o rkvdec-h264.o rkvdec-vp9.odrivers/staging/media/rkvdec/rkvdec-vp9.c new file +0
-0@@ -0,0 +1,1577 @@<snip>
Quoted 65 lines
drivers/staging/media/rkvdec/rkvdec-vp9.c new file +65
-0+static void init_inter_probs (struct rkvdec_ctx *ctx, +const struct rkvdec_vp9_run *run) +{ +const struct v4l2_ctrl_vp9_frame_decode_params *dec_params; +struct rkvdec_vp9_ctx *vp9_ctx = ctx->priv; +struct rkvdec_vp9_priv_tbl *tbl = vp9_ctx->priv_tbl.cpu; +struct rkvdec_vp9_inter_frame_probs *rkprobs; +const struct v4l2_vp9_probabilities *probs; +unsigned int i, j, k; + + rkprobs = &tbl->probs.inter; + dec_params = run->decode_params; + probs = &dec_params->probs; + +/* +* inter probs +* 151 x 128 bits, aligned to 152 x 128 bits +* inter only +* intra_y_mode & inter_block info 6 x 128 bits +*/ + +memcpy (rkprobs->y_mode, probs->y_mode,sizeof (rkprobs->y_mode)); +memcpy (rkprobs->comp_mode, probs->comp_mode, +sizeof (rkprobs->comp_mode)); +memcpy (rkprobs->comp_ref, probs->comp_ref, +sizeof (rkprobs->comp_ref)); +memcpy (rkprobs->single_ref, probs->single_ref, +sizeof (rkprobs->single_ref)); +memcpy (rkprobs->inter_mode, probs->inter_mode, +sizeof (rkprobs->inter_mode)); +memcpy (rkprobs->interp_filter, probs->interp_filter, +sizeof (rkprobs->interp_filter)); + +/* 128 x 128 bits coeff related */ +for (i =0 ; i <ARRAY_SIZE (probs->coef); i++) { +for (j =0 ; j <ARRAY_SIZE (probs->coef[0 ]); j++) { +for (k =0 ; k <ARRAY_SIZE (probs->coef[0 ][0 ]); k++) +write_coeff_plane (probs->coef[i][j][k], + rkprobs->coef[k][i][j]); + } + } + +/* intra uv mode 6 x 128 */ +memcpy (rkprobs->uv_mode_0_2, &probs->uv_mode[0 ], +sizeof (rkprobs->uv_mode_0_2)); +memcpy (rkprobs->uv_mode_3_5, &probs->uv_mode[3 ], +sizeof (rkprobs->uv_mode_3_5)); +memcpy (rkprobs->uv_mode_6_8, &probs->uv_mode[6 ], +sizeof (rkprobs->uv_mode_6_8)); +memcpy (rkprobs->uv_mode_9, &probs->uv_mode[9 ], +sizeof (rkprobs->uv_mode_9)); + +/* mv related 6 x 128 */ +memcpy (rkprobs->mv.joint, probs->mv.joint, +sizeof (rkprobs->mv.joint)); +memcpy (rkprobs->mv.sign, probs->mv.sign, +sizeof (rkprobs->mv.sign)); +memcpy (rkprobs->mv.class, probs->mv.class, +sizeof (rkprobs->mv.class)); +memcpy (rkprobs->mv.class0_bit, probs->mv.class0_bit, +sizeof (rkprobs->mv.class0_bit)); +memcpy (rkprobs->mv.bits, probs->mv.bits, +sizeof (rkprobs->mv.bits)); +memcpy (rkprobs->mv.class0_fr, probs->mv.class0_fr, +sizeof (rkprobs->mv.class0_fr));Quoted 9 lines
drivers/staging/media/rkvdec/rkvdec-vp9.c
Quoted 6 lines
drivers/staging/media/rkvdec/rkvdec-vp9.c new file +6
-0+memcpy (rkprobs->mv.fr, probs->mv.fr, +sizeof (rkprobs->mv.fr)); +memcpy (rkprobs->mv.class0_hp, probs->mv.class0_hp, +sizeof (rkprobs->mv.class0_hp)); +memcpy (rkprobs->mv.hp, probs->mv.hp, +sizeof (rkprobs->mv.hp));Can't you just do: '
rkprobs->mv=probs->mv'?I think I'd like to keep this as-is.
Quoted 7 lines
Quoted 3 lines
Having the memcpy makes it explicit that we are copying these structs around. While the assignment would bring type checking, it can be misleading for readers.
On the other hand, it's not obvious from the code that all fields of the structure are copied. Perhaps
memcpy(&rkprobs->mv, &probs->mv, sizeof(rkprobs->mv))would be a good compromise?
Well, that would effectively (inadvertedly) tie probs:mv to rkprobs:mv, so we might as well do something like:
(thanks Boris for the suggestion)
drivers/staging/media/rkvdec/rkvdec-vp9.c +1 -11
@@ -48,17 +48,7 @@ struct rkvdec_vp9_inter_frame_probs {u8 uv_mode_9[9 ];u8 padding4[7 ];u8 padding5[16 ]; -struct { -u8 joint[3 ]; -u8 sign[2 ]; -u8 class[2 ][10 ]; -u8 class0_bit[2 ]; -u8 bits[2 ][10 ]; -u8 class0_fr[2 ][2 ][3 ]; -u8 fr[2 ][3 ]; -u8 class0_hp[2 ]; -u8 hp[2 ]; - } mv; +struct v4l2_vp9_mv_probabilities mv; };
However, I'm reluctant to consider any of these suggestions because we are effectively matching a hardware descriptor to a software interface, where the latter is unstable.
If we were to change the software interface at any point, we'd break this driver.
The more I think about this, the more I think the drivers needs to stay as-is.
Thanks,
Ezequiel
Re: [PATCH v3 3/3] media: rkvdec: Add the VP9 backend
In reply to Ezequiel GarciaQuoted 7 lines · Ezequiel Garcia
From: Boris Brezillon <boris.brezillon-ZGY8ohtN/[email protected]>
The Rockchip VDEC supports VP9 profile 0 up to 4096x2304@30fps. Add a backend for this new format.
I have tested this series on a Rock Pi 4 RK3399 device using the ffmpeg vp9 request api hwaccel at [1].
Decoding up to 3840x2160@30fps media is working great. (frame scaling was not tested or fully implemented in ffmpeg hwaccel)
[1] https://github.com/Kwiboo/FFmpeg/commits/v4l2-request-hwaccel-4.2.2
Regards,
Jonas
Quoted 8 lines
--- drivers/staging/media/rkvdec/Makefile | 2 +- drivers/staging/media/rkvdec/rkvdec-vp9.c | 1577 +++++++++++++++++++++ drivers/staging/media/rkvdec/rkvdec.c | 56 +- drivers/staging/media/rkvdec/rkvdec.h | 6 + 4 files changed, 1637 insertions(+), 4 deletions(-) create mode 100644 drivers/staging/media/rkvdec/rkvdec-vp9.c
[...]