Skip to contentSkip to conversation

media: amphion: decoder add support to RealVideo

PATCH Superseded Ming Qian 2 participants 5 messages Jan 13, 2023 – Jan 17, 2023

Subsystems (2)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Ming Qian

author maintainer
#0 lore raw

v3 cover media: amphion: decoder add support to RealVideo

RealVideo, or also spelled as Real Video, is a suite of proprietary video compression formats developed by RealNetworks - the specific format changes with the version.
RealVideo codecs are identified by four-character codes. RV30 and RV40 are RealNetworks' proprietary H.264-based codecs.

amphion decoder can support decoding RV30 and RV40 streams.

v3
- define respective formats for RV30 and RV40

v2
- correct the format name from RealMedia to RealVideo

Ming Qian (2):
media: add RealVideo format RV30 and RV40
media: amphion: support to decode RealVideo video

 .../media/v4l/pixfmt-compressed.rst           | 21 +++++++++++++++++
 drivers/media/platform/amphion/vdec.c         | 14 +++++++++++
 drivers/media/platform/amphion/vpu_malone.c   | 23 +++++++++++++++++++
 drivers/media/platform/amphion/vpu_malone.h   |  1 +
 drivers/media/v4l2-core/v4l2-ioctl.c          |  2 ++
 include/uapi/linux/videodev2.h                |  2 ++
 6 files changed, 63 insertions(+)
Signature · 7 lines
2.38.1


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
author maintainer
#1 lore raw

PATCH v3 1/2 media: add RealVideo format RV30 and RV40

RealVideo, or also spelled as Real Video, is a suite of proprietary video compression formats developed by RealNetworks - the specific format changes with the version.
RealVideo codecs are identified by four-character codes. RV30 and RV40 are RealNetworks' proprietary H.264-based codecs.

---
 .../media/v4l/pixfmt-compressed.rst           | 21 +++++++++++++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c          |  2 ++
 include/uapi/linux/videodev2.h                |  2 ++
 3 files changed, 25 insertions(+)
Documentation/userspace-api/media/v4l/pixfmt-compressed.rst +21 -0
index a0230f357680..50ac095c1f16 100644
@@ -237,6 +237,27 @@ Compressed Formats
         Metadata associated with the frame to decode is required to be passed
         through the ``V4L2_CID_STATELESS_FWHT_PARAMS`` control.
 	See the :ref:`associated Codec Control ID <codec-stateless-fwht>`.
+    * .. _V4L2-PIX-FMT-RV30:
+
+      - ``V4L2_PIX_FMT_RV30``
+      - 'RV30'
+      - RealVideo, or also spelled as Real Video, is a suite of proprietary
+        video compression formats developed by RealNetworks -
+        the specific format changes with the version.
+        RealVideo codecs are identified by four-character codes.
+        RV30 corresponds to RealVideo 8, suspected to based largely on an early draft of H.264
+    * .. _V4L2-PIX-FMT-RV40:
+
+      - ``V4L2_PIX_FMT_RV40``
+      - 'RV40'
+      - RV40 represents RealVideo 9 and RealVideo 10.
+        RealVideo 9, suspected to be based on H.264,
+        RealVideo 10, aka RV9 EHQ, This refers to
+        an improved encoder for the RV9 format
+        that is fully backwards compatible with RV9 players -
+        the format and decoder did not change,
+        only the encoder did.
+        As a result, it uses the same FourCC.
 
 .. raw:: latex
 
drivers/media/v4l2-core/v4l2-ioctl.c +2 -0
index 9b5b04b8aa69..0546b00d3fc9 100644
@@ -1473,6 +1473,8 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 		case V4L2_PIX_FMT_FWHT:		descr = "FWHT"; break; /* used in vicodec */
 		case V4L2_PIX_FMT_FWHT_STATELESS:	descr = "FWHT Stateless"; break; /* used in vicodec */
 		case V4L2_PIX_FMT_SPK:		descr = "Sorenson Spark"; break;
+		case V4L2_PIX_FMT_RV30:		descr = "RealVideo 8"; break;
+		case V4L2_PIX_FMT_RV40:		descr = "RealVideo 9 & 10"; break;
 		case V4L2_PIX_FMT_CPIA1:	descr = "GSPCA CPiA YUV"; break;
 		case V4L2_PIX_FMT_WNVA:		descr = "WNVA"; break;
 		case V4L2_PIX_FMT_SN9C10X:	descr = "GSPCA SN9C10X"; break;
include/uapi/linux/videodev2.h +2 -0
index 262ef10cfa02..b121154a6e24 100644
@@ -740,6 +740,8 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
 #define V4L2_PIX_FMT_HEVC_SLICE v4l2_fourcc('S', '2', '6', '5') /* HEVC parsed slices */
 #define V4L2_PIX_FMT_SPK      v4l2_fourcc('S', 'P', 'K', '0') /* Sorenson Spark */
+#define V4L2_PIX_FMT_RV30     v4l2_fourcc('R', 'V', '3', '0') /* RealVideo 8 */
+#define V4L2_PIX_FMT_RV40     v4l2_fourcc('R', 'V', '4', '0') /* RealVideo 9 & 10 */
 
 /*  Vendor-specific formats   */
 #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /* cpia1 YUV */
Signature · 7 lines
2.38.1


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Reviewed-by on v3 1/2
#2 lore raw

Re: [PATCH v3 1/2] media: add RealVideo format RV30 and RV40

Le vendredi 13 janvier 2023 à 12:13 +0800, Ming Qian a écrit :

Quoted 20 lines

RealVideo, or also spelled as Real Video, is a suite of proprietary video compression formats developed by RealNetworks - the specific format changes with the version.
RealVideo codecs are identified by four-character codes. RV30 and RV40 are RealNetworks' proprietary H.264-based codecs.

---
 .../media/v4l/pixfmt-compressed.rst           | 21 +++++++++++++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c          |  2 ++
 include/uapi/linux/videodev2.h                |  2 ++
 3 files changed, 25 insertions(+)
Documentation/userspace-api/media/v4l/pixfmt-compressed.rst +0 -0
index a0230f357680..50ac095c1f16 100644
@@ -237,6 +237,27 @@ Compressed Formats
         Metadata associated with the frame to decode is required to be passed
         through the ``V4L2_CID_STATELESS_FWHT_PARAMS`` control.

Documentation/userspace-api/media/v4l/pixfmt-compressed.rst Compressed Formats · line 248

Quoted 10 lines
Documentation/userspace-api/media/v4l/pixfmt-compressed.rst +9 -0
 	See the :ref:`associated Codec Control ID <codec-stateless-fwht>`.
+    * .. _V4L2-PIX-FMT-RV30:
+
+      - ``V4L2_PIX_FMT_RV30``
+      - 'RV30'
+      - RealVideo, or also spelled as Real Video, is a suite of proprietary
+        video compression formats developed by RealNetworks -
+        the specific format changes with the version.
+        RealVideo codecs are identified by four-character codes.
+        RV30 corresponds to RealVideo 8, suspected to based largely on an early draft of H.264

to *be* based on.

Just a style comment too, but the flow could be improved while fixing this typo. Lines in this paragraph seems very uneven.

Documentation/userspace-api/media/v4l/pixfmt-compressed.rst Compressed Formats · line 260

Quoted 12 lines
Documentation/userspace-api/media/v4l/pixfmt-compressed.rst +12 -0
+    * .. _V4L2-PIX-FMT-RV40:
+
+      - ``V4L2_PIX_FMT_RV40``
+      - 'RV40'
+      - RV40 represents RealVideo 9 and RealVideo 10.
+        RealVideo 9, suspected to be based on H.264,
+        RealVideo 10, aka RV9 EHQ, This refers to
+        an improved encoder for the RV9 format
+        that is fully backwards compatible with RV9 players -
+        the format and decoder did not change,
+        only the encoder did.
+        As a result, it uses the same FourCC.

It feels this one could be wider ? But again, this is cosmetic at this point. For the API changes, assuming fixed typo and improved style:

Quoted 29 lines
Documentation/userspace-api/media/v4l/pixfmt-compressed.rst +0 -0
 .. raw:: latex
 
drivers/media/v4l2-core/v4l2-ioctl.c +2 -0
index 9b5b04b8aa69..0546b00d3fc9 100644
@@ -1473,6 +1473,8 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 		case V4L2_PIX_FMT_FWHT:		descr = "FWHT"; break; /* used in vicodec */
 		case V4L2_PIX_FMT_FWHT_STATELESS:	descr = "FWHT Stateless"; break; /* used in vicodec */
 		case V4L2_PIX_FMT_SPK:		descr = "Sorenson Spark"; break;
+		case V4L2_PIX_FMT_RV30:		descr = "RealVideo 8"; break;
+		case V4L2_PIX_FMT_RV40:		descr = "RealVideo 9 & 10"; break;
 		case V4L2_PIX_FMT_CPIA1:	descr = "GSPCA CPiA YUV"; break;
 		case V4L2_PIX_FMT_WNVA:		descr = "WNVA"; break;
 		case V4L2_PIX_FMT_SN9C10X:	descr = "GSPCA SN9C10X"; break;
include/uapi/linux/videodev2.h +2 -0
index 262ef10cfa02..b121154a6e24 100644
@@ -740,6 +740,8 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
 #define V4L2_PIX_FMT_HEVC_SLICE v4l2_fourcc('S', '2', '6', '5') /* HEVC parsed slices */
 #define V4L2_PIX_FMT_SPK      v4l2_fourcc('S', 'P', 'K', '0') /* Sorenson Spark */
+#define V4L2_PIX_FMT_RV30     v4l2_fourcc('R', 'V', '3', '0') /* RealVideo 8 */
+#define V4L2_PIX_FMT_RV40     v4l2_fourcc('R', 'V', '4', '0') /* RealVideo 9 & 10 */
 
 /*  Vendor-specific formats   */
 #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /* cpia1 YUV */
author maintainer
#3 lore raw

RE: [EXT] Re: [PATCH v3 1/2] media: add RealVideo format RV30 and RV40

Quoted 70 lines

-----Original Message-----
From: Nicolas Dufresne <[email protected]>
Sent: 2023年1月17日 4:47
To: Ming Qian <[email protected]>; [email protected]; hverkuil- [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; dl-linux-imx <linux- [email protected]>; X.H. Bao <[email protected]>; Eagle Zhou <[email protected]>; Tao Jiang <[email protected]>; linux- [email protected]; [email protected]; linux-arm- [email protected]
Subject: [EXT] Re: [PATCH v3 1/2] media: add RealVideo format RV30 and RV40

Caution: EXT Email

Quoted 23 lines · Ming Qian

RealVideo, or also spelled as Real Video, is a suite of proprietary video compression formats developed by RealNetworks - the specific format changes with the version.
RealVideo codecs are identified by four-character codes. RV30 and RV40 are RealNetworks' proprietary H.264-based codecs.

---
 .../media/v4l/pixfmt-compressed.rst           | 21 +++++++++++++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c          |  2 ++
 include/uapi/linux/videodev2.h                |  2 ++
 3 files changed, 25 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst index a0230f357680..50ac095c1f16 100644

Documentation/userspace-api/media/v4l/pixfmt-compressed.rst +0 -0
@@ -237,6 +237,27 @@ Compressed Formats
         Metadata associated with the frame to decode is required to be passed
         through the ``V4L2_CID_STATELESS_FWHT_PARAMS`` control.
      See the :ref:`associated Codec Control ID <codec-stateless-fwht>`.

Documentation/userspace-api/media/v4l/pixfmt-compressed.rst Compressed Formats · line 249

Quoted 10 lines
Documentation/userspace-api/media/v4l/pixfmt-compressed.rst +10 -0
+    * .. _V4L2-PIX-FMT-RV30:
+
+      - ``V4L2_PIX_FMT_RV30``
+      - 'RV30'
+      - RealVideo, or also spelled as Real Video, is a suite of proprietary
+        video compression formats developed by RealNetworks -
+        the specific format changes with the version.
+        RealVideo codecs are identified by four-character codes.
+        RV30 corresponds to RealVideo 8, suspected to based largely
+ on an early draft of H.264

to *be* based on.

Just a style comment too, but the flow could be improved while fixing this typo.
Lines in this paragraph seems very uneven.

Documentation/userspace-api/media/v4l/pixfmt-compressed.rst Compressed Formats · line 260

Quoted 12 lines
Documentation/userspace-api/media/v4l/pixfmt-compressed.rst +12 -0
+    * .. _V4L2-PIX-FMT-RV40:
+
+      - ``V4L2_PIX_FMT_RV40``
+      - 'RV40'
+      - RV40 represents RealVideo 9 and RealVideo 10.
+        RealVideo 9, suspected to be based on H.264,
+        RealVideo 10, aka RV9 EHQ, This refers to
+        an improved encoder for the RV9 format
+        that is fully backwards compatible with RV9 players -
+        the format and decoder did not change,
+        only the encoder did.
+        As a result, it uses the same FourCC.
Quoted 4 lines

It feels this one could be wider ? But again, this is cosmetic at this point. For the API changes, assuming fixed typo and improved style:

Hi Nicolas,
I'll fix the typo and improve style in v4 patch,
And for RV40, maybe a new fourcc like RV40E can be defined for encoding RealVideo 10. But this patch won't use this format, some encoder driver can add it in the future if it can support encoding the RealVideo 10

Ming

Quoted 39 lines

drivers/media/v4l2-core/v4l2-ioctl.c static void v4l_fill_fmtdesc(struct v4l2_fmtdesc

Quoted 10 lines
Documentation/userspace-api/media/v4l/pixfmt-compressed.rst +0 -0
 .. raw:: latex
 
drivers/media/v4l2-core/v4l2-ioctl.c +0 -0
index 9b5b04b8aa69..0546b00d3fc9 100644
@@ -1473,6 +1473,8 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc

*fmt)

drivers/media/v4l2-core/v4l2-ioctl.c static void v4l_fill_fmtdesc(struct v4l2_fmtdesc · line 1473

Quoted 1 line
drivers/media/v4l2-core/v4l2-ioctl.c +0 -0
              case V4L2_PIX_FMT_FWHT:         descr = "FWHT"; break; /* used in

vicodec */

drivers/media/v4l2-core/v4l2-ioctl.c static void v4l_fill_fmtdesc(struct v4l2_fmtdesc · line 1474

Quoted 1 line
drivers/media/v4l2-core/v4l2-ioctl.c +0 -0
              case V4L2_PIX_FMT_FWHT_STATELESS:       descr = "FWHT Stateless";

break; /* used in vicodec */

Quoted 12 lines
drivers/media/v4l2-core/v4l2-ioctl.c +2 -0
              case V4L2_PIX_FMT_SPK:          descr = "Sorenson Spark"; break;
+             case V4L2_PIX_FMT_RV30:         descr = "RealVideo 8"; break;
+             case V4L2_PIX_FMT_RV40:         descr = "RealVideo 9 & 10"; break;
              case V4L2_PIX_FMT_CPIA1:        descr = "GSPCA CPiA YUV"; break;
              case V4L2_PIX_FMT_WNVA:         descr = "WNVA"; break;
              case V4L2_PIX_FMT_SN9C10X:      descr = "GSPCA SN9C10X"; break;
(quoted diff) +0 -0

b/include/uapi/linux/videodev2.h index 262ef10cfa02..b121154a6e24 100644

include/uapi/linux/videodev2.h +0 -0
@@ -740,6 +740,8 @@ struct v4l2_pix_format {  #define
Quoted 2 lines

V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */ #define V4L2_PIX_FMT_HEVC_SLICE v4l2_fourcc('S', '2', '6', '5') /*

HEVC parsed slices */

Quoted 1 line

#define V4L2_PIX_FMT_SPK v4l2_fourcc('S', 'P', 'K', '0') /* Sorenson Spark

*/

Quoted 1 line

+#define V4L2_PIX_FMT_RV30 v4l2_fourcc('R', 'V', '3', '0') /* RealVideo 8

*/

Quoted 1 line

+#define V4L2_PIX_FMT_RV40 v4l2_fourcc('R', 'V', '4', '0') /* RealVideo 9 &

10 */

Quoted 3 lines

/* Vendor-specific formats */
#define V4L2_PIX_FMT_CPIA1 v4l2_fourcc('C', 'P', 'I', 'A') /* cpia1 YUV */

author maintainer
#4 lore raw

PATCH v3 2/2 media: amphion: support to decode RealVideo video

In reply to Ming Qian

amphion decoder add support for the RealVideo video. RV30 and RV40 are support by amphion decoder.

---
 drivers/media/platform/amphion/vdec.c       | 14 +++++++++++++
 drivers/media/platform/amphion/vpu_malone.c | 23 +++++++++++++++++++++
 drivers/media/platform/amphion/vpu_malone.h |  1 +
 3 files changed, 38 insertions(+)
drivers/media/platform/amphion/vdec.c +14 -0
index 09304b96f40d..03477998b9eb 100644
@@ -172,6 +172,20 @@ static const struct vpu_format vdec_formats[] = {
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
 		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
 	},
+	{
+		.pixfmt = V4L2_PIX_FMT_RV30,
+		.mem_planes = 1,
+		.comp_planes = 1,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
+	},
+	{
+		.pixfmt = V4L2_PIX_FMT_RV40,
+		.mem_planes = 1,
+		.comp_planes = 1,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+		.flags = V4L2_FMT_FLAG_DYN_RESOLUTION | V4L2_FMT_FLAG_COMPRESSED
+	},
 	{0, 0, 0, 0},
 };
 
drivers/media/platform/amphion/vpu_malone.c +23 -0
index 67ba637c4c7f..26039741166a 100644
@@ -68,6 +68,8 @@
 #define STREAM_CONFIG_NUM_DBE_SET(x, y)		CONFIG_SET(x, y, 30, 0x40000000)
 #define STREAM_CONFIG_FS_CTRL_MODE_SET(x, y)	CONFIG_SET(x, y, 31, 0x80000000)
 
+#define MALONE_DEC_FMT_RV_MASK			BIT(21)
+
 enum vpu_malone_stream_input_mode {
 	INVALID_MODE = 0,
 	FRAME_LVL,
@@ -478,6 +480,9 @@ u32 vpu_malone_get_version(struct vpu_shared_addr *shared)
 {
 	struct malone_iface *iface = shared->iface;
 
+	vpu_malone_enable_format(V4L2_PIX_FMT_RV30, iface->fw_version & MALONE_DEC_FMT_RV_MASK);
+	vpu_malone_enable_format(V4L2_PIX_FMT_RV40, iface->fw_version & MALONE_DEC_FMT_RV_MASK);
+
 	return iface->fw_version;
 }
 
@@ -563,8 +568,22 @@ static struct malone_fmt_mapping fmt_mappings[] = {
 	{V4L2_PIX_FMT_JPEG,        MALONE_FMT_JPG},
 	{V4L2_PIX_FMT_VP8,         MALONE_FMT_VP8},
 	{V4L2_PIX_FMT_SPK,         MALONE_FMT_SPK},
+	{V4L2_PIX_FMT_RV30,        MALONE_FMT_RV},
+	{V4L2_PIX_FMT_RV40,        MALONE_FMT_RV},
 };
 
+void vpu_malone_enable_format(u32 pixelformat, int enable)
+{
+	u32 i;
+
+	for (i = 0; i < ARRAY_SIZE(fmt_mappings); i++) {
+		if (pixelformat == fmt_mappings[i].pixelformat) {
+			fmt_mappings[i].is_disabled = enable ? 0 : 1;
+			return;
+		}
+	}
+}
+
 static enum vpu_malone_format vpu_malone_format_remap(u32 pixelformat)
 {
 	u32 i;
@@ -989,6 +1008,8 @@ static const struct malone_padding_scode padding_scodes[] = {
 	{SCODE_PADDING_EOS,      V4L2_PIX_FMT_H263,        {0xb1010000, 0x0}},
 	{SCODE_PADDING_EOS,      V4L2_PIX_FMT_VP8,         {0x34010000, 0x0}},
 	{SCODE_PADDING_EOS,      V4L2_PIX_FMT_SPK,         {0x34010000, 0x0}},
+	{SCODE_PADDING_EOS,      V4L2_PIX_FMT_RV30,        {0x34010000, 0x0}},
+	{SCODE_PADDING_EOS,      V4L2_PIX_FMT_RV40,        {0x34010000, 0x0}},
 	{SCODE_PADDING_EOS,      V4L2_PIX_FMT_JPEG,        {0xefff0000, 0x0}},
 	{SCODE_PADDING_ABORT,    V4L2_PIX_FMT_H264,        {0x0B010000, 0}},
 	{SCODE_PADDING_ABORT,    V4L2_PIX_FMT_H264_MVC,    {0x0B010000, 0}},
@@ -1001,6 +1022,8 @@ static const struct malone_padding_scode padding_scodes[] = {
 	{SCODE_PADDING_ABORT,    V4L2_PIX_FMT_H263,        {0xb1010000, 0x0}},
 	{SCODE_PADDING_ABORT,    V4L2_PIX_FMT_VP8,         {0x34010000, 0x0}},
 	{SCODE_PADDING_ABORT,    V4L2_PIX_FMT_SPK,         {0x34010000, 0x0}},
+	{SCODE_PADDING_ABORT,    V4L2_PIX_FMT_RV30,        {0x34010000, 0x0}},
+	{SCODE_PADDING_ABORT,    V4L2_PIX_FMT_RV40,        {0x34010000, 0x0}},
 	{SCODE_PADDING_EOS,      V4L2_PIX_FMT_JPEG,        {0x0, 0x0}},
 	{SCODE_PADDING_BUFFLUSH, V4L2_PIX_FMT_H264,        {0x15010000, 0x0}},
 	{SCODE_PADDING_BUFFLUSH, V4L2_PIX_FMT_H264_MVC,    {0x15010000, 0x0}},
drivers/media/platform/amphion/vpu_malone.h +1 -0
index 02a9d9530970..c95b53629199 100644
@@ -41,5 +41,6 @@ int vpu_malone_post_cmd(struct vpu_shared_addr *shared, u32 instance);
 int vpu_malone_init_instance(struct vpu_shared_addr *shared, u32 instance);
 u32 vpu_malone_get_max_instance_count(struct vpu_shared_addr *shared);
 bool vpu_malone_check_fmt(enum vpu_core_type type, u32 pixelfmt);
+void vpu_malone_enable_format(u32 pixelformat, int enable);
 
 #endif
Signature · 7 lines
2.38.1


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel