Skip to contentSkip to conversation

Add MPEG-2 decoding to Rockchip VPU

PATCH Superseded Ezequiel Garcia 3 participants 17 messages Feb 5, 2019 – Feb 21, 2019

Subsystems (3)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover Add MPEG-2 decoding to Rockchip VPU

This series introduces the decoding infrastructure that will be used to add support for other codecs such as VP8, VP9 and H.264.

There are a number of v4l2-compliance issues, which I
will be addressing shortly. For those wanting to take
a peep, I've pasted the v4l2-compliance issues here: http://ix.io/1AfJ.

Minor issues aside, "release early" they say, so here it is!

About the JPEG bounce buffer, we can probably get rid of it, but for now, it's fine to keep it as it won't affect video decoding.

About the media controller topology change, which is no doubt the nastiest change in this series, it's important to mention that we need multiple video interface nodes: one video node for encoding, and one video node for decoding.

Since the VPU is a single engine, it needs a single mem2mem device to serialize the codec jobs thru it, taking advantage of the whole mem2mem infrastructure.

This works perfectly well, but requires a somewhat involved topology. For the encoder the graph goes like this:

┌────────────────────────────────┐
│ rockchip,rk3399-vpu-enc-source │
│          /dev/video0           │
└────────────────────────────────┘
  ┃
  ┃
  ▼
┌────────────────────────────────┐
│  rockchip,rk3399-vpu-enc-proc  │
└────────────────────────────────┘
  ┃
  ┃
  ▼
┌────────────────────────────────┐
│  rockchip,rk3399-vpu-enc-sink  │
│          /dev/video0           │
└────────────────────────────────┘

For the decoder the graph goes like this:

┌────────────────────────────────┐
│ rockchip,rk3399-vpu-dec-source │
│          /dev/video1           │
└────────────────────────────────┘
  ┃
  ┃
  ▼
┌────────────────────────────────┐
│  rockchip,rk3399-vpu-dec-proc  │
└────────────────────────────────┘
  ┃
  ┃
  ▼
┌────────────────────────────────┐
│  rockchip,rk3399-vpu-dec-sink  │
│          /dev/video1           │
└────────────────────────────────┘

Both are tied to the same media device,
i.e. /dev/media0. Does this comply with the media
controller specification?

The pixel format helpers deserve a discussion of its own. Note that these helpers will be useful for probably most V4L drivers. See for instance, this recent bug caused by bad math in vivid's sizeimage calculations:

https://patchwork.kernel.org/patch/10796201/

Finally, I have to thank Jonas Karlman, who did the initial MPEG-2 decoding work and also for getting mpv+ffmpeg to work using the Request API.

This driver can be tested using mpv+ffmpeg for the video decoding side, and the Panfrost mesa driver for rendering.

I should be posting instructions to set all of this up, and also will be submitting the support for H264, VP8 and VP9, hopefully very soon.

Ezequiel Garcia (9):
media: Introduce helpers to fill pixel format structs rockchip/vpu: Use pixel format helpers rockchip/vpu: Use v4l2_m2m_buf_copy_data rockchip/vpu: Cleanup macroblock alignment rockchip/vpu: Cleanup JPEG bounce buffer management rockchip/vpu: Open-code media controller register rockchip/vpu: Support the Request API rockchip/vpu: Add decoder boilerplate rockchip/vpu: Add support for non-standard controls

Jonas Karlman (1):
rockchip/vpu: Add support for MPEG-2 decoding

 drivers/media/v4l2-core/Makefile              |   2 +-
 drivers/media/v4l2-core/v4l2-common.c         |  71 +++
 drivers/media/v4l2-core/v4l2-fourcc.c         | 109 ++++
 drivers/staging/media/rockchip/vpu/Makefile   |   5 +-
 .../media/rockchip/vpu/rk3288_vpu_hw.c        |   4 +-
 .../rockchip/vpu/rk3288_vpu_hw_jpeg_enc.c     |   4 +-
 .../media/rockchip/vpu/rk3399_vpu_hw.c        |  61 +-
 .../rockchip/vpu/rk3399_vpu_hw_jpeg_enc.c     |  10 +-
 .../rockchip/vpu/rk3399_vpu_hw_mpeg2_dec.c    | 263 +++++++++
 .../staging/media/rockchip/vpu/rockchip_vpu.h | 115 +++-
 .../media/rockchip/vpu/rockchip_vpu_common.h  |  10 +
 .../media/rockchip/vpu/rockchip_vpu_dec.c     | 557 ++++++++++++++++++
 .../media/rockchip/vpu/rockchip_vpu_drv.c     | 426 ++++++++++++--
 .../media/rockchip/vpu/rockchip_vpu_enc.c     | 152 ++---
 .../media/rockchip/vpu/rockchip_vpu_hw.h      |  42 ++
 .../media/rockchip/vpu/rockchip_vpu_jpeg.c    |  25 +
 .../media/rockchip/vpu/rockchip_vpu_mpeg2.c   |  61 ++
 include/media/v4l2-common.h                   |   5 +
 include/media/v4l2-fourcc.h                   |  53 ++
 19 files changed, 1803 insertions(+), 172 deletions(-)
 create mode 100644 drivers/media/v4l2-core/v4l2-fourcc.c
 create mode 100644 drivers/staging/media/rockchip/vpu/rk3399_vpu_hw_mpeg2_dec.c
 create mode 100644 drivers/staging/media/rockchip/vpu/rockchip_vpu_dec.c
 create mode 100644 drivers/staging/media/rockchip/vpu/rockchip_vpu_mpeg2.c
 create mode 100644 include/media/v4l2-fourcc.h
Signature · 7 lines
2.20.1


_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip
#1 lore raw

PATCH 1/10 media: Introduce helpers to fill pixel format structs

Add two new API helpers, v4l2_fill_pixfmt and v4l2_fill_pixfmt_mp, to be used by drivers to calculate plane sizes and bytes per lines.

Note that driver-specific paddig and alignment are not taken into account, and must be done by drivers using this API.

---
 drivers/media/v4l2-core/Makefile      |   2 +-
 drivers/media/v4l2-core/v4l2-common.c |  71 +++++++++++++++++
 drivers/media/v4l2-core/v4l2-fourcc.c | 109 ++++++++++++++++++++++++++
 include/media/v4l2-common.h           |   5 ++
 include/media/v4l2-fourcc.h           |  53 +++++++++++++
 5 files changed, 239 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/v4l2-core/v4l2-fourcc.c
 create mode 100644 include/media/v4l2-fourcc.h
drivers/media/v4l2-core/Makefile +1 -1
index 9ee57e1efefe..bc23c3407c17 100644
@@ -7,7 +7,7 @@ tuner-objs	:=	tuner-core.o
 
 videodev-objs	:=	v4l2-dev.o v4l2-ioctl.o v4l2-device.o v4l2-fh.o \
 			v4l2-event.o v4l2-ctrls.o v4l2-subdev.o v4l2-clk.o \
-			v4l2-async.o
+			v4l2-async.o v4l2-fourcc.o
 ifeq ($(CONFIG_COMPAT),y)
   videodev-objs += v4l2-compat-ioctl32.o
 endif
drivers/media/v4l2-core/v4l2-common.c +71 -0

80 lines not on this page · raw message

drivers/media/v4l2-core/v4l2-fourcc.c new file +109 -0

109 lines not on this page · raw message

include/media/v4l2-common.h +5 -0
index 0c511ed8ffb0..6461ce747d90 100644
@@ -327,6 +327,11 @@ void v4l_bound_align_image(unsigned int *width, unsigned int wmin,
 			   unsigned int hmax, unsigned int halign,
 			   unsigned int salign);
 
+void v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, int pixelformat,
+		      int width, int height);
+void v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt, int pixelformat,
+			 int width, int height);
+
 /**
  * v4l2_find_nearest_size - Find the nearest size among a discrete
  *	set of resolutions contained in an array of a driver specific struct.
include/media/v4l2-fourcc.h new file +53 -0

53 lines not on this page · raw message

Signature · 1 line
2.20.1
#2 lore raw

Re: [PATCH 01/10] media: Introduce helpers to fill pixel format structs

Hi Ezequiel,

A quick review below. This looks really useful, BTW.

Quoted 4 lines · Ezequiel Garcia

Add two new API helpers, v4l2_fill_pixfmt and v4l2_fill_pixfmt_mp, to be used by drivers to calculate plane sizes and bytes per lines.

Note that driver-specific paddig and alignment are not

paddig -> padding

Quoted 31 lines

taken into account, and must be done by drivers using this API.

---
 drivers/media/v4l2-core/Makefile      |   2 +-
 drivers/media/v4l2-core/v4l2-common.c |  71 +++++++++++++++++
 drivers/media/v4l2-core/v4l2-fourcc.c | 109 ++++++++++++++++++++++++++
 include/media/v4l2-common.h           |   5 ++
 include/media/v4l2-fourcc.h           |  53 +++++++++++++
 5 files changed, 239 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/v4l2-core/v4l2-fourcc.c
 create mode 100644 include/media/v4l2-fourcc.h
drivers/media/v4l2-core/Makefile +1 -1
index 9ee57e1efefe..bc23c3407c17 100644
@@ -7,7 +7,7 @@ tuner-objs	:=	tuner-core.o
 
 videodev-objs	:=	v4l2-dev.o v4l2-ioctl.o v4l2-device.o v4l2-fh.o \
 			v4l2-event.o v4l2-ctrls.o v4l2-subdev.o v4l2-clk.o \
-			v4l2-async.o
+			v4l2-async.o v4l2-fourcc.o
 ifeq ($(CONFIG_COMPAT),y)
   videodev-objs += v4l2-compat-ioctl32.o
 endif
drivers/media/v4l2-core/v4l2-common.c +0 -0
index 50763fb42a1b..39d86a389cae 100644

drivers/media/v4l2-core/v4l2-common.c line 64

Quoted 4 lines
drivers/media/v4l2-core/v4l2-common.c +1 -0
@@ -61,6 +61,7 @@
 #include <media/v4l2-common.h>
 #include <media/v4l2-device.h>
 #include <media/v4l2-ctrls.h>
+#include <media/v4l2-fourcc.h>

Either create a v4l2-fourcc.c source using this header, or move the contents of v4l2-fourcc.h to v4l2-common.h.

Creating a new header but not a new source is a bit weird.

Quoted 8 lines
drivers/media/v4l2-core/v4l2-common.c +1 -0
 #include <linux/videodev2.h>
 
@@ -455,3 +456,73 @@ int v4l2_s_parm_cap(struct video_device *vdev,
 	return ret;
 }
 EXPORT_SYMBOL_GPL(v4l2_s_parm_cap);
+

drivers/media/v4l2-core/v4l2-common.c int v4l2_s_parm_cap(struct video_device *vdev, · line 469

Quoted 10 lines
drivers/media/v4l2-core/v4l2-common.c +10 -0
+void v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
+			 int pixelformat, int width, int height)
+{
+	const struct v4l2_format_info *info;
+	struct v4l2_plane_pix_format *plane;
+	int i;
+
+	info = v4l2_format_info(pixelformat);
+	if (!info)
+		return;

You should return a bool or something to indicate whether or not the pixelformat was known.

drivers/media/v4l2-core/v4l2-common.c int v4l2_s_parm_cap(struct video_device *vdev, · line 475

Quoted 6 lines
drivers/media/v4l2-core/v4l2-common.c +6 -0
+
+	pixfmt->width = width;
+	pixfmt->height = height;
+	pixfmt->pixelformat = pixelformat;
+
+	if (!info->multiplanar) {

It would make much more sense if multiplanar contained the number of planes to use (i.e. equal to pixfmt->num_planes).

See more about this below.

Quoted 26 lines
drivers/media/v4l2-core/v4l2-common.c +26 -0
+		pixfmt->num_planes = 1;
+		plane = &pixfmt->plane_fmt[0];
+		plane->bytesperline = width * info->cpp[0];
+		plane->sizeimage = 0;
+		for (i = 0; i < info->num_planes; i++) {
+			unsigned int hsub = (i == 0) ? 1 : info->hsub;
+			unsigned int vsub = (i == 0) ? 1 : info->vsub;
+
+			plane->sizeimage += info->cpp[i] *
+				DIV_ROUND_UP(width, hsub) *
+				DIV_ROUND_UP(height, vsub);
+		}
+	} else {
+		pixfmt->num_planes = info->num_planes;
+		for (i = 0; i < info->num_planes; i++) {
+			unsigned int hsub = (i == 0) ? 1 : info->hsub;
+			unsigned int vsub = (i == 0) ? 1 : info->vsub;
+
+			plane = &pixfmt->plane_fmt[i];
+			plane->bytesperline =
+				info->cpp[i] * DIV_ROUND_UP(width, hsub);
+			plane->sizeimage =
+				plane->bytesperline * DIV_ROUND_UP(height, vsub);
+		}
+	}
+}

drivers/media/v4l2-core/v4l2-common.c int v4l2_s_parm_cap(struct video_device *vdev, · line 511

Quoted 10 lines
drivers/media/v4l2-core/v4l2-common.c +10 -0
+EXPORT_SYMBOL_GPL(v4l2_fill_pixfmt_mp);
+
+void v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, int pixelformat, int width, int height)
+{
+	const struct v4l2_format_info *info;
+	int i;
+
+	info = v4l2_format_info(pixelformat);
+	if (!info)
+		return;

You have to check if pixelformat was a multiplanar format and reject it.

Quoted 48 lines
drivers/media/v4l2-core/v4l2-common.c +17 -0
+
+	pixfmt->width = width;
+	pixfmt->height = height;
+	pixfmt->pixelformat = pixelformat;
+	pixfmt->bytesperline = width * info->cpp[0];
+	pixfmt->sizeimage = 0;
+
+	for (i = 0; i < info->num_planes; i++) {
+		unsigned int hsub = (i == 0) ? 1 : info->hsub;
+		unsigned int vsub = (i == 0) ? 1 : info->vsub;
+
+		pixfmt->sizeimage += info->cpp[i] *
+			DIV_ROUND_UP(width, hsub) *
+			DIV_ROUND_UP(height, vsub);
+	}
+}
+EXPORT_SYMBOL_GPL(v4l2_fill_pixfmt);
drivers/media/v4l2-core/v4l2-fourcc.c new file +25 -0
new file mode 100644
index 000000000000..982c0ffa1a66
@@ -0,0 +1,109 @@
+/*
+ * Copyright (c) 2018 Collabora, Ltd.
+ *
+ * Based on drm-fourcc:
+ * Copyright (c) 2016 Laurent Pinchart <laurent.pinchart-ryLnwIuWjnjg/[email protected]>
+ *
+ * Permission to use, copy, modify, distribute, and sell this software and its
+ * documentation for any purpose is hereby granted without fee, provided that
+ * the above copyright notice appear in all copies and that both that copyright
+ * notice and this permission notice appear in supporting documentation, and
+ * that the name of the copyright holders not be used in advertising or
+ * publicity pertaining to distribution of the software without specific,
+ * written prior permission.  The copyright holders make no representations
+ * about the suitability of this software for any purpose.  It is provided "as
+ * is" without express or implied warranty.
+ *
+ * THE COPYRIGHT HOLDERS DISCLAIM ALL WARRANTIES WITH REGARD TO THIS SOFTWARE,
+ * INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN NO
+ * EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY SPECIAL, INDIRECT OR
+ * CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE,
+ * DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
+ * TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE
+ * OF THIS SOFTWARE.
+ */
+

drivers/media/v4l2-core/v4l2-fourcc.c line 35

Quoted 10 lines
drivers/media/v4l2-core/v4l2-fourcc.c new file +10 -0
+#include <linux/ctype.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-fourcc.h>
+
+static char printable_char(int c)
+{
+	return isascii(c) && isprint(c) ? c : '?';
+}
+
+const char *v4l2_get_format_name(uint32_t format)

This should be called v4l2_get_fourcc_name. The format name is what ENUMFMT returns.

drivers/media/v4l2-core/v4l2-fourcc.c line 44

Quoted 9 lines
drivers/media/v4l2-core/v4l2-fourcc.c new file +9 -0
+{
+	static char buf[4];
+
+	snprintf(buf, 4,
+		 "%c%c%c%c",
+		 printable_char(format & 0xff),
+		 printable_char((format >> 8) & 0xff),
+		 printable_char((format >> 16) & 0xff),
+		 printable_char((format >> 24) & 0x7f));

If bit 31 is set, then add a '-BE' suffix to indicate that this is a big endian variant of the same pixelformat with bit 31 set to 0.

See also v4l_fill_fmtdesc() in v4l2-ioctl.c.

drivers/media/v4l2-core/v4l2-fourcc.c line 48

Quoted 4 lines
drivers/media/v4l2-core/v4l2-fourcc.c new file +4 -0
+
+	return buf;
+}
+EXPORT_SYMBOL(v4l2_get_format_name);

I remember that Sakari tried to make a macro for this in a public header, but it was either rejected or fizzled out:

https://www.mail-archive.com/[email protected]/msg128702.html

Quoted 46 lines
drivers/media/v4l2-core/v4l2-fourcc.c new file +46 -0
+
+const struct v4l2_format_info *v4l2_format_info(u32 format)
+{
+	static const struct v4l2_format_info formats[] = {
+		/* RGB formats */
+		{ .format = V4L2_PIX_FMT_BGR24,		.num_planes = 1, .cpp = { 3, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_RGB24,		.num_planes = 1, .cpp = { 3, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_HSV24,		.num_planes = 1, .cpp = { 3, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_BGR32,		.num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_XBGR32,	.num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_RGB32,		.num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_XRGB32,	.num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_HSV32,		.num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_ARGB32,	.num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_ABGR32,	.num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_GREY,		.num_planes = 1, .cpp = { 1, 0, 0 }, .hsub = 1, .vsub = 1 },
+
+		/* YUV formats */
+		{ .format = V4L2_PIX_FMT_YUYV,		.num_planes = 1, .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_YVYU,		.num_planes = 1, .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_UYVY,		.num_planes = 1, .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_VYUY,		.num_planes = 1, .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
+
+		{ .format = V4L2_PIX_FMT_NV12,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 2 },
+		{ .format = V4L2_PIX_FMT_NV21,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 2 },
+		{ .format = V4L2_PIX_FMT_NV16,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_NV61,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_NV24,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_NV42,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 1, .vsub = 1 },
+
+		{ .format = V4L2_PIX_FMT_YUV410,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 4, .vsub = 4 },
+		{ .format = V4L2_PIX_FMT_YVU410,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 4, .vsub = 4 },
+		{ .format = V4L2_PIX_FMT_YUV411P,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 4, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_YUV420,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 2, .vsub = 2 },
+		{ .format = V4L2_PIX_FMT_YVU420,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 2, .vsub = 2 },
+		{ .format = V4L2_PIX_FMT_YUV422P,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 2, .vsub = 1 },
+
+		{ .format = V4L2_PIX_FMT_YUV420M,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 2, .vsub = 2, .multiplanar = 1 },
+		{ .format = V4L2_PIX_FMT_YVU420M,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 2, .vsub = 2, .multiplanar = 1 },
+		{ .format = V4L2_PIX_FMT_YUV422M,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 2, .vsub = 1, .multiplanar = 1 },
+		{ .format = V4L2_PIX_FMT_YVU422M,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 2, .vsub = 1, .multiplanar = 1 },
+		{ .format = V4L2_PIX_FMT_YUV444M,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 1, .vsub = 1, .multiplanar = 1 },
+		{ .format = V4L2_PIX_FMT_YVU444M,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 1, .vsub = 1, .multiplanar = 1 },
+
+		{ .format = V4L2_PIX_FMT_NV12M,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 2, .multiplanar = 1 },
+		{ .format = V4L2_PIX_FMT_NV21M,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 2, .multiplanar = 1 },

drivers/media/v4l2-core/v4l2-fourcc.c line 104

Quoted 10 lines
drivers/media/v4l2-core/v4l2-fourcc.c new file +10 -0
+		{ .format = V4L2_PIX_FMT_NV16M,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 1, .multiplanar = 1 },
+		{ .format = V4L2_PIX_FMT_NV61M,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 1, .multiplanar = 1 },
+
+	};
+	unsigned int i;
+
+	for (i = 0; i < ARRAY_SIZE(formats); ++i) {
+		if (formats[i].format == format)
+			return &formats[i];
+	}

No need for {}

drivers/media/v4l2-core/v4l2-fourcc.c line 106

Quoted 2 lines
drivers/media/v4l2-core/v4l2-fourcc.c new file +2 -0
+
+	pr_warn("Unsupported V4L 4CC format %s (%08x)\n", v4l2_get_format_name(format), format);

Make this a pr_dev or remove it altogether. I prefer the latter.

Quoted 38 lines
drivers/media/v4l2-core/v4l2-fourcc.c new file +3 -0
+	return NULL;
+}
+EXPORT_SYMBOL(v4l2_format_info);
include/media/v4l2-common.h +5 -0
index 0c511ed8ffb0..6461ce747d90 100644
@@ -327,6 +327,11 @@ void v4l_bound_align_image(unsigned int *width, unsigned int wmin,
 			   unsigned int hmax, unsigned int halign,
 			   unsigned int salign);
 
+void v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, int pixelformat,
+		      int width, int height);
+void v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt, int pixelformat,
+			 int width, int height);
+
 /**
  * v4l2_find_nearest_size - Find the nearest size among a discrete
  *	set of resolutions contained in an array of a driver specific struct.
include/media/v4l2-fourcc.h new file +13 -0
new file mode 100644
index 000000000000..3d24f442aaf5
@@ -0,0 +1,53 @@
+/*
+ * Copyright (c) 2018 Collabora, Ltd.
+ *
+ * Based on drm-fourcc:
+ * Copyright (c) 2016 Laurent Pinchart <laurent.pinchart-ryLnwIuWjnjg/[email protected]>
+ *
+ * Permission to use, copy, modify, distribute, and sell this software and its
+ * documentation for any purpose is hereby granted without fee, provided that
+ * the above copyright notice appear in all copies and that both that copyright
+ * notice and this permission notice appear in supporting documentation, and
+ * that the name of the copyright holders not be used in advertising or
+ * publicity pertaining to distribution of the software without specific,
+ * written prior permission.  The copyright holders make no representations

include/media/v4l2-fourcc.h line 23

Quoted 10 lines
include/media/v4l2-fourcc.h new file +10 -0
+ * about the suitability of this software for any purpose.  It is provided "as
+ * is" without express or implied warranty.
+ *
+ * THE COPYRIGHT HOLDERS DISCLAIM ALL WARRANTIES WITH REGARD TO THIS SOFTWARE,
+ * INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN NO
+ * EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY SPECIAL, INDIRECT OR
+ * CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE,
+ * DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
+ * TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE
+ * OF THIS SOFTWARE.

Should be an SPDX ID.

include/media/v4l2-fourcc.h line 34

Quoted 11 lines
include/media/v4l2-fourcc.h new file +11 -0
+ */
+#ifndef __V4L2_FOURCC_H__
+#define __V4L2_FOURCC_H__
+
+#include <linux/types.h>
+
+/**
+ * struct v4l2_format_info - information about a V4L2 format
+ * @format: 4CC format identifier (V4L2_PIX_FMT_*)
+ * @header_size: Size of header, optional and used by compressed formats
+ * @num_planes: Number of planes (1 to 3)

This is actually 1-4 since there may be an alpha channel as well. Not that we have such formats since the only formats with an alpha channel are interleaved formats, but it is possible.

So this value is 2 for both NV12 and NV12M.

include/media/v4l2-fourcc.h line 35

Quoted 1 line
include/media/v4l2-fourcc.h new file +1 -0
+ * @cpp: Number of bytes per pixel (per plane)

cpp? Shouldn't that be bpp?

Note that this can differ per plane (see e.g. NV24).

include/media/v4l2-fourcc.h line 37

Quoted 2 lines
include/media/v4l2-fourcc.h new file +2 -0
+ * @hsub: Horizontal chroma subsampling factor
+ * @vsub: Vertical chroma subsampling factor

A bit too cryptic IMHO. I would prefer hdiv or hsubsampling. 'hsub' suggests subtraction :-)

include/media/v4l2-fourcc.h line 38

Quoted 1 line
include/media/v4l2-fourcc.h new file +1 -0
+ * @multiplanar: Is it a multiplanar variant format? (e.g. NV12M)

This should, I think, be renamed to num_non_contig_planes to indicate how many non-contiguous planes there are in the format.

So this value is 1 for NV12 and 2 for NV12M. For V4L2_PIX_FMT_YUV444M it is 3.

You can stick this value directly into pixfmt_mp->num_planes.

As an aside: perhaps we should start calling the 'multiplanar API' the 'multiple non-contiguous planes API', at least in the documentation. It's the first time that I found a description that actually covers the real meaning.

Quoted 6 lines
include/media/v4l2-fourcc.h new file +5 -0
+ */
+struct v4l2_format_info {
+	u32 format;
+	u32 header_size;
+	u8 num_planes;

include/media/v4l2-fourcc.h line 53

Quoted 10 lines
include/media/v4l2-fourcc.h new file +10 -0
+	u8 cpp[3];
+	u8 hsub;
+	u8 vsub;
+	u8 multiplanar;
+};
+
+const struct v4l2_format_info *v4l2_format_info(u32 format);
+const char *v4l2_get_format_name(u32 format);
+
+#endif

Regards,

Hans

#3 lore raw

Re: [PATCH 01/10] media: Introduce helpers to fill pixel format structs

Quoted 51 lines · Hans Verkuil

Hi Ezequiel,

A quick review below. This looks really useful, BTW.

Quoted 4 lines · Ezequiel Garcia

Add two new API helpers, v4l2_fill_pixfmt and v4l2_fill_pixfmt_mp, to be used by drivers to calculate plane sizes and bytes per lines.

Note that driver-specific paddig and alignment are not

paddig -> padding

Quoted 31 lines

taken into account, and must be done by drivers using this API.

---
 drivers/media/v4l2-core/Makefile      |   2 +-
 drivers/media/v4l2-core/v4l2-common.c |  71 +++++++++++++++++
 drivers/media/v4l2-core/v4l2-fourcc.c | 109 ++++++++++++++++++++++++++
 include/media/v4l2-common.h           |   5 ++
 include/media/v4l2-fourcc.h           |  53 +++++++++++++
 5 files changed, 239 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/v4l2-core/v4l2-fourcc.c
 create mode 100644 include/media/v4l2-fourcc.h
drivers/media/v4l2-core/Makefile +1 -1
index 9ee57e1efefe..bc23c3407c17 100644
@@ -7,7 +7,7 @@ tuner-objs	:=	tuner-core.o
 
 videodev-objs	:=	v4l2-dev.o v4l2-ioctl.o v4l2-device.o v4l2-fh.o \
 			v4l2-event.o v4l2-ctrls.o v4l2-subdev.o v4l2-clk.o \
-			v4l2-async.o
+			v4l2-async.o v4l2-fourcc.o
 ifeq ($(CONFIG_COMPAT),y)
   videodev-objs += v4l2-compat-ioctl32.o
 endif
drivers/media/v4l2-core/v4l2-common.c +0 -0
index 50763fb42a1b..39d86a389cae 100644

drivers/media/v4l2-core/v4l2-common.c line 64

Quoted 4 lines
drivers/media/v4l2-core/v4l2-common.c +1 -0
@@ -61,6 +61,7 @@
 #include <media/v4l2-common.h>
 #include <media/v4l2-device.h>
 #include <media/v4l2-ctrls.h>
+#include <media/v4l2-fourcc.h>

Either create a v4l2-fourcc.c source using this header, or move the contents of v4l2-fourcc.h to v4l2-common.h.

Quoted 2 lines

Creating a new header but not a new source is a bit weird.

This patch adds v4l2-fourcc.c and v4l2-fourcc.h, containing only the pixel format array and the fourcc name helper.

Quoted 13 lines
Quoted 13 lines
drivers/media/v4l2-core/v4l2-common.c +6 -0
 #include <linux/videodev2.h>
 
@@ -455,3 +456,73 @@ int v4l2_s_parm_cap(struct video_device *vdev,
 	return ret;
 }
 EXPORT_SYMBOL_GPL(v4l2_s_parm_cap);
+
+void v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
+			 int pixelformat, int width, int height)
+{
+	const struct v4l2_format_info *info;
+	struct v4l2_plane_pix_format *plane;
Quoted 9 lines

drivers/media/v4l2-core/v4l2-common.c int v4l2_s_parm_cap(struct video_device *vdev, · line 469

Quoted 5 lines
drivers/media/v4l2-core/v4l2-common.c +5 -0
+	int i;
+
+	info = v4l2_format_info(pixelformat);
+	if (!info)
+		return;

You should return a bool or something to indicate whether or not the pixelformat was known.

Got it.

Quoted 42 lines

drivers/media/v4l2-core/v4l2-common.c int v4l2_s_parm_cap(struct video_device *vdev, · line 475

Quoted 6 lines
drivers/media/v4l2-core/v4l2-common.c +6 -0
+
+	pixfmt->width = width;
+	pixfmt->height = height;
+	pixfmt->pixelformat = pixelformat;
+
+	if (!info->multiplanar) {

It would make much more sense if multiplanar contained the number of planes to use (i.e. equal to pixfmt->num_planes).

See more about this below.

Quoted 30 lines
drivers/media/v4l2-core/v4l2-common.c +30 -0
+		pixfmt->num_planes = 1;
+		plane = &pixfmt->plane_fmt[0];
+		plane->bytesperline = width * info->cpp[0];
+		plane->sizeimage = 0;
+		for (i = 0; i < info->num_planes; i++) {
+			unsigned int hsub = (i == 0) ? 1 : info->hsub;
+			unsigned int vsub = (i == 0) ? 1 : info->vsub;
+
+			plane->sizeimage += info->cpp[i] *
+				DIV_ROUND_UP(width, hsub) *
+				DIV_ROUND_UP(height, vsub);
+		}
+	} else {
+		pixfmt->num_planes = info->num_planes;
+		for (i = 0; i < info->num_planes; i++) {
+			unsigned int hsub = (i == 0) ? 1 : info->hsub;
+			unsigned int vsub = (i == 0) ? 1 : info->vsub;
+
+			plane = &pixfmt->plane_fmt[i];
+			plane->bytesperline =
+				info->cpp[i] * DIV_ROUND_UP(width, hsub);
+			plane->sizeimage =
+				plane->bytesperline * DIV_ROUND_UP(height, vsub);
+		}
+	}
+}
+EXPORT_SYMBOL_GPL(v4l2_fill_pixfmt_mp);
+
+void v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, int pixelformat, int width, int height)
+{
Quoted 9 lines

drivers/media/v4l2-core/v4l2-common.c int v4l2_s_parm_cap(struct video_device *vdev, · line 469

Quoted 6 lines
drivers/media/v4l2-core/v4l2-common.c +6 -0
+	const struct v4l2_format_info *info;
+	int i;
+
+	info = v4l2_format_info(pixelformat);
+	if (!info)
+		return;

You have to check if pixelformat was a multiplanar format and reject it.

OK.

Quoted 52 lines
Quoted 52 lines
drivers/media/v4l2-core/v4l2-common.c +17 -0
+
+	pixfmt->width = width;
+	pixfmt->height = height;
+	pixfmt->pixelformat = pixelformat;
+	pixfmt->bytesperline = width * info->cpp[0];
+	pixfmt->sizeimage = 0;
+
+	for (i = 0; i < info->num_planes; i++) {
+		unsigned int hsub = (i == 0) ? 1 : info->hsub;
+		unsigned int vsub = (i == 0) ? 1 : info->vsub;
+
+		pixfmt->sizeimage += info->cpp[i] *
+			DIV_ROUND_UP(width, hsub) *
+			DIV_ROUND_UP(height, vsub);
+	}
+}
+EXPORT_SYMBOL_GPL(v4l2_fill_pixfmt);
drivers/media/v4l2-core/v4l2-fourcc.c new file +29 -0
new file mode 100644
index 000000000000..982c0ffa1a66
@@ -0,0 +1,109 @@
+/*
+ * Copyright (c) 2018 Collabora, Ltd.
+ *
+ * Based on drm-fourcc:
+ * Copyright (c) 2016 Laurent Pinchart <laurent.pinchart-ryLnwIuWjnjg/[email protected]>
+ *
+ * Permission to use, copy, modify, distribute, and sell this software and its
+ * documentation for any purpose is hereby granted without fee, provided that
+ * the above copyright notice appear in all copies and that both that copyright
+ * notice and this permission notice appear in supporting documentation, and
+ * that the name of the copyright holders not be used in advertising or
+ * publicity pertaining to distribution of the software without specific,
+ * written prior permission.  The copyright holders make no representations
+ * about the suitability of this software for any purpose.  It is provided "as
+ * is" without express or implied warranty.
+ *
+ * THE COPYRIGHT HOLDERS DISCLAIM ALL WARRANTIES WITH REGARD TO THIS SOFTWARE,
+ * INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN NO
+ * EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY SPECIAL, INDIRECT OR
+ * CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE,
+ * DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
+ * TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE
+ * OF THIS SOFTWARE.
+ */
+
+#include <linux/ctype.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-fourcc.h>
+
Quoted 9 lines

drivers/media/v4l2-core/v4l2-fourcc.c line 35

Quoted 6 lines
drivers/media/v4l2-core/v4l2-fourcc.c new file +6 -0
+static char printable_char(int c)
+{
+	return isascii(c) && isprint(c) ? c : '?';
+}
+
+const char *v4l2_get_format_name(uint32_t format)

This should be called v4l2_get_fourcc_name. The format name is what ENUMFMT returns.

Got it.

Quoted 13 lines

drivers/media/v4l2-core/v4l2-fourcc.c line 44

Quoted 9 lines
drivers/media/v4l2-core/v4l2-fourcc.c new file +9 -0
+{
+	static char buf[4];
+
+	snprintf(buf, 4,
+		 "%c%c%c%c",
+		 printable_char(format & 0xff),
+		 printable_char((format >> 8) & 0xff),
+		 printable_char((format >> 16) & 0xff),
+		 printable_char((format >> 24) & 0x7f));

If bit 31 is set, then add a '-BE' suffix to indicate that this is a big endian variant of the same pixelformat with bit 31 set to 0.

Quoted 2 lines

See also v4l_fill_fmtdesc() in v4l2-ioctl.c.

I omitted endianness and alpha because they weren't used by V4L2, but let me add them to make the code generic and support out-of-tree stuff.

Quoted 10 lines

drivers/media/v4l2-core/v4l2-fourcc.c line 48

Quoted 4 lines
drivers/media/v4l2-core/v4l2-fourcc.c new file +4 -0
+
+	return buf;
+}
+EXPORT_SYMBOL(v4l2_get_format_name);

I remember that Sakari tried to make a macro for this in a public header, but it was either rejected or fizzled out:

https://www.mail-archive.com/[email protected]/msg128702.html

I see. This time we are adding the fourcc name helper to the private API, which shouldn't cause too much debate. Adding Sakari to the loop.

Quoted 50 lines
Quoted 50 lines
drivers/media/v4l2-core/v4l2-fourcc.c new file +50 -0
+
+const struct v4l2_format_info *v4l2_format_info(u32 format)
+{
+	static const struct v4l2_format_info formats[] = {
+		/* RGB formats */
+		{ .format = V4L2_PIX_FMT_BGR24,		.num_planes = 1, .cpp = { 3, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_RGB24,		.num_planes = 1, .cpp = { 3, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_HSV24,		.num_planes = 1, .cpp = { 3, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_BGR32,		.num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_XBGR32,	.num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_RGB32,		.num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_XRGB32,	.num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_HSV32,		.num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_ARGB32,	.num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_ABGR32,	.num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_GREY,		.num_planes = 1, .cpp = { 1, 0, 0 }, .hsub = 1, .vsub = 1 },
+
+		/* YUV formats */
+		{ .format = V4L2_PIX_FMT_YUYV,		.num_planes = 1, .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_YVYU,		.num_planes = 1, .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_UYVY,		.num_planes = 1, .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_VYUY,		.num_planes = 1, .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
+
+		{ .format = V4L2_PIX_FMT_NV12,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 2 },
+		{ .format = V4L2_PIX_FMT_NV21,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 2 },
+		{ .format = V4L2_PIX_FMT_NV16,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_NV61,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_NV24,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_NV42,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 1, .vsub = 1 },
+
+		{ .format = V4L2_PIX_FMT_YUV410,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 4, .vsub = 4 },
+		{ .format = V4L2_PIX_FMT_YVU410,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 4, .vsub = 4 },
+		{ .format = V4L2_PIX_FMT_YUV411P,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 4, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_YUV420,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 2, .vsub = 2 },
+		{ .format = V4L2_PIX_FMT_YVU420,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 2, .vsub = 2 },
+		{ .format = V4L2_PIX_FMT_YUV422P,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 2, .vsub = 1 },
+
+		{ .format = V4L2_PIX_FMT_YUV420M,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 2, .vsub = 2, .multiplanar = 1 },
+		{ .format = V4L2_PIX_FMT_YVU420M,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 2, .vsub = 2, .multiplanar = 1 },
+		{ .format = V4L2_PIX_FMT_YUV422M,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 2, .vsub = 1, .multiplanar = 1 },
+		{ .format = V4L2_PIX_FMT_YVU422M,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 2, .vsub = 1, .multiplanar = 1 },
+		{ .format = V4L2_PIX_FMT_YUV444M,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 1, .vsub = 1, .multiplanar = 1 },
+		{ .format = V4L2_PIX_FMT_YVU444M,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 1, .vsub = 1, .multiplanar = 1 },
+
+		{ .format = V4L2_PIX_FMT_NV12M,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 2, .multiplanar = 1 },
+		{ .format = V4L2_PIX_FMT_NV21M,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 2, .multiplanar = 1 },
+		{ .format = V4L2_PIX_FMT_NV16M,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 1, .multiplanar = 1 },
+		{ .format = V4L2_PIX_FMT_NV61M,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 1, .multiplanar = 1 },
+
+	};
Quoted 9 lines

drivers/media/v4l2-core/v4l2-fourcc.c line 104

Quoted 6 lines
drivers/media/v4l2-core/v4l2-fourcc.c new file +6 -0
+	unsigned int i;
+
+	for (i = 0; i < ARRAY_SIZE(formats); ++i) {
+		if (formats[i].format == format)
+			return &formats[i];
+	}

No need for {}

OK

Quoted 5 lines

drivers/media/v4l2-core/v4l2-fourcc.c line 106

Quoted 2 lines
drivers/media/v4l2-core/v4l2-fourcc.c new file +2 -0
+
+	pr_warn("Unsupported V4L 4CC format %s (%08x)\n", v4l2_get_format_name(format), format);

Make this a pr_dev or remove it altogether. I prefer the latter.

Right. If the function returns an error condition, we can drop it.

Quoted 67 lines
Quoted 38 lines
drivers/media/v4l2-core/v4l2-fourcc.c new file +3 -0
+	return NULL;
+}
+EXPORT_SYMBOL(v4l2_format_info);
include/media/v4l2-common.h +5 -0
index 0c511ed8ffb0..6461ce747d90 100644
@@ -327,6 +327,11 @@ void v4l_bound_align_image(unsigned int *width, unsigned int wmin,
 			   unsigned int hmax, unsigned int halign,
 			   unsigned int salign);
 
+void v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, int pixelformat,
+		      int width, int height);
+void v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt, int pixelformat,
+			 int width, int height);
+
 /**
  * v4l2_find_nearest_size - Find the nearest size among a discrete
  *	set of resolutions contained in an array of a driver specific struct.
include/media/v4l2-fourcc.h new file +13 -0
new file mode 100644
index 000000000000..3d24f442aaf5
@@ -0,0 +1,53 @@
+/*
+ * Copyright (c) 2018 Collabora, Ltd.
+ *
+ * Based on drm-fourcc:
+ * Copyright (c) 2016 Laurent Pinchart <laurent.pinchart-ryLnwIuWjnjg/[email protected]>
+ *
+ * Permission to use, copy, modify, distribute, and sell this software and its
+ * documentation for any purpose is hereby granted without fee, provided that
+ * the above copyright notice appear in all copies and that both that copyright
+ * notice and this permission notice appear in supporting documentation, and
+ * that the name of the copyright holders not be used in advertising or
+ * publicity pertaining to distribution of the software without specific,
+ * written prior permission.  The copyright holders make no representations

include/media/v4l2-fourcc.h line 23

Quoted 10 lines
include/media/v4l2-fourcc.h new file +10 -0
+ * about the suitability of this software for any purpose.  It is provided "as
+ * is" without express or implied warranty.
+ *
+ * THE COPYRIGHT HOLDERS DISCLAIM ALL WARRANTIES WITH REGARD TO THIS SOFTWARE,
+ * INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN NO
+ * EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY SPECIAL, INDIRECT OR
+ * CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE,
+ * DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
+ * TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE
+ * OF THIS SOFTWARE.

Should be an SPDX ID.

include/media/v4l2-fourcc.h line 34

Quoted 11 lines
include/media/v4l2-fourcc.h new file +11 -0
+ */
+#ifndef __V4L2_FOURCC_H__
+#define __V4L2_FOURCC_H__
+
+#include <linux/types.h>
+
+/**
+ * struct v4l2_format_info - information about a V4L2 format
+ * @format: 4CC format identifier (V4L2_PIX_FMT_*)
+ * @header_size: Size of header, optional and used by compressed formats
+ * @num_planes: Number of planes (1 to 3)

This is actually 1-4 since there may be an alpha channel as well. Not that we have such formats since the only formats with an alpha channel are interleaved formats, but it is possible.

Quoted 8 lines

So this value is 2 for both NV12 and NV12M.

include/media/v4l2-fourcc.h line 35

Quoted 1 line
include/media/v4l2-fourcc.h new file +1 -0
+ * @cpp: Number of bytes per pixel (per plane)

cpp? Shouldn't that be bpp?

Note that this can differ per plane (see e.g. NV24).

Yes, the comment should specify that it's an array of bytes per pixel.

And the naming follows drm-fourcc. cpp stands for char-per-pixel. It may be confusing but at the same time, I really like to have consistent naming across the board (think for instance in the DMA coherent/consistent aliases).

Also, note that drm-fourcc deprecates cpp, to support tile formats. Hopefully we don't need that here?

Quoted 6 lines

include/media/v4l2-fourcc.h line 37

Quoted 2 lines
include/media/v4l2-fourcc.h new file +2 -0
+ * @hsub: Horizontal chroma subsampling factor
+ * @vsub: Vertical chroma subsampling factor

A bit too cryptic IMHO. I would prefer hdiv or hsubsampling. 'hsub' suggests subtraction :-)

Ditto, this name follows drm-fourcc. I'm fine either way.

Quoted 9 lines

include/media/v4l2-fourcc.h line 38

Quoted 1 line
include/media/v4l2-fourcc.h new file +1 -0
+ * @multiplanar: Is it a multiplanar variant format? (e.g. NV12M)

This should, I think, be renamed to num_non_contig_planes to indicate how many non-contiguous planes there are in the format.

So this value is 1 for NV12 and 2 for NV12M. For V4L2_PIX_FMT_YUV444M it is 3.

You can stick this value directly into pixfmt_mp->num_planes.

Fine by me, but I have to admit I don't see the value of adding the number of non-contiguous planes. For multiplanar non-contiguous formats the number of planes is equal to the number of planes.

Although maybe it will be clear this way for readers?

Quoted 4 lines

As an aside: perhaps we should start calling the 'multiplanar API' the 'multiple non-contiguous planes API', at least in the documentation. It's the first time that I found a description that actually covers the real meaning.

Yes, indeed. In fact, my first version of this code had something like "is_noncontiguous" instead of the "multiplanar" field.

Quoted 8 lines
Quoted 8 lines
include/media/v4l2-fourcc.h new file +8 -0
+ */
+struct v4l2_format_info {
+	u32 format;
+	u32 header_size;
+	u8 num_planes;
+	u8 cpp[3];
+	u8 hsub;
+	u8 vsub;
Quoted 9 lines

include/media/v4l2-fourcc.h line 53

Quoted 8 lines
include/media/v4l2-fourcc.h new file +7 -0
+	u8 multiplanar;
+};
+
+const struct v4l2_format_info *v4l2_format_info(u32 format);
+const char *v4l2_get_format_name(u32 format);
+
+#endif

Thanks a lot for the thorough review!

Quoted 3 lines

Regards,

Hans

#4 lore raw

Re: [PATCH 01/10] media: Introduce helpers to fill pixel format structs

Quoted 52 lines · Ezequiel Garcia
Quoted 51 lines · Hans Verkuil

Hi Ezequiel,

A quick review below. This looks really useful, BTW.

Quoted 4 lines · Ezequiel Garcia

Add two new API helpers, v4l2_fill_pixfmt and v4l2_fill_pixfmt_mp, to be used by drivers to calculate plane sizes and bytes per lines.

Note that driver-specific paddig and alignment are not

paddig -> padding

Quoted 31 lines

taken into account, and must be done by drivers using this API.

---
 drivers/media/v4l2-core/Makefile      |   2 +-
 drivers/media/v4l2-core/v4l2-common.c |  71 +++++++++++++++++
 drivers/media/v4l2-core/v4l2-fourcc.c | 109 ++++++++++++++++++++++++++
 include/media/v4l2-common.h           |   5 ++
 include/media/v4l2-fourcc.h           |  53 +++++++++++++
 5 files changed, 239 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/v4l2-core/v4l2-fourcc.c
 create mode 100644 include/media/v4l2-fourcc.h
drivers/media/v4l2-core/Makefile +1 -1
index 9ee57e1efefe..bc23c3407c17 100644
@@ -7,7 +7,7 @@ tuner-objs	:=	tuner-core.o
 
 videodev-objs	:=	v4l2-dev.o v4l2-ioctl.o v4l2-device.o v4l2-fh.o \
 			v4l2-event.o v4l2-ctrls.o v4l2-subdev.o v4l2-clk.o \
-			v4l2-async.o
+			v4l2-async.o v4l2-fourcc.o
 ifeq ($(CONFIG_COMPAT),y)
   videodev-objs += v4l2-compat-ioctl32.o
 endif
drivers/media/v4l2-core/v4l2-common.c +0 -0
index 50763fb42a1b..39d86a389cae 100644

drivers/media/v4l2-core/v4l2-common.c line 64

Quoted 4 lines
drivers/media/v4l2-core/v4l2-common.c +1 -0
@@ -61,6 +61,7 @@
 #include <media/v4l2-common.h>
 #include <media/v4l2-device.h>
 #include <media/v4l2-ctrls.h>
+#include <media/v4l2-fourcc.h>

Either create a v4l2-fourcc.c source using this header, or move the contents of v4l2-fourcc.h to v4l2-common.h.

Quoted 5 lines
Quoted 2 lines

Creating a new header but not a new source is a bit weird.

This patch adds v4l2-fourcc.c and v4l2-fourcc.h, containing only the pixel format array and the fourcc name helper.

Ah, I now see that I got confused because some functions were added to v4l2-fourcc.c and others to v4l2-common.c.

I still stand by my comment: either everything goes to v4l2-common, or to v4l2-fourcc. But it makes no sense IMHO to split it over two sources.

Quoted 324 lines
Quoted 14 lines
Quoted 14 lines
drivers/media/v4l2-core/v4l2-common.c +6 -0
 #include <linux/videodev2.h>
 
@@ -455,3 +456,73 @@ int v4l2_s_parm_cap(struct video_device *vdev,
 	return ret;
 }
 EXPORT_SYMBOL_GPL(v4l2_s_parm_cap);
+
+void v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
+			 int pixelformat, int width, int height)
+{
+	const struct v4l2_format_info *info;
+	struct v4l2_plane_pix_format *plane;
Quoted 9 lines

drivers/media/v4l2-core/v4l2-common.c int v4l2_s_parm_cap(struct video_device *vdev, · line 469

Quoted 5 lines
drivers/media/v4l2-core/v4l2-common.c +5 -0
+	int i;
+
+	info = v4l2_format_info(pixelformat);
+	if (!info)
+		return;

You should return a bool or something to indicate whether or not the pixelformat was known.

Got it.

Quoted 42 lines

drivers/media/v4l2-core/v4l2-common.c int v4l2_s_parm_cap(struct video_device *vdev, · line 475

Quoted 6 lines
drivers/media/v4l2-core/v4l2-common.c +6 -0
+
+	pixfmt->width = width;
+	pixfmt->height = height;
+	pixfmt->pixelformat = pixelformat;
+
+	if (!info->multiplanar) {

It would make much more sense if multiplanar contained the number of planes to use (i.e. equal to pixfmt->num_planes).

See more about this below.

Quoted 30 lines
drivers/media/v4l2-core/v4l2-common.c +30 -0
+		pixfmt->num_planes = 1;
+		plane = &pixfmt->plane_fmt[0];
+		plane->bytesperline = width * info->cpp[0];
+		plane->sizeimage = 0;
+		for (i = 0; i < info->num_planes; i++) {
+			unsigned int hsub = (i == 0) ? 1 : info->hsub;
+			unsigned int vsub = (i == 0) ? 1 : info->vsub;
+
+			plane->sizeimage += info->cpp[i] *
+				DIV_ROUND_UP(width, hsub) *
+				DIV_ROUND_UP(height, vsub);
+		}
+	} else {
+		pixfmt->num_planes = info->num_planes;
+		for (i = 0; i < info->num_planes; i++) {
+			unsigned int hsub = (i == 0) ? 1 : info->hsub;
+			unsigned int vsub = (i == 0) ? 1 : info->vsub;
+
+			plane = &pixfmt->plane_fmt[i];
+			plane->bytesperline =
+				info->cpp[i] * DIV_ROUND_UP(width, hsub);
+			plane->sizeimage =
+				plane->bytesperline * DIV_ROUND_UP(height, vsub);
+		}
+	}
+}
+EXPORT_SYMBOL_GPL(v4l2_fill_pixfmt_mp);
+
+void v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, int pixelformat, int width, int height)
+{
Quoted 9 lines

drivers/media/v4l2-core/v4l2-common.c int v4l2_s_parm_cap(struct video_device *vdev, · line 469

Quoted 6 lines
drivers/media/v4l2-core/v4l2-common.c +6 -0
+	const struct v4l2_format_info *info;
+	int i;
+
+	info = v4l2_format_info(pixelformat);
+	if (!info)
+		return;

You have to check if pixelformat was a multiplanar format and reject it.

OK.

Quoted 52 lines
Quoted 52 lines
drivers/media/v4l2-core/v4l2-common.c +17 -0
+
+	pixfmt->width = width;
+	pixfmt->height = height;
+	pixfmt->pixelformat = pixelformat;
+	pixfmt->bytesperline = width * info->cpp[0];
+	pixfmt->sizeimage = 0;
+
+	for (i = 0; i < info->num_planes; i++) {
+		unsigned int hsub = (i == 0) ? 1 : info->hsub;
+		unsigned int vsub = (i == 0) ? 1 : info->vsub;
+
+		pixfmt->sizeimage += info->cpp[i] *
+			DIV_ROUND_UP(width, hsub) *
+			DIV_ROUND_UP(height, vsub);
+	}
+}
+EXPORT_SYMBOL_GPL(v4l2_fill_pixfmt);
drivers/media/v4l2-core/v4l2-fourcc.c new file +29 -0
new file mode 100644
index 000000000000..982c0ffa1a66
@@ -0,0 +1,109 @@
+/*
+ * Copyright (c) 2018 Collabora, Ltd.
+ *
+ * Based on drm-fourcc:
+ * Copyright (c) 2016 Laurent Pinchart <laurent.pinchart-ryLnwIuWjnjg/[email protected]>
+ *
+ * Permission to use, copy, modify, distribute, and sell this software and its
+ * documentation for any purpose is hereby granted without fee, provided that
+ * the above copyright notice appear in all copies and that both that copyright
+ * notice and this permission notice appear in supporting documentation, and
+ * that the name of the copyright holders not be used in advertising or
+ * publicity pertaining to distribution of the software without specific,
+ * written prior permission.  The copyright holders make no representations
+ * about the suitability of this software for any purpose.  It is provided "as
+ * is" without express or implied warranty.
+ *
+ * THE COPYRIGHT HOLDERS DISCLAIM ALL WARRANTIES WITH REGARD TO THIS SOFTWARE,
+ * INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN NO
+ * EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY SPECIAL, INDIRECT OR
+ * CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE,
+ * DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
+ * TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE
+ * OF THIS SOFTWARE.
+ */
+
+#include <linux/ctype.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-fourcc.h>
+
Quoted 9 lines

drivers/media/v4l2-core/v4l2-fourcc.c line 35

Quoted 6 lines
drivers/media/v4l2-core/v4l2-fourcc.c new file +6 -0
+static char printable_char(int c)
+{
+	return isascii(c) && isprint(c) ? c : '?';
+}
+
+const char *v4l2_get_format_name(uint32_t format)

This should be called v4l2_get_fourcc_name. The format name is what ENUMFMT returns.

Got it.

Quoted 13 lines

drivers/media/v4l2-core/v4l2-fourcc.c line 44

Quoted 9 lines
drivers/media/v4l2-core/v4l2-fourcc.c new file +9 -0
+{
+	static char buf[4];
+
+	snprintf(buf, 4,
+		 "%c%c%c%c",
+		 printable_char(format & 0xff),
+		 printable_char((format >> 8) & 0xff),
+		 printable_char((format >> 16) & 0xff),
+		 printable_char((format >> 24) & 0x7f));

If bit 31 is set, then add a '-BE' suffix to indicate that this is a big endian variant of the same pixelformat with bit 31 set to 0.

Quoted 2 lines

See also v4l_fill_fmtdesc() in v4l2-ioctl.c.

I omitted endianness and alpha because they weren't used by V4L2, but let me add them to make the code generic and support out-of-tree stuff.

Quoted 10 lines

drivers/media/v4l2-core/v4l2-fourcc.c line 48

Quoted 4 lines
drivers/media/v4l2-core/v4l2-fourcc.c new file +4 -0
+
+	return buf;
+}
+EXPORT_SYMBOL(v4l2_get_format_name);

I remember that Sakari tried to make a macro for this in a public header, but it was either rejected or fizzled out:

https://www.mail-archive.com/[email protected]/msg128702.html

I see. This time we are adding the fourcc name helper to the private API, which shouldn't cause too much debate. Adding Sakari to the loop.

Quoted 50 lines
Quoted 50 lines
drivers/media/v4l2-core/v4l2-fourcc.c new file +50 -0
+
+const struct v4l2_format_info *v4l2_format_info(u32 format)
+{
+	static const struct v4l2_format_info formats[] = {
+		/* RGB formats */
+		{ .format = V4L2_PIX_FMT_BGR24,		.num_planes = 1, .cpp = { 3, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_RGB24,		.num_planes = 1, .cpp = { 3, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_HSV24,		.num_planes = 1, .cpp = { 3, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_BGR32,		.num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_XBGR32,	.num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_RGB32,		.num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_XRGB32,	.num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_HSV32,		.num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_ARGB32,	.num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_ABGR32,	.num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_GREY,		.num_planes = 1, .cpp = { 1, 0, 0 }, .hsub = 1, .vsub = 1 },
+
+		/* YUV formats */
+		{ .format = V4L2_PIX_FMT_YUYV,		.num_planes = 1, .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_YVYU,		.num_planes = 1, .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_UYVY,		.num_planes = 1, .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_VYUY,		.num_planes = 1, .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
+
+		{ .format = V4L2_PIX_FMT_NV12,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 2 },
+		{ .format = V4L2_PIX_FMT_NV21,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 2 },
+		{ .format = V4L2_PIX_FMT_NV16,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_NV61,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_NV24,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 1, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_NV42,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 1, .vsub = 1 },
+
+		{ .format = V4L2_PIX_FMT_YUV410,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 4, .vsub = 4 },
+		{ .format = V4L2_PIX_FMT_YVU410,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 4, .vsub = 4 },
+		{ .format = V4L2_PIX_FMT_YUV411P,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 4, .vsub = 1 },
+		{ .format = V4L2_PIX_FMT_YUV420,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 2, .vsub = 2 },
+		{ .format = V4L2_PIX_FMT_YVU420,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 2, .vsub = 2 },
+		{ .format = V4L2_PIX_FMT_YUV422P,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 2, .vsub = 1 },
+
+		{ .format = V4L2_PIX_FMT_YUV420M,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 2, .vsub = 2, .multiplanar = 1 },
+		{ .format = V4L2_PIX_FMT_YVU420M,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 2, .vsub = 2, .multiplanar = 1 },
+		{ .format = V4L2_PIX_FMT_YUV422M,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 2, .vsub = 1, .multiplanar = 1 },
+		{ .format = V4L2_PIX_FMT_YVU422M,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 2, .vsub = 1, .multiplanar = 1 },
+		{ .format = V4L2_PIX_FMT_YUV444M,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 1, .vsub = 1, .multiplanar = 1 },
+		{ .format = V4L2_PIX_FMT_YVU444M,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 1, .vsub = 1, .multiplanar = 1 },
+
+		{ .format = V4L2_PIX_FMT_NV12M,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 2, .multiplanar = 1 },
+		{ .format = V4L2_PIX_FMT_NV21M,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 2, .multiplanar = 1 },
+		{ .format = V4L2_PIX_FMT_NV16M,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 1, .multiplanar = 1 },
+		{ .format = V4L2_PIX_FMT_NV61M,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 1, .multiplanar = 1 },
+
+	};
Quoted 9 lines

drivers/media/v4l2-core/v4l2-fourcc.c line 104

Quoted 6 lines
drivers/media/v4l2-core/v4l2-fourcc.c new file +6 -0
+	unsigned int i;
+
+	for (i = 0; i < ARRAY_SIZE(formats); ++i) {
+		if (formats[i].format == format)
+			return &formats[i];
+	}

No need for {}

OK

Quoted 5 lines

drivers/media/v4l2-core/v4l2-fourcc.c line 106

Quoted 2 lines
drivers/media/v4l2-core/v4l2-fourcc.c new file +2 -0
+
+	pr_warn("Unsupported V4L 4CC format %s (%08x)\n", v4l2_get_format_name(format), format);

Make this a pr_dev or remove it altogether. I prefer the latter.

Right. If the function returns an error condition, we can drop it.

Quoted 67 lines
Quoted 38 lines
drivers/media/v4l2-core/v4l2-fourcc.c new file +3 -0
+	return NULL;
+}
+EXPORT_SYMBOL(v4l2_format_info);
include/media/v4l2-common.h +5 -0
index 0c511ed8ffb0..6461ce747d90 100644
@@ -327,6 +327,11 @@ void v4l_bound_align_image(unsigned int *width, unsigned int wmin,
 			   unsigned int hmax, unsigned int halign,
 			   unsigned int salign);
 
+void v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, int pixelformat,
+		      int width, int height);
+void v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt, int pixelformat,
+			 int width, int height);
+
 /**
  * v4l2_find_nearest_size - Find the nearest size among a discrete
  *	set of resolutions contained in an array of a driver specific struct.
include/media/v4l2-fourcc.h new file +13 -0
new file mode 100644
index 000000000000..3d24f442aaf5
@@ -0,0 +1,53 @@
+/*
+ * Copyright (c) 2018 Collabora, Ltd.
+ *
+ * Based on drm-fourcc:
+ * Copyright (c) 2016 Laurent Pinchart <laurent.pinchart-ryLnwIuWjnjg/[email protected]>
+ *
+ * Permission to use, copy, modify, distribute, and sell this software and its
+ * documentation for any purpose is hereby granted without fee, provided that
+ * the above copyright notice appear in all copies and that both that copyright
+ * notice and this permission notice appear in supporting documentation, and
+ * that the name of the copyright holders not be used in advertising or
+ * publicity pertaining to distribution of the software without specific,
+ * written prior permission.  The copyright holders make no representations

include/media/v4l2-fourcc.h line 23

Quoted 10 lines
include/media/v4l2-fourcc.h new file +10 -0
+ * about the suitability of this software for any purpose.  It is provided "as
+ * is" without express or implied warranty.
+ *
+ * THE COPYRIGHT HOLDERS DISCLAIM ALL WARRANTIES WITH REGARD TO THIS SOFTWARE,
+ * INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN NO
+ * EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY SPECIAL, INDIRECT OR
+ * CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE,
+ * DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
+ * TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE
+ * OF THIS SOFTWARE.

Should be an SPDX ID.

include/media/v4l2-fourcc.h line 34

Quoted 11 lines
include/media/v4l2-fourcc.h new file +11 -0
+ */
+#ifndef __V4L2_FOURCC_H__
+#define __V4L2_FOURCC_H__
+
+#include <linux/types.h>
+
+/**
+ * struct v4l2_format_info - information about a V4L2 format
+ * @format: 4CC format identifier (V4L2_PIX_FMT_*)
+ * @header_size: Size of header, optional and used by compressed formats
+ * @num_planes: Number of planes (1 to 3)

This is actually 1-4 since there may be an alpha channel as well. Not that we have such formats since the only formats with an alpha channel are interleaved formats, but it is possible.

Quoted 8 lines

So this value is 2 for both NV12 and NV12M.

include/media/v4l2-fourcc.h line 35

Quoted 1 line
include/media/v4l2-fourcc.h new file +1 -0
+ * @cpp: Number of bytes per pixel (per plane)

cpp? Shouldn't that be bpp?

Note that this can differ per plane (see e.g. NV24).

Quoted 6 lines

Yes, the comment should specify that it's an array of bytes per pixel.

And the naming follows drm-fourcc. cpp stands for char-per-pixel. It may be confusing but at the same time, I really like to have consistent naming across the board (think for instance in the DMA coherent/consistent aliases).

We never use cpp in the media subsystem, it's always bpp. Please keep that name.

And the v4l2 fourccs are unrelated to the drm fourccs. Historically they developed along separate paths and I actually think it can be confusing to suggest that they are related.

Now whether there should be an effort to unify the two is an entirely different question.

Quoted 3 lines

Also, note that drm-fourcc deprecates cpp, to support tile formats. Hopefully we don't need that here?

We do have tile formats (V4L2_PIX_FMT_NV12MT_16X16), but it is up to the driver to align width/height accordingly.

Quoted 15 lines
Quoted 7 lines

include/media/v4l2-fourcc.h line 37

Quoted 3 lines
include/media/v4l2-fourcc.h new file +2 -0
+ * @hsub: Horizontal chroma subsampling factor
+ * @vsub: Vertical chroma subsampling factor

A bit too cryptic IMHO. I would prefer hdiv or hsubsampling. 'hsub' suggests subtraction :-)

Ditto, this name follows drm-fourcc. I'm fine either way.

Quoted 5 lines

include/media/v4l2-fourcc.h line 38

Quoted 1 line
include/media/v4l2-fourcc.h new file +1 -0
+ * @multiplanar: Is it a multiplanar variant format? (e.g. NV12M)

This should, I think, be renamed to num_non_contig_planes to indicate how many non-contiguous planes there are in the format.

Quoted 8 lines
Quoted 4 lines

So this value is 1 for NV12 and 2 for NV12M. For V4L2_PIX_FMT_YUV444M it is 3.

You can stick this value directly into pixfmt_mp->num_planes.

Fine by me, but I have to admit I don't see the value of adding the number of non-contiguous planes. For multiplanar non-contiguous formats the number of planes is equal to the number of planes.

Hmm, that's true. Choose whatever gives you the shortest code :-)

Quoted 10 lines

Although maybe it will be clear this way for readers?

Quoted 4 lines

As an aside: perhaps we should start calling the 'multiplanar API' the 'multiple non-contiguous planes API', at least in the documentation. It's the first time that I found a description that actually covers the real meaning.

Yes, indeed. In fact, my first version of this code had something like "is_noncontiguous" instead of the "multiplanar" field.

I'm fine with that. Add a comment after it like: /* aka multiplanar */

Quoted 19 lines
Quoted 18 lines
Quoted 7 lines
include/media/v4l2-fourcc.h new file +5 -0
+ */
+struct v4l2_format_info {
+	u32 format;
+	u32 header_size;
+	u8 num_planes;

include/media/v4l2-fourcc.h line 53

Quoted 10 lines
include/media/v4l2-fourcc.h new file +10 -0
+	u8 cpp[3];
+	u8 hsub;
+	u8 vsub;
+	u8 multiplanar;
+};
+
+const struct v4l2_format_info *v4l2_format_info(u32 format);
+const char *v4l2_get_format_name(u32 format);
+
+#endif
Quoted 8 lines

Thanks a lot for the thorough review!

Quoted 3 lines

Regards,

Hans

Regards,

Hans

#5 lore raw

Re: [PATCH 01/10] media: Introduce helpers to fill pixel format structs

Quoted 8 lines · Hans Verkuil
Quoted 6 lines · Ezequiel Garcia
Quoted 5 lines · Hans Verkuil

Hi Ezequiel,

A quick review below. This looks really useful, BTW.

On 2/5/19 9:24 PM, Ezequiel Garcia wrote:

[snip]

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 5 lines

+/**
+ * struct v4l2_format_info - information about a V4L2 format
+ * @format: 4CC format identifier (V4L2_PIX_FMT_*)
+ * @header_size: Size of header, optional and used by compressed formats
+ * @num_planes: Number of planes (1 to 3)

This is actually 1-4 since there may be an alpha channel as well. Not that we have such formats since the only formats with an alpha channel are interleaved formats, but it is possible.

How about 1 to VIDEO_MAX_PLANES to be a bit more consistent? Tbh. I'm not sure why we have that defined to 8, but if we have such constant already, it could make sense to use it here as well.

[snip]

Quoted 7 lines
Quoted 3 lines

Also, note that drm-fourcc deprecates cpp, to support tile formats. Hopefully we don't need that here?

We do have tile formats (V4L2_PIX_FMT_NV12MT_16X16), but it is up to the driver to align width/height accordingly.

I'd still make these helpers align to the constraints defined by the format itself (e.g. 16x16), since it doesn't cost us anything, and have the driver do any further alignment only if they need so.

Quoted 10 lines
Quoted 10 lines
Quoted 7 lines
Quoted 3 lines

+ * @hsub: Horizontal chroma subsampling factor
+ * @vsub: Vertical chroma subsampling factor

A bit too cryptic IMHO. I would prefer hdiv or hsubsampling. 'hsub' suggests subtraction :-)

Ditto, this name follows drm-fourcc. I'm fine either way.

I personally like hsub and vsub too, but maybe I just spent too much time with DRM code. *subsampling would make the initializers super wide, so if we decide that we don't like *sub, I'd go with *div.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 1 line

+ * @multiplanar: Is it a multiplanar variant format? (e.g. NV12M)

This should, I think, be renamed to num_non_contig_planes to indicate how many non-contiguous planes there are in the format.

So this value is 1 for NV12 and 2 for NV12M. For V4L2_PIX_FMT_YUV444M it is 3.

Quoted 14 lines
Quoted 6 lines
Quoted 2 lines

You can stick this value directly into pixfmt_mp->num_planes.

Fine by me, but I have to admit I don't see the value of adding the number of non-contiguous planes. For multiplanar non-contiguous formats the number of planes is equal to the number of planes.

Hmm, that's true. Choose whatever gives you the shortest code :-)

Quoted 5 lines

Although maybe it will be clear this way for readers?

Quoted 2 lines

As an aside: perhaps we should start calling the 'multiplanar API' the 'multiple non-contiguous planes API', at least in the documentation. It's the

To me, "multiple non-contiguous planes API" would suggest that the planes themselves are non-contiguous.

Many drivers (especially Samsung ones) have a distinction between "color planes" and "memory planes" internally, so maybe "Multiple memory planes API" could make sense?

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

first time that I found a description that actually covers the real meaning.

Yes, indeed. In fact, my first version of this code had something like "is_noncontiguous" instead of the "multiplanar" field.

I'm fine with that. Add a comment after it like: /* aka multiplanar */

FWIW, some of the drivers have .num_cplanes and .num_mplanes in their format descriptors.

Best regards,
Tomasz

#6 lore raw

Re: [PATCH 01/10] media: Introduce helpers to fill pixel format structs

Quoted 17 lines · Tomasz Figa
Quoted 8 lines · Hans Verkuil
Quoted 6 lines · Ezequiel Garcia
Quoted 5 lines · Hans Verkuil

Hi Ezequiel,

A quick review below. This looks really useful, BTW.

On 2/5/19 9:24 PM, Ezequiel Garcia wrote:

[snip]

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

+/**
+ * struct v4l2_format_info - information about a V4L2 format
+ * @format: 4CC format identifier (V4L2_PIX_FMT_*)
+ * @header_size: Size of header, optional and used by compressed formats
+ * @num_planes: Number of planes (1 to 3)

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

This is actually 1-4 since there may be an alpha channel as well. Not that we have such formats since the only formats with an alpha channel are interleaved formats, but it is possible.

How about 1 to VIDEO_MAX_PLANES to be a bit more consistent? Tbh. I'm not sure why we have that defined to 8, but if we have such constant already, it could make sense to use it here as well.

We didn't know at the time how many planes we would need. I think we chose 8 because 1) that fit inside struct v4l2_format and 2) it allowed room for planes carrying meta data.

In hindsight we probably should have chosen 4 instead of 8.

In any case, since this is an internal API I think chosing MAX_PLANES here would waste unnecessary memory.

Quoted 6 lines

[snip]

Quoted 4 lines
Quoted 3 lines

Also, note that drm-fourcc deprecates cpp, to support tile formats. Hopefully we don't need that here?

Quoted 7 lines
Quoted 3 lines

We do have tile formats (V4L2_PIX_FMT_NV12MT_16X16), but it is up to the driver to align width/height accordingly.

I'd still make these helpers align to the constraints defined by the format itself (e.g. 16x16), since it doesn't cost us anything, and have the driver do any further alignment only if they need so.

Yes, sorry, I should have said that: for tiled pixel formats this struct should give the alignments.

But those alignments differ from hsub/vsub: those values restrict the resolution, but the 'tiled' alignments are on top of that.

Quoted 38 lines
Quoted 11 lines
Quoted 11 lines
Quoted 8 lines
Quoted 4 lines

+ * @hsub: Horizontal chroma subsampling factor
+ * @vsub: Vertical chroma subsampling factor

A bit too cryptic IMHO. I would prefer hdiv or hsubsampling. 'hsub' suggests subtraction :-)

Ditto, this name follows drm-fourcc. I'm fine either way.

I personally like hsub and vsub too, but maybe I just spent too much time with DRM code. *subsampling would make the initializers super wide, so if we decide that we don't like *sub, I'd go with *div.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 1 line

+ * @multiplanar: Is it a multiplanar variant format? (e.g. NV12M)

This should, I think, be renamed to num_non_contig_planes to indicate how many non-contiguous planes there are in the format.

So this value is 1 for NV12 and 2 for NV12M. For V4L2_PIX_FMT_YUV444M it is 3.

Quoted 14 lines
Quoted 6 lines
Quoted 2 lines

You can stick this value directly into pixfmt_mp->num_planes.

Fine by me, but I have to admit I don't see the value of adding the number of non-contiguous planes. For multiplanar non-contiguous formats the number of planes is equal to the number of planes.

Hmm, that's true. Choose whatever gives you the shortest code :-)

Quoted 5 lines

Although maybe it will be clear this way for readers?

Quoted 2 lines

As an aside: perhaps we should start calling the 'multiplanar API' the 'multiple non-contiguous planes API', at least in the documentation. It's the

Quoted 6 lines

To me, "multiple non-contiguous planes API" would suggest that the planes themselves are non-contiguous.

Many drivers (especially Samsung ones) have a distinction between "color planes" and "memory planes" internally, so maybe "Multiple memory planes API" could make sense?

Huh, that's an idea. So _MPLANE should have been _MMPLANE?

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

first time that I found a description that actually covers the real meaning.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

Yes, indeed. In fact, my first version of this code had something like "is_noncontiguous" instead of the "multiplanar" field.

I'm fine with that. Add a comment after it like: /* aka multiplanar */

FWIW, some of the drivers have .num_cplanes and .num_mplanes in their format descriptors.

I think that makes sense. Good suggestion.

Regards,

Hans

#7 lore raw

Re: [PATCH 01/10] media: Introduce helpers to fill pixel format structs

Quoted 25 lines · Hans Verkuil
Quoted 16 lines · Tomasz Figa
Quoted 7 lines · Hans Verkuil
Quoted 6 lines · Ezequiel Garcia
Quoted 5 lines · Hans Verkuil

Hi Ezequiel,

A quick review below. This looks really useful, BTW.

On 2/5/19 9:24 PM, Ezequiel Garcia wrote:

[snip]

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

+/**
+ * struct v4l2_format_info - information about a V4L2 format
+ * @format: 4CC format identifier (V4L2_PIX_FMT_*)
+ * @header_size: Size of header, optional and used by compressed formats
+ * @num_planes: Number of planes (1 to 3)

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

This is actually 1-4 since there may be an alpha channel as well. Not that we have such formats since the only formats with an alpha channel are interleaved formats, but it is possible.

How about 1 to VIDEO_MAX_PLANES to be a bit more consistent? Tbh. I'm not sure why we have that defined to 8, but if we have such constant already, it could make sense to use it here as well.

Quoted 9 lines

We didn't know at the time how many planes we would need. I think we chose 8 because 1) that fit inside struct v4l2_format and 2) it allowed room for planes carrying meta data.

In hindsight we probably should have chosen 4 instead of 8.

In any case, since this is an internal API I think chosing MAX_PLANES here would waste unnecessary memory.

4 it is then.

Quoted 12 lines
Quoted 11 lines

[snip]

Quoted 6 lines
Quoted 2 lines

Also, note that drm-fourcc deprecates cpp, to support tile formats. Hopefully we don't need that here?

We do have tile formats (V4L2_PIX_FMT_NV12MT_16X16), but it is up to the driver to align width/height accordingly.

I'd still make these helpers align to the constraints defined by the format itself (e.g. 16x16), since it doesn't cost us anything, and have the driver do any further alignment only if they need so.

Quoted 6 lines

Yes, sorry, I should have said that: for tiled pixel formats this struct should give the alignments.

But those alignments differ from hsub/vsub: those values restrict the resolution, but the 'tiled' alignments are on top of that.

Hm, OK. So the idea is support tiling formats on this helpers?

Might need to change the way we describe formats, as DRM does, to incorporate some notion of pixel blocks.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 2 lines

+ * @hsub: Horizontal chroma subsampling factor
+ * @vsub: Vertical chroma subsampling factor

A bit too cryptic IMHO. I would prefer hdiv or hsubsampling. 'hsub' suggests subtraction :-)

Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines

Ditto, this name follows drm-fourcc. I'm fine either way.

I personally like hsub and vsub too, but maybe I just spent too much time with DRM code. *subsampling would make the initializers super wide, so if we decide that we don't like *sub, I'd go with *div.

Note that there's a v2 patch, where I went with *div :-)

Quoted 21 lines
Quoted 21 lines
Quoted 20 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line

+ * @multiplanar: Is it a multiplanar variant format? (e.g. NV12M)

This should, I think, be renamed to num_non_contig_planes to indicate how many non-contiguous planes there are in the format.

Quoted 8 lines
Quoted 4 lines

So this value is 1 for NV12 and 2 for NV12M. For V4L2_PIX_FMT_YUV444M it is 3.

You can stick this value directly into pixfmt_mp->num_planes.

Fine by me, but I have to admit I don't see the value of adding the number of non-contiguous planes. For multiplanar non-contiguous formats the number of planes is equal to the number of planes.

Hmm, that's true. Choose whatever gives you the shortest code :-)

Quoted 4 lines

Although maybe it will be clear this way for readers?

Quoted 2 lines

As an aside: perhaps we should start calling the 'multiplanar API' the 'multiple non-contiguous planes API', at least in the documentation. It's the

Quoted 22 lines
Quoted 6 lines

To me, "multiple non-contiguous planes API" would suggest that the planes themselves are non-contiguous.

Many drivers (especially Samsung ones) have a distinction between "color planes" and "memory planes" internally, so maybe "Multiple memory planes API" could make sense?

Huh, that's an idea. So _MPLANE should have been _MMPLANE?

Quoted 11 lines
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

first time that I found a description that actually covers the real meaning.

Yes, indeed. In fact, my first version of this code had something like "is_noncontiguous" instead of the "multiplanar" field.

I'm fine with that. Add a comment after it like: /* aka multiplanar */

FWIW, some of the drivers have .num_cplanes and .num_mplanes in their format descriptors.

This sounds nice.

To be clear, this means that instead of a boolean is_noncontigous, we describe the number of (non-contiguous) memory planes and the number of component planes.

I really dislike the name "color planes", because it could be misleading as chroma planes, so I think "component planes" makes it more clear.

Thanks for reviewing!
Eze

#8 lore raw

PATCH 2/10 rockchip/vpu: Use pixel format helpers

In reply to Ezequiel Garcia

Now that we've introduced the pixel format helpers, use them in vpu driver, and get rid of the internal helpers.

---
 .../media/rockchip/vpu/rockchip_vpu_enc.c     | 91 +------------------
 1 file changed, 2 insertions(+), 89 deletions(-)
drivers/staging/media/rockchip/vpu/rockchip_vpu_enc.c +2 -89
index ab0fb2053620..4451bb2dc3d7 100644
@@ -30,93 +30,6 @@
 #include "rockchip_vpu_hw.h"
 #include "rockchip_vpu_common.h"
 
-/**
- * struct v4l2_format_info - information about a V4L2 format
- * @format: 4CC format identifier (V4L2_PIX_FMT_*)
- * @header_size: Size of header, optional and used by compressed formats
- * @num_planes: Number of planes (1 to 3)
- * @cpp: Number of bytes per pixel (per plane)
- * @hsub: Horizontal chroma subsampling factor
- * @vsub: Vertical chroma subsampling factor
- * @is_compressed: Is it a compressed format?
- * @multiplanar: Is it a multiplanar variant format? (e.g. NV12M)
- */
-struct v4l2_format_info {
-	u32 format;
-	u32 header_size;
-	u8 num_planes;
-	u8 cpp[3];
-	u8 hsub;
-	u8 vsub;
-	u8 is_compressed;
-	u8 multiplanar;
-};
-
-static const struct v4l2_format_info *
-v4l2_format_info(u32 format)
-{
-	static const struct v4l2_format_info formats[] = {
-		{ .format = V4L2_PIX_FMT_YUV420M,	.num_planes = 3, .cpp = { 1, 1, 1 }, .hsub = 2, .vsub = 2, .multiplanar = 1 },
-		{ .format = V4L2_PIX_FMT_NV12M,		.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 2, .multiplanar = 1 },
-		{ .format = V4L2_PIX_FMT_YUYV,		.num_planes = 1, .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
-		{ .format = V4L2_PIX_FMT_UYVY,		.num_planes = 1, .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
-	};
-	unsigned int i;
-
-	for (i = 0; i < ARRAY_SIZE(formats); ++i) {
-		if (formats[i].format == format)
-			return &formats[i];
-	}
-
-	vpu_err("Unsupported V4L 4CC format (%08x)\n", format);
-	return NULL;
-}
-
-static void
-fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
-	       int pixelformat, int width, int height)
-{
-	const struct v4l2_format_info *info;
-	struct v4l2_plane_pix_format *plane;
-	int i;
-
-	info = v4l2_format_info(pixelformat);
-	if (!info)
-		return;
-
-	pixfmt->width = width;
-	pixfmt->height = height;
-	pixfmt->pixelformat = pixelformat;
-
-	if (!info->multiplanar) {
-		pixfmt->num_planes = 1;
-		plane = &pixfmt->plane_fmt[0];
-		plane->bytesperline = info->is_compressed ?
-					0 : width * info->cpp[0];
-		plane->sizeimage = info->header_size;
-		for (i = 0; i < info->num_planes; i++) {
-			unsigned int hsub = (i == 0) ? 1 : info->hsub;
-			unsigned int vsub = (i == 0) ? 1 : info->vsub;
-
-			plane->sizeimage += info->cpp[i] *
-				DIV_ROUND_UP(width, hsub) *
-				DIV_ROUND_UP(height, vsub);
-		}
-	} else {
-		pixfmt->num_planes = info->num_planes;
-		for (i = 0; i < info->num_planes; i++) {
-			unsigned int hsub = (i == 0) ? 1 : info->hsub;
-			unsigned int vsub = (i == 0) ? 1 : info->vsub;
-
-			plane = &pixfmt->plane_fmt[i];
-			plane->bytesperline =
-				info->cpp[i] * DIV_ROUND_UP(width, hsub);
-			plane->sizeimage =
-				plane->bytesperline * DIV_ROUND_UP(height, vsub);
-		}
-	}
-}
-
 static const struct rockchip_vpu_fmt *
 rockchip_vpu_find_format(struct rockchip_vpu_ctx *ctx, u32 fourcc)
 {
@@ -339,7 +252,7 @@ vidioc_try_fmt_out_mplane(struct file *file, void *priv, struct v4l2_format *f)
 	height = round_up(height, JPEG_MB_DIM);
 
 	/* Fill remaining fields */
-	fill_pixfmt_mp(pix_mp, fmt->fourcc, width, height);
+	v4l2_fill_pixfmt_mp(pix_mp, fmt->fourcc, width, height);
 
 	for (i = 0; i < pix_mp->num_planes; i++) {
 		memset(pix_mp->plane_fmt[i].reserved, 0,
@@ -393,7 +306,7 @@ void rockchip_vpu_enc_reset_src_fmt(struct rockchip_vpu_dev *vpu,
 	fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
 	fmt->xfer_func = V4L2_XFER_FUNC_DEFAULT;
 
-	fill_pixfmt_mp(fmt, ctx->vpu_src_fmt->fourcc, width, height);
+	v4l2_fill_pixfmt_mp(fmt, ctx->vpu_src_fmt->fourcc, width, height);
 }
 
 static int
Signature · 1 line
2.20.1
#9 lore raw

PATCH 3/10 rockchip/vpu: Use v4l2_m2m_buf_copy_data

In reply to Ezequiel Garcia

Use the recently introduced v4l2_m2m_buf_copy_data helper and get rid of some code.

---
 drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c | 9 +--------
 1 file changed, 1 insertion(+), 8 deletions(-)
drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c +1 -8
index 962412c79b91..c332dbeb9dbc 100644
@@ -59,14 +59,7 @@ static void rockchip_vpu_job_finish(struct rockchip_vpu_dev *vpu,
 	src->sequence = ctx->sequence_out++;
 	dst->sequence = ctx->sequence_cap++;
 
-	dst->field = src->field;
-	if (src->flags & V4L2_BUF_FLAG_TIMECODE)
-		dst->timecode = src->timecode;
-	dst->vb2_buf.timestamp = src->vb2_buf.timestamp;
-	dst->flags &= ~(V4L2_BUF_FLAG_TSTAMP_SRC_MASK |
-			V4L2_BUF_FLAG_TIMECODE);
-	dst->flags |= src->flags & (V4L2_BUF_FLAG_TSTAMP_SRC_MASK |
-				    V4L2_BUF_FLAG_TIMECODE);
+	v4l2_m2m_buf_copy_data(src, dst, true);
 
 	avail_size = vb2_plane_size(&dst->vb2_buf, 0) -
 		     ctx->vpu_dst_fmt->header_size;
Signature · 1 line
2.20.1
#10 lore raw

PATCH 4/10 rockchip/vpu: Cleanup macroblock alignment

In reply to Ezequiel Garcia

We need to make the macrobock alignment generic, in order to support multiple codecs.

---
 .../media/rockchip/vpu/rockchip_vpu_enc.c     | 20 ++++++++-----------
 1 file changed, 8 insertions(+), 12 deletions(-)
drivers/staging/media/rockchip/vpu/rockchip_vpu_enc.c +8 -12
index 4451bb2dc3d7..ae1ff3d9b9d2 100644
@@ -203,8 +203,8 @@ vidioc_try_fmt_cap_mplane(struct file *file, void *priv, struct v4l2_format *f)
 			       fmt->frmsize.min_height,
 			       fmt->frmsize.max_height);
 	/* Round up to macroblocks. */
-	pix_mp->width = round_up(pix_mp->width, JPEG_MB_DIM);
-	pix_mp->height = round_up(pix_mp->height, JPEG_MB_DIM);
+	pix_mp->width = round_up(pix_mp->width, fmt->frmsize.step_width);
+	pix_mp->height = round_up(pix_mp->height, fmt->frmsize.step_height);
 
 	/*
 	 * For compressed formats the application can specify
@@ -248,8 +248,8 @@ vidioc_try_fmt_out_mplane(struct file *file, void *priv, struct v4l2_format *f)
 		       ctx->vpu_dst_fmt->frmsize.min_height,
 		       ctx->vpu_dst_fmt->frmsize.max_height);
 	/* Round up to macroblocks. */
-	width = round_up(width, JPEG_MB_DIM);
-	height = round_up(height, JPEG_MB_DIM);
+	width = round_up(width, ctx->vpu_dst_fmt->frmsize.step_width);
+	height = round_up(height, ctx->vpu_dst_fmt->frmsize.step_height);
 
 	/* Fill remaining fields */
 	v4l2_fill_pixfmt_mp(pix_mp, fmt->fourcc, width, height);
@@ -338,10 +338,8 @@ vidioc_s_fmt_out_mplane(struct file *file, void *priv, struct v4l2_format *f)
 	ctx->dst_fmt.height = pix_mp->height;
 
 	vpu_debug(0, "OUTPUT codec mode: %d\n", ctx->vpu_src_fmt->codec_mode);
-	vpu_debug(0, "fmt - w: %d, h: %d, mb - w: %d, h: %d\n",
-		  pix_mp->width, pix_mp->height,
-		  JPEG_MB_WIDTH(pix_mp->width),
-		  JPEG_MB_HEIGHT(pix_mp->height));
+	vpu_debug(0, "fmt - w: %d, h: %d\n",
+		  pix_mp->width, pix_mp->height);
 	return 0;
 }
 
@@ -380,10 +378,8 @@ vidioc_s_fmt_cap_mplane(struct file *file, void *priv, struct v4l2_format *f)
 	ctx->dst_fmt = *pix_mp;
 
 	vpu_debug(0, "CAPTURE codec mode: %d\n", ctx->vpu_dst_fmt->codec_mode);
-	vpu_debug(0, "fmt - w: %d, h: %d, mb - w: %d, h: %d\n",
-		  pix_mp->width, pix_mp->height,
-		  JPEG_MB_WIDTH(pix_mp->width),
-		  JPEG_MB_HEIGHT(pix_mp->height));
+	vpu_debug(0, "fmt - w: %d, h: %d\n",
+		  pix_mp->width, pix_mp->height);
 
 	/*
 	 * Current raw format might have become invalid with newly
Signature · 1 line
2.20.1
#11 lore raw

PATCH 5/10 rockchip/vpu: Cleanup JPEG bounce buffer management

In reply to Ezequiel Garcia

In order to make the code more generic, introduce a pair of start/stop codec operations, and use them to allocate and release the JPEG bounce buffer.

---
 .../media/rockchip/vpu/rk3288_vpu_hw.c        |  2 ++
 .../rockchip/vpu/rk3288_vpu_hw_jpeg_enc.c     |  4 +--
 .../media/rockchip/vpu/rk3399_vpu_hw.c        |  2 ++
 .../rockchip/vpu/rk3399_vpu_hw_jpeg_enc.c     |  4 +--
 .../staging/media/rockchip/vpu/rockchip_vpu.h | 12 ++++----
 .../media/rockchip/vpu/rockchip_vpu_drv.c     | 10 +++++--
 .../media/rockchip/vpu/rockchip_vpu_enc.c     | 23 +++++----------
 .../media/rockchip/vpu/rockchip_vpu_hw.h      | 28 +++++++++++++++++++
 .../media/rockchip/vpu/rockchip_vpu_jpeg.c    | 25 +++++++++++++++++
 9 files changed, 81 insertions(+), 29 deletions(-)
drivers/staging/media/rockchip/vpu/rk3288_vpu_hw.c +2 -0
index a5e9d183fffd..056ee017c798 100644
@@ -98,6 +98,8 @@ static const struct rockchip_vpu_codec_ops rk3288_vpu_codec_ops[] = {
 	[RK_VPU_MODE_JPEG_ENC] = {
 		.run = rk3288_vpu_jpeg_enc_run,
 		.reset = rk3288_vpu_enc_reset,
+		.start = rockchip_vpu_jpeg_enc_start,
+		.stop = rockchip_vpu_jpeg_enc_stop,
 	},
 };
 
drivers/staging/media/rockchip/vpu/rk3288_vpu_hw_jpeg_enc.c +2 -2
index 5282236d1bb1..9275905e2276 100644
@@ -37,9 +37,9 @@ static void rk3288_vpu_jpeg_enc_set_buffers(struct rockchip_vpu_dev *vpu,
 
 	WARN_ON(pix_fmt->num_planes > 3);
 
-	vepu_write_relaxed(vpu, ctx->bounce_dma_addr,
+	vepu_write_relaxed(vpu, ctx->jpeg_enc_ctx.bounce_buffer.dma,
 			   VEPU_REG_ADDR_OUTPUT_STREAM);
-	vepu_write_relaxed(vpu, ctx->bounce_size,
+	vepu_write_relaxed(vpu, ctx->jpeg_enc_ctx.bounce_buffer.size,
 			   VEPU_REG_STR_BUF_LIMIT);
 
 	if (pix_fmt->num_planes == 1) {
drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c +2 -0
index 6fdef61e2127..8469e474c975 100644
@@ -98,6 +98,8 @@ static const struct rockchip_vpu_codec_ops rk3399_vpu_codec_ops[] = {
 	[RK_VPU_MODE_JPEG_ENC] = {
 		.run = rk3399_vpu_jpeg_enc_run,
 		.reset = rk3399_vpu_enc_reset,
+		.start = rockchip_vpu_jpeg_enc_start,
+		.stop = rockchip_vpu_jpeg_enc_stop,
 	},
 };
 
drivers/staging/media/rockchip/vpu/rk3399_vpu_hw_jpeg_enc.c +2 -2
index dbc86d95fe3b..222b6a3aa25e 100644
@@ -69,9 +69,9 @@ static void rk3399_vpu_jpeg_enc_set_buffers(struct rockchip_vpu_dev *vpu,
 
 	WARN_ON(pix_fmt->num_planes > 3);
 
-	vepu_write_relaxed(vpu, ctx->bounce_dma_addr,
+	vepu_write_relaxed(vpu, ctx->jpeg_enc_ctx.bounce_buffer.dma,
 			   VEPU_REG_ADDR_OUTPUT_STREAM);
-	vepu_write_relaxed(vpu, ctx->bounce_size,
+	vepu_write_relaxed(vpu, ctx->jpeg_enc_ctx.bounce_buffer.size,
 			   VEPU_REG_STR_BUF_LIMIT);
 
 	if (pix_fmt->num_planes == 1) {
drivers/staging/media/rockchip/vpu/rockchip_vpu.h +5 -7
index 1ec2be483e27..76ee24abc141 100644
@@ -124,10 +124,7 @@ struct rockchip_vpu_dev {
  * @jpeg_quality:	User-specified JPEG compression quality.
  *
  * @codec_ops:		Set of operations related to codec mode.
- *
- * @bounce_dma_addr:	Bounce buffer bus address.
- * @bounce_buf:		Bounce buffer pointer.
- * @bounce_size:	Bounce buffer size.
+ * @jpeg_enc_ctx:	JPEG-encoding context.
  */
 struct rockchip_vpu_ctx {
 	struct rockchip_vpu_dev *dev;
@@ -146,9 +143,10 @@ struct rockchip_vpu_ctx {
 
 	const struct rockchip_vpu_codec_ops *codec_ops;
 
-	dma_addr_t bounce_dma_addr;
-	void *bounce_buf;
-	size_t bounce_size;
+	/* Specific for particular codec modes. */
+	union {
+		struct rockchip_vpu_jpeg_enc_hw_ctx jpeg_enc_ctx;
+	};
 };
 
 /**
drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c +8 -2
index c332dbeb9dbc..962748ae3822 100644
@@ -64,10 +64,16 @@ static void rockchip_vpu_job_finish(struct rockchip_vpu_dev *vpu,
 	avail_size = vb2_plane_size(&dst->vb2_buf, 0) -
 		     ctx->vpu_dst_fmt->header_size;
 	if (bytesused <= avail_size) {
-		if (ctx->bounce_buf) {
+		/*
+		 * This works while JPEG is the only encoder this driver
+		 * supports. We will have to abstract this step, or get
+		 * rid of the bounce buffer before we can support
+		 * encoding other codecs.
+		 */
+		if (ctx->jpeg_enc_ctx.bounce_buffer.cpu) {
 			memcpy(vb2_plane_vaddr(&dst->vb2_buf, 0) +
 			       ctx->vpu_dst_fmt->header_size,
-			       ctx->bounce_buf, bytesused);
+			       ctx->jpeg_enc_ctx.bounce_buffer.cpu, bytesused);
 		}
 		dst->vb2_buf.planes[0].bytesused =
 			ctx->vpu_dst_fmt->header_size + bytesused;
drivers/staging/media/rockchip/vpu/rockchip_vpu_enc.c +7 -16
index ae1ff3d9b9d2..2b28403314bc 100644
@@ -514,6 +514,7 @@ static int rockchip_vpu_start_streaming(struct vb2_queue *q, unsigned int count)
 {
 	struct rockchip_vpu_ctx *ctx = vb2_get_drv_priv(q);
 	enum rockchip_vpu_codec_mode codec_mode;
+	int ret = 0;
 
 	if (V4L2_TYPE_IS_OUTPUT(q->type))
 		ctx->sequence_out = 0;
@@ -526,17 +527,10 @@ static int rockchip_vpu_start_streaming(struct vb2_queue *q, unsigned int count)
 	vpu_debug(4, "Codec mode = %d\n", codec_mode);
 	ctx->codec_ops = &ctx->dev->variant->codec_ops[codec_mode];
 
-	/* A bounce buffer is needed for the JPEG payload */
-	if (!V4L2_TYPE_IS_OUTPUT(q->type)) {
-		ctx->bounce_size = ctx->dst_fmt.plane_fmt[0].sizeimage -
-				  ctx->vpu_dst_fmt->header_size;
-		ctx->bounce_buf = dma_alloc_attrs(ctx->dev->dev,
-						  ctx->bounce_size,
-						  &ctx->bounce_dma_addr,
-						  GFP_KERNEL,
-						  DMA_ATTR_ALLOC_SINGLE_PAGES);
-	}
-	return 0;
+	if (!V4L2_TYPE_IS_OUTPUT(q->type))
+		if (ctx->codec_ops && ctx->codec_ops->start)
+			ret = ctx->codec_ops->start(ctx);
+	return ret;
 }
 
 static void rockchip_vpu_stop_streaming(struct vb2_queue *q)
@@ -544,11 +538,8 @@ static void rockchip_vpu_stop_streaming(struct vb2_queue *q)
 	struct rockchip_vpu_ctx *ctx = vb2_get_drv_priv(q);
 
 	if (!V4L2_TYPE_IS_OUTPUT(q->type))
-		dma_free_attrs(ctx->dev->dev,
-			       ctx->bounce_size,
-			       ctx->bounce_buf,
-			       ctx->bounce_dma_addr,
-			       DMA_ATTR_ALLOC_SINGLE_PAGES);
+		if (ctx->codec_ops && ctx->codec_ops->stop)
+			ctx->codec_ops->stop(ctx);
 
 	/*
 	 * The mem2mem framework calls v4l2_m2m_cancel_job before
drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h +28 -0
index 2b955da1be1a..e2e84526f263 100644
@@ -18,9 +18,33 @@ struct rockchip_vpu_ctx;
 struct rockchip_vpu_buf;
 struct rockchip_vpu_variant;
 
+/**
+ * struct rockchip_vpu_aux_buf - auxiliary DMA buffer for hardware data
+ * @cpu:	CPU pointer to the buffer.
+ * @dma:	DMA address of the buffer.
+ * @size:	Size of the buffer.
+ */
+struct rockchip_vpu_aux_buf {
+	void *cpu;
+	dma_addr_t dma;
+	size_t size;
+};
+
+/**
+ * struct rockchip_vpu_jpeg_enc_hw_ctx
+ * @bounce_buffer:	Bounce buffer
+ */
+struct rockchip_vpu_jpeg_enc_hw_ctx {
+	struct rockchip_vpu_aux_buf bounce_buffer;
+};
+
 /**
  * struct rockchip_vpu_codec_ops - codec mode specific operations
  *
+ * @start:	If needed, can be used for initialization.
+ *		Optional and called from process context.
+ * @stop:	If needed, can be used to undo the .start phase.
+ *		Optional and called from process context.
  * @run:	Start single {en,de)coding job. Called from atomic context
  *		to indicate that a pair of buffers is ready and the hardware
  *		should be programmed and started.
@@ -28,6 +52,8 @@ struct rockchip_vpu_variant;
  * @reset:	Reset the hardware in case of a timeout.
  */
 struct rockchip_vpu_codec_ops {
+	int (*start)(struct rockchip_vpu_ctx *ctx);
+	void (*stop)(struct rockchip_vpu_ctx *ctx);
 	void (*run)(struct rockchip_vpu_ctx *ctx);
 	void (*done)(struct rockchip_vpu_ctx *ctx, enum vb2_buffer_state);
 	void (*reset)(struct rockchip_vpu_ctx *ctx);
@@ -54,5 +80,7 @@ void rockchip_vpu_irq_done(struct rockchip_vpu_dev *vpu,
 
 void rk3288_vpu_jpeg_enc_run(struct rockchip_vpu_ctx *ctx);
 void rk3399_vpu_jpeg_enc_run(struct rockchip_vpu_ctx *ctx);
+int rockchip_vpu_jpeg_enc_start(struct rockchip_vpu_ctx *ctx);
+void rockchip_vpu_jpeg_enc_stop(struct rockchip_vpu_ctx *ctx);
 
 #endif /* ROCKCHIP_VPU_HW_H_ */
drivers/staging/media/rockchip/vpu/rockchip_vpu_jpeg.c +25 -0
index 0ff0badc1f7a..3244cca4e915 100644
@@ -6,9 +6,11 @@
  * Copyright (C) Jean-Francois Moine (http://moinejf.free.fr)
  * Copyright (C) 2014 Philipp Zabel, Pengutronix
  */
+#include <linux/dma-mapping.h>
 #include <linux/kernel.h>
 #include <linux/string.h>
 #include "rockchip_vpu_jpeg.h"
+#include "rockchip_vpu.h"
 
 #define LUMA_QUANT_OFF		7
 #define CHROMA_QUANT_OFF	72
@@ -288,3 +290,26 @@ void rockchip_vpu_jpeg_header_assemble(struct rockchip_vpu_jpeg_ctx *ctx)
 
 	jpeg_set_quality(buf, ctx->quality);
 }
+
+int rockchip_vpu_jpeg_enc_start(struct rockchip_vpu_ctx *ctx)
+{
+	ctx->jpeg_enc_ctx.bounce_buffer.size = ctx->dst_fmt.plane_fmt[0].sizeimage -
+			  ctx->vpu_dst_fmt->header_size;
+	ctx->jpeg_enc_ctx.bounce_buffer.cpu = dma_alloc_attrs(ctx->dev->dev,
+					ctx->jpeg_enc_ctx.bounce_buffer.size,
+					&ctx->jpeg_enc_ctx.bounce_buffer.dma,
+					GFP_KERNEL,
+					DMA_ATTR_ALLOC_SINGLE_PAGES);
+	if (!ctx->jpeg_enc_ctx.bounce_buffer.cpu)
+		return -ENOMEM;
+	return 0;
+}
+
+void rockchip_vpu_jpeg_enc_stop(struct rockchip_vpu_ctx *ctx)
+{
+	dma_free_attrs(ctx->dev->dev,
+		       ctx->jpeg_enc_ctx.bounce_buffer.size,
+		       ctx->jpeg_enc_ctx.bounce_buffer.cpu,
+		       ctx->jpeg_enc_ctx.bounce_buffer.dma,
+		       DMA_ATTR_ALLOC_SINGLE_PAGES);
+}
Signature · 1 line
2.20.1
#12 lore raw

PATCH 6/10 rockchip/vpu: Open-code media controller register

In reply to Ezequiel Garcia

In preparation to support decoders, using a single memory-to-memory device, we need to roll our own media controller entities registration.

---
 .../staging/media/rockchip/vpu/rockchip_vpu.h |  35 ++++
 .../media/rockchip/vpu/rockchip_vpu_drv.c     | 177 ++++++++++++++++--
 2 files changed, 200 insertions(+), 12 deletions(-)
drivers/staging/media/rockchip/vpu/rockchip_vpu.h +35 -0
index 76ee24abc141..084f58cadda1 100644
@@ -71,6 +71,38 @@ enum rockchip_vpu_codec_mode {
 	RK_VPU_MODE_JPEG_ENC,
 };
 
+/*
+ * struct rockchip_vpu_mc - media controller data
+ *
+ * @source:		&struct media_entity pointer with the source entity
+ *			Used only when the M2M device is registered via
+ *			v4l2_m2m_unregister_media_controller().
+ * @source_pad:		&struct media_pad with the source pad.
+ *			Used only when the M2M device is registered via
+ *			v4l2_m2m_unregister_media_controller().
+ * @sink:		&struct media_entity pointer with the sink entity
+ *			Used only when the M2M device is registered via
+ *			v4l2_m2m_unregister_media_controller().
+ * @sink_pad:		&struct media_pad with the sink pad.
+ *			Used only when the M2M device is registered via
+ *			v4l2_m2m_unregister_media_controller().
+ * @proc:		&struct media_entity pointer with the M2M device itself.
+ * @proc_pads:		&struct media_pad with the @proc pads.
+ *			Used only when the M2M device is registered via
+ *			v4l2_m2m_unregister_media_controller().
+ * @intf_devnode:	&struct media_intf devnode pointer with the interface
+ *			with controls the M2M device.
+ */
+struct rockchip_vpu_mc {
+	struct media_entity	*source;
+	struct media_pad	source_pad;
+	struct media_entity	sink;
+	struct media_pad	sink_pad;
+	struct media_entity	proc;
+	struct media_pad	proc_pads[2];
+	struct media_intf_devnode *intf_devnode;
+};
+
 /**
  * struct rockchip_vpu_dev - driver data
  * @v4l2_dev:		V4L2 device to register video devices for.
@@ -78,6 +110,8 @@ enum rockchip_vpu_codec_mode {
  * @mdev:		media device associated to this device.
  * @vfd_enc:		Video device for encoder.
  * @pdev:		Pointer to VPU platform device.
+ * @mc:			Array of media controller topology structs
+ *			for encoder and decoder.
  * @dev:		Pointer to device for convenient logging using
  *			dev_ macros.
  * @clocks:		Array of clock handles.
@@ -95,6 +129,7 @@ struct rockchip_vpu_dev {
 	struct media_device mdev;
 	struct video_device *vfd_enc;
 	struct platform_device *pdev;
+	struct rockchip_vpu_mc mc[2];
 	struct device *dev;
 	struct clk_bulk_data clocks[ROCKCHIP_VPU_MAX_CLOCKS];
 	void __iomem *base;
drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c +165 -12

215 lines not on this page · raw message

Signature · 1 line
2.20.1
#13 lore raw

PATCH 7/10 rockchip/vpu: Support the Request API

In reply to Ezequiel Garcia

Introduce support for the Request API. Although the JPEG encoder does not mandate using the Request API, it's perfectly possible to use it, if the application wants to.

In addition, add helpers that will be used by video decoders.

---
 .../rockchip/vpu/rk3399_vpu_hw_jpeg_enc.c     |  6 +++++
 .../media/rockchip/vpu/rockchip_vpu_common.h  |  3 +++
 .../media/rockchip/vpu/rockchip_vpu_drv.c     | 25 +++++++++++++++++++
 .../media/rockchip/vpu/rockchip_vpu_enc.c     | 18 +++++++++++++
 4 files changed, 52 insertions(+)
drivers/staging/media/rockchip/vpu/rk3399_vpu_hw_jpeg_enc.c +6 -0
index 222b6a3aa25e..1e2e51457106 100644
@@ -113,11 +113,15 @@ void rk3399_vpu_jpeg_enc_run(struct rockchip_vpu_ctx *ctx)
 	struct rockchip_vpu_dev *vpu = ctx->dev;
 	struct vb2_buffer *src_buf, *dst_buf;
 	struct rockchip_vpu_jpeg_ctx jpeg_ctx;
+	struct media_request *src_req;
 	u32 reg;
 
 	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
 	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
 
+	src_req = src_buf->req_obj.req;
+	v4l2_ctrl_request_setup(src_req, &ctx->ctrl_handler);
+
 	memset(&jpeg_ctx, 0, sizeof(jpeg_ctx));
 	jpeg_ctx.buffer = vb2_plane_vaddr(dst_buf, 0);
 	jpeg_ctx.width = ctx->dst_fmt.width;
@@ -153,6 +157,8 @@ void rk3399_vpu_jpeg_enc_run(struct rockchip_vpu_ctx *ctx)
 		| VEPU_REG_ENCODE_FORMAT_JPEG
 		| VEPU_REG_ENCODE_ENABLE;
 
+	v4l2_ctrl_request_complete(src_req, &ctx->ctrl_handler);
+
 	/* Kick the watchdog and start encoding */
 	schedule_delayed_work(&vpu->watchdog_work, msecs_to_jiffies(2000));
 	vepu_write(vpu, reg, VEPU_REG_ENCODE_START);
drivers/staging/media/rockchip/vpu/rockchip_vpu_common.h +3 -0
index ca77668d9579..ac018136a7bc 100644
@@ -26,4 +26,7 @@ void rockchip_vpu_enc_reset_src_fmt(struct rockchip_vpu_dev *vpu,
 void rockchip_vpu_enc_reset_dst_fmt(struct rockchip_vpu_dev *vpu,
 				    struct rockchip_vpu_ctx *ctx);
 
+void *rockchip_vpu_get_ctrl(struct rockchip_vpu_ctx *ctx, u32 id);
+dma_addr_t rockchip_vpu_get_ref(struct vb2_queue *q, u64 ts);
+
 #endif /* ROCKCHIP_VPU_COMMON_H_ */
drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c +25 -0
index 28e9e97ff257..968dd6700afd 100644
@@ -36,6 +36,24 @@ module_param_named(debug, rockchip_vpu_debug, int, 0644);
 MODULE_PARM_DESC(debug,
 		 "Debug level - higher value produces more verbose messages");
 
+void *rockchip_vpu_get_ctrl(struct rockchip_vpu_ctx *ctx, u32 id)
+{
+	struct v4l2_ctrl *ctrl;
+
+	ctrl = v4l2_ctrl_find(&ctx->ctrl_handler, id);
+	return ctrl ? ctrl->p_cur.p : NULL;
+}
+
+dma_addr_t rockchip_vpu_get_ref(struct vb2_queue *q, u64 ts)
+{
+	int index;
+
+	index = vb2_find_timestamp(q, ts, 0);
+	if (index >= 0)
+		return vb2_dma_contig_plane_dma_addr(q->bufs[index], 0);
+	return 0;
+}
+
 static void rockchip_vpu_job_finish(struct rockchip_vpu_dev *vpu,
 				    struct rockchip_vpu_ctx *ctx,
 				    unsigned int bytesused,
@@ -164,6 +182,7 @@ enc_queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq)
 	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
 	src_vq->lock = &ctx->dev->vpu_mutex;
 	src_vq->dev = ctx->dev->v4l2_dev.dev;
+	src_vq->supports_requests = true;
 
 	ret = vb2_queue_init(src_vq);
 	if (ret)
@@ -328,6 +347,11 @@ static const struct of_device_id of_rockchip_vpu_match[] = {
 };
 MODULE_DEVICE_TABLE(of, of_rockchip_vpu_match);
 
+static const struct media_device_ops rockchip_m2m_media_ops = {
+	.req_validate = vb2_request_validate,
+	.req_queue = v4l2_m2m_request_queue,
+};
+
 static int rockchip_vpu_video_device_register(struct rockchip_vpu_dev *vpu)
 {
 	const struct of_device_id *match;
@@ -608,6 +632,7 @@ static int rockchip_vpu_probe(struct platform_device *pdev)
 	vpu->mdev.dev = vpu->dev;
 	strlcpy(vpu->mdev.model, DRIVER_NAME, sizeof(vpu->mdev.model));
 	media_device_init(&vpu->mdev);
+	vpu->mdev.ops = &rockchip_m2m_media_ops;
 	vpu->v4l2_dev.mdev = &vpu->mdev;
 
 	ret = rockchip_vpu_video_device_register(vpu);
drivers/staging/media/rockchip/vpu/rockchip_vpu_enc.c +18 -0
index 2b28403314bc..1b5a675ef24f 100644
@@ -555,14 +555,32 @@ static void rockchip_vpu_stop_streaming(struct vb2_queue *q)
 			vbuf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 		if (!vbuf)
 			break;
+		v4l2_ctrl_request_complete(vbuf->vb2_buf.req_obj.req, &ctx->ctrl_handler);
 		v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR);
 	}
 }
 
+static void rockchip_vpu_buf_request_complete(struct vb2_buffer *vb)
+{
+	struct rockchip_vpu_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+
+	v4l2_ctrl_request_complete(vb->req_obj.req, &ctx->ctrl_handler);
+}
+
+static int rockchip_vpu_buf_out_validate(struct vb2_buffer *vb)
+{
+	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+
+	vbuf->field = V4L2_FIELD_NONE;
+	return 0;
+}
+
 const struct vb2_ops rockchip_vpu_enc_queue_ops = {
 	.queue_setup = rockchip_vpu_queue_setup,
 	.buf_prepare = rockchip_vpu_buf_prepare,
 	.buf_queue = rockchip_vpu_buf_queue,
+	.buf_out_validate = rockchip_vpu_buf_out_validate,
+	.buf_request_complete = rockchip_vpu_buf_request_complete,
 	.start_streaming = rockchip_vpu_start_streaming,
 	.stop_streaming = rockchip_vpu_stop_streaming,
 	.wait_prepare = vb2_ops_wait_prepare,
Signature · 1 line
2.20.1
#14 lore raw

PATCH 8/10 rockchip/vpu: Add decoder boilerplate

In reply to Ezequiel Garcia

This commit adds the needed boilerplate code to support the VPU in decoding operation. Two v4l2 interfaces are exposed, one for encoding and one for decoding, but a single m2m device is shared by them, so jobs are properly serialized.

---
 drivers/staging/media/rockchip/vpu/Makefile   |   1 +
 .../media/rockchip/vpu/rk3399_vpu_hw.c        |  19 +
 .../staging/media/rockchip/vpu/rockchip_vpu.h |  35 ++
 .../media/rockchip/vpu/rockchip_vpu_common.h  |   6 +
 .../media/rockchip/vpu/rockchip_vpu_dec.c     | 557 ++++++++++++++++++
 .../media/rockchip/vpu/rockchip_vpu_drv.c     | 130 +++-
 6 files changed, 737 insertions(+), 11 deletions(-)
 create mode 100644 drivers/staging/media/rockchip/vpu/rockchip_vpu_dec.c
drivers/staging/media/rockchip/vpu/Makefile +1 -0
index e9d733bb7632..b9041a139212 100644
@@ -3,6 +3,7 @@ obj-$(CONFIG_VIDEO_ROCKCHIP_VPU) += rockchip-vpu.o
 rockchip-vpu-y += \
 		rockchip_vpu_drv.o \
 		rockchip_vpu_enc.o \
+		rockchip_vpu_dec.o \
 		rk3288_vpu_hw.o \
 		rk3288_vpu_hw_jpeg_enc.o \
 		rk3399_vpu_hw.o \
drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c +19 -0
index 8469e474c975..0263584e616d 100644
@@ -74,6 +74,24 @@ static irqreturn_t rk3399_vepu_irq(int irq, void *dev_id)
 	return IRQ_HANDLED;
 }
 
+static irqreturn_t rk3399_vdpu_irq(int irq, void *dev_id)
+{
+	struct rockchip_vpu_dev *vpu = dev_id;
+	enum vb2_buffer_state state;
+	u32 status;
+
+	status = vdpu_read(vpu, VDPU_REG_INTERRUPT);
+	state = (status & VDPU_REG_INTERRUPT_DEC_IRQ) ?
+		VB2_BUF_STATE_DONE : VB2_BUF_STATE_ERROR;
+
+	vdpu_write(vpu, 0, VDPU_REG_INTERRUPT);
+	vdpu_write(vpu, 0, VDPU_REG_AXI_CTRL);
+
+	rockchip_vpu_irq_done(vpu, 0, state);
+
+	return IRQ_HANDLED;
+}
+
 static int rk3399_vpu_hw_init(struct rockchip_vpu_dev *vpu)
 {
 	/* Bump ACLK to max. possible freq. to improve performance. */
@@ -114,6 +132,7 @@ const struct rockchip_vpu_variant rk3399_vpu_variant = {
 	.codec = RK_VPU_CODEC_JPEG,
 	.codec_ops = rk3399_vpu_codec_ops,
 	.vepu_irq = rk3399_vepu_irq,
+	.vdpu_irq = rk3399_vdpu_irq,
 	.init = rk3399_vpu_hw_init,
 	.clk_names = {"aclk", "hclk"},
 	.num_clocks = 2
drivers/staging/media/rockchip/vpu/rockchip_vpu.h +35 -0
index 084f58cadda1..b383c89ecc17 100644
@@ -40,23 +40,31 @@ struct rockchip_vpu_codec_ops;
  * struct rockchip_vpu_variant - information about VPU hardware variant
  *
  * @enc_offset:			Offset from VPU base to encoder registers.
+ * @dec_offset:			Offset from VPU base to decoder registers.
  * @enc_fmts:			Encoder formats.
  * @num_enc_fmts:		Number of encoder formats.
+ * @dec_fmts:			Decoder formats.
+ * @num_dec_fmts:		Number of decoder formats.
  * @codec:			Supported codecs
  * @codec_ops:			Codec ops.
  * @init:			Initialize hardware.
  * @vepu_irq:			encoder interrupt handler
+ * @vdpu_irq:			decoder interrupt handler
  * @clk_names:			array of clock names
  * @num_clocks:			number of clocks in the array
  */
 struct rockchip_vpu_variant {
 	unsigned int enc_offset;
+	unsigned int dec_offset;
 	const struct rockchip_vpu_fmt *enc_fmts;
 	unsigned int num_enc_fmts;
+	const struct rockchip_vpu_fmt *dec_fmts;
+	unsigned int num_dec_fmts;
 	unsigned int codec;
 	const struct rockchip_vpu_codec_ops *codec_ops;
 	int (*init)(struct rockchip_vpu_dev *vpu);
 	irqreturn_t (*vepu_irq)(int irq, void *priv);
+	irqreturn_t (*vdpu_irq)(int irq, void *priv);
 	const char *clk_names[ROCKCHIP_VPU_MAX_CLOCKS];
 	int num_clocks;
 };
@@ -109,6 +117,7 @@ struct rockchip_vpu_mc {
  * @m2m_dev:		mem2mem device associated to this device.
  * @mdev:		media device associated to this device.
  * @vfd_enc:		Video device for encoder.
+ * @vfd_dec:		Video device for decoder.
  * @pdev:		Pointer to VPU platform device.
  * @mc:			Array of media controller topology structs
  *			for encoder and decoder.
@@ -117,6 +126,7 @@ struct rockchip_vpu_mc {
  * @clocks:		Array of clock handles.
  * @base:		Mapped address of VPU registers.
  * @enc_base:		Mapped address of VPU encoder register for convenience.
+ * @dec_base:		Mapped address of VPU decoder register for convenience.
  * @vpu_mutex:		Mutex to synchronize V4L2 calls.
  * @irqlock:		Spinlock to synchronize access to data structures
  *			shared with interrupt handlers.
@@ -128,12 +138,14 @@ struct rockchip_vpu_dev {
 	struct v4l2_m2m_dev *m2m_dev;
 	struct media_device mdev;
 	struct video_device *vfd_enc;
+	struct video_device *vfd_dec;
 	struct platform_device *pdev;
 	struct rockchip_vpu_mc mc[2];
 	struct device *dev;
 	struct clk_bulk_data clocks[ROCKCHIP_VPU_MAX_CLOCKS];
 	void __iomem *base;
 	void __iomem *enc_base;
+	void __iomem *dec_base;
 
 	struct mutex vpu_mutex;	/* video_device lock */
 	spinlock_t irqlock;
@@ -146,6 +158,7 @@ struct rockchip_vpu_dev {
  *
  * @dev:		VPU driver data to which the context belongs.
  * @fh:			V4L2 file handler.
+ * @is_enc:		Encoder or decoder?
  *
  * @sequence_cap:       Sequence counter for capture queue
  * @sequence_out:       Sequence counter for output queue
@@ -164,6 +177,7 @@ struct rockchip_vpu_dev {
 struct rockchip_vpu_ctx {
 	struct rockchip_vpu_dev *dev;
 	struct v4l2_fh fh;
+	unsigned int is_enc;
 
 	u32 sequence_cap;
 	u32 sequence_out;
@@ -262,4 +276,25 @@ static inline u32 vepu_read(struct rockchip_vpu_dev *vpu, u32 reg)
 	return val;
 }
 
+static inline void vdpu_write_relaxed(struct rockchip_vpu_dev *vpu,
+				      u32 val, u32 reg)
+{
+	vpu_debug(6, "0x%04x = 0x%08x\n", reg / 4, val);
+	writel_relaxed(val, vpu->dec_base + reg);
+}
+
+static inline void vdpu_write(struct rockchip_vpu_dev *vpu, u32 val, u32 reg)
+{
+	vpu_debug(6, "0x%04x = 0x%08x\n", reg / 4, val);
+	writel(val, vpu->dec_base + reg);
+}
+
+static inline u32 vdpu_read(struct rockchip_vpu_dev *vpu, u32 reg)
+{
+	u32 val = readl(vpu->dec_base + reg);
+
+	vpu_debug(6, "0x%04x = 0x%08x\n", reg / 4, val);
+	return val;
+}
+
 #endif /* ROCKCHIP_VPU_H_ */
drivers/staging/media/rockchip/vpu/rockchip_vpu_common.h +6 -0
index ac018136a7bc..7e5fce3bf215 100644
@@ -19,12 +19,18 @@
 #include "rockchip_vpu.h"
 
 extern const struct v4l2_ioctl_ops rockchip_vpu_enc_ioctl_ops;
+extern const struct v4l2_ioctl_ops rockchip_vpu_dec_ioctl_ops;
 extern const struct vb2_ops rockchip_vpu_enc_queue_ops;
+extern const struct vb2_ops rockchip_vpu_dec_queue_ops;
 
 void rockchip_vpu_enc_reset_src_fmt(struct rockchip_vpu_dev *vpu,
 				    struct rockchip_vpu_ctx *ctx);
 void rockchip_vpu_enc_reset_dst_fmt(struct rockchip_vpu_dev *vpu,
 				    struct rockchip_vpu_ctx *ctx);
+void rockchip_vpu_dec_reset_src_fmt(struct rockchip_vpu_dev *vpu,
+				    struct rockchip_vpu_ctx *ctx);
+void rockchip_vpu_dec_reset_dst_fmt(struct rockchip_vpu_dev *vpu,
+				    struct rockchip_vpu_ctx *ctx);
 
 void *rockchip_vpu_get_ctrl(struct rockchip_vpu_ctx *ctx, u32 id);
 dma_addr_t rockchip_vpu_get_ref(struct vb2_queue *q, u64 ts);
drivers/staging/media/rockchip/vpu/rockchip_vpu_dec.c new file +557 -0

557 lines not on this page · raw message

drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c +119 -11

212 lines not on this page · raw message

Signature · 1 line
2.20.1
#15 lore raw

PATCH 9/10 rockchip/vpu: Add support for non-standard controls

In reply to Ezequiel Garcia

Rework the way controls are registered by the driver, so it can support non-standard controls, such as those used by stateless codecs.

---
 .../media/rockchip/vpu/rk3288_vpu_hw.c        |  2 +-
 .../media/rockchip/vpu/rk3399_vpu_hw.c        |  2 +-
 .../staging/media/rockchip/vpu/rockchip_vpu.h | 24 ++++++-
 .../media/rockchip/vpu/rockchip_vpu_common.h  |  1 +
 .../media/rockchip/vpu/rockchip_vpu_drv.c     | 65 +++++++++++++++++--
 5 files changed, 84 insertions(+), 10 deletions(-)
drivers/staging/media/rockchip/vpu/rk3288_vpu_hw.c +1 -1
index 056ee017c798..630eded99c68 100644
@@ -112,7 +112,7 @@ const struct rockchip_vpu_variant rk3288_vpu_variant = {
 	.enc_fmts = rk3288_vpu_enc_fmts,
 	.num_enc_fmts = ARRAY_SIZE(rk3288_vpu_enc_fmts),
 	.codec_ops = rk3288_vpu_codec_ops,
-	.codec = RK_VPU_CODEC_JPEG,
+	.codec = RK_VPU_JPEG_ENCODER,
 	.vepu_irq = rk3288_vepu_irq,
 	.init = rk3288_vpu_hw_init,
 	.clk_names = {"aclk", "hclk"},
drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c +1 -1
index 0263584e616d..9eae1e6f1393 100644
@@ -129,7 +129,7 @@ const struct rockchip_vpu_variant rk3399_vpu_variant = {
 	.enc_offset = 0x0,
 	.enc_fmts = rk3399_vpu_enc_fmts,
 	.num_enc_fmts = ARRAY_SIZE(rk3399_vpu_enc_fmts),
-	.codec = RK_VPU_CODEC_JPEG,
+	.codec = RK_VPU_JPEG_ENCODER,
 	.codec_ops = rk3399_vpu_codec_ops,
 	.vepu_irq = rk3399_vepu_irq,
 	.vdpu_irq = rk3399_vdpu_irq,
drivers/staging/media/rockchip/vpu/rockchip_vpu.h +23 -1
index b383c89ecc17..a90fc2dfae99 100644
@@ -25,6 +25,7 @@
 
 #include "rockchip_vpu_hw.h"
 
+#define ROCKCHIP_VPU_MAX_CTRLS          32
 #define ROCKCHIP_VPU_MAX_CLOCKS		4
 
 #define JPEG_MB_DIM			16
@@ -34,7 +35,10 @@
 struct rockchip_vpu_ctx;
 struct rockchip_vpu_codec_ops;
 
-#define RK_VPU_CODEC_JPEG BIT(0)
+#define RK_VPU_JPEG_ENCODER	BIT(0)
+#define RK_VPU_ENCODERS		0x0000ffff
+
+#define RK_VPU_DECODERS		0xffff0000
 
 /**
  * struct rockchip_vpu_variant - information about VPU hardware variant
@@ -79,6 +83,20 @@ enum rockchip_vpu_codec_mode {
 	RK_VPU_MODE_JPEG_ENC,
 };
 
+/*
+ * struct rockchip_vpu_ctrl - helper type to declare supported controls
+ * @id:		V4L2 control ID (V4L2_CID_xxx)
+ * @is_std:	boolean to distinguish standard from customs control.
+ * @codec:	codec id this control belong to (RK_VPU_JPEG_ENCODER, etc.)
+ * @cfg:	control configuration
+ */
+struct rockchip_vpu_ctrl {
+	unsigned int id;
+	unsigned int is_std;
+	unsigned int codec;
+	struct v4l2_ctrl_config cfg;
+};
+
 /*
  * struct rockchip_vpu_mc - media controller data
  *
@@ -169,6 +187,8 @@ struct rockchip_vpu_dev {
  * @dst_fmt:		V4L2 pixel format of active destination format.
  *
  * @ctrl_handler:	Control handler used to register controls.
+ * @ctrls:		Array of supported controls.
+ * @num_ctrls:		Number of controls populated in the array.
  * @jpeg_quality:	User-specified JPEG compression quality.
  *
  * @codec_ops:		Set of operations related to codec mode.
@@ -188,6 +208,8 @@ struct rockchip_vpu_ctx {
 	struct v4l2_pix_format_mplane dst_fmt;
 
 	struct v4l2_ctrl_handler ctrl_handler;
+	struct v4l2_ctrl *ctrls[ROCKCHIP_VPU_MAX_CTRLS];
+	unsigned int num_ctrls;
 	int jpeg_quality;
 
 	const struct rockchip_vpu_codec_ops *codec_ops;
drivers/staging/media/rockchip/vpu/rockchip_vpu_common.h +1 -0
index 7e5fce3bf215..70b8ac1c7503 100644
@@ -23,6 +23,7 @@ extern const struct v4l2_ioctl_ops rockchip_vpu_dec_ioctl_ops;
 extern const struct vb2_ops rockchip_vpu_enc_queue_ops;
 extern const struct vb2_ops rockchip_vpu_dec_queue_ops;
 
+void *rockchip_vpu_find_control_data(struct rockchip_vpu_ctx *ctx, unsigned int id);
 void rockchip_vpu_enc_reset_src_fmt(struct rockchip_vpu_dev *vpu,
 				    struct rockchip_vpu_ctx *ctx);
 void rockchip_vpu_enc_reset_dst_fmt(struct rockchip_vpu_dev *vpu,
drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c +58 -7

84 lines not on this page · raw message

Signature · 1 line
2.20.1
#16 lore raw

PATCH 10/10 rockchip/vpu: Add support for MPEG-2 decoding

In reply to Ezequiel Garcia

From: Jonas Karlman <[email protected]>

Add MPEG-2 decoding, only on RK3399. Other SoCs and support for other codecs will be added in the future.

---
 drivers/staging/media/rockchip/vpu/Makefile   |   4 +-
 .../media/rockchip/vpu/rk3399_vpu_hw.c        |  40 ++-
 .../rockchip/vpu/rk3399_vpu_hw_mpeg2_dec.c    | 263 ++++++++++++++++++
 .../staging/media/rockchip/vpu/rockchip_vpu.h |   9 +
 .../media/rockchip/vpu/rockchip_vpu_drv.c     |  12 +
 .../media/rockchip/vpu/rockchip_vpu_hw.h      |  14 +
 .../media/rockchip/vpu/rockchip_vpu_mpeg2.c   |  61 ++++
 7 files changed, 401 insertions(+), 2 deletions(-)
 create mode 100644 drivers/staging/media/rockchip/vpu/rk3399_vpu_hw_mpeg2_dec.c
 create mode 100644 drivers/staging/media/rockchip/vpu/rockchip_vpu_mpeg2.c
drivers/staging/media/rockchip/vpu/Makefile +3 -1
index b9041a139212..86e57776f0ec 100644
@@ -8,4 +8,6 @@ rockchip-vpu-y += \
 		rk3288_vpu_hw_jpeg_enc.o \
 		rk3399_vpu_hw.o \
 		rk3399_vpu_hw_jpeg_enc.o \
-		rockchip_vpu_jpeg.o
+		rockchip_vpu_jpeg.o \
+		rk3399_vpu_hw_mpeg2_dec.o \
+		rockchip_vpu_mpeg2.o
drivers/staging/media/rockchip/vpu/rk3399_vpu_hw.c +39 -1
index 9eae1e6f1393..0ea184162a6c 100644
@@ -55,6 +55,26 @@ static const struct rockchip_vpu_fmt rk3399_vpu_enc_fmts[] = {
 	},
 };
 
+static const struct rockchip_vpu_fmt rk3399_vpu_dec_fmts[] = {
+	{
+		.fourcc = V4L2_PIX_FMT_NV12,
+		.codec_mode = RK_VPU_MODE_NONE,
+	},
+	{
+		.fourcc = V4L2_PIX_FMT_MPEG2_SLICE,
+		.codec_mode = RK_VPU_MODE_MPEG2_DEC,
+		.max_depth = 2,
+		.frmsize = {
+			.min_width = 48,
+			.max_width = 1920,
+			.step_width = MPEG2_MB_DIM,
+			.min_height = 48,
+			.max_height = 1088,
+			.step_height = MPEG2_MB_DIM,
+		},
+	},
+};
+
 static irqreturn_t rk3399_vepu_irq(int irq, void *dev_id)
 {
 	struct rockchip_vpu_dev *vpu = dev_id;
@@ -108,6 +128,15 @@ static void rk3399_vpu_enc_reset(struct rockchip_vpu_ctx *ctx)
 	vepu_write(vpu, 0, VEPU_REG_AXI_CTRL);
 }
 
+static void rk3399_vpu_dec_reset(struct rockchip_vpu_ctx *ctx)
+{
+	struct rockchip_vpu_dev *vpu = ctx->dev;
+
+	vdpu_write(vpu, VDPU_REG_INTERRUPT_DEC_IRQ_DIS, VDPU_REG_INTERRUPT);
+	vdpu_write(vpu, 0, VDPU_REG_EN_FLAGS);
+	vdpu_write(vpu, 1, VDPU_REG_SOFT_RESET);
+}
+
 /*
  * Supported codec ops.
  */
@@ -119,6 +148,12 @@ static const struct rockchip_vpu_codec_ops rk3399_vpu_codec_ops[] = {
 		.start = rockchip_vpu_jpeg_enc_start,
 		.stop = rockchip_vpu_jpeg_enc_stop,
 	},
+	[RK_VPU_MODE_MPEG2_DEC] = {
+		.run = rk3399_vpu_mpeg2_dec_run,
+		.reset = rk3399_vpu_dec_reset,
+		.start = rockchip_vpu_mpeg2_dec_start,
+		.stop = rockchip_vpu_mpeg2_dec_stop,
+	},
 };
 
 /*
@@ -129,7 +164,10 @@ const struct rockchip_vpu_variant rk3399_vpu_variant = {
 	.enc_offset = 0x0,
 	.enc_fmts = rk3399_vpu_enc_fmts,
 	.num_enc_fmts = ARRAY_SIZE(rk3399_vpu_enc_fmts),
-	.codec = RK_VPU_JPEG_ENCODER,
+	.dec_offset = 0x400,
+	.dec_fmts = rk3399_vpu_dec_fmts,
+	.num_dec_fmts = ARRAY_SIZE(rk3399_vpu_dec_fmts),
+	.codec = RK_VPU_JPEG_ENCODER | RK_VPU_MPEG2_DECODER,
 	.codec_ops = rk3399_vpu_codec_ops,
 	.vepu_irq = rk3399_vepu_irq,
 	.vdpu_irq = rk3399_vdpu_irq,
drivers/staging/media/rockchip/vpu/rk3399_vpu_hw_mpeg2_dec.c new file +263 -0

263 lines not on this page · raw message

drivers/staging/media/rockchip/vpu/rockchip_vpu.h +9 -0
index a90fc2dfae99..2e178b745c64 100644
@@ -28,6 +28,10 @@
 #define ROCKCHIP_VPU_MAX_CTRLS          32
 #define ROCKCHIP_VPU_MAX_CLOCKS		4
 
+#define MPEG2_MB_DIM			16
+#define MPEG2_MB_WIDTH(w)		DIV_ROUND_UP(w, MPEG2_MB_DIM)
+#define MPEG2_MB_HEIGHT(h)		DIV_ROUND_UP(h, MPEG2_MB_DIM)
+
 #define JPEG_MB_DIM			16
 #define JPEG_MB_WIDTH(w)		DIV_ROUND_UP(w, JPEG_MB_DIM)
 #define JPEG_MB_HEIGHT(h)		DIV_ROUND_UP(h, JPEG_MB_DIM)
@@ -38,6 +42,7 @@ struct rockchip_vpu_codec_ops;
 #define RK_VPU_JPEG_ENCODER	BIT(0)
 #define RK_VPU_ENCODERS		0x0000ffff
 
+#define RK_VPU_MPEG2_DECODER	BIT(16)
 #define RK_VPU_DECODERS		0xffff0000
 
 /**
@@ -77,10 +82,12 @@ struct rockchip_vpu_variant {
  * enum rockchip_vpu_codec_mode - codec operating mode.
  * @RK_VPU_MODE_NONE:  No operating mode. Used for RAW video formats.
  * @RK_VPU_MODE_JPEG_ENC: JPEG encoder.
+ * @RK_VPU_MODE_MPEG2_DEC: MPEG-2 decoder.
  */
 enum rockchip_vpu_codec_mode {
 	RK_VPU_MODE_NONE = -1,
 	RK_VPU_MODE_JPEG_ENC,
+	RK_VPU_MODE_MPEG2_DEC,
 };
 
 /*
@@ -193,6 +200,7 @@ struct rockchip_vpu_dev {
  *
  * @codec_ops:		Set of operations related to codec mode.
  * @jpeg_enc_ctx:	JPEG-encoding context.
+ * @mpeg2_dec_ctx:	MPEG-2-decoding context.
  */
 struct rockchip_vpu_ctx {
 	struct rockchip_vpu_dev *dev;
@@ -217,6 +225,7 @@ struct rockchip_vpu_ctx {
 	/* Specific for particular codec modes. */
 	union {
 		struct rockchip_vpu_jpeg_enc_hw_ctx jpeg_enc_ctx;
+		struct rockchip_vpu_mpeg2_dec_hw_ctx mpeg2_dec_ctx;
 	};
 };
 
drivers/staging/media/rockchip/vpu/rockchip_vpu_drv.c +12 -0
index df4c4e953742..82cd11b8d4d6 100644
@@ -300,6 +300,18 @@ static struct rockchip_vpu_ctrl controls[] = {
 			.step = 1,
 			.def = 50,
 		},
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS,
+		.codec = RK_VPU_MPEG2_DECODER,
+		.cfg = {
+			.elem_size = sizeof(struct v4l2_ctrl_mpeg2_slice_params),
+		},
+	}, {
+		.id = V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION,
+		.codec = RK_VPU_MPEG2_DECODER,
+		.cfg = {
+			.elem_size = sizeof(struct v4l2_ctrl_mpeg2_quantization),
+		},
 	},
 };
 
drivers/staging/media/rockchip/vpu/rockchip_vpu_hw.h +14 -0
index e2e84526f263..fa0a17840da7 100644
@@ -38,6 +38,14 @@ struct rockchip_vpu_jpeg_enc_hw_ctx {
 	struct rockchip_vpu_aux_buf bounce_buffer;
 };
 
+/**
+ * struct rockchip_vpu_mpeg2_dec_hw_ctx
+ * @qtable:		Quantization table
+ */
+struct rockchip_vpu_mpeg2_dec_hw_ctx {
+	struct rockchip_vpu_aux_buf qtable;
+};
+
 /**
  * struct rockchip_vpu_codec_ops - codec mode specific operations
  *
@@ -83,4 +91,10 @@ void rk3399_vpu_jpeg_enc_run(struct rockchip_vpu_ctx *ctx);
 int rockchip_vpu_jpeg_enc_start(struct rockchip_vpu_ctx *ctx);
 void rockchip_vpu_jpeg_enc_stop(struct rockchip_vpu_ctx *ctx);
 
+void rk3399_vpu_mpeg2_dec_run(struct rockchip_vpu_ctx *ctx);
+void rockchip_vpu_mpeg2_dec_copy_qtable(u8 *qtable,
+	const struct v4l2_ctrl_mpeg2_quantization *ctrl);
+int rockchip_vpu_mpeg2_dec_start(struct rockchip_vpu_ctx *ctx);
+void rockchip_vpu_mpeg2_dec_stop(struct rockchip_vpu_ctx *ctx);
+
 #endif /* ROCKCHIP_VPU_HW_H_ */
drivers/staging/media/rockchip/vpu/rockchip_vpu_mpeg2.c new file +61 -0

61 lines not on this page · raw message

Signature · 1 line
2.20.1