media: vim2m: add multiplanar API support
cover media: vim2m: add multiplanar API support
Hi everyone,
This patch series adds multiplane API support for the virtual M2M driver, along with some minor driver refactoring/improvements.
I followed the lead of the vivid driver and made multiplanar support selectable with a module parameter, and the default is to use the single planar api.
Although there are not yet any pixelformats in the driver that make use of multiple memory planes, adding support for these should be easier now with the API level changes taken care of.
v4l2-compliance reports the following with multiplane support disabled:
Total for vim2m device /dev/video0: 48, Succeeded: 48, Failed: 0, Warnings: 0
and the same with multiplane support enabled:
Total for vim2m device /dev/video0: 48, Succeeded: 48, Failed: 0, Warnings: 0
Patches need to be applied in increasing numerical order (Patch [3/3] depends on [1/3] and [2/3]).
Since the multi-plane changes had to touch a lot of the driver, I did a basic regression test with the following script which generates a test input image with vivid and an output image from vim2m for each supported format. I confirmed all outputs visually and verified they were identical to the outputs before the change. Testing was done on an x86_64 qemu image.
#!/bin/sh
# tested with HDMI vivid emulation # modprobe vivid num_inputs=1 input_types=3
vim2m=/dev/video0
vivid=/dev/video1
width=640
height=480
out_width=320
out_height=240
capture_formats=$(v4l2-ctl -d $vim2m --list-formats | awk '/\]:/ {print $2}' | sed "s/'//g") output_formats=$(v4l2-ctl -d $vim2m --list-formats-out | awk '/\]:/ {print $2}' | sed "s/'//g")
# Turn off text mode so that images will be identical v4l2-ctl -d $vivid -c osd_text_mode=2
for ofmt in ${output_formats}; do
# generate input image
inname="${width}x${height}.${ofmt}"
v4l2-ctl -d $vivid -v pixelformat=$ofmt,width=$width,height=$height,field=none \
--stream-mmap --stream-count=1 --stream-to=$inname
for cfmt in ${capture_formats}; do
outname="${out_width}x${out_height}-out.${cfmt}"
v4l2-ctl -d $vim2m -x pixelformat=$ofmt,width=$width,height=$height \
-v pixelformat=$cfmt,width=$out_width,height=$out_height \
--stream-from=$inname --stream-to=$outname --stream-mmap --stream-out-mmap \
--stream-count=1
done
doneMatthew Majewski (3):
media: v4l2-common: Add RGBR format info
media: vim2m: Simplify try_fmt
media: vim2m: Add parametized support for multiplanar API
drivers/media/test-drivers/vim2m.c | 327 +++++++++++++++++++++----- drivers/media/v4l2-core/v4l2-common.c | 1 + 2 files changed, 274 insertions(+), 54 deletions(-)
Signature · 1 line
2.25.1
PATCH 1/3 media: v4l2-common: Add RGBR format info
Add missing RGBR entry in the v4l2_format_info[] table.
RGBR has identical format information to RGBP, as it is a big endian variant of RGB-5-6-5 pixel encoding according to the description in videodev2.h.
--- drivers/media/v4l2-core/v4l2-common.c | 1 + 1 file changed, 1 insertion(+)
drivers/media/v4l2-core/v4l2-common.c +1 -0
@@ -250,6 +250,7 @@ const struct v4l2_format_info *v4l2_format_info(u32 format) { .format = V4L2_PIX_FMT_ABGR32, .pixel_enc = V4L2_PIXEL_ENC_RGB, .mem_planes = 1, .comp_planes = 1, .bpp = { 4, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 }, { .format = V4L2_PIX_FMT_BGRA32, .pixel_enc = V4L2_PIXEL_ENC_RGB, .mem_planes = 1, .comp_planes = 1, .bpp = { 4, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 }, { .format = V4L2_PIX_FMT_RGB565, .pixel_enc = V4L2_PIXEL_ENC_RGB, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 }, + { .format = V4L2_PIX_FMT_RGB565X, .pixel_enc = V4L2_PIXEL_ENC_RGB, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 }, { .format = V4L2_PIX_FMT_RGB555, .pixel_enc = V4L2_PIXEL_ENC_RGB, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 }, { .format = V4L2_PIX_FMT_BGR666, .pixel_enc = V4L2_PIXEL_ENC_RGB, .mem_planes = 1, .comp_planes = 1, .bpp = { 4, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 }, { .format = V4L2_PIX_FMT_BGR48_12, .pixel_enc = V4L2_PIXEL_ENC_RGB, .mem_planes = 1, .comp_planes = 1, .bpp = { 6, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
Signature · 1 line
2.25.1
PATCH 2/3 media: vim2m: Simplify try_fmt
In reply to Matthew MajewskiClean up vidioc_try_fmt with the following changes:
1. remove unsused vim2m_fmt parameter
2. use clamp() macro to restrain width/height bounds
3. use ALIGN() macro to align width/height
4. use v4l2_fill_pixfmt to set bytesperline/sizeimage
--- drivers/media/test-drivers/vim2m.c | 33 +++++++++++------------------- 1 file changed, 12 insertions(+), 21 deletions(-)
drivers/media/test-drivers/vim2m.c +12 -21
@@ -26,6 +26,7 @@#include <media/v4l2-ctrls.h> #include <media/v4l2-event.h> #include <media/videobuf2-vmalloc.h> +#include <media/v4l2-common.h> MODULE_DESCRIPTION ("Virtual device for mem2mem framework testing" );MODULE_AUTHOR ("Pawel Osciak, <[email protected]>" );
@@ -755,31 +756,21 @@ static int vidioc_g_fmt_vid_cap(struct file *file, void *priv,return vidioc_g_fmt (file2ctx (file), f); } -static int vidioc_try_fmt (struct v4l2_format *f,struct vim2m_fmt *fmt) +static int vidioc_try_fmt (struct v4l2_format *f) { -int walign, halign; -/* -* V4L2 specification specifies the driver corrects the -* format struct if any of the dimensions is unsupported -*/ -if (f->fmt.pix.height <MIN_H ) - f->fmt.pix.height =MIN_H ; -else if (f->fmt.pix.height >MAX_H ) - f->fmt.pix.height =MAX_H ; +int width, height, walign, halign; -if (f->fmt.pix.width <MIN_W ) - f->fmt.pix.width =MIN_W ; -else if (f->fmt.pix.width >MAX_W ) - f->fmt.pix.width =MAX_W ; + width =clamp (f->fmt.pix.width,MIN_W ,MAX_W ); + height =clamp (f->fmt.pix.width,MIN_H ,MAX_H );get_alignment (f->fmt.pix.pixelformat, &walign, &halign); - f->fmt.pix.width &= ~(walign -1 ); - f->fmt.pix.height &= ~(halign -1 ); - f->fmt.pix.bytesperline = (f->fmt.pix.width * fmt->depth) >>3 ; - f->fmt.pix.sizeimage = f->fmt.pix.height * f->fmt.pix.bytesperline; + width =ALIGN (width, walign); + height =ALIGN (height, halign); + f->fmt.pix.field =V4L2_FIELD_NONE ; -return 0 ; +return v4l2_fill_pixfmt (&f->fmt.pix, f->fmt.pix.pixelformat, + width, height); } static int vidioc_try_fmt_vid_cap(struct file *file, void *priv,
@@ -804,7 +795,7 @@ static int vidioc_try_fmt_vid_cap(struct file *file, void *priv, f->fmt.pix.ycbcr_enc = ctx->ycbcr_enc; f->fmt.pix.quantization = ctx->quant; -return vidioc_try_fmt (f, fmt); +return vidioc_try_fmt (f); } static int vidioc_try_fmt_vid_out(struct file *file, void *priv,
@@ -827,7 +818,7 @@ static int vidioc_try_fmt_vid_out(struct file *file, void *priv,if (!f->fmt.pix.colorspace) f->fmt.pix.colorspace =V4L2_COLORSPACE_REC709 ; -return vidioc_try_fmt (f, fmt); +return vidioc_try_fmt (f); }static int vidioc_s_fmt (struct vim2m_ctx *ctx,struct v4l2_format *f)
Signature · 1 line
2.25.1
PATCH 3/3 media: vim2m: Add parametized support for multiplanar API
In reply to Matthew MajewskiAdd support for the mulitiplaner API. The device can now act as either a multi-planar or a single-planar device depending on a module parameter, similar to the way vivid behaves.
Multiplanar support was added by implementing the appropate try/get/set mplane functions, and by modifying the queue_setup() and buf_prepare() functions to handle multiple planes. Implementation was inspired by vivid.
--- drivers/media/test-drivers/vim2m.c | 306 +++++++++++++++++++++++++---- 1 file changed, 267 insertions(+), 39 deletions(-)
drivers/media/test-drivers/vim2m.c +267 -39
@@ -43,6 +43,10 @@ static unsigned int default_transtime = 40; /* Max 25 fps */module_param (default_transtime, uint,0644 );MODULE_PARM_DESC (default_transtime,"default transaction time in ms" ); +static unsigned int multiplanar =1 ; +module_param (multiplanar, uint,0644 ); +MODULE_PARM_DESC (multiplanar,"1 (default) creates a single planar device, 2 creates multiplanar device." ); +#define MIN_W 32#define MIN_H 32#define MAX_W 640
@@ -135,7 +139,8 @@ static struct vim2m_fmt formats[] = {struct vim2m_q_data {unsigned int width;unsigned int height; -unsigned int sizeimage; +unsigned int num_mem_planes; +unsigned int sizeimage[VIDEO_MAX_PLANES ];unsigned int sequence;struct vim2m_fmt *fmt; };
@@ -194,6 +199,7 @@ struct vim2m_dev {struct mutex dev_mutex;struct v4l2_m2m_dev *m2m_dev; +bool multiplanar; };struct vim2m_ctx {
@@ -238,8 +244,10 @@ static struct vim2m_q_data *get_q_data(struct vim2m_ctx *ctx, {switch (type) {case V4L2_BUF_TYPE_VIDEO_OUTPUT : +case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE :return &ctx->q_data[V4L2_M2M_SRC ];case V4L2_BUF_TYPE_VIDEO_CAPTURE : +case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE :return &ctx->q_data[V4L2_M2M_DST ];default :return NULL ;
@@ -250,8 +258,10 @@ static const char *type_name(enum v4l2_buf_type type) {switch (type) {case V4L2_BUF_TYPE_VIDEO_OUTPUT : +case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE :return "Output" ;case V4L2_BUF_TYPE_VIDEO_CAPTURE : +case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE :return "Capture" ;default :return "Invalid" ;
@@ -721,6 +731,7 @@ static int vidioc_g_fmt(struct vim2m_ctx *ctx, struct v4l2_format *f) {struct vb2_queue *vq;struct vim2m_q_data *q_data; +int ret; vq =v4l2_m2m_get_vq (ctx->fh.m2m_ctx, f->type);if (!vq)
@@ -730,12 +741,12 @@ static int vidioc_g_fmt(struct vim2m_ctx *ctx, struct v4l2_format *f)if (!q_data)return -EINVAL ; - f->fmt.pix.width = q_data->width; - f->fmt.pix.height = q_data->height; + ret =v4l2_fill_pixfmt (&f->fmt.pix, q_data->fmt->fourcc, + q_data->width, q_data->height); +if (ret) +return ret; + f->fmt.pix.field =V4L2_FIELD_NONE ; - f->fmt.pix.pixelformat = q_data->fmt->fourcc; - f->fmt.pix.bytesperline = (q_data->width * q_data->fmt->depth) >>3 ; - f->fmt.pix.sizeimage = q_data->sizeimage; f->fmt.pix.colorspace = ctx->colorspace; f->fmt.pix.xfer_func = ctx->xfer_func; f->fmt.pix.ycbcr_enc = ctx->ycbcr_enc;
@@ -744,33 +755,102 @@ static int vidioc_g_fmt(struct vim2m_ctx *ctx, struct v4l2_format *f)return 0 ; } +static int vidioc_g_fmt_mplane (struct vim2m_ctx *ctx,struct v4l2_format *f) +{ +struct vb2_queue *vq; +struct vim2m_q_data *q_data; +int ret; + + vq =v4l2_m2m_get_vq (ctx->fh.m2m_ctx, f->type); +if (!vq) +return -EINVAL ; + + q_data =get_q_data (ctx, f->type); +if (!q_data) +return -EINVAL ; + + ret =v4l2_fill_pixfmt_mp (&f->fmt.pix_mp, q_data->fmt->fourcc, + q_data->width, q_data->height); +if (ret) +return ret; + + f->fmt.pix_mp.field =V4L2_FIELD_NONE ; + f->fmt.pix_mp.colorspace = ctx->colorspace; + f->fmt.pix_mp.xfer_func = ctx->xfer_func; + f->fmt.pix_mp.ycbcr_enc = ctx->ycbcr_enc; + f->fmt.pix_mp.quantization = ctx->quant; + +return 0 ; +} +static int vidioc_g_fmt_vid_out (struct file *file,void *priv,struct v4l2_format *f) { +struct vim2m_dev *dev =video_drvdata (file); + +if (dev->multiplanar) +return -ENOTTY ; +return vidioc_g_fmt (file2ctx (file), f); }static int vidioc_g_fmt_vid_cap (struct file *file,void *priv,struct v4l2_format *f) { +struct vim2m_dev *dev =video_drvdata (file); + +if (dev->multiplanar) +return -ENOTTY ; +return vidioc_g_fmt (file2ctx (file), f); } -static int vidioc_try_fmt (struct v4l2_format *f) +static int vidioc_g_fmt_vid_out_mplane (struct file *file,void *priv, +struct v4l2_format *f) { -int width, height, walign, halign; +struct vim2m_dev *dev =video_drvdata (file); + +if (!dev->multiplanar) +return -ENOTTY ; + +return vidioc_g_fmt_mplane (file2ctx (file), f); +} + +static int vidioc_g_fmt_vid_cap_mplane (struct file *file,void *priv, +struct v4l2_format *f) +{ +struct vim2m_dev *dev =video_drvdata (file); + +if (!dev->multiplanar) +return -ENOTTY ; - width =clamp (f->fmt.pix.width,MIN_W ,MAX_W ); - height =clamp (f->fmt.pix.width,MIN_H ,MAX_H ); +return vidioc_g_fmt_mplane (file2ctx (file), f); +} + +static int vidioc_try_fmt (struct v4l2_format *f,bool is_mplane) +{ +int walign, halign, ret; +int width = (is_mplane) ? f->fmt.pix_mp.width : f->fmt.pix.width; +int height = (is_mplane) ? f->fmt.pix_mp.height : f->fmt.pix.height; +u32 pixfmt = (is_mplane) ? f->fmt.pix_mp.pixelformat : + f->fmt.pix.pixelformat; + + width =clamp (width,MIN_W ,MAX_W ); + height =clamp (height,MIN_H ,MAX_H ); -get_alignment (f->fmt.pix.pixelformat, &walign, &halign); +get_alignment (pixfmt, &walign, &halign); width =ALIGN (width, walign); height =ALIGN (height, halign); f->fmt.pix.field =V4L2_FIELD_NONE ; -return v4l2_fill_pixfmt (&f->fmt.pix, f->fmt.pix.pixelformat, - width, height); +if (is_mplane) { + ret =v4l2_fill_pixfmt_mp (&f->fmt.pix_mp, pixfmt, width, + height); + }else { + ret =v4l2_fill_pixfmt (&f->fmt.pix, pixfmt, width, height); + } +return ret; } static int vidioc_try_fmt_vid_cap(struct file *file, void *priv,
@@ -778,6 +858,10 @@ static int vidioc_try_fmt_vid_cap(struct file *file, void *priv, {struct vim2m_fmt *fmt;struct vim2m_ctx *ctx =file2ctx (file); +struct vim2m_dev *dev =video_drvdata (file); + +if (dev->multiplanar) +return -ENOTTY ; fmt =find_format (f->fmt.pix.pixelformat);if (!fmt) {
@@ -795,7 +879,36 @@ static int vidioc_try_fmt_vid_cap(struct file *file, void *priv, f->fmt.pix.ycbcr_enc = ctx->ycbcr_enc; f->fmt.pix.quantization = ctx->quant; -return vidioc_try_fmt (f); +return vidioc_try_fmt (f, false); +} + +static int vidioc_try_fmt_vid_cap_mplane (struct file *file,void *priv, +struct v4l2_format *f) +{ +struct vim2m_fmt *fmt; +struct vim2m_ctx *ctx =file2ctx (file); +struct vim2m_dev *dev =video_drvdata (file); + +if (!dev->multiplanar) +return -ENOTTY ; + + fmt =find_format (f->fmt.pix_mp.pixelformat); +if (!fmt) { + f->fmt.pix_mp.pixelformat = formats[0 ].fourcc; + fmt =find_format (f->fmt.pix_mp.pixelformat); + } +if (!(fmt->types &MEM2MEM_CAPTURE )) { +v4l2_err (&ctx->dev->v4l2_dev, +"Fourcc format (0x%08x) invalid.\n" , + f->fmt.pix.pixelformat); +return -EINVAL ; + } + f->fmt.pix_mp.colorspace = ctx->colorspace; + f->fmt.pix_mp.xfer_func = ctx->xfer_func; + f->fmt.pix_mp.ycbcr_enc = ctx->ycbcr_enc; + f->fmt.pix_mp.quantization = ctx->quant; + +return vidioc_try_fmt (f, true); } static int vidioc_try_fmt_vid_out(struct file *file, void *priv,
@@ -803,6 +916,10 @@ static int vidioc_try_fmt_vid_out(struct file *file, void *priv, {struct vim2m_fmt *fmt;struct vim2m_ctx *ctx =file2ctx (file); +struct vim2m_dev *dev =video_drvdata (file); + +if (dev->multiplanar) +return -ENOTTY ; fmt =find_format (f->fmt.pix.pixelformat);if (!fmt) {
@@ -818,13 +935,45 @@ static int vidioc_try_fmt_vid_out(struct file *file, void *priv,if (!f->fmt.pix.colorspace) f->fmt.pix.colorspace =V4L2_COLORSPACE_REC709 ; -return vidioc_try_fmt (f); +return vidioc_try_fmt (f, false); +} + +static int vidioc_try_fmt_vid_out_mplane (struct file *file,void *priv, +struct v4l2_format *f) +{ +struct vim2m_fmt *fmt; +struct vim2m_ctx *ctx =file2ctx (file); +struct vim2m_dev *dev =video_drvdata (file); + +if (!dev->multiplanar) +return -ENOTTY ; + + fmt =find_format (f->fmt.pix_mp.pixelformat); +if (!fmt) { + f->fmt.pix_mp.pixelformat = formats[0 ].fourcc; + fmt =find_format (f->fmt.pix_mp.pixelformat); + } +if (!(fmt->types &MEM2MEM_OUTPUT )) { +v4l2_err (&ctx->dev->v4l2_dev, +"Fourcc format (0x%08x) invalid.\n" , + f->fmt.pix_mp.pixelformat); +return -EINVAL ; + } +if (!f->fmt.pix_mp.colorspace) + f->fmt.pix_mp.colorspace =V4L2_COLORSPACE_REC709 ; + +return vidioc_try_fmt (f, true); }static int vidioc_s_fmt (struct vim2m_ctx *ctx,struct v4l2_format *f) {struct vim2m_q_data *q_data;struct vb2_queue *vq; +unsigned int i; +bool is_mplane = ctx->dev->multiplanar; +u32 pixfmt = (is_mplane) ? f->fmt.pix_mp.pixelformat : f->fmt.pix.pixelformat; +u32 width = (is_mplane) ? f->fmt.pix_mp.width : f->fmt.pix.width; +u32 height = (is_mplane) ? f->fmt.pix_mp.height : f->fmt.pix.height; vq =v4l2_m2m_get_vq (ctx->fh.m2m_ctx, f->type);if (!vq)
@@ -839,11 +988,17 @@ static int vidioc_s_fmt(struct vim2m_ctx *ctx, struct v4l2_format *f)return -EBUSY ; } - q_data->fmt =find_format (f->fmt.pix.pixelformat); - q_data->width = f->fmt.pix.width; - q_data->height = f->fmt.pix.height; - q_data->sizeimage = q_data->width * q_data->height - * q_data->fmt->depth >>3 ; + q_data->fmt =find_format (pixfmt); + q_data->width = width; + q_data->height = height; +if (is_mplane) { + q_data->num_mem_planes = f->fmt.pix_mp.num_planes; +for (i =0 ; i < f->fmt.pix_mp.num_planes; i++) + q_data->sizeimage[i] = f->fmt.pix_mp.plane_fmt[i].sizeimage; + }else { + q_data->sizeimage[0 ] = f->fmt.pix.sizeimage; + q_data->num_mem_planes =1 ; + } dprintk(ctx->dev, 1, "Format for type %s: %dx%d (%d bpp), fmt: %c%c%c%c\n",
@@ -861,6 +1016,10 @@ static int vidioc_s_fmt_vid_cap(struct file *file, void *priv, struct v4l2_format *f) {int ret; +struct vim2m_dev *dev =video_drvdata (file); + +if (dev->multiplanar) +return -ENOTTY ; ret =vidioc_try_fmt_vid_cap (file, priv, f);if (ret)
@@ -869,12 +1028,32 @@ static int vidioc_s_fmt_vid_cap(struct file *file, void *priv,return vidioc_s_fmt (file2ctx (file), f); } +static int vidioc_s_fmt_vid_cap_mplane (struct file *file,void *priv, +struct v4l2_format *f) +{ +int ret; +struct vim2m_dev *dev =video_drvdata (file); + +if (!dev->multiplanar) +return -ENOTTY ; + + ret =vidioc_try_fmt_vid_cap_mplane (file, priv, f); +if (ret) +return ret; + +return vidioc_s_fmt (file2ctx (file), f); +} +static int vidioc_s_fmt_vid_out (struct file *file,void *priv,struct v4l2_format *f) {struct vim2m_ctx *ctx =file2ctx (file); +struct vim2m_dev *dev =video_drvdata (file);int ret; +if (dev->multiplanar) +return -ENOTTY ; + ret =vidioc_try_fmt_vid_out (file, priv, f);if (ret)return ret;
@@ -889,6 +1068,30 @@ static int vidioc_s_fmt_vid_out(struct file *file, void *priv,return ret; } +static int vidioc_s_fmt_vid_out_mplane (struct file *file,void *priv, +struct v4l2_format *f) +{ +struct vim2m_ctx *ctx =file2ctx (file); +struct vim2m_dev *dev =video_drvdata (file); +int ret; + +if (!dev->multiplanar) +return -ENOTTY ; + + ret =vidioc_try_fmt_vid_out_mplane (file, priv, f); +if (ret) +return ret; + + ret =vidioc_s_fmt (file2ctx (file), f); +if (!ret) { + ctx->colorspace = f->fmt.pix_mp.colorspace; + ctx->xfer_func = f->fmt.pix_mp.xfer_func; + ctx->ycbcr_enc = f->fmt.pix_mp.ycbcr_enc; + ctx->quant = f->fmt.pix_mp.quantization; + } +return ret; +} +static int vim2m_s_ctrl (struct v4l2_ctrl *ctrl) { struct vim2m_ctx *ctx =
@@ -939,11 +1142,17 @@ static const struct v4l2_ioctl_ops vim2m_ioctl_ops = { .vidioc_g_fmt_vid_cap = vidioc_g_fmt_vid_cap, .vidioc_try_fmt_vid_cap = vidioc_try_fmt_vid_cap, .vidioc_s_fmt_vid_cap = vidioc_s_fmt_vid_cap, + .vidioc_g_fmt_vid_cap_mplane = vidioc_g_fmt_vid_cap_mplane, + .vidioc_try_fmt_vid_cap_mplane = vidioc_try_fmt_vid_cap_mplane, + .vidioc_s_fmt_vid_cap_mplane = vidioc_s_fmt_vid_cap_mplane, .vidioc_enum_fmt_vid_out = vidioc_enum_fmt_vid_out, .vidioc_g_fmt_vid_out = vidioc_g_fmt_vid_out, .vidioc_try_fmt_vid_out = vidioc_try_fmt_vid_out, .vidioc_s_fmt_vid_out = vidioc_s_fmt_vid_out, + .vidioc_g_fmt_vid_out_mplane = vidioc_g_fmt_vid_out_mplane, + .vidioc_try_fmt_vid_out_mplane = vidioc_try_fmt_vid_out_mplane, + .vidioc_s_fmt_vid_out_mplane = vidioc_s_fmt_vid_out_mplane, .vidioc_reqbufs = v4l2_m2m_ioctl_reqbufs, .vidioc_querybuf = v4l2_m2m_ioctl_querybuf,
@@ -972,23 +1181,32 @@ static int vim2m_queue_setup(struct vb2_queue *vq, {struct vim2m_ctx *ctx =vb2_get_drv_priv (vq);struct vim2m_q_data *q_data; -unsigned int size, count = *nbuffers; +unsigned int size, p, count = *nbuffers; q_data =get_q_data (ctx, vq->type);if (!q_data)return -EINVAL ; - size = q_data->width * q_data->height * q_data->fmt->depth >>3 ; + size =0 ; +for (p =0 ; p < q_data->num_mem_planes; p++) + size += q_data->sizeimage[p];while (size * count >MEM2MEM_VID_MEM_LIMIT ) (count)--; *nbuffers = count; -if (*nplanes) -return sizes[0 ] < size ? -EINVAL :0 ; - - *nplanes =1 ; - sizes[0 ] = size; +if (*nplanes) { +if (*nplanes != q_data->num_mem_planes) +return -EINVAL ; +for (p =0 ; p < q_data->num_mem_planes; p++) { +if (sizes[p] < q_data->sizeimage[p]) +return -EINVAL ; + } + }else { + *nplanes = q_data->num_mem_planes; +for (p =0 ; p < q_data->num_mem_planes; p++) + sizes[p] = q_data->sizeimage[p]; + }dprintk (ctx->dev,1 ,"%s: get %d buffer(s) of size %d each.\n" ,type_name (vq->type), count, size);
@@ -1015,21 +1233,24 @@ static int vim2m_buf_prepare(struct vb2_buffer *vb) {struct vim2m_ctx *ctx =vb2_get_drv_priv (vb->vb2_queue);struct vim2m_q_data *q_data; +unsigned int p;dprintk (ctx->dev,2 ,"type: %s\n" ,type_name (vb->vb2_queue->type)); q_data =get_q_data (ctx, vb->vb2_queue->type);if (!q_data)return -EINVAL ; -if (vb2_plane_size (vb,0 ) < q_data->sizeimage) { -dprintk (ctx->dev,1 , -"%s data will not fit into plane (%lu < %lu)\n" , - __func__,vb2_plane_size (vb,0 ), - (long )q_data->sizeimage); -return -EINVAL ; - } -vb2_set_plane_payload (vb,0 , q_data->sizeimage); +for (p =0 ; p < q_data->num_mem_planes; p++) { +if (vb2_plane_size (vb, p) < q_data->sizeimage[p]) { +dprintk (ctx->dev,1 , +"%s data will not fit into plane (%lu < %lu)\n" , + __func__,vb2_plane_size (vb, p), + (long )q_data->sizeimage[p]); +return -EINVAL ; + } +vb2_set_plane_payload (vb, p, q_data->sizeimage[p]); + }return 0 ; }
@@ -1100,7 +1321,8 @@ static int queue_init(void *priv, struct vb2_queue *src_vq,struct vim2m_ctx *ctx = priv;int ret; - src_vq->type =V4L2_BUF_TYPE_VIDEO_OUTPUT ; + src_vq->type = (ctx->dev->multiplanar) ?V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE : +V4L2_BUF_TYPE_VIDEO_OUTPUT ; src_vq->io_modes =VB2_MMAP |VB2_USERPTR |VB2_DMABUF ; src_vq->drv_priv = ctx; src_vq->buf_struct_size =sizeof (struct v4l2_m2m_buffer );
@@ -1114,7 +1336,8 @@ static int queue_init(void *priv, struct vb2_queue *src_vq,if (ret)return ret; - dst_vq->type =V4L2_BUF_TYPE_VIDEO_CAPTURE ; + dst_vq->type = (ctx->dev->multiplanar) ?V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE : +V4L2_BUF_TYPE_VIDEO_CAPTURE ; dst_vq->io_modes =VB2_MMAP |VB2_USERPTR |VB2_DMABUF ; dst_vq->drv_priv = ctx; dst_vq->buf_struct_size =sizeof (struct v4l2_m2m_buffer );
@@ -1188,10 +1411,11 @@ static int vim2m_open(struct file *file) ctx->q_data[V4L2_M2M_SRC ].fmt = &formats[0 ]; ctx->q_data[V4L2_M2M_SRC ].width =640 ; ctx->q_data[V4L2_M2M_SRC ].height =480 ; - ctx->q_data[V4L2_M2M_SRC ].sizeimage = + ctx->q_data[V4L2_M2M_SRC ].sizeimage[0 ] = ctx->q_data[V4L2_M2M_SRC ].width * ctx->q_data[V4L2_M2M_SRC ].height * (ctx->q_data[V4L2_M2M_SRC ].fmt->depth >>3 ); + ctx->q_data[V4L2_M2M_SRC ].num_mem_planes =1 ; ctx->q_data[V4L2_M2M_DST ] = ctx->q_data[V4L2_M2M_SRC ]; ctx->colorspace =V4L2_COLORSPACE_REC709 ;
@@ -1268,7 +1492,7 @@ static const struct video_device vim2m_videodev = { .ioctl_ops = &vim2m_ioctl_ops, .minor = -1, .release = vim2m_device_release, - .device_caps = V4L2_CAP_VIDEO_M2M | V4L2_CAP_STREAMING, + .device_caps = V4L2_CAP_STREAMING, };static const struct v4l2_m2m_ops m2m_ops = {
@@ -1299,10 +1523,14 @@ static int vim2m_probe(struct platform_device *pdev)atomic_set (&dev->num_inst,0 );mutex_init (&dev->dev_mutex); + dev->multiplanar = (multiplanar ==2 ); + dev->vfd = vim2m_videodev; vfd = &dev->vfd; vfd->lock = &dev->dev_mutex; vfd->v4l2_dev = &dev->v4l2_dev; + vfd->device_caps |= (dev->multiplanar) ?V4L2_CAP_VIDEO_M2M_MPLANE : +V4L2_CAP_VIDEO_M2M ;video_set_drvdata (vfd, dev); v4l2_info(&dev->v4l2_dev,
Signature · 1 line
2.25.1
Hi Matthew,
Quoted 5 lines · Matthew Majewski
Hi everyone,
This patch series adds multiplane API support for the virtual M2M driver, along with some minor driver refactoring/improvements.
Quoted 7 lines
I followed the lead of the vivid driver and made multiplanar support selectable with a module parameter, and the default is to use the single planar api.
Although there are not yet any pixelformats in the driver that make use of multiple memory planes, adding support for these should be easier now with the API level changes taken care of.
Are you planning follow-up patches adding support for at least one multiplanar format? That would be really nice.
Regards,
Hans
Quoted 61 lines
v4l2-compliance reports the following with multiplane support disabled:
Total for vim2m device
/dev/video0: 48, Succeeded: 48, Failed: 0, Warnings: 0and the same with multiplane support enabled:
Total for vim2m device
/dev/video0: 48, Succeeded: 48, Failed: 0, Warnings: 0Patches need to be applied in increasing numerical order (Patch [3/3] depends on [1/3] and [2/3]).
Since the multi-plane changes had to touch a lot of the driver, I did a basic regression test with the following script which generates a test input image with vivid and an output image from vim2m for each supported format. I confirmed all outputs visually and verified they were identical to the outputs before the change. Testing was done on an
x86_64qemu image.#!/bin/sh
# tested with HDMI vivid emulation # modprobe vivid num_inputs=1 input_types=3vim2m=/dev/video0
vivid=/dev/video1width=640
height=480
out_width=320
out_height=240capture_formats=$(v4l2-ctl -d $vim2m --list-formats | awk '/\]:/ {print $2}' | sed "s/'//g") output_formats=$(v4l2-ctl -d $vim2m --list-formats-out | awk '/\]:/ {print $2}' | sed "s/'//g")
# Turn off text mode so that images will be identical v4l2-ctl -d $vivid -c osd_text_mode=2
for ofmt in ${output_formats}; do
# generate input image inname="${width}x${height}.${ofmt}" v4l2-ctl -d $vivid -v pixelformat=$ofmt,width=$width,height=$height,field=none \ --stream-mmap --stream-count=1 --stream-to=$inname for cfmt in ${capture_formats}; do outname="${out_width}x${out_height}-out.${cfmt}" v4l2-ctl -d $vim2m -x pixelformat=$ofmt,width=$width,height=$height \ -v pixelformat=$cfmt,width=$out_width,height=$out_height \ --stream-from=$inname --stream-to=$outname --stream-mmap --stream-out-mmap \ --stream-count=1 done doneMatthew Majewski (3):
media: v4l2-common: Add RGBR format info
media: vim2m: Simplifytry_fmt
media: vim2m: Add parametized support for multiplanar APIdrivers/media/test-drivers/vim2m.c | 327 +++++++++++++++++++++----- drivers/media/v4l2-core/v4l2-common.c | 1 + 2 files changed, 274 insertions(+), 54 deletions(-)
Hello Hans,
Quoted 17 lines · Hans Verkuil
Hi Matthew,
Quoted 5 lines · Matthew Majewski
Hi everyone,
This patch series adds multiplane API support for the virtual M2M driver, along with some minor driver refactoring/improvements.
Quoted 8 lines
I followed the lead of the vivid driver and made multiplanar support
selectable with a module parameter, and the default is to use the single planar api.Although there are not yet any pixelformats in the driver that make use of multiple memory planes, adding support for these should be easier now with the API level changes taken care of.
Quoted 7 lines
Are you planning follow-up patches adding support for at least one multiplanar format? That would be really nice.
Regards,
Hans
Yes, I currently have a WIP patch series to add support for all of the formats that vivid currently supports to vim2m. It involves a huge refactor though, so I split the groundwork for supporting the multi- planar formats into a separate patch series first.
Best,
Matthew
Quoted 76 lines
Quoted 75 lines
v4l2-compliance reports the following with multiplane support disabled:
Total for vim2m device
/dev/video0: 48, Succeeded: 48, Failed: 0,
Warnings: 0and the same with multiplane support enabled:
Total for vim2m device
/dev/video0: 48, Succeeded: 48, Failed: 0,
Warnings: 0Patches need to be applied in increasing numerical order (Patch [3/3]
depends on [1/3] and [2/3]).Since the multi-plane changes had to touch a lot of the driver, I did
a basic regression test with the following script which generates a test input image with vivid and an output image from vim2m for each supported format. I confirmed all outputs visually and verified they
were identical to the outputs before the change. Testing was done on
anx86_64qemu image.#!/bin/sh
# tested with HDMI vivid emulation # modprobe vivid num_inputs=1 input_types=3vim2m=/dev/video0
vivid=/dev/video1width=640
height=480
out_width=320
out_height=240capture_formats=$(v4l2-ctl -d $vim2m --list-formats | awk '/\]:/ {print $2}' | sed "s/'//g")
output_formats=$(v4l2-ctl -d $vim2m --list-formats-out | awk '/\]:/ {print $2}' | sed "s/'//g")# Turn off text mode so that images will be identical v4l2-ctl -d $vivid -c osd_text_mode=2
for ofmt in ${output_formats}; do
# generate input image inname="${width}x${height}.${ofmt}" v4l2-ctl -d $vivid -v pixelformat=$ofmt,width=$width,height=$height,field=none \ --stream-mmap --stream-count=1 --stream-to=$inname for cfmt in ${capture_formats}; do outname="${out_width}x${out_height}-out.${cfmt}" v4l2-ctl -d $vim2m -x pixelformat=$ofmt,width=$width,height=$height \ -v pixelformat=$cfmt,width=$out_width,height=$out_height \ --stream-from=$inname --stream-to=$outname -- stream-mmap --stream-out-mmap \ --stream-count=1 done doneMatthew Majewski (3):
media: v4l2-common: Add RGBR format info
media: vim2m: Simplifytry_fmt
media: vim2m: Add parametized support for multiplanar APIdrivers/media/test-drivers/vim2m.c | 327 +++++++++++++++++++++-
----
drivers/media/v4l2-core/v4l2-common.c | 1 +2 files changed, 274 insertions(+), 54 deletions(-)
Quoted 25 lines · Matthew Majewski
Hello Hans,
Quoted 22 lines · Hans Verkuil
Hi Matthew,
Quoted 5 lines · Matthew Majewski
Hi everyone,
This patch series adds multiplane API support for the virtual M2M driver, along with some minor driver refactoring/improvements.
Quoted 8 lines
I followed the lead of the vivid driver and made multiplanar support
selectable with a module parameter, and the default is to use the single planar api.Although there are not yet any pixelformats in the driver that make use of multiple memory planes, adding support for these should be easier now with the API level changes taken care of.
Are you planning follow-up patches adding support for at least one multiplanar format? That would be really nice.
Regards,
Quoted 7 lines
Quoted 2 lines
Hans
Yes, I currently have a WIP patch series to add support for all of the formats that vivid currently supports to vim2m. It involves a huge refactor though, so I split the groundwork for supporting the multi- planar formats into a separate patch series first.
Ah, great news! Looking forward to that.
FYI: I've added vim2m multiplanar tests to contrib/test/test-media in v4l-utils. That script is part of our regression testing, so this helps testing multiplanar support.
Regards,
Hans
Quoted 82 lines
Best,
Matthew
Quoted 76 lines
Quoted 75 lines
v4l2-compliance reports the following with multiplane support disabled:
Total for vim2m device
/dev/video0: 48, Succeeded: 48, Failed: 0,
Warnings: 0and the same with multiplane support enabled:
Total for vim2m device
/dev/video0: 48, Succeeded: 48, Failed: 0,
Warnings: 0Patches need to be applied in increasing numerical order (Patch [3/3]
depends on [1/3] and [2/3]).Since the multi-plane changes had to touch a lot of the driver, I did
a basic regression test with the following script which generates a test input image with vivid and an output image from vim2m for each supported format. I confirmed all outputs visually and verified they
were identical to the outputs before the change. Testing was done on
anx86_64qemu image.#!/bin/sh
# tested with HDMI vivid emulation # modprobe vivid num_inputs=1 input_types=3vim2m=/dev/video0
vivid=/dev/video1width=640
height=480
out_width=320
out_height=240capture_formats=$(v4l2-ctl -d $vim2m --list-formats | awk '/\]:/ {print $2}' | sed "s/'//g")
output_formats=$(v4l2-ctl -d $vim2m --list-formats-out | awk '/\]:/ {print $2}' | sed "s/'//g")# Turn off text mode so that images will be identical v4l2-ctl -d $vivid -c osd_text_mode=2
for ofmt in ${output_formats}; do
# generate input image inname="${width}x${height}.${ofmt}" v4l2-ctl -d $vivid -v pixelformat=$ofmt,width=$width,height=$height,field=none \ --stream-mmap --stream-count=1 --stream-to=$inname for cfmt in ${capture_formats}; do outname="${out_width}x${out_height}-out.${cfmt}" v4l2-ctl -d $vim2m -x pixelformat=$ofmt,width=$width,height=$height \ -v pixelformat=$cfmt,width=$out_width,height=$out_height \ --stream-from=$inname --stream-to=$outname -- stream-mmap --stream-out-mmap \ --stream-count=1 done doneMatthew Majewski (3):
media: v4l2-common: Add RGBR format info
media: vim2m: Simplifytry_fmt
media: vim2m: Add parametized support for multiplanar APIdrivers/media/test-drivers/vim2m.c | 327 +++++++++++++++++++++-
----
drivers/media/v4l2-core/v4l2-common.c | 1 +2 files changed, 274 insertions(+), 54 deletions(-)