Skip to contentSkip to conversation

media: imx: staging cleanup and FIM export to v4l2-core

PATCH Superseded Jiaqi Shi 2 participants 5 messages Nov 21

Subsystems (4)

v2 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Jiaqi Shi

author
#0 lore raw

v2 cover media: imx: staging cleanup and FIM export to v4l2-core

Changes in v2:
- Drop the ICAP_CHANNEL s_ctrl patch. The ICAP controls are clustered and s_ctrl is always called on ICAP_EDGE, so the added case was dead code.
- Initialize the default field and colorimetry with imx_media_init_mbus_fmt(), and resolve try-format colorimetry using the selected format's media bus code.
- Clamp dimensions to non-zero values and reject formats whose image size exceeds the 32-bit sizeimage field.
- Validate the selected pixel format against the source media bus format, including an exact match on non-IPU passthrough links.
- Update the v4l2-ctl examples to specify width and height explicitly.
- During FIM control updates, read new values only from the active cluster and committed values from other clusters.

This series continues the i.MX staging media driver cleanup by removing legacy video node handling and exporting the Frame Interval Monitor to v4l2-core.

Patch 1 removes control inheritance and source-dependent format handling from capture video nodes in favor of the subdev-based userspace API.

Patch 2 moves the generic FIM implementation to v4l2-core while keeping the i.MX-specific input capture controls in the i.MX adapter. The error calculation uses signed arithmetic to avoid underflow when the measured interval is smaller than the nominal interval.

Validation:
- allmodconfig build and sparse
- local FIM KUnit tests, not included in this series: 13/13 passed
- local white-box i.MX capture format tests, not included: 15/15 passed
- QEMU vivid: v4l2-compliance 125/125, 30-frame streaming passed
- QEMU vim2m: v4l2-compliance 49/49, 10-frame streaming passed
- i.MX register-level and hardware DMA paths remain untested

Jiaqi Shi (2):
media: imx: remove legacy video node control and format handling
media: v4l2-core: add frame interval monitor helper

 Documentation/admin-guide/media/imx.rst       |  20 +-
 drivers/media/v4l2-core/Kconfig               |   4 +
 drivers/media/v4l2-core/Makefile              |   1 +
 drivers/media/v4l2-core/v4l2-fim.c            | 331 ++++++++++++++++
 drivers/staging/media/imx/Kconfig             |   1 +
 drivers/staging/media/imx/TODO                |  13 -
 drivers/staging/media/imx/imx-ic-prpencvf.c   |   7 +-
 drivers/staging/media/imx/imx-media-capture.c | 230 +++---------
 drivers/staging/media/imx/imx-media-csi.c     |   7 +-
 .../staging/media/imx/imx-media-dev-common.c  | 113 +-----
 drivers/staging/media/imx/imx-media-fim.c     | 355 +++---------------
 drivers/staging/media/imx/imx-media.h         |   9 +-
 include/media/v4l2-fim.h                      |  40 ++
 13 files changed, 490 insertions(+), 641 deletions(-)
 create mode 100644 drivers/media/v4l2-core/v4l2-fim.c
 delete mode 100644 drivers/staging/media/imx/TODO
 create mode 100644 include/media/v4l2-fim.h
Signature · 1 line
2.53.0
author
#1 lore raw

PATCH v2 1/2 media: imx: remove legacy video node control and format handling

Remove control inheritance from pipeline subdevices to video nodes. Userspace should access controls through subdev nodes instead.

Use source-independent format ioctls for all capture nodes and initialize them to the existing UYVY 640x480 default instead of the source subdev active format. Keep stream-time format validation.

Update the usage documentation and remove the completed TODO entries.

---
 Documentation/admin-guide/media/imx.rst       |  20 +-
 drivers/staging/media/imx/TODO                |  10 -
 drivers/staging/media/imx/imx-ic-prpencvf.c   |   7 +-
 drivers/staging/media/imx/imx-media-capture.c | 230 ++++--------------
 drivers/staging/media/imx/imx-media-csi.c     |   7 +-
 .../staging/media/imx/imx-media-dev-common.c  | 113 +--------
 drivers/staging/media/imx/imx-media.h         |   9 +-
 7 files changed, 61 insertions(+), 335 deletions(-)
Documentation/admin-guide/media/imx.rst +8 -12
index bb68100d8acb..812c6ac09ab8 100644
@@ -394,13 +394,9 @@ scaling up to 1024x1024, CSC, flip, and rotation:
 Usage Notes
 -----------
 
-To aid in configuration and for backward compatibility with V4L2
-applications that access controls only from video device nodes, the
-capture device interfaces inherit controls from the active entities
-in the current pipeline, so controls can be accessed either directly
-from the subdev or from the active capture device interface. For
-example, the FIM controls are available either from the ipuX_csiY
-subdevs or from the active capture device.
+Controls for pipeline entities are accessed through their subdev device
+nodes. For example, the FIM controls are available from the ipuX_csiY
+subdevs.
 
 The following are specific usage notes for the Sabre* reference
 boards:
@@ -483,7 +479,7 @@ NTSC, or 'alternate'):
    media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/720x480]"
    media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/720x480]"
    # Configure "ipu1_csi0 capture" interface (assumed at /dev/video4)
-   v4l2-ctl -d4 --set-fmt-video=field=interlaced_bt
+   v4l2-ctl -d4 --set-fmt-video=width=720,height=480,field=interlaced_bt
 
 Streaming can then begin on /dev/video4. The v4l2-ctl tool can also be
 used to select any supported YUV pixelformat on /dev/video4.
@@ -510,7 +506,7 @@ alternating fields (field type 'seq-tb' for PAL, or 'alternate').
    media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x576 field:none]"
    media-ctl -V "'ipu1_ic_prpvf':1 [fmt:AYUV32/720x576 field:none]"
    # Configure "ipu1_ic_prpvf capture" interface (assumed at /dev/video2)
-   v4l2-ctl -d2 --set-fmt-video=field=none
+   v4l2-ctl -d2 --set-fmt-video=width=720,height=576,field=none
 
 Streaming can then begin on /dev/video2. The v4l2-ctl tool can also be
 used to select any supported YUV pixelformat on /dev/video2.
@@ -541,7 +537,7 @@ NTSC, or 'alternate'):
    media-ctl -V "'ipu1_csi0_mux':5 [fmt:UYVY2X8/720x480]"
    media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/720x480]"
    # Configure "ipu1_csi0 capture" interface (assumed at /dev/video0)
-   v4l2-ctl -d0 --set-fmt-video=field=interlaced_bt
+   v4l2-ctl -d0 --set-fmt-video=width=720,height=480,field=interlaced_bt
 
 Streaming can then begin on /dev/video0. The v4l2-ctl tool can also be
 used to select any supported YUV pixelformat on /dev/video0.
@@ -568,7 +564,7 @@ alternating fields (field type 'seq-tb' for PAL, or 'alternate').
    media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x576 field:none]"
    media-ctl -V "'ipu1_ic_prpvf':1 [fmt:AYUV32/720x576 field:none]"
    # Configure "ipu1_ic_prpvf capture" interface (assumed at /dev/video2)
-   v4l2-ctl -d2 --set-fmt-video=field=none
+   v4l2-ctl -d2 --set-fmt-video=width=720,height=576,field=none
 
 Streaming can then begin on /dev/video2. The v4l2-ctl tool can also be
 used to select any supported YUV pixelformat on /dev/video2.
@@ -652,7 +648,7 @@ shows colorspace conversion and scaling at IC output.
    media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/640x480]"
    media-ctl -V "'ipu1_ic_prpenc':1 [fmt:ARGB8888_1X32/800x600]"
    # Set a format at the capture interface
-   v4l2-ctl -d /dev/video1 --set-fmt-video=pixelformat=RGB3
+   v4l2-ctl -d /dev/video1 --set-fmt-video=width=800,height=600,pixelformat=RGB3
 
 Streaming can then begin on "ipu1_ic_prpenc capture" node.
 
drivers/staging/media/imx/TODO +0 -10
index 11c9e10d34ae..43cd030d4bbd 100644
@@ -1,13 +1,3 @@
 
 - The Frame Interval Monitor could be exported to v4l2-core for
   general use.
-
-- This media driver supports inheriting V4L2 controls to the
-  video capture devices, from the subdevices in the capture device's
-  pipeline. The controls for each capture device are updated in the
-  link_notify callback when the pipeline is modified. This feature should be
-  removed, userspace should use the subdev-based userspace API instead.
-
-- Similarly to the legacy control handling, legacy format handling where
-  formats on the video nodes are influenced by the active format of the
-  connected subdev should be removed.
drivers/staging/media/imx/imx-ic-prpencvf.c +2 -5
index 3430ddeac356..372d25c791d3 100644
@@ -1280,13 +1280,13 @@ static int prp_registered(struct v4l2_subdev *sd)
 
 	priv->vdev = imx_media_capture_device_init(ic_priv->ipu_dev,
 						   &ic_priv->sd,
-						   PRPENCVF_SRC_PAD, true);
+						   PRPENCVF_SRC_PAD, true, true);
 	if (IS_ERR(priv->vdev))
 		return PTR_ERR(priv->vdev);
 
 	ret = imx_media_capture_device_register(priv->vdev, 0);
 	if (ret)
-		goto remove_vdev;
+		return ret;
 
 	ret = prp_init_controls(priv);
 	if (ret)
@@ -1296,8 +1296,6 @@ static int prp_registered(struct v4l2_subdev *sd)
 
 unreg_vdev:
 	imx_media_capture_device_unregister(priv->vdev);
-remove_vdev:
-	imx_media_capture_device_remove(priv->vdev);
 	return ret;
 }
 
@@ -1306,7 +1304,6 @@ static void prp_unregistered(struct v4l2_subdev *sd)
 	struct prp_priv *priv = sd_to_priv(sd);
 
 	imx_media_capture_device_unregister(priv->vdev);
-	imx_media_capture_device_remove(priv->vdev);
 
 	v4l2_ctrl_handler_free(&priv->ctrl_hdlr);
 }
drivers/staging/media/imx/imx-media-capture.c +45 -185

349 lines not on this page · raw message

drivers/staging/media/imx/imx-media-csi.c +2 -5
index 43c637b333f2..7e655260f241 100644
@@ -1815,7 +1815,7 @@ static int csi_registered(struct v4l2_subdev *sd)
 	}
 
 	priv->vdev = imx_media_capture_device_init(priv->sd.dev, &priv->sd,
-						   CSI_SRC_PAD_IDMAC, true);
+						   CSI_SRC_PAD_IDMAC, true, false);
 	if (IS_ERR(priv->vdev)) {
 		ret = PTR_ERR(priv->vdev);
 		goto free_fim;
@@ -1823,12 +1823,10 @@ static int csi_registered(struct v4l2_subdev *sd)
 
 	ret = imx_media_capture_device_register(priv->vdev, 0);
 	if (ret)
-		goto remove_vdev;
+		goto free_fim;
 
 	return 0;
 
-remove_vdev:
-	imx_media_capture_device_remove(priv->vdev);
 free_fim:
 	if (priv->fim)
 		imx_media_fim_free(priv->fim);
@@ -1842,7 +1840,6 @@ static void csi_unregistered(struct v4l2_subdev *sd)
 	struct csi_priv *priv = v4l2_get_subdevdata(sd);
 
 	imx_media_capture_device_unregister(priv->vdev);
-	imx_media_capture_device_remove(priv->vdev);
 
 	if (priv->fim)
 		imx_media_fim_free(priv->fim);
drivers/staging/media/imx/imx-media-dev-common.c +1 -112

131 lines not on this page · raw message

drivers/staging/media/imx/imx-media.h +3 -6
index 537558a7bece..2f32a787a600 100644
@@ -125,10 +125,8 @@ static inline struct imx_media_buffer *to_imx_media_vb(struct vb2_buffer *vb)
 }
 
 /*
- * to support control inheritance to video devices, this
- * retrieves a pad's list_head of video devices that can
- * be reached from the pad. Note that only the lists in
- * source pads get populated, sink pads have empty lists.
+ * Retrieve a pad's list of video devices reachable from the pad for
+ * subdev event forwarding. Only source pad lists are populated.
  */
 static inline struct list_head *
 to_pad_vdev_list(struct v4l2_subdev *sd, int pad_index)
@@ -270,8 +268,7 @@ int imx_media_ic_unregister(struct v4l2_subdev *sd);
 /* imx-media-capture.c */
 struct imx_media_video_dev *
 imx_media_capture_device_init(struct device *dev, struct v4l2_subdev *src_sd,
-			      int pad, bool legacy_api);
-void imx_media_capture_device_remove(struct imx_media_video_dev *vdev);
+			      int pad, bool legacy_api, bool ic_route);
 int imx_media_capture_device_register(struct imx_media_video_dev *vdev,
 				      u32 link_flags);
 void imx_media_capture_device_unregister(struct imx_media_video_dev *vdev);
Signature · 1 line
2.53.0
#2 lore raw

Re: [PATCH 1/2] media: imx: remove legacy video node control and format handling

Thank you for your contribution! Sashiko AI review found 6 potential issues to consider: - [High] media: imx: broken pixel format validation in capture_find_format() - [High] media: imx: 32-bit…

Thank you for your contribution! Sashiko AI review found 6 potential issues to consider:
- [High] media: imx: broken pixel format validation in capture_find_format()
- [High] media: imx: 32-bit planar sizeimage overflow in __capture_try_fmt()
- [Medium] media: imx: __capture_try_fmt() fails with -EINVAL on large dimensions
- [Medium] media: imx: unsanitized field handling in __capture_try_fmt()
- [Medium] media: imx: wrong default RGB quantization in __capture_try_fmt()
- [Medium] media: imx: callers of imx_media_capture_device_init() set legacy_api
--

commit 110cb63254b68ca17643e4ce4ba642ee6210e734
Author: Jiaqi Shi <[email protected]>

media: imx: remove legacy video node control and format handling

This commit removes control inheritance from pipeline subdevices to capture video nodes and switches capture format handling to source-independent format ioctls initialized to a default UYVY 640x480 format.

drivers/staging/media/imx/imx-ic-prpencvf.c static int prp_registered(struct v4l2_subdev *sd) · line 1283

Quoted 10 lines
drivers/staging/media/imx/imx-ic-prpencvf.c +1 -1
index 3430ddeac3569..372d25c791d35 100644
@@ -1280,13 +1280,13 @@ static int prp_registered(struct v4l2_subdev *sd)
 
 	priv->vdev = imx_media_capture_device_init(ic_priv->ipu_dev,
 						   &ic_priv->sd,
-						   PRPENCVF_SRC_PAD, true);
+						   PRPENCVF_SRC_PAD, true, true);

[Severity: Medium]
Should prp_registered() and csi_registered() pass legacy_api = false (or remove the legacy_api parameter from imx_media_capture_device_init()) now that legacy format and control handling is being removed?

Because both callers still pass legacy_api = true, imx_media_capture_device_init() never sets V4L2_CAP_IO_MC on any capture video node, which causes v4l_enum_fmt() in v4l2-core to zero f->mbus_code before calling capture_enum_fmt_vid_cap(). It also leaves capture_legacy_ioctl_ops active (while capture_ioctl_ops is unused), so VIDIOC_ENUM_FRAMESIZES still calls capture_legacy_enum_framesizes(), which queries the source subdev's active format and fails with -EINVAL on IC capture nodes for any pixel format whose cc->codes[0] is not an IPU-internal media bus code.

[ ... ]

Quoted 4 lines
drivers/staging/media/imx/imx-media-capture.c +0 -0
index 9216a221b72f4..adc524038ed81 100644

[ ... ]

drivers/staging/media/imx/imx-media-capture.c static const struct imx_media_pixfmt *capture_find_format(u32 code, u32 fourcc) · line 78

Quoted 11 lines
drivers/staging/media/imx/imx-media-capture.c +5 -1
@@ -74,7 +70,11 @@ static const struct imx_media_pixfmt *capture_find_format(u32 code, u32 fourcc)
 		return cc;
 	}
 
-	return imx_media_find_mbus_format(code, PIXFMT_SEL_ANY);
+	cc = imx_media_find_mbus_format(code, PIXFMT_SEL_ANY);
+	if (cc && fourcc && cc->fourcc != fourcc)
+		return NULL;
+
+	return cc;
 }

[Severity: High]
Does capture_find_format() misvalidate both passthrough and IPU-internal formats when called from capture_validate_fmt() at stream start?

In the non-IPU branch above, imx_media_find_mbus_format(code, PIXFMT_SEL_ANY) returns only the first entry in pixel_formats[] matching code:

drivers/staging/media/imx/imx-media-utils.c:pixel_formats[] {
    ...
	}, {
		.fourcc = V4L2_PIX_FMT_GREY,
		.codes = IMX_BUS_FMTS(
			MEDIA_BUS_FMT_Y8_1X8,
			MEDIA_BUS_FMT_Y10_1X10,
			MEDIA_BUS_FMT_Y12_1X12
		),
		.cs     = IPUV3_COLORSPACE_RGB,
		.bpp    = 8,
		.bayer  = true,
	}, {
		.fourcc = V4L2_PIX_FMT_Y10,
		.codes = IMX_BUS_FMTS(MEDIA_BUS_FMT_Y10_1X10),
		.cs     = IPUV3_COLORSPACE_RGB,
		.bpp    = 16,
		.bayer  = true,
    ...
}

Because V4L2_PIX_FMT_GREY precedes V4L2_PIX_FMT_Y10 and V4L2_PIX_FMT_Y12 and lists MEDIA_BUS_FMT_Y10_1X10 and MEDIA_BUS_FMT_Y12_1X12, imx_media_find_mbus_format() returns V4L2_PIX_FMT_GREY for those codes. Checking cc->fourcc != fourcc then rejects valid V4L2_PIX_FMT_Y10 and V4L2_PIX_FMT_Y12 capture with -EPIPE while accepting 8-bpp V4L2_PIX_FMT_GREY buffers for 16-bit passthrough DMA.

In the IPU branch of capture_find_format(), when fourcc is not found in fmt_sel, it falls back to the first format in fmt_sel instead of returning NULL:

drivers/staging/media/imx/imx-media-capture.c:capture_find_format() {
	cc = imx_media_find_ipu_format(code, PIXFMT_SEL_YUV_RGB);
	if (cc) {
		enum imx_pixfmt_sel fmt_sel = cc->cs == IPUV3_COLORSPACE_YUV
					    ? PIXFMT_SEL_YUV : PIXFMT_SEL_RGB;

		cc = imx_media_find_pixel_format(fourcc, fmt_sel);
		if (!cc) {
			imx_media_enum_pixel_formats(&fourcc, 0, fmt_sel, 0);
			cc = imx_media_find_pixel_format(fourcc, fmt_sel);
		}

		return cc;
	}
    ...
}

Because all Bayer and greyscale entries in pixel_formats[] have .cs = IPUV3_COLORSPACE_RGB, configuring a Bayer or greyscale pixel format on a video node fed by MEDIA_BUS_FMT_ARGB8888_1X32 falls back to a PIXFMT_SEL_RGB descriptor with cc->cs == IPUV3_COLORSPACE_RGB, passes priv->vdev.cc->cs != cc->cs in capture_validate_fmt(), and reaches ipu_cpmem_set_image(), triggering WARN_ON(1) for V4L2_PIX_FMT_Y10/Y12 or leaving the CPMEM channel at 32 bpp after an unchecked ipu_cpmem_set_fmt() -EINVAL error and overflowing 8-bpp or 16-bpp DMA buffers.

Quoted 15 lines
drivers/staging/media/imx/imx-media-capture.c +3 -1
@@ -136,10 +136,12 @@ static int capture_g_fmt_vid_cap(struct file *file, void *fh,
 }
 
 static const struct imx_media_pixfmt *
-__capture_try_fmt(struct v4l2_pix_format *pixfmt, struct v4l2_rect *compose)
+__capture_try_fmt(struct capture_priv *priv, struct v4l2_pix_format *pixfmt,
+		  struct v4l2_rect *compose)
 {
 	struct v4l2_mbus_framefmt fmt_src;
 	const struct imx_media_pixfmt *cc;
+	u64 sizeimage;
 
 	/*
 	 * Find the pixel format, default to the first supported format if not

drivers/staging/media/imx/imx-media-capture.c __capture_try_fmt(struct v4l2_pix_format *pixfmt, struct v4l2_rect *compose) · line 170

Quoted 2 lines
drivers/staging/media/imx/imx-media-capture.c +0 -0
@@ -167,9 +169,21 @@ __capture_try_fmt(struct v4l2_pix_format *pixfmt, struct v4l2_rect *compose)
 		}
 	}

[Severity: Medium]
Now that all capture nodes use __capture_try_fmt(), does this leave pixfmt->field unsanitized and unvalidated against the source subdev?

In __capture_try_fmt(), the V4L2_FIELD_SEQ_TB and V4L2_FIELD_SEQ_BT cases sit inside if (V4L2_FIELD_IS_INTERLACED(pixfmt->field)), which is false for sequential fields, making that switch unreachable. No other check sanitizes pixfmt->field, so V4L2_FIELD_ANY or invalid field values are returned as-is from VIDIOC_TRY_FMT and VIDIOC_S_FMT, and capture_validate_fmt() does not verify priv->vdev.fmt.field against fmt_src.format.field before configuring IDMAC interweaving.

drivers/staging/media/imx/imx-media-capture.c __capture_try_fmt(struct v4l2_pix_format *pixfmt, struct v4l2_rect *compose) · line 178

Quoted 8 lines
drivers/staging/media/imx/imx-media-capture.c +7 -1
-	v4l2_fill_mbus_format(&fmt_src, pixfmt, 0);
+	/* Clamp dimensions to prevent zero-sized buffers. */
+	pixfmt->width = clamp(pixfmt->width, 1, 65535);
+	pixfmt->height = clamp(pixfmt->height, 1, 65535);
+
+	v4l2_fill_mbus_format(&fmt_src, pixfmt,
+			      cc->codes ? cc->codes[0] : 0);
+	imx_media_try_colorimetry(&fmt_src, priv->ic_route);

[Severity: Medium]
Does passing cc->codes ? cc->codes[0] : 0 give in-memory-only RGB formats the wrong default quantization?

V4L2_PIX_FMT_BGR24, V4L2_PIX_FMT_XBGR32, V4L2_PIX_FMT_BGRX32 and V4L2_PIX_FMT_RGBX32 have .codes = NULL in pixel_formats[]. When fmt_src.code is 0, imx_media_try_colorimetry() cannot find a format descriptor, leaves is_rgb = false, and maps V4L2_QUANTIZATION_DEFAULT to V4L2_QUANTIZATION_LIM_RANGE instead of V4L2_QUANTIZATION_FULL_RANGE.

drivers/staging/media/imx/imx-media-capture.c __capture_try_fmt(struct v4l2_pix_format *pixfmt, struct v4l2_rect *compose) · line 185

Quoted 7 lines
drivers/staging/media/imx/imx-media-capture.c +5 -0
 	imx_media_mbus_fmt_to_pix_fmt(pixfmt, &fmt_src, cc);
 
+	sizeimage = (u64)pixfmt->bytesperline * pixfmt->height;
+	if (cc->planar)
+		sizeimage = sizeimage * cc->bpp / 8;
+	if (sizeimage > U32_MAX)
+		return NULL;

[Severity: High]
Can pixfmt->sizeimage wrap to 0 for planar formats even when the 64-bit sizeimage check here succeeds?

imx_media_mbus_fmt_to_pix_fmt() computes pix->sizeimage in 32-bit arithmetic before shifting right by 3:

drivers/staging/media/imx/imx-media-utils.c:imx_media_mbus_fmt_to_pix_fmt() {
    ...
	pix->sizeimage = cc->planar ? ((stride * pix->height * cc->bpp) >> 3) :
			 stride * pix->height;
}

For a 16-bpp planar format such as V4L2_PIX_FMT_NV16 or V4L2_PIX_FMT_YUV422P at 16384x16384, stride * pix->height * cc->bpp is 2^32, so pixfmt->sizeimage wraps to 0 in u32 while the 64-bit sizeimage in __capture_try_fmt() is 512 MiB (<= U32_MAX) and is never written back to pixfmt->sizeimage. A subsequent VIDIOC_REQBUFS or VIDIOC_CREATE_BUFS then divides by zero in capture_queue_setup() at VID_MEM_LIMIT / pix->sizeimage.

drivers/staging/media/imx/imx-media-capture.c __capture_try_fmt(struct v4l2_pix_format *pixfmt, struct v4l2_rect *compose) · line 204

Quoted 12 lines
drivers/staging/media/imx/imx-media-capture.c +5 -1
@@ -181,7 +195,11 @@ __capture_try_fmt(struct v4l2_pix_format *pixfmt, struct v4l2_rect *compose)
 static int capture_try_fmt_vid_cap(struct file *file, void *fh,
 				   struct v4l2_format *f)
 {
-	__capture_try_fmt(&f->fmt.pix, NULL);
+	struct capture_priv *priv = video_drvdata(file);
+
+	if (!__capture_try_fmt(priv, &f->fmt.pix, NULL))
+		return -EINVAL;
+
 	return 0;
 }

[Severity: Medium]
Should VIDIOC_TRY_FMT and VIDIOC_S_FMT clamp width and height to dimensions that fit within sizeimage rather than returning -EINVAL when sizeimage exceeds U32_MAX?

capture_enum_framesizes() advertises dimensions up to 65535x65535, and __capture_try_fmt() clamps width and height to 65535 (rounded up to 65536 in imx_media_mbus_fmt_to_pix_fmt()). For any format with bpp >= 12 (including the default UYVY), requesting 65535x65535 or probing maximum dimensions with ~0U x ~0U yields sizeimage > U32_MAX and causes VIDIOC_TRY_FMT and VIDIOC_S_FMT to fail with -EINVAL instead of adjusting the dimensions into a valid range.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
author
#3 lore raw

PATCH v2 2/2 media: v4l2-core: add frame interval monitor helper

In reply to Jiaqi Shi

Move the generic frame interval monitoring implementation from i.MX to v4l2-core. The helper supports driver-selected control and event IDs and accepts EOF or external timestamps.

Keep the i.MX-specific input capture controls in the i.MX adapter, sharing the original control handler and preserving the existing UAPI and event semantics.

Select V4L2_FIM from VIDEO_IMX_MEDIA and remove the completed TODO.

---
 drivers/media/v4l2-core/Kconfig           |   4 +
 drivers/media/v4l2-core/Makefile          |   1 +
 drivers/media/v4l2-core/v4l2-fim.c        | 331 ++++++++++++++++++++
 drivers/staging/media/imx/Kconfig         |   1 +
 drivers/staging/media/imx/TODO            |   3 -
 drivers/staging/media/imx/imx-media-fim.c | 355 ++++------------------
 include/media/v4l2-fim.h                  |  40 +++
 7 files changed, 429 insertions(+), 306 deletions(-)
 create mode 100644 drivers/media/v4l2-core/v4l2-fim.c
 delete mode 100644 drivers/staging/media/imx/TODO
 create mode 100644 include/media/v4l2-fim.h
drivers/media/v4l2-core/Kconfig +4 -0
index d50ccac9733c..cff961b24c04 100644
@@ -40,6 +40,10 @@ config VIDEO_TUNER
 config V4L2_JPEG_HELPER
 	tristate
 
+# Used by drivers that need v4l2-fim.ko
+config V4L2_FIM
+	tristate
+
 # Used by drivers that need v4l2-h264.ko
 config V4L2_H264
 	tristate
drivers/media/v4l2-core/Makefile +1 -0
index 329f0eadce99..d5f6699a2669 100644
@@ -26,6 +26,7 @@ videodev-$(CONFIG_VIDEO_V4L2_I2C) += v4l2-i2c.o
 
 obj-$(CONFIG_V4L2_ASYNC) += v4l2-async.o
 obj-$(CONFIG_V4L2_CCI) += v4l2-cci.o
+obj-$(CONFIG_V4L2_FIM) += v4l2-fim.o
 obj-$(CONFIG_V4L2_FLASH_LED_CLASS) += v4l2-flash-led-class.o
 obj-$(CONFIG_V4L2_FWNODE) += v4l2-fwnode.o
 obj-$(CONFIG_V4L2_H264) += v4l2-h264.o
drivers/media/v4l2-core/v4l2-fim.c new file +331 -0

331 lines not on this page · raw message

drivers/staging/media/imx/Kconfig +1 -0
index 1cd48028b641..f559b9979039 100644
@@ -6,6 +6,7 @@ config VIDEO_IMX_MEDIA
 	depends on VIDEO_DEV
 	depends on IMX_IPUV3_CORE
 	select MEDIA_CONTROLLER
+	select V4L2_FIM
 	select V4L2_FWNODE
 	select V4L2_MEM2MEM_DEV
 	select VIDEOBUF2_DMA_CONTIG
drivers/staging/media/imx/TODO deleted +0 -3
deleted file mode 100644
index 43cd030d4bbd..000000000000
@@ -1,3 +0,0 @@
-
-- The Frame Interval Monitor could be exported to v4l2-core for
-  general use.
drivers/staging/media/imx/imx-media-fim.c +52 -303

465 lines not on this page · raw message

include/media/v4l2-fim.h new file +40 -0
new file mode 100644
index 000000000000..6d649a56b08f
@@ -0,0 +1,40 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * V4L2 frame interval monitor helpers
+ *
+ * Copyright (c) 2016 Mentor Graphics Inc.
+ */
+
+#ifndef _V4L2_FIM_H
+#define _V4L2_FIM_H
+
+#include <linux/ktime.h>
+#include <linux/types.h>
+
+struct v4l2_ctrl_handler;
+struct v4l2_fract;
+struct v4l2_subdev;
+
+struct v4l2_fim;
+
+#define V4L2_FIM_NUM_CONTROLS	5
+
+struct v4l2_fim_config {
+	u32 ctrl_base;
+	u32 event_type;
+	void *priv;
+	bool (*external_timestamp)(void *priv, bool curval, bool use_val);
+	void (*wait_first_timestamp)(void *priv);
+};
+
+struct v4l2_fim *v4l2_fim_init(struct v4l2_subdev *sd,
+			       struct v4l2_ctrl_handler *hdlr,
+			       const struct v4l2_fim_config *config);
+void v4l2_fim_reset(struct v4l2_fim *fim, bool curval, struct v4l2_ctrl *ctrl);
+void v4l2_fim_set_stream(struct v4l2_fim *fim,
+			 const struct v4l2_fract *frame_interval, bool on);
+void v4l2_fim_eof_monitor(struct v4l2_fim *fim, ktime_t timestamp);
+void v4l2_fim_monitor_timestamp(struct v4l2_fim *fim, ktime_t timestamp);
+bool v4l2_fim_is_streaming(struct v4l2_fim *fim);
+
+#endif
Signature · 1 line
2.53.0
#4 lore raw

Re: [PATCH 2/2] media: v4l2-core: add frame interval monitor helper

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Low] media: v4l2-fim: missing struct v4l2_ctrl forward declaration in v4l2-fim.h --

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Low] media: v4l2-fim: missing struct v4l2_ctrl forward declaration in v4l2-fim.h
--

commit 4054bba4817179535aaa3077407029639e3caccd
Author: Jiaqi Shi <[email protected]>

media: v4l2-core: add frame interval monitor helper

This commit extracts the generic frame interval monitoring logic from the staging i.MX media driver into a shared v4l2-core helper and converts the i.MX driver to use it.

Quoted 6 lines
include/media/v4l2-fim.h new file +0 -0
new file mode 100644
index 0000000000000..6d649a56b08f4
@@ -0,0 +1,40 @@

[ ... ]

Quoted 13 lines
include/media/v4l2-fim.h new file +13 -0
+#include <linux/ktime.h>
+#include <linux/types.h>
+
+struct v4l2_ctrl_handler;
+struct v4l2_fract;
+struct v4l2_subdev;
+
+struct v4l2_fim;
+
+#define V4L2_FIM_NUM_CONTROLS	5
+
+struct v4l2_fim_config {
+	u32 ctrl_base;

include/media/v4l2-fim.h line 33

Quoted 10 lines
include/media/v4l2-fim.h new file +10 -0
+	u32 event_type;
+	void *priv;
+	bool (*external_timestamp)(void *priv, bool curval, bool use_val);
+	void (*wait_first_timestamp)(void *priv);
+};
+
+struct v4l2_fim *v4l2_fim_init(struct v4l2_subdev *sd,
+			       struct v4l2_ctrl_handler *hdlr,
+			       const struct v4l2_fim_config *config);
+void v4l2_fim_reset(struct v4l2_fim *fim, bool curval, struct v4l2_ctrl *ctrl);

[Severity: Low]
Should struct v4l2_ctrl also be forward-declared here? Without it, any translation unit that includes media/v4l2-fim.h before media/v4l2-ctrls.h scopes struct v4l2_ctrl to the v4l2_fim_reset() parameter list and fails to compile with -Wvisibility and incompatible-pointer-type errors.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2