Skip to contentSkip to conversation

media: imx: staging cleanup and FIM export to v4l2-core

PATCH Superseded Jiaqi Shi 2 participants 5 messages Nov 21

Subsystems (4)

v3 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Jiaqi Shi

author
#0 lore raw

v3 cover media: imx: staging cleanup and FIM export to v4l2-core

Changes in v3:
- Add the missing struct v4l2_ctrl forward declaration to v4l2-fim.h.
- Validate exact media-bus-code/pixel-format pairs without falling back to another format, while preserving the intentional Y10/Y12 to GREY clamping supported by the CSI driver.
- Use 64-bit arithmetic for image size calculation and reduce the height when necessary to fit the 32-bit sizeimage field instead of failing.
- Sanitize ANY/invalid field values and validate field compatibility with the source at stream start, including IDMAC interweave field ordering.
- Default RGB formats without a media bus code to full-range quantization.
- Keep legacy_api enabled for the remaining STD and frame-interval ioctls; removing those interfaces is outside the scope of this cleanup.

This series continues the i.MX staging media driver cleanup by removing legacy control inheritance and source-dependent format handling from capture video nodes, and exports the generic Frame Interval Monitor to v4l2-core while retaining the i.MX input capture controls locally.

Validation:
- allmodconfig build and sparse
- local FIM KUnit tests, not included in this series: 13/13 passed
- local white-box i.MX capture format tests, not included: 20/20 passed
- QEMU vivid: v4l2-compliance 125/125, 30-frame streaming passed
- QEMU vim2m: v4l2-compliance 49/49, 10-frame streaming passed
- i.MX register-level and hardware DMA paths remain untested

Jiaqi Shi (2):
media: imx: remove legacy video node control and format handling
media: v4l2-core: add frame interval monitor helper

 Documentation/admin-guide/media/imx.rst       |  20 +-
 drivers/media/v4l2-core/Kconfig               |   4 +
 drivers/media/v4l2-core/Makefile              |   1 +
 drivers/media/v4l2-core/v4l2-fim.c            | 331 ++++++++++++++++
 drivers/staging/media/imx/Kconfig             |   1 +
 drivers/staging/media/imx/TODO                |  13 -
 drivers/staging/media/imx/imx-ic-prpencvf.c   |   7 +-
 drivers/staging/media/imx/imx-media-capture.c | 326 ++++++----------
 drivers/staging/media/imx/imx-media-csi.c     |   7 +-
 .../staging/media/imx/imx-media-dev-common.c  | 113 +-----
 drivers/staging/media/imx/imx-media-fim.c     | 355 +++---------------
 drivers/staging/media/imx/imx-media-utils.c   |   9 +-
 drivers/staging/media/imx/imx-media.h         |   9 +-
 include/media/v4l2-fim.h                      |  41 ++
 14 files changed, 573 insertions(+), 664 deletions(-)
 create mode 100644 drivers/media/v4l2-core/v4l2-fim.c
 delete mode 100644 drivers/staging/media/imx/TODO
 create mode 100644 include/media/v4l2-fim.h
Signature · 1 line
2.53.0
author
#1 lore raw

PATCH v3 1/2 media: imx: remove legacy video node control and format handling

Remove control inheritance from pipeline subdevices to video nodes. Userspace should access controls through subdev nodes instead.

Use source-independent format ioctls for all capture nodes and initialize them to the existing UYVY 640x480 default instead of the source subdev active format. Keep stream-time format validation.

Update the usage documentation and remove the completed TODO entries.

---
 Documentation/admin-guide/media/imx.rst       |  20 +-
 drivers/staging/media/imx/TODO                |  10 -
 drivers/staging/media/imx/imx-ic-prpencvf.c   |   7 +-
 drivers/staging/media/imx/imx-media-capture.c | 326 +++++++-----------
 drivers/staging/media/imx/imx-media-csi.c     |   7 +-
 .../staging/media/imx/imx-media-dev-common.c  | 113 +-----
 drivers/staging/media/imx/imx-media-utils.c   |   9 +-
 drivers/staging/media/imx/imx-media.h         |   9 +-
 8 files changed, 143 insertions(+), 358 deletions(-)
Documentation/admin-guide/media/imx.rst +8 -12
index bb68100d8acb..812c6ac09ab8 100644
@@ -394,13 +394,9 @@ scaling up to 1024x1024, CSC, flip, and rotation:
 Usage Notes
 -----------
 
-To aid in configuration and for backward compatibility with V4L2
-applications that access controls only from video device nodes, the
-capture device interfaces inherit controls from the active entities
-in the current pipeline, so controls can be accessed either directly
-from the subdev or from the active capture device interface. For
-example, the FIM controls are available either from the ipuX_csiY
-subdevs or from the active capture device.
+Controls for pipeline entities are accessed through their subdev device
+nodes. For example, the FIM controls are available from the ipuX_csiY
+subdevs.
 
 The following are specific usage notes for the Sabre* reference
 boards:
@@ -483,7 +479,7 @@ NTSC, or 'alternate'):
    media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/720x480]"
    media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/720x480]"
    # Configure "ipu1_csi0 capture" interface (assumed at /dev/video4)
-   v4l2-ctl -d4 --set-fmt-video=field=interlaced_bt
+   v4l2-ctl -d4 --set-fmt-video=width=720,height=480,field=interlaced_bt
 
 Streaming can then begin on /dev/video4. The v4l2-ctl tool can also be
 used to select any supported YUV pixelformat on /dev/video4.
@@ -510,7 +506,7 @@ alternating fields (field type 'seq-tb' for PAL, or 'alternate').
    media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x576 field:none]"
    media-ctl -V "'ipu1_ic_prpvf':1 [fmt:AYUV32/720x576 field:none]"
    # Configure "ipu1_ic_prpvf capture" interface (assumed at /dev/video2)
-   v4l2-ctl -d2 --set-fmt-video=field=none
+   v4l2-ctl -d2 --set-fmt-video=width=720,height=576,field=none
 
 Streaming can then begin on /dev/video2. The v4l2-ctl tool can also be
 used to select any supported YUV pixelformat on /dev/video2.
@@ -541,7 +537,7 @@ NTSC, or 'alternate'):
    media-ctl -V "'ipu1_csi0_mux':5 [fmt:UYVY2X8/720x480]"
    media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/720x480]"
    # Configure "ipu1_csi0 capture" interface (assumed at /dev/video0)
-   v4l2-ctl -d0 --set-fmt-video=field=interlaced_bt
+   v4l2-ctl -d0 --set-fmt-video=width=720,height=480,field=interlaced_bt
 
 Streaming can then begin on /dev/video0. The v4l2-ctl tool can also be
 used to select any supported YUV pixelformat on /dev/video0.
@@ -568,7 +564,7 @@ alternating fields (field type 'seq-tb' for PAL, or 'alternate').
    media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x576 field:none]"
    media-ctl -V "'ipu1_ic_prpvf':1 [fmt:AYUV32/720x576 field:none]"
    # Configure "ipu1_ic_prpvf capture" interface (assumed at /dev/video2)
-   v4l2-ctl -d2 --set-fmt-video=field=none
+   v4l2-ctl -d2 --set-fmt-video=width=720,height=576,field=none
 
 Streaming can then begin on /dev/video2. The v4l2-ctl tool can also be
 used to select any supported YUV pixelformat on /dev/video2.
@@ -652,7 +648,7 @@ shows colorspace conversion and scaling at IC output.
    media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/640x480]"
    media-ctl -V "'ipu1_ic_prpenc':1 [fmt:ARGB8888_1X32/800x600]"
    # Set a format at the capture interface
-   v4l2-ctl -d /dev/video1 --set-fmt-video=pixelformat=RGB3
+   v4l2-ctl -d /dev/video1 --set-fmt-video=width=800,height=600,pixelformat=RGB3
 
 Streaming can then begin on "ipu1_ic_prpenc capture" node.
 
drivers/staging/media/imx/TODO +0 -10
index 11c9e10d34ae..43cd030d4bbd 100644
@@ -1,13 +1,3 @@
 
 - The Frame Interval Monitor could be exported to v4l2-core for
   general use.
-
-- This media driver supports inheriting V4L2 controls to the
-  video capture devices, from the subdevices in the capture device's
-  pipeline. The controls for each capture device are updated in the
-  link_notify callback when the pipeline is modified. This feature should be
-  removed, userspace should use the subdev-based userspace API instead.
-
-- Similarly to the legacy control handling, legacy format handling where
-  formats on the video nodes are influenced by the active format of the
-  connected subdev should be removed.
drivers/staging/media/imx/imx-ic-prpencvf.c +2 -5
index 3430ddeac356..372d25c791d3 100644
@@ -1280,13 +1280,13 @@ static int prp_registered(struct v4l2_subdev *sd)
 
 	priv->vdev = imx_media_capture_device_init(ic_priv->ipu_dev,
 						   &ic_priv->sd,
-						   PRPENCVF_SRC_PAD, true);
+						   PRPENCVF_SRC_PAD, true, true);
 	if (IS_ERR(priv->vdev))
 		return PTR_ERR(priv->vdev);
 
 	ret = imx_media_capture_device_register(priv->vdev, 0);
 	if (ret)
-		goto remove_vdev;
+		return ret;
 
 	ret = prp_init_controls(priv);
 	if (ret)
@@ -1296,8 +1296,6 @@ static int prp_registered(struct v4l2_subdev *sd)
 
 unreg_vdev:
 	imx_media_capture_device_unregister(priv->vdev);
-remove_vdev:
-	imx_media_capture_device_remove(priv->vdev);
 	return ret;
 }
 
@@ -1306,7 +1304,6 @@ static void prp_unregistered(struct v4l2_subdev *sd)
 	struct prp_priv *priv = sd_to_priv(sd);
 
 	imx_media_capture_device_unregister(priv->vdev);
-	imx_media_capture_device_remove(priv->vdev);
 
 	v4l2_ctrl_handler_free(&priv->ctrl_hdlr);
 }
drivers/staging/media/imx/imx-media-capture.c +120 -206

468 lines not on this page · raw message

drivers/staging/media/imx/imx-media-csi.c +2 -5
index 43c637b333f2..7e655260f241 100644
@@ -1815,7 +1815,7 @@ static int csi_registered(struct v4l2_subdev *sd)
 	}
 
 	priv->vdev = imx_media_capture_device_init(priv->sd.dev, &priv->sd,
-						   CSI_SRC_PAD_IDMAC, true);
+						   CSI_SRC_PAD_IDMAC, true, false);
 	if (IS_ERR(priv->vdev)) {
 		ret = PTR_ERR(priv->vdev);
 		goto free_fim;
@@ -1823,12 +1823,10 @@ static int csi_registered(struct v4l2_subdev *sd)
 
 	ret = imx_media_capture_device_register(priv->vdev, 0);
 	if (ret)
-		goto remove_vdev;
+		goto free_fim;
 
 	return 0;
 
-remove_vdev:
-	imx_media_capture_device_remove(priv->vdev);
 free_fim:
 	if (priv->fim)
 		imx_media_fim_free(priv->fim);
@@ -1842,7 +1840,6 @@ static void csi_unregistered(struct v4l2_subdev *sd)
 	struct csi_priv *priv = v4l2_get_subdevdata(sd);
 
 	imx_media_capture_device_unregister(priv->vdev);
-	imx_media_capture_device_remove(priv->vdev);
 
 	if (priv->fim)
 		imx_media_fim_free(priv->fim);
drivers/staging/media/imx/imx-media-dev-common.c +1 -112

131 lines not on this page · raw message

drivers/staging/media/imx/imx-media-utils.c +7 -2
index 7bdcb88a686b..7cd69a698124 100644
@@ -522,6 +522,7 @@ int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format *pix,
 {
 	u32 width;
 	u32 stride;
+	u64 sizeimage;
 
 	if (!cc) {
 		cc = imx_media_find_ipu_format(mbus->code,
@@ -562,8 +563,12 @@ int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format *pix,
 	pix->quantization = mbus->quantization;
 	pix->field = mbus->field;
 	pix->bytesperline = stride;
-	pix->sizeimage = cc->planar ? ((stride * pix->height * cc->bpp) >> 3) :
-			 stride * pix->height;
+	sizeimage = (u64)stride * pix->height;
+	if (cc->planar)
+		sizeimage = (sizeimage * cc->bpp) >> 3;
+	if (sizeimage > U32_MAX)
+		return -EINVAL;
+	pix->sizeimage = sizeimage;
 
 	return 0;
 }
drivers/staging/media/imx/imx-media.h +3 -6
index 537558a7bece..2f32a787a600 100644
@@ -125,10 +125,8 @@ static inline struct imx_media_buffer *to_imx_media_vb(struct vb2_buffer *vb)
 }
 
 /*
- * to support control inheritance to video devices, this
- * retrieves a pad's list_head of video devices that can
- * be reached from the pad. Note that only the lists in
- * source pads get populated, sink pads have empty lists.
+ * Retrieve a pad's list of video devices reachable from the pad for
+ * subdev event forwarding. Only source pad lists are populated.
  */
 static inline struct list_head *
 to_pad_vdev_list(struct v4l2_subdev *sd, int pad_index)
@@ -270,8 +268,7 @@ int imx_media_ic_unregister(struct v4l2_subdev *sd);
 /* imx-media-capture.c */
 struct imx_media_video_dev *
 imx_media_capture_device_init(struct device *dev, struct v4l2_subdev *src_sd,
-			      int pad, bool legacy_api);
-void imx_media_capture_device_remove(struct imx_media_video_dev *vdev);
+			      int pad, bool legacy_api, bool ic_route);
 int imx_media_capture_device_register(struct imx_media_video_dev *vdev,
 				      u32 link_flags);
 void imx_media_capture_device_unregister(struct imx_media_video_dev *vdev);
Signature · 1 line
2.53.0
#2 lore raw

Re: [PATCH v3 1/2] media: imx: remove legacy video node control and format handling

Thank you for your contribution! Sashiko AI review found 2 potential issues to consider: - [Medium] media: imx: capture_validate_field() mutates vdev.fmt.field on stream start - [Medium] media: imx…

Thank you for your contribution! Sashiko AI review found 2 potential issues to consider:
- [Medium] media: imx: capture_validate_field() mutates vdev.fmt.field on stream start
- [Medium] media: imx: callers of imx_media_capture_device_init() still pass legacy_api
--

commit f4224bc45a1438de57dbc43b915f511c93024a3e
Author: Jiaqi Shi <[email protected]>

media: imx: remove legacy video node control and format handling

This commit removes subdev control inheritance from capture video nodes and switches capture format handling and initialization to source-independent format ioctls while keeping stream-time format validation.

Quoted 4 lines
drivers/staging/media/imx/imx-media-capture.c +0 -0
index 9216a221b72f4..2e1cafc5f433e 100644

[ ... ]

drivers/staging/media/imx/imx-media-capture.c static const struct v4l2_ioctl_ops capture_legacy_ioctl_ops = { · line 479

Quoted 11 lines
drivers/staging/media/imx/imx-media-capture.c +3 -3
@@ -566,10 +473,10 @@ static const struct v4l2_ioctl_ops capture_legacy_ioctl_ops = {
 	.vidioc_enum_framesizes		= capture_legacy_enum_framesizes,
 	.vidioc_enum_frameintervals	= capture_legacy_enum_frameintervals,
 
-	.vidioc_enum_fmt_vid_cap	= capture_legacy_enum_fmt_vid_cap,
+	.vidioc_enum_fmt_vid_cap	= capture_enum_fmt_vid_cap,
 	.vidioc_g_fmt_vid_cap		= capture_g_fmt_vid_cap,
-	.vidioc_try_fmt_vid_cap		= capture_legacy_try_fmt_vid_cap,
-	.vidioc_s_fmt_vid_cap		= capture_legacy_s_fmt_vid_cap,
+	.vidioc_try_fmt_vid_cap		= capture_try_fmt_vid_cap,
+	.vidioc_s_fmt_vid_cap		= capture_s_fmt_vid_cap,

[Severity: Medium]
Should the callers of imx_media_capture_device_init() stop setting legacy_api to true now that video node format handling is decoupled from the source subdev?

Both in-tree callers (prp_registered() and csi_registered()) still pass legacy_api = true, which leaves V4L2_CAP_IO_MC unset on every capture device (so v4l_enum_fmt() zeroes f->mbus_code before calling capture_enum_fmt_vid_cap()) and leaves capture_ioctl_ops, capture_enum_framesizes(), and capture_subscribe_event() unused.

In addition, pairing capture_enum_fmt_vid_cap() with capture_legacy_enum_framesizes() breaks VIDIOC_ENUM_FRAMESIZES on the IC capture nodes: capture_enum_fmt_vid_cap() enumerates all non-IPU formats in pixel_formats[], and capture_legacy_enum_framesizes() forwards their media bus code (or 0 for in-memory-only formats) to prp_enum_frame_size(), which rejects any non-IPU media bus code with -EINVAL.

Quoted 17 lines
drivers/staging/media/imx/imx-media-capture.c +13 -0
@@ -672,6 +579,44 @@ static void capture_buf_queue(struct vb2_buffer *vb)
 	spin_unlock_irqrestore(&priv->q_lock, flags);
 }
 
+static int capture_validate_field(struct capture_priv *priv,
+				  const struct v4l2_mbus_framefmt *src)
+{
+	struct v4l2_pix_format *pix = &priv->vdev.fmt;
+
+	switch (pix->field) {
+	case V4L2_FIELD_INTERLACED:
+		switch (src->field) {
+		case V4L2_FIELD_SEQ_TB:
+			pix->field = V4L2_FIELD_INTERLACED_TB;
+			return 0;
+		case V4L2_FIELD_SEQ_BT:
+			pix->field = V4L2_FIELD_INTERLACED_BT;

drivers/staging/media/imx/imx-media-capture.c static void capture_buf_queue(struct vb2_buffer *vb) · line 604

Quoted 10 lines
drivers/staging/media/imx/imx-media-capture.c +10 -0
+			return 0;
+		case V4L2_FIELD_INTERLACED:
+			return 0;
+		case V4L2_FIELD_INTERLACED_TB:
+		case V4L2_FIELD_INTERLACED_BT:
+			pix->field = src->field;
+			return 0;
+		default:
+			return -EPIPE;
+		}

[Severity: Medium]
Should capture_validate_field() avoid overwriting priv->vdev.fmt.field during VIDIOC_STREAMON?

When userspace configures V4L2_FIELD_INTERLACED via VIDIOC_S_FMT, capture_start_streaming() -> capture_validate_fmt() -> capture_validate_field() mutates priv->vdev.fmt.field in place to V4L2_FIELD_INTERLACED_TB or V4L2_FIELD_INTERLACED_BT. This changes the format reported by VIDIOC_G_FMT (even if imx_media_pipeline_set_stream()
subsequently fails), and if the upstream subdev field order is later
switched between top-bottom and bottom-top without another VIDIOC_S_FMT, a subsequent VIDIOC_STREAMON will validate against the overwritten field order and fail with -EPIPE.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
author
#3 lore raw

PATCH v3 2/2 media: v4l2-core: add frame interval monitor helper

In reply to Jiaqi Shi

Move the generic frame interval monitoring implementation from i.MX to v4l2-core. The helper supports driver-selected control and event IDs and accepts EOF or external timestamps.

Keep the i.MX-specific input capture controls in the i.MX adapter, sharing the original control handler and preserving the existing UAPI and event semantics.

Select V4L2_FIM from VIDEO_IMX_MEDIA and remove the completed TODO.

---
 drivers/media/v4l2-core/Kconfig           |   4 +
 drivers/media/v4l2-core/Makefile          |   1 +
 drivers/media/v4l2-core/v4l2-fim.c        | 331 ++++++++++++++++++++
 drivers/staging/media/imx/Kconfig         |   1 +
 drivers/staging/media/imx/TODO            |   3 -
 drivers/staging/media/imx/imx-media-fim.c | 355 ++++------------------
 include/media/v4l2-fim.h                  |  41 +++
 7 files changed, 430 insertions(+), 306 deletions(-)
 create mode 100644 drivers/media/v4l2-core/v4l2-fim.c
 delete mode 100644 drivers/staging/media/imx/TODO
 create mode 100644 include/media/v4l2-fim.h
drivers/media/v4l2-core/Kconfig +4 -0
index d50ccac9733c..cff961b24c04 100644
@@ -40,6 +40,10 @@ config VIDEO_TUNER
 config V4L2_JPEG_HELPER
 	tristate
 
+# Used by drivers that need v4l2-fim.ko
+config V4L2_FIM
+	tristate
+
 # Used by drivers that need v4l2-h264.ko
 config V4L2_H264
 	tristate
drivers/media/v4l2-core/Makefile +1 -0
index 329f0eadce99..d5f6699a2669 100644
@@ -26,6 +26,7 @@ videodev-$(CONFIG_VIDEO_V4L2_I2C) += v4l2-i2c.o
 
 obj-$(CONFIG_V4L2_ASYNC) += v4l2-async.o
 obj-$(CONFIG_V4L2_CCI) += v4l2-cci.o
+obj-$(CONFIG_V4L2_FIM) += v4l2-fim.o
 obj-$(CONFIG_V4L2_FLASH_LED_CLASS) += v4l2-flash-led-class.o
 obj-$(CONFIG_V4L2_FWNODE) += v4l2-fwnode.o
 obj-$(CONFIG_V4L2_H264) += v4l2-h264.o
drivers/media/v4l2-core/v4l2-fim.c new file +331 -0

331 lines not on this page · raw message

drivers/staging/media/imx/Kconfig +1 -0
index 1cd48028b641..f559b9979039 100644
@@ -6,6 +6,7 @@ config VIDEO_IMX_MEDIA
 	depends on VIDEO_DEV
 	depends on IMX_IPUV3_CORE
 	select MEDIA_CONTROLLER
+	select V4L2_FIM
 	select V4L2_FWNODE
 	select V4L2_MEM2MEM_DEV
 	select VIDEOBUF2_DMA_CONTIG
drivers/staging/media/imx/TODO deleted +0 -3
deleted file mode 100644
index 43cd030d4bbd..000000000000
@@ -1,3 +0,0 @@
-
-- The Frame Interval Monitor could be exported to v4l2-core for
-  general use.
drivers/staging/media/imx/imx-media-fim.c +52 -303

465 lines not on this page · raw message

include/media/v4l2-fim.h new file +41 -0

41 lines not on this page · raw message

Signature · 1 line
2.53.0
#4 lore raw

Re: [PATCH v3 2/2] media: v4l2-core: add frame interval monitor helper

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Low] media: v4l2-core: missing kernel-doc for new v4l2-fim public API --

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Low] media: v4l2-core: missing kernel-doc for new v4l2-fim public API --

commit 70fa26e77e3e68f76d9d83df7734532469e6749d
Author: Jiaqi Shi <[email protected]>

media: v4l2-core: add frame interval monitor helper

This commit moves the generic frame interval monitoring implementation from the i.MX staging media driver to v4l2-core as a reusable helper module.

Quoted 6 lines
include/media/v4l2-fim.h new file +0 -0
new file mode 100644
index 0000000000000..e44dd4ac90105
@@ -0,0 +1,41 @@

[ ... ]

Quoted 11 lines
include/media/v4l2-fim.h new file +11 -0
+#define V4L2_FIM_NUM_CONTROLS	5
+
+struct v4l2_fim_config {
+	u32 ctrl_base;
+	u32 event_type;
+	void *priv;
+	bool (*external_timestamp)(void *priv, bool curval, bool use_val);
+	void (*wait_first_timestamp)(void *priv);
+};
+
+struct v4l2_fim *v4l2_fim_init(struct v4l2_subdev *sd,

include/media/v4l2-fim.h line 41

Quoted 10 lines
include/media/v4l2-fim.h new file +10 -0
+			       struct v4l2_ctrl_handler *hdlr,
+			       const struct v4l2_fim_config *config);
+void v4l2_fim_reset(struct v4l2_fim *fim, bool curval, struct v4l2_ctrl *ctrl);
+void v4l2_fim_set_stream(struct v4l2_fim *fim,
+			 const struct v4l2_fract *frame_interval, bool on);
+void v4l2_fim_eof_monitor(struct v4l2_fim *fim, ktime_t timestamp);
+void v4l2_fim_monitor_timestamp(struct v4l2_fim *fim, ktime_t timestamp);
+bool v4l2_fim_is_streaming(struct v4l2_fim *fim);
+
+#endif

[Severity: Low]
This isn't a bug, but the new public v4l2-core helper API has no kernel-doc comments describing allocation, cleanup, or locking expectations:

- v4l2_fim_init() allocates struct v4l2_fim with devm_kzalloc(sd->dev, ...) and adds V4L2_FIM_NUM_CONTROLS controls starting at config->ctrl_base to hdlr, which the caller remains responsible for freeing with v4l2_ctrl_handler_free().
- config->external_timestamp() is called under spin_lock_irqsave(&fim->lock) with the control handler lock held, while config->wait_first_timestamp() is called from process context in v4l2_fim_set_stream() with v4l2_ctrl_lock() held.
- v4l2_fim_set_stream() acquires v4l2_ctrl_lock(), v4l2_fim_reset() and v4l2_fim_is_streaming() expect the caller to hold the control handler lock, and v4l2_fim_eof_monitor() / v4l2_fim_monitor_timestamp() acquire spin_lock_irqsave(&fim->lock) and may be called from interrupt context.

Also, reset_fim() invokes config->external_timestamp(fim->config.priv, curval, !curval && !core_active); passing both curval and use_val is redundant since the callback only needs a single boolean indicating whether to read uncommitted control->val or committed control->cur.val.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2