Skip to contentSkip to conversation

media: imx: Create media links in bound notifiers

PATCH Superseded Steve Longerbeam 3 participants 38 messages Mar 3, 2020 – Apr 18, 2020

Subsystems (7)

v4 of 6 Superseded Latest is v6 →

lore
Share & tools
lore

Steve Longerbeam

author maintainer
#0 lore raw

v4 cover media: imx: Create media links in bound notifiers

Move media link creation into the notifier bound callbacks in the minimum set of subdevices required by imx (imx5/6/7 CSI, imx6/7 MIPI CSI-2, and video mux).

History:

v4:
- Removed the endpoint parsing callback APIs from video-mux and imx drivers as suggested by Sakari, replacing with endpoint parsing local to the drivers and the use of v4l2_async_notifier_add_fwnode_remote_subdev(). As a result convenience function v4l2_async_register_fwnode_subdev() is no longer used and is reverted.

v3:
- The changes to the default behaviour in media_entity_get_fwnode_pad(), and the fixes to current media drivers that call it inconsistently, have been put-off to another time. Instead this version implements the get_fwnode_pad operation where required in the imx and video-mux subdevices to make media link creation work correctly. The improvements to media_entity_get_fwnode_pad() can wait to another patch series.

v2:
- rename/move the notifier-to-state inlines in imx7-mipi-csis.c and imx7-media-csi.c, suggested by Rui Silva.
- rewrite imx_media_create_links() to only add the missing media links from the imx6 MIPI CSI-2 receiver.

Steve Longerbeam (17):
media: entity: Pass entity to get_fwnode_pad operation
media: video-mux: Parse information from firmware without using
callbacks
media: imx: Parse information from firmware without using callbacks Revert "media: v4l2-fwnode: Add a convenience function for registering
subdevs with notifiers"
media: imx: csi: Implement get_fwnode_pad op
media: imx: mipi csi-2: Implement get_fwnode_pad op
media: video-mux: Implement get_fwnode_pad op
media: imx: Add imx_media_create_fwnode_pad_link()
media: video-mux: Create media links in bound notifier
media: imx: mipi csi-2: Create media links in bound notifier
media: imx7: mipi csis: Create media links in bound notifier
media: imx7: csi: Create media links in bound notifier
media: imx: csi: Create media links in bound notifier
media: imx: csi: Lookup upstream endpoint with
imx_media_get_pad_fwnode
media: imx: Create missing links from CSI-2 receiver
media: imx: silence a couple debug messages
media: imx: TODO: Remove media link creation todos

 drivers/media/mc/mc-entity.c                  |   2 +-
 drivers/media/platform/video-mux.c            | 185 ++++++++++++++++--
 drivers/media/v4l2-core/v4l2-fwnode.c         |  62 ------
 drivers/staging/media/imx/TODO                |  29 ---
 drivers/staging/media/imx/imx-media-csi.c     | 136 ++++++++-----
 .../staging/media/imx/imx-media-dev-common.c  |  50 ++---
 drivers/staging/media/imx/imx-media-dev.c     |   2 +-
 .../staging/media/imx/imx-media-internal-sd.c |   6 +-
 drivers/staging/media/imx/imx-media-of.c      | 114 -----------
 drivers/staging/media/imx/imx-media-utils.c   | 124 ++++++++++++
 drivers/staging/media/imx/imx-media.h         |   9 +-
 drivers/staging/media/imx/imx6-mipi-csi2.c    | 119 +++++++++--
 drivers/staging/media/imx/imx7-media-csi.c    | 100 +++++++---
 drivers/staging/media/imx/imx7-mipi-csis.c    | 105 +++++++---
 include/media/media-entity.h                  |   3 +-
 include/media/v4l2-fwnode.h                   |  38 ----
 16 files changed, 654 insertions(+), 430 deletions(-)
Signature · 1 line
2.17.1
author maintainer
#1 lore raw

PATCH v4 1/17 media: entity: Pass entity to get_fwnode_pad operation

Add a missing pointer to the entity in the media_entity operation get_fwnode_pad. There are no implementers of this op yet, but a future entity that does so will almost certainly need a reference to itself to carry out the work.

Fixes ae45cd5efc120 ("[media] media: entity: Add get_fwnode_pad entity

operation")

---
 drivers/media/mc/mc-entity.c | 2 +-
 include/media/media-entity.h | 3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)
drivers/media/mc/mc-entity.c +1 -1
index 211279c5fd77..12b45e669bcc 100644
@@ -386,7 +386,7 @@ int media_entity_get_fwnode_pad(struct media_entity *entity,
 	if (ret)
 		return ret;
 
-	ret = entity->ops->get_fwnode_pad(&endpoint);
+	ret = entity->ops->get_fwnode_pad(entity, &endpoint);
 	if (ret < 0)
 		return ret;
 
include/media/media-entity.h +2 -1
index 8cb2c504a05c..cde80ad029b7 100644
@@ -212,7 +212,8 @@ struct media_pad {
  *    mutex held.
  */
 struct media_entity_operations {
-	int (*get_fwnode_pad)(struct fwnode_endpoint *endpoint);
+	int (*get_fwnode_pad)(struct media_entity *entity,
+			      struct fwnode_endpoint *endpoint);
 	int (*link_setup)(struct media_entity *entity,
 			  const struct media_pad *local,
 			  const struct media_pad *remote, u32 flags);
Signature · 1 line
2.17.1
maintainer Reviewed-by on v4 1/17
#2 lore raw

Re: [PATCH v4 01/17] media: entity: Pass entity to get_fwnode_pad operation

Hi Steve,

Thank you for the patch.

Quoted 8 lines · Steve Longerbeam

Add a missing pointer to the entity in the media_entity operation get_fwnode_pad. There are no implementers of this op yet, but a future entity that does so will almost certainly need a reference to itself to carry out the work.

Fixes ae45cd5efc120 ("[media] media: entity: Add get_fwnode_pad entity

operation")

Quoted 32 lines
---
 drivers/media/mc/mc-entity.c | 2 +-
 include/media/media-entity.h | 3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)
drivers/media/mc/mc-entity.c +1 -1
index 211279c5fd77..12b45e669bcc 100644
@@ -386,7 +386,7 @@ int media_entity_get_fwnode_pad(struct media_entity *entity,
 	if (ret)
 		return ret;
 
-	ret = entity->ops->get_fwnode_pad(&endpoint);
+	ret = entity->ops->get_fwnode_pad(entity, &endpoint);
 	if (ret < 0)
 		return ret;
 
include/media/media-entity.h +2 -1
index 8cb2c504a05c..cde80ad029b7 100644
@@ -212,7 +212,8 @@ struct media_pad {
  *    mutex held.
  */
 struct media_entity_operations {
-	int (*get_fwnode_pad)(struct fwnode_endpoint *endpoint);
+	int (*get_fwnode_pad)(struct media_entity *entity,
+			      struct fwnode_endpoint *endpoint);
 	int (*link_setup)(struct media_entity *entity,
 			  const struct media_pad *local,
 			  const struct media_pad *remote, u32 flags);
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#3 lore raw

PATCH v4 2/17 media: video-mux: Parse information from firmware without using callbacks

In reply to Steve Longerbeam

Instead of using the convenience function v4l2_async_register_fwnode_subdev(), parse the video-mux input endpoints and set up the async sub-devices without using callbacks. The video-mux knows which ports it must parse (the input ports) and how to handle unconnected remotes, so it makes the code simpler to transfer control of endpoint parsing to the driver.

---
 drivers/media/platform/video-mux.c | 70 ++++++++++++++++++++----------
 1 file changed, 46 insertions(+), 24 deletions(-)
drivers/media/platform/video-mux.c +46 -24
index ddd0e338f9e4..7b6c96a29aa5 100644
@@ -21,6 +21,7 @@
 
 struct video_mux {
 	struct v4l2_subdev subdev;
+	struct v4l2_async_notifier notifier;
 	struct media_pad *pads;
 	struct v4l2_mbus_framefmt *format_mbus;
 	struct mux_control *mux;
@@ -330,36 +331,49 @@ static const struct v4l2_subdev_ops video_mux_subdev_ops = {
 	.video = &video_mux_subdev_video_ops,
 };
 
-static int video_mux_parse_endpoint(struct device *dev,
-				    struct v4l2_fwnode_endpoint *vep,
-				    struct v4l2_async_subdev *asd)
-{
-	/*
-	 * it's not an error if remote is missing on a video-mux
-	 * input port, return -ENOTCONN to skip this endpoint with
-	 * no error.
-	 */
-	return fwnode_device_is_available(asd->match.fwnode) ? 0 : -ENOTCONN;
-}
-
 static int video_mux_async_register(struct video_mux *vmux,
 				    unsigned int num_input_pads)
 {
-	unsigned int i, *ports;
+	unsigned int i;
 	int ret;
 
-	ports = kcalloc(num_input_pads, sizeof(*ports), GFP_KERNEL);
-	if (!ports)
-		return -ENOMEM;
-	for (i = 0; i < num_input_pads; i++)
-		ports[i] = i;
+	v4l2_async_notifier_init(&vmux->notifier);
 
-	ret = v4l2_async_register_fwnode_subdev(
-		&vmux->subdev, sizeof(struct v4l2_async_subdev),
-		ports, num_input_pads, video_mux_parse_endpoint);
+	for (i = 0; i < num_input_pads; i++) {
+		struct v4l2_async_subdev *asd;
+		struct fwnode_handle *ep;
 
-	kfree(ports);
-	return ret;
+		ep = fwnode_graph_get_endpoint_by_id(
+			dev_fwnode(vmux->subdev.dev), i, 0,
+			FWNODE_GRAPH_ENDPOINT_NEXT);
+		if (!ep)
+			continue;
+
+		asd = kzalloc(sizeof(*asd), GFP_KERNEL);
+		if (!asd) {
+			fwnode_handle_put(ep);
+			return -ENOMEM;
+		}
+
+		ret = v4l2_async_notifier_add_fwnode_remote_subdev(
+			&vmux->notifier, ep, asd);
+
+		fwnode_handle_put(ep);
+
+		if (ret) {
+			kfree(asd);
+			/* OK if asd already exists */
+			if (ret != -EEXIST)
+				return ret;
+		}
+	}
+
+	ret = v4l2_async_subdev_notifier_register(&vmux->subdev,
+						  &vmux->notifier);
+	if (ret)
+		return ret;
+
+	return v4l2_async_register_subdev(&vmux->subdev);
 }
 
 static int video_mux_probe(struct platform_device *pdev)
@@ -434,7 +448,13 @@ static int video_mux_probe(struct platform_device *pdev)
 
 	vmux->subdev.entity.ops = &video_mux_ops;
 
-	return video_mux_async_register(vmux, num_pads - 1);
+	ret = video_mux_async_register(vmux, num_pads - 1);
+	if (ret) {
+		v4l2_async_notifier_unregister(&vmux->notifier);
+		v4l2_async_notifier_cleanup(&vmux->notifier);
+	}
+
+	return ret;
 }
 
 static int video_mux_remove(struct platform_device *pdev)
@@ -442,6 +462,8 @@ static int video_mux_remove(struct platform_device *pdev)
 	struct video_mux *vmux = platform_get_drvdata(pdev);
 	struct v4l2_subdev *sd = &vmux->subdev;
 
+	v4l2_async_notifier_unregister(&vmux->notifier);
+	v4l2_async_notifier_cleanup(&vmux->notifier);
 	v4l2_async_unregister_subdev(sd);
 	media_entity_cleanup(&sd->entity);
 
Signature · 1 line
2.17.1
author maintainer
#4 lore raw

PATCH v4 3/17 media: imx: Parse information from firmware without using callbacks

In reply to Steve Longerbeam

Instead of using the convenience functions v4l2_async_notifier_parse_fwnode_endpoints*() or v4l2_async_register_fwnode_subdev(), parse the input endpoints and set up the async sub-devices without using callbacks. The drivers know which ports it must parse and how to handle unconnected remotes, so it makes the code simpler to transfer control of endpoint parsing to the driver.

---
 drivers/staging/media/imx/imx-media-csi.c  | 78 +++++++++----------
 drivers/staging/media/imx/imx6-mipi-csi2.c | 70 ++++++++++++-----
 drivers/staging/media/imx/imx7-media-csi.c | 53 ++++++++++---
 drivers/staging/media/imx/imx7-mipi-csis.c | 91 +++++++++++++++-------
 4 files changed, 192 insertions(+), 100 deletions(-)
drivers/staging/media/imx/imx-media-csi.c +38 -40
index b60ed4f22f6d..f409fca88dcf 100644
@@ -58,6 +58,8 @@ struct csi_priv {
 	struct ipu_soc *ipu;
 	struct v4l2_subdev sd;
 	struct media_pad pad[CSI_NUM_PADS];
+	struct v4l2_async_notifier notifier;
+
 	/* the video device at IDMAC output pad */
 	struct imx_media_video_dev *vdev;
 	struct imx_media_fim *fim;
@@ -1864,59 +1866,49 @@ static const struct v4l2_subdev_internal_ops csi_internal_ops = {
 	.unregistered = csi_unregistered,
 };
 
-static int imx_csi_parse_endpoint(struct device *dev,
-				  struct v4l2_fwnode_endpoint *vep,
-				  struct v4l2_async_subdev *asd)
-{
-	return fwnode_device_is_available(asd->match.fwnode) ? 0 : -ENOTCONN;
-}
-
 static int imx_csi_async_register(struct csi_priv *priv)
 {
-	struct v4l2_async_notifier *notifier;
-	struct fwnode_handle *fwnode;
+	struct v4l2_async_subdev *asd = NULL;
+	struct fwnode_handle *ep;
 	unsigned int port;
 	int ret;
 
-	notifier = kzalloc(sizeof(*notifier), GFP_KERNEL);
-	if (!notifier)
-		return -ENOMEM;
-
-	v4l2_async_notifier_init(notifier);
-
-	fwnode = dev_fwnode(priv->dev);
+	v4l2_async_notifier_init(&priv->notifier);
 
 	/* get this CSI's port id */
-	ret = fwnode_property_read_u32(fwnode, "reg", &port);
-	if (ret < 0)
-		goto out_free;
-
-	ret = v4l2_async_notifier_parse_fwnode_endpoints_by_port(
-		priv->dev->parent, notifier, sizeof(struct v4l2_async_subdev),
-		port, imx_csi_parse_endpoint);
+	ret = fwnode_property_read_u32(dev_fwnode(priv->dev), "reg", &port);
 	if (ret < 0)
-		goto out_cleanup;
+		return ret;
 
-	ret = v4l2_async_subdev_notifier_register(&priv->sd, notifier);
-	if (ret < 0)
-		goto out_cleanup;
+	ep = fwnode_graph_get_endpoint_by_id(dev_fwnode(priv->dev->parent),
+					     port, 0,
+					     FWNODE_GRAPH_ENDPOINT_NEXT);
+	if (ep) {
+		asd = kzalloc(sizeof(*asd), GFP_KERNEL);
+		if (!asd) {
+			fwnode_handle_put(ep);
+			return -ENOMEM;
+		}
 
-	ret = v4l2_async_register_subdev(&priv->sd);
-	if (ret < 0)
-		goto out_unregister;
+		ret = v4l2_async_notifier_add_fwnode_remote_subdev(
+			&priv->notifier, ep, asd);
 
-	priv->sd.subdev_notifier = notifier;
+		fwnode_handle_put(ep);
 
-	return 0;
+		if (ret) {
+			kfree(asd);
+			/* OK if asd already exists */
+			if (ret != -EEXIST)
+				return ret;
+		}
+	}
 
-out_unregister:
-	v4l2_async_notifier_unregister(notifier);
-out_cleanup:
-	v4l2_async_notifier_cleanup(notifier);
-out_free:
-	kfree(notifier);
+	ret = v4l2_async_subdev_notifier_register(&priv->sd,
+						  &priv->notifier);
+	if (ret)
+		return ret;
 
-	return ret;
+	return v4l2_async_register_subdev(&priv->sd);
 }
 
 static int imx_csi_probe(struct platform_device *pdev)
@@ -1996,9 +1988,13 @@ static int imx_csi_probe(struct platform_device *pdev)
 
 	ret = imx_csi_async_register(priv);
 	if (ret)
-		goto free;
+		goto cleanup;
 
 	return 0;
+
+cleanup:
+	v4l2_async_notifier_unregister(&priv->notifier);
+	v4l2_async_notifier_cleanup(&priv->notifier);
 free:
 	v4l2_ctrl_handler_free(&priv->ctrl_hdlr);
 	mutex_destroy(&priv->lock);
@@ -2012,6 +2008,8 @@ static int imx_csi_remove(struct platform_device *pdev)
 
 	v4l2_ctrl_handler_free(&priv->ctrl_hdlr);
 	mutex_destroy(&priv->lock);
+	v4l2_async_notifier_unregister(&priv->notifier);
+	v4l2_async_notifier_cleanup(&priv->notifier);
 	v4l2_async_unregister_subdev(sd);
 	media_entity_cleanup(&sd->entity);
 
drivers/staging/media/imx/imx6-mipi-csi2.c +49 -21

110 lines not on this page · raw message

drivers/staging/media/imx/imx7-media-csi.c +44 -9

86 lines not on this page · raw message

drivers/staging/media/imx/imx7-mipi-csis.c +61 -30

136 lines not on this page · raw message

Signature · 1 line
2.17.1
author maintainer
#5 lore raw

PATCH v4 4/17 Revert "media: v4l2-fwnode: Add a convenience function for registering subdevs with notifiers"

In reply to Steve Longerbeam

The users of v4l2_async_register_fwnode_subdev() have switched to parsing their endpoints and setting up async sub-device lists in their notifiers locally, without using the endpoint parsing callbacks. There are no more users of v4l2_async_register_fwnode_subdev() so this convenience function can be removed.

This reverts commit 1634f0eded87d1f150e823fa56cd782ea0775eb2.

---
 drivers/media/v4l2-core/v4l2-fwnode.c | 62 ---------------------------
 include/media/v4l2-fwnode.h           | 38 ----------------
 2 files changed, 100 deletions(-)
drivers/media/v4l2-core/v4l2-fwnode.c +0 -62
index 6ece4320e1d2..1dc18940ac0e 100644
@@ -1163,68 +1163,6 @@ int v4l2_async_register_subdev_sensor_common(struct v4l2_subdev *sd)
 }
 EXPORT_SYMBOL_GPL(v4l2_async_register_subdev_sensor_common);
 
-int v4l2_async_register_fwnode_subdev(struct v4l2_subdev *sd,
-				      size_t asd_struct_size,
-				      unsigned int *ports,
-				      unsigned int num_ports,
-				      parse_endpoint_func parse_endpoint)
-{
-	struct v4l2_async_notifier *notifier;
-	struct device *dev = sd->dev;
-	struct fwnode_handle *fwnode;
-	int ret;
-
-	if (WARN_ON(!dev))
-		return -ENODEV;
-
-	fwnode = dev_fwnode(dev);
-	if (!fwnode_device_is_available(fwnode))
-		return -ENODEV;
-
-	notifier = kzalloc(sizeof(*notifier), GFP_KERNEL);
-	if (!notifier)
-		return -ENOMEM;
-
-	v4l2_async_notifier_init(notifier);
-
-	if (!ports) {
-		ret = v4l2_async_notifier_parse_fwnode_endpoints(dev, notifier,
-								 asd_struct_size,
-								 parse_endpoint);
-		if (ret < 0)
-			goto out_cleanup;
-	} else {
-		unsigned int i;
-
-		for (i = 0; i < num_ports; i++) {
-			ret = v4l2_async_notifier_parse_fwnode_endpoints_by_port(dev, notifier, asd_struct_size, ports[i], parse_endpoint);
-			if (ret < 0)
-				goto out_cleanup;
-		}
-	}
-
-	ret = v4l2_async_subdev_notifier_register(sd, notifier);
-	if (ret < 0)
-		goto out_cleanup;
-
-	ret = v4l2_async_register_subdev(sd);
-	if (ret < 0)
-		goto out_unregister;
-
-	sd->subdev_notifier = notifier;
-
-	return 0;
-
-out_unregister:
-	v4l2_async_notifier_unregister(notifier);
-out_cleanup:
-	v4l2_async_notifier_cleanup(notifier);
-	kfree(notifier);
-
-	return ret;
-}
-EXPORT_SYMBOL_GPL(v4l2_async_register_fwnode_subdev);
-
 MODULE_LICENSE("GPL");
 MODULE_AUTHOR("Sakari Ailus <[email protected]>");
 MODULE_AUTHOR("Sylwester Nawrocki <[email protected]>");
include/media/v4l2-fwnode.h +0 -38
index f6a7bcd13197..caa854d2a867 100644
@@ -20,7 +20,6 @@
 #include <linux/types.h>
 
 #include <media/v4l2-mediabus.h>
-#include <media/v4l2-subdev.h>
 
 struct fwnode_handle;
 struct v4l2_async_notifier;
@@ -369,41 +368,4 @@ v4l2_async_notifier_parse_fwnode_endpoints_by_port(struct device *dev,
 int v4l2_async_notifier_parse_fwnode_sensor_common(struct device *dev,
 						   struct v4l2_async_notifier *notifier);
 
-/**
- * v4l2_async_register_fwnode_subdev - registers a sub-device to the
- *					asynchronous sub-device framework
- *					and parses fwnode endpoints
- *
- * @sd: pointer to struct &v4l2_subdev
- * @asd_struct_size: size of the driver's async sub-device struct, including
- *		     sizeof(struct v4l2_async_subdev). The &struct
- *		     v4l2_async_subdev shall be the first member of
- *		     the driver's async sub-device struct, i.e. both
- *		     begin at the same memory address.
- * @ports: array of port id's to parse for fwnode endpoints. If NULL, will
- *	   parse all ports owned by the sub-device.
- * @num_ports: number of ports in @ports array. Ignored if @ports is NULL.
- * @parse_endpoint: Driver's callback function called on each V4L2 fwnode
- *		    endpoint. Optional.
- *
- * This function is just like v4l2_async_register_subdev() with the
- * exception that calling it will also allocate a notifier for the
- * sub-device, parse the sub-device's firmware node endpoints using
- * v4l2_async_notifier_parse_fwnode_endpoints() or
- * v4l2_async_notifier_parse_fwnode_endpoints_by_port(), and
- * registers the sub-device notifier. The sub-device is similarly
- * unregistered by calling v4l2_async_unregister_subdev().
- *
- * While registered, the subdev module is marked as in-use.
- *
- * An error is returned if the module is no longer loaded on any attempts
- * to register it.
- */
-int
-v4l2_async_register_fwnode_subdev(struct v4l2_subdev *sd,
-				  size_t asd_struct_size,
-				  unsigned int *ports,
-				  unsigned int num_ports,
-				  parse_endpoint_func parse_endpoint);
-
 #endif /* _V4L2_FWNODE_H */
Signature · 1 line
2.17.1
author maintainer
#6 lore raw

PATCH v4 5/17 media: imx: csi: Implement get_fwnode_pad op

In reply to Steve Longerbeam

The CSI does not have a 1:1 relationship between fwnode port numbers and pad indexes. In fact the CSI fwnode device is itself a port which is the sink, containing only a single fwnode endpoint. Implement media_entity operation get_fwnode_pad to first verify the given endpoint is the CSI's sink endpoint, and if so return the CSI sink pad index.

---
 drivers/staging/media/imx/imx-media-csi.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
drivers/staging/media/imx/imx-media-csi.c +23 -0
index f409fca88dcf..35f2512ed2a9 100644
@@ -1827,9 +1827,32 @@ static void csi_unregistered(struct v4l2_subdev *sd)
 		ipu_csi_put(priv->csi);
 }
 
+/*
+ * The CSI has only one fwnode endpoint, at the sink pad. Verify the
+ * endpoint belongs to us, and return CSI_SINK_PAD.
+ */
+static int csi_get_fwnode_pad(struct media_entity *entity,
+			      struct fwnode_endpoint *endpoint)
+{
+	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+	struct csi_priv *priv = v4l2_get_subdevdata(sd);
+	struct fwnode_handle *csi_port = dev_fwnode(priv->dev);
+	struct fwnode_handle *csi_ep;
+	int ret;
+
+	csi_ep = fwnode_get_next_child_node(csi_port, NULL);
+
+	ret = endpoint->local_fwnode == csi_ep ? CSI_SINK_PAD : -ENXIO;
+
+	fwnode_handle_put(csi_ep);
+
+	return ret;
+}
+
 static const struct media_entity_operations csi_entity_ops = {
 	.link_setup = csi_link_setup,
 	.link_validate = v4l2_subdev_link_validate,
+	.get_fwnode_pad = csi_get_fwnode_pad,
 };
 
 static const struct v4l2_subdev_core_ops csi_core_ops = {
Signature · 1 line
2.17.1
maintainer Reviewed-by on v4 5/17
#7 lore raw

Re: [PATCH v4 05/17] media: imx: csi: Implement get_fwnode_pad op

Hi Steve,

Thank you for the patch.

Quoted 7 lines · Steve Longerbeam

The CSI does not have a 1:1 relationship between fwnode port numbers and pad indexes. In fact the CSI fwnode device is itself a port which is the sink, containing only a single fwnode endpoint. Implement media_entity operation get_fwnode_pad to first verify the given endpoint is the CSI's sink endpoint, and if so return the CSI sink pad index.

Quoted 41 lines
---
 drivers/staging/media/imx/imx-media-csi.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
drivers/staging/media/imx/imx-media-csi.c +23 -0
index f409fca88dcf..35f2512ed2a9 100644
@@ -1827,9 +1827,32 @@ static void csi_unregistered(struct v4l2_subdev *sd)
 		ipu_csi_put(priv->csi);
 }
 
+/*
+ * The CSI has only one fwnode endpoint, at the sink pad. Verify the
+ * endpoint belongs to us, and return CSI_SINK_PAD.
+ */
+static int csi_get_fwnode_pad(struct media_entity *entity,
+			      struct fwnode_endpoint *endpoint)
+{
+	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+	struct csi_priv *priv = v4l2_get_subdevdata(sd);
+	struct fwnode_handle *csi_port = dev_fwnode(priv->dev);
+	struct fwnode_handle *csi_ep;
+	int ret;
+
+	csi_ep = fwnode_get_next_child_node(csi_port, NULL);
+
+	ret = endpoint->local_fwnode == csi_ep ? CSI_SINK_PAD : -ENXIO;
+
+	fwnode_handle_put(csi_ep);
+
+	return ret;
+}
+
 static const struct media_entity_operations csi_entity_ops = {
 	.link_setup = csi_link_setup,
 	.link_validate = v4l2_subdev_link_validate,
+	.get_fwnode_pad = csi_get_fwnode_pad,
 };
 
 static const struct v4l2_subdev_core_ops csi_core_ops = {
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#8 lore raw

PATCH v4 6/17 media: imx: mipi csi-2: Implement get_fwnode_pad op

In reply to Steve Longerbeam

Implement get_fwnode_pad operation. If the endpoint is owned by the MIPI CSI-2 receiver, return the endpoint's port number. The MIPI CSI-2 receiver maps port numbers and pad indexes 1:1.

---
 drivers/staging/media/imx/imx6-mipi-csi2.c | 28 ++++++++++++++++++++++
 1 file changed, 28 insertions(+)
drivers/staging/media/imx/imx6-mipi-csi2.c +28 -0
index fdd763587e6c..8500207e5ea9 100644
@@ -507,9 +507,37 @@ static int csi2_registered(struct v4l2_subdev *sd)
 				      640, 480, 0, V4L2_FIELD_NONE, NULL);
 }
 
+static int csi2_get_fwnode_pad(struct media_entity *entity,
+			       struct fwnode_endpoint *endpoint)
+{
+	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+	struct csi2_dev *csi2 = sd_to_dev(sd);
+	struct fwnode_handle *csi2_ep;
+
+	/*
+	 * If the endpoint is one of ours, return the endpoint's port
+	 * number. This device maps port numbers and pad indexes 1:1.
+	 */
+	fwnode_graph_for_each_endpoint(dev_fwnode(csi2->dev), csi2_ep) {
+		if (endpoint->local_fwnode == csi2_ep) {
+			struct fwnode_endpoint fwep;
+			int ret;
+
+			ret = fwnode_graph_parse_endpoint(csi2_ep, &fwep);
+
+			fwnode_handle_put(csi2_ep);
+
+			return ret ? ret : fwep.port;
+		}
+	}
+
+	return -ENXIO;
+}
+
 static const struct media_entity_operations csi2_entity_ops = {
 	.link_setup = csi2_link_setup,
 	.link_validate = v4l2_subdev_link_validate,
+	.get_fwnode_pad = csi2_get_fwnode_pad,
 };
 
 static const struct v4l2_subdev_video_ops csi2_video_ops = {
Signature · 1 line
2.17.1
maintainer
#9 lore raw

Re: [PATCH v4 06/17] media: imx: mipi csi-2: Implement get_fwnode_pad op

Hi Steve,

Thank you for the patch.

Quoted 33 lines · Steve Longerbeam

Implement get_fwnode_pad operation. If the endpoint is owned by the MIPI CSI-2 receiver, return the endpoint's port number. The MIPI CSI-2 receiver maps port numbers and pad indexes 1:1.

---
 drivers/staging/media/imx/imx6-mipi-csi2.c | 28 ++++++++++++++++++++++
 1 file changed, 28 insertions(+)
drivers/staging/media/imx/imx6-mipi-csi2.c +16 -0
index fdd763587e6c..8500207e5ea9 100644
@@ -507,9 +507,37 @@ static int csi2_registered(struct v4l2_subdev *sd)
 				      640, 480, 0, V4L2_FIELD_NONE, NULL);
 }
 
+static int csi2_get_fwnode_pad(struct media_entity *entity,
+			       struct fwnode_endpoint *endpoint)
+{
+	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+	struct csi2_dev *csi2 = sd_to_dev(sd);
+	struct fwnode_handle *csi2_ep;
+
+	/*
+	 * If the endpoint is one of ours, return the endpoint's port
+	 * number. This device maps port numbers and pad indexes 1:1.
+	 */
+	fwnode_graph_for_each_endpoint(dev_fwnode(csi2->dev), csi2_ep) {
+		if (endpoint->local_fwnode == csi2_ep) {
+			struct fwnode_endpoint fwep;
+			int ret;
+

drivers/staging/media/imx/imx6-mipi-csi2.c static int csi2_registered(struct v4l2_subdev *sd) · line 535

Quoted 10 lines
drivers/staging/media/imx/imx6-mipi-csi2.c +10 -0
+			ret = fwnode_graph_parse_endpoint(csi2_ep, &fwep);
+
+			fwnode_handle_put(csi2_ep);
+
+			return ret ? ret : fwep.port;
+		}
+	}
+
+	return -ENXIO;
+}

As the 1:1 mapping is the common case, would it make sense to modify media_entity_get_fwnode_pad() accordingly when .get_fwnode_pad is not set ? The current behaviour is to return the first pad that matches the requested direction, which could be preserved as a second-level fallback if the 1:1 mapping doesn't give the right direction (but I'm not sure there's a use case for that, the 1:1 mapping seems to be all we need if there's no specific .get_fwnode_pad implementation).

Quoted 8 lines
drivers/staging/media/imx/imx6-mipi-csi2.c +2 -0
+
 static const struct media_entity_operations csi2_entity_ops = {
 	.link_setup = csi2_link_setup,
 	.link_validate = v4l2_subdev_link_validate,
+	.get_fwnode_pad = csi2_get_fwnode_pad,
 };
 
 static const struct v4l2_subdev_video_ops csi2_video_ops = {
Signature · 3 lines
Regards,

Laurent Pinchart
maintainer
#10 lore raw

Re: [PATCH v4 06/17] media: imx: mipi csi-2: Implement get_fwnode_pad op

Hi Laurent,

Quoted 44 lines · Laurent Pinchart

Hi Steve,

Thank you for the patch.

Quoted 39 lines · Steve Longerbeam

Implement get_fwnode_pad operation. If the endpoint is owned by the MIPI CSI-2 receiver, return the endpoint's port number. The MIPI CSI-2 receiver maps port numbers and pad indexes 1:1.

---
 drivers/staging/media/imx/imx6-mipi-csi2.c | 28 ++++++++++++++++++++++
 1 file changed, 28 insertions(+)
drivers/staging/media/imx/imx6-mipi-csi2.c +22 -0
index fdd763587e6c..8500207e5ea9 100644
@@ -507,9 +507,37 @@ static int csi2_registered(struct v4l2_subdev *sd)
 				      640, 480, 0, V4L2_FIELD_NONE, NULL);
 }
 
+static int csi2_get_fwnode_pad(struct media_entity *entity,
+			       struct fwnode_endpoint *endpoint)
+{
+	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+	struct csi2_dev *csi2 = sd_to_dev(sd);
+	struct fwnode_handle *csi2_ep;
+
+	/*
+	 * If the endpoint is one of ours, return the endpoint's port
+	 * number. This device maps port numbers and pad indexes 1:1.
+	 */
+	fwnode_graph_for_each_endpoint(dev_fwnode(csi2->dev), csi2_ep) {
+		if (endpoint->local_fwnode == csi2_ep) {
+			struct fwnode_endpoint fwep;
+			int ret;
+
+			ret = fwnode_graph_parse_endpoint(csi2_ep, &fwep);
+
+			fwnode_handle_put(csi2_ep);
+
+			return ret ? ret : fwep.port;
+		}
Quoted 8 lines

drivers/staging/media/imx/imx6-mipi-csi2.c static int csi2_registered(struct v4l2_subdev *sd) · line 535

Quoted 4 lines
drivers/staging/media/imx/imx6-mipi-csi2.c +4 -0
+	}
+
+	return -ENXIO;
+}

As the 1:1 mapping is the common case, would it make sense to modify media_entity_get_fwnode_pad() accordingly when .get_fwnode_pad is not set ? The current behaviour is to return the first pad that matches the

I also think this would make sense.

Quoted 4 lines

requested direction, which could be preserved as a second-level fallback if the 1:1 mapping doesn't give the right direction (but I'm not sure there's a use case for that, the 1:1 mapping seems to be all we need if there's no specific .get_fwnode_pad implementation).

I believe at least the smiapp driver breaks if you do that, so the current behaviour should be retained (secondary to the 1:1 mapping).

Signature · 3 lines
Regards,

Sakari Ailus
maintainer
#11 lore raw

Re: [PATCH v4 06/17] media: imx: mipi csi-2: Implement get_fwnode_pad op

Hi Sakari,

Quoted 52 lines · Sakari Ailus
Quoted 40 lines · Laurent Pinchart
Quoted 39 lines · Steve Longerbeam

Implement get_fwnode_pad operation. If the endpoint is owned by the MIPI CSI-2 receiver, return the endpoint's port number. The MIPI CSI-2 receiver maps port numbers and pad indexes 1:1.

---
 drivers/staging/media/imx/imx6-mipi-csi2.c | 28 ++++++++++++++++++++++
 1 file changed, 28 insertions(+)
drivers/staging/media/imx/imx6-mipi-csi2.c +22 -0
index fdd763587e6c..8500207e5ea9 100644
@@ -507,9 +507,37 @@ static int csi2_registered(struct v4l2_subdev *sd)
 				      640, 480, 0, V4L2_FIELD_NONE, NULL);
 }
 
+static int csi2_get_fwnode_pad(struct media_entity *entity,
+			       struct fwnode_endpoint *endpoint)
+{
+	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+	struct csi2_dev *csi2 = sd_to_dev(sd);
+	struct fwnode_handle *csi2_ep;
+
+	/*
+	 * If the endpoint is one of ours, return the endpoint's port
+	 * number. This device maps port numbers and pad indexes 1:1.
+	 */
+	fwnode_graph_for_each_endpoint(dev_fwnode(csi2->dev), csi2_ep) {
+		if (endpoint->local_fwnode == csi2_ep) {
+			struct fwnode_endpoint fwep;
+			int ret;
+
+			ret = fwnode_graph_parse_endpoint(csi2_ep, &fwep);
+
+			fwnode_handle_put(csi2_ep);
+
+			return ret ? ret : fwep.port;
+		}
Quoted 8 lines

drivers/staging/media/imx/imx6-mipi-csi2.c static int csi2_registered(struct v4l2_subdev *sd) · line 535

Quoted 4 lines
drivers/staging/media/imx/imx6-mipi-csi2.c +4 -0
+	}
+
+	return -ENXIO;
+}

As the 1:1 mapping is the common case, would it make sense to modify media_entity_get_fwnode_pad() accordingly when .get_fwnode_pad is not set ? The current behaviour is to return the first pad that matches the

I also think this would make sense.

Quoted 7 lines
Quoted 4 lines

requested direction, which could be preserved as a second-level fallback if the 1:1 mapping doesn't give the right direction (but I'm not sure there's a use case for that, the 1:1 mapping seems to be all we need if there's no specific .get_fwnode_pad implementation).

I believe at least the smiapp driver breaks if you do that, so the current behaviour should be retained (secondary to the 1:1 mapping).

Shouldn't the smiapp driver get a .get_fwnode_pad() implementation then ? It could be argued that the smiapp use case is also quite common, and would deserve handling in the core, as a second fallback. It makes me wonder if we should really try to have a core fallback in the first place then, we could instead create helpers for the common cases that would be used by drivers as their .get_fwnode_pad() implementation, forcing each driver to think about which version matches its needs best.

Signature · 3 lines
Regards,

Laurent Pinchart
maintainer
#12 lore raw

Re: [PATCH v4 06/17] media: imx: mipi csi-2: Implement get_fwnode_pad op

Hi Laurent,

Quoted 63 lines · Laurent Pinchart

Hi Sakari,

Quoted 52 lines · Sakari Ailus
Quoted 40 lines · Laurent Pinchart
Quoted 39 lines · Steve Longerbeam

Implement get_fwnode_pad operation. If the endpoint is owned by the MIPI CSI-2 receiver, return the endpoint's port number. The MIPI CSI-2 receiver maps port numbers and pad indexes 1:1.

---
 drivers/staging/media/imx/imx6-mipi-csi2.c | 28 ++++++++++++++++++++++
 1 file changed, 28 insertions(+)
drivers/staging/media/imx/imx6-mipi-csi2.c +22 -0
index fdd763587e6c..8500207e5ea9 100644
@@ -507,9 +507,37 @@ static int csi2_registered(struct v4l2_subdev *sd)
 				      640, 480, 0, V4L2_FIELD_NONE, NULL);
 }
 
+static int csi2_get_fwnode_pad(struct media_entity *entity,
+			       struct fwnode_endpoint *endpoint)
+{
+	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+	struct csi2_dev *csi2 = sd_to_dev(sd);
+	struct fwnode_handle *csi2_ep;
+
+	/*
+	 * If the endpoint is one of ours, return the endpoint's port
+	 * number. This device maps port numbers and pad indexes 1:1.
+	 */
+	fwnode_graph_for_each_endpoint(dev_fwnode(csi2->dev), csi2_ep) {
+		if (endpoint->local_fwnode == csi2_ep) {
+			struct fwnode_endpoint fwep;
+			int ret;
+
+			ret = fwnode_graph_parse_endpoint(csi2_ep, &fwep);
+
+			fwnode_handle_put(csi2_ep);
+
+			return ret ? ret : fwep.port;
+		}
Quoted 8 lines

drivers/staging/media/imx/imx6-mipi-csi2.c static int csi2_registered(struct v4l2_subdev *sd) · line 535

Quoted 4 lines
drivers/staging/media/imx/imx6-mipi-csi2.c +4 -0
+	}
+
+	return -ENXIO;
+}

As the 1:1 mapping is the common case, would it make sense to modify media_entity_get_fwnode_pad() accordingly when .get_fwnode_pad is not set ? The current behaviour is to return the first pad that matches the

I also think this would make sense.

Quoted 7 lines
Quoted 4 lines

requested direction, which could be preserved as a second-level fallback if the 1:1 mapping doesn't give the right direction (but I'm not sure there's a use case for that, the 1:1 mapping seems to be all we need if there's no specific .get_fwnode_pad implementation).

I believe at least the smiapp driver breaks if you do that, so the current behaviour should be retained (secondary to the 1:1 mapping).

Quoted 7 lines

Shouldn't the smiapp driver get a .get_fwnode_pad() implementation then ? It could be argued that the smiapp use case is also quite common, and would deserve handling in the core, as a second fallback. It makes me wonder if we should really try to have a core fallback in the first place then, we could instead create helpers for the common cases that would be used by drivers as their .get_fwnode_pad() implementation, forcing each driver to think about which version matches its needs best.

I guess this could be a little cleaner with explicit helpers set by drives. I'm not against that, but at the same time I don't think this is an area the cleaning of which would significantly help something.

Signature · 3 lines
Regards,

Sakari Ailus
author maintainer
#13 lore raw

Re: [PATCH v4 06/17] media: imx: mipi csi-2: Implement get_fwnode_pad op

In reply to Sakari Ailus

Hi Sakari, Laurent,

Quoted 54 lines · Sakari Ailus

Hi Laurent,

Quoted 43 lines · Laurent Pinchart

Hi Steve,

Thank you for the patch.

Quoted 38 lines · Steve Longerbeam

Implement get_fwnode_pad operation. If the endpoint is owned by the MIPI CSI-2 receiver, return the endpoint's port number. The MIPI CSI-2 receiver maps port numbers and pad indexes 1:1.

---
  drivers/staging/media/imx/imx6-mipi-csi2.c | 28 ++++++++++++++++++++++
  1 file changed, 28 insertions(+)
drivers/staging/media/imx/imx6-mipi-csi2.c +21 -0
index fdd763587e6c..8500207e5ea9 100644
@@ -507,9 +507,37 @@ static int csi2_registered(struct v4l2_subdev *sd)
  				      640, 480, 0, V4L2_FIELD_NONE, NULL);
  }
 
+static int csi2_get_fwnode_pad(struct media_entity *entity,
+			       struct fwnode_endpoint *endpoint)
+{
+	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+	struct csi2_dev *csi2 = sd_to_dev(sd);
+	struct fwnode_handle *csi2_ep;
+
+	/*
+	 * If the endpoint is one of ours, return the endpoint's port
+	 * number. This device maps port numbers and pad indexes 1:1.
+	 */
+	fwnode_graph_for_each_endpoint(dev_fwnode(csi2->dev), csi2_ep) {
+		if (endpoint->local_fwnode == csi2_ep) {
+			struct fwnode_endpoint fwep;
+			int ret;
+
+			ret = fwnode_graph_parse_endpoint(csi2_ep, &fwep);
+
+			fwnode_handle_put(csi2_ep);
+
+			return ret ? ret : fwep.port;
Quoted 8 lines

drivers/staging/media/imx/imx6-mipi-csi2.c static int csi2_registered(struct v4l2_subdev *sd) · line 535

Quoted 5 lines
drivers/staging/media/imx/imx6-mipi-csi2.c +5 -0
+		}
+	}
+
+	return -ENXIO;
+}

As the 1:1 mapping is the common case, would it make sense to modify media_entity_get_fwnode_pad() accordingly when .get_fwnode_pad is not set ? The current behaviour is to return the first pad that matches the

Quoted 1 line

I also think this would make sense.

What do you think about https://patchwork.linuxtv.org/patch/60312/ ? I'm planning to resurrect it for v5.

Steve

Quoted 7 lines
Quoted 4 lines

requested direction, which could be preserved as a second-level fallback if the 1:1 mapping doesn't give the right direction (but I'm not sure there's a use case for that, the 1:1 mapping seems to be all we need if there's no specific .get_fwnode_pad implementation).

I believe at least the smiapp driver breaks if you do that, so the current behaviour should be retained (secondary to the 1:1 mapping).

maintainer
#14 lore raw

Re: [PATCH v4 06/17] media: imx: mipi csi-2: Implement get_fwnode_pad op

Hi Steve,

Quoted 51 lines · Steve Longerbeam
Quoted 50 lines · Sakari Ailus
Quoted 48 lines · Laurent Pinchart
Quoted 33 lines · Steve Longerbeam

Implement get_fwnode_pad operation. If the endpoint is owned by the MIPI CSI-2 receiver, return the endpoint's port number. The MIPI CSI-2 receiver maps port numbers and pad indexes 1:1.

---
  drivers/staging/media/imx/imx6-mipi-csi2.c | 28 ++++++++++++++++++++++
  1 file changed, 28 insertions(+)
drivers/staging/media/imx/imx6-mipi-csi2.c +16 -0
index fdd763587e6c..8500207e5ea9 100644
@@ -507,9 +507,37 @@ static int csi2_registered(struct v4l2_subdev *sd)
  				      640, 480, 0, V4L2_FIELD_NONE, NULL);
  }
 
+static int csi2_get_fwnode_pad(struct media_entity *entity,
+			       struct fwnode_endpoint *endpoint)
+{
+	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+	struct csi2_dev *csi2 = sd_to_dev(sd);
+	struct fwnode_handle *csi2_ep;
+
+	/*
+	 * If the endpoint is one of ours, return the endpoint's port
+	 * number. This device maps port numbers and pad indexes 1:1.
+	 */
+	fwnode_graph_for_each_endpoint(dev_fwnode(csi2->dev), csi2_ep) {
+		if (endpoint->local_fwnode == csi2_ep) {
+			struct fwnode_endpoint fwep;
+			int ret;
+

drivers/staging/media/imx/imx6-mipi-csi2.c static int csi2_registered(struct v4l2_subdev *sd) · line 535

Quoted 10 lines
drivers/staging/media/imx/imx6-mipi-csi2.c +10 -0
+			ret = fwnode_graph_parse_endpoint(csi2_ep, &fwep);
+
+			fwnode_handle_put(csi2_ep);
+
+			return ret ? ret : fwep.port;
+		}
+	}
+
+	return -ENXIO;
+}

As the 1:1 mapping is the common case, would it make sense to modify media_entity_get_fwnode_pad() accordingly when .get_fwnode_pad is not set ? The current behaviour is to return the first pad that matches the

Quoted 4 lines
Quoted 1 line

I also think this would make sense.

What do you think about https://patchwork.linuxtv.org/patch/60312/ ? I'm planning to resurrect it for v5.

The approach looks good to me.

Quoted 7 lines
Quoted 7 lines
Quoted 4 lines

requested direction, which could be preserved as a second-level fallback if the 1:1 mapping doesn't give the right direction (but I'm not sure there's a use case for that, the 1:1 mapping seems to be all we need if there's no specific .get_fwnode_pad implementation).

I believe at least the smiapp driver breaks if you do that, so the current behaviour should be retained (secondary to the 1:1 mapping).

Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#15 lore raw

Re: [PATCH v4 06/17] media: imx: mipi csi-2: Implement get_fwnode_pad op

In reply to Laurent Pinchart

Hi Laurent,

Quoted 43 lines · Laurent Pinchart

Hi Steve,

Thank you for the patch.

Quoted 38 lines · Steve Longerbeam

Implement get_fwnode_pad operation. If the endpoint is owned by the MIPI CSI-2 receiver, return the endpoint's port number. The MIPI CSI-2 receiver maps port numbers and pad indexes 1:1.

---
  drivers/staging/media/imx/imx6-mipi-csi2.c | 28 ++++++++++++++++++++++
  1 file changed, 28 insertions(+)
drivers/staging/media/imx/imx6-mipi-csi2.c +21 -0
index fdd763587e6c..8500207e5ea9 100644
@@ -507,9 +507,37 @@ static int csi2_registered(struct v4l2_subdev *sd)
  				      640, 480, 0, V4L2_FIELD_NONE, NULL);
  }
 
+static int csi2_get_fwnode_pad(struct media_entity *entity,
+			       struct fwnode_endpoint *endpoint)
+{
+	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+	struct csi2_dev *csi2 = sd_to_dev(sd);
+	struct fwnode_handle *csi2_ep;
+
+	/*
+	 * If the endpoint is one of ours, return the endpoint's port
+	 * number. This device maps port numbers and pad indexes 1:1.
+	 */
+	fwnode_graph_for_each_endpoint(dev_fwnode(csi2->dev), csi2_ep) {
+		if (endpoint->local_fwnode == csi2_ep) {
+			struct fwnode_endpoint fwep;
+			int ret;
+
+			ret = fwnode_graph_parse_endpoint(csi2_ep, &fwep);
+
+			fwnode_handle_put(csi2_ep);
+
+			return ret ? ret : fwep.port;
Quoted 8 lines

drivers/staging/media/imx/imx6-mipi-csi2.c static int csi2_registered(struct v4l2_subdev *sd) · line 535

Quoted 5 lines
drivers/staging/media/imx/imx6-mipi-csi2.c +5 -0
+		}
+	}
+
+	return -ENXIO;
+}

As the 1:1 mapping is the common case, would it make sense to modify media_entity_get_fwnode_pad() accordingly when .get_fwnode_pad is not set ?

Yes! In fact that was a previous revision of this patchset (v2). But I dropped that idea in v3 and v4 because it didn't seem to be getting any traction.

I guess I'll try again. I'll bring back [1] in v5.

Steve

[1] https://patchwork.linuxtv.org/patch/60312/

Quoted 14 lines

The current behaviour is to return the first pad that matches the
requested direction, which could be preserved as a second-level fallback if the 1:1 mapping doesn't give the right direction (but I'm not sure there's a use case for that, the 1:1 mapping seems to be all we need if there's no specific .get_fwnode_pad implementation).

Quoted 8 lines
drivers/staging/media/imx/imx6-mipi-csi2.c +2 -0
+
  static const struct media_entity_operations csi2_entity_ops = {
  	.link_setup = csi2_link_setup,
  	.link_validate = v4l2_subdev_link_validate,
+	.get_fwnode_pad = csi2_get_fwnode_pad,
  };
 
  static const struct v4l2_subdev_video_ops csi2_video_ops = {
author maintainer
#16 lore raw

PATCH v4 7/17 media: video-mux: Implement get_fwnode_pad op

In reply to Steve Longerbeam

Implement get_fwnode_pad operation. If the endpoint is owned by the video mux, return the endpoint's port number. The video mux maps fwnode port numbers and pad indexes 1:1.

---
 drivers/media/platform/video-mux.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)
drivers/media/platform/video-mux.c +29 -0
index 7b6c96a29aa5..f446ada82176 100644
@@ -94,9 +94,38 @@ static int video_mux_link_setup(struct media_entity *entity,
 	return ret;
 }
 
+static int video_mux_get_fwnode_pad(struct media_entity *entity,
+				    struct fwnode_endpoint *endpoint)
+{
+	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+	struct video_mux *vmux = v4l2_subdev_to_video_mux(sd);
+	struct fwnode_handle *vmux_fwnode = dev_fwnode(vmux->subdev.dev);
+	struct fwnode_handle *vmux_ep;
+
+	/*
+	 * If the endpoint is one of ours, return the endpoint's port
+	 * number. This device maps port numbers and pad indexes 1:1.
+	 */
+	fwnode_graph_for_each_endpoint(vmux_fwnode, vmux_ep) {
+		if (endpoint->local_fwnode == vmux_ep) {
+			struct fwnode_endpoint fwep;
+			int ret;
+
+			ret = fwnode_graph_parse_endpoint(vmux_ep, &fwep);
+
+			fwnode_handle_put(vmux_ep);
+
+			return ret ? ret : fwep.port;
+		}
+	}
+
+	return -ENXIO;
+}
+
 static const struct media_entity_operations video_mux_ops = {
 	.link_setup = video_mux_link_setup,
 	.link_validate = v4l2_subdev_link_validate,
+	.get_fwnode_pad = video_mux_get_fwnode_pad,
 };
 
 static int video_mux_s_stream(struct v4l2_subdev *sd, int enable)
Signature · 1 line
2.17.1
maintainer
#17 lore raw

Re: [PATCH v4 07/17] media: video-mux: Implement get_fwnode_pad op

Hi Steve,

Thank you for the patch.

Quoted 3 lines · Steve Longerbeam

Implement get_fwnode_pad operation. If the endpoint is owned by the video mux, return the endpoint's port number. The video mux maps fwnode port numbers and pad indexes 1:1.

If you follow my suggestion from 06/17, this patch could be dropped.

Quoted 48 lines
---
 drivers/media/platform/video-mux.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)
drivers/media/platform/video-mux.c +29 -0
index 7b6c96a29aa5..f446ada82176 100644
@@ -94,9 +94,38 @@ static int video_mux_link_setup(struct media_entity *entity,
 	return ret;
 }
 
+static int video_mux_get_fwnode_pad(struct media_entity *entity,
+				    struct fwnode_endpoint *endpoint)
+{
+	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+	struct video_mux *vmux = v4l2_subdev_to_video_mux(sd);
+	struct fwnode_handle *vmux_fwnode = dev_fwnode(vmux->subdev.dev);
+	struct fwnode_handle *vmux_ep;
+
+	/*
+	 * If the endpoint is one of ours, return the endpoint's port
+	 * number. This device maps port numbers and pad indexes 1:1.
+	 */
+	fwnode_graph_for_each_endpoint(vmux_fwnode, vmux_ep) {
+		if (endpoint->local_fwnode == vmux_ep) {
+			struct fwnode_endpoint fwep;
+			int ret;
+
+			ret = fwnode_graph_parse_endpoint(vmux_ep, &fwep);
+
+			fwnode_handle_put(vmux_ep);
+
+			return ret ? ret : fwep.port;
+		}
+	}
+
+	return -ENXIO;
+}
+
 static const struct media_entity_operations video_mux_ops = {
 	.link_setup = video_mux_link_setup,
 	.link_validate = v4l2_subdev_link_validate,
+	.get_fwnode_pad = video_mux_get_fwnode_pad,
 };
 
 static int video_mux_s_stream(struct v4l2_subdev *sd, int enable)
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#18 lore raw

Re: [PATCH v4 07/17] media: video-mux: Implement get_fwnode_pad op

Quoted 9 lines · Laurent Pinchart

Hi Steve,

Thank you for the patch.

Quoted 3 lines · Steve Longerbeam

Implement get_fwnode_pad operation. If the endpoint is owned by the video mux, return the endpoint's port number. The video mux maps fwnode port numbers and pad indexes 1:1.

If you follow my suggestion from 06/17, this patch could be dropped.

Agreed.

Steve

Quoted 49 lines
Quoted 49 lines
---
  drivers/media/platform/video-mux.c | 29 +++++++++++++++++++++++++++++
  1 file changed, 29 insertions(+)
drivers/media/platform/video-mux.c +29 -0
index 7b6c96a29aa5..f446ada82176 100644
@@ -94,9 +94,38 @@ static int video_mux_link_setup(struct media_entity *entity,
  	return ret;
  }
 
+static int video_mux_get_fwnode_pad(struct media_entity *entity,
+				    struct fwnode_endpoint *endpoint)
+{
+	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+	struct video_mux *vmux = v4l2_subdev_to_video_mux(sd);
+	struct fwnode_handle *vmux_fwnode = dev_fwnode(vmux->subdev.dev);
+	struct fwnode_handle *vmux_ep;
+
+	/*
+	 * If the endpoint is one of ours, return the endpoint's port
+	 * number. This device maps port numbers and pad indexes 1:1.
+	 */
+	fwnode_graph_for_each_endpoint(vmux_fwnode, vmux_ep) {
+		if (endpoint->local_fwnode == vmux_ep) {
+			struct fwnode_endpoint fwep;
+			int ret;
+
+			ret = fwnode_graph_parse_endpoint(vmux_ep, &fwep);
+
+			fwnode_handle_put(vmux_ep);
+
+			return ret ? ret : fwep.port;
+		}
+	}
+
+	return -ENXIO;
+}
+
  static const struct media_entity_operations video_mux_ops = {
  	.link_setup = video_mux_link_setup,
  	.link_validate = v4l2_subdev_link_validate,
+	.get_fwnode_pad = video_mux_get_fwnode_pad,
  };
 
  static int video_mux_s_stream(struct v4l2_subdev *sd, int enable)
author maintainer
#19 lore raw

PATCH v4 8/17 media: imx: Add imx_media_create_fwnode_pad_link()

In reply to Steve Longerbeam

Add functions to create media links between a source and sink subdevice based on fwnode endpoint connections between them.

---
 drivers/staging/media/imx/imx-media-utils.c | 91 +++++++++++++++++++++
 drivers/staging/media/imx/imx-media.h       |  4 +
 2 files changed, 95 insertions(+)
drivers/staging/media/imx/imx-media-utils.c +91 -0
index 0788a1874557..87152bd9af22 100644
@@ -729,6 +729,97 @@ void imx_media_grp_id_to_sd_name(char *sd_name, int sz, u32 grp_id, int ipu_id)
 }
 EXPORT_SYMBOL_GPL(imx_media_grp_id_to_sd_name);
 
+/*
+ * Look for and create a single fwnode link that connects a source
+ * subdevice to a sink pad.
+ */
+int imx_media_create_fwnode_pad_link(struct v4l2_subdev *src_sd,
+				     struct media_pad *sink)
+{
+	struct fwnode_handle *endpoint;
+
+	/* loop thru the source's fwnode endpoints */
+	fwnode_graph_for_each_endpoint(dev_fwnode(src_sd->dev), endpoint) {
+		struct fwnode_handle *remote_ep;
+		int src_idx, sink_idx, ret;
+		struct media_pad *src;
+
+		remote_ep = fwnode_graph_get_remote_endpoint(endpoint);
+		if (!remote_ep)
+			continue;
+
+		/*
+		 * ask the sink entity to verify that this fwnode link
+		 * actually does connect with the entity, and if so that
+		 * it connects to its requested sink pad.
+		 */
+		sink_idx = media_entity_get_fwnode_pad(sink->entity,
+						       remote_ep,
+						       MEDIA_PAD_FL_SINK);
+		fwnode_handle_put(remote_ep);
+
+		if (sink_idx < 0 || sink_idx != sink->index)
+			continue;
+
+		src_idx = media_entity_get_fwnode_pad(&src_sd->entity,
+						      endpoint,
+						      MEDIA_PAD_FL_SOURCE);
+		if (src_idx < 0)
+			continue;
+
+		/*
+		 * found the fwnode link that works, create the media
+		 * link for it.
+		 */
+
+		fwnode_handle_put(endpoint);
+
+		src = &src_sd->entity.pads[src_idx];
+
+		/* success if it already exists */
+		if (media_entity_find_link(src, sink))
+			return 0;
+
+		dev_dbg(src_sd->dev, "%s:%d -> %s:%d\n",
+			src_sd->entity.name, src_idx,
+			sink->entity->name, sink_idx);
+
+		ret = media_create_pad_link(&src_sd->entity, src_idx,
+					    sink->entity, sink_idx, 0);
+		if (ret)
+			dev_err(src_sd->dev,
+				"%s:%d -> %s:%d failed with %d\n",
+				src_sd->entity.name, src_idx,
+				sink->entity->name, sink_idx, ret);
+
+		return ret;
+	}
+
+	return -ENXIO;
+}
+EXPORT_SYMBOL_GPL(imx_media_create_fwnode_pad_link);
+
+int imx_media_create_fwnode_pad_links(struct v4l2_subdev *src_sd,
+				      struct v4l2_subdev *sink_sd)
+{
+	int i;
+
+	for (i = 0; i < sink_sd->entity.num_pads; i++) {
+		struct media_pad *pad = &sink_sd->entity.pads[i];
+		int ret;
+
+		if (!(pad->flags & MEDIA_PAD_FL_SINK))
+			continue;
+
+		ret = imx_media_create_fwnode_pad_link(src_sd, pad);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(imx_media_create_fwnode_pad_links);
+
 struct v4l2_subdev *
 imx_media_find_subdev_by_fwnode(struct imx_media_dev *imxmd,
 				struct fwnode_handle *fwnode)
drivers/staging/media/imx/imx-media.h +4 -0
index 11861191324a..f90a65ba4ced 100644
@@ -183,6 +183,10 @@ int imx_media_ipu_image_to_mbus_fmt(struct v4l2_mbus_framefmt *mbus,
 				    struct ipu_image *image);
 void imx_media_grp_id_to_sd_name(char *sd_name, int sz,
 				 u32 grp_id, int ipu_id);
+int imx_media_create_fwnode_pad_link(struct v4l2_subdev *src_sd,
+				     struct media_pad *sink);
+int imx_media_create_fwnode_pad_links(struct v4l2_subdev *src_sd,
+				      struct v4l2_subdev *sink_sd);
 struct v4l2_subdev *
 imx_media_find_subdev_by_fwnode(struct imx_media_dev *imxmd,
 				struct fwnode_handle *fwnode);
Signature · 1 line
2.17.1
maintainer
#20 lore raw

Re: [PATCH v4 08/17] media: imx: Add imx_media_create_fwnode_pad_link()

Hi Steve,

Thank you for the patch.

Quoted 97 lines · Steve Longerbeam

Add functions to create media links between a source and sink subdevice based on fwnode endpoint connections between them.

---
 drivers/staging/media/imx/imx-media-utils.c | 91 +++++++++++++++++++++
 drivers/staging/media/imx/imx-media.h       |  4 +
 2 files changed, 95 insertions(+)
drivers/staging/media/imx/imx-media-utils.c +80 -0
index 0788a1874557..87152bd9af22 100644
@@ -729,6 +729,97 @@ void imx_media_grp_id_to_sd_name(char *sd_name, int sz, u32 grp_id, int ipu_id)
 }
 EXPORT_SYMBOL_GPL(imx_media_grp_id_to_sd_name);
 
+/*
+ * Look for and create a single fwnode link that connects a source
+ * subdevice to a sink pad.
+ */
+int imx_media_create_fwnode_pad_link(struct v4l2_subdev *src_sd,
+				     struct media_pad *sink)
+{
+	struct fwnode_handle *endpoint;
+
+	/* loop thru the source's fwnode endpoints */
+	fwnode_graph_for_each_endpoint(dev_fwnode(src_sd->dev), endpoint) {
+		struct fwnode_handle *remote_ep;
+		int src_idx, sink_idx, ret;
+		struct media_pad *src;
+
+		remote_ep = fwnode_graph_get_remote_endpoint(endpoint);
+		if (!remote_ep)
+			continue;
+
+		/*
+		 * ask the sink entity to verify that this fwnode link
+		 * actually does connect with the entity, and if so that
+		 * it connects to its requested sink pad.
+		 */
+		sink_idx = media_entity_get_fwnode_pad(sink->entity,
+						       remote_ep,
+						       MEDIA_PAD_FL_SINK);
+		fwnode_handle_put(remote_ep);
+
+		if (sink_idx < 0 || sink_idx != sink->index)
+			continue;
+
+		src_idx = media_entity_get_fwnode_pad(&src_sd->entity,
+						      endpoint,
+						      MEDIA_PAD_FL_SOURCE);
+		if (src_idx < 0)
+			continue;
+
+		/*
+		 * found the fwnode link that works, create the media
+		 * link for it.
+		 */
+
+		fwnode_handle_put(endpoint);
+
+		src = &src_sd->entity.pads[src_idx];
+
+		/* success if it already exists */
+		if (media_entity_find_link(src, sink))
+			return 0;
+
+		dev_dbg(src_sd->dev, "%s:%d -> %s:%d\n",
+			src_sd->entity.name, src_idx,
+			sink->entity->name, sink_idx);
+
+		ret = media_create_pad_link(&src_sd->entity, src_idx,
+					    sink->entity, sink_idx, 0);
+		if (ret)
+			dev_err(src_sd->dev,
+				"%s:%d -> %s:%d failed with %d\n",
+				src_sd->entity.name, src_idx,
+				sink->entity->name, sink_idx, ret);
+
+		return ret;
+	}
+
+	return -ENXIO;
+}
+EXPORT_SYMBOL_GPL(imx_media_create_fwnode_pad_link);
+
+int imx_media_create_fwnode_pad_links(struct v4l2_subdev *src_sd,
+				      struct v4l2_subdev *sink_sd)
+{
+	int i;
+
+	for (i = 0; i < sink_sd->entity.num_pads; i++) {
+		struct media_pad *pad = &sink_sd->entity.pads[i];
+		int ret;
+
+		if (!(pad->flags & MEDIA_PAD_FL_SINK))

drivers/staging/media/imx/imx-media-utils.c void imx_media_grp_id_to_sd_name(char *sd_name, int sz, u32 grp_id, int ipu_id) · line 821

Quoted 10 lines
drivers/staging/media/imx/imx-media-utils.c +10 -0
+			continue;
+
+		ret = imx_media_create_fwnode_pad_link(src_sd, pad);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(imx_media_create_fwnode_pad_links);

As commented in 09/17, I think these should be core helpers.

Quoted 19 lines
drivers/staging/media/imx/imx-media-utils.c +1 -0
+
 struct v4l2_subdev *
 imx_media_find_subdev_by_fwnode(struct imx_media_dev *imxmd,
 				struct fwnode_handle *fwnode)
drivers/staging/media/imx/imx-media.h +4 -0
index 11861191324a..f90a65ba4ced 100644
@@ -183,6 +183,10 @@ int imx_media_ipu_image_to_mbus_fmt(struct v4l2_mbus_framefmt *mbus,
 				    struct ipu_image *image);
 void imx_media_grp_id_to_sd_name(char *sd_name, int sz,
 				 u32 grp_id, int ipu_id);
+int imx_media_create_fwnode_pad_link(struct v4l2_subdev *src_sd,
+				     struct media_pad *sink);
+int imx_media_create_fwnode_pad_links(struct v4l2_subdev *src_sd,
+				      struct v4l2_subdev *sink_sd);
 struct v4l2_subdev *
 imx_media_find_subdev_by_fwnode(struct imx_media_dev *imxmd,
 				struct fwnode_handle *fwnode);
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#21 lore raw

PATCH v4 9/17 media: video-mux: Create media links in bound notifier

In reply to Steve Longerbeam

Implement a notifier bound op to register media links from the remote sub-device's source pad(s) to the video-mux sink pad(s).

---
Changes in v3:
- this version does the work inline. The previous version called a media_create_fwnode_links() which is removed in v3.

---
 drivers/media/platform/video-mux.c | 92 ++++++++++++++++++++++++++++++
 1 file changed, 92 insertions(+)
drivers/media/platform/video-mux.c +92 -0
index f446ada82176..3991b1ea671c 100644
@@ -36,6 +36,12 @@ static const struct v4l2_mbus_framefmt video_mux_format_mbus_default = {
 	.field = V4L2_FIELD_NONE,
 };
 
+static inline struct video_mux *
+notifier_to_video_mux(struct v4l2_async_notifier *n)
+{
+	return container_of(n, struct video_mux, notifier);
+}
+
 static inline struct video_mux *v4l2_subdev_to_video_mux(struct v4l2_subdev *sd)
 {
 	return container_of(sd, struct video_mux, subdev);
@@ -360,6 +366,90 @@ static const struct v4l2_subdev_ops video_mux_subdev_ops = {
 	.video = &video_mux_subdev_video_ops,
 };
 
+static int video_mux_notify_bound(struct v4l2_async_notifier *notifier,
+				  struct v4l2_subdev *sd,
+				  struct v4l2_async_subdev *asd)
+{
+	struct video_mux *vmux = notifier_to_video_mux(notifier);
+	struct fwnode_handle *vmux_fwnode = dev_fwnode(vmux->subdev.dev);
+	struct fwnode_handle *sd_fwnode = dev_fwnode(sd->dev);
+	struct fwnode_handle *vmux_ep;
+
+	fwnode_graph_for_each_endpoint(vmux_fwnode, vmux_ep) {
+		struct fwnode_handle *remote_ep, *sd_ep;
+		struct media_pad *src_pad, *sink_pad;
+		struct fwnode_endpoint fwep;
+		int src_idx, sink_idx, ret;
+		bool remote_ep_belongs;
+
+		ret = fwnode_graph_parse_endpoint(vmux_ep, &fwep);
+		if (ret)
+			continue;
+
+		/* only create links to the vmux sink pads */
+		if (fwep.port >= vmux->subdev.entity.num_pads - 1)
+			continue;
+
+		sink_idx = fwep.port;
+		sink_pad = &vmux->subdev.entity.pads[sink_idx];
+
+		remote_ep = fwnode_graph_get_remote_endpoint(vmux_ep);
+		if (!remote_ep)
+			continue;
+
+		/*
+		 * verify that this remote endpoint is owned by the
+		 * sd, in case the sd does not check for that in its
+		 * .get_fwnode_pad operation or does not implement it.
+		 */
+		remote_ep_belongs = false;
+		fwnode_graph_for_each_endpoint(sd_fwnode, sd_ep) {
+			if (sd_ep == remote_ep) {
+				remote_ep_belongs = true;
+				fwnode_handle_put(sd_ep);
+				break;
+			}
+		}
+		if (!remote_ep_belongs)
+			continue;
+
+		src_idx = media_entity_get_fwnode_pad(&sd->entity, remote_ep,
+						      MEDIA_PAD_FL_SOURCE);
+		fwnode_handle_put(remote_ep);
+
+		if (src_idx < 0)
+			continue;
+
+		src_pad = &sd->entity.pads[src_idx];
+
+		/* skip this link if it already exists */
+		if (media_entity_find_link(src_pad, sink_pad))
+			continue;
+
+		ret = media_create_pad_link(&sd->entity, src_idx,
+					    &vmux->subdev.entity,
+					    sink_idx, 0);
+		if (ret) {
+			dev_err(vmux->subdev.dev,
+				"%s:%d -> %s:%d failed with %d\n",
+				sd->entity.name, src_idx,
+				vmux->subdev.entity.name, sink_idx, ret);
+			fwnode_handle_put(vmux_ep);
+			return ret;
+		}
+
+		dev_dbg(vmux->subdev.dev, "%s:%d -> %s:%d\n",
+			sd->entity.name, src_idx,
+			vmux->subdev.entity.name, sink_idx);
+	}
+
+	return 0;
+}
+
+static const struct v4l2_async_notifier_operations video_mux_notify_ops = {
+	.bound = video_mux_notify_bound,
+};
+
 static int video_mux_async_register(struct video_mux *vmux,
 				    unsigned int num_input_pads)
 {
@@ -397,6 +487,8 @@ static int video_mux_async_register(struct video_mux *vmux,
 		}
 	}
 
+	vmux->notifier.ops = &video_mux_notify_ops;
+
 	ret = v4l2_async_subdev_notifier_register(&vmux->subdev,
 						  &vmux->notifier);
 	if (ret)
Signature · 1 line
2.17.1
maintainer
#22 lore raw

Re: [PATCH v4 09/17] media: video-mux: Create media links in bound notifier

Hi Steve,

Thank you for the patch.

Quoted 31 lines · Steve Longerbeam

Implement a notifier bound op to register media links from the remote sub-device's source pad(s) to the video-mux sink pad(s).

---
Changes in v3:
- this version does the work inline. The previous version called a media_create_fwnode_links() which is removed in v3.

---
 drivers/media/platform/video-mux.c | 92 ++++++++++++++++++++++++++++++
 1 file changed, 92 insertions(+)
drivers/media/platform/video-mux.c +6 -0
index f446ada82176..3991b1ea671c 100644
@@ -36,6 +36,12 @@ static const struct v4l2_mbus_framefmt video_mux_format_mbus_default = {
 	.field = V4L2_FIELD_NONE,
 };
 
+static inline struct video_mux *
+notifier_to_video_mux(struct v4l2_async_notifier *n)
+{
+	return container_of(n, struct video_mux, notifier);
+}
+
 static inline struct video_mux *v4l2_subdev_to_video_mux(struct v4l2_subdev *sd)
 {
 	return container_of(sd, struct video_mux, subdev);
@@ -360,6 +366,90 @@ static const struct v4l2_subdev_ops video_mux_subdev_ops = {
 	.video = &video_mux_subdev_video_ops,

drivers/media/platform/video-mux.c static const struct v4l2_subdev_ops video_mux_subdev_ops = { · line 376

Quoted 10 lines
drivers/media/platform/video-mux.c +8 -0
 };
 
+static int video_mux_notify_bound(struct v4l2_async_notifier *notifier,
+				  struct v4l2_subdev *sd,
+				  struct v4l2_async_subdev *asd)
+{
+	struct video_mux *vmux = notifier_to_video_mux(notifier);
+	struct fwnode_handle *vmux_fwnode = dev_fwnode(vmux->subdev.dev);
+	struct fwnode_handle *sd_fwnode = dev_fwnode(sd->dev);
+	struct fwnode_handle *vmux_ep;

There doesn't seem to be anything in this function that is specific to the video_mux driver. I think it would make sense to turn it into a generic helper that creates links between two subdevs based on their fwnode connections.

I also wonder if it wouldn't be more efficient to create links at complete() time instead of bound() time, in which case the helper would create all links for a given subdevice, not links between two specific subdevices (or maybe all links for a given pad direction, to be able to remove the existing link check below).

Quoted 41 lines
drivers/media/platform/video-mux.c +41 -0
+
+	fwnode_graph_for_each_endpoint(vmux_fwnode, vmux_ep) {
+		struct fwnode_handle *remote_ep, *sd_ep;
+		struct media_pad *src_pad, *sink_pad;
+		struct fwnode_endpoint fwep;
+		int src_idx, sink_idx, ret;
+		bool remote_ep_belongs;
+
+		ret = fwnode_graph_parse_endpoint(vmux_ep, &fwep);
+		if (ret)
+			continue;
+
+		/* only create links to the vmux sink pads */
+		if (fwep.port >= vmux->subdev.entity.num_pads - 1)
+			continue;
+
+		sink_idx = fwep.port;
+		sink_pad = &vmux->subdev.entity.pads[sink_idx];
+
+		remote_ep = fwnode_graph_get_remote_endpoint(vmux_ep);
+		if (!remote_ep)
+			continue;
+
+		/*
+		 * verify that this remote endpoint is owned by the
+		 * sd, in case the sd does not check for that in its
+		 * .get_fwnode_pad operation or does not implement it.
+		 */
+		remote_ep_belongs = false;
+		fwnode_graph_for_each_endpoint(sd_fwnode, sd_ep) {
+			if (sd_ep == remote_ep) {
+				remote_ep_belongs = true;
+				fwnode_handle_put(sd_ep);
+				break;
+			}
+		}
+		if (!remote_ep_belongs)
+			continue;
+
+		src_idx = media_entity_get_fwnode_pad(&sd->entity, remote_ep,
+						      MEDIA_PAD_FL_SOURCE);

drivers/media/platform/video-mux.c static const struct v4l2_subdev_ops video_mux_subdev_ops = { · line 427

Quoted 10 lines
drivers/media/platform/video-mux.c +10 -0
+		fwnode_handle_put(remote_ep);
+
+		if (src_idx < 0)
+			continue;
+
+		src_pad = &sd->entity.pads[src_idx];
+
+		/* skip this link if it already exists */
+		if (media_entity_find_link(src_pad, sink_pad))
+			continue;

Have you encountered this in practice ? If we only create links for sink pads this shouldn't happen.

Quoted 37 lines
drivers/media/platform/video-mux.c +27 -0
+
+		ret = media_create_pad_link(&sd->entity, src_idx,
+					    &vmux->subdev.entity,
+					    sink_idx, 0);
+		if (ret) {
+			dev_err(vmux->subdev.dev,
+				"%s:%d -> %s:%d failed with %d\n",
+				sd->entity.name, src_idx,
+				vmux->subdev.entity.name, sink_idx, ret);
+			fwnode_handle_put(vmux_ep);
+			return ret;
+		}
+
+		dev_dbg(vmux->subdev.dev, "%s:%d -> %s:%d\n",
+			sd->entity.name, src_idx,
+			vmux->subdev.entity.name, sink_idx);
+	}
+
+	return 0;
+}
+
+static const struct v4l2_async_notifier_operations video_mux_notify_ops = {
+	.bound = video_mux_notify_bound,
+};
+
 static int video_mux_async_register(struct video_mux *vmux,
 				    unsigned int num_input_pads)
 {
@@ -397,6 +487,8 @@ static int video_mux_async_register(struct video_mux *vmux,
 		}
 	}
 
+	vmux->notifier.ops = &video_mux_notify_ops;
+
 	ret = v4l2_async_subdev_notifier_register(&vmux->subdev,
 						  &vmux->notifier);
 	if (ret)
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#23 lore raw

Re: [PATCH v4 09/17] media: video-mux: Create media links in bound notifier

Hi Laurent,

Quoted 38 lines · Laurent Pinchart

Hi Steve,

Thank you for the patch.

Quoted 33 lines · Steve Longerbeam

Implement a notifier bound op to register media links from the remote sub-device's source pad(s) to the video-mux sink pad(s).

---
Changes in v3:
- this version does the work inline. The previous version called a media_create_fwnode_links() which is removed in v3.

---
  drivers/media/platform/video-mux.c | 92 ++++++++++++++++++++++++++++++
  1 file changed, 92 insertions(+)
drivers/media/platform/video-mux.c +6 -0
index f446ada82176..3991b1ea671c 100644
@@ -36,6 +36,12 @@ static const struct v4l2_mbus_framefmt video_mux_format_mbus_default = {
  	.field = V4L2_FIELD_NONE,
  };
 
+static inline struct video_mux *
+notifier_to_video_mux(struct v4l2_async_notifier *n)
+{
+	return container_of(n, struct video_mux, notifier);
+}
+
  static inline struct video_mux *v4l2_subdev_to_video_mux(struct v4l2_subdev *sd)
  {
  	return container_of(sd, struct video_mux, subdev);
@@ -360,6 +366,90 @@ static const struct v4l2_subdev_ops video_mux_subdev_ops = {
  	.video = &video_mux_subdev_video_ops,
  };
 
Quoted 12 lines

drivers/media/platform/video-mux.c static const struct v4l2_subdev_ops video_mux_subdev_ops = { · line 376

Quoted 8 lines
drivers/media/platform/video-mux.c +8 -0
+static int video_mux_notify_bound(struct v4l2_async_notifier *notifier,
+				  struct v4l2_subdev *sd,
+				  struct v4l2_async_subdev *asd)
+{
+	struct video_mux *vmux = notifier_to_video_mux(notifier);
+	struct fwnode_handle *vmux_fwnode = dev_fwnode(vmux->subdev.dev);
+	struct fwnode_handle *sd_fwnode = dev_fwnode(sd->dev);
+	struct fwnode_handle *vmux_ep;

There doesn't seem to be anything in this function that is specific to the video_mux driver. I think it would make sense to turn it into a generic helper that creates links between two subdevs based on their fwnode connections.

Agreed, in fact I wrote imx_media_create_fwnode_pad_links(src_sd, sink_sd) (patch 8 in this series), and it is completely generic, it could simply be renamed v4l2_create_fwnode_pad_links() and moved to core.

Quoted 5 lines

I also wonder if it wouldn't be more efficient to create links at complete() time instead of bound() time, in which case the helper would create all links for a given subdevice, not links between two specific subdevices (or maybe all links for a given pad direction, to be able to remove the existing link check below).

It looks like that should be possible. The bound sub-devices at complete() time are available in the notifier->done list. I'll start looking at that for v5.

Steve

Quoted 92 lines
Quoted 42 lines
drivers/media/platform/video-mux.c +41 -0
+
+	fwnode_graph_for_each_endpoint(vmux_fwnode, vmux_ep) {
+		struct fwnode_handle *remote_ep, *sd_ep;
+		struct media_pad *src_pad, *sink_pad;
+		struct fwnode_endpoint fwep;
+		int src_idx, sink_idx, ret;
+		bool remote_ep_belongs;
+
+		ret = fwnode_graph_parse_endpoint(vmux_ep, &fwep);
+		if (ret)
+			continue;
+
+		/* only create links to the vmux sink pads */
+		if (fwep.port >= vmux->subdev.entity.num_pads - 1)
+			continue;
+
+		sink_idx = fwep.port;
+		sink_pad = &vmux->subdev.entity.pads[sink_idx];
+
+		remote_ep = fwnode_graph_get_remote_endpoint(vmux_ep);
+		if (!remote_ep)
+			continue;
+
+		/*
+		 * verify that this remote endpoint is owned by the
+		 * sd, in case the sd does not check for that in its
+		 * .get_fwnode_pad operation or does not implement it.
+		 */
+		remote_ep_belongs = false;
+		fwnode_graph_for_each_endpoint(sd_fwnode, sd_ep) {
+			if (sd_ep == remote_ep) {
+				remote_ep_belongs = true;
+				fwnode_handle_put(sd_ep);
+				break;
+			}
+		}
+		if (!remote_ep_belongs)
+			continue;
+
+		src_idx = media_entity_get_fwnode_pad(&sd->entity, remote_ep,
+						      MEDIA_PAD_FL_SOURCE);

drivers/media/platform/video-mux.c static const struct v4l2_subdev_ops video_mux_subdev_ops = { · line 427

Quoted 10 lines
drivers/media/platform/video-mux.c +10 -0
+		fwnode_handle_put(remote_ep);
+
+		if (src_idx < 0)
+			continue;
+
+		src_pad = &sd->entity.pads[src_idx];
+
+		/* skip this link if it already exists */
+		if (media_entity_find_link(src_pad, sink_pad))
+			continue;

Have you encountered this in practice ? If we only create links for sink pads this shouldn't happen.

Quoted 37 lines
drivers/media/platform/video-mux.c +27 -0
+
+		ret = media_create_pad_link(&sd->entity, src_idx,
+					    &vmux->subdev.entity,
+					    sink_idx, 0);
+		if (ret) {
+			dev_err(vmux->subdev.dev,
+				"%s:%d -> %s:%d failed with %d\n",
+				sd->entity.name, src_idx,
+				vmux->subdev.entity.name, sink_idx, ret);
+			fwnode_handle_put(vmux_ep);
+			return ret;
+		}
+
+		dev_dbg(vmux->subdev.dev, "%s:%d -> %s:%d\n",
+			sd->entity.name, src_idx,
+			vmux->subdev.entity.name, sink_idx);
+	}
+
+	return 0;
+}
+
+static const struct v4l2_async_notifier_operations video_mux_notify_ops = {
+	.bound = video_mux_notify_bound,
+};
+
  static int video_mux_async_register(struct video_mux *vmux,
  				    unsigned int num_input_pads)
  {
@@ -397,6 +487,8 @@ static int video_mux_async_register(struct video_mux *vmux,
  		}
  	}
 
+	vmux->notifier.ops = &video_mux_notify_ops;
+
  	ret = v4l2_async_subdev_notifier_register(&vmux->subdev,
  						  &vmux->notifier);
  	if (ret)
author maintainer
#24 lore raw

Re: [PATCH v4 09/17] media: video-mux: Create media links in bound notifier

Hi Laurent,

Quoted 57 lines · Steve Longerbeam

Hi Laurent,

Quoted 43 lines · Laurent Pinchart

Hi Steve,

Thank you for the patch.

Quoted 38 lines · Steve Longerbeam

Implement a notifier bound op to register media links from the remote sub-device's source pad(s) to the video-mux sink pad(s).

---
Changes in v3:
- this version does the work inline. The previous version called a media_create_fwnode_links() which is removed in v3.
---
  drivers/media/platform/video-mux.c | 92
++++++++++++++++++++++++++++++

  1 file changed, 92 insertions(+)
drivers/media/platform/video-mux.c +0 -0
index f446ada82176..3991b1ea671c 100644
@@ -36,6 +36,12 @@ static const struct v4l2_mbus_framefmt 
 video_mux_format_mbus_default = {
       .field = V4L2_FIELD_NONE,
   };

  +static inline struct video_mux *

drivers/media/platform/video-mux.c +5 -0
+notifier_to_video_mux(struct v4l2_async_notifier *n)
+{
+    return container_of(n, struct video_mux, notifier);
+}
+

  static inline struct video_mux *v4l2_subdev_to_video_mux(struct v4l2_subdev *sd) {
      return container_of(sd, struct video_mux, subdev);

drivers/media/platform/video-mux.c +3 -0
@@ -360,6 +366,90 @@ static const struct v4l2_subdev_ops 
 video_mux_subdev_ops = {
       .video = &video_mux_subdev_video_ops,
   };
   +static int video_mux_notify_bound(struct v4l2_async_notifier *notifier,
+                  struct v4l2_subdev *sd,
+                  struct v4l2_async_subdev *asd)
+{
Quoted 8 lines

drivers/media/platform/video-mux.c static const struct v4l2_subdev_ops video_mux_subdev_ops = { · line 376

Quoted 4 lines
drivers/media/platform/video-mux.c +4 -0
+    struct video_mux *vmux = notifier_to_video_mux(notifier);
+    struct fwnode_handle *vmux_fwnode = dev_fwnode(vmux->subdev.dev);
+    struct fwnode_handle *sd_fwnode = dev_fwnode(sd->dev);
+    struct fwnode_handle *vmux_ep;

There doesn't seem to be anything in this function that is specific to the video_mux driver. I think it would make sense to turn it into a generic helper that creates links between two subdevs based on their fwnode connections.

Agreed, in fact I wrote imx_media_create_fwnode_pad_links(src_sd, sink_sd) (patch 8 in this series), and it is completely generic, it could simply be renamed v4l2_create_fwnode_pad_links() and moved to core.

Quoted 7 lines
Quoted 5 lines

I also wonder if it wouldn't be more efficient to create links at complete() time instead of bound() time, in which case the helper would create all links for a given subdevice, not links between two specific subdevices (or maybe all links for a given pad direction, to be able to remove the existing link check below).

It looks like that should be possible. The bound sub-devices at complete() time are available in the notifier->done list. I'll start looking at that for v5.

Actually for sub-devices that won't work. The .complete() callback is only called on the root notifier, not on subdev notifiers. I don't think it's a big deal, it works fine to create links from one source subdev at a time in .bound(), and I don't see it as much of a efficiency hit.

Steve

Quoted 97 lines
Quoted 96 lines
Quoted 45 lines
drivers/media/platform/video-mux.c +41 -0
+
+    fwnode_graph_for_each_endpoint(vmux_fwnode, vmux_ep) {
+        struct fwnode_handle *remote_ep, *sd_ep;
+        struct media_pad *src_pad, *sink_pad;
+        struct fwnode_endpoint fwep;
+        int src_idx, sink_idx, ret;
+        bool remote_ep_belongs;
+
+        ret = fwnode_graph_parse_endpoint(vmux_ep, &fwep);
+        if (ret)
+            continue;
+
+        /* only create links to the vmux sink pads */
+        if (fwep.port >= vmux->subdev.entity.num_pads - 1)
+            continue;
+
+        sink_idx = fwep.port;
+        sink_pad = &vmux->subdev.entity.pads[sink_idx];
+
+        remote_ep = fwnode_graph_get_remote_endpoint(vmux_ep);
+        if (!remote_ep)
+            continue;
+
+        /*
+         * verify that this remote endpoint is owned by the
+         * sd, in case the sd does not check for that in its
+         * .get_fwnode_pad operation or does not implement it.
+         */
+        remote_ep_belongs = false;
+        fwnode_graph_for_each_endpoint(sd_fwnode, sd_ep) {
+            if (sd_ep == remote_ep) {
+                remote_ep_belongs = true;
+                fwnode_handle_put(sd_ep);
+                break;
+            }
+        }
+        if (!remote_ep_belongs)
+            continue;
+
+        src_idx = media_entity_get_fwnode_pad(&sd->entity, remote_ep,
+                              MEDIA_PAD_FL_SOURCE);

drivers/media/platform/video-mux.c static const struct v4l2_subdev_ops video_mux_subdev_ops = { · line 427

Quoted 10 lines
drivers/media/platform/video-mux.c +10 -0
+        fwnode_handle_put(remote_ep);
+
+        if (src_idx < 0)
+            continue;
+
+        src_pad = &sd->entity.pads[src_idx];
+
+        /* skip this link if it already exists */
+        if (media_entity_find_link(src_pad, sink_pad))
+            continue;

Have you encountered this in practice ? If we only create links for sink pads this shouldn't happen.

Quoted 38 lines
drivers/media/platform/video-mux.c +25 -0
+
+        ret = media_create_pad_link(&sd->entity, src_idx,
+                        &vmux->subdev.entity,
+                        sink_idx, 0);
+        if (ret) {
+            dev_err(vmux->subdev.dev,
+                "%s:%d -> %s:%d failed with %d\n",
+                sd->entity.name, src_idx,
+                vmux->subdev.entity.name, sink_idx, ret);
+            fwnode_handle_put(vmux_ep);
+            return ret;
+        }
+
+        dev_dbg(vmux->subdev.dev, "%s:%d -> %s:%d\n",
+            sd->entity.name, src_idx,
+            vmux->subdev.entity.name, sink_idx);
+    }
+
+    return 0;
+}
+
+static const struct v4l2_async_notifier_operations 
 video_mux_notify_ops = {
+    .bound = video_mux_notify_bound,
+};
+
   static int video_mux_async_register(struct video_mux *vmux,

                      unsigned int num_input_pads) {

drivers/media/platform/video-mux.c +1 -0
@@ -397,6 +487,8 @@ static int video_mux_async_register(struct 
 video_mux *vmux,
           }
       }
   +    vmux->notifier.ops = &video_mux_notify_ops;
+
       ret = v4l2_async_subdev_notifier_register(&vmux->subdev,
                             &vmux->notifier);

      if (ret)

author maintainer
#25 lore raw

PATCH v4 10/17 media: imx: mipi csi-2: Create media links in bound notifier

In reply to Steve Longerbeam

Implement a notifier bound op to register media links from the remote sub-device's source pad(s) to the mipi csi-2 receiver sink pad.

---
Changes in v3:
- call a local imx-media utility imx_media_create_fwnode_pad_link() that creates a single link.

---
 drivers/staging/media/imx/imx6-mipi-csi2.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
drivers/staging/media/imx/imx6-mipi-csi2.c +21 -0
index 8500207e5ea9..26c0b579e6ae 100644
@@ -91,6 +91,11 @@ static inline struct csi2_dev *sd_to_dev(struct v4l2_subdev *sdev)
 	return container_of(sdev, struct csi2_dev, sd);
 }
 
+static inline struct csi2_dev *notifier_to_dev(struct v4l2_async_notifier *n)
+{
+	return container_of(n, struct csi2_dev, notifier);
+}
+
 /*
  * The required sequence of MIPI CSI-2 startup as specified in the i.MX6
  * reference manual is as follows:
@@ -559,6 +564,20 @@ static const struct v4l2_subdev_internal_ops csi2_internal_ops = {
 	.registered = csi2_registered,
 };
 
+static int csi2_notify_bound(struct v4l2_async_notifier *notifier,
+			     struct v4l2_subdev *sd,
+			     struct v4l2_async_subdev *asd)
+{
+	struct csi2_dev *csi2 = notifier_to_dev(notifier);
+	struct media_pad *sink = &csi2->sd.entity.pads[CSI2_SINK_PAD];
+
+	return imx_media_create_fwnode_pad_link(sd, sink);
+}
+
+static const struct v4l2_async_notifier_operations csi2_notify_ops = {
+	.bound = csi2_notify_bound,
+};
+
 static int csi2_async_register(struct csi2_dev *csi2)
 {
 	struct v4l2_fwnode_endpoint vep = {
@@ -597,6 +616,8 @@ static int csi2_async_register(struct csi2_dev *csi2)
 
 	fwnode_handle_put(ep);
 
+	csi2->notifier.ops = &csi2_notify_ops;
+
 	ret = v4l2_async_subdev_notifier_register(&csi2->sd,
 						  &csi2->notifier);
 	if (ret)
Signature · 1 line
2.17.1
author maintainer
#26 lore raw

PATCH v4 11/17 media: imx7: mipi csis: Create media links in bound notifier

In reply to Steve Longerbeam

Implement a notifier bound op to register media links from the remote sub-device's source pad(s) to the mipi csi-2 receiver sink pad.

---
Changes in v3:
- call a local imx-media utility imx_media_create_fwnode_pad_link(). Changes in v2:
- Move notifier_to_csis_state() next to mipi_sd_to_csis_state(), remove unnecessary inline, and rename to mipi_notifier_to_csis_state(). Suggested by Rui Silva.

---
 drivers/staging/media/imx/imx7-mipi-csis.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
drivers/staging/media/imx/imx7-mipi-csis.c +22 -0
index 7e872d8f51e0..363545127047 100644
@@ -317,6 +317,12 @@ static int mipi_csis_dump_regs(struct csi_state *state)
 	return 0;
 }
 
+static struct csi_state *
+mipi_notifier_to_csis_state(struct v4l2_async_notifier *n)
+{
+	return container_of(n, struct csi_state, notifier);
+}
+
 static struct csi_state *mipi_sd_to_csis_state(struct v4l2_subdev *sdev)
 {
 	return container_of(sdev, struct csi_state, mipi_sd);
@@ -828,6 +834,20 @@ static int mipi_csis_parse_dt(struct platform_device *pdev,
 
 static int mipi_csis_pm_resume(struct device *dev, bool runtime);
 
+static int mipi_csis_notify_bound(struct v4l2_async_notifier *notifier,
+				  struct v4l2_subdev *sd,
+				  struct v4l2_async_subdev *asd)
+{
+	struct csi_state *state = mipi_notifier_to_csis_state(notifier);
+	struct media_pad *sink = &state->mipi_sd.entity.pads[CSIS_PAD_SINK];
+
+	return imx_media_create_fwnode_pad_link(sd, sink);
+}
+
+static const struct v4l2_async_notifier_operations mipi_csis_notify_ops = {
+	.bound = mipi_csis_notify_bound,
+};
+
 static int mipi_csis_subdev_init(struct v4l2_subdev *mipi_sd,
 				 struct platform_device *pdev,
 				 const struct v4l2_subdev_ops *ops)
@@ -899,6 +919,8 @@ static int mipi_csis_async_register(struct csi_state *state)
 
 	fwnode_handle_put(ep);
 
+	state->notifier.ops = &mipi_csis_notify_ops;
+
 	ret = v4l2_async_subdev_notifier_register(&state->mipi_sd,
 						  &state->notifier);
 	if (ret)
Signature · 1 line
2.17.1
author maintainer
#27 lore raw

PATCH v4 12/17 media: imx7: csi: Create media links in bound notifier

In reply to Steve Longerbeam

Implement a notifier bound op to register media links from the remote sub-device's source pad(s) to the CSI sink pad.

---
Changes in v3:
- call a local imx-media utility imx_media_create_fwnode_pad_link(). Changes in v2:
- Rename notifier_to_dev() to imx7_csi_notifier_to_dev() and remove unnecessary inline. Suggested by Rui Silva.

---
 drivers/staging/media/imx/imx7-media-csi.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
drivers/staging/media/imx/imx7-media-csi.c +22 -0
index 776eb42ae5c8..88e21c13e420 100644
@@ -196,6 +196,12 @@ struct imx7_csi {
 	struct completion last_eof_completion;
 };
 
+static struct imx7_csi *
+imx7_csi_notifier_to_dev(struct v4l2_async_notifier *n)
+{
+	return container_of(n, struct imx7_csi, notifier);
+}
+
 static u32 imx7_csi_reg_read(struct imx7_csi *csi, unsigned int offset)
 {
 	return readl(csi->regbase + offset);
@@ -1180,6 +1186,20 @@ static const struct v4l2_subdev_internal_ops imx7_csi_internal_ops = {
 	.unregistered	= imx7_csi_unregistered,
 };
 
+static int imx7_csi_notify_bound(struct v4l2_async_notifier *notifier,
+				 struct v4l2_subdev *sd,
+				 struct v4l2_async_subdev *asd)
+{
+	struct imx7_csi *csi = imx7_csi_notifier_to_dev(notifier);
+	struct media_pad *sink = &csi->sd.entity.pads[IMX7_CSI_PAD_SINK];
+
+	return imx_media_create_fwnode_pad_link(sd, sink);
+}
+
+static const struct v4l2_async_notifier_operations imx7_csi_notify_ops = {
+	.bound = imx7_csi_notify_bound,
+};
+
 static int imx7_csi_async_register(struct imx7_csi *csi)
 {
 	struct v4l2_async_subdev *asd = NULL;
@@ -1210,6 +1230,8 @@ static int imx7_csi_async_register(struct imx7_csi *csi)
 		}
 	}
 
+	csi->notifier.ops = &imx7_csi_notify_ops;
+
 	ret = v4l2_async_subdev_notifier_register(&csi->sd, &csi->notifier);
 	if (ret)
 		return ret;
Signature · 1 line
2.17.1
author maintainer
#28 lore raw

PATCH v4 13/17 media: imx: csi: Create media links in bound notifier

In reply to Steve Longerbeam

Implement a notifier bound op to register media links from the remote sub-device's source pad(s) to the CSI sink pad.

---
Changes in v3:
- call a local imx-media utility imx_media_create_fwnode_pad_link().

---
 drivers/staging/media/imx/imx-media-csi.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
drivers/staging/media/imx/imx-media-csi.c +21 -0
index 35f2512ed2a9..59ab1cf72841 100644
@@ -120,6 +120,11 @@ static inline struct csi_priv *sd_to_dev(struct v4l2_subdev *sdev)
 	return container_of(sdev, struct csi_priv, sd);
 }
 
+static inline struct csi_priv *notifier_to_dev(struct v4l2_async_notifier *n)
+{
+	return container_of(n, struct csi_priv, notifier);
+}
+
 static inline bool is_parallel_bus(struct v4l2_fwnode_endpoint *ep)
 {
 	return ep->bus_type != V4L2_MBUS_CSI2_DPHY;
@@ -1889,6 +1894,20 @@ static const struct v4l2_subdev_internal_ops csi_internal_ops = {
 	.unregistered = csi_unregistered,
 };
 
+static int imx_csi_notify_bound(struct v4l2_async_notifier *notifier,
+				struct v4l2_subdev *sd,
+				struct v4l2_async_subdev *asd)
+{
+	struct csi_priv *priv = notifier_to_dev(notifier);
+	struct media_pad *sink = &priv->sd.entity.pads[CSI_SINK_PAD];
+
+	return imx_media_create_fwnode_pad_link(sd, sink);
+}
+
+static const struct v4l2_async_notifier_operations csi_notify_ops = {
+	.bound = imx_csi_notify_bound,
+};
+
 static int imx_csi_async_register(struct csi_priv *priv)
 {
 	struct v4l2_async_subdev *asd = NULL;
@@ -1926,6 +1945,8 @@ static int imx_csi_async_register(struct csi_priv *priv)
 		}
 	}
 
+	priv->notifier.ops = &csi_notify_ops;
+
 	ret = v4l2_async_subdev_notifier_register(&priv->sd,
 						  &priv->notifier);
 	if (ret)
Signature · 1 line
2.17.1
author maintainer
#29 lore raw

PATCH v4 14/17 media: imx: csi: Lookup upstream endpoint with imx_media_get_pad_fwnode

In reply to Steve Longerbeam

Fix the 1:1 port-id:pad-index assumption for the upstream subdevice, by searching the upstream subdevice's endpoints for one that maps to the pad's index. This is carried out by a new reverse mapping function imx_media_get_pad_fwnode().

---
 drivers/staging/media/imx/imx-media-csi.c   | 22 ++++----------
 drivers/staging/media/imx/imx-media-utils.c | 33 +++++++++++++++++++++
 drivers/staging/media/imx/imx-media.h       |  1 +
 drivers/staging/media/imx/imx7-media-csi.c  | 25 +++++-----------
 4 files changed, 47 insertions(+), 34 deletions(-)
drivers/staging/media/imx/imx-media-csi.c +6 -16
index 59ab1cf72841..0388d73e916c 100644
@@ -164,7 +164,7 @@ static inline bool requires_passthrough(struct v4l2_fwnode_endpoint *ep,
 static int csi_get_upstream_endpoint(struct csi_priv *priv,
 				     struct v4l2_fwnode_endpoint *ep)
 {
-	struct device_node *endpoint, *port;
+	struct fwnode_handle *endpoint;
 	struct media_entity *src;
 	struct v4l2_subdev *sd;
 	struct media_pad *pad;
@@ -203,23 +203,13 @@ static int csi_get_upstream_endpoint(struct csi_priv *priv,
 	if (!pad)
 		return -ENODEV;
 
-	sd = media_entity_to_v4l2_subdev(pad->entity);
+	endpoint = imx_media_get_pad_fwnode(pad);
+	if (IS_ERR(endpoint))
+		return PTR_ERR(endpoint);
 
-	/*
-	 * NOTE: this assumes an OF-graph port id is the same as a
-	 * media pad index.
-	 */
-	port = of_graph_get_port_by_id(sd->dev->of_node, pad->index);
-	if (!port)
-		return -ENODEV;
-
-	endpoint = of_get_next_child(port, NULL);
-	of_node_put(port);
-	if (!endpoint)
-		return -ENODEV;
+	v4l2_fwnode_endpoint_parse(endpoint, ep);
 
-	v4l2_fwnode_endpoint_parse(of_fwnode_handle(endpoint), ep);
-	of_node_put(endpoint);
+	fwnode_handle_put(endpoint);
 
 	return 0;
 }
drivers/staging/media/imx/imx-media-utils.c +33 -0
index 87152bd9af22..61752c6b074d 100644
@@ -1007,6 +1007,39 @@ imx_media_pipeline_video_device(struct media_entity *start_entity,
 }
 EXPORT_SYMBOL_GPL(imx_media_pipeline_video_device);
 
+/*
+ * Find a fwnode endpoint that maps to the given subdevice's pad.
+ * If there are multiple endpoints that map to the pad, only the
+ * first endpoint encountered is returned.
+ *
+ * On success the refcount of the returned fwnode endpoint is
+ * incremented.
+ */
+struct fwnode_handle *imx_media_get_pad_fwnode(struct media_pad *pad)
+{
+	struct fwnode_handle *endpoint;
+	struct v4l2_subdev *sd;
+
+	if (!is_media_entity_v4l2_subdev(pad->entity))
+		return ERR_PTR(-ENODEV);
+
+	sd = media_entity_to_v4l2_subdev(pad->entity);
+
+	fwnode_graph_for_each_endpoint(dev_fwnode(sd->dev), endpoint) {
+		int pad_idx = media_entity_get_fwnode_pad(&sd->entity,
+							  endpoint,
+							  pad->flags);
+		if (pad_idx < 0)
+			continue;
+
+		if (pad_idx == pad->index)
+			return endpoint;
+	}
+
+	return ERR_PTR(-ENODEV);
+}
+EXPORT_SYMBOL_GPL(imx_media_get_pad_fwnode);
+
 /*
  * Turn current pipeline streaming on/off starting from entity.
  */
drivers/staging/media/imx/imx-media.h +1 -0
index f90a65ba4ced..5f23d852122f 100644
@@ -205,6 +205,7 @@ imx_media_pipeline_subdev(struct media_entity *start_entity, u32 grp_id,
 struct video_device *
 imx_media_pipeline_video_device(struct media_entity *start_entity,
 				enum v4l2_buf_type buftype, bool upstream);
+struct fwnode_handle *imx_media_get_pad_fwnode(struct media_pad *pad);
 
 struct imx_media_dma_buf {
 	void          *virt;
drivers/staging/media/imx/imx7-media-csi.c +7 -18
index 88e21c13e420..9b31379611ac 100644
@@ -439,9 +439,8 @@ static int imx7_csi_get_upstream_endpoint(struct imx7_csi *csi,
 					  struct v4l2_fwnode_endpoint *ep,
 					  bool skip_mux)
 {
-	struct device_node *endpoint, *port;
+	struct fwnode_handle *endpoint;
 	struct media_entity *src;
-	struct v4l2_subdev *sd;
 	struct media_pad *pad;
 
 	if (!csi->src_sd)
@@ -463,29 +462,19 @@ static int imx7_csi_get_upstream_endpoint(struct imx7_csi *csi,
 	if (!pad)
 		return -ENODEV;
 
-	sd = media_entity_to_v4l2_subdev(pad->entity);
-
 	/* To get bus type we may need to skip video mux */
 	if (skip_mux && src->function == MEDIA_ENT_F_VID_MUX) {
-		src = &sd->entity;
+		src = pad->entity;
 		goto skip_video_mux;
 	}
 
-	/*
-	 * NOTE: this assumes an OF-graph port id is the same as a
-	 * media pad index.
-	 */
-	port = of_graph_get_port_by_id(sd->dev->of_node, pad->index);
-	if (!port)
-		return -ENODEV;
+	endpoint = imx_media_get_pad_fwnode(pad);
+	if (IS_ERR(endpoint))
+		return PTR_ERR(endpoint);
 
-	endpoint = of_get_next_child(port, NULL);
-	of_node_put(port);
-	if (!endpoint)
-		return -ENODEV;
+	v4l2_fwnode_endpoint_parse(endpoint, ep);
 
-	v4l2_fwnode_endpoint_parse(of_fwnode_handle(endpoint), ep);
-	of_node_put(endpoint);
+	fwnode_handle_put(endpoint);
 
 	return 0;
 }
Signature · 1 line
2.17.1
maintainer
#30 lore raw

Re: [PATCH v4 14/17] media: imx: csi: Lookup upstream endpoint with imx_media_get_pad_fwnode

Hi Steve,

Thank you for the patch.

Quoted 4 lines · Steve Longerbeam

Fix the 1:1 port-id:pad-index assumption for the upstream subdevice, by searching the upstream subdevice's endpoints for one that maps to the pad's index. This is carried out by a new reverse mapping function imx_media_get_pad_fwnode().

This gets pretty inefficient :-S I think it's fine for now, but I believe there's room for additional refactoring to avoid the need to get the upstream endpoint in the first place.

Quoted 157 lines
---
 drivers/staging/media/imx/imx-media-csi.c   | 22 ++++----------
 drivers/staging/media/imx/imx-media-utils.c | 33 +++++++++++++++++++++
 drivers/staging/media/imx/imx-media.h       |  1 +
 drivers/staging/media/imx/imx7-media-csi.c  | 25 +++++-----------
 4 files changed, 47 insertions(+), 34 deletions(-)
drivers/staging/media/imx/imx-media-csi.c +6 -16
index 59ab1cf72841..0388d73e916c 100644
@@ -164,7 +164,7 @@ static inline bool requires_passthrough(struct v4l2_fwnode_endpoint *ep,
 static int csi_get_upstream_endpoint(struct csi_priv *priv,
 				     struct v4l2_fwnode_endpoint *ep)
 {
-	struct device_node *endpoint, *port;
+	struct fwnode_handle *endpoint;
 	struct media_entity *src;
 	struct v4l2_subdev *sd;
 	struct media_pad *pad;
@@ -203,23 +203,13 @@ static int csi_get_upstream_endpoint(struct csi_priv *priv,
 	if (!pad)
 		return -ENODEV;
 
-	sd = media_entity_to_v4l2_subdev(pad->entity);
+	endpoint = imx_media_get_pad_fwnode(pad);
+	if (IS_ERR(endpoint))
+		return PTR_ERR(endpoint);
 
-	/*
-	 * NOTE: this assumes an OF-graph port id is the same as a
-	 * media pad index.
-	 */
-	port = of_graph_get_port_by_id(sd->dev->of_node, pad->index);
-	if (!port)
-		return -ENODEV;
-
-	endpoint = of_get_next_child(port, NULL);
-	of_node_put(port);
-	if (!endpoint)
-		return -ENODEV;
+	v4l2_fwnode_endpoint_parse(endpoint, ep);
 
-	v4l2_fwnode_endpoint_parse(of_fwnode_handle(endpoint), ep);
-	of_node_put(endpoint);
+	fwnode_handle_put(endpoint);
 
 	return 0;
 }
drivers/staging/media/imx/imx-media-utils.c +33 -0
index 87152bd9af22..61752c6b074d 100644
@@ -1007,6 +1007,39 @@ imx_media_pipeline_video_device(struct media_entity *start_entity,
 }
 EXPORT_SYMBOL_GPL(imx_media_pipeline_video_device);
 
+/*
+ * Find a fwnode endpoint that maps to the given subdevice's pad.
+ * If there are multiple endpoints that map to the pad, only the
+ * first endpoint encountered is returned.
+ *
+ * On success the refcount of the returned fwnode endpoint is
+ * incremented.
+ */
+struct fwnode_handle *imx_media_get_pad_fwnode(struct media_pad *pad)
+{
+	struct fwnode_handle *endpoint;
+	struct v4l2_subdev *sd;
+
+	if (!is_media_entity_v4l2_subdev(pad->entity))
+		return ERR_PTR(-ENODEV);
+
+	sd = media_entity_to_v4l2_subdev(pad->entity);
+
+	fwnode_graph_for_each_endpoint(dev_fwnode(sd->dev), endpoint) {
+		int pad_idx = media_entity_get_fwnode_pad(&sd->entity,
+							  endpoint,
+							  pad->flags);
+		if (pad_idx < 0)
+			continue;
+
+		if (pad_idx == pad->index)
+			return endpoint;
+	}
+
+	return ERR_PTR(-ENODEV);
+}
+EXPORT_SYMBOL_GPL(imx_media_get_pad_fwnode);
+
 /*
  * Turn current pipeline streaming on/off starting from entity.
  */
drivers/staging/media/imx/imx-media.h +1 -0
index f90a65ba4ced..5f23d852122f 100644
@@ -205,6 +205,7 @@ imx_media_pipeline_subdev(struct media_entity *start_entity, u32 grp_id,
 struct video_device *
 imx_media_pipeline_video_device(struct media_entity *start_entity,
 				enum v4l2_buf_type buftype, bool upstream);
+struct fwnode_handle *imx_media_get_pad_fwnode(struct media_pad *pad);
 
 struct imx_media_dma_buf {
 	void          *virt;
drivers/staging/media/imx/imx7-media-csi.c +7 -18
index 88e21c13e420..9b31379611ac 100644
@@ -439,9 +439,8 @@ static int imx7_csi_get_upstream_endpoint(struct imx7_csi *csi,
 					  struct v4l2_fwnode_endpoint *ep,
 					  bool skip_mux)
 {
-	struct device_node *endpoint, *port;
+	struct fwnode_handle *endpoint;
 	struct media_entity *src;
-	struct v4l2_subdev *sd;
 	struct media_pad *pad;
 
 	if (!csi->src_sd)
@@ -463,29 +462,19 @@ static int imx7_csi_get_upstream_endpoint(struct imx7_csi *csi,
 	if (!pad)
 		return -ENODEV;
 
-	sd = media_entity_to_v4l2_subdev(pad->entity);
-
 	/* To get bus type we may need to skip video mux */
 	if (skip_mux && src->function == MEDIA_ENT_F_VID_MUX) {
-		src = &sd->entity;
+		src = pad->entity;
 		goto skip_video_mux;
 	}
 
-	/*
-	 * NOTE: this assumes an OF-graph port id is the same as a
-	 * media pad index.
-	 */
-	port = of_graph_get_port_by_id(sd->dev->of_node, pad->index);
-	if (!port)
-		return -ENODEV;
+	endpoint = imx_media_get_pad_fwnode(pad);
+	if (IS_ERR(endpoint))
+		return PTR_ERR(endpoint);
 
-	endpoint = of_get_next_child(port, NULL);
-	of_node_put(port);
-	if (!endpoint)
-		return -ENODEV;
+	v4l2_fwnode_endpoint_parse(endpoint, ep);
 
-	v4l2_fwnode_endpoint_parse(of_fwnode_handle(endpoint), ep);
-	of_node_put(endpoint);
+	fwnode_handle_put(endpoint);
 
 	return 0;
 }
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#31 lore raw

PATCH v4 15/17 media: imx: Create missing links from CSI-2 receiver

In reply to Steve Longerbeam

The entities external to the i.MX6 IPU and i.MX7 now create the links to their fwnode-endpoint connected entities in their notifier bound callbacks. Which means imx_media_create_of_links() and imx_media_create_csi_of_links() are no longer needed and are removed.

However there is still one case in which imx-media needs to create fwnode-endpoint based links at probe completion. The v4l2-async framework does not allow multiple subdevice notifiers to contain a duplicate subdevice in their asd_list. Only the first subdev notifier that discovers and adds that one subdevice to its asd_list will receive a bound callback for it. Other subdevices that also have firmware endpoint connections to this duplicate subdevice will not have it in their asd_list, and thus will never receive a bound callback for it. In the case of imx-media, the one duplicate subdevice in question is the i.MX6 MIPI CSI-2 receiver.

Until there is a solution to that problem, rewrite imx_media_create_links() to add the missing links from the CSI-2 receiver to the CSIs and CSI muxes. The function is renamed imx_media_create_csi2_links().

---
Changes in v3:
- call a local imx-media utility imx_media_create_fwnode_pad_links(). Changes in v2:
- this is a rewrite of v1 "media: imx: Use media_create_fwnode_links for external links", which only adds the missing CSI-2 receiver links.

---
 .../staging/media/imx/imx-media-dev-common.c  |  46 +++----
 drivers/staging/media/imx/imx-media-of.c      | 114 ------------------
 drivers/staging/media/imx/imx-media.h         |   4 -
 3 files changed, 17 insertions(+), 147 deletions(-)
drivers/staging/media/imx/imx-media-dev-common.c +17 -29
index 66b505f7e8df..f7ad3cbbeec2 100644
@@ -30,41 +30,31 @@ static int imx_media_subdev_bound(struct v4l2_async_notifier *notifier,
 }
 
 /*
- * Create the media links for all subdevs that registered.
+ * Create the missing media links from the CSI-2 receiver.
  * Called after all async subdevs have bound.
  */
-static int imx_media_create_links(struct v4l2_async_notifier *notifier)
+static void imx_media_create_csi2_links(struct imx_media_dev *imxmd)
 {
-	struct imx_media_dev *imxmd = notifier2dev(notifier);
-	struct v4l2_subdev *sd;
+	struct v4l2_subdev *sd, *csi2 = NULL;
 
 	list_for_each_entry(sd, &imxmd->v4l2_dev.subdevs, list) {
-		switch (sd->grp_id) {
-		case IMX_MEDIA_GRP_ID_IPU_VDIC:
-		case IMX_MEDIA_GRP_ID_IPU_IC_PRP:
-		case IMX_MEDIA_GRP_ID_IPU_IC_PRPENC:
-		case IMX_MEDIA_GRP_ID_IPU_IC_PRPVF:
-			/*
-			 * links have already been created for the
-			 * sync-registered subdevs.
-			 */
-			break;
-		case IMX_MEDIA_GRP_ID_IPU_CSI0:
-		case IMX_MEDIA_GRP_ID_IPU_CSI1:
-		case IMX_MEDIA_GRP_ID_CSI:
-			imx_media_create_csi_of_links(imxmd, sd);
-			break;
-		default:
-			/*
-			 * if this subdev has fwnode links, create media
-			 * links for them.
-			 */
-			imx_media_create_of_links(imxmd, sd);
+		if (sd->grp_id == IMX_MEDIA_GRP_ID_CSI2) {
+			csi2 = sd;
 			break;
 		}
 	}
+	if (!csi2)
+		return;
 
-	return 0;
+	list_for_each_entry(sd, &imxmd->v4l2_dev.subdevs, list) {
+		/* skip if not a CSI or a video mux */
+		if (!(sd->grp_id & IMX_MEDIA_GRP_ID_IPU_CSI) &&
+		    !(sd->grp_id & IMX_MEDIA_GRP_ID_CSI) &&
+		    sd->entity.function != MEDIA_ENT_F_VID_MUX)
+			continue;
+
+		imx_media_create_fwnode_pad_links(csi2, sd);
+	}
 }
 
 /*
@@ -196,9 +186,7 @@ int imx_media_probe_complete(struct v4l2_async_notifier *notifier)
 
 	mutex_lock(&imxmd->mutex);
 
-	ret = imx_media_create_links(notifier);
-	if (ret)
-		goto unlock;
+	imx_media_create_csi2_links(imxmd);
 
 	ret = imx_media_create_pad_vdev_lists(imxmd);
 	if (ret)
drivers/staging/media/imx/imx-media-of.c +0 -114

117 lines not on this page · raw message

drivers/staging/media/imx/imx-media.h +0 -4
index 5f23d852122f..5271b84bea9a 100644
@@ -248,10 +248,6 @@ void imx_media_unregister_ipu_internal_subdevs(struct imx_media_dev *imxmd);
 /* imx-media-of.c */
 int imx_media_add_of_subdevs(struct imx_media_dev *dev,
 			     struct device_node *np);
-int imx_media_create_of_links(struct imx_media_dev *imxmd,
-			      struct v4l2_subdev *sd);
-int imx_media_create_csi_of_links(struct imx_media_dev *imxmd,
-				  struct v4l2_subdev *csi);
 int imx_media_of_add_csi(struct imx_media_dev *imxmd,
 			 struct device_node *csi_np);
 
Signature · 1 line
2.17.1
maintainer Reviewed-by on v4 15/17
#32 lore raw

Re: [PATCH v4 15/17] media: imx: Create missing links from CSI-2 receiver

Hi Steve,

Thank you for the patch.

Quoted 77 lines · Steve Longerbeam

The entities external to the i.MX6 IPU and i.MX7 now create the links to their fwnode-endpoint connected entities in their notifier bound callbacks. Which means imx_media_create_of_links() and imx_media_create_csi_of_links() are no longer needed and are removed.

However there is still one case in which imx-media needs to create fwnode-endpoint based links at probe completion. The v4l2-async framework does not allow multiple subdevice notifiers to contain a duplicate subdevice in their asd_list. Only the first subdev notifier that discovers and adds that one subdevice to its asd_list will receive a bound callback for it. Other subdevices that also have firmware endpoint connections to this duplicate subdevice will not have it in their asd_list, and thus will never receive a bound callback for it. In the case of imx-media, the one duplicate subdevice in question is the i.MX6 MIPI CSI-2 receiver.

Until there is a solution to that problem, rewrite imx_media_create_links() to add the missing links from the CSI-2 receiver to the CSIs and CSI muxes. The function is renamed imx_media_create_csi2_links().

---
Changes in v3:
- call a local imx-media utility imx_media_create_fwnode_pad_links(). Changes in v2:
- this is a rewrite of v1 "media: imx: Use media_create_fwnode_links for external links", which only adds the missing CSI-2 receiver links.

---
 .../staging/media/imx/imx-media-dev-common.c  |  46 +++----
 drivers/staging/media/imx/imx-media-of.c      | 114 ------------------
 drivers/staging/media/imx/imx-media.h         |   4 -
 3 files changed, 17 insertions(+), 147 deletions(-)
drivers/staging/media/imx/imx-media-dev-common.c +5 -25
index 66b505f7e8df..f7ad3cbbeec2 100644
@@ -30,41 +30,31 @@ static int imx_media_subdev_bound(struct v4l2_async_notifier *notifier,
 }
 
 /*
- * Create the media links for all subdevs that registered.
+ * Create the missing media links from the CSI-2 receiver.
  * Called after all async subdevs have bound.
  */
-static int imx_media_create_links(struct v4l2_async_notifier *notifier)
+static void imx_media_create_csi2_links(struct imx_media_dev *imxmd)
 {
-	struct imx_media_dev *imxmd = notifier2dev(notifier);
-	struct v4l2_subdev *sd;
+	struct v4l2_subdev *sd, *csi2 = NULL;
 
 	list_for_each_entry(sd, &imxmd->v4l2_dev.subdevs, list) {
-		switch (sd->grp_id) {
-		case IMX_MEDIA_GRP_ID_IPU_VDIC:
-		case IMX_MEDIA_GRP_ID_IPU_IC_PRP:
-		case IMX_MEDIA_GRP_ID_IPU_IC_PRPENC:
-		case IMX_MEDIA_GRP_ID_IPU_IC_PRPVF:
-			/*
-			 * links have already been created for the
-			 * sync-registered subdevs.
-			 */
-			break;
-		case IMX_MEDIA_GRP_ID_IPU_CSI0:
-		case IMX_MEDIA_GRP_ID_IPU_CSI1:
-		case IMX_MEDIA_GRP_ID_CSI:
-			imx_media_create_csi_of_links(imxmd, sd);
-			break;
-		default:
-			/*
-			 * if this subdev has fwnode links, create media
-			 * links for them.
-			 */
-			imx_media_create_of_links(imxmd, sd);
+		if (sd->grp_id == IMX_MEDIA_GRP_ID_CSI2) {
+			csi2 = sd;
 			break;
 		}

drivers/staging/media/imx/imx-media-dev-common.c static int imx_media_subdev_bound(struct v4l2_async_notifier *notifier, · line 53

Quoted 10 lines
drivers/staging/media/imx/imx-media-dev-common.c +7 -1
 	}
+	if (!csi2)
+		return;
 
-	return 0;
+	list_for_each_entry(sd, &imxmd->v4l2_dev.subdevs, list) {
+		/* skip if not a CSI or a video mux */
+		if (!(sd->grp_id & IMX_MEDIA_GRP_ID_IPU_CSI) &&
+		    !(sd->grp_id & IMX_MEDIA_GRP_ID_CSI) &&
+		    sd->entity.function != MEDIA_ENT_F_VID_MUX)

This is a bit dangerous, as the external source connected to the CSI-2 receiver could be a video mux. How about assigning a group id to the internal mux, as done in the "[PATCH 2/2] media: staging/imx: Don't create links between external entities" patch that I've posted, and checking the group id here ?

With that fixed,

Quoted 155 lines
drivers/staging/media/imx/imx-media-dev-common.c +5 -3
+			continue;
+
+		imx_media_create_fwnode_pad_links(csi2, sd);
+	}
 }
 
 /*
@@ -196,9 +186,7 @@ int imx_media_probe_complete(struct v4l2_async_notifier *notifier)
 
 	mutex_lock(&imxmd->mutex);
 
-	ret = imx_media_create_links(notifier);
-	if (ret)
-		goto unlock;
+	imx_media_create_csi2_links(imxmd);
 
 	ret = imx_media_create_pad_vdev_lists(imxmd);
 	if (ret)
drivers/staging/media/imx/imx-media-of.c +0 -114
index 2d3efd2a6dde..82e13e972e23 100644
@@ -74,117 +74,3 @@ int imx_media_add_of_subdevs(struct imx_media_dev *imxmd,
 	return ret;
 }
 EXPORT_SYMBOL_GPL(imx_media_add_of_subdevs);
-
-/*
- * Create a single media link to/from sd using a fwnode link.
- *
- * NOTE: this function assumes an OF port node is equivalent to
- * a media pad (port id equal to media pad index), and that an
- * OF endpoint node is equivalent to a media link.
- */
-static int create_of_link(struct imx_media_dev *imxmd,
-			  struct v4l2_subdev *sd,
-			  struct v4l2_fwnode_link *link)
-{
-	struct v4l2_subdev *remote, *src, *sink;
-	int src_pad, sink_pad;
-
-	if (link->local_port >= sd->entity.num_pads)
-		return -EINVAL;
-
-	remote = imx_media_find_subdev_by_fwnode(imxmd, link->remote_node);
-	if (!remote)
-		return 0;
-
-	if (sd->entity.pads[link->local_port].flags & MEDIA_PAD_FL_SINK) {
-		src = remote;
-		src_pad = link->remote_port;
-		sink = sd;
-		sink_pad = link->local_port;
-	} else {
-		src = sd;
-		src_pad = link->local_port;
-		sink = remote;
-		sink_pad = link->remote_port;
-	}
-
-	/* make sure link doesn't already exist before creating */
-	if (media_entity_find_link(&src->entity.pads[src_pad],
-				   &sink->entity.pads[sink_pad]))
-		return 0;
-
-	v4l2_info(sd->v4l2_dev, "%s:%d -> %s:%d\n",
-		  src->name, src_pad, sink->name, sink_pad);
-
-	return media_create_pad_link(&src->entity, src_pad,
-				     &sink->entity, sink_pad, 0);
-}
-
-/*
- * Create media links to/from sd using its device-tree endpoints.
- */
-int imx_media_create_of_links(struct imx_media_dev *imxmd,
-			      struct v4l2_subdev *sd)
-{
-	struct v4l2_fwnode_link link;
-	struct device_node *ep;
-	int ret;
-
-	for_each_endpoint_of_node(sd->dev->of_node, ep) {
-		ret = v4l2_fwnode_parse_link(of_fwnode_handle(ep), &link);
-		if (ret)
-			continue;
-
-		ret = create_of_link(imxmd, sd, &link);
-		v4l2_fwnode_put_link(&link);
-		if (ret)
-			return ret;
-	}
-
-	return 0;
-}
-EXPORT_SYMBOL_GPL(imx_media_create_of_links);
-
-/*
- * Create media links to the given CSI subdevice's sink pads,
- * using its device-tree endpoints.
- */
-int imx_media_create_csi_of_links(struct imx_media_dev *imxmd,
-				  struct v4l2_subdev *csi)
-{
-	struct device_node *csi_np = csi->dev->of_node;
-	struct device_node *ep;
-
-	for_each_child_of_node(csi_np, ep) {
-		struct fwnode_handle *fwnode, *csi_ep;
-		struct v4l2_fwnode_link link;
-		int ret;
-
-		memset(&link, 0, sizeof(link));
-
-		link.local_node = of_fwnode_handle(csi_np);
-		link.local_port = CSI_SINK_PAD;
-
-		csi_ep = of_fwnode_handle(ep);
-
-		fwnode = fwnode_graph_get_remote_endpoint(csi_ep);
-		if (!fwnode)
-			continue;
-
-		fwnode = fwnode_get_parent(fwnode);
-		fwnode_property_read_u32(fwnode, "reg", &link.remote_port);
-		fwnode = fwnode_get_next_parent(fwnode);
-		if (is_of_node(fwnode) &&
-		    of_node_name_eq(to_of_node(fwnode), "ports"))
-			fwnode = fwnode_get_next_parent(fwnode);
-		link.remote_node = fwnode;
-
-		ret = create_of_link(imxmd, csi, &link);
-		fwnode_handle_put(link.remote_node);
-		if (ret)
-			return ret;
-	}
-
-	return 0;
-}
-EXPORT_SYMBOL_GPL(imx_media_create_csi_of_links);
drivers/staging/media/imx/imx-media.h +0 -4
index 5f23d852122f..5271b84bea9a 100644
@@ -248,10 +248,6 @@ void imx_media_unregister_ipu_internal_subdevs(struct imx_media_dev *imxmd);
 /* imx-media-of.c */
 int imx_media_add_of_subdevs(struct imx_media_dev *dev,
 			     struct device_node *np);
-int imx_media_create_of_links(struct imx_media_dev *imxmd,
-			      struct v4l2_subdev *sd);
-int imx_media_create_csi_of_links(struct imx_media_dev *imxmd,
-				  struct v4l2_subdev *csi);
 int imx_media_of_add_csi(struct imx_media_dev *imxmd,
 			 struct device_node *csi_np);
 
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#33 lore raw

Re: [PATCH v4 15/17] media: imx: Create missing links from CSI-2 receiver

Quoted 86 lines · Laurent Pinchart

Hi Steve,

Thank you for the patch.

Quoted 81 lines · Steve Longerbeam

The entities external to the i.MX6 IPU and i.MX7 now create the links to their fwnode-endpoint connected entities in their notifier bound callbacks. Which means imx_media_create_of_links() and imx_media_create_csi_of_links() are no longer needed and are removed.

However there is still one case in which imx-media needs to create fwnode-endpoint based links at probe completion. The v4l2-async framework does not allow multiple subdevice notifiers to contain a duplicate subdevice in their asd_list. Only the first subdev notifier that discovers and adds that one subdevice to its asd_list will receive a bound callback for it. Other subdevices that also have firmware endpoint connections to this duplicate subdevice will not have it in their asd_list, and thus will never receive a bound callback for it. In the case of imx-media, the one duplicate subdevice in question is the i.MX6 MIPI CSI-2 receiver.

Until there is a solution to that problem, rewrite imx_media_create_links() to add the missing links from the CSI-2 receiver to the CSIs and CSI muxes. The function is renamed imx_media_create_csi2_links().

---
Changes in v3:
- call a local imx-media utility imx_media_create_fwnode_pad_links(). Changes in v2:
- this is a rewrite of v1 "media: imx: Use media_create_fwnode_links for external links", which only adds the missing CSI-2 receiver links.

---
  .../staging/media/imx/imx-media-dev-common.c  |  46 +++----
  drivers/staging/media/imx/imx-media-of.c      | 114 ------------------
  drivers/staging/media/imx/imx-media.h         |   4 -
  3 files changed, 17 insertions(+), 147 deletions(-)
drivers/staging/media/imx/imx-media-dev-common.c +7 -25
index 66b505f7e8df..f7ad3cbbeec2 100644
@@ -30,41 +30,31 @@ static int imx_media_subdev_bound(struct v4l2_async_notifier *notifier,
  }
 
  /*
- * Create the media links for all subdevs that registered.
+ * Create the missing media links from the CSI-2 receiver.
   * Called after all async subdevs have bound.
   */
-static int imx_media_create_links(struct v4l2_async_notifier *notifier)
+static void imx_media_create_csi2_links(struct imx_media_dev *imxmd)
  {
-	struct imx_media_dev *imxmd = notifier2dev(notifier);
-	struct v4l2_subdev *sd;
+	struct v4l2_subdev *sd, *csi2 = NULL;
 
  	list_for_each_entry(sd, &imxmd->v4l2_dev.subdevs, list) {
-		switch (sd->grp_id) {
-		case IMX_MEDIA_GRP_ID_IPU_VDIC:
-		case IMX_MEDIA_GRP_ID_IPU_IC_PRP:
-		case IMX_MEDIA_GRP_ID_IPU_IC_PRPENC:
-		case IMX_MEDIA_GRP_ID_IPU_IC_PRPVF:
-			/*
-			 * links have already been created for the
-			 * sync-registered subdevs.
-			 */
-			break;
-		case IMX_MEDIA_GRP_ID_IPU_CSI0:
-		case IMX_MEDIA_GRP_ID_IPU_CSI1:
-		case IMX_MEDIA_GRP_ID_CSI:
-			imx_media_create_csi_of_links(imxmd, sd);
-			break;
-		default:
-			/*
-			 * if this subdev has fwnode links, create media
-			 * links for them.
-			 */
-			imx_media_create_of_links(imxmd, sd);
+		if (sd->grp_id == IMX_MEDIA_GRP_ID_CSI2) {
+			csi2 = sd;
  			break;
  		}
  	}
+	if (!csi2)
+		return;
 
Quoted 8 lines

drivers/staging/media/imx/imx-media-dev-common.c static int imx_media_subdev_bound(struct v4l2_async_notifier *notifier, · line 53

Quoted 6 lines
drivers/staging/media/imx/imx-media-dev-common.c +5 -1
-	return 0;
+	list_for_each_entry(sd, &imxmd->v4l2_dev.subdevs, list) {
+		/* skip if not a CSI or a video mux */
+		if (!(sd->grp_id & IMX_MEDIA_GRP_ID_IPU_CSI) &&
+		    !(sd->grp_id & IMX_MEDIA_GRP_ID_CSI) &&
+		    sd->entity.function != MEDIA_ENT_F_VID_MUX)

This is a bit dangerous, as the external source connected to the CSI-2 receiver could be a video mux.

Yes I agree it looks dangerous on the face of it, but if there is a video-mux as a source to the csi-2 receiver, the video-mux's .get_fwnode_pad() verifies that the passed fwnode endpoint is actually owned by the video-mux, and imx_media_create_fwnode_pad_link() is only looking for a sink pad... Well, maybe that wasn't clear but if you follow the code you'll see that a weird link won't be created in that case.

But I agree, to make this at least look less dangerous, I'll do what you have done in "[PATCH 2/2] media: staging/imx: Don't create links between external entities" and mark the video-mux in the ipu-csi bound callbacks as one of the imx6 internal video muxes via a imx grp_id.

Steve

Quoted 164 lines

How about assigning a group id to the
internal mux, as done in the "[PATCH 2/2] media: staging/imx: Don't create links between external entities" patch that I've posted, and checking the group id here ?

With that fixed,

Quoted 155 lines
drivers/staging/media/imx/imx-media-dev-common.c +5 -3
+			continue;
+
+		imx_media_create_fwnode_pad_links(csi2, sd);
+	}
  }
 
  /*
@@ -196,9 +186,7 @@ int imx_media_probe_complete(struct v4l2_async_notifier *notifier)
 
  	mutex_lock(&imxmd->mutex);
 
-	ret = imx_media_create_links(notifier);
-	if (ret)
-		goto unlock;
+	imx_media_create_csi2_links(imxmd);
 
  	ret = imx_media_create_pad_vdev_lists(imxmd);
  	if (ret)
drivers/staging/media/imx/imx-media-of.c +0 -114
index 2d3efd2a6dde..82e13e972e23 100644
@@ -74,117 +74,3 @@ int imx_media_add_of_subdevs(struct imx_media_dev *imxmd,
  	return ret;
  }
  EXPORT_SYMBOL_GPL(imx_media_add_of_subdevs);
-
-/*
- * Create a single media link to/from sd using a fwnode link.
- *
- * NOTE: this function assumes an OF port node is equivalent to
- * a media pad (port id equal to media pad index), and that an
- * OF endpoint node is equivalent to a media link.
- */
-static int create_of_link(struct imx_media_dev *imxmd,
-			  struct v4l2_subdev *sd,
-			  struct v4l2_fwnode_link *link)
-{
-	struct v4l2_subdev *remote, *src, *sink;
-	int src_pad, sink_pad;
-
-	if (link->local_port >= sd->entity.num_pads)
-		return -EINVAL;
-
-	remote = imx_media_find_subdev_by_fwnode(imxmd, link->remote_node);
-	if (!remote)
-		return 0;
-
-	if (sd->entity.pads[link->local_port].flags & MEDIA_PAD_FL_SINK) {
-		src = remote;
-		src_pad = link->remote_port;
-		sink = sd;
-		sink_pad = link->local_port;
-	} else {
-		src = sd;
-		src_pad = link->local_port;
-		sink = remote;
-		sink_pad = link->remote_port;
-	}
-
-	/* make sure link doesn't already exist before creating */
-	if (media_entity_find_link(&src->entity.pads[src_pad],
-				   &sink->entity.pads[sink_pad]))
-		return 0;
-
-	v4l2_info(sd->v4l2_dev, "%s:%d -> %s:%d\n",
-		  src->name, src_pad, sink->name, sink_pad);
-
-	return media_create_pad_link(&src->entity, src_pad,
-				     &sink->entity, sink_pad, 0);
-}
-
-/*
- * Create media links to/from sd using its device-tree endpoints.
- */
-int imx_media_create_of_links(struct imx_media_dev *imxmd,
-			      struct v4l2_subdev *sd)
-{
-	struct v4l2_fwnode_link link;
-	struct device_node *ep;
-	int ret;
-
-	for_each_endpoint_of_node(sd->dev->of_node, ep) {
-		ret = v4l2_fwnode_parse_link(of_fwnode_handle(ep), &link);
-		if (ret)
-			continue;
-
-		ret = create_of_link(imxmd, sd, &link);
-		v4l2_fwnode_put_link(&link);
-		if (ret)
-			return ret;
-	}
-
-	return 0;
-}
-EXPORT_SYMBOL_GPL(imx_media_create_of_links);
-
-/*
- * Create media links to the given CSI subdevice's sink pads,
- * using its device-tree endpoints.
- */
-int imx_media_create_csi_of_links(struct imx_media_dev *imxmd,
-				  struct v4l2_subdev *csi)
-{
-	struct device_node *csi_np = csi->dev->of_node;
-	struct device_node *ep;
-
-	for_each_child_of_node(csi_np, ep) {
-		struct fwnode_handle *fwnode, *csi_ep;
-		struct v4l2_fwnode_link link;
-		int ret;
-
-		memset(&link, 0, sizeof(link));
-
-		link.local_node = of_fwnode_handle(csi_np);
-		link.local_port = CSI_SINK_PAD;
-
-		csi_ep = of_fwnode_handle(ep);
-
-		fwnode = fwnode_graph_get_remote_endpoint(csi_ep);
-		if (!fwnode)
-			continue;
-
-		fwnode = fwnode_get_parent(fwnode);
-		fwnode_property_read_u32(fwnode, "reg", &link.remote_port);
-		fwnode = fwnode_get_next_parent(fwnode);
-		if (is_of_node(fwnode) &&
-		    of_node_name_eq(to_of_node(fwnode), "ports"))
-			fwnode = fwnode_get_next_parent(fwnode);
-		link.remote_node = fwnode;
-
-		ret = create_of_link(imxmd, csi, &link);
-		fwnode_handle_put(link.remote_node);
-		if (ret)
-			return ret;
-	}
-
-	return 0;
-}
-EXPORT_SYMBOL_GPL(imx_media_create_csi_of_links);
drivers/staging/media/imx/imx-media.h +0 -4
index 5f23d852122f..5271b84bea9a 100644
@@ -248,10 +248,6 @@ void imx_media_unregister_ipu_internal_subdevs(struct imx_media_dev *imxmd);
  /* imx-media-of.c */
  int imx_media_add_of_subdevs(struct imx_media_dev *dev,
  			     struct device_node *np);
-int imx_media_create_of_links(struct imx_media_dev *imxmd,
-			      struct v4l2_subdev *sd);
-int imx_media_create_csi_of_links(struct imx_media_dev *imxmd,
-				  struct v4l2_subdev *csi);
  int imx_media_of_add_csi(struct imx_media_dev *imxmd,
  			 struct device_node *csi_np);
 
author maintainer
#34 lore raw

PATCH v4 16/17 media: imx: silence a couple debug messages

In reply to Steve Longerbeam

Convert to dev_dbg the "subdev bound" and IPU-internal media-link creation messages.

---
 drivers/staging/media/imx/imx-media-dev-common.c  | 4 +++-
 drivers/staging/media/imx/imx-media-dev.c         | 2 +-
 drivers/staging/media/imx/imx-media-internal-sd.c | 6 +++---
 3 files changed, 7 insertions(+), 5 deletions(-)
drivers/staging/media/imx/imx-media-dev-common.c +3 -1
index f7ad3cbbeec2..b23bbfab388a 100644
@@ -24,7 +24,9 @@ static int imx_media_subdev_bound(struct v4l2_async_notifier *notifier,
 				  struct v4l2_subdev *sd,
 				  struct v4l2_async_subdev *asd)
 {
-	v4l2_info(sd->v4l2_dev, "subdev %s bound\n", sd->name);
+	struct imx_media_dev *imxmd = notifier2dev(notifier);
+
+	dev_dbg(imxmd->md.dev, "subdev %s bound\n", sd->name);
 
 	return 0;
 }
drivers/staging/media/imx/imx-media-dev.c +1 -1
index 2c3c2adca683..6d2205461e56 100644
@@ -32,7 +32,7 @@ static int imx_media_subdev_bound(struct v4l2_async_notifier *notifier,
 			return ret;
 	}
 
-	v4l2_info(&imxmd->v4l2_dev, "subdev %s bound\n", sd->name);
+	dev_dbg(imxmd->md.dev, "subdev %s bound\n", sd->name);
 
 	return 0;
 }
drivers/staging/media/imx/imx-media-internal-sd.c +3 -3
index d4237e1a4241..da4109b2fd13 100644
@@ -142,9 +142,9 @@ static int create_internal_link(struct imx_media_dev *imxmd,
 				   &sink->entity.pads[link->remote_pad]))
 		return 0;
 
-	v4l2_info(&imxmd->v4l2_dev, "%s:%d -> %s:%d\n",
-		  src->name, link->local_pad,
-		  sink->name, link->remote_pad);
+	dev_dbg(imxmd->md.dev, "%s:%d -> %s:%d\n",
+		src->name, link->local_pad,
+		sink->name, link->remote_pad);
 
 	ret = media_create_pad_link(&src->entity, link->local_pad,
 				    &sink->entity, link->remote_pad, 0);
Signature · 1 line
2.17.1
maintainer Reviewed-by on v4 16/17
#35 lore raw

Re: [PATCH v4 16/17] media: imx: silence a couple debug messages

Hi Steve,

Thank you for the patch.

Quoted 4 lines · Steve Longerbeam

Convert to dev_dbg the "subdev bound" and IPU-internal media-link creation messages.

Quoted 51 lines
---
 drivers/staging/media/imx/imx-media-dev-common.c  | 4 +++-
 drivers/staging/media/imx/imx-media-dev.c         | 2 +-
 drivers/staging/media/imx/imx-media-internal-sd.c | 6 +++---
 3 files changed, 7 insertions(+), 5 deletions(-)
drivers/staging/media/imx/imx-media-dev-common.c +3 -1
index f7ad3cbbeec2..b23bbfab388a 100644
@@ -24,7 +24,9 @@ static int imx_media_subdev_bound(struct v4l2_async_notifier *notifier,
 				  struct v4l2_subdev *sd,
 				  struct v4l2_async_subdev *asd)
 {
-	v4l2_info(sd->v4l2_dev, "subdev %s bound\n", sd->name);
+	struct imx_media_dev *imxmd = notifier2dev(notifier);
+
+	dev_dbg(imxmd->md.dev, "subdev %s bound\n", sd->name);
 
 	return 0;
 }
drivers/staging/media/imx/imx-media-dev.c +1 -1
index 2c3c2adca683..6d2205461e56 100644
@@ -32,7 +32,7 @@ static int imx_media_subdev_bound(struct v4l2_async_notifier *notifier,
 			return ret;
 	}
 
-	v4l2_info(&imxmd->v4l2_dev, "subdev %s bound\n", sd->name);
+	dev_dbg(imxmd->md.dev, "subdev %s bound\n", sd->name);
 
 	return 0;
 }
drivers/staging/media/imx/imx-media-internal-sd.c +3 -3
index d4237e1a4241..da4109b2fd13 100644
@@ -142,9 +142,9 @@ static int create_internal_link(struct imx_media_dev *imxmd,
 				   &sink->entity.pads[link->remote_pad]))
 		return 0;
 
-	v4l2_info(&imxmd->v4l2_dev, "%s:%d -> %s:%d\n",
-		  src->name, link->local_pad,
-		  sink->name, link->remote_pad);
+	dev_dbg(imxmd->md.dev, "%s:%d -> %s:%d\n",
+		src->name, link->local_pad,
+		sink->name, link->remote_pad);
 
 	ret = media_create_pad_link(&src->entity, link->local_pad,
 				    &sink->entity, link->remote_pad, 0);
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#36 lore raw

PATCH v4 17/17 media: imx: TODO: Remove media link creation todos

In reply to Steve Longerbeam

Remove the TODO items regarding media link creation, these issues are resolved by moving media link creation to individual entity bound callbacks and the implementation of the get_fwnode_pad operation.

---
 drivers/staging/media/imx/TODO | 29 -----------------------------
 1 file changed, 29 deletions(-)
drivers/staging/media/imx/TODO +0 -29
index 6f29b5ca5324..a371cdedcdb0 100644
@@ -17,35 +17,6 @@
   decided whether this feature is useful enough to make it generally
   available by exporting to v4l2-core.
 
-- After all async subdevices have been bound, v4l2_fwnode_parse_link()
-  is used to form the media links between the devices discovered in
-  the OF graph.
-
-  While this approach allows support for arbitrary OF graphs, there
-  are some assumptions for this to work:
-
-  1. If a port owned by a device in the graph has endpoint nodes, the
-     port is treated as a media pad.
-
-     This presents problems for devices that don't make this port = pad
-     assumption. Examples are SMIAPP compatible cameras which define only
-     a single output port node, but which define multiple pads owned
-     by multiple subdevices (pixel-array, binner, scaler). Or video
-     decoders (entity function MEDIA_ENT_F_ATV_DECODER), which also define
-     only a single output port node, but define multiple pads for video,
-     VBI, and audio out.
-
-     A workaround at present is to set the port reg properties to
-     correspond to the media pad index that the port represents. A
-     possible long-term solution is to implement a subdev API that
-     maps a port id to a media pad index.
-
-  2. Every endpoint of a port owned by a device in the graph is treated
-     as a media link.
-
-     Which means a port must not contain mixed-use endpoints, they
-     must all refer to media links between V4L2 subdevices.
-
 - i.MX7: all of the above, since it uses the imx media core
 
 - i.MX7: use Frame Interval Monitor
Signature · 1 line
2.17.1
author maintainer
#37 lore raw
In reply to Steve Longerbeam

Hi Sakari,

When you have time can you review this series.

TIA!

Steve

Quoted 69 lines · Steve Longerbeam

Move media link creation into the notifier bound callbacks in the minimum set of subdevices required by imx (imx5/6/7 CSI, imx6/7 MIPI CSI-2, and video mux).

History:

v4:
- Removed the endpoint parsing callback APIs from video-mux and imx drivers as suggested by Sakari, replacing with endpoint parsing local to the drivers and the use of v4l2_async_notifier_add_fwnode_remote_subdev(). As a result convenience function v4l2_async_register_fwnode_subdev() is no longer used and is reverted.

v3:
- The changes to the default behaviour in media_entity_get_fwnode_pad(), and the fixes to current media drivers that call it inconsistently, have been put-off to another time. Instead this version implements the get_fwnode_pad operation where required in the imx and video-mux subdevices to make media link creation work correctly. The improvements to media_entity_get_fwnode_pad() can wait to another patch series.

v2:
- rename/move the notifier-to-state inlines in imx7-mipi-csis.c and imx7-media-csi.c, suggested by Rui Silva.
- rewrite imx_media_create_links() to only add the missing media links from the imx6 MIPI CSI-2 receiver.

Steve Longerbeam (17):
media: entity: Pass entity to get_fwnode_pad operation
media: video-mux: Parse information from firmware without using
callbacks
media: imx: Parse information from firmware without using callbacks Revert "media: v4l2-fwnode: Add a convenience function for registering
subdevs with notifiers"
media: imx: csi: Implement get_fwnode_pad op
media: imx: mipi csi-2: Implement get_fwnode_pad op
media: video-mux: Implement get_fwnode_pad op
media: imx: Add imx_media_create_fwnode_pad_link()
media: video-mux: Create media links in bound notifier
media: imx: mipi csi-2: Create media links in bound notifier
media: imx7: mipi csis: Create media links in bound notifier
media: imx7: csi: Create media links in bound notifier
media: imx: csi: Create media links in bound notifier
media: imx: csi: Lookup upstream endpoint with
imx_media_get_pad_fwnode
media: imx: Create missing links from CSI-2 receiver
media: imx: silence a couple debug messages
media: imx: TODO: Remove media link creation todos

  drivers/media/mc/mc-entity.c                  |   2 +-
  drivers/media/platform/video-mux.c            | 185 ++++++++++++++++--
  drivers/media/v4l2-core/v4l2-fwnode.c         |  62 ------
  drivers/staging/media/imx/TODO                |  29 ---
  drivers/staging/media/imx/imx-media-csi.c     | 136 ++++++++-----
  .../staging/media/imx/imx-media-dev-common.c  |  50 ++---
  drivers/staging/media/imx/imx-media-dev.c     |   2 +-
  .../staging/media/imx/imx-media-internal-sd.c |   6 +-
  drivers/staging/media/imx/imx-media-of.c      | 114 -----------
  drivers/staging/media/imx/imx-media-utils.c   | 124 ++++++++++++
  drivers/staging/media/imx/imx-media.h         |   9 +-
  drivers/staging/media/imx/imx6-mipi-csi2.c    | 119 +++++++++--
  drivers/staging/media/imx/imx7-media-csi.c    | 100 +++++++---
  drivers/staging/media/imx/imx7-mipi-csis.c    | 105 +++++++---
  include/media/media-entity.h                  |   3 +-
  include/media/v4l2-fwnode.h                   |  38 ----
  16 files changed, 654 insertions(+), 430 deletions(-)