Skip to contentSkip to conversation

media: i2c: ov5640: Implement get_mbus_config

PATCH In review Marcel Ziswiler 9 participants 21 messages Mar 6, 2023 – Mar 22, 2023

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 media: i2c: ov5640: Implement get_mbus_config

From: Aishwarya Kothari <[email protected]>

Implement the introduced get_mbus_config operation to report the config of the MIPI CSI-2, BT.656 and Parallel interface.

---

Changes in v2:
- Take care of MIPI CSI-2, BT.656 and Parallel interface as pointed out by Jacopo. Thanks!

 drivers/media/i2c/ov5640.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
drivers/media/i2c/ov5640.c +19 -0
index 1536649b9e90..43373416fcba 100644
@@ -3774,6 +3774,24 @@ static int ov5640_init_cfg(struct v4l2_subdev *sd,
 	return 0;
 }
 
+static int ov5640_get_mbus_config(struct v4l2_subdev *sd,
+				   unsigned int pad,
+				   struct v4l2_mbus_config *cfg)
+{
+	struct ov5640_dev *sensor = to_ov5640_dev(sd);
+
+	cfg->type = sensor->ep.bus_type;
+	if (ov5640_is_csi2(sensor)) {
+		cfg->bus.mipi_csi2.num_data_lanes =
+			sensor->ep.bus.mipi_csi2.num_data_lanes;
+		cfg->bus.mipi_csi2.flags = sensor->ep.bus.mipi_csi2.flags;
+	} else {
+		cfg->bus.parallel.flags = sensor->ep.bus.parallel.flags;
+	}
+
+	return 0;
+}
+
 static const struct v4l2_subdev_core_ops ov5640_core_ops = {
 	.log_status = v4l2_ctrl_subdev_log_status,
 	.subscribe_event = v4l2_ctrl_subdev_subscribe_event,
@@ -3794,6 +3812,7 @@ static const struct v4l2_subdev_pad_ops ov5640_pad_ops = {
 	.get_selection = ov5640_get_selection,
 	.enum_frame_size = ov5640_enum_frame_size,
 	.enum_frame_interval = ov5640_enum_frame_interval,
+	.get_mbus_config = ov5640_get_mbus_config,
 };
 
 static const struct v4l2_subdev_ops ov5640_subdev_ops = {
Signature · 1 line
2.36.1
#1 lore raw

Hi Marcel,

Quoted 47 lines · Marcel Ziswiler

From: Aishwarya Kothari <[email protected]>

Implement the introduced get_mbus_config operation to report the config of the MIPI CSI-2, BT.656 and Parallel interface.

---

Changes in v2:
- Take care of MIPI CSI-2, BT.656 and Parallel interface as pointed out by Jacopo. Thanks!

 drivers/media/i2c/ov5640.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
drivers/media/i2c/ov5640.c +18 -0
index 1536649b9e90..43373416fcba 100644
@@ -3774,6 +3774,24 @@ static int ov5640_init_cfg(struct v4l2_subdev *sd,
 	return 0;
 }
 
+static int ov5640_get_mbus_config(struct v4l2_subdev *sd,
+				   unsigned int pad,
+				   struct v4l2_mbus_config *cfg)
+{
+	struct ov5640_dev *sensor = to_ov5640_dev(sd);
+
+	cfg->type = sensor->ep.bus_type;
+	if (ov5640_is_csi2(sensor)) {
+		cfg->bus.mipi_csi2.num_data_lanes =
+			sensor->ep.bus.mipi_csi2.num_data_lanes;
+		cfg->bus.mipi_csi2.flags = sensor->ep.bus.mipi_csi2.flags;
+	} else {
+		cfg->bus.parallel.flags = sensor->ep.bus.parallel.flags;
+	}
+
+	return 0;
+}
+
 static const struct v4l2_subdev_core_ops ov5640_core_ops = {
 	.log_status = v4l2_ctrl_subdev_log_status,
 	.subscribe_event = v4l2_ctrl_subdev_subscribe_event,

drivers/media/i2c/ov5640.c static const struct v4l2_subdev_pad_ops ov5640_pad_ops = { · line 3815

Quoted 4 lines
drivers/media/i2c/ov5640.c +1 -0
@@ -3794,6 +3812,7 @@ static const struct v4l2_subdev_pad_ops ov5640_pad_ops = {
 	.get_selection = ov5640_get_selection,
 	.enum_frame_size = ov5640_enum_frame_size,
 	.enum_frame_interval = ov5640_enum_frame_interval,
+	.get_mbus_config = ov5640_get_mbus_config,

What's the reasoning for this patch?

Drivers that don't have e.g. dynamic lane configuration shouldn't need to implement get_mbus_config.

Signature · 3 lines
Kind regards,

Sakari Ailus
#2 lore raw

Hello Sakari,

Quoted 46 lines · Sakari Ailus
Quoted 45 lines · Marcel Ziswiler

From: Aishwarya Kothari <[email protected]>

Implement the introduced get_mbus_config operation to report the config of the MIPI CSI-2, BT.656 and Parallel interface.

---

Changes in v2:
- Take care of MIPI CSI-2, BT.656 and Parallel interface as pointed out by Jacopo. Thanks!

 drivers/media/i2c/ov5640.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
drivers/media/i2c/ov5640.c +18 -0
index 1536649b9e90..43373416fcba 100644
@@ -3774,6 +3774,24 @@ static int ov5640_init_cfg(struct v4l2_subdev *sd,
 	return 0;
 }
 
+static int ov5640_get_mbus_config(struct v4l2_subdev *sd,
+				   unsigned int pad,
+				   struct v4l2_mbus_config *cfg)
+{
+	struct ov5640_dev *sensor = to_ov5640_dev(sd);
+
+	cfg->type = sensor->ep.bus_type;
+	if (ov5640_is_csi2(sensor)) {
+		cfg->bus.mipi_csi2.num_data_lanes =
+			sensor->ep.bus.mipi_csi2.num_data_lanes;
+		cfg->bus.mipi_csi2.flags = sensor->ep.bus.mipi_csi2.flags;
+	} else {
+		cfg->bus.parallel.flags = sensor->ep.bus.parallel.flags;
+	}
+
+	return 0;
+}
+
 static const struct v4l2_subdev_core_ops ov5640_core_ops = {
 	.log_status = v4l2_ctrl_subdev_log_status,
Quoted 8 lines

drivers/media/i2c/ov5640.c static const struct v4l2_subdev_pad_ops ov5640_pad_ops = { · line 3815

Quoted 6 lines
drivers/media/i2c/ov5640.c +1 -0
 	.subscribe_event = v4l2_ctrl_subdev_subscribe_event,
@@ -3794,6 +3812,7 @@ static const struct v4l2_subdev_pad_ops ov5640_pad_ops = {
 	.get_selection = ov5640_get_selection,
 	.enum_frame_size = ov5640_enum_frame_size,
 	.enum_frame_interval = ov5640_enum_frame_interval,
+	.get_mbus_config = ov5640_get_mbus_config,

What's the reasoning for this patch?

Without this it's not possible to use it on i.MX6, drivers/staging/media/imx/imx6-mipi-csi2.c requires it, some more details from Jacopo here [0].

Everything used to work fine up to v5.18, after that kernel version various changes broke it [1][2] (I assume you are pretty much aware of the history here, you commented on a few emails).

[0] https://lore.kernel.org/all/[email protected]/
[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/

Quoted 2 lines

Drivers that don't have e.g. dynamic lane configuration shouldn't need to implement get_mbus_config.

Francesco

#3 lore raw

Hi Francesco,

Quoted 62 lines · Francesco Dolcini

Hello Sakari,

Quoted 46 lines · Sakari Ailus
Quoted 45 lines · Marcel Ziswiler

From: Aishwarya Kothari <[email protected]>

Implement the introduced get_mbus_config operation to report the config of the MIPI CSI-2, BT.656 and Parallel interface.

---

Changes in v2:
- Take care of MIPI CSI-2, BT.656 and Parallel interface as pointed out by Jacopo. Thanks!

 drivers/media/i2c/ov5640.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
drivers/media/i2c/ov5640.c +18 -0
index 1536649b9e90..43373416fcba 100644
@@ -3774,6 +3774,24 @@ static int ov5640_init_cfg(struct v4l2_subdev *sd,
 	return 0;
 }
 
+static int ov5640_get_mbus_config(struct v4l2_subdev *sd,
+				   unsigned int pad,
+				   struct v4l2_mbus_config *cfg)
+{
+	struct ov5640_dev *sensor = to_ov5640_dev(sd);
+
+	cfg->type = sensor->ep.bus_type;
+	if (ov5640_is_csi2(sensor)) {
+		cfg->bus.mipi_csi2.num_data_lanes =
+			sensor->ep.bus.mipi_csi2.num_data_lanes;
+		cfg->bus.mipi_csi2.flags = sensor->ep.bus.mipi_csi2.flags;
+	} else {
+		cfg->bus.parallel.flags = sensor->ep.bus.parallel.flags;
+	}
+
+	return 0;
+}
+
 static const struct v4l2_subdev_core_ops ov5640_core_ops = {
 	.log_status = v4l2_ctrl_subdev_log_status,
Quoted 8 lines

drivers/media/i2c/ov5640.c static const struct v4l2_subdev_pad_ops ov5640_pad_ops = { · line 3815

Quoted 6 lines
drivers/media/i2c/ov5640.c +1 -0
 	.subscribe_event = v4l2_ctrl_subdev_subscribe_event,
@@ -3794,6 +3812,7 @@ static const struct v4l2_subdev_pad_ops ov5640_pad_ops = {
 	.get_selection = ov5640_get_selection,
 	.enum_frame_size = ov5640_enum_frame_size,
 	.enum_frame_interval = ov5640_enum_frame_interval,
+	.get_mbus_config = ov5640_get_mbus_config,

What's the reasoning for this patch?

Without this it's not possible to use it on i.MX6, drivers/staging/media/imx/imx6-mipi-csi2.c requires it, some more details from Jacopo here [0].

Quoted 10 lines

Everything used to work fine up to v5.18, after that kernel version various changes broke it [1][2] (I assume you are pretty much aware of the history here, you commented on a few emails).

[0] https://lore.kernel.org/all/[email protected]/
[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/

Quoted 2 lines

Drivers that don't have e.g. dynamic lane configuration shouldn't need to implement get_mbus_config.

Not even for staging drivers. The driver should be fixed to get that information from the endpoint instead.

I don't object having a helper in the framework to do this though. There are many receiver drivers that need this to work with those devices that have variable number of lanes.

Signature · 3 lines
Regards,

Sakari Ailus
#4 lore raw

+Marco

Quoted 25 lines · Sakari Ailus
Quoted 24 lines · Francesco Dolcini
Quoted 10 lines · Sakari Ailus
Quoted 7 lines · Marcel Ziswiler

From: Aishwarya Kothari <[email protected]>

Implement the introduced get_mbus_config operation to report the config of the MIPI CSI-2, BT.656 and Parallel interface.

What's the reasoning for this patch?

Without this it's not possible to use it on i.MX6, drivers/staging/media/imx/imx6-mipi-csi2.c requires it, some more details from Jacopo here [0].

Everything used to work fine up to v5.18, after that kernel version various changes broke it [1][2] (I assume you are pretty much aware of the history here, you commented on a few emails).

[0] https://lore.kernel.org/all/[email protected]/
[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/

Quoted 5 lines
Quoted 2 lines
Quoted 2 lines

Drivers that don't have e.g. dynamic lane configuration shouldn't need to implement get_mbus_config.

Not even for staging drivers. The driver should be fixed to get that information from the endpoint instead.

This seems exactly the opposite of what commit
7318abface48 ("media: imx: Use get_mbus_config instead of parsing upstream DT endpoints") did.

Given that I am somehow confused, but I am not that familiar with this subsystem, so I guess this is expected :-). Can someone provide some additional hint here?

Quoted 3 lines

I don't object having a helper in the framework to do this though. There are many receiver drivers that need this to work with those devices that have variable number of lanes.

Thanks in advance,
Francesco

#5 lore raw

Hello

Quoted 34 lines · Francesco Dolcini

+Marco

Quoted 25 lines · Sakari Ailus
Quoted 24 lines · Francesco Dolcini
Quoted 10 lines · Sakari Ailus
Quoted 7 lines · Marcel Ziswiler

From: Aishwarya Kothari <[email protected]>

Implement the introduced get_mbus_config operation to report the config of the MIPI CSI-2, BT.656 and Parallel interface.

What's the reasoning for this patch?

Without this it's not possible to use it on i.MX6, drivers/staging/media/imx/imx6-mipi-csi2.c requires it, some more details from Jacopo here [0].

Everything used to work fine up to v5.18, after that kernel version various changes broke it [1][2] (I assume you are pretty much aware of the history here, you commented on a few emails).

[0] https://lore.kernel.org/all/[email protected]/
[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/

Quoted 5 lines
Quoted 2 lines
Quoted 2 lines

Drivers that don't have e.g. dynamic lane configuration shouldn't need to implement get_mbus_config.

Not even for staging drivers. The driver should be fixed to get that information from the endpoint instead.

Quoted 8 lines

This seems exactly the opposite of what commit
7318abface48 ("media: imx: Use get_mbus_config instead of parsing upstream DT endpoints") did.

Given that I am somehow confused, but I am not that familiar with this subsystem, so I guess this is expected :-). Can someone provide some additional hint here?

As per my understanding, the i.MX6 IPU CSI driver connects to the CSI-2 receiver and/or two video muxes. One figure's worth a thousands
words: "Figure 19-1. CSI2IPU gasket connectivity" of the IMX6DQRM TRM.

So the local endpoint might not provide the required information on the bus configuration as it connects to a video-mux.

That's why the imx_media_pipeline_subdev() helper is used in csi_get_upstream_mbus_config().

My gut feeling is that it would be better to always call get_mbus_config() on the next subdev (the mux or the CSI-2 rx) and there parse the local endpoint as it's the mux or the CSI-2 rx that connect to the actual source.

To be honest my understanding is that this patch has always been needed to work on imx6 and this is not a regression but something that was kept as an out-of-tree patch downstream. Is this correct or is this a regression ? The above mentioned patches that move fwnode matching to endpoints don't seem related, don't they ?

Thanks
j

Quoted 7 lines
Quoted 3 lines

I don't object having a helper in the framework to do this though. There are many receiver drivers that need this to work with those devices that have variable number of lanes.

Thanks in advance,
Francesco

maintainer
#6 lore raw

Hi Jacopo,

Quoted 41 lines · Jacopo Mondi

Hello

Quoted 38 lines · Francesco Dolcini

+Marco

Quoted 25 lines · Sakari Ailus
Quoted 24 lines · Francesco Dolcini
Quoted 10 lines · Sakari Ailus
Quoted 7 lines · Marcel Ziswiler

From: Aishwarya Kothari <[email protected]>

Implement the introduced get_mbus_config operation to report the config of the MIPI CSI-2, BT.656 and Parallel interface.

What's the reasoning for this patch?

Without this it's not possible to use it on i.MX6, drivers/staging/media/imx/imx6-mipi-csi2.c requires it, some more details from Jacopo here [0].

Everything used to work fine up to v5.18, after that kernel version various changes broke it [1][2] (I assume you are pretty much aware of the history here, you commented on a few emails).

[0] https://lore.kernel.org/all/[email protected]/
[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/

Quoted 5 lines
Quoted 2 lines
Quoted 2 lines

Drivers that don't have e.g. dynamic lane configuration shouldn't need to implement get_mbus_config.

Not even for staging drivers. The driver should be fixed to get that information from the endpoint instead.

This seems exactly the opposite of what commit
7318abface48 ("media: imx: Use get_mbus_config instead of parsing upstream DT endpoints") did.

Quoted 8 lines
Quoted 4 lines

Given that I am somehow confused, but I am not that familiar with this subsystem, so I guess this is expected :-). Can someone provide some additional hint here?

As per my understanding, the i.MX6 IPU CSI driver connects to the CSI-2 receiver and/or two video muxes. One figure's worth a thousands
words: "Figure 19-1. CSI2IPU gasket connectivity" of the IMX6DQRM TRM.

I don't have that document.

Quoted 11 lines

So the local endpoint might not provide the required information on the bus configuration as it connects to a video-mux.

That's why the imx_media_pipeline_subdev() helper is used in csi_get_upstream_mbus_config().

My gut feeling is that it would be better to always call get_mbus_config() on the next subdev (the mux or the CSI-2 rx) and there parse the local endpoint as it's the mux or the CSI-2 rx that connect to the actual source.

Isn't this still a different endpoint in DT? I understand you have a single pad with two links?

Signature · 3 lines
Kind regards,

Sakari Ailus
#7 lore raw
Quoted 43 lines · Sakari Ailus
Quoted 42 lines · Jacopo Mondi
Quoted 40 lines · Francesco Dolcini
Quoted 25 lines · Sakari Ailus
Quoted 24 lines · Francesco Dolcini
Quoted 10 lines · Sakari Ailus
Quoted 7 lines · Marcel Ziswiler

From: Aishwarya Kothari <[email protected]>

Implement the introduced get_mbus_config operation to report the config of the MIPI CSI-2, BT.656 and Parallel interface.

What's the reasoning for this patch?

Without this it's not possible to use it on i.MX6, drivers/staging/media/imx/imx6-mipi-csi2.c requires it, some more details from Jacopo here [0].

Everything used to work fine up to v5.18, after that kernel version various changes broke it [1][2] (I assume you are pretty much aware of the history here, you commented on a few emails).

[0] https://lore.kernel.org/all/[email protected]/
[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/

Quoted 5 lines
Quoted 2 lines
Quoted 2 lines

Drivers that don't have e.g. dynamic lane configuration shouldn't need to implement get_mbus_config.

Not even for staging drivers. The driver should be fixed to get that information from the endpoint instead.

This seems exactly the opposite of what commit
7318abface48 ("media: imx: Use get_mbus_config instead of parsing upstream DT endpoints") did.

Given that I am somehow confused, but I am not that familiar with this subsystem, so I guess this is expected :-). Can someone provide some additional hint here?

Quoted 5 lines
Quoted 3 lines

As per my understanding, the i.MX6 IPU CSI driver connects to the CSI-2 receiver and/or two video muxes. One figure's worth a thousands
words: "Figure 19-1. CSI2IPU gasket connectivity" of the IMX6DQRM TRM.

I don't have that document.

https://www.nxp.com/webapp/Download?colCode=IMX6DQRM

You'll need a user account on nxp.com though.

Quoted 6 lines
Quoted 6 lines

So the local endpoint might not provide the required information on the bus configuration as it connects to a video-mux.

That's why the imx_media_pipeline_subdev() helper is used in csi_get_upstream_mbus_config().

Quoted 7 lines
Quoted 4 lines

My gut feeling is that it would be better to always call get_mbus_config() on the next subdev (the mux or the CSI-2 rx) and there parse the local endpoint as it's the mux or the CSI-2 rx that connect to the actual source.

Isn't this still a different endpoint in DT? I understand you have a single pad with two links?

In a (simplified) nutshell,

---------+     +----------+     +---------+     +-----+     +-----+
| Camera | --> | CSI-2 RX | --> | CSI2IPU | --> | Mux | --> | IPU |
| Sensor |     |          |     | Gasket  |     |     |     |     |
---------+     +----------+     +---------+     +-----+     +-----+

All those blocks, except for the gasket, have a node in DT.

The IPU driver needs to know the number of CSI-2 data lanes, which is encoded in the data-lanes DT property present in both the sensor output endpoint and the CSI-2 RX input endpoint, but not the other endpoints in the pipeline.

Signature · 3 lines
Regards,

Laurent Pinchart
maintainer
#8 lore raw

Hi Laurent,

Quoted 6 lines · Laurent Pinchart

In a (simplified) nutshell,

---------+     +----------+     +---------+     +-----+     +-----+
| Camera | --> | CSI-2 RX | --> | CSI2IPU | --> | Mux | --> | IPU |
| Sensor |     |          |     | Gasket  |     |     |     |     |
---------+     +----------+     +---------+     +-----+     +-----+

Thank you, this is helpful.

I suppose the mux here at least won't actively do anything to the data. So presumably its endpoint won't contain the active configuration, but its superset.

Quoted 7 lines

All those blocks, except for the gasket, have a node in DT.

The IPU driver needs to know the number of CSI-2 data lanes, which is encoded in the data-lanes DT property present in both the sensor output endpoint and the CSI-2 RX input endpoint, but not the other endpoints in the pipeline.

This doesn't yet explain why the sensor would need to implement get_mbus_config if its bus configuration remains constant.

I suppose those blocks in between would probably need something to convey their active configuration from upstream sub-devices.

Signature · 3 lines
Kind regards,

Sakari Ailus
#9 lore raw
Quoted 19 lines · Sakari Ailus

Hi Laurent,

Quoted 6 lines · Laurent Pinchart

In a (simplified) nutshell,

---------+     +----------+     +---------+     +-----+     +-----+
| Camera | --> | CSI-2 RX | --> | CSI2IPU | --> | Mux | --> | IPU |
| Sensor |     |          |     | Gasket  |     |     |     |     |
---------+     +----------+     +---------+     +-----+     +-----+

Thank you, this is helpful.

I suppose the mux here at least won't actively do anything to the data. So presumably its endpoint won't contain the active configuration, but its superset.

Quoted 3 lines

All those blocks, except for the gasket, have a node in DT.

Quoted 7 lines
Quoted 4 lines

The IPU driver needs to know the number of CSI-2 data lanes, which is encoded in the data-lanes DT property present in both the sensor output endpoint and the CSI-2 RX input endpoint, but not the other endpoints in the pipeline.

This doesn't yet explain why the sensor would need to implement get_mbus_config if its bus configuration remains constant.

If I recall correctly, the IPU driver calls .g_mbus_config() on the camera sensor to get the number of lanes, as it can't get it from its own endpoint. That's a hack, and as Jacopo proposed, calling .g_mbus_config() on the CSI-2 RX would be better, as the CSI-2 RX driver can then get the value from its own endpoint, without requiring all sensor drivers to implement .g_mbus_config().

Quoted 2 lines

I suppose those blocks in between would probably need something to convey their active configuration from upstream sub-devices.

Signature · 3 lines
Regards,

Laurent Pinchart
maintainer
#10 lore raw

Hi Laurent,

Quoted 28 lines · Laurent Pinchart
Quoted 19 lines · Sakari Ailus

Hi Laurent,

Quoted 6 lines · Laurent Pinchart

In a (simplified) nutshell,

---------+     +----------+     +---------+     +-----+     +-----+
| Camera | --> | CSI-2 RX | --> | CSI2IPU | --> | Mux | --> | IPU |
| Sensor |     |          |     | Gasket  |     |     |     |     |
---------+     +----------+     +---------+     +-----+     +-----+

Thank you, this is helpful.

I suppose the mux here at least won't actively do anything to the data. So presumably its endpoint won't contain the active configuration, but its superset.

Quoted 3 lines

All those blocks, except for the gasket, have a node in DT.

Quoted 7 lines
Quoted 4 lines

The IPU driver needs to know the number of CSI-2 data lanes, which is encoded in the data-lanes DT property present in both the sensor output endpoint and the CSI-2 RX input endpoint, but not the other endpoints in the pipeline.

This doesn't yet explain why the sensor would need to implement get_mbus_config if its bus configuration remains constant.

Quoted 6 lines

If I recall correctly, the IPU driver calls .g_mbus_config() on the camera sensor to get the number of lanes, as it can't get it from its own endpoint. That's a hack, and as Jacopo proposed, calling .g_mbus_config() on the CSI-2 RX would be better, as the CSI-2 RX driver can then get the value from its own endpoint, without requiring all sensor drivers to implement .g_mbus_config().

The g_mbus_config op could be implemented by the CSI2IPU and mux, by simply requesting the information from the upstream sub-device. No hacks would be needed.

Signature · 3 lines
Kind regrads,

Sakari Ailus
#11 lore raw
Quoted 38 lines · Sakari Ailus

Hi Laurent,

Quoted 28 lines · Laurent Pinchart
Quoted 19 lines · Sakari Ailus

Hi Laurent,

Quoted 6 lines · Laurent Pinchart

In a (simplified) nutshell,

---------+     +----------+     +---------+     +-----+     +-----+
> Camera | --> | CSI-2 RX | --> | CSI2IPU | --> | Mux | --> | IPU |
> Sensor |     |          |     | Gasket  |     |     |     |     |
---------+     +----------+     +---------+     +-----+     +-----+

Thank you, this is helpful.

I suppose the mux here at least won't actively do anything to the data. So presumably its endpoint won't contain the active configuration, but its superset.

Quoted 3 lines

All those blocks, except for the gasket, have a node in DT.

Quoted 7 lines
Quoted 4 lines

The IPU driver needs to know the number of CSI-2 data lanes, which is encoded in the data-lanes DT property present in both the sensor output endpoint and the CSI-2 RX input endpoint, but not the other endpoints in the pipeline.

This doesn't yet explain why the sensor would need to implement get_mbus_config if its bus configuration remains constant.

Quoted 6 lines

If I recall correctly, the IPU driver calls .g_mbus_config() on the camera sensor to get the number of lanes, as it can't get it from its own endpoint. That's a hack, and as Jacopo proposed, calling .g_mbus_config() on the CSI-2 RX would be better, as the CSI-2 RX driver can then get the value from its own endpoint, without requiring all sensor drivers to implement .g_mbus_config().

Quoted 3 lines

The g_mbus_config op could be implemented by the CSI2IPU and mux, by simply requesting the information from the upstream sub-device. No hacks would be needed.

I think implementing get_mbus_config on the mux might be enough. The IPU driver already recognizes the CSI-2 RX by its grp_id and could infer that it has to use MIPI CSI-2 mode from that.

The video-mux would have to forward get_mbus_config to its active upstream port and if the upstream sensor does not implement it read bus width from the active upstream endpoint.

regards
Philipp

#12 lore raw
Quoted 39 lines · Philipp Zabel
Quoted 34 lines · Sakari Ailus
Quoted 26 lines · Laurent Pinchart
Quoted 17 lines · Sakari Ailus
Quoted 6 lines · Laurent Pinchart

In a (simplified) nutshell,

---------+     +----------+     +---------+     +-----+     +-----+
> Camera | --> | CSI-2 RX | --> | CSI2IPU | --> | Mux | --> | IPU |
> Sensor |     |          |     | Gasket  |     |     |     |     |
---------+     +----------+     +---------+     +-----+     +-----+

Thank you, this is helpful.

I suppose the mux here at least won't actively do anything to the data. So presumably its endpoint won't contain the active configuration, but its superset.

Quoted 3 lines

All those blocks, except for the gasket, have a node in DT.

Quoted 7 lines
Quoted 4 lines

The IPU driver needs to know the number of CSI-2 data lanes, which is encoded in the data-lanes DT property present in both the sensor output endpoint and the CSI-2 RX input endpoint, but not the other endpoints in the pipeline.

This doesn't yet explain why the sensor would need to implement get_mbus_config if its bus configuration remains constant.

Quoted 6 lines

If I recall correctly, the IPU driver calls .g_mbus_config() on the camera sensor to get the number of lanes, as it can't get it from its own endpoint. That's a hack, and as Jacopo proposed, calling .g_mbus_config() on the CSI-2 RX would be better, as the CSI-2 RX driver can then get the value from its own endpoint, without requiring all sensor drivers to implement .g_mbus_config().

Quoted 3 lines

The g_mbus_config op could be implemented by the CSI2IPU and mux, by simply requesting the information from the upstream sub-device. No hacks would be needed.

Quoted 7 lines

I think implementing get_mbus_config on the mux might be enough. The IPU driver already recognizes the CSI-2 RX by its grp_id and could infer that it has to use MIPI CSI-2 mode from that.

The video-mux would have to forward get_mbus_config to its active upstream port and if the upstream sensor does not implement it read bus width from the active upstream endpoint.

I'm fine with implementing it in the mux as well, but I think we can take a shortcut here and call it on the CSI-2 RX from the IPU driver, as the IPU driver knows about the architecture of the whole pipeline.

Signature · 3 lines
Regards,

Laurent Pinchart
#13 lore raw
Quoted 45 lines · Laurent Pinchart
Quoted 43 lines · Philipp Zabel
Quoted 34 lines · Sakari Ailus
Quoted 26 lines · Laurent Pinchart
Quoted 17 lines · Sakari Ailus
Quoted 6 lines · Laurent Pinchart

In a (simplified) nutshell,

---------+     +----------+     +---------+     +-----+     +-----+
> Camera | --> | CSI-2 RX | --> | CSI2IPU | --> | Mux | --> | IPU |
> Sensor |     |          |     | Gasket  |     |     |     |     |
---------+     +----------+     +---------+     +-----+     +-----+

Thank you, this is helpful.

I suppose the mux here at least won't actively do anything to the data. So presumably its endpoint won't contain the active configuration, but its superset.

Quoted 3 lines

All those blocks, except for the gasket, have a node in DT.

Quoted 7 lines
Quoted 4 lines

The IPU driver needs to know the number of CSI-2 data lanes, which is encoded in the data-lanes DT property present in both the sensor output endpoint and the CSI-2 RX input endpoint, but not the other endpoints in the pipeline.

This doesn't yet explain why the sensor would need to implement get_mbus_config if its bus configuration remains constant.

Quoted 6 lines

If I recall correctly, the IPU driver calls .g_mbus_config() on the camera sensor to get the number of lanes, as it can't get it from its own endpoint. That's a hack, and as Jacopo proposed, calling .g_mbus_config() on the CSI-2 RX would be better, as the CSI-2 RX driver can then get the value from its own endpoint, without requiring all sensor drivers to implement .g_mbus_config().

Quoted 3 lines

The g_mbus_config op could be implemented by the CSI2IPU and mux, by simply requesting the information from the upstream sub-device. No hacks would be needed.

I think implementing get_mbus_config on the mux might be enough. The IPU driver already recognizes the CSI-2 RX by its grp_id and could infer that it has to use MIPI CSI-2 mode from that.

Quoted 7 lines
Quoted 3 lines

The video-mux would have to forward get_mbus_config to its active upstream port and if the upstream sensor does not implement it read bus width from the active upstream endpoint.

I'm fine with implementing it in the mux as well, but I think we can take a shortcut here and call it on the CSI-2 RX from the IPU driver, as the IPU driver knows about the architecture of the whole pipeline.

FWIW I have made use of video-mux and implementing g_mbus_config on it for D-PHY switch chips[1] where the different input ports may have different configurations. I'll admit that I've made the easy assumption that it's CSI-2 D-PHY in and out, when it could quite legitimately be working with any of the other bus types.

I had been intending to send this[2] upstream when I get a chance, but am I reading imx6q.dtsi[3] correctly in that the mux is accepting parallel on some ports and CSI-2 on others? The mux hardware is therefore far more than just a simple switch between inputs? Although as this is after the CSI2 rx block, this is effectively parallel data within the SoC, therefore is the configuration and get_mbus_config really relevant?
I'd like to understand how this is being used on imx6 before trying to rework my patch into a generic solution.

Thanks
Dave

[1] eg Arducam's 2 and 4 port muxes -
https://www.arducam.com/product-category/camera-multiplexers/, which
IIRC use OnSemi's FSA644
https://www.onsemi.com/products/interfaces/analog-switches/fsa644
[2] https://github.com/raspberrypi/linux/commit/bf653318475cf4db0ec59e92139f477f7cc0a544
[3] https://elixir.bootlin.com/linux/latest/source/arch/arm/boot/dts/imx6q.dtsi#L349

#14 lore raw

Hi Dave,

Quoted 53 lines · Dave Stevenson
Quoted 44 lines · Laurent Pinchart
Quoted 43 lines · Philipp Zabel
Quoted 34 lines · Sakari Ailus
Quoted 26 lines · Laurent Pinchart
Quoted 17 lines · Sakari Ailus
Quoted 6 lines · Laurent Pinchart

In a (simplified) nutshell,

---------+     +----------+     +---------+     +-----+     +-----+
> Camera | --> | CSI-2 RX | --> | CSI2IPU | --> | Mux | --> | IPU |
> Sensor |     |          |     | Gasket  |     |     |     |     |
---------+     +----------+     +---------+     +-----+     +-----+

Thank you, this is helpful.

I suppose the mux here at least won't actively do anything to the data. So presumably its endpoint won't contain the active configuration, but its superset.

Quoted 3 lines

All those blocks, except for the gasket, have a node in DT.

Quoted 7 lines
Quoted 4 lines

The IPU driver needs to know the number of CSI-2 data lanes, which is encoded in the data-lanes DT property present in both the sensor output endpoint and the CSI-2 RX input endpoint, but not the other endpoints in the pipeline.

This doesn't yet explain why the sensor would need to implement get_mbus_config if its bus configuration remains constant.

Quoted 6 lines

If I recall correctly, the IPU driver calls .g_mbus_config() on the camera sensor to get the number of lanes, as it can't get it from its own endpoint. That's a hack, and as Jacopo proposed, calling .g_mbus_config() on the CSI-2 RX would be better, as the CSI-2 RX driver can then get the value from its own endpoint, without requiring all sensor drivers to implement .g_mbus_config().

Quoted 3 lines

The g_mbus_config op could be implemented by the CSI2IPU and mux, by simply requesting the information from the upstream sub-device. No hacks would be needed.

I think implementing get_mbus_config on the mux might be enough. The IPU driver already recognizes the CSI-2 RX by its grp_id and could infer that it has to use MIPI CSI-2 mode from that.

Quoted 7 lines
Quoted 3 lines

The video-mux would have to forward get_mbus_config to its active upstream port and if the upstream sensor does not implement it read bus width from the active upstream endpoint.

I'm fine with implementing it in the mux as well, but I think we can take a shortcut here and call it on the CSI-2 RX from the IPU driver, as the IPU driver knows about the architecture of the whole pipeline.

Quoted 5 lines

FWIW I have made use of video-mux and implementing g_mbus_config on it for D-PHY switch chips[1] where the different input ports may have different configurations. I'll admit that I've made the easy assumption that it's CSI-2 D-PHY in and out, when it could quite legitimately be working with any of the other bus types.

That's a use case I hadn't considered. .get_mbus_config() makes sense.

Quoted 7 lines

I had been intending to send this[2] upstream when I get a chance, but am I reading imx6q.dtsi[3] correctly in that the mux is accepting parallel on some ports and CSI-2 on others? The mux hardware is therefore far more than just a simple switch between inputs? Although as this is after the CSI2 rx block, this is effectively parallel data within the SoC, therefore is the configuration and get_mbus_config really relevant?

The exact hardware architecture isn't clear, but I indeed expect the mux to receive parallel data from the CSI-2 RX and parallel inputs.

I had a closer look at the IPU driver code, and realized I was wrong when I mentioned it needed to know the number of lanes. What the IPU need is the bus type (CSI-2, BT656 or parallel) and, for parallel buses, the bus width. It may be possible to refactor the IPU driver code to replace that information with the media bus code in some places, but not everywhere. Whether the input comes from the CSI-2 RX could be deduced internally from the selected mux input, but we can't differentiate BT656 from parallel without querying the mux's input bus type. Calling .get_mbus_config() on the mux is thus required, and the mux driver should implement the operation.

Quoted 10 lines

I'd like to understand how this is being used on imx6 before trying to rework my patch into a generic solution.

Thanks
Dave

[1] eg Arducam's 2 and 4 port muxes - https://www.arducam.com/product-category/camera-multiplexers/, which IIRC use OnSemi's FSA644 https://www.onsemi.com/products/interfaces/analog-switches/fsa644
[2] https://github.com/raspberrypi/linux/commit/bf653318475cf4db0ec59e92139f477f7cc0a544
[3] https://elixir.bootlin.com/linux/latest/source/arch/arm/boot/dts/imx6q.dtsi#L349

Signature · 3 lines
Regards,

Laurent Pinchart
maintainer
#15 lore raw
In reply to Laurent Pinchart

Hi Laurent,

Quoted 44 lines · Laurent Pinchart
Quoted 43 lines · Philipp Zabel
Quoted 34 lines · Sakari Ailus
Quoted 26 lines · Laurent Pinchart
Quoted 17 lines · Sakari Ailus
Quoted 6 lines · Laurent Pinchart

In a (simplified) nutshell,

---------+     +----------+     +---------+     +-----+     +-----+
> Camera | --> | CSI-2 RX | --> | CSI2IPU | --> | Mux | --> | IPU |
> Sensor |     |          |     | Gasket  |     |     |     |     |
---------+     +----------+     +---------+     +-----+     +-----+

Thank you, this is helpful.

I suppose the mux here at least won't actively do anything to the data. So presumably its endpoint won't contain the active configuration, but its superset.

Quoted 3 lines

All those blocks, except for the gasket, have a node in DT.

Quoted 7 lines
Quoted 4 lines

The IPU driver needs to know the number of CSI-2 data lanes, which is encoded in the data-lanes DT property present in both the sensor output endpoint and the CSI-2 RX input endpoint, but not the other endpoints in the pipeline.

This doesn't yet explain why the sensor would need to implement get_mbus_config if its bus configuration remains constant.

Quoted 6 lines

If I recall correctly, the IPU driver calls .g_mbus_config() on the camera sensor to get the number of lanes, as it can't get it from its own endpoint. That's a hack, and as Jacopo proposed, calling .g_mbus_config() on the CSI-2 RX would be better, as the CSI-2 RX driver can then get the value from its own endpoint, without requiring all sensor drivers to implement .g_mbus_config().

Quoted 3 lines

The g_mbus_config op could be implemented by the CSI2IPU and mux, by simply requesting the information from the upstream sub-device. No hacks would be needed.

I think implementing get_mbus_config on the mux might be enough. The IPU driver already recognizes the CSI-2 RX by its grp_id and could infer that it has to use MIPI CSI-2 mode from that.

Quoted 7 lines
Quoted 3 lines

The video-mux would have to forward get_mbus_config to its active upstream port and if the upstream sensor does not implement it read bus width from the active upstream endpoint.

I'm fine with implementing it in the mux as well, but I think we can take a shortcut here and call it on the CSI-2 RX from the IPU driver, as the IPU driver knows about the architecture of the whole pipeline.

If that's the case then I guess that's fine. But can these drivers be used elsewhere than with IMX6?

It'd be safest to implement g_mbus_config for the all the way to CSI-2 RX.

But if the answer to the question is "no", then making that shortcut should be fine (and this can be always reworked if need be).

Signature · 3 lines
Kind regards,

Sakari Ailus
#16 lore raw
In reply to Laurent Pinchart
Quoted 28 lines · Laurent Pinchart
Quoted 19 lines · Sakari Ailus

Hi Laurent,

Quoted 6 lines · Laurent Pinchart

In a (simplified) nutshell,

---------+     +----------+     +---------+     +-----+     +-----+
> Camera | --> | CSI-2 RX | --> | CSI2IPU | --> | Mux | --> | IPU |
> Sensor |     |          |     | Gasket  |     |     |     |     |
---------+     +----------+     +---------+     +-----+     +-----+

Thank you, this is helpful.

I suppose the mux here at least won't actively do anything to the data. So presumably its endpoint won't contain the active configuration, but its superset.

Quoted 3 lines

All those blocks, except for the gasket, have a node in DT.

Quoted 7 lines
Quoted 4 lines

The IPU driver needs to know the number of CSI-2 data lanes, which is encoded in the data-lanes DT property present in both the sensor output endpoint and the CSI-2 RX input endpoint, but not the other endpoints in the pipeline.

This doesn't yet explain why the sensor would need to implement get_mbus_config if its bus configuration remains constant.

Quoted 6 lines

If I recall correctly, the IPU driver calls .g_mbus_config() on the camera sensor to get the number of lanes, as it can't get it from its own endpoint. That's a hack, and as Jacopo proposed, calling .g_mbus_config() on the CSI-2 RX would be better, as the CSI-2 RX driver can then get the value from its own endpoint, without requiring all sensor drivers to implement .g_mbus_config().

The IPU driver doesn't call get_mbus_config, the CSI-2 RX driver does (csi2_get_active_lanes in imx6-mipi-csi2.c). It could just fall back to looking at its own endpoint if the upstream driver does not implement get_mbus_config.

Of course that will only help if the DT contains this information and all connected lanes are active.

regards
Philipp

#17 lore raw

Hi Philipp

Quoted 29 lines · Philipp Zabel
Quoted 28 lines · Laurent Pinchart
Quoted 26 lines · Sakari Ailus

Hi Laurent,

Quoted 6 lines · Laurent Pinchart

In a (simplified) nutshell,

---------+     +----------+     +---------+     +-----+     +-----+
> Camera | --> | CSI-2 RX | --> | CSI2IPU | --> | Mux | --> | IPU |
> Sensor |     |          |     | Gasket  |     |     |     |     |
---------+     +----------+     +---------+     +-----+     +-----+

Thank you, this is helpful.

I suppose the mux here at least won't actively do anything to the data. So presumably its endpoint won't contain the active configuration, but its superset.

Quoted 7 lines

All those blocks, except for the gasket, have a node in DT.

The IPU driver needs to know the number of CSI-2 data lanes, which is encoded in the data-lanes DT property present in both the sensor output endpoint and the CSI-2 RX input endpoint, but not the other endpoints in the pipeline.

This doesn't yet explain why the sensor would need to implement get_mbus_config if its bus configuration remains constant.

Quoted 8 lines
Quoted 6 lines

If I recall correctly, the IPU driver calls .g_mbus_config() on the camera sensor to get the number of lanes, as it can't get it from its own endpoint. That's a hack, and as Jacopo proposed, calling .g_mbus_config() on the CSI-2 RX would be better, as the CSI-2 RX driver can then get the value from its own endpoint, without requiring all sensor drivers to implement .g_mbus_config().

The IPU driver doesn't call get_mbus_config, the CSI-2 RX driver does

Am I confusing IPU CSI with CSI-2 ?
https://elixir.bootlin.com/linux/latest/source/drivers/staging/media/imx/imx-media-csi.c#L211

Quoted 6 lines

(csi2_get_active_lanes in imx6-mipi-csi2.c). It could just fall back to looking at its own endpoint if the upstream driver does not implement get_mbus_config.

Of course that will only help if the DT contains this information and all connected lanes are active.

We should assume DTs are correct, otherwise we're screwed most of the times...

Quoted 3 lines

regards
Philipp

#18 lore raw
Quoted 39 lines · Jacopo Mondi

Hi Philipp

Quoted 36 lines · Philipp Zabel
Quoted 34 lines · Laurent Pinchart
Quoted 19 lines · Sakari Ailus

Hi Laurent,

Quoted 6 lines · Laurent Pinchart

In a (simplified) nutshell,

---------+     +----------+     +---------+     +-----+     +-----+
> Camera | --> | CSI-2 RX | --> | CSI2IPU | --> | Mux | --> | IPU |
> Sensor |     |          |     | Gasket  |     |     |     |     |
---------+     +----------+     +---------+     +-----+     +-----+

Thank you, this is helpful.

I suppose the mux here at least won't actively do anything to the data. So presumably its endpoint won't contain the active configuration, but its superset.

Quoted 3 lines

All those blocks, except for the gasket, have a node in DT.

Quoted 7 lines
Quoted 4 lines

The IPU driver needs to know the number of CSI-2 data lanes, which is encoded in the data-lanes DT property present in both the sensor output endpoint and the CSI-2 RX input endpoint, but not the other endpoints in the pipeline.

This doesn't yet explain why the sensor would need to implement get_mbus_config if its bus configuration remains constant.

If I recall correctly, the IPU driver calls .g_mbus_config() on the camera sensor to get the number of lanes, as it can't get it from its own endpoint. That's a hack, and as Jacopo proposed, calling .g_mbus_config() on the CSI-2 RX would be better, as the CSI-2 RX driver can then get the value from its own endpoint, without requiring all sensor drivers to implement .g_mbus_config().

Quoted 4 lines
Quoted 1 line

The IPU driver doesn't call get_mbus_config, the CSI-2 RX driver does

Am I confusing IPU CSI with CSI-2 ?
https://elixir.bootlin.com/linux/latest/source/drivers/staging/media/imx/imx-media-csi.c#L211

What the eyesight ... Sorry for the confusion. You are right, it's right there. Yes, that is IPU CSI calling get_mbus_config.

I had git grepped for get_mbus_config this morning, and was convinced I only found the call in imx-media-csi2, which is where the number of lanes is evaluated. The call in imx-media-csi.c is used to determine whether the sensor is parallel or CSI-2 and whether the CSI has to operate in bypass mode (for 16-bit parallel bus).

regards
Philipp

#19 lore raw
In reply to Jacopo Mondi
Quoted 37 lines · Jacopo Mondi
Quoted 36 lines · Francesco Dolcini
Quoted 25 lines · Sakari Ailus
Quoted 24 lines · Francesco Dolcini
Quoted 10 lines · Sakari Ailus
Quoted 7 lines · Marcel Ziswiler

From: Aishwarya Kothari <[email protected]>

Implement the introduced get_mbus_config operation to report the config of the MIPI CSI-2, BT.656 and Parallel interface.

What's the reasoning for this patch?

Without this it's not possible to use it on i.MX6, drivers/staging/media/imx/imx6-mipi-csi2.c requires it, some more details from Jacopo here [0].

Everything used to work fine up to v5.18, after that kernel version various changes broke it [1][2] (I assume you are pretty much aware of the history here, you commented on a few emails).

[0] https://lore.kernel.org/all/[email protected]/
[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/

Quoted 5 lines
Quoted 2 lines
Quoted 2 lines

Drivers that don't have e.g. dynamic lane configuration shouldn't need to implement get_mbus_config.

Not even for staging drivers. The driver should be fixed to get that information from the endpoint instead.

This seems exactly the opposite of what commit
7318abface48 ("media: imx: Use get_mbus_config instead of parsing upstream DT endpoints") did.

Quoted 8 lines
Quoted 4 lines

Given that I am somehow confused, but I am not that familiar with this subsystem, so I guess this is expected :-). Can someone provide some additional hint here?

To be honest my understanding is that this patch has always been needed to work on imx6 and this is not a regression but something that was kept as an out-of-tree patch downstream. Is this correct or is this a regression ?

I confirm that v5.18 was/is fine. Aishwarya: correct? In the end you tested it, not me :-)

Francesco

#20 lore raw
Quoted 47 lines · Francesco Dolcini
Quoted 37 lines · Jacopo Mondi
Quoted 36 lines · Francesco Dolcini
Quoted 25 lines · Sakari Ailus
Quoted 24 lines · Francesco Dolcini
Quoted 10 lines · Sakari Ailus
Quoted 7 lines · Marcel Ziswiler

From: Aishwarya Kothari <[email protected]>

Implement the introduced get_mbus_config operation to report the config of the MIPI CSI-2, BT.656 and Parallel interface.

What's the reasoning for this patch?

Without this it's not possible to use it on i.MX6, drivers/staging/media/imx/imx6-mipi-csi2.c requires it, some more details from Jacopo here [0].

Everything used to work fine up to v5.18, after that kernel version various changes broke it [1][2] (I assume you are pretty much aware of the history here, you commented on a few emails).

[0] https://lore.kernel.org/all/[email protected]/
[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/

Quoted 5 lines
Quoted 2 lines
Quoted 2 lines

Drivers that don't have e.g. dynamic lane configuration shouldn't need to implement get_mbus_config.

Not even for staging drivers. The driver should be fixed to get that information from the endpoint instead.

This seems exactly the opposite of what commit
7318abface48 ("media: imx: Use get_mbus_config instead of parsing upstream DT endpoints") did.

Quoted 8 lines
Quoted 4 lines

Given that I am somehow confused, but I am not that familiar with this subsystem, so I guess this is expected :-). Can someone provide some additional hint here?

To be honest my understanding is that this patch has always been needed to work on imx6 and this is not a regression but something that was kept as an out-of-tree patch downstream. Is this correct or is this a regression ?

Quoted 6 lines

I confirm that v5.18 was/is fine. Aishwarya: correct? In the end you tested it, not me :-)

Francesco

It worked on the v5.18 without this patch.

Aishwarya