Skip to contentSkip to conversation

media: cadence,ti: CSI2RX Multistream Support

PATCH In review Jai Luthra 5 participants 47 messages Jun 27, 2024 – Jul 19, 2024

Subsystems (4)
lore
Share & tools
lore

Jai Luthra

author
#0 lore raw

v2 cover media: cadence,ti: CSI2RX Multistream Support

This series adds multi-stream support for Cadence CSI2RX and TI CSI2RX Shim drivers.

PATCH 1: Runtime Power Management for Cadence CSI2RX
PATCH 2-7: Support multiple DMA contexts/video nodes in TI CSI2RX
PATCH 8-9: Use get_frame_desc to propagate virtual channel information
across Cadence and TI CSI-RX subdevs
PATCH 10-12: Use new multi-stream APIs across the drivers to support
multiplexed cameras from sources like UB960 (FPDLink)
PATCH 13: Optimize stream on by submitting all queued buffers to DMA

This applies on top of today's linux-next (next-20240626) (also tested rebase with media_stage.git master)

---
Changes in v2:

- Change the multi-camera capture architecture to be similar to that of Tomi's RPi5 FE series, where the driver will wait for userspace to start streaming on all "actively routed" video nodes before starting streaming on the source. This simplifies things a lot from the HW perspective, which might run into deadlocks due to a shared FIFO between multiple DMA channels.

- Drop a few fixes that were posted separately and are already merged
- Fix dtschema warnings reported by Rob on [02/13]
- Fix warnings for uninitialized used_vc variable in cdns-csi2rx.c
- Return -EBUSY if someone updates routes for j721e-csi2rx subdev while streaming
- Only allow single-streams to be routed to the source pads (linked to video nodes) of the j721e-csi2rx device
- Squash the patches marked "SQUASH" in the v1 RFC series

- Link to RFC (v1):
https://lore.kernel.org/r/[email protected]

---
Jai Luthra (8):
dt-bindings: media: ti,j721e-csi2rx-shim: Support 32 dma chans
media: ti: j721e-csi2rx: separate out device and context
media: ti: j721e-csi2rx: add a subdev for the core device
media: ti: j721e-csi2rx: add support for processing virtual channels
media: cadence: csi2rx: Use new enable stream APIs
media: cadence: csi2rx: Enable multi-stream support
media: ti: j721e-csi2rx: add multistream support
media: ti: j721e-csi2rx: Submit all available buffers

Jayshri Pawar (1):
media: cadence: csi2rx: Support runtime PM

Pratyush Yadav (4):
media: ti: j721e-csi2rx: prepare SHIM code for multiple contexts
media: ti: j721e-csi2rx: allocate DMA channel based on context index
media: ti: j721e-csi2rx: get number of contexts from device tree
media: cadence: csi2rx: add get_frame_desc wrapper

 .../bindings/media/ti,j721e-csi2rx-shim.yaml       |  39 +-
 drivers/media/platform/cadence/cdns-csi2rx.c       | 440 +++++++++--
 .../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c  | 879 ++++++++++++++++-----
 3 files changed, 1071 insertions(+), 287 deletions(-)

---
base-commit: df9574a57d02b265322e77fb8628d4d33641dda9
change-id: 20240221-multistream-fbba6ffe47a3

Best regards,

Signature · 1 line
Jai Luthra <[email protected]>
author
#1 lore raw

PATCH v2 1/13 media: cadence: csi2rx: Support runtime PM

From: Jayshri Pawar <[email protected]>

Use runtime power management hooks to save power when CSI-RX is not in use. Also stop/start any in-progress streams, which might happen during a system suspend/resume cycle.

---
 drivers/media/platform/cadence/cdns-csi2rx.c | 43 +++++++++++++++++++++++++++-
 1 file changed, 42 insertions(+), 1 deletion(-)
drivers/media/platform/cadence/cdns-csi2rx.c +42 -1
index 6f7d27a48eff..751eadbe61ef 100644
@@ -366,6 +366,12 @@ static int csi2rx_s_stream(struct v4l2_subdev *subdev, int enable)
 	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
 	int ret = 0;
 
+	if (enable) {
+		ret = pm_runtime_resume_and_get(csi2rx->dev);
+		if (ret < 0)
+			return ret;
+	}
+
 	mutex_lock(&csi2rx->lock);
 
 	if (enable) {
@@ -375,8 +381,10 @@ static int csi2rx_s_stream(struct v4l2_subdev *subdev, int enable)
 		 */
 		if (!csi2rx->count) {
 			ret = csi2rx_start(csi2rx);
-			if (ret)
+			if (ret) {
+				pm_runtime_put(csi2rx->dev);
 				goto out;
+			}
 		}
 
 		csi2rx->count++;
@@ -388,6 +396,8 @@ static int csi2rx_s_stream(struct v4l2_subdev *subdev, int enable)
 		 */
 		if (!csi2rx->count)
 			csi2rx_stop(csi2rx);
+
+		pm_runtime_put(csi2rx->dev);
 	}
 
 out:
@@ -661,6 +671,29 @@ static int csi2rx_parse_dt(struct csi2rx_priv *csi2rx)
 	return ret;
 }
 
+static int csi2rx_suspend(struct device *dev)
+{
+	struct csi2rx_priv *csi2rx = dev_get_drvdata(dev);
+
+	mutex_lock(&csi2rx->lock);
+	if (csi2rx->count)
+		csi2rx_stop(csi2rx);
+	mutex_unlock(&csi2rx->lock);
+
+	return 0;
+}
+
+static int csi2rx_resume(struct device *dev)
+{
+	struct csi2rx_priv *csi2rx = dev_get_drvdata(dev);
+
+	mutex_lock(&csi2rx->lock);
+	if (csi2rx->count)
+		csi2rx_start(csi2rx);
+	mutex_unlock(&csi2rx->lock);
+	return 0;
+}
+
 static int csi2rx_probe(struct platform_device *pdev)
 {
 	struct csi2rx_priv *csi2rx;
@@ -707,6 +740,7 @@ static int csi2rx_probe(struct platform_device *pdev)
 	if (ret)
 		goto err_cleanup;
 
+	pm_runtime_enable(csi2rx->dev);
 	ret = v4l2_async_register_subdev(&csi2rx->subdev);
 	if (ret < 0)
 		goto err_free_state;
@@ -721,6 +755,7 @@ static int csi2rx_probe(struct platform_device *pdev)
 
 err_free_state:
 	v4l2_subdev_cleanup(&csi2rx->subdev);
+	pm_runtime_disable(csi2rx->dev);
 err_cleanup:
 	v4l2_async_nf_unregister(&csi2rx->notifier);
 	v4l2_async_nf_cleanup(&csi2rx->notifier);
@@ -739,9 +774,14 @@ static void csi2rx_remove(struct platform_device *pdev)
 	v4l2_async_unregister_subdev(&csi2rx->subdev);
 	v4l2_subdev_cleanup(&csi2rx->subdev);
 	media_entity_cleanup(&csi2rx->subdev.entity);
+	pm_runtime_disable(csi2rx->dev);
 	kfree(csi2rx);
 }
 
+static const struct dev_pm_ops csi2rx_pm_ops = {
+	SET_RUNTIME_PM_OPS(csi2rx_suspend, csi2rx_resume, NULL)
+};
+
 static const struct of_device_id csi2rx_of_table[] = {
 	{ .compatible = "starfive,jh7110-csi2rx" },
 	{ .compatible = "cdns,csi2rx" },
@@ -756,6 +796,7 @@ static struct platform_driver csi2rx_driver = {
 	.driver	= {
 		.name		= "cdns-csi2rx",
 		.of_match_table	= csi2rx_of_table,
+		.pm		= &csi2rx_pm_ops,
 	},
 };
 module_platform_driver(csi2rx_driver);
Signature · 1 line
2.43.0
#2 lore raw

Re: [PATCH v2 01/13] media: cadence: csi2rx: Support runtime PM

Hi,

Quoted 68 lines · Jai Luthra

From: Jayshri Pawar <[email protected]>

Use runtime power management hooks to save power when CSI-RX is not in use. Also stop/start any in-progress streams, which might happen during a system suspend/resume cycle.

---
  drivers/media/platform/cadence/cdns-csi2rx.c | 43 +++++++++++++++++++++++++++-
  1 file changed, 42 insertions(+), 1 deletion(-)
drivers/media/platform/cadence/cdns-csi2rx.c +24 -1
index 6f7d27a48eff..751eadbe61ef 100644
@@ -366,6 +366,12 @@ static int csi2rx_s_stream(struct v4l2_subdev *subdev, int enable)
  	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
  	int ret = 0;
 
+	if (enable) {
+		ret = pm_runtime_resume_and_get(csi2rx->dev);
+		if (ret < 0)
+			return ret;
+	}
+
  	mutex_lock(&csi2rx->lock);
 
  	if (enable) {
@@ -375,8 +381,10 @@ static int csi2rx_s_stream(struct v4l2_subdev *subdev, int enable)
  		 */
  		if (!csi2rx->count) {
  			ret = csi2rx_start(csi2rx);
-			if (ret)
+			if (ret) {
+				pm_runtime_put(csi2rx->dev);
  				goto out;
+			}
  		}
 
  		csi2rx->count++;
@@ -388,6 +396,8 @@ static int csi2rx_s_stream(struct v4l2_subdev *subdev, int enable)
  		 */
  		if (!csi2rx->count)
  			csi2rx_stop(csi2rx);
+
+		pm_runtime_put(csi2rx->dev);
  	}
 
  out:
@@ -661,6 +671,29 @@ static int csi2rx_parse_dt(struct csi2rx_priv *csi2rx)
  	return ret;
  }
 
+static int csi2rx_suspend(struct device *dev)
+{
+	struct csi2rx_priv *csi2rx = dev_get_drvdata(dev);
+
+	mutex_lock(&csi2rx->lock);
+	if (csi2rx->count)
+		csi2rx_stop(csi2rx);
+	mutex_unlock(&csi2rx->lock);
+
+	return 0;
+}
+
+static int csi2rx_resume(struct device *dev)

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_parse_dt(struct csi2rx_priv *csi2rx) · line 696

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +10 -0
+{
+	struct csi2rx_priv *csi2rx = dev_get_drvdata(dev);
+
+	mutex_lock(&csi2rx->lock);
+	if (csi2rx->count)
+		csi2rx_start(csi2rx);
+	mutex_unlock(&csi2rx->lock);
+	return 0;
+}
+

I don't think this looks correct. Afaik the runtime suspend/resume is not called on system suspend/resume. You could change the SET_RUNTIME_PM_OPS to use the same callbacks for runtime and system suspend, but I think that's a bad idea. Runtime suspend is not supposed to turn off the streaming. The driver is supposed to turn off the streaming, then call runtime_put, which would result in runtime suspend callback getting called.

Tomi

Quoted 43 lines
drivers/media/platform/cadence/cdns-csi2rx.c +8 -0
  static int csi2rx_probe(struct platform_device *pdev)
  {
  	struct csi2rx_priv *csi2rx;
@@ -707,6 +740,7 @@ static int csi2rx_probe(struct platform_device *pdev)
  	if (ret)
  		goto err_cleanup;
 
+	pm_runtime_enable(csi2rx->dev);
  	ret = v4l2_async_register_subdev(&csi2rx->subdev);
  	if (ret < 0)
  		goto err_free_state;
@@ -721,6 +755,7 @@ static int csi2rx_probe(struct platform_device *pdev)
 
  err_free_state:
  	v4l2_subdev_cleanup(&csi2rx->subdev);
+	pm_runtime_disable(csi2rx->dev);
  err_cleanup:
  	v4l2_async_nf_unregister(&csi2rx->notifier);
  	v4l2_async_nf_cleanup(&csi2rx->notifier);
@@ -739,9 +774,14 @@ static void csi2rx_remove(struct platform_device *pdev)
  	v4l2_async_unregister_subdev(&csi2rx->subdev);
  	v4l2_subdev_cleanup(&csi2rx->subdev);
  	media_entity_cleanup(&csi2rx->subdev.entity);
+	pm_runtime_disable(csi2rx->dev);
  	kfree(csi2rx);
  }
 
+static const struct dev_pm_ops csi2rx_pm_ops = {
+	SET_RUNTIME_PM_OPS(csi2rx_suspend, csi2rx_resume, NULL)
+};
+
  static const struct of_device_id csi2rx_of_table[] = {
  	{ .compatible = "starfive,jh7110-csi2rx" },
  	{ .compatible = "cdns,csi2rx" },
@@ -756,6 +796,7 @@ static struct platform_driver csi2rx_driver = {
  	.driver	= {
  		.name		= "cdns-csi2rx",
  		.of_match_table	= csi2rx_of_table,
+		.pm		= &csi2rx_pm_ops,
  	},
  };
  module_platform_driver(csi2rx_driver);
#3 lore raw

回复: [PATCH v2 01/13] media: cadence: csi2rx: Support runtime PM

Hi Tomi,

[...]

Quoted 20 lines
Quoted 20 lines
drivers/media/platform/cadence/cdns-csi2rx.c +20 -0
+static int csi2rx_suspend(struct device *dev) {
+	struct csi2rx_priv *csi2rx = dev_get_drvdata(dev);
+
+	mutex_lock(&csi2rx->lock);
+	if (csi2rx->count)
+		csi2rx_stop(csi2rx);
+	mutex_unlock(&csi2rx->lock);
+
+	return 0;
+}
+
+static int csi2rx_resume(struct device *dev) {
+	struct csi2rx_priv *csi2rx = dev_get_drvdata(dev);
+
+	mutex_lock(&csi2rx->lock);
+	if (csi2rx->count)
+		csi2rx_start(csi2rx);
+	mutex_unlock(&csi2rx->lock);
+	return 0;
+}
Quoted 9 lines
Quoted 1 line

+

I don't think this looks correct. Afaik the runtime suspend/resume is not called on system suspend/resume. You could change the SET_RUNTIME_PM_OPS to use the same callbacks for runtime and system suspend, but I think that's a bad idea. Runtime suspend is not supposed to turn off the streaming. The driver is supposed to turn off the streaming, then call runtime_put, which would result in runtime suspend callback getting called.

I implemented system suspend/resume based on this patch, I feel good about it.

https://lore.kernel.org/all/[email protected]/

Regards,
Changhuang

author
#4 lore raw

Re: 回复: [PATCH v2 01/13] media: cadence: csi2rx: Support runtime PM

Hi Changhuang,

Quoted 33 lines · Changhuang Liang

Hi Tomi,

[...]

Quoted 20 lines
Quoted 20 lines
drivers/media/platform/cadence/cdns-csi2rx.c +20 -0
+static int csi2rx_suspend(struct device *dev) {
+	struct csi2rx_priv *csi2rx = dev_get_drvdata(dev);
+
+	mutex_lock(&csi2rx->lock);
+	if (csi2rx->count)
+		csi2rx_stop(csi2rx);
+	mutex_unlock(&csi2rx->lock);
+
+	return 0;
+}
+
+static int csi2rx_resume(struct device *dev) {
+	struct csi2rx_priv *csi2rx = dev_get_drvdata(dev);
+
+	mutex_lock(&csi2rx->lock);
+	if (csi2rx->count)
+		csi2rx_start(csi2rx);
+	mutex_unlock(&csi2rx->lock);
+	return 0;
+}
Quoted 9 lines
Quoted 1 line

+

I don't think this looks correct. Afaik the runtime suspend/resume is not called on system suspend/resume. You could change the SET_RUNTIME_PM_OPS to use the same callbacks for runtime and system suspend, but I think that's a bad idea. Runtime suspend is not supposed to turn off the streaming. The driver is supposed to turn off the streaming, then call runtime_put, which would result in runtime suspend callback getting called.

Quoted 3 lines

I implemented system suspend/resume based on this patch, I feel good about it.

https://lore.kernel.org/all/[email protected]/

Thanks for carrying this patch in your series.

I think Tomi's point still holds - only the system suspend hook should try to stop/start the CSI2RX device.

Runtime PM hooks are usually only called when there are no users, so no active streams.

Quoted 3 lines

Regards,
Changhuang

Signature · 4 lines
Thanks,
Jai

GPG Fingerprint: 4DE0 D818 E5D5 75E8 D45A AFC5 43DE 91F9 249A 7145
#5 lore raw

回复: 回复: [PATCH v2 01/13] media: cadence: csi2rx: Support runtime PM

Hi, Jai,

Quoted 44 lines

Hi Changhuang,

Quoted 28 lines · Changhuang Liang

Hi Tomi,

[...]

Quoted 25 lines
Quoted 11 lines
drivers/media/platform/cadence/cdns-csi2rx.c +11 -0
+static int csi2rx_suspend(struct device *dev) {
+	struct csi2rx_priv *csi2rx = dev_get_drvdata(dev);
+
+	mutex_lock(&csi2rx->lock);
+	if (csi2rx->count)
+		csi2rx_stop(csi2rx);
+	mutex_unlock(&csi2rx->lock);
+
+	return 0;
+}
+

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_parse_dt(struct csi2rx_priv *csi2rx) · line 696

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +10 -0
+static int csi2rx_resume(struct device *dev) {
+	struct csi2rx_priv *csi2rx = dev_get_drvdata(dev);
+
+	mutex_lock(&csi2rx->lock);
+	if (csi2rx->count)
+		csi2rx_start(csi2rx);
+	mutex_unlock(&csi2rx->lock);
+	return 0;
+}
+

I don't think this looks correct. Afaik the runtime suspend/resume is not called on system suspend/resume. You could change the SET_RUNTIME_PM_OPS to use the same callbacks for runtime and

system

Quoted 3 lines
Quoted 3 lines

suspend, but I think that's a bad idea. Runtime suspend is not supposed to turn off the streaming. The driver is supposed to turn off the streaming, then call runtime_put, which would result in runtime

suspend callback getting called.

Quoted 3 lines
Quoted 1 line

I implemented system suspend/resume based on this patch, I feel good

about it.

Quoted 3 lines

https://lore.kernel.org/all/20240326031237.25331-1-changhuang.liang@st
arfivetech.com/

Quoted 8 lines

Thanks for carrying this patch in your series.

I think Tomi's point still holds - only the system suspend hook should try to stop/start the CSI2RX device.

Runtime PM hooks are usually only called when there are no users, so no active streams.

OK, I will modify this in my series.

Regards,
Changhuang

author
#6 lore raw

Re: Re: [PATCH v2 01/13] media: cadence: csi2rx: Support runtime PM

In reply to Tomi Valkeinen

Hi Tomi,

Quoted 75 lines · Tomi Valkeinen

Hi,

Quoted 72 lines · Jai Luthra

From: Jayshri Pawar <[email protected]>

Use runtime power management hooks to save power when CSI-RX is not in use. Also stop/start any in-progress streams, which might happen during a system suspend/resume cycle.

---
  drivers/media/platform/cadence/cdns-csi2rx.c | 43 +++++++++++++++++++++++++++-
  1 file changed, 42 insertions(+), 1 deletion(-)
drivers/media/platform/cadence/cdns-csi2rx.c +33 -1
index 6f7d27a48eff..751eadbe61ef 100644
@@ -366,6 +366,12 @@ static int csi2rx_s_stream(struct v4l2_subdev *subdev, int enable)
  	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
  	int ret = 0;
+	if (enable) {
+		ret = pm_runtime_resume_and_get(csi2rx->dev);
+		if (ret < 0)
+			return ret;
+	}
+
  	mutex_lock(&csi2rx->lock);
  	if (enable) {
@@ -375,8 +381,10 @@ static int csi2rx_s_stream(struct v4l2_subdev *subdev, int enable)
  		 */
  		if (!csi2rx->count) {
  			ret = csi2rx_start(csi2rx);
-			if (ret)
+			if (ret) {
+				pm_runtime_put(csi2rx->dev);
  				goto out;
+			}
  		}
  		csi2rx->count++;
@@ -388,6 +396,8 @@ static int csi2rx_s_stream(struct v4l2_subdev *subdev, int enable)
  		 */
  		if (!csi2rx->count)
  			csi2rx_stop(csi2rx);
+
+		pm_runtime_put(csi2rx->dev);
  	}
  out:
@@ -661,6 +671,29 @@ static int csi2rx_parse_dt(struct csi2rx_priv *csi2rx)
  	return ret;
  }
+static int csi2rx_suspend(struct device *dev)
+{
+	struct csi2rx_priv *csi2rx = dev_get_drvdata(dev);
+
+	mutex_lock(&csi2rx->lock);
+	if (csi2rx->count)
+		csi2rx_stop(csi2rx);
+	mutex_unlock(&csi2rx->lock);
+
+	return 0;
+}
+
+static int csi2rx_resume(struct device *dev)
+{
+	struct csi2rx_priv *csi2rx = dev_get_drvdata(dev);
+
+	mutex_lock(&csi2rx->lock);
+	if (csi2rx->count)
+		csi2rx_start(csi2rx);
+	mutex_unlock(&csi2rx->lock);
+	return 0;
+}
Quoted 8 lines

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_parse_dt(struct csi2rx_priv *csi2rx) · line 695

Quoted 1 line
drivers/media/platform/cadence/cdns-csi2rx.c +1 -0
+

I don't think this looks correct. Afaik the runtime suspend/resume is not called on system suspend/resume. You could change the SET_RUNTIME_PM_OPS to use the same callbacks for runtime and system suspend, but I think that's a bad idea. Runtime suspend is not supposed to turn off the streaming. The driver is supposed to turn off the streaming, then call runtime_put, which would result in runtime suspend callback getting called.

Agreed, runtime PM should only be called once streams are stopped and device is not in use.

These hooks look like system suspend/resume hooks to me.

I don't think this patch makes much sense in this series anyway, I was carrying it to bring the drivers at parity with the vendor kernel.

I will drop this from my series and let Changhuang carry it in his series for overall PM support for cdns-csi2rx.

Quoted 44 lines

Tomi

Quoted 40 lines
drivers/media/platform/cadence/cdns-csi2rx.c +8 -0
  static int csi2rx_probe(struct platform_device *pdev)
  {
  	struct csi2rx_priv *csi2rx;
@@ -707,6 +740,7 @@ static int csi2rx_probe(struct platform_device *pdev)
  	if (ret)
  		goto err_cleanup;
+	pm_runtime_enable(csi2rx->dev);
  	ret = v4l2_async_register_subdev(&csi2rx->subdev);
  	if (ret < 0)
  		goto err_free_state;
@@ -721,6 +755,7 @@ static int csi2rx_probe(struct platform_device *pdev)
  err_free_state:
  	v4l2_subdev_cleanup(&csi2rx->subdev);
+	pm_runtime_disable(csi2rx->dev);
  err_cleanup:
  	v4l2_async_nf_unregister(&csi2rx->notifier);
  	v4l2_async_nf_cleanup(&csi2rx->notifier);
@@ -739,9 +774,14 @@ static void csi2rx_remove(struct platform_device *pdev)
  	v4l2_async_unregister_subdev(&csi2rx->subdev);
  	v4l2_subdev_cleanup(&csi2rx->subdev);
  	media_entity_cleanup(&csi2rx->subdev.entity);
+	pm_runtime_disable(csi2rx->dev);
  	kfree(csi2rx);
  }
+static const struct dev_pm_ops csi2rx_pm_ops = {
+	SET_RUNTIME_PM_OPS(csi2rx_suspend, csi2rx_resume, NULL)
+};
+
  static const struct of_device_id csi2rx_of_table[] = {
  	{ .compatible = "starfive,jh7110-csi2rx" },
  	{ .compatible = "cdns,csi2rx" },
@@ -756,6 +796,7 @@ static struct platform_driver csi2rx_driver = {
  	.driver	= {
  		.name		= "cdns-csi2rx",
  		.of_match_table	= csi2rx_of_table,
+		.pm		= &csi2rx_pm_ops,
  	},
  };
  module_platform_driver(csi2rx_driver);
Signature · 4 lines
Thanks,
Jai

GPG Fingerprint: 4DE0 D818 E5D5 75E8 D45A AFC5 43DE 91F9 249A 7145
author
#7 lore raw

PATCH v2 2/13 dt-bindings: media: ti,j721e-csi2rx-shim: Support 32 dma chans

In reply to Jai Luthra

The CSI2RX SHIM IP can support a maximum of 32x DMA channels.

These can be used to split incoming "streams" of data on the CSI-RX port, distinguished by MIPI Virtual Channel (or Data Type), into different locations in memory (/dev/videoX nodes).

Actual number of DMA channels reserved is different for each SoC integrating this IP, but a maximum of 32x channels are always available in this IP's register space, so set minimum as 1 and maximum as 32.

---
 .../bindings/media/ti,j721e-csi2rx-shim.yaml       | 39 ++++++++++++++++++++--
 1 file changed, 36 insertions(+), 3 deletions(-)
Documentation/devicetree/bindings/media/ti,j721e-csi2rx-shim.yaml +36 -3
index f762fdc05e4d..0e00533c7b68 100644
@@ -20,11 +20,44 @@ properties:
     const: ti,j721e-csi2rx-shim
 
   dmas:
-    maxItems: 1
+    minItems: 1
+    maxItems: 32
 
   dma-names:
+    minItems: 1
     items:
       - const: rx0
+      - const: rx1
+      - const: rx2
+      - const: rx3
+      - const: rx4
+      - const: rx5
+      - const: rx6
+      - const: rx7
+      - const: rx8
+      - const: rx9
+      - const: rx10
+      - const: rx11
+      - const: rx12
+      - const: rx13
+      - const: rx14
+      - const: rx15
+      - const: rx16
+      - const: rx17
+      - const: rx18
+      - const: rx19
+      - const: rx20
+      - const: rx21
+      - const: rx22
+      - const: rx23
+      - const: rx24
+      - const: rx25
+      - const: rx26
+      - const: rx27
+      - const: rx28
+      - const: rx29
+      - const: rx30
+      - const: rx31
 
   reg:
     maxItems: 1
@@ -62,8 +95,8 @@ examples:
 
     ti_csi2rx0: ticsi2rx@4500000 {
         compatible = "ti,j721e-csi2rx-shim";
-        dmas = <&main_udmap 0x4940>;
-        dma-names = "rx0";
+        dmas = <&main_udmap 0x4940>, <&main_udmap 0x4941>;
+        dma-names = "rx0", "rx1";
         reg = <0x4500000 0x1000>;
         power-domains = <&k3_pds 26 TI_SCI_PD_EXCLUSIVE>;
         #address-cells = <1>;
Signature · 1 line
2.43.0
Reviewed-by on v2 2/13
#8 lore raw

Re: [PATCH v2 02/13] dt-bindings: media: ti,j721e-csi2rx-shim: Support 32 dma chans

Quoted 5 lines · Jai Luthra

The CSI2RX SHIM IP can support a maximum of 32x DMA channels.

These can be used to split incoming "streams" of data on the CSI-RX port, distinguished by MIPI Virtual Channel (or Data Type), into different locations in memory (/dev/videoX nodes).

Usually you shouldn't talk about Linux specifics in DT bindings. The DT bindings are only about the HW, and the OS doesn't matter. It doesn't really matter much, but I'd just leave out the mention to /dev/videoX.

Quoted 3 lines

Actual number of DMA channels reserved is different for each SoC integrating this IP, but a maximum of 32x channels are always available in this IP's register space, so set minimum as 1 and maximum as 32.

So in the SoC's dts file you will set the number of channels to the maximum supported by that SoC? I guess that's fine.

Tomi

Quoted 68 lines
---
  .../bindings/media/ti,j721e-csi2rx-shim.yaml       | 39 ++++++++++++++++++++--
  1 file changed, 36 insertions(+), 3 deletions(-)
Documentation/devicetree/bindings/media/ti,j721e-csi2rx-shim.yaml +36 -3
index f762fdc05e4d..0e00533c7b68 100644
@@ -20,11 +20,44 @@ properties:
      const: ti,j721e-csi2rx-shim
 
    dmas:
-    maxItems: 1
+    minItems: 1
+    maxItems: 32
 
    dma-names:
+    minItems: 1
      items:
        - const: rx0
+      - const: rx1
+      - const: rx2
+      - const: rx3
+      - const: rx4
+      - const: rx5
+      - const: rx6
+      - const: rx7
+      - const: rx8
+      - const: rx9
+      - const: rx10
+      - const: rx11
+      - const: rx12
+      - const: rx13
+      - const: rx14
+      - const: rx15
+      - const: rx16
+      - const: rx17
+      - const: rx18
+      - const: rx19
+      - const: rx20
+      - const: rx21
+      - const: rx22
+      - const: rx23
+      - const: rx24
+      - const: rx25
+      - const: rx26
+      - const: rx27
+      - const: rx28
+      - const: rx29
+      - const: rx30
+      - const: rx31
 
    reg:
      maxItems: 1
@@ -62,8 +95,8 @@ examples:
 
      ti_csi2rx0: ticsi2rx@4500000 {
          compatible = "ti,j721e-csi2rx-shim";
-        dmas = <&main_udmap 0x4940>;
-        dma-names = "rx0";
+        dmas = <&main_udmap 0x4940>, <&main_udmap 0x4941>;
+        dma-names = "rx0", "rx1";
          reg = <0x4500000 0x1000>;
          power-domains = <&k3_pds 26 TI_SCI_PD_EXCLUSIVE>;
          #address-cells = <1>;
author
#9 lore raw

Re: Re: [PATCH v2 02/13] dt-bindings: media: ti,j721e-csi2rx-shim: Support 32 dma chans

Thanks for the review.

Quoted 10 lines · Tomi Valkeinen
Quoted 5 lines · Jai Luthra

The CSI2RX SHIM IP can support a maximum of 32x DMA channels.

These can be used to split incoming "streams" of data on the CSI-RX port, distinguished by MIPI Virtual Channel (or Data Type), into different locations in memory (/dev/videoX nodes).

Usually you shouldn't talk about Linux specifics in DT bindings. The DT bindings are only about the HW, and the OS doesn't matter. It doesn't really matter much, but I'd just leave out the mention to /dev/videoX.

My bad, will drop the reference to /dev/videoX in next revision.

Quoted 14 lines
Quoted 4 lines

Actual number of DMA channels reserved is different for each SoC integrating this IP, but a maximum of 32x channels are always available in this IP's register space, so set minimum as 1 and maximum as 32.

So in the SoC's dts file you will set the number of channels to the maximum supported by that SoC? I guess that's fine.

Tomi

[...]

Signature · 4 lines
Thanks,
Jai

GPG Fingerprint: 4DE0 D818 E5D5 75E8 D45A AFC5 43DE 91F9 249A 7145
#10 lore raw

Re: [PATCH v2 02/13] dt-bindings: media: ti,j721e-csi2rx-shim: Support 32 dma chans

In reply to Jai Luthra
Quoted 22 lines · Jai Luthra

The CSI2RX SHIM IP can support a maximum of 32x DMA channels.

These can be used to split incoming "streams" of data on the CSI-RX port, distinguished by MIPI Virtual Channel (or Data Type), into different locations in memory (/dev/videoX nodes).

Actual number of DMA channels reserved is different for each SoC integrating this IP, but a maximum of 32x channels are always available in this IP's register space, so set minimum as 1 and maximum as 32.

---
 .../bindings/media/ti,j721e-csi2rx-shim.yaml       | 39 ++++++++++++++++++++--
 1 file changed, 36 insertions(+), 3 deletions(-)
Documentation/devicetree/bindings/media/ti,j721e-csi2rx-shim.yaml +0 -0
index f762fdc05e4d..0e00533c7b68 100644
@@ -20,11 +20,44 @@ properties:
     const: ti,j721e-csi2rx-shim

Documentation/devicetree/bindings/media/ti,j721e-csi2rx-shim.yaml properties: · line 29

Quoted 10 lines
Documentation/devicetree/bindings/media/ti,j721e-csi2rx-shim.yaml +3 -1
 
   dmas:
-    maxItems: 1
+    minItems: 1
+    maxItems: 32
 
   dma-names:
+    minItems: 1
     items:
       - const: rx0

So all variants now get total random number of DMAs? I don't understand why this is not constrained.

Best regards,
Krzysztof

author
#11 lore raw

Re: Re: [PATCH v2 02/13] dt-bindings: media: ti,j721e-csi2rx-shim: Support 32 dma chans

Hi Krzysztof,

Thanks for the review.

Quoted 10 lines · Krzysztof Kozlowski
Quoted 9 lines · Jai Luthra

The CSI2RX SHIM IP can support a maximum of 32x DMA channels.

These can be used to split incoming "streams" of data on the CSI-RX port, distinguished by MIPI Virtual Channel (or Data Type), into different locations in memory (/dev/videoX nodes).

Actual number of DMA channels reserved is different for each SoC integrating this IP, but a maximum of 32x channels are always available in this IP's register space, so set minimum as 1 and maximum as 32.

Sorry, this above paragraph is incorrect. All variants of SoCs integrating this IP can in-fact support 32 channels.

I will update the commit description in the next revision to make this clearer.

Quoted 19 lines
Quoted 19 lines
---
 .../bindings/media/ti,j721e-csi2rx-shim.yaml       | 39 ++++++++++++++++++++--
 1 file changed, 36 insertions(+), 3 deletions(-)
Documentation/devicetree/bindings/media/ti,j721e-csi2rx-shim.yaml +2 -1
index f762fdc05e4d..0e00533c7b68 100644
@@ -20,11 +20,44 @@ properties:
     const: ti,j721e-csi2rx-shim
 
   dmas:
-    maxItems: 1
+    minItems: 1
+    maxItems: 32
 
Quoted 7 lines

Documentation/devicetree/bindings/media/ti,j721e-csi2rx-shim.yaml properties: · line 29

Quoted 4 lines
Documentation/devicetree/bindings/media/ti,j721e-csi2rx-shim.yaml +1 -0
   dma-names:
+    minItems: 1
     items:
       - const: rx0

So all variants now get total random number of DMAs? I don't understand why this is not constrained.

The default system-level resource partitioning allocates < 32 channels for some platforms, though that can be changed by the user [1].

I don't think a separate compatible for constraints makes sense here, as this is not an IP or SoC-level constraint, and only depends on how the end-user allocates the channels across peripherals.

Quoted 4 lines

Best regards,
Krzysztof

[1] https://software-dl.ti.com/processor-sdk-linux/esd/AM62X/latest/exports/docs/linux/How_to_Guides/Host/K3_Resource_Partitioning_Tool.html

Signature · 4 lines
Thanks,
Jai

GPG Fingerprint: 4DE0 D818 E5D5 75E8 D45A AFC5 43DE 91F9 249A 7145
author
#12 lore raw

PATCH v2 3/13 media: ti: j721e-csi2rx: separate out device and context

In reply to Jai Luthra

The TI CSI2RX wrapper has two parts: the main device and the DMA contexts. The driver was originally written with single camera capture in mind, so only one DMA context was needed. For the sake of simplicity, the context specific stuff was not modeled different to the main device.

To enable multiplexed stream capture, the contexts need to be separated out from the main device. Create a struct ti_csi2rx_ctx that holds the DMA context specific things. Separate out functions handling the device and context related functionality.

---
 .../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c  | 423 ++++++++++++---------
 1 file changed, 242 insertions(+), 181 deletions(-)
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +242 -181
index 22442fce7607..d8dfe0002b72 100644
@@ -40,6 +40,8 @@
 #define SHIM_PSI_CFG0_DST_TAG		GENMASK(31, 16)
 
 #define PSIL_WORD_SIZE_BYTES		16
+#define TI_CSI2RX_NUM_CTX		1
+
 /*
  * There are no hard limits on the width or height. The DMA engine can handle
  * all sizes. The max width and height are arbitrary numbers for this driver.
@@ -64,7 +66,7 @@ struct ti_csi2rx_buffer {
 	/* Common v4l2 buffer. Must be first. */
 	struct vb2_v4l2_buffer		vb;
 	struct list_head		list;
-	struct ti_csi2rx_dev		*csi;
+	struct ti_csi2rx_ctx		*ctx;
 };
 
 enum ti_csi2rx_dma_state {
@@ -84,29 +86,37 @@ struct ti_csi2rx_dma {
 	 * Queue of buffers submitted to DMA engine.
 	 */
 	struct list_head		submitted;
-	/* Buffer to drain stale data from PSI-L endpoint */
-	struct {
-		void			*vaddr;
-		dma_addr_t		paddr;
-		size_t			len;
-	} drain;
+};
+
+struct ti_csi2rx_dev;
+
+struct ti_csi2rx_ctx {
+	struct ti_csi2rx_dev		*csi;
+	struct video_device		vdev;
+	struct vb2_queue		vidq;
+	struct mutex			mutex; /* To serialize ioctls. */
+	struct v4l2_format		v_fmt;
+	struct ti_csi2rx_dma		dma;
+	u32				sequence;
+	u32				idx;
 };
 
 struct ti_csi2rx_dev {
 	struct device			*dev;
 	void __iomem			*shim;
 	struct v4l2_device		v4l2_dev;
-	struct video_device		vdev;
 	struct media_device		mdev;
 	struct media_pipeline		pipe;
 	struct media_pad		pad;
 	struct v4l2_async_notifier	notifier;
 	struct v4l2_subdev		*source;
-	struct vb2_queue		vidq;
-	struct mutex			mutex; /* To serialize ioctls. */
-	struct v4l2_format		v_fmt;
-	struct ti_csi2rx_dma		dma;
-	u32				sequence;
+	struct ti_csi2rx_ctx		ctx[TI_CSI2RX_NUM_CTX];
+	/* Buffer to drain stale data from PSI-L endpoint */
+	struct {
+		void			*vaddr;
+		dma_addr_t		paddr;
+		size_t			len;
+	} drain;
 };
 
 static const struct ti_csi2rx_fmt ti_csi2rx_formats[] = {
@@ -212,7 +222,7 @@ static const struct ti_csi2rx_fmt ti_csi2rx_formats[] = {
 };
 
 /* Forward declaration needed by ti_csi2rx_dma_callback. */
-static int ti_csi2rx_start_dma(struct ti_csi2rx_dev *csi,
+static int ti_csi2rx_start_dma(struct ti_csi2rx_ctx *ctx,
 			       struct ti_csi2rx_buffer *buf);
 
 static const struct ti_csi2rx_fmt *find_format_by_fourcc(u32 pixelformat)
@@ -302,7 +312,7 @@ static int ti_csi2rx_enum_fmt_vid_cap(struct file *file, void *priv,
 static int ti_csi2rx_g_fmt_vid_cap(struct file *file, void *prov,
 				   struct v4l2_format *f)
 {
-	struct ti_csi2rx_dev *csi = video_drvdata(file);
+	struct ti_csi2rx_ctx *csi = video_drvdata(file);
 
 	*f = csi->v_fmt;
 
@@ -333,7 +343,7 @@ static int ti_csi2rx_try_fmt_vid_cap(struct file *file, void *priv,
 static int ti_csi2rx_s_fmt_vid_cap(struct file *file, void *priv,
 				   struct v4l2_format *f)
 {
-	struct ti_csi2rx_dev *csi = video_drvdata(file);
+	struct ti_csi2rx_ctx *csi = video_drvdata(file);
 	struct vb2_queue *q = &csi->vidq;
 	int ret;
 
@@ -419,25 +429,33 @@ static int csi_async_notifier_bound(struct v4l2_async_notifier *notifier,
 static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
 {
 	struct ti_csi2rx_dev *csi = dev_get_drvdata(notifier->v4l2_dev->dev);
-	struct video_device *vdev = &csi->vdev;
-	int ret;
+	int ret, i;
 
-	ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
-	if (ret)
-		return ret;
+	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
+		struct ti_csi2rx_ctx *ctx = &csi->ctx[i];
+		struct video_device *vdev = &ctx->vdev;
 
-	ret = v4l2_create_fwnode_links_to_pad(csi->source, &csi->pad,
-					      MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED);
-
-	if (ret) {
-		video_unregister_device(vdev);
-		return ret;
+		ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
+		if (ret)
+			goto unregister_dev;
 	}
 
+	ret = v4l2_create_fwnode_links_to_pad(csi->source, &csi->pad,
+					      MEDIA_LNK_FL_IMMUTABLE |
+					      MEDIA_LNK_FL_ENABLED);
+	if (ret)
+		goto unregister_dev;
+
 	ret = v4l2_device_register_subdev_nodes(&csi->v4l2_dev);
 	if (ret)
-		video_unregister_device(vdev);
+		goto unregister_dev;
+
+	return 0;
 
+unregister_dev:
+	i--;
+	for (; i >= 0; i--)
+		video_unregister_device(&csi->ctx[i].vdev);
 	return ret;
 }
 
@@ -483,12 +501,13 @@ static int ti_csi2rx_notifier_register(struct ti_csi2rx_dev *csi)
 	return 0;
 }
 
-static void ti_csi2rx_setup_shim(struct ti_csi2rx_dev *csi)
+static void ti_csi2rx_setup_shim(struct ti_csi2rx_ctx *ctx)
 {
+	struct ti_csi2rx_dev *csi = ctx->csi;
 	const struct ti_csi2rx_fmt *fmt;
 	unsigned int reg;
 
-	fmt = find_format_by_fourcc(csi->v_fmt.fmt.pix.pixelformat);
+	fmt = find_format_by_fourcc(ctx->v_fmt.fmt.pix.pixelformat);
 
 	/* De-assert the pixel interface reset. */
 	reg = SHIM_CNTL_PIX_RST;
@@ -555,8 +574,9 @@ static void ti_csi2rx_drain_callback(void *param)
  * To prevent that stale data corrupting the subsequent transactions, it is
  * required to issue DMA requests to drain it out.
  */
-static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
+static int ti_csi2rx_drain_dma(struct ti_csi2rx_ctx *ctx)
 {
+	struct ti_csi2rx_dev *csi = ctx->csi;
 	struct dma_async_tx_descriptor *desc;
 	struct completion drain_complete;
 	dma_cookie_t cookie;
@@ -564,8 +584,8 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
 
 	init_completion(&drain_complete);
 
-	desc = dmaengine_prep_slave_single(csi->dma.chan, csi->dma.drain.paddr,
-					   csi->dma.drain.len, DMA_DEV_TO_MEM,
+	desc = dmaengine_prep_slave_single(ctx->dma.chan, csi->drain.paddr,
+					   csi->drain.len, DMA_DEV_TO_MEM,
 					   DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
 	if (!desc) {
 		ret = -EIO;
@@ -580,11 +600,11 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
 	if (ret)
 		goto out;
 
-	dma_async_issue_pending(csi->dma.chan);
+	dma_async_issue_pending(ctx->dma.chan);
 
 	if (!wait_for_completion_timeout(&drain_complete,
 					 msecs_to_jiffies(DRAIN_TIMEOUT_MS))) {
-		dmaengine_terminate_sync(csi->dma.chan);
+		dmaengine_terminate_sync(ctx->dma.chan);
 		dev_dbg(csi->dev, "DMA transfer timed out for drain buffer\n");
 		ret = -ETIMEDOUT;
 		goto out;
@@ -596,8 +616,8 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
 static void ti_csi2rx_dma_callback(void *param)
 {
 	struct ti_csi2rx_buffer *buf = param;
-	struct ti_csi2rx_dev *csi = buf->csi;
-	struct ti_csi2rx_dma *dma = &csi->dma;
+	struct ti_csi2rx_ctx *ctx = buf->ctx;
+	struct ti_csi2rx_dma *dma = &ctx->dma;
 	unsigned long flags;
 
 	/*
@@ -605,7 +625,7 @@ static void ti_csi2rx_dma_callback(void *param)
 	 * hardware monitor registers.
 	 */
 	buf->vb.vb2_buf.timestamp = ktime_get_ns();
-	buf->vb.sequence = csi->sequence++;
+	buf->vb.sequence = ctx->sequence++;
 
 	spin_lock_irqsave(&dma->lock, flags);
 
@@ -617,8 +637,9 @@ static void ti_csi2rx_dma_callback(void *param)
 	while (!list_empty(&dma->queue)) {
 		buf = list_entry(dma->queue.next, struct ti_csi2rx_buffer, list);
 
-		if (ti_csi2rx_start_dma(csi, buf)) {
-			dev_err(csi->dev, "Failed to queue the next buffer for DMA\n");
+		if (ti_csi2rx_start_dma(ctx, buf)) {
+			dev_err(ctx->csi->dev,
+				"Failed to queue the next buffer for DMA\n");
 			vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
 		} else {
 			list_move_tail(&buf->list, &dma->submitted);
@@ -631,17 +652,17 @@ static void ti_csi2rx_dma_callback(void *param)
 	spin_unlock_irqrestore(&dma->lock, flags);
 }
 
-static int ti_csi2rx_start_dma(struct ti_csi2rx_dev *csi,
+static int ti_csi2rx_start_dma(struct ti_csi2rx_ctx *ctx,
 			       struct ti_csi2rx_buffer *buf)
 {
 	unsigned long addr;
 	struct dma_async_tx_descriptor *desc;
-	size_t len = csi->v_fmt.fmt.pix.sizeimage;
+	size_t len = ctx->v_fmt.fmt.pix.sizeimage;
 	dma_cookie_t cookie;
 	int ret = 0;
 
 	addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
-	desc = dmaengine_prep_slave_single(csi->dma.chan, addr, len,
+	desc = dmaengine_prep_slave_single(ctx->dma.chan, addr, len,
 					   DMA_DEV_TO_MEM,
 					   DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
 	if (!desc)
@@ -655,20 +676,20 @@ static int ti_csi2rx_start_dma(struct ti_csi2rx_dev *csi,
 	if (ret)
 		return ret;
 
-	dma_async_issue_pending(csi->dma.chan);
+	dma_async_issue_pending(ctx->dma.chan);
 
 	return 0;
 }
 
-static void ti_csi2rx_stop_dma(struct ti_csi2rx_dev *csi)
+static void ti_csi2rx_stop_dma(struct ti_csi2rx_ctx *ctx)
 {
-	struct ti_csi2rx_dma *dma = &csi->dma;
+	struct ti_csi2rx_dma *dma = &ctx->dma;
 	enum ti_csi2rx_dma_state state;
 	unsigned long flags;
 	int ret;
 
 	spin_lock_irqsave(&dma->lock, flags);
-	state = csi->dma.state;
+	state = ctx->dma.state;
 	dma->state = TI_CSI2RX_DMA_STOPPED;
 	spin_unlock_irqrestore(&dma->lock, flags);
 
@@ -679,30 +700,30 @@ static void ti_csi2rx_stop_dma(struct ti_csi2rx_dev *csi)
 		 * is stopped, as the module-level pixel reset cannot be
 		 * enforced before terminating DMA.
 		 */
-		ret = ti_csi2rx_drain_dma(csi);
+		ret = ti_csi2rx_drain_dma(ctx);
 		if (ret && ret != -ETIMEDOUT)
-			dev_warn(csi->dev,
+			dev_warn(ctx->csi->dev,
 				 "Failed to drain DMA. Next frame might be bogus\n");
 	}
 
-	ret = dmaengine_terminate_sync(csi->dma.chan);
+	ret = dmaengine_terminate_sync(ctx->dma.chan);
 	if (ret)
-		dev_err(csi->dev, "Failed to stop DMA: %d\n", ret);
+		dev_err(ctx->csi->dev, "Failed to stop DMA: %d\n", ret);
 }
 
-static void ti_csi2rx_cleanup_buffers(struct ti_csi2rx_dev *csi,
+static void ti_csi2rx_cleanup_buffers(struct ti_csi2rx_ctx *ctx,
 				      enum vb2_buffer_state state)
 {
-	struct ti_csi2rx_dma *dma = &csi->dma;
+	struct ti_csi2rx_dma *dma = &ctx->dma;
 	struct ti_csi2rx_buffer *buf, *tmp;
 	unsigned long flags;
 
 	spin_lock_irqsave(&dma->lock, flags);
-	list_for_each_entry_safe(buf, tmp, &csi->dma.queue, list) {
+	list_for_each_entry_safe(buf, tmp, &ctx->dma.queue, list) {
 		list_del(&buf->list);
 		vb2_buffer_done(&buf->vb.vb2_buf, state);
 	}
-	list_for_each_entry_safe(buf, tmp, &csi->dma.submitted, list) {
+	list_for_each_entry_safe(buf, tmp, &ctx->dma.submitted, list) {
 		list_del(&buf->list);
 		vb2_buffer_done(&buf->vb.vb2_buf, state);
 	}
@@ -713,8 +734,8 @@ static int ti_csi2rx_queue_setup(struct vb2_queue *q, unsigned int *nbuffers,
 				 unsigned int *nplanes, unsigned int sizes[],
 				 struct device *alloc_devs[])
 {
-	struct ti_csi2rx_dev *csi = vb2_get_drv_priv(q);
-	unsigned int size = csi->v_fmt.fmt.pix.sizeimage;
+	struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(q);
+	unsigned int size = ctx->v_fmt.fmt.pix.sizeimage;
 
 	if (*nplanes) {
 		if (sizes[0] < size)
@@ -730,11 +751,11 @@ static int ti_csi2rx_queue_setup(struct vb2_queue *q, unsigned int *nbuffers,
 
 static int ti_csi2rx_buffer_prepare(struct vb2_buffer *vb)
 {
-	struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vb->vb2_queue);
-	unsigned long size = csi->v_fmt.fmt.pix.sizeimage;
+	struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+	unsigned long size = ctx->v_fmt.fmt.pix.sizeimage;
 
 	if (vb2_plane_size(vb, 0) < size) {
-		dev_err(csi->dev, "Data will not fit into plane\n");
+		dev_err(ctx->csi->dev, "Data will not fit into plane\n");
 		return -EINVAL;
 	}
 
@@ -744,15 +765,15 @@ static int ti_csi2rx_buffer_prepare(struct vb2_buffer *vb)
 
 static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
 {
-	struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vb->vb2_queue);
+	struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
 	struct ti_csi2rx_buffer *buf;
-	struct ti_csi2rx_dma *dma = &csi->dma;
+	struct ti_csi2rx_dma *dma = &ctx->dma;
 	bool restart_dma = false;
 	unsigned long flags = 0;
 	int ret;
 
 	buf = container_of(vb, struct ti_csi2rx_buffer, vb.vb2_buf);
-	buf->csi = csi;
+	buf->ctx = ctx;
 
 	spin_lock_irqsave(&dma->lock, flags);
 	/*
@@ -781,18 +802,18 @@ static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
 		 * the application and will only confuse it. Issue a DMA
 		 * transaction to drain that up.
 		 */
-		ret = ti_csi2rx_drain_dma(csi);
+		ret = ti_csi2rx_drain_dma(ctx);
 		if (ret && ret != -ETIMEDOUT)
-			dev_warn(csi->dev,
+			dev_warn(ctx->csi->dev,
 				 "Failed to drain DMA. Next frame might be bogus\n");
 
 		spin_lock_irqsave(&dma->lock, flags);
-		ret = ti_csi2rx_start_dma(csi, buf);
+		ret = ti_csi2rx_start_dma(ctx, buf);
 		if (ret) {
 			vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
 			dma->state = TI_CSI2RX_DMA_IDLE;
 			spin_unlock_irqrestore(&dma->lock, flags);
-			dev_err(csi->dev, "Failed to start DMA: %d\n", ret);
+			dev_err(ctx->csi->dev, "Failed to start DMA: %d\n", ret);
 		} else {
 			list_add_tail(&buf->list, &dma->submitted);
 			spin_unlock_irqrestore(&dma->lock, flags);
@@ -802,8 +823,9 @@ static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
 
 static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 {
-	struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vq);
-	struct ti_csi2rx_dma *dma = &csi->dma;
+	struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vq);
+	struct ti_csi2rx_dev *csi = ctx->csi;
+	struct ti_csi2rx_dma *dma = &ctx->dma;
 	struct ti_csi2rx_buffer *buf;
 	unsigned long flags;
 	int ret = 0;
@@ -815,18 +837,18 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 	if (ret)
 		return ret;
 
-	ret = video_device_pipeline_start(&csi->vdev, &csi->pipe);
+	ret = video_device_pipeline_start(&ctx->vdev, &csi->pipe);
 	if (ret)
 		goto err;
 
-	ti_csi2rx_setup_shim(csi);
+	ti_csi2rx_setup_shim(ctx);
 
-	csi->sequence = 0;
+	ctx->sequence = 0;
 
 	spin_lock_irqsave(&dma->lock, flags);
 	buf = list_entry(dma->queue.next, struct ti_csi2rx_buffer, list);
 
-	ret = ti_csi2rx_start_dma(csi, buf);
+	ret = ti_csi2rx_start_dma(ctx, buf);
 	if (ret) {
 		dev_err(csi->dev, "Failed to start DMA: %d\n", ret);
 		spin_unlock_irqrestore(&dma->lock, flags);
@@ -844,22 +866,23 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 	return 0;
 
 err_dma:
-	ti_csi2rx_stop_dma(csi);
+	ti_csi2rx_stop_dma(ctx);
 err_pipeline:
-	video_device_pipeline_stop(&csi->vdev);
+	video_device_pipeline_stop(&ctx->vdev);
 	writel(0, csi->shim + SHIM_CNTL);
 	writel(0, csi->shim + SHIM_DMACNTX);
 err:
-	ti_csi2rx_cleanup_buffers(csi, VB2_BUF_STATE_QUEUED);
+	ti_csi2rx_cleanup_buffers(ctx, VB2_BUF_STATE_QUEUED);
 	return ret;
 }
 
 static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
 {
-	struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vq);
+	struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vq);
+	struct ti_csi2rx_dev *csi = ctx->csi;
 	int ret;
 
-	video_device_pipeline_stop(&csi->vdev);
+	video_device_pipeline_stop(&ctx->vdev);
 
 	writel(0, csi->shim + SHIM_CNTL);
 	writel(0, csi->shim + SHIM_DMACNTX);
@@ -868,8 +891,8 @@ static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
 	if (ret)
 		dev_err(csi->dev, "Failed to stop subdev stream\n");
 
-	ti_csi2rx_stop_dma(csi);
-	ti_csi2rx_cleanup_buffers(csi, VB2_BUF_STATE_ERROR);
+	ti_csi2rx_stop_dma(ctx);
+	ti_csi2rx_cleanup_buffers(ctx, VB2_BUF_STATE_ERROR);
 }
 
 static const struct vb2_ops csi_vb2_qops = {
@@ -882,27 +905,60 @@ static const struct vb2_ops csi_vb2_qops = {
 	.wait_finish = vb2_ops_wait_finish,
 };
 
-static int ti_csi2rx_init_vb2q(struct ti_csi2rx_dev *csi)
+static void ti_csi2rx_cleanup_dma(struct ti_csi2rx_ctx *ctx)
 {
-	struct vb2_queue *q = &csi->vidq;
+	dma_release_channel(ctx->dma.chan);
+}
+
+static void ti_csi2rx_cleanup_v4l2(struct ti_csi2rx_dev *csi)
+{
+	media_device_unregister(&csi->mdev);
+	v4l2_device_unregister(&csi->v4l2_dev);
+	media_device_cleanup(&csi->mdev);
+}
+
+static void ti_csi2rx_cleanup_notifier(struct ti_csi2rx_dev *csi)
+{
+	v4l2_async_nf_unregister(&csi->notifier);
+	v4l2_async_nf_cleanup(&csi->notifier);
+}
+
+static void ti_csi2rx_cleanup_vb2q(struct ti_csi2rx_ctx *ctx)
+{
+	vb2_queue_release(&ctx->vidq);
+}
+
+static void ti_csi2rx_cleanup_ctx(struct ti_csi2rx_ctx *ctx)
+{
+	ti_csi2rx_cleanup_dma(ctx);
+	ti_csi2rx_cleanup_vb2q(ctx);
+
+	video_unregister_device(&ctx->vdev);
+
+	mutex_destroy(&ctx->mutex);
+}
+
+static int ti_csi2rx_init_vb2q(struct ti_csi2rx_ctx *ctx)
+{
+	struct vb2_queue *q = &ctx->vidq;
 	int ret;
 
 	q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
 	q->io_modes = VB2_MMAP | VB2_DMABUF;
-	q->drv_priv = csi;
+	q->drv_priv = ctx;
 	q->buf_struct_size = sizeof(struct ti_csi2rx_buffer);
 	q->ops = &csi_vb2_qops;
 	q->mem_ops = &vb2_dma_contig_memops;
 	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
-	q->dev = dmaengine_get_dma_device(csi->dma.chan);
-	q->lock = &csi->mutex;
+	q->dev = dmaengine_get_dma_device(ctx->dma.chan);
+	q->lock = &ctx->mutex;
 	q->min_queued_buffers = 1;
 
 	ret = vb2_queue_init(q);
 	if (ret)
 		return ret;
 
-	csi->vdev.queue = q;
+	ctx->vdev.queue = q;
 
 	return 0;
 }
@@ -911,8 +967,9 @@ static int ti_csi2rx_link_validate(struct media_link *link)
 {
 	struct media_entity *entity = link->sink->entity;
 	struct video_device *vdev = media_entity_to_video_device(entity);
-	struct ti_csi2rx_dev *csi = container_of(vdev, struct ti_csi2rx_dev, vdev);
-	struct v4l2_pix_format *csi_fmt = &csi->v_fmt.fmt.pix;
+	struct ti_csi2rx_ctx *ctx = container_of(vdev, struct ti_csi2rx_ctx, vdev);
+	struct ti_csi2rx_dev *csi = ctx->csi;
+	struct v4l2_pix_format *csi_fmt = &ctx->v_fmt.fmt.pix;
 	struct v4l2_subdev_format source_fmt = {
 		.which	= V4L2_SUBDEV_FORMAT_ACTIVE,
 		.pad	= link->source->index,
@@ -965,47 +1022,69 @@ static const struct media_entity_operations ti_csi2rx_video_entity_ops = {
 	.link_validate = ti_csi2rx_link_validate,
 };
 
-static int ti_csi2rx_init_dma(struct ti_csi2rx_dev *csi)
+static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
 {
 	struct dma_slave_config cfg = {
 		.src_addr_width = DMA_SLAVE_BUSWIDTH_16_BYTES,
 	};
 	int ret;
 
-	INIT_LIST_HEAD(&csi->dma.queue);
-	INIT_LIST_HEAD(&csi->dma.submitted);
-	spin_lock_init(&csi->dma.lock);
+	INIT_LIST_HEAD(&ctx->dma.queue);
+	INIT_LIST_HEAD(&ctx->dma.submitted);
+	spin_lock_init(&ctx->dma.lock);
 
-	csi->dma.state = TI_CSI2RX_DMA_STOPPED;
+	ctx->dma.state = TI_CSI2RX_DMA_STOPPED;
 
-	csi->dma.chan = dma_request_chan(csi->dev, "rx0");
-	if (IS_ERR(csi->dma.chan))
-		return PTR_ERR(csi->dma.chan);
+	ctx->dma.chan = dma_request_chan(ctx->csi->dev, "rx0");
+	if (IS_ERR(ctx->dma.chan))
+		return PTR_ERR(ctx->dma.chan);
 
-	ret = dmaengine_slave_config(csi->dma.chan, &cfg);
+	ret = dmaengine_slave_config(ctx->dma.chan, &cfg);
 	if (ret) {
-		dma_release_channel(csi->dma.chan);
+		dma_release_channel(ctx->dma.chan);
 		return ret;
 	}
 
-	csi->dma.drain.len = DRAIN_BUFFER_SIZE;
-	csi->dma.drain.vaddr = dma_alloc_coherent(csi->dev, csi->dma.drain.len,
-						  &csi->dma.drain.paddr,
-						  GFP_KERNEL);
-	if (!csi->dma.drain.vaddr)
-		return -ENOMEM;
-
 	return 0;
 }
 
 static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
 {
 	struct media_device *mdev = &csi->mdev;
-	struct video_device *vdev = &csi->vdev;
+	int ret;
+
+	mdev->dev = csi->dev;
+	mdev->hw_revision = 1;
+	strscpy(mdev->model, "TI-CSI2RX", sizeof(mdev->model));
+
+	media_device_init(mdev);
+
+	csi->v4l2_dev.mdev = mdev;
+
+	ret = v4l2_device_register(csi->dev, &csi->v4l2_dev);
+	if (ret)
+		return ret;
+
+	ret = media_device_register(mdev);
+	if (ret) {
+		v4l2_device_unregister(&csi->v4l2_dev);
+		media_device_cleanup(mdev);
+		return ret;
+	}
+
+	return 0;
+}
+
+static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
+{
+	struct ti_csi2rx_dev *csi = ctx->csi;
+	struct video_device *vdev = &ctx->vdev;
 	const struct ti_csi2rx_fmt *fmt;
-	struct v4l2_pix_format *pix_fmt = &csi->v_fmt.fmt.pix;
+	struct v4l2_pix_format *pix_fmt = &ctx->v_fmt.fmt.pix;
 	int ret;
 
+	mutex_init(&ctx->mutex);
+
 	fmt = find_format_by_fourcc(V4L2_PIX_FMT_UYVY);
 	if (!fmt)
 		return -EINVAL;
@@ -1018,15 +1097,16 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
 	pix_fmt->quantization = V4L2_QUANTIZATION_LIM_RANGE,
 	pix_fmt->xfer_func = V4L2_XFER_FUNC_SRGB,
 
-	ti_csi2rx_fill_fmt(fmt, &csi->v_fmt);
-
-	mdev->dev = csi->dev;
-	mdev->hw_revision = 1;
-	strscpy(mdev->model, "TI-CSI2RX", sizeof(mdev->model));
+	ti_csi2rx_fill_fmt(fmt, &ctx->v_fmt);
 
-	media_device_init(mdev);
+	csi->pad.flags = MEDIA_PAD_FL_SINK;
+	vdev->entity.ops = &ti_csi2rx_video_entity_ops;
+	ret = media_entity_pads_init(&ctx->vdev.entity, 1, &csi->pad);
+	if (ret)
+		return ret;
 
-	strscpy(vdev->name, TI_CSI2RX_MODULE_NAME, sizeof(vdev->name));
+	snprintf(vdev->name, sizeof(vdev->name), "%s context %u",
+		 dev_name(csi->dev), ctx->idx);
 	vdev->v4l2_dev = &csi->v4l2_dev;
 	vdev->vfl_dir = VFL_DIR_RX;
 	vdev->fops = &csi_fops;
@@ -1034,61 +1114,28 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
 	vdev->release = video_device_release_empty;
 	vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING |
 			    V4L2_CAP_IO_MC;
-	vdev->lock = &csi->mutex;
-	video_set_drvdata(vdev, csi);
+	vdev->lock = &ctx->mutex;
+	video_set_drvdata(vdev, ctx);
 
-	csi->pad.flags = MEDIA_PAD_FL_SINK;
-	vdev->entity.ops = &ti_csi2rx_video_entity_ops;
-	ret = media_entity_pads_init(&csi->vdev.entity, 1, &csi->pad);
+	ret = ti_csi2rx_init_dma(ctx);
 	if (ret)
 		return ret;
 
-	csi->v4l2_dev.mdev = mdev;
-
-	ret = v4l2_device_register(csi->dev, &csi->v4l2_dev);
+	ret = ti_csi2rx_init_vb2q(ctx);
 	if (ret)
-		return ret;
-
-	ret = media_device_register(mdev);
-	if (ret) {
-		v4l2_device_unregister(&csi->v4l2_dev);
-		media_device_cleanup(mdev);
-		return ret;
-	}
+		goto cleanup_dma;
 
 	return 0;
-}
 
-static void ti_csi2rx_cleanup_dma(struct ti_csi2rx_dev *csi)
-{
-	dma_free_coherent(csi->dev, csi->dma.drain.len,
-			  csi->dma.drain.vaddr, csi->dma.drain.paddr);
-	csi->dma.drain.vaddr = NULL;
-	dma_release_channel(csi->dma.chan);
-}
-
-static void ti_csi2rx_cleanup_v4l2(struct ti_csi2rx_dev *csi)
-{
-	media_device_unregister(&csi->mdev);
-	v4l2_device_unregister(&csi->v4l2_dev);
-	media_device_cleanup(&csi->mdev);
-}
-
-static void ti_csi2rx_cleanup_subdev(struct ti_csi2rx_dev *csi)
-{
-	v4l2_async_nf_unregister(&csi->notifier);
-	v4l2_async_nf_cleanup(&csi->notifier);
-}
-
-static void ti_csi2rx_cleanup_vb2q(struct ti_csi2rx_dev *csi)
-{
-	vb2_queue_release(&csi->vidq);
+cleanup_dma:
+	ti_csi2rx_cleanup_dma(ctx);
+	return ret;
 }
 
 static int ti_csi2rx_probe(struct platform_device *pdev)
 {
 	struct ti_csi2rx_dev *csi;
-	int ret;
+	int ret, i;
 
 	csi = devm_kzalloc(&pdev->dev, sizeof(*csi), GFP_KERNEL);
 	if (!csi)
@@ -1097,62 +1144,76 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
 	csi->dev = &pdev->dev;
 	platform_set_drvdata(pdev, csi);
 
-	mutex_init(&csi->mutex);
 	csi->shim = devm_platform_ioremap_resource(pdev, 0);
 	if (IS_ERR(csi->shim)) {
 		ret = PTR_ERR(csi->shim);
-		goto err_mutex;
+		return ret;
 	}
 
-	ret = ti_csi2rx_init_dma(csi);
-	if (ret)
-		goto err_mutex;
+	csi->drain.len = DRAIN_BUFFER_SIZE;
+	csi->drain.vaddr = dma_alloc_coherent(csi->dev, csi->drain.len,
+					      &csi->drain.paddr,
+					      GFP_KERNEL);
+	if (!csi->drain.vaddr)
+		return -ENOMEM;
 
 	ret = ti_csi2rx_v4l2_init(csi);
-	if (ret)
-		goto err_dma;
-
-	ret = ti_csi2rx_init_vb2q(csi);
 	if (ret)
 		goto err_v4l2;
 
+	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
+		csi->ctx[i].idx = i;
+		csi->ctx[i].csi = csi;
+		ret = ti_csi2rx_init_ctx(&csi->ctx[i]);
+		if (ret)
+			goto err_ctx;
+	}
+
 	ret = ti_csi2rx_notifier_register(csi);
 	if (ret)
-		goto err_vb2q;
+		goto err_ctx;
 
 	ret = of_platform_populate(csi->dev->of_node, NULL, NULL, csi->dev);
 	if (ret) {
 		dev_err(csi->dev, "Failed to create children: %d\n", ret);
-		goto err_subdev;
+		goto err_notifier;
 	}
 
 	return 0;
 
-err_subdev:
-	ti_csi2rx_cleanup_subdev(csi);
-err_vb2q:
-	ti_csi2rx_cleanup_vb2q(csi);
-err_v4l2:
+err_notifier:
+	ti_csi2rx_cleanup_notifier(csi);
+err_ctx:
+	i--;
+	for (; i >= 0; i--)
+		ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
 	ti_csi2rx_cleanup_v4l2(csi);
-err_dma:
-	ti_csi2rx_cleanup_dma(csi);
-err_mutex:
-	mutex_destroy(&csi->mutex);
+err_v4l2:
+	dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
+			  csi->drain.paddr);
 	return ret;
 }
 
 static void ti_csi2rx_remove(struct platform_device *pdev)
 {
 	struct ti_csi2rx_dev *csi = platform_get_drvdata(pdev);
+	int i;
+
+	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
+		if (vb2_is_busy(&csi->ctx[i].vidq))
+			dev_err(csi->dev,
+				"Failed to remove as queue busy for ctx %u\n",
+				i);
+	}
 
-	video_unregister_device(&csi->vdev);
+	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++)
+		ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
 
-	ti_csi2rx_cleanup_vb2q(csi);
-	ti_csi2rx_cleanup_subdev(csi);
+	ti_csi2rx_cleanup_notifier(csi);
 	ti_csi2rx_cleanup_v4l2(csi);
-	ti_csi2rx_cleanup_dma(csi);
 
-	mutex_destroy(&csi->mutex);
+	dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
+			  csi->drain.paddr);
 }
 
 static const struct of_device_id ti_csi2rx_of_match[] = {
Signature · 1 line
2.43.0
#13 lore raw

Re: [PATCH v2 03/13] media: ti: j721e-csi2rx: separate out device and context

Hi Jai

Quoted 475 lines · Jai Luthra

The TI CSI2RX wrapper has two parts: the main device and the DMA contexts. The driver was originally written with single camera capture in mind, so only one DMA context was needed. For the sake of simplicity, the context specific stuff was not modeled different to the main device.

To enable multiplexed stream capture, the contexts need to be separated out from the main device. Create a struct ti_csi2rx_ctx that holds the DMA context specific things. Separate out functions handling the device and context related functionality.

---
 .../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c  | 423 ++++++++++++---------
 1 file changed, 242 insertions(+), 181 deletions(-)
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +103 -80
index 22442fce7607..d8dfe0002b72 100644
@@ -40,6 +40,8 @@
 #define SHIM_PSI_CFG0_DST_TAG		GENMASK(31, 16)
 
 #define PSIL_WORD_SIZE_BYTES		16
+#define TI_CSI2RX_NUM_CTX		1
+
 /*
  * There are no hard limits on the width or height. The DMA engine can handle
  * all sizes. The max width and height are arbitrary numbers for this driver.
@@ -64,7 +66,7 @@ struct ti_csi2rx_buffer {
 	/* Common v4l2 buffer. Must be first. */
 	struct vb2_v4l2_buffer		vb;
 	struct list_head		list;
-	struct ti_csi2rx_dev		*csi;
+	struct ti_csi2rx_ctx		*ctx;
 };
 
 enum ti_csi2rx_dma_state {
@@ -84,29 +86,37 @@ struct ti_csi2rx_dma {
 	 * Queue of buffers submitted to DMA engine.
 	 */
 	struct list_head		submitted;
-	/* Buffer to drain stale data from PSI-L endpoint */
-	struct {
-		void			*vaddr;
-		dma_addr_t		paddr;
-		size_t			len;
-	} drain;
+};
+
+struct ti_csi2rx_dev;
+
+struct ti_csi2rx_ctx {
+	struct ti_csi2rx_dev		*csi;
+	struct video_device		vdev;
+	struct vb2_queue		vidq;
+	struct mutex			mutex; /* To serialize ioctls. */
+	struct v4l2_format		v_fmt;
+	struct ti_csi2rx_dma		dma;
+	u32				sequence;
+	u32				idx;
 };
 
 struct ti_csi2rx_dev {
 	struct device			*dev;
 	void __iomem			*shim;
 	struct v4l2_device		v4l2_dev;
-	struct video_device		vdev;
 	struct media_device		mdev;
 	struct media_pipeline		pipe;
 	struct media_pad		pad;
 	struct v4l2_async_notifier	notifier;
 	struct v4l2_subdev		*source;
-	struct vb2_queue		vidq;
-	struct mutex			mutex; /* To serialize ioctls. */
-	struct v4l2_format		v_fmt;
-	struct ti_csi2rx_dma		dma;
-	u32				sequence;
+	struct ti_csi2rx_ctx		ctx[TI_CSI2RX_NUM_CTX];
+	/* Buffer to drain stale data from PSI-L endpoint */
+	struct {
+		void			*vaddr;
+		dma_addr_t		paddr;
+		size_t			len;
+	} drain;
 };
 
 static const struct ti_csi2rx_fmt ti_csi2rx_formats[] = {
@@ -212,7 +222,7 @@ static const struct ti_csi2rx_fmt ti_csi2rx_formats[] = {
 };
 
 /* Forward declaration needed by ti_csi2rx_dma_callback. */
-static int ti_csi2rx_start_dma(struct ti_csi2rx_dev *csi,
+static int ti_csi2rx_start_dma(struct ti_csi2rx_ctx *ctx,
 			       struct ti_csi2rx_buffer *buf);
 
 static const struct ti_csi2rx_fmt *find_format_by_fourcc(u32 pixelformat)
@@ -302,7 +312,7 @@ static int ti_csi2rx_enum_fmt_vid_cap(struct file *file, void *priv,
 static int ti_csi2rx_g_fmt_vid_cap(struct file *file, void *prov,
 				   struct v4l2_format *f)
 {
-	struct ti_csi2rx_dev *csi = video_drvdata(file);
+	struct ti_csi2rx_ctx *csi = video_drvdata(file);
 
 	*f = csi->v_fmt;
 
@@ -333,7 +343,7 @@ static int ti_csi2rx_try_fmt_vid_cap(struct file *file, void *priv,
 static int ti_csi2rx_s_fmt_vid_cap(struct file *file, void *priv,
 				   struct v4l2_format *f)
 {
-	struct ti_csi2rx_dev *csi = video_drvdata(file);
+	struct ti_csi2rx_ctx *csi = video_drvdata(file);
 	struct vb2_queue *q = &csi->vidq;
 	int ret;
 
@@ -419,25 +429,33 @@ static int csi_async_notifier_bound(struct v4l2_async_notifier *notifier,
 static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
 {
 	struct ti_csi2rx_dev *csi = dev_get_drvdata(notifier->v4l2_dev->dev);
-	struct video_device *vdev = &csi->vdev;
-	int ret;
+	int ret, i;
 
-	ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
-	if (ret)
-		return ret;
+	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
+		struct ti_csi2rx_ctx *ctx = &csi->ctx[i];
+		struct video_device *vdev = &ctx->vdev;
 
-	ret = v4l2_create_fwnode_links_to_pad(csi->source, &csi->pad,
-					      MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED);
-
-	if (ret) {
-		video_unregister_device(vdev);
-		return ret;
+		ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
+		if (ret)
+			goto unregister_dev;
 	}
 
+	ret = v4l2_create_fwnode_links_to_pad(csi->source, &csi->pad,
+					      MEDIA_LNK_FL_IMMUTABLE |
+					      MEDIA_LNK_FL_ENABLED);
+	if (ret)
+		goto unregister_dev;
+
 	ret = v4l2_device_register_subdev_nodes(&csi->v4l2_dev);
 	if (ret)
-		video_unregister_device(vdev);
+		goto unregister_dev;
+
+	return 0;
 
+unregister_dev:
+	i--;
+	for (; i >= 0; i--)
+		video_unregister_device(&csi->ctx[i].vdev);
 	return ret;
 }
 
@@ -483,12 +501,13 @@ static int ti_csi2rx_notifier_register(struct ti_csi2rx_dev *csi)
 	return 0;
 }
 
-static void ti_csi2rx_setup_shim(struct ti_csi2rx_dev *csi)
+static void ti_csi2rx_setup_shim(struct ti_csi2rx_ctx *ctx)
 {
+	struct ti_csi2rx_dev *csi = ctx->csi;
 	const struct ti_csi2rx_fmt *fmt;
 	unsigned int reg;
 
-	fmt = find_format_by_fourcc(csi->v_fmt.fmt.pix.pixelformat);
+	fmt = find_format_by_fourcc(ctx->v_fmt.fmt.pix.pixelformat);
 
 	/* De-assert the pixel interface reset. */
 	reg = SHIM_CNTL_PIX_RST;
@@ -555,8 +574,9 @@ static void ti_csi2rx_drain_callback(void *param)
  * To prevent that stale data corrupting the subsequent transactions, it is
  * required to issue DMA requests to drain it out.
  */
-static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
+static int ti_csi2rx_drain_dma(struct ti_csi2rx_ctx *ctx)
 {
+	struct ti_csi2rx_dev *csi = ctx->csi;
 	struct dma_async_tx_descriptor *desc;
 	struct completion drain_complete;
 	dma_cookie_t cookie;
@@ -564,8 +584,8 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
 
 	init_completion(&drain_complete);
 
-	desc = dmaengine_prep_slave_single(csi->dma.chan, csi->dma.drain.paddr,
-					   csi->dma.drain.len, DMA_DEV_TO_MEM,
+	desc = dmaengine_prep_slave_single(ctx->dma.chan, csi->drain.paddr,
+					   csi->drain.len, DMA_DEV_TO_MEM,
 					   DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
 	if (!desc) {
 		ret = -EIO;
@@ -580,11 +600,11 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
 	if (ret)
 		goto out;
 
-	dma_async_issue_pending(csi->dma.chan);
+	dma_async_issue_pending(ctx->dma.chan);
 
 	if (!wait_for_completion_timeout(&drain_complete,
 					 msecs_to_jiffies(DRAIN_TIMEOUT_MS))) {
-		dmaengine_terminate_sync(csi->dma.chan);
+		dmaengine_terminate_sync(ctx->dma.chan);
 		dev_dbg(csi->dev, "DMA transfer timed out for drain buffer\n");
 		ret = -ETIMEDOUT;
 		goto out;
@@ -596,8 +616,8 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
 static void ti_csi2rx_dma_callback(void *param)
 {
 	struct ti_csi2rx_buffer *buf = param;
-	struct ti_csi2rx_dev *csi = buf->csi;
-	struct ti_csi2rx_dma *dma = &csi->dma;
+	struct ti_csi2rx_ctx *ctx = buf->ctx;
+	struct ti_csi2rx_dma *dma = &ctx->dma;
 	unsigned long flags;
 
 	/*
@@ -605,7 +625,7 @@ static void ti_csi2rx_dma_callback(void *param)
 	 * hardware monitor registers.
 	 */
 	buf->vb.vb2_buf.timestamp = ktime_get_ns();
-	buf->vb.sequence = csi->sequence++;
+	buf->vb.sequence = ctx->sequence++;
 
 	spin_lock_irqsave(&dma->lock, flags);
 
@@ -617,8 +637,9 @@ static void ti_csi2rx_dma_callback(void *param)
 	while (!list_empty(&dma->queue)) {
 		buf = list_entry(dma->queue.next, struct ti_csi2rx_buffer, list);
 
-		if (ti_csi2rx_start_dma(csi, buf)) {
-			dev_err(csi->dev, "Failed to queue the next buffer for DMA\n");
+		if (ti_csi2rx_start_dma(ctx, buf)) {
+			dev_err(ctx->csi->dev,
+				"Failed to queue the next buffer for DMA\n");
 			vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
 		} else {
 			list_move_tail(&buf->list, &dma->submitted);
@@ -631,17 +652,17 @@ static void ti_csi2rx_dma_callback(void *param)
 	spin_unlock_irqrestore(&dma->lock, flags);
 }
 
-static int ti_csi2rx_start_dma(struct ti_csi2rx_dev *csi,
+static int ti_csi2rx_start_dma(struct ti_csi2rx_ctx *ctx,
 			       struct ti_csi2rx_buffer *buf)
 {
 	unsigned long addr;
 	struct dma_async_tx_descriptor *desc;
-	size_t len = csi->v_fmt.fmt.pix.sizeimage;
+	size_t len = ctx->v_fmt.fmt.pix.sizeimage;
 	dma_cookie_t cookie;
 	int ret = 0;
 
 	addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
-	desc = dmaengine_prep_slave_single(csi->dma.chan, addr, len,
+	desc = dmaengine_prep_slave_single(ctx->dma.chan, addr, len,
 					   DMA_DEV_TO_MEM,
 					   DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
 	if (!desc)
@@ -655,20 +676,20 @@ static int ti_csi2rx_start_dma(struct ti_csi2rx_dev *csi,
 	if (ret)
 		return ret;
 
-	dma_async_issue_pending(csi->dma.chan);
+	dma_async_issue_pending(ctx->dma.chan);
 
 	return 0;
 }
 
-static void ti_csi2rx_stop_dma(struct ti_csi2rx_dev *csi)
+static void ti_csi2rx_stop_dma(struct ti_csi2rx_ctx *ctx)
 {
-	struct ti_csi2rx_dma *dma = &csi->dma;
+	struct ti_csi2rx_dma *dma = &ctx->dma;
 	enum ti_csi2rx_dma_state state;
 	unsigned long flags;
 	int ret;
 
 	spin_lock_irqsave(&dma->lock, flags);
-	state = csi->dma.state;
+	state = ctx->dma.state;
 	dma->state = TI_CSI2RX_DMA_STOPPED;
 	spin_unlock_irqrestore(&dma->lock, flags);
 
@@ -679,30 +700,30 @@ static void ti_csi2rx_stop_dma(struct ti_csi2rx_dev *csi)
 		 * is stopped, as the module-level pixel reset cannot be
 		 * enforced before terminating DMA.
 		 */
-		ret = ti_csi2rx_drain_dma(csi);
+		ret = ti_csi2rx_drain_dma(ctx);
 		if (ret && ret != -ETIMEDOUT)
-			dev_warn(csi->dev,
+			dev_warn(ctx->csi->dev,
 				 "Failed to drain DMA. Next frame might be bogus\n");
 	}
 
-	ret = dmaengine_terminate_sync(csi->dma.chan);
+	ret = dmaengine_terminate_sync(ctx->dma.chan);
 	if (ret)
-		dev_err(csi->dev, "Failed to stop DMA: %d\n", ret);
+		dev_err(ctx->csi->dev, "Failed to stop DMA: %d\n", ret);
 }
 
-static void ti_csi2rx_cleanup_buffers(struct ti_csi2rx_dev *csi,
+static void ti_csi2rx_cleanup_buffers(struct ti_csi2rx_ctx *ctx,
 				      enum vb2_buffer_state state)
 {
-	struct ti_csi2rx_dma *dma = &csi->dma;
+	struct ti_csi2rx_dma *dma = &ctx->dma;
 	struct ti_csi2rx_buffer *buf, *tmp;
 	unsigned long flags;
 
 	spin_lock_irqsave(&dma->lock, flags);
-	list_for_each_entry_safe(buf, tmp, &csi->dma.queue, list) {
+	list_for_each_entry_safe(buf, tmp, &ctx->dma.queue, list) {
 		list_del(&buf->list);
 		vb2_buffer_done(&buf->vb.vb2_buf, state);
 	}
-	list_for_each_entry_safe(buf, tmp, &csi->dma.submitted, list) {
+	list_for_each_entry_safe(buf, tmp, &ctx->dma.submitted, list) {
 		list_del(&buf->list);
 		vb2_buffer_done(&buf->vb.vb2_buf, state);
 	}
@@ -713,8 +734,8 @@ static int ti_csi2rx_queue_setup(struct vb2_queue *q, unsigned int *nbuffers,
 				 unsigned int *nplanes, unsigned int sizes[],
 				 struct device *alloc_devs[])
 {
-	struct ti_csi2rx_dev *csi = vb2_get_drv_priv(q);
-	unsigned int size = csi->v_fmt.fmt.pix.sizeimage;
+	struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(q);
+	unsigned int size = ctx->v_fmt.fmt.pix.sizeimage;
 
 	if (*nplanes) {
 		if (sizes[0] < size)
@@ -730,11 +751,11 @@ static int ti_csi2rx_queue_setup(struct vb2_queue *q, unsigned int *nbuffers,
 
 static int ti_csi2rx_buffer_prepare(struct vb2_buffer *vb)
 {
-	struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vb->vb2_queue);
-	unsigned long size = csi->v_fmt.fmt.pix.sizeimage;
+	struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+	unsigned long size = ctx->v_fmt.fmt.pix.sizeimage;
 
 	if (vb2_plane_size(vb, 0) < size) {
-		dev_err(csi->dev, "Data will not fit into plane\n");
+		dev_err(ctx->csi->dev, "Data will not fit into plane\n");
 		return -EINVAL;
 	}
 
@@ -744,15 +765,15 @@ static int ti_csi2rx_buffer_prepare(struct vb2_buffer *vb)
 
 static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
 {
-	struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vb->vb2_queue);
+	struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
 	struct ti_csi2rx_buffer *buf;
-	struct ti_csi2rx_dma *dma = &csi->dma;
+	struct ti_csi2rx_dma *dma = &ctx->dma;
 	bool restart_dma = false;
 	unsigned long flags = 0;
 	int ret;
 
 	buf = container_of(vb, struct ti_csi2rx_buffer, vb.vb2_buf);
-	buf->csi = csi;
+	buf->ctx = ctx;
 
 	spin_lock_irqsave(&dma->lock, flags);
 	/*
@@ -781,18 +802,18 @@ static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
 		 * the application and will only confuse it. Issue a DMA
 		 * transaction to drain that up.
 		 */
-		ret = ti_csi2rx_drain_dma(csi);
+		ret = ti_csi2rx_drain_dma(ctx);
 		if (ret && ret != -ETIMEDOUT)
-			dev_warn(csi->dev,
+			dev_warn(ctx->csi->dev,
 				 "Failed to drain DMA. Next frame might be bogus\n");
 
 		spin_lock_irqsave(&dma->lock, flags);
-		ret = ti_csi2rx_start_dma(csi, buf);
+		ret = ti_csi2rx_start_dma(ctx, buf);
 		if (ret) {
 			vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
 			dma->state = TI_CSI2RX_DMA_IDLE;
 			spin_unlock_irqrestore(&dma->lock, flags);
-			dev_err(csi->dev, "Failed to start DMA: %d\n", ret);
+			dev_err(ctx->csi->dev, "Failed to start DMA: %d\n", ret);
 		} else {
 			list_add_tail(&buf->list, &dma->submitted);
 			spin_unlock_irqrestore(&dma->lock, flags);
@@ -802,8 +823,9 @@ static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
 
 static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 {
-	struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vq);
-	struct ti_csi2rx_dma *dma = &csi->dma;
+	struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vq);
+	struct ti_csi2rx_dev *csi = ctx->csi;
+	struct ti_csi2rx_dma *dma = &ctx->dma;
 	struct ti_csi2rx_buffer *buf;
 	unsigned long flags;
 	int ret = 0;
@@ -815,18 +837,18 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 	if (ret)
 		return ret;
 
-	ret = video_device_pipeline_start(&csi->vdev, &csi->pipe);
+	ret = video_device_pipeline_start(&ctx->vdev, &csi->pipe);
 	if (ret)
 		goto err;
 
-	ti_csi2rx_setup_shim(csi);
+	ti_csi2rx_setup_shim(ctx);
 
-	csi->sequence = 0;
+	ctx->sequence = 0;
 
 	spin_lock_irqsave(&dma->lock, flags);
 	buf = list_entry(dma->queue.next, struct ti_csi2rx_buffer, list);
 
-	ret = ti_csi2rx_start_dma(csi, buf);
+	ret = ti_csi2rx_start_dma(ctx, buf);
 	if (ret) {
 		dev_err(csi->dev, "Failed to start DMA: %d\n", ret);
 		spin_unlock_irqrestore(&dma->lock, flags);
@@ -844,22 +866,23 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 	return 0;
 
 err_dma:
-	ti_csi2rx_stop_dma(csi);
+	ti_csi2rx_stop_dma(ctx);
 err_pipeline:
-	video_device_pipeline_stop(&csi->vdev);
+	video_device_pipeline_stop(&ctx->vdev);
 	writel(0, csi->shim + SHIM_CNTL);
 	writel(0, csi->shim + SHIM_DMACNTX);
 err:
-	ti_csi2rx_cleanup_buffers(csi, VB2_BUF_STATE_QUEUED);
+	ti_csi2rx_cleanup_buffers(ctx, VB2_BUF_STATE_QUEUED);
 	return ret;
 }
 
 static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
 {
-	struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vq);
+	struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vq);
+	struct ti_csi2rx_dev *csi = ctx->csi;
 	int ret;
 
-	video_device_pipeline_stop(&csi->vdev);
+	video_device_pipeline_stop(&ctx->vdev);
 
 	writel(0, csi->shim + SHIM_CNTL);
 	writel(0, csi->shim + SHIM_DMACNTX);
@@ -868,8 +891,8 @@ static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
 	if (ret)
 		dev_err(csi->dev, "Failed to stop subdev stream\n");
 
-	ti_csi2rx_stop_dma(csi);
-	ti_csi2rx_cleanup_buffers(csi, VB2_BUF_STATE_ERROR);
+	ti_csi2rx_stop_dma(ctx);
+	ti_csi2rx_cleanup_buffers(ctx, VB2_BUF_STATE_ERROR);
 }
 
 static const struct vb2_ops csi_vb2_qops = {

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct vb2_ops csi_vb2_qops = { · line 908

Quoted 5 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +1 -1
@@ -882,27 +905,60 @@ static const struct vb2_ops csi_vb2_qops = {
 	.wait_finish = vb2_ops_wait_finish,
 };
 
-static int ti_csi2rx_init_vb2q(struct ti_csi2rx_dev *csi)
+static void ti_csi2rx_cleanup_dma(struct ti_csi2rx_ctx *ctx)

Is it worth a wrapper function ?

Quoted 9 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +7 -1
 {
-	struct vb2_queue *q = &csi->vidq;
+	dma_release_channel(ctx->dma.chan);
+}
+
+static void ti_csi2rx_cleanup_v4l2(struct ti_csi2rx_dev *csi)
+{
+	media_device_unregister(&csi->mdev);
+	v4l2_device_unregister(&csi->v4l2_dev);

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct vb2_ops csi_vb2_qops = { · line 926

Quoted 10 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +10 -0
+	media_device_cleanup(&csi->mdev);
+}
+
+static void ti_csi2rx_cleanup_notifier(struct ti_csi2rx_dev *csi)
+{
+	v4l2_async_nf_unregister(&csi->notifier);
+	v4l2_async_nf_cleanup(&csi->notifier);
+}
+
+static void ti_csi2rx_cleanup_vb2q(struct ti_csi2rx_ctx *ctx)

Has a single caller, can be inlined maybe ?

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct vb2_ops csi_vb2_qops = { · line 934

Quoted 8 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +8 -0
+{
+	vb2_queue_release(&ctx->vidq);
+}
+
+static void ti_csi2rx_cleanup_ctx(struct ti_csi2rx_ctx *ctx)
+{
+	ti_csi2rx_cleanup_dma(ctx);
+	ti_csi2rx_cleanup_vb2q(ctx);

So this would become

        dma_release_channel(ctx->dma.chan);
        vb2_queue_release(&ctx->vidq);
Quoted 62 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +18 -10
+
+	video_unregister_device(&ctx->vdev);
+
+	mutex_destroy(&ctx->mutex);
+}
+
+static int ti_csi2rx_init_vb2q(struct ti_csi2rx_ctx *ctx)
+{
+	struct vb2_queue *q = &ctx->vidq;
 	int ret;
 
 	q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
 	q->io_modes = VB2_MMAP | VB2_DMABUF;
-	q->drv_priv = csi;
+	q->drv_priv = ctx;
 	q->buf_struct_size = sizeof(struct ti_csi2rx_buffer);
 	q->ops = &csi_vb2_qops;
 	q->mem_ops = &vb2_dma_contig_memops;
 	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
-	q->dev = dmaengine_get_dma_device(csi->dma.chan);
-	q->lock = &csi->mutex;
+	q->dev = dmaengine_get_dma_device(ctx->dma.chan);
+	q->lock = &ctx->mutex;
 	q->min_queued_buffers = 1;
 
 	ret = vb2_queue_init(q);
 	if (ret)
 		return ret;
 
-	csi->vdev.queue = q;
+	ctx->vdev.queue = q;
 
 	return 0;
 }
@@ -911,8 +967,9 @@ static int ti_csi2rx_link_validate(struct media_link *link)
 {
 	struct media_entity *entity = link->sink->entity;
 	struct video_device *vdev = media_entity_to_video_device(entity);
-	struct ti_csi2rx_dev *csi = container_of(vdev, struct ti_csi2rx_dev, vdev);
-	struct v4l2_pix_format *csi_fmt = &csi->v_fmt.fmt.pix;
+	struct ti_csi2rx_ctx *ctx = container_of(vdev, struct ti_csi2rx_ctx, vdev);
+	struct ti_csi2rx_dev *csi = ctx->csi;
+	struct v4l2_pix_format *csi_fmt = &ctx->v_fmt.fmt.pix;
 	struct v4l2_subdev_format source_fmt = {
 		.which	= V4L2_SUBDEV_FORMAT_ACTIVE,
 		.pad	= link->source->index,
@@ -965,47 +1022,69 @@ static const struct media_entity_operations ti_csi2rx_video_entity_ops = {
 	.link_validate = ti_csi2rx_link_validate,
 };
 
-static int ti_csi2rx_init_dma(struct ti_csi2rx_dev *csi)
+static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
 {
 	struct dma_slave_config cfg = {
 		.src_addr_width = DMA_SLAVE_BUSWIDTH_16_BYTES,
 	};
 	int ret;
 
-	INIT_LIST_HEAD(&csi->dma.queue);
-	INIT_LIST_HEAD(&csi->dma.submitted);
-	spin_lock_init(&csi->dma.lock);
+	INIT_LIST_HEAD(&ctx->dma.queue);

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct media_entity_operations ti_csi2rx_video_entity_ops = { · line 1038

Quoted 10 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +4 -4
+	INIT_LIST_HEAD(&ctx->dma.submitted);
+	spin_lock_init(&ctx->dma.lock);
 
-	csi->dma.state = TI_CSI2RX_DMA_STOPPED;
+	ctx->dma.state = TI_CSI2RX_DMA_STOPPED;
 
-	csi->dma.chan = dma_request_chan(csi->dev, "rx0");
-	if (IS_ERR(csi->dma.chan))
-		return PTR_ERR(csi->dma.chan);
+	ctx->dma.chan = dma_request_chan(ctx->csi->dev, "rx0");

The channel name should probably be parametrized, I presume it will happen next

Quoted 226 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +75 -80
+	if (IS_ERR(ctx->dma.chan))
+		return PTR_ERR(ctx->dma.chan);
 
-	ret = dmaengine_slave_config(csi->dma.chan, &cfg);
+	ret = dmaengine_slave_config(ctx->dma.chan, &cfg);
 	if (ret) {
-		dma_release_channel(csi->dma.chan);
+		dma_release_channel(ctx->dma.chan);
 		return ret;
 	}
 
-	csi->dma.drain.len = DRAIN_BUFFER_SIZE;
-	csi->dma.drain.vaddr = dma_alloc_coherent(csi->dev, csi->dma.drain.len,
-						  &csi->dma.drain.paddr,
-						  GFP_KERNEL);
-	if (!csi->dma.drain.vaddr)
-		return -ENOMEM;
-
 	return 0;
 }
 
 static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
 {
 	struct media_device *mdev = &csi->mdev;
-	struct video_device *vdev = &csi->vdev;
+	int ret;
+
+	mdev->dev = csi->dev;
+	mdev->hw_revision = 1;
+	strscpy(mdev->model, "TI-CSI2RX", sizeof(mdev->model));
+
+	media_device_init(mdev);
+
+	csi->v4l2_dev.mdev = mdev;
+
+	ret = v4l2_device_register(csi->dev, &csi->v4l2_dev);
+	if (ret)
+		return ret;
+
+	ret = media_device_register(mdev);
+	if (ret) {
+		v4l2_device_unregister(&csi->v4l2_dev);
+		media_device_cleanup(mdev);
+		return ret;
+	}
+
+	return 0;
+}
+
+static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
+{
+	struct ti_csi2rx_dev *csi = ctx->csi;
+	struct video_device *vdev = &ctx->vdev;
 	const struct ti_csi2rx_fmt *fmt;
-	struct v4l2_pix_format *pix_fmt = &csi->v_fmt.fmt.pix;
+	struct v4l2_pix_format *pix_fmt = &ctx->v_fmt.fmt.pix;
 	int ret;
 
+	mutex_init(&ctx->mutex);
+
 	fmt = find_format_by_fourcc(V4L2_PIX_FMT_UYVY);
 	if (!fmt)
 		return -EINVAL;
@@ -1018,15 +1097,16 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
 	pix_fmt->quantization = V4L2_QUANTIZATION_LIM_RANGE,
 	pix_fmt->xfer_func = V4L2_XFER_FUNC_SRGB,
 
-	ti_csi2rx_fill_fmt(fmt, &csi->v_fmt);
-
-	mdev->dev = csi->dev;
-	mdev->hw_revision = 1;
-	strscpy(mdev->model, "TI-CSI2RX", sizeof(mdev->model));
+	ti_csi2rx_fill_fmt(fmt, &ctx->v_fmt);
 
-	media_device_init(mdev);
+	csi->pad.flags = MEDIA_PAD_FL_SINK;
+	vdev->entity.ops = &ti_csi2rx_video_entity_ops;
+	ret = media_entity_pads_init(&ctx->vdev.entity, 1, &csi->pad);
+	if (ret)
+		return ret;
 
-	strscpy(vdev->name, TI_CSI2RX_MODULE_NAME, sizeof(vdev->name));
+	snprintf(vdev->name, sizeof(vdev->name), "%s context %u",
+		 dev_name(csi->dev), ctx->idx);
 	vdev->v4l2_dev = &csi->v4l2_dev;
 	vdev->vfl_dir = VFL_DIR_RX;
 	vdev->fops = &csi_fops;
@@ -1034,61 +1114,28 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
 	vdev->release = video_device_release_empty;
 	vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING |
 			    V4L2_CAP_IO_MC;
-	vdev->lock = &csi->mutex;
-	video_set_drvdata(vdev, csi);
+	vdev->lock = &ctx->mutex;
+	video_set_drvdata(vdev, ctx);
 
-	csi->pad.flags = MEDIA_PAD_FL_SINK;
-	vdev->entity.ops = &ti_csi2rx_video_entity_ops;
-	ret = media_entity_pads_init(&csi->vdev.entity, 1, &csi->pad);
+	ret = ti_csi2rx_init_dma(ctx);
 	if (ret)
 		return ret;
 
-	csi->v4l2_dev.mdev = mdev;
-
-	ret = v4l2_device_register(csi->dev, &csi->v4l2_dev);
+	ret = ti_csi2rx_init_vb2q(ctx);
 	if (ret)
-		return ret;
-
-	ret = media_device_register(mdev);
-	if (ret) {
-		v4l2_device_unregister(&csi->v4l2_dev);
-		media_device_cleanup(mdev);
-		return ret;
-	}
+		goto cleanup_dma;
 
 	return 0;
-}
 
-static void ti_csi2rx_cleanup_dma(struct ti_csi2rx_dev *csi)
-{
-	dma_free_coherent(csi->dev, csi->dma.drain.len,
-			  csi->dma.drain.vaddr, csi->dma.drain.paddr);
-	csi->dma.drain.vaddr = NULL;
-	dma_release_channel(csi->dma.chan);
-}
-
-static void ti_csi2rx_cleanup_v4l2(struct ti_csi2rx_dev *csi)
-{
-	media_device_unregister(&csi->mdev);
-	v4l2_device_unregister(&csi->v4l2_dev);
-	media_device_cleanup(&csi->mdev);
-}
-
-static void ti_csi2rx_cleanup_subdev(struct ti_csi2rx_dev *csi)
-{
-	v4l2_async_nf_unregister(&csi->notifier);
-	v4l2_async_nf_cleanup(&csi->notifier);
-}
-
-static void ti_csi2rx_cleanup_vb2q(struct ti_csi2rx_dev *csi)
-{
-	vb2_queue_release(&csi->vidq);
+cleanup_dma:
+	ti_csi2rx_cleanup_dma(ctx);
+	return ret;
 }
 
 static int ti_csi2rx_probe(struct platform_device *pdev)
 {
 	struct ti_csi2rx_dev *csi;
-	int ret;
+	int ret, i;
 
 	csi = devm_kzalloc(&pdev->dev, sizeof(*csi), GFP_KERNEL);
 	if (!csi)
@@ -1097,62 +1144,76 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
 	csi->dev = &pdev->dev;
 	platform_set_drvdata(pdev, csi);
 
-	mutex_init(&csi->mutex);
 	csi->shim = devm_platform_ioremap_resource(pdev, 0);
 	if (IS_ERR(csi->shim)) {
 		ret = PTR_ERR(csi->shim);
-		goto err_mutex;
+		return ret;
 	}
 
-	ret = ti_csi2rx_init_dma(csi);
-	if (ret)
-		goto err_mutex;
+	csi->drain.len = DRAIN_BUFFER_SIZE;
+	csi->drain.vaddr = dma_alloc_coherent(csi->dev, csi->drain.len,
+					      &csi->drain.paddr,
+					      GFP_KERNEL);
+	if (!csi->drain.vaddr)
+		return -ENOMEM;
 
 	ret = ti_csi2rx_v4l2_init(csi);
-	if (ret)
-		goto err_dma;
-
-	ret = ti_csi2rx_init_vb2q(csi);
 	if (ret)
 		goto err_v4l2;
 
+	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
+		csi->ctx[i].idx = i;
+		csi->ctx[i].csi = csi;
+		ret = ti_csi2rx_init_ctx(&csi->ctx[i]);
+		if (ret)
+			goto err_ctx;
+	}
+
 	ret = ti_csi2rx_notifier_register(csi);
 	if (ret)
-		goto err_vb2q;
+		goto err_ctx;
 
 	ret = of_platform_populate(csi->dev->of_node, NULL, NULL, csi->dev);
 	if (ret) {
 		dev_err(csi->dev, "Failed to create children: %d\n", ret);
-		goto err_subdev;
+		goto err_notifier;
 	}
 
 	return 0;
 
-err_subdev:
-	ti_csi2rx_cleanup_subdev(csi);
-err_vb2q:
-	ti_csi2rx_cleanup_vb2q(csi);
-err_v4l2:
+err_notifier:
+	ti_csi2rx_cleanup_notifier(csi);
+err_ctx:
+	i--;
+	for (; i >= 0; i--)
+		ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
 	ti_csi2rx_cleanup_v4l2(csi);
-err_dma:
-	ti_csi2rx_cleanup_dma(csi);
-err_mutex:
-	mutex_destroy(&csi->mutex);

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int ti_csi2rx_probe(struct platform_device *pdev) · line 1200

Quoted 10 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +4 -0
+err_v4l2:
+	dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
+			  csi->drain.paddr);
 	return ret;
 }
 
 static void ti_csi2rx_remove(struct platform_device *pdev)
 {
 	struct ti_csi2rx_dev *csi = platform_get_drvdata(pdev);
+	int i;

unsigned

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int ti_csi2rx_probe(struct platform_device *pdev) · line 1203

Quoted 3 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +3 -0
+
+	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
+		if (vb2_is_busy(&csi->ctx[i].vidq))

Can this happen ?

Quoted 26 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +9 -6
+			dev_err(csi->dev,
+				"Failed to remove as queue busy for ctx %u\n",
+				i);
+	}
 
-	video_unregister_device(&csi->vdev);
+	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++)
+		ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
 
-	ti_csi2rx_cleanup_vb2q(csi);
-	ti_csi2rx_cleanup_subdev(csi);
+	ti_csi2rx_cleanup_notifier(csi);
 	ti_csi2rx_cleanup_v4l2(csi);
-	ti_csi2rx_cleanup_dma(csi);
 
-	mutex_destroy(&csi->mutex);
+	dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
+			  csi->drain.paddr);
 }
 
 static const struct of_device_id ti_csi2rx_of_match[] = {
 
--

2.43.0

author
#14 lore raw

Re: Re: [PATCH v2 03/13] media: ti: j721e-csi2rx: separate out device and context

Hi Jacopo,

Thanks for the review.

Quoted 519 lines · Jacopo Mondi

Hi Jai

Quoted 475 lines · Jai Luthra

The TI CSI2RX wrapper has two parts: the main device and the DMA contexts. The driver was originally written with single camera capture in mind, so only one DMA context was needed. For the sake of simplicity, the context specific stuff was not modeled different to the main device.

To enable multiplexed stream capture, the contexts need to be separated out from the main device. Create a struct ti_csi2rx_ctx that holds the DMA context specific things. Separate out functions handling the device and context related functionality.

---
 .../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c  | 423 ++++++++++++---------
 1 file changed, 242 insertions(+), 181 deletions(-)
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +103 -80
index 22442fce7607..d8dfe0002b72 100644
@@ -40,6 +40,8 @@
 #define SHIM_PSI_CFG0_DST_TAG		GENMASK(31, 16)
 
 #define PSIL_WORD_SIZE_BYTES		16
+#define TI_CSI2RX_NUM_CTX		1
+
 /*
  * There are no hard limits on the width or height. The DMA engine can handle
  * all sizes. The max width and height are arbitrary numbers for this driver.
@@ -64,7 +66,7 @@ struct ti_csi2rx_buffer {
 	/* Common v4l2 buffer. Must be first. */
 	struct vb2_v4l2_buffer		vb;
 	struct list_head		list;
-	struct ti_csi2rx_dev		*csi;
+	struct ti_csi2rx_ctx		*ctx;
 };
 
 enum ti_csi2rx_dma_state {
@@ -84,29 +86,37 @@ struct ti_csi2rx_dma {
 	 * Queue of buffers submitted to DMA engine.
 	 */
 	struct list_head		submitted;
-	/* Buffer to drain stale data from PSI-L endpoint */
-	struct {
-		void			*vaddr;
-		dma_addr_t		paddr;
-		size_t			len;
-	} drain;
+};
+
+struct ti_csi2rx_dev;
+
+struct ti_csi2rx_ctx {
+	struct ti_csi2rx_dev		*csi;
+	struct video_device		vdev;
+	struct vb2_queue		vidq;
+	struct mutex			mutex; /* To serialize ioctls. */
+	struct v4l2_format		v_fmt;
+	struct ti_csi2rx_dma		dma;
+	u32				sequence;
+	u32				idx;
 };
 
 struct ti_csi2rx_dev {
 	struct device			*dev;
 	void __iomem			*shim;
 	struct v4l2_device		v4l2_dev;
-	struct video_device		vdev;
 	struct media_device		mdev;
 	struct media_pipeline		pipe;
 	struct media_pad		pad;
 	struct v4l2_async_notifier	notifier;
 	struct v4l2_subdev		*source;
-	struct vb2_queue		vidq;
-	struct mutex			mutex; /* To serialize ioctls. */
-	struct v4l2_format		v_fmt;
-	struct ti_csi2rx_dma		dma;
-	u32				sequence;
+	struct ti_csi2rx_ctx		ctx[TI_CSI2RX_NUM_CTX];
+	/* Buffer to drain stale data from PSI-L endpoint */
+	struct {
+		void			*vaddr;
+		dma_addr_t		paddr;
+		size_t			len;
+	} drain;
 };
 
 static const struct ti_csi2rx_fmt ti_csi2rx_formats[] = {
@@ -212,7 +222,7 @@ static const struct ti_csi2rx_fmt ti_csi2rx_formats[] = {
 };
 
 /* Forward declaration needed by ti_csi2rx_dma_callback. */
-static int ti_csi2rx_start_dma(struct ti_csi2rx_dev *csi,
+static int ti_csi2rx_start_dma(struct ti_csi2rx_ctx *ctx,
 			       struct ti_csi2rx_buffer *buf);
 
 static const struct ti_csi2rx_fmt *find_format_by_fourcc(u32 pixelformat)
@@ -302,7 +312,7 @@ static int ti_csi2rx_enum_fmt_vid_cap(struct file *file, void *priv,
 static int ti_csi2rx_g_fmt_vid_cap(struct file *file, void *prov,
 				   struct v4l2_format *f)
 {
-	struct ti_csi2rx_dev *csi = video_drvdata(file);
+	struct ti_csi2rx_ctx *csi = video_drvdata(file);
 
 	*f = csi->v_fmt;
 
@@ -333,7 +343,7 @@ static int ti_csi2rx_try_fmt_vid_cap(struct file *file, void *priv,
 static int ti_csi2rx_s_fmt_vid_cap(struct file *file, void *priv,
 				   struct v4l2_format *f)
 {
-	struct ti_csi2rx_dev *csi = video_drvdata(file);
+	struct ti_csi2rx_ctx *csi = video_drvdata(file);
 	struct vb2_queue *q = &csi->vidq;
 	int ret;
 
@@ -419,25 +429,33 @@ static int csi_async_notifier_bound(struct v4l2_async_notifier *notifier,
 static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
 {
 	struct ti_csi2rx_dev *csi = dev_get_drvdata(notifier->v4l2_dev->dev);
-	struct video_device *vdev = &csi->vdev;
-	int ret;
+	int ret, i;
 
-	ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
-	if (ret)
-		return ret;
+	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
+		struct ti_csi2rx_ctx *ctx = &csi->ctx[i];
+		struct video_device *vdev = &ctx->vdev;
 
-	ret = v4l2_create_fwnode_links_to_pad(csi->source, &csi->pad,
-					      MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED);
-
-	if (ret) {
-		video_unregister_device(vdev);
-		return ret;
+		ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
+		if (ret)
+			goto unregister_dev;
 	}
 
+	ret = v4l2_create_fwnode_links_to_pad(csi->source, &csi->pad,
+					      MEDIA_LNK_FL_IMMUTABLE |
+					      MEDIA_LNK_FL_ENABLED);
+	if (ret)
+		goto unregister_dev;
+
 	ret = v4l2_device_register_subdev_nodes(&csi->v4l2_dev);
 	if (ret)
-		video_unregister_device(vdev);
+		goto unregister_dev;
+
+	return 0;
 
+unregister_dev:
+	i--;
+	for (; i >= 0; i--)
+		video_unregister_device(&csi->ctx[i].vdev);
 	return ret;
 }
 
@@ -483,12 +501,13 @@ static int ti_csi2rx_notifier_register(struct ti_csi2rx_dev *csi)
 	return 0;
 }
 
-static void ti_csi2rx_setup_shim(struct ti_csi2rx_dev *csi)
+static void ti_csi2rx_setup_shim(struct ti_csi2rx_ctx *ctx)
 {
+	struct ti_csi2rx_dev *csi = ctx->csi;
 	const struct ti_csi2rx_fmt *fmt;
 	unsigned int reg;
 
-	fmt = find_format_by_fourcc(csi->v_fmt.fmt.pix.pixelformat);
+	fmt = find_format_by_fourcc(ctx->v_fmt.fmt.pix.pixelformat);
 
 	/* De-assert the pixel interface reset. */
 	reg = SHIM_CNTL_PIX_RST;
@@ -555,8 +574,9 @@ static void ti_csi2rx_drain_callback(void *param)
  * To prevent that stale data corrupting the subsequent transactions, it is
  * required to issue DMA requests to drain it out.
  */
-static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
+static int ti_csi2rx_drain_dma(struct ti_csi2rx_ctx *ctx)
 {
+	struct ti_csi2rx_dev *csi = ctx->csi;
 	struct dma_async_tx_descriptor *desc;
 	struct completion drain_complete;
 	dma_cookie_t cookie;
@@ -564,8 +584,8 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
 
 	init_completion(&drain_complete);
 
-	desc = dmaengine_prep_slave_single(csi->dma.chan, csi->dma.drain.paddr,
-					   csi->dma.drain.len, DMA_DEV_TO_MEM,
+	desc = dmaengine_prep_slave_single(ctx->dma.chan, csi->drain.paddr,
+					   csi->drain.len, DMA_DEV_TO_MEM,
 					   DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
 	if (!desc) {
 		ret = -EIO;
@@ -580,11 +600,11 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
 	if (ret)
 		goto out;
 
-	dma_async_issue_pending(csi->dma.chan);
+	dma_async_issue_pending(ctx->dma.chan);
 
 	if (!wait_for_completion_timeout(&drain_complete,
 					 msecs_to_jiffies(DRAIN_TIMEOUT_MS))) {
-		dmaengine_terminate_sync(csi->dma.chan);
+		dmaengine_terminate_sync(ctx->dma.chan);
 		dev_dbg(csi->dev, "DMA transfer timed out for drain buffer\n");
 		ret = -ETIMEDOUT;
 		goto out;
@@ -596,8 +616,8 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
 static void ti_csi2rx_dma_callback(void *param)
 {
 	struct ti_csi2rx_buffer *buf = param;
-	struct ti_csi2rx_dev *csi = buf->csi;
-	struct ti_csi2rx_dma *dma = &csi->dma;
+	struct ti_csi2rx_ctx *ctx = buf->ctx;
+	struct ti_csi2rx_dma *dma = &ctx->dma;
 	unsigned long flags;
 
 	/*
@@ -605,7 +625,7 @@ static void ti_csi2rx_dma_callback(void *param)
 	 * hardware monitor registers.
 	 */
 	buf->vb.vb2_buf.timestamp = ktime_get_ns();
-	buf->vb.sequence = csi->sequence++;
+	buf->vb.sequence = ctx->sequence++;
 
 	spin_lock_irqsave(&dma->lock, flags);
 
@@ -617,8 +637,9 @@ static void ti_csi2rx_dma_callback(void *param)
 	while (!list_empty(&dma->queue)) {
 		buf = list_entry(dma->queue.next, struct ti_csi2rx_buffer, list);
 
-		if (ti_csi2rx_start_dma(csi, buf)) {
-			dev_err(csi->dev, "Failed to queue the next buffer for DMA\n");
+		if (ti_csi2rx_start_dma(ctx, buf)) {
+			dev_err(ctx->csi->dev,
+				"Failed to queue the next buffer for DMA\n");
 			vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
 		} else {
 			list_move_tail(&buf->list, &dma->submitted);
@@ -631,17 +652,17 @@ static void ti_csi2rx_dma_callback(void *param)
 	spin_unlock_irqrestore(&dma->lock, flags);
 }
 
-static int ti_csi2rx_start_dma(struct ti_csi2rx_dev *csi,
+static int ti_csi2rx_start_dma(struct ti_csi2rx_ctx *ctx,
 			       struct ti_csi2rx_buffer *buf)
 {
 	unsigned long addr;
 	struct dma_async_tx_descriptor *desc;
-	size_t len = csi->v_fmt.fmt.pix.sizeimage;
+	size_t len = ctx->v_fmt.fmt.pix.sizeimage;
 	dma_cookie_t cookie;
 	int ret = 0;
 
 	addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
-	desc = dmaengine_prep_slave_single(csi->dma.chan, addr, len,
+	desc = dmaengine_prep_slave_single(ctx->dma.chan, addr, len,
 					   DMA_DEV_TO_MEM,
 					   DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
 	if (!desc)
@@ -655,20 +676,20 @@ static int ti_csi2rx_start_dma(struct ti_csi2rx_dev *csi,
 	if (ret)
 		return ret;
 
-	dma_async_issue_pending(csi->dma.chan);
+	dma_async_issue_pending(ctx->dma.chan);
 
 	return 0;
 }
 
-static void ti_csi2rx_stop_dma(struct ti_csi2rx_dev *csi)
+static void ti_csi2rx_stop_dma(struct ti_csi2rx_ctx *ctx)
 {
-	struct ti_csi2rx_dma *dma = &csi->dma;
+	struct ti_csi2rx_dma *dma = &ctx->dma;
 	enum ti_csi2rx_dma_state state;
 	unsigned long flags;
 	int ret;
 
 	spin_lock_irqsave(&dma->lock, flags);
-	state = csi->dma.state;
+	state = ctx->dma.state;
 	dma->state = TI_CSI2RX_DMA_STOPPED;
 	spin_unlock_irqrestore(&dma->lock, flags);
 
@@ -679,30 +700,30 @@ static void ti_csi2rx_stop_dma(struct ti_csi2rx_dev *csi)
 		 * is stopped, as the module-level pixel reset cannot be
 		 * enforced before terminating DMA.
 		 */
-		ret = ti_csi2rx_drain_dma(csi);
+		ret = ti_csi2rx_drain_dma(ctx);
 		if (ret && ret != -ETIMEDOUT)
-			dev_warn(csi->dev,
+			dev_warn(ctx->csi->dev,
 				 "Failed to drain DMA. Next frame might be bogus\n");
 	}
 
-	ret = dmaengine_terminate_sync(csi->dma.chan);
+	ret = dmaengine_terminate_sync(ctx->dma.chan);
 	if (ret)
-		dev_err(csi->dev, "Failed to stop DMA: %d\n", ret);
+		dev_err(ctx->csi->dev, "Failed to stop DMA: %d\n", ret);
 }
 
-static void ti_csi2rx_cleanup_buffers(struct ti_csi2rx_dev *csi,
+static void ti_csi2rx_cleanup_buffers(struct ti_csi2rx_ctx *ctx,
 				      enum vb2_buffer_state state)
 {
-	struct ti_csi2rx_dma *dma = &csi->dma;
+	struct ti_csi2rx_dma *dma = &ctx->dma;
 	struct ti_csi2rx_buffer *buf, *tmp;
 	unsigned long flags;
 
 	spin_lock_irqsave(&dma->lock, flags);
-	list_for_each_entry_safe(buf, tmp, &csi->dma.queue, list) {
+	list_for_each_entry_safe(buf, tmp, &ctx->dma.queue, list) {
 		list_del(&buf->list);
 		vb2_buffer_done(&buf->vb.vb2_buf, state);
 	}
-	list_for_each_entry_safe(buf, tmp, &csi->dma.submitted, list) {
+	list_for_each_entry_safe(buf, tmp, &ctx->dma.submitted, list) {
 		list_del(&buf->list);
 		vb2_buffer_done(&buf->vb.vb2_buf, state);
 	}
@@ -713,8 +734,8 @@ static int ti_csi2rx_queue_setup(struct vb2_queue *q, unsigned int *nbuffers,
 				 unsigned int *nplanes, unsigned int sizes[],
 				 struct device *alloc_devs[])
 {
-	struct ti_csi2rx_dev *csi = vb2_get_drv_priv(q);
-	unsigned int size = csi->v_fmt.fmt.pix.sizeimage;
+	struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(q);
+	unsigned int size = ctx->v_fmt.fmt.pix.sizeimage;
 
 	if (*nplanes) {
 		if (sizes[0] < size)
@@ -730,11 +751,11 @@ static int ti_csi2rx_queue_setup(struct vb2_queue *q, unsigned int *nbuffers,
 
 static int ti_csi2rx_buffer_prepare(struct vb2_buffer *vb)
 {
-	struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vb->vb2_queue);
-	unsigned long size = csi->v_fmt.fmt.pix.sizeimage;
+	struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+	unsigned long size = ctx->v_fmt.fmt.pix.sizeimage;
 
 	if (vb2_plane_size(vb, 0) < size) {
-		dev_err(csi->dev, "Data will not fit into plane\n");
+		dev_err(ctx->csi->dev, "Data will not fit into plane\n");
 		return -EINVAL;
 	}
 
@@ -744,15 +765,15 @@ static int ti_csi2rx_buffer_prepare(struct vb2_buffer *vb)
 
 static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
 {
-	struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vb->vb2_queue);
+	struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
 	struct ti_csi2rx_buffer *buf;
-	struct ti_csi2rx_dma *dma = &csi->dma;
+	struct ti_csi2rx_dma *dma = &ctx->dma;
 	bool restart_dma = false;
 	unsigned long flags = 0;
 	int ret;
 
 	buf = container_of(vb, struct ti_csi2rx_buffer, vb.vb2_buf);
-	buf->csi = csi;
+	buf->ctx = ctx;
 
 	spin_lock_irqsave(&dma->lock, flags);
 	/*
@@ -781,18 +802,18 @@ static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
 		 * the application and will only confuse it. Issue a DMA
 		 * transaction to drain that up.
 		 */
-		ret = ti_csi2rx_drain_dma(csi);
+		ret = ti_csi2rx_drain_dma(ctx);
 		if (ret && ret != -ETIMEDOUT)
-			dev_warn(csi->dev,
+			dev_warn(ctx->csi->dev,
 				 "Failed to drain DMA. Next frame might be bogus\n");
 
 		spin_lock_irqsave(&dma->lock, flags);
-		ret = ti_csi2rx_start_dma(csi, buf);
+		ret = ti_csi2rx_start_dma(ctx, buf);
 		if (ret) {
 			vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
 			dma->state = TI_CSI2RX_DMA_IDLE;
 			spin_unlock_irqrestore(&dma->lock, flags);
-			dev_err(csi->dev, "Failed to start DMA: %d\n", ret);
+			dev_err(ctx->csi->dev, "Failed to start DMA: %d\n", ret);
 		} else {
 			list_add_tail(&buf->list, &dma->submitted);
 			spin_unlock_irqrestore(&dma->lock, flags);
@@ -802,8 +823,9 @@ static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
 
 static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 {
-	struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vq);
-	struct ti_csi2rx_dma *dma = &csi->dma;
+	struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vq);
+	struct ti_csi2rx_dev *csi = ctx->csi;
+	struct ti_csi2rx_dma *dma = &ctx->dma;
 	struct ti_csi2rx_buffer *buf;
 	unsigned long flags;
 	int ret = 0;
@@ -815,18 +837,18 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 	if (ret)
 		return ret;
 
-	ret = video_device_pipeline_start(&csi->vdev, &csi->pipe);
+	ret = video_device_pipeline_start(&ctx->vdev, &csi->pipe);
 	if (ret)
 		goto err;
 
-	ti_csi2rx_setup_shim(csi);
+	ti_csi2rx_setup_shim(ctx);
 
-	csi->sequence = 0;
+	ctx->sequence = 0;
 
 	spin_lock_irqsave(&dma->lock, flags);
 	buf = list_entry(dma->queue.next, struct ti_csi2rx_buffer, list);
 
-	ret = ti_csi2rx_start_dma(csi, buf);
+	ret = ti_csi2rx_start_dma(ctx, buf);
 	if (ret) {
 		dev_err(csi->dev, "Failed to start DMA: %d\n", ret);
 		spin_unlock_irqrestore(&dma->lock, flags);
@@ -844,22 +866,23 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 	return 0;
 
 err_dma:
-	ti_csi2rx_stop_dma(csi);
+	ti_csi2rx_stop_dma(ctx);
 err_pipeline:
-	video_device_pipeline_stop(&csi->vdev);
+	video_device_pipeline_stop(&ctx->vdev);
 	writel(0, csi->shim + SHIM_CNTL);
 	writel(0, csi->shim + SHIM_DMACNTX);
 err:
-	ti_csi2rx_cleanup_buffers(csi, VB2_BUF_STATE_QUEUED);
+	ti_csi2rx_cleanup_buffers(ctx, VB2_BUF_STATE_QUEUED);
 	return ret;
 }
 
 static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
 {
-	struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vq);
+	struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vq);
+	struct ti_csi2rx_dev *csi = ctx->csi;
 	int ret;
 
-	video_device_pipeline_stop(&csi->vdev);
+	video_device_pipeline_stop(&ctx->vdev);
 
 	writel(0, csi->shim + SHIM_CNTL);
 	writel(0, csi->shim + SHIM_DMACNTX);
@@ -868,8 +891,8 @@ static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
 	if (ret)
 		dev_err(csi->dev, "Failed to stop subdev stream\n");
 
-	ti_csi2rx_stop_dma(csi);
-	ti_csi2rx_cleanup_buffers(csi, VB2_BUF_STATE_ERROR);
+	ti_csi2rx_stop_dma(ctx);
+	ti_csi2rx_cleanup_buffers(ctx, VB2_BUF_STATE_ERROR);
 }
 
 static const struct vb2_ops csi_vb2_qops = {

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct vb2_ops csi_vb2_qops = { · line 908

Quoted 5 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +1 -1
@@ -882,27 +905,60 @@ static const struct vb2_ops csi_vb2_qops = {
 	.wait_finish = vb2_ops_wait_finish,
 };
 
-static int ti_csi2rx_init_vb2q(struct ti_csi2rx_dev *csi)
+static void ti_csi2rx_cleanup_dma(struct ti_csi2rx_ctx *ctx)

Is it worth a wrapper function ?

Quoted 9 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +7 -1
 {
-	struct vb2_queue *q = &csi->vidq;
+	dma_release_channel(ctx->dma.chan);
+}
+
+static void ti_csi2rx_cleanup_v4l2(struct ti_csi2rx_dev *csi)
+{
+	media_device_unregister(&csi->mdev);
+	v4l2_device_unregister(&csi->v4l2_dev);

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct vb2_ops csi_vb2_qops = { · line 926

Quoted 10 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +10 -0
+	media_device_cleanup(&csi->mdev);
+}
+
+static void ti_csi2rx_cleanup_notifier(struct ti_csi2rx_dev *csi)
+{
+	v4l2_async_nf_unregister(&csi->notifier);
+	v4l2_async_nf_cleanup(&csi->notifier);
+}
+
+static void ti_csi2rx_cleanup_vb2q(struct ti_csi2rx_ctx *ctx)

Has a single caller, can be inlined maybe ?

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct vb2_ops csi_vb2_qops = { · line 934

Quoted 8 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +8 -0
+{
+	vb2_queue_release(&ctx->vidq);
+}
+
+static void ti_csi2rx_cleanup_ctx(struct ti_csi2rx_ctx *ctx)
+{
+	ti_csi2rx_cleanup_dma(ctx);
+	ti_csi2rx_cleanup_vb2q(ctx);

So this would become

Quoted 3 lines
        dma_release_channel(ctx->dma.chan);
        vb2_queue_release(&ctx->vidq);

Good catch, will update in v3.

Quoted 68 lines
Quoted 68 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +21 -11
+
+	video_unregister_device(&ctx->vdev);
+
+	mutex_destroy(&ctx->mutex);
+}
+
+static int ti_csi2rx_init_vb2q(struct ti_csi2rx_ctx *ctx)
+{
+	struct vb2_queue *q = &ctx->vidq;
 	int ret;
 
 	q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
 	q->io_modes = VB2_MMAP | VB2_DMABUF;
-	q->drv_priv = csi;
+	q->drv_priv = ctx;
 	q->buf_struct_size = sizeof(struct ti_csi2rx_buffer);
 	q->ops = &csi_vb2_qops;
 	q->mem_ops = &vb2_dma_contig_memops;
 	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
-	q->dev = dmaengine_get_dma_device(csi->dma.chan);
-	q->lock = &csi->mutex;
+	q->dev = dmaengine_get_dma_device(ctx->dma.chan);
+	q->lock = &ctx->mutex;
 	q->min_queued_buffers = 1;
 
 	ret = vb2_queue_init(q);
 	if (ret)
 		return ret;
 
-	csi->vdev.queue = q;
+	ctx->vdev.queue = q;
 
 	return 0;
 }
@@ -911,8 +967,9 @@ static int ti_csi2rx_link_validate(struct media_link *link)
 {
 	struct media_entity *entity = link->sink->entity;
 	struct video_device *vdev = media_entity_to_video_device(entity);
-	struct ti_csi2rx_dev *csi = container_of(vdev, struct ti_csi2rx_dev, vdev);
-	struct v4l2_pix_format *csi_fmt = &csi->v_fmt.fmt.pix;
+	struct ti_csi2rx_ctx *ctx = container_of(vdev, struct ti_csi2rx_ctx, vdev);
+	struct ti_csi2rx_dev *csi = ctx->csi;
+	struct v4l2_pix_format *csi_fmt = &ctx->v_fmt.fmt.pix;
 	struct v4l2_subdev_format source_fmt = {
 		.which	= V4L2_SUBDEV_FORMAT_ACTIVE,
 		.pad	= link->source->index,
@@ -965,47 +1022,69 @@ static const struct media_entity_operations ti_csi2rx_video_entity_ops = {
 	.link_validate = ti_csi2rx_link_validate,
 };
 
-static int ti_csi2rx_init_dma(struct ti_csi2rx_dev *csi)
+static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
 {
 	struct dma_slave_config cfg = {
 		.src_addr_width = DMA_SLAVE_BUSWIDTH_16_BYTES,
 	};
 	int ret;
 
-	INIT_LIST_HEAD(&csi->dma.queue);
-	INIT_LIST_HEAD(&csi->dma.submitted);
-	spin_lock_init(&csi->dma.lock);
+	INIT_LIST_HEAD(&ctx->dma.queue);
+	INIT_LIST_HEAD(&ctx->dma.submitted);
+	spin_lock_init(&ctx->dma.lock);
 
-	csi->dma.state = TI_CSI2RX_DMA_STOPPED;
+	ctx->dma.state = TI_CSI2RX_DMA_STOPPED;
 
Quoted 7 lines

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct media_entity_operations ti_csi2rx_video_entity_ops = { · line 1038

Quoted 4 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +1 -3
-	csi->dma.chan = dma_request_chan(csi->dev, "rx0");
-	if (IS_ERR(csi->dma.chan))
-		return PTR_ERR(csi->dma.chan);
+	ctx->dma.chan = dma_request_chan(ctx->csi->dev, "rx0");

The channel name should probably be parametrized, I presume it will happen next

Yes in subsequent patches.

Quoted 233 lines
Quoted 233 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +78 -80
+	if (IS_ERR(ctx->dma.chan))
+		return PTR_ERR(ctx->dma.chan);
 
-	ret = dmaengine_slave_config(csi->dma.chan, &cfg);
+	ret = dmaengine_slave_config(ctx->dma.chan, &cfg);
 	if (ret) {
-		dma_release_channel(csi->dma.chan);
+		dma_release_channel(ctx->dma.chan);
 		return ret;
 	}
 
-	csi->dma.drain.len = DRAIN_BUFFER_SIZE;
-	csi->dma.drain.vaddr = dma_alloc_coherent(csi->dev, csi->dma.drain.len,
-						  &csi->dma.drain.paddr,
-						  GFP_KERNEL);
-	if (!csi->dma.drain.vaddr)
-		return -ENOMEM;
-
 	return 0;
 }
 
 static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
 {
 	struct media_device *mdev = &csi->mdev;
-	struct video_device *vdev = &csi->vdev;
+	int ret;
+
+	mdev->dev = csi->dev;
+	mdev->hw_revision = 1;
+	strscpy(mdev->model, "TI-CSI2RX", sizeof(mdev->model));
+
+	media_device_init(mdev);
+
+	csi->v4l2_dev.mdev = mdev;
+
+	ret = v4l2_device_register(csi->dev, &csi->v4l2_dev);
+	if (ret)
+		return ret;
+
+	ret = media_device_register(mdev);
+	if (ret) {
+		v4l2_device_unregister(&csi->v4l2_dev);
+		media_device_cleanup(mdev);
+		return ret;
+	}
+
+	return 0;
+}
+
+static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
+{
+	struct ti_csi2rx_dev *csi = ctx->csi;
+	struct video_device *vdev = &ctx->vdev;
 	const struct ti_csi2rx_fmt *fmt;
-	struct v4l2_pix_format *pix_fmt = &csi->v_fmt.fmt.pix;
+	struct v4l2_pix_format *pix_fmt = &ctx->v_fmt.fmt.pix;
 	int ret;
 
+	mutex_init(&ctx->mutex);
+
 	fmt = find_format_by_fourcc(V4L2_PIX_FMT_UYVY);
 	if (!fmt)
 		return -EINVAL;
@@ -1018,15 +1097,16 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
 	pix_fmt->quantization = V4L2_QUANTIZATION_LIM_RANGE,
 	pix_fmt->xfer_func = V4L2_XFER_FUNC_SRGB,
 
-	ti_csi2rx_fill_fmt(fmt, &csi->v_fmt);
-
-	mdev->dev = csi->dev;
-	mdev->hw_revision = 1;
-	strscpy(mdev->model, "TI-CSI2RX", sizeof(mdev->model));
+	ti_csi2rx_fill_fmt(fmt, &ctx->v_fmt);
 
-	media_device_init(mdev);
+	csi->pad.flags = MEDIA_PAD_FL_SINK;
+	vdev->entity.ops = &ti_csi2rx_video_entity_ops;
+	ret = media_entity_pads_init(&ctx->vdev.entity, 1, &csi->pad);
+	if (ret)
+		return ret;
 
-	strscpy(vdev->name, TI_CSI2RX_MODULE_NAME, sizeof(vdev->name));
+	snprintf(vdev->name, sizeof(vdev->name), "%s context %u",
+		 dev_name(csi->dev), ctx->idx);
 	vdev->v4l2_dev = &csi->v4l2_dev;
 	vdev->vfl_dir = VFL_DIR_RX;
 	vdev->fops = &csi_fops;
@@ -1034,61 +1114,28 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
 	vdev->release = video_device_release_empty;
 	vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING |
 			    V4L2_CAP_IO_MC;
-	vdev->lock = &csi->mutex;
-	video_set_drvdata(vdev, csi);
+	vdev->lock = &ctx->mutex;
+	video_set_drvdata(vdev, ctx);
 
-	csi->pad.flags = MEDIA_PAD_FL_SINK;
-	vdev->entity.ops = &ti_csi2rx_video_entity_ops;
-	ret = media_entity_pads_init(&csi->vdev.entity, 1, &csi->pad);
+	ret = ti_csi2rx_init_dma(ctx);
 	if (ret)
 		return ret;
 
-	csi->v4l2_dev.mdev = mdev;
-
-	ret = v4l2_device_register(csi->dev, &csi->v4l2_dev);
+	ret = ti_csi2rx_init_vb2q(ctx);
 	if (ret)
-		return ret;
-
-	ret = media_device_register(mdev);
-	if (ret) {
-		v4l2_device_unregister(&csi->v4l2_dev);
-		media_device_cleanup(mdev);
-		return ret;
-	}
+		goto cleanup_dma;
 
 	return 0;
-}
 
-static void ti_csi2rx_cleanup_dma(struct ti_csi2rx_dev *csi)
-{
-	dma_free_coherent(csi->dev, csi->dma.drain.len,
-			  csi->dma.drain.vaddr, csi->dma.drain.paddr);
-	csi->dma.drain.vaddr = NULL;
-	dma_release_channel(csi->dma.chan);
-}
-
-static void ti_csi2rx_cleanup_v4l2(struct ti_csi2rx_dev *csi)
-{
-	media_device_unregister(&csi->mdev);
-	v4l2_device_unregister(&csi->v4l2_dev);
-	media_device_cleanup(&csi->mdev);
-}
-
-static void ti_csi2rx_cleanup_subdev(struct ti_csi2rx_dev *csi)
-{
-	v4l2_async_nf_unregister(&csi->notifier);
-	v4l2_async_nf_cleanup(&csi->notifier);
-}
-
-static void ti_csi2rx_cleanup_vb2q(struct ti_csi2rx_dev *csi)
-{
-	vb2_queue_release(&csi->vidq);
+cleanup_dma:
+	ti_csi2rx_cleanup_dma(ctx);
+	return ret;
 }
 
 static int ti_csi2rx_probe(struct platform_device *pdev)
 {
 	struct ti_csi2rx_dev *csi;
-	int ret;
+	int ret, i;
 
 	csi = devm_kzalloc(&pdev->dev, sizeof(*csi), GFP_KERNEL);
 	if (!csi)
@@ -1097,62 +1144,76 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
 	csi->dev = &pdev->dev;
 	platform_set_drvdata(pdev, csi);
 
-	mutex_init(&csi->mutex);
 	csi->shim = devm_platform_ioremap_resource(pdev, 0);
 	if (IS_ERR(csi->shim)) {
 		ret = PTR_ERR(csi->shim);
-		goto err_mutex;
+		return ret;
 	}
 
-	ret = ti_csi2rx_init_dma(csi);
-	if (ret)
-		goto err_mutex;
+	csi->drain.len = DRAIN_BUFFER_SIZE;
+	csi->drain.vaddr = dma_alloc_coherent(csi->dev, csi->drain.len,
+					      &csi->drain.paddr,
+					      GFP_KERNEL);
+	if (!csi->drain.vaddr)
+		return -ENOMEM;
 
 	ret = ti_csi2rx_v4l2_init(csi);
-	if (ret)
-		goto err_dma;
-
-	ret = ti_csi2rx_init_vb2q(csi);
 	if (ret)
 		goto err_v4l2;
 
+	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
+		csi->ctx[i].idx = i;
+		csi->ctx[i].csi = csi;
+		ret = ti_csi2rx_init_ctx(&csi->ctx[i]);
+		if (ret)
+			goto err_ctx;
+	}
+
 	ret = ti_csi2rx_notifier_register(csi);
 	if (ret)
-		goto err_vb2q;
+		goto err_ctx;
 
 	ret = of_platform_populate(csi->dev->of_node, NULL, NULL, csi->dev);
 	if (ret) {
 		dev_err(csi->dev, "Failed to create children: %d\n", ret);
-		goto err_subdev;
+		goto err_notifier;
 	}
 
 	return 0;
 
-err_subdev:
-	ti_csi2rx_cleanup_subdev(csi);
-err_vb2q:
-	ti_csi2rx_cleanup_vb2q(csi);
-err_v4l2:
+err_notifier:
+	ti_csi2rx_cleanup_notifier(csi);
+err_ctx:
+	i--;
+	for (; i >= 0; i--)
+		ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
 	ti_csi2rx_cleanup_v4l2(csi);
-err_dma:
-	ti_csi2rx_cleanup_dma(csi);
-err_mutex:
-	mutex_destroy(&csi->mutex);
+err_v4l2:
+	dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
+			  csi->drain.paddr);
 	return ret;
 }
 
Quoted 7 lines

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int ti_csi2rx_probe(struct platform_device *pdev) · line 1200

Quoted 4 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +1 -0
 static void ti_csi2rx_remove(struct platform_device *pdev)
 {
 	struct ti_csi2rx_dev *csi = platform_get_drvdata(pdev);
+	int i;

unsigned

Will fix.

Quoted 6 lines

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int ti_csi2rx_probe(struct platform_device *pdev) · line 1203

Quoted 3 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +3 -0
+
+	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
+		if (vb2_is_busy(&csi->ctx[i].vidq))

Can this happen ?

Probably not. While this driver is in use modprobe -r j721e_csi2rx would fail anyway because the module refcnt > 0. And when it's not in use the queue would never be busy...

Thanks for the catch, will drop this check (and move to .remove_new API)

Quoted 26 lines
Quoted 26 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +9 -6
+			dev_err(csi->dev,
+				"Failed to remove as queue busy for ctx %u\n",
+				i);
+	}
 
-	video_unregister_device(&csi->vdev);
+	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++)
+		ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
 
-	ti_csi2rx_cleanup_vb2q(csi);
-	ti_csi2rx_cleanup_subdev(csi);
+	ti_csi2rx_cleanup_notifier(csi);
 	ti_csi2rx_cleanup_v4l2(csi);
-	ti_csi2rx_cleanup_dma(csi);
 
-	mutex_destroy(&csi->mutex);
+	dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
+			  csi->drain.paddr);
 }
 
 static const struct of_device_id ti_csi2rx_of_match[] = {
 
--

2.43.0

Signature · 4 lines
Thanks,
Jai

GPG Fingerprint: 4DE0 D818 E5D5 75E8 D45A AFC5 43DE 91F9 249A 7145
author
#15 lore raw

PATCH v2 4/13 media: ti: j721e-csi2rx: prepare SHIM code for multiple contexts

In reply to Jai Luthra

From: Pratyush Yadav <[email protected]>

Currently the SHIM code to configure the context only touches the first context. Add support for writing to the context's registers based on the context index.

---
 drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +6 -6
index d8dfe0002b72..bffc8165fd33 100644
@@ -25,7 +25,7 @@
 #define SHIM_CNTL			0x10
 #define SHIM_CNTL_PIX_RST		BIT(0)
 
-#define SHIM_DMACNTX			0x20
+#define SHIM_DMACNTX(i)			(0x20 + ((i) * 0x20))
 #define SHIM_DMACNTX_EN			BIT(31)
 #define SHIM_DMACNTX_YUV422		GENMASK(27, 26)
 #define SHIM_DMACNTX_SIZE		GENMASK(21, 20)
@@ -35,7 +35,7 @@
 #define SHIM_DMACNTX_SIZE_16		1
 #define SHIM_DMACNTX_SIZE_32		2
 
-#define SHIM_PSI_CFG0			0x24
+#define SHIM_PSI_CFG0(i)		(0x24 + ((i) * 0x20))
 #define SHIM_PSI_CFG0_SRC_TAG		GENMASK(15, 0)
 #define SHIM_PSI_CFG0_DST_TAG		GENMASK(31, 16)
 
@@ -549,11 +549,11 @@ static void ti_csi2rx_setup_shim(struct ti_csi2rx_ctx *ctx)
 
 	reg |= FIELD_PREP(SHIM_DMACNTX_SIZE, fmt->size);
 
-	writel(reg, csi->shim + SHIM_DMACNTX);
+	writel(reg, csi->shim + SHIM_DMACNTX(ctx->idx));
 
 	reg = FIELD_PREP(SHIM_PSI_CFG0_SRC_TAG, 0) |
 	      FIELD_PREP(SHIM_PSI_CFG0_DST_TAG, 0);
-	writel(reg, csi->shim + SHIM_PSI_CFG0);
+	writel(reg, csi->shim + SHIM_PSI_CFG0(ctx->idx));
 }
 
 static void ti_csi2rx_drain_callback(void *param)
@@ -870,7 +870,7 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 err_pipeline:
 	video_device_pipeline_stop(&ctx->vdev);
 	writel(0, csi->shim + SHIM_CNTL);
-	writel(0, csi->shim + SHIM_DMACNTX);
+	writel(0, csi->shim + SHIM_DMACNTX(ctx->idx));
 err:
 	ti_csi2rx_cleanup_buffers(ctx, VB2_BUF_STATE_QUEUED);
 	return ret;
@@ -885,7 +885,7 @@ static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
 	video_device_pipeline_stop(&ctx->vdev);
 
 	writel(0, csi->shim + SHIM_CNTL);
-	writel(0, csi->shim + SHIM_DMACNTX);
+	writel(0, csi->shim + SHIM_DMACNTX(ctx->idx));
 
 	ret = v4l2_subdev_call(csi->source, video, s_stream, 0);
 	if (ret)
Signature · 1 line
2.43.0
Reviewed-by on v2 4/13
#16 lore raw

Re: [PATCH v2 04/13] media: ti: j721e-csi2rx: prepare SHIM code for multiple contexts

Hi Jai

Quoted 8 lines · Jai Luthra

From: Pratyush Yadav <[email protected]>

Currently the SHIM code to configure the context only touches the first context. Add support for writing to the context's registers based on the context index.

Thanks
j

Quoted 63 lines
---
 drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +6 -7
index d8dfe0002b72..bffc8165fd33 100644
@@ -25,7 +25,7 @@
 #define SHIM_CNTL			0x10
 #define SHIM_CNTL_PIX_RST		BIT(0)
 
-#define SHIM_DMACNTX			0x20
+#define SHIM_DMACNTX(i)			(0x20 + ((i) * 0x20))
 #define SHIM_DMACNTX_EN			BIT(31)
 #define SHIM_DMACNTX_YUV422		GENMASK(27, 26)
 #define SHIM_DMACNTX_SIZE		GENMASK(21, 20)
@@ -35,7 +35,7 @@
 #define SHIM_DMACNTX_SIZE_16		1
 #define SHIM_DMACNTX_SIZE_32		2
 
-#define SHIM_PSI_CFG0			0x24
+#define SHIM_PSI_CFG0(i)		(0x24 + ((i) * 0x20))
 #define SHIM_PSI_CFG0_SRC_TAG		GENMASK(15, 0)
 #define SHIM_PSI_CFG0_DST_TAG		GENMASK(31, 16)
 
@@ -549,11 +549,11 @@ static void ti_csi2rx_setup_shim(struct ti_csi2rx_ctx *ctx)
 
 	reg |= FIELD_PREP(SHIM_DMACNTX_SIZE, fmt->size);
 
-	writel(reg, csi->shim + SHIM_DMACNTX);
+	writel(reg, csi->shim + SHIM_DMACNTX(ctx->idx));
 
 	reg = FIELD_PREP(SHIM_PSI_CFG0_SRC_TAG, 0) |
 	      FIELD_PREP(SHIM_PSI_CFG0_DST_TAG, 0);
-	writel(reg, csi->shim + SHIM_PSI_CFG0);
+	writel(reg, csi->shim + SHIM_PSI_CFG0(ctx->idx));
 }
 
 static void ti_csi2rx_drain_callback(void *param)
@@ -870,7 +870,7 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 err_pipeline:
 	video_device_pipeline_stop(&ctx->vdev);
 	writel(0, csi->shim + SHIM_CNTL);
-	writel(0, csi->shim + SHIM_DMACNTX);
+	writel(0, csi->shim + SHIM_DMACNTX(ctx->idx));
 err:
 	ti_csi2rx_cleanup_buffers(ctx, VB2_BUF_STATE_QUEUED);
 	return ret;
@@ -885,7 +885,7 @@ static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
 	video_device_pipeline_stop(&ctx->vdev);
 
 	writel(0, csi->shim + SHIM_CNTL);
-	writel(0, csi->shim + SHIM_DMACNTX);
+	writel(0, csi->shim + SHIM_DMACNTX(ctx->idx));
 
 	ret = v4l2_subdev_call(csi->source, video, s_stream, 0);
 	if (ret)
 
--

2.43.0

author
#17 lore raw

PATCH v2 5/13 media: ti: j721e-csi2rx: allocate DMA channel based on context index

In reply to Jai Luthra

From: Pratyush Yadav <[email protected]>

With multiple contexts, there needs to be a different DMA channel for each context. Earlier, the DMA channel name was hard coded to "rx0" for the sake of simplicity. Generate the DMA channel name based on its index and get the channel corresponding to the context.

---
 drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +3 -1
index bffc8165fd33..361b0ea8e0d9 100644
@@ -1027,6 +1027,7 @@ static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
 	struct dma_slave_config cfg = {
 		.src_addr_width = DMA_SLAVE_BUSWIDTH_16_BYTES,
 	};
+	char name[32];
 	int ret;
 
 	INIT_LIST_HEAD(&ctx->dma.queue);
@@ -1035,7 +1036,8 @@ static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
 
 	ctx->dma.state = TI_CSI2RX_DMA_STOPPED;
 
-	ctx->dma.chan = dma_request_chan(ctx->csi->dev, "rx0");
+	snprintf(name, sizeof(name), "rx%u", ctx->idx);
+	ctx->dma.chan = dma_request_chan(ctx->csi->dev, name);
 	if (IS_ERR(ctx->dma.chan))
 		return PTR_ERR(ctx->dma.chan);
 
Signature · 1 line
2.43.0
Reviewed-by on v2 5/13
#18 lore raw

Re: [PATCH v2 05/13] media: ti: j721e-csi2rx: allocate DMA channel based on context index

Hi Jai

Quoted 9 lines · Jai Luthra

From: Pratyush Yadav <[email protected]>

With multiple contexts, there needs to be a different DMA channel for each context. Earlier, the DMA channel name was hard coded to "rx0" for the sake of simplicity. Generate the DMA channel name based on its index and get the channel corresponding to the context.

Thanks
j

Quoted 31 lines
---
 drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +3 -2
index bffc8165fd33..361b0ea8e0d9 100644
@@ -1027,6 +1027,7 @@ static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
 	struct dma_slave_config cfg = {
 		.src_addr_width = DMA_SLAVE_BUSWIDTH_16_BYTES,
 	};
+	char name[32];
 	int ret;
 
 	INIT_LIST_HEAD(&ctx->dma.queue);
@@ -1035,7 +1036,8 @@ static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
 
 	ctx->dma.state = TI_CSI2RX_DMA_STOPPED;
 
-	ctx->dma.chan = dma_request_chan(ctx->csi->dev, "rx0");
+	snprintf(name, sizeof(name), "rx%u", ctx->idx);
+	ctx->dma.chan = dma_request_chan(ctx->csi->dev, name);
 	if (IS_ERR(ctx->dma.chan))
 		return PTR_ERR(ctx->dma.chan);
 
 
--

2.43.0

author
#19 lore raw

PATCH v2 6/13 media: ti: j721e-csi2rx: add a subdev for the core device

In reply to Jai Luthra

With single stream capture, it was simpler to use the video device as the media entity representing the main TI CSI2RX device. Now with multi stream capture coming into the picture, the model has shifted to each video device having a link to the main device's subdev. The routing would then be set on this subdev.

Add this subdev, link each context to this subdev's entity and link the subdev's entity to the source. Also add an array of media pads. It will have one sink pad and source pads equal to the number of contexts.

---
 .../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c  | 237 ++++++++++++++++++---
 1 file changed, 212 insertions(+), 25 deletions(-)
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +212 -25
index 361b0ea8e0d9..13d7426ab4ba 100644
@@ -51,6 +51,11 @@
 #define MAX_WIDTH_BYTES			SZ_16K
 #define MAX_HEIGHT_LINES		SZ_16K
 
+#define TI_CSI2RX_PAD_SINK		0
+#define TI_CSI2RX_PAD_FIRST_SOURCE	1
+#define TI_CSI2RX_NUM_SOURCE_PADS	1
+#define TI_CSI2RX_NUM_PADS		(1 + TI_CSI2RX_NUM_SOURCE_PADS)
+
 #define DRAIN_TIMEOUT_MS		50
 #define DRAIN_BUFFER_SIZE		SZ_32K
 
@@ -97,6 +102,7 @@ struct ti_csi2rx_ctx {
 	struct mutex			mutex; /* To serialize ioctls. */
 	struct v4l2_format		v_fmt;
 	struct ti_csi2rx_dma		dma;
+	struct media_pad		pad;
 	u32				sequence;
 	u32				idx;
 };
@@ -104,12 +110,15 @@ struct ti_csi2rx_ctx {
 struct ti_csi2rx_dev {
 	struct device			*dev;
 	void __iomem			*shim;
+	struct mutex			mutex; /* To serialize ioctls. */
+	unsigned int			enable_count;
 	struct v4l2_device		v4l2_dev;
 	struct media_device		mdev;
 	struct media_pipeline		pipe;
-	struct media_pad		pad;
+	struct media_pad		pads[TI_CSI2RX_NUM_PADS];
 	struct v4l2_async_notifier	notifier;
 	struct v4l2_subdev		*source;
+	struct v4l2_subdev		subdev;
 	struct ti_csi2rx_ctx		ctx[TI_CSI2RX_NUM_CTX];
 	/* Buffer to drain stale data from PSI-L endpoint */
 	struct {
@@ -431,6 +440,15 @@ static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
 	struct ti_csi2rx_dev *csi = dev_get_drvdata(notifier->v4l2_dev->dev);
 	int ret, i;
 
+	/* Create link from source to subdev */
+	ret = v4l2_create_fwnode_links_to_pad(csi->source,
+					      &csi->pads[TI_CSI2RX_PAD_SINK],
+					      MEDIA_LNK_FL_IMMUTABLE |
+					      MEDIA_LNK_FL_ENABLED);
+	if (ret)
+		return ret;
+
+	/* Create and link video nodes for all DMA contexts */
 	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
 		struct ti_csi2rx_ctx *ctx = &csi->ctx[i];
 		struct video_device *vdev = &ctx->vdev;
@@ -438,13 +456,17 @@ static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
 		ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
 		if (ret)
 			goto unregister_dev;
-	}
 
-	ret = v4l2_create_fwnode_links_to_pad(csi->source, &csi->pad,
-					      MEDIA_LNK_FL_IMMUTABLE |
-					      MEDIA_LNK_FL_ENABLED);
-	if (ret)
-		goto unregister_dev;
+		ret = media_create_pad_link(&csi->subdev.entity,
+					    TI_CSI2RX_PAD_FIRST_SOURCE + ctx->idx,
+					    &vdev->entity, 0,
+					    MEDIA_LNK_FL_IMMUTABLE |
+					    MEDIA_LNK_FL_ENABLED);
+		if (ret) {
+			video_unregister_device(vdev);
+			goto unregister_dev;
+		}
+	}
 
 	ret = v4l2_device_register_subdev_nodes(&csi->v4l2_dev);
 	if (ret)
@@ -859,7 +881,7 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 	dma->state = TI_CSI2RX_DMA_ACTIVE;
 	spin_unlock_irqrestore(&dma->lock, flags);
 
-	ret = v4l2_subdev_call(csi->source, video, s_stream, 1);
+	ret = v4l2_subdev_call(&csi->subdev, video, s_stream, 1);
 	if (ret)
 		goto err_dma;
 
@@ -887,7 +909,7 @@ static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
 	writel(0, csi->shim + SHIM_CNTL);
 	writel(0, csi->shim + SHIM_DMACNTX(ctx->idx));
 
-	ret = v4l2_subdev_call(csi->source, video, s_stream, 0);
+	ret = v4l2_subdev_call(&csi->subdev, video, s_stream, 0);
 	if (ret)
 		dev_err(csi->dev, "Failed to stop subdev stream\n");
 
@@ -905,6 +927,119 @@ static const struct vb2_ops csi_vb2_qops = {
 	.wait_finish = vb2_ops_wait_finish,
 };
 
+static inline struct ti_csi2rx_dev *to_csi2rx_dev(struct v4l2_subdev *sd)
+{
+	return container_of(sd, struct ti_csi2rx_dev, subdev);
+}
+
+static int ti_csi2rx_sd_set_fmt(struct v4l2_subdev *sd,
+				struct v4l2_subdev_state *state,
+				struct v4l2_subdev_format *format)
+{
+	struct v4l2_mbus_framefmt *fmt;
+	int ret = 0;
+
+	/* No transcoding, don't allow setting source fmt */
+	if (format->pad >= TI_CSI2RX_PAD_FIRST_SOURCE)
+		return v4l2_subdev_get_fmt(sd, state, format);
+
+	if (!find_format_by_code(format->format.code))
+		format->format.code = ti_csi2rx_formats[0].code;
+
+	format->format.field = V4L2_FIELD_NONE;
+
+	fmt = v4l2_subdev_state_get_format(state, format->pad, format->stream);
+	if (!fmt) {
+		ret = -EINVAL;
+		goto out;
+	}
+	*fmt = format->format;
+
+	fmt = v4l2_subdev_state_get_format(state, TI_CSI2RX_PAD_FIRST_SOURCE,
+					   format->stream);
+	if (!fmt) {
+		ret = -EINVAL;
+		goto out;
+	}
+	*fmt = format->format;
+
+out:
+	return ret;
+}
+
+static int ti_csi2rx_sd_init_state(struct v4l2_subdev *sd,
+				   struct v4l2_subdev_state *state)
+{
+	struct v4l2_subdev_format format = {
+		.pad = TI_CSI2RX_PAD_SINK,
+		.format = {
+			.width = 640,
+			.height = 480,
+			.code = MEDIA_BUS_FMT_UYVY8_1X16,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.ycbcr_enc = V4L2_YCBCR_ENC_601,
+			.quantization = V4L2_QUANTIZATION_LIM_RANGE,
+			.xfer_func = V4L2_XFER_FUNC_SRGB,
+		},
+	};
+
+	return ti_csi2rx_sd_set_fmt(sd, state, &format);
+}
+
+static int ti_csi2rx_sd_s_stream(struct v4l2_subdev *sd, int enable)
+{
+	struct ti_csi2rx_dev *csi = to_csi2rx_dev(sd);
+	int ret = 0;
+
+	mutex_lock(&csi->mutex);
+
+	if (enable) {
+		if (csi->enable_count > 0) {
+			csi->enable_count++;
+			goto out;
+		}
+
+		ret = v4l2_subdev_call(csi->source, video, s_stream, 1);
+		if (ret)
+			goto out;
+
+		csi->enable_count++;
+	} else {
+		if (csi->enable_count == 0) {
+			ret = -EINVAL;
+			goto out;
+		}
+
+		if (--csi->enable_count > 0)
+			goto out;
+
+		ret = v4l2_subdev_call(csi->source, video, s_stream, 0);
+	}
+
+out:
+	mutex_unlock(&csi->mutex);
+	return ret;
+}
+
+static const struct v4l2_subdev_pad_ops ti_csi2rx_subdev_pad_ops = {
+	.get_fmt = v4l2_subdev_get_fmt,
+	.set_fmt = ti_csi2rx_sd_set_fmt,
+};
+
+static const struct v4l2_subdev_video_ops ti_csi2rx_subdev_video_ops = {
+	.s_stream = ti_csi2rx_sd_s_stream,
+};
+
+static const struct v4l2_subdev_ops ti_csi2rx_subdev_ops = {
+	.video = &ti_csi2rx_subdev_video_ops,
+	.pad = &ti_csi2rx_subdev_pad_ops,
+};
+
+static const struct v4l2_subdev_internal_ops ti_csi2rx_internal_ops = {
+	.init_state = ti_csi2rx_sd_init_state,
+};
+
 static void ti_csi2rx_cleanup_dma(struct ti_csi2rx_ctx *ctx)
 {
 	dma_release_channel(ctx->dma.chan);
@@ -912,6 +1047,7 @@ static void ti_csi2rx_cleanup_dma(struct ti_csi2rx_ctx *ctx)
 
 static void ti_csi2rx_cleanup_v4l2(struct ti_csi2rx_dev *csi)
 {
+	v4l2_subdev_cleanup(&csi->subdev);
 	media_device_unregister(&csi->mdev);
 	v4l2_device_unregister(&csi->v4l2_dev);
 	media_device_cleanup(&csi->mdev);
@@ -973,14 +1109,22 @@ static int ti_csi2rx_link_validate(struct media_link *link)
 	struct v4l2_subdev_format source_fmt = {
 		.which	= V4L2_SUBDEV_FORMAT_ACTIVE,
 		.pad	= link->source->index,
+		.stream = 0,
 	};
+	struct v4l2_subdev_state *state;
 	const struct ti_csi2rx_fmt *ti_fmt;
 	int ret;
 
-	ret = v4l2_subdev_call_state_active(csi->source, pad,
-					    get_fmt, &source_fmt);
-	if (ret)
-		return ret;
+	state = v4l2_subdev_lock_and_get_active_state(&csi->subdev);
+	ret = v4l2_subdev_call(&csi->subdev, pad, get_fmt, state, &source_fmt);
+	v4l2_subdev_unlock_state(state);
+
+	if (ret) {
+		dev_dbg(csi->dev,
+			"Skipping validation as no format present on \"%s\":%u:0\n",
+			link->source->entity->name, link->source->index);
+		return 0;
+	}
 
 	if (source_fmt.format.width != csi_fmt->width) {
 		dev_dbg(csi->dev, "Width does not match (source %u, sink %u)\n",
@@ -1010,8 +1154,9 @@ static int ti_csi2rx_link_validate(struct media_link *link)
 
 	if (ti_fmt->fourcc != csi_fmt->pixelformat) {
 		dev_dbg(csi->dev,
-			"Cannot transform source fmt 0x%x to sink fmt 0x%x\n",
-			ti_fmt->fourcc, csi_fmt->pixelformat);
+			"Cannot transform \"%s\":%u format %p4cc to %p4cc\n",
+			link->source->entity->name, link->source->index,
+			&ti_fmt->fourcc, &csi_fmt->pixelformat);
 		return -EPIPE;
 	}
 
@@ -1022,6 +1167,10 @@ static const struct media_entity_operations ti_csi2rx_video_entity_ops = {
 	.link_validate = ti_csi2rx_link_validate,
 };
 
+static const struct media_entity_operations ti_csi2rx_subdev_entity_ops = {
+	.link_validate = v4l2_subdev_link_validate,
+};
+
 static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
 {
 	struct dma_slave_config cfg = {
@@ -1053,7 +1202,8 @@ static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
 static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
 {
 	struct media_device *mdev = &csi->mdev;
-	int ret;
+	struct v4l2_subdev *sd = &csi->subdev;
+	int ret, i;
 
 	mdev->dev = csi->dev;
 	mdev->hw_revision = 1;
@@ -1065,16 +1215,50 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
 
 	ret = v4l2_device_register(csi->dev, &csi->v4l2_dev);
 	if (ret)
-		return ret;
+		goto cleanup_media;
 
 	ret = media_device_register(mdev);
-	if (ret) {
-		v4l2_device_unregister(&csi->v4l2_dev);
-		media_device_cleanup(mdev);
-		return ret;
-	}
+	if (ret)
+		goto unregister_v4l2;
+
+	v4l2_subdev_init(sd, &ti_csi2rx_subdev_ops);
+	sd->internal_ops = &ti_csi2rx_internal_ops;
+	sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
+	sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
+	strscpy(sd->name, dev_name(csi->dev), sizeof(sd->name));
+	sd->dev = csi->dev;
+	sd->entity.ops = &ti_csi2rx_subdev_entity_ops;
+
+	csi->pads[TI_CSI2RX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+
+	for (i = TI_CSI2RX_PAD_FIRST_SOURCE; i < TI_CSI2RX_NUM_PADS; i++)
+		csi->pads[i].flags = MEDIA_PAD_FL_SOURCE;
+
+	ret = media_entity_pads_init(&sd->entity, ARRAY_SIZE(csi->pads),
+				     csi->pads);
+	if (ret)
+		goto unregister_media;
+
+	ret = v4l2_subdev_init_finalize(sd);
+	if (ret)
+		goto unregister_media;
+
+	ret = v4l2_device_register_subdev(&csi->v4l2_dev, sd);
+	if (ret)
+		goto cleanup_subdev;
 
 	return 0;
+
+cleanup_subdev:
+	v4l2_subdev_cleanup(sd);
+unregister_media:
+	media_device_unregister(mdev);
+unregister_v4l2:
+	v4l2_device_unregister(&csi->v4l2_dev);
+cleanup_media:
+	media_device_cleanup(mdev);
+
+	return ret;
 }
 
 static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
@@ -1101,9 +1285,9 @@ static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
 
 	ti_csi2rx_fill_fmt(fmt, &ctx->v_fmt);
 
-	csi->pad.flags = MEDIA_PAD_FL_SINK;
+	ctx->pad.flags = MEDIA_PAD_FL_SINK;
 	vdev->entity.ops = &ti_csi2rx_video_entity_ops;
-	ret = media_entity_pads_init(&ctx->vdev.entity, 1, &csi->pad);
+	ret = media_entity_pads_init(&ctx->vdev.entity, 1, &ctx->pad);
 	if (ret)
 		return ret;
 
@@ -1159,6 +1343,8 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
 	if (!csi->drain.vaddr)
 		return -ENOMEM;
 
+	mutex_init(&csi->mutex);
+
 	ret = ti_csi2rx_v4l2_init(csi);
 	if (ret)
 		goto err_v4l2;
@@ -1191,6 +1377,7 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
 		ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
 	ti_csi2rx_cleanup_v4l2(csi);
 err_v4l2:
+	mutex_destroy(&csi->mutex);
 	dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
 			  csi->drain.paddr);
 	return ret;
@@ -1213,7 +1400,7 @@ static void ti_csi2rx_remove(struct platform_device *pdev)
 
 	ti_csi2rx_cleanup_notifier(csi);
 	ti_csi2rx_cleanup_v4l2(csi);
-
+	mutex_destroy(&csi->mutex);
 	dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
 			  csi->drain.paddr);
 }
Signature · 1 line
2.43.0
#20 lore raw

Re: [PATCH v2 06/13] media: ti: j721e-csi2rx: add a subdev for the core device

Quoted 59 lines · Jai Luthra

With single stream capture, it was simpler to use the video device as the media entity representing the main TI CSI2RX device. Now with multi stream capture coming into the picture, the model has shifted to each video device having a link to the main device's subdev. The routing would then be set on this subdev.

Add this subdev, link each context to this subdev's entity and link the subdev's entity to the source. Also add an array of media pads. It will have one sink pad and source pads equal to the number of contexts.

---
 .../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c  | 237 ++++++++++++++++++---
 1 file changed, 212 insertions(+), 25 deletions(-)
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +10 -1
index 361b0ea8e0d9..13d7426ab4ba 100644
@@ -51,6 +51,11 @@
 #define MAX_WIDTH_BYTES			SZ_16K
 #define MAX_HEIGHT_LINES		SZ_16K
 
+#define TI_CSI2RX_PAD_SINK		0
+#define TI_CSI2RX_PAD_FIRST_SOURCE	1
+#define TI_CSI2RX_NUM_SOURCE_PADS	1
+#define TI_CSI2RX_NUM_PADS		(1 + TI_CSI2RX_NUM_SOURCE_PADS)
+
 #define DRAIN_TIMEOUT_MS		50
 #define DRAIN_BUFFER_SIZE		SZ_32K
 
@@ -97,6 +102,7 @@ struct ti_csi2rx_ctx {
 	struct mutex			mutex; /* To serialize ioctls. */
 	struct v4l2_format		v_fmt;
 	struct ti_csi2rx_dma		dma;
+	struct media_pad		pad;
 	u32				sequence;
 	u32				idx;
 };
@@ -104,12 +110,15 @@ struct ti_csi2rx_ctx {
 struct ti_csi2rx_dev {
 	struct device			*dev;
 	void __iomem			*shim;
+	struct mutex			mutex; /* To serialize ioctls. */
+	unsigned int			enable_count;
 	struct v4l2_device		v4l2_dev;
 	struct media_device		mdev;
 	struct media_pipeline		pipe;
-	struct media_pad		pad;
+	struct media_pad		pads[TI_CSI2RX_NUM_PADS];
 	struct v4l2_async_notifier	notifier;
 	struct v4l2_subdev		*source;
+	struct v4l2_subdev		subdev;
 	struct ti_csi2rx_ctx		ctx[TI_CSI2RX_NUM_CTX];
 	/* Buffer to drain stale data from PSI-L endpoint */
 	struct {

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier) · line 444

Quoted 5 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +2 -0
@@ -431,6 +440,15 @@ static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
 	struct ti_csi2rx_dev *csi = dev_get_drvdata(notifier->v4l2_dev->dev);
 	int ret, i;
 
+	/* Create link from source to subdev */
+	ret = v4l2_create_fwnode_links_to_pad(csi->source,

Isn't this a single link from the image source to the RX's sink pad ? Wouldn't media_create_pad_link() do here ?

Quoted 19 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +7 -4
+					      &csi->pads[TI_CSI2RX_PAD_SINK],
+					      MEDIA_LNK_FL_IMMUTABLE |
+					      MEDIA_LNK_FL_ENABLED);
+	if (ret)
+		return ret;
+
+	/* Create and link video nodes for all DMA contexts */
 	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
 		struct ti_csi2rx_ctx *ctx = &csi->ctx[i];
 		struct video_device *vdev = &ctx->vdev;
@@ -438,13 +456,17 @@ static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
 		ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
 		if (ret)
 			goto unregister_dev;
-	}
 
-	ret = v4l2_create_fwnode_links_to_pad(csi->source, &csi->pad,
-					      MEDIA_LNK_FL_IMMUTABLE |
-					      MEDIA_LNK_FL_ENABLED);

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier) · line 467

Quoted 10 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +8 -2
-	if (ret)
-		goto unregister_dev;
+		ret = media_create_pad_link(&csi->subdev.entity,
+					    TI_CSI2RX_PAD_FIRST_SOURCE + ctx->idx,
+					    &vdev->entity, 0,
+					    MEDIA_LNK_FL_IMMUTABLE |
+					    MEDIA_LNK_FL_ENABLED);
+		if (ret) {
+			video_unregister_device(vdev);
+			goto unregister_dev;

Should you call media_entity_remove_links() on the csi2 entity on the error path ?

Quoted 31 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +8 -2
+		}
+	}
 
 	ret = v4l2_device_register_subdev_nodes(&csi->v4l2_dev);
 	if (ret)
@@ -859,7 +881,7 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 	dma->state = TI_CSI2RX_DMA_ACTIVE;
 	spin_unlock_irqrestore(&dma->lock, flags);
 
-	ret = v4l2_subdev_call(csi->source, video, s_stream, 1);
+	ret = v4l2_subdev_call(&csi->subdev, video, s_stream, 1);
 	if (ret)
 		goto err_dma;
 
@@ -887,7 +909,7 @@ static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
 	writel(0, csi->shim + SHIM_CNTL);
 	writel(0, csi->shim + SHIM_DMACNTX(ctx->idx));
 
-	ret = v4l2_subdev_call(csi->source, video, s_stream, 0);
+	ret = v4l2_subdev_call(&csi->subdev, video, s_stream, 0);
 	if (ret)
 		dev_err(csi->dev, "Failed to stop subdev stream\n");
 
@@ -905,6 +927,119 @@ static const struct vb2_ops csi_vb2_qops = {
 	.wait_finish = vb2_ops_wait_finish,
 };
 
+static inline struct ti_csi2rx_dev *to_csi2rx_dev(struct v4l2_subdev *sd)
+{
+	return container_of(sd, struct ti_csi2rx_dev, subdev);
+}

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct vb2_ops csi_vb2_qops = { · line 943

Quoted 10 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +10 -0
+
+static int ti_csi2rx_sd_set_fmt(struct v4l2_subdev *sd,
+				struct v4l2_subdev_state *state,
+				struct v4l2_subdev_format *format)
+{
+	struct v4l2_mbus_framefmt *fmt;
+	int ret = 0;
+
+	/* No transcoding, don't allow setting source fmt */
+	if (format->pad >= TI_CSI2RX_PAD_FIRST_SOURCE)

> TI_CSI2RX_PAD_SINK

might be clearer

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct vb2_ops csi_vb2_qops = { · line 949

Quoted 6 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +6 -0
+		return v4l2_subdev_get_fmt(sd, state, format);
+
+	if (!find_format_by_code(format->format.code))
+		format->format.code = ti_csi2rx_formats[0].code;
+
+	format->format.field = V4L2_FIELD_NONE;

What about other colorspace related fields ?

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct vb2_ops csi_vb2_qops = { · line 952

Quoted 3 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +3 -0
+
+	fmt = v4l2_subdev_state_get_format(state, format->pad, format->stream);
+	if (!fmt) {

Can this happen ?

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct vb2_ops csi_vb2_qops = { · line 961

Quoted 9 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +9 -0
+		ret = -EINVAL;
+		goto out;
+	}
+	*fmt = format->format;
+
+	fmt = v4l2_subdev_state_get_format(state, TI_CSI2RX_PAD_FIRST_SOURCE,
+					   format->stream);
+	if (!fmt) {
+		ret = -EINVAL;

ditto

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct vb2_ops csi_vb2_qops = { · line 962

Quoted 1 line
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +1 -0
+		goto out;

kind of missing the point of a jump here where you can simply return -EINVAL

Quoted 18 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +18 -0
+	}
+	*fmt = format->format;
+
+out:
+	return ret;
+}
+
+static int ti_csi2rx_sd_init_state(struct v4l2_subdev *sd,
+				   struct v4l2_subdev_state *state)
+{
+	struct v4l2_subdev_format format = {
+		.pad = TI_CSI2RX_PAD_SINK,
+		.format = {
+			.width = 640,
+			.height = 480,
+			.code = MEDIA_BUS_FMT_UYVY8_1X16,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct vb2_ops csi_vb2_qops = { · line 990

Quoted 10 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +10 -0
+			.ycbcr_enc = V4L2_YCBCR_ENC_601,
+			.quantization = V4L2_QUANTIZATION_LIM_RANGE,
+			.xfer_func = V4L2_XFER_FUNC_SRGB,
+		},
+	};
+
+	return ti_csi2rx_sd_set_fmt(sd, state, &format);
+}
+
+static int ti_csi2rx_sd_s_stream(struct v4l2_subdev *sd, int enable)

Even if I never used that API myself, but I have a feeling enable_streams() (which is meant to replace s_stream for multiplexed subdev) would help you avoiding the manual enable_count booking here below ?

Quoted 64 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +53 -0
+{
+	struct ti_csi2rx_dev *csi = to_csi2rx_dev(sd);
+	int ret = 0;
+
+	mutex_lock(&csi->mutex);
+
+	if (enable) {
+		if (csi->enable_count > 0) {
+			csi->enable_count++;
+			goto out;
+		}
+
+		ret = v4l2_subdev_call(csi->source, video, s_stream, 1);
+		if (ret)
+			goto out;
+
+		csi->enable_count++;
+	} else {
+		if (csi->enable_count == 0) {
+			ret = -EINVAL;
+			goto out;
+		}
+
+		if (--csi->enable_count > 0)
+			goto out;
+
+		ret = v4l2_subdev_call(csi->source, video, s_stream, 0);
+	}
+
+out:
+	mutex_unlock(&csi->mutex);
+	return ret;
+}
+
+static const struct v4l2_subdev_pad_ops ti_csi2rx_subdev_pad_ops = {
+	.get_fmt = v4l2_subdev_get_fmt,
+	.set_fmt = ti_csi2rx_sd_set_fmt,
+};
+
+static const struct v4l2_subdev_video_ops ti_csi2rx_subdev_video_ops = {
+	.s_stream = ti_csi2rx_sd_s_stream,
+};
+
+static const struct v4l2_subdev_ops ti_csi2rx_subdev_ops = {
+	.video = &ti_csi2rx_subdev_video_ops,
+	.pad = &ti_csi2rx_subdev_pad_ops,
+};
+
+static const struct v4l2_subdev_internal_ops ti_csi2rx_internal_ops = {
+	.init_state = ti_csi2rx_sd_init_state,
+};
+
 static void ti_csi2rx_cleanup_dma(struct ti_csi2rx_ctx *ctx)
 {
 	dma_release_channel(ctx->dma.chan);
@@ -912,6 +1047,7 @@ static void ti_csi2rx_cleanup_dma(struct ti_csi2rx_ctx *ctx)
 
 static void ti_csi2rx_cleanup_v4l2(struct ti_csi2rx_dev *csi)
 {
+	v4l2_subdev_cleanup(&csi->subdev);
 	media_device_unregister(&csi->mdev);
 	v4l2_device_unregister(&csi->v4l2_dev);
 	media_device_cleanup(&csi->mdev);

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int ti_csi2rx_link_validate(struct media_link *link) · line 1112

Quoted 4 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +1 -0
@@ -973,14 +1109,22 @@ static int ti_csi2rx_link_validate(struct media_link *link)
 	struct v4l2_subdev_format source_fmt = {
 		.which	= V4L2_SUBDEV_FORMAT_ACTIVE,
 		.pad	= link->source->index,
+		.stream = 0,

I don't think it hurts, but this seems not to be strictly related to this patch ?

Quoted 7 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +1 -2
 	};
+	struct v4l2_subdev_state *state;
 	const struct ti_csi2rx_fmt *ti_fmt;
 	int ret;
 
-	ret = v4l2_subdev_call_state_active(csi->source, pad,
-					    get_fmt, &source_fmt);

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int ti_csi2rx_link_validate(struct media_link *link) · line 1125

Quoted 10 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +8 -2
-	if (ret)
-		return ret;
+	state = v4l2_subdev_lock_and_get_active_state(&csi->subdev);
+	ret = v4l2_subdev_call(&csi->subdev, pad, get_fmt, state, &source_fmt);
+	v4l2_subdev_unlock_state(state);
+
+	if (ret) {
+		dev_dbg(csi->dev,
+			"Skipping validation as no format present on \"%s\":%u:0\n",
+			link->source->entity->name, link->source->index);

If get_fmt returns an error, should you simply continue or fail ? In which cases get_fmt on a subdev fails ? Only if the pad or the stream are not valid right ?

Quoted 55 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +16 -9
+		return 0;
+	}
 
 	if (source_fmt.format.width != csi_fmt->width) {
 		dev_dbg(csi->dev, "Width does not match (source %u, sink %u)\n",
@@ -1010,8 +1154,9 @@ static int ti_csi2rx_link_validate(struct media_link *link)
 
 	if (ti_fmt->fourcc != csi_fmt->pixelformat) {
 		dev_dbg(csi->dev,
-			"Cannot transform source fmt 0x%x to sink fmt 0x%x\n",
-			ti_fmt->fourcc, csi_fmt->pixelformat);
+			"Cannot transform \"%s\":%u format %p4cc to %p4cc\n",
+			link->source->entity->name, link->source->index,
+			&ti_fmt->fourcc, &csi_fmt->pixelformat);
 		return -EPIPE;
 	}
 
@@ -1022,6 +1167,10 @@ static const struct media_entity_operations ti_csi2rx_video_entity_ops = {
 	.link_validate = ti_csi2rx_link_validate,
 };
 
+static const struct media_entity_operations ti_csi2rx_subdev_entity_ops = {
+	.link_validate = v4l2_subdev_link_validate,
+};
+
 static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
 {
 	struct dma_slave_config cfg = {
@@ -1053,7 +1202,8 @@ static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
 static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
 {
 	struct media_device *mdev = &csi->mdev;
-	int ret;
+	struct v4l2_subdev *sd = &csi->subdev;
+	int ret, i;
 
 	mdev->dev = csi->dev;
 	mdev->hw_revision = 1;
@@ -1065,16 +1215,50 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
 
 	ret = v4l2_device_register(csi->dev, &csi->v4l2_dev);
 	if (ret)
-		return ret;
+		goto cleanup_media;
 
 	ret = media_device_register(mdev);
-	if (ret) {
-		v4l2_device_unregister(&csi->v4l2_dev);
-		media_device_cleanup(mdev);
-		return ret;
-	}
+	if (ret)
+		goto unregister_v4l2;
+
+	v4l2_subdev_init(sd, &ti_csi2rx_subdev_ops);

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi) · line 1234

Quoted 10 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +10 -0
+	sd->internal_ops = &ti_csi2rx_internal_ops;
+	sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
+	sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
+	strscpy(sd->name, dev_name(csi->dev), sizeof(sd->name));
+	sd->dev = csi->dev;
+	sd->entity.ops = &ti_csi2rx_subdev_entity_ops;
+
+	csi->pads[TI_CSI2RX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+
+	for (i = TI_CSI2RX_PAD_FIRST_SOURCE; i < TI_CSI2RX_NUM_PADS; i++)
        for (unsigned int i = 0;
Quoted 31 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +25 -0
+		csi->pads[i].flags = MEDIA_PAD_FL_SOURCE;
+
+	ret = media_entity_pads_init(&sd->entity, ARRAY_SIZE(csi->pads),
+				     csi->pads);
+	if (ret)
+		goto unregister_media;
+
+	ret = v4l2_subdev_init_finalize(sd);
+	if (ret)
+		goto unregister_media;
+
+	ret = v4l2_device_register_subdev(&csi->v4l2_dev, sd);
+	if (ret)
+		goto cleanup_subdev;
 
 	return 0;
+
+cleanup_subdev:
+	v4l2_subdev_cleanup(sd);
+unregister_media:
+	media_device_unregister(mdev);
+unregister_v4l2:
+	v4l2_device_unregister(&csi->v4l2_dev);
+cleanup_media:
+	media_device_cleanup(mdev);
+
+	return ret;
 }
 
 static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx) · line 1104

Quoted 4 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +0 -1
@@ -1101,9 +1285,9 @@ static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
 
 	ti_csi2rx_fill_fmt(fmt, &ctx->v_fmt);
 
-	csi->pad.flags = MEDIA_PAD_FL_SINK;

Ah was this re-initializing the same pad multiple times ?

Quoted 38 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +6 -3
+	ctx->pad.flags = MEDIA_PAD_FL_SINK;
 	vdev->entity.ops = &ti_csi2rx_video_entity_ops;
-	ret = media_entity_pads_init(&ctx->vdev.entity, 1, &csi->pad);
+	ret = media_entity_pads_init(&ctx->vdev.entity, 1, &ctx->pad);
 	if (ret)
 		return ret;
 
@@ -1159,6 +1343,8 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
 	if (!csi->drain.vaddr)
 		return -ENOMEM;
 
+	mutex_init(&csi->mutex);
+
 	ret = ti_csi2rx_v4l2_init(csi);
 	if (ret)
 		goto err_v4l2;
@@ -1191,6 +1377,7 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
 		ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
 	ti_csi2rx_cleanup_v4l2(csi);
 err_v4l2:
+	mutex_destroy(&csi->mutex);
 	dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
 			  csi->drain.paddr);
 	return ret;
@@ -1213,7 +1400,7 @@ static void ti_csi2rx_remove(struct platform_device *pdev)
 
 	ti_csi2rx_cleanup_notifier(csi);
 	ti_csi2rx_cleanup_v4l2(csi);
-
+	mutex_destroy(&csi->mutex);
 	dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
 			  csi->drain.paddr);
 }
 
--

2.43.0

author
#21 lore raw

Re: Re: [PATCH v2 06/13] media: ti: j721e-csi2rx: add a subdev for the core device

Quoted 63 lines · Jacopo Mondi
Quoted 62 lines · Jai Luthra

With single stream capture, it was simpler to use the video device as the media entity representing the main TI CSI2RX device. Now with multi stream capture coming into the picture, the model has shifted to each video device having a link to the main device's subdev. The routing would then be set on this subdev.

Add this subdev, link each context to this subdev's entity and link the subdev's entity to the source. Also add an array of media pads. It will have one sink pad and source pads equal to the number of contexts.

---
 .../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c  | 237 ++++++++++++++++++---
 1 file changed, 212 insertions(+), 25 deletions(-)
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +10 -1
index 361b0ea8e0d9..13d7426ab4ba 100644
@@ -51,6 +51,11 @@
 #define MAX_WIDTH_BYTES			SZ_16K
 #define MAX_HEIGHT_LINES		SZ_16K
 
+#define TI_CSI2RX_PAD_SINK		0
+#define TI_CSI2RX_PAD_FIRST_SOURCE	1
+#define TI_CSI2RX_NUM_SOURCE_PADS	1
+#define TI_CSI2RX_NUM_PADS		(1 + TI_CSI2RX_NUM_SOURCE_PADS)
+
 #define DRAIN_TIMEOUT_MS		50
 #define DRAIN_BUFFER_SIZE		SZ_32K
 
@@ -97,6 +102,7 @@ struct ti_csi2rx_ctx {
 	struct mutex			mutex; /* To serialize ioctls. */
 	struct v4l2_format		v_fmt;
 	struct ti_csi2rx_dma		dma;
+	struct media_pad		pad;
 	u32				sequence;
 	u32				idx;
 };
@@ -104,12 +110,15 @@ struct ti_csi2rx_ctx {
 struct ti_csi2rx_dev {
 	struct device			*dev;
 	void __iomem			*shim;
+	struct mutex			mutex; /* To serialize ioctls. */
+	unsigned int			enable_count;
 	struct v4l2_device		v4l2_dev;
 	struct media_device		mdev;
 	struct media_pipeline		pipe;
-	struct media_pad		pad;
+	struct media_pad		pads[TI_CSI2RX_NUM_PADS];
 	struct v4l2_async_notifier	notifier;
 	struct v4l2_subdev		*source;
+	struct v4l2_subdev		subdev;
 	struct ti_csi2rx_ctx		ctx[TI_CSI2RX_NUM_CTX];
 	/* Buffer to drain stale data from PSI-L endpoint */
 	struct {
@@ -431,6 +440,15 @@ static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
 	struct ti_csi2rx_dev *csi = dev_get_drvdata(notifier->v4l2_dev->dev);
 	int ret, i;
 
Quoted 5 lines

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier) · line 444

Quoted 2 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +2 -0
+	/* Create link from source to subdev */
+	ret = v4l2_create_fwnode_links_to_pad(csi->source,

Isn't this a single link from the image source to the RX's sink pad ? Wouldn't media_create_pad_link() do here ?

The source here is always cdns-csi2rx bridge (and not the sensor) which has multiple pads.

If we use media_create_pad_link() here, we would need to figure out the correct source pad using fwnode APIs. v4l2_create_fwnode_links_to_pad() helper already does that for us.

But because of your comment I realized that we are missing .get_fwnode_pad() callbacks in both cdns-csi2rx and this driver.

It is working by sheer luck as media_entity_get_fwnode_pad() returns the first pad as fallback. I will fix this in v3.

Quoted 18 lines
Quoted 18 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +7 -1
+					      &csi->pads[TI_CSI2RX_PAD_SINK],
+					      MEDIA_LNK_FL_IMMUTABLE |
+					      MEDIA_LNK_FL_ENABLED);
+	if (ret)
+		return ret;
+
+	/* Create and link video nodes for all DMA contexts */
 	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
 		struct ti_csi2rx_ctx *ctx = &csi->ctx[i];
 		struct video_device *vdev = &ctx->vdev;
@@ -438,13 +456,17 @@ static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
 		ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
 		if (ret)
 			goto unregister_dev;
-	}
 
Quoted 17 lines

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier) · line 467

Quoted 13 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +8 -5
-	ret = v4l2_create_fwnode_links_to_pad(csi->source, &csi->pad,
-					      MEDIA_LNK_FL_IMMUTABLE |
-					      MEDIA_LNK_FL_ENABLED);
-	if (ret)
-		goto unregister_dev;
+		ret = media_create_pad_link(&csi->subdev.entity,
+					    TI_CSI2RX_PAD_FIRST_SOURCE + ctx->idx,
+					    &vdev->entity, 0,
+					    MEDIA_LNK_FL_IMMUTABLE |
+					    MEDIA_LNK_FL_ENABLED);
+		if (ret) {
+			video_unregister_device(vdev);
+			goto unregister_dev;

Should you call media_entity_remove_links() on the csi2 entity on the error path ?

Will fix.

Quoted 45 lines
Quoted 31 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +8 -2
+		}
+	}
 
 	ret = v4l2_device_register_subdev_nodes(&csi->v4l2_dev);
 	if (ret)
@@ -859,7 +881,7 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 	dma->state = TI_CSI2RX_DMA_ACTIVE;
 	spin_unlock_irqrestore(&dma->lock, flags);
 
-	ret = v4l2_subdev_call(csi->source, video, s_stream, 1);
+	ret = v4l2_subdev_call(&csi->subdev, video, s_stream, 1);
 	if (ret)
 		goto err_dma;
 
@@ -887,7 +909,7 @@ static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
 	writel(0, csi->shim + SHIM_CNTL);
 	writel(0, csi->shim + SHIM_DMACNTX(ctx->idx));
 
-	ret = v4l2_subdev_call(csi->source, video, s_stream, 0);
+	ret = v4l2_subdev_call(&csi->subdev, video, s_stream, 0);
 	if (ret)
 		dev_err(csi->dev, "Failed to stop subdev stream\n");
 
@@ -905,6 +927,119 @@ static const struct vb2_ops csi_vb2_qops = {
 	.wait_finish = vb2_ops_wait_finish,
 };
 
+static inline struct ti_csi2rx_dev *to_csi2rx_dev(struct v4l2_subdev *sd)
+{
+	return container_of(sd, struct ti_csi2rx_dev, subdev);
+}

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct vb2_ops csi_vb2_qops = { · line 943

Quoted 10 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +10 -0
+
+static int ti_csi2rx_sd_set_fmt(struct v4l2_subdev *sd,
+				struct v4l2_subdev_state *state,
+				struct v4l2_subdev_format *format)
+{
+	struct v4l2_mbus_framefmt *fmt;
+	int ret = 0;
+
+	/* No transcoding, don't allow setting source fmt */
+	if (format->pad >= TI_CSI2RX_PAD_FIRST_SOURCE)

> TI_CSI2RX_PAD_SINK

Quoted 1 line

might be clearer

Will fix.

Quoted 9 lines

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct vb2_ops csi_vb2_qops = { · line 949

Quoted 7 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +6 -0
+		return v4l2_subdev_get_fmt(sd, state, format);
+
+	if (!find_format_by_code(format->format.code))
+		format->format.code = ti_csi2rx_formats[0].code;
+
+	format->format.field = V4L2_FIELD_NONE;

What about other colorspace related fields ?

The HW does not care about colorspace/encoding etc. and acts as passthrough, so we accept whatever userspace sets for those.

Quoted 6 lines

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct vb2_ops csi_vb2_qops = { · line 952

Quoted 4 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +3 -0
+
+	fmt = v4l2_subdev_state_get_format(state, format->pad, format->stream);
+	if (!fmt) {

Can this happen ?

Yes __v4l2_subdev_state_get_format() may return NULL if userspace tried to set format with an invalid stream (i.e. no routes exist for that stream)

Quoted 13 lines

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct vb2_ops csi_vb2_qops = { · line 961

Quoted 10 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +9 -0
+		ret = -EINVAL;
+		goto out;
+	}
+	*fmt = format->format;
+
+	fmt = v4l2_subdev_state_get_format(state, TI_CSI2RX_PAD_FIRST_SOURCE,
+					   format->stream);
+	if (!fmt) {
+		ret = -EINVAL;

ditto

Quoted 4 lines

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct vb2_ops csi_vb2_qops = { · line 962

Quoted 1 line
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +1 -0
+		goto out;

kind of missing the point of a jump here where you can simply return -EINVAL

Will fix.

Quoted 26 lines
Quoted 26 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +25 -0
+	}
+	*fmt = format->format;
+
+out:
+	return ret;
+}
+
+static int ti_csi2rx_sd_init_state(struct v4l2_subdev *sd,
+				   struct v4l2_subdev_state *state)
+{
+	struct v4l2_subdev_format format = {
+		.pad = TI_CSI2RX_PAD_SINK,
+		.format = {
+			.width = 640,
+			.height = 480,
+			.code = MEDIA_BUS_FMT_UYVY8_1X16,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.ycbcr_enc = V4L2_YCBCR_ENC_601,
+			.quantization = V4L2_QUANTIZATION_LIM_RANGE,
+			.xfer_func = V4L2_XFER_FUNC_SRGB,
+		},
+	};
+
+	return ti_csi2rx_sd_set_fmt(sd, state, &format);
Quoted 8 lines

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct vb2_ops csi_vb2_qops = { · line 990

Quoted 3 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +3 -0
+}
+
+static int ti_csi2rx_sd_s_stream(struct v4l2_subdev *sd, int enable)

Even if I never used that API myself, but I have a feeling enable_streams() (which is meant to replace s_stream for multiplexed subdev) would help you avoiding the manual enable_count booking here below ?

In PATCH 12/13 we switch to using enable_streams() APIs here.

We still keep this enable_count bookkeeping, but I agree it can be dropped with minor changes. Will fix in v3.

Quoted 78 lines
Quoted 65 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +53 -0
+{
+	struct ti_csi2rx_dev *csi = to_csi2rx_dev(sd);
+	int ret = 0;
+
+	mutex_lock(&csi->mutex);
+
+	if (enable) {
+		if (csi->enable_count > 0) {
+			csi->enable_count++;
+			goto out;
+		}
+
+		ret = v4l2_subdev_call(csi->source, video, s_stream, 1);
+		if (ret)
+			goto out;
+
+		csi->enable_count++;
+	} else {
+		if (csi->enable_count == 0) {
+			ret = -EINVAL;
+			goto out;
+		}
+
+		if (--csi->enable_count > 0)
+			goto out;
+
+		ret = v4l2_subdev_call(csi->source, video, s_stream, 0);
+	}
+
+out:
+	mutex_unlock(&csi->mutex);
+	return ret;
+}
+
+static const struct v4l2_subdev_pad_ops ti_csi2rx_subdev_pad_ops = {
+	.get_fmt = v4l2_subdev_get_fmt,
+	.set_fmt = ti_csi2rx_sd_set_fmt,
+};
+
+static const struct v4l2_subdev_video_ops ti_csi2rx_subdev_video_ops = {
+	.s_stream = ti_csi2rx_sd_s_stream,
+};
+
+static const struct v4l2_subdev_ops ti_csi2rx_subdev_ops = {
+	.video = &ti_csi2rx_subdev_video_ops,
+	.pad = &ti_csi2rx_subdev_pad_ops,
+};
+
+static const struct v4l2_subdev_internal_ops ti_csi2rx_internal_ops = {
+	.init_state = ti_csi2rx_sd_init_state,
+};
+
 static void ti_csi2rx_cleanup_dma(struct ti_csi2rx_ctx *ctx)
 {
 	dma_release_channel(ctx->dma.chan);
@@ -912,6 +1047,7 @@ static void ti_csi2rx_cleanup_dma(struct ti_csi2rx_ctx *ctx)
 
 static void ti_csi2rx_cleanup_v4l2(struct ti_csi2rx_dev *csi)
 {
+	v4l2_subdev_cleanup(&csi->subdev);
 	media_device_unregister(&csi->mdev);
 	v4l2_device_unregister(&csi->v4l2_dev);
 	media_device_cleanup(&csi->mdev);

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int ti_csi2rx_link_validate(struct media_link *link) · line 1112

Quoted 4 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +1 -0
@@ -973,14 +1109,22 @@ static int ti_csi2rx_link_validate(struct media_link *link)
 	struct v4l2_subdev_format source_fmt = {
 		.which	= V4L2_SUBDEV_FORMAT_ACTIVE,
 		.pad	= link->source->index,
+		.stream = 0,

I don't think it hurts, but this seems not to be strictly related to this patch ?

Quoted 5 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +1 -0
 	};
+	struct v4l2_subdev_state *state;
 	const struct ti_csi2rx_fmt *ti_fmt;
 	int ret;
 
Quoted 16 lines

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int ti_csi2rx_link_validate(struct media_link *link) · line 1125

Quoted 12 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +8 -4
-	ret = v4l2_subdev_call_state_active(csi->source, pad,
-					    get_fmt, &source_fmt);
-	if (ret)
-		return ret;
+	state = v4l2_subdev_lock_and_get_active_state(&csi->subdev);
+	ret = v4l2_subdev_call(&csi->subdev, pad, get_fmt, state, &source_fmt);
+	v4l2_subdev_unlock_state(state);
+
+	if (ret) {
+		dev_dbg(csi->dev,
+			"Skipping validation as no format present on \"%s\":%u:0\n",
+			link->source->entity->name, link->source->index);

If get_fmt returns an error, should you simply continue or fail ? In which cases get_fmt on a subdev fails ? Only if the pad or the stream are not valid right ?

The subdev introduced in this patch has multiple source pads, all actively linked to separate video nodes, one for each DMA channel.

We continue (return 0) here as we don't want __media_pipeline_start to fail because of pads that are "unused" i.e. have no routes and no formats set.

If the user actually routes a stream to any pad the pipeline validation will check that a format is also set (and thus the link will get validated here).

I wonder if we can keep the links mutable, and expect userspace to disable links to unused video nodes. Will have to think more about that approach.

Quoted 60 lines
Quoted 60 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +20 -9
+		return 0;
+	}
 
 	if (source_fmt.format.width != csi_fmt->width) {
 		dev_dbg(csi->dev, "Width does not match (source %u, sink %u)\n",
@@ -1010,8 +1154,9 @@ static int ti_csi2rx_link_validate(struct media_link *link)
 
 	if (ti_fmt->fourcc != csi_fmt->pixelformat) {
 		dev_dbg(csi->dev,
-			"Cannot transform source fmt 0x%x to sink fmt 0x%x\n",
-			ti_fmt->fourcc, csi_fmt->pixelformat);
+			"Cannot transform \"%s\":%u format %p4cc to %p4cc\n",
+			link->source->entity->name, link->source->index,
+			&ti_fmt->fourcc, &csi_fmt->pixelformat);
 		return -EPIPE;
 	}
 
@@ -1022,6 +1167,10 @@ static const struct media_entity_operations ti_csi2rx_video_entity_ops = {
 	.link_validate = ti_csi2rx_link_validate,
 };
 
+static const struct media_entity_operations ti_csi2rx_subdev_entity_ops = {
+	.link_validate = v4l2_subdev_link_validate,
+};
+
 static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
 {
 	struct dma_slave_config cfg = {
@@ -1053,7 +1202,8 @@ static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
 static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
 {
 	struct media_device *mdev = &csi->mdev;
-	int ret;
+	struct v4l2_subdev *sd = &csi->subdev;
+	int ret, i;
 
 	mdev->dev = csi->dev;
 	mdev->hw_revision = 1;
@@ -1065,16 +1215,50 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
 
 	ret = v4l2_device_register(csi->dev, &csi->v4l2_dev);
 	if (ret)
-		return ret;
+		goto cleanup_media;
 
 	ret = media_device_register(mdev);
-	if (ret) {
-		v4l2_device_unregister(&csi->v4l2_dev);
-		media_device_cleanup(mdev);
-		return ret;
-	}
+	if (ret)
+		goto unregister_v4l2;
+
+	v4l2_subdev_init(sd, &ti_csi2rx_subdev_ops);
+	sd->internal_ops = &ti_csi2rx_internal_ops;
+	sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
+	sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
+	strscpy(sd->name, dev_name(csi->dev), sizeof(sd->name));
Quoted 8 lines

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi) · line 1234

Quoted 6 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +6 -0
+	sd->dev = csi->dev;
+	sd->entity.ops = &ti_csi2rx_subdev_entity_ops;
+
+	csi->pads[TI_CSI2RX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+
+	for (i = TI_CSI2RX_PAD_FIRST_SOURCE; i < TI_CSI2RX_NUM_PADS; i++)
        for (unsigned int i = 0;

Will fix.

Quoted 30 lines
Quoted 30 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +25 -0
+		csi->pads[i].flags = MEDIA_PAD_FL_SOURCE;
+
+	ret = media_entity_pads_init(&sd->entity, ARRAY_SIZE(csi->pads),
+				     csi->pads);
+	if (ret)
+		goto unregister_media;
+
+	ret = v4l2_subdev_init_finalize(sd);
+	if (ret)
+		goto unregister_media;
+
+	ret = v4l2_device_register_subdev(&csi->v4l2_dev, sd);
+	if (ret)
+		goto cleanup_subdev;
 
 	return 0;
+
+cleanup_subdev:
+	v4l2_subdev_cleanup(sd);
+unregister_media:
+	media_device_unregister(mdev);
+unregister_v4l2:
+	v4l2_device_unregister(&csi->v4l2_dev);
+cleanup_media:
+	media_device_cleanup(mdev);
+
+	return ret;
 }
 
Quoted 8 lines

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx) · line 1104

Quoted 6 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +0 -1
 static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
@@ -1101,9 +1285,9 @@ static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
 
 	ti_csi2rx_fill_fmt(fmt, &ctx->v_fmt);
 
-	csi->pad.flags = MEDIA_PAD_FL_SINK;

Ah was this re-initializing the same pad multiple times ?

Not sure I understand the comment fully. We have only 1 video context here, more contexts will be introduced in subsequent patches.

Quoted 39 lines
Quoted 39 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +6 -3
+	ctx->pad.flags = MEDIA_PAD_FL_SINK;
 	vdev->entity.ops = &ti_csi2rx_video_entity_ops;
-	ret = media_entity_pads_init(&ctx->vdev.entity, 1, &csi->pad);
+	ret = media_entity_pads_init(&ctx->vdev.entity, 1, &ctx->pad);
 	if (ret)
 		return ret;
 
@@ -1159,6 +1343,8 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
 	if (!csi->drain.vaddr)
 		return -ENOMEM;
 
+	mutex_init(&csi->mutex);
+
 	ret = ti_csi2rx_v4l2_init(csi);
 	if (ret)
 		goto err_v4l2;
@@ -1191,6 +1377,7 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
 		ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
 	ti_csi2rx_cleanup_v4l2(csi);
 err_v4l2:
+	mutex_destroy(&csi->mutex);
 	dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
 			  csi->drain.paddr);
 	return ret;
@@ -1213,7 +1400,7 @@ static void ti_csi2rx_remove(struct platform_device *pdev)
 
 	ti_csi2rx_cleanup_notifier(csi);
 	ti_csi2rx_cleanup_v4l2(csi);
-
+	mutex_destroy(&csi->mutex);
 	dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
 			  csi->drain.paddr);
 }
 
--

2.43.0

Signature · 4 lines
Thanks,
Jai

GPG Fingerprint: 4DE0 D818 E5D5 75E8 D45A AFC5 43DE 91F9 249A 7145
#22 lore raw

Re: [PATCH v2 06/13] media: ti: j721e-csi2rx: add a subdev for the core device

Hi Jai
+ Laurent + Sakari for a question below

Quoted 77 lines · Jai Luthra
Quoted 63 lines · Jacopo Mondi
Quoted 62 lines · Jai Luthra

With single stream capture, it was simpler to use the video device as the media entity representing the main TI CSI2RX device. Now with multi stream capture coming into the picture, the model has shifted to each video device having a link to the main device's subdev. The routing would then be set on this subdev.

Add this subdev, link each context to this subdev's entity and link the subdev's entity to the source. Also add an array of media pads. It will have one sink pad and source pads equal to the number of contexts.

---
 .../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c  | 237 ++++++++++++++++++---
 1 file changed, 212 insertions(+), 25 deletions(-)
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +10 -1
index 361b0ea8e0d9..13d7426ab4ba 100644
@@ -51,6 +51,11 @@
 #define MAX_WIDTH_BYTES			SZ_16K
 #define MAX_HEIGHT_LINES		SZ_16K
 
+#define TI_CSI2RX_PAD_SINK		0
+#define TI_CSI2RX_PAD_FIRST_SOURCE	1
+#define TI_CSI2RX_NUM_SOURCE_PADS	1
+#define TI_CSI2RX_NUM_PADS		(1 + TI_CSI2RX_NUM_SOURCE_PADS)
+
 #define DRAIN_TIMEOUT_MS		50
 #define DRAIN_BUFFER_SIZE		SZ_32K
 
@@ -97,6 +102,7 @@ struct ti_csi2rx_ctx {
 	struct mutex			mutex; /* To serialize ioctls. */
 	struct v4l2_format		v_fmt;
 	struct ti_csi2rx_dma		dma;
+	struct media_pad		pad;
 	u32				sequence;
 	u32				idx;
 };
@@ -104,12 +110,15 @@ struct ti_csi2rx_ctx {
 struct ti_csi2rx_dev {
 	struct device			*dev;
 	void __iomem			*shim;
+	struct mutex			mutex; /* To serialize ioctls. */
+	unsigned int			enable_count;
 	struct v4l2_device		v4l2_dev;
 	struct media_device		mdev;
 	struct media_pipeline		pipe;
-	struct media_pad		pad;
+	struct media_pad		pads[TI_CSI2RX_NUM_PADS];
 	struct v4l2_async_notifier	notifier;
 	struct v4l2_subdev		*source;
+	struct v4l2_subdev		subdev;
 	struct ti_csi2rx_ctx		ctx[TI_CSI2RX_NUM_CTX];
 	/* Buffer to drain stale data from PSI-L endpoint */
 	struct {
@@ -431,6 +440,15 @@ static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
 	struct ti_csi2rx_dev *csi = dev_get_drvdata(notifier->v4l2_dev->dev);
 	int ret, i;
 
Quoted 5 lines

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier) · line 444

Quoted 2 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +2 -0
+	/* Create link from source to subdev */
+	ret = v4l2_create_fwnode_links_to_pad(csi->source,

Isn't this a single link from the image source to the RX's sink pad ? Wouldn't media_create_pad_link() do here ?

The source here is always cdns-csi2rx bridge (and not the sensor) which has multiple pads.

If we use media_create_pad_link() here, we would need to figure out the correct source pad using fwnode APIs. v4l2_create_fwnode_links_to_pad() helper already does that for us.

Quoted 6 lines

But because of your comment I realized that we are missing .get_fwnode_pad() callbacks in both cdns-csi2rx and this driver.

It is working by sheer luck as media_entity_get_fwnode_pad() returns the first pad as fallback. I will fix this in v3.

Ack

Quoted 97 lines
Quoted 18 lines
Quoted 18 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +7 -1
+					      &csi->pads[TI_CSI2RX_PAD_SINK],
+					      MEDIA_LNK_FL_IMMUTABLE |
+					      MEDIA_LNK_FL_ENABLED);
+	if (ret)
+		return ret;
+
+	/* Create and link video nodes for all DMA contexts */
 	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
 		struct ti_csi2rx_ctx *ctx = &csi->ctx[i];
 		struct video_device *vdev = &ctx->vdev;
@@ -438,13 +456,17 @@ static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
 		ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
 		if (ret)
 			goto unregister_dev;
-	}
 
Quoted 17 lines

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier) · line 467

Quoted 13 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +8 -5
-	ret = v4l2_create_fwnode_links_to_pad(csi->source, &csi->pad,
-					      MEDIA_LNK_FL_IMMUTABLE |
-					      MEDIA_LNK_FL_ENABLED);
-	if (ret)
-		goto unregister_dev;
+		ret = media_create_pad_link(&csi->subdev.entity,
+					    TI_CSI2RX_PAD_FIRST_SOURCE + ctx->idx,
+					    &vdev->entity, 0,
+					    MEDIA_LNK_FL_IMMUTABLE |
+					    MEDIA_LNK_FL_ENABLED);
+		if (ret) {
+			video_unregister_device(vdev);
+			goto unregister_dev;

Should you call media_entity_remove_links() on the csi2 entity on the error path ?

Will fix.

Quoted 45 lines
Quoted 31 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +8 -2
+		}
+	}
 
 	ret = v4l2_device_register_subdev_nodes(&csi->v4l2_dev);
 	if (ret)
@@ -859,7 +881,7 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 	dma->state = TI_CSI2RX_DMA_ACTIVE;
 	spin_unlock_irqrestore(&dma->lock, flags);
 
-	ret = v4l2_subdev_call(csi->source, video, s_stream, 1);
+	ret = v4l2_subdev_call(&csi->subdev, video, s_stream, 1);
 	if (ret)
 		goto err_dma;
 
@@ -887,7 +909,7 @@ static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
 	writel(0, csi->shim + SHIM_CNTL);
 	writel(0, csi->shim + SHIM_DMACNTX(ctx->idx));
 
-	ret = v4l2_subdev_call(csi->source, video, s_stream, 0);
+	ret = v4l2_subdev_call(&csi->subdev, video, s_stream, 0);
 	if (ret)
 		dev_err(csi->dev, "Failed to stop subdev stream\n");
 
@@ -905,6 +927,119 @@ static const struct vb2_ops csi_vb2_qops = {
 	.wait_finish = vb2_ops_wait_finish,
 };
 
+static inline struct ti_csi2rx_dev *to_csi2rx_dev(struct v4l2_subdev *sd)
+{
+	return container_of(sd, struct ti_csi2rx_dev, subdev);
+}

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct vb2_ops csi_vb2_qops = { · line 943

Quoted 10 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +10 -0
+
+static int ti_csi2rx_sd_set_fmt(struct v4l2_subdev *sd,
+				struct v4l2_subdev_state *state,
+				struct v4l2_subdev_format *format)
+{
+	struct v4l2_mbus_framefmt *fmt;
+	int ret = 0;
+
+	/* No transcoding, don't allow setting source fmt */
+	if (format->pad >= TI_CSI2RX_PAD_FIRST_SOURCE)

> TI_CSI2RX_PAD_SINK

Quoted 1 line

might be clearer

Will fix.

Quoted 9 lines

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct vb2_ops csi_vb2_qops = { · line 949

Quoted 7 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +6 -0
+		return v4l2_subdev_get_fmt(sd, state, format);
+
+	if (!find_format_by_code(format->format.code))
+		format->format.code = ti_csi2rx_formats[0].code;
+
+	format->format.field = V4L2_FIELD_NONE;

What about other colorspace related fields ?

Quoted 3 lines

The HW does not care about colorspace/encoding etc. and acts as passthrough, so we accept whatever userspace sets for those.

Or should it rather propagate to userspace what is set by the downstream entity ?

Quoted 11 lines
Quoted 6 lines

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct vb2_ops csi_vb2_qops = { · line 952

Quoted 4 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +3 -0
+
+	fmt = v4l2_subdev_state_get_format(state, format->pad, format->stream);
+	if (!fmt) {

Can this happen ?

Yes __v4l2_subdev_state_get_format() may return NULL if userspace tried to set format with an invalid stream (i.e. no routes exist for that stream)

The core makes sure the stream is valid afaict

See check_state() in v4l2-subdev.c

Quoted 55 lines
Quoted 13 lines

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct vb2_ops csi_vb2_qops = { · line 961

Quoted 10 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +9 -0
+		ret = -EINVAL;
+		goto out;
+	}
+	*fmt = format->format;
+
+	fmt = v4l2_subdev_state_get_format(state, TI_CSI2RX_PAD_FIRST_SOURCE,
+					   format->stream);
+	if (!fmt) {
+		ret = -EINVAL;

ditto

Quoted 4 lines

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct vb2_ops csi_vb2_qops = { · line 962

Quoted 1 line
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +1 -0
+		goto out;

kind of missing the point of a jump here where you can simply return -EINVAL

Will fix.

Quoted 26 lines
Quoted 26 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +25 -0
+	}
+	*fmt = format->format;
+
+out:
+	return ret;
+}
+
+static int ti_csi2rx_sd_init_state(struct v4l2_subdev *sd,
+				   struct v4l2_subdev_state *state)
+{
+	struct v4l2_subdev_format format = {
+		.pad = TI_CSI2RX_PAD_SINK,
+		.format = {
+			.width = 640,
+			.height = 480,
+			.code = MEDIA_BUS_FMT_UYVY8_1X16,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.ycbcr_enc = V4L2_YCBCR_ENC_601,
+			.quantization = V4L2_QUANTIZATION_LIM_RANGE,
+			.xfer_func = V4L2_XFER_FUNC_SRGB,
+		},
+	};
+
+	return ti_csi2rx_sd_set_fmt(sd, state, &format);
Quoted 8 lines

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static const struct vb2_ops csi_vb2_qops = { · line 990

Quoted 3 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +3 -0
+}
+
+static int ti_csi2rx_sd_s_stream(struct v4l2_subdev *sd, int enable)

Even if I never used that API myself, but I have a feeling enable_streams() (which is meant to replace s_stream for multiplexed subdev) would help you avoiding the manual enable_count booking here below ?

Quoted 5 lines

In PATCH 12/13 we switch to using enable_streams() APIs here.

We still keep this enable_count bookkeeping, but I agree it can be dropped with minor changes. Will fix in v3.

Thanks, if it gets troublesome to remove it and you later switch to the new API then don't bother

Quoted 102 lines
Quoted 78 lines
Quoted 65 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +53 -0
+{
+	struct ti_csi2rx_dev *csi = to_csi2rx_dev(sd);
+	int ret = 0;
+
+	mutex_lock(&csi->mutex);
+
+	if (enable) {
+		if (csi->enable_count > 0) {
+			csi->enable_count++;
+			goto out;
+		}
+
+		ret = v4l2_subdev_call(csi->source, video, s_stream, 1);
+		if (ret)
+			goto out;
+
+		csi->enable_count++;
+	} else {
+		if (csi->enable_count == 0) {
+			ret = -EINVAL;
+			goto out;
+		}
+
+		if (--csi->enable_count > 0)
+			goto out;
+
+		ret = v4l2_subdev_call(csi->source, video, s_stream, 0);
+	}
+
+out:
+	mutex_unlock(&csi->mutex);
+	return ret;
+}
+
+static const struct v4l2_subdev_pad_ops ti_csi2rx_subdev_pad_ops = {
+	.get_fmt = v4l2_subdev_get_fmt,
+	.set_fmt = ti_csi2rx_sd_set_fmt,
+};
+
+static const struct v4l2_subdev_video_ops ti_csi2rx_subdev_video_ops = {
+	.s_stream = ti_csi2rx_sd_s_stream,
+};
+
+static const struct v4l2_subdev_ops ti_csi2rx_subdev_ops = {
+	.video = &ti_csi2rx_subdev_video_ops,
+	.pad = &ti_csi2rx_subdev_pad_ops,
+};
+
+static const struct v4l2_subdev_internal_ops ti_csi2rx_internal_ops = {
+	.init_state = ti_csi2rx_sd_init_state,
+};
+
 static void ti_csi2rx_cleanup_dma(struct ti_csi2rx_ctx *ctx)
 {
 	dma_release_channel(ctx->dma.chan);
@@ -912,6 +1047,7 @@ static void ti_csi2rx_cleanup_dma(struct ti_csi2rx_ctx *ctx)
 
 static void ti_csi2rx_cleanup_v4l2(struct ti_csi2rx_dev *csi)
 {
+	v4l2_subdev_cleanup(&csi->subdev);
 	media_device_unregister(&csi->mdev);
 	v4l2_device_unregister(&csi->v4l2_dev);
 	media_device_cleanup(&csi->mdev);

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int ti_csi2rx_link_validate(struct media_link *link) · line 1112

Quoted 4 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +1 -0
@@ -973,14 +1109,22 @@ static int ti_csi2rx_link_validate(struct media_link *link)
 	struct v4l2_subdev_format source_fmt = {
 		.which	= V4L2_SUBDEV_FORMAT_ACTIVE,
 		.pad	= link->source->index,
+		.stream = 0,

I don't think it hurts, but this seems not to be strictly related to this patch ?

Quoted 5 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +1 -0
 	};
+	struct v4l2_subdev_state *state;
 	const struct ti_csi2rx_fmt *ti_fmt;
 	int ret;
 
Quoted 16 lines

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int ti_csi2rx_link_validate(struct media_link *link) · line 1125

Quoted 12 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +8 -4
-	ret = v4l2_subdev_call_state_active(csi->source, pad,
-					    get_fmt, &source_fmt);
-	if (ret)
-		return ret;
+	state = v4l2_subdev_lock_and_get_active_state(&csi->subdev);
+	ret = v4l2_subdev_call(&csi->subdev, pad, get_fmt, state, &source_fmt);
+	v4l2_subdev_unlock_state(state);
+
+	if (ret) {
+		dev_dbg(csi->dev,
+			"Skipping validation as no format present on \"%s\":%u:0\n",
+			link->source->entity->name, link->source->index);

If get_fmt returns an error, should you simply continue or fail ? In which cases get_fmt on a subdev fails ? Only if the pad or the stream are not valid right ?

The subdev introduced in this patch has multiple source pads, all actively linked to separate video nodes, one for each DMA channel.

We continue (return 0) here as we don't want __media_pipeline_start to fail because of pads that are "unused" i.e. have no routes and no formats set.

Quoted 8 lines

If the user actually routes a stream to any pad the pipeline validation will check that a format is also set (and thus the link will get validated here).

I wonder if we can keep the links mutable, and expect userspace to disable links to unused video nodes. Will have to think more about that approach.

We recently had a discussion about this (link enablement for 'active' dma paths) with Sakari and Laurent on the Mali-C55 ISP. Let's see if they have feedbacks.

Quoted 104 lines
Quoted 60 lines
Quoted 60 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +20 -9
+		return 0;
+	}
 
 	if (source_fmt.format.width != csi_fmt->width) {
 		dev_dbg(csi->dev, "Width does not match (source %u, sink %u)\n",
@@ -1010,8 +1154,9 @@ static int ti_csi2rx_link_validate(struct media_link *link)
 
 	if (ti_fmt->fourcc != csi_fmt->pixelformat) {
 		dev_dbg(csi->dev,
-			"Cannot transform source fmt 0x%x to sink fmt 0x%x\n",
-			ti_fmt->fourcc, csi_fmt->pixelformat);
+			"Cannot transform \"%s\":%u format %p4cc to %p4cc\n",
+			link->source->entity->name, link->source->index,
+			&ti_fmt->fourcc, &csi_fmt->pixelformat);
 		return -EPIPE;
 	}
 
@@ -1022,6 +1167,10 @@ static const struct media_entity_operations ti_csi2rx_video_entity_ops = {
 	.link_validate = ti_csi2rx_link_validate,
 };
 
+static const struct media_entity_operations ti_csi2rx_subdev_entity_ops = {
+	.link_validate = v4l2_subdev_link_validate,
+};
+
 static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
 {
 	struct dma_slave_config cfg = {
@@ -1053,7 +1202,8 @@ static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
 static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
 {
 	struct media_device *mdev = &csi->mdev;
-	int ret;
+	struct v4l2_subdev *sd = &csi->subdev;
+	int ret, i;
 
 	mdev->dev = csi->dev;
 	mdev->hw_revision = 1;
@@ -1065,16 +1215,50 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
 
 	ret = v4l2_device_register(csi->dev, &csi->v4l2_dev);
 	if (ret)
-		return ret;
+		goto cleanup_media;
 
 	ret = media_device_register(mdev);
-	if (ret) {
-		v4l2_device_unregister(&csi->v4l2_dev);
-		media_device_cleanup(mdev);
-		return ret;
-	}
+	if (ret)
+		goto unregister_v4l2;
+
+	v4l2_subdev_init(sd, &ti_csi2rx_subdev_ops);
+	sd->internal_ops = &ti_csi2rx_internal_ops;
+	sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
+	sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
+	strscpy(sd->name, dev_name(csi->dev), sizeof(sd->name));
Quoted 8 lines

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi) · line 1234

Quoted 6 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +6 -0
+	sd->dev = csi->dev;
+	sd->entity.ops = &ti_csi2rx_subdev_entity_ops;
+
+	csi->pads[TI_CSI2RX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+
+	for (i = TI_CSI2RX_PAD_FIRST_SOURCE; i < TI_CSI2RX_NUM_PADS; i++)
        for (unsigned int i = 0;

Will fix.

Quoted 33 lines
Quoted 33 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +25 -0
+		csi->pads[i].flags = MEDIA_PAD_FL_SOURCE;
+
+	ret = media_entity_pads_init(&sd->entity, ARRAY_SIZE(csi->pads),
+				     csi->pads);
+	if (ret)
+		goto unregister_media;
+
+	ret = v4l2_subdev_init_finalize(sd);
+	if (ret)
+		goto unregister_media;
+
+	ret = v4l2_device_register_subdev(&csi->v4l2_dev, sd);
+	if (ret)
+		goto cleanup_subdev;
 
 	return 0;
+
+cleanup_subdev:
+	v4l2_subdev_cleanup(sd);
+unregister_media:
+	media_device_unregister(mdev);
+unregister_v4l2:
+	v4l2_device_unregister(&csi->v4l2_dev);
+cleanup_media:
+	media_device_cleanup(mdev);
+
+	return ret;
 }
 
 static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
@@ -1101,9 +1285,9 @@ static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
 
Quoted 8 lines
Quoted 5 lines

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx) · line 1104

Quoted 3 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +0 -1
 	ti_csi2rx_fill_fmt(fmt, &ctx->v_fmt);
 
-	csi->pad.flags = MEDIA_PAD_FL_SINK;

Ah was this re-initializing the same pad multiple times ?

Not sure I understand the comment fully. We have only 1 video context here, more contexts will be introduced in subsequent patches.

Yeah, I meant that before thsi patch there was a single csi->pad which I thought was re-initialized multiple times. But before this patch you had no contexts, so no re-initialization

Quoted 46 lines
Quoted 36 lines
Quoted 36 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +6 -2
+	ctx->pad.flags = MEDIA_PAD_FL_SINK;
 	vdev->entity.ops = &ti_csi2rx_video_entity_ops;
-	ret = media_entity_pads_init(&ctx->vdev.entity, 1, &csi->pad);
+	ret = media_entity_pads_init(&ctx->vdev.entity, 1, &ctx->pad);
 	if (ret)
 		return ret;
 
@@ -1159,6 +1343,8 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
 	if (!csi->drain.vaddr)
 		return -ENOMEM;
 
+	mutex_init(&csi->mutex);
+
 	ret = ti_csi2rx_v4l2_init(csi);
 	if (ret)
 		goto err_v4l2;
@@ -1191,6 +1377,7 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
 		ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
 	ti_csi2rx_cleanup_v4l2(csi);
 err_v4l2:
+	mutex_destroy(&csi->mutex);
 	dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
 			  csi->drain.paddr);
 	return ret;
@@ -1213,7 +1400,7 @@ static void ti_csi2rx_remove(struct platform_device *pdev)
 
 	ti_csi2rx_cleanup_notifier(csi);
 	ti_csi2rx_cleanup_v4l2(csi);
-
+	mutex_destroy(&csi->mutex);
 	dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
 			  csi->drain.paddr);
 }
Quoted 4 lines
Quoted 4 lines

--
2.43.0

--
Thanks,
Jai

GPG Fingerprint: 4DE0 D818 E5D5 75E8 D45A AFC5 43DE 91F9 249A 7145

author
#23 lore raw

PATCH v2 7/13 media: ti: j721e-csi2rx: get number of contexts from device tree

In reply to Jai Luthra

From: Pratyush Yadav <[email protected]>

Different platforms that use this driver might have different number of DMA channels allocated for CSI. So only as many DMA contexts can be used as the number of DMA channels available. Get the number of channels provided via device tree and only configure that many contexts, and hence only that many pads.

---
 .../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c  | 44 ++++++++++++++++------
 1 file changed, 32 insertions(+), 12 deletions(-)
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +32 -12
index 13d7426ab4ba..b4b4bb69c88a 100644
@@ -40,7 +40,7 @@
 #define SHIM_PSI_CFG0_DST_TAG		GENMASK(31, 16)
 
 #define PSIL_WORD_SIZE_BYTES		16
-#define TI_CSI2RX_NUM_CTX		1
+#define TI_CSI2RX_MAX_CTX		32
 
 /*
  * There are no hard limits on the width or height. The DMA engine can handle
@@ -53,8 +53,8 @@
 
 #define TI_CSI2RX_PAD_SINK		0
 #define TI_CSI2RX_PAD_FIRST_SOURCE	1
-#define TI_CSI2RX_NUM_SOURCE_PADS	1
-#define TI_CSI2RX_NUM_PADS		(1 + TI_CSI2RX_NUM_SOURCE_PADS)
+#define TI_CSI2RX_MAX_SOURCE_PADS	TI_CSI2RX_MAX_CTX
+#define TI_CSI2RX_MAX_PADS		(1 + TI_CSI2RX_MAX_SOURCE_PADS)
 
 #define DRAIN_TIMEOUT_MS		50
 #define DRAIN_BUFFER_SIZE		SZ_32K
@@ -112,14 +112,15 @@ struct ti_csi2rx_dev {
 	void __iomem			*shim;
 	struct mutex			mutex; /* To serialize ioctls. */
 	unsigned int			enable_count;
+	unsigned int			num_ctx;
 	struct v4l2_device		v4l2_dev;
 	struct media_device		mdev;
 	struct media_pipeline		pipe;
-	struct media_pad		pads[TI_CSI2RX_NUM_PADS];
+	struct media_pad		pads[TI_CSI2RX_MAX_PADS];
 	struct v4l2_async_notifier	notifier;
 	struct v4l2_subdev		*source;
 	struct v4l2_subdev		subdev;
-	struct ti_csi2rx_ctx		ctx[TI_CSI2RX_NUM_CTX];
+	struct ti_csi2rx_ctx		ctx[TI_CSI2RX_MAX_CTX];
 	/* Buffer to drain stale data from PSI-L endpoint */
 	struct {
 		void			*vaddr;
@@ -449,7 +450,7 @@ static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
 		return ret;
 
 	/* Create and link video nodes for all DMA contexts */
-	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
+	for (i = 0; i < csi->num_ctx; i++) {
 		struct ti_csi2rx_ctx *ctx = &csi->ctx[i];
 		struct video_device *vdev = &ctx->vdev;
 
@@ -1231,10 +1232,12 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
 
 	csi->pads[TI_CSI2RX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
 
-	for (i = TI_CSI2RX_PAD_FIRST_SOURCE; i < TI_CSI2RX_NUM_PADS; i++)
+	for (i = TI_CSI2RX_PAD_FIRST_SOURCE;
+	     i < TI_CSI2RX_PAD_FIRST_SOURCE + csi->num_ctx; i++)
 		csi->pads[i].flags = MEDIA_PAD_FL_SOURCE;
 
-	ret = media_entity_pads_init(&sd->entity, ARRAY_SIZE(csi->pads),
+	ret = media_entity_pads_init(&sd->entity,
+				     TI_CSI2RX_PAD_FIRST_SOURCE + csi->num_ctx,
 				     csi->pads);
 	if (ret)
 		goto unregister_media;
@@ -1320,8 +1323,9 @@ static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
 
 static int ti_csi2rx_probe(struct platform_device *pdev)
 {
+	struct device_node *np = pdev->dev.of_node;
 	struct ti_csi2rx_dev *csi;
-	int ret, i;
+	int ret, i, count;
 
 	csi = devm_kzalloc(&pdev->dev, sizeof(*csi), GFP_KERNEL);
 	if (!csi)
@@ -1343,13 +1347,29 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
 	if (!csi->drain.vaddr)
 		return -ENOMEM;
 
+	/* Only use as many contexts as the number of DMA channels allocated. */
+	count = of_property_count_strings(np, "dma-names");
+	if (count < 0) {
+		dev_err(csi->dev, "Failed to get DMA channel count: %d\n",
+			count);
+		return count;
+	}
+
+	csi->num_ctx = count;
+	if (csi->num_ctx > TI_CSI2RX_MAX_CTX) {
+		dev_warn(csi->dev,
+			 "%u DMA channels passed. Maximum is %u. Ignoring the rest.\n",
+			 csi->num_ctx, TI_CSI2RX_MAX_CTX);
+		csi->num_ctx = TI_CSI2RX_MAX_CTX;
+	}
+
 	mutex_init(&csi->mutex);
 
 	ret = ti_csi2rx_v4l2_init(csi);
 	if (ret)
 		goto err_v4l2;
 
-	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
+	for (i = 0; i < csi->num_ctx; i++) {
 		csi->ctx[i].idx = i;
 		csi->ctx[i].csi = csi;
 		ret = ti_csi2rx_init_ctx(&csi->ctx[i]);
@@ -1388,14 +1408,14 @@ static void ti_csi2rx_remove(struct platform_device *pdev)
 	struct ti_csi2rx_dev *csi = platform_get_drvdata(pdev);
 	int i;
 
-	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
+	for (i = 0; i < csi->num_ctx; i++) {
 		if (vb2_is_busy(&csi->ctx[i].vidq))
 			dev_err(csi->dev,
 				"Failed to remove as queue busy for ctx %u\n",
 				i);
 	}
 
-	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++)
+	for (i = 0; i < csi->num_ctx; i++)
 		ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
 
 	ti_csi2rx_cleanup_notifier(csi);
Signature · 1 line
2.43.0
#24 lore raw

Re: [PATCH v2 07/13] media: ti: j721e-csi2rx: get number of contexts from device tree

Quoted 93 lines · Jai Luthra

From: Pratyush Yadav <[email protected]>

Different platforms that use this driver might have different number of DMA channels allocated for CSI. So only as many DMA contexts can be used as the number of DMA channels available. Get the number of channels provided via device tree and only configure that many contexts, and hence only that many pads.

---
 .../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c  | 44 ++++++++++++++++------
 1 file changed, 32 insertions(+), 12 deletions(-)
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +13 -9
index 13d7426ab4ba..b4b4bb69c88a 100644
@@ -40,7 +40,7 @@
 #define SHIM_PSI_CFG0_DST_TAG		GENMASK(31, 16)
 
 #define PSIL_WORD_SIZE_BYTES		16
-#define TI_CSI2RX_NUM_CTX		1
+#define TI_CSI2RX_MAX_CTX		32
 
 /*
  * There are no hard limits on the width or height. The DMA engine can handle
@@ -53,8 +53,8 @@
 
 #define TI_CSI2RX_PAD_SINK		0
 #define TI_CSI2RX_PAD_FIRST_SOURCE	1
-#define TI_CSI2RX_NUM_SOURCE_PADS	1
-#define TI_CSI2RX_NUM_PADS		(1 + TI_CSI2RX_NUM_SOURCE_PADS)
+#define TI_CSI2RX_MAX_SOURCE_PADS	TI_CSI2RX_MAX_CTX
+#define TI_CSI2RX_MAX_PADS		(1 + TI_CSI2RX_MAX_SOURCE_PADS)
 
 #define DRAIN_TIMEOUT_MS		50
 #define DRAIN_BUFFER_SIZE		SZ_32K
@@ -112,14 +112,15 @@ struct ti_csi2rx_dev {
 	void __iomem			*shim;
 	struct mutex			mutex; /* To serialize ioctls. */
 	unsigned int			enable_count;
+	unsigned int			num_ctx;
 	struct v4l2_device		v4l2_dev;
 	struct media_device		mdev;
 	struct media_pipeline		pipe;
-	struct media_pad		pads[TI_CSI2RX_NUM_PADS];
+	struct media_pad		pads[TI_CSI2RX_MAX_PADS];
 	struct v4l2_async_notifier	notifier;
 	struct v4l2_subdev		*source;
 	struct v4l2_subdev		subdev;
-	struct ti_csi2rx_ctx		ctx[TI_CSI2RX_NUM_CTX];
+	struct ti_csi2rx_ctx		ctx[TI_CSI2RX_MAX_CTX];
 	/* Buffer to drain stale data from PSI-L endpoint */
 	struct {
 		void			*vaddr;
@@ -449,7 +450,7 @@ static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
 		return ret;
 
 	/* Create and link video nodes for all DMA contexts */
-	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
+	for (i = 0; i < csi->num_ctx; i++) {
 		struct ti_csi2rx_ctx *ctx = &csi->ctx[i];
 		struct video_device *vdev = &ctx->vdev;
 
@@ -1231,10 +1232,12 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
 
 	csi->pads[TI_CSI2RX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
 
-	for (i = TI_CSI2RX_PAD_FIRST_SOURCE; i < TI_CSI2RX_NUM_PADS; i++)
+	for (i = TI_CSI2RX_PAD_FIRST_SOURCE;
+	     i < TI_CSI2RX_PAD_FIRST_SOURCE + csi->num_ctx; i++)
 		csi->pads[i].flags = MEDIA_PAD_FL_SOURCE;
 
-	ret = media_entity_pads_init(&sd->entity, ARRAY_SIZE(csi->pads),
+	ret = media_entity_pads_init(&sd->entity,
+				     TI_CSI2RX_PAD_FIRST_SOURCE + csi->num_ctx,
 				     csi->pads);
 	if (ret)
 		goto unregister_media;
@@ -1320,8 +1323,9 @@ static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
 
 static int ti_csi2rx_probe(struct platform_device *pdev)
 {
+	struct device_node *np = pdev->dev.of_node;
 	struct ti_csi2rx_dev *csi;
-	int ret, i;
+	int ret, i, count;
 
 	csi = devm_kzalloc(&pdev->dev, sizeof(*csi), GFP_KERNEL);
 	if (!csi)

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int ti_csi2rx_probe(struct platform_device *pdev) · line 1351

Quoted 5 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +2 -0
@@ -1343,13 +1347,29 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
 	if (!csi->drain.vaddr)
 		return -ENOMEM;
 
+	/* Only use as many contexts as the number of DMA channels allocated. */
+	count = of_property_count_strings(np, "dma-names");

Is the property marked as required in the DTS bindings ?

If that's the case, I thought the assumption was that dts are 'correct'.

Quoted 47 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +17 -4
+	if (count < 0) {
+		dev_err(csi->dev, "Failed to get DMA channel count: %d\n",
+			count);
+		return count;
+	}
+
+	csi->num_ctx = count;
+	if (csi->num_ctx > TI_CSI2RX_MAX_CTX) {
+		dev_warn(csi->dev,
+			 "%u DMA channels passed. Maximum is %u. Ignoring the rest.\n",
+			 csi->num_ctx, TI_CSI2RX_MAX_CTX);
+		csi->num_ctx = TI_CSI2RX_MAX_CTX;
+	}
+
 	mutex_init(&csi->mutex);
 
 	ret = ti_csi2rx_v4l2_init(csi);
 	if (ret)
 		goto err_v4l2;
 
-	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
+	for (i = 0; i < csi->num_ctx; i++) {
 		csi->ctx[i].idx = i;
 		csi->ctx[i].csi = csi;
 		ret = ti_csi2rx_init_ctx(&csi->ctx[i]);
@@ -1388,14 +1408,14 @@ static void ti_csi2rx_remove(struct platform_device *pdev)
 	struct ti_csi2rx_dev *csi = platform_get_drvdata(pdev);
 	int i;
 
-	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
+	for (i = 0; i < csi->num_ctx; i++) {
 		if (vb2_is_busy(&csi->ctx[i].vidq))
 			dev_err(csi->dev,
 				"Failed to remove as queue busy for ctx %u\n",
 				i);
 	}
 
-	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++)
+	for (i = 0; i < csi->num_ctx; i++)
 		ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
 
 	ti_csi2rx_cleanup_notifier(csi);
 
--

2.43.0

author
#25 lore raw

Re: Re: [PATCH v2 07/13] media: ti: j721e-csi2rx: get number of contexts from device tree

Hi Jacopo,

Thanks for the review.

Quoted 97 lines · Jacopo Mondi
Quoted 96 lines · Jai Luthra

From: Pratyush Yadav <[email protected]>

Different platforms that use this driver might have different number of DMA channels allocated for CSI. So only as many DMA contexts can be used as the number of DMA channels available. Get the number of channels provided via device tree and only configure that many contexts, and hence only that many pads.

---
 .../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c  | 44 ++++++++++++++++------
 1 file changed, 32 insertions(+), 12 deletions(-)
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +13 -9
index 13d7426ab4ba..b4b4bb69c88a 100644
@@ -40,7 +40,7 @@
 #define SHIM_PSI_CFG0_DST_TAG		GENMASK(31, 16)
 
 #define PSIL_WORD_SIZE_BYTES		16
-#define TI_CSI2RX_NUM_CTX		1
+#define TI_CSI2RX_MAX_CTX		32
 
 /*
  * There are no hard limits on the width or height. The DMA engine can handle
@@ -53,8 +53,8 @@
 
 #define TI_CSI2RX_PAD_SINK		0
 #define TI_CSI2RX_PAD_FIRST_SOURCE	1
-#define TI_CSI2RX_NUM_SOURCE_PADS	1
-#define TI_CSI2RX_NUM_PADS		(1 + TI_CSI2RX_NUM_SOURCE_PADS)
+#define TI_CSI2RX_MAX_SOURCE_PADS	TI_CSI2RX_MAX_CTX
+#define TI_CSI2RX_MAX_PADS		(1 + TI_CSI2RX_MAX_SOURCE_PADS)
 
 #define DRAIN_TIMEOUT_MS		50
 #define DRAIN_BUFFER_SIZE		SZ_32K
@@ -112,14 +112,15 @@ struct ti_csi2rx_dev {
 	void __iomem			*shim;
 	struct mutex			mutex; /* To serialize ioctls. */
 	unsigned int			enable_count;
+	unsigned int			num_ctx;
 	struct v4l2_device		v4l2_dev;
 	struct media_device		mdev;
 	struct media_pipeline		pipe;
-	struct media_pad		pads[TI_CSI2RX_NUM_PADS];
+	struct media_pad		pads[TI_CSI2RX_MAX_PADS];
 	struct v4l2_async_notifier	notifier;
 	struct v4l2_subdev		*source;
 	struct v4l2_subdev		subdev;
-	struct ti_csi2rx_ctx		ctx[TI_CSI2RX_NUM_CTX];
+	struct ti_csi2rx_ctx		ctx[TI_CSI2RX_MAX_CTX];
 	/* Buffer to drain stale data from PSI-L endpoint */
 	struct {
 		void			*vaddr;
@@ -449,7 +450,7 @@ static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
 		return ret;
 
 	/* Create and link video nodes for all DMA contexts */
-	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
+	for (i = 0; i < csi->num_ctx; i++) {
 		struct ti_csi2rx_ctx *ctx = &csi->ctx[i];
 		struct video_device *vdev = &ctx->vdev;
 
@@ -1231,10 +1232,12 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
 
 	csi->pads[TI_CSI2RX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
 
-	for (i = TI_CSI2RX_PAD_FIRST_SOURCE; i < TI_CSI2RX_NUM_PADS; i++)
+	for (i = TI_CSI2RX_PAD_FIRST_SOURCE;
+	     i < TI_CSI2RX_PAD_FIRST_SOURCE + csi->num_ctx; i++)
 		csi->pads[i].flags = MEDIA_PAD_FL_SOURCE;
 
-	ret = media_entity_pads_init(&sd->entity, ARRAY_SIZE(csi->pads),
+	ret = media_entity_pads_init(&sd->entity,
+				     TI_CSI2RX_PAD_FIRST_SOURCE + csi->num_ctx,
 				     csi->pads);
 	if (ret)
 		goto unregister_media;
@@ -1320,8 +1323,9 @@ static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
 
 static int ti_csi2rx_probe(struct platform_device *pdev)
 {
+	struct device_node *np = pdev->dev.of_node;
 	struct ti_csi2rx_dev *csi;
-	int ret, i;
+	int ret, i, count;
 
 	csi = devm_kzalloc(&pdev->dev, sizeof(*csi), GFP_KERNEL);
 	if (!csi)
@@ -1343,13 +1347,29 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
 	if (!csi->drain.vaddr)
 		return -ENOMEM;
 
Quoted 8 lines

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int ti_csi2rx_probe(struct platform_device *pdev) · line 1351

Quoted 2 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +2 -0
+	/* Only use as many contexts as the number of DMA channels allocated. */
+	count = of_property_count_strings(np, "dma-names");

Is the property marked as required in the DTS bindings ?

If that's the case, I thought the assumption was that dts are 'correct'.

I see. Yes it is a required property, so correct dts should have at least one channel. In that case I don't see a problem dropping these checks.

Quoted 47 lines
Quoted 47 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +17 -4
+	if (count < 0) {
+		dev_err(csi->dev, "Failed to get DMA channel count: %d\n",
+			count);
+		return count;
+	}
+
+	csi->num_ctx = count;
+	if (csi->num_ctx > TI_CSI2RX_MAX_CTX) {
+		dev_warn(csi->dev,
+			 "%u DMA channels passed. Maximum is %u. Ignoring the rest.\n",
+			 csi->num_ctx, TI_CSI2RX_MAX_CTX);
+		csi->num_ctx = TI_CSI2RX_MAX_CTX;
+	}
+
 	mutex_init(&csi->mutex);
 
 	ret = ti_csi2rx_v4l2_init(csi);
 	if (ret)
 		goto err_v4l2;
 
-	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
+	for (i = 0; i < csi->num_ctx; i++) {
 		csi->ctx[i].idx = i;
 		csi->ctx[i].csi = csi;
 		ret = ti_csi2rx_init_ctx(&csi->ctx[i]);
@@ -1388,14 +1408,14 @@ static void ti_csi2rx_remove(struct platform_device *pdev)
 	struct ti_csi2rx_dev *csi = platform_get_drvdata(pdev);
 	int i;
 
-	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
+	for (i = 0; i < csi->num_ctx; i++) {
 		if (vb2_is_busy(&csi->ctx[i].vidq))
 			dev_err(csi->dev,
 				"Failed to remove as queue busy for ctx %u\n",
 				i);
 	}
 
-	for (i = 0; i < TI_CSI2RX_NUM_CTX; i++)
+	for (i = 0; i < csi->num_ctx; i++)
 		ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
 
 	ti_csi2rx_cleanup_notifier(csi);
 
--

2.43.0

Signature · 4 lines
Thanks,
Jai

GPG Fingerprint: 4DE0 D818 E5D5 75E8 D45A AFC5 43DE 91F9 249A 7145
author
#26 lore raw

PATCH v2 8/13 media: cadence: csi2rx: add get_frame_desc wrapper

In reply to Jai Luthra

From: Pratyush Yadav <[email protected]>

J721E wrapper CSI2RX driver needs to get the frame descriptor from the source to find out info about virtual channel. This driver itself does not touch the routing or virtual channels in any way. So simply pass the descriptor through from the source.

---
 drivers/media/platform/cadence/cdns-csi2rx.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
drivers/media/platform/cadence/cdns-csi2rx.c +24 -0
index 751eadbe61ef..b6054d62c799 100644
@@ -135,6 +135,21 @@ static const struct csi2rx_fmt *csi2rx_get_fmt_by_code(u32 code)
 	return NULL;
 }
 
+static int csi2rx_get_frame_desc_from_source(struct csi2rx_priv *csi2rx,
+					     struct v4l2_mbus_frame_desc *fd)
+{
+	struct media_pad *remote_pad;
+
+	remote_pad = media_entity_remote_source_pad_unique(&csi2rx->subdev.entity);
+	if (!remote_pad) {
+		dev_err(csi2rx->dev, "No remote pad found for sink\n");
+		return -ENODEV;
+	}
+
+	return v4l2_subdev_call(csi2rx->source_subdev, pad, get_frame_desc,
+				remote_pad->index, fd);
+}
+
 static inline
 struct csi2rx_priv *v4l2_subdev_to_csi2rx(struct v4l2_subdev *subdev)
 {
@@ -466,10 +481,19 @@ static int csi2rx_init_state(struct v4l2_subdev *subdev,
 	return csi2rx_set_fmt(subdev, state, &format);
 }
 
+static int csi2rx_get_frame_desc(struct v4l2_subdev *subdev, unsigned int pad,
+				 struct v4l2_mbus_frame_desc *fd)
+{
+	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
+
+	return csi2rx_get_frame_desc_from_source(csi2rx, fd);
+}
+
 static const struct v4l2_subdev_pad_ops csi2rx_pad_ops = {
 	.enum_mbus_code	= csi2rx_enum_mbus_code,
 	.get_fmt	= v4l2_subdev_get_fmt,
 	.set_fmt	= csi2rx_set_fmt,
+	.get_frame_desc = csi2rx_get_frame_desc,
 };
 
 static const struct v4l2_subdev_video_ops csi2rx_video_ops = {
Signature · 1 line
2.43.0
Reviewed-by on v2 8/13
#27 lore raw

Re: [PATCH v2 08/13] media: cadence: csi2rx: add get_frame_desc wrapper

Hi Jai

Quoted 46 lines · Jai Luthra

From: Pratyush Yadav <[email protected]>

J721E wrapper CSI2RX driver needs to get the frame descriptor from the source to find out info about virtual channel. This driver itself does not touch the routing or virtual channels in any way. So simply pass the descriptor through from the source.

---
 drivers/media/platform/cadence/cdns-csi2rx.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
drivers/media/platform/cadence/cdns-csi2rx.c +18 -0
index 751eadbe61ef..b6054d62c799 100644
@@ -135,6 +135,21 @@ static const struct csi2rx_fmt *csi2rx_get_fmt_by_code(u32 code)
 	return NULL;
 }
 
+static int csi2rx_get_frame_desc_from_source(struct csi2rx_priv *csi2rx,
+					     struct v4l2_mbus_frame_desc *fd)
+{
+	struct media_pad *remote_pad;
+
+	remote_pad = media_entity_remote_source_pad_unique(&csi2rx->subdev.entity);
+	if (!remote_pad) {
+		dev_err(csi2rx->dev, "No remote pad found for sink\n");
+		return -ENODEV;
+	}
+
+	return v4l2_subdev_call(csi2rx->source_subdev, pad, get_frame_desc,
+				remote_pad->index, fd);
+}
+
 static inline
 struct csi2rx_priv *v4l2_subdev_to_csi2rx(struct v4l2_subdev *subdev)
 {
@@ -466,10 +481,19 @@ static int csi2rx_init_state(struct v4l2_subdev *subdev,
 	return csi2rx_set_fmt(subdev, state, &format);
 }
 
+static int csi2rx_get_frame_desc(struct v4l2_subdev *subdev, unsigned int pad,
+				 struct v4l2_mbus_frame_desc *fd)
+{

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_init_state(struct v4l2_subdev *subdev, · line 496

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +6 -0
+	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
+
+	return csi2rx_get_frame_desc_from_source(csi2rx, fd);
+}
+
 static const struct v4l2_subdev_pad_ops csi2rx_pad_ops = {
 	.enum_mbus_code	= csi2rx_enum_mbus_code,
 	.get_fmt	= v4l2_subdev_get_fmt,
 	.set_fmt	= csi2rx_set_fmt,
+	.get_frame_desc = csi2rx_get_frame_desc,

The previous entries are indented with a tab

With this fixed

Thanks
j

Quoted 8 lines
drivers/media/platform/cadence/cdns-csi2rx.c +0 -1
 };
 
 static const struct v4l2_subdev_video_ops csi2rx_video_ops = {
 
--

2.43.0

author
#28 lore raw

Re: Re: [PATCH v2 08/13] media: cadence: csi2rx: add get_frame_desc wrapper

Quoted 46 lines · Jacopo Mondi

Hi Jai

Quoted 43 lines · Jai Luthra

From: Pratyush Yadav <[email protected]>

J721E wrapper CSI2RX driver needs to get the frame descriptor from the source to find out info about virtual channel. This driver itself does not touch the routing or virtual channels in any way. So simply pass the descriptor through from the source.

---
 drivers/media/platform/cadence/cdns-csi2rx.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
drivers/media/platform/cadence/cdns-csi2rx.c +15 -0
index 751eadbe61ef..b6054d62c799 100644
@@ -135,6 +135,21 @@ static const struct csi2rx_fmt *csi2rx_get_fmt_by_code(u32 code)
 	return NULL;
 }
 
+static int csi2rx_get_frame_desc_from_source(struct csi2rx_priv *csi2rx,
+					     struct v4l2_mbus_frame_desc *fd)
+{
+	struct media_pad *remote_pad;
+
+	remote_pad = media_entity_remote_source_pad_unique(&csi2rx->subdev.entity);
+	if (!remote_pad) {
+		dev_err(csi2rx->dev, "No remote pad found for sink\n");
+		return -ENODEV;
+	}
+
+	return v4l2_subdev_call(csi2rx->source_subdev, pad, get_frame_desc,
+				remote_pad->index, fd);
+}
+
 static inline
 struct csi2rx_priv *v4l2_subdev_to_csi2rx(struct v4l2_subdev *subdev)
 {
@@ -466,10 +481,19 @@ static int csi2rx_init_state(struct v4l2_subdev *subdev,
 	return csi2rx_set_fmt(subdev, state, &format);
 }
 
Quoted 15 lines

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_init_state(struct v4l2_subdev *subdev, · line 496

Quoted 13 lines
drivers/media/platform/cadence/cdns-csi2rx.c +9 -0
+static int csi2rx_get_frame_desc(struct v4l2_subdev *subdev, unsigned int pad,
+				 struct v4l2_mbus_frame_desc *fd)
+{
+	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
+
+	return csi2rx_get_frame_desc_from_source(csi2rx, fd);
+}
+
 static const struct v4l2_subdev_pad_ops csi2rx_pad_ops = {
 	.enum_mbus_code	= csi2rx_enum_mbus_code,
 	.get_fmt	= v4l2_subdev_get_fmt,
 	.set_fmt	= csi2rx_set_fmt,
+	.get_frame_desc = csi2rx_get_frame_desc,

The previous entries are indented with a tab

Ah thanks, will fix.

Quoted 15 lines

With this fixed

Thanks
j

Quoted 8 lines
drivers/media/platform/cadence/cdns-csi2rx.c +0 -1
 };
 
 static const struct v4l2_subdev_video_ops csi2rx_video_ops = {
 
--

2.43.0

Signature · 4 lines
Thanks,
Jai

GPG Fingerprint: 4DE0 D818 E5D5 75E8 D45A AFC5 43DE 91F9 249A 7145
author
#29 lore raw

PATCH v2 9/13 media: ti: j721e-csi2rx: add support for processing virtual channels

In reply to Jai Luthra

Use get_frame_desc() to get the frame desc from the connected source, and use the provided virtual channel instead of hardcoded one.

get_frame_desc() works per stream, but as we don't support multiple streams yet, we will just always use stream 0. If the source doesn't support get_frame_desc(), fall back to the previous method of always capturing virtual channel 0.

---
 .../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c  | 39 ++++++++++++++++++++++
 1 file changed, 39 insertions(+)
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +39 -0
index b4b4bb69c88a..c0916ca1a6f8 100644
@@ -29,6 +29,7 @@
 #define SHIM_DMACNTX_EN			BIT(31)
 #define SHIM_DMACNTX_YUV422		GENMASK(27, 26)
 #define SHIM_DMACNTX_SIZE		GENMASK(21, 20)
+#define SHIM_DMACNTX_VC			GENMASK(9, 6)
 #define SHIM_DMACNTX_FMT		GENMASK(5, 0)
 #define SHIM_DMACNTX_YUV422_MODE_11	3
 #define SHIM_DMACNTX_SIZE_8		0
@@ -105,6 +106,8 @@ struct ti_csi2rx_ctx {
 	struct media_pad		pad;
 	u32				sequence;
 	u32				idx;
+	u32				vc;
+	u32				stream;
 };
 
 struct ti_csi2rx_dev {
@@ -571,6 +574,7 @@ static void ti_csi2rx_setup_shim(struct ti_csi2rx_ctx *ctx)
 	}
 
 	reg |= FIELD_PREP(SHIM_DMACNTX_SIZE, fmt->size);
+	reg |= FIELD_PREP(SHIM_DMACNTX_VC, ctx->vc);
 
 	writel(reg, csi->shim + SHIM_DMACNTX(ctx->idx));
 
@@ -844,6 +848,33 @@ static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
 	}
 }
 
+static int ti_csi2rx_get_vc(struct ti_csi2rx_ctx *ctx)
+{
+	struct ti_csi2rx_dev *csi = ctx->csi;
+	struct v4l2_mbus_frame_desc fd;
+	struct media_pad *pad;
+	int ret, i;
+
+	pad = media_entity_remote_pad_unique(&csi->subdev.entity, MEDIA_PAD_FL_SOURCE);
+	if (!pad)
+		return -ENODEV;
+
+	ret = v4l2_subdev_call(csi->source, pad, get_frame_desc, pad->index,
+			       &fd);
+	if (ret)
+		return ret;
+
+	if (fd.type != V4L2_MBUS_FRAME_DESC_TYPE_CSI2)
+		return -EINVAL;
+
+	for (i = 0; i < fd.num_entries; i++) {
+		if (ctx->stream == fd.entry[i].stream)
+			return fd.entry[i].bus.csi2.vc;
+	}
+
+	return -ENODEV;
+}
+
 static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 {
 	struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vq);
@@ -864,6 +895,14 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 	if (ret)
 		goto err;
 
+	ret = ti_csi2rx_get_vc(ctx);
+	if (ret == -ENOIOCTLCMD)
+		ctx->vc = 0;
+	else if (ret < 0)
+		goto err;
+	else
+		ctx->vc = ret;
+
 	ti_csi2rx_setup_shim(ctx);
 
 	ctx->sequence = 0;
Signature · 1 line
2.43.0
#30 lore raw

Re: [PATCH v2 09/13] media: ti: j721e-csi2rx: add support for processing virtual channels

Hi Jai

Quoted 53 lines · Jai Luthra

Use get_frame_desc() to get the frame desc from the connected source, and use the provided virtual channel instead of hardcoded one.

get_frame_desc() works per stream, but as we don't support multiple streams yet, we will just always use stream 0. If the source doesn't support get_frame_desc(), fall back to the previous method of always capturing virtual channel 0.

---
 .../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c  | 39 ++++++++++++++++++++++
 1 file changed, 39 insertions(+)
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +9 -0
index b4b4bb69c88a..c0916ca1a6f8 100644
@@ -29,6 +29,7 @@
 #define SHIM_DMACNTX_EN			BIT(31)
 #define SHIM_DMACNTX_YUV422		GENMASK(27, 26)
 #define SHIM_DMACNTX_SIZE		GENMASK(21, 20)
+#define SHIM_DMACNTX_VC			GENMASK(9, 6)
 #define SHIM_DMACNTX_FMT		GENMASK(5, 0)
 #define SHIM_DMACNTX_YUV422_MODE_11	3
 #define SHIM_DMACNTX_SIZE_8		0
@@ -105,6 +106,8 @@ struct ti_csi2rx_ctx {
 	struct media_pad		pad;
 	u32				sequence;
 	u32				idx;
+	u32				vc;
+	u32				stream;
 };
 
 struct ti_csi2rx_dev {
@@ -571,6 +574,7 @@ static void ti_csi2rx_setup_shim(struct ti_csi2rx_ctx *ctx)
 	}
 
 	reg |= FIELD_PREP(SHIM_DMACNTX_SIZE, fmt->size);
+	reg |= FIELD_PREP(SHIM_DMACNTX_VC, ctx->vc);
 
 	writel(reg, csi->shim + SHIM_DMACNTX(ctx->idx));
 
@@ -844,6 +848,33 @@ static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
 	}
 }
 
+static int ti_csi2rx_get_vc(struct ti_csi2rx_ctx *ctx)
+{
+	struct ti_csi2rx_dev *csi = ctx->csi;
+	struct v4l2_mbus_frame_desc fd;
+	struct media_pad *pad;

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb) · line 865

Quoted 10 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +10 -0
+	int ret, i;
+
+	pad = media_entity_remote_pad_unique(&csi->subdev.entity, MEDIA_PAD_FL_SOURCE);
+	if (!pad)
+		return -ENODEV;
+
+	ret = v4l2_subdev_call(csi->source, pad, get_frame_desc, pad->index,
+			       &fd);
+	if (ret)
+		return ret;

Would it be better to fail at bound() time if the remote subdev doesn't support get_frame_desc ? you can use

        if (!v4l2_subdev_has_op(subdev, pad, get_frame_desc)) {

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb) · line 870

Quoted 5 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +5 -0
+
+	if (fd.type != V4L2_MBUS_FRAME_DESC_TYPE_CSI2)
+		return -EINVAL;
+
+	for (i = 0; i < fd.num_entries; i++) {

for (unsigned int i

should num_entries be validated ?

Quoted 11 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +7 -0
+		if (ctx->stream == fd.entry[i].stream)
+			return fd.entry[i].bus.csi2.vc;
+	}
+
+	return -ENODEV;
+}
+
 static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 {
 	struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vq);

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count) · line 900

Quoted 6 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +3 -0
@@ -864,6 +895,14 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 	if (ret)
 		goto err;
 
+	ret = ti_csi2rx_get_vc(ctx);
+	if (ret == -ENOIOCTLCMD)
+		ctx->vc = 0;

Ah, so you fallback to 0 in case the subdev doesn't support get_frame_desc. I'm not sure what would be better here maybe wait for other's opinions as well.

Personally I would fail earlier to make sure subdev drivers are forced to impement get_frame_desc

Quoted 13 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +5 -1
+	else if (ret < 0)
+		goto err;
+	else
+		ctx->vc = ret;
+
 	ti_csi2rx_setup_shim(ctx);
 
 	ctx->sequence = 0;
 
--

2.43.0

author
#31 lore raw

Re: Re: [PATCH v2 09/13] media: ti: j721e-csi2rx: add support for processing virtual channels

Hi Jacopo,

Quoted 104 lines · Jacopo Mondi

Hi Jai

Quoted 53 lines · Jai Luthra

Use get_frame_desc() to get the frame desc from the connected source, and use the provided virtual channel instead of hardcoded one.

get_frame_desc() works per stream, but as we don't support multiple streams yet, we will just always use stream 0. If the source doesn't support get_frame_desc(), fall back to the previous method of always capturing virtual channel 0.

---
 .../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c  | 39 ++++++++++++++++++++++
 1 file changed, 39 insertions(+)
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +9 -0
index b4b4bb69c88a..c0916ca1a6f8 100644
@@ -29,6 +29,7 @@
 #define SHIM_DMACNTX_EN			BIT(31)
 #define SHIM_DMACNTX_YUV422		GENMASK(27, 26)
 #define SHIM_DMACNTX_SIZE		GENMASK(21, 20)
+#define SHIM_DMACNTX_VC			GENMASK(9, 6)
 #define SHIM_DMACNTX_FMT		GENMASK(5, 0)
 #define SHIM_DMACNTX_YUV422_MODE_11	3
 #define SHIM_DMACNTX_SIZE_8		0
@@ -105,6 +106,8 @@ struct ti_csi2rx_ctx {
 	struct media_pad		pad;
 	u32				sequence;
 	u32				idx;
+	u32				vc;
+	u32				stream;
 };
 
 struct ti_csi2rx_dev {
@@ -571,6 +574,7 @@ static void ti_csi2rx_setup_shim(struct ti_csi2rx_ctx *ctx)
 	}
 
 	reg |= FIELD_PREP(SHIM_DMACNTX_SIZE, fmt->size);
+	reg |= FIELD_PREP(SHIM_DMACNTX_VC, ctx->vc);
 
 	writel(reg, csi->shim + SHIM_DMACNTX(ctx->idx));
 
@@ -844,6 +848,33 @@ static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
 	}
 }
 
+static int ti_csi2rx_get_vc(struct ti_csi2rx_ctx *ctx)
+{
+	struct ti_csi2rx_dev *csi = ctx->csi;
+	struct v4l2_mbus_frame_desc fd;
+	struct media_pad *pad;

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb) · line 865

Quoted 10 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +10 -0
+	int ret, i;
+
+	pad = media_entity_remote_pad_unique(&csi->subdev.entity, MEDIA_PAD_FL_SOURCE);
+	if (!pad)
+		return -ENODEV;
+
+	ret = v4l2_subdev_call(csi->source, pad, get_frame_desc, pad->index,
+			       &fd);
+	if (ret)
+		return ret;

Would it be better to fail at bound() time if the remote subdev doesn't support get_frame_desc ? you can use

        if (!v4l2_subdev_has_op(subdev, pad, get_frame_desc)) {

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb) · line 870

Quoted 5 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +5 -0
+
+	if (fd.type != V4L2_MBUS_FRAME_DESC_TYPE_CSI2)
+		return -EINVAL;
+
+	for (i = 0; i < fd.num_entries; i++) {

for (unsigned int i

should num_entries be validated ?

Quoted 11 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +7 -0
+		if (ctx->stream == fd.entry[i].stream)
+			return fd.entry[i].bus.csi2.vc;
+	}
+
+	return -ENODEV;
+}
+
 static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 {
 	struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vq);

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count) · line 900

Quoted 6 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +3 -0
@@ -864,6 +895,14 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 	if (ret)
 		goto err;
 
+	ret = ti_csi2rx_get_vc(ctx);
+	if (ret == -ENOIOCTLCMD)
+		ctx->vc = 0;

Ah, so you fallback to 0 in case the subdev doesn't support get_frame_desc. I'm not sure what would be better here maybe wait for other's opinions as well.

Quoted 3 lines

Personally I would fail earlier to make sure subdev drivers are forced to impement get_frame_desc

Before we break support for all non-streams-aware subdevs, I think maybe we need a default v4l2_subdev_get_frame_desc() helper that returns a single entry with stream = 0 and csi2.vc = 0, as that is usually what most sensors do.

Quoted 13 lines
Quoted 13 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +5 -1
+	else if (ret < 0)
+		goto err;
+	else
+		ctx->vc = ret;
+
 	ti_csi2rx_setup_shim(ctx);
 
 	ctx->sequence = 0;
 
--

2.43.0

Signature · 4 lines
Thanks,
Jai

GPG Fingerprint: 4DE0 D818 E5D5 75E8 D45A AFC5 43DE 91F9 249A 7145
#32 lore raw

Re: [PATCH v2 09/13] media: ti: j721e-csi2rx: add support for processing virtual channels

In reply to Jacopo Mondi
Quoted 104 lines · Jacopo Mondi

Hi Jai

Quoted 53 lines · Jai Luthra

Use get_frame_desc() to get the frame desc from the connected source, and use the provided virtual channel instead of hardcoded one.

get_frame_desc() works per stream, but as we don't support multiple streams yet, we will just always use stream 0. If the source doesn't support get_frame_desc(), fall back to the previous method of always capturing virtual channel 0.

---
  .../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c  | 39 ++++++++++++++++++++++
  1 file changed, 39 insertions(+)
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +9 -0
index b4b4bb69c88a..c0916ca1a6f8 100644
@@ -29,6 +29,7 @@
  #define SHIM_DMACNTX_EN			BIT(31)
  #define SHIM_DMACNTX_YUV422		GENMASK(27, 26)
  #define SHIM_DMACNTX_SIZE		GENMASK(21, 20)
+#define SHIM_DMACNTX_VC			GENMASK(9, 6)
  #define SHIM_DMACNTX_FMT		GENMASK(5, 0)
  #define SHIM_DMACNTX_YUV422_MODE_11	3
  #define SHIM_DMACNTX_SIZE_8		0
@@ -105,6 +106,8 @@ struct ti_csi2rx_ctx {
  	struct media_pad		pad;
  	u32				sequence;
  	u32				idx;
+	u32				vc;
+	u32				stream;
  };
 
  struct ti_csi2rx_dev {
@@ -571,6 +574,7 @@ static void ti_csi2rx_setup_shim(struct ti_csi2rx_ctx *ctx)
  	}
 
  	reg |= FIELD_PREP(SHIM_DMACNTX_SIZE, fmt->size);
+	reg |= FIELD_PREP(SHIM_DMACNTX_VC, ctx->vc);
 
  	writel(reg, csi->shim + SHIM_DMACNTX(ctx->idx));
 
@@ -844,6 +848,33 @@ static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
  	}
  }
 
+static int ti_csi2rx_get_vc(struct ti_csi2rx_ctx *ctx)
+{
+	struct ti_csi2rx_dev *csi = ctx->csi;
+	struct v4l2_mbus_frame_desc fd;
+	struct media_pad *pad;

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb) · line 865

Quoted 10 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +10 -0
+	int ret, i;
+
+	pad = media_entity_remote_pad_unique(&csi->subdev.entity, MEDIA_PAD_FL_SOURCE);
+	if (!pad)
+		return -ENODEV;
+
+	ret = v4l2_subdev_call(csi->source, pad, get_frame_desc, pad->index,
+			       &fd);
+	if (ret)
+		return ret;

Would it be better to fail at bound() time if the remote subdev doesn't support get_frame_desc ? you can use

         if (!v4l2_subdev_has_op(subdev, pad, get_frame_desc)) {

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb) · line 870

Quoted 5 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +5 -0
+
+	if (fd.type != V4L2_MBUS_FRAME_DESC_TYPE_CSI2)
+		return -EINVAL;
+
+	for (i = 0; i < fd.num_entries; i++) {

for (unsigned int i

should num_entries be validated ?

Quoted 11 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +7 -0
+		if (ctx->stream == fd.entry[i].stream)
+			return fd.entry[i].bus.csi2.vc;
+	}
+
+	return -ENODEV;
+}
+
  static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
  {
  	struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vq);

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count) · line 900

Quoted 6 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +3 -0
@@ -864,6 +895,14 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
  	if (ret)
  		goto err;
 
+	ret = ti_csi2rx_get_vc(ctx);
+	if (ret == -ENOIOCTLCMD)
+		ctx->vc = 0;

Ah, so you fallback to 0 in case the subdev doesn't support get_frame_desc. I'm not sure what would be better here maybe wait for other's opinions as well.

Quoted 2 lines

Personally I would fail earlier to make sure subdev drivers are forced to impement get_frame_desc

As this driver is already in upstream, with single-stream support, I think requiring frame_desc could break currently working setups.

I do a similar thing as above in CAL and RPi CFE drivers.

Tomi

author
#33 lore raw

PATCH v2 10/13 media: cadence: csi2rx: Use new enable stream APIs

In reply to Jai Luthra

The enable_streams() API in v4l2 supports passing a bitmask to enable each pad/stream combination individually on any media subdev instead of doing s_stream(1) to start all streams on the subdev at once.

This API is implemented by ds90ub960 driver (FPDLink deser) and thus the caller (cdns-csi2x) is required to use it. For now we only enable stream0.

---
 drivers/media/platform/cadence/cdns-csi2rx.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)
drivers/media/platform/cadence/cdns-csi2rx.c +16 -2
index b6054d62c799..2ec34fc9c524 100644
@@ -222,10 +222,18 @@ static int csi2rx_configure_ext_dphy(struct csi2rx_priv *csi2rx)
 static int csi2rx_start(struct csi2rx_priv *csi2rx)
 {
 	unsigned int i;
+	struct media_pad *remote_pad;
 	unsigned long lanes_used = 0;
 	u32 reg;
 	int ret;
 
+	remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
+	if (!remote_pad) {
+		dev_err(csi2rx->dev,
+			"Failed to find connected source\n");
+		return -ENODEV;
+	}
+
 	ret = clk_prepare_enable(csi2rx->p_clk);
 	if (ret)
 		return ret;
@@ -309,7 +317,8 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
 
 	reset_control_deassert(csi2rx->sys_rst);
 
-	ret = v4l2_subdev_call(csi2rx->source_subdev, video, s_stream, true);
+	ret = v4l2_subdev_enable_streams(csi2rx->source_subdev,
+					 remote_pad->index, BIT(0));
 	if (ret)
 		goto err_disable_sysclk;
 
@@ -337,6 +346,7 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
 
 static void csi2rx_stop(struct csi2rx_priv *csi2rx)
 {
+	struct media_pad *remote_pad;
 	unsigned int i;
 	u32 val;
 	int ret;
@@ -365,8 +375,12 @@ static void csi2rx_stop(struct csi2rx_priv *csi2rx)
 	reset_control_assert(csi2rx->p_rst);
 	clk_disable_unprepare(csi2rx->p_clk);
 
-	if (v4l2_subdev_call(csi2rx->source_subdev, video, s_stream, false))
+	remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
+	if (!remote_pad ||
+	    v4l2_subdev_disable_streams(csi2rx->source_subdev,
+					remote_pad->index, BIT(0))) {
 		dev_warn(csi2rx->dev, "Couldn't disable our subdev\n");
+	}
 
 	if (csi2rx->dphy) {
 		writel(0, csi2rx->base + CSI2RX_DPHY_LANE_CTRL_REG);
Signature · 1 line
2.43.0
author
#34 lore raw

PATCH v2 11/13 media: cadence: csi2rx: Enable multi-stream support

In reply to Jai Luthra

Cadence CSI-2 bridge IP supports capturing multiple virtual "streams" of data over the same physical interface using MIPI Virtual Channels.

The V4L2 subdev APIs should reflect this capability and allow per-stream routing and controls.

While the hardware IP supports usecases where streams coming in the sink pad can be broadcasted to multiple source pads, the driver will need significant re-architecture to make that possible. The two users of this IP in mainline linux are TI Shim and StarFive JH7110 CAMSS, and both have only integrated the first source pad i.e stream0 of this IP. So for now keep it simple and only allow 1-to-1 mapping of streams from sink to source, without any broadcasting.

With stream routing now supported in the driver, implement the enable_stream and disable_stream hooks in place of the stream-unaware s_stream hook.

This allows consumer devices like a DMA bridge or ISP, to enable particular streams on a source pad, which in turn can be used to enable only particular streams on the CSI-TX device connected on the sink pad.

Implement a fallback s_stream hook that internally calls enable_stream on each source pad, for consumer drivers that don't use multi-stream APIs to still work.

---
 drivers/media/platform/cadence/cdns-csi2rx.c | 407 ++++++++++++++++++++-------
 1 file changed, 313 insertions(+), 94 deletions(-)
drivers/media/platform/cadence/cdns-csi2rx.c +313 -94
index 2ec34fc9c524..b0c91a9c65e8 100644
@@ -90,6 +90,7 @@ struct csi2rx_priv {
 	struct reset_control		*pixel_rst[CSI2RX_STREAMS_MAX];
 	struct phy			*dphy;
 
+	u32				vc_select[CSI2RX_STREAMS_MAX];
 	u8				lanes[CSI2RX_LANES_MAX];
 	u8				num_lanes;
 	u8				max_lanes;
@@ -179,27 +180,43 @@ static void csi2rx_reset(struct csi2rx_priv *csi2rx)
 
 static int csi2rx_configure_ext_dphy(struct csi2rx_priv *csi2rx)
 {
+	struct v4l2_ctrl_handler *handler = csi2rx->source_subdev->ctrl_handler;
 	union phy_configure_opts opts = { };
 	struct phy_configure_opts_mipi_dphy *cfg = &opts.mipi_dphy;
-	struct v4l2_subdev_format sd_fmt = {
-		.which	= V4L2_SUBDEV_FORMAT_ACTIVE,
-		.pad	= CSI2RX_PAD_SINK,
-	};
+	struct v4l2_mbus_framefmt *framefmt;
+	struct v4l2_subdev_state *state;
 	const struct csi2rx_fmt *fmt;
 	s64 link_freq;
 	int ret;
 
-	ret = v4l2_subdev_call_state_active(&csi2rx->subdev, pad, get_fmt,
-					    &sd_fmt);
-	if (ret < 0)
-		return ret;
+	if (v4l2_ctrl_find(handler, V4L2_CID_LINK_FREQ)) {
+		link_freq = v4l2_get_link_freq(handler, 0, 0);
+	} else {
+		state = v4l2_subdev_get_locked_active_state(&csi2rx->subdev);
+		framefmt = v4l2_subdev_state_get_format(state, CSI2RX_PAD_SINK,
+							0);
+
+		if (framefmt) {
+			fmt = csi2rx_get_fmt_by_code(framefmt->code);
+		} else {
+			dev_err(csi2rx->dev,
+				"Did not find active sink format\n");
+			return -EINVAL;
+		}
 
-	fmt = csi2rx_get_fmt_by_code(sd_fmt.format.code);
+		link_freq = v4l2_get_link_freq(handler, fmt->bpp,
+					       2 * csi2rx->num_lanes);
 
-	link_freq = v4l2_get_link_freq(csi2rx->source_subdev->ctrl_handler,
-				       fmt->bpp, 2 * csi2rx->num_lanes);
-	if (link_freq < 0)
+		dev_warn(csi2rx->dev,
+			 "Guessing link frequency using bitdepth of stream 0.\n");
+		dev_warn(csi2rx->dev,
+			 "V4L2_CID_LINK_FREQ control is required for multi format sources.\n");
+	}
+
+	if (link_freq < 0) {
+		dev_err(csi2rx->dev, "Unable to calculate link frequency\n");
 		return link_freq;
+	}
 
 	ret = phy_mipi_dphy_get_default_config_for_hsclk(link_freq,
 							 csi2rx->num_lanes, cfg);
@@ -222,18 +239,10 @@ static int csi2rx_configure_ext_dphy(struct csi2rx_priv *csi2rx)
 static int csi2rx_start(struct csi2rx_priv *csi2rx)
 {
 	unsigned int i;
-	struct media_pad *remote_pad;
 	unsigned long lanes_used = 0;
 	u32 reg;
 	int ret;
 
-	remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
-	if (!remote_pad) {
-		dev_err(csi2rx->dev,
-			"Failed to find connected source\n");
-		return -ENODEV;
-	}
-
 	ret = clk_prepare_enable(csi2rx->p_clk);
 	if (ret)
 		return ret;
@@ -300,11 +309,7 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
 		writel(CSI2RX_STREAM_CFG_FIFO_MODE_LARGE_BUF,
 		       csi2rx->base + CSI2RX_STREAM_CFG_REG(i));
 
-		/*
-		 * Enable one virtual channel. When multiple virtual channels
-		 * are supported this will have to be changed.
-		 */
-		writel(CSI2RX_STREAM_DATA_CFG_VC_SELECT(0),
+		writel(csi2rx->vc_select[i],
 		       csi2rx->base + CSI2RX_STREAM_DATA_CFG_REG(i));
 
 		writel(CSI2RX_STREAM_CTRL_START,
@@ -317,17 +322,10 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
 
 	reset_control_deassert(csi2rx->sys_rst);
 
-	ret = v4l2_subdev_enable_streams(csi2rx->source_subdev,
-					 remote_pad->index, BIT(0));
-	if (ret)
-		goto err_disable_sysclk;
-
 	clk_disable_unprepare(csi2rx->p_clk);
 
 	return 0;
 
-err_disable_sysclk:
-	clk_disable_unprepare(csi2rx->sys_clk);
 err_disable_pixclk:
 	for (; i > 0; i--) {
 		reset_control_assert(csi2rx->pixel_rst[i - 1]);
@@ -346,7 +344,6 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
 
 static void csi2rx_stop(struct csi2rx_priv *csi2rx)
 {
-	struct media_pad *remote_pad;
 	unsigned int i;
 	u32 val;
 	int ret;
@@ -375,13 +372,6 @@ static void csi2rx_stop(struct csi2rx_priv *csi2rx)
 	reset_control_assert(csi2rx->p_rst);
 	clk_disable_unprepare(csi2rx->p_clk);
 
-	remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
-	if (!remote_pad ||
-	    v4l2_subdev_disable_streams(csi2rx->source_subdev,
-					remote_pad->index, BIT(0))) {
-		dev_warn(csi2rx->dev, "Couldn't disable our subdev\n");
-	}
-
 	if (csi2rx->dphy) {
 		writel(0, csi2rx->base + CSI2RX_DPHY_LANE_CTRL_REG);
 
@@ -390,47 +380,167 @@ static void csi2rx_stop(struct csi2rx_priv *csi2rx)
 	}
 }
 
-static int csi2rx_s_stream(struct v4l2_subdev *subdev, int enable)
+static void csi2rx_update_vc_select(struct csi2rx_priv *csi2rx,
+				    struct v4l2_subdev_state *state)
 {
-	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
-	int ret = 0;
+	struct v4l2_mbus_frame_desc fd = {0};
+	struct v4l2_subdev_route *route;
+	unsigned int i;
+	int ret;
 
-	if (enable) {
-		ret = pm_runtime_resume_and_get(csi2rx->dev);
-		if (ret < 0)
-			return ret;
+	for (i = 0; i < CSI2RX_STREAMS_MAX; i++)
+		csi2rx->vc_select[i] = 0;
+
+	ret = csi2rx_get_frame_desc_from_source(csi2rx, &fd);
+	if (ret || fd.type != V4L2_MBUS_FRAME_DESC_TYPE_CSI2) {
+		dev_dbg(csi2rx->dev,
+			"Failed to get source frame desc, allowing only VC=0\n");
+		goto err_no_fd;
 	}
 
-	mutex_lock(&csi2rx->lock);
+	/* If source provides per-stream VC info, use it to filter by VC */
+	for_each_active_route(&state->routing, route) {
+		int cdns_stream = route->source_pad - CSI2RX_PAD_SOURCE_STREAM0;
+		u8 used_vc = 0;
 
-	if (enable) {
-		/*
-		 * If we're not the first users, there's no need to
-		 * enable the whole controller.
-		 */
-		if (!csi2rx->count) {
-			ret = csi2rx_start(csi2rx);
-			if (ret) {
-				pm_runtime_put(csi2rx->dev);
-				goto out;
+		for (i = 0; i < fd.num_entries; i++) {
+			if (fd.entry[i].stream == route->sink_stream) {
+				used_vc = fd.entry[i].bus.csi2.vc;
+				break;
 			}
 		}
+		csi2rx->vc_select[cdns_stream] |=
+			CSI2RX_STREAM_DATA_CFG_VC_SELECT(used_vc);
+	}
 
-		csi2rx->count++;
-	} else {
-		csi2rx->count--;
+err_no_fd:
+	for (i = 0; i < CSI2RX_STREAMS_MAX; i++) {
+		if (!csi2rx->vc_select[i]) {
+			csi2rx->vc_select[i] =
+				CSI2RX_STREAM_DATA_CFG_VC_SELECT(0);
+		}
+	}
+}
 
-		/*
-		 * Let the last user turn off the lights.
-		 */
-		if (!csi2rx->count)
-			csi2rx_stop(csi2rx);
+static int csi2rx_enable_streams(struct v4l2_subdev *subdev,
+				 struct v4l2_subdev_state *state, u32 pad,
+				 u64 streams_mask)
+{
+	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
+	struct media_pad *remote_pad;
+	u64 sink_streams;
+	int ret;
+
+	remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
+	if (!remote_pad) {
+		dev_err(csi2rx->dev,
+			"Failed to find connected source\n");
+		return -ENODEV;
+	}
+
+	ret = pm_runtime_resume_and_get(csi2rx->dev);
+	if (ret < 0)
+		return ret;
+
+	sink_streams = v4l2_subdev_state_xlate_streams(state, pad,
+						       CSI2RX_PAD_SINK,
+						       &streams_mask);
 
-		pm_runtime_put(csi2rx->dev);
+	mutex_lock(&csi2rx->lock);
+	/*
+	 * If we're not the first users, there's no need to
+	 * enable the whole controller.
+	 */
+	if (!csi2rx->count) {
+		ret = csi2rx_start(csi2rx);
+		if (ret)
+			goto err_stream_start;
 	}
 
-out:
+	/* Start streaming on the source */
+	ret = v4l2_subdev_enable_streams(csi2rx->source_subdev, remote_pad->index,
+					 sink_streams);
+	if (ret) {
+		dev_err(csi2rx->dev,
+			"Failed to start streams %#llx on subdev\n",
+			sink_streams);
+		goto err_subdev_enable;
+	}
+
+	csi2rx->count++;
+	mutex_unlock(&csi2rx->lock);
+
+	return 0;
+
+err_subdev_enable:
+	if (!csi2rx->count)
+		csi2rx_stop(csi2rx);
+err_stream_start:
 	mutex_unlock(&csi2rx->lock);
+	pm_runtime_put(csi2rx->dev);
+	return ret;
+}
+
+static int csi2rx_disable_streams(struct v4l2_subdev *subdev,
+				  struct v4l2_subdev_state *state, u32 pad,
+				  u64 streams_mask)
+{
+	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
+	struct media_pad *remote_pad;
+	u64 sink_streams;
+
+	sink_streams = v4l2_subdev_state_xlate_streams(state, pad,
+						       CSI2RX_PAD_SINK,
+						       &streams_mask);
+
+	remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
+	if (!remote_pad ||
+	    v4l2_subdev_disable_streams(csi2rx->source_subdev,
+					remote_pad->index, sink_streams)) {
+		dev_err(csi2rx->dev, "Couldn't disable our subdev\n");
+	}
+
+	mutex_lock(&csi2rx->lock);
+	csi2rx->count--;
+	/*
+	 * Let the last user turn off the lights.
+	 */
+	if (!csi2rx->count)
+		csi2rx_stop(csi2rx);
+	mutex_unlock(&csi2rx->lock);
+
+	pm_runtime_put(csi2rx->dev);
+
+	return 0;
+}
+
+static int csi2rx_s_stream_fallback(struct v4l2_subdev *sd, int enable)
+{
+	struct v4l2_subdev_state *state;
+	struct v4l2_subdev_route *route;
+	u64 mask[CSI2RX_PAD_MAX] = {0};
+	int i, ret;
+
+	/* Find the stream mask on all source pads */
+	state = v4l2_subdev_lock_and_get_active_state(sd);
+	for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++) {
+		for_each_active_route(&state->routing, route) {
+			if (route->source_pad == i)
+				mask[i] |= BIT_ULL(route->source_stream);
+		}
+	}
+	v4l2_subdev_unlock_state(state);
+
+	/* Start streaming on each pad */
+	for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++) {
+		if (enable)
+			ret = v4l2_subdev_enable_streams(sd, i, mask[i]);
+		else
+			ret = v4l2_subdev_disable_streams(sd, i, mask[i]);
+		if (ret)
+			return ret;
+	}
+
 	return ret;
 }
 
@@ -446,12 +556,63 @@ static int csi2rx_enum_mbus_code(struct v4l2_subdev *subdev,
 	return 0;
 }
 
+static int _csi2rx_set_routing(struct v4l2_subdev *subdev,
+			       struct v4l2_subdev_state *state,
+			       struct v4l2_subdev_krouting *routing)
+{
+	static const struct v4l2_mbus_framefmt format = {
+		.width = 640,
+		.height = 480,
+		.code = MEDIA_BUS_FMT_UYVY8_1X16,
+		.field = V4L2_FIELD_NONE,
+		.colorspace = V4L2_COLORSPACE_SRGB,
+		.ycbcr_enc = V4L2_YCBCR_ENC_601,
+		.quantization = V4L2_QUANTIZATION_LIM_RANGE,
+		.xfer_func = V4L2_XFER_FUNC_SRGB,
+	};
+	int ret;
+
+	if (routing->num_routes > V4L2_FRAME_DESC_ENTRY_MAX)
+		return -EINVAL;
+
+	ret = v4l2_subdev_routing_validate(subdev, routing,
+					   V4L2_SUBDEV_ROUTING_ONLY_1_TO_1);
+	if (ret)
+		return ret;
+
+	ret = v4l2_subdev_set_routing_with_fmt(subdev, state, routing, &format);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int csi2rx_set_routing(struct v4l2_subdev *subdev,
+			      struct v4l2_subdev_state *state,
+			      enum v4l2_subdev_format_whence which,
+			      struct v4l2_subdev_krouting *routing)
+{
+	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
+	int ret;
+
+	if (which == V4L2_SUBDEV_FORMAT_ACTIVE && csi2rx->count)
+		return -EBUSY;
+
+	ret = _csi2rx_set_routing(subdev, state, routing);
+
+	if (ret)
+		return ret;
+
+	csi2rx_update_vc_select(csi2rx, state);
+
+	return 0;
+}
+
 static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
 			  struct v4l2_subdev_state *state,
 			  struct v4l2_subdev_format *format)
 {
 	struct v4l2_mbus_framefmt *fmt;
-	unsigned int i;
 
 	/* No transcoding, source and sink formats must match. */
 	if (format->pad != CSI2RX_PAD_SINK)
@@ -463,14 +624,19 @@ static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
 	format->format.field = V4L2_FIELD_NONE;
 
 	/* Set sink format */
-	fmt = v4l2_subdev_state_get_format(state, format->pad);
+	fmt = v4l2_subdev_state_get_format(state, format->pad, format->stream);
+	if (!fmt)
+		return -EINVAL;
+
 	*fmt = format->format;
 
-	/* Propagate to source formats */
-	for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++) {
-		fmt = v4l2_subdev_state_get_format(state, i);
-		*fmt = format->format;
-	}
+	/* Propagate to source format */
+	fmt = v4l2_subdev_state_get_opposite_stream_format(state, format->pad,
+							   format->stream);
+	if (!fmt)
+		return -EINVAL;
+
+	*fmt = format->format;
 
 	return 0;
 }
@@ -478,40 +644,92 @@ static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
 static int csi2rx_init_state(struct v4l2_subdev *subdev,
 			     struct v4l2_subdev_state *state)
 {
-	struct v4l2_subdev_format format = {
-		.pad = CSI2RX_PAD_SINK,
-		.format = {
-			.width = 640,
-			.height = 480,
-			.code = MEDIA_BUS_FMT_UYVY8_1X16,
-			.field = V4L2_FIELD_NONE,
-			.colorspace = V4L2_COLORSPACE_SRGB,
-			.ycbcr_enc = V4L2_YCBCR_ENC_601,
-			.quantization = V4L2_QUANTIZATION_LIM_RANGE,
-			.xfer_func = V4L2_XFER_FUNC_SRGB,
+	struct v4l2_subdev_route routes[] = {
+		{
+			.sink_pad = CSI2RX_PAD_SINK,
+			.sink_stream = 0,
+			.source_pad = CSI2RX_PAD_SOURCE_STREAM0,
+			.source_stream = 0,
+			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
 		},
 	};
 
-	return csi2rx_set_fmt(subdev, state, &format);
+	struct v4l2_subdev_krouting routing = {
+		.num_routes = ARRAY_SIZE(routes),
+		.routes = routes,
+	};
+
+	return _csi2rx_set_routing(subdev, state, &routing);
 }
 
 static int csi2rx_get_frame_desc(struct v4l2_subdev *subdev, unsigned int pad,
 				 struct v4l2_mbus_frame_desc *fd)
 {
 	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
+	struct v4l2_mbus_frame_desc source_fd = {0};
+	struct v4l2_subdev_route *route;
+	struct v4l2_subdev_state *state;
+	int ret;
+
+	ret = csi2rx_get_frame_desc_from_source(csi2rx, &source_fd);
+	if (ret)
+		return ret;
+
+	fd->type = V4L2_MBUS_FRAME_DESC_TYPE_CSI2;
 
-	return csi2rx_get_frame_desc_from_source(csi2rx, fd);
+	state = v4l2_subdev_lock_and_get_active_state(subdev);
+
+	for_each_active_route(&state->routing, route) {
+		struct v4l2_mbus_frame_desc_entry *source_entry = NULL;
+		unsigned int i;
+
+		if (route->source_pad != pad)
+			continue;
+
+		for (i = 0; i < source_fd.num_entries; i++) {
+			if (source_fd.entry[i].stream == route->sink_stream) {
+				source_entry = &source_fd.entry[i];
+				break;
+			}
+		}
+
+		if (!source_entry) {
+			dev_err(csi2rx->dev,
+				"Failed to find stream from source frame desc\n");
+			ret = -EPIPE;
+			goto err_missing_stream;
+		}
+
+		fd->entry[fd->num_entries].stream = route->source_stream;
+		fd->entry[fd->num_entries].flags = source_entry->flags;
+		fd->entry[fd->num_entries].length = source_entry->length;
+		fd->entry[fd->num_entries].pixelcode = source_entry->pixelcode;
+		fd->entry[fd->num_entries].bus.csi2.vc =
+			source_entry->bus.csi2.vc;
+		fd->entry[fd->num_entries].bus.csi2.dt =
+			source_entry->bus.csi2.dt;
+
+		fd->num_entries++;
+	}
+
+err_missing_stream:
+	v4l2_subdev_unlock_state(state);
+
+	return ret;
 }
 
 static const struct v4l2_subdev_pad_ops csi2rx_pad_ops = {
-	.enum_mbus_code	= csi2rx_enum_mbus_code,
-	.get_fmt	= v4l2_subdev_get_fmt,
-	.set_fmt	= csi2rx_set_fmt,
-	.get_frame_desc = csi2rx_get_frame_desc,
+	.enum_mbus_code		= csi2rx_enum_mbus_code,
+	.get_fmt		= v4l2_subdev_get_fmt,
+	.set_fmt		= csi2rx_set_fmt,
+	.get_frame_desc		= csi2rx_get_frame_desc,
+	.set_routing		= csi2rx_set_routing,
+	.enable_streams		= csi2rx_enable_streams,
+	.disable_streams	= csi2rx_disable_streams,
 };
 
 static const struct v4l2_subdev_video_ops csi2rx_video_ops = {
-	.s_stream	= csi2rx_s_stream,
+	.s_stream	= csi2rx_s_stream_fallback,
 };
 
 static const struct v4l2_subdev_ops csi2rx_subdev_ops = {
@@ -766,7 +984,8 @@ static int csi2rx_probe(struct platform_device *pdev)
 	csi2rx->pads[CSI2RX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
 	for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++)
 		csi2rx->pads[i].flags = MEDIA_PAD_FL_SOURCE;
-	csi2rx->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+	csi2rx->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE |
+		V4L2_SUBDEV_FL_STREAMS;
 	csi2rx->subdev.entity.ops = &csi2rx_media_ops;
 
 	ret = media_entity_pads_init(&csi2rx->subdev.entity, CSI2RX_PAD_MAX,
Signature · 1 line
2.43.0
#35 lore raw

Re: [PATCH v2 11/13] media: cadence: csi2rx: Enable multi-stream support

Hi Jai

Quoted 55 lines · Jai Luthra

Cadence CSI-2 bridge IP supports capturing multiple virtual "streams" of data over the same physical interface using MIPI Virtual Channels.

The V4L2 subdev APIs should reflect this capability and allow per-stream routing and controls.

While the hardware IP supports usecases where streams coming in the sink pad can be broadcasted to multiple source pads, the driver will need significant re-architecture to make that possible. The two users of this IP in mainline linux are TI Shim and StarFive JH7110 CAMSS, and both have only integrated the first source pad i.e stream0 of this IP. So for now keep it simple and only allow 1-to-1 mapping of streams from sink to source, without any broadcasting.

With stream routing now supported in the driver, implement the enable_stream and disable_stream hooks in place of the stream-unaware s_stream hook.

This allows consumer devices like a DMA bridge or ISP, to enable particular streams on a source pad, which in turn can be used to enable only particular streams on the CSI-TX device connected on the sink pad.

Implement a fallback s_stream hook that internally calls enable_stream on each source pad, for consumer drivers that don't use multi-stream APIs to still work.

---
 drivers/media/platform/cadence/cdns-csi2rx.c | 407 ++++++++++++++++++++-------
 1 file changed, 313 insertions(+), 94 deletions(-)
drivers/media/platform/cadence/cdns-csi2rx.c +3 -4
index 2ec34fc9c524..b0c91a9c65e8 100644
@@ -90,6 +90,7 @@ struct csi2rx_priv {
 	struct reset_control		*pixel_rst[CSI2RX_STREAMS_MAX];
 	struct phy			*dphy;
 
+	u32				vc_select[CSI2RX_STREAMS_MAX];
 	u8				lanes[CSI2RX_LANES_MAX];
 	u8				num_lanes;
 	u8				max_lanes;
@@ -179,27 +180,43 @@ static void csi2rx_reset(struct csi2rx_priv *csi2rx)
 
 static int csi2rx_configure_ext_dphy(struct csi2rx_priv *csi2rx)
 {
+	struct v4l2_ctrl_handler *handler = csi2rx->source_subdev->ctrl_handler;
 	union phy_configure_opts opts = { };
 	struct phy_configure_opts_mipi_dphy *cfg = &opts.mipi_dphy;
-	struct v4l2_subdev_format sd_fmt = {
-		.which	= V4L2_SUBDEV_FORMAT_ACTIVE,
-		.pad	= CSI2RX_PAD_SINK,
-	};
+	struct v4l2_mbus_framefmt *framefmt;

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_reset(struct csi2rx_priv *csi2rx) · line 192

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +2 -4
+	struct v4l2_subdev_state *state;
 	const struct csi2rx_fmt *fmt;
 	s64 link_freq;
 	int ret;
 
-	ret = v4l2_subdev_call_state_active(&csi2rx->subdev, pad, get_fmt,
-					    &sd_fmt);
-	if (ret < 0)
-		return ret;
+	if (v4l2_ctrl_find(handler, V4L2_CID_LINK_FREQ)) {

Do you need to do this by yourself ? afaict v4l2_get_link_freq() already checks if V4L2_CID_LINK_FREQ is available, and if not, fallsback to use PIXEL_RATE.

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_reset(struct csi2rx_priv *csi2rx) · line 204

Quoted 12 lines
drivers/media/platform/cadence/cdns-csi2rx.c +12 -0
+		link_freq = v4l2_get_link_freq(handler, 0, 0);
+	} else {
+		state = v4l2_subdev_get_locked_active_state(&csi2rx->subdev);
+		framefmt = v4l2_subdev_state_get_format(state, CSI2RX_PAD_SINK,
+							0);
+
+		if (framefmt) {
+			fmt = csi2rx_get_fmt_by_code(framefmt->code);
+		} else {
+			dev_err(csi2rx->dev,
+				"Did not find active sink format\n");
+			return -EINVAL;

Is this possibile ?

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_reset(struct csi2rx_priv *csi2rx) · line 208

Quoted 5 lines
drivers/media/platform/cadence/cdns-csi2rx.c +3 -1
+		}
 
-	fmt = csi2rx_get_fmt_by_code(sd_fmt.format.code);
+		link_freq = v4l2_get_link_freq(handler, fmt->bpp,
+					       2 * csi2rx->num_lanes);

Do we want to allow falling back on PIXEL_RATE for multiplexed transmitters ? I presume this will give you invalid results anyway.

I would simply call v4l2_get_link_freq(handler, 0, 0) to force the usage of LINK_FREQ which will become mandatory for transmitters

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_reset(struct csi2rx_priv *csi2rx) · line 213

Quoted 8 lines
drivers/media/platform/cadence/cdns-csi2rx.c +4 -3
-	link_freq = v4l2_get_link_freq(csi2rx->source_subdev->ctrl_handler,
-				       fmt->bpp, 2 * csi2rx->num_lanes);
-	if (link_freq < 0)
+		dev_warn(csi2rx->dev,
+			 "Guessing link frequency using bitdepth of stream 0.\n");
+		dev_warn(csi2rx->dev,
+			 "V4L2_CID_LINK_FREQ control is required for multi format sources.\n");

Exactly :)

Quoted 92 lines
drivers/media/platform/cadence/cdns-csi2rx.c +9 -31
+	}
+
+	if (link_freq < 0) {
+		dev_err(csi2rx->dev, "Unable to calculate link frequency\n");
 		return link_freq;
+	}
 
 	ret = phy_mipi_dphy_get_default_config_for_hsclk(link_freq,
 							 csi2rx->num_lanes, cfg);
@@ -222,18 +239,10 @@ static int csi2rx_configure_ext_dphy(struct csi2rx_priv *csi2rx)
 static int csi2rx_start(struct csi2rx_priv *csi2rx)
 {
 	unsigned int i;
-	struct media_pad *remote_pad;
 	unsigned long lanes_used = 0;
 	u32 reg;
 	int ret;
 
-	remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
-	if (!remote_pad) {
-		dev_err(csi2rx->dev,
-			"Failed to find connected source\n");
-		return -ENODEV;
-	}
-
 	ret = clk_prepare_enable(csi2rx->p_clk);
 	if (ret)
 		return ret;
@@ -300,11 +309,7 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
 		writel(CSI2RX_STREAM_CFG_FIFO_MODE_LARGE_BUF,
 		       csi2rx->base + CSI2RX_STREAM_CFG_REG(i));
 
-		/*
-		 * Enable one virtual channel. When multiple virtual channels
-		 * are supported this will have to be changed.
-		 */
-		writel(CSI2RX_STREAM_DATA_CFG_VC_SELECT(0),
+		writel(csi2rx->vc_select[i],
 		       csi2rx->base + CSI2RX_STREAM_DATA_CFG_REG(i));
 
 		writel(CSI2RX_STREAM_CTRL_START,
@@ -317,17 +322,10 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
 
 	reset_control_deassert(csi2rx->sys_rst);
 
-	ret = v4l2_subdev_enable_streams(csi2rx->source_subdev,
-					 remote_pad->index, BIT(0));
-	if (ret)
-		goto err_disable_sysclk;
-
 	clk_disable_unprepare(csi2rx->p_clk);
 
 	return 0;
 
-err_disable_sysclk:
-	clk_disable_unprepare(csi2rx->sys_clk);
 err_disable_pixclk:
 	for (; i > 0; i--) {
 		reset_control_assert(csi2rx->pixel_rst[i - 1]);
@@ -346,7 +344,6 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
 
 static void csi2rx_stop(struct csi2rx_priv *csi2rx)
 {
-	struct media_pad *remote_pad;
 	unsigned int i;
 	u32 val;
 	int ret;
@@ -375,13 +372,6 @@ static void csi2rx_stop(struct csi2rx_priv *csi2rx)
 	reset_control_assert(csi2rx->p_rst);
 	clk_disable_unprepare(csi2rx->p_clk);
 
-	remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
-	if (!remote_pad ||
-	    v4l2_subdev_disable_streams(csi2rx->source_subdev,
-					remote_pad->index, BIT(0))) {
-		dev_warn(csi2rx->dev, "Couldn't disable our subdev\n");
-	}
-
 	if (csi2rx->dphy) {
 		writel(0, csi2rx->base + CSI2RX_DPHY_LANE_CTRL_REG);
 
@@ -390,47 +380,167 @@ static void csi2rx_stop(struct csi2rx_priv *csi2rx)
 	}
 }
 
-static int csi2rx_s_stream(struct v4l2_subdev *subdev, int enable)
+static void csi2rx_update_vc_select(struct csi2rx_priv *csi2rx,
+				    struct v4l2_subdev_state *state)
 {
-	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
-	int ret = 0;
+	struct v4l2_mbus_frame_desc fd = {0};

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_stop(struct csi2rx_priv *csi2rx) · line 392

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +5 -4
+	struct v4l2_subdev_route *route;
+	unsigned int i;
+	int ret;
 
-	if (enable) {
-		ret = pm_runtime_resume_and_get(csi2rx->dev);
-		if (ret < 0)
-			return ret;
+	for (i = 0; i < CSI2RX_STREAMS_MAX; i++)
+		csi2rx->vc_select[i] = 0;

or just memset ?

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_stop(struct csi2rx_priv *csi2rx) · line 397

Quoted 5 lines
drivers/media/platform/cadence/cdns-csi2rx.c +5 -0
+
+	ret = csi2rx_get_frame_desc_from_source(csi2rx, &fd);
+	if (ret || fd.type != V4L2_MBUS_FRAME_DESC_TYPE_CSI2) {
+		dev_dbg(csi2rx->dev,
+			"Failed to get source frame desc, allowing only VC=0\n");

I'm still not sure if picking a default fallback is a good idea or not, as this will only work by chance

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_stop(struct csi2rx_priv *csi2rx) · line 403

Quoted 7 lines
drivers/media/platform/cadence/cdns-csi2rx.c +4 -1
+		goto err_no_fd;
 	}
 
-	mutex_lock(&csi2rx->lock);
+	/* If source provides per-stream VC info, use it to filter by VC */
+	for_each_active_route(&state->routing, route) {
+		int cdns_stream = route->source_pad - CSI2RX_PAD_SOURCE_STREAM0;

unsigned maybe ?

Quoted 9 lines
drivers/media/platform/cadence/cdns-csi2rx.c +1 -7
+		u8 used_vc = 0;
 
-	if (enable) {
-		/*
-		 * If we're not the first users, there's no need to
-		 * enable the whole controller.
-		 */
-		if (!csi2rx->count) {
-			ret = csi2rx_start(csi2rx);

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_stop(struct csi2rx_priv *csi2rx) · line 412

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +5 -3
-			if (ret) {
-				pm_runtime_put(csi2rx->dev);
-				goto out;
+		for (i = 0; i < fd.num_entries; i++) {
+			if (fd.entry[i].stream == route->sink_stream) {
+				used_vc = fd.entry[i].bus.csi2.vc;
+				break;
 			}
 		}
+		csi2rx->vc_select[cdns_stream] |=

Why |= ? do you expect to be able to capture multiple VC per stream ? I don't think that's allowed ?

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_stop(struct csi2rx_priv *csi2rx) · line 413

Quoted 1 line
drivers/media/platform/cadence/cdns-csi2rx.c +1 -0
+			CSI2RX_STREAM_DATA_CFG_VC_SELECT(used_vc);

In case no stream matching route->sink_stream is found in the remote's frame descriptor this sets CSI2RX_STREAM_DATA_CFG_VC_SELECT(0) for vc_select[x]...

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_stop(struct csi2rx_priv *csi2rx) · line 420

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +6 -3
+	}
 
-		csi2rx->count++;
-	} else {
-		csi2rx->count--;
+err_no_fd:
+	for (i = 0; i < CSI2RX_STREAMS_MAX; i++) {
+		if (!csi2rx->vc_select[i]) {
+			csi2rx->vc_select[i] =
+				CSI2RX_STREAM_DATA_CFG_VC_SELECT(0);

... so why doing it again here (assuming the no_fd case should be dropped).

Also, setting multiple streams to capture on VC=0 will multiplex the same stream or cause troubles ?

Also, as you have an initial loop already this could maybe be:

        /* Initialize all streams to capture VC 0 by default. */
	for (i = 0; i < CSI2RX_STREAMS_MAX; i++)
		csi2rx->vc_select[i] =  CSI2RX_STREAM_DATA_CFG_VC_SELECT(0);

	ret = csi2rx_get_frame_desc_from_source(csi2rx, &fd);
	if (ret || fd.type != V4L2_MBUS_FRAME_DESC_TYPE_CSI2) {
		dev_dbg(csi2rx->dev,
			"Failed to get source frame desc, allowing only VC=0\n");
		return;
	}

	/* If source provides per-stream VC info, use it to filter by VC */
	for_each_active_route(&state->routing, route) {
		int cdns_stream = route->source_pad - CSI2RX_PAD_SOURCE_STREAM0;

		for (i = 0; i < fd.num_entries; i++) {
			if (fd.entry[i].stream != route->sink_stream)
                                continue;

                        csi2rx->vc_select[cdns_stream] =
                                CSI2RX_STREAM_DATA_CFG_VC_SELECT(fd.entry[i].bus.csi2.vc);
		}
	}
Quoted 33 lines
drivers/media/platform/cadence/cdns-csi2rx.c +26 -5
+		}
+	}
+}
 
-		/*
-		 * Let the last user turn off the lights.
-		 */
-		if (!csi2rx->count)
-			csi2rx_stop(csi2rx);
+static int csi2rx_enable_streams(struct v4l2_subdev *subdev,
+				 struct v4l2_subdev_state *state, u32 pad,
+				 u64 streams_mask)
+{
+	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
+	struct media_pad *remote_pad;
+	u64 sink_streams;
+	int ret;
+
+	remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
+	if (!remote_pad) {
+		dev_err(csi2rx->dev,
+			"Failed to find connected source\n");
+		return -ENODEV;
+	}
+
+	ret = pm_runtime_resume_and_get(csi2rx->dev);
+	if (ret < 0)
+		return ret;
+
+	sink_streams = v4l2_subdev_state_xlate_streams(state, pad,
+						       CSI2RX_PAD_SINK,
+						       &streams_mask);
 

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_stop(struct csi2rx_priv *csi2rx) · line 457

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +9 -1
-		pm_runtime_put(csi2rx->dev);
+	mutex_lock(&csi2rx->lock);
+	/*
+	 * If we're not the first users, there's no need to
+	 * enable the whole controller.
+	 */
+	if (!csi2rx->count) {
+		ret = csi2rx_start(csi2rx);
+		if (ret)
+			goto err_stream_start;

I'm not 100% familiar with this new API but I was expecting you wouldn't need to do so

Quoted 102 lines
drivers/media/platform/cadence/cdns-csi2rx.c +91 -1
 	}
 
-out:
+	/* Start streaming on the source */
+	ret = v4l2_subdev_enable_streams(csi2rx->source_subdev, remote_pad->index,
+					 sink_streams);
+	if (ret) {
+		dev_err(csi2rx->dev,
+			"Failed to start streams %#llx on subdev\n",
+			sink_streams);
+		goto err_subdev_enable;
+	}
+
+	csi2rx->count++;
+	mutex_unlock(&csi2rx->lock);
+
+	return 0;
+
+err_subdev_enable:
+	if (!csi2rx->count)
+		csi2rx_stop(csi2rx);
+err_stream_start:
 	mutex_unlock(&csi2rx->lock);
+	pm_runtime_put(csi2rx->dev);
+	return ret;
+}
+
+static int csi2rx_disable_streams(struct v4l2_subdev *subdev,
+				  struct v4l2_subdev_state *state, u32 pad,
+				  u64 streams_mask)
+{
+	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
+	struct media_pad *remote_pad;
+	u64 sink_streams;
+
+	sink_streams = v4l2_subdev_state_xlate_streams(state, pad,
+						       CSI2RX_PAD_SINK,
+						       &streams_mask);
+
+	remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
+	if (!remote_pad ||
+	    v4l2_subdev_disable_streams(csi2rx->source_subdev,
+					remote_pad->index, sink_streams)) {
+		dev_err(csi2rx->dev, "Couldn't disable our subdev\n");
+	}
+
+	mutex_lock(&csi2rx->lock);
+	csi2rx->count--;
+	/*
+	 * Let the last user turn off the lights.
+	 */
+	if (!csi2rx->count)
+		csi2rx_stop(csi2rx);
+	mutex_unlock(&csi2rx->lock);
+
+	pm_runtime_put(csi2rx->dev);
+
+	return 0;
+}
+
+static int csi2rx_s_stream_fallback(struct v4l2_subdev *sd, int enable)
+{
+	struct v4l2_subdev_state *state;
+	struct v4l2_subdev_route *route;
+	u64 mask[CSI2RX_PAD_MAX] = {0};
+	int i, ret;
+
+	/* Find the stream mask on all source pads */
+	state = v4l2_subdev_lock_and_get_active_state(sd);
+	for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++) {
+		for_each_active_route(&state->routing, route) {
+			if (route->source_pad == i)
+				mask[i] |= BIT_ULL(route->source_stream);
+		}
+	}
+	v4l2_subdev_unlock_state(state);
+
+	/* Start streaming on each pad */
+	for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++) {
+		if (enable)
+			ret = v4l2_subdev_enable_streams(sd, i, mask[i]);
+		else
+			ret = v4l2_subdev_disable_streams(sd, i, mask[i]);
+		if (ret)
+			return ret;
+	}
+
 	return ret;
 }
 
@@ -446,12 +556,63 @@ static int csi2rx_enum_mbus_code(struct v4l2_subdev *subdev,
 	return 0;
 }
 
+static int _csi2rx_set_routing(struct v4l2_subdev *subdev,
+			       struct v4l2_subdev_state *state,
+			       struct v4l2_subdev_krouting *routing)
+{
+	static const struct v4l2_mbus_framefmt format = {
+		.width = 640,
+		.height = 480,
+		.code = MEDIA_BUS_FMT_UYVY8_1X16,

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_enum_mbus_code(struct v4l2_subdev *subdev, · line 576

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +10 -0
+		.field = V4L2_FIELD_NONE,
+		.colorspace = V4L2_COLORSPACE_SRGB,
+		.ycbcr_enc = V4L2_YCBCR_ENC_601,
+		.quantization = V4L2_QUANTIZATION_LIM_RANGE,
+		.xfer_func = V4L2_XFER_FUNC_SRGB,
+	};
+	int ret;
+
+	if (routing->num_routes > V4L2_FRAME_DESC_ENTRY_MAX)
+		return -EINVAL;

Isn't there a max number of routes supported by the HW ?

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_enum_mbus_code(struct v4l2_subdev *subdev, · line 583

Quoted 7 lines
drivers/media/platform/cadence/cdns-csi2rx.c +7 -0
+
+	ret = v4l2_subdev_routing_validate(subdev, routing,
+					   V4L2_SUBDEV_ROUTING_ONLY_1_TO_1);
+	if (ret)
+		return ret;
+
+	ret = v4l2_subdev_set_routing_with_fmt(subdev, state, routing, &format);
        return ...
Quoted 9 lines
drivers/media/platform/cadence/cdns-csi2rx.c +9 -0
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int csi2rx_set_routing(struct v4l2_subdev *subdev,
+			      struct v4l2_subdev_state *state,
+			      enum v4l2_subdev_format_whence which,

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_enum_mbus_code(struct v4l2_subdev *subdev, · line 602

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +10 -0
+			      struct v4l2_subdev_krouting *routing)
+{
+	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
+	int ret;
+
+	if (which == V4L2_SUBDEV_FORMAT_ACTIVE && csi2rx->count)
+		return -EBUSY;
+
+	ret = _csi2rx_set_routing(subdev, state, routing);
+

no empty line

Quoted 28 lines
drivers/media/platform/cadence/cdns-csi2rx.c +12 -2
+	if (ret)
+		return ret;
+
+	csi2rx_update_vc_select(csi2rx, state);
+
+	return 0;
+}
+
 static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
 			  struct v4l2_subdev_state *state,
 			  struct v4l2_subdev_format *format)
 {
 	struct v4l2_mbus_framefmt *fmt;
-	unsigned int i;
 
 	/* No transcoding, source and sink formats must match. */
 	if (format->pad != CSI2RX_PAD_SINK)
@@ -463,14 +624,19 @@ static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
 	format->format.field = V4L2_FIELD_NONE;
 
 	/* Set sink format */
-	fmt = v4l2_subdev_state_get_format(state, format->pad);
+	fmt = v4l2_subdev_state_get_format(state, format->pad, format->stream);
+	if (!fmt)
+		return -EINVAL;
+
 	*fmt = format->format;
 

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_set_fmt(struct v4l2_subdev *subdev, · line 637

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +5 -5
-	/* Propagate to source formats */
-	for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++) {
-		fmt = v4l2_subdev_state_get_format(state, i);
-		*fmt = format->format;
-	}
+	/* Propagate to source format */
+	fmt = v4l2_subdev_state_get_opposite_stream_format(state, format->pad,
+							   format->stream);
+	if (!fmt)
+		return -EINVAL;

again I'm not sure this can happen

Quoted 11 lines
drivers/media/platform/cadence/cdns-csi2rx.c +2 -2
+
+	*fmt = format->format;
 
 	return 0;
 }
@@ -478,40 +644,92 @@ static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
 static int csi2rx_init_state(struct v4l2_subdev *subdev,
 			     struct v4l2_subdev_state *state)
 {
-	struct v4l2_subdev_format format = {
-		.pad = CSI2RX_PAD_SINK,

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_set_fmt(struct v4l2_subdev *subdev, · line 647

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +1 -9
-		.format = {
-			.width = 640,
-			.height = 480,
-			.code = MEDIA_BUS_FMT_UYVY8_1X16,
-			.field = V4L2_FIELD_NONE,
-			.colorspace = V4L2_COLORSPACE_SRGB,
-			.ycbcr_enc = V4L2_YCBCR_ENC_601,
-			.quantization = V4L2_QUANTIZATION_LIM_RANGE,
-			.xfer_func = V4L2_XFER_FUNC_SRGB,
+	struct v4l2_subdev_route routes[] = {

static const ?

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_set_fmt(struct v4l2_subdev *subdev, · line 657

Quoted 11 lines
drivers/media/platform/cadence/cdns-csi2rx.c +7 -1
+		{
+			.sink_pad = CSI2RX_PAD_SINK,
+			.sink_stream = 0,
+			.source_pad = CSI2RX_PAD_SOURCE_STREAM0,
+			.source_stream = 0,
+			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
 		},
 	};
 
-	return csi2rx_set_fmt(subdev, state, &format);
+	struct v4l2_subdev_krouting routing = {

static const ?

Quoted 7 lines
drivers/media/platform/cadence/cdns-csi2rx.c +5 -0
+		.num_routes = ARRAY_SIZE(routes),
+		.routes = routes,
+	};
+
+	return _csi2rx_set_routing(subdev, state, &routing);
 }
 

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_set_fmt(struct v4l2_subdev *subdev, · line 674

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +6 -0
 static int csi2rx_get_frame_desc(struct v4l2_subdev *subdev, unsigned int pad,
 				 struct v4l2_mbus_frame_desc *fd)
 {
 	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
+	struct v4l2_mbus_frame_desc source_fd = {0};
+	struct v4l2_subdev_route *route;
+	struct v4l2_subdev_state *state;
+	int ret;
+
+	ret = csi2rx_get_frame_desc_from_source(csi2rx, &source_fd);

Should this inspect the remote frame desc or only the local routing table ?

Quoted 29 lines
drivers/media/platform/cadence/cdns-csi2rx.c +27 -1
+	if (ret)
+		return ret;
+
+	fd->type = V4L2_MBUS_FRAME_DESC_TYPE_CSI2;
 
-	return csi2rx_get_frame_desc_from_source(csi2rx, fd);
+	state = v4l2_subdev_lock_and_get_active_state(subdev);
+
+	for_each_active_route(&state->routing, route) {
+		struct v4l2_mbus_frame_desc_entry *source_entry = NULL;
+		unsigned int i;
+
+		if (route->source_pad != pad)
+			continue;
+
+		for (i = 0; i < source_fd.num_entries; i++) {
+			if (source_fd.entry[i].stream == route->sink_stream) {
+				source_entry = &source_fd.entry[i];
+				break;
+			}
+		}
+
+		if (!source_entry) {
+			dev_err(csi2rx->dev,
+				"Failed to find stream from source frame desc\n");
+			ret = -EPIPE;
+			goto err_missing_stream;
+		}
+

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_set_fmt(struct v4l2_subdev *subdev, · line 712

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +10 -0
+		fd->entry[fd->num_entries].stream = route->source_stream;
+		fd->entry[fd->num_entries].flags = source_entry->flags;
+		fd->entry[fd->num_entries].length = source_entry->length;
+		fd->entry[fd->num_entries].pixelcode = source_entry->pixelcode;
+		fd->entry[fd->num_entries].bus.csi2.vc =
+			source_entry->bus.csi2.vc;
+		fd->entry[fd->num_entries].bus.csi2.dt =
+			source_entry->bus.csi2.dt;
+
+		fd->num_entries++;

as you

		if (route->source_pad != pad)
			continue;

can you have multiple entris on a pad ?

Quoted 29 lines
drivers/media/platform/cadence/cdns-csi2rx.c +14 -5
+	}
+
+err_missing_stream:
+	v4l2_subdev_unlock_state(state);
+
+	return ret;
 }
 
 static const struct v4l2_subdev_pad_ops csi2rx_pad_ops = {
-	.enum_mbus_code	= csi2rx_enum_mbus_code,
-	.get_fmt	= v4l2_subdev_get_fmt,
-	.set_fmt	= csi2rx_set_fmt,
-	.get_frame_desc = csi2rx_get_frame_desc,
+	.enum_mbus_code		= csi2rx_enum_mbus_code,
+	.get_fmt		= v4l2_subdev_get_fmt,
+	.set_fmt		= csi2rx_set_fmt,
+	.get_frame_desc		= csi2rx_get_frame_desc,
+	.set_routing		= csi2rx_set_routing,
+	.enable_streams		= csi2rx_enable_streams,
+	.disable_streams	= csi2rx_disable_streams,
 };
 
 static const struct v4l2_subdev_video_ops csi2rx_video_ops = {
-	.s_stream	= csi2rx_s_stream,
+	.s_stream	= csi2rx_s_stream_fallback,
 };
 
 static const struct v4l2_subdev_ops csi2rx_subdev_ops = {

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_probe(struct platform_device *pdev) · line 988

Quoted 6 lines
drivers/media/platform/cadence/cdns-csi2rx.c +2 -1
@@ -766,7 +984,8 @@ static int csi2rx_probe(struct platform_device *pdev)
 	csi2rx->pads[CSI2RX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
 	for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++)
 		csi2rx->pads[i].flags = MEDIA_PAD_FL_SOURCE;
-	csi2rx->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+	csi2rx->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE |
+		V4L2_SUBDEV_FL_STREAMS;

nit: please align to the previous one

	csi2rx->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE |
		                V4L2_SUBDEV_FL_STREAMS;

and maybe the |= should be just an =

Thanks
j

Quoted 8 lines
drivers/media/platform/cadence/cdns-csi2rx.c +0 -1
 	csi2rx->subdev.entity.ops = &csi2rx_media_ops;
 
 	ret = media_entity_pads_init(&csi2rx->subdev.entity, CSI2RX_PAD_MAX,
 
--

2.43.0

author
#36 lore raw

Re: Re: [PATCH v2 11/13] media: cadence: csi2rx: Enable multi-stream support

Hi Jacopo,

Thanks for the review.

Quoted 88 lines · Jacopo Mondi

Hi Jai

Quoted 55 lines · Jai Luthra

Cadence CSI-2 bridge IP supports capturing multiple virtual "streams" of data over the same physical interface using MIPI Virtual Channels.

The V4L2 subdev APIs should reflect this capability and allow per-stream routing and controls.

While the hardware IP supports usecases where streams coming in the sink pad can be broadcasted to multiple source pads, the driver will need significant re-architecture to make that possible. The two users of this IP in mainline linux are TI Shim and StarFive JH7110 CAMSS, and both have only integrated the first source pad i.e stream0 of this IP. So for now keep it simple and only allow 1-to-1 mapping of streams from sink to source, without any broadcasting.

With stream routing now supported in the driver, implement the enable_stream and disable_stream hooks in place of the stream-unaware s_stream hook.

This allows consumer devices like a DMA bridge or ISP, to enable particular streams on a source pad, which in turn can be used to enable only particular streams on the CSI-TX device connected on the sink pad.

Implement a fallback s_stream hook that internally calls enable_stream on each source pad, for consumer drivers that don't use multi-stream APIs to still work.

---
 drivers/media/platform/cadence/cdns-csi2rx.c | 407 ++++++++++++++++++++-------
 1 file changed, 313 insertions(+), 94 deletions(-)
drivers/media/platform/cadence/cdns-csi2rx.c +3 -4
index 2ec34fc9c524..b0c91a9c65e8 100644
@@ -90,6 +90,7 @@ struct csi2rx_priv {
 	struct reset_control		*pixel_rst[CSI2RX_STREAMS_MAX];
 	struct phy			*dphy;
 
+	u32				vc_select[CSI2RX_STREAMS_MAX];
 	u8				lanes[CSI2RX_LANES_MAX];
 	u8				num_lanes;
 	u8				max_lanes;
@@ -179,27 +180,43 @@ static void csi2rx_reset(struct csi2rx_priv *csi2rx)
 
 static int csi2rx_configure_ext_dphy(struct csi2rx_priv *csi2rx)
 {
+	struct v4l2_ctrl_handler *handler = csi2rx->source_subdev->ctrl_handler;
 	union phy_configure_opts opts = { };
 	struct phy_configure_opts_mipi_dphy *cfg = &opts.mipi_dphy;
-	struct v4l2_subdev_format sd_fmt = {
-		.which	= V4L2_SUBDEV_FORMAT_ACTIVE,
-		.pad	= CSI2RX_PAD_SINK,
-	};
+	struct v4l2_mbus_framefmt *framefmt;

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_reset(struct csi2rx_priv *csi2rx) · line 192

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +2 -4
+	struct v4l2_subdev_state *state;
 	const struct csi2rx_fmt *fmt;
 	s64 link_freq;
 	int ret;
 
-	ret = v4l2_subdev_call_state_active(&csi2rx->subdev, pad, get_fmt,
-					    &sd_fmt);
-	if (ret < 0)
-		return ret;
+	if (v4l2_ctrl_find(handler, V4L2_CID_LINK_FREQ)) {

Do you need to do this by yourself ? afaict v4l2_get_link_freq() already checks if V4L2_CID_LINK_FREQ is available, and if not, fallsback to use PIXEL_RATE.

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_reset(struct csi2rx_priv *csi2rx) · line 204

Quoted 12 lines
drivers/media/platform/cadence/cdns-csi2rx.c +12 -0
+		link_freq = v4l2_get_link_freq(handler, 0, 0);
+	} else {
+		state = v4l2_subdev_get_locked_active_state(&csi2rx->subdev);
+		framefmt = v4l2_subdev_state_get_format(state, CSI2RX_PAD_SINK,
+							0);
+
+		if (framefmt) {
+			fmt = csi2rx_get_fmt_by_code(framefmt->code);
+		} else {
+			dev_err(csi2rx->dev,
+				"Did not find active sink format\n");
+			return -EINVAL;

Is this possibile ?

Quoted 8 lines

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_reset(struct csi2rx_priv *csi2rx) · line 208

Quoted 5 lines
drivers/media/platform/cadence/cdns-csi2rx.c +3 -1
+		}
 
-	fmt = csi2rx_get_fmt_by_code(sd_fmt.format.code);
+		link_freq = v4l2_get_link_freq(handler, fmt->bpp,
+					       2 * csi2rx->num_lanes);

Do we want to allow falling back on PIXEL_RATE for multiplexed transmitters ? I presume this will give you invalid results anyway.

This is mostly done to avoid breaking any single stream sensor that does not have the LINK_FREQ control, and was working with this bridge before. Thus the warning below for multi-format sources.

Quoted 3 lines

I would simply call v4l2_get_link_freq(handler, 0, 0) to force the usage of LINK_FREQ which will become mandatory for transmitters

Ah I did not know LINK_FREQ will be mandatory soon. Any threads I can look at where this was discussed?

Quoted 108 lines

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_reset(struct csi2rx_priv *csi2rx) · line 213

Quoted 9 lines
drivers/media/platform/cadence/cdns-csi2rx.c +4 -3
-	link_freq = v4l2_get_link_freq(csi2rx->source_subdev->ctrl_handler,
-				       fmt->bpp, 2 * csi2rx->num_lanes);
-	if (link_freq < 0)
+		dev_warn(csi2rx->dev,
+			 "Guessing link frequency using bitdepth of stream 0.\n");
+		dev_warn(csi2rx->dev,
+			 "V4L2_CID_LINK_FREQ control is required for multi format sources.\n");

Exactly :)

Quoted 96 lines
drivers/media/platform/cadence/cdns-csi2rx.c +12 -31
+	}
+
+	if (link_freq < 0) {
+		dev_err(csi2rx->dev, "Unable to calculate link frequency\n");
 		return link_freq;
+	}
 
 	ret = phy_mipi_dphy_get_default_config_for_hsclk(link_freq,
 							 csi2rx->num_lanes, cfg);
@@ -222,18 +239,10 @@ static int csi2rx_configure_ext_dphy(struct csi2rx_priv *csi2rx)
 static int csi2rx_start(struct csi2rx_priv *csi2rx)
 {
 	unsigned int i;
-	struct media_pad *remote_pad;
 	unsigned long lanes_used = 0;
 	u32 reg;
 	int ret;
 
-	remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
-	if (!remote_pad) {
-		dev_err(csi2rx->dev,
-			"Failed to find connected source\n");
-		return -ENODEV;
-	}
-
 	ret = clk_prepare_enable(csi2rx->p_clk);
 	if (ret)
 		return ret;
@@ -300,11 +309,7 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
 		writel(CSI2RX_STREAM_CFG_FIFO_MODE_LARGE_BUF,
 		       csi2rx->base + CSI2RX_STREAM_CFG_REG(i));
 
-		/*
-		 * Enable one virtual channel. When multiple virtual channels
-		 * are supported this will have to be changed.
-		 */
-		writel(CSI2RX_STREAM_DATA_CFG_VC_SELECT(0),
+		writel(csi2rx->vc_select[i],
 		       csi2rx->base + CSI2RX_STREAM_DATA_CFG_REG(i));
 
 		writel(CSI2RX_STREAM_CTRL_START,
@@ -317,17 +322,10 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
 
 	reset_control_deassert(csi2rx->sys_rst);
 
-	ret = v4l2_subdev_enable_streams(csi2rx->source_subdev,
-					 remote_pad->index, BIT(0));
-	if (ret)
-		goto err_disable_sysclk;
-
 	clk_disable_unprepare(csi2rx->p_clk);
 
 	return 0;
 
-err_disable_sysclk:
-	clk_disable_unprepare(csi2rx->sys_clk);
 err_disable_pixclk:
 	for (; i > 0; i--) {
 		reset_control_assert(csi2rx->pixel_rst[i - 1]);
@@ -346,7 +344,6 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
 
 static void csi2rx_stop(struct csi2rx_priv *csi2rx)
 {
-	struct media_pad *remote_pad;
 	unsigned int i;
 	u32 val;
 	int ret;
@@ -375,13 +372,6 @@ static void csi2rx_stop(struct csi2rx_priv *csi2rx)
 	reset_control_assert(csi2rx->p_rst);
 	clk_disable_unprepare(csi2rx->p_clk);
 
-	remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
-	if (!remote_pad ||
-	    v4l2_subdev_disable_streams(csi2rx->source_subdev,
-					remote_pad->index, BIT(0))) {
-		dev_warn(csi2rx->dev, "Couldn't disable our subdev\n");
-	}
-
 	if (csi2rx->dphy) {
 		writel(0, csi2rx->base + CSI2RX_DPHY_LANE_CTRL_REG);
 
@@ -390,47 +380,167 @@ static void csi2rx_stop(struct csi2rx_priv *csi2rx)
 	}
 }
 
-static int csi2rx_s_stream(struct v4l2_subdev *subdev, int enable)
+static void csi2rx_update_vc_select(struct csi2rx_priv *csi2rx,
+				    struct v4l2_subdev_state *state)
 {
-	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
-	int ret = 0;
+	struct v4l2_mbus_frame_desc fd = {0};
+	struct v4l2_subdev_route *route;
+	unsigned int i;
+	int ret;
 
Quoted 8 lines

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_stop(struct csi2rx_priv *csi2rx) · line 392

Quoted 6 lines
drivers/media/platform/cadence/cdns-csi2rx.c +2 -4
-	if (enable) {
-		ret = pm_runtime_resume_and_get(csi2rx->dev);
-		if (ret < 0)
-			return ret;
+	for (i = 0; i < CSI2RX_STREAMS_MAX; i++)
+		csi2rx->vc_select[i] = 0;

or just memset ?

Will do.

Quoted 13 lines

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_stop(struct csi2rx_priv *csi2rx) · line 397

Quoted 6 lines
drivers/media/platform/cadence/cdns-csi2rx.c +5 -0
+
+	ret = csi2rx_get_frame_desc_from_source(csi2rx, &fd);
+	if (ret || fd.type != V4L2_MBUS_FRAME_DESC_TYPE_CSI2) {
+		dev_dbg(csi2rx->dev,
+			"Failed to get source frame desc, allowing only VC=0\n");

I'm still not sure if picking a default fallback is a good idea or not, as this will only work by chance

Quoted 3 lines
drivers/media/platform/cadence/cdns-csi2rx.c +1 -0
+		goto err_no_fd;
 	}
 
Quoted 6 lines

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_stop(struct csi2rx_priv *csi2rx) · line 403

Quoted 4 lines
drivers/media/platform/cadence/cdns-csi2rx.c +3 -1
-	mutex_lock(&csi2rx->lock);
+	/* If source provides per-stream VC info, use it to filter by VC */
+	for_each_active_route(&state->routing, route) {
+		int cdns_stream = route->source_pad - CSI2RX_PAD_SOURCE_STREAM0;

unsigned maybe ?

Will fix.

Quoted 15 lines
Quoted 15 lines
drivers/media/platform/cadence/cdns-csi2rx.c +3 -10
+		u8 used_vc = 0;
 
-	if (enable) {
-		/*
-		 * If we're not the first users, there's no need to
-		 * enable the whole controller.
-		 */
-		if (!csi2rx->count) {
-			ret = csi2rx_start(csi2rx);
-			if (ret) {
-				pm_runtime_put(csi2rx->dev);
-				goto out;
+		for (i = 0; i < fd.num_entries; i++) {
+			if (fd.entry[i].stream == route->sink_stream) {
Quoted 8 lines

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_stop(struct csi2rx_priv *csi2rx) · line 412

Quoted 5 lines
drivers/media/platform/cadence/cdns-csi2rx.c +3 -0
+				used_vc = fd.entry[i].bus.csi2.vc;
+				break;
 			}
 		}
+		csi2rx->vc_select[cdns_stream] |=

Why |= ? do you expect to be able to capture multiple VC per stream ? I don't think that's allowed ?

The intention here is to allow multiple VC on the same source pad. Confusingly enough cadence calls the source pads as "stream0,1,2,3" and this driver uses that convention in the macros :/

This cdns-csi2rx bridge is an intermediary between the sensor and ti's wrapper IP (platform/ti/j721e-csi2rx.c) which is connected to the first source pad of cdns-csi2rx on TI SoCs like AM62A [1]

Ultimately the wrapper will separate incoming streams by VC/DT into different DMA channels, so no filtering is done here.

[1]: Section 12.6.4.1 CSI_RX_IF Block Diagram in AM62A TRM
https://www.ti.com/lit/pdf/spruj16

Quoted 9 lines

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_stop(struct csi2rx_priv *csi2rx) · line 413

Quoted 2 lines
drivers/media/platform/cadence/cdns-csi2rx.c +1 -0
+			CSI2RX_STREAM_DATA_CFG_VC_SELECT(used_vc);

In case no stream matching route->sink_stream is found in the remote's frame descriptor this sets CSI2RX_STREAM_DATA_CFG_VC_SELECT(0) for vc_select[x]...

Quoted 2 lines
drivers/media/platform/cadence/cdns-csi2rx.c +1 -0
+	}
 
Quoted 11 lines

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_stop(struct csi2rx_priv *csi2rx) · line 420

Quoted 8 lines
drivers/media/platform/cadence/cdns-csi2rx.c +5 -3
-		csi2rx->count++;
-	} else {
-		csi2rx->count--;
+err_no_fd:
+	for (i = 0; i < CSI2RX_STREAMS_MAX; i++) {
+		if (!csi2rx->vc_select[i]) {
+			csi2rx->vc_select[i] =
+				CSI2RX_STREAM_DATA_CFG_VC_SELECT(0);

... so why doing it again here (assuming the no_fd case should be dropped).

Ah good catch, will fix.

Quoted 19 lines

Also, setting multiple streams to capture on VC=0 will multiplex the same stream or cause troubles ?

Also, as you have an initial loop already this could maybe be:

        /* Initialize all streams to capture VC 0 by default. */
	for (i = 0; i < CSI2RX_STREAMS_MAX; i++)
		csi2rx->vc_select[i] =  CSI2RX_STREAM_DATA_CFG_VC_SELECT(0);

	ret = csi2rx_get_frame_desc_from_source(csi2rx, &fd);
	if (ret || fd.type != V4L2_MBUS_FRAME_DESC_TYPE_CSI2) {
		dev_dbg(csi2rx->dev,
			"Failed to get source frame desc, allowing only VC=0\n");
		return;
	}

	/* If source provides per-stream VC info, use it to filter by VC */
	for_each_active_route(&state->routing, route) {
Quoted 8 lines
		int cdns_stream = route->source_pad - CSI2RX_PAD_SOURCE_STREAM0;

		for (i = 0; i < fd.num_entries; i++) {
			if (fd.entry[i].stream != route->sink_stream)
                                continue;

                        csi2rx->vc_select[cdns_stream] =
                                CSI2RX_STREAM_DATA_CFG_VC_SELECT(fd.entry[i].bus.csi2.vc);

Thanks will do this in v3.

Quoted 156 lines
		}
	}
Quoted 33 lines
drivers/media/platform/cadence/cdns-csi2rx.c +26 -5
+		}
+	}
+}
 
-		/*
-		 * Let the last user turn off the lights.
-		 */
-		if (!csi2rx->count)
-			csi2rx_stop(csi2rx);
+static int csi2rx_enable_streams(struct v4l2_subdev *subdev,
+				 struct v4l2_subdev_state *state, u32 pad,
+				 u64 streams_mask)
+{
+	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
+	struct media_pad *remote_pad;
+	u64 sink_streams;
+	int ret;
+
+	remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
+	if (!remote_pad) {
+		dev_err(csi2rx->dev,
+			"Failed to find connected source\n");
+		return -ENODEV;
+	}
+
+	ret = pm_runtime_resume_and_get(csi2rx->dev);
+	if (ret < 0)
+		return ret;
+
+	sink_streams = v4l2_subdev_state_xlate_streams(state, pad,
+						       CSI2RX_PAD_SINK,
+						       &streams_mask);
 

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_stop(struct csi2rx_priv *csi2rx) · line 457

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +9 -1
-		pm_runtime_put(csi2rx->dev);
+	mutex_lock(&csi2rx->lock);
+	/*
+	 * If we're not the first users, there's no need to
+	 * enable the whole controller.
+	 */
+	if (!csi2rx->count) {
+		ret = csi2rx_start(csi2rx);
+		if (ret)
+			goto err_stream_start;

I'm not 100% familiar with this new API but I was expecting you wouldn't need to do so

Quoted 106 lines
drivers/media/platform/cadence/cdns-csi2rx.c +95 -1
 	}
 
-out:
+	/* Start streaming on the source */
+	ret = v4l2_subdev_enable_streams(csi2rx->source_subdev, remote_pad->index,
+					 sink_streams);
+	if (ret) {
+		dev_err(csi2rx->dev,
+			"Failed to start streams %#llx on subdev\n",
+			sink_streams);
+		goto err_subdev_enable;
+	}
+
+	csi2rx->count++;
+	mutex_unlock(&csi2rx->lock);
+
+	return 0;
+
+err_subdev_enable:
+	if (!csi2rx->count)
+		csi2rx_stop(csi2rx);
+err_stream_start:
 	mutex_unlock(&csi2rx->lock);
+	pm_runtime_put(csi2rx->dev);
+	return ret;
+}
+
+static int csi2rx_disable_streams(struct v4l2_subdev *subdev,
+				  struct v4l2_subdev_state *state, u32 pad,
+				  u64 streams_mask)
+{
+	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
+	struct media_pad *remote_pad;
+	u64 sink_streams;
+
+	sink_streams = v4l2_subdev_state_xlate_streams(state, pad,
+						       CSI2RX_PAD_SINK,
+						       &streams_mask);
+
+	remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
+	if (!remote_pad ||
+	    v4l2_subdev_disable_streams(csi2rx->source_subdev,
+					remote_pad->index, sink_streams)) {
+		dev_err(csi2rx->dev, "Couldn't disable our subdev\n");
+	}
+
+	mutex_lock(&csi2rx->lock);
+	csi2rx->count--;
+	/*
+	 * Let the last user turn off the lights.
+	 */
+	if (!csi2rx->count)
+		csi2rx_stop(csi2rx);
+	mutex_unlock(&csi2rx->lock);
+
+	pm_runtime_put(csi2rx->dev);
+
+	return 0;
+}
+
+static int csi2rx_s_stream_fallback(struct v4l2_subdev *sd, int enable)
+{
+	struct v4l2_subdev_state *state;
+	struct v4l2_subdev_route *route;
+	u64 mask[CSI2RX_PAD_MAX] = {0};
+	int i, ret;
+
+	/* Find the stream mask on all source pads */
+	state = v4l2_subdev_lock_and_get_active_state(sd);
+	for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++) {
+		for_each_active_route(&state->routing, route) {
+			if (route->source_pad == i)
+				mask[i] |= BIT_ULL(route->source_stream);
+		}
+	}
+	v4l2_subdev_unlock_state(state);
+
+	/* Start streaming on each pad */
+	for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++) {
+		if (enable)
+			ret = v4l2_subdev_enable_streams(sd, i, mask[i]);
+		else
+			ret = v4l2_subdev_disable_streams(sd, i, mask[i]);
+		if (ret)
+			return ret;
+	}
+
 	return ret;
 }
 
@@ -446,12 +556,63 @@ static int csi2rx_enum_mbus_code(struct v4l2_subdev *subdev,
 	return 0;
 }
 
+static int _csi2rx_set_routing(struct v4l2_subdev *subdev,
+			       struct v4l2_subdev_state *state,
+			       struct v4l2_subdev_krouting *routing)
+{
+	static const struct v4l2_mbus_framefmt format = {
+		.width = 640,
+		.height = 480,
+		.code = MEDIA_BUS_FMT_UYVY8_1X16,
+		.field = V4L2_FIELD_NONE,
+		.colorspace = V4L2_COLORSPACE_SRGB,
+		.ycbcr_enc = V4L2_YCBCR_ENC_601,
+		.quantization = V4L2_QUANTIZATION_LIM_RANGE,
Quoted 9 lines

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_enum_mbus_code(struct v4l2_subdev *subdev, · line 576

Quoted 6 lines
drivers/media/platform/cadence/cdns-csi2rx.c +6 -0
+		.xfer_func = V4L2_XFER_FUNC_SRGB,
+	};
+	int ret;
+
+	if (routing->num_routes > V4L2_FRAME_DESC_ENTRY_MAX)
+		return -EINVAL;

Isn't there a max number of routes supported by the HW ?

I guess yes, 16 VC x 4 output pads should be the total routes possible, but that is much greater than the current FRAME_DESC_ENTRY_MAX (= 8)

This check was re-used from ds90ub9xx.c drivers.

Quoted 9 lines

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_enum_mbus_code(struct v4l2_subdev *subdev, · line 583

Quoted 7 lines
drivers/media/platform/cadence/cdns-csi2rx.c +7 -0
+
+	ret = v4l2_subdev_routing_validate(subdev, routing,
+					   V4L2_SUBDEV_ROUTING_ONLY_1_TO_1);
+	if (ret)
+		return ret;
+
+	ret = v4l2_subdev_set_routing_with_fmt(subdev, state, routing, &format);
        return ...

Will fix.

Quoted 14 lines
Quoted 14 lines
drivers/media/platform/cadence/cdns-csi2rx.c +13 -0
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int csi2rx_set_routing(struct v4l2_subdev *subdev,
+			      struct v4l2_subdev_state *state,
+			      enum v4l2_subdev_format_whence which,
+			      struct v4l2_subdev_krouting *routing)
+{
+	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
+	int ret;
Quoted 8 lines

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_enum_mbus_code(struct v4l2_subdev *subdev, · line 602

Quoted 6 lines
drivers/media/platform/cadence/cdns-csi2rx.c +6 -0
+
+	if (which == V4L2_SUBDEV_FORMAT_ACTIVE && csi2rx->count)
+		return -EBUSY;
+
+	ret = _csi2rx_set_routing(subdev, state, routing);
+

no empty line

Will fix.

Quoted 29 lines
Quoted 29 lines
drivers/media/platform/cadence/cdns-csi2rx.c +12 -2
+	if (ret)
+		return ret;
+
+	csi2rx_update_vc_select(csi2rx, state);
+
+	return 0;
+}
+
 static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
 			  struct v4l2_subdev_state *state,
 			  struct v4l2_subdev_format *format)
 {
 	struct v4l2_mbus_framefmt *fmt;
-	unsigned int i;
 
 	/* No transcoding, source and sink formats must match. */
 	if (format->pad != CSI2RX_PAD_SINK)
@@ -463,14 +624,19 @@ static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
 	format->format.field = V4L2_FIELD_NONE;
 
 	/* Set sink format */
-	fmt = v4l2_subdev_state_get_format(state, format->pad);
+	fmt = v4l2_subdev_state_get_format(state, format->pad, format->stream);
+	if (!fmt)
+		return -EINVAL;
+
 	*fmt = format->format;
 
Quoted 12 lines

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_set_fmt(struct v4l2_subdev *subdev, · line 637

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +5 -5
-	/* Propagate to source formats */
-	for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++) {
-		fmt = v4l2_subdev_state_get_format(state, i);
-		*fmt = format->format;
-	}
+	/* Propagate to source format */
+	fmt = v4l2_subdev_state_get_opposite_stream_format(state, format->pad,
+							   format->stream);
+	if (!fmt)
+		return -EINVAL;

again I'm not sure this can happen

Will fix.

Quoted 16 lines
Quoted 16 lines
drivers/media/platform/cadence/cdns-csi2rx.c +2 -6
+
+	*fmt = format->format;
 
 	return 0;
 }
@@ -478,40 +644,92 @@ static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
 static int csi2rx_init_state(struct v4l2_subdev *subdev,
 			     struct v4l2_subdev_state *state)
 {
-	struct v4l2_subdev_format format = {
-		.pad = CSI2RX_PAD_SINK,
-		.format = {
-			.width = 640,
-			.height = 480,
-			.code = MEDIA_BUS_FMT_UYVY8_1X16,
Quoted 8 lines

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_set_fmt(struct v4l2_subdev *subdev, · line 647

Quoted 6 lines
drivers/media/platform/cadence/cdns-csi2rx.c +1 -5
-			.field = V4L2_FIELD_NONE,
-			.colorspace = V4L2_COLORSPACE_SRGB,
-			.ycbcr_enc = V4L2_YCBCR_ENC_601,
-			.quantization = V4L2_QUANTIZATION_LIM_RANGE,
-			.xfer_func = V4L2_XFER_FUNC_SRGB,
+	struct v4l2_subdev_route routes[] = {

static const ?

Will fix.

Quoted 10 lines
Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +6 -0
+		{
+			.sink_pad = CSI2RX_PAD_SINK,
+			.sink_stream = 0,
+			.source_pad = CSI2RX_PAD_SOURCE_STREAM0,
+			.source_stream = 0,
+			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
 		},
 	};
 
Quoted 4 lines

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_set_fmt(struct v4l2_subdev *subdev, · line 657

Quoted 2 lines
drivers/media/platform/cadence/cdns-csi2rx.c +1 -1
-	return csi2rx_set_fmt(subdev, state, &format);
+	struct v4l2_subdev_krouting routing = {

static const ?

Will fix.

Quoted 8 lines
Quoted 8 lines
drivers/media/platform/cadence/cdns-csi2rx.c +5 -0
+		.num_routes = ARRAY_SIZE(routes),
+		.routes = routes,
+	};
+
+	return _csi2rx_set_routing(subdev, state, &routing);
 }
 
Quoted 13 lines

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_set_fmt(struct v4l2_subdev *subdev, · line 674

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +6 -0
 static int csi2rx_get_frame_desc(struct v4l2_subdev *subdev, unsigned int pad,
 				 struct v4l2_mbus_frame_desc *fd)
 {
 	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
+	struct v4l2_mbus_frame_desc source_fd = {0};
+	struct v4l2_subdev_route *route;
+	struct v4l2_subdev_state *state;
+	int ret;
+
+	ret = csi2rx_get_frame_desc_from_source(csi2rx, &source_fd);

Should this inspect the remote frame desc or only the local routing table ?

We want to use the correct VC, if the source is sending two streams with VC=1 and VC=2, we want to make sure the output streams are also setting that data in the frame descriptors for this subdev.

Quoted 43 lines
Quoted 30 lines
drivers/media/platform/cadence/cdns-csi2rx.c +27 -1
+	if (ret)
+		return ret;
+
+	fd->type = V4L2_MBUS_FRAME_DESC_TYPE_CSI2;
 
-	return csi2rx_get_frame_desc_from_source(csi2rx, fd);
+	state = v4l2_subdev_lock_and_get_active_state(subdev);
+
+	for_each_active_route(&state->routing, route) {
+		struct v4l2_mbus_frame_desc_entry *source_entry = NULL;
+		unsigned int i;
+
+		if (route->source_pad != pad)
+			continue;
+
+		for (i = 0; i < source_fd.num_entries; i++) {
+			if (source_fd.entry[i].stream == route->sink_stream) {
+				source_entry = &source_fd.entry[i];
+				break;
+			}
+		}
+
+		if (!source_entry) {
+			dev_err(csi2rx->dev,
+				"Failed to find stream from source frame desc\n");
+			ret = -EPIPE;
+			goto err_missing_stream;
+		}
+

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_set_fmt(struct v4l2_subdev *subdev, · line 712

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +10 -0
+		fd->entry[fd->num_entries].stream = route->source_stream;
+		fd->entry[fd->num_entries].flags = source_entry->flags;
+		fd->entry[fd->num_entries].length = source_entry->length;
+		fd->entry[fd->num_entries].pixelcode = source_entry->pixelcode;
+		fd->entry[fd->num_entries].bus.csi2.vc =
+			source_entry->bus.csi2.vc;
+		fd->entry[fd->num_entries].bus.csi2.dt =
+			source_entry->bus.csi2.dt;
+
+		fd->num_entries++;

as you

Quoted 4 lines
		if (route->source_pad != pad)
			continue;

can you have multiple entris on a pad ?

Yes, this IP will pass through all incoming streams on sink to the first source.

Quoted 39 lines
Quoted 30 lines
drivers/media/platform/cadence/cdns-csi2rx.c +14 -5
+	}
+
+err_missing_stream:
+	v4l2_subdev_unlock_state(state);
+
+	return ret;
 }
 
 static const struct v4l2_subdev_pad_ops csi2rx_pad_ops = {
-	.enum_mbus_code	= csi2rx_enum_mbus_code,
-	.get_fmt	= v4l2_subdev_get_fmt,
-	.set_fmt	= csi2rx_set_fmt,
-	.get_frame_desc = csi2rx_get_frame_desc,
+	.enum_mbus_code		= csi2rx_enum_mbus_code,
+	.get_fmt		= v4l2_subdev_get_fmt,
+	.set_fmt		= csi2rx_set_fmt,
+	.get_frame_desc		= csi2rx_get_frame_desc,
+	.set_routing		= csi2rx_set_routing,
+	.enable_streams		= csi2rx_enable_streams,
+	.disable_streams	= csi2rx_disable_streams,
 };
 
 static const struct v4l2_subdev_video_ops csi2rx_video_ops = {
-	.s_stream	= csi2rx_s_stream,
+	.s_stream	= csi2rx_s_stream_fallback,
 };
 
 static const struct v4l2_subdev_ops csi2rx_subdev_ops = {

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_probe(struct platform_device *pdev) · line 988

Quoted 6 lines
drivers/media/platform/cadence/cdns-csi2rx.c +2 -1
@@ -766,7 +984,8 @@ static int csi2rx_probe(struct platform_device *pdev)
 	csi2rx->pads[CSI2RX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
 	for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++)
 		csi2rx->pads[i].flags = MEDIA_PAD_FL_SOURCE;
-	csi2rx->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+	csi2rx->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE |
+		V4L2_SUBDEV_FL_STREAMS;

nit: please align to the previous one

Quoted 4 lines
	csi2rx->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE |
		                V4L2_SUBDEV_FL_STREAMS;

and maybe the |= should be just an =

Will fix.

Quoted 12 lines

Thanks
j

Quoted 8 lines
drivers/media/platform/cadence/cdns-csi2rx.c +0 -1
 	csi2rx->subdev.entity.ops = &csi2rx_media_ops;
 
 	ret = media_entity_pads_init(&csi2rx->subdev.entity, CSI2RX_PAD_MAX,
 
--

2.43.0

Signature · 4 lines
Thanks,
Jai

GPG Fingerprint: 4DE0 D818 E5D5 75E8 D45A AFC5 43DE 91F9 249A 7145
#37 lore raw

Re: [PATCH v2 11/13] media: cadence: csi2rx: Enable multi-stream support

Hi Jai

Quoted 102 lines · Jai Luthra

Hi Jacopo,

Thanks for the review.

Quoted 88 lines · Jacopo Mondi

Hi Jai

Quoted 55 lines · Jai Luthra

Cadence CSI-2 bridge IP supports capturing multiple virtual "streams" of data over the same physical interface using MIPI Virtual Channels.

The V4L2 subdev APIs should reflect this capability and allow per-stream routing and controls.

While the hardware IP supports usecases where streams coming in the sink pad can be broadcasted to multiple source pads, the driver will need significant re-architecture to make that possible. The two users of this IP in mainline linux are TI Shim and StarFive JH7110 CAMSS, and both have only integrated the first source pad i.e stream0 of this IP. So for now keep it simple and only allow 1-to-1 mapping of streams from sink to source, without any broadcasting.

With stream routing now supported in the driver, implement the enable_stream and disable_stream hooks in place of the stream-unaware s_stream hook.

This allows consumer devices like a DMA bridge or ISP, to enable particular streams on a source pad, which in turn can be used to enable only particular streams on the CSI-TX device connected on the sink pad.

Implement a fallback s_stream hook that internally calls enable_stream on each source pad, for consumer drivers that don't use multi-stream APIs to still work.

---
 drivers/media/platform/cadence/cdns-csi2rx.c | 407 ++++++++++++++++++++-------
 1 file changed, 313 insertions(+), 94 deletions(-)
drivers/media/platform/cadence/cdns-csi2rx.c +3 -4
index 2ec34fc9c524..b0c91a9c65e8 100644
@@ -90,6 +90,7 @@ struct csi2rx_priv {
 	struct reset_control		*pixel_rst[CSI2RX_STREAMS_MAX];
 	struct phy			*dphy;
 
+	u32				vc_select[CSI2RX_STREAMS_MAX];
 	u8				lanes[CSI2RX_LANES_MAX];
 	u8				num_lanes;
 	u8				max_lanes;
@@ -179,27 +180,43 @@ static void csi2rx_reset(struct csi2rx_priv *csi2rx)
 
 static int csi2rx_configure_ext_dphy(struct csi2rx_priv *csi2rx)
 {
+	struct v4l2_ctrl_handler *handler = csi2rx->source_subdev->ctrl_handler;
 	union phy_configure_opts opts = { };
 	struct phy_configure_opts_mipi_dphy *cfg = &opts.mipi_dphy;
-	struct v4l2_subdev_format sd_fmt = {
-		.which	= V4L2_SUBDEV_FORMAT_ACTIVE,
-		.pad	= CSI2RX_PAD_SINK,
-	};
+	struct v4l2_mbus_framefmt *framefmt;

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_reset(struct csi2rx_priv *csi2rx) · line 192

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +2 -4
+	struct v4l2_subdev_state *state;
 	const struct csi2rx_fmt *fmt;
 	s64 link_freq;
 	int ret;
 
-	ret = v4l2_subdev_call_state_active(&csi2rx->subdev, pad, get_fmt,
-					    &sd_fmt);
-	if (ret < 0)
-		return ret;
+	if (v4l2_ctrl_find(handler, V4L2_CID_LINK_FREQ)) {

Do you need to do this by yourself ? afaict v4l2_get_link_freq() already checks if V4L2_CID_LINK_FREQ is available, and if not, fallsback to use PIXEL_RATE.

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_reset(struct csi2rx_priv *csi2rx) · line 204

Quoted 12 lines
drivers/media/platform/cadence/cdns-csi2rx.c +12 -0
+		link_freq = v4l2_get_link_freq(handler, 0, 0);
+	} else {
+		state = v4l2_subdev_get_locked_active_state(&csi2rx->subdev);
+		framefmt = v4l2_subdev_state_get_format(state, CSI2RX_PAD_SINK,
+							0);
+
+		if (framefmt) {
+			fmt = csi2rx_get_fmt_by_code(framefmt->code);
+		} else {
+			dev_err(csi2rx->dev,
+				"Did not find active sink format\n");
+			return -EINVAL;

Is this possibile ?

Quoted 8 lines

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_reset(struct csi2rx_priv *csi2rx) · line 208

Quoted 5 lines
drivers/media/platform/cadence/cdns-csi2rx.c +3 -1
+		}
 
-	fmt = csi2rx_get_fmt_by_code(sd_fmt.format.code);
+		link_freq = v4l2_get_link_freq(handler, fmt->bpp,
+					       2 * csi2rx->num_lanes);

Do we want to allow falling back on PIXEL_RATE for multiplexed transmitters ? I presume this will give you invalid results anyway.

Quoted 3 lines

This is mostly done to avoid breaking any single stream sensor that does not have the LINK_FREQ control, and was working with this bridge before. Thus the warning below for multi-format sources.

Is it possible to allow usage of PIXEL_LINK only for non-multiplexed transmitters ?

Quoted 7 lines
Quoted 3 lines

I would simply call v4l2_get_link_freq(handler, 0, 0) to force the usage of LINK_FREQ which will become mandatory for transmitters

Ah I did not know LINK_FREQ will be mandatory soon. Any threads I can look at where this was discussed?

I meant mandatory for multiplexed transmitters, which will have to be 'forced' to use LINK_FREQ as PIXEL_RATE doesn't make much sense for them

Quoted 398 lines
Quoted 108 lines

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_reset(struct csi2rx_priv *csi2rx) · line 213

Quoted 9 lines
drivers/media/platform/cadence/cdns-csi2rx.c +4 -3
-	link_freq = v4l2_get_link_freq(csi2rx->source_subdev->ctrl_handler,
-				       fmt->bpp, 2 * csi2rx->num_lanes);
-	if (link_freq < 0)
+		dev_warn(csi2rx->dev,
+			 "Guessing link frequency using bitdepth of stream 0.\n");
+		dev_warn(csi2rx->dev,
+			 "V4L2_CID_LINK_FREQ control is required for multi format sources.\n");

Exactly :)

Quoted 96 lines
drivers/media/platform/cadence/cdns-csi2rx.c +12 -31
+	}
+
+	if (link_freq < 0) {
+		dev_err(csi2rx->dev, "Unable to calculate link frequency\n");
 		return link_freq;
+	}
 
 	ret = phy_mipi_dphy_get_default_config_for_hsclk(link_freq,
 							 csi2rx->num_lanes, cfg);
@@ -222,18 +239,10 @@ static int csi2rx_configure_ext_dphy(struct csi2rx_priv *csi2rx)
 static int csi2rx_start(struct csi2rx_priv *csi2rx)
 {
 	unsigned int i;
-	struct media_pad *remote_pad;
 	unsigned long lanes_used = 0;
 	u32 reg;
 	int ret;
 
-	remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
-	if (!remote_pad) {
-		dev_err(csi2rx->dev,
-			"Failed to find connected source\n");
-		return -ENODEV;
-	}
-
 	ret = clk_prepare_enable(csi2rx->p_clk);
 	if (ret)
 		return ret;
@@ -300,11 +309,7 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
 		writel(CSI2RX_STREAM_CFG_FIFO_MODE_LARGE_BUF,
 		       csi2rx->base + CSI2RX_STREAM_CFG_REG(i));
 
-		/*
-		 * Enable one virtual channel. When multiple virtual channels
-		 * are supported this will have to be changed.
-		 */
-		writel(CSI2RX_STREAM_DATA_CFG_VC_SELECT(0),
+		writel(csi2rx->vc_select[i],
 		       csi2rx->base + CSI2RX_STREAM_DATA_CFG_REG(i));
 
 		writel(CSI2RX_STREAM_CTRL_START,
@@ -317,17 +322,10 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
 
 	reset_control_deassert(csi2rx->sys_rst);
 
-	ret = v4l2_subdev_enable_streams(csi2rx->source_subdev,
-					 remote_pad->index, BIT(0));
-	if (ret)
-		goto err_disable_sysclk;
-
 	clk_disable_unprepare(csi2rx->p_clk);
 
 	return 0;
 
-err_disable_sysclk:
-	clk_disable_unprepare(csi2rx->sys_clk);
 err_disable_pixclk:
 	for (; i > 0; i--) {
 		reset_control_assert(csi2rx->pixel_rst[i - 1]);
@@ -346,7 +344,6 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
 
 static void csi2rx_stop(struct csi2rx_priv *csi2rx)
 {
-	struct media_pad *remote_pad;
 	unsigned int i;
 	u32 val;
 	int ret;
@@ -375,13 +372,6 @@ static void csi2rx_stop(struct csi2rx_priv *csi2rx)
 	reset_control_assert(csi2rx->p_rst);
 	clk_disable_unprepare(csi2rx->p_clk);
 
-	remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
-	if (!remote_pad ||
-	    v4l2_subdev_disable_streams(csi2rx->source_subdev,
-					remote_pad->index, BIT(0))) {
-		dev_warn(csi2rx->dev, "Couldn't disable our subdev\n");
-	}
-
 	if (csi2rx->dphy) {
 		writel(0, csi2rx->base + CSI2RX_DPHY_LANE_CTRL_REG);
 
@@ -390,47 +380,167 @@ static void csi2rx_stop(struct csi2rx_priv *csi2rx)
 	}
 }
 
-static int csi2rx_s_stream(struct v4l2_subdev *subdev, int enable)
+static void csi2rx_update_vc_select(struct csi2rx_priv *csi2rx,
+				    struct v4l2_subdev_state *state)
 {
-	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
-	int ret = 0;
+	struct v4l2_mbus_frame_desc fd = {0};
+	struct v4l2_subdev_route *route;
+	unsigned int i;
+	int ret;
 
Quoted 8 lines

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_stop(struct csi2rx_priv *csi2rx) · line 392

Quoted 6 lines
drivers/media/platform/cadence/cdns-csi2rx.c +2 -4
-	if (enable) {
-		ret = pm_runtime_resume_and_get(csi2rx->dev);
-		if (ret < 0)
-			return ret;
+	for (i = 0; i < CSI2RX_STREAMS_MAX; i++)
+		csi2rx->vc_select[i] = 0;

or just memset ?

Will do.

Quoted 13 lines

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_stop(struct csi2rx_priv *csi2rx) · line 397

Quoted 6 lines
drivers/media/platform/cadence/cdns-csi2rx.c +5 -0
+
+	ret = csi2rx_get_frame_desc_from_source(csi2rx, &fd);
+	if (ret || fd.type != V4L2_MBUS_FRAME_DESC_TYPE_CSI2) {
+		dev_dbg(csi2rx->dev,
+			"Failed to get source frame desc, allowing only VC=0\n");

I'm still not sure if picking a default fallback is a good idea or not, as this will only work by chance

Quoted 3 lines
drivers/media/platform/cadence/cdns-csi2rx.c +1 -0
+		goto err_no_fd;
 	}
 
Quoted 6 lines

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_stop(struct csi2rx_priv *csi2rx) · line 403

Quoted 4 lines
drivers/media/platform/cadence/cdns-csi2rx.c +3 -1
-	mutex_lock(&csi2rx->lock);
+	/* If source provides per-stream VC info, use it to filter by VC */
+	for_each_active_route(&state->routing, route) {
+		int cdns_stream = route->source_pad - CSI2RX_PAD_SOURCE_STREAM0;

unsigned maybe ?

Will fix.

Quoted 15 lines
Quoted 15 lines
drivers/media/platform/cadence/cdns-csi2rx.c +3 -10
+		u8 used_vc = 0;
 
-	if (enable) {
-		/*
-		 * If we're not the first users, there's no need to
-		 * enable the whole controller.
-		 */
-		if (!csi2rx->count) {
-			ret = csi2rx_start(csi2rx);
-			if (ret) {
-				pm_runtime_put(csi2rx->dev);
-				goto out;
+		for (i = 0; i < fd.num_entries; i++) {
+			if (fd.entry[i].stream == route->sink_stream) {
Quoted 8 lines

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_stop(struct csi2rx_priv *csi2rx) · line 412

Quoted 5 lines
drivers/media/platform/cadence/cdns-csi2rx.c +3 -0
+				used_vc = fd.entry[i].bus.csi2.vc;
+				break;
 			}
 		}
+		csi2rx->vc_select[cdns_stream] |=

Why |= ? do you expect to be able to capture multiple VC per stream ? I don't think that's allowed ?

The intention here is to allow multiple VC on the same source pad. Confusingly enough cadence calls the source pads as "stream0,1,2,3" and this driver uses that convention in the macros :/

This cdns-csi2rx bridge is an intermediary between the sensor and ti's wrapper IP (platform/ti/j721e-csi2rx.c) which is connected to the first source pad of cdns-csi2rx on TI SoCs like AM62A [1]

Ultimately the wrapper will separate incoming streams by VC/DT into different DMA channels, so no filtering is done here.

[1]: Section 12.6.4.1 CSI_RX_IF Block Diagram in AM62A TRM
https://www.ti.com/lit/pdf/spruj16

Quoted 9 lines

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_stop(struct csi2rx_priv *csi2rx) · line 413

Quoted 2 lines
drivers/media/platform/cadence/cdns-csi2rx.c +1 -0
+			CSI2RX_STREAM_DATA_CFG_VC_SELECT(used_vc);

In case no stream matching route->sink_stream is found in the remote's frame descriptor this sets CSI2RX_STREAM_DATA_CFG_VC_SELECT(0) for vc_select[x]...

Quoted 2 lines
drivers/media/platform/cadence/cdns-csi2rx.c +1 -0
+	}
 
Quoted 11 lines

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_stop(struct csi2rx_priv *csi2rx) · line 420

Quoted 8 lines
drivers/media/platform/cadence/cdns-csi2rx.c +5 -3
-		csi2rx->count++;
-	} else {
-		csi2rx->count--;
+err_no_fd:
+	for (i = 0; i < CSI2RX_STREAMS_MAX; i++) {
+		if (!csi2rx->vc_select[i]) {
+			csi2rx->vc_select[i] =
+				CSI2RX_STREAM_DATA_CFG_VC_SELECT(0);

... so why doing it again here (assuming the no_fd case should be dropped).

Ah good catch, will fix.

Quoted 19 lines

Also, setting multiple streams to capture on VC=0 will multiplex the same stream or cause troubles ?

Also, as you have an initial loop already this could maybe be:

        /* Initialize all streams to capture VC 0 by default. */
	for (i = 0; i < CSI2RX_STREAMS_MAX; i++)
		csi2rx->vc_select[i] =  CSI2RX_STREAM_DATA_CFG_VC_SELECT(0);

	ret = csi2rx_get_frame_desc_from_source(csi2rx, &fd);
	if (ret || fd.type != V4L2_MBUS_FRAME_DESC_TYPE_CSI2) {
		dev_dbg(csi2rx->dev,
			"Failed to get source frame desc, allowing only VC=0\n");
		return;
	}

	/* If source provides per-stream VC info, use it to filter by VC */
	for_each_active_route(&state->routing, route) {
Quoted 8 lines
		int cdns_stream = route->source_pad - CSI2RX_PAD_SOURCE_STREAM0;

		for (i = 0; i < fd.num_entries; i++) {
			if (fd.entry[i].stream != route->sink_stream)
                                continue;

                        csi2rx->vc_select[cdns_stream] =
                                CSI2RX_STREAM_DATA_CFG_VC_SELECT(fd.entry[i].bus.csi2.vc);

Thanks will do this in v3.

Quoted 156 lines
		}
	}
Quoted 33 lines
drivers/media/platform/cadence/cdns-csi2rx.c +26 -5
+		}
+	}
+}
 
-		/*
-		 * Let the last user turn off the lights.
-		 */
-		if (!csi2rx->count)
-			csi2rx_stop(csi2rx);
+static int csi2rx_enable_streams(struct v4l2_subdev *subdev,
+				 struct v4l2_subdev_state *state, u32 pad,
+				 u64 streams_mask)
+{
+	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
+	struct media_pad *remote_pad;
+	u64 sink_streams;
+	int ret;
+
+	remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
+	if (!remote_pad) {
+		dev_err(csi2rx->dev,
+			"Failed to find connected source\n");
+		return -ENODEV;
+	}
+
+	ret = pm_runtime_resume_and_get(csi2rx->dev);
+	if (ret < 0)
+		return ret;
+
+	sink_streams = v4l2_subdev_state_xlate_streams(state, pad,
+						       CSI2RX_PAD_SINK,
+						       &streams_mask);
 

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_stop(struct csi2rx_priv *csi2rx) · line 457

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +9 -1
-		pm_runtime_put(csi2rx->dev);
+	mutex_lock(&csi2rx->lock);
+	/*
+	 * If we're not the first users, there's no need to
+	 * enable the whole controller.
+	 */
+	if (!csi2rx->count) {
+		ret = csi2rx_start(csi2rx);
+		if (ret)
+			goto err_stream_start;

I'm not 100% familiar with this new API but I was expecting you wouldn't need to do so

Quoted 106 lines
drivers/media/platform/cadence/cdns-csi2rx.c +95 -1
 	}
 
-out:
+	/* Start streaming on the source */
+	ret = v4l2_subdev_enable_streams(csi2rx->source_subdev, remote_pad->index,
+					 sink_streams);
+	if (ret) {
+		dev_err(csi2rx->dev,
+			"Failed to start streams %#llx on subdev\n",
+			sink_streams);
+		goto err_subdev_enable;
+	}
+
+	csi2rx->count++;
+	mutex_unlock(&csi2rx->lock);
+
+	return 0;
+
+err_subdev_enable:
+	if (!csi2rx->count)
+		csi2rx_stop(csi2rx);
+err_stream_start:
 	mutex_unlock(&csi2rx->lock);
+	pm_runtime_put(csi2rx->dev);
+	return ret;
+}
+
+static int csi2rx_disable_streams(struct v4l2_subdev *subdev,
+				  struct v4l2_subdev_state *state, u32 pad,
+				  u64 streams_mask)
+{
+	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
+	struct media_pad *remote_pad;
+	u64 sink_streams;
+
+	sink_streams = v4l2_subdev_state_xlate_streams(state, pad,
+						       CSI2RX_PAD_SINK,
+						       &streams_mask);
+
+	remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
+	if (!remote_pad ||
+	    v4l2_subdev_disable_streams(csi2rx->source_subdev,
+					remote_pad->index, sink_streams)) {
+		dev_err(csi2rx->dev, "Couldn't disable our subdev\n");
+	}
+
+	mutex_lock(&csi2rx->lock);
+	csi2rx->count--;
+	/*
+	 * Let the last user turn off the lights.
+	 */
+	if (!csi2rx->count)
+		csi2rx_stop(csi2rx);
+	mutex_unlock(&csi2rx->lock);
+
+	pm_runtime_put(csi2rx->dev);
+
+	return 0;
+}
+
+static int csi2rx_s_stream_fallback(struct v4l2_subdev *sd, int enable)
+{
+	struct v4l2_subdev_state *state;
+	struct v4l2_subdev_route *route;
+	u64 mask[CSI2RX_PAD_MAX] = {0};
+	int i, ret;
+
+	/* Find the stream mask on all source pads */
+	state = v4l2_subdev_lock_and_get_active_state(sd);
+	for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++) {
+		for_each_active_route(&state->routing, route) {
+			if (route->source_pad == i)
+				mask[i] |= BIT_ULL(route->source_stream);
+		}
+	}
+	v4l2_subdev_unlock_state(state);
+
+	/* Start streaming on each pad */
+	for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++) {
+		if (enable)
+			ret = v4l2_subdev_enable_streams(sd, i, mask[i]);
+		else
+			ret = v4l2_subdev_disable_streams(sd, i, mask[i]);
+		if (ret)
+			return ret;
+	}
+
 	return ret;
 }
 
@@ -446,12 +556,63 @@ static int csi2rx_enum_mbus_code(struct v4l2_subdev *subdev,
 	return 0;
 }
 
+static int _csi2rx_set_routing(struct v4l2_subdev *subdev,
+			       struct v4l2_subdev_state *state,
+			       struct v4l2_subdev_krouting *routing)
+{
+	static const struct v4l2_mbus_framefmt format = {
+		.width = 640,
+		.height = 480,
+		.code = MEDIA_BUS_FMT_UYVY8_1X16,
+		.field = V4L2_FIELD_NONE,
+		.colorspace = V4L2_COLORSPACE_SRGB,
+		.ycbcr_enc = V4L2_YCBCR_ENC_601,
+		.quantization = V4L2_QUANTIZATION_LIM_RANGE,
Quoted 9 lines

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_enum_mbus_code(struct v4l2_subdev *subdev, · line 576

Quoted 6 lines
drivers/media/platform/cadence/cdns-csi2rx.c +6 -0
+		.xfer_func = V4L2_XFER_FUNC_SRGB,
+	};
+	int ret;
+
+	if (routing->num_routes > V4L2_FRAME_DESC_ENTRY_MAX)
+		return -EINVAL;

Isn't there a max number of routes supported by the HW ?

Quoted 2 lines

I guess yes, 16 VC x 4 output pads should be the total routes possible, but that is much greater than the current FRAME_DESC_ENTRY_MAX (= 8)

I was not expecting FRAME_DESC_ENTRY_MAX to be that small :/

Quoted 150 lines

This check was re-used from ds90ub9xx.c drivers.

Quoted 9 lines

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_enum_mbus_code(struct v4l2_subdev *subdev, · line 583

Quoted 7 lines
drivers/media/platform/cadence/cdns-csi2rx.c +7 -0
+
+	ret = v4l2_subdev_routing_validate(subdev, routing,
+					   V4L2_SUBDEV_ROUTING_ONLY_1_TO_1);
+	if (ret)
+		return ret;
+
+	ret = v4l2_subdev_set_routing_with_fmt(subdev, state, routing, &format);
        return ...

Will fix.

Quoted 14 lines
Quoted 14 lines
drivers/media/platform/cadence/cdns-csi2rx.c +13 -0
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int csi2rx_set_routing(struct v4l2_subdev *subdev,
+			      struct v4l2_subdev_state *state,
+			      enum v4l2_subdev_format_whence which,
+			      struct v4l2_subdev_krouting *routing)
+{
+	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
+	int ret;
Quoted 8 lines

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_enum_mbus_code(struct v4l2_subdev *subdev, · line 602

Quoted 6 lines
drivers/media/platform/cadence/cdns-csi2rx.c +6 -0
+
+	if (which == V4L2_SUBDEV_FORMAT_ACTIVE && csi2rx->count)
+		return -EBUSY;
+
+	ret = _csi2rx_set_routing(subdev, state, routing);
+

no empty line

Will fix.

Quoted 29 lines
Quoted 29 lines
drivers/media/platform/cadence/cdns-csi2rx.c +12 -2
+	if (ret)
+		return ret;
+
+	csi2rx_update_vc_select(csi2rx, state);
+
+	return 0;
+}
+
 static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
 			  struct v4l2_subdev_state *state,
 			  struct v4l2_subdev_format *format)
 {
 	struct v4l2_mbus_framefmt *fmt;
-	unsigned int i;
 
 	/* No transcoding, source and sink formats must match. */
 	if (format->pad != CSI2RX_PAD_SINK)
@@ -463,14 +624,19 @@ static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
 	format->format.field = V4L2_FIELD_NONE;
 
 	/* Set sink format */
-	fmt = v4l2_subdev_state_get_format(state, format->pad);
+	fmt = v4l2_subdev_state_get_format(state, format->pad, format->stream);
+	if (!fmt)
+		return -EINVAL;
+
 	*fmt = format->format;
 
Quoted 12 lines

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_set_fmt(struct v4l2_subdev *subdev, · line 637

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +5 -5
-	/* Propagate to source formats */
-	for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++) {
-		fmt = v4l2_subdev_state_get_format(state, i);
-		*fmt = format->format;
-	}
+	/* Propagate to source format */
+	fmt = v4l2_subdev_state_get_opposite_stream_format(state, format->pad,
+							   format->stream);
+	if (!fmt)
+		return -EINVAL;

again I'm not sure this can happen

Will fix.

Quoted 16 lines
Quoted 16 lines
drivers/media/platform/cadence/cdns-csi2rx.c +2 -6
+
+	*fmt = format->format;
 
 	return 0;
 }
@@ -478,40 +644,92 @@ static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
 static int csi2rx_init_state(struct v4l2_subdev *subdev,
 			     struct v4l2_subdev_state *state)
 {
-	struct v4l2_subdev_format format = {
-		.pad = CSI2RX_PAD_SINK,
-		.format = {
-			.width = 640,
-			.height = 480,
-			.code = MEDIA_BUS_FMT_UYVY8_1X16,
Quoted 8 lines

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_set_fmt(struct v4l2_subdev *subdev, · line 647

Quoted 6 lines
drivers/media/platform/cadence/cdns-csi2rx.c +1 -5
-			.field = V4L2_FIELD_NONE,
-			.colorspace = V4L2_COLORSPACE_SRGB,
-			.ycbcr_enc = V4L2_YCBCR_ENC_601,
-			.quantization = V4L2_QUANTIZATION_LIM_RANGE,
-			.xfer_func = V4L2_XFER_FUNC_SRGB,
+	struct v4l2_subdev_route routes[] = {

static const ?

Will fix.

Quoted 10 lines
Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +6 -0
+		{
+			.sink_pad = CSI2RX_PAD_SINK,
+			.sink_stream = 0,
+			.source_pad = CSI2RX_PAD_SOURCE_STREAM0,
+			.source_stream = 0,
+			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
 		},
 	};
 
Quoted 4 lines

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_set_fmt(struct v4l2_subdev *subdev, · line 657

Quoted 2 lines
drivers/media/platform/cadence/cdns-csi2rx.c +1 -1
-	return csi2rx_set_fmt(subdev, state, &format);
+	struct v4l2_subdev_krouting routing = {

static const ?

Will fix.

Quoted 8 lines
Quoted 8 lines
drivers/media/platform/cadence/cdns-csi2rx.c +5 -0
+		.num_routes = ARRAY_SIZE(routes),
+		.routes = routes,
+	};
+
+	return _csi2rx_set_routing(subdev, state, &routing);
 }
 
Quoted 13 lines

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_set_fmt(struct v4l2_subdev *subdev, · line 674

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +6 -0
 static int csi2rx_get_frame_desc(struct v4l2_subdev *subdev, unsigned int pad,
 				 struct v4l2_mbus_frame_desc *fd)
 {
 	struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
+	struct v4l2_mbus_frame_desc source_fd = {0};
+	struct v4l2_subdev_route *route;
+	struct v4l2_subdev_state *state;
+	int ret;
+
+	ret = csi2rx_get_frame_desc_from_source(csi2rx, &source_fd);

Should this inspect the remote frame desc or only the local routing table ?

Quoted 4 lines

We want to use the correct VC, if the source is sending two streams with VC=1 and VC=2, we want to make sure the output streams are also setting that data in the frame descriptors for this subdev.

Isn't it something related to the routing table validation ?

Quoted 115 lines
Quoted 43 lines
Quoted 30 lines
drivers/media/platform/cadence/cdns-csi2rx.c +27 -1
+	if (ret)
+		return ret;
+
+	fd->type = V4L2_MBUS_FRAME_DESC_TYPE_CSI2;
 
-	return csi2rx_get_frame_desc_from_source(csi2rx, fd);
+	state = v4l2_subdev_lock_and_get_active_state(subdev);
+
+	for_each_active_route(&state->routing, route) {
+		struct v4l2_mbus_frame_desc_entry *source_entry = NULL;
+		unsigned int i;
+
+		if (route->source_pad != pad)
+			continue;
+
+		for (i = 0; i < source_fd.num_entries; i++) {
+			if (source_fd.entry[i].stream == route->sink_stream) {
+				source_entry = &source_fd.entry[i];
+				break;
+			}
+		}
+
+		if (!source_entry) {
+			dev_err(csi2rx->dev,
+				"Failed to find stream from source frame desc\n");
+			ret = -EPIPE;
+			goto err_missing_stream;
+		}
+

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_set_fmt(struct v4l2_subdev *subdev, · line 712

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +10 -0
+		fd->entry[fd->num_entries].stream = route->source_stream;
+		fd->entry[fd->num_entries].flags = source_entry->flags;
+		fd->entry[fd->num_entries].length = source_entry->length;
+		fd->entry[fd->num_entries].pixelcode = source_entry->pixelcode;
+		fd->entry[fd->num_entries].bus.csi2.vc =
+			source_entry->bus.csi2.vc;
+		fd->entry[fd->num_entries].bus.csi2.dt =
+			source_entry->bus.csi2.dt;
+
+		fd->num_entries++;

as you

Quoted 4 lines
		if (route->source_pad != pad)
			continue;

can you have multiple entris on a pad ?

Yes, this IP will pass through all incoming streams on sink to the first source.

Quoted 39 lines
Quoted 30 lines
drivers/media/platform/cadence/cdns-csi2rx.c +14 -5
+	}
+
+err_missing_stream:
+	v4l2_subdev_unlock_state(state);
+
+	return ret;
 }
 
 static const struct v4l2_subdev_pad_ops csi2rx_pad_ops = {
-	.enum_mbus_code	= csi2rx_enum_mbus_code,
-	.get_fmt	= v4l2_subdev_get_fmt,
-	.set_fmt	= csi2rx_set_fmt,
-	.get_frame_desc = csi2rx_get_frame_desc,
+	.enum_mbus_code		= csi2rx_enum_mbus_code,
+	.get_fmt		= v4l2_subdev_get_fmt,
+	.set_fmt		= csi2rx_set_fmt,
+	.get_frame_desc		= csi2rx_get_frame_desc,
+	.set_routing		= csi2rx_set_routing,
+	.enable_streams		= csi2rx_enable_streams,
+	.disable_streams	= csi2rx_disable_streams,
 };
 
 static const struct v4l2_subdev_video_ops csi2rx_video_ops = {
-	.s_stream	= csi2rx_s_stream,
+	.s_stream	= csi2rx_s_stream_fallback,
 };
 
 static const struct v4l2_subdev_ops csi2rx_subdev_ops = {

drivers/media/platform/cadence/cdns-csi2rx.c static int csi2rx_probe(struct platform_device *pdev) · line 988

Quoted 6 lines
drivers/media/platform/cadence/cdns-csi2rx.c +2 -1
@@ -766,7 +984,8 @@ static int csi2rx_probe(struct platform_device *pdev)
 	csi2rx->pads[CSI2RX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
 	for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++)
 		csi2rx->pads[i].flags = MEDIA_PAD_FL_SOURCE;
-	csi2rx->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+	csi2rx->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE |
+		V4L2_SUBDEV_FL_STREAMS;

nit: please align to the previous one

Quoted 4 lines
	csi2rx->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE |
		                V4L2_SUBDEV_FL_STREAMS;

and maybe the |= should be just an =

Will fix.

Quoted 12 lines

Thanks
j

Quoted 8 lines
drivers/media/platform/cadence/cdns-csi2rx.c +0 -1
 	csi2rx->subdev.entity.ops = &csi2rx_media_ops;
 
 	ret = media_entity_pads_init(&csi2rx->subdev.entity, CSI2RX_PAD_MAX,
 
--

2.43.0

--
Thanks,
Jai

GPG Fingerprint: 4DE0 D818 E5D5 75E8 D45A AFC5 43DE 91F9 249A 7145

#38 lore raw

Re: [PATCH v2 11/13] media: cadence: csi2rx: Enable multi-stream support

Quoted 116 lines · Jacopo Mondi

Hi Jai

Quoted 102 lines · Jai Luthra

Hi Jacopo,

Thanks for the review.

Quoted 88 lines · Jacopo Mondi

Hi Jai

Quoted 55 lines · Jai Luthra

Cadence CSI-2 bridge IP supports capturing multiple virtual "streams" of data over the same physical interface using MIPI Virtual Channels.

The V4L2 subdev APIs should reflect this capability and allow per-stream routing and controls.

While the hardware IP supports usecases where streams coming in the sink pad can be broadcasted to multiple source pads, the driver will need significant re-architecture to make that possible. The two users of this IP in mainline linux are TI Shim and StarFive JH7110 CAMSS, and both have only integrated the first source pad i.e stream0 of this IP. So for now keep it simple and only allow 1-to-1 mapping of streams from sink to source, without any broadcasting.

With stream routing now supported in the driver, implement the enable_stream and disable_stream hooks in place of the stream-unaware s_stream hook.

This allows consumer devices like a DMA bridge or ISP, to enable particular streams on a source pad, which in turn can be used to enable only particular streams on the CSI-TX device connected on the sink pad.

Implement a fallback s_stream hook that internally calls enable_stream on each source pad, for consumer drivers that don't use multi-stream APIs to still work.

---
  drivers/media/platform/cadence/cdns-csi2rx.c | 407 ++++++++++++++++++++-------
  1 file changed, 313 insertions(+), 94 deletions(-)
drivers/media/platform/cadence/cdns-csi2rx.c +3 -4
index 2ec34fc9c524..b0c91a9c65e8 100644
@@ -90,6 +90,7 @@ struct csi2rx_priv {
  	struct reset_control		*pixel_rst[CSI2RX_STREAMS_MAX];
  	struct phy			*dphy;
 
+	u32				vc_select[CSI2RX_STREAMS_MAX];
  	u8				lanes[CSI2RX_LANES_MAX];
  	u8				num_lanes;
  	u8				max_lanes;
@@ -179,27 +180,43 @@ static void csi2rx_reset(struct csi2rx_priv *csi2rx)
 
  static int csi2rx_configure_ext_dphy(struct csi2rx_priv *csi2rx)
  {
+	struct v4l2_ctrl_handler *handler = csi2rx->source_subdev->ctrl_handler;
  	union phy_configure_opts opts = { };
  	struct phy_configure_opts_mipi_dphy *cfg = &opts.mipi_dphy;
-	struct v4l2_subdev_format sd_fmt = {
-		.which	= V4L2_SUBDEV_FORMAT_ACTIVE,
-		.pad	= CSI2RX_PAD_SINK,
-	};
+	struct v4l2_mbus_framefmt *framefmt;

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_reset(struct csi2rx_priv *csi2rx) · line 192

Quoted 10 lines
drivers/media/platform/cadence/cdns-csi2rx.c +2 -4
+	struct v4l2_subdev_state *state;
  	const struct csi2rx_fmt *fmt;
  	s64 link_freq;
  	int ret;
 
-	ret = v4l2_subdev_call_state_active(&csi2rx->subdev, pad, get_fmt,
-					    &sd_fmt);
-	if (ret < 0)
-		return ret;
+	if (v4l2_ctrl_find(handler, V4L2_CID_LINK_FREQ)) {

Do you need to do this by yourself ? afaict v4l2_get_link_freq() already checks if V4L2_CID_LINK_FREQ is available, and if not, fallsback to use PIXEL_RATE.

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_reset(struct csi2rx_priv *csi2rx) · line 204

Quoted 12 lines
drivers/media/platform/cadence/cdns-csi2rx.c +12 -0
+		link_freq = v4l2_get_link_freq(handler, 0, 0);
+	} else {
+		state = v4l2_subdev_get_locked_active_state(&csi2rx->subdev);
+		framefmt = v4l2_subdev_state_get_format(state, CSI2RX_PAD_SINK,
+							0);
+
+		if (framefmt) {
+			fmt = csi2rx_get_fmt_by_code(framefmt->code);
+		} else {
+			dev_err(csi2rx->dev,
+				"Did not find active sink format\n");
+			return -EINVAL;

Is this possibile ?

Quoted 8 lines

drivers/media/platform/cadence/cdns-csi2rx.c static void csi2rx_reset(struct csi2rx_priv *csi2rx) · line 208

Quoted 5 lines
drivers/media/platform/cadence/cdns-csi2rx.c +3 -1
+		}
 
-	fmt = csi2rx_get_fmt_by_code(sd_fmt.format.code);
+		link_freq = v4l2_get_link_freq(handler, fmt->bpp,
+					       2 * csi2rx->num_lanes);

Do we want to allow falling back on PIXEL_RATE for multiplexed transmitters ? I presume this will give you invalid results anyway.

Quoted 3 lines

This is mostly done to avoid breaking any single stream sensor that does not have the LINK_FREQ control, and was working with this bridge before. Thus the warning below for multi-format sources.

Is it possible to allow usage of PIXEL_LINK only for non-multiplexed transmitters ?

Quoted 4 lines
Quoted 3 lines

I would simply call v4l2_get_link_freq(handler, 0, 0) to force the usage of LINK_FREQ which will become mandatory for transmitters

Quoted 7 lines
Quoted 3 lines

Ah I did not know LINK_FREQ will be mandatory soon. Any threads I can look at where this was discussed?

I meant mandatory for multiplexed transmitters, which will have to be 'forced' to use LINK_FREQ as PIXEL_RATE doesn't make much sense for them

In CAL driver's multiplexed streams patch I have:

Quoted 27 lines
	/*
	 * v4l2_get_link_freq() uses V4L2_CID_LINK_FREQ first, and falls back
	 * to V4L2_CID_PIXEL_RATE if V4L2_CID_LINK_FREQ is not available.
	 *
	 * With multistream input there is no single pixel rate, and thus we
	 * cannot use V4L2_CID_PIXEL_RATE, so we pass 0 as the bpp which
	 * causes v4l2_get_link_freq() to return an error if it falls back to
	 * V4L2_CID_PIXEL_RATE.
	 */

	state = v4l2_subdev_get_locked_active_state(&phy->subdev);

	if (state->routing.num_routes > 1) {
		bpp = 0;
	} else {
		struct v4l2_subdev_route *route = &state->routing.routes[0];
		const struct cal_format_info *fmtinfo;
		struct v4l2_mbus_framefmt *fmt;

		fmt = v4l2_subdev_state_get_format(state,
			route->sink_pad, route->sink_stream);

		fmtinfo = cal_format_by_code(fmt->code);
		if (!fmtinfo)
			return -EINVAL;

		bpp = fmtinfo->bpp;
Quoted 8 lines
	}

	freq = v4l2_get_link_freq(phy->source->ctrl_handler, bpp, 2 * num_lanes);
	if (freq < 0) {
		phy_err(phy, "failed to get link freq for subdev '%s'\n",
			phy->source->name);
		return freq;
	}

Tomi

author
#39 lore raw

PATCH v2 12/13 media: ti: j721e-csi2rx: add multistream support

In reply to Jai Luthra

Each CSI2 stream can be multiplexed into 4 independent streams, each identified by its virtual channel number. To capture this multiplexed stream, the application needs to tell the driver how it wants to route the data. It needs to specify which context should process which stream. This is done via the new routing APIs.

Add ioctls to accept routing information from the application and save that in the driver. This can be used when starting streaming on a context to determine which route and consequently which virtual channel it should process.

Support the new enable_stream()/disable_stream() APIs in the subdev instead of s_stream() hook. We wait for the userspace to start capturing on all video nodes that have active routes, and once all video nodes are STREAMON, we request the source to enable_stream() for all the sink streams.

---
 .../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c  | 231 +++++++++++++++++----
 1 file changed, 185 insertions(+), 46 deletions(-)
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +185 -46
index c0916ca1a6f8..84b972c251e8 100644
@@ -124,6 +124,7 @@ struct ti_csi2rx_dev {
 	struct v4l2_subdev		*source;
 	struct v4l2_subdev		subdev;
 	struct ti_csi2rx_ctx		ctx[TI_CSI2RX_MAX_CTX];
+	u64				streams_mask; /* Enabled sink streams */
 	/* Buffer to drain stale data from PSI-L endpoint */
 	struct {
 		void			*vaddr;
@@ -535,10 +536,6 @@ static void ti_csi2rx_setup_shim(struct ti_csi2rx_ctx *ctx)
 
 	fmt = find_format_by_fourcc(ctx->v_fmt.fmt.pix.pixelformat);
 
-	/* De-assert the pixel interface reset. */
-	reg = SHIM_CNTL_PIX_RST;
-	writel(reg, csi->shim + SHIM_CNTL);
-
 	reg = SHIM_DMACNTX_EN;
 	reg |= FIELD_PREP(SHIM_DMACNTX_FMT, fmt->csi_dt);
 
@@ -881,8 +878,12 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 	struct ti_csi2rx_dev *csi = ctx->csi;
 	struct ti_csi2rx_dma *dma = &ctx->dma;
 	struct ti_csi2rx_buffer *buf;
+	struct v4l2_subdev_krouting *routing;
+	struct v4l2_subdev_route *route = NULL;
+	struct media_pad *remote_pad;
 	unsigned long flags;
-	int ret = 0;
+	int ret = 0, i;
+	struct v4l2_subdev_state *state;
 
 	spin_lock_irqsave(&dma->lock, flags);
 	if (list_empty(&dma->queue))
@@ -895,6 +896,40 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 	if (ret)
 		goto err;
 
+	remote_pad = media_entity_remote_source_pad_unique(ctx->pad.entity);
+	if (!remote_pad) {
+		ret = -ENODEV;
+		goto err;
+	}
+
+	state = v4l2_subdev_lock_and_get_active_state(&csi->subdev);
+
+	routing = &state->routing;
+
+	/* Find the stream to process. */
+	for (i = 0; i < routing->num_routes; i++) {
+		struct v4l2_subdev_route *r = &routing->routes[i];
+
+		if (!(r->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
+			continue;
+
+		if (r->source_pad != remote_pad->index)
+			continue;
+
+		route = r;
+		break;
+	}
+
+	if (!route) {
+		ret = -ENODEV;
+		v4l2_subdev_unlock_state(state);
+		goto err;
+	}
+
+	ctx->stream = route->sink_stream;
+
+	v4l2_subdev_unlock_state(state);
+
 	ret = ti_csi2rx_get_vc(ctx);
 	if (ret == -ENOIOCTLCMD)
 		ctx->vc = 0;
@@ -921,7 +956,10 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 	dma->state = TI_CSI2RX_DMA_ACTIVE;
 	spin_unlock_irqrestore(&dma->lock, flags);
 
-	ret = v4l2_subdev_call(&csi->subdev, video, s_stream, 1);
+	/* Start stream 0, we don't allow multiple streams on the source pad */
+	ret = v4l2_subdev_enable_streams(&csi->subdev,
+					 TI_CSI2RX_PAD_FIRST_SOURCE + ctx->idx,
+					 BIT(0));
 	if (ret)
 		goto err_dma;
 
@@ -944,12 +982,16 @@ static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
 	struct ti_csi2rx_dev *csi = ctx->csi;
 	int ret;
 
-	video_device_pipeline_stop(&ctx->vdev);
-
+	/* assert pixel reset to prevent stale data */
 	writel(0, csi->shim + SHIM_CNTL);
+
+	video_device_pipeline_stop(&ctx->vdev);
 	writel(0, csi->shim + SHIM_DMACNTX(ctx->idx));
 
-	ret = v4l2_subdev_call(&csi->subdev, video, s_stream, 0);
+	ret = v4l2_subdev_disable_streams(&csi->subdev,
+					  TI_CSI2RX_PAD_FIRST_SOURCE + ctx->idx,
+					  BIT(0));
+
 	if (ret)
 		dev_err(csi->dev, "Failed to stop subdev stream\n");
 
@@ -995,8 +1037,8 @@ static int ti_csi2rx_sd_set_fmt(struct v4l2_subdev *sd,
 	}
 	*fmt = format->format;
 
-	fmt = v4l2_subdev_state_get_format(state, TI_CSI2RX_PAD_FIRST_SOURCE,
-					   format->stream);
+	fmt = v4l2_subdev_state_get_opposite_stream_format(state, format->pad,
+							   format->stream);
 	if (!fmt) {
 		ret = -EINVAL;
 		goto out;
@@ -1007,72 +1049,169 @@ static int ti_csi2rx_sd_set_fmt(struct v4l2_subdev *sd,
 	return ret;
 }
 
+static int _ti_csi2rx_sd_set_routing(struct v4l2_subdev *sd,
+				     struct v4l2_subdev_state *state,
+				     struct v4l2_subdev_krouting *routing)
+{
+	int ret;
+
+	const struct v4l2_mbus_framefmt format = {
+		.width = 640,
+		.height = 480,
+		.code = MEDIA_BUS_FMT_UYVY8_1X16,
+		.field = V4L2_FIELD_NONE,
+		.colorspace = V4L2_COLORSPACE_SRGB,
+		.ycbcr_enc = V4L2_YCBCR_ENC_601,
+		.quantization = V4L2_QUANTIZATION_LIM_RANGE,
+		.xfer_func = V4L2_XFER_FUNC_SRGB,
+	};
+
+	ret = v4l2_subdev_routing_validate(sd, routing,
+					   V4L2_SUBDEV_ROUTING_ONLY_1_TO_1 |
+					   V4L2_SUBDEV_ROUTING_NO_SOURCE_MULTIPLEXING);
+
+	if (ret)
+		return ret;
+
+	/* Only stream ID 0 allowed on source pads */
+	for (unsigned int i = 0; i < routing->num_routes; ++i) {
+		const struct v4l2_subdev_route *route = &routing->routes[i];
+
+		if (route->source_stream != 0)
+			return -EINVAL;
+	}
+
+	ret = v4l2_subdev_set_routing_with_fmt(sd, state, routing, &format);
+
+	return ret;
+}
+
+static int ti_csi2rx_sd_set_routing(struct v4l2_subdev *sd,
+				    struct v4l2_subdev_state *state,
+				    enum v4l2_subdev_format_whence which,
+				    struct v4l2_subdev_krouting *routing)
+{
+	struct ti_csi2rx_dev *csi = to_csi2rx_dev(sd);
+
+	if (csi->enable_count > 0)
+		return -EBUSY;
+
+	return _ti_csi2rx_sd_set_routing(sd, state, routing);
+}
+
 static int ti_csi2rx_sd_init_state(struct v4l2_subdev *sd,
 				   struct v4l2_subdev_state *state)
 {
-	struct v4l2_subdev_format format = {
-		.pad = TI_CSI2RX_PAD_SINK,
-		.format = {
-			.width = 640,
-			.height = 480,
-			.code = MEDIA_BUS_FMT_UYVY8_1X16,
-			.field = V4L2_FIELD_NONE,
-			.colorspace = V4L2_COLORSPACE_SRGB,
-			.ycbcr_enc = V4L2_YCBCR_ENC_601,
-			.quantization = V4L2_QUANTIZATION_LIM_RANGE,
-			.xfer_func = V4L2_XFER_FUNC_SRGB,
-		},
+	struct v4l2_subdev_route routes[] = { {
+		.sink_pad = 0,
+		.sink_stream = 0,
+		.source_pad = TI_CSI2RX_PAD_FIRST_SOURCE,
+		.source_stream = 0,
+		.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
+	} };
+
+	struct v4l2_subdev_krouting routing = {
+		.num_routes = 1,
+		.routes = routes,
 	};
 
-	return ti_csi2rx_sd_set_fmt(sd, state, &format);
+	/* Initialize routing to single route to the fist source pad */
+	return _ti_csi2rx_sd_set_routing(sd, state, &routing);
 }
 
-static int ti_csi2rx_sd_s_stream(struct v4l2_subdev *sd, int enable)
+static int ti_csi2rx_sd_enable_streams(struct v4l2_subdev *sd,
+				       struct v4l2_subdev_state *state,
+				       u32 pad, u64 streams_mask)
 {
 	struct ti_csi2rx_dev *csi = to_csi2rx_dev(sd);
+	struct v4l2_subdev_route *route;
+	struct media_pad *remote_pad;
+	u64 sink_streams;
 	int ret = 0;
+	u32 reg;
 
+	remote_pad = media_entity_remote_source_pad_unique(&csi->subdev.entity);
+	if (!remote_pad)
+		return -ENODEV;
+	sink_streams = v4l2_subdev_state_xlate_streams(state, pad,
+						       TI_CSI2RX_PAD_SINK,
+						       &streams_mask);
 	mutex_lock(&csi->mutex);
 
-	if (enable) {
-		if (csi->enable_count > 0) {
-			csi->enable_count++;
+	csi->streams_mask |= sink_streams;
+	/*
+	 * Check if all active streams are enabled, and only then start
+	 * streaming on the source
+	 */
+	for_each_active_route(&state->routing, route) {
+		if (!(csi->streams_mask & BIT_ULL(route->sink_stream)))
 			goto out;
-		}
+	}
 
-		ret = v4l2_subdev_call(csi->source, video, s_stream, 1);
-		if (ret)
-			goto out;
+	/* De-assert the pixel interface reset. */
+	reg = SHIM_CNTL_PIX_RST;
+	writel(reg, csi->shim + SHIM_CNTL);
 
+	ret = v4l2_subdev_enable_streams(csi->source, remote_pad->index,
+					 csi->streams_mask);
+out:
+	if (!ret)
 		csi->enable_count++;
-	} else {
-		if (csi->enable_count == 0) {
-			ret = -EINVAL;
-			goto out;
-		}
+	mutex_unlock(&csi->mutex);
 
-		if (--csi->enable_count > 0)
-			goto out;
+	return ret;
+}
+
+static int ti_csi2rx_sd_disable_streams(struct v4l2_subdev *sd,
+					struct v4l2_subdev_state *state,
+					u32 pad, u64 streams_mask)
+{
+	struct ti_csi2rx_dev *csi = to_csi2rx_dev(sd);
+	struct v4l2_subdev_route *route;
+	struct media_pad *remote_pad;
+	u64 sink_streams;
+	int ret = 0;
 
-		ret = v4l2_subdev_call(csi->source, video, s_stream, 0);
+	remote_pad = media_entity_remote_source_pad_unique(&csi->subdev.entity);
+	if (!remote_pad)
+		return -ENODEV;
+	sink_streams = v4l2_subdev_state_xlate_streams(state, pad,
+						       TI_CSI2RX_PAD_SINK,
+						       &streams_mask);
+
+	mutex_lock(&csi->mutex);
+	if (csi->enable_count == 0) {
+		ret = -EINVAL;
+		goto out;
 	}
 
+	/* Check if any of the active streams are already disabled */
+	for_each_active_route(&state->routing, route) {
+		if (!(csi->streams_mask & BIT_ULL(route->sink_stream))) {
+			--csi->enable_count;
+			goto out;
+		}
+	}
+	/* Disable all streams on the source */
+	ret = v4l2_subdev_disable_streams(csi->source, remote_pad->index,
+					  csi->streams_mask);
+	if (!ret)
+		--csi->enable_count;
 out:
+	csi->streams_mask &= ~sink_streams;
 	mutex_unlock(&csi->mutex);
 	return ret;
 }
 
 static const struct v4l2_subdev_pad_ops ti_csi2rx_subdev_pad_ops = {
+	.set_routing = ti_csi2rx_sd_set_routing,
 	.get_fmt = v4l2_subdev_get_fmt,
 	.set_fmt = ti_csi2rx_sd_set_fmt,
-};
-
-static const struct v4l2_subdev_video_ops ti_csi2rx_subdev_video_ops = {
-	.s_stream = ti_csi2rx_sd_s_stream,
+	.enable_streams = ti_csi2rx_sd_enable_streams,
+	.disable_streams = ti_csi2rx_sd_disable_streams,
 };
 
 static const struct v4l2_subdev_ops ti_csi2rx_subdev_ops = {
-	.video = &ti_csi2rx_subdev_video_ops,
 	.pad = &ti_csi2rx_subdev_pad_ops,
 };
 
@@ -1264,7 +1403,7 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
 	v4l2_subdev_init(sd, &ti_csi2rx_subdev_ops);
 	sd->internal_ops = &ti_csi2rx_internal_ops;
 	sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
-	sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
+	sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_STREAMS;
 	strscpy(sd->name, dev_name(csi->dev), sizeof(sd->name));
 	sd->dev = csi->dev;
 	sd->entity.ops = &ti_csi2rx_subdev_entity_ops;
Signature · 1 line
2.43.0
#40 lore raw

Re: [PATCH v2 12/13] media: ti: j721e-csi2rx: add multistream support

Quoted 67 lines · Jai Luthra

Each CSI2 stream can be multiplexed into 4 independent streams, each identified by its virtual channel number. To capture this multiplexed stream, the application needs to tell the driver how it wants to route the data. It needs to specify which context should process which stream. This is done via the new routing APIs.

Add ioctls to accept routing information from the application and save that in the driver. This can be used when starting streaming on a context to determine which route and consequently which virtual channel it should process.

Support the new enable_stream()/disable_stream() APIs in the subdev instead of s_stream() hook. We wait for the userspace to start capturing on all video nodes that have active routes, and once all video nodes are STREAMON, we request the source to enable_stream() for all the sink streams.

---
  .../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c  | 231 +++++++++++++++++----
  1 file changed, 185 insertions(+), 46 deletions(-)
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +8 -5
index c0916ca1a6f8..84b972c251e8 100644
@@ -124,6 +124,7 @@ struct ti_csi2rx_dev {
  	struct v4l2_subdev		*source;
  	struct v4l2_subdev		subdev;
  	struct ti_csi2rx_ctx		ctx[TI_CSI2RX_MAX_CTX];
+	u64				streams_mask; /* Enabled sink streams */
  	/* Buffer to drain stale data from PSI-L endpoint */
  	struct {
  		void			*vaddr;
@@ -535,10 +536,6 @@ static void ti_csi2rx_setup_shim(struct ti_csi2rx_ctx *ctx)
 
  	fmt = find_format_by_fourcc(ctx->v_fmt.fmt.pix.pixelformat);
 
-	/* De-assert the pixel interface reset. */
-	reg = SHIM_CNTL_PIX_RST;
-	writel(reg, csi->shim + SHIM_CNTL);
-
  	reg = SHIM_DMACNTX_EN;
  	reg |= FIELD_PREP(SHIM_DMACNTX_FMT, fmt->csi_dt);
 
@@ -881,8 +878,12 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
  	struct ti_csi2rx_dev *csi = ctx->csi;
  	struct ti_csi2rx_dma *dma = &ctx->dma;
  	struct ti_csi2rx_buffer *buf;
+	struct v4l2_subdev_krouting *routing;
+	struct v4l2_subdev_route *route = NULL;
+	struct media_pad *remote_pad;
  	unsigned long flags;
-	int ret = 0;
+	int ret = 0, i;
+	struct v4l2_subdev_state *state;
 
  	spin_lock_irqsave(&dma->lock, flags);
  	if (list_empty(&dma->queue))
@@ -895,6 +896,40 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
  	if (ret)
  		goto err;
 
+	remote_pad = media_entity_remote_source_pad_unique(ctx->pad.entity);
+	if (!remote_pad) {

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count) · line 910

Quoted 10 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +10 -0
+		ret = -ENODEV;
+		goto err;
+	}
+
+	state = v4l2_subdev_lock_and_get_active_state(&csi->subdev);
+
+	routing = &state->routing;
+
+	/* Find the stream to process. */
+	for (i = 0; i < routing->num_routes; i++) {

for_each_active_route()

Quoted 69 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +35 -6
+		struct v4l2_subdev_route *r = &routing->routes[i];
+
+		if (!(r->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
+			continue;
+
+		if (r->source_pad != remote_pad->index)
+			continue;
+
+		route = r;
+		break;
+	}
+
+	if (!route) {
+		ret = -ENODEV;
+		v4l2_subdev_unlock_state(state);
+		goto err;
+	}
+
+	ctx->stream = route->sink_stream;
+
+	v4l2_subdev_unlock_state(state);
+
  	ret = ti_csi2rx_get_vc(ctx);
  	if (ret == -ENOIOCTLCMD)
  		ctx->vc = 0;
@@ -921,7 +956,10 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
  	dma->state = TI_CSI2RX_DMA_ACTIVE;
  	spin_unlock_irqrestore(&dma->lock, flags);
 
-	ret = v4l2_subdev_call(&csi->subdev, video, s_stream, 1);
+	/* Start stream 0, we don't allow multiple streams on the source pad */
+	ret = v4l2_subdev_enable_streams(&csi->subdev,
+					 TI_CSI2RX_PAD_FIRST_SOURCE + ctx->idx,
+					 BIT(0));
  	if (ret)
  		goto err_dma;
 
@@ -944,12 +982,16 @@ static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
  	struct ti_csi2rx_dev *csi = ctx->csi;
  	int ret;
 
-	video_device_pipeline_stop(&ctx->vdev);
-
+	/* assert pixel reset to prevent stale data */
  	writel(0, csi->shim + SHIM_CNTL);
+
+	video_device_pipeline_stop(&ctx->vdev);
  	writel(0, csi->shim + SHIM_DMACNTX(ctx->idx));
 
-	ret = v4l2_subdev_call(&csi->subdev, video, s_stream, 0);
+	ret = v4l2_subdev_disable_streams(&csi->subdev,
+					  TI_CSI2RX_PAD_FIRST_SOURCE + ctx->idx,
+					  BIT(0));
+
  	if (ret)
  		dev_err(csi->dev, "Failed to stop subdev stream\n");
 
@@ -995,8 +1037,8 @@ static int ti_csi2rx_sd_set_fmt(struct v4l2_subdev *sd,
  	}
  	*fmt = format->format;
 
-	fmt = v4l2_subdev_state_get_format(state, TI_CSI2RX_PAD_FIRST_SOURCE,
-					   format->stream);
+	fmt = v4l2_subdev_state_get_opposite_stream_format(state, format->pad,
+							   format->stream);
  	if (!fmt) {
  		ret = -EINVAL;
  		goto out;

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int ti_csi2rx_sd_set_fmt(struct v4l2_subdev *sd, · line 1058

Quoted 10 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +7 -0
@@ -1007,72 +1049,169 @@ static int ti_csi2rx_sd_set_fmt(struct v4l2_subdev *sd,
  	return ret;
  }
 
+static int _ti_csi2rx_sd_set_routing(struct v4l2_subdev *sd,
+				     struct v4l2_subdev_state *state,
+				     struct v4l2_subdev_krouting *routing)
+{
+	int ret;
+
+	const struct v4l2_mbus_framefmt format = {

static const

Tomi

author
#41 lore raw

PATCH v2 13/13 media: ti: j721e-csi2rx: Submit all available buffers

In reply to Jai Luthra

We already make sure to submit all available buffers to DMA in each DMA completion callback.

Move that logic in a separate function, and use it during stream start as well, as most application queue all their buffers before stream on.

---
 .../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c  | 43 ++++++++++++----------
 1 file changed, 24 insertions(+), 19 deletions(-)
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +24 -19
index 84b972c251e8..a258477b9084 100644
@@ -637,6 +637,27 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_ctx *ctx)
 	return ret;
 }
 
+static int ti_csi2rx_dma_submit_pending(struct ti_csi2rx_ctx *ctx)
+{
+	struct ti_csi2rx_dma *dma = &ctx->dma;
+	struct ti_csi2rx_buffer *buf;
+	int ret = 0;
+
+	/* If there are more buffers to process then start their transfer. */
+	while (!list_empty(&dma->queue)) {
+		buf = list_entry(dma->queue.next, struct ti_csi2rx_buffer, list);
+		ret = ti_csi2rx_start_dma(ctx, buf);
+		if (ret) {
+			dev_err(ctx->csi->dev,
+				"Failed to queue the next buffer for DMA\n");
+			vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
+			break;
+		}
+		list_move_tail(&buf->list, &dma->submitted);
+	}
+	return ret;
+}
+
 static void ti_csi2rx_dma_callback(void *param)
 {
 	struct ti_csi2rx_buffer *buf = param;
@@ -657,18 +678,7 @@ static void ti_csi2rx_dma_callback(void *param)
 	vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
 	list_del(&buf->list);
 
-	/* If there are more buffers to process then start their transfer. */
-	while (!list_empty(&dma->queue)) {
-		buf = list_entry(dma->queue.next, struct ti_csi2rx_buffer, list);
-
-		if (ti_csi2rx_start_dma(ctx, buf)) {
-			dev_err(ctx->csi->dev,
-				"Failed to queue the next buffer for DMA\n");
-			vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
-		} else {
-			list_move_tail(&buf->list, &dma->submitted);
-		}
-	}
+	ti_csi2rx_dma_submit_pending(ctx);
 
 	if (list_empty(&dma->submitted))
 		dma->state = TI_CSI2RX_DMA_IDLE;
@@ -877,7 +887,6 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 	struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vq);
 	struct ti_csi2rx_dev *csi = ctx->csi;
 	struct ti_csi2rx_dma *dma = &ctx->dma;
-	struct ti_csi2rx_buffer *buf;
 	struct v4l2_subdev_krouting *routing;
 	struct v4l2_subdev_route *route = NULL;
 	struct media_pad *remote_pad;
@@ -943,16 +952,13 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 	ctx->sequence = 0;
 
 	spin_lock_irqsave(&dma->lock, flags);
-	buf = list_entry(dma->queue.next, struct ti_csi2rx_buffer, list);
 
-	ret = ti_csi2rx_start_dma(ctx, buf);
+	ret = ti_csi2rx_dma_submit_pending(ctx);
 	if (ret) {
-		dev_err(csi->dev, "Failed to start DMA: %d\n", ret);
 		spin_unlock_irqrestore(&dma->lock, flags);
-		goto err_pipeline;
+		goto err_dma;
 	}
 
-	list_move_tail(&buf->list, &dma->submitted);
 	dma->state = TI_CSI2RX_DMA_ACTIVE;
 	spin_unlock_irqrestore(&dma->lock, flags);
 
@@ -967,7 +973,6 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 
 err_dma:
 	ti_csi2rx_stop_dma(ctx);
-err_pipeline:
 	video_device_pipeline_stop(&ctx->vdev);
 	writel(0, csi->shim + SHIM_CNTL);
 	writel(0, csi->shim + SHIM_DMACNTX(ctx->idx));
Signature · 1 line
2.43.0
Reviewed-by on v2 13/13
#42 lore raw

Re: [PATCH v2 13/13] media: ti: j721e-csi2rx: Submit all available buffers

Quoted 92 lines · Jai Luthra

We already make sure to submit all available buffers to DMA in each DMA completion callback.

Move that logic in a separate function, and use it during stream start as well, as most application queue all their buffers before stream on.

---
  .../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c  | 43 ++++++++++++----------
  1 file changed, 24 insertions(+), 19 deletions(-)
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +24 -18
index 84b972c251e8..a258477b9084 100644
@@ -637,6 +637,27 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_ctx *ctx)
  	return ret;
  }
 
+static int ti_csi2rx_dma_submit_pending(struct ti_csi2rx_ctx *ctx)
+{
+	struct ti_csi2rx_dma *dma = &ctx->dma;
+	struct ti_csi2rx_buffer *buf;
+	int ret = 0;
+
+	/* If there are more buffers to process then start their transfer. */
+	while (!list_empty(&dma->queue)) {
+		buf = list_entry(dma->queue.next, struct ti_csi2rx_buffer, list);
+		ret = ti_csi2rx_start_dma(ctx, buf);
+		if (ret) {
+			dev_err(ctx->csi->dev,
+				"Failed to queue the next buffer for DMA\n");
+			vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
+			break;
+		}
+		list_move_tail(&buf->list, &dma->submitted);
+	}
+	return ret;
+}
+
  static void ti_csi2rx_dma_callback(void *param)
  {
  	struct ti_csi2rx_buffer *buf = param;
@@ -657,18 +678,7 @@ static void ti_csi2rx_dma_callback(void *param)
  	vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
  	list_del(&buf->list);
 
-	/* If there are more buffers to process then start their transfer. */
-	while (!list_empty(&dma->queue)) {
-		buf = list_entry(dma->queue.next, struct ti_csi2rx_buffer, list);
-
-		if (ti_csi2rx_start_dma(ctx, buf)) {
-			dev_err(ctx->csi->dev,
-				"Failed to queue the next buffer for DMA\n");
-			vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
-		} else {
-			list_move_tail(&buf->list, &dma->submitted);
-		}
-	}
+	ti_csi2rx_dma_submit_pending(ctx);
 
  	if (list_empty(&dma->submitted))
  		dma->state = TI_CSI2RX_DMA_IDLE;
@@ -877,7 +887,6 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
  	struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vq);
  	struct ti_csi2rx_dev *csi = ctx->csi;
  	struct ti_csi2rx_dma *dma = &ctx->dma;
-	struct ti_csi2rx_buffer *buf;
  	struct v4l2_subdev_krouting *routing;
  	struct v4l2_subdev_route *route = NULL;
  	struct media_pad *remote_pad;
@@ -943,16 +952,13 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
  	ctx->sequence = 0;
 
  	spin_lock_irqsave(&dma->lock, flags);
-	buf = list_entry(dma->queue.next, struct ti_csi2rx_buffer, list);
 
-	ret = ti_csi2rx_start_dma(ctx, buf);
+	ret = ti_csi2rx_dma_submit_pending(ctx);
  	if (ret) {
-		dev_err(csi->dev, "Failed to start DMA: %d\n", ret);
  		spin_unlock_irqrestore(&dma->lock, flags);
-		goto err_pipeline;
+		goto err_dma;
  	}
 
-	list_move_tail(&buf->list, &dma->submitted);
  	dma->state = TI_CSI2RX_DMA_ACTIVE;
  	spin_unlock_irqrestore(&dma->lock, flags);
 

drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count) · line 978

Quoted 7 lines
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c +0 -1
@@ -967,7 +973,6 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
 
  err_dma:
  	ti_csi2rx_stop_dma(ctx);
-err_pipeline:
  	video_device_pipeline_stop(&ctx->vdev);
  	writel(0, csi->shim + SHIM_CNTL);
  	writel(0, csi->shim + SHIM_DMACNTX(ctx->idx));

Tomi

#43 lore raw
In reply to Jai Luthra

Hi Jai,

Quoted 57 lines · Jai Luthra

This series adds multi-stream support for Cadence CSI2RX and TI CSI2RX Shim drivers.

PATCH 1: Runtime Power Management for Cadence CSI2RX
PATCH 2-7: Support multiple DMA contexts/video nodes in TI CSI2RX
PATCH 8-9: Use get_frame_desc to propagate virtual channel information
across Cadence and TI CSI-RX subdevs
PATCH 10-12: Use new multi-stream APIs across the drivers to support
multiplexed cameras from sources like UB960 (FPDLink)
PATCH 13: Optimize stream on by submitting all queued buffers to DMA

This applies on top of today's linux-next (next-20240626) (also tested rebase with media_stage.git master)

---
Changes in v2:

- Change the multi-camera capture architecture to be similar to that of Tomi's RPi5 FE series, where the driver will wait for userspace to start streaming on all "actively routed" video nodes before starting streaming on the source. This simplifies things a lot from the HW perspective, which might run into deadlocks due to a shared FIFO between multiple DMA channels.

- Drop a few fixes that were posted separately and are already merged
- Fix dtschema warnings reported by Rob on [02/13]
- Fix warnings for uninitialized used_vc variable in cdns-csi2rx.c
- Return -EBUSY if someone updates routes for j721e-csi2rx subdev while streaming
- Only allow single-streams to be routed to the source pads (linked to video nodes) of the j721e-csi2rx device
- Squash the patches marked "SQUASH" in the v1 RFC series

- Link to RFC (v1):

   https://lore.kernel.org/r/[email protected]

---
Jai Luthra (8):
dt-bindings: media: ti,j721e-csi2rx-shim: Support 32 dma chans
media: ti: j721e-csi2rx: separate out device and context
media: ti: j721e-csi2rx: add a subdev for the core device
media: ti: j721e-csi2rx: add support for processing virtual channels
media: cadence: csi2rx: Use new enable stream APIs
media: cadence: csi2rx: Enable multi-stream support
media: ti: j721e-csi2rx: add multistream support
media: ti: j721e-csi2rx: Submit all available buffers

Jayshri Pawar (1):
media: cadence: csi2rx: Support runtime PM

Pratyush Yadav (4):
media: ti: j721e-csi2rx: prepare SHIM code for multiple contexts
media: ti: j721e-csi2rx: allocate DMA channel based on context index
media: ti: j721e-csi2rx: get number of contexts from device tree
media: cadence: csi2rx: add get_frame_desc wrapper

Quoted 7 lines
  .../bindings/media/ti,j721e-csi2rx-shim.yaml       |  39 +-
  drivers/media/platform/cadence/cdns-csi2rx.c       | 440 +++++++++--
  .../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c  | 879 ++++++++++++++++-----
  3 files changed, 1071 insertions(+), 287 deletions(-)

---
base-commit: df9574a57d02b265322e77fb8628d4d33641dda9
change-id: 20240221-multistream-fbba6ffe47a3

You have based this series on top of your private branch. Don't do that. Base on top of a kernel tag, or a commonly known tree (linux-media-stage for example), and preferably mention the base in the cover letter.

Your private branch contains e.g. dtsos needed for testing. If you have such a branch, you should point to it in the cover letter as it's valuable for reviewers/testers.

Only base on top of a private branch if your patches compile-time depend on something from there, and in that case point to the branch and mention this dependency clearly in the cover letter.

Tomi

author
#44 lore raw

Hi Tomi,

Quoted 15 lines · Tomi Valkeinen

Hi Jai,

Quoted 12 lines · Jai Luthra

This series adds multi-stream support for Cadence CSI2RX and TI CSI2RX Shim drivers.

PATCH 1: Runtime Power Management for Cadence CSI2RX
PATCH 2-7: Support multiple DMA contexts/video nodes in TI CSI2RX
PATCH 8-9: Use get_frame_desc to propagate virtual channel information
across Cadence and TI CSI-RX subdevs
PATCH 10-12: Use new multi-stream APIs across the drivers to support
multiplexed cameras from sources like UB960 (FPDLink)
PATCH 13: Optimize stream on by submitting all queued buffers to DMA

This applies on top of today's linux-next (next-20240626)

This series is based on top of next-20240626

Quoted 53 lines
Quoted 45 lines

(also tested rebase with media_stage.git master)

---
Changes in v2:

- Change the multi-camera capture architecture to be similar to that of Tomi's RPi5 FE series, where the driver will wait for userspace to start streaming on all "actively routed" video nodes before starting streaming on the source. This simplifies things a lot from the HW perspective, which might run into deadlocks due to a shared FIFO between multiple DMA channels.

- Drop a few fixes that were posted separately and are already merged
- Fix dtschema warnings reported by Rob on [02/13]
- Fix warnings for uninitialized used_vc variable in cdns-csi2rx.c
- Return -EBUSY if someone updates routes for j721e-csi2rx subdev while streaming
- Only allow single-streams to be routed to the source pads (linked to video nodes) of the j721e-csi2rx device
- Squash the patches marked "SQUASH" in the v1 RFC series

- Link to RFC (v1):

   https://lore.kernel.org/r/[email protected]

---
Jai Luthra (8):
dt-bindings: media: ti,j721e-csi2rx-shim: Support 32 dma chans
media: ti: j721e-csi2rx: separate out device and context
media: ti: j721e-csi2rx: add a subdev for the core device
media: ti: j721e-csi2rx: add support for processing virtual channels
media: cadence: csi2rx: Use new enable stream APIs
media: cadence: csi2rx: Enable multi-stream support
media: ti: j721e-csi2rx: add multistream support
media: ti: j721e-csi2rx: Submit all available buffers

Jayshri Pawar (1):
media: cadence: csi2rx: Support runtime PM

Pratyush Yadav (4):
media: ti: j721e-csi2rx: prepare SHIM code for multiple contexts
media: ti: j721e-csi2rx: allocate DMA channel based on context index
media: ti: j721e-csi2rx: get number of contexts from device tree
media: cadence: csi2rx: add get_frame_desc wrapper

Quoted 7 lines
  .../bindings/media/ti,j721e-csi2rx-shim.yaml       |  39 +-
  drivers/media/platform/cadence/cdns-csi2rx.c       | 440 +++++++++--
  .../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c  | 879 ++++++++++++++++-----
  3 files changed, 1071 insertions(+), 287 deletions(-)

---
base-commit: df9574a57d02b265322e77fb8628d4d33641dda9
change-id: 20240221-multistream-fbba6ffe47a3

Quoted 3 lines

You have based this series on top of your private branch. Don't do that. Base on top of a kernel tag, or a commonly known tree (linux-media-stage for example), and preferably mention the base in the cover letter.

The base commit SHA populated by b4 is the same as next-20240626 as mentioned above

https://gitlab.com/linux-kernel/linux-next/-/commits/df9574a57d02b265322e77fb8628d4d33641dda9

I chose to not use media-stage as the base, but this series applies cleanly (and compiles) on top of that as well.

Quoted 4 lines

Your private branch contains e.g. dtsos needed for testing. If you have such a branch, you should point to it in the cover letter as it's valuable for reviewers/testers.

Ah my bad, I missed mentioning my github branch that can be used for testing the content of this series. It contains some DTSOs and defconfig updates, along with support for FPDLink/V3Link sensors.

https://github.com/jailuthra/linux/commits/b4/multistream/

Quoted 4 lines

Only base on top of a private branch if your patches compile-time depend on something from there, and in that case point to the branch and mention this dependency clearly in the cover letter.

Makes sense, will take extra care to mention the dependencies and base branch from next version.

Quoted 3 lines

Tomi

Signature · 4 lines
Thanks,
Jai

GPG Fingerprint: 4DE0 D818 E5D5 75E8 D45A AFC5 43DE 91F9 249A 7145
#45 lore raw
Quoted 74 lines · Jai Luthra

Hi Tomi,

Quoted 15 lines · Tomi Valkeinen

Hi Jai,

Quoted 12 lines · Jai Luthra

This series adds multi-stream support for Cadence CSI2RX and TI CSI2RX Shim drivers.

PATCH 1: Runtime Power Management for Cadence CSI2RX
PATCH 2-7: Support multiple DMA contexts/video nodes in TI CSI2RX
PATCH 8-9: Use get_frame_desc to propagate virtual channel information
across Cadence and TI CSI-RX subdevs
PATCH 10-12: Use new multi-stream APIs across the drivers to support
multiplexed cameras from sources like UB960 (FPDLink)
PATCH 13: Optimize stream on by submitting all queued buffers to DMA

This applies on top of today's linux-next (next-20240626)

This series is based on top of next-20240626

Quoted 53 lines
Quoted 52 lines

(also tested rebase with media_stage.git master)

---
Changes in v2:

- Change the multi-camera capture architecture to be similar to that of Tomi's RPi5 FE series, where the driver will wait for userspace to start streaming on all "actively routed" video nodes before starting streaming on the source. This simplifies things a lot from the HW perspective, which might run into deadlocks due to a shared FIFO between multiple DMA channels.

- Drop a few fixes that were posted separately and are already merged
- Fix dtschema warnings reported by Rob on [02/13]
- Fix warnings for uninitialized used_vc variable in cdns-csi2rx.c
- Return -EBUSY if someone updates routes for j721e-csi2rx subdev while streaming
- Only allow single-streams to be routed to the source pads (linked to video nodes) of the j721e-csi2rx device
- Squash the patches marked "SQUASH" in the v1 RFC series

- Link to RFC (v1):

    https://lore.kernel.org/r/[email protected]

---
Jai Luthra (8):
dt-bindings: media: ti,j721e-csi2rx-shim: Support 32 dma chans
media: ti: j721e-csi2rx: separate out device and context
media: ti: j721e-csi2rx: add a subdev for the core device
media: ti: j721e-csi2rx: add support for processing virtual channels
media: cadence: csi2rx: Use new enable stream APIs
media: cadence: csi2rx: Enable multi-stream support
media: ti: j721e-csi2rx: add multistream support
media: ti: j721e-csi2rx: Submit all available buffers

Jayshri Pawar (1):
media: cadence: csi2rx: Support runtime PM

Pratyush Yadav (4):
media: ti: j721e-csi2rx: prepare SHIM code for multiple contexts
media: ti: j721e-csi2rx: allocate DMA channel based on context index
media: ti: j721e-csi2rx: get number of contexts from device tree
media: cadence: csi2rx: add get_frame_desc wrapper

   .../bindings/media/ti,j721e-csi2rx-shim.yaml       |  39 +-
   drivers/media/platform/cadence/cdns-csi2rx.c       | 440 +++++++++--
   .../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c  | 879 ++++++++++++++++-----
   3 files changed, 1071 insertions(+), 287 deletions(-)

---
base-commit: df9574a57d02b265322e77fb8628d4d33641dda9
change-id: 20240221-multistream-fbba6ffe47a3

Quoted 6 lines
Quoted 3 lines

You have based this series on top of your private branch. Don't do that. Base on top of a kernel tag, or a commonly known tree (linux-media-stage for example), and preferably mention the base in the cover letter.

The base commit SHA populated by b4 is the same as next-20240626 as mentioned above

Ah, right, my bad. I took your branch https://github.com/jailuthra/linux/commits/b4/multistream/, and assumed it's the one you used to send these patches. In that branch these patches are not based on linux-next.

Quoted 4 lines

https://gitlab.com/linux-kernel/linux-next/-/commits/df9574a57d02b265322e77fb8628d4d33641dda9

I chose to not use media-stage as the base, but this series applies cleanly (and compiles) on top of that as well.

I'd still recommend media-stage, as that's where these patches would be merged (or just linux-media). linux-next is good for testing, but I wouldn't normally base patches on top of that, or at last send patches based on that.

Quoted 9 lines
Quoted 3 lines

Your private branch contains e.g. dtsos needed for testing. If you have such a branch, you should point to it in the cover letter as it's valuable for reviewers/testers.

Ah my bad, I missed mentioning my github branch that can be used for testing the content of this series. It contains some DTSOs and defconfig updates, along with support for FPDLink/V3Link sensors.

https://github.com/jailuthra/linux/commits/b4/multistream/

Jfyi, I've tested this with am62a and arducam's fpdlink board with imx219 sensors, and works fine for me.

I only tested with pixel streams, I'd like to also add all the patches needed for embedded data and test that (I think all of those have been posted to the lists), but I don't think I'll have time for that right now.

Tomi

Quoted 11 lines
Quoted 3 lines

Only base on top of a private branch if your patches compile-time depend on something from there, and in that case point to the branch and mention this dependency clearly in the cover letter.

Makes sense, will take extra care to mention the dependencies and base branch from next version.

Quoted 3 lines

Tomi

author
#46 lore raw
Quoted 79 lines · Tomi Valkeinen
Quoted 78 lines · Jai Luthra

Hi Tomi,

Quoted 15 lines · Tomi Valkeinen

Hi Jai,

Quoted 12 lines · Jai Luthra

This series adds multi-stream support for Cadence CSI2RX and TI CSI2RX Shim drivers.

PATCH 1: Runtime Power Management for Cadence CSI2RX
PATCH 2-7: Support multiple DMA contexts/video nodes in TI CSI2RX
PATCH 8-9: Use get_frame_desc to propagate virtual channel information
across Cadence and TI CSI-RX subdevs
PATCH 10-12: Use new multi-stream APIs across the drivers to support
multiplexed cameras from sources like UB960 (FPDLink)
PATCH 13: Optimize stream on by submitting all queued buffers to DMA

This applies on top of today's linux-next (next-20240626)

This series is based on top of next-20240626

Quoted 56 lines
Quoted 45 lines

(also tested rebase with media_stage.git master)

---
Changes in v2:

- Change the multi-camera capture architecture to be similar to that of Tomi's RPi5 FE series, where the driver will wait for userspace to start streaming on all "actively routed" video nodes before starting streaming on the source. This simplifies things a lot from the HW perspective, which might run into deadlocks due to a shared FIFO between multiple DMA channels.

- Drop a few fixes that were posted separately and are already merged
- Fix dtschema warnings reported by Rob on [02/13]
- Fix warnings for uninitialized used_vc variable in cdns-csi2rx.c
- Return -EBUSY if someone updates routes for j721e-csi2rx subdev while streaming
- Only allow single-streams to be routed to the source pads (linked to video nodes) of the j721e-csi2rx device
- Squash the patches marked "SQUASH" in the v1 RFC series

- Link to RFC (v1):

    https://lore.kernel.org/r/[email protected]

---
Jai Luthra (8):
dt-bindings: media: ti,j721e-csi2rx-shim: Support 32 dma chans
media: ti: j721e-csi2rx: separate out device and context
media: ti: j721e-csi2rx: add a subdev for the core device
media: ti: j721e-csi2rx: add support for processing virtual channels
media: cadence: csi2rx: Use new enable stream APIs
media: cadence: csi2rx: Enable multi-stream support
media: ti: j721e-csi2rx: add multistream support
media: ti: j721e-csi2rx: Submit all available buffers

Jayshri Pawar (1):
media: cadence: csi2rx: Support runtime PM

Pratyush Yadav (4):
media: ti: j721e-csi2rx: prepare SHIM code for multiple contexts
media: ti: j721e-csi2rx: allocate DMA channel based on context index
media: ti: j721e-csi2rx: get number of contexts from device tree
media: cadence: csi2rx: add get_frame_desc wrapper

Quoted 7 lines
   .../bindings/media/ti,j721e-csi2rx-shim.yaml       |  39 +-
   drivers/media/platform/cadence/cdns-csi2rx.c       | 440 +++++++++--
   .../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c  | 879 ++++++++++++++++-----
   3 files changed, 1071 insertions(+), 287 deletions(-)

---
base-commit: df9574a57d02b265322e77fb8628d4d33641dda9
change-id: 20240221-multistream-fbba6ffe47a3

You have based this series on top of your private branch. Don't do that. Base on top of a kernel tag, or a commonly known tree (linux-media-stage for example), and preferably mention the base in the cover letter.

Quoted 7 lines
Quoted 2 lines

The base commit SHA populated by b4 is the same as next-20240626 as mentioned above

Ah, right, my bad. I took your branch
https://github.com/jailuthra/linux/commits/b4/multistream/, and assumed it's the one you used to send these patches. In that branch these patches are not based on linux-next.

Ah, yes I cherry-picked them in after posting.

Quoted 9 lines
Quoted 5 lines

https://gitlab.com/linux-kernel/linux-next/-/commits/df9574a57d02b265322e77fb8628d4d33641dda9

I chose to not use media-stage as the base, but this series applies cleanly (and compiles) on top of that as well.

I'd still recommend media-stage, as that's where these patches would be merged (or just linux-media). linux-next is good for testing, but I wouldn't normally base patches on top of that, or at last send patches based on that.

Understood, will use media-stage while posting future revisions.

Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

Your private branch contains e.g. dtsos needed for testing. If you have such a branch, you should point to it in the cover letter as it's valuable for reviewers/testers.

Quoted 8 lines
Quoted 5 lines

Ah my bad, I missed mentioning my github branch that can be used for testing the content of this series. It contains some DTSOs and defconfig updates, along with support for FPDLink/V3Link sensors.

https://github.com/jailuthra/linux/commits/b4/multistream/

Jfyi, I've tested this with am62a and arducam's fpdlink board with imx219 sensors, and works fine for me.

Neat! Thanks for testing it out.

Quoted 4 lines

I only tested with pixel streams, I'd like to also add all the patches needed for embedded data and test that (I think all of those have been posted to the lists), but I don't think I'll have time for that right now.

I see, I haven't been following the recent series adding support for internal pads and embedded data in IMX219. I will try to merge those in and see if I can get something working.

Quoted 15 lines

Tomi

Quoted 11 lines
Quoted 3 lines

Only base on top of a private branch if your patches compile-time depend on something from there, and in that case point to the branch and mention this dependency clearly in the cover letter.

Makes sense, will take extra care to mention the dependencies and base branch from next version.

Quoted 3 lines

Tomi

Signature · 4 lines
Thanks,
Jai

GPG Fingerprint: 4DE0 D818 E5D5 75E8 D45A AFC5 43DE 91F9 249A 7145