Skip to contentSkip to conversation

Streaming control for MC with metadata or streams otherwise

PATCH In review Sakari Ailus 7 participants 67 messages Jun 19, 2025 – Aug 4, 2025

Subsystems (2)
lore
Share & tools
lore

Sakari Ailus

author maintainer
#0 lore raw

cover Streaming control for MC with metadata or streams otherwise

Hi all,

Starting and stopping streaming is a non-trivial problem in all but most simple cases where there's only one stream. But when multiple streams come into play, the problem quickly gets difficult to solve properly as it generally spans multiple hardware deveices, not only including transmitters (e.g. camera sensors) and receivers but also devices in between, such as CSI-2 aggregators and serdes devices.

The following document summarises the known use cases:
<URL:https://pilvi.retiisi.eu/s/44RWYjKG92R6X49>, please let me know if you have something to add that's not included here.

What this patchset implements is a partial solution, but it's also a solution that can be extended to cover the above cases with relatively small amendments.

The current functionality is best documented in patch "media: v4l2-mc:
Introduce v4l2_mc_pipeline_enabled()".

This set, in some form, is also effectively required by metadata support patchset of which I'm about to post a new version. Starting per VC (for CSI-2) is next on my to-do list when it comes to this set.

Getting these both in for 6.18 would be great.

Comments are very welcome.

Sakari Ailus (13):
media: ipu6: Use correct pads for xlate_streams()
media: ipu6: Set minimum height to 1
media: ipu6: Enable and disable each stream at CSI-2 subdev source pad
media: v4l2-subdev: Add a helper to figure out the pad streaming state
media: v4l: Make media_entity_to_video_device() NULL-safe
media: v4l2-subdev: Mark both streams of a route enabled
media: ipu6: Set up CSI-2 receiver at correct moment
media: v4l2-subdev: Print early in

    v4l2_subdev_{enable,disable}_streams()

media: v4l2-subdev: Collect streams on source pads only
media: v4l2-subdev: Add debug prints to v4l2_subdev_collect_streams()
media: v4l2-subdev: Introduce v4l2_subdev_find_route()
media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()
media: ipu6: isys: Rework stream starting and stopping

 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c |  63 +++--
 .../media/pci/intel/ipu6/ipu6-isys-queue.c    |  48 ++--
 .../media/pci/intel/ipu6/ipu6-isys-video.c    |  96 ++++---
 .../media/pci/intel/ipu6/ipu6-isys-video.h    |   9 +-
 drivers/media/pci/intel/ipu6/ipu6-isys.h      |   2 +-
 drivers/media/v4l2-core/v4l2-mc.c             | 243 ++++++++++++++++++
 drivers/media/v4l2-core/v4l2-subdev.c         | 107 +++++---
 include/media/v4l2-dev.h                      |  14 +-
 include/media/v4l2-mc.h                       |  44 ++++
 include/media/v4l2-subdev.h                   |  22 ++
 10 files changed, 539 insertions(+), 109 deletions(-)
Signature · 1 line
2.39.5
author maintainer
#1 lore raw

PATCH 1/13 media: ipu6: Use correct pads for xlate_streams()

The arguments to v4l2_subdev_state_xlate_streams() were incorrect, the source pads was used as the sink pad and the source pad was a constant (rather than the actual source pad). Fix these.

Fixes 3a5c59ad926b ("media: ipu6: Rework CSI-2 sub-device streaming control")
---
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +6 -6
index da8581a37e22..6030bd23b4b9 100644
@@ -354,9 +354,9 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
 	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
 	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
 
-	sink_streams = v4l2_subdev_state_xlate_streams(state, CSI2_PAD_SRC,
-						       CSI2_PAD_SINK,
-						       &streams_mask);
+	sink_streams =
+		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
+						&streams_mask);
 
 	ret = ipu6_isys_csi2_calc_timing(csi2, &timing, CSI2_ACCINV);
 	if (ret)
@@ -384,9 +384,9 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,
 	struct media_pad *remote_pad;
 	u64 sink_streams;
 
-	sink_streams = v4l2_subdev_state_xlate_streams(state, CSI2_PAD_SRC,
-						       CSI2_PAD_SINK,
-						       &streams_mask);
+	sink_streams =
+		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
+						&streams_mask);
 
 	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
 	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
Signature · 1 line
2.39.5
#2 lore raw

Re: [PATCH 01/13] media: ipu6: Use correct pads for xlate_streams()

Hi Sakari,

Thank you for the patch.

Quoted 1 line · Sakari Ailus

The arguments to v4l2_subdev_state_xlate_streams() were incorrect, the

s/were/are/

Quoted 1 line

source pads was used as the sink pad and the source pad was a constant

s/pads was/pad is/
s/pad was/pad is/

Are you sure though ? Unless I misread the code, you replace

pad0 = CSI2_PAD_SRC
pad1 = CSI2_PAD_SINK

with

pad0 = pad
pad1 = CSI2_PAD_SINK

This seems to be a correct fix, but I don't see where "the source pad was used as the sink pad".

Quoted 14 lines

(rather than the actual source pad). Fix these.

Fixes 3a5c59ad926b ("media: ipu6: Rework CSI-2 sub-device streaming control")
---
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0 -0
index da8581a37e22..6030bd23b4b9 100644

drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd, · line 359

Quoted 9 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +3 -3
@@ -354,9 +354,9 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
 	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
 	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
 
-	sink_streams = v4l2_subdev_state_xlate_streams(state, CSI2_PAD_SRC,
-						       CSI2_PAD_SINK,
-						       &streams_mask);
+	sink_streams =
+		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
+						&streams_mask);

This is one of the cases where I'd write

	sink_streams = v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
						       &streams_mask);

even if it goes to 81 columns.

drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd, · line 389

Quoted 13 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +3 -3
 	ret = ipu6_isys_csi2_calc_timing(csi2, &timing, CSI2_ACCINV);
 	if (ret)
@@ -384,9 +384,9 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,
 	struct media_pad *remote_pad;
 	u64 sink_streams;
 
-	sink_streams = v4l2_subdev_state_xlate_streams(state, CSI2_PAD_SRC,
-						       CSI2_PAD_SINK,
-						       &streams_mask);
+	sink_streams =
+		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
+						&streams_mask);

Same here.

Quoted 3 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0 -0
 	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
 	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#3 lore raw

Re: [PATCH 01/13] media: ipu6: Use correct pads for xlate_streams()

Hi Laurent,

Quoted 3 lines · Laurent Pinchart

Hi Sakari,

Thank you for the patch.

Thanks for the review!

Quoted 16 lines · Sakari Ailus
Quoted 1 line

The arguments to v4l2_subdev_state_xlate_streams() were incorrect, the

s/were/are/

Quoted 1 line

source pads was used as the sink pad and the source pad was a constant

s/pads was/pad is/
s/pad was/pad is/

Are you sure though ? Unless I misread the code, you replace

pad0 = CSI2_PAD_SRC
pad1 = CSI2_PAD_SINK

Quoted 7 lines

with

pad0 = pad
pad1 = CSI2_PAD_SINK

This seems to be a correct fix, but I don't see where "the source pad was used as the sink pad".

Right, I'll reword it for v2.

Quoted 27 lines
Quoted 15 lines

(rather than the actual source pad). Fix these.

Fixes 3a5c59ad926b ("media: ipu6: Rework CSI-2 sub-device streaming control")
---
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0 -0
index da8581a37e22..6030bd23b4b9 100644

drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd, · line 359

Quoted 9 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +3 -3
@@ -354,9 +354,9 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
 	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
 	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
 
-	sink_streams = v4l2_subdev_state_xlate_streams(state, CSI2_PAD_SRC,
-						       CSI2_PAD_SINK,
-						       &streams_mask);
+	sink_streams =
+		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
+						&streams_mask);

This is one of the cases where I'd write

Quoted 4 lines
	sink_streams = v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
						       &streams_mask);

even if it goes to 81 columns.

The limit is still 80, not 81.

Either way, a number of the new functions in the V4L2 subdev KAPI have very long names, I'll post patches to shorten them later on.

Quoted 21 lines
Quoted 5 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0 -0
 	ret = ipu6_isys_csi2_calc_timing(csi2, &timing, CSI2_ACCINV);
 	if (ret)

drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd, · line 389

Quoted 9 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +3 -3
@@ -384,9 +384,9 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,
 	struct media_pad *remote_pad;
 	u64 sink_streams;
 
-	sink_streams = v4l2_subdev_state_xlate_streams(state, CSI2_PAD_SRC,
-						       CSI2_PAD_SINK,
-						       &streams_mask);
+	sink_streams =
+		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
+						&streams_mask);

Same here.

Quoted 3 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0 -0
 	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
 	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
Signature · 3 lines
Regards,

Sakari Ailus
#4 lore raw

Re: [PATCH 01/13] media: ipu6: Use correct pads for xlate_streams()

Quoted 58 lines · Sakari Ailus
Quoted 3 lines · Laurent Pinchart

Hi Sakari,

Thank you for the patch.

Thanks for the review!

Quoted 15 lines · Sakari Ailus
Quoted 1 line

The arguments to v4l2_subdev_state_xlate_streams() were incorrect, the

s/were/are/

Quoted 1 line

source pads was used as the sink pad and the source pad was a constant

s/pads was/pad is/
s/pad was/pad is/

Are you sure though ? Unless I misread the code, you replace

pad0 = CSI2_PAD_SRC
pad1 = CSI2_PAD_SINK

Quoted 7 lines

with

pad0 = pad
pad1 = CSI2_PAD_SINK

This seems to be a correct fix, but I don't see where "the source pad was used as the sink pad".

Right, I'll reword it for v2.

Quoted 26 lines
Quoted 14 lines

(rather than the actual source pad). Fix these.

Fixes 3a5c59ad926b ("media: ipu6: Rework CSI-2 sub-device streaming control")
---
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0 -0
index da8581a37e22..6030bd23b4b9 100644

drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd, · line 359

Quoted 9 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +3 -3
@@ -354,9 +354,9 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
 	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
 	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
 
-	sink_streams = v4l2_subdev_state_xlate_streams(state, CSI2_PAD_SRC,
-						       CSI2_PAD_SINK,
-						       &streams_mask);
+	sink_streams =
+		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
+						&streams_mask);

This is one of the cases where I'd write

Quoted 6 lines
Quoted 4 lines
	sink_streams = v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
						       &streams_mask);

even if it goes to 81 columns.

The limit is still 80, not 81.

The global limit is 100 for the kernel. There's somehow of a consensus in the media subsystem to keep it closer to 80, with different people have different sensitivities. We occasionally go over 80, and that's usually left as a driver maintainer decision.

Quoted 2 lines

Either way, a number of the new functions in the V4L2 subdev KAPI have very long names, I'll post patches to shorten them later on.

As long as the name remains equally readable and understandable, and all functions use the same conventions, I'm fine with that.

Quoted 19 lines
Quoted 19 lines

drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd, · line 389

Quoted 13 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +3 -3
 	ret = ipu6_isys_csi2_calc_timing(csi2, &timing, CSI2_ACCINV);
 	if (ret)
@@ -384,9 +384,9 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,
 	struct media_pad *remote_pad;
 	u64 sink_streams;
 
-	sink_streams = v4l2_subdev_state_xlate_streams(state, CSI2_PAD_SRC,
-						       CSI2_PAD_SINK,
-						       &streams_mask);
+	sink_streams =
+		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
+						&streams_mask);

Same here.

Quoted 3 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0 -0
 	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
 	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#5 lore raw

Re: [PATCH 01/13] media: ipu6: Use correct pads for xlate_streams()

Hi Laurent,

Quoted 62 lines · Laurent Pinchart
Quoted 61 lines · Sakari Ailus
Quoted 3 lines · Laurent Pinchart

Hi Sakari,

Thank you for the patch.

Thanks for the review!

Quoted 15 lines · Sakari Ailus
Quoted 1 line

The arguments to v4l2_subdev_state_xlate_streams() were incorrect, the

s/were/are/

Quoted 1 line

source pads was used as the sink pad and the source pad was a constant

s/pads was/pad is/
s/pad was/pad is/

Are you sure though ? Unless I misread the code, you replace

pad0 = CSI2_PAD_SRC
pad1 = CSI2_PAD_SINK

Quoted 7 lines

with

pad0 = pad
pad1 = CSI2_PAD_SINK

This seems to be a correct fix, but I don't see where "the source pad was used as the sink pad".

Right, I'll reword it for v2.

Quoted 29 lines
Quoted 14 lines

(rather than the actual source pad). Fix these.

Fixes 3a5c59ad926b ("media: ipu6: Rework CSI-2 sub-device streaming control")
---
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0 -0
index da8581a37e22..6030bd23b4b9 100644

drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd, · line 359

Quoted 9 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +3 -3
@@ -354,9 +354,9 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
 	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
 	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
 
-	sink_streams = v4l2_subdev_state_xlate_streams(state, CSI2_PAD_SRC,
-						       CSI2_PAD_SINK,
-						       &streams_mask);
+	sink_streams =
+		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
+						&streams_mask);

This is one of the cases where I'd write

	sink_streams = v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
						       &streams_mask);
Quoted 8 lines
Quoted 3 lines
Quoted 1 line

even if it goes to 81 columns.

The limit is still 80, not 81.

The global limit is 100 for the kernel. There's somehow of a consensus in the media subsystem to keep it closer to 80, with different people have different sensitivities. We occasionally go over 80, and that's usually left as a driver maintainer decision.

It's 80, not 100. The checkpatch.pl limit is higher than 80 though, as there are still commonly cases where the code is more readable with longer lines. See e.g. Documentation/process/coding-style.rst .

Signature · 3 lines
Regards,

Sakari Ailus
#6 lore raw

Re: [PATCH 01/13] media: ipu6: Use correct pads for xlate_streams()

Quoted 61 lines · Sakari Ailus
Quoted 60 lines · Laurent Pinchart
Quoted 58 lines · Sakari Ailus
Quoted 15 lines · Laurent Pinchart
Quoted 1 line · Sakari Ailus

The arguments to v4l2_subdev_state_xlate_streams() were incorrect, the

s/were/are/

Quoted 1 line

source pads was used as the sink pad and the source pad was a constant

s/pads was/pad is/
s/pad was/pad is/

Are you sure though ? Unless I misread the code, you replace

pad0 = CSI2_PAD_SRC
pad1 = CSI2_PAD_SINK

Quoted 7 lines

with

pad0 = pad
pad1 = CSI2_PAD_SINK

This seems to be a correct fix, but I don't see where "the source pad was used as the sink pad".

Right, I'll reword it for v2.

Quoted 26 lines
Quoted 14 lines

(rather than the actual source pad). Fix these.

Fixes 3a5c59ad926b ("media: ipu6: Rework CSI-2 sub-device streaming control")
---
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0 -0
index da8581a37e22..6030bd23b4b9 100644

drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd, · line 359

Quoted 9 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +3 -3
@@ -354,9 +354,9 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
 	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
 	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
 
-	sink_streams = v4l2_subdev_state_xlate_streams(state, CSI2_PAD_SRC,
-						       CSI2_PAD_SINK,
-						       &streams_mask);
+	sink_streams =
+		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
+						&streams_mask);

This is one of the cases where I'd write

Quoted 4 lines
	sink_streams = v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
						       &streams_mask);

even if it goes to 81 columns.

The limit is still 80, not 81.

Quoted 8 lines
Quoted 4 lines

The global limit is 100 for the kernel. There's somehow of a consensus in the media subsystem to keep it closer to 80, with different people have different sensitivities. We occasionally go over 80, and that's usually left as a driver maintainer decision.

It's 80, not 100. The checkpatch.pl limit is higher than 80 though, as there are still commonly cases where the code is more readable with longer lines. See e.g. Documentation/process/coding-style.rst .

It states

----
The preferred limit on the length of a single line is 80 columns.

Statements longer than 80 columns should be broken into sensible chunks, unless exceeding 80 columns significantly increases readability and does not hide information.
----

In this specific case, going over 80 columns improves readability in my opinion, and doesn't hide information. As I wrote, we treat this as a driver maintainer preference at the moment, so I won't make a call for the ipu6 driver.

Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#7 lore raw

PATCH 2/13 media: ipu6: Set minimum height to 1

In reply to Sakari Ailus

For image data generally 2 seems like a minimum height that surely won't cause any issues, but some sensors have metadata the height of which is just one line. Set the minimum height to 1.

---
 drivers/media/pci/intel/ipu6/ipu6-isys.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/media/pci/intel/ipu6/ipu6-isys.h +1 -1
index f488e782c26e..0e2c8b71edfc 100644
@@ -40,7 +40,7 @@ struct ipu6_bus_device;
 #define IPU6_ISYS_NUM_RECV_QUEUE 1
 
 #define IPU6_ISYS_MIN_WIDTH		2U
-#define IPU6_ISYS_MIN_HEIGHT		2U
+#define IPU6_ISYS_MIN_HEIGHT		1U
 #define IPU6_ISYS_MAX_WIDTH		4672U
 #define IPU6_ISYS_MAX_HEIGHT		3416U
 
Signature · 1 line
2.39.5
Reviewed-by on v1 2/13
#8 lore raw

Re: [PATCH 02/13] media: ipu6: Set minimum height to 1

Hi Sakari,

Thank you for the patch.

Quoted 5 lines · Sakari Ailus

For image data generally 2 seems like a minimum height that surely won't cause any issues, but some sensors have metadata the height of which is just one line. Set the minimum height to 1.

Quoted 17 lines
---
 drivers/media/pci/intel/ipu6/ipu6-isys.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/media/pci/intel/ipu6/ipu6-isys.h +1 -1
index f488e782c26e..0e2c8b71edfc 100644
@@ -40,7 +40,7 @@ struct ipu6_bus_device;
 #define IPU6_ISYS_NUM_RECV_QUEUE 1
 
 #define IPU6_ISYS_MIN_WIDTH		2U
-#define IPU6_ISYS_MIN_HEIGHT		2U
+#define IPU6_ISYS_MIN_HEIGHT		1U
 #define IPU6_ISYS_MAX_WIDTH		4672U
 #define IPU6_ISYS_MAX_HEIGHT		3416U
 
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#9 lore raw

PATCH 3/13 media: ipu6: Enable and disable each stream at CSI-2 subdev source pad

In reply to Sakari Ailus

On UAPI, streaming is started a video device at a time. The IPU6 ISYS driver only starts streaming on the source sub-device when all relevant video devices have been set streaming. This also needs to be reflected in the sub-device pads, hence set them all streaming, one at a time.

---
 .../media/pci/intel/ipu6/ipu6-isys-video.c    | 43 ++++++++++++-------
 1 file changed, 28 insertions(+), 15 deletions(-)
drivers/media/pci/intel/ipu6/ipu6-isys-video.c +28 -15
index 24a2ef93474c..54006b5e2ccd 100644
@@ -990,6 +990,7 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 	struct device *dev = &av->isys->adev->auxdev.dev;
 	struct v4l2_subdev *sd;
 	struct media_pad *r_pad;
+	unsigned int i;
 	u32 sink_pad, sink_stream;
 	u64 r_stream;
 	u64 stream_mask = 0;
@@ -1018,14 +1019,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 		stop_streaming_firmware(av);
 
 		/* stop sub-device which connects with video */
-		dev_dbg(dev, "stream off entity %s pad:%d mask:0x%llx\n",
-			sd->name, r_pad->index, stream_mask);
-		ret = v4l2_subdev_disable_streams(sd, r_pad->index,
-						  stream_mask);
-		if (ret) {
-			dev_err(dev, "stream off %s failed with %d\n", sd->name,
-				ret);
-			return ret;
+		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
+			if (!media_pad_pipeline(&sd->entity.pads[i]))
+				continue;
+			ret = v4l2_subdev_disable_streams(sd, i, 1U);
+			if (ret) {
+				dev_err(dev, "stream off %s failed with %d\n",
+					sd->name, ret);
+				return ret;
+			}
 		}
 		close_streaming_firmware(av);
 	} else {
@@ -1036,13 +1038,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 		}
 
 		/* start sub-device which connects with video */
-		dev_dbg(dev, "stream on %s pad %d mask 0x%llx\n", sd->name,
-			r_pad->index, stream_mask);
-		ret = v4l2_subdev_enable_streams(sd, r_pad->index, stream_mask);
-		if (ret) {
-			dev_err(dev, "stream on %s failed with %d\n", sd->name,
-				ret);
-			goto out_media_entity_stop_streaming_firmware;
+		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
+			if (!media_pad_pipeline(&sd->entity.pads[i]))
+				continue;
+			ret = v4l2_subdev_enable_streams(sd, i, 1U);
+			if (ret) {
+				dev_err(dev, "stream on %s failed with %d\n",
+					sd->name, ret);
+				goto out_media_entity_stop_streaming_firmware;
+			}
 		}
 	}
 
@@ -1051,6 +1055,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 	return 0;
 
 out_media_entity_stop_streaming_firmware:
+	while (i-- > CSI2_PAD_SRC) {
+		int ret2;
+
+		if (!media_pad_pipeline(&sd->entity.pads[i]))
+			continue;
+		ret2 = v4l2_subdev_disable_streams(sd, i, 1U);
+		dev_err(dev, "stream off %s failed with %d\n", sd->name, ret2);
+	}
+
 	stop_streaming_firmware(av);
 
 	return ret;
Signature · 1 line
2.39.5
#10 lore raw

Re: [PATCH 03/13] media: ipu6: Enable and disable each stream at CSI-2 subdev source pad

kernel test robot noticed the following build warnings:

Hi Sakari,

kernel test robot noticed the following build warnings:

[auto build test WARNING on linuxtv-media-pending/master] [also build test WARNING on linus/master v6.16-rc2 next-20250618] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Sakari-Ailus/media-ipu6-Use-correct-pads-for-xlate_streams/20250619-161847
base:   https://git.linuxtv.org/media-ci/media-pending.git master
patch link:    https://lore.kernel.org/r/20250619081546.1582969-4-sakari.ailus%40linux.intel.com
patch subject: [PATCH 03/13] media: ipu6: Enable and disable each stream at CSI-2 subdev source pad
config: i386-buildonly-randconfig-005-20250619 (https://download.01.org/0day-ci/archive/20250619/[email protected]/config)
compiler: clang version 20.1.2 (https://github.com/llvm/llvm-project 58df0ef89dd64126512e4ee27b4ac3fd8ddf6247)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250619/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All warnings (new ones prefixed by >>):

>> drivers/media/pci/intel/ipu6/ipu6-isys-video.c:996:6: warning: variable 'stream_mask' set but not used [-Wunused-but-set-variable]
     996 |         u64 stream_mask = 0;
         |             ^

1 warning generated.

vim +/stream_mask +996 drivers/media/pci/intel/ipu6/ipu6-isys-video.c

3c1dfb5a69cf83 Bingbu Cao   2024-01-31   983  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   984  int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   985  				  struct ipu6_isys_buffer_list *bl)
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   986  {
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   987  	struct v4l2_subdev_krouting *routing;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   988  	struct ipu6_isys_stream *stream = av->stream;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   989  	struct v4l2_subdev_state *subdev_state;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   990  	struct device *dev = &av->isys->adev->auxdev.dev;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   991  	struct v4l2_subdev *sd;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   992  	struct media_pad *r_pad;
e232e72fbaa788 Sakari Ailus 2025-06-19   993  	unsigned int i;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   994  	u32 sink_pad, sink_stream;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   995  	u64 r_stream;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  @996  	u64 stream_mask = 0;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   997  	int ret = 0;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   998  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   999  	dev_dbg(dev, "set stream: %d\n", state);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1000  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1001  	if (WARN(!stream->source_entity, "No source entity for stream\n"))
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1002  		return -ENODEV;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1003  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1004  	sd = &stream->asd->sd;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1005  	r_pad = media_pad_remote_pad_first(&av->pad);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1006  	r_stream = ipu6_isys_get_src_stream_by_src_pad(sd, r_pad->index);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1007  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1008  	subdev_state = v4l2_subdev_lock_and_get_active_state(sd);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1009  	routing = &subdev_state->routing;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1010  	ret = v4l2_subdev_routing_find_opposite_end(routing, r_pad->index,
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1011  						    r_stream, &sink_pad,
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1012  						    &sink_stream);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1013  	v4l2_subdev_unlock_state(subdev_state);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1014  	if (ret)
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1015  		return ret;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1016  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1017  	stream_mask = get_stream_mask_by_pipeline(av);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1018  	if (!state) {
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1019  		stop_streaming_firmware(av);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1020  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1021  		/* stop sub-device which connects with video */
e232e72fbaa788 Sakari Ailus 2025-06-19  1022  		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
e232e72fbaa788 Sakari Ailus 2025-06-19  1023  			if (!media_pad_pipeline(&sd->entity.pads[i]))
e232e72fbaa788 Sakari Ailus 2025-06-19  1024  				continue;
e232e72fbaa788 Sakari Ailus 2025-06-19  1025  			ret = v4l2_subdev_disable_streams(sd, i, 1U);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1026  			if (ret) {
e232e72fbaa788 Sakari Ailus 2025-06-19  1027  				dev_err(dev, "stream off %s failed with %d\n",
e232e72fbaa788 Sakari Ailus 2025-06-19  1028  					sd->name, ret);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1029  				return ret;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1030  			}
e232e72fbaa788 Sakari Ailus 2025-06-19  1031  		}
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1032  		close_streaming_firmware(av);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1033  	} else {
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1034  		ret = start_stream_firmware(av, bl);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1035  		if (ret) {
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1036  			dev_err(dev, "start stream of firmware failed\n");
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1037  			return ret;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1038  		}
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1039  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1040  		/* start sub-device which connects with video */
e232e72fbaa788 Sakari Ailus 2025-06-19  1041  		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
e232e72fbaa788 Sakari Ailus 2025-06-19  1042  			if (!media_pad_pipeline(&sd->entity.pads[i]))
e232e72fbaa788 Sakari Ailus 2025-06-19  1043  				continue;
e232e72fbaa788 Sakari Ailus 2025-06-19  1044  			ret = v4l2_subdev_enable_streams(sd, i, 1U);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1045  			if (ret) {
e232e72fbaa788 Sakari Ailus 2025-06-19  1046  				dev_err(dev, "stream on %s failed with %d\n",
e232e72fbaa788 Sakari Ailus 2025-06-19  1047  					sd->name, ret);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1048  				goto out_media_entity_stop_streaming_firmware;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1049  			}
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1050  		}
e232e72fbaa788 Sakari Ailus 2025-06-19  1051  	}
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1052  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1053  	av->streaming = state;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1054  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1055  	return 0;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1056  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1057  out_media_entity_stop_streaming_firmware:
e232e72fbaa788 Sakari Ailus 2025-06-19  1058  	while (i-- > CSI2_PAD_SRC) {
e232e72fbaa788 Sakari Ailus 2025-06-19  1059  		int ret2;
e232e72fbaa788 Sakari Ailus 2025-06-19  1060  
e232e72fbaa788 Sakari Ailus 2025-06-19  1061  		if (!media_pad_pipeline(&sd->entity.pads[i]))
e232e72fbaa788 Sakari Ailus 2025-06-19  1062  			continue;
e232e72fbaa788 Sakari Ailus 2025-06-19  1063  		ret2 = v4l2_subdev_disable_streams(sd, i, 1U);
e232e72fbaa788 Sakari Ailus 2025-06-19  1064  		dev_err(dev, "stream off %s failed with %d\n", sd->name, ret2);
e232e72fbaa788 Sakari Ailus 2025-06-19  1065  	}
e232e72fbaa788 Sakari Ailus 2025-06-19  1066  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1067  	stop_streaming_firmware(av);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1068  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1069  	return ret;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1070  }
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1071  
Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#11 lore raw

Re: [PATCH 03/13] media: ipu6: Enable and disable each stream at CSI-2 subdev source pad

In reply to Sakari Ailus

Hi Sakari,

Thank you for the patch.

Quoted 4 lines · Sakari Ailus

On UAPI, streaming is started a video device at a time. The IPU6 ISYS driver only starts streaming on the source sub-device when all relevant video devices have been set streaming. This also needs to be reflected in the sub-device pads, hence set them all streaming, one at a time.

Why ? What's wrong with enabling/disabling streams on each pad as the corresponding video device is started/stopped ?

Quoted 80 lines
---
 .../media/pci/intel/ipu6/ipu6-isys-video.c    | 43 ++++++++++++-------
 1 file changed, 28 insertions(+), 15 deletions(-)
drivers/media/pci/intel/ipu6/ipu6-isys-video.c +28 -15
index 24a2ef93474c..54006b5e2ccd 100644
@@ -990,6 +990,7 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 	struct device *dev = &av->isys->adev->auxdev.dev;
 	struct v4l2_subdev *sd;
 	struct media_pad *r_pad;
+	unsigned int i;
 	u32 sink_pad, sink_stream;
 	u64 r_stream;
 	u64 stream_mask = 0;
@@ -1018,14 +1019,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 		stop_streaming_firmware(av);
 
 		/* stop sub-device which connects with video */
-		dev_dbg(dev, "stream off entity %s pad:%d mask:0x%llx\n",
-			sd->name, r_pad->index, stream_mask);
-		ret = v4l2_subdev_disable_streams(sd, r_pad->index,
-						  stream_mask);
-		if (ret) {
-			dev_err(dev, "stream off %s failed with %d\n", sd->name,
-				ret);
-			return ret;
+		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
+			if (!media_pad_pipeline(&sd->entity.pads[i]))
+				continue;
+			ret = v4l2_subdev_disable_streams(sd, i, 1U);
+			if (ret) {
+				dev_err(dev, "stream off %s failed with %d\n",
+					sd->name, ret);
+				return ret;
+			}
 		}
 		close_streaming_firmware(av);
 	} else {
@@ -1036,13 +1038,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 		}
 
 		/* start sub-device which connects with video */
-		dev_dbg(dev, "stream on %s pad %d mask 0x%llx\n", sd->name,
-			r_pad->index, stream_mask);
-		ret = v4l2_subdev_enable_streams(sd, r_pad->index, stream_mask);
-		if (ret) {
-			dev_err(dev, "stream on %s failed with %d\n", sd->name,
-				ret);
-			goto out_media_entity_stop_streaming_firmware;
+		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
+			if (!media_pad_pipeline(&sd->entity.pads[i]))
+				continue;
+			ret = v4l2_subdev_enable_streams(sd, i, 1U);
+			if (ret) {
+				dev_err(dev, "stream on %s failed with %d\n",
+					sd->name, ret);
+				goto out_media_entity_stop_streaming_firmware;
+			}
 		}
 	}
 
@@ -1051,6 +1055,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 	return 0;
 
 out_media_entity_stop_streaming_firmware:
+	while (i-- > CSI2_PAD_SRC) {
+		int ret2;
+
+		if (!media_pad_pipeline(&sd->entity.pads[i]))
+			continue;
+		ret2 = v4l2_subdev_disable_streams(sd, i, 1U);
+		dev_err(dev, "stream off %s failed with %d\n", sd->name, ret2);
+	}
+
 	stop_streaming_firmware(av);
 
 	return ret;
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#12 lore raw

Re: [PATCH 03/13] media: ipu6: Enable and disable each stream at CSI-2 subdev source pad

Hi Laurent,

Quoted 4 lines · Laurent Pinchart

Hi Sakari,

Thank you for the patch.

Quoted 8 lines
Quoted 4 lines · Sakari Ailus

On UAPI, streaming is started a video device at a time. The IPU6 ISYS driver only starts streaming on the source sub-device when all relevant video devices have been set streaming. This also needs to be reflected in the sub-device pads, hence set them all streaming, one at a time.

Why ? What's wrong with enabling/disabling streams on each pad as the corresponding video device is started/stopped ?

Streaming is started once all the related video nodes are set streaming, otherwise there's a chance of losing data.

I've squashed this with the last patch after posting the set.

Quoted 82 lines
Quoted 81 lines
---
 .../media/pci/intel/ipu6/ipu6-isys-video.c    | 43 ++++++++++++-------
 1 file changed, 28 insertions(+), 15 deletions(-)
drivers/media/pci/intel/ipu6/ipu6-isys-video.c +28 -15
index 24a2ef93474c..54006b5e2ccd 100644
@@ -990,6 +990,7 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 	struct device *dev = &av->isys->adev->auxdev.dev;
 	struct v4l2_subdev *sd;
 	struct media_pad *r_pad;
+	unsigned int i;
 	u32 sink_pad, sink_stream;
 	u64 r_stream;
 	u64 stream_mask = 0;
@@ -1018,14 +1019,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 		stop_streaming_firmware(av);
 
 		/* stop sub-device which connects with video */
-		dev_dbg(dev, "stream off entity %s pad:%d mask:0x%llx\n",
-			sd->name, r_pad->index, stream_mask);
-		ret = v4l2_subdev_disable_streams(sd, r_pad->index,
-						  stream_mask);
-		if (ret) {
-			dev_err(dev, "stream off %s failed with %d\n", sd->name,
-				ret);
-			return ret;
+		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
+			if (!media_pad_pipeline(&sd->entity.pads[i]))
+				continue;
+			ret = v4l2_subdev_disable_streams(sd, i, 1U);
+			if (ret) {
+				dev_err(dev, "stream off %s failed with %d\n",
+					sd->name, ret);
+				return ret;
+			}
 		}
 		close_streaming_firmware(av);
 	} else {
@@ -1036,13 +1038,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 		}
 
 		/* start sub-device which connects with video */
-		dev_dbg(dev, "stream on %s pad %d mask 0x%llx\n", sd->name,
-			r_pad->index, stream_mask);
-		ret = v4l2_subdev_enable_streams(sd, r_pad->index, stream_mask);
-		if (ret) {
-			dev_err(dev, "stream on %s failed with %d\n", sd->name,
-				ret);
-			goto out_media_entity_stop_streaming_firmware;
+		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
+			if (!media_pad_pipeline(&sd->entity.pads[i]))
+				continue;
+			ret = v4l2_subdev_enable_streams(sd, i, 1U);
+			if (ret) {
+				dev_err(dev, "stream on %s failed with %d\n",
+					sd->name, ret);
+				goto out_media_entity_stop_streaming_firmware;
+			}
 		}
 	}
 
@@ -1051,6 +1055,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 	return 0;
 
 out_media_entity_stop_streaming_firmware:
+	while (i-- > CSI2_PAD_SRC) {
+		int ret2;
+
+		if (!media_pad_pipeline(&sd->entity.pads[i]))
+			continue;
+		ret2 = v4l2_subdev_disable_streams(sd, i, 1U);
+		dev_err(dev, "stream off %s failed with %d\n", sd->name, ret2);
+	}
+
 	stop_streaming_firmware(av);
 
 	return ret;
Signature · 3 lines
Regards,

Sakari Ailus
#13 lore raw

Re: [PATCH 03/13] media: ipu6: Enable and disable each stream at CSI-2 subdev source pad

Quoted 7 lines · Sakari Ailus
Quoted 6 lines · Laurent Pinchart
Quoted 4 lines · Sakari Ailus

On UAPI, streaming is started a video device at a time. The IPU6 ISYS driver only starts streaming on the source sub-device when all relevant video devices have been set streaming. This also needs to be reflected in the sub-device pads, hence set them all streaming, one at a time.

Quoted 5 lines
Quoted 2 lines

Why ? What's wrong with enabling/disabling streams on each pad as the corresponding video device is started/stopped ?

Streaming is started once all the related video nodes are set streaming, otherwise there's a chance of losing data.

Yes, but that's something that can be handled in the subdev, to only start the ISYS and its source when all the streams that are part of the pipeline have been started.

Quoted 1 line

I've squashed this with the last patch after posting the set.

Maybe it will become clearer when I review the rest of the series.

Quoted 80 lines
Quoted 80 lines
Quoted 80 lines
---
 .../media/pci/intel/ipu6/ipu6-isys-video.c    | 43 ++++++++++++-------
 1 file changed, 28 insertions(+), 15 deletions(-)
drivers/media/pci/intel/ipu6/ipu6-isys-video.c +28 -15
index 24a2ef93474c..54006b5e2ccd 100644
@@ -990,6 +990,7 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 	struct device *dev = &av->isys->adev->auxdev.dev;
 	struct v4l2_subdev *sd;
 	struct media_pad *r_pad;
+	unsigned int i;
 	u32 sink_pad, sink_stream;
 	u64 r_stream;
 	u64 stream_mask = 0;
@@ -1018,14 +1019,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 		stop_streaming_firmware(av);
 
 		/* stop sub-device which connects with video */
-		dev_dbg(dev, "stream off entity %s pad:%d mask:0x%llx\n",
-			sd->name, r_pad->index, stream_mask);
-		ret = v4l2_subdev_disable_streams(sd, r_pad->index,
-						  stream_mask);
-		if (ret) {
-			dev_err(dev, "stream off %s failed with %d\n", sd->name,
-				ret);
-			return ret;
+		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
+			if (!media_pad_pipeline(&sd->entity.pads[i]))
+				continue;
+			ret = v4l2_subdev_disable_streams(sd, i, 1U);
+			if (ret) {
+				dev_err(dev, "stream off %s failed with %d\n",
+					sd->name, ret);
+				return ret;
+			}
 		}
 		close_streaming_firmware(av);
 	} else {
@@ -1036,13 +1038,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 		}
 
 		/* start sub-device which connects with video */
-		dev_dbg(dev, "stream on %s pad %d mask 0x%llx\n", sd->name,
-			r_pad->index, stream_mask);
-		ret = v4l2_subdev_enable_streams(sd, r_pad->index, stream_mask);
-		if (ret) {
-			dev_err(dev, "stream on %s failed with %d\n", sd->name,
-				ret);
-			goto out_media_entity_stop_streaming_firmware;
+		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
+			if (!media_pad_pipeline(&sd->entity.pads[i]))
+				continue;
+			ret = v4l2_subdev_enable_streams(sd, i, 1U);
+			if (ret) {
+				dev_err(dev, "stream on %s failed with %d\n",
+					sd->name, ret);
+				goto out_media_entity_stop_streaming_firmware;
+			}
 		}
 	}
 
@@ -1051,6 +1055,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 	return 0;
 
 out_media_entity_stop_streaming_firmware:
+	while (i-- > CSI2_PAD_SRC) {
+		int ret2;
+
+		if (!media_pad_pipeline(&sd->entity.pads[i]))
+			continue;
+		ret2 = v4l2_subdev_disable_streams(sd, i, 1U);
+		dev_err(dev, "stream off %s failed with %d\n", sd->name, ret2);
+	}
+
 	stop_streaming_firmware(av);
 
 	return ret;
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#14 lore raw

Re: [PATCH 03/13] media: ipu6: Enable and disable each stream at CSI-2 subdev source pad

Quoted 11 lines · Laurent Pinchart
Quoted 10 lines · Sakari Ailus
Quoted 8 lines · Laurent Pinchart
Quoted 4 lines · Sakari Ailus

On UAPI, streaming is started a video device at a time. The IPU6 ISYS driver only starts streaming on the source sub-device when all relevant video devices have been set streaming. This also needs to be reflected in the sub-device pads, hence set them all streaming, one at a time.

Why ? What's wrong with enabling/disabling streams on each pad as the corresponding video device is started/stopped ?

Quoted 6 lines
Quoted 2 lines

Streaming is started once all the related video nodes are set streaming, otherwise there's a chance of losing data.

Yes, but that's something that can be handled in the subdev, to only start the ISYS and its source when all the streams that are part of the pipeline have been started.

I agree, this is done in the CSI-2 receiver driver actually.

There's some work to do in cleaning up the IPU6 driver. The combination of videobuf2 and streams isn't very driver-friendly currently. Things generally need to happen in certain order and this may not be always veyr convenient for drivers.

Quoted 86 lines
Quoted 2 lines

I've squashed this with the last patch after posting the set.

Maybe it will become clearer when I review the rest of the series.

Quoted 80 lines
Quoted 80 lines
Quoted 80 lines
---
 .../media/pci/intel/ipu6/ipu6-isys-video.c    | 43 ++++++++++++-------
 1 file changed, 28 insertions(+), 15 deletions(-)
drivers/media/pci/intel/ipu6/ipu6-isys-video.c +28 -15
index 24a2ef93474c..54006b5e2ccd 100644
@@ -990,6 +990,7 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 	struct device *dev = &av->isys->adev->auxdev.dev;
 	struct v4l2_subdev *sd;
 	struct media_pad *r_pad;
+	unsigned int i;
 	u32 sink_pad, sink_stream;
 	u64 r_stream;
 	u64 stream_mask = 0;
@@ -1018,14 +1019,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 		stop_streaming_firmware(av);
 
 		/* stop sub-device which connects with video */
-		dev_dbg(dev, "stream off entity %s pad:%d mask:0x%llx\n",
-			sd->name, r_pad->index, stream_mask);
-		ret = v4l2_subdev_disable_streams(sd, r_pad->index,
-						  stream_mask);
-		if (ret) {
-			dev_err(dev, "stream off %s failed with %d\n", sd->name,
-				ret);
-			return ret;
+		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
+			if (!media_pad_pipeline(&sd->entity.pads[i]))
+				continue;
+			ret = v4l2_subdev_disable_streams(sd, i, 1U);
+			if (ret) {
+				dev_err(dev, "stream off %s failed with %d\n",
+					sd->name, ret);
+				return ret;
+			}
 		}
 		close_streaming_firmware(av);
 	} else {
@@ -1036,13 +1038,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 		}
 
 		/* start sub-device which connects with video */
-		dev_dbg(dev, "stream on %s pad %d mask 0x%llx\n", sd->name,
-			r_pad->index, stream_mask);
-		ret = v4l2_subdev_enable_streams(sd, r_pad->index, stream_mask);
-		if (ret) {
-			dev_err(dev, "stream on %s failed with %d\n", sd->name,
-				ret);
-			goto out_media_entity_stop_streaming_firmware;
+		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
+			if (!media_pad_pipeline(&sd->entity.pads[i]))
+				continue;
+			ret = v4l2_subdev_enable_streams(sd, i, 1U);
+			if (ret) {
+				dev_err(dev, "stream on %s failed with %d\n",
+					sd->name, ret);
+				goto out_media_entity_stop_streaming_firmware;
+			}
 		}
 	}
 
@@ -1051,6 +1055,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 	return 0;
 
 out_media_entity_stop_streaming_firmware:
+	while (i-- > CSI2_PAD_SRC) {
+		int ret2;
+
+		if (!media_pad_pipeline(&sd->entity.pads[i]))
+			continue;
+		ret2 = v4l2_subdev_disable_streams(sd, i, 1U);
+		dev_err(dev, "stream off %s failed with %d\n", sd->name, ret2);
+	}
+
 	stop_streaming_firmware(av);
 
 	return ret;
Signature · 3 lines
Regards,

Sakari Ailus
author maintainer
#15 lore raw

PATCH 4/13 media: v4l2-subdev: Add a helper to figure out the pad streaming state

In reply to Sakari Ailus

Add a helper, v4l2_subdev_state_streams_enabled(), to tell which streams are enabled on a pad. This is useful to e.g. figure out in a driver whether a hardware configuration change is necessary to enable a given stream.

---
 drivers/media/v4l2-core/v4l2-subdev.c | 17 +++++++++++++++++
 include/media/v4l2-subdev.h           |  3 +++
 2 files changed, 20 insertions(+)
drivers/media/v4l2-core/v4l2-subdev.c +17 -0
index a3074f469b15..60b8febd3339 100644
@@ -2453,6 +2453,23 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_disable_streams);
 
+u64 v4l2_subdev_state_streams_enabled(struct v4l2_subdev *sd,
+				      struct v4l2_subdev_state *state, u32 pad)
+{
+	u64 streams_mask = 0;
+
+	for (unsigned int i = 0; i < state->stream_configs.num_configs; i++) {
+		struct v4l2_subdev_stream_config *cfg =
+			&state->stream_configs.configs[i];
+
+		if (cfg->pad == pad && cfg->enabled)
+			streams_mask |= BIT_ULL(cfg->stream);
+	}
+
+	return streams_mask;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_state_streams_enabled);
+
 int v4l2_subdev_s_stream_helper(struct v4l2_subdev *sd, int enable)
 {
 	struct v4l2_subdev_state *state;
include/media/v4l2-subdev.h +3 -0
index 57f2bcb4eb16..deab128a4779 100644
@@ -1731,6 +1731,9 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
 int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
 				u64 streams_mask);
 
+u64 v4l2_subdev_state_streams_enabled(struct v4l2_subdev *sd,
+				      struct v4l2_subdev_state *state, u32 pad);
+
 /**
  * v4l2_subdev_s_stream_helper() - Helper to implement the subdev s_stream
  *	operation using enable_streams and disable_streams
Signature · 1 line
2.39.5
#16 lore raw

Re: [PATCH 04/13] media: v4l2-subdev: Add a helper to figure out the pad streaming state

Hi Sakari,

Thank you for the patch.

Quoted 16 lines · Sakari Ailus

Add a helper, v4l2_subdev_state_streams_enabled(), to tell which streams are enabled on a pad. This is useful to e.g. figure out in a driver whether a hardware configuration change is necessary to enable a given stream.

---
 drivers/media/v4l2-core/v4l2-subdev.c | 17 +++++++++++++++++
 include/media/v4l2-subdev.h           |  3 +++
 2 files changed, 20 insertions(+)
drivers/media/v4l2-core/v4l2-subdev.c +0 -0
index a3074f469b15..60b8febd3339 100644

drivers/media/v4l2-core/v4l2-subdev.c int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad, · line 2457

Quoted 5 lines
drivers/media/v4l2-core/v4l2-subdev.c +2 -0
@@ -2453,6 +2453,23 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_disable_streams);
 
+u64 v4l2_subdev_state_streams_enabled(struct v4l2_subdev *sd,
+				      struct v4l2_subdev_state *state, u32 pad)

You can make the state const.

drivers/media/v4l2-core/v4l2-subdev.c int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad, · line 2462

Quoted 5 lines
drivers/media/v4l2-core/v4l2-subdev.c +5 -0
+{
+	u64 streams_mask = 0;
+
+	for (unsigned int i = 0; i < state->stream_configs.num_configs; i++) {
+		struct v4l2_subdev_stream_config *cfg =

const

Quoted 18 lines
drivers/media/v4l2-core/v4l2-subdev.c +10 -0
+			&state->stream_configs.configs[i];
+
+		if (cfg->pad == pad && cfg->enabled)
+			streams_mask |= BIT_ULL(cfg->stream);
+	}
+
+	return streams_mask;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_state_streams_enabled);
+
 int v4l2_subdev_s_stream_helper(struct v4l2_subdev *sd, int enable)
 {
 	struct v4l2_subdev_state *state;
include/media/v4l2-subdev.h +0 -0
index 57f2bcb4eb16..deab128a4779 100644

include/media/v4l2-subdev.h int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad, · line 1733

Quoted 3 lines
include/media/v4l2-subdev.h +0 -0
@@ -1731,6 +1731,9 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
 int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
 				u64 streams_mask);
 

Missing documentation.

The function looks fine. It's a bit costly though, so I'll see in subsequent patches if there could be a better way to handle whatever issue this solves.

Quoted 6 lines
include/media/v4l2-subdev.h +3 -0
+u64 v4l2_subdev_state_streams_enabled(struct v4l2_subdev *sd,
+				      struct v4l2_subdev_state *state, u32 pad);
+
 /**
  * v4l2_subdev_s_stream_helper() - Helper to implement the subdev s_stream
  *	operation using enable_streams and disable_streams
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#17 lore raw

PATCH 5/13 media: v4l: Make media_entity_to_video_device() NULL-safe

In reply to Sakari Ailus

Make media_entity_to_video_device(NULL) return NULL, instead of an invalid pointer value.

---
 include/media/v4l2-dev.h | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
include/media/v4l2-dev.h +10 -4
index 1b6222fab24e..069c2f14b473 100644
@@ -313,10 +313,16 @@ struct video_device {
  * media_entity_to_video_device - Returns a &struct video_device from
  *	the &struct media_entity embedded on it.
  *
- * @__entity: pointer to &struct media_entity
- */
-#define media_entity_to_video_device(__entity) \
-	container_of(__entity, struct video_device, entity)
+ * @__entity: pointer to &struct media_entity, may be NULL
+ */
+#define media_entity_to_video_device(__entity)				\
+	({								\
+		typeof (__entity) __me_to_vdev_ent = __entity;		\
+									\
+		__me_to_vdev_ent ? container_of(__me_to_vdev_ent,	\
+						struct video_device, entity) : \
+			NULL;						\
+	})
 
 /**
  * to_video_device - Returns a &struct video_device from the
Signature · 1 line
2.39.5
#18 lore raw

Re: [PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe

Hi Sakari,

Thank you for the patch.

Quoted 20 lines · Sakari Ailus

Make media_entity_to_video_device(NULL) return NULL, instead of an invalid pointer value.

---
 include/media/v4l2-dev.h | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
include/media/v4l2-dev.h +0 -4
index 1b6222fab24e..069c2f14b473 100644
@@ -313,10 +313,16 @@ struct video_device {
  * media_entity_to_video_device - Returns a &struct video_device from
  *	the &struct media_entity embedded on it.
  *
- * @__entity: pointer to &struct media_entity
- */
-#define media_entity_to_video_device(__entity) \
-	container_of(__entity, struct video_device, entity)

include/media/v4l2-dev.h struct video_device { · line 325

Quoted 10 lines
include/media/v4l2-dev.h +10 -0
+ * @__entity: pointer to &struct media_entity, may be NULL
+ */
+#define media_entity_to_video_device(__entity)				\
+	({								\
+		typeof (__entity) __me_to_vdev_ent = __entity;		\
+									\
+		__me_to_vdev_ent ? container_of(__me_to_vdev_ent,	\
+						struct video_device, entity) : \
+			NULL;						\
+	})

This makes the macro safer, it's a good idea. Wouldn't it be better implemented as a container_of_null() (name to be bikeshedded) though ? I don't think media_entity_to_video_device() is the only macro that could benefit from this. It could even be integrated in container_of(), but I fear that could introduce issues.

Quoted 3 lines
include/media/v4l2-dev.h +0 -0
 /**
  * to_video_device - Returns a &struct video_device from the
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#19 lore raw

Re: [PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe

Hi Laurent,

Quoted 33 lines · Laurent Pinchart

Hi Sakari,

Thank you for the patch.

Quoted 28 lines · Sakari Ailus

Make media_entity_to_video_device(NULL) return NULL, instead of an invalid pointer value.

---
 include/media/v4l2-dev.h | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
include/media/v4l2-dev.h +8 -4
index 1b6222fab24e..069c2f14b473 100644
@@ -313,10 +313,16 @@ struct video_device {
  * media_entity_to_video_device - Returns a &struct video_device from
  *	the &struct media_entity embedded on it.
  *
- * @__entity: pointer to &struct media_entity
- */
-#define media_entity_to_video_device(__entity) \
-	container_of(__entity, struct video_device, entity)
+ * @__entity: pointer to &struct media_entity, may be NULL
+ */
+#define media_entity_to_video_device(__entity)				\
+	({								\
+		typeof (__entity) __me_to_vdev_ent = __entity;		\
+									\
+		__me_to_vdev_ent ? container_of(__me_to_vdev_ent,	\
+						struct video_device, entity) : \
Quoted 8 lines

include/media/v4l2-dev.h struct video_device { · line 325

Quoted 2 lines
include/media/v4l2-dev.h +2 -0
+			NULL;						\
+	})

This makes the macro safer, it's a good idea. Wouldn't it be better implemented as a container_of_null() (name to be bikeshedded) though ? I don't think media_entity_to_video_device() is the only macro that could benefit from this. It could even be integrated in container_of(), but I fear that could introduce issues.

That sounds like a good idea. I'll first see how this would look like with container_of_const()...

Signature · 3 lines
Regards,

Sakari Ailus
#20 lore raw

Re: [PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe

Quoted 39 lines · Sakari Ailus
Quoted 29 lines · Laurent Pinchart
Quoted 28 lines · Sakari Ailus

Make media_entity_to_video_device(NULL) return NULL, instead of an invalid pointer value.

---
 include/media/v4l2-dev.h | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
include/media/v4l2-dev.h +8 -4
index 1b6222fab24e..069c2f14b473 100644
@@ -313,10 +313,16 @@ struct video_device {
  * media_entity_to_video_device - Returns a &struct video_device from
  *	the &struct media_entity embedded on it.
  *
- * @__entity: pointer to &struct media_entity
- */
-#define media_entity_to_video_device(__entity) \
-	container_of(__entity, struct video_device, entity)
+ * @__entity: pointer to &struct media_entity, may be NULL
+ */
+#define media_entity_to_video_device(__entity)				\
+	({								\
+		typeof (__entity) __me_to_vdev_ent = __entity;		\
+									\
+		__me_to_vdev_ent ? container_of(__me_to_vdev_ent,	\
+						struct video_device, entity) : \
Quoted 8 lines

include/media/v4l2-dev.h struct video_device { · line 325

Quoted 2 lines
include/media/v4l2-dev.h +2 -0
+			NULL;						\
+	})

This makes the macro safer, it's a good idea. Wouldn't it be better implemented as a container_of_null() (name to be bikeshedded) though ? I don't think media_entity_to_video_device() is the only macro that could benefit from this. It could even be integrated in container_of(), but I fear that could introduce issues.

Quoted 2 lines

That sounds like a good idea. I'll first see how this would look like with container_of_const()...

Thinking some more about this, I think we can move forward without waiting for container_of_null().

Should we however add a check to ensure the entity is a video device (by checking that the function is MEDIA_ENT_F_IO_V4L), and return NULL if it's not ? It would make the macro even safer to use. There would be a small additional runtime cost for call sites that guarantee the entity is a video device.

I checked the current users of the macro, and the vast majority of them are in the .link_validate() operation, where they know that the entity is a video device. There are just a handful of locations where a check precedes the media_entity_to_video_device() call. So maybe it's not worth it ?

I also found no caller that checks for entity != NULL before calling the macro. Is this change actually needed ?

Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#21 lore raw

Re: [PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe

Hi Laurent,

Quoted 52 lines · Laurent Pinchart
Quoted 39 lines · Sakari Ailus
Quoted 29 lines · Laurent Pinchart
Quoted 28 lines · Sakari Ailus

Make media_entity_to_video_device(NULL) return NULL, instead of an invalid pointer value.

---
 include/media/v4l2-dev.h | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
include/media/v4l2-dev.h +8 -4
index 1b6222fab24e..069c2f14b473 100644
@@ -313,10 +313,16 @@ struct video_device {
  * media_entity_to_video_device - Returns a &struct video_device from
  *	the &struct media_entity embedded on it.
  *
- * @__entity: pointer to &struct media_entity
- */
-#define media_entity_to_video_device(__entity) \
-	container_of(__entity, struct video_device, entity)
+ * @__entity: pointer to &struct media_entity, may be NULL
+ */
+#define media_entity_to_video_device(__entity)				\
+	({								\
+		typeof (__entity) __me_to_vdev_ent = __entity;		\
+									\
+		__me_to_vdev_ent ? container_of(__me_to_vdev_ent,	\
+						struct video_device, entity) : \
Quoted 8 lines

include/media/v4l2-dev.h struct video_device { · line 325

Quoted 2 lines
include/media/v4l2-dev.h +2 -0
+			NULL;						\
+	})

This makes the macro safer, it's a good idea. Wouldn't it be better implemented as a container_of_null() (name to be bikeshedded) though ? I don't think media_entity_to_video_device() is the only macro that could benefit from this. It could even be integrated in container_of(), but I fear that could introduce issues.

Quoted 2 lines

That sounds like a good idea. I'll first see how this would look like with container_of_const()...

Thinking some more about this, I think we can move forward without waiting for container_of_null().

Should we however add a check to ensure the entity is a video device (by checking that the function is MEDIA_ENT_F_IO_V4L), and return NULL if it's not ? It would make the macro even safer to use. There would be a small additional runtime cost for call sites that guarantee the entity is a video device.

Quoted 8 lines

I checked the current users of the macro, and the vast majority of them are in the .link_validate() operation, where they know that the entity is a video device. There are just a handful of locations where a check precedes the media_entity_to_video_device() call. So maybe it's not worth it ?

I also found no caller that checks for entity != NULL before calling the macro. Is this change actually needed ?

We have a similar check in media_entity_to_v4l2_subdev() macro. In the patches for streaming control there's a need for an explicit check without this patch. That's of course fine as well if we decide so.

Signature · 3 lines
Regards,

Sakari Ailus
Reviewed-by
#22 lore raw

Re: [PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe

Quoted 21 lines · Sakari Ailus
Quoted 20 lines · Laurent Pinchart
Quoted 19 lines · Sakari Ailus
Quoted 18 lines · Laurent Pinchart
Quoted 17 lines · Sakari Ailus

Make media_entity_to_video_device(NULL) return NULL, instead of an invalid pointer value.

---
 include/media/v4l2-dev.h | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
include/media/v4l2-dev.h +0 -1
index 1b6222fab24e..069c2f14b473 100644
@@ -313,10 +313,16 @@ struct video_device {
  * media_entity_to_video_device - Returns a &struct video_device from
  *	the &struct media_entity embedded on it.
  *
- * @__entity: pointer to &struct media_entity
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

include/media/v4l2-dev.h struct video_device { · line 320

Quoted 8 lines
include/media/v4l2-dev.h +5 -3
- */
-#define media_entity_to_video_device(__entity) \
-	container_of(__entity, struct video_device, entity)
+ * @__entity: pointer to &struct media_entity, may be NULL
+ */
+#define media_entity_to_video_device(__entity)				\
+	({								\
+		typeof (__entity) __me_to_vdev_ent = __entity;		\

This should be __me_vdev_ent to align the naming with media_entity_to_v4l2_subdev().

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

include/media/v4l2-dev.h struct video_device { · line 323

Quoted 3 lines
include/media/v4l2-dev.h +3 -0
+									\
+		__me_to_vdev_ent ? container_of(__me_to_vdev_ent,	\
+						struct video_device, entity) : \

And here you should write

	__me_to_vdev_ent ?						   \
		container_of(__me_vdev_ent, struct video_device, entity) : \
		NULL;							   \
Quoted 27 lines
Quoted 27 lines
Quoted 11 lines
Quoted 8 lines

include/media/v4l2-dev.h struct video_device { · line 325

Quoted 2 lines
include/media/v4l2-dev.h +2 -0
+			NULL;						\
+	})

This makes the macro safer, it's a good idea. Wouldn't it be better implemented as a container_of_null() (name to be bikeshedded) though ? I don't think media_entity_to_video_device() is the only macro that could benefit from this. It could even be integrated in container_of(), but I fear that could introduce issues.

That sounds like a good idea. I'll first see how this would look like with container_of_const()...

Thinking some more about this, I think we can move forward without waiting for container_of_null().

Should we however add a check to ensure the entity is a video device (by checking that the function is MEDIA_ENT_F_IO_V4L), and return NULL if it's not ? It would make the macro even safer to use. There would be a small additional runtime cost for call sites that guarantee the entity is a video device.

I checked the current users of the macro, and the vast majority of them are in the .link_validate() operation, where they know that the entity is a video device. There are just a handful of locations where a check precedes the media_entity_to_video_device() call. So maybe it's not worth it ?

Quoted 4 lines
Quoted 2 lines

I also found no caller that checks for entity != NULL before calling the macro. Is this change actually needed ?

We have a similar check in media_entity_to_v4l2_subdev() macro.

That's a good point. Different behaviours for the two macros would be confusing I suppose, so

I'm tempted to convert the macro to an inline function, but that would make it more difficult to switch to container_of_const().

Quoted 3 lines

In the
patches for streaming control there's a need for an explicit check without this patch. That's of course fine as well if we decide so.

Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#23 lore raw

Re: [PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe

Hi Laurent,

Quoted 40 lines · Laurent Pinchart
Quoted 21 lines · Sakari Ailus
Quoted 20 lines · Laurent Pinchart
Quoted 19 lines · Sakari Ailus
Quoted 18 lines · Laurent Pinchart
Quoted 17 lines · Sakari Ailus

Make media_entity_to_video_device(NULL) return NULL, instead of an invalid pointer value.

---
 include/media/v4l2-dev.h | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
include/media/v4l2-dev.h +0 -1
index 1b6222fab24e..069c2f14b473 100644
@@ -313,10 +313,16 @@ struct video_device {
  * media_entity_to_video_device - Returns a &struct video_device from
  *	the &struct media_entity embedded on it.
  *
- * @__entity: pointer to &struct media_entity
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

include/media/v4l2-dev.h struct video_device { · line 320

Quoted 8 lines
include/media/v4l2-dev.h +5 -3
- */
-#define media_entity_to_video_device(__entity) \
-	container_of(__entity, struct video_device, entity)
+ * @__entity: pointer to &struct media_entity, may be NULL
+ */
+#define media_entity_to_video_device(__entity)				\
+	({								\
+		typeof (__entity) __me_to_vdev_ent = __entity;		\

This should be __me_vdev_ent to align the naming with media_entity_to_v4l2_subdev().

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

include/media/v4l2-dev.h struct video_device { · line 323

Quoted 3 lines
include/media/v4l2-dev.h +3 -0
+									\
+		__me_to_vdev_ent ? container_of(__me_to_vdev_ent,	\
+						struct video_device, entity) : \

And here you should write

Quoted 3 lines
	__me_to_vdev_ent ?						   \
		container_of(__me_vdev_ent, struct video_device, entity) : \
		NULL;							   \

I'm fine wrap after '?' as well, but another wrap is needed due to indentation in any case.

Quoted 31 lines
Quoted 31 lines
Quoted 30 lines
Quoted 10 lines
Quoted 9 lines

include/media/v4l2-dev.h struct video_device { · line 325

Quoted 3 lines
include/media/v4l2-dev.h +2 -0
+			NULL;						\
+	})

This makes the macro safer, it's a good idea. Wouldn't it be better implemented as a container_of_null() (name to be bikeshedded) though ? I don't think media_entity_to_video_device() is the only macro that could benefit from this. It could even be integrated in container_of(), but I fear that could introduce issues.

Quoted 2 lines

That sounds like a good idea. I'll first see how this would look like with container_of_const()...

Thinking some more about this, I think we can move forward without waiting for container_of_null().

Should we however add a check to ensure the entity is a video device (by checking that the function is MEDIA_ENT_F_IO_V4L), and return NULL if it's not ? It would make the macro even safer to use. There would be a small additional runtime cost for call sites that guarantee the entity is a video device.

I checked the current users of the macro, and the vast majority of them are in the .link_validate() operation, where they know that the entity is a video device. There are just a handful of locations where a check precedes the media_entity_to_video_device() call. So maybe it's not worth it ?

I also found no caller that checks for entity != NULL before calling the macro. Is this change actually needed ?

Quoted 6 lines
Quoted 1 line

We have a similar check in media_entity_to_v4l2_subdev() macro.

That's a good point. Different behaviours for the two macros would be confusing I suppose, so

Thank you.

Quoted 3 lines

I'm tempted to convert the macro to an inline function, but that would make it more difficult to switch to container_of_const().

Let's not do that. Eventually container_of() should become what container_of_const() is at the moment.

But once this is in, I can introduce container_of_null() which we could later use. I wonder how it will look like.

Signature · 3 lines
Regards,

Sakari Ailus
#24 lore raw

Re: [PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe

Quoted 43 lines · Sakari Ailus

Hi Laurent,

Quoted 40 lines · Laurent Pinchart
Quoted 21 lines · Sakari Ailus
Quoted 20 lines · Laurent Pinchart
Quoted 19 lines · Sakari Ailus
Quoted 18 lines · Laurent Pinchart
Quoted 17 lines · Sakari Ailus

Make media_entity_to_video_device(NULL) return NULL, instead of an invalid pointer value.

---
 include/media/v4l2-dev.h | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
include/media/v4l2-dev.h +0 -1
index 1b6222fab24e..069c2f14b473 100644
@@ -313,10 +313,16 @@ struct video_device {
  * media_entity_to_video_device - Returns a &struct video_device from
  *	the &struct media_entity embedded on it.
  *
- * @__entity: pointer to &struct media_entity
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

include/media/v4l2-dev.h struct video_device { · line 320

Quoted 8 lines
include/media/v4l2-dev.h +5 -3
- */
-#define media_entity_to_video_device(__entity) \
-	container_of(__entity, struct video_device, entity)
+ * @__entity: pointer to &struct media_entity, may be NULL
+ */
+#define media_entity_to_video_device(__entity)				\
+	({								\
+		typeof (__entity) __me_to_vdev_ent = __entity;		\

This should be __me_vdev_ent to align the naming with media_entity_to_v4l2_subdev().

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

include/media/v4l2-dev.h struct video_device { · line 323

Quoted 3 lines
include/media/v4l2-dev.h +3 -0
+									\
+		__me_to_vdev_ent ? container_of(__me_to_vdev_ent,	\
+						struct video_device, entity) : \

And here you should write

Quoted 6 lines
Quoted 3 lines
	__me_to_vdev_ent ?						   \
		container_of(__me_vdev_ent, struct video_device, entity) : \
		NULL;							   \

I'm fine wrap after '?' as well, but another wrap is needed due to indentation in any case.

I didn't make it clear, my comment was about replicating the style of media_entity_to_v4l2_subdev().

Quoted 47 lines
Quoted 30 lines
Quoted 30 lines
Quoted 29 lines
Quoted 11 lines
Quoted 8 lines

include/media/v4l2-dev.h struct video_device { · line 325

Quoted 2 lines
include/media/v4l2-dev.h +2 -0
+			NULL;						\
+	})

This makes the macro safer, it's a good idea. Wouldn't it be better implemented as a container_of_null() (name to be bikeshedded) though ? I don't think media_entity_to_video_device() is the only macro that could benefit from this. It could even be integrated in container_of(), but I fear that could introduce issues.

That sounds like a good idea. I'll first see how this would look like with container_of_const()...

Thinking some more about this, I think we can move forward without waiting for container_of_null().

Should we however add a check to ensure the entity is a video device (by checking that the function is MEDIA_ENT_F_IO_V4L), and return NULL if it's not ? It would make the macro even safer to use. There would be a small additional runtime cost for call sites that guarantee the entity is a video device.

I checked the current users of the macro, and the vast majority of them are in the .link_validate() operation, where they know that the entity is a video device. There are just a handful of locations where a check precedes the media_entity_to_video_device() call. So maybe it's not worth it ?

I also found no caller that checks for entity != NULL before calling the macro. Is this change actually needed ?

Quoted 6 lines
Quoted 1 line

We have a similar check in media_entity_to_v4l2_subdev() macro.

That's a good point. Different behaviours for the two macros would be confusing I suppose, so

Thank you.

Quoted 2 lines

I'm tempted to convert the macro to an inline function, but that would make it more difficult to switch to container_of_const().

Let's not do that. Eventually container_of() should become what container_of_const() is at the moment.

But once this is in, I can introduce container_of_null() which we could later use. I wonder how it will look like.

Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#25 lore raw

Re: [PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe

Quoted 48 lines · Laurent Pinchart
Quoted 47 lines · Sakari Ailus

Hi Laurent,

Quoted 43 lines · Laurent Pinchart
Quoted 21 lines · Sakari Ailus
Quoted 20 lines · Laurent Pinchart
Quoted 19 lines · Sakari Ailus
Quoted 18 lines · Laurent Pinchart
Quoted 17 lines · Sakari Ailus

Make media_entity_to_video_device(NULL) return NULL, instead of an invalid pointer value.

---
 include/media/v4l2-dev.h | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
include/media/v4l2-dev.h +0 -1
index 1b6222fab24e..069c2f14b473 100644
@@ -313,10 +313,16 @@ struct video_device {
  * media_entity_to_video_device - Returns a &struct video_device from
  *	the &struct media_entity embedded on it.
  *
- * @__entity: pointer to &struct media_entity
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

include/media/v4l2-dev.h struct video_device { · line 320

Quoted 8 lines
include/media/v4l2-dev.h +5 -3
- */
-#define media_entity_to_video_device(__entity) \
-	container_of(__entity, struct video_device, entity)
+ * @__entity: pointer to &struct media_entity, may be NULL
+ */
+#define media_entity_to_video_device(__entity)				\
+	({								\
+		typeof (__entity) __me_to_vdev_ent = __entity;		\

This should be __me_vdev_ent to align the naming with media_entity_to_v4l2_subdev().

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

include/media/v4l2-dev.h struct video_device { · line 323

Quoted 3 lines
include/media/v4l2-dev.h +3 -0
+									\
+		__me_to_vdev_ent ? container_of(__me_to_vdev_ent,	\
+						struct video_device, entity) : \

And here you should write

	__me_to_vdev_ent ?						   \
		container_of(__me_vdev_ent, struct video_device, entity) : \
		NULL;							   \
Quoted 5 lines
Quoted 2 lines

I'm fine wrap after '?' as well, but another wrap is needed due to indentation in any case.

I didn't make it clear, my comment was about replicating the style of media_entity_to_v4l2_subdev().

Ok, I'll use that in v3.

Signature · 1 line
Sakari Ailus
author maintainer
#26 lore raw

PATCH 6/13 media: v4l2-subdev: Mark both streams of a route enabled

In reply to Sakari Ailus

Streams are stored in sink-source pairs in sub-device state. When a stream was marked enabled (or disabled), only the state of one end of the stream was modified, leaving the stream in an incoherent state. Mark both ends of the stream enabled (or disabled).

---
 drivers/media/v4l2-core/v4l2-subdev.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/media/v4l2-core/v4l2-subdev.c +3 -1
index 60b8febd3339..5afdd9d548b5 100644
@@ -2254,9 +2254,11 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
 	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
 		struct v4l2_subdev_stream_config *cfg =
 			&state->stream_configs.configs[i];
+		struct v4l2_subdev_stream_config *cfg2 =
+			&state->stream_configs.configs[i ^ 1U];
 
 		if (cfg->pad == pad && (streams_mask & BIT_ULL(cfg->stream)))
-			cfg->enabled = enabled;
+			cfg->enabled = cfg2->enabled = enabled;
 	}
 }
 
Signature · 1 line
2.39.5
#27 lore raw

Re: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled

Hi Sakari,

Thank you for the patch.

Quoted 1 line · Sakari Ailus

Streams are stored in sink-source pairs in sub-device state.

Is it really this simple ? Don't we support for instance stream merging where two streams on possibly different sink pads are routed to a single stream on a source pad ?

Quoted 27 lines

When a stream
was marked enabled (or disabled), only the state of one end of the stream was modified, leaving the stream in an incoherent state. Mark both ends of the stream enabled (or disabled).

---
 drivers/media/v4l2-core/v4l2-subdev.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/media/v4l2-core/v4l2-subdev.c +3 -1
index 60b8febd3339..5afdd9d548b5 100644
@@ -2254,9 +2254,11 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
 	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
 		struct v4l2_subdev_stream_config *cfg =
 			&state->stream_configs.configs[i];
+		struct v4l2_subdev_stream_config *cfg2 =
+			&state->stream_configs.configs[i ^ 1U];
 
 		if (cfg->pad == pad && (streams_mask & BIT_ULL(cfg->stream)))
-			cfg->enabled = enabled;
+			cfg->enabled = cfg2->enabled = enabled;
 	}
 }
 
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#28 lore raw

Re: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled

Hi Laurent,

Quoted 3 lines · Laurent Pinchart

Hi Sakari,

Thank you for the patch.

Thanks for the review.

Quoted 7 lines · Sakari Ailus
Quoted 1 line

Streams are stored in sink-source pairs in sub-device state.

Is it really this simple ? Don't we support for instance stream merging where two streams on possibly different sink pads are routed to a single stream on a source pad ?

As far as I'm concerned, yes. Each route has a sink and a source pad, you may have multiple routes from a pad or to a pad, but these are all separate routes. The UAPI as well as v4l2-subdev.c is aligned with this and I wouldn't expect it to be otherwise.

I'd like to have Tomi's view on this, too.

Quoted 5 lines
Quoted 5 lines

When a stream
was marked enabled (or disabled), only the state of one end of the stream was modified, leaving the stream in an incoherent state. Mark both ends of the stream enabled (or disabled).

This should discuss "route" in fact, not "stream".

Signature · 3 lines
Regards,

Sakari Ailus
#29 lore raw

Re: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled

Quoted 4 lines · Sakari Ailus
Quoted 3 lines · Laurent Pinchart
Quoted 1 line · Sakari Ailus

Streams are stored in sink-source pairs in sub-device state.

Quoted 8 lines
Quoted 3 lines

Is it really this simple ? Don't we support for instance stream merging where two streams on possibly different sink pads are routed to a single stream on a source pad ?

As far as I'm concerned, yes. Each route has a sink and a source pad, you may have multiple routes from a pad or to a pad, but these are all separate routes. The UAPI as well as v4l2-subdev.c is aligned with this and I wouldn't expect it to be otherwise.

Should we then refactor v4l2_subdev_stream_configs to contain a list of route configs, with each route config containing two stream configs, one for the sink pad and one for the source pad ? We would still have duplicated entries when multiple routes originate from or end at the same pad/stream, but the data structures would be more explicit and we would rely less on the fact that each route has a pair of consecutive stream configs.

Quoted 8 lines

I'd like to have Tomi's view on this, too.

Quoted 4 lines
Quoted 4 lines

When a stream
was marked enabled (or disabled), only the state of one end of the stream was modified, leaving the stream in an incoherent state. Mark both ends of the stream enabled (or disabled).

This should discuss "route" in fact, not "stream".

That would make it a bit clearer.

Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#30 lore raw

Re: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled

Hi Laurent,

Quoted 14 lines · Laurent Pinchart
Quoted 4 lines · Sakari Ailus
Quoted 3 lines · Laurent Pinchart
Quoted 1 line · Sakari Ailus

Streams are stored in sink-source pairs in sub-device state.

Quoted 8 lines
Quoted 3 lines

Is it really this simple ? Don't we support for instance stream merging where two streams on possibly different sink pads are routed to a single stream on a source pad ?

As far as I'm concerned, yes. Each route has a sink and a source pad, you may have multiple routes from a pad or to a pad, but these are all separate routes. The UAPI as well as v4l2-subdev.c is aligned with this and I wouldn't expect it to be otherwise.

Quoted 7 lines

Should we then refactor v4l2_subdev_stream_configs to contain a list of route configs, with each route config containing two stream configs, one for the sink pad and one for the source pad ? We would still have duplicated entries when multiple routes originate from or end at the same pad/stream, but the data structures would be more explicit and we would rely less on the fact that each route has a pair of consecutive stream configs.

Most of the code dealing with stream configurations works on either sink or source pads and the current data structure makes that code more simple. I think this was probably a factor why Tomi wrote it like this.

Tomi, any comment on this?

Quoted 11 lines
Quoted 9 lines

I'd like to have Tomi's view on this, too.

Quoted 4 lines
Quoted 4 lines

When a stream
was marked enabled (or disabled), only the state of one end of the stream was modified, leaving the stream in an incoherent state. Mark both ends of the stream enabled (or disabled).

This should discuss "route" in fact, not "stream".

That would make it a bit clearer.

I'll fix this for v2.

Signature · 3 lines
Regards,

Sakari Ailus
#31 lore raw

Re: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled

In reply to Sakari Ailus
Quoted 13 lines · Sakari Ailus

Hi Laurent,

Quoted 3 lines · Laurent Pinchart

Hi Sakari,

Thank you for the patch.

Thanks for the review.

Quoted 4 lines · Sakari Ailus
Quoted 1 line

Streams are stored in sink-source pairs in sub-device state.

Quoted 8 lines
Quoted 3 lines

Is it really this simple ? Don't we support for instance stream merging where two streams on possibly different sink pads are routed to a single stream on a source pad ?

As far as I'm concerned, yes. Each route has a sink and a source pad, you may have multiple routes from a pad or to a pad, but these are all separate routes. The UAPI as well as v4l2-subdev.c is aligned with this and I wouldn't expect it to be otherwise.

But with stream merging/splitting we'll have one stream config on one pad, and two stream configs on the other. And thus two routes from that one config.

That said, if I recall right, I don't think merging/splitting has ever been really explored...

Tomi

#32 lore raw

Re: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled

Quoted 9 lines · Tomi Valkeinen
Quoted 8 lines · Sakari Ailus
Quoted 6 lines · Laurent Pinchart
Quoted 1 line · Sakari Ailus

Streams are stored in sink-source pairs in sub-device state.

Is it really this simple ? Don't we support for instance stream merging where two streams on possibly different sink pads are routed to a single stream on a source pad ?

Quoted 8 lines
Quoted 4 lines

As far as I'm concerned, yes. Each route has a sink and a source pad, you may have multiple routes from a pad or to a pad, but these are all separate routes. The UAPI as well as v4l2-subdev.c is aligned with this and I wouldn't expect it to be otherwise.

But with stream merging/splitting we'll have one stream config on one pad, and two stream configs on the other. And thus two routes from that one config.

With the current code, we will have duplicated stream config entries for the same stream in that case. Only the first one will be seen by the v4l2_subdev_state_get_format() function, but both will exist.

Quoted 2 lines

That said, if I recall right, I don't think merging/splitting has ever been really explored...

Signature · 3 lines
Regards,

Laurent Pinchart
#33 lore raw

Re: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled

In reply to Sakari Ailus

Hi,

Quoted 15 lines · Sakari Ailus

Streams are stored in sink-source pairs in sub-device state. When a stream was marked enabled (or disabled), only the state of one end of the stream was modified, leaving the stream in an incoherent state. Mark both ends of the stream enabled (or disabled).

---
 drivers/media/v4l2-core/v4l2-subdev.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/media/v4l2-core/v4l2-subdev.c +0 -0
index 60b8febd3339..5afdd9d548b5 100644

drivers/media/v4l2-core/v4l2-subdev.c static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd, · line 2258

Quoted 5 lines
drivers/media/v4l2-core/v4l2-subdev.c +2 -0
@@ -2254,9 +2254,11 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
 	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
 		struct v4l2_subdev_stream_config *cfg =
 			&state->stream_configs.configs[i];
+		struct v4l2_subdev_stream_config *cfg2 =
+			&state->stream_configs.configs[i ^ 1U];

I don't think this is correct. There's no specific rule that the stream configs would be in a particular order. Or that there even are a pair.

You should look for the other end of the stream from the routing table.

Tomi

Quoted 7 lines
drivers/media/v4l2-core/v4l2-subdev.c +1 -1
 		if (cfg->pad == pad && (streams_mask & BIT_ULL(cfg->stream)))
-			cfg->enabled = enabled;
+			cfg->enabled = cfg2->enabled = enabled;
 	}
 }
 
author maintainer
#34 lore raw

Re: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled

Moi,

Quoted 27 lines · Tomi Valkeinen

Hi,

Quoted 15 lines · Sakari Ailus

Streams are stored in sink-source pairs in sub-device state. When a stream was marked enabled (or disabled), only the state of one end of the stream was modified, leaving the stream in an incoherent state. Mark both ends of the stream enabled (or disabled).

---
 drivers/media/v4l2-core/v4l2-subdev.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/media/v4l2-core/v4l2-subdev.c +0 -0
index 60b8febd3339..5afdd9d548b5 100644

drivers/media/v4l2-core/v4l2-subdev.c static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd, · line 2258

Quoted 5 lines
drivers/media/v4l2-core/v4l2-subdev.c +2 -0
@@ -2254,9 +2254,11 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
 	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
 		struct v4l2_subdev_stream_config *cfg =
 			&state->stream_configs.configs[i];
+		struct v4l2_subdev_stream_config *cfg2 =
+			&state->stream_configs.configs[i ^ 1U];

I don't think this is correct. There's no specific rule that the stream configs would be in a particular order. Or that there even are a pair.

Quoted 1 line

You should look for the other end of the stream from the routing table.

Currently the code managing the streams stores them in this array in pairs, the sink followed by the source for each route. As noted in the commit message, this patch also fixes a bug.

I can change the patch to use routes to find the other end, but over time we might want to use a different data structure to store this information (a linked list perhaps). The number of routes and stream configurations is presumably fairly small in most cases so right now I presume there's no noteworthy effect on performance.

Signature · 3 lines
Terveisin,

Sakari Ailus
#35 lore raw

Re: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled

Quoted 32 lines · Sakari Ailus
Quoted 25 lines · Tomi Valkeinen
Quoted 15 lines · Sakari Ailus

Streams are stored in sink-source pairs in sub-device state. When a stream was marked enabled (or disabled), only the state of one end of the stream was modified, leaving the stream in an incoherent state. Mark both ends of the stream enabled (or disabled).

---
 drivers/media/v4l2-core/v4l2-subdev.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/media/v4l2-core/v4l2-subdev.c +0 -0
index 60b8febd3339..5afdd9d548b5 100644

drivers/media/v4l2-core/v4l2-subdev.c static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd, · line 2258

Quoted 5 lines
drivers/media/v4l2-core/v4l2-subdev.c +2 -0
@@ -2254,9 +2254,11 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
 	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
 		struct v4l2_subdev_stream_config *cfg =
 			&state->stream_configs.configs[i];
+		struct v4l2_subdev_stream_config *cfg2 =
+			&state->stream_configs.configs[i ^ 1U];

I don't think this is correct. There's no specific rule that the stream configs would be in a particular order. Or that there even are a pair.

Quoted 1 line

You should look for the other end of the stream from the routing table.

Currently the code managing the streams stores them in this array in pairs, the sink followed by the source for each route. As noted in the commit message, this patch also fixes a bug.

Quoted 5 lines

I can change the patch to use routes to find the other end, but over time we might want to use a different data structure to store this information (a linked list perhaps). The number of routes and stream configurations is presumably fairly small in most cases so right now I presume there's no noteworthy effect on performance.

At the very least I would like to document somewhere that each route stores a pair of stream config. We also really, *really*, *REALLY* need to make sure that as little code as possible depends on this, and especially that no driver accesses the stream configs directly... aargghghhhh the ds90ub913 driver does already :'-(

I've submitted [1] to make the v4l2_subdev_stream_config structure private.

[1] https://lore.kernel.org/r/[email protected]

Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#36 lore raw

PATCH 7/13 media: ipu6: Set up CSI-2 receiver at correct moment

In reply to Sakari Ailus

Enable the CSI-2 receiver before the first stream is started and disable it when the last stream is stopped. Before this patch, every time a stream was started, the CSI-2 receiver was enabled and similarly, it was disabled when any stream was stopped.

---
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 25 ++++++++++++-------
 1 file changed, 16 insertions(+), 9 deletions(-)
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +16 -9
index 6030bd23b4b9..3b837e9ccffe 100644
@@ -348,7 +348,7 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
 	struct ipu6_isys_csi2_timing timing = { };
 	struct v4l2_subdev *remote_sd;
 	struct media_pad *remote_pad;
-	u64 sink_streams;
+	u64 sink_streams, already_enabled;
 	int ret;
 
 	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
@@ -358,13 +358,17 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
 		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
 						&streams_mask);
 
-	ret = ipu6_isys_csi2_calc_timing(csi2, &timing, CSI2_ACCINV);
-	if (ret)
-		return ret;
+	already_enabled = v4l2_subdev_state_streams_enabled(sd, state,
+							    CSI2_PAD_SINK);
+	if (!already_enabled) {
+		ret = ipu6_isys_csi2_calc_timing(csi2, &timing, CSI2_ACCINV);
+		if (ret)
+			return ret;
 
-	ret = ipu6_isys_csi2_set_stream(sd, &timing, csi2->nlanes, true);
-	if (ret)
-		return ret;
+		ret = ipu6_isys_csi2_set_stream(sd, &timing, csi2->nlanes, true);
+		if (ret)
+			return ret;
+	}
 
 	ret = v4l2_subdev_enable_streams(remote_sd, remote_pad->index,
 					 sink_streams);
@@ -382,7 +386,7 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,
 {
 	struct v4l2_subdev *remote_sd;
 	struct media_pad *remote_pad;
-	u64 sink_streams;
+	u64 sink_streams, still_enabled;
 
 	sink_streams =
 		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
@@ -391,7 +395,10 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,
 	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
 	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
 
-	ipu6_isys_csi2_set_stream(sd, NULL, 0, false);
+	still_enabled = v4l2_subdev_state_streams_enabled(sd, state,
+							  CSI2_PAD_SINK);
+	if (still_enabled == sink_streams)
+		ipu6_isys_csi2_set_stream(sd, NULL, 0, false);
 
 	v4l2_subdev_disable_streams(remote_sd, remote_pad->index, sink_streams);
 
Signature · 1 line
2.39.5
#37 lore raw

Re: [PATCH 07/13] media: ipu6: Set up CSI-2 receiver at correct moment

Hi Sakari,

Thank you for the patch.

Quoted 4 lines · Sakari Ailus

Enable the CSI-2 receiver before the first stream is started and disable it when the last stream is stopped. Before this patch, every time a stream was started, the CSI-2 receiver was enabled and similarly, it was disabled when any stream was stopped.

You mentioned in 03/13 that we should only enable the source when all video capture devices have been started. Shouldn't you enable the CSI-2 receiver when *all* streams are enabled, not when the first one is enabled (and similarly when stopping) ?

Quoted 19 lines
---
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 25 ++++++++++++-------
 1 file changed, 16 insertions(+), 9 deletions(-)
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +1 -1
index 6030bd23b4b9..3b837e9ccffe 100644
@@ -348,7 +348,7 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
 	struct ipu6_isys_csi2_timing timing = { };
 	struct v4l2_subdev *remote_sd;
 	struct media_pad *remote_pad;
-	u64 sink_streams;
+	u64 sink_streams, already_enabled;
 	int ret;
 
 	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);

drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd, · line 362

Quoted 8 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +2 -3
@@ -358,13 +358,17 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
 		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
 						&streams_mask);
 
-	ret = ipu6_isys_csi2_calc_timing(csi2, &timing, CSI2_ACCINV);
-	if (ret)
-		return ret;
+	already_enabled = v4l2_subdev_state_streams_enabled(sd, state,
+							    CSI2_PAD_SINK);

It would be cheaper to store a bitmask of enabled streams on the csi2 structure.

Quoted 36 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +13 -5
+	if (!already_enabled) {
+		ret = ipu6_isys_csi2_calc_timing(csi2, &timing, CSI2_ACCINV);
+		if (ret)
+			return ret;
 
-	ret = ipu6_isys_csi2_set_stream(sd, &timing, csi2->nlanes, true);
-	if (ret)
-		return ret;
+		ret = ipu6_isys_csi2_set_stream(sd, &timing, csi2->nlanes, true);
+		if (ret)
+			return ret;
+	}
 
 	ret = v4l2_subdev_enable_streams(remote_sd, remote_pad->index,
 					 sink_streams);
@@ -382,7 +386,7 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,
 {
 	struct v4l2_subdev *remote_sd;
 	struct media_pad *remote_pad;
-	u64 sink_streams;
+	u64 sink_streams, still_enabled;
 
 	sink_streams =
 		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
@@ -391,7 +395,10 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,
 	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
 	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
 
-	ipu6_isys_csi2_set_stream(sd, NULL, 0, false);
+	still_enabled = v4l2_subdev_state_streams_enabled(sd, state,
+							  CSI2_PAD_SINK);
+	if (still_enabled == sink_streams)
+		ipu6_isys_csi2_set_stream(sd, NULL, 0, false);
 
 	v4l2_subdev_disable_streams(remote_sd, remote_pad->index, sink_streams);
 
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#38 lore raw

Re: [PATCH 07/13] media: ipu6: Set up CSI-2 receiver at correct moment

Hi Laurent,

Quoted 3 lines · Laurent Pinchart

Hi Sakari,

Thank you for the patch.

Thanks for the review.

Quoted 11 lines · Sakari Ailus
Quoted 4 lines

Enable the CSI-2 receiver before the first stream is started and disable it when the last stream is stopped. Before this patch, every time a stream was started, the CSI-2 receiver was enabled and similarly, it was disabled when any stream was stopped.

You mentioned in 03/13 that we should only enable the source when all video capture devices have been started. Shouldn't you enable the CSI-2 receiver when *all* streams are enabled, not when the first one is enabled (and similarly when stopping) ?

Indeed. That only comes in the last patch, but I think some of these can be squashed, too.

Quoted 23 lines
Quoted 23 lines
---
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 25 ++++++++++++-------
 1 file changed, 16 insertions(+), 9 deletions(-)
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +1 -1
index 6030bd23b4b9..3b837e9ccffe 100644
@@ -348,7 +348,7 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
 	struct ipu6_isys_csi2_timing timing = { };
 	struct v4l2_subdev *remote_sd;
 	struct media_pad *remote_pad;
-	u64 sink_streams;
+	u64 sink_streams, already_enabled;
 	int ret;
 
 	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
@@ -358,13 +358,17 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
 		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
 						&streams_mask);
 
Quoted 8 lines

drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd, · line 362

Quoted 5 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +2 -3
-	ret = ipu6_isys_csi2_calc_timing(csi2, &timing, CSI2_ACCINV);
-	if (ret)
-		return ret;
+	already_enabled = v4l2_subdev_state_streams_enabled(sd, state,
+							    CSI2_PAD_SINK);

It would be cheaper to store a bitmask of enabled streams on the csi2 structure.

I think I'll do just that. Indeed, this soon won't be even usable here in any case.

Quoted 38 lines
Quoted 37 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +13 -5
+	if (!already_enabled) {
+		ret = ipu6_isys_csi2_calc_timing(csi2, &timing, CSI2_ACCINV);
+		if (ret)
+			return ret;
 
-	ret = ipu6_isys_csi2_set_stream(sd, &timing, csi2->nlanes, true);
-	if (ret)
-		return ret;
+		ret = ipu6_isys_csi2_set_stream(sd, &timing, csi2->nlanes, true);
+		if (ret)
+			return ret;
+	}
 
 	ret = v4l2_subdev_enable_streams(remote_sd, remote_pad->index,
 					 sink_streams);
@@ -382,7 +386,7 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,
 {
 	struct v4l2_subdev *remote_sd;
 	struct media_pad *remote_pad;
-	u64 sink_streams;
+	u64 sink_streams, still_enabled;
 
 	sink_streams =
 		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
@@ -391,7 +395,10 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,
 	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
 	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
 
-	ipu6_isys_csi2_set_stream(sd, NULL, 0, false);
+	still_enabled = v4l2_subdev_state_streams_enabled(sd, state,
+							  CSI2_PAD_SINK);
+	if (still_enabled == sink_streams)
+		ipu6_isys_csi2_set_stream(sd, NULL, 0, false);
 
 	v4l2_subdev_disable_streams(remote_sd, remote_pad->index, sink_streams);
 
Signature · 3 lines
Regards,

Sakari Ailus
author maintainer
#39 lore raw

PATCH 8/13 media: v4l2-subdev: Print early in v4l2_subdev_{enable,disable}_streams()

In reply to Sakari Ailus

Print debug messages early in v4l2_subdev_enable_streams() and v4l2_subdev_disable_streams(), before sanity checks take place. This can help figuring out why something goes wrong, in driver development or otherwise.

Also print the name of the sub-device where streaming is to be enabled or disabled.

---
 drivers/media/v4l2-core/v4l2-subdev.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)
drivers/media/v4l2-core/v4l2-subdev.c +6 -4
index 5afdd9d548b5..6bc855058ca6 100644
@@ -2273,6 +2273,9 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
 	bool use_s_stream;
 	int ret;
 
+	dev_dbg(dev, "enable streams %s:%u/%#llx\n", sd->entity.name, pad,
+		streams_mask);
+
 	/* A few basic sanity checks first. */
 	if (pad >= sd->entity.num_pads)
 		return -EINVAL;
@@ -2320,8 +2323,6 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
 		goto done;
 	}
 
-	dev_dbg(dev, "enable streams %u:%#llx\n", pad, streams_mask);
-
 	already_streaming = v4l2_subdev_is_streaming(sd);
 
 	if (!use_s_stream) {
@@ -2373,6 +2374,9 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
 	bool use_s_stream;
 	int ret;
 
+	dev_dbg(dev, "disable streams %s:%u/%#llx\n", sd->entity.name, pad,
+		streams_mask);
+
 	/* A few basic sanity checks first. */
 	if (pad >= sd->entity.num_pads)
 		return -EINVAL;
@@ -2420,8 +2424,6 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
 		goto done;
 	}
 
-	dev_dbg(dev, "disable streams %u:%#llx\n", pad, streams_mask);
-
 	if (!use_s_stream) {
 		/* Call the .disable_streams() operation. */
 		ret = v4l2_subdev_call(sd, pad, disable_streams, state, pad,
Signature · 1 line
2.39.5
Reviewed-by on v1 8/13
#40 lore raw

Re: [PATCH 08/13] media: v4l2-subdev: Print early in v4l2_subdev_{enable,disable}_streams()

Hi Sakari,

Quoted 18 lines · Sakari Ailus

Print debug messages early in v4l2_subdev_enable_streams() and v4l2_subdev_disable_streams(), before sanity checks take place. This can help figuring out why something goes wrong, in driver development or otherwise.

Also print the name of the sub-device where streaming is to be enabled or disabled.

---
 drivers/media/v4l2-core/v4l2-subdev.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)
drivers/media/v4l2-core/v4l2-subdev.c +0 -0
index 5afdd9d548b5..6bc855058ca6 100644

drivers/media/v4l2-core/v4l2-subdev.c int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad, · line 2277

Quoted 5 lines
drivers/media/v4l2-core/v4l2-subdev.c +2 -0
@@ -2273,6 +2273,9 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
 	bool use_s_stream;
 	int ret;
 
+	dev_dbg(dev, "enable streams %s:%u/%#llx\n", sd->entity.name, pad,
+		streams_mask);

I suppose there can be multiple subdevs per struct device, so printing the entity name can be useful. I'd put quotes around '%s' as the entity name can contain spaces. Apart from that,

Quoted 32 lines
drivers/media/v4l2-core/v4l2-subdev.c +4 -4
+
 	/* A few basic sanity checks first. */
 	if (pad >= sd->entity.num_pads)
 		return -EINVAL;
@@ -2320,8 +2323,6 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
 		goto done;
 	}
 
-	dev_dbg(dev, "enable streams %u:%#llx\n", pad, streams_mask);
-
 	already_streaming = v4l2_subdev_is_streaming(sd);
 
 	if (!use_s_stream) {
@@ -2373,6 +2374,9 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
 	bool use_s_stream;
 	int ret;
 
+	dev_dbg(dev, "disable streams %s:%u/%#llx\n", sd->entity.name, pad,
+		streams_mask);
+
 	/* A few basic sanity checks first. */
 	if (pad >= sd->entity.num_pads)
 		return -EINVAL;
@@ -2420,8 +2424,6 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
 		goto done;
 	}
 
-	dev_dbg(dev, "disable streams %u:%#llx\n", pad, streams_mask);
-
 	if (!use_s_stream) {
 		/* Call the .disable_streams() operation. */
 		ret = v4l2_subdev_call(sd, pad, disable_streams, state, pad,
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#41 lore raw

Re: [PATCH 08/13] media: v4l2-subdev: Print early in v4l2_subdev_{enable,disable}_streams()

Hi Laurent,

Quoted 24 lines · Laurent Pinchart

Hi Sakari,

Quoted 21 lines · Sakari Ailus

Print debug messages early in v4l2_subdev_enable_streams() and v4l2_subdev_disable_streams(), before sanity checks take place. This can help figuring out why something goes wrong, in driver development or otherwise.

Also print the name of the sub-device where streaming is to be enabled or disabled.

---
 drivers/media/v4l2-core/v4l2-subdev.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)
drivers/media/v4l2-core/v4l2-subdev.c +0 -0
index 5afdd9d548b5..6bc855058ca6 100644
@@ -2273,6 +2273,9 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
 	bool use_s_stream;
 	int ret;
 
Quoted 6 lines

drivers/media/v4l2-core/v4l2-subdev.c int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad, · line 2277

Quoted 2 lines
drivers/media/v4l2-core/v4l2-subdev.c +2 -0
+	dev_dbg(dev, "enable streams %s:%u/%#llx\n", sd->entity.name, pad,
+		streams_mask);

I suppose there can be multiple subdevs per struct device, so printing the entity name can be useful. I'd put quotes around '%s' as the entity name can contain spaces. Apart from that,

I'll use double quotes as that is how sub-device (or entity) name is typically quoted in debug prints.

Quoted 2 lines

Thank you.

Signature · 3 lines
Regards,

Sakari Ailus
author maintainer
#42 lore raw

PATCH 9/13 media: v4l2-subdev: Collect streams on source pads only

In reply to Sakari Ailus

v4l2_subdev_collect_streams() is used to find the streams present on source pads only. Only iterate through the streams on source pads, i.e. on odd array indices.

---
 drivers/media/v4l2-core/v4l2-subdev.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)
drivers/media/v4l2-core/v4l2-subdev.c +8 -7
index 6bc855058ca6..932fca795d4a 100644
@@ -2225,16 +2225,17 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
 	*found_streams = 0;
 	*enabled_streams = 0;
 
-	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
-		const struct v4l2_subdev_stream_config *cfg =
-			&state->stream_configs.configs[i];
+	for (unsigned int i = 0; i < state->stream_configs.num_configs; i += 2) {
+		const struct v4l2_subdev_stream_config *src_cfg =
+			&state->stream_configs.configs[i + 1];
 
-		if (cfg->pad != pad || !(streams_mask & BIT_ULL(cfg->stream)))
+		if (src_cfg->pad != pad ||
+		    !(streams_mask & BIT_ULL(src_cfg->stream)))
 			continue;
 
-		*found_streams |= BIT_ULL(cfg->stream);
-		if (cfg->enabled)
-			*enabled_streams |= BIT_ULL(cfg->stream);
+		*found_streams |= BIT_ULL(src_cfg->stream);
+		if (src_cfg->enabled)
+			*enabled_streams |= BIT_ULL(src_cfg->stream);
 	}
 }
 
Signature · 1 line
2.39.5
#43 lore raw

Re: [PATCH 09/13] media: v4l2-subdev: Collect streams on source pads only

Hi Sakari,

Thank you for the patch.

Quoted 14 lines · Sakari Ailus

v4l2_subdev_collect_streams() is used to find the streams present on source pads only. Only iterate through the streams on source pads, i.e. on odd array indices.

---
 drivers/media/v4l2-core/v4l2-subdev.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)
drivers/media/v4l2-core/v4l2-subdev.c +0 -0
index 6bc855058ca6..932fca795d4a 100644

drivers/media/v4l2-core/v4l2-subdev.c static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd, · line 2230

Quoted 9 lines
drivers/media/v4l2-core/v4l2-subdev.c +3 -3
@@ -2225,16 +2225,17 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
 	*found_streams = 0;
 	*enabled_streams = 0;
 
-	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
-		const struct v4l2_subdev_stream_config *cfg =
-			&state->stream_configs.configs[i];
+	for (unsigned int i = 0; i < state->stream_configs.num_configs; i += 2) {
+		const struct v4l2_subdev_stream_config *src_cfg =
+			&state->stream_configs.configs[i + 1];

You could start at i = 1 and avoid the + 1 here.

Unless I missed it, I don't think we document anywhere that we store two values per route in the array, and that, if multiple routes have the same sink or source stream, multiple entries will exist for the same stream. I'd like to see this clearly explained somewhere, instead of relying on an implemetation detail only known by few people.

Quoted 15 lines
drivers/media/v4l2-core/v4l2-subdev.c +5 -4
-		if (cfg->pad != pad || !(streams_mask & BIT_ULL(cfg->stream)))
+		if (src_cfg->pad != pad ||
+		    !(streams_mask & BIT_ULL(src_cfg->stream)))
 			continue;
 
-		*found_streams |= BIT_ULL(cfg->stream);
-		if (cfg->enabled)
-			*enabled_streams |= BIT_ULL(cfg->stream);
+		*found_streams |= BIT_ULL(src_cfg->stream);
+		if (src_cfg->enabled)
+			*enabled_streams |= BIT_ULL(src_cfg->stream);
 	}
 }
 
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#44 lore raw

Re: [PATCH 09/13] media: v4l2-subdev: Collect streams on source pads only

Hi Laurent,

Quoted 3 lines · Laurent Pinchart

Hi Sakari,

Thank you for the patch.

Thank you for the review!

Quoted 28 lines · Sakari Ailus
Quoted 14 lines

v4l2_subdev_collect_streams() is used to find the streams present on source pads only. Only iterate through the streams on source pads, i.e. on odd array indices.

---
 drivers/media/v4l2-core/v4l2-subdev.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)
drivers/media/v4l2-core/v4l2-subdev.c +0 -0
index 6bc855058ca6..932fca795d4a 100644

drivers/media/v4l2-core/v4l2-subdev.c static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd, · line 2230

Quoted 9 lines
drivers/media/v4l2-core/v4l2-subdev.c +3 -3
@@ -2225,16 +2225,17 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
 	*found_streams = 0;
 	*enabled_streams = 0;
 
-	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
-		const struct v4l2_subdev_stream_config *cfg =
-			&state->stream_configs.configs[i];
+	for (unsigned int i = 0; i < state->stream_configs.num_configs; i += 2) {
+		const struct v4l2_subdev_stream_config *src_cfg =
+			&state->stream_configs.configs[i + 1];

You could start at i = 1 and avoid the + 1 here.

Quoted 5 lines

Unless I missed it, I don't think we document anywhere that we store two values per route in the array, and that, if multiple routes have the same sink or source stream, multiple entries will exist for the same stream. I'd like to see this clearly explained somewhere, instead of relying on an implemetation detail only known by few people.

I'll add this in a separate patch.

Signature · 3 lines
Regards,

Sakari Ailus
author maintainer
#45 lore raw

PATCH 10/13 media: v4l2-subdev: Add debug prints to v4l2_subdev_collect_streams()

In reply to Sakari Ailus

Print streams found by v4l2_subdev_collect_streams() at debug level.

---
 drivers/media/v4l2-core/v4l2-subdev.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/media/v4l2-core/v4l2-subdev.c +5 -0
index 932fca795d4a..c549a462dac7 100644
@@ -2219,6 +2219,8 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
 		*found_streams = BIT_ULL(0);
 		*enabled_streams =
 			(sd->enabled_pads & BIT_ULL(pad)) ? BIT_ULL(0) : 0;
+		dev_dbg(sd->dev,
+			"collect_streams: sub-device does not support streams\n");
 		return;
 	}
 
@@ -2237,6 +2239,9 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
 		if (src_cfg->enabled)
 			*enabled_streams |= BIT_ULL(src_cfg->stream);
 	}
+	dev_dbg(sd->dev,
+		"collect_streams: %s:%u found %#llx enabled %#llx\n",
+		sd->entity.name, pad, *found_streams, *enabled_streams);
 }
 
 static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
Signature · 1 line
2.39.5
Reviewed-by on v1 10/13
#46 lore raw

Re: [PATCH 10/13] media: v4l2-subdev: Add debug prints to v4l2_subdev_collect_streams()

Hi Sakari,

Thank you for the patch.

Quoted 1 line · Sakari Ailus

Print streams found by v4l2_subdev_collect_streams() at debug level.

Explaining why you found this useful could help.

Quoted 10 lines
---
 drivers/media/v4l2-core/v4l2-subdev.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/media/v4l2-core/v4l2-subdev.c +0 -0
index 932fca795d4a..c549a462dac7 100644

drivers/media/v4l2-core/v4l2-subdev.c static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd, · line 2223

Quoted 5 lines
drivers/media/v4l2-core/v4l2-subdev.c +2 -0
@@ -2219,6 +2219,8 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
 		*found_streams = BIT_ULL(0);
 		*enabled_streams =
 			(sd->enabled_pads & BIT_ULL(pad)) ? BIT_ULL(0) : 0;
+		dev_dbg(sd->dev,
+			"collect_streams: sub-device does not support streams\n");

You print the subdev name below, I would print it here too. Maybe

			"collect_streams: '%s':%u: streams not supported\n",
			sd->entity.name, pad);

drivers/media/v4l2-core/v4l2-subdev.c static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd, · line 2241

Quoted 7 lines
drivers/media/v4l2-core/v4l2-subdev.c +0 -0
 		return;
 	}
 
@@ -2237,6 +2239,9 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
 		if (src_cfg->enabled)
 			*enabled_streams |= BIT_ULL(src_cfg->stream);
 	}

Please add a blank line here.

drivers/media/v4l2-core/v4l2-subdev.c static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd, · line 2243

Quoted 2 lines
drivers/media/v4l2-core/v4l2-subdev.c +2 -0
+	dev_dbg(sd->dev,
+		"collect_streams: %s:%u found %#llx enabled %#llx\n",
		"collect_streams: '%s':%u: found %#llx, enabled: %#llx\n",

or possibly

		"%s: '%s':%u: found %#llx, enabled: %#llx\n", __func__,

(same above).

Quoted 4 lines
drivers/media/v4l2-core/v4l2-subdev.c +1 -0
+		sd->entity.name, pad, *found_streams, *enabled_streams);
 }
 
 static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#47 lore raw

Re: [PATCH 10/13] media: v4l2-subdev: Add debug prints to v4l2_subdev_collect_streams()

Hi Laurent,

Quoted 27 lines · Laurent Pinchart

Hi Sakari,

Thank you for the patch.

Quoted 1 line · Sakari Ailus

Print streams found by v4l2_subdev_collect_streams() at debug level.

Explaining why you found this useful could help.

Quoted 10 lines
---
 drivers/media/v4l2-core/v4l2-subdev.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/media/v4l2-core/v4l2-subdev.c +0 -0
index 932fca795d4a..c549a462dac7 100644

drivers/media/v4l2-core/v4l2-subdev.c static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd, · line 2223

Quoted 5 lines
drivers/media/v4l2-core/v4l2-subdev.c +2 -0
@@ -2219,6 +2219,8 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
 		*found_streams = BIT_ULL(0);
 		*enabled_streams =
 			(sd->enabled_pads & BIT_ULL(pad)) ? BIT_ULL(0) : 0;
+		dev_dbg(sd->dev,
+			"collect_streams: sub-device does not support streams\n");

You print the subdev name below, I would print it here too. Maybe

Quoted 2 lines
			"collect_streams: '%s':%u: streams not supported\n",
			sd->entity.name, pad);

Makes sense.

Quoted 16 lines

drivers/media/v4l2-core/v4l2-subdev.c static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd, · line 2241

Quoted 8 lines
drivers/media/v4l2-core/v4l2-subdev.c +0 -0
 		return;
 	}
 
@@ -2237,6 +2239,9 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
 		if (src_cfg->enabled)
 			*enabled_streams |= BIT_ULL(src_cfg->stream);
 	}

Please add a blank line here.

drivers/media/v4l2-core/v4l2-subdev.c static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd, · line 2243

Quoted 2 lines
drivers/media/v4l2-core/v4l2-subdev.c +2 -0
+	dev_dbg(sd->dev,
+		"collect_streams: %s:%u found %#llx enabled %#llx\n",
		"collect_streams: '%s':%u: found %#llx, enabled: %#llx\n",
Quoted 5 lines

or possibly

		"%s: '%s':%u: found %#llx, enabled: %#llx\n", __func__,

(same above).

I prefer collect_streams: only as it's shorter, there's no possibility for confusion in any case.

Quoted 2 lines

Thank you!

Quoted 6 lines
Quoted 5 lines
drivers/media/v4l2-core/v4l2-subdev.c +1 -0
+		sd->entity.name, pad, *found_streams, *enabled_streams);
 }
 
 static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
Signature · 3 lines
Regards,

Sakari Ailus
author maintainer
#48 lore raw

PATCH 11/13 media: v4l2-subdev: Introduce v4l2_subdev_find_route()

In reply to Sakari Ailus

v4l2_subdev_find_route() is like v4l2_subdev_routing_find_opposite_end(), with the difference that it's more flexible: it can look up only active routes and can find multiple routes, too.

v4l2_subdev_find_route() is intended to replace
v4l2_subdev_routing_find_opposite_end().

---
 drivers/media/v4l2-core/v4l2-subdev.c | 56 ++++++++++++++++++---------
 include/media/v4l2-subdev.h           | 19 +++++++++
 2 files changed, 56 insertions(+), 19 deletions(-)
drivers/media/v4l2-core/v4l2-subdev.c +37 -19
index c549a462dac7..13d6e96daf3a 100644
@@ -1996,34 +1996,52 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_set_routing_with_fmt);
 
-int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
-					  u32 pad, u32 stream, u32 *other_pad,
-					  u32 *other_stream)
+struct v4l2_subdev_route *
+v4l2_subdev_find_route(const struct v4l2_subdev_krouting *routing,
+		       u32 pad, u32 stream, bool active, unsigned int index)
 {
 	unsigned int i;
 
 	for (i = 0; i < routing->num_routes; ++i) {
 		struct v4l2_subdev_route *route = &routing->routes[i];
 
-		if (route->source_pad == pad &&
-		    route->source_stream == stream) {
-			if (other_pad)
-				*other_pad = route->sink_pad;
-			if (other_stream)
-				*other_stream = route->sink_stream;
-			return 0;
-		}
+		if (active && !(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
+			continue;
 
-		if (route->sink_pad == pad && route->sink_stream == stream) {
-			if (other_pad)
-				*other_pad = route->source_pad;
-			if (other_stream)
-				*other_stream = route->source_stream;
-			return 0;
-		}
+		if ((route->source_pad != pad ||
+		     route->source_stream != stream) &&
+		    (route->sink_pad != pad || route->sink_stream != stream))
+			continue;
+
+		if (index--)
+			continue;
+
+		return route;
 	}
 
-	return -EINVAL;
+	return ERR_PTR(-ENOENT);
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_find_route);
+
+int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
+					  u32 pad, u32 stream, u32 *other_pad,
+					  u32 *other_stream)
+{
+	struct v4l2_subdev_route *route;
+
+	route = v4l2_subdev_find_route(routing, pad, stream, false, 0);
+	if (IS_ERR(route))
+		return PTR_ERR(route);
+
+	bool is_source = route->source_pad == pad;
+
+	if (other_pad)
+		*other_pad = is_source ? route->sink_pad : route->source_pad;
+	if (other_stream)
+		*other_stream = is_source ?
+			route->sink_stream : route->source_stream;
+
+	return 0;
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_routing_find_opposite_end);
 
include/media/v4l2-subdev.h +19 -0
index deab128a4779..9ed8600ba3d4 100644
@@ -1547,6 +1547,23 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
 				     const struct v4l2_subdev_krouting *routing,
 				     const struct v4l2_mbus_framefmt *fmt);
 
+/**
+ * v4l2_subdev_find_route() - Find routes from a (pad, stream) pair
+ * @routing: routing used to find the opposite side
+ * @pad: pad id
+ * @stream: stream id
+ * @active: set to true for looking up only active routes
+ * @index: for accessing more than one route from the pad
+ *
+ * Find a route from the routing table where one end has (pad, stream) pair
+ * matching @pad and @stream.
+ *
+ * Returns the route on success or -ENOENT if no matching route is found.
+ */
+struct v4l2_subdev_route *
+v4l2_subdev_find_route(const struct v4l2_subdev_krouting *routing,
+		       u32 pad, u32 stream, bool active, unsigned int index);
+
 /**
  * v4l2_subdev_routing_find_opposite_end() - Find the opposite stream
  * @routing: routing used to find the opposite side
@@ -1555,6 +1572,8 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
  * @other_pad: pointer used to return the opposite pad
  * @other_stream: pointer used to return the opposite stream
  *
+ * Prefer v4l2_subdev_find_route() over v4l2_subdev_routing_find_opposite_end().
+ *
  * This function uses the routing table to find the pad + stream which is
  * opposite the given pad + stream.
  *
Signature · 1 line
2.39.5
#49 lore raw

Re: [PATCH 11/13] media: v4l2-subdev: Introduce v4l2_subdev_find_route()

Hi Sakari

Quoted 6 lines · Sakari Ailus

v4l2_subdev_find_route() is like v4l2_subdev_routing_find_opposite_end(), with the difference that it's more flexible: it can look up only active routes and can find multiple routes, too.

v4l2_subdev_find_route() is intended to replace
v4l2_subdev_routing_find_opposite_end().

To me this feels like v4l2_subdev_find_route() could be used to implement more helpers like v4l2_subdev_routing_find_opposite_end() for drivers instead of going the other way around.

let's see what the use cases are

Quoted 27 lines
---
 drivers/media/v4l2-core/v4l2-subdev.c | 56 ++++++++++++++++++---------
 include/media/v4l2-subdev.h           | 19 +++++++++
 2 files changed, 56 insertions(+), 19 deletions(-)
drivers/media/v4l2-core/v4l2-subdev.c +3 -3
index c549a462dac7..13d6e96daf3a 100644
@@ -1996,34 +1996,52 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_set_routing_with_fmt);
 
-int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
-					  u32 pad, u32 stream, u32 *other_pad,
-					  u32 *other_stream)
+struct v4l2_subdev_route *
+v4l2_subdev_find_route(const struct v4l2_subdev_krouting *routing,
+		       u32 pad, u32 stream, bool active, unsigned int index)
 {
 	unsigned int i;
 
 	for (i = 0; i < routing->num_routes; ++i) {
 		struct v4l2_subdev_route *route = &routing->routes[i];
 

drivers/media/v4l2-core/v4l2-subdev.c int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 2009

Quoted 10 lines
drivers/media/v4l2-core/v4l2-subdev.c +2 -8
-		if (route->source_pad == pad &&
-		    route->source_stream == stream) {
-			if (other_pad)
-				*other_pad = route->sink_pad;
-			if (other_stream)
-				*other_stream = route->sink_stream;
-			return 0;
-		}
+		if (active && !(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
+			continue;

I know currently v4l2_subdev_routing_find_opposite_end() does return any route that matches the provided 'pad' and 'stream' included non-active ones, but I wonder if this is desirable. What is the use case for enumerating a non-active route between two pads ?

(it is also my impression that all drivers that use v4l2_subdev_routing_find_opposite_end() assume the route is active)

Also I wonder if the usage of V4L2_SUBDEV_ROUTE_FL_ACTIVE is clearly defined, or, in other words, what is the use case for userspace to create non-active routes, given that any new VIDIOC_SUBDEV_S_ROUTING will anyway re-create the routing table (that's a different question, on the ioctl definition and not on this change though)

Quoted 51 lines
drivers/media/v4l2-core/v4l2-subdev.c +32 -8
-		if (route->sink_pad == pad && route->sink_stream == stream) {
-			if (other_pad)
-				*other_pad = route->source_pad;
-			if (other_stream)
-				*other_stream = route->source_stream;
-			return 0;
-		}
+		if ((route->source_pad != pad ||
+		     route->source_stream != stream) &&
+		    (route->sink_pad != pad || route->sink_stream != stream))
+			continue;
+
+		if (index--)
+			continue;
+
+		return route;
 	}
 
-	return -EINVAL;
+	return ERR_PTR(-ENOENT);
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_find_route);
+
+int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
+					  u32 pad, u32 stream, u32 *other_pad,
+					  u32 *other_stream)
+{
+	struct v4l2_subdev_route *route;
+
+	route = v4l2_subdev_find_route(routing, pad, stream, false, 0);
+	if (IS_ERR(route))
+		return PTR_ERR(route);
+
+	bool is_source = route->source_pad == pad;
+
+	if (other_pad)
+		*other_pad = is_source ? route->sink_pad : route->source_pad;
+	if (other_stream)
+		*other_stream = is_source ?
+			route->sink_stream : route->source_stream;
+
+	return 0;
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_routing_find_opposite_end);
 
include/media/v4l2-subdev.h +0 -0
index deab128a4779..9ed8600ba3d4 100644

include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1551

Quoted 5 lines
include/media/v4l2-subdev.h +2 -0
@@ -1547,6 +1547,23 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
 				     const struct v4l2_subdev_krouting *routing,
 				     const struct v4l2_mbus_framefmt *fmt);
 
+/**
+ * v4l2_subdev_find_route() - Find routes from a (pad, stream) pair

from or for ?

include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1552

Quoted 1 line
include/media/v4l2-subdev.h +1 -0
+ * @routing: routing used to find the opposite side

I would not say "opposite side" but rather

@routing: routing table used to enumerate routes

include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1556

Quoted 4 lines
include/media/v4l2-subdev.h +4 -0
+ * @pad: pad id
+ * @stream: stream id
+ * @active: set to true for looking up only active routes
+ * @index: for accessing more than one route from the pad

I understand this but maybe

@index: route index for enumerating multiple routes
?

include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1559

Quoted 3 lines
include/media/v4l2-subdev.h +3 -0
+ *
+ * Find a route from the routing table where one end has (pad, stream) pair
+ * matching @pad and @stream.

* If multiple routes in @routing match @pad and @stream, return
* the @index one.
*
* Set @active to true to only enumerate active routes.

include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1561

Quoted 2 lines
include/media/v4l2-subdev.h +2 -0
+ *
+ * Returns the route on success or -ENOENT if no matching route is found.

I see other functions documentation using

* Return:

is this a kernel-doc thing ?

Quoted 9 lines
include/media/v4l2-subdev.h +5 -0
+ */
+struct v4l2_subdev_route *
+v4l2_subdev_find_route(const struct v4l2_subdev_krouting *routing,
+		       u32 pad, u32 stream, bool active, unsigned int index);
+
 /**
  * v4l2_subdev_routing_find_opposite_end() - Find the opposite stream
  * @routing: routing used to find the opposite side

include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1576

Quoted 5 lines
include/media/v4l2-subdev.h +2 -0
@@ -1555,6 +1572,8 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
  * @other_pad: pointer used to return the opposite pad
  * @other_stream: pointer used to return the opposite stream
  *
+ * Prefer v4l2_subdev_find_route() over v4l2_subdev_routing_find_opposite_end().
+ *

As said, I'm not sure if that's preferred or we should rather create more helpers using v4l2_subdev_find_route() internally. Time will tell I guess ?

Thanks
j

Quoted 7 lines
include/media/v4l2-subdev.h +0 -1
  * This function uses the routing table to find the pad + stream which is
  * opposite the given pad + stream.
  *
--

2.39.5

author maintainer
#50 lore raw

Re: [PATCH 11/13] media: v4l2-subdev: Introduce v4l2_subdev_find_route()

Hi Jacopo,

Thanks for the review.

Quoted 43 lines · Jacopo Mondi

Hi Sakari

Quoted 6 lines · Sakari Ailus

v4l2_subdev_find_route() is like v4l2_subdev_routing_find_opposite_end(), with the difference that it's more flexible: it can look up only active routes and can find multiple routes, too.

v4l2_subdev_find_route() is intended to replace
v4l2_subdev_routing_find_opposite_end().

To me this feels like v4l2_subdev_find_route() could be used to implement more helpers like v4l2_subdev_routing_find_opposite_end() for drivers instead of going the other way around.

let's see what the use cases are

Quoted 27 lines
---
 drivers/media/v4l2-core/v4l2-subdev.c | 56 ++++++++++++++++++---------
 include/media/v4l2-subdev.h           | 19 +++++++++
 2 files changed, 56 insertions(+), 19 deletions(-)
drivers/media/v4l2-core/v4l2-subdev.c +3 -3
index c549a462dac7..13d6e96daf3a 100644
@@ -1996,34 +1996,52 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_set_routing_with_fmt);
 
-int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
-					  u32 pad, u32 stream, u32 *other_pad,
-					  u32 *other_stream)
+struct v4l2_subdev_route *
+v4l2_subdev_find_route(const struct v4l2_subdev_krouting *routing,
+		       u32 pad, u32 stream, bool active, unsigned int index)
 {
 	unsigned int i;
 
 	for (i = 0; i < routing->num_routes; ++i) {
 		struct v4l2_subdev_route *route = &routing->routes[i];
 
Quoted 15 lines

drivers/media/v4l2-core/v4l2-subdev.c int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 2009

Quoted 10 lines
drivers/media/v4l2-core/v4l2-subdev.c +2 -8
-		if (route->source_pad == pad &&
-		    route->source_stream == stream) {
-			if (other_pad)
-				*other_pad = route->sink_pad;
-			if (other_stream)
-				*other_stream = route->sink_stream;
-			return 0;
-		}
+		if (active && !(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
+			continue;

I know currently v4l2_subdev_routing_find_opposite_end() does return any route that matches the provided 'pad' and 'stream' included non-active ones, but I wonder if this is desirable. What is the use case for enumerating a non-active route between two pads ?

Good question. v4l2_subdev_routing_find_opposite_end() nevertheless returns them. And the caller won't get the route for checking the state either.

Quoted 9 lines

(it is also my impression that all drivers that use v4l2_subdev_routing_find_opposite_end() assume the route is active)

Also I wonder if the usage of V4L2_SUBDEV_ROUTE_FL_ACTIVE is clearly defined, or, in other words, what is the use case for userspace to create non-active routes, given that any new VIDIOC_SUBDEV_S_ROUTING will anyway re-create the routing table (that's a different question, on the ioctl definition and not on this change though)

I think it is. Please review the UAPI documentation in the metadata series. :-)

Quoted 55 lines
Quoted 55 lines
drivers/media/v4l2-core/v4l2-subdev.c +32 -8
-		if (route->sink_pad == pad && route->sink_stream == stream) {
-			if (other_pad)
-				*other_pad = route->source_pad;
-			if (other_stream)
-				*other_stream = route->source_stream;
-			return 0;
-		}
+		if ((route->source_pad != pad ||
+		     route->source_stream != stream) &&
+		    (route->sink_pad != pad || route->sink_stream != stream))
+			continue;
+
+		if (index--)
+			continue;
+
+		return route;
 	}
 
-	return -EINVAL;
+	return ERR_PTR(-ENOENT);
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_find_route);
+
+int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
+					  u32 pad, u32 stream, u32 *other_pad,
+					  u32 *other_stream)
+{
+	struct v4l2_subdev_route *route;
+
+	route = v4l2_subdev_find_route(routing, pad, stream, false, 0);
+	if (IS_ERR(route))
+		return PTR_ERR(route);
+
+	bool is_source = route->source_pad == pad;
+
+	if (other_pad)
+		*other_pad = is_source ? route->sink_pad : route->source_pad;
+	if (other_stream)
+		*other_stream = is_source ?
+			route->sink_stream : route->source_stream;
+
+	return 0;
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_routing_find_opposite_end);
 
include/media/v4l2-subdev.h +0 -0
index deab128a4779..9ed8600ba3d4 100644
@@ -1547,6 +1547,23 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
 				     const struct v4l2_subdev_krouting *routing,
 				     const struct v4l2_mbus_framefmt *fmt);
 
Quoted 4 lines

include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1551

Quoted 2 lines
include/media/v4l2-subdev.h +2 -0
+/**
+ * v4l2_subdev_find_route() - Find routes from a (pad, stream) pair

from or for ?

From or to. I'll fix this in the next version.

Quoted 6 lines

include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1552

Quoted 2 lines
include/media/v4l2-subdev.h +1 -0
+ * @routing: routing used to find the opposite side

I would not say "opposite side" but rather

@routing: routing table used to enumerate routes

How about simply "the routing table"?

Quoted 10 lines

include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1556

Quoted 5 lines
include/media/v4l2-subdev.h +4 -0
+ * @pad: pad id
+ * @stream: stream id
+ * @active: set to true for looking up only active routes
+ * @index: for accessing more than one route from the pad

I understand this but maybe

@index: route index for enumerating multiple routes
?

Sounds good.

Quoted 10 lines

include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1559

Quoted 4 lines
include/media/v4l2-subdev.h +3 -0
+ *
+ * Find a route from the routing table where one end has (pad, stream) pair
+ * matching @pad and @stream.

* If multiple routes in @routing match @pad and @stream, return
* the @index one.
*
* Set @active to true to only enumerate active routes.

Quoted 8 lines

include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1561

Quoted 2 lines
include/media/v4l2-subdev.h +2 -0
+ *
+ * Returns the route on success or -ENOENT if no matching route is found.

I see other functions documentation using

* Return:

is this a kernel-doc thing ?

Yes, makes sense.

Quoted 11 lines
Quoted 11 lines
include/media/v4l2-subdev.h +5 -0
+ */
+struct v4l2_subdev_route *
+v4l2_subdev_find_route(const struct v4l2_subdev_krouting *routing,
+		       u32 pad, u32 stream, bool active, unsigned int index);
+
 /**
  * v4l2_subdev_routing_find_opposite_end() - Find the opposite stream
  * @routing: routing used to find the opposite side
@@ -1555,6 +1572,8 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
  * @other_pad: pointer used to return the opposite pad
Quoted 8 lines

include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1576

Quoted 4 lines
include/media/v4l2-subdev.h +2 -0
  * @other_stream: pointer used to return the opposite stream
  *
+ * Prefer v4l2_subdev_find_route() over v4l2_subdev_routing_find_opposite_end().
+ *

As said, I'm not sure if that's preferred or we should rather create more helpers using v4l2_subdev_find_route() internally. Time will tell I guess ?

I agree.

The benefit of the older function was that it returns information that doesn't need a lock for accessing it.

Signature · 3 lines
Regards,

Sakari Ailus
#51 lore raw

Re: [PATCH 11/13] media: v4l2-subdev: Introduce v4l2_subdev_find_route()

Quoted 57 lines · Sakari Ailus
Quoted 40 lines · Jacopo Mondi
Quoted 6 lines · Sakari Ailus

v4l2_subdev_find_route() is like v4l2_subdev_routing_find_opposite_end(), with the difference that it's more flexible: it can look up only active routes and can find multiple routes, too.

v4l2_subdev_find_route() is intended to replace
v4l2_subdev_routing_find_opposite_end().

To me this feels like v4l2_subdev_find_route() could be used to implement more helpers like v4l2_subdev_routing_find_opposite_end() for drivers instead of going the other way around.

let's see what the use cases are

Quoted 26 lines
---
 drivers/media/v4l2-core/v4l2-subdev.c | 56 ++++++++++++++++++---------
 include/media/v4l2-subdev.h           | 19 +++++++++
 2 files changed, 56 insertions(+), 19 deletions(-)
drivers/media/v4l2-core/v4l2-subdev.c +3 -3
index c549a462dac7..13d6e96daf3a 100644
@@ -1996,34 +1996,52 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_set_routing_with_fmt);
 
-int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
-					  u32 pad, u32 stream, u32 *other_pad,
-					  u32 *other_stream)
+struct v4l2_subdev_route *
+v4l2_subdev_find_route(const struct v4l2_subdev_krouting *routing,
+		       u32 pad, u32 stream, bool active, unsigned int index)
 {
 	unsigned int i;
 
 	for (i = 0; i < routing->num_routes; ++i) {
 		struct v4l2_subdev_route *route = &routing->routes[i];
 
Quoted 15 lines

drivers/media/v4l2-core/v4l2-subdev.c int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 2009

Quoted 10 lines
drivers/media/v4l2-core/v4l2-subdev.c +2 -8
-		if (route->source_pad == pad &&
-		    route->source_stream == stream) {
-			if (other_pad)
-				*other_pad = route->sink_pad;
-			if (other_stream)
-				*other_stream = route->sink_stream;
-			return 0;
-		}
+		if (active && !(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
+			continue;

I know currently v4l2_subdev_routing_find_opposite_end() does return any route that matches the provided 'pad' and 'stream' included non-active ones, but I wonder if this is desirable. What is the use case for enumerating a non-active route between two pads ?

Quoted 2 lines

Good question. v4l2_subdev_routing_find_opposite_end() nevertheless returns them. And the caller won't get the route for checking the state either.

I'm tempted to check the callers of the function and change the behaviour to only return active routes.

Quoted 11 lines
Quoted 8 lines

(it is also my impression that all drivers that use v4l2_subdev_routing_find_opposite_end() assume the route is active)

Also I wonder if the usage of V4L2_SUBDEV_ROUTE_FL_ACTIVE is clearly defined, or, in other words, what is the use case for userspace to create non-active routes, given that any new VIDIOC_SUBDEV_S_ROUTING will anyway re-create the routing table (that's a different question, on the ioctl definition and not on this change though)

I think it is. Please review the UAPI documentation in the metadata series. :-)

Section "Device types and routing setup" in Documentation/userspace-api/media/v4l/dev-subdev.rst is the best documentation we have at the moment.

We essentially have two models for internal routing. In the first model, which I'll nickname the crossbar switch model, a large number of routes are possible (up to any input to any output types of scenarios). In this case, routes are created by userspace, and all routes in the routing table are expected to be active. Any inactive route provided by userspace would be dropped by the driver and not be included in the routing table.

The second model covers devices such as camera sensors, where a small fixed set of routes are hardcoded. The routing table is fixed, an some routes (the ones not marked with the IMMUTABLE flag) can be enabled/disabled by userspace using the V4L2_SUBDEV_ROUTE_FL_ACTIVE. This allows userspace to enumerate the available routes.

Documentation/userspace-api/media/v4l/dev-subdev.rst should document more clearly that we do not allow any hybrid behaviour at the moment.

Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
drivers/media/v4l2-core/v4l2-subdev.c +22 -8
-		if (route->sink_pad == pad && route->sink_stream == stream) {
-			if (other_pad)
-				*other_pad = route->source_pad;
-			if (other_stream)
-				*other_stream = route->source_stream;
-			return 0;
-		}
+		if ((route->source_pad != pad ||
+		     route->source_stream != stream) &&
+		    (route->sink_pad != pad || route->sink_stream != stream))
+			continue;
+
+		if (index--)
+			continue;
+
+		return route;
 	}
 
-	return -EINVAL;
+	return ERR_PTR(-ENOENT);
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_find_route);
+
+int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
+					  u32 pad, u32 stream, u32 *other_pad,
+					  u32 *other_stream)
+{
+	struct v4l2_subdev_route *route;
+
+	route = v4l2_subdev_find_route(routing, pad, stream, false, 0);
+	if (IS_ERR(route))
+		return PTR_ERR(route);
Quoted 8 lines
Quoted 8 lines

drivers/media/v4l2-core/v4l2-subdev.c int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 2042

Quoted 8 lines
drivers/media/v4l2-core/v4l2-subdev.c +8 -0
+
+	bool is_source = route->source_pad == pad;
+
+	if (other_pad)
+		*other_pad = is_source ? route->sink_pad : route->source_pad;
+	if (other_stream)
+		*other_stream = is_source ?
+			route->sink_stream : route->source_stream;

Having to do this is_source dance makes v4l2_subdev_find_route() annoying to use. It may be fine when using the function to implement other helpers, but I wouldn't like to see this being done in drivers. Maybe we can avoid exporting v4l2_subdev_find_route() for now to ensure that, the only user in this series is in v4l2-mc.c.

Quoted 35 lines
Quoted 13 lines
Quoted 13 lines
drivers/media/v4l2-core/v4l2-subdev.c +2 -0
+
+	return 0;
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_routing_find_opposite_end);
 
include/media/v4l2-subdev.h +0 -0
index deab128a4779..9ed8600ba3d4 100644
@@ -1547,6 +1547,23 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
 				     const struct v4l2_subdev_krouting *routing,
 				     const struct v4l2_mbus_framefmt *fmt);
 
Quoted 4 lines

include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1551

Quoted 2 lines
include/media/v4l2-subdev.h +2 -0
+/**
+ * v4l2_subdev_find_route() - Find routes from a (pad, stream) pair

from or for ?

From or to. I'll fix this in the next version.

Quoted 6 lines

include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1552

Quoted 2 lines
include/media/v4l2-subdev.h +1 -0
+ * @routing: routing used to find the opposite side

I would not say "opposite side" but rather

@routing: routing table used to enumerate routes

How about simply "the routing table"?

Quoted 6 lines
Quoted 5 lines
include/media/v4l2-subdev.h +4 -0
+ * @pad: pad id
+ * @stream: stream id
+ * @active: set to true for looking up only active routes
+ * @index: for accessing more than one route from the pad
Quoted 6 lines
Quoted 4 lines

I understand this but maybe

@index: route index for enumerating multiple routes
?

Sounds good.

I'm curious to know how this parameter will be used. In the only user (in patch 12/13), it is hardcoded to 0. I'm not sure indexing routes will be very useful for drivers.

Quoted 39 lines
Quoted 9 lines

include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1559

Quoted 3 lines
include/media/v4l2-subdev.h +3 -0
+ *
+ * Find a route from the routing table where one end has (pad, stream) pair
+ * matching @pad and @stream.

* If multiple routes in @routing match @pad and @stream, return
* the @index one.
*
* Set @active to true to only enumerate active routes.

Quoted 8 lines

include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1561

Quoted 2 lines
include/media/v4l2-subdev.h +2 -0
+ *
+ * Returns the route on success or -ENOENT if no matching route is found.

I see other functions documentation using

* Return:

is this a kernel-doc thing ?

Yes, makes sense.

Quoted 10 lines
Quoted 10 lines
include/media/v4l2-subdev.h +5 -0
+ */
+struct v4l2_subdev_route *
+v4l2_subdev_find_route(const struct v4l2_subdev_krouting *routing,
+		       u32 pad, u32 stream, bool active, unsigned int index);
+
 /**
  * v4l2_subdev_routing_find_opposite_end() - Find the opposite stream
  * @routing: routing used to find the opposite side
@@ -1555,6 +1572,8 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
  * @other_pad: pointer used to return the opposite pad
Quoted 8 lines

include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1576

Quoted 4 lines
include/media/v4l2-subdev.h +2 -0
  * @other_stream: pointer used to return the opposite stream
  *
+ * Prefer v4l2_subdev_find_route() over v4l2_subdev_routing_find_opposite_end().
+ *

As said, I'm not sure if that's preferred or we should rather create more helpers using v4l2_subdev_find_route() internally. Time will tell I guess ?

Quoted 4 lines

I agree.

The benefit of the older function was that it returns information that doesn't need a lock for accessing it.

I'm not sure to follow you here. v4l2_subdev_find_route() has the same locking requirements as v4l2_subdev_routing_find_opposite_end().

Given that v4l2_subdev_find_route() has a single user, and that the only difference in that user compared to v4l2_subdev_routing_find_opposite_end() is that only active routes are considered, I would prefer modifying v4l2_subdev_routing_find_opposite_end() to ignore inactive routes after checking that no caller would break.

Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#52 lore raw

Re: [PATCH 11/13] media: v4l2-subdev: Introduce v4l2_subdev_find_route()

Hi Laurent,

Quoted 58 lines · Laurent Pinchart
Quoted 57 lines · Sakari Ailus
Quoted 55 lines · Jacopo Mondi
Quoted 6 lines · Sakari Ailus

v4l2_subdev_find_route() is like v4l2_subdev_routing_find_opposite_end(), with the difference that it's more flexible: it can look up only active routes and can find multiple routes, too.

v4l2_subdev_find_route() is intended to replace
v4l2_subdev_routing_find_opposite_end().

To me this feels like v4l2_subdev_find_route() could be used to implement more helpers like v4l2_subdev_routing_find_opposite_end() for drivers instead of going the other way around.

let's see what the use cases are

Quoted 26 lines
---
 drivers/media/v4l2-core/v4l2-subdev.c | 56 ++++++++++++++++++---------
 include/media/v4l2-subdev.h           | 19 +++++++++
 2 files changed, 56 insertions(+), 19 deletions(-)
drivers/media/v4l2-core/v4l2-subdev.c +3 -3
index c549a462dac7..13d6e96daf3a 100644
@@ -1996,34 +1996,52 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_set_routing_with_fmt);
 
-int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
-					  u32 pad, u32 stream, u32 *other_pad,
-					  u32 *other_stream)
+struct v4l2_subdev_route *
+v4l2_subdev_find_route(const struct v4l2_subdev_krouting *routing,
+		       u32 pad, u32 stream, bool active, unsigned int index)
 {
 	unsigned int i;
 
 	for (i = 0; i < routing->num_routes; ++i) {
 		struct v4l2_subdev_route *route = &routing->routes[i];
 

drivers/media/v4l2-core/v4l2-subdev.c int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 2009

Quoted 10 lines
drivers/media/v4l2-core/v4l2-subdev.c +2 -8
-		if (route->source_pad == pad &&
-		    route->source_stream == stream) {
-			if (other_pad)
-				*other_pad = route->sink_pad;
-			if (other_stream)
-				*other_stream = route->sink_stream;
-			return 0;
-		}
+		if (active && !(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
+			continue;

I know currently v4l2_subdev_routing_find_opposite_end() does return any route that matches the provided 'pad' and 'stream' included non-active ones, but I wonder if this is desirable. What is the use case for enumerating a non-active route between two pads ?

Quoted 5 lines
Quoted 2 lines

Good question. v4l2_subdev_routing_find_opposite_end() nevertheless returns them. And the caller won't get the route for checking the state either.

I'm tempted to check the callers of the function and change the behaviour to only return active routes.

I can prepend a patch to do the change there so it won't happen here.

Quoted 25 lines
Quoted 4 lines
Quoted 4 lines

(it is also my impression that all drivers that use v4l2_subdev_routing_find_opposite_end() assume the route is active)

Quoted 8 lines
Quoted 5 lines

Also I wonder if the usage of V4L2_SUBDEV_ROUTE_FL_ACTIVE is clearly defined, or, in other words, what is the use case for userspace to create non-active routes, given that any new VIDIOC_SUBDEV_S_ROUTING will anyway re-create the routing table (that's a different question, on the ioctl definition and not on this change though)

I think it is. Please review the UAPI documentation in the metadata series. :-)

Section "Device types and routing setup" in Documentation/userspace-api/media/v4l/dev-subdev.rst is the best documentation we have at the moment.

We essentially have two models for internal routing. In the first model, which I'll nickname the crossbar switch model, a large number of routes are possible (up to any input to any output types of scenarios). In this case, routes are created by userspace, and all routes in the routing table are expected to be active. Any inactive route provided by userspace would be dropped by the driver and not be included in the routing table.

Quoted 8 lines

The second model covers devices such as camera sensors, where a small fixed set of routes are hardcoded. The routing table is fixed, an some routes (the ones not marked with the IMMUTABLE flag) can be enabled/disabled by userspace using the V4L2_SUBDEV_ROUTE_FL_ACTIVE. This allows userspace to enumerate the available routes.

Documentation/userspace-api/media/v4l/dev-subdev.rst should document more clearly that we do not allow any hybrid behaviour at the moment.

I agree.

Quoted 40 lines
Quoted 40 lines
Quoted 40 lines
Quoted 40 lines
drivers/media/v4l2-core/v4l2-subdev.c +28 -8
-		if (route->sink_pad == pad && route->sink_stream == stream) {
-			if (other_pad)
-				*other_pad = route->source_pad;
-			if (other_stream)
-				*other_stream = route->source_stream;
-			return 0;
-		}
+		if ((route->source_pad != pad ||
+		     route->source_stream != stream) &&
+		    (route->sink_pad != pad || route->sink_stream != stream))
+			continue;
+
+		if (index--)
+			continue;
+
+		return route;
 	}
 
-	return -EINVAL;
+	return ERR_PTR(-ENOENT);
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_find_route);
+
+int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
+					  u32 pad, u32 stream, u32 *other_pad,
+					  u32 *other_stream)
+{
+	struct v4l2_subdev_route *route;
+
+	route = v4l2_subdev_find_route(routing, pad, stream, false, 0);
+	if (IS_ERR(route))
+		return PTR_ERR(route);
+
+	bool is_source = route->source_pad == pad;
+
+	if (other_pad)
+		*other_pad = is_source ? route->sink_pad : route->source_pad;
+	if (other_stream)
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines

drivers/media/v4l2-core/v4l2-subdev.c int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 2042

Quoted 2 lines
drivers/media/v4l2-core/v4l2-subdev.c +2 -0
+		*other_stream = is_source ?
+			route->sink_stream : route->source_stream;

Having to do this is_source dance makes v4l2_subdev_find_route() annoying to use. It may be fine when using the function to implement other helpers, but I wouldn't like to see this being done in drivers. Maybe we can avoid exporting v4l2_subdev_find_route() for now to ensure that, the only user in this series is in v4l2-mc.c.

Sounds reasonable.

Quoted 38 lines
Quoted 38 lines
Quoted 14 lines
Quoted 14 lines
drivers/media/v4l2-core/v4l2-subdev.c +2 -0
+
+	return 0;
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_routing_find_opposite_end);
 
include/media/v4l2-subdev.h +0 -0
index deab128a4779..9ed8600ba3d4 100644
@@ -1547,6 +1547,23 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
 				     const struct v4l2_subdev_krouting *routing,
 				     const struct v4l2_mbus_framefmt *fmt);
 
Quoted 4 lines

include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1551

Quoted 2 lines
include/media/v4l2-subdev.h +2 -0
+/**
+ * v4l2_subdev_find_route() - Find routes from a (pad, stream) pair

from or for ?

From or to. I'll fix this in the next version.

Quoted 6 lines

include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1552

Quoted 2 lines
include/media/v4l2-subdev.h +1 -0
+ * @routing: routing used to find the opposite side

I would not say "opposite side" but rather

@routing: routing table used to enumerate routes

How about simply "the routing table"?

Quoted 8 lines

include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1556

Quoted 5 lines
include/media/v4l2-subdev.h +4 -0
+ * @pad: pad id
+ * @stream: stream id
+ * @active: set to true for looking up only active routes
+ * @index: for accessing more than one route from the pad

I understand this but maybe

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

@index: route index for enumerating multiple routes
?

Sounds good.

I'm curious to know how this parameter will be used. In the only user (in patch 12/13), it is hardcoded to 0. I'm not sure indexing routes will be very useful for drivers.

Looks like it'll be needed by v4l2_subdev_set_streams_enabled(), as a result on discussion related to it.

Quoted 40 lines
Quoted 40 lines
Quoted 10 lines

include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1559

Quoted 4 lines
include/media/v4l2-subdev.h +3 -0
+ *
+ * Find a route from the routing table where one end has (pad, stream) pair
+ * matching @pad and @stream.

* If multiple routes in @routing match @pad and @stream, return
* the @index one.
*
* Set @active to true to only enumerate active routes.

Quoted 8 lines

include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1561

Quoted 2 lines
include/media/v4l2-subdev.h +2 -0
+ *
+ * Returns the route on success or -ENOENT if no matching route is found.

I see other functions documentation using

* Return:

is this a kernel-doc thing ?

Yes, makes sense.

Quoted 18 lines
Quoted 9 lines
include/media/v4l2-subdev.h +5 -0
+ */
+struct v4l2_subdev_route *
+v4l2_subdev_find_route(const struct v4l2_subdev_krouting *routing,
+		       u32 pad, u32 stream, bool active, unsigned int index);
+
 /**
  * v4l2_subdev_routing_find_opposite_end() - Find the opposite stream
  * @routing: routing used to find the opposite side

include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1576

Quoted 5 lines
include/media/v4l2-subdev.h +2 -0
@@ -1555,6 +1572,8 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
  * @other_pad: pointer used to return the opposite pad
  * @other_stream: pointer used to return the opposite stream
  *
+ * Prefer v4l2_subdev_find_route() over v4l2_subdev_routing_find_opposite_end().
+ *

As said, I'm not sure if that's preferred or we should rather create more helpers using v4l2_subdev_find_route() internally. Time will tell I guess ?

Quoted 7 lines
Quoted 4 lines

I agree.

The benefit of the older function was that it returns information that doesn't need a lock for accessing it.

I'm not sure to follow you here. v4l2_subdev_find_route() has the same locking requirements as v4l2_subdev_routing_find_opposite_end().

The function itself does, but a lock is not required for accessing information returned by v4l2_subdev_routing_find_opposite_end() (unlike v4l2_subdev_find_route()).

Quoted 7 lines

Given that v4l2_subdev_find_route() has a single user, and that the only difference in that user compared to v4l2_subdev_routing_find_opposite_end() is that only active routes are considered, I would prefer modifying v4l2_subdev_routing_find_opposite_end() to ignore inactive routes after checking that no caller would break.

Signature · 3 lines
Regards,

Sakari Ailus
author maintainer
#53 lore raw

PATCH 12/13 media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

In reply to Sakari Ailus

v4l2_mc_pipeline_enabled() helps solving a problem known for long but lacked any sort of general solution: with multiple streams, when streaming is started on video nodes one by one, when should streaming be started in the source?

v4l2_mc_pipeline_enabled() traverses the pipeline towards the source, queries the streams generated by the source and traces them back to the video nodes.

---
 drivers/media/v4l2-core/v4l2-mc.c | 243 ++++++++++++++++++++++++++++++
 include/media/v4l2-mc.h           |  44 ++++++
 2 files changed, 287 insertions(+)
drivers/media/v4l2-core/v4l2-mc.c +243 -0
index 937d358697e1..1731088ad436 100644
@@ -612,3 +612,246 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
 	return ret;
 }
 EXPORT_SYMBOL_GPL(v4l2_pipeline_link_notify);
+
+static int
+__v4l2_mc_pipeline_enabled(struct v4l2_subdev_state *state,
+			   struct media_pad *src_pad, u64 __src_streams,
+			   struct media_pad **__sink_pad, u64 *__sink_streams)
+{
+	struct v4l2_subdev_route *route;
+	u64 src_streams = 0, sink_streams = 0;
+	bool has_sink_pad = false;
+	unsigned int sink_pad;
+
+	dev_dbg(state->sd->dev, "%s: source enabled, pad/streams %u/%#llx\n",
+		state->sd->entity.name, src_pad->index, __src_streams);
+	for_each_active_route(&state->routing, route) {
+		dev_dbg(state->sd->dev, "%s: %u/%u -> %u/%u, flags %x\n",
+			state->sd->entity.name,
+			route->sink_pad, route->sink_stream, route->source_pad,
+			route->source_stream, route->flags);
+		if (route->source_pad != src_pad->index)
+			continue;
+
+		if (!(BIT_ULL(route->source_stream) & __src_streams))
+			continue;
+
+		if (!has_sink_pad) {
+			has_sink_pad = true;
+			sink_pad = route->sink_pad;
+		}
+
+		if (route->sink_pad != sink_pad) {
+			dev_dbg(state->sd->dev,
+				"sink pads (%u vs. %u) differ\n",
+				route->sink_pad, sink_pad);
+			return -EMLINK;
+		}
+
+		sink_streams |= BIT_ULL(route->sink_stream);
+		src_streams |= BIT_ULL(route->source_stream);
+	}
+
+	*__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] : NULL;
+	*__sink_streams = sink_streams;
+
+	return 0;
+}
+
+static int v4l2_mc_downpath_enabled(struct media_pad *sink_pad,
+				    unsigned int sink_stream,
+				    bool (*func)(struct video_device *vdev),
+				    struct media_pad **__sink_pad,
+				    u64 *__sink_streams)
+{
+	struct v4l2_subdev_state *state;
+	struct v4l2_subdev_route *route;
+	struct v4l2_subdev *sd;
+	struct media_pad *source_pad, *tmp_pad;
+	u32 source_stream;
+
+	if (!is_media_entity_v4l2_subdev(sink_pad->entity))
+		return -ENXIO;
+
+	sd = media_entity_to_v4l2_subdev(sink_pad->entity);
+	dev_dbg(sd->dev, "path_enabled: found sub-device %s\n",
+		sd->entity.name);
+
+	state = v4l2_subdev_lock_and_get_active_state(sd);
+	route = v4l2_subdev_find_route(&state->routing, sink_pad->index,
+				       sink_stream, true, 0);
+	if (IS_ERR(route)) {
+		v4l2_subdev_unlock_state(state);
+		dev_dbg(sd->dev,
+			"path_enabled: can't find opposite route for %s:%u/%u",
+			sd->entity.name, sink_pad->index, sink_stream);
+		return 2;
+	}
+
+	source_pad = &sd->entity.pads[route->source_pad];
+	v4l2_subdev_unlock_state(state);
+
+	tmp_pad = sink_pad;
+	sink_pad = media_pad_remote_pad_unique(source_pad);
+	if (IS_ERR(sink_pad)) {
+		dev_dbg(sd->dev,
+			"path_enabled: can't find remote source for %s:%u\n",
+			source_pad->entity->name, source_pad->index);
+		return PTR_ERR(sink_pad);
+	}
+
+	if (is_media_entity_v4l2_video_device(sink_pad->entity)) {
+		struct video_device *vdev;
+
+		vdev = media_entity_to_video_device(sink_pad->entity);
+		if (!vdev)
+			return -ENXIO;
+
+		dev_dbg(vdev->dev_parent,
+			"path_enabled: found video device %s\n",
+			vdev->name);
+
+		if (!*__sink_pad) {
+			*__sink_pad = tmp_pad;
+			dev_dbg(sd->dev, "path_enabled: sink %u/%u\n",
+				tmp_pad->index, sink_stream);
+		} else if (tmp_pad != *__sink_pad) {
+			dev_dbg(sd->dev,
+				"path_enabled: pads %s/%u and %s/%u differ\n",
+				tmp_pad->entity->name, tmp_pad->index,
+				(*__sink_pad)->entity->name,
+				(*__sink_pad)->index);
+			return -EXDEV;
+		}
+
+		*__sink_streams |= BIT_ULL(sink_stream);
+
+		return func(vdev);
+	}
+
+	return v4l2_mc_downpath_enabled(sink_pad, source_stream, func,
+					__sink_pad, __sink_streams);
+}
+
+static int v4l2_mc_source_get_streams(struct v4l2_subdev *sd, unsigned int pad,
+				      u64 *__streams)
+{
+	struct v4l2_mbus_frame_desc desc;
+	u64 streams = 0;
+	int ret;
+
+	if (!__streams)
+		return -EINVAL;
+
+	ret = v4l2_subdev_call(sd, pad, get_frame_desc, pad, &desc);
+	if (ret == -ENOIOCTLCMD) {
+		*__streams = 1ULL;
+		return 0;
+	}
+	if (ret)
+		return ret;
+
+	for (unsigned int i = 0; i < desc.num_entries; i++) {
+		if (streams & BIT_ULL(desc.entry[i].stream))
+			return -EINVAL;
+
+		streams |= BIT_ULL(desc.entry[i].stream);
+	}
+
+	dev_dbg(sd->dev, "found streams %#llx based on streams %#llx\n",
+		*__streams, streams);
+	if (*__streams & ~streams)
+		return -EINVAL;
+
+	*__streams = streams;
+
+	return 0;
+}
+
+int v4l2_mc_pipeline_enabled(struct video_device *vdev,
+			     bool (*func)(struct video_device *vdev),
+			     struct media_pad **__sink_pad, u64 *__sink_streams)
+{
+	u64 sink_streams = 1U;
+	struct media_pad *src_pad;
+	u64 src_streams;
+	struct v4l2_subdev_state *state;
+	struct media_pad *sink_pad = vdev->entity.pads;
+	struct v4l2_subdev *sd = NULL;
+	bool streaming = true;
+	struct media_pad *tmp_pad;
+	u64 tmp_streams;
+	int ret;
+
+	if (!__sink_pad)
+		__sink_pad = &tmp_pad;
+	if (!__sink_streams)
+		__sink_streams = &tmp_streams;
+	*__sink_pad = NULL;
+	*__sink_streams = 0;
+
+	do {
+		src_pad = media_pad_remote_pad_unique(sink_pad);
+		if (IS_ERR(src_pad)) {
+			dev_dbg(sd ? sd->dev : vdev->dev_parent,
+				"no unique remote pad found from %s:%u\n",
+				sink_pad->entity->name, sink_pad->index);
+			return PTR_ERR(src_pad);
+		}
+
+		sd = media_entity_to_v4l2_subdev(src_pad->entity);
+		if (!sd) {
+			dev_dbg(sd->dev,
+				"media entity %s is not a V4L2 sub-device\n",
+				src_pad->entity->name);
+			return -ENXIO;
+		}
+
+		/* Source streams match sink. */
+		src_streams = sink_streams;
+
+		state = v4l2_subdev_lock_and_get_active_state(sd);
+		ret = __v4l2_mc_pipeline_enabled(state, src_pad,
+						 src_streams, &sink_pad,
+						 &sink_streams);
+		v4l2_subdev_unlock_state(state);
+		if (ret)
+			return ret;
+	} while (sink_pad);
+
+	ret = v4l2_mc_source_get_streams(sd, src_pad->index, &src_streams);
+	if (ret)
+		return ret;
+
+	sd = media_entity_to_v4l2_subdev(src_pad->entity);
+
+	dev_dbg(sd->dev, "following %s:%u/%#llx\n", sd->entity.name,
+		src_pad->index, src_streams);
+
+	for (unsigned int i = __ffs(src_streams); src_streams;
+	     src_streams &= ~BIT_ULL(i), i = __ffs(src_streams)) {
+		sink_pad = media_pad_remote_pad_unique(src_pad);
+		if (IS_ERR(src_pad)) {
+			dev_dbg(sd->dev,
+				"no unique remote pad found from %s:%u\n",
+				sink_pad->entity->name, sink_pad->index);
+			return PTR_ERR(src_pad);
+		}
+
+		ret = v4l2_mc_downpath_enabled(sink_pad, i, func, __sink_pad,
+					       __sink_streams);
+		if (ret == 2)
+			continue;
+		if (ret < 0)
+			return ret;
+		if (!ret)
+			streaming = false;
+	}
+
+	dev_dbg(media_entity_to_v4l2_subdev((*__sink_pad)->entity)->dev,
+		"sink pad %s:%u/%#llx\n", (*__sink_pad)->entity->name,
+		(*__sink_pad)->index, *__sink_streams);
+
+	return streaming;
+}
+EXPORT_SYMBOL_GPL(v4l2_mc_pipeline_enabled);
include/media/v4l2-mc.h +44 -0

50 lines not on this page · raw message

Signature · 1 line
2.39.5
#54 lore raw

Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

kernel test robot noticed the following build warnings:

Hi Sakari,

kernel test robot noticed the following build warnings:

[auto build test WARNING on linuxtv-media-pending/master]
[also build test WARNING on linus/master media-tree/master v6.16-rc2 next-20250618] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Sakari-Ailus/media-ipu6-Use-correct-pads-for-xlate_streams/20250619-161847
base:   https://git.linuxtv.org/media-ci/media-pending.git master
patch link:    https://lore.kernel.org/r/20250619081546.1582969-13-sakari.ailus%40linux.intel.com
patch subject: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()
config: x86_64-buildonly-randconfig-003-20250619 (https://download.01.org/0day-ci/archive/20250619/[email protected]/config)
compiler: clang version 20.1.2 (https://github.com/llvm/llvm-project 58df0ef89dd64126512e4ee27b4ac3fd8ddf6247)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250619/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All warnings (new ones prefixed by >>):

>> drivers/media/v4l2-core/v4l2-mc.c:622:6: warning: variable 'src_streams' set but not used [-Wunused-but-set-variable]
     622 |         u64 src_streams = 0, sink_streams = 0;
         |             ^
>> drivers/media/v4l2-core/v4l2-mc.c:732:44: warning: variable 'source_stream' is uninitialized when used here [-Wuninitialized]
     732 |         return v4l2_mc_downpath_enabled(sink_pad, source_stream, func,
         |                                                   ^~~~~~~~~~~~~
   drivers/media/v4l2-core/v4l2-mc.c:671:19: note: initialize the variable 'source_stream' to silence this warning
     671 |         u32 source_stream;
         |                          ^
         |                           = 0

2 warnings generated.

vim +/src_streams +622 drivers/media/v4l2-core/v4l2-mc.c

615
616 static int
617 __v4l2_mc_pipeline_enabled(struct v4l2_subdev_state *state,
618 struct media_pad *src_pad, u64 __src_streams,
619 struct media_pad **__sink_pad, u64 *__sink_streams)
620 {
621 struct v4l2_subdev_route *route;

 > 622		u64 src_streams = 0, sink_streams = 0;
   623		bool has_sink_pad = false;
   624		unsigned int sink_pad;
   625	
   626		dev_dbg(state->sd->dev, "%s: source enabled, pad/streams %u/%#llx\n",
   627			state->sd->entity.name, src_pad->index, __src_streams);
   628		for_each_active_route(&state->routing, route) {
   629			dev_dbg(state->sd->dev, "%s: %u/%u -> %u/%u, flags %x\n",
   630				state->sd->entity.name,
   631				route->sink_pad, route->sink_stream, route->source_pad,
   632				route->source_stream, route->flags);
   633			if (route->source_pad != src_pad->index)
   634				continue;
   635	
   636			if (!(BIT_ULL(route->source_stream) & __src_streams))
   637				continue;
   638	
   639			if (!has_sink_pad) {
   640				has_sink_pad = true;
   641				sink_pad = route->sink_pad;
   642			}
   643	
   644			if (route->sink_pad != sink_pad) {
   645				dev_dbg(state->sd->dev,
   646					"sink pads (%u vs. %u) differ\n",
   647					route->sink_pad, sink_pad);
   648				return -EMLINK;
   649			}
   650	
   651			sink_streams |= BIT_ULL(route->sink_stream);
   652			src_streams |= BIT_ULL(route->source_stream);
   653		}
   654	
   655		*__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] : NULL;
   656		*__sink_streams = sink_streams;
   657	
   658		return 0;
   659	}
   660	
   661	static int v4l2_mc_downpath_enabled(struct media_pad *sink_pad,
   662					    unsigned int sink_stream,
   663					    bool (*func)(struct video_device *vdev),
   664					    struct media_pad **__sink_pad,
   665					    u64 *__sink_streams)
   666	{
   667		struct v4l2_subdev_state *state;
   668		struct v4l2_subdev_route *route;
   669		struct v4l2_subdev *sd;
   670		struct media_pad *source_pad, *tmp_pad;
   671		u32 source_stream;
   672	
   673		if (!is_media_entity_v4l2_subdev(sink_pad->entity))
   674			return -ENXIO;
   675	
   676		sd = media_entity_to_v4l2_subdev(sink_pad->entity);
   677		dev_dbg(sd->dev, "path_enabled: found sub-device %s\n",
   678			sd->entity.name);
   679	
   680		state = v4l2_subdev_lock_and_get_active_state(sd);
   681		route = v4l2_subdev_find_route(&state->routing, sink_pad->index,
   682					       sink_stream, true, 0);
   683		if (IS_ERR(route)) {
   684			v4l2_subdev_unlock_state(state);
   685			dev_dbg(sd->dev,
   686				"path_enabled: can't find opposite route for %s:%u/%u",
   687				sd->entity.name, sink_pad->index, sink_stream);
   688			return 2;
   689		}
   690	
   691		source_pad = &sd->entity.pads[route->source_pad];
   692		v4l2_subdev_unlock_state(state);
   693	
   694		tmp_pad = sink_pad;
   695		sink_pad = media_pad_remote_pad_unique(source_pad);
   696		if (IS_ERR(sink_pad)) {
   697			dev_dbg(sd->dev,
   698				"path_enabled: can't find remote source for %s:%u\n",
   699				source_pad->entity->name, source_pad->index);
   700			return PTR_ERR(sink_pad);
   701		}
   702	
   703		if (is_media_entity_v4l2_video_device(sink_pad->entity)) {
   704			struct video_device *vdev;
   705	
   706			vdev = media_entity_to_video_device(sink_pad->entity);
   707			if (!vdev)
   708				return -ENXIO;
   709	
   710			dev_dbg(vdev->dev_parent,
   711				"path_enabled: found video device %s\n",
   712				vdev->name);
   713	
   714			if (!*__sink_pad) {
   715				*__sink_pad = tmp_pad;
   716				dev_dbg(sd->dev, "path_enabled: sink %u/%u\n",
   717					tmp_pad->index, sink_stream);
   718			} else if (tmp_pad != *__sink_pad) {
   719				dev_dbg(sd->dev,
   720					"path_enabled: pads %s/%u and %s/%u differ\n",
   721					tmp_pad->entity->name, tmp_pad->index,
   722					(*__sink_pad)->entity->name,
   723					(*__sink_pad)->index);
   724				return -EXDEV;
   725			}
   726	
   727			*__sink_streams |= BIT_ULL(sink_stream);
   728	
   729			return func(vdev);
   730		}
   731	
 > 732		return v4l2_mc_downpath_enabled(sink_pad, source_stream, func,

733 __sink_pad, __sink_streams);
734 }
735

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#55 lore raw

Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

In reply to Sakari Ailus

Hi Sakari,

kernel test robot noticed the following build warnings:

https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Sakari-Ailus/media-ipu6-Use-correct-pads-for-xlate_streams/20250619-161847
base:   https://git.linuxtv.org/media-ci/media-pending.git master
patch link:    https://lore.kernel.org/r/20250619081546.1582969-13-sakari.ailus%40linux.intel.com
patch subject: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()
config: i386-randconfig-141-20250620 (https://download.01.org/0day-ci/archive/20250620/[email protected]/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags

| Reported-by: kernel test robot <[email protected]>
| Reported-by: Dan Carpenter <[email protected]>
| Closes: https://lore.kernel.org/r/[email protected]/

smatch warnings:
drivers/media/v4l2-core/v4l2-mc.c:732 v4l2_mc_downpath_enabled() error: uninitialized symbol 'source_stream'. drivers/media/v4l2-core/v4l2-mc.c:834 v4l2_mc_pipeline_enabled() warn: variable dereferenced before IS_ERR check 'src_pad' (see line 802)

vim +/source_stream +732 drivers/media/v4l2-core/v4l2-mc.c

bc83f513f8ad94 Sakari Ailus 2025-06-19  771  int v4l2_mc_pipeline_enabled(struct video_device *vdev,
bc83f513f8ad94 Sakari Ailus 2025-06-19  772  			     bool (*func)(struct video_device *vdev),
bc83f513f8ad94 Sakari Ailus 2025-06-19  773  			     struct media_pad **__sink_pad, u64 *__sink_streams)
bc83f513f8ad94 Sakari Ailus 2025-06-19  774  {
bc83f513f8ad94 Sakari Ailus 2025-06-19  775  	u64 sink_streams = 1U;
bc83f513f8ad94 Sakari Ailus 2025-06-19  776  	struct media_pad *src_pad;
bc83f513f8ad94 Sakari Ailus 2025-06-19  777  	u64 src_streams;
bc83f513f8ad94 Sakari Ailus 2025-06-19  778  	struct v4l2_subdev_state *state;
bc83f513f8ad94 Sakari Ailus 2025-06-19  779  	struct media_pad *sink_pad = vdev->entity.pads;
bc83f513f8ad94 Sakari Ailus 2025-06-19  780  	struct v4l2_subdev *sd = NULL;
bc83f513f8ad94 Sakari Ailus 2025-06-19  781  	bool streaming = true;
bc83f513f8ad94 Sakari Ailus 2025-06-19  782  	struct media_pad *tmp_pad;
bc83f513f8ad94 Sakari Ailus 2025-06-19  783  	u64 tmp_streams;
bc83f513f8ad94 Sakari Ailus 2025-06-19  784  	int ret;
bc83f513f8ad94 Sakari Ailus 2025-06-19  785  
bc83f513f8ad94 Sakari Ailus 2025-06-19  786  	if (!__sink_pad)
bc83f513f8ad94 Sakari Ailus 2025-06-19  787  		__sink_pad = &tmp_pad;
bc83f513f8ad94 Sakari Ailus 2025-06-19  788  	if (!__sink_streams)
bc83f513f8ad94 Sakari Ailus 2025-06-19  789  		__sink_streams = &tmp_streams;
bc83f513f8ad94 Sakari Ailus 2025-06-19  790  	*__sink_pad = NULL;
bc83f513f8ad94 Sakari Ailus 2025-06-19  791  	*__sink_streams = 0;
bc83f513f8ad94 Sakari Ailus 2025-06-19  792  
bc83f513f8ad94 Sakari Ailus 2025-06-19  793  	do {
bc83f513f8ad94 Sakari Ailus 2025-06-19  794  		src_pad = media_pad_remote_pad_unique(sink_pad);
bc83f513f8ad94 Sakari Ailus 2025-06-19  795  		if (IS_ERR(src_pad)) {
bc83f513f8ad94 Sakari Ailus 2025-06-19  796  			dev_dbg(sd ? sd->dev : vdev->dev_parent,
bc83f513f8ad94 Sakari Ailus 2025-06-19  797  				"no unique remote pad found from %s:%u\n",
bc83f513f8ad94 Sakari Ailus 2025-06-19  798  				sink_pad->entity->name, sink_pad->index);
bc83f513f8ad94 Sakari Ailus 2025-06-19  799  			return PTR_ERR(src_pad);
bc83f513f8ad94 Sakari Ailus 2025-06-19  800  		}
bc83f513f8ad94 Sakari Ailus 2025-06-19  801  
bc83f513f8ad94 Sakari Ailus 2025-06-19 @802  		sd = media_entity_to_v4l2_subdev(src_pad->entity);
bc83f513f8ad94 Sakari Ailus 2025-06-19  803  		if (!sd) {
bc83f513f8ad94 Sakari Ailus 2025-06-19  804  			dev_dbg(sd->dev,
bc83f513f8ad94 Sakari Ailus 2025-06-19  805  				"media entity %s is not a V4L2 sub-device\n",
bc83f513f8ad94 Sakari Ailus 2025-06-19  806  				src_pad->entity->name);
bc83f513f8ad94 Sakari Ailus 2025-06-19  807  			return -ENXIO;
bc83f513f8ad94 Sakari Ailus 2025-06-19  808  		}
bc83f513f8ad94 Sakari Ailus 2025-06-19  809  
bc83f513f8ad94 Sakari Ailus 2025-06-19  810  		/* Source streams match sink. */
bc83f513f8ad94 Sakari Ailus 2025-06-19  811  		src_streams = sink_streams;
bc83f513f8ad94 Sakari Ailus 2025-06-19  812  
bc83f513f8ad94 Sakari Ailus 2025-06-19  813  		state = v4l2_subdev_lock_and_get_active_state(sd);
bc83f513f8ad94 Sakari Ailus 2025-06-19  814  		ret = __v4l2_mc_pipeline_enabled(state, src_pad,
bc83f513f8ad94 Sakari Ailus 2025-06-19  815  						 src_streams, &sink_pad,
bc83f513f8ad94 Sakari Ailus 2025-06-19  816  						 &sink_streams);
bc83f513f8ad94 Sakari Ailus 2025-06-19  817  		v4l2_subdev_unlock_state(state);
bc83f513f8ad94 Sakari Ailus 2025-06-19  818  		if (ret)
bc83f513f8ad94 Sakari Ailus 2025-06-19  819  			return ret;
bc83f513f8ad94 Sakari Ailus 2025-06-19  820  	} while (sink_pad);
bc83f513f8ad94 Sakari Ailus 2025-06-19  821  
bc83f513f8ad94 Sakari Ailus 2025-06-19  822  	ret = v4l2_mc_source_get_streams(sd, src_pad->index, &src_streams);
bc83f513f8ad94 Sakari Ailus 2025-06-19  823  	if (ret)
bc83f513f8ad94 Sakari Ailus 2025-06-19  824  		return ret;
bc83f513f8ad94 Sakari Ailus 2025-06-19  825  
bc83f513f8ad94 Sakari Ailus 2025-06-19  826  	sd = media_entity_to_v4l2_subdev(src_pad->entity);
bc83f513f8ad94 Sakari Ailus 2025-06-19  827  
bc83f513f8ad94 Sakari Ailus 2025-06-19  828  	dev_dbg(sd->dev, "following %s:%u/%#llx\n", sd->entity.name,
bc83f513f8ad94 Sakari Ailus 2025-06-19  829  		src_pad->index, src_streams);
bc83f513f8ad94 Sakari Ailus 2025-06-19  830  
bc83f513f8ad94 Sakari Ailus 2025-06-19  831  	for (unsigned int i = __ffs(src_streams); src_streams;
bc83f513f8ad94 Sakari Ailus 2025-06-19  832  	     src_streams &= ~BIT_ULL(i), i = __ffs(src_streams)) {
bc83f513f8ad94 Sakari Ailus 2025-06-19  833  		sink_pad = media_pad_remote_pad_unique(src_pad);
bc83f513f8ad94 Sakari Ailus 2025-06-19 @834  		if (IS_ERR(src_pad)) {
                                                                   ^^^^^^^

Copy and paste. s/src_pad/sink_pad/.

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#56 lore raw

Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

In reply to Sakari Ailus

Hi Sakari

Quoted 4 lines · Sakari Ailus

v4l2_mc_pipeline_enabled() helps solving a problem known for long but lacked any sort of general solution: with multiple streams, when streaming is started on video nodes one by one, when should streaming be started in the source?

I tried quite some time to understand this, but if I'm not mistaken, a stream-aware subdev, which links to vdev, will always "demux" streams to different pads and will connect to the vdev from there

Source
subdev

 +-----------------+
 |                (1/0) ------> vdev0
 |                 |
(0)[1,2,3]        (2/0  ------> vdev1
 |                 |
 |                (3/0) ------> vdev2
 +-----------------+

With

(0) multiplexed sink pad with 3 streams
(1) (2) and (3) source pad with a single stream

Can't we relay on the media-link state between the source pads and the video devices with something like what Dan has proposed here ?
https://patchwork.linuxtv.org/project/linux-media/patch/[email protected]/

What am I missing ?

Quoted 321 lines

v4l2_mc_pipeline_enabled() traverses the pipeline towards the source, queries the streams generated by the source and traces them back to the video nodes.

---
 drivers/media/v4l2-core/v4l2-mc.c | 243 ++++++++++++++++++++++++++++++
 include/media/v4l2-mc.h           |  44 ++++++
 2 files changed, 287 insertions(+)
drivers/media/v4l2-core/v4l2-mc.c +243 -0
index 937d358697e1..1731088ad436 100644
@@ -612,3 +612,246 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
 	return ret;
 }
 EXPORT_SYMBOL_GPL(v4l2_pipeline_link_notify);
+
+static int
+__v4l2_mc_pipeline_enabled(struct v4l2_subdev_state *state,
+			   struct media_pad *src_pad, u64 __src_streams,
+			   struct media_pad **__sink_pad, u64 *__sink_streams)
+{
+	struct v4l2_subdev_route *route;
+	u64 src_streams = 0, sink_streams = 0;
+	bool has_sink_pad = false;
+	unsigned int sink_pad;
+
+	dev_dbg(state->sd->dev, "%s: source enabled, pad/streams %u/%#llx\n",
+		state->sd->entity.name, src_pad->index, __src_streams);
+	for_each_active_route(&state->routing, route) {
+		dev_dbg(state->sd->dev, "%s: %u/%u -> %u/%u, flags %x\n",
+			state->sd->entity.name,
+			route->sink_pad, route->sink_stream, route->source_pad,
+			route->source_stream, route->flags);
+		if (route->source_pad != src_pad->index)
+			continue;
+
+		if (!(BIT_ULL(route->source_stream) & __src_streams))
+			continue;
+
+		if (!has_sink_pad) {
+			has_sink_pad = true;
+			sink_pad = route->sink_pad;
+		}
+
+		if (route->sink_pad != sink_pad) {
+			dev_dbg(state->sd->dev,
+				"sink pads (%u vs. %u) differ\n",
+				route->sink_pad, sink_pad);
+			return -EMLINK;
+		}
+
+		sink_streams |= BIT_ULL(route->sink_stream);
+		src_streams |= BIT_ULL(route->source_stream);
+	}
+
+	*__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] : NULL;
+	*__sink_streams = sink_streams;
+
+	return 0;
+}
+
+static int v4l2_mc_downpath_enabled(struct media_pad *sink_pad,
+				    unsigned int sink_stream,
+				    bool (*func)(struct video_device *vdev),
+				    struct media_pad **__sink_pad,
+				    u64 *__sink_streams)
+{
+	struct v4l2_subdev_state *state;
+	struct v4l2_subdev_route *route;
+	struct v4l2_subdev *sd;
+	struct media_pad *source_pad, *tmp_pad;
+	u32 source_stream;
+
+	if (!is_media_entity_v4l2_subdev(sink_pad->entity))
+		return -ENXIO;
+
+	sd = media_entity_to_v4l2_subdev(sink_pad->entity);
+	dev_dbg(sd->dev, "path_enabled: found sub-device %s\n",
+		sd->entity.name);
+
+	state = v4l2_subdev_lock_and_get_active_state(sd);
+	route = v4l2_subdev_find_route(&state->routing, sink_pad->index,
+				       sink_stream, true, 0);
+	if (IS_ERR(route)) {
+		v4l2_subdev_unlock_state(state);
+		dev_dbg(sd->dev,
+			"path_enabled: can't find opposite route for %s:%u/%u",
+			sd->entity.name, sink_pad->index, sink_stream);
+		return 2;
+	}
+
+	source_pad = &sd->entity.pads[route->source_pad];
+	v4l2_subdev_unlock_state(state);
+
+	tmp_pad = sink_pad;
+	sink_pad = media_pad_remote_pad_unique(source_pad);
+	if (IS_ERR(sink_pad)) {
+		dev_dbg(sd->dev,
+			"path_enabled: can't find remote source for %s:%u\n",
+			source_pad->entity->name, source_pad->index);
+		return PTR_ERR(sink_pad);
+	}
+
+	if (is_media_entity_v4l2_video_device(sink_pad->entity)) {
+		struct video_device *vdev;
+
+		vdev = media_entity_to_video_device(sink_pad->entity);
+		if (!vdev)
+			return -ENXIO;
+
+		dev_dbg(vdev->dev_parent,
+			"path_enabled: found video device %s\n",
+			vdev->name);
+
+		if (!*__sink_pad) {
+			*__sink_pad = tmp_pad;
+			dev_dbg(sd->dev, "path_enabled: sink %u/%u\n",
+				tmp_pad->index, sink_stream);
+		} else if (tmp_pad != *__sink_pad) {
+			dev_dbg(sd->dev,
+				"path_enabled: pads %s/%u and %s/%u differ\n",
+				tmp_pad->entity->name, tmp_pad->index,
+				(*__sink_pad)->entity->name,
+				(*__sink_pad)->index);
+			return -EXDEV;
+		}
+
+		*__sink_streams |= BIT_ULL(sink_stream);
+
+		return func(vdev);
+	}
+
+	return v4l2_mc_downpath_enabled(sink_pad, source_stream, func,
+					__sink_pad, __sink_streams);
+}
+
+static int v4l2_mc_source_get_streams(struct v4l2_subdev *sd, unsigned int pad,
+				      u64 *__streams)
+{
+	struct v4l2_mbus_frame_desc desc;
+	u64 streams = 0;
+	int ret;
+
+	if (!__streams)
+		return -EINVAL;
+
+	ret = v4l2_subdev_call(sd, pad, get_frame_desc, pad, &desc);
+	if (ret == -ENOIOCTLCMD) {
+		*__streams = 1ULL;
+		return 0;
+	}
+	if (ret)
+		return ret;
+
+	for (unsigned int i = 0; i < desc.num_entries; i++) {
+		if (streams & BIT_ULL(desc.entry[i].stream))
+			return -EINVAL;
+
+		streams |= BIT_ULL(desc.entry[i].stream);
+	}
+
+	dev_dbg(sd->dev, "found streams %#llx based on streams %#llx\n",
+		*__streams, streams);
+	if (*__streams & ~streams)
+		return -EINVAL;
+
+	*__streams = streams;
+
+	return 0;
+}
+
+int v4l2_mc_pipeline_enabled(struct video_device *vdev,
+			     bool (*func)(struct video_device *vdev),
+			     struct media_pad **__sink_pad, u64 *__sink_streams)
+{
+	u64 sink_streams = 1U;
+	struct media_pad *src_pad;
+	u64 src_streams;
+	struct v4l2_subdev_state *state;
+	struct media_pad *sink_pad = vdev->entity.pads;
+	struct v4l2_subdev *sd = NULL;
+	bool streaming = true;
+	struct media_pad *tmp_pad;
+	u64 tmp_streams;
+	int ret;
+
+	if (!__sink_pad)
+		__sink_pad = &tmp_pad;
+	if (!__sink_streams)
+		__sink_streams = &tmp_streams;
+	*__sink_pad = NULL;
+	*__sink_streams = 0;
+
+	do {
+		src_pad = media_pad_remote_pad_unique(sink_pad);
+		if (IS_ERR(src_pad)) {
+			dev_dbg(sd ? sd->dev : vdev->dev_parent,
+				"no unique remote pad found from %s:%u\n",
+				sink_pad->entity->name, sink_pad->index);
+			return PTR_ERR(src_pad);
+		}
+
+		sd = media_entity_to_v4l2_subdev(src_pad->entity);
+		if (!sd) {
+			dev_dbg(sd->dev,
+				"media entity %s is not a V4L2 sub-device\n",
+				src_pad->entity->name);
+			return -ENXIO;
+		}
+
+		/* Source streams match sink. */
+		src_streams = sink_streams;
+
+		state = v4l2_subdev_lock_and_get_active_state(sd);
+		ret = __v4l2_mc_pipeline_enabled(state, src_pad,
+						 src_streams, &sink_pad,
+						 &sink_streams);
+		v4l2_subdev_unlock_state(state);
+		if (ret)
+			return ret;
+	} while (sink_pad);
+
+	ret = v4l2_mc_source_get_streams(sd, src_pad->index, &src_streams);
+	if (ret)
+		return ret;
+
+	sd = media_entity_to_v4l2_subdev(src_pad->entity);
+
+	dev_dbg(sd->dev, "following %s:%u/%#llx\n", sd->entity.name,
+		src_pad->index, src_streams);
+
+	for (unsigned int i = __ffs(src_streams); src_streams;
+	     src_streams &= ~BIT_ULL(i), i = __ffs(src_streams)) {
+		sink_pad = media_pad_remote_pad_unique(src_pad);
+		if (IS_ERR(src_pad)) {
+			dev_dbg(sd->dev,
+				"no unique remote pad found from %s:%u\n",
+				sink_pad->entity->name, sink_pad->index);
+			return PTR_ERR(src_pad);
+		}
+
+		ret = v4l2_mc_downpath_enabled(sink_pad, i, func, __sink_pad,
+					       __sink_streams);
+		if (ret == 2)
+			continue;
+		if (ret < 0)
+			return ret;
+		if (!ret)
+			streaming = false;
+	}
+
+	dev_dbg(media_entity_to_v4l2_subdev((*__sink_pad)->entity)->dev,
+		"sink pad %s:%u/%#llx\n", (*__sink_pad)->entity->name,
+		(*__sink_pad)->index, *__sink_streams);
+
+	return streaming;
+}
+EXPORT_SYMBOL_GPL(v4l2_mc_pipeline_enabled);
include/media/v4l2-mc.h +44 -1
index 1837c9fd78cf..e72c0f62fa34 100644
@@ -193,6 +193,50 @@ void v4l2_pipeline_pm_put(struct media_entity *entity);
 int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
 			      unsigned int notification);
 
+/**
+ * v4l2_mc_pipeline_enabled - Tell when to start streaming
+ * @vdev: The video device
+ * @func: Caller-provided function to tell a video device's streaming state
+ * @__sink_pad: sink pad at the root of the local pipeline
+ * @__sink_streams: streams to start
+ *
+ * Use to tell whether streaming should start on a video node. @func returns
+ * true if streaming has been started on a given video node. @__sink_pad and
+ * @__sink_streams are filled with pad and streams on the sub-device closest to
+ * the video nodes, to be used for calling v4l2_subdev_enable_streams() and
+ * v4l2_subdev_disable_streams().
+ *
+ * Using v4l2_mc_pipeline_enabled() has a few limitations currently (consider it
+ * a to-do list):
+ * * only unbranched streams can be supported albeit adding support for
+ *   downstream branches would be fairly trivial,
+ * * streams within a single source sub-device are considered to start at the
+ *   same time, more control could be added in two ways: 1) for sources to
+ *   determine stream starting, a control could be added to UAPI and 2) sources
+ *   could tell which streams start at the same time using a sub-device
+ *   operation,
+ * * CSI-2 VC framing is ignored currently, but VC-based stream starting could
+ *   be implemented by letting the caller to provide a function to determine
+ *   which streams are of interest and
+ * * routes leading to nowhere are ignored, on some hardware this is a problem,
+ *   but this can also be rather trivially addressed.
+ *
+ * Return:
+ * * 0: Success, but don't start streaming yet
+ * * 1: Success, now it's time to start streaming
+ * * -ENXIO: Route traversal encountered a non-video device/sub-device entity
+ * * -ENOTUNIQ: No unique remote pad
+ * * -ENOLINK: No remote pad found
+ * * -ENOENT: Enabled upstream route not found
+ * * -EMLINK: No unique downstream route found
+ * * -EINVAL: Stream could not be followed to source or was not produced by
+ *            the source
+ */
+int v4l2_mc_pipeline_enabled(struct video_device *vdev,
+			     bool (*func)(struct video_device *vdev),
+			     struct media_pad **__sink_pad,
+			     u64 *__sink_streams);
+
 #else /* CONFIG_MEDIA_CONTROLLER */
 
 static inline int v4l2_mc_create_media_graph(struct media_device *mdev)
--

2.39.5

#57 lore raw

Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

Hi Jacopo,

Thank you for the comments.

Quoted 25 lines · Jacopo Mondi

Hi Sakari

Quoted 4 lines · Sakari Ailus

v4l2_mc_pipeline_enabled() helps solving a problem known for long but lacked any sort of general solution: with multiple streams, when streaming is started on video nodes one by one, when should streaming be started in the source?

I tried quite some time to understand this, but if I'm not mistaken, a stream-aware subdev, which links to vdev, will always "demux" streams to different pads and will connect to the vdev from there

Source
subdev

 +-----------------+
 |                (1/0) ------> vdev0
 |                 |
(0)[1,2,3]        (2/0  ------> vdev1
 |                 |
 |                (3/0) ------> vdev2
 +-----------------+

With

Quoted 8 lines

(0) multiplexed sink pad with 3 streams
(1) (2) and (3) source pad with a single stream

Can't we relay on the media-link state between the source pads and the video devices with something like what Dan has proposed here ?
https://patchwork.linuxtv.org/project/linux-media/patch/[email protected]/

What am I missing ?

There are two different concepts here that are probably underdocumented at the moment: Media entity pipeline state and then when and in which order streaming control is applied on hardware. But... maybe the latter can be derived from the former when it comes to video nodes. The pipeline is set for entities that are upstream from the video device, it's not set on downstream entities. I'll see if I could use that instead. Otherwise, I'll check if Dan's approach could be used for the purpose.

Signature · 3 lines
Regards,

Sakari Ailus
author maintainer
#58 lore raw

Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

In reply to Jacopo Mondi

Hi Jacopo,

Quoted 23 lines · Jacopo Mondi

Hi Sakari

Quoted 4 lines · Sakari Ailus

v4l2_mc_pipeline_enabled() helps solving a problem known for long but lacked any sort of general solution: with multiple streams, when streaming is started on video nodes one by one, when should streaming be started in the source?

I tried quite some time to understand this, but if I'm not mistaken, a stream-aware subdev, which links to vdev, will always "demux" streams to different pads and will connect to the vdev from there

Source
subdev

 +-----------------+
 |                (1/0) ------> vdev0
 |                 |
(0)[1,2,3]        (2/0  ------> vdev1
 |                 |
 |                (3/0) ------> vdev2
 +-----------------+
Quoted 8 lines

With

(0) multiplexed sink pad with 3 streams
(1) (2) and (3) source pad with a single stream

Can't we relay on the media-link state between the source pads and the video devices with something like what Dan has proposed here ?
https://patchwork.linuxtv.org/project/linux-media/patch/[email protected]/

This isn't entirely the same thing: pipeline is specific to the pad but not streams, ad here we're interested in streams. Two streams may start at different points of time even if both are part of the same pipeline.

We definitely should have just one way to figure this out.

I'll send v2 with comments and before that also see if (or how) we could get rid of the callback.

Cc Dan.

Quoted 320 lines

What am I missing ?

Quoted 317 lines

v4l2_mc_pipeline_enabled() traverses the pipeline towards the source, queries the streams generated by the source and traces them back to the video nodes.

---
 drivers/media/v4l2-core/v4l2-mc.c | 243 ++++++++++++++++++++++++++++++
 include/media/v4l2-mc.h           |  44 ++++++
 2 files changed, 287 insertions(+)
drivers/media/v4l2-core/v4l2-mc.c +243 -0
index 937d358697e1..1731088ad436 100644
@@ -612,3 +612,246 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
 	return ret;
 }
 EXPORT_SYMBOL_GPL(v4l2_pipeline_link_notify);
+
+static int
+__v4l2_mc_pipeline_enabled(struct v4l2_subdev_state *state,
+			   struct media_pad *src_pad, u64 __src_streams,
+			   struct media_pad **__sink_pad, u64 *__sink_streams)
+{
+	struct v4l2_subdev_route *route;
+	u64 src_streams = 0, sink_streams = 0;
+	bool has_sink_pad = false;
+	unsigned int sink_pad;
+
+	dev_dbg(state->sd->dev, "%s: source enabled, pad/streams %u/%#llx\n",
+		state->sd->entity.name, src_pad->index, __src_streams);
+	for_each_active_route(&state->routing, route) {
+		dev_dbg(state->sd->dev, "%s: %u/%u -> %u/%u, flags %x\n",
+			state->sd->entity.name,
+			route->sink_pad, route->sink_stream, route->source_pad,
+			route->source_stream, route->flags);
+		if (route->source_pad != src_pad->index)
+			continue;
+
+		if (!(BIT_ULL(route->source_stream) & __src_streams))
+			continue;
+
+		if (!has_sink_pad) {
+			has_sink_pad = true;
+			sink_pad = route->sink_pad;
+		}
+
+		if (route->sink_pad != sink_pad) {
+			dev_dbg(state->sd->dev,
+				"sink pads (%u vs. %u) differ\n",
+				route->sink_pad, sink_pad);
+			return -EMLINK;
+		}
+
+		sink_streams |= BIT_ULL(route->sink_stream);
+		src_streams |= BIT_ULL(route->source_stream);
+	}
+
+	*__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] : NULL;
+	*__sink_streams = sink_streams;
+
+	return 0;
+}
+
+static int v4l2_mc_downpath_enabled(struct media_pad *sink_pad,
+				    unsigned int sink_stream,
+				    bool (*func)(struct video_device *vdev),
+				    struct media_pad **__sink_pad,
+				    u64 *__sink_streams)
+{
+	struct v4l2_subdev_state *state;
+	struct v4l2_subdev_route *route;
+	struct v4l2_subdev *sd;
+	struct media_pad *source_pad, *tmp_pad;
+	u32 source_stream;
+
+	if (!is_media_entity_v4l2_subdev(sink_pad->entity))
+		return -ENXIO;
+
+	sd = media_entity_to_v4l2_subdev(sink_pad->entity);
+	dev_dbg(sd->dev, "path_enabled: found sub-device %s\n",
+		sd->entity.name);
+
+	state = v4l2_subdev_lock_and_get_active_state(sd);
+	route = v4l2_subdev_find_route(&state->routing, sink_pad->index,
+				       sink_stream, true, 0);
+	if (IS_ERR(route)) {
+		v4l2_subdev_unlock_state(state);
+		dev_dbg(sd->dev,
+			"path_enabled: can't find opposite route for %s:%u/%u",
+			sd->entity.name, sink_pad->index, sink_stream);
+		return 2;
+	}
+
+	source_pad = &sd->entity.pads[route->source_pad];
+	v4l2_subdev_unlock_state(state);
+
+	tmp_pad = sink_pad;
+	sink_pad = media_pad_remote_pad_unique(source_pad);
+	if (IS_ERR(sink_pad)) {
+		dev_dbg(sd->dev,
+			"path_enabled: can't find remote source for %s:%u\n",
+			source_pad->entity->name, source_pad->index);
+		return PTR_ERR(sink_pad);
+	}
+
+	if (is_media_entity_v4l2_video_device(sink_pad->entity)) {
+		struct video_device *vdev;
+
+		vdev = media_entity_to_video_device(sink_pad->entity);
+		if (!vdev)
+			return -ENXIO;
+
+		dev_dbg(vdev->dev_parent,
+			"path_enabled: found video device %s\n",
+			vdev->name);
+
+		if (!*__sink_pad) {
+			*__sink_pad = tmp_pad;
+			dev_dbg(sd->dev, "path_enabled: sink %u/%u\n",
+				tmp_pad->index, sink_stream);
+		} else if (tmp_pad != *__sink_pad) {
+			dev_dbg(sd->dev,
+				"path_enabled: pads %s/%u and %s/%u differ\n",
+				tmp_pad->entity->name, tmp_pad->index,
+				(*__sink_pad)->entity->name,
+				(*__sink_pad)->index);
+			return -EXDEV;
+		}
+
+		*__sink_streams |= BIT_ULL(sink_stream);
+
+		return func(vdev);
+	}
+
+	return v4l2_mc_downpath_enabled(sink_pad, source_stream, func,
+					__sink_pad, __sink_streams);
+}
+
+static int v4l2_mc_source_get_streams(struct v4l2_subdev *sd, unsigned int pad,
+				      u64 *__streams)
+{
+	struct v4l2_mbus_frame_desc desc;
+	u64 streams = 0;
+	int ret;
+
+	if (!__streams)
+		return -EINVAL;
+
+	ret = v4l2_subdev_call(sd, pad, get_frame_desc, pad, &desc);
+	if (ret == -ENOIOCTLCMD) {
+		*__streams = 1ULL;
+		return 0;
+	}
+	if (ret)
+		return ret;
+
+	for (unsigned int i = 0; i < desc.num_entries; i++) {
+		if (streams & BIT_ULL(desc.entry[i].stream))
+			return -EINVAL;
+
+		streams |= BIT_ULL(desc.entry[i].stream);
+	}
+
+	dev_dbg(sd->dev, "found streams %#llx based on streams %#llx\n",
+		*__streams, streams);
+	if (*__streams & ~streams)
+		return -EINVAL;
+
+	*__streams = streams;
+
+	return 0;
+}
+
+int v4l2_mc_pipeline_enabled(struct video_device *vdev,
+			     bool (*func)(struct video_device *vdev),
+			     struct media_pad **__sink_pad, u64 *__sink_streams)
+{
+	u64 sink_streams = 1U;
+	struct media_pad *src_pad;
+	u64 src_streams;
+	struct v4l2_subdev_state *state;
+	struct media_pad *sink_pad = vdev->entity.pads;
+	struct v4l2_subdev *sd = NULL;
+	bool streaming = true;
+	struct media_pad *tmp_pad;
+	u64 tmp_streams;
+	int ret;
+
+	if (!__sink_pad)
+		__sink_pad = &tmp_pad;
+	if (!__sink_streams)
+		__sink_streams = &tmp_streams;
+	*__sink_pad = NULL;
+	*__sink_streams = 0;
+
+	do {
+		src_pad = media_pad_remote_pad_unique(sink_pad);
+		if (IS_ERR(src_pad)) {
+			dev_dbg(sd ? sd->dev : vdev->dev_parent,
+				"no unique remote pad found from %s:%u\n",
+				sink_pad->entity->name, sink_pad->index);
+			return PTR_ERR(src_pad);
+		}
+
+		sd = media_entity_to_v4l2_subdev(src_pad->entity);
+		if (!sd) {
+			dev_dbg(sd->dev,
+				"media entity %s is not a V4L2 sub-device\n",
+				src_pad->entity->name);
+			return -ENXIO;
+		}
+
+		/* Source streams match sink. */
+		src_streams = sink_streams;
+
+		state = v4l2_subdev_lock_and_get_active_state(sd);
+		ret = __v4l2_mc_pipeline_enabled(state, src_pad,
+						 src_streams, &sink_pad,
+						 &sink_streams);
+		v4l2_subdev_unlock_state(state);
+		if (ret)
+			return ret;
+	} while (sink_pad);
+
+	ret = v4l2_mc_source_get_streams(sd, src_pad->index, &src_streams);
+	if (ret)
+		return ret;
+
+	sd = media_entity_to_v4l2_subdev(src_pad->entity);
+
+	dev_dbg(sd->dev, "following %s:%u/%#llx\n", sd->entity.name,
+		src_pad->index, src_streams);
+
+	for (unsigned int i = __ffs(src_streams); src_streams;
+	     src_streams &= ~BIT_ULL(i), i = __ffs(src_streams)) {
+		sink_pad = media_pad_remote_pad_unique(src_pad);
+		if (IS_ERR(src_pad)) {
+			dev_dbg(sd->dev,
+				"no unique remote pad found from %s:%u\n",
+				sink_pad->entity->name, sink_pad->index);
+			return PTR_ERR(src_pad);
+		}
+
+		ret = v4l2_mc_downpath_enabled(sink_pad, i, func, __sink_pad,
+					       __sink_streams);
+		if (ret == 2)
+			continue;
+		if (ret < 0)
+			return ret;
+		if (!ret)
+			streaming = false;
+	}
+
+	dev_dbg(media_entity_to_v4l2_subdev((*__sink_pad)->entity)->dev,
+		"sink pad %s:%u/%#llx\n", (*__sink_pad)->entity->name,
+		(*__sink_pad)->index, *__sink_streams);
+
+	return streaming;
+}
+EXPORT_SYMBOL_GPL(v4l2_mc_pipeline_enabled);
include/media/v4l2-mc.h +44 -0
index 1837c9fd78cf..e72c0f62fa34 100644
@@ -193,6 +193,50 @@ void v4l2_pipeline_pm_put(struct media_entity *entity);
 int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
 			      unsigned int notification);
 
+/**
+ * v4l2_mc_pipeline_enabled - Tell when to start streaming
+ * @vdev: The video device
+ * @func: Caller-provided function to tell a video device's streaming state
+ * @__sink_pad: sink pad at the root of the local pipeline
+ * @__sink_streams: streams to start
+ *
+ * Use to tell whether streaming should start on a video node. @func returns
+ * true if streaming has been started on a given video node. @__sink_pad and
+ * @__sink_streams are filled with pad and streams on the sub-device closest to
+ * the video nodes, to be used for calling v4l2_subdev_enable_streams() and
+ * v4l2_subdev_disable_streams().
+ *
+ * Using v4l2_mc_pipeline_enabled() has a few limitations currently (consider it
+ * a to-do list):
+ * * only unbranched streams can be supported albeit adding support for
+ *   downstream branches would be fairly trivial,
+ * * streams within a single source sub-device are considered to start at the
+ *   same time, more control could be added in two ways: 1) for sources to
+ *   determine stream starting, a control could be added to UAPI and 2) sources
+ *   could tell which streams start at the same time using a sub-device
+ *   operation,
+ * * CSI-2 VC framing is ignored currently, but VC-based stream starting could
+ *   be implemented by letting the caller to provide a function to determine
+ *   which streams are of interest and
+ * * routes leading to nowhere are ignored, on some hardware this is a problem,
+ *   but this can also be rather trivially addressed.
+ *
+ * Return:
+ * * 0: Success, but don't start streaming yet
+ * * 1: Success, now it's time to start streaming
+ * * -ENXIO: Route traversal encountered a non-video device/sub-device entity
+ * * -ENOTUNIQ: No unique remote pad
+ * * -ENOLINK: No remote pad found
+ * * -ENOENT: Enabled upstream route not found
+ * * -EMLINK: No unique downstream route found
+ * * -EINVAL: Stream could not be followed to source or was not produced by
+ *            the source
+ */
+int v4l2_mc_pipeline_enabled(struct video_device *vdev,
+			     bool (*func)(struct video_device *vdev),
+			     struct media_pad **__sink_pad,
+			     u64 *__sink_streams);
+
 #else /* CONFIG_MEDIA_CONTROLLER */
 
 static inline int v4l2_mc_create_media_graph(struct media_device *mdev)
Signature · 3 lines
Kind regards,

Sakari Ailus
#59 lore raw

Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

Quoted 31 lines · Sakari Ailus
Quoted 21 lines · Jacopo Mondi
Quoted 4 lines · Sakari Ailus

v4l2_mc_pipeline_enabled() helps solving a problem known for long but lacked any sort of general solution: with multiple streams, when streaming is started on video nodes one by one, when should streaming be started in the source?

I tried quite some time to understand this, but if I'm not mistaken, a stream-aware subdev, which links to vdev, will always "demux" streams to different pads and will connect to the vdev from there

Source
subdev

 +-----------------+
 |                (1/0) ------> vdev0
 |                 |
(0)[1,2,3]        (2/0  ------> vdev1
 |                 |
 |                (3/0) ------> vdev2
 +-----------------+
Quoted 8 lines

With

(0) multiplexed sink pad with 3 streams
(1) (2) and (3) source pad with a single stream

Can't we relay on the media-link state between the source pads and the video devices with something like what Dan has proposed here ?
https://patchwork.linuxtv.org/project/linux-media/patch/[email protected]/

Quoted 5 lines

This isn't entirely the same thing: pipeline is specific to the pad but not streams, ad here we're interested in streams. Two streams may start at different points of time even if both are part of the same pipeline.

We definitely should have just one way to figure this out.

But can we ? Isn't it dependent on use cases ? I can imagine an application wanting to capture two image streams where one of them is enabled all the time and the other one is regularly turned on and off, in which case you would need to start the pipeline when the first stream starts. A different application may instead want to capture the same two streams and make sure it gets all frames on both, which possibly requires delaying the start of capture until both video capture devices are started.

Quoted 328 lines

I'll send v2 with comments and before that also see if (or how) we could get rid of the callback.

Cc Dan.

Quoted 318 lines

What am I missing ?

Quoted 316 lines

v4l2_mc_pipeline_enabled() traverses the pipeline towards the source, queries the streams generated by the source and traces them back to the video nodes.

---
 drivers/media/v4l2-core/v4l2-mc.c | 243 ++++++++++++++++++++++++++++++
 include/media/v4l2-mc.h           |  44 ++++++
 2 files changed, 287 insertions(+)
drivers/media/v4l2-core/v4l2-mc.c +243 -0
index 937d358697e1..1731088ad436 100644
@@ -612,3 +612,246 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
 	return ret;
 }
 EXPORT_SYMBOL_GPL(v4l2_pipeline_link_notify);
+
+static int
+__v4l2_mc_pipeline_enabled(struct v4l2_subdev_state *state,
+			   struct media_pad *src_pad, u64 __src_streams,
+			   struct media_pad **__sink_pad, u64 *__sink_streams)
+{
+	struct v4l2_subdev_route *route;
+	u64 src_streams = 0, sink_streams = 0;
+	bool has_sink_pad = false;
+	unsigned int sink_pad;
+
+	dev_dbg(state->sd->dev, "%s: source enabled, pad/streams %u/%#llx\n",
+		state->sd->entity.name, src_pad->index, __src_streams);
+	for_each_active_route(&state->routing, route) {
+		dev_dbg(state->sd->dev, "%s: %u/%u -> %u/%u, flags %x\n",
+			state->sd->entity.name,
+			route->sink_pad, route->sink_stream, route->source_pad,
+			route->source_stream, route->flags);
+		if (route->source_pad != src_pad->index)
+			continue;
+
+		if (!(BIT_ULL(route->source_stream) & __src_streams))
+			continue;
+
+		if (!has_sink_pad) {
+			has_sink_pad = true;
+			sink_pad = route->sink_pad;
+		}
+
+		if (route->sink_pad != sink_pad) {
+			dev_dbg(state->sd->dev,
+				"sink pads (%u vs. %u) differ\n",
+				route->sink_pad, sink_pad);
+			return -EMLINK;
+		}
+
+		sink_streams |= BIT_ULL(route->sink_stream);
+		src_streams |= BIT_ULL(route->source_stream);
+	}
+
+	*__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] : NULL;
+	*__sink_streams = sink_streams;
+
+	return 0;
+}
+
+static int v4l2_mc_downpath_enabled(struct media_pad *sink_pad,
+				    unsigned int sink_stream,
+				    bool (*func)(struct video_device *vdev),
+				    struct media_pad **__sink_pad,
+				    u64 *__sink_streams)
+{
+	struct v4l2_subdev_state *state;
+	struct v4l2_subdev_route *route;
+	struct v4l2_subdev *sd;
+	struct media_pad *source_pad, *tmp_pad;
+	u32 source_stream;
+
+	if (!is_media_entity_v4l2_subdev(sink_pad->entity))
+		return -ENXIO;
+
+	sd = media_entity_to_v4l2_subdev(sink_pad->entity);
+	dev_dbg(sd->dev, "path_enabled: found sub-device %s\n",
+		sd->entity.name);
+
+	state = v4l2_subdev_lock_and_get_active_state(sd);
+	route = v4l2_subdev_find_route(&state->routing, sink_pad->index,
+				       sink_stream, true, 0);
+	if (IS_ERR(route)) {
+		v4l2_subdev_unlock_state(state);
+		dev_dbg(sd->dev,
+			"path_enabled: can't find opposite route for %s:%u/%u",
+			sd->entity.name, sink_pad->index, sink_stream);
+		return 2;
+	}
+
+	source_pad = &sd->entity.pads[route->source_pad];
+	v4l2_subdev_unlock_state(state);
+
+	tmp_pad = sink_pad;
+	sink_pad = media_pad_remote_pad_unique(source_pad);
+	if (IS_ERR(sink_pad)) {
+		dev_dbg(sd->dev,
+			"path_enabled: can't find remote source for %s:%u\n",
+			source_pad->entity->name, source_pad->index);
+		return PTR_ERR(sink_pad);
+	}
+
+	if (is_media_entity_v4l2_video_device(sink_pad->entity)) {
+		struct video_device *vdev;
+
+		vdev = media_entity_to_video_device(sink_pad->entity);
+		if (!vdev)
+			return -ENXIO;
+
+		dev_dbg(vdev->dev_parent,
+			"path_enabled: found video device %s\n",
+			vdev->name);
+
+		if (!*__sink_pad) {
+			*__sink_pad = tmp_pad;
+			dev_dbg(sd->dev, "path_enabled: sink %u/%u\n",
+				tmp_pad->index, sink_stream);
+		} else if (tmp_pad != *__sink_pad) {
+			dev_dbg(sd->dev,
+				"path_enabled: pads %s/%u and %s/%u differ\n",
+				tmp_pad->entity->name, tmp_pad->index,
+				(*__sink_pad)->entity->name,
+				(*__sink_pad)->index);
+			return -EXDEV;
+		}
+
+		*__sink_streams |= BIT_ULL(sink_stream);
+
+		return func(vdev);
+	}
+
+	return v4l2_mc_downpath_enabled(sink_pad, source_stream, func,
+					__sink_pad, __sink_streams);
+}
+
+static int v4l2_mc_source_get_streams(struct v4l2_subdev *sd, unsigned int pad,
+				      u64 *__streams)
+{
+	struct v4l2_mbus_frame_desc desc;
+	u64 streams = 0;
+	int ret;
+
+	if (!__streams)
+		return -EINVAL;
+
+	ret = v4l2_subdev_call(sd, pad, get_frame_desc, pad, &desc);
+	if (ret == -ENOIOCTLCMD) {
+		*__streams = 1ULL;
+		return 0;
+	}
+	if (ret)
+		return ret;
+
+	for (unsigned int i = 0; i < desc.num_entries; i++) {
+		if (streams & BIT_ULL(desc.entry[i].stream))
+			return -EINVAL;
+
+		streams |= BIT_ULL(desc.entry[i].stream);
+	}
+
+	dev_dbg(sd->dev, "found streams %#llx based on streams %#llx\n",
+		*__streams, streams);
+	if (*__streams & ~streams)
+		return -EINVAL;
+
+	*__streams = streams;
+
+	return 0;
+}
+
+int v4l2_mc_pipeline_enabled(struct video_device *vdev,
+			     bool (*func)(struct video_device *vdev),
+			     struct media_pad **__sink_pad, u64 *__sink_streams)
+{
+	u64 sink_streams = 1U;
+	struct media_pad *src_pad;
+	u64 src_streams;
+	struct v4l2_subdev_state *state;
+	struct media_pad *sink_pad = vdev->entity.pads;
+	struct v4l2_subdev *sd = NULL;
+	bool streaming = true;
+	struct media_pad *tmp_pad;
+	u64 tmp_streams;
+	int ret;
+
+	if (!__sink_pad)
+		__sink_pad = &tmp_pad;
+	if (!__sink_streams)
+		__sink_streams = &tmp_streams;
+	*__sink_pad = NULL;
+	*__sink_streams = 0;
+
+	do {
+		src_pad = media_pad_remote_pad_unique(sink_pad);
+		if (IS_ERR(src_pad)) {
+			dev_dbg(sd ? sd->dev : vdev->dev_parent,
+				"no unique remote pad found from %s:%u\n",
+				sink_pad->entity->name, sink_pad->index);
+			return PTR_ERR(src_pad);
+		}
+
+		sd = media_entity_to_v4l2_subdev(src_pad->entity);
+		if (!sd) {
+			dev_dbg(sd->dev,
+				"media entity %s is not a V4L2 sub-device\n",
+				src_pad->entity->name);
+			return -ENXIO;
+		}
+
+		/* Source streams match sink. */
+		src_streams = sink_streams;
+
+		state = v4l2_subdev_lock_and_get_active_state(sd);
+		ret = __v4l2_mc_pipeline_enabled(state, src_pad,
+						 src_streams, &sink_pad,
+						 &sink_streams);
+		v4l2_subdev_unlock_state(state);
+		if (ret)
+			return ret;
+	} while (sink_pad);
+
+	ret = v4l2_mc_source_get_streams(sd, src_pad->index, &src_streams);
+	if (ret)
+		return ret;
+
+	sd = media_entity_to_v4l2_subdev(src_pad->entity);
+
+	dev_dbg(sd->dev, "following %s:%u/%#llx\n", sd->entity.name,
+		src_pad->index, src_streams);
+
+	for (unsigned int i = __ffs(src_streams); src_streams;
+	     src_streams &= ~BIT_ULL(i), i = __ffs(src_streams)) {
+		sink_pad = media_pad_remote_pad_unique(src_pad);
+		if (IS_ERR(src_pad)) {
+			dev_dbg(sd->dev,
+				"no unique remote pad found from %s:%u\n",
+				sink_pad->entity->name, sink_pad->index);
+			return PTR_ERR(src_pad);
+		}
+
+		ret = v4l2_mc_downpath_enabled(sink_pad, i, func, __sink_pad,
+					       __sink_streams);
+		if (ret == 2)
+			continue;
+		if (ret < 0)
+			return ret;
+		if (!ret)
+			streaming = false;
+	}
+
+	dev_dbg(media_entity_to_v4l2_subdev((*__sink_pad)->entity)->dev,
+		"sink pad %s:%u/%#llx\n", (*__sink_pad)->entity->name,
+		(*__sink_pad)->index, *__sink_streams);
+
+	return streaming;
+}
+EXPORT_SYMBOL_GPL(v4l2_mc_pipeline_enabled);
include/media/v4l2-mc.h +44 -0
index 1837c9fd78cf..e72c0f62fa34 100644
@@ -193,6 +193,50 @@ void v4l2_pipeline_pm_put(struct media_entity *entity);
 int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
 			      unsigned int notification);
 
+/**
+ * v4l2_mc_pipeline_enabled - Tell when to start streaming
+ * @vdev: The video device
+ * @func: Caller-provided function to tell a video device's streaming state
+ * @__sink_pad: sink pad at the root of the local pipeline
+ * @__sink_streams: streams to start
+ *
+ * Use to tell whether streaming should start on a video node. @func returns
+ * true if streaming has been started on a given video node. @__sink_pad and
+ * @__sink_streams are filled with pad and streams on the sub-device closest to
+ * the video nodes, to be used for calling v4l2_subdev_enable_streams() and
+ * v4l2_subdev_disable_streams().
+ *
+ * Using v4l2_mc_pipeline_enabled() has a few limitations currently (consider it
+ * a to-do list):
+ * * only unbranched streams can be supported albeit adding support for
+ *   downstream branches would be fairly trivial,
+ * * streams within a single source sub-device are considered to start at the
+ *   same time, more control could be added in two ways: 1) for sources to
+ *   determine stream starting, a control could be added to UAPI and 2) sources
+ *   could tell which streams start at the same time using a sub-device
+ *   operation,
+ * * CSI-2 VC framing is ignored currently, but VC-based stream starting could
+ *   be implemented by letting the caller to provide a function to determine
+ *   which streams are of interest and
+ * * routes leading to nowhere are ignored, on some hardware this is a problem,
+ *   but this can also be rather trivially addressed.
+ *
+ * Return:
+ * * 0: Success, but don't start streaming yet
+ * * 1: Success, now it's time to start streaming
+ * * -ENXIO: Route traversal encountered a non-video device/sub-device entity
+ * * -ENOTUNIQ: No unique remote pad
+ * * -ENOLINK: No remote pad found
+ * * -ENOENT: Enabled upstream route not found
+ * * -EMLINK: No unique downstream route found
+ * * -EINVAL: Stream could not be followed to source or was not produced by
+ *            the source
+ */
+int v4l2_mc_pipeline_enabled(struct video_device *vdev,
+			     bool (*func)(struct video_device *vdev),
+			     struct media_pad **__sink_pad,
+			     u64 *__sink_streams);
+
 #else /* CONFIG_MEDIA_CONTROLLER */
 
 static inline int v4l2_mc_create_media_graph(struct media_device *mdev)
Signature · 3 lines
Kind regards,

Sakari Ailus
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#60 lore raw

Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

Hi Laurent,

Quoted 38 lines · Laurent Pinchart
Quoted 31 lines · Sakari Ailus
Quoted 21 lines · Jacopo Mondi
Quoted 4 lines · Sakari Ailus

v4l2_mc_pipeline_enabled() helps solving a problem known for long but lacked any sort of general solution: with multiple streams, when streaming is started on video nodes one by one, when should streaming be started in the source?

I tried quite some time to understand this, but if I'm not mistaken, a stream-aware subdev, which links to vdev, will always "demux" streams to different pads and will connect to the vdev from there

Source
subdev

 +-----------------+
 |                (1/0) ------> vdev0
 |                 |
(0)[1,2,3]        (2/0  ------> vdev1
 |                 |
 |                (3/0) ------> vdev2
 +-----------------+
Quoted 8 lines

With

(0) multiplexed sink pad with 3 streams
(1) (2) and (3) source pad with a single stream

Can't we relay on the media-link state between the source pads and the video devices with something like what Dan has proposed here ?
https://patchwork.linuxtv.org/project/linux-media/patch/[email protected]/

Quoted 5 lines

This isn't entirely the same thing: pipeline is specific to the pad but not streams, ad here we're interested in streams. Two streams may start at different points of time even if both are part of the same pipeline.

We definitely should have just one way to figure this out.

Quoted 8 lines

But can we ? Isn't it dependent on use cases ? I can imagine an application wanting to capture two image streams where one of them is enabled all the time and the other one is regularly turned on and off, in which case you would need to start the pipeline when the first stream starts. A different application may instead want to capture the same two streams and make sure it gets all frames on both, which possibly requires delaying the start of capture until both video capture devices are started.

Exactly. It's indeed dependent on the use case but the driver is the same in both cases. This is why the generic solution needs more information from the user space but I think we can work out how to support this after merging the metadata series.

Signature · 3 lines
Regards,

Sakari Ailus
#61 lore raw

Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

Quoted 48 lines · Sakari Ailus
Quoted 38 lines · Laurent Pinchart
Quoted 31 lines · Sakari Ailus
Quoted 21 lines · Jacopo Mondi
Quoted 4 lines · Sakari Ailus

v4l2_mc_pipeline_enabled() helps solving a problem known for long but lacked any sort of general solution: with multiple streams, when streaming is started on video nodes one by one, when should streaming be started in the source?

I tried quite some time to understand this, but if I'm not mistaken, a stream-aware subdev, which links to vdev, will always "demux" streams to different pads and will connect to the vdev from there

Source
subdev

 +-----------------+
 |                (1/0) ------> vdev0
 |                 |
(0)[1,2,3]        (2/0  ------> vdev1
 |                 |
 |                (3/0) ------> vdev2
 +-----------------+
Quoted 8 lines

With

(0) multiplexed sink pad with 3 streams
(1) (2) and (3) source pad with a single stream

Can't we relay on the media-link state between the source pads and the video devices with something like what Dan has proposed here ?
https://patchwork.linuxtv.org/project/linux-media/patch/[email protected]/

Quoted 5 lines

This isn't entirely the same thing: pipeline is specific to the pad but not streams, ad here we're interested in streams. Two streams may start at different points of time even if both are part of the same pipeline.

We definitely should have just one way to figure this out.

Quoted 8 lines

But can we ? Isn't it dependent on use cases ? I can imagine an application wanting to capture two image streams where one of them is enabled all the time and the other one is regularly turned on and off, in which case you would need to start the pipeline when the first stream starts. A different application may instead want to capture the same two streams and make sure it gets all frames on both, which possibly requires delaying the start of capture until both video capture devices are started.

Quoted 4 lines

Exactly. It's indeed dependent on the use case but the driver is the same in both cases. This is why the generic solution needs more information from the user space but I think we can work out how to support this after merging the metadata series.

Yes, we need more information from userspace.

It's also a driver-dependent behaviour. If the streams come from multiple cameras that all go through the same deserializer (e.g. a quad FPD-Link or GMSL deserializer), then the decision on when to start streaming on some of the components in the pipeline could depend on which deserializer is used. Some will require all cameras to be started together, while other could allow starting and stopping sources independently. The nature of upstream elements could influence when downstream elements will need to be started, for the same userspace-facing behaviour.

Signature · 3 lines
Regards,

Laurent Pinchart
#62 lore raw

Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

In reply to Sakari Ailus
kernel test robot noticed the following build errors:

Hi Sakari,

kernel test robot noticed the following build errors:

[auto build test ERROR on linuxtv-media-pending/master]
[also build test ERROR on linus/master media-tree/master v6.16-rc3 next-20250623] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Sakari-Ailus/media-ipu6-Use-correct-pads-for-xlate_streams/20250619-161847
base:   https://git.linuxtv.org/media-ci/media-pending.git master
patch link:    https://lore.kernel.org/r/20250619081546.1582969-13-sakari.ailus%40linux.intel.com
patch subject: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()
config: arm-imx_v4_v5_defconfig (https://download.01.org/0day-ci/archive/20250623/[email protected]/config)
compiler: clang version 21.0.0git (https://github.com/llvm/llvm-project 875b36a8742437b95f623bab1e0332562c7b4b3f)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250623/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

>> drivers/media/v4l2-core/v4l2-mc.c:628:2: error: call to undeclared function 'for_each_active_route'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
     628 |         for_each_active_route(&state->routing, route) {
         |         ^
   drivers/media/v4l2-core/v4l2-mc.c:628:47: error: expected ';' after expression
     628 |         for_each_active_route(&state->routing, route) {
         |                                                      ^
         |                                                      ;
   drivers/media/v4l2-core/v4l2-mc.c:634:4: error: 'continue' statement not in loop statement
     634 |                         continue;
         |                         ^
   drivers/media/v4l2-core/v4l2-mc.c:637:4: error: 'continue' statement not in loop statement
     637 |                         continue;
         |                         ^
>> drivers/media/v4l2-core/v4l2-mc.c:681:10: error: call to undeclared function 'v4l2_subdev_find_route'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
     681 |         route = v4l2_subdev_find_route(&state->routing, sink_pad->index,
         |                 ^
>> drivers/media/v4l2-core/v4l2-mc.c:681:8: error: incompatible integer to pointer conversion assigning to 'struct v4l2_subdev_route *' from 'int' [-Wint-conversion]
     681 |         route = v4l2_subdev_find_route(&state->routing, sink_pad->index,
         |               ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     682 |                                        sink_stream, true, 0);
         |                                        ~~~~~~~~~~~~~~~~~~~~~

6 errors generated.

vim +/for_each_active_route +628 drivers/media/v4l2-core/v4l2-mc.c

615
616 static int
617 __v4l2_mc_pipeline_enabled(struct v4l2_subdev_state *state,
618 struct media_pad *src_pad, u64 __src_streams,
619 struct media_pad **__sink_pad, u64 *__sink_streams)
620 {
621 struct v4l2_subdev_route *route;
622 u64 src_streams = 0, sink_streams = 0;
623 bool has_sink_pad = false;
624 unsigned int sink_pad;
625
626 dev_dbg(state->sd->dev, "%s: source enabled, pad/streams %u/%#llx\n",
627 state->sd->entity.name, src_pad->index, __src_streams);

 > 628		for_each_active_route(&state->routing, route) {
   629			dev_dbg(state->sd->dev, "%s: %u/%u -> %u/%u, flags %x\n",
   630				state->sd->entity.name,
   631				route->sink_pad, route->sink_stream, route->source_pad,
   632				route->source_stream, route->flags);
   633			if (route->source_pad != src_pad->index)
   634				continue;
   635	
   636			if (!(BIT_ULL(route->source_stream) & __src_streams))
   637				continue;
   638	
   639			if (!has_sink_pad) {
   640				has_sink_pad = true;
   641				sink_pad = route->sink_pad;
   642			}
   643	
   644			if (route->sink_pad != sink_pad) {
   645				dev_dbg(state->sd->dev,
   646					"sink pads (%u vs. %u) differ\n",
   647					route->sink_pad, sink_pad);
   648				return -EMLINK;
   649			}
   650	
   651			sink_streams |= BIT_ULL(route->sink_stream);
   652			src_streams |= BIT_ULL(route->source_stream);
   653		}
   654	
   655		*__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] : NULL;
   656		*__sink_streams = sink_streams;
   657	
   658		return 0;
   659	}
   660	
   661	static int v4l2_mc_downpath_enabled(struct media_pad *sink_pad,
   662					    unsigned int sink_stream,
   663					    bool (*func)(struct video_device *vdev),
   664					    struct media_pad **__sink_pad,
   665					    u64 *__sink_streams)
   666	{
   667		struct v4l2_subdev_state *state;
   668		struct v4l2_subdev_route *route;
   669		struct v4l2_subdev *sd;
   670		struct media_pad *source_pad, *tmp_pad;
   671		u32 source_stream;
   672	
   673		if (!is_media_entity_v4l2_subdev(sink_pad->entity))
   674			return -ENXIO;
   675	
   676		sd = media_entity_to_v4l2_subdev(sink_pad->entity);
   677		dev_dbg(sd->dev, "path_enabled: found sub-device %s\n",
   678			sd->entity.name);
   679	
   680		state = v4l2_subdev_lock_and_get_active_state(sd);
 > 681		route = v4l2_subdev_find_route(&state->routing, sink_pad->index,

682 sink_stream, true, 0);
683 if (IS_ERR(route)) {
684 v4l2_subdev_unlock_state(state);
685 dev_dbg(sd->dev,
686 "path_enabled: can't find opposite route for %s:%u/%u",
687 sd->entity.name, sink_pad->index, sink_stream);
688 return 2;
689 }
690
691 source_pad = &sd->entity.pads[route->source_pad];
692 v4l2_subdev_unlock_state(state);
693
694 tmp_pad = sink_pad;
695 sink_pad = media_pad_remote_pad_unique(source_pad);
696 if (IS_ERR(sink_pad)) {
697 dev_dbg(sd->dev,
698 "path_enabled: can't find remote source for %s:%u\n",
699 source_pad->entity->name, source_pad->index);
700 return PTR_ERR(sink_pad);
701 }
702
703 if (is_media_entity_v4l2_video_device(sink_pad->entity)) {
704 struct video_device *vdev;
705
706 vdev = media_entity_to_video_device(sink_pad->entity);
707 if (!vdev)
708 return -ENXIO;
709
710 dev_dbg(vdev->dev_parent,
711 "path_enabled: found video device %s\n",
712 vdev->name);
713
714 if (!*__sink_pad) {
715 *__sink_pad = tmp_pad;
716 dev_dbg(sd->dev, "path_enabled: sink %u/%u\n",
717 tmp_pad->index, sink_stream);
718 } else if (tmp_pad != *__sink_pad) {
719 dev_dbg(sd->dev,
720 "path_enabled: pads %s/%u and %s/%u differ\n",
721 tmp_pad->entity->name, tmp_pad->index,
722 (*__sink_pad)->entity->name,
723 (*__sink_pad)->index);
724 return -EXDEV;
725 }
726
727 *__sink_streams |= BIT_ULL(sink_stream);
728
729 return func(vdev);
730 }
731
732 return v4l2_mc_downpath_enabled(sink_pad, source_stream, func,
733 __sink_pad, __sink_streams);
734 }
735

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#63 lore raw

Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

In reply to Sakari Ailus

Hi Sakari,

Thank you for the patch.

Quoted 271 lines · Sakari Ailus

v4l2_mc_pipeline_enabled() helps solving a problem known for long but lacked any sort of general solution: with multiple streams, when streaming is started on video nodes one by one, when should streaming be started in the source?

v4l2_mc_pipeline_enabled() traverses the pipeline towards the source, queries the streams generated by the source and traces them back to the video nodes.

---
 drivers/media/v4l2-core/v4l2-mc.c | 243 ++++++++++++++++++++++++++++++
 include/media/v4l2-mc.h           |  44 ++++++
 2 files changed, 287 insertions(+)
drivers/media/v4l2-core/v4l2-mc.c +243 -0
index 937d358697e1..1731088ad436 100644
@@ -612,3 +612,246 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
 	return ret;
 }
 EXPORT_SYMBOL_GPL(v4l2_pipeline_link_notify);
+
+static int
+__v4l2_mc_pipeline_enabled(struct v4l2_subdev_state *state,
+			   struct media_pad *src_pad, u64 __src_streams,
+			   struct media_pad **__sink_pad, u64 *__sink_streams)
+{
+	struct v4l2_subdev_route *route;
+	u64 src_streams = 0, sink_streams = 0;
+	bool has_sink_pad = false;
+	unsigned int sink_pad;
+
+	dev_dbg(state->sd->dev, "%s: source enabled, pad/streams %u/%#llx\n",
+		state->sd->entity.name, src_pad->index, __src_streams);
+	for_each_active_route(&state->routing, route) {
+		dev_dbg(state->sd->dev, "%s: %u/%u -> %u/%u, flags %x\n",
+			state->sd->entity.name,
+			route->sink_pad, route->sink_stream, route->source_pad,
+			route->source_stream, route->flags);
+		if (route->source_pad != src_pad->index)
+			continue;
+
+		if (!(BIT_ULL(route->source_stream) & __src_streams))
+			continue;
+
+		if (!has_sink_pad) {
+			has_sink_pad = true;
+			sink_pad = route->sink_pad;
+		}
+
+		if (route->sink_pad != sink_pad) {
+			dev_dbg(state->sd->dev,
+				"sink pads (%u vs. %u) differ\n",
+				route->sink_pad, sink_pad);
+			return -EMLINK;
+		}
+
+		sink_streams |= BIT_ULL(route->sink_stream);
+		src_streams |= BIT_ULL(route->source_stream);
+	}
+
+	*__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] : NULL;
+	*__sink_streams = sink_streams;
+
+	return 0;
+}
+
+static int v4l2_mc_downpath_enabled(struct media_pad *sink_pad,
+				    unsigned int sink_stream,
+				    bool (*func)(struct video_device *vdev),
+				    struct media_pad **__sink_pad,
+				    u64 *__sink_streams)
+{
+	struct v4l2_subdev_state *state;
+	struct v4l2_subdev_route *route;
+	struct v4l2_subdev *sd;
+	struct media_pad *source_pad, *tmp_pad;
+	u32 source_stream;
+
+	if (!is_media_entity_v4l2_subdev(sink_pad->entity))
+		return -ENXIO;
+
+	sd = media_entity_to_v4l2_subdev(sink_pad->entity);
+	dev_dbg(sd->dev, "path_enabled: found sub-device %s\n",
+		sd->entity.name);
+
+	state = v4l2_subdev_lock_and_get_active_state(sd);
+	route = v4l2_subdev_find_route(&state->routing, sink_pad->index,
+				       sink_stream, true, 0);
+	if (IS_ERR(route)) {
+		v4l2_subdev_unlock_state(state);
+		dev_dbg(sd->dev,
+			"path_enabled: can't find opposite route for %s:%u/%u",
+			sd->entity.name, sink_pad->index, sink_stream);
+		return 2;
+	}
+
+	source_pad = &sd->entity.pads[route->source_pad];
+	v4l2_subdev_unlock_state(state);
+
+	tmp_pad = sink_pad;
+	sink_pad = media_pad_remote_pad_unique(source_pad);
+	if (IS_ERR(sink_pad)) {
+		dev_dbg(sd->dev,
+			"path_enabled: can't find remote source for %s:%u\n",
+			source_pad->entity->name, source_pad->index);
+		return PTR_ERR(sink_pad);
+	}
+
+	if (is_media_entity_v4l2_video_device(sink_pad->entity)) {
+		struct video_device *vdev;
+
+		vdev = media_entity_to_video_device(sink_pad->entity);
+		if (!vdev)
+			return -ENXIO;
+
+		dev_dbg(vdev->dev_parent,
+			"path_enabled: found video device %s\n",
+			vdev->name);
+
+		if (!*__sink_pad) {
+			*__sink_pad = tmp_pad;
+			dev_dbg(sd->dev, "path_enabled: sink %u/%u\n",
+				tmp_pad->index, sink_stream);
+		} else if (tmp_pad != *__sink_pad) {
+			dev_dbg(sd->dev,
+				"path_enabled: pads %s/%u and %s/%u differ\n",
+				tmp_pad->entity->name, tmp_pad->index,
+				(*__sink_pad)->entity->name,
+				(*__sink_pad)->index);
+			return -EXDEV;
+		}
+
+		*__sink_streams |= BIT_ULL(sink_stream);
+
+		return func(vdev);
+	}
+
+	return v4l2_mc_downpath_enabled(sink_pad, source_stream, func,
+					__sink_pad, __sink_streams);
+}
+
+static int v4l2_mc_source_get_streams(struct v4l2_subdev *sd, unsigned int pad,
+				      u64 *__streams)
+{
+	struct v4l2_mbus_frame_desc desc;
+	u64 streams = 0;
+	int ret;
+
+	if (!__streams)
+		return -EINVAL;
+
+	ret = v4l2_subdev_call(sd, pad, get_frame_desc, pad, &desc);
+	if (ret == -ENOIOCTLCMD) {
+		*__streams = 1ULL;
+		return 0;
+	}
+	if (ret)
+		return ret;
+
+	for (unsigned int i = 0; i < desc.num_entries; i++) {
+		if (streams & BIT_ULL(desc.entry[i].stream))
+			return -EINVAL;
+
+		streams |= BIT_ULL(desc.entry[i].stream);
+	}
+
+	dev_dbg(sd->dev, "found streams %#llx based on streams %#llx\n",
+		*__streams, streams);
+	if (*__streams & ~streams)
+		return -EINVAL;
+
+	*__streams = streams;
+
+	return 0;
+}
+
+int v4l2_mc_pipeline_enabled(struct video_device *vdev,
+			     bool (*func)(struct video_device *vdev),
+			     struct media_pad **__sink_pad, u64 *__sink_streams)
+{
+	u64 sink_streams = 1U;
+	struct media_pad *src_pad;
+	u64 src_streams;
+	struct v4l2_subdev_state *state;
+	struct media_pad *sink_pad = vdev->entity.pads;
+	struct v4l2_subdev *sd = NULL;
+	bool streaming = true;
+	struct media_pad *tmp_pad;
+	u64 tmp_streams;
+	int ret;
+
+	if (!__sink_pad)
+		__sink_pad = &tmp_pad;
+	if (!__sink_streams)
+		__sink_streams = &tmp_streams;
+	*__sink_pad = NULL;
+	*__sink_streams = 0;
+
+	do {
+		src_pad = media_pad_remote_pad_unique(sink_pad);
+		if (IS_ERR(src_pad)) {
+			dev_dbg(sd ? sd->dev : vdev->dev_parent,
+				"no unique remote pad found from %s:%u\n",
+				sink_pad->entity->name, sink_pad->index);
+			return PTR_ERR(src_pad);
+		}
+
+		sd = media_entity_to_v4l2_subdev(src_pad->entity);
+		if (!sd) {
+			dev_dbg(sd->dev,
+				"media entity %s is not a V4L2 sub-device\n",
+				src_pad->entity->name);
+			return -ENXIO;
+		}
+
+		/* Source streams match sink. */
+		src_streams = sink_streams;
+
+		state = v4l2_subdev_lock_and_get_active_state(sd);
+		ret = __v4l2_mc_pipeline_enabled(state, src_pad,
+						 src_streams, &sink_pad,
+						 &sink_streams);
+		v4l2_subdev_unlock_state(state);
+		if (ret)
+			return ret;
+	} while (sink_pad);
+
+	ret = v4l2_mc_source_get_streams(sd, src_pad->index, &src_streams);
+	if (ret)
+		return ret;
+
+	sd = media_entity_to_v4l2_subdev(src_pad->entity);
+
+	dev_dbg(sd->dev, "following %s:%u/%#llx\n", sd->entity.name,
+		src_pad->index, src_streams);
+
+	for (unsigned int i = __ffs(src_streams); src_streams;
+	     src_streams &= ~BIT_ULL(i), i = __ffs(src_streams)) {
+		sink_pad = media_pad_remote_pad_unique(src_pad);
+		if (IS_ERR(src_pad)) {
+			dev_dbg(sd->dev,
+				"no unique remote pad found from %s:%u\n",
+				sink_pad->entity->name, sink_pad->index);
+			return PTR_ERR(src_pad);
+		}
+
+		ret = v4l2_mc_downpath_enabled(sink_pad, i, func, __sink_pad,
+					       __sink_streams);
+		if (ret == 2)
+			continue;
+		if (ret < 0)
+			return ret;
+		if (!ret)
+			streaming = false;
+	}
+
+	dev_dbg(media_entity_to_v4l2_subdev((*__sink_pad)->entity)->dev,
+		"sink pad %s:%u/%#llx\n", (*__sink_pad)->entity->name,
+		(*__sink_pad)->index, *__sink_streams);
+
+	return streaming;
+}
+EXPORT_SYMBOL_GPL(v4l2_mc_pipeline_enabled);
include/media/v4l2-mc.h +0 -0
index 1837c9fd78cf..e72c0f62fa34 100644

include/media/v4l2-mc.h void v4l2_pipeline_pm_put(struct media_entity *entity); · line 201

Quoted 9 lines
include/media/v4l2-mc.h +6 -0
@@ -193,6 +193,50 @@ void v4l2_pipeline_pm_put(struct media_entity *entity);
 int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
 			      unsigned int notification);
 
+/**
+ * v4l2_mc_pipeline_enabled - Tell when to start streaming
+ * @vdev: The video device
+ * @func: Caller-provided function to tell a video device's streaming state
+ * @__sink_pad: sink pad at the root of the local pipeline
+ * @__sink_streams: streams to start

Any reason for the double underscore ?

include/media/v4l2-mc.h void v4l2_pipeline_pm_put(struct media_entity *entity); · line 212

Quoted 11 lines
include/media/v4l2-mc.h +11 -0
+ *
+ * Use to tell whether streaming should start on a video node. @func returns
+ * true if streaming has been started on a given video node. @__sink_pad and
+ * @__sink_streams are filled with pad and streams on the sub-device closest to
+ * the video nodes, to be used for calling v4l2_subdev_enable_streams() and
+ * v4l2_subdev_disable_streams().
+ *
+ * Using v4l2_mc_pipeline_enabled() has a few limitations currently (consider it
+ * a to-do list):
+ * * only unbranched streams can be supported albeit adding support for
+ *   downstream branches would be fairly trivial,

I can't tell from the documentation here what you mean exactly by "unbranched streams".

include/media/v4l2-mc.h void v4l2_pipeline_pm_put(struct media_entity *entity); · line 222

Quoted 10 lines
include/media/v4l2-mc.h +10 -0
+ * * streams within a single source sub-device are considered to start at the
+ *   same time, more control could be added in two ways: 1) for sources to
+ *   determine stream starting, a control could be added to UAPI and 2) sources
+ *   could tell which streams start at the same time using a sub-device
+ *   operation,
+ * * CSI-2 VC framing is ignored currently, but VC-based stream starting could
+ *   be implemented by letting the caller to provide a function to determine
+ *   which streams are of interest and
+ * * routes leading to nowhere are ignored, on some hardware this is a problem,
+ *   but this can also be rather trivially addressed.

I'm afraid this function looks like a hack, to solve a problem that is not even explicitly described. You don't explain the issue in the cover letter or in Documentation/, the cover letter merely states that this is a "partial solution". The documentation of the function doesn't explain what criteria the decision is based on. We need a proper explanation of the problem in Documentation/, with a description of the behaviour (or behaviours) drivers are expected to implement.

Furthermore, on the implementation side, things are fairly inefficient. We already traverse the whole pipeline in media_pipeline_start(), based on links and routes, and populate the media_pipeline structure. We shouldn't do the same here, but instead inspect media_pipeline to extract the information we need. If you're missing information there, let's add it.

Quoted 20 lines
include/media/v4l2-mc.h +17 -0
+ *
+ * Return:
+ * * 0: Success, but don't start streaming yet
+ * * 1: Success, now it's time to start streaming
+ * * -ENXIO: Route traversal encountered a non-video device/sub-device entity
+ * * -ENOTUNIQ: No unique remote pad
+ * * -ENOLINK: No remote pad found
+ * * -ENOENT: Enabled upstream route not found
+ * * -EMLINK: No unique downstream route found
+ * * -EINVAL: Stream could not be followed to source or was not produced by
+ *            the source
+ */
+int v4l2_mc_pipeline_enabled(struct video_device *vdev,
+			     bool (*func)(struct video_device *vdev),
+			     struct media_pad **__sink_pad,
+			     u64 *__sink_streams);
+
 #else /* CONFIG_MEDIA_CONTROLLER */
 
 static inline int v4l2_mc_create_media_graph(struct media_device *mdev)
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#64 lore raw

Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

Hi Laurent,

Thanks for the review.

Quoted 279 lines · Laurent Pinchart

Hi Sakari,

Thank you for the patch.

Quoted 274 lines · Sakari Ailus

v4l2_mc_pipeline_enabled() helps solving a problem known for long but lacked any sort of general solution: with multiple streams, when streaming is started on video nodes one by one, when should streaming be started in the source?

v4l2_mc_pipeline_enabled() traverses the pipeline towards the source, queries the streams generated by the source and traces them back to the video nodes.

---
 drivers/media/v4l2-core/v4l2-mc.c | 243 ++++++++++++++++++++++++++++++
 include/media/v4l2-mc.h           |  44 ++++++
 2 files changed, 287 insertions(+)
drivers/media/v4l2-core/v4l2-mc.c +243 -0
index 937d358697e1..1731088ad436 100644
@@ -612,3 +612,246 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
 	return ret;
 }
 EXPORT_SYMBOL_GPL(v4l2_pipeline_link_notify);
+
+static int
+__v4l2_mc_pipeline_enabled(struct v4l2_subdev_state *state,
+			   struct media_pad *src_pad, u64 __src_streams,
+			   struct media_pad **__sink_pad, u64 *__sink_streams)
+{
+	struct v4l2_subdev_route *route;
+	u64 src_streams = 0, sink_streams = 0;
+	bool has_sink_pad = false;
+	unsigned int sink_pad;
+
+	dev_dbg(state->sd->dev, "%s: source enabled, pad/streams %u/%#llx\n",
+		state->sd->entity.name, src_pad->index, __src_streams);
+	for_each_active_route(&state->routing, route) {
+		dev_dbg(state->sd->dev, "%s: %u/%u -> %u/%u, flags %x\n",
+			state->sd->entity.name,
+			route->sink_pad, route->sink_stream, route->source_pad,
+			route->source_stream, route->flags);
+		if (route->source_pad != src_pad->index)
+			continue;
+
+		if (!(BIT_ULL(route->source_stream) & __src_streams))
+			continue;
+
+		if (!has_sink_pad) {
+			has_sink_pad = true;
+			sink_pad = route->sink_pad;
+		}
+
+		if (route->sink_pad != sink_pad) {
+			dev_dbg(state->sd->dev,
+				"sink pads (%u vs. %u) differ\n",
+				route->sink_pad, sink_pad);
+			return -EMLINK;
+		}
+
+		sink_streams |= BIT_ULL(route->sink_stream);
+		src_streams |= BIT_ULL(route->source_stream);
+	}
+
+	*__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] : NULL;
+	*__sink_streams = sink_streams;
+
+	return 0;
+}
+
+static int v4l2_mc_downpath_enabled(struct media_pad *sink_pad,
+				    unsigned int sink_stream,
+				    bool (*func)(struct video_device *vdev),
+				    struct media_pad **__sink_pad,
+				    u64 *__sink_streams)
+{
+	struct v4l2_subdev_state *state;
+	struct v4l2_subdev_route *route;
+	struct v4l2_subdev *sd;
+	struct media_pad *source_pad, *tmp_pad;
+	u32 source_stream;
+
+	if (!is_media_entity_v4l2_subdev(sink_pad->entity))
+		return -ENXIO;
+
+	sd = media_entity_to_v4l2_subdev(sink_pad->entity);
+	dev_dbg(sd->dev, "path_enabled: found sub-device %s\n",
+		sd->entity.name);
+
+	state = v4l2_subdev_lock_and_get_active_state(sd);
+	route = v4l2_subdev_find_route(&state->routing, sink_pad->index,
+				       sink_stream, true, 0);
+	if (IS_ERR(route)) {
+		v4l2_subdev_unlock_state(state);
+		dev_dbg(sd->dev,
+			"path_enabled: can't find opposite route for %s:%u/%u",
+			sd->entity.name, sink_pad->index, sink_stream);
+		return 2;
+	}
+
+	source_pad = &sd->entity.pads[route->source_pad];
+	v4l2_subdev_unlock_state(state);
+
+	tmp_pad = sink_pad;
+	sink_pad = media_pad_remote_pad_unique(source_pad);
+	if (IS_ERR(sink_pad)) {
+		dev_dbg(sd->dev,
+			"path_enabled: can't find remote source for %s:%u\n",
+			source_pad->entity->name, source_pad->index);
+		return PTR_ERR(sink_pad);
+	}
+
+	if (is_media_entity_v4l2_video_device(sink_pad->entity)) {
+		struct video_device *vdev;
+
+		vdev = media_entity_to_video_device(sink_pad->entity);
+		if (!vdev)
+			return -ENXIO;
+
+		dev_dbg(vdev->dev_parent,
+			"path_enabled: found video device %s\n",
+			vdev->name);
+
+		if (!*__sink_pad) {
+			*__sink_pad = tmp_pad;
+			dev_dbg(sd->dev, "path_enabled: sink %u/%u\n",
+				tmp_pad->index, sink_stream);
+		} else if (tmp_pad != *__sink_pad) {
+			dev_dbg(sd->dev,
+				"path_enabled: pads %s/%u and %s/%u differ\n",
+				tmp_pad->entity->name, tmp_pad->index,
+				(*__sink_pad)->entity->name,
+				(*__sink_pad)->index);
+			return -EXDEV;
+		}
+
+		*__sink_streams |= BIT_ULL(sink_stream);
+
+		return func(vdev);
+	}
+
+	return v4l2_mc_downpath_enabled(sink_pad, source_stream, func,
+					__sink_pad, __sink_streams);
+}
+
+static int v4l2_mc_source_get_streams(struct v4l2_subdev *sd, unsigned int pad,
+				      u64 *__streams)
+{
+	struct v4l2_mbus_frame_desc desc;
+	u64 streams = 0;
+	int ret;
+
+	if (!__streams)
+		return -EINVAL;
+
+	ret = v4l2_subdev_call(sd, pad, get_frame_desc, pad, &desc);
+	if (ret == -ENOIOCTLCMD) {
+		*__streams = 1ULL;
+		return 0;
+	}
+	if (ret)
+		return ret;
+
+	for (unsigned int i = 0; i < desc.num_entries; i++) {
+		if (streams & BIT_ULL(desc.entry[i].stream))
+			return -EINVAL;
+
+		streams |= BIT_ULL(desc.entry[i].stream);
+	}
+
+	dev_dbg(sd->dev, "found streams %#llx based on streams %#llx\n",
+		*__streams, streams);
+	if (*__streams & ~streams)
+		return -EINVAL;
+
+	*__streams = streams;
+
+	return 0;
+}
+
+int v4l2_mc_pipeline_enabled(struct video_device *vdev,
+			     bool (*func)(struct video_device *vdev),
+			     struct media_pad **__sink_pad, u64 *__sink_streams)
+{
+	u64 sink_streams = 1U;
+	struct media_pad *src_pad;
+	u64 src_streams;
+	struct v4l2_subdev_state *state;
+	struct media_pad *sink_pad = vdev->entity.pads;
+	struct v4l2_subdev *sd = NULL;
+	bool streaming = true;
+	struct media_pad *tmp_pad;
+	u64 tmp_streams;
+	int ret;
+
+	if (!__sink_pad)
+		__sink_pad = &tmp_pad;
+	if (!__sink_streams)
+		__sink_streams = &tmp_streams;
+	*__sink_pad = NULL;
+	*__sink_streams = 0;
+
+	do {
+		src_pad = media_pad_remote_pad_unique(sink_pad);
+		if (IS_ERR(src_pad)) {
+			dev_dbg(sd ? sd->dev : vdev->dev_parent,
+				"no unique remote pad found from %s:%u\n",
+				sink_pad->entity->name, sink_pad->index);
+			return PTR_ERR(src_pad);
+		}
+
+		sd = media_entity_to_v4l2_subdev(src_pad->entity);
+		if (!sd) {
+			dev_dbg(sd->dev,
+				"media entity %s is not a V4L2 sub-device\n",
+				src_pad->entity->name);
+			return -ENXIO;
+		}
+
+		/* Source streams match sink. */
+		src_streams = sink_streams;
+
+		state = v4l2_subdev_lock_and_get_active_state(sd);
+		ret = __v4l2_mc_pipeline_enabled(state, src_pad,
+						 src_streams, &sink_pad,
+						 &sink_streams);
+		v4l2_subdev_unlock_state(state);
+		if (ret)
+			return ret;
+	} while (sink_pad);
+
+	ret = v4l2_mc_source_get_streams(sd, src_pad->index, &src_streams);
+	if (ret)
+		return ret;
+
+	sd = media_entity_to_v4l2_subdev(src_pad->entity);
+
+	dev_dbg(sd->dev, "following %s:%u/%#llx\n", sd->entity.name,
+		src_pad->index, src_streams);
+
+	for (unsigned int i = __ffs(src_streams); src_streams;
+	     src_streams &= ~BIT_ULL(i), i = __ffs(src_streams)) {
+		sink_pad = media_pad_remote_pad_unique(src_pad);
+		if (IS_ERR(src_pad)) {
+			dev_dbg(sd->dev,
+				"no unique remote pad found from %s:%u\n",
+				sink_pad->entity->name, sink_pad->index);
+			return PTR_ERR(src_pad);
+		}
+
+		ret = v4l2_mc_downpath_enabled(sink_pad, i, func, __sink_pad,
+					       __sink_streams);
+		if (ret == 2)
+			continue;
+		if (ret < 0)
+			return ret;
+		if (!ret)
+			streaming = false;
+	}
+
+	dev_dbg(media_entity_to_v4l2_subdev((*__sink_pad)->entity)->dev,
+		"sink pad %s:%u/%#llx\n", (*__sink_pad)->entity->name,
+		(*__sink_pad)->index, *__sink_streams);
+
+	return streaming;
+}
+EXPORT_SYMBOL_GPL(v4l2_mc_pipeline_enabled);
include/media/v4l2-mc.h +0 -0
index 1837c9fd78cf..e72c0f62fa34 100644
@@ -193,6 +193,50 @@ void v4l2_pipeline_pm_put(struct media_entity *entity);
 int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
 			      unsigned int notification);
 
Quoted 8 lines

include/media/v4l2-mc.h void v4l2_pipeline_pm_put(struct media_entity *entity); · line 201

Quoted 6 lines
include/media/v4l2-mc.h +6 -0
+/**
+ * v4l2_mc_pipeline_enabled - Tell when to start streaming
+ * @vdev: The video device
+ * @func: Caller-provided function to tell a video device's streaming state
+ * @__sink_pad: sink pad at the root of the local pipeline
+ * @__sink_streams: streams to start

Any reason for the double underscore ?

The function used internally a variable without the underscores for a different purpose.

Quoted 13 lines

include/media/v4l2-mc.h void v4l2_pipeline_pm_put(struct media_entity *entity); · line 212

Quoted 12 lines
include/media/v4l2-mc.h +11 -0
+ *
+ * Use to tell whether streaming should start on a video node. @func returns
+ * true if streaming has been started on a given video node. @__sink_pad and
+ * @__sink_streams are filled with pad and streams on the sub-device closest to
+ * the video nodes, to be used for calling v4l2_subdev_enable_streams() and
+ * v4l2_subdev_disable_streams().
+ *
+ * Using v4l2_mc_pipeline_enabled() has a few limitations currently (consider it
+ * a to-do list):
+ * * only unbranched streams can be supported albeit adding support for
+ *   downstream branches would be fairly trivial,
Quoted 2 lines

I can't tell from the documentation here what you mean exactly by "unbranched streams".

These may have been referred to as "linear" streams elsewhere. I'll address this in the next version.

Quoted 12 lines

include/media/v4l2-mc.h void v4l2_pipeline_pm_put(struct media_entity *entity); · line 222

Quoted 11 lines
include/media/v4l2-mc.h +10 -0
+ * * streams within a single source sub-device are considered to start at the
+ *   same time, more control could be added in two ways: 1) for sources to
+ *   determine stream starting, a control could be added to UAPI and 2) sources
+ *   could tell which streams start at the same time using a sub-device
+ *   operation,
+ * * CSI-2 VC framing is ignored currently, but VC-based stream starting could
+ *   be implemented by letting the caller to provide a function to determine
+ *   which streams are of interest and
+ * * routes leading to nowhere are ignored, on some hardware this is a problem,
+ *   but this can also be rather trivially addressed.
Quoted 7 lines

I'm afraid this function looks like a hack, to solve a problem that is not even explicitly described. You don't explain the issue in the cover letter or in Documentation/, the cover letter merely states that this is a "partial solution". The documentation of the function doesn't explain what criteria the decision is based on. We need a proper explanation of the problem in Documentation/, with a description of the behaviour (or behaviours) drivers are expected to implement.

I believe this is discussed in the cover letter, not in detail though. I'm fine with adding more documentation, there isn't much as I also wanted to get feedback on the approach itself. No alternatives have been proposed so far either.

Quoted 7 lines

Furthermore, on the implementation side, things are fairly inefficient. We already traverse the whole pipeline in media_pipeline_start(), based on links and routes, and populate the media_pipeline structure. We shouldn't do the same here, but instead inspect media_pipeline to extract the information we need. If you're missing information there, let's add it.

The media pipeline is created and traversed, yes, but the media pipeline does not include streams which are strictly a V4L2 concept. I agree there is some overlap between the two but as long as MC remains separate from V4L2, we can't reasonably access streams from the pipeline traversal.

Quoted 21 lines
Quoted 21 lines
include/media/v4l2-mc.h +17 -0
+ *
+ * Return:
+ * * 0: Success, but don't start streaming yet
+ * * 1: Success, now it's time to start streaming
+ * * -ENXIO: Route traversal encountered a non-video device/sub-device entity
+ * * -ENOTUNIQ: No unique remote pad
+ * * -ENOLINK: No remote pad found
+ * * -ENOENT: Enabled upstream route not found
+ * * -EMLINK: No unique downstream route found
+ * * -EINVAL: Stream could not be followed to source or was not produced by
+ *            the source
+ */
+int v4l2_mc_pipeline_enabled(struct video_device *vdev,
+			     bool (*func)(struct video_device *vdev),
+			     struct media_pad **__sink_pad,
+			     u64 *__sink_streams);
+
 #else /* CONFIG_MEDIA_CONTROLLER */
 
 static inline int v4l2_mc_create_media_graph(struct media_device *mdev)
Signature · 3 lines
Kind regards,

Sakari Ailus
#65 lore raw

Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

Quoted 285 lines · Sakari Ailus
Quoted 275 lines · Laurent Pinchart
Quoted 274 lines · Sakari Ailus

v4l2_mc_pipeline_enabled() helps solving a problem known for long but lacked any sort of general solution: with multiple streams, when streaming is started on video nodes one by one, when should streaming be started in the source?

v4l2_mc_pipeline_enabled() traverses the pipeline towards the source, queries the streams generated by the source and traces them back to the video nodes.

---
 drivers/media/v4l2-core/v4l2-mc.c | 243 ++++++++++++++++++++++++++++++
 include/media/v4l2-mc.h           |  44 ++++++
 2 files changed, 287 insertions(+)
drivers/media/v4l2-core/v4l2-mc.c +243 -0
index 937d358697e1..1731088ad436 100644
@@ -612,3 +612,246 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
 	return ret;
 }
 EXPORT_SYMBOL_GPL(v4l2_pipeline_link_notify);
+
+static int
+__v4l2_mc_pipeline_enabled(struct v4l2_subdev_state *state,
+			   struct media_pad *src_pad, u64 __src_streams,
+			   struct media_pad **__sink_pad, u64 *__sink_streams)
+{
+	struct v4l2_subdev_route *route;
+	u64 src_streams = 0, sink_streams = 0;
+	bool has_sink_pad = false;
+	unsigned int sink_pad;
+
+	dev_dbg(state->sd->dev, "%s: source enabled, pad/streams %u/%#llx\n",
+		state->sd->entity.name, src_pad->index, __src_streams);
+	for_each_active_route(&state->routing, route) {
+		dev_dbg(state->sd->dev, "%s: %u/%u -> %u/%u, flags %x\n",
+			state->sd->entity.name,
+			route->sink_pad, route->sink_stream, route->source_pad,
+			route->source_stream, route->flags);
+		if (route->source_pad != src_pad->index)
+			continue;
+
+		if (!(BIT_ULL(route->source_stream) & __src_streams))
+			continue;
+
+		if (!has_sink_pad) {
+			has_sink_pad = true;
+			sink_pad = route->sink_pad;
+		}
+
+		if (route->sink_pad != sink_pad) {
+			dev_dbg(state->sd->dev,
+				"sink pads (%u vs. %u) differ\n",
+				route->sink_pad, sink_pad);
+			return -EMLINK;
+		}
+
+		sink_streams |= BIT_ULL(route->sink_stream);
+		src_streams |= BIT_ULL(route->source_stream);
+	}
+
+	*__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] : NULL;
+	*__sink_streams = sink_streams;
+
+	return 0;
+}
+
+static int v4l2_mc_downpath_enabled(struct media_pad *sink_pad,
+				    unsigned int sink_stream,
+				    bool (*func)(struct video_device *vdev),
+				    struct media_pad **__sink_pad,
+				    u64 *__sink_streams)
+{
+	struct v4l2_subdev_state *state;
+	struct v4l2_subdev_route *route;
+	struct v4l2_subdev *sd;
+	struct media_pad *source_pad, *tmp_pad;
+	u32 source_stream;
+
+	if (!is_media_entity_v4l2_subdev(sink_pad->entity))
+		return -ENXIO;
+
+	sd = media_entity_to_v4l2_subdev(sink_pad->entity);
+	dev_dbg(sd->dev, "path_enabled: found sub-device %s\n",
+		sd->entity.name);
+
+	state = v4l2_subdev_lock_and_get_active_state(sd);
+	route = v4l2_subdev_find_route(&state->routing, sink_pad->index,
+				       sink_stream, true, 0);
+	if (IS_ERR(route)) {
+		v4l2_subdev_unlock_state(state);
+		dev_dbg(sd->dev,
+			"path_enabled: can't find opposite route for %s:%u/%u",
+			sd->entity.name, sink_pad->index, sink_stream);
+		return 2;
+	}
+
+	source_pad = &sd->entity.pads[route->source_pad];
+	v4l2_subdev_unlock_state(state);
+
+	tmp_pad = sink_pad;
+	sink_pad = media_pad_remote_pad_unique(source_pad);
+	if (IS_ERR(sink_pad)) {
+		dev_dbg(sd->dev,
+			"path_enabled: can't find remote source for %s:%u\n",
+			source_pad->entity->name, source_pad->index);
+		return PTR_ERR(sink_pad);
+	}
+
+	if (is_media_entity_v4l2_video_device(sink_pad->entity)) {
+		struct video_device *vdev;
+
+		vdev = media_entity_to_video_device(sink_pad->entity);
+		if (!vdev)
+			return -ENXIO;
+
+		dev_dbg(vdev->dev_parent,
+			"path_enabled: found video device %s\n",
+			vdev->name);
+
+		if (!*__sink_pad) {
+			*__sink_pad = tmp_pad;
+			dev_dbg(sd->dev, "path_enabled: sink %u/%u\n",
+				tmp_pad->index, sink_stream);
+		} else if (tmp_pad != *__sink_pad) {
+			dev_dbg(sd->dev,
+				"path_enabled: pads %s/%u and %s/%u differ\n",
+				tmp_pad->entity->name, tmp_pad->index,
+				(*__sink_pad)->entity->name,
+				(*__sink_pad)->index);
+			return -EXDEV;
+		}
+
+		*__sink_streams |= BIT_ULL(sink_stream);
+
+		return func(vdev);
+	}
+
+	return v4l2_mc_downpath_enabled(sink_pad, source_stream, func,
+					__sink_pad, __sink_streams);
+}
+
+static int v4l2_mc_source_get_streams(struct v4l2_subdev *sd, unsigned int pad,
+				      u64 *__streams)
+{
+	struct v4l2_mbus_frame_desc desc;
+	u64 streams = 0;
+	int ret;
+
+	if (!__streams)
+		return -EINVAL;
+
+	ret = v4l2_subdev_call(sd, pad, get_frame_desc, pad, &desc);
+	if (ret == -ENOIOCTLCMD) {
+		*__streams = 1ULL;
+		return 0;
+	}
+	if (ret)
+		return ret;
+
+	for (unsigned int i = 0; i < desc.num_entries; i++) {
+		if (streams & BIT_ULL(desc.entry[i].stream))
+			return -EINVAL;
+
+		streams |= BIT_ULL(desc.entry[i].stream);
+	}
+
+	dev_dbg(sd->dev, "found streams %#llx based on streams %#llx\n",
+		*__streams, streams);
+	if (*__streams & ~streams)
+		return -EINVAL;
+
+	*__streams = streams;
+
+	return 0;
+}
+
+int v4l2_mc_pipeline_enabled(struct video_device *vdev,
+			     bool (*func)(struct video_device *vdev),
+			     struct media_pad **__sink_pad, u64 *__sink_streams)
+{
+	u64 sink_streams = 1U;
+	struct media_pad *src_pad;
+	u64 src_streams;
+	struct v4l2_subdev_state *state;
+	struct media_pad *sink_pad = vdev->entity.pads;
+	struct v4l2_subdev *sd = NULL;
+	bool streaming = true;
+	struct media_pad *tmp_pad;
+	u64 tmp_streams;
+	int ret;
+
+	if (!__sink_pad)
+		__sink_pad = &tmp_pad;
+	if (!__sink_streams)
+		__sink_streams = &tmp_streams;
+	*__sink_pad = NULL;
+	*__sink_streams = 0;
+
+	do {
+		src_pad = media_pad_remote_pad_unique(sink_pad);
+		if (IS_ERR(src_pad)) {
+			dev_dbg(sd ? sd->dev : vdev->dev_parent,
+				"no unique remote pad found from %s:%u\n",
+				sink_pad->entity->name, sink_pad->index);
+			return PTR_ERR(src_pad);
+		}
+
+		sd = media_entity_to_v4l2_subdev(src_pad->entity);
+		if (!sd) {
+			dev_dbg(sd->dev,
+				"media entity %s is not a V4L2 sub-device\n",
+				src_pad->entity->name);
+			return -ENXIO;
+		}
+
+		/* Source streams match sink. */
+		src_streams = sink_streams;
+
+		state = v4l2_subdev_lock_and_get_active_state(sd);
+		ret = __v4l2_mc_pipeline_enabled(state, src_pad,
+						 src_streams, &sink_pad,
+						 &sink_streams);
+		v4l2_subdev_unlock_state(state);
+		if (ret)
+			return ret;
+	} while (sink_pad);
+
+	ret = v4l2_mc_source_get_streams(sd, src_pad->index, &src_streams);
+	if (ret)
+		return ret;
+
+	sd = media_entity_to_v4l2_subdev(src_pad->entity);
+
+	dev_dbg(sd->dev, "following %s:%u/%#llx\n", sd->entity.name,
+		src_pad->index, src_streams);
+
+	for (unsigned int i = __ffs(src_streams); src_streams;
+	     src_streams &= ~BIT_ULL(i), i = __ffs(src_streams)) {
+		sink_pad = media_pad_remote_pad_unique(src_pad);
+		if (IS_ERR(src_pad)) {
+			dev_dbg(sd->dev,
+				"no unique remote pad found from %s:%u\n",
+				sink_pad->entity->name, sink_pad->index);
+			return PTR_ERR(src_pad);
+		}
+
+		ret = v4l2_mc_downpath_enabled(sink_pad, i, func, __sink_pad,
+					       __sink_streams);
+		if (ret == 2)
+			continue;
+		if (ret < 0)
+			return ret;
+		if (!ret)
+			streaming = false;
+	}
+
+	dev_dbg(media_entity_to_v4l2_subdev((*__sink_pad)->entity)->dev,
+		"sink pad %s:%u/%#llx\n", (*__sink_pad)->entity->name,
+		(*__sink_pad)->index, *__sink_streams);
+
+	return streaming;
+}
+EXPORT_SYMBOL_GPL(v4l2_mc_pipeline_enabled);
include/media/v4l2-mc.h +0 -0
index 1837c9fd78cf..e72c0f62fa34 100644
@@ -193,6 +193,50 @@ void v4l2_pipeline_pm_put(struct media_entity *entity);
 int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
 			      unsigned int notification);
 
Quoted 8 lines

include/media/v4l2-mc.h void v4l2_pipeline_pm_put(struct media_entity *entity); · line 201

Quoted 6 lines
include/media/v4l2-mc.h +6 -0
+/**
+ * v4l2_mc_pipeline_enabled - Tell when to start streaming
+ * @vdev: The video device
+ * @func: Caller-provided function to tell a video device's streaming state
+ * @__sink_pad: sink pad at the root of the local pipeline
+ * @__sink_streams: streams to start

Any reason for the double underscore ?

Quoted 2 lines

The function used internally a variable without the underscores for a different purpose.

Then it's better to use the __ for internal variables instead of the publicly documented API. Or rename internal variables so they don't clash.

Quoted 37 lines
Quoted 12 lines

include/media/v4l2-mc.h void v4l2_pipeline_pm_put(struct media_entity *entity); · line 212

Quoted 11 lines
include/media/v4l2-mc.h +11 -0
+ *
+ * Use to tell whether streaming should start on a video node. @func returns
+ * true if streaming has been started on a given video node. @__sink_pad and
+ * @__sink_streams are filled with pad and streams on the sub-device closest to
+ * the video nodes, to be used for calling v4l2_subdev_enable_streams() and
+ * v4l2_subdev_disable_streams().
+ *
+ * Using v4l2_mc_pipeline_enabled() has a few limitations currently (consider it
+ * a to-do list):
+ * * only unbranched streams can be supported albeit adding support for
+ *   downstream branches would be fairly trivial,
Quoted 2 lines

I can't tell from the documentation here what you mean exactly by "unbranched streams".

These may have been referred to as "linear" streams elsewhere. I'll address this in the next version.

Quoted 11 lines

include/media/v4l2-mc.h void v4l2_pipeline_pm_put(struct media_entity *entity); · line 222

Quoted 10 lines
include/media/v4l2-mc.h +10 -0
+ * * streams within a single source sub-device are considered to start at the
+ *   same time, more control could be added in two ways: 1) for sources to
+ *   determine stream starting, a control could be added to UAPI and 2) sources
+ *   could tell which streams start at the same time using a sub-device
+ *   operation,
+ * * CSI-2 VC framing is ignored currently, but VC-based stream starting could
+ *   be implemented by letting the caller to provide a function to determine
+ *   which streams are of interest and
+ * * routes leading to nowhere are ignored, on some hardware this is a problem,
+ *   but this can also be rather trivially addressed.
Quoted 7 lines

I'm afraid this function looks like a hack, to solve a problem that is not even explicitly described. You don't explain the issue in the cover letter or in Documentation/, the cover letter merely states that this is a "partial solution". The documentation of the function doesn't explain what criteria the decision is based on. We need a proper explanation of the problem in Documentation/, with a description of the behaviour (or behaviours) drivers are expected to implement.

Quoted 3 lines

I believe this is discussed in the cover letter, not in detail though. I'm fine with adding more documentation, there isn't much as I also wanted to get feedback on the approach itself.

But that's my point, I have trouble giving feedback on the approach if you don't explain what problem you're trying to solve, and why you picked this particular solution. It may be all clear in your head, but it isn't for me.

Quoted 35 lines

No alternatives have been proposed so
far either.

Quoted 6 lines

Furthermore, on the implementation side, things are fairly inefficient. We already traverse the whole pipeline in media_pipeline_start(), based on links and routes, and populate the media_pipeline structure. We shouldn't do the same here, but instead inspect media_pipeline to extract the information we need. If you're missing information there, let's add it.

The media pipeline is created and traversed, yes, but the media pipeline does not include streams which are strictly a V4L2 concept. I agree there is some overlap between the two but as long as MC remains separate from V4L2, we can't reasonably access streams from the pipeline traversal.

Quoted 20 lines
Quoted 20 lines
include/media/v4l2-mc.h +17 -0
+ *
+ * Return:
+ * * 0: Success, but don't start streaming yet
+ * * 1: Success, now it's time to start streaming
+ * * -ENXIO: Route traversal encountered a non-video device/sub-device entity
+ * * -ENOTUNIQ: No unique remote pad
+ * * -ENOLINK: No remote pad found
+ * * -ENOENT: Enabled upstream route not found
+ * * -EMLINK: No unique downstream route found
+ * * -EINVAL: Stream could not be followed to source or was not produced by
+ *            the source
+ */
+int v4l2_mc_pipeline_enabled(struct video_device *vdev,
+			     bool (*func)(struct video_device *vdev),
+			     struct media_pad **__sink_pad,
+			     u64 *__sink_streams);
+
 #else /* CONFIG_MEDIA_CONTROLLER */
 
 static inline int v4l2_mc_create_media_graph(struct media_device *mdev)
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#66 lore raw

PATCH 13/13 media: ipu6: isys: Rework stream starting and stopping

In reply to Sakari Ailus

Rework starting and stopping streams, based on determining which streams are available on the source. Rely on v4l2_mc_pipeline_enabled() to obtain information on when to start streaming.

This effectively both complicates and simplifies driver implementation. There's some cleanup to be done in the driver after this patch, as starting and stopping streaming is currently a quite complicated process.

---
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 38 ++++++++++---
 .../media/pci/intel/ipu6/ipu6-isys-queue.c    | 48 ++++++++++-------
 .../media/pci/intel/ipu6/ipu6-isys-video.c    | 53 ++++++++++++-------
 .../media/pci/intel/ipu6/ipu6-isys-video.h    |  9 ++--
 4 files changed, 99 insertions(+), 49 deletions(-)
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +31 -7
index 3b837e9ccffe..8d57cdb6aeb0 100644
@@ -346,21 +346,31 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
 	struct ipu6_isys_subdev *asd = to_ipu6_isys_subdev(sd);
 	struct ipu6_isys_csi2 *csi2 = to_ipu6_isys_csi2(asd);
 	struct ipu6_isys_csi2_timing timing = { };
+	struct ipu6_isys_video *iv;
+	struct ipu6_isys_stream *stream;
 	struct v4l2_subdev *remote_sd;
-	struct media_pad *remote_pad;
-	u64 sink_streams, already_enabled;
+	struct media_pad *remote_pad, *vdev_pad;
+	u64 sink_streams;
 	int ret;
 
-	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
-	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
+	vdev_pad = media_pad_remote_pad_first(&sd->entity.pads[pad]);
+	iv = container_of_const(media_entity_to_video_device(vdev_pad->entity),
+				struct ipu6_isys_video, vdev);
+	stream = iv->stream;
 
 	sink_streams =
 		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
 						&streams_mask);
 
-	already_enabled = v4l2_subdev_state_streams_enabled(sd, state,
-							    CSI2_PAD_SINK);
-	if (!already_enabled) {
+	if ((sink_streams | stream->streams_enabled) != stream->streams) {
+		stream->streams_enabled |= sink_streams;
+		return 0;
+	}
+
+	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
+	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
+
+	if (!stream->streaming) {
 		ret = ipu6_isys_csi2_calc_timing(csi2, &timing, CSI2_ACCINV);
 		if (ret)
 			return ret;
@@ -371,12 +381,15 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
 	}
 
 	ret = v4l2_subdev_enable_streams(remote_sd, remote_pad->index,
+					 stream->streams_enabled |
 					 sink_streams);
 	if (ret) {
 		ipu6_isys_csi2_set_stream(sd, NULL, 0, false);
 		return ret;
 	}
 
+	stream->streams_enabled |= sink_streams;
+
 	return 0;
 }
 
@@ -402,6 +415,17 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,
 
 	v4l2_subdev_disable_streams(remote_sd, remote_pad->index, sink_streams);
 
+	struct media_pad *vdev_pad;
+	struct ipu6_isys_video *iv;
+	struct ipu6_isys_stream *stream;
+
+	vdev_pad = media_pad_remote_pad_first(&sd->entity.pads[pad]);
+	iv = container_of_const(media_entity_to_video_device(vdev_pad->entity),
+				struct ipu6_isys_video, vdev);
+	stream = iv->stream;
+
+	stream->streams_enabled &= ~sink_streams;
+
 	return 0;
 }
 
drivers/media/pci/intel/ipu6/ipu6-isys-queue.c +29 -19

116 lines not on this page · raw message

drivers/media/pci/intel/ipu6/ipu6-isys-video.c +35 -18

110 lines not on this page · raw message

drivers/media/pci/intel/ipu6/ipu6-isys-video.h +4 -5
index 1dd36f2a077e..6f95544794cf 100644
@@ -44,6 +44,8 @@ struct sequence_info {
 struct ipu6_isys_stream {
 	struct mutex mutex;
 	struct media_entity *source_entity;
+	u64 streams;
+	u64 streams_enabled;
 	atomic_t sequence;
 	unsigned int seq_index;
 	struct sequence_info seq[IPU6_ISYS_MAX_PARALLEL_SOF];
@@ -52,8 +54,6 @@ struct ipu6_isys_stream {
 	unsigned int nr_output_pins;
 	struct ipu6_isys_subdev *asd;
 
-	int nr_queues;	/* Number of capture queues */
-	int nr_streaming;
 	int streaming;	/* Has streaming been really started? */
 	struct list_head queues;
 	struct completion stream_open_completion;
@@ -106,14 +106,13 @@ extern const struct ipu6_isys_pixelformat ipu6_isys_pfmts_packed[];
 const struct ipu6_isys_pixelformat *
 ipu6_isys_get_isys_format(u32 pixelformat, u32 code);
 int ipu6_isys_video_prepare_stream(struct ipu6_isys_video *av,
-				   struct media_entity *source_entity,
-				   int nr_queues);
+				   struct media_entity *source_entity);
 int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 				  struct ipu6_isys_buffer_list *bl);
 int ipu6_isys_fw_open(struct ipu6_isys *isys);
 void ipu6_isys_fw_close(struct ipu6_isys *isys);
 int ipu6_isys_setup_video(struct ipu6_isys_video *av,
-			  struct media_entity **source_entity, int *nr_queues);
+			  struct media_entity **source_entity);
 int ipu6_isys_video_init(struct ipu6_isys_video *av);
 void ipu6_isys_video_cleanup(struct ipu6_isys_video *av);
 void ipu6_isys_put_stream(struct ipu6_isys_stream *stream);
Signature · 1 line
2.39.5