Streaming control for MC with metadata or streams otherwise
Subsystems (2)
Sakari Ailus
maintainercover Streaming control for MC with metadata or streams otherwise
Hi all,
Starting and stopping streaming is a non-trivial problem in all but most simple cases where there's only one stream. But when multiple streams come into play, the problem quickly gets difficult to solve properly as it generally spans multiple hardware deveices, not only including transmitters (e.g. camera sensors) and receivers but also devices in between, such as CSI-2 aggregators and serdes devices.
The following document summarises the known use cases:
<URL:https://pilvi.retiisi.eu/s/44RWYjKG92R6X49>, please let me know if you have something to add that's not included here.
What this patchset implements is a partial solution, but it's also a solution that can be extended to cover the above cases with relatively small amendments.
The current functionality is best documented in patch "media: v4l2-mc:
Introduce v4l2_mc_pipeline_enabled()".
This set, in some form, is also effectively required by metadata support patchset of which I'm about to post a new version. Starting per VC (for CSI-2) is next on my to-do list when it comes to this set.
Getting these both in for 6.18 would be great.
Comments are very welcome.
Sakari Ailus (13):
media: ipu6: Use correct pads for xlate_streams()
media: ipu6: Set minimum height to 1
media: ipu6: Enable and disable each stream at CSI-2 subdev source pad
media: v4l2-subdev: Add a helper to figure out the pad streaming state
media: v4l: Make media_entity_to_video_device() NULL-safe
media: v4l2-subdev: Mark both streams of a route enabled
media: ipu6: Set up CSI-2 receiver at correct moment
media: v4l2-subdev: Print early in
v4l2_subdev_{enable,disable}_streams() media: v4l2-subdev: Collect streams on source pads only
media: v4l2-subdev: Add debug prints to v4l2_subdev_collect_streams()
media: v4l2-subdev: Introduce v4l2_subdev_find_route()
media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()
media: ipu6: isys: Rework stream starting and stopping
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 63 +++-- .../media/pci/intel/ipu6/ipu6-isys-queue.c | 48 ++-- .../media/pci/intel/ipu6/ipu6-isys-video.c | 96 ++++--- .../media/pci/intel/ipu6/ipu6-isys-video.h | 9 +- drivers/media/pci/intel/ipu6/ipu6-isys.h | 2 +- drivers/media/v4l2-core/v4l2-mc.c | 243 ++++++++++++++++++ drivers/media/v4l2-core/v4l2-subdev.c | 107 +++++--- include/media/v4l2-dev.h | 14 +- include/media/v4l2-mc.h | 44 ++++ include/media/v4l2-subdev.h | 22 ++ 10 files changed, 539 insertions(+), 109 deletions(-)
Signature · 1 line
2.39.5
Sakari Ailus
maintainerPATCH 1/13 media: ipu6: Use correct pads for xlate_streams()
The arguments to v4l2_subdev_state_xlate_streams() were incorrect, the source pads was used as the sink pad and the source pad was a constant (rather than the actual source pad). Fix these.
--- drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +6 -6
@@ -354,9 +354,9 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd, remote_pad =media_pad_remote_pad_first (&sd->entity.pads[CSI2_PAD_SINK ]); remote_sd =media_entity_to_v4l2_subdev (remote_pad->entity); - sink_streams =v4l2_subdev_state_xlate_streams (state,CSI2_PAD_SRC , -CSI2_PAD_SINK , - &streams_mask); + sink_streams = +v4l2_subdev_state_xlate_streams (state, pad,CSI2_PAD_SINK , + &streams_mask); ret =ipu6_isys_csi2_calc_timing (csi2, &timing,CSI2_ACCINV );if (ret)
@@ -384,9 +384,9 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,struct media_pad *remote_pad;u64 sink_streams; - sink_streams =v4l2_subdev_state_xlate_streams (state,CSI2_PAD_SRC , -CSI2_PAD_SINK , - &streams_mask); + sink_streams = +v4l2_subdev_state_xlate_streams (state, pad,CSI2_PAD_SINK , + &streams_mask); remote_pad =media_pad_remote_pad_first (&sd->entity.pads[CSI2_PAD_SINK ]); remote_sd =media_entity_to_v4l2_subdev (remote_pad->entity);
Signature · 1 line
2.39.5
Re: [PATCH 01/13] media: ipu6: Use correct pads for xlate_streams()
Hi Sakari,
Thank you for the patch.
Quoted 1 line · Sakari Ailus
The arguments to
v4l2_subdev_state_xlate_streams()were incorrect, the
s/were/are/
Quoted 1 line
source pads was used as the sink pad and the source pad was a constant
s/pads was/pad is/
s/pad was/pad is/
Are you sure though ? Unless I misread the code, you replace
pad0 = CSI2_PAD_SRC
pad1 = CSI2_PAD_SINK
with
pad0 = pad
pad1 = CSI2_PAD_SINK
This seems to be a correct fix, but I don't see where "the source pad was used as the sink pad".
Quoted 14 lines
(rather than the actual source pad). Fix these.
Fixes 3a5c59ad926b ("media: ipu6: Rework CSI-2 sub-device streaming control")Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0
-0
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd, · line 359
Quoted 9 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +3
-3@@ -354,9 +354,9 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd, remote_pad =media_pad_remote_pad_first (&sd->entity.pads[CSI2_PAD_SINK ]); remote_sd =media_entity_to_v4l2_subdev (remote_pad->entity); - sink_streams =v4l2_subdev_state_xlate_streams (state,CSI2_PAD_SRC , -CSI2_PAD_SINK , - &streams_mask); + sink_streams = +v4l2_subdev_state_xlate_streams (state, pad,CSI2_PAD_SINK , + &streams_mask);
This is one of the cases where I'd write
sink_streams = v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK, &streams_mask);
even if it goes to 81 columns.
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd, · line 389
Quoted 13 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +3
-3ret =ipu6_isys_csi2_calc_timing (csi2, &timing,CSI2_ACCINV );if (ret)@@ -384,9 +384,9 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,struct media_pad *remote_pad;u64 sink_streams; - sink_streams =v4l2_subdev_state_xlate_streams (state,CSI2_PAD_SRC , -CSI2_PAD_SINK , - &streams_mask); + sink_streams = +v4l2_subdev_state_xlate_streams (state, pad,CSI2_PAD_SINK , + &streams_mask);
Same here.
Quoted 3 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0
-0remote_pad =media_pad_remote_pad_first (&sd->entity.pads[CSI2_PAD_SINK ]); remote_sd =media_entity_to_v4l2_subdev (remote_pad->entity);
Signature · 3 lines
Regards, Laurent Pinchart
Sakari Ailus
maintainerRe: [PATCH 01/13] media: ipu6: Use correct pads for xlate_streams()
Hi Laurent,
Quoted 3 lines · Laurent Pinchart
Hi Sakari,
Thank you for the patch.
Thanks for the review!
Quoted 16 lines · Sakari Ailus
Quoted 1 line
The arguments to
v4l2_subdev_state_xlate_streams()were incorrect, thes/were/are/
Quoted 1 line
source pads was used as the sink pad and the source pad was a constant
s/pads was/pad is/
s/pad was/pad is/Are you sure though ? Unless I misread the code, you replace
pad0 =
CSI2_PAD_SRC
pad1 =CSI2_PAD_SINK
Quoted 7 lines
with
pad0 = pad
pad1 =CSI2_PAD_SINKThis seems to be a correct fix, but I don't see where "the source pad was used as the sink pad".
Right, I'll reword it for v2.
Quoted 27 lines
Quoted 15 lines
(rather than the actual source pad). Fix these.
Fixes 3a5c59ad926b ("media: ipu6: Rework CSI-2 sub-device streaming control")Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0
-0drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd, · line 359
Quoted 9 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +3
-3@@ -354,9 +354,9 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd, remote_pad =media_pad_remote_pad_first (&sd->entity.pads[CSI2_PAD_SINK ]); remote_sd =media_entity_to_v4l2_subdev (remote_pad->entity); - sink_streams =v4l2_subdev_state_xlate_streams (state,CSI2_PAD_SRC , -CSI2_PAD_SINK , - &streams_mask); + sink_streams = +v4l2_subdev_state_xlate_streams (state, pad,CSI2_PAD_SINK , + &streams_mask);This is one of the cases where I'd write
Quoted 4 lines
sink_streams = v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK, &streams_mask);even if it goes to 81 columns.
The limit is still 80, not 81.
Either way, a number of the new functions in the V4L2 subdev KAPI have very long names, I'll post patches to shorten them later on.
Quoted 21 lines
Quoted 5 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0
-0ret =ipu6_isys_csi2_calc_timing (csi2, &timing,CSI2_ACCINV );if (ret)drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd, · line 389
Quoted 9 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +3
-3@@ -384,9 +384,9 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,struct media_pad *remote_pad;u64 sink_streams; - sink_streams =v4l2_subdev_state_xlate_streams (state,CSI2_PAD_SRC , -CSI2_PAD_SINK , - &streams_mask); + sink_streams = +v4l2_subdev_state_xlate_streams (state, pad,CSI2_PAD_SINK , + &streams_mask);Same here.
Quoted 3 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0
-0remote_pad =media_pad_remote_pad_first (&sd->entity.pads[CSI2_PAD_SINK ]); remote_sd =media_entity_to_v4l2_subdev (remote_pad->entity);
Signature · 3 lines
Regards, Sakari Ailus
Re: [PATCH 01/13] media: ipu6: Use correct pads for xlate_streams()
Quoted 58 lines · Sakari Ailus
Quoted 3 lines · Laurent Pinchart
Hi Sakari,
Thank you for the patch.
Thanks for the review!
Quoted 15 lines · Sakari Ailus
Quoted 1 line
The arguments to
v4l2_subdev_state_xlate_streams()were incorrect, thes/were/are/
Quoted 1 line
source pads was used as the sink pad and the source pad was a constant
s/pads was/pad is/
s/pad was/pad is/Are you sure though ? Unless I misread the code, you replace
pad0 =
CSI2_PAD_SRC
pad1 =CSI2_PAD_SINKQuoted 7 lines
with
pad0 = pad
pad1 =CSI2_PAD_SINKThis seems to be a correct fix, but I don't see where "the source pad was used as the sink pad".
Right, I'll reword it for v2.
Quoted 26 lines
Quoted 14 lines
(rather than the actual source pad). Fix these.
Fixes 3a5c59ad926b ("media: ipu6: Rework CSI-2 sub-device streaming control")Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0
-0drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd, · line 359
Quoted 9 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +3
-3@@ -354,9 +354,9 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd, remote_pad =media_pad_remote_pad_first (&sd->entity.pads[CSI2_PAD_SINK ]); remote_sd =media_entity_to_v4l2_subdev (remote_pad->entity); - sink_streams =v4l2_subdev_state_xlate_streams (state,CSI2_PAD_SRC , -CSI2_PAD_SINK , - &streams_mask); + sink_streams = +v4l2_subdev_state_xlate_streams (state, pad,CSI2_PAD_SINK , + &streams_mask);This is one of the cases where I'd write
Quoted 6 lines
Quoted 4 lines
sink_streams = v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK, &streams_mask);even if it goes to 81 columns.
The limit is still 80, not 81.
The global limit is 100 for the kernel. There's somehow of a consensus in the media subsystem to keep it closer to 80, with different people have different sensitivities. We occasionally go over 80, and that's usually left as a driver maintainer decision.
Quoted 2 lines
Either way, a number of the new functions in the V4L2 subdev KAPI have very long names, I'll post patches to shorten them later on.
As long as the name remains equally readable and understandable, and all functions use the same conventions, I'm fine with that.
Quoted 19 lines
Quoted 19 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd, · line 389
Quoted 13 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +3
-3ret =ipu6_isys_csi2_calc_timing (csi2, &timing,CSI2_ACCINV );if (ret)@@ -384,9 +384,9 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,struct media_pad *remote_pad;u64 sink_streams; - sink_streams =v4l2_subdev_state_xlate_streams (state,CSI2_PAD_SRC , -CSI2_PAD_SINK , - &streams_mask); + sink_streams = +v4l2_subdev_state_xlate_streams (state, pad,CSI2_PAD_SINK , + &streams_mask);Same here.
Quoted 3 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0
-0remote_pad =media_pad_remote_pad_first (&sd->entity.pads[CSI2_PAD_SINK ]); remote_sd =media_entity_to_v4l2_subdev (remote_pad->entity);
Signature · 3 lines
Regards, Laurent Pinchart
Sakari Ailus
maintainerRe: [PATCH 01/13] media: ipu6: Use correct pads for xlate_streams()
Hi Laurent,
Quoted 62 lines · Laurent Pinchart
Quoted 61 lines · Sakari Ailus
Quoted 3 lines · Laurent Pinchart
Hi Sakari,
Thank you for the patch.
Thanks for the review!
Quoted 15 lines · Sakari Ailus
Quoted 1 line
The arguments to
v4l2_subdev_state_xlate_streams()were incorrect, thes/were/are/
Quoted 1 line
source pads was used as the sink pad and the source pad was a constant
s/pads was/pad is/
s/pad was/pad is/Are you sure though ? Unless I misread the code, you replace
pad0 =
CSI2_PAD_SRC
pad1 =CSI2_PAD_SINKQuoted 7 lines
with
pad0 = pad
pad1 =CSI2_PAD_SINKThis seems to be a correct fix, but I don't see where "the source pad was used as the sink pad".
Right, I'll reword it for v2.
Quoted 29 lines
Quoted 14 lines
(rather than the actual source pad). Fix these.
Fixes 3a5c59ad926b ("media: ipu6: Rework CSI-2 sub-device streaming control")Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0
-0drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd, · line 359
Quoted 9 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +3
-3@@ -354,9 +354,9 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd, remote_pad =media_pad_remote_pad_first (&sd->entity.pads[CSI2_PAD_SINK ]); remote_sd =media_entity_to_v4l2_subdev (remote_pad->entity); - sink_streams =v4l2_subdev_state_xlate_streams (state,CSI2_PAD_SRC , -CSI2_PAD_SINK , - &streams_mask); + sink_streams = +v4l2_subdev_state_xlate_streams (state, pad,CSI2_PAD_SINK , + &streams_mask);This is one of the cases where I'd write
sink_streams = v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK, &streams_mask);
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
even if it goes to 81 columns.
The limit is still 80, not 81.
The global limit is 100 for the kernel. There's somehow of a consensus in the media subsystem to keep it closer to 80, with different people have different sensitivities. We occasionally go over 80, and that's usually left as a driver maintainer decision.
It's 80, not 100. The checkpatch.pl limit is higher than 80 though, as there are still commonly cases where the code is more readable with longer lines. See e.g. Documentation/process/coding-style.rst .
Signature · 3 lines
Regards, Sakari Ailus
Re: [PATCH 01/13] media: ipu6: Use correct pads for xlate_streams()
Quoted 61 lines · Sakari Ailus
Quoted 60 lines · Laurent Pinchart
Quoted 58 lines · Sakari Ailus
Quoted 15 lines · Laurent Pinchart
Quoted 1 line · Sakari Ailus
The arguments to
v4l2_subdev_state_xlate_streams()were incorrect, thes/were/are/
Quoted 1 line
source pads was used as the sink pad and the source pad was a constant
s/pads was/pad is/
s/pad was/pad is/Are you sure though ? Unless I misread the code, you replace
pad0 =
CSI2_PAD_SRC
pad1 =CSI2_PAD_SINKQuoted 7 lines
with
pad0 = pad
pad1 =CSI2_PAD_SINKThis seems to be a correct fix, but I don't see where "the source pad was used as the sink pad".
Right, I'll reword it for v2.
Quoted 26 lines
Quoted 14 lines
(rather than the actual source pad). Fix these.
Fixes 3a5c59ad926b ("media: ipu6: Rework CSI-2 sub-device streaming control")Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0
-0drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd, · line 359
Quoted 9 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +3
-3@@ -354,9 +354,9 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd, remote_pad =media_pad_remote_pad_first (&sd->entity.pads[CSI2_PAD_SINK ]); remote_sd =media_entity_to_v4l2_subdev (remote_pad->entity); - sink_streams =v4l2_subdev_state_xlate_streams (state,CSI2_PAD_SRC , -CSI2_PAD_SINK , - &streams_mask); + sink_streams = +v4l2_subdev_state_xlate_streams (state, pad,CSI2_PAD_SINK , + &streams_mask);This is one of the cases where I'd write
Quoted 4 lines
sink_streams = v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK, &streams_mask);even if it goes to 81 columns.
The limit is still 80, not 81.
Quoted 8 lines
Quoted 4 lines
The global limit is 100 for the kernel. There's somehow of a consensus in the media subsystem to keep it closer to 80, with different people have different sensitivities. We occasionally go over 80, and that's usually left as a driver maintainer decision.
It's 80, not 100. The checkpatch.pl limit is higher than 80 though, as there are still commonly cases where the code is more readable with longer lines. See e.g.
Documentation/process/coding-style.rst.
It states
----
The preferred limit on the length of a single line is 80 columns.
Statements longer than 80 columns should be broken into sensible chunks, unless exceeding 80 columns significantly increases readability and does not hide information.
----
In this specific case, going over 80 columns improves readability in my opinion, and doesn't hide information. As I wrote, we treat this as a driver maintainer preference at the moment, so I won't make a call for the ipu6 driver.
Signature · 3 lines
Regards, Laurent Pinchart
Sakari Ailus
maintainerPATCH 2/13 media: ipu6: Set minimum height to 1
In reply to Sakari AilusFor image data generally 2 seems like a minimum height that surely won't cause any issues, but some sensors have metadata the height of which is just one line. Set the minimum height to 1.
--- drivers/media/pci/intel/ipu6/ipu6-isys.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/media/pci/intel/ipu6/ipu6-isys.h +1 -1
@@ -40,7 +40,7 @@ struct ipu6_bus_device;#define IPU6_ISYS_NUM_RECV_QUEUE 1#define IPU6_ISYS_MIN_WIDTH 2U -#define IPU6_ISYS_MIN_HEIGHT 2U +#define IPU6_ISYS_MIN_HEIGHT 1U#define IPU6_ISYS_MAX_WIDTH 4672U#define IPU6_ISYS_MAX_HEIGHT 3416U
Signature · 1 line
2.39.5
Re: [PATCH 02/13] media: ipu6: Set minimum height to 1
Hi Sakari,
Thank you for the patch.
Quoted 5 lines · Sakari Ailus
For image data generally 2 seems like a minimum height that surely won't cause any issues, but some sensors have metadata the height of which is just one line. Set the minimum height to 1.
Signed-off-by Sakari Ailus <[email protected]>
Quoted 17 lines
--- drivers/media/pci/intel/ipu6/ipu6-isys.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/media/pci/intel/ipu6/ipu6-isys.h +1
-1@@ -40,7 +40,7 @@ struct ipu6_bus_device;#define IPU6_ISYS_NUM_RECV_QUEUE 1#define IPU6_ISYS_MIN_WIDTH 2U -#define IPU6_ISYS_MIN_HEIGHT 2U +#define IPU6_ISYS_MIN_HEIGHT 1U#define IPU6_ISYS_MAX_WIDTH 4672U#define IPU6_ISYS_MAX_HEIGHT 3416U
Signature · 3 lines
Regards, Laurent Pinchart
Sakari Ailus
maintainerPATCH 3/13 media: ipu6: Enable and disable each stream at CSI-2 subdev source pad
In reply to Sakari AilusOn UAPI, streaming is started a video device at a time. The IPU6 ISYS driver only starts streaming on the source sub-device when all relevant video devices have been set streaming. This also needs to be reflected in the sub-device pads, hence set them all streaming, one at a time.
--- .../media/pci/intel/ipu6/ipu6-isys-video.c | 43 ++++++++++++------- 1 file changed, 28 insertions(+), 15 deletions(-)
drivers/media/pci/intel/ipu6/ipu6-isys-video.c +28 -15
@@ -990,6 +990,7 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,struct device *dev = &av->isys->adev->auxdev.dev;struct v4l2_subdev *sd;struct media_pad *r_pad; +unsigned int i;u32 sink_pad, sink_stream;u64 r_stream;u64 stream_mask =0 ;
@@ -1018,14 +1019,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,stop_streaming_firmware (av);/* stop sub-device which connects with video */ -dev_dbg (dev,"stream off entity %s pad:%d mask:0x%llx\n" , - sd->name, r_pad->index, stream_mask); - ret =v4l2_subdev_disable_streams (sd, r_pad->index, - stream_mask); -if (ret) { -dev_err (dev,"stream off %s failed with %d\n" , sd->name, - ret); -return ret; +for (i =CSI2_PAD_SRC ; i <NR_OF_CSI2_SRC_PADS ; i++) { +if (!media_pad_pipeline (&sd->entity.pads[i])) +continue ; + ret =v4l2_subdev_disable_streams (sd, i,1U ); +if (ret) { +dev_err (dev,"stream off %s failed with %d\n" , + sd->name, ret); +return ret; + } }close_streaming_firmware (av); }else {
@@ -1036,13 +1038,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state, }/* start sub-device which connects with video */ -dev_dbg (dev,"stream on %s pad %d mask 0x%llx\n" , sd->name, - r_pad->index, stream_mask); - ret =v4l2_subdev_enable_streams (sd, r_pad->index, stream_mask); -if (ret) { -dev_err (dev,"stream on %s failed with %d\n" , sd->name, - ret); - goto out_media_entity_stop_streaming_firmware; +for (i =CSI2_PAD_SRC ; i <NR_OF_CSI2_SRC_PADS ; i++) { +if (!media_pad_pipeline (&sd->entity.pads[i])) +continue ; + ret =v4l2_subdev_enable_streams (sd, i,1U ); +if (ret) { +dev_err (dev,"stream on %s failed with %d\n" , + sd->name, ret); + goto out_media_entity_stop_streaming_firmware; + } } }
@@ -1051,6 +1055,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,return 0 ; out_media_entity_stop_streaming_firmware: +while (i-- >CSI2_PAD_SRC ) { +int ret2; + +if (!media_pad_pipeline (&sd->entity.pads[i])) +continue ; + ret2 =v4l2_subdev_disable_streams (sd, i,1U ); +dev_err (dev,"stream off %s failed with %d\n" , sd->name, ret2); + } +stop_streaming_firmware (av);return ret;
Signature · 1 line
2.39.5
kernel test robot
build botRe: [PATCH 03/13] media: ipu6: Enable and disable each stream at CSI-2 subdev source pad
kernel test robot noticed the following build warnings:
Hi Sakari,
kernel test robot noticed the following build warnings:
[auto build test WARNING on linuxtv-media-pending/master] [also build test WARNING on linus/master v6.16-rc2 next-20250618] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Sakari-Ailus/media-ipu6-Use-correct-pads-for-xlate_streams/20250619-161847 base: https://git.linuxtv.org/media-ci/media-pending.git master patch link: https://lore.kernel.org/r/20250619081546.1582969-4-sakari.ailus%40linux.intel.com patch subject: [PATCH 03/13] media: ipu6: Enable and disable each stream at CSI-2 subdev source pad config: i386-buildonly-randconfig-005-20250619 (https://download.01.org/0day-ci/archive/20250619/[email protected]/config) compiler: clang version 20.1.2 (https://github.com/llvm/llvm-project 58df0ef89dd64126512e4ee27b4ac3fd8ddf6247) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250619/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
>> drivers/media/pci/intel/ipu6/ipu6-isys-video.c:996:6: warning: variable 'stream_mask' set but not used [-Wunused-but-set-variable]
996 | u64 stream_mask = 0;
| ^1 warning generated.
vim +/stream_mask +996 drivers/media/pci/intel/ipu6/ipu6-isys-video.c
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 983
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 984 int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 985 struct ipu6_isys_buffer_list *bl)
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 986 {
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 987 struct v4l2_subdev_krouting *routing;
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 988 struct ipu6_isys_stream *stream = av->stream;
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 989 struct v4l2_subdev_state *subdev_state;
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 990 struct device *dev = &av->isys->adev->auxdev.dev;
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 991 struct v4l2_subdev *sd;
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 992 struct media_pad *r_pad;
e232e72fbaa788 Sakari Ailus 2025-06-19 993 unsigned int i;
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 994 u32 sink_pad, sink_stream;
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 995 u64 r_stream;
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 @996 u64 stream_mask = 0;
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 997 int ret = 0;
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 998
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 999 dev_dbg(dev, "set stream: %d\n", state);
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1000
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1001 if (WARN(!stream->source_entity, "No source entity for stream\n"))
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1002 return -ENODEV;
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1003
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1004 sd = &stream->asd->sd;
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1005 r_pad = media_pad_remote_pad_first(&av->pad);
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1006 r_stream = ipu6_isys_get_src_stream_by_src_pad(sd, r_pad->index);
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1007
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1008 subdev_state = v4l2_subdev_lock_and_get_active_state(sd);
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1009 routing = &subdev_state->routing;
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1010 ret = v4l2_subdev_routing_find_opposite_end(routing, r_pad->index,
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1011 r_stream, &sink_pad,
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1012 &sink_stream);
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1013 v4l2_subdev_unlock_state(subdev_state);
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1014 if (ret)
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1015 return ret;
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1016
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1017 stream_mask = get_stream_mask_by_pipeline(av);
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1018 if (!state) {
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1019 stop_streaming_firmware(av);
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1020
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1021 /* stop sub-device which connects with video */
e232e72fbaa788 Sakari Ailus 2025-06-19 1022 for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
e232e72fbaa788 Sakari Ailus 2025-06-19 1023 if (!media_pad_pipeline(&sd->entity.pads[i]))
e232e72fbaa788 Sakari Ailus 2025-06-19 1024 continue;
e232e72fbaa788 Sakari Ailus 2025-06-19 1025 ret = v4l2_subdev_disable_streams(sd, i, 1U);
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1026 if (ret) {
e232e72fbaa788 Sakari Ailus 2025-06-19 1027 dev_err(dev, "stream off %s failed with %d\n",
e232e72fbaa788 Sakari Ailus 2025-06-19 1028 sd->name, ret);
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1029 return ret;
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1030 }
e232e72fbaa788 Sakari Ailus 2025-06-19 1031 }
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1032 close_streaming_firmware(av);
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1033 } else {
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1034 ret = start_stream_firmware(av, bl);
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1035 if (ret) {
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1036 dev_err(dev, "start stream of firmware failed\n");
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1037 return ret;
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1038 }
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1039
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1040 /* start sub-device which connects with video */
e232e72fbaa788 Sakari Ailus 2025-06-19 1041 for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
e232e72fbaa788 Sakari Ailus 2025-06-19 1042 if (!media_pad_pipeline(&sd->entity.pads[i]))
e232e72fbaa788 Sakari Ailus 2025-06-19 1043 continue;
e232e72fbaa788 Sakari Ailus 2025-06-19 1044 ret = v4l2_subdev_enable_streams(sd, i, 1U);
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1045 if (ret) {
e232e72fbaa788 Sakari Ailus 2025-06-19 1046 dev_err(dev, "stream on %s failed with %d\n",
e232e72fbaa788 Sakari Ailus 2025-06-19 1047 sd->name, ret);
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1048 goto out_media_entity_stop_streaming_firmware;
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1049 }
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1050 }
e232e72fbaa788 Sakari Ailus 2025-06-19 1051 }
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1052
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1053 av->streaming = state;
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1054
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1055 return 0;
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1056
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1057 out_media_entity_stop_streaming_firmware:
e232e72fbaa788 Sakari Ailus 2025-06-19 1058 while (i-- > CSI2_PAD_SRC) {
e232e72fbaa788 Sakari Ailus 2025-06-19 1059 int ret2;
e232e72fbaa788 Sakari Ailus 2025-06-19 1060
e232e72fbaa788 Sakari Ailus 2025-06-19 1061 if (!media_pad_pipeline(&sd->entity.pads[i]))
e232e72fbaa788 Sakari Ailus 2025-06-19 1062 continue;
e232e72fbaa788 Sakari Ailus 2025-06-19 1063 ret2 = v4l2_subdev_disable_streams(sd, i, 1U);
e232e72fbaa788 Sakari Ailus 2025-06-19 1064 dev_err(dev, "stream off %s failed with %d\n", sd->name, ret2);
e232e72fbaa788 Sakari Ailus 2025-06-19 1065 }
e232e72fbaa788 Sakari Ailus 2025-06-19 1066
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1067 stop_streaming_firmware(av);
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1068
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1069 return ret;
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1070 }
3c1dfb5a69cf83 Bingbu Cao 2024-01-31 1071 Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Re: [PATCH 03/13] media: ipu6: Enable and disable each stream at CSI-2 subdev source pad
In reply to Sakari AilusHi Sakari,
Thank you for the patch.
Quoted 4 lines · Sakari Ailus
On UAPI, streaming is started a video device at a time. The IPU6 ISYS driver only starts streaming on the source sub-device when all relevant video devices have been set streaming. This also needs to be reflected in the sub-device pads, hence set them all streaming, one at a time.
Why ? What's wrong with enabling/disabling streams on each pad as the corresponding video device is started/stopped ?
Quoted 80 lines
Signed-off-by Sakari Ailus <[email protected]>--- .../media/pci/intel/ipu6/ipu6-isys-video.c | 43 ++++++++++++------- 1 file changed, 28 insertions(+), 15 deletions(-)drivers/media/pci/intel/ipu6/ipu6-isys-video.c +28
-15@@ -990,6 +990,7 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,struct device *dev = &av->isys->adev->auxdev.dev;struct v4l2_subdev *sd;struct media_pad *r_pad; +unsigned int i;u32 sink_pad, sink_stream;u64 r_stream;u64 stream_mask =0 ;@@ -1018,14 +1019,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,stop_streaming_firmware (av);/* stop sub-device which connects with video */ -dev_dbg (dev,"stream off entity %s pad:%d mask:0x%llx\n" , - sd->name, r_pad->index, stream_mask); - ret =v4l2_subdev_disable_streams (sd, r_pad->index, - stream_mask); -if (ret) { -dev_err (dev,"stream off %s failed with %d\n" , sd->name, - ret); -return ret; +for (i =CSI2_PAD_SRC ; i <NR_OF_CSI2_SRC_PADS ; i++) { +if (!media_pad_pipeline (&sd->entity.pads[i])) +continue ; + ret =v4l2_subdev_disable_streams (sd, i,1U ); +if (ret) { +dev_err (dev,"stream off %s failed with %d\n" , + sd->name, ret); +return ret; + } }close_streaming_firmware (av); }else {@@ -1036,13 +1038,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state, }/* start sub-device which connects with video */ -dev_dbg (dev,"stream on %s pad %d mask 0x%llx\n" , sd->name, - r_pad->index, stream_mask); - ret =v4l2_subdev_enable_streams (sd, r_pad->index, stream_mask); -if (ret) { -dev_err (dev,"stream on %s failed with %d\n" , sd->name, - ret); - goto out_media_entity_stop_streaming_firmware; +for (i =CSI2_PAD_SRC ; i <NR_OF_CSI2_SRC_PADS ; i++) { +if (!media_pad_pipeline (&sd->entity.pads[i])) +continue ; + ret =v4l2_subdev_enable_streams (sd, i,1U ); +if (ret) { +dev_err (dev,"stream on %s failed with %d\n" , + sd->name, ret); + goto out_media_entity_stop_streaming_firmware; + } } }@@ -1051,6 +1055,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,return 0 ; out_media_entity_stop_streaming_firmware: +while (i-- >CSI2_PAD_SRC ) { +int ret2; + +if (!media_pad_pipeline (&sd->entity.pads[i])) +continue ; + ret2 =v4l2_subdev_disable_streams (sd, i,1U ); +dev_err (dev,"stream off %s failed with %d\n" , sd->name, ret2); + } +stop_streaming_firmware (av);return ret;
Signature · 3 lines
Regards, Laurent Pinchart
Sakari Ailus
maintainerRe: [PATCH 03/13] media: ipu6: Enable and disable each stream at CSI-2 subdev source pad
Hi Laurent,
Quoted 4 lines · Laurent Pinchart
Hi Sakari,
Thank you for the patch.
Quoted 8 lines
Quoted 4 lines · Sakari Ailus
On UAPI, streaming is started a video device at a time. The IPU6 ISYS driver only starts streaming on the source sub-device when all relevant video devices have been set streaming. This also needs to be reflected in the sub-device pads, hence set them all streaming, one at a time.
Why ? What's wrong with enabling/disabling streams on each pad as the corresponding video device is started/stopped ?
Streaming is started once all the related video nodes are set streaming, otherwise there's a chance of losing data.
I've squashed this with the last patch after posting the set.
Quoted 82 lines
Quoted 81 lines
Signed-off-by Sakari Ailus <[email protected]>--- .../media/pci/intel/ipu6/ipu6-isys-video.c | 43 ++++++++++++------- 1 file changed, 28 insertions(+), 15 deletions(-)drivers/media/pci/intel/ipu6/ipu6-isys-video.c +28
-15@@ -990,6 +990,7 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,struct device *dev = &av->isys->adev->auxdev.dev;struct v4l2_subdev *sd;struct media_pad *r_pad; +unsigned int i;u32 sink_pad, sink_stream;u64 r_stream;u64 stream_mask =0 ;@@ -1018,14 +1019,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,stop_streaming_firmware (av);/* stop sub-device which connects with video */ -dev_dbg (dev,"stream off entity %s pad:%d mask:0x%llx\n" , - sd->name, r_pad->index, stream_mask); - ret =v4l2_subdev_disable_streams (sd, r_pad->index, - stream_mask); -if (ret) { -dev_err (dev,"stream off %s failed with %d\n" , sd->name, - ret); -return ret; +for (i =CSI2_PAD_SRC ; i <NR_OF_CSI2_SRC_PADS ; i++) { +if (!media_pad_pipeline (&sd->entity.pads[i])) +continue ; + ret =v4l2_subdev_disable_streams (sd, i,1U ); +if (ret) { +dev_err (dev,"stream off %s failed with %d\n" , + sd->name, ret); +return ret; + } }close_streaming_firmware (av); }else {@@ -1036,13 +1038,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state, }/* start sub-device which connects with video */ -dev_dbg (dev,"stream on %s pad %d mask 0x%llx\n" , sd->name, - r_pad->index, stream_mask); - ret =v4l2_subdev_enable_streams (sd, r_pad->index, stream_mask); -if (ret) { -dev_err (dev,"stream on %s failed with %d\n" , sd->name, - ret); - goto out_media_entity_stop_streaming_firmware; +for (i =CSI2_PAD_SRC ; i <NR_OF_CSI2_SRC_PADS ; i++) { +if (!media_pad_pipeline (&sd->entity.pads[i])) +continue ; + ret =v4l2_subdev_enable_streams (sd, i,1U ); +if (ret) { +dev_err (dev,"stream on %s failed with %d\n" , + sd->name, ret); + goto out_media_entity_stop_streaming_firmware; + } } }@@ -1051,6 +1055,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,return 0 ; out_media_entity_stop_streaming_firmware: +while (i-- >CSI2_PAD_SRC ) { +int ret2; + +if (!media_pad_pipeline (&sd->entity.pads[i])) +continue ; + ret2 =v4l2_subdev_disable_streams (sd, i,1U ); +dev_err (dev,"stream off %s failed with %d\n" , sd->name, ret2); + } +stop_streaming_firmware (av);return ret;
Signature · 3 lines
Regards, Sakari Ailus
Re: [PATCH 03/13] media: ipu6: Enable and disable each stream at CSI-2 subdev source pad
Quoted 7 lines · Sakari Ailus
Quoted 6 lines · Laurent Pinchart
Quoted 4 lines · Sakari Ailus
On UAPI, streaming is started a video device at a time. The IPU6 ISYS driver only starts streaming on the source sub-device when all relevant video devices have been set streaming. This also needs to be reflected in the sub-device pads, hence set them all streaming, one at a time.
Quoted 5 lines
Quoted 2 lines
Why ? What's wrong with enabling/disabling streams on each pad as the corresponding video device is started/stopped ?
Streaming is started once all the related video nodes are set streaming, otherwise there's a chance of losing data.
Yes, but that's something that can be handled in the subdev, to only start the ISYS and its source when all the streams that are part of the pipeline have been started.
Quoted 1 line
I've squashed this with the last patch after posting the set.
Maybe it will become clearer when I review the rest of the series.
Quoted 80 lines
Quoted 80 lines
Quoted 80 lines
Signed-off-by Sakari Ailus <[email protected]>--- .../media/pci/intel/ipu6/ipu6-isys-video.c | 43 ++++++++++++------- 1 file changed, 28 insertions(+), 15 deletions(-)drivers/media/pci/intel/ipu6/ipu6-isys-video.c +28
-15@@ -990,6 +990,7 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,struct device *dev = &av->isys->adev->auxdev.dev;struct v4l2_subdev *sd;struct media_pad *r_pad; +unsigned int i;u32 sink_pad, sink_stream;u64 r_stream;u64 stream_mask =0 ;@@ -1018,14 +1019,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,stop_streaming_firmware (av);/* stop sub-device which connects with video */ -dev_dbg (dev,"stream off entity %s pad:%d mask:0x%llx\n" , - sd->name, r_pad->index, stream_mask); - ret =v4l2_subdev_disable_streams (sd, r_pad->index, - stream_mask); -if (ret) { -dev_err (dev,"stream off %s failed with %d\n" , sd->name, - ret); -return ret; +for (i =CSI2_PAD_SRC ; i <NR_OF_CSI2_SRC_PADS ; i++) { +if (!media_pad_pipeline (&sd->entity.pads[i])) +continue ; + ret =v4l2_subdev_disable_streams (sd, i,1U ); +if (ret) { +dev_err (dev,"stream off %s failed with %d\n" , + sd->name, ret); +return ret; + } }close_streaming_firmware (av); }else {@@ -1036,13 +1038,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state, }/* start sub-device which connects with video */ -dev_dbg (dev,"stream on %s pad %d mask 0x%llx\n" , sd->name, - r_pad->index, stream_mask); - ret =v4l2_subdev_enable_streams (sd, r_pad->index, stream_mask); -if (ret) { -dev_err (dev,"stream on %s failed with %d\n" , sd->name, - ret); - goto out_media_entity_stop_streaming_firmware; +for (i =CSI2_PAD_SRC ; i <NR_OF_CSI2_SRC_PADS ; i++) { +if (!media_pad_pipeline (&sd->entity.pads[i])) +continue ; + ret =v4l2_subdev_enable_streams (sd, i,1U ); +if (ret) { +dev_err (dev,"stream on %s failed with %d\n" , + sd->name, ret); + goto out_media_entity_stop_streaming_firmware; + } } }@@ -1051,6 +1055,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,return 0 ; out_media_entity_stop_streaming_firmware: +while (i-- >CSI2_PAD_SRC ) { +int ret2; + +if (!media_pad_pipeline (&sd->entity.pads[i])) +continue ; + ret2 =v4l2_subdev_disable_streams (sd, i,1U ); +dev_err (dev,"stream off %s failed with %d\n" , sd->name, ret2); + } +stop_streaming_firmware (av);return ret;
Signature · 3 lines
Regards, Laurent Pinchart
Sakari Ailus
maintainerRe: [PATCH 03/13] media: ipu6: Enable and disable each stream at CSI-2 subdev source pad
Quoted 11 lines · Laurent Pinchart
Quoted 10 lines · Sakari Ailus
Quoted 8 lines · Laurent Pinchart
Quoted 4 lines · Sakari Ailus
On UAPI, streaming is started a video device at a time. The IPU6 ISYS driver only starts streaming on the source sub-device when all relevant video devices have been set streaming. This also needs to be reflected in the sub-device pads, hence set them all streaming, one at a time.
Why ? What's wrong with enabling/disabling streams on each pad as the corresponding video device is started/stopped ?
Quoted 6 lines
Quoted 2 lines
Streaming is started once all the related video nodes are set streaming, otherwise there's a chance of losing data.
Yes, but that's something that can be handled in the subdev, to only start the ISYS and its source when all the streams that are part of the pipeline have been started.
I agree, this is done in the CSI-2 receiver driver actually.
There's some work to do in cleaning up the IPU6 driver. The combination of videobuf2 and streams isn't very driver-friendly currently. Things generally need to happen in certain order and this may not be always veyr convenient for drivers.
Quoted 86 lines
Quoted 2 lines
I've squashed this with the last patch after posting the set.
Maybe it will become clearer when I review the rest of the series.
Quoted 80 lines
Quoted 80 lines
Quoted 80 lines
Signed-off-by Sakari Ailus <[email protected]>--- .../media/pci/intel/ipu6/ipu6-isys-video.c | 43 ++++++++++++------- 1 file changed, 28 insertions(+), 15 deletions(-)drivers/media/pci/intel/ipu6/ipu6-isys-video.c +28
-15@@ -990,6 +990,7 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,struct device *dev = &av->isys->adev->auxdev.dev;struct v4l2_subdev *sd;struct media_pad *r_pad; +unsigned int i;u32 sink_pad, sink_stream;u64 r_stream;u64 stream_mask =0 ;@@ -1018,14 +1019,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,stop_streaming_firmware (av);/* stop sub-device which connects with video */ -dev_dbg (dev,"stream off entity %s pad:%d mask:0x%llx\n" , - sd->name, r_pad->index, stream_mask); - ret =v4l2_subdev_disable_streams (sd, r_pad->index, - stream_mask); -if (ret) { -dev_err (dev,"stream off %s failed with %d\n" , sd->name, - ret); -return ret; +for (i =CSI2_PAD_SRC ; i <NR_OF_CSI2_SRC_PADS ; i++) { +if (!media_pad_pipeline (&sd->entity.pads[i])) +continue ; + ret =v4l2_subdev_disable_streams (sd, i,1U ); +if (ret) { +dev_err (dev,"stream off %s failed with %d\n" , + sd->name, ret); +return ret; + } }close_streaming_firmware (av); }else {@@ -1036,13 +1038,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state, }/* start sub-device which connects with video */ -dev_dbg (dev,"stream on %s pad %d mask 0x%llx\n" , sd->name, - r_pad->index, stream_mask); - ret =v4l2_subdev_enable_streams (sd, r_pad->index, stream_mask); -if (ret) { -dev_err (dev,"stream on %s failed with %d\n" , sd->name, - ret); - goto out_media_entity_stop_streaming_firmware; +for (i =CSI2_PAD_SRC ; i <NR_OF_CSI2_SRC_PADS ; i++) { +if (!media_pad_pipeline (&sd->entity.pads[i])) +continue ; + ret =v4l2_subdev_enable_streams (sd, i,1U ); +if (ret) { +dev_err (dev,"stream on %s failed with %d\n" , + sd->name, ret); + goto out_media_entity_stop_streaming_firmware; + } } }@@ -1051,6 +1055,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,return 0 ; out_media_entity_stop_streaming_firmware: +while (i-- >CSI2_PAD_SRC ) { +int ret2; + +if (!media_pad_pipeline (&sd->entity.pads[i])) +continue ; + ret2 =v4l2_subdev_disable_streams (sd, i,1U ); +dev_err (dev,"stream off %s failed with %d\n" , sd->name, ret2); + } +stop_streaming_firmware (av);return ret;
Signature · 3 lines
Regards, Sakari Ailus
Sakari Ailus
maintainerPATCH 4/13 media: v4l2-subdev: Add a helper to figure out the pad streaming state
In reply to Sakari AilusAdd a helper, v4l2_subdev_state_streams_enabled(), to tell which streams are enabled on a pad. This is useful to e.g. figure out in a driver whether a hardware configuration change is necessary to enable a given stream.
--- drivers/media/v4l2-core/v4l2-subdev.c | 17 +++++++++++++++++ include/media/v4l2-subdev.h | 3 +++ 2 files changed, 20 insertions(+)
drivers/media/v4l2-core/v4l2-subdev.c +17 -0
@@ -2453,6 +2453,23 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad, }EXPORT_SYMBOL_GPL (v4l2_subdev_disable_streams); +u64 v4l2_subdev_state_streams_enabled (struct v4l2_subdev *sd, +struct v4l2_subdev_state *state,u32 pad) +{ +u64 streams_mask =0 ; + +for (unsigned int i =0 ; i < state->stream_configs.num_configs; i++) { +struct v4l2_subdev_stream_config *cfg = + &state->stream_configs.configs[i]; + +if (cfg->pad == pad && cfg->enabled) + streams_mask |=BIT_ULL (cfg->stream); + } + +return streams_mask; +} +EXPORT_SYMBOL_GPL (v4l2_subdev_state_streams_enabled); +int v4l2_subdev_s_stream_helper (struct v4l2_subdev *sd,int enable) {struct v4l2_subdev_state *state;
include/media/v4l2-subdev.h +3 -0
@@ -1731,6 +1731,9 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,int v4l2_subdev_disable_streams (struct v4l2_subdev *sd,u32 pad,u64 streams_mask); +u64 v4l2_subdev_state_streams_enabled (struct v4l2_subdev *sd, +struct v4l2_subdev_state *state,u32 pad); +/** * v4l2_subdev_s_stream_helper() - Helper to implement the subdev s_stream * operation using enable_streams and disable_streams
Signature · 1 line
2.39.5
Re: [PATCH 04/13] media: v4l2-subdev: Add a helper to figure out the pad streaming state
Hi Sakari,
Thank you for the patch.
Quoted 16 lines · Sakari Ailus
Add a helper,
v4l2_subdev_state_streams_enabled(), to tell which streams are enabled on a pad. This is useful to e.g. figure out in a driver whether a hardware configuration change is necessary to enable a given stream.Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/v4l2-core/v4l2-subdev.c | 17 +++++++++++++++++ include/media/v4l2-subdev.h | 3 +++ 2 files changed, 20 insertions(+)drivers/media/v4l2-core/v4l2-subdev.c +0
-0
drivers/media/v4l2-core/v4l2-subdev.c int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad, · line 2457
Quoted 5 lines
drivers/media/v4l2-core/v4l2-subdev.c +2
-0@@ -2453,6 +2453,23 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad, }EXPORT_SYMBOL_GPL (v4l2_subdev_disable_streams); +u64 v4l2_subdev_state_streams_enabled (struct v4l2_subdev *sd, +struct v4l2_subdev_state *state,u32 pad)
You can make the state const.
drivers/media/v4l2-core/v4l2-subdev.c int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad, · line 2462
Quoted 5 lines
drivers/media/v4l2-core/v4l2-subdev.c +5
-0+{ +u64 streams_mask =0 ; + +for (unsigned int i =0 ; i < state->stream_configs.num_configs; i++) { + struct v4l2_subdev_stream_config *cfg =
const
Quoted 18 lines
drivers/media/v4l2-core/v4l2-subdev.c +10
-0+ &state->stream_configs.configs[i]; + +if (cfg->pad == pad && cfg->enabled) + streams_mask |=BIT_ULL (cfg->stream); + } + +return streams_mask; +} +EXPORT_SYMBOL_GPL (v4l2_subdev_state_streams_enabled); +int v4l2_subdev_s_stream_helper (struct v4l2_subdev *sd,int enable) {struct v4l2_subdev_state *state;include/media/v4l2-subdev.h +0
-0
include/media/v4l2-subdev.h int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad, · line 1733
Quoted 3 lines
include/media/v4l2-subdev.h +0
-0@@ -1731,6 +1731,9 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,int v4l2_subdev_disable_streams (struct v4l2_subdev *sd,u32 pad,u64 streams_mask);
Missing documentation.
The function looks fine. It's a bit costly though, so I'll see in subsequent patches if there could be a better way to handle whatever issue this solves.
Quoted 6 lines
include/media/v4l2-subdev.h +3
-0+u64 v4l2_subdev_state_streams_enabled (struct v4l2_subdev *sd, +struct v4l2_subdev_state *state,u32 pad); +/** * v4l2_subdev_s_stream_helper() - Helper to implement the subdev s_stream * operation using enable_streams and disable_streams
Signature · 3 lines
Regards, Laurent Pinchart
Sakari Ailus
maintainerPATCH 5/13 media: v4l: Make media_entity_to_video_device() NULL-safe
In reply to Sakari AilusMake media_entity_to_video_device(NULL) return NULL, instead of an invalid pointer value.
--- include/media/v4l2-dev.h | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)
include/media/v4l2-dev.h +10 -4
@@ -313,10 +313,16 @@ struct video_device {* media_entity_to_video_device - Returns a &struct video_device from * the &struct media_entity embedded on it. * -* @__entity: pointer to &struct media_entity -*/ -#define media_entity_to_video_device (__entity) \ - container_of(__entity, struct video_device, entity) +* @__entity: pointer to &struct media_entity, may be NULL +*/ +#define media_entity_to_video_device (__entity) \ + ({ \ + typeof (__entity) __me_to_vdev_ent = __entity; \ + \ + __me_to_vdev_ent ? container_of(__me_to_vdev_ent, \ + struct video_device, entity) : \ + NULL; \ + })/** * to_video_device - Returns a &struct video_device from the
Signature · 1 line
2.39.5
Re: [PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe
Hi Sakari,
Thank you for the patch.
Quoted 20 lines · Sakari Ailus
Make
media_entity_to_video_device(NULL)return NULL, instead of an invalid pointer value.Signed-off-by Sakari Ailus <[email protected]>--- include/media/v4l2-dev.h | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)include/media/v4l2-dev.h +0
-4@@ -313,10 +313,16 @@ struct video_device {* media_entity_to_video_device - Returns a &struct video_device from * the &struct media_entity embedded on it. * -* @__entity: pointer to &struct media_entity -*/ -#define media_entity_to_video_device (__entity) \ - container_of(__entity, struct video_device, entity)
include/media/v4l2-dev.h struct video_device { · line 325
Quoted 10 lines
include/media/v4l2-dev.h +10
-0+* @__entity: pointer to &struct media_entity, may be NULL +*/ +#define media_entity_to_video_device (__entity) \ + ({ \ + typeof (__entity) __me_to_vdev_ent = __entity; \ + \ + __me_to_vdev_ent ? container_of(__me_to_vdev_ent, \ + struct video_device, entity) : \ + NULL; \ + })
This makes the macro safer, it's a good idea. Wouldn't it be better implemented as a container_of_null() (name to be bikeshedded) though ? I don't think media_entity_to_video_device() is the only macro that could benefit from this. It could even be integrated in container_of(), but I fear that could introduce issues.
Quoted 3 lines
include/media/v4l2-dev.h +0
-0/** * to_video_device - Returns a &struct video_device from the
Signature · 3 lines
Regards, Laurent Pinchart
Sakari Ailus
maintainerRe: [PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe
Hi Laurent,
Quoted 33 lines · Laurent Pinchart
Hi Sakari,
Thank you for the patch.
Quoted 28 lines · Sakari Ailus
Make
media_entity_to_video_device(NULL)return NULL, instead of an invalid pointer value.Signed-off-by Sakari Ailus <[email protected]>--- include/media/v4l2-dev.h | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)include/media/v4l2-dev.h +8
-4@@ -313,10 +313,16 @@ struct video_device {* media_entity_to_video_device - Returns a &struct video_device from * the &struct media_entity embedded on it. * -* @__entity: pointer to &struct media_entity -*/ -#define media_entity_to_video_device (__entity) \ - container_of(__entity, struct video_device, entity) +* @__entity: pointer to &struct media_entity, may be NULL +*/ +#define media_entity_to_video_device (__entity) \ + ({ \ + typeof (__entity) __me_to_vdev_ent = __entity; \ + \ + __me_to_vdev_ent ? container_of(__me_to_vdev_ent, \ + struct video_device, entity) : \
Quoted 8 lines
include/media/v4l2-dev.h struct video_device { · line 325
Quoted 2 lines
include/media/v4l2-dev.h +2
-0+NULL ; \ + })This makes the macro safer, it's a good idea. Wouldn't it be better implemented as a
container_of_null()(name to be bikeshedded) though ? I don't thinkmedia_entity_to_video_device()is the only macro that could benefit from this. It could even be integrated incontainer_of(), but I fear that could introduce issues.
That sounds like a good idea. I'll first see how this would look like with container_of_const()...
Signature · 3 lines
Regards, Sakari Ailus
Re: [PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe
Quoted 39 lines · Sakari Ailus
Quoted 29 lines · Laurent Pinchart
Quoted 28 lines · Sakari Ailus
Make
media_entity_to_video_device(NULL)return NULL, instead of an invalid pointer value.Signed-off-by Sakari Ailus <[email protected]>--- include/media/v4l2-dev.h | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)include/media/v4l2-dev.h +8
-4@@ -313,10 +313,16 @@ struct video_device {* media_entity_to_video_device - Returns a &struct video_device from * the &struct media_entity embedded on it. * -* @__entity: pointer to &struct media_entity -*/ -#define media_entity_to_video_device (__entity) \ - container_of(__entity, struct video_device, entity) +* @__entity: pointer to &struct media_entity, may be NULL +*/ +#define media_entity_to_video_device (__entity) \ + ({ \ + typeof (__entity) __me_to_vdev_ent = __entity; \ + \ + __me_to_vdev_ent ? container_of(__me_to_vdev_ent, \ + struct video_device, entity) : \Quoted 8 lines
include/media/v4l2-dev.h struct video_device { · line 325
Quoted 2 lines
include/media/v4l2-dev.h +2
-0+NULL ; \ + })This makes the macro safer, it's a good idea. Wouldn't it be better implemented as a
container_of_null()(name to be bikeshedded) though ? I don't thinkmedia_entity_to_video_device()is the only macro that could benefit from this. It could even be integrated incontainer_of(), but I fear that could introduce issues.
Quoted 2 lines
That sounds like a good idea. I'll first see how this would look like with
container_of_const()...
Thinking some more about this, I think we can move forward without waiting for container_of_null().
Should we however add a check to ensure the entity is a video device (by checking that the function is MEDIA_ENT_F_IO_V4L), and return NULL if it's not ? It would make the macro even safer to use. There would be a small additional runtime cost for call sites that guarantee the entity is a video device.
I checked the current users of the macro, and the vast majority of them are in the .link_validate() operation, where they know that the entity is a video device. There are just a handful of locations where a check precedes the media_entity_to_video_device() call. So maybe it's not worth it ?
I also found no caller that checks for entity != NULL before calling the macro. Is this change actually needed ?
Signature · 3 lines
Regards, Laurent Pinchart
Sakari Ailus
maintainerRe: [PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe
Hi Laurent,
Quoted 52 lines · Laurent Pinchart
Quoted 39 lines · Sakari Ailus
Quoted 29 lines · Laurent Pinchart
Quoted 28 lines · Sakari Ailus
Make
media_entity_to_video_device(NULL)return NULL, instead of an invalid pointer value.Signed-off-by Sakari Ailus <[email protected]>--- include/media/v4l2-dev.h | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)include/media/v4l2-dev.h +8
-4@@ -313,10 +313,16 @@ struct video_device {* media_entity_to_video_device - Returns a &struct video_device from * the &struct media_entity embedded on it. * -* @__entity: pointer to &struct media_entity -*/ -#define media_entity_to_video_device (__entity) \ - container_of(__entity, struct video_device, entity) +* @__entity: pointer to &struct media_entity, may be NULL +*/ +#define media_entity_to_video_device (__entity) \ + ({ \ + typeof (__entity) __me_to_vdev_ent = __entity; \ + \ + __me_to_vdev_ent ? container_of(__me_to_vdev_ent, \ + struct video_device, entity) : \Quoted 8 lines
include/media/v4l2-dev.h struct video_device { · line 325
Quoted 2 lines
include/media/v4l2-dev.h +2
-0+NULL ; \ + })This makes the macro safer, it's a good idea. Wouldn't it be better implemented as a
container_of_null()(name to be bikeshedded) though ? I don't thinkmedia_entity_to_video_device()is the only macro that could benefit from this. It could even be integrated incontainer_of(), but I fear that could introduce issues.Quoted 2 lines
That sounds like a good idea. I'll first see how this would look like with
container_of_const()...Thinking some more about this, I think we can move forward without waiting for
container_of_null().Should we however add a check to ensure the entity is a video device (by checking that the function is
MEDIA_ENT_F_IO_V4L), and return NULL if it's not ? It would make the macro even safer to use. There would be a small additional runtime cost for call sites that guarantee the entity is a video device.
Quoted 8 lines
I checked the current users of the macro, and the vast majority of them are in the .link_validate() operation, where they know that the entity is a video device. There are just a handful of locations where a check precedes the
media_entity_to_video_device()call. So maybe it's not worth it ?I also found no caller that checks for entity != NULL before calling the macro. Is this change actually needed ?
We have a similar check in media_entity_to_v4l2_subdev() macro. In the patches for streaming control there's a need for an explicit check without this patch. That's of course fine as well if we decide so.
Signature · 3 lines
Regards, Sakari Ailus
Laurent Pinchart
Reviewed-byRe: [PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe
Quoted 21 lines · Sakari Ailus
Quoted 20 lines · Laurent Pinchart
Quoted 19 lines · Sakari Ailus
Quoted 18 lines · Laurent Pinchart
Quoted 17 lines · Sakari Ailus
Make
media_entity_to_video_device(NULL)return NULL, instead of an invalid pointer value.Signed-off-by Sakari Ailus <[email protected]>--- include/media/v4l2-dev.h | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)include/media/v4l2-dev.h +0
-1@@ -313,10 +313,16 @@ struct video_device {* media_entity_to_video_device - Returns a &struct video_device from * the &struct media_entity embedded on it. * -* @__entity: pointer to &struct media_entity
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
include/media/v4l2-dev.h struct video_device { · line 320
Quoted 8 lines
include/media/v4l2-dev.h +5
-3-*/ -#define media_entity_to_video_device (__entity) \ - container_of(__entity, struct video_device, entity) +* @__entity: pointer to &struct media_entity, may be NULL +*/ +#define media_entity_to_video_device (__entity) \ + ({ \ + typeof (__entity) __me_to_vdev_ent = __entity; \
This should be __me_vdev_ent to align the naming with media_entity_to_v4l2_subdev().
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
include/media/v4l2-dev.h struct video_device { · line 323
Quoted 3 lines
include/media/v4l2-dev.h +3
-0+ \ + __me_to_vdev_ent ?container_of (__me_to_vdev_ent, \ + struct video_device, entity) : \
And here you should write
__me_to_vdev_ent ? \ container_of(__me_vdev_ent, struct video_device, entity) : \ NULL; \
Quoted 27 lines
Quoted 27 lines
Quoted 11 lines
Quoted 8 lines
include/media/v4l2-dev.h struct video_device { · line 325
Quoted 2 lines
include/media/v4l2-dev.h +2
-0+NULL ; \ + })This makes the macro safer, it's a good idea. Wouldn't it be better implemented as a
container_of_null()(name to be bikeshedded) though ? I don't thinkmedia_entity_to_video_device()is the only macro that could benefit from this. It could even be integrated incontainer_of(), but I fear that could introduce issues.That sounds like a good idea. I'll first see how this would look like with
container_of_const()...Thinking some more about this, I think we can move forward without waiting for
container_of_null().Should we however add a check to ensure the entity is a video device (by checking that the function is
MEDIA_ENT_F_IO_V4L), and return NULL if it's not ? It would make the macro even safer to use. There would be a small additional runtime cost for call sites that guarantee the entity is a video device.I checked the current users of the macro, and the vast majority of them are in the .link_validate() operation, where they know that the entity is a video device. There are just a handful of locations where a check precedes the
media_entity_to_video_device()call. So maybe it's not worth it ?
Quoted 4 lines
Quoted 2 lines
I also found no caller that checks for entity != NULL before calling the macro. Is this change actually needed ?
We have a similar check in
media_entity_to_v4l2_subdev()macro.
That's a good point. Different behaviours for the two macros would be confusing I suppose, so
I'm tempted to convert the macro to an inline function, but that would make it more difficult to switch to container_of_const().
Quoted 3 lines
In the
patches for streaming control there's a need for an explicit check without this patch. That's of course fine as well if we decide so.
Signature · 3 lines
Regards, Laurent Pinchart
Sakari Ailus
maintainerRe: [PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe
Hi Laurent,
Quoted 40 lines · Laurent Pinchart
Quoted 21 lines · Sakari Ailus
Quoted 20 lines · Laurent Pinchart
Quoted 19 lines · Sakari Ailus
Quoted 18 lines · Laurent Pinchart
Quoted 17 lines · Sakari Ailus
Make
media_entity_to_video_device(NULL)return NULL, instead of an invalid pointer value.Signed-off-by Sakari Ailus <[email protected]>--- include/media/v4l2-dev.h | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)include/media/v4l2-dev.h +0
-1@@ -313,10 +313,16 @@ struct video_device {* media_entity_to_video_device - Returns a &struct video_device from * the &struct media_entity embedded on it. * -* @__entity: pointer to &struct media_entity Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
include/media/v4l2-dev.h struct video_device { · line 320
Quoted 8 lines
include/media/v4l2-dev.h +5
-3-*/ -#define media_entity_to_video_device (__entity) \ - container_of(__entity, struct video_device, entity) +* @__entity: pointer to &struct media_entity, may be NULL +*/ +#define media_entity_to_video_device (__entity) \ + ({ \ + typeof (__entity) __me_to_vdev_ent = __entity; \This should be
__me_vdev_entto align the naming withmedia_entity_to_v4l2_subdev().Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
include/media/v4l2-dev.h struct video_device { · line 323
Quoted 3 lines
include/media/v4l2-dev.h +3
-0+ \ + __me_to_vdev_ent ?container_of (__me_to_vdev_ent, \ + struct video_device, entity) : \And here you should write
Quoted 3 lines
__me_to_vdev_ent ? \ container_of(__me_vdev_ent, struct video_device, entity) : \ NULL; \
I'm fine wrap after '?' as well, but another wrap is needed due to indentation in any case.
Quoted 31 lines
Quoted 31 lines
Quoted 30 lines
Quoted 10 lines
Quoted 9 lines
include/media/v4l2-dev.h struct video_device { · line 325
Quoted 3 lines
include/media/v4l2-dev.h +2
-0+NULL ; \ + })This makes the macro safer, it's a good idea. Wouldn't it be better implemented as a
container_of_null()(name to be bikeshedded) though ? I don't thinkmedia_entity_to_video_device()is the only macro that could benefit from this. It could even be integrated incontainer_of(), but I fear that could introduce issues.Quoted 2 lines
That sounds like a good idea. I'll first see how this would look like with
container_of_const()...Thinking some more about this, I think we can move forward without waiting for
container_of_null().Should we however add a check to ensure the entity is a video device (by checking that the function is
MEDIA_ENT_F_IO_V4L), and return NULL if it's not ? It would make the macro even safer to use. There would be a small additional runtime cost for call sites that guarantee the entity is a video device.I checked the current users of the macro, and the vast majority of them are in the .link_validate() operation, where they know that the entity is a video device. There are just a handful of locations where a check precedes the
media_entity_to_video_device()call. So maybe it's not worth it ?I also found no caller that checks for entity != NULL before calling the macro. Is this change actually needed ?
Quoted 6 lines
Quoted 1 line
We have a similar check in
media_entity_to_v4l2_subdev()macro.That's a good point. Different behaviours for the two macros would be confusing I suppose, so
Reviewed-by Laurent Pinchart <[email protected]>
Thank you.
Quoted 3 lines
I'm tempted to convert the macro to an inline function, but that would make it more difficult to switch to
container_of_const().
Let's not do that. Eventually container_of() should become what container_of_const() is at the moment.
But once this is in, I can introduce container_of_null() which we could later use. I wonder how it will look like.
Signature · 3 lines
Regards, Sakari Ailus
Re: [PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe
Quoted 43 lines · Sakari Ailus
Hi Laurent,
Quoted 40 lines · Laurent Pinchart
Quoted 21 lines · Sakari Ailus
Quoted 20 lines · Laurent Pinchart
Quoted 19 lines · Sakari Ailus
Quoted 18 lines · Laurent Pinchart
Quoted 17 lines · Sakari Ailus
Make
media_entity_to_video_device(NULL)return NULL, instead of an invalid pointer value.Signed-off-by Sakari Ailus <[email protected]>--- include/media/v4l2-dev.h | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)include/media/v4l2-dev.h +0
-1@@ -313,10 +313,16 @@ struct video_device {* media_entity_to_video_device - Returns a &struct video_device from * the &struct media_entity embedded on it. * -* @__entity: pointer to &struct media_entity Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
include/media/v4l2-dev.h struct video_device { · line 320
Quoted 8 lines
include/media/v4l2-dev.h +5
-3-*/ -#define media_entity_to_video_device (__entity) \ - container_of(__entity, struct video_device, entity) +* @__entity: pointer to &struct media_entity, may be NULL +*/ +#define media_entity_to_video_device (__entity) \ + ({ \ + typeof (__entity) __me_to_vdev_ent = __entity; \This should be
__me_vdev_entto align the naming withmedia_entity_to_v4l2_subdev().Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
include/media/v4l2-dev.h struct video_device { · line 323
Quoted 3 lines
include/media/v4l2-dev.h +3
-0+ \ + __me_to_vdev_ent ?container_of (__me_to_vdev_ent, \ + struct video_device, entity) : \And here you should write
Quoted 6 lines
Quoted 3 lines
__me_to_vdev_ent ? \ container_of(__me_vdev_ent, struct video_device, entity) : \ NULL; \I'm fine wrap after '?' as well, but another wrap is needed due to indentation in any case.
I didn't make it clear, my comment was about replicating the style of media_entity_to_v4l2_subdev().
Quoted 47 lines
Quoted 30 lines
Quoted 30 lines
Quoted 29 lines
Quoted 11 lines
Quoted 8 lines
include/media/v4l2-dev.h struct video_device { · line 325
Quoted 2 lines
include/media/v4l2-dev.h +2
-0+NULL ; \ + })This makes the macro safer, it's a good idea. Wouldn't it be better implemented as a
container_of_null()(name to be bikeshedded) though ? I don't thinkmedia_entity_to_video_device()is the only macro that could benefit from this. It could even be integrated incontainer_of(), but I fear that could introduce issues.That sounds like a good idea. I'll first see how this would look like with
container_of_const()...Thinking some more about this, I think we can move forward without waiting for
container_of_null().Should we however add a check to ensure the entity is a video device (by checking that the function is
MEDIA_ENT_F_IO_V4L), and return NULL if it's not ? It would make the macro even safer to use. There would be a small additional runtime cost for call sites that guarantee the entity is a video device.I checked the current users of the macro, and the vast majority of them are in the .link_validate() operation, where they know that the entity is a video device. There are just a handful of locations where a check precedes the
media_entity_to_video_device()call. So maybe it's not worth it ?I also found no caller that checks for entity != NULL before calling the macro. Is this change actually needed ?
Quoted 6 lines
Quoted 1 line
We have a similar check in
media_entity_to_v4l2_subdev()macro.That's a good point. Different behaviours for the two macros would be confusing I suppose, so
Reviewed-by Laurent Pinchart <[email protected]>Thank you.
Quoted 2 lines
I'm tempted to convert the macro to an inline function, but that would make it more difficult to switch to
container_of_const().Let's not do that. Eventually
container_of()should become whatcontainer_of_const()is at the moment.But once this is in, I can introduce
container_of_null()which we could later use. I wonder how it will look like.
Signature · 3 lines
Regards, Laurent Pinchart
Sakari Ailus
maintainerRe: [PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe
Quoted 48 lines · Laurent Pinchart
Quoted 47 lines · Sakari Ailus
Hi Laurent,
Quoted 43 lines · Laurent Pinchart
Quoted 21 lines · Sakari Ailus
Quoted 20 lines · Laurent Pinchart
Quoted 19 lines · Sakari Ailus
Quoted 18 lines · Laurent Pinchart
Quoted 17 lines · Sakari Ailus
Make
media_entity_to_video_device(NULL)return NULL, instead of an invalid pointer value.Signed-off-by Sakari Ailus <[email protected]>--- include/media/v4l2-dev.h | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)include/media/v4l2-dev.h +0
-1@@ -313,10 +313,16 @@ struct video_device {* media_entity_to_video_device - Returns a &struct video_device from * the &struct media_entity embedded on it. * -* @__entity: pointer to &struct media_entity Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
include/media/v4l2-dev.h struct video_device { · line 320
Quoted 8 lines
include/media/v4l2-dev.h +5
-3-*/ -#define media_entity_to_video_device (__entity) \ - container_of(__entity, struct video_device, entity) +* @__entity: pointer to &struct media_entity, may be NULL +*/ +#define media_entity_to_video_device (__entity) \ + ({ \ + typeof (__entity) __me_to_vdev_ent = __entity; \This should be
__me_vdev_entto align the naming withmedia_entity_to_v4l2_subdev().Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
include/media/v4l2-dev.h struct video_device { · line 323
Quoted 3 lines
include/media/v4l2-dev.h +3
-0+ \ + __me_to_vdev_ent ?container_of (__me_to_vdev_ent, \ + struct video_device, entity) : \And here you should write
__me_to_vdev_ent ? \ container_of(__me_vdev_ent, struct video_device, entity) : \ NULL; \
Quoted 5 lines
Quoted 2 lines
I'm fine wrap after '?' as well, but another wrap is needed due to indentation in any case.
I didn't make it clear, my comment was about replicating the style of
media_entity_to_v4l2_subdev().
Ok, I'll use that in v3.
Signature · 1 line
Sakari Ailus
Sakari Ailus
maintainerPATCH 6/13 media: v4l2-subdev: Mark both streams of a route enabled
In reply to Sakari AilusStreams are stored in sink-source pairs in sub-device state. When a stream was marked enabled (or disabled), only the state of one end of the stream was modified, leaving the stream in an incoherent state. Mark both ends of the stream enabled (or disabled).
--- drivers/media/v4l2-core/v4l2-subdev.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/media/v4l2-core/v4l2-subdev.c +3 -1
@@ -2254,9 +2254,11 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,for (unsigned int i =0 ; i < state->stream_configs.num_configs; ++i) {struct v4l2_subdev_stream_config *cfg = &state->stream_configs.configs[i]; +struct v4l2_subdev_stream_config *cfg2 = + &state->stream_configs.configs[i ^1U ];if (cfg->pad == pad && (streams_mask &BIT_ULL (cfg->stream))) - cfg->enabled = enabled; + cfg->enabled = cfg2->enabled = enabled; } }
Signature · 1 line
2.39.5
Re: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled
Hi Sakari,
Thank you for the patch.
Quoted 1 line · Sakari Ailus
Streams are stored in sink-source pairs in sub-device state.
Is it really this simple ? Don't we support for instance stream merging where two streams on possibly different sink pads are routed to a single stream on a source pad ?
Quoted 27 lines
When a stream
was marked enabled (or disabled), only the state of one end of the stream was modified, leaving the stream in an incoherent state. Mark both ends of the stream enabled (or disabled).Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/v4l2-core/v4l2-subdev.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)drivers/media/v4l2-core/v4l2-subdev.c +3
-1@@ -2254,9 +2254,11 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,for (unsigned int i =0 ; i < state->stream_configs.num_configs; ++i) {struct v4l2_subdev_stream_config *cfg = &state->stream_configs.configs[i]; +struct v4l2_subdev_stream_config *cfg2 = + &state->stream_configs.configs[i ^1U ];if (cfg->pad == pad && (streams_mask &BIT_ULL (cfg->stream))) - cfg->enabled = enabled; + cfg->enabled = cfg2->enabled = enabled; } }
Signature · 3 lines
Regards, Laurent Pinchart
Sakari Ailus
maintainerRe: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled
Hi Laurent,
Quoted 3 lines · Laurent Pinchart
Hi Sakari,
Thank you for the patch.
Thanks for the review.
Quoted 7 lines · Sakari Ailus
Quoted 1 line
Streams are stored in sink-source pairs in sub-device state.
Is it really this simple ? Don't we support for instance stream merging where two streams on possibly different sink pads are routed to a single stream on a source pad ?
As far as I'm concerned, yes. Each route has a sink and a source pad, you may have multiple routes from a pad or to a pad, but these are all separate routes. The UAPI as well as v4l2-subdev.c is aligned with this and I wouldn't expect it to be otherwise.
I'd like to have Tomi's view on this, too.
Quoted 5 lines
Quoted 5 lines
When a stream
was marked enabled (or disabled), only the state of one end of the stream was modified, leaving the stream in an incoherent state. Mark both ends of the stream enabled (or disabled).
This should discuss "route" in fact, not "stream".
Signature · 3 lines
Regards, Sakari Ailus
Re: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled
Quoted 4 lines · Sakari Ailus
Quoted 3 lines · Laurent Pinchart
Quoted 1 line · Sakari Ailus
Streams are stored in sink-source pairs in sub-device state.
Quoted 8 lines
Quoted 3 lines
Is it really this simple ? Don't we support for instance stream merging where two streams on possibly different sink pads are routed to a single stream on a source pad ?
As far as I'm concerned, yes. Each route has a sink and a source pad, you may have multiple routes from a pad or to a pad, but these are all separate routes. The UAPI as well as
v4l2-subdev.cis aligned with this and I wouldn't expect it to be otherwise.
Should we then refactor v4l2_subdev_stream_configs to contain a list of route configs, with each route config containing two stream configs, one for the sink pad and one for the source pad ? We would still have duplicated entries when multiple routes originate from or end at the same pad/stream, but the data structures would be more explicit and we would rely less on the fact that each route has a pair of consecutive stream configs.
Quoted 8 lines
I'd like to have Tomi's view on this, too.
Quoted 4 lines
Quoted 4 lines
When a stream
was marked enabled (or disabled), only the state of one end of the stream was modified, leaving the stream in an incoherent state. Mark both ends of the stream enabled (or disabled).This should discuss "route" in fact, not "stream".
That would make it a bit clearer.
Signature · 3 lines
Regards, Laurent Pinchart
Sakari Ailus
maintainerRe: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled
Hi Laurent,
Quoted 14 lines · Laurent Pinchart
Quoted 4 lines · Sakari Ailus
Quoted 3 lines · Laurent Pinchart
Quoted 1 line · Sakari Ailus
Streams are stored in sink-source pairs in sub-device state.
Quoted 8 lines
Quoted 3 lines
Is it really this simple ? Don't we support for instance stream merging where two streams on possibly different sink pads are routed to a single stream on a source pad ?
As far as I'm concerned, yes. Each route has a sink and a source pad, you may have multiple routes from a pad or to a pad, but these are all separate routes. The UAPI as well as
v4l2-subdev.cis aligned with this and I wouldn't expect it to be otherwise.
Quoted 7 lines
Should we then refactor
v4l2_subdev_stream_configsto contain a list of route configs, with each route config containing two stream configs, one for the sink pad and one for the source pad ? We would still have duplicated entries when multiple routes originate from or end at the same pad/stream, but the data structures would be more explicit and we would rely less on the fact that each route has a pair of consecutive stream configs.
Most of the code dealing with stream configurations works on either sink or source pads and the current data structure makes that code more simple. I think this was probably a factor why Tomi wrote it like this.
Tomi, any comment on this?
Quoted 11 lines
Quoted 9 lines
I'd like to have Tomi's view on this, too.
Quoted 4 lines
Quoted 4 lines
When a stream
was marked enabled (or disabled), only the state of one end of the stream was modified, leaving the stream in an incoherent state. Mark both ends of the stream enabled (or disabled).This should discuss "route" in fact, not "stream".
That would make it a bit clearer.
I'll fix this for v2.
Signature · 3 lines
Regards, Sakari Ailus
Re: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled
In reply to Sakari AilusQuoted 13 lines · Sakari Ailus
Hi Laurent,
Quoted 3 lines · Laurent Pinchart
Hi Sakari,
Thank you for the patch.
Thanks for the review.
Quoted 4 lines · Sakari Ailus
Quoted 1 line
Streams are stored in sink-source pairs in sub-device state.
Quoted 8 lines
Quoted 3 lines
Is it really this simple ? Don't we support for instance stream merging where two streams on possibly different sink pads are routed to a single stream on a source pad ?
As far as I'm concerned, yes. Each route has a sink and a source pad, you may have multiple routes from a pad or to a pad, but these are all separate routes. The UAPI as well as
v4l2-subdev.cis aligned with this and I wouldn't expect it to be otherwise.
But with stream merging/splitting we'll have one stream config on one pad, and two stream configs on the other. And thus two routes from that one config.
That said, if I recall right, I don't think merging/splitting has ever been really explored...
Tomi
Re: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled
Quoted 9 lines · Tomi Valkeinen
Quoted 8 lines · Sakari Ailus
Quoted 6 lines · Laurent Pinchart
Quoted 1 line · Sakari Ailus
Streams are stored in sink-source pairs in sub-device state.
Is it really this simple ? Don't we support for instance stream merging where two streams on possibly different sink pads are routed to a single stream on a source pad ?
Quoted 8 lines
Quoted 4 lines
As far as I'm concerned, yes. Each route has a sink and a source pad, you may have multiple routes from a pad or to a pad, but these are all separate routes. The UAPI as well as
v4l2-subdev.cis aligned with this and I wouldn't expect it to be otherwise.But with stream merging/splitting we'll have one stream config on one pad, and two stream configs on the other. And thus two routes from that one config.
With the current code, we will have duplicated stream config entries for the same stream in that case. Only the first one will be seen by the v4l2_subdev_state_get_format() function, but both will exist.
Quoted 2 lines
That said, if I recall right, I don't think merging/splitting has ever been really explored...
Signature · 3 lines
Regards, Laurent Pinchart
Re: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled
In reply to Sakari AilusHi,
Quoted 15 lines · Sakari Ailus
Streams are stored in sink-source pairs in sub-device state. When a stream was marked enabled (or disabled), only the state of one end of the stream was modified, leaving the stream in an incoherent state. Mark both ends of the stream enabled (or disabled).
Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/v4l2-core/v4l2-subdev.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)drivers/media/v4l2-core/v4l2-subdev.c +0
-0
drivers/media/v4l2-core/v4l2-subdev.c static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd, · line 2258
Quoted 5 lines
drivers/media/v4l2-core/v4l2-subdev.c +2
-0@@ -2254,9 +2254,11 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,for (unsigned int i =0 ; i < state->stream_configs.num_configs; ++i) {struct v4l2_subdev_stream_config *cfg = &state->stream_configs.configs[i]; +struct v4l2_subdev_stream_config *cfg2 = + &state->stream_configs.configs[i ^1U ];
I don't think this is correct. There's no specific rule that the stream configs would be in a particular order. Or that there even are a pair.
You should look for the other end of the stream from the routing table.
Tomi
Quoted 7 lines
drivers/media/v4l2-core/v4l2-subdev.c +1
-1if (cfg->pad == pad && (streams_mask &BIT_ULL (cfg->stream))) - cfg->enabled = enabled; + cfg->enabled = cfg2->enabled = enabled; } }
Sakari Ailus
maintainerRe: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled
Moi,
Quoted 27 lines · Tomi Valkeinen
Hi,
Quoted 15 lines · Sakari Ailus
Streams are stored in sink-source pairs in sub-device state. When a stream was marked enabled (or disabled), only the state of one end of the stream was modified, leaving the stream in an incoherent state. Mark both ends of the stream enabled (or disabled).
Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/v4l2-core/v4l2-subdev.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)drivers/media/v4l2-core/v4l2-subdev.c +0
-0drivers/media/v4l2-core/v4l2-subdev.c static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd, · line 2258
Quoted 5 lines
drivers/media/v4l2-core/v4l2-subdev.c +2
-0@@ -2254,9 +2254,11 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,for (unsigned int i =0 ; i < state->stream_configs.num_configs; ++i) {struct v4l2_subdev_stream_config *cfg = &state->stream_configs.configs[i]; +struct v4l2_subdev_stream_config *cfg2 = + &state->stream_configs.configs[i ^1U ];I don't think this is correct. There's no specific rule that the stream configs would be in a particular order. Or that there even are a pair.
Quoted 1 line
You should look for the other end of the stream from the routing table.
Currently the code managing the streams stores them in this array in pairs, the sink followed by the source for each route. As noted in the commit message, this patch also fixes a bug.
I can change the patch to use routes to find the other end, but over time we might want to use a different data structure to store this information (a linked list perhaps). The number of routes and stream configurations is presumably fairly small in most cases so right now I presume there's no noteworthy effect on performance.
Signature · 3 lines
Terveisin, Sakari Ailus
Re: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled
Quoted 32 lines · Sakari Ailus
Quoted 25 lines · Tomi Valkeinen
Quoted 15 lines · Sakari Ailus
Streams are stored in sink-source pairs in sub-device state. When a stream was marked enabled (or disabled), only the state of one end of the stream was modified, leaving the stream in an incoherent state. Mark both ends of the stream enabled (or disabled).
Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/v4l2-core/v4l2-subdev.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)drivers/media/v4l2-core/v4l2-subdev.c +0
-0drivers/media/v4l2-core/v4l2-subdev.c static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd, · line 2258
Quoted 5 lines
drivers/media/v4l2-core/v4l2-subdev.c +2
-0@@ -2254,9 +2254,11 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,for (unsigned int i =0 ; i < state->stream_configs.num_configs; ++i) {struct v4l2_subdev_stream_config *cfg = &state->stream_configs.configs[i]; +struct v4l2_subdev_stream_config *cfg2 = + &state->stream_configs.configs[i ^1U ];I don't think this is correct. There's no specific rule that the stream configs would be in a particular order. Or that there even are a pair.
Quoted 1 line
You should look for the other end of the stream from the routing table.
Currently the code managing the streams stores them in this array in pairs, the sink followed by the source for each route. As noted in the commit message, this patch also fixes a bug.
Quoted 5 lines
I can change the patch to use routes to find the other end, but over time we might want to use a different data structure to store this information (a linked list perhaps). The number of routes and stream configurations is presumably fairly small in most cases so right now I presume there's no noteworthy effect on performance.
At the very least I would like to document somewhere that each route stores a pair of stream config. We also really, *really*, *REALLY* need to make sure that as little code as possible depends on this, and especially that no driver accesses the stream configs directly... aargghghhhh the ds90ub913 driver does already :'-(
I've submitted [1] to make the v4l2_subdev_stream_config structure private.
[1] https://lore.kernel.org/r/[email protected]
Signature · 3 lines
Regards, Laurent Pinchart
Sakari Ailus
maintainerPATCH 7/13 media: ipu6: Set up CSI-2 receiver at correct moment
In reply to Sakari AilusEnable the CSI-2 receiver before the first stream is started and disable it when the last stream is stopped. Before this patch, every time a stream was started, the CSI-2 receiver was enabled and similarly, it was disabled when any stream was stopped.
--- drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 25 ++++++++++++------- 1 file changed, 16 insertions(+), 9 deletions(-)
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +16 -9
@@ -348,7 +348,7 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,struct ipu6_isys_csi2_timing timing = { };struct v4l2_subdev *remote_sd;struct media_pad *remote_pad; -u64 sink_streams; +u64 sink_streams, already_enabled;int ret; remote_pad =media_pad_remote_pad_first (&sd->entity.pads[CSI2_PAD_SINK ]);
@@ -358,13 +358,17 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,v4l2_subdev_state_xlate_streams (state, pad,CSI2_PAD_SINK , &streams_mask); - ret =ipu6_isys_csi2_calc_timing (csi2, &timing,CSI2_ACCINV ); -if (ret) -return ret; + already_enabled =v4l2_subdev_state_streams_enabled (sd, state, +CSI2_PAD_SINK ); +if (!already_enabled) { + ret =ipu6_isys_csi2_calc_timing (csi2, &timing,CSI2_ACCINV ); +if (ret) +return ret; - ret =ipu6_isys_csi2_set_stream (sd, &timing, csi2->nlanes, true); -if (ret) -return ret; + ret =ipu6_isys_csi2_set_stream (sd, &timing, csi2->nlanes, true); +if (ret) +return ret; + } ret =v4l2_subdev_enable_streams (remote_sd, remote_pad->index, sink_streams);
@@ -382,7 +386,7 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd, {struct v4l2_subdev *remote_sd;struct media_pad *remote_pad; -u64 sink_streams; +u64 sink_streams, still_enabled; sink_streams = v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
@@ -391,7 +395,10 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd, remote_pad =media_pad_remote_pad_first (&sd->entity.pads[CSI2_PAD_SINK ]); remote_sd =media_entity_to_v4l2_subdev (remote_pad->entity); -ipu6_isys_csi2_set_stream (sd,NULL ,0 , false); + still_enabled =v4l2_subdev_state_streams_enabled (sd, state, +CSI2_PAD_SINK ); +if (still_enabled == sink_streams) +ipu6_isys_csi2_set_stream (sd,NULL ,0 , false);v4l2_subdev_disable_streams (remote_sd, remote_pad->index, sink_streams);
Signature · 1 line
2.39.5
Re: [PATCH 07/13] media: ipu6: Set up CSI-2 receiver at correct moment
Hi Sakari,
Thank you for the patch.
Quoted 4 lines · Sakari Ailus
Enable the CSI-2 receiver before the first stream is started and disable it when the last stream is stopped. Before this patch, every time a stream was started, the CSI-2 receiver was enabled and similarly, it was disabled when any stream was stopped.
You mentioned in 03/13 that we should only enable the source when all video capture devices have been started. Shouldn't you enable the CSI-2 receiver when *all* streams are enabled, not when the first one is enabled (and similarly when stopping) ?
Quoted 19 lines
Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 25 ++++++++++++------- 1 file changed, 16 insertions(+), 9 deletions(-)drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +1
-1@@ -348,7 +348,7 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,struct ipu6_isys_csi2_timing timing = { };struct v4l2_subdev *remote_sd;struct media_pad *remote_pad; -u64 sink_streams; +u64 sink_streams, already_enabled;int ret; remote_pad =media_pad_remote_pad_first (&sd->entity.pads[CSI2_PAD_SINK ]);
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd, · line 362
Quoted 8 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +2
-3@@ -358,13 +358,17 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,v4l2_subdev_state_xlate_streams (state, pad,CSI2_PAD_SINK , &streams_mask); - ret =ipu6_isys_csi2_calc_timing (csi2, &timing,CSI2_ACCINV ); -if (ret) -return ret; + already_enabled =v4l2_subdev_state_streams_enabled (sd, state, +CSI2_PAD_SINK );
It would be cheaper to store a bitmask of enabled streams on the csi2 structure.
Quoted 36 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +13
-5+if (!already_enabled) { + ret =ipu6_isys_csi2_calc_timing (csi2, &timing,CSI2_ACCINV ); +if (ret) +return ret; - ret =ipu6_isys_csi2_set_stream (sd, &timing, csi2->nlanes, true); -if (ret) -return ret; + ret =ipu6_isys_csi2_set_stream (sd, &timing, csi2->nlanes, true); +if (ret) +return ret; + } ret =v4l2_subdev_enable_streams (remote_sd, remote_pad->index, sink_streams);@@ -382,7 +386,7 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd, {struct v4l2_subdev *remote_sd;struct media_pad *remote_pad; -u64 sink_streams; +u64 sink_streams, still_enabled; sink_streams = v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,@@ -391,7 +395,10 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd, remote_pad =media_pad_remote_pad_first (&sd->entity.pads[CSI2_PAD_SINK ]); remote_sd =media_entity_to_v4l2_subdev (remote_pad->entity); -ipu6_isys_csi2_set_stream (sd,NULL ,0 , false); + still_enabled =v4l2_subdev_state_streams_enabled (sd, state, +CSI2_PAD_SINK ); +if (still_enabled == sink_streams) +ipu6_isys_csi2_set_stream (sd,NULL ,0 , false);v4l2_subdev_disable_streams (remote_sd, remote_pad->index, sink_streams);
Signature · 3 lines
Regards, Laurent Pinchart
Sakari Ailus
maintainerRe: [PATCH 07/13] media: ipu6: Set up CSI-2 receiver at correct moment
Hi Laurent,
Quoted 3 lines · Laurent Pinchart
Hi Sakari,
Thank you for the patch.
Thanks for the review.
Quoted 11 lines · Sakari Ailus
Quoted 4 lines
Enable the CSI-2 receiver before the first stream is started and disable it when the last stream is stopped. Before this patch, every time a stream was started, the CSI-2 receiver was enabled and similarly, it was disabled when any stream was stopped.
You mentioned in 03/13 that we should only enable the source when all video capture devices have been started. Shouldn't you enable the CSI-2 receiver when *all* streams are enabled, not when the first one is enabled (and similarly when stopping) ?
Indeed. That only comes in the last patch, but I think some of these can be squashed, too.
Quoted 23 lines
Quoted 23 lines
Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 25 ++++++++++++------- 1 file changed, 16 insertions(+), 9 deletions(-)drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +1
-1@@ -348,7 +348,7 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,struct ipu6_isys_csi2_timing timing = { };struct v4l2_subdev *remote_sd;struct media_pad *remote_pad; -u64 sink_streams; +u64 sink_streams, already_enabled;int ret; remote_pad =media_pad_remote_pad_first (&sd->entity.pads[CSI2_PAD_SINK ]);@@ -358,13 +358,17 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,v4l2_subdev_state_xlate_streams (state, pad,CSI2_PAD_SINK , &streams_mask);
Quoted 8 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd, · line 362
Quoted 5 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +2
-3- ret =ipu6_isys_csi2_calc_timing (csi2, &timing,CSI2_ACCINV ); -if (ret) -return ret; + already_enabled =v4l2_subdev_state_streams_enabled (sd, state, +CSI2_PAD_SINK );It would be cheaper to store a bitmask of enabled streams on the csi2 structure.
I think I'll do just that. Indeed, this soon won't be even usable here in any case.
Quoted 38 lines
Quoted 37 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +13
-5+if (!already_enabled) { + ret =ipu6_isys_csi2_calc_timing (csi2, &timing,CSI2_ACCINV ); +if (ret) +return ret; - ret =ipu6_isys_csi2_set_stream (sd, &timing, csi2->nlanes, true); -if (ret) -return ret; + ret =ipu6_isys_csi2_set_stream (sd, &timing, csi2->nlanes, true); +if (ret) +return ret; + } ret =v4l2_subdev_enable_streams (remote_sd, remote_pad->index, sink_streams);@@ -382,7 +386,7 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd, {struct v4l2_subdev *remote_sd;struct media_pad *remote_pad; -u64 sink_streams; +u64 sink_streams, still_enabled; sink_streams = v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,@@ -391,7 +395,10 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd, remote_pad =media_pad_remote_pad_first (&sd->entity.pads[CSI2_PAD_SINK ]); remote_sd =media_entity_to_v4l2_subdev (remote_pad->entity); -ipu6_isys_csi2_set_stream (sd,NULL ,0 , false); + still_enabled =v4l2_subdev_state_streams_enabled (sd, state, +CSI2_PAD_SINK ); +if (still_enabled == sink_streams) +ipu6_isys_csi2_set_stream (sd,NULL ,0 , false);v4l2_subdev_disable_streams (remote_sd, remote_pad->index, sink_streams);
Signature · 3 lines
Regards, Sakari Ailus
Sakari Ailus
maintainerPATCH 8/13 media: v4l2-subdev: Print early in v4l2_subdev_{enable,disable}_streams()
In reply to Sakari AilusPrint debug messages early in v4l2_subdev_enable_streams() and v4l2_subdev_disable_streams(), before sanity checks take place. This can help figuring out why something goes wrong, in driver development or otherwise.
Also print the name of the sub-device where streaming is to be enabled or disabled.
--- drivers/media/v4l2-core/v4l2-subdev.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
drivers/media/v4l2-core/v4l2-subdev.c +6 -4
@@ -2273,6 +2273,9 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,bool use_s_stream;int ret; +dev_dbg (dev,"enable streams %s:%u/%#llx\n" , sd->entity.name, pad, + streams_mask); +/* A few basic sanity checks first. */ if (pad >= sd->entity.num_pads)return -EINVAL ;
@@ -2320,8 +2323,6 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad, goto done; } -dev_dbg (dev,"enable streams %u:%#llx\n" , pad, streams_mask); - already_streaming =v4l2_subdev_is_streaming (sd);if (!use_s_stream) {
@@ -2373,6 +2374,9 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,bool use_s_stream;int ret; +dev_dbg (dev,"disable streams %s:%u/%#llx\n" , sd->entity.name, pad, + streams_mask); +/* A few basic sanity checks first. */ if (pad >= sd->entity.num_pads)return -EINVAL ;
@@ -2420,8 +2424,6 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad, goto done; } -dev_dbg (dev,"disable streams %u:%#llx\n" , pad, streams_mask); -if (!use_s_stream) {/* Call the .disable_streams() operation. */ ret = v4l2_subdev_call(sd, pad, disable_streams, state, pad,
Signature · 1 line
2.39.5
Re: [PATCH 08/13] media: v4l2-subdev: Print early in v4l2_subdev_{enable,disable}_streams()
Hi Sakari,
Quoted 18 lines · Sakari Ailus
Print debug messages early in
v4l2_subdev_enable_streams()andv4l2_subdev_disable_streams(), before sanity checks take place. This can help figuring out why something goes wrong, in driver development or otherwise.Also print the name of the sub-device where streaming is to be enabled or disabled.
Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/v4l2-core/v4l2-subdev.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)drivers/media/v4l2-core/v4l2-subdev.c +0
-0
drivers/media/v4l2-core/v4l2-subdev.c int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad, · line 2277
Quoted 5 lines
drivers/media/v4l2-core/v4l2-subdev.c +2
-0@@ -2273,6 +2273,9 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,bool use_s_stream;int ret; +dev_dbg (dev,"enable streams %s:%u/%#llx\n" , sd->entity.name, pad, + streams_mask);
I suppose there can be multiple subdevs per struct device, so printing the entity name can be useful. I'd put quotes around '%s' as the entity name can contain spaces. Apart from that,
Quoted 32 lines
drivers/media/v4l2-core/v4l2-subdev.c +4
-4+/* A few basic sanity checks first. */ if (pad >= sd->entity.num_pads)return -EINVAL ;@@ -2320,8 +2323,6 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad, goto done; } -dev_dbg (dev,"enable streams %u:%#llx\n" , pad, streams_mask); - already_streaming =v4l2_subdev_is_streaming (sd);if (!use_s_stream) {@@ -2373,6 +2374,9 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,bool use_s_stream;int ret; +dev_dbg (dev,"disable streams %s:%u/%#llx\n" , sd->entity.name, pad, + streams_mask); +/* A few basic sanity checks first. */ if (pad >= sd->entity.num_pads)return -EINVAL ;@@ -2420,8 +2424,6 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad, goto done; } -dev_dbg (dev,"disable streams %u:%#llx\n" , pad, streams_mask); -if (!use_s_stream) {/* Call the .disable_streams() operation. */ ret = v4l2_subdev_call(sd, pad, disable_streams, state, pad,
Signature · 3 lines
Regards, Laurent Pinchart
Sakari Ailus
maintainerRe: [PATCH 08/13] media: v4l2-subdev: Print early in v4l2_subdev_{enable,disable}_streams()
Hi Laurent,
Quoted 24 lines · Laurent Pinchart
Hi Sakari,
Quoted 21 lines · Sakari Ailus
Print debug messages early in
v4l2_subdev_enable_streams()andv4l2_subdev_disable_streams(), before sanity checks take place. This can help figuring out why something goes wrong, in driver development or otherwise.Also print the name of the sub-device where streaming is to be enabled or disabled.
Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/v4l2-core/v4l2-subdev.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)drivers/media/v4l2-core/v4l2-subdev.c +0
-0@@ -2273,6 +2273,9 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,bool use_s_stream;int ret;
Quoted 6 lines
drivers/media/v4l2-core/v4l2-subdev.c int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad, · line 2277
Quoted 2 lines
drivers/media/v4l2-core/v4l2-subdev.c +2
-0+dev_dbg (dev,"enable streams %s:%u/%#llx\n" , sd->entity.name, pad, + streams_mask);I suppose there can be multiple subdevs per struct device, so printing the entity name can be useful. I'd put quotes around '%s' as the entity name can contain spaces. Apart from that,
I'll use double quotes as that is how sub-device (or entity) name is typically quoted in debug prints.
Quoted 2 lines
Reviewed-by Laurent Pinchart <[email protected]>
Thank you.
Signature · 3 lines
Regards, Sakari Ailus
Sakari Ailus
maintainerPATCH 9/13 media: v4l2-subdev: Collect streams on source pads only
In reply to Sakari Ailusv4l2_subdev_collect_streams() is used to find the streams present on source pads only. Only iterate through the streams on source pads, i.e. on odd array indices.
--- drivers/media/v4l2-core/v4l2-subdev.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-)
drivers/media/v4l2-core/v4l2-subdev.c +8 -7
@@ -2225,16 +2225,17 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd, *found_streams =0 ; *enabled_streams =0 ; -for (unsigned int i =0 ; i < state->stream_configs.num_configs; ++i) { -const struct v4l2_subdev_stream_config *cfg = - &state->stream_configs.configs[i]; +for (unsigned int i =0 ; i < state->stream_configs.num_configs; i +=2 ) { +const struct v4l2_subdev_stream_config *src_cfg = + &state->stream_configs.configs[i +1 ]; -if (cfg->pad != pad || !(streams_mask &BIT_ULL (cfg->stream))) +if (src_cfg->pad != pad || + !(streams_mask &BIT_ULL (src_cfg->stream)))continue ; - *found_streams |=BIT_ULL (cfg->stream); -if (cfg->enabled) - *enabled_streams |=BIT_ULL (cfg->stream); + *found_streams |=BIT_ULL (src_cfg->stream); +if (src_cfg->enabled) + *enabled_streams |=BIT_ULL (src_cfg->stream); } }
Signature · 1 line
2.39.5
Re: [PATCH 09/13] media: v4l2-subdev: Collect streams on source pads only
Hi Sakari,
Thank you for the patch.
Quoted 14 lines · Sakari Ailus
v4l2_subdev_collect_streams()is used to find the streams present on source pads only. Only iterate through the streams on source pads, i.e. on odd array indices.Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/v4l2-core/v4l2-subdev.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-)drivers/media/v4l2-core/v4l2-subdev.c +0
-0
drivers/media/v4l2-core/v4l2-subdev.c static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd, · line 2230
Quoted 9 lines
drivers/media/v4l2-core/v4l2-subdev.c +3
-3@@ -2225,16 +2225,17 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd, *found_streams =0 ; *enabled_streams =0 ; -for (unsigned int i =0 ; i < state->stream_configs.num_configs; ++i) { -const struct v4l2_subdev_stream_config *cfg = - &state->stream_configs.configs[i]; +for (unsigned int i =0 ; i < state->stream_configs.num_configs; i +=2 ) { +const struct v4l2_subdev_stream_config *src_cfg = + &state->stream_configs.configs[i +1 ];
You could start at i = 1 and avoid the + 1 here.
Unless I missed it, I don't think we document anywhere that we store two values per route in the array, and that, if multiple routes have the same sink or source stream, multiple entries will exist for the same stream. I'd like to see this clearly explained somewhere, instead of relying on an implemetation detail only known by few people.
Quoted 15 lines
drivers/media/v4l2-core/v4l2-subdev.c +5
-4-if (cfg->pad != pad || !(streams_mask &BIT_ULL (cfg->stream))) +if (src_cfg->pad != pad || + !(streams_mask &BIT_ULL (src_cfg->stream)))continue ; - *found_streams |=BIT_ULL (cfg->stream); -if (cfg->enabled) - *enabled_streams |=BIT_ULL (cfg->stream); + *found_streams |=BIT_ULL (src_cfg->stream); +if (src_cfg->enabled) + *enabled_streams |=BIT_ULL (src_cfg->stream); } }
Signature · 3 lines
Regards, Laurent Pinchart
Sakari Ailus
maintainerRe: [PATCH 09/13] media: v4l2-subdev: Collect streams on source pads only
Hi Laurent,
Quoted 3 lines · Laurent Pinchart
Hi Sakari,
Thank you for the patch.
Thank you for the review!
Quoted 28 lines · Sakari Ailus
Quoted 14 lines
v4l2_subdev_collect_streams()is used to find the streams present on source pads only. Only iterate through the streams on source pads, i.e. on odd array indices.Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/v4l2-core/v4l2-subdev.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-)drivers/media/v4l2-core/v4l2-subdev.c +0
-0drivers/media/v4l2-core/v4l2-subdev.c static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd, · line 2230
Quoted 9 lines
drivers/media/v4l2-core/v4l2-subdev.c +3
-3@@ -2225,16 +2225,17 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd, *found_streams =0 ; *enabled_streams =0 ; -for (unsigned int i =0 ; i < state->stream_configs.num_configs; ++i) { -const struct v4l2_subdev_stream_config *cfg = - &state->stream_configs.configs[i]; +for (unsigned int i =0 ; i < state->stream_configs.num_configs; i +=2 ) { +const struct v4l2_subdev_stream_config *src_cfg = + &state->stream_configs.configs[i +1 ];You could start at i = 1 and avoid the + 1 here.
Quoted 5 lines
Unless I missed it, I don't think we document anywhere that we store two values per route in the array, and that, if multiple routes have the same sink or source stream, multiple entries will exist for the same stream. I'd like to see this clearly explained somewhere, instead of relying on an implemetation detail only known by few people.
I'll add this in a separate patch.
Signature · 3 lines
Regards, Sakari Ailus
Sakari Ailus
maintainerPATCH 10/13 media: v4l2-subdev: Add debug prints to v4l2_subdev_collect_streams()
In reply to Sakari AilusPrint streams found by v4l2_subdev_collect_streams() at debug level.
--- drivers/media/v4l2-core/v4l2-subdev.c | 5 +++++ 1 file changed, 5 insertions(+)
drivers/media/v4l2-core/v4l2-subdev.c +5 -0
@@ -2219,6 +2219,8 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd, *found_streams =BIT_ULL (0 ); *enabled_streams = (sd->enabled_pads &BIT_ULL (pad)) ?BIT_ULL (0 ) :0 ; +dev_dbg (sd->dev, +"collect_streams: sub-device does not support streams\n" );return ; }
@@ -2237,6 +2239,9 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,if (src_cfg->enabled) *enabled_streams |=BIT_ULL (src_cfg->stream); } +dev_dbg (sd->dev, +"collect_streams: %s:%u found %#llx enabled %#llx\n" , + sd->entity.name, pad, *found_streams, *enabled_streams); } static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
Signature · 1 line
2.39.5
Re: [PATCH 10/13] media: v4l2-subdev: Add debug prints to v4l2_subdev_collect_streams()
Hi Sakari,
Thank you for the patch.
Quoted 1 line · Sakari Ailus
Print streams found by
v4l2_subdev_collect_streams()at debug level.
Explaining why you found this useful could help.
Quoted 10 lines
Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/v4l2-core/v4l2-subdev.c | 5 +++++ 1 file changed, 5 insertions(+)drivers/media/v4l2-core/v4l2-subdev.c +0
-0
drivers/media/v4l2-core/v4l2-subdev.c static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd, · line 2223
Quoted 5 lines
drivers/media/v4l2-core/v4l2-subdev.c +2
-0@@ -2219,6 +2219,8 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd, *found_streams =BIT_ULL (0 ); *enabled_streams = (sd->enabled_pads &BIT_ULL (pad)) ?BIT_ULL (0 ) :0 ; +dev_dbg (sd->dev, +"collect_streams: sub-device does not support streams\n" );
You print the subdev name below, I would print it here too. Maybe
"collect_streams: '%s':%u: streams not supported\n", sd->entity.name, pad);
drivers/media/v4l2-core/v4l2-subdev.c static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd, · line 2241
Quoted 7 lines
drivers/media/v4l2-core/v4l2-subdev.c +0
-0return ; }@@ -2237,6 +2239,9 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,if (src_cfg->enabled) *enabled_streams |=BIT_ULL (src_cfg->stream); }
Please add a blank line here.
drivers/media/v4l2-core/v4l2-subdev.c static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd, · line 2243
Quoted 2 lines
drivers/media/v4l2-core/v4l2-subdev.c +2
-0+ dev_dbg(sd->dev, + "collect_streams: %s:%u found %#llx enabled %#llx\n",
"collect_streams: '%s':%u: found %#llx, enabled: %#llx\n",
or possibly
"%s: '%s':%u: found %#llx, enabled: %#llx\n", __func__,
(same above).
Quoted 4 lines
drivers/media/v4l2-core/v4l2-subdev.c +1
-0+ sd->entity.name, pad, *found_streams, *enabled_streams); } static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
Signature · 3 lines
Regards, Laurent Pinchart
Sakari Ailus
maintainerRe: [PATCH 10/13] media: v4l2-subdev: Add debug prints to v4l2_subdev_collect_streams()
Hi Laurent,
Quoted 27 lines · Laurent Pinchart
Hi Sakari,
Thank you for the patch.
Quoted 1 line · Sakari Ailus
Print streams found by
v4l2_subdev_collect_streams()at debug level.Explaining why you found this useful could help.
Quoted 10 lines
Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/v4l2-core/v4l2-subdev.c | 5 +++++ 1 file changed, 5 insertions(+)drivers/media/v4l2-core/v4l2-subdev.c +0
-0drivers/media/v4l2-core/v4l2-subdev.c static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd, · line 2223
Quoted 5 lines
drivers/media/v4l2-core/v4l2-subdev.c +2
-0@@ -2219,6 +2219,8 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd, *found_streams =BIT_ULL (0 ); *enabled_streams = (sd->enabled_pads &BIT_ULL (pad)) ?BIT_ULL (0 ) :0 ; +dev_dbg (sd->dev, +"collect_streams: sub-device does not support streams\n" );You print the subdev name below, I would print it here too. Maybe
Quoted 2 lines
"collect_streams: '%s':%u: streams not supported\n", sd->entity.name, pad);
Makes sense.
Quoted 16 lines
drivers/media/v4l2-core/v4l2-subdev.c static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd, · line 2241
Quoted 8 lines
drivers/media/v4l2-core/v4l2-subdev.c +0
-0return ; }@@ -2237,6 +2239,9 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,if (src_cfg->enabled) *enabled_streams |=BIT_ULL (src_cfg->stream); }Please add a blank line here.
drivers/media/v4l2-core/v4l2-subdev.c static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd, · line 2243
Quoted 2 lines
drivers/media/v4l2-core/v4l2-subdev.c +2
-0+ dev_dbg(sd->dev, + "collect_streams: %s:%u found %#llx enabled %#llx\n","collect_streams: '%s':%u: found %#llx, enabled: %#llx\n",
Quoted 5 lines
or possibly
"%s: '%s':%u: found %#llx, enabled: %#llx\n", __func__,(same above).
I prefer collect_streams: only as it's shorter, there's no possibility for confusion in any case.
Quoted 2 lines
Reviewed-by Laurent Pinchart <[email protected]>
Thank you!
Quoted 6 lines
Quoted 5 lines
drivers/media/v4l2-core/v4l2-subdev.c +1
-0+ sd->entity.name, pad, *found_streams, *enabled_streams); } static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
Signature · 3 lines
Regards, Sakari Ailus
Sakari Ailus
maintainerPATCH 11/13 media: v4l2-subdev: Introduce v4l2_subdev_find_route()
In reply to Sakari Ailusv4l2_subdev_find_route() is like v4l2_subdev_routing_find_opposite_end(), with the difference that it's more flexible: it can look up only active routes and can find multiple routes, too.
v4l2_subdev_find_route() is intended to replacev4l2_subdev_routing_find_opposite_end().
--- drivers/media/v4l2-core/v4l2-subdev.c | 56 ++++++++++++++++++--------- include/media/v4l2-subdev.h | 19 +++++++++ 2 files changed, 56 insertions(+), 19 deletions(-)
drivers/media/v4l2-core/v4l2-subdev.c +37 -19
@@ -1996,34 +1996,52 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, }EXPORT_SYMBOL_GPL (v4l2_subdev_set_routing_with_fmt); -int v4l2_subdev_routing_find_opposite_end (const struct v4l2_subdev_krouting *routing, -u32 pad,u32 stream,u32 *other_pad, -u32 *other_stream) +struct v4l2_subdev_route * +v4l2_subdev_find_route (const struct v4l2_subdev_krouting *routing, +u32 pad,u32 stream,bool active,unsigned int index) {unsigned int i;for (i =0 ; i < routing->num_routes; ++i) {struct v4l2_subdev_route *route = &routing->routes[i]; -if (route->source_pad == pad && - route->source_stream == stream) { -if (other_pad) - *other_pad = route->sink_pad; -if (other_stream) - *other_stream = route->sink_stream; -return 0 ; - } +if (active && !(route->flags &V4L2_SUBDEV_ROUTE_FL_ACTIVE )) +continue ; -if (route->sink_pad == pad && route->sink_stream == stream) { -if (other_pad) - *other_pad = route->source_pad; -if (other_stream) - *other_stream = route->source_stream; -return 0 ; - } +if ((route->source_pad != pad || + route->source_stream != stream) && + (route->sink_pad != pad || route->sink_stream != stream)) +continue ; + +if (index--) +continue ; + +return route; } -return -EINVAL ; +return ERR_PTR (-ENOENT ); +} +EXPORT_SYMBOL_GPL (v4l2_subdev_find_route); + +int v4l2_subdev_routing_find_opposite_end (const struct v4l2_subdev_krouting *routing, +u32 pad,u32 stream,u32 *other_pad, +u32 *other_stream) +{ +struct v4l2_subdev_route *route; + + route =v4l2_subdev_find_route (routing, pad, stream, false,0 ); +if (IS_ERR (route)) +return PTR_ERR (route); + +bool is_source = route->source_pad == pad; + +if (other_pad) + *other_pad = is_source ? route->sink_pad : route->source_pad; +if (other_stream) + *other_stream = is_source ? + route->sink_stream : route->source_stream; + +return 0 ; }EXPORT_SYMBOL_GPL (v4l2_subdev_routing_find_opposite_end);
include/media/v4l2-subdev.h +19 -0
@@ -1547,6 +1547,23 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,const struct v4l2_subdev_krouting *routing, const struct v4l2_mbus_framefmt *fmt); +/** +* v4l2_subdev_find_route() - Find routes from a (pad, stream) pair +* @routing: routing used to find the opposite side +* @pad: pad id +* @stream: stream id +* @active: set to true for looking up only active routes +* @index: for accessing more than one route from the pad +* +* Find a route from the routing table where one end has (pad, stream) pair +* matching @pad and @stream. +* +* Returns the route on success or -ENOENT if no matching route is found. +*/ +struct v4l2_subdev_route * +v4l2_subdev_find_route (const struct v4l2_subdev_krouting *routing, +u32 pad,u32 stream,bool active,unsigned int index); +/** * v4l2_subdev_routing_find_opposite_end() - Find the opposite stream * @routing: routing used to find the opposite side
@@ -1555,6 +1572,8 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,* @other_pad: pointer used to return the opposite pad * @other_stream: pointer used to return the opposite stream * +* Prefer v4l2_subdev_find_route() over v4l2_subdev_routing_find_opposite_end(). +* * This function uses the routing table to find the pad + stream which is * opposite the given pad + stream. *
Signature · 1 line
2.39.5
Re: [PATCH 11/13] media: v4l2-subdev: Introduce v4l2_subdev_find_route()
Hi Sakari
Quoted 6 lines · Sakari Ailus
v4l2_subdev_find_route()is likev4l2_subdev_routing_find_opposite_end(), with the difference that it's more flexible: it can look up only active routes and can find multiple routes, too.
v4l2_subdev_find_route()is intended to replacev4l2_subdev_routing_find_opposite_end().
To me this feels like v4l2_subdev_find_route() could be used to implement more helpers like v4l2_subdev_routing_find_opposite_end() for drivers instead of going the other way around.
let's see what the use cases are
Quoted 27 lines
Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/v4l2-core/v4l2-subdev.c | 56 ++++++++++++++++++--------- include/media/v4l2-subdev.h | 19 +++++++++ 2 files changed, 56 insertions(+), 19 deletions(-)drivers/media/v4l2-core/v4l2-subdev.c +3
-3@@ -1996,34 +1996,52 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, }EXPORT_SYMBOL_GPL (v4l2_subdev_set_routing_with_fmt); -int v4l2_subdev_routing_find_opposite_end (const struct v4l2_subdev_krouting *routing, -u32 pad,u32 stream,u32 *other_pad, -u32 *other_stream) +struct v4l2_subdev_route * +v4l2_subdev_find_route (const struct v4l2_subdev_krouting *routing, +u32 pad,u32 stream,bool active,unsigned int index) {unsigned int i;for (i =0 ; i < routing->num_routes; ++i) {struct v4l2_subdev_route *route = &routing->routes[i];
drivers/media/v4l2-core/v4l2-subdev.c int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 2009
Quoted 10 lines
drivers/media/v4l2-core/v4l2-subdev.c +2
-8-if (route->source_pad == pad && - route->source_stream == stream) { -if (other_pad) - *other_pad = route->sink_pad; -if (other_stream) - *other_stream = route->sink_stream; -return 0 ; - } +if (active && !(route->flags &V4L2_SUBDEV_ROUTE_FL_ACTIVE )) +continue ;
I know currently v4l2_subdev_routing_find_opposite_end() does return any route that matches the provided 'pad' and 'stream' included non-active ones, but I wonder if this is desirable. What is the use case for enumerating a non-active route between two pads ?
(it is also my impression that all drivers that use v4l2_subdev_routing_find_opposite_end() assume the route is active)
Also I wonder if the usage of V4L2_SUBDEV_ROUTE_FL_ACTIVE is clearly defined, or, in other words, what is the use case for userspace to create non-active routes, given that any new VIDIOC_SUBDEV_S_ROUTING will anyway re-create the routing table (that's a different question, on the ioctl definition and not on this change though)
Quoted 51 lines
drivers/media/v4l2-core/v4l2-subdev.c +32
-8-if (route->sink_pad == pad && route->sink_stream == stream) { -if (other_pad) - *other_pad = route->source_pad; -if (other_stream) - *other_stream = route->source_stream; -return 0 ; - } +if ((route->source_pad != pad || + route->source_stream != stream) && + (route->sink_pad != pad || route->sink_stream != stream)) +continue ; + +if (index--) +continue ; + +return route; } -return -EINVAL ; +return ERR_PTR (-ENOENT ); +} +EXPORT_SYMBOL_GPL (v4l2_subdev_find_route); + +int v4l2_subdev_routing_find_opposite_end (const struct v4l2_subdev_krouting *routing, +u32 pad,u32 stream,u32 *other_pad, +u32 *other_stream) +{ +struct v4l2_subdev_route *route; + + route =v4l2_subdev_find_route (routing, pad, stream, false,0 ); +if (IS_ERR (route)) +return PTR_ERR (route); + +bool is_source = route->source_pad == pad; + +if (other_pad) + *other_pad = is_source ? route->sink_pad : route->source_pad; +if (other_stream) + *other_stream = is_source ? + route->sink_stream : route->source_stream; + +return 0 ; }EXPORT_SYMBOL_GPL (v4l2_subdev_routing_find_opposite_end);include/media/v4l2-subdev.h +0
-0
include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1551
Quoted 5 lines
include/media/v4l2-subdev.h +2
-0@@ -1547,6 +1547,23 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,const struct v4l2_subdev_krouting *routing, const struct v4l2_mbus_framefmt *fmt); +/** +* v4l2_subdev_find_route() - Find routes from a (pad, stream) pair
from or for ?
include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1552
Quoted 1 line
include/media/v4l2-subdev.h +1
-0+* @routing: routing used to find the opposite side
I would not say "opposite side" but rather
@routing: routing table used to enumerate routes
include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1556
Quoted 4 lines
include/media/v4l2-subdev.h +4
-0+* @pad: pad id +* @stream: stream id +* @active: set to true for looking up only active routes +* @index: for accessing more than one route from the pad
I understand this but maybe
@index: route index for enumerating multiple routes
?
include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1559
Quoted 3 lines
include/media/v4l2-subdev.h +3
-0+* +* Find a route from the routing table where one end has (pad, stream) pair +* matching @pad and @stream.
* If multiple routes in @routing match @pad and @stream, return
* the @index one.
*
* Set @active to true to only enumerate active routes.
include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1561
Quoted 2 lines
include/media/v4l2-subdev.h +2
-0+* +* Returns the route on success or -ENOENT if no matching route is found.
I see other functions documentation using
* Return:
is this a kernel-doc thing ?
Quoted 9 lines
include/media/v4l2-subdev.h +5
-0+*/ +struct v4l2_subdev_route * +v4l2_subdev_find_route (const struct v4l2_subdev_krouting *routing, +u32 pad,u32 stream,bool active,unsigned int index); +/** * v4l2_subdev_routing_find_opposite_end() - Find the opposite stream * @routing: routing used to find the opposite side
include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1576
Quoted 5 lines
include/media/v4l2-subdev.h +2
-0@@ -1555,6 +1572,8 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,* @other_pad: pointer used to return the opposite pad * @other_stream: pointer used to return the opposite stream * +* Prefer v4l2_subdev_find_route() over v4l2_subdev_routing_find_opposite_end(). +*
As said, I'm not sure if that's preferred or we should rather create more helpers using v4l2_subdev_find_route() internally. Time will tell I guess ?
Thanks
j
Quoted 7 lines
include/media/v4l2-subdev.h +0
-1* This function uses the routing table to find the pad + stream which is * opposite the given pad + stream. * -- 2.39.5
Sakari Ailus
maintainerRe: [PATCH 11/13] media: v4l2-subdev: Introduce v4l2_subdev_find_route()
Hi Jacopo,
Thanks for the review.
Quoted 43 lines · Jacopo Mondi
Hi Sakari
Quoted 6 lines · Sakari Ailus
v4l2_subdev_find_route()is likev4l2_subdev_routing_find_opposite_end(), with the difference that it's more flexible: it can look up only active routes and can find multiple routes, too.
v4l2_subdev_find_route()is intended to replacev4l2_subdev_routing_find_opposite_end().To me this feels like
v4l2_subdev_find_route()could be used to implement more helpers likev4l2_subdev_routing_find_opposite_end()for drivers instead of going the other way around.let's see what the use cases are
Quoted 27 lines
Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/v4l2-core/v4l2-subdev.c | 56 ++++++++++++++++++--------- include/media/v4l2-subdev.h | 19 +++++++++ 2 files changed, 56 insertions(+), 19 deletions(-)drivers/media/v4l2-core/v4l2-subdev.c +3
-3@@ -1996,34 +1996,52 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, }EXPORT_SYMBOL_GPL (v4l2_subdev_set_routing_with_fmt); -int v4l2_subdev_routing_find_opposite_end (const struct v4l2_subdev_krouting *routing, -u32 pad,u32 stream,u32 *other_pad, -u32 *other_stream) +struct v4l2_subdev_route * +v4l2_subdev_find_route (const struct v4l2_subdev_krouting *routing, +u32 pad,u32 stream,bool active,unsigned int index) {unsigned int i;for (i =0 ; i < routing->num_routes; ++i) {struct v4l2_subdev_route *route = &routing->routes[i];
Quoted 15 lines
drivers/media/v4l2-core/v4l2-subdev.c int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 2009
Quoted 10 lines
drivers/media/v4l2-core/v4l2-subdev.c +2
-8-if (route->source_pad == pad && - route->source_stream == stream) { -if (other_pad) - *other_pad = route->sink_pad; -if (other_stream) - *other_stream = route->sink_stream; -return 0 ; - } +if (active && !(route->flags &V4L2_SUBDEV_ROUTE_FL_ACTIVE )) +continue ;I know currently
v4l2_subdev_routing_find_opposite_end()does return any route that matches the provided 'pad' and 'stream' included non-active ones, but I wonder if this is desirable. What is the use case for enumerating a non-active route between two pads ?
Good question. v4l2_subdev_routing_find_opposite_end() nevertheless returns them. And the caller won't get the route for checking the state either.
Quoted 9 lines
(it is also my impression that all drivers that use
v4l2_subdev_routing_find_opposite_end()assume the route is active)Also I wonder if the usage of
V4L2_SUBDEV_ROUTE_FL_ACTIVEis clearly defined, or, in other words, what is the use case for userspace to create non-active routes, given that any newVIDIOC_SUBDEV_S_ROUTINGwill anyway re-create the routing table (that's a different question, on the ioctl definition and not on this change though)
I think it is. Please review the UAPI documentation in the metadata series. :-)
Quoted 55 lines
Quoted 55 lines
drivers/media/v4l2-core/v4l2-subdev.c +32
-8-if (route->sink_pad == pad && route->sink_stream == stream) { -if (other_pad) - *other_pad = route->source_pad; -if (other_stream) - *other_stream = route->source_stream; -return 0 ; - } +if ((route->source_pad != pad || + route->source_stream != stream) && + (route->sink_pad != pad || route->sink_stream != stream)) +continue ; + +if (index--) +continue ; + +return route; } -return -EINVAL ; +return ERR_PTR (-ENOENT ); +} +EXPORT_SYMBOL_GPL (v4l2_subdev_find_route); + +int v4l2_subdev_routing_find_opposite_end (const struct v4l2_subdev_krouting *routing, +u32 pad,u32 stream,u32 *other_pad, +u32 *other_stream) +{ +struct v4l2_subdev_route *route; + + route =v4l2_subdev_find_route (routing, pad, stream, false,0 ); +if (IS_ERR (route)) +return PTR_ERR (route); + +bool is_source = route->source_pad == pad; + +if (other_pad) + *other_pad = is_source ? route->sink_pad : route->source_pad; +if (other_stream) + *other_stream = is_source ? + route->sink_stream : route->source_stream; + +return 0 ; }EXPORT_SYMBOL_GPL (v4l2_subdev_routing_find_opposite_end);include/media/v4l2-subdev.h +0
-0@@ -1547,6 +1547,23 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,const struct v4l2_subdev_krouting *routing, const struct v4l2_mbus_framefmt *fmt);
Quoted 4 lines
include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1551
Quoted 2 lines
include/media/v4l2-subdev.h +2
-0+/** +* v4l2_subdev_find_route() - Find routes from a (pad, stream) pair from or for ?
From or to. I'll fix this in the next version.
Quoted 6 lines
include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1552
Quoted 2 lines
include/media/v4l2-subdev.h +1
-0+* @routing: routing used to find the opposite side I would not say "opposite side" but rather
@routing: routing table used to enumerate routes
How about simply "the routing table"?
Quoted 10 lines
include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1556
Quoted 5 lines
include/media/v4l2-subdev.h +4
-0+* @pad: pad id +* @stream: stream id +* @active: set to true for looking up only active routes +* @index: for accessing more than one route from the pad I understand this but maybe
@index: route index for enumerating multiple routes
?
Sounds good.
Quoted 10 lines
include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1559
Quoted 4 lines
include/media/v4l2-subdev.h +3
-0+* +* Find a route from the routing table where one end has (pad, stream) pair +* matching @pad and @stream. * If multiple routes in @routing match @pad and @stream, return
* the @index one.
*
* Set @active to true to only enumerate active routes.
Quoted 8 lines
include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1561
Quoted 2 lines
include/media/v4l2-subdev.h +2
-0+* +* Returns the route on success or -ENOENT if no matching route is found. I see other functions documentation using
* Return:
is this a kernel-doc thing ?
Yes, makes sense.
Quoted 11 lines
Quoted 11 lines
include/media/v4l2-subdev.h +5
-0+*/ +struct v4l2_subdev_route * +v4l2_subdev_find_route (const struct v4l2_subdev_krouting *routing, +u32 pad,u32 stream,bool active,unsigned int index); +/** * v4l2_subdev_routing_find_opposite_end() - Find the opposite stream * @routing: routing used to find the opposite side @@ -1555,6 +1572,8 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,* @other_pad: pointer used to return the opposite pad
Quoted 8 lines
include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1576
Quoted 4 lines
include/media/v4l2-subdev.h +2
-0* @other_stream: pointer used to return the opposite stream * +* Prefer v4l2_subdev_find_route() over v4l2_subdev_routing_find_opposite_end(). +* As said, I'm not sure if that's preferred or we should rather create more helpers using
v4l2_subdev_find_route()internally. Time will tell I guess ?
I agree.
The benefit of the older function was that it returns information that doesn't need a lock for accessing it.
Signature · 3 lines
Regards, Sakari Ailus
Re: [PATCH 11/13] media: v4l2-subdev: Introduce v4l2_subdev_find_route()
Quoted 57 lines · Sakari Ailus
Quoted 40 lines · Jacopo Mondi
Quoted 6 lines · Sakari Ailus
v4l2_subdev_find_route()is likev4l2_subdev_routing_find_opposite_end(), with the difference that it's more flexible: it can look up only active routes and can find multiple routes, too.
v4l2_subdev_find_route()is intended to replacev4l2_subdev_routing_find_opposite_end().To me this feels like
v4l2_subdev_find_route()could be used to implement more helpers likev4l2_subdev_routing_find_opposite_end()for drivers instead of going the other way around.let's see what the use cases are
Quoted 26 lines
Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/v4l2-core/v4l2-subdev.c | 56 ++++++++++++++++++--------- include/media/v4l2-subdev.h | 19 +++++++++ 2 files changed, 56 insertions(+), 19 deletions(-)drivers/media/v4l2-core/v4l2-subdev.c +3
-3@@ -1996,34 +1996,52 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, }EXPORT_SYMBOL_GPL (v4l2_subdev_set_routing_with_fmt); -int v4l2_subdev_routing_find_opposite_end (const struct v4l2_subdev_krouting *routing, -u32 pad,u32 stream,u32 *other_pad, -u32 *other_stream) +struct v4l2_subdev_route * +v4l2_subdev_find_route (const struct v4l2_subdev_krouting *routing, +u32 pad,u32 stream,bool active,unsigned int index) {unsigned int i;for (i =0 ; i < routing->num_routes; ++i) {struct v4l2_subdev_route *route = &routing->routes[i];Quoted 15 lines
drivers/media/v4l2-core/v4l2-subdev.c int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 2009
Quoted 10 lines
drivers/media/v4l2-core/v4l2-subdev.c +2
-8-if (route->source_pad == pad && - route->source_stream == stream) { -if (other_pad) - *other_pad = route->sink_pad; -if (other_stream) - *other_stream = route->sink_stream; -return 0 ; - } +if (active && !(route->flags &V4L2_SUBDEV_ROUTE_FL_ACTIVE )) +continue ;I know currently
v4l2_subdev_routing_find_opposite_end()does return any route that matches the provided 'pad' and 'stream' included non-active ones, but I wonder if this is desirable. What is the use case for enumerating a non-active route between two pads ?
Quoted 2 lines
Good question.
v4l2_subdev_routing_find_opposite_end()nevertheless returns them. And the caller won't get the route for checking the state either.
I'm tempted to check the callers of the function and change the behaviour to only return active routes.
Quoted 11 lines
Quoted 8 lines
(it is also my impression that all drivers that use
v4l2_subdev_routing_find_opposite_end()assume the route is active)Also I wonder if the usage of
V4L2_SUBDEV_ROUTE_FL_ACTIVEis clearly defined, or, in other words, what is the use case for userspace to create non-active routes, given that any newVIDIOC_SUBDEV_S_ROUTINGwill anyway re-create the routing table (that's a different question, on the ioctl definition and not on this change though)I think it is. Please review the UAPI documentation in the metadata series. :-)
Section "Device types and routing setup" in Documentation/userspace-api/media/v4l/dev-subdev.rst is the best documentation we have at the moment.
We essentially have two models for internal routing. In the first model, which I'll nickname the crossbar switch model, a large number of routes are possible (up to any input to any output types of scenarios). In this case, routes are created by userspace, and all routes in the routing table are expected to be active. Any inactive route provided by userspace would be dropped by the driver and not be included in the routing table.
The second model covers devices such as camera sensors, where a small fixed set of routes are hardcoded. The routing table is fixed, an some routes (the ones not marked with the IMMUTABLE flag) can be enabled/disabled by userspace using the V4L2_SUBDEV_ROUTE_FL_ACTIVE. This allows userspace to enumerate the available routes.
Documentation/userspace-api/media/v4l/dev-subdev.rst should document more clearly that we do not allow any hybrid behaviour at the moment.
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
drivers/media/v4l2-core/v4l2-subdev.c +22
-8-if (route->sink_pad == pad && route->sink_stream == stream) { -if (other_pad) - *other_pad = route->source_pad; -if (other_stream) - *other_stream = route->source_stream; -return 0 ; - } +if ((route->source_pad != pad || + route->source_stream != stream) && + (route->sink_pad != pad || route->sink_stream != stream)) +continue ; + +if (index--) +continue ; + +return route; } -return -EINVAL ; +return ERR_PTR (-ENOENT ); +} +EXPORT_SYMBOL_GPL (v4l2_subdev_find_route); + +int v4l2_subdev_routing_find_opposite_end (const struct v4l2_subdev_krouting *routing, +u32 pad,u32 stream,u32 *other_pad, +u32 *other_stream) +{ +struct v4l2_subdev_route *route; + + route =v4l2_subdev_find_route (routing, pad, stream, false,0 ); +if (IS_ERR (route)) +return PTR_ERR (route);
Quoted 8 lines
Quoted 8 lines
drivers/media/v4l2-core/v4l2-subdev.c int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 2042
Quoted 8 lines
drivers/media/v4l2-core/v4l2-subdev.c +8
-0+ +bool is_source = route->source_pad == pad; + +if (other_pad) + *other_pad = is_source ? route->sink_pad : route->source_pad; +if (other_stream) + *other_stream = is_source ? + route->sink_stream : route->source_stream;
Having to do this is_source dance makes v4l2_subdev_find_route() annoying to use. It may be fine when using the function to implement other helpers, but I wouldn't like to see this being done in drivers. Maybe we can avoid exporting v4l2_subdev_find_route() for now to ensure that, the only user in this series is in v4l2-mc.c.
Quoted 35 lines
Quoted 13 lines
Quoted 13 lines
drivers/media/v4l2-core/v4l2-subdev.c +2
-0+ +return 0 ; }EXPORT_SYMBOL_GPL (v4l2_subdev_routing_find_opposite_end);include/media/v4l2-subdev.h +0
-0@@ -1547,6 +1547,23 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,const struct v4l2_subdev_krouting *routing, const struct v4l2_mbus_framefmt *fmt);Quoted 4 lines
include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1551
Quoted 2 lines
include/media/v4l2-subdev.h +2
-0+/** +* v4l2_subdev_find_route() - Find routes from a (pad, stream) pair from or for ?
From or to. I'll fix this in the next version.
Quoted 6 lines
include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1552
Quoted 2 lines
include/media/v4l2-subdev.h +1
-0+* @routing: routing used to find the opposite side I would not say "opposite side" but rather
@routing: routing table used to enumerate routes
How about simply "the routing table"?
Quoted 6 lines
Quoted 5 lines
include/media/v4l2-subdev.h +4
-0+* @pad: pad id +* @stream: stream id +* @active: set to true for looking up only active routes +* @index: for accessing more than one route from the pad
Quoted 6 lines
Quoted 4 lines
I understand this but maybe
@index: route index for enumerating multiple routes
?Sounds good.
I'm curious to know how this parameter will be used. In the only user (in patch 12/13), it is hardcoded to 0. I'm not sure indexing routes will be very useful for drivers.
Quoted 39 lines
Quoted 9 lines
include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1559
Quoted 3 lines
include/media/v4l2-subdev.h +3
-0+* +* Find a route from the routing table where one end has (pad, stream) pair +* matching @pad and @stream. * If multiple routes in @routing match @pad and @stream, return
* the @index one.
*
* Set @active to true to only enumerate active routes.Quoted 8 lines
include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1561
Quoted 2 lines
include/media/v4l2-subdev.h +2
-0+* +* Returns the route on success or -ENOENT if no matching route is found. I see other functions documentation using
* Return:
is this a kernel-doc thing ?
Yes, makes sense.
Quoted 10 lines
Quoted 10 lines
include/media/v4l2-subdev.h +5
-0+*/ +struct v4l2_subdev_route * +v4l2_subdev_find_route (const struct v4l2_subdev_krouting *routing, +u32 pad,u32 stream,bool active,unsigned int index); +/** * v4l2_subdev_routing_find_opposite_end() - Find the opposite stream * @routing: routing used to find the opposite side @@ -1555,6 +1572,8 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,* @other_pad: pointer used to return the opposite pad Quoted 8 lines
include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1576
Quoted 4 lines
include/media/v4l2-subdev.h +2
-0* @other_stream: pointer used to return the opposite stream * +* Prefer v4l2_subdev_find_route() over v4l2_subdev_routing_find_opposite_end(). +* As said, I'm not sure if that's preferred or we should rather create more helpers using
v4l2_subdev_find_route()internally. Time will tell I guess ?
Quoted 4 lines
I agree.
The benefit of the older function was that it returns information that doesn't need a lock for accessing it.
I'm not sure to follow you here. v4l2_subdev_find_route() has the same locking requirements as v4l2_subdev_routing_find_opposite_end().
Given that v4l2_subdev_find_route() has a single user, and that the only difference in that user compared to v4l2_subdev_routing_find_opposite_end() is that only active routes are considered, I would prefer modifying v4l2_subdev_routing_find_opposite_end() to ignore inactive routes after checking that no caller would break.
Signature · 3 lines
Regards, Laurent Pinchart
Sakari Ailus
maintainerRe: [PATCH 11/13] media: v4l2-subdev: Introduce v4l2_subdev_find_route()
Hi Laurent,
Quoted 58 lines · Laurent Pinchart
Quoted 57 lines · Sakari Ailus
Quoted 55 lines · Jacopo Mondi
Quoted 6 lines · Sakari Ailus
v4l2_subdev_find_route()is likev4l2_subdev_routing_find_opposite_end(), with the difference that it's more flexible: it can look up only active routes and can find multiple routes, too.
v4l2_subdev_find_route()is intended to replacev4l2_subdev_routing_find_opposite_end().To me this feels like
v4l2_subdev_find_route()could be used to implement more helpers likev4l2_subdev_routing_find_opposite_end()for drivers instead of going the other way around.let's see what the use cases are
Quoted 26 lines
Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/v4l2-core/v4l2-subdev.c | 56 ++++++++++++++++++--------- include/media/v4l2-subdev.h | 19 +++++++++ 2 files changed, 56 insertions(+), 19 deletions(-)drivers/media/v4l2-core/v4l2-subdev.c +3
-3@@ -1996,34 +1996,52 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, }EXPORT_SYMBOL_GPL (v4l2_subdev_set_routing_with_fmt); -int v4l2_subdev_routing_find_opposite_end (const struct v4l2_subdev_krouting *routing, -u32 pad,u32 stream,u32 *other_pad, -u32 *other_stream) +struct v4l2_subdev_route * +v4l2_subdev_find_route (const struct v4l2_subdev_krouting *routing, +u32 pad,u32 stream,bool active,unsigned int index) {unsigned int i;for (i =0 ; i < routing->num_routes; ++i) {struct v4l2_subdev_route *route = &routing->routes[i];drivers/media/v4l2-core/v4l2-subdev.c int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 2009
Quoted 10 lines
drivers/media/v4l2-core/v4l2-subdev.c +2
-8-if (route->source_pad == pad && - route->source_stream == stream) { -if (other_pad) - *other_pad = route->sink_pad; -if (other_stream) - *other_stream = route->sink_stream; -return 0 ; - } +if (active && !(route->flags &V4L2_SUBDEV_ROUTE_FL_ACTIVE )) +continue ;I know currently
v4l2_subdev_routing_find_opposite_end()does return any route that matches the provided 'pad' and 'stream' included non-active ones, but I wonder if this is desirable. What is the use case for enumerating a non-active route between two pads ?
Quoted 5 lines
Quoted 2 lines
Good question.
v4l2_subdev_routing_find_opposite_end()nevertheless returns them. And the caller won't get the route for checking the state either.I'm tempted to check the callers of the function and change the behaviour to only return active routes.
I can prepend a patch to do the change there so it won't happen here.
Quoted 25 lines
Quoted 4 lines
Quoted 4 lines
(it is also my impression that all drivers that use
v4l2_subdev_routing_find_opposite_end()assume the route is active)Quoted 8 lines
Quoted 5 lines
Also I wonder if the usage of
V4L2_SUBDEV_ROUTE_FL_ACTIVEis clearly defined, or, in other words, what is the use case for userspace to create non-active routes, given that any newVIDIOC_SUBDEV_S_ROUTINGwill anyway re-create the routing table (that's a different question, on the ioctl definition and not on this change though)I think it is. Please review the UAPI documentation in the metadata series. :-)
Section "Device types and routing setup" in
Documentation/userspace-api/media/v4l/dev-subdev.rstis the best documentation we have at the moment.We essentially have two models for internal routing. In the first model, which I'll nickname the crossbar switch model, a large number of routes are possible (up to any input to any output types of scenarios). In this case, routes are created by userspace, and all routes in the routing table are expected to be active. Any inactive route provided by userspace would be dropped by the driver and not be included in the routing table.
Quoted 8 lines
The second model covers devices such as camera sensors, where a small fixed set of routes are hardcoded. The routing table is fixed, an some routes (the ones not marked with the IMMUTABLE flag) can be enabled/disabled by userspace using the
V4L2_SUBDEV_ROUTE_FL_ACTIVE. This allows userspace to enumerate the available routes.
Documentation/userspace-api/media/v4l/dev-subdev.rstshould document more clearly that we do not allow any hybrid behaviour at the moment.
I agree.
Quoted 40 lines
Quoted 40 lines
Quoted 40 lines
Quoted 40 lines
drivers/media/v4l2-core/v4l2-subdev.c +28
-8-if (route->sink_pad == pad && route->sink_stream == stream) { -if (other_pad) - *other_pad = route->source_pad; -if (other_stream) - *other_stream = route->source_stream; -return 0 ; - } +if ((route->source_pad != pad || + route->source_stream != stream) && + (route->sink_pad != pad || route->sink_stream != stream)) +continue ; + +if (index--) +continue ; + +return route; } -return -EINVAL ; +return ERR_PTR (-ENOENT ); +} +EXPORT_SYMBOL_GPL (v4l2_subdev_find_route); + +int v4l2_subdev_routing_find_opposite_end (const struct v4l2_subdev_krouting *routing, +u32 pad,u32 stream,u32 *other_pad, +u32 *other_stream) +{ +struct v4l2_subdev_route *route; + + route =v4l2_subdev_find_route (routing, pad, stream, false,0 ); +if (IS_ERR (route)) +return PTR_ERR (route); + +bool is_source = route->source_pad == pad; + +if (other_pad) + *other_pad = is_source ? route->sink_pad : route->source_pad; +if (other_stream)
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
drivers/media/v4l2-core/v4l2-subdev.c int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 2042
Quoted 2 lines
drivers/media/v4l2-core/v4l2-subdev.c +2
-0+ *other_stream = is_source ? + route->sink_stream : route->source_stream;Having to do this
is_sourcedance makesv4l2_subdev_find_route()annoying to use. It may be fine when using the function to implement other helpers, but I wouldn't like to see this being done in drivers. Maybe we can avoid exportingv4l2_subdev_find_route()for now to ensure that, the only user in this series is inv4l2-mc.c.
Sounds reasonable.
Quoted 38 lines
Quoted 38 lines
Quoted 14 lines
Quoted 14 lines
drivers/media/v4l2-core/v4l2-subdev.c +2
-0+ +return 0 ; }EXPORT_SYMBOL_GPL (v4l2_subdev_routing_find_opposite_end);include/media/v4l2-subdev.h +0
-0@@ -1547,6 +1547,23 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,const struct v4l2_subdev_krouting *routing, const struct v4l2_mbus_framefmt *fmt);Quoted 4 lines
include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1551
Quoted 2 lines
include/media/v4l2-subdev.h +2
-0+/** +* v4l2_subdev_find_route() - Find routes from a (pad, stream) pair from or for ?
From or to. I'll fix this in the next version.
Quoted 6 lines
include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1552
Quoted 2 lines
include/media/v4l2-subdev.h +1
-0+* @routing: routing used to find the opposite side I would not say "opposite side" but rather
@routing: routing table used to enumerate routes
How about simply "the routing table"?
Quoted 8 lines
include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1556
Quoted 5 lines
include/media/v4l2-subdev.h +4
-0+* @pad: pad id +* @stream: stream id +* @active: set to true for looking up only active routes +* @index: for accessing more than one route from the pad I understand this but maybe
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
@index: route index for enumerating multiple routes
?Sounds good.
I'm curious to know how this parameter will be used. In the only user (in patch 12/13), it is hardcoded to 0. I'm not sure indexing routes will be very useful for drivers.
Looks like it'll be needed by v4l2_subdev_set_streams_enabled(), as a result on discussion related to it.
Quoted 40 lines
Quoted 40 lines
Quoted 10 lines
include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1559
Quoted 4 lines
include/media/v4l2-subdev.h +3
-0+* +* Find a route from the routing table where one end has (pad, stream) pair +* matching @pad and @stream. * If multiple routes in @routing match @pad and @stream, return
* the @index one.
*
* Set @active to true to only enumerate active routes.Quoted 8 lines
include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1561
Quoted 2 lines
include/media/v4l2-subdev.h +2
-0+* +* Returns the route on success or -ENOENT if no matching route is found. I see other functions documentation using
* Return:
is this a kernel-doc thing ?
Yes, makes sense.
Quoted 18 lines
Quoted 9 lines
include/media/v4l2-subdev.h +5
-0+*/ +struct v4l2_subdev_route * +v4l2_subdev_find_route (const struct v4l2_subdev_krouting *routing, +u32 pad,u32 stream,bool active,unsigned int index); +/** * v4l2_subdev_routing_find_opposite_end() - Find the opposite stream * @routing: routing used to find the opposite side include/media/v4l2-subdev.h int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, · line 1576
Quoted 5 lines
include/media/v4l2-subdev.h +2
-0@@ -1555,6 +1572,8 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,* @other_pad: pointer used to return the opposite pad * @other_stream: pointer used to return the opposite stream * +* Prefer v4l2_subdev_find_route() over v4l2_subdev_routing_find_opposite_end(). +* As said, I'm not sure if that's preferred or we should rather create more helpers using
v4l2_subdev_find_route()internally. Time will tell I guess ?
Quoted 7 lines
Quoted 4 lines
I agree.
The benefit of the older function was that it returns information that doesn't need a lock for accessing it.
I'm not sure to follow you here.
v4l2_subdev_find_route()has the same locking requirements asv4l2_subdev_routing_find_opposite_end().
The function itself does, but a lock is not required for accessing information returned by v4l2_subdev_routing_find_opposite_end() (unlike v4l2_subdev_find_route()).
Quoted 7 lines
Given that
v4l2_subdev_find_route()has a single user, and that the only difference in that user compared tov4l2_subdev_routing_find_opposite_end()is that only active routes are considered, I would prefer modifyingv4l2_subdev_routing_find_opposite_end()to ignore inactive routes after checking that no caller would break.
Signature · 3 lines
Regards, Sakari Ailus
Sakari Ailus
maintainerPATCH 12/13 media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()
In reply to Sakari Ailusv4l2_mc_pipeline_enabled() helps solving a problem known for long but lacked any sort of general solution: with multiple streams, when streaming is started on video nodes one by one, when should streaming be started in the source?
v4l2_mc_pipeline_enabled() traverses the pipeline towards the source, queries the streams generated by the source and traces them back to the video nodes.
--- drivers/media/v4l2-core/v4l2-mc.c | 243 ++++++++++++++++++++++++++++++ include/media/v4l2-mc.h | 44 ++++++ 2 files changed, 287 insertions(+)
drivers/media/v4l2-core/v4l2-mc.c +243 -0
@@ -612,3 +612,246 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,return ret; }EXPORT_SYMBOL_GPL (v4l2_pipeline_link_notify); + +static int +__v4l2_mc_pipeline_enabled (struct v4l2_subdev_state *state, +struct media_pad *src_pad,u64 __src_streams, +struct media_pad **__sink_pad,u64 *__sink_streams) +{ +struct v4l2_subdev_route *route; +u64 src_streams =0 , sink_streams =0 ; +bool has_sink_pad = false; +unsigned int sink_pad; + +dev_dbg (state->sd->dev,"%s: source enabled, pad/streams %u/%#llx\n" , + state->sd->entity.name, src_pad->index, __src_streams); +for_each_active_route (&state->routing, route) { +dev_dbg (state->sd->dev,"%s: %u/%u -> %u/%u, flags %x\n" , + state->sd->entity.name, + route->sink_pad, route->sink_stream, route->source_pad, + route->source_stream, route->flags); +if (route->source_pad != src_pad->index) +continue ; + +if (!(BIT_ULL (route->source_stream) & __src_streams)) +continue ; + +if (!has_sink_pad) { + has_sink_pad = true; + sink_pad = route->sink_pad; + } + +if (route->sink_pad != sink_pad) { +dev_dbg (state->sd->dev, +"sink pads (%u vs. %u) differ\n" , + route->sink_pad, sink_pad); +return -EMLINK ; + } + + sink_streams |=BIT_ULL (route->sink_stream); + src_streams |=BIT_ULL (route->source_stream); + } + + *__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] :NULL ; + *__sink_streams = sink_streams; + +return 0 ; +} + +static int v4l2_mc_downpath_enabled (struct media_pad *sink_pad, +unsigned int sink_stream, +bool (*func)(struct video_device *vdev), +struct media_pad **__sink_pad, +u64 *__sink_streams) +{ +struct v4l2_subdev_state *state; +struct v4l2_subdev_route *route; +struct v4l2_subdev *sd; +struct media_pad *source_pad, *tmp_pad; +u32 source_stream; + +if (!is_media_entity_v4l2_subdev (sink_pad->entity)) +return -ENXIO ; + + sd =media_entity_to_v4l2_subdev (sink_pad->entity); +dev_dbg (sd->dev,"path_enabled: found sub-device %s\n" , + sd->entity.name); + + state =v4l2_subdev_lock_and_get_active_state (sd); + route =v4l2_subdev_find_route (&state->routing, sink_pad->index, + sink_stream, true,0 ); +if (IS_ERR (route)) { +v4l2_subdev_unlock_state (state); +dev_dbg (sd->dev, +"path_enabled: can't find opposite route for %s:%u/%u" , + sd->entity.name, sink_pad->index, sink_stream); +return 2 ; + } + + source_pad = &sd->entity.pads[route->source_pad]; +v4l2_subdev_unlock_state (state); + + tmp_pad = sink_pad; + sink_pad =media_pad_remote_pad_unique (source_pad); +if (IS_ERR (sink_pad)) { +dev_dbg (sd->dev, +"path_enabled: can't find remote source for %s:%u\n" , + source_pad->entity->name, source_pad->index); +return PTR_ERR (sink_pad); + } + +if (is_media_entity_v4l2_video_device (sink_pad->entity)) { +struct video_device *vdev; + + vdev =media_entity_to_video_device (sink_pad->entity); +if (!vdev) +return -ENXIO ; + +dev_dbg (vdev->dev_parent, +"path_enabled: found video device %s\n" , + vdev->name); + +if (!*__sink_pad) { + *__sink_pad = tmp_pad; +dev_dbg (sd->dev,"path_enabled: sink %u/%u\n" , + tmp_pad->index, sink_stream); + }else if (tmp_pad != *__sink_pad) { +dev_dbg (sd->dev, +"path_enabled: pads %s/%u and %s/%u differ\n" , + tmp_pad->entity->name, tmp_pad->index, + (*__sink_pad)->entity->name, + (*__sink_pad)->index); +return -EXDEV ; + } + + *__sink_streams |=BIT_ULL (sink_stream); + +return func (vdev); + } + +return v4l2_mc_downpath_enabled (sink_pad, source_stream, func, + __sink_pad, __sink_streams); +} + +static int v4l2_mc_source_get_streams (struct v4l2_subdev *sd,unsigned int pad, +u64 *__streams) +{ +struct v4l2_mbus_frame_desc desc; +u64 streams =0 ; +int ret; + +if (!__streams) +return -EINVAL ; + + ret =v4l2_subdev_call (sd, pad, get_frame_desc, pad, &desc); +if (ret == -ENOIOCTLCMD ) { + *__streams =1ULL ; +return 0 ; + } +if (ret) +return ret; + +for (unsigned int i =0 ; i < desc.num_entries; i++) { +if (streams &BIT_ULL (desc.entry[i].stream)) +return -EINVAL ; + + streams |=BIT_ULL (desc.entry[i].stream); + } + +dev_dbg (sd->dev,"found streams %#llx based on streams %#llx\n" , + *__streams, streams); +if (*__streams & ~streams) +return -EINVAL ; + + *__streams = streams; + +return 0 ; +} + +int v4l2_mc_pipeline_enabled (struct video_device *vdev, +bool (*func)(struct video_device *vdev), +struct media_pad **__sink_pad,u64 *__sink_streams) +{ +u64 sink_streams =1U ; +struct media_pad *src_pad; +u64 src_streams; +struct v4l2_subdev_state *state; +struct media_pad *sink_pad = vdev->entity.pads; +struct v4l2_subdev *sd =NULL ; +bool streaming = true; +struct media_pad *tmp_pad; +u64 tmp_streams; +int ret; + +if (!__sink_pad) + __sink_pad = &tmp_pad; +if (!__sink_streams) + __sink_streams = &tmp_streams; + *__sink_pad =NULL ; + *__sink_streams =0 ; + +do { + src_pad =media_pad_remote_pad_unique (sink_pad); +if (IS_ERR (src_pad)) { +dev_dbg (sd ? sd->dev : vdev->dev_parent, +"no unique remote pad found from %s:%u\n" , + sink_pad->entity->name, sink_pad->index); +return PTR_ERR (src_pad); + } + + sd =media_entity_to_v4l2_subdev (src_pad->entity); +if (!sd) { +dev_dbg (sd->dev, +"media entity %s is not a V4L2 sub-device\n" , + src_pad->entity->name); +return -ENXIO ; + } + +/* Source streams match sink. */ + src_streams = sink_streams; + + state =v4l2_subdev_lock_and_get_active_state (sd); + ret =__v4l2_mc_pipeline_enabled (state, src_pad, + src_streams, &sink_pad, + &sink_streams); +v4l2_subdev_unlock_state (state); +if (ret) +return ret; + }while (sink_pad); + + ret =v4l2_mc_source_get_streams (sd, src_pad->index, &src_streams); +if (ret) +return ret; + + sd =media_entity_to_v4l2_subdev (src_pad->entity); + +dev_dbg (sd->dev,"following %s:%u/%#llx\n" , sd->entity.name, + src_pad->index, src_streams); + +for (unsigned int i =__ffs (src_streams); src_streams; + src_streams &= ~BIT_ULL (i), i =__ffs (src_streams)) { + sink_pad =media_pad_remote_pad_unique (src_pad); +if (IS_ERR (src_pad)) { +dev_dbg (sd->dev, +"no unique remote pad found from %s:%u\n" , + sink_pad->entity->name, sink_pad->index); +return PTR_ERR (src_pad); + } + + ret =v4l2_mc_downpath_enabled (sink_pad, i, func, __sink_pad, + __sink_streams); +if (ret ==2 ) +continue ; +if (ret <0 ) +return ret; +if (!ret) + streaming = false; + } + +dev_dbg (media_entity_to_v4l2_subdev ((*__sink_pad)->entity)->dev, +"sink pad %s:%u/%#llx\n" , (*__sink_pad)->entity->name, + (*__sink_pad)->index, *__sink_streams); + +return streaming; +} +EXPORT_SYMBOL_GPL (v4l2_mc_pipeline_enabled);
include/media/v4l2-mc.h +44 -0
50 lines not on this page · raw message
Signature · 1 line
2.39.5
kernel test robot
build botRe: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()
kernel test robot noticed the following build warnings:
Hi Sakari,
kernel test robot noticed the following build warnings:
[auto build test WARNING on linuxtv-media-pending/master]
[also build test WARNING on linus/master media-tree/master v6.16-rc2 next-20250618] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Sakari-Ailus/media-ipu6-Use-correct-pads-for-xlate_streams/20250619-161847 base: https://git.linuxtv.org/media-ci/media-pending.git master patch link: https://lore.kernel.org/r/20250619081546.1582969-13-sakari.ailus%40linux.intel.com patch subject: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled() config: x86_64-buildonly-randconfig-003-20250619 (https://download.01.org/0day-ci/archive/20250619/[email protected]/config) compiler: clang version 20.1.2 (https://github.com/llvm/llvm-project 58df0ef89dd64126512e4ee27b4ac3fd8ddf6247) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250619/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
>> drivers/media/v4l2-core/v4l2-mc.c:622:6: warning: variable 'src_streams' set but not used [-Wunused-but-set-variable]
622 | u64 src_streams = 0, sink_streams = 0;
| ^
>> drivers/media/v4l2-core/v4l2-mc.c:732:44: warning: variable 'source_stream' is uninitialized when used here [-Wuninitialized]
732 | return v4l2_mc_downpath_enabled(sink_pad, source_stream, func,
| ^~~~~~~~~~~~~
drivers/media/v4l2-core/v4l2-mc.c:671:19: note: initialize the variable 'source_stream' to silence this warning
671 | u32 source_stream;
| ^
| = 02 warnings generated.
vim +/src_streams +622 drivers/media/v4l2-core/v4l2-mc.c
615
616 static int
617 __v4l2_mc_pipeline_enabled(struct v4l2_subdev_state *state,
618 struct media_pad *src_pad, u64 __src_streams,
619 struct media_pad **__sink_pad, u64 *__sink_streams)
620 {
621 struct v4l2_subdev_route *route;
> 622 u64 src_streams = 0, sink_streams = 0;
623 bool has_sink_pad = false;
624 unsigned int sink_pad;
625
626 dev_dbg(state->sd->dev, "%s: source enabled, pad/streams %u/%#llx\n",
627 state->sd->entity.name, src_pad->index, __src_streams);
628 for_each_active_route(&state->routing, route) {
629 dev_dbg(state->sd->dev, "%s: %u/%u -> %u/%u, flags %x\n",
630 state->sd->entity.name,
631 route->sink_pad, route->sink_stream, route->source_pad,
632 route->source_stream, route->flags);
633 if (route->source_pad != src_pad->index)
634 continue;
635
636 if (!(BIT_ULL(route->source_stream) & __src_streams))
637 continue;
638
639 if (!has_sink_pad) {
640 has_sink_pad = true;
641 sink_pad = route->sink_pad;
642 }
643
644 if (route->sink_pad != sink_pad) {
645 dev_dbg(state->sd->dev,
646 "sink pads (%u vs. %u) differ\n",
647 route->sink_pad, sink_pad);
648 return -EMLINK;
649 }
650
651 sink_streams |= BIT_ULL(route->sink_stream);
652 src_streams |= BIT_ULL(route->source_stream);
653 }
654
655 *__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] : NULL;
656 *__sink_streams = sink_streams;
657
658 return 0;
659 }
660
661 static int v4l2_mc_downpath_enabled(struct media_pad *sink_pad,
662 unsigned int sink_stream,
663 bool (*func)(struct video_device *vdev),
664 struct media_pad **__sink_pad,
665 u64 *__sink_streams)
666 {
667 struct v4l2_subdev_state *state;
668 struct v4l2_subdev_route *route;
669 struct v4l2_subdev *sd;
670 struct media_pad *source_pad, *tmp_pad;
671 u32 source_stream;
672
673 if (!is_media_entity_v4l2_subdev(sink_pad->entity))
674 return -ENXIO;
675
676 sd = media_entity_to_v4l2_subdev(sink_pad->entity);
677 dev_dbg(sd->dev, "path_enabled: found sub-device %s\n",
678 sd->entity.name);
679
680 state = v4l2_subdev_lock_and_get_active_state(sd);
681 route = v4l2_subdev_find_route(&state->routing, sink_pad->index,
682 sink_stream, true, 0);
683 if (IS_ERR(route)) {
684 v4l2_subdev_unlock_state(state);
685 dev_dbg(sd->dev,
686 "path_enabled: can't find opposite route for %s:%u/%u",
687 sd->entity.name, sink_pad->index, sink_stream);
688 return 2;
689 }
690
691 source_pad = &sd->entity.pads[route->source_pad];
692 v4l2_subdev_unlock_state(state);
693
694 tmp_pad = sink_pad;
695 sink_pad = media_pad_remote_pad_unique(source_pad);
696 if (IS_ERR(sink_pad)) {
697 dev_dbg(sd->dev,
698 "path_enabled: can't find remote source for %s:%u\n",
699 source_pad->entity->name, source_pad->index);
700 return PTR_ERR(sink_pad);
701 }
702
703 if (is_media_entity_v4l2_video_device(sink_pad->entity)) {
704 struct video_device *vdev;
705
706 vdev = media_entity_to_video_device(sink_pad->entity);
707 if (!vdev)
708 return -ENXIO;
709
710 dev_dbg(vdev->dev_parent,
711 "path_enabled: found video device %s\n",
712 vdev->name);
713
714 if (!*__sink_pad) {
715 *__sink_pad = tmp_pad;
716 dev_dbg(sd->dev, "path_enabled: sink %u/%u\n",
717 tmp_pad->index, sink_stream);
718 } else if (tmp_pad != *__sink_pad) {
719 dev_dbg(sd->dev,
720 "path_enabled: pads %s/%u and %s/%u differ\n",
721 tmp_pad->entity->name, tmp_pad->index,
722 (*__sink_pad)->entity->name,
723 (*__sink_pad)->index);
724 return -EXDEV;
725 }
726
727 *__sink_streams |= BIT_ULL(sink_stream);
728
729 return func(vdev);
730 }
731
> 732 return v4l2_mc_downpath_enabled(sink_pad, source_stream, func, 733 __sink_pad, __sink_streams);
734 }
735
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()
In reply to Sakari AilusHi Sakari,
kernel test robot noticed the following build warnings:
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Sakari-Ailus/media-ipu6-Use-correct-pads-for-xlate_streams/20250619-161847 base: https://git.linuxtv.org/media-ci/media-pending.git master patch link: https://lore.kernel.org/r/20250619081546.1582969-13-sakari.ailus%40linux.intel.com patch subject: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled() config: i386-randconfig-141-20250620 (https://download.01.org/0day-ci/archive/20250620/[email protected]/config) compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]> | Reported-by: Dan Carpenter <[email protected]> | Closes: https://lore.kernel.org/r/[email protected]/
smatch warnings:drivers/media/v4l2-core/v4l2-mc.c:732 v4l2_mc_downpath_enabled() error: uninitialized symbol 'source_stream'. drivers/media/v4l2-core/v4l2-mc.c:834 v4l2_mc_pipeline_enabled() warn: variable dereferenced before IS_ERR check 'src_pad' (see line 802)
vim +/source_stream +732 drivers/media/v4l2-core/v4l2-mc.c
bc83f513f8ad94 Sakari Ailus 2025-06-19 771 int v4l2_mc_pipeline_enabled(struct video_device *vdev,
bc83f513f8ad94 Sakari Ailus 2025-06-19 772 bool (*func)(struct video_device *vdev),
bc83f513f8ad94 Sakari Ailus 2025-06-19 773 struct media_pad **__sink_pad, u64 *__sink_streams)
bc83f513f8ad94 Sakari Ailus 2025-06-19 774 {
bc83f513f8ad94 Sakari Ailus 2025-06-19 775 u64 sink_streams = 1U;
bc83f513f8ad94 Sakari Ailus 2025-06-19 776 struct media_pad *src_pad;
bc83f513f8ad94 Sakari Ailus 2025-06-19 777 u64 src_streams;
bc83f513f8ad94 Sakari Ailus 2025-06-19 778 struct v4l2_subdev_state *state;
bc83f513f8ad94 Sakari Ailus 2025-06-19 779 struct media_pad *sink_pad = vdev->entity.pads;
bc83f513f8ad94 Sakari Ailus 2025-06-19 780 struct v4l2_subdev *sd = NULL;
bc83f513f8ad94 Sakari Ailus 2025-06-19 781 bool streaming = true;
bc83f513f8ad94 Sakari Ailus 2025-06-19 782 struct media_pad *tmp_pad;
bc83f513f8ad94 Sakari Ailus 2025-06-19 783 u64 tmp_streams;
bc83f513f8ad94 Sakari Ailus 2025-06-19 784 int ret;
bc83f513f8ad94 Sakari Ailus 2025-06-19 785
bc83f513f8ad94 Sakari Ailus 2025-06-19 786 if (!__sink_pad)
bc83f513f8ad94 Sakari Ailus 2025-06-19 787 __sink_pad = &tmp_pad;
bc83f513f8ad94 Sakari Ailus 2025-06-19 788 if (!__sink_streams)
bc83f513f8ad94 Sakari Ailus 2025-06-19 789 __sink_streams = &tmp_streams;
bc83f513f8ad94 Sakari Ailus 2025-06-19 790 *__sink_pad = NULL;
bc83f513f8ad94 Sakari Ailus 2025-06-19 791 *__sink_streams = 0;
bc83f513f8ad94 Sakari Ailus 2025-06-19 792
bc83f513f8ad94 Sakari Ailus 2025-06-19 793 do {
bc83f513f8ad94 Sakari Ailus 2025-06-19 794 src_pad = media_pad_remote_pad_unique(sink_pad);
bc83f513f8ad94 Sakari Ailus 2025-06-19 795 if (IS_ERR(src_pad)) {
bc83f513f8ad94 Sakari Ailus 2025-06-19 796 dev_dbg(sd ? sd->dev : vdev->dev_parent,
bc83f513f8ad94 Sakari Ailus 2025-06-19 797 "no unique remote pad found from %s:%u\n",
bc83f513f8ad94 Sakari Ailus 2025-06-19 798 sink_pad->entity->name, sink_pad->index);
bc83f513f8ad94 Sakari Ailus 2025-06-19 799 return PTR_ERR(src_pad);
bc83f513f8ad94 Sakari Ailus 2025-06-19 800 }
bc83f513f8ad94 Sakari Ailus 2025-06-19 801
bc83f513f8ad94 Sakari Ailus 2025-06-19 @802 sd = media_entity_to_v4l2_subdev(src_pad->entity);
bc83f513f8ad94 Sakari Ailus 2025-06-19 803 if (!sd) {
bc83f513f8ad94 Sakari Ailus 2025-06-19 804 dev_dbg(sd->dev,
bc83f513f8ad94 Sakari Ailus 2025-06-19 805 "media entity %s is not a V4L2 sub-device\n",
bc83f513f8ad94 Sakari Ailus 2025-06-19 806 src_pad->entity->name);
bc83f513f8ad94 Sakari Ailus 2025-06-19 807 return -ENXIO;
bc83f513f8ad94 Sakari Ailus 2025-06-19 808 }
bc83f513f8ad94 Sakari Ailus 2025-06-19 809
bc83f513f8ad94 Sakari Ailus 2025-06-19 810 /* Source streams match sink. */
bc83f513f8ad94 Sakari Ailus 2025-06-19 811 src_streams = sink_streams;
bc83f513f8ad94 Sakari Ailus 2025-06-19 812
bc83f513f8ad94 Sakari Ailus 2025-06-19 813 state = v4l2_subdev_lock_and_get_active_state(sd);
bc83f513f8ad94 Sakari Ailus 2025-06-19 814 ret = __v4l2_mc_pipeline_enabled(state, src_pad,
bc83f513f8ad94 Sakari Ailus 2025-06-19 815 src_streams, &sink_pad,
bc83f513f8ad94 Sakari Ailus 2025-06-19 816 &sink_streams);
bc83f513f8ad94 Sakari Ailus 2025-06-19 817 v4l2_subdev_unlock_state(state);
bc83f513f8ad94 Sakari Ailus 2025-06-19 818 if (ret)
bc83f513f8ad94 Sakari Ailus 2025-06-19 819 return ret;
bc83f513f8ad94 Sakari Ailus 2025-06-19 820 } while (sink_pad);
bc83f513f8ad94 Sakari Ailus 2025-06-19 821
bc83f513f8ad94 Sakari Ailus 2025-06-19 822 ret = v4l2_mc_source_get_streams(sd, src_pad->index, &src_streams);
bc83f513f8ad94 Sakari Ailus 2025-06-19 823 if (ret)
bc83f513f8ad94 Sakari Ailus 2025-06-19 824 return ret;
bc83f513f8ad94 Sakari Ailus 2025-06-19 825
bc83f513f8ad94 Sakari Ailus 2025-06-19 826 sd = media_entity_to_v4l2_subdev(src_pad->entity);
bc83f513f8ad94 Sakari Ailus 2025-06-19 827
bc83f513f8ad94 Sakari Ailus 2025-06-19 828 dev_dbg(sd->dev, "following %s:%u/%#llx\n", sd->entity.name,
bc83f513f8ad94 Sakari Ailus 2025-06-19 829 src_pad->index, src_streams);
bc83f513f8ad94 Sakari Ailus 2025-06-19 830
bc83f513f8ad94 Sakari Ailus 2025-06-19 831 for (unsigned int i = __ffs(src_streams); src_streams;
bc83f513f8ad94 Sakari Ailus 2025-06-19 832 src_streams &= ~BIT_ULL(i), i = __ffs(src_streams)) {
bc83f513f8ad94 Sakari Ailus 2025-06-19 833 sink_pad = media_pad_remote_pad_unique(src_pad);
bc83f513f8ad94 Sakari Ailus 2025-06-19 @834 if (IS_ERR(src_pad)) {
^^^^^^^Copy and paste. s/src_pad/sink_pad/.
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()
In reply to Sakari AilusHi Sakari
Quoted 4 lines · Sakari Ailus
v4l2_mc_pipeline_enabled()helps solving a problem known for long but lacked any sort of general solution: with multiple streams, when streaming is started on video nodes one by one, when should streaming be started in the source?
I tried quite some time to understand this, but if I'm not mistaken, a stream-aware subdev, which links to vdev, will always "demux" streams to different pads and will connect to the vdev from there
Source
subdev
+-----------------+ | (1/0) ------> vdev0 | | (0)[1,2,3] (2/0 ------> vdev1 | | | (3/0) ------> vdev2 +-----------------+
With
(0) multiplexed sink pad with 3 streams
(1) (2) and (3) source pad with a single stream
Can't we relay on the media-link state between the source pads and the video devices with something like what Dan has proposed here ?
https://patchwork.linuxtv.org/project/linux-media/patch/[email protected]/
What am I missing ?
Quoted 321 lines
v4l2_mc_pipeline_enabled()traverses the pipeline towards the source, queries the streams generated by the source and traces them back to the video nodes.Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/v4l2-core/v4l2-mc.c | 243 ++++++++++++++++++++++++++++++ include/media/v4l2-mc.h | 44 ++++++ 2 files changed, 287 insertions(+)drivers/media/v4l2-core/v4l2-mc.c +243
-0@@ -612,3 +612,246 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,return ret; }EXPORT_SYMBOL_GPL (v4l2_pipeline_link_notify); + +static int +__v4l2_mc_pipeline_enabled (struct v4l2_subdev_state *state, +struct media_pad *src_pad,u64 __src_streams, +struct media_pad **__sink_pad,u64 *__sink_streams) +{ +struct v4l2_subdev_route *route; +u64 src_streams =0 , sink_streams =0 ; +bool has_sink_pad = false; +unsigned int sink_pad; + +dev_dbg (state->sd->dev,"%s: source enabled, pad/streams %u/%#llx\n" , + state->sd->entity.name, src_pad->index, __src_streams); +for_each_active_route (&state->routing, route) { +dev_dbg (state->sd->dev,"%s: %u/%u -> %u/%u, flags %x\n" , + state->sd->entity.name, + route->sink_pad, route->sink_stream, route->source_pad, + route->source_stream, route->flags); +if (route->source_pad != src_pad->index) +continue ; + +if (!(BIT_ULL (route->source_stream) & __src_streams)) +continue ; + +if (!has_sink_pad) { + has_sink_pad = true; + sink_pad = route->sink_pad; + } + +if (route->sink_pad != sink_pad) { +dev_dbg (state->sd->dev, +"sink pads (%u vs. %u) differ\n" , + route->sink_pad, sink_pad); +return -EMLINK ; + } + + sink_streams |=BIT_ULL (route->sink_stream); + src_streams |=BIT_ULL (route->source_stream); + } + + *__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] :NULL ; + *__sink_streams = sink_streams; + +return 0 ; +} + +static int v4l2_mc_downpath_enabled (struct media_pad *sink_pad, +unsigned int sink_stream, +bool (*func)(struct video_device *vdev), +struct media_pad **__sink_pad, +u64 *__sink_streams) +{ +struct v4l2_subdev_state *state; +struct v4l2_subdev_route *route; +struct v4l2_subdev *sd; +struct media_pad *source_pad, *tmp_pad; +u32 source_stream; + +if (!is_media_entity_v4l2_subdev (sink_pad->entity)) +return -ENXIO ; + + sd =media_entity_to_v4l2_subdev (sink_pad->entity); +dev_dbg (sd->dev,"path_enabled: found sub-device %s\n" , + sd->entity.name); + + state =v4l2_subdev_lock_and_get_active_state (sd); + route =v4l2_subdev_find_route (&state->routing, sink_pad->index, + sink_stream, true,0 ); +if (IS_ERR (route)) { +v4l2_subdev_unlock_state (state); +dev_dbg (sd->dev, +"path_enabled: can't find opposite route for %s:%u/%u" , + sd->entity.name, sink_pad->index, sink_stream); +return 2 ; + } + + source_pad = &sd->entity.pads[route->source_pad]; +v4l2_subdev_unlock_state (state); + + tmp_pad = sink_pad; + sink_pad =media_pad_remote_pad_unique (source_pad); +if (IS_ERR (sink_pad)) { +dev_dbg (sd->dev, +"path_enabled: can't find remote source for %s:%u\n" , + source_pad->entity->name, source_pad->index); +return PTR_ERR (sink_pad); + } + +if (is_media_entity_v4l2_video_device (sink_pad->entity)) { +struct video_device *vdev; + + vdev =media_entity_to_video_device (sink_pad->entity); +if (!vdev) +return -ENXIO ; + +dev_dbg (vdev->dev_parent, +"path_enabled: found video device %s\n" , + vdev->name); + +if (!*__sink_pad) { + *__sink_pad = tmp_pad; +dev_dbg (sd->dev,"path_enabled: sink %u/%u\n" , + tmp_pad->index, sink_stream); + }else if (tmp_pad != *__sink_pad) { +dev_dbg (sd->dev, +"path_enabled: pads %s/%u and %s/%u differ\n" , + tmp_pad->entity->name, tmp_pad->index, + (*__sink_pad)->entity->name, + (*__sink_pad)->index); +return -EXDEV ; + } + + *__sink_streams |=BIT_ULL (sink_stream); + +return func (vdev); + } + +return v4l2_mc_downpath_enabled (sink_pad, source_stream, func, + __sink_pad, __sink_streams); +} + +static int v4l2_mc_source_get_streams (struct v4l2_subdev *sd,unsigned int pad, +u64 *__streams) +{ +struct v4l2_mbus_frame_desc desc; +u64 streams =0 ; +int ret; + +if (!__streams) +return -EINVAL ; + + ret =v4l2_subdev_call (sd, pad, get_frame_desc, pad, &desc); +if (ret == -ENOIOCTLCMD ) { + *__streams =1ULL ; +return 0 ; + } +if (ret) +return ret; + +for (unsigned int i =0 ; i < desc.num_entries; i++) { +if (streams &BIT_ULL (desc.entry[i].stream)) +return -EINVAL ; + + streams |=BIT_ULL (desc.entry[i].stream); + } + +dev_dbg (sd->dev,"found streams %#llx based on streams %#llx\n" , + *__streams, streams); +if (*__streams & ~streams) +return -EINVAL ; + + *__streams = streams; + +return 0 ; +} + +int v4l2_mc_pipeline_enabled (struct video_device *vdev, +bool (*func)(struct video_device *vdev), +struct media_pad **__sink_pad,u64 *__sink_streams) +{ +u64 sink_streams =1U ; +struct media_pad *src_pad; +u64 src_streams; +struct v4l2_subdev_state *state; +struct media_pad *sink_pad = vdev->entity.pads; +struct v4l2_subdev *sd =NULL ; +bool streaming = true; +struct media_pad *tmp_pad; +u64 tmp_streams; +int ret; + +if (!__sink_pad) + __sink_pad = &tmp_pad; +if (!__sink_streams) + __sink_streams = &tmp_streams; + *__sink_pad =NULL ; + *__sink_streams =0 ; + +do { + src_pad =media_pad_remote_pad_unique (sink_pad); +if (IS_ERR (src_pad)) { +dev_dbg (sd ? sd->dev : vdev->dev_parent, +"no unique remote pad found from %s:%u\n" , + sink_pad->entity->name, sink_pad->index); +return PTR_ERR (src_pad); + } + + sd =media_entity_to_v4l2_subdev (src_pad->entity); +if (!sd) { +dev_dbg (sd->dev, +"media entity %s is not a V4L2 sub-device\n" , + src_pad->entity->name); +return -ENXIO ; + } + +/* Source streams match sink. */ + src_streams = sink_streams; + + state =v4l2_subdev_lock_and_get_active_state (sd); + ret =__v4l2_mc_pipeline_enabled (state, src_pad, + src_streams, &sink_pad, + &sink_streams); +v4l2_subdev_unlock_state (state); +if (ret) +return ret; + }while (sink_pad); + + ret =v4l2_mc_source_get_streams (sd, src_pad->index, &src_streams); +if (ret) +return ret; + + sd =media_entity_to_v4l2_subdev (src_pad->entity); + +dev_dbg (sd->dev,"following %s:%u/%#llx\n" , sd->entity.name, + src_pad->index, src_streams); + +for (unsigned int i =__ffs (src_streams); src_streams; + src_streams &= ~BIT_ULL (i), i =__ffs (src_streams)) { + sink_pad =media_pad_remote_pad_unique (src_pad); +if (IS_ERR (src_pad)) { +dev_dbg (sd->dev, +"no unique remote pad found from %s:%u\n" , + sink_pad->entity->name, sink_pad->index); +return PTR_ERR (src_pad); + } + + ret =v4l2_mc_downpath_enabled (sink_pad, i, func, __sink_pad, + __sink_streams); +if (ret ==2 ) +continue ; +if (ret <0 ) +return ret; +if (!ret) + streaming = false; + } + +dev_dbg (media_entity_to_v4l2_subdev ((*__sink_pad)->entity)->dev, +"sink pad %s:%u/%#llx\n" , (*__sink_pad)->entity->name, + (*__sink_pad)->index, *__sink_streams); + +return streaming; +} +EXPORT_SYMBOL_GPL (v4l2_mc_pipeline_enabled);include/media/v4l2-mc.h +44
-1@@ -193,6 +193,50 @@ void v4l2_pipeline_pm_put(struct media_entity *entity);int v4l2_pipeline_link_notify (struct media_link *link,u32 flags,unsigned int notification); +/** +* v4l2_mc_pipeline_enabled - Tell when to start streaming +* @vdev: The video device +* @func: Caller-provided function to tell a video device's streaming state +* @__sink_pad: sink pad at the root of the local pipeline +* @__sink_streams: streams to start +* +* Use to tell whether streaming should start on a video node. @func returns +* true if streaming has been started on a given video node. @__sink_pad and +* @__sink_streams are filled with pad and streams on the sub-device closest to +* the video nodes, to be used for calling v4l2_subdev_enable_streams() and +* v4l2_subdev_disable_streams(). +* +* Using v4l2_mc_pipeline_enabled() has a few limitations currently (consider it +* a to-do list): +* * only unbranched streams can be supported albeit adding support for +* downstream branches would be fairly trivial, +* * streams within a single source sub-device are considered to start at the +* same time, more control could be added in two ways: 1) for sources to +* determine stream starting, a control could be added to UAPI and 2) sources +* could tell which streams start at the same time using a sub-device +* operation, +* * CSI-2 VC framing is ignored currently, but VC-based stream starting could +* be implemented by letting the caller to provide a function to determine +* which streams are of interest and +* * routes leading to nowhere are ignored, on some hardware this is a problem, +* but this can also be rather trivially addressed. +* +* Return: +* * 0: Success, but don't start streaming yet +* * 1: Success, now it's time to start streaming +* * -ENXIO: Route traversal encountered a non-video device/sub-device entity +* * -ENOTUNIQ: No unique remote pad +* * -ENOLINK: No remote pad found +* * -ENOENT: Enabled upstream route not found +* * -EMLINK: No unique downstream route found +* * -EINVAL: Stream could not be followed to source or was not produced by +* the source +*/ +int v4l2_mc_pipeline_enabled (struct video_device *vdev, +bool (*func)(struct video_device *vdev), +struct media_pad **__sink_pad, +u64 *__sink_streams); +#else /* CONFIG_MEDIA_CONTROLLER */ static inline int v4l2_mc_create_media_graph (struct media_device *mdev) --2.39.5
Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()
Hi Jacopo,
Thank you for the comments.
Quoted 25 lines · Jacopo Mondi
Hi Sakari
Quoted 4 lines · Sakari Ailus
v4l2_mc_pipeline_enabled()helps solving a problem known for long but lacked any sort of general solution: with multiple streams, when streaming is started on video nodes one by one, when should streaming be started in the source?I tried quite some time to understand this, but if I'm not mistaken, a stream-aware subdev, which links to vdev, will always "demux" streams to different pads and will connect to the vdev from there
Source
subdev+-----------------+ | (1/0) ------> vdev0 | | (0)[1,2,3] (2/0 ------> vdev1 | | | (3/0) ------> vdev2 +-----------------+With
Quoted 8 lines
(0) multiplexed sink pad with 3 streams
(1) (2) and (3) source pad with a single streamCan't we relay on the media-link state between the source pads and the video devices with something like what Dan has proposed here ?
https://patchwork.linuxtv.org/project/linux-media/patch/[email protected]/What am I missing ?
There are two different concepts here that are probably underdocumented at the moment: Media entity pipeline state and then when and in which order streaming control is applied on hardware. But... maybe the latter can be derived from the former when it comes to video nodes. The pipeline is set for entities that are upstream from the video device, it's not set on downstream entities. I'll see if I could use that instead. Otherwise, I'll check if Dan's approach could be used for the purpose.
Signature · 3 lines
Regards, Sakari Ailus
Sakari Ailus
maintainerRe: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()
In reply to Jacopo MondiHi Jacopo,
Quoted 23 lines · Jacopo Mondi
Hi Sakari
Quoted 4 lines · Sakari Ailus
v4l2_mc_pipeline_enabled()helps solving a problem known for long but lacked any sort of general solution: with multiple streams, when streaming is started on video nodes one by one, when should streaming be started in the source?I tried quite some time to understand this, but if I'm not mistaken, a stream-aware subdev, which links to vdev, will always "demux" streams to different pads and will connect to the vdev from there
Source
subdev+-----------------+ | (1/0) ------> vdev0 | | (0)[1,2,3] (2/0 ------> vdev1 | | | (3/0) ------> vdev2 +-----------------+
Quoted 8 lines
With
(0) multiplexed sink pad with 3 streams
(1) (2) and (3) source pad with a single streamCan't we relay on the media-link state between the source pads and the video devices with something like what Dan has proposed here ?
https://patchwork.linuxtv.org/project/linux-media/patch/[email protected]/
This isn't entirely the same thing: pipeline is specific to the pad but not streams, ad here we're interested in streams. Two streams may start at different points of time even if both are part of the same pipeline.
We definitely should have just one way to figure this out.
I'll send v2 with comments and before that also see if (or how) we could get rid of the callback.
Cc Dan.
Quoted 320 lines
What am I missing ?
Quoted 317 lines
v4l2_mc_pipeline_enabled()traverses the pipeline towards the source, queries the streams generated by the source and traces them back to the video nodes.Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/v4l2-core/v4l2-mc.c | 243 ++++++++++++++++++++++++++++++ include/media/v4l2-mc.h | 44 ++++++ 2 files changed, 287 insertions(+)drivers/media/v4l2-core/v4l2-mc.c +243
-0@@ -612,3 +612,246 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,return ret; }EXPORT_SYMBOL_GPL (v4l2_pipeline_link_notify); + +static int +__v4l2_mc_pipeline_enabled (struct v4l2_subdev_state *state, +struct media_pad *src_pad,u64 __src_streams, +struct media_pad **__sink_pad,u64 *__sink_streams) +{ +struct v4l2_subdev_route *route; +u64 src_streams =0 , sink_streams =0 ; +bool has_sink_pad = false; +unsigned int sink_pad; + +dev_dbg (state->sd->dev,"%s: source enabled, pad/streams %u/%#llx\n" , + state->sd->entity.name, src_pad->index, __src_streams); +for_each_active_route (&state->routing, route) { +dev_dbg (state->sd->dev,"%s: %u/%u -> %u/%u, flags %x\n" , + state->sd->entity.name, + route->sink_pad, route->sink_stream, route->source_pad, + route->source_stream, route->flags); +if (route->source_pad != src_pad->index) +continue ; + +if (!(BIT_ULL (route->source_stream) & __src_streams)) +continue ; + +if (!has_sink_pad) { + has_sink_pad = true; + sink_pad = route->sink_pad; + } + +if (route->sink_pad != sink_pad) { +dev_dbg (state->sd->dev, +"sink pads (%u vs. %u) differ\n" , + route->sink_pad, sink_pad); +return -EMLINK ; + } + + sink_streams |=BIT_ULL (route->sink_stream); + src_streams |=BIT_ULL (route->source_stream); + } + + *__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] :NULL ; + *__sink_streams = sink_streams; + +return 0 ; +} + +static int v4l2_mc_downpath_enabled (struct media_pad *sink_pad, +unsigned int sink_stream, +bool (*func)(struct video_device *vdev), +struct media_pad **__sink_pad, +u64 *__sink_streams) +{ +struct v4l2_subdev_state *state; +struct v4l2_subdev_route *route; +struct v4l2_subdev *sd; +struct media_pad *source_pad, *tmp_pad; +u32 source_stream; + +if (!is_media_entity_v4l2_subdev (sink_pad->entity)) +return -ENXIO ; + + sd =media_entity_to_v4l2_subdev (sink_pad->entity); +dev_dbg (sd->dev,"path_enabled: found sub-device %s\n" , + sd->entity.name); + + state =v4l2_subdev_lock_and_get_active_state (sd); + route =v4l2_subdev_find_route (&state->routing, sink_pad->index, + sink_stream, true,0 ); +if (IS_ERR (route)) { +v4l2_subdev_unlock_state (state); +dev_dbg (sd->dev, +"path_enabled: can't find opposite route for %s:%u/%u" , + sd->entity.name, sink_pad->index, sink_stream); +return 2 ; + } + + source_pad = &sd->entity.pads[route->source_pad]; +v4l2_subdev_unlock_state (state); + + tmp_pad = sink_pad; + sink_pad =media_pad_remote_pad_unique (source_pad); +if (IS_ERR (sink_pad)) { +dev_dbg (sd->dev, +"path_enabled: can't find remote source for %s:%u\n" , + source_pad->entity->name, source_pad->index); +return PTR_ERR (sink_pad); + } + +if (is_media_entity_v4l2_video_device (sink_pad->entity)) { +struct video_device *vdev; + + vdev =media_entity_to_video_device (sink_pad->entity); +if (!vdev) +return -ENXIO ; + +dev_dbg (vdev->dev_parent, +"path_enabled: found video device %s\n" , + vdev->name); + +if (!*__sink_pad) { + *__sink_pad = tmp_pad; +dev_dbg (sd->dev,"path_enabled: sink %u/%u\n" , + tmp_pad->index, sink_stream); + }else if (tmp_pad != *__sink_pad) { +dev_dbg (sd->dev, +"path_enabled: pads %s/%u and %s/%u differ\n" , + tmp_pad->entity->name, tmp_pad->index, + (*__sink_pad)->entity->name, + (*__sink_pad)->index); +return -EXDEV ; + } + + *__sink_streams |=BIT_ULL (sink_stream); + +return func (vdev); + } + +return v4l2_mc_downpath_enabled (sink_pad, source_stream, func, + __sink_pad, __sink_streams); +} + +static int v4l2_mc_source_get_streams (struct v4l2_subdev *sd,unsigned int pad, +u64 *__streams) +{ +struct v4l2_mbus_frame_desc desc; +u64 streams =0 ; +int ret; + +if (!__streams) +return -EINVAL ; + + ret =v4l2_subdev_call (sd, pad, get_frame_desc, pad, &desc); +if (ret == -ENOIOCTLCMD ) { + *__streams =1ULL ; +return 0 ; + } +if (ret) +return ret; + +for (unsigned int i =0 ; i < desc.num_entries; i++) { +if (streams &BIT_ULL (desc.entry[i].stream)) +return -EINVAL ; + + streams |=BIT_ULL (desc.entry[i].stream); + } + +dev_dbg (sd->dev,"found streams %#llx based on streams %#llx\n" , + *__streams, streams); +if (*__streams & ~streams) +return -EINVAL ; + + *__streams = streams; + +return 0 ; +} + +int v4l2_mc_pipeline_enabled (struct video_device *vdev, +bool (*func)(struct video_device *vdev), +struct media_pad **__sink_pad,u64 *__sink_streams) +{ +u64 sink_streams =1U ; +struct media_pad *src_pad; +u64 src_streams; +struct v4l2_subdev_state *state; +struct media_pad *sink_pad = vdev->entity.pads; +struct v4l2_subdev *sd =NULL ; +bool streaming = true; +struct media_pad *tmp_pad; +u64 tmp_streams; +int ret; + +if (!__sink_pad) + __sink_pad = &tmp_pad; +if (!__sink_streams) + __sink_streams = &tmp_streams; + *__sink_pad =NULL ; + *__sink_streams =0 ; + +do { + src_pad =media_pad_remote_pad_unique (sink_pad); +if (IS_ERR (src_pad)) { +dev_dbg (sd ? sd->dev : vdev->dev_parent, +"no unique remote pad found from %s:%u\n" , + sink_pad->entity->name, sink_pad->index); +return PTR_ERR (src_pad); + } + + sd =media_entity_to_v4l2_subdev (src_pad->entity); +if (!sd) { +dev_dbg (sd->dev, +"media entity %s is not a V4L2 sub-device\n" , + src_pad->entity->name); +return -ENXIO ; + } + +/* Source streams match sink. */ + src_streams = sink_streams; + + state =v4l2_subdev_lock_and_get_active_state (sd); + ret =__v4l2_mc_pipeline_enabled (state, src_pad, + src_streams, &sink_pad, + &sink_streams); +v4l2_subdev_unlock_state (state); +if (ret) +return ret; + }while (sink_pad); + + ret =v4l2_mc_source_get_streams (sd, src_pad->index, &src_streams); +if (ret) +return ret; + + sd =media_entity_to_v4l2_subdev (src_pad->entity); + +dev_dbg (sd->dev,"following %s:%u/%#llx\n" , sd->entity.name, + src_pad->index, src_streams); + +for (unsigned int i =__ffs (src_streams); src_streams; + src_streams &= ~BIT_ULL (i), i =__ffs (src_streams)) { + sink_pad =media_pad_remote_pad_unique (src_pad); +if (IS_ERR (src_pad)) { +dev_dbg (sd->dev, +"no unique remote pad found from %s:%u\n" , + sink_pad->entity->name, sink_pad->index); +return PTR_ERR (src_pad); + } + + ret =v4l2_mc_downpath_enabled (sink_pad, i, func, __sink_pad, + __sink_streams); +if (ret ==2 ) +continue ; +if (ret <0 ) +return ret; +if (!ret) + streaming = false; + } + +dev_dbg (media_entity_to_v4l2_subdev ((*__sink_pad)->entity)->dev, +"sink pad %s:%u/%#llx\n" , (*__sink_pad)->entity->name, + (*__sink_pad)->index, *__sink_streams); + +return streaming; +} +EXPORT_SYMBOL_GPL (v4l2_mc_pipeline_enabled);include/media/v4l2-mc.h +44
-0@@ -193,6 +193,50 @@ void v4l2_pipeline_pm_put(struct media_entity *entity);int v4l2_pipeline_link_notify (struct media_link *link,u32 flags,unsigned int notification); +/** +* v4l2_mc_pipeline_enabled - Tell when to start streaming +* @vdev: The video device +* @func: Caller-provided function to tell a video device's streaming state +* @__sink_pad: sink pad at the root of the local pipeline +* @__sink_streams: streams to start +* +* Use to tell whether streaming should start on a video node. @func returns +* true if streaming has been started on a given video node. @__sink_pad and +* @__sink_streams are filled with pad and streams on the sub-device closest to +* the video nodes, to be used for calling v4l2_subdev_enable_streams() and +* v4l2_subdev_disable_streams(). +* +* Using v4l2_mc_pipeline_enabled() has a few limitations currently (consider it +* a to-do list): +* * only unbranched streams can be supported albeit adding support for +* downstream branches would be fairly trivial, +* * streams within a single source sub-device are considered to start at the +* same time, more control could be added in two ways: 1) for sources to +* determine stream starting, a control could be added to UAPI and 2) sources +* could tell which streams start at the same time using a sub-device +* operation, +* * CSI-2 VC framing is ignored currently, but VC-based stream starting could +* be implemented by letting the caller to provide a function to determine +* which streams are of interest and +* * routes leading to nowhere are ignored, on some hardware this is a problem, +* but this can also be rather trivially addressed. +* +* Return: +* * 0: Success, but don't start streaming yet +* * 1: Success, now it's time to start streaming +* * -ENXIO: Route traversal encountered a non-video device/sub-device entity +* * -ENOTUNIQ: No unique remote pad +* * -ENOLINK: No remote pad found +* * -ENOENT: Enabled upstream route not found +* * -EMLINK: No unique downstream route found +* * -EINVAL: Stream could not be followed to source or was not produced by +* the source +*/ +int v4l2_mc_pipeline_enabled (struct video_device *vdev, +bool (*func)(struct video_device *vdev), +struct media_pad **__sink_pad, +u64 *__sink_streams); +#else /* CONFIG_MEDIA_CONTROLLER */ static inline int v4l2_mc_create_media_graph (struct media_device *mdev)
Signature · 3 lines
Kind regards, Sakari Ailus
Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()
Quoted 31 lines · Sakari Ailus
Quoted 21 lines · Jacopo Mondi
Quoted 4 lines · Sakari Ailus
v4l2_mc_pipeline_enabled()helps solving a problem known for long but lacked any sort of general solution: with multiple streams, when streaming is started on video nodes one by one, when should streaming be started in the source?I tried quite some time to understand this, but if I'm not mistaken, a stream-aware subdev, which links to vdev, will always "demux" streams to different pads and will connect to the vdev from there
Source
subdev+-----------------+ | (1/0) ------> vdev0 | | (0)[1,2,3] (2/0 ------> vdev1 | | | (3/0) ------> vdev2 +-----------------+Quoted 8 lines
With
(0) multiplexed sink pad with 3 streams
(1) (2) and (3) source pad with a single streamCan't we relay on the media-link state between the source pads and the video devices with something like what Dan has proposed here ?
https://patchwork.linuxtv.org/project/linux-media/patch/[email protected]/
Quoted 5 lines
This isn't entirely the same thing: pipeline is specific to the pad but not streams, ad here we're interested in streams. Two streams may start at different points of time even if both are part of the same pipeline.
We definitely should have just one way to figure this out.
But can we ? Isn't it dependent on use cases ? I can imagine an application wanting to capture two image streams where one of them is enabled all the time and the other one is regularly turned on and off, in which case you would need to start the pipeline when the first stream starts. A different application may instead want to capture the same two streams and make sure it gets all frames on both, which possibly requires delaying the start of capture until both video capture devices are started.
Quoted 328 lines
I'll send v2 with comments and before that also see if (or how) we could get rid of the callback.
Cc Dan.
Quoted 318 lines
What am I missing ?
Quoted 316 lines
v4l2_mc_pipeline_enabled()traverses the pipeline towards the source, queries the streams generated by the source and traces them back to the video nodes.Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/v4l2-core/v4l2-mc.c | 243 ++++++++++++++++++++++++++++++ include/media/v4l2-mc.h | 44 ++++++ 2 files changed, 287 insertions(+)drivers/media/v4l2-core/v4l2-mc.c +243
-0@@ -612,3 +612,246 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,return ret; }EXPORT_SYMBOL_GPL (v4l2_pipeline_link_notify); + +static int +__v4l2_mc_pipeline_enabled (struct v4l2_subdev_state *state, +struct media_pad *src_pad,u64 __src_streams, +struct media_pad **__sink_pad,u64 *__sink_streams) +{ +struct v4l2_subdev_route *route; +u64 src_streams =0 , sink_streams =0 ; +bool has_sink_pad = false; +unsigned int sink_pad; + +dev_dbg (state->sd->dev,"%s: source enabled, pad/streams %u/%#llx\n" , + state->sd->entity.name, src_pad->index, __src_streams); +for_each_active_route (&state->routing, route) { +dev_dbg (state->sd->dev,"%s: %u/%u -> %u/%u, flags %x\n" , + state->sd->entity.name, + route->sink_pad, route->sink_stream, route->source_pad, + route->source_stream, route->flags); +if (route->source_pad != src_pad->index) +continue ; + +if (!(BIT_ULL (route->source_stream) & __src_streams)) +continue ; + +if (!has_sink_pad) { + has_sink_pad = true; + sink_pad = route->sink_pad; + } + +if (route->sink_pad != sink_pad) { +dev_dbg (state->sd->dev, +"sink pads (%u vs. %u) differ\n" , + route->sink_pad, sink_pad); +return -EMLINK ; + } + + sink_streams |=BIT_ULL (route->sink_stream); + src_streams |=BIT_ULL (route->source_stream); + } + + *__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] :NULL ; + *__sink_streams = sink_streams; + +return 0 ; +} + +static int v4l2_mc_downpath_enabled (struct media_pad *sink_pad, +unsigned int sink_stream, +bool (*func)(struct video_device *vdev), +struct media_pad **__sink_pad, +u64 *__sink_streams) +{ +struct v4l2_subdev_state *state; +struct v4l2_subdev_route *route; +struct v4l2_subdev *sd; +struct media_pad *source_pad, *tmp_pad; +u32 source_stream; + +if (!is_media_entity_v4l2_subdev (sink_pad->entity)) +return -ENXIO ; + + sd =media_entity_to_v4l2_subdev (sink_pad->entity); +dev_dbg (sd->dev,"path_enabled: found sub-device %s\n" , + sd->entity.name); + + state =v4l2_subdev_lock_and_get_active_state (sd); + route =v4l2_subdev_find_route (&state->routing, sink_pad->index, + sink_stream, true,0 ); +if (IS_ERR (route)) { +v4l2_subdev_unlock_state (state); +dev_dbg (sd->dev, +"path_enabled: can't find opposite route for %s:%u/%u" , + sd->entity.name, sink_pad->index, sink_stream); +return 2 ; + } + + source_pad = &sd->entity.pads[route->source_pad]; +v4l2_subdev_unlock_state (state); + + tmp_pad = sink_pad; + sink_pad =media_pad_remote_pad_unique (source_pad); +if (IS_ERR (sink_pad)) { +dev_dbg (sd->dev, +"path_enabled: can't find remote source for %s:%u\n" , + source_pad->entity->name, source_pad->index); +return PTR_ERR (sink_pad); + } + +if (is_media_entity_v4l2_video_device (sink_pad->entity)) { +struct video_device *vdev; + + vdev =media_entity_to_video_device (sink_pad->entity); +if (!vdev) +return -ENXIO ; + +dev_dbg (vdev->dev_parent, +"path_enabled: found video device %s\n" , + vdev->name); + +if (!*__sink_pad) { + *__sink_pad = tmp_pad; +dev_dbg (sd->dev,"path_enabled: sink %u/%u\n" , + tmp_pad->index, sink_stream); + }else if (tmp_pad != *__sink_pad) { +dev_dbg (sd->dev, +"path_enabled: pads %s/%u and %s/%u differ\n" , + tmp_pad->entity->name, tmp_pad->index, + (*__sink_pad)->entity->name, + (*__sink_pad)->index); +return -EXDEV ; + } + + *__sink_streams |=BIT_ULL (sink_stream); + +return func (vdev); + } + +return v4l2_mc_downpath_enabled (sink_pad, source_stream, func, + __sink_pad, __sink_streams); +} + +static int v4l2_mc_source_get_streams (struct v4l2_subdev *sd,unsigned int pad, +u64 *__streams) +{ +struct v4l2_mbus_frame_desc desc; +u64 streams =0 ; +int ret; + +if (!__streams) +return -EINVAL ; + + ret =v4l2_subdev_call (sd, pad, get_frame_desc, pad, &desc); +if (ret == -ENOIOCTLCMD ) { + *__streams =1ULL ; +return 0 ; + } +if (ret) +return ret; + +for (unsigned int i =0 ; i < desc.num_entries; i++) { +if (streams &BIT_ULL (desc.entry[i].stream)) +return -EINVAL ; + + streams |=BIT_ULL (desc.entry[i].stream); + } + +dev_dbg (sd->dev,"found streams %#llx based on streams %#llx\n" , + *__streams, streams); +if (*__streams & ~streams) +return -EINVAL ; + + *__streams = streams; + +return 0 ; +} + +int v4l2_mc_pipeline_enabled (struct video_device *vdev, +bool (*func)(struct video_device *vdev), +struct media_pad **__sink_pad,u64 *__sink_streams) +{ +u64 sink_streams =1U ; +struct media_pad *src_pad; +u64 src_streams; +struct v4l2_subdev_state *state; +struct media_pad *sink_pad = vdev->entity.pads; +struct v4l2_subdev *sd =NULL ; +bool streaming = true; +struct media_pad *tmp_pad; +u64 tmp_streams; +int ret; + +if (!__sink_pad) + __sink_pad = &tmp_pad; +if (!__sink_streams) + __sink_streams = &tmp_streams; + *__sink_pad =NULL ; + *__sink_streams =0 ; + +do { + src_pad =media_pad_remote_pad_unique (sink_pad); +if (IS_ERR (src_pad)) { +dev_dbg (sd ? sd->dev : vdev->dev_parent, +"no unique remote pad found from %s:%u\n" , + sink_pad->entity->name, sink_pad->index); +return PTR_ERR (src_pad); + } + + sd =media_entity_to_v4l2_subdev (src_pad->entity); +if (!sd) { +dev_dbg (sd->dev, +"media entity %s is not a V4L2 sub-device\n" , + src_pad->entity->name); +return -ENXIO ; + } + +/* Source streams match sink. */ + src_streams = sink_streams; + + state =v4l2_subdev_lock_and_get_active_state (sd); + ret =__v4l2_mc_pipeline_enabled (state, src_pad, + src_streams, &sink_pad, + &sink_streams); +v4l2_subdev_unlock_state (state); +if (ret) +return ret; + }while (sink_pad); + + ret =v4l2_mc_source_get_streams (sd, src_pad->index, &src_streams); +if (ret) +return ret; + + sd =media_entity_to_v4l2_subdev (src_pad->entity); + +dev_dbg (sd->dev,"following %s:%u/%#llx\n" , sd->entity.name, + src_pad->index, src_streams); + +for (unsigned int i =__ffs (src_streams); src_streams; + src_streams &= ~BIT_ULL (i), i =__ffs (src_streams)) { + sink_pad =media_pad_remote_pad_unique (src_pad); +if (IS_ERR (src_pad)) { +dev_dbg (sd->dev, +"no unique remote pad found from %s:%u\n" , + sink_pad->entity->name, sink_pad->index); +return PTR_ERR (src_pad); + } + + ret =v4l2_mc_downpath_enabled (sink_pad, i, func, __sink_pad, + __sink_streams); +if (ret ==2 ) +continue ; +if (ret <0 ) +return ret; +if (!ret) + streaming = false; + } + +dev_dbg (media_entity_to_v4l2_subdev ((*__sink_pad)->entity)->dev, +"sink pad %s:%u/%#llx\n" , (*__sink_pad)->entity->name, + (*__sink_pad)->index, *__sink_streams); + +return streaming; +} +EXPORT_SYMBOL_GPL (v4l2_mc_pipeline_enabled);include/media/v4l2-mc.h +44
-0@@ -193,6 +193,50 @@ void v4l2_pipeline_pm_put(struct media_entity *entity);int v4l2_pipeline_link_notify (struct media_link *link,u32 flags,unsigned int notification); +/** +* v4l2_mc_pipeline_enabled - Tell when to start streaming +* @vdev: The video device +* @func: Caller-provided function to tell a video device's streaming state +* @__sink_pad: sink pad at the root of the local pipeline +* @__sink_streams: streams to start +* +* Use to tell whether streaming should start on a video node. @func returns +* true if streaming has been started on a given video node. @__sink_pad and +* @__sink_streams are filled with pad and streams on the sub-device closest to +* the video nodes, to be used for calling v4l2_subdev_enable_streams() and +* v4l2_subdev_disable_streams(). +* +* Using v4l2_mc_pipeline_enabled() has a few limitations currently (consider it +* a to-do list): +* * only unbranched streams can be supported albeit adding support for +* downstream branches would be fairly trivial, +* * streams within a single source sub-device are considered to start at the +* same time, more control could be added in two ways: 1) for sources to +* determine stream starting, a control could be added to UAPI and 2) sources +* could tell which streams start at the same time using a sub-device +* operation, +* * CSI-2 VC framing is ignored currently, but VC-based stream starting could +* be implemented by letting the caller to provide a function to determine +* which streams are of interest and +* * routes leading to nowhere are ignored, on some hardware this is a problem, +* but this can also be rather trivially addressed. +* +* Return: +* * 0: Success, but don't start streaming yet +* * 1: Success, now it's time to start streaming +* * -ENXIO: Route traversal encountered a non-video device/sub-device entity +* * -ENOTUNIQ: No unique remote pad +* * -ENOLINK: No remote pad found +* * -ENOENT: Enabled upstream route not found +* * -EMLINK: No unique downstream route found +* * -EINVAL: Stream could not be followed to source or was not produced by +* the source +*/ +int v4l2_mc_pipeline_enabled (struct video_device *vdev, +bool (*func)(struct video_device *vdev), +struct media_pad **__sink_pad, +u64 *__sink_streams); +#else /* CONFIG_MEDIA_CONTROLLER */ static inline int v4l2_mc_create_media_graph (struct media_device *mdev)Signature · 3 lines
Kind regards, Sakari Ailus
Signature · 3 lines
Regards, Laurent Pinchart
Sakari Ailus
maintainerRe: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()
Hi Laurent,
Quoted 38 lines · Laurent Pinchart
Quoted 31 lines · Sakari Ailus
Quoted 21 lines · Jacopo Mondi
Quoted 4 lines · Sakari Ailus
v4l2_mc_pipeline_enabled()helps solving a problem known for long but lacked any sort of general solution: with multiple streams, when streaming is started on video nodes one by one, when should streaming be started in the source?I tried quite some time to understand this, but if I'm not mistaken, a stream-aware subdev, which links to vdev, will always "demux" streams to different pads and will connect to the vdev from there
Source
subdev+-----------------+ | (1/0) ------> vdev0 | | (0)[1,2,3] (2/0 ------> vdev1 | | | (3/0) ------> vdev2 +-----------------+Quoted 8 lines
With
(0) multiplexed sink pad with 3 streams
(1) (2) and (3) source pad with a single streamCan't we relay on the media-link state between the source pads and the video devices with something like what Dan has proposed here ?
https://patchwork.linuxtv.org/project/linux-media/patch/[email protected]/Quoted 5 lines
This isn't entirely the same thing: pipeline is specific to the pad but not streams, ad here we're interested in streams. Two streams may start at different points of time even if both are part of the same pipeline.
We definitely should have just one way to figure this out.
Quoted 8 lines
But can we ? Isn't it dependent on use cases ? I can imagine an application wanting to capture two image streams where one of them is enabled all the time and the other one is regularly turned on and off, in which case you would need to start the pipeline when the first stream starts. A different application may instead want to capture the same two streams and make sure it gets all frames on both, which possibly requires delaying the start of capture until both video capture devices are started.
Exactly. It's indeed dependent on the use case but the driver is the same in both cases. This is why the generic solution needs more information from the user space but I think we can work out how to support this after merging the metadata series.
Signature · 3 lines
Regards, Sakari Ailus
Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()
Quoted 48 lines · Sakari Ailus
Quoted 38 lines · Laurent Pinchart
Quoted 31 lines · Sakari Ailus
Quoted 21 lines · Jacopo Mondi
Quoted 4 lines · Sakari Ailus
v4l2_mc_pipeline_enabled()helps solving a problem known for long but lacked any sort of general solution: with multiple streams, when streaming is started on video nodes one by one, when should streaming be started in the source?I tried quite some time to understand this, but if I'm not mistaken, a stream-aware subdev, which links to vdev, will always "demux" streams to different pads and will connect to the vdev from there
Source
subdev+-----------------+ | (1/0) ------> vdev0 | | (0)[1,2,3] (2/0 ------> vdev1 | | | (3/0) ------> vdev2 +-----------------+Quoted 8 lines
With
(0) multiplexed sink pad with 3 streams
(1) (2) and (3) source pad with a single streamCan't we relay on the media-link state between the source pads and the video devices with something like what Dan has proposed here ?
https://patchwork.linuxtv.org/project/linux-media/patch/[email protected]/Quoted 5 lines
This isn't entirely the same thing: pipeline is specific to the pad but not streams, ad here we're interested in streams. Two streams may start at different points of time even if both are part of the same pipeline.
We definitely should have just one way to figure this out.
Quoted 8 lines
But can we ? Isn't it dependent on use cases ? I can imagine an application wanting to capture two image streams where one of them is enabled all the time and the other one is regularly turned on and off, in which case you would need to start the pipeline when the first stream starts. A different application may instead want to capture the same two streams and make sure it gets all frames on both, which possibly requires delaying the start of capture until both video capture devices are started.
Quoted 4 lines
Exactly. It's indeed dependent on the use case but the driver is the same in both cases. This is why the generic solution needs more information from the user space but I think we can work out how to support this after merging the metadata series.
Yes, we need more information from userspace.
It's also a driver-dependent behaviour. If the streams come from multiple cameras that all go through the same deserializer (e.g. a quad FPD-Link or GMSL deserializer), then the decision on when to start streaming on some of the components in the pipeline could depend on which deserializer is used. Some will require all cameras to be started together, while other could allow starting and stopping sources independently. The nature of upstream elements could influence when downstream elements will need to be started, for the same userspace-facing behaviour.
Signature · 3 lines
Regards, Laurent Pinchart
kernel test robot
build botRe: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()
In reply to Sakari Ailuskernel test robot noticed the following build errors:
Hi Sakari,
kernel test robot noticed the following build errors:
[auto build test ERROR on linuxtv-media-pending/master]
[also build test ERROR on linus/master media-tree/master v6.16-rc3 next-20250623] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Sakari-Ailus/media-ipu6-Use-correct-pads-for-xlate_streams/20250619-161847 base: https://git.linuxtv.org/media-ci/media-pending.git master patch link: https://lore.kernel.org/r/20250619081546.1582969-13-sakari.ailus%40linux.intel.com patch subject: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled() config: arm-imx_v4_v5_defconfig (https://download.01.org/0day-ci/archive/20250623/[email protected]/config) compiler: clang version 21.0.0git (https://github.com/llvm/llvm-project 875b36a8742437b95f623bab1e0332562c7b4b3f) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250623/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
>> drivers/media/v4l2-core/v4l2-mc.c:628:2: error: call to undeclared function 'for_each_active_route'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
628 | for_each_active_route(&state->routing, route) {
| ^
drivers/media/v4l2-core/v4l2-mc.c:628:47: error: expected ';' after expression
628 | for_each_active_route(&state->routing, route) {
| ^
| ;
drivers/media/v4l2-core/v4l2-mc.c:634:4: error: 'continue' statement not in loop statement
634 | continue;
| ^
drivers/media/v4l2-core/v4l2-mc.c:637:4: error: 'continue' statement not in loop statement
637 | continue;
| ^
>> drivers/media/v4l2-core/v4l2-mc.c:681:10: error: call to undeclared function 'v4l2_subdev_find_route'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
681 | route = v4l2_subdev_find_route(&state->routing, sink_pad->index,
| ^
>> drivers/media/v4l2-core/v4l2-mc.c:681:8: error: incompatible integer to pointer conversion assigning to 'struct v4l2_subdev_route *' from 'int' [-Wint-conversion]
681 | route = v4l2_subdev_find_route(&state->routing, sink_pad->index,
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
682 | sink_stream, true, 0);
| ~~~~~~~~~~~~~~~~~~~~~6 errors generated.
vim +/for_each_active_route +628 drivers/media/v4l2-core/v4l2-mc.c
615
616 static int
617 __v4l2_mc_pipeline_enabled(struct v4l2_subdev_state *state,
618 struct media_pad *src_pad, u64 __src_streams,
619 struct media_pad **__sink_pad, u64 *__sink_streams)
620 {
621 struct v4l2_subdev_route *route;
622 u64 src_streams = 0, sink_streams = 0;
623 bool has_sink_pad = false;
624 unsigned int sink_pad;
625
626 dev_dbg(state->sd->dev, "%s: source enabled, pad/streams %u/%#llx\n",
627 state->sd->entity.name, src_pad->index, __src_streams);
> 628 for_each_active_route(&state->routing, route) {
629 dev_dbg(state->sd->dev, "%s: %u/%u -> %u/%u, flags %x\n",
630 state->sd->entity.name,
631 route->sink_pad, route->sink_stream, route->source_pad,
632 route->source_stream, route->flags);
633 if (route->source_pad != src_pad->index)
634 continue;
635
636 if (!(BIT_ULL(route->source_stream) & __src_streams))
637 continue;
638
639 if (!has_sink_pad) {
640 has_sink_pad = true;
641 sink_pad = route->sink_pad;
642 }
643
644 if (route->sink_pad != sink_pad) {
645 dev_dbg(state->sd->dev,
646 "sink pads (%u vs. %u) differ\n",
647 route->sink_pad, sink_pad);
648 return -EMLINK;
649 }
650
651 sink_streams |= BIT_ULL(route->sink_stream);
652 src_streams |= BIT_ULL(route->source_stream);
653 }
654
655 *__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] : NULL;
656 *__sink_streams = sink_streams;
657
658 return 0;
659 }
660
661 static int v4l2_mc_downpath_enabled(struct media_pad *sink_pad,
662 unsigned int sink_stream,
663 bool (*func)(struct video_device *vdev),
664 struct media_pad **__sink_pad,
665 u64 *__sink_streams)
666 {
667 struct v4l2_subdev_state *state;
668 struct v4l2_subdev_route *route;
669 struct v4l2_subdev *sd;
670 struct media_pad *source_pad, *tmp_pad;
671 u32 source_stream;
672
673 if (!is_media_entity_v4l2_subdev(sink_pad->entity))
674 return -ENXIO;
675
676 sd = media_entity_to_v4l2_subdev(sink_pad->entity);
677 dev_dbg(sd->dev, "path_enabled: found sub-device %s\n",
678 sd->entity.name);
679
680 state = v4l2_subdev_lock_and_get_active_state(sd);
> 681 route = v4l2_subdev_find_route(&state->routing, sink_pad->index, 682 sink_stream, true, 0);
683 if (IS_ERR(route)) {
684 v4l2_subdev_unlock_state(state);
685 dev_dbg(sd->dev,
686 "path_enabled: can't find opposite route for %s:%u/%u",
687 sd->entity.name, sink_pad->index, sink_stream);
688 return 2;
689 }
690
691 source_pad = &sd->entity.pads[route->source_pad];
692 v4l2_subdev_unlock_state(state);
693
694 tmp_pad = sink_pad;
695 sink_pad = media_pad_remote_pad_unique(source_pad);
696 if (IS_ERR(sink_pad)) {
697 dev_dbg(sd->dev,
698 "path_enabled: can't find remote source for %s:%u\n",
699 source_pad->entity->name, source_pad->index);
700 return PTR_ERR(sink_pad);
701 }
702
703 if (is_media_entity_v4l2_video_device(sink_pad->entity)) {
704 struct video_device *vdev;
705
706 vdev = media_entity_to_video_device(sink_pad->entity);
707 if (!vdev)
708 return -ENXIO;
709
710 dev_dbg(vdev->dev_parent,
711 "path_enabled: found video device %s\n",
712 vdev->name);
713
714 if (!*__sink_pad) {
715 *__sink_pad = tmp_pad;
716 dev_dbg(sd->dev, "path_enabled: sink %u/%u\n",
717 tmp_pad->index, sink_stream);
718 } else if (tmp_pad != *__sink_pad) {
719 dev_dbg(sd->dev,
720 "path_enabled: pads %s/%u and %s/%u differ\n",
721 tmp_pad->entity->name, tmp_pad->index,
722 (*__sink_pad)->entity->name,
723 (*__sink_pad)->index);
724 return -EXDEV;
725 }
726
727 *__sink_streams |= BIT_ULL(sink_stream);
728
729 return func(vdev);
730 }
731
732 return v4l2_mc_downpath_enabled(sink_pad, source_stream, func,
733 __sink_pad, __sink_streams);
734 }
735
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()
In reply to Sakari AilusHi Sakari,
Thank you for the patch.
Quoted 271 lines · Sakari Ailus
v4l2_mc_pipeline_enabled()helps solving a problem known for long but lacked any sort of general solution: with multiple streams, when streaming is started on video nodes one by one, when should streaming be started in the source?
v4l2_mc_pipeline_enabled()traverses the pipeline towards the source, queries the streams generated by the source and traces them back to the video nodes.Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/v4l2-core/v4l2-mc.c | 243 ++++++++++++++++++++++++++++++ include/media/v4l2-mc.h | 44 ++++++ 2 files changed, 287 insertions(+)drivers/media/v4l2-core/v4l2-mc.c +243
-0@@ -612,3 +612,246 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,return ret; }EXPORT_SYMBOL_GPL (v4l2_pipeline_link_notify); + +static int +__v4l2_mc_pipeline_enabled (struct v4l2_subdev_state *state, +struct media_pad *src_pad,u64 __src_streams, +struct media_pad **__sink_pad,u64 *__sink_streams) +{ +struct v4l2_subdev_route *route; +u64 src_streams =0 , sink_streams =0 ; +bool has_sink_pad = false; +unsigned int sink_pad; + +dev_dbg (state->sd->dev,"%s: source enabled, pad/streams %u/%#llx\n" , + state->sd->entity.name, src_pad->index, __src_streams); +for_each_active_route (&state->routing, route) { +dev_dbg (state->sd->dev,"%s: %u/%u -> %u/%u, flags %x\n" , + state->sd->entity.name, + route->sink_pad, route->sink_stream, route->source_pad, + route->source_stream, route->flags); +if (route->source_pad != src_pad->index) +continue ; + +if (!(BIT_ULL (route->source_stream) & __src_streams)) +continue ; + +if (!has_sink_pad) { + has_sink_pad = true; + sink_pad = route->sink_pad; + } + +if (route->sink_pad != sink_pad) { +dev_dbg (state->sd->dev, +"sink pads (%u vs. %u) differ\n" , + route->sink_pad, sink_pad); +return -EMLINK ; + } + + sink_streams |=BIT_ULL (route->sink_stream); + src_streams |=BIT_ULL (route->source_stream); + } + + *__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] :NULL ; + *__sink_streams = sink_streams; + +return 0 ; +} + +static int v4l2_mc_downpath_enabled (struct media_pad *sink_pad, +unsigned int sink_stream, +bool (*func)(struct video_device *vdev), +struct media_pad **__sink_pad, +u64 *__sink_streams) +{ +struct v4l2_subdev_state *state; +struct v4l2_subdev_route *route; +struct v4l2_subdev *sd; +struct media_pad *source_pad, *tmp_pad; +u32 source_stream; + +if (!is_media_entity_v4l2_subdev (sink_pad->entity)) +return -ENXIO ; + + sd =media_entity_to_v4l2_subdev (sink_pad->entity); +dev_dbg (sd->dev,"path_enabled: found sub-device %s\n" , + sd->entity.name); + + state =v4l2_subdev_lock_and_get_active_state (sd); + route =v4l2_subdev_find_route (&state->routing, sink_pad->index, + sink_stream, true,0 ); +if (IS_ERR (route)) { +v4l2_subdev_unlock_state (state); +dev_dbg (sd->dev, +"path_enabled: can't find opposite route for %s:%u/%u" , + sd->entity.name, sink_pad->index, sink_stream); +return 2 ; + } + + source_pad = &sd->entity.pads[route->source_pad]; +v4l2_subdev_unlock_state (state); + + tmp_pad = sink_pad; + sink_pad =media_pad_remote_pad_unique (source_pad); +if (IS_ERR (sink_pad)) { +dev_dbg (sd->dev, +"path_enabled: can't find remote source for %s:%u\n" , + source_pad->entity->name, source_pad->index); +return PTR_ERR (sink_pad); + } + +if (is_media_entity_v4l2_video_device (sink_pad->entity)) { +struct video_device *vdev; + + vdev =media_entity_to_video_device (sink_pad->entity); +if (!vdev) +return -ENXIO ; + +dev_dbg (vdev->dev_parent, +"path_enabled: found video device %s\n" , + vdev->name); + +if (!*__sink_pad) { + *__sink_pad = tmp_pad; +dev_dbg (sd->dev,"path_enabled: sink %u/%u\n" , + tmp_pad->index, sink_stream); + }else if (tmp_pad != *__sink_pad) { +dev_dbg (sd->dev, +"path_enabled: pads %s/%u and %s/%u differ\n" , + tmp_pad->entity->name, tmp_pad->index, + (*__sink_pad)->entity->name, + (*__sink_pad)->index); +return -EXDEV ; + } + + *__sink_streams |=BIT_ULL (sink_stream); + +return func (vdev); + } + +return v4l2_mc_downpath_enabled (sink_pad, source_stream, func, + __sink_pad, __sink_streams); +} + +static int v4l2_mc_source_get_streams (struct v4l2_subdev *sd,unsigned int pad, +u64 *__streams) +{ +struct v4l2_mbus_frame_desc desc; +u64 streams =0 ; +int ret; + +if (!__streams) +return -EINVAL ; + + ret =v4l2_subdev_call (sd, pad, get_frame_desc, pad, &desc); +if (ret == -ENOIOCTLCMD ) { + *__streams =1ULL ; +return 0 ; + } +if (ret) +return ret; + +for (unsigned int i =0 ; i < desc.num_entries; i++) { +if (streams &BIT_ULL (desc.entry[i].stream)) +return -EINVAL ; + + streams |=BIT_ULL (desc.entry[i].stream); + } + +dev_dbg (sd->dev,"found streams %#llx based on streams %#llx\n" , + *__streams, streams); +if (*__streams & ~streams) +return -EINVAL ; + + *__streams = streams; + +return 0 ; +} + +int v4l2_mc_pipeline_enabled (struct video_device *vdev, +bool (*func)(struct video_device *vdev), +struct media_pad **__sink_pad,u64 *__sink_streams) +{ +u64 sink_streams =1U ; +struct media_pad *src_pad; +u64 src_streams; +struct v4l2_subdev_state *state; +struct media_pad *sink_pad = vdev->entity.pads; +struct v4l2_subdev *sd =NULL ; +bool streaming = true; +struct media_pad *tmp_pad; +u64 tmp_streams; +int ret; + +if (!__sink_pad) + __sink_pad = &tmp_pad; +if (!__sink_streams) + __sink_streams = &tmp_streams; + *__sink_pad =NULL ; + *__sink_streams =0 ; + +do { + src_pad =media_pad_remote_pad_unique (sink_pad); +if (IS_ERR (src_pad)) { +dev_dbg (sd ? sd->dev : vdev->dev_parent, +"no unique remote pad found from %s:%u\n" , + sink_pad->entity->name, sink_pad->index); +return PTR_ERR (src_pad); + } + + sd =media_entity_to_v4l2_subdev (src_pad->entity); +if (!sd) { +dev_dbg (sd->dev, +"media entity %s is not a V4L2 sub-device\n" , + src_pad->entity->name); +return -ENXIO ; + } + +/* Source streams match sink. */ + src_streams = sink_streams; + + state =v4l2_subdev_lock_and_get_active_state (sd); + ret =__v4l2_mc_pipeline_enabled (state, src_pad, + src_streams, &sink_pad, + &sink_streams); +v4l2_subdev_unlock_state (state); +if (ret) +return ret; + }while (sink_pad); + + ret =v4l2_mc_source_get_streams (sd, src_pad->index, &src_streams); +if (ret) +return ret; + + sd =media_entity_to_v4l2_subdev (src_pad->entity); + +dev_dbg (sd->dev,"following %s:%u/%#llx\n" , sd->entity.name, + src_pad->index, src_streams); + +for (unsigned int i =__ffs (src_streams); src_streams; + src_streams &= ~BIT_ULL (i), i =__ffs (src_streams)) { + sink_pad =media_pad_remote_pad_unique (src_pad); +if (IS_ERR (src_pad)) { +dev_dbg (sd->dev, +"no unique remote pad found from %s:%u\n" , + sink_pad->entity->name, sink_pad->index); +return PTR_ERR (src_pad); + } + + ret =v4l2_mc_downpath_enabled (sink_pad, i, func, __sink_pad, + __sink_streams); +if (ret ==2 ) +continue ; +if (ret <0 ) +return ret; +if (!ret) + streaming = false; + } + +dev_dbg (media_entity_to_v4l2_subdev ((*__sink_pad)->entity)->dev, +"sink pad %s:%u/%#llx\n" , (*__sink_pad)->entity->name, + (*__sink_pad)->index, *__sink_streams); + +return streaming; +} +EXPORT_SYMBOL_GPL (v4l2_mc_pipeline_enabled);include/media/v4l2-mc.h +0
-0
include/media/v4l2-mc.h void v4l2_pipeline_pm_put(struct media_entity *entity); · line 201
Quoted 9 lines
include/media/v4l2-mc.h +6
-0@@ -193,6 +193,50 @@ void v4l2_pipeline_pm_put(struct media_entity *entity);int v4l2_pipeline_link_notify (struct media_link *link,u32 flags,unsigned int notification); +/** +* v4l2_mc_pipeline_enabled - Tell when to start streaming +* @vdev: The video device +* @func: Caller-provided function to tell a video device's streaming state +* @__sink_pad: sink pad at the root of the local pipeline +* @__sink_streams: streams to start
Any reason for the double underscore ?
include/media/v4l2-mc.h void v4l2_pipeline_pm_put(struct media_entity *entity); · line 212
Quoted 11 lines
include/media/v4l2-mc.h +11
-0+* +* Use to tell whether streaming should start on a video node. @func returns +* true if streaming has been started on a given video node. @__sink_pad and +* @__sink_streams are filled with pad and streams on the sub-device closest to +* the video nodes, to be used for calling v4l2_subdev_enable_streams() and +* v4l2_subdev_disable_streams(). +* +* Using v4l2_mc_pipeline_enabled() has a few limitations currently (consider it +* a to-do list): +* * only unbranched streams can be supported albeit adding support for +* downstream branches would be fairly trivial,
I can't tell from the documentation here what you mean exactly by "unbranched streams".
include/media/v4l2-mc.h void v4l2_pipeline_pm_put(struct media_entity *entity); · line 222
Quoted 10 lines
include/media/v4l2-mc.h +10
-0+* * streams within a single source sub-device are considered to start at the +* same time, more control could be added in two ways: 1) for sources to +* determine stream starting, a control could be added to UAPI and 2) sources +* could tell which streams start at the same time using a sub-device +* operation, +* * CSI-2 VC framing is ignored currently, but VC-based stream starting could +* be implemented by letting the caller to provide a function to determine +* which streams are of interest and +* * routes leading to nowhere are ignored, on some hardware this is a problem, +* but this can also be rather trivially addressed.
I'm afraid this function looks like a hack, to solve a problem that is not even explicitly described. You don't explain the issue in the cover letter or in Documentation/, the cover letter merely states that this is a "partial solution". The documentation of the function doesn't explain what criteria the decision is based on. We need a proper explanation of the problem in Documentation/, with a description of the behaviour (or behaviours) drivers are expected to implement.
Furthermore, on the implementation side, things are fairly inefficient. We already traverse the whole pipeline in media_pipeline_start(), based on links and routes, and populate the media_pipeline structure. We shouldn't do the same here, but instead inspect media_pipeline to extract the information we need. If you're missing information there, let's add it.
Quoted 20 lines
include/media/v4l2-mc.h +17
-0+* +* Return: +* * 0: Success, but don't start streaming yet +* * 1: Success, now it's time to start streaming +* * -ENXIO: Route traversal encountered a non-video device/sub-device entity +* * -ENOTUNIQ: No unique remote pad +* * -ENOLINK: No remote pad found +* * -ENOENT: Enabled upstream route not found +* * -EMLINK: No unique downstream route found +* * -EINVAL: Stream could not be followed to source or was not produced by +* the source +*/ +int v4l2_mc_pipeline_enabled (struct video_device *vdev, +bool (*func)(struct video_device *vdev), +struct media_pad **__sink_pad, +u64 *__sink_streams); +#else /* CONFIG_MEDIA_CONTROLLER */ static inline int v4l2_mc_create_media_graph (struct media_device *mdev)
Signature · 3 lines
Regards, Laurent Pinchart
Sakari Ailus
maintainerRe: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()
Hi Laurent,
Thanks for the review.
Quoted 279 lines · Laurent Pinchart
Hi Sakari,
Thank you for the patch.
Quoted 274 lines · Sakari Ailus
v4l2_mc_pipeline_enabled()helps solving a problem known for long but lacked any sort of general solution: with multiple streams, when streaming is started on video nodes one by one, when should streaming be started in the source?
v4l2_mc_pipeline_enabled()traverses the pipeline towards the source, queries the streams generated by the source and traces them back to the video nodes.Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/v4l2-core/v4l2-mc.c | 243 ++++++++++++++++++++++++++++++ include/media/v4l2-mc.h | 44 ++++++ 2 files changed, 287 insertions(+)drivers/media/v4l2-core/v4l2-mc.c +243
-0@@ -612,3 +612,246 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,return ret; }EXPORT_SYMBOL_GPL (v4l2_pipeline_link_notify); + +static int +__v4l2_mc_pipeline_enabled (struct v4l2_subdev_state *state, +struct media_pad *src_pad,u64 __src_streams, +struct media_pad **__sink_pad,u64 *__sink_streams) +{ +struct v4l2_subdev_route *route; +u64 src_streams =0 , sink_streams =0 ; +bool has_sink_pad = false; +unsigned int sink_pad; + +dev_dbg (state->sd->dev,"%s: source enabled, pad/streams %u/%#llx\n" , + state->sd->entity.name, src_pad->index, __src_streams); +for_each_active_route (&state->routing, route) { +dev_dbg (state->sd->dev,"%s: %u/%u -> %u/%u, flags %x\n" , + state->sd->entity.name, + route->sink_pad, route->sink_stream, route->source_pad, + route->source_stream, route->flags); +if (route->source_pad != src_pad->index) +continue ; + +if (!(BIT_ULL (route->source_stream) & __src_streams)) +continue ; + +if (!has_sink_pad) { + has_sink_pad = true; + sink_pad = route->sink_pad; + } + +if (route->sink_pad != sink_pad) { +dev_dbg (state->sd->dev, +"sink pads (%u vs. %u) differ\n" , + route->sink_pad, sink_pad); +return -EMLINK ; + } + + sink_streams |=BIT_ULL (route->sink_stream); + src_streams |=BIT_ULL (route->source_stream); + } + + *__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] :NULL ; + *__sink_streams = sink_streams; + +return 0 ; +} + +static int v4l2_mc_downpath_enabled (struct media_pad *sink_pad, +unsigned int sink_stream, +bool (*func)(struct video_device *vdev), +struct media_pad **__sink_pad, +u64 *__sink_streams) +{ +struct v4l2_subdev_state *state; +struct v4l2_subdev_route *route; +struct v4l2_subdev *sd; +struct media_pad *source_pad, *tmp_pad; +u32 source_stream; + +if (!is_media_entity_v4l2_subdev (sink_pad->entity)) +return -ENXIO ; + + sd =media_entity_to_v4l2_subdev (sink_pad->entity); +dev_dbg (sd->dev,"path_enabled: found sub-device %s\n" , + sd->entity.name); + + state =v4l2_subdev_lock_and_get_active_state (sd); + route =v4l2_subdev_find_route (&state->routing, sink_pad->index, + sink_stream, true,0 ); +if (IS_ERR (route)) { +v4l2_subdev_unlock_state (state); +dev_dbg (sd->dev, +"path_enabled: can't find opposite route for %s:%u/%u" , + sd->entity.name, sink_pad->index, sink_stream); +return 2 ; + } + + source_pad = &sd->entity.pads[route->source_pad]; +v4l2_subdev_unlock_state (state); + + tmp_pad = sink_pad; + sink_pad =media_pad_remote_pad_unique (source_pad); +if (IS_ERR (sink_pad)) { +dev_dbg (sd->dev, +"path_enabled: can't find remote source for %s:%u\n" , + source_pad->entity->name, source_pad->index); +return PTR_ERR (sink_pad); + } + +if (is_media_entity_v4l2_video_device (sink_pad->entity)) { +struct video_device *vdev; + + vdev =media_entity_to_video_device (sink_pad->entity); +if (!vdev) +return -ENXIO ; + +dev_dbg (vdev->dev_parent, +"path_enabled: found video device %s\n" , + vdev->name); + +if (!*__sink_pad) { + *__sink_pad = tmp_pad; +dev_dbg (sd->dev,"path_enabled: sink %u/%u\n" , + tmp_pad->index, sink_stream); + }else if (tmp_pad != *__sink_pad) { +dev_dbg (sd->dev, +"path_enabled: pads %s/%u and %s/%u differ\n" , + tmp_pad->entity->name, tmp_pad->index, + (*__sink_pad)->entity->name, + (*__sink_pad)->index); +return -EXDEV ; + } + + *__sink_streams |=BIT_ULL (sink_stream); + +return func (vdev); + } + +return v4l2_mc_downpath_enabled (sink_pad, source_stream, func, + __sink_pad, __sink_streams); +} + +static int v4l2_mc_source_get_streams (struct v4l2_subdev *sd,unsigned int pad, +u64 *__streams) +{ +struct v4l2_mbus_frame_desc desc; +u64 streams =0 ; +int ret; + +if (!__streams) +return -EINVAL ; + + ret =v4l2_subdev_call (sd, pad, get_frame_desc, pad, &desc); +if (ret == -ENOIOCTLCMD ) { + *__streams =1ULL ; +return 0 ; + } +if (ret) +return ret; + +for (unsigned int i =0 ; i < desc.num_entries; i++) { +if (streams &BIT_ULL (desc.entry[i].stream)) +return -EINVAL ; + + streams |=BIT_ULL (desc.entry[i].stream); + } + +dev_dbg (sd->dev,"found streams %#llx based on streams %#llx\n" , + *__streams, streams); +if (*__streams & ~streams) +return -EINVAL ; + + *__streams = streams; + +return 0 ; +} + +int v4l2_mc_pipeline_enabled (struct video_device *vdev, +bool (*func)(struct video_device *vdev), +struct media_pad **__sink_pad,u64 *__sink_streams) +{ +u64 sink_streams =1U ; +struct media_pad *src_pad; +u64 src_streams; +struct v4l2_subdev_state *state; +struct media_pad *sink_pad = vdev->entity.pads; +struct v4l2_subdev *sd =NULL ; +bool streaming = true; +struct media_pad *tmp_pad; +u64 tmp_streams; +int ret; + +if (!__sink_pad) + __sink_pad = &tmp_pad; +if (!__sink_streams) + __sink_streams = &tmp_streams; + *__sink_pad =NULL ; + *__sink_streams =0 ; + +do { + src_pad =media_pad_remote_pad_unique (sink_pad); +if (IS_ERR (src_pad)) { +dev_dbg (sd ? sd->dev : vdev->dev_parent, +"no unique remote pad found from %s:%u\n" , + sink_pad->entity->name, sink_pad->index); +return PTR_ERR (src_pad); + } + + sd =media_entity_to_v4l2_subdev (src_pad->entity); +if (!sd) { +dev_dbg (sd->dev, +"media entity %s is not a V4L2 sub-device\n" , + src_pad->entity->name); +return -ENXIO ; + } + +/* Source streams match sink. */ + src_streams = sink_streams; + + state =v4l2_subdev_lock_and_get_active_state (sd); + ret =__v4l2_mc_pipeline_enabled (state, src_pad, + src_streams, &sink_pad, + &sink_streams); +v4l2_subdev_unlock_state (state); +if (ret) +return ret; + }while (sink_pad); + + ret =v4l2_mc_source_get_streams (sd, src_pad->index, &src_streams); +if (ret) +return ret; + + sd =media_entity_to_v4l2_subdev (src_pad->entity); + +dev_dbg (sd->dev,"following %s:%u/%#llx\n" , sd->entity.name, + src_pad->index, src_streams); + +for (unsigned int i =__ffs (src_streams); src_streams; + src_streams &= ~BIT_ULL (i), i =__ffs (src_streams)) { + sink_pad =media_pad_remote_pad_unique (src_pad); +if (IS_ERR (src_pad)) { +dev_dbg (sd->dev, +"no unique remote pad found from %s:%u\n" , + sink_pad->entity->name, sink_pad->index); +return PTR_ERR (src_pad); + } + + ret =v4l2_mc_downpath_enabled (sink_pad, i, func, __sink_pad, + __sink_streams); +if (ret ==2 ) +continue ; +if (ret <0 ) +return ret; +if (!ret) + streaming = false; + } + +dev_dbg (media_entity_to_v4l2_subdev ((*__sink_pad)->entity)->dev, +"sink pad %s:%u/%#llx\n" , (*__sink_pad)->entity->name, + (*__sink_pad)->index, *__sink_streams); + +return streaming; +} +EXPORT_SYMBOL_GPL (v4l2_mc_pipeline_enabled);include/media/v4l2-mc.h +0
-0@@ -193,6 +193,50 @@ void v4l2_pipeline_pm_put(struct media_entity *entity);int v4l2_pipeline_link_notify (struct media_link *link,u32 flags,unsigned int notification);
Quoted 8 lines
include/media/v4l2-mc.h void v4l2_pipeline_pm_put(struct media_entity *entity); · line 201
Quoted 6 lines
include/media/v4l2-mc.h +6
-0+/** +* v4l2_mc_pipeline_enabled - Tell when to start streaming +* @vdev: The video device +* @func: Caller-provided function to tell a video device's streaming state +* @__sink_pad: sink pad at the root of the local pipeline +* @__sink_streams: streams to start Any reason for the double underscore ?
The function used internally a variable without the underscores for a different purpose.
Quoted 13 lines
include/media/v4l2-mc.h void v4l2_pipeline_pm_put(struct media_entity *entity); · line 212
Quoted 12 lines
include/media/v4l2-mc.h +11
-0+* +* Use to tell whether streaming should start on a video node. @func returns +* true if streaming has been started on a given video node. @__sink_pad and +* @__sink_streams are filled with pad and streams on the sub-device closest to +* the video nodes, to be used for calling v4l2_subdev_enable_streams() and +* v4l2_subdev_disable_streams(). +* +* Using v4l2_mc_pipeline_enabled() has a few limitations currently (consider it +* a to-do list): +* * only unbranched streams can be supported albeit adding support for +* downstream branches would be fairly trivial,
Quoted 2 lines
I can't tell from the documentation here what you mean exactly by "unbranched streams".
These may have been referred to as "linear" streams elsewhere. I'll address this in the next version.
Quoted 12 lines
include/media/v4l2-mc.h void v4l2_pipeline_pm_put(struct media_entity *entity); · line 222
Quoted 11 lines
include/media/v4l2-mc.h +10
-0+* * streams within a single source sub-device are considered to start at the +* same time, more control could be added in two ways: 1) for sources to +* determine stream starting, a control could be added to UAPI and 2) sources +* could tell which streams start at the same time using a sub-device +* operation, +* * CSI-2 VC framing is ignored currently, but VC-based stream starting could +* be implemented by letting the caller to provide a function to determine +* which streams are of interest and +* * routes leading to nowhere are ignored, on some hardware this is a problem, +* but this can also be rather trivially addressed.
Quoted 7 lines
I'm afraid this function looks like a hack, to solve a problem that is not even explicitly described. You don't explain the issue in the cover letter or in
Documentation/, the cover letter merely states that this is a "partial solution". The documentation of the function doesn't explain what criteria the decision is based on. We need a proper explanation of the problem inDocumentation/, with a description of the behaviour (or behaviours) drivers are expected to implement.
I believe this is discussed in the cover letter, not in detail though. I'm fine with adding more documentation, there isn't much as I also wanted to get feedback on the approach itself. No alternatives have been proposed so far either.
Quoted 7 lines
Furthermore, on the implementation side, things are fairly inefficient. We already traverse the whole pipeline in
media_pipeline_start(), based on links and routes, and populate themedia_pipelinestructure. We shouldn't do the same here, but instead inspectmedia_pipelineto extract the information we need. If you're missing information there, let's add it.
The media pipeline is created and traversed, yes, but the media pipeline does not include streams which are strictly a V4L2 concept. I agree there is some overlap between the two but as long as MC remains separate from V4L2, we can't reasonably access streams from the pipeline traversal.
Quoted 21 lines
Quoted 21 lines
include/media/v4l2-mc.h +17
-0+* +* Return: +* * 0: Success, but don't start streaming yet +* * 1: Success, now it's time to start streaming +* * -ENXIO: Route traversal encountered a non-video device/sub-device entity +* * -ENOTUNIQ: No unique remote pad +* * -ENOLINK: No remote pad found +* * -ENOENT: Enabled upstream route not found +* * -EMLINK: No unique downstream route found +* * -EINVAL: Stream could not be followed to source or was not produced by +* the source +*/ +int v4l2_mc_pipeline_enabled (struct video_device *vdev, +bool (*func)(struct video_device *vdev), +struct media_pad **__sink_pad, +u64 *__sink_streams); +#else /* CONFIG_MEDIA_CONTROLLER */ static inline int v4l2_mc_create_media_graph (struct media_device *mdev)
Signature · 3 lines
Kind regards, Sakari Ailus
Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()
Quoted 285 lines · Sakari Ailus
Quoted 275 lines · Laurent Pinchart
Quoted 274 lines · Sakari Ailus
v4l2_mc_pipeline_enabled()helps solving a problem known for long but lacked any sort of general solution: with multiple streams, when streaming is started on video nodes one by one, when should streaming be started in the source?
v4l2_mc_pipeline_enabled()traverses the pipeline towards the source, queries the streams generated by the source and traces them back to the video nodes.Signed-off-by Sakari Ailus <[email protected]>--- drivers/media/v4l2-core/v4l2-mc.c | 243 ++++++++++++++++++++++++++++++ include/media/v4l2-mc.h | 44 ++++++ 2 files changed, 287 insertions(+)drivers/media/v4l2-core/v4l2-mc.c +243
-0@@ -612,3 +612,246 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,return ret; }EXPORT_SYMBOL_GPL (v4l2_pipeline_link_notify); + +static int +__v4l2_mc_pipeline_enabled (struct v4l2_subdev_state *state, +struct media_pad *src_pad,u64 __src_streams, +struct media_pad **__sink_pad,u64 *__sink_streams) +{ +struct v4l2_subdev_route *route; +u64 src_streams =0 , sink_streams =0 ; +bool has_sink_pad = false; +unsigned int sink_pad; + +dev_dbg (state->sd->dev,"%s: source enabled, pad/streams %u/%#llx\n" , + state->sd->entity.name, src_pad->index, __src_streams); +for_each_active_route (&state->routing, route) { +dev_dbg (state->sd->dev,"%s: %u/%u -> %u/%u, flags %x\n" , + state->sd->entity.name, + route->sink_pad, route->sink_stream, route->source_pad, + route->source_stream, route->flags); +if (route->source_pad != src_pad->index) +continue ; + +if (!(BIT_ULL (route->source_stream) & __src_streams)) +continue ; + +if (!has_sink_pad) { + has_sink_pad = true; + sink_pad = route->sink_pad; + } + +if (route->sink_pad != sink_pad) { +dev_dbg (state->sd->dev, +"sink pads (%u vs. %u) differ\n" , + route->sink_pad, sink_pad); +return -EMLINK ; + } + + sink_streams |=BIT_ULL (route->sink_stream); + src_streams |=BIT_ULL (route->source_stream); + } + + *__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] :NULL ; + *__sink_streams = sink_streams; + +return 0 ; +} + +static int v4l2_mc_downpath_enabled (struct media_pad *sink_pad, +unsigned int sink_stream, +bool (*func)(struct video_device *vdev), +struct media_pad **__sink_pad, +u64 *__sink_streams) +{ +struct v4l2_subdev_state *state; +struct v4l2_subdev_route *route; +struct v4l2_subdev *sd; +struct media_pad *source_pad, *tmp_pad; +u32 source_stream; + +if (!is_media_entity_v4l2_subdev (sink_pad->entity)) +return -ENXIO ; + + sd =media_entity_to_v4l2_subdev (sink_pad->entity); +dev_dbg (sd->dev,"path_enabled: found sub-device %s\n" , + sd->entity.name); + + state =v4l2_subdev_lock_and_get_active_state (sd); + route =v4l2_subdev_find_route (&state->routing, sink_pad->index, + sink_stream, true,0 ); +if (IS_ERR (route)) { +v4l2_subdev_unlock_state (state); +dev_dbg (sd->dev, +"path_enabled: can't find opposite route for %s:%u/%u" , + sd->entity.name, sink_pad->index, sink_stream); +return 2 ; + } + + source_pad = &sd->entity.pads[route->source_pad]; +v4l2_subdev_unlock_state (state); + + tmp_pad = sink_pad; + sink_pad =media_pad_remote_pad_unique (source_pad); +if (IS_ERR (sink_pad)) { +dev_dbg (sd->dev, +"path_enabled: can't find remote source for %s:%u\n" , + source_pad->entity->name, source_pad->index); +return PTR_ERR (sink_pad); + } + +if (is_media_entity_v4l2_video_device (sink_pad->entity)) { +struct video_device *vdev; + + vdev =media_entity_to_video_device (sink_pad->entity); +if (!vdev) +return -ENXIO ; + +dev_dbg (vdev->dev_parent, +"path_enabled: found video device %s\n" , + vdev->name); + +if (!*__sink_pad) { + *__sink_pad = tmp_pad; +dev_dbg (sd->dev,"path_enabled: sink %u/%u\n" , + tmp_pad->index, sink_stream); + }else if (tmp_pad != *__sink_pad) { +dev_dbg (sd->dev, +"path_enabled: pads %s/%u and %s/%u differ\n" , + tmp_pad->entity->name, tmp_pad->index, + (*__sink_pad)->entity->name, + (*__sink_pad)->index); +return -EXDEV ; + } + + *__sink_streams |=BIT_ULL (sink_stream); + +return func (vdev); + } + +return v4l2_mc_downpath_enabled (sink_pad, source_stream, func, + __sink_pad, __sink_streams); +} + +static int v4l2_mc_source_get_streams (struct v4l2_subdev *sd,unsigned int pad, +u64 *__streams) +{ +struct v4l2_mbus_frame_desc desc; +u64 streams =0 ; +int ret; + +if (!__streams) +return -EINVAL ; + + ret =v4l2_subdev_call (sd, pad, get_frame_desc, pad, &desc); +if (ret == -ENOIOCTLCMD ) { + *__streams =1ULL ; +return 0 ; + } +if (ret) +return ret; + +for (unsigned int i =0 ; i < desc.num_entries; i++) { +if (streams &BIT_ULL (desc.entry[i].stream)) +return -EINVAL ; + + streams |=BIT_ULL (desc.entry[i].stream); + } + +dev_dbg (sd->dev,"found streams %#llx based on streams %#llx\n" , + *__streams, streams); +if (*__streams & ~streams) +return -EINVAL ; + + *__streams = streams; + +return 0 ; +} + +int v4l2_mc_pipeline_enabled (struct video_device *vdev, +bool (*func)(struct video_device *vdev), +struct media_pad **__sink_pad,u64 *__sink_streams) +{ +u64 sink_streams =1U ; +struct media_pad *src_pad; +u64 src_streams; +struct v4l2_subdev_state *state; +struct media_pad *sink_pad = vdev->entity.pads; +struct v4l2_subdev *sd =NULL ; +bool streaming = true; +struct media_pad *tmp_pad; +u64 tmp_streams; +int ret; + +if (!__sink_pad) + __sink_pad = &tmp_pad; +if (!__sink_streams) + __sink_streams = &tmp_streams; + *__sink_pad =NULL ; + *__sink_streams =0 ; + +do { + src_pad =media_pad_remote_pad_unique (sink_pad); +if (IS_ERR (src_pad)) { +dev_dbg (sd ? sd->dev : vdev->dev_parent, +"no unique remote pad found from %s:%u\n" , + sink_pad->entity->name, sink_pad->index); +return PTR_ERR (src_pad); + } + + sd =media_entity_to_v4l2_subdev (src_pad->entity); +if (!sd) { +dev_dbg (sd->dev, +"media entity %s is not a V4L2 sub-device\n" , + src_pad->entity->name); +return -ENXIO ; + } + +/* Source streams match sink. */ + src_streams = sink_streams; + + state =v4l2_subdev_lock_and_get_active_state (sd); + ret =__v4l2_mc_pipeline_enabled (state, src_pad, + src_streams, &sink_pad, + &sink_streams); +v4l2_subdev_unlock_state (state); +if (ret) +return ret; + }while (sink_pad); + + ret =v4l2_mc_source_get_streams (sd, src_pad->index, &src_streams); +if (ret) +return ret; + + sd =media_entity_to_v4l2_subdev (src_pad->entity); + +dev_dbg (sd->dev,"following %s:%u/%#llx\n" , sd->entity.name, + src_pad->index, src_streams); + +for (unsigned int i =__ffs (src_streams); src_streams; + src_streams &= ~BIT_ULL (i), i =__ffs (src_streams)) { + sink_pad =media_pad_remote_pad_unique (src_pad); +if (IS_ERR (src_pad)) { +dev_dbg (sd->dev, +"no unique remote pad found from %s:%u\n" , + sink_pad->entity->name, sink_pad->index); +return PTR_ERR (src_pad); + } + + ret =v4l2_mc_downpath_enabled (sink_pad, i, func, __sink_pad, + __sink_streams); +if (ret ==2 ) +continue ; +if (ret <0 ) +return ret; +if (!ret) + streaming = false; + } + +dev_dbg (media_entity_to_v4l2_subdev ((*__sink_pad)->entity)->dev, +"sink pad %s:%u/%#llx\n" , (*__sink_pad)->entity->name, + (*__sink_pad)->index, *__sink_streams); + +return streaming; +} +EXPORT_SYMBOL_GPL (v4l2_mc_pipeline_enabled);include/media/v4l2-mc.h +0
-0@@ -193,6 +193,50 @@ void v4l2_pipeline_pm_put(struct media_entity *entity);int v4l2_pipeline_link_notify (struct media_link *link,u32 flags,unsigned int notification);Quoted 8 lines
include/media/v4l2-mc.h void v4l2_pipeline_pm_put(struct media_entity *entity); · line 201
Quoted 6 lines
include/media/v4l2-mc.h +6
-0+/** +* v4l2_mc_pipeline_enabled - Tell when to start streaming +* @vdev: The video device +* @func: Caller-provided function to tell a video device's streaming state +* @__sink_pad: sink pad at the root of the local pipeline +* @__sink_streams: streams to start Any reason for the double underscore ?
Quoted 2 lines
The function used internally a variable without the underscores for a different purpose.
Then it's better to use the __ for internal variables instead of the publicly documented API. Or rename internal variables so they don't clash.
Quoted 37 lines
Quoted 12 lines
include/media/v4l2-mc.h void v4l2_pipeline_pm_put(struct media_entity *entity); · line 212
Quoted 11 lines
include/media/v4l2-mc.h +11
-0+* +* Use to tell whether streaming should start on a video node. @func returns +* true if streaming has been started on a given video node. @__sink_pad and +* @__sink_streams are filled with pad and streams on the sub-device closest to +* the video nodes, to be used for calling v4l2_subdev_enable_streams() and +* v4l2_subdev_disable_streams(). +* +* Using v4l2_mc_pipeline_enabled() has a few limitations currently (consider it +* a to-do list): +* * only unbranched streams can be supported albeit adding support for +* downstream branches would be fairly trivial, Quoted 2 lines
I can't tell from the documentation here what you mean exactly by "unbranched streams".
These may have been referred to as "linear" streams elsewhere. I'll address this in the next version.
Quoted 11 lines
include/media/v4l2-mc.h void v4l2_pipeline_pm_put(struct media_entity *entity); · line 222
Quoted 10 lines
include/media/v4l2-mc.h +10
-0+* * streams within a single source sub-device are considered to start at the +* same time, more control could be added in two ways: 1) for sources to +* determine stream starting, a control could be added to UAPI and 2) sources +* could tell which streams start at the same time using a sub-device +* operation, +* * CSI-2 VC framing is ignored currently, but VC-based stream starting could +* be implemented by letting the caller to provide a function to determine +* which streams are of interest and +* * routes leading to nowhere are ignored, on some hardware this is a problem, +* but this can also be rather trivially addressed. Quoted 7 lines
I'm afraid this function looks like a hack, to solve a problem that is not even explicitly described. You don't explain the issue in the cover letter or in
Documentation/, the cover letter merely states that this is a "partial solution". The documentation of the function doesn't explain what criteria the decision is based on. We need a proper explanation of the problem inDocumentation/, with a description of the behaviour (or behaviours) drivers are expected to implement.
Quoted 3 lines
I believe this is discussed in the cover letter, not in detail though. I'm fine with adding more documentation, there isn't much as I also wanted to get feedback on the approach itself.
But that's my point, I have trouble giving feedback on the approach if you don't explain what problem you're trying to solve, and why you picked this particular solution. It may be all clear in your head, but it isn't for me.
Quoted 35 lines
No alternatives have been proposed so
far either.Quoted 6 lines
Furthermore, on the implementation side, things are fairly inefficient. We already traverse the whole pipeline in
media_pipeline_start(), based on links and routes, and populate themedia_pipelinestructure. We shouldn't do the same here, but instead inspectmedia_pipelineto extract the information we need. If you're missing information there, let's add it.The media pipeline is created and traversed, yes, but the media pipeline does not include streams which are strictly a V4L2 concept. I agree there is some overlap between the two but as long as MC remains separate from V4L2, we can't reasonably access streams from the pipeline traversal.
Quoted 20 lines
Quoted 20 lines
include/media/v4l2-mc.h +17
-0+* +* Return: +* * 0: Success, but don't start streaming yet +* * 1: Success, now it's time to start streaming +* * -ENXIO: Route traversal encountered a non-video device/sub-device entity +* * -ENOTUNIQ: No unique remote pad +* * -ENOLINK: No remote pad found +* * -ENOENT: Enabled upstream route not found +* * -EMLINK: No unique downstream route found +* * -EINVAL: Stream could not be followed to source or was not produced by +* the source +*/ +int v4l2_mc_pipeline_enabled (struct video_device *vdev, +bool (*func)(struct video_device *vdev), +struct media_pad **__sink_pad, +u64 *__sink_streams); +#else /* CONFIG_MEDIA_CONTROLLER */ static inline int v4l2_mc_create_media_graph (struct media_device *mdev)
Signature · 3 lines
Regards, Laurent Pinchart
Sakari Ailus
maintainerPATCH 13/13 media: ipu6: isys: Rework stream starting and stopping
In reply to Sakari AilusRework starting and stopping streams, based on determining which streams are available on the source. Rely on v4l2_mc_pipeline_enabled() to obtain information on when to start streaming.
This effectively both complicates and simplifies driver implementation. There's some cleanup to be done in the driver after this patch, as starting and stopping streaming is currently a quite complicated process.
--- drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 38 ++++++++++--- .../media/pci/intel/ipu6/ipu6-isys-queue.c | 48 ++++++++++------- .../media/pci/intel/ipu6/ipu6-isys-video.c | 53 ++++++++++++------- .../media/pci/intel/ipu6/ipu6-isys-video.h | 9 ++-- 4 files changed, 99 insertions(+), 49 deletions(-)
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +31 -7
@@ -346,21 +346,31 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,struct ipu6_isys_subdev *asd =to_ipu6_isys_subdev (sd);struct ipu6_isys_csi2 *csi2 =to_ipu6_isys_csi2 (asd);struct ipu6_isys_csi2_timing timing = { }; +struct ipu6_isys_video *iv; +struct ipu6_isys_stream *stream;struct v4l2_subdev *remote_sd; -struct media_pad *remote_pad; -u64 sink_streams, already_enabled; +struct media_pad *remote_pad, *vdev_pad; +u64 sink_streams;int ret; - remote_pad =media_pad_remote_pad_first (&sd->entity.pads[CSI2_PAD_SINK ]); - remote_sd =media_entity_to_v4l2_subdev (remote_pad->entity); + vdev_pad =media_pad_remote_pad_first (&sd->entity.pads[pad]); + iv =container_of_const (media_entity_to_video_device (vdev_pad->entity), + struct ipu6_isys_video, vdev); + stream = iv->stream; sink_streams =v4l2_subdev_state_xlate_streams (state, pad,CSI2_PAD_SINK , &streams_mask); - already_enabled =v4l2_subdev_state_streams_enabled (sd, state, -CSI2_PAD_SINK ); -if (!already_enabled) { +if ((sink_streams | stream->streams_enabled) != stream->streams) { + stream->streams_enabled |= sink_streams; +return 0 ; + } + + remote_pad =media_pad_remote_pad_first (&sd->entity.pads[CSI2_PAD_SINK ]); + remote_sd =media_entity_to_v4l2_subdev (remote_pad->entity); + +if (!stream->streaming) { ret =ipu6_isys_csi2_calc_timing (csi2, &timing,CSI2_ACCINV );if (ret)return ret;
@@ -371,12 +381,15 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd, } ret =v4l2_subdev_enable_streams (remote_sd, remote_pad->index, + stream->streams_enabled | sink_streams);if (ret) {ipu6_isys_csi2_set_stream (sd,NULL ,0 , false);return ret; } + stream->streams_enabled |= sink_streams; +return 0 ; }
@@ -402,6 +415,17 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,v4l2_subdev_disable_streams (remote_sd, remote_pad->index, sink_streams); +struct media_pad *vdev_pad; +struct ipu6_isys_video *iv; +struct ipu6_isys_stream *stream; + + vdev_pad =media_pad_remote_pad_first (&sd->entity.pads[pad]); + iv =container_of_const (media_entity_to_video_device (vdev_pad->entity), + struct ipu6_isys_video, vdev); + stream = iv->stream; + + stream->streams_enabled &= ~sink_streams; +return 0 ; }
drivers/media/pci/intel/ipu6/ipu6-isys-queue.c +29 -19
116 lines not on this page · raw message
drivers/media/pci/intel/ipu6/ipu6-isys-video.c +35 -18
110 lines not on this page · raw message
drivers/media/pci/intel/ipu6/ipu6-isys-video.h +4 -5
@@ -44,6 +44,8 @@ struct sequence_info {struct ipu6_isys_stream {struct mutex mutex;struct media_entity *source_entity; +u64 streams; +u64 streams_enabled;atomic_t sequence;unsigned int seq_index;struct sequence_info seq[IPU6_ISYS_MAX_PARALLEL_SOF ];
@@ -52,8 +54,6 @@ struct ipu6_isys_stream {unsigned int nr_output_pins;struct ipu6_isys_subdev *asd; -int nr_queues;/* Number of capture queues */ -int nr_streaming;int streaming;/* Has streaming been really started? */ struct list_head queues;struct completion stream_open_completion;
@@ -106,14 +106,13 @@ extern const struct ipu6_isys_pixelformat ipu6_isys_pfmts_packed[];const struct ipu6_isys_pixelformat *ipu6_isys_get_isys_format (u32 pixelformat,u32 code);int ipu6_isys_video_prepare_stream (struct ipu6_isys_video *av, -struct media_entity *source_entity, -int nr_queues); +struct media_entity *source_entity);int ipu6_isys_video_set_streaming (struct ipu6_isys_video *av,int state,struct ipu6_isys_buffer_list *bl);int ipu6_isys_fw_open (struct ipu6_isys *isys);void ipu6_isys_fw_close (struct ipu6_isys *isys);int ipu6_isys_setup_video (struct ipu6_isys_video *av, -struct media_entity **source_entity,int *nr_queues); +struct media_entity **source_entity);int ipu6_isys_video_init (struct ipu6_isys_video *av);void ipu6_isys_video_cleanup (struct ipu6_isys_video *av);void ipu6_isys_put_stream (struct ipu6_isys_stream *stream);
Signature · 1 line
2.39.5