v4l: routing and streams support
Subsystems (2)
v16 cover v4l: routing and streams support
Hi,
This is v16 of the streams series. The v15 can be found from:
https://lore.kernel.org/all/[email protected]/
My work branch with additional drivers can be found from:
git://git.kernel.org/pub/scm/linux/kernel/git/tomba/linux.git streams/work-v16
And there's also the v4l-utils series v2 to add support to v4l2-ctl and media-ctl:
https://lore.kernel.org/all/[email protected]/
(Note: the v4l-utils above has not been changed to accommodate the drop of V4L2_SUBDEV_ROUTE_FL_SOURCE yet)
Changes in v16:
- v4l2_subdev_link_validate_get_format: Drop use of for_each_set_bit(), as the sink_streams_mask is not unsigned long.
- v4l2_subdev_set_routing: Cast parameters given to check_mul_overflow() to size_t to avoid compiler warnings.
- Drop V4L2_SUBDEV_ROUTE_FL_SOURCE
- Require a code change and kernel compilation to enable Streams API.
A bit more details for the last two changes:
V4L2_SUBDEV_ROUTE_FL_SOURCE felt somewhat hacky, and the whole question of how sensors' streams in various cases should be handled was unclear. Thus it was decided (me, Laurent, Sakari) that we should only support the current single-stream sensors and work on multi-streams sensors on top of this series. This should not hinder any use case that was supportable in the v15, as most likely the second stream from a sensor would be an embedded data stream, and embedded data is not (really) supported upstream.
The routing and stream support has been worked on for years by multiple persons, and has proven to be very complicated feature, or a set of features. While this series has been used on multiple different platforms for various use cases, we would still like to be on the safe side, especially regarding the userspace APIs. Thus, we decided that it would be good to get this series merged while still making it clear that the uAPI is not quite stable yet, and to achieve that, there's a static variable in v4l2-subdev.c that the user needs to set to enable the API.
Tomi
Jacopo Mondi (2):
media: Documentation: Add GS_ROUTING documentation
media: subdev: Add for_each_active_route() macro
Laurent Pinchart (5):
media: subdev: Add [GS]_ROUTING subdev ioctls and operations
media: subdev: add v4l2_subdev_routing_validate() helper
media: v4l2-subdev: Add v4l2_subdev_state_xlate_streams() helper
media: v4l2-subdev: Add subdev .(enable|disable)_streams() operations
media: v4l2-subdev: Add v4l2_subdev_s_stream_helper() function
Sakari Ailus (1):
media: Add stream to frame descriptor
Tomi Valkeinen (12):
media: v4l2-subdev: Sort includes
media: add V4L2_SUBDEV_FL_STREAMS
media: add V4L2_SUBDEV_CAP_STREAMS
media: subdev: Require code change to enable [GS]_ROUTING
media: subdev: add v4l2_subdev_has_pad_interdep()
media: subdev: add v4l2_subdev_set_routing helper()
media: Documentation: add multiplexed streams documentation
media: subdev: add stream based configuration
media: subdev: use streams in v4l2_subdev_link_validate()
media: subdev: add "opposite" stream helper funcs
media: subdev: add streams to v4l2_subdev_get_fmt() helper function
media: subdev: add v4l2_subdev_set_routing_with_fmt() helper
.clang-format | 1 + .../driver-api/media/v4l2-subdev.rst | 8 + .../userspace-api/media/v4l/dev-subdev.rst | 140 +++ .../userspace-api/media/v4l/user-func.rst | 1 + .../v4l/vidioc-subdev-enum-frame-interval.rst | 5 +- .../v4l/vidioc-subdev-enum-frame-size.rst | 5 +- .../v4l/vidioc-subdev-enum-mbus-code.rst | 5 +- .../media/v4l/vidioc-subdev-g-crop.rst | 5 +- .../media/v4l/vidioc-subdev-g-fmt.rst | 5 +- .../v4l/vidioc-subdev-g-frame-interval.rst | 5 +- .../media/v4l/vidioc-subdev-g-routing.rst | 147 +++ .../media/v4l/vidioc-subdev-g-selection.rst | 5 +- drivers/media/v4l2-core/v4l2-ioctl.c | 25 +- drivers/media/v4l2-core/v4l2-subdev.c | 1000 ++++++++++++++++- include/media/v4l2-subdev.h | 377 +++++++ include/uapi/linux/v4l2-subdev.h | 74 +- 16 files changed, 1758 insertions(+), 50 deletions(-) create mode 100644 Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
Signature · 1 line
2.34.1
PATCH v16 1/20 media: v4l2-subdev: Sort includes
Sort the includes alphabetically.
--- drivers/media/v4l2-core/v4l2-subdev.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
drivers/media/v4l2-core/v4l2-subdev.c +4 -4
@@ -8,20 +8,20 @@* Sakari Ailus <[email protected]> */ +#include <linux/export.h> #include <linux/ioctl.h> #include <linux/mm.h> #include <linux/module.h> #include <linux/slab.h> #include <linux/types.h> -#include <linux/videodev2.h> -#include <linux/export.h> #include <linux/version.h> +#include <linux/videodev2.h> #include <media/v4l2-ctrls.h> #include <media/v4l2-device.h> -#include <media/v4l2-ioctl.h> -#include <media/v4l2-fh.h> #include <media/v4l2-event.h> +#include <media/v4l2-fh.h> +#include <media/v4l2-ioctl.h> #if defined(CONFIG_VIDEO_V4L2_SUBDEV_API )static int subdev_fh_init (struct v4l2_subdev_fh *fh,struct v4l2_subdev *sd)
Signature · 1 line
2.34.1
PATCH v16 2/20 media: add V4L2_SUBDEV_FL_STREAMS
In reply to Tomi ValkeinenAdd subdev flag V4L2_SUBDEV_FL_STREAMS. It is used to indicate that the subdev supports the new API with multiplexed streams (routing, stream configs).
--- include/media/v4l2-subdev.h | 11 +++++++++++ 1 file changed, 11 insertions(+)
include/media/v4l2-subdev.h +11 -0
@@ -879,6 +879,17 @@ struct v4l2_subdev_internal_ops {* should set this flag. */ #define V4L2_SUBDEV_FL_HAS_EVENTS (1U << 3) +/* +* Set this flag if this subdev supports multiplexed streams. This means +* that the driver supports routing and handles the stream parameter in its +* v4l2_subdev_pad_ops handlers. More specifically, this means: +* +* - Centrally managed subdev active state is enabled +* - Legacy pad config is _not_ supported (state->pads is NULL) +* - Routing ioctls are available +* - Multiple streams per pad are supported +*/ +#define V4L2_SUBDEV_FL_STREAMS (1U << 4)struct regulator_bulk_data ;
Signature · 1 line
2.34.1
Re: [PATCH v16 02/20] media: add V4L2_SUBDEV_FL_STREAMS
Hi Tomi,
Thank you for the patch.
Quoted 15 lines · Tomi Valkeinen
Add subdev flag
V4L2_SUBDEV_FL_STREAMS. It is used to indicate that the subdev supports the new API with multiplexed streams (routing, stream configs).Signed-off-by Tomi Valkeinen <[email protected]>--- include/media/v4l2-subdev.h | 11 +++++++++++ 1 file changed, 11 insertions(+)include/media/v4l2-subdev.h +0
-0@@ -879,6 +879,17 @@ struct v4l2_subdev_internal_ops {* should set this flag.
include/media/v4l2-subdev.h struct v4l2_subdev_internal_ops { · line 889
Quoted 10 lines
include/media/v4l2-subdev.h +8
-0*/ #define V4L2_SUBDEV_FL_HAS_EVENTS (1U << 3) +/* +* Set this flag if this subdev supports multiplexed streams. This means +* that the driver supports routing and handles the stream parameter in its +* v4l2_subdev_pad_ops handlers. More specifically, this means: +* +* - Centrally managed subdev active state is enabled +* - Legacy pad config is _not_ supported (state->pads is NULL) +* - Routing ioctls are available
I wonder, as a sanity check, should the subdev core verify that the s_routing operation is available when this flag is set ? This could be done on top, so
Quoted 6 lines
include/media/v4l2-subdev.h +3
-0+* - Multiple streams per pad are supported +*/ +#define V4L2_SUBDEV_FL_STREAMS (1U << 4)struct regulator_bulk_data ;
Signature · 3 lines
Regards, Laurent Pinchart
PATCH v16 3/20 media: add V4L2_SUBDEV_CAP_STREAMS
In reply to Tomi ValkeinenAdd a subdev capability flag to expose to userspace if a subdev supports multiplexed streams.
--- drivers/media/v4l2-core/v4l2-subdev.c | 5 ++++- include/uapi/linux/v4l2-subdev.h | 3 +++ 2 files changed, 7 insertions(+), 1 deletion(-)
drivers/media/v4l2-core/v4l2-subdev.c +4 -1
@@ -431,6 +431,7 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,struct v4l2_subdev *sd =vdev_to_v4l2_subdev (vdev);struct v4l2_fh *vfh = file->private_data;bool ro_subdev =test_bit (V4L2_FL_SUBDEV_RO_DEVNODE , &vdev->flags); +bool streams_subdev = sd->flags &V4L2_SUBDEV_FL_STREAMS ;int rval;switch (cmd) {
@@ -439,7 +440,9 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,memset (cap->reserved,0 ,sizeof (cap->reserved)); cap->version =LINUX_VERSION_CODE ; - cap->capabilities = ro_subdev ?V4L2_SUBDEV_CAP_RO_SUBDEV :0 ; + cap->capabilities = + (ro_subdev ?V4L2_SUBDEV_CAP_RO_SUBDEV :0 ) | + (streams_subdev ?V4L2_SUBDEV_CAP_STREAMS :0 );return 0 ; }
include/uapi/linux/v4l2-subdev.h +3 -0
@@ -188,6 +188,9 @@ struct v4l2_subdev_capability {/* The v4l2 sub-device video device node is registered in read-only mode. */ #define V4L2_SUBDEV_CAP_RO_SUBDEV 0x00000001 +/* The v4l2 sub-device supports routing and multiplexed streams. */ +#define V4L2_SUBDEV_CAP_STREAMS 0x00000002 +/* Backwards compatibility define --- to be removed */ #define v4l2_subdev_edid v4l2_edid
Signature · 1 line
2.34.1
PATCH v16 4/20 media: Documentation: Add GS_ROUTING documentation
In reply to Tomi ValkeinenFrom: Jacopo Mondi <[email protected]>
Add documentation for VIDIOC_SUBDEV_G/S_ROUTING ioctl and add description of multiplexed media pads and internal routing to the V4L2-subdev documentation section.
--- .../userspace-api/media/v4l/dev-subdev.rst | 2 + .../userspace-api/media/v4l/user-func.rst | 1 + .../media/v4l/vidioc-subdev-g-routing.rst | 147 ++++++++++++++++++ 3 files changed, 150 insertions(+) create mode 100644 Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
Documentation/userspace-api/media/v4l/dev-subdev.rst +2 -0
@@ -29,6 +29,8 @@ will feature a character device node on which ioctls can be called to - negotiate image formats on individual pads +- inspect and modify internal data routing between pads of the same entity + Sub-device character device nodes, conventionally named ``/dev/v4l-subdev*``, use major number 81.
Documentation/userspace-api/media/v4l/user-func.rst +1 -0
@@ -70,6 +70,7 @@ Function Reference vidioc-subdev-g-crop vidioc-subdev-g-fmt vidioc-subdev-g-frame-interval + vidioc-subdev-g-routing vidioc-subdev-g-selection vidioc-subdev-querycap vidioc-subscribe-event
Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst new file +147 -0
147 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v16 5/20 media: subdev: Add [GS]_ROUTING subdev ioctls and operations
In reply to Tomi ValkeinenFrom: Laurent Pinchart <[email protected]>
Add support for subdev internal routing. A route is defined as a single stream from a sink pad to a source pad.
The userspace can configure the routing via two new ioctls, VIDIOC_SUBDEV_G_ROUTING and VIDIOC_SUBDEV_S_ROUTING, and subdevs can implement the functionality with v4l2_subdev_pad_ops.set_routing().
- Add sink and source streams for multiplexed links
- Copy the argument back in case of an error. This is needed to let the caller know the number of routes.
- Expand and refine documentation.
- Make the 'routes' pointer a __u64 __user pointer so that a compat32 version of the ioctl is not required.
- Add struct v4l2_subdev_krouting to be used for subdevice operations.
- Fix typecasing warnings
- Check sink & source pad types
- Add 'which' field
- Routing to subdev state
- Dropped get_routing subdev op
--- drivers/media/v4l2-core/v4l2-ioctl.c | 25 +++++++- drivers/media/v4l2-core/v4l2-subdev.c | 84 +++++++++++++++++++++++++++ include/media/v4l2-subdev.h | 22 +++++++ include/uapi/linux/v4l2-subdev.h | 43 ++++++++++++++ 4 files changed, 173 insertions(+), 1 deletion(-)
drivers/media/v4l2-core/v4l2-ioctl.c +24 -1
@@ -16,6 +16,7 @@#include <linux/kernel.h> #include <linux/version.h> +#include <linux/v4l2-subdev.h> #include <linux/videodev2.h> #include <media/media-device.h> /* for media_set_bus_info() */
@@ -3151,6 +3152,21 @@ static int check_array_args(unsigned int cmd, void *parg, size_t *array_size, ret =1 ;break ; } + +case VIDIOC_SUBDEV_G_ROUTING : +case VIDIOC_SUBDEV_S_ROUTING : { +struct v4l2_subdev_routing *routing = parg; + +if (routing->num_routes >256 ) +return -E2BIG ; + + *user_ptr =u64_to_user_ptr (routing->routes); + *kernel_ptr = (void **)&routing->routes; + *array_size =sizeof (struct v4l2_subdev_route ) + * routing->num_routes; + ret =1 ; +break ; + } }return ret;
@@ -3397,8 +3413,15 @@ video_usercopy(struct file *file, unsigned int orig_cmd, unsigned long arg,/* * Some ioctls can return an error, but still have valid * results that must be returned. +* +* FIXME: subdev IOCTLS are partially handled here and partially in +* v4l2-subdev.c and the 'always_copy' flag can only be set for IOCTLS +* defined here as part of the 'v4l2_ioctls' array. As +* VIDIOC_SUBDEV_G_ROUTING needs to return results to applications even +* in case of failure, but it is not defined here as part of the +* 'v4l2_ioctls' array, insert an ad-hoc check to address that. */ -if (err <0 && !always_copy) +if (err <0 && !always_copy && cmd !=VIDIOC_SUBDEV_G_ROUTING ) goto out;if (has_array_args) {
drivers/media/v4l2-core/v4l2-subdev.c +84 -0
108 lines not on this page · raw message
include/media/v4l2-subdev.h +22 -0
@@ -695,12 +695,26 @@ struct v4l2_subdev_pad_config {struct v4l2_rect try_compose; }; +/** +* struct v4l2_subdev_krouting - subdev routing table +* +* @num_routes: number of routes +* @routes: &struct v4l2_subdev_route +* +* This structure contains the routing table for a subdev. +*/ +struct v4l2_subdev_krouting { +unsigned int num_routes; +struct v4l2_subdev_route *routes; +}; +/** * struct v4l2_subdev_state - Used for storing subdev state information. * * @_lock: default for 'lock' * @lock: mutex for the state. May be replaced by the user. * @pads: &struct v4l2_subdev_pad_config array +* @routing: routing table for the subdev * * This structure only needs to be passed to the pad op if the 'which' field * of the main argument is set to %V4L2_SUBDEV_FORMAT_TRY. For
@@ -711,6 +725,7 @@ struct v4l2_subdev_state {struct mutex _lock;struct mutex *lock;struct v4l2_subdev_pad_config *pads; +struct v4l2_subdev_krouting routing; };/**
@@ -763,6 +778,9 @@ struct v4l2_subdev_state {* this operation as close as possible to stream on time. The * operation shall fail if the pad index it has been called on * is not valid or in case of unrecoverable failures. +* +* @set_routing: enable or disable data connection routes described in the +* subdevice routing table. */ struct v4l2_subdev_pad_ops { int (*init_cfg)(struct v4l2_subdev *sd,
@@ -805,6 +823,10 @@ struct v4l2_subdev_pad_ops { struct v4l2_mbus_frame_desc *fd);int (*get_mbus_config)(struct v4l2_subdev *sd,unsigned int pad,struct v4l2_mbus_config *config); +int (*set_routing)(struct v4l2_subdev *sd, +struct v4l2_subdev_state *state, +enum v4l2_subdev_format_whence which, +struct v4l2_subdev_krouting *route); };/**
include/uapi/linux/v4l2-subdev.h +43 -0
61 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v16 6/20 media: subdev: Require code change to enable [GS]_ROUTING
In reply to Tomi ValkeinenStreams API is an experimental feature. To use Streams API, the user needs to change a variable in v4l2-subdev.c and recompile the kernel.
This commit should be reverted when the Streams API is deemed ready for production use.
--- drivers/media/v4l2-core/v4l2-subdev.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
drivers/media/v4l2-core/v4l2-subdev.c +13 -0
@@ -23,6 +23,13 @@#include <media/v4l2-fh.h> #include <media/v4l2-ioctl.h> +/* +* Streams API is an experimental feature. To use Streams API, set +* 'v4l2_subdev_enable_streams_api' to 1 below. +*/ + +static bool v4l2_subdev_enable_streams_api =0 ; +/* * Maximum stream ID is 63 for now, as we use u64 bitmask to represent a set * of streams.
@@ -751,6 +758,9 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,struct v4l2_subdev_routing *routing = arg;struct v4l2_subdev_krouting *krouting; +if (!v4l2_subdev_enable_streams_api) +return -ENOIOCTLCMD ; +if (!(sd->flags &V4L2_SUBDEV_FL_STREAMS ))return -ENOIOCTLCMD ;
@@ -778,6 +788,9 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,struct v4l2_subdev_krouting krouting = {};unsigned int i; +if (!v4l2_subdev_enable_streams_api) +return -ENOIOCTLCMD ; +if (!(sd->flags &V4L2_SUBDEV_FL_STREAMS ))return -ENOIOCTLCMD ;
Signature · 1 line
2.34.1
Re: [PATCH v16 06/20] media: subdev: Require code change to enable [GS]_ROUTING
Moi,
Quoted 16 lines · Tomi Valkeinen
Streams API is an experimental feature. To use Streams API, the user needs to change a variable in
v4l2-subdev.cand recompile the kernel.This commit should be reverted when the Streams API is deemed ready for production use.
Signed-off-by Tomi Valkeinen <[email protected]>--- drivers/media/v4l2-core/v4l2-subdev.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)drivers/media/v4l2-core/v4l2-subdev.c +0
-0
drivers/media/v4l2-core/v4l2-subdev.c line 31
Quoted 9 lines
drivers/media/v4l2-core/v4l2-subdev.c +6
-0@@ -23,6 +23,13 @@#include <media/v4l2-fh.h> #include <media/v4l2-ioctl.h> +/* +* Streams API is an experimental feature. To use Streams API, set +* 'v4l2_subdev_enable_streams_api' to 1 below. +*/ + +static bool v4l2_subdev_enable_streams_api =0 ;
Static variables shouldn't be explicitly assigned to zero. I'll address this while applying.
Quoted 24 lines
drivers/media/v4l2-core/v4l2-subdev.c +7
-0+/* * Maximum stream ID is 63 for now, as we use u64 bitmask to represent a set * of streams. @@ -751,6 +758,9 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,struct v4l2_subdev_routing *routing = arg;struct v4l2_subdev_krouting *krouting; +if (!v4l2_subdev_enable_streams_api) +return -ENOIOCTLCMD ; +if (!(sd->flags &V4L2_SUBDEV_FL_STREAMS ))return -ENOIOCTLCMD ;@@ -778,6 +788,9 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,struct v4l2_subdev_krouting krouting = {};unsigned int i; +if (!v4l2_subdev_enable_streams_api) +return -ENOIOCTLCMD ; +if (!(sd->flags &V4L2_SUBDEV_FL_STREAMS ))return -ENOIOCTLCMD ;
Signature · 3 lines
Terveisin, Sakari Ailus
PATCH 1/1 media: subdev: Require code change to enable [GS]_ROUTING
In reply to Tomi ValkeinenStreams API is an experimental feature. To use Streams API, the user needs to change a variable in v4l2-subdev.c and recompile the kernel.
This commit should be reverted when the Streams API is deemed ready for production use.
---
Here's a minor update to the patch:
- Fix unused var warning
- Don't initialize the static var to 0
drivers/media/v4l2-core/v4l2-subdev.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
drivers/media/v4l2-core/v4l2-subdev.c +15 -0
@@ -23,6 +23,15 @@#include <media/v4l2-fh.h> #include <media/v4l2-ioctl.h> +#if defined(CONFIG_VIDEO_V4L2_SUBDEV_API ) +/* +* Streams API is an experimental feature. To use Streams API, set +* 'v4l2_subdev_enable_streams_api' to 1 below. +*/ + +static bool v4l2_subdev_enable_streams_api; +#endif +/* * Maximum stream ID is 63 for now, as we use u64 bitmask to represent a set * of streams.
@@ -751,6 +760,9 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,struct v4l2_subdev_routing *routing = arg;struct v4l2_subdev_krouting *krouting; +if (!v4l2_subdev_enable_streams_api) +return -ENOIOCTLCMD ; +if (!(sd->flags &V4L2_SUBDEV_FL_STREAMS ))return -ENOIOCTLCMD ;
@@ -778,6 +790,9 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,struct v4l2_subdev_krouting krouting = {};unsigned int i; +if (!v4l2_subdev_enable_streams_api) +return -ENOIOCTLCMD ; +if (!(sd->flags &V4L2_SUBDEV_FL_STREAMS ))return -ENOIOCTLCMD ;
Signature · 1 line
2.34.1
Re: [PATCH v16.1] media: subdev: Require code change to enable [GS]_ROUTING
Hi Tomi,
Thank you for the patch.
Quoted 1 line · Tomi Valkeinen
Streams API is an experimental feature. To use Streams API, the user
Nitpicking, I'd write "The Streams API" and "To use the Streams API".
I'm sure Sakari can handle this in the second version of the pull request.
Quoted 6 lines
needs to change a variable in
v4l2-subdev.cand recompile the kernel.This commit should be reverted when the Streams API is deemed ready for production use.
Signed-off-by Tomi Valkeinen <[email protected]>
Quoted 49 lines
---
Here's a minor update to the patch:
- Fix unused var warning
- Don't initialize the static var to 0drivers/media/v4l2-core/v4l2-subdev.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)drivers/media/v4l2-core/v4l2-subdev.c +15
-0@@ -23,6 +23,15 @@#include <media/v4l2-fh.h> #include <media/v4l2-ioctl.h> +#if defined(CONFIG_VIDEO_V4L2_SUBDEV_API ) +/* +* Streams API is an experimental feature. To use Streams API, set +* 'v4l2_subdev_enable_streams_api' to 1 below. +*/ + +static bool v4l2_subdev_enable_streams_api; +#endif +/* * Maximum stream ID is 63 for now, as we use u64 bitmask to represent a set * of streams. @@ -751,6 +760,9 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,struct v4l2_subdev_routing *routing = arg;struct v4l2_subdev_krouting *krouting; +if (!v4l2_subdev_enable_streams_api) +return -ENOIOCTLCMD ; +if (!(sd->flags &V4L2_SUBDEV_FL_STREAMS ))return -ENOIOCTLCMD ;@@ -778,6 +790,9 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,struct v4l2_subdev_krouting krouting = {};unsigned int i; +if (!v4l2_subdev_enable_streams_api) +return -ENOIOCTLCMD ; +if (!(sd->flags &V4L2_SUBDEV_FL_STREAMS ))return -ENOIOCTLCMD ;
Signature · 3 lines
Regards, Laurent Pinchart
Re: [PATCH v16.1] media: subdev: Require code change to enable [GS]_ROUTING
Quoted 38 lines · Laurent Pinchart
Hi Tomi,
Thank you for the patch.
Quoted 1 line · Tomi Valkeinen
Streams API is an experimental feature. To use Streams API, the user
Nitpicking, I'd write "The Streams API" and "To use the Streams API".
I'm sure Sakari can handle this in the second version of the pull request.
Quoted 6 lines
needs to change a variable in
v4l2-subdev.cand recompile the kernel.This commit should be reverted when the Streams API is deemed ready for production use.
Signed-off-by Tomi Valkeinen <[email protected]>Reviewed-by Laurent Pinchart <[email protected]>Quoted 17 lines
---
Here's a minor update to the patch:
- Fix unused var warning
- Don't initialize the static var to 0drivers/media/v4l2-core/v4l2-subdev.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)drivers/media/v4l2-core/v4l2-subdev.c +0
-0@@ -23,6 +23,15 @@#include <media/v4l2-fh.h> #include <media/v4l2-ioctl.h>
Quoted 3 lines
drivers/media/v4l2-core/v4l2-subdev.c line 28
Quoted 3 lines
drivers/media/v4l2-core/v4l2-subdev.c +3
-0+#if defined(CONFIG_VIDEO_V4L2_SUBDEV_API ) +/* +* Streams API is an experimental feature. To use Streams API, set
Oh, and here too.
Quoted 29 lines
Quoted 29 lines
drivers/media/v4l2-core/v4l2-subdev.c +12
-0+* 'v4l2_subdev_enable_streams_api' to 1 below. +*/ + +static bool v4l2_subdev_enable_streams_api; +#endif +/* * Maximum stream ID is 63 for now, as we use u64 bitmask to represent a set * of streams. @@ -751,6 +760,9 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,struct v4l2_subdev_routing *routing = arg;struct v4l2_subdev_krouting *krouting; +if (!v4l2_subdev_enable_streams_api) +return -ENOIOCTLCMD ; +if (!(sd->flags &V4L2_SUBDEV_FL_STREAMS ))return -ENOIOCTLCMD ;@@ -778,6 +790,9 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,struct v4l2_subdev_krouting krouting = {};unsigned int i; +if (!v4l2_subdev_enable_streams_api) +return -ENOIOCTLCMD ; +if (!(sd->flags &V4L2_SUBDEV_FL_STREAMS ))return -ENOIOCTLCMD ;
Signature · 3 lines
Regards, Laurent Pinchart
PATCH v16 7/20 media: subdev: add v4l2_subdev_has_pad_interdep()
In reply to Tomi ValkeinenAdd a v4l2_subdev_has_pad_interdep() helper function which can be used for media_entity_operations.has_pad_interdep op.
It considers two pads interdependent if there is an active route between pad0 and pad1.
--- drivers/media/v4l2-core/v4l2-subdev.c | 31 +++++++++++++++++++++++++++ include/media/v4l2-subdev.h | 18 ++++++++++++++++ 2 files changed, 49 insertions(+)
drivers/media/v4l2-core/v4l2-subdev.c +31 -0
@@ -1056,6 +1056,37 @@ int v4l2_subdev_link_validate(struct media_link *link) }EXPORT_SYMBOL_GPL (v4l2_subdev_link_validate); +bool v4l2_subdev_has_pad_interdep (struct media_entity *entity, +unsigned int pad0,unsigned int pad1) +{ +struct v4l2_subdev *sd =media_entity_to_v4l2_subdev (entity); +struct v4l2_subdev_krouting *routing; +struct v4l2_subdev_state *state; +unsigned int i; + + state =v4l2_subdev_lock_and_get_active_state (sd); + + routing = &state->routing; + +for (i =0 ; i < routing->num_routes; ++i) { +struct v4l2_subdev_route *route = &routing->routes[i]; + +if (!(route->flags &V4L2_SUBDEV_ROUTE_FL_ACTIVE )) +continue ; + +if ((route->sink_pad == pad0 && route->source_pad == pad1) || + (route->source_pad == pad0 && route->sink_pad == pad1)) { +v4l2_subdev_unlock_state (state); +return true; + } + } + +v4l2_subdev_unlock_state (state); + +return false; +} +EXPORT_SYMBOL_GPL (v4l2_subdev_has_pad_interdep); +struct v4l2_subdev_state *__v4l2_subdev_state_alloc (struct v4l2_subdev *sd,const char *lock_name,struct lock_class_key *lock_key)
include/media/v4l2-subdev.h +18 -0
@@ -1241,6 +1241,24 @@ int v4l2_subdev_link_validate_default(struct v4l2_subdev *sd,*/ int v4l2_subdev_link_validate (struct media_link *link); +/** +* v4l2_subdev_has_pad_interdep - MC has_pad_interdep implementation for subdevs +* +* @entity: pointer to &struct media_entity +* @pad0: pad number for the first pad +* @pad1: pad number for the second pad +* +* This function is an implementation of the +* media_entity_operations.has_pad_interdep operation for subdevs that +* implement the multiplexed streams API (as indicated by the +* V4L2_SUBDEV_FL_STREAMS subdev flag). +* +* It considers two pads interdependent if there is an active route between pad0 +* and pad1. +*/ +bool v4l2_subdev_has_pad_interdep (struct media_entity *entity, +unsigned int pad0,unsigned int pad1); +/** * __v4l2_subdev_state_alloc - allocate v4l2_subdev_state *
Signature · 1 line
2.34.1
PATCH v16 8/20 media: subdev: add v4l2_subdev_set_routing helper()
In reply to Tomi ValkeinenAdd a helper function to set the subdev routing. The helper can be used from subdev driver's set_routing op to store the routing table.
--- drivers/media/v4l2-core/v4l2-subdev.c | 31 +++++++++++++++++++++++++++ include/media/v4l2-subdev.h | 16 ++++++++++++++ 2 files changed, 47 insertions(+)
drivers/media/v4l2-core/v4l2-subdev.c +31 -0
@@ -12,6 +12,7 @@#include <linux/ioctl.h> #include <linux/mm.h> #include <linux/module.h> +#include <linux/overflow.h> #include <linux/slab.h> #include <linux/types.h> #include <linux/version.h>
@@ -1191,6 +1192,36 @@ int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state, }EXPORT_SYMBOL_GPL (v4l2_subdev_get_fmt); +int v4l2_subdev_set_routing (struct v4l2_subdev *sd, +struct v4l2_subdev_state *state, +const struct v4l2_subdev_krouting *routing) +{ +struct v4l2_subdev_krouting *dst = &state->routing; +const struct v4l2_subdev_krouting *src = routing; +struct v4l2_subdev_krouting new_routing = {0 }; +size_t bytes; + +if (unlikely (check_mul_overflow ((size_t )src->num_routes, +sizeof (*src->routes), &bytes))) +return -EOVERFLOW ; + +lockdep_assert_held (state->lock); + +if (src->num_routes >0 ) { + new_routing.routes =kmemdup (src->routes, bytes,GFP_KERNEL ); +if (!new_routing.routes) +return -ENOMEM ; + } + + new_routing.num_routes = src->num_routes; + +kfree (dst->routes); + *dst = new_routing; + +return 0 ; +} +EXPORT_SYMBOL_GPL (v4l2_subdev_set_routing); +#endif /* CONFIG_VIDEO_V4L2_SUBDEV_API */ #endif /* CONFIG_MEDIA_CONTROLLER */
include/media/v4l2-subdev.h +16 -0
@@ -1419,6 +1419,22 @@ v4l2_subdev_lock_and_get_active_state(struct v4l2_subdev *sd)int v4l2_subdev_get_fmt (struct v4l2_subdev *sd,struct v4l2_subdev_state *state,struct v4l2_subdev_format *format); +/** +* v4l2_subdev_set_routing() - Set given routing to subdev state +* @sd: The subdevice +* @state: The subdevice state +* @routing: Routing that will be copied to subdev state +* +* This will release old routing table (if any) from the state, allocate +* enough space for the given routing, and copy the routing. +* +* This can be used from the subdev driver's set_routing op, after validating +* the routing. +*/ +int v4l2_subdev_set_routing (struct v4l2_subdev *sd, +struct v4l2_subdev_state *state, +const struct v4l2_subdev_krouting *routing); +#endif /* CONFIG_VIDEO_V4L2_SUBDEV_API */ #endif /* CONFIG_MEDIA_CONTROLLER */
Signature · 1 line
2.34.1
PATCH v16 9/20 media: subdev: Add for_each_active_route() macro
In reply to Tomi ValkeinenFrom: Jacopo Mondi <[email protected]>
Add a for_each_active_route() macro to replace the repeated pattern of iterating on the active routes of a routing table.
--- .clang-format | 1 + drivers/media/v4l2-core/v4l2-subdev.c | 20 ++++++++++++++++++++ include/media/v4l2-subdev.h | 13 +++++++++++++ 3 files changed, 34 insertions(+)
.clang-format +1 -0
@@ -190,6 +190,7 @@ ForEachMacros: - 'for_each_active_dev_scope' - 'for_each_active_drhd_unit' - 'for_each_active_iommu' + - 'for_each_active_route' - 'for_each_aggr_pgid' - 'for_each_available_child_of_node' - 'for_each_bench'
drivers/media/v4l2-core/v4l2-subdev.c +20 -0
@@ -1222,6 +1222,26 @@ int v4l2_subdev_set_routing(struct v4l2_subdev *sd, }EXPORT_SYMBOL_GPL (v4l2_subdev_set_routing); +struct v4l2_subdev_route * +__v4l2_subdev_next_active_route (const struct v4l2_subdev_krouting *routing, +struct v4l2_subdev_route *route) +{ +if (route) + ++route; +else + route = &routing->routes[0 ]; + +for (; route < routing->routes + routing->num_routes; ++route) { +if (!(route->flags &V4L2_SUBDEV_ROUTE_FL_ACTIVE )) +continue ; + +return route; + } + +return NULL ; +} +EXPORT_SYMBOL_GPL (__v4l2_subdev_next_active_route); +#endif /* CONFIG_VIDEO_V4L2_SUBDEV_API */ #endif /* CONFIG_MEDIA_CONTROLLER */
include/media/v4l2-subdev.h +13 -0
@@ -1435,6 +1435,19 @@ int v4l2_subdev_set_routing(struct v4l2_subdev *sd,struct v4l2_subdev_state *state, const struct v4l2_subdev_krouting *routing); +struct v4l2_subdev_route * +__v4l2_subdev_next_active_route (const struct v4l2_subdev_krouting *routing, +struct v4l2_subdev_route *route); + +/** +* for_each_active_route - iterate on all active routes of a routing table +* @routing: The routing table +* @route: The route iterator +*/ +#define for_each_active_route (routing, route) \ + for ((route) = NULL; \ + ((route) = __v4l2_subdev_next_active_route((routing), (route)));) +#endif /* CONFIG_VIDEO_V4L2_SUBDEV_API */ #endif /* CONFIG_MEDIA_CONTROLLER */
Signature · 1 line
2.34.1
PATCH v16 10/20 media: Documentation: add multiplexed streams documentation
In reply to Tomi ValkeinenAdd documentation related to multiplexed streams.
--- .../driver-api/media/v4l2-subdev.rst | 8 + .../userspace-api/media/v4l/dev-subdev.rst | 138 ++++++++++++++++++ 2 files changed, 146 insertions(+)
Documentation/driver-api/media/v4l2-subdev.rst +8 -0
@@ -593,6 +593,14 @@ before calling v4l2_subdev_init_finalize(): This shares the driver's private mutex between the controls and the states. +Streams, multiplexed media pads and internal routing +---------------------------------------------------- + +A subdevice driver can implement support for multiplexed streams by setting +the V4L2_SUBDEV_FL_STREAMS subdev flag and implementing support for +centrally managed subdev active state, routing and stream based +configuration. + V4L2 sub-device functions and data structures ---------------------------------------------
Documentation/userspace-api/media/v4l/dev-subdev.rst +138 -0
141 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v16 11/20 media: subdev: add stream based configuration
In reply to Tomi ValkeinenAdd support to manage configurations (format, crop, compose) per stream, instead of per pad. This is accomplished with data structures that hold an array of all subdev's stream configurations.
The number of streams can vary at runtime based on routing. Every time the routing is changed, the stream configurations need to be re-initialized.
--- .../v4l/vidioc-subdev-enum-frame-interval.rst | 5 +- .../v4l/vidioc-subdev-enum-frame-size.rst | 5 +- .../v4l/vidioc-subdev-enum-mbus-code.rst | 5 +- .../media/v4l/vidioc-subdev-g-crop.rst | 5 +- .../media/v4l/vidioc-subdev-g-fmt.rst | 5 +- .../v4l/vidioc-subdev-g-frame-interval.rst | 5 +- .../media/v4l/vidioc-subdev-g-selection.rst | 5 +- drivers/media/v4l2-core/v4l2-subdev.c | 150 +++++++++++++++++- include/media/v4l2-subdev.h | 79 +++++++++ include/uapi/linux/v4l2-subdev.h | 28 +++- 10 files changed, 271 insertions(+), 21 deletions(-)
Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-interval.rst +4 -1
@@ -92,7 +92,10 @@ multiple pads of the same sub-device is not defined. - Frame intervals to be enumerated, from enum :ref:`v4l2_subdev_format_whence <v4l2-subdev-format-whence>`. * - __u32 - - ``reserved``\ [8] + - ``stream`` + - Stream identifier. + * - __u32 + - ``reserved``\ [7] - Reserved for future extensions. Applications and drivers must set the array to zero.
Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-size.rst +4 -1
@@ -97,7 +97,10 @@ information about try formats. - Frame sizes to be enumerated, from enum :ref:`v4l2_subdev_format_whence <v4l2-subdev-format-whence>`. * - __u32 - - ``reserved``\ [8] + - ``stream`` + - Stream identifier. + * - __u32 + - ``reserved``\ [7] - Reserved for future extensions. Applications and drivers must set the array to zero.
Documentation/userspace-api/media/v4l/vidioc-subdev-enum-mbus-code.rst +4 -1
@@ -73,7 +73,10 @@ information about the try formats. - ``flags`` - See :ref:`v4l2-subdev-mbus-code-flags` * - __u32 - - ``reserved``\ [7] + - ``stream`` + - Stream identifier. + * - __u32 + - ``reserved``\ [6] - Reserved for future extensions. Applications and drivers must set the array to zero.
Documentation/userspace-api/media/v4l/vidioc-subdev-g-crop.rst +4 -1
@@ -96,7 +96,10 @@ modified format should be as close as possible to the original request. - ``rect`` - Crop rectangle boundaries, in pixels. * - __u32 - - ``reserved``\ [8] + - ``stream`` + - Stream identifier. + * - __u32 + - ``reserved``\ [7] - Reserved for future extensions. Applications and drivers must set the array to zero.
Documentation/userspace-api/media/v4l/vidioc-subdev-g-fmt.rst +4 -1
@@ -102,7 +102,10 @@ should be as close as possible to the original request. - Definition of an image format, see :c:type:`v4l2_mbus_framefmt` for details. * - __u32 - - ``reserved``\ [8] + - ``stream`` + - Stream identifier. + * - __u32 + - ``reserved``\ [7] - Reserved for future extensions. Applications and drivers must set the array to zero.
Documentation/userspace-api/media/v4l/vidioc-subdev-g-frame-interval.rst +4 -1
@@ -90,7 +90,10 @@ the same sub-device is not defined. - ``interval`` - Period, in seconds, between consecutive video frames. * - __u32 - - ``reserved``\ [9] + - ``stream`` + - Stream identifier. + * - __u32 + - ``reserved``\ [8] - Reserved for future extensions. Applications and drivers must set the array to zero.
Documentation/userspace-api/media/v4l/vidioc-subdev-g-selection.rst +4 -1
@@ -94,7 +94,10 @@ Selection targets and flags are documented in - ``r`` - Selection rectangle, in pixels. * - __u32 - - ``reserved``\ [8] + - ``stream`` + - Stream identifier. + * - __u32 + - ``reserved``\ [7] - Reserved for future extensions. Applications and drivers must set the array to zero.
drivers/media/v4l2-core/v4l2-subdev.c +143 -7
223 lines not on this page · raw message
include/media/v4l2-subdev.h +79 -0
103 lines not on this page · raw message
include/uapi/linux/v4l2-subdev.h +21 -7
@@ -45,13 +45,15 @@ enum v4l2_subdev_format_whence {* @which: format type (from enum v4l2_subdev_format_whence) * @pad: pad number, as reported by the media API * @format: media bus format (format code and frame size) +* @stream: stream number, defined in subdev routing * @reserved: drivers and applications must zero this array */ struct v4l2_subdev_format {__u32 which;__u32 pad;struct v4l2_mbus_framefmt format; -__u32 reserved[8 ]; +__u32 stream; +__u32 reserved[7 ]; };/**
@@ -59,13 +61,15 @@ struct v4l2_subdev_format {* @which: format type (from enum v4l2_subdev_format_whence) * @pad: pad number, as reported by the media API * @rect: pad crop rectangle boundaries +* @stream: stream number, defined in subdev routing * @reserved: drivers and applications must zero this array */ struct v4l2_subdev_crop {__u32 which;__u32 pad;struct v4l2_rect rect; -__u32 reserved[8 ]; +__u32 stream; +__u32 reserved[7 ]; };#define V4L2_SUBDEV_MBUS_CODE_CSC_COLORSPACE 0x00000001
@@ -81,6 +85,7 @@ struct v4l2_subdev_crop {* @code: format code (MEDIA_BUS_FMT_ definitions) * @which: format type (from enum v4l2_subdev_format_whence) * @flags: flags set by the driver, (V4L2_SUBDEV_MBUS_CODE_*) +* @stream: stream number, defined in subdev routing * @reserved: drivers and applications must zero this array */ struct v4l2_subdev_mbus_code_enum {
@@ -89,7 +94,8 @@ struct v4l2_subdev_mbus_code_enum {__u32 code;__u32 which;__u32 flags; -__u32 reserved[7 ]; +__u32 stream; +__u32 reserved[6 ]; };/**
@@ -102,6 +108,7 @@ struct v4l2_subdev_mbus_code_enum {* @min_height: minimum frame height, in pixels * @max_height: maximum frame height, in pixels * @which: format type (from enum v4l2_subdev_format_whence) +* @stream: stream number, defined in subdev routing * @reserved: drivers and applications must zero this array */ struct v4l2_subdev_frame_size_enum {
@@ -113,19 +120,22 @@ struct v4l2_subdev_frame_size_enum {__u32 min_height;__u32 max_height;__u32 which; -__u32 reserved[8 ]; +__u32 stream; +__u32 reserved[7 ]; };/** * struct v4l2_subdev_frame_interval - Pad-level frame rate * @pad: pad number, as reported by the media API * @interval: frame interval in seconds +* @stream: stream number, defined in subdev routing * @reserved: drivers and applications must zero this array */ struct v4l2_subdev_frame_interval {__u32 pad;struct v4l2_fract interval; -__u32 reserved[9 ]; +__u32 stream; +__u32 reserved[8 ]; };/**
@@ -137,6 +147,7 @@ struct v4l2_subdev_frame_interval {* @height: frame height in pixels * @interval: frame interval in seconds * @which: format type (from enum v4l2_subdev_format_whence) +* @stream: stream number, defined in subdev routing * @reserved: drivers and applications must zero this array */ struct v4l2_subdev_frame_interval_enum {
@@ -147,7 +158,8 @@ struct v4l2_subdev_frame_interval_enum {__u32 height;struct v4l2_fract interval;__u32 which; -__u32 reserved[8 ]; +__u32 stream; +__u32 reserved[7 ]; };/**
@@ -159,6 +171,7 @@ struct v4l2_subdev_frame_interval_enum {* defined in v4l2-common.h; V4L2_SEL_TGT_* . * @flags: constraint flags, defined in v4l2-common.h; V4L2_SEL_FLAG_*. * @r: coordinates of the selection window +* @stream: stream number, defined in subdev routing * @reserved: for future use, set to zero for now * * Hardware may use multiple helper windows to process a video stream.
@@ -171,7 +184,8 @@ struct v4l2_subdev_selection {__u32 target;__u32 flags;struct v4l2_rect r; -__u32 reserved[8 ]; +__u32 stream; +__u32 reserved[7 ]; };/**
Signature · 1 line
2.34.1
PATCH v16 12/20 media: subdev: use streams in v4l2_subdev_link_validate()
In reply to Tomi ValkeinenUpdate v4l2_subdev_link_validate() to use routing and streams for validation.
Instead of just looking at the format on the pad on both ends of the link, the routing tables are used to collect all the streams going from the source to the sink over the link, and the streams' formats on both ends of the link are verified.
--- drivers/media/v4l2-core/v4l2-subdev.c | 185 +++++++++++++++++++++++--- 1 file changed, 165 insertions(+), 20 deletions(-)
drivers/media/v4l2-core/v4l2-subdev.c +165 -20
@@ -1024,7 +1024,7 @@ int v4l2_subdev_link_validate_default(struct v4l2_subdev *sd,EXPORT_SYMBOL_GPL (v4l2_subdev_link_validate_default);static int -v4l2_subdev_link_validate_get_format (struct media_pad *pad, +v4l2_subdev_link_validate_get_format (struct media_pad *pad,u32 stream,struct v4l2_subdev_format *fmt) {if (is_media_entity_v4l2_subdev (pad->entity)) {
@@ -1033,7 +1033,11 @@ v4l2_subdev_link_validate_get_format(struct media_pad *pad, fmt->which =V4L2_SUBDEV_FORMAT_ACTIVE ; fmt->pad = pad->index; -return v4l2_subdev_call_state_active (sd, pad, get_fmt, fmt); + fmt->stream = stream; + +return v4l2_subdev_call (sd, pad, get_fmt, +v4l2_subdev_get_locked_active_state (sd), + fmt); } WARN(pad->entity->function != MEDIA_ENT_F_IO_V4L,
@@ -1043,31 +1047,172 @@ v4l2_subdev_link_validate_get_format(struct media_pad *pad,return -EINVAL ; } +#if defined(CONFIG_VIDEO_V4L2_SUBDEV_API ) + +static void __v4l2_link_validate_get_streams (struct media_pad *pad, +u64 *streams_mask) +{ +struct v4l2_subdev_route *route; +struct v4l2_subdev_state *state; +struct v4l2_subdev *subdev; + + subdev =media_entity_to_v4l2_subdev (pad->entity); + + *streams_mask =0 ; + + state =v4l2_subdev_get_locked_active_state (subdev); +if (WARN_ON (!state)) +return ; + +for_each_active_route (&state->routing, route) { +u32 route_pad; +u32 route_stream; + +if (pad->flags &MEDIA_PAD_FL_SOURCE ) { + route_pad = route->source_pad; + route_stream = route->source_stream; + }else { + route_pad = route->sink_pad; + route_stream = route->sink_stream; + } + +if (route_pad != pad->index) +continue ; + + *streams_mask |=BIT_ULL (route_stream); + } +} + +#endif /* CONFIG_VIDEO_V4L2_SUBDEV_API */ + +static void v4l2_link_validate_get_streams (struct media_pad *pad, +u64 *streams_mask) +{ +struct v4l2_subdev *subdev =media_entity_to_v4l2_subdev (pad->entity); + +if (!(subdev->flags &V4L2_SUBDEV_FL_STREAMS )) { +/* Non-streams subdevs have an implicit stream 0 */ + *streams_mask =BIT_ULL (0 ); +return ; + } + +#if defined(CONFIG_VIDEO_V4L2_SUBDEV_API ) +__v4l2_link_validate_get_streams (pad, streams_mask); +#else +/* This shouldn't happen */ + *streams_mask =0 ; +#endif +} + +static int v4l2_subdev_link_validate_locked (struct media_link *link) +{ +struct v4l2_subdev *sink_subdev = +media_entity_to_v4l2_subdev (link->sink->entity); +struct device *dev = sink_subdev->entity.graph_obj.mdev->dev; +u64 source_streams_mask; +u64 sink_streams_mask; +u64 dangling_sink_streams; +u32 stream; +int ret; + +dev_dbg (dev,"validating link \"%s\":%u -> \"%s\":%u\n" , + link->source->entity->name, link->source->index, + link->sink->entity->name, link->sink->index); + +v4l2_link_validate_get_streams (link->source, &source_streams_mask); +v4l2_link_validate_get_streams (link->sink, &sink_streams_mask); + +/* +* It is ok to have more source streams than sink streams as extra +* source streams can just be ignored by the receiver, but having extra +* sink streams is an error as streams must have a source. +*/ + dangling_sink_streams = (source_streams_mask ^ sink_streams_mask) & + sink_streams_mask; +if (dangling_sink_streams) { +dev_err (dev,"Dangling sink streams: mask %#llx\n" , + dangling_sink_streams); +return -EINVAL ; + } + +/* Validate source and sink stream formats */ + +for (stream =0 ; stream <sizeof (sink_streams_mask) *8 ; ++stream) { +struct v4l2_subdev_format sink_fmt, source_fmt; + +if (!(sink_streams_mask &BIT_ULL (stream))) +continue ; + +dev_dbg (dev,"validating stream \"%s\":%u:%u -> \"%s\":%u:%u\n" , + link->source->entity->name, link->source->index, stream, + link->sink->entity->name, link->sink->index, stream); + + ret =v4l2_subdev_link_validate_get_format (link->source, stream, + &source_fmt); +if (ret <0 ) { +dev_dbg (dev, +"Failed to get format for \"%s\":%u:%u (but that's ok)\n" , + link->source->entity->name, link->source->index, + stream); +continue ; + } + + ret =v4l2_subdev_link_validate_get_format (link->sink, stream, + &sink_fmt); +if (ret <0 ) { +dev_dbg (dev, +"Failed to get format for \"%s\":%u:%u (but that's ok)\n" , + link->sink->entity->name, link->sink->index, + stream); +continue ; + } + +/* TODO: add stream number to link_validate() */ + ret =v4l2_subdev_call (sink_subdev, pad, link_validate, link, + &source_fmt, &sink_fmt); +if (!ret) +continue ; + +if (ret != -ENOIOCTLCMD ) +return ret; + + ret =v4l2_subdev_link_validate_default (sink_subdev, link, + &source_fmt, &sink_fmt); + +if (ret) +return ret; + } + +return 0 ; +} +int v4l2_subdev_link_validate (struct media_link *link) { -struct v4l2_subdev *sink; -struct v4l2_subdev_format sink_fmt, source_fmt; -int rval; +struct v4l2_subdev *source_sd, *sink_sd; +struct v4l2_subdev_state *source_state, *sink_state; +int ret; - rval =v4l2_subdev_link_validate_get_format ( - link->source, &source_fmt); -if (rval <0 ) -return 0 ; + sink_sd =media_entity_to_v4l2_subdev (link->sink->entity); + source_sd =media_entity_to_v4l2_subdev (link->source->entity); - rval =v4l2_subdev_link_validate_get_format ( - link->sink, &sink_fmt); -if (rval <0 ) -return 0 ; + sink_state =v4l2_subdev_get_unlocked_active_state (sink_sd); + source_state =v4l2_subdev_get_unlocked_active_state (source_sd); - sink =media_entity_to_v4l2_subdev (link->sink->entity); +if (sink_state) +v4l2_subdev_lock_state (sink_state); - rval =v4l2_subdev_call (sink, pad, link_validate, link, - &source_fmt, &sink_fmt); -if (rval != -ENOIOCTLCMD ) -return rval; +if (source_state) +v4l2_subdev_lock_state (source_state); -return v4l2_subdev_link_validate_default ( - sink, link, &source_fmt, &sink_fmt); + ret =v4l2_subdev_link_validate_locked (link); + +if (sink_state) +v4l2_subdev_unlock_state (sink_state); + +if (source_state) +v4l2_subdev_unlock_state (source_state); + +return ret; }EXPORT_SYMBOL_GPL (v4l2_subdev_link_validate);
Signature · 1 line
2.34.1
PATCH v16 13/20 media: subdev: add "opposite" stream helper funcs
In reply to Tomi ValkeinenAdd two helper functions to make dealing with streams easier:
v4l2_subdev_routing_find_opposite_end - given a routing table and a pad
+ stream, return the pad + stream on the opposite side of the subdev.
v4l2_subdev_state_get_opposite_stream_format - return a pointer to the format on the pad + stream on the opposite side from the given pad + stream.
--- drivers/media/v4l2-core/v4l2-subdev.c | 49 +++++++++++++++++++++++++++ include/media/v4l2-subdev.h | 36 ++++++++++++++++++++ 2 files changed, 85 insertions(+)
drivers/media/v4l2-core/v4l2-subdev.c +49 -0
@@ -1523,6 +1523,55 @@ v4l2_subdev_state_get_stream_compose(struct v4l2_subdev_state *state, }EXPORT_SYMBOL_GPL (v4l2_subdev_state_get_stream_compose); +int v4l2_subdev_routing_find_opposite_end (const struct v4l2_subdev_krouting *routing, +u32 pad,u32 stream,u32 *other_pad, +u32 *other_stream) +{ +unsigned int i; + +for (i =0 ; i < routing->num_routes; ++i) { +struct v4l2_subdev_route *route = &routing->routes[i]; + +if (route->source_pad == pad && + route->source_stream == stream) { +if (other_pad) + *other_pad = route->sink_pad; +if (other_stream) + *other_stream = route->sink_stream; +return 0 ; + } + +if (route->sink_pad == pad && route->sink_stream == stream) { +if (other_pad) + *other_pad = route->source_pad; +if (other_stream) + *other_stream = route->source_stream; +return 0 ; + } + } + +return -EINVAL ; +} +EXPORT_SYMBOL_GPL (v4l2_subdev_routing_find_opposite_end); + +struct v4l2_mbus_framefmt * +v4l2_subdev_state_get_opposite_stream_format (struct v4l2_subdev_state *state, +u32 pad,u32 stream) +{ +u32 other_pad, other_stream; +int ret; + + ret =v4l2_subdev_routing_find_opposite_end (&state->routing, + pad, stream, + &other_pad, &other_stream); +if (ret) +return NULL ; + +return v4l2_subdev_state_get_stream_format (state, other_pad, + other_stream); +} +EXPORT_SYMBOL_GPL (v4l2_subdev_state_get_opposite_stream_format); +#endif /* CONFIG_VIDEO_V4L2_SUBDEV_API */ #endif /* CONFIG_MEDIA_CONTROLLER */
include/media/v4l2-subdev.h +36 -0
@@ -1527,6 +1527,42 @@ struct v4l2_rect * v4l2_subdev_state_get_stream_compose(struct v4l2_subdev_state *state, unsigned int pad, u32 stream); +/** +* v4l2_subdev_routing_find_opposite_end() - Find the opposite stream +* @routing: routing used to find the opposite side +* @pad: pad id +* @stream: stream id +* @other_pad: pointer used to return the opposite pad +* @other_stream: pointer used to return the opposite stream +* +* This function uses the routing table to find the pad + stream which is +* opposite the given pad + stream. +* +* @other_pad and/or @other_stream can be NULL if the caller does not need the +* value. +* +* Returns 0 on success, or -EINVAL if no matching route is found. +*/ +int v4l2_subdev_routing_find_opposite_end (const struct v4l2_subdev_krouting *routing, +u32 pad,u32 stream,u32 *other_pad, +u32 *other_stream); + +/** +* v4l2_subdev_state_get_opposite_stream_format() - Get pointer to opposite +* stream format +* @state: subdevice state +* @pad: pad id +* @stream: stream id +* +* This returns a pointer to &struct v4l2_mbus_framefmt for the pad + stream +* that is opposite the given pad + stream in the subdev state. +* +* If the state does not contain the given pad + stream, NULL is returned. +*/ +struct v4l2_mbus_framefmt * +v4l2_subdev_state_get_opposite_stream_format (struct v4l2_subdev_state *state, +u32 pad,u32 stream); +#endif /* CONFIG_VIDEO_V4L2_SUBDEV_API */ #endif /* CONFIG_MEDIA_CONTROLLER */
Signature · 1 line
2.34.1
PATCH v16 14/20 media: subdev: add streams to v4l2_subdev_get_fmt() helper function
In reply to Tomi ValkeinenAdd streams support to v4l2_subdev_get_fmt() helper function. Subdev drivers that do not need to do anything special in their get_fmt op can use this helper directly for v4l2_subdev_pad_ops.get_fmt.
--- drivers/media/v4l2-core/v4l2-subdev.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
drivers/media/v4l2-core/v4l2-subdev.c +7 -3
@@ -1389,10 +1389,14 @@ int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state, {struct v4l2_mbus_framefmt *fmt; -if (format->pad >= sd->entity.num_pads) -return -EINVAL ; +if (sd->flags &V4L2_SUBDEV_FL_STREAMS ) + fmt =v4l2_subdev_state_get_stream_format (state, format->pad, + format->stream); +else if (format->pad < sd->entity.num_pads && format->stream ==0 ) + fmt =v4l2_subdev_get_pad_format (sd, state, format->pad); +else + fmt =NULL ; - fmt =v4l2_subdev_get_pad_format (sd, state, format->pad);if (!fmt)return -EINVAL ;
Signature · 1 line
2.34.1
PATCH v16 15/20 media: subdev: add v4l2_subdev_set_routing_with_fmt() helper
In reply to Tomi Valkeinenv4l2_subdev_set_routing_with_fmt() is the same as v4l2_subdev_set_routing(), but additionally initializes all the streams with the given format.
--- drivers/media/v4l2-core/v4l2-subdev.c | 22 ++++++++++++++++++++++ include/media/v4l2-subdev.h | 16 ++++++++++++++++ 2 files changed, 38 insertions(+)
drivers/media/v4l2-core/v4l2-subdev.c +22 -0
@@ -1464,6 +1464,28 @@ __v4l2_subdev_next_active_route(const struct v4l2_subdev_krouting *routing, }EXPORT_SYMBOL_GPL (__v4l2_subdev_next_active_route); +int v4l2_subdev_set_routing_with_fmt (struct v4l2_subdev *sd, +struct v4l2_subdev_state *state, +struct v4l2_subdev_krouting *routing, +const struct v4l2_mbus_framefmt *fmt) +{ +struct v4l2_subdev_stream_configs *stream_configs; +unsigned int i; +int ret; + + ret =v4l2_subdev_set_routing (sd, state, routing); +if (ret) +return ret; + + stream_configs = &state->stream_configs; + +for (i =0 ; i < stream_configs->num_configs; ++i) + stream_configs->configs[i].fmt = *fmt; + +return 0 ; +} +EXPORT_SYMBOL_GPL (v4l2_subdev_set_routing_with_fmt); +struct v4l2_mbus_framefmt *v4l2_subdev_state_get_stream_format (struct v4l2_subdev_state *state,unsigned int pad,u32 stream)
include/media/v4l2-subdev.h +16 -0
@@ -1481,6 +1481,22 @@ __v4l2_subdev_next_active_route(const struct v4l2_subdev_krouting *routing,for ((route) =NULL ; \ ((route) =__v4l2_subdev_next_active_route ((routing), (route)));) +/** +* v4l2_subdev_set_routing_with_fmt() - Set given routing and format to subdev +* state +* @sd: The subdevice +* @state: The subdevice state +* @routing: Routing that will be copied to subdev state +* @fmt: Format used to initialize all the streams +* +* This is the same as v4l2_subdev_set_routing, but additionally initializes +* all the streams using the given format. +*/ +int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd, + struct v4l2_subdev_state *state, + struct v4l2_subdev_krouting *routing, + const struct v4l2_mbus_framefmt *fmt); +/** * v4l2_subdev_state_get_stream_format() - Get pointer to a stream format * @state: subdevice state
Signature · 1 line
2.34.1
PATCH v16 16/20 media: subdev: add v4l2_subdev_routing_validate() helper
In reply to Tomi ValkeinenFrom: Laurent Pinchart <[email protected]>
Add a v4l2_subdev_routing_validate() helper for verifying routing for common cases like only allowing non-overlapping 1-to-1 streams.
--- drivers/media/v4l2-core/v4l2-subdev.c | 102 ++++++++++++++++++++++++++ include/media/v4l2-subdev.h | 39 ++++++++++ 2 files changed, 141 insertions(+)
drivers/media/v4l2-core/v4l2-subdev.c +102 -0
@@ -1598,6 +1598,108 @@ v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state, }EXPORT_SYMBOL_GPL (v4l2_subdev_state_get_opposite_stream_format); +int v4l2_subdev_routing_validate (struct v4l2_subdev *sd, +const struct v4l2_subdev_krouting *routing, +enum v4l2_subdev_routing_restriction disallow) +{ +u32 *remote_pads =NULL ; +unsigned int i, j; +int ret = -EINVAL ; + +if (disallow &V4L2_SUBDEV_ROUTING_NO_STREAM_MIX ) { + remote_pads =kcalloc (sd->entity.num_pads,sizeof (*remote_pads), +GFP_KERNEL ); +if (!remote_pads) +return -ENOMEM ; + +for (i =0 ; i < sd->entity.num_pads; ++i) + remote_pads[i] =U32_MAX ; + } + +for (i =0 ; i < routing->num_routes; ++i) { +const struct v4l2_subdev_route *route = &routing->routes[i]; + +/* Validate the sink and source pad numbers. */ +if (route->sink_pad >= sd->entity.num_pads || + !(sd->entity.pads[route->sink_pad].flags &MEDIA_PAD_FL_SINK )) { +dev_dbg (sd->dev,"route %u sink (%u) is not a sink pad\n" , + i, route->sink_pad); + goto out; + } + +if (route->source_pad >= sd->entity.num_pads || + !(sd->entity.pads[route->source_pad].flags &MEDIA_PAD_FL_SOURCE )) { +dev_dbg (sd->dev,"route %u source (%u) is not a source pad\n" , + i, route->source_pad); + goto out; + } + +/* +* V4L2_SUBDEV_ROUTING_NO_STREAM_MIX: Streams on the same pad +* may not be routed to streams on different pads. +*/ +if (disallow &V4L2_SUBDEV_ROUTING_NO_STREAM_MIX ) { +if (remote_pads[route->sink_pad] !=U32_MAX && + remote_pads[route->sink_pad] != route->source_pad) { +dev_dbg (sd->dev, +"route %u attempts to mix %s streams\n" , + i,"sink" ); + goto out; + } + +if (remote_pads[route->source_pad] !=U32_MAX && + remote_pads[route->source_pad] != route->sink_pad) { +dev_dbg (sd->dev, +"route %u attempts to mix %s streams\n" , + i,"source" ); + goto out; + } + + remote_pads[route->sink_pad] = route->source_pad; + remote_pads[route->source_pad] = route->sink_pad; + } + +for (j = i +1 ; j < routing->num_routes; ++j) { +const struct v4l2_subdev_route *r = &routing->routes[j]; + +/* +* V4L2_SUBDEV_ROUTING_NO_1_TO_N: No two routes can +* originate from the same (sink) stream. +*/ +if ((disallow &V4L2_SUBDEV_ROUTING_NO_1_TO_N ) && + route->sink_pad == r->sink_pad && + route->sink_stream == r->sink_stream) { +dev_dbg (sd->dev, +"routes %u and %u originate from same sink (%u/%u)\n" , + i, j, route->sink_pad, + route->sink_stream); + goto out; + } + +/* +* V4L2_SUBDEV_ROUTING_NO_N_TO_1: No two routes can end +* at the same (source) stream. +*/ +if ((disallow &V4L2_SUBDEV_ROUTING_NO_N_TO_1 ) && + route->source_pad == r->source_pad && + route->source_stream == r->source_stream) { +dev_dbg (sd->dev, +"routes %u and %u end at same source (%u/%u)\n" , + i, j, route->source_pad, + route->source_stream); + goto out; + } + } + } + + ret =0 ; + +out: +kfree (remote_pads); +return ret; +} +EXPORT_SYMBOL_GPL (v4l2_subdev_routing_validate); +#endif /* CONFIG_VIDEO_V4L2_SUBDEV_API */ #endif /* CONFIG_MEDIA_CONTROLLER */
include/media/v4l2-subdev.h +39 -0
@@ -1579,6 +1579,45 @@ struct v4l2_mbus_framefmt * v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state, u32 pad, u32 stream); +/** +* enum v4l2_subdev_routing_restriction - Subdevice internal routing restrictions +* +* @V4L2_SUBDEV_ROUTING_NO_1_TO_N: +* an input stream may not be routed to multiple output streams (stream +* duplication) +* @V4L2_SUBDEV_ROUTING_NO_N_TO_1: +* multiple input streams may not be routed to the same output stream +* (stream merging) +* @V4L2_SUBDEV_ROUTING_NO_STREAM_MIX: +* streams on the same pad may not be routed to streams on different pads +* @V4L2_SUBDEV_ROUTING_ONLY_1_TO_1: +* only non-overlapping 1-to-1 stream routing is allowed (a combination of +* @V4L2_SUBDEV_ROUTING_NO_1_TO_N and @V4L2_SUBDEV_ROUTING_NO_N_TO_1) +*/ +enum v4l2_subdev_routing_restriction { +V4L2_SUBDEV_ROUTING_NO_1_TO_N =BIT (0 ), +V4L2_SUBDEV_ROUTING_NO_N_TO_1 =BIT (1 ), +V4L2_SUBDEV_ROUTING_NO_STREAM_MIX =BIT (2 ), +V4L2_SUBDEV_ROUTING_ONLY_1_TO_1 = +V4L2_SUBDEV_ROUTING_NO_1_TO_N | +V4L2_SUBDEV_ROUTING_NO_N_TO_1 , +}; + +/** +* v4l2_subdev_routing_validate() - Verify that routes comply with driver +* constraints +* @sd: The subdevice +* @routing: Routing to verify +* @disallow: Restrictions on routes +* +* This verifies that the given routing complies with the @disallow constraints. +* +* Returns 0 on success, error value otherwise. +*/ +int v4l2_subdev_routing_validate (struct v4l2_subdev *sd, +const struct v4l2_subdev_krouting *routing, +enum v4l2_subdev_routing_restriction disallow); +#endif /* CONFIG_VIDEO_V4L2_SUBDEV_API */ #endif /* CONFIG_MEDIA_CONTROLLER */
Signature · 1 line
2.34.1
PATCH v16 17/20 media: v4l2-subdev: Add v4l2_subdev_state_xlate_streams() helper
In reply to Tomi ValkeinenFrom: Laurent Pinchart <[email protected]>
Add a helper function to translate streams between two pads of a subdev, using the subdev's internal routing table.
--- drivers/media/v4l2-core/v4l2-subdev.c | 26 ++++++++++++++++++++++++++ include/media/v4l2-subdev.h | 23 +++++++++++++++++++++++ 2 files changed, 49 insertions(+)
drivers/media/v4l2-core/v4l2-subdev.c +26 -0
@@ -1598,6 +1598,32 @@ v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state, }EXPORT_SYMBOL_GPL (v4l2_subdev_state_get_opposite_stream_format); +u64 v4l2_subdev_state_xlate_streams (const struct v4l2_subdev_state *state, +u32 pad0,u32 pad1,u64 *streams) +{ +const struct v4l2_subdev_krouting *routing = &state->routing; +struct v4l2_subdev_route *route; +u64 streams0 =0 ; +u64 streams1 =0 ; + +for_each_active_route (routing, route) { +if (route->sink_pad == pad0 && route->source_pad == pad1 && + (*streams &BIT_ULL (route->sink_stream))) { + streams0 |=BIT_ULL (route->sink_stream); + streams1 |=BIT_ULL (route->source_stream); + } +if (route->source_pad == pad0 && route->sink_pad == pad1 && + (*streams &BIT_ULL (route->source_stream))) { + streams0 |=BIT_ULL (route->source_stream); + streams1 |=BIT_ULL (route->sink_stream); + } + } + + *streams = streams0; +return streams1; +} +EXPORT_SYMBOL_GPL (v4l2_subdev_state_xlate_streams); +int v4l2_subdev_routing_validate (struct v4l2_subdev *sd,const struct v4l2_subdev_krouting *routing,enum v4l2_subdev_routing_restriction disallow)
include/media/v4l2-subdev.h +23 -0
@@ -1579,6 +1579,29 @@ struct v4l2_mbus_framefmt * v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state, u32 pad, u32 stream); +/** +* v4l2_subdev_state_xlate_streams() - Translate streams from one pad to another +* +* @state: Subdevice state +* @pad0: The first pad +* @pad1: The second pad +* @streams: Streams bitmask on the first pad +* +* Streams on sink pads of a subdev are routed to source pads as expressed in +* the subdev state routing table. Stream numbers don't necessarily match on +* the sink and source side of a route. This function translates stream numbers +* on @pad0, expressed as a bitmask in @streams, to the corresponding streams +* on @pad1 using the routing table from the @state. It returns the stream mask +* on @pad1, and updates @streams with the streams that have been found in the +* routing table. +* +* @pad0 and @pad1 must be a sink and a source, in any order. +* +* Return: The bitmask of streams of @pad1 that are routed to @streams on @pad0. +*/ +u64 v4l2_subdev_state_xlate_streams (const struct v4l2_subdev_state *state, +u32 pad0,u32 pad1,u64 *streams); +/** * enum v4l2_subdev_routing_restriction - Subdevice internal routing restrictions *
Signature · 1 line
2.34.1
PATCH v16 18/20 media: v4l2-subdev: Add subdev .(enable|disable)_streams() operations
In reply to Tomi ValkeinenFrom: Laurent Pinchart <[email protected]>
Add two new subdev pad operations, .enable_streams() and .disable_streams(), to allow control of individual streams per pad. This is a superset of what the video .s_stream() operation implements.
To help with handling of backward compatibility, add two wrapper functions around those operations, and require their usage in drivers.
--- drivers/media/v4l2-core/v4l2-subdev.c | 224 ++++++++++++++++++++++++++ include/media/v4l2-subdev.h | 85 ++++++++++ 2 files changed, 309 insertions(+)
drivers/media/v4l2-core/v4l2-subdev.c +224 -0
@@ -1726,6 +1726,230 @@ int v4l2_subdev_routing_validate(struct v4l2_subdev *sd, }EXPORT_SYMBOL_GPL (v4l2_subdev_routing_validate); +static int v4l2_subdev_enable_streams_fallback (struct v4l2_subdev *sd,u32 pad, +u64 streams_mask) +{ +struct device *dev = sd->entity.graph_obj.mdev->dev; +unsigned int i; +int ret; + +/* +* The subdev doesn't implement pad-based stream enable, fall back +* on the .s_stream() operation. This can only be done for subdevs that +* have a single source pad, as sd->enabled_streams is global to the +* subdev. +*/ +if (!(sd->entity.pads[pad].flags &MEDIA_PAD_FL_SOURCE )) +return -EOPNOTSUPP ; + +for (i =0 ; i < sd->entity.num_pads; ++i) { +if (i != pad && sd->entity.pads[i].flags &MEDIA_PAD_FL_SOURCE ) +return -EOPNOTSUPP ; + } + +if (sd->enabled_streams & streams_mask) { +dev_dbg (dev,"set of streams %#llx already enabled on %s:%u\n" , + streams_mask, sd->entity.name, pad); +return -EALREADY ; + } + +/* Start streaming when the first streams are enabled. */ +if (!sd->enabled_streams) { + ret =v4l2_subdev_call (sd, video, s_stream,1 ); +if (ret) +return ret; + } + + sd->enabled_streams |= streams_mask; + +return 0 ; +} + +int v4l2_subdev_enable_streams (struct v4l2_subdev *sd,u32 pad, +u64 streams_mask) +{ +struct device *dev = sd->entity.graph_obj.mdev->dev; +struct v4l2_subdev_state *state; +u64 found_streams =0 ; +unsigned int i; +int ret; + +/* A few basic sanity checks first. */ +if (pad >= sd->entity.num_pads) +return -EINVAL ; + +if (!streams_mask) +return 0 ; + +/* Fallback on .s_stream() if .enable_streams() isn't available. */ +if (!sd->ops->pad || !sd->ops->pad->enable_streams) +return v4l2_subdev_enable_streams_fallback (sd, pad, + streams_mask); + + state =v4l2_subdev_lock_and_get_active_state (sd); + +/* +* Verify that the requested streams exist and that they are not +* already enabled. +*/ +for (i =0 ; i < state->stream_configs.num_configs; ++i) { +struct v4l2_subdev_stream_config *cfg = + &state->stream_configs.configs[i]; + +if (cfg->pad != pad || !(streams_mask &BIT_ULL (cfg->stream))) +continue ; + + found_streams |=BIT_ULL (cfg->stream); + +if (cfg->enabled) { +dev_dbg (dev,"stream %u already enabled on %s:%u\n" , + cfg->stream, sd->entity.name, pad); + ret = -EALREADY ; + goto done; + } + } + +if (found_streams != streams_mask) { +dev_dbg (dev,"streams 0x%llx not found on %s:%u\n" , + streams_mask & ~found_streams, sd->entity.name, pad); + ret = -EINVAL ; + goto done; + } + +/* Call the .enable_streams() operation. */ + ret =v4l2_subdev_call (sd, pad, enable_streams, state, pad, + streams_mask); +if (ret) + goto done; + +/* Mark the streams as enabled. */ +for (i =0 ; i < state->stream_configs.num_configs; ++i) { +struct v4l2_subdev_stream_config *cfg = + &state->stream_configs.configs[i]; + +if (cfg->pad == pad && (streams_mask &BIT_ULL (cfg->stream))) + cfg->enabled = true; + } + +done: +v4l2_subdev_unlock_state (state); + +return ret; +} +EXPORT_SYMBOL_GPL (v4l2_subdev_enable_streams); + +static int v4l2_subdev_disable_streams_fallback (struct v4l2_subdev *sd,u32 pad, +u64 streams_mask) +{ +struct device *dev = sd->entity.graph_obj.mdev->dev; +unsigned int i; +int ret; + +/* +* If the subdev doesn't implement pad-based stream enable, fall back +* on the .s_stream() operation. This can only be done for subdevs that +* have a single source pad, as sd->enabled_streams is global to the +* subdev. +*/ +if (!(sd->entity.pads[pad].flags &MEDIA_PAD_FL_SOURCE )) +return -EOPNOTSUPP ; + +for (i =0 ; i < sd->entity.num_pads; ++i) { +if (i != pad && sd->entity.pads[i].flags &MEDIA_PAD_FL_SOURCE ) +return -EOPNOTSUPP ; + } + +if ((sd->enabled_streams & streams_mask) != streams_mask) { +dev_dbg (dev,"set of streams %#llx already disabled on %s:%u\n" , + streams_mask, sd->entity.name, pad); +return -EALREADY ; + } + +/* Stop streaming when the last streams are disabled. */ +if (!(sd->enabled_streams & ~streams_mask)) { + ret =v4l2_subdev_call (sd, video, s_stream,0 ); +if (ret) +return ret; + } + + sd->enabled_streams &= ~streams_mask; + +return 0 ; +} + +int v4l2_subdev_disable_streams (struct v4l2_subdev *sd,u32 pad, +u64 streams_mask) +{ +struct device *dev = sd->entity.graph_obj.mdev->dev; +struct v4l2_subdev_state *state; +u64 found_streams =0 ; +unsigned int i; +int ret; + +/* A few basic sanity checks first. */ +if (pad >= sd->entity.num_pads) +return -EINVAL ; + +if (!streams_mask) +return 0 ; + +/* Fallback on .s_stream() if .disable_streams() isn't available. */ +if (!sd->ops->pad || !sd->ops->pad->disable_streams) +return v4l2_subdev_disable_streams_fallback (sd, pad, + streams_mask); + + state =v4l2_subdev_lock_and_get_active_state (sd); + +/* +* Verify that the requested streams exist and that they are not +* already disabled. +*/ +for (i =0 ; i < state->stream_configs.num_configs; ++i) { +struct v4l2_subdev_stream_config *cfg = + &state->stream_configs.configs[i]; + +if (cfg->pad != pad || !(streams_mask &BIT_ULL (cfg->stream))) +continue ; + + found_streams |=BIT_ULL (cfg->stream); + +if (!cfg->enabled) { +dev_dbg (dev,"stream %u already disabled on %s:%u\n" , + cfg->stream, sd->entity.name, pad); + ret = -EALREADY ; + goto done; + } + } + +if (found_streams != streams_mask) { +dev_dbg (dev,"streams 0x%llx not found on %s:%u\n" , + streams_mask & ~found_streams, sd->entity.name, pad); + ret = -EINVAL ; + goto done; + } + +/* Call the .disable_streams() operation. */ + ret =v4l2_subdev_call (sd, pad, disable_streams, state, pad, + streams_mask); +if (ret) + goto done; + +/* Mark the streams as disabled. */ +for (i =0 ; i < state->stream_configs.num_configs; ++i) { +struct v4l2_subdev_stream_config *cfg = + &state->stream_configs.configs[i]; + +if (cfg->pad == pad && (streams_mask &BIT_ULL (cfg->stream))) + cfg->enabled = false; + } + +done: +v4l2_subdev_unlock_state (state); + +return ret; +} +EXPORT_SYMBOL_GPL (v4l2_subdev_disable_streams); +#endif /* CONFIG_VIDEO_V4L2_SUBDEV_API */ #endif /* CONFIG_MEDIA_CONTROLLER */
include/media/v4l2-subdev.h +85 -0
127 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v16 19/20 media: v4l2-subdev: Add v4l2_subdev_s_stream_helper() function
In reply to Tomi ValkeinenFrom: Laurent Pinchart <[email protected]>
The v4l2_subdev_s_stream_helper() helper can be used by subdevs that implement the stream-aware .enable_streams() and .disable_streams() operations to implement .s_stream(). This is limited to subdevs that have a single source pad.
--- drivers/media/v4l2-core/v4l2-subdev.c | 40 +++++++++++++++++++++++++++ include/media/v4l2-subdev.h | 17 ++++++++++++ 2 files changed, 57 insertions(+)
drivers/media/v4l2-core/v4l2-subdev.c +40 -0
@@ -1950,6 +1950,46 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad, }EXPORT_SYMBOL_GPL (v4l2_subdev_disable_streams); +int v4l2_subdev_s_stream_helper (struct v4l2_subdev *sd,int enable) +{ +struct v4l2_subdev_state *state; +struct v4l2_subdev_route *route; +struct media_pad *pad; +u64 source_mask =0 ; +int pad_index =-1 ; + +/* +* Find the source pad. This helper is meant for subdevs that have a +* single source pad, so failures shouldn't happen, but catch them +* loudly nonetheless as they indicate a driver bug. +*/ +media_entity_for_each_pad (&sd->entity, pad) { +if (pad->flags &MEDIA_PAD_FL_SOURCE ) { + pad_index = pad->index; +break ; + } + } + +if (WARN_ON (pad_index ==-1 )) +return -EINVAL ; + +/* +* As there's a single source pad, just collect all the source streams. +*/ + state =v4l2_subdev_lock_and_get_active_state (sd); + +for_each_active_route (&state->routing, route) + source_mask |=BIT_ULL (route->source_stream); + +v4l2_subdev_unlock_state (state); + +if (enable) +return v4l2_subdev_enable_streams (sd, pad_index, source_mask); +else +return v4l2_subdev_disable_streams (sd, pad_index, source_mask); +} +EXPORT_SYMBOL_GPL (v4l2_subdev_s_stream_helper); +#endif /* CONFIG_VIDEO_V4L2_SUBDEV_API */ #endif /* CONFIG_MEDIA_CONTROLLER */
include/media/v4l2-subdev.h +17 -0
@@ -1726,6 +1726,23 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,int v4l2_subdev_disable_streams (struct v4l2_subdev *sd,u32 pad,u64 streams_mask); +/** +* v4l2_subdev_s_stream_helper() - Helper to implement the subdev s_stream +* operation using enable_streams and disable_streams +* @sd: The subdevice +* @enable: Enable or disable streaming +* +* Subdevice drivers that implement the streams-aware +* &v4l2_subdev_pad_ops.enable_streams and &v4l2_subdev_pad_ops.disable_streams +* operations can use this helper to implement the legacy +* &v4l2_subdev_video_ops.s_stream operation. +* +* This helper can only be used by subdevs that have a single source pad. +* +* Return: 0 on success, or a negative error code otherwise. +*/ +int v4l2_subdev_s_stream_helper (struct v4l2_subdev *sd,int enable); +#endif /* CONFIG_VIDEO_V4L2_SUBDEV_API */ #endif /* CONFIG_MEDIA_CONTROLLER */
Signature · 1 line
2.34.1
PATCH v16 20/20 media: Add stream to frame descriptor
In reply to Tomi ValkeinenFrom: Sakari Ailus <[email protected]>
The stream field identifies the stream this frame descriptor applies to in routing configuration across a multiplexed link.
--- include/media/v4l2-subdev.h | 2 ++ 1 file changed, 2 insertions(+)
include/media/v4l2-subdev.h +2 -0
@@ -342,6 +342,7 @@ enum v4l2_mbus_frame_desc_flags {* struct v4l2_mbus_frame_desc_entry - media bus frame description structure * * @flags: bitmask flags, as defined by &enum v4l2_mbus_frame_desc_flags. +* @stream: stream in routing configuration * @pixelcode: media bus pixel code, valid if @flags * %FRAME_DESC_FL_BLOB is not set. * @length: number of octets per frame, valid if @flags
@@ -351,6 +352,7 @@ enum v4l2_mbus_frame_desc_flags {*/ struct v4l2_mbus_frame_desc_entry {enum v4l2_mbus_frame_desc_flags flags; +u32 stream;u32 pixelcode;u32 length;union {
Signature · 1 line
2.34.1
PATCH 1/1 media: Documentation: Update documentation for streams
In reply to Tomi ValkeinenDocument how streams interacts with formats and selections.
Update documentation in respect to what is allowed, in particular streams are only supported via full routes, source-only routes are not supported right now.
The centerpiece of the API additions are streams. Albeit routes are configured via S_ROUTING IOCTL that also declares streams, it is streams that are accessed through other APIs. Thus refer to streams instead of routes in documentation.
---
Hi folks,
This replaces my earlier documetation patch. The commit message and the subject have changed and the content reflects this. Largely this means removing a few features of the interface --- for now.
The intent is to be able to merge this very soon, thus those portions that are still debated have been dropped (no more dependencies between streams, for instance).
.../userspace-api/media/v4l/dev-subdev.rst | 121 +++++++++--------- 1 file changed, 58 insertions(+), 63 deletions(-)
Documentation/userspace-api/media/v4l/dev-subdev.rst +58 -63
@@ -406,6 +406,8 @@ pixel array is not rectangular but cross-shaped or round. The maximum size may also be smaller than the BOUNDS rectangle. +.. _format-propagation: + Order of configuration and format propagation ---------------------------------------------
@@ -507,12 +509,12 @@ source pads. Streams, multiplexed media pads and internal routing ---------------------------------------------------- -Commonly V4L2 subdevices support only separate video streams, that is, only a -single stream can pass through a media link and a media pad. Thus each pad -contains a format configuration for that single stream. In some cases a subdev -can do stream processing and split a stream into two or compose two streams -into one, but the inputs and outputs for the subdev are still a single stream -per pad. +Simple V4L2 subdevices do not support multiple, unrelated video streams, +and only a single stream can pass through a media link and a media pad. +Thus each pad contains a format and selection configuration for that +single stream. A subdev can do stream processing and split a stream into +two or compose two streams into one, but the inputs and outputs for the +subdev are still a single stream per pad. Some hardware, e.g. MIPI CSI-2, support multiplexed streams, that is, multiple data streams are transmitted on the same bus, which is represented by a media
@@ -539,14 +541,35 @@ streams from one end of the link to the other, and subdevices have routing tables which describe how the incoming streams from sink pads are routed to the source pads. -A stream ID (often just "stream") is a media link-local identifier for a stream. -In other words, a particular stream ID must exist on both sides of a media +A stream ID is a media pad-local identifier for a stream. Streams IDs of +the same stream must be equal on both ends of a link. In other words, +a particular stream ID must exist on both sides of a media link, but another stream ID can be used for the same stream at the other side -of the subdevice. +of the subdevice. The same stream ID is used to refer to the stream on +both pads of the link on all ioctls operating on pads. + +A stream at a specific point in the media pipeline is identified by the +sub-devdev and a (pad, stream) pair. For subdevices that do not support +multiplexed streams the 'stream' field is always 0. + +Interaction between routes, streams, formats and selections +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The addition of streams to the V4L2 sub-device interface moves the sub-device +formats and selections from pads to (pad, stream) pairs. Besides the +usual pad, also the stream ID needs to be provided for setting formats and +selections. The order of configuring formats and selections along a stream is +the same as without streams (see :ref:`format-propagation`). + +Instead of the sub-device wide merging of streams from all source pads +towards all sink pads, data flows for each route are separate from each +other. Any number of routes from streams on sink pads towards streams on +source pads is allowed, to the extent supported by drivers. For every +stream on a sink pad, however, only a single route is allowed. -A stream at a specific point in the media pipeline is identified with the -subdev and a (pad, stream) pair. For subdevices that do not support -multiplexed streams the 'stream' is always 0. +Any configurations of a stream within a pad, such as format or selections, +are independent of similar configurations on other streams. This is +subject to change in the future. Configuring streams ^^^^^^^^^^^^^^^^^^^
@@ -560,34 +583,37 @@ There are three steps in configuring the streams: 1) Set up links. Connect the pads between subdevices using the :ref:`Media Controller API <media_controller>` -2) Routing. The routing table for the subdevice must be set with +2) Streams. Streams are declared and their routing is configured by +setting the routing table for the subdevice must be set with :ref:`VIDIOC_SUBDEV_S_ROUTING <VIDIOC_SUBDEV_G_ROUTING>` ioctl. Note that -setting the routing table will reset all the stream configurations in a media -entity. +setting the routing table will reset formats and selections in the +sub-device to default values. -3) Configure streams. Each route endpoint must be configured -with :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>`. +3) Configure formats and selections. Formats and selections of each stream +are configured separately as documented for plain subdevices in +:ref:`<format-propagation>`. The stream ID is set to the same stream ID +associated with either sink or source pads of routes configured using the +:ref:`VIDIOC_SUBDEV_S_ROUTING <VIDIOC_SUBDEV_G_ROUTING>` ioctl. Multiplexed streams setup example ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ A simple example of a multiplexed stream setup might be as follows: -- Two identical sensors (Sensor A and Sensor B). Each sensor has a single source - pad (pad 0) which carries two streams, pixel data stream and metadata - stream. +- Two identical sensors (Sensor A and Sensor B). The sensors produce image + data only and hence do not need specific support for streams. - Multiplexer bridge (Bridge). The bridge has two sink pads, connected to the - sensors (pads 0, 1), and one source pad (pad 2), which outputs all 4 - streams. + sensors (pads 0, 1), and one source pad (pad 2), which outputs both of + the streams. - Receiver in the SoC (Receiver). The receiver has a single sink pad (pad 0), connected to the bridge, and four source pads (pads 1-4), going to the DMA - engine. The receiver demultiplexes the incoming streams to the four source + engine. The receiver demultiplexes the incoming streams to two the source pads. - Four DMA Engines in the SoC (DMA Engine). Each DMA engine is connected to a - single source pad in the receiver. + single source pad in the receive via a link, two of which are active. The sensors, the bridge and the receiver are modeled as V4L2 subdevices, exposed to userspace via /dev/v4l-subdevX device nodes. The DMA engines are
@@ -599,23 +625,7 @@ To configure this pipeline, the userspace must take the following steps: bridge to the receiver, and the receiver to the DMA engines. This step does not differ from normal non-multiplexed media controller setup. -2) Configure routing. - -.. flat-table:: Sensor routing table (identical on both sensors) - :header-rows: 1 - - * - Sink Pad/Stream - - Source Pad/Stream - - Routing Flags - - Comments - * - 0/0 (unused) - - 0/0 - - V4L2_SUBDEV_ROUTE_FL_ACTIVE | V4L2_SUBDEV_ROUTE_FL_SOURCE_ONLY - - Pixel data stream. Source route, i.e. the sink fields are unused. - * - 0/0 (unused) - - 0/1 - - V4L2_SUBDEV_ROUTE_FL_ACTIVE | V4L2_SUBDEV_ROUTE_FL_SOURCE_ONLY - - Metadata stream. Source route, i.e. the sink fields are unused. +2) Configure routing .. flat-table:: Bridge routing table :header-rows: 1
@@ -628,18 +638,10 @@ not differ from normal non-multiplexed media controller setup. - 2/0 - V4L2_SUBDEV_ROUTE_FL_ACTIVE - Pixel data stream from Sensor A - * - 0/1 - - 2/1 - - V4L2_SUBDEV_ROUTE_FL_ACTIVE - - Metadata stream from Sensor A * - 1/0 - - 2/2 + - 2/1 - V4L2_SUBDEV_ROUTE_FL_ACTIVE - Pixel data stream from Sensor B - * - 1/1 - - 2/3 - - V4L2_SUBDEV_ROUTE_FL_ACTIVE - - Metadata stream from Sensor B .. flat-table:: Receiver routing table :header-rows: 1
@@ -655,22 +657,15 @@ not differ from normal non-multiplexed media controller setup. * - 0/1 - 2/0 - V4L2_SUBDEV_ROUTE_FL_ACTIVE - - Metadata stream from Sensor A - * - 0/2 - - 3/0 - - V4L2_SUBDEV_ROUTE_FL_ACTIVE - Pixel data stream from Sensor B - * - 0/3 - - 4/0 - - V4L2_SUBDEV_ROUTE_FL_ACTIVE - - Metadata stream from Sensor B -3) Configure streams +3) Configure formats and selections -After configuring the routing table, the next step is configuring the streams. -This step is similar to configuring the pads in a non-multiplexed streams -setup, with the difference that we need to configure each (pad, stream) pair -(i.e. route endpoint) instead of just a pad. +After configuring the routing table, the next step is configuring the +formats and selections for the streams. This step is similar to +configuring the pads in a non-multiplexed streams setup, with the +difference that we need to configure each (pad, stream) pair instead of +just a pad. A common way to accomplish this is to start from the sensors and propagate the configurations along the stream towards the receiver,
Signature · 1 line
2.30.2
Re: [PATCH 1/1] media: Documentation: Update documentation for streams
Quoted 61 lines · Sakari Ailus
Document how streams interacts with formats and selections.
Update documentation in respect to what is allowed, in particular streams are only supported via full routes, source-only routes are not supported right now.
The centerpiece of the API additions are streams. Albeit routes are configured via
S_ROUTINGIOCTL that also declares streams, it is streams that are accessed through other APIs. Thus refer to streams instead of routes in documentation.Signed-off-by Sakari Ailus <[email protected]>---
Hi folks,This replaces my earlier documetation patch. The commit message and the subject have changed and the content reflects this. Largely this means removing a few features of the interface --- for now.
The intent is to be able to merge this very soon, thus those portions that are still debated have been dropped (no more dependencies between streams, for instance).
.../userspace-api/media/v4l/dev-subdev.rst | 121 +++++++++--------- 1 file changed, 58 insertions(+), 63 deletions(-)Documentation/userspace-api/media/v4l/dev-subdev.rst +8
-6@@ -406,6 +406,8 @@ pixel array is not rectangular but cross-shaped or round. The maximum size may also be smaller than the BOUNDS rectangle. +.. _format-propagation: + Order of configuration and format propagation ---------------------------------------------@@ -507,12 +509,12 @@ source pads. Streams, multiplexed media pads and internal routing ---------------------------------------------------- -Commonly V4L2 subdevices support only separate video streams, that is, only a -single stream can pass through a media link and a media pad. Thus each pad -contains a format configuration for that single stream. In some cases a subdev -can do stream processing and split a stream into two or compose two streams -into one, but the inputs and outputs for the subdev are still a single stream -per pad. +Simple V4L2 subdevices do not support multiple, unrelated video streams, +and only a single stream can pass through a media link and a media pad. +Thus each pad contains a format and selection configuration for that +single stream. A subdev can do stream processing and split a stream into +two or compose two streams into one, but the inputs and outputs for the +subdev are still a single stream per pad. Some hardware, e.g. MIPI CSI-2, support multiplexed streams, that is, multiple data streams are transmitted on the same bus, which is represented by a media@@ -539,14 +541,35 @@ streams from one end of the link to the other, and subdevices have routing tables which describe how the incoming streams from sink pads are routed to the source pads.
Documentation/userspace-api/media/v4l/dev-subdev.rst streams from one end of the link to the other, and subdevices have routing · line 549
Quoted 10 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +5
-3-A stream ID (often just "stream") is a media link-local identifier for a stream. -In other words, a particular stream ID must exist on both sides of a media +A stream ID is a media pad-local identifier for a stream. Streams IDs of +the same stream must be equal on both ends of a link. In other words, +a particular stream ID must exist on both sides of a media link, but another stream ID can be used for the same stream at the other side -of the subdevice. +of the subdevice. The same stream ID is used to refer to the stream on +both pads of the link on all ioctls operating on pads.
This sentence feels a bit confusing. What's it trying to say? That the same Stream ID has to be on the pads on both sides of the link? Wasn't that already covered earlier?
Documentation/userspace-api/media/v4l/dev-subdev.rst streams from one end of the link to the other, and subdevices have routing · line 552
Quoted 2 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +2
-0+A stream at a specific point in the media pipeline is identified by the +sub-devdev and a (pad, stream) pair. For subdevices that do not support
Typo there in "sub-devdev". Also, there seems to be a lot of "sub-device" and "subdevice" words in the text. Which one is it?
Documentation/userspace-api/media/v4l/dev-subdev.rst streams from one end of the link to the other, and subdevices have routing · line 560
Quoted 8 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +8
-0+multiplexed streams the 'stream' field is always 0. + +Interaction between routes, streams, formats and selections +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The addition of streams to the V4L2 sub-device interface moves the sub-device +formats and selections from pads to (pad, stream) pairs. Besides the +usual pad, also the stream ID needs to be provided for setting formats and
Here, and already earlier, you first say "stream" and then "stream ID". Are they the same thing? You removed my "stream ID (often just "stream")" text, which I had included to clarify that those two terms are interchangeable in certain contexts where it's obvious.
Documentation/userspace-api/media/v4l/dev-subdev.rst streams from one end of the link to the other, and subdevices have routing · line 565
Quoted 5 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +5
-0+selections. The order of configuring formats and selections along a stream is +the same as without streams (see :ref:`format-propagation`). + +Instead of the sub-device wide merging of streams from all source pads +towards all sink pads, data flows for each route are separate from each
Shouldn't the above be from sink pads towards source pads?
Documentation/userspace-api/media/v4l/dev-subdev.rst streams from one end of the link to the other, and subdevices have routing · line 568
Quoted 3 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +3
-0+other. Any number of routes from streams on sink pads towards streams on +source pads is allowed, to the extent supported by drivers. For every +stream on a sink pad, however, only a single route is allowed.
I don't follow here. Don't you first say that any number of routes is ok, then you say only a single route is ok?
Documentation/userspace-api/media/v4l/dev-subdev.rst streams from one end of the link to the other, and subdevices have routing · line 572
Quoted 6 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +3
-3-A stream at a specific point in the media pipeline is identified with the -subdev and a (pad, stream) pair. For subdevices that do not support -multiplexed streams the 'stream' is always 0. +Any configurations of a stream within a pad, such as format or selections, +are independent of similar configurations on other streams. This is +subject to change in the future.
Hmm, what does this mean? Isn't this device specific? The format for a video stream will affect the format for a metadata stream from the same source.
Documentation/userspace-api/media/v4l/dev-subdev.rst There are three steps in configuring the streams: · line 587
Quoted 9 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +2
-1Configuring streams ^^^^^^^^^^^^^^^^^^^@@ -560,34 +583,37 @@ There are three steps in configuring the streams: 1) Set up links. Connect the pads between subdevices using the :ref:`Media Controller API <media_controller>` -2) Routing. The routing table for the subdevice must be set with +2) Streams. Streams are declared and their routing is configured by +setting the routing table for the subdevice must be set with
Hmm, maybe s/must be set with/using/
Documentation/userspace-api/media/v4l/dev-subdev.rst There are three steps in configuring the streams: · line 596
Quoted 13 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +7
-4:ref:`VIDIOC_SUBDEV_S_ROUTING <VIDIOC_SUBDEV_G_ROUTING>` ioctl. Note that -setting the routing table will reset all the stream configurations in a media -entity. +setting the routing table will reset formats and selections in the +sub-device to default values. -3) Configure streams. Each route endpoint must be configured -with :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>`. +3) Configure formats and selections. Formats and selections of each stream +are configured separately as documented for plain subdevices in +:ref:`<format-propagation>`. The stream ID is set to the same stream ID +associated with either sink or source pads of routes configured using the +:ref:`VIDIOC_SUBDEV_S_ROUTING <VIDIOC_SUBDEV_G_ROUTING>` ioctl.
Is this trying to say that the stream ID numbers used in the VIDIOC_SUBDEV_G_ROUTING call are the same used in the format and selection configuration? Isn't that obvious already? I at least find the sentence a bit confusing.
Documentation/userspace-api/media/v4l/dev-subdev.rst There are three steps in configuring the streams: · line 604
Quoted 10 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +2
-3Multiplexed streams setup example ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ A simple example of a multiplexed stream setup might be as follows: -- Two identical sensors (Sensor A and Sensor B). Each sensor has a single source - pad (pad 0) which carries two streams, pixel data stream and metadata - stream. +- Two identical sensors (Sensor A and Sensor B). The sensors produce image + data only and hence do not need specific support for streams.
Hmm, I have already changed all this in the v16, and I think I had other changes in the docs too. What's this patch based on?
Quoted 77 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +6
-38- Multiplexer bridge (Bridge). The bridge has two sink pads, connected to the - sensors (pads 0, 1), and one source pad (pad 2), which outputs all 4 - streams. + sensors (pads 0, 1), and one source pad (pad 2), which outputs both of + the streams. - Receiver in the SoC (Receiver). The receiver has a single sink pad (pad 0), connected to the bridge, and four source pads (pads 1-4), going to the DMA - engine. The receiver demultiplexes the incoming streams to the four source + engine. The receiver demultiplexes the incoming streams to two the source pads. - Four DMA Engines in the SoC (DMA Engine). Each DMA engine is connected to a - single source pad in the receiver. + single source pad in the receive via a link, two of which are active. The sensors, the bridge and the receiver are modeled as V4L2 subdevices, exposed to userspace via /dev/v4l-subdevX device nodes. The DMA engines are@@ -599,23 +625,7 @@ To configure this pipeline, the userspace must take the following steps: bridge to the receiver, and the receiver to the DMA engines. This step does not differ from normal non-multiplexed media controller setup. -2) Configure routing. - -.. flat-table:: Sensor routing table (identical on both sensors) - :header-rows: 1 - - * - Sink Pad/Stream - - Source Pad/Stream - - Routing Flags - - Comments - * - 0/0 (unused) - - 0/0 - - V4L2_SUBDEV_ROUTE_FL_ACTIVE | V4L2_SUBDEV_ROUTE_FL_SOURCE_ONLY - - Pixel data stream. Source route, i.e. the sink fields are unused. - * - 0/0 (unused) - - 0/1 - - V4L2_SUBDEV_ROUTE_FL_ACTIVE | V4L2_SUBDEV_ROUTE_FL_SOURCE_ONLY - - Metadata stream. Source route, i.e. the sink fields are unused. +2) Configure routing .. flat-table:: Bridge routing table :header-rows: 1@@ -628,18 +638,10 @@ not differ from normal non-multiplexed media controller setup. - 2/0 - V4L2_SUBDEV_ROUTE_FL_ACTIVE - Pixel data stream from Sensor A - * - 0/1 - - 2/1 - - V4L2_SUBDEV_ROUTE_FL_ACTIVE - - Metadata stream from Sensor A * - 1/0 - - 2/2 + - 2/1 - V4L2_SUBDEV_ROUTE_FL_ACTIVE - Pixel data stream from Sensor B - * - 1/1 - - 2/3 - - V4L2_SUBDEV_ROUTE_FL_ACTIVE - - Metadata stream from Sensor B .. flat-table:: Receiver routing table :header-rows: 1@@ -655,22 +657,15 @@ not differ from normal non-multiplexed media controller setup. * - 0/1 - 2/0 - V4L2_SUBDEV_ROUTE_FL_ACTIVE - - Metadata stream from Sensor A - * - 0/2 - - 3/0 - - V4L2_SUBDEV_ROUTE_FL_ACTIVE - Pixel data stream from Sensor B - * - 0/3 - - 4/0 - - V4L2_SUBDEV_ROUTE_FL_ACTIVE - - Metadata stream from Sensor B
Documentation/userspace-api/media/v4l/dev-subdev.rst not differ from normal non-multiplexed media controller setup. · line 666
Quoted 10 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +4
-5-3) Configure streams +3) Configure formats and selections -After configuring the routing table, the next step is configuring the streams. -This step is similar to configuring the pads in a non-multiplexed streams -setup, with the difference that we need to configure each (pad, stream) pair -(i.e. route endpoint) instead of just a pad. +After configuring the routing table, the next step is configuring the +formats and selections for the streams. This step is similar to +configuring the pads in a non-multiplexed streams setup, with the
As you say "configuring the formats and selections for the streams", maybe similarly you could say "is similar to configuring the formats and selections for the pads".
Documentation/userspace-api/media/v4l/dev-subdev.rst not differ from normal non-multiplexed media controller setup. · line 668
Quoted 2 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +2
-0+difference that we need to configure each (pad, stream) pair instead of +just a pad.
Do we "configure the streams/pads", or do we "configure the formats and selections for the streams/pads"?
Tomi
Re: [PATCH 1/1] media: Documentation: Update documentation for streams
Moi,
Quoted 62 lines · Tomi Valkeinen
Quoted 61 lines · Sakari Ailus
Document how streams interacts with formats and selections.
Update documentation in respect to what is allowed, in particular streams are only supported via full routes, source-only routes are not supported right now.
The centerpiece of the API additions are streams. Albeit routes are configured via
S_ROUTINGIOCTL that also declares streams, it is streams that are accessed through other APIs. Thus refer to streams instead of routes in documentation.Signed-off-by Sakari Ailus <[email protected]>---
Hi folks,This replaces my earlier documetation patch. The commit message and the subject have changed and the content reflects this. Largely this means removing a few features of the interface --- for now.
The intent is to be able to merge this very soon, thus those portions that are still debated have been dropped (no more dependencies between streams, for instance).
.../userspace-api/media/v4l/dev-subdev.rst | 121 +++++++++--------- 1 file changed, 58 insertions(+), 63 deletions(-)Documentation/userspace-api/media/v4l/dev-subdev.rst +11
-8@@ -406,6 +406,8 @@ pixel array is not rectangular but cross-shaped or round. The maximum size may also be smaller than the BOUNDS rectangle. +.. _format-propagation: + Order of configuration and format propagation ---------------------------------------------@@ -507,12 +509,12 @@ source pads. Streams, multiplexed media pads and internal routing ---------------------------------------------------- -Commonly V4L2 subdevices support only separate video streams, that is, only a -single stream can pass through a media link and a media pad. Thus each pad -contains a format configuration for that single stream. In some cases a subdev -can do stream processing and split a stream into two or compose two streams -into one, but the inputs and outputs for the subdev are still a single stream -per pad. +Simple V4L2 subdevices do not support multiple, unrelated video streams, +and only a single stream can pass through a media link and a media pad. +Thus each pad contains a format and selection configuration for that +single stream. A subdev can do stream processing and split a stream into +two or compose two streams into one, but the inputs and outputs for the +subdev are still a single stream per pad. Some hardware, e.g. MIPI CSI-2, support multiplexed streams, that is, multiple data streams are transmitted on the same bus, which is represented by a media@@ -539,14 +541,35 @@ streams from one end of the link to the other, and subdevices have routing tables which describe how the incoming streams from sink pads are routed to the source pads. -A stream ID (often just "stream") is a media link-local identifier for a stream. -In other words, a particular stream ID must exist on both sides of a media +A stream ID is a media pad-local identifier for a stream. Streams IDs of +the same stream must be equal on both ends of a link. In other words, +a particular stream ID must exist on both sides of a media
Quoted 8 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst streams from one end of the link to the other, and subdevices have routing · line 549
Quoted 4 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +2
-1link, but another stream ID can be used for the same stream at the other side -of the subdevice. +of the subdevice. The same stream ID is used to refer to the stream on +both pads of the link on all ioctls operating on pads.This sentence feels a bit confusing. What's it trying to say? That the same Stream ID has to be on the pads on both sides of the link? Wasn't that already covered earlier?
After reading the paragraph again, I think the last sentence can be dropped indeed.
Quoted 6 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst streams from one end of the link to the other, and subdevices have routing · line 552
Quoted 3 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +2
-0+A stream at a specific point in the media pipeline is identified by the +sub-devdev and a (pad, stream) pair. For subdevices that do not supportTypo there in "sub-devdev". Also, there seems to be a lot of "sub-device" and "subdevice" words in the text. Which one is it?
It's intended to be "sub-device". Sometimes it is shortedned to "subdev", but this is documentation so I'd avoid it here. "Subdevice" seems to be only present on routing documentation. I'll replace all instances with "sub-device".
Quoted 14 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst streams from one end of the link to the other, and subdevices have routing · line 560
Quoted 9 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +8
-0+multiplexed streams the 'stream' field is always 0. + +Interaction between routes, streams, formats and selections +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The addition of streams to the V4L2 sub-device interface moves the sub-device +formats and selections from pads to (pad, stream) pairs. Besides the +usual pad, also the stream ID needs to be provided for setting formats andHere, and already earlier, you first say "stream" and then "stream ID". Are they the same thing? You removed my "stream ID (often just "stream")" text, which I had included to clarify that those two terms are interchangeable in certain contexts where it's obvious.
It says above that "A stream ID is a media pad-local identifier for a stream".
I think (pad, stream) pair is fine --- we don't talk about pad IDs either.
Quoted 8 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst streams from one end of the link to the other, and subdevices have routing · line 565
Quoted 6 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +5
-0+selections. The order of configuring formats and selections along a stream is +the same as without streams (see :ref:`format-propagation`). + +Instead of the sub-device wide merging of streams from all source pads +towards all sink pads, data flows for each route are separate from eachShouldn't the above be from sink pads towards source pads?
Uh, yes. I'll fix for v2.
Quoted 7 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst streams from one end of the link to the other, and subdevices have routing · line 568
Quoted 4 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +3
-0+other. Any number of routes from streams on sink pads towards streams on +source pads is allowed, to the extent supported by drivers. For every +stream on a sink pad, however, only a single route is allowed.I don't follow here. Don't you first say that any number of routes is ok, then you say only a single route is ok?
The latter "sink" should have been "source". I'll fix it.
Quoted 10 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst streams from one end of the link to the other, and subdevices have routing · line 572
Quoted 7 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +3
-3-A stream at a specific point in the media pipeline is identified with the -subdev and a (pad, stream) pair. For subdevices that do not support -multiplexed streams the 'stream' is always 0. +Any configurations of a stream within a pad, such as format or selections, +are independent of similar configurations on other streams. This is +subject to change in the future.Hmm, what does this mean? Isn't this device specific? The format for a video stream will affect the format for a metadata stream from the same source.
Correct, but we don't support these yet. They should of course follow soon after the initial merge.
Quoted 11 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst There are three steps in configuring the streams: · line 586
Quoted 9 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +2
-1Configuring streams ^^^^^^^^^^^^^^^^^^^@@ -560,34 +583,37 @@ There are three steps in configuring the streams: 1) Set up links. Connect the pads between subdevices using the :ref:`Media Controller API <media_controller>` -2) Routing. The routing table for the subdevice must be set with +2) Streams. Streams are declared and their routing is configured by +setting the routing table for the subdevice must be set withHmm, maybe s/must be set with/using/
Yes.
Quoted 28 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst There are three steps in configuring the streams: · line 596
Quoted 13 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +7
-4:ref:`VIDIOC_SUBDEV_S_ROUTING <VIDIOC_SUBDEV_G_ROUTING>` ioctl. Note that -setting the routing table will reset all the stream configurations in a media -entity. +setting the routing table will reset formats and selections in the +sub-device to default values. -3) Configure streams. Each route endpoint must be configured -with :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>`. +3) Configure formats and selections. Formats and selections of each stream +are configured separately as documented for plain subdevices in +:ref:`<format-propagation>`. The stream ID is set to the same stream ID +associated with either sink or source pads of routes configured using the +:ref:`VIDIOC_SUBDEV_S_ROUTING <VIDIOC_SUBDEV_G_ROUTING>` ioctl.Is this trying to say that the stream ID numbers used in the
VIDIOC_SUBDEV_G_ROUTINGcall are the same used in the format and selection configuration? Isn't that obvious already? I at least find the sentence a bit confusing.Documentation/userspace-api/media/v4l/dev-subdev.rst There are three steps in configuring the streams: · line 604
Quoted 8 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +2
-3Multiplexed streams setup example ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ A simple example of a multiplexed stream setup might be as follows: -- Two identical sensors (Sensor A and Sensor B). Each sensor has a single source - pad (pad 0) which carries two streams, pixel data stream and metadata - stream. +- Two identical sensors (Sensor A and Sensor B). The sensors produce image + data only and hence do not need specific support for streams.
Quoted 2 lines
Hmm, I have already changed all this in the v16, and I think I had other changes in the docs too. What's this patch based on?
As discussed, this patch was based on your streams/work-v16 branch. I'll rebase this on the last patch of the set, which also means removing this and related changes.
Quoted 76 lines
Quoted 76 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +7
-42- Multiplexer bridge (Bridge). The bridge has two sink pads, connected to the - sensors (pads 0, 1), and one source pad (pad 2), which outputs all 4 - streams. + sensors (pads 0, 1), and one source pad (pad 2), which outputs both of + the streams. - Receiver in the SoC (Receiver). The receiver has a single sink pad (pad 0), connected to the bridge, and four source pads (pads 1-4), going to the DMA - engine. The receiver demultiplexes the incoming streams to the four source + engine. The receiver demultiplexes the incoming streams to two the source pads. - Four DMA Engines in the SoC (DMA Engine). Each DMA engine is connected to a - single source pad in the receiver. + single source pad in the receive via a link, two of which are active. The sensors, the bridge and the receiver are modeled as V4L2 subdevices, exposed to userspace via /dev/v4l-subdevX device nodes. The DMA engines are@@ -599,23 +625,7 @@ To configure this pipeline, the userspace must take the following steps: bridge to the receiver, and the receiver to the DMA engines. This step does not differ from normal non-multiplexed media controller setup. -2) Configure routing. - -.. flat-table:: Sensor routing table (identical on both sensors) - :header-rows: 1 - - * - Sink Pad/Stream - - Source Pad/Stream - - Routing Flags - - Comments - * - 0/0 (unused) - - 0/0 - - V4L2_SUBDEV_ROUTE_FL_ACTIVE | V4L2_SUBDEV_ROUTE_FL_SOURCE_ONLY - - Pixel data stream. Source route, i.e. the sink fields are unused. - * - 0/0 (unused) - - 0/1 - - V4L2_SUBDEV_ROUTE_FL_ACTIVE | V4L2_SUBDEV_ROUTE_FL_SOURCE_ONLY - - Metadata stream. Source route, i.e. the sink fields are unused. +2) Configure routing .. flat-table:: Bridge routing table :header-rows: 1@@ -628,18 +638,10 @@ not differ from normal non-multiplexed media controller setup. - 2/0 - V4L2_SUBDEV_ROUTE_FL_ACTIVE - Pixel data stream from Sensor A - * - 0/1 - - 2/1 - - V4L2_SUBDEV_ROUTE_FL_ACTIVE - - Metadata stream from Sensor A * - 1/0 - - 2/2 + - 2/1 - V4L2_SUBDEV_ROUTE_FL_ACTIVE - Pixel data stream from Sensor B - * - 1/1 - - 2/3 - - V4L2_SUBDEV_ROUTE_FL_ACTIVE - - Metadata stream from Sensor B .. flat-table:: Receiver routing table :header-rows: 1@@ -655,22 +657,15 @@ not differ from normal non-multiplexed media controller setup. * - 0/1 - 2/0 - V4L2_SUBDEV_ROUTE_FL_ACTIVE - - Metadata stream from Sensor A - * - 0/2 - - 3/0 - - V4L2_SUBDEV_ROUTE_FL_ACTIVE - Pixel data stream from Sensor B - * - 0/3 - - 4/0 - - V4L2_SUBDEV_ROUTE_FL_ACTIVE - - Metadata stream from Sensor B -3) Configure streams +3) Configure formats and selections -After configuring the routing table, the next step is configuring the streams. -This step is similar to configuring the pads in a non-multiplexed streams -setup, with the difference that we need to configure each (pad, stream) pair
Quoted 8 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst not differ from normal non-multiplexed media controller setup. · line 666
Quoted 4 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +3
-1-(i.e. route endpoint) instead of just a pad. +After configuring the routing table, the next step is configuring the +formats and selections for the streams. This step is similar to +configuring the pads in a non-multiplexed streams setup, with theAs you say "configuring the formats and selections for the streams", maybe similarly you could say "is similar to configuring the formats and selections for the pads".
We haven't referred the non-stream aware configuration as being specifically performed on pads. I think this should be rewritten, not just slightly adjusted.
How about:
After configuring routing, the next step is configuring the formats and selections for the streams. This is similar to performing this step without streams, with just one exception: the ``stream`` field needs to be assigned to the value of the stream ID.
Quoted 9 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst not differ from normal non-multiplexed media controller setup. · line 668
Quoted 3 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +2
-0+difference that we need to configure each (pad, stream) pair instead of +just a pad.Do we "configure the streams/pads", or do we "configure the formats and selections for the streams/pads"?
Tomi
Signature · 3 lines
Terveisin, Sakari Ailus
PATCH v2 1/1 media: Documentation: Update documentation for streams
In reply to Tomi ValkeinenDocument how streams interacts with formats and selections.
Update documentation in respect to what is allowed, in particular streams are only supported via full routes, source-only routes are not supported right now.
The centerpiece of the API additions are streams. Albeit routes are configured via S_ROUTING IOCTL that also declares streams, it is streams that are accessed through other APIs. Thus refer to streams instead of routes in documentation.
---
since v1:
- Address Tomi's comments:
- Fix wrong terms (sink vs. source, others)
- Align subdev, subdevice to sub-device consistently
- Improve wording in a number of places
- Rebase on plain v16 streams set (i.e. not additional patches in between)
- Drop a few redundant sentences
See: <URL:https://lore.kernel.org/linux-media/[email protected]/T/#me8bb137ec1bc7fb20efe8042be04b40753928f1d>
.../userspace-api/media/v4l/dev-subdev.rst | 74 +++++++++++++------ 1 file changed, 51 insertions(+), 23 deletions(-)
Documentation/userspace-api/media/v4l/dev-subdev.rst +51 -23
@@ -406,6 +406,8 @@ pixel array is not rectangular but cross-shaped or round. The maximum size may also be smaller than the BOUNDS rectangle. +.. _format-propagation: + Order of configuration and format propagation ---------------------------------------------
@@ -507,12 +509,12 @@ source pads. Streams, multiplexed media pads and internal routing ---------------------------------------------------- -Commonly V4L2 subdevices support only separate video streams, that is, only a -single stream can pass through a media link and a media pad. Thus each pad -contains a format configuration for that single stream. In some cases a subdev -can do stream processing and split a stream into two or compose two streams -into one, but the inputs and outputs for the subdev are still a single stream -per pad. +Simple V4L2 subdevices do not support multiple, unrelated video streams, +and only a single stream can pass through a media link and a media pad. +Thus each pad contains a format and selection configuration for that +single stream. A subdev can do stream processing and split a stream into +two or compose two streams into one, but the inputs and outputs for the +subdev are still a single stream per pad. Some hardware, e.g. MIPI CSI-2, support multiplexed streams, that is, multiple data streams are transmitted on the same bus, which is represented by a media
@@ -539,14 +541,35 @@ streams from one end of the link to the other, and subdevices have routing tables which describe how the incoming streams from sink pads are routed to the source pads. -A stream ID (often just "stream") is a media link-local identifier for a stream. -In other words, a particular stream ID must exist on both sides of a media +A stream ID is a media pad-local identifier for a stream. Streams IDs of +the same stream must be equal on both ends of a link. In other words, +a particular stream ID must exist on both sides of a media link, but another stream ID can be used for the same stream at the other side -of the subdevice. +of the subdevice. The same stream ID is used to refer to the stream on +both pads of the link on all ioctls operating on pads. + +A stream at a specific point in the media pipeline is identified by the +sub-devdev and a (pad, stream) pair. For subdevices that do not support +multiplexed streams the 'stream' field is always 0. + +Interaction between routes, streams, formats and selections +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The addition of streams to the V4L2 sub-device interface moves the sub-device +formats and selections from pads to (pad, stream) pairs. Besides the +usual pad, also the stream ID needs to be provided for setting formats and +selections. The order of configuring formats and selections along a stream is +the same as without streams (see :ref:`format-propagation`). + +Instead of the sub-device wide merging of streams from all source pads +towards all sink pads, data flows for each route are separate from each +other. Any number of routes from streams on sink pads towards streams on +source pads is allowed, to the extent supported by drivers. For every +stream on a sink pad, however, only a single route is allowed. -A stream at a specific point in the media pipeline is identified with the -subdev and a (pad, stream) pair. For subdevices that do not support -multiplexed streams the 'stream' is always 0. +Any configurations of a stream within a pad, such as format or selections, +are independent of similar configurations on other streams. This is +subject to change in the future. Configuring streams ^^^^^^^^^^^^^^^^^^^
@@ -560,13 +583,17 @@ There are three steps in configuring the streams: 1) Set up links. Connect the pads between subdevices using the :ref:`Media Controller API <media_controller>` -2) Routing. The routing table for the subdevice must be set with +2) Streams. Streams are declared and their routing is configured by +setting the routing table for the subdevice must be set with :ref:`VIDIOC_SUBDEV_S_ROUTING <VIDIOC_SUBDEV_G_ROUTING>` ioctl. Note that -setting the routing table will reset all the stream configurations in a media -entity. +setting the routing table will reset formats and selections in the +sub-device to default values. -3) Configure streams. Each route endpoint must be configured -with :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>`. +3) Configure formats and selections. Formats and selections of each stream +are configured separately as documented for plain subdevices in +:ref:`<format-propagation>`. The stream ID is set to the same stream ID +associated with either sink or source pads of routes configured using the +:ref:`VIDIOC_SUBDEV_S_ROUTING <VIDIOC_SUBDEV_G_ROUTING>` ioctl. Multiplexed streams setup example ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -596,7 +623,7 @@ To configure this pipeline, the userspace must take the following steps: bridge to the receiver, and the receiver to the DMA engines. This step does not differ from normal non-multiplexed media controller setup. -2) Configure routing. +2) Configure routing .. flat-table:: Bridge routing table :header-rows: 1
@@ -630,12 +657,13 @@ not differ from normal non-multiplexed media controller setup. - V4L2_SUBDEV_ROUTE_FL_ACTIVE - Pixel data stream from Sensor B -3) Configure streams +3) Configure formats and selections -After configuring the routing table, the next step is configuring the streams. -This step is similar to configuring the pads in a non-multiplexed streams -setup, with the difference that we need to configure each (pad, stream) pair -(i.e. route endpoint) instead of just a pad. +After configuring the routing table, the next step is configuring the +formats and selections for the streams. This step is similar to +configuring the pads in a non-multiplexed streams setup, with the +difference that we need to configure each (pad, stream) pair instead of +just a pad. A common way to accomplish this is to start from the sensors and propagate the configurations along the stream towards the receiver,
Signature · 1 line
2.30.2
Re: [PATCH v2 1/1] media: Documentation: Update documentation for streams
Quoted 70 lines · Sakari Ailus
Document how streams interacts with formats and selections.
Update documentation in respect to what is allowed, in particular streams are only supported via full routes, source-only routes are not supported right now.
The centerpiece of the API additions are streams. Albeit routes are configured via
S_ROUTINGIOCTL that also declares streams, it is streams that are accessed through other APIs. Thus refer to streams instead of routes in documentation.Signed-off-by Sakari Ailus <[email protected]>---
since v1:- Address Tomi's comments:
- Fix wrong terms (sink vs. source, others)
- Align subdev, subdevice to sub-device consistently
- Improve wording in a number of places
- Rebase on plain v16 streams set (i.e. not additional patches in between)
- Drop a few redundant sentences
See: <URL:https://lore.kernel.org/linux-media/[email protected]/T/#me8bb137ec1bc7fb20efe8042be04b40753928f1d>.../userspace-api/media/v4l/dev-subdev.rst | 74 +++++++++++++------ 1 file changed, 51 insertions(+), 23 deletions(-)Documentation/userspace-api/media/v4l/dev-subdev.rst +8
-8@@ -406,6 +406,8 @@ pixel array is not rectangular but cross-shaped or round. The maximum size may also be smaller than the BOUNDS rectangle. +.. _format-propagation: + Order of configuration and format propagation ---------------------------------------------@@ -507,12 +509,12 @@ source pads. Streams, multiplexed media pads and internal routing ---------------------------------------------------- -Commonly V4L2 subdevices support only separate video streams, that is, only a -single stream can pass through a media link and a media pad. Thus each pad -contains a format configuration for that single stream. In some cases a subdev -can do stream processing and split a stream into two or compose two streams -into one, but the inputs and outputs for the subdev are still a single stream -per pad. +Simple V4L2 subdevices do not support multiple, unrelated video streams, +and only a single stream can pass through a media link and a media pad. +Thus each pad contains a format and selection configuration for that +single stream. A subdev can do stream processing and split a stream into +two or compose two streams into one, but the inputs and outputs for the +subdev are still a single stream per pad. Some hardware, e.g. MIPI CSI-2, support multiplexed streams, that is, multiple data streams are transmitted on the same bus, which is represented by a media@@ -539,14 +541,35 @@ streams from one end of the link to the other, and subdevices have routing tables which describe how the incoming streams from sink pads are routed to the source pads. -A stream ID (often just "stream") is a media link-local identifier for a stream. -In other words, a particular stream ID must exist on both sides of a media
Documentation/userspace-api/media/v4l/dev-subdev.rst streams from one end of the link to the other, and subdevices have routing · line 552
Quoted 10 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +8
-1+A stream ID is a media pad-local identifier for a stream. Streams IDs of +the same stream must be equal on both ends of a link. In other words, +a particular stream ID must exist on both sides of a media link, but another stream ID can be used for the same stream at the other side -of the subdevice. +of the subdevice. The same stream ID is used to refer to the stream on +both pads of the link on all ioctls operating on pads. + +A stream at a specific point in the media pipeline is identified by the +sub-devdev and a (pad, stream) pair. For subdevices that do not support
"sub-devdev". Also, you have "subdevice" (also in other places in the text), not "sub-device".
Other than that, looks fine:
Tomi
PATCH v3 1/1 media: Documentation: Update documentation for streams
In reply to Tomi ValkeinenDocument how streams interacts with formats and selections.
Update documentation in respect to what is allowed, in particular streams are only supported via full routes, source-only routes are not supported right now.
The centerpiece of the API additions are streams. Albeit routes are configured via S_ROUTING IOCTL that also declares streams, it is streams that are accessed through other APIs. Thus refer to streams instead of routes in documentation.
---
since v2:
- Include sub-device term alignment changes and rewrite of the 3rd point of stream-based configuration missed in v2.
.../userspace-api/media/v4l/dev-subdev.rst | 84 ++++++++++++------- 1 file changed, 55 insertions(+), 29 deletions(-)
Documentation/userspace-api/media/v4l/dev-subdev.rst +55 -29
@@ -406,6 +406,8 @@ pixel array is not rectangular but cross-shaped or round. The maximum size may also be smaller than the BOUNDS rectangle. +.. _format-propagation: + Order of configuration and format propagation ---------------------------------------------
@@ -507,12 +509,12 @@ source pads. Streams, multiplexed media pads and internal routing ---------------------------------------------------- -Commonly V4L2 subdevices support only separate video streams, that is, only a -single stream can pass through a media link and a media pad. Thus each pad -contains a format configuration for that single stream. In some cases a subdev -can do stream processing and split a stream into two or compose two streams -into one, but the inputs and outputs for the subdev are still a single stream -per pad. +Simple V4L2 sub-devices do not support multiple, unrelated video streams, +and only a single stream can pass through a media link and a media pad. +Thus each pad contains a format and selection configuration for that +single stream. A subdev can do stream processing and split a stream into +two or compose two streams into one, but the inputs and outputs for the +subdev are still a single stream per pad. Some hardware, e.g. MIPI CSI-2, support multiplexed streams, that is, multiple data streams are transmitted on the same bus, which is represented by a media
@@ -535,38 +537,62 @@ Understanding streams A stream is a stream of content (e.g. pixel data or metadata) flowing through the media pipeline from a source (e.g. a sensor) towards the final sink (e.g. a receiver and demultiplexer in a SoC). Each media link carries all the enabled -streams from one end of the link to the other, and subdevices have routing +streams from one end of the link to the other, and sub-devices have routing tables which describe how the incoming streams from sink pads are routed to the source pads. -A stream ID (often just "stream") is a media link-local identifier for a stream. -In other words, a particular stream ID must exist on both sides of a media +A stream ID is a media pad-local identifier for a stream. Streams IDs of +the same stream must be equal on both ends of a link. In other words, +a particular stream ID must exist on both sides of a media link, but another stream ID can be used for the same stream at the other side -of the subdevice. +of the sub-device. + +A stream at a specific point in the media pipeline is identified by the +sub-device and a (pad, stream) pair. For sub-devices that do not support +multiplexed streams the 'stream' field is always 0. + +Interaction between routes, streams, formats and selections +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The addition of streams to the V4L2 sub-device interface moves the sub-device +formats and selections from pads to (pad, stream) pairs. Besides the +usual pad, also the stream ID needs to be provided for setting formats and +selections. The order of configuring formats and selections along a stream is +the same as without streams (see :ref:`format-propagation`). + +Instead of the sub-device wide merging of streams from all sink pads +towards all source pads, data flows for each route are separate from each +other. Any number of routes from streams on sink pads towards streams on +source pads is allowed, to the extent supported by drivers. For every +stream on a source pad, however, only a single route is allowed. -A stream at a specific point in the media pipeline is identified with the -subdev and a (pad, stream) pair. For subdevices that do not support -multiplexed streams the 'stream' is always 0. +Any configurations of a stream within a pad, such as format or selections, +are independent of similar configurations on other streams. This is +subject to change in the future. Configuring streams ^^^^^^^^^^^^^^^^^^^ -The configuration of the streams is done individually for each subdevice and -the validity of the streams between subdevices is validated when the pipeline +The configuration of the streams is done individually for each sub-device and +the validity of the streams between sub-devices is validated when the pipeline is started. There are three steps in configuring the streams: -1) Set up links. Connect the pads between subdevices using the :ref:`Media +1) Set up links. Connect the pads between sub-devices using the :ref:`Media Controller API <media_controller>` -2) Routing. The routing table for the subdevice must be set with +2) Streams. Streams are declared and their routing is configured by +setting the routing table for the sub-device using :ref:`VIDIOC_SUBDEV_S_ROUTING <VIDIOC_SUBDEV_G_ROUTING>` ioctl. Note that -setting the routing table will reset all the stream configurations in a media -entity. +setting the routing table will reset formats and selections in the +sub-device to default values. -3) Configure streams. Each route endpoint must be configured -with :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>`. +3) Configure formats and selections. Formats and selections of each stream +are configured separately as documented for plain sub-devices in +:ref:`<format-propagation>`. The stream ID is set to the same stream ID +associated with either sink or source pads of routes configured using the +:ref:`VIDIOC_SUBDEV_S_ROUTING <VIDIOC_SUBDEV_G_ROUTING>` ioctl. Multiplexed streams setup example ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -586,7 +612,7 @@ A simple example of a multiplexed stream setup might be as follows: - DMA Engines in the SoC (DMA Engine), one for each stream. Each DMA engine is connected to a single source pad in the receiver. -The sensors, the bridge and the receiver are modeled as V4L2 subdevices, +The sensors, the bridge and the receiver are modeled as V4L2 sub-devices, exposed to userspace via /dev/v4l-subdevX device nodes. The DMA engines are modeled as V4L2 devices, exposed to userspace via /dev/videoX nodes.
@@ -596,7 +622,7 @@ To configure this pipeline, the userspace must take the following steps: bridge to the receiver, and the receiver to the DMA engines. This step does not differ from normal non-multiplexed media controller setup. -2) Configure routing. +2) Configure routing .. flat-table:: Bridge routing table :header-rows: 1
@@ -630,14 +656,14 @@ not differ from normal non-multiplexed media controller setup. - V4L2_SUBDEV_ROUTE_FL_ACTIVE - Pixel data stream from Sensor B -3) Configure streams +3) Configure formats and selections -After configuring the routing table, the next step is configuring the streams. -This step is similar to configuring the pads in a non-multiplexed streams -setup, with the difference that we need to configure each (pad, stream) pair -(i.e. route endpoint) instead of just a pad. +After configuring routing, the next step is configuring the formats and +selections for the streams. This is similar to performing this step without +streams, with just one exception: the ``stream`` field needs to be assigned +to the value of the stream ID. A common way to accomplish this is to start from the sensors and propagate the configurations along the stream towards the receiver, using :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctls to configure each -stream endpoint in each subdev. +stream endpoint in each sub-device.
Signature · 1 line
2.30.2
Re: [PATCH v3 1/1] media: Documentation: Update documentation for streams
Quoted 6 lines · Sakari Ailus
Document how streams interacts with formats and selections.
Update documentation in respect to what is allowed, in particular streams are only supported via full routes, source-only routes are not supported right now.
Quoted 6 lines
The centerpiece of the API additions are streams. Albeit routes are configured via
S_ROUTINGIOCTL that also declares streams, it is streams that are accessed through other APIs. Thus refer to streams instead of routes in documentation.Signed-off-by Sakari Ailus <[email protected]>
Tomi
Quoted 158 lines
---
since v2:- Include sub-device term alignment changes and rewrite of the 3rd point of stream-based configuration missed in v2.
.../userspace-api/media/v4l/dev-subdev.rst | 84 ++++++++++++------- 1 file changed, 55 insertions(+), 29 deletions(-)Documentation/userspace-api/media/v4l/dev-subdev.rst +55
-29@@ -406,6 +406,8 @@ pixel array is not rectangular but cross-shaped or round. The maximum size may also be smaller than the BOUNDS rectangle. +.. _format-propagation: + Order of configuration and format propagation ---------------------------------------------@@ -507,12 +509,12 @@ source pads. Streams, multiplexed media pads and internal routing ---------------------------------------------------- -Commonly V4L2 subdevices support only separate video streams, that is, only a -single stream can pass through a media link and a media pad. Thus each pad -contains a format configuration for that single stream. In some cases a subdev -can do stream processing and split a stream into two or compose two streams -into one, but the inputs and outputs for the subdev are still a single stream -per pad. +Simple V4L2 sub-devices do not support multiple, unrelated video streams, +and only a single stream can pass through a media link and a media pad. +Thus each pad contains a format and selection configuration for that +single stream. A subdev can do stream processing and split a stream into +two or compose two streams into one, but the inputs and outputs for the +subdev are still a single stream per pad. Some hardware, e.g. MIPI CSI-2, support multiplexed streams, that is, multiple data streams are transmitted on the same bus, which is represented by a media@@ -535,38 +537,62 @@ Understanding streams A stream is a stream of content (e.g. pixel data or metadata) flowing through the media pipeline from a source (e.g. a sensor) towards the final sink (e.g. a receiver and demultiplexer in a SoC). Each media link carries all the enabled -streams from one end of the link to the other, and subdevices have routing +streams from one end of the link to the other, and sub-devices have routing tables which describe how the incoming streams from sink pads are routed to the source pads. -A stream ID (often just "stream") is a media link-local identifier for a stream. -In other words, a particular stream ID must exist on both sides of a media +A stream ID is a media pad-local identifier for a stream. Streams IDs of +the same stream must be equal on both ends of a link. In other words, +a particular stream ID must exist on both sides of a media link, but another stream ID can be used for the same stream at the other side -of the subdevice. +of the sub-device. + +A stream at a specific point in the media pipeline is identified by the +sub-device and a (pad, stream) pair. For sub-devices that do not support +multiplexed streams the 'stream' field is always 0. + +Interaction between routes, streams, formats and selections +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The addition of streams to the V4L2 sub-device interface moves the sub-device +formats and selections from pads to (pad, stream) pairs. Besides the +usual pad, also the stream ID needs to be provided for setting formats and +selections. The order of configuring formats and selections along a stream is +the same as without streams (see :ref:`format-propagation`). + +Instead of the sub-device wide merging of streams from all sink pads +towards all source pads, data flows for each route are separate from each +other. Any number of routes from streams on sink pads towards streams on +source pads is allowed, to the extent supported by drivers. For every +stream on a source pad, however, only a single route is allowed. -A stream at a specific point in the media pipeline is identified with the -subdev and a (pad, stream) pair. For subdevices that do not support -multiplexed streams the 'stream' is always 0. +Any configurations of a stream within a pad, such as format or selections, +are independent of similar configurations on other streams. This is +subject to change in the future. Configuring streams ^^^^^^^^^^^^^^^^^^^ -The configuration of the streams is done individually for each subdevice and -the validity of the streams between subdevices is validated when the pipeline +The configuration of the streams is done individually for each sub-device and +the validity of the streams between sub-devices is validated when the pipeline is started. There are three steps in configuring the streams: -1) Set up links. Connect the pads between subdevices using the :ref:`Media +1) Set up links. Connect the pads between sub-devices using the :ref:`Media Controller API <media_controller>` -2) Routing. The routing table for the subdevice must be set with +2) Streams. Streams are declared and their routing is configured by +setting the routing table for the sub-device using :ref:`VIDIOC_SUBDEV_S_ROUTING <VIDIOC_SUBDEV_G_ROUTING>` ioctl. Note that -setting the routing table will reset all the stream configurations in a media -entity. +setting the routing table will reset formats and selections in the +sub-device to default values. -3) Configure streams. Each route endpoint must be configured -with :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>`. +3) Configure formats and selections. Formats and selections of each stream +are configured separately as documented for plain sub-devices in +:ref:`<format-propagation>`. The stream ID is set to the same stream ID +associated with either sink or source pads of routes configured using the +:ref:`VIDIOC_SUBDEV_S_ROUTING <VIDIOC_SUBDEV_G_ROUTING>` ioctl. Multiplexed streams setup example ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^@@ -586,7 +612,7 @@ A simple example of a multiplexed stream setup might be as follows: - DMA Engines in the SoC (DMA Engine), one for each stream. Each DMA engine is connected to a single source pad in the receiver. -The sensors, the bridge and the receiver are modeled as V4L2 subdevices, +The sensors, the bridge and the receiver are modeled as V4L2 sub-devices, exposed to userspace via /dev/v4l-subdevX device nodes. The DMA engines are modeled as V4L2 devices, exposed to userspace via /dev/videoX nodes.@@ -596,7 +622,7 @@ To configure this pipeline, the userspace must take the following steps: bridge to the receiver, and the receiver to the DMA engines. This step does not differ from normal non-multiplexed media controller setup. -2) Configure routing. +2) Configure routing .. flat-table:: Bridge routing table :header-rows: 1@@ -630,14 +656,14 @@ not differ from normal non-multiplexed media controller setup. - V4L2_SUBDEV_ROUTE_FL_ACTIVE - Pixel data stream from Sensor B -3) Configure streams +3) Configure formats and selections -After configuring the routing table, the next step is configuring the streams. -This step is similar to configuring the pads in a non-multiplexed streams -setup, with the difference that we need to configure each (pad, stream) pair -(i.e. route endpoint) instead of just a pad. +After configuring routing, the next step is configuring the formats and +selections for the streams. This is similar to performing this step without +streams, with just one exception: the ``stream`` field needs to be assigned +to the value of the stream ID. A common way to accomplish this is to start from the sensors and propagate the configurations along the stream towards the receiver, using :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctls to configure each -stream endpoint in each subdev. +stream endpoint in each sub-device.
PATCH v4 1/1 media: Documentation: Update documentation for streams
In reply to Tomi ValkeinenDocument how streams interacts with formats and selections.
Update documentation in respect to what is allowed, in particular streams are only supported via full routes, source-only routes are not supported right now.
The centerpiece of the API additions are streams. Albeit routes are configured via S_ROUTING IOCTL that also declares streams, it is streams that are accessed through other APIs. Thus refer to streams instead of routes in documentation.
---
since v3:
- Fix documentation reference
.../userspace-api/media/v4l/dev-subdev.rst | 84 ++++++++++++------- 1 file changed, 55 insertions(+), 29 deletions(-)
Documentation/userspace-api/media/v4l/dev-subdev.rst +55 -29
@@ -406,6 +406,8 @@ pixel array is not rectangular but cross-shaped or round. The maximum size may also be smaller than the BOUNDS rectangle. +.. _format-propagation: + Order of configuration and format propagation ---------------------------------------------
@@ -507,12 +509,12 @@ source pads. Streams, multiplexed media pads and internal routing ---------------------------------------------------- -Commonly V4L2 subdevices support only separate video streams, that is, only a -single stream can pass through a media link and a media pad. Thus each pad -contains a format configuration for that single stream. In some cases a subdev -can do stream processing and split a stream into two or compose two streams -into one, but the inputs and outputs for the subdev are still a single stream -per pad. +Simple V4L2 sub-devices do not support multiple, unrelated video streams, +and only a single stream can pass through a media link and a media pad. +Thus each pad contains a format and selection configuration for that +single stream. A subdev can do stream processing and split a stream into +two or compose two streams into one, but the inputs and outputs for the +subdev are still a single stream per pad. Some hardware, e.g. MIPI CSI-2, support multiplexed streams, that is, multiple data streams are transmitted on the same bus, which is represented by a media
@@ -535,38 +537,62 @@ Understanding streams A stream is a stream of content (e.g. pixel data or metadata) flowing through the media pipeline from a source (e.g. a sensor) towards the final sink (e.g. a receiver and demultiplexer in a SoC). Each media link carries all the enabled -streams from one end of the link to the other, and subdevices have routing +streams from one end of the link to the other, and sub-devices have routing tables which describe how the incoming streams from sink pads are routed to the source pads. -A stream ID (often just "stream") is a media link-local identifier for a stream. -In other words, a particular stream ID must exist on both sides of a media +A stream ID is a media pad-local identifier for a stream. Streams IDs of +the same stream must be equal on both ends of a link. In other words, +a particular stream ID must exist on both sides of a media link, but another stream ID can be used for the same stream at the other side -of the subdevice. +of the sub-device. + +A stream at a specific point in the media pipeline is identified by the +sub-device and a (pad, stream) pair. For sub-devices that do not support +multiplexed streams the 'stream' field is always 0. + +Interaction between routes, streams, formats and selections +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The addition of streams to the V4L2 sub-device interface moves the sub-device +formats and selections from pads to (pad, stream) pairs. Besides the +usual pad, also the stream ID needs to be provided for setting formats and +selections. The order of configuring formats and selections along a stream is +the same as without streams (see :ref:`format-propagation`). + +Instead of the sub-device wide merging of streams from all sink pads +towards all source pads, data flows for each route are separate from each +other. Any number of routes from streams on sink pads towards streams on +source pads is allowed, to the extent supported by drivers. For every +stream on a source pad, however, only a single route is allowed. -A stream at a specific point in the media pipeline is identified with the -subdev and a (pad, stream) pair. For subdevices that do not support -multiplexed streams the 'stream' is always 0. +Any configurations of a stream within a pad, such as format or selections, +are independent of similar configurations on other streams. This is +subject to change in the future. Configuring streams ^^^^^^^^^^^^^^^^^^^ -The configuration of the streams is done individually for each subdevice and -the validity of the streams between subdevices is validated when the pipeline +The configuration of the streams is done individually for each sub-device and +the validity of the streams between sub-devices is validated when the pipeline is started. There are three steps in configuring the streams: -1) Set up links. Connect the pads between subdevices using the :ref:`Media +1) Set up links. Connect the pads between sub-devices using the :ref:`Media Controller API <media_controller>` -2) Routing. The routing table for the subdevice must be set with +2) Streams. Streams are declared and their routing is configured by +setting the routing table for the sub-device using :ref:`VIDIOC_SUBDEV_S_ROUTING <VIDIOC_SUBDEV_G_ROUTING>` ioctl. Note that -setting the routing table will reset all the stream configurations in a media -entity. +setting the routing table will reset formats and selections in the +sub-device to default values. -3) Configure streams. Each route endpoint must be configured -with :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>`. +3) Configure formats and selections. Formats and selections of each stream +are configured separately as documented for plain sub-devices in +:ref:`format-propagation`. The stream ID is set to the same stream ID +associated with either sink or source pads of routes configured using the +:ref:`VIDIOC_SUBDEV_S_ROUTING <VIDIOC_SUBDEV_G_ROUTING>` ioctl. Multiplexed streams setup example ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -586,7 +612,7 @@ A simple example of a multiplexed stream setup might be as follows: - DMA Engines in the SoC (DMA Engine), one for each stream. Each DMA engine is connected to a single source pad in the receiver. -The sensors, the bridge and the receiver are modeled as V4L2 subdevices, +The sensors, the bridge and the receiver are modeled as V4L2 sub-devices, exposed to userspace via /dev/v4l-subdevX device nodes. The DMA engines are modeled as V4L2 devices, exposed to userspace via /dev/videoX nodes.
@@ -596,7 +622,7 @@ To configure this pipeline, the userspace must take the following steps: bridge to the receiver, and the receiver to the DMA engines. This step does not differ from normal non-multiplexed media controller setup. -2) Configure routing. +2) Configure routing .. flat-table:: Bridge routing table :header-rows: 1
@@ -630,14 +656,14 @@ not differ from normal non-multiplexed media controller setup. - V4L2_SUBDEV_ROUTE_FL_ACTIVE - Pixel data stream from Sensor B -3) Configure streams +3) Configure formats and selections -After configuring the routing table, the next step is configuring the streams. -This step is similar to configuring the pads in a non-multiplexed streams -setup, with the difference that we need to configure each (pad, stream) pair -(i.e. route endpoint) instead of just a pad. +After configuring routing, the next step is configuring the formats and +selections for the streams. This is similar to performing this step without +streams, with just one exception: the ``stream`` field needs to be assigned +to the value of the stream ID. A common way to accomplish this is to start from the sensors and propagate the configurations along the stream towards the receiver, using :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctls to configure each -stream endpoint in each subdev. +stream endpoint in each sub-device.
Signature · 1 line
2.30.2