Generic line based metadata support, internal pads
Subsystems (3)
Sakari Ailus
maintainerv7 cover Generic line based metadata support, internal pads
Hi folks,
Here are a few patches to add support generic, line based metadata as well as internal pads. While the amount of code is not very large, to the contrary it is quite small actually IMO, I presume what this is about and why it is being proposed requires some explaining.
Metadata mbus codes and formats have existed for some time in V4L2. They however have been only used by drivers that produce the data itself and effectively this metadata has always been statistics of some sort (at least when it comes to ISPs). What is different here is that we intend to add support for metadata originating from camera sensors.
Camera sensors produce different kinds of metadata, embedded data (usually register address--value pairs used to capture the frame, in a more or less sensor specific format), histograms (in a very sensor specific format), dark pixels etc. The number of these formats is probably going to be about as large as image data formats if not larger, as the image data formats are much better standardised but a smaller subset of them will be supported by V4L2, at least initially but possibly much more in the long run.
Having this many device specific formats would be a major problem for all the other drivers along that pipeline (not to mention the users of those drivers), including bridge (e.g. CSI-2 to parallel) but especially CSI-2 receiver drivers that have DMA: the poor driver developer would not only need to know camera sensor specific formats but to choose the specific packing of that format suitable for the DMA used by the hardware. It is unlikely many of these would ever get tested while being present on the driver API. Also adding new sensors with new embedded data formats would involve updating all bridge and CSI-2 receiver drivers. I don't expect this to be a workable approach.
Instead what I'm proposing is to use specific metadata formats on the sensor devices only, on internal pads (more about those soon) of the sensors, only visible in the UAPI, and then generic mbus formats along the pipeline and finally generic V4L2 metadata formats on the DMAs (specific to bit depth and packing). This would unsnarl the two, defining what data there is (specific mbus code) and how that is transported and packed (generic mbus codes and V4L2 formats).
The user space would be required to "know" the path of that data from the sensor's internal pad to the V4L2 video node. I do not see this as these devices require at least some knowledge of the pipeline, i.e. hardware at hand. Separating what the data means and how it is packed may even be
beneficial: it allows separating code that interprets the data (sensor internal mbus code) from the code that accesses it (packing).
These formats are in practice line based, meaning that there may be padding at the end of the line, depending on the bus as well as the DMA. If non-line based formats are needed, it is always possible to set the "height" field to 1.
The internal (source) pads are an alternative to source routes [1]. The source routes were not universally liked and I do have to say I like re-using existing interface concepts (pads and everything you can do with pads, including access format, selections etc.) wherever it makes sense, instead of duplicating functionality.
Effectively internal source pads behave mostly just like sink pads, but they describe a flow of data that originates from a sub-device instead of arriving to a sub-device. The SUBDEV_S_ROUTING IOCTLs are used to enable and disable routes from internal source pads to sub-device's source pads. The subdev format IOCTLs are usable, too, so one can find which subdev format is available on given internal source pad.
This set depends on these patches:
<URL:https://lore.kernel.org/linux-media/[email protected]/T/#t>
I've also pushed these here and I'll keep updating the branch, I've also included untested OV2740 patches:
<URL:https://git.linuxtv.org/sailus/media_tree.git/log/?h=metadata>
Questions and comments are most welcome.
Preliminary media-ctl and yavta patches can be found here:
<URL:https://git.retiisi.eu/?p=~sailus/yavta.git;a=shortlog;h=refs/heads/metadata> <URL:https://git.retiisi.eu/?p=~sailus/v4l-utils.git;a=shortlog;h=refs/heads/metadata>
I have used IMX219 as an example on routing in a sensor driver in this version. I also hope I can add OV2740 support in the next version after testing the patches.
[1] <URL:https://lore.kernel.org/linux-media/[email protected]/>
since v6:
- Improve embedded data UAPI documentation on camera sensors.
- Improve wording of stream glossary entry.
- Improve internal pad flag documentation.
- Fix definition of "data unit" and remove an extra "only" in INTERNAL pad flag description (1st patch).
- Use IMX219 driver in examples consistently.
- Remove the CSI-2 to parallel bridge from the example to simplify the example.
- Minor rewording of some parts of the routing examples.
- Rebase on unified sub-device state information access functions:
<URL:https://lore.kernel.org/linux-media/[email protected]/T/#t>
- In CCS driver, do not maintain current active configuration in driver's device context struct (apart from mbus codes). Rely on sub-device state locking and clean up the code. (Multiple patches towards the end of the set.)
- Arrange the CCS patches early in the set towards the end of the set.
- Move the patch enabling streams API to the end of the set.
- Rework IOCTL argument copying condition for [GS]_ROUTING).
- Handle copying back routes in S_ROUTING, do not rely on G_ROUTING IOCTL implementation.
- Rebase on metadata preparation patchset v6:
<URL:https://lore.kernel.org/linux-media/[email protected]/T/#t>.
since v5:
- Rebase on new set of preparation patches.
- Switch CCS driver from s_stream to enable_streams/disable_streams. Keep streaming state information --- the sensor remains in streaming state if any of the streams is enabled.
- Fix setting mbus code on embedded data in get_frame_desc() op in the CCS driver.
since v4:
- Add a patch to acquire two sub-device states that may use the same lock.
- Add a patch for CCS driver to remove ccs_get_crop_compose() helper.
- Add a patch for CCS driver moving acquiring and releasing the mutex to the s_stream callback.
- Add a patch for CCS driver to rely on sub-device state locking using a single driver-provided lock.
- Fixed calculating minimum number of routes in copying the routes (thanks, Laurent).
- Moved a label in S_ROUTING handling to make Clang happy (hopefully).
- Fixed setting emb_data_ctrl register for CCS embedded data support.
- Rebase on Laurent's cleanup patches.
- Wrap a few long lines.
- Write in embedded data documentation sensor drivers generally don't allow configuring it.
since v3:
- Separate preparation patches from this set.
- Add a definition for "Data unit", a pixel that is not image data and use it instead in format documentation.
- Fix more numbered lists in dev-subdev.rst.
- Remove a redundant definition for V4L2_META_FMT_GENERIC_CSI2_2_24 --- V4L2_META_FMT_GENERIC_CSI2_12 can be used instead.
- Use "X" instead of "p" to denote padding in format documentation.
- Use IMX219 in examples instead of CCS.
- Document that the generic V4L2 CSI-2 metadata formats use padding defined in CSI-2 spec and packing defined in CCS spec.
- Add patches to align [GS]_ROUTING behaviour with V4L2. This means mainly returning configured routes as part of S_ROUTING as well. "len_routes" field is added to denote the length of the array and having more routes than fits in the array is no longer an error. Also added more reserved fields.
- Added trivial support for S_ROUTING (via G_ROUTING implementation) for use in drivers with static-only routes.
- Added helper functions to obtain mbus format as well as crop and compose rectangles that are streams-independent.
- Added a patch to define generic CSI-2 long packet types.
- Removed MEDIA_BUS_FMT_IS_META() macro. It didn't seem useful in the end.
- Use a single CCS embedded data format. The bit depth can be selected using the meta stream on the source pad.
- Fix mbus code numbers (there were holes due to removed redundant formats).
- Fix generic mbus code documentation (byte was being used instead of bit).
- Fix spelling of "length".
- Added a patch to remove v4l2_subdev_enable_streams_api that disables streams API. This should be merged once libcamera support for streams works nicely.
- Don't use strings in printing frame descriptor flags.
- Warn on string truncation in printing frame descriptor.
since v2:
- Add a better example, with formats.
- Add CCS static data media bus codes.
- Added an example demonstrating the use of internal pads. --- Is the level of detail enough for the purpose?
- Improved documentation.
- Added a macro to tell whether a format is a metadata format. (Documentation could be added.)
- A small ReST syntax fix in the same section.
- Drop leftovers of a patch checking for the INTERNAL_SOURCE flag.
since v1:
- Make the new pad flag just "INTERNAL", requiring either SINK or SOURCE pad flag to accompany it. Removed the union in struct v4l2_subdev_route.
- Add the term "stream" to MC glossary.
- Improved and fixed documentation (according to comments).
- Note these formats are little endian.
- Remove 1X8 from the names of the mbus codes. These formats have generally 8 bits per pixel.
- Fix mbus code numbering (had holes in RFC).
- Add new metadata fields to debug prints.
- Fix a minor documentation build issue.
Sakari Ailus (31):
media: mc: Add INTERNAL pad flag
media: Documentation: Add "stream" into glossary
media: uapi: Add generic serial metadata mbus formats
media: uapi: Document which mbus format fields are valid for metadata
media: uapi: Add generic 8-bit metadata format definitions
media: v4l: Support line-based metadata capture
media: Documentation: Additional streams generally don't harm capture
media: Documentation: Document embedded data guidelines for camera
sensors
media: Documentation: v4l: Document internal source pads
media: Documentation: Document S_ROUTING behaviour
media: v4l: subdev: Add a function to lock two sub-device states, use
it
media: v4l: subdev: Move G_ROUTING handling below S_ROUTING
media: v4l: subdev: Copy argument back to user also for S_ROUTING
media: v4l: subdev: Add len_routes field to struct v4l2_subdev_routing
media: v4l: subdev: Return routes set using S_ROUTING
media: v4l: subdev: Allow a larger number of routes than there's room
for
media: v4l: subdev: Add trivial set_routing support
media: ccs: No need to set streaming to false in power off
media: ccs: Use {enable,disable}_streams operations
media: ccs: Track streaming state
media: ccs: Move ccs_validate_csi_data_format up
media: ccs: Support frame descriptors
media: uapi: ccs: Add media bus code for MIPI CCS embedded data
media: ccs: Add support for embedded data stream
media: ccs: Remove ccs_get_crop_compose helper
media: ccs: Rely on sub-device state locking
media: ccs: Compute binning configuration from sub-device state
media: ccs: Compute scaling configuration from sub-device state
media: ccs: Remove which parameter from ccs_propagate
media: Documentation: ccs: Document routing
media: uapi: v4l: subdev: Enable streams API
.../media/drivers/camera-sensor.rst | 29 + .../userspace-api/media/drivers/ccs.rst | 34 +- .../userspace-api/media/glossary.rst | 15 + .../media/mediactl/media-types.rst | 8 + .../userspace-api/media/v4l/dev-meta.rst | 15 + .../userspace-api/media/v4l/dev-subdev.rst | 177 ++- .../userspace-api/media/v4l/meta-formats.rst | 1 + .../media/v4l/metafmt-generic.rst | 304 +++++ .../media/v4l/subdev-formats.rst | 288 +++++ .../media/v4l/vidioc-enum-fmt.rst | 7 + .../media/v4l/vidioc-subdev-g-routing.rst | 40 +- .../media/videodev2.h.rst.exceptions | 1 + drivers/media/i2c/ccs/ccs-core.c | 1014 +++++++++++------ drivers/media/i2c/ccs/ccs-quirk.h | 7 + drivers/media/i2c/ccs/ccs.h | 27 +- drivers/media/mc/mc-entity.c | 10 +- drivers/media/v4l2-core/v4l2-ioctl.c | 25 +- drivers/media/v4l2-core/v4l2-subdev.c | 109 +- include/media/v4l2-subdev.h | 42 + include/uapi/linux/media-bus-format.h | 12 + include/uapi/linux/media.h | 1 + include/uapi/linux/v4l2-mediabus.h | 18 +- include/uapi/linux/v4l2-subdev.h | 8 +- include/uapi/linux/videodev2.h | 18 + 24 files changed, 1784 insertions(+), 426 deletions(-) create mode 100644 Documentation/userspace-api/media/v4l/metafmt-generic.rst
base-commit: 5b1efab1ccf0358c33605661492bcd478bd3b677
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 1/31 media: mc: Add INTERNAL pad flag
Internal source pads will be used as routing endpoints in V4L2 [GS]_ROUTING IOCTLs, to indicate that the stream begins in the entity. Internal source pads are pads that have both SINK and INTERNAL flags set.
Also prevent creating links to pads that have been flagged as internal and initialising SOURCE pads with INTERNAL flag set.
--- .../userspace-api/media/mediactl/media-types.rst | 8 ++++++++ drivers/media/mc/mc-entity.c | 10 ++++++++-- include/uapi/linux/media.h | 1 + 3 files changed, 17 insertions(+), 2 deletions(-)
Documentation/userspace-api/media/mediactl/media-types.rst +8 -0
@@ -361,6 +361,7 @@ Types and flags used to represent the media graph elements .. _MEDIA-PAD-FL-SINK: .. _MEDIA-PAD-FL-SOURCE: .. _MEDIA-PAD-FL-MUST-CONNECT: +.. _MEDIA-PAD-FL-INTERNAL: .. flat-table:: Media pad flags :header-rows: 0
@@ -382,6 +383,13 @@ Types and flags used to represent the media graph elements when this flag isn't set; the absence of the flag doesn't imply there is none. + * - ``MEDIA_PAD_FL_INTERNAL`` + - The internal flag indicates an internal pad that has no external + connections. Such a pad shall not be connected with a link. + + The internal flag may currently be present only in a source pad where + it indicates that the :ref:``stream <media-glossary-stream>`` + originates from within the entity. One and only one of ``MEDIA_PAD_FL_SINK`` and ``MEDIA_PAD_FL_SOURCE`` must be set for every pad.
drivers/media/mc/mc-entity.c +8 -2
@@ -213,7 +213,9 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads, iter->index = i++;if (hweight32 (iter->flags & (MEDIA_PAD_FL_SINK | -MEDIA_PAD_FL_SOURCE )) !=1 ) { +MEDIA_PAD_FL_SOURCE )) !=1 || + (iter->flags &MEDIA_PAD_FL_INTERNAL && + !(iter->flags &MEDIA_PAD_FL_SINK ))) { ret = -EINVAL ;break ; }
@@ -1075,7 +1077,8 @@ int media_get_pad_index(struct media_entity *entity, u32 pad_type,for (i =0 ; i < entity->num_pads; i++) {if ((entity->pads[i].flags & - (MEDIA_PAD_FL_SINK |MEDIA_PAD_FL_SOURCE )) != pad_type) + (MEDIA_PAD_FL_SINK |MEDIA_PAD_FL_SOURCE | +MEDIA_PAD_FL_INTERNAL )) != pad_type)continue ;if (entity->pads[i].sig_type == sig_type)
@@ -1100,6 +1103,9 @@ media_create_pad_link(struct media_entity *source, u16 source_pad,return -EINVAL ;if (WARN_ON (!(sink->pads[sink_pad].flags &MEDIA_PAD_FL_SINK )))return -EINVAL ; +if (WARN_ON (source->pads[source_pad].flags &MEDIA_PAD_FL_INTERNAL ) || +WARN_ON (source->pads[sink_pad].flags &MEDIA_PAD_FL_INTERNAL )) +return -EINVAL ; link =media_add_link (&source->links);if (link ==NULL )
include/uapi/linux/media.h +1 -0
@@ -208,6 +208,7 @@ struct media_entity_desc {#define MEDIA_PAD_FL_SINK (1U << 0)#define MEDIA_PAD_FL_SOURCE (1U << 1)#define MEDIA_PAD_FL_MUST_CONNECT (1U << 2) +#define MEDIA_PAD_FL_INTERNAL (1U << 3)struct media_pad_desc {__u32 entity;/* entity ID */
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 2/31 media: Documentation: Add "stream" into glossary
In reply to Sakari AilusAdd term "stream" to the glossary of the Media subsystem documentation.
--- Documentation/userspace-api/media/glossary.rst | 6 ++++++ 1 file changed, 6 insertions(+)
Documentation/userspace-api/media/glossary.rst +6 -0
@@ -173,6 +173,12 @@ Glossary An integrated circuit that integrates all components of a computer or other electronic systems. +_media-glossary-stream: + Stream + A distinct flow of data (image data or metadata) from an initial source + to a final sink. The initial source may be e.g. an image sensor and the + final sink e.g. a memory buffer. + V4L2 API **V4L2 userspace API**
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 3/31 media: uapi: Add generic serial metadata mbus formats
In reply to Sakari AilusAdd generic serial metadata mbus formats. These formats describe data width and packing but not the content itself. The reason for specifying such formats is that the formats as such are fairly device specific but they are still handled by CSI-2 receiver drivers that should not be aware of device specific formats. What makes generic metadata formats possible is that these formats are parsed by software only, after capturing the data to system memory.
Also add a definition for "Data unit" to cover what is essentially a pixel but is not image data.
--- .../userspace-api/media/glossary.rst | 9 + .../media/v4l/subdev-formats.rst | 258 ++++++++++++++++++ include/uapi/linux/media-bus-format.h | 9 + 3 files changed, 276 insertions(+)
Documentation/userspace-api/media/glossary.rst +9 -0
@@ -25,6 +25,15 @@ Glossary See :ref:`cec`. +.. _media-glossary-data-unit: + + Data unit + + Unit of data transported by a bus. On parallel buses, the data unit + consists of one or more related samples while on serial buses the data + unit is logical. If the data unit is image data, it may also be called a + pixel. + Device Driver Part of the Linux Kernel that implements support for a hardware component.
Documentation/userspace-api/media/v4l/subdev-formats.rst +258 -0
261 lines not on this page · raw message
include/uapi/linux/media-bus-format.h +9 -0
@@ -173,4 +173,13 @@*/ #define MEDIA_BUS_FMT_METADATA_FIXED 0x7001 +/* Generic line based metadata formats for serial buses. Next is 0x8008. */ +#define MEDIA_BUS_FMT_META_8 0x8001 +#define MEDIA_BUS_FMT_META_10 0x8002 +#define MEDIA_BUS_FMT_META_12 0x8003 +#define MEDIA_BUS_FMT_META_14 0x8004 +#define MEDIA_BUS_FMT_META_16 0x8005 +#define MEDIA_BUS_FMT_META_20 0x8006 +#define MEDIA_BUS_FMT_META_24 0x8007 +#endif /* __LINUX_MEDIA_BUS_FORMAT_H */
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 4/31 media: uapi: Document which mbus format fields are valid for metadata
In reply to Sakari AilusNow that metadata mbus formats have been added, it is necessary to define which fields in struct v4l2_mbus_format are applicable to them (not many).
--- include/uapi/linux/v4l2-mediabus.h | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-)
include/uapi/linux/v4l2-mediabus.h +12 -6
@@ -19,12 +19,18 @@* @width: image width * @height: image height * @code: data format code (from enum v4l2_mbus_pixelcode) -* @field: used interlacing type (from enum v4l2_field) -* @colorspace: colorspace of the data (from enum v4l2_colorspace) -* @ycbcr_enc: YCbCr encoding of the data (from enum v4l2_ycbcr_encoding) -* @hsv_enc: HSV encoding of the data (from enum v4l2_hsv_encoding) -* @quantization: quantization of the data (from enum v4l2_quantization) -* @xfer_func: transfer function of the data (from enum v4l2_xfer_func) +* @field: used interlacing type (from enum v4l2_field), zero on metadata +* mbus codes +* @colorspace: colorspace of the data (from enum v4l2_colorspace), zero on +* metadata mbus codes +* @ycbcr_enc: YCbCr encoding of the data (from enum v4l2_ycbcr_encoding), zero +* on metadata mbus codes +* @hsv_enc: HSV encoding of the data (from enum v4l2_hsv_encoding), zero on +* metadata mbus codes +* @quantization: quantization of the data (from enum v4l2_quantization), zero +* on metadata mbus codes +* @xfer_func: transfer function of the data (from enum v4l2_xfer_func), zero +* on metadata mbus codes * @flags: flags (V4L2_MBUS_FRAMEFMT_*) * @reserved: reserved bytes that can be later used */
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 5/31 media: uapi: Add generic 8-bit metadata format definitions
In reply to Sakari AilusGeneric 8-bit metadata formats define the in-memory data layout but not the format of the data itself. The reasoning for having such formats is to allow CSI-2 receiver drivers to receive and DMA drivers to write the data to memory without knowing a large number of device specific formats.
These formats may be used only in conjunction of a Media controller pipeline where the internal pad of the source sub-device defines the specific format of the data (using an mbus code).
--- .../userspace-api/media/v4l/dev-subdev.rst | 2 + .../userspace-api/media/v4l/meta-formats.rst | 1 + .../media/v4l/metafmt-generic.rst | 304 ++++++++++++++++++ drivers/media/v4l2-core/v4l2-ioctl.c | 7 + include/uapi/linux/videodev2.h | 8 + 5 files changed, 322 insertions(+) create mode 100644 Documentation/userspace-api/media/v4l/metafmt-generic.rst
Documentation/userspace-api/media/v4l/dev-subdev.rst +2 -0
@@ -506,6 +506,8 @@ source pads. subdev-formats +.. _subdev-routing: + Streams, multiplexed media pads and internal routing ----------------------------------------------------
Documentation/userspace-api/media/v4l/meta-formats.rst +1 -0
@@ -19,3 +19,4 @@ These formats are used for the :ref:`metadata` interface only. metafmt-vsp1-hgo metafmt-vsp1-hgt metafmt-vivid + metafmt-generic
Documentation/userspace-api/media/v4l/metafmt-generic.rst new file +304 -0
304 lines not on this page · raw message
drivers/media/v4l2-core/v4l2-ioctl.c +7 -0
@@ -1452,6 +1452,13 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)case V4L2_PIX_FMT_Y210 : descr ="10-bit YUYV Packed" ;break ;case V4L2_PIX_FMT_Y212 : descr ="12-bit YUYV Packed" ;break ;case V4L2_PIX_FMT_Y216 : descr ="16-bit YUYV Packed" ;break ; +case V4L2_META_FMT_GENERIC_8 : descr ="8-bit Generic Metadata" ;break ; +case V4L2_META_FMT_GENERIC_CSI2_10 : descr ="8b Generic Meta, 10b CSI-2" ;break ; +case V4L2_META_FMT_GENERIC_CSI2_12 : descr ="8b Generic Meta, 12b CSI-2" ;break ; +case V4L2_META_FMT_GENERIC_CSI2_14 : descr ="8b Generic Meta, 14b CSI-2" ;break ; +case V4L2_META_FMT_GENERIC_CSI2_16 : descr ="8b Generic Meta, 16b CSI-2" ;break ; +case V4L2_META_FMT_GENERIC_CSI2_20 : descr ="8b Generic Meta, 20b CSI-2" ;break ; +case V4L2_META_FMT_GENERIC_CSI2_24 : descr ="8b Generic Meta, 24b CSI-2" ;break ;default :/* Compressed formats */
include/uapi/linux/videodev2.h +8 -0
@@ -839,6 +839,14 @@ struct v4l2_pix_format {#define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P')/* Rockchip ISP1 3A Parameters */ #define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S')/* Rockchip ISP1 3A Statistics */ +#define V4L2_META_FMT_GENERIC_8 v4l2_fourcc('M', 'E', 'T', '8')/* Generic 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_10 v4l2_fourcc('M', 'C', '1', 'A')/* 10-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_12 v4l2_fourcc('M', 'C', '1', 'C')/* 12-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_14 v4l2_fourcc('M', 'C', '1', 'E')/* 14-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_16 v4l2_fourcc('M', 'C', '1', 'G')/* 16-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_20 v4l2_fourcc('M', 'C', '1', 'K')/* 20-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_24 v4l2_fourcc('M', 'C', '1', 'O')/* 24-bit CSI-2 packed 8-bit metadata */ +/* priv field value to indicates that subsequent fields are valid. */ #define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 6/31 media: v4l: Support line-based metadata capture
In reply to Sakari Ailusmany camera sensors, among other devices, transmit embedded data and image data for each CSI-2 frame. This embedded data typically contains register configuration of the sensor that has been used to capture the image data of the same frame.
The embedded data is received by the CSI-2 receiver and has the same properties as the image data, including that it is line based: it has width, height and bytesperline (stride).
Add these fields to struct v4l2_meta_format and document them.
Also add V4L2_FMT_FLAG_META_LINE_BASED to tell a given format is line-based i.e. these fields of struct v4l2_meta_format are valid for it.
--- .../userspace-api/media/v4l/dev-meta.rst | 15 +++++++++++++++ .../userspace-api/media/v4l/vidioc-enum-fmt.rst | 7 +++++++ .../media/videodev2.h.rst.exceptions | 1 + drivers/media/v4l2-core/v4l2-ioctl.c | 5 +++-- include/uapi/linux/videodev2.h | 10 ++++++++++ 5 files changed, 36 insertions(+), 2 deletions(-)
Documentation/userspace-api/media/v4l/dev-meta.rst +15 -0
@@ -65,3 +65,18 @@ to 0. - ``buffersize`` - Maximum buffer size in bytes required for data. The value is set by the driver. + * - __u32 + - ``width`` + - Width of a line of metadata in Data units. Valid when + :c:type`v4l2_fmtdesc` flag ``V4L2_FMT_FLAG_META_LINE_BASED`` is set, + otherwise zero. See :c:func:`VIDIOC_ENUM_FMT`. + * - __u32 + - ``height`` + - Number of rows of metadata. Valid when :c:type`v4l2_fmtdesc` flag + ``V4L2_FMT_FLAG_META_LINE_BASED`` is set, otherwise zero. See + :c:func:`VIDIOC_ENUM_FMT`. + * - __u32 + - ``bytesperline`` + - Offset in bytes between the beginning of two consecutive lines. Valid + when :c:type`v4l2_fmtdesc` flag ``V4L2_FMT_FLAG_META_LINE_BASED`` is + set, otherwise zero. See :c:func:`VIDIOC_ENUM_FMT`.
Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst +7 -0
@@ -227,6 +227,13 @@ the ``mbus_code`` field is handled differently: The application can ask to configure the quantization of the capture device when calling the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl with :ref:`V4L2_PIX_FMT_FLAG_SET_CSC <v4l2-pix-fmt-flag-set-csc>` set. + * - ``V4L2_FMT_FLAG_META_LINE_BASED`` + - 0x0200 + - The metadata format is line-based. In this case the ``width``, + ``height`` and ``bytesperline`` fields of :c:type:`v4l2_meta_format` are + valid. The buffer consists of ``height`` lines, each having ``width`` + Data units of data and offset (in bytes) between the beginning of each + two consecutive lines is ``bytesperline``. Return Value ============
Documentation/userspace-api/media/videodev2.h.rst.exceptions +1 -0
@@ -215,6 +215,7 @@ replace define V4L2_FMT_FLAG_CSC_XFER_FUNC fmtdesc-flags replace define V4L2_FMT_FLAG_CSC_YCBCR_ENC fmtdesc-flags replace define V4L2_FMT_FLAG_CSC_HSV_ENC fmtdesc-flags replace define V4L2_FMT_FLAG_CSC_QUANTIZATION fmtdesc-flags +replace define V4L2_FMT_FLAG_META_LINE_BASED fmtdesc-flags # V4L2 timecode types replace define V4L2_TC_TYPE_24FPS timecode-type
drivers/media/v4l2-core/v4l2-ioctl.c +3 -2
@@ -343,8 +343,9 @@ static void v4l_print_format(const void *arg, bool write_only)case V4L2_BUF_TYPE_META_OUTPUT : meta = &p->fmt.meta; pixelformat = meta->dataformat; -pr_cont (", dataformat=%p4cc, buffersize=%u\n" , - &pixelformat, meta->buffersize); +pr_cont (", dataformat=%p4cc, buffersize=%u, width=%u, height=%u, bytesperline=%u\n" , + &pixelformat, meta->buffersize, meta->width, + meta->height, meta->bytesperline);break ; } }
include/uapi/linux/videodev2.h +10 -0
@@ -877,6 +877,7 @@ struct v4l2_fmtdesc {#define V4L2_FMT_FLAG_CSC_YCBCR_ENC 0x0080#define V4L2_FMT_FLAG_CSC_HSV_ENC V4L2_FMT_FLAG_CSC_YCBCR_ENC#define V4L2_FMT_FLAG_CSC_QUANTIZATION 0x0100 +#define V4L2_FMT_FLAG_META_LINE_BASED 0x0200/* Frame Size and frame rate enumeration */ /*
@@ -2420,10 +2421,19 @@ struct v4l2_sdr_format {* struct v4l2_meta_format - metadata format definition * @dataformat: little endian four character code (fourcc) * @buffersize: maximum size in bytes required for data +* @width: number of data units of data per line (valid for line +* based formats only, see format documentation) +* @height: number of lines of data per buffer (valid for line based +* formats only) +* @bytesperline: offset between the beginnings of two adjacent lines in +* bytes (valid for line based formats only) */ struct v4l2_meta_format {__u32 dataformat;__u32 buffersize; +__u32 width; +__u32 height; +__u32 bytesperline; } __attribute__ ((packed));/**
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 7/31 media: Documentation: Additional streams generally don't harm capture
In reply to Sakari AilusHaving extra streams on the source end of the link that cannot be captured by the sink sub-device generally are not an issue, at least not on CSI-2 bus. Still document that there may be hardware specific limitations. For example on parallel bus this might not work on all cases.
--- Documentation/userspace-api/media/v4l/dev-subdev.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
Documentation/userspace-api/media/v4l/dev-subdev.rst +3 -3
@@ -529,9 +529,9 @@ the its sink pad and allows to route them individually to one of its source pads. Subdevice drivers that support multiplexed streams are compatible with -non-multiplexed subdev drivers, but, of course, require a routing configuration -where the link between those two types of drivers contains only a single -stream. +non-multiplexed subdev drivers. However, if the driver at the sink end of a link +does not support streams, then only the stream 0 on source end may be +captured. There may be additional hardware specific limitations. Understanding streams ^^^^^^^^^^^^^^^^^^^^^
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 8/31 media: Documentation: Document embedded data guidelines for camera sensors
In reply to Sakari AilusDocument how embedded data support should be implemented for camera sensors, and when and how CCS embedded data format should be referenced.
--- .../media/drivers/camera-sensor.rst | 29 +++++++++++++++++++ 1 file changed, 29 insertions(+)
Documentation/userspace-api/media/drivers/camera-sensor.rst +29 -0
@@ -102,3 +102,32 @@ register programming sequences shall initialize the :ref:`V4L2_CID_HFLIP values programmed by the register sequences. The default values of these controls shall be 0 (disabled). Especially these controls shall not be inverted, independently of the sensor's mounting rotation. + +Embedded data +------------- + +Many sensors, mostly raw sensors, support embedded data which is used to convey +the sensor configuration for the captured frame back to the host. While CSI-2 is +the most common bus used by such sensors, embedded data can be available on +other bus types as well. + +Embedded data support shall use an internal source pad (a pad that has +``MEDIA_PAD_FL_SINK <MEDIA-PAD-FL-SINK>`` and ``MEDIA_PAD_FL_INTERNAL +<MEDIA-PAD-FL-INTERNAL>`` flags set) and route to the external pad. If embedded +data output can be disabled in hardware, it should be possible to disable the +embedded data route via ``VIDIOC_SUBDEV_S_ROUTING`` IOCTL. + +In general, changing the embedded data format from the driver-configured values +is not supported. The height of the metadata is hardware specific and the width +is that (or less of that) of the image width, as configured on the pixel data +stream. + +CCS and non-CCS embedded data +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Embedded data which is compliant with CCS definitions shall use ``CCS embedded +data format <MEDIA-BUS-FMT-CCS-EMBEDDED>``. Device specific embedded data which +is compliant up to MIPI CCS embedded data levels 1 or 2 only shall refer to CCS +embedded data formats and document the level of conformance. The rest of the +device specific embedded data format shall be documented in the context of the +data format itself.
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 9/31 media: Documentation: v4l: Document internal source pads
In reply to Sakari AilusDocument internal source pads, pads that have both SINK and INTERNAL flags set. Use the IMX219 camera sensor as an example.
--- .../userspace-api/media/v4l/dev-subdev.rst | 145 ++++++++++++++++++ 1 file changed, 145 insertions(+)
Documentation/userspace-api/media/v4l/dev-subdev.rst +145 -0
@@ -553,6 +553,27 @@ A stream at a specific point in the media pipeline is identified by the sub-device and a (pad, stream) pair. For sub-devices that do not support multiplexed streams the 'stream' field is always 0. +.. _v4l2-subdev-internal-source-pads: + +Internal source pads and routing +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Cases where a single sub-device source pad is traversed by multiple streams, one +or more of which originate from within the sub-device itself, are special as +there is no external sink pad for such routes. In those cases, the sources of +the internally generated streams are represented by internal source pads, which +are sink pads that have the :ref:`MEDIA_PAD_FL_INTERNAL <MEDIA-PAD-FL-INTERNAL>` +pad flag set. + +Internal pads have all the properties of an external pad, including formats and +selections. The format in this case is the source format of the stream. An +internal pad always has a single stream only (0). + +Routes from an internal source pad to an external source pad are typically not +modifiable but they can be activated and deactivated using the +:ref:`V4L2_SUBDEV_ROUTE_FL_ACTIVE <v4l2-subdev-routing-flags>` flag, depending +on driver capabilities. + Interaction between routes, streams, formats and selections ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -668,3 +689,127 @@ To configure this pipeline, the userspace must take the following steps: the configurations along the stream towards the receiver, using :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctls to configure each stream endpoint in each sub-device. + +Internal pads setup example +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +A simple example of a multiplexed stream setup might be as follows: + +- An IMX219 camera sensor source sub-device, with one sink pad (0), one source pad + (1), an internal sink pad (2) that represents the source of embedded + data. There are two routes, one from the sink pad to the source, and another + from the internal sink pad to the source pad. Both streams are always active, + i.e. there is no need to separately enable the embedded data stream. The + sensor uses the CSI-2 bus. + +- A CSI-2 receiver in the SoC (Receiver). The receiver has a single sink pad + (pad 0), connected to the bridge, and two source pads (pads 1-2), going to the + DMA engine. The receiver demultiplexes the incoming streams to the source + pads. + +- DMA Engines in the SoC (DMA Engine), one for each stream. Each DMA engine is + connected to a single source pad in the receiver. + +The sensor, the bridge and the receiver are modeled as V4L2 sub-devices, +exposed to userspace via /dev/v4l-subdevX device nodes. The DMA engines are +modeled as V4L2 devices, exposed to userspace via /dev/videoX nodes. + +To configure this pipeline, the userspace must take the following steps: + +1) Set up media links between entities: connect the sensors to the bridge, + bridge to the receiver, and the receiver to the DMA engines. This step does + not differ from normal non-multiplexed media controller setup. + +2) Configure routing + +.. flat-table:: Camera sensor. There are no configurable routes. + :header-rows: 1 + + * - Sink Pad/Stream + - Source Pad/Stream + - Routing Flags + - Comments + * - 0/0 + - 1/0 + - V4L2_SUBDEV_ROUTE_FL_ACTIVE + - Pixel data stream from the sink pad + * - 2/0 + - 1/1 + - V4L2_SUBDEV_ROUTE_FL_ACTIVE + - Metadata stream from the internal sink pad + +.. flat-table:: Receiver routing table. Typically both routes need to be + explicitly set. + :header-rows: 1 + + * - Sink Pad/Stream + - Source Pad/Stream + - Routing Flags + - Comments + * - 0/0 + - 1/0 + - V4L2_SUBDEV_ROUTE_FL_ACTIVE + - Pixel data stream from camera sensor + * - 0/1 + - 2/0 + - V4L2_SUBDEV_ROUTE_FL_ACTIVE + - Metadata stream from camera sensor + +The options available in sensor's routing configuration are dictated by hardware +capabilities: typically camera sensors always produce an image data stream while +it may be possible to enable and disable the embedded data stream. + +3) Configure formats and selections + + This example assumes that the formats are propagated from sink pad to the + source pad as-is. The tables contain fields of both struct v4l2_subdev_format + and struct v4l2_mbus_framefmt. + +.. flat-table:: Formats set on the sub-devices. Bold values are set, others are + static or propagated. The order is aligned with configured + routes. + :header-rows: 1 + :fill-cells: + + * - Sub-device + - Pad/Stream + - Width + - Height + - Code + * - :rspan:`3` Camera sensor sub-device (IMX219) + - 1/0 + - 3296 + - 2480 + - MEDIA_BUS_FMT_SRGGB10 + * - 0/0 + - **3296** + - **2480** + - **MEDIA_BUS_FMT_SRGGB10** + * - 2/0 + - 3296 + - 2 + - MEDIA_BUS_FMT_IMX219_EMBEDDED + * - 1/1 + - 3296 + - 2 + - MEDIA_BUS_FMT_META_10 + * - :rspan:`3` Receiver + - 0/0 + - **3296** + - **2480** + - **MEDIA_BUS_FMT_SRGGB10** + * - 1/0 + - 3296 + - 2480 + - MEDIA_BUS_FMT_SRGGB10 + * - 0/1 + - **3296** + - **2** + - **MEDIA_BUS_FMT_META_10** + * - 2/0 + - 3296 + - 2 + - MEDIA_BUS_FMT_META_10 + +The embedded data format does not need to be configured as the format is +dictated by the pixel data format in this case.
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 10/31 media: Documentation: Document S_ROUTING behaviour
In reply to Sakari AilusDocument S_ROUTING behaviour for different devices.
Generally in devices that produce streams the streams are static and some can be enabled and disabled, whereas in devices that just transport them or write them to memory, more configurability is allowed. Document this.
--- .../userspace-api/media/v4l/dev-subdev.rst | 24 +++++++++++++++++++ 1 file changed, 24 insertions(+)
Documentation/userspace-api/media/v4l/dev-subdev.rst +24 -0
@@ -593,6 +593,30 @@ Any configurations of a stream within a pad, such as format or selections, are independent of similar configurations on other streams. This is subject to change in the future. +Device types and routing setup +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Different kinds of sub-devices have differing behaviour for route activation, +depending on the hardware. In all cases, however, only routes that have the +``V4L2_SUBDEV_STREAM_FL_ACTIVE`` flag set are active. + +Devices generating the streams may allow enabling and disabling some of the +routes or the configuration is fixed. If the routes can be disabled, not +declaring the routes (or declaring them without +``VIDIOC_SUBDEV_STREAM_FL_ACTIVE`` flag set) in ``VIDIOC_SUBDEV_S_ROUTING`` will +disable the routes while the sub-device driver retains the streams and their +configuration. The ``VIDIOC_SUBDEV_S_ROUTING`` will still return such routes +back to the user in the routes array, with the ``V4L2_SUBDEV_STREAM_FL_ACTIVE`` +flag unset. + +Devices transporting the streams almost always have more configurability with +respect to routing. Typically any route between the sub-device's sink and source +pads is possible, and multiple routes (usually up to certain limited number) may +be active simultaneously. For such devices, no routes are created by the driver +and user-created routes are fully replaced when ``VIDIOC_SUBDEV_S_ROUTING`` is +called on the sub-device. Such newly created routes have the device's default +configuration for format and selection rectangles. + Configuring streams ^^^^^^^^^^^^^^^^^^^
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 11/31 media: v4l: subdev: Add a function to lock two sub-device states, use it
In reply to Sakari AilusAdd two new functions, v4l2_subdev_lock_states() and v4l2_subdev_unclock_states(), to acquire and release the state of two sub-devices. They differ from calling v4l2_subdev_{un,}lock_state() so that if the two states share the same lock, the lock is acquired only once.
Also use the new functions in v4l2_subdev_link_validate().
--- drivers/media/v4l2-core/v4l2-subdev.c | 12 +++----- include/media/v4l2-subdev.h | 40 +++++++++++++++++++++++++++ 2 files changed, 44 insertions(+), 8 deletions(-)
drivers/media/v4l2-core/v4l2-subdev.c +4 -8
@@ -1377,17 +1377,13 @@ int v4l2_subdev_link_validate(struct media_link *link) states_locked = sink_state && source_state; -if (states_locked) { -v4l2_subdev_lock_state (sink_state); -v4l2_subdev_lock_state (source_state); - } +if (states_locked) +v4l2_subdev_lock_states (sink_state, source_state); ret =v4l2_subdev_link_validate_locked (link, states_locked); -if (states_locked) { -v4l2_subdev_unlock_state (sink_state); -v4l2_subdev_unlock_state (source_state); - } +if (states_locked) +v4l2_subdev_unlock_states (sink_state, source_state);return ret; }
include/media/v4l2-subdev.h +40 -0
@@ -1341,6 +1341,46 @@ static inline void v4l2_subdev_unlock_state(struct v4l2_subdev_state *state)mutex_unlock (state->lock); } +/** +* v4l2_subdev_lock_states - Lock two sub-device states +* @state1: One subdevice state +* @state2: The other subdevice state +* +* Locks the state of two sub-devices. +* +* The states must be unlocked with v4l2_subdev_unlock_states() after use. +* +* This differs from calling v4l2_subdev_lock_state() on both states so that if +* the states share the same lock, the lock is acquired only once (so no +* deadlock occurs). The caller is responsible for ensuring the locks will +* always be acquired in the same order. +*/ +static inline void v4l2_subdev_lock_states (struct v4l2_subdev_state *state1, +struct v4l2_subdev_state *state2) +{ +mutex_lock (state1->lock); +if (state1->lock != state2->lock) +mutex_lock (state2->lock); +} + +/** +* v4l2_subdev_unlock_states() - Unlock two sub-device states +* @state1: One subdevice state +* @state2: The other subdevice state +* +* Unlocks the state of two sub-devices. +* +* This differs from calling v4l2_subdev_unlock_state() on both states so that +* if the states share the same lock, the lock is released only once. +*/ +static inline void v4l2_subdev_unlock_states (struct v4l2_subdev_state *state1, +struct v4l2_subdev_state *state2) +{ +mutex_unlock (state1->lock); +if (state1->lock != state2->lock) +mutex_unlock (state2->lock); +} +/** * v4l2_subdev_get_unlocked_active_state() - Checks that the active subdev state * is unlocked and returns it
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 12/31 media: v4l: subdev: Move G_ROUTING handling below S_ROUTING
In reply to Sakari AilusMove G_ROUTING IOCTL handling below that of S_ROUTING. G_ROUTING implementation will soon needed in handling S_ROUTING as well.
--- drivers/media/v4l2-core/v4l2-subdev.c | 54 +++++++++++++-------------- 1 file changed, 27 insertions(+), 27 deletions(-)
drivers/media/v4l2-core/v4l2-subdev.c +27 -27
@@ -887,33 +887,6 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,case VIDIOC_SUBDEV_QUERYSTD :return v4l2_subdev_call (sd, video, querystd, arg); -case VIDIOC_SUBDEV_G_ROUTING : { -struct v4l2_subdev_routing *routing = arg; -struct v4l2_subdev_krouting *krouting; - -if (!v4l2_subdev_enable_streams_api) -return -ENOIOCTLCMD ; - -if (!(sd->flags &V4L2_SUBDEV_FL_STREAMS )) -return -ENOIOCTLCMD ; - -memset (routing->reserved,0 ,sizeof (routing->reserved)); - - krouting = &state->routing; - -if (routing->num_routes < krouting->num_routes) { - routing->num_routes = krouting->num_routes; -return -ENOSPC ; - } - -memcpy ((struct v4l2_subdev_route *)(uintptr_t )routing->routes, - krouting->routes, - krouting->num_routes *sizeof (*krouting->routes)); - routing->num_routes = krouting->num_routes; - -return 0 ; - } -case VIDIOC_SUBDEV_S_ROUTING : {struct v4l2_subdev_routing *routing = arg; struct v4l2_subdev_route *routes =
@@ -962,6 +935,33 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg, routing->which, &krouting); } +case VIDIOC_SUBDEV_G_ROUTING : { +struct v4l2_subdev_routing *routing = arg; +struct v4l2_subdev_krouting *krouting; + +if (!v4l2_subdev_enable_streams_api) +return -ENOIOCTLCMD ; + +if (!(sd->flags &V4L2_SUBDEV_FL_STREAMS )) +return -ENOIOCTLCMD ; + +memset (routing->reserved,0 ,sizeof (routing->reserved)); + + krouting = &state->routing; + +if (routing->num_routes < krouting->num_routes) { + routing->num_routes = krouting->num_routes; +return -ENOSPC ; + } + +memcpy ((struct v4l2_subdev_route *)(uintptr_t )routing->routes, + krouting->routes, + krouting->num_routes *sizeof (*krouting->routes)); + routing->num_routes = krouting->num_routes; + +return 0 ; + } +case VIDIOC_SUBDEV_G_CLIENT_CAP : {struct v4l2_subdev_client_capability *client_cap = arg;
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 13/31 media: v4l: subdev: Copy argument back to user also for S_ROUTING
In reply to Sakari AilusAs the user needs to know what went wrong for S_ROUTING, copy array arguments back to the user.
--- drivers/media/v4l2-core/v4l2-ioctl.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
drivers/media/v4l2-core/v4l2-ioctl.c +6 -3
@@ -3415,11 +3415,14 @@ video_usercopy(struct file *file, unsigned int orig_cmd, unsigned long arg,* FIXME: subdev IOCTLS are partially handled here and partially in * v4l2-subdev.c and the 'always_copy' flag can only be set for IOCTLS * defined here as part of the 'v4l2_ioctls' array. As -* VIDIOC_SUBDEV_G_ROUTING needs to return results to applications even -* in case of failure, but it is not defined here as part of the +* VIDIOC_SUBDEV_[GS]_ROUTING needs to return results to applications +* even in case of failure, but it is not defined here as part of the * 'v4l2_ioctls' array, insert an ad-hoc check to address that. */ -if (err <0 && !always_copy && cmd !=VIDIOC_SUBDEV_G_ROUTING ) +if (cmd ==VIDIOC_SUBDEV_G_ROUTING || cmd ==VIDIOC_SUBDEV_S_ROUTING ) + always_copy = true; + +if (err <0 && !always_copy) goto out;if (has_array_args) {
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 14/31 media: v4l: subdev: Add len_routes field to struct v4l2_subdev_routing
In reply to Sakari AilusThe len_routes field is used to tell the size of the routes array in struct v4l2_subdev_routing. This way the number of routes returned from S_ROUTING IOCTL may be larger than the number of routes provided, in case there are more routes returned by the driver.
Note that this changes the (now-disabled) UAPI, users must be updated.
--- .../media/v4l/vidioc-subdev-g-routing.rst | 31 ++++++++++++------- drivers/media/v4l2-core/v4l2-ioctl.c | 4 +-- drivers/media/v4l2-core/v4l2-subdev.c | 6 +++- include/media/v4l2-subdev.h | 2 ++ include/uapi/linux/v4l2-subdev.h | 8 +++-- 5 files changed, 34 insertions(+), 17 deletions(-)
Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst +20 -11
@@ -46,20 +46,26 @@ with the ``VIDIOC_SUBDEV_S_ROUTING`` ioctl, by adding or removing routes and setting or clearing flags of the ``flags`` field of a struct :c:type:`v4l2_subdev_route`. -All stream configurations are reset when ``VIDIOC_SUBDEV_S_ROUTING`` is called. This -means that the userspace must reconfigure all streams after calling the ioctl -with e.g. ``VIDIOC_SUBDEV_S_FMT``. +All stream configurations are reset when ``VIDIOC_SUBDEV_S_ROUTING`` is +called. This means that the userspace must reconfigure all streams after calling +the ioctl with e.g. ``VIDIOC_SUBDEV_S_FMT``. Only subdevices which have both sink and source pads can support routing. -When inspecting routes through ``VIDIOC_SUBDEV_G_ROUTING`` and the application -provided ``num_routes`` is not big enough to contain all the available routes -the subdevice exposes, drivers return the ENOSPC error code and adjust the -value of the ``num_routes`` field. Application should then reserve enough memory -for all the route entries and call ``VIDIOC_SUBDEV_G_ROUTING`` again. +The ``num_routes`` field is used to denote the number of routes set (set by user +space on ``VIDIOC_SUBDEV_S_ROUTING`` argument) on the routing table as well as +the number of routes returned back from both IOCTLs. The ``len_routes`` +signifies the number of routes that can fit into the ``routes`` array. The +userspace shall set ``len_routes`` for both IOCTLs and ``num_routes`` for +``VIDIOC_SUBDEV_S_ROUTING``. -On a successful ``VIDIOC_SUBDEV_G_ROUTING`` call the driver updates the -``num_routes`` field to reflect the actual number of routes returned. +On a ``VIDIOC_SUBDEV_G_ROUTING`` call the driver updates the ``num_routes`` +field to reflect the actual number of routes known by the driver. ``num_routes`` +larger than ``len_routes`` in both IOCTLs. In this ``len_routes`` were returned +back to the userspace. This is not an error. + +Also ``VIDIOC_SUBDEV_S_ROUTING`` may return more route than the user provided in +``num_routes`` field due to e.g. hardware properties. .. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.7cm}|
@@ -74,6 +80,9 @@ On a successful ``VIDIOC_SUBDEV_G_ROUTING`` call the driver updates the - ``which`` - Format to modified, from enum :ref:`v4l2_subdev_format_whence <v4l2-subdev-format-whence>`. + * - __u32 + - ``len_routes`` + - The length of the array (as in memory reserved for the array) * - struct :c:type:`v4l2_subdev_route` - ``routes[]`` - Array of struct :c:type:`v4l2_subdev_route` entries
@@ -81,7 +90,7 @@ On a successful ``VIDIOC_SUBDEV_G_ROUTING`` call the driver updates the - ``num_routes`` - Number of entries of the routes array * - __u32 - - ``reserved``\ [5] + - ``reserved``\ [11] - Reserved for future extensions. Applications and drivers must set the array to zero.
drivers/media/v4l2-core/v4l2-ioctl.c +2 -2
@@ -3155,13 +3155,13 @@ static int check_array_args(unsigned int cmd, void *parg, size_t *array_size,case VIDIOC_SUBDEV_S_ROUTING : {struct v4l2_subdev_routing *routing = parg; -if (routing->num_routes >256 ) +if (routing->len_routes >256 )return -E2BIG ; *user_ptr =u64_to_user_ptr (routing->routes); *kernel_ptr = (void **)&routing->routes; *array_size =sizeof (struct v4l2_subdev_route ) - * routing->num_routes; + * routing->len_routes; ret =1 ;break ; }
drivers/media/v4l2-core/v4l2-subdev.c +5 -1
@@ -903,6 +903,9 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,if (routing->which !=V4L2_SUBDEV_FORMAT_TRY && ro_subdev)return -EPERM ; +if (routing->num_routes > routing->len_routes) +return -EINVAL ; +memset (routing->reserved,0 ,sizeof (routing->reserved));for (i =0 ; i < routing->num_routes; ++i) {
@@ -929,6 +932,7 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg, } krouting.num_routes = routing->num_routes; + krouting.len_routes = routing->len_routes; krouting.routes = routes; return v4l2_subdev_call(sd, pad, set_routing, state,
@@ -949,7 +953,7 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg, krouting = &state->routing; -if (routing->num_routes < krouting->num_routes) { +if (routing->len_routes < krouting->num_routes) { routing->num_routes = krouting->num_routes;return -ENOSPC ; }
include/media/v4l2-subdev.h +2 -0
@@ -741,12 +741,14 @@ struct v4l2_subdev_stream_configs {/** * struct v4l2_subdev_krouting - subdev routing table * +* @len_routes: length of routes array, in routes * @num_routes: number of routes * @routes: &struct v4l2_subdev_route * * This structure contains the routing table for a subdev. */ struct v4l2_subdev_krouting { +unsigned int len_routes;unsigned int num_routes;struct v4l2_subdev_route *routes; };
include/uapi/linux/v4l2-subdev.h +5 -3
@@ -222,15 +222,17 @@ struct v4l2_subdev_route {* struct v4l2_subdev_routing - Subdev routing information * * @which: configuration type (from enum v4l2_subdev_format_whence) -* @num_routes: the total number of routes in the routes array +* @len_routes: the length of the routes array, in routes * @routes: pointer to the routes array +* @num_routes: the total number of routes in the routes array * @reserved: drivers and applications must zero this array */ struct v4l2_subdev_routing {__u32 which; -__u32 num_routes; +__u32 len_routes;__u64 routes; -__u32 reserved[6 ]; +__u32 num_routes; +__u32 reserved[11 ]; };/*
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 15/31 media: v4l: subdev: Return routes set using S_ROUTING
In reply to Sakari AilusReturn the routes set using S_ROUTING back to the user. Also reflect this in documentation.
--- .../media/v4l/vidioc-subdev-g-routing.rst | 5 +++-- drivers/media/v4l2-core/v4l2-subdev.c | 12 +++++++++++- 2 files changed, 14 insertions(+), 3 deletions(-)
Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst +3 -2
@@ -43,8 +43,9 @@ The routing configuration determines the flows of data inside an entity. Drivers report their current routing tables using the ``VIDIOC_SUBDEV_G_ROUTING`` ioctl and application may enable or disable routes with the ``VIDIOC_SUBDEV_S_ROUTING`` ioctl, by adding or removing routes and -setting or clearing flags of the ``flags`` field of a -struct :c:type:`v4l2_subdev_route`. +setting or clearing flags of the ``flags`` field of a struct +:c:type:`v4l2_subdev_route`. Similarly to ``VIDIOC_SUBDEV_G_ROUTING``, also +``VIDIOC_SUBDEV_S_ROUTING`` returns the routes back to the user. All stream configurations are reset when ``VIDIOC_SUBDEV_S_ROUTING`` is called. This means that the userspace must reconfigure all streams after calling
drivers/media/v4l2-core/v4l2-subdev.c +11 -1
@@ -935,8 +935,18 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg, krouting.len_routes = routing->len_routes; krouting.routes = routes; -return v4l2_subdev_call (sd, pad, set_routing, state, + rval =v4l2_subdev_call (sd, pad, set_routing, state, routing->which, &krouting); +if (rval <0 ) +return rval; + +memcpy ((struct v4l2_subdev_route *)(uintptr_t )routing->routes, + state->routing.routes, + state->routing.num_routes * +sizeof (*state->routing.routes)); + routing->num_routes = state->routing.num_routes; + +return 0 ; }case VIDIOC_SUBDEV_G_ROUTING : {
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 16/31 media: v4l: subdev: Allow a larger number of routes than there's room for
In reply to Sakari AilusOn VIDIOC_SUBDEV_[GS]_ROUTING, only return as many routes back to the user as there's room. Do not consider it an error if more routes existed. Simply inform the user there are more routes.
--- .../media/v4l/vidioc-subdev-g-routing.rst | 4 ---- drivers/media/v4l2-core/v4l2-subdev.c | 10 +++------- 2 files changed, 3 insertions(+), 11 deletions(-)
Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst +0 -4
@@ -145,10 +145,6 @@ On success 0 is returned, on error -1 and the ``errno`` variable is set appropriately. The generic error codes are described at the :ref:`Generic Error Codes <gen-errors>` chapter. -ENOSPC - The application provided ``num_routes`` is not big enough to contain - all the available routes the subdevice exposes. - EINVAL The sink or source pad identifiers reference a non-existing pad, or reference pads of different types (ie. the sink_pad identifiers refers to a source pad).
drivers/media/v4l2-core/v4l2-subdev.c +3 -7
@@ -942,7 +942,7 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,memcpy ((struct v4l2_subdev_route *)(uintptr_t )routing->routes, state->routing.routes, - state->routing.num_routes * +min (state->routing.num_routes, routing->len_routes) *sizeof (*state->routing.routes)); routing->num_routes = state->routing.num_routes;
@@ -963,14 +963,10 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg, krouting = &state->routing; -if (routing->len_routes < krouting->num_routes) { - routing->num_routes = krouting->num_routes; -return -ENOSPC ; - } -memcpy ((struct v4l2_subdev_route *)(uintptr_t )routing->routes, krouting->routes, - krouting->num_routes *sizeof (*krouting->routes)); +min (krouting->num_routes, routing->len_routes) * +sizeof (*krouting->routes)); routing->num_routes = krouting->num_routes;return 0 ;
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 17/31 media: v4l: subdev: Add trivial set_routing support
In reply to Sakari AilusAdd trivial S_ROUTING IOCTL support for drivers where routing is static. Essentially this means returning the same information G_ROUTING call would have done.
--- drivers/media/v4l2-core/v4l2-subdev.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
drivers/media/v4l2-core/v4l2-subdev.c +14 -0
@@ -908,6 +908,20 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,memset (routing->reserved,0 ,sizeof (routing->reserved)); +/* +* If the driver doesn't support setting routing, just return +* the routing table here. +*/ +if (!v4l2_subdev_has_op (sd, pad, set_routing)) { +memcpy ((struct v4l2_subdev_route *)(uintptr_t )routing->routes, + state->routing.routes, +min (state->routing.num_routes, routing->len_routes) * +sizeof (*state->routing.routes)); + routing->num_routes = state->routing.num_routes; + +return 0 ; + } +for (i =0 ; i < routing->num_routes; ++i) {const struct v4l2_subdev_route *route = &routes[i];const struct media_pad *pads = sd->entity.pads;
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 18/31 media: ccs: No need to set streaming to false in power off
In reply to Sakari AilusStreaming will have been stopped by the sensor is powered off, and so sensor->streaming is also false already. Do not set it as part of the runtime suspend callback.
--- drivers/media/i2c/ccs/ccs-core.c | 1 - 1 file changed, 1 deletion(-)
drivers/media/i2c/ccs/ccs-core.c +0 -1
@@ -1700,7 +1700,6 @@ static int ccs_power_off(struct device *dev)usleep_range (5000 ,5000 );regulator_bulk_disable (ARRAY_SIZE (ccs_regulators), sensor->regulators); - sensor->streaming = false;return 0 ; }
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 19/31 media: ccs: Use {enable,disable}_streams operations
In reply to Sakari AilusSwitch from s_stream() video op to enable_streams() and disable_streams() pad operations. They are preferred and required for streams support.
--- drivers/media/i2c/ccs/ccs-core.c | 183 +++++++++++++++---------------- 1 file changed, 86 insertions(+), 97 deletions(-)
drivers/media/i2c/ccs/ccs-core.c +86 -97
@@ -1705,22 +1705,64 @@ static int ccs_power_off(struct device *dev) }/* ----------------------------------------------------------------------------- -* Video stream management +* V4L2 subdev video operations */ -static int ccs_start_streaming (struct ccs_sensor *sensor) +static int ccs_pm_get_init (struct ccs_sensor *sensor) { +struct i2c_client *client =v4l2_get_subdevdata (&sensor->src->sd); +int rval; + +/* +* It can't use pm_runtime_resume_and_get() here, as the driver +* relies at the returned value to detect if the device was already +* active or not. +*/ + rval =pm_runtime_get_sync (&client->dev); +if (rval <0 ) + goto error; + +/* Device was already active, so don't set controls */ +if (rval ==1 ) +return 0 ; + +/* Restore V4L2 controls to the previously suspended device */ + rval =__v4l2_ctrl_handler_setup (&sensor->pixel_array->ctrl_handler); +if (rval) + goto error; + + rval =__v4l2_ctrl_handler_setup (&sensor->src->ctrl_handler); +if (rval) + goto error; + +/* Keep PM runtime usage_count incremented on success */ +return 0 ; +error: +pm_runtime_put (&client->dev); +return rval; +} + +static int ccs_enable_streams (struct v4l2_subdev *subdev, +struct v4l2_subdev_state *state,u32 pad, +u64 streams_mask) +{ +struct ccs_sensor *sensor =to_ccs_sensor (subdev);struct i2c_client *client =v4l2_get_subdevdata (&sensor->src->sd);unsigned int binning_mode;int rval; -mutex_lock (&sensor->mutex); +if (pad !=CCS_PAD_SRC ) +return -EINVAL ; + + rval =ccs_pm_get_init (sensor); +if (rval) +return rval; rval =ccs_write (sensor,CSI_DATA_FORMAT , (sensor->csi_format->width <<8 ) | sensor->csi_format->compressed);if (rval) - goto out; + goto err_pm_put;/* Binning configuration */ if (sensor->binning_horizontal == 1 &&
@@ -1733,38 +1775,38 @@ static int ccs_start_streaming(struct ccs_sensor *sensor) rval =ccs_write (sensor,BINNING_TYPE , binning_type);if (rval <0 ) - goto out; + goto err_pm_put; binning_mode =1 ; } rval =ccs_write (sensor,BINNING_MODE , binning_mode);if (rval <0 ) - goto out; + goto err_pm_put;/* Set up PLL */ rval =ccs_pll_configure (sensor);if (rval) - goto out; + goto err_pm_put;/* Analog crop start coordinates */ rval =ccs_write (sensor,X_ADDR_START , sensor->pa_src.left);if (rval <0 ) - goto out; + goto err_pm_put; rval =ccs_write (sensor,Y_ADDR_START , sensor->pa_src.top);if (rval <0 ) - goto out; + goto err_pm_put;/* Analog crop end coordinates */ rval =ccs_write (sensor,X_ADDR_END , sensor->pa_src.left + sensor->pa_src.width -1 );if (rval <0 ) - goto out; + goto err_pm_put; rval =ccs_write (sensor,Y_ADDR_END , sensor->pa_src.top + sensor->pa_src.height -1 );if (rval <0 ) - goto out; + goto err_pm_put;/* * Output from pixel array, including blanking, is set using
@@ -1777,22 +1819,22 @@ static int ccs_start_streaming(struct ccs_sensor *sensor) rval =ccs_write (sensor,DIGITAL_CROP_X_OFFSET , sensor->scaler_sink.left);if (rval <0 ) - goto out; + goto err_pm_put; rval =ccs_write (sensor,DIGITAL_CROP_Y_OFFSET , sensor->scaler_sink.top);if (rval <0 ) - goto out; + goto err_pm_put; rval =ccs_write (sensor,DIGITAL_CROP_IMAGE_WIDTH , sensor->scaler_sink.width);if (rval <0 ) - goto out; + goto err_pm_put; rval =ccs_write (sensor,DIGITAL_CROP_IMAGE_HEIGHT , sensor->scaler_sink.height);if (rval <0 ) - goto out; + goto err_pm_put; }/* Scaling */
@@ -1800,20 +1842,20 @@ static int ccs_start_streaming(struct ccs_sensor *sensor) != CCS_SCALING_CAPABILITY_NONE) { rval =ccs_write (sensor,SCALING_MODE , sensor->scaling_mode);if (rval <0 ) - goto out; + goto err_pm_put; rval =ccs_write (sensor,SCALE_M , sensor->scale_m);if (rval <0 ) - goto out; + goto err_pm_put; }/* Output size from sensor */ rval =ccs_write (sensor,X_OUTPUT_SIZE , sensor->src_src.width);if (rval <0 ) - goto out; + goto err_pm_put; rval =ccs_write (sensor,Y_OUTPUT_SIZE , sensor->src_src.height);if (rval <0 ) - goto out; + goto err_pm_put; if (CCS_LIM(sensor, FLASH_MODE_CAPABILITY) & (CCS_FLASH_MODE_CAPABILITY_SINGLE_STROBE |
@@ -1822,109 +1864,52 @@ static int ccs_start_streaming(struct ccs_sensor *sensor) sensor->hwcfg.strobe_setup->trigger != 0) { rval =ccs_setup_flash_strobe (sensor);if (rval) - goto out; + goto err_pm_put; } rval =ccs_call_quirk (sensor, pre_streamon);if (rval) {dev_err (&client->dev,"pre_streamon quirks failed\n" ); - goto out; + goto err_pm_put; } rval =ccs_write (sensor,MODE_SELECT ,CCS_MODE_SELECT_STREAMING ); -out: -mutex_unlock (&sensor->mutex); - -return rval; -} - -static int ccs_stop_streaming (struct ccs_sensor *sensor) -{ -struct i2c_client *client =v4l2_get_subdevdata (&sensor->src->sd); -int rval; - -mutex_lock (&sensor->mutex); - rval =ccs_write (sensor,MODE_SELECT ,CCS_MODE_SELECT_SOFTWARE_STANDBY ); -if (rval) - goto out; - - rval =ccs_call_quirk (sensor, post_streamoff); -if (rval) -dev_err (&client->dev,"post_streamoff quirks failed\n" ); - -out: -mutex_unlock (&sensor->mutex); -return rval; -} - -/* ----------------------------------------------------------------------------- -* V4L2 subdev video operations -*/ - -static int ccs_pm_get_init (struct ccs_sensor *sensor) -{ -struct i2c_client *client =v4l2_get_subdevdata (&sensor->src->sd); -int rval; - -/* -* It can't use pm_runtime_resume_and_get() here, as the driver -* relies at the returned value to detect if the device was already -* active or not. -*/ - rval =pm_runtime_get_sync (&client->dev); -if (rval <0 ) - goto error; - -/* Device was already active, so don't set controls */ -if (rval ==1 ) -return 0 ; + sensor->streaming = true; -/* Restore V4L2 controls to the previously suspended device */ - rval =v4l2_ctrl_handler_setup (&sensor->pixel_array->ctrl_handler); -if (rval) - goto error; +return 0 ; - rval =v4l2_ctrl_handler_setup (&sensor->src->ctrl_handler); -if (rval) - goto error; +err_pm_put: +pm_runtime_mark_last_busy (&client->dev); +pm_runtime_put_autosuspend (&client->dev); -/* Keep PM runtime usage_count incremented on success */ -return 0 ; -error: -pm_runtime_put (&client->dev);return rval; } -static int ccs_set_stream (struct v4l2_subdev *subdev,int enable) +static int ccs_disable_streams (struct v4l2_subdev *subdev, +struct v4l2_subdev_state *state,u32 pad, +u64 streams_mask) {struct ccs_sensor *sensor =to_ccs_sensor (subdev);struct i2c_client *client =v4l2_get_subdevdata (&sensor->src->sd);int rval; -if (!enable) { -ccs_stop_streaming (sensor); - sensor->streaming = false; -pm_runtime_mark_last_busy (&client->dev); -pm_runtime_put_autosuspend (&client->dev); - -return 0 ; - } +if (pad !=CCS_PAD_SRC ) +return -EINVAL ; - rval =ccs_pm_get_init (sensor); + rval =ccs_write (sensor,MODE_SELECT ,CCS_MODE_SELECT_SOFTWARE_STANDBY );if (rval)return rval; - sensor->streaming = true; + rval =ccs_call_quirk (sensor, post_streamoff); +if (rval) +dev_err (&client->dev,"post_streamoff quirks failed\n" ); - rval =ccs_start_streaming (sensor); -if (rval <0 ) { - sensor->streaming = false; -pm_runtime_mark_last_busy (&client->dev); -pm_runtime_put_autosuspend (&client->dev); - } + sensor->streaming = false; +pm_runtime_mark_last_busy (&client->dev); +pm_runtime_put_autosuspend (&client->dev); -return rval; +return 0 ; }static int ccs_pre_streamon (struct v4l2_subdev *subdev,u32 flags)
@@ -1950,7 +1935,9 @@ static int ccs_pre_streamon(struct v4l2_subdev *subdev, u32 flags) } } +mutex_lock (&sensor->mutex); rval =ccs_pm_get_init (sensor); +mutex_unlock (&sensor->mutex);if (rval)return rval;
@@ -3046,7 +3033,7 @@ static int ccs_init_cfg(struct v4l2_subdev *sd, }static const struct v4l2_subdev_video_ops ccs_video_ops = { - .s_stream = ccs_set_stream, + .s_stream = v4l2_subdev_s_stream_helper, .pre_streamon = ccs_pre_streamon, .post_streamoff = ccs_post_streamoff, };
@@ -3058,6 +3045,8 @@ static const struct v4l2_subdev_pad_ops ccs_pad_ops = { .set_fmt = ccs_set_format, .get_selection = ccs_get_selection, .set_selection = ccs_set_selection, + .enable_streams = ccs_enable_streams, + .disable_streams = ccs_disable_streams, };static const struct v4l2_subdev_sensor_ops ccs_sensor_ops = {
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 20/31 media: ccs: Track streaming state
In reply to Sakari AilusWith enable_streams and disable_streams, the driver for a device where streams are not independently started and stopped needs to maintain state information on streams that have been requested to be started. Do that now.
In the future, a helper function in the framework is a desirable way to do this instead.
--- drivers/media/i2c/ccs/ccs-core.c | 13 ++++++++++--- drivers/media/i2c/ccs/ccs.h | 2 +- 2 files changed, 11 insertions(+), 4 deletions(-)
drivers/media/i2c/ccs/ccs-core.c +10 -3
@@ -1754,6 +1754,11 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev,if (pad !=CCS_PAD_SRC )return -EINVAL ; +if (sensor->streaming) { + sensor->streaming |= streams_mask; +return 0 ; + } + rval =ccs_pm_get_init (sensor);if (rval)return rval;
@@ -1875,7 +1880,7 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev, rval =ccs_write (sensor,MODE_SELECT ,CCS_MODE_SELECT_STREAMING ); - sensor->streaming = true; + sensor->streaming |= streams_mask;return 0 ;
@@ -1897,6 +1902,10 @@ static int ccs_disable_streams(struct v4l2_subdev *subdev,if (pad !=CCS_PAD_SRC )return -EINVAL ; + sensor->streaming &= ~streams_mask; +if (sensor->streaming) +return 0 ; + rval =ccs_write (sensor,MODE_SELECT ,CCS_MODE_SELECT_SOFTWARE_STANDBY );if (rval)return rval;
@@ -1905,7 +1914,6 @@ static int ccs_disable_streams(struct v4l2_subdev *subdev,if (rval)dev_err (&client->dev,"post_streamoff quirks failed\n" ); - sensor->streaming = false;pm_runtime_mark_last_busy (&client->dev);pm_runtime_put_autosuspend (&client->dev);
@@ -3518,7 +3526,6 @@ static int ccs_probe(struct i2c_client *client) goto out_cleanup; } - sensor->streaming = false; sensor->dev_init_done = true; rval =ccs_write_msr_regs (sensor);
drivers/media/i2c/ccs/ccs.h +1 -1
@@ -234,7 +234,7 @@ struct ccs_sensor {u16 image_start;/* image data start line */ u16 visible_pixel_start;/* start pixel of the visible image */ -bool streaming; +u8 streaming;bool dev_init_done;u8 compressed_min_bpp;
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 21/31 media: ccs: Move ccs_validate_csi_data_format up
In reply to Sakari Ailusccs_validate_csi_data_format() will soon be needed elsewhere, above its current location. Move it up.
--- drivers/media/i2c/ccs/ccs-core.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-)
drivers/media/i2c/ccs/ccs-core.c +14 -14
@@ -1967,6 +1967,20 @@ static int ccs_post_streamoff(struct v4l2_subdev *subdev)return pm_runtime_put (&client->dev); } +static const struct ccs_csi_data_format +*ccs_validate_csi_data_format (struct ccs_sensor *sensor,u32 code) +{ +unsigned int i; + +for (i =0 ; i <ARRAY_SIZE (ccs_csi_data_formats); i++) { +if (sensor->mbus_frame_fmts & (1 << i) && + ccs_csi_data_formats[i].code == code) +return &ccs_csi_data_formats[i]; + } + +return sensor->csi_format; +} +static int ccs_enum_mbus_code (struct v4l2_subdev *subdev,struct v4l2_subdev_state *sd_state,struct v4l2_subdev_mbus_code_enum *code)
@@ -2102,20 +2116,6 @@ static void ccs_propagate(struct v4l2_subdev *subdev, } } -static const struct ccs_csi_data_format -*ccs_validate_csi_data_format (struct ccs_sensor *sensor,u32 code) -{ -unsigned int i; - -for (i =0 ; i <ARRAY_SIZE (ccs_csi_data_formats); i++) { -if (sensor->mbus_frame_fmts & (1 << i) && - ccs_csi_data_formats[i].code == code) -return &ccs_csi_data_formats[i]; - } - -return sensor->csi_format; -} -static int ccs_set_format_source (struct v4l2_subdev *subdev,struct v4l2_subdev_state *sd_state,struct v4l2_subdev_format *fmt)
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 22/31 media: ccs: Support frame descriptors
In reply to Sakari AilusProvide information on the frame layout using frame descriptors.
--- drivers/media/i2c/ccs/ccs-core.c | 60 +++++++++++++++++++++++++++++++ drivers/media/i2c/ccs/ccs-quirk.h | 7 ++++ drivers/media/i2c/ccs/ccs.h | 4 +++ 3 files changed, 71 insertions(+)
drivers/media/i2c/ccs/ccs-core.c +60 -0
@@ -25,6 +25,7 @@#include <linux/slab.h> #include <linux/smiapp.h> #include <linux/v4l2-mediabus.h> +#include <media/mipi-csi2.h> #include <media/v4l2-fwnode.h> #include <media/v4l2-device.h> #include <uapi/linux/ccs.h>
@@ -235,6 +236,33 @@ static int ccs_read_all_limits(struct ccs_sensor *sensor)return ret; } +static u8 ccs_mipi_csi2_data_type (unsigned int bpp) +{ +switch (bpp) { +case 6 : +return MIPI_CSI2_DT_RAW6 ; +case 7 : +return MIPI_CSI2_DT_RAW7 ; +case 8 : +return MIPI_CSI2_DT_RAW8 ; +case 10 : +return MIPI_CSI2_DT_RAW10 ; +case 12 : +return MIPI_CSI2_DT_RAW12 ; +case 14 : +return MIPI_CSI2_DT_RAW14 ; +case 16 : +return MIPI_CSI2_DT_RAW16 ; +case 20 : +return MIPI_CSI2_DT_RAW20 ; +case 24 : +return MIPI_CSI2_DT_RAW24 ; +default : +WARN_ON (1 ); +return 0 ; + } +} +static int ccs_read_frame_fmt (struct ccs_sensor *sensor) {struct i2c_client *client =v4l2_get_subdevdata (&sensor->src->sd);
@@ -2620,6 +2648,37 @@ static int ccs_set_selection(struct v4l2_subdev *subdev,return ret; } +static int ccs_get_frame_desc (struct v4l2_subdev *subdev,unsigned int pad, +struct v4l2_mbus_frame_desc *desc) +{ +struct ccs_sensor *sensor =to_ccs_sensor (subdev); +struct v4l2_mbus_frame_desc_entry *entry = desc->entry; + +if (ccs_has_quirk (sensor, frame_desc)) +return ccs_call_quirk (sensor, frame_desc, desc); + +switch (sensor->hwcfg.csi_signalling_mode) { +case CCS_CSI_SIGNALING_MODE_CSI_2_DPHY : +case CCS_CSI_SIGNALING_MODE_CSI_2_CPHY : + desc->type =V4L2_MBUS_FRAME_DESC_TYPE_CSI2 ; +break ; +default : +/* FIXME: CCP2 support */ +return -EINVAL ; + } + + entry->pixelcode = sensor->csi_format->code; + entry->stream =CCS_STREAM_PIXEL ; + entry->bus.csi2.dt = + sensor->csi_format->width == sensor->csi_format->compressed ? +ccs_mipi_csi2_data_type (sensor->csi_format->compressed) : +CCS_DEFAULT_COMPRESSED_DT ; + entry++; + desc->num_entries++; + +return 0 ; +} +static int ccs_get_skip_frames (struct v4l2_subdev *subdev,u32 *frames) {struct ccs_sensor *sensor =to_ccs_sensor (subdev);
@@ -3055,6 +3114,7 @@ static const struct v4l2_subdev_pad_ops ccs_pad_ops = { .set_selection = ccs_set_selection, .enable_streams = ccs_enable_streams, .disable_streams = ccs_disable_streams, + .get_frame_desc = ccs_get_frame_desc, };static const struct v4l2_subdev_sensor_ops ccs_sensor_ops = {
drivers/media/i2c/ccs/ccs-quirk.h +7 -0
@@ -36,6 +36,7 @@ struct ccs_sensor;* access may be done by the caller (default read * value is zero), else negative error code on error * @flags: Quirk flags +* @frame_desc: Obtain the frame descriptor */ struct ccs_quirk {int (*limits)(struct ccs_sensor *sensor);
@@ -46,6 +47,8 @@ struct ccs_quirk {int (*init)(struct ccs_sensor *sensor);int (*reg_access)(struct ccs_sensor *sensor,bool write,u32 *reg,u32 *val); +int (*frame_desc)(struct ccs_sensor *sensor, +struct v4l2_mbus_frame_desc *desc);unsigned long flags; };
@@ -62,6 +65,10 @@ struct ccs_reg_8 { .val = _val, \ } +#define ccs_has_quirk (sensor, _quirk) \ + ((sensor)->minfo.quirk && \ + (sensor)->minfo.quirk->_quirk) +#define ccs_call_quirk (sensor, _quirk, ...) \ ((sensor)->minfo.quirk && \ (sensor)->minfo.quirk->_quirk ? \
drivers/media/i2c/ccs/ccs.h +4 -0
@@ -45,6 +45,8 @@#define CCS_COLOUR_COMPONENTS 4 +#define CCS_DEFAULT_COMPRESSED_DT 0x30 +#define SMIAPP_NAME "smiapp"#define CCS_NAME "ccs"
@@ -174,6 +176,8 @@ struct ccs_csi_data_format {#define CCS_PAD_SRC 1#define CCS_PADS 2 +#define CCS_STREAM_PIXEL 0 +struct ccs_binning_subtype {u8 horizontal:4 ;u8 vertical:4 ;
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 23/31 media: uapi: ccs: Add media bus code for MIPI CCS embedded data
In reply to Sakari AilusAdd new MIPI CCS embedded data media bus code
(MEDIA_BUS_FMT_CCS_EMBEDDED).
--- .../media/v4l/subdev-formats.rst | 28 +++++++++++++++++++ include/uapi/linux/media-bus-format.h | 3 ++ 2 files changed, 31 insertions(+)
Documentation/userspace-api/media/v4l/subdev-formats.rst +28 -0
@@ -8492,3 +8492,31 @@ and finally the bit number in subscript. "X" indicates a padding bit. - X - X - X + +.. _MEDIA-BUS-FMT-CCS-EMBEDDED: + +MIPI CCS Embedded Data Formats +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +`MIPI CCS <https://www.mipi.org/specifications/camera-command-set>`_ defines a +metadata format for sensor embedded data, which is used to store the register +configuration used for capturing a given frame. The format is defined in the CCS +specification. The media bus code for this format is +``MEDIA_BUS_FMT_CCS_EMBEDDED``. + +The CCS embedded data format definition includes three levels: + +1. Padding within CSI-2 bus :ref:`Data unit <media-glossary-data-unit>` as + documented in the MIPI CCS specification. + +2. The tagged data format as documented in the MIPI CCS specification. + +3. Register addresses and register documentation as documented in the MIPI CCS + specification. + +The format definition shall be used only by devices that fulfill all three +levels above. + +This mbus code are only used for "2-byte simplified tagged data format" (code +0xa) but their use may be extended further in the future, to cover other CCS +embedded data format codes.
include/uapi/linux/media-bus-format.h +3 -0
@@ -182,4 +182,7 @@#define MEDIA_BUS_FMT_META_20 0x8006#define MEDIA_BUS_FMT_META_24 0x8007 +/* Specific metadata formats. Next is 0x9002. */ +#define MEDIA_BUS_FMT_CCS_EMBEDDED 0x9001 +#endif /* __LINUX_MEDIA_BUS_FORMAT_H */
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 24/31 media: ccs: Add support for embedded data stream
In reply to Sakari AilusAdd support for embedded data stream, in UAPI and frame descriptor.
This patch adds also a new embedded data pad (2) to the source sub-device.
--- drivers/media/i2c/ccs/ccs-core.c | 357 +++++++++++++++++++++++++++++-- drivers/media/i2c/ccs/ccs.h | 18 +- 2 files changed, 350 insertions(+), 25 deletions(-)
drivers/media/i2c/ccs/ccs-core.c +336 -21
@@ -1890,6 +1890,13 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev,if (rval <0 ) goto err_pm_put; +/* Configure embedded data */ +if (sensor->csi_format->compressed >=16 ) { + rval =ccs_write (sensor,EMB_DATA_CTRL , sensor->emb_data_ctrl); +if (rval <0 ) + goto err_pm_put; + } + if (CCS_LIM(sensor, FLASH_MODE_CAPABILITY) & (CCS_FLASH_MODE_CAPABILITY_SINGLE_STROBE | SMIAPP_FLASH_MODE_CAPABILITY_MULTIPLE_STROBE) &&
@@ -2009,6 +2016,57 @@ static const struct ccs_csi_data_formatreturn sensor->csi_format; } +#define CCS_EMBEDDED_CODE_DEPTH (depth, half_depth) \ + depth, \ + CCS_EMB_DATA_CAPABILITY_TWO_BYTES_PER_RAW##depth, \ + CCS_EMB_DATA_CAPABILITY_NO_ONE_BYTE_PER_RAW##depth, \ + CCS_EMB_DATA_CTRL_RAW##half_depth##_PACKING_FOR_RAW##depth, \ + MEDIA_BUS_FMT_META_##half_depth, \ + MEDIA_BUS_FMT_META_##depth, \ + +static const struct ccs_embedded_code { +u8 depth; +u8 cap_two_bytes_per_sample; +u8 cap_no_legacy; +u8 ctrl; +u32 code_two_bytes; +u32 code_legacy; +} ccs_embedded_codes[] = { + {CCS_EMBEDDED_CODE_DEPTH (16 ,8 ) }, + {CCS_EMBEDDED_CODE_DEPTH (20 ,10 ) }, + {CCS_EMBEDDED_CODE_DEPTH (24 ,12 ) }, +}; + +static const struct ccs_embedded_code *ccs_embedded_code (unsigned int bpp) +{ +unsigned int i; + +for (i =0 ; i <ARRAY_SIZE (ccs_embedded_codes); i++) +if (ccs_embedded_codes[i].depth == bpp) +return ccs_embedded_codes + i; + +WARN_ON (1 ); + +return ccs_embedded_codes; +} + +static u32 +ccs_default_embedded_code (struct ccs_sensor *sensor, +const struct ccs_embedded_code *embedded_code) +{ +if (CCS_LIM (sensor,EMB_DATA_CAPABILITY ) & +BIT (embedded_code->cap_two_bytes_per_sample)) +return embedded_code->code_two_bytes; + +if (!(CCS_LIM (sensor,EMB_DATA_CAPABILITY ) & +BIT (embedded_code->cap_no_legacy))) +return embedded_code->code_legacy; + +WARN_ON (1 ); + +return embedded_code->code_legacy; +} +static int ccs_enum_mbus_code (struct v4l2_subdev *subdev,struct v4l2_subdev_state *sd_state,struct v4l2_subdev_mbus_code_enum *code)
@@ -2024,6 +2082,69 @@ static int ccs_enum_mbus_code(struct v4l2_subdev *subdev,dev_err (&client->dev,"subdev %s, pad %u, index %u\n" , subdev->name, code->pad, code->index); +if (subdev == &sensor->src->sd) { +if (code->pad ==CCS_PAD_META ) { +if (code->index) + goto out; + + code->code =MEDIA_BUS_FMT_CCS_EMBEDDED ; + + rval =0 ; + goto out; + } +if (code->stream ==CCS_STREAM_META ) { +struct v4l2_mbus_framefmt *pix_fmt = +v4l2_subdev_state_get_format (sd_state, +CCS_PAD_SRC , +CCS_STREAM_PIXEL ); +const struct ccs_csi_data_format *csi_format = +ccs_validate_csi_data_format (sensor, + pix_fmt->code); +unsigned int i =0 ; +u32 codes[2 ]; + +switch (csi_format->compressed) { +case 8 : + codes[i++] =MEDIA_BUS_FMT_META_8 ; +break ; +case 10 : + codes[i++] =MEDIA_BUS_FMT_META_10 ; +break ; +case 12 : + codes[i++] =MEDIA_BUS_FMT_META_12 ; +break ; +case 14 : + codes[i++] =MEDIA_BUS_FMT_META_14 ; +break ; +case 16 : +case 20 : +case 24 : { +const struct ccs_embedded_code *embedded_code = +ccs_embedded_code (csi_format->compressed); + +if (CCS_LIM (sensor,EMB_DATA_CAPABILITY ) & +BIT (embedded_code->cap_two_bytes_per_sample)) + codes[i++] = + embedded_code->code_two_bytes; + +if (!(CCS_LIM (sensor,EMB_DATA_CAPABILITY ) & +BIT (embedded_code->cap_no_legacy))) + codes[i++] = embedded_code->code_legacy; +break ; + } +default : +WARN_ON (1 ); + } + +if (WARN_ON (i >ARRAY_SIZE (codes)) || code->index >= i) + goto out; + + code->code = codes[code->index]; + rval =0 ; + goto out; + } + } +if (subdev != &sensor->src->sd || code->pad !=CCS_PAD_SRC ) {if (code->index) goto out;
@@ -2066,8 +2187,11 @@ static int __ccs_get_format(struct v4l2_subdev *subdev,struct v4l2_subdev_state *sd_state, struct v4l2_subdev_format *fmt) { - fmt->format = *v4l2_subdev_state_get_format(sd_state, fmt->pad); - fmt->format.code =__ccs_get_mbus_code (subdev, fmt->pad); + fmt->format = *v4l2_subdev_state_get_format(sd_state, fmt->pad, + fmt->stream); + +if (fmt->pad !=CCS_PAD_META && fmt->stream !=CCS_STREAM_META ) + fmt->format.code =__ccs_get_mbus_code (subdev, fmt->pad);return 0 ; }
@@ -2097,10 +2221,11 @@ static void ccs_get_crop_compose(struct v4l2_subdev *subdev,if (crops)for (i =0 ; i < subdev->entity.num_pads; i++) crops[i] = -v4l2_subdev_state_get_crop (sd_state, i); +v4l2_subdev_state_get_crop (sd_state, i, +CCS_STREAM_PIXEL );if (comps) - *comps =v4l2_subdev_state_get_compose (sd_state, - ssd->sink_pad); + *comps =v4l2_subdev_state_get_compose (sd_state, ssd->sink_pad, +CCS_STREAM_PIXEL ); }/* Changes require propagation only on sink pad. */
@@ -2133,7 +2258,8 @@ static void ccs_propagate(struct v4l2_subdev *subdev, fallthrough;case V4L2_SEL_TGT_COMPOSE : *crops[CCS_PAD_SRC ] = *comp; - fmt =v4l2_subdev_state_get_format (sd_state,CCS_PAD_SRC ); + fmt =v4l2_subdev_state_get_format (sd_state,CCS_PAD_SRC , +CCS_STREAM_PIXEL ); fmt->width = comp->width; fmt->height = comp->height;if (which ==V4L2_SUBDEV_FORMAT_ACTIVE && ssd == sensor->src)
@@ -2197,6 +2323,81 @@ static int ccs_set_format_source(struct v4l2_subdev *subdev,return ccs_pll_update (sensor); } +static int ccs_set_format_meta (struct v4l2_subdev *subdev, +struct v4l2_subdev_state *sd_state, +struct v4l2_mbus_framefmt *fmt) +{ +struct ccs_sensor *sensor =to_ccs_sensor (subdev); +const struct ccs_csi_data_format *csi_format; +struct v4l2_mbus_framefmt *pix_fmt; +struct v4l2_mbus_framefmt *meta_fmt; +struct v4l2_mbus_framefmt *meta_out_fmt; +u32 code; + + pix_fmt =v4l2_subdev_state_get_format (sd_state,CCS_PAD_SRC , +CCS_STREAM_PIXEL ); + meta_fmt =v4l2_subdev_state_get_format (sd_state,CCS_PAD_META ,0 ); + meta_out_fmt =v4l2_subdev_state_get_format (sd_state,CCS_PAD_SRC , +CCS_STREAM_META ); + + code = fmt ? fmt->code : meta_out_fmt->code; + + meta_out_fmt->width = meta_fmt->width = pix_fmt->width; + meta_out_fmt->height = meta_fmt->height = + sensor->embedded_end - sensor->embedded_start; + meta_fmt->code =MEDIA_BUS_FMT_CCS_EMBEDDED ; + + csi_format =ccs_validate_csi_data_format (sensor, pix_fmt->code); + +switch (csi_format->compressed) { +case 8 : + meta_out_fmt->code =MEDIA_BUS_FMT_META_8 ; +break ; +case 10 : + meta_out_fmt->code =MEDIA_BUS_FMT_META_10 ; +break ; +case 12 : + meta_out_fmt->code =MEDIA_BUS_FMT_META_12 ; +break ; +case 14 : + meta_out_fmt->code =MEDIA_BUS_FMT_META_14 ; +break ; +case 16 : +case 20 : +case 24 : { +const struct ccs_embedded_code *embedded_code; + + embedded_code =ccs_embedded_code (csi_format->compressed); + meta_out_fmt->code = +ccs_default_embedded_code (sensor, embedded_code); + +if (!(CCS_LIM (sensor,EMB_DATA_CAPABILITY ) & +BIT (embedded_code->cap_no_legacy)) && + code == embedded_code->code_legacy) { + meta_out_fmt->code = embedded_code->code_legacy; + sensor->emb_data_ctrl =0 ; + } + +if (CCS_LIM (sensor,EMB_DATA_CAPABILITY ) & +BIT (embedded_code->cap_two_bytes_per_sample) && + code == embedded_code->code_two_bytes) { + meta_out_fmt->code = embedded_code->code_two_bytes; + sensor->emb_data_ctrl = embedded_code->ctrl; + } + +break ; + } +default : +WARN_ON (1 ); +return 0 ; + } + +if (fmt) + *fmt = *meta_out_fmt; + +return 0 ; +} +static int ccs_set_format (struct v4l2_subdev *subdev,struct v4l2_subdev_state *sd_state,struct v4l2_subdev_format *fmt)
@@ -2205,12 +2406,25 @@ static int ccs_set_format(struct v4l2_subdev *subdev,struct ccs_subdev *ssd =to_ccs_subdev (subdev);struct v4l2_rect *crops[CCS_PADS ]; +if (subdev == &sensor->src->sd && fmt->pad ==CCS_PAD_META ) +return ccs_get_format (subdev, sd_state, fmt); +mutex_lock (&sensor->mutex); +if (subdev == &sensor->src->sd && fmt->stream ==CCS_STREAM_META ) { +ccs_set_format_meta (subdev, sd_state, &fmt->format); + +mutex_unlock (&sensor->mutex); + +return 0 ; + } +if (fmt->pad == ssd->source_pad) {int rval; rval =ccs_set_format_source (subdev, sd_state, fmt); +if (sensor->embedded_start != sensor->embedded_end) +ccs_set_format_meta (subdev, sd_state,NULL );mutex_unlock (&sensor->mutex);
@@ -2485,6 +2699,12 @@ static int ccs_sel_supported(struct v4l2_subdev *subdev,struct ccs_sensor *sensor =to_ccs_sensor (subdev);struct ccs_subdev *ssd =to_ccs_subdev (subdev); +if (sel->stream !=CCS_STREAM_PIXEL ) +return -EINVAL ; + +if (sel->pad ==CCS_PAD_META ) +return -EINVAL ; +/* We only implement crop in three places. */ switch (sel->target) {case V4L2_SEL_TGT_CROP :
@@ -2529,7 +2749,8 @@ static int ccs_set_crop(struct v4l2_subdev *subdev,if (sel->pad == ssd->sink_pad) {struct v4l2_mbus_framefmt *mfmt = -v4l2_subdev_state_get_format (sd_state, sel->pad); +v4l2_subdev_state_get_format (sd_state, sel->pad, +CCS_STREAM_PIXEL ); src_size.width = mfmt->width; src_size.height = mfmt->height;
@@ -2590,7 +2811,9 @@ static int ccs_get_selection(struct v4l2_subdev *subdev, }else if (sel->pad == ssd->sink_pad) {struct v4l2_mbus_framefmt *sink_fmt =v4l2_subdev_state_get_format (sd_state, - ssd->sink_pad); + ssd->sink_pad, +CCS_STREAM_PIXEL ); + sel->r.top = sel->r.left =0 ; sel->r.width = sink_fmt->width; sel->r.height = sink_fmt->height;
@@ -2676,6 +2899,14 @@ static int ccs_get_frame_desc(struct v4l2_subdev *subdev, unsigned int pad, entry++; desc->num_entries++; +if (sensor->embedded_start != sensor->embedded_end) { + entry->pixelcode =MEDIA_BUS_FMT_CCS_EMBEDDED ; + entry->stream =CCS_STREAM_META ; + entry->bus.csi2.dt =MIPI_CSI2_DT_EMBEDDED_8B ; + entry++; + desc->num_entries++; + } +return 0 ; }
@@ -3026,7 +3257,8 @@ static int ccs_init_subdev(struct ccs_sensor *sensor, ssd->sensor = sensor; ssd->npads = num_pads; - ssd->source_pad = num_pads -1 ; + ssd->source_pad = + ssd == sensor->pixel_array ?CCS_PA_PAD_SRC :CCS_PAD_SRC ;v4l2_i2c_subdev_set_name (&ssd->sd, client, sensor->minfo.name, name);
@@ -3040,6 +3272,10 @@ static int ccs_init_subdev(struct ccs_sensor *sensor, ssd->sd.owner =THIS_MODULE ; ssd->sd.dev = &client->dev;v4l2_set_subdevdata (&ssd->sd, client); + }else { + ssd->sd.flags |=V4L2_SUBDEV_FL_STREAMS ; + ssd->pads[CCS_PAD_META ].flags = +MEDIA_PAD_FL_SINK |MEDIA_PAD_FL_INTERNAL ; } rval =media_entity_pads_init (&ssd->sd.entity, ssd->npads, ssd->pads);
@@ -3057,21 +3293,19 @@ static int ccs_init_subdev(struct ccs_sensor *sensor,return 0 ; } -static int ccs_init_cfg (struct v4l2_subdev *sd, -struct v4l2_subdev_state *sd_state) +static int __ccs_init_cfg (struct v4l2_subdev *sd, +struct v4l2_subdev_state *sd_state) {struct ccs_subdev *ssd =to_ccs_subdev (sd);struct ccs_sensor *sensor = ssd->sensor;unsigned int pad = ssd == sensor->pixel_array ?CCS_PA_PAD_SRC :CCS_PAD_SINK ;struct v4l2_mbus_framefmt *fmt = -v4l2_subdev_state_get_format (sd_state, pad); +v4l2_subdev_state_get_format (sd_state, pad,CCS_STREAM_PIXEL );struct v4l2_rect *crop = -v4l2_subdev_state_get_crop (sd_state, pad); +v4l2_subdev_state_get_crop (sd_state, pad,CCS_STREAM_PIXEL );bool is_active = !sd->active_state || sd->active_state == sd_state; -mutex_lock (&sensor->mutex); -ccs_get_native_size (ssd, crop); fmt->width = crop->width;
@@ -3083,20 +3317,78 @@ static int ccs_init_cfg(struct v4l2_subdev *sd,if (is_active) sensor->pa_src = *crop; -mutex_unlock (&sensor->mutex);return 0 ; } - fmt =v4l2_subdev_state_get_format (sd_state,CCS_PAD_SRC ); + fmt =v4l2_subdev_state_get_format (sd_state,CCS_PAD_SRC , +CCS_STREAM_PIXEL ); fmt->code = ssd == sensor->src ? sensor->csi_format->code : sensor->internal_csi_format->code; fmt->field =V4L2_FIELD_NONE ;ccs_propagate (sd, sd_state, is_active,V4L2_SEL_TGT_CROP ); +return 0 ; +} + +static int ccs_init_cfg (struct v4l2_subdev *sd, +struct v4l2_subdev_state *sd_state) +{ +struct ccs_subdev *ssd =to_ccs_subdev (sd); +struct ccs_sensor *sensor = ssd->sensor; +int rval; + +mutex_lock (&sensor->mutex); + rval =__ccs_init_cfg (sd, sd_state);mutex_unlock (&sensor->mutex); -return 0 ; +return rval; +} + +static int ccs_src_init_cfg (struct v4l2_subdev *sd, +struct v4l2_subdev_state *sd_state) +{ +struct v4l2_subdev_route routes[] = { + { + .sink_pad =CCS_PAD_SINK , + .source_pad =CCS_PAD_SRC , + .source_stream =CCS_STREAM_PIXEL , + .flags =V4L2_SUBDEV_ROUTE_FL_ACTIVE , + }, { + .sink_pad =CCS_PAD_META , + .source_pad =CCS_PAD_SRC , + .source_stream =CCS_STREAM_META , + .flags =V4L2_SUBDEV_ROUTE_FL_ACTIVE , + } + }; +struct v4l2_subdev_krouting routing = { + .routes = routes, + .num_routes =1 , + }; +struct ccs_subdev *ssd =to_ccs_subdev (sd); +struct ccs_sensor *sensor = ssd->sensor; +int rval; + +mutex_lock (&sensor->mutex); + +if (sensor->embedded_start != sensor->embedded_end) + routing.num_routes++; + + rval =v4l2_subdev_set_routing (sd, sd_state, &routing); +if (rval) + goto out; + + rval =__ccs_init_cfg (sd, sd_state); +if (rval) + goto out; + +if (sensor->embedded_start != sensor->embedded_end) +ccs_set_format_meta (sd, sd_state,NULL ); + +out: +mutex_unlock (&sensor->mutex); + +return rval; }static const struct v4l2_subdev_video_ops ccs_video_ops = {
@@ -3112,6 +3404,15 @@ static const struct v4l2_subdev_pad_ops ccs_pad_ops = { .set_fmt = ccs_set_format, .get_selection = ccs_get_selection, .set_selection = ccs_set_selection, +}; + +static const struct v4l2_subdev_pad_ops ccs_src_pad_ops = { + .init_cfg = ccs_src_init_cfg, + .enum_mbus_code = ccs_enum_mbus_code, + .get_fmt = ccs_get_format, + .set_fmt = ccs_set_format, + .get_selection = ccs_get_selection, + .set_selection = ccs_set_selection, .enable_streams = ccs_enable_streams, .disable_streams = ccs_disable_streams, .get_frame_desc = ccs_get_frame_desc,
@@ -3128,6 +3429,12 @@ static const struct v4l2_subdev_ops ccs_ops = { .sensor = &ccs_sensor_ops, }; +static const struct v4l2_subdev_ops ccs_src_ops = { + .video = &ccs_video_ops, + .pad = &ccs_src_pad_ops, + .sensor = &ccs_sensor_ops, +}; +static const struct media_entity_operations ccs_entity_ops = { .link_validate = v4l2_subdev_link_validate, };
@@ -3283,7 +3590,7 @@ static int ccs_probe(struct i2c_client *client) sensor->src = &sensor->ssds[sensor->ssds_used]; -v4l2_i2c_subdev_init (&sensor->src->sd, client, &ccs_ops); +v4l2_i2c_subdev_init (&sensor->src->sd, client, &ccs_src_ops); sensor->src->sd.internal_ops = &ccs_internal_src_ops; sensor->regulators = devm_kcalloc(&client->dev,
@@ -3548,12 +3855,20 @@ static int ccs_probe(struct i2c_client *client) goto out_cleanup; } - rval =ccs_init_subdev (sensor, sensor->scaler," scaler" ,2 , + rval =ccs_init_subdev (sensor, sensor->scaler," scaler" , + sensor->ssds_used !=CCS_SUBDEVS ? +CCS_PADS_NOMETA : + sensor->embedded_start == sensor->embedded_end ? +CCS_PADS_NOMETA :CCS_PADS ,MEDIA_ENT_F_PROC_VIDEO_SCALER ,"ccs scaler mutex" , &scaler_lock_key);if (rval) goto out_cleanup; - rval =ccs_init_subdev (sensor, sensor->binner," binner" ,2 , + rval =ccs_init_subdev (sensor, sensor->binner," binner" , + sensor->ssds_used ==CCS_SUBDEVS ? +CCS_PADS_NOMETA : + sensor->embedded_start == sensor->embedded_end ? +CCS_PADS_NOMETA :CCS_PADS ,MEDIA_ENT_F_PROC_VIDEO_SCALER ,"ccs binner mutex" , &binner_lock_key);if (rval)
drivers/media/i2c/ccs/ccs.h +14 -4
@@ -172,11 +172,18 @@ struct ccs_csi_data_format {#define CCS_SUBDEVS 3#define CCS_PA_PAD_SRC 0 -#define CCS_PAD_SINK 0 -#define CCS_PAD_SRC 1 -#define CCS_PADS 2 +enum { +CCS_PAD_SINK , +CCS_PAD_SRC , +CCS_PAD_META , +CCS_PADS_NOMETA =CCS_PAD_META , +CCS_PADS , +}; -#define CCS_STREAM_PIXEL 0 +enum { +CCS_STREAM_PIXEL , +CCS_STREAM_META , +};struct ccs_binning_subtype {u8 horizontal:4 ;
@@ -226,6 +233,9 @@ struct ccs_sensor {int default_pixel_order;struct ccs_data_container sdata, mdata; +u32 embedded_mbus_code; +u8 emb_data_ctrl; +u8 binning_horizontal;u8 binning_vertical;
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 25/31 media: ccs: Remove ccs_get_crop_compose helper
In reply to Sakari AilusAs it's now easier to obtain the necessary information on crop and compose rectangles after moving to sub-device state, remove the ccs_get_crop_compose helper.
--- drivers/media/i2c/ccs/ccs-core.c | 130 ++++++++++++++----------------- 1 file changed, 58 insertions(+), 72 deletions(-)
drivers/media/i2c/ccs/ccs-core.c +58 -72
@@ -2210,24 +2210,6 @@ static int ccs_get_format(struct v4l2_subdev *subdev,return rval; } -static void ccs_get_crop_compose (struct v4l2_subdev *subdev, -struct v4l2_subdev_state *sd_state, -struct v4l2_rect **crops, -struct v4l2_rect **comps) -{ -struct ccs_subdev *ssd =to_ccs_subdev (subdev); -unsigned int i; - -if (crops) -for (i =0 ; i < subdev->entity.num_pads; i++) - crops[i] = -v4l2_subdev_state_get_crop (sd_state, i, -CCS_STREAM_PIXEL ); -if (comps) - *comps =v4l2_subdev_state_get_compose (sd_state, ssd->sink_pad, -CCS_STREAM_PIXEL ); -} -/* Changes require propagation only on sink pad. */ static void ccs_propagate(struct v4l2_subdev *subdev, struct v4l2_subdev_state *sd_state, int which,
@@ -2235,15 +2217,17 @@ static void ccs_propagate(struct v4l2_subdev *subdev, {struct ccs_sensor *sensor =to_ccs_sensor (subdev);struct ccs_subdev *ssd =to_ccs_subdev (subdev); -struct v4l2_rect *comp, *crops[CCS_PADS ]; +struct v4l2_rect *comp, *crop;struct v4l2_mbus_framefmt *fmt; -ccs_get_crop_compose (subdev, sd_state, crops, &comp); - + comp =v4l2_subdev_state_get_compose (sd_state, ssd->sink_pad, +CCS_STREAM_PIXEL );switch (target) {case V4L2_SEL_TGT_CROP : - comp->width = crops[CCS_PAD_SINK ]->width; - comp->height = crops[CCS_PAD_SINK ]->height; + crop =v4l2_subdev_state_get_crop (sd_state,CCS_PAD_SINK , +CCS_STREAM_PIXEL ); + comp->width = crop->width; + comp->height = crop->height;if (which ==V4L2_SUBDEV_FORMAT_ACTIVE ) {if (ssd == sensor->scaler) { sensor->scale_m =CCS_LIM (sensor,SCALER_N_MIN );
@@ -2257,13 +2241,15 @@ static void ccs_propagate(struct v4l2_subdev *subdev, } fallthrough;case V4L2_SEL_TGT_COMPOSE : - *crops[CCS_PAD_SRC ] = *comp; + crop =v4l2_subdev_state_get_crop (sd_state,CCS_PAD_SRC , +CCS_STREAM_PIXEL ); + *crop = *comp; fmt =v4l2_subdev_state_get_format (sd_state,CCS_PAD_SRC ,CCS_STREAM_PIXEL ); fmt->width = comp->width; fmt->height = comp->height;if (which ==V4L2_SUBDEV_FORMAT_ACTIVE && ssd == sensor->src) - sensor->src_src = *crops[CCS_PAD_SRC ]; + sensor->src_src = *crop;break ;default :WARN_ON_ONCE (1 );
@@ -2404,7 +2390,7 @@ static int ccs_set_format(struct v4l2_subdev *subdev, {struct ccs_sensor *sensor =to_ccs_sensor (subdev);struct ccs_subdev *ssd =to_ccs_subdev (subdev); -struct v4l2_rect *crops[CCS_PADS ]; +struct v4l2_rect *crop;if (subdev == &sensor->src->sd && fmt->pad ==CCS_PAD_META )return ccs_get_format (subdev, sd_state, fmt);
@@ -2446,12 +2432,13 @@ static int ccs_set_format(struct v4l2_subdev *subdev,CCS_LIM (sensor,MIN_Y_OUTPUT_SIZE ), CCS_LIM(sensor, MAX_Y_OUTPUT_SIZE)); -ccs_get_crop_compose (subdev, sd_state, crops,NULL ); + crop =v4l2_subdev_state_get_crop (sd_state, ssd->sink_pad, +CCS_STREAM_PIXEL ); - crops[ssd->sink_pad]->left =0 ; - crops[ssd->sink_pad]->top =0 ; - crops[ssd->sink_pad]->width = fmt->format.width; - crops[ssd->sink_pad]->height = fmt->format.height; + crop->left =0 ; + crop->top =0 ; + crop->width = fmt->format.width; + crop->height = fmt->format.height;ccs_propagate (subdev, sd_state, fmt->which,V4L2_SEL_TGT_CROP );mutex_unlock (&sensor->mutex);
@@ -2506,24 +2493,23 @@ static int scaling_goodness(struct v4l2_subdev *subdev, int w, int ask_w,static void ccs_set_compose_binner (struct v4l2_subdev *subdev,struct v4l2_subdev_state *sd_state,struct v4l2_subdev_selection *sel, -struct v4l2_rect **crops, +struct v4l2_rect *sink_crop,struct v4l2_rect *comp) {struct ccs_sensor *sensor =to_ccs_sensor (subdev);unsigned int i;unsigned int binh =1 , binv =1 ;int best =scaling_goodness ( - subdev, - crops[CCS_PAD_SINK ]->width, sel->r.width, - crops[CCS_PAD_SINK ]->height, sel->r.height, sel->flags); + subdev, sink_crop->width, sel->r.width, + sink_crop->height, sel->r.height, sel->flags);for (i =0 ; i < sensor->nbinning_subtypes; i++) {int this =scaling_goodness ( subdev, - crops[CCS_PAD_SINK ]->width + sink_crop->width / sensor->binning_subtypes[i].horizontal, sel->r.width, - crops[CCS_PAD_SINK ]->height + sink_crop->height / sensor->binning_subtypes[i].vertical, sel->r.height, sel->flags);
@@ -2538,8 +2524,8 @@ static void ccs_set_compose_binner(struct v4l2_subdev *subdev, sensor->binning_horizontal = binh; } - sel->r.width = (crops[CCS_PAD_SINK ]->width / binh) & ~1 ; - sel->r.height = (crops[CCS_PAD_SINK ]->height / binv) & ~1 ; + sel->r.width = (sink_crop->width / binh) & ~1 ; + sel->r.height = (sink_crop->height / binv) & ~1 ; }/*
@@ -2554,7 +2540,7 @@ static void ccs_set_compose_binner(struct v4l2_subdev *subdev,static void ccs_set_compose_scaler (struct v4l2_subdev *subdev,struct v4l2_subdev_state *sd_state,struct v4l2_subdev_selection *sel, -struct v4l2_rect **crops, +struct v4l2_rect *sink_crop,struct v4l2_rect *comp) {struct i2c_client *client =v4l2_get_subdevdata (subdev);
@@ -2567,17 +2553,12 @@ static void ccs_set_compose_scaler(struct v4l2_subdev *subdev,unsigned int i;int best =INT_MIN ; - sel->r.width = min_t(unsigned int, sel->r.width, - crops[CCS_PAD_SINK ]->width); - sel->r.height = min_t(unsigned int, sel->r.height, - crops[CCS_PAD_SINK ]->height); - - a = crops[CCS_PAD_SINK ]->width - *CCS_LIM (sensor,SCALER_N_MIN ) / sel->r.width; - b = crops[CCS_PAD_SINK ]->height - *CCS_LIM (sensor,SCALER_N_MIN ) / sel->r.height; - max_m = crops[CCS_PAD_SINK ]->width - *CCS_LIM (sensor,SCALER_N_MIN ) + sel->r.width = min_t(unsigned int, sel->r.width, sink_crop->width); + sel->r.height = min_t(unsigned int, sel->r.height, sink_crop->height); + + a = sink_crop->width *CCS_LIM (sensor,SCALER_N_MIN ) / sel->r.width; + b = sink_crop->height *CCS_LIM (sensor,SCALER_N_MIN ) / sel->r.height; + max_m = sink_crop->width *CCS_LIM (sensor,SCALER_N_MIN ) /CCS_LIM (sensor,MIN_X_OUTPUT_SIZE ); a = clamp(a, CCS_LIM(sensor, SCALER_M_MIN),
@@ -2610,10 +2591,10 @@ static void ccs_set_compose_scaler(struct v4l2_subdev *subdev,for (i =0 ; i < ntry; i++) {int this =scaling_goodness ( subdev, - crops[CCS_PAD_SINK ]->width + sink_crop->width / try[i] *CCS_LIM (sensor,SCALER_N_MIN ), sel->r.width, - crops[CCS_PAD_SINK ]->height, + sink_crop->height, sel->r.height, sel->flags);
@@ -2630,12 +2611,10 @@ static void ccs_set_compose_scaler(struct v4l2_subdev *subdev,continue ; this =scaling_goodness ( - subdev, crops[CCS_PAD_SINK ]->width - / try[i] + subdev, sink_crop->width / try[i] *CCS_LIM (sensor,SCALER_N_MIN ), sel->r.width, - crops[CCS_PAD_SINK ]->height - / try[i] + sink_crop->height / try[i] *CCS_LIM (sensor,SCALER_N_MIN ), sel->r.height, sel->flags);
@@ -2648,17 +2627,15 @@ static void ccs_set_compose_scaler(struct v4l2_subdev *subdev, } sel->r.width = - (crops[CCS_PAD_SINK ]->width - / scale_m + (sink_crop->width / scale_m *CCS_LIM (sensor,SCALER_N_MIN )) & ~1 ;if (mode ==SMIAPP_SCALING_MODE_BOTH ) sel->r.height = - (crops[CCS_PAD_SINK ]->height - / scale_m + (sink_crop->height / scale_m *CCS_LIM (sensor,SCALER_N_MIN )) & ~1 ;else - sel->r.height = crops[CCS_PAD_SINK ]->height; + sel->r.height = sink_crop->height;if (sel->which ==V4L2_SUBDEV_FORMAT_ACTIVE ) { sensor->scale_m = scale_m;
@@ -2672,17 +2649,20 @@ static int ccs_set_compose(struct v4l2_subdev *subdev, {struct ccs_sensor *sensor =to_ccs_sensor (subdev);struct ccs_subdev *ssd =to_ccs_subdev (subdev); -struct v4l2_rect *comp, *crops[CCS_PADS ]; +struct v4l2_rect *comp, *sink_crop; -ccs_get_crop_compose (subdev, sd_state, crops, &comp); + sink_crop =v4l2_subdev_state_get_crop (sd_state,CCS_PAD_SINK , +CCS_STREAM_PIXEL ); + comp =v4l2_subdev_state_get_compose (sd_state, ssd->sink_pad, +CCS_STREAM_PIXEL ); sel->r.top =0 ; sel->r.left =0 ;if (ssd == sensor->binner) -ccs_set_compose_binner (subdev, sd_state, sel, crops, comp); +ccs_set_compose_binner (subdev, sd_state, sel, sink_crop, comp);else -ccs_set_compose_scaler (subdev, sd_state, sel, crops, comp); +ccs_set_compose_scaler (subdev, sd_state, sel, sink_crop, comp); *comp = sel->r;ccs_propagate (subdev, sd_state, sel->which,V4L2_SEL_TGT_COMPOSE );
@@ -2743,9 +2723,12 @@ static int ccs_set_crop(struct v4l2_subdev *subdev, {struct ccs_sensor *sensor =to_ccs_sensor (subdev);struct ccs_subdev *ssd =to_ccs_subdev (subdev); -struct v4l2_rect src_size = {0 }, *crops[CCS_PADS ], *comp; +struct v4l2_rect src_size = {0 }, *crop, *comp; -ccs_get_crop_compose (subdev, sd_state, crops, &comp); + crop =v4l2_subdev_state_get_crop (sd_state, sel->pad, +CCS_STREAM_PIXEL ); + comp =v4l2_subdev_state_get_compose (sd_state, ssd->sink_pad, +CCS_STREAM_PIXEL );if (sel->pad == ssd->sink_pad) { struct v4l2_mbus_framefmt *mfmt =
@@ -2769,7 +2752,7 @@ static int ccs_set_crop(struct v4l2_subdev *subdev, sel->r.left =min_t (int, sel->r.left, src_size.width - sel->r.width); sel->r.top =min_t (int, sel->r.top, src_size.height - sel->r.height); - *crops[sel->pad] = sel->r; + *crop = sel->r;if (ssd != sensor->pixel_array && sel->pad ==CCS_PAD_SINK )ccs_propagate (subdev, sd_state, sel->which,V4L2_SEL_TGT_CROP );
@@ -2794,14 +2777,17 @@ static int ccs_get_selection(struct v4l2_subdev *subdev, {struct ccs_sensor *sensor =to_ccs_sensor (subdev);struct ccs_subdev *ssd =to_ccs_subdev (subdev); -struct v4l2_rect *comp, *crops[CCS_PADS ]; +struct v4l2_rect *crop, *comp;int ret; ret =ccs_sel_supported (subdev, sel);if (ret)return ret; -ccs_get_crop_compose (subdev, sd_state, crops, &comp); + crop =v4l2_subdev_state_get_crop (sd_state, sel->pad, +CCS_STREAM_PIXEL ); + comp =v4l2_subdev_state_get_compose (sd_state, ssd->sink_pad, +CCS_STREAM_PIXEL );switch (sel->target) {case V4L2_SEL_TGT_CROP_BOUNDS :
@@ -2823,7 +2809,7 @@ static int ccs_get_selection(struct v4l2_subdev *subdev,break ;case V4L2_SEL_TGT_CROP :case V4L2_SEL_TGT_COMPOSE_BOUNDS : - sel->r = *crops[sel->pad]; + sel->r = *crop;break ;case V4L2_SEL_TGT_COMPOSE : sel->r = *comp;
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 26/31 media: ccs: Rely on sub-device state locking
In reply to Sakari AilusRely on sub-device state locking to serialise access to driver's data structures. The driver-provided mutex is used as the state lock for all driver sub-devices.
--- drivers/media/i2c/ccs/ccs-core.c | 192 +++++++++++++------------------ drivers/media/i2c/ccs/ccs.h | 1 - 2 files changed, 81 insertions(+), 112 deletions(-)
drivers/media/i2c/ccs/ccs-core.c +81 -111
@@ -531,12 +531,13 @@ static int ccs_pll_update(struct ccs_sensor *sensor)* */ -static void __ccs_update_exposure_limits (struct ccs_sensor *sensor) +static void __ccs_update_exposure_limits (struct ccs_sensor *sensor, +struct v4l2_rect *pa_src) {struct v4l2_ctrl *ctrl = sensor->exposure;int max; - max = sensor->pa_src.height + sensor->vblank->val - + max = pa_src->height + sensor->vblank->val -CCS_LIM (sensor,COARSE_INTEGRATION_TIME_MAX_MARGIN );__v4l2_ctrl_modify_range (ctrl, ctrl->minimum, max, ctrl->step, max);
@@ -639,12 +640,20 @@ static int ccs_set_ctrl(struct v4l2_ctrl *ctrl) container_of(ctrl->handler, struct ccs_subdev, ctrl_handler) ->sensor;struct i2c_client *client =v4l2_get_subdevdata (&sensor->src->sd); +struct v4l2_subdev_state *state; +struct v4l2_rect *pa_src;int pm_status;u32 orient =0 ;unsigned int i;int exposure;int rval; +if (ctrl->id ==V4L2_CID_VBLANK || ctrl->id ==V4L2_CID_HBLANK ) { + state =v4l2_subdev_get_locked_active_state (&sensor->pixel_array->sd); + pa_src =v4l2_subdev_state_get_crop (state,CCS_PA_PAD_SRC , +CCS_STREAM_PIXEL ); + } +switch (ctrl->id) {case V4L2_CID_HFLIP :case V4L2_CID_VFLIP :
@@ -663,7 +672,7 @@ static int ccs_set_ctrl(struct v4l2_ctrl *ctrl)case V4L2_CID_VBLANK : exposure = sensor->exposure->val; -__ccs_update_exposure_limits (sensor); +__ccs_update_exposure_limits (sensor, pa_src);if (exposure > sensor->exposure->maximum) { sensor->exposure->val = sensor->exposure->maximum;
@@ -755,12 +764,12 @@ static int ccs_set_ctrl(struct v4l2_ctrl *ctrl)break ;case V4L2_CID_VBLANK : rval =ccs_write (sensor,FRAME_LENGTH_LINES , - sensor->pa_src.height + ctrl->val); + pa_src->height + ctrl->val);break ;case V4L2_CID_HBLANK : rval =ccs_write (sensor,LINE_LENGTH_PCK , - sensor->pa_src.width + ctrl->val); + pa_src->width + ctrl->val);break ;case V4L2_CID_TEST_PATTERN :
@@ -1215,7 +1224,8 @@ static int ccs_get_mbus_formats(struct ccs_sensor *sensor)return 0 ; } -static void ccs_update_blanking (struct ccs_sensor *sensor) +static void ccs_update_blanking (struct ccs_sensor *sensor, +struct v4l2_rect *pa_src) {struct v4l2_ctrl *vblank = sensor->vblank;struct v4l2_ctrl *hblank = sensor->hblank;
@@ -1238,21 +1248,26 @@ static void ccs_update_blanking(struct ccs_sensor *sensor) min =max_t (int,CCS_LIM (sensor,MIN_FRAME_BLANKING_LINES ), - min_fll - sensor->pa_src.height); - max = max_fll - sensor->pa_src.height; + min_fll - pa_src->height); + max = max_fll - pa_src->height;__v4l2_ctrl_modify_range (vblank, min, max, vblank->step, min); - min =max_t (int, min_llp - sensor->pa_src.width, min_lbp); - max = max_llp - sensor->pa_src.width; + min =max_t (int, min_llp - pa_src->width, min_lbp); + max = max_llp - pa_src->width;__v4l2_ctrl_modify_range (hblank, min, max, hblank->step, min); -__ccs_update_exposure_limits (sensor); +__ccs_update_exposure_limits (sensor, pa_src); }static int ccs_pll_blanking_update (struct ccs_sensor *sensor) { +struct v4l2_subdev_state *state = +v4l2_subdev_get_locked_active_state (&sensor->pixel_array->sd); +struct v4l2_rect *pa_src = +v4l2_subdev_state_get_crop (state,CCS_PA_PAD_SRC , +CCS_STREAM_PIXEL );struct i2c_client *client =v4l2_get_subdevdata (&sensor->src->sd);int rval;
@@ -1261,15 +1276,15 @@ static int ccs_pll_blanking_update(struct ccs_sensor *sensor)return rval;/* Output from pixel array, including blanking */ -ccs_update_blanking (sensor); +ccs_update_blanking (sensor, pa_src);dev_dbg (&client->dev,"vblank\t\t%d\n" , sensor->vblank->val);dev_dbg (&client->dev,"hblank\t\t%d\n" , sensor->hblank->val);dev_dbg (&client->dev,"real timeperframe\t100/%d\n" , sensor->pll.pixel_rate_pixel_array / - ((sensor->pa_src.width + sensor->hblank->val) * - (sensor->pa_src.height + sensor->vblank->val) /100 )); + ((pa_src->width + sensor->hblank->val) * + (pa_src->height + sensor->vblank->val) /100 ));return 0 ; }
@@ -1775,6 +1790,16 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev, u64 streams_mask) {struct ccs_sensor *sensor =to_ccs_sensor (subdev); +struct v4l2_subdev_state *pa_state = +v4l2_subdev_get_locked_active_state (&sensor->pixel_array->sd); +struct v4l2_subdev_state *src_state = +v4l2_subdev_get_locked_active_state (&sensor->src->sd); +struct v4l2_rect *pa_src = +v4l2_subdev_state_get_crop (pa_state,CCS_PA_PAD_SRC , +CCS_STREAM_PIXEL ); +struct v4l2_rect *src_src = +v4l2_subdev_state_get_crop (src_state,CCS_PAD_SRC , +CCS_STREAM_PIXEL );struct i2c_client *client =v4l2_get_subdevdata (&sensor->src->sd);unsigned int binning_mode;int rval;
@@ -1822,22 +1847,20 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev, goto err_pm_put;/* Analog crop start coordinates */ - rval =ccs_write (sensor,X_ADDR_START , sensor->pa_src.left); + rval =ccs_write (sensor,X_ADDR_START , pa_src->left);if (rval <0 ) goto err_pm_put; - rval =ccs_write (sensor,Y_ADDR_START , sensor->pa_src.top); + rval =ccs_write (sensor,Y_ADDR_START , pa_src->top);if (rval <0 ) goto err_pm_put;/* Analog crop end coordinates */ - rval =ccs_write (sensor,X_ADDR_END , - sensor->pa_src.left + sensor->pa_src.width -1 ); + rval =ccs_write (sensor,X_ADDR_END , pa_src->left + pa_src->width -1 );if (rval <0 ) goto err_pm_put; - rval =ccs_write (sensor,Y_ADDR_END , - sensor->pa_src.top + sensor->pa_src.height -1 ); + rval =ccs_write (sensor,Y_ADDR_END , pa_src->top + pa_src->height -1 );if (rval <0 ) goto err_pm_put;
@@ -1849,23 +1872,30 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev,/* Digital crop */ if (CCS_LIM (sensor,DIGITAL_CROP_CAPABILITY ) ==CCS_DIGITAL_CROP_CAPABILITY_INPUT_CROP ) { +struct v4l2_subdev_state *scaler_state = +v4l2_subdev_get_locked_active_state (&sensor->scaler->sd); +struct v4l2_rect *scaler_sink = +v4l2_subdev_state_get_crop (scaler_state, + sensor->scaler->sink_pad, +CCS_STREAM_PIXEL ); + rval =ccs_write (sensor,DIGITAL_CROP_X_OFFSET , - sensor->scaler_sink.left); + scaler_sink->left);if (rval <0 ) goto err_pm_put; rval =ccs_write (sensor,DIGITAL_CROP_Y_OFFSET , - sensor->scaler_sink.top); + scaler_sink->top);if (rval <0 ) goto err_pm_put; rval =ccs_write (sensor,DIGITAL_CROP_IMAGE_WIDTH , - sensor->scaler_sink.width); + scaler_sink->width);if (rval <0 ) goto err_pm_put; rval =ccs_write (sensor,DIGITAL_CROP_IMAGE_HEIGHT , - sensor->scaler_sink.height); + scaler_sink->height);if (rval <0 ) goto err_pm_put; }
@@ -1883,10 +1913,10 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev, }/* Output size from sensor */ - rval =ccs_write (sensor,X_OUTPUT_SIZE , sensor->src_src.width); + rval =ccs_write (sensor,X_OUTPUT_SIZE , src_src->width);if (rval <0 ) goto err_pm_put; - rval =ccs_write (sensor,Y_OUTPUT_SIZE , sensor->src_src.height); + rval =ccs_write (sensor,Y_OUTPUT_SIZE , src_src->height);if (rval <0 ) goto err_pm_put;
@@ -2075,9 +2105,6 @@ static int ccs_enum_mbus_code(struct v4l2_subdev *subdev,struct ccs_sensor *sensor =to_ccs_sensor (subdev);unsigned int i;int idx =-1 ; -int rval = -EINVAL ; - -mutex_lock (&sensor->mutex);dev_err (&client->dev,"subdev %s, pad %u, index %u\n" , subdev->name, code->pad, code->index);
@@ -2085,12 +2112,11 @@ static int ccs_enum_mbus_code(struct v4l2_subdev *subdev,if (subdev == &sensor->src->sd) {if (code->pad ==CCS_PAD_META ) {if (code->index) - goto out; +return -EINVAL ; code->code =MEDIA_BUS_FMT_CCS_EMBEDDED ; - rval =0 ; - goto out; +return 0 ; }if (code->stream ==CCS_STREAM_META ) { struct v4l2_mbus_framefmt *pix_fmt =
@@ -2137,21 +2163,21 @@ static int ccs_enum_mbus_code(struct v4l2_subdev *subdev, }if (WARN_ON (i >ARRAY_SIZE (codes)) || code->index >= i) - goto out; +return -EINVAL ; code->code = codes[code->index]; - rval =0 ; - goto out; + +return 0 ; } }if (subdev != &sensor->src->sd || code->pad !=CCS_PAD_SRC ) {if (code->index) - goto out; +return -EINVAL ; code->code = sensor->internal_csi_format->code; - rval =0 ; - goto out; + +return 0 ; }for (i =0 ; i <ARRAY_SIZE (ccs_csi_data_formats); i++) {
@@ -2162,18 +2188,14 @@ static int ccs_enum_mbus_code(struct v4l2_subdev *subdev, code->code = ccs_csi_data_formats[i].code;dev_err (&client->dev,"found index %u, i %u, code %x\n" , code->index, i, code->code); - rval =0 ; -break ; +return 0 ; } } -out: -mutex_unlock (&sensor->mutex); - -return rval; +return -EINVAL ; } -static u32 __ccs_get_mbus_code (struct v4l2_subdev *subdev,unsigned int pad) +static u32 ccs_get_mbus_code (struct v4l2_subdev *subdev,unsigned int pad) {struct ccs_sensor *sensor =to_ccs_sensor (subdev);
@@ -2183,33 +2205,19 @@ static u32 __ccs_get_mbus_code(struct v4l2_subdev *subdev, unsigned int pad)return sensor->internal_csi_format->code; } -static int __ccs_get_format (struct v4l2_subdev *subdev, -struct v4l2_subdev_state *sd_state, -struct v4l2_subdev_format *fmt) +static int ccs_get_format (struct v4l2_subdev *subdev, +struct v4l2_subdev_state *sd_state, +struct v4l2_subdev_format *fmt) { fmt->format = *v4l2_subdev_state_get_format (sd_state, fmt->pad, fmt->stream);if (fmt->pad !=CCS_PAD_META && fmt->stream !=CCS_STREAM_META ) - fmt->format.code =__ccs_get_mbus_code (subdev, fmt->pad); + fmt->format.code =ccs_get_mbus_code (subdev, fmt->pad);return 0 ; } -static int ccs_get_format (struct v4l2_subdev *subdev, -struct v4l2_subdev_state *sd_state, -struct v4l2_subdev_format *fmt) -{ -struct ccs_sensor *sensor =to_ccs_sensor (subdev); -int rval; - -mutex_lock (&sensor->mutex); - rval =__ccs_get_format (subdev, sd_state, fmt); -mutex_unlock (&sensor->mutex); - -return rval; -} -/* Changes require propagation only on sink pad. */ static void ccs_propagate(struct v4l2_subdev *subdev, struct v4l2_subdev_state *sd_state, int which,
@@ -2233,7 +2241,6 @@ static void ccs_propagate(struct v4l2_subdev *subdev, sensor->scale_m =CCS_LIM (sensor,SCALER_N_MIN ); sensor->scaling_mode =CCS_SCALING_MODE_NO_SCALING ; - sensor->scaler_sink = *comp; }else if (ssd == sensor->binner) { sensor->binning_horizontal =1 ; sensor->binning_vertical =1 ;
@@ -2248,8 +2255,6 @@ static void ccs_propagate(struct v4l2_subdev *subdev, CCS_STREAM_PIXEL); fmt->width = comp->width; fmt->height = comp->height; -if (which ==V4L2_SUBDEV_FORMAT_ACTIVE && ssd == sensor->src) - sensor->src_src = *crop;break ;default :WARN_ON_ONCE (1 );
@@ -2268,7 +2273,7 @@ static int ccs_set_format_source(struct v4l2_subdev *subdev,unsigned int i;int rval; - rval =__ccs_get_format (subdev, sd_state, fmt); + rval =ccs_get_format (subdev, sd_state, fmt);if (rval)return rval;
@@ -2395,13 +2400,9 @@ static int ccs_set_format(struct v4l2_subdev *subdev,if (subdev == &sensor->src->sd && fmt->pad ==CCS_PAD_META )return ccs_get_format (subdev, sd_state, fmt); -mutex_lock (&sensor->mutex); -if (subdev == &sensor->src->sd && fmt->stream ==CCS_STREAM_META ) {ccs_set_format_meta (subdev, sd_state, &fmt->format); -mutex_unlock (&sensor->mutex); -return 0 ; }
@@ -2412,13 +2413,12 @@ static int ccs_set_format(struct v4l2_subdev *subdev,if (sensor->embedded_start != sensor->embedded_end)ccs_set_format_meta (subdev, sd_state,NULL ); -mutex_unlock (&sensor->mutex); -return rval; }/* Sink pad. Width and height are changeable here. */ - fmt->format.code =__ccs_get_mbus_code (subdev, fmt->pad); + fmt->format.code =ccs_get_mbus_code (subdev, fmt->pad); + fmt->format.width &= ~1 ; fmt->format.height &= ~1 ; fmt->format.field =V4L2_FIELD_NONE ;
@@ -2441,8 +2441,6 @@ static int ccs_set_format(struct v4l2_subdev *subdev, crop->height = fmt->format.height;ccs_propagate (subdev, sd_state, fmt->which,V4L2_SEL_TGT_CROP ); -mutex_unlock (&sensor->mutex); -return 0 ; }
@@ -2756,9 +2754,6 @@ static int ccs_set_crop(struct v4l2_subdev *subdev,if (ssd != sensor->pixel_array && sel->pad ==CCS_PAD_SINK )ccs_propagate (subdev, sd_state, sel->which,V4L2_SEL_TGT_CROP ); -else if (sel->which ==V4L2_SUBDEV_FORMAT_ACTIVE && - ssd == sensor->pixel_array) - sensor->pa_src = sel->r;return 0 ; }
@@ -2830,8 +2825,6 @@ static int ccs_set_selection(struct v4l2_subdev *subdev,if (ret)return ret; -mutex_lock (&sensor->mutex); - sel->r.left =max (0 , sel->r.left & ~1 ); sel->r.top =max (0 , sel->r.top & ~1 ); sel->r.width =CCS_ALIGN_DIM (sel->r.width, sel->flags);
@@ -2853,7 +2846,6 @@ static int ccs_set_selection(struct v4l2_subdev *subdev, ret = -EINVAL ; } -mutex_unlock (&sensor->mutex);return ret; }
@@ -3240,6 +3232,7 @@ static int ccs_init_subdev(struct ccs_sensor *sensor, ssd->sd.flags |=V4L2_SUBDEV_FL_HAS_DEVNODE ; ssd->sd.entity.function = function; + ssd->sd.state_lock = &sensor->mutex; ssd->sensor = sensor; ssd->npads = num_pads;
@@ -3279,8 +3272,8 @@ static int ccs_init_subdev(struct ccs_sensor *sensor,return 0 ; } -static int __ccs_init_cfg (struct v4l2_subdev *sd, -struct v4l2_subdev_state *sd_state) +static int ccs_init_cfg (struct v4l2_subdev *sd, +struct v4l2_subdev_state *sd_state) {struct ccs_subdev *ssd =to_ccs_subdev (sd);struct ccs_sensor *sensor = ssd->sensor;
@@ -3299,12 +3292,8 @@ static int __ccs_init_cfg(struct v4l2_subdev *sd, fmt->code = sensor->internal_csi_format->code; fmt->field =V4L2_FIELD_NONE ; -if (ssd == sensor->pixel_array) { -if (is_active) - sensor->pa_src = *crop; - +if (ssd == sensor->pixel_array)return 0 ; - } fmt =v4l2_subdev_state_get_format (sd_state,CCS_PAD_SRC ,CCS_STREAM_PIXEL );
@@ -3317,20 +3306,6 @@ static int __ccs_init_cfg(struct v4l2_subdev *sd,return 0 ; } -static int ccs_init_cfg (struct v4l2_subdev *sd, -struct v4l2_subdev_state *sd_state) -{ -struct ccs_subdev *ssd =to_ccs_subdev (sd); -struct ccs_sensor *sensor = ssd->sensor; -int rval; - -mutex_lock (&sensor->mutex); - rval =__ccs_init_cfg (sd, sd_state); -mutex_unlock (&sensor->mutex); - -return rval; -} -static int ccs_src_init_cfg (struct v4l2_subdev *sd,struct v4l2_subdev_state *sd_state) {
@@ -3355,25 +3330,20 @@ static int ccs_src_init_cfg(struct v4l2_subdev *sd,struct ccs_sensor *sensor = ssd->sensor;int rval; -mutex_lock (&sensor->mutex); -if (sensor->embedded_start != sensor->embedded_end) routing.num_routes++; rval =v4l2_subdev_set_routing (sd, sd_state, &routing);if (rval) - goto out; +return 0 ; - rval =__ccs_init_cfg (sd, sd_state); + rval =ccs_init_cfg (sd, sd_state);if (rval) - goto out; +return 0 ;if (sensor->embedded_start != sensor->embedded_end)ccs_set_format_meta (sd, sd_state,NULL ); -out: -mutex_unlock (&sensor->mutex); -return rval; }
drivers/media/i2c/ccs/ccs.h +0 -1
@@ -228,7 +228,6 @@ struct ccs_sensor {u32 mbus_frame_fmts;const struct ccs_csi_data_format *csi_format;const struct ccs_csi_data_format *internal_csi_format; -struct v4l2_rect pa_src, scaler_sink, src_src;u32 default_mbus_frame_fmts;int default_pixel_order;struct ccs_data_container sdata, mdata;
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 27/31 media: ccs: Compute binning configuration from sub-device state
In reply to Sakari AilusCalculate binning configuration from sub-device state so the state related configuration can be removed from the driver's device context struct.
--- drivers/media/i2c/ccs/ccs-core.c | 72 ++++++++++++++++++++++---------- drivers/media/i2c/ccs/ccs.h | 3 -- 2 files changed, 49 insertions(+), 26 deletions(-)
drivers/media/i2c/ccs/ccs-core.c +49 -23
@@ -501,13 +501,52 @@ static int ccs_pll_try(struct ccs_sensor *sensor, struct ccs_pll *pll)return ccs_pll_calculate (&client->dev, &lim, pll); } +static void +ccs_get_binning (struct ccs_sensor *sensor,u8 *binning_mode,u8 *binh,u8 *binv) +{ +struct v4l2_subdev_state *binner_state = +v4l2_subdev_get_locked_active_state (&sensor->binner->sd); +struct v4l2_rect *binner_sink_crop = +v4l2_subdev_state_get_crop (binner_state,CCS_PAD_SINK , +CCS_STREAM_PIXEL ); +struct v4l2_rect *binner_sink_comp = +v4l2_subdev_state_get_compose (binner_state,CCS_PAD_SINK , +CCS_STREAM_PIXEL ); + +if (binner_sink_crop->width == binner_sink_comp->width && + binner_sink_crop->height == binner_sink_comp->height) { +if (binning_mode) + *binning_mode =0 ; + +if (binh) + *binh =1 ; + +if (binv) + *binv =1 ; + +return ; + } + +if (binning_mode) + *binning_mode =1 ; + +if (binh) + *binh = binner_sink_crop->width / binner_sink_comp->width; + +if (binv) + *binv = binner_sink_crop->height / binner_sink_comp->height; +} +static int ccs_pll_update (struct ccs_sensor *sensor) {struct ccs_pll *pll = &sensor->pll; +u8 binh, binv;int rval; - pll->binning_horizontal = sensor->binning_horizontal; - pll->binning_vertical = sensor->binning_vertical; +ccs_get_binning (sensor,NULL , &binh, &binv); + + pll->binning_horizontal = binh; + pll->binning_vertical = binv; pll->link_freq = sensor->link_freq->qmenu_int[sensor->link_freq->val]; pll->scale_m = sensor->scale_m;
@@ -1231,8 +1270,11 @@ static void ccs_update_blanking(struct ccs_sensor *sensor,struct v4l2_ctrl *hblank = sensor->hblank;u16 min_fll, max_fll, min_llp, max_llp, min_lbp;int min, max; +u8 binh, binv; + +ccs_get_binning (sensor,NULL , &binh, &binv); -if (sensor->binning_vertical >1 || sensor->binning_horizontal >1 ) { +if (binv >1 || binh >1 ) { min_fll =CCS_LIM (sensor,MIN_FRAME_LENGTH_LINES_BIN ); max_fll =CCS_LIM (sensor,MAX_FRAME_LENGTH_LINES_BIN ); min_llp =CCS_LIM (sensor,MIN_LINE_LENGTH_PCK_BIN );
@@ -1801,7 +1843,7 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev,v4l2_subdev_state_get_crop (src_state,CCS_PAD_SRC ,CCS_STREAM_PIXEL );struct i2c_client *client =v4l2_get_subdevdata (&sensor->src->sd); -unsigned int binning_mode; +u8 binning_mode, binh, binv;int rval;if (pad !=CCS_PAD_SRC )
@@ -1823,19 +1865,12 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev, goto err_pm_put;/* Binning configuration */ -if (sensor->binning_horizontal ==1 && - sensor->binning_vertical ==1 ) { - binning_mode =0 ; - }else { -u8 binning_type = - (sensor->binning_horizontal <<4 ) - | sensor->binning_vertical; +ccs_get_binning (sensor, &binning_mode, &binh, &binv); - rval =ccs_write (sensor,BINNING_TYPE , binning_type); +if (binning_mode) { + rval =ccs_write (sensor,BINNING_TYPE , (binh <<4 ) | binv);if (rval <0 ) goto err_pm_put; - - binning_mode =1 ; } rval =ccs_write (sensor,BINNING_MODE , binning_mode);if (rval <0 )
@@ -2241,9 +2276,6 @@ static void ccs_propagate(struct v4l2_subdev *subdev, sensor->scale_m =CCS_LIM (sensor,SCALER_N_MIN ); sensor->scaling_mode =CCS_SCALING_MODE_NO_SCALING ; - }else if (ssd == sensor->binner) { - sensor->binning_horizontal =1 ; - sensor->binning_vertical =1 ; } } fallthrough;
@@ -2517,10 +2549,6 @@ static void ccs_set_compose_binner(struct v4l2_subdev *subdev, best = this; } } -if (sel->which ==V4L2_SUBDEV_FORMAT_ACTIVE ) { - sensor->binning_vertical = binv; - sensor->binning_horizontal = binh; - } sel->r.width = (sink_crop->width / binh) & ~1 ; sel->r.height = (sink_crop->height / binv) & ~1 ;
@@ -3707,8 +3735,6 @@ static int ccs_probe(struct i2c_client *client) sensor->binning_subtypes[i].vertical); } } - sensor->binning_horizontal =1 ; - sensor->binning_vertical =1 ;if (device_create_file (&client->dev, &dev_attr_ident) !=0 ) {dev_err (&client->dev,"sysfs ident entry creation failed\n" );
drivers/media/i2c/ccs/ccs.h +0 -3
@@ -235,9 +235,6 @@ struct ccs_sensor {u32 embedded_mbus_code;u8 emb_data_ctrl; -u8 binning_horizontal; -u8 binning_vertical; -u8 scale_m;u8 scaling_mode;
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 28/31 media: ccs: Compute scaling configuration from sub-device state
In reply to Sakari AilusCompute scaling configuration from sub-device state instead of storing it to the driver's device context struct.
--- drivers/media/i2c/ccs/ccs-core.c | 60 ++++++++++++++++++++++---------- drivers/media/i2c/ccs/ccs.h | 3 -- 2 files changed, 41 insertions(+), 22 deletions(-)
drivers/media/i2c/ccs/ccs-core.c +41 -19
@@ -537,19 +537,52 @@ ccs_get_binning(struct ccs_sensor *sensor, u8 *binning_mode, u8 *binh, u8 *binv) *binv = binner_sink_crop->height / binner_sink_comp->height; } +static void ccs_get_scaling (struct ccs_sensor *sensor, +u8 *scaling_mode,u8 *scale_m) +{ +struct v4l2_subdev_state *scaler_state = +v4l2_subdev_get_locked_active_state (&sensor->scaler->sd); +struct v4l2_rect *scaler_sink_crop = +v4l2_subdev_state_get_crop (scaler_state,CCS_PAD_SINK , +CCS_STREAM_PIXEL ); +struct v4l2_rect *scaler_sink_comp = +v4l2_subdev_state_get_compose (scaler_state,CCS_PAD_SINK , +CCS_STREAM_PIXEL ); + +if (scale_m) + *scale_m = scaler_sink_crop->width * +CCS_LIM (sensor,SCALER_N_MIN ) / + scaler_sink_comp->width; + +if (scaling_mode) { +if (scaler_sink_crop->width == scaler_sink_comp->width) + *scaling_mode =CCS_SCALING_MODE_NO_SCALING ; +else if (scaler_sink_crop->height == scaler_sink_comp->height) + *scaling_mode =CCS_SCALING_MODE_HORIZONTAL ; +else + *scaling_mode =SMIAPP_SCALING_MODE_BOTH ; + } +} +static int ccs_pll_update (struct ccs_sensor *sensor) {struct ccs_pll *pll = &sensor->pll;u8 binh, binv; +u8 scale_m;int rval;ccs_get_binning (sensor,NULL , &binh, &binv); +if (sensor->scaler) +ccs_get_scaling (sensor,NULL , &scale_m); +else + scale_m =CCS_LIM (sensor,SCALER_N_MIN ); + pll->binning_horizontal = binh; pll->binning_vertical = binv; pll->link_freq = sensor->link_freq->qmenu_int[sensor->link_freq->val]; - pll->scale_m = sensor->scale_m; + pll->scale_m = scale_m; pll->bits_per_pixel = sensor->csi_format->compressed; rval =ccs_pll_try (sensor, pll);
@@ -1192,7 +1225,7 @@ static int ccs_get_mbus_formats(struct ccs_sensor *sensor)/* Figure out which BPP values can be used with which formats. */ pll->binning_horizontal =1 ; pll->binning_vertical =1 ; - pll->scale_m = sensor->scale_m; + pll->scale_m =CCS_LIM (sensor,SCALER_N_MIN );for (i =0 ; i <ARRAY_SIZE (ccs_csi_data_formats); i++) { sensor->compressed_min_bpp =
@@ -1938,11 +1971,15 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev,/* Scaling */ if (CCS_LIM (sensor,SCALING_CAPABILITY ) !=CCS_SCALING_CAPABILITY_NONE ) { - rval =ccs_write (sensor,SCALING_MODE , sensor->scaling_mode); +u8 scaling_mode, scale_m; + +ccs_get_scaling (sensor, &scaling_mode, &scale_m); + + rval =ccs_write (sensor,SCALING_MODE , scaling_mode);if (rval <0 ) goto err_pm_put; - rval =ccs_write (sensor,SCALE_M , sensor->scale_m); + rval =ccs_write (sensor,SCALE_M , scale_m);if (rval <0 ) goto err_pm_put; }
@@ -2258,7 +2295,6 @@ static void ccs_propagate(struct v4l2_subdev *subdev, struct v4l2_subdev_state *sd_state, int which, int target) { - struct ccs_sensor *sensor = to_ccs_sensor(subdev); struct ccs_subdev *ssd = to_ccs_subdev(subdev);struct v4l2_rect *comp, *crop;struct v4l2_mbus_framefmt *fmt;
@@ -2271,13 +2307,6 @@ static void ccs_propagate(struct v4l2_subdev *subdev, CCS_STREAM_PIXEL); comp->width = crop->width; comp->height = crop->height; -if (which ==V4L2_SUBDEV_FORMAT_ACTIVE ) { -if (ssd == sensor->scaler) { - sensor->scale_m =CCS_LIM (sensor,SCALER_N_MIN ); - sensor->scaling_mode = -CCS_SCALING_MODE_NO_SCALING ; - } - } fallthrough;case V4L2_SEL_TGT_COMPOSE : crop = v4l2_subdev_state_get_crop(sd_state, CCS_PAD_SRC,
@@ -2662,11 +2691,6 @@ static void ccs_set_compose_scaler(struct v4l2_subdev *subdev, & ~1 ; else sel->r.height = sink_crop->height; - -if (sel->which ==V4L2_SUBDEV_FORMAT_ACTIVE ) { - sensor->scale_m = scale_m; - sensor->scaling_mode = mode; - } }/* We're only called on source pads. This function sets scaling. */ static int ccs_set_compose(struct v4l2_subdev *subdev,
@@ -3771,8 +3795,6 @@ static int ccs_probe(struct i2c_client *client) sensor->pixel_array = &sensor->ssds[sensor->ssds_used]; sensor->ssds_used++; - sensor->scale_m =CCS_LIM (sensor,SCALER_N_MIN ); -/* prepare PLL configuration input values */ sensor->pll.bus_type =CCS_PLL_BUS_TYPE_CSI2_DPHY ; sensor->pll.csi2.lanes = sensor->hwcfg.lanes;
drivers/media/i2c/ccs/ccs.h +0 -3
@@ -235,9 +235,6 @@ struct ccs_sensor {u32 embedded_mbus_code;u8 emb_data_ctrl; -u8 scale_m; -u8 scaling_mode; -u8 frame_skip;u16 embedded_start;/* embedded data start line */ u16 embedded_end;
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 29/31 media: ccs: Remove which parameter from ccs_propagate
In reply to Sakari Ailusccs_propagate() no longer stores information in the driver's context struct. The which parameter can thus be removed.
--- drivers/media/i2c/ccs/ccs-core.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-)
drivers/media/i2c/ccs/ccs-core.c +5 -7
@@ -2292,8 +2292,7 @@ static int ccs_get_format(struct v4l2_subdev *subdev,/* Changes require propagation only on sink pad. */ static void ccs_propagate (struct v4l2_subdev *subdev, -struct v4l2_subdev_state *sd_state,int which, -int target) +struct v4l2_subdev_state *sd_state,int target) {struct ccs_subdev *ssd =to_ccs_subdev (subdev);struct v4l2_rect *comp, *crop;
@@ -2500,7 +2499,7 @@ static int ccs_set_format(struct v4l2_subdev *subdev, crop->top =0 ; crop->width = fmt->format.width; crop->height = fmt->format.height; -ccs_propagate (subdev, sd_state, fmt->which,V4L2_SEL_TGT_CROP ); +ccs_propagate (subdev, sd_state,V4L2_SEL_TGT_CROP );return 0 ; }
@@ -2715,7 +2714,7 @@ static int ccs_set_compose(struct v4l2_subdev *subdev,ccs_set_compose_scaler (subdev, sd_state, sel, sink_crop, comp); *comp = sel->r; -ccs_propagate (subdev, sd_state, sel->which,V4L2_SEL_TGT_COMPOSE ); +ccs_propagate (subdev, sd_state,V4L2_SEL_TGT_COMPOSE );if (sel->which ==V4L2_SUBDEV_FORMAT_ACTIVE )return ccs_pll_blanking_update (sensor);
@@ -2805,7 +2804,7 @@ static int ccs_set_crop(struct v4l2_subdev *subdev, *crop = sel->r;if (ssd != sensor->pixel_array && sel->pad ==CCS_PAD_SINK ) -ccs_propagate (subdev, sd_state, sel->which,V4L2_SEL_TGT_CROP ); +ccs_propagate (subdev, sd_state,V4L2_SEL_TGT_CROP );return 0 ; }
@@ -3335,7 +3334,6 @@ static int ccs_init_cfg(struct v4l2_subdev *sd,v4l2_subdev_state_get_format (sd_state, pad,CCS_STREAM_PIXEL );struct v4l2_rect *crop =v4l2_subdev_state_get_crop (sd_state, pad,CCS_STREAM_PIXEL ); -bool is_active = !sd->active_state || sd->active_state == sd_state;ccs_get_native_size (ssd, crop);
@@ -3353,7 +3351,7 @@ static int ccs_init_cfg(struct v4l2_subdev *sd, sensor->csi_format->code : sensor->internal_csi_format->code; fmt->field =V4L2_FIELD_NONE ; -ccs_propagate (sd, sd_state, is_active,V4L2_SEL_TGT_CROP ); +ccs_propagate (sd, sd_state,V4L2_SEL_TGT_CROP );return 0 ; }
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 30/31 media: Documentation: ccs: Document routing
In reply to Sakari AilusDocument which routes are available for the CCS driver (source) sub-device and what configuration are possible.
Also update copyright.
--- .../userspace-api/media/drivers/ccs.rst | 34 ++++++++++++++++++- .../media/v4l/subdev-formats.rst | 2 ++ 2 files changed, 35 insertions(+), 1 deletion(-)
Documentation/userspace-api/media/drivers/ccs.rst +33 -1
@@ -107,4 +107,36 @@ than in the centre. Shading correction needs to be enabled for luminance correction level to have an effect. -**Copyright** |copy| 2020 Intel Corporation +.. _media-ccs-routes: + +Routes +------ + +The CCS driver implements one or two :ref:`routes <subdev-routing>` in +its source sub-device (scaler sub-device if exists for the device, otherwise +binner) depending on whether the sensor supports embedded data. (All CCS +compliant sensors do but the CCS driver supports preceding standards that did +not require embedded data support, too.) + +The first route of the CCS source sub-device is for pixel data (internal pad +1/stream 0 -> pad 0/stream 0) and the second one is for embedded data (internal +pad 2/stream 0 -> pad 0/stream 1). + +Embedded data +~~~~~~~~~~~~~ + +MIPI CCS supports generation of camera sensor embedded data. The media bus code +used for this format is :ref:`MEDIA_BUS_FMT_CCS_EMBEDDDED +<MEDIA-BUS-FMT-CCS-EMBEDDED>`. + +The bit depth of the CCS pixel data affects how the sensor will output the +embedded data, adding padding to align with CSI-2 bus :ref:`Data units +<media-glossary-data-unit>` for that particular bit depth. This is indicated by +the generic metadata format on the sensor's source sub-device's source pad. + +Embedded data for bit depths greater than or equal to 16 may support more dense +packing or legacy single metadata byte per data unit, or both of these, +depending on the device. The supported embedded data formats can be enumerated +and configured on stream 1 of the source pad (1) of the CCS source sub-device. + +**Copyright** |copy| 2020, 2023 Intel Corporation
Documentation/userspace-api/media/v4l/subdev-formats.rst +2 -0
@@ -8520,3 +8520,5 @@ levels above. This mbus code are only used for "2-byte simplified tagged data format" (code 0xa) but their use may be extended further in the future, to cover other CCS embedded data format codes. + +Also see :ref:`CCS driver documentation <media-ccs-routes>`.
Signature · 1 line
2.39.2
Sakari Ailus
maintainerPATCH v7 31/31 media: uapi: v4l: subdev: Enable streams API
In reply to Sakari AilusRemove v4l2_subdev_enable_streams_api variable that was used to easily enable streams API for development, and conditions that use the variable.
This patch enables the streams API for V4L2 sub-device interface which allows transporting multiple streams on a single MC link.
--- drivers/media/v4l2-core/v4l2-subdev.c | 23 ----------------------- 1 file changed, 23 deletions(-)
drivers/media/v4l2-core/v4l2-subdev.c +0 -23
@@ -26,15 +26,6 @@#include <media/v4l2-fh.h> #include <media/v4l2-ioctl.h> -#if defined(CONFIG_VIDEO_V4L2_SUBDEV_API ) -/* -* The Streams API is an experimental feature. To use the Streams API, set -* 'v4l2_subdev_enable_streams_api' to 1 below. -*/ - -static bool v4l2_subdev_enable_streams_api; -#endif -/* * Maximum stream ID is 63 for now, as we use u64 bitmask to represent a set * of streams.
@@ -894,9 +885,6 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,struct v4l2_subdev_krouting krouting = {};unsigned int i; -if (!v4l2_subdev_enable_streams_api) -return -ENOIOCTLCMD ; -if (!(sd->flags &V4L2_SUBDEV_FL_STREAMS ))return -ENOIOCTLCMD ;
@@ -967,9 +955,6 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,struct v4l2_subdev_routing *routing = arg;struct v4l2_subdev_krouting *krouting; -if (!v4l2_subdev_enable_streams_api) -return -ENOIOCTLCMD ; -if (!(sd->flags &V4L2_SUBDEV_FL_STREAMS ))return -ENOIOCTLCMD ;
@@ -997,14 +982,6 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,case VIDIOC_SUBDEV_S_CLIENT_CAP : {struct v4l2_subdev_client_capability *client_cap = arg; -/* -* Clear V4L2_SUBDEV_CLIENT_CAP_STREAMS if streams API is not -* enabled. Remove this when streams API is no longer -* experimental. -*/ -if (!v4l2_subdev_enable_streams_api) - client_cap->capabilities &= ~V4L2_SUBDEV_CLIENT_CAP_STREAMS ; -/* Filter out unsupported capabilities */ client_cap->capabilities &=V4L2_SUBDEV_CLIENT_CAP_STREAMS ;
Signature · 1 line
2.39.2
Quoted 1 line · Sakari Ailus
<URL:https://git.linuxtv.org/sailus/media_tree.git/log/?h=metadata>
I've updated this, the patches are on top of the soon-to-be-merged sub-device kAPI changes.
Signature · 1 line
Sakari Ailus