Sub-device configuration models
v5 cover Sub-device configuration models
Hello everyone,
I've been recently working (with others) on sub-device streams support as well as on internal pads. The two can be used to make sub-device configuration more versatile.
At the same time, the added interfaces are much more useful if we require specific semantics of those interfaces, so that the user space knows exactly what e.g. a given selection target signifies. However, as the same selection rectangle could be used for a different purpose on a non-raw sensor device, we need a way to tell how should the user space determine how to use a given interface.
I'm proposing to solve this problem by introducing sub-device configuration models, and by the common raw sensor model, also present in this patchset, in particular.
This has been (and will, for some time, continue to be) the reason why I have reviewed few sensor driver related patches lately. As we're introducing a new interface, it's beneficial to be able to use that interface right from the start, rather than trying to later on offer compatibility support, which is almost always a fair amount of work with less than desirable results in the driver.
With this solved, I believe we can enable the use of the streams UAPI.
Comments are welcome.
RFC v4 is <[email protected]> on LMML.
RFC v3 is <[email protected]> on LMML.
RFC v2 (labelled as PATCH) is <[email protected]> on LMML.
RFC v1 is <[email protected]> on LMML.
We may end up postponing merging three last patches (those adding controls) until current control values are moved to sub-device (and video device) state. Discussion is needed on the topic.
The compiled documentation can be found here <URL:https://www.retiisi.eu/~sailus/v4l2/tmp/common-raw/output/userspace-api/media/v4l/dev-subdev.html#media-subdev-config-model-common-raw-sensor>.
since RFC v4:
- Document the use of luma formats for generic raw data on both mbus codes and V4L2 pixelformats, with new controls V4L2_CID_COLOUR_PATTERN and V4L2_CID_COLOUR_PATTERN_FLIP determine the pattern in different flipping configurations. This enables the use of generic pattern-agnostic formats whereas controls are used to express colour components and their order.
- Add a patch that reworks the metadata format 4ccs. These haven't been in use in any case so far.
- Improve documentation column widths.
since RFC v3:
- Fix binning control name in documentation.
- Note V4L2_CID_CONFIG_MODEL is read-only.
- Clarify wording in config model documentation based on Jacopo's comments.
- Clarify post-scaler digital crop behaviour.
since RFC v2:
- Remove the CROP_BOUNDS rectangle on the image data sink pad. It's redundant.
- Remove scaling from the sink compose rectangle, leaving binning and sub-sampling.
- Introduce scaling as well as post-scaler digital crop on source pad (image stream). This is a new patch, preceding control changes.
- Use monospace font for selection targets.
- Improvements in referring to previous configuration steps in text description.
since RFC v1:
- Rework the documentation according to the comments (spelling fixes, alignment mostly).
- Include separate patches for binning and sub-sampling controls.
- Allow binning, scaling and scaling to be configured using the compose rectangle. This will change to just scaling in the three last patches.
Sakari Ailus (15):
media: Documentation: Rework embedded data documentation
media: Documentation: Add a hyphen to list-based
media: Documentation: Reword split of sensor driver to two classes
media: Documentation: Add subdev configuration models, raw sensor
model
media: Documentation: Add scaling and post-scaler crop for common raw
media: uapi: Add V4L2_CID_CONFIG_MODEL control
media: uapi: Add V4L2_CID_COLOUR_PATTERN for describing colour
patterns
media: uapi: Correct generic CSI-2 metadata format 4cc
media: uapi: Documentation: Improve column width hints for examples
media: v4l: uapi: Add a control for colour pattern flipping effect
media: Documentation: Document luma-only mbus codes and CFA for
cameras
media: uapi: Documentation: Use luma formats with CFA pattern control
media: uapi: Add V4L2_CID_BINNING control for binning configuration
media: uapi: Add controls for sub-sampling configuration
media: Documentation: Add binning and sub-sampling controls
.../media/drivers/camera-sensor.rst | 66 ++- .../media/v4l/common-raw-sensor.dia | 441 ++++++++++++++++++ .../media/v4l/common-raw-sensor.svg | 134 ++++++ .../userspace-api/media/v4l/dev-subdev.rst | 2 + .../media/v4l/ext-ctrls-camera.rst | 40 ++ .../media/v4l/ext-ctrls-image-process.rst | 4 + .../media/v4l/ext-ctrls-image-source.rst | 22 + .../media/v4l/metafmt-generic.rst | 10 +- .../media/v4l/pixfmt-yuv-luma.rst | 5 + .../media/v4l/subdev-config-model.rst | 233 +++++++++ drivers/media/v4l2-core/v4l2-ctrls-defs.c | 12 + include/uapi/linux/v4l2-controls.h | 27 ++ include/uapi/linux/videodev2.h | 12 +- 13 files changed, 983 insertions(+), 25 deletions(-) create mode 100644 Documentation/userspace-api/media/v4l/common-raw-sensor.dia create mode 100644 Documentation/userspace-api/media/v4l/common-raw-sensor.svg create mode 100644 Documentation/userspace-api/media/v4l/subdev-config-model.rst
Signature · 1 line
2.39.5
RFC v5 1/15 media: Documentation: Rework embedded data documentation
Rework embedded data documentation by removing the reference to the pixel data stream. The specific documentation of the embedded data interface will be elsewhere, in the near future either CCS or common raw camera sensor model documentation.
--- .../userspace-api/media/drivers/camera-sensor.rst | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-)
Documentation/userspace-api/media/drivers/camera-sensor.rst +6 -7
@@ -111,13 +111,12 @@ the sensor configuration for the captured frame back to the host. While CSI-2 is the most common data interface used by such sensors, embedded data can be available on other interfaces as well. -Such sensors expose two internal sink pads (pads that have both the -``MEDIA_PAD_FL_SINK <MEDIA-PAD-FL-SINK>`` and ``MEDIA_PAD_FL_INTERNAL -<MEDIA-PAD-FL-INTERNAL>`` flags set) to model the source of the image and -embedded data streams. Both of these pads produces a single stream, and the -sub-device routes those streams to the external (source) pad. If the sub-device -driver supports disabling embedded data, this can be done by disabling the -embedded data route via the ``VIDIOC_SUBDEV_S_ROUTING`` IOCTL. +Embedded data support is indicated by the precence of an internal sink pad (pad +that has both the ``MEDIA_PAD_FL_SINK <MEDIA-PAD-FL-SINK>`` and +``MEDIA_PAD_FL_INTERNAL <MEDIA-PAD-FL-INTERNAL>`` flags set) with a metadata +format to model the embedded data stream. If the sub-device driver supports +disabling embedded data, this can be done by disabling the embedded data route +via the ``VIDIOC_SUBDEV_S_ROUTING`` IOCTL. In general, changing the embedded data format from the driver-configured values is not supported. The height of the metadata is device-specific and the width
Signature · 1 line
2.39.5
Re: [RFC v5 01/15] media: Documentation: Rework embedded data documentation
Quoted 6 lines · Sakari Ailus
Rework embedded data documentation by removing the reference to the pixel data stream. The specific documentation of the embedded data interface will be elsewhere, in the near future either CCS or common raw camera sensor model documentation.
Quoted 8 lines
Signed-off-by Sakari Ailus <[email protected]>Reviewed-by Jacopo Mondi <[email protected]>Reviewed-by Laurent Pinchart <[email protected]>Reviewed-by Tomi Valkeinen <[email protected]>--- .../userspace-api/media/drivers/camera-sensor.rst | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-)
Cheers,
Prabhakar
Quoted 27 lines
Documentation/userspace-api/media/drivers/camera-sensor.rst +6
-8@@ -111,13 +111,12 @@ the sensor configuration for the captured frame back to the host. While CSI-2 is the most common data interface used by such sensors, embedded data can be available on other interfaces as well. -Such sensors expose two internal sink pads (pads that have both the -``MEDIA_PAD_FL_SINK <MEDIA-PAD-FL-SINK>`` and ``MEDIA_PAD_FL_INTERNAL -<MEDIA-PAD-FL-INTERNAL>`` flags set) to model the source of the image and -embedded data streams. Both of these pads produces a single stream, and the -sub-device routes those streams to the external (source) pad. If the sub-device -driver supports disabling embedded data, this can be done by disabling the -embedded data route via the ``VIDIOC_SUBDEV_S_ROUTING`` IOCTL. +Embedded data support is indicated by the precence of an internal sink pad (pad +that has both the ``MEDIA_PAD_FL_SINK <MEDIA-PAD-FL-SINK>`` and +``MEDIA_PAD_FL_INTERNAL <MEDIA-PAD-FL-INTERNAL>`` flags set) with a metadata +format to model the embedded data stream. If the sub-device driver supports +disabling embedded data, this can be done by disabling the embedded data route +via the ``VIDIOC_SUBDEV_S_ROUTING`` IOCTL. In general, changing the embedded data format from the driver-configured values is not supported. The height of the metadata is device-specific and the width --2.39.5
Re: RFC v5 01/15] media: Documentation: Rework embedded data documentation
In reply to Sakari AilusHi Sakari,
Quoted 19 lines · Sakari Ailus
Rework embedded data documentation by removing the reference to the pixel data stream. The specific documentation of the embedded data interface will be elsewhere, in the near future either CCS or common raw camera sensor model documentation.
Signed-off-by Sakari Ailus <[email protected]>Reviewed-by Jacopo Mondi <[email protected]>Reviewed-by Laurent Pinchart <[email protected]>Reviewed-by Tomi Valkeinen <[email protected]>--- .../userspace-api/media/drivers/camera-sensor.rst | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-)Documentation/userspace-api/media/drivers/camera-sensor.rst +0
-0@@ -111,13 +111,12 @@ the sensor configuration for the captured frame back to the host. While CSI-2 is the most common data interface used by such sensors, embedded data can be
Documentation/userspace-api/media/drivers/camera-sensor.rst the sensor configuration for the captured frame back to the host. While CSI-2 is · line 114
Quoted 10 lines
Documentation/userspace-api/media/drivers/camera-sensor.rst +1
-7available on other interfaces as well. -Such sensors expose two internal sink pads (pads that have both the -``MEDIA_PAD_FL_SINK <MEDIA-PAD-FL-SINK>`` and ``MEDIA_PAD_FL_INTERNAL -<MEDIA-PAD-FL-INTERNAL>`` flags set) to model the source of the image and -embedded data streams. Both of these pads produces a single stream, and the -sub-device routes those streams to the external (source) pad. If the sub-device -driver supports disabling embedded data, this can be done by disabling the -embedded data route via the ``VIDIOC_SUBDEV_S_ROUTING`` IOCTL. +Embedded data support is indicated by the precence of an internal sink pad (pad
Just a small typo here: precence.
Other than that, Reviewed-by: Mirela Rabulea <[email protected]>
Regards,
Mirela
Quoted 11 lines
Documentation/userspace-api/media/drivers/camera-sensor.rst +5
-1+that has both the ``MEDIA_PAD_FL_SINK <MEDIA-PAD-FL-SINK>`` and +``MEDIA_PAD_FL_INTERNAL <MEDIA-PAD-FL-INTERNAL>`` flags set) with a metadata +format to model the embedded data stream. If the sub-device driver supports +disabling embedded data, this can be done by disabling the embedded data route +via the ``VIDIOC_SUBDEV_S_ROUTING`` IOCTL. In general, changing the embedded data format from the driver-configured values is not supported. The height of the metadata is device-specific and the width --2.39.5
RFC v5 2/15 media: Documentation: Add a hyphen to list-based
In reply to Sakari AilusAdd a hyphen to list-based for uniform spelling in camera-sensor.rst.
--- Documentation/userspace-api/media/drivers/camera-sensor.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
Documentation/userspace-api/media/drivers/camera-sensor.rst +3 -3
@@ -26,10 +26,10 @@ of cropping and scaling operations from the device's pixel array's size. An example of such a driver is the CCS driver. -Register list based drivers +Register list-based drivers ~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Register list based drivers generally, instead of able to configure the device +Register list-based drivers generally, instead of able to configure the device they control based on user requests, are limited to a number of preset configurations that combine a number of different parameters that on hardware level are independent. How a driver picks such configuration is based on the
@@ -67,7 +67,7 @@ is pixels and the unit of the ``V4L2_CID_VBLANK`` is lines. The pixel rate in the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the same sub-device. The unit of that control is pixels per second. -Register list based drivers need to implement read-only sub-device nodes for the +Register list-based drivers need to implement read-only sub-device nodes for the purpose. Devices that are not register list based need these to configure the device's internal processing pipeline.
Signature · 1 line
2.39.5
Re: [RFC v5 02/15] media: Documentation: Add a hyphen to list-based
Quoted 10 lines · Sakari Ailus
Add a hyphen to list-based for uniform spelling in
camera-sensor.rst.Signed-off-by Sakari Ailus <[email protected]>Reviewed-by Jacopo Mondi <[email protected]>Reviewed-by Tomi Valkeinen <[email protected]>--- Documentation/userspace-api/media/drivers/camera-sensor.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
Cheers,
Prabhakar
Quoted 29 lines
Documentation/userspace-api/media/drivers/camera-sensor.rst +3
-4@@ -26,10 +26,10 @@ of cropping and scaling operations from the device's pixel array's size. An example of such a driver is the CCS driver. -Register list based drivers +Register list-based drivers ~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Register list based drivers generally, instead of able to configure the device +Register list-based drivers generally, instead of able to configure the device they control based on user requests, are limited to a number of preset configurations that combine a number of different parameters that on hardware level are independent. How a driver picks such configuration is based on the@@ -67,7 +67,7 @@ is pixels and the unit of the ``V4L2_CID_VBLANK`` is lines. The pixel rate in the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the same sub-device. The unit of that control is pixels per second. -Register list based drivers need to implement read-only sub-device nodes for the +Register list-based drivers need to implement read-only sub-device nodes for the purpose. Devices that are not register list based need these to configure the device's internal processing pipeline. --2.39.5
RFC v5 3/15 media: Documentation: Reword split of sensor driver to two classes
In reply to Sakari AilusThe sensor drivers do not configure the output size of the sensors but the entire internal pipeline. Reflect this in the documentation.
--- .../userspace-api/media/drivers/camera-sensor.rst | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
Documentation/userspace-api/media/drivers/camera-sensor.rst +6 -4
@@ -10,11 +10,13 @@ used to control the camera sensor drivers. You may also find :ref:`media_writing_camera_sensor_drivers` useful. -Frame size ----------- +Sensor internal pipeline configuration +-------------------------------------- -There are two distinct ways to configure the frame size produced by camera -sensors. +Camera sensors have an internal processing pipeline including cropping and +binning functionality. The sensor drivers belong to two distinct classes, freely +configurable and register list-based drivers, depending on how the driver +configures this functionality. Freely configurable camera sensor drivers ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Signature · 1 line
2.39.5
Re: [RFC v5 03/15] media: Documentation: Reword split of sensor driver to two classes
Quoted 4 lines · Sakari Ailus
The sensor drivers do not configure the output size of the sensors but the entire internal pipeline. Reflect this in the documentation.
Quoted 8 lines
Signed-off-by Sakari Ailus <[email protected]>Reviewed-by Laurent Pinchart <[email protected]>Reviewed-by Jacopo Mondi <[email protected]>Reviewed-by Tomi Valkeinen <[email protected]>--- .../userspace-api/media/drivers/camera-sensor.rst | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
Cheers,
Prabhakar
Quoted 25 lines
Documentation/userspace-api/media/drivers/camera-sensor.rst +6
-5@@ -10,11 +10,13 @@ used to control the camera sensor drivers. You may also find :ref:`media_writing_camera_sensor_drivers` useful. -Frame size ----------- +Sensor internal pipeline configuration +-------------------------------------- -There are two distinct ways to configure the frame size produced by camera -sensors. +Camera sensors have an internal processing pipeline including cropping and +binning functionality. The sensor drivers belong to two distinct classes, freely +configurable and register list-based drivers, depending on how the driver +configures this functionality. Freely configurable camera sensor drivers ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ --2.39.5
RFC v5 4/15 media: Documentation: Add subdev configuration models, raw sensor model
In reply to Sakari AilusSub-device configuration models define what V4L2 API elements are available on a compliant sub-device and how do they behave.
The patch also adds a model for common raw sensors.
--- .../media/drivers/camera-sensor.rst | 4 + .../media/v4l/common-raw-sensor.dia | 441 ++++++++++++++++++ .../media/v4l/common-raw-sensor.svg | 134 ++++++ .../userspace-api/media/v4l/dev-subdev.rst | 2 + .../media/v4l/subdev-config-model.rst | 209 +++++++++ 5 files changed, 790 insertions(+) create mode 100644 Documentation/userspace-api/media/v4l/common-raw-sensor.dia create mode 100644 Documentation/userspace-api/media/v4l/common-raw-sensor.svg create mode 100644 Documentation/userspace-api/media/v4l/subdev-config-model.rst
Documentation/userspace-api/media/drivers/camera-sensor.rst +4 -0
@@ -18,6 +18,8 @@ binning functionality. The sensor drivers belong to two distinct classes, freely configurable and register list-based drivers, depending on how the driver configures this functionality. +Also see :ref:`media_subdev_config_model_common_raw_sensor`. + Freely configurable camera sensor drivers ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -105,6 +107,8 @@ values programmed by the register sequences. The default values of these controls shall be 0 (disabled). Especially these controls shall not be inverted, independently of the sensor's mounting rotation. +.. _media_using_camera_sensor_drivers_embedded_data: + Embedded data -------------
Documentation/userspace-api/media/v4l/common-raw-sensor.dia new file +441 -0
441 lines not on this page · raw message
Documentation/userspace-api/media/v4l/common-raw-sensor.svg new file +134 -0
134 lines not on this page · raw message
Documentation/userspace-api/media/v4l/dev-subdev.rst +2 -0
@@ -838,3 +838,5 @@ stream while it may be possible to enable and disable the embedded data stream. The embedded data format does not need to be configured on the sensor's pads as the format is dictated by the pixel data format in this case. + +.. include:: subdev-config-model.rst
Documentation/userspace-api/media/v4l/subdev-config-model.rst new file +209 -0
209 lines not on this page · raw message
Signature · 1 line
2.39.5
Re: [RFC v5 04/15] media: Documentation: Add subdev configuration models, raw sensor model
Hi Sakari,
Thank you for the patch.
Quoted 40 lines · Sakari Ailus
Sub-device configuration models define what V4L2 API elements are available on a compliant sub-device and how do they behave.
The patch also adds a model for common raw sensors.
Signed-off-by Sakari Ailus <[email protected]>Reviewed-by Tomi Valkeinen <[email protected]>--- .../media/drivers/camera-sensor.rst | 4 + .../media/v4l/common-raw-sensor.dia | 441 ++++++++++++++++++ .../media/v4l/common-raw-sensor.svg | 134 ++++++ .../userspace-api/media/v4l/dev-subdev.rst | 2 + .../media/v4l/subdev-config-model.rst | 209 +++++++++ 5 files changed, 790 insertions(+) create mode 100644 Documentation/userspace-api/media/v4l/common-raw-sensor.dia create mode 100644 Documentation/userspace-api/media/v4l/common-raw-sensor.svg create mode 100644 Documentation/userspace-api/media/v4l/subdev-config-model.rstDocumentation/userspace-api/media/drivers/camera-sensor.rst +4
-0@@ -18,6 +18,8 @@ binning functionality. The sensor drivers belong to two distinct classes, freely configurable and register list-based drivers, depending on how the driver configures this functionality. +Also see :ref:`media_subdev_config_model_common_raw_sensor`. + Freely configurable camera sensor drivers ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~@@ -105,6 +107,8 @@ values programmed by the register sequences. The default values of these controls shall be 0 (disabled). Especially these controls shall not be inverted, independently of the sensor's mounting rotation. +.. _media_using_camera_sensor_drivers_embedded_data: + Embedded data -------------
<snip>
Quoted 104 lines
Documentation/userspace-api/media/v4l/dev-subdev.rst +2
-0@@ -838,3 +838,5 @@ stream while it may be possible to enable and disable the embedded data stream. The embedded data format does not need to be configured on the sensor's pads as the format is dictated by the pixel data format in this case. + +.. include:: subdev-config-model.rstDocumentation/userspace-api/media/v4l/subdev-config-model.rst new file +88
-0@@ -0,0 +1,209 @@ +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later + +.. _media_subdev_config_model: + +Sub-device configuration models +=============================== + +A sub-device configuration model specifies in detail what the user space can +expect from a sub-device in terms of V4L2 sub-device interface support, +semantics included. + +A sub-device may implement more than one configuration model at the same +time. The implemented configuration models can be obtained from the sub-device's +``V4L2_CID_CONFIG_MODEL`` control. + +.. _media_subdev_config_model_common_raw_sensor: + +Common raw camera sensor model +------------------------------ + +The common raw camera sensor model defines a set of enumeration and +configuration interfaces (formats, selections etc.) that cover the vast majority +of functionality of raw camera sensors. Not all of the interfaces are +necessarily offered by all drivers. + +A sub-device complies with the common raw sensor model if the +``V4L2_CONFIG_MODEL_COMMON_RAW`` bit is set in the ``V4L2_CID_CONFIG_MODEL`` +control of the sub-device. + +The common raw camera sensor model is aligned with +:ref:`media_using_camera_sensor_drivers`. Please refer to that regarding aspects +not specified here. + +Each camera sensor implementing the common raw sensor model exposes a single +V4L2 sub-device. The sub-device contains a single source pad (0) and two or more +internal pads: an image data internal pad (1) and optionally an embedded data +pad (2). Additionally, further internal pads may be supported for other +features, in which case they are documented separately for the given device. + +This is shown in :ref:`media_subdev_config_model_common_raw_sensor_subdev`. + +.. _media_subdev_config_model_common_raw_sensor_subdev: + +.. kernel-figure:: common-raw-sensor.svg + :alt: common-raw-sensor.svg + :align: center + + **Common raw sensor sub-device** + +Routes +^^^^^^ + +A sub-device conforming to common raw camera sensor model implements the +following routes. + +.. flat-table:: Routes + :header-rows: 1 + + * - Sink pad/stream + - Source pad/stream + - Static (X/M(aybe)/-) + - Mandatory (X/-) + - Synopsis + * - 1/0 + - 0/0 + - X + - X + - Image data + * - 2/0 + - 0/1 + - M + - \- + - Embedded data + +Support for the embedded data stream is optional. Drivers supporting the +embedded data stream may allow disabling and enabling the route when the +streaming is disabled. + +Sensor pixel array size, cropping and binning +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The sensor's pixel array is divided into one or more areas. The areas around the +edge of the pixel array, usually one or more sides, may contain optical black +pixels, dummy pixels and other non-image pixels. The entire pixel array size is +conveyed by the format on (pad, stream) pair 1/0. + +A rectangle within the pixel array contains the visible pixels. Capturing the +non-visible pixels outside the visible pixel area may be supported by the
Documentation/userspace-api/media/v4l/subdev-config-model.rst line 98
Quoted 10 lines
Documentation/userspace-api/media/v4l/subdev-config-model.rst new file +10
-0+sensor. The visible pixel area corresponds to the ``V4L2_SEL_TGT_CROP_DEFAULT`` +selection target on (pad, stream) pair 1/0. + +Sensors can perform multiple operations that affect the output image size. First +of these is the analogue crop. Analogue crop limits the area of the pixel array +which the sensor will read, affecting sensor timing as well. The granularity of +the analogue crop configuration varies greatly across sensors: some sensors +support only a few different analogue crop configurations whereas others may +support anything divisible by a given number of pixels. The analogue crop +configuration corresponds the ``V4L2_SEL_TGT_CROP`` selection target on (pad,
s/ corresponds the/ corresponds to the
Quoted 19 lines
Documentation/userspace-api/media/v4l/subdev-config-model.rst new file +19
-0+stream) pair 1/0. The default analogue crop rectangle corresponds to the visible +pixel area. + +In the next step, binning is performed on the image data read from camera +sensor's pixel array, as determined by the analogue crop configuration. Enabling +binning will effectively result in an image smaller than the original by given +binning factors horizontally and vertically. Typical values are 1/2 and 1/3 but +others may well be supported by the hardware as well. + +Sub-sampling follows binning. Sub-sampling, like binning, reduces the size of +the image by including only a subset of samples read from the sensor's pixel +matrix, typically every n'th pixel horizontally and vertically, taking the +sensor's colour pattern into account. Sub-sampling is generally configurable +separately horizontally and vertically. + +Binning and sub-sampling are configured using the ``V4L2_SEL_TGT_COMPOSE`` +rectangle, relative to the analogue crop rectangle, on (pad, stream) pair +1/0. The driver implementation determines how to configure binning and +sub-sampling to achieve the desired size.
Documentation/userspace-api/media/v4l/subdev-config-model.rst line 127
Quoted 10 lines
Documentation/userspace-api/media/v4l/subdev-config-model.rst new file +10
-0+ +The digital crop operation takes place after binning and sub-sampling. It is +configured by setting the ``V4L2_SEL_TGT_CROP`` rectangle on (pad, stream) pair +0/0. The resulting image size is further output by the sensor. + +The sensor's output mbus code is configured by setting the format on the (pad, +stream) pair 0/0. When setting the format, always use the same width and height +as for the digital crop setting. + +Drivers may only support some of even none of these configurations, in which
s/some of/some or
Quoted 83 lines
Documentation/userspace-api/media/v4l/subdev-config-model.rst new file +82
-1+case they do not expose the corresponding selection rectangles. If any selection +targets are omitted, the further selection rectangle or format is instead +related to the previous implemented selection rectangle. For instance, if the +sensor supports binning but not analogue crop, then the binning configuration +(``V4L2_SEL_TGT_COMPOSE`` selection target) is done in relation to the visible +pixel area (``V4L2_SEL_TGT_CROP_DEFAULT`` selection target). + +Also refer to :ref:`Selection targets <v4l2-selection-targets-table>`. + +.. flat-table:: Selection targets on pads + :header-rows: 1 + + * - Pad/Stream + - Selection target/format + - Mandatory (X/-) + - Modifiable (X/-) + - Synopsis + * - 1/0 + - Format + - X + - \- + - Image data format. The width and the height fields indicates the full + size of the pixel array, including non-visible pixels. The media bus + code of this format reflects the native pixel depth of the sensor. + * - 1/0 + - ``V4L2_SEL_TGT_CROP_DEFAULT`` + - X + - \ + - The visible pixel area. This rectangle is relative to the format on the + same (pad, stream). + * - 1/0 + - ``V4L2_SEL_TGT_CROP`` + - \- + - X + - Analogue crop. Analogue crop typically has a coarse granularity. This + rectangle is relative to the format on the same (pad, stream). + * - 1/0 + - ``V4L2_SEL_TGT_COMPOSE`` + - \- + - X + - Binning and sub-sampling. This rectangle is relative to the + ``V4L2_SEL_TGT_CROP`` rectangle on the same (pad, stream). The + combination of binning and sub-sampling is configured using this + selection target. + * - 2/0 + - Format + - X + - \- + - Embedded data format. + * - 0/0 + - ``V4L2_SEL_TGT_CROP`` + - \- + - X + - Digital crop. This rectangle is relative to the ``V4L2_SEL_TGT_COMPOSE`` + rectangle on (pad, stream) pair 1/0. + * - 0/0 + - Format + - X + - X + - Image data source format. Always assign the width and height fields of + the format to the same values than for the ``V4L2_SEL_TGT_CROP`` + rectangle on (pad, stream) pair 0/0. The media bus code reflects the + pixel data output of the sensor. + * - 0/1 + - Format + - X + - \- + - Embedded data source format. + +Embedded data +^^^^^^^^^^^^^ + +The embedded data stream is produced by the sensor when the corresponding route +is enabled. The embedded data route may also be immutable or not exist at all, +in case the sensor (or the driver) does not support it. + +Generally the sensor embedded data width is determined by the width of the image +data whereas the number of lines are constant for the embedded data. The user +space may obtain the size of the embedded data once the image data size on the +source pad has been configured. + +Also see :ref:`media_using_camera_sensor_drivers_embedded_data`. --
Quoted 2 lines
2.39.5
Cheers,
Prabhakar
Re: [RFC v5 04/15] media: Documentation: Add subdev configuration models, raw sensor model
In reply to Sakari AilusHi Sakari,
Regards,
Mirela
Quoted 855 lines · Sakari Ailus
Sub-device configuration models define what V4L2 API elements are available on a compliant sub-device and how do they behave.
The patch also adds a model for common raw sensors.
Signed-off-by Sakari Ailus <[email protected]>Reviewed-by Tomi Valkeinen <[email protected]>--- .../media/drivers/camera-sensor.rst | 4 + .../media/v4l/common-raw-sensor.dia | 441 ++++++++++++++++++ .../media/v4l/common-raw-sensor.svg | 134 ++++++ .../userspace-api/media/v4l/dev-subdev.rst | 2 + .../media/v4l/subdev-config-model.rst | 209 +++++++++ 5 files changed, 790 insertions(+)create mode 100644
Documentation/userspace-api/media/v4l/common-raw-sensor.diacreate mode 100644Documentation/userspace-api/media/v4l/common-raw-sensor.svgcreate mode 100644Documentation/userspace-api/media/v4l/subdev-config-model.rstDocumentation/userspace-api/media/drivers/camera-sensor.rst +4
-0@@ -18,6 +18,8 @@ binning functionality. The sensor drivers belong to two distinct classes, freely configurable and register list-based drivers, depending on how the driver configures this functionality. +Also see :ref:`media_subdev_config_model_common_raw_sensor`. + Freely configurable camera sensor drivers ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~@@ -105,6 +107,8 @@ values programmed by the register sequences. The default values of these controls shall be 0 (disabled). Especially these controls shall not be inverted, independently of the sensor's mounting rotation. +.. _media_using_camera_sensor_drivers_embedded_data: + Embedded data -------------Documentation/userspace-api/media/v4l/common-raw-sensor.dia new file +441
-0@@ -0,0 +1,441 @@ +<?xml version="1.0" encoding="UTF-8"?> +<dia:diagram xmlns:dia="http://www.lysator.liu.se/~alla/dia/"> + <dia:diagramdata> + <dia:attribute name="background"> + <dia:color val="#ffffffff"/> + </dia:attribute> + <dia:attribute name="pagebreak"> + <dia:color val="#000099ff"/> + </dia:attribute> + <dia:attribute name="paper"> + <dia:composite type="paper"> + <dia:attribute name="name"> + <dia:string>#A4#</dia:string> + </dia:attribute> + <dia:attribute name="tmargin"> + <dia:real val="2.8222"/> + </dia:attribute> + <dia:attribute name="bmargin"> + <dia:real val="2.8222"/> + </dia:attribute> + <dia:attribute name="lmargin"> + <dia:real val="2.8222"/> + </dia:attribute> + <dia:attribute name="rmargin"> + <dia:real val="2.8222"/> + </dia:attribute> + <dia:attribute name="is_portrait"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="scaling"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="fitto"> + <dia:boolean val="false"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="grid"> + <dia:composite type="grid"> + <dia:attribute name="dynamic"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="width_x"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="width_y"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="visible_x"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="visible_y"> + <dia:int val="1"/> + </dia:attribute> + <dia:composite type="color"/> + </dia:composite> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#d8e5e5ff"/> + </dia:attribute> + <dia:attribute name="guides"/> + <dia:attribute name="guide_color"> + <dia:color val="#00ff00ff"/> + </dia:attribute> + <dia:attribute name="display"> + <dia:composite type="display"> + <dia:attribute name="antialiased"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="snap-to-grid"> + <dia:boolean val="false"/> + </dia:attribute> + <dia:attribute name="snap-to-guides"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="snap-to-object"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="show-grid"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="show-guides"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="show-connection-points"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:diagramdata> + <dia:layer name="Background" visible="true" connectable="false"/> + <dia:layer name="Background" visible="true" connectable="false"/> + <dia:layer name="Background" visible="true" connectable="true" active="true"> + <dia:object type="Standard - Box" version="0" id="O0"> + <dia:attribute name="obj_pos"> + <dia:point val="16.35,11.5"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="16.3,11.45;25.8,18.45"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="16.35,11.5"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9.4000000000000021"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="6.9000000000000021"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Geometric - Perfect Circle" version="1" id="O1"> + <dia:attribute name="obj_pos"> + <dia:point val="25.5627,14.578"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="25.4627,14.478;26.0191,15.0344"/> + </dia:attribute> + <dia:attribute name="meta"> + <dia:composite type="dict"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="25.5627,14.578"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="0.35638032780853468"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="0.35638032780853468"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.10000000000000001"/> + </dia:attribute> + <dia:attribute name="line_colour"> + <dia:color val="#000000ff"/> + </dia:attribute> + <dia:attribute name="fill_colour"> + <dia:color val="#ffffffff"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="line_style"> + <dia:enum val="0"/> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="flip_horizontal"> + <dia:boolean val="false"/> + </dia:attribute> + <dia:attribute name="flip_vertical"> + <dia:boolean val="false"/> + </dia:attribute> + <dia:attribute name="subscale"> + <dia:real val="1"/> + </dia:attribute> + </dia:object> + <dia:object type="Geometric - Perfect Circle" version="1" id="O2"> + <dia:attribute name="obj_pos"> + <dia:point val="17.7702,13.2656"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="17.6702,13.1656;18.24,13.7354"/> + </dia:attribute> + <dia:attribute name="meta"> + <dia:composite type="dict"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="17.7702,13.2656"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="0.36980024191863681"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="0.36980024191863681"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.10000000000000001"/> + </dia:attribute> + <dia:attribute name="line_colour"> + <dia:color val="#000000ff"/> + </dia:attribute> + <dia:attribute name="fill_colour"> + <dia:color val="#ffffffff"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="line_style"> + <dia:enum val="0"/> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="flip_horizontal"> + <dia:boolean val="false"/> + </dia:attribute> + <dia:attribute name="flip_vertical"> + <dia:boolean val="false"/> + </dia:attribute> + <dia:attribute name="subscale"> + <dia:real val="1"/> + </dia:attribute> + </dia:object> + <dia:object type="Geometric - Perfect Circle" version="1" id="O3"> + <dia:attribute name="obj_pos"> + <dia:point val="17.7464,16.2056"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="17.6464,16.1056;18.2162,16.6754"/> + </dia:attribute> + <dia:attribute name="meta"> + <dia:composite type="dict"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="17.7464,16.2056"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="0.36980024191863681"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="0.36980024191863681"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.10000000000000001"/> + </dia:attribute> + <dia:attribute name="line_colour"> + <dia:color val="#000000ff"/> + </dia:attribute> + <dia:attribute name="fill_colour"> + <dia:color val="#ffffffff"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="line_style"> + <dia:enum val="0"/> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="flip_horizontal"> + <dia:boolean val="false"/> + </dia:attribute> + <dia:attribute name="flip_vertical"> + <dia:boolean val="false"/> + </dia:attribute> + <dia:attribute name="subscale"> + <dia:real val="1"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O4"> + <dia:attribute name="obj_pos"> + <dia:point val="18.1609,16.3413"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="18.1016,14.5712;25.6221,16.4007"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="18.1609,16.3413"/> + <dia:point val="25.5627,14.7562"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_style"> + <dia:enum val="1"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="1"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O3" connection="8"/> + <dia:connection handle="1" to="O1" connection="2"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O5"> + <dia:attribute name="obj_pos"> + <dia:point val="18.14,13.4505"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="18.0821,13.3926;25.6206,14.9674"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="18.14,13.4505"/> + <dia:point val="25.5627,14.7562"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="1"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O2" connection="3"/> + <dia:connection handle="1" to="O1" connection="2"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O6"> + <dia:attribute name="obj_pos"> + <dia:point val="25.919,14.7562"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="25.8686,14.3879;31.0068,15.0497"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="25.919,14.7562"/> + <dia:point val="30.9564,14.7131"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="1"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O1" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O7"> + <dia:attribute name="obj_pos"> + <dia:point val="17.9551,13.2656"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="17.9551,12.5181;22.7051,13.2656"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#image data (1)#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="17.9551,13.1131"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000ff"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O2" connection="1"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O8"> + <dia:attribute name="obj_pos"> + <dia:point val="17.9313,16.5754"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="17.9313,16.5754;24.1238,17.3229"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#embedded data (2)#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="17.9313,17.1704"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000ff"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="0"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O3" connection="0"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O9"> + <dia:attribute name="obj_pos"> + <dia:point val="26.1,14.125"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="26.1,13.53;30.7475,14.2775"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#source pad (0)#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="26.1,14.125"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000ff"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + </dia:object> + </dia:layer> +</dia:diagram>Documentation/userspace-api/media/v4l/common-raw-sensor.svg new file +134
-0@@ -0,0 +1,134 @@ +<?xml version="1.0" encoding="UTF-8"?> +<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="296pt" height="142pt" viewBox="0 0 296 142" version="1.1"> +<defs> +<g> +<symbol overflow="visible" id="glyph0-0"> +<path style="stroke:none;" d="M 0.640625 2.265625 L 0.640625 -9.015625 L 7.03125 -9.015625 L 7.03125 2.265625 Z M 1.359375 1.546875 L 6.328125 1.546875 L 6.328125 -8.296875 L 1.359375 -8.296875 Z M 1.359375 1.546875 "/> +</symbol> +<symbol overflow="visible" id="glyph0-1"> +<path style="stroke:none;" d="M 1.203125 -7 L 2.359375 -7 L 2.359375 0 L 1.203125 0 Z M 1.203125 -9.71875 L 2.359375 -9.71875 L 2.359375 -8.265625 L 1.203125 -8.265625 Z M 1.203125 -9.71875 "/> +</symbol> +<symbol overflow="visible" id="glyph0-2"> +<path style="stroke:none;" d="M 6.65625 -5.65625 C 6.9375 -6.164062 7.273438 -6.546875 7.671875 -6.796875 C 8.078125 -7.046875 8.550781 -7.171875 9.09375 -7.171875 C 9.820312 -7.171875 10.382812 -6.914062 10.78125 -6.40625 C 11.175781 -5.894531 11.375 -5.164062 11.375 -4.21875 L 11.375 0 L 10.21875 0 L 10.21875 -4.1875 C 10.21875 -4.851562 10.097656 -5.347656 9.859375 -5.671875 C 9.628906 -6.003906 9.269531 -6.171875 8.78125 -6.171875 C 8.1875 -6.171875 7.710938 -5.972656 7.359375 -5.578125 C 7.015625 -5.179688 6.84375 -4.640625 6.84375 -3.953125 L 6.84375 0 L 5.6875 0 L 5.6875 -4.1875 C 5.6875 -4.863281 5.566406 -5.363281 5.328125 -5.6875 C 5.097656 -6.007812 4.734375 -6.171875 4.234375 -6.171875 C 3.648438 -6.171875 3.179688 -5.96875 2.828125 -5.5625 C 2.484375 -5.164062 2.3125 -4.628906 2.3125 -3.953125 L 2.3125 0 L 1.15625 0 L 1.15625 -7 L 2.3125 -7 L 2.3125 -5.90625 C 2.582031 -6.332031 2.898438 -6.648438 3.265625 -6.859375 C 3.628906 -7.066406 4.0625 -7.171875 4.5625 -7.171875 C 5.070312 -7.171875 5.503906 -7.039062 5.859375 -6.78125 C 6.222656 -6.519531 6.488281 -6.144531 6.65625 -5.65625 Z M 6.65625 -5.65625 "/> +</symbol> +<symbol overflow="visible" id="glyph0-3"> +<path style="stroke:none;" d="M 4.390625 -3.515625 C 3.460938 -3.515625 2.816406 -3.40625 2.453125 -3.1875 C 2.097656 -2.976562 1.921875 -2.617188 1.921875 -2.109375 C 1.921875 -1.703125 2.050781 -1.378906 2.3125 -1.140625 C 2.582031 -0.898438 2.953125 -0.78125 3.421875 -0.78125 C 4.054688 -0.78125 4.566406 -1.003906 4.953125 -1.453125 C 5.335938 -1.910156 5.53125 -2.515625 5.53125 -3.265625 L 5.53125 -3.515625 Z M 6.671875 -4 L 6.671875 0 L 5.53125 0 L 5.53125 -1.0625 C 5.269531 -0.632812 4.941406 -0.316406 4.546875 -0.109375 C 4.160156 0.0859375 3.679688 0.1875 3.109375 0.1875 C 2.390625 0.1875 1.816406 -0.015625 1.390625 -0.421875 C 0.972656 -0.828125 0.765625 -1.363281 0.765625 -2.03125 C 0.765625 -2.820312 1.023438 -3.414062 1.546875 -3.8125 C 2.078125 -4.21875 2.867188 -4.421875 3.921875 -4.421875 L 5.53125 -4.421875 L 5.53125 -4.53125 C 5.53125 -5.0625 5.351562 -5.46875 5 -5.75 C 4.65625 -6.039062 4.171875 -6.1875 3.546875 -6.1875 C 3.140625 -6.1875 2.75 -6.140625 2.375 -6.046875 C 2 -5.953125 1.632812 -5.8125 1.28125 -5.625 L 1.28125 -6.671875 C 1.695312 -6.835938 2.101562 -6.960938 2.5 -7.046875 C 2.894531 -7.128906 3.28125 -7.171875 3.65625 -7.171875 C 4.675781 -7.171875 5.429688 -6.90625 5.921875 -6.375 C 6.421875 -5.851562 6.671875 -5.0625 6.671875 -4 Z M 6.671875 -4 "/> +</symbol> +<symbol overflow="visible" id="glyph0-4"> +<path style="stroke:none;" d="M 5.8125 -3.578125 C 5.8125 -4.410156 5.640625 -5.054688 5.296875 -5.515625 C 4.953125 -5.972656 4.46875 -6.203125 3.84375 -6.203125 C 3.226562 -6.203125 2.75 -5.972656 2.40625 -5.515625 C 2.0625 -5.054688 1.890625 -4.410156 1.890625 -3.578125 C 1.890625 -2.753906 2.0625 -2.113281 2.40625 -1.65625 C 2.75 -1.195312 3.226562 -0.96875 3.84375 -0.96875 C 4.46875 -0.96875 4.953125 -1.195312 5.296875 -1.65625 C 5.640625 -2.113281 5.8125 -2.753906 5.8125 -3.578125 Z M 6.953125 -0.875 C 6.953125 0.320312 6.6875 1.207031 6.15625 1.78125 C 5.632812 2.363281 4.828125 2.65625 3.734375 2.65625 C 3.328125 2.65625 2.945312 2.625 2.59375 2.5625 C 2.238281 2.507812 1.890625 2.421875 1.546875 2.296875 L 1.546875 1.171875 C 1.890625 1.359375 2.222656 1.492188 2.546875 1.578125 C 2.878906 1.671875 3.21875 1.71875 3.5625 1.71875 C 4.3125 1.71875 4.875 1.519531 5.25 1.125 C 5.625 0.726562 5.8125 0.132812 5.8125 -0.65625 L 5.8125 -1.234375 C 5.570312 -0.816406 5.265625 -0.503906 4.890625 -0.296875 C 4.523438 -0.0976562 4.082031 0 3.5625 0 C 2.707031 0 2.015625 -0.328125 1.484375 -0.984375 C 0.960938 -1.640625 0.703125 -2.503906 0.703125 -3.578125 C 0.703125 -4.660156 0.960938 -5.53125 1.484375 -6.1875 C 2.015625 -6.84375 2.707031 -7.171875 3.5625 -7.171875 C 4.082031 -7.171875 4.523438 -7.066406 4.890625 -6.859375 C 5.265625 -6.648438 5.570312 -6.34375 5.8125 -5.9375 L 5.8125 -7 L 6.953125 -7 Z M 6.953125 -0.875 "/> +</symbol> +<symbol overflow="visible" id="glyph0-5"> +<path style="stroke:none;" d="M 7.1875 -3.78125 L 7.1875 -3.21875 L 1.90625 -3.21875 C 1.957031 -2.425781 2.195312 -1.820312 2.625 -1.40625 C 3.050781 -1 3.644531 -0.796875 4.40625 -0.796875 C 4.84375 -0.796875 5.269531 -0.847656 5.6875 -0.953125 C 6.101562 -1.066406 6.515625 -1.226562 6.921875 -1.4375 L 6.921875 -0.359375 C 6.515625 -0.179688 6.09375 -0.046875 5.65625 0.046875 C 5.21875 0.140625 4.78125 0.1875 4.34375 0.1875 C 3.21875 0.1875 2.328125 -0.132812 1.671875 -0.78125 C 1.023438 -1.4375 0.703125 -2.320312 0.703125 -3.4375 C 0.703125 -4.582031 1.007812 -5.488281 1.625 -6.15625 C 2.25 -6.832031 3.085938 -7.171875 4.140625 -7.171875 C 5.078125 -7.171875 5.816406 -6.863281 6.359375 -6.25 C 6.910156 -5.644531 7.1875 -4.820312 7.1875 -3.78125 Z M 6.046875 -4.125 C 6.035156 -4.75 5.859375 -5.25 5.515625 -5.625 C 5.171875 -6 4.71875 -6.1875 4.15625 -6.1875 C 3.507812 -6.1875 2.992188 -6.003906 2.609375 -5.640625 C 2.222656 -5.285156 2 -4.78125 1.9375 -4.125 Z M 6.046875 -4.125 "/> +</symbol> +<symbol overflow="visible" id="glyph0-6"> +<path style="stroke:none;" d=""/> +</symbol> +<symbol overflow="visible" id="glyph0-7"> +<path style="stroke:none;" d="M 5.8125 -5.9375 L 5.8125 -9.71875 L 6.953125 -9.71875 L 6.953125 0 L 5.8125 0 L 5.8125 -1.046875 C 5.570312 -0.628906 5.265625 -0.316406 4.890625 -0.109375 C 4.523438 0.0859375 4.082031 0.1875 3.5625 0.1875 C 2.71875 0.1875 2.03125 -0.148438 1.5 -0.828125 C 0.96875 -1.503906 0.703125 -2.394531 0.703125 -3.5 C 0.703125 -4.59375 0.96875 -5.476562 1.5 -6.15625 C 2.03125 -6.832031 2.71875 -7.171875 3.5625 -7.171875 C 4.082031 -7.171875 4.523438 -7.066406 4.890625 -6.859375 C 5.265625 -6.660156 5.570312 -6.351562 5.8125 -5.9375 Z M 1.890625 -3.5 C 1.890625 -2.644531 2.0625 -1.976562 2.40625 -1.5 C 2.757812 -1.019531 3.238281 -0.78125 3.84375 -0.78125 C 4.457031 -0.78125 4.9375 -1.019531 5.28125 -1.5 C 5.632812 -1.976562 5.8125 -2.644531 5.8125 -3.5 C 5.8125 -4.34375 5.632812 -5.003906 5.28125 -5.484375 C 4.9375 -5.960938 4.457031 -6.203125 3.84375 -6.203125 C 3.238281 -6.203125 2.757812 -5.960938 2.40625 -5.484375 C 2.0625 -5.003906 1.890625 -4.34375 1.890625 -3.5 Z M 1.890625 -3.5 "/> +</symbol> +<symbol overflow="visible" id="glyph0-8"> +<path style="stroke:none;" d="M 2.34375 -8.984375 L 2.34375 -7 L 4.71875 -7 L 4.71875 -6.109375 L 2.34375 -6.109375 L 2.34375 -2.3125 C 2.34375 -1.738281 2.421875 -1.367188 2.578125 -1.203125 C 2.734375 -1.046875 3.050781 -0.96875 3.53125 -0.96875 L 4.71875 -0.96875 L 4.71875 0 L 3.53125 0 C 2.644531 0 2.03125 -0.164062 1.6875 -0.5 C 1.351562 -0.832031 1.1875 -1.4375 1.1875 -2.3125 L 1.1875 -6.109375 L 0.34375 -6.109375 L 0.34375 -7 L 1.1875 -7 L 1.1875 -8.984375 Z M 2.34375 -8.984375 "/> +</symbol> +<symbol overflow="visible" id="glyph0-9"> +<path style="stroke:none;" d="M 3.96875 -9.703125 C 3.40625 -8.742188 2.988281 -7.796875 2.71875 -6.859375 C 2.445312 -5.929688 2.3125 -4.984375 2.3125 -4.015625 C 2.3125 -3.054688 2.445312 -2.101562 2.71875 -1.15625 C 3 -0.21875 3.414062 0.726562 3.96875 1.6875 L 2.96875 1.6875 C 2.34375 0.707031 1.875 -0.253906 1.5625 -1.203125 C 1.25 -2.148438 1.09375 -3.085938 1.09375 -4.015625 C 1.09375 -4.941406 1.25 -5.875 1.5625 -6.8125 C 1.875 -7.757812 2.34375 -8.722656 2.96875 -9.703125 Z M 3.96875 -9.703125 "/> +</symbol> +<symbol overflow="visible" id="glyph0-10"> +<path style="stroke:none;" d="M 1.59375 -1.0625 L 3.65625 -1.0625 L 3.65625 -8.171875 L 1.40625 -7.734375 L 1.40625 -8.875 L 3.640625 -9.328125 L 4.90625 -9.328125 L 4.90625 -1.0625 L 6.953125 -1.0625 L 6.953125 0 L 1.59375 0 Z M 1.59375 -1.0625 "/> +</symbol> +<symbol overflow="visible" id="glyph0-11"> +<path style="stroke:none;" d="M 1.03125 -9.703125 L 2.03125 -9.703125 C 2.65625 -8.722656 3.117188 -7.757812 3.421875 -6.8125 C 3.734375 -5.875 3.890625 -4.941406 3.890625 -4.015625 C 3.890625 -3.085938 3.734375 -2.148438 3.421875 -1.203125 C 3.117188 -0.253906 2.65625 0.707031 2.03125 1.6875 L 1.03125 1.6875 C 1.582031 0.726562 1.992188 -0.21875 2.265625 -1.15625 C 2.535156 -2.101562 2.671875 -3.054688 2.671875 -4.015625 C 2.671875 -4.984375 2.535156 -5.929688 2.265625 -6.859375 C 1.992188 -7.796875 1.582031 -8.742188 1.03125 -9.703125 Z M 1.03125 -9.703125 "/> +</symbol> +<symbol overflow="visible" id="glyph0-12"> +<path style="stroke:none;" d="M 6.234375 -3.5 C 6.234375 -4.34375 6.054688 -5.003906 5.703125 -5.484375 C 5.359375 -5.960938 4.882812 -6.203125 4.28125 -6.203125 C 3.664062 -6.203125 3.179688 -5.960938 2.828125 -5.484375 C 2.484375 -5.003906 2.3125 -4.34375 2.3125 -3.5 C 2.3125 -2.644531 2.484375 -1.976562 2.828125 -1.5 C 3.179688 -1.019531 3.664062 -0.78125 4.28125 -0.78125 C 4.882812 -0.78125 5.359375 -1.019531 5.703125 -1.5 C 6.054688 -1.976562 6.234375 -2.644531 6.234375 -3.5 Z M 2.3125 -5.9375 C 2.5625 -6.351562 2.867188 -6.660156 3.234375 -6.859375 C 3.597656 -7.066406 4.039062 -7.171875 4.5625 -7.171875 C 5.40625 -7.171875 6.09375 -6.832031 6.625 -6.15625 C 7.15625 -5.476562 7.421875 -4.59375 7.421875 -3.5 C 7.421875 -2.394531 7.15625 -1.503906 6.625 -0.828125 C 6.09375 -0.148438 5.40625 0.1875 4.5625 0.1875 C 4.039062 0.1875 3.597656 0.0859375 3.234375 -0.109375 C 2.867188 -0.316406 2.5625 -0.628906 2.3125 -1.046875 L 2.3125 0 L 1.15625 0 L 1.15625 -9.71875 L 2.3125 -9.71875 Z M 2.3125 -5.9375 "/> +</symbol> +<symbol overflow="visible" id="glyph0-13"> +<path style="stroke:none;" d="M 2.453125 -1.0625 L 6.859375 -1.0625 L 6.859375 0 L 0.9375 0 L 0.9375 -1.0625 C 1.414062 -1.5625 2.066406 -2.226562 2.890625 -3.0625 C 3.722656 -3.894531 4.242188 -4.429688 4.453125 -4.671875 C 4.859375 -5.128906 5.140625 -5.515625 5.296875 -5.828125 C 5.460938 -6.140625 5.546875 -6.445312 5.546875 -6.75 C 5.546875 -7.25 5.367188 -7.65625 5.015625 -7.96875 C 4.671875 -8.28125 4.21875 -8.4375 3.65625 -8.4375 C 3.257812 -8.4375 2.84375 -8.363281 2.40625 -8.21875 C 1.96875 -8.082031 1.5 -7.878906 1 -7.609375 L 1 -8.875 C 1.507812 -9.082031 1.984375 -9.238281 2.421875 -9.34375 C 2.867188 -9.445312 3.273438 -9.5 3.640625 -9.5 C 4.609375 -9.5 5.378906 -9.253906 5.953125 -8.765625 C 6.523438 -8.285156 6.8125 -7.640625 6.8125 -6.828125 C 6.8125 -6.453125 6.738281 -6.09375 6.59375 -5.75 C 6.445312 -5.40625 6.1875 -5 5.8125 -4.53125 C 5.707031 -4.40625 5.375 -4.054688 4.8125 -3.484375 C 4.257812 -2.910156 3.472656 -2.101562 2.453125 -1.0625 Z M 2.453125 -1.0625 "/> +</symbol> +<symbol overflow="visible" id="glyph0-14"> +<path style="stroke:none;" d="M 5.671875 -6.796875 L 5.671875 -5.703125 C 5.347656 -5.867188 5.007812 -5.992188 4.65625 -6.078125 C 4.300781 -6.160156 3.9375 -6.203125 3.5625 -6.203125 C 3 -6.203125 2.570312 -6.113281 2.28125 -5.9375 C 2 -5.769531 1.859375 -5.507812 1.859375 -5.15625 C 1.859375 -4.882812 1.957031 -4.671875 2.15625 -4.515625 C 2.363281 -4.367188 2.773438 -4.226562 3.390625 -4.09375 L 3.78125 -4 C 4.601562 -3.832031 5.1875 -3.585938 5.53125 -3.265625 C 5.875 -2.941406 6.046875 -2.5 6.046875 -1.9375 C 6.046875 -1.28125 5.785156 -0.757812 5.265625 -0.375 C 4.753906 0 4.050781 0.1875 3.15625 0.1875 C 2.78125 0.1875 2.390625 0.148438 1.984375 0.078125 C 1.578125 0.00390625 1.144531 -0.101562 0.6875 -0.25 L 0.6875 -1.4375 C 1.113281 -1.21875 1.53125 -1.050781 1.9375 -0.9375 C 2.351562 -0.832031 2.765625 -0.78125 3.171875 -0.78125 C 3.710938 -0.78125 4.128906 -0.875 4.421875 -1.0625 C 4.710938 -1.25 4.859375 -1.507812 4.859375 -1.84375 C 4.859375 -2.15625 4.753906 -2.394531 4.546875 -2.5625 C 4.335938 -2.726562 3.875 -2.890625 3.15625 -3.046875 L 2.765625 -3.140625 C 2.046875 -3.285156 1.53125 -3.515625 1.21875 -3.828125 C 0.90625 -4.140625 0.75 -4.566406 0.75 -5.109375 C 0.75 -5.765625 0.976562 -6.269531 1.4375 -6.625 C 1.90625 -6.988281 2.570312 -7.171875 3.4375 -7.171875 C 3.851562 -7.171875 4.25 -7.140625 4.625 -7.078125 C 5 -7.015625 5.347656 -6.921875 5.671875 -6.796875 Z M 5.671875 -6.796875 "/> +</symbol> +<symbol overflow="visible" id="glyph0-15"> +<path style="stroke:none;" d="M 3.921875 -6.1875 C 3.304688 -6.1875 2.816406 -5.945312 2.453125 -5.46875 C 2.097656 -4.988281 1.921875 -4.332031 1.921875 -3.5 C 1.921875 -2.65625 2.097656 -1.992188 2.453125 -1.515625 C 2.804688 -1.035156 3.296875 -0.796875 3.921875 -0.796875 C 4.535156 -0.796875 5.019531 -1.035156 5.375 -1.515625 C 5.726562 -2.003906 5.90625 -2.664062 5.90625 -3.5 C 5.90625 -4.320312 5.726562 -4.972656 5.375 -5.453125 C 5.019531 -5.941406 4.535156 -6.1875 3.921875 -6.1875 Z M 3.921875 -7.171875 C 4.921875 -7.171875 5.703125 -6.84375 6.265625 -6.1875 C 6.835938 -5.539062 7.125 -4.644531 7.125 -3.5 C 7.125 -2.351562 6.835938 -1.453125 6.265625 -0.796875 C 5.703125 -0.140625 4.921875 0.1875 3.921875 0.1875 C 2.910156 0.1875 2.117188 -0.140625 1.546875 -0.796875 C 0.984375 -1.453125 0.703125 -2.351562 0.703125 -3.5 C 0.703125 -4.644531 0.984375 -5.539062 1.546875 -6.1875 C 2.117188 -6.84375 2.910156 -7.171875 3.921875 -7.171875 Z M 3.921875 -7.171875 "/> +</symbol> +<symbol overflow="visible" id="glyph0-16"> +<path style="stroke:none;" d="M 1.09375 -2.765625 L 1.09375 -7 L 2.234375 -7 L 2.234375 -2.8125 C 2.234375 -2.144531 2.363281 -1.644531 2.625 -1.3125 C 2.882812 -0.976562 3.269531 -0.8125 3.78125 -0.8125 C 4.40625 -0.8125 4.894531 -1.007812 5.25 -1.40625 C 5.613281 -1.800781 5.796875 -2.34375 5.796875 -3.03125 L 5.796875 -7 L 6.953125 -7 L 6.953125 0 L 5.796875 0 L 5.796875 -1.078125 C 5.515625 -0.648438 5.191406 -0.332031 4.828125 -0.125 C 4.460938 0.0820312 4.035156 0.1875 3.546875 0.1875 C 2.742188 0.1875 2.132812 -0.0625 1.71875 -0.5625 C 1.300781 -1.0625 1.09375 -1.796875 1.09375 -2.765625 Z M 3.984375 -7.171875 Z M 3.984375 -7.171875 "/> +</symbol> +<symbol overflow="visible" id="glyph0-17"> +<path style="stroke:none;" d="M 5.265625 -5.921875 C 5.128906 -5.992188 4.984375 -6.046875 4.828125 -6.078125 C 4.679688 -6.117188 4.519531 -6.140625 4.34375 -6.140625 C 3.6875 -6.140625 3.179688 -5.925781 2.828125 -5.5 C 2.484375 -5.082031 2.3125 -4.476562 2.3125 -3.6875 L 2.3125 0 L 1.15625 0 L 1.15625 -7 L 2.3125 -7 L 2.3125 -5.90625 C 2.5625 -6.332031 2.878906 -6.648438 3.265625 -6.859375 C 3.648438 -7.066406 4.117188 -7.171875 4.671875 -7.171875 C 4.753906 -7.171875 4.84375 -7.164062 4.9375 -7.15625 C 5.03125 -7.144531 5.132812 -7.128906 5.25 -7.109375 Z M 5.265625 -5.921875 "/> +</symbol> +<symbol overflow="visible" id="glyph0-18"> +<path style="stroke:none;" d="M 6.25 -6.734375 L 6.25 -5.65625 C 5.914062 -5.832031 5.585938 -5.960938 5.265625 -6.046875 C 4.941406 -6.140625 4.613281 -6.1875 4.28125 -6.1875 C 3.53125 -6.1875 2.945312 -5.953125 2.53125 -5.484375 C 2.125 -5.015625 1.921875 -4.351562 1.921875 -3.5 C 1.921875 -2.644531 2.125 -1.976562 2.53125 -1.5 C 2.945312 -1.03125 3.53125 -0.796875 4.28125 -0.796875 C 4.613281 -0.796875 4.941406 -0.835938 5.265625 -0.921875 C 5.585938 -1.015625 5.914062 -1.148438 6.25 -1.328125 L 6.25 -0.265625 C 5.925781 -0.117188 5.59375 -0.0078125 5.25 0.0625 C 4.90625 0.144531 4.539062 0.1875 4.15625 0.1875 C 3.09375 0.1875 2.25 -0.144531 1.625 -0.8125 C 1.007812 -1.476562 0.703125 -2.375 0.703125 -3.5 C 0.703125 -4.632812 1.015625 -5.53125 1.640625 -6.1875 C 2.273438 -6.84375 3.132812 -7.171875 4.21875 -7.171875 C 4.570312 -7.171875 4.914062 -7.132812 5.25 -7.0625 C 5.59375 -6.988281 5.925781 -6.878906 6.25 -6.734375 Z M 6.25 -6.734375 "/> +</symbol> +<symbol overflow="visible" id="glyph0-19"> +<path style="stroke:none;" d="M 2.3125 -1.046875 L 2.3125 2.65625 L 1.15625 2.65625 L 1.15625 -7 L 2.3125 -7 L 2.3125 -5.9375 C 2.5625 -6.351562 2.867188 -6.660156 3.234375 -6.859375 C 3.597656 -7.066406 4.039062 -7.171875 4.5625 -7.171875 C 5.40625 -7.171875 6.09375 -6.832031 6.625 -6.15625 C 7.15625 -5.476562 7.421875 -4.59375 7.421875 -3.5 C 7.421875 -2.394531 7.15625 -1.503906 6.625 -0.828125 C 6.09375 -0.148438 5.40625 0.1875 4.5625 0.1875 C 4.039062 0.1875 3.597656 0.0859375 3.234375 -0.109375 C 2.867188 -0.316406 2.5625 -0.628906 2.3125 -1.046875 Z M 6.234375 -3.5 C 6.234375 -4.34375 6.054688 -5.003906 5.703125 -5.484375 C 5.359375 -5.960938 4.882812 -6.203125 4.28125 -6.203125 C 3.664062 -6.203125 3.179688 -5.960938 2.828125 -5.484375 C 2.484375 -5.003906 2.3125 -4.34375 2.3125 -3.5 C 2.3125 -2.644531 2.484375 -1.976562 2.828125 -1.5 C 3.179688 -1.019531 3.664062 -0.78125 4.28125 -0.78125 C 4.882812 -0.78125 5.359375 -1.019531 5.703125 -1.5 C 6.054688 -1.976562 6.234375 -2.644531 6.234375 -3.5 Z M 6.234375 -3.5 "/> +</symbol> +<symbol overflow="visible" id="glyph0-20"> +<path style="stroke:none;" d="M 4.0625 -8.5 C 3.414062 -8.5 2.925781 -8.175781 2.59375 -7.53125 C 2.269531 -6.894531 2.109375 -5.9375 2.109375 -4.65625 C 2.109375 -3.375 2.269531 -2.410156 2.59375 -1.765625 C 2.925781 -1.128906 3.414062 -0.8125 4.0625 -0.8125 C 4.71875 -0.8125 5.207031 -1.128906 5.53125 -1.765625 C 5.863281 -2.410156 6.03125 -3.375 6.03125 -4.65625 C 6.03125 -5.9375 5.863281 -6.894531 5.53125 -7.53125 C 5.207031 -8.175781 4.71875 -8.5 4.0625 -8.5 Z M 4.0625 -9.5 C 5.113281 -9.5 5.914062 -9.082031 6.46875 -8.25 C 7.019531 -7.425781 7.296875 -6.226562 7.296875 -4.65625 C 7.296875 -3.082031 7.019531 -1.878906 6.46875 -1.046875 C 5.914062 -0.222656 5.113281 0.1875 4.0625 0.1875 C 3.019531 0.1875 2.222656 -0.222656 1.671875 -1.046875 C 1.117188 -1.878906 0.84375 -3.082031 0.84375 -4.65625 C 0.84375 -6.226562 1.117188 -7.425781 1.671875 -8.25 C 2.222656 -9.082031 3.019531 -9.5 4.0625 -9.5 Z M 4.0625 -9.5 "/> +</symbol> +</g> +</defs> +<g id="surface11910"> +<rect x="0" y="0" width="296" height="142" style="fill:rgb(100%,100%,100%);fill-opacity:1;stroke:none;"/> +<path style="fill-rule:evenodd;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 16.35 11.5 L 25.75 11.5 L 25.75 18.4 L 16.35 18.4 Z M 16.35 11.5 " transform="matrix(20,0,0,20,-325,-228)"/> +<path style="fill-rule:evenodd;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 25.919141 14.75625 C 25.919141 14.99375 25.562695 14.99375 25.562695 14.75625 C 25.562695 14.518555 25.919141 14.518555 25.919141 14.75625 " transform="matrix(20,0,0,20,-325,-228)"/> +<path style="fill-rule:evenodd;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 18.140039 13.450586 C 18.140039 13.69707 17.770117 13.69707 17.770117 13.450586 C 17.770117 13.203906 18.140039 13.203906 18.140039 13.450586 " transform="matrix(20,0,0,20,-325,-228)"/> +<path style="fill-rule:evenodd;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 18.116211 16.39043 C 18.116211 16.637109 17.746484 16.637109 17.746484 16.39043 C 17.746484 16.143945 18.116211 16.143945 18.116211 16.39043 " transform="matrix(20,0,0,20,-325,-228)"/> +<path style="fill:none;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-dasharray:1,1;stroke-miterlimit:10;" d="M 18.160937 16.341406 L 25.344141 14.80293 " transform="matrix(20,0,0,20,-325,-228)"/> +<path style="fill:none;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 25.016797 15.128711 L 25.45332 14.779687 L 24.912109 14.639844 " transform="matrix(20,0,0,20,-325,-228)"/> +<path style="fill:none;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 18.140039 13.450586 L 25.342383 14.717383 " transform="matrix(20,0,0,20,-325,-228)"/> +<path style="fill:none;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 24.916797 14.896484 L 25.452539 14.736914 L 25.003516 14.403906 " transform="matrix(20,0,0,20,-325,-228)"/> +<path style="fill:none;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 25.918945 14.75625 L 30.732813 14.715039 " transform="matrix(20,0,0,20,-325,-228)"/> +<path style="fill:none;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 30.34668 14.968359 L 30.844531 14.714062 L 30.342383 14.468359 " transform="matrix(20,0,0,20,-325,-228)"/> +<g style="fill:rgb(0%,0%,0%);fill-opacity:1;"> + <use xlink:href="#glyph0-1" x="34.101562" y="34.267687"/> + <use xlink:href="#glyph0-2" x="37.712674" y="34.267687"/> + <use xlink:href="#glyph0-3" x="50.212674" y="34.267687"/> + <use xlink:href="#glyph0-4" x="57.990451" y="34.267687"/> + <use xlink:href="#glyph0-5" x="66.046007" y="34.267687"/> + <use xlink:href="#glyph0-6" x="73.823785" y="34.267687"/> + <use xlink:href="#glyph0-7" x="77.990451" y="34.267687"/> + <use xlink:href="#glyph0-3" x="86.046007" y="34.267687"/> + <use xlink:href="#glyph0-8" x="93.823785" y="34.267687"/> + <use xlink:href="#glyph0-3" x="98.823785" y="34.267687"/> + <use xlink:href="#glyph0-6" x="106.601562" y="34.267687"/> + <use xlink:href="#glyph0-9" x="110.768229" y="34.267687"/> + <use xlink:href="#glyph0-10" x="115.768229" y="34.267687"/> + <use xlink:href="#glyph0-11" x="123.823785" y="34.267687"/> +</g> +<g style="fill:rgb(0%,0%,0%);fill-opacity:1;"> + <use xlink:href="#glyph0-5" x="33.625" y="115.412218"/> + <use xlink:href="#glyph0-2" x="41.402778" y="115.412218"/> + <use xlink:href="#glyph0-12" x="53.902778" y="115.412218"/> + <use xlink:href="#glyph0-5" x="61.958333" y="115.412218"/> + <use xlink:href="#glyph0-7" x="69.736111" y="115.412218"/> + <use xlink:href="#glyph0-7" x="77.791667" y="115.412218"/> + <use xlink:href="#glyph0-5" x="85.847222" y="115.412218"/> + <use xlink:href="#glyph0-7" x="93.625" y="115.412218"/> + <use xlink:href="#glyph0-6" x="101.680556" y="115.412218"/> + <use xlink:href="#glyph0-7" x="105.847222" y="115.412218"/> + <use xlink:href="#glyph0-3" x="113.902778" y="115.412218"/> + <use xlink:href="#glyph0-8" x="121.680556" y="115.412218"/> + <use xlink:href="#glyph0-3" x="126.680556" y="115.412218"/> + <use xlink:href="#glyph0-6" x="134.458333" y="115.412218"/> + <use xlink:href="#glyph0-9" x="138.625" y="115.412218"/> + <use xlink:href="#glyph0-13" x="143.625" y="115.412218"/> + <use xlink:href="#glyph0-11" x="151.680556" y="115.412218"/> +</g> +<g style="fill:rgb(0%,0%,0%);fill-opacity:1;"> + <use xlink:href="#glyph0-14" x="197" y="54.505968"/> + <use xlink:href="#glyph0-15" x="203.666667" y="54.505968"/> + <use xlink:href="#glyph0-16" x="211.444444" y="54.505968"/> + <use xlink:href="#glyph0-17" x="219.5" y="54.505968"/> + <use xlink:href="#glyph0-18" x="224.5" y="54.505968"/> + <use xlink:href="#glyph0-5" x="231.444444" y="54.505968"/> + <use xlink:href="#glyph0-6" x="239.222222" y="54.505968"/> + <use xlink:href="#glyph0-19" x="243.388889" y="54.505968"/> + <use xlink:href="#glyph0-3" x="251.444444" y="54.505968"/> + <use xlink:href="#glyph0-7" x="259.222222" y="54.505968"/> + <use xlink:href="#glyph0-6" x="267.277778" y="54.505968"/> + <use xlink:href="#glyph0-9" x="271.444444" y="54.505968"/> + <use xlink:href="#glyph0-20" x="276.444444" y="54.505968"/> + <use xlink:href="#glyph0-11" x="284.5" y="54.505968"/> +</g> +</g> +</svg>Documentation/userspace-api/media/v4l/dev-subdev.rst +2
-0@@ -838,3 +838,5 @@ stream while it may be possible to enable and disable the embedded data stream. The embedded data format does not need to be configured on the sensor's pads as the format is dictated by the pixel data format in this case. + +.. include:: subdev-config-model.rstDocumentation/userspace-api/media/v4l/subdev-config-model.rst new file +209
-1@@ -0,0 +1,209 @@ +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later + +.. _media_subdev_config_model: + +Sub-device configuration models +=============================== + +A sub-device configuration model specifies in detail what the user space can +expect from a sub-device in terms of V4L2 sub-device interface support, +semantics included. + +A sub-device may implement more than one configuration model at the same +time. The implemented configuration models can be obtained from the sub-device's +``V4L2_CID_CONFIG_MODEL`` control. + +.. _media_subdev_config_model_common_raw_sensor: + +Common raw camera sensor model +------------------------------ + +The common raw camera sensor model defines a set of enumeration and +configuration interfaces (formats, selections etc.) that cover the vast majority +of functionality of raw camera sensors. Not all of the interfaces are +necessarily offered by all drivers. + +A sub-device complies with the common raw sensor model if the +``V4L2_CONFIG_MODEL_COMMON_RAW`` bit is set in the ``V4L2_CID_CONFIG_MODEL`` +control of the sub-device. + +The common raw camera sensor model is aligned with +:ref:`media_using_camera_sensor_drivers`. Please refer to that regarding aspects +not specified here. + +Each camera sensor implementing the common raw sensor model exposes a single +V4L2 sub-device. The sub-device contains a single source pad (0) and two or more +internal pads: an image data internal pad (1) and optionally an embedded data +pad (2). Additionally, further internal pads may be supported for other +features, in which case they are documented separately for the given device. + +This is shown in :ref:`media_subdev_config_model_common_raw_sensor_subdev`. + +.. _media_subdev_config_model_common_raw_sensor_subdev: + +.. kernel-figure:: common-raw-sensor.svg + :alt: common-raw-sensor.svg + :align: center + + **Common raw sensor sub-device** + +Routes +^^^^^^ + +A sub-device conforming to common raw camera sensor model implements the +following routes. + +.. flat-table:: Routes + :header-rows: 1 + + * - Sink pad/stream + - Source pad/stream + - Static (X/M(aybe)/-) + - Mandatory (X/-) + - Synopsis + * - 1/0 + - 0/0 + - X + - X + - Image data + * - 2/0 + - 0/1 + - M + - \- + - Embedded data + +Support for the embedded data stream is optional. Drivers supporting the +embedded data stream may allow disabling and enabling the route when the +streaming is disabled. + +Sensor pixel array size, cropping and binning +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The sensor's pixel array is divided into one or more areas. The areas around the +edge of the pixel array, usually one or more sides, may contain optical black +pixels, dummy pixels and other non-image pixels. The entire pixel array size is +conveyed by the format on (pad, stream) pair 1/0. + +A rectangle within the pixel array contains the visible pixels. Capturing the +non-visible pixels outside the visible pixel area may be supported by the +sensor. The visible pixel area corresponds to the ``V4L2_SEL_TGT_CROP_DEFAULT`` +selection target on (pad, stream) pair 1/0. + +Sensors can perform multiple operations that affect the output image size. First +of these is the analogue crop. Analogue crop limits the area of the pixel array +which the sensor will read, affecting sensor timing as well. The granularity of +the analogue crop configuration varies greatly across sensors: some sensors +support only a few different analogue crop configurations whereas others may +support anything divisible by a given number of pixels. The analogue crop +configuration corresponds the ``V4L2_SEL_TGT_CROP`` selection target on (pad, +stream) pair 1/0. The default analogue crop rectangle corresponds to the visible +pixel area. + +In the next step, binning is performed on the image data read from camera +sensor's pixel array, as determined by the analogue crop configuration. Enabling +binning will effectively result in an image smaller than the original by given +binning factors horizontally and vertically. Typical values are 1/2 and 1/3 but +others may well be supported by the hardware as well. + +Sub-sampling follows binning. Sub-sampling, like binning, reduces the size of +the image by including only a subset of samples read from the sensor's pixel +matrix, typically every n'th pixel horizontally and vertically, taking the +sensor's colour pattern into account. Sub-sampling is generally configurable +separately horizontally and vertically. + +Binning and sub-sampling are configured using the ``V4L2_SEL_TGT_COMPOSE`` +rectangle, relative to the analogue crop rectangle, on (pad, stream) pair +1/0. The driver implementation determines how to configure binning and +sub-sampling to achieve the desired size. + +The digital crop operation takes place after binning and sub-sampling. It is +configured by setting the ``V4L2_SEL_TGT_CROP`` rectangle on (pad, stream) pair +0/0. The resulting image size is further output by the sensor. + +The sensor's output mbus code is configured by setting the format on the (pad, +stream) pair 0/0. When setting the format, always use the same width and height +as for the digital crop setting. + +Drivers may only support some of even none of these configurations, in which +case they do not expose the corresponding selection rectangles. If any selection +targets are omitted, the further selection rectangle or format is instead +related to the previous implemented selection rectangle. For instance, if the +sensor supports binning but not analogue crop, then the binning configuration +(``V4L2_SEL_TGT_COMPOSE`` selection target) is done in relation to the visible +pixel area (``V4L2_SEL_TGT_CROP_DEFAULT`` selection target). + +Also refer to :ref:`Selection targets <v4l2-selection-targets-table>`. + +.. flat-table:: Selection targets on pads + :header-rows: 1 + + * - Pad/Stream + - Selection target/format + - Mandatory (X/-) + - Modifiable (X/-) + - Synopsis + * - 1/0 + - Format + - X + - \- + - Image data format. The width and the height fields indicates the full + size of the pixel array, including non-visible pixels. The media bus + code of this format reflects the native pixel depth of the sensor. + * - 1/0 + - ``V4L2_SEL_TGT_CROP_DEFAULT`` + - X + - \ + - The visible pixel area. This rectangle is relative to the format on the + same (pad, stream). + * - 1/0 + - ``V4L2_SEL_TGT_CROP`` + - \- + - X + - Analogue crop. Analogue crop typically has a coarse granularity. This + rectangle is relative to the format on the same (pad, stream). + * - 1/0 + - ``V4L2_SEL_TGT_COMPOSE`` + - \- + - X + - Binning and sub-sampling. This rectangle is relative to the + ``V4L2_SEL_TGT_CROP`` rectangle on the same (pad, stream). The + combination of binning and sub-sampling is configured using this + selection target. + * - 2/0 + - Format + - X + - \- + - Embedded data format. + * - 0/0 + - ``V4L2_SEL_TGT_CROP`` + - \- + - X + - Digital crop. This rectangle is relative to the ``V4L2_SEL_TGT_COMPOSE`` + rectangle on (pad, stream) pair 1/0. + * - 0/0 + - Format + - X + - X + - Image data source format. Always assign the width and height fields of + the format to the same values than for the ``V4L2_SEL_TGT_CROP`` + rectangle on (pad, stream) pair 0/0. The media bus code reflects the + pixel data output of the sensor. + * - 0/1 + - Format + - X + - \- + - Embedded data source format. + +Embedded data +^^^^^^^^^^^^^ + +The embedded data stream is produced by the sensor when the corresponding route +is enabled. The embedded data route may also be immutable or not exist at all, +in case the sensor (or the driver) does not support it. + +Generally the sensor embedded data width is determined by the width of the image +data whereas the number of lines are constant for the embedded data. The user +space may obtain the size of the embedded data once the image data size on the +source pad has been configured. + +Also see :ref:`media_using_camera_sensor_drivers_embedded_data`. --2.39.5
Re: [RFC v5 04/15] media: Documentation: Add subdev configuration models, raw sensor model
Hi Sakari,
Quoted 50 lines · Mirela Rabulea
Hi Sakari,
Reviewed-by Mirela Rabulea <[email protected]>Regards,
Mirela
Quoted 41 lines · Sakari Ailus
Sub-device configuration models define what V4L2 API elements are available on a compliant sub-device and how do they behave.
The patch also adds a model for common raw sensors.
Signed-off-by Sakari Ailus <[email protected]>Reviewed-by Tomi Valkeinen <[email protected]>---
.../media/drivers/camera-sensor.rst | 4 + .../media/v4l/common-raw-sensor.dia | 441 ++++++++++++++++++ .../media/v4l/common-raw-sensor.svg | 134 ++++++ .../userspace-api/media/v4l/dev-subdev.rst | 2 + .../media/v4l/subdev-config-model.rst | 209 +++++++++ 5 files changed, 790 insertions(+)create mode 100644
Documentation/userspace-api/media/v4l/common-raw-sensor.diacreate mode 100644Documentation/userspace-api/media/v4l/common-raw-sensor.svgcreate mode 100644Documentation/userspace-api/media/v4l/subdev-config-model.rst...
Documentation/userspace-api/media/v4l/subdev-config-model.rst new file +6
-0+ +Common raw camera sensor model +------------------------------ + +The common raw camera sensor model defines a set of enumeration and +configuration interfaces (formats, selections etc.) that cover thevast majority
Documentation/userspace-api/media/v4l/subdev-config-model.rst new file +5
-0+of functionality of raw camera sensors. Not all of the interfaces are +necessarily offered by all drivers. + +A sub-device complies with the common raw sensor model if the +``V4L2_CONFIG_MODEL_COMMON_RAW`` bit is set in the``V4L2_CID_CONFIG_MODEL``
Documentation/userspace-api/media/v4l/subdev-config-model.rst new file +4
-0+control of the sub-device. + +The common raw camera sensor model is aligned with +:ref:`media_using_camera_sensor_drivers`. Please refer to thatregarding aspects
Documentation/userspace-api/media/v4l/subdev-config-model.rst new file +3
-0+not specified here. + +Each camera sensor implementing the common raw sensor model exposes
Quoted 8 lines
Quoted 8 lines
a single
+V4L2 sub-device. The sub-device contains a single source pad (0) and two or more
+internal pads: an image data internal pad (1) and optionally an embedded data(quoted diff) +2
-0+pad (2). Additionally, further internal pads may be supported for other +features, in which case they are documented separately for the givendevice.
One more question here, to be sure I understand correctly. So, in order to be compliant with the common raw camera sensor model, in case a raw sensor will need more than 2 internal pads, the expectation is to keep image on pad 0, edata on pad 1, and add more needed pads? Or is it acceptable that a driver completely changes the pad numbering according to it's needs, as long as it is documented?
The case I am targeting is os08a20 hdr sensor, with: pad1-image long exposure, pad2-edata, pad3-image short exposure. So far, so good, I guess.
I don't know if there will be such a case for a sensor driver which does not support embedded data, but will need to support other internal pads, and in such a case, how should pad numbering look like?
Thanks,
Mirela
Re: [RFC v5 04/15] media: Documentation: Add subdev configuration models, raw sensor model
Hi Mirela,
Quoted 72 lines · Mirela Rabulea
Hi Sakari,
Quoted 53 lines · Mirela Rabulea
Hi Sakari,
Reviewed-by Mirela Rabulea <[email protected]>Regards,
Mirela
Quoted 44 lines · Sakari Ailus
Sub-device configuration models define what V4L2 API elements are available on a compliant sub-device and how do they behave.
The patch also adds a model for common raw sensors.
Signed-off-by Sakari Ailus <[email protected]>Reviewed-by Tomi Valkeinen <[email protected]>---
.../media/drivers/camera-sensor.rst | 4 + .../media/v4l/common-raw-sensor.dia | 441 ++++++++++++++++++ .../media/v4l/common-raw-sensor.svg | 134 ++++++ .../userspace-api/media/v4l/dev-subdev.rst | 2 + .../media/v4l/subdev-config-model.rst | 209 +++++++++ 5 files changed, 790 insertions(+)create mode 100644
Documentation/userspace-api/media/v4l/common-raw-sensor.diacreate mode 100644Documentation/userspace-api/media/v4l/common-raw-sensor.svgcreate mode 100644Documentation/userspace-api/media/v4l/subdev-config-model.rst...
Documentation/userspace-api/media/v4l/subdev-config-model.rst new file +6
-0+ +Common raw camera sensor model +------------------------------ + +The common raw camera sensor model defines a set of enumeration and +configuration interfaces (formats, selections etc.) that cover thevast majority
Documentation/userspace-api/media/v4l/subdev-config-model.rst new file +5
-0+of functionality of raw camera sensors. Not all of the interfaces are +necessarily offered by all drivers. + +A sub-device complies with the common raw sensor model if the +``V4L2_CONFIG_MODEL_COMMON_RAW`` bit is set in the``V4L2_CID_CONFIG_MODEL``
Documentation/userspace-api/media/v4l/subdev-config-model.rst new file +4
-0+control of the sub-device. + +The common raw camera sensor model is aligned with +:ref:`media_using_camera_sensor_drivers`. Please refer to thatregarding aspects
Documentation/userspace-api/media/v4l/subdev-config-model.rst new file +3
-0+not specified here. + +Each camera sensor implementing the common raw sensor model exposesQuoted 8 lines
Quoted 8 lines
a single
+V4L2 sub-device. The sub-device contains a single source pad (0) and two or more
+internal pads: an image data internal pad (1) and optionally an embedded data(quoted diff) +2
-0+pad (2). Additionally, further internal pads may be supported for other +features, in which case they are documented separately for thegiven device.
One more question here, to be sure I understand correctly. So, in order to be compliant with the common raw camera sensor model, in case a raw sensor will need more than 2 internal pads, the expectation is to keep image on pad 0, edata on pad 1, and add more needed pads? Or is it acceptable that a driver completely changes the pad numbering according to it's needs, as long as it is documented?
Quoted 6 lines
The case I am targeting is os08a20 hdr sensor, with: pad1-image long exposure, pad2-edata, pad3-image short exposure. So far, so good, I guess.
I don't know if there will be such a case for a sensor driver which does not support embedded data, but will need to support other internal pads, and in such a case, how should pad numbering look like?
Good question.
I think we could change the common raw sensor model in such a way that the embedded data internal pad is the first one using a metadata mbus code, instead of using a plain number.
Laurent, any thoughts?
Signature · 3 lines
Kind regards, Sakari Ailus
RFC v5 5/15 media: Documentation: Add scaling and post-scaler crop for common raw
In reply to Sakari AilusDocument scaling and post-scaler digital crop operations for the common raw sensor model.
--- .../media/v4l/subdev-config-model.rst | 24 +++++++++++++++---- 1 file changed, 19 insertions(+), 5 deletions(-)
Documentation/userspace-api/media/v4l/subdev-config-model.rst +19 -5
@@ -118,11 +118,18 @@ sub-sampling to achieve the desired size. The digital crop operation takes place after binning and sub-sampling. It is configured by setting the ``V4L2_SEL_TGT_CROP`` rectangle on (pad, stream) pair -0/0. The resulting image size is further output by the sensor. +0/0. + +The scaling operation is performed after the digital crop. It is configured by +setting the ``V4L2_SEL_TGT_COMPOSE`` rectangle on (pad, stream) pair 0/0, +relative to the digital crop. The sensor's output mbus code is configured by setting the format on the (pad, -stream) pair 0/0. When setting the format, always use the same width and height -as for the digital crop setting. +stream) pair 0/0. The width and height fields are used to configure post-scaler +digital crop if supported by the driver, affecting the right and bottom edges of +the frame. If post-scaler digital crop is not supported, the width and height +fields of the format will match the compose rectangle sizes applied on the same +0/0 (pad, stream) pair. Drivers may only support some of even none of these configurations, in which case they do not expose the corresponding selection rectangles. If any selection
@@ -180,12 +187,19 @@ Also refer to :ref:`Selection targets <v4l2-selection-targets-table>`. - X - Digital crop. This rectangle is relative to the ``V4L2_SEL_TGT_COMPOSE`` rectangle on (pad, stream) pair 1/0. + * - 0/0 + - ``V4L2_SEL_TGT_COMPOSE`` + - \- + - X + - Scaling. This rectangle is relative to the ``V4L2_SEL_TGT_CROP`` + rectangle on (pad, stream) pair 0/0. * - 0/0 - Format - X - X - - Image data source format. Always assign the width and height fields of - the format to the same values than for the ``V4L2_SEL_TGT_CROP`` + - Image data source format and post-scaler crop. The width and height + fields of the format, used to configure post-scaler crop on the right + and bottom edges of the image, are related to the ``V4L2_SEL_TGT_COMPOSE`` rectangle on (pad, stream) pair 0/0. The media bus code reflects the pixel data output of the sensor. * - 0/1
Signature · 1 line
2.39.5
Re: [RFC v5 05/15] media: Documentation: Add scaling and post-scaler crop for common raw
Quoted 10 lines · Sakari Ailus
Document scaling and post-scaler digital crop operations for the common raw sensor model.
Signed-off-by Sakari Ailus <[email protected]>Reviewed-by Tomi Valkeinen <[email protected]>--- .../media/v4l/subdev-config-model.rst | 24 +++++++++++++++---- 1 file changed, 19 insertions(+), 5 deletions(-)
Cheers,
Prabhakar
Quoted 51 lines
Documentation/userspace-api/media/v4l/subdev-config-model.rst +19
-6@@ -118,11 +118,18 @@ sub-sampling to achieve the desired size. The digital crop operation takes place after binning and sub-sampling. It is configured by setting the ``V4L2_SEL_TGT_CROP`` rectangle on (pad, stream) pair -0/0. The resulting image size is further output by the sensor. +0/0. + +The scaling operation is performed after the digital crop. It is configured by +setting the ``V4L2_SEL_TGT_COMPOSE`` rectangle on (pad, stream) pair 0/0, +relative to the digital crop. The sensor's output mbus code is configured by setting the format on the (pad, -stream) pair 0/0. When setting the format, always use the same width and height -as for the digital crop setting. +stream) pair 0/0. The width and height fields are used to configure post-scaler +digital crop if supported by the driver, affecting the right and bottom edges of +the frame. If post-scaler digital crop is not supported, the width and height +fields of the format will match the compose rectangle sizes applied on the same +0/0 (pad, stream) pair. Drivers may only support some of even none of these configurations, in which case they do not expose the corresponding selection rectangles. If any selection@@ -180,12 +187,19 @@ Also refer to :ref:`Selection targets <v4l2-selection-targets-table>`. - X - Digital crop. This rectangle is relative to the ``V4L2_SEL_TGT_COMPOSE`` rectangle on (pad, stream) pair 1/0. + * - 0/0 + - ``V4L2_SEL_TGT_COMPOSE`` + - \- + - X + - Scaling. This rectangle is relative to the ``V4L2_SEL_TGT_CROP`` + rectangle on (pad, stream) pair 0/0. * - 0/0 - Format - X - X - - Image data source format. Always assign the width and height fields of - the format to the same values than for the ``V4L2_SEL_TGT_CROP`` + - Image data source format and post-scaler crop. The width and height + fields of the format, used to configure post-scaler crop on the right + and bottom edges of the image, are related to the ``V4L2_SEL_TGT_COMPOSE`` rectangle on (pad, stream) pair 0/0. The media bus code reflects the pixel data output of the sensor. * - 0/1 --2.39.5
RFC v5 6/15 media: uapi: Add V4L2_CID_CONFIG_MODEL control
In reply to Sakari AilusAdd the V4L2_CID_CONFIG_MODEL control for the configuration model.
--- .../userspace-api/media/v4l/ext-ctrls-image-process.rst | 4 ++++ drivers/media/v4l2-core/v4l2-ctrls-defs.c | 5 +++++ include/uapi/linux/v4l2-controls.h | 3 +++ 3 files changed, 12 insertions(+)
Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst +4 -0
@@ -55,3 +55,7 @@ Image Process Control IDs control value divided by e.g. 0x100, meaning that to get no digital gain the control value needs to be 0x100. The no-gain configuration is also typically the default. + +``V4L2_CID_CONFIG_MODEL (bitmask)`` + Which configuration models the sub-device supports. Please see + :ref:`media_subdev_config_model`. This is a read-only control.
drivers/media/v4l2-core/v4l2-ctrls-defs.c +5 -0
@@ -1164,6 +1164,7 @@ const char *v4l2_ctrl_get_name(u32 id)case V4L2_CID_TEST_PATTERN :return "Test Pattern" ;case V4L2_CID_DEINTERLACING_MODE :return "Deinterlacing Mode" ;case V4L2_CID_DIGITAL_GAIN :return "Digital Gain" ; +case V4L2_CID_CONFIG_MODEL :return "Sub-device configuration model" ;/* DV controls */ /* Keep the order of the 'case's the same as in v4l2-controls.h! */
@@ -1481,6 +1482,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,case V4L2_CID_DV_RX_POWER_PRESENT : *type =V4L2_CTRL_TYPE_BITMASK ;break ; +case V4L2_CID_CONFIG_MODEL : + *flags |=V4L2_CTRL_FLAG_READ_ONLY ; + *type =V4L2_CTRL_TYPE_BITMASK ; +break ;case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE :case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT : *type =V4L2_CTRL_TYPE_INTEGER ;
include/uapi/linux/v4l2-controls.h +3 -0
@@ -1225,6 +1225,9 @@ enum v4l2_jpeg_chroma_subsampling {#define V4L2_CID_TEST_PATTERN (V4L2_CID_IMAGE_PROC_CLASS_BASE + 3)#define V4L2_CID_DEINTERLACING_MODE (V4L2_CID_IMAGE_PROC_CLASS_BASE + 4)#define V4L2_CID_DIGITAL_GAIN (V4L2_CID_IMAGE_PROC_CLASS_BASE + 5) +#define V4L2_CID_CONFIG_MODEL (V4L2_CID_IMAGE_PROC_CLASS_BASE + 6) + +#define V4L2_CID_CONFIG_MODEL_COMMON_RAW (1U << 0)/* DV-class control IDs defined by V4L2 */ #define V4L2_CID_DV_CLASS_BASE (V4L2_CTRL_CLASS_DV | 0x900)
Signature · 1 line
2.39.5
Re: [RFC v5 06/15] media: uapi: Add V4L2_CID_CONFIG_MODEL control
Quoted 11 lines · Sakari Ailus
Add the
V4L2_CID_CONFIG_MODELcontrol for the configuration model.Signed-off-by Sakari Ailus <[email protected]>Reviewed-by Laurent Pinchart <[email protected]>--- .../userspace-api/media/v4l/ext-ctrls-image-process.rst | 4 ++++ drivers/media/v4l2-core/v4l2-ctrls-defs.c | 5 +++++ include/uapi/linux/v4l2-controls.h | 3 +++ 3 files changed, 12 insertions(+)
Cheers,
Prabhakar
Quoted 52 lines
Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst +4
-0@@ -55,3 +55,7 @@ Image Process Control IDs control value divided by e.g. 0x100, meaning that to get no digital gain the control value needs to be 0x100. The no-gain configuration is also typically the default. + +``V4L2_CID_CONFIG_MODEL (bitmask)`` + Which configuration models the sub-device supports. Please see + :ref:`media_subdev_config_model`. This is a read-only control.drivers/media/v4l2-core/v4l2-ctrls-defs.c +5
-0@@ -1164,6 +1164,7 @@ const char *v4l2_ctrl_get_name(u32 id)case V4L2_CID_TEST_PATTERN :return "Test Pattern" ;case V4L2_CID_DEINTERLACING_MODE :return "Deinterlacing Mode" ;case V4L2_CID_DIGITAL_GAIN :return "Digital Gain" ; +case V4L2_CID_CONFIG_MODEL :return "Sub-device configuration model" ;/* DV controls */ /* Keep the order of the 'case's the same as in v4l2-controls.h! */ @@ -1481,6 +1482,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,case V4L2_CID_DV_RX_POWER_PRESENT : *type =V4L2_CTRL_TYPE_BITMASK ;break ; +case V4L2_CID_CONFIG_MODEL : + *flags |=V4L2_CTRL_FLAG_READ_ONLY ; + *type =V4L2_CTRL_TYPE_BITMASK ; +break ;case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE :case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT : *type =V4L2_CTRL_TYPE_INTEGER ;include/uapi/linux/v4l2-controls.h +3
-1@@ -1225,6 +1225,9 @@ enum v4l2_jpeg_chroma_subsampling {#define V4L2_CID_TEST_PATTERN (V4L2_CID_IMAGE_PROC_CLASS_BASE + 3)#define V4L2_CID_DEINTERLACING_MODE (V4L2_CID_IMAGE_PROC_CLASS_BASE + 4)#define V4L2_CID_DIGITAL_GAIN (V4L2_CID_IMAGE_PROC_CLASS_BASE + 5) +#define V4L2_CID_CONFIG_MODEL (V4L2_CID_IMAGE_PROC_CLASS_BASE + 6) + +#define V4L2_CID_CONFIG_MODEL_COMMON_RAW (1U << 0)/* DV-class control IDs defined by V4L2 */ #define V4L2_CID_DV_CLASS_BASE (V4L2_CTRL_CLASS_DV | 0x900) --2.39.5
Re: [RFC v5 06/15] media: uapi: Add V4L2_CID_CONFIG_MODEL control
In reply to Sakari AilusHi Sakari,
Quoted 50 lines · Sakari Ailus
Add the
V4L2_CID_CONFIG_MODELcontrol for the configuration model.Signed-off-by Sakari Ailus <[email protected]>Reviewed-by Laurent Pinchart <[email protected]>--- .../userspace-api/media/v4l/ext-ctrls-image-process.rst | 4 ++++ drivers/media/v4l2-core/v4l2-ctrls-defs.c | 5 +++++ include/uapi/linux/v4l2-controls.h | 3 +++ 3 files changed, 12 insertions(+)Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst +4
-0@@ -55,3 +55,7 @@ Image Process Control IDs control value divided by e.g. 0x100, meaning that to get no digital gain the control value needs to be 0x100. The no-gain configuration is also typically the default. + +``V4L2_CID_CONFIG_MODEL (bitmask)`` + Which configuration models the sub-device supports. Please see + :ref:`media_subdev_config_model`. This is a read-only control.drivers/media/v4l2-core/v4l2-ctrls-defs.c +5
-0@@ -1164,6 +1164,7 @@ const char *v4l2_ctrl_get_name(u32 id)case V4L2_CID_TEST_PATTERN :return "Test Pattern" ;case V4L2_CID_DEINTERLACING_MODE :return "Deinterlacing Mode" ;case V4L2_CID_DIGITAL_GAIN :return "Digital Gain" ; +case V4L2_CID_CONFIG_MODEL :return "Sub-device configuration model" ;/* DV controls */ /* Keep the order of the 'case's the same as in v4l2-controls.h! */ @@ -1481,6 +1482,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,case V4L2_CID_DV_RX_POWER_PRESENT : *type =V4L2_CTRL_TYPE_BITMASK ;break ; +case V4L2_CID_CONFIG_MODEL : + *flags |=V4L2_CTRL_FLAG_READ_ONLY ; + *type =V4L2_CTRL_TYPE_BITMASK ; +break ;case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE :case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT : *type =V4L2_CTRL_TYPE_INTEGER ;include/uapi/linux/v4l2-controls.h +0
-0
include/uapi/linux/v4l2-controls.h enum v4l2_jpeg_chroma_subsampling { · line 1230
Quoted 6 lines
include/uapi/linux/v4l2-controls.h +3
-0@@ -1225,6 +1225,9 @@ enum v4l2_jpeg_chroma_subsampling {#define V4L2_CID_TEST_PATTERN (V4L2_CID_IMAGE_PROC_CLASS_BASE + 3)#define V4L2_CID_DEINTERLACING_MODE (V4L2_CID_IMAGE_PROC_CLASS_BASE + 4)#define V4L2_CID_DIGITAL_GAIN (V4L2_CID_IMAGE_PROC_CLASS_BASE + 5) +#define V4L2_CID_CONFIG_MODEL (V4L2_CID_IMAGE_PROC_CLASS_BASE + 6) + +#define V4L2_CID_CONFIG_MODEL_COMMON_RAW (1U << 0)
I think I mentioned this before, but what's the point of this?
You are adding a control describing a configuration model, but it has just a single possible configuration model. I see no description anywhere about when a new model would need to be added, or what userspace is supposed to do with this.
And as long as there is only one model anyway, I don't see the point of this control.
Is the intention that all sensor drivers will set this control? The RFC series isn't clear about this.
The problem I see with this series is that it seems to mix seemingly unrelated changes:
adding COLOUR_PATTERN/BINNING controls doesn't seem to depend on configuration models. Or if they do, I clearly didn't get that.
Regards,
Hans
Quoted 3 lines
include/uapi/linux/v4l2-controls.h +0
-0/* DV-class control IDs defined by V4L2 */ #define V4L2_CID_DV_CLASS_BASE (V4L2_CTRL_CLASS_DV | 0x900)
Re: [RFC v5 06/15] media: uapi: Add V4L2_CID_CONFIG_MODEL control
Hi Hans,
Quoted 53 lines · Hans Verkuil
Hi Sakari,
Quoted 50 lines · Sakari Ailus
Add the
V4L2_CID_CONFIG_MODELcontrol for the configuration model.Signed-off-by Sakari Ailus <[email protected]>Reviewed-by Laurent Pinchart <[email protected]>--- .../userspace-api/media/v4l/ext-ctrls-image-process.rst | 4 ++++ drivers/media/v4l2-core/v4l2-ctrls-defs.c | 5 +++++ include/uapi/linux/v4l2-controls.h | 3 +++ 3 files changed, 12 insertions(+)Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst +4
-0@@ -55,3 +55,7 @@ Image Process Control IDs control value divided by e.g. 0x100, meaning that to get no digital gain the control value needs to be 0x100. The no-gain configuration is also typically the default. + +``V4L2_CID_CONFIG_MODEL (bitmask)`` + Which configuration models the sub-device supports. Please see + :ref:`media_subdev_config_model`. This is a read-only control.drivers/media/v4l2-core/v4l2-ctrls-defs.c +5
-0@@ -1164,6 +1164,7 @@ const char *v4l2_ctrl_get_name(u32 id)case V4L2_CID_TEST_PATTERN :return "Test Pattern" ;case V4L2_CID_DEINTERLACING_MODE :return "Deinterlacing Mode" ;case V4L2_CID_DIGITAL_GAIN :return "Digital Gain" ; +case V4L2_CID_CONFIG_MODEL :return "Sub-device configuration model" ;/* DV controls */ /* Keep the order of the 'case's the same as in v4l2-controls.h! */ @@ -1481,6 +1482,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,case V4L2_CID_DV_RX_POWER_PRESENT : *type =V4L2_CTRL_TYPE_BITMASK ;break ; +case V4L2_CID_CONFIG_MODEL : + *flags |=V4L2_CTRL_FLAG_READ_ONLY ; + *type =V4L2_CTRL_TYPE_BITMASK ; +break ;case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE :case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT : *type =V4L2_CTRL_TYPE_INTEGER ;include/uapi/linux/v4l2-controls.h +0
-0@@ -1225,6 +1225,9 @@ enum v4l2_jpeg_chroma_subsampling {
Quoted 8 lines
include/uapi/linux/v4l2-controls.h enum v4l2_jpeg_chroma_subsampling { · line 1230
Quoted 6 lines
include/uapi/linux/v4l2-controls.h +3
-0#define V4L2_CID_TEST_PATTERN (V4L2_CID_IMAGE_PROC_CLASS_BASE + 3)#define V4L2_CID_DEINTERLACING_MODE (V4L2_CID_IMAGE_PROC_CLASS_BASE + 4)#define V4L2_CID_DIGITAL_GAIN (V4L2_CID_IMAGE_PROC_CLASS_BASE + 5) +#define V4L2_CID_CONFIG_MODEL (V4L2_CID_IMAGE_PROC_CLASS_BASE + 6) + +#define V4L2_CID_CONFIG_MODEL_COMMON_RAW (1U << 0)I think I mentioned this before, but what's the point of this?
I recall Laurent was last to reply to the thread, with a good explanation of the purpose. The message id is <[email protected]> .
Quoted 5 lines
You are adding a control describing a configuration model, but it has just a single possible configuration model. I see no description anywhere about when a new model would need to be added, or what userspace is supposed to do with this.
At this point I'm not sure how many other configuration models might be needed or when they would be needed.
Quoted 3 lines
And as long as there is only one model anyway, I don't see the point of this control.
I could create a control just for the common raw sensor model but
Quoted 3 lines
Is the intention that all sensor drivers will set this control? The RFC series isn't clear about this.
I'd expect almost all new raw sensor drivers to expose this control with the common raw bit set.
Quoted 4 lines
The problem I see with this series is that it seems to mix seemingly unrelated changes: adding COLOUR_PATTERN/BINNING controls doesn't seem to depend on configuration models. Or if they do, I clearly didn't get that.
These are all related to sensor API improvements. There is no direct dependency, no, but I expect drivers implementing the common raw sensor model would also support these controls. I can document this.
Signature · 3 lines
Kind regards, Sakari Ailus
Re: [RFC v5 06/15] media: uapi: Add V4L2_CID_CONFIG_MODEL control
Quoted 65 lines · Sakari Ailus
Hi Hans,
Quoted 53 lines · Hans Verkuil
Hi Sakari,
Quoted 50 lines · Sakari Ailus
Add the
V4L2_CID_CONFIG_MODELcontrol for the configuration model.Signed-off-by Sakari Ailus <[email protected]>Reviewed-by Laurent Pinchart <[email protected]>--- .../userspace-api/media/v4l/ext-ctrls-image-process.rst | 4 ++++ drivers/media/v4l2-core/v4l2-ctrls-defs.c | 5 +++++ include/uapi/linux/v4l2-controls.h | 3 +++ 3 files changed, 12 insertions(+)Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst +4
-0@@ -55,3 +55,7 @@ Image Process Control IDs control value divided by e.g. 0x100, meaning that to get no digital gain the control value needs to be 0x100. The no-gain configuration is also typically the default. + +``V4L2_CID_CONFIG_MODEL (bitmask)`` + Which configuration models the sub-device supports. Please see + :ref:`media_subdev_config_model`. This is a read-only control.drivers/media/v4l2-core/v4l2-ctrls-defs.c +5
-0@@ -1164,6 +1164,7 @@ const char *v4l2_ctrl_get_name(u32 id)case V4L2_CID_TEST_PATTERN :return "Test Pattern" ;case V4L2_CID_DEINTERLACING_MODE :return "Deinterlacing Mode" ;case V4L2_CID_DIGITAL_GAIN :return "Digital Gain" ; +case V4L2_CID_CONFIG_MODEL :return "Sub-device configuration model" ;/* DV controls */ /* Keep the order of the 'case's the same as in v4l2-controls.h! */ @@ -1481,6 +1482,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,case V4L2_CID_DV_RX_POWER_PRESENT : *type =V4L2_CTRL_TYPE_BITMASK ;break ; +case V4L2_CID_CONFIG_MODEL : + *flags |=V4L2_CTRL_FLAG_READ_ONLY ; + *type =V4L2_CTRL_TYPE_BITMASK ; +break ;case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE :case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT : *type =V4L2_CTRL_TYPE_INTEGER ;include/uapi/linux/v4l2-controls.h +0
-0@@ -1225,6 +1225,9 @@ enum v4l2_jpeg_chroma_subsampling {Quoted 8 lines
include/uapi/linux/v4l2-controls.h enum v4l2_jpeg_chroma_subsampling { · line 1230
Quoted 6 lines
include/uapi/linux/v4l2-controls.h +3
-0#define V4L2_CID_TEST_PATTERN (V4L2_CID_IMAGE_PROC_CLASS_BASE + 3)#define V4L2_CID_DEINTERLACING_MODE (V4L2_CID_IMAGE_PROC_CLASS_BASE + 4)#define V4L2_CID_DIGITAL_GAIN (V4L2_CID_IMAGE_PROC_CLASS_BASE + 5) +#define V4L2_CID_CONFIG_MODEL (V4L2_CID_IMAGE_PROC_CLASS_BASE + 6) + +#define V4L2_CID_CONFIG_MODEL_COMMON_RAW (1U << 0)I think I mentioned this before, but what's the point of this?
Quoted 3 lines
I recall Laurent was last to reply to the thread, with a good explanation of the purpose. The message id is <[email protected]> .
Now document that in this patch series. I double checked patch 04/15 again, and there is no mention of that explanation from Laurent. It should be incorporated in that patch.
Open questions:
1) If this control is not available, but it is still a camera sensor, what is userspace supposed to to? I guess all it can do is assume that the driver follows the standard rules, since there is no way to figure out if there are differences. So again, how does this help? In any case, this should be documented as well.
2) Are there compliance tests for this? Because there is no point adding this without having tests for it as well. Otherwise I can 100% guarantee that drivers will set this even if it deviates from what it should do in some way. Relying on code review alone is going to be a very tough job.
Quoted 24 lines
Quoted 6 lines
You are adding a control describing a configuration model, but it has just a single possible configuration model. I see no description anywhere about when a new model would need to be added, or what userspace is supposed to do with this.
At this point I'm not sure how many other configuration models might be needed or when they would be needed.
Quoted 3 lines
And as long as there is only one model anyway, I don't see the point of this control.
I could create a control just for the common raw sensor model but
Quoted 3 lines
Is the intention that all sensor drivers will set this control? The RFC series isn't clear about this.
I'd expect almost all new raw sensor drivers to expose this control with the common raw bit set.
Quoted 1 line
Quoted 7 lines
Quoted 3 lines
The problem I see with this series is that it seems to mix seemingly unrelated changes: adding COLOUR_PATTERN/BINNING controls doesn't seem to depend on configuration models. Or if they do, I clearly didn't get that.
These are all related to sensor API improvements. There is no direct dependency, no, but I expect drivers implementing the common raw sensor model would also support these controls. I can document this.
Just split it up in two separate series. I have no objections to the sensor API improvements, so it is much easier to get that in.
But I think that the 'config_model' part is poorly documented and I am quite skeptical about the whole thing. So that shouldn't block the other changes from this RFC series.
Regards,
Hans
Re: [RFC v5 06/15] media: uapi: Add V4L2_CID_CONFIG_MODEL control
Hi Hans,
Quoted 74 lines · Hans Verkuil
Quoted 65 lines · Sakari Ailus
Hi Hans,
Quoted 53 lines · Hans Verkuil
Hi Sakari,
Quoted 50 lines · Sakari Ailus
Add the
V4L2_CID_CONFIG_MODELcontrol for the configuration model.Signed-off-by Sakari Ailus <[email protected]>Reviewed-by Laurent Pinchart <[email protected]>--- .../userspace-api/media/v4l/ext-ctrls-image-process.rst | 4 ++++ drivers/media/v4l2-core/v4l2-ctrls-defs.c | 5 +++++ include/uapi/linux/v4l2-controls.h | 3 +++ 3 files changed, 12 insertions(+)Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst +4
-0@@ -55,3 +55,7 @@ Image Process Control IDs control value divided by e.g. 0x100, meaning that to get no digital gain the control value needs to be 0x100. The no-gain configuration is also typically the default. + +``V4L2_CID_CONFIG_MODEL (bitmask)`` + Which configuration models the sub-device supports. Please see + :ref:`media_subdev_config_model`. This is a read-only control.drivers/media/v4l2-core/v4l2-ctrls-defs.c +5
-0@@ -1164,6 +1164,7 @@ const char *v4l2_ctrl_get_name(u32 id)case V4L2_CID_TEST_PATTERN :return "Test Pattern" ;case V4L2_CID_DEINTERLACING_MODE :return "Deinterlacing Mode" ;case V4L2_CID_DIGITAL_GAIN :return "Digital Gain" ; +case V4L2_CID_CONFIG_MODEL :return "Sub-device configuration model" ;/* DV controls */ /* Keep the order of the 'case's the same as in v4l2-controls.h! */ @@ -1481,6 +1482,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,case V4L2_CID_DV_RX_POWER_PRESENT : *type =V4L2_CTRL_TYPE_BITMASK ;break ; +case V4L2_CID_CONFIG_MODEL : + *flags |=V4L2_CTRL_FLAG_READ_ONLY ; + *type =V4L2_CTRL_TYPE_BITMASK ; +break ;case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE :case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT : *type =V4L2_CTRL_TYPE_INTEGER ;include/uapi/linux/v4l2-controls.h +0
-0@@ -1225,6 +1225,9 @@ enum v4l2_jpeg_chroma_subsampling {Quoted 8 lines
include/uapi/linux/v4l2-controls.h enum v4l2_jpeg_chroma_subsampling { · line 1230
Quoted 6 lines
include/uapi/linux/v4l2-controls.h +3
-0#define V4L2_CID_TEST_PATTERN (V4L2_CID_IMAGE_PROC_CLASS_BASE + 3)#define V4L2_CID_DEINTERLACING_MODE (V4L2_CID_IMAGE_PROC_CLASS_BASE + 4)#define V4L2_CID_DIGITAL_GAIN (V4L2_CID_IMAGE_PROC_CLASS_BASE + 5) +#define V4L2_CID_CONFIG_MODEL (V4L2_CID_IMAGE_PROC_CLASS_BASE + 6) + +#define V4L2_CID_CONFIG_MODEL_COMMON_RAW (1U << 0)I think I mentioned this before, but what's the point of this?
Quoted 3 lines
I recall Laurent was last to reply to the thread, with a good explanation of the purpose. The message id is <[email protected]> .
Now document that in this patch series. I double checked patch 04/15 again, and there is no mention of that explanation from Laurent. It should be incorporated in that patch.
Quoted 7 lines
Open questions:
1) If this control is not available, but it is still a camera sensor, what is userspace supposed to to? I guess all it can do is assume that the driver follows the standard rules, since there is no way to figure out if there are differences. So again, how does this help? In any case, this should be documented as well.
The user space cannot expect the sub-device UAPI conforms to the common raw sensor model without this control. The purpose of the model is indeed to allow the user space to expect certain things to be supported in the API and if they are supported, they work in a certain way. The alternative is to do this driver by driver and there are quite a few sensor drivers out there, with more to come.
Quoted 5 lines
2) Are there compliance tests for this? Because there is no point adding this without having tests for it as well. Otherwise I can 100% guarantee that drivers will set this even if it deviates from what it should do in some way. Relying on code review alone is going to be a very tough job.
Compliance tests need to be added, I agree.
Quoted 33 lines
Quoted 25 lines
Quoted 7 lines
You are adding a control describing a configuration model, but it has just a single possible configuration model. I see no description anywhere about when a new model would need to be added, or what userspace is supposed to do with this.
At this point I'm not sure how many other configuration models might be needed or when they would be needed.
Quoted 3 lines
And as long as there is only one model anyway, I don't see the point of this control.
I could create a control just for the common raw sensor model but
Quoted 3 lines
Is the intention that all sensor drivers will set this control? The RFC series isn't clear about this.
I'd expect almost all new raw sensor drivers to expose this control with the common raw bit set.
Quoted 1 line
Quoted 7 lines
Quoted 3 lines
The problem I see with this series is that it seems to mix seemingly unrelated changes: adding COLOUR_PATTERN/BINNING controls doesn't seem to depend on configuration models. Or if they do, I clearly didn't get that.
These are all related to sensor API improvements. There is no direct dependency, no, but I expect drivers implementing the common raw sensor model would also support these controls. I can document this.
Quoted 6 lines
Just split it up in two separate series. I have no objections to the sensor API improvements, so it is much easier to get that in.
But I think that the '
config_model' part is poorly documented and I am quite skeptical about the whole thing. So that shouldn't block the other changes from this RFC series.
One option could be indeed to postpone this, and continue working on the user space one driver at a time. My initial thought really was we could require the common raw sensor model to be used with internal pads, otherwise it'll be harder to ensure sensor drivers using them will be using them consistently: compliance tests would require quite a few exceptions for instance (doing the tests only for sensors that are non-CCS for instance).
I wonder what Laurent thinks.
Signature · 3 lines
Regards, Sakari Ailus
RFC v5 7/15 media: uapi: Add V4L2_CID_COLOUR_PATTERN for describing colour patterns
In reply to Sakari AilusAdd V4L2_CID_COLOUR_PATTERN to tell the camera sensor's native colour pattern.
--- .../userspace-api/media/v4l/ext-ctrls-image-source.rst | 10 ++++++++++ drivers/media/v4l2-core/v4l2-ctrls-defs.c | 1 + include/uapi/linux/v4l2-controls.h | 6 ++++++ 3 files changed, 17 insertions(+)
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +10 -0
@@ -92,3 +92,13 @@ Image Source Control IDs representing a gain of exactly 1.0. For example, if this default value is reported as being (say) 128, then a value of 192 would represent a gain of exactly 1.5. + +``V4L2_CID_COLOUR_PATTERN (integer)`` + This control determines the colour components and pixel order in the + sensor's CFA (Colour Filter Array) when used in conjunction with + :ref:`luma-only mbus codes MEDIA_BUS_FMT_Yx_1Xx (where 'x' is the bit depth) + <v4l2-mbus-pixelcode-yuv8>` pixelformats. + + This control may only be used on a V4L2 sub-device. + + This is a read-only control.
drivers/media/v4l2-core/v4l2-ctrls-defs.c +1 -0
@@ -1155,6 +1155,7 @@ const char *v4l2_ctrl_get_name(u32 id)case V4L2_CID_TEST_PATTERN_BLUE :return "Blue Pixel Value" ;case V4L2_CID_TEST_PATTERN_GREENB :return "Green (Blue) Pixel Value" ;case V4L2_CID_NOTIFY_GAINS :return "Notify Gains" ; +case V4L2_CID_COLOUR_PATTERN :return "Colour Pattern" ;/* Image processing controls */ /* Keep the order of the 'case's the same as in v4l2-controls.h! */
include/uapi/linux/v4l2-controls.h +6 -0
@@ -1214,6 +1214,12 @@ enum v4l2_jpeg_chroma_subsampling {#define V4L2_CID_UNIT_CELL_SIZE (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 8)#define V4L2_CID_NOTIFY_GAINS (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 9) +#define V4L2_CID_COLOUR_PATTERN (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 10) +#define V4L2_COLOUR_PATTERN_GRBG 0 +#define V4L2_COLOUR_PATTERN_RGGB 1 +#define V4L2_COLOUR_PATTERN_BGGR 2 +#define V4L2_COLOUR_PATTERN_GBRG 3 +/* Image processing controls */
Signature · 1 line
2.39.5
Re: [RFC v5 07/15] media: uapi: Add V4L2_CID_COLOUR_PATTERN for describing colour patterns
Hi Sakari,
sorry for my late response.
So, this control and the luma-only mbus code is expected to be used for all raw sensors? Or can raw sensors still use BGGR mbus codes without the color pattern control?
I noted this is now read-only, is the intention that the user space will only query the current CFA pattern? In case the user space will want to change the CFA, is it expected to achieve it via V4L2_CID_HFLIP or/and V4L2_CID_VFLIP and/or crop?
What exactly is the V4L2_CID_COLOUR_PATTERN expected to report? Is it the sensor's native CFA pattern, or the current CFA pattern with the current flip & crop setting?
One more suggestion below.
Quoted 45 lines · Sakari Ailus
Add
V4L2_CID_COLOUR_PATTERNto tell the camera sensor's native colour pattern.Signed-off-by Sakari Ailus <[email protected]>--- .../userspace-api/media/v4l/ext-ctrls-image-source.rst | 10 ++++++++++ drivers/media/v4l2-core/v4l2-ctrls-defs.c | 1 + include/uapi/linux/v4l2-controls.h | 6 ++++++ 3 files changed, 17 insertions(+)Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +10
-0@@ -92,3 +92,13 @@ Image Source Control IDs representing a gain of exactly 1.0. For example, if this default value is reported as being (say) 128, then a value of 192 would represent a gain of exactly 1.5. + +``V4L2_CID_COLOUR_PATTERN (integer)`` + This control determines the colour components and pixel order in the + sensor's CFA (Colour Filter Array) when used in conjunction with + :ref:`luma-only mbus codes MEDIA_BUS_FMT_Yx_1Xx (where 'x' is the bit depth) + <v4l2-mbus-pixelcode-yuv8>` pixelformats. + + This control may only be used on a V4L2 sub-device. + + This is a read-only control.drivers/media/v4l2-core/v4l2-ctrls-defs.c +1
-0@@ -1155,6 +1155,7 @@ const char *v4l2_ctrl_get_name(u32 id)case V4L2_CID_TEST_PATTERN_BLUE :return "Blue Pixel Value" ;case V4L2_CID_TEST_PATTERN_GREENB :return "Green (Blue) Pixel Value" ;case V4L2_CID_NOTIFY_GAINS :return "Notify Gains" ; +case V4L2_CID_COLOUR_PATTERN :return "Colour Pattern" ;/* Image processing controls */ /* Keep the order of the 'case's the same as in v4l2-controls.h! */ include/uapi/linux/v4l2-controls.h +0
-0
include/uapi/linux/v4l2-controls.h enum v4l2_jpeg_chroma_subsampling { · line 1222
Quoted 9 lines
include/uapi/linux/v4l2-controls.h +6
-0@@ -1214,6 +1214,12 @@ enum v4l2_jpeg_chroma_subsampling {#define V4L2_CID_UNIT_CELL_SIZE (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 8)#define V4L2_CID_NOTIFY_GAINS (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 9) +#define V4L2_CID_COLOUR_PATTERN (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 10) +#define V4L2_COLOUR_PATTERN_GRBG 0 +#define V4L2_COLOUR_PATTERN_RGGB 1 +#define V4L2_COLOUR_PATTERN_BGGR 2 +#define V4L2_COLOUR_PATTERN_GBRG 3 +
I see no pattern for RGB-Ir sensors.
For Omnivision ox05b1s RGB-Ir sensor for example, it has a 4x4 pattern, we would need something like:
B G R G...
G Ir G Ir...
R G B G...
G Ir G Ir...
V4L2_COLOUR_PATTERN_BGRGGIGIRGBGGIGI
And also 3 other combinations from mirror and flip:
V4L2_COLOUR_PATTERN_GRGBIGIGGBGRIGIG
V4L2_COLOUR_PATTERN_GIGIRGBGGIGIBGRG
V4L2_COLOUR_PATTERN_IGIGGBGRIGIGGRGB
At which point we could have these patterns? Is it possible from the beginning, or should we add them at the time when there is at least a driver that needs them?
Also, another question, on which tree does this patch set apply?
Thanks,
Mirela
Quoted 6 lines
include/uapi/linux/v4l2-controls.h +0
-1/* Image processing controls */ --2.39.5
Re: [RFC v5 07/15] media: uapi: Add V4L2_CID_COLOUR_PATTERN for describing colour patterns
Hi Mirela,
Thank you for the review.
Quoted 7 lines · Mirela Rabulea
Hi Sakari,
sorry for my late response.
So, this control and the luma-only mbus code is expected to be used for all raw sensors? Or can raw sensors still use BGGR mbus codes without the color pattern control?
Once we have these merged, I'd expect new drivers to use common raw sensor model (as long as it's suitable for the device) and the Yxx mbus codes and these controls. I think it's fine to keep using the old pattern specific mbus codes in addition to that.
Quoted 5 lines
I noted this is now read-only, is the intention that the user space will only query the current CFA pattern? In case the user space will want to change the CFA, is it expected to achieve it via V4L2_CID_HFLIP or/and
V4L2_CID_VFLIPand/or crop?
Correct. The CFA pattern isn't directly changeable by the user space so this is similar to existing pattern specific mbus codes.
Quoted 4 lines
What exactly is the
V4L2_CID_COLOUR_PATTERNexpected to report? Is it the sensor's native CFA pattern, or the current CFA pattern with the current flip & crop setting?
The native pattern which isn't affected by flipping or cropping. I'll add it to the documentation here, too.
Quoted 52 lines
One more suggestion below.
Quoted 48 lines · Sakari Ailus
Add
V4L2_CID_COLOUR_PATTERNto tell the camera sensor's native colour pattern.Signed-off-by Sakari Ailus <[email protected]>--- .../userspace-api/media/v4l/ext-ctrls-image-source.rst | 10 ++++++++++ drivers/media/v4l2-core/v4l2-ctrls-defs.c | 1 + include/uapi/linux/v4l2-controls.h | 6 ++++++ 3 files changed, 17 insertions(+)Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +10
-0@@ -92,3 +92,13 @@ Image Source Control IDs representing a gain of exactly 1.0. For example, if this default value is reported as being (say) 128, then a value of 192 would represent a gain of exactly 1.5. + +``V4L2_CID_COLOUR_PATTERN (integer)`` + This control determines the colour components and pixel order in the + sensor's CFA (Colour Filter Array) when used in conjunction with + :ref:`luma-only mbus codes MEDIA_BUS_FMT_Yx_1Xx (where 'x' is the bit depth) + <v4l2-mbus-pixelcode-yuv8>` pixelformats. + + This control may only be used on a V4L2 sub-device. + + This is a read-only control.drivers/media/v4l2-core/v4l2-ctrls-defs.c +1
-0@@ -1155,6 +1155,7 @@ const char *v4l2_ctrl_get_name(u32 id)case V4L2_CID_TEST_PATTERN_BLUE :return "Blue Pixel Value" ;case V4L2_CID_TEST_PATTERN_GREENB :return "Green (Blue) Pixel Value" ;case V4L2_CID_NOTIFY_GAINS :return "Notify Gains" ; +case V4L2_CID_COLOUR_PATTERN :return "Colour Pattern" ;/* Image processing controls */ /* Keep the order of the 'case's the same as in v4l2-controls.h! */ include/uapi/linux/v4l2-controls.h +0
-0@@ -1214,6 +1214,12 @@ enum v4l2_jpeg_chroma_subsampling {#define V4L2_CID_UNIT_CELL_SIZE (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 8)#define V4L2_CID_NOTIFY_GAINS (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 9)
Quoted 8 lines
include/uapi/linux/v4l2-controls.h enum v4l2_jpeg_chroma_subsampling { · line 1222
Quoted 6 lines
include/uapi/linux/v4l2-controls.h +6
-0+#define V4L2_CID_COLOUR_PATTERN (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 10) +#define V4L2_COLOUR_PATTERN_GRBG 0 +#define V4L2_COLOUR_PATTERN_RGGB 1 +#define V4L2_COLOUR_PATTERN_BGGR 2 +#define V4L2_COLOUR_PATTERN_GBRG 3 +I see no pattern for RGB-Ir sensors.
Correct. Could you write a patch for that, please? :-)
I'll add more documentation to the above four patterns, that should be a better starting point than this version.
Quoted 20 lines
For Omnivision ox05b1s RGB-Ir sensor for example, it has a 4x4 pattern, we would need something like:
B G R G...
G Ir G Ir...
R G B G...
G Ir G Ir...
V4L2_COLOUR_PATTERN_BGRGGIGIRGBGGIGIAnd also 3 other combinations from mirror and flip:
V4L2_COLOUR_PATTERN_GRGBIGIGGBGRIGIG
V4L2_COLOUR_PATTERN_GIGIRGBGGIGIBGRG
Quoted 8 lines
V4L2_COLOUR_PATTERN_IGIGGBGRIGIGGRGBAt which point we could have these patterns? Is it possible from the beginning, or should we add them at the time when there is at least a driver that needs them?
Also, another question, on which tree does this patch set apply?
They're all here <URL:https://git.linuxtv.org/sailus/media_tree.git/log/?h=metadata-raw>. It seems I forgot this from the cover page.
Signature · 3 lines
Kind regards, Sakari Ailus
Re: [RFC v5 07/15] media: uapi: Add V4L2_CID_COLOUR_PATTERN for describing colour patterns
In reply to Sakari AilusHi Sakari,
Quoted 15 lines · Sakari Ailus
Add
V4L2_CID_COLOUR_PATTERNto tell the camera sensor's native colour pattern.Signed-off-by Sakari Ailus <[email protected]>--- .../userspace-api/media/v4l/ext-ctrls-image-source.rst | 10 ++++++++++ drivers/media/v4l2-core/v4l2-ctrls-defs.c | 1 + include/uapi/linux/v4l2-controls.h | 6 ++++++ 3 files changed, 17 insertions(+)Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +0
-0
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst Image Source Control IDs · line 96
Quoted 5 lines
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +2
-0@@ -92,3 +92,13 @@ Image Source Control IDs representing a gain of exactly 1.0. For example, if this default value is reported as being (say) 128, then a value of 192 would represent a gain of exactly 1.5. + +``V4L2_CID_COLOUR_PATTERN (integer)``
Please use 'COLOR' instead of 'COLOUR' to be consistent with other color related control names. So do a search-and-replace for this in code and documentation.
Regards,
Hans
Quoted 37 lines
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +8
-0+ This control determines the colour components and pixel order in the + sensor's CFA (Colour Filter Array) when used in conjunction with + :ref:`luma-only mbus codes MEDIA_BUS_FMT_Yx_1Xx (where 'x' is the bit depth) + <v4l2-mbus-pixelcode-yuv8>` pixelformats. + + This control may only be used on a V4L2 sub-device. + + This is a read-only control.drivers/media/v4l2-core/v4l2-ctrls-defs.c +1
-0@@ -1155,6 +1155,7 @@ const char *v4l2_ctrl_get_name(u32 id)case V4L2_CID_TEST_PATTERN_BLUE :return "Blue Pixel Value" ;case V4L2_CID_TEST_PATTERN_GREENB :return "Green (Blue) Pixel Value" ;case V4L2_CID_NOTIFY_GAINS :return "Notify Gains" ; +case V4L2_CID_COLOUR_PATTERN :return "Colour Pattern" ;/* Image processing controls */ /* Keep the order of the 'case's the same as in v4l2-controls.h! */ include/uapi/linux/v4l2-controls.h +6
-0@@ -1214,6 +1214,12 @@ enum v4l2_jpeg_chroma_subsampling {#define V4L2_CID_UNIT_CELL_SIZE (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 8)#define V4L2_CID_NOTIFY_GAINS (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 9) +#define V4L2_CID_COLOUR_PATTERN (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 10) +#define V4L2_COLOUR_PATTERN_GRBG 0 +#define V4L2_COLOUR_PATTERN_RGGB 1 +#define V4L2_COLOUR_PATTERN_BGGR 2 +#define V4L2_COLOUR_PATTERN_GBRG 3 +/* Image processing controls */
Re: [RFC v5 07/15] media: uapi: Add V4L2_CID_COLOUR_PATTERN for describing colour patterns
Hi Hans,
Quoted 13 lines · Hans Verkuil
Hi Sakari,
Quoted 10 lines · Sakari Ailus
Add
V4L2_CID_COLOUR_PATTERNto tell the camera sensor's native colour pattern.Signed-off-by Sakari Ailus <[email protected]>--- .../userspace-api/media/v4l/ext-ctrls-image-source.rst | 10 ++++++++++ drivers/media/v4l2-core/v4l2-ctrls-defs.c | 1 + include/uapi/linux/v4l2-controls.h | 6 ++++++ 3 files changed, 17 insertions(+)
Quoted 13 lines
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst Image Source Control IDs · line 96
Quoted 10 lines
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +2
-0@@ -92,3 +92,13 @@ Image Source Control IDs representing a gain of exactly 1.0. For example, if this default value is reported as being (say) 128, then a value of 192 would represent a gain of exactly 1.5. + +``V4L2_CID_COLOUR_PATTERN (integer)``Please use 'COLOR' instead of 'COLOUR' to be consistent with other color related control names. So do a search-and-replace for this in code and documentation.
Ack, I'll address this in v6.
Signature · 3 lines
Regards, Sakari Ailus
RFC v5 8/15 media: uapi: Correct generic CSI-2 metadata format 4cc
In reply to Sakari AilusRework the pixelformat 4cc for CSI-2 generic metadata. This can be done as no driver uses this yet and the interface is disabled.
--- .../userspace-api/media/v4l/metafmt-generic.rst | 2 +- include/uapi/linux/videodev2.h | 12 ++++++------ 2 files changed, 7 insertions(+), 7 deletions(-)
Documentation/userspace-api/media/v4l/metafmt-generic.rst +1 -1
@@ -1,7 +1,7 @@ .. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later ******************************************************************************************************************************************************************************************************************************************************************************** -V4L2_META_FMT_GENERIC_8 ('MET8'), V4L2_META_FMT_GENERIC_CSI2_10 ('MC1A'), V4L2_META_FMT_GENERIC_CSI2_12 ('MC1C'), V4L2_META_FMT_GENERIC_CSI2_14 ('MC1E'), V4L2_META_FMT_GENERIC_CSI2_16 ('MC1G'), V4L2_META_FMT_GENERIC_CSI2_20 ('MC1K'), V4L2_META_FMT_GENERIC_CSI2_24 ('MC1O') +V4L2_META_FMT_GENERIC_8 ('MET8'), V4L2_META_FMT_GENERIC_CSI2_10 ('MECA'), V4L2_META_FMT_GENERIC_CSI2_12 ('MECC'), V4L2_META_FMT_GENERIC_CSI2_14 ('MECE'), V4L2_META_FMT_GENERIC_CSI2_16 ('MECG'), V4L2_META_FMT_GENERIC_CSI2_20 ('MECK'), V4L2_META_FMT_GENERIC_CSI2_24 ('MECO') ********************************************************************************************************************************************************************************************************************************************************************************
include/uapi/linux/videodev2.h +6 -6
@@ -869,12 +869,12 @@ struct v4l2_pix_format {* adding new ones! */ #define V4L2_META_FMT_GENERIC_8 v4l2_fourcc('M', 'E', 'T', '8')/* Generic 8-bit metadata */ -#define V4L2_META_FMT_GENERIC_CSI2_10 v4l2_fourcc('M', 'C', '1', 'A')/* 10-bit CSI-2 packed 8-bit metadata */ -#define V4L2_META_FMT_GENERIC_CSI2_12 v4l2_fourcc('M', 'C', '1', 'C')/* 12-bit CSI-2 packed 8-bit metadata */ -#define V4L2_META_FMT_GENERIC_CSI2_14 v4l2_fourcc('M', 'C', '1', 'E')/* 14-bit CSI-2 packed 8-bit metadata */ -#define V4L2_META_FMT_GENERIC_CSI2_16 v4l2_fourcc('M', 'C', '1', 'G')/* 16-bit CSI-2 packed 8-bit metadata */ -#define V4L2_META_FMT_GENERIC_CSI2_20 v4l2_fourcc('M', 'C', '1', 'K')/* 20-bit CSI-2 packed 8-bit metadata */ -#define V4L2_META_FMT_GENERIC_CSI2_24 v4l2_fourcc('M', 'C', '1', 'O')/* 24-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_10 v4l2_fourcc('M', 'E', 'C', 'A')/* 10-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_12 v4l2_fourcc('M', 'E', 'C', 'C')/* 12-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_14 v4l2_fourcc('M', 'E', 'C', 'E')/* 14-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_16 v4l2_fourcc('M', 'E', 'C', 'G')/* 16-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_20 v4l2_fourcc('M', 'E', 'C', 'K')/* 20-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_24 v4l2_fourcc('M', 'E', 'C', 'O')/* 24-bit CSI-2 packed 8-bit metadata */ #endif /* priv field value to indicates that subsequent fields are valid. */
Signature · 1 line
2.39.5
RFC v5 9/15 media: uapi: Documentation: Improve column width hints for examples
In reply to Sakari AilusUse less arbitrary widths for the columns in metadata layout examples.
--- Documentation/userspace-api/media/v4l/metafmt-generic.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
Documentation/userspace-api/media/v4l/metafmt-generic.rst +4 -4
@@ -74,7 +74,7 @@ This format is little endian. **Byte Order Of V4L2_META_FMT_GENERIC_CSI2_10.** Each cell is one byte. "M" denotes a byte of metadata and "x" a byte of padding. -.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{.8cm}| +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}| .. flat-table:: Sample 4x2 Metadata Frame :header-rows: 0
@@ -118,7 +118,7 @@ This format is little endian. **Byte Order Of V4L2_META_FMT_GENERIC_CSI2_12.** Each cell is one byte. "M" denotes a byte of metadata and "x" a byte of padding. -.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{.8cm}|p{.8cm}| +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}| .. flat-table:: Sample 4x2 Metadata Frame :header-rows: 0
@@ -159,7 +159,7 @@ This format is little endian. **Byte Order Of V4L2_META_FMT_GENERIC_CSI2_14.** Each cell is one byte. "M" denotes a byte of metadata and "x" a byte of padding. -.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{.8cm}| +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|p{1.8cm}|p{1.8cm}| .. flat-table:: Sample 4x2 Metadata Frame :header-rows: 0
@@ -255,7 +255,7 @@ This format is little endian. **Byte Order Of V4L2_META_FMT_GENERIC_CSI2_20.** Each cell is one byte. "M" denotes a byte of metadata and "x" a byte of padding. -.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}| +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.8cm} .. flat-table:: Sample 4x2 Metadata Frame :header-rows: 0
Signature · 1 line
2.39.5
RFC v5 10/15 media: v4l: uapi: Add a control for colour pattern flipping effect
In reply to Sakari AilusAdd a bitmask control (V4L2_CID_COLOUR_PATTERN_FLIP) to tell whether flipping results in a change in the sensor's colour pattern, separately horizontally and vertically. The information is essential for raw formats when using generic raw mbus codes.
--- .../media/v4l/ext-ctrls-image-source.rst | 8 ++++++++ drivers/media/v4l2-core/v4l2-ctrls-defs.c | 2 ++ include/uapi/linux/v4l2-controls.h | 15 +++++++++++++++ 3 files changed, 25 insertions(+)
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +8 -0
@@ -102,3 +102,11 @@ Image Source Control IDs This control may only be used on a V4L2 sub-device. This is a read-only control. + +``V4L2_CID_COLOUR_PATTERN_FLIP (bitmask)`` + Whether the horizontal or vertical flipping controls (V4L2_CID_HFLIP and + V4L2_CID_VFLIP) have an effect on the pixel order of the output colour + pattern. Macros ``V4L2_COLOUR_PATTERN_FLIP_HORIZONTAL`` and + ``V4L2_COLOUR_PATTERN_FLIP_VERTICAL`` define bitmasks for both bits. If + either horizontal or vertical bit is set, the readout pattern order is that + of the reversed readout.
drivers/media/v4l2-core/v4l2-ctrls-defs.c +2 -0
@@ -1156,6 +1156,7 @@ const char *v4l2_ctrl_get_name(u32 id)case V4L2_CID_TEST_PATTERN_GREENB :return "Green (Blue) Pixel Value" ;case V4L2_CID_NOTIFY_GAINS :return "Notify Gains" ;case V4L2_CID_COLOUR_PATTERN :return "Colour Pattern" ; +case V4L2_CID_COLOUR_PATTERN_FLIP :return "Colour Pattern Flip" ;/* Image processing controls */ /* Keep the order of the 'case's the same as in v4l2-controls.h! */
@@ -1474,6 +1475,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type, *max =0xffff ;break ;case V4L2_CID_FLASH_FAULT : +case V4L2_CID_COLOUR_PATTERN_FLIP :case V4L2_CID_JPEG_ACTIVE_MARKER :case V4L2_CID_3A_LOCK :case V4L2_CID_AUTO_FOCUS_STATUS :
include/uapi/linux/v4l2-controls.h +15 -0
@@ -1215,11 +1215,26 @@ enum v4l2_jpeg_chroma_subsampling {#define V4L2_CID_NOTIFY_GAINS (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 9)#define V4L2_CID_COLOUR_PATTERN (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 10) +/* +* CFA pattern start shall be aligned with the number of patterns for colour +* components. Patterns shall be calculable based on flipping when it affects +* the pattern the following way: +* +* flipped_pattern = native_pattern ^ +* ((hflip ? V4L2_COLOUR_PATTERN_FLIP_HORIZONTAL : 0) | +* (vflip ? V4L2_COLOUR_PATTERN_FLIP_VERTICAL : 0)) +* +* where hflip and vflip are the values of V4L2_CID_HFLIP and V4L2_CID_VFLIP +* controls, respectively. +*/ #define V4L2_COLOUR_PATTERN_GRBG 0#define V4L2_COLOUR_PATTERN_RGGB 1#define V4L2_COLOUR_PATTERN_BGGR 2#define V4L2_COLOUR_PATTERN_GBRG 3 +#define V4L2_CID_COLOUR_PATTERN_FLIP (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 11) +#define V4L2_COLOUR_PATTERN_FLIP_HORIZONTAL (1U << 0) +#define V4L2_COLOUR_PATTERN_FLIP_VERTICAL (1U << 1)/* Image processing controls */
Signature · 1 line
2.39.5
Re: [RFC v5 10/15] media: v4l: uapi: Add a control for colour pattern flipping effect
Hi Sakari,
Quoted 4 lines · Sakari Ailus
Add a bitmask control (
V4L2_CID_COLOUR_PATTERN_FLIP) to tell whether flipping results in a change in the sensor's colour pattern, separately horizontally and vertically. The information is essential for raw formats when using generic raw mbus codes.
Is this control really necessary? Are there cases when V4L2_CID_HFLIP or/and V4L2_CID_VFLIP is enabled and the CFA pattern does not change?
Could a raw sensor driver be capable of reporting the V4L2_CID_COLOUR_PATTERN based on current values of the HFLIP/VFLIP? I'm not sure I understood all the aspects of patch #7 (V4L2_CID_COLOUR_PATTERN).
Regards,
Mirela
Quoted 78 lines
Signed-off-by Sakari Ailus <[email protected]>--- .../media/v4l/ext-ctrls-image-source.rst | 8 ++++++++ drivers/media/v4l2-core/v4l2-ctrls-defs.c | 2 ++ include/uapi/linux/v4l2-controls.h | 15 +++++++++++++++ 3 files changed, 25 insertions(+)Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +8
-0@@ -102,3 +102,11 @@ Image Source Control IDs This control may only be used on a V4L2 sub-device. This is a read-only control. + +``V4L2_CID_COLOUR_PATTERN_FLIP (bitmask)`` + Whether the horizontal or vertical flipping controls (V4L2_CID_HFLIP and + V4L2_CID_VFLIP) have an effect on the pixel order of the output colour + pattern. Macros ``V4L2_COLOUR_PATTERN_FLIP_HORIZONTAL`` and + ``V4L2_COLOUR_PATTERN_FLIP_VERTICAL`` define bitmasks for both bits. If + either horizontal or vertical bit is set, the readout pattern order is that + of the reversed readout.drivers/media/v4l2-core/v4l2-ctrls-defs.c +2
-0@@ -1156,6 +1156,7 @@ const char *v4l2_ctrl_get_name(u32 id)case V4L2_CID_TEST_PATTERN_GREENB :return "Green (Blue) Pixel Value" ;case V4L2_CID_NOTIFY_GAINS :return "Notify Gains" ;case V4L2_CID_COLOUR_PATTERN :return "Colour Pattern" ; +case V4L2_CID_COLOUR_PATTERN_FLIP :return "Colour Pattern Flip" ;/* Image processing controls */ /* Keep the order of the 'case's the same as in v4l2-controls.h! */ @@ -1474,6 +1475,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type, *max =0xffff ;break ;case V4L2_CID_FLASH_FAULT : +case V4L2_CID_COLOUR_PATTERN_FLIP :case V4L2_CID_JPEG_ACTIVE_MARKER :case V4L2_CID_3A_LOCK :case V4L2_CID_AUTO_FOCUS_STATUS :include/uapi/linux/v4l2-controls.h +15
-1@@ -1215,11 +1215,26 @@ enum v4l2_jpeg_chroma_subsampling {#define V4L2_CID_NOTIFY_GAINS (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 9)#define V4L2_CID_COLOUR_PATTERN (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 10) +/* +* CFA pattern start shall be aligned with the number of patterns for colour +* components. Patterns shall be calculable based on flipping when it affects +* the pattern the following way: +* +* flipped_pattern = native_pattern ^ +* ((hflip ? V4L2_COLOUR_PATTERN_FLIP_HORIZONTAL : 0) | +* (vflip ? V4L2_COLOUR_PATTERN_FLIP_VERTICAL : 0)) +* +* where hflip and vflip are the values of V4L2_CID_HFLIP and V4L2_CID_VFLIP +* controls, respectively. +*/ #define V4L2_COLOUR_PATTERN_GRBG 0#define V4L2_COLOUR_PATTERN_RGGB 1#define V4L2_COLOUR_PATTERN_BGGR 2#define V4L2_COLOUR_PATTERN_GBRG 3 +#define V4L2_CID_COLOUR_PATTERN_FLIP (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 11) +#define V4L2_COLOUR_PATTERN_FLIP_HORIZONTAL (1U << 0) +#define V4L2_COLOUR_PATTERN_FLIP_VERTICAL (1U << 1)/* Image processing controls */ --2.39.5
Re: [RFC v5 10/15] media: v4l: uapi: Add a control for colour pattern flipping effect
Hi Mirela,
Quoted 10 lines · Mirela Rabulea
Hi Sakari,
Quoted 4 lines · Sakari Ailus
Add a bitmask control (
V4L2_CID_COLOUR_PATTERN_FLIP) to tell whether flipping results in a change in the sensor's colour pattern, separately horizontally and vertically. The information is essential for raw formats when using generic raw mbus codes.Is this control really necessary? Are there cases when
V4L2_CID_HFLIPor/andV4L2_CID_VFLIPis enabled and the CFA pattern does not change?
Some sensors have some tricks in firmware to avoid changing the pattern (I suppose in reality they crop one line / column) but this may not be controllable by the driver, so this possibility should be taken into account.
Quoted 4 lines
Could a raw sensor driver be capable of reporting the
V4L2_CID_COLOUR_PATTERNbased on current values of the HFLIP/VFLIP? I'm not sure I understood all the aspects of patch #7 (V4L2_CID_COLOUR_PATTERN).
In principle it could. The two were originally made independent also because there was an inconvenient dependency between mbus codes and controls. If the pattern control does not specify the native pattern but an active pattern (whatever has an effect on it), the userspace also must check that configuration to determine the native pattern. My understanding is the preference on that side is the native pattern.
Signature · 3 lines
Regards, Sakari Ailus
Re: [RFC v5 10/15] media: v4l: uapi: Add a control for colour pattern flipping effect
Hi Mirela and Sakari
Quoted 12 lines · Sakari Ailus
Hi Mirela,
Quoted 8 lines · Mirela Rabulea
Hi Sakari,
Quoted 4 lines · Sakari Ailus
Add a bitmask control (
V4L2_CID_COLOUR_PATTERN_FLIP) to tell whether flipping results in a change in the sensor's colour pattern, separately horizontally and vertically. The information is essential for raw formats when using generic raw mbus codes.
Quoted 7 lines
Quoted 2 lines
Is this control really necessary? Are there cases when
V4L2_CID_HFLIPor/andV4L2_CID_VFLIPis enabled and the CFA pattern does not change?Some sensors have some tricks in firmware to avoid changing the pattern (I suppose in reality they crop one line / column) but this may not be controllable by the driver, so this possibility should be taken into account.
To give examples of sensors I've encountered that do this.
Sony IMX327, IMX290, IMX462, IMX415 and probably most of the Starvis range. OnSemi AR0144 and AR0234.
Sony's datasheets do explicitly show the pixel array as being an odd number of pixels in each direction, and that it therefore has the same colour pixel (generally red) in each of the 4 corners. The flip registers will be shifting readout by 1 pixel.
Readout mode restrictions generally say that the width and height must be at least a multiple of 4, and offsets must be even, so there is no way to read out that extra border pixel.
Dave
Quoted 16 lines
Quoted 4 lines
Could a raw sensor driver be capable of reporting the
V4L2_CID_COLOUR_PATTERNbased on current values of the HFLIP/VFLIP? I'm not sure I understood all the aspects of patch #7 (V4L2_CID_COLOUR_PATTERN).In principle it could. The two were originally made independent also because there was an inconvenient dependency between mbus codes and controls. If the pattern control does not specify the native pattern but an active pattern (whatever has an effect on it), the userspace also must check that configuration to determine the native pattern. My understanding is the preference on that side is the native pattern.
--
Regards,Sakari Ailus
RFC v5 11/15 media: Documentation: Document luma-only mbus codes and CFA for cameras
In reply to Sakari AilusDocument the use of luma-only mbus codes for camera sensors and how the V4L2_CID_COLOUR_PATTERN and V4L2_CID_COLOUR_PATTERN_FLIP controls are used to convey the colour filter array pattern on UAPI.
--- .../userspace-api/media/drivers/camera-sensor.rst | 13 +++++++++++++ .../media/v4l/ext-ctrls-image-source.rst | 4 ++++ 2 files changed, 17 insertions(+)
Documentation/userspace-api/media/drivers/camera-sensor.rst +13 -0
@@ -41,6 +41,19 @@ format set on a source pad at the end of the device's internal pipeline. Most sensor drivers are implemented this way. +V4L2_CID_COLOUR_PATTERN, luma-only mbus formats, flipping and cropping +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For raw image data originating from camera sensors, :ref:`luma-only mbus codes +MEDIA_BUS_FMT_Yx_1Xx (where 'x' is the bit depth) <v4l2-mbus-pixelcode-yuv8>` +are used as Colour Filter Array (CFA) agnostic raw formats. The +``V4L2_CID_COLOUR_PATTERN <image-source-control-colour-pattern>`` control in the +same sub-device defines the native colour pattern of the device. Flipping may +further affect the readout pattern as indicated by the +``V4L2_CID_COLOUR_PATTERN_FLIP <image-source-control-colour-pattern-flip>`` +control. Further on, cropping also has an effect on the pattern if cropped +amount is not divisible by the size of the pattern, horizontally and vertically. + Frame interval configuration ----------------------------
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +4 -0
@@ -93,6 +93,8 @@ Image Source Control IDs is reported as being (say) 128, then a value of 192 would represent a gain of exactly 1.5. +.. _image-source-control-colour-pattern: + ``V4L2_CID_COLOUR_PATTERN (integer)`` This control determines the colour components and pixel order in the sensor's CFA (Colour Filter Array) when used in conjunction with
@@ -103,6 +105,8 @@ Image Source Control IDs This is a read-only control. +.. _image-source-control-colour-pattern-flip: + ``V4L2_CID_COLOUR_PATTERN_FLIP (bitmask)`` Whether the horizontal or vertical flipping controls (V4L2_CID_HFLIP and V4L2_CID_VFLIP) have an effect on the pixel order of the output colour
Signature · 1 line
2.39.5
Re: [EXT] [RFC v5 11/15] media: Documentation: Document luma-only mbus codes and CFA for cameras
Hi Sakari,
Quoted 24 lines · Sakari Ailus
Caution: This is an external email. Please take care when clicking links or opening attachments. When in doubt, report the message using the 'Report this email' button
Document the use of luma-only mbus codes for camera sensors and how the
V4L2_CID_COLOUR_PATTERNandV4L2_CID_COLOUR_PATTERN_FLIPcontrols are used to convey the colour filter array pattern on UAPI.Signed-off-by Sakari Ailus <[email protected]>--- .../userspace-api/media/drivers/camera-sensor.rst | 13 +++++++++++++ .../media/v4l/ext-ctrls-image-source.rst | 4 ++++ 2 files changed, 17 insertions(+)Documentation/userspace-api/media/drivers/camera-sensor.rst +3
-0@@ -41,6 +41,19 @@ format set on a source pad at the end of the device's internal pipeline. Most sensor drivers are implemented this way. +V4L2_CID_COLOUR_PATTERN, luma-only mbus formats, flipping and cropping +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +
Documentation/userspace-api/media/drivers/camera-sensor.rst format set on a source pad at the end of the device's internal pipeline. · line 56
Quoted 10 lines
Documentation/userspace-api/media/drivers/camera-sensor.rst +10
-0+For raw image data originating from camera sensors, :ref:`luma-only mbus codes +MEDIA_BUS_FMT_Yx_1Xx (where 'x' is the bit depth) <v4l2-mbus-pixelcode-yuv8>` +are used as Colour Filter Array (CFA) agnostic raw formats. The +``V4L2_CID_COLOUR_PATTERN <image-source-control-colour-pattern>`` control in the +same sub-device defines the native colour pattern of the device. Flipping may +further affect the readout pattern as indicated by the +``V4L2_CID_COLOUR_PATTERN_FLIP <image-source-control-colour-pattern-flip>`` +control. Further on, cropping also has an effect on the pattern if cropped +amount is not divisible by the size of the pattern, horizontally and vertically. +
Ok, so it is clear now, the V4L2_CID_COLOUR_PATTERN stands for the "native colour pattern", as opposed to the "readout pattern". It is good to make the distinction.
Maybe the V4L2_CID_COLOUR_PATTERN control description in patch #7 could also be made more clear with regards to native vs readout pattern?
"This control determines the colour components and pixel order in the sensor's CFA (Colour Filter Array)..." => "This control reports the colour components and native pixel order in the sensor's CFA (Colour Filter Array)..." ?
Thanks,
Mirela
Quoted 28 lines
Documentation/userspace-api/media/drivers/camera-sensor.rst +0
-0Frame interval configuration ----------------------------Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +4
-1@@ -93,6 +93,8 @@ Image Source Control IDs is reported as being (say) 128, then a value of 192 would represent a gain of exactly 1.5. +.. _image-source-control-colour-pattern: + ``V4L2_CID_COLOUR_PATTERN (integer)`` This control determines the colour components and pixel order in the sensor's CFA (Colour Filter Array) when used in conjunction with@@ -103,6 +105,8 @@ Image Source Control IDs This is a read-only control. +.. _image-source-control-colour-pattern-flip: + ``V4L2_CID_COLOUR_PATTERN_FLIP (bitmask)`` Whether the horizontal or vertical flipping controls (V4L2_CID_HFLIP and V4L2_CID_VFLIP) have an effect on the pixel order of the output colour --2.39.5
Re: [EXT] [RFC v5 11/15] media: Documentation: Document luma-only mbus codes and CFA for cameras
Hi Mirela,
Quoted 42 lines · Mirela Rabulea
Hi Sakari,
Quoted 24 lines · Sakari Ailus
Caution: This is an external email. Please take care when clicking links or opening attachments. When in doubt, report the message using the 'Report this email' button
Document the use of luma-only mbus codes for camera sensors and how the
V4L2_CID_COLOUR_PATTERNandV4L2_CID_COLOUR_PATTERN_FLIPcontrols are used to convey the colour filter array pattern on UAPI.Signed-off-by Sakari Ailus <[email protected]>--- .../userspace-api/media/drivers/camera-sensor.rst | 13 +++++++++++++ .../media/v4l/ext-ctrls-image-source.rst | 4 ++++ 2 files changed, 17 insertions(+)Documentation/userspace-api/media/drivers/camera-sensor.rst +3
-0@@ -41,6 +41,19 @@ format set on a source pad at the end of the device's internal pipeline. Most sensor drivers are implemented this way. +V4L2_CID_COLOUR_PATTERN, luma-only mbus formats, flipping and cropping +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Documentation/userspace-api/media/drivers/camera-sensor.rst format set on a source pad at the end of the device's internal pipeline. · line 56
Quoted 10 lines
Documentation/userspace-api/media/drivers/camera-sensor.rst +10
-0+For raw image data originating from camera sensors, :ref:`luma-only mbus codes +MEDIA_BUS_FMT_Yx_1Xx (where 'x' is the bit depth) <v4l2-mbus-pixelcode-yuv8>` +are used as Colour Filter Array (CFA) agnostic raw formats. The +``V4L2_CID_COLOUR_PATTERN <image-source-control-colour-pattern>`` control in the +same sub-device defines the native colour pattern of the device. Flipping may +further affect the readout pattern as indicated by the +``V4L2_CID_COLOUR_PATTERN_FLIP <image-source-control-colour-pattern-flip>`` +control. Further on, cropping also has an effect on the pattern if cropped +amount is not divisible by the size of the pattern, horizontally and vertically. +Ok, so it is clear now, the
V4L2_CID_COLOUR_PATTERNstands for the "native colour pattern", as opposed to the "readout pattern". It is good to make the distinction.
Quoted 7 lines
Maybe the
V4L2_CID_COLOUR_PATTERNcontrol description in patch #7 could also be made more clear with regards to native vs readout pattern?"This control determines the colour components and pixel order in the sensor's CFA (Colour Filter Array)..." => "This control reports the colour components and native pixel order in the sensor's CFA (Colour Filter Array)..." ?
I agree, I'll change this in v6.
Signature · 3 lines
Kind regards, Sakari Ailus
RFC v5 12/15 media: uapi: Documentation: Use luma formats with CFA pattern control
In reply to Sakari AilusDocument that the luma-only, greyscale formats (Y10 etc.) are used also for non-greyscale images with the CFA pattern specified by the V4L2_CID_COLOUR_PATTERN control.
The advantage of this approach is that there is little (or none at all) work needed to be able to capture such images with existing CSI-2 receiver drivers as many already support greyscale formats when compared to new CFA-agnostic pixelformats.
--- Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst | 5 +++++ 1 file changed, 5 insertions(+)
Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst +5 -0
@@ -9,6 +9,11 @@ Luma-Only Formats This family of formats only store the luma component of a Y'CbCr image. They are often referred to as greyscale formats. +These pixel formats are also used on :term:`MC-centric` devices when the CFA +pattern is specified by the :ref:`V4L2_CID_COLOUR_PATTERN +<image-source-control-colour-pattern>` control in the image source, typically a +camera sensor. + .. note:: - In all the tables that follow, bit 7 is the most significant bit in a byte.
Signature · 1 line
2.39.5
Re: [RFC v5 12/15] media: uapi: Documentation: Use luma formats with CFA pattern control
Hi Sakari,
Thanks,
Mirela
Quoted 33 lines · Sakari Ailus
Document that the luma-only, greyscale formats (Y10 etc.) are used also for non-greyscale images with the CFA pattern specified by the
V4L2_CID_COLOUR_PATTERNcontrol.The advantage of this approach is that there is little (or none at all) work needed to be able to capture such images with existing CSI-2 receiver drivers as many already support greyscale formats when compared to new CFA-agnostic pixelformats.
Signed-off-by Sakari Ailus <[email protected]>--- Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst | 5 +++++ 1 file changed, 5 insertions(+)Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst +5
-1@@ -9,6 +9,11 @@ Luma-Only Formats This family of formats only store the luma component of a Y'CbCr image. They are often referred to as greyscale formats. +These pixel formats are also used on :term:`MC-centric` devices when the CFA +pattern is specified by the :ref:`V4L2_CID_COLOUR_PATTERN +<image-source-control-colour-pattern>` control in the image source, typically a +camera sensor. + .. note:: - In all the tables that follow, bit 7 is the most significant bit in a byte. --2.39.5
RFC v5 13/15 media: uapi: Add V4L2_CID_BINNING control for binning configuration
In reply to Sakari AilusAdd V4L2_CID_BINNING control for configuring binning and enumerating a camera sensor's binning capabilities. The control combines horizontal and vertical binning into a single control as the two are generally related.
New drivers should use this control to configure binning.
--- .../media/drivers/camera-sensor.rst | 10 +++++++ .../media/v4l/ext-ctrls-camera.rst | 29 +++++++++++++++++++ drivers/media/v4l2-core/v4l2-ctrls-defs.c | 2 ++ include/uapi/linux/v4l2-controls.h | 1 + 4 files changed, 42 insertions(+)
Documentation/userspace-api/media/drivers/camera-sensor.rst +10 -0
@@ -120,6 +120,16 @@ values programmed by the register sequences. The default values of these controls shall be 0 (disabled). Especially these controls shall not be inverted, independently of the sensor's mounting rotation. +Binning +------- + +Binning has traditionally been configured using :ref:`the compose selection +rectangle <v4l2-selection-targets-table>`. The :ref:`V4L2_CID_BINNING +<v4l2-cid-camera-sensor-binning>` is also available for binning configuration and +users should use it when it's available. Drivers supporting the control shall +also support the compose rectangle, albeit the rectangle may be read-only when +the control is present. + .. _media_using_camera_sensor_drivers_embedded_data: Embedded data
Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst +29 -0
@@ -672,3 +672,32 @@ enum v4l2_scene_mode - As modes differ for each sensor, menu items are not standardized by this control and are left to the programmer. + +.. _v4l2-cid-camera-sensor-binning: + +``V4L2_CID_BINNING_FACTORS (integer menu)`` + + Horizontal and vertical binning factors. Binning combines several + horizontal, vertical or both pixel values into a single pixel. It is a way + to scale an image. Binning typically produces fairly good quality output. + + Determines both horizontal and vertical binning factors for a camera + sensor. The values are encoded in the following way: + +.. flat-table:: + :header-rows: 1 + :stub-columns: 0 + + * - Bits + - Synopsis + * - 48--63 + - Horizontal binning numerator. + * - 32--47 + - Horizontal binning denominator. + * - 16--31 + - Vertical binning numerator. + * - 0--15 + - Vertical binning denominator. + +For instance, a value of ``0x0001000300020003`` indicates binning by 3 +(horizontally) * 3/2 (vertically).
drivers/media/v4l2-core/v4l2-ctrls-defs.c +2 -0
@@ -1087,6 +1087,7 @@ const char *v4l2_ctrl_get_name(u32 id)case V4L2_CID_CAMERA_ORIENTATION :return "Camera Orientation" ;case V4L2_CID_CAMERA_SENSOR_ROTATION :return "Camera Sensor Rotation" ;case V4L2_CID_HDR_SENSOR_MODE :return "HDR Sensor Mode" ; +case V4L2_CID_BINNING_FACTORS :return "Binning Factors" ;/* FM Radio Modulator controls */ /* Keep the order of the 'case's the same as in v4l2-controls.h! */
@@ -1426,6 +1427,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,case V4L2_CID_HDR_SENSOR_MODE : *type =V4L2_CTRL_TYPE_MENU ;break ; +case V4L2_CID_BINNING_FACTORS :case V4L2_CID_LINK_FREQ : *type =V4L2_CTRL_TYPE_INTEGER_MENU ;break ;
include/uapi/linux/v4l2-controls.h +1 -0
@@ -1088,6 +1088,7 @@ enum v4l2_auto_focus_range {#define V4L2_CID_CAMERA_SENSOR_ROTATION (V4L2_CID_CAMERA_CLASS_BASE+35)#define V4L2_CID_HDR_SENSOR_MODE (V4L2_CID_CAMERA_CLASS_BASE+36) +#define V4L2_CID_BINNING_FACTORS (V4L2_CID_CAMERA_CLASS_BASE+37)/* FM Modulator class control IDs */
Signature · 1 line
2.39.5
Re: [RFC v5 13/15] media: uapi: Add V4L2_CID_BINNING control for binning configuration
Quoted 60 lines · Sakari Ailus
Add
V4L2_CID_BINNINGcontrol for configuring binning and enumerating a camera sensor's binning capabilities. The control combines horizontal and vertical binning into a single control as the two are generally related.New drivers should use this control to configure binning.
Signed-off-by Sakari Ailus <[email protected]>Reviewed-by Tomi Valkeinen <[email protected]>--- .../media/drivers/camera-sensor.rst | 10 +++++++ .../media/v4l/ext-ctrls-camera.rst | 29 +++++++++++++++++++ drivers/media/v4l2-core/v4l2-ctrls-defs.c | 2 ++ include/uapi/linux/v4l2-controls.h | 1 + 4 files changed, 42 insertions(+)Documentation/userspace-api/media/drivers/camera-sensor.rst +10
-0@@ -120,6 +120,16 @@ values programmed by the register sequences. The default values of these controls shall be 0 (disabled). Especially these controls shall not be inverted, independently of the sensor's mounting rotation. +Binning +------- + +Binning has traditionally been configured using :ref:`the compose selection +rectangle <v4l2-selection-targets-table>`. The :ref:`V4L2_CID_BINNING +<v4l2-cid-camera-sensor-binning>` is also available for binning configuration and +users should use it when it's available. Drivers supporting the control shall +also support the compose rectangle, albeit the rectangle may be read-only when +the control is present. + .. _media_using_camera_sensor_drivers_embedded_data: Embedded dataDocumentation/userspace-api/media/v4l/ext-ctrls-camera.rst +16
-0@@ -672,3 +672,32 @@ enum v4l2_scene_mode - As modes differ for each sensor, menu items are not standardized by this control and are left to the programmer. + +.. _v4l2-cid-camera-sensor-binning: + +``V4L2_CID_BINNING_FACTORS (integer menu)`` + + Horizontal and vertical binning factors. Binning combines several + horizontal, vertical or both pixel values into a single pixel. It is a way + to scale an image. Binning typically produces fairly good quality output. + + Determines both horizontal and vertical binning factors for a camera + sensor. The values are encoded in the following way: + +.. flat-table:: + :header-rows: 1 + :stub-columns: 0 +
Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst enum v4l2_scene_mode - · line 700
Quoted 10 lines
Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst +10
-0+ * - Bits + - Synopsis + * - 48--63 + - Horizontal binning numerator. + * - 32--47 + - Horizontal binning denominator. + * - 16--31 + - Vertical binning numerator. + * - 0--15 + - Vertical binning denominator.
This suggests that the vertical and horizontal binning factors are not independent settings, is that correct?
How does this control (and the subsampling controls) relate to the ROTATE control? Does it happen before or after the rotate?
Regards,
Hans
Quoted 35 lines
Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst +3
-0+ +For instance, a value of ``0x0001000300020003`` indicates binning by 3 +(horizontally) * 3/2 (vertically).drivers/media/v4l2-core/v4l2-ctrls-defs.c +2
-0@@ -1087,6 +1087,7 @@ const char *v4l2_ctrl_get_name(u32 id)case V4L2_CID_CAMERA_ORIENTATION :return "Camera Orientation" ;case V4L2_CID_CAMERA_SENSOR_ROTATION :return "Camera Sensor Rotation" ;case V4L2_CID_HDR_SENSOR_MODE :return "HDR Sensor Mode" ; +case V4L2_CID_BINNING_FACTORS :return "Binning Factors" ;/* FM Radio Modulator controls */ /* Keep the order of the 'case's the same as in v4l2-controls.h! */ @@ -1426,6 +1427,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,case V4L2_CID_HDR_SENSOR_MODE : *type =V4L2_CTRL_TYPE_MENU ;break ; +case V4L2_CID_BINNING_FACTORS :case V4L2_CID_LINK_FREQ : *type =V4L2_CTRL_TYPE_INTEGER_MENU ;break ;include/uapi/linux/v4l2-controls.h +1
-0@@ -1088,6 +1088,7 @@ enum v4l2_auto_focus_range {#define V4L2_CID_CAMERA_SENSOR_ROTATION (V4L2_CID_CAMERA_CLASS_BASE+35)#define V4L2_CID_HDR_SENSOR_MODE (V4L2_CID_CAMERA_CLASS_BASE+36) +#define V4L2_CID_BINNING_FACTORS (V4L2_CID_CAMERA_CLASS_BASE+37)/* FM Modulator class control IDs */
Re: [RFC v5 13/15] media: uapi: Add V4L2_CID_BINNING control for binning configuration
Hi Hans,
Thank you for the review.
Quoted 66 lines · Hans Verkuil
Quoted 65 lines · Sakari Ailus
Add
V4L2_CID_BINNINGcontrol for configuring binning and enumerating a camera sensor's binning capabilities. The control combines horizontal and vertical binning into a single control as the two are generally related.New drivers should use this control to configure binning.
Signed-off-by Sakari Ailus <[email protected]>Reviewed-by Tomi Valkeinen <[email protected]>--- .../media/drivers/camera-sensor.rst | 10 +++++++ .../media/v4l/ext-ctrls-camera.rst | 29 +++++++++++++++++++ drivers/media/v4l2-core/v4l2-ctrls-defs.c | 2 ++ include/uapi/linux/v4l2-controls.h | 1 + 4 files changed, 42 insertions(+)Documentation/userspace-api/media/drivers/camera-sensor.rst +10
-0@@ -120,6 +120,16 @@ values programmed by the register sequences. The default values of these controls shall be 0 (disabled). Especially these controls shall not be inverted, independently of the sensor's mounting rotation. +Binning +------- + +Binning has traditionally been configured using :ref:`the compose selection +rectangle <v4l2-selection-targets-table>`. The :ref:`V4L2_CID_BINNING +<v4l2-cid-camera-sensor-binning>` is also available for binning configuration and +users should use it when it's available. Drivers supporting the control shall +also support the compose rectangle, albeit the rectangle may be read-only when +the control is present. + .. _media_using_camera_sensor_drivers_embedded_data: Embedded dataDocumentation/userspace-api/media/v4l/ext-ctrls-camera.rst +21
-0@@ -672,3 +672,32 @@ enum v4l2_scene_mode - As modes differ for each sensor, menu items are not standardized by this control and are left to the programmer. + +.. _v4l2-cid-camera-sensor-binning: + +``V4L2_CID_BINNING_FACTORS (integer menu)`` + + Horizontal and vertical binning factors. Binning combines several + horizontal, vertical or both pixel values into a single pixel. It is a way + to scale an image. Binning typically produces fairly good quality output. + + Determines both horizontal and vertical binning factors for a camera + sensor. The values are encoded in the following way: + +.. flat-table:: + :header-rows: 1 + :stub-columns: 0 + + * - Bits + - Synopsis + * - 48--63 + - Horizontal binning numerator. + * - 32--47
Quoted 8 lines
Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst enum v4l2_scene_mode - · line 700
Quoted 5 lines
Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst +5
-0+ - Horizontal binning denominator. + * - 16--31 + - Vertical binning numerator. + * - 0--15 + - Vertical binning denominator.This suggests that the vertical and horizontal binning factors are not independent settings, is that correct?
There are dependencies between the two, yes.
Quoted 3 lines
How does this control (and the subsampling controls) relate to the ROTATE control? Does it happen before or after the rotate?
In general there's no rotation support in (raw) camera sensors, just flipping. And flipping isn't affecting binning.
Signature · 3 lines
Kind regards, Sakari Ailus
RFC v5 14/15 media: uapi: Add controls for sub-sampling configuration
In reply to Sakari AilusSub-sampling is a way to decrease the data rates after the pixel array by systematically discarding some samples, either vertically or horizontally or both. Add two controls for the purpose and document them. The sub-sampling configuration is taken into account in the compose rectangle.
--- .../userspace-api/media/drivers/camera-sensor.rst | 14 ++++++++++++-- .../userspace-api/media/v4l/ext-ctrls-camera.rst | 11 +++++++++++ drivers/media/v4l2-core/v4l2-ctrls-defs.c | 2 ++ include/uapi/linux/v4l2-controls.h | 2 ++ 4 files changed, 27 insertions(+), 2 deletions(-)
Documentation/userspace-api/media/drivers/camera-sensor.rst +12 -2
@@ -120,8 +120,8 @@ values programmed by the register sequences. The default values of these controls shall be 0 (disabled). Especially these controls shall not be inverted, independently of the sensor's mounting rotation. -Binning -------- +Binning and sub-sampling +------------------------ Binning has traditionally been configured using :ref:`the compose selection rectangle <v4l2-selection-targets-table>`. The :ref:`V4L2_CID_BINNING
@@ -130,6 +130,16 @@ users should use it when it's available. Drivers supporting the control shall also support the compose rectangle, albeit the rectangle may be read-only when the control is present. +Sub-sampling is often supported as part of a camera sensor's binning +functionality and performed after the binning operation. Sub-sampling typically +produces quality-wise worse results than binning. Sub-sampling factors are +independent horizontally and vertically and they are controlled using two +controls, :ref:`V4L2_CID_SUBSAMPLING_HORIZONTAL and +V4L2_CID_SUBSAMPLING_VERTICAL <v4l2-cid-camera-sensor-subsampling>`. In +sub-sampling, the image size before sub-sampling is horizontally and vertically +divided by the respective sub-sampling factors. Drivers supporting the control shall +also reflect the sub-sampling configuration in the compose rectangle. + .. _media_using_camera_sensor_drivers_embedded_data: Embedded data
Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst +11 -0
@@ -701,3 +701,14 @@ enum v4l2_scene_mode - For instance, a value of ``0x0001000300020003`` indicates binning by 3 (horizontally) * 3/2 (vertically). + +.. _v4l2-cid-camera-sensor-subsampling: + +``V4L2_CID_SUBSAMPLING_HORIZONTAL`` and ``V4L2_CID_SUBSAMPLING_VERTICAL`` +(integer) + + Horizontal and vertical subsampling factors. + + Sub-sampling is used to downscale an image, horizontally and vertically, by + discarding a part of the image data. Typically sub-sampling produces lower + quality images than binning.
drivers/media/v4l2-core/v4l2-ctrls-defs.c +2 -0
@@ -1088,6 +1088,8 @@ const char *v4l2_ctrl_get_name(u32 id)case V4L2_CID_CAMERA_SENSOR_ROTATION :return "Camera Sensor Rotation" ;case V4L2_CID_HDR_SENSOR_MODE :return "HDR Sensor Mode" ;case V4L2_CID_BINNING_FACTORS :return "Binning Factors" ; +case V4L2_CID_SUBSAMPLING_HORIZONTAL :return "Sub-Sampling Factor, Horizontal" ; +case V4L2_CID_SUBSAMPLING_VERTICAL :return "Sub-Sampling Factor, Vertical" ;/* FM Radio Modulator controls */ /* Keep the order of the 'case's the same as in v4l2-controls.h! */
include/uapi/linux/v4l2-controls.h +2 -0
@@ -1089,6 +1089,8 @@ enum v4l2_auto_focus_range {#define V4L2_CID_HDR_SENSOR_MODE (V4L2_CID_CAMERA_CLASS_BASE+36)#define V4L2_CID_BINNING_FACTORS (V4L2_CID_CAMERA_CLASS_BASE+37) +#define V4L2_CID_SUBSAMPLING_HORIZONTAL (V4L2_CID_CAMERA_CLASS_BASE+38) +#define V4L2_CID_SUBSAMPLING_VERTICAL (V4L2_CID_CAMERA_CLASS_BASE+39)/* FM Modulator class control IDs */
Signature · 1 line
2.39.5
RFC v5 15/15 media: Documentation: Add binning and sub-sampling controls
In reply to Sakari AilusDocument the binning and scaling controls (V4L2_CID_BINNING and V4L2_CID_SUBSAMPLING_{HORIZONTAL,VERTICAL}) in the common raw sensor model.
--- .../media/v4l/subdev-config-model.rst | 20 ++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-)
Documentation/userspace-api/media/v4l/subdev-config-model.rst +15 -5
@@ -113,8 +113,12 @@ separately horizontally and vertically. Binning and sub-sampling are configured using the ``V4L2_SEL_TGT_COMPOSE`` rectangle, relative to the analogue crop rectangle, on (pad, stream) pair -1/0. The driver implementation determines how to configure binning and -sub-sampling to achieve the desired size. +1/0. It depends on the driver which of these operations are being used to +achieve the resulting size. Binning and sub-sampling are also directly +configured using :ref:`V4L2_CID_BINNING_FACTORS +<v4l2-cid-camera-sensor-binning>` and :ref:`V4L2_CID_SUBSAMPLING_HORIZONTAL and +V4L2_CID_SUBSAMPLING_VERTICAL <v4l2-cid-camera-sensor-subsampling>` controls on +drivers that support them. The digital crop operation takes place after binning and sub-sampling. It is configured by setting the ``V4L2_SEL_TGT_CROP`` rectangle on (pad, stream) pair
@@ -173,9 +177,15 @@ Also refer to :ref:`Selection targets <v4l2-selection-targets-table>`. - \- - X - Binning and sub-sampling. This rectangle is relative to the - ``V4L2_SEL_TGT_CROP`` rectangle on the same (pad, stream). The - combination of binning and sub-sampling is configured using this - selection target. + ``V4L2_SEL_TGT_CROP`` rectangle on the same (pad, stream). Binning is + configured using the :ref:`V4L2_CID_BINNING_FACTORS + <v4l2-cid-camera-sensor-binning>` control and sub-sampling is configured + using the :ref:`V4L2_CID_SUBSAMPLING_HORIZONTAL and + V4L2_CID_SUBSAMPLING_VERTICAL <v4l2-cid-camera-sensor-subsampling>` + controls on drivers that support these controls. To configure binning + and sub-sampling on drivers that do not support these controls, the + selection rectangle may be changed directly to configure the combined + effect on the image size. * - 2/0 - Format - X
Signature · 1 line
2.39.5
Quoted 5 lines · Sakari Ailus
Hello everyone,
I've been recently working (with others) on sub-device streams support as well as on internal pads. The two can be used to make sub-device configuration more versatile.
The patches are all available here (including streams + internal pad support) <URL:https://git.linuxtv.org/sailus/media_tree.git/log/?h=metadata-raw>.
Signature · 1 line
Sakari Ailus