Skip to contentSkip to conversation

media: Documentation: Improve PIXEL_RATE control documentation

PATCH In review Sakari Ailus 2 participants 5 messages Apr 14 – Apr 20 Media (V4L/DVB)

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 media: Documentation: Improve PIXEL_RATE control documentation

Document explicitly that the PIXEL_RATE control reflects the actual frequency at which the pixels are read in the pixel array. It is thus orthogonal to analogue binning.

---
 .../userspace-api/media/v4l/ext-ctrls-image-process.rst      | 5 +++++
 1 file changed, 5 insertions(+)
Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst +5 -0
index 6d516f041ca2..8616bcd67270 100644
@@ -41,6 +41,11 @@ Image Process Control IDs
     The configuration of the frame rate is performed by selecting the desired
     horizontal and vertical blanking. The unit of this control is Hz.
 
+    Note that this control isn't affected by analogue binning as the pixels are
+    still being read at the same frequency as without analogue binning, only
+    what is being read is different (a single pixel value vs. a binned pixel
+    value based on the values of two or more pixels).
+
 ``V4L2_CID_TEST_PATTERN (menu)``
     Some capture/display/sensor devices have the capability to generate
     test pattern images. These hardware specific test patterns can be
Signature · 1 line
2.47.3
#1 lore raw

Hi Sakari

Quoted 14 lines · Sakari Ailus

Document explicitly that the PIXEL_RATE control reflects the actual frequency at which the pixels are read in the pixel array. It is thus orthogonal to analogue binning.

---
 .../userspace-api/media/v4l/ext-ctrls-image-process.rst      | 5 +++++
 1 file changed, 5 insertions(+)
Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst +0 -0
index 6d516f041ca2..8616bcd67270 100644

Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst Image Process Control IDs · line 48

Quoted 8 lines
Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst +5 -0
@@ -41,6 +41,11 @@ Image Process Control IDs
     The configuration of the frame rate is performed by selecting the desired
     horizontal and vertical blanking. The unit of this control is Hz.
 
+    Note that this control isn't affected by analogue binning as the pixels are
+    still being read at the same frequency as without analogue binning, only
+    what is being read is different (a single pixel value vs. a binned pixel
+    value based on the values of two or more pixels).
+

This breaks the model implemented to support "special" (aka analogue ?) binning mode on imx219 where we double the pixel rate to express an higher frame rate and halve the blanking values before writing them to registers

I guess we have to keep this mode working not to break existing userspace, also because it's the only way we can have it working without the introduction of the FLL and LLP controls.

With the new model and FLL and LLP controls, I presume analog binning would require userspace to program halved FLL and LLP values ?

Currently in your proposal of V4L2_CID_BINNING_FACTORS it doesn't seem there is a way to distinguish between a digital and an analogue binning mode, they're both 2x2 so I guess it's the driver that has to realize that if FLL and LLP are smaller the [output width + blank] it has to activate the analogue binning mode ?

Quoted 6 lines
Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst +0 -1
 ``V4L2_CID_TEST_PATTERN (menu)``
     Some capture/display/sensor devices have the capability to generate
     test pattern images. These hardware specific test patterns can be
--

2.47.3

#2 lore raw

Hi Jacopo,

Quoted 20 lines · Jacopo Mondi

Hi Sakari

Quoted 17 lines · Sakari Ailus

Document explicitly that the PIXEL_RATE control reflects the actual frequency at which the pixels are read in the pixel array. It is thus orthogonal to analogue binning.

---
 .../userspace-api/media/v4l/ext-ctrls-image-process.rst      | 5 +++++
 1 file changed, 5 insertions(+)
Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst +0 -0
index 6d516f041ca2..8616bcd67270 100644
@@ -41,6 +41,11 @@ Image Process Control IDs
     The configuration of the frame rate is performed by selecting the desired
     horizontal and vertical blanking. The unit of this control is Hz.
 
Quoted 10 lines

Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst Image Process Control IDs · line 48

Quoted 5 lines
Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst +5 -0
+    Note that this control isn't affected by analogue binning as the pixels are
+    still being read at the same frequency as without analogue binning, only
+    what is being read is different (a single pixel value vs. a binned pixel
+    value based on the values of two or more pixels).
+

This breaks the model implemented to support "special" (aka analogue ?) binning mode on imx219 where we double the pixel rate to express an higher frame rate and halve the blanking values before writing them to registers

Note that this patch is a clarification to the existing documentation, not a change as such. It was never expected to be used as it is currently in the imx219 driver.

Quoted 4 lines

I guess we have to keep this mode working not to break existing userspace, also because it's the only way we can have it working without the introduction of the FLL and LLP controls.

The other option is indeed to leave the HBLANK as-is but based on a discussion with Jai I decided to write a patch to remove the rate multiplier. Overall I'd say it'd be better to change this now rather than leave imx219 using a different interface behaviour than the rest.

Quoted 3 lines

With the new model and FLL and LLP controls, I presume analog binning would require userspace to program halved FLL and LLP values ?

The userspace would in fact use LLP and FLL as the sensor does, there's no need to divide or multiply these values with another variable.

Quoted 6 lines

Currently in your proposal of V4L2_CID_BINNING_FACTORS it doesn't seem there is a way to distinguish between a digital and an analogue binning mode, they're both 2x2 so I guess it's the driver that has to realize that if FLL and LLP are smaller the [output width + blank] it has to activate the analogue binning mode ?

The driver is indeed responsible for setting the limits for FLL and LLP (or blanking) controls depending on binning and analogue cropping configuration, a sensor supporting analogue binning is not a special case as such.

Quoted 5 lines
Quoted 5 lines
Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst +0 -0
 ``V4L2_CID_TEST_PATTERN (menu)``
     Some capture/display/sensor devices have the capability to generate
     test pattern images. These hardware specific test patterns can be
Signature · 3 lines
Kind regards,

Sakari Ailus
#3 lore raw

Hi Sakari

Quoted 39 lines · Sakari Ailus

Hi Jacopo,

Quoted 20 lines · Jacopo Mondi

Hi Sakari

Quoted 17 lines · Sakari Ailus

Document explicitly that the PIXEL_RATE control reflects the actual frequency at which the pixels are read in the pixel array. It is thus orthogonal to analogue binning.

---
 .../userspace-api/media/v4l/ext-ctrls-image-process.rst      | 5 +++++
 1 file changed, 5 insertions(+)
Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst +0 -0
index 6d516f041ca2..8616bcd67270 100644
@@ -41,6 +41,11 @@ Image Process Control IDs
     The configuration of the frame rate is performed by selecting the desired
     horizontal and vertical blanking. The unit of this control is Hz.
 
Quoted 10 lines

Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst Image Process Control IDs · line 48

Quoted 5 lines
Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst +5 -0
+    Note that this control isn't affected by analogue binning as the pixels are
+    still being read at the same frequency as without analogue binning, only
+    what is being read is different (a single pixel value vs. a binned pixel
+    value based on the values of two or more pixels).
+

This breaks the model implemented to support "special" (aka analogue ?) binning mode on imx219 where we double the pixel rate to express an higher frame rate and halve the blanking values before writing them to registers

Note that this patch is a clarification to the existing documentation, not a change as such. It was never expected to be used as it is currently in the imx219 driver.

Quoted 1 line
Quoted 9 lines
Quoted 3 lines

I guess we have to keep this mode working not to break existing userspace, also because it's the only way we can have it working without the introduction of the FLL and LLP controls.

The other option is indeed to leave the HBLANK as-is but based on a discussion with Jai I decided to write a patch to remove the rate multiplier. Overall I'd say it'd be better to change this now rather than leave imx219 using a different interface behaviour than the rest.

imx219 is PiCamera module v2, it's very popular, I would be careful changing it. If I'm not mistaken (please confirm) your series now require userspace to program FLL and LLP in units of two lines when using binning. What would happen to existing userspace with the new version ?

Again, I would defer it to Dave as the maintainer.

Quoted 14 lines
Quoted 3 lines

With the new model and FLL and LLP controls, I presume analog binning would require userspace to program halved FLL and LLP values ?

The userspace would in fact use LLP and FLL as the sensor does, there's no need to divide or multiply these values with another variable.

Quoted 6 lines

Currently in your proposal of V4L2_CID_BINNING_FACTORS it doesn't seem there is a way to distinguish between a digital and an analogue binning mode, they're both 2x2 so I guess it's the driver that has to realize that if FLL and LLP are smaller the [output width + blank] it has to activate the analogue binning mode ?

Quoted 4 lines

The driver is indeed responsible for setting the limits for FLL and LLP (or blanking) controls depending on binning and analogue cropping configuration, a sensor supporting analogue binning is not a special case as such.

Maybe I wasn't clear

The imx219 supports 2 different 2x2 binning modes

#define IMX219_BINNING_X2 0x01
#define IMX219_BINNING_X2_ANALOG 0x03

We currently use X2_ANALOG unconditionally, and your patches do not change this, I agree.

My thinking was in general, is there a need to distinguish between a 2x2 digital binning and a 2x2 analog binning ? Does the new API allow to do so ?

Quoted 11 lines
Quoted 6 lines

Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst Image Process Control IDs · line 51

Quoted 6 lines
Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst +0 -0
 ``V4L2_CID_TEST_PATTERN (menu)``
     Some capture/display/sensor devices have the capability to generate
     test pattern images. These hardware specific test patterns can be

--
Kind regards,

Sakari Ailus

#4 lore raw

Hi Jacopo,

Quoted 52 lines · Jacopo Mondi

Hi Sakari

Quoted 39 lines · Sakari Ailus

Hi Jacopo,

Quoted 20 lines · Jacopo Mondi

Hi Sakari

Quoted 17 lines · Sakari Ailus

Document explicitly that the PIXEL_RATE control reflects the actual frequency at which the pixels are read in the pixel array. It is thus orthogonal to analogue binning.

---
 .../userspace-api/media/v4l/ext-ctrls-image-process.rst      | 5 +++++
 1 file changed, 5 insertions(+)
Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst +0 -0
index 6d516f041ca2..8616bcd67270 100644
@@ -41,6 +41,11 @@ Image Process Control IDs
     The configuration of the frame rate is performed by selecting the desired
     horizontal and vertical blanking. The unit of this control is Hz.
 
Quoted 10 lines

Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst Image Process Control IDs · line 48

Quoted 5 lines
Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst +5 -0
+    Note that this control isn't affected by analogue binning as the pixels are
+    still being read at the same frequency as without analogue binning, only
+    what is being read is different (a single pixel value vs. a binned pixel
+    value based on the values of two or more pixels).
+

This breaks the model implemented to support "special" (aka analogue ?) binning mode on imx219 where we double the pixel rate to express an higher frame rate and halve the blanking values before writing them to registers

Note that this patch is a clarification to the existing documentation, not a change as such. It was never expected to be used as it is currently in the imx219 driver.

Quoted 1 line
Quoted 9 lines
Quoted 3 lines

I guess we have to keep this mode working not to break existing userspace, also because it's the only way we can have it working without the introduction of the FLL and LLP controls.

The other option is indeed to leave the HBLANK as-is but based on a discussion with Jai I decided to write a patch to remove the rate multiplier. Overall I'd say it'd be better to change this now rather than leave imx219 using a different interface behaviour than the rest.

Quoted 5 lines

imx219 is PiCamera module v2, it's very popular, I would be careful changing it. If I'm not mistaken (please confirm) your series now require userspace to program FLL and LLP in units of two lines when using binning. What would happen to existing userspace with the new version ?

The unit of LLP and FLL are pixels and lines, respectively. Please see my other reply <[email protected]>. This patch isn't directly related to the imx219 driver, the purpose here was to define it in a way that left no room for interpretation.

The imx219 driver is a different discussion as it does affect an existing UAPI.

Quoted 29 lines

Again, I would defer it to Dave as the maintainer.

Quoted 14 lines
Quoted 3 lines

With the new model and FLL and LLP controls, I presume analog binning would require userspace to program halved FLL and LLP values ?

The userspace would in fact use LLP and FLL as the sensor does, there's no need to divide or multiply these values with another variable.

Quoted 6 lines

Currently in your proposal of V4L2_CID_BINNING_FACTORS it doesn't seem there is a way to distinguish between a digital and an analogue binning mode, they're both 2x2 so I guess it's the driver that has to realize that if FLL and LLP are smaller the [output width + blank] it has to activate the analogue binning mode ?

Quoted 4 lines

The driver is indeed responsible for setting the limits for FLL and LLP (or blanking) controls depending on binning and analogue cropping configuration, a sensor supporting analogue binning is not a special case as such.

Maybe I wasn't clear

The imx219 supports 2 different 2x2 binning modes

#define IMX219_BINNING_X2 0x01
#define IMX219_BINNING_X2_ANALOG 0x03

Quoted 6 lines

We currently use X2_ANALOG unconditionally, and your patches do not change this, I agree.

My thinking was in general, is there a need to distinguish between a 2x2 digital binning and a 2x2 analog binning ? Does the new API allow to do so ?

I think we'd need a new control to do that, if we needed to tell them apart. I'm fairly certain we'll never need the digital option.

Signature · 3 lines
Kind regards,

Sakari Ailus