Skip to contentSkip to conversation

New V4L2 controls V4L2_CID_NOTIFY_GAIN_XXX

PATCH Superseded David Plowman 4 participants 9 messages May 17, 2021 – May 27, 2021 Media (V4L/DVB)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

cover New V4L2 controls V4L2_CID_NOTIFY_GAIN_XXX

Hi

I'd like to propose some new V4L2 controls as defined in the attached patches. The controls are:

V4L2_CID_NOTIFY_GAIN_RED
V4L2_CID_NOTIFY_GAIN_GREENR
V4L2_CID_NOTIFY_GAIN_BLUE
V4L2_CID_NOTIFY_GAIN_GREENB

The purpose of these controls is to be able to notify a raw sensor what colour gains will be applied by subsequent processing (such as by an ISP). Equivalently we can think of them as telling the sensor what the white point is. Note that the sensor is told these gains but does not apply them - which the choice of name is trying to convey.

Some sensors need to know these numbers for the processing that they perform. Here I'm thinking in particular of so-called "quad Bayer" (also sometimes "tetracell") sensors that have a special read-out mode that converts the non-standard Bayer pattern into a standard one, also at full resolution.

Sensors of this type are becoming quite common on cell phones where, for example, a 48MP sensor may be able to deliver multiple exposures at 12MP (for HDR processing perhaps) but they may also have a mode as described above where they can generate a standard Bayer output at 48MP. This processing works better - we might expect less colour aliasing? - when the sensor knows what values correspond to "white".

One question in my mind is whether it's worth having a control for each green channel. The sensor I'm currently looking at only wants a single green gain, but perhaps it's one of those instances where it would be annoying to put in a single gain and discover, sometime later, a sensor that wants both. Opinions on the matter always appreciated!

Thanks very much and best regards

David

David Plowman (2):
media: v4l2-ctrls: Add V4L2_CID_NOTIFY_GAIN_XXX controls
media: v4l2-ctrls: Document V4L2_CID_NOTIFY_GAIN_XXX controls

 .../media/v4l/ext-ctrls-image-source.rst      | 28 +++++++++++++++++++
 drivers/media/v4l2-core/v4l2-ctrls.c          |  4 +++
 include/uapi/linux/v4l2-controls.h            |  4 +++
 3 files changed, 36 insertions(+)
Signature · 1 line
2.17.1
#1 lore raw

PATCH 1/2 media: v4l2-ctrls: Add V4L2_CID_NOTIFY_GAIN_XXX controls

We add new controls, one for each of the four usual Bayer channels:

V4L2_CID_NOTIFY_GAIN_RED
V4L2_CID_NOTIFY_GAIN_GREENR
V4L2_CID_NOTIFY_GAIN_BLUE
V4L2_CID_NOTIFY_GAIN_GREENB

These are provided for sensors that need to know what colour gains will be applied to the Bayer channel by subsequent processing (such as by an ISP), even though the sensor will not apply this gain itself.

The units, as with analogue gain, are determined by the driver.

---
 drivers/media/v4l2-core/v4l2-ctrls.c | 4 ++++
 include/uapi/linux/v4l2-controls.h   | 4 ++++
 2 files changed, 8 insertions(+)
drivers/media/v4l2-core/v4l2-ctrls.c +4 -0
index 0d7fe1bd975a..2f4436e04cf9 100644
@@ -1150,6 +1150,10 @@ const char *v4l2_ctrl_get_name(u32 id)
 	case V4L2_CID_TEST_PATTERN_GREENR:	return "Green (Red) Pixel Value";
 	case V4L2_CID_TEST_PATTERN_BLUE:	return "Blue Pixel Value";
 	case V4L2_CID_TEST_PATTERN_GREENB:	return "Green (Blue) Pixel Value";
+	case V4L2_CID_NOTIFY_GAIN_RED:		return "Notify Red Gain";
+	case V4L2_CID_NOTIFY_GAIN_GREENR:	return "Notify Green (Red) Gain";
+	case V4L2_CID_NOTIFY_GAIN_BLUE:		return "Notify Blue Gain";
+	case V4L2_CID_NOTIFY_GAIN_GREENB:	return "Notify Green (Blue) Gain";
 
 	/* Image processing controls */
 	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
include/uapi/linux/v4l2-controls.h +4 -0
index d43bec5f1afd..dff5f0d26d4a 100644
@@ -1116,6 +1116,10 @@ enum v4l2_jpeg_chroma_subsampling {
 #define V4L2_CID_TEST_PATTERN_BLUE		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 6)
 #define V4L2_CID_TEST_PATTERN_GREENB		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 7)
 #define V4L2_CID_UNIT_CELL_SIZE			(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 8)
+#define V4L2_CID_NOTIFY_GAIN_RED		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 9)
+#define V4L2_CID_NOTIFY_GAIN_GREENR		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 10)
+#define V4L2_CID_NOTIFY_GAIN_BLUE		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 11)
+#define V4L2_CID_NOTIFY_GAIN_GREENB		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 12)
 
 
 /* Image processing controls */
Signature · 1 line
2.17.1
#2 lore raw

PATCH 2/2 media: v4l2-ctrls: Document V4L2_CID_NOTIFY_GAIN_XXX controls

In reply to David Plowman

Add documentation for each of the controls

V4L2_CID_NOTIFY_GAIN_RED
V4L2_CID_NOTIFY_GAIN_GREENR
V4L2_CID_NOTIFY_GAIN_BLUE
V4L2_CID_NOTIFY_GAIN_GREENB

These controls are required by sensors that need to know what colour gains will be applied to pixels by downstream processing (such as by an ISP), though the sensor does not apply these gains itself.

---
 .../media/v4l/ext-ctrls-image-source.rst      | 28 +++++++++++++++++++
 1 file changed, 28 insertions(+)
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +28 -0
index de43f5c8486d..f824d6c36ae8 100644
@@ -72,3 +72,31 @@ Image Source Control IDs
     * - __u32
       - ``height``
       - Height of the area.
+
+``V4L2_CID_NOTIFY_GAIN_RED (integer)``
+    Notify the sensor what gain will be applied to red pixels by the
+    subsequent processing (such as by an ISP). The sensor is merely
+    informed of this value in case it performs processing that requires
+    it, but it is not applied to the output pixels themselves. The
+    units are determined by the sensor driver.
+
+``V4L2_CID_NOTIFY_GAIN_GREENR (integer)``
+    Notify the sensor what gain will be applied to green pixels (on
+    red rows) by subsequent processing (such as by an ISP). The sensor
+    is merely informed of this value in case it performs processing
+    that requires it, but it is not applied to the output pixels
+    themselves. The units are determined by the sensor driver.
+
+``V4L2_CID_NOTIFY_GAIN_BLUE (integer)``
+    Notify the sensor what gain will be applied to blue pixels by the
+    subsequent processing (such as by an ISP). The sensor is merely
+    informed of this value in case it performs processing that requires
+    it, but it is not applied to the output pixels themselves. The
+    units are determined by the sensor driver.
+
+``V4L2_CID_NOTIFY_GAIN_GREENB (integer)``
+    Notify the sensor what gain will be applied to green pixels (on
+    blue rows) by subsequent processing (such as by an ISP). The sensor
+    is merely informed of this value in case it performs processing
+    that requires it, but it is not applied to the output pixels
+    themselves. The units are determined by the sensor driver.
Signature · 1 line
2.17.1
#3 lore raw

Re: [PATCH 2/2] media: v4l2-ctrls: Document V4L2_CID_NOTIFY_GAIN_XXX controls

Hi David,

Thanks for the patch.

Cc'ing Laurent, too.

Quoted 21 lines · David Plowman

Add documentation for each of the controls

V4L2_CID_NOTIFY_GAIN_RED
V4L2_CID_NOTIFY_GAIN_GREENR
V4L2_CID_NOTIFY_GAIN_BLUE
V4L2_CID_NOTIFY_GAIN_GREENB

These controls are required by sensors that need to know what colour gains will be applied to pixels by downstream processing (such as by an ISP), though the sensor does not apply these gains itself.

---
 .../media/v4l/ext-ctrls-image-source.rst      | 28 +++++++++++++++++++
 1 file changed, 28 insertions(+)
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +0 -0
index de43f5c8486d..f824d6c36ae8 100644

Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst Image Source Control IDs · line 81

Quoted 10 lines
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +7 -0
@@ -72,3 +72,31 @@ Image Source Control IDs
     * - __u32
       - ``height``
       - Height of the area.
+
+``V4L2_CID_NOTIFY_GAIN_RED (integer)``
+    Notify the sensor what gain will be applied to red pixels by the
+    subsequent processing (such as by an ISP). The sensor is merely
+    informed of this value in case it performs processing that requires
+    it, but it is not applied to the output pixels themselves. The
+    units are determined by the sensor driver.

I wonder if this should say the default value should reflect gain of 1. It probably wouldn't hurt at least.

Quoted 21 lines
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +21 -0
+
+``V4L2_CID_NOTIFY_GAIN_GREENR (integer)``
+    Notify the sensor what gain will be applied to green pixels (on
+    red rows) by subsequent processing (such as by an ISP). The sensor
+    is merely informed of this value in case it performs processing
+    that requires it, but it is not applied to the output pixels
+    themselves. The units are determined by the sensor driver.
+
+``V4L2_CID_NOTIFY_GAIN_BLUE (integer)``
+    Notify the sensor what gain will be applied to blue pixels by the
+    subsequent processing (such as by an ISP). The sensor is merely
+    informed of this value in case it performs processing that requires
+    it, but it is not applied to the output pixels themselves. The
+    units are determined by the sensor driver.
+
+``V4L2_CID_NOTIFY_GAIN_GREENB (integer)``
+    Notify the sensor what gain will be applied to green pixels (on
+    blue rows) by subsequent processing (such as by an ISP). The sensor
+    is merely informed of this value in case it performs processing
+    that requires it, but it is not applied to the output pixels
+    themselves. The units are determined by the sensor driver.
Signature · 3 lines
Kind regards,

Sakari Ailus
#4 lore raw

Re: [PATCH 2/2] media: v4l2-ctrls: Document V4L2_CID_NOTIFY_GAIN_XXX controls

Added Laurent to cc.

Quoted 68 lines · Sakari Ailus

Hi David,

Thanks for the patch.

Cc'ing Laurent, too.

Quoted 21 lines · David Plowman

Add documentation for each of the controls

V4L2_CID_NOTIFY_GAIN_RED
V4L2_CID_NOTIFY_GAIN_GREENR
V4L2_CID_NOTIFY_GAIN_BLUE
V4L2_CID_NOTIFY_GAIN_GREENB

These controls are required by sensors that need to know what colour gains will be applied to pixels by downstream processing (such as by an ISP), though the sensor does not apply these gains itself.

---
 .../media/v4l/ext-ctrls-image-source.rst      | 28 +++++++++++++++++++
 1 file changed, 28 insertions(+)
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +0 -0
index de43f5c8486d..f824d6c36ae8 100644

Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst Image Source Control IDs · line 81

Quoted 10 lines
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +7 -0
@@ -72,3 +72,31 @@ Image Source Control IDs
     * - __u32
       - ``height``
       - Height of the area.
+
+``V4L2_CID_NOTIFY_GAIN_RED (integer)``
+    Notify the sensor what gain will be applied to red pixels by the
+    subsequent processing (such as by an ISP). The sensor is merely
+    informed of this value in case it performs processing that requires
+    it, but it is not applied to the output pixels themselves. The
+    units are determined by the sensor driver.

I wonder if this should say the default value should reflect gain of 1. It probably wouldn't hurt at least.

Quoted 21 lines
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +21 -0
+
+``V4L2_CID_NOTIFY_GAIN_GREENR (integer)``
+    Notify the sensor what gain will be applied to green pixels (on
+    red rows) by subsequent processing (such as by an ISP). The sensor
+    is merely informed of this value in case it performs processing
+    that requires it, but it is not applied to the output pixels
+    themselves. The units are determined by the sensor driver.
+
+``V4L2_CID_NOTIFY_GAIN_BLUE (integer)``
+    Notify the sensor what gain will be applied to blue pixels by the
+    subsequent processing (such as by an ISP). The sensor is merely
+    informed of this value in case it performs processing that requires
+    it, but it is not applied to the output pixels themselves. The
+    units are determined by the sensor driver.
+
+``V4L2_CID_NOTIFY_GAIN_GREENB (integer)``
+    Notify the sensor what gain will be applied to green pixels (on
+    blue rows) by subsequent processing (such as by an ISP). The sensor
+    is merely informed of this value in case it performs processing
+    that requires it, but it is not applied to the output pixels
+    themselves. The units are determined by the sensor driver.
Signature · 3 lines
Kind regards,

Sakari Ailus
Signature · 1 line
Sakari Ailus
#5 lore raw

Re: [PATCH 2/2] media: v4l2-ctrls: Document V4L2_CID_NOTIFY_GAIN_XXX controls

Hello,

Quoted 44 lines · Sakari Ailus

Added Laurent to cc.

Quoted 41 lines · Sakari Ailus

Hi David,

Thanks for the patch.

Cc'ing Laurent, too.

Quoted 21 lines · David Plowman

Add documentation for each of the controls

V4L2_CID_NOTIFY_GAIN_RED
V4L2_CID_NOTIFY_GAIN_GREENR
V4L2_CID_NOTIFY_GAIN_BLUE
V4L2_CID_NOTIFY_GAIN_GREENB

These controls are required by sensors that need to know what colour gains will be applied to pixels by downstream processing (such as by an ISP), though the sensor does not apply these gains itself.

---
 .../media/v4l/ext-ctrls-image-source.rst      | 28 +++++++++++++++++++
 1 file changed, 28 insertions(+)
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +0 -0
index de43f5c8486d..f824d6c36ae8 100644

Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst Image Source Control IDs · line 81

Quoted 10 lines
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +7 -0
@@ -72,3 +72,31 @@ Image Source Control IDs
     * - __u32
       - ``height``
       - Height of the area.
+
+``V4L2_CID_NOTIFY_GAIN_RED (integer)``
+    Notify the sensor what gain will be applied to red pixels by the
+    subsequent processing (such as by an ISP). The sensor is merely
+    informed of this value in case it performs processing that requires
+    it, but it is not applied to the output pixels themselves. The
+    units are determined by the sensor driver.

I wonder if this should say the default value should reflect gain of 1. It probably wouldn't hurt at least.

Seems reasonable to me.

David, do you think we could also document that the values of these controls are linear, or would that be too restrictive ?

Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +21 -0
+
+``V4L2_CID_NOTIFY_GAIN_GREENR (integer)``
+    Notify the sensor what gain will be applied to green pixels (on
+    red rows) by subsequent processing (such as by an ISP). The sensor
+    is merely informed of this value in case it performs processing
+    that requires it, but it is not applied to the output pixels
+    themselves. The units are determined by the sensor driver.
+
+``V4L2_CID_NOTIFY_GAIN_BLUE (integer)``
+    Notify the sensor what gain will be applied to blue pixels by the
+    subsequent processing (such as by an ISP). The sensor is merely
+    informed of this value in case it performs processing that requires
+    it, but it is not applied to the output pixels themselves. The
+    units are determined by the sensor driver.
+
+``V4L2_CID_NOTIFY_GAIN_GREENB (integer)``
+    Notify the sensor what gain will be applied to green pixels (on
+    blue rows) by subsequent processing (such as by an ISP). The sensor
+    is merely informed of this value in case it performs processing
+    that requires it, but it is not applied to the output pixels
+    themselves. The units are determined by the sensor driver.
Signature · 3 lines
Regards,

Laurent Pinchart
#6 lore raw

Re: [PATCH 2/2] media: v4l2-ctrls: Document V4L2_CID_NOTIFY_GAIN_XXX controls

Hi Sakari, Laurent, everyone

Thanks for the feedback.

Quoted 49 lines · Laurent Pinchart

Hello,

Quoted 44 lines · Sakari Ailus

Added Laurent to cc.

Quoted 41 lines · Sakari Ailus

Hi David,

Thanks for the patch.

Cc'ing Laurent, too.

Quoted 21 lines · David Plowman

Add documentation for each of the controls

V4L2_CID_NOTIFY_GAIN_RED
V4L2_CID_NOTIFY_GAIN_GREENR
V4L2_CID_NOTIFY_GAIN_BLUE
V4L2_CID_NOTIFY_GAIN_GREENB

These controls are required by sensors that need to know what colour gains will be applied to pixels by downstream processing (such as by an ISP), though the sensor does not apply these gains itself.

---
 .../media/v4l/ext-ctrls-image-source.rst      | 28 +++++++++++++++++++
 1 file changed, 28 insertions(+)
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +0 -0
index de43f5c8486d..f824d6c36ae8 100644

Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst Image Source Control IDs · line 81

Quoted 10 lines
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +7 -0
@@ -72,3 +72,31 @@ Image Source Control IDs
     * - __u32
       - ``height``
       - Height of the area.
+
+``V4L2_CID_NOTIFY_GAIN_RED (integer)``
+    Notify the sensor what gain will be applied to red pixels by the
+    subsequent processing (such as by an ISP). The sensor is merely
+    informed of this value in case it performs processing that requires
+    it, but it is not applied to the output pixels themselves. The
+    units are determined by the sensor driver.

I wonder if this should say the default value should reflect gain of 1. It probably wouldn't hurt at least.

Quoted 1 line

Seems reasonable to me.

Yes, I think this is a good idea.

Quoted 3 lines

David, do you think we could also document that the values of these controls are linear, or would that be too restrictive ?

That's an interesting point. I guess I was drawing on the precedent set by analogue gain, but in many respects mandating a linear response, and not letting device specific units escape the driver, might be more convenient for application code.

The typical use would, I expect, involve firmware reading colour gains from an ISP and passing them in here. As such, linear is quite likely to be desirable. In the event that there is a conversion to be done, the driver can always take care of it.

Further, if we want to go with linear values, I wonder whether we should go all the way and specify it fully. For example we could mandate u8.8 values, so that 256 means 1.0x - this already feels like more than enough range and precision.

There's a slight question in my mind as to whether we should specify the range of the control too, in particular whether gains can be less than unity or not. Again, this would be kind to applications, but drivers might have to re-interpret it internally for "fussy" sensors. Any other opinions on that?

One side effect of specifying the meaning precisely is that all four necessarily become identical. I hadn't mentioned this previously, but having them different would clearly be a right nuisance!

If that sounds good to everyone I'll make up a second version with updated documentation.

Thanks and best regards
David

Quoted 27 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +13 -0
+
+``V4L2_CID_NOTIFY_GAIN_GREENR (integer)``
+    Notify the sensor what gain will be applied to green pixels (on
+    red rows) by subsequent processing (such as by an ISP). The sensor
+    is merely informed of this value in case it performs processing
+    that requires it, but it is not applied to the output pixels
+    themselves. The units are determined by the sensor driver.
+
+``V4L2_CID_NOTIFY_GAIN_BLUE (integer)``
+    Notify the sensor what gain will be applied to blue pixels by the
+    subsequent processing (such as by an ISP). The sensor is merely
+    informed of this value in case it performs processing that requires
+    it, but it is not applied to the output pixels themselves. The
Quoted 8 lines
Quoted 8 lines

Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst Image Source Control IDs · line 102

Quoted 8 lines
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +8 -0
+    units are determined by the sensor driver.
+
+``V4L2_CID_NOTIFY_GAIN_GREENB (integer)``
+    Notify the sensor what gain will be applied to green pixels (on
+    blue rows) by subsequent processing (such as by an ISP). The sensor
+    is merely informed of this value in case it performs processing
+    that requires it, but it is not applied to the output pixels
+    themselves. The units are determined by the sensor driver.

--
Regards,

Laurent Pinchart

#7 lore raw

Re: [PATCH 2/2] media: v4l2-ctrls: Document V4L2_CID_NOTIFY_GAIN_XXX controls

Hi David,

Quoted 68 lines · David Plowman

Hi Sakari, Laurent, everyone

Thanks for the feedback.

Quoted 49 lines · Laurent Pinchart

Hello,

Quoted 44 lines · Sakari Ailus

Added Laurent to cc.

Quoted 41 lines · Sakari Ailus

Hi David,

Thanks for the patch.

Cc'ing Laurent, too.

Quoted 21 lines · David Plowman

Add documentation for each of the controls

V4L2_CID_NOTIFY_GAIN_RED
V4L2_CID_NOTIFY_GAIN_GREENR
V4L2_CID_NOTIFY_GAIN_BLUE
V4L2_CID_NOTIFY_GAIN_GREENB

These controls are required by sensors that need to know what colour gains will be applied to pixels by downstream processing (such as by an ISP), though the sensor does not apply these gains itself.

---
 .../media/v4l/ext-ctrls-image-source.rst      | 28 +++++++++++++++++++
 1 file changed, 28 insertions(+)
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +0 -0
index de43f5c8486d..f824d6c36ae8 100644

Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst Image Source Control IDs · line 81

Quoted 10 lines
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +7 -0
@@ -72,3 +72,31 @@ Image Source Control IDs
     * - __u32
       - ``height``
       - Height of the area.
+
+``V4L2_CID_NOTIFY_GAIN_RED (integer)``
+    Notify the sensor what gain will be applied to red pixels by the
+    subsequent processing (such as by an ISP). The sensor is merely
+    informed of this value in case it performs processing that requires
+    it, but it is not applied to the output pixels themselves. The
+    units are determined by the sensor driver.

I wonder if this should say the default value should reflect gain of 1. It probably wouldn't hurt at least.

Quoted 1 line

Seems reasonable to me.

Yes, I think this is a good idea.

Quoted 3 lines

David, do you think we could also document that the values of these controls are linear, or would that be too restrictive ?

That's an interesting point. I guess I was drawing on the precedent set by analogue gain, but in many respects mandating a linear response, and not letting device specific units escape the driver, might be more convenient for application code.

Quoted 4 lines

The typical use would, I expect, involve firmware reading colour gains from an ISP and passing them in here. As such, linear is quite likely to be desirable. In the event that there is a conversion to be done, the driver can always take care of it.

I don't see issues with the approach.

Quoted 11 lines

Further, if we want to go with linear values, I wonder whether we should go all the way and specify it fully. For example we could mandate u8.8 values, so that 256 means 1.0x - this already feels like more than enough range and precision.

There's a slight question in my mind as to whether we should specify the range of the control too, in particular whether gains can be less than unity or not. Again, this would be kind to applications, but drivers might have to re-interpret it internally for "fussy" sensors. Any other opinions on that?

The controls API already can provide this information, I wouldn't add it to the documentation of a specific control.

Quoted 4 lines

One side effect of specifying the meaning precisely is that all four necessarily become identical. I hadn't mentioned this previously, but having them different would clearly be a right nuisance!

Agreed.

Quoted 3 lines

If that sounds good to everyone I'll make up a second version with updated documentation.

I wonder how it'd look like if all these controls had a single explanation instead of one for each. I don't think we've done this earlier, but they're all pretty much alike and used together in any case.

Signature · 3 lines
Regards,

Sakari Ailus
#8 lore raw
In reply to David Plowman

Hi David,

Quoted 29 lines · David Plowman

Hi

I'd like to propose some new V4L2 controls as defined in the attached patches. The controls are:

V4L2_CID_NOTIFY_GAIN_RED
V4L2_CID_NOTIFY_GAIN_GREENR
V4L2_CID_NOTIFY_GAIN_BLUE
V4L2_CID_NOTIFY_GAIN_GREENB

The purpose of these controls is to be able to notify a raw sensor what colour gains will be applied by subsequent processing (such as by an ISP). Equivalently we can think of them as telling the sensor what the white point is. Note that the sensor is told these gains but does not apply them - which the choice of name is trying to convey.

Some sensors need to know these numbers for the processing that they perform. Here I'm thinking in particular of so-called "quad Bayer" (also sometimes "tetracell") sensors that have a special read-out mode that converts the non-standard Bayer pattern into a standard one, also at full resolution.

Sensors of this type are becoming quite common on cell phones where, for example, a 48MP sensor may be able to deliver multiple exposures at 12MP (for HDR processing perhaps) but they may also have a mode as described above where they can generate a standard Bayer output at 48MP. This processing works better - we might expect less colour aliasing? - when the sensor knows what values correspond to "white".

Quoted 6 lines

One question in my mind is whether it's worth having a control for each green channel. The sensor I'm currently looking at only wants a single green gain, but perhaps it's one of those instances where it would be annoying to put in a single gain and discover, sometime later, a sensor that wants both. Opinions on the matter always appreciated!

I think I would like to see this series in combination with an actual sensor driver + ISP that uses them.

I am a bit concerned about the fact that the units are sensor-specific, so how would an ISP driver be able to set them in a sensor-independent manner? It seems that this would require knowledge about the sensor internals, which is something you want to avoid in a ISP driver.

Perhaps this should be a multiplication factor instead? I.e. if the value is X, then the ISP calculated red as the original red value times (X / 100). So if X = 212, then the red gain factor is 2.12.

It's probably best to have two controls for green IMHO.

Regards,

Hans

Quoted 14 lines

Thanks very much and best regards

David

David Plowman (2):
media: v4l2-ctrls: Add V4L2_CID_NOTIFY_GAIN_XXX controls
media: v4l2-ctrls: Document V4L2_CID_NOTIFY_GAIN_XXX controls

 .../media/v4l/ext-ctrls-image-source.rst      | 28 +++++++++++++++++++
 drivers/media/v4l2-core/v4l2-ctrls.c          |  4 +++
 include/uapi/linux/v4l2-controls.h            |  4 +++
 3 files changed, 36 insertions(+)