Skip to contentSkip to conversation

New V4L2 control V4L2_CID_NOTIFY_GAINS

PATCH Superseded David Plowman 4 participants 9 messages Aug 9, 2021 Media (V4L/DVB)

v4 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

v4 cover New V4L2 control V4L2_CID_NOTIFY_GAINS

Hi Laurent, Sakari, everyone

Thanks for the various comments on the previous version of these patches. This latest revision is identical in just about all respects, the difference being that the four previous controls (V4L2_CID_NOTIFY_GAIN_XXX) have been replaced by a single "array" control (V4L2_CID_NOTIFY_GAINS), as per that discussion.

Thanks and best regards
David

David Plowman (2):
media: v4l2-ctrls: Add V4L2_CID_NOTIFY_GAINS control
media: v4l2-ctrls: Document V4L2_CID_NOTIFY_GAINS control

 .../media/v4l/ext-ctrls-image-source.rst          | 15 +++++++++++++++
 drivers/media/v4l2-core/v4l2-ctrls-defs.c         |  1 +
 include/uapi/linux/v4l2-controls.h                |  1 +
 3 files changed, 17 insertions(+)
Signature · 1 line
2.30.2
#1 lore raw

PATCH v4 1/2 media: v4l2-ctrls: Add V4L2_CID_NOTIFY_GAINS control

We add a new control V4L2_CID_NOTIFY_GAINS which allows the sensor to be notified what gains will be applied to the different colour channels by subsequent processing (such as by an ISP), even though the sensor will not apply any of these gains itself.

For Bayer sensors this will be an array control taking 4 values which are the 4 gains arranged in the fixed order B, Gb, Gr and R, irrespective of the exact Bayer order of the sensor itself.

The units are in all cases linear with the default value indicating a gain of exactly 1.

---
 drivers/media/v4l2-core/v4l2-ctrls-defs.c | 1 +
 include/uapi/linux/v4l2-controls.h        | 1 +
 2 files changed, 2 insertions(+)
drivers/media/v4l2-core/v4l2-ctrls-defs.c +1 -0
index 421300e13a41..f87053c83249 100644
@@ -1107,6 +1107,7 @@ const char *v4l2_ctrl_get_name(u32 id)
 	case V4L2_CID_TEST_PATTERN_GREENR:	return "Green (Red) Pixel Value";
 	case V4L2_CID_TEST_PATTERN_BLUE:	return "Blue Pixel Value";
 	case V4L2_CID_TEST_PATTERN_GREENB:	return "Green (Blue) Pixel Value";
+	case V4L2_CID_NOTIFY_GAINS:		return "Notify Gains";
 
 	/* Image processing controls */
 	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
include/uapi/linux/v4l2-controls.h +1 -0
index 5532b5f68493..133e20444939 100644
@@ -1118,6 +1118,7 @@ enum v4l2_jpeg_chroma_subsampling {
 #define V4L2_CID_TEST_PATTERN_BLUE		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 6)
 #define V4L2_CID_TEST_PATTERN_GREENB		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 7)
 #define V4L2_CID_UNIT_CELL_SIZE			(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 8)
+#define V4L2_CID_NOTIFY_GAINS			(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 9)
 
 
 /* Image processing controls */
Signature · 1 line
2.30.2
#2 lore raw

Re: [PATCH v4 1/2] media: v4l2-ctrls: Add V4L2_CID_NOTIFY_GAINS control

Quoted 11 lines · David Plowman

We add a new control V4L2_CID_NOTIFY_GAINS which allows the sensor to be notified what gains will be applied to the different colour channels by subsequent processing (such as by an ISP), even though the sensor will not apply any of these gains itself.

For Bayer sensors this will be an array control taking 4 values which are the 4 gains arranged in the fixed order B, Gb, Gr and R, irrespective of the exact Bayer order of the sensor itself.

The units are in all cases linear with the default value indicating a gain of exactly 1.

So a value of 2 means a gain of 2? Or are these fixed point values? How do I represent a gain of 1.5?

Quoted 12 lines
---
 drivers/media/v4l2-core/v4l2-ctrls-defs.c | 1 +
 include/uapi/linux/v4l2-controls.h        | 1 +
 2 files changed, 2 insertions(+)
drivers/media/v4l2-core/v4l2-ctrls-defs.c +0 -0
index 421300e13a41..f87053c83249 100644

drivers/media/v4l2-core/v4l2-ctrls-defs.c const char *v4l2_ctrl_get_name(u32 id) · line 1113

Quoted 7 lines
drivers/media/v4l2-core/v4l2-ctrls-defs.c +1 -0
@@ -1107,6 +1107,7 @@ const char *v4l2_ctrl_get_name(u32 id)
 	case V4L2_CID_TEST_PATTERN_GREENR:	return "Green (Red) Pixel Value";
 	case V4L2_CID_TEST_PATTERN_BLUE:	return "Blue Pixel Value";
 	case V4L2_CID_TEST_PATTERN_GREENB:	return "Green (Blue) Pixel Value";
+	case V4L2_CID_NOTIFY_GAINS:		return "Notify Gains";
 
 	/* Image processing controls */
 	/* Keep the order of the 'case's the same as in v4l2-controls.h! */

Since this is a standard control, it should also be configured correctly in v4l2_ctrl_fill().

Instead of an array, would a compound control (aka a struct) be better? Then you can explicitly have field names g, gb, gr and r.

Is there a specific reason we want an array instead of that? I'm not opposed, but I'd like to see a rationale for that.

Regards,

Hans

Quoted 13 lines
include/uapi/linux/v4l2-controls.h +1 -0
index 5532b5f68493..133e20444939 100644
@@ -1118,6 +1118,7 @@ enum v4l2_jpeg_chroma_subsampling {
 #define V4L2_CID_TEST_PATTERN_BLUE		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 6)
 #define V4L2_CID_TEST_PATTERN_GREENB		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 7)
 #define V4L2_CID_UNIT_CELL_SIZE			(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 8)
+#define V4L2_CID_NOTIFY_GAINS			(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 9)
 
 
 /* Image processing controls */
#3 lore raw

Re: [PATCH v4 1/2] media: v4l2-ctrls: Add V4L2_CID_NOTIFY_GAINS control

Hi Hans,

Quoted 10 lines · Hans Verkuil
Quoted 9 lines · David Plowman

We add a new control V4L2_CID_NOTIFY_GAINS which allows the sensor to be notified what gains will be applied to the different colour channels by subsequent processing (such as by an ISP), even though the sensor will not apply any of these gains itself.

For Bayer sensors this will be an array control taking 4 values which are the 4 gains arranged in the fixed order B, Gb, Gr and R, irrespective of the exact Bayer order of the sensor itself.

Quoted 5 lines
Quoted 2 lines

The units are in all cases linear with the default value indicating a gain of exactly 1.

So a value of 2 means a gain of 2? Or are these fixed point values? How do I represent a gain of 1.5?

No, the default value corresponds to a x1.0 gain, but it's not 1. If the default is, let's say, 128, then x2.0 will be 256.

Quoted 22 lines
Quoted 11 lines
---
 drivers/media/v4l2-core/v4l2-ctrls-defs.c | 1 +
 include/uapi/linux/v4l2-controls.h        | 1 +
 2 files changed, 2 insertions(+)
drivers/media/v4l2-core/v4l2-ctrls-defs.c +0 -0
index 421300e13a41..f87053c83249 100644

drivers/media/v4l2-core/v4l2-ctrls-defs.c const char *v4l2_ctrl_get_name(u32 id) · line 1113

Quoted 7 lines
drivers/media/v4l2-core/v4l2-ctrls-defs.c +1 -0
@@ -1107,6 +1107,7 @@ const char *v4l2_ctrl_get_name(u32 id)
 	case V4L2_CID_TEST_PATTERN_GREENR:	return "Green (Red) Pixel Value";
 	case V4L2_CID_TEST_PATTERN_BLUE:	return "Blue Pixel Value";
 	case V4L2_CID_TEST_PATTERN_GREENB:	return "Green (Blue) Pixel Value";
+	case V4L2_CID_NOTIFY_GAINS:		return "Notify Gains";
 
 	/* Image processing controls */
 	/* Keep the order of the 'case's the same as in v4l2-controls.h! */

Since this is a standard control, it should also be configured correctly in v4l2_ctrl_fill().

Quoted 5 lines

Instead of an array, would a compound control (aka a struct) be better? Then you can explicitly have field names g, gb, gr and r.

Is there a specific reason we want an array instead of that? I'm not opposed, but I'd like to see a rationale for that.

Bayer ia only one of the possible CFA patterns for sensors. With a structure containing named b, gb, gr and r fields, we wouldn't be able to support, for instance, RGBW filters. It's not clear at this point what other CFA patterns will be seen in sensors that require this feature, but an array control will be able to more easily support these use cases.

Quoted 12 lines
Quoted 12 lines
include/uapi/linux/v4l2-controls.h +1 -0
index 5532b5f68493..133e20444939 100644
@@ -1118,6 +1118,7 @@ enum v4l2_jpeg_chroma_subsampling {
 #define V4L2_CID_TEST_PATTERN_BLUE		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 6)
 #define V4L2_CID_TEST_PATTERN_GREENB		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 7)
 #define V4L2_CID_UNIT_CELL_SIZE			(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 8)
+#define V4L2_CID_NOTIFY_GAINS			(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 9)
 
 
 /* Image processing controls */
Signature · 3 lines
Regards,

Laurent Pinchart
#4 lore raw

Re: [PATCH v4 1/2] media: v4l2-ctrls: Add V4L2_CID_NOTIFY_GAINS control

Quoted 13 lines · Laurent Pinchart

Hi Hans,

Quoted 10 lines · Hans Verkuil
Quoted 9 lines · David Plowman

We add a new control V4L2_CID_NOTIFY_GAINS which allows the sensor to be notified what gains will be applied to the different colour channels by subsequent processing (such as by an ISP), even though the sensor will not apply any of these gains itself.

For Bayer sensors this will be an array control taking 4 values which are the 4 gains arranged in the fixed order B, Gb, Gr and R, irrespective of the exact Bayer order of the sensor itself.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

The units are in all cases linear with the default value indicating a gain of exactly 1.

So a value of 2 means a gain of 2? Or are these fixed point values? How do I represent a gain of 1.5?

No, the default value corresponds to a x1.0 gain, but it's not 1. If the default is, let's say, 128, then x2.0 will be 256.

Ah, now I get it. Perhaps a small example of this in the documentation patch will help clarify this.

Quoted 29 lines
Quoted 23 lines
Quoted 12 lines
---
 drivers/media/v4l2-core/v4l2-ctrls-defs.c | 1 +
 include/uapi/linux/v4l2-controls.h        | 1 +
 2 files changed, 2 insertions(+)
drivers/media/v4l2-core/v4l2-ctrls-defs.c +0 -0
index 421300e13a41..f87053c83249 100644

drivers/media/v4l2-core/v4l2-ctrls-defs.c const char *v4l2_ctrl_get_name(u32 id) · line 1113

Quoted 7 lines
drivers/media/v4l2-core/v4l2-ctrls-defs.c +1 -0
@@ -1107,6 +1107,7 @@ const char *v4l2_ctrl_get_name(u32 id)
 	case V4L2_CID_TEST_PATTERN_GREENR:	return "Green (Red) Pixel Value";
 	case V4L2_CID_TEST_PATTERN_BLUE:	return "Blue Pixel Value";
 	case V4L2_CID_TEST_PATTERN_GREENB:	return "Green (Blue) Pixel Value";
+	case V4L2_CID_NOTIFY_GAINS:		return "Notify Gains";
 
 	/* Image processing controls */
 	/* Keep the order of the 'case's the same as in v4l2-controls.h! */

Since this is a standard control, it should also be configured correctly in v4l2_ctrl_fill().

Quoted 5 lines

Instead of an array, would a compound control (aka a struct) be better? Then you can explicitly have field names g, gb, gr and r.

Is there a specific reason we want an array instead of that? I'm not opposed, but I'd like to see a rationale for that.

Quoted 6 lines

Bayer ia only one of the possible CFA patterns for sensors. With a structure containing named b, gb, gr and r fields, we wouldn't be able to support, for instance, RGBW filters. It's not clear at this point what other CFA patterns will be seen in sensors that require this feature, but an array control will be able to more easily support these use cases.

OK. It is probably a good idea to mention this in the commit log at least.

Regards,

Hans

Quoted 14 lines
Quoted 13 lines
Quoted 13 lines
include/uapi/linux/v4l2-controls.h +1 -0
index 5532b5f68493..133e20444939 100644
@@ -1118,6 +1118,7 @@ enum v4l2_jpeg_chroma_subsampling {
 #define V4L2_CID_TEST_PATTERN_BLUE		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 6)
 #define V4L2_CID_TEST_PATTERN_GREENB		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 7)
 #define V4L2_CID_UNIT_CELL_SIZE			(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 8)
+#define V4L2_CID_NOTIFY_GAINS			(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 9)
 
 
 /* Image processing controls */
#5 lore raw

Re: [PATCH v4 1/2] media: v4l2-ctrls: Add V4L2_CID_NOTIFY_GAINS control

Hi everyone

Quoted 18 lines · Hans Verkuil
Quoted 16 lines · Laurent Pinchart

Hi Hans,

Quoted 13 lines · Hans Verkuil
Quoted 11 lines · David Plowman

We add a new control V4L2_CID_NOTIFY_GAINS which allows the sensor to be notified what gains will be applied to the different colour channels by subsequent processing (such as by an ISP), even though the sensor will not apply any of these gains itself.

For Bayer sensors this will be an array control taking 4 values which are the 4 gains arranged in the fixed order B, Gb, Gr and R, irrespective of the exact Bayer order of the sensor itself.

The units are in all cases linear with the default value indicating a gain of exactly 1.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

So a value of 2 means a gain of 2? Or are these fixed point values? How do I represent a gain of 1.5?

No, the default value corresponds to a x1.0 gain, but it's not 1. If the default is, let's say, 128, then x2.0 will be 256.

Ah, now I get it. Perhaps a small example of this in the documentation patch will help clarify this.

Yes I agree that would be helpful. I'll put that in the next version shortly (just waiting to see if there are any other changes suggested).

Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
---
 drivers/media/v4l2-core/v4l2-ctrls-defs.c | 1 +
 include/uapi/linux/v4l2-controls.h        | 1 +
 2 files changed, 2 insertions(+)
drivers/media/v4l2-core/v4l2-ctrls-defs.c +1 -0
index 421300e13a41..f87053c83249 100644
@@ -1107,6 +1107,7 @@ const char *v4l2_ctrl_get_name(u32 id)
    case V4L2_CID_TEST_PATTERN_GREENR:      return "Green (Red) Pixel Value";
    case V4L2_CID_TEST_PATTERN_BLUE:        return "Blue Pixel Value";
    case V4L2_CID_TEST_PATTERN_GREENB:      return "Green (Blue) Pixel Value";
+   case V4L2_CID_NOTIFY_GAINS:             return "Notify Gains";
 
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

drivers/media/v4l2-core/v4l2-ctrls-defs.c const char *v4l2_ctrl_get_name(u32 id) · line 1113

Quoted 2 lines
drivers/media/v4l2-core/v4l2-ctrls-defs.c +0 -0
    /* Image processing controls */
    /* Keep the order of the 'case's the same as in v4l2-controls.h! */

Since this is a standard control, it should also be configured correctly in v4l2_ctrl_fill().

Just a small clarification on this. Given that the min/max/default values are really up to the sensor what would I fill in here, maybe just the control type?

Quoted 7 lines
Quoted 7 lines
Quoted 6 lines

Instead of an array, would a compound control (aka a struct) be better? Then you can explicitly have field names g, gb, gr and r.

Is there a specific reason we want an array instead of that? I'm not opposed, but I'd like to see a rationale for that.

Quoted 8 lines
Quoted 6 lines

Bayer ia only one of the possible CFA patterns for sensors. With a structure containing named b, gb, gr and r fields, we wouldn't be able to support, for instance, RGBW filters. It's not clear at this point what other CFA patterns will be seen in sensors that require this feature, but an array control will be able to more easily support these use cases.

OK. It is probably a good idea to mention this in the commit log at least.

Will do!

Thanks and best regards
David

Quoted 20 lines

Regards,

Hans

Quoted 14 lines
Quoted 13 lines
Quoted 13 lines
include/uapi/linux/v4l2-controls.h +1 -0
index 5532b5f68493..133e20444939 100644
@@ -1118,6 +1118,7 @@ enum v4l2_jpeg_chroma_subsampling {
 #define V4L2_CID_TEST_PATTERN_BLUE         (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 6)
 #define V4L2_CID_TEST_PATTERN_GREENB               (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 7)
 #define V4L2_CID_UNIT_CELL_SIZE                    (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 8)
+#define V4L2_CID_NOTIFY_GAINS                      (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 9)
 
 
 /* Image processing controls */
#6 lore raw

Re: [PATCH v4 1/2] media: v4l2-ctrls: Add V4L2_CID_NOTIFY_GAINS control

Hi David,

Quoted 27 lines · David Plowman

Hi everyone

Quoted 24 lines · Hans Verkuil
Quoted 21 lines · Laurent Pinchart

Hi Hans,

Quoted 10 lines · Hans Verkuil
Quoted 9 lines · David Plowman

We add a new control V4L2_CID_NOTIFY_GAINS which allows the sensor to be notified what gains will be applied to the different colour channels by subsequent processing (such as by an ISP), even though the sensor will not apply any of these gains itself.

For Bayer sensors this will be an array control taking 4 values which are the 4 gains arranged in the fixed order B, Gb, Gr and R, irrespective of the exact Bayer order of the sensor itself.

Quoted 5 lines
Quoted 2 lines

The units are in all cases linear with the default value indicating a gain of exactly 1.

So a value of 2 means a gain of 2? Or are these fixed point values? How do I represent a gain of 1.5?

No, the default value corresponds to a x1.0 gain, but it's not 1. If the default is, let's say, 128, then x2.0 will be 256.

Quoted 6 lines
Quoted 2 lines

Ah, now I get it. Perhaps a small example of this in the documentation patch will help clarify this.

Yes I agree that would be helpful. I'll put that in the next version shortly (just waiting to see if there are any other changes suggested).

The digital gain control has the same semantics, see Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst .

Signature · 3 lines
Regards,

Sakari Ailus
#7 lore raw

PATCH v4 2/2 media: v4l2-ctrls: Document V4L2_CID_NOTIFY_GAINS control

In reply to David Plowman

Add documentation for the V4L2_CID_NOTIFY_GAINS control.

This control is required by sensors that need to know what colour gains will be applied to pixels by downstream processing (such as by an ISP), though the sensor does not apply these gains itself.

---
 .../media/v4l/ext-ctrls-image-source.rst          | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +15 -0
index de43f5c8486d..c1793fda1429 100644
@@ -72,3 +72,18 @@ Image Source Control IDs
     * - __u32
       - ``height``
       - Height of the area.
+
+``V4L2_CID_NOTIFY_GAINS (integer)``
+    The sensor is notified what gains will be applied to the different
+    colour channels by subsequent processing (such as by an ISP). The
+    sensor is merely informed of these values in case it performs
+    processing that requires them, but it does not apply them itself to
+    the output pixels.
+
+    For Bayer sensors this is an array control taking 4 gain values,
+    being the gains for each of the Bayer channels. The gains are always
+    in the order B, Gb, Gr and R, irrespective of the exact Bayer order
+    of the sensor itself.
+
+    The units for the gain values are linear, with the default value
+    representing a gain of exactly 1.
Signature · 1 line
2.30.2
#8 lore raw

Re: [PATCH v4 2/2] media: v4l2-ctrls: Document V4L2_CID_NOTIFY_GAINS control

Quoted 16 lines · David Plowman

Add documentation for the V4L2_CID_NOTIFY_GAINS control.

This control is required by sensors that need to know what colour gains will be applied to pixels by downstream processing (such as by an ISP), though the sensor does not apply these gains itself.

---
 .../media/v4l/ext-ctrls-image-source.rst          | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +0 -0
index de43f5c8486d..c1793fda1429 100644

Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst Image Source Control IDs · line 76

Quoted 5 lines
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +2 -0
@@ -72,3 +72,18 @@ Image Source Control IDs
     * - __u32
       - ``height``
       - Height of the area.
+
+``V4L2_CID_NOTIFY_GAINS (integer)``

Say '(integer array)' here to clarify that this is an array.

Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst Image Source Control IDs · line 89

Quoted 13 lines
Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst +13 -0
+    The sensor is notified what gains will be applied to the different
+    colour channels by subsequent processing (such as by an ISP). The
+    sensor is merely informed of these values in case it performs
+    processing that requires them, but it does not apply them itself to
+    the output pixels.
+
+    For Bayer sensors this is an array control taking 4 gain values,
+    being the gains for each of the Bayer channels. The gains are always
+    in the order B, Gb, Gr and R, irrespective of the exact Bayer order
+    of the sensor itself.
+
+    The units for the gain values are linear, with the default value
+    representing a gain of exactly 1.

Add an example here to clarify this.

Regards,

Hans

Quoted 1 line