V4L2 Integer (?) menu control
Hi,
I was wondering how to implement in v4l2 a standard menu control having integer values as the menu items. The menu item values would be irregular, e.g. ascending logarithmically and thus the step value would not be a constant.
I'm not interested in private control and symbolic enumeration for each value at the series. It should be a standard control where drivers could define an array of integers reflecting the control menu items. And then the applications could enumerate what integer values are valid and can be happily applied to a device.
I don't seem to find a way to implement this in current v4l2 control framework. Such functionality isn't there, or is it ?
Signature · 2 lines
Regards, Sylwester Nawrocki
On Wed, 23 Nov 2011 23:26:22 +0100, Sylwester Nawrocki <[email protected]>
Quoted 3 lines · wrote:
I don't seem to find a way to implement this in current v4l2 control
framework. Such functionality isn't there, or is it ?
You can use the menu control type, but you will need to remap the control
values so they are continuous.
Signature · 3 lines
Rémi Denis-Courmont http://www.remlab.net/
Quoted 10 lines · On Wednesday, November 23, 2011 23:26:22 Sylwester Nawrocki wrote:
Hi,
I was wondering how to implement in v4l2 a standard menu control having integer values as the menu items. The menu item values would be irregular, e.g. ascending logarithmically and thus the step value would not be a constant.
I'm not interested in private control and symbolic enumeration for each value at the series. It should be a standard control where drivers could define an array of integers reflecting the control menu items. And then the applications could enumerate what integer values are valid and can be happily applied to a device.
Quoted 2 lines
I don't seem to find a way to implement this in current v4l2 control framework. Such functionality isn't there, or is it ?
No it doesn't exist.
I'd have sworn that I saw a proposal for adding something like that on the mailinglist some time ago, but I can't find it anymore. It wouldn't be difficult to add.
Regards,
Hans
Quoted 10 lines · Sylwester Nawrocki
Hi,
I was wondering how to implement in v4l2 a standard menu control having integer values as the menu items. The menu item values would be irregular, e.g. ascending logarithmically and thus the step value would not be a constant.
I'm not interested in private control and symbolic enumeration for each value at the series. It should be a standard control where drivers could define an array of integers reflecting the control menu items. And then the applications could enumerate what integer values are valid and can be happily applied to a device.
Quoted 2 lines
I don't seem to find a way to implement this in current v4l2 control framework. Such functionality isn't there, or is it ?
Hi Sylwester,
There is not currently, but I have patches for it. The issue is that I need them myself but the driver I need them for isn't ready to be released yet. And as usual, I assume others than vivo is required to show they're really useful so I haven't sent them.
Good that you asked so we won't end up writing essentially the same code again. I'll try to send the patches today.
Kind regards,
Signature · 2 lines
Sakari Ailus e-mail: [email protected] jabber/XMPP/Gmail: [email protected]
Quoted 7 lines · Rémi Denis-Courmont
Quoted 2 lines · Sylwester Nawrocki
I don't seem to find a way to implement this in current v4l2 control framework. Such functionality isn't there, or is it ?
You can use the menu control type, but you will need to remap the control values so they are continuous.
Yes, but what I'm missing is a method for the drivers to inform the application how the mapping looks like. Something like custom queryctrl for standard control CID.
So for instance if we have a standard control V4L2_CID_CAMERA_ISO, two devices could support different series of values, e.g.
50, 200, 400, 800, ..
100, 180, 300, 600, ...
Currently the menu items are hard coded in the kernel for standard controls, and AFAIU we can only query the control names.
In fact the continuous enumeration in the driver might do, which would be then mapped to register values. But the meaning of this values need to be made known to the applications.
--
Thanks,
Sylwester
Thank you all for the comments.
Quoted 6 lines · Sakari Ailus
Hi Sylwester,
There is not currently, but I have patches for it. The issue is that I need them myself but the driver I need them for isn't ready to be released yet. And as usual, I assume others than vivo is required to show they're really useful so I haven't sent them.
That's great news. Then I might not need to do all the work on my own;)
Quoted 3 lines
Good that you asked so we won't end up writing essentially the same code again. I'll try to send the patches today.
All right, there is no rush. I was just looking around how to support the camera scene mode with m5mols sort of sensors. The scene mode is essentially a compilation of several different parameters, for some of which there are standard controls in V4L2 but for many there are not.
I've got a feeling the best way to handle this would be to create controls for each single parameter and then do a batch set from user space, and keep the scene mode mappings in user space. The only concern is there is a couple of ISP-specific parameters involved with that scene mode thing. Perhaps they just could be set initially to fixed values.
Signature · 2 lines
Regards, Sylwester
Hi Sylwester,
Quoted 11 lines · Sylwester Nawrocki
Thank you all for the comments.
Quoted 6 lines · Sakari Ailus
Hi Sylwester,
There is not currently, but I have patches for it. The issue is that I need them myself but the driver I need them for isn't ready to be released yet. And as usual, I assume others than vivo is required to show they're really useful so I haven't sent them.
That's great news. Then I might not need to do all the work on my own;)
I hope mine will do. ;-)
I'm working on 2.6.32 kernel (ouch!) so I haven't been able to test them properly yet. Please provide feedback on them if you find any issues.
Quoted 8 lines
Quoted 3 lines
Good that you asked so we won't end up writing essentially the same code again. I'll try to send the patches today.
All right, there is no rush. I was just looking around how to support the camera scene mode with m5mols sort of sensors. The scene mode is essentially a compilation of several different parameters, for some of which there are standard controls in V4L2 but for many there are not.
I fully agree with this approach. Scene modes should not be implemented at the level of the V4L2 API. Instead, the parameters that the scene modes consist of must be shown separately on the V4L2 API, if that is the level of API they belong to. Depending on your camera stack control algorithms could reside in the user space, which I believe is however not the case with the M5-MOLS.
Quoted 5 lines
I've got a feeling the best way to handle this would be to create controls for each single parameter and then do a batch set from user space, and keep the scene mode mappings in user space. The only concern is there is a couple of ISP-specific parameters involved with that scene mode thing. Perhaps they just could be set initially to fixed values.
Can you describe what kind of parameters this is about? Is there an issue in just setting those using the ISP driver V4L2 subdev API?
This makes your user space to depend both on the sensor and the ISP, but there's really no way around that if both do non-trivial hardware-specific things.
I think we need to further standardise image processing configuration such as RGB-to-RGB matrices and gamma tables. This would make the ISP interfaces less hardware specific.
Kind regards,
Signature · 2 lines
Sakari Ailus [email protected]
Hi Sakari,
Quoted 24 lines · Sakari Ailus
Quoted 8 lines · Sylwester Nawrocki
Quoted 5 lines · Sakari Ailus
There is not currently, but I have patches for it. The issue is that I need them myself but the driver I need them for isn't ready to be released yet. And as usual, I assume others than vivo is required to show they're really useful so I haven't sent them.
That's great news. Then I might not need to do all the work on my own;)
I hope mine will do. ;-)
I'm working on 2.6.32 kernel (ouch!) so I haven't been able to test them properly yet. Please provide feedback on them if you find any issues.
Quoted 8 lines
Quoted 3 lines
Good that you asked so we won't end up writing essentially the same code again. I'll try to send the patches today.
All right, there is no rush. I was just looking around how to support the camera scene mode with m5mols sort of sensors. The scene mode is essentially a compilation of several different parameters, for some of which there are standard controls in V4L2 but for many there are not.
Quoted 5 lines
I fully agree with this approach. Scene modes should not be implemented at the level of the V4L2 API. Instead, the parameters that the scene modes consist of must be shown separately on the V4L2 API, if that is the level of API they belong to. Depending on your camera stack control algorithms could reside in the user space, which I believe is however not the case with the M5-MOLS.
No, with these hybrid camera devices the algorithms are built in their own ISP. And there is quite many advanced algorithms, e.g. auto focus/face detection that are difficult to control at the subdevice API level.
Quoted 9 lines
Quoted 6 lines
I've got a feeling the best way to handle this would be to create controls for each single parameter and then do a batch set from user space, and keep the scene mode mappings in user space. The only concern is there is a couple of ISP-specific parameters involved with that scene mode thing. Perhaps they just could be set initially to fixed values.
Can you describe what kind of parameters this is about? Is there an issue in just setting those using the ISP driver V4L2 subdev API?
Please see struct m5mols_scenemode in file m5mols.h
for a brief overview. I have also been preparing a list of the parameters and their exact meaning, but it's not ready yet.
The issue is that the subdev API seems to low level for the device but it's the only API available at the user space ;)
Quoted 3 lines
This makes your user space to depend both on the sensor and the ISP, but there's really no way around that if both do non-trivial hardware-specific things.
I guess a dedicated library for the sensor itself is needed on top of subdevice API to be able to use advanced features. And even then subdevice/V4L2 API is a limitation.
Quoted 4 lines
I think we need to further standardise image processing configuration such as RGB-to-RGB matrices and gamma tables. This would make the ISP interfaces less hardware specific.
I guess first we need at least one more OMAP3 ISP like device driver in mainline to identify common features and design APIs for them. On the other hand gamma tables are also present in some embedded ISPs, e.g. in s5k6aafx IIRC.
Signature · 2 lines
Regards, Sylwester
Hi Sylwester,
Quoted 31 lines · Sylwester Nawrocki
Quoted 24 lines · Sakari Ailus
Quoted 8 lines · Sylwester Nawrocki
Quoted 5 lines · Sakari Ailus
There is not currently, but I have patches for it. The issue is that I need them myself but the driver I need them for isn't ready to be released yet. And as usual, I assume others than vivo is required to show they're really useful so I haven't sent them.
That's great news. Then I might not need to do all the work on my own;)
I hope mine will do. ;-)
I'm working on 2.6.32 kernel (ouch!) so I haven't been able to test them properly yet. Please provide feedback on them if you find any issues.
Quoted 8 lines
Quoted 3 lines
Good that you asked so we won't end up writing essentially the same code again. I'll try to send the patches today.
All right, there is no rush. I was just looking around how to support the camera scene mode with m5mols sort of sensors. The scene mode is essentially a compilation of several different parameters, for some of which there are standard controls in V4L2 but for many there are not.
Quoted 5 lines
I fully agree with this approach. Scene modes should not be implemented at the level of the V4L2 API. Instead, the parameters that the scene modes consist of must be shown separately on the V4L2 API, if that is the level of API they belong to. Depending on your camera stack control algorithms could reside in the user space, which I believe is however not the case with the M5-MOLS.
Quoted 3 lines
No, with these hybrid camera devices the algorithms are built in their own ISP. And there is quite many advanced algorithms, e.g. auto focus/face detection that are difficult to control at the subdevice API level.
Can you tell what makes it difficult?
Quoted 2 lines
The issue is that the subdev API seems to low level for the device but it's the only API available at the user space ;)
...
Quoted 5 lines
Quoted 2 lines
This makes your user space to depend both on the sensor and the ISP, but there's really no way around that if both do non-trivial hardware-specific things.
I guess a dedicated library for the sensor itself is needed on top of subdevice API to be able to use advanced features. And even then subdevice/V4L2 API is a limitation.
How is it a limitation?
Whe whole intent is to provide as standard as possible way to access the hardware features through an interface provided by the driver. So what is missing in your opinion? :-)
Quoted 7 lines
Quoted 3 lines
I think we need to further standardise image processing configuration such as RGB-to-RGB matrices and gamma tables. This would make the ISP interfaces less hardware specific.
I guess first we need at least one more OMAP3 ISP like device driver in mainline to identify common features and design APIs for them. On the other hand gamma tables are also present in some embedded ISPs, e.g. in s5k6aafx IIRC.
Or get more public specs for different ISPs. Or just read the existing specs more. ;-) The OMAP 4 ISS spec is public. Even if it's from TI as well it's very different from the OMAP 3 ISP.
Signature · 2 lines
Sakari Ailus e-mail: [email protected] jabber/XMPP/Gmail: [email protected]
Hi Sakari,
Quoted 10 lines · Sakari Ailus
Quoted 9 lines · Sylwester Nawrocki
Quoted 8 lines · Sakari Ailus
Quoted 7 lines · Sylwester Nawrocki
Quoted 5 lines · Sakari Ailus
There is not currently, but I have patches for it. The issue is that I need them myself but the driver I need them for isn't ready to be released yet. And as usual, I assume others than vivo is required to show they're really useful so I haven't sent them.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
That's great news. Then I might not need to do all the work on my own;)
I hope mine will do. ;-)
I'm working on 2.6.32 kernel (ouch!) so I haven't been able to test them properly yet. Please provide feedback on them if you find any issues.
Sure, I just need to reserve some time to try these things out.
Quoted 16 lines
Quoted 16 lines
Quoted 15 lines
Quoted 9 lines
Quoted 4 lines
Good that you asked so we won't end up writing essentially the same code again. I'll try to send the patches today.
All right, there is no rush. I was just looking around how to support the camera scene mode with m5mols sort of sensors. The scene mode is essentially a compilation of several different parameters, for some of which there are standard controls in V4L2 but for many there are not.
I fully agree with this approach. Scene modes should not be implemented at the level of the V4L2 API. Instead, the parameters that the scene modes consist of must be shown separately on the V4L2 API, if that is the level of API they belong to. Depending on your camera stack control algorithms could reside in the user space, which I believe is however not the case with the M5-MOLS.
Quoted 5 lines
Quoted 3 lines
No, with these hybrid camera devices the algorithms are built in their own ISP. And there is quite many advanced algorithms, e.g. auto focus/face detection that are difficult to control at the subdevice API level.
Can you tell what makes it difficult?
I think the problem boils down to the following issues:
1) lack of controls for several parameters at the camera control class, e.g.
- V4L2_CID_METERING_MODE (menu),
- V4L2_CID_AUTO_EXPOSURE_BIAS (integer),
- V4L2_CID_ISO (integer menu),
- various auto focus modes,
- capture mode: single, multi, auto bracketing, anti hand-shake, etc.
2) separate H/W contexts (image format, some controls) for viewfinder, snapshot,
preview; introducing the SNAPSHOT control could partially resolve this;
3) proper support for encoded data on the media bus
4) some sensors with embedded ISPs provide features which might be considered "high level", like for example scene mode (white balance) presets, exposure bracketing, drawing some additional markers in viewfinder video stream, e.g. a face detection rectangle, etc.
That's just what comes to my mind right now. 1), 2), 3) shouldn't be difficult to solve, only 4) seems to be a real issue.
I was planning to prepare an RFC on controls listed in 1). Some of them would need your recent integer menu control patches.
Quoted 9 lines
Quoted 3 lines
The issue is that the subdev API seems to low level for the device but it's the only API available at the user space ;)
...
Quoted 3 lines
Quoted 2 lines
This makes your user space to depend both on the sensor and the ISP, but there's really no way around that if both do non-trivial hardware-specific things.
Quoted 8 lines
Quoted 2 lines
I guess a dedicated library for the sensor itself is needed on top of subdevice API to be able to use advanced features. And even then subdevice/V4L2 API is a limitation.
How is it a limitation?
Whe whole intent is to provide as standard as possible way to access the hardware features through an interface provided by the driver. So what is missing in your opinion? :-)
What I tried to say was that there might different logic implemented at each hybrid sensor/ISP device. Some device specific sequences might be needed, which might not be possible to abstract in a v4l2 subdevice driver itself. With regards to what is missing please see above.
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
I think we need to further standardise image processing configuration such as RGB-to-RGB matrices and gamma tables. This would make the ISP interfaces less hardware specific.
Quoted 7 lines
Quoted 3 lines
I guess first we need at least one more OMAP3 ISP like device driver in mainline to identify common features and design APIs for them. On the other hand gamma tables are also present in some embedded ISPs, e.g. in s5k6aafx IIRC.
Or get more public specs for different ISPs. Or just read the existing specs more. ;-) The OMAP 4 ISS spec is public. Even if it's from TI as well it's very different from the OMAP 3 ISP.
Right, a very valid point:) I just had a look at the OMAP4 ISS, thanks for the hint! I'll try to obtain more detailed documentation on similar Samsung devices, unfortunately as you know it still isn't public.
Signature · 2 lines
Best regards, Sylwester Nawrocki
Hi Sylwester,
Quoted 44 lines · Sylwester Nawrocki
Hi Sakari,
Quoted 10 lines · Sakari Ailus
Quoted 9 lines · Sylwester Nawrocki
Quoted 8 lines · Sakari Ailus
Quoted 7 lines · Sylwester Nawrocki
Quoted 5 lines · Sakari Ailus
There is not currently, but I have patches for it. The issue is that I need them myself but the driver I need them for isn't ready to be released yet. And as usual, I assume others than vivo is required to show they're really useful so I haven't sent them.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
That's great news. Then I might not need to do all the work on my own;)
I hope mine will do. ;-)
I'm working on 2.6.32 kernel (ouch!) so I haven't been able to test them properly yet. Please provide feedback on them if you find any issues.
Sure, I just need to reserve some time to try these things out.
Quoted 16 lines
Quoted 16 lines
Quoted 15 lines
Quoted 9 lines
Quoted 4 lines
Good that you asked so we won't end up writing essentially the same code again. I'll try to send the patches today.
All right, there is no rush. I was just looking around how to support the camera scene mode with m5mols sort of sensors. The scene mode is essentially a compilation of several different parameters, for some of which there are standard controls in V4L2 but for many there are not.
I fully agree with this approach. Scene modes should not be implemented at the level of the V4L2 API. Instead, the parameters that the scene modes consist of must be shown separately on the V4L2 API, if that is the level of API they belong to. Depending on your camera stack control algorithms could reside in the user space, which I believe is however not the case with the M5-MOLS.
Quoted 5 lines
Quoted 3 lines
No, with these hybrid camera devices the algorithms are built in their own ISP. And there is quite many advanced algorithms, e.g. auto focus/face detection that are difficult to control at the subdevice API level.
Can you tell what makes it difficult?
Quoted 8 lines
I think the problem boils down to the following issues:
1) lack of controls for several parameters at the camera control class, e.g.
-V4L2_CID_METERING_MODE(menu),
-V4L2_CID_AUTO_EXPOSURE_BIAS(integer),
-V4L2_CID_ISO(integer menu),
- various auto focus modes,
- capture mode: single, multi, auto bracketing, anti hand-shake, etc.
These are indeed quite high level controls. You need some of this when your camera control algorithms run on the camera module itself.
I think it's just a matter of adding them to the control class, or to export them as private controls. Some autofocus modes might require a private control, for example, if they need additional information from the user.
Quoted 2 lines
2) separate H/W contexts (image format, some controls) for viewfinder, snapshot,
preview; introducing the SNAPSHOT control could partially resolve this;
This could be more tricky. Guennadi did measurements on how much time could be saved by doing this, and the answer was in his case some 30 milliseconds. The sensor he was using used archaic 20 kHz bus speed instead of the 400 kHz one standardised in the beginning of 90s.
How about M5-MOLS?
Quoted 6 lines
3) proper support for encoded data on the media bus
4) some sensors with embedded ISPs provide features which might be considered "high level", like for example scene mode (white balance) presets, exposure bracketing, drawing some additional markers in viewfinder video stream, e.g. a face detection rectangle, etc.
Much of the time this kind of features bind together many parameters and it may be best to provide a private interface to these. That's also how it often is in the higher layers --- if your functionality is something that no-one else has chances are low that you can provide a fully standardised interface for it.
Quoted 20 lines
That's just what comes to my mind right now. 1), 2), 3) shouldn't be difficult to solve, only 4) seems to be a real issue.
I was planning to prepare an RFC on controls listed in 1). Some of them would need your recent integer menu control patches.
Quoted 14 lines
Quoted 3 lines
The issue is that the subdev API seems to low level for the device but it's the only API available at the user space ;)
...
Quoted 5 lines
Quoted 2 lines
This makes your user space to depend both on the sensor and the ISP, but there's really no way around that if both do non-trivial hardware-specific things.
I guess a dedicated library for the sensor itself is needed on top of subdevice API to be able to use advanced features. And even then subdevice/V4L2 API is a limitation.
How is it a limitation?
Quoted 8 lines
Quoted 3 lines
Whe whole intent is to provide as standard as possible way to access the hardware features through an interface provided by the driver. So what is missing in your opinion? :-)
What I tried to say was that there might different logic implemented at each hybrid sensor/ISP device. Some device specific sequences might be needed, which might not be possible to abstract in a v4l2 subdevice driver itself. With regards to what is missing please see above.
Indeed. The V4L2 subdev API provides access to subdevices themselves, not the whole media device.
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
Quoted 3 lines
I think we need to further standardise image processing configuration such as RGB-to-RGB matrices and gamma tables. This would make the ISP interfaces less hardware specific.
I guess first we need at least one more OMAP3 ISP like device driver in mainline to identify common features and design APIs for them. On the other hand gamma tables are also present in some embedded ISPs, e.g. in s5k6aafx IIRC.
Quoted 7 lines
Quoted 3 lines
Or get more public specs for different ISPs. Or just read the existing specs more. ;-) The OMAP 4 ISS spec is public. Even if it's from TI as well it's very different from the OMAP 3 ISP.
Right, a very valid point:) I just had a look at the OMAP4 ISS, thanks for the hint! I'll try to obtain more detailed documentation on similar Samsung devices, unfortunately as you know it still isn't public.
That definitely would be very interesting and useful, and also good for V4L2 support for Samsung ISPs...
Kind regards,
Signature · 2 lines
Sakari Ailus [email protected]