RFC: API to query webcams for various webcam specific properties

Hi All,

With libv4l giving us the ability to do some much needed image conversion in userspace, it has become clear that for (some) webcams more processing then just format conversion is necessary.

So far I've been keeping various proposed patches for doing things like software white balance correction out of libv4l as I first want a proper API for drivers to signal they need this to libv4l.

Part of the problem is that various cams needs various additional processing steps for best results, and currently there is no way to ask a driver which additional steps should be done (and using which values). Another part is that we do not have a complete picture of all possible existing processing steps we want to do, so what ever we come up with needs to be extensible.

To give an idea, here are a few things which libv4l should know about an video input source:
-does this cam need software whitebalance
-does this cam need software auto exposure
-does this cam need gamma correction, and what initial gamma to use
-if the sensor is mounted upside down, and the hardware cannot flip the image
itself

This has been discussed at the plumbers conference, and there the solution we came up with for "does this cam need software whitebalance?" was (AFAIK), check if has a V4L2_CID_AUTO_WHITE_BALANCE, if it does not do software whitebalance. This of course means we will be doing software whitebalance on things like framefrabbers etc. too, so the plan was to combine this with an "is_webcam" flag in the capabilities struct. The is_webcam workaround, already shows what is wrong with this approach, we are checking for something not being there, were we should be checking for the driver asking something actively,

So we need an extensible mechanism to query devices if they could benefit from certain additional processing being done on the generated image data.

This sounds a lot like the existing mechanism for v4l2 controls, except that these are all read only controls, and not controls which we want to show up in v4l control panels like v4l2ucp.

Still I think that using the existing controls mechanism is the best way todo this, so therefor I propose to add a number of standard CID's to query the things listed above. All these CID's will always be shown by the driver as readonly and disabled (as they are not really controls).

Here is an initial proposal for the new CID's, I'm sure the list will grow this is just a first revision:

#define V4L2_CTRL_CLASS_CAMERA_PROPERTY 0x009b0000

#define V4L2_CID_CAMERA_PROPERTY_CLASS_BASE \
	(V4L2_CTRL_CLASS_CAMERA_PROPERTY | 0x900)
#define V4L2_CID_CAMERA_PROPERTY_CLASS \
	(V4L2_CTRL_CLASS_CAMERA_PROPERTY | 1)

/* Booleans */
#define V4L2_CID_WANTS_SW_WHITEBALANCE (V4L2_CID_CAMERA_PROPERTY_CLASS_BASE+1) #define V4L2_CID_WANTS_SW_AUTO_EXPOSURE (V4L2_CID_CAMERA_PROPERTY_CLASS_BASE+2) #define V4L2_CID_WANTS_SW_GAMMA_CORRECT (V4L2_CID_CAMERA_PROPERTY_CLASS_BASE+3)
#define V4L2_CID_SENSOR_UPSIDE_DOWN (V4L2_CID_CAMERA_PROPERTY_CLASS_BASE+4)

/* Fixed point, 16.16 stored in 32 bit integer */
#define V4L2_CID_DEF_GAMMA_CORR_FACTOR (V4L2_CID_CAMERA_PROPERTY_CLASS_BASE+5)

Please let me know what you think of this proposal, as I would like to move forward with this soon.

Regards,

Hans

--
video4linux-list mailing list
Unsubscribe mailto:[email protected]?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

<snip>

There also seem to be so many things we might want to control that such an inference based system is going to hit other limitations.

I can see this leading to a lot of drivers having to implement a whole bunch of cases in a switch statement to handle these values. Could a simpler approach be to have a single ioctl to query the set of controls the driver would like to have implemented and the driver then responds with the list of tags and default values for the controls it would like implemented.

Someting like:

struct tag
{
u32 tag_id;
u32 tag_value;
};

const tag default_tags[] = { {LIBV4L_CTL_GAMMA, 0x34}, {LIBV4L_CTL_LRFLIP,1} };

This could also be a mechanism to address your other RFC as to how to store the current settings. The fact that you are already adding code to the kernel to provide the list of controls somewhat argues against your point that you don't want to add code to the kernel to store the current control settings. The driver could therefore copy the default control values into somewhere in it's device struct to provide a per device instance volatile storage for the data.

The reason I prefer in driver storage is that it simplifies the task of associating the data with the device. If you have a machine with multiple webcams they need to have independent sets of controls per device and you shouldn't retain the previous values if the user unplugs one webcam and plugs in another that gets the same /dev/videox name.

If you do use shared memory have you considered wheter to use the SysV or Posix variant? Both variants provide the required retain while not in use functionality but have different naming rules.

Is it necessary to provide a mechanism to notify other libv4l instances that the set values have changed? With driver stored values I think it is but if you use shared memory they could simply be read each time a frame is received.

Adam

--
video4linux-list mailing list
Unsubscribe mailto:[email protected]?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Yes, implementing v4l2 controls in drivers can take up quite a bit of code. This is a general problem though, not limited to my proposal for new CID's for the device/driver to be able to give hints to userspace that certain post-processing would be good todo.

We really need to add some framework code to the v4l2 core to make handling controls at the driver level easier.

What I envision is at the end having something like you propose above in the driver, and then have the v4l2 core do the translating to / from v4l2 control calls.

Yes, the idea is to simply read the values each frame, which is why shared memory is prefered as a solution to this. Your worries can be solved by simply putting the "card" member of the v4l2_capability struct + the minor no in the shared memory segment name / id.

Regards,

Hans

--
video4linux-list mailing list
Unsubscribe mailto:[email protected]?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list