Skip to contentSkip to conversation

Minimal libv4l2 support for complex cameras

PATCH In review Pavel Machek 4 participants 9 messages Mar 23, 2020 – Apr 14, 2020 Media (V4L/DVB)

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Minimal libv4l2 support for complex cameras

Hi!

We now have easy-to-install support for complex camera in form of Maemo Leste on N900.... Unfortunately we don't have anything in userspace that can be used to work with the camera.

This attempts to be minimal solution to get libv4l2 to work.

It enables passing camera pipeline description to libv4l2, so that controls are mapped to appopriate devices, and libv4l2 can be used with complex camera.

It is useful for testing kernel parts of N900 subsystem, and it will serve as a basis for autofocus work and autogain improvements.

lib/include/libv4l2.h +4 -0
index ea1870db0..a0ec0a91e 100644
@@ -58,6 +58,10 @@ LIBV4L_PUBLIC extern FILE *v4l2_log_file;
    invalid memory address will not lead to failure with errno being EFAULT,
    as it would with a real ioctl, but will cause libv4l2 to break, and you
    get to keep both pieces.
+
+   You can open complex pipelines by passing ".cv" file with pipeline
+   description to v4l2_open(). libv4l2 will open all the required
+   devices automatically in that case.
 */
 
 LIBV4L_PUBLIC int v4l2_open(const char *file, int oflag, ...);
lib/libv4l2/libv4l2-priv.h +17 -0
index 1924c91f6..1ee697a62 100644
@@ -104,6 +104,7 @@ struct v4l2_dev_info {
 	void *plugin_library;
 	void *dev_ops_priv;
 	const struct libv4l_dev_ops *dev_ops;
+	struct v4l2_controls_map *map;
 };
 
 /* From v4l2-plugin.c */
@@ -130,4 +131,20 @@ static inline void v4l2_plugin_cleanup(void *plugin_lib, void *plugin_priv,
 extern const char *v4l2_ioctls[];
 void v4l2_log_ioctl(unsigned long int request, void *arg, int result);
 
+
+struct v4l2_control_map {
+	unsigned long control;
+	int fd;
+};
+
+struct v4l2_controls_map {
+	int main_fd;
+	int num_fds;
+	int num_controls;
+	struct v4l2_control_map map[];
+};
+
+int v4l2_open_pipeline(struct v4l2_controls_map *map, int v4l2_flags);
+LIBV4L_PUBLIC int v4l2_get_fd_for_control(int fd, unsigned long control);
+
 #endif
lib/libv4l2/libv4l2.c +255 -4

317 lines not on this page · raw message

lib/libv4lconvert/control/libv4lcontrol.c +9 -3
index 0b0a346e4..8d2516126 100644
@@ -865,6 +865,7 @@ int v4lcontrol_vidioc_queryctrl(struct v4lcontrol_data *data, void *arg)
 	struct v4l2_queryctrl *ctrl = arg;
 	int retval;
 	uint32_t orig_id = ctrl->id;
+	int fd;
 
 	/* if we have an exact match return it */
 	for (i = 0; i < V4LCONTROL_COUNT; i++)
@@ -874,8 +875,9 @@ int v4lcontrol_vidioc_queryctrl(struct v4lcontrol_data *data, void *arg)
 			return 0;
 		}
 
+	fd = v4l2_get_fd_for_control(data->fd, ctrl->id);
 	/* find out what the kernel driver would respond. */
-	retval = data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
+	retval = data->dev_ops->ioctl(data->dev_ops_priv, fd,
 			VIDIOC_QUERYCTRL, arg);
 
 	if ((data->priv_flags & V4LCONTROL_SUPPORTS_NEXT_CTRL) &&
@@ -905,6 +907,7 @@ int v4lcontrol_vidioc_g_ctrl(struct v4lcontrol_data *data, void *arg)
 {
 	int i;
 	struct v4l2_control *ctrl = arg;
+	int fd;
 
 	for (i = 0; i < V4LCONTROL_COUNT; i++)
 		if ((data->controls & (1 << i)) &&
@@ -913,7 +916,8 @@ int v4lcontrol_vidioc_g_ctrl(struct v4lcontrol_data *data, void *arg)
 			return 0;
 		}
 
-	return data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
+	fd = v4l2_get_fd_for_control(data->fd, ctrl->id);
+	return data->dev_ops->ioctl(data->dev_ops_priv, fd,
 			VIDIOC_G_CTRL, arg);
 }
 
@@ -996,6 +1000,7 @@ int v4lcontrol_vidioc_s_ctrl(struct v4lcontrol_data *data, void *arg)
 {
 	int i;
 	struct v4l2_control *ctrl = arg;
+	int fd;
 
 	for (i = 0; i < V4LCONTROL_COUNT; i++)
 		if ((data->controls & (1 << i)) &&
@@ -1010,7 +1015,8 @@ int v4lcontrol_vidioc_s_ctrl(struct v4lcontrol_data *data, void *arg)
 			return 0;
 		}
 
-	return data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
+	fd = v4l2_get_fd_for_control(data->fd, ctrl->id);
+	return data->dev_ops->ioctl(data->dev_ops_priv, fd,
 			VIDIOC_S_CTRL, arg);
 }
 
v4l-utils-po/de.po +542 -460

2,220 lines not on this page · raw message

Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
#1 lore raw

Hi Pavel,

Thank you for the patch.

Quoted 7 lines · Pavel Machek

Hi!

We now have easy-to-install support for complex camera in form of Maemo Leste on N900.... Unfortunately we don't have anything in userspace that can be used to work with the camera.

This attempts to be minimal solution to get libv4l2 to work.

libv4l2 is mostly deprecated. How about contributing an OMAP3 ISP pipeline handler to libcamera instead ? :-)

Quoted 8 lines

It enables passing camera pipeline description to libv4l2, so that controls are mapped to appopriate devices, and libv4l2 can be used with complex camera.

It is useful for testing kernel parts of N900 subsystem, and it will serve as a basis for autofocus work and autogain improvements.

[snip]

Signature · 3 lines
Regards,

Laurent Pinchart
#2 lore raw

Hi!

Quoted 5 lines

Thank you for the patch.

Quoted 2 lines · Pavel Machek

Hi!

Quoted 8 lines
Quoted 5 lines

We now have easy-to-install support for complex camera in form of Maemo Leste on N900.... Unfortunately we don't have anything in userspace that can be used to work with the camera.

This attempts to be minimal solution to get libv4l2 to work.

libv4l2 is mostly deprecated. How about contributing an OMAP3 ISP pipeline handler to libcamera instead ? :-)

Why should it be instead?

I need something for kernel testing, and there is ton of apps using it. Let me do this. libcamera might be future, but...

Quoted 8 lines
Quoted 8 lines

It enables passing camera pipeline description to libv4l2, so that controls are mapped to appopriate devices, and libv4l2 can be used with complex camera.

It is useful for testing kernel parts of N900 subsystem, and it will serve as a basis for autofocus work and autogain improvements.

Best regards,
Pavel

Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
#3 lore raw

Hi Pavel,

Quoted 9 lines · Pavel Machek
Quoted 9 lines · Pavel Machek
Quoted 7 lines

Hi!

We now have easy-to-install support for complex camera in form of Maemo Leste on N900.... Unfortunately we don't have anything in userspace that can be used to work with the camera.

This attempts to be minimal solution to get libv4l2 to work.

Quoted 7 lines
Quoted 2 lines

libv4l2 is mostly deprecated. How about contributing an OMAP3 ISP pipeline handler to libcamera instead ? :-)

Why should it be instead?

I need something for kernel testing, and there is ton of apps using it. Let me do this. libcamera might be future, but...

Sure, if it's useful for you, I won't prevent you from developing any code you want :-) But there's very little chance of getting it merged, and it would be useful to more people to have a support for that platform in libcamera. It's really your decision, and I'm not blaming you.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

It enables passing camera pipeline description to libv4l2, so that controls are mapped to appopriate devices, and libv4l2 can be used with complex camera.

It is useful for testing kernel parts of N900 subsystem, and it will serve as a basis for autofocus work and autogain improvements.

Signature · 3 lines
Regards,

Laurent Pinchart
#4 lore raw

Em Mon, 23 Mar 2020 14:24:42 +0200
Laurent Pinchart <[email protected]> escreveu:

Quoted 13 lines

Hi Pavel,

Quoted 10 lines · Pavel Machek
Quoted 10 lines · Pavel Machek
Quoted 7 lines

Hi!

We now have easy-to-install support for complex camera in form of Maemo Leste on N900.... Unfortunately we don't have anything in userspace that can be used to work with the camera.

This attempts to be minimal solution to get libv4l2 to work.

libv4l2 is mostly deprecated.

Well, not really... I guess lots of userspace apps rely on it (qv4l2, xawtv, tvtime, camorama, zbar, ...).

In order to be able to deprecate it, we need to add some code there to let them bind via libcamera and test them with different hardware, including the non-UVC ones.

Quoted 5 lines
Quoted 5 lines
Quoted 2 lines

How about contributing an OMAP3 ISP
pipeline handler to libcamera instead ? :-)

Why should it be instead?

Quoted 8 lines
Quoted 2 lines

I need something for kernel testing, and there is ton of apps using it. Let me do this. libcamera might be future, but...

Sure, if it's useful for you, I won't prevent you from developing any code you want :-) But there's very little chance of getting it merged, and it would be useful to more people to have a support for that platform in libcamera. It's really your decision, and I'm not blaming you.

I created some time ago a fork of v4l-utils in order to be able to merge the N900 work from Pavel. We can add the N900 work there - or on a separate branch at the main v4l-utils tree.

Thanks,
Mauro

#5 lore raw

Hi Mauro,

Quoted 11 lines · Mauro Carvalho Chehab

Em Mon, 23 Mar 2020 14:24:42 +0200 Laurent Pinchart escreveu:

Quoted 10 lines · Pavel Machek
Quoted 9 lines
Quoted 9 lines · Pavel Machek
Quoted 7 lines

Hi!

We now have easy-to-install support for complex camera in form of Maemo Leste on N900.... Unfortunately we don't have anything in userspace that can be used to work with the camera.

This attempts to be minimal solution to get libv4l2 to work.

Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line

libv4l2 is mostly deprecated.

Well, not really... I guess lots of userspace apps rely on it (qv4l2, xawtv, tvtime, camorama, zbar, ...).

In order to be able to deprecate it, we need to add some code there to let them bind via libcamera and test them with different hardware, including the non-UVC ones.

qv4l2 is out of scope, as that's a V4L2 test application, so it really needs to stay as close as possible to V4L2 :-) TV applications are also not candidates for porting to libcamera, as libcamera doesn't support TV capture. Sure, they can use non-TV capture devices, but it's really a completely different world. I don't mind if someone wants to try and integrate libcamera support in the above applications, but I'd also be happy to let them as they are and move on.

I'm not calling for libv4l2 to be removed, but new developments shouldn't use it. We've been telling developers to go for the V4L2 API directly instead unless some libv4l2 features are mandatory for their use case (such as support for the custom compression formats of those ancient webcams). It makes no sense, from a community point of view, to invest in that project to support complex cameras.

Quoted 14 lines
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines

How about contributing an OMAP3 ISP
pipeline handler to libcamera instead ? :-)

Why should it be instead?

Quoted 8 lines
Quoted 2 lines

I need something for kernel testing, and there is ton of apps using it. Let me do this. libcamera might be future, but...

Sure, if it's useful for you, I won't prevent you from developing any code you want :-) But there's very little chance of getting it merged, and it would be useful to more people to have a support for that platform in libcamera. It's really your decision, and I'm not blaming you.

Quoted 3 lines

I created some time ago a fork of v4l-utils in order to be able to merge the N900 work from Pavel. We can add the N900 work there - or on a separate branch at the main v4l-utils tree.

It's open source, forks are encouraged. I believe it would be a waste of time, but it's not my time, so I don't mind :-)

Signature · 3 lines
Regards,

Laurent Pinchart
#6 lore raw
In reply to Laurent Pinchart

Hi!

Quoted 15 lines · Pavel Machek
Quoted 14 lines
Quoted 11 lines · Pavel Machek
Quoted 7 lines

Hi!

We now have easy-to-install support for complex camera in form of Maemo Leste on N900.... Unfortunately we don't have anything in userspace that can be used to work with the camera.

This attempts to be minimal solution to get libv4l2 to work.

libv4l2 is mostly deprecated. How about contributing an OMAP3 ISP pipeline handler to libcamera instead ? :-)

Why should it be instead?

Quoted 8 lines
Quoted 2 lines

I need something for kernel testing, and there is ton of apps using it. Let me do this. libcamera might be future, but...

Sure, if it's useful for you, I won't prevent you from developing any code you want :-) But there's very little chance of getting it merged, and it would be useful to more people to have a support for that platform in libcamera. It's really your decision, and I'm not blaming you.

When you have libcamera ready, you'll still need some hardware and kernel to work with it. Do you have something more suitable than N9/N900? Droid 4 has separate CPU to run the camera, Librem 5 camera does not have autofocus (and I believe nor has PinePhone).

My patch is not complex, and libv4l2 is full of similar hardware support code. What would be reason not to merge it?

Best regards,
Pavel

Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
#7 lore raw

Hi,

Quoted 24 lines · Pavel Machek
Quoted 15 lines · Pavel Machek
Quoted 14 lines
Quoted 11 lines · Pavel Machek
Quoted 7 lines

Hi!

We now have easy-to-install support for complex camera in form of Maemo Leste on N900.... Unfortunately we don't have anything in userspace that can be used to work with the camera.

This attempts to be minimal solution to get libv4l2 to work.

libv4l2 is mostly deprecated. How about contributing an OMAP3 ISP pipeline handler to libcamera instead ? :-)

Why should it be instead?

Quoted 8 lines
Quoted 2 lines

I need something for kernel testing, and there is ton of apps using it. Let me do this. libcamera might be future, but...

Sure, if it's useful for you, I won't prevent you from developing any code you want :-) But there's very little chance of getting it merged, and it would be useful to more people to have a support for that platform in libcamera. It's really your decision, and I'm not blaming you.

Quoted 4 lines

When you have libcamera ready, you'll still need some hardware and kernel to work with it. Do you have something more suitable than N9/N900? Droid 4 has separate CPU to run the camera, Librem 5 camera does not have autofocus (and I believe nor has PinePhone).

OMAP3 is great as a test bed for camera opinion. It's widely available, has amazing documentation, and even though the hardware is aging a little bit, it's still very decent and has a nice ISP. Even better, there's good kernel support for it, so it's a really good platform in my opinion. The only reason we don't support it yet in libcamera is lack of spare time, I would really love to see that happening, and it would be first-class citizen in the code base.

Quoted 2 lines

My patch is not complex, and libv4l2 is full of similar hardware support code. What would be reason not to merge it?

It has to be reviewed and maintained. From a V4L2 community point of view, I don't think it makes sense to invest time on a dying component (I'm talking about libv4l2, not OMAP3). But from a personal point of view, if you find libv4l2 useful to develop code for the OMAP3 and experiment, I see no issue with that. If other reviewers and maintainers of libv4l2 disagree and want to invest their time in this, I won't stop them, but I wanted to warn that this is a dying project.

Signature · 3 lines
Regards,

Laurent Pinchart
#8 lore raw
Quoted 43 lines · Laurent Pinchart

Hi,

Quoted 24 lines · Pavel Machek
Quoted 15 lines · Pavel Machek
Quoted 14 lines
Quoted 11 lines · Pavel Machek
Quoted 7 lines

Hi!

We now have easy-to-install support for complex camera in form of Maemo Leste on N900.... Unfortunately we don't have anything in userspace that can be used to work with the camera.

This attempts to be minimal solution to get libv4l2 to work.

libv4l2 is mostly deprecated. How about contributing an OMAP3 ISP pipeline handler to libcamera instead ? :-)

Why should it be instead?

Quoted 8 lines
Quoted 2 lines

I need something for kernel testing, and there is ton of apps using it. Let me do this. libcamera might be future, but...

Sure, if it's useful for you, I won't prevent you from developing any code you want :-) But there's very little chance of getting it merged, and it would be useful to more people to have a support for that platform in libcamera. It's really your decision, and I'm not blaming you.

Quoted 4 lines

When you have libcamera ready, you'll still need some hardware and kernel to work with it. Do you have something more suitable than N9/N900? Droid 4 has separate CPU to run the camera, Librem 5 camera does not have autofocus (and I believe nor has PinePhone).

OMAP3 is great as a test bed for camera opinion. It's widely available, has amazing documentation, and even though the hardware is aging a little bit, it's still very decent and has a nice ISP. Even better, there's good kernel support for it, so it's a really good platform in my opinion. The only reason we don't support it yet in libcamera is lack of spare time, I would really love to see that happening, and it would be first-class citizen in the code base.

Quoted 2 lines

My patch is not complex, and libv4l2 is full of similar hardware support code. What would be reason not to merge it?

Quoted 8 lines

It has to be reviewed and maintained. From a V4L2 community point of view, I don't think it makes sense to invest time on a dying component (I'm talking about libv4l2, not OMAP3). But from a personal point of view, if you find libv4l2 useful to develop code for the OMAP3 and experiment, I see no issue with that. If other reviewers and maintainers of libv4l2 disagree and want to invest their time in this, I won't stop them, but I wanted to warn that this is a dying project.

It's not so much the reviewing/maintaining part that is a concern for me, it is the fact that this would create yet another way to support complex cameras, when we made the decision to use libcamera for that instead.

I don't want two APIs, and vendors certainly don't want two APIs.

I agree with Laurent that adding OMAP3 support to libcamera would be much more useful than adding a libv4l hack.

Regards,

Hans