Creating a V4L driver for a USB camera

Hi all,

I'm a senior software engineer [0] with a small startup. Our product is Linux based and makes use of a 3M pixel camera. Unfortunately, the camera we have been using for the last 3 years is no longer being produced.

We have found two candidate replacement cameras, one with a binary only driver and user space library and one with a windows driver but no Linux driver.

My questions:

- How difficult is it to create a GPL V4L driver for a USB camera by snooping the USB traffic of the device when connected to a windows machine? The intention is to merge this work into the V4L mainline and ultimately the kernel.

- How much work is involved in the above for someone experienced in writing V4L drivers?

- Are there people involved with the V4L project that would be willing to undertake this project under contract?

Obviously, answers to the last question can be send directly to me :-). The others are probably best sent to the list.

Cheers,
Erik

[0] I am also a FOSS developer. My main two projects are
http://www.mega-nerd.com/libsndfile/
http://www.mega-nerd.com/libsamplerate/
I am also involved with Xiph.org and the Debian project.

=======================
erik de castro lopo
senior design engineer

bCODE
level 2, 2a glen street
milsons point
sydney nsw 2061
australia

tel +61 (0)2 9954 4411
fax +61 (0)2 9954 4422
www.bcode.com

Hi,

2009/6/3 Erik de Castro Lopo <[email protected]>:

Do you have any datasheet available on what usb bridge / sensor that is used? If the chipsets are undocumented and some proprietary image compression technique is used, the time to reverse-engineer them can be quite lengthy.

Best regards,
Erik

Hi,

That depends mainly on the format of the image data by the cam, if the cam sends raw bayer data, or raw yuv / rgb then this is doable, if it uses plain JPEG it is also doable. If it uses some custom compression then you need a wizzkid to crack the code. I've tried this myself, and I failed, you really need someone with the right mindset to reverse engineer a compression algorithm. Merely being a good programmer is not enough.

This can vary wildly, assuming the video data format is a known one, a wild estimate would be that this takes 2 fulltime weeks (with hands on hardware access). But it could be that it takes much longer if somehow the cam is strange (or worse, like buggy).

Your welcome to send me a couple of cams, that is usually all the payment I expect, I also don't make any promises (I do this on top of my dayjob). But first things first, what are the usb-id's of the cams, can you send me (offlist) the windows drivers ? Chances are the chipset used is already supported.

Regards,

Hans

The USB device itself comes up as :

    Bus 001 Device 011: ID 0547:8031 Anchor Chips, Inc

The sensor is a Micron MT9T001P12STC and I have the data sheet for it.

I've asked the manufacturer for source code to the windows driver and docs/source/whatever for the USB interface.

I happen to know that the sensor/camera (via the windows driver) can provide raw bayer data which is what I'm after (our application is machine vision and bayer works best).

Cheers,
Erik

=======================
erik de castro lopo
senior design engineer

bCODE
level 2, 2a glen street
milsons point
sydney nsw 2061
australia

tel +61 (0)2 9954 4411
fax +61 (0)2 9954 4422
www.bcode.com

If this is the case, then it ought not to be terribly difficult to write a basic driver. If you wanted still camera support, with which I have a bit more experience than with streaming support, I would say it would take me about a week or so. The real obstacle is proprietary data compression which is usually totally undocumented, as others have already said.

Of course, I said above "basic" driver. That does not include things like color balance, contrast, or brightness controls. Such would probably take a little bit longer.

through with this project.

Theodore Kilgore

(Greetings from the monthly meeting of the East Alabama LUG)

Yep, only interested in still images ATM.

Als need contol over things like this. We have pretty good control over the lighting the camera works under so we tweak contrast/brightness/ whatever in the camera to provide the bext possible image to the image processing.

Cheers,
Erik

=======================
erik de castro lopo
senior design engineer

bCODE
level 2, 2a glen street
milsons point
sydney nsw 2061
australia

tel +61 (0)2 9954 4411
fax +61 (0)2 9954 4422
www.bcode.com

Well, if you are interested in using the camera as a still camera, then probably you ought also to send an inquiry over to

[email protected]

because that is, basically, where the still camera support is done, not here. At the same time, though, you have gotten some responses here from some pretty clever folks who have volunteered to help. So if I were you I would certainly not want to drop those offers into the sea.

The reason for the distinction between V4L type of things and Gphoto type of things is the old and time-honored rigid distinction between kernelspace and userspace in Linux. Basically, the idea is that a video device or a streaming camera (webcam) requires a kernel device to be created, usually named /dev/videoX where X is a number, and if you have just one such device hooked up then X=0, of course. But userspace applications do not require that, and a device can be communicated with through libusb, for example, or even can be communicated with directly.

So, Gphoto is a project which supports still cameras, using libusb as its infrastructure. It was conceived and intended to support proprietary cameras, which do not use any standard communication protocol. More recently, it has also evolved into supporting the cameras which use the PTP protocol. If your camera is proprietary or uses PTP and you want to get still photos off it, then Gphoto is the way to go. If it can also stream and you want _that_ supported, then here is the right place.

Many still cameras, though, are not using a proprietary or a PTP communication protocol, but instead are using Mass Storage, usually with protocol Transparent SCSI, and with Bulk Transport. Such still cameras do not in general need any particular driver software at all. Rather, one gets access to such a camera by mounting it. For, as a USB device it acts just like any USB external hard drive or flash drive and the photos on it are files ready to download.

Unfortunately, if you have mentioned what are the characteristics of your camera as a USB device, then that mention did not register on me. What might really help in getting started (whichever way this is going) is the full output of lsusb. Here, for example, is part of the output from a camera, from lsusb -v:

Bus 005 Device 002: ID 093a:010f Pixart Imaging, Inc. Argus DC-1610/DC-1620/Emprex PCD3600/Philips P44417B keychain camera/Precision Mini,Model HA513A/Vivitar Vivicam 55 Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 1.10

  bDeviceClass          255 Vendor Specific Class
  bDeviceSubClass       255 Vendor Specific Subclass
  bDeviceProtocol       255 Vendor Specific Protocol

bMaxPacketSize0 8
idVendor 0x093a Pixart Imaging, Inc.
idProduct 0x010f Argus DC-1610/DC-1620/Emprex PCD3600/Philips
P44417B keychain camera/Precision Mini,Model HA513A/Vivitar Vivicam 55
bcdDevice 1.00
iManufacturer 0
iProduct 2 USB Dual-mode Camera

Note in particular these lines

  bDeviceClass          255 Vendor Specific Class
  bDeviceSubClass       255 Vendor Specific Subclass
  bDeviceProtocol       255 Vendor Specific Protocol

which tell that, whatever else, this camera needs to have a driver provided or else one can do nothing at all with it. It does have a stillcam driver, in fact, in libgphoto2. A webcam driver for it is under development, too, over here, since the camera is "Dual-mode" and can be used both as a still camera and as a webcam.

Now, here is another camera:

Bus 005 Device 003: ID 2770:915d NHJ, Ltd Cyberpix S-210S / Little Tikes My Real Digital Camera Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 1.10

  bDeviceClass            0 (Defined at Interface level)
  bDeviceSubClass         0
  bDeviceProtocol         0
  bMaxPacketSize0         8

idVendor 0x2770 NHJ, Ltd
idProduct 0x915d Cyberpix S-210S / Little Tikes My Real Digital
Camera

So, the note says "Defined at Interface level" so we can scroll down a bit and we see

bInterfaceClass 8 Mass Storage
bInterfaceSubClass 6 SCSI
bInterfaceProtocol 80 Bulk (Zip)

so in other words, to get access to this camera one must mount it. Period. No particular additional support is needed for this second camera, and, as a matter of fact, no additional support is possible, either, neither from the kernel nor from a project like libgphoto.

So the reason for these examples is, it is this kind of information which one must use to judge the situation, and it is this kind of information which one would have to provide in order to begin to get definitive answers. This, in fact, is where one must start.

Hope this helps.

Theodore Kilgore

Well our current camera has a V4L based driver so we'd like to stick to that :-).

Ok, to the lsusb -v info:

Bus 001 Device 011: ID 0547:8031 Anchor Chips, Inc. Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 2.00
bDeviceClass 0 (Defined at Interface level)
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 64
idVendor 0x0547 Anchor Chips, Inc.
idProduct 0x8031
bcdDevice 0.00
iManufacturer 1
iProduct 2
iSerial 0
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 32
bNumInterfaces 1
bConfigurationValue 1
iConfiguration 0
bmAttributes 0x80
(Bus Powered)
MaxPower 100mA
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 2
bInterfaceClass 255 Vendor Specific Class
bInterfaceSubClass 0
bInterfaceProtocol 0
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 3
Transfer Type Interrupt
Synch Type None
Usage Type Data
wMaxPacketSize 0x0004 1x 4 bytes
bInterval 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x82 EP 2 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 0

The "Vendor Specific Class" above suggests that this camera does not behave like a proper USV video or still camera, but rather uses its own protocol (just like the camera we are replacing).

I have managed to convince the manufactuer of the fact that its a good idea to provide some information and/or windows source code, but as yet I can't predict how good that information will be.

Erik

=======================
erik de castro lopo
senior design engineer

bCODE
level 2, 2a glen street
milsons point
sydney nsw 2061
australia

tel +61 (0)2 9954 4411
fax +61 (0)2 9954 4422
www.bcode.com

As I explained, it is a matter of what one wants to do with the camera which kind of driver one wants to construct. Also it is a matter of what exactly that the particular camera will do. To take the two extreme cases:

-- if you have a camera which will not act as a webcam, a V4L driver will probably not materialize.

-- if you have a webcam which has no ability to act as a still camera, then no Gphoto driver will materialize.

And if the camera can do both of the above, then it is possible (and it occurs, too) that the camera has a Gphoto driver for its still camera functionality and it has a V4L driver for its webcam functionality. But the still camera and webcam functions are conceptually different and require distinct methodologies to support them in Linux.

In other words, the kind of software support which is required by new camera X is determined by the properties of camera X alone. That was my point.

Yes. That seems quite clear.

If the manufacturer truly provides meaningful information, that would be very good. Therefore, I restrain myself from expressing general pessimism about the prospects.

Theodore Kilgore