Skip to contentSkip to conversation

libv4l2 and the Hauppauge HVR1600 (cx18 driver) not working well together

DISCUSSION Simon Farnsworth 3 participants 20 messages Sep 2, 2009 – Sep 4, 2009 Media (V4L/DVB)

lore
Share & tools
lore
#1 lore raw
In reply to Simon Farnsworth

Hi,

Quoted 6 lines · Simon Farnsworth

Hello,

I'm in the process of reworking Xine's input_v4l to use libv4l2, so that
it gets the benefit of all the work done on modern cards and webcams,
and I've hit a stumbling block.

Quoted 9 lines

I have a Hauppauge HVR1600 for NTSC and ATSC support, and it appears to simply not work with libv4l2, due to lack of mmap support. My code works adequately (modulo a nice pile of bugs) with a HVR1110r3, so it appears to be driver level.

Which is the better route to handling this; adding code to input_v4l to use libv4lconvert when mmap isn't available, or converting the cx18 driver to use mmap?

Or modify libv4l2 to also handle devices which can only do read. There have been some changes to libv4l2 recently which would make doing that feasible.

Quoted 4 lines

If it's a case of converting the cx18 driver, how would I go about doing that? I have no experience of the driver, so I'm not sure what I'd have to do - noting that if I break the existing read() support, other users will get upset.

I don't believe that modifying the driver is the answer, we need to either fix this at the libv4l or xine level.

I wonder though, doesn't the cx18 offer any format that xine can handle directly?

As stated libv4l2 currently does not support devices that cannot do read, what this comes down to in practice (or should, if not that is a bug), is that it passes all calls directly to the driver. So if the driver has any pixfmt's xine can handle directly things should work fine.

Regards,

Hans

#0 lore raw

Hello,

I'm in the process of reworking Xine's input_v4l to use libv4l2, so that
it gets the benefit of all the work done on modern cards and webcams,
and I've hit a stumbling block.

I have a Hauppauge HVR1600 for NTSC and ATSC support, and it appears to simply not work with libv4l2, due to lack of mmap support. My code works adequately (modulo a nice pile of bugs) with a HVR1110r3, so it appears to be driver level.

Which is the better route to handling this; adding code to input_v4l to use libv4lconvert when mmap isn't available, or converting the cx18 driver to use mmap?

If it's a case of converting the cx18 driver, how would I go about doing that? I have no experience of the driver, so I'm not sure what I'd have to do - noting that if I break the existing read() support, other users will get upset.

Signature · 3 lines
Advice appreciated,

Simon Farnsworth
#2 lore raw
In reply to Hans de Goede
Quoted 8 lines · Hans de Goede

Hi,

Quoted 5 lines · Simon Farnsworth

I have a Hauppauge HVR1600 for NTSC and ATSC support, and it appears to simply not work with libv4l2, due to lack of mmap support. My code works adequately (modulo a nice pile of bugs) with a HVR1110r3, so it appears to be driver level.

Quoted 8 lines
Quoted 4 lines

Which is the better route to handling this; adding code to input_v4l to use libv4lconvert when mmap isn't available, or converting the cx18 driver to use mmap?

Or modify libv4l2 to also handle devices which can only do read. There have been some changes to libv4l2 recently which would make doing that feasible.

Roughly what would I need to do to libv4l2?

I can see four new cases to handle:

1) driver supports read(), client wants read(), format supported by both. Just pass read() through to the driver.

2) driver supports read(), client wants mmap(), format supported by both. Allocate buffers when REQBUFs happens, handle QBUF and DQBUF by read()ing frame size chunks into the buffer at the head of the internal queue when a DQBUF happens, and passing it back out.

3) As 1, but needs conversion. read() into a temporary buffer, convert with libv4lconvert (I think this needs a secondary buffer), and supply data from the secondary buffer to read()

4) As 2, but needs conversion. Change DQBUF handling to read() frame size chunks into a temporary buffer, then use libv4lconvert to copy those chunks from the temporary buffer into the buffer you're about to pass out.

Have I missed anything?

Quoted 11 lines
Quoted 4 lines

If it's a case of converting the cx18 driver, how would I go about doing that? I have no experience of the driver, so I'm not sure what I'd have to do - noting that if I break the existing read() support, other users will get upset.

I don't believe that modifying the driver is the answer, we need to either fix this at the libv4l or xine level.

I wonder though, doesn't the cx18 offer any format that xine can handle directly?

Not sensibly; it offers HM12 only, and Xine needs an RGB format, YV12, or YUYV. With a lot of rework, I could have the cx18 encode video to MPEG-2, then have Xine decode the resulting MPEG-2 stream, but this seems like overkill for uncompressed video. I could also teach Xine to handle HM12 natively, but that would duplicate effort already done in libv4l2 and libv4lconvert, which seems a bit silly to me.

Signature · 1 line
Simon Farnsworth
#3 lore raw

Hi,

Quoted 17 lines · Simon Farnsworth
Quoted 8 lines · Hans de Goede

Hi,

Quoted 5 lines · Simon Farnsworth

I have a Hauppauge HVR1600 for NTSC and ATSC support, and it appears to simply not work with libv4l2, due to lack of mmap support. My code works adequately (modulo a nice pile of bugs) with a HVR1110r3, so it appears to be driver level.

Quoted 8 lines
Quoted 4 lines

Which is the better route to handling this; adding code to input_v4l to use libv4lconvert when mmap isn't available, or converting the cx18 driver to use mmap?

Or modify libv4l2 to also handle devices which can only do read. There have been some changes to libv4l2 recently which would make doing that feasible.

Quoted 7 lines

Roughly what would I need to do to libv4l2?

I can see four new cases to handle:

1) driver supports read(), client wants read(), format supported by both. Just pass read() through to the driver.

This is already handled

Quoted 5 lines

2) driver supports read(), client wants mmap(), format supported by both. Allocate buffers when REQBUFs happens, handle QBUF and DQBUF by read()ing frame size chunks into the buffer at the head of the internal queue when a DQBUF happens, and passing it back out.

This is going to be a case of though luck for the client.

Quoted 4 lines

3) As 1, but needs conversion. read() into a temporary buffer, convert with libv4lconvert (I think this needs a secondary buffer), and supply data from the secondary buffer to read()

This is already handled, the problem why we currently do not support read() only devices, is that we used to use mmap under the hood while emulating read() like this (faster). This turns out to even be a problem with certain mmap() capable devices, so recently I added the capability to fall back to using read() to get data from the driver when doing conversion and the client is doing read.

Quoted 5 lines

4) As 2, but needs conversion. Change DQBUF handling to read() frame size chunks into a temporary buffer, then use libv4lconvert to copy those chunks from the temporary buffer into the buffer you're about to pass out.

As 2: this is going to be a case of though luck for the client.

Quoted 2 lines

Have I missed anything?

Nope, actually most if the code is already there. I will take a shot at implementing libv4l2 support for devices which only support read() and not mmap() mode and send you a patch to test.

Quoted 9 lines
Quoted 3 lines

I wonder though, doesn't the cx18 offer any format that xine can handle directly?

Not sensibly; it offers HM12 only, and Xine needs an RGB format, YV12, or YUYV. With a lot of rework, I could have the cx18 encode video to MPEG-2, then have Xine decode the resulting MPEG-2 stream, but this seems like overkill for uncompressed video. I could also teach Xine to handle HM12 natively, but that would duplicate effort already done in libv4l2 and libv4lconvert, which seems a bit silly to me.

Ah, I already thought it might be that device, but I wasn't sure.

Regards,

Hans

#4 lore raw
In reply to Simon Farnsworth

Hi,

Quoted 21 lines · Simon Farnsworth
Quoted 8 lines · Hans de Goede

Hi,

Quoted 5 lines · Simon Farnsworth

I have a Hauppauge HVR1600 for NTSC and ATSC support, and it appears to simply not work with libv4l2, due to lack of mmap support. My code works adequately (modulo a nice pile of bugs) with a HVR1110r3, so it appears to be driver level.

Quoted 8 lines
Quoted 4 lines

Which is the better route to handling this; adding code to input_v4l to use libv4lconvert when mmap isn't available, or converting the cx18 driver to use mmap?

Or modify libv4l2 to also handle devices which can only do read. There have been some changes to libv4l2 recently which would make doing that feasible.

Roughly what would I need to do to libv4l2?

I can see four new cases to handle:

Quoted 8 lines

1) driver supports read(), client wants read(), format supported by both. Just pass read() through to the driver.

3) As 1, but needs conversion. read() into a temporary buffer, convert with libv4lconvert (I think this needs a secondary buffer), and supply data from the secondary buffer to read()

Ok,

That was even easier then I thought it would be. Attached is a patch (against libv4l-0.6.1), which implements 1) and 3) from above.

Please give this a try.

Regards,

Hans

Quoted 11 lines
Quoted 11 lines
Quoted 4 lines

If it's a case of converting the cx18 driver, how would I go about doing that? I have no experience of the driver, so I'm not sure what I'd have to do - noting that if I break the existing read() support, other users will get upset.

I don't believe that modifying the driver is the answer, we need to either fix this at the libv4l or xine level.

I wonder though, doesn't the cx18 offer any format that xine can handle directly?

Quoted 6 lines

Not sensibly; it offers HM12 only, and Xine needs an RGB format, YV12, or YUYV. With a lot of rework, I could have the cx18 encode video to MPEG-2, then have Xine decode the resulting MPEG-2 stream, but this seems like overkill for uncompressed video. I could also teach Xine to handle HM12 natively, but that would duplicate effort already done in libv4l2 and libv4lconvert, which seems a bit silly to me.

v4l2-apps/libv4l/libv4l2/libv4l2.c +5 -4
diff -r 88a9c2ed612e v4l2-apps/libv4l/libv4l2/libv4l2.c
@@ -526,10 +526,9 @@
     return -1;
   }
 
-  /* we only add functionality for video capture devices, and we do not
-     handle devices which don't do mmap */
+  /* we only add functionality for video capture devices */
   if (!(cap.capabilities & V4L2_CAP_VIDEO_CAPTURE) ||
-      !(cap.capabilities & V4L2_CAP_STREAMING))
+      !(cap.capabilities & (V4L2_CAP_STREAMING|V4L2_CAP_READWRITE))
     return fd;
 
   /* Get current cam format */
@@ -564,6 +563,8 @@
   devices[index].flags = v4l2_flags;
   if (cap.capabilities & V4L2_CAP_READWRITE)
     devices[index].flags |= V4L2_SUPPORTS_READ;
+  if (!(cap.capabilities & V4L2_CAP_STREAMING))
+    devices[index].flags |= V4L2_USE_READ_FOR_READ;
   if (!strcmp((char *)cap.driver, "uvcvideo"))
     devices[index].flags |= V4L2_IS_UVC;
   devices[index].open_count = 1;
@@ -571,7 +572,7 @@
   devices[index].dest_fmt = fmt;
 
   /* When a user does a try_fmt with the current dest_fmt and the dest_fmt
-     is a supported one we will align the resulution (see try_fmt for why).
+     is a supported one we will align the resolution (see try_fmt for why).
      Do the same here now, so that a try_fmt on the result of a get_fmt done
      immediately after open leaves the fmt unchanged. */
   if (v4lconvert_supported_dst_format(
author Applied
#5 lore raw
Quoted 5 lines · Hans de Goede

Ok,

That was even easier then I thought it would be. Attached is a patch (against libv4l-0.6.1), which implements 1) and 3) from above.

I applied it to a clone of your HG repository, and had to make a minor change to get it to compile. I've attached the updated patch.

It looks like the read() from the card isn't reading entire frames ata a time - I'm using a piece of test gear that I have to return in a couple of hours to send colourbars to it, and I'm seeing bad colour, and the picture moving across the screen. I'll try and chase this, see whether there's something obviously wrong.

The repository I went against was http://linuxtv.org/hg/~hgoede/libv4l/
identified as:

$ hg identify
c51a90c0f62f+ tip
Signature · 1 line
Simon Farnsworth
v4l2-apps/libv4l/libv4l2/libv4l2.c +5 -4
diff -r c51a90c0f62f v4l2-apps/libv4l/libv4l2/libv4l2.c
@@ -526,10 +526,9 @@
     return -1;
   }
 
-  /* we only add functionality for video capture devices, and we do not
-     handle devices which don't do mmap */
+  /* we only add functionality for video capture devices */
   if (!(cap.capabilities & V4L2_CAP_VIDEO_CAPTURE) ||
-      !(cap.capabilities & V4L2_CAP_STREAMING))
+      !(cap.capabilities & (V4L2_CAP_STREAMING|V4L2_CAP_READWRITE)))
     return fd;
 
   /* Get current cam format */
@@ -564,6 +563,8 @@
   devices[index].flags = v4l2_flags;
   if (cap.capabilities & V4L2_CAP_READWRITE)
     devices[index].flags |= V4L2_SUPPORTS_READ;
+  if (!(cap.capabilities & V4L2_CAP_STREAMING))
+    devices[index].flags |= V4L2_USE_READ_FOR_READ;
   if (!strcmp((char *)cap.driver, "uvcvideo"))
     devices[index].flags |= V4L2_IS_UVC;
   devices[index].open_count = 1;
@@ -571,7 +572,7 @@
   devices[index].dest_fmt = fmt;
 
   /* When a user does a try_fmt with the current dest_fmt and the dest_fmt
-     is a supported one we will align the resulution (see try_fmt for why).
+     is a supported one we will align the resolution (see try_fmt for why).
      Do the same here now, so that a try_fmt on the result of a get_fmt done
      immediately after open leaves the fmt unchanged. */
   if (v4lconvert_supported_dst_format(
#6 lore raw
Quoted 7 lines · Simon Farnsworth
Quoted 6 lines · Hans de Goede

Ok,

That was even easier then I thought it would be. Attached is a patch (against libv4l-0.6.1), which implements 1) and 3) from above.

Quoted 9 lines

I applied it to a clone of your HG repository, and had to make a minor change to get it to compile. I've attached the updated patch.

It looks like the read() from the card isn't reading entire frames ata a time - I'm using a piece of test gear that I have to return in a couple of hours to send colourbars to it, and I'm seeing bad colour, and the picture moving across the screen. I'll try and chase this, see whether there's something obviously wrong.

There is indeed something obviously wrong; at line 315 of libv4l2.c, we expand the buffer we read into, then ask for that many bytes.

v4l2-apps/libv4l/libv4l2/libv4l2.c Thu Sep 03 11:32:40 2009 +0100 renamed from v4l2-apps/libv4l/libv4l2/libv4l2.c Tue Sep 01 10:03:27 2009 +0200 +1 -1
diff -r c51a90c0f62f v4l2-apps/libv4l/libv4l2/libv4l2.c
@@ -326,7 +326,7 @@
   }
 
   do {
-    result = SYS_READ(devices[index].fd, devices[index].readbuf, buf_size);
+    result = SYS_READ(devices[index].fd, devices[index].readbuf, devices[index].dest_fmt.fmt.pix.sizeimage);
     if (result <= 0) {
       if (result && errno != EAGAIN) {
        int saved_err = errno;

fixes it for me.

I appear to lose colour after a few seconds of capture, which I shall chase further.

Signature · 1 line
Simon Farnsworth
#7 lore raw
Quoted 2 lines · Simon Farnsworth

I appear to lose colour after a few seconds of capture, which I shall chase further.

And resolved - I was off-frequency by 2MHz, which leaves me surprised that I got picture. Only thing left for me to sort out is audio support.

Signature · 3 lines
Thanks for your help,

Simon Farnsworth
#15 lore raw
In reply to Hans de Goede
Quoted 14 lines · Hans de Goede

Hi,

Quoted 11 lines · Simon Farnsworth

Hello,

I'm in the process of reworking Xine's input_v4l to use libv4l2, so that
it gets the benefit of all the work done on modern cards and webcams,
and I've hit a stumbling block.

I have a Hauppauge HVR1600 for NTSC and ATSC support, and it appears to simply not work with libv4l2, due to lack of mmap support. My code works adequately (modulo a nice pile of bugs) with a HVR1110r3, so it appears to be driver level.

Quoted 12 lines
Quoted 4 lines

Which is the better route to handling this; adding code to input_v4l to use libv4lconvert when mmap isn't available, or converting the cx18 driver to use mmap?

Or modify libv4l2 to also handle devices which can only do read. There have been some changes to libv4l2 recently which would make doing that feasible.

Quoted 4 lines

If it's a case of converting the cx18 driver, how would I go about doing that? I have no experience of the driver, so I'm not sure what I'd have to do - noting that if I break the existing read() support, other users will get upset.

Modifying the cx18 driver to support mmap would be time consuming and non-trivial. Since cx18 is a rework of the ivtv driver, you will have the same problem with ivtv and PVR-150's and PVR-500's.

Implementing mmap() in these drivers for MPEG PS and TS streams would be interesting, because of the MPEG frames have variable length, not a fixed length. Since MPEG compressed video is the main format for which people purchase a CX2341[568] based board for analog TV, mmap() mode doesn't have a big payoff.

The CX2341[568] can output YUV video in the HM12 format, so mmap() may make sense for that. The challenges: implementing a {cx18,ivtv}-alsa module to provide ALSA PCM nodes instead of /dev/video24 PCM audio; and switching the primary stream handling and queuing used by the drivers internally to be different for different stream types (YUV/mmap, MPEG/read, and PCM/read but V4L2 and ALSA).

But I suspect no user pays for the extra cost of the CX2341[568] hardware MPEG encoder, if the user primarily wants uncompressed YUV video as their main format.

Quoted 5 lines

I don't believe that modifying the driver is the answer, we need to either fix this at the libv4l or xine level.

I wonder though, doesn't the cx18 offer any format that xine can handle directly?

The CX2341[568] can output a DVD compatible MPEG-2 PS, if the default MPEG-2 PS is something xine can't handle.

Quoted 4 lines

As stated libv4l2 currently does not support devices that cannot do read, what this comes down to in practice (or should, if not that is a bug), is that it passes all calls directly to the driver. So if the driver has any pixfmt's xine can handle directly things should work fine.

The cx18 and ivtv drivers report 2 video capture pixel formats:

        static struct v4l2_fmtdesc formats[] = {
                { 0, V4L2_BUF_TYPE_VIDEO_CAPTURE, 0,
                  "HM12 (YUV 4:1:1)", V4L2_PIX_FMT_HM12, { 0, 0, 0, 0 }
                },
                { 1, V4L2_BUF_TYPE_VIDEO_CAPTURE, V4L2_FMT_FLAG_COMPRESSED,
                  "MPEG", V4L2_PIX_FMT_MPEG, { 0, 0, 0, 0 }
                }
        };

But MPEG controls can be used to select the exact format of the MPEG stream, including an MPEG-2 TS in the case of the CX23418.

Regards,
Andy

Quoted 3 lines

Regards,

Hans

#13 lore raw
In reply to Simon Farnsworth
Quoted 9 lines · Simon Farnsworth
Quoted 5 lines · Hans de Goede

Ok,

That was even easier then I thought it would be. Attached is a patch (against libv4l-0.6.1), which implements 1) and 3) from above.

I applied it to a clone of your HG repository, and had to make a minor change to get it to compile. I've attached the updated patch.

Woopsie, sorry I should have test compiled it myself.

Regards,

Hans

#9 lore raw
In reply to Simon Farnsworth

Hans Verkuil,

I think we have found a bug in the read() implementation of the cx18 driver, see below.

Hi all,

Quoted 11 lines · Simon Farnsworth
Quoted 10 lines · Simon Farnsworth
Quoted 6 lines · Hans de Goede

Ok,

That was even easier then I thought it would be. Attached is a patch (against libv4l-0.6.1), which implements 1) and 3) from above.

I applied it to a clone of your HG repository, and had to make a minor change to get it to compile. I've attached the updated patch.

Quoted 9 lines
Quoted 6 lines

It looks like the read() from the card isn't reading entire frames ata a time - I'm using a piece of test gear that I have to return in a couple of hours to send colourbars to it, and I'm seeing bad colour, and the picture moving across the screen. I'll try and chase this, see whether there's something obviously wrong.

There is indeed something obviously wrong; at line 315 of libv4l2.c, we expand the buffer we read into, then ask for that many bytes.

Ah, actually this is a driver bug, not a libv4l2 bug, but I'll fix things in libv4l to work around it for now.

read() should always return an entire frame (or as much of it as will fit and throw away the rest). Think for example of jpeg streams, where the exact size of the image isn't known by the client (as it differs from frame to frame). dest_fmt.fmt.pix.sizeimage purely is an upper limit, and so is the value passed in to read(), the driver itself should clamp it so that it returns exactly one frame (for formats which are frame based).

The page alignment (2 pages on i386 / one on x86_64) is done because some drivers internally use page aligned buffer sizes and thus for example with jpeg streams, can have frames queued for read() slightly bigger then dest_fmt.fmt.pix.sizeimage, but when this happens that is really a driver bug, because as said dest_fmt.fmt.pix.sizeimage should report an upper limit of the the frame sizes to be expected. I'll remove the align workaround, as that bug is much less likely to be hit (and probably easier to fix at the driver level) then the issue we're now seeing with read().

Regards,

Hans

#8 lore raw
In reply to Simon Farnsworth
Quoted 6 lines · Simon Farnsworth
Quoted 2 lines · Simon Farnsworth

I appear to lose colour after a few seconds of capture, which I shall chase further.

And resolved - I was off-frequency by 2MHz, which leaves me surprised that I got picture. Only thing left for me to sort out is audio support.

I've got a start at cx18-alsa support in

http://linuxtv.org/hg/~awalls/mc-lab (IIRC)

But it's only really the non-working skeleton of things to get ALSA device nodes. The behind the scenes work of actually opening the PCM stream for ALSA PCM nodes and proper locking with the /dev/video24 PCM stream nodes is not there.

You'll have to use /dev/video24 for now.

Regards,
Andy

#16 lore raw
In reply to Andy Walls
Quoted 3 lines · Andy Walls

But I suspect no user pays for the extra cost of the CX2341[568] hardware MPEG encoder, if the user primarily wants uncompressed YUV video as their main format.

Actually, we're doing exactly that. We want a PCI card from a reputable manufacturer which provides uncompressed YUV and ATSC (both OTA and ClearQAM cable). As we already buy Hauppauge HVR-1110s for DVB-T and uncompressed analogue, a Hauppauge card suits us, and the only thing they have that fits the needs is the HVR-1600; the MPEG encoder is thus left idle.

Signature · 1 line
Simon Farnsworth
#10 lore raw
In reply to Hans de Goede
Quoted 40 lines · Hans de Goede

Hans Verkuil,

I think we have found a bug in the read() implementation of the cx18 driver, see below.

Hi all,

Quoted 11 lines · Simon Farnsworth
Quoted 10 lines · Simon Farnsworth
Quoted 6 lines · Hans de Goede

Ok,

That was even easier then I thought it would be. Attached is a patch (against libv4l-0.6.1), which implements 1) and 3) from above.

I applied it to a clone of your HG repository, and had to make a minor change to get it to compile. I've attached the updated patch.

Quoted 9 lines
Quoted 6 lines

It looks like the read() from the card isn't reading entire frames ata a time - I'm using a piece of test gear that I have to return in a couple of hours to send colourbars to it, and I'm seeing bad colour, and the picture moving across the screen. I'll try and chase this, see whether there's something obviously wrong.

There is indeed something obviously wrong; at line 315 of libv4l2.c, we expand the buffer we read into, then ask for that many bytes.

Ah, actually this is a driver bug, not a libv4l2 bug, but I'll fix things in libv4l to work around it for now.

read() should always return an entire frame (or as much of it as will fit and throw away the rest). Think for example of jpeg streams, where the exact size of the image isn't known by the client (as it differs from frame to frame). dest_fmt.fmt.pix.sizeimage purely is an upper limit, and so is the value passed in to read(), the driver itself should clamp it so that it returns exactly one frame (for formats which are frame based).

Quoted 8 lines

The page alignment (2 pages on i386 / one on x86_64) is done because some drivers internally use page aligned buffer sizes and thus for example with jpeg streams, can have frames queued for read() slightly bigger then dest_fmt.fmt.pix.sizeimage, but when this happens that is really a driver bug, because as said dest_fmt.fmt.pix.sizeimage should report an upper limit of the the frame sizes to be expected. I'll remove the align workaround, as that bug is much less likely to be hit (and probably easier to fix at the driver level) then the issue we're now seeing with read().

Hans and Hans,

I'll have time to look into this on Friday and see what can be done.

Hans de Goede,

I may ask for more information/explanation later.

Regards,
Andy

Quoted 3 lines

Regards,

Hans

#17 lore raw
In reply to Simon Farnsworth
Quoted 11 lines · Simon Farnsworth
Quoted 3 lines · Andy Walls

But I suspect no user pays for the extra cost of the CX2341[568] hardware MPEG encoder, if the user primarily wants uncompressed YUV video as their main format.

Actually, we're doing exactly that. We want a PCI card from a reputable manufacturer which provides uncompressed YUV and ATSC (both OTA and ClearQAM cable). As we already buy Hauppauge HVR-1110s for DVB-T and uncompressed analogue, a Hauppauge card suits us, and the only thing they have that fits the needs is the HVR-1600; the MPEG encoder is thus left idle.

Ah. OK, this is more than an academic exercise. :)

If you can prioritize your needs for the cx18 driver, I can see what I can get done.

If you'd like to submit patches, I'll be happy to review them to make sure they don't break anything and then get them integrated.

Regards,
Andy

#18 lore raw
In reply to Andy Walls
Quoted 12 lines · Andy Walls
Quoted 10 lines · Simon Farnsworth
Quoted 3 lines · Andy Walls

But I suspect no user pays for the extra cost of the CX2341[568] hardware MPEG encoder, if the user primarily wants uncompressed YUV video as their main format.

Actually, we're doing exactly that. We want a PCI card from a reputable manufacturer which provides uncompressed YUV and ATSC (both OTA and ClearQAM cable). As we already buy Hauppauge HVR-1110s for DVB-T and uncompressed analogue, a Hauppauge card suits us, and the only thing they have that fits the needs is the HVR-1600; the MPEG encoder is thus left idle.

Quoted 5 lines

Ah. OK, this is more than an academic exercise. :)

If you can prioritize your needs for the cx18 driver, I can see what I can get done.

The video side is now working well for me on the HVR1110 and HVR1600 - in an ideal world, I'd have an ALSA driver for cx18 audio instead of the video24 device. The read() bug isn't major for me, as I can just use a slightly modified libv4l2 to cope.

Quoted 3 lines

If you'd like to submit patches, I'll be happy to review them to make sure they don't break anything and then get them integrated.

I'll be discussing this with my management next week, and they'll be making a judgement call on whether we can cope without analogue audio on the RF input only. If we can't, I'll be tasked with working on this; I take it you'd prefer to have ALSA added to the driver than for me to teach Xine to read from /dev/video24 for PCM audio?

In any case, I have some more work to do on the reworked input_v4l for Xine, as I need to get it into a state where I can work with the Xine guys on merging it into their mainline. This will have to be finished before I can really dive into the cx18 driver.

Signature · 4 lines
Thank you for the offer of help - I'll let you know if I'm going to
start working on ALSA support in cx18,

Simon Farnsworth
#19 lore raw
Quoted 23 lines · Simon Farnsworth
Quoted 12 lines · Andy Walls
Quoted 10 lines · Simon Farnsworth
Quoted 3 lines · Andy Walls

But I suspect no user pays for the extra cost of the CX2341[568] hardware MPEG encoder, if the user primarily wants uncompressed YUV video as their main format.

Actually, we're doing exactly that. We want a PCI card from a reputable manufacturer which provides uncompressed YUV and ATSC (both OTA and ClearQAM cable). As we already buy Hauppauge HVR-1110s for DVB-T and uncompressed analogue, a Hauppauge card suits us, and the only thing they have that fits the needs is the HVR-1600; the MPEG encoder is thus left idle.

Quoted 5 lines

Ah. OK, this is more than an academic exercise. :)

If you can prioritize your needs for the cx18 driver, I can see what I can get done.

The video side is now working well for me on the HVR1110 and HVR1600 - in an ideal world, I'd have an ALSA driver for cx18 audio instead of the video24 device. The read() bug isn't major for me, as I can just use a slightly modified libv4l2 to cope.

Quoted 8 lines
Quoted 3 lines

If you'd like to submit patches, I'll be happy to review them to make sure they don't break anything and then get them integrated.

I'll be discussing this with my management next week, and they'll be making a judgement call on whether we can cope without analogue audio on the RF input only. If we can't, I'll be tasked with working on this; I take it you'd prefer to have ALSA added to the driver than for me to teach Xine to read from /dev/video24 for PCM audio?

I have no personal preference. Implementing a working cx18-alsa will mean a number of applications can then use PCM audio from the CX23418 without modification. That would be better for Linux users overall.

Implementing changes in Xine to use the /dev/video24 (V4L2 PCM device), I suspect, is the approach to use to meet your management's or customers' requirements on a short schedule.

Regards,
Andy

Quoted 4 lines

In any case, I have some more work to do on the reworked input_v4l for Xine, as I need to get it into a state where I can work with the Xine guys on merging it into their mainline. This will have to be finished before I can really dive into the cx18 driver.

#11 lore raw
In reply to Hans de Goede
Quoted 29 lines · Hans de Goede

Hans Verkuil,

I think we have found a bug in the read() implementation of the cx18 driver, see below.

Hi all,

Quoted 20 lines · Simon Farnsworth
Quoted 7 lines · Simon Farnsworth
Quoted 6 lines · Hans de Goede

Ok,

That was even easier then I thought it would be. Attached is a patch (against libv4l-0.6.1), which implements 1) and 3) from above.

Quoted 9 lines

I applied it to a clone of your HG repository, and had to make a minor change to get it to compile. I've attached the updated patch.

It looks like the read() from the card isn't reading entire frames ata a time - I'm using a piece of test gear that I have to return in a couple of hours to send colourbars to it, and I'm seeing bad colour, and the picture moving across the screen. I'll try and chase this, see whether there's something obviously wrong.

There is indeed something obviously wrong; at line 315 of libv4l2.c, we expand the buffer we read into, then ask for that many bytes.

Hans de Goede,

Quoted 1 line

Ah, actually this is a driver bug,

I agree.

Quoted 2 lines

not a libv4l2 bug, but I'll fix things
in libv4l to work around it for now.

OK, thanks.

Quoted 1 line

read() should always return an entire frame (or as much of it as will fit

I agree

Quoted 1 line

and throw away the rest).

That sounds fine to me.

Hans and Hans,

The V4L2 spec for the read() call seems unlcear to me:

"Return Value
On success, the number of bytes read is returned. It is not an error if this number is smaller than the number of bytes requested, or the amount of data required for one frame. This may happen for example because read() was interrupted by a signal. On error, -1 is returned, and the errno variable is set appropriately. In this case the next read will start at the beginning of a new frame."

To me, the spec only says the remainder of a frame is thrown away when read() exits with an error.

BTW, should select() return "data available", if less than one whole frame is available? It can happen if the buffers we give to the CX23418 firmware don't exactly match the YUV framesize. The V4l2 spec for the read() call seems to imply that read() should block (or return with EAGAIN) until at least one whole frame is available. Is that correct?

Regards,
Andy

Quoted 23 lines

Think for example of jpeg streams, where the
exact size of the image isn't known by the client (as it differs from frame to frame). dest_fmt.fmt.pix.sizeimage purely is an upper limit, and so is the value passed in to read(), the driver itself should clamp it so that it returns exactly one frame (for formats which are frame based).

The page alignment (2 pages on i386 / one on x86_64) is done because some drivers internally use page aligned buffer sizes and thus for example with jpeg streams, can have frames queued for read() slightly bigger then dest_fmt.fmt.pix.sizeimage, but when this happens that is really a driver bug, because as said dest_fmt.fmt.pix.sizeimage should report an upper limit of the the frame sizes to be expected. I'll remove the align workaround, as that bug is much less likely to be hit (and probably easier to fix at the driver level) then the issue we're now seeing with read().

Regards,

Hans
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#12 lore raw

Hi,

Quoted 10 lines · Andy Walls

The V4L2 spec for the read() call seems unlcear to me:

"Return Value
On success, the number of bytes read is returned. It is not an error if this number is smaller than the number of bytes requested, or the amount of data required for one frame. This may happen for example because read() was interrupted by a signal. On error, -1 is returned, and the errno variable is set appropriately. In this case the next read will start at the beginning of a new frame."

Quoted 3 lines

To me, the spec only says the remainder of a frame is thrown away when read() exits with an error.

It does seem to say that yes, as said in a previous mail of me, this part of the spec needs some fixing. It seems to try and describe the queuing that happens inside the driver in too much detail and leaves out other much me important bits about how read() on video devices behaves.

Quoted 6 lines

BTW, should select() return "data available", if less than one whole frame is available? It can happen if the buffers we give to the CX23418 firmware don't exactly match the YUV framesize. The V4l2 spec for the read() call seems to imply that read() should block (or return with EAGAIN) until at least one whole frame is available. Is that correct?

I agree that waiting until at least one whole frame is available. Is the correct behaviour.

Regards,

Hans