HD-PVR fails consistently on Linux, works on Windows
I recently purchased a Hauppauge HD-PVR (the 1212 version, label on bottom 49001LF, Rev F2). I have consistent capture failures on Linux where data from the device simply stops, generally within a few minutes of starting a capture. Yet, the device works flawlessly on Windows with the same USB and component cables, same power supply, and same physical position. This suggests that the device itself has acceptable power, is not overheating, etc. I'll detail below the testing I've done thus far and would appreciate any suggestions on how to further test or address the problem.
The good news is that I have a highly reproducible failure on Linux, but then that's the bad news too.
Thanks.
Keith
-- Linux tests --
I started trying to use the HD-PVR directly with my MythTV backend. I have subsequently switched all of my testing to simple direct captures from the /dev/video? device using /bin/cat to eliminate as many variables as possible.
I've done a large number of tests with combinations of the following:
OS: gentoo 3.4.7, gentoo 3.5.4
HD-PVR firmware: 1.5.7.0 (0x15), 1.7.1.30059 (0x1e)
Input resolution: fixed to 720p, fixed to 1080i, floating based on input
USB ports: motherboard ports on Intel DP45SG, motherboard ports on MSI X58 Pro-E, ports on SIIG USB PCIe card
Captures fail consistently.
I've verified that the HD-PVR is the only device on the USB bus and that the bus shows as "Linux Foundation 2.0 root hub" in all tests. I've increased the debug output level for the hdpvr driver to 6 (/sys/module/hdpvr/parameters/hdpvr_debug) and collected the following:
Sep 21 17:01:00 mythbe kernel: [535043.504450] usb 9-1: New USB device found, idVendor=2040, idProduct=4903 Sep 21 17:01:00 mythbe kernel: [535043.504453] usb 9-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3 Sep 21 17:01:00 mythbe kernel: [535043.504456] usb 9-1: Product: Hauppauge HD PVR Sep 21 17:01:00 mythbe kernel: [535043.504458] usb 9-1: Manufacturer: AMBA Sep 21 17:01:00 mythbe kernel: [535043.504459] usb 9-1: SerialNumber: 00A6DD48 Sep 21 17:01:00 mythbe kernel: [535043.504523] usb 9-1: ep 0x1 - rounding interval to 32768 microframes, ep desc says 0 microframes Sep 21 17:01:00 mythbe kernel: [535043.504528] usb 9-1: ep 0x81 - rounding interval to 32768 microframes, ep desc says 0 microframes Sep 21 17:01:01 mythbe kernel: [535043.703947] hdpvr 9-1:1.0: firmware version 0x15 dated Jun 17 2010 09:26:53 Sep 21 17:01:01 mythbe kernel: [535043.889144] IR keymap rc-hauppauge not found Sep 21 17:01:01 mythbe kernel: [535043.889146] Registered IR keymap rc-empty Sep 21 17:01:01 mythbe kernel: [535043.889190] input: i2c IR (HD-PVR) as /devices/virtual/rc/rc5/input16 Sep 21 17:01:01 mythbe kernel: [535043.889415] rc5: i2c IR (HD-PVR) as /devices/virtual/rc/rc5 Sep 21 17:01:01 mythbe kernel: [535043.889417] ir-kbd-i2c: i2c IR (HD-PVR) detected at i2c-8/8-0071/ir0 [Hauppage HD PVR I2C] Sep 21 17:01:01 mythbe kernel: [535043.889518] hdpvr 9-1:1.0: device now attached to video6 Sep 21 17:01:01 mythbe kernel: [535043.889534] usbcore: registered new interface driver hdpvr Sep 21 17:05:11 mythbe kernel: [535293.776318] hdpvr 9-1:1.0: video signal: 1920x1080@30hz Sep 21 17:05:14 mythbe kernel: [535297.312589] hdpvr 9-1:1.0: encoder start control request returned 0 Sep 21 17:05:15 mythbe kernel: [535297.670830] hdpvr 9-1:1.0: config call request for value 0x700 returned 1 Sep 21 17:05:15 mythbe kernel: [535297.670833] hdpvr 9-1:1.0: streaming started Sep 21 17:05:15 mythbe kernel: [535297.670839] hdpvr 9-1:1.0: hdpvr_read:442 buffer stat: 64 free, 0 proc Sep 21 17:05:15 mythbe kernel: [535297.670882] hdpvr 9-1:1.0: hdpvr_submit_buffers:209 buffer stat: 0 free, 64 proc Sep 21 17:05:15 mythbe kernel: [535297.709079] hdpvr 9-1:1.0: hdpvr_read:502 buffer stat: 1 free, 63 proc Sep 21 17:05:15 mythbe kernel: [535297.709088] hdpvr 9-1:1.0: hdpvr_submit_buffers:209 buffer stat: 0 free, 64 proc
(many repeats of the above two line sequence)
Sep 21 17:17:09 mythbe kernel: [536011.936858] hdpvr 9-1:1.0: hdpvr_read:502 buffer stat: 1 free, 63 proc Sep 21 17:17:09 mythbe kernel: [536011.936866] hdpvr 9-1:1.0: hdpvr_submit_buffers:209 buffer stat: 0 free, 64 proc Sep 21 17:17:36 mythbe kernel: [536038.853044] hdpvr 9-1:1.0: config call request for value 0x800 returned -110 Sep 21 17:17:36 mythbe kernel: [536038.853052] hdpvr 9-1:1.0: transmit worker exited Sep 21 17:17:36 mythbe kernel: [536038.996035] hdpvr 9-1:1.0: used 0 urbs to empty device buffers
If I understand correctly, this is showing a ETIMEDOUT error. When I've looked at the cat with strace, it is always blocked on a read. So, it seems like the HD-PVR just stops sending.
I also ran a USB capture with wireshark and see much the same thing. While I haven't tried to decode the USB packets, the pattern is that the HD-PVR sends, the host sends a message/ack, this pattern repeats, and then nothing. The majority of the failures occur in less than 15 minutes.
-- Windows tests --
I installed the Hauppauge software on a Windows 7 system and moved the USB cable to the Windows system. Nothing else was changed. I've run many hours of successful captures. I've checked each recording with ffprobe and verified that each is the expected length (i.e., no data drops at all). The recordings are of good quality. This shows that the HD-PVR is capable of working as expected and quite reliably.
Quoted 106 lines · On Wed, 26 Sep 2012 21:42:32 -0500 Keith Pyle <[email protected]> wrote:
I recently purchased a Hauppauge HD-PVR (the 1212 version, label on bottom 49001LF, Rev F2). I have consistent capture failures on Linux where data from the device simply stops, generally within a few minutes of starting a capture. Yet, the device works flawlessly on Windows with the same USB and component cables, same power supply, and same physical position. This suggests that the device itself has acceptable power, is not overheating, etc. I'll detail below the testing I've done thus far and would appreciate any suggestions on how to further test or address the problem.
The good news is that I have a highly reproducible failure on Linux, but then that's the bad news too.
Thanks.
Keith
-- Linux tests --
I started trying to use the HD-PVR directly with my MythTV backend. I have subsequently switched all of my testing to simple direct captures from the/dev/video? device using/bin/catto eliminate as many variables as possible.I've done a large number of tests with combinations of the following:
OS: gentoo 3.4.7, gentoo 3.5.4
HD-PVR firmware: 1.5.7.0 (0x15), 1.7.1.30059 (0x1e)
Input resolution: fixed to 720p, fixed to 1080i, floating based on input USB ports: motherboard ports on Intel DP45SG, motherboard ports on MSI X58 Pro-E, ports on SIIG USB PCIe cardCaptures fail consistently.
I've verified that the HD-PVR is the only device on the USB bus and that the bus shows as "Linux Foundation 2.0 root hub" in all tests. I've increased the debug output level for the hdpvr driver to 6 (
/sys/module/hdpvr/parameters/hdpvr_debug) and collected the following:Sep 21 17:01:00 mythbe kernel: [535043.504450] usb 9-1: New USB device found, idVendor=2040, idProduct=4903 Sep 21 17:01:00 mythbe kernel: [535043.504453] usb 9-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3 Sep 21 17:01:00 mythbe kernel: [535043.504456] usb 9-1: Product: Hauppauge HD PVR Sep 21 17:01:00 mythbe kernel: [535043.504458] usb 9-1: Manufacturer: AMBA Sep 21 17:01:00 mythbe kernel: [535043.504459] usb 9-1: SerialNumber: 00A6DD48 Sep 21 17:01:00 mythbe kernel: [535043.504523] usb 9-1: ep 0x1 - rounding interval to 32768 microframes, ep desc says 0 microframes Sep 21 17:01:00 mythbe kernel: [535043.504528] usb 9-1: ep 0x81 - rounding interval to 32768 microframes, ep desc says 0 microframes Sep 21 17:01:01 mythbe kernel: [535043.703947] hdpvr 9-1:1.0: firmware version 0x15 dated Jun 17 2010 09:26:53 Sep 21 17:01:01 mythbe kernel: [535043.889144] IR keymap rc-hauppauge not found Sep 21 17:01:01 mythbe kernel: [535043.889146] Registered IR keymap rc-empty Sep 21 17:01:01 mythbe kernel: [535043.889190] input: i2c IR (HD-PVR) as /devices/virtual/rc/rc5/input16 Sep 21 17:01:01 mythbe kernel: [535043.889415] rc5: i2c IR (HD-PVR) as /devices/virtual/rc/rc5 Sep 21 17:01:01 mythbe kernel: [535043.889417] ir-kbd-i2c: i2c IR (HD-PVR) detected at i2c-8/8-0071/ir0 [Hauppage HD PVR I2C] Sep 21 17:01:01 mythbe kernel: [535043.889518] hdpvr 9-1:1.0: device now attached to video6 Sep 21 17:01:01 mythbe kernel: [535043.889534] usbcore: registered new interface driver hdpvr Sep 21 17:05:11 mythbe kernel: [535293.776318] hdpvr 9-1:1.0: video signal: 1920x1080@30hz Sep 21 17:05:14 mythbe kernel: [535297.312589] hdpvr 9-1:1.0: encoder start control request returned 0 Sep 21 17:05:15 mythbe kernel: [535297.670830] hdpvr 9-1:1.0: config call request for value 0x700 returned 1 Sep 21 17:05:15 mythbe kernel: [535297.670833] hdpvr 9-1:1.0: streaming started Sep 21 17:05:15 mythbe kernel: [535297.670839] hdpvr 9-1:1.0: hdpvr_read:442 buffer stat: 64 free, 0 proc Sep 21 17:05:15 mythbe kernel: [535297.670882] hdpvr 9-1:1.0: hdpvr_submit_buffers:209 buffer stat: 0 free, 64 proc Sep 21 17:05:15 mythbe kernel: [535297.709079] hdpvr 9-1:1.0: hdpvr_read:502 buffer stat: 1 free, 63 proc Sep 21 17:05:15 mythbe kernel: [535297.709088] hdpvr 9-1:1.0:hdpvr_submit_buffers:209 buffer stat: 0 free, 64 proc
(many repeats of the above two line sequence)
Sep 21 17:17:09 mythbe kernel: [536011.936858] hdpvr 9-1:1.0: hdpvr_read:502 buffer stat: 1 free, 63 proc Sep 21 17:17:09 mythbe kernel: [536011.936866] hdpvr 9-1:1.0: hdpvr_submit_buffers:209 buffer stat: 0 free, 64 proc Sep 21 17:17:36 mythbe kernel: [536038.853044] hdpvr 9-1:1.0: config call request for value 0x800 returned -110 Sep 21 17:17:36 mythbe kernel: [536038.853052] hdpvr 9-1:1.0: transmit worker exited Sep 21 17:17:36 mythbe kernel: [536038.996035] hdpvr 9-1:1.0: used 0urbs to empty device buffers
If I understand correctly, this is showing a ETIMEDOUT error. When I've looked at the cat with strace, it is always blocked on a read. So, it seems like the HD-PVR just stops sending.
I also ran a USB capture with wireshark and see much the same thing. While I haven't tried to decode the USB packets, the pattern is that the HD-PVR sends, the host sends a message/ack, this pattern repeats, and then nothing. The majority of the failures occur in less than 15 minutes.
Quoted 8 lines
-- Windows tests --
I installed the Hauppauge software on a Windows 7 system and moved the USB cable to the Windows system. Nothing else was changed. I've run many hours of successful captures. I've checked each recording with ffprobe and verified that each is the expected length (i.e., no data drops at all). The recordings are of good quality. This shows that the HD-PVR is capable of working as expected and quite reliably.
It may be a coincidence but I since I started using irqbalance ( https://code.google.com/p/irqbalance/ ) my HD-PVR has been completely stable. Before that I was experiencing daily lockups.
Quoted 9 lines · On Wed, 26 Sep 2012 21:42:32 -0500 Keith Pyle <[email protected]> wrote:
I recently purchased a Hauppauge HD-PVR (the 1212 version, label on bottom 49001LF, Rev F2). I have consistent capture failures on Linux where data from the device simply stops, generally within a few minutes of starting a capture.
[...]
Sep 21 17:01:01 mythbe kernel: [535043.703947] hdpvr 9-1:1.0: firmwareversion
0x15dated Jun 17 2010 09:26:53
When we contacted Hauppauge regarding the stability issue, they recommended upgrading to the latest firmware dated Mar 26 2012. We *think* this has improved stability, but it certainly hasn't fixed it completely.
Upgrading the firmware requires a Windows PC -- see
http://www.hauppauge.com/site/support/support_hdpvr.html
Quoted 3 lines · Jonathan
It may be a coincidence but I since I started using irqbalance (
https://code.google.com/p/irqbalance/ ) my HD-PVR has been completely stable. Before that I was experiencing daily lockups.
Interesting. You definitely didn't upgrade the firmware around the same time?
We think the stability is worse when the Linux PC is heavily loaded: We do real-time image processing on the video stream from the HD PVR, so the CPUs are maxed out, and we get frequent lock-ups. We also think the lock-ups are more frequent when we have several HD PVRs connected to the same PC, all running at the same time. I'll have to try this irqbalance.
--Dave.
Quoted 11 lines · David Röthlisberger
Quoted 9 lines · On Wed, 26 Sep 2012 21:42:32 -0500 Keith Pyle <[email protected]> wrote:
I recently purchased a Hauppauge HD-PVR (the 1212 version, label on bottom 49001LF, Rev F2). I have consistent capture failures on Linux where data from the device simply stops, generally within a few minutes of starting a capture.
[...]
Sep 21 17:01:01 mythbe kernel: [535043.703947] hdpvr 9-1:1.0: firmwareversion
0x15dated Jun 17 2010 09:26:53
Quoted 7 lines
When we contacted Hauppauge regarding the stability issue, they recommended upgrading to the latest firmware dated Mar 26 2012. We *think* this has improved stability, but it certainly hasn't fixed it completely.
Upgrading the firmware requires a Windows PC -- see
http://www.hauppauge.com/site/support/support_hdpvr.html
The Mar 26, 2012 firmware is the 1.7.1.30059 version that shows as 0x1e in the kernel messages. In my case, I've tried both 0x1e and 0x15 with no discernible difference in stability. That is, both hang reliably and repeatedly on Linux but not when using a Windows system.
Quoted 9 lines · Jonathan
Quoted 4 lines
It may be a coincidence but I since I started using irqbalance (
https://code.google.com/p/irqbalance/ ) my HD-PVR has been completely stable. Before that I was experiencing daily lockups.Interesting. You definitely didn't upgrade the firmware around the same time?
Quoted 8 lines
We think the stability is worse when the Linux PC is heavily loaded: We do real-time image processing on the video stream from the HD PVR, so the CPUs are maxed out, and we get frequent lock-ups. We also think the lock-ups are more frequent when we have several HD PVRs connected to the same PC, all running at the same time. I'll have to try this irqbalance.
--Dave.
All of my tests have been using only one HD-PVR on otherwise nearly idle systems with *lots* of system resources available. One test system is an Intel Core 2 Quad Q9400 (2.66 GHz) and the other is an Intel i7 950 (3.0 GHz). CPU loads are low during captures - not anywhere near having any core at 100%. While system load could certainly be a factor for some, I do not believe it is in my tests.
Keith
Quoted 30 lines · On Sat, 13 Oct 2012 20:17:59 +0100 David Röthlisberger <[email protected]>…
Quoted 9 lines · On Wed, 26 Sep 2012 21:42:32 -0500 Keith Pyle <[email protected]> wrote:
I recently purchased a Hauppauge HD-PVR (the 1212 version, label on bottom 49001LF, Rev F2). I have consistent capture failures on Linux where data from the device simply stops, generally within a few minutes of starting a capture.
[...]
Sep 21 17:01:01 mythbe kernel: [535043.703947] hdpvr 9-1:1.0: firmwareversion
0x15dated Jun 17 2010 09:26:53When we contacted Hauppauge regarding the stability issue, they recommended upgrading to the latest firmware dated Mar 26 2012. We *think* this has improved stability, but it certainly hasn't fixed it completely.
Upgrading the firmware requires a Windows PC -- see
http://www.hauppauge.com/site/support/support_hdpvr.htmlQuoted 3 lines · Jonathan
It may be a coincidence but I since I started using irqbalance (
https://code.google.com/p/irqbalance/ ) my HD-PVR has been completely stable. Before that I was experiencing daily lockups.Interesting. You definitely didn't upgrade the firmware around the same time?
Quoted 7 lines
We think the stability is worse when the Linux PC is heavily loaded: We do real-time image processing on the video stream from the HD PVR, so the CPUs are maxed out, and we get frequent lock-ups. We also think the lock-ups are more frequent when we have several HD PVRs connected to the same PC, all running at the same time. I'll have to try this irqbalance.
--Dave.
No change in my firmware; still on version 0x15. FWIW, after using irqblance for about 10 days, cat /proc/interrupts shows the interrupt for xhci_hcd (the USB3 bus my HD-PVR is attached to) is now spread across all 4 cores whereas before it was loaded up on CPU0. Same thing was shown for ahci, and a lot of data is being written to disk when the HD-PVR is working so I guess that could be a factor as well.
Jonathan
Quoted 11 lines · Keith Pyle
I recently purchased a Hauppauge HD-PVR (the 1212 version, label on bottom 49001LF, Rev F2). I have consistent capture failures on Linux where data from the device simply stops, generally within a few minutes of starting a capture. Yet, the device works flawlessly on Windows with the same USB and component cables, same power supply, and same physical position. This suggests that the device itself has acceptable power, is not overheating, etc. I'll detail below the testing I've done thus far and would appreciate any suggestions on how to further test or address the problem.
The good news is that I have a highly reproducible failure on Linux, but then that's the bad news too.
Thanks.
Keith
-- Linux tests --
I started trying to use the HD-PVR directly with my MythTV backend. I have subsequently switched all of my testing to simple direct captures from the/dev/video? device using/bin/catto eliminate as many variables as possible.
Quoted 8 lines
I've done a large number of tests with combinations of the following:
OS: gentoo 3.4.7, gentoo 3.5.4
HD-PVR firmware: 1.5.7.0 (0x15), 1.7.1.30059 (0x1e)
Input resolution: fixed to 720p, fixed to 1080i, floating based on input
USB ports: motherboard ports on Intel DP45SG, motherboard ports on MSI X58 Pro-E, ports on SIIG USB PCIe cardCaptures fail consistently.
Here's some interesting new information. On noticing that the 3.6 kernel included several USB related commits, I updated the kernel of my test system (MSI X58 Pro-E) from 3.5.4 to 3.6.2. I ran a series of capture tests with firmware 0x1e and all other variables exactly as before - same USB cable, port, physical position, etc. I have 20 successful 1-hour captures and 1 failure. There are no logged messages for the failed capture but the timing coincides within a minute after initializing a wireless joystick on a different USB bus. I've been unable to reproduce this failure, so I cannot conclusively state that there is a correlation.
The results of the kernel change are dramatic. Under 3.5.x and earlier, the failure rate for 1-hour captures was 100%. Most failed in less than 10 minutes. There were some instances of hard hangs on the HD-PVR (i.e., power cycle required). With 3.6.2, it is less than 5% failure. The one failure was a truncated recording.
I am continuing capture tests on my test system to build more data. I will next be updating my MythTV backend to 3.6 and trying the capture tests on it since it has a different motherboard.
It would be quite interesting if others with HD-PVR problems see similar results for 3.6.2 or better.
Keith
Quoted 10 lines · Keith Pyle
Quoted 8 lines · Keith Pyle
I recently purchased a Hauppauge HD-PVR (the 1212 version, label on bottom 49001LF, Rev F2). I have consistent capture failures on Linux where data from the device simply stops, generally within a few minutes of starting a capture. Yet, the device works flawlessly on Windows with the same USB and component cables, same power supply, and same physical position. This suggests that the device itself has acceptable power, is not overheating, etc. I'll detail below the testing I've done thus far and would appreciate any suggestions on how to further test or address the problem.
Quoted 5 lines
Here's some interesting new information. On noticing that the 3.6 kernel included several USB related commits, I updated the kernel of my test system (MSI X58 Pro-E) from 3.5.4 to 3.6.2. I ran a series of capture tests with firmware
0x1eand all other variables exactly as before - same USB cable, port, physical position, etc. I have 20 successful 1-hour captures and 1 failure.
We have run extensive tests comparing the HD PVR's stability on kernel 3.6.6 vs. kernel <3.6; having a single HD PVR connected to the PC vs. multiple HD PVRs on the same USB controller; adding better heatsinks; running irqbalance. None of these made any difference. The only significant improvement we found was from removing the lid and adding fans pointing directly at the circuit board. Even this didn't cure the problem completely, it only increased the time-to-failure.
For details, including the time-to-failure measurements and the shell script we used, see http://stb-tester.com/hardware.html#hauppauge-hd-pvr
Note that our workload is quite different from yours: An individual recording typically lasts 2-10 minutes, but as soon as it has finished we start another one, so we are stopping and starting all day. Hauppauge have hinted that under these conditions the HD PVR is unreliable on Windows too, which implies a problem in the hardware or firmware, rather than the Linux hdpvr driver.
Dave.