Support for Elgato Game Capture HD / MStar MST3367CMK
Hello everyone!
Over the past weeks, I've been busy capturing USB packets between the Elgato Game Capture HD and my PC. It's using the MStar MST3367CMK chip, which seems to have proprietary Linux support available only for hardware vendors in form of an SDK. Problem is, that this SDK is strictly kept under an NDA, making it kinda impossible for us to get our hands on.
So, I got my hands dirty and have found some very good stuff! First of all, in contrast to many sources, the Elgato Game Capture HD outputs compressed video and audio via USB! It's already encoded, so there is no need for reencoding, this will save CPU power. For testing purposes, I've only tried capturing 720p data for now, but this should be more than enough.
Basically, we need to read raw USB traffic, write an MPEG-TS file header, put in the raw USB data and close the file. I'm not super experienced in C / kernel development (especially V4L), but I'll give my best to get this project forward. My next step is getting a prototype working with libusb in userland; after that's done, I'll try porting it over to kernel / V4L.
Project page can be found here:
https://github.com/tolga9009/elgato-gchd
USB logs and docs:
v1.0 as 7zip: https://docs.google.com/file/d/0B29z6-xPIPLEQVBMTWZHbUswYjg
v1.0 as rar: https://docs.google.com/file/d/0B29z6-xPIPLEcENMWnh1MklPdTQ
v1.0 as zip: https://docs.google.com/file/d/0B29z6-xPIPLEQWtibWk3T3AtVjA
Is anyone interested in getting involved / taking over? Overall, it seems doable and not too complex. I'd be happy about any help! Also, if you need more information, just ask me. I'll provide you everything I have about this little device.
Cheers,
Tolga Cakir
Quoted 1 line · Tolga Cakir
Hello everyone!
Hi Tolga!
Quoted 6 lines
Over the past weeks, I've been busy capturing USB packets between the Elgato Game Capture HD and my PC. It's using the MStar MST3367CMK chip, which seems to have proprietary Linux support available only for hardware vendors in form of an SDK. Problem is, that this SDK is strictly kept under an NDA, making it kinda impossible for us to get our hands on.
Thanks for raising the subject.
While your comment is true, it would have been more appropriate to the development community to say that it truly uses the Fujitsu USB encoder, a fujitsu USB API along with a series of smaller subsystems for HDMI receivers and transmitters. Your capture logs indicate (largely) interaction with the Fujitsu USB bridge + integral encoder. The distinction is important.
We outlined the architecture of the device (along with the brief tear down) here: http://www.kernellabs.com/blog/?p=1959
Quoted 6 lines
So, I got my hands dirty and have found some very good stuff! First of all, in contrast to many sources, the Elgato Game Capture HD outputs compressed video and audio via USB! It's already encoded, so there is no need for reencoding, this will save CPU power. For testing purposes, I've only tried capturing 720p data for now, but this should be more than enough.
Have you posted any source code? I don't see any in the zips or on github.
Paging through a 600MB usb capture to find an occasional comment (assuming you have inserted them) doesn't encourage me to contribute.
Quoted 9 lines
Basically, we need to read raw USB traffic, write an MPEG-TS file header, put in the raw USB data and close the file. I'm not super experienced in C / kernel development (especially V4L), but I'll give my best to get this project forward. My next step is getting a prototype working with libusb in userland; after that's done, I'll try porting it over to kernel / V4L
Project page can be found here:
https://github.com/tolga9009/elgato-gchd
I must be missing something. your repo contains a LICENSE file and README. Did you forget to checking a homebrew datasheet or working sample source code?
Quoted 5 lines
USB logs and docs:
v1.0 as 7zip: https://docs.google.com/file/d/0B29z6-xPIPLEQVBMTWZHbUswYjg
v1.0 as rar: https://docs.google.com/file/d/0B29z6-xPIPLEcENMWnh1MklPdTQ
v1.0 as zip: https://docs.google.com/file/d/0B29z6-xPIPLEQWtibWk3T3AtVjA
Ahh, thank you for circulating the datasheets and images from our blog post, you are most welcome! The internet is a wonderful thing, I'm glad you found them useful.
Quoted 5 lines
Is anyone interested in getting involved / taking over? Overall, it seems doable and not too complex. I'd be happy about any help! Also, if you need more information, just ask me. I'll provide you everything I have about this little device.
How about instead of some usb dumps, pictures and pdfs, a working program and a description of the device protocol? This would help.
I spent a few days late 2012 with the usb analyzer and brought together a primitive collection of personal notes on the API. Sadly I'm struggling to locate them currently. From memory, the device has an odd protocol which isn't exactly obvious. Its firmware like, not i2c based. You don't appear to control the HDMI rx/tx silicon by hand, the fijutsu firmware does this via firmware APIs. you would think, YAY! firmware API, easy, surprisingly not. A lot of byte guess to be done. If you have any significant homebrew documentation on the byte sequences that control the device, this would help.
Part of the problem is that the device also streams (with the windows/osx drivers I was using) permanently on, making it difficult to see the wood from the noise. So, even when you are not 'using it', its streaming payload via USB to the host. Urgh. I hope they've fixed this.
The device outputs native ISO13818 TS packets which are easily playable in VLC as is. I don't even think you need to add a header, unless you are electing to create an updated PMT.
I have datasheets and/or source on everything except the fujitsu encoder, sorry - I can't share.
Keep going with your project, this should be a fun to follow. libusb is easy to work with, you should have the device running in no time.
If you can make the device run at both 720p and 1080i then you should find enough variance in the protocol bytes, build that into your app, to be useful for some people.
- Steve
Signature · 2 lines
Steven Toth - Kernel Labs http://www.kernellabs.com
Am 09.05.2014 um 14:38 schrieb Steven Toth:
Quoted 81 lines · Tolga Cakir
Quoted 1 line
Hello everyone!
Hi Tolga!
Quoted 5 lines
Over the past weeks, I've been busy capturing USB packets between the Elgato Game Capture HD and my PC. It's using the MStar MST3367CMK chip, which seems to have proprietary Linux support available only for hardware vendors in form of an SDK. Problem is, that this SDK is strictly kept under an NDA, making it kinda impossible for us to get our hands on.
Thanks for raising the subject.
While your comment is true, it would have been more appropriate to the development community to say that it truly uses the Fujitsu USB encoder, a fujitsu USB API along with a series of smaller subsystems for HDMI receivers and transmitters. Your capture logs indicate (largely) interaction with the Fujitsu USB bridge + integral encoder. The distinction is important.
We outlined the architecture of the device (along with the brief tear down) here: http://www.kernellabs.com/blog/?p=1959
Quoted 5 lines
So, I got my hands dirty and have found some very good stuff! First of all, in contrast to many sources, the Elgato Game Capture HD outputs compressed video and audio via USB! It's already encoded, so there is no need for reencoding, this will save CPU power. For testing purposes, I've only tried capturing 720p data for now, but this should be more than enough.
Have you posted any source code? I don't see any in the zips or on github.
Paging through a 600MB usb capture to find an occasional comment (assuming you have inserted them) doesn't encourage me to contribute.
Quoted 8 lines
Basically, we need to read raw USB traffic, write an MPEG-TS file header, put in the raw USB data and close the file. I'm not super experienced in C / kernel development (especially V4L), but I'll give my best to get this project forward. My next step is getting a prototype working with libusb in userland; after that's done, I'll try porting it over to kernel / V4L
Project page can be found here:
https://github.com/tolga9009/elgato-gchdI must be missing something. your repo contains a LICENSE file and README. Did you forget to checking a homebrew datasheet or working sample source code?
Quoted 4 lines
USB logs and docs:
v1.0 as 7zip: https://docs.google.com/file/d/0B29z6-xPIPLEQVBMTWZHbUswYjg
v1.0 as rar: https://docs.google.com/file/d/0B29z6-xPIPLEcENMWnh1MklPdTQ
v1.0 as zip: https://docs.google.com/file/d/0B29z6-xPIPLEQWtibWk3T3AtVjAAhh, thank you for circulating the datasheets and images from our blog post, you are most welcome! The internet is a wonderful thing, I'm glad you found them useful.
Quoted 4 lines
Is anyone interested in getting involved / taking over? Overall, it seems doable and not too complex. I'd be happy about any help! Also, if you need more information, just ask me. I'll provide you everything I have about this little device.
How about instead of some usb dumps, pictures and pdfs, a working program and a description of the device protocol? This would help.
I spent a few days late 2012 with the usb analyzer and brought together a primitive collection of personal notes on the API. Sadly I'm struggling to locate them currently. From memory, the device has an odd protocol which isn't exactly obvious. Its firmware like, not i2c based. You don't appear to control the HDMI rx/tx silicon by hand, the fijutsu firmware does this via firmware APIs. you would think, YAY! firmware API, easy, surprisingly not. A lot of byte guess to be done. If you have any significant homebrew documentation on the byte sequences that control the device, this would help.
Part of the problem is that the device also streams (with the windows/osx drivers I was using) permanently on, making it difficult to see the wood from the noise. So, even when you are not 'using it', its streaming payload via USB to the host. Urgh. I hope they've fixed this.
The device outputs native ISO13818 TS packets which are easily playable in VLC as is. I don't even think you need to add a header, unless you are electing to create an updated PMT.
I have datasheets and/or source on everything except the fujitsu encoder, sorry - I can't share.
Quoted 8 lines
Keep going with your project, this should be a fun to follow. libusb is easy to work with, you should have the device running in no time.
If you can make the device run at both 720p and 1080i then you should find enough variance in the protocol bytes, build that into your app, to be useful for some people.
- Steve
Hi Steven!
thank you for your hints so far! They really helped me moving forward. 6 months have passed and I was able to learn more about the device. I have captured some more USB logs using the OpenVizsla USB 2.0 analyzer, comparing Windows and Mac drivers side-by-side. There are some findings, I want to share with you:
- The device has 5 endpoints, excluding EP0. EP1 and EP4 is used for video / audio transmission and supports bulk transfers only; EP2 is used for booting firmwares (which are not flashed onto the device, like FPGAs - so, no brick chance!); EP5 is used for flashing a firmware onto the device and shouldn't be touched by us. I have no idea about EP3, as I haven't come across of it yet. You can find the output of "lsusb -v" here: https://github.com/tolga9009/elgato-gchd/blob/master/descriptors/lsusb_usb_descriptor
- Setting up the device is done in several, non-trivial steps. There are 2 firmwares loaded onto the device (which we were able to transfer onto the device!).
- I have come across a bootup script, which was bundled with the Elgato Mac drivers: https://drive.google.com/file/d/0B29z6-xPIPLEZTZhSzJiQTVoR3c/view?usp=sharing. This is a very interesting, documented script. You should definitely take a look at it. The device seems to be using I2C for configuration. I wasn't able to translate between I2C and the USB packets so far, but I'm sure it can be done.
I'm currently stuck after successfully loading the 2nd firmware onto the device. That's a point, where things are getting really complicated. I have studied and compared the (new) USB capture logs countless hours, but wasn't able to find anything to move forward. Can you look into this a second time? You can find everything in the GitHub repo: https://github.com/tolga9009/elgato-gchd. By the way: Elgato has improved alot. The new v2.0 Mac drivers stripped away tons of unnecessary USB packets. That helps concentrating on the important stuff.
Thank you!
Cheers,
Tolga
Comments inline
Quoted 102 lines · Tolga Cakir
Am 09.05.2014 um 14:38 schrieb Steven Toth:
Quoted 101 lines · Tolga Cakir
Quoted 2 lines
Hello everyone!
Hi Tolga!
Quoted 7 lines
Over the past weeks, I've been busy capturing USB packets between the Elgato
Game Capture HD and my PC. It's using the MStar MST3367CMK chip, which seems
to have proprietary Linux support available only for hardware vendors in form of an SDK. Problem is, that this SDK is strictly kept under an NDA, making it kinda impossible for us to get our hands on.Thanks for raising the subject.
While your comment is true, it would have been more appropriate to the development community to say that it truly uses the Fujitsu USB encoder, a fujitsu USB API along with a series of smaller subsystems for HDMI receivers and transmitters. Your capture logs indicate (largely) interaction with the Fujitsu USB bridge + integral encoder. The distinction is important.
We outlined the architecture of the device (along with the brief tear down) here: http://www.kernellabs.com/blog/?p=1959
Quoted 8 lines
So, I got my hands dirty and have found some very good stuff! First of all,
in contrast to many sources, the Elgato Game Capture HD outputs compressed
video and audio via USB! It's already encoded, so there is no need for reencoding, this will save CPU power. For testing purposes, I've only tried
capturing 720p data for now, but this should be more than enough.Have you posted any source code? I don't see any in the zips or on github.
Paging through a 600MB usb capture to find an occasional comment (assuming you have inserted them) doesn't encourage me to contribute.
Quoted 10 lines
Basically, we need to read raw USB traffic, write an MPEG-TS file header, put in the raw USB data and close the file. I'm not super experienced in C /
kernel development (especially V4L), but I'll give my best to get this project forward. My next step is getting a prototype working with libusb in
userland; after that's done, I'll try porting it over to kernel / V4LProject page can be found here:
https://github.com/tolga9009/elgato-gchdI must be missing something. your repo contains a LICENSE file and README. Did you forget to checking a homebrew datasheet or working sample source code?
Quoted 4 lines
USB logs and docs:
v1.0 as 7zip: https://docs.google.com/file/d/0B29z6-xPIPLEQVBMTWZHbUswYjg
v1.0 as rar: https://docs.google.com/file/d/0B29z6-xPIPLEcENMWnh1MklPdTQ
v1.0 as zip: https://docs.google.com/file/d/0B29z6-xPIPLEQWtibWk3T3AtVjAAhh, thank you for circulating the datasheets and images from our blog post, you are most welcome! The internet is a wonderful thing, I'm glad you found them useful.
Quoted 6 lines
Is anyone interested in getting involved / taking over? Overall, it seems doable and not too complex. I'd be happy about any help! Also, if you need
more information, just ask me. I'll provide you everything I have about this
little device.How about instead of some usb dumps, pictures and pdfs, a working program and a description of the device protocol? This would help.
I spent a few days late 2012 with the usb analyzer and brought together a primitive collection of personal notes on the API. Sadly I'm struggling to locate them currently. From memory, the device has an odd protocol which isn't exactly obvious. Its firmware like, not i2c based. You don't appear to control the HDMI rx/tx silicon by hand, the fijutsu firmware does this via firmware APIs. you would think, YAY! firmware API, easy, surprisingly not. A lot of byte guess to be done. If you have any significant homebrew documentation on the byte sequences that control the device, this would help.
Part of the problem is that the device also streams (with the windows/osx drivers I was using) permanently on, making it difficult to see the wood from the noise. So, even when you are not 'using it', its streaming payload via USB to the host. Urgh. I hope they've fixed this.
The device outputs native ISO13818 TS packets which are easily playable in VLC as is. I don't even think you need to add a header, unless you are electing to create an updated PMT.
I have datasheets and/or source on everything except the fujitsu encoder, sorry - I can't share.
Keep going with your project, this should be a fun to follow. libusb is easy to work with, you should have the device running in no time.
Quoted 8 lines
Quoted 5 lines
If you can make the device run at both 720p and 1080i then you should find enough variance in the protocol bytes, build that into your app, to be useful for some people.
- Steve
Hi Steven!
Hello Tolga.
Quoted 19 lines
thank you for your hints so far! They really helped me moving forward. 6 months have passed and I was able to learn more about the device. I have captured some more USB logs using the OpenVizsla USB 2.0 analyzer, comparing Windows and Mac drivers side-by-side. There are some findings, I want to share with you:
- The device has 5 endpoints, excluding EP0. EP1 and EP4 is used for video / audio transmission and supports bulk transfers only; EP2 is used for booting firmwares (which are not flashed onto the device, like FPGAs - so, no brick chance!); EP5 is used for flashing a firmware onto the device and shouldn't be touched by us. I have no idea about EP3, as I haven't come across of it yet. You can find the output of "lsusb -v" here:
https://github.com/tolga9009/elgato-gchd/blob/master/descriptors/lsusb_usb_descriptor- Setting up the device is done in several, non-trivial steps. There are 2 firmwares loaded onto the device (which we were able to transfer onto the device!).
Quoted 7 lines
- I have come across a bootup script, which was bundled with the Elgato Mac drivers:
https://drive.google.com/file/d/0B29z6-xPIPLEZTZhSzJiQTVoR3c/view?usp=sharing.
This is a very interesting, documented script. You should definitely take a look at it. The device seems to be using I2C for configuration. I wasn't able to translate between I2C and the USB packets so far, but I'm sure it can be done.
Based on my first glance, I don't think the driver was talking directly to the parts via i2c, it was requesting features from the driver (such as "set hdmi mode") and the firmware itself was making that happen. Those are generally a little more difficult to debug, but not impossible.
My earlier comment is the best advise I can give you:
"If you can make the device run at both 720p and 1080i then you should find enough variance in the protocol bytes, build that into your app, to be useful for some people."
^^^ This is the technique I usually begin with - for any undocumented hardware.
Quoted 3 lines
I'm currently stuck after successfully loading the 2nd firmware onto the device. That's a point, where things are getting really complicated. I have
Ah yes, the joy of complication. Keep at it.
Quoted 2 lines
studied and compared the (new) USB capture logs countless hours, but wasn't able to find anything to move forward. Can you look into this a second time?
I'm swamped with a number of other projects and probably won't get a chance to look at this in any detail, but I'd be happy to comment on any specific patterns or control byte sequences that you think may be relevant.
Quoted 4 lines
You can find everything in the GitHub repo:
https://github.com/tolga9009/elgato-gchd. By the way: Elgato has improved alot. The new v2.0 Mac drivers stripped away tons of unnecessary USB packets. That helps concentrating on the important stuff.
great!
Quoted 2 lines
Thank you!
No, thank you.
- Steve
Signature · 2 lines
Steven Toth - Kernel Labs http://www.kernellabs.com
Elgato Game Capture HD on Linux
Hi Steven,
I wish you a happy new year, it's been a while. I've made great success on the Elgato Game Capture HD driver for Linux - the major features are working. Currently, it's supporting capturing / streaming HDMI with 1080p30 and 720p60, and component with 1080p30, 1080i60, 720p60 and 576p50 (PAL). Eventhough I've just quickly hacked together everything, I've learned alot.
The official Mac OS X drivers included files with comments. They were my cheatsheet. One important thing, I've just recently figured out: the official driver is not doing stuff sequentially, like I've always assumed. It's doing many things in parallel using threads. This is the reason, why reverse engineering has been a nightmare for me, cause lots of unrelated, random packets showed up and I couldn't interpret them. No two USB logs looked nearly the same - there were always differences. Sometimes major, sometimes minor.
There seems to be one thread, which is trying to receive bulk data on endpoints 0x04 and 0x01 (maybe even one thread per endpoint?), without any sleep. This was causing LOTS of USB log entries in conjunction with my USB hardware analyzer - talking about 5 - 8 million lines for 20 seconds.
Another thread is constantly checking the input source's information, like resolution (small sleeps, unlike bulk receive). If it changes, this thread writes new, customized configuration to the device. I have logged lots of different traffic and am sure, that I can track this down.
There seems to be another thread, which seems to do just the things, I was actually trying to track down in the first place: plain simple device setup / initialization. Just sending over some packets, like bitrate configuration and video output resolution. Luckily, most of these values are commented in Elgato's official Mac OS X driver and I've also figured out, how most of them work (https://github.com/tolga9009/elgato-gchd/blob/master/commands.h).
Eventhough lots and lots of refactoring is urgently waiting for me (honestly, I'm ashamed of it), I think it's atleast a good point to start. I'm gonna work on refactoring / tidying up next major hacking session in about 2 - 3 months. You can find the latest code here: https://github.com/tolga9009/elgato-gchd. Looking forward to port this over to V4L2, once the refactoring is done.
If you're interested in how things turned out, I've even uploaded a small video presentation on YouTube: https://youtu.be/rpm6TJu6HkE?t=6m34s. You can skip it to around 6:30, cause I'm just showing how to compile & firmware extraction first.
Cheers,
Tolga