V4L2 Advanced Codec questions
Hello!
I want to open a couple of HVR22xx items up for discussion.
The HVR-22xx analog encoder is capable of encoded to all kinds of video and audio codecs in various containers formats.
From memory, wm9, mpeg4, mpeg2, divx, AAC, AC3, Windows audio codecs in asf, ts, ps, avi containers, depending on various firmware license enablements and configuration options. Maybe more, maybe, I'll draw up a complete list when I begin to focus on analog.
Any single encoder on the HVR22xx can produce (if licensed) any of the formats above. However, due to a lack of CPU horsepower in the RISC engine, the board is not completely symmetrical when the encoders are running concurrently. This is the main reason why Hauppauge have disabled these features in the windows driver.
It's possible for example to get two concurrent MPEG2 PS streams but only if the bitrate is limited to 6Mbps, which we also do in the windows driver.
Apart from the fact that we (the LinuxTV community) will need to determine what's possible concurrently, and what isn't, it does raise interesting issues for the V4L2 API.
So, how do we expose this advanced codec and hardware encoder limitation information through v4l2 to the applications?
Do we, don't we?
Suggestions?
- Steve
Quoted 10 lines · Steven Toth
I want to open a couple of HVR22xx items up for discussion.
The HVR-22xx analog encoder is capable of encoded to all kinds of video and audio codecs in various containers formats.
From memory, wm9, mpeg4, mpeg2, divx, AAC, AC3, Windows audio codecs in asf,
ts, ps, avi containers, depending on various firmware license enablements and configuration options. Maybe more, maybe, I'll draw up a complete list when I begin to focus on analog.
Well, the current list is already impressive. I guess MPEG audio layer 2 or 3 are missing.
Quoted 4 lines
Any single encoder on the HVR22xx can produce (if licensed) any of the formats above. However, due to a lack of CPU horsepower in the RISC engine, the board is not completely symmetrical when the encoders are running concurrently. This is the main reason why Hauppauge have disabled these features in the windows driver.
I guess that will also be a problem within the v4l2 driver since user space has generally no idea if two video devices share hardware.
Quoted 9 lines
It's possible for example to get two concurrent MPEG2 PS streams but only if the bitrate is limited to 6Mbps, which we also do in the windows driver.
Apart from the fact that we (the LinuxTV community) will need to determine what's possible concurrently, and what isn't, it does raise interesting issues for the V4L2 API.
So, how do we expose this advanced codec and hardware encoder limitation information through v4l2 to the applications?
One possibility is adding an mpeg control which request concurrent usage of the hardware encoder and limits the control values to allow concurrent usage. If the control is not set, the hardware encoder will be only useable on one device. User space applications need to discover the devices sharing hardware and can set the concurrent mode if they wishes.
Current applications won't break but can only use hardware decoding on a single device. A module option could force concurrent mode in which case only the encoding parameters are limited but both devices can use hardware encoding.
Janne
Quoted 22 lines · On Thursday 26 March 2009 16:59:11 Steven Toth wrote:
Hello!
I want to open a couple of HVR22xx items up for discussion.
The HVR-22xx analog encoder is capable of encoded to all kinds of video and audio codecs in various containers formats.
From memory, wm9, mpeg4, mpeg2, divx, AAC, AC3, Windows audio codecs in
asf, ts, ps, avi containers, depending on various firmware license enablements and configuration options. Maybe more, maybe, I'll draw up a complete list when I begin to focus on analog.Any single encoder on the HVR22xx can produce (if licensed) any of the formats above. However, due to a lack of CPU horsepower in the RISC engine, the board is not completely symmetrical when the encoders are running concurrently. This is the main reason why Hauppauge have disabled these features in the windows driver.
It's possible for example to get two concurrent MPEG2 PS streams but only if the bitrate is limited to 6Mbps, which we also do in the windows driver.
Quoted 8 lines
Apart from the fact that we (the LinuxTV community) will need to determine what's possible concurrently, and what isn't, it does raise interesting issues for the V4L2 API.
So, how do we expose this advanced codec and hardware encoder limitation information through v4l2 to the applications?
Do we, don't we?
Hi Steve,
If I understand it correctly, then a single analog source can be encoded to multiple formats at the same time, right?
Or is it that multiple analog sources can each be encoded to some format? Or a combination of both?
Is there a limit to the number of concurrent encoders (except for CPU horsepower)?
Basically, since you can have multiple encoders, you also need multiple videoX nodes, once for each encoder. And I would expect that an application can just setup each encoder. Whenever you start an encoder the driver might either accept it or return -ENOSPC if there aren't enough resources.
You have to document the restrictions in a document, but otherwise I don't see any reason why implementing this would cause any problems.
Adding new containers and codecs is easy: just add the missing ones to enum v4l2_mpeg_stream_type, v4l2_mpeg_audio_encoding and v4l2_mpeg_video_encoding and add any additional controls that are needed to implement each codec/container.
In theory you can reduce the number of possible containers/codecs/bitrates in the controls according to the remaining resources. But I think that will be too complicated to do for too little gain, not only in the driver but also in the application.
Regards,
Hans
Signature · 1 line
Hans Verkuil - video4linux developer - sponsored by TANDBERG
Quoted 27 lines · Hans Verkuil
Quoted 26 lines · On Thursday 26 March 2009 16:59:11 Steven Toth wrote:
Hello!
I want to open a couple of HVR22xx items up for discussion.
The HVR-22xx analog encoder is capable of encoded to all kinds of video and audio codecs in various containers formats.
From memory, wm9, mpeg4, mpeg2, divx, AAC, AC3, Windows audio codecs in
asf, ts, ps, avi containers, depending on various firmware license enablements and configuration options. Maybe more, maybe, I'll draw up a complete list when I begin to focus on analog.Any single encoder on the HVR22xx can produce (if licensed) any of the formats above. However, due to a lack of CPU horsepower in the RISC engine, the board is not completely symmetrical when the encoders are running concurrently. This is the main reason why Hauppauge have disabled these features in the windows driver.
It's possible for example to get two concurrent MPEG2 PS streams but only if the bitrate is limited to 6Mbps, which we also do in the windows driver.
Apart from the fact that we (the LinuxTV community) will need to determine what's possible concurrently, and what isn't, it does raise interesting issues for the V4L2 API.
Quoted 8 lines
Quoted 4 lines
So, how do we expose this advanced codec and hardware encoder limitation information through v4l2 to the applications?
Do we, don't we?
Hi Steve,
If I understand it correctly, then a single analog source can be encoded to multiple formats at the same time, right?
No.
Quoted 2 lines
Or is it that multiple analog sources can each be encoded to some format? Or a combination of both?
Multiple analog sources can produce multiple formats, CPU power permitting.
Quoted 2 lines
Is there a limit to the number of concurrent encoders (except for CPU horsepower)?
Not that I'm aware of, yet.
Quoted 2 lines
Basically, since you can have multiple encoders, you also need multiple videoX nodes, once for each encoder. And I would expect that an application can just setup each encoder. Whenever you start an encoder the driver might either accept it or return
-ENOSPCif there aren't enough resources.
This is fine, and expected.
Quoted 4 lines
You have to document the restrictions in a document, but otherwise I don't see any reason why implementing this would cause any problems.
Adding new containers and codecs is easy: just add the missing ones to enum
v4l2_mpeg_stream_type,v4l2_mpeg_audio_encodingandv4l2_mpeg_video_encodingand add any additional controls that are needed to implement each codec/container.
Ahh, this is the magic information I was looking for.
Quoted 2 lines
In theory you can reduce the number of possible containers/codecs/bitrates in the controls according to the remaining resources. But I think that will be too complicated to do for too little gain, not only in the driver but also in the application.
I think this is going to be the major issue and it will start to reflect itself through the API into the application. We'll see, maybe not.
Thanks,
Steve
Quoted 38 lines · On Saturday 28 March 2009 18:03:47 Steven Toth wrote:
Quoted 30 lines · Hans Verkuil
Quoted 29 lines · On Thursday 26 March 2009 16:59:11 Steven Toth wrote:
Hello!
I want to open a couple of HVR22xx items up for discussion.
The HVR-22xx analog encoder is capable of encoded to all kinds of video and audio codecs in various containers formats.
From memory, wm9, mpeg4, mpeg2, divx, AAC, AC3, Windows audio codecs
in asf, ts, ps, avi containers, depending on various firmware license enablements and configuration options. Maybe more, maybe, I'll draw up a complete list when I begin to focus on analog.Any single encoder on the HVR22xx can produce (if licensed) any of the formats above. However, due to a lack of CPU horsepower in the RISC engine, the board is not completely symmetrical when the encoders are running concurrently. This is the main reason why Hauppauge have disabled these features in the windows driver.
It's possible for example to get two concurrent MPEG2 PS streams but only if the bitrate is limited to 6Mbps, which we also do in the windows driver.
Apart from the fact that we (the LinuxTV community) will need to determine what's possible concurrently, and what isn't, it does raise interesting issues for the V4L2 API.
So, how do we expose this advanced codec and hardware encoder limitation information through v4l2 to the applications?
Quoted 6 lines
Quoted 1 line
Do we, don't we?
Hi Steve,
If I understand it correctly, then a single analog source can be encoded to multiple formats at the same time, right?
Quoted 7 lines
No.
Quoted 2 lines
Or is it that multiple analog sources can each be encoded to some format? Or a combination of both?
Multiple analog sources can produce multiple formats, CPU power permitting.
So it is always: 'one source -> one encoder', right? Never 'one source -> multiple encoders'?
Quoted 5 lines
Quoted 3 lines
Is there a limit to the number of concurrent encoders (except for CPU horsepower)?
Not that I'm aware of, yet.
There must be a limit to the number of sources? Or can this device act as a codec as well: you present it with raw video from memory and it compresses it to e.g. mpeg?
Quoted 19 lines
Quoted 6 lines
Basically, since you can have multiple encoders, you also need multiple videoX nodes, once for each encoder. And I would expect that an application can just setup each encoder. Whenever you start an encoder the driver might either accept it or return
-ENOSPCif there aren't enough resources.This is fine, and expected.
Quoted 7 lines
You have to document the restrictions in a document, but otherwise I don't see any reason why implementing this would cause any problems.
Adding new containers and codecs is easy: just add the missing ones to enum
v4l2_mpeg_stream_type,v4l2_mpeg_audio_encodingandv4l2_mpeg_video_encodingand add any additional controls that are needed to implement each codec/container.Ahh, this is the magic information I was looking for.
Quoted 7 lines
Quoted 4 lines
In theory you can reduce the number of possible containers/codecs/bitrates in the controls according to the remaining resources. But I think that will be too complicated to do for too little gain, not only in the driver but also in the application.
I think this is going to be the major issue and it will start to reflect itself through the API into the application. We'll see, maybe not.
The first step will be to get a single encoder going :-)
At this stage I don't know enough about the capabilities of the hardware to tell how important it is to have such info available. I suspect any solution to this might well need something like a media controller device about which I've written before, which can be used to present such meta information.
I'm ever more of the opinion that v4l needs such a top-level device as v4l hardware becomes more complex.
Regards,
Hans
Signature · 1 line
Hans Verkuil - video4linux developer - sponsored by TANDBERG