Skip to contentSkip to conversation

V4L2 Advanced Codec questions

DISCUSSION Steven Toth 3 participants 5 messages Mar 26, 2009 – Mar 28, 2009 Media (V4L/DVB)

lore
Share & tools
lore
#0 lore raw

Hello!

I want to open a couple of HVR22xx items up for discussion.

The HVR-22xx analog encoder is capable of encoded to all kinds of video and audio codecs in various containers formats.

From memory, wm9, mpeg4, mpeg2, divx, AAC, AC3, Windows audio codecs in asf, ts, ps, avi containers, depending on various firmware license enablements and configuration options. Maybe more, maybe, I'll draw up a complete list when I begin to focus on analog.

Any single encoder on the HVR22xx can produce (if licensed) any of the formats above. However, due to a lack of CPU horsepower in the RISC engine, the board is not completely symmetrical when the encoders are running concurrently. This is the main reason why Hauppauge have disabled these features in the windows driver.

It's possible for example to get two concurrent MPEG2 PS streams but only if the bitrate is limited to 6Mbps, which we also do in the windows driver.

Apart from the fact that we (the LinuxTV community) will need to determine what's possible concurrently, and what isn't, it does raise interesting issues for the V4L2 API.

So, how do we expose this advanced codec and hardware encoder limitation information through v4l2 to the applications?

Do we, don't we?

Suggestions?

- Steve

#1 lore raw
Quoted 10 lines · Steven Toth

I want to open a couple of HVR22xx items up for discussion.

The HVR-22xx analog encoder is capable of encoded to all kinds of video and audio codecs in various containers formats.

From memory, wm9, mpeg4, mpeg2, divx, AAC, AC3, Windows audio codecs in asf,
ts, ps, avi containers, depending on various firmware license enablements and configuration options. Maybe more, maybe, I'll draw up a complete list when I begin to focus on analog.

Well, the current list is already impressive. I guess MPEG audio layer 2 or 3 are missing.

Quoted 4 lines

Any single encoder on the HVR22xx can produce (if licensed) any of the formats above. However, due to a lack of CPU horsepower in the RISC engine, the board is not completely symmetrical when the encoders are running concurrently. This is the main reason why Hauppauge have disabled these features in the windows driver.

I guess that will also be a problem within the v4l2 driver since user space has generally no idea if two video devices share hardware.

Quoted 9 lines

It's possible for example to get two concurrent MPEG2 PS streams but only if the bitrate is limited to 6Mbps, which we also do in the windows driver.

Apart from the fact that we (the LinuxTV community) will need to determine what's possible concurrently, and what isn't, it does raise interesting issues for the V4L2 API.

So, how do we expose this advanced codec and hardware encoder limitation information through v4l2 to the applications?

One possibility is adding an mpeg control which request concurrent usage of the hardware encoder and limits the control values to allow concurrent usage. If the control is not set, the hardware encoder will be only useable on one device. User space applications need to discover the devices sharing hardware and can set the concurrent mode if they wishes.

Current applications won't break but can only use hardware decoding on a single device. A module option could force concurrent mode in which case only the encoding parameters are limited but both devices can use hardware encoding.

Janne

#2 lore raw
In reply to Steven Toth
Quoted 22 lines · On Thursday 26 March 2009 16:59:11 Steven Toth wrote:

Hello!

I want to open a couple of HVR22xx items up for discussion.

The HVR-22xx analog encoder is capable of encoded to all kinds of video and audio codecs in various containers formats.

From memory, wm9, mpeg4, mpeg2, divx, AAC, AC3, Windows audio codecs in
asf, ts, ps, avi containers, depending on various firmware license enablements and configuration options. Maybe more, maybe, I'll draw up a complete list when I begin to focus on analog.

Any single encoder on the HVR22xx can produce (if licensed) any of the formats above. However, due to a lack of CPU horsepower in the RISC engine, the board is not completely symmetrical when the encoders are running concurrently. This is the main reason why Hauppauge have disabled these features in the windows driver.

It's possible for example to get two concurrent MPEG2 PS streams but only if the bitrate is limited to 6Mbps, which we also do in the windows driver.

Quoted 8 lines

Apart from the fact that we (the LinuxTV community) will need to determine what's possible concurrently, and what isn't, it does raise interesting issues for the V4L2 API.

So, how do we expose this advanced codec and hardware encoder limitation information through v4l2 to the applications?

Do we, don't we?

Hi Steve,

If I understand it correctly, then a single analog source can be encoded to multiple formats at the same time, right?

Or is it that multiple analog sources can each be encoded to some format? Or a combination of both?

Is there a limit to the number of concurrent encoders (except for CPU horsepower)?

Basically, since you can have multiple encoders, you also need multiple videoX nodes, once for each encoder. And I would expect that an application can just setup each encoder. Whenever you start an encoder the driver might either accept it or return -ENOSPC if there aren't enough resources.

You have to document the restrictions in a document, but otherwise I don't see any reason why implementing this would cause any problems.

Adding new containers and codecs is easy: just add the missing ones to enum v4l2_mpeg_stream_type, v4l2_mpeg_audio_encoding and v4l2_mpeg_video_encoding and add any additional controls that are needed to implement each codec/container.

In theory you can reduce the number of possible containers/codecs/bitrates in the controls according to the remaining resources. But I think that will be too complicated to do for too little gain, not only in the driver but also in the application.

Regards,

Hans

Signature · 1 line
Hans Verkuil - video4linux developer - sponsored by TANDBERG
#3 lore raw
Quoted 27 lines · Hans Verkuil
Quoted 26 lines · On Thursday 26 March 2009 16:59:11 Steven Toth wrote:

Hello!

I want to open a couple of HVR22xx items up for discussion.

The HVR-22xx analog encoder is capable of encoded to all kinds of video and audio codecs in various containers formats.

From memory, wm9, mpeg4, mpeg2, divx, AAC, AC3, Windows audio codecs in
asf, ts, ps, avi containers, depending on various firmware license enablements and configuration options. Maybe more, maybe, I'll draw up a complete list when I begin to focus on analog.

Any single encoder on the HVR22xx can produce (if licensed) any of the formats above. However, due to a lack of CPU horsepower in the RISC engine, the board is not completely symmetrical when the encoders are running concurrently. This is the main reason why Hauppauge have disabled these features in the windows driver.

It's possible for example to get two concurrent MPEG2 PS streams but only if the bitrate is limited to 6Mbps, which we also do in the windows driver.

Apart from the fact that we (the LinuxTV community) will need to determine what's possible concurrently, and what isn't, it does raise interesting issues for the V4L2 API.

Quoted 8 lines
Quoted 4 lines

So, how do we expose this advanced codec and hardware encoder limitation information through v4l2 to the applications?

Do we, don't we?

Hi Steve,

If I understand it correctly, then a single analog source can be encoded to multiple formats at the same time, right?

No.

Quoted 2 lines

Or is it that multiple analog sources can each be encoded to some format? Or a combination of both?

Multiple analog sources can produce multiple formats, CPU power permitting.

Quoted 2 lines

Is there a limit to the number of concurrent encoders (except for CPU horsepower)?

Not that I'm aware of, yet.

Quoted 2 lines

Basically, since you can have multiple encoders, you also need multiple videoX nodes, once for each encoder. And I would expect that an application can just setup each encoder. Whenever you start an encoder the driver might either accept it or return -ENOSPC if there aren't enough resources.

This is fine, and expected.

Quoted 4 lines

You have to document the restrictions in a document, but otherwise I don't see any reason why implementing this would cause any problems.

Adding new containers and codecs is easy: just add the missing ones to enum v4l2_mpeg_stream_type, v4l2_mpeg_audio_encoding and v4l2_mpeg_video_encoding and add any additional controls that are needed to implement each codec/container.

Ahh, this is the magic information I was looking for.

Quoted 2 lines

In theory you can reduce the number of possible containers/codecs/bitrates in the controls according to the remaining resources. But I think that will be too complicated to do for too little gain, not only in the driver but also in the application.

I think this is going to be the major issue and it will start to reflect itself through the API into the application. We'll see, maybe not.

Thanks,

Steve

#4 lore raw
Quoted 38 lines · On Saturday 28 March 2009 18:03:47 Steven Toth wrote:
Quoted 30 lines · Hans Verkuil
Quoted 29 lines · On Thursday 26 March 2009 16:59:11 Steven Toth wrote:

Hello!

I want to open a couple of HVR22xx items up for discussion.

The HVR-22xx analog encoder is capable of encoded to all kinds of video and audio codecs in various containers formats.

From memory, wm9, mpeg4, mpeg2, divx, AAC, AC3, Windows audio codecs
in asf, ts, ps, avi containers, depending on various firmware license enablements and configuration options. Maybe more, maybe, I'll draw up a complete list when I begin to focus on analog.

Any single encoder on the HVR22xx can produce (if licensed) any of the formats above. However, due to a lack of CPU horsepower in the RISC engine, the board is not completely symmetrical when the encoders are running concurrently. This is the main reason why Hauppauge have disabled these features in the windows driver.

It's possible for example to get two concurrent MPEG2 PS streams but only if the bitrate is limited to 6Mbps, which we also do in the windows driver.

Apart from the fact that we (the LinuxTV community) will need to determine what's possible concurrently, and what isn't, it does raise interesting issues for the V4L2 API.

So, how do we expose this advanced codec and hardware encoder limitation information through v4l2 to the applications?

Quoted 6 lines
Quoted 1 line

Do we, don't we?

Hi Steve,

If I understand it correctly, then a single analog source can be encoded to multiple formats at the same time, right?

Quoted 7 lines

No.

Quoted 2 lines

Or is it that multiple analog sources can each be encoded to some format? Or a combination of both?

Multiple analog sources can produce multiple formats, CPU power permitting.

So it is always: 'one source -> one encoder', right? Never 'one source -> multiple encoders'?

Quoted 5 lines
Quoted 3 lines

Is there a limit to the number of concurrent encoders (except for CPU horsepower)?

Not that I'm aware of, yet.

There must be a limit to the number of sources? Or can this device act as a codec as well: you present it with raw video from memory and it compresses it to e.g. mpeg?

Quoted 19 lines
Quoted 6 lines

Basically, since you can have multiple encoders, you also need multiple videoX nodes, once for each encoder. And I would expect that an application can just setup each encoder. Whenever you start an encoder the driver might either accept it or return -ENOSPC if there aren't enough resources.

This is fine, and expected.

Quoted 7 lines

You have to document the restrictions in a document, but otherwise I don't see any reason why implementing this would cause any problems.

Adding new containers and codecs is easy: just add the missing ones to enum v4l2_mpeg_stream_type, v4l2_mpeg_audio_encoding and v4l2_mpeg_video_encoding and add any additional controls that are needed to implement each codec/container.

Ahh, this is the magic information I was looking for.

Quoted 7 lines
Quoted 4 lines

In theory you can reduce the number of possible containers/codecs/bitrates in the controls according to the remaining resources. But I think that will be too complicated to do for too little gain, not only in the driver but also in the application.

I think this is going to be the major issue and it will start to reflect itself through the API into the application. We'll see, maybe not.

The first step will be to get a single encoder going :-)

At this stage I don't know enough about the capabilities of the hardware to tell how important it is to have such info available. I suspect any solution to this might well need something like a media controller device about which I've written before, which can be used to present such meta information.

I'm ever more of the opinion that v4l needs such a top-level device as v4l hardware becomes more complex.

Regards,

Hans

Signature · 1 line
Hans Verkuil - video4linux developer - sponsored by TANDBERG