Videobuf improvements to allow its usage with USB drivers

Videobuf Improvements for USB Devices
=====================================

Videobuf history
================

If you are familiar with V4L and kernel development, you probably know that one
of the hardest part of the driver is to control the video stream that arrives
from the device, and need to be sent to an userspace application.

To address this task, a kernel module, called
videobuf, is widely used for
quite a long time at the PCI devices.

However, previously, this driver were restricted to DMA devices. Specifically,
it used to work only with PCI devices whose steams are provided via DMA, and
for chipsets that supports scatter/gather mode.

On scatter/gatter DMA mode, the data I/O is splitted into several small buffers.
This worked fine for bttv, saa7134 and cx88 devices, whose PCI bridge is
capable of handling such transfers.

However, due to their specific hardware restrictions, videobuf couldn't be used
by USB devices. Due to that, each V4L USB driver had to implement their own
buffering schema.

I've started some time ago a project to make
videobuf more generic. In order to
to that, videobuf were splitted into two files:

- videobuf-core - with core features, not specific
to DMA or PCI;
- videobuf-dma-sg - for PCI DMA scatter/gather
devices.

After that, I've created another videobuf instance, called videobuf-vmalloc.

This one uses memory alloced with vmalloc_user(). This is the same approach used
by other USB drivers. The first test for the newer core driver were to port the
virtual video driver (vivi) to work with it.

The videobuf split revealed several bad locks inside the driver.

Several developers helped to solve those issues,
including: Brandon Philips,
Nick Piggin, Jonathan Corbet, Trent Piepho, Adrian Bunk and Andrew Morton.

Also, Guennadi Liakhovetski removed PCI specific
details, on videobuf-dma-sg.
Now, the same driver can also be used by other
architectures that don't provide
PCI interfaces, like ARM.

videobuf for USB devices
========================

The last round for the code improvement just
happened this week: the conversion
of em28xx to use videobuf.

This round happened thanks to the help of Aidan
Thornton, that got a proposed
patch I send him, and help me to fix and address
several issues on this complex
task.

So, the first USB driver that is working perfectly is available for testing at:

	http://linuxtv.org/hg/~mchehab/em28xx-vb

This driver were tested with several widely used userspace apps: tvtime,
mplayer, xawtv and mythtv. It were also tested with the testing tool v4l2-apps/test/code_example, available inside the tree.

I'll port the changesets soon to the master
development trees.

The new approach has several advantages over the old one:
- buffering code inside em28xx-video is now clean
and easy to
understand;

- the same buffering code can be easily ported to
other USB drivers;

- by using the same videobuf code, all drivers will
have the same
behaviour. This will help userspace apps to be more independent of specific
devices;

- the performance of the newer code is much more
optimized than the
previous code;

- redundant streaming handling code is now inside
V4L core;

- It is now easy to add overlay and userptr support
for those drivers;

- It is now easy to use videobuf-dvb for USB
devices also.

Things yet to be done
=====================

1) videobuf operating memory modes:

The old videobuf and the newvideobuf-dma-sg supports all streaming modes
present at V4L1 and V4L2 API:

	VIDIOCMBUF;
	reading /dev/video?;
	kernel mmapped memory;
	userspace mmapped memory;
	overlay mode, to send video input streams directly

into video adapter memory.

The new videobuf-vmalloc shares the same core stuff. However, it currently
doesn't implement userspace mmapped memory or overlay mode. The other three
modes are already supported.

It doesn't seem to be hard to add the missing modes. Probably, the only
function that will need more code is videobuf_iolock(). I've already started to code a patch to add userspace mmap. This may be useful to allow some userspace
apps that relies on this method to work.

2) videobuf-dvb

This driver allows using the same videobuf handling also for DVB devices. This
driver works fine with videobuf-dma-sg. Theoretically, it should work fine also with USB drivers.

3) tm6000

This is the first driver I've made to use videobuf. Unfortunately, the driver
is loosing URB frames. I suspect that it is a hardware problem. I expect to
finish this driver soon, since there are several new TV devices using this
chipset.

4) porting other usb drivers to use videobuf

This will help to cleanup their source code, and fix some API non-compliance.

5) videobuf for non scatter/gather DMA

There are a few devices in the market that supports only contiguous DMA
transfer. This is the case, for example of Marvel cafe chips, used on OLPC.

For those devices, it would be interesting to create a new videobuf module, and
migrate them to this solution.

I'd like to thank all of you that helped with this development, and to the
improvement of the Linux kernel support for video input devices.

APPENDIX: Performance tests of the both versions of em28xx driver

This is the performance tests I did, running code_example to get 1,000 frames
@29.995 Hz (about 35 seconds of stream), tested on a i386 machine, running at
1,5GHz:

=== message truncated ===
This is the performance tests I did, running >

@29.995 Hz (about 35 seconds of stream), tested on a

running at
1,5GHz:

The old driver:

$ time -f "%E: %Us User time, %Ss Kernel time, %P

./capture_example
0:34.21: 8.22s User time, 25.16s Kernel time, 97%

The videobuf-based driver:

$ time -f "%E: %Us User time, %Ss Kernel time, %P

./capture_example
0:35.36: 0.01s User time, 0.05s Kernel time, 0% CPU

Conclusion:

The time consumption to receive the stream where

seconds to 0.05 seconds

Quoted 3 lines

Conclusion:

The time consumption to receive the stream where reduced from about 33.38 seconds to 0.05 seconds

the question is moreover what made capture_example go up to 100% CPU in the first try and to 0% in the second one.
I'm not sure about the old implementation in the original driver, although I'm just curious about the details here. xawtv usually uses very little cputime at all.
If I use
"$ time mplayer tv:// -tv driver=v4l2" it shows up

real 0m40.972s
user 0m0.230s
sys 0m0.050s

your benchmark is a bit unclear.

Quoted 3 lines · On Sun, 13 Apr 2008 18:17:54 +0200 (CEST) Markus Rechberger…

Quoted 3 lines

Conclusion:

The time consumption to receive the stream where

reduced from about 33.38 seconds to 0.05 seconds

Quoted 2 lines

the question is moreover what made capture_example

go

Quoted 4 lines

up to 100% CPU in the first try and to 0% in the
second one.
I'm not sure about the old implementation in the
original driver, although I'm just curious about

the

Quoted 1 line

details here. xawtv usually uses very little

cputime

Quoted 10 lines

at all.
If I use
"$ time mplayer tv:// -tv driver=v4l2" it shows up

real 0m40.972s
user 0m0.230s
sys 0m0.050s

your benchmark is a bit unclear.

The advantage of using capture_example for benchmark tests is that it is a very
simple mmap loop, without multi-thread, and just discarding the return. With
this, you're timing just the minimal requirements for receiving frames.

A TV application will also need to use the video adapter to present images, and
may do some other tasks, like running DSP algorithms for de-interlacing. It may
also discard frames, if there's not enough CPU to work will all of them. So,
you will never know how much of those times are due to V4L kernelspace part.

On the tests I did here with TV applications, the amount of performance,
reported by "top" also indicated that the previous approach were worse.

For example, on the same centino machine @1.5 GHZ, mplayer with "-tv driver=v4l2"
were ranging from 30% to 75% of CPU. With videobuf, the CPU consumption were
close to 23%, without much variation.

--- Mauro Carvalho Chehab <[email protected]>

Quoted 49 lines · schrieb:

Quoted 3 lines · On Sun, 13 Apr 2008 18:17:54 +0200 (CEST) Markus Rechberger…

Quoted 3 lines

Conclusion:

The time consumption to receive the stream where

reduced from about 33.38 seconds to 0.05 seconds

Quoted 2 lines

the question is moreover what made capture_example

go

Quoted 4 lines

up to 100% CPU in the first try and to 0% in the
second one.
I'm not sure about the old implementation in the
original driver, although I'm just curious about

the

Quoted 1 line

details here. xawtv usually uses very little

cputime

Quoted 10 lines

at all.
If I use
"$ time mplayer tv:// -tv driver=v4l2" it shows up

real 0m40.972s
user 0m0.230s
sys 0m0.050s

your benchmark is a bit unclear.

The advantage of using capture_example for benchmark tests is that it is a very
simple mmap loop, without multi-thread, and just discarding the return. With
this, you're timing just the minimal requirements for receiving frames.

A TV application will also need to use the video adapter to present images, and
may do some other tasks, like running DSP algorithms for de-interlacing. It may
also discard frames, if there's not enough CPU to work will all of them. So,
you will never know how much of those times are due to V4L kernelspace part.

On the tests I did here with TV applications, the amount of performance,
reported by "top" also indicated that the previous approach were worse.

Quoted 6 lines

For example, on the same centino machine @1.5 GHZ, mplayer with "-tv driver=v4l2"
were ranging from 30% to 75% of CPU. With videobuf, the CPU consumption were
close to 23%, without much variation.

my eeePC shows up 0-5% CPU usage with mplayer
fullscreen without videobuf, seems more like
something's broken in your testapplication or
somewhere else?

mplayer uses the memory mapped interface for this.

Also the command
"$ mplayer -benchmark tv:// -tv driver=v4l2"

CPU Intel 620 Mhz

Markus

--
video4linux-list mailing list
Unsubscribe mailto:[email protected]?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Quoted 8 lines · Markus Rechberger

--- Mauro Carvalho Chehab <[email protected]>

Quoted 2 lines · schrieb:

On Sun, 13 Apr 2008 18:17:54 +0200 (CEST) Markus Rechberger <[email protected]>

Quoted 4 lines · wrote:

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Conclusion:

The time consumption to receive the stream

where

Quoted 1 line

Quoted 1 line

reduced from about 33.38 seconds to 0.05

seconds

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

the question is moreover what made

capture_example

Quoted 2 lines

Quoted 2 lines

go

Quoted 1 line

up to 100% CPU in the first try and to 0% in

the

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

second one.
I'm not sure about the old implementation in

the

Quoted 1 line

Quoted 1 line

Quoted 1 line

original driver, although I'm just curious

about

Quoted 6 lines

Quoted 6 lines

the

Quoted 1 line

details here. xawtv usually uses very little

cputime

Quoted 3 lines

at all.
If I use
"$ time mplayer tv:// -tv driver=v4l2" it

shows up

Quoted 9 lines

Quoted 9 lines

Quoted 7 lines

real 0m40.972s
user 0m0.230s
sys 0m0.050s

your benchmark is a bit unclear.

The advantage of using capture_example for

benchmark

Quoted 2 lines

Quoted 2 lines

tests is that it is a very
simple mmap loop, without multi-thread, and

just

Quoted 2 lines

Quoted 2 lines

discarding the return. With this, you're timing just the minimal

requirements

Quoted 3 lines

Quoted 3 lines

for receiving frames.

A TV application will also need to use the

video

Quoted 2 lines

Quoted 2 lines

adapter to present images, and
may do some other tasks, like running DSP

algorithms

Quoted 2 lines

Quoted 2 lines

for de-interlacing. It may
also discard frames, if there's not enough CPU

to

Quoted 2 lines

Quoted 2 lines

work will all of them. So,
you will never know how much of those times are

due

Quoted 3 lines

Quoted 3 lines

to V4L kernelspace part.

On the tests I did here with TV applications,

the

Quoted 2 lines

Quoted 2 lines

amount of performance,
reported by "top" also indicated that the

previous

Quoted 3 lines

Quoted 3 lines

approach were worse.

For example, on the same centino machine @1.5

GHZ,

Quoted 2 lines

Quoted 2 lines

mplayer with "-tv driver=v4l2"
were ranging from 30% to 75% of CPU. With

videobuf,

Quoted 4 lines

Quoted 3 lines

the CPU consumption were
close to 23%, without much variation.

Quoted 5 lines

my eeePC shows up 0-5% CPU usage with mplayer
fullscreen without videobuf, seems more like
something's broken in your testapplication or
somewhere else?

Using the em28xx?
My is about 40-70% CPU...

I am new to this user group. Will someone help?

I have mailed 2 queries for help to '[email protected]'. First, I emailed this on Apr 9, but have had no response to them. 2nd, in looking at
the archive list, the queries are not even listed.

What am I doing wrong?

Quoted 8 lines · --- Karl Kobata

I am new to this user group. Will someone help?

I have mailed 2 queries for help to '[email protected]'. First, I emailed this on Apr 9, but have had no response to them. 2nd, in looking at
the archive list, the queries are not even listed.

Quoted 2 lines

What am I doing wrong?

good question, probably because your mail didn't get through for some reason.
At least I don't have anything in my mail account.

Markus

--
video4linux-list mailing list
Unsubscribe mailto:[email protected]?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

I am new to this user group. Will someone help?

I have mailed 2 queries for help to '[email protected]'. First, I emailed this on Apr 9, but have had no response to them. 2nd, in looking at the archive list, the queries are not even listed.

my eeePC shows up 0-5% CPU usage with mplayer
fullscreen without videobuf, seems more like
something's broken in your testapplication or
somewhere else?

Quoted 4 lines

my eeePC shows up 0-5% CPU usage with mplayer
fullscreen without videobuf, seems more like
something's broken in your testapplication or
somewhere else?

The test application (capture_example) is the one documented at the V4L2 spec.
The only difference is that I've incremented count to 1000, to get more frames.
I don't see what's wrong on it.

--- Mauro Carvalho Chehab <[email protected]>

Quoted 11 lines · schrieb:

Quoted 4 lines

my eeePC shows up 0-5% CPU usage with mplayer
fullscreen without videobuf, seems more like
something's broken in your testapplication or
somewhere else?

The test application (capture_example) is the one documented at the V4L2 spec.
The only difference is that I've incremented count to 1000, to get more frames.
I don't see what's wrong on it.

I just tested capture_example on the eeePC (again non videobuf).

$ time ./capture_example
.................. (printed this around 100 times?)
real   0m4.312s
user   0m0.010s
sys    0m0.000s

strace clearly shows up VIDIOC_QBUF, VIDIOC_DQBUF.

So the question is rather what makes the results so bad on your system.
How can the userspace application go up to 100%, while the system isn't that busy?

Quoted 10 lines · On Sun, 13 Apr 2008 23:38:24 +0200 (CEST) Markus Rechberger…

--- Mauro Carvalho Chehab <[email protected]>

Quoted 6 lines · schrieb:

Quoted 4 lines

my eeePC shows up 0-5% CPU usage with mplayer
fullscreen without videobuf, seems more like
something's broken in your testapplication or
somewhere else?

The test application (capture_example) is the

one

Quoted 2 lines

Quoted 2 lines

documented at the V4L2 spec.
The only difference is that I've incremented

count

Quoted 5 lines

Quoted 3 lines

to 1000, to get more frames.
I don't see what's wrong on it.

I just tested capture_example on the eeePC (again

non

Quoted 4 lines

videobuf).

$ time ./capture_example
.................. (printed this around 100

times?)

Quoted 7 lines

real   0m4.312s
user   0m0.010s
sys    0m0.000s

strace clearly shows up VIDIOC_QBUF, VIDIOC_DQBUF.

So the question is rather what makes the results

so

Quoted 2 lines

bad on your system.
How can the userspace application go up to 100%,

while

Quoted 1 line

the system isn't that busy?

Are you running your tests against the in-kernel
version?

Also, you're running
with count = 100 (the 100 dots). You need to edit
the example code, and move to
1000 to have the same basement as I used here.

--- Mauro Carvalho Chehab <[email protected]>

Quoted 2 lines · schrieb:

On Sun, 13 Apr 2008 23:38:24 +0200 (CEST)
Markus Rechberger <[email protected]>

Quoted 3 lines · wrote:

Quoted 3 lines

--- Mauro Carvalho Chehab

<[email protected]>

Quoted 3 lines

Quoted 3 lines

Quoted 1 line · schrieb:

Quoted 1 line

my eeePC shows up 0-5% CPU usage with

mplayer

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

fullscreen without videobuf, seems more like
something's broken in your testapplication

or

Quoted 12 lines

Quoted 3 lines

Quoted 3 lines

Quoted 1 line

somewhere else?

The test application (capture_example) is the

one

Quoted 2 lines

Quoted 2 lines

documented at the V4L2 spec.
The only difference is that I've incremented

count

Quoted 5 lines

Quoted 3 lines

to 1000, to get more frames.
I don't see what's wrong on it.

I just tested capture_example on the eeePC

(again

Quoted 11 lines

non

Quoted 4 lines

videobuf).

$ time ./capture_example
.................. (printed this around 100

times?)

Quoted 5 lines

real   0m4.312s
user   0m0.010s
sys    0m0.000s

strace clearly shows up VIDIOC_QBUF,

VIDIOC_DQBUF.

Quoted 10 lines

Quoted 2 lines

So the question is rather what makes the results

so

Quoted 2 lines

bad on your system.
How can the userspace application go up to 100%,

while

Quoted 1 line

the system isn't that busy?

Are you running your tests against the in-kernel
version?

v4l-dvb-experimental/em28xx-userspace 2 from
mcentral.de use the same algorithms.

I ran capture_example again with count=1000

first run:
real 0m40.393s
user 0m0.000s
sys 0m0.020s
second run:
real 0m40.294s
user 0m0.000s
sys 0m0.030s
third run:
real 0m40.394s
user 0m0.000s
sys 0m0.010s