tm6000 calculating urb buffer

Am 30.04.2010 18:34, schrieb Stefan Ringel:

Quoted 5 lines

Hi Mauro,

Today I'm writing directly to you, because it doesn't work the mailing list. I thought over the calculating urb buffer and I have follow idea:

Quoted 9 lines

buffer = endpoint fifo size (3072 Bytes) * block size (184 Bytes)

The actually calculating is a video frame size (image = width * hight * 2 Bytes/Pixel), so that this buffer has to begin and to end an uncomplete block. followed blocks are setting the logic to an err_mgs block, so that going to lost frames.

I forgot a log with old calculating.

Am 30.04.2010 21:31, schrieb Stefan Ringel:

Quoted 9 lines

Am 30.04.2010 18:34, schrieb Stefan Ringel:

Quoted 7 lines

Hi Mauro,

Today I'm writing directly to you, because it doesn't work the mailing list. I thought over the calculating urb buffer and I have follow idea:

buffer = endpoint fifo size (3072 Bytes) * block size (184 Bytes)

Quoted 10 lines

Quoted 7 lines

The actually calculating is a video frame size (image = width * hight * 2 Bytes/Pixel), so that this buffer has to begin and to end an uncomplete block. followed blocks are setting the logic to an err_mgs block, so that going to lost frames.

I forgot a log with old calculating.

datagram from urb to videobuf

urb copy to temp copy to 1. videobuf
buffer 2. audiobuf
3. vbi
184 Packets -------> 184 * 3072 ----------> 4. etc.
a 3072 bytes bytes

               184 *                   3072 *
             3072 bytes              180 bytes
                                (184 bytes - 4 bytes
                                    header )

step 1

copy from urb to temp buffer

snip
----

for (i = 0; i < urb->number_of_packets; i++) {
	int status = urb->iso_frame_desc[i].status;
	
	if (status<0) {
		print_err_status (dev,i,status);
		continue;
	}

	len=urb->iso_frame_desc[i].actual_length;

	memcpy (t_buf[i*len], urb->transfer_buffer[i*len], len);
	copied += len;
	if (copied >= size || !buf)
		break;

}

if (!urb->iso_frame_desc[i].status) {
	if ((buf->fmt->fourcc)==V4L2_PIX_FMT_TM6000) {
		rc=copy_multiplexed(t_buf, outp, len, urb, &buf);

		if (rc<=0)
			return rc;
	} else {
		copy_streams(t_buf, outp, len, urb, &buf);
	}
}

---
snip

step 2

copy from temp buffer into videobuffer

snip
---

for (i=0;i<3072;i++) {

	switch(cmd) {
		case TM6000_URB_MSG_VIDEO:
			/* Fills video buffer */
			memcpy(&out_p[(line << 1 + field) * block * 180],
				ptr[(i*184)+4], 180);
			printk (KERN_INFO "cmd=%s, size=%d\n",
			tm6000_msg_type[cmd],size);
			break;
		case TM6000_URB_MSG_PTS:
			printk (KERN_INFO "cmd=%s, size=%d\n",
			tm6000_msg_type[cmd],size);
			break;
		case TM6000_URB_MSG_AUDIO:
			/* Need some code to process audio */
			printk ("%ld: cmd=%s, size=%d\n", jiffies,
			tm6000_msg_type[cmd],size);
			break;
		default:
			dprintk (dev, V4L2_DEBUG_ISOC, "cmd=%s, size=%d\n",
			printk (KERN_INFO "cmd=%s, size=%d\n",
			tm6000_msg_type[cmd],size);
		}
	}
}

---
snip

This is a schemata to copy in videobuf.

temp_buf = fifo size * block size

viodeobuf = hight * wight * 2

Questions

1. Is it right if I copy the block without header to videobufer?
2. Can I full the videobuffer have more temp_bufs?
3. How are the actually data schema from urb to videobuffer?

Hi Stefan,

Quoted 30 lines · Stefan Ringel

Am 30.04.2010 21:31, schrieb Stefan Ringel:

Quoted 9 lines

Am 30.04.2010 18:34, schrieb Stefan Ringel:

Quoted 7 lines

Hi Mauro,

Today I'm writing directly to you, because it doesn't work the mailing list. I thought over the calculating urb buffer and I have follow idea:

buffer = endpoint fifo size (3072 Bytes) * block size (184 Bytes)

Quoted 10 lines

Quoted 7 lines

The actually calculating is a video frame size (image = width * hight * 2 Bytes/Pixel), so that this buffer has to begin and to end an uncomplete block. followed blocks are setting the logic to an err_mgs block, so that going to lost frames.

I forgot a log with old calculating.

datagram from urb to videobuf

urb copy to temp copy to 1. videobuf
buffer 2. audiobuf
3. vbi
184 Packets -------> 184 * 3072 ----------> 4. etc.
a 3072 bytes bytes

Quoted 4 lines

               184 *                   3072 *
             3072 bytes              180 bytes
                                (184 bytes - 4 bytes
                                    header )

In order to receive 184 packets with 3072 bytes each, the USB code will try to allocate the next power-of-two memory block capable of receiving such data block. As: 184 * 3072 = 565248, the kernel allocator will seek for a continuous block of 1 MB, that can do DMA transfers (required by ehci driver). On a typical machine, due to memory fragmentation, in general, there aren't many of such blocks. So, this will increase the probability of not having any such large block available, causing an

dump at kernel, plus a -ENOMEM on the driver, generally requiring a reboot if you want to run the driver again.

Quoted 5 lines

step 1

copy from urb to temp buffer

Why do you want to do triple buffering? This is a very bad idea. If you do it at the wrong way, by handling the copy at interrupt time, you're eating more power (and batteries, on notebooks), and reducing the machine speed. If you split it into two halves, you'll need a larger buffer area, since kernel will eventually join a few consecutive

into one, to avoid damaging other kernel process. Also, it will risk

frames or introduce a high delay.

It is already bad enough to have a double buffering with those usb

Just as an example, the last time I've measured em28xx driver performance, after doing lots of optimization at the code, it were still consuming about 30% of CPU time of the machine I used for test (a typical mono-core Intel CPU).

I know that the code would be simpler if we use a temporary buffer, but this way, we save CPU time. Also, if we do triple buffering, you'll likely add some delay when syncing between audio and video, due to the workqueue time.

So, in summary, what we need to do is to validate the code and simplify it to be faster. If you take a look at tm6000-video.c, you'll see that I've tried already some different approaches. The one that is currently working is the first approach I did. As the newer solutions didn't solve the loss of data, but introduced newer bugs, I did a rollback to the code. At

I stopped working on tm6000, I was about to write a new (simpler) approach, but still avoiding the double buffering.

Quoted 18 lines

snip
----

for (i = 0; i < urb->number_of_packets; i++) {
    int status = urb->iso_frame_desc[i].status;
   
    if (status<0) {
        print_err_status (dev,i,status);
        continue;
    }

    len=urb->iso_frame_desc[i].actual_length;

    memcpy (t_buf[i*len], urb->transfer_buffer[i*len], len);
    copied += len;
    if (copied >= size || !buf)
        break;

Quoted 5 lines

}

if (!urb->iso_frame_desc[i].status) {
    if ((buf->fmt->fourcc)==V4L2_PIX_FMT_TM6000) {
        rc=copy_multiplexed(t_buf, outp, len, urb, &buf);

copy_multiplexed() is about what you want: It just copies everything (except for the URB headers), into a buffer, allowing decoding the data on userspace. There's an userspace application that gets those data, at v4l-utils tree. With this approach, you may add a decoder at libv4l for TM6000 format, and let userspace to do the audio/video/TS decoding.

Quoted 11 lines

        if (rc<=0)
            return rc;
    } else {
        copy_streams(t_buf, outp, len, urb, &buf);
    }
}

---
snip

step 2

Quoted 6 lines

copy from temp buffer into videobuffer

snip
---

for (i=0;i<3072;i++) {

Doesn't work: nothing warrants that the device will start with a frame.

Quoted 40 lines

    switch(cmd) {
        case TM6000_URB_MSG_VIDEO:
            /* Fills video buffer */
            memcpy(&out_p[(line << 1 + field) * block * 180],
                ptr[(i*184)+4], 180);
            printk (KERN_INFO "cmd=%s, size=%d\n",
            tm6000_msg_type[cmd],size);
            break;
        case TM6000_URB_MSG_PTS:
            printk (KERN_INFO "cmd=%s, size=%d\n",
            tm6000_msg_type[cmd],size);
            break;
        case TM6000_URB_MSG_AUDIO:
            /* Need some code to process audio */
            printk ("%ld: cmd=%s, size=%d\n", jiffies,
            tm6000_msg_type[cmd],size);
            break;
        default:
            dprintk (dev, V4L2_DEBUG_ISOC, "cmd=%s, size=%d\n",
            printk (KERN_INFO "cmd=%s, size=%d\n",
            tm6000_msg_type[cmd],size);
        }
    }
}

---
snip

This is a schemata to copy in videobuf.

temp_buf = fifo size * block size

viodeobuf = hight * wight * 2

Questions

1. Is it right if I copy the block without header to videobufer?
2. Can I full the videobuffer have more temp_bufs?
3. How are the actually data schema from urb to videobuffer?

Quoted 11 lines

Quoted 11 lines

datagram from urb to videobuf

urb copy to temp copy to 1. videobuf
buffer 2. audiobuf
3. vbi
184 Packets -------> 184 * 3072 ----------> 4. etc.
a 3072 bytes bytes

               184 *                   3072 *
             3072 bytes              180 bytes
                                (184 bytes - 4 bytes
                                    header )

Quoted 7 lines

In order to receive 184 packets with 3072 bytes each, the USB code will try to allocate the next power-of-two memory block capable of receiving such data block. As: 184 * 3072 = 565248, the kernel allocator will seek for a continuous block of 1 MB, that can do DMA transfers (required by ehci driver). On a typical machine, due to memory fragmentation, in general, there aren't many of such blocks. So, this will increase the probability of not having any such large block available, causing an

horrible

Quoted 3 lines

dump at kernel, plus a -ENOMEM on the driver, generally requiring a reboot if you want to run the driver again.

And direct copy from urb to videobuf/alsa/vbi in 184 Bytes segments.

urb 1. videobuf
copy to 2. audiobuf
3. vbi
184 Packets -------> 4. etc.
a 3072 bytes
180 Bytes (without headers)

or how can I copy 180 Bytes Data from 184 Bytes block with an anligment of 184 urb pipe (184 * 3072 Bytes)?

Quoted 18 lines · Stefan Ringel

Quoted 18 lines

Quoted 11 lines

datagram from urb to videobuf

urb copy to temp copy to 1. videobuf
buffer 2. audiobuf
3. vbi
184 Packets -------> 184 * 3072 ----------> 4. etc.
a 3072 bytes bytes

               184 *                   3072 *
             3072 bytes              180 bytes
                                (184 bytes - 4 bytes
                                    header )

In order to receive 184 packets with 3072 bytes each, the USB code will try to allocate the next power-of-two memory block capable of receiving such data block. As: 184 * 3072 = 565248, the kernel allocator will seek for a continuous block of 1 MB, that can do DMA transfers (required by ehci driver). On a typical machine, due to memory fragmentation, in general, there aren't many of such blocks. So, this will increase the probability of not having any such large block available, causing an

Quoted 2 lines

horrible

Quoted 1 line

dump at kernel, plus a -ENOMEM on the driver, generally requiring a

Quoted 4 lines

Quoted 2 lines

if you want to run the driver again.

And direct copy from urb to videobuf/alsa/vbi in 184 Bytes segments.

Quoted 6 lines

urb 1. videobuf
copy to 2. audiobuf
3. vbi
184 Packets -------> 4. etc.
a 3072 bytes
180 Bytes (without headers)

That's basically what that logic does. It preserves the header if you

TM6000 format (so, no checks for the start of the block, etc), or copies just the data, if you select YUY2 or UYUV.

Quoted 2 lines

or how can I copy 180 Bytes Data from 184 Bytes block with an anligment of 184 urb pipe (184 * 3072 Bytes)?

A 184 x 3072 URB pipe is a big problem. We used a large pipe in the

won't work. For example, on a notebook I used to run some tests with 1

ram after starting X and do anything (like opening a browser), the URB allocation used to fail, as there weren't any available 1MB segment at the DMA area. Even without starting X, after a few tests, it would

have fragmented the memory and the driver stops working.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Am 04.05.2010 21:50, schrieb Mauro Carvalho Chehab:

Quoted 20 lines

Quoted 18 lines · Stefan Ringel

Quoted 18 lines

Quoted 11 lines

datagram from urb to videobuf

urb copy to temp copy to 1. videobuf
buffer 2. audiobuf
3. vbi
184 Packets -------> 184 * 3072 ----------> 4. etc.
a 3072 bytes bytes

               184 *                   3072 *
             3072 bytes              180 bytes
                                (184 bytes - 4 bytes
                                    header )

In order to receive 184 packets with 3072 bytes each, the USB code will try to allocate the next power-of-two memory block capable of receiving such data block. As: 184 * 3072 = 565248, the kernel allocator will seek for a continuous block of 1 MB, that can do DMA transfers (required by ehci driver). On a typical machine, due to memory fragmentation, in general, there aren't many of such blocks. So, this will increase the probability of not having any such large block available, causing an

Quoted 2 lines

Quoted 2 lines

horrible

Quoted 1 line

dump at kernel, plus a -ENOMEM on the driver, generally requiring a

reboot

Quoted 11 lines

Quoted 10 lines

Quoted 2 lines

if you want to run the driver again.

And direct copy from urb to videobuf/alsa/vbi in 184 Bytes segments.

urb 1. videobuf
copy to 2. audiobuf
3. vbi
184 Packets -------> 4. etc.
a 3072 bytes
180 Bytes (without headers)

That's basically what that logic does. It preserves the header if you

select

Quoted 6 lines

TM6000 format (so, no checks for the start of the block, etc), or copies just the data, if you select YUY2 or UYUV.

Quoted 2 lines

or how can I copy 180 Bytes Data from 184 Bytes block with an anligment of 184 urb pipe (184 * 3072 Bytes)?

A 184 x 3072 URB pipe is a big problem. We used a large pipe in the

past, and this

Quoted 1 line

won't work. For example, on a notebook I used to run some tests with 1

GB of

Quoted 3 lines

ram after starting X and do anything (like opening a browser), the URB allocation used to fail, as there weren't any available 1MB segment at the DMA area. Even without starting X, after a few tests, it would

eventually

Quoted 3 lines

have fragmented the memory and the driver stops working.

and 3072 * 46 = 141312 bytes and it can through 184 ! it's 1/4 smaller.