Skip to contentSkip to conversation

Linux tty layer hackery: Heads up and RFC

DISCUSSION Alan Cox 4 participants 7 messages Jul 21, 2005 – Jul 26, 2005

lore
Share & tools
lore

Alan Cox

author
#0 lore raw

At the moment tty buffers are attached directly to the tty. This is causing a lot of the problems related to tty layer locking, also problems at high speed and also with bursty data (such as occurs in virtualised environments)

I'm working on ripping out the flip buffers and replacing them with a pool of dynamically allocated buffers. This allows both for old style "byte I/O" devices and also helps virtualisation and smart devices where large blocks of data suddenely materialise and need storing.

So far so good. Lots of drivers reference tty->flip.*. Several of them also call directly and unsafely into function pointers it provides. This will all break. Most drivers can use tty_insert_flip_char which can be kept as an API but others need more.

At the moment I've added the following interfaces, if people think more will be needed now is a good time to say

int tty_buffer_request_room(tty, size)

Try and ensure at least size bytes are available, returns actual room (may be zero). At the moment it just uses the flipbuf space but that will change. Repeated calls without characters being added are not cumulative. (ie if you call it with 1, 1, 1, and then 4 you'll have four characters of space. The other functions will also try and grow buffers in future but this will be a more efficient way when you know block sizes.

int tty_insert_flip_char(tty, ch, flag)

As before insert a character if there is room. Now returns 1 for success, 0 for failure.

int tty_insert_flip_string(tty, str, len)

Insert a block of non error characters. Returns the number inserted.

int tty_prepare_flip_string(tty, strptr, len)

Adjust the buffer to allow len characters to be added. Returns a buffer pointer in strptr and the length available. This allows for hardware that needs to use functions like insl or mencpy_fromio.

I've converted a fair number of drivers to this API ready and I'll post some patches for review soon.

Alan

#1 lore raw
Quoted 9 lines · Alan Cox

At the moment tty buffers are attached directly to the tty. This is causing a lot of the problems related to tty layer locking, also problems at high speed and also with bursty data (such as occurs in virtualised environments)

I'm working on ripping out the flip buffers and replacing them with a pool of dynamically allocated buffers. This allows both for old style "byte I/O" devices and also helps virtualisation and smart devices where large blocks of data suddenely materialise and need storing.

Great! Really good news!

Quoted 26 lines

So far so good. Lots of drivers reference tty->flip.*. Several of them also call directly and unsafely into function pointers it provides. This will all break. Most drivers can use tty_insert_flip_char which can be kept as an API but others need more.

At the moment I've added the following interfaces, if people think more will be needed now is a good time to say

int tty_buffer_request_room(tty, size)

Try and ensure at least size bytes are available, returns actual room (may be zero). At the moment it just uses the flipbuf space but that will change. Repeated calls without characters being added are not cumulative. (ie if you call it with 1, 1, 1, and then 4 you'll have four characters of space. The other functions will also try and grow buffers in future but this will be a more efficient way when you know block sizes.

int tty_insert_flip_char(tty, ch, flag)

As before insert a character if there is room. Now returns 1 for success, 0 for failure.

int tty_insert_flip_string(tty, str, len)

Quoted 7 lines

Insert a block of non error characters. Returns the number inserted.

int tty_prepare_flip_string(tty, strptr, len)

Adjust the buffer to allow len characters to be added. Returns a buffer pointer in strptr and the length available. This allows for hardware that needs to use functions like insl or mencpy_fromio.

As you are going to replace flip buffers with different implementation anyway, isn't it better to get rid of "flip" in the interface names as well (maybe providing some synonyms for backward compatibility)? What I mean is that names like

tty_buffer_insert_char()
tty_buffer_insert_string()

for the new interfaces would probably make more sense.

Otherwise I find the interfaces you suggest just fine and suitable for the task I was unable to achieve without ugly hacks using current flip buffers interfaces (reliable high-speed bulk USB tty driver).

Signature · 1 line
Sergei.
#2 lore raw
In reply to Alan Cox
Quoted 5 lines · Alan Cox

int tty_prepare_flip_string(tty, strptr, len)

Adjust the buffer to allow len characters to be added. Returns a buffer pointer in strptr and the length available. This allows for hardware that needs to use functions like insl or mencpy_fromio.

Ok, So then I start copying characters into the flipstring, but how do I say I'm done?

Or is there a race between that I call tty_prepare_flip_string, and other processes start pulling my not-yet-filled string from the buffer? (Surely not!)

Roger.

Signature · 5 lines
** [email protected] ** http://www.BitWizard.nl/ ** +31-15-2600998 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous statement. 
Does it sit on the couch all day? Is it unemployed? Please be specific! 
Define 'it' and what it isn't doing. --------- Adapted from lxrbot FAQ
author
#3 lore raw
Quoted 2 lines · Rogier Wolff

Ok, So then I start copying characters into the flipstring, but how do I say I'm done?

tty_flip_buffer_push() (or its possibly renamed equivalent). At that point as now the buffer may get processed or queued to the ldisc. At that point its now owned by the ldisc and the tty buffer layer will give you new buffers.

#4 lore raw
In reply to Rogier Wolff

Hi Rogier,

Having just written a DMA UART driver I can say this is good news :-). Here are some things to think about:

What my driver would like to do is to handle its own input buffers. It would pass the buffer to the tty layer when it is full and the tty layer would pass the buffer back once it has drained the data from it. The problem is that I don't always receive a block worth of characters so I also need to pass the tty layer a buffer (which I'm still DMAing into) with a count of how many chars there are in the buffer and a offset of where to start from. If I can't do this then I have to stop the DMA transfer, pass you a mostly empty buffer with a char count, and setup DMA transfer into another buffer. If during this I get a burst of data I could well lose it :-(.

Best Regards,

Mark

Quoted 15 lines · --- Rogier Wolff
Quoted 3 lines · Alan Cox

int tty_prepare_flip_string(tty, strptr, len)

Adjust the buffer to allow len characters to be

added. Returns a buffer

Quoted 1 line

pointer in strptr and the length available. This

allows for hardware

Quoted 1 line

that needs to use functions like insl or

mencpy_fromio.

Ok, So then I start copying characters into the flipstring, but how do
I say I'm done?

Quoted 27 lines

Or is there a race between that I call tty_prepare_flip_string, and
other processes start pulling my not-yet-filled string from the
buffer? (Surely not!)

Roger.

Signature · 17 lines
** [email protected] **
http://www.BitWizard.nl/ ** +31-15-2600998 **
*-- BitWizard writes Linux device drivers for any
device you may have! --*
Q: It doesn't work. A: Look buddy, doesn't work is
an ambiguous statement. 
Does it sit on the couch all day? Is it unemployed?
Please be specific! 
Define 'it' and what it isn't doing. ---------
Adapted from lxrbot FAQ
-
To unsubscribe from this list: send the line
"unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at 
http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

___________________________________________________________
Yahoo! Messenger - NEW crystal clear PC to PC calling worldwide with voicemail http://uk.messenger.yahoo.com

author
#5 lore raw
Quoted 3 lines · Mark Underwood

What my driver would like to do is to handle its own input buffers. It would pass the buffer to the tty layer when it is full and the tty layer would pass the

In theory you can do that already, although the locking is a bit screwed up for it. Actually all the tty locking is broken for rx I believe. Everyone should be holding the tty read lock when updating flip buffers but right now we don't

Quoted 6 lines

buffer back once it has drained the data from it. The problem is that I don't always receive a block worth of characters so I also need to pass the tty layer a buffer (which I'm still DMAing into) with a count of how many chars there are in the buffer and a offset of where to start from.

You can do this now providing you don't do it blindly from IRQ context.

Quoted 1 line

From a workqueue do

	struct tty_ldisc *ld = tty_ldisc_ref(tty);
	int space;

	if(ld == NULL)	/* Bin/defer */
		return;
	space = ld->receive_room(tty);
	if(count > space) count = space;

	ld->receive_buf(tty, charbuf, flagbuf, count);

There is a corner case if TTY_DONT_FLIP is set where you should queue but not all drivers do this and the DONT_FLIP hack 'has to die'

#6 lore raw

Hi Alan,

Thanks for your help, I might give this ago once I've fixed some flow control problems in my driver.

On a loosely related topic I have extended serial_core.c to handle DMA UARTS (only the TX path is effected). Once I'm happy with my changes I post a patch.

Best Regards,

Mark

Quoted 46 lines · --- Alan Cox
Quoted 1 line · Mark Underwood

What my driver would like to do is to handle its

own

Quoted 2 lines

input buffers. It would pass the buffer to the tty layer when it is full and the tty layer would pass

the

In theory you can do that already, although the locking is a bit screwed
up for it. Actually all the tty locking is broken for rx I believe.
Everyone should be holding the tty read lock when updating flip buffers
but right now we don't

Quoted 4 lines

buffer back once it has drained the data from it. The problem is that I don't always receive a block worth of characters so I also need to pass the tty layer a buffer (which I'm still DMAing into) with

a

Quoted 1 line

count of how many chars there are in the buffer

and a

Quoted 1 line

offset of where to start from.

You can do this now providing you don't do it blindly from IRQ context.

Quoted 1 line

From a workqueue do

	struct tty_ldisc *ld = tty_ldisc_ref(tty);
	int space;

	if(ld == NULL)	/* Bin/defer */
		return;
	space = ld->receive_room(tty);
	if(count > space) count = space;

	ld->receive_buf(tty, charbuf, flagbuf, count);

There is a corner case if TTY_DONT_FLIP is set where you should queue
but not all drivers do this and the DONT_FLIP hack 'has to die'

Quoted 8 lines

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected] More majordomo info at
http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

___________________________________________________________
How much free photo storage do you get? Store your holiday snaps for FREE with Yahoo! Photos http://uk.photos.yahoo.com