Linux tty layer hackery: Heads up and RFC
At the moment tty buffers are attached directly to the tty. This is causing a lot of the problems related to tty layer locking, also problems at high speed and also with bursty data (such as occurs in virtualised environments)
I'm working on ripping out the flip buffers and replacing them with a pool of dynamically allocated buffers. This allows both for old style "byte I/O" devices and also helps virtualisation and smart devices where large blocks of data suddenely materialise and need storing.
So far so good. Lots of drivers reference tty->flip.*. Several of them also call directly and unsafely into function pointers it provides. This will all break. Most drivers can use tty_insert_flip_char which can be kept as an API but others need more.
At the moment I've added the following interfaces, if people think more will be needed now is a good time to say
int tty_buffer_request_room(tty, size)
Try and ensure at least size bytes are available, returns actual room (may be zero). At the moment it just uses the flipbuf space but that will change. Repeated calls without characters being added are not cumulative. (ie if you call it with 1, 1, 1, and then 4 you'll have four characters of space. The other functions will also try and grow buffers in future but this will be a more efficient way when you know block sizes.
int tty_insert_flip_char(tty, ch, flag)
As before insert a character if there is room. Now returns 1 for success, 0 for failure.
int tty_insert_flip_string(tty, str, len)
Insert a block of non error characters. Returns the number inserted.
int tty_prepare_flip_string(tty, strptr, len)
Adjust the buffer to allow len characters to be added. Returns a buffer pointer in strptr and the length available. This allows for hardware that needs to use functions like insl or mencpy_fromio.
I've converted a fair number of drivers to this API ready and I'll post some patches for review soon.
Alan
Quoted 9 lines · Alan Cox
At the moment tty buffers are attached directly to the tty. This is causing a lot of the problems related to tty layer locking, also problems at high speed and also with bursty data (such as occurs in virtualised environments)
I'm working on ripping out the flip buffers and replacing them with a pool of dynamically allocated buffers. This allows both for old style "byte I/O" devices and also helps virtualisation and smart devices where large blocks of data suddenely materialise and need storing.
Great! Really good news!
Quoted 26 lines
So far so good. Lots of drivers reference tty->flip.*. Several of them also call directly and unsafely into function pointers it provides. This will all break. Most drivers can use
tty_insert_flip_charwhich can be kept as an API but others need more.At the moment I've added the following interfaces, if people think more will be needed now is a good time to say
int
tty_buffer_request_room(tty, size)Try and ensure at least size bytes are available, returns actual room (may be zero). At the moment it just uses the flipbuf space but that will change. Repeated calls without characters being added are not cumulative. (ie if you call it with 1, 1, 1, and then 4 you'll have four characters of space. The other functions will also try and grow buffers in future but this will be a more efficient way when you know block sizes.
int
tty_insert_flip_char(tty, ch, flag)As before insert a character if there is room. Now returns 1 for success, 0 for failure.
int
tty_insert_flip_string(tty, str, len)
Quoted 7 lines
Insert a block of non error characters. Returns the number inserted.
int
tty_prepare_flip_string(tty, strptr, len)Adjust the buffer to allow len characters to be added. Returns a buffer pointer in strptr and the length available. This allows for hardware that needs to use functions like insl or
mencpy_fromio.
As you are going to replace flip buffers with different implementation anyway, isn't it better to get rid of "flip" in the interface names as well (maybe providing some synonyms for backward compatibility)? What I mean is that names like
tty_buffer_insert_char()tty_buffer_insert_string()
for the new interfaces would probably make more sense.
Otherwise I find the interfaces you suggest just fine and suitable for the task I was unable to achieve without ugly hacks using current flip buffers interfaces (reliable high-speed bulk USB tty driver).
Signature · 1 line
Sergei.
Quoted 5 lines · Alan Cox
int
tty_prepare_flip_string(tty, strptr, len)Adjust the buffer to allow len characters to be added. Returns a buffer pointer in strptr and the length available. This allows for hardware that needs to use functions like insl or
mencpy_fromio.
Ok, So then I start copying characters into the flipstring, but how do I say I'm done?
Or is there a race between that I call tty_prepare_flip_string, and other processes start pulling my not-yet-filled string from the buffer? (Surely not!)
Roger.
Signature · 5 lines
** [email protected] ** http://www.BitWizard.nl/ ** +31-15-2600998 ** *-- BitWizard writes Linux device drivers for any device you may have! --* Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous statement. Does it sit on the couch all day? Is it unemployed? Please be specific! Define 'it' and what it isn't doing. --------- Adapted from lxrbot FAQ
Quoted 2 lines · Rogier Wolff
Ok, So then I start copying characters into the flipstring, but how do I say I'm done?
tty_flip_buffer_push() (or its possibly renamed equivalent). At that point as now the buffer may get processed or queued to the ldisc. At that point its now owned by the ldisc and the tty buffer layer will give you new buffers.
Hi Rogier,
Having just written a DMA UART driver I can say this is good news :-). Here are some things to think about:
What my driver would like to do is to handle its own input buffers. It would pass the buffer to the tty layer when it is full and the tty layer would pass the buffer back once it has drained the data from it. The problem is that I don't always receive a block worth of characters so I also need to pass the tty layer a buffer (which I'm still DMAing into) with a count of how many chars there are in the buffer and a offset of where to start from. If I can't do this then I have to stop the DMA transfer, pass you a mostly empty buffer with a char count, and setup DMA transfer into another buffer. If during this I get a burst of data I could well lose it :-(.
Best Regards,
Mark
Quoted 15 lines · --- Rogier Wolff
Quoted 3 lines · Alan Cox
int
tty_prepare_flip_string(tty, strptr, len)Adjust the buffer to allow len characters to be
added. Returns a buffer
Quoted 1 line
pointer in strptr and the length available. This
allows for hardware
Quoted 1 line
that needs to use functions like insl or
mencpy_fromio.Ok, So then I start copying characters into the flipstring, but how do
I say I'm done?
Quoted 27 lines
Or is there a race between that I call
tty_prepare_flip_string, and
other processes start pulling my not-yet-filled string from the
buffer? (Surely not!)Roger.
Signature · 17 lines
** [email protected] ** http://www.BitWizard.nl/ ** +31-15-2600998 ** *-- BitWizard writes Linux device drivers for any device you may have! --* Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous statement. Does it sit on the couch all day? Is it unemployed? Please be specific! Define 'it' and what it isn't doing. --------- Adapted from lxrbot FAQ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
___________________________________________________________
Yahoo! Messenger - NEW crystal clear PC to PC calling worldwide with voicemail http://uk.messenger.yahoo.com
Quoted 3 lines · Mark Underwood
What my driver would like to do is to handle its own input buffers. It would pass the buffer to the tty layer when it is full and the tty layer would pass the
In theory you can do that already, although the locking is a bit screwed up for it. Actually all the tty locking is broken for rx I believe. Everyone should be holding the tty read lock when updating flip buffers but right now we don't
Quoted 6 lines
buffer back once it has drained the data from it. The problem is that I don't always receive a block worth of characters so I also need to pass the tty layer a buffer (which I'm still DMAing into) with a count of how many chars there are in the buffer and a offset of where to start from.
You can do this now providing you don't do it blindly from IRQ context.
Quoted 1 line
From a workqueue do
struct tty_ldisc *ld = tty_ldisc_ref(tty); int space; if(ld == NULL) /* Bin/defer */ return; space = ld->receive_room(tty); if(count > space) count = space; ld->receive_buf(tty, charbuf, flagbuf, count);
There is a corner case if TTY_DONT_FLIP is set where you should queue but not all drivers do this and the DONT_FLIP hack 'has to die'
Hi Alan,
Thanks for your help, I might give this ago once I've fixed some flow control problems in my driver.
On a loosely related topic I have extended serial_core.c to handle DMA UARTS (only the TX path is effected). Once I'm happy with my changes I post a patch.
Best Regards,
Mark
Quoted 46 lines · --- Alan Cox
Quoted 1 line · Mark Underwood
What my driver would like to do is to handle its
own
Quoted 2 lines
input buffers. It would pass the buffer to the tty layer when it is full and the tty layer would pass
the
In theory you can do that already, although the locking is a bit screwed
up for it. Actually all the tty locking is broken for rx I believe.
Everyone should be holding the tty read lock when updating flip buffers
but right now we don'tQuoted 4 lines
buffer back once it has drained the data from it. The problem is that I don't always receive a block worth of characters so I also need to pass the tty layer a buffer (which I'm still DMAing into) with
a
Quoted 1 line
count of how many chars there are in the buffer
and a
Quoted 1 line
offset of where to start from.
You can do this now providing you don't do it blindly from IRQ context.
Quoted 1 line
From a workqueue do
struct tty_ldisc *ld = tty_ldisc_ref(tty); int space; if(ld == NULL) /* Bin/defer */ return; space = ld->receive_room(tty); if(count > space) count = space; ld->receive_buf(tty, charbuf, flagbuf, count);There is a corner case if
TTY_DONT_FLIPis set where you should queue
but not all drivers do this and theDONT_FLIPhack 'has to die'
Quoted 8 lines
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected] More majordomo info at
http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
___________________________________________________________
How much free photo storage do you get? Store your holiday snaps for FREE with Yahoo! Photos http://uk.photos.yahoo.com