Skip to contentSkip to conversation

read(2) syscall bloat

mlmmj Moritz Wilhelmy 4 participants 6 messages Sep 5, 2011 – Sep 9, 2011

lore
Share & tools
lore
#0 lore raw

Hello,

mlmmj currently does a read(2) system call for every single byte it reads from a file descriptor. This is unnecessarily inefficient and slow.

Strace output is similar to the following:
open("/var/spool/mlmmj/foo/control/listaddress", O_RDONLY) = 4

read(4, "f", 1)                         = 1
read(4, "o", 1)                         = 1
read(4, "o", 1)                         = 1
read(4, "@", 1)                         = 1
read(4, "l", 1)                         = 1
read(4, "i", 1)                         = 1
read(4, "s", 1)                         = 1
read(4, "t", 1)                         = 1
read(4, "s", 1)                         = 1
read(4, ".", 1)                         = 1
read(4, "e", 1)                         = 1
read(4, "x", 1)                         = 1
read(4, "a", 1)                         = 1
read(4, "m", 1)                         = 1
read(4, "p", 1)                         = 1
read(4, "l", 1)                         = 1
read(4, "e", 1)                         = 1
read(4, ".", 1)                         = 1
read(4, "c", 1)                         = 1
read(4, "o", 1)                         = 1
read(4, "m", 1)                         = 1
read(4, "\n", 1)                        = 1
close(4)                                = 0

Given that there is a getline(3) function in POSIX.1-2008, shouldn't it be possible to retire mygetline?

I've previously posted this issue to the musl mailing list [1], which has an "anti-bloat side project", but I've been putting the mail to this list off.

I don't see where any of Rich's arguments from [2] apply. Can anyone please explain why it was done this way in the first place?

[1] http://www.openwall.com/lists/musl/2011/08/16/8
[2] http://www.openwall.com/lists/musl/2011/08/16/11

Moritz

#1 lore raw
Quoted 35 lines · Moritz Wilhelmy

Hello,

mlmmj currently does a read(2) system call for every single byte it reads from a file descriptor. This is unnecessarily inefficient and slow.

Strace output is similar to the following:
open("/var/spool/mlmmj/foo/control/listaddress", O_RDONLY) = 4

read(4, "f", 1)                         = 1
read(4, "o", 1)                         = 1
read(4, "o", 1)                         = 1
read(4, "@", 1)                         = 1
read(4, "l", 1)                         = 1
read(4, "i", 1)                         = 1
read(4, "s", 1)                         = 1
read(4, "t", 1)                         = 1
read(4, "s", 1)                         = 1
read(4, ".", 1)                         = 1
read(4, "e", 1)                         = 1
read(4, "x", 1)                         = 1
read(4, "a", 1)                         = 1
read(4, "m", 1)                         = 1
read(4, "p", 1)                         = 1
read(4, "l", 1)                         = 1
read(4, "e", 1)                         = 1
read(4, ".", 1)                         = 1
read(4, "c", 1)                         = 1
read(4, "o", 1)                         = 1
read(4, "m", 1)                         = 1
read(4, "\n", 1)                        = 1
close(4)                                = 0

Given that there is a getline(3) function in POSIX.1-2008, shouldn't it be possible to retire mygetline?

Quoted 6 lines

I've previously posted this issue to the musl mailing list [1], which has an "anti-bloat side project", but I've been putting the mail to this list off.

I don't see where any of Rich's arguments from [2] apply. Can anyone please explain why it was done this way in the first place?

Not sure why it was done like that. Anyway, this shouldn't be too hard to fix, given that we already use our own getline() implementation (see "mygetline.c") and it shouldn't be too hard to add our own buffering implementation or use some stream implementation here.

I can have a look at that around next week if no one else wants to.

Quoted 5 lines

[1] http://www.openwall.com/lists/musl/2011/08/16/8
[2] http://www.openwall.com/lists/musl/2011/08/16/11

Moritz

#2 lore raw
In reply to Moritz Wilhelmy
Quoted 3 lines · Moritz Wilhelmy

mlmmj currently does a read(2) system call for every single byte it reads from a file descriptor. This is unnecessarily inefficient and slow.

Mmm. There've gotta be a lot of context switches happening there....

Quoted 5 lines

Strace output is similar to the following:
open("/var/spool/mlmmj/foo/control/listaddress", O_RDONLY) = 4

read(4, "f", 1)                         = 1
read(4, "o", 1)                         = 1
read(4, "o", 1)                         = 1

[...]

Quoted 5 lines

read(4, "\n", 1) = 1
close(4) = 0

Given that there is a getline(3) function in POSIX.1-2008, shouldn't it be possible to retire mygetline?

Not if getline() is new as of 2008; there are a lot of systems older than that around, and since Mlmmj is so nice and slim, it is an ideal candidate for running on older systems. I don't want to compromise that.

Quoted 5 lines

I've previously posted this issue to the musl mailing list [1], which has an "anti-bloat side project", but I've been putting the mail to this list off.

I don't see where any of Rich's arguments from [2] apply.

He's just pointing out that you can't reimplement mygetline() to read in larger chunks without some kind of buffering. This is because reading a larger chunk might read past end-of-line. If it does, then you have to rewind the stream (not always possible) or buffer the extra output so that the next call to mygetline() can use it.

Quoted 1 line

Can anyone please explain why it was done this way in the first place?

Not me.

Maybe we should do some profiling to see if this truly is a bottleneck or not.

Ben.

Quoted 2 lines

[1] http://www.openwall.com/lists/musl/2011/08/16/8
[2] http://www.openwall.com/lists/musl/2011/08/16/11

#3 lore raw
In reply to Moritz Wilhelmy
Quoted 16 lines · Ben Schmidt
Quoted 3 lines · Moritz Wilhelmy

mlmmj currently does a read(2) system call for every single byte it reads from a file descriptor. This is unnecessarily inefficient and slow.

Mmm. There've gotta be a lot of context switches happening there....

Quoted 5 lines

Strace output is similar to the following:
open("/var/spool/mlmmj/foo/control/listaddress", O_RDONLY) = 4

read(4, "f", 1)                         = 1
read(4, "o", 1)                         = 1
read(4, "o", 1)                         = 1

[...]

Quoted 3 lines

read(4, "\n", 1) = 1
close(4) = 0

Quoted 6 lines
Quoted 2 lines

Given that there is a getline(3) function in POSIX.1-2008, shouldn't it be possible to retire mygetline?

Not if getline() is new as of 2008; there are a lot of systems older than that around, and since Mlmmj is so nice and slim, it is an ideal candidate for running on older systems. I don't want to compromise that.

Well, if you really care about that, consider using fgets() which is part of C89, even. Or just use our own buffer implementation.

Quoted 13 lines
Quoted 6 lines

I've previously posted this issue to the musl mailing list [1], which has an "anti-bloat side project", but I've been putting the mail to this list off.

I don't see where any of Rich's arguments from [2] apply.

He's just pointing out that you can't reimplement mygetline() to read in larger chunks without some kind of buffering. This is because reading a larger chunk might read past end-of-line. If it does, then you have to rewind the stream (not always possible) or buffer the extra output so that the next call to mygetline() can use it.

Quoted 6 lines
Quoted 1 line

Can anyone please explain why it was done this way in the first place?

Not me.

Maybe we should do some profiling to see if this truly is a bottleneck or not.

Agreed, some numbers would be nice. Anyway, this shouldn't be too hard to implement and this will imply some performance improvements for sure...

Quoted 7 lines

Ben.

Quoted 2 lines

[1] http://www.openwall.com/lists/musl/2011/08/16/8
[2] http://www.openwall.com/lists/musl/2011/08/16/11

#4 lore raw
In reply to Moritz Wilhelmy
Quoted 6 lines · Ben Schmidt
Quoted 3 lines · Moritz Wilhelmy

mlmmj currently does a read(2) system call for every single byte it reads from a file descriptor. This is unnecessarily inefficient and slow.

Mmm. There've gotta be a lot of context switches happening there....

That's the point :-)

Quoted 9 lines
Quoted 5 lines

Strace output is similar to the following:
open("/var/spool/mlmmj/foo/control/listaddress", O_RDONLY) = 4

read(4, "f", 1)                         = 1
read(4, "o", 1)                         = 1
read(4, "o", 1)                         = 1

[...]

Quoted 3 lines

read(4, "\n", 1) = 1
close(4) = 0

Quoted 6 lines
Quoted 2 lines

Given that there is a getline(3) function in POSIX.1-2008, shouldn't it be possible to retire mygetline?

Not if getline() is new as of 2008; there are a lot of systems older than that around, and since Mlmmj is so nice and slim, it is an ideal candidate for running on older systems. I don't want to compromise that.

It has been in glibc long before and can be implemented in about 50 lines. You could detect if the libc has a getline function, and use your own otherwise (you do have autotools after all!)

You could copy the FreeBSD implementation of getline/getdelim with small changes, which is (obviously) BSD licensed. It doesn't look too specific to BSD stdio. I've seen some kind of getline.c floating around in many projects since many years, before it was finally put into the standard.

http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libc/stdio/getline.c?rev=1.1.2.1.6.1;content-type=text%2Fplain
http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libc/stdio/getdelim.c?rev=1.2.2.2.4.1;content-type=text%2Fplain

It would require switching to FILE*s though, but I see very little reason not to do just that for local files.

Quoted 5 lines

He's just pointing out that you can't reimplement mygetline() to read in larger chunks without some kind of buffering. This is because reading a larger chunk might read past end-of-line. If it does, then you have to rewind the stream (not always possible) or buffer the extra output so that the next call to mygetline() can use it.

Alright, that's actually obvious.

Moritz

#5 lore raw
In reply to Moritz Wilhelmy
Quoted 4 lines · Ben Schmidt

Maybe we should do some profiling to see if this truly is a bottleneck or not.

Ben.

Without looking, my bet is that it's NOT a bottleneck. Implementing the buffering yourself is the kind of thing you do *not* want to do, because it's error prone and can lead very easily to buffer overflow issue. At least, take the implementation from something that already exists (ideally, use a known shared lib). Working on this kind of "by hand" optimization is, IMHO a waste of time, considering all what happens when receiving a mail.

As a consequence, I vote for using getline() if it's available, and keep the old (slower) implementation if it's not.

Thomas