horrible disk thorughput on itanium
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 4 lines · [email protected] (Linus Torvalds) writes:
"
putc()" is a standard function. If it sucks, let's get it fixed. And instead of changing bonnie, how about pinging the _real_ people who write sucky code?
It is easy to fix. Just do #define putc putc_unlocked
There is just a slight problem: it'll fail if your application is threaded and wants to use the same FILE from multiple threads.
It is a common problem on all OS that eventually got threadsafe stdio. I bet putc sucks on Solaris too.
-Andi
Quoted 6 lines · Andi Kleen
Quoted 4 lines · [email protected] (Linus Torvalds) writes:
"
putc()" is a standard function. If it sucks, let's get it fixed. And instead of changing bonnie, how about pinging the _real_ people who write sucky code?
Quoted 6 lines
It is easy to fix. Just do #define putc
putc_unlocked
There is just a slight problem: it'll fail if your application is threaded and wants to use the same FILE from multiple threads.It is a common problem on all OS that eventually got threadsafe stdio. I bet putc sucks on Solaris too.
This kind of thing should be covered by _REENTRANT. If you don't compile with _REENTRANT (the default), then putc() should be the unlocked version.
Regards, Richard.... Permanent: [email protected] Current: [email protected]
Quoted 3 lines · Richard Gooch
This kind of thing should be covered by
_REENTRANT. If you don't compile with_REENTRANT(the default), thenputc()should be the unlocked version.
The link to the mailing list post from bug-glibc says otherwise, that is the problem. Using the unlocked version isn't implied by not setting __REENTRANT.
Robert Love
Quoted 9 lines · Robert Love
Quoted 3 lines · Richard Gooch
This kind of thing should be covered by
_REENTRANT. If you don't compile with_REENTRANT(the default), thenputc()should be the unlocked version.The link to the mailing list post from bug-glibc says otherwise, that is the problem. Using the unlocked version isn't implied by not setting
__REENTRANT.
The bug is in glibc. An application shouldn't need to be changed to work around that bug. putc() is a well-known interface, and people shouldn't have to code around a change in that interface.
Regards, Richard.... Permanent: [email protected] Current: [email protected]
Quoted 7 lines · Richard Gooch
Quoted 3 lines
The link to the mailing list post from bug-glibc says otherwise, that is the problem. Using the unlocked version isn't implied by not setting
__REENTRANT.The bug is in glibc. An application shouldn't need to be changed to work around that bug.
putc()is a well-known interface, and people shouldn't have to code around a change in that interface.
Right. That's why I referenced a post on bug-glibc and called the issue a problem. I'm not defending the heaping mass known as glibc ... it should be fixed.
Robert Love
Quoted 7 lines · Andi Kleen
Quoted 4 lines · [email protected] (Linus Torvalds) writes:
"
putc()" is a standard function. If it sucks, let's get it fixed. And instead of changing bonnie, how about pinging the _real_ people who write sucky code?It is easy to fix. Just do #define putc
putc_unlocked
Sure. And why don't you also do
#define sin(x) (1) #define sqrt(x) (1) #define strlen(x) (1) ...
to make other benchmarks happier?
bonnie is a _benchmark_. It's meant for finding bad performance. Changing it to make it work better when performance is bad is _pointless_. You've now made the whole point of bonnie go away.
Quoted 4 lines
There is just a slight problem: it'll fail if your application is threaded and wants to use the same FILE from multiple threads.
It is a common problem on all OS that eventually got threadsafe stdio.
It's a common problem with bad programming.
You can be thread-safe without sucking dead baby donkeys through a straw. I already mentioned two possible ways to fix it so that you have locking when you need to, and no locking when you don't.
Linus
In article <[email protected]>,
Quoted 3 lines · Linus Torvalds
bonnie is a _benchmark_. It's meant for finding bad performance. Changing it to make it work better when performance is bad is _pointless_. You've now made the whole point of bonnie go away.
It isn't just bonnie showing bad performance. My application shows it, bonnie shows it, and tiobench shows it. I think the focus on putc may be too intense. It isn't suprizing to me that either the kernel or glibc may not be optimised on ia64, I'm just trying to figure out how to get better io rates out of my itanium machine.
Quoted 11 lines · Greg Hennessy
In article <[email protected]>,
Quoted 3 lines · Linus Torvalds
bonnie is a _benchmark_. It's meant for finding bad performance. Changing it to make it work better when performance is bad is _pointless_. You've now made the whole point of bonnie go away.
It isn't just bonnie showing bad performance. My application shows it, bonnie shows it, and tiobench shows it. I think the focus on putc may be too intense. It isn't suprizing to me that either the kernel or glibc may not be optimised on ia64, I'm just trying to figure out how to get better io rates out of my itanium machine.
Does a simple 'dd' show the problem? I mean,
time dd if=/dev/zero of=/somelargefile count=somelargenumber bs=8k
is it much slower on the itanium, too? dd doesn't use putc(), I hope.
Just for comparison, I've run the following command here:
# time dd if=/dev/zero of=/u2/test count=250000 bs=8k 250000+0 records in 250000+0 records out 0.14user 12.95system 1:23.15elapsed 15%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (117major+18minor)pagefaults 0swaps
(Almost idle box, 256MB RAM, UDMA disk, 2.2.x kernel)
You may try with a bigger file, expecially if you have more RAM.
.TM.
Signature · 5 lines
____/ ____/ /
/ / / Marco Colombo
___/ ___ / / Technical Manager
/ / / ESI s.r.l.
_____/ _____/ _/ [email protected]Quoted 3 lines
You can be thread-safe without sucking dead baby donkeys through a straw. I already mentioned two possible ways to fix it so that you have locking when you need to, and no locking when you don't.
Your proposals sound rather dangerous. They would silently break recompiled threaded programs that need the locking and don't use -D__REENTRANT (most people do not seem to use it). I doubt the possible pain from that is worth it for speeding up an basically obsolete interface like putc.
i.e. if someone wants speed they definitely shouldn't use putc()
-Andi
Quoted 3 lines · Andi Kleen
Your proposals sound rather dangerous. They would silently break recompiled threaded programs that need the locking and don't use -D__REENTRANT
No it wouldn't.
Once you do a pthread_create(), the locking is there.
Before you do a pthread_create(), it doesn't lock.
What's the problem? Before you do a pthread_create(), you don't _NEED_ locking, because there is only one thread that accesses the stdio data structures.
And there are no races - if there is only one thread, then another thread couldn't be suddenly doing a pthread_create() during a stdio operations.
Safe, and efficient. Yes, it adds a flag test or a indirect branch, but considering that you avoid a serialized locking instruction, the optimization sounds obvious.
Linus
Quoted 24 lines · Linus Torvalds
Quoted 3 lines · Andi Kleen
Your proposals sound rather dangerous. They would silently break recompiled threaded programs that need the locking and don't use -D__REENTRANT
No it wouldn't.
Once you do a
pthread_create(), the locking is there.Before you do a
pthread_create(), it doesn't lock.What's the problem? Before you do a
pthread_create(), you don't _NEED_ locking, because there is only one thread that accesses the stdio data structures.And there are no races - if there is only one thread, then another thread couldn't be suddenly doing a
pthread_create()during a stdio operations.Safe, and efficient. Yes, it adds a flag test or a indirect branch, but considering that you avoid a serialized locking instruction, the optimization sounds obvious.
Linus
Quoted 10 lines · Andi Kleen
Quoted 3 lines
You can be thread-safe without sucking dead baby donkeys through a straw. I already mentioned two possible ways to fix it so that you have locking when you need to, and no locking when you don't.
Your proposals sound rather dangerous. They would silently break recompiled threaded programs that need the locking and don't use -D__REENTRANT (most people do not seem to use it). I doubt the possible pain from that is worth it for speeding up an basically obsolete interface like putc.
i.e. if someone wants speed they definitely shouldn't use
putc()
Threaded programs that need locking and don't define _THREAD_SAFE or _REENTRANT or whatever is appropriate are already broken -- they just don't know it yet.
FreeBSD #defines putc and getc to their unlocked versions unless _THREAD_SAFE is defined, and people don't seem to think its libc is broken. Many lightly threaded programs, in fact, wouldn't need or even want the locked variants to be the default. One app I've worked with only reads and writes any given FILE* from one thread, and I saw an 4x speedup by switching to the unlocked variants.
It's generally a bad idea to make people pay for a feature they don't ask for. FreeBSD's libc understands this; glibc apaprently doesn't.
-- Michael Poole
Quoted 8 lines · Andi Kleen
Quoted 4 lines
You can be thread-safe without sucking dead baby donkeys through a straw. I already mentioned two possible ways to fix it so that you have locking when you need to, and no locking when you don't.
Your proposals sound rather dangerous. They would silently break recompiled threaded programs that need the locking and don't use -D__REENTRANT (most people do not seem to use it).
I would worry about threaded progs that don't -D_REENTRANT as they are broken.
Quoted 3 lines
I doubt the possible pain from that is worth it for speeding up an basically obsolete interface like putc.
i.e. if someone wants speed they definitely shouldn't use
putc()
It's not just putc, it's all of stdio.
Padraig.
Quoted 18 lines · Michael Poole
Quoted 10 lines · Andi Kleen
Quoted 4 lines
You can be thread-safe without sucking dead baby donkeys through a straw. I already mentioned two possible ways to fix it so that you have locking when you need to, and no locking when you don't.
Your proposals sound rather dangerous. They would silently break recompiled
threaded programs that need the locking and don't use -D__REENTRANT (most
people do not seem to use it). I doubt the possible pain from that is worth it for speeding up an basically obsolete interface like putc.i.e. if someone wants speed they definitely shouldn't use
putc()Threaded programs that need locking and don't define
_THREAD_SAFEor_REENTRANTor whatever is appropriate are already broken -- they just don't know it yet.
Quoted 6 lines
FreeBSD #defines putc and getc to their unlocked versions unless
_THREAD_SAFEis defined, and people don't seem to think its libc is broken. Many lightly threaded programs, in fact, wouldn't need or even want the locked variants to be the default. One app I've worked with only reads and writes any given FILE* from one thread, and I saw an 4x speedup by switching to the unlocked variants.
This breaks for the case discussed @
http://sources.redhat.com/ml/bug-glibc/2001-11/msg00079.html
I.E. if you have a multithreaded lib being linked by single threaded apps (Note multithreaded lib, not just a threadsafe lib (I.E. the lib calls pthread_create())).
Padraig.
From: Andi Kleen <[email protected]>
Date: 07 Dec 2001 14:54:49 +0100
It is a common problem on all OS that eventually got threadsafe stdio. I bet putc sucks on Solaris too.
Nope, they even inline the full implementation it in their header files so they beat even our putc_unlocked() to the point it is embarassing.
Quoted 5 lines · Padraig Brady
This breaks for the case discussed @
http://sources.redhat.com/ml/bug-glibc/2001-11/msg00079.html
I.E. if you have a multithreaded lib being linked by single threaded apps (Note multithreaded lib, not just a threadsafe lib (I.E. the lib callspthread_create())).
That's an interesting, but very contrived, example. Can you find a single multi-threaded lib that uses FILE*'s shared with the application using it?
Linus's suggestion to add hooks to pthread_create() gets around that problem, anyway. Alternatively, the multi-threaded library could require any application linking to it to define _REENTRANT.
After all, it's silly to talk about a 'multi-threaded' library linked to a 'single-threaded' application -- the application plus any libraries, as a whole, are either multithreaded or not. They have to be on the same page to deal with *any* locking issues.
-- Michael
Quoted 10 lines · Michael Poole
Quoted 6 lines · Padraig Brady
This breaks for the case discussed @
http://sources.redhat.com/ml/bug-glibc/2001-11/msg00079.html
I.E. if you have a multithreaded lib being linked by single threaded apps (Note multithreaded lib, not just a threadsafe lib (I.E. the lib callspthread_create())).
Quoted 3 lines
That's an interesting, but very contrived, example. Can you find a single multi-threaded lib that uses FILE*'s shared with the application using it?
Well you have to deal with the general case. A single threaded
app linking against a multithreaded lib. It mightn't be just shared FILE*'s that could cause problems.
Quoted 2 lines
Linus's suggestion to add hooks to
pthread_create()gets around that problem, anyway.
I don't think this will work as I said before current apps that use _unlocked() functions directly manipulate the stdio structures, hence a "new smarter locking stdio" would never get used by existing compiled apps.
Quoted 2 lines
Alternatively, the multi-threaded library could require any application linking to it to define
_REENTRANT.
It could, but what if an existing interface (lib) is changed
from signle to multithreaded. You can't preclude this.
Quoted 6 lines
After all, it's silly to talk about a 'multi-threaded' library linked to a 'single-threaded' application -- the application plus any libraries, as a whole, are either multithreaded or not. They have to be on the same page to deal with *any* locking issues.
-- Michael
Padraig.
Quoted 3 lines · Padraig Brady
Well you have to deal with the general case. A single threaded app linking against a multithreaded lib. It mightn't be just shared FILE*'s that could cause problems.
The question was about putc(), and presumably other stdio functions. They deal with FILE*'s. They are also commonly used and small operations, so forcing locking on an app that doesn't ask for it can (and has in the past) cause a major performance degredation. I bet you could find very similar results for memory allocation in other applications.
Quoted 7 lines
Quoted 2 lines
Linus's suggestion to add hooks to
pthread_create()gets around that problem, anyway.I don't think this will work as I said before current apps that use
_unlocked()functions directly manipulate the stdio structures, hence a "new smarter locking stdio" would never get used by existing compiled apps.
As Linus pointed out, you just need another test in those macros, so they can be forced to call functions rather than using inline code. When you call a function, it can use whatever new smarter locking library you want.
Quoted 5 lines
Quoted 2 lines
Alternatively, the multi-threaded library could require any application linking to it to define
_REENTRANT.It could, but what if an existing interface (lib) is changed from signle to multithreaded. You can't preclude this.
I certainly can preclude that. I'd consider adding threads and their locking considerations to the library a change in the API and ABI -- and quite a big one at that. If you change the ABI (rather than just extending it), you must increase the major version number, which prevents linking with those applications that expect the semantics of earlier versions.
(BSD variants have in the past had both libc and libc_r, where only libc_r makes you pay locking overhead. This is yet another way to enforce the ABI separation between single- and multi-threaded applications, with different tradeoffs than the others mentioned.)
-- Michael
[OT] fputc vs putc Re: horrible disk thorughput on itanium
In reply to Andi KleenQuoted 2 lines · David S. Miller
They do it also for
putc()if you haven't defined the appropriate defines.
what exactly is the difference between putc() and fputc()? the man page is very vague.
Signature · 2 lines
Tom Vier <[email protected]> DSA Key id 0x27371A2C
Re: [OT] fputc vs putc Re: horrible disk thorughput on itanium
In reply to Andi KleenQuoted 6 lines · Tom Vier
Quoted 2 lines · David S. Miller
They do it also for
putc()if you haven't defined the appropriate defines.what exactly is the difference between
putc()andfputc()? the man page is very vague.
fputc is a function putc is a macro because the overhead of a function call in writing a character has always been a problem...
Eric
Re: [OT] fputc vs putc Re: horrible disk thorughput on itanium
In reply to Andi KleenFollowup to: <[email protected]> By author: [email protected] (Eric W. Biederman) In newsgroup: linux.dev.kernel
Quoted 4 lines
fputc is a function putc is a macro because the overhead of a function call in writing a character has always been a problem...
putc() is frequently defined as
#define putc(__C) fputc((__C), stdout)
... or some equivalent; I think the best way to say it's that it's a shorthand.
-hpa
Signature · 3 lines
<[email protected]> at work, <[email protected]> in private! "Unix gives you enough rope to shoot yourself in the foot." http://www.zytor.com/~hpa/puzzle.txt <[email protected]>
Re: [OT] fputc vs putc Re: horrible disk thorughput on itanium
In reply to Andi KleenQuoted 6 lines · H. Peter Anvin
putc()is frequently defined as#define
putc(__C)fputc((__C), stdout)... or some equivalent; I think the best way to say it's that it's a shorthand.
according to the putc man page in debian stable, it takes the same args as fputc. maybe it varies by glibc version (mine is 2.1.3-19). i guess anyone using putc better use autoconf. also, "unix system programming for SVr4" says the only difference is that putc is an inlined macro version of fputc.
Signature · 2 lines
Tom Vier <[email protected]> DSA Key id 0x27371A2C
Re: [OT] fputc vs putc Re: horrible disk thorughput on itanium
In reply to Andi KleenFollowup to: <20011209141408.A17671@zero> By author: Tom Vier <[email protected]> In newsgroup: linux.dev.kernel
Quoted 6 lines
according to the putc man page in debian stable, it takes the same args as fputc. maybe it varies by glibc version (mine is 2.1.3-19). i guess anyone using putc better use autoconf. also, "unix system programming for SVr4" says the only difference is that putc is an inlined macro version of fputc.
Nevermind. I was thinking about putc() and putchar(), additionally confused by the annoying fact that fputs() is to putc() as puts() is to putchar()... makes no fscking sense.
-hpa
Signature · 3 lines
<[email protected]> at work, <[email protected]> in private! "Unix gives you enough rope to shoot yourself in the foot." http://www.zytor.com/~hpa/puzzle.txt <[email protected]>
Quoted 7 lines · Andi Kleen
Quoted 4 lines · [email protected] (Linus Torvalds) writes:
"
putc()" is a standard function. If it sucks, let's get it fixed. And instead of changing bonnie, how about pinging the _real_ people who write sucky code?
Quoted 7 lines
It is easy to fix. Just do #define putc
putc_unlocked
There is just a slight problem: it'll fail if your application is threaded and wants to use the same FILE from multiple threads.It is a common problem on all OS that eventually got threadsafe stdio. I bet putc sucks on Solaris too.
-Andi
Interesting thread on this:
http://sources.redhat.com/ml/bug-glibc/2001-11/msg00079.html
for glibc 2.2.4 with the following program with input file of 354371 lines of text(/usr/share/doc/*), where the average line length was 37 chars.
getc/putc
real 2.181s
user 2.150s
sys 0.030s
getc_unlocked/putc_unlocked
real 0.326s
user 0.280s
sys 0.040s
I.E. 669% faster!
Padraig.
-------------------
#include <stdio.h>
#ifndef _REENTRANT
# undef getc # define getc(x) getc_unlocked(x) # undef putc # define putc(x,y) putc_unlocked(x,y)
#endif //_REENTRANT
void main(void)
{
int c;
while((c=getc(stdin))!=EOF) putc(c,stdout);
}