deinline sleep/delay functions
PATCH 1/1 deinline sleep/delay functions
Hi Andrew,
Optimizing delay functions for speed is utterly pointless.
This patch turns ssleep(n), mdelay(n), udelay(n) and ndelay(n) into functions, thus they generate the smallest possible code at the callsite. Previously they were more or less inlined.
Run tested. Saved a few kb off vmlinux.
--
vda
include/asm-i386/delay.h +2 -6
@@ -15,12 +15,8 @@ extern void __ndelay(unsigned long nsecsextern void __const_udelay (unsigned long usecs);extern void __delay (unsigned long loops); -#define udelay (n) (__builtin_constant_p(n) ? \ - ((n) > 20000 ? __bad_udelay() : __const_udelay((n) * 0x10c7ul)) : \ - __udelay(n)) +#define udelay (n) __udelay(n) -#define ndelay (n) (__builtin_constant_p(n) ? \ - ((n) > 20000 ? __bad_ndelay() : __const_udelay((n) * 5ul)) : \ - __ndelay(n)) +#define ndelay (n) __ndelay(n)#endif /* defined(_I386_DELAY_H) */
include/linux/delay.h +3 -29
@@ -11,40 +11,14 @@ extern unsigned long loops_per_jiffy;#include <asm/delay.h> -/* -* Using udelay() for intervals greater than a few milliseconds can -* risk overflow for high loops_per_jiffy (high bogomips) machines. The -* mdelay() provides a wrapper to prevent this. For delays greater -* than MAX_UDELAY_MS milliseconds, the wrapper is used. Architecture -* specific values can be defined in asm-???/delay.h as an override. -* The 2nd mdelay() definition ensures GCC will optimize away the -* while loop for the common cases where n <= MAX_UDELAY_MS -- Paul G. -*/ - -#ifndef MAX_UDELAY_MS -#define MAX_UDELAY_MS 5 -#endif - -#ifdef notdef -#define mdelay (n) (\ - {unsigned long __ms=(n); while (__ms--) udelay(1000);}) -#else -#define mdelay (n) (\ - (__builtin_constant_p(n) && (n)<=MAX_UDELAY_MS) ? udelay((n)*1000) : \ - ({unsigned long __ms=(n); while (__ms--) udelay(1000);})) -#endif - +void calibrate_delay (void ); +void mdelay (unsigned int msecs);#ifndef ndelay#define ndelay (x) udelay(((x)+999)/1000)#endif -void calibrate_delay (void );void msleep (unsigned int msecs); +void ssleep (unsigned int secs);unsigned long msleep_interruptible (unsigned int msecs); - -static inline void ssleep (unsigned int seconds) -{ -msleep (seconds *1000 ); -}#endif /* defined(_LINUX_DELAY_H) */
kernel/timer.c +22 -0
@@ -33,6 +33,7 @@#include <linux/posix-timers.h> #include <linux/cpu.h> #include <linux/syscalls.h> +#include <linux/delay.h> #include <asm/uaccess.h> #include <asm/unistd.h>
@@ -89,6 +90,20 @@ static inline void set_running_timer(tve/* Fake initialization */ static DEFINE_PER_CPU(tvec_base_t, tvec_bases) = { SPIN_LOCK_UNLOCKED }; +/*** +* Using udelay() for intervals greater than a few milliseconds can +* risk overflow for high loops_per_jiffy (high bogomips) machines. The +* mdelay() provides a wrapper to prevent this. +* +* It's not inlined because we do not optimize delays for speed ;) +*/ +void mdelay (unsigned int msecs) +{ +while (msecs--)udelay (1000 ); +} + +EXPORT_SYMBOL (mdelay); +static void check_timer_failed (struct timer_list *timer) {static int whine_count;
@@ -1592,6 +1607,13 @@ void msleep(unsigned int msecs) }EXPORT_SYMBOL (msleep); + +void ssleep (unsigned int secs) +{ +msleep (secs *1000 ); +} + +EXPORT_SYMBOL (ssleep);/** * msleep_interruptible - sleep waiting for waitqueue interruptions
Quoted 11 lines · Denis Vlasenko
Hi Andrew,
Optimizing delay functions for speed is utterly pointless.
This patch turns ssleep(n), mdelay(n), udelay(n) and ndelay(n) into functions, thus they generate the smallest possible code at the callsite. Previously they were more or less inlined.
Run tested. Saved a few kb off vmlinux.
Signed-off-by Denis Vlasenko <[email protected]>
Rejected-by: Russell King 8)
The reason is that now we're unable to find out if anyone's doing udelay(100000000000000000) which breaks on most architectures.
There are a number of compile-time checks that your patch has removed which catch such things, and as such your patch is not acceptable. Some architectures have a lower threshold of acceptability for the maximum udelay value, so it's absolutely necessary to keep this.
Signature · 3 lines
Russell King Linux kernel 2.6 ARM Linux - http://www.arm.linux.org.uk/ maintainer of: 2.6 Serial core
Quoted 16 lines · Russell King
Quoted 11 lines · Denis Vlasenko
Hi Andrew,
Optimizing delay functions for speed is utterly pointless.
This patch turns ssleep(n), mdelay(n), udelay(n) and ndelay(n) into functions, thus they generate the smallest possible code at the callsite. Previously they were more or less inlined.
Run tested. Saved a few kb off vmlinux.
Signed-off-by Denis Vlasenko <[email protected]>Rejected-by: Russell King 8)
Quoted 7 lines
The reason is that now we're unable to find out if anyone's doing udelay(100000000000000000) which breaks on most architectures.
There are a number of compile-time checks that your patch has removed which catch such things, and as such your patch is not acceptable. Some architectures have a lower threshold of acceptability for the maximum udelay value, so it's absolutely necessary to keep this.
It removes that check from x86 - other architectures retain it.
I don't recall seeing anyone trigger the check, and it hardly seems worth adding a "few kb" to vmlinux for it?
Quoted 9 lines
Quoted 5 lines
There are a number of compile-time checks that your patch has removed which catch such things, and as such your patch is not acceptable. Some architectures have a lower threshold of acceptability for the maximum udelay value, so it's absolutely necessary to keep this.
It removes that check from x86 - other architectures retain it.
I don't recall seeing anyone trigger the check,
I do ;) Esp in vendor out of tree crap. It's a good compile time diagnostic so the junk code doesnt' hit mainline but gets fixed first.
Quoted 3 lines
and it hardly seems worth
adding a "few kb" to vmlinux for it?
but it can be fixed to not add that few kb while retaining the checking value. All that needs is for it to be a define that calls the worker function. Eg the check gets optimized out and all that remains is the call to the worker function that does the actual delay.
Quoted 6 lines · Arjan van de Ven
Quoted 6 lines
Quoted 5 lines
There are a number of compile-time checks that your patch has removed which catch such things, and as such your patch is not acceptable. Some architectures have a lower threshold of acceptability for the maximum udelay value, so it's absolutely necessary to keep this.
Quoted 6 lines
Quoted 3 lines
It removes that check from x86 - other architectures retain it.
I don't recall seeing anyone trigger the check,
I do ;) Esp in vendor out of tree crap. It's a good compile time diagnostic so the junk code doesnt' hit mainline but gets fixed first.
It seems my patch was incomplete.
Thinking more about it, since it exists in all arches and also since delaying is not performance critical (hey, if we're going to delay/sleep, we surely can burn a few more cycles), this can be done as follows:
linux/timer.c:
void ndelay(unsigned int nsecs)
{
unsigned int m = nsecs/(1024*1024);
while (m--)
__ndelay(1024);
__ndelay(nsecs % (1024*1024));
}void udelay(unsigned int usecs)
{
unsigned int k = usecs/1024;
while (k--)
__udelay(1024);
__udelay(usecs % 1024);
}void mdelay(unsigned int msecs)
{
while (msecs--)
udelay(1000);
}void ssleep(unsigned int secs)
{
msleep(secs * 1000);
}and arches will need to only supply two functions:
__udelay(n) [n is guaranteed <1024]__ndelay(n) [n is guaranteed <1024*1024]
For users, _any_ value, however large, will work for any delay function.
Comments?
--
vda
Quoted 7 lines · Denis Vlasenko
Quoted 6 lines · Arjan van de Ven
Quoted 6 lines
Quoted 5 lines
There are a number of compile-time checks that your patch has removed which catch such things, and as such your patch is not acceptable. Some architectures have a lower threshold of acceptability for the maximum udelay value, so it's absolutely necessary to keep this.
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
It removes that check from x86 - other architectures retain it.
For users, _any_ value, however large, will work for any delay function.
that's not desired though. Desired is to limit udelay() to say 2000 or so. And force anything above that to go via mdelay() (just to make it stand out as broken code ;)
Over time we also want to phase out mdelay of course...
Quoted 11 lines · Arjan van de Ven
Quoted 10 lines · Denis Vlasenko
Quoted 9 lines · Arjan van de Ven
Quoted 9 lines
Quoted 5 lines
There are a number of compile-time checks that your patch has removed which catch such things, and as such your patch is not acceptable. Some architectures have a lower threshold of acceptability for the maximum udelay value, so it's absolutely necessary to keep this.
It removes that check from x86 - other architectures retain it.
Quoted 6 lines
Quoted 2 lines
For users, _any_ value, however large, will work for any delay function.
that's not desired though. Desired is to limit
udelay()to say 2000 or so. And force anything above that to go viamdelay()(just to make it stand out as broken code ;)
An if(usec > 2000) { printk(..); dump_stack(); } will do.
But do you really want to do this? There might be legitimate reasons to compute udelay's parameter with results which are sometimes large.
If you really want to, let's decide on this limit now, before patch cooking. I err to conservative (large) limit (mostly in order to catch math underflows) or no limit at all. --
vda
Quoted 12 lines · Denis Vlasenko
Quoted 11 lines · Arjan van de Ven
Quoted 10 lines · Denis Vlasenko
Quoted 9 lines · Arjan van de Ven
Quoted 9 lines
Quoted 5 lines
There are a number of compile-time checks that your patch has removed which catch such things, and as such your patch is not acceptable. Some architectures have a lower threshold of acceptability for the maximum udelay value, so it's absolutely necessary to keep this.
It removes that check from x86 - other architectures retain it.
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
For users, _any_ value, however large, will work for any delay function.
that's not desired though. Desired is to limit
udelay()to say 2000 or so. And force anything above that to go viamdelay()(just to make it stand out as broken code ;)An if(usec > 2000) { printk(..);
dump_stack(); } will do.
No it won't - that's a run time test which will only get caught if the code is run. There's no guarantees of that.
Quoted 2 lines
But do you really want to do this? There might be legitimate reasons to compute udelay's parameter with results which are sometimes large.
Yes. udelay() has overflow issues - if you pass too large a number to udelay() you get a randomised delay because you've lost the top bits.
The maximum delay is dependent on the architecture implementation, and it depends on bogomips. There is no one single value for it. Architectures have to decide this from the way that they do the math and the expected range of bogomips.
Please - leave asm-*/delay.h alone.
Signature · 3 lines
Russell King Linux kernel 2.6 ARM Linux - http://www.arm.linux.org.uk/ maintainer of: 2.6 Serial core
Quoted 2 lines
An if(usec > 2000) { printk(..);
dump_stack(); } will do.
that's runtime not compile time.
The old situation was a compile time check which is far more powerful.
Quoted 3 lines
But do you really want to do this? There might be legitimate reasons to compute udelay's parameter with results which are sometimes large.
however that's not valid, because a really large udelay parameter will cause the delay loop math to overflow. That was the main reason for the "no more than 2ms or so" restriction.
Quoted 2 lines
If you really want to, let's decide on this limit now,
the existing code already has a limit so why not just use that?
Quoted 5 lines · Arjan van de Ven
Quoted 2 lines
An if(usec > 2000) { printk(..);
dump_stack(); } will do.that's runtime not compile time.
The old situation was a compile time check which is far more powerful.
Ok I like compile checks too, it will stay.
But it won't help on accidental mdelay(some math) == mdelay(-1) == mdelay(4 000 000 000), so we _also_ will have an if() inside udelay(), ok?
Quoted 3 lines · Russell King
Yes.
udelay()has overflow issues - if you pass too large a number toudelay()you get a randomised delay because you've lost the top bits.
Thus [umn]delay may fail in unpredictable ways with non-const parameter which is too big. And this is good exactly why?
I'm ok with making it fail, but _predictably_. With printk(), trace, whatever.
Quoted 4 lines
The maximum delay is dependent on the architecture implementation, and it depends on bogomips. There is no one single value for it. Architectures have to decide this from the way that they do the math and the expected range of bogomips.
In example I posted these limitations are lifted. Granted these limitations were not critical, but removing them can't do harm, I guess?
Quoted 1 line
Please - leave
asm-*/delay.halone.
Let's see what udelay(const) will compile down to on ppc:
asm-ppc/delay.h
===============
extern unsigned long loops_per_jiffy;
extern void __delay(unsigned int loops);
...
#define __MAX_UDELAY (226050910UL/HZ) /* maximum udelay argument */ #define __MAX_NDELAY (4294967295UL/HZ) /* maximum ndelay argument */
extern __inline__ void __udelay(unsigned int x)
{
unsigned int loops;
__asm__("mulhwu %0,%1,%2" : "=r" (loops) :
"r" (x), "r" (loops_per_jiffy * 226));
__delay(loops);
}extern __inline__ void __ndelay(unsigned int x)
{
unsigned int loops;
__asm__("mulhwu %0,%1,%2" : "=r" (loops) :
"r" (x), "r" (loops_per_jiffy * 5));
__delay(loops);
}extern void __bad_udelay(void); /* deliberately undefined */
extern void __bad_ndelay(void); /* deliberately undefined */
#define udelay(n) (__builtin_constant_p(n)? \
((n) > __MAX_UDELAY? __bad_udelay(): __udelay((n) * (19 * HZ))) : \
__udelay((n) * (19 * HZ)))#define ndelay(n) (__builtin_constant_p(n)? \
((n) > __MAX_NDELAY? __bad_ndelay(): __ndelay((n) * HZ)) : \
__ndelay((n) * HZ))Thus:
udelay(const) = loops_per_jiffy * 5; mulhwu thing; call to __delay()
While with proposed code:
udelay(const) = call to udelay()
Which is smaller.
--
vda
Quoted 2 lines · Denis Vlasenko
I'm ok with making it fail, but _predictably_. With
printk(), trace, whatever.
BUG_ON() is probably a good candidate, especially as the implementation
is expected to be compact and can even be removed for those saving every byte by undefining CONFIG_BUG.
Maciej
Quoted 9 lines · Denis Vlasenko
Quoted 4 lines · Russell King
The maximum delay is dependent on the architecture implementation, and it depends on bogomips. There is no one single value for it. Architectures have to decide this from the way that they do the math and the expected range of bogomips.
In example I posted these limitations are lifted. Granted these limitations were not critical, but removing them can't do harm, I guess?
They're lifted poorly. You include a mandatory division in the path. On systems where division has to be done in code, this is not acceptable, especially when we're trying to get short delays on embedded CPUs running below 100MHz. The time it takes to do the division could swamp the required delay value.
Signature · 3 lines
Russell King Linux kernel 2.6 ARM Linux - http://www.arm.linux.org.uk/ maintainer of: 2.6 Serial core
Quoted 11 lines · Russell King
Quoted 9 lines · Denis Vlasenko
Quoted 4 lines · Russell King
The maximum delay is dependent on the architecture implementation, and it depends on bogomips. There is no one single value for it. Architectures have to decide this from the way that they do the math and the expected range of bogomips.
In example I posted these limitations are lifted. Granted these limitations were not critical, but removing them can't do harm, I guess?
Quoted 5 lines
They're lifted poorly. You include a mandatory division in the path. On systems where division has to be done in code, this is not acceptable, especially when we're trying to get short delays on embedded CPUs running below 100MHz. The time it takes to do the division could swamp the required delay value.
What divisions? Where?
void udelay(unsigned int usecs)
{
unsigned int k = usecs/1024;
while (k--)
__udelay(1024);
__udelay(usecs % 1024);
}
I see no divisions. I see shifts and ANDs.
I can code them explicitly:
void udelay(unsigned int usecs)
{
unsigned int k = usecs >> 10; /* divide by 1024 */ while (k--)
__udelay(0x400); /* 1024 */ __udelay(usecs & 0x3ff); /* mod 1024 */
}
Should be ok now.
--
vda
Quoted 11 lines · Arjan van de Ven
Quoted 10 lines · Denis Vlasenko
Quoted 9 lines · Arjan van de Ven
Quoted 9 lines
Quoted 5 lines
There are a number of compile-time checks that your patch has removed which catch such things, and as such your patch is not acceptable. Some architectures have a lower threshold of acceptability for the maximum udelay value, so it's absolutely necessary to keep this.
It removes that check from x86 - other architectures retain it.
Quoted 8 lines
Quoted 2 lines
For users, _any_ value, however large, will work for any delay function.
that's not desired though. Desired is to limit
udelay()to say 2000 or so. And force anything above that to go viamdelay()(just to make it stand out as broken code ;)Over time we also want to phase out mdelay of course...
The joystick drivers will (sadly) need mdelay forever, due to hardware crappines.
Signature · 2 lines
Vojtech Pavlik SuSE Labs, SuSE CR
Quoted 6 lines · Andrew Morton
Quoted 5 lines · Russell King
Rejected-by: Russell King 8)
The reason is that now we're unable to find out if anyone's doing udelay(100000000000000000) which breaks on most architectures.
Quoted 7 lines
Quoted 4 lines
There are a number of compile-time checks that your patch has removed which catch such things, and as such your patch is not acceptable. Some architectures have a lower threshold of acceptability for the maximum udelay value, so it's absolutely necessary to keep this.
I don't recall seeing anyone trigger the check, and it hardly seems worth adding a "few kb" to vmlinux for it?
Maybe we can have both - would the space saving be achieved by just moving mdelay and ssleep out of linux/delay.h and not touching asm-i386/delay.h?
Signature · 3 lines
Russell King Linux kernel 2.6 ARM Linux - http://www.arm.linux.org.uk/ maintainer of: 2.6 Serial core
Quoted 7 lines · Denis Vlasenko
Hi Andrew,
Optimizing delay functions for speed is utterly pointless.
This patch turns ssleep(n), mdelay(n), udelay(n) and ndelay(n) into functions, thus they generate the smallest possible code at the callsite. Previously they were more or less inlined.
Optimizing mdelay() and udelay() for speed is pointless, but optimizing ndelay() makes a lot of sense - if the setup time (call, etc) of the delay is large, the delay time will be off by many percent.
Signature · 2 lines
Vojtech Pavlik SuSE Labs, SuSE CR