ia64: disable preemption in udelay()
Resubmitting using Tony Luck's suggested alternative.
The udelay() inline for ia64 uses the ITC. If CONFIG_PREEMPT is enabled and the platform has unsynchronized ITCs and the calling task migrates to another CPU while doing the udelay loop, then the effective delay may be too short or very, very long.
This patch disables preemption around 100 usec chunks of the overall desired udelay time. This minimizes preemption-holdoffs.
Index: linux/include/asm-ia64/delay.h
=================================--- linux.orig/include/asm-ia64/delay.h 2005-12-15 15:34:16.000000000 -0800
+++ linux/include/asm-ia64/delay.h 2005-12-15 15:34:57.000000000 -0800
@@ -84,14 +84,6 @@
ia64_delay_loop (loops - 1);
}
-static __inline__ void
-udelay (unsigned long usecs)
-{
- unsigned long start = ia64_get_itc();
- unsigned long cycles = usecs*local_cpu_data->cyc_per_usec;
-
- while (ia64_get_itc() - start < cycles)
- cpu_relax();
-}
+extern void udelay (unsigned long usecs);
#endif /* _ASM_IA64_DELAY_H */
Index: linux/arch/ia64/kernel/time.c
=================================--- linux.orig/arch/ia64/kernel/time.c 2005-12-15 15:34:16.000000000 -0800
+++ linux/arch/ia64/kernel/time.c 2005-12-15 15:34:57.000000000 -0800
@@ -249,3 +249,31 @@
*/
set_normalized_timespec(&wall_to_monotonic, -xtime.tv_sec, -xtime.tv_nsec);
}
+
+#define SMALLUSECS 100
+
+void
+udelay (unsigned long usecs)
+{
+ unsigned long start;
+ unsigned long cycles;
+ unsigned long smallusecs;
+
+ /*
+ * Execute the non-preemptible delay loop (because the ITC might
+ * not be synchronized between CPUS) in relatively short time
+ * chunks, allowing preemption between the chunks.
+ */
+ while (usecs > 0) {
+ smallusecs = (usecs > SMALLUSECS) ? SMALLUSECS : usecs;
+ preempt_disable();
+ cycles = smallusecs*local_cpu_data->cyc_per_usec;
+ start = ia64_get_itc();
+
+ while (ia64_get_itc() - start < cycles)
+ cpu_relax();
+
+ preempt_enable();
+ usecs -= smallusecs;
+ }
+}
Review
Quoted 2 lines · On Thu, Dec 15, 2005 at 06:42:52PM -0800, [email protected] wrote:
(quoted diff) +2
-0+ +#define SMALLUSECS 100
John, I did not see your posts until this had already made it out. I would think that the folks running realtime applications would expect udelay to hold off for even shorter periods of time. I would expect something along the line of 20 or 25 uSec.
Quoted 16 lines
(quoted diff) +16
-0+ +void +udelay (unsigned long usecs) +{ + unsigned long start; + unsigned long cycles; + unsigned long smallusecs; + + /* + * Execute the non-preemptible delay loop (because the ITC might + * not be synchronized between CPUS) in relatively short time + * chunks, allowing preemption between the chunks. + */ + while (usecs > 0) { + smallusecs = (usecs > SMALLUSECS) ? SMALLUSECS : usecs; + preempt_disable();
Review
Quoted 10 lines
(quoted diff) +10
-0+ cycles = smallusecs*local_cpu_data->cyc_per_usec; + start = ia64_get_itc(); + + while (ia64_get_itc() - start < cycles) + cpu_relax(); + + preempt_enable(); + usecs -= smallusecs; + } +}
How much drift would you expect from this? I have not tried this, but what about something more along the lines of:
#define MAX_USECS_WHILE_NOT_PREMPTIBLE 20
void
udelay (unsigned long usecs)
{
unsigned long next, timeout;
long last_processor = -1;
/*
* Execute the non-preemptible delay loop (because the ITC might
* not be synchronized between CPUS) in relatively short time
* chunks, allowing preemption between the chunks.
*/
while (usecs > 0) {
next = min(usecs, MAX_USECS_WHILE_NOT_PREMPTIBLE);
preempt_disable;
if (last_processor != smp_processor_id()) {
last_processor = smp_processor_id();
timeout = ia64_get_itc();
}
timeout += next * local_cpu_data->cyc_per_usec;
while (ia64_get_itc() < timeout)
cpu_relax();
preempt_enable;
usecs -= next;
}
}Quoted 6 lines · Robin Holt
Quoted 1 line
+#define SMALLUSECS 100
John, I did not see your posts until this had already made it out. I would think that the folks running realtime applications would expect udelay to hold off for even shorter periods of time. I would expect something along the line of 20 or 25 uSec.
A good question ... I'm going to put John's change in as-is for now so that 2.6.15 can benefit from the reduced code size of the out-of-line and avoid the ugly bug when preemption is enabled on a drifty system.
We can make fine tune changes to the udelay() implementation after we get some data on what is needed.
Quoted 4 lines
How much drift would you expect from this? I have not tried this, but what about something more along the lines of:
#define
MAX_USECS_WHILE_NOT_PREMPTIBLE20
As we reduce the non-preemtible window drift in my version of udelay() would get worse ... but I haven't done any measurements on how much worse.
Quoted 3 lines
timeout += next * local_cpu_data->cyc_per_usec; while (ia64_get_itc() < timeout) cpu_relax();
Bad news if your ar.itc wraps around (less than four centuries of uptime at 1.6GHz :-)
-Tony
From: "Luck, Tony" <[email protected]>
...
Quoted 3 lines
A good question ... I'm going to put John's change in as-is for now so that 2.6.15 can benefit from the reduced code size of the out-of-line and avoid the ugly bug when preemption is enabled on a drifty system.
Okay, but I'll propose some changes to that soon, probably using Robin Holt's suggestions. I'm concerned about the impact of interrupts outside of the inner loop and inside the outer loop, which would increase the effective delay time.
John Hawkes