Work around for periodic do_gettimeofday hang
PATCH 1/1 Work around for periodic do_gettimeofday hang
On the voyager systems particularly (but also on some of my slower parisc boxes) I periodically get hangs where the system just seems to stop (although it does remain able to execute alt-sysrq). The traces always seem to implicate do_gettimeofday in the xtime seqlock.
I've not been able to trace an exact cause for this, but it does seem to be pretty much eliminated by lowering the clock speed to 100HZ. I propose the following patch to do this for the slower intel processors (basically pentiums and below)
James
--
===== include/asm-i386/param.h 1.6 vs edited =====
include/asm-i386/param.h +8 -1
@@ -1,8 +1,15 @@#ifndef _ASMi386_PARAM_H#define _ASMi386_PARAM_H +#include <linux/config.h> +#ifdef __KERNEL__ -# define HZ 1000/* Internal kernel timer frequency */ +# if defined(CONFIG_M386 ) || defined(CONFIG_M486 ) || defined(CONFIG_M586 ) \ + || defined(CONFIG_M586TSC ) || defined(CONFIG_M586MMX ) +# define HZ 100 +# else +# define HZ 1000/* Internal kernel timer frequency */ +# endif # define USER_HZ 100/* .. some user interfaces are in "ticks" */ # define CLOCKS_PER_SEC (USER_HZ)/* like times() */ #endif
Quoted 5 lines
I've not been able to trace an exact cause for this, but it does seem to be pretty much eliminated by lowering the clock speed to 100HZ. I propose the following patch to do this for the slower intel processors (basically pentiums and below)
while I agree with 100Hz for slower cpus, I rather have a config option for it so people (and distros) can select it independent of the exact cpu type they want to compile a kernel for
Quoted 2 lines · Arjan van de Ven
while I agree with 100Hz for slower cpus, I rather have a config option for it so people (and distros) can select it independent of the exact cpu type they want to compile a kernel for
How about this then?
James
===== arch/i386/Kconfig 1.134 vs edited =====
arch/i386/Kconfig +8 -0
@@ -330,6 +330,14 @@ endchoice +config X86_HZ + int "Clock Tick Rate" + default 1000 if !(M386 || M486 || M586 || M586TSC || M586MMX) + default 100 if (M386 || M486 || M586 || M586TSC || M586MMX) + help + Select the kernel clock tick rate in interrupts per second. + Slower processors should choose 100; everything else 1000. + config X86_GENERIC bool "Generic x86 support" help
===== include/asm-i386/param.h 1.6 vs edited =====
include/asm-i386/param.h +3 -1
@@ -1,8 +1,10 @@#ifndef _ASMi386_PARAM_H#define _ASMi386_PARAM_H +#include <linux/config.h> +#ifdef __KERNEL__ -# define HZ 1000/* Internal kernel timer frequency */ +# define HZ (CONFIG_X86_HZ)# define USER_HZ 100/* .. some user interfaces are in "ticks" */ # define CLOCKS_PER_SEC (USER_HZ)/* like times() */ #endif
Quoted 8 lines · James Bottomley
+config
X86_HZ+ int "Clock Tick Rate" + default 1000 if !(M386 || M486 || M586 || M586TSC || M586MMX) + default 100 if (M386 || M486 || M586 || M586TSC || M586MMX) + help+ Select the kernel clock tick rate in interrupts per second.
+ Slower processors should choose 100; everything else 1000.
I guess we don't need the help, given that it's not a menuisable option. (There was a make-HZ-selectable patch once, and Linus spat it out).
Silly question: how come do_gettimeofday() is hanging?
Quoted 1 line · Andrew Morton
Silly question: how come
do_gettimeofday()is hanging?
That's what I don't know. like I said, the xtime seqlock seemed to be implicated, but I've never managed to trace it down.
I also notice that under heavy load the system starts to lose time (at least according to ntp). I suspect there's something wrong with the PIT timer routine (since that's what voyager has to use) for adding lost ticks.
James
Quoted 10 lines · Andrew Morton
Quoted 8 lines · James Bottomley
+config
X86_HZ+ int "Clock Tick Rate" + default 1000 if !(M386 || M486 || M586 || M586TSC || M586MMX) + default 100 if (M386 || M486 || M586 || M586TSC || M586MMX) + help+ Select the kernel clock tick rate in interrupts per second.
+ Slower processors should choose 100; everything else 1000.
Quoted 2 lines
I guess we don't need the help, given that it's not a menuisable option. (There was a make-HZ-selectable patch once, and Linus spat it out).
I'd rather have it as menu-visible option, but I can see how arbitrary HZ gets spat out. How about a radio-button like thing with "100" and "1000" in it.
HZ=1000 costs you 1% HPC performance, and for slower machines probably more. Also some hw (with slow smm) really doesn't likeit.
Quoted 1 line · Arjan van de Ven
HZ=1000 costs you 1% HPC performance, and for slower machines probably more. Also some hw (with slow smm) really doesn't likeit.
I actually profiled the timer ISR on a 600Mhz C3. It runs for 22 usec. So with HZ=1000 the residency of the timer interrupt is 2.2%.
Lee
Quoted 5 lines · Arjan van de Ven
Quoted 2 lines
while I agree with 100Hz for slower cpus, I rather have a config option for it so people (and distros) can select it independent of the exact cpu type they want to compile a kernel for
How about this then?
I'd rather make it a number field rather than "choose-me", so the wise user can choose any Hz he likes.
Jan Engelhardt
Signature · 2 lines
Gesellschaft für Wissenschaftliche Datenverarbeitung Am Fassberg, 37077 Göttingen, www.gwdg.de
Hi,
arch/i386/Kconfig line 336
Quoted 4 lines · James Bottomley
arch/i386/Kconfig +4
-0+config X86_HZ + int "Clock Tick Rate" + default 1000 if !(M386 || M486 || M586 || M586TSC || M586MMX) + default 100 if (M386 || M486 || M586 || M586TSC || M586MMX)
You don't have to duplicate the conditions, kconfig takes the first match.
bye, Roman
Quoted 4 lines · James Bottomley
On the voyager systems particularly (but also on some of my slower parisc boxes) I periodically get hangs where the system just seems to stop (although it does remain able to execute alt-sysrq). The traces always seem to implicate
do_gettimeofdayin the xtime seqlock.
What are the other CPUs doing in this case ?
Quoted 1 line · Alan Cox
What are the other CPUs doing in this case ?
Actually, I don't know ... I can't seem to persuade the voyagers to broadcast an NMI to give up that information.
However, unfortunately, I got a similar hang at HZ=100 this time not implicating do_gettimeofday.
I'll continue investigating.
James