2.6.19-rc6-rt4, changed yum repository
i've released the 2.6.18-rc6-rt4 tree, which can be downloaded from the usual place:
http://redhat.com/~mingo/realtime-preempt/
NOTE: the YUM repository has changed the -rt kernel's package name, it's now kernel-rt, so it does not override the kernel package. If you have rt.repo already then just do a "yum install kernel-rt".
If there's no rt repository yet, the -rt YUM repository for Fedora Core 6 can be activated via:
cd /etc/yum.repos.d
wget http://people.redhat.com/~mingo/realtime-preempt/rt.repo
yum install kernel-rt
a number of fixes were done since -rt3, and merges of Linus' latest -git tree.
to build a 2.6.19-rc6-rt4 tree, the following patches should be applied:
http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.18.tar.bz2
http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.19-rc6.bz2
http://redhat.com/~mingo/realtime-preempt/patch-2.6.19-rc6-rt4
as usual, bugreports, fixes and suggestions are welcome,
Ingo
Hi,
Boot bugs happening here on fc6 running locally compiled 2.6.19-rc-rt UP i386 kernels on a K8.
They happen on some boots, some are ok.
If bug happens, boot seams to stop after the
"for interactive setup , press 'I'" sort of message
and continues after I enter control+c.
Next message after that is about the time having been set. If bug happened, the time is incorrectly set mostly to 3600s in the future.
This is the last part of SysRq+T right after such a bug happened:
=======================
init S [f7e6d000] F7EB1F60 0 296 1 297 286 (NOTLB)
f7eb1f28 00000086 c014928e f7eb1f60 f7fb07b8 c191d6c4 f7eeb1a0 c191d600
00000007 f7e6d12c f7e6d000 d4b9b4a7 00000007 0000262b c014a6ab f6867568
00018a21 00000000 f7fc1000 00000001 f7eb1f88 f7eb1f40 c02ca94b f7feee00
Call Trace:
[<c014928e>] do_wp_page+0x34a/0x395
[<c014a6ab>] __handle_mm_fault+0x7b9/0x7db
[<c02ca94b>] schedule+0xcd/0xe9
[<c0118903>] eligible_child+0x9f/0xb0
[<c0119d53>] do_wait+0x9f1/0xacd
[<c0114409>] default_wake_function+0x0/0x16
[<c0119e60>] sys_wait4+0x31/0x34
[<c0119e8a>] sys_waitpid+0x27/0x2b
[<c0102e49>] sysenter_past_esp+0x56/0x79
=======================
rc.sysinit S [f7fc1000] FF889000 0 297 296 322 (NOTLB)
f7e76f28 00000086 c01492cf ff889000 08113fcc f6873860 f7fee600 c17f5760
00000006 f7fc112c f7fc1000 dd1e3416 00000008 0000df7d c014a642 f687744c
011543fb 00000000 f7e7baa0 00000001 f7e76f88 f7e76f40 c02ca94b f7fee600
Call Trace:
[<c01492cf>] do_wp_page+0x38b/0x395
[<c014a642>] __handle_mm_fault+0x750/0x7db
[<c02ca94b>] schedule+0xcd/0xe9
[<c0118903>] eligible_child+0x9f/0xb0
[<c0119d53>] do_wait+0x9f1/0xacd
[<c012fd91>] rt_down+0xe/0x25
[<c0114409>] default_wake_function+0x0/0x16
[<c0119e60>] sys_wait4+0x31/0x34
[<c0119e8a>] sys_waitpid+0x27/0x2b
[<c0102e49>] sysenter_past_esp+0x56/0x79
=======================
hwclock R [f7e7baa0] F6873224 [on rq #0] 0 322 297 (NOTLB)
c1902fb4 00003086 f7fee434 f6873224 f7fee434 c02cd498 00000000 00000000
00000001 f7e7bbcc f7e7baa0 795f1ab5 0000000d 003cda22 c02e8fc2 00000004
4e8951e9 00000002 00240384 00000000 00000000 c1902000 c0102f22 00240384
Call Trace:
[<c02cd498>] do_page_fault+0x2b9/0x552
[<c0102f22>] work_resched+0x6/0x19
=======================work_resched sits in entry.S:
work_resched:
DISABLE_INTERRUPTS
call __schedule
# make sure we don't miss an interrupt # setting need_resched or sigpending # between sampling and the iret
movl TI_flags(%ebp), %ecx
andl $_TIF_WORK_MASK, %ecx # is there any work to be done other
# than syscall tracing?
jz restore_all
testl $(_TIF_NEED_RESCHED|_TIF_NEED_RESCHED_DELAYED), %ecx jnz work_resched
The hwclock page_fault happens at the
movl TI_flags(%ebp), %ecx
line.
Will try the yum repo kernel next.
Karsten
Quoted 13 lines · Karsten Wiese
work_resched:
DISABLE_INTERRUPTS
call__schedule
# make sure we don't miss an interrupt # settingneed_reschedor sigpending # between sampling and the iret
movlTI_flags(%ebp), %ecx
andl $_TIF_WORK_MASK, %ecx # is there any work to be done other
# than syscall tracing?
jzrestore_all
testl $(_TIF_NEED_RESCHED|_TIF_NEED_RESCHED_DELAYED), %ecx jnzwork_resched
Quoted 3 lines
The hwclock
page_faulthappens at the
movlTI_flags(%ebp), %ecx
line.
hm, weird - maybe something corrupts %ebp here? Could you try to add this to before the faulting instruction:
GET_THREAD_INFO(%ebp)
this will make sure %ebp has the right contents.
Ingo
Ingo,
Quoted 2 lines · Ingo Molnar
i've released the 2.6.18-rc6-rt4 tree, which can be downloaded from the usual place:
I have problems runinng cyclictest on later -rt kernels, up to -rt4.
This is a Dell Latitude D610, with Pentium M & ICH6 chipset.
Symptoms:
- "cyclictest -n -p 90 -t 4" is just "running upwards"
With 2.6.18-rt5 it looked like this, much better:
isonoe:/home/rsc/tmp/rtpreempt/cyclictest-0.11# ./cyclictest -n -t 4 -p 90 -l 1000 1.51 0.99 0.41 1/117 9885
T: 0 ( 9802) P:90 I: 1000 C: 1000 Min: 7 Act: 11 Avg: 16 Max: 561 T: 1 ( 9803) P:89 I: 1500 C: 671 Min: 7 Act: 11 Avg: 15 Max: 498 T: 2 ( 9804) P:88 I: 2000 C: 504 Min: 7 Act: 11 Avg: 16 Max: 533 T: 3 ( 9805) P:87 I: 2500 C: 403 Min: 6 Act: 11 Avg: 13 Max: 394
- Using a relative timer (-r), I get huge latencies:
isonoe:/home/rsc/tmp/rtpreempt/cyclictest-0.11# ./cyclictest -n -p 90 -t 4 -r -l 1000 0.47 0.76 0.61 3/128 16112
T: 0 (15901) P:90 I: 1000 C: 1000 Min: 2963 Act: 2990 Avg: 4334 Max: 7039 T: 1 (15902) P:89 I: 1500 C: 667 Min: 4003 Act: 6495 Avg: 6492 Max: 6541 T: 2 (15903) P:88 I: 2000 C: 667 Min: 3498 Act: 5995 Avg: 5993 Max: 6041 T: 3 (15904) P:87 I: 2500 C: 667 Min: 5433 Act: 5488 Avg: 5498 Max: 6898
This is with "lapic lapictimer" on the kernel commandline. I've manually switched the clocksource from pit to tsc:
isonoe:/sys/devices/system/clocksource/clocksource0# cat available_clocksource jiffies tsc pit isonoe:/sys/devices/system/clocksource/clocksource0# cat current_clocksource tsc
Some related dmesg output:
Kernel command line: root=/dev/sda7 ro lapic lapictimer
Found and enabled local APIC!
mapped APIC to ffffd000 (fee00000)
...
WARNING: experimental RCU implementation.
Clock event device pit configured with caps set: 07
...
Clock event device pit new caps set: 03
Clock event device lapic configured with caps set: 04
...
Time: pit clocksource has been installed.
...
*****************************************************************************
* *
* REMINDER, the following debugging options are turned on in your .config: *
* * * CONFIG_CRITICAL_PREEMPT_TIMING * * CONFIG_CRITICAL_IRQSOFF_TIMING * * CONFIG_FUNCTION_TRACE * * * * they may increase runtime overhead and latencies. * * *
*****************************************************************************
Last time this machine worked was with 2.6.18-rt5, 2.6.18-rt7 didn't work any more. If it is helpful, I can do some further boot tests to find out which patch broke it.
Robert
Signature · 5 lines
Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
Pengutronix - Linux Solutions for Science and Industry
Handelsregister: Amtsgericht Hildesheim, HRA 2686
Hannoversche Str. 2, 31134 Hildesheim, Germany
Phone: +49-5121-206917-0 | Fax: +49-5121-206917-9Am Sonntag, 19. November 2006 14:43 schrieb Ingo Molnar:
Quoted 20 lines · Karsten Wiese
Quoted 14 lines
work_resched:
DISABLE_INTERRUPTS
call__schedule
# make sure we don't miss an interrupt # settingneed_reschedor sigpending # between sampling and the iret
movlTI_flags(%ebp), %ecx
andl $_TIF_WORK_MASK, %ecx # is there any work to be done other
# than syscall tracing?
jzrestore_all
testl $(_TIF_NEED_RESCHED|_TIF_NEED_RESCHED_DELAYED), %ecx jnzwork_reschedQuoted 3 lines
The hwclock
page_faulthappens at the
movlTI_flags(%ebp), %ecx
line.
Quoted 6 lines
hm, weird - maybe something corrupts %ebp here? Could you try to add this to before the faulting instruction:
GET_THREAD_INFO(%ebp)this will make sure %ebp has the right contents.
Doesn't make a difference:
clock is still set an hour early during boot occasionally. An hour offset I also get when I comment out the hwclock call in rc.sysinit.
The Sysrq+T output with GET_THREAD_INFO(%ebp) has:
=======================
hwclock R [f7f76550] C1B07224 [on rq #0] 0 329 304 (NOTLB)
f7f6efb4 00003086 c1907434 c1b07224 c1907434 c02d320f 00000000 00000000
00000001 f7f7667c f7f76550 ad91991e 00000008 001349cd c02ef1fe 00000004
d1292e17 00000000 000cc113 00000000 00000000 f7f6e000 c0102f22 000cc113
Call Trace:
[<c02d320f>] do_page_fault+0x2b9/0x552
[<c0102f22>] work_resched+0x6/0x20
=======================The [<c0102f22>] work_resched+0x6/0x20 corresponds to
mov $0xfffff000,%ebp in: (gdb) disassemble work_resched Dump of assembler code for function work_resched: 0x000001c0 <work_resched+0>: cli 0x000001c1 <work_resched+1>: call 0x1c2 <work_resched+2> 0x000001c6 <work_resched+6>: mov $0xfffff000,%ebp 0x000001cb <work_resched+11>: and %esp,%ebp 0x000001cd <work_resched+13>: mov 0x8(%ebp),%ecx 0x000001d0 <work_resched+16>: and $0xfe3e,%ecx 0x000001d6 <work_resched+22>: je 0x16a <restore_all> 0x000001d8 <work_resched+24>: test $0x80008,%ecx 0x000001de <work_resched+30>: jne 0x1c0 <work_resched>
End of assembler dump.
But "mov $0xfffff000,%ebp" can't cause a pagefault.
So either the Sysrq+T output is wrong or the actual page_fault happens inside the "call __schedule" with __schedule missing from the Call Trace.
Your yum-repo kernel seams to stay clear of above problem. Obvious differences to my .config are SMP <> UP and M686+X86_GENERIC <> MK8.
Karsten
Quoted 9 lines · Karsten Wiese
Call Trace: [<c02d320f>] do_page_fault+0x2b9/0x552 [<c0102f22>] work_resched+0x6/0x20The [<
c0102f22>]work_resched+0x6/0x20corresponds tomov $0xfffff000,%ebp0x000001c1 <work_resched+1>: call 0x1c2 <work_resched+2> 0x000001c6 <work_resched+6>: mov $0xfffff000,%ebp
no, it's the call's return address that is work_resched+6.
to get a more usable snapshot of what this task is doing you'd need something like SysRq-P output. (that works on PREEMPT_RT only if you enable /proc/sys/kernel/debug_direct_keyboard - but careful, it might break if you generate too many interrupts - i usually only to do the SysRq-P and hope that it doesnt break then.)
Ingo
Quoted 11 lines · Ingo Molnar
i've released the 2.6.18-rc6-rt4 tree, which can be downloaded from the usual place:
http://redhat.com/~mingo/realtime-preempt/
NOTE: the YUM repository has changed the -rt kernel's package name, it's now kernel-rt, so it does not override the kernel package. If you have rt.repo already then just do a "yum install kernel-rt".
If there's no rt repository yet, the -rt YUM repository for Fedora Core 6 can be activated via:
Can you provide kernel-rt-devel for compile nvidia DRI kernel module ?
I had tried 2.6.19-rc6-rt4, and no regressions from 2.6.18+dyntick.
but I still can freeze computer when make heavy copy to SATA HD. Thanks,
Sérgio M. B.
Am Sonntag, 19. November 2006 22:14 schrieb Ingo Molnar:
Quoted 13 lines · Karsten Wiese
Quoted 10 lines
Call Trace: [<c02d320f>] do_page_fault+0x2b9/0x552 [<c0102f22>] work_resched+0x6/0x20The [<
c0102f22>]work_resched+0x6/0x20corresponds tomov $0xfffff000,%ebp0x000001c1 <work_resched+1>: call 0x1c2 <work_resched+2> 0x000001c6 <work_resched+6>: mov $0xfffff000,%ebp
Quoted 7 lines
no, it's the call's return address that is work_resched+6.
to get a more usable snapshot of what this task is doing you'd need something like SysRq-P output. (that works on
PREEMPT_RTonly if you enable/proc/sys/kernel/debug_direct_keyboard- but careful, it might break if you generate too many interrupts - i usually only to do the SysRq-P and hope that it doesnt break then.)
Thanks. It turned out, it was me having missed to .config enable
"Enhanced Real Time Clock Support"
. Sorry for having bothered you about this.
hwclock then couldn't open /dev/rtc, fell back to iopl(3) hw access. And that failed sometimes ;-)
I'd still like to know, why are there the do_page_fault() SysRq+T traces under hwclock context, while hwclock userspace is in a loop doing iopl(3)ed io-access?
hwclock's loop active then is:
for (i = 0; !cmos_clock_busy(); i++)
if (i >= 10000000)
return 1;
with:
int cmos_clock_busy()
{
return /* poll bit 7 (UIP) of Control Register A */
(hclock_read(10) & 0x80);
}
with hand interpreted:
unsigned long hclock_read(unsigned long reg)
{
unsigned long v;
__asm__ volatile ("cli");
outb (reg, clock_ctl_addr);
v = inb (clock_data_addr);
__asm__ volatile ("sti");
return v;
}I guess:
SysRq+T doesn't care about userspace eip and just displays hwclock's last used kernel stack?
Karsten