CONFIG_PRINTK_TIME broken on git HEAD ?
Hi all,
I've updated today to 2.6.26-rc1-00166-gc0a1811 , running an 64bit kernel at the moment.
dmesg shows here :
...
[ 0.000000] Allocating PCI resources starting at d0000000 (gap: cf600000:2f800000) [ 0.000000] SMP: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] PERCPU: Allocating 42756 bytes of per cpu data [ 0.000000] NR_CPUS: 8, nr_cpu_ids: 4 [4294014.506571] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 1010753 [4294014.506571] Kernel command line: root=/dev/sdb1 ro debug vga=0x317 [4294014.506571] Initializing CPU#0 [4294014.506571] Preemptible RCU implementation. [4294014.506571] PID hash table entries: 4096 (order: 12, 32768 bytes)
...
Is this a known issue ?
Regards,
Gabriel
Quoted 3 lines · On Thu, 08 May 2008 00:26:04 +0200 Gabriel C <[email protected]> wrote:
Hi all,
I've updated today to 2.6.26-rc1-00166-gc0a1811 , running an 64bit kernel at the moment.
Which kernel were you runing previously?
Quoted 15 lines
dmesg shows here :
...
[ 0.000000] Allocating PCI resources starting at d0000000 (gap: cf600000:2f800000) [ 0.000000] SMP: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] PERCPU: Allocating 42756 bytes of per cpu data [ 0.000000] NR_CPUS: 8, nr_cpu_ids: 4 [4294014.506571] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 1010753 [4294014.506571] Kernel command line: root=/dev/sdb1 ro debug vga=0x317 [4294014.506571] Initializing CPU#0 [4294014.506571] Preemptible RCU implementation. [4294014.506571] PID hash table entries: 4096 (order: 12, 32768 bytes)
Quoted 5 lines
...
Is this a known issue ?
Not to me.
Quoted 8 lines · Andrew Morton
Quoted 3 lines · On Thu, 08 May 2008 00:26:04 +0200 Gabriel C <[email protected]> wrote:
Hi all,
I've updated today to 2.6.26-rc1-00166-gc0a1811 , running an 64bit kernel at the moment.
Which kernel were you runing previously?
2.6.25* , tested 2.6.25-git* , 2.6.26-rc1 , and 2.6.26-rc1-00065-g5717922.
Quoted 1 line
From my log 2.6.26-rc1-00065-g5717922 was fine.
...
May 6 00:13:14 thor kernel: [ 0.000000] Allocating PCI resources starting at d0000000 (gap: cf600000:2f800000) May 6 00:13:14 thor kernel: [ 0.000000] SMP: Allowing 4 CPUs, 0 hotplug CPUs May 6 00:13:14 thor kernel: [ 0.000000] PERCPU: Allocating 41732 bytes of per cpu data May 6 00:13:14 thor kernel: [ 0.000000] Kernel command line: root=/dev/sdb1 ro debug vga=0x317 May 6 00:13:14 thor kernel: [ 0.000000] Preemptible RCU implementation. May 6 00:13:14 thor kernel: [ 0.000000] Extended CMOS year: 2000 May 6 00:13:14 thor kernel: [ 0.000000] TSC calibrated against PM_TIMER May 6 00:13:14 thor kernel: [ 0.000014] time.c: Detected 2499.952 MHz processor. May 6 00:13:14 thor kernel: [ 0.000032] console [tty0] enabled May 6 00:13:14 thor kernel: [ 0.000999] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes) May 6 00:13:14 thor kernel: [ 0.000999] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes) May 6 00:13:14 thor kernel: [ 0.000999] Checking aperture...
...
Quoted 16 lines
Quoted 16 lines
dmesg shows here :
...
[ 0.000000] Allocating PCI resources starting at d0000000 (gap: cf600000:2f800000) [ 0.000000] SMP: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] PERCPU: Allocating 42756 bytes of per cpu data [ 0.000000] NR_CPUS: 8, nr_cpu_ids: 4 [4294014.506571] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 1010753 [4294014.506571] Kernel command line: root=/dev/sdb1 ro debug vga=0x317 [4294014.506571] Initializing CPU#0 [4294014.506571] Preemptible RCU implementation. [4294014.506571] PID hash table entries: 4096 (order: 12, 32768 bytes)
Quoted 8 lines
Quoted 5 lines
...
Is this a known issue ?
Not to me.
Gabriel
Quoted 29 lines · Gabriel C
Quoted 7 lines · Andrew Morton
Quoted 3 lines · On Thu, 08 May 2008 00:26:04 +0200 Gabriel C <[email protected]> wrote:
Hi all,
I've updated today to 2.6.26-rc1-00166-gc0a1811 , running an 64bit kernel at the moment.
Which kernel were you runing previously?
2.6.25* , tested 2.6.25-git* , 2.6.26-rc1 , and 2.6.26-rc1-00065-g5717922.
Quoted 1 line
From my log 2.6.26-rc1-00065-g5717922 was fine.
...
May 6 00:13:14 thor kernel: [ 0.000000] Allocating PCI resources starting at d0000000 (gap: cf600000:2f800000) May 6 00:13:14 thor kernel: [ 0.000000] SMP: Allowing 4 CPUs, 0 hotplug CPUs May 6 00:13:14 thor kernel: [ 0.000000] PERCPU: Allocating 41732 bytes of per cpu data May 6 00:13:14 thor kernel: [ 0.000000] Kernel command line: root=/dev/sdb1 ro debug vga=0x317 May 6 00:13:14 thor kernel: [ 0.000000] Preemptible RCU implementation. May 6 00:13:14 thor kernel: [ 0.000000] Extended CMOS year: 2000 May 6 00:13:14 thor kernel: [ 0.000000] TSC calibrated against PM_TIMER May 6 00:13:14 thor kernel: [ 0.000014] time.c: Detected 2499.952 MHz processor. May 6 00:13:14 thor kernel: [ 0.000032] console [tty0] enabled May 6 00:13:14 thor kernel: [ 0.000999] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes) May 6 00:13:14 thor kernel: [ 0.000999] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes) May 6 00:13:14 thor kernel: [ 0.000999] Checking aperture...
Quoted 25 lines
...
Quoted 22 lines
Quoted 15 lines
dmesg shows here :
...
[ 0.000000] Allocating PCI resources starting at d0000000 (gap: cf600000:2f800000) [ 0.000000] SMP: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] PERCPU: Allocating 42756 bytes of per cpu data [ 0.000000] NR_CPUS: 8, nr_cpu_ids: 4 [4294014.506571] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 1010753 [4294014.506571] Kernel command line: root=/dev/sdb1 ro debug vga=0x317 [4294014.506571] Initializing CPU#0 [4294014.506571] Preemptible RCU implementation. [4294014.506571] PID hash table entries: 4096 (order: 12, 32768 bytes)Quoted 5 lines
...
Is this a known issue ?
Not to me.
I've bisected it.
git bisect log
git-bisect start
# good: [2ddcca36c8bcfa251724fe342c8327451988be0d] Linux 2.6.26-rc1
git-bisect good 2ddcca36c8bcfa251724fe342c8327451988be0d
# bad: [3de2403e6659d71b36ec820dc9b942762ddfe6eb] Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/sparc-2.6
git-bisect bad 3de2403e6659d71b36ec820dc9b942762ddfe6eb
# bad: [aac6abca858386438d9a7233c3471d2ecfa2f704] sched: default to n for GROUP_SCHED and FAIR_GROUP_SCHED
git-bisect bad aac6abca858386438d9a7233c3471d2ecfa2f704
# good: [45ea2103d8856454503b30464cc1dba378748d00] Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/x86/linux-2.6-x86-fixes
git-bisect good 45ea2103d8856454503b30464cc1dba378748d00
# good: [5717922a1b8cc477f45a0f4f11fe619392ba7133] Merge branch 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jwessel/linux-2.6-kgdb
git-bisect good 5717922a1b8cc477f45a0f4f11fe619392ba7133
# good: [b328ca182f01c2a04b85e0ee8a410720b104fbcc] sched: fix hrtick_start_fair and CPU-Hotplug
git-bisect good b328ca182f01c2a04b85e0ee8a410720b104fbcc
# good: [712555ee4f873515612f89554ad1a3fda5fa887e] sched: fix missing locking in sched_domains code
git-bisect good 712555ee4f873515612f89554ad1a3fda5fa887e
# good: [dfbf4a1bc319f0f9a31e39b2da1fa5c55e85af89] sched: fix cpu clock
git-bisect good dfbf4a1bc319f0f9a31e39b2da1fa5c55e85af89
# bad: [3e51f33fcc7f55e6df25d15b55ed10c8b4da84cd] sched: add optional support for CONFIG_HAVE_UNSTABLE_SCHED_CLOCK git-bisect bad 3e51f33fcc7f55e6df25d15b55ed10c8b4da84cd
# good: [a5574cf65b5f03ce9ade3918764fe22e5e2371e3] sched, x86: add HAVE_UNSTABLE_SCHED_CLOCK
git-bisect good a5574cf65b5f03ce9ade3918764fe22e5e2371e3
cat ./.git/refs/bisect/bad3e51f33fcc7f55e6df25d15b55ed10c8b4da84cd
git show 3e51f33fcc7f55e6df25d15b55ed10c8b4da84cd commit 3e51f33fcc7f55e6df25d15b55ed10c8b4da84cd
Author: Peter Zijlstra <[email protected]>
Date: Sat May 3 18:29:28 2008 +0200
sched: add optional support for CONFIG_HAVE_UNSTABLE_SCHED_CLOCK
this replaces the rq->clock stuff (and possibly cpu_clock()).
- architectures that have an 'imperfect' hardware clock can set CONFIG_HAVE_UNSTABLE_SCHED_CLOCK
- the 'jiffie' window might be superfulous when we update tick_gtod before the __update_sched_clock() call in sched_clock_tick()
- cpu_clock() might be implemented as:
sched_clock_cpu(smp_processor_id())
if the accuracy proves good enough - how far can TSC drift in a single jiffie when considering the filtering and idle hooks?
[ [email protected]: various fixes and cleanups ]
Gabriel
Quoted 58 lines · Gabriel C
Quoted 16 lines
Quoted 1 line
From my log 2.6.26-rc1-00065-g5717922 was fine.
May 6 00:13:14 thor kernel: [ 0.000000] Allocating PCI resources starting at d0000000 (gap: cf600000:2f800000) May 6 00:13:14 thor kernel: [ 0.000000] SMP: Allowing 4 CPUs, 0 hotplug CPUs May 6 00:13:14 thor kernel: [ 0.000000] PERCPU: Allocating 41732 bytes of per cpu data May 6 00:13:14 thor kernel: [ 0.000000] Kernel command line: root=/dev/sdb1 ro debug vga=0x317 May 6 00:13:14 thor kernel: [ 0.000000] Preemptible RCU implementation. May 6 00:13:14 thor kernel: [ 0.000000] Extended CMOS year: 2000 May 6 00:13:14 thor kernel: [ 0.000000] TSC calibrated against PM_TIMER May 6 00:13:14 thor kernel: [ 0.000014] time.c: Detected 2499.952 MHz processor. May 6 00:13:14 thor kernel: [ 0.000032] console [tty0] enabled May 6 00:13:14 thor kernel: [ 0.000999] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes) May 6 00:13:14 thor kernel: [ 0.000999] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes) May 6 00:13:14 thor kernel: [ 0.000999] Checking aperture...Quoted 16 lines
...
Quoted 14 lines
Quoted 14 lines
dmesg shows here :
...
[ 0.000000] Allocating PCI resources starting at d0000000 (gap: cf600000:2f800000) [ 0.000000] SMP: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] PERCPU: Allocating 42756 bytes of per cpu data [ 0.000000] NR_CPUS: 8, nr_cpu_ids: 4 [4294014.506571] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 1010753 [4294014.506571] Kernel command line: root=/dev/sdb1 ro debug vga=0x317 [4294014.506571] Initializing CPU#0 [4294014.506571] Preemptible RCU implementation. [4294014.506571] PID hash table entries: 4096 (order: 12, 32768 bytes)I've bisected it.
cat ./.git/refs/bisect/bad
3e51f33fcc7f55e6df25d15b55ed10c8b4da84cdgit show
3e51f33fcc7f55e6df25d15b55ed10c8b4da84cdcommit3e51f33fcc7f55e6df25d15b55ed10c8b4da84cd
Author: Peter Zijlstra <[email protected]>
Date: Sat May 3 18:29:28 2008 +0200sched: add optional support for
CONFIG_HAVE_UNSTABLE_SCHED_CLOCKthis replaces the
rq->clockstuff (and possiblycpu_clock()).- architectures that have an 'imperfect' hardware clock can set
CONFIG_HAVE_UNSTABLE_SCHED_CLOCK- the 'jiffie' window might be superfulous when we update
tick_gtodbefore the__update_sched_clock()call insched_clock_tick()-
cpu_clock()might be implemented as:sched_clock_cpu(smp_processor_id())
Quoted 8 lines
if the accuracy proves good enough - how far can TSC drift in a single jiffie when considering the filtering and idle hooks?
[ [email protected]: various fixes and cleanups ]
Signed-off-by Peter Zijlstra <[email protected]>Signed-off-by Ingo Molnar <[email protected]>
Ok, so the issue is that the clock doesn't start at 0, but at:
4294014.506571 ?
I guess that's a side-effect of using ktime to base the clock off. Thomas, do we have a clock that is uptime based?
Quoted 63 lines · Peter Zijlstra
Quoted 61 lines · Gabriel C
Quoted 16 lines
Quoted 1 line
From my log 2.6.26-rc1-00065-g5717922 was fine.
May 6 00:13:14 thor kernel: [ 0.000000] Allocating PCI resources starting at d0000000 (gap: cf600000:2f800000) May 6 00:13:14 thor kernel: [ 0.000000] SMP: Allowing 4 CPUs, 0 hotplug CPUs May 6 00:13:14 thor kernel: [ 0.000000] PERCPU: Allocating 41732 bytes of per cpu data May 6 00:13:14 thor kernel: [ 0.000000] Kernel command line: root=/dev/sdb1 ro debug vga=0x317 May 6 00:13:14 thor kernel: [ 0.000000] Preemptible RCU implementation. May 6 00:13:14 thor kernel: [ 0.000000] Extended CMOS year: 2000 May 6 00:13:14 thor kernel: [ 0.000000] TSC calibrated against PM_TIMER May 6 00:13:14 thor kernel: [ 0.000014] time.c: Detected 2499.952 MHz processor. May 6 00:13:14 thor kernel: [ 0.000032] console [tty0] enabled May 6 00:13:14 thor kernel: [ 0.000999] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes) May 6 00:13:14 thor kernel: [ 0.000999] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes) May 6 00:13:14 thor kernel: [ 0.000999] Checking aperture...Quoted 16 lines
...
Quoted 14 lines
Quoted 14 lines
dmesg shows here :
...
[ 0.000000] Allocating PCI resources starting at d0000000 (gap: cf600000:2f800000) [ 0.000000] SMP: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] PERCPU: Allocating 42756 bytes of per cpu data [ 0.000000] NR_CPUS: 8, nr_cpu_ids: 4 [4294014.506571] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 1010753 [4294014.506571] Kernel command line: root=/dev/sdb1 ro debug vga=0x317 [4294014.506571] Initializing CPU#0 [4294014.506571] Preemptible RCU implementation. [4294014.506571] PID hash table entries: 4096 (order: 12, 32768 bytes)I've bisected it.
cat ./.git/refs/bisect/bad
3e51f33fcc7f55e6df25d15b55ed10c8b4da84cdgit show
3e51f33fcc7f55e6df25d15b55ed10c8b4da84cdcommit3e51f33fcc7f55e6df25d15b55ed10c8b4da84cd
Author: Peter Zijlstra <[email protected]>
Date: Sat May 3 18:29:28 2008 +0200sched: add optional support for
CONFIG_HAVE_UNSTABLE_SCHED_CLOCKthis replaces the
rq->clockstuff (and possiblycpu_clock()).- architectures that have an 'imperfect' hardware clock can set
CONFIG_HAVE_UNSTABLE_SCHED_CLOCK- the 'jiffie' window might be superfulous when we update
tick_gtodbefore the__update_sched_clock()call insched_clock_tick()-
cpu_clock()might be implemented as:sched_clock_cpu(smp_processor_id())
if the accuracy proves good enough - how far can TSC drift in a single jiffie when considering the filtering and idle hooks?
Quoted 8 lines
Quoted 5 lines
[ [email protected]: various fixes and cleanups ]
Signed-off-by Peter Zijlstra <[email protected]>Signed-off-by Ingo Molnar <[email protected]>Ok, so the issue is that the clock doesn't start at 0, but at:
4294014.506571 ?
At least on my box , yes.
There is a similar report http://lkml.org/lkml/2008/5/8/100 , there it jumps to 42949372.961619
Quoted 3 lines
I guess that's a side-effect of using ktime to base the clock off. Thomas, do we have a clock that is uptime based?
Gabriel
Quoted 5 lines · Gabriel C
Quoted 2 lines
Ok, so the issue is that the clock doesn't start at 0, but at:
4294014.506571 ?At least on my box , yes.
Does this work for you?
---
Subject: sched: fixup sched-clock offset
From: Peter Zijlstra <[email protected]>
It could happen that ktime doesn't start at 0.
--- kernel/sched_clock.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-)
kernel/sched_clock.c +9 -7
@@ -59,21 +59,23 @@ static inline struct sched_clock_data *creturn &per_cpu (sched_clock_data, cpu); } +static __read_mostly u64 ktime_offset; +void sched_clock_init (void ) { -u64 ktime_now =ktime_to_ns (ktime_get ()); -u64 now =0 ;int cpu; + ktime_offset =ktime_to_ns (ktime_get ()); +for_each_possible_cpu (cpu) {struct sched_clock_data *scd =cpu_sdc (cpu); scd->lock = (raw_spinlock_t )__RAW_SPIN_LOCK_UNLOCKED; scd->prev_jiffies = jiffies; - scd->prev_raw = now; - scd->tick_raw = now; - scd->tick_gtod = ktime_now; - scd->clock = ktime_now; + scd->prev_raw =0 ; + scd->tick_raw =0 ; + scd->tick_gtod =0 ; + scd->clock =0 ; } }
@@ -177,7 +179,7 @@ void sched_clock_tick(void)WARN_ON_ONCE (!irqs_disabled ()); now =sched_clock (); - now_gtod =ktime_to_ns (ktime_get ()); + now_gtod =ktime_to_ns (ktime_get ()) - ktime_offset;__raw_spin_lock (&scd->lock);__update_sched_clock (scd, now);
Ingo Molnar
AppliedQuoted 4 lines · Peter Zijlstra
Subject: sched: fixup sched-clock offset
From: Peter Zijlstra <[email protected]>It could happen that ktime doesn't start at 0.
applied, thanks Peter. I'm wondering which clocks do this - HPET perhaps?
Ingo
Quoted 8 lines · Peter Zijlstra
Quoted 4 lines · Gabriel C
Quoted 2 lines
Ok, so the issue is that the clock doesn't start at 0, but at:
4294014.506571 ?At least on my box , yes.
Does this work for you?
No it does not , I still get :
....
[ 0.000000] PERCPU: Allocating 42756 bytes of per cpu data [ 0.000000] NR_CPUS: 8, nr_cpu_ids: 4 [4294014.506571] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 1010723
...
Quoted 59 lines
---
Subject: sched: fixup sched-clock offset
From: Peter Zijlstra <[email protected]>It could happen that ktime doesn't start at 0.
Signed-off-by Peter Zijlstra <[email protected]>--- kernel/sched_clock.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-)Index:
linux-2.6-2/kernel/sched_clock.c
===================================================================kernel/sched_clock.c +9
-7@@ -59,21 +59,23 @@ static inline struct sched_clock_data *creturn &per_cpu (sched_clock_data, cpu); } +static __read_mostly u64 ktime_offset; +void sched_clock_init (void ) { -u64 ktime_now =ktime_to_ns (ktime_get ()); -u64 now =0 ;int cpu; + ktime_offset =ktime_to_ns (ktime_get ()); +for_each_possible_cpu (cpu) {struct sched_clock_data *scd =cpu_sdc (cpu); scd->lock = (raw_spinlock_t )__RAW_SPIN_LOCK_UNLOCKED; scd->prev_jiffies = jiffies; - scd->prev_raw = now; - scd->tick_raw = now; - scd->tick_gtod = ktime_now; - scd->clock = ktime_now; + scd->prev_raw =0 ; + scd->tick_raw =0 ; + scd->tick_gtod =0 ; + scd->clock =0 ; } }@@ -177,7 +179,7 @@ void sched_clock_tick(void)WARN_ON_ONCE (!irqs_disabled ()); now =sched_clock (); - now_gtod =ktime_to_ns (ktime_get ()); + now_gtod =ktime_to_ns (ktime_get ()) - ktime_offset;__raw_spin_lock (&scd->lock);__update_sched_clock (scd, now);
Quoted 9 lines · Ingo Molnar
Quoted 4 lines · Peter Zijlstra
Subject: sched: fixup sched-clock offset
From: Peter Zijlstra <[email protected]>It could happen that ktime doesn't start at 0.
applied, thanks Peter. I'm wondering which clocks do this - HPET perhaps?
The box is using TSC clocksource right now , for some reason in >=2.6.25* kernels HPET registered to late.
http://lkml.org/lkml/2008/5/8/164
Quoted 3 lines
Ingo
Gabriel