mmotm 2008-12-01-19-41: early exception (page fault -- deref of 0x20)
Hi,
while writing out warning about empty mtrr (or any other in the early stage), I'm getting this:
PANIC: early exception 0e rip 10:ffffffff8025be26 error 0 cr2 20
Pid: 0, comm: swapper Not tainted 2.6.28-rc6-mm1_64 #484
Call Trace: [<ffffffff80770195>] early_idt_handler+0x55/0x69 [<ffffffff8025be26>] ? getnstimeofday+0x46/0xc0 [<ffffffff80258a81>] ktime_get_real+0x11/0x50 [<ffffffff803aa8d1>] seed_std_data+0x11/0x30 [<ffffffff8020ff60>] ? show_trace+0x10/0x20 [<ffffffff803aa900>] seed_random_pools+0x10/0x30 [<ffffffff8023dda9>] init_oops_id+0x9/0x40 [<ffffffff8023dde9>] print_oops_end_marker+0x9/0x20 [<ffffffff8023dfed>] warn_slowpath+0x9d/0xd0 [<ffffffff802595e4>] ? up+0x34/0x50 [<ffffffff8023e77d>] ? release_console_sem+0x1bd/0x210 [<ffffffff80592c38>] ? printk+0x3c/0x44 [<ffffffff80776e67>] mtrr_trim_uncached_memory+0x166/0x390 [<ffffffff8077fca0>] ? early_gart_iommu_check+0xaf/0x2a7 [<ffffffff80772574>] setup_arch+0x3ec/0x6a8 [<ffffffff80770adf>] start_kernel+0x65/0x3c6 [<ffffffff8077027d>] x86_64_start_reservations+0x7d/0x89 [<ffffffff80770384>] x86_64_start_kernel+0xd8/0xdf
RIP 0x10
The clock is still null in kernel/time/timekeeping.c.
This one is to blame:
random-add-a-way-to-get-some-random-bits-into-the-entropy-pools-early-on.patch
Quoted 25 lines · On Wed, 03 Dec 2008 00:09:46 +0100 Jiri Slaby <[email protected]> wrote:
Hi,
while writing out warning about empty mtrr (or any other in the early stage), I'm getting this:
PANIC: early exception 0e rip 10:ffffffff8025be26 error 0 cr2 20
Pid: 0, comm: swapper Not tainted 2.6.28-rc6-mm1_64 #484Call Trace: [<ffffffff80770195>] early_idt_handler+0x55/0x69 [<ffffffff8025be26>] ? getnstimeofday+0x46/0xc0 [<ffffffff80258a81>] ktime_get_real+0x11/0x50 [<ffffffff803aa8d1>] seed_std_data+0x11/0x30 [<ffffffff8020ff60>] ? show_trace+0x10/0x20 [<ffffffff803aa900>] seed_random_pools+0x10/0x30 [<ffffffff8023dda9>] init_oops_id+0x9/0x40 [<ffffffff8023dde9>] print_oops_end_marker+0x9/0x20 [<ffffffff8023dfed>] warn_slowpath+0x9d/0xd0 [<ffffffff802595e4>] ? up+0x34/0x50 [<ffffffff8023e77d>] ? release_console_sem+0x1bd/0x210 [<ffffffff80592c38>] ? printk+0x3c/0x44 [<ffffffff80776e67>] mtrr_trim_uncached_memory+0x166/0x390 [<ffffffff8077fca0>] ? early_gart_iommu_check+0xaf/0x2a7 [<ffffffff80772574>] setup_arch+0x3ec/0x6a8 [<ffffffff80770adf>] start_kernel+0x65/0x3c6 [<ffffffff8077027d>] x86_64_start_reservations+0x7d/0x89 [<ffffffff80770384>] x86_64_start_kernel+0xd8/0xdf
Quoted 6 lines
RIP
0x10The clock is still null in
kernel/time/timekeeping.c.This one is to blame:
random-add-a-way-to-get-some-random-bits-into-the-entropy-pools-early-on.patch
urgh, OK, thanks, I'll drop it.
Presumably your machine picked a different clocksource from Arjan's, mine and others. Now, how do we work out what clocksource you're using?
dmesg -s 1000000 | grep -i clock
nope
dmesg -s 1000000 | grep -i using
nope
hrm.
Quoted 37 lines · Andrew Morton
Quoted 25 lines · On Wed, 03 Dec 2008 00:09:46 +0100 Jiri Slaby <[email protected]> wrote:
Hi,
while writing out warning about empty mtrr (or any other in the early stage), I'm getting this:
PANIC: early exception 0e rip 10:ffffffff8025be26 error 0 cr2 20
Pid: 0, comm: swapper Not tainted 2.6.28-rc6-mm1_64 #484Call Trace: [<ffffffff80770195>] early_idt_handler+0x55/0x69 [<ffffffff8025be26>] ? getnstimeofday+0x46/0xc0 [<ffffffff80258a81>] ktime_get_real+0x11/0x50 [<ffffffff803aa8d1>] seed_std_data+0x11/0x30 [<ffffffff8020ff60>] ? show_trace+0x10/0x20 [<ffffffff803aa900>] seed_random_pools+0x10/0x30 [<ffffffff8023dda9>] init_oops_id+0x9/0x40 [<ffffffff8023dde9>] print_oops_end_marker+0x9/0x20 [<ffffffff8023dfed>] warn_slowpath+0x9d/0xd0 [<ffffffff802595e4>] ? up+0x34/0x50 [<ffffffff8023e77d>] ? release_console_sem+0x1bd/0x210 [<ffffffff80592c38>] ? printk+0x3c/0x44 [<ffffffff80776e67>] mtrr_trim_uncached_memory+0x166/0x390 [<ffffffff8077fca0>] ? early_gart_iommu_check+0xaf/0x2a7 [<ffffffff80772574>] setup_arch+0x3ec/0x6a8 [<ffffffff80770adf>] start_kernel+0x65/0x3c6 [<ffffffff8077027d>] x86_64_start_reservations+0x7d/0x89 [<ffffffff80770384>] x86_64_start_kernel+0xd8/0xdfQuoted 6 lines
RIP
0x10The clock is still null in
kernel/time/timekeeping.c.This one is to blame:
random-add-a-way-to-get-some-random-bits-into-the-entropy-pools-early-on.patchurgh, OK, thanks, I'll drop it.
Quoted 8 lines
Presumably your machine picked a different clocksource from Arjan's, mine and others. Now, how do we work out what clocksource you're using?
dmesg -s 1000000 | grep -i clock
nope
dmesg -s 1000000 | grep -i using
nope
If we oops or warn while picking a timesource, we'll have lots of fun?
Signature · 1 line
Mathematics is the supreme nostalgia of our time.
Quoted 3 lines · Matt Mackall
If we oops or warn while picking a timesource, we'll have lots of fun?
we really only need to mix in the tsc; ktime_get() is just an arch friendly way to get that I supposed (wrongly).
but yes we need to do a few things
1) seed on demand with a platform time source
2) have a way where arch init can just hand semi random data during the boot process to increase the randomness (even if it doesn't count as entropy)
I got pulled in some big project at work so I might not get to it this week, but if nobody beats me to it I will get to it ;-(
Quoted 10 lines · Arjan van de Ven
Quoted 3 lines · Matt Mackall
If we oops or warn while picking a timesource, we'll have lots of fun?
we really only need to mix in the tsc;
ktime_get()is just an arch friendly way to get that I supposed (wrongly).but yes we need to do a few things
1) seed on demand with a platform time source
Currently we use jiffies + get_cycles(). That's going to have somewhere between, oh, 3 bits of entropy (very stable boot with only jiffies) and 25 bits of entropy (TSC with lots of waiting for hardware) at boot. Ideally, we'd have access to a wall clock of some sort as well. But that's also a fairly limited source - wall clocks are both low resolution and predictable/collision-prone.
Quoted 2 lines
2) have a way where arch init can just hand semi random data during the boot process to increase the randomness (even if it doesn't count as entropy)
A simple wrapper around mix_pool_bytes probably fits the ticket.
But I don't think this will solve the general problem of 'large numbers of practically identical machines booting up with the same pre-init random number pools'. Beyond things like MAC addresses and serial numbers (predictable/observable but at least not collision-prone), we have no way to differentiate some boxes. We may need to forcibly generate some timing entropy. Perhaps something like this:
http://markmail.org/message/xwsbywr6ziil2qu2
(which is way too slow in its current form)
There's a related problem of systems with no way to store a seed across boots.
Signature · 1 line
Mathematics is the supreme nostalgia of our time.
Quoted 7 lines · Matt Mackall
Quoted 6 lines · Arjan van de Ven
Quoted 2 lines · Matt Mackall
If we oops or warn while picking a timesource, we'll have lots of fun?
we really only need to mix in the tsc;
ktime_get()is just an arch friendly way to get that I supposed (wrongly).
Quoted 8 lines
Quoted 2 lines
but yes we need to do a few things
1) seed on demand with a platform time sourceCurrently we use jiffies +
get_cycles(). That's going to have somewhere
between, oh, 3 bits of entropy (very stable boot with only jiffies) and
25 bits of entropy (TSC with lots of waiting for hardware) at boot. Ideally, we'd have access to a wall clock of some sort as well. But that's also a fairly limited source - wall clocks are both low
resolution and predictable/collision-prone.
tsc taken quite some time appart in the boot, we will get multiple times a bunch of entropy. This comes from variations in time various hardware things take (due to spread spectrum clocks) and because of cpu speculation not being deterministic to this level of multi-milliseconds.
Quoted 7 lines · Andrew Morton
Presumably your machine picked a different clocksource from Arjan's, mine and others. Now, how do we work out what clocksource you're using? dmesg -s 1000000 | grep -i clock
nope
dmesg -s 1000000 | grep -i using
nope
hrm.
cat /sys/devices/system/clocksource/clocksource0/current_clocksource
Signature · 2 lines
Jiri Kosina SUSE Labs
Quoted 11 lines · Jiri Kosina
Quoted 7 lines · Andrew Morton
Presumably your machine picked a different clocksource from Arjan's, mine and others. Now, how do we work out what clocksource you're using? dmesg -s 1000000 | grep -i clock
nope
dmesg -s 1000000 | grep -i using
nope
hrm.cat
/sys/devices/system/clocksource/clocksource0/current_clocksource
tsc
It ran from qemu, but it oopsed even on my desktop (tsc too).
Quoted 10 lines · Jiri Slaby
Quoted 8 lines
Quoted 7 lines
Presumably your machine picked a different clocksource from Arjan's, mine and others. Now, how do we work out what clocksource you're using? dmesg -s 1000000 | grep -i clock
nope
dmesg -s 1000000 | grep -i using
nope
hrm.cat
/sys/devices/system/clocksource/clocksource0/current_clocksourcetsc
It ran from qemu, but it oopsed even on my desktop (tsc too).
Could you please send the full oops to Thomas and Ingo?
Signature · 2 lines
Jiri Kosina SUSE Labs
Quoted 25 lines · Jiri Slaby
Hi,
while writing out warning about empty mtrr (or any other in the early stage), I'm getting this:
PANIC: early exception 0e rip 10:ffffffff8025be26 error 0 cr2 20
Pid: 0, comm: swapper Not tainted 2.6.28-rc6-mm1_64 #484Call Trace: [<ffffffff80770195>] early_idt_handler+0x55/0x69 [<ffffffff8025be26>] ? getnstimeofday+0x46/0xc0 [<ffffffff80258a81>] ktime_get_real+0x11/0x50 [<ffffffff803aa8d1>] seed_std_data+0x11/0x30 [<ffffffff8020ff60>] ? show_trace+0x10/0x20 [<ffffffff803aa900>] seed_random_pools+0x10/0x30 [<ffffffff8023dda9>] init_oops_id+0x9/0x40 [<ffffffff8023dde9>] print_oops_end_marker+0x9/0x20 [<ffffffff8023dfed>] warn_slowpath+0x9d/0xd0 [<ffffffff802595e4>] ? up+0x34/0x50 [<ffffffff8023e77d>] ? release_console_sem+0x1bd/0x210 [<ffffffff80592c38>] ? printk+0x3c/0x44 [<ffffffff80776e67>] mtrr_trim_uncached_memory+0x166/0x390 [<ffffffff8077fca0>] ? early_gart_iommu_check+0xaf/0x2a7 [<ffffffff80772574>] setup_arch+0x3ec/0x6a8 [<ffffffff80770adf>] start_kernel+0x65/0x3c6 [<ffffffff8077027d>] x86_64_start_reservations+0x7d/0x89 [<ffffffff80770384>] x86_64_start_kernel+0xd8/0xdf
Quoted 6 lines
RIP
0x10The clock is still null in
kernel/time/timekeeping.c.This one is to blame:
random-add-a-way-to-get-some-random-bits-into-the-entropy-pools-early-on.patch
Nice. I didn't like that patch anyway.
Seems to me we really need to simplify all our startup dependencies. Some things should -always- work. Time is one of them, random number generation may be another, memory allocation ought to be one as well.
So I think:
a) we need to come up with a solution/rule for early ktime faulting
b) the RNG should initialize itself on demand
c) seeding the RNG should include a (void *, len)
Signature · 1 line
Mathematics is the supreme nostalgia of our time.