kernel 2.6.16-1.2097_FC6 unbootable on Itanium
Quoted 11 lines · Prarit Bhargava
Quoted 6 lines · Émeric Maschino
Hi,
There's something wrong with kernel 2.6.16-1.2097_FC6. The last displayed message is "Freeing unused kernel memory: 400kB freed" and then nothing. kernel-2.6.16-1.2088_FC6 runs fine.
I saw an error spit out while the RPM install was assembling the initrd.
I'll look into it.
Looks like upstream 2.6.16-1 doesn't boot either.
Adding linux-ia64 to the list ...
P.
Quoted 3 lines
P.
Quoted 4 lines · Prarit Bhargava
Quoted 2 lines · Émeric Maschino
Hi,
Quoted 8 lines
Quoted 4 lines
There's something wrong with kernel 2.6.16-1.2097_FC6. The last displayed message is "Freeing unused kernel memory: 400kB freed" and then nothing. kernel-2.6.16-1.2088_FC6 runs fine.
I saw an error spit out while the RPM install was assembling the initrd.
I'll look into it.
Looks like upstream 2.6.16-1 doesn't boot either.
Adding linux-ia64 to the list ...
P.
Quoted 2 lines
P.
Prarit Bhargava wrote on Wednesday, March 29, 2006 8:34 AM
Quoted 3 lines
Looks like upstream 2.6.16-1 doesn't boot either.
Adding linux-ia64 to the list ...
Just tried upstream 2.6.16.1 on my Intel tiger4 machine. It boots OK. What kind of problem do you have? Where does it hang?
- Ken
Quoted 10 lines · Kenneth W
Prarit Bhargava wrote on Wednesday, March 29, 2006 8:34 AM
Quoted 3 lines
Looks like upstream 2.6.16-1 doesn't boot either.
Adding linux-ia64 to the list ...
Just tried upstream 2.6.16.1 on my Intel tiger4 machine. It boots OK. What kind of problem do you have? Where does it hang?
I'm trying to get a hold of Emeric to find out what system he's using :)
But it looks like it is hanging on Altix's at
Calling initcall 0xa0000001006ad230: bictcp_register+0x0/0x40() TCP bic registered
Calling initcall 0xa0000001006b1160: xfrm_user_init+0x0/0xe0() Initializing IPsec netlink socket
Calling initcall 0xa0000001006ade70: af_unix_init+0x0/0x120()
NET: Registered protocol family 1
Calling initcall 0xa0000001006adfc0: packet_init+0x0/0x100()
NET: Registered protocol family 17
Calling initcall 0xa0000001006a9670: acpi_poweroff_init+0x0/0x100() Calling initcall 0xa0000001006af2d0: seqgen_init+0x0/0x40() Calling initcall 0xa0000001006b18d0: early_uart_console_switch+0x0/0x120() Calling initcall 0xa0000001006a8e10: net_random_reseed+0x0/0x100() Freeing unused kernel memory: 400kB freed
P.
Quoted 6 lines
- Ken
-
To unsubscribe from this list: send the line "unsubscribe linux-ia64" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Mar 29, 2006 at 01:55:26PM -0500, Prarit Bhargava wrote:
| But it looks like it is hanging on Altix's at | | Calling initcall 0xa0000001006ad230: bictcp_register+0x0/0x40() | TCP bic registered | Calling initcall 0xa0000001006b1160: xfrm_user_init+0x0/0xe0() | Initializing IPsec netlink socket | Calling initcall 0xa0000001006ade70: af_unix_init+0x0/0x120() | NET: Registered protocol family 1 | Calling initcall 0xa0000001006adfc0: packet_init+0x0/0x100() | NET: Registered protocol family 17 | Calling initcall 0xa0000001006a9670: acpi_poweroff_init+0x0/0x100() | Calling initcall 0xa0000001006af2d0: seqgen_init+0x0/0x40() | Calling initcall 0xa0000001006b18d0: early_uart_console_switch+0x0/0x120() | Calling initcall 0xa0000001006a8e10: net_random_reseed+0x0/0x100() | Freeing unused kernel memory: 400kB freed
Is it hung, or did it MCA? Check the leds (ctrl-t to L2 from console, then 'leds'). If you repeat this procedure, you should see the LED values changing.
Greg
Prarit Bhargava wrote on Wednesday, March 29, 2006 10:55 AM
Quoted 8 lines
Quoted 3 lines
Just tried upstream 2.6.16.1 on my Intel tiger4 machine. It boots OK. What kind of problem do you have? Where does it hang?
I'm trying to get a hold of Emeric to find out what system he's using :)
But it looks like it is hanging on Altix's at
Quoted 13 lines
Calling initcall
0xa0000001006ad230: bictcp_register+0x0/0x40() TCP bic registered
Calling initcall0xa0000001006b1160: xfrm_user_init+0x0/0xe0() Initializing IPsec netlink socket
Calling initcall0xa0000001006ade70: af_unix_init+0x0/0x120()
NET: Registered protocol family 1
Calling initcall0xa0000001006adfc0: packet_init+0x0/0x100()
NET: Registered protocol family 17
Calling initcall0xa0000001006a9670: acpi_poweroff_init+0x0/0x100() Calling initcall0xa0000001006af2d0: seqgen_init+0x0/0x40() Calling initcall0xa0000001006b18d0: early_uart_console_switch+0x0/0x120() Calling initcall0xa0000001006a8e10: net_random_reseed+0x0/0x100() Freeing unused kernel memory: 400kB freed
I also tried yesterday's snapshot 2.6.16-git17 on my tiger4, it doesn't boot :-(
So far we have reports on:
(1) 2.6.16.1 doesn't boot on altix
(2) 2.6.16-git17 doesn't boot on tiger4
(3) Jes's strange .section causing some hangs.
Hmm, I don't know what's going on ....
- Ken
Quoted 8 lines
I also tried yesterday's snapshot 2.6.16-git17 on my tiger4, it doesn't boot :-(
So far we have reports on:
(1) 2.6.16.1 doesn't boot on altix
(2) 2.6.16-git17 doesn't boot on tiger4
(3) Jes's strange .section causing some hangs.
Tiger still boots with my release tree (just has seven patches that haven't been sent to Linus yet, including Russ' fix for my merge goof that created item "3" above). I haven't pulled in from Linus in about three days, so I'm about 458 commits behind the bleeding edge. So a "git bisect" between Linus head and the point where I last merged (commit 64bc0430) might yield the problem (for Ken's tiger boot issue).
-Tony
Here's what's in release/test that hasn't gone to Linus yet:
arch/ia64/kernel/iosapic.c | 279 +++++++++++++++++++++--------------- arch/ia64/kernel/vmlinux.lds.S | 18 +- arch/ia64/mm/init.c | 8 - arch/ia64/mm/tlb.c | 12 - arch/ia64/sn/kernel/sn2/sn_hwperf.c | 8 - 5 files changed, 198 insertions(+), 127 deletions(-)
Chen, Kenneth W:
[IA64] optimize flush_tlb_range on large numa box
Dean Roe:
[IA64-SGI] fix for-loop in sn_hwperf_geoid_to_cnode()
[email protected]:
[IA64-SGI] sn_hwperf use of num_online_cpus()
Russ Anderson:
[IA64] Move __mca_table out of the __init section
Satoru Takeuchi:
[IA64] correct some messages and fixes some minor things [IA64] simplify some condition checks in iosapic_check_gsi_range
Zhang, Yanmin:
[IA64] lazy_mmu_prot_update needs to be aware of huge pages
Quoted 8 lines · Kenneth W
I also tried yesterday's snapshot 2.6.16-git17 on my tiger4, it doesn't boot :-(
So far we have reports on:
(1) 2.6.16.1 doesn't boot on altix
(2) 2.6.16-git17 doesn't boot on tiger4
(3) Jes's strange .section causing some hangs.
(4) James Bottomley couldn't boot his ZX2000 (HP ZX1 chipset) from git yesterday either.
He had to update firmware to 2.31 version because of some changes in 2.6.16 SAL usage exposed an older SAL bug. Alex Williamson had dug up the details and I'm blanking on them right now.
But his zx2000 is still not able to boot.
Could be multiple, unrelated problems being reported now.
grant
Re: [Fedora-ia64-list] kernel 2.6.16-1.2097_FC6 unbootable on
In reply to Prarit BhargavaQuoted 8 lines · Grant Grundler
(4) James Bottomley couldn't boot his ZX2000 (HP ZX1 chipset) from git yesterday either.
He had to update firmware to 2.31 version because of some changes in 2.6.16 SAL usage exposed an older SAL bug. Alex Williamson had dug up the details and I'm blanking on them right now.
But his zx2000 is still not able to boot.Could be multiple, unrelated problems being reported now.
No, I can confirm that reverting
[IA64] MCA Recovery: kernel context recovery table
fixes my boot problems.
I actually got slightly further than Jes with the analysis. The problem occurs because something in this changeset makes code that dynamically loads the TLS libraries segfault. If you look, Jes, you'll find init is continually segfaulting and respawning.
James
Quoted 7 lines
Tiger still boots with my release tree (just has seven patches that haven't been sent to Linus yet, including Russ' fix for my merge goof that created item "3" above). I haven't pulled in from Linus in about three days, so I'm about 458 commits behind the bleeding edge. So a "git bisect" between Linus head and the point where I last merged (commit
64bc0430) might yield the problem (for Ken's tiger boot issue).
Just started the bisection ... and the current tip of Linus' git tree (f3cab8a0) booted just fine on tiger! There are 85 commits in there since git-17 was cut, but a quick glance at gitk doesn't show any obvious fixes or reverts that might affect ia64.
-Tony
This is my turn to fill in the report: yesterday's Rawhide kernel 2.6.16-1.2097_FC6 (based on 2.6.16-git9, git10 and git 13) and today's Rawhide kernel 2.6.16-1.2102_FC6 (based on 2.6.16-git14, 2.6.16-git15) don't boot on hp workstation zx6000.
Émeric
Quoted 9 lines
So far we have reports on:
(1) 2.6.16.1 doesn't boot on altix
(2) 2.6.16-git17 doesn't boot on tiger4
(3) Jes's strange .section causing some hangs.Hmm, I don't know what's going on ....
- Ken
Quoted 8 lines · Tony
Quoted 7 lines
Tiger still boots with my release tree (just has seven patches that haven't been sent to Linus yet, including Russ' fix for my merge goof that created item "3" above). I haven't pulled in from Linus in about three days, so I'm about 458 commits behind the bleeding edge. So a "git bisect" between Linus head and the point where I last merged (commit
64bc0430) might yield the problem (for Ken's tiger boot issue).
Quoted 5 lines
Just started the bisection ... and the current tip of Linus' git tree (
f3cab8a0) booted just fine on tiger! There are 85 commits in there since git-17 was cut, but a quick glance at gitk doesn't show any obvious fixes or reverts that might affect ia64.
I don't know if this helps or not.
I ran Jes's kernel on the simulator. Unfortunately, he sent me a stripped kernel so I have no symbol table.
The kernel blows up right after printing:
... Virtual mem_map starts at 0xa0007fffd5f2c000 Built 1 zonelists Kernel command line: root=/dev/hda2 init=/bin/bash console=ttyS0 PID hash table entries: 1024 (order: 10, 32768 bytes) Console: colour dummy device 80x25
The failure is an MCA caused by a cache hit on a memory reference to an uncached address. The simulator detects this error & stops.
The code that took the failure was memcopy (or equiv). I recognized the code from the prefetchs and ld/st sequence. The data appears to be an ACPI table that is being copied into kernel memory. The current reference is using uncached addresses but 15M instructions in the past, the table was referenced cached.
Does this ring any bells with anyone? This failure may occur only in our simulator environment, so don't spent any time on this unless it sounds familar.
As soon as I get a symbol table, I'll know a lot more about the failure.
---
Jack
Quoted 9 lines
Quoted 9 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Tony" = Luck, Tony <[email protected]> writes:
I also tried yesterday's snapshot 2.6.16-git17 on my tiger4, it doesn't boot :-(
So far we have reports on:
(1) 2.6.16.1 doesn't boot on altix (2) 2.6.16-git17 doesn't boot on tiger4 (3) Jes's strange .section causing some hangs.
Tony> Tiger still boots with my release tree (just has seven patches Tony> that haven't been sent to Linus yet, including Russ' fix for my Tony> merge goof that created item "3" above). I haven't pulled in Tony> from Linus in about three days, so I'm about 458 commits behind Tony> the bleeding edge. So a "git bisect" between Linus head and the Tony> point where I last merged (commit 64bc0430) might yield the Tony> problem (for Ken's tiger boot issue).
Tony,
Please note that if I compile the kernel on a box with an old installation and old toolchain the kernel boots, same source tree built on a system with a new installation and it doesn't boot.
Cheers,
Jes
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Jack" = Jack Steiner <[email protected]> writes:
Jack> ... Virtual mem_map starts at 0xa0007fffd5f2c000 Built 1
Jack> zonelists Kernel command line: root=/dev/hda2 init=/bin/bash Jack> console=ttyS0 PID hash table entries: 1024 (order: 10, 32768 Jack> bytes) Console: colour dummy device 80x25
Jack> The failure is an MCA caused by a cache hit on a memory Jack> reference to an uncached address. The simulator detects this Jack> error & stops.
Hi Jack,
Sorry about the missing symbol table, was a bit late.
I am not sure it's the same thing, as my kernel gets a lot further and normally spawns init before exploding.
Jack> The code that took the failure was memcopy (or equiv). I Jack> recognized the code from the prefetchs and ld/st sequence. The Jack> data appears to be an ACPI table that is being copied into Jack> kernel memory. The current reference is using uncached addresses Jack> but 15M instructions in the past, the table was referenced Jack> cached.
Jack> Does this ring any bells with anyone? This failure may occur Jack> only in our simulator environment, so don't spent any time on Jack> this unless it sounds familar.
Jack> As soon as I get a symbol table, I'll know a lot more about the Jack> failure.
I put one up on attica for you ~jes/System.map-jack
It seems the kernel explodes in __copy_user in your case.
Cheers,
Jes
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Prarit" = Prarit Bhargava <[email protected]> writes:
Prarit> Looks like upstream 2.6.16-1 doesn't boot either.
Prarit> Adding linux-ia64 to the list ...
Please don't cross post to linux-ia64 when you have a fscked up list on the CC, the fedora list is one of those hopeless subscribers only things ;-(
Jes
Quoted 18 lines · Jes Sorensen (on 30 Mar 2006 03:57:04 -0500) wrote:
Quoted 9 lines
Quoted 9 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Tony" = Luck, Tony <[email protected]> writes:
I also tried yesterday's snapshot 2.6.16-git17 on my tiger4, it doesn't boot :-(
So far we have reports on:
(1) 2.6.16.1 doesn't boot on altix (2) 2.6.16-git17 doesn't boot on tiger4 (3) Jes's strange .section causing some hangs.
Tony> Tiger still boots with my release tree (just has seven patches Tony> that haven't been sent to Linus yet, including Russ' fix for my Tony> merge goof that created item "3" above). I haven't pulled in Tony> from Linus in about three days, so I'm about 458 commits behind Tony> the bleeding edge. So a "git bisect" between Linus head and the Tony> point where I last merged (commit
64bc0430) might yield the Tony> problem (for Ken's tiger boot issue).
Quoted 5 lines
Tony,
Please note that if I compile the kernel on a box with an old installation and old toolchain the kernel boots, same source tree built on a system with a new installation and it doesn't boot.
This sounds like a random data overwrite which depends on the exact kernel layout. Somebody else in SGI (Prarit ?) said that they compiled your failing tree on the same platform and their kernel worked. Since the build userid and the pathname to the kernel source tree are embedded in the binary, changing either of those will disturb the kernel layout, possibly making the problem move.
Jes, try renaming your source tree to something with a different pathname length and rebuild the kernel. Does the problem change or go away?
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Keith" = Keith Owens <[email protected]> writes:
Quoted 3 lines · Keith> Jes Sorensen (on 30 Mar 2006 03:57:04 -0500) wrote:
Quoted 3 lines
Please note that if I compile the kernel on a box with an old installation and old toolchain the kernel boots, same source tree built on a system with a new installation and it doesn't boot.
Keith> This sounds like a random data overwrite which depends on the Keith> exact kernel layout. Somebody else in SGI (Prarit ?) said that Keith> they compiled your failing tree on the same platform and their Keith> kernel worked. Since the build userid and the pathname to the Keith> kernel source tree are embedded in the binary, changing either Keith> of those will disturb the kernel layout, possibly making the Keith> problem move.
Keith> Jes, try renaming your source tree to something with a Keith> different pathname length and rebuild the kernel. Does the Keith> problem change or go away?
Moved it to /tmp/a and built there - no go.
Cheers,
Jes
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"James" = James Bottomley <[email protected]> writes:
James> No, I can confirm that reverting
James> [IA64] MCA Recovery: kernel context recovery table
James> fixes my boot problems.
James> I actually got slightly further than Jes with the analysis. James> The problem occurs because something in this changeset makes James> code that dynamically loads the TLS libraries segfault. If you James> look, Jes, you'll find init is continually segfaulting and James> respawning.
Yay!
I am not the only one seeing this! I am not insane!
I bet that if you revert the patch above but then add these two lines to arch/ia64/kernel/gate.S you'll see the same failure:
.section "__mca_table", "a" .previous
Tracked it down further. If I take the gate.o file from a working kernel and copy it into the broken tree and run make again, the resulting kernel boots.
I compared the code between a working and a broken gate.o with objdump and it's identical. However the good file is 3552 bytes whereas the broken one is 3648.
objdump -h reveals one extra empty section which seems to be what makes all the difference. The below output is from a broken gate.o, the only difference I can find comparing it to the good one is that idx 4 is missing in the good one and the following ones are renamed. Do we have some strange restrictions in the linker on the number of sections it can handle?
Cheers,
Jes
gate.o: file format elf64-ia64-little
Sections:
Idx Name Size VMA LMA File off Algn 0 .text 000003b0 0000000000000000 0000000000000000 00000040 2**5
CONTENTS, ALLOC, LOAD, READONLY, CODE
1 .data 00000000 0000000000000000 0000000000000000 000003f0 2**0
CONTENTS, ALLOC, LOAD, DATA
2 .bss 00000000 0000000000000000 0000000000000000 000003f0 2**0
ALLOC
3 __ex_table 00000000 0000000000000000 0000000000000000 000003f0 2**0
CONTENTS, ALLOC, LOAD, READONLY, DATA
4 __mca_table 00000000 0000000000000000 0000000000000000 000003f0 2**0
CONTENTS, ALLOC, LOAD, READONLY, DATA
5 .data.patch.vtop 00000000 0000000000000000 0000000000000000 000003f0 2**0
CONTENTS, ALLOC, LOAD, DATA
6 .data.patch.mckinley_e9 00000004 0000000000000000 0000000000000000 000003f0 2**2
CONTENTS, ALLOC, LOAD, RELOC, DATA
7 .data.patch.fsyscall_table 00000004 0000000000000000 0000000000000000 000003f4 2**2
CONTENTS, ALLOC, LOAD, RELOC, DATA
8 .data.patch.brl_fsys_bubble_down 00000004 0000000000000000 0000000000000000 000003f8 2**2
CONTENTS, ALLOC, LOAD, RELOC, DATA
9 .IA_64.unwind_info 000000b8 0000000000000000 0000000000000000 00000400 2**3
CONTENTS, ALLOC, LOAD, READONLY, DATA
10 .IA_64.unwind 00000048 0000000000000000 0000000000000000 000004b8 2**3
CONTENTS, ALLOC, LOAD, RELOC, READONLY, DATA
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Jes" = Jes Sorensen <[email protected]> writes:
Jes> Tracked it down further. If I take the gate.o file from a working Jes> kernel and copy it into the broken tree and run make again, the Jes> resulting kernel boots.
Jes> I compared the code between a working and a broken gate.o with Jes> objdump and it's identical. However the good file is 3552 bytes Jes> whereas the broken one is 3648.
Looks like something goes wrong with some linking references due to this. I ran objdump on the two kernels I get when replacing gate.o and the diff is as attached below. Not sure if this is an ia64 specific bug or we're just 'lucky'.
Cheers,
Jes
broken/vmlinux.s renamed from good/vmlinux.s +6 -6
@@ -4981,8 +4981,8 @@ a00000010000bd96: 30 01 44 44 08 40 mov.m r19=ar.bsp a00000010000bd9c: 01 00 40 00 mov f10ð a00000010000bda0: 05 00 00 00 01 00 [MLX] nop.m 0x0 -a00000010000bda6: 00 00 00 00 20 c0 movl r14=0xa000000000010640;; -a00000010000bdac: 01 24 30 68 +a00000010000bda6: 00 00 00 00 00 c0 movl r14=0x10080;; +a00000010000bdac: 01 20 04 60 a00000010000bdb0: 08 00 00 32 2a 04 [MMI] mov.m ar25=r0 a00000010000bdb6: 00 00 80 54 08 e0 mov.m ar.ccv=r0 a00000010000bdbc: e0 08 00 07 mov b7=r14
@@ -8192,8 +8192,8 @@ a00000010000fe06: 00 01 00 00 20 c0 movl r14=0xa00000010000bc60 a00000010000fe0c: 01 06 e0 6d a00000010000fe10: 05 00 00 00 01 00 [MLX] nop.m 0x0 -a00000010000fe16: 00 00 00 00 20 80 movl r28=0xa000000000010620;; -a00000010000fe1c: 03 22 30 68 +a00000010000fe16: 00 00 00 00 00 80 movl r28=0x10060;; +a00000010000fe1c: 03 26 00 60 a00000010000fe20: 01 10 00 20 00 21 [MII] mov r2=r16 a00000010000fe26: 00 21 42 0c 42 e0 adds r16ƒ6,r16 a00000010000fe2c: 03 00 cc 00 mov r31=pr;;
@@ -1563052,7 +1563052,7 @@ a00000010091856c: 03 73 ff 00 data8 0x1fee607fe a000000100918570: 73 ec 02 f0 05 44 [MBB] (p35) mov pmd[r120]=r0 a000000100918576: ee 02 f0 06 46 e8 data8 0x1181bc00bb9 -a00000010091857c: 02 f0 03 4c br.ctop.sptk.few.clr a0000000ff957570 <__kernel_sigtramp+0xff946e50>;; +a00000010091857c: 02 f0 03 4c br.ctop.sptk.few.clr a0000000ff957570 <__end_gate_brl_fsys_bubble_down_patchlist+0xff957064>;; a000000100918580: e4 02 f0 01 36 e6 [MLX] (p23) mov r0=-2090116 a000000100918586: 02 f0 02 38 e1 02 data8 0x1be0edc605 a00000010091858c: e3 76 f0 0d
@@ -1563061,7 +1563061,7 @@ a00000010091859c: 03 73 ff 00 data8 0x1fee607fe a0000001009185a0: 73 ec 00 f0 05 44 [MBB] (p35) mov pmd[r120]=r0 a0000001009185a6: ee 00 f0 06 46 e8 data8 0x1181bc003b9 -a0000001009185ac: 00 f0 03 4c br.ctop.sptk.few.clr a0000000ff9575a0 <__kernel_sigtramp+0xff946e80>;; +a0000001009185ac: 00 f0 03 4c br.ctop.sptk.few.clr a0000000ff9575a0 <__end_gate_brl_fsys_bubble_down_patchlist+0xff957094>;; a0000001009185b0: e4 00 f0 01 36 e6 [MLX] (p07) mov r0=-2090116 a0000001009185b6: 00 f0 02 38 e1 00 data8 0x1cdf2edc601 a0000001009185bc: e3 76 f9 e6
Quoted 14 lines
I don't know if this helps or not.
I ran Jes's kernel on the simulator. Unfortunately, he sent me a stripped kernel so I have no symbol table.
The kernel blows up right after printing:
... Virtual mem_map starts at 0xa0007fffd5f2c000 Built 1 zonelists Kernel command line: root=/dev/hda2 init=/bin/bash console=ttyS0 PID hash table entries: 1024 (order: 10, 32768 bytes) Console: colour dummy device 80x25
Quoted 8 lines
The failure is an MCA caused by a cache hit on a memory reference to an uncached address. The simulator detects this error & stops.
The code that took the failure was memcopy (or equiv). I recognized the code from the prefetchs and ld/st sequence. The data appears to be an ACPI table that is being copied into kernel memory. The current reference is using uncached addresses but 15M instructions in the past, the table was referenced cached.
I found the problem that is causing the cache/uncached collision. This is most likely NOT what is causing the problem you are currently chasing, but it is a problem never-the-less.
(Background: I noticed that recent IA64 kernels are making references to the same address using both cached & uncached addresses. The kernel fails to boot on the simulator because it detects a cache-hit on an uncached reference. Mixing cached & uncached references is not supported).
acpi_os_map_memory() has changed the order of checking the EFI attributes on pages being mapped. In 2.6.15, if the EFI map indicates that an address supports both cached & uncached references, acpi_os_map_memory() would map it as cached.
The most recent kernel has changed this behavior so that the address will be referenced as uncached.
OLD:
acpi_os_map_memory
if (EFI_MEMORY_WB & efi_mem_attributes(phys)) {
*virt = (void __iomem *)phys_to_virt(phys);
} else {
*virt = ioremap(phys, size);
}NEW
acpi_os_map_memory() unconditionally calls ioremap() to do the mapping
ioremap
if (efi_mem_attribute_range(offset, size, EFI_MEMORY_UC))
return __ioremap(offset, size);
if (efi_mem_attribute_range(offset, size, EFI_MEMORY_WB))
return phys_to_virt(offset);
Earlier in the boot, the ACPI tables are unconditionally references as cached:
char *__acpi_map_table(unsigned long phys_addr, unsigned long size)
{
return __va(phys_addr);
}I suspect there was a reason for this change. I'll add Bjorn to the cc list.
Is this problem unique to SN systems. The BIOS reports that most memory ranges support both CACHED & UNCACHED references. I _think_ this is correct.
Hello all,
Quoted 5 lines
--- arch/ia64/kernel/gate.lds.S | 1 + include/asm-ia64/asmmacro.h | 4 ++++ 2 files changed, 5 insertions(+)
I applied this patch and compiled using gcc-4.1.0-3.
I now see hundreds of warnings like:
WARNING: drivers/atm/he.o - Section mismatch: reference to .init.text:he_start from .text after 'he_init_one' (at offset 0x81a2)
WARNING: drivers/block/cpqarray.o - Section mismatch: reference to .init.text:cpqarray_init_one from .data.rel.local between 'cpqarray_pci_driver' (at offset 0x20) and 'smart1_access'
Just an FYI,
P.
Quoted 1 line · Jack Steiner
Is this problem unique to SN systems
No, the same will happen on all other systems (that I know of).
Quoted 3 lines
The BIOS reports that most
memory ranges support both CACHED & UNCACHED references. I _think_ this is correct.
That's correct. The map shows the ways the page *can* be mapped, not the way it *should* be mapped.
It's strange that ACPI would prefer to use WC when WB mapping is possible. They definitely need to pick one way and stick with it though. As you say, mapping the same page with different cacheability is a no-no (and at least in theory, it should cause an MCA even on real hardware).
--david
--
Mosberger Consulting LLC, http://www.mosberger-consulting.com/
Quoted 10 lines · David Mosberger-Tang
Quoted 1 line · Jack Steiner
Is this problem unique to SN systems
No, the same will happen on all other systems (that I know of).
Quoted 3 lines
The BIOS reports that most
memory ranges support both CACHED & UNCACHED references. I _think_ this is correct.
Quoted 8 lines
That's correct. The map shows the ways the page *can* be mapped, not the way it *should* be mapped.
It's strange that ACPI would prefer to use WC when WB mapping is possible. They definitely need to pick one way and stick with it though. As you say, mapping the same page with different cacheability is a no-no (and at least in theory, it should cause an MCA even on real hardware).
It does, at least on our chipset. If the chipset detects a simultaneous UC & C reference, it generates a BUS error. It is surprising how quickly this MCA occurs when we break the rules.
Quoted 5 lines
--david
--
Mosberger Consulting LLC, http://www.mosberger-consulting.com/
Signature · 1 line
Jack
Quoted 7 lines · Jack Steiner
Quoted 2 lines · David Mosberger-Tang
(and at least in theory, it should cause an MCA even on real hardware).
It does, at least on our chipset. If the chipset detects a simultaneous UC & C reference, it generates a BUS error. It is surprising how quickly this MCA occurs when we break the rules.
That's good. While MCAs are a pain, it's a lot better than having to track down data corruption due to (in-)coherency.
--david
--
Mosberger Consulting LLC, http://www.mosberger-consulting.com/
Quoted 10 lines · Prarit Bhargava
Hello all,
Quoted 5 lines
---
arch/ia64/kernel/gate.lds.S | 1 + include/asm-ia64/asmmacro.h | 4 ++++ 2 files changed, 5 insertions(+)I applied this patch and compiled using gcc-4.1.0-3.
Quoted 8 lines
I now see hundreds of warnings like:
WARNING:
drivers/atm/he.o- Section mismatch: reference to .init.text:he_start from .text after 'he_init_one' (at offset0x81a2)
WARNING:drivers/block/cpqarray.o- Section mismatch: reference to .init.text:cpqarray_init_one from .data.rel.local between 'cpqarray_pci_driver' (at offset0x20) and 'smart1_access'
Where did they come from?
H.J.
Quoted 4 lines
Where did they come from?
During the 'make compressed' phase ...
P.
Quoted 3 lines
H.J.
Quoted 7 lines · Prarit Bhargava
Quoted 4 lines
Where did they come from?
During the 'make compressed' phase ...
Which program generated them?
H.J.
Quoted 3 lines
Which program generated them?
[root@altix2 ~]# gcc --version
gcc (GCC) 4.1.0 20060304 (Red Hat 4.1.0-3)
P.
Quoted 17 lines · Jack Steiner
acpi_os_map_memory()has changed the order of checking the EFI attributes on pages being mapped. In 2.6.15, if the EFI map indicates that an address supports both cached & uncached references,acpi_os_map_memory()would map it as cached.OLD:
acpi_os_map_memory if (EFI_MEMORY_WB & efi_mem_attributes(phys)) { *virt = (void __iomem *)phys_to_virt(phys); } else { *virt = ioremap(phys, size); }NEW
acpi_os_map_memory()unconditionally callsioremap()to do the mapping
Quoted 6 lines
ioremap
if (efi_mem_attribute_range(offset, size, EFI_MEMORY_UC))
return__ioremap(offset, size);if (
efi_mem_attribute_range(offset, size, EFI_MEMORY_WB))
returnphys_to_virt(offset);
I debated whether to check for UC or WB first, and I think I got it wrong. My reasoning for UC first was that it was a smaller change for the typical case CSR mapping case, because ioremap() used to *always* return UC mappings.
But efi_memmap_init() collects full granules of WB memory, without regard for whether they also support UC. So if ioremap() needs to work for main memory (which is slightly strange in itself, but seems to be the precedent), it needs to prefer WB mappings when possible.
Quoted 6 lines
Earlier in the boot, the ACPI tables are unconditionally references as cached:
char *__acpi_map_table(unsigned long phys_addr, unsigned long size) { return __va(phys_addr); }
This seems suspicious to me. But we look at some of these ACPI tables prior to calling efi_memmap_init(), so it might not be safe to use ioremap() yet. It's conceivable that a platform could place ACPI tables in UC-only ROM, and this would blow up. But I guess we haven't tripped over that yet.
Does the patch below solve your immediate problem?
[IA64] ioremap() should prefer WB over UC
efi_memmap_init() collects full granules of WB memory, without regard for whether they also support UC. So in order for ioremap() to work for main memory, it must prefer WB mappings when possible.
Index: work-mm5/arch/ia64/mm/ioremap.c
=================================--- work-mm5.orig/arch/ia64/mm/ioremap.c 2006-03-23 10:22:40.000000000 -0700
+++ work-mm5/arch/ia64/mm/ioremap.c 2006-03-30 09:36:57.000000000 -0700
@@ -21,12 +21,12 @@
void __iomem *
ioremap (unsigned long offset, unsigned long size)
{
- if (efi_mem_attribute_range(offset, size, EFI_MEMORY_UC))
- return __ioremap(offset, size);
-
if (efi_mem_attribute_range(offset, size, EFI_MEMORY_WB))
return phys_to_virt(offset);
+ if (efi_mem_attribute_range(offset, size, EFI_MEMORY_UC))
+ return __ioremap(offset, size);
+
/*
* Someday this should check ACPI resources so we
* can do the right thing for hot-plugged regions.
Quoted 42 lines · Bjorn Helgaas
Quoted 17 lines · Jack Steiner
acpi_os_map_memory()has changed the order of checking the EFI attributes on pages being mapped. In 2.6.15, if the EFI map indicates that an address supports both cached & uncached references,acpi_os_map_memory()would map it as cached.OLD:
acpi_os_map_memory if (EFI_MEMORY_WB & efi_mem_attributes(phys)) { *virt = (void __iomem *)phys_to_virt(phys); } else { *virt = ioremap(phys, size); }NEW
acpi_os_map_memory()unconditionally callsioremap()to do the mappingQuoted 6 lines
ioremap
if (efi_mem_attribute_range(offset, size, EFI_MEMORY_UC))
return__ioremap(offset, size);if (
efi_mem_attribute_range(offset, size, EFI_MEMORY_WB))
returnphys_to_virt(offset);I debated whether to check for UC or WB first, and I think I got it wrong. My reasoning for UC first was that it was a smaller change for the typical case CSR mapping case, because
ioremap()used to *always* return UC mappings.But
efi_memmap_init()collects full granules of WB memory, without regard for whether they also support UC. So ifioremap()needs to work for main memory (which is slightly strange in itself, but seems to be the precedent), it needs to prefer WB mappings when possible.Quoted 6 lines
Earlier in the boot, the ACPI tables are unconditionally references as cached:
char *__acpi_map_table(unsigned long phys_addr, unsigned long size) { return __va(phys_addr); }
Quoted 7 lines
This seems suspicious to me. But we look at some of these ACPI tables prior to calling
efi_memmap_init(), so it might not be safe to useioremap()yet. It's conceivable that a platform could place ACPI tables in UC-only ROM, and this would blow up. But I guess we haven't tripped over that yet.Does the patch below solve your immediate problem?
Looks good.
Quoted 29 lines
[IA64]
ioremap()should prefer WB over UC
efi_memmap_init()collects full granules of WB memory, without regard for whether they also support UC. So in order forioremap()to work for main memory, it must prefer WB mappings when possible.Signed-off-by Bjorn Helgaas <[email protected]>Index:
work-mm5/arch/ia64/mm/ioremap.c
=================================> ---work-mm5.orig/arch/ia64/mm/ioremap.c2006-03-23 10:22:40.000000000 -0700
+++work-mm5/arch/ia64/mm/ioremap.c2006-03-30 09:36:57.000000000 -0700(quoted diff) +3
-3@@ -21,12 +21,12 @@ void __iomem * ioremap (unsigned long offset, unsigned long size) { - if (efi_mem_attribute_range(offset, size, EFI_MEMORY_UC)) - return __ioremap(offset, size); - if (efi_mem_attribute_range(offset, size, EFI_MEMORY_WB)) return phys_to_virt(offset); + if (efi_mem_attribute_range(offset, size, EFI_MEMORY_UC)) + return __ioremap(offset, size); + /* * Someday this should check ACPI resources so we * can do the right thing for hot-plugged regions.
Signature · 4 lines
Thanks Jack Steiner ([email protected]) 651-683-5302 Principal Engineer SGI - Silicon Graphics, Inc.
Quoted 9 lines
I applied this patch and compiled using gcc-4.1.0-3.
I now see hundreds of warnings like:
WARNING:
drivers/atm/he.o- Section mismatch: reference to .init.text:he_start from .text after 'he_init_one' (at offset0x81a2)
WARNING:drivers/block/cpqarray.o- Section mismatch: reference to .init.text:cpqarray_init_one from .data.rel.local between 'cpqarray_pci_driver' (at offset0x20) and 'smart1_access'
Just tried with vanilla gcc 4.1.0, and I don't see these warnings.
BTW, these messages presumably come from scripts/mod/modpost.c
-Tony
Quoted 7 lines · Prarit Bhargava
Quoted 3 lines
Which program generated them?
[root@altix2 ~]# gcc --version
gcc (GCC) 4.1.0 20060304 (Red Hat 4.1.0-3)
Humph. I can't get this to happen again :/. Oh well ... if anyone else sees this issue please let me know.
P.
Quoted 6 lines
P.
-
To unsubscribe from this list: send the line "unsubscribe linux-ia64" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Mar 30, 2006 at 09:16:11AM -0600, Jack Steiner wrote:
...
Quoted 2 lines
The most recent kernel has changed this behavior so that the address will be referenced as uncached.
...
Quoted 7 lines
NEW
acpi_os_map_memory()unconditionally callsioremap()to do the mappingioremap
if (efi_mem_attribute_range(offset, size, EFI_MEMORY_UC))
return__ioremap(offset, size);
Could this be related to the issue of ioremap() vs ioremap_nocache() that Mathew Wilcox proposed a patch for?
http://lkml.org/lkml/2006/3/30/251
grant