Skip to contentSkip to conversation

2.6.3 Heisenbug in unwind.c

DISCUSSION Keith Owens 2 participants 9 messages Feb 24, 2004 – Mar 17, 2004

lore
Share & tools
lore
#0 lore raw

I am seeing a Heisenbug in the 2.6.3 kernel unwind code. The symptoms are that the backtrace terminates early, usually failing to unwind past an interrupt frame. Andreas, this _may_ be what you are seeing.

Changing the config options (sn2->dig) makes backtrace work again. Turning on UNW_DEBUG to debug the unwinder makes backtrace work again :(. Adding 30 dummy functions (which only call printk and are never called themselves) to unwind.c makes the backtrace work again.

That last one really worries me. All it does is shift the position of the real unwind code within the kernel without changing the unwind code itself. Looks like an uninitialised pointer somewhere.

gcc version 3.2.3 20030502 (Red Hat Linux 3.2.3-24)
GNU assembler version 2.14.90.0.4 (ia64-unknown-linux-gnu) using BFD version 2.14.90.0.4 20030523

#1 lore raw
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Keith Owens

Keith> I am seeing a Heisenbug in the 2.6.3 kernel unwind code. The Keith> symptoms are that the backtrace terminates early, usually Keith> failing to unwind past an interrupt frame.

I haven't seen that in quite some time.

Keith> Andreas, this _may_ be what you are seeing.

Keith> Changing the config options (sn2->dig) makes backtrace work Keith> again. Turning on UNW_DEBUG to debug the unwinder makes Keith> backtrace work again :(. Adding 30 dummy functions (which Keith> only call printk and are never called themselves) to unwind.c Keith> makes the backtrace work again.

Keith> That last one really worries me. All it does is shift the Keith> position of the real unwind code within the kernel without Keith> changing the unwind code itself. Looks like an uninitialised Keith> pointer somewhere.

Keith> gcc version 3.2.3 20030502 (Red Hat Linux 3.2.3-24) GNU Keith> assembler version 2.14.90.0.4 (ia64-unknown-linux-gnu) using Keith> BFD version 2.14.90.0.4 20030523

It doesn't sound like a timing-related bug though, which is good news. If you could reproduce the problem with Ski, that would almosts certainly make it possible to root-cause it relatively quickly.

It might also be worthwhile to see if gcc 3.3.3 or 3.4 makes any difference.

	--david
#2 lore raw
In reply to Keith Owens
Quoted 3 lines · Keith Owens

I am seeing a Heisenbug in the 2.6.3 kernel unwind code. The symptoms are that the backtrace terminates early, usually failing to unwind past an interrupt frame.

One possible contender for this unwind Heisenbug. Building a 2.6.4-rc3 kernel with gcc version 3.2.3 20030502 (Red Hat Linux 3.2.3-24) and GNU ld version 2.14.90.0.4 20030523. The unwind data in vmlinux is invalid, with overlapping entries. If this command reports anything at all then your unwind data is stuffed.

readelf -u vmlinux | grep '+[a-f0-9]*>:' | head -5

An extract of the descriptor triplets looks like this. It seems that the unwind descriptors for .text and .init.text have been merged together, as if both sections started at the same offset.

00015a00 00015a70 005ced40
00015a80 00015b20 005ced58
00015ac0 00015d10 005c6828 Illegal insert, belongs to __init text 00015b20 00015ca0 005ced70
00015ca0 00015d30 005ced90
00015d20 00015e50 005c6848 Illegal insert, belongs to __init text 00015d40 00015f90 005ceda8
00015e60 00015f90 005c6868
00015fa0 000162e0 005c6888

Depending on precisely where the interrupt occurs, you may pick up a correct or an incorrect unwind descriptor. Which in turn affects the backtrace, and explains why changing code size may the Heisenbug move.

Using the same toolchain to build a 2.4 kernel is not a problem.

#3 lore raw
In reply to Keith Owens
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Keith Owens

Keith> One possible contender for this unwind Heisenbug. Building a Keith> 2.6.4-rc3 kernel with gcc version 3.2.3 20030502 (Red Hat Keith> Linux 3.2.3-24) and GNU ld version 2.14.90.0.4 20030523. The Keith> unwind data in vmlinux is invalid, with overlapping entries. Keith> If this command reports anything at all then your unwind data Keith> is stuffed.

Keith> readelf -u vmlinux | grep '+[a-f0-9]*>:' | head -5

Keith> An extract of the descriptor triplets looks like this. It Keith> seems that the unwind descriptors for .text and .init.text Keith> have been merged together, as if both sections started at the Keith> same offset.

Keith> 00015a00 00015a70 005ced40
Keith> 00015a80 00015b20 005ced58
Keith> 00015ac0 00015d10 005c6828 Illegal insert, belongs to __init text Keith> 00015b20 00015ca0 005ced70
Keith> 00015ca0 00015d30 005ced90
Keith> 00015d20 00015e50 005c6848 Illegal insert, belongs to __init text Keith> 00015d40 00015f90 005ceda8
Keith> 00015e60 00015f90 005c6868
Keith> 00015fa0 000162e0 005c6888

How nasty!

I don't see this problem with:

	gcc version 3.3.3 20040125
	ld version 2.14.90.0.7 20031029 Debian GNU/Linux

I doubt it's a gcc bug. Can you try with a more recent version of binutils?

Keith> Depending on precisely where the interrupt occurs, you may Keith> pick up a correct or an incorrect unwind descriptor. Which Keith> in turn affects the backtrace, and explains why changing code Keith> size may the Heisenbug move.

Yeah, that would certainly explain it! Good hunting!

	--david
#4 lore raw
In reply to Keith Owens
Quoted 6 lines · David Mosberger
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Keith Owens

Keith> 00015a00 00015a70 005ced40
Keith> 00015a80 00015b20 005ced58
Keith> 00015ac0 00015d10 005c6828 Illegal insert, belongs to __init text
How nasty!

Quoted 7 lines

I don't see this problem with:

	gcc version 3.3.3 20040125
	ld version 2.14.90.0.7 20031029 Debian GNU/Linux

I doubt it's a gcc bug. Can you try with a more recent version of binutils?

The bug only manifests itself when one particular SGI patch is applied to the kernel. When one particular object from that patch is linked into the kernel then it trips the unwind glitch. My problem is that the code looks fine, and so does the unwind data in its object. Still digging, valid C code should not be able to do this to ld :(.

#5 lore raw
In reply to Keith Owens
Quoted 11 lines

An extract of the descriptor triplets looks like this. It seems that the unwind descriptors for .text and .init.text have been merged together, as if both sections started at the same offset.

00015a00 00015a70 005ced40
00015a80 00015b20 005ced58
00015ac0 00015d10 005c6828 Illegal insert, belongs to __init text 00015b20 00015ca0 005ced70
00015ca0 00015d30 005ced90
00015d20 00015e50 005c6848 Illegal insert, belongs to __init text 00015d40 00015f90 005ceda8

One step further on. The illegal unwind descriptors are being caused by the linker (GNU ld version 2.14.90.0.4 20030523) sometimes putting everything from .init.text onwards in an extra output program header. Has anybody seen this symptom before?

The error is horribly sensitive. Deleting a single asm(nop) bundle from a .c file is enough to move from a bad to a good link. Rerunning the very last ld step with no change to the input files but adding -q to ld makes the problem go away.

Good link, with 4 program headers.

ELF Header:
Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00

  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              EXEC (Executable file)
  Machine:                           Intel IA-64
  Version:                           0x1
  Entry point address:               0x4008320
  Start of program headers:          64 (bytes into file)
  Start of section headers:          9240232 (bytes into file)
  Flags:                             0x50, 64-bit, constant gp
  Size of this header:               64 (bytes)
  Size of program headers:           56 (bytes)
  Number of program headers:         4
  Size of section headers:           64 (bytes)
  Number of section headers:         34

Section header string table index: 31

Section Headers:

  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0]                   NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] .text             PROGBITS         a000000100000000  00010000
       00000000005c05b0  0000000000000000  AX       0     0     32768
  [ 2] __ex_table        PROGBITS         a0000001005c05b0  005d05b0
       0000000000002a10  0000000000000000   A       0     0     4
  [ 3] .data.patch.vtop  PROGBITS         a0000001005c2fc0  005d2fc0
       0000000000000020  0000000000000000   A       0     0     4
  [ 4] .data.patch.mckin PROGBITS         a0000001005c2fe0  005d2fe0
       0000000000000020  0000000000000000   A       0     0     4
  [ 5] .IA_64.unwind_inf PROGBITS         a0000001005c3000  005d3000
       0000000000045c08  0000000000000000   A       0     0     8
  [ 6] .IA_64.unwind     IA_64_UNWIND     a000000100608c08  00618c08
       0000000000034758  0000000000000000  AL       1     1     8
  [ 7] .rodata           PROGBITS         a00000010063d360  0064d360
       0000000000058a41  0000000000000000   A       0     0     8
  [ 8] __ksymtab         PROGBITS         a000000100695da8  006a5da8
       0000000000007480  0000000000000000   A       0     0     8
  [ 9] __ksymtab_gpl     PROGBITS         a00000010069d228  006ad228
       0000000000000170  0000000000000000   A       0     0     8
  [10] __kcrctab         PROGBITS         a00000010069d398  006ad398
       0000000000003a40  0000000000000000   A       0     0     8
  [11] __kcrctab_gpl     PROGBITS         a0000001006a0dd8  006b0dd8
       00000000000000b8  0000000000000000   A       0     0     8
  [12] __ksymtab_strings PROGBITS         a0000001006a0e90  006b0e90
       000000000000921a  0000000000000000   A       0     0     8
  [13] .opd              PROGBITS         a0000001006aa0b0  006ba0b0
       0000000000011f30  0000000000000000   A       0     0     16
  [14] .init.text        PROGBITS         a0000001006bc000  006cc000
       000000000002eb70  0000000000000000 WAX       0     0     32
  [15] .init.data        PROGBITS         a0000001006eab70  006fab70
       000000000001a0c8  0000000000000000  WA       0     0     8
  [16] .init.ramfs       PROGBITS         a000000100704c38  00714c38
       0000000000000086  0000000000000000   A       0     0     1
  [17] .init.setup       PROGBITS         a000000100704cc0  00714cc0
       0000000000000290  0000000000000000  WA       0     0     8
  [18] .kdb_initcall.ini PROGBITS         a000000100704f50  00714f50
       0000000000000010  0000000000000000  WA       0     0     8
  [19] __param           PROGBITS         a000000100704f60  00714f60
       00000000000004d8  0000000000000000   A       0     0     8
  [20] .initcall.init    PROGBITS         a000000100705438  00715438
       0000000000000348  0000000000000000  WA       0     0     8
  [21] .con_initcall.ini PROGBITS         a000000100705780  00715780
       0000000000000010  0000000000000000  WA       0     0     8
  [22] .data.init_task   PROGBITS         a000000100708000  00718000
       0000000000008000  0000000000000000  WA       0     0     16
  [23] .data.page_aligne PROGBITS         a000000100710000  00720000
       000000000000c710  0000000000000000 WAX       0     0     1
  [24] .data.cacheline_a PROGBITS         a000000100720000  00730000
       000000000000cd00  0000000000000000  WA       0     0     128
  [25] .data.percpu      PROGBITS         ffffffffffff0000  00740000
       0000000000009278  0000000000000000  WA       0     0     128
  [26] .data             PROGBITS         a000000100740000  00750000
       000000000016cb97  0000000000000000  WA       0     0     128
  [27] .got              PROGBITS         a0000001008acba0  008bcba0
       0000000000011648  0000000000000000 WAp       0     0     8
  [28] .sdata            PROGBITS         a0000001008be1e8  008ce1e8
       0000000000001b18  0000000000000000 WAp       0     0     8
  [29] .sbss             NOBITS           a0000001008bfd00  008cfd00
       00000000000011e8  0000000000000000 WAp       0     0     8
  [30] .bss              NOBITS           a0000001008c0f00  008cfd18
       00000000001eca80  0000000000000000  WA       0     0     128
  [31] .shstrtab         STRTAB           0000000000000000  008cfd18
       0000000000000189  0000000000000000           0     0     1
  [32] .symtab           SYMTAB           0000000000000000  008d0728
       00000000000aefc0  0000000000000018          33   53ec     8
  [33] .strtab           STRTAB           0000000000000000  0097f6e8
       00000000000874d3  0000000000000000           0     0     1

Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings)
I (info), L (link order), G (group), x (unknown)
O (extra OS processing required) o (OS specific), p (processor specific)

Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
LOAD 0x0000000000010000 0xa000000100000000 0x0000000004000000

                 0x000000000072cd00 0x000000000072cd00  RWE    10000
  LOAD           0x0000000000740000 0xffffffffffff0000 0x0000000004730000
                 0x0000000000009278 0x0000000000009278  RW     10000
  LOAD           0x0000000000750000 0xa000000100740000 0x0000000004740000
                 0x000000000017fd00 0x000000000036d980  RW     10000
  IA_64_UNWIND   0x0000000000618c08 0xa000000100608c08 0x0000000004608c08
                 0x0000000000034758 0x0000000000034758  R      8

Section to Segment mapping:
Segment Sections...

   00     .text __ex_table .data.patch.vtop .data.patch.mckinley_e9 .IA_64.unwind_info .IA_64.unwind .rodata __ksymtab __ksymtab_gpl __kcrctab __kcrctab_gpl __ksymtab_strings .opd .init.text .init.data .init.ramfs .init.setup .kdb_initcall.init __param .initcall.init .con_initcall.init .data.init_task .data.page_aligned .data.cacheline_aligned 
   01     .data.percpu 
   02     .data .got .sdata .sbss .bss 
   03     .IA_64.unwind 

Bad link, with 5 program headers.

ELF Header:
Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00

  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              EXEC (Executable file)
  Machine:                           Intel IA-64
  Version:                           0x1
  Entry point address:               0x4568be0
  Start of program headers:          64 (bytes into file)
  Start of section headers:          8808232 (bytes into file)
  Flags:                             0x50, 64-bit, constant gp
  Size of this header:               64 (bytes)
  Size of program headers:           56 (bytes)
  Number of program headers:         5
  Size of section headers:           64 (bytes)
  Number of section headers:         34

Section header string table index: 31

Section Headers:

  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0]                   NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] .text             PROGBITS         a000000100000000  00010000
       00000000005c05d0  0000000000000000  AX       0     0     32768
  [ 2] __ex_table        PROGBITS         a0000001005c05d0  005d05d0
       0000000000002a10  0000000000000000   A       0     0     4
  [ 3] .data.patch.vtop  PROGBITS         a0000001005c2fe0  005d2fe0
       0000000000000020  0000000000000000   A       0     0     4
  [ 4] .data.patch.mckin PROGBITS         a0000001005c3000  005d3000
       0000000000000020  0000000000000000   A       0     0     4
  [ 5] .IA_64.unwind_inf PROGBITS         a0000001005c4000  005d4000
       0000000000045c08  0000000000000000   A       0     0     8
  [ 6] .IA_64.unwind     IA_64_UNWIND     a000000100609c08  00619c08
       0000000000034758  0000000000000000  AL       1     1     8
  [ 7] .rodata           PROGBITS         a00000010063e360  0064e360
       0000000000058a3d  0000000000000000   A       0     0     8
  [ 8] __ksymtab         PROGBITS         a000000100696da0  006a6da0
       0000000000007480  0000000000000000   A       0     0     8
  [ 9] __ksymtab_gpl     PROGBITS         a00000010069e220  006ae220
       0000000000000170  0000000000000000   A       0     0     8
  [10] __kcrctab         PROGBITS         a00000010069e390  006ae390
       0000000000003a40  0000000000000000   A       0     0     8
  [11] __kcrctab_gpl     PROGBITS         a0000001006a1dd0  006b1dd0
       00000000000000b8  0000000000000000   A       0     0     8
  [12] __ksymtab_strings PROGBITS         a0000001006a1e88  006b1e88
       000000000000921c  0000000000000000   A       0     0     8
  [13] .opd              PROGBITS         a0000001006ab0b0  006bb0b0
       0000000000011f30  0000000000000000   A       0     0     16
  [14] .init.text        PROGBITS         a0000001006c0000  006d0000
       000000000002eb80  0000000000000000 WAX       0     0     32
  [15] .init.data        PROGBITS         a0000001006eeb80  006feb80
       000000000001a0c1  0000000000000000  WA       0     0     8
  [16] .init.ramfs       PROGBITS         a000000100708c41  00718c41
       0000000000000086  0000000000000000   A       0     0     1
  [17] .init.setup       PROGBITS         a000000100708cd0  00718cd0
       0000000000000290  0000000000000000  WA       0     0     8
  [18] .kdb_initcall.ini PROGBITS         a000000100708f60  00718f60
       0000000000000010  0000000000000000  WA       0     0     8
  [19] __param           PROGBITS         a000000100708f70  00718f70
       00000000000004d8  0000000000000000   A       0     0     8
  [20] .initcall.init    PROGBITS         a000000100709448  00719448
       0000000000000348  0000000000000000  WA       0     0     8
  [21] .con_initcall.ini PROGBITS         a000000100709790  00719790
       0000000000000010  0000000000000000  WA       0     0     8
  [22] .data.init_task   PROGBITS         a00000010070c000  0071c000
       0000000000008000  0000000000000000  WA       0     0     16
  [23] .data.page_aligne PROGBITS         a000000100714000  00724000
       000000000000c710  0000000000000000 WAX       0     0     1
  [24] .data.cacheline_a PROGBITS         a000000100724000  00734000
       000000000000cc84  0000000000000000  WA       0     0     128
  [25] .data.percpu      PROGBITS         ffffffffffff0000  00750000
       0000000000009260  0000000000000000  WA       0     0     128
  [26] .data             PROGBITS         a000000100750000  00760000
       00000000000f33c0  0000000000000000  WA       0     0     128
  [27] .got              PROGBITS         a0000001008433c0  008533c0
       0000000000011648  0000000000000000 WAp       0     0     8
  [28] .sdata            PROGBITS         a000000100854a08  00864a08
       0000000000001b18  0000000000000000 WAp       0     0     8
  [29] .sbss             NOBITS           a000000100856520  00866520
       00000000000011e4  0000000000000000 WAp       0     0     8
  [30] .bss              NOBITS           a000000100857780  0086659c
       00000000001ecb20  0000000000000000  WA       0     0     128
  [31] .shstrtab         STRTAB           0000000000000000  0086659c
       0000000000000189  0000000000000000           0     0     1
  [32] .symtab           SYMTAB           0000000000000000  00866fa8
       00000000000aef78  0000000000000018          33   53e9     8
  [33] .strtab           STRTAB           0000000000000000  00915f20
       00000000000874d3  0000000000000000           0     0     1

Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings)
I (info), L (link order), G (group), x (unknown)
O (extra OS processing required) o (OS specific), p (processor specific)

Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
LOAD 0x0000000000010000 0xa000000100000000 0x0000000004000000

                 0x00000000006bcfe0 0x00000000006bcfe0  R E    10000
  LOAD           0x00000000006d0000 0xa0000001006c0000 0x00000000046c0000
                 0x0000000000070c84 0x0000000000070c84  RWE    10000
  LOAD           0x0000000000750000 0xffffffffffff0000 0x0000000004740000
                 0x0000000000009260 0x0000000000009260  RW     10000
  LOAD           0x0000000000760000 0xa000000100750000 0x0000000004750000
                 0x0000000000106520 0x00000000002f42a0  RW     10000
  IA_64_UNWIND   0x0000000000619c08 0xa000000100609c08 0x0000000004609c08
                 0x0000000000034758 0x0000000000034758  R      8

Section to Segment mapping:
Segment Sections...

   00     .text __ex_table .data.patch.vtop .data.patch.mckinley_e9 .IA_64.unwind_info .IA_64.unwind .rodata __ksymtab __ksymtab_gpl __kcrctab __kcrctab_gpl __ksymtab_strings .opd 
   01     .init.text .init.data .init.ramfs .init.setup .kdb_initcall.init __param .initcall.init .con_initcall.init .data.init_task .data.page_aligned .data.cacheline_aligned 
   02     .data.percpu 
   03     .data .got .sdata .sbss .bss 
   04     .IA_64.unwind 
#6 lore raw
In reply to Keith Owens
Quoted 3 lines · Keith Owens

One step further on. The illegal unwind descriptors are being caused by the linker (GNU ld version 2.14.90.0.4 20030523) sometimes putting everything from .init.text onwards in an extra output program header.

Which of course it is quite entitled to do. vmlinux.lds.S does not define the mapping of sections to segments so the linker can choose the layout itself. Most of the time .text and .init.text are in the same segment, but sometimes they are in separate segments. When they are in separate segments _and_ binutils has buggy SEGREL code then the unwind data is useless.

Refuse to build 2.6 ia64 with a buggy binutils.

Index: linux-2.6.4/arch/ia64/scripts/toolchain-flags
=================================--- linux-2.6.4.orig/arch/ia64/scripts/toolchain-flags 2004-03-10 18:55:49.000000000 -0800
+++ linux-2.6.4/arch/ia64/scripts/toolchain-flags 2004-03-15 22:11:02.000000000 -0800
@@ -18,11 +18,6 @@
rm -f $out
if [ $res != 00000a00 ]; then
CPPFLAGS="$CPPFLAGS -DHAVE_BUGGY_SEGREL"
- cat >&2 <<EOF
-warning: your linker cannot handle cross-segment segment-relative relocations.
- please upgrade to a newer version (it is safe to use this linker, but
- the kernel will be bigger than strictly necessary).
-EOF
fi

# Check whether .align inside a function works as expected.
Index: linux-2.6.4/arch/ia64/Makefile
=================================--- linux-2.6.4.orig/arch/ia64/Makefile 2004-03-11 22:29:00.000000000 -0800
+++ linux-2.6.4/arch/ia64/Makefile 2004-03-15 22:18:06.000000000 -0800
@@ -37,6 +37,12 @@
ftp://ftp.hpl.hp.com/pub/linux-ia64/gas-030124.tar.gz)
endif

+ifneq ($(findstring -DHAVE_BUGGY_SEGREL,$(CPPFLAGS)),)
+$(error Sorry, you need a newer version of binutils, one that is built from \
+ a source-tree that post-dates 17-Jul-2003. binutils-2.14.90.0.8 is \
+ known to work)
+endif
+
ifeq ($(GCC_VERSION),2)
$(error Sorry, your compiler is too old. GCC v2.96 is known to generate bad code.) endif
Index: linux-2.6.4/arch/ia64/kernel/gate.lds.S
=================================--- linux-2.6.4.orig/arch/ia64/kernel/gate.lds.S 2004-03-10 18:55:24.000000000 -0800
+++ linux-2.6.4/arch/ia64/kernel/gate.lds.S 2004-03-15 21:21:02.000000000 -0800
@@ -46,12 +46,8 @@
} :readable
.IA_64.unwind_info : { *(.IA_64.unwind_info*) }
.IA_64.unwind : { *(.IA_64.unwind*) } :readable :unwind
-#ifdef HAVE_BUGGY_SEGREL
- .text (GATE_ADDR + PAGE_SIZE) : { *(.text) *(.text.*) } :readable
-#else
. = ALIGN (PERCPU_PAGE_SIZE) + (. & (PERCPU_PAGE_SIZE - 1));
.text : { *(.text) *(.text.*) } :epc
-#endif

/DISCARD/ : {
*(.got.plt) *(.got)
@@ -69,9 +65,7 @@
PHDRS
{
readable PT_LOAD FILEHDR PHDRS FLAGS(4); /* PF_R */
-#ifndef HAVE_BUGGY_SEGREL
epc PT_LOAD FILEHDR PHDRS FLAGS(1); /* PF_X */
-#endif
dynamic PT_DYNAMIC FLAGS(4); /* PF_R */
unwind 0x70000001; /* PT_IA_64_UNWIND, but ld doesn't match the name */
}
Index: linux-2.6.4/arch/ia64/mm/init.c
=================================--- linux-2.6.4.orig/arch/ia64/mm/init.c 2004-03-10 18:55:26.000000000 -0800
+++ linux-2.6.4/arch/ia64/mm/init.c 2004-03-15 21:21:18.000000000 -0800
@@ -265,12 +265,7 @@
*/
page = virt_to_page(ia64_imva(__start_gate_section)); put_kernel_page(page, GATE_ADDR, PAGE_READONLY);
-#ifdef HAVE_BUGGY_SEGREL
- page = virt_to_page(ia64_imva(__start_gate_section + PAGE_SIZE));
- put_kernel_page(page, GATE_ADDR + PAGE_SIZE, PAGE_GATE);
-#else
put_kernel_page(page, GATE_ADDR + PERCPU_PAGE_SIZE, PAGE_GATE);
-#endif
ia64_patch_gate();
}

#7 lore raw
In reply to Keith Owens
Quoted 3 lines · Keith Owens

One step further on. The illegal unwind descriptors are being caused by the linker (GNU ld version 2.14.90.0.4 20030523) sometimes putting everything from .init.text onwards in an extra output program header.

Which of course it is entitled to do. vmlinux.lds.S does not explicitly assign sections to segments so ld is allowed to choose its own mapping. Most of the time the .text and .init.text sections are in the same segment, but sometimes the linker assigns them to different segments, depending on the precise contents of the input files. When those sections are in different segments _and_ you are using binutils with buggy ia64 SEGREL code then the result is corrupt unwind data.

binutils-2.14.90.0.4-26.3 (RH AS3.0) is not good enough, binutils-2.14.90.0.8-8.1 (Fedora) works.

Patch to 2.6.4 to prevent building with a buggy version of binutils.

Index: linux-2.6.4/arch/ia64/scripts/toolchain-flags
=================================--- linux-2.6.4.orig/arch/ia64/scripts/toolchain-flags 2004-03-10 18:55:49.000000000 -0800
+++ linux-2.6.4/arch/ia64/scripts/toolchain-flags 2004-03-15 22:11:02.000000000 -0800
@@ -18,11 +18,6 @@
rm -f $out
if [ $res != 00000a00 ]; then
CPPFLAGS="$CPPFLAGS -DHAVE_BUGGY_SEGREL"
- cat >&2 <<EOF
-warning: your linker cannot handle cross-segment segment-relative relocations.
- please upgrade to a newer version (it is safe to use this linker, but
- the kernel will be bigger than strictly necessary).
-EOF
fi

# Check whether .align inside a function works as expected.
Index: linux-2.6.4/arch/ia64/kernel/gate.lds.S
=================================--- linux-2.6.4.orig/arch/ia64/kernel/gate.lds.S 2004-03-10 18:55:24.000000000 -0800
+++ linux-2.6.4/arch/ia64/kernel/gate.lds.S 2004-03-15 21:21:02.000000000 -0800
@@ -46,12 +46,8 @@
} :readable
.IA_64.unwind_info : { *(.IA_64.unwind_info*) }
.IA_64.unwind : { *(.IA_64.unwind*) } :readable :unwind
-#ifdef HAVE_BUGGY_SEGREL
- .text (GATE_ADDR + PAGE_SIZE) : { *(.text) *(.text.*) } :readable
-#else
. = ALIGN (PERCPU_PAGE_SIZE) + (. & (PERCPU_PAGE_SIZE - 1));
.text : { *(.text) *(.text.*) } :epc
-#endif

/DISCARD/ : {
*(.got.plt) *(.got)
@@ -69,9 +65,7 @@
PHDRS
{
readable PT_LOAD FILEHDR PHDRS FLAGS(4); /* PF_R */
-#ifndef HAVE_BUGGY_SEGREL
epc PT_LOAD FILEHDR PHDRS FLAGS(1); /* PF_X */
-#endif
dynamic PT_DYNAMIC FLAGS(4); /* PF_R */
unwind 0x70000001; /* PT_IA_64_UNWIND, but ld doesn't match the name */
}
Index: linux-2.6.4/arch/ia64/mm/init.c
=================================--- linux-2.6.4.orig/arch/ia64/mm/init.c 2004-03-10 18:55:26.000000000 -0800
+++ linux-2.6.4/arch/ia64/mm/init.c 2004-03-15 21:21:18.000000000 -0800
@@ -265,12 +265,7 @@
*/
page = virt_to_page(ia64_imva(__start_gate_section)); put_kernel_page(page, GATE_ADDR, PAGE_READONLY);
-#ifdef HAVE_BUGGY_SEGREL
- page = virt_to_page(ia64_imva(__start_gate_section + PAGE_SIZE));
- put_kernel_page(page, GATE_ADDR + PAGE_SIZE, PAGE_GATE);
-#else
put_kernel_page(page, GATE_ADDR + PERCPU_PAGE_SIZE, PAGE_GATE);
-#endif
ia64_patch_gate();
}

Index: linux-2.6.4/arch/ia64/Makefile
=================================--- linux-2.6.4.orig/arch/ia64/Makefile 2004-03-11 22:29:00.000000000 -0800
+++ linux-2.6.4/arch/ia64/Makefile 2004-03-15 22:18:06.000000000 -0800
@@ -37,6 +37,12 @@
ftp://ftp.hpl.hp.com/pub/linux-ia64/gas-030124.tar.gz)
endif

+ifneq ($(findstring -DHAVE_BUGGY_SEGREL,$(CPPFLAGS)),)
+$(error Sorry, you need a newer version of binutils, one that is built from \
+ a source-tree that post-dates 17-Jul-2003. binutils-2.14.90.0.8 is \
+ known to work)
+endif
+
ifeq ($(GCC_VERSION),2)
$(error Sorry, your compiler is too old. GCC v2.96 is known to generate bad code.) endif

#8 lore raw
In reply to Keith Owens
Quoted 10 lines · David Mosberger

Yes, that basically looks good. I do think you have to change the order to this:

PHDRS {
code PT_LOAD;
percpu PT_LOAD;
data PT_LOAD;
}

to avoid extra padding, though.

Done.

Index: linux-2.6.4/arch/ia64/kernel/vmlinux.lds.S
=================================--- linux-2.6.4.orig/arch/ia64/kernel/vmlinux.lds.S Mon Mar 15 10:14:58 2004
+++ linux-2.6.4/arch/ia64/kernel/vmlinux.lds.S Wed Mar 17 14:44:14 2004
@@ -12,6 +12,11 @@
OUTPUT_ARCH(ia64)
ENTRY(phys_start)
jiffies = jiffies_64;
+PHDRS {

+  code   PT_LOAD;
+  percpu PT_LOAD;
+  data   PT_LOAD;

+}
SECTIONS
{
/* Sections to be discarded */
@@ -27,6 +32,7 @@
v = PAGE_OFFSET; /* this symbol is here to make debugging easier... */
phys_start = _start - LOAD_OFFSET;

+ code : { } :code
. = KERNEL_START;

_text = .;
@@ -180,6 +186,7 @@

         { *(.data.cacheline_aligned) }

/* Per-cpu data: */
+ percpu : { } :percpu
. = ALIGN(PERCPU_PAGE_SIZE);
__phys_per_cpu_start = .;
.data.percpu PERCPU_ADDR : AT(__phys_per_cpu_start - LOAD_OFFSET)
@@ -190,6 +197,7 @@
}
. = __phys_per_cpu_start + PERCPU_PAGE_SIZE; /* ensure percpu data fits into percpu page size */

+ data : { } :data
.data : AT(ADDR(.data) - LOAD_OFFSET)
{ *(.data) *(.data1) *(.gnu.linkonce.d*) CONSTRUCTORS }

@@ -212,6 +220,7 @@

   _end = .;

+ code : { } :code
/* Stabs debugging sections. */
.stab 0 : { *(.stab) }
.stabstr 0 : { *(.stabstr) }

Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
LOAD 0x0000000000010000 0xa000000100000000 0x0000000004000000

                 0x0000000000730d00 0x0000000000730d00  RWE    10000
  LOAD           0x0000000000750000 0xffffffffffff0000 0x0000000004740000
                 0x0000000000009278 0x0000000000009278  RW     10000
  LOAD           0x0000000000760000 0xa000000100750000 0x0000000004750000
                 0x000000000017e238 0x000000000036bf00  RW     10000
  IA_64_UNWIND   0x0000000000618c48 0xa000000100608c48 0x0000000004608c48
                 0x0000000000034758 0x0000000000034758  R      8

Section to Segment mapping:
Segment Sections...

   00     .text __ex_table .data.patch.vtop .data.patch.mckinley_e9 .IA_64.unwind_info .IA_64.unwind .rodata __ksymtab __ksymtab_gpl __kcrctab __kcrctab_gpl __ksymtab_strings .opd .init.text .init.data .init.ramfs .init.setup .kdb_initcall.init __param .initcall.init .con_initcall.init .data.init_task .data.page_aligned .data.cacheline_aligned 
   01     .data.percpu 
   02     .data .got .sdata .sbss .bss 
   03     .IA_64.unwind 
Quoted 3 lines

Also, what happens for the case that was giving you grief in the first place? Does the linker report an error or does it actually do the right thing?

It does the right thing. The load map and section to segment mapping above is from my test case, without PHDR it puts .init.text in a separate segment.

Quoted 2 lines

I guess I'm wondering why the linker decided to put it into a different segment in the first place.

The only difference is the size of the input objects, literally a single extra nop bundle in one object is enough to trip the error case. Also add -q to ld makes the problem disappear. I am assuming that the size difference on input is enough to affect one of the linker's hashing algorithms. That in turn lets the linker see a difference between

{ .text __ex_table .data.patch.vtop .data.patch.mckinley_e9 .IA_64.unwind_info .IA_64.unwind .rodata __ksymtab __ksymtab_gpl __kcrctab __kcrctab_gpl __ksymtab_strings .opd }

which it puts in a 'R E' segment and these sections

{ .init.text .init.data .init.ramfs .init.setup .kdb_initcall.init __param .initcall.init .con_initcall.init .data.init_task .data.page_aligned .data.cacheline_aligned }

which are marked 'RWE'. In the normal case all those sections are in a single RWE segment.