Out of order unwind entry warning

WARNING: Out of order unwind entry!

I started to look into the problem why we get the

Quoted 1 line

WARNING: Out of order unwind entry!

warning messages at kernel bootup in dmesg.

It's related to a binutils/linker bug, that ld does not correctly removes unlink information for unused weak symbols, which have been replaced by their non-weak ones.

Looks like you did some good sleuthing... Although... I notice that ia64 doesn't seem to bother to do this sanity check on the unwind table.

tausq, should we just drop it in ours too? I can't see any evidence ia64 does anything differently in bfd that would circumvent this kind of issue.

Quoted 6 lines

Looks like you did some good sleuthing... Although... I notice that ia64 doesn't seem to bother to do this sanity check on the unwind table.

tausq, should we just drop it in ours too? I can't see any evidence ia64 does anything differently in bfd that would circumvent this kind of issue.

IIRC we do a binary search for the unwind entry. If the entries are not sorted, the binary search will give wrong results.

I started to look into the problem why we get the

Quoted 1 line

WARNING: Out of order unwind entry!

warning messages at kernel bootup in dmesg.

It's related to a binutils/linker bug, that ld does not correctly removes unlink information for unused weak symbols, which have been replaced by their non-weak ones.

In my kernel I could track it down to 2 symbols:
- arch_mod_section_prepend() from kernel/module.c, and
- save_stack_trace_tsk().

Here is the extract of the readelf output on vmlinux:
...
<inb>: [0x1011fbd4-0x1011fc48]
Entry_GR=1 Save_SP Save_RP Total_frame_size=8
<arch_mod_section_prepend>: [0x1011fc4c-0x1011fc74]
Entry_GR=1 Save_SP Total_frame_size=8
<arch_mod_section_prepend>: [0x1011fc4c-0x10167cf4]
Entry_GR=1 Save_SP Total_frame_size=8
<module_arch_cleanup>: [0x1011fc78-0x1011fcb8]
Entry_GR=2 Save_SP Save_RP Total_frame_size=8
...

Quoted 13 lines

I started to look into the problem why we get the

Quoted 1 line

WARNING: Out of order unwind entry!

warning messages at kernel bootup in dmesg.

It's related to a binutils/linker bug, that ld does not correctly removes unlink information for unused weak symbols, which have been replaced by their non-weak ones.

In my kernel I could track it down to 2 symbols:
- arch_mod_section_prepend() from kernel/module.c, and
- save_stack_trace_tsk().

Quoted 11 lines

Here is the extract of the readelf output on vmlinux:
...
<inb>: [0x1011fbd4-0x1011fc48]
Entry_GR=1 Save_SP Save_RP Total_frame_size=8
<arch_mod_section_prepend>: [0x1011fc4c-0x1011fc74]
Entry_GR=1 Save_SP Total_frame_size=8
<arch_mod_section_prepend>: [0x1011fc4c-0x10167cf4]
Entry_GR=1 Save_SP Total_frame_size=8
<module_arch_cleanup>: [0x1011fc78-0x1011fcb8]
Entry_GR=2 Save_SP Save_RP Total_frame_size=8
...

It might be useful to know the actual location arch_mod_section_prepend and whether it follows inb in its .o. From my understanding of things, it shouldn't be necessary to remove the unwind info for unused weak symbols if they aren't garbage collected. A simple testcase would be helpful.

Quoted 4 lines

<arch_mod_section_prepend>: [0x1011fc4c-0x1011fc74]
Entry_GR=1 Save_SP Total_frame_size=8
<arch_mod_section_prepend>: [0x1011fc4c-0x10167cf4]
Entry_GR=1 Save_SP Total_frame_size=8

Quoted 2 lines

From my understanding of things, it shouldn't be necessary to remove the unwind info for unused weak symbols if they aren't garbage collected.

Hi Dave,

Quoted 3 lines · John David Anglin

Quoted 2 lines

From my understanding of things, it shouldn't be necessary to remove the unwind info for unused weak symbols if they aren't garbage collected.

What do you mean with "garbage collected" ? "dropped from the executable" ? AFAICS, the weak function was completely dropped from the vmlinux file.

Some more info:
The unwind function in the parisc linux kernel walks the (sorted!) unwind list. If one entry is wrong this can lead to incorrect unwinding.

In this case:

<arch_mod_section_prepend>: [0x1011fc4c-0x1011fc74] - non-weak function
Entry_GR=1 Save_SP Total_frame_size=8
<arch_mod_section_prepend>: [0x1011fc4c-0x10167cf4] - weak function
Entry_GR=1 Save_SP Total_frame_size=8
<module_arch_cleanup>: [0x1011fc78-0x1011fcb8]
Entry_GR=2 Save_SP Save_RP Total_frame_size=8

I see a those options:
a) drop the second (weak) unwind info entry
b) for the weak function, use the same start- and end-addresses as the non-weak entry.

Option b) would probably need some adjustments to the existing parisc specific unwind function.

Do you know if linux uses the --gc-sections ld option? This together with the GCC -function-sections option might contribute to this problem.

I see a those options:
a) drop the second (weak) unwind info entry
b) for the weak function, use the same start- and end-addresses as the non-weak entry.

Option b) would probably need some adjustments to the existing parisc specific unwind function.

Quoted 25 lines

Quoted 13 lines

I started to look into the problem why we get the

Quoted 1 line

WARNING: Out of order unwind entry!

warning messages at kernel bootup in dmesg.

It's related to a binutils/linker bug, that ld does not correctly removes unlink information for unused weak symbols, which have been replaced by their non-weak ones.

In my kernel I could track it down to 2 symbols:
- arch_mod_section_prepend() from kernel/module.c, and
- save_stack_trace_tsk().

Quoted 11 lines

Here is the extract of the readelf output on vmlinux:
...
<inb>: [0x1011fbd4-0x1011fc48]
Entry_GR=1 Save_SP Save_RP Total_frame_size=8
<arch_mod_section_prepend>: [0x1011fc4c-0x1011fc74]
Entry_GR=1 Save_SP Total_frame_size=8
<arch_mod_section_prepend>: [0x1011fc4c-0x10167cf4]
Entry_GR=1 Save_SP Total_frame_size=8
<module_arch_cleanup>: [0x1011fc78-0x1011fcb8]
Entry_GR=2 Save_SP Save_RP Total_frame_size=8
...

Quoted 5 lines

It might be useful to know the actual location arch_mod_section_prepend and whether it follows inb in its .o. From my understanding of things, it shouldn't be necessary to remove the unwind info for unused weak symbols if they aren't garbage collected. A simple testcase would be helpful.

arch_mod_section_prepend() is here:
* arch/parisc/kernel/module.c - non-weak function
* kernel/module.c - weak function

inb() is not defined in arch/parisc/kernel/module.c.
But during kernel build, all object files in arch/parisc/kernel/*.o are linked together into arch/parisc/kernel/built-in.o:

Running readelf: "hppa-linux-readelf -u arch/parisc/kernel/built-in.o" gives:
...
<inb>: [0x0-0xe8]
Entry_GR=1 Save_SP Save_RP Total_frame_size=8
<arch_mod_section_prepend>: [0x0-0x50]
Entry_GR=1 Save_SP Total_frame_size=8
<module_arch_cleanup>: [0x0-0x80]
Entry_GR=2 Save_SP Save_RP Total_frame_size=8

So, yes, the non-weak arch_mod_section_prepend() function follows the inb() function in the object file (and in the final vmlinux file as well).

Quoted 4 lines

Quoted 4 lines

<arch_mod_section_prepend>: [0x1011fc4c-0x1011fc74]
Entry_GR=1 Save_SP Total_frame_size=8
<arch_mod_section_prepend>: [0x1011fc4c-0x10167cf4]
Entry_GR=1 Save_SP Total_frame_size=8

The first entry is the non-weak one.
The second entry is the remaining part of the weak (dropped) function. It seems only the start address was changed to the start address of the non-weak function, while the end address has been kept unchanged. [0x1011fc4c-0x10167cf4] would mean, that the weak function would be 295KB in size...?

Quoted 57 lines · John David Anglin

Quoted 25 lines

Quoted 13 lines

I started to look into the problem why we get the

Quoted 1 line

WARNING: Out of order unwind entry!

warning messages at kernel bootup in dmesg.

It's related to a binutils/linker bug, that ld does not correctly removes unlink information for unused weak symbols, which have been replaced by their non-weak ones.

In my kernel I could track it down to 2 symbols:
- arch_mod_section_prepend() from kernel/module.c, and
- save_stack_trace_tsk().

Quoted 11 lines

Here is the extract of the readelf output on vmlinux:
...
<inb>: [0x1011fbd4-0x1011fc48]
Entry_GR=1 Save_SP Save_RP Total_frame_size=8
<arch_mod_section_prepend>: [0x1011fc4c-0x1011fc74]
Entry_GR=1 Save_SP Total_frame_size=8
<arch_mod_section_prepend>: [0x1011fc4c-0x10167cf4]
Entry_GR=1 Save_SP Total_frame_size=8
<module_arch_cleanup>: [0x1011fc78-0x1011fcb8]
Entry_GR=2 Save_SP Save_RP Total_frame_size=8
...

Quoted 5 lines

It might be useful to know the actual location arch_mod_section_prepend and whether it follows inb in its .o. From my understanding of things, it shouldn't be necessary to remove the unwind info for unused weak symbols if they aren't garbage collected. A simple testcase would be helpful.

arch_mod_section_prepend() is here:
* arch/parisc/kernel/module.c - non-weak function
* kernel/module.c - weak function

inb() is not defined in arch/parisc/kernel/module.c.
But during kernel build, all object files in arch/parisc/kernel/*.o are linked together into arch/parisc/kernel/built-in.o:

Running readelf: "hppa-linux-readelf -u arch/parisc/kernel/built-in.o" gives:
...
<inb>: [0x0-0xe8]
Entry_GR=1 Save_SP Save_RP Total_frame_size=8
<arch_mod_section_prepend>: [0x0-0x50]
Entry_GR=1 Save_SP Total_frame_size=8
<module_arch_cleanup>: [0x0-0x80]
Entry_GR=2 Save_SP Save_RP Total_frame_size=8

So, yes, the non-weak arch_mod_section_prepend() function follows the inb() function in the object file (and in the final vmlinux file as well).

Quoted 4 lines

Quoted 4 lines

<arch_mod_section_prepend>: [0x1011fc4c-0x1011fc74]
Entry_GR=1 Save_SP Total_frame_size=8
<arch_mod_section_prepend>: [0x1011fc4c-0x10167cf4]
Entry_GR=1 Save_SP Total_frame_size=8

Quoted 5 lines

The first entry is the non-weak one.
The second entry is the remaining part of the weak (dropped) function. It seems only the start address was changed to the start address of the non-weak function, while the end address has been kept unchanged. [0x1011fc4c-0x10167cf4] would mean, that the weak function would be 295KB in size...?

If you run readelf on built-in.o, is the problem present there?

Does this happen with both 32 and 64 bit kernels?

Need to figure out why the start address of the weak function is the same as the non-weak function. My initial guess is that RELOC_FOR_GLOBAL_SYMBOL is messing up.

Quoted 4 lines · John David Anglin

Quoted 4 lines

Quoted 4 lines

and whether it follows inb in its .o. From my understanding of things, it shouldn't be necessary to remove the unwind info for unused weak symbols if they aren't garbage collected. A simple testcase would be helpful.

Quoted 5 lines · Helge Deller

Quoted 4 lines · John David Anglin

Quoted 4 lines

Quoted 4 lines

and whether it follows inb in its .o. From my understanding of things, it shouldn't be necessary to remove the unwind info for unused weak symbols if they aren't garbage collected. A simple testcase would be helpful.

Attached is a testcase (t1.c and t2.c):
t1.c contains weak function f().
t2.c contains non-weak function f().

hppa-linux-gcc -c t1.c t2.c
hppa-linux-ld -r -o all.o t1.o t2.o

hppa-linux-readelf -s t1.o gives:
Symbol table '.symtab' contains 10 entries:

   Num:    Value  Size Type    Bind   Vis      Ndx Name
     6: 00000000    28 FUNC    GLOBAL DEFAULT    1 a
     7: 0000001c    32 FUNC    WEAK   DEFAULT    1 f
     8: 0000003c    28 FUNC    GLOBAL DEFAULT    1 z
     9: 00000058    80 FUNC    GLOBAL DEFAULT    1 main

hppa-linux-readelf -s t2.o gives:
Symbol table '.symtab' contains 7 entries:
Num: Value Size Type Bind Vis Ndx Name

     6: 00000000    32 FUNC    GLOBAL DEFAULT    1 f

hppa-linux-readelf -s all.o gives:
Symbol table '.symtab' contains 11 entries:

   Num:    Value  Size Type    Bind   Vis      Ndx Name
     7: 00000000    32 FUNC    GLOBAL DEFAULT    3 f
     8: 0000003c    28 FUNC    GLOBAL DEFAULT    1 z
     9: 00000058    80 FUNC    GLOBAL DEFAULT    1 main
    10: 00000000    28 FUNC    GLOBAL DEFAULT    1 a

hppa-linux-readelf -u all.o gives:
Unwind section '.PARISC.unwind' at offset 0xfc contains 5 entries:
<a>: [0x0-0x30]
Entry_GR=1 Save_SP Total_frame_size=8
<f>: [0x0-0x70]
Entry_GR=1 Save_SP Total_frame_size=8
<z>: [0x3c-0xa8]
Entry_GR=1 Save_SP Total_frame_size=8
<main>: [0x58-0x148]
Entry_GR=2 Save_SP Save_RP Total_frame_size=8
<f>: [0x0-0x38]
Entry_GR=1 Save_SP Total_frame_size=8

Function f() is listed twice and with different lengths. Sadly in this example f() starts at offset 0. I think if you tweak the example, try to see what you get when the start of f() is not zero.

Quoted 49 lines · Helge Deller

Quoted 5 lines · Helge Deller

Quoted 4 lines · John David Anglin

Quoted 4 lines

Quoted 4 lines

and whether it follows inb in its .o. From my understanding of things, it shouldn't be necessary to remove the unwind info for unused weak symbols if they aren't garbage collected. A simple testcase would be helpful.

Attached is a testcase (t1.c and t2.c):
t1.c contains weak function f().
t2.c contains non-weak function f().

hppa-linux-gcc -c t1.c t2.c
hppa-linux-ld -r -o all.o t1.o t2.o

hppa-linux-readelf -s t1.o gives:
Symbol table '.symtab' contains 10 entries:

   Num:    Value  Size Type    Bind   Vis      Ndx Name
     6: 00000000    28 FUNC    GLOBAL DEFAULT    1 a
     7: 0000001c    32 FUNC    WEAK   DEFAULT    1 f
     8: 0000003c    28 FUNC    GLOBAL DEFAULT    1 z
     9: 00000058    80 FUNC    GLOBAL DEFAULT    1 main

hppa-linux-readelf -s t2.o gives:
Symbol table '.symtab' contains 7 entries:
Num: Value Size Type Bind Vis Ndx Name

     6: 00000000    32 FUNC    GLOBAL DEFAULT    1 f

hppa-linux-readelf -s all.o gives:
Symbol table '.symtab' contains 11 entries:

   Num:    Value  Size Type    Bind   Vis      Ndx Name
     7: 00000000    32 FUNC    GLOBAL DEFAULT    3 f
     8: 0000003c    28 FUNC    GLOBAL DEFAULT    1 z
     9: 00000058    80 FUNC    GLOBAL DEFAULT    1 main
    10: 00000000    28 FUNC    GLOBAL DEFAULT    1 a

hppa-linux-readelf -u all.o gives:
Unwind section '.PARISC.unwind' at offset 0xfc contains 5 entries:
<a>: [0x0-0x30]
Entry_GR=1 Save_SP Total_frame_size=8
<f>: [0x0-0x70]
Entry_GR=1 Save_SP Total_frame_size=8
<z>: [0x3c-0xa8]
Entry_GR=1 Save_SP Total_frame_size=8
<main>: [0x58-0x148]
Entry_GR=2 Save_SP Save_RP Total_frame_size=8
<f>: [0x0-0x38]
Entry_GR=1 Save_SP Total_frame_size=8

Quoted 3 lines

Function f() is listed twice and with different lengths. Sadly in this example f() starts at offset 0. I think if you tweak the example, try to see what you get when the start of f() is not zero.

Looking at all.o with objdump, the code for <f> is still present. I would say the regions should be:

<a>: [0x0-0x18]
<f>: [0x1c-0x38]
<z>: [0x3c-0x54]
<main>: [0x58-0xa4]
<f>: [0x0-0x1c]

There are two .text sections, so the offsets for the second <f> are relative to the start of this section.

My compilation probably differs a bit from yours but I see the same readelf -u output. Suspect readelf is broken. The end values seem off by a factor 2.

Quoted 5 lines · Helge Deller

Quoted 4 lines · John David Anglin

Quoted 4 lines

Quoted 4 lines

and whether it follows inb in its .o. From my understanding of things, it shouldn't be necessary to remove the unwind info for unused weak symbols if they aren't garbage collected. A simple testcase would be helpful.

Attached is a testcase (t1.c and t2.c):
t1.c contains weak function f().
t2.c contains non-weak function f().

hppa-linux-gcc -c t1.c t2.c
hppa-linux-ld -r -o all.o t1.o t2.o

Quoted 11 lines · Helge Deller

Quoted 5 lines · Helge Deller

Quoted 4 lines · John David Anglin

Quoted 4 lines

Quoted 4 lines

and whether it follows inb in its .o. From my understanding of things, it shouldn't be necessary to remove the unwind info for unused weak symbols if they aren't garbage collected. A simple testcase would be helpful.

Attached is a testcase (t1.c and t2.c):
t1.c contains weak function f().
t2.c contains non-weak function f().

Quoted 2 lines

hppa-linux-gcc -c t1.c t2.c
hppa-linux-ld -r -o all.o t1.o t2.o

Attached is a patch for gas that seems to produce reasonable unwind info for the testcase when all.o is linked into a final executable. I think it will fix the linux unwind problem.

Still need to look at readelf -u on .o's.

Quoted 1 line

Still need to look at readelf -u on .o's.

Yes, I see this too. readelf -u doesn't print any longer the second line of:
<arch_mod_section_prepend>: [0x1011fc4c-0x1011fc74]
Entry_GR=1 Save_SP Total_frame_size=8

Quoted 5 lines

Quoted 1 line

Still need to look at readelf -u on .o's.

Yes, I see this too. readelf -u doesn't print any longer the second line of:
<arch_mod_section_prepend>: [0x1011fc4c-0x1011fc74]
Entry_GR=1 Save_SP Total_frame_size=8

Actually, the entries are still there. For weak functions that are not used, they appear as "<foo+x>:" where foo is the closest preceding symbol before the unused weak function. In practice, the code shouldn't be entered, so the lack of an associated function symbol shouldn't matter.

I have a fix for the readelf -u bug. See below.

Quoted 12 lines

Quoted 5 lines

Quoted 1 line

Still need to look at readelf -u on .o's.

Yes, I see this too. readelf -u doesn't print any longer the second line of:
<arch_mod_section_prepend>: [0x1011fc4c-0x1011fc74]
Entry_GR=1 Save_SP Total_frame_size=8

Actually, the entries are still there. For weak functions that are not used, they appear as "<foo+x>:" where foo is the closest preceding symbol before the unused weak function. In practice, the code shouldn't be entered, so the lack of an associated function symbol shouldn't matter.

Quoted 1 line

I have a fix for the readelf -u bug. See below.

Sorry, but I don't fully understand...

With your patch to gas, the unwind info for the weak functions are completely gone in the final executable (which is good).

But "readelf -u" doesn't show the "Entry_GR=..." lines for _any_ function any longer (in the final executable). I really mean _all_ functions, weak, non-weak, standard, ...

The patch you attached only changes the start and end addresses of the symbols...?

Quoted 14 lines · John David Anglin

Quoted 12 lines

Quoted 5 lines

Quoted 1 line

Still need to look at readelf -u on .o's.

Yes, I see this too. readelf -u doesn't print any longer the second line of:
<arch_mod_section_prepend>: [0x1011fc4c-0x1011fc74]
Entry_GR=1 Save_SP Total_frame_size=8

Actually, the entries are still there. For weak functions that are not used, they appear as "<foo+x>:" where foo is the closest preceding symbol before the unused weak function. In practice, the code shouldn't be entered, so the lack of an associated function symbol shouldn't matter.

Quoted 5 lines

Quoted 1 line

I have a fix for the readelf -u bug. See below.

Sorry, but I don't fully understand...

With your patch to gas, the unwind info for the weak functions are completely gone in the final executable (which is good).

I don't think I changed this. In the testcase that you sent, I see

<a>: [0x104a0-0x104b8]
Entry_GR=1 Save_SP Total_frame_size=8
<a+1c>: [0x104bc-0x104d8]
Entry_GR=1 Save_SP Total_frame_size=8
<z>: [0x104dc-0x104f4]
Entry_GR=1 Save_SP Total_frame_size=8
<main>: [0x104f8-0x10544]
Entry_GR=1 Save_SP Total_frame_size=8
<f>: [0x10548-0x10564]
Entry_GR=2 Save_SP Save_RP Total_frame_size=8

The weak version of f is the<a+1c> entry.

Quoted 2 lines

But "readelf -u" doesn't show the "Entry_GR=..." lines for _any_ function any longer (in the final executable). I really mean _all_ functions, weak, non-weak, standard, ...

That's not what I'm seeing. The old and new versions seem similar (i.e., with respect to missing "Entry_GR=..."). Didn't touch the code that extracts and prints this stuff.

The lack of "Entry_GR=..." stuff for an entry probably indicates that there isn't a .CALLINFO directive for the function in question. This could happen if the function is handcoded assembler. However, it is legitimate for a function not to have any of the unwind stuff set (e.g., function doesn't save any registers or allocate a frame). I think GCC gets this right but it never has been looked at with a magnifying glass.

Quoted 1 line

The patch you attached only changes the start and end addresses of the symbols...?

Yes, and only for .o's. In final executables, there are no relocations in the .PARISC.unwind section. So, the values printed should be unaffected.

In the .o case, the code was in effect doing the SEGREL relocation twice leading to offsets twice as big as they should have been. It still runs through the relocations if present, but uses the value from doing the relocation if one is present. Compare what you see now with objdump -d and readelf -u on a .o.

Quoted 52 lines

Quoted 14 lines · John David Anglin

Quoted 12 lines

Quoted 5 lines

Quoted 1 line

Still need to look at readelf -u on .o's.

Yes, I see this too. readelf -u doesn't print any longer the second line of:
<arch_mod_section_prepend>: [0x1011fc4c-0x1011fc74]
Entry_GR=1 Save_SP Total_frame_size=8

Actually, the entries are still there. For weak functions that are not used, they appear as "<foo+x>:" where foo is the closest preceding symbol before the unused weak function. In practice, the code shouldn't be entered, so the lack of an associated function symbol shouldn't matter.

Quoted 5 lines

Quoted 1 line

I have a fix for the readelf -u bug. See below.

Sorry, but I don't fully understand...

With your patch to gas, the unwind info for the weak functions are completely gone in the final executable (which is good).

I don't think I changed this. In the testcase that you sent, I see

<a>: [0x104a0-0x104b8]
Entry_GR=1 Save_SP Total_frame_size=8
<a+1c>: [0x104bc-0x104d8]
Entry_GR=1 Save_SP Total_frame_size=8
<z>: [0x104dc-0x104f4]
Entry_GR=1 Save_SP Total_frame_size=8
<main>: [0x104f8-0x10544]
Entry_GR=1 Save_SP Total_frame_size=8
<f>: [0x10548-0x10564]
Entry_GR=2 Save_SP Save_RP Total_frame_size=8

The weak version of f is the<a+1c> entry.

Quoted 2 lines

But "readelf -u" doesn't show the "Entry_GR=..." lines for _any_ function any longer (in the final executable). I really mean _all_ functions, weak, non-weak, standard, ...

That's not what I'm seeing. The old and new versions seem similar (i.e., with respect to missing "Entry_GR=..."). Didn't touch the code that extracts and prints this stuff.

The lack of "Entry_GR=..." stuff for an entry probably indicates that there isn't a .CALLINFO directive for the function in question. This could happen if the function is handcoded assembler. However, it is legitimate for a function not to have any of the unwind stuff set (e.g., function doesn't save any registers or allocate a frame). I think GCC gets this right but it never has been looked at with a magnifying glass.

Quoted 1 line

The patch you attached only changes the start and end addresses of the symbols...?

Quoted 8 lines

Yes, and only for .o's. In final executables, there are no relocations in the .PARISC.unwind section. So, the values printed should be unaffected.

In the .o case, the code was in effect doing the SEGREL relocation twice leading to offsets twice as big as they should have been. It still runs through the relocations if present, but uses the value from doing the relocation if one is present. Compare what you see now with objdump -d and readelf -u on a .o.

Hi Dave,

you are probably correct. Nevertheless, your patches seem to have side-effects. With your patches, the testcase from me now looks correct (as you explained above).

But when compiling the kernel something is still wrong.
I tried it a few times now, recompiled binutils, the kernel and so on. I'm not 100% sure that I did everything correct, but I still think there is something wrong...

This is what I get with your patches:
/opt/cross-hppa-4.4.neu/bin/hppa-linux-readelf -u vmlinux
...
--------------------------------------------------------------
<arch_mod_section_prepend>: [0x1011fc00-0x1011fc28]

<module_arch_cleanup>: [0x1011fc2c-0x1011fc6c]

<apply_relocate>: [0x1011fc70-0x1011fca4]
Entry_GR=2 Save_SP Save_RP Total_frame_size=8
<module_finalize>: [0x1011fca8-0x1011fe8c]
Entry_GR=1 Save_SP Save_RP Total_frame_size=24
<get_stub>: [0x1011fe90-0x1011ffb4]

<apply_relocate_add>: [0x1011ffb8-0x101205b8]

<module_frob_arch_sections>: [0x101205bc-0x10120758]

<module_free>: [0x1012075c-0x101207a4]

<module_alloc>: [0x101207a8-0x101207e4]

<dump_trace>: [0x101207e8-0x10120904]

<save_stack_trace_tsk>: [0x10120908-0x10120958]

<save_stack_trace>: [0x1012095c-0x101209b8]

<sgl_frnd>: [0x101209bc-0x10120b8c]
Entry_GR=1 Save_SP Total_frame_size=8
<dbl_frnd>: [0x10120b90-0x10120e74]
Entry_GR=1 Save_SP Total_frame_size=8
--------------------------------------------------------------

As you can see, a few Entry_GR lines are missing.
Even for C-functions for which Entry_GR was there with the old toolchain. I sadly can not see any reason or logic in why specific functions don't have their Entry_GR lines any longer.

Here for reference what I see with the old (unpatched) toolchain:
(as you can see arch_mod_section_prepend() is there twice again -> because of the old unpatched toolchain)

--------------------------------------------------------------
/opt/cross-hppa-4.4/bin/hppa-linux-readelf -u vmlinux
...
<arch_mod_section_prepend>: [0x1011fc00-0x1011fc28]
Entry_GR=1 Save_SP Total_frame_size=8
<arch_mod_section_prepend>: [0x1011fc00-0x10167dec]
Entry_GR=1 Save_SP Total_frame_size=8
<module_arch_cleanup>: [0x1011fc2c-0x1011fc6c]
Entry_GR=2 Save_SP Save_RP Total_frame_size=8
<apply_relocate>: [0x1011fc70-0x1011fca4]
Entry_GR=1 Save_SP Save_RP Total_frame_size=8
<module_finalize>: [0x1011fca8-0x1011fe8c]
Entry_GR=11 Save_SP Save_RP Total_frame_size=16
<get_stub>: [0x1011fe90-0x1011ffb4]
Entry_GR=1 Save_SP Total_frame_size=8
<apply_relocate_add>: [0x1011ffb8-0x101205b8]
Entry_GR=16 Save_SP Save_RP Total_frame_size=24
<module_frob_arch_sections>: [0x101205bc-0x10120758]
Entry_GR=10 Save_SP Save_RP Total_frame_size=16
<module_free>: [0x1012075c-0x101207a4]
Entry_GR=3 Save_SP Save_RP Total_frame_size=8
<module_alloc>: [0x101207a8-0x101207e4]
Entry_GR=1 Save_SP Save_RP Total_frame_size=8
<dump_trace>: [0x101207e8-0x10120904]
Entry_GR=7 Save_SP Save_RP Total_frame_size=80
<save_stack_trace_tsk>: [0x10120908-0x10120958]
Entry_GR=2 Save_SP Save_RP Total_frame_size=8
<save_stack_trace_tsk>: [0x10120908-0x1015c7ac]

Entry_GR=2 Save_SP Save_RP Total_frame_size=8
<save_stack_trace>: [0x1012095c-0x101209b8]
Entry_GR=2 Save_SP Save_RP Total_frame_size=8
<sgl_frnd>: [0x101209bc-0x10120b8c]
Entry_GR=1 Save_SP Total_frame_size=8
<dbl_frnd>: [0x10120b90-0x10120e74]
Entry_GR=1 Save_SP Total_frame_size=8
--------------------------------------------------------------

There seems to a problem with readelf -u on 32-bit kernels. The address ranges are wrong. The segment offset doesn't seem to be being handled correctly.

Quoted 3 lines · John David Anglin

There seems to a problem with readelf -u on 32-bit kernels. The address ranges are wrong. The segment offset doesn't seem to be being handled correctly.

readelf -u uses the base address of the segment containing the .PARISC.unwind section. On 64-bit kernels, the .PARISC.unwind section is in the same segment as the .text section, so the addresses print are correct. However, on recent 32-bit kernels, this is not the case:

Section to Segment mapping:
Segment Sections...

    00     .text .rodata .init.rodata __bug_table .pci_fixup .builtin_fw __ksymtab __ksymtab_gpl __kcrctab __kcrctab_gpl __ksymtab_strings __param
    01     __ex_table .PARISC.unwind .data.read_mostly .data .got .data.vmpages .bss
    02     .data.init_task .init.text .init.data .init.setup .initcall.init .con_initcall.init .exit.text
    03

.PARISC.unwind is readonly, so it shouldn't be in a W segment.

But when compiling the kernel something is still wrong.
I tried it a few times now, recompiled binutils, the kernel and so on. I'm not 100% sure that I did everything correct, but I still think there is something wrong...

Quoted 4 lines · Helge Deller

But when compiling the kernel something is still wrong.
I tried it a few times now, recompiled binutils, the kernel and so on. I'm not 100% sure that I did everything correct, but I still think there is something wrong...

I believe that you are correct but I think we have another unrelated bug. It appears that the unwind data is getting messed up in the link of built-in.o.

Quoted 7 lines · John David Anglin

Quoted 4 lines · Helge Deller

But when compiling the kernel something is still wrong.
I tried it a few times now, recompiled binutils, the kernel and so on. I'm not 100% sure that I did everything correct, but I still think there is something wrong...

Quoted 3 lines

I believe that you are correct but I think we have another unrelated bug. It appears that the unwind data is getting messed up in the link of built-in.o.

It seems the unwind data is being sorted by ld -r.

Quoted 9 lines · John David Anglin

Quoted 7 lines · John David Anglin

Quoted 4 lines · Helge Deller

But when compiling the kernel something is still wrong.
I tried it a few times now, recompiled binutils, the kernel and so on. I'm not 100% sure that I did everything correct, but I still think there is something wrong...

Quoted 5 lines

Quoted 3 lines

I believe that you are correct but I think we have another unrelated bug. It appears that the unwind data is getting messed up in the link of built-in.o.

It seems the unwind data is being sorted by ld -r.

Does the attached patch fix the problem?

Quoted 15 lines · John David Anglin

Quoted 14 lines · John David Anglin

Quoted 11 lines · Helge Deller

Quoted 5 lines

But when compiling the kernel something is still wrong.
I tried it a few times now, recompiled binutils, the kernel and so on. I'm not 100% sure that I did everything correct, but I still think there is something wrong...

I believe that you are correct but I think we have another unrelated bug. It appears that the unwind data is getting messed up in the link of built-in.o.

Quoted 3 lines

Quoted 1 line

It seems the unwind data is being sorted by ld -r.

Does the attached patch fix the problem?

Yes, it does :-)

Quoted 18 lines · John David Anglin

Quoted 17 lines · John David Anglin

Quoted 15 lines · John David Anglin

Quoted 11 lines · Helge Deller

Quoted 5 lines

But when compiling the kernel something is still wrong.
I tried it a few times now, recompiled binutils, the kernel and so on. I'm not 100% sure that I did everything correct, but I still think there is something wrong...

I believe that you are correct but I think we have another unrelated bug. It appears that the unwind data is getting messed up in the link of built-in.o.

It seems the unwind data is being sorted by ld -r.

Quoted 3 lines

Quoted 1 line

Does the attached patch fix the problem?

Yes, it does :-)

Thanks for testing! I have built a 32-bit kernel with the revised assembler and linker, and the readelf -u output looked good to me. I will install the remaining two changes tonight (the assembler change was installed a couple of days ago).

I haven't had a chance to boot the new kernel (waiting for a GCC build to complete). There is a USB issue that appeared on my c3750 with 2.6.30.9 (2.6.30.8 is ok) and 2.6.31.5. Here is the dmesg output from 2.6.31.5:

Linux version 2.6.31.5 (root@hiauly6) (gcc version 4.3.4 (Debian 4.3.4-5) ) #6 Sun Nov 1 14:20:36 EST 2009
unwind_init: start = 0x10524000, end = 0x10559a60, entries = 13734
FP[0] enabled: Rev 1 Model 19
The 32-bit Kernel has started...
...
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
...
usbmon: debugfs is not available
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
ohci_hcd 0000:00:0e.2: OHCI Host Controller
ohci_hcd 0000:00:0e.2: new USB bus registered, assigned bus number 1
ohci_hcd 0000:00:0e.2: irq 1, io mem 0xf4007000
...
usb usb1: configuration #1 chosen from 1 choice
sdb5
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 3 ports detected
...
usb 1-1: new low speed USB device using ohci_hcd and address 2
EXT3-fs: mounted filesystem with writeback data mode.
VFS: Mounted root (ext3 filesystem) readonly on device 8:19.
Freeing unused kernel memory: 124k freed
usb 1-1: configuration #1 chosen from 1 choice
usb 1-2: new low speed USB device using ohci_hcd and address 3
usb 1-2: configuration #1 chosen from 1 choice
sysfs: cannot create duplicate filename '/module/usbhid/sections/.text'

------------[ cut here ]------------
Badness at fs/sysfs/dir.c:487

YZrvWESTHLNXBCVMcbcbcbcbOGFRQPDI
PSW: 00000000000001000000000000001111 Not tainted

r00-03  0004000f 10576200 101cb5dc 4edc9000
r04-07  ffffffef 4fbb4288 4edc8a88 00074844
r08-11  4f8db000 00000000 00063000 00000064
r12-15  00000124 00067bbb 104c36a0 00000058
r16-19  00068fc8 00000062 4fbb4148 10576244
r20-23  00002b68 4f16bca0 0027f808 ffffffff
r24-27  ffffffff 00002b68 10576240 1055c2d0
r28-31  0000004e 00000000 4fbb4340 102a0244

sr00-03 00000000 00000000 00000000 00000010
sr04-07 00000000 00000000 00000000 00000000

IASQ: 00000000 00000000 IAOQ: 101cb5dc 101cb5e0
IIR: 03ffe01f ISR: 00000000 IOR: 00000000
CPU: 0 CR30: 4fbb4000 CR31: ffffffff
ORIG_R28: 00000001
IAOQ[0]: sysfs_add_one+0x8c/0xb4
IAOQ[1]: sysfs_add_one+0x90/0xb4
RP(r2): sysfs_add_one+0x8c/0xb4

Backtrace:
  [<101cadb0>] sysfs_add_file_mode+0x58/0xb4
  [<101cd4b4>] internal_create_group+0x118/0x1dc
  [<10158434>] load_module+0xf14/0x1214

input: Logitech N48 as /devices/parisc/10/10:0/pci0000:00/0000:00:0e.2/usb1/1-1/1-1:1.0/input/input0 generic-usb 0003:046D:C001.0001: input: USB HID v1.00 Mouse [Logitech N48] on usb-0000:00:0e.2-1/input0
input: SILITEK USB Keyboard and Mouse as /devices/parisc/10/10:0/pci0000:00/0000:00:0e.2/usb1/1-2/1-2:1.0/input/input1 generic-usb 0003:047B:0002.0002: input: USB HID v1.00 Keyboard [SILITEK USB Keyboard and Mouse] on usb-0000:00:0e.2-2/input0
input: SILITEK USB Keyboard and Mouse as /devices/parisc/10/10:0/pci0000:00/0000:00:0e.2/usb1/1-2/1-2:1.1/input/input2 generic-usb 0003:047B:0002.0003: input: USB HID v1.00 Mouse [SILITEK USB Keyboard and Mouse] on usb-0000:00:0e.2-2/input1
usbcore: registered new interface driver usbhid
usbhid: v2.6:USB HID core driver
...

As you can see, there is a USB mouse and keyboard. Want to see backtrace with new assembler and linker. However, at the least the first part is consistent with the sysfs warning.

I "upgraded" this system to testing last weekend. The keyboard is broken under X. In gdm/kdm, one has to press a key twice. The Xorg.0.log seems ok:

(**) Option "CoreKeyboard"
(**) Generic Keyboard: always reports core events
(**) Option "Protocol" "standard"
(**) Generic Keyboard: Protocol: standard
(**) Option "AutoRepeat" "500 30"
(**) Option "XkbRules" "xorg"
(**) Generic Keyboard: XkbRules: "xorg"
(**) Option "XkbModel" "pc104"
(**) Generic Keyboard: XkbModel: "pc104"
(**) Option "XkbLayout" "us"
(**) Generic Keyboard: XkbLayout: "us"
(**) Option "CustomKeycodes" "off"
(**) Generic Keyboard: CustomKeycodes disabled
(II) XINPUT: Adding extended input device "Generic Keyboard" (type: KEYBOARD)

Mouse works ok.
Thoughts?

Quoted 3 lines · John David Anglin

sysfs: cannot create duplicate filename '/module/usbhid/sections/.text'

------------[ cut here ]------------
Badness at fs/sysfs/dir.c:487

Yes, this is known.
See here:
http://patchwork.kernel.org/patch/42190/
Would be nice if this could get fixed.

Quoted 9 lines · Helge Deller

Quoted 3 lines · John David Anglin

sysfs: cannot create duplicate filename '/module/usbhid/sections/.text'

------------[ cut here ]------------
Badness at fs/sysfs/dir.c:487

Yes, this is known.
See here:
http://patchwork.kernel.org/patch/42190/
Would be nice if this could get fixed.

Actually, it is fixed; the patch is upstream as commit 1b364bf438cf337a3818aee77d68c0713f3e1fc4

The patch was designed to fix the panic that occurred. This warn on is coming out of the sysfs code when we try to register a duplicate section name. The panic that followed is what the patch fixed ... but we still get left with the warn on.

Quoted 4 lines

Thanks for testing! I have built a 32-bit kernel with the revised assembler and linker, and the readelf -u output looked good to me. I will install the remaining two changes tonight (the assembler change was installed a couple of days ago).

I did tested the 64bit version now as well.
It looks good too.

sadly I answered too early....
With 64bit kernel I get those unwind warnings again with nearly all kernel modules now:
WARNING: Out of order unwind entry! 0000000000d4ad40 and 0000000000d4ad50

Quoted 4 lines · Helge Deller

sadly I answered too early....
With 64bit kernel I get those unwind warnings again with nearly all kernel modules now:
WARNING: Out of order unwind entry! 0000000000d4ad40 and 0000000000d4ad50

What's readelf say?

Unwind section '.PARISC.unwind' at offset 0x11c contains 3 entries:

<libcrc32c_mod_fini>: [0x0-0x2c]
Entry_GR=1 Save_SP Save_RP Total_frame_size=8
<libcrc32c_mod_init>: [0x0-0x44]
Entry_GR=1 Save_SP Save_RP Total_frame_size=8
<crc32c>: [0x0-0x6c]
Entry_GR=2 Save_SP Save_RP Total_frame_size=8

Quoted 8 lines

Unwind section '.PARISC.unwind' at offset 0x11c contains 3 entries:

<libcrc32c_mod_fini>: [0x0-0x2c]
Entry_GR=1 Save_SP Save_RP Total_frame_size=8
<libcrc32c_mod_init>: [0x0-0x44]
Entry_GR=1 Save_SP Save_RP Total_frame_size=8
<crc32c>: [0x0-0x6c]
Entry_GR=2 Save_SP Save_RP Total_frame_size=8

What do you see for '.rela.PARISC.unwind'?

I suspect that this is a problem in handling the R_PARISC_SEGREL32 relocations is the kernel loader. The above would be fine if libcrc32c_mod_fini and libcRc32c_mod_init are in different sections (.exit.text and .inti.text).

Quoted 9 lines

Quoted 8 lines

Unwind section '.PARISC.unwind' at offset 0x11c contains 3 entries:

<libcrc32c_mod_fini>: [0x0-0x2c]
Entry_GR=1 Save_SP Save_RP Total_frame_size=8
<libcrc32c_mod_init>: [0x0-0x44]
Entry_GR=1 Save_SP Save_RP Total_frame_size=8
<crc32c>: [0x0-0x6c]
Entry_GR=2 Save_SP Save_RP Total_frame_size=8

Quoted 6 lines

What do you see for '.rela.PARISC.unwind'?

I suspect that this is a problem in handling the R_PARISC_SEGREL32 relocations is the kernel loader. The above would be fine if libcrc32c_mod_fini and libcRc32c_mod_init are in different sections (.exit.text and .inti.text).

Relocation section '.rela.PARISC.unwind' at offset 0x9b8 contains 6 entries:

  Offset     Info    Type            Sym.Value  Sym. Name + Addend
00000000  00000231 R_PARISC_SEGREL32 00000000   .exit.text + 0
00000004  00000231 R_PARISC_SEGREL32 00000000   .exit.text + 2c
00000010  00000331 R_PARISC_SEGREL32 00000000   .init.text + 0
00000014  00000331 R_PARISC_SEGREL32 00000000   .init.text + 44
00000020  00000431 R_PARISC_SEGREL32 00000000   .text.crc32c + 0
00000024  00000431 R_PARISC_SEGREL32 00000000   .text.crc32c + 6c

Quoted 17 lines · John David Anglin

Quoted 9 lines

Quoted 8 lines

Unwind section '.PARISC.unwind' at offset 0x11c contains 3 entries:

<libcrc32c_mod_fini>: [0x0-0x2c]
Entry_GR=1 Save_SP Save_RP Total_frame_size=8
<libcrc32c_mod_init>: [0x0-0x44]
Entry_GR=1 Save_SP Save_RP Total_frame_size=8
<crc32c>: [0x0-0x6c]
Entry_GR=2 Save_SP Save_RP Total_frame_size=8

Quoted 6 lines

What do you see for '.rela.PARISC.unwind'?

I suspect that this is a problem in handling the R_PARISC_SEGREL32 relocations is the kernel loader. The above would be fine if libcrc32c_mod_fini and libcRc32c_mod_init are in different sections (.exit.text and .inti.text).

Quoted 8 lines

Relocation section '.rela.PARISC.unwind' at offset 0x9b8 contains 6 entries:
Offset Info Type Sym.Value Sym. Name + Addend

00000000  00000231 R_PARISC_SEGREL32 00000000   .exit.text + 0
00000004  00000231 R_PARISC_SEGREL32 00000000   .exit.text + 2c
00000010  00000331 R_PARISC_SEGREL32 00000000   .init.text + 0
00000014  00000331 R_PARISC_SEGREL32 00000000   .init.text + 44
00000020  00000431 R_PARISC_SEGREL32 00000000   .text.crc32c + 0
00000024  00000431 R_PARISC_SEGREL32 00000000   .text.crc32c + 6c

That's exactly what I expected. There's no overlap. The module loader needs to sort the unwind entries after doing the relocations.

Quoted 11 lines

Quoted 8 lines

Relocation section '.rela.PARISC.unwind' at offset 0x9b8 contains 6 entries:
Offset Info Type Sym.Value Sym. Name + Addend

00000000  00000231 R_PARISC_SEGREL32 00000000   .exit.text + 0
00000004  00000231 R_PARISC_SEGREL32 00000000   .exit.text + 2c
00000010  00000331 R_PARISC_SEGREL32 00000000   .init.text + 0
00000014  00000331 R_PARISC_SEGREL32 00000000   .init.text + 44
00000020  00000431 R_PARISC_SEGREL32 00000000   .text.crc32c + 0
00000024  00000431 R_PARISC_SEGREL32 00000000   .text.crc32c + 6c

That's exactly what I expected. There's no overlap. The module loader needs to sort the unwind entries after doing the relocations.

It's already sorting the entries, but creating the relocation is currently a hack.

See top of arch/parisc/kernel/module.c:
* - SEGREL32 handling
* We are not doing SEGREL32 handling correctly. According to the ABI, we
* should do a value offset, like this:
* if (in_init(me, (void *)val))
* val -= (uint32_t)me->module_init;
* else
* val -= (uint32_t)me->module_core;
* However, SEGREL32 is used only for PARISC unwind entries, and we want
* those entries to have an absolute address, and not just an offset.
*

  *      The unwind table mechanism has the ability to specify an offset for
  *      the unwind table; however, because we split off the init functions into
  *      a different piece of memory, it is not possible to do this using a
  *      single offset. Instead, we use the above hack for now.

If we change this, don't we require a binutils update then at the same time from the users?

Quoted 11 lines

Quoted 11 lines

Quoted 8 lines

Relocation section '.rela.PARISC.unwind' at offset 0x9b8 contains 6 entries:
Offset Info Type Sym.Value Sym. Name + Addend

00000000  00000231 R_PARISC_SEGREL32 00000000   .exit.text + 0
00000004  00000231 R_PARISC_SEGREL32 00000000   .exit.text + 2c
00000010  00000331 R_PARISC_SEGREL32 00000000   .init.text + 0
00000014  00000331 R_PARISC_SEGREL32 00000000   .init.text + 44
00000020  00000431 R_PARISC_SEGREL32 00000000   .text.crc32c + 0
00000024  00000431 R_PARISC_SEGREL32 00000000   .text.crc32c + 6c

That's exactly what I expected. There's no overlap. The module loader needs to sort the unwind entries after doing the relocations.

Quoted 1 line

It's already sorting the entries,

Quoted 1 line

If we change this, don't we require a binutils update then at the same time from the users?

Don't think fixing the offsets requires a binutils update. However, the change to using section offsets may have made the problem worse. The binutils update only corrected the issue with overloaded symbols.

Kyle, I see you applied my original patch into your "next" tree already:
http://git.kernel.org/?p=linux/kernel/git/kyle/parisc-2.6.git;a=commit;h=b8d46862d910c3be7c28c627204da968c174aa9a

Do you want me to send you a new patch against your "next" tree instead?