Skip to contentSkip to conversation

x86_64: Add "-m elf_i386" when linking i386 object files.

RFC PATCH Tri Vo 3 participants 4 messages Dec 12, 2018 – Jan 8, 2019 4 messages not in the archive

lore
Share & tools
lore

Tri Vo

author
#3 lore raw
In reply to a message not in the archive

Adding appropriate people, lists.

Quoted 117 lines · Nick Desaulniers
Quoted 51 lines · Nick Desaulniers
Quoted 20 lines · Tri Vo

From: George Rimar <[email protected]>

Linux kernel uses OUTPUT_FORMAT in it's linker scripts. Most of the time -m option is passed to the linker with correct architecture, but sometimes (at least for x86_64) the -m option contradicts OUTPUT_FORMAT directive. Specifically, arch/x86/boot and arch/x86/realmode/rm modules have i386 object files, but are linked with -m elf_x86_64 linker flag when building for x86_64.

BFD and Gold linkers are OK with this, but lld fails:
ld.lld: error: arch/x86/realmode/rm/header.o is incompatible with elf_x86_64

Suggested fix: just add correct -m after incorrect one (it overrides it), so the linker invocation looks like this: ld -m elf_x86_64 -z max-page-size=0x200000 -m elf_i386 --emit-relocs -T realmode.lds header.o trampoline_64.o stack.o reboot.o -o realmode.elf (it will also work with GNU ld, because it supports OUTPUT_FORMAT and just ignores -m options if this directive is in the linker script).

Quoted 6 lines

Tested by building x86_64 kernel with GNU gcc/ld toolchain and booting it in QEMU.

This fixes the following linkage error I observe when linking an x86 kernel with LLD:

```
  LD      arch/x86/realmode/rm/realmode.elf
ld.lld: error: arch/x86/realmode/rm/header.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/trampoline_64.o is incompatible
with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/stack.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/reboot.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/wakeup_asm.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/wakemain.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/video-mode.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/copy.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/bioscall.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/regs.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/video-vga.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/video-vesa.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/video-bios.o is incompatible with elf_x86_64
arch/x86/realmode/rm/Makefile:55: recipe for target
'arch/x86/realmode/rm/realmode.elf' failed
```
Quoted 6 lines

Looks like we still have a few other (unrelated) issues to track down with LLD, but this gets us one step closer.

Thanks for sending this, Tri!

Just some additional thoughts on this:

The kernel is adding -m elf_x86_64 to the linker command line parameters, but then relying on binutils tie-breaking-behavior by specifying a different architecture via linker script. So it's ambiguous which arch the kernel is trying to link for, and the kernel gets lucky/relies on the way ld.bfd happens to resolve this ambiguity without warning. An alternative fix may be to just filter out/remove the -m elf_x86_64 for this target's linker flags.

Quoted 31 lines
Quoted 31 lines
---
 arch/x86/boot/Makefile        | 2 +-
 arch/x86/realmode/rm/Makefile | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
arch/x86/boot/Makefile +1 -1
index 9b5adae9cc40..e2839b5c246c 100644
@@ -100,7 +100,7 @@ $(obj)/zoffset.h: $(obj)/compressed/vmlinux FORCE
 AFLAGS_header.o += -I$(objtree)/$(obj)
 $(obj)/header.o: $(obj)/zoffset.h
 
-LDFLAGS_setup.elf      := -T
+LDFLAGS_setup.elf      := -m elf_i386 -T
 $(obj)/setup.elf: $(src)/setup.ld $(SETUP_OBJS) FORCE
        $(call if_changed,ld)
 
arch/x86/realmode/rm/Makefile +1 -1
index 4463fa72db94..96cb20de08af 100644
@@ -47,7 +47,7 @@ $(obj)/pasyms.h: $(REALMODE_OBJS) FORCE
 targets += realmode.lds
 $(obj)/realmode.lds: $(obj)/pasyms.h
 
-LDFLAGS_realmode.elf := --emit-relocs -T
+LDFLAGS_realmode.elf := -m elf_i386 --emit-relocs -T
 CPPFLAGS_realmode.lds += -P -C -I$(objtree)/$(obj)
 
Quoted 9 lines
Quoted 4 lines
arch/x86/realmode/rm/Makefile +0 -1
 targets += realmode.elf
--

2.20.0.rc2.403.gdbc3b29805-goog

--
Thanks,
~Nick Desaulniers

--
Thanks,
~Nick Desaulniers

#4 lore raw

Added Rui, an LLD code owner.

This patch contains the approach we discussed earlier during LLD development when faced this issue first time (and as a result of the discussion,
the same fix was suggested: https://bugzilla.kernel.org/show_bug.cgi?id=194091#c0),
so I think it is fine.

Best regards,
George | Developer | Access Softek, Inc

________________________________________
От: Tri Vo <[email protected]>
Отправлено: 12 декабря 2018 г. 22:48
Кому: Nick Desaulniers
Копия: [email protected]; [email protected]; [email protected]; [email protected]; George Rimar; Dmitry Golovin; Bill Wendling; [email protected]; [email protected] Тема: Re: [RFC PATCH] x86_64: Add "-m elf_i386" when linking i386 object files.

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.

Adding appropriate people, lists.

Quoted 117 lines · Nick Desaulniers
Quoted 51 lines · Nick Desaulniers
Quoted 20 lines · Tri Vo

From: George Rimar <[email protected]>

Linux kernel uses OUTPUT_FORMAT in it's linker scripts. Most of the time -m option is passed to the linker with correct architecture, but sometimes (at least for x86_64) the -m option contradicts OUTPUT_FORMAT directive. Specifically, arch/x86/boot and arch/x86/realmode/rm modules have i386 object files, but are linked with -m elf_x86_64 linker flag when building for x86_64.

BFD and Gold linkers are OK with this, but lld fails:
ld.lld: error: arch/x86/realmode/rm/header.o is incompatible with elf_x86_64

Suggested fix: just add correct -m after incorrect one (it overrides it), so the linker invocation looks like this: ld -m elf_x86_64 -z max-page-size=0x200000 -m elf_i386 --emit-relocs -T realmode.lds header.o trampoline_64.o stack.o reboot.o -o realmode.elf (it will also work with GNU ld, because it supports OUTPUT_FORMAT and just ignores -m options if this directive is in the linker script).

Quoted 6 lines

Tested by building x86_64 kernel with GNU gcc/ld toolchain and booting it in QEMU.

This fixes the following linkage error I observe when linking an x86 kernel with LLD:

```
  LD      arch/x86/realmode/rm/realmode.elf
ld.lld: error: arch/x86/realmode/rm/header.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/trampoline_64.o is incompatible
with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/stack.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/reboot.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/wakeup_asm.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/wakemain.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/video-mode.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/copy.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/bioscall.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/regs.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/video-vga.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/video-vesa.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/video-bios.o is incompatible with elf_x86_64
arch/x86/realmode/rm/Makefile:55: recipe for target
'arch/x86/realmode/rm/realmode.elf' failed
```
Quoted 6 lines

Looks like we still have a few other (unrelated) issues to track down with LLD, but this gets us one step closer.

Thanks for sending this, Tri!

Just some additional thoughts on this:

The kernel is adding -m elf_x86_64 to the linker command line parameters, but then relying on binutils tie-breaking-behavior by specifying a different architecture via linker script. So it's ambiguous which arch the kernel is trying to link for, and the kernel gets lucky/relies on the way ld.bfd happens to resolve this ambiguity without warning. An alternative fix may be to just filter out/remove the -m elf_x86_64 for this target's linker flags.

Quoted 31 lines
Quoted 31 lines
---
 arch/x86/boot/Makefile        | 2 +-
 arch/x86/realmode/rm/Makefile | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
arch/x86/boot/Makefile +1 -1
index 9b5adae9cc40..e2839b5c246c 100644
@@ -100,7 +100,7 @@ $(obj)/zoffset.h: $(obj)/compressed/vmlinux FORCE
 AFLAGS_header.o += -I$(objtree)/$(obj)
 $(obj)/header.o: $(obj)/zoffset.h
 
-LDFLAGS_setup.elf      := -T
+LDFLAGS_setup.elf      := -m elf_i386 -T
 $(obj)/setup.elf: $(src)/setup.ld $(SETUP_OBJS) FORCE
        $(call if_changed,ld)
 
arch/x86/realmode/rm/Makefile +1 -1
index 4463fa72db94..96cb20de08af 100644
@@ -47,7 +47,7 @@ $(obj)/pasyms.h: $(REALMODE_OBJS) FORCE
 targets += realmode.lds
 $(obj)/realmode.lds: $(obj)/pasyms.h
 
-LDFLAGS_realmode.elf := --emit-relocs -T
+LDFLAGS_realmode.elf := -m elf_i386 --emit-relocs -T
 CPPFLAGS_realmode.lds += -P -C -I$(objtree)/$(obj)
 
Quoted 9 lines
Quoted 4 lines
arch/x86/realmode/rm/Makefile +0 -1
 targets += realmode.elf
--

2.20.0.rc2.403.gdbc3b29805-goog

--
Thanks,
~Nick Desaulniers

--
Thanks,
~Nick Desaulniers

author
#6 lore raw
In reply to a message not in the archive

Re-sending as plain text. Sorry.

Quoted 145 lines · Tri Vo

Hello, hope everyone's had great holidays.
Could someone review at this patch?

Quoted 140 lines · George Rimar

Added Rui, an LLD code owner.

This patch contains the approach we discussed earlier during LLD development when faced this issue first time (and as a result of the discussion,
the same fix was suggested: https://bugzilla.kernel.org/show_bug.cgi?id=194091#c0),
so I think it is fine.

Best regards,
George | Developer | Access Softek, Inc

________________________________________
От: Tri Vo <[email protected]>
Отправлено: 12 декабря 2018 г. 22:48
Кому: Nick Desaulniers
Копия: [email protected]; [email protected]; [email protected]; [email protected]; George Rimar; Dmitry Golovin; Bill Wendling; [email protected]; [email protected] Тема: Re: [RFC PATCH] x86_64: Add "-m elf_i386" when linking i386 object files.

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.

Adding appropriate people, lists.

Quoted 117 lines · Nick Desaulniers
Quoted 51 lines · Nick Desaulniers
Quoted 20 lines · Tri Vo

From: George Rimar <[email protected]>

Linux kernel uses OUTPUT_FORMAT in it's linker scripts. Most of the time -m option is passed to the linker with correct architecture, but sometimes (at least for x86_64) the -m option contradicts OUTPUT_FORMAT directive. Specifically, arch/x86/boot and arch/x86/realmode/rm modules have i386 object files, but are linked with -m elf_x86_64 linker flag when building for x86_64.

BFD and Gold linkers are OK with this, but lld fails:
ld.lld: error: arch/x86/realmode/rm/header.o is incompatible with elf_x86_64

Suggested fix: just add correct -m after incorrect one (it overrides it), so the linker invocation looks like this: ld -m elf_x86_64 -z max-page-size=0x200000 -m elf_i386 --emit-relocs -T realmode.lds header.o trampoline_64.o stack.o reboot.o -o realmode.elf (it will also work with GNU ld, because it supports OUTPUT_FORMAT and just ignores -m options if this directive is in the linker script).

Quoted 6 lines

Tested by building x86_64 kernel with GNU gcc/ld toolchain and booting it in QEMU.

This fixes the following linkage error I observe when linking an x86 kernel with LLD:

```
  LD      arch/x86/realmode/rm/realmode.elf
ld.lld: error: arch/x86/realmode/rm/header.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/trampoline_64.o is incompatible
with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/stack.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/reboot.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/wakeup_asm.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/wakemain.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/video-mode.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/copy.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/bioscall.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/regs.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/video-vga.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/video-vesa.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/video-bios.o is incompatible with elf_x86_64
arch/x86/realmode/rm/Makefile:55: recipe for target
'arch/x86/realmode/rm/realmode.elf' failed
```
Quoted 6 lines

Looks like we still have a few other (unrelated) issues to track down with LLD, but this gets us one step closer.

Thanks for sending this, Tri!

Just some additional thoughts on this:

The kernel is adding -m elf_x86_64 to the linker command line parameters, but then relying on binutils tie-breaking-behavior by specifying a different architecture via linker script. So it's ambiguous which arch the kernel is trying to link for, and the kernel gets lucky/relies on the way ld.bfd happens to resolve this ambiguity without warning. An alternative fix may be to just filter out/remove the -m elf_x86_64 for this target's linker flags.

Quoted 31 lines
Quoted 31 lines
---
 arch/x86/boot/Makefile        | 2 +-
 arch/x86/realmode/rm/Makefile | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
arch/x86/boot/Makefile +1 -1
index 9b5adae9cc40..e2839b5c246c 100644
@@ -100,7 +100,7 @@ $(obj)/zoffset.h: $(obj)/compressed/vmlinux FORCE
 AFLAGS_header.o += -I$(objtree)/$(obj)
 $(obj)/header.o: $(obj)/zoffset.h
 
-LDFLAGS_setup.elf      := -T
+LDFLAGS_setup.elf      := -m elf_i386 -T
 $(obj)/setup.elf: $(src)/setup.ld $(SETUP_OBJS) FORCE
        $(call if_changed,ld)
 
arch/x86/realmode/rm/Makefile +1 -1
index 4463fa72db94..96cb20de08af 100644
@@ -47,7 +47,7 @@ $(obj)/pasyms.h: $(REALMODE_OBJS) FORCE
 targets += realmode.lds
 $(obj)/realmode.lds: $(obj)/pasyms.h
 
-LDFLAGS_realmode.elf := --emit-relocs -T
+LDFLAGS_realmode.elf := -m elf_i386 --emit-relocs -T
 CPPFLAGS_realmode.lds += -P -C -I$(objtree)/$(obj)
 
Quoted 9 lines
Quoted 4 lines
arch/x86/realmode/rm/Makefile +0 -1
 targets += realmode.elf
--

2.20.0.rc2.403.gdbc3b29805-goog

--
Thanks,
~Nick Desaulniers

--
Thanks,
~Nick Desaulniers

#7 lore raw
Quoted 4 lines · Tri Vo
Quoted 3 lines

Hello, hope everyone's had great holidays.
Could someone review at this patch?

Boris,
I assume Tri meant to post this to you in the To: line? I'm happy with this patch, and am happy to add my reviewed by tag (in addition to my tested-by below), but I think Tri meant to ask you. WDYT?

Quoted 130 lines
Quoted 130 lines · George Rimar
Quoted 128 lines

Added Rui, an LLD code owner.

This patch contains the approach we discussed earlier during LLD development when faced this issue first time (and as a result of the discussion,
the same fix was suggested: https://bugzilla.kernel.org/show_bug.cgi?id=194091#c0),
so I think it is fine.

Best regards,
George | Developer | Access Softek, Inc

________________________________________
От: Tri Vo <[email protected]>
Отправлено: 12 декабря 2018 г. 22:48
Кому: Nick Desaulniers
Копия: [email protected]; [email protected]; [email protected]; [email protected]; George Rimar; Dmitry Golovin; Bill Wendling; [email protected]; [email protected] Тема: Re: [RFC PATCH] x86_64: Add "-m elf_i386" when linking i386 object files.

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.

Adding appropriate people, lists.

Quoted 105 lines · Nick Desaulniers
Quoted 51 lines · Nick Desaulniers
Quoted 20 lines · Tri Vo

From: George Rimar <[email protected]>

Linux kernel uses OUTPUT_FORMAT in it's linker scripts. Most of the time -m option is passed to the linker with correct architecture, but sometimes (at least for x86_64) the -m option contradicts OUTPUT_FORMAT directive. Specifically, arch/x86/boot and arch/x86/realmode/rm modules have i386 object files, but are linked with -m elf_x86_64 linker flag when building for x86_64.

BFD and Gold linkers are OK with this, but lld fails:
ld.lld: error: arch/x86/realmode/rm/header.o is incompatible with elf_x86_64

Suggested fix: just add correct -m after incorrect one (it overrides it), so the linker invocation looks like this: ld -m elf_x86_64 -z max-page-size=0x200000 -m elf_i386 --emit-relocs -T realmode.lds header.o trampoline_64.o stack.o reboot.o -o realmode.elf (it will also work with GNU ld, because it supports OUTPUT_FORMAT and just ignores -m options if this directive is in the linker script).

Quoted 6 lines

Tested by building x86_64 kernel with GNU gcc/ld toolchain and booting it in QEMU.

This fixes the following linkage error I observe when linking an x86 kernel with LLD:

```
  LD      arch/x86/realmode/rm/realmode.elf
ld.lld: error: arch/x86/realmode/rm/header.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/trampoline_64.o is incompatible
with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/stack.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/reboot.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/wakeup_asm.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/wakemain.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/video-mode.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/copy.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/bioscall.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/regs.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/video-vga.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/video-vesa.o is incompatible with elf_x86_64
ld.lld: error: arch/x86/realmode/rm/video-bios.o is incompatible with elf_x86_64
arch/x86/realmode/rm/Makefile:55: recipe for target
'arch/x86/realmode/rm/realmode.elf' failed
```
Quoted 6 lines

Looks like we still have a few other (unrelated) issues to track down with LLD, but this gets us one step closer.

Thanks for sending this, Tri!

Just some additional thoughts on this:

The kernel is adding -m elf_x86_64 to the linker command line parameters, but then relying on binutils tie-breaking-behavior by specifying a different architecture via linker script. So it's ambiguous which arch the kernel is trying to link for, and the kernel gets lucky/relies on the way ld.bfd happens to resolve this ambiguity without warning. An alternative fix may be to just filter out/remove the -m elf_x86_64 for this target's linker flags.

Quoted 34 lines
Quoted 34 lines
---
 arch/x86/boot/Makefile        | 2 +-
 arch/x86/realmode/rm/Makefile | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
arch/x86/boot/Makefile +1 -1
index 9b5adae9cc40..e2839b5c246c 100644
@@ -100,7 +100,7 @@ $(obj)/zoffset.h: $(obj)/compressed/vmlinux FORCE
 AFLAGS_header.o += -I$(objtree)/$(obj)
 $(obj)/header.o: $(obj)/zoffset.h
 
-LDFLAGS_setup.elf      := -T
+LDFLAGS_setup.elf      := -m elf_i386 -T
 $(obj)/setup.elf: $(src)/setup.ld $(SETUP_OBJS) FORCE
        $(call if_changed,ld)
 
arch/x86/realmode/rm/Makefile +1 -2
index 4463fa72db94..96cb20de08af 100644
@@ -47,7 +47,7 @@ $(obj)/pasyms.h: $(REALMODE_OBJS) FORCE
 targets += realmode.lds
 $(obj)/realmode.lds: $(obj)/pasyms.h
 
-LDFLAGS_realmode.elf := --emit-relocs -T
+LDFLAGS_realmode.elf := -m elf_i386 --emit-relocs -T
 CPPFLAGS_realmode.lds += -P -C -I$(objtree)/$(obj)
 
 targets += realmode.elf
--

2.20.0.rc2.403.gdbc3b29805-goog

Signature · 2 lines
Thanks,
~Nick Desaulniers