Fully support standalone Clang/LLVM toolchains
v4 cover Fully support standalone Clang/LLVM toolchains
This patch series is comprised of five parts.
The first patch fixes a warning seen when building with LLVM due to a bogus combination of assembly directives.
The second patch fixes errors seen when trying to build OpenSBI as a position-independent binary using LLD that may or may not be an LLD bug (the exact meaning of -N/--omagic isn't clear) but can easily be worked around without any issue either way.
The third patch bypasses the Clang driver's helpful nature of not honouring -pie for bare-metal binaries as it's normally not something you want (arguably that should be an error though, or passed on, since just giving a -Wunused-command-line-argument warning can get lost, and may be disabled).
The fourth patch adds build system support for using Clang and LLVM binutils, provided your Clang is able to locate a libgcc.a in its search path.
However, pure LLVM toolchains do not use libgcc, they use compiler-rt (libclang_rt.builtins.<arch>.a). We could change the Makefile to not hard-code -lgcc and instead use -print-libgcc-file-name, but that still requires a bare metal compiler-rt built for the right -march/-mabi to be present, which is often not the case. Moreover, we need very little from libgcc/compiler-rt; RV64 needs nothing, and RV32 only needs 64-bit division. Thus, the fifth patch vendors part of FreeBSD's libquad and stops linking against libgcc entirely, allowing OpenSBI to be built with just a cross-compiler. This means that building with any distro-provided LLVM just works, as does compiling with the system Clang compiler and LLD linker on FreeBSD without any external packages needed (beyond GNU make).
Changes in v4:
* Avoid read-only dynamic relocation error with LLD
* Support FW_PIC with LLD when using a bare-metal triple
Changes in v3:
* Fixed the binding warning seen when building with LLVM
* Provide a default -target when CROSS_COMPILE isn't set so the variable isn't normally required to be set when cross-compiling with LLVM (but is still honoured if set).
* Pass -fuse=bfd when not using LLD to ensure an ld.bfd gets correctly picked up over a plain ld when ld is LLD. Now we always explicitly request Clang and GCC use the linker type corresponding to LD. This allows BFD to be used on FreeBSD for testing purposes despite LLD being the system linker.
Changes in v2:
* Add documentation to README.md
* Pass -fuse-ld=lld to Clang when using LLD
* Modify commit message of first commit to explain -N -> -Wl,-N change
* Bring back the old ?= uses for the non-CROSS_COMPILE (and, now, non-LLVM) case; whilst I still think that's how it should be, it's not required for this patch, just related cleanup I did to reduce the complexity of adding the LLVM case.
Jessica Clarke (5):
fw_base: Don't mark fw_platform_init as both global and weak
fw_base: Put data in .data rather than .text
firmware: Explicitly pass -pie to the linker, not just the driver
Makefile: Support building with Clang and LLVM binutils Drop dependency on libgcc by importing part of FreeBSD's libquad
Makefile | 65 +++++- README.md | 43 +++- firmware/fw_base.S | 2 +- firmware/objects.mk | 2 +- lib/utils/libquad/divdi3.c | 64 ++++++ lib/utils/libquad/include/limits.h | 12 ++ lib/utils/libquad/include/sys/cdefs.h | 12 ++ lib/utils/libquad/include/sys/types.h | 25 +++ lib/utils/libquad/moddi3.c | 66 ++++++ lib/utils/libquad/objects.mk | 14 ++ lib/utils/libquad/qdivrem.c | 278 ++++++++++++++++++++++++++ lib/utils/libquad/quad.h | 105 ++++++++++ lib/utils/libquad/udivdi3.c | 52 +++++ lib/utils/libquad/umoddi3.c | 54 +++++ 14 files changed, 784 insertions(+), 10 deletions(-) create mode 100644 lib/utils/libquad/divdi3.c create mode 100644 lib/utils/libquad/include/limits.h create mode 100644 lib/utils/libquad/include/sys/cdefs.h create mode 100644 lib/utils/libquad/include/sys/types.h create mode 100644 lib/utils/libquad/moddi3.c create mode 100644 lib/utils/libquad/objects.mk create mode 100644 lib/utils/libquad/qdivrem.c create mode 100644 lib/utils/libquad/quad.h create mode 100644 lib/utils/libquad/udivdi3.c create mode 100644 lib/utils/libquad/umoddi3.c
Signature · 1 line
2.31.0
PATCH v4 1/5 fw_base: Don't mark fw_platform_init as both global and weak
These are mutually exclusive. GNU as and LLVM both let later binding directives override earlier ones so this works as intended, but LLVM 12 turned this into a warning as there's no good reason to do such a thing and could be a potential bug. Thus, remove the redundant and incorrect .globl directive for fw_platform_init.
--- firmware/fw_base.S | 1 - 1 file changed, 1 deletion(-)
firmware/fw_base.S +0 -1
@@ -553,7 +553,6 @@ _start_hang: .section .entry, "ax", %progbits .align 3 - .globl fw_platform_init .weak fw_platform_init fw_platform_init: add a0, a1, zero
Signature · 1 line
2.31.0
PATCH v4 1/5 fw_base: Don't mark fw_platform_init as both global and weak
Quoted 10 lines · Jessica Clarke
These are mutually exclusive. GNU as and LLVM both let later binding directives override earlier ones so this works as intended, but LLVM 12 turned this into a warning as there's no good reason to do such a thing and could be a potential bug. Thus, remove the redundant and incorrect .globl directive for
fw_platform_init.--- firmware/fw_base.S | 1 - 1 file changed, 1 deletion(-)
You SoB tag was missed again:
And please include people's RB / TB tags in the newer version, if nothing changed:
PATCH v4 2/5 fw_base: Put data in .data rather than .text
In reply to Jessica ClarkeThe -N linker option is supposed to make .text writable, but GNU ld and LLD differ in interpreting what that means. GNU ld will happily let you have relocations in it, but LLD will see that the input section is read-only (even though the output section is writable) and give an error. It's unclear if either of them intend to have that behaviour in this edge case, but regardless there's no reason not to just put the data in a writable .data section.
--- firmware/fw_base.S | 1 + 1 file changed, 1 insertion(+)
firmware/fw_base.S +1 -0
@@ -501,6 +501,7 @@ _skip_trap_exit_rv32_hyp: /* We don't expect to reach here hence just hang */ j _start_hang + .data .align 3 #ifdef FW_PIC _runtime_offset:
Signature · 1 line
2.31.0
PATCH v4 2/5 fw_base: Put data in .data rather than .text
Quoted 8 lines · Jessica Clarke
The -N linker option is supposed to make .text writable, but GNU ld and LLD differ in interpreting what that means. GNU ld will happily let you have relocations in it, but LLD will see that the input section is read-only (even though the output section is writable) and give an error. It's unclear if either of them intend to have that behaviour in this edge case, but regardless there's no reason not to just put the data in a writable .data section.
Quoted 4 lines
--- firmware/fw_base.S | 1 + 1 file changed, 1 insertion(+)
Again, your SoB tag was missed:
With this patch the following error seen when building with full LLVM toolchain no longer exists:
ELF platform/generic/firmware/fw_dynamic.elf
ld.lld: error: can't create dynamic relocation R_RISCV_64 against
symbol: _fw_start in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
defined in opensbi/build/platform/generic/firmware/fw_dynamic.elf.ld:8 referenced by fw_base.S:502 (
opensbi/firmware/fw_base.S:502)opensbi/build/platform/generic/firmware/fw_dynamic.o:(.entry+0x3A0)
ld.lld: error: can't create dynamic relocation R_RISCV_64 against
symbol: _fw_reloc_end in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
defined in opensbi/build/platform/generic/firmware/fw_dynamic.elf.ld:92 referenced by fw_base.S:502 (
opensbi/firmware/fw_base.S:502)opensbi/build/platform/generic/firmware/fw_dynamic.o:(.entry+0x3B0)
clang-12: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Makefile:396:
opensbi/build/platform/generic/firmware/fw_dynamic.elf] Error 1
PATCH v4 3/5 firmware: Explicitly pass -pie to the linker, not just the driver
In reply to Jessica ClarkeWhen using Clang with a bare-metal triple, -pie does not get passed to the linker as it's not normally a thing that makes sense. However, in our case it is, and manually forwarding it on works as desired, so do so to fully support FW_PIC with Clang, including when linking with LLD.
--- firmware/objects.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
firmware/objects.mk +1 -1
@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y) firmware-genflags-y += -DFW_PIC firmware-asflags-y += -fpic firmware-cflags-y += -fPIE -pie -firmware-ldflags-y += -Wl,--no-dynamic-linker +firmware-ldflags-y += -Wl,--no-dynamic-linker -Wl,-pie endif ifdef FW_TEXT_START
Signature · 1 line
2.31.0
PATCH v4 3/5 firmware: Explicitly pass -pie to the linker, not just the driver
Quoted 14 lines · Jessica Clarke
When using Clang with a bare-metal triple, -pie does not get passed to the linker as it's not normally a thing that makes sense. However, in our case it is, and manually forwarding it on works as desired, so do so to fully support
FW_PICwith Clang, including when linking with LLD.--- firmware/objects.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)firmware/objects.mk +0
-0
firmware/objects.mk ifeq ($(FW_PIC),y) · line 25
Quoted 6 lines
firmware/objects.mk +1
-1@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y) firmware-genflags-y += -DFW_PIC firmware-asflags-y += -fpic firmware-cflags-y += -fPIE -pie -firmware-ldflags-y += -Wl,--no-dynamic-linker +firmware-ldflags-y += -Wl,--no-dynamic-linker -Wl,-pie endif
Does this manual forwarding also work for GNU ld? If so, I think we don't need to detect bare-metal triple and turn off FW_PIC in Makefile?
Regards,
Bin
PATCH v4 3/5 firmware: Explicitly pass -pie to the linker, not just the driver
Quoted 11 lines · Bin Meng
Quoted 9 lines · Jessica Clarke
When using Clang with a bare-metal triple, -pie does not get passed to the linker as it's not normally a thing that makes sense. However, in our case it is, and manually forwarding it on works as desired, so do so to fully support
FW_PICwith Clang, including when linking with LLD.--- firmware/objects.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 15 lines
firmware/objects.mk ifeq ($(FW_PIC),y) · line 25
Quoted 11 lines
firmware/objects.mk +1
-1@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y) firmware-genflags-y += -DFW_PIC firmware-asflags-y += -fpic firmware-cflags-y += -fPIE -pie -firmware-ldflags-y += -Wl,--no-dynamic-linker +firmware-ldflags-y += -Wl,--no-dynamic-linker -Wl,-pie endifDoes this manual forwarding also work for GNU ld? If so, I think we don't need to detect bare-metal triple and turn off
FW_PICinMakefile?
I just built a riscv64-unknown-elf-gcc toolchain and used it to build the current HEAD of opensbi/master. Indeed it's broken that pie is not supported with the bare-metal triple.
The GNU ld simply complains:
ELF platform/generic/firmware/payloads/test.elf/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
-pie not supported
Regards,
Bin
PATCH v4 3/5 firmware: Explicitly pass -pie to the linker, not just the driver
Hi Bin,
Quoted 33 lines · Bin Meng
Quoted 11 lines · Bin Meng
Quoted 9 lines · Jessica Clarke
When using Clang with a bare-metal triple, -pie does not get passed to the linker as it's not normally a thing that makes sense. However, in our case it is, and manually forwarding it on works as desired, so do so to fully support
FW_PICwith Clang, including when linking with LLD.--- firmware/objects.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Quoted 15 lines
firmware/objects.mk ifeq ($(FW_PIC),y) · line 25
Quoted 11 lines
firmware/objects.mk +1
-1@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y) firmware-genflags-y += -DFW_PIC firmware-asflags-y += -fpic firmware-cflags-y += -fPIE -pie -firmware-ldflags-y += -Wl,--no-dynamic-linker +firmware-ldflags-y += -Wl,--no-dynamic-linker -Wl,-pie endifDoes this manual forwarding also work for GNU ld? If so, I think we don't need to detect bare-metal triple and turn off
FW_PICinMakefile?I just built a riscv64-unknown-elf-gcc toolchain and used it to build the current HEAD of opensbi/master. Indeed it's broken that pie is not supported with the bare-metal triple.
Quoted 5 lines
The GNU ld simply complains:
ELF platform/generic/firmware/payloads/test.elf
/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
-pie not supported
The firmware/objects.mk is doing the following:
ifndef FW_PICFW_PIC := y
endif
Instead of above, we should set FW_PIC=y only when the underlying toolchain supports pie.
We need a patch for this to be merged before we can merge this series. Can you send such a patch ? If not then I can send it.
Regards,
Anup
Quoted 8 lines
Regards,
Bin--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi
PATCH v4 3/5 firmware: Explicitly pass -pie to the linker, not just the driver
Hi Anup,
Quoted 43 lines · Anup Patel
Hi Bin,
Quoted 33 lines · Bin Meng
Quoted 11 lines · Bin Meng
Quoted 9 lines · Jessica Clarke
When using Clang with a bare-metal triple, -pie does not get passed to the linker as it's not normally a thing that makes sense. However, in our case it is, and manually forwarding it on works as desired, so do so to fully support
FW_PICwith Clang, including when linking with LLD.--- firmware/objects.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Quoted 15 lines
firmware/objects.mk ifeq ($(FW_PIC),y) · line 25
Quoted 11 lines
firmware/objects.mk +1
-1@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y) firmware-genflags-y += -DFW_PIC firmware-asflags-y += -fpic firmware-cflags-y += -fPIE -pie -firmware-ldflags-y += -Wl,--no-dynamic-linker +firmware-ldflags-y += -Wl,--no-dynamic-linker -Wl,-pie endifDoes this manual forwarding also work for GNU ld? If so, I think we don't need to detect bare-metal triple and turn off
FW_PICinMakefile?I just built a riscv64-unknown-elf-gcc toolchain and used it to build the current HEAD of opensbi/master. Indeed it's broken that pie is not supported with the bare-metal triple.
Quoted 5 lines
The GNU ld simply complains:
ELF platform/generic/firmware/payloads/test.elf
/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
-pie not supported
Quoted 9 lines
The
firmware/objects.mkis doing the following:ifndef
FW_PICFW_PIC:= y
endifInstead of above, we should set FW_PIC=y only when the underlying toolchain supports pie.
Agree.
Quoted 2 lines
We need a patch for this to be merged before we can merge this series. Can you send such a patch ? If not then I can send it.
Do you have some reliable ways to check whether a toolchain supports PIE?
Regards,
Bin
PATCH v4 3/5 firmware: Explicitly pass -pie to the linker, not just the driver
Quoted 57 lines · Bin Meng
Hi Anup,
Quoted 43 lines · Anup Patel
Hi Bin,
Quoted 33 lines · Bin Meng
Quoted 11 lines · Bin Meng
Quoted 9 lines · Jessica Clarke
When using Clang with a bare-metal triple, -pie does not get passed to the linker as it's not normally a thing that makes sense. However, in our case it is, and manually forwarding it on works as desired, so do so to fully support
FW_PICwith Clang, including when linking with LLD.--- firmware/objects.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Quoted 15 lines
firmware/objects.mk ifeq ($(FW_PIC),y) · line 25
Quoted 11 lines
firmware/objects.mk +1
-1@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y) firmware-genflags-y += -DFW_PIC firmware-asflags-y += -fpic firmware-cflags-y += -fPIE -pie -firmware-ldflags-y += -Wl,--no-dynamic-linker +firmware-ldflags-y += -Wl,--no-dynamic-linker -Wl,-pie endifDoes this manual forwarding also work for GNU ld? If so, I think we don't need to detect bare-metal triple and turn off
FW_PICinMakefile?I just built a riscv64-unknown-elf-gcc toolchain and used it to build the current HEAD of opensbi/master. Indeed it's broken that pie is not supported with the bare-metal triple.
Quoted 5 lines
The GNU ld simply complains:
ELF platform/generic/firmware/payloads/test.elf
/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
-pie not supportedQuoted 9 lines
The
firmware/objects.mkis doing the following:ifndef
FW_PICFW_PIC:= y
endifInstead of above, we should set FW_PIC=y only when the underlying toolchain supports pie.
Quoted 6 lines
Agree.
Quoted 2 lines
We need a patch for this to be merged before we can merge this series. Can you send such a patch ? If not then I can send it.
Do you have some reliable ways to check whether a toolchain supports PIE?
How about checking "-linux-" in CROSS_COMPILE prefix ? If it's available then we set FW_PIC=y else we set FW_PIC=n. I did not find any info in the "${CROSS_COMPILE}gcc -v" output.
Maybe also add some comment in objects.mk that GCC bare-metal toolchain does not have PIE enabled.
Regards,
Anup
Quoted 3 lines
Regards,
Bin
PATCH v4 3/5 firmware: Explicitly pass -pie to the linker, not just the driver
Quoted 61 lines · Anup Patel
Quoted 59 lines · Bin Meng
Hi Anup,
Quoted 43 lines · Anup Patel
Hi Bin,
Quoted 33 lines · Bin Meng
Quoted 11 lines · Bin Meng
Quoted 9 lines · Jessica Clarke
When using Clang with a bare-metal triple, -pie does not get passed to the linker as it's not normally a thing that makes sense. However, in our case it is, and manually forwarding it on works as desired, so do so to fully support
FW_PICwith Clang, including when linking with LLD.--- firmware/objects.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Quoted 15 lines
firmware/objects.mk ifeq ($(FW_PIC),y) · line 25
Quoted 11 lines
firmware/objects.mk +1
-1@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y) firmware-genflags-y += -DFW_PIC firmware-asflags-y += -fpic firmware-cflags-y += -fPIE -pie -firmware-ldflags-y += -Wl,--no-dynamic-linker +firmware-ldflags-y += -Wl,--no-dynamic-linker -Wl,-pie endifDoes this manual forwarding also work for GNU ld? If so, I think we don't need to detect bare-metal triple and turn off
FW_PICinMakefile?I just built a riscv64-unknown-elf-gcc toolchain and used it to build the current HEAD of opensbi/master. Indeed it's broken that pie is not supported with the bare-metal triple.
Quoted 5 lines
The GNU ld simply complains:
ELF platform/generic/firmware/payloads/test.elf
/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
-pie not supportedQuoted 9 lines
The
firmware/objects.mkis doing the following:ifndef
FW_PICFW_PIC:= y
endifInstead of above, we should set FW_PIC=y only when the underlying toolchain supports pie.
Agree.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
We need a patch for this to be merged before we can merge this series. Can you send such a patch ? If not then I can send it.
Do you have some reliable ways to check whether a toolchain supports PIE?
How about checking "-linux-" in
CROSS_COMPILEprefix ? If it's available then we set FW_PIC=y else we set FW_PIC=n.
That works for cross-compile toolchains. But how about native toolchains (building OpenSBI on a RISC-V machine)?
Quoted 1 line
I did not find any info in the "${CROSS_COMPILE}gcc -v" output.
Me neither.
Quoted 3 lines
Maybe also add some comment in objects.mk that GCC bare-metal toolchain does not have PIE enabled.
Or we can just document if using bare-metal toolchain FW_PIC has to be set to n in the build.
Regards,
Bin
PATCH v4 3/5 firmware: Explicitly pass -pie to the linker, not just the driver
Quoted 66 lines · Bin Meng
Quoted 64 lines · Anup Patel
Quoted 62 lines · Bin Meng
Hi Anup,
Quoted 43 lines · Anup Patel
Hi Bin,
Quoted 33 lines · Bin Meng
Quoted 11 lines · Bin Meng
Quoted 9 lines · Jessica Clarke
When using Clang with a bare-metal triple, -pie does not get passed to the linker as it's not normally a thing that makes sense. However, in our case it is, and manually forwarding it on works as desired, so do so to fully support
FW_PICwith Clang, including when linking with LLD.--- firmware/objects.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Quoted 15 lines
firmware/objects.mk ifeq ($(FW_PIC),y) · line 25
Quoted 11 lines
firmware/objects.mk +1
-1@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y) firmware-genflags-y += -DFW_PIC firmware-asflags-y += -fpic firmware-cflags-y += -fPIE -pie -firmware-ldflags-y += -Wl,--no-dynamic-linker +firmware-ldflags-y += -Wl,--no-dynamic-linker -Wl,-pie endifDoes this manual forwarding also work for GNU ld? If so, I think we don't need to detect bare-metal triple and turn off
FW_PICinMakefile?I just built a riscv64-unknown-elf-gcc toolchain and used it to build the current HEAD of opensbi/master. Indeed it's broken that pie is not supported with the bare-metal triple.
Quoted 5 lines
The GNU ld simply complains:
ELF platform/generic/firmware/payloads/test.elf
/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
-pie not supportedQuoted 9 lines
The
firmware/objects.mkis doing the following:ifndef
FW_PICFW_PIC:= y
endifInstead of above, we should set FW_PIC=y only when the underlying toolchain supports pie.
Agree.
Quoted 2 lines
We need a patch for this to be merged before we can merge this series. Can you send such a patch ? If not then I can send it.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Do you have some reliable ways to check whether a toolchain supports PIE?
How about checking "-linux-" in
CROSS_COMPILEprefix ? If it's available then we set FW_PIC=y else we set FW_PIC=n.That works for cross-compile toolchains. But how about native toolchains (building OpenSBI on a RISC-V machine)?
Yes, it will not work for native compilation of OpenSBI.
Quoted 11 lines
Quoted 2 lines
I did not find any info in the "${CROSS_COMPILE}gcc -v" output.
Me neither.
Quoted 3 lines
Maybe also add some comment in objects.mk that GCC bare-metal toolchain does not have PIE enabled.
Or we can just document if using bare-metal toolchain
FW_PIChas to be set to n in the build.
Sounds good.
We should document it under section "Required Toolchain" of top-level README.md. Basically, we prefer toolchains with PIE support and for toolchains not having PIE users have to pass the "FW_PIC=n" option.
Also, we should replace references of bare-metal toolchain prefix with linux toolchain prefix everywhere in documentation.
Regards,
Anup
Quoted 3 lines
Regards,
Bin
PATCH v4 3/5 firmware: Explicitly pass -pie to the linker, not just the driver
? 2021-07-10???? 20:26 +0530?Anup Patel???
Quoted 118 lines · Bin Meng
Quoted 88 lines
Quoted 85 lines · Anup Patel
Quoted 81 lines · Bin Meng
Hi Anup,
Quoted 58 lines · Anup Patel
Hi Bin,
Quoted 44 lines · Bin Meng
Quoted 26 lines · Bin Meng
Quoted 23 lines · Jessica Clarke
When using Clang with a bare-metal triple, -pie does not get passed to
the linker as it's not normally a thing that makes sense. However, in
our case it is, and manually forwarding it on works as desired, so do so
to fully supportFW_PICwith Clang, including when linking with LLD.
---
?firmware/objects.mk | 2 +-
?1 file changed, 1 insertion(+), 1 deletion(-)firmware/objects.mk +0
-0@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)?firmware-genflags-y += -DFW_PIC
?firmware-asflags-y? += -fpic
?firmware-cflags-y?? += -fPIE -piefirmware/objects.mk +1
-1-firmware-ldflags-y? +=? -Wl,--no-dynamic-linker +firmware-ldflags-y? +=? -Wl,--no-dynamic-linker -Wl,-Quoted 8 lines
Quoted 2 lines
pie
?endifDoes this manual forwarding also work for GNU ld? If so, I think we
don't need to detect bare-metal triple and turn offFW_PICinMakefile?I just built a riscv64-unknown-elf-gcc toolchain and used it to build
the current HEAD of opensbi/master. Indeed it's broken that pie is not
supported with the bare-metal triple.Quoted 6 lines
The GNU ld simply complains:
?ELF?????? platform/generic/firmware/payloads/test.elf
/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
-pie not supportedThe
firmware/objects.mkis doing the following:Quoted 8 lines
ifndef
FW_PICFW_PIC:= y
endifInstead of above, we should set FW_PIC=y only when the underlying
toolchain supports pie.Agree.
Quoted 4 lines
We need a patch for this to be merged before we can merge this
series. Can you send such a patch ? If not then I can send it.Do you have some reliable ways to check whether a toolchain supports PIE?
Quoted 6 lines
Quoted 3 lines
How about checking "-linux-" in
CROSS_COMPILEprefix ? If it's available then
we set FW_PIC=y else we set FW_PIC=n.That works for cross-compile toolchains. But how about native toolchains (building OpenSBI on a RISC-V machine)?
Yes, it will not work for native compilation of OpenSBI.
Quoted 6 lines
Quoted 2 lines
I did not find any info in the "${CROSS_COMPILE}gcc -v" output.
Me neither.
Quoted 1 line
Quoted 7 lines
Quoted 3 lines
Maybe also add some comment in objects.mk that GCC bare-metal toolchain
does not have PIE enabled.Or we can just document if using bare-metal toolchain
FW_PIChas to be
set to n in the build.Sounds good.
We should document it under section "Required Toolchain" of top-level README.md. Basically, we prefer toolchains with PIE support and for toolchains not having PIE users have to pass the "FW_PIC=n" option.
Quoted 6 lines
Also, we should replace references of bare-metal toolchain prefix with
linux toolchain prefix everywhere in documentation.Regards,
Anup
We can add the following code to the makefile to detect pie support
pie_support=$(shell $(CC) -nostdlib -fPIE -Wl,-pie /dev/null 2>/dev/null && echo y || echo n)
Quoted 5 lines
Quoted 4 lines
Regards,
Bin
PATCH v4 3/5 firmware: Explicitly pass -pie to the linker, not just the driver
Quoted 124 lines · Xiang W
? 2021-07-10???? 20:26 +0530?Anup Patel???
Quoted 122 lines · Bin Meng
Quoted 88 lines
Quoted 85 lines · Anup Patel
Quoted 81 lines · Bin Meng
Hi Anup,
Quoted 58 lines · Anup Patel
Hi Bin,
Quoted 44 lines · Bin Meng
Quoted 26 lines · Bin Meng
Quoted 23 lines · Jessica Clarke
When using Clang with a bare-metal triple, -pie does not get passed to
the linker as it's not normally a thing that makes sense. However, in
our case it is, and manually forwarding it on works as desired, so do so
to fully supportFW_PICwith Clang, including when linking with LLD.--- firmware/objects.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)firmware/objects.mk +1
-1@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y) firmware-genflags-y += -DFW_PIC firmware-asflags-y += -fpic firmware-cflags-y += -fPIE -pie -firmware-ldflags-y += -Wl,--no-dynamic-linker +firmware-ldflags-y += -Wl,--no-dynamic-linker -Wl,-Quoted 8 lines
Quoted 2 lines
pie
endifDoes this manual forwarding also work for GNU ld? If so, I think we
don't need to detect bare-metal triple and turn offFW_PICinMakefile?I just built a riscv64-unknown-elf-gcc toolchain and used it to build
the current HEAD of opensbi/master. Indeed it's broken that pie is not
supported with the bare-metal triple.Quoted 6 lines
The GNU ld simply complains:
ELF platform/generic/firmware/payloads/test.elf
/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
-pie not supportedThe
firmware/objects.mkis doing the following:Quoted 8 lines
ifndef
FW_PICFW_PIC:= y
endifInstead of above, we should set FW_PIC=y only when the underlying
toolchain supports pie.Agree.
Quoted 4 lines
We need a patch for this to be merged before we can merge this
series. Can you send such a patch ? If not then I can send it.Do you have some reliable ways to check whether a toolchain supports PIE?
Quoted 6 lines
Quoted 3 lines
How about checking "-linux-" in
CROSS_COMPILEprefix ? If it's available then
we set FW_PIC=y else we set FW_PIC=n.That works for cross-compile toolchains. But how about native toolchains (building OpenSBI on a RISC-V machine)?
Yes, it will not work for native compilation of OpenSBI.
Quoted 6 lines
Quoted 2 lines
I did not find any info in the "${CROSS_COMPILE}gcc -v" output.
Me neither.
Quoted 1 line
Quoted 7 lines
Quoted 3 lines
Maybe also add some comment in objects.mk that GCC bare-metal toolchain
does not have PIE enabled.Or we can just document if using bare-metal toolchain
FW_PIChas to be
set to n in the build.Sounds good.
We should document it under section "Required Toolchain" of top-level README.md. Basically, we prefer toolchains with PIE support and for toolchains not having PIE users have to pass the "FW_PIC=n" option.
Also, we should replace references of bare-metal toolchain prefix with
linux toolchain prefix everywhere in documentation.
Quoted 6 lines
Quoted 2 lines
Regards,
AnupWe can add the following code to the makefile to detect pie support
pie_support=$(shell $(CC) -nostdlib -fPIE -Wl,-pie
/dev/null2>/dev/null && echo y || echo n)
That doesn?t work, /dev/null is not a valid object file so this will always fail with any toolchain.
Jess
PATCH v4 3/5 firmware: Explicitly pass -pie to the linker, not just the driver
? 2021-07-10???? 19:10 +0100?Jessica Clarke???
Quoted 144 lines · Xiang W
Quoted 143 lines
? 2021-07-10???? 20:26 +0530?Anup Patel???
Quoted 141 lines
Quoted 98 lines · Bin Meng
Quoted 94 lines · Anup Patel
Quoted 90 lines · Bin Meng
Hi Anup,
Quoted 66 lines · Anup Patel
Hi Bin,
Quoted 51 lines · Bin Meng
Quoted 31 lines · Bin Meng
Quoted 28 lines · Jessica Clarke
When using Clang with a bare-metal triple, -pie does
not get passed to
the linker as it's not normally a thing that makes sense. However, in
our case it is, and manually forwarding it on works as
desired, so do so
to fully supportFW_PICwith Clang, including when linking with LLD.
---
?firmware/objects.mk | 2 +-
?1 file changed, 1 insertion(+), 1 deletion(-)firmware/objects.mk +0
-0@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)?firmware-genflags-y += -DFW_PIC
?firmware-asflags-y? += -fpic
?firmware-cflags-y?? += -fPIE -piefirmware/objects.mk +1
-1-firmware-ldflags-y? +=? -Wl,--no-dynamic-linker +firmware-ldflags-y? +=? -Wl,--no-dynamic-linker -Wl,-
pieQuoted 8 lines
Quoted 1 line
?endif
Does this manual forwarding also work for GNU ld? If so,
I think we
don't need to detect bare-metal triple and turn offFW_PICinMakefile?I just built a riscv64-unknown-elf-gcc toolchain and used
it to build
the current HEAD of opensbi/master. Indeed it's broken that
pie is not
supported with the bare-metal triple.Quoted 6 lines
The GNU ld simply complains:
?ELF?????? platform/generic/firmware/payloads/test.elf
/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
-pie not supportedThe
firmware/objects.mkis doing the following:Quoted 8 lines
ifndef
FW_PICFW_PIC:= y
endifInstead of above, we should set FW_PIC=y only when the underlying
toolchain supports pie.Agree.
Quoted 5 lines
We need a patch for this to be merged before we can merge this
series. Can you send such a patch ? If not then I can send
it.Do you have some reliable ways to check whether a toolchain supports PIE?
Quoted 7 lines
Quoted 4 lines
How about checking "-linux-" in
CROSS_COMPILEprefix ? If it's
available then
we set FW_PIC=y else we set FW_PIC=n.That works for cross-compile toolchains. But how about native toolchains (building OpenSBI on a RISC-V machine)?
Yes, it will not work for native compilation of OpenSBI.
Quoted 7 lines
Quoted 3 lines
I did not find any info in the "${CROSS_COMPILE}gcc -v" output.
Me neither.
Quoted 1 line
Quoted 8 lines
Quoted 3 lines
Maybe also add some comment in objects.mk that GCC bare-metal toolchain
does not have PIE enabled.Or we can just document if using bare-metal toolchain
FW_PIChas to
be
set to n in the build.Sounds good.
We should document it under section "Required Toolchain" of top- level
README.md. Basically, we prefer toolchains with PIE support and for
toolchains not having PIE users have to pass the "FW_PIC=n" option.Also, we should replace references of bare-metal toolchain prefix with
linux toolchain prefix everywhere in documentation.Regards,
Anup
Quoted 7 lines
Quoted 4 lines
We can add the following code to the makefile to detect pie support
pie_support=$(shell $(CC) -nostdlib -fPIE -Wl,-pie
/dev/null2>/dev/null && echo y || echo n)That doesn?t work,
/dev/nullis not a valid object file so this will always fail with any toolchain.
I have tested it, so that the following commands no longer report errors
CROSS_COMPILE=riscv64-unknown-elf- make PLATFORM=generic
Xiang W
Quoted 2 lines
Jess
PATCH v4 3/5 firmware: Explicitly pass -pie to the linker, not just the driver
Quoted 151 lines · Xiang W
? 2021-07-10???? 19:10 +0100?Jessica Clarke???
Quoted 149 lines · Xiang W
Quoted 147 lines
? 2021-07-10???? 20:26 +0530?Anup Patel???
Quoted 135 lines
Quoted 98 lines · Bin Meng
Quoted 94 lines · Anup Patel
Quoted 90 lines · Bin Meng
Hi Anup,
Quoted 66 lines · Anup Patel
Hi Bin,
Quoted 51 lines · Bin Meng
Quoted 31 lines · Bin Meng
Quoted 28 lines · Jessica Clarke
When using Clang with a bare-metal triple, -pie does
not get passed to
the linker as it's not normally a thing that makes sense. However, in
our case it is, and manually forwarding it on works as
desired, so do so
to fully supportFW_PICwith Clang, including when linking with LLD.--- firmware/objects.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)firmware/objects.mk +1
-1@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y) firmware-genflags-y += -DFW_PIC firmware-asflags-y += -fpic firmware-cflags-y += -fPIE -pie -firmware-ldflags-y += -Wl,--no-dynamic-linker +firmware-ldflags-y += -Wl,--no-dynamic-linker -Wl,-
pieQuoted 8 lines
Quoted 1 line
endif
Does this manual forwarding also work for GNU ld? If so,
I think we
don't need to detect bare-metal triple and turn offFW_PICinMakefile?I just built a riscv64-unknown-elf-gcc toolchain and used
it to build
the current HEAD of opensbi/master. Indeed it's broken that
pie is not
supported with the bare-metal triple.Quoted 6 lines
The GNU ld simply complains:
ELF platform/generic/firmware/payloads/test.elf
/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
-pie not supportedThe
firmware/objects.mkis doing the following:Quoted 8 lines
ifndef
FW_PICFW_PIC:= y
endifInstead of above, we should set FW_PIC=y only when the underlying
toolchain supports pie.Agree.
Quoted 5 lines
We need a patch for this to be merged before we can merge this
series. Can you send such a patch ? If not then I can send
it.Do you have some reliable ways to check whether a toolchain supports PIE?
Quoted 7 lines
Quoted 4 lines
How about checking "-linux-" in
CROSS_COMPILEprefix ? If it's
available then
we set FW_PIC=y else we set FW_PIC=n.That works for cross-compile toolchains. But how about native toolchains (building OpenSBI on a RISC-V machine)?
Yes, it will not work for native compilation of OpenSBI.
Quoted 7 lines
Quoted 3 lines
I did not find any info in the "${CROSS_COMPILE}gcc -v" output.
Me neither.
Quoted 1 line
Quoted 8 lines
Quoted 3 lines
Maybe also add some comment in objects.mk that GCC bare-metal toolchain
does not have PIE enabled.Or we can just document if using bare-metal toolchain
FW_PIChas to
be
set to n in the build.Sounds good.
We should document it under section "Required Toolchain" of top- level
README.md. Basically, we prefer toolchains with PIE support and for
toolchains not having PIE users have to pass the "FW_PIC=n" option.Quoted 6 lines
Also, we should replace references of bare-metal toolchain prefix with
linux toolchain prefix everywhere in documentation.Regards,
AnupWe can add the following code to the makefile to detect pie support
pie_support=$(shell $(CC) -nostdlib -fPIE -Wl,-pie
/dev/null2>/dev/null && echo y || echo n)
Quoted 6 lines
Quoted 2 lines
That doesn?t work,
/dev/nullis not a valid object file so this will always fail with any toolchain.I have tested it, so that the following commands no longer report errors
CROSS_COMPILE=riscv64-unknown-elf- make PLATFORM=generic
It doesn?t work with Clang. It will always give an error as it regards /dev/null as being an object file. You need to force it to parse /dev/null as a C file with -x c. The same thing also happens with riscv64-unknown-freebsd12.1-gcc. I don?t know what?s special about riscv64-unknown-elf-gcc, though that *does* seem to do the right thing currently, but only that. Even my native x86_64-linux-gnu-gcc treats /dev/null as an object file. So something weird is going on in GCC land, but -x c should ensure you always get what you want.
Jess
PATCH v4 4/5 Makefile: Support building with Clang and LLVM binutils
In reply to Jessica ClarkeThis is intended to mirror the Linux kernel. Building with CC=clang will use Clang as the compiler but default to using the existing binutils. Building with LLVM=1 will default to using Clang and LLVM binutils.
Whilst GCC will accept the -N linker option and forward it on to the linker, Clang will not, and so in order to support both compilers we must use -Wl, to forward it to the linker as is required for most other linker options.
--- Makefile | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++----- README.md | 43 ++++++++++++++++++++++++++++++++++-- 2 files changed, 100 insertions(+), 8 deletions(-)
Makefile +59 -6
@@ -76,26 +76,54 @@ OPENSBI_VERSION_MINOR=`grep "define OPENSBI_VERSION_MINOR" $(include_dir)/sbi/sb OPENSBI_VERSION_GIT=$(shell if [ -d $(src_dir)/.git ]; then git describe 2> /dev/null; fi) # Setup compilation commands +ifneq ($(LLVM),) +CC = clang +AR = llvm-ar +LD = ld.lld +OBJCOPY = llvm-objcopy +else ifdef CROSS_COMPILE CC = $(CROSS_COMPILE)gcc -CPP = $(CROSS_COMPILE)cpp AR = $(CROSS_COMPILE)ar LD = $(CROSS_COMPILE)ld OBJCOPY = $(CROSS_COMPILE)objcopy else CC ?= gcc -CPP ?= cpp AR ?= ar LD ?= ld OBJCOPY ?= objcopy endif +endif +CPP = $(CC) -E AS = $(CC) DTC = dtc -# Guess the compillers xlen -OPENSBI_CC_XLEN := $(shell TMP=`$(CC) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP}) +ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),) +CC_IS_CLANG = y +else +CC_IS_CLANG = n +endif + +ifneq ($(shell $(LD) --version 2>&1 | head -n 1 | grep LLD),) +LD_IS_LLD = y +else +LD_IS_LLD = n +endif + +ifeq ($(CC_IS_CLANG),y) +ifneq ($(CROSS_COMPILE),) +CLANG_TARGET = -target $(notdir $(CROSS_COMPILE:%-=%)) +endif +endif + +# Guess the compiler's XLEN +OPENSBI_CC_XLEN := $(shell TMP=`$(CC) $(CLANG_TARGET) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP}) + +# Guess the compiler's ABI and ISA +ifneq ($(CC_IS_CLANG),y) OPENSBI_CC_ABI := $(shell TMP=`$(CC) -v 2>&1 | sed -n 's/.*\(with\-abi=\([a-zA-Z0-9]*\)\).*/\2/p'`; echo $${TMP}) OPENSBI_CC_ISA := $(shell TMP=`$(CC) -v 2>&1 | sed -n 's/.*\(with\-arch=\([a-zA-Z0-9]*\)\).*/\2/p'`; echo $${TMP}) +endif # Setup platform XLEN ifndef PLATFORM_RISCV_XLEN
@@ -106,6 +134,12 @@ ifndef PLATFORM_RISCV_XLEN endif endif +ifeq ($(CC_IS_CLANG),y) +ifeq ($(CROSS_COMPILE),) +CLANG_TARGET = -target riscv$(PLATFORM_RISCV_XLEN)-unknown-elf +endif +endif + # Setup list of objects.mk files ifdef PLATFORM platform-object-mks=$(shell if [ -d $(platform_src_dir)/ ]; then find $(platform_src_dir) -iname "objects.mk" | sort -r; fi)
@@ -194,7 +228,11 @@ else endif # Setup compilation commands flags -GENFLAGS = -I$(platform_src_dir)/include +ifeq ($(CC_IS_CLANG),y) +GENFLAGS += $(CLANG_TARGET) +GENFLAGS += -Wno-unused-command-line-argument +endif +GENFLAGS += -I$(platform_src_dir)/include GENFLAGS += -I$(include_dir) ifneq ($(OPENSBI_VERSION_GIT),) GENFLAGS += -DOPENSBI_VERSION_GIT="\"$(OPENSBI_VERSION_GIT)\""
@@ -208,6 +246,9 @@ CFLAGS += -fno-omit-frame-pointer -fno-optimize-sibling-calls CFLAGS += -mno-save-restore -mstrict-align CFLAGS += -mabi=$(PLATFORM_RISCV_ABI) -march=$(PLATFORM_RISCV_ISA) CFLAGS += -mcmodel=$(PLATFORM_RISCV_CODE_MODEL) +ifeq ($(LD_IS_LLD),y) +CFLAGS += -mno-relax +endif CFLAGS += $(GENFLAGS) CFLAGS += $(platform-cflags-y) CFLAGS += -fno-pie -no-pie
@@ -222,18 +263,30 @@ ASFLAGS += -fno-omit-frame-pointer -fno-optimize-sibling-calls ASFLAGS += -mno-save-restore -mstrict-align ASFLAGS += -mabi=$(PLATFORM_RISCV_ABI) -march=$(PLATFORM_RISCV_ISA) ASFLAGS += -mcmodel=$(PLATFORM_RISCV_CODE_MODEL) +ifeq ($(LD_IS_LLD),y) +ASFLAGS += -mno-relax +endif ASFLAGS += $(GENFLAGS) ASFLAGS += $(platform-asflags-y) ASFLAGS += $(firmware-asflags-y) ARFLAGS = rcs -ELFFLAGS += -Wl,--build-id=none -N -static-libgcc -lgcc +ifeq ($(LD_IS_LLD),y) +ELFFLAGS += -fuse-ld=lld +else +ELFFLAGS += -fuse-ld=bfd +endif +ELFFLAGS += -Wl,--build-id=none -Wl,-N -static-libgcc -lgcc ELFFLAGS += $(platform-ldflags-y) ELFFLAGS += $(firmware-ldflags-y) MERGEFLAGS += -r +ifeq ($(LD_IS_LLD),y) +MERGEFLAGS += -b elf +else MERGEFLAGS += -b elf$(PLATFORM_RISCV_XLEN)-littleriscv +endif MERGEFLAGS += -m elf$(PLATFORM_RISCV_XLEN)lriscv DTSCPPFLAGS = $(CPPFLAGS) -nostdinc -nostdlib -fno-builtin -D__DTS__ -x assembler-with-cpp
README.md +41 -2
55 lines not on this page · raw message
Signature · 1 line
2.31.0
PATCH v4 4/5 Makefile: Support building with Clang and LLVM binutils
Quoted 10 lines · Jessica Clarke
This is intended to mirror the Linux kernel. Building with CC=clang will use Clang as the compiler but default to using the existing binutils. Building with LLVM=1 will default to using Clang and LLVM binutils.
Whilst GCC will accept the -N linker option and forward it on to the linker, Clang will not, and so in order to support both compilers we must use -Wl, to forward it to the linker as is required for most other linker options.
Quoted 6 lines
Signed-off-by Jessica Clarke <[email protected]>--- Makefile | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++----- README.md | 43 ++++++++++++++++++++++++++++++++++-- 2 files changed, 100 insertions(+), 8 deletions(-)
Here are test results:
Building with "riscv64-linux-gcc",
$ file build/platform/generic/firmware/fw_dynamic.elf build/platform/generic/firmware/fw_dynamic.elf: ELF 64-bit LSB executable, UCB RISC-V, version 1 (SYSV), dynamically linked, with debug_info, not stripped
$ riscv64-linux-readelf -r build/platform/generic/firmware/fw_dynamic.elf Relocation section '.rela.dyn' at offset 0x140f8 contains 184 entries: Offset Info Type Sym. Value Sym. Name + Addend 000080013090 000000000003 R_RISCV_RELATIVE 80000db4 ... 000080013628 000200000002 R_RISCV_64 0000000080013720 fdt_serial_uart8250 + 0 000080013830 000d00000002 R_RISCV_64 00000000800138d8 fdt_reset_sifive_test + 0
Building with "LLVM=1",
$ file build/platform/generic/firmware/fw_dynamic.elf build/platform/generic/firmware/fw_dynamic.elf: ELF 64-bit LSB shared object, UCB RISC-V, version 1 (SYSV), dynamically linked, with debug_info, not stripped
$ riscv64-linux-readelf -r build/platform/generic/firmware/fw_dynamic.elf Relocation section '.rela.dyn' at offset 0x17d98 contains 188 entries: Offset Info Type Sym. Value Sym. Name + Addend 000080017000 000000000003 R_RISCV_RELATIVE 8000b680 000080017030 000000000003 R_RISCV_RELATIVE 8001b1b8 ... 000080017c90 000000000003 R_RISCV_RELATIVE 80017628
There are two differences:
1. LLVM toolchain generates a "shared object" firmware image, while GCC generates "executable".
2. LLVM one has 4 more entries in .rela.dyn than the GCC. All entries of LLVM have the R_RISCV_RELATIVE type, but GCC one has two R_RISCV_64 entries.
I am not sure whether GCC / LLVM is doing things correctly for the above 2 differences. fw_dynamic image of both can boot to S-mode U-Boot on QEMU 'virt' though.
Using clang and GNU binutils, fw_dynamic image does not boot on QEMU 'virt', as reported before.
$ make CC=clang CROSS_COMPILE=riscv64-linux- PLATFORM=generic
Regards,
Bin
PATCH v4 4/5 Makefile: Support building with Clang and LLVM binutils
Quoted 53 lines · Bin Meng
Quoted 10 lines · Jessica Clarke
This is intended to mirror the Linux kernel. Building with CC=clang will use Clang as the compiler but default to using the existing binutils. Building with LLVM=1 will default to using Clang and LLVM binutils.
Whilst GCC will accept the -N linker option and forward it on to the linker, Clang will not, and so in order to support both compilers we must use -Wl, to forward it to the linker as is required for most other linker options.
Quoted 6 lines
Signed-off-by Jessica Clarke <[email protected]>--- Makefile | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++----- README.md | 43 ++++++++++++++++++++++++++++++++++-- 2 files changed, 100 insertions(+), 8 deletions(-)Here are test results:
Building with "riscv64-linux-gcc",
$ file build/platform/generic/firmware/fw_dynamic.elf build/platform/generic/firmware/fw_dynamic.elf: ELF 64-bit LSB executable, UCB RISC-V, version 1 (SYSV), dynamically linked, with debug_info, not stripped$ riscv64-linux-readelf -r build/platform/generic/firmware/fw_dynamic.elf Relocation section '.rela.dyn' at offset 0x140f8 contains 184 entries: Offset Info Type Sym. Value Sym. Name + Addend 000080013090 000000000003 R_RISCV_RELATIVE 80000db4 ... 000080013628 000200000002 R_RISCV_64 0000000080013720 fdt_serial_uart8250 + 0 000080013830 000d00000002 R_RISCV_64 00000000800138d8 fdt_reset_sifive_test + 0Building with "LLVM=1",
$ file build/platform/generic/firmware/fw_dynamic.elf build/platform/generic/firmware/fw_dynamic.elf: ELF 64-bit LSB shared object, UCB RISC-V, version 1 (SYSV), dynamically linked, with debug_info, not stripped$ riscv64-linux-readelf -r build/platform/generic/firmware/fw_dynamic.elf Relocation section '.rela.dyn' at offset 0x17d98 contains 188 entries: Offset Info Type Sym. Value Sym. Name + Addend 000080017000 000000000003 R_RISCV_RELATIVE 8000b680 000080017030 000000000003 R_RISCV_RELATIVE 8001b1b8 ... 000080017c90 000000000003 R_RISCV_RELATIVE 80017628
Quoted 7 lines
There are two differences:
1. LLVM toolchain generates a "shared object" firmware image, while GCC generates "executable".
2. LLVM one has 4 more entries in .rela.dyn than the GCC. All entries of LLVM have theR_RISCV_RELATIVEtype, but GCC one has twoR_RISCV_64entries.
Do you have any explanations on these 2 differences? Are these possible toolchain bugs?
Quoted 8 lines
I am not sure whether GCC / LLVM is doing things correctly for the above 2 differences.
fw_dynamicimage of both can boot to S-mode U-Boot on QEMU 'virt' though.Using clang and GNU binutils,
fw_dynamicimage does not boot on QEMU 'virt', as reported before.$ make CC=clang CROSS_COMPILE=riscv64-linux- PLATFORM=generic
Regards,
Bin
PATCH v4 4/5 Makefile: Support building with Clang and LLVM binutils
Quoted 57 lines · Bin Meng
Quoted 55 lines · Bin Meng
Quoted 10 lines · Jessica Clarke
This is intended to mirror the Linux kernel. Building with CC=clang will use Clang as the compiler but default to using the existing binutils. Building with LLVM=1 will default to using Clang and LLVM binutils.
Whilst GCC will accept the -N linker option and forward it on to the linker, Clang will not, and so in order to support both compilers we must use -Wl, to forward it to the linker as is required for most other linker options.
Quoted 6 lines
Signed-off-by Jessica Clarke <[email protected]>---
Makefile| 65 ++++++++++++++++++++++++++++++++++++++++++++++++++----- README.md | 43 ++++++++++++++++++++++++++++++++++--2 files changed, 100 insertions(+), 8 deletions(-)Here are test results:
Building with "riscv64-linux-gcc",
$ file build/platform/generic/firmware/fw_dynamic.elf build/platform/generic/firmware/fw_dynamic.elf: ELF 64-bit LSB executable, UCB RISC-V, version 1 (SYSV), dynamically linked, with debug_info, not stripped$ riscv64-linux-readelf -r build/platform/generic/firmware/fw_dynamic.elf Relocation section '.rela.dyn' at offset 0x140f8 contains 184 entries: Offset Info Type Sym. Value Sym. Name + Addend 000080013090 000000000003 R_RISCV_RELATIVE 80000db4 ... 000080013628 000200000002 R_RISCV_64 0000000080013720 fdt_serial_uart8250 + 0 000080013830 000d00000002 R_RISCV_64 00000000800138d8 fdt_reset_sifive_test + 0Building with "LLVM=1",
$ file build/platform/generic/firmware/fw_dynamic.elf build/platform/generic/firmware/fw_dynamic.elf: ELF 64-bit LSB shared object, UCB RISC-V, version 1 (SYSV), dynamically linked, with debug_info, not stripped$ riscv64-linux-readelf -r build/platform/generic/firmware/fw_dynamic.elf Relocation section '.rela.dyn' at offset 0x17d98 contains 188 entries: Offset Info Type Sym. Value Sym. Name + Addend 000080017000 000000000003 R_RISCV_RELATIVE 8000b680 000080017030 000000000003 R_RISCV_RELATIVE 8001b1b8 ... 000080017c90 000000000003 R_RISCV_RELATIVE 80017628There are two differences:
Quoted 8 lines
Quoted 5 lines
1. LLVM toolchain generates a "shared object" firmware image, while GCC generates "executable".
2. LLVM one has 4 more entries in .rela.dyn than the GCC. All entries of LLVM have theR_RISCV_RELATIVEtype, but GCC one has twoR_RISCV_64entries.Do you have any explanations on these 2 differences? Are these possible toolchain bugs?
[Hm, I composed this on the 11th but seems I never sent it...]
The first one smells to me like GNU ld is wrong, as executables are inherently not position-independent, PIEs are always shared objects, and I?m pretty sure that?s true of the recent -static-pie support too. In practice for our use cases it doesn?t matter though. It seems this is yet another undocumented, likely unintended (as it?s a very old legacy, and mostly unused, option) consequence of using -N/--omagic (in that it just blindly sets various flags internally and nobody thought about whether that made sense once -pie was added). Incidentally, I don?t think we actually need -N/--omagic any more, but that?s a separate thing.
For the minor difference in number of relocations, that probably just comes down to minor codegen differences and I wouldn?t worry about it; with a large enough code base small differences are to be expected.
As for R_RISCV_64, there?s no reason for GNU ld to emit R_RISCV_64 here. It?s technically correct but entirely unnecessary (and can break legitimate code that assumes only R_RISCV_RELATIVE gets emitted, which *should* be the case; kernel and run-time linker self-relocation code, that looks a lot like what OpenSBI is doing here, often likes to assume that, possibly with R_RISCV_IRELATIVE too if IFUNCs are used). I see it locally for fdt_serial_uart8250 and fdt_reset_sifive which should in no way be special, there?s nothing stopping those being evaluated at link time and leaving R_RISCV_RELATIVE to adjust them at run time as needed like with all the other symbols.
So I?d regard -N + -pie giving EXEC not DYN, and R_RISCV_64 being emitted here, as being GNU ld sort-of-bugs, albeit with the former being extremely ill-defined over what that combination means (beyond ?it does what it does?), with LLD?s output for both being what I would expect.
Jess
Quoted 10 lines
Quoted 7 lines
I am not sure whether GCC / LLVM is doing things correctly for the above 2 differences.
fw_dynamicimage of both can boot to S-mode U-Boot on QEMU 'virt' though.Using clang and GNU binutils,
fw_dynamicimage does not boot on QEMU 'virt', as reported before.$ make CC=clang CROSS_COMPILE=riscv64-linux- PLATFORM=genericRegards,
Bin
PATCH v4 4/5 Makefile: Support building with Clang and LLVM binutils
Quoted 81 lines · Jessica Clarke
Quoted 57 lines · Bin Meng
Quoted 55 lines · Bin Meng
Quoted 10 lines · Jessica Clarke
This is intended to mirror the Linux kernel. Building with CC=clang will use Clang as the compiler but default to using the existing binutils. Building with LLVM=1 will default to using Clang and LLVM binutils.
Whilst GCC will accept the -N linker option and forward it on to the linker, Clang will not, and so in order to support both compilers we must use -Wl, to forward it to the linker as is required for most other linker options.
Quoted 6 lines
Signed-off-by Jessica Clarke <[email protected]>---
Makefile| 65 ++++++++++++++++++++++++++++++++++++++++++++++++++----- README.md | 43 ++++++++++++++++++++++++++++++++++--2 files changed, 100 insertions(+), 8 deletions(-)Here are test results:
Building with "riscv64-linux-gcc",
$ file build/platform/generic/firmware/fw_dynamic.elf build/platform/generic/firmware/fw_dynamic.elf: ELF 64-bit LSB executable, UCB RISC-V, version 1 (SYSV), dynamically linked, with debug_info, not stripped$ riscv64-linux-readelf -r build/platform/generic/firmware/fw_dynamic.elf Relocation section '.rela.dyn' at offset 0x140f8 contains 184 entries: Offset Info Type Sym. Value Sym. Name + Addend 000080013090 000000000003 R_RISCV_RELATIVE 80000db4 ... 000080013628 000200000002 R_RISCV_64 0000000080013720 fdt_serial_uart8250 + 0 000080013830 000d00000002 R_RISCV_64 00000000800138d8 fdt_reset_sifive_test + 0Building with "LLVM=1",
$ file build/platform/generic/firmware/fw_dynamic.elf build/platform/generic/firmware/fw_dynamic.elf: ELF 64-bit LSB shared object, UCB RISC-V, version 1 (SYSV), dynamically linked, with debug_info, not stripped$ riscv64-linux-readelf -r build/platform/generic/firmware/fw_dynamic.elf Relocation section '.rela.dyn' at offset 0x17d98 contains 188 entries: Offset Info Type Sym. Value Sym. Name + Addend 000080017000 000000000003 R_RISCV_RELATIVE 8000b680 000080017030 000000000003 R_RISCV_RELATIVE 8001b1b8 ... 000080017c90 000000000003 R_RISCV_RELATIVE 80017628There are two differences:
Quoted 8 lines
Quoted 5 lines
1. LLVM toolchain generates a "shared object" firmware image, while GCC generates "executable".
2. LLVM one has 4 more entries in .rela.dyn than the GCC. All entries of LLVM have theR_RISCV_RELATIVEtype, but GCC one has twoR_RISCV_64entries.Do you have any explanations on these 2 differences? Are these possible toolchain bugs?
[Hm, I composed this on the 11th but seems I never sent it...]
The first one smells to me like GNU ld is wrong, as executables are inherently not position-independent, PIEs are always shared objects, and I?m pretty sure that?s true of the recent -static-pie support too. In practice for our use cases it doesn?t matter though. It seems this is yet another undocumented, likely unintended (as it?s a very old legacy, and mostly unused, option) consequence of using -N/--omagic (in that it just blindly sets various flags internally and nobody thought about whether that made sense once -pie was added). Incidentally, I don?t think we actually need -N/--omagic any more, but that?s a separate thing.
Quoted 7 lines
For the minor difference in number of relocations, that probably just comes down to minor codegen differences and I wouldn?t worry about it; with a large enough code base small differences are to be expected.
As for
R_RISCV_64, there?s no reason for GNU ld to emitR_RISCV_64here. It?s technically correct but entirely unnecessary (and can break legitimate code that assumes onlyR_RISCV_RELATIVEgets emitted, which
Fortunately OpenSBI, as well as U-Boot, handle both R_RISCV_RELATIVE and R_RISCV_{64,32} here, so they are not broken due to these unnecessary entries.
Quoted 8 lines
*should* be the case; kernel and run-time linker self-relocation code, that looks a lot like what OpenSBI is doing here, often likes to assume that, possibly with
R_RISCV_IRELATIVEtoo if IFUNCs are used). I see it locally forfdt_serial_uart8250andfdt_reset_sifivewhich should in no way be special, there?s nothing stopping those being evaluated at link time and leavingR_RISCV_RELATIVEto adjust them at run time as needed like with all the other symbols.
Quoted 5 lines
So I?d regard -N + -pie giving EXEC not DYN, and
R_RISCV_64being emitted here, as being GNU ld sort-of-bugs, albeit with the former being extremely ill-defined over what that combination means (beyond ?it does what it does?), with LLD?s output for both being what I would expect.
Thanks a lot for the explanation! Someone can file a defect to the GNU ld :)
Regards,
Bin
PATCH v4 5/5 Drop dependency on libgcc by importing part of FreeBSD's libquad
In reply to Jessica ClarkeWe only need libgcc for 64-bit division on RV32. Whilst GCC toolchains bundle libgcc, Clang toolchains tend not to ship libclang_rt.builtins given every compiler is a cross-compiler for every target and so you would need a silly number of builds of it, with only the native library available; only vendor-provided Clang toolchains specifically for bare metal cross-compiling are likely to provide it.
Thus, import part of FreeBSD's implementation of the division support functions needed and stop linking against libgcc.
--- Makefile | 2 +- lib/utils/libquad/divdi3.c | 64 ++++++ lib/utils/libquad/include/limits.h | 12 ++ lib/utils/libquad/include/sys/cdefs.h | 12 ++ lib/utils/libquad/include/sys/types.h | 25 +++ lib/utils/libquad/moddi3.c | 66 ++++++ lib/utils/libquad/objects.mk | 14 ++ lib/utils/libquad/qdivrem.c | 278 ++++++++++++++++++++++++++ lib/utils/libquad/quad.h | 105 ++++++++++ lib/utils/libquad/udivdi3.c | 52 +++++ lib/utils/libquad/umoddi3.c | 54 +++++ 11 files changed, 683 insertions(+), 1 deletion(-)
Makefile +1 -1
@@ -277,7 +277,7 @@ ELFFLAGS += -fuse-ld=lld else ELFFLAGS += -fuse-ld=bfd endif -ELFFLAGS += -Wl,--build-id=none -Wl,-N -static-libgcc -lgcc +ELFFLAGS += -Wl,--build-id=none -Wl,-N ELFFLAGS += $(platform-ldflags-y) ELFFLAGS += $(firmware-ldflags-y)
lib/utils/libquad/divdi3.c new file +64 -0
64 lines not on this page · raw message
lib/utils/libquad/include/limits.h new file +12 -0
@@ -0,0 +1,12 @@ +/* +* SPDX-License-Identifier: BSD-2-Clause +* +* Copyright (c) 2021 Jessica Clarke <[email protected]> +*/ + +#ifndef __LIMITS_H__ +#define __LIMITS_H__ + +#define CHAR_BIT 8 + +#endif
lib/utils/libquad/include/sys/cdefs.h new file +12 -0
@@ -0,0 +1,12 @@ +/* +* SPDX-License-Identifier: BSD-2-Clause +* +* Copyright (c) 2021 Jessica Clarke <[email protected]> +*/ + +#ifndef __SYS_CDEFS_H__ +#define __SYS_CDEFS_H__ + +#define __FBSDID (s) struct __hack + +#endif
lib/utils/libquad/include/sys/types.h new file +25 -0
@@ -0,0 +1,25 @@ +/* +* SPDX-License-Identifier: BSD-2-Clause +* +* Copyright (c) 2021 Jessica Clarke <[email protected]> +*/ + +#ifndef __SYS_TYPES_H__ +#define __SYS_TYPES_H__ + +#include <sbi/sbi_types.h> + +typedef unsigned long u_long ; + +typedef int64_t quad_t ; +typedef uint64_t u_quad_t ; + +#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__ +#define _QUAD_LOWWORD 1 +#define _QUAD_HIGHWORD 0 +#else +#define _QUAD_LOWWORD 0 +#define _QUAD_HIGHWORD 1 +#endif + +#endif
lib/utils/libquad/moddi3.c new file +66 -0
66 lines not on this page · raw message
lib/utils/libquad/objects.mk new file +14 -0
@@ -0,0 +1,14 @@ +# +# SPDX-License-Identifier: BSD-2-Clause +# +# Copyright (c) 2021 Jessica Clarke <[email protected]> +# + +ifeq ($(PLATFORM_RISCV_XLEN),32) +libsbiutils-objs-y += libquad/divdi3.o +libsbiutils-objs-y += libquad/moddi3.o +libsbiutils-objs-y += libquad/qdivrem.o +libsbiutils-objs-y += libquad/udivdi3.o +libsbiutils-objs-y += libquad/umoddi3.o +libsbiutils-genflags-y += -I$(libsbiutils_dir)/libquad/include +endif
lib/utils/libquad/qdivrem.c new file +278 -0
278 lines not on this page · raw message
lib/utils/libquad/quad.h new file +105 -0
105 lines not on this page · raw message
lib/utils/libquad/udivdi3.c new file +52 -0
52 lines not on this page · raw message
lib/utils/libquad/umoddi3.c new file +54 -0
54 lines not on this page · raw message
Signature · 1 line
2.31.0
v4 cover Fully support standalone Clang/LLVM toolchains
In reply to Jessica ClarkeQuoted 80 lines · Jessica Clarke
This patch series is comprised of five parts.
The first patch fixes a warning seen when building with LLVM due to a bogus combination of assembly directives.
The second patch fixes errors seen when trying to build OpenSBI as a position-independent binary using LLD that may or may not be an LLD bug (the exact meaning of -N/--omagic isn't clear) but can easily be worked around without any issue either way.
The third patch bypasses the Clang driver's helpful nature of not honouring -pie for bare-metal binaries as it's normally not something you want (arguably that should be an error though, or passed on, since just giving a -Wunused-command-line-argument warning can get lost, and may be disabled).
The fourth patch adds build system support for using Clang and LLVM binutils, provided your Clang is able to locate a libgcc.a in its search path.
However, pure LLVM toolchains do not use libgcc, they use compiler-rt (libclang_rt.builtins.<arch>.a). We could change the
Makefileto not hard-code -lgcc and instead use -print-libgcc-file-name, but that still requires a bare metal compiler-rt built for the right -march/-mabi to be present, which is often not the case. Moreover, we need very little from libgcc/compiler-rt; RV64 needs nothing, and RV32 only needs 64-bit division. Thus, the fifth patch vendors part of FreeBSD's libquad and stops linking against libgcc entirely, allowing OpenSBI to be built with just a cross-compiler. This means that building with any distro-provided LLVM just works, as does compiling with the system Clang compiler and LLD linker on FreeBSD without any external packages needed (beyond GNU make).Changes in v4:
* Avoid read-only dynamic relocation error with LLD
* SupportFW_PICwith LLD when using a bare-metal tripleChanges in v3:
* Fixed the binding warning seen when building with LLVM
* Provide a default -target whenCROSS_COMPILEisn't set so the variable isn't normally required to be set when cross-compiling with LLVM (but is still honoured if set).
* Pass -fuse=bfd when not using LLD to ensure an ld.bfd gets correctly picked up over a plain ld when ld is LLD. Now we always explicitly request Clang and GCC use the linker type corresponding to LD. This allows BFD to be used on FreeBSD for testing purposes despite LLD being the system linker.Changes in v2:
* Add documentation to README.md
* Pass -fuse-ld=lld to Clang when using LLD
* Modify commit message of first commit to explain -N -> -Wl,-N change
* Bring back the old ?= uses for the non-CROSS_COMPILE (and, now, non-LLVM) case; whilst I still think that's how it should be, it's not required for this patch, just related cleanup I did to reduce the complexity of adding the LLVM case.Jessica Clarke (5):
fw_base: Don't markfw_platform_initas both global and weak
fw_base: Put data in .data rather than .text
firmware: Explicitly pass -pie to the linker, not just the driver
Makefile: Support building with Clang and LLVM binutils Drop dependency on libgcc by importing part of FreeBSD's libquadMakefile | 65 +++++- README.md | 43 +++- firmware/fw_base.S | 2 +- firmware/objects.mk | 2 +- lib/utils/libquad/divdi3.c | 64 ++++++ lib/utils/libquad/include/limits.h | 12 ++ lib/utils/libquad/include/sys/cdefs.h | 12 ++ lib/utils/libquad/include/sys/types.h | 25 +++ lib/utils/libquad/moddi3.c | 66 ++++++ lib/utils/libquad/objects.mk | 14 ++ lib/utils/libquad/qdivrem.c | 278 ++++++++++++++++++++++++++ lib/utils/libquad/quad.h | 105 ++++++++++ lib/utils/libquad/udivdi3.c | 52 +++++ lib/utils/libquad/umoddi3.c | 54 +++++ 14 files changed, 784 insertions(+), 10 deletions(-)
Quoted 10 lines
create mode 100644
lib/utils/libquad/divdi3.ccreate mode 100644lib/utils/libquad/include/limits.hcreate mode 100644lib/utils/libquad/include/sys/cdefs.hcreate mode 100644lib/utils/libquad/include/sys/types.hcreate mode 100644lib/utils/libquad/moddi3.ccreate mode 100644lib/utils/libquad/objects.mkcreate mode 100644lib/utils/libquad/qdivrem.ccreate mode 100644lib/utils/libquad/quad.hcreate mode 100644lib/utils/libquad/udivdi3.ccreate mode 100644lib/utils/libquad/umoddi3.c
... all Signed-off-by: Jessica Clarke <[email protected]>, managed to forget that yet again :(
Jess