Skip to contentSkip to conversation

Fully support standalone Clang/LLVM toolchains

PATCH Superseded Jessica Clarke 4 participants 24 messages Jul 9, 2021 – Jul 21, 2021 Kbuild

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v4 cover Fully support standalone Clang/LLVM toolchains

This patch series is comprised of five parts.

The first patch fixes a warning seen when building with LLVM due to a bogus combination of assembly directives.

The second patch fixes errors seen when trying to build OpenSBI as a position-independent binary using LLD that may or may not be an LLD bug (the exact meaning of -N/--omagic isn't clear) but can easily be worked around without any issue either way.

The third patch bypasses the Clang driver's helpful nature of not honouring -pie for bare-metal binaries as it's normally not something you want (arguably that should be an error though, or passed on, since just giving a -Wunused-command-line-argument warning can get lost, and may be disabled).

The fourth patch adds build system support for using Clang and LLVM binutils, provided your Clang is able to locate a libgcc.a in its search path.

However, pure LLVM toolchains do not use libgcc, they use compiler-rt (libclang_rt.builtins.<arch>.a). We could change the Makefile to not hard-code -lgcc and instead use -print-libgcc-file-name, but that still requires a bare metal compiler-rt built for the right -march/-mabi to be present, which is often not the case. Moreover, we need very little from libgcc/compiler-rt; RV64 needs nothing, and RV32 only needs 64-bit division. Thus, the fifth patch vendors part of FreeBSD's libquad and stops linking against libgcc entirely, allowing OpenSBI to be built with just a cross-compiler. This means that building with any distro-provided LLVM just works, as does compiling with the system Clang compiler and LLD linker on FreeBSD without any external packages needed (beyond GNU make).

Changes in v4:
* Avoid read-only dynamic relocation error with LLD
* Support FW_PIC with LLD when using a bare-metal triple

Changes in v3:
* Fixed the binding warning seen when building with LLVM
* Provide a default -target when CROSS_COMPILE isn't set so the variable isn't normally required to be set when cross-compiling with LLVM (but is still honoured if set).
* Pass -fuse=bfd when not using LLD to ensure an ld.bfd gets correctly picked up over a plain ld when ld is LLD. Now we always explicitly request Clang and GCC use the linker type corresponding to LD. This allows BFD to be used on FreeBSD for testing purposes despite LLD being the system linker.

Changes in v2:
* Add documentation to README.md
* Pass -fuse-ld=lld to Clang when using LLD
* Modify commit message of first commit to explain -N -> -Wl,-N change
* Bring back the old ?= uses for the non-CROSS_COMPILE (and, now, non-LLVM) case; whilst I still think that's how it should be, it's not required for this patch, just related cleanup I did to reduce the complexity of adding the LLVM case.

Jessica Clarke (5):
fw_base: Don't mark fw_platform_init as both global and weak
fw_base: Put data in .data rather than .text
firmware: Explicitly pass -pie to the linker, not just the driver
Makefile: Support building with Clang and LLVM binutils Drop dependency on libgcc by importing part of FreeBSD's libquad

 Makefile                              |  65 +++++-
 README.md                             |  43 +++-
 firmware/fw_base.S                    |   2 +-
 firmware/objects.mk                   |   2 +-
 lib/utils/libquad/divdi3.c            |  64 ++++++
 lib/utils/libquad/include/limits.h    |  12 ++
 lib/utils/libquad/include/sys/cdefs.h |  12 ++
 lib/utils/libquad/include/sys/types.h |  25 +++
 lib/utils/libquad/moddi3.c            |  66 ++++++
 lib/utils/libquad/objects.mk          |  14 ++
 lib/utils/libquad/qdivrem.c           | 278 ++++++++++++++++++++++++++
 lib/utils/libquad/quad.h              | 105 ++++++++++
 lib/utils/libquad/udivdi3.c           |  52 +++++
 lib/utils/libquad/umoddi3.c           |  54 +++++
 14 files changed, 784 insertions(+), 10 deletions(-)
 create mode 100644 lib/utils/libquad/divdi3.c
 create mode 100644 lib/utils/libquad/include/limits.h
 create mode 100644 lib/utils/libquad/include/sys/cdefs.h
 create mode 100644 lib/utils/libquad/include/sys/types.h
 create mode 100644 lib/utils/libquad/moddi3.c
 create mode 100644 lib/utils/libquad/objects.mk
 create mode 100644 lib/utils/libquad/qdivrem.c
 create mode 100644 lib/utils/libquad/quad.h
 create mode 100644 lib/utils/libquad/udivdi3.c
 create mode 100644 lib/utils/libquad/umoddi3.c
Signature · 1 line
2.31.0
#1 lore raw

PATCH v4 1/5 fw_base: Don't mark fw_platform_init as both global and weak

These are mutually exclusive. GNU as and LLVM both let later binding directives override earlier ones so this works as intended, but LLVM 12 turned this into a warning as there's no good reason to do such a thing and could be a potential bug. Thus, remove the redundant and incorrect .globl directive for fw_platform_init.

---
 firmware/fw_base.S | 1 -
 1 file changed, 1 deletion(-)
firmware/fw_base.S +0 -1
index a5ce946..ee2a51b 100644
@@ -553,7 +553,6 @@ _start_hang:
 
 	.section .entry, "ax", %progbits
 	.align 3
-	.globl fw_platform_init
 	.weak fw_platform_init
 fw_platform_init:
 	add	a0, a1, zero
Signature · 1 line
2.31.0
#2 lore raw

PATCH v4 1/5 fw_base: Don't mark fw_platform_init as both global and weak

Quoted 10 lines · Jessica Clarke

These are mutually exclusive. GNU as and LLVM both let later binding directives override earlier ones so this works as intended, but LLVM 12 turned this into a warning as there's no good reason to do such a thing and could be a potential bug. Thus, remove the redundant and incorrect .globl directive for fw_platform_init.

---
 firmware/fw_base.S | 1 -
 1 file changed, 1 deletion(-)

You SoB tag was missed again:

And please include people's RB / TB tags in the newer version, if nothing changed:

#3 lore raw

PATCH v4 2/5 fw_base: Put data in .data rather than .text

In reply to Jessica Clarke

The -N linker option is supposed to make .text writable, but GNU ld and LLD differ in interpreting what that means. GNU ld will happily let you have relocations in it, but LLD will see that the input section is read-only (even though the output section is writable) and give an error. It's unclear if either of them intend to have that behaviour in this edge case, but regardless there's no reason not to just put the data in a writable .data section.

---
 firmware/fw_base.S | 1 +
 1 file changed, 1 insertion(+)
firmware/fw_base.S +1 -0
index ee2a51b..f8aea05 100644
@@ -501,6 +501,7 @@ _skip_trap_exit_rv32_hyp:
 	/* We don't expect to reach here hence just hang */
 	j	_start_hang
 
+	.data
 	.align 3
 #ifdef FW_PIC
 _runtime_offset:
Signature · 1 line
2.31.0
#4 lore raw

PATCH v4 2/5 fw_base: Put data in .data rather than .text

Quoted 8 lines · Jessica Clarke

The -N linker option is supposed to make .text writable, but GNU ld and LLD differ in interpreting what that means. GNU ld will happily let you have relocations in it, but LLD will see that the input section is read-only (even though the output section is writable) and give an error. It's unclear if either of them intend to have that behaviour in this edge case, but regardless there's no reason not to just put the data in a writable .data section.

Quoted 4 lines
---
 firmware/fw_base.S | 1 +
 1 file changed, 1 insertion(+)

Again, your SoB tag was missed:

With this patch the following error seen when building with full LLVM toolchain no longer exists:

ELF platform/generic/firmware/fw_dynamic.elf
ld.lld: error: can't create dynamic relocation R_RISCV_64 against
symbol: _fw_start in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

defined in opensbi/build/platform/generic/firmware/fw_dynamic.elf.ld:8 referenced by fw_base.S:502 (opensbi/firmware/fw_base.S:502)

              opensbi/build/platform/generic/firmware/fw_dynamic.o:(.entry+0x3A0)

ld.lld: error: can't create dynamic relocation R_RISCV_64 against
symbol: _fw_reloc_end in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

defined in opensbi/build/platform/generic/firmware/fw_dynamic.elf.ld:92 referenced by fw_base.S:502 (opensbi/firmware/fw_base.S:502)

              opensbi/build/platform/generic/firmware/fw_dynamic.o:(.entry+0x3B0)

clang-12: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Makefile:396:
opensbi/build/platform/generic/firmware/fw_dynamic.elf] Error 1

#5 lore raw

PATCH v4 3/5 firmware: Explicitly pass -pie to the linker, not just the driver

In reply to Jessica Clarke

When using Clang with a bare-metal triple, -pie does not get passed to the linker as it's not normally a thing that makes sense. However, in our case it is, and manually forwarding it on works as desired, so do so to fully support FW_PIC with Clang, including when linking with LLD.

---
 firmware/objects.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
firmware/objects.mk +1 -1
index ce91c2f..3bc83cd 100644
@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)
 firmware-genflags-y +=	-DFW_PIC
 firmware-asflags-y  +=	-fpic
 firmware-cflags-y   +=	-fPIE -pie
-firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
+firmware-ldflags-y  +=  -Wl,--no-dynamic-linker -Wl,-pie
 endif
 
 ifdef FW_TEXT_START
Signature · 1 line
2.31.0
#6 lore raw

PATCH v4 3/5 firmware: Explicitly pass -pie to the linker, not just the driver

Quoted 14 lines · Jessica Clarke

When using Clang with a bare-metal triple, -pie does not get passed to the linker as it's not normally a thing that makes sense. However, in our case it is, and manually forwarding it on works as desired, so do so to fully support FW_PIC with Clang, including when linking with LLD.

---
 firmware/objects.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
firmware/objects.mk +0 -0
index ce91c2f..3bc83cd 100644

firmware/objects.mk ifeq ($(FW_PIC),y) · line 25

Quoted 6 lines
firmware/objects.mk +1 -1
@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)
 firmware-genflags-y += -DFW_PIC
 firmware-asflags-y  += -fpic
 firmware-cflags-y   += -fPIE -pie
-firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
+firmware-ldflags-y  +=  -Wl,--no-dynamic-linker -Wl,-pie
 endif

Does this manual forwarding also work for GNU ld? If so, I think we don't need to detect bare-metal triple and turn off FW_PIC in Makefile?

Regards,
Bin

#7 lore raw

PATCH v4 3/5 firmware: Explicitly pass -pie to the linker, not just the driver

Quoted 11 lines · Bin Meng
Quoted 9 lines · Jessica Clarke

When using Clang with a bare-metal triple, -pie does not get passed to the linker as it's not normally a thing that makes sense. However, in our case it is, and manually forwarding it on works as desired, so do so to fully support FW_PIC with Clang, including when linking with LLD.

---
 firmware/objects.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 15 lines

firmware/objects.mk ifeq ($(FW_PIC),y) · line 25

Quoted 11 lines
firmware/objects.mk +1 -1
index ce91c2f..3bc83cd 100644
@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)
 firmware-genflags-y += -DFW_PIC
 firmware-asflags-y  += -fpic
 firmware-cflags-y   += -fPIE -pie
-firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
+firmware-ldflags-y  +=  -Wl,--no-dynamic-linker -Wl,-pie
 endif

Does this manual forwarding also work for GNU ld? If so, I think we don't need to detect bare-metal triple and turn off FW_PIC in Makefile?

I just built a riscv64-unknown-elf-gcc toolchain and used it to build the current HEAD of opensbi/master. Indeed it's broken that pie is not supported with the bare-metal triple.

The GNU ld simply complains:

ELF platform/generic/firmware/payloads/test.elf
/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
-pie not supported

Regards,
Bin

#8 lore raw

PATCH v4 3/5 firmware: Explicitly pass -pie to the linker, not just the driver

Hi Bin,

Quoted 33 lines · Bin Meng
Quoted 11 lines · Bin Meng
Quoted 9 lines · Jessica Clarke

When using Clang with a bare-metal triple, -pie does not get passed to the linker as it's not normally a thing that makes sense. However, in our case it is, and manually forwarding it on works as desired, so do so to fully support FW_PIC with Clang, including when linking with LLD.

---
 firmware/objects.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 15 lines

firmware/objects.mk ifeq ($(FW_PIC),y) · line 25

Quoted 11 lines
firmware/objects.mk +1 -1
index ce91c2f..3bc83cd 100644
@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)
 firmware-genflags-y += -DFW_PIC
 firmware-asflags-y  += -fpic
 firmware-cflags-y   += -fPIE -pie
-firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
+firmware-ldflags-y  +=  -Wl,--no-dynamic-linker -Wl,-pie
 endif

Does this manual forwarding also work for GNU ld? If so, I think we don't need to detect bare-metal triple and turn off FW_PIC in Makefile?

I just built a riscv64-unknown-elf-gcc toolchain and used it to build the current HEAD of opensbi/master. Indeed it's broken that pie is not supported with the bare-metal triple.

Quoted 5 lines

The GNU ld simply complains:

ELF platform/generic/firmware/payloads/test.elf
/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
-pie not supported

The firmware/objects.mk is doing the following:

ifndef FW_PIC
FW_PIC := y
endif

Instead of above, we should set FW_PIC=y only when the underlying toolchain supports pie.

We need a patch for this to be merged before we can merge this series. Can you send such a patch ? If not then I can send it.

Regards,
Anup

Quoted 8 lines

Regards,
Bin

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

#9 lore raw

PATCH v4 3/5 firmware: Explicitly pass -pie to the linker, not just the driver

Hi Anup,

Quoted 43 lines · Anup Patel

Hi Bin,

Quoted 33 lines · Bin Meng
Quoted 11 lines · Bin Meng
Quoted 9 lines · Jessica Clarke

When using Clang with a bare-metal triple, -pie does not get passed to the linker as it's not normally a thing that makes sense. However, in our case it is, and manually forwarding it on works as desired, so do so to fully support FW_PIC with Clang, including when linking with LLD.

---
 firmware/objects.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 15 lines

firmware/objects.mk ifeq ($(FW_PIC),y) · line 25

Quoted 11 lines
firmware/objects.mk +1 -1
index ce91c2f..3bc83cd 100644
@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)
 firmware-genflags-y += -DFW_PIC
 firmware-asflags-y  += -fpic
 firmware-cflags-y   += -fPIE -pie
-firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
+firmware-ldflags-y  +=  -Wl,--no-dynamic-linker -Wl,-pie
 endif

Does this manual forwarding also work for GNU ld? If so, I think we don't need to detect bare-metal triple and turn off FW_PIC in Makefile?

I just built a riscv64-unknown-elf-gcc toolchain and used it to build the current HEAD of opensbi/master. Indeed it's broken that pie is not supported with the bare-metal triple.

Quoted 5 lines

The GNU ld simply complains:

ELF platform/generic/firmware/payloads/test.elf
/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
-pie not supported

Quoted 9 lines

The firmware/objects.mk is doing the following:

ifndef FW_PIC
FW_PIC := y
endif

Instead of above, we should set FW_PIC=y only when the underlying toolchain supports pie.

Agree.

Quoted 2 lines

We need a patch for this to be merged before we can merge this series. Can you send such a patch ? If not then I can send it.

Do you have some reliable ways to check whether a toolchain supports PIE?

Regards,
Bin

#10 lore raw

PATCH v4 3/5 firmware: Explicitly pass -pie to the linker, not just the driver

Quoted 57 lines · Bin Meng

Hi Anup,

Quoted 43 lines · Anup Patel

Hi Bin,

Quoted 33 lines · Bin Meng
Quoted 11 lines · Bin Meng
Quoted 9 lines · Jessica Clarke

When using Clang with a bare-metal triple, -pie does not get passed to the linker as it's not normally a thing that makes sense. However, in our case it is, and manually forwarding it on works as desired, so do so to fully support FW_PIC with Clang, including when linking with LLD.

---
 firmware/objects.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 15 lines

firmware/objects.mk ifeq ($(FW_PIC),y) · line 25

Quoted 11 lines
firmware/objects.mk +1 -1
index ce91c2f..3bc83cd 100644
@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)
 firmware-genflags-y += -DFW_PIC
 firmware-asflags-y  += -fpic
 firmware-cflags-y   += -fPIE -pie
-firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
+firmware-ldflags-y  +=  -Wl,--no-dynamic-linker -Wl,-pie
 endif

Does this manual forwarding also work for GNU ld? If so, I think we don't need to detect bare-metal triple and turn off FW_PIC in Makefile?

I just built a riscv64-unknown-elf-gcc toolchain and used it to build the current HEAD of opensbi/master. Indeed it's broken that pie is not supported with the bare-metal triple.

Quoted 5 lines

The GNU ld simply complains:

ELF platform/generic/firmware/payloads/test.elf
/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
-pie not supported

Quoted 9 lines

The firmware/objects.mk is doing the following:

ifndef FW_PIC
FW_PIC := y
endif

Instead of above, we should set FW_PIC=y only when the underlying toolchain supports pie.

Quoted 6 lines

Agree.

Quoted 2 lines

We need a patch for this to be merged before we can merge this series. Can you send such a patch ? If not then I can send it.

Do you have some reliable ways to check whether a toolchain supports PIE?

How about checking "-linux-" in CROSS_COMPILE prefix ? If it's available then we set FW_PIC=y else we set FW_PIC=n. I did not find any info in the "${CROSS_COMPILE}gcc -v" output.

Maybe also add some comment in objects.mk that GCC bare-metal toolchain does not have PIE enabled.

Regards,
Anup

Quoted 3 lines

Regards,
Bin

#11 lore raw

PATCH v4 3/5 firmware: Explicitly pass -pie to the linker, not just the driver

Quoted 61 lines · Anup Patel
Quoted 59 lines · Bin Meng

Hi Anup,

Quoted 43 lines · Anup Patel

Hi Bin,

Quoted 33 lines · Bin Meng
Quoted 11 lines · Bin Meng
Quoted 9 lines · Jessica Clarke

When using Clang with a bare-metal triple, -pie does not get passed to the linker as it's not normally a thing that makes sense. However, in our case it is, and manually forwarding it on works as desired, so do so to fully support FW_PIC with Clang, including when linking with LLD.

---
 firmware/objects.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 15 lines

firmware/objects.mk ifeq ($(FW_PIC),y) · line 25

Quoted 11 lines
firmware/objects.mk +1 -1
index ce91c2f..3bc83cd 100644
@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)
 firmware-genflags-y += -DFW_PIC
 firmware-asflags-y  += -fpic
 firmware-cflags-y   += -fPIE -pie
-firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
+firmware-ldflags-y  +=  -Wl,--no-dynamic-linker -Wl,-pie
 endif

Does this manual forwarding also work for GNU ld? If so, I think we don't need to detect bare-metal triple and turn off FW_PIC in Makefile?

I just built a riscv64-unknown-elf-gcc toolchain and used it to build the current HEAD of opensbi/master. Indeed it's broken that pie is not supported with the bare-metal triple.

Quoted 5 lines

The GNU ld simply complains:

ELF platform/generic/firmware/payloads/test.elf
/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
-pie not supported

Quoted 9 lines

The firmware/objects.mk is doing the following:

ifndef FW_PIC
FW_PIC := y
endif

Instead of above, we should set FW_PIC=y only when the underlying toolchain supports pie.

Agree.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

We need a patch for this to be merged before we can merge this series. Can you send such a patch ? If not then I can send it.

Do you have some reliable ways to check whether a toolchain supports PIE?

How about checking "-linux-" in CROSS_COMPILE prefix ? If it's available then we set FW_PIC=y else we set FW_PIC=n.

That works for cross-compile toolchains. But how about native toolchains (building OpenSBI on a RISC-V machine)?

Quoted 1 line

I did not find any info in the "${CROSS_COMPILE}gcc -v" output.

Me neither.

Quoted 3 lines

Maybe also add some comment in objects.mk that GCC bare-metal toolchain does not have PIE enabled.

Or we can just document if using bare-metal toolchain FW_PIC has to be set to n in the build.

Regards,
Bin

#12 lore raw

PATCH v4 3/5 firmware: Explicitly pass -pie to the linker, not just the driver

Quoted 66 lines · Bin Meng
Quoted 64 lines · Anup Patel
Quoted 62 lines · Bin Meng

Hi Anup,

Quoted 43 lines · Anup Patel

Hi Bin,

Quoted 33 lines · Bin Meng
Quoted 11 lines · Bin Meng
Quoted 9 lines · Jessica Clarke

When using Clang with a bare-metal triple, -pie does not get passed to the linker as it's not normally a thing that makes sense. However, in our case it is, and manually forwarding it on works as desired, so do so to fully support FW_PIC with Clang, including when linking with LLD.

---
 firmware/objects.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 15 lines

firmware/objects.mk ifeq ($(FW_PIC),y) · line 25

Quoted 11 lines
firmware/objects.mk +1 -1
index ce91c2f..3bc83cd 100644
@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)
 firmware-genflags-y += -DFW_PIC
 firmware-asflags-y  += -fpic
 firmware-cflags-y   += -fPIE -pie
-firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
+firmware-ldflags-y  +=  -Wl,--no-dynamic-linker -Wl,-pie
 endif

Does this manual forwarding also work for GNU ld? If so, I think we don't need to detect bare-metal triple and turn off FW_PIC in Makefile?

I just built a riscv64-unknown-elf-gcc toolchain and used it to build the current HEAD of opensbi/master. Indeed it's broken that pie is not supported with the bare-metal triple.

Quoted 5 lines

The GNU ld simply complains:

ELF platform/generic/firmware/payloads/test.elf
/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
-pie not supported

Quoted 9 lines

The firmware/objects.mk is doing the following:

ifndef FW_PIC
FW_PIC := y
endif

Instead of above, we should set FW_PIC=y only when the underlying toolchain supports pie.

Agree.

Quoted 2 lines

We need a patch for this to be merged before we can merge this series. Can you send such a patch ? If not then I can send it.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

Do you have some reliable ways to check whether a toolchain supports PIE?

How about checking "-linux-" in CROSS_COMPILE prefix ? If it's available then we set FW_PIC=y else we set FW_PIC=n.

That works for cross-compile toolchains. But how about native toolchains (building OpenSBI on a RISC-V machine)?

Yes, it will not work for native compilation of OpenSBI.

Quoted 11 lines
Quoted 2 lines

I did not find any info in the "${CROSS_COMPILE}gcc -v" output.

Me neither.

Quoted 3 lines

Maybe also add some comment in objects.mk that GCC bare-metal toolchain does not have PIE enabled.

Or we can just document if using bare-metal toolchain FW_PIC has to be set to n in the build.

Sounds good.

We should document it under section "Required Toolchain" of top-level README.md. Basically, we prefer toolchains with PIE support and for toolchains not having PIE users have to pass the "FW_PIC=n" option.

Also, we should replace references of bare-metal toolchain prefix with linux toolchain prefix everywhere in documentation.

Regards,
Anup

Quoted 3 lines

Regards,
Bin

#13 lore raw

PATCH v4 3/5 firmware: Explicitly pass -pie to the linker, not just the driver

? 2021-07-10???? 20:26 +0530?Anup Patel???

Quoted 118 lines · Bin Meng
Quoted 88 lines
Quoted 85 lines · Anup Patel
Quoted 81 lines · Bin Meng

Hi Anup,

Quoted 58 lines · Anup Patel

Hi Bin,

Quoted 44 lines · Bin Meng
Quoted 26 lines · Bin Meng
Quoted 23 lines · Jessica Clarke

When using Clang with a bare-metal triple, -pie does not get passed to
the linker as it's not normally a thing that makes sense. However, in
our case it is, and manually forwarding it on works as desired, so do so
to fully support FW_PIC with Clang, including when linking with LLD.
---
?firmware/objects.mk | 2 +-
?1 file changed, 1 insertion(+), 1 deletion(-)

firmware/objects.mk +0 -0
index ce91c2f..3bc83cd 100644
@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)

?firmware-genflags-y += -DFW_PIC
?firmware-asflags-y? += -fpic
?firmware-cflags-y?? += -fPIE -pie

firmware/objects.mk +1 -1
-firmware-ldflags-y? +=? -Wl,--no-dynamic-linker
+firmware-ldflags-y? +=? -Wl,--no-dynamic-linker -Wl,-
Quoted 8 lines
Quoted 2 lines

pie
?endif

Does this manual forwarding also work for GNU ld? If so, I think we
don't need to detect bare-metal triple and turn off FW_PIC in
Makefile?

I just built a riscv64-unknown-elf-gcc toolchain and used it to build
the current HEAD of opensbi/master. Indeed it's broken that pie is not
supported with the bare-metal triple.

Quoted 6 lines

The GNU ld simply complains:

?ELF?????? platform/generic/firmware/payloads/test.elf /opt/riscv-unknown-elf/lib/gcc/riscv64-unknown- elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
-pie not supported

The firmware/objects.mk is doing the following:

Quoted 8 lines

ifndef FW_PIC
FW_PIC := y
endif

Instead of above, we should set FW_PIC=y only when the underlying
toolchain supports pie.

Agree.

Quoted 4 lines

We need a patch for this to be merged before we can merge this
series. Can you send such a patch ? If not then I can send it.

Do you have some reliable ways to check whether a toolchain supports PIE?

Quoted 6 lines
Quoted 3 lines

How about checking "-linux-" in CROSS_COMPILE prefix ? If it's available then
we set FW_PIC=y else we set FW_PIC=n.

That works for cross-compile toolchains. But how about native toolchains (building OpenSBI on a RISC-V machine)?

Yes, it will not work for native compilation of OpenSBI.

Quoted 6 lines
Quoted 2 lines

I did not find any info in the "${CROSS_COMPILE}gcc -v" output.

Me neither.

Quoted 1 line
Quoted 7 lines
Quoted 3 lines

Maybe also add some comment in objects.mk that GCC bare-metal toolchain
does not have PIE enabled.

Or we can just document if using bare-metal toolchain FW_PIC has to be
set to n in the build.

Sounds good.

We should document it under section "Required Toolchain" of top-level README.md. Basically, we prefer toolchains with PIE support and for toolchains not having PIE users have to pass the "FW_PIC=n" option.

Quoted 6 lines

Also, we should replace references of bare-metal toolchain prefix with
linux toolchain prefix everywhere in documentation.

Regards,
Anup

We can add the following code to the makefile to detect pie support

pie_support=$(shell $(CC) -nostdlib -fPIE -Wl,-pie /dev/null 2>/dev/null && echo y || echo n)

Quoted 5 lines
Quoted 4 lines

Regards,
Bin

#14 lore raw

PATCH v4 3/5 firmware: Explicitly pass -pie to the linker, not just the driver

Quoted 124 lines · Xiang W

? 2021-07-10???? 20:26 +0530?Anup Patel???

Quoted 122 lines · Bin Meng
Quoted 88 lines
Quoted 85 lines · Anup Patel
Quoted 81 lines · Bin Meng

Hi Anup,

Quoted 58 lines · Anup Patel

Hi Bin,

Quoted 44 lines · Bin Meng
Quoted 26 lines · Bin Meng
Quoted 23 lines · Jessica Clarke

When using Clang with a bare-metal triple, -pie does not get passed to
the linker as it's not normally a thing that makes sense. However, in
our case it is, and manually forwarding it on works as desired, so do so
to fully support FW_PIC with Clang, including when linking with LLD.

---
 firmware/objects.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
firmware/objects.mk +1 -1
index ce91c2f..3bc83cd 100644
@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)
 firmware-genflags-y += -DFW_PIC
 firmware-asflags-y  += -fpic
 firmware-cflags-y   += -fPIE -pie
-firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
+firmware-ldflags-y  +=  -Wl,--no-dynamic-linker -Wl,-
Quoted 8 lines
Quoted 2 lines

pie
endif

Does this manual forwarding also work for GNU ld? If so, I think we
don't need to detect bare-metal triple and turn off FW_PIC in
Makefile?

I just built a riscv64-unknown-elf-gcc toolchain and used it to build
the current HEAD of opensbi/master. Indeed it's broken that pie is not
supported with the bare-metal triple.

Quoted 6 lines

The GNU ld simply complains:

ELF platform/generic/firmware/payloads/test.elf
/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown- elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
-pie not supported

The firmware/objects.mk is doing the following:

Quoted 8 lines

ifndef FW_PIC
FW_PIC := y
endif

Instead of above, we should set FW_PIC=y only when the underlying
toolchain supports pie.

Agree.

Quoted 4 lines

We need a patch for this to be merged before we can merge this
series. Can you send such a patch ? If not then I can send it.

Do you have some reliable ways to check whether a toolchain supports PIE?

Quoted 6 lines
Quoted 3 lines

How about checking "-linux-" in CROSS_COMPILE prefix ? If it's available then
we set FW_PIC=y else we set FW_PIC=n.

That works for cross-compile toolchains. But how about native toolchains (building OpenSBI on a RISC-V machine)?

Yes, it will not work for native compilation of OpenSBI.

Quoted 6 lines
Quoted 2 lines

I did not find any info in the "${CROSS_COMPILE}gcc -v" output.

Me neither.

Quoted 1 line
Quoted 7 lines
Quoted 3 lines

Maybe also add some comment in objects.mk that GCC bare-metal toolchain
does not have PIE enabled.

Or we can just document if using bare-metal toolchain FW_PIC has to be
set to n in the build.

Sounds good.

We should document it under section "Required Toolchain" of top-level README.md. Basically, we prefer toolchains with PIE support and for toolchains not having PIE users have to pass the "FW_PIC=n" option.

Also, we should replace references of bare-metal toolchain prefix with
linux toolchain prefix everywhere in documentation.

Quoted 6 lines
Quoted 2 lines

Regards,
Anup

We can add the following code to the makefile to detect pie support

pie_support=$(shell $(CC) -nostdlib -fPIE -Wl,-pie /dev/null 2>/dev/null && echo y || echo n)

That doesn?t work, /dev/null is not a valid object file so this will always fail with any toolchain.

Jess

#15 lore raw

PATCH v4 3/5 firmware: Explicitly pass -pie to the linker, not just the driver

? 2021-07-10???? 19:10 +0100?Jessica Clarke???

Quoted 144 lines · Xiang W
Quoted 143 lines

? 2021-07-10???? 20:26 +0530?Anup Patel???

Quoted 141 lines
Quoted 98 lines · Bin Meng
Quoted 94 lines · Anup Patel
Quoted 90 lines · Bin Meng

Hi Anup,

Quoted 66 lines · Anup Patel

Hi Bin,

Quoted 51 lines · Bin Meng
Quoted 31 lines · Bin Meng
Quoted 28 lines · Jessica Clarke

When using Clang with a bare-metal triple, -pie does
not get passed to
the linker as it's not normally a thing that makes sense. However, in
our case it is, and manually forwarding it on works as
desired, so do so
to fully support FW_PIC with Clang, including when linking with LLD.
---
?firmware/objects.mk | 2 +-
?1 file changed, 1 insertion(+), 1 deletion(-)

firmware/objects.mk +0 -0
index ce91c2f..3bc83cd 100644
@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)

?firmware-genflags-y += -DFW_PIC
?firmware-asflags-y? += -fpic
?firmware-cflags-y?? += -fPIE -pie

firmware/objects.mk +1 -1
-firmware-ldflags-y? +=? -Wl,--no-dynamic-linker
+firmware-ldflags-y? +=? -Wl,--no-dynamic-linker -

Wl,-
pie

Quoted 8 lines
Quoted 1 line

?endif

Does this manual forwarding also work for GNU ld? If so,
I think we
don't need to detect bare-metal triple and turn off FW_PIC in
Makefile?

I just built a riscv64-unknown-elf-gcc toolchain and used
it to build
the current HEAD of opensbi/master. Indeed it's broken that
pie is not
supported with the bare-metal triple.

Quoted 6 lines

The GNU ld simply complains:

?ELF?????? platform/generic/firmware/payloads/test.elf /opt/riscv-unknown-elf/lib/gcc/riscv64-unknown- elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
-pie not supported

The firmware/objects.mk is doing the following:

Quoted 8 lines

ifndef FW_PIC
FW_PIC := y
endif

Instead of above, we should set FW_PIC=y only when the underlying
toolchain supports pie.

Agree.

Quoted 5 lines

We need a patch for this to be merged before we can merge this
series. Can you send such a patch ? If not then I can send
it.

Do you have some reliable ways to check whether a toolchain supports PIE?

Quoted 7 lines
Quoted 4 lines

How about checking "-linux-" in CROSS_COMPILE prefix ? If it's
available then
we set FW_PIC=y else we set FW_PIC=n.

That works for cross-compile toolchains. But how about native toolchains (building OpenSBI on a RISC-V machine)?

Yes, it will not work for native compilation of OpenSBI.

Quoted 7 lines
Quoted 3 lines

I did not find any info in the "${CROSS_COMPILE}gcc -v" output.

Me neither.

Quoted 1 line
Quoted 8 lines
Quoted 3 lines

Maybe also add some comment in objects.mk that GCC bare-metal toolchain
does not have PIE enabled.

Or we can just document if using bare-metal toolchain FW_PIC has to
be
set to n in the build.

Sounds good.

We should document it under section "Required Toolchain" of top- level
README.md. Basically, we prefer toolchains with PIE support and for
toolchains not having PIE users have to pass the "FW_PIC=n" option.

Also, we should replace references of bare-metal toolchain prefix with
linux toolchain prefix everywhere in documentation.

Regards,
Anup

Quoted 7 lines
Quoted 4 lines

We can add the following code to the makefile to detect pie support

pie_support=$(shell $(CC) -nostdlib -fPIE -Wl,-pie /dev/null 2>/dev/null && echo y || echo n)

That doesn?t work, /dev/null is not a valid object file so this will always fail with any toolchain.

I have tested it, so that the following commands no longer report errors

CROSS_COMPILE=riscv64-unknown-elf- make PLATFORM=generic

Xiang W

Quoted 2 lines

Jess

#16 lore raw

PATCH v4 3/5 firmware: Explicitly pass -pie to the linker, not just the driver

Quoted 151 lines · Xiang W

? 2021-07-10???? 19:10 +0100?Jessica Clarke???

Quoted 149 lines · Xiang W
Quoted 147 lines

? 2021-07-10???? 20:26 +0530?Anup Patel???

Quoted 135 lines
Quoted 98 lines · Bin Meng
Quoted 94 lines · Anup Patel
Quoted 90 lines · Bin Meng

Hi Anup,

Quoted 66 lines · Anup Patel

Hi Bin,

Quoted 51 lines · Bin Meng
Quoted 31 lines · Bin Meng
Quoted 28 lines · Jessica Clarke

When using Clang with a bare-metal triple, -pie does
not get passed to
the linker as it's not normally a thing that makes sense. However, in
our case it is, and manually forwarding it on works as
desired, so do so
to fully support FW_PIC with Clang, including when linking with LLD.

---
 firmware/objects.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
firmware/objects.mk +1 -1
index ce91c2f..3bc83cd 100644
@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)
 firmware-genflags-y += -DFW_PIC
 firmware-asflags-y  += -fpic
 firmware-cflags-y   += -fPIE -pie
-firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
+firmware-ldflags-y  +=  -Wl,--no-dynamic-linker -

Wl,-
pie

Quoted 8 lines
Quoted 1 line

endif

Does this manual forwarding also work for GNU ld? If so,
I think we
don't need to detect bare-metal triple and turn off FW_PIC in
Makefile?

I just built a riscv64-unknown-elf-gcc toolchain and used
it to build
the current HEAD of opensbi/master. Indeed it's broken that
pie is not
supported with the bare-metal triple.

Quoted 6 lines

The GNU ld simply complains:

ELF platform/generic/firmware/payloads/test.elf
/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown- elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld.bfd:
-pie not supported

The firmware/objects.mk is doing the following:

Quoted 8 lines

ifndef FW_PIC
FW_PIC := y
endif

Instead of above, we should set FW_PIC=y only when the underlying
toolchain supports pie.

Agree.

Quoted 5 lines

We need a patch for this to be merged before we can merge this
series. Can you send such a patch ? If not then I can send
it.

Do you have some reliable ways to check whether a toolchain supports PIE?

Quoted 7 lines
Quoted 4 lines

How about checking "-linux-" in CROSS_COMPILE prefix ? If it's
available then
we set FW_PIC=y else we set FW_PIC=n.

That works for cross-compile toolchains. But how about native toolchains (building OpenSBI on a RISC-V machine)?

Yes, it will not work for native compilation of OpenSBI.

Quoted 7 lines
Quoted 3 lines

I did not find any info in the "${CROSS_COMPILE}gcc -v" output.

Me neither.

Quoted 1 line
Quoted 8 lines
Quoted 3 lines

Maybe also add some comment in objects.mk that GCC bare-metal toolchain
does not have PIE enabled.

Or we can just document if using bare-metal toolchain FW_PIC has to
be
set to n in the build.

Sounds good.

We should document it under section "Required Toolchain" of top- level
README.md. Basically, we prefer toolchains with PIE support and for
toolchains not having PIE users have to pass the "FW_PIC=n" option.

Quoted 6 lines

Also, we should replace references of bare-metal toolchain prefix with
linux toolchain prefix everywhere in documentation.

Regards,
Anup

We can add the following code to the makefile to detect pie support

pie_support=$(shell $(CC) -nostdlib -fPIE -Wl,-pie /dev/null 2>/dev/null && echo y || echo n)

Quoted 6 lines
Quoted 2 lines

That doesn?t work, /dev/null is not a valid object file so this will always fail with any toolchain.

I have tested it, so that the following commands no longer report errors

CROSS_COMPILE=riscv64-unknown-elf- make PLATFORM=generic

It doesn?t work with Clang. It will always give an error as it regards /dev/null as being an object file. You need to force it to parse /dev/null as a C file with -x c. The same thing also happens with riscv64-unknown-freebsd12.1-gcc. I don?t know what?s special about riscv64-unknown-elf-gcc, though that *does* seem to do the right thing currently, but only that. Even my native x86_64-linux-gnu-gcc treats /dev/null as an object file. So something weird is going on in GCC land, but -x c should ensure you always get what you want.

Jess

#17 lore raw

PATCH v4 4/5 Makefile: Support building with Clang and LLVM binutils

In reply to Jessica Clarke

This is intended to mirror the Linux kernel. Building with CC=clang will use Clang as the compiler but default to using the existing binutils. Building with LLVM=1 will default to using Clang and LLVM binutils.

Whilst GCC will accept the -N linker option and forward it on to the linker, Clang will not, and so in order to support both compilers we must use -Wl, to forward it to the linker as is required for most other linker options.

---
 Makefile  | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++-----
 README.md | 43 ++++++++++++++++++++++++++++++++++--
 2 files changed, 100 insertions(+), 8 deletions(-)
Makefile +59 -6
index 6b64205..50b9f89 100644
@@ -76,26 +76,54 @@ OPENSBI_VERSION_MINOR=`grep "define OPENSBI_VERSION_MINOR" $(include_dir)/sbi/sb
 OPENSBI_VERSION_GIT=$(shell if [ -d $(src_dir)/.git ]; then git describe 2> /dev/null; fi)
 
 # Setup compilation commands
+ifneq ($(LLVM),)
+CC		=	clang
+AR		=	llvm-ar
+LD		=	ld.lld
+OBJCOPY		=	llvm-objcopy
+else
 ifdef CROSS_COMPILE
 CC		=	$(CROSS_COMPILE)gcc
-CPP		=	$(CROSS_COMPILE)cpp
 AR		=	$(CROSS_COMPILE)ar
 LD		=	$(CROSS_COMPILE)ld
 OBJCOPY		=	$(CROSS_COMPILE)objcopy
 else
 CC		?=	gcc
-CPP		?=	cpp
 AR		?=	ar
 LD		?=	ld
 OBJCOPY		?=	objcopy
 endif
+endif
+CPP		=	$(CC) -E
 AS		=	$(CC)
 DTC		=	dtc
 
-# Guess the compillers xlen
-OPENSBI_CC_XLEN := $(shell TMP=`$(CC) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP})
+ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),)
+CC_IS_CLANG	=	y
+else
+CC_IS_CLANG	=	n
+endif
+
+ifneq ($(shell $(LD) --version 2>&1 | head -n 1 | grep LLD),)
+LD_IS_LLD	=	y
+else
+LD_IS_LLD	=	n
+endif
+
+ifeq ($(CC_IS_CLANG),y)
+ifneq ($(CROSS_COMPILE),)
+CLANG_TARGET	=	-target $(notdir $(CROSS_COMPILE:%-=%))
+endif
+endif
+
+# Guess the compiler's XLEN
+OPENSBI_CC_XLEN := $(shell TMP=`$(CC) $(CLANG_TARGET) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP})
+
+# Guess the compiler's ABI and ISA
+ifneq ($(CC_IS_CLANG),y)
 OPENSBI_CC_ABI := $(shell TMP=`$(CC) -v 2>&1 | sed -n 's/.*\(with\-abi=\([a-zA-Z0-9]*\)\).*/\2/p'`; echo $${TMP})
 OPENSBI_CC_ISA := $(shell TMP=`$(CC) -v 2>&1 | sed -n 's/.*\(with\-arch=\([a-zA-Z0-9]*\)\).*/\2/p'`; echo $${TMP})
+endif
 
 # Setup platform XLEN
 ifndef PLATFORM_RISCV_XLEN
@@ -106,6 +134,12 @@ ifndef PLATFORM_RISCV_XLEN
   endif
 endif
 
+ifeq ($(CC_IS_CLANG),y)
+ifeq ($(CROSS_COMPILE),)
+CLANG_TARGET	=	-target riscv$(PLATFORM_RISCV_XLEN)-unknown-elf
+endif
+endif
+
 # Setup list of objects.mk files
 ifdef PLATFORM
 platform-object-mks=$(shell if [ -d $(platform_src_dir)/ ]; then find $(platform_src_dir) -iname "objects.mk" | sort -r; fi)
@@ -194,7 +228,11 @@ else
 endif
 
 # Setup compilation commands flags
-GENFLAGS	=	-I$(platform_src_dir)/include
+ifeq ($(CC_IS_CLANG),y)
+GENFLAGS	+=	$(CLANG_TARGET)
+GENFLAGS	+=	-Wno-unused-command-line-argument
+endif
+GENFLAGS	+=	-I$(platform_src_dir)/include
 GENFLAGS	+=	-I$(include_dir)
 ifneq ($(OPENSBI_VERSION_GIT),)
 GENFLAGS	+=	-DOPENSBI_VERSION_GIT="\"$(OPENSBI_VERSION_GIT)\""
@@ -208,6 +246,9 @@ CFLAGS		+=	-fno-omit-frame-pointer -fno-optimize-sibling-calls
 CFLAGS		+=	-mno-save-restore -mstrict-align
 CFLAGS		+=	-mabi=$(PLATFORM_RISCV_ABI) -march=$(PLATFORM_RISCV_ISA)
 CFLAGS		+=	-mcmodel=$(PLATFORM_RISCV_CODE_MODEL)
+ifeq ($(LD_IS_LLD),y)
+CFLAGS		+=	-mno-relax
+endif
 CFLAGS		+=	$(GENFLAGS)
 CFLAGS		+=	$(platform-cflags-y)
 CFLAGS		+=	-fno-pie -no-pie
@@ -222,18 +263,30 @@ ASFLAGS		+=	-fno-omit-frame-pointer -fno-optimize-sibling-calls
 ASFLAGS		+=	-mno-save-restore -mstrict-align
 ASFLAGS		+=	-mabi=$(PLATFORM_RISCV_ABI) -march=$(PLATFORM_RISCV_ISA)
 ASFLAGS		+=	-mcmodel=$(PLATFORM_RISCV_CODE_MODEL)
+ifeq ($(LD_IS_LLD),y)
+ASFLAGS		+=	-mno-relax
+endif
 ASFLAGS		+=	$(GENFLAGS)
 ASFLAGS		+=	$(platform-asflags-y)
 ASFLAGS		+=	$(firmware-asflags-y)
 
 ARFLAGS		=	rcs
 
-ELFFLAGS	+=	-Wl,--build-id=none -N -static-libgcc -lgcc
+ifeq ($(LD_IS_LLD),y)
+ELFFLAGS	+=	-fuse-ld=lld
+else
+ELFFLAGS	+=	-fuse-ld=bfd
+endif
+ELFFLAGS	+=	-Wl,--build-id=none -Wl,-N -static-libgcc -lgcc
 ELFFLAGS	+=	$(platform-ldflags-y)
 ELFFLAGS	+=	$(firmware-ldflags-y)
 
 MERGEFLAGS	+=	-r
+ifeq ($(LD_IS_LLD),y)
+MERGEFLAGS	+=	-b elf
+else
 MERGEFLAGS	+=	-b elf$(PLATFORM_RISCV_XLEN)-littleriscv
+endif
 MERGEFLAGS	+=	-m elf$(PLATFORM_RISCV_XLEN)lriscv
 
 DTSCPPFLAGS	=	$(CPPFLAGS) -nostdinc -nostdlib -fno-builtin -D__DTS__ -x assembler-with-cpp
README.md +41 -2

55 lines not on this page · raw message

Signature · 1 line
2.31.0
#18 lore raw

PATCH v4 4/5 Makefile: Support building with Clang and LLVM binutils

Quoted 10 lines · Jessica Clarke

This is intended to mirror the Linux kernel. Building with CC=clang will use Clang as the compiler but default to using the existing binutils. Building with LLVM=1 will default to using Clang and LLVM binutils.

Whilst GCC will accept the -N linker option and forward it on to the linker, Clang will not, and so in order to support both compilers we must use -Wl, to forward it to the linker as is required for most other linker options.

Quoted 6 lines
---
 Makefile  | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++-----
 README.md | 43 ++++++++++++++++++++++++++++++++++--
 2 files changed, 100 insertions(+), 8 deletions(-)

Here are test results:

Building with "riscv64-linux-gcc",

$ file build/platform/generic/firmware/fw_dynamic.elf
build/platform/generic/firmware/fw_dynamic.elf: ELF 64-bit LSB
executable, UCB RISC-V, version 1 (SYSV), dynamically linked, with
debug_info, not stripped
$ riscv64-linux-readelf -r build/platform/generic/firmware/fw_dynamic.elf
Relocation section '.rela.dyn' at offset 0x140f8 contains 184 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
000080013090  000000000003 R_RISCV_RELATIVE                     80000db4
...
000080013628  000200000002 R_RISCV_64        0000000080013720
fdt_serial_uart8250 + 0
000080013830  000d00000002 R_RISCV_64        00000000800138d8
fdt_reset_sifive_test + 0

Building with "LLVM=1",

$ file build/platform/generic/firmware/fw_dynamic.elf
build/platform/generic/firmware/fw_dynamic.elf: ELF 64-bit LSB shared
object, UCB RISC-V, version 1 (SYSV), dynamically linked, with
debug_info, not stripped
$ riscv64-linux-readelf -r build/platform/generic/firmware/fw_dynamic.elf
Relocation section '.rela.dyn' at offset 0x17d98 contains 188 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
000080017000  000000000003 R_RISCV_RELATIVE                     8000b680
000080017030  000000000003 R_RISCV_RELATIVE                     8001b1b8
...
000080017c90  000000000003 R_RISCV_RELATIVE                     80017628

There are two differences:

1. LLVM toolchain generates a "shared object" firmware image, while GCC generates "executable".
2. LLVM one has 4 more entries in .rela.dyn than the GCC. All entries of LLVM have the R_RISCV_RELATIVE type, but GCC one has two R_RISCV_64 entries.

I am not sure whether GCC / LLVM is doing things correctly for the above 2 differences. fw_dynamic image of both can boot to S-mode U-Boot on QEMU 'virt' though.

Using clang and GNU binutils, fw_dynamic image does not boot on QEMU 'virt', as reported before.

$ make CC=clang CROSS_COMPILE=riscv64-linux- PLATFORM=generic

Regards,
Bin

#19 lore raw

PATCH v4 4/5 Makefile: Support building with Clang and LLVM binutils

Quoted 53 lines · Bin Meng
Quoted 10 lines · Jessica Clarke

This is intended to mirror the Linux kernel. Building with CC=clang will use Clang as the compiler but default to using the existing binutils. Building with LLVM=1 will default to using Clang and LLVM binutils.

Whilst GCC will accept the -N linker option and forward it on to the linker, Clang will not, and so in order to support both compilers we must use -Wl, to forward it to the linker as is required for most other linker options.

Quoted 6 lines
---
 Makefile  | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++-----
 README.md | 43 ++++++++++++++++++++++++++++++++++--
 2 files changed, 100 insertions(+), 8 deletions(-)

Here are test results:

Building with "riscv64-linux-gcc",

$ file build/platform/generic/firmware/fw_dynamic.elf
build/platform/generic/firmware/fw_dynamic.elf: ELF 64-bit LSB
executable, UCB RISC-V, version 1 (SYSV), dynamically linked, with
debug_info, not stripped
$ riscv64-linux-readelf -r build/platform/generic/firmware/fw_dynamic.elf
Relocation section '.rela.dyn' at offset 0x140f8 contains 184 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
000080013090  000000000003 R_RISCV_RELATIVE                     80000db4
...
000080013628  000200000002 R_RISCV_64        0000000080013720
fdt_serial_uart8250 + 0
000080013830  000d00000002 R_RISCV_64        00000000800138d8
fdt_reset_sifive_test + 0

Building with "LLVM=1",

$ file build/platform/generic/firmware/fw_dynamic.elf
build/platform/generic/firmware/fw_dynamic.elf: ELF 64-bit LSB shared
object, UCB RISC-V, version 1 (SYSV), dynamically linked, with
debug_info, not stripped
$ riscv64-linux-readelf -r build/platform/generic/firmware/fw_dynamic.elf
Relocation section '.rela.dyn' at offset 0x17d98 contains 188 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
000080017000  000000000003 R_RISCV_RELATIVE                     8000b680
000080017030  000000000003 R_RISCV_RELATIVE                     8001b1b8
...
000080017c90  000000000003 R_RISCV_RELATIVE                     80017628
Quoted 7 lines

There are two differences:

1. LLVM toolchain generates a "shared object" firmware image, while GCC generates "executable".
2. LLVM one has 4 more entries in .rela.dyn than the GCC. All entries of LLVM have the R_RISCV_RELATIVE type, but GCC one has two R_RISCV_64 entries.

Do you have any explanations on these 2 differences? Are these possible toolchain bugs?

Quoted 8 lines

I am not sure whether GCC / LLVM is doing things correctly for the above 2 differences. fw_dynamic image of both can boot to S-mode U-Boot on QEMU 'virt' though.

Using clang and GNU binutils, fw_dynamic image does not boot on QEMU 'virt', as reported before.

$ make CC=clang CROSS_COMPILE=riscv64-linux- PLATFORM=generic

Regards,
Bin

#20 lore raw

PATCH v4 4/5 Makefile: Support building with Clang and LLVM binutils

Quoted 57 lines · Bin Meng
Quoted 55 lines · Bin Meng
Quoted 10 lines · Jessica Clarke

This is intended to mirror the Linux kernel. Building with CC=clang will use Clang as the compiler but default to using the existing binutils. Building with LLVM=1 will default to using Clang and LLVM binutils.

Whilst GCC will accept the -N linker option and forward it on to the linker, Clang will not, and so in order to support both compilers we must use -Wl, to forward it to the linker as is required for most other linker options.

Quoted 6 lines

---
Makefile | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++----- README.md | 43 ++++++++++++++++++++++++++++++++++--

2 files changed, 100 insertions(+), 8 deletions(-)

Here are test results:

Building with "riscv64-linux-gcc",

$ file build/platform/generic/firmware/fw_dynamic.elf
build/platform/generic/firmware/fw_dynamic.elf: ELF 64-bit LSB
executable, UCB RISC-V, version 1 (SYSV), dynamically linked, with
debug_info, not stripped
$ riscv64-linux-readelf -r build/platform/generic/firmware/fw_dynamic.elf
Relocation section '.rela.dyn' at offset 0x140f8 contains 184 entries:
 Offset          Info           Type           Sym. Value    Sym. Name + Addend
000080013090  000000000003 R_RISCV_RELATIVE                     80000db4
...
000080013628  000200000002 R_RISCV_64        0000000080013720
fdt_serial_uart8250 + 0
000080013830  000d00000002 R_RISCV_64        00000000800138d8
fdt_reset_sifive_test + 0

Building with "LLVM=1",

$ file build/platform/generic/firmware/fw_dynamic.elf
build/platform/generic/firmware/fw_dynamic.elf: ELF 64-bit LSB shared
object, UCB RISC-V, version 1 (SYSV), dynamically linked, with
debug_info, not stripped
$ riscv64-linux-readelf -r build/platform/generic/firmware/fw_dynamic.elf
Relocation section '.rela.dyn' at offset 0x17d98 contains 188 entries:
 Offset          Info           Type           Sym. Value    Sym. Name + Addend
000080017000  000000000003 R_RISCV_RELATIVE                     8000b680
000080017030  000000000003 R_RISCV_RELATIVE                     8001b1b8
...
000080017c90  000000000003 R_RISCV_RELATIVE                     80017628

There are two differences:

Quoted 8 lines
Quoted 5 lines

1. LLVM toolchain generates a "shared object" firmware image, while GCC generates "executable".
2. LLVM one has 4 more entries in .rela.dyn than the GCC. All entries of LLVM have the R_RISCV_RELATIVE type, but GCC one has two R_RISCV_64 entries.

Do you have any explanations on these 2 differences? Are these possible toolchain bugs?

[Hm, I composed this on the 11th but seems I never sent it...]

The first one smells to me like GNU ld is wrong, as executables are inherently not position-independent, PIEs are always shared objects, and I?m pretty sure that?s true of the recent -static-pie support too. In practice for our use cases it doesn?t matter though. It seems this is yet another undocumented, likely unintended (as it?s a very old legacy, and mostly unused, option) consequence of using -N/--omagic (in that it just blindly sets various flags internally and nobody thought about whether that made sense once -pie was added). Incidentally, I don?t think we actually need -N/--omagic any more, but that?s a separate thing.

For the minor difference in number of relocations, that probably just comes down to minor codegen differences and I wouldn?t worry about it; with a large enough code base small differences are to be expected.

As for R_RISCV_64, there?s no reason for GNU ld to emit R_RISCV_64 here. It?s technically correct but entirely unnecessary (and can break legitimate code that assumes only R_RISCV_RELATIVE gets emitted, which *should* be the case; kernel and run-time linker self-relocation code, that looks a lot like what OpenSBI is doing here, often likes to assume that, possibly with R_RISCV_IRELATIVE too if IFUNCs are used). I see it locally for fdt_serial_uart8250 and fdt_reset_sifive which should in no way be special, there?s nothing stopping those being evaluated at link time and leaving R_RISCV_RELATIVE to adjust them at run time as needed like with all the other symbols.

So I?d regard -N + -pie giving EXEC not DYN, and R_RISCV_64 being emitted here, as being GNU ld sort-of-bugs, albeit with the former being extremely ill-defined over what that combination means (beyond ?it does what it does?), with LLD?s output for both being what I would expect.

Jess

Quoted 10 lines
Quoted 7 lines

I am not sure whether GCC / LLVM is doing things correctly for the above 2 differences. fw_dynamic image of both can boot to S-mode U-Boot on QEMU 'virt' though.

Using clang and GNU binutils, fw_dynamic image does not boot on QEMU 'virt', as reported before.

$ make CC=clang CROSS_COMPILE=riscv64-linux- PLATFORM=generic

Regards,
Bin

#21 lore raw

PATCH v4 4/5 Makefile: Support building with Clang and LLVM binutils

Quoted 81 lines · Jessica Clarke
Quoted 57 lines · Bin Meng
Quoted 55 lines · Bin Meng
Quoted 10 lines · Jessica Clarke

This is intended to mirror the Linux kernel. Building with CC=clang will use Clang as the compiler but default to using the existing binutils. Building with LLVM=1 will default to using Clang and LLVM binutils.

Whilst GCC will accept the -N linker option and forward it on to the linker, Clang will not, and so in order to support both compilers we must use -Wl, to forward it to the linker as is required for most other linker options.

Quoted 6 lines

---
Makefile | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++----- README.md | 43 ++++++++++++++++++++++++++++++++++--

2 files changed, 100 insertions(+), 8 deletions(-)

Here are test results:

Building with "riscv64-linux-gcc",

$ file build/platform/generic/firmware/fw_dynamic.elf
build/platform/generic/firmware/fw_dynamic.elf: ELF 64-bit LSB
executable, UCB RISC-V, version 1 (SYSV), dynamically linked, with
debug_info, not stripped
$ riscv64-linux-readelf -r build/platform/generic/firmware/fw_dynamic.elf
Relocation section '.rela.dyn' at offset 0x140f8 contains 184 entries:
 Offset          Info           Type           Sym. Value    Sym. Name + Addend
000080013090  000000000003 R_RISCV_RELATIVE                     80000db4
...
000080013628  000200000002 R_RISCV_64        0000000080013720
fdt_serial_uart8250 + 0
000080013830  000d00000002 R_RISCV_64        00000000800138d8
fdt_reset_sifive_test + 0

Building with "LLVM=1",

$ file build/platform/generic/firmware/fw_dynamic.elf
build/platform/generic/firmware/fw_dynamic.elf: ELF 64-bit LSB shared
object, UCB RISC-V, version 1 (SYSV), dynamically linked, with
debug_info, not stripped
$ riscv64-linux-readelf -r build/platform/generic/firmware/fw_dynamic.elf
Relocation section '.rela.dyn' at offset 0x17d98 contains 188 entries:
 Offset          Info           Type           Sym. Value    Sym. Name + Addend
000080017000  000000000003 R_RISCV_RELATIVE                     8000b680
000080017030  000000000003 R_RISCV_RELATIVE                     8001b1b8
...
000080017c90  000000000003 R_RISCV_RELATIVE                     80017628

There are two differences:

Quoted 8 lines
Quoted 5 lines

1. LLVM toolchain generates a "shared object" firmware image, while GCC generates "executable".
2. LLVM one has 4 more entries in .rela.dyn than the GCC. All entries of LLVM have the R_RISCV_RELATIVE type, but GCC one has two R_RISCV_64 entries.

Do you have any explanations on these 2 differences? Are these possible toolchain bugs?

[Hm, I composed this on the 11th but seems I never sent it...]

The first one smells to me like GNU ld is wrong, as executables are inherently not position-independent, PIEs are always shared objects, and I?m pretty sure that?s true of the recent -static-pie support too. In practice for our use cases it doesn?t matter though. It seems this is yet another undocumented, likely unintended (as it?s a very old legacy, and mostly unused, option) consequence of using -N/--omagic (in that it just blindly sets various flags internally and nobody thought about whether that made sense once -pie was added). Incidentally, I don?t think we actually need -N/--omagic any more, but that?s a separate thing.

Quoted 7 lines

For the minor difference in number of relocations, that probably just comes down to minor codegen differences and I wouldn?t worry about it; with a large enough code base small differences are to be expected.

As for R_RISCV_64, there?s no reason for GNU ld to emit R_RISCV_64 here. It?s technically correct but entirely unnecessary (and can break legitimate code that assumes only R_RISCV_RELATIVE gets emitted, which

Fortunately OpenSBI, as well as U-Boot, handle both R_RISCV_RELATIVE and R_RISCV_{64,32} here, so they are not broken due to these unnecessary entries.

Quoted 8 lines

*should* be the case; kernel and run-time linker self-relocation code, that looks a lot like what OpenSBI is doing here, often likes to assume that, possibly with R_RISCV_IRELATIVE too if IFUNCs are used). I see it locally for fdt_serial_uart8250 and fdt_reset_sifive which should in no way be special, there?s nothing stopping those being evaluated at link time and leaving R_RISCV_RELATIVE to adjust them at run time as needed like with all the other symbols.

Quoted 5 lines

So I?d regard -N + -pie giving EXEC not DYN, and R_RISCV_64 being emitted here, as being GNU ld sort-of-bugs, albeit with the former being extremely ill-defined over what that combination means (beyond ?it does what it does?), with LLD?s output for both being what I would expect.

Thanks a lot for the explanation! Someone can file a defect to the GNU ld :)

Regards,
Bin

#22 lore raw

PATCH v4 5/5 Drop dependency on libgcc by importing part of FreeBSD's libquad

In reply to Jessica Clarke

We only need libgcc for 64-bit division on RV32. Whilst GCC toolchains bundle libgcc, Clang toolchains tend not to ship libclang_rt.builtins given every compiler is a cross-compiler for every target and so you would need a silly number of builds of it, with only the native library available; only vendor-provided Clang toolchains specifically for bare metal cross-compiling are likely to provide it.

Thus, import part of FreeBSD's implementation of the division support functions needed and stop linking against libgcc.

---
 Makefile                              |   2 +-
 lib/utils/libquad/divdi3.c            |  64 ++++++
 lib/utils/libquad/include/limits.h    |  12 ++
 lib/utils/libquad/include/sys/cdefs.h |  12 ++
 lib/utils/libquad/include/sys/types.h |  25 +++
 lib/utils/libquad/moddi3.c            |  66 ++++++
 lib/utils/libquad/objects.mk          |  14 ++
 lib/utils/libquad/qdivrem.c           | 278 ++++++++++++++++++++++++++
 lib/utils/libquad/quad.h              | 105 ++++++++++
 lib/utils/libquad/udivdi3.c           |  52 +++++
 lib/utils/libquad/umoddi3.c           |  54 +++++
 11 files changed, 683 insertions(+), 1 deletion(-)
Makefile +1 -1
index 50b9f89..0672558 100644
@@ -277,7 +277,7 @@ ELFFLAGS	+=	-fuse-ld=lld
 else
 ELFFLAGS	+=	-fuse-ld=bfd
 endif
-ELFFLAGS	+=	-Wl,--build-id=none -Wl,-N -static-libgcc -lgcc
+ELFFLAGS	+=	-Wl,--build-id=none -Wl,-N
 ELFFLAGS	+=	$(platform-ldflags-y)
 ELFFLAGS	+=	$(firmware-ldflags-y)
 
lib/utils/libquad/divdi3.c new file +64 -0

64 lines not on this page · raw message

lib/utils/libquad/include/limits.h new file +12 -0
new file mode 100644
index 0000000..9c69d27
@@ -0,0 +1,12 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2021 Jessica Clarke <[email protected]>
+ */
+
+#ifndef __LIMITS_H__
+#define __LIMITS_H__
+
+#define CHAR_BIT 8
+
+#endif
lib/utils/libquad/include/sys/cdefs.h new file +12 -0
new file mode 100644
index 0000000..e8d6faa
@@ -0,0 +1,12 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2021 Jessica Clarke <[email protected]>
+ */
+
+#ifndef __SYS_CDEFS_H__
+#define __SYS_CDEFS_H__
+
+#define __FBSDID(s) struct __hack
+
+#endif
lib/utils/libquad/include/sys/types.h new file +25 -0
new file mode 100644
index 0000000..e372437
@@ -0,0 +1,25 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2021 Jessica Clarke <[email protected]>
+ */
+
+#ifndef __SYS_TYPES_H__
+#define __SYS_TYPES_H__
+
+#include <sbi/sbi_types.h>
+
+typedef unsigned long u_long;
+
+typedef int64_t quad_t;
+typedef uint64_t u_quad_t;
+
+#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
+#define _QUAD_LOWWORD 1
+#define _QUAD_HIGHWORD 0
+#else
+#define _QUAD_LOWWORD 0
+#define _QUAD_HIGHWORD 1
+#endif
+
+#endif
lib/utils/libquad/moddi3.c new file +66 -0

66 lines not on this page · raw message

lib/utils/libquad/objects.mk new file +14 -0
new file mode 100644
index 0000000..5e53cd0
@@ -0,0 +1,14 @@
+#
+# SPDX-License-Identifier: BSD-2-Clause
+#
+# Copyright (c) 2021 Jessica Clarke <[email protected]>
+#
+
+ifeq ($(PLATFORM_RISCV_XLEN),32)
+libsbiutils-objs-y += libquad/divdi3.o
+libsbiutils-objs-y += libquad/moddi3.o
+libsbiutils-objs-y += libquad/qdivrem.o
+libsbiutils-objs-y += libquad/udivdi3.o
+libsbiutils-objs-y += libquad/umoddi3.o
+libsbiutils-genflags-y += -I$(libsbiutils_dir)/libquad/include
+endif
lib/utils/libquad/qdivrem.c new file +278 -0

278 lines not on this page · raw message

lib/utils/libquad/quad.h new file +105 -0

105 lines not on this page · raw message

lib/utils/libquad/udivdi3.c new file +52 -0

52 lines not on this page · raw message

lib/utils/libquad/umoddi3.c new file +54 -0

54 lines not on this page · raw message

Signature · 1 line
2.31.0
#23 lore raw

v4 cover Fully support standalone Clang/LLVM toolchains

In reply to Jessica Clarke
Quoted 80 lines · Jessica Clarke

This patch series is comprised of five parts.

The first patch fixes a warning seen when building with LLVM due to a bogus combination of assembly directives.

The second patch fixes errors seen when trying to build OpenSBI as a position-independent binary using LLD that may or may not be an LLD bug (the exact meaning of -N/--omagic isn't clear) but can easily be worked around without any issue either way.

The third patch bypasses the Clang driver's helpful nature of not honouring -pie for bare-metal binaries as it's normally not something you want (arguably that should be an error though, or passed on, since just giving a -Wunused-command-line-argument warning can get lost, and may be disabled).

The fourth patch adds build system support for using Clang and LLVM binutils, provided your Clang is able to locate a libgcc.a in its search path.

However, pure LLVM toolchains do not use libgcc, they use compiler-rt (libclang_rt.builtins.<arch>.a). We could change the Makefile to not hard-code -lgcc and instead use -print-libgcc-file-name, but that still requires a bare metal compiler-rt built for the right -march/-mabi to be present, which is often not the case. Moreover, we need very little from libgcc/compiler-rt; RV64 needs nothing, and RV32 only needs 64-bit division. Thus, the fifth patch vendors part of FreeBSD's libquad and stops linking against libgcc entirely, allowing OpenSBI to be built with just a cross-compiler. This means that building with any distro-provided LLVM just works, as does compiling with the system Clang compiler and LLD linker on FreeBSD without any external packages needed (beyond GNU make).

Changes in v4:
* Avoid read-only dynamic relocation error with LLD
* Support FW_PIC with LLD when using a bare-metal triple

Changes in v3:
* Fixed the binding warning seen when building with LLVM
* Provide a default -target when CROSS_COMPILE isn't set so the variable isn't normally required to be set when cross-compiling with LLVM (but is still honoured if set).
* Pass -fuse=bfd when not using LLD to ensure an ld.bfd gets correctly picked up over a plain ld when ld is LLD. Now we always explicitly request Clang and GCC use the linker type corresponding to LD. This allows BFD to be used on FreeBSD for testing purposes despite LLD being the system linker.

Changes in v2:
* Add documentation to README.md
* Pass -fuse-ld=lld to Clang when using LLD
* Modify commit message of first commit to explain -N -> -Wl,-N change
* Bring back the old ?= uses for the non-CROSS_COMPILE (and, now, non-LLVM) case; whilst I still think that's how it should be, it's not required for this patch, just related cleanup I did to reduce the complexity of adding the LLVM case.

Jessica Clarke (5):
fw_base: Don't mark fw_platform_init as both global and weak
fw_base: Put data in .data rather than .text
firmware: Explicitly pass -pie to the linker, not just the driver
Makefile: Support building with Clang and LLVM binutils Drop dependency on libgcc by importing part of FreeBSD's libquad

Makefile                              |  65 +++++-
README.md                             |  43 +++-
firmware/fw_base.S                    |   2 +-
firmware/objects.mk                   |   2 +-
lib/utils/libquad/divdi3.c            |  64 ++++++
lib/utils/libquad/include/limits.h    |  12 ++
lib/utils/libquad/include/sys/cdefs.h |  12 ++
lib/utils/libquad/include/sys/types.h |  25 +++
lib/utils/libquad/moddi3.c            |  66 ++++++
lib/utils/libquad/objects.mk          |  14 ++
lib/utils/libquad/qdivrem.c           | 278 ++++++++++++++++++++++++++
lib/utils/libquad/quad.h              | 105 ++++++++++
lib/utils/libquad/udivdi3.c           |  52 +++++
lib/utils/libquad/umoddi3.c           |  54 +++++
14 files changed, 784 insertions(+), 10 deletions(-)
Quoted 10 lines

create mode 100644 lib/utils/libquad/divdi3.c create mode 100644 lib/utils/libquad/include/limits.h create mode 100644 lib/utils/libquad/include/sys/cdefs.h create mode 100644 lib/utils/libquad/include/sys/types.h create mode 100644 lib/utils/libquad/moddi3.c create mode 100644 lib/utils/libquad/objects.mk create mode 100644 lib/utils/libquad/qdivrem.c create mode 100644 lib/utils/libquad/quad.h create mode 100644 lib/utils/libquad/udivdi3.c create mode 100644 lib/utils/libquad/umoddi3.c

... all Signed-off-by: Jessica Clarke <[email protected]>, managed to forget that yet again :(

Jess