Skip to contentSkip to conversation

Fully support standalone Clang/LLVM toolchains

PATCH Superseded Jessica Clarke 4 participants 34 messages Jul 11, 2021 – Jul 12, 2021 Kbuild

lore
Share & tools
lore
#0 lore raw

v5 cover Fully support standalone Clang/LLVM toolchains

This patch series is comprised of six parts.

The first patch fixes a warning seen when building with LLVM due to a bogus combination of assembly directives.

The second patch fixes errors seen when trying to build OpenSBI as a position-independent binary using LLD that may or may not be an LLD bug (the exact meaning of -N/--omagic isn't clear) but can easily be worked around without any issue either way.

The third patch bypasses the Clang driver's helpful nature of not honouring -pie for bare-metal binaries as it's normally not something you want (arguably that should be an error though, or passed on, since just giving a -Wunused-command-line-argument warning can get lost, and may be disabled).

The fourth patch fixes the FW_PIC support added recently to be disabled by default when unsupported, since bare-metal GNU ld does not support PIE and so currently building with a bare-metal GNU ld will fail unless FW_PIC is manually disabled. Disabling FW_PIC is suboptimal but restores the pre-FW_PIC behaviour so is no longer a regression.

The fifth patch adds build system support for using Clang and LLVM binutils, provided your Clang is able to locate a libgcc.a in its search path.

However, pure LLVM toolchains do not use libgcc, they use compiler-rt (libclang_rt.builtins.<arch>.a). We could change the Makefile to not hard-code -lgcc and instead use -print-libgcc-file-name, but that still requires a bare metal compiler-rt built for the right -march/-mabi to be present, which is often not the case. Moreover, we need very little from libgcc/compiler-rt; RV64 needs nothing, and RV32 only needs 64-bit division. Thus, the sixth patch vendors part of FreeBSD's libquad and stops linking against libgcc entirely, allowing OpenSBI to be built with just a cross-compiler. This means that building with any distro-provided LLVM just works, as does compiling with the system Clang compiler and LLD linker on FreeBSD without any external packages needed (beyond GNU make).

Note that using Clang with the current bleeding-edge riscv64 glibc toolchain from Bootlin's linker is currently broken; around 60 relocations, or about a third of the number expected, go missing, with a big gap in the relocations emitted, causing various global data structures to not be correctly initialised. In particular, root's possible_harts and regions fields lose their relocations during linking so are left as NULL, causing sanitize_domain to immediately fail with SBI_EINVAL. The relocations exist in the object files and look completely sane so this appears to be a GNU ld bug, but I have not narrowed it down enough to be sure it's not something weird that OpenSBI or Clang are doing. I do not think this should be a blocker, and have added a note about this currently-broken combination to the README.

Changes in v5:
* Disable FW_PIC by default when unsupported
* Pulled out -mno-relax and -fuse-ld flags into variables for reusing in the code for the above change, and to deduplicate the existing -mno-relax code
* Note in the README that riscv64-linux-gnu-ld + Clang is currently broken due to what appears to be a GNU ld bug

Changes in v4:
* Avoid read-only dynamic relocation error with LLD
* Support FW_PIC with LLD when using a bare-metal triple

Changes in v3:
* Fixed the binding warning seen when building with LLVM
* Provide a default -target when CROSS_COMPILE isn't set so the variable isn't normally required to be set when cross-compiling with LLVM (but is still honoured if set).
* Pass -fuse=bfd when not using LLD to ensure an ld.bfd gets correctly picked up over a plain ld when ld is LLD. Now we always explicitly request Clang and GCC use the linker type corresponding to LD. This allows BFD to be used on FreeBSD for testing purposes despite LLD being the system linker.

Changes in v2:
* Add documentation to README.md
* Pass -fuse-ld=lld to Clang when using LLD
* Modify commit message of first commit to explain -N -> -Wl,-N change
* Bring back the old ?= uses for the non-CROSS_COMPILE (and, now, non-LLVM) case; whilst I still think that's how it should be, it's not required for this patch, just related cleanup I did to reduce the complexity of adding the LLVM case.

Jessica Clarke (6):
fw_base: Don't mark fw_platform_init as both global and weak
fw_base: Put data in .data rather than .text
firmware: Explicitly pass -pie to the linker, not just the driver
firmware: Only default FW_PIC to y if supported
Makefile: Support building with Clang and LLVM binutils Drop dependency on libgcc by importing part of FreeBSD's libquad

 Makefile                              |  67 ++++++-
 README.md                             |  48 ++++-
 firmware/fw_base.S                    |   2 +-
 firmware/objects.mk                   |   4 +-
 lib/utils/libquad/divdi3.c            |  64 ++++++
 lib/utils/libquad/include/limits.h    |  12 ++
 lib/utils/libquad/include/sys/cdefs.h |  12 ++
 lib/utils/libquad/include/sys/types.h |  25 +++
 lib/utils/libquad/moddi3.c            |  66 ++++++
 lib/utils/libquad/objects.mk          |  14 ++
 lib/utils/libquad/qdivrem.c           | 278 ++++++++++++++++++++++++++
 lib/utils/libquad/quad.h              | 105 ++++++++++
 lib/utils/libquad/udivdi3.c           |  52 +++++
 lib/utils/libquad/umoddi3.c           |  54 +++++
 14 files changed, 792 insertions(+), 11 deletions(-)
 create mode 100644 lib/utils/libquad/divdi3.c
 create mode 100644 lib/utils/libquad/include/limits.h
 create mode 100644 lib/utils/libquad/include/sys/cdefs.h
 create mode 100644 lib/utils/libquad/include/sys/types.h
 create mode 100644 lib/utils/libquad/moddi3.c
 create mode 100644 lib/utils/libquad/objects.mk
 create mode 100644 lib/utils/libquad/qdivrem.c
 create mode 100644 lib/utils/libquad/quad.h
 create mode 100644 lib/utils/libquad/udivdi3.c
 create mode 100644 lib/utils/libquad/umoddi3.c
Signature · 1 line
2.31.0
#1 lore raw

PATCH v5 1/6 fw_base: Don't mark fw_platform_init as both global and weak

These are mutually exclusive. GNU as and LLVM both let later binding directives override earlier ones so this works as intended, but LLVM 12 turned this into a warning as there's no good reason to do such a thing and could be a potential bug. Thus, remove the redundant and incorrect .globl directive for fw_platform_init.

---
 firmware/fw_base.S | 1 -
 1 file changed, 1 deletion(-)
firmware/fw_base.S +0 -1
index a5ce946..ee2a51b 100644
@@ -553,7 +553,6 @@ _start_hang:
 
 	.section .entry, "ax", %progbits
 	.align 3
-	.globl fw_platform_init
 	.weak fw_platform_init
 fw_platform_init:
 	add	a0, a1, zero
Signature · 1 line
2.31.0
Applied
#2 lore raw

PATCH v5 1/6 fw_base: Don't mark fw_platform_init as both global and weak

Quoted 10 lines · Jessica Clarke

These are mutually exclusive. GNU as and LLVM both let later binding directives override earlier ones so this works as intended, but LLVM 12 turned this into a warning as there's no good reason to do such a thing and could be a potential bug. Thus, remove the redundant and incorrect .globl directive for fw_platform_init.

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

Quoted 24 lines
---
 firmware/fw_base.S | 1 -
 1 file changed, 1 deletion(-)
firmware/fw_base.S +0 -2
index a5ce946..ee2a51b 100644
@@ -553,7 +553,6 @@ _start_hang:
 
        .section .entry, "ax", %progbits
        .align 3
-       .globl fw_platform_init
        .weak fw_platform_init
 fw_platform_init:
        add     a0, a1, zero
--

2.31.0

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

#3 lore raw

PATCH v5 2/6 fw_base: Put data in .data rather than .text

In reply to Jessica Clarke

The -N linker option is supposed to make .text writable, but GNU ld and LLD differ in interpreting what that means. GNU ld will happily let you have relocations in it, but LLD will see that the input section is read-only (even though the output section is writable) and give an error. It's unclear if either of them intend to have that behaviour in this edge case, but regardless there's no reason not to just put the data in a writable .data section.

---
 firmware/fw_base.S | 1 +
 1 file changed, 1 insertion(+)
firmware/fw_base.S +1 -0
index ee2a51b..f8aea05 100644
@@ -501,6 +501,7 @@ _skip_trap_exit_rv32_hyp:
 	/* We don't expect to reach here hence just hang */
 	j	_start_hang
 
+	.data
 	.align 3
 #ifdef FW_PIC
 _runtime_offset:
Signature · 1 line
2.31.0
Applied
#4 lore raw

PATCH v5 2/6 fw_base: Put data in .data rather than .text

Quoted 9 lines · Jessica Clarke

The -N linker option is supposed to make .text writable, but GNU ld and LLD differ in interpreting what that means. GNU ld will happily let you have relocations in it, but LLD will see that the input section is read-only (even though the output section is writable) and give an error. It's unclear if either of them intend to have that behaviour in this edge case, but regardless there's no reason not to just put the data in a writable .data section.

Quoted 3 lines

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

Quoted 24 lines
---
 firmware/fw_base.S | 1 +
 1 file changed, 1 insertion(+)
firmware/fw_base.S +1 -1
index ee2a51b..f8aea05 100644
@@ -501,6 +501,7 @@ _skip_trap_exit_rv32_hyp:
        /* We don't expect to reach here hence just hang */
        j       _start_hang
 
+       .data
        .align 3
 #ifdef FW_PIC
 _runtime_offset:
--

2.31.0

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

#5 lore raw

PATCH v5 3/6 firmware: Explicitly pass -pie to the linker, not just the driver

In reply to Jessica Clarke

When using Clang with a bare-metal triple, -pie does not get passed to the linker as it's not normally a thing that makes sense, unlike GCC which will unconditionally forward it on and potentially result in a linker error. However, LLD does support it, and manually forwarding it on works as desired, so do so to fully support FW_PIC with Clang and LLD.

---
 firmware/objects.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
firmware/objects.mk +1 -1
index ce91c2f..3bc83cd 100644
@@ -21,7 +21,7 @@ ifeq ($(FW_PIC),y)
 firmware-genflags-y +=	-DFW_PIC
 firmware-asflags-y  +=	-fpic
 firmware-cflags-y   +=	-fPIE -pie
-firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
+firmware-ldflags-y  +=  -Wl,--no-dynamic-linker -Wl,-pie
 endif
 
 ifdef FW_TEXT_START
Signature · 1 line
2.31.0
#6 lore raw

PATCH v5 3/6 firmware: Explicitly pass -pie to the linker, not just the driver

Quoted 8 lines · Jessica Clarke

When using Clang with a bare-metal triple, -pie does not get passed to the linker as it's not normally a thing that makes sense, unlike GCC which will unconditionally forward it on and potentially result in a linker error. However, LLD does support it, and manually forwarding it on works as desired, so do so to fully support FW_PIC with Clang and LLD.

Quoted 5 lines
---
 firmware/objects.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Applied
#7 lore raw

PATCH v5 3/6 firmware: Explicitly pass -pie to the linker, not just the driver

Quoted 10 lines · Bin Meng
Quoted 8 lines · Jessica Clarke

When using Clang with a bare-metal triple, -pie does not get passed to the linker as it's not normally a thing that makes sense, unlike GCC which will unconditionally forward it on and potentially result in a linker error. However, LLD does support it, and manually forwarding it on works as desired, so do so to fully support FW_PIC with Clang and LLD.

Quoted 8 lines
Quoted 5 lines
---
 firmware/objects.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

Quoted 5 lines

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

#8 lore raw

PATCH v5 3/6 firmware: Explicitly pass -pie to the linker, not just the driver

In reply to Jessica Clarke
Quoted 4 lines · Jessica Clarke

When using Clang with a bare-metal triple, -pie does not get passed to the linker as it's not normally a thing that makes sense, unlike GCC which will unconditionally forward it on and potentially result in a

It seems GCC driver does not unconditionally forward it to the linker, see:

$ riscv64-unknown-elf-gcc -fPIE -nostdlib -x c /dev/null -o /dev/null
2>&1 && echo y
/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld:
warning: cannot find entry symbol _start; defaulting to
0000000000010040
y

But

$ riscv64-unknown-elf-gcc -fPIE -nostdlib -Wl,-pie -x c /dev/null -o
/dev/null 2>&1 && echo y
/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld:
-pie not supported
collect2: error: ld returned 1 exit status

So regarding bare-metal triple, GCC driver and clang driver behave the same way. It's just GNU bare-metal ld does not support PIE, while LLD supports.

So we need to revise the above commit message.

Quoted 9 lines

linker error. However, LLD does support it, and manually forwarding it on works as desired, so do so to fully support FW_PIC with Clang and LLD.

---
 firmware/objects.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Regards,
Bin

#9 lore raw

PATCH v5 3/6 firmware: Explicitly pass -pie to the linker, not just the driver

Quoted 24 lines · Bin Meng
Quoted 4 lines · Jessica Clarke

When using Clang with a bare-metal triple, -pie does not get passed to the linker as it's not normally a thing that makes sense, unlike GCC which will unconditionally forward it on and potentially result in a

It seems GCC driver does not unconditionally forward it to the linker, see:

$ riscv64-unknown-elf-gcc -fPIE -nostdlib -x c /dev/null -o /dev/null
2>&1 && echo y
/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld:
warning: cannot find entry symbol _start; defaulting to
0000000000010040
y

But

$ riscv64-unknown-elf-gcc -fPIE -nostdlib -Wl,-pie -x c /dev/null -o
/dev/null 2>&1 && echo y
/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld:
-pie not supported
collect2: error: ld returned 1 exit status
Quoted 5 lines

So regarding bare-metal triple, GCC driver and clang driver behave the same way. It's just GNU bare-metal ld does not support PIE, while LLD supports.

So we need to revise the above commit message.

You didn?t pass -pie, only -fPIE.

Jess

Quoted 12 lines
Quoted 9 lines

linker error. However, LLD does support it, and manually forwarding it on works as desired, so do so to fully support FW_PIC with Clang and LLD.

---
firmware/objects.mk | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-)

Regards,
Bin

#10 lore raw

PATCH v5 3/6 firmware: Explicitly pass -pie to the linker, not just the driver

Quoted 26 lines · Jessica Clarke
Quoted 24 lines · Bin Meng
Quoted 4 lines · Jessica Clarke

When using Clang with a bare-metal triple, -pie does not get passed to the linker as it's not normally a thing that makes sense, unlike GCC which will unconditionally forward it on and potentially result in a

It seems GCC driver does not unconditionally forward it to the linker, see:

$ riscv64-unknown-elf-gcc -fPIE -nostdlib -x c /dev/null -o /dev/null
2>&1 && echo y
/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld:
warning: cannot find entry symbol _start; defaulting to
0000000000010040
y

But

$ riscv64-unknown-elf-gcc -fPIE -nostdlib -Wl,-pie -x c /dev/null -o
/dev/null 2>&1 && echo y
/opt/riscv-unknown-elf/lib/gcc/riscv64-unknown-elf/10.1.0/../../../../riscv64-unknown-elf/bin/ld:
-pie not supported
collect2: error: ld returned 1 exit status
Quoted 7 lines
Quoted 5 lines

So regarding bare-metal triple, GCC driver and clang driver behave the same way. It's just GNU bare-metal ld does not support PIE, while LLD supports.

So we need to revise the above commit message.

You didn?t pass -pie, only -fPIE.

Ah, yes. Indeed -pie is the linker option. Then we are good.

Regards,
Bin

#11 lore raw

PATCH v5 4/6 firmware: Only default FW_PIC to y if supported

In reply to Jessica Clarke

Bare-metal GNU ld does not support PIE, so if using it this will result in a failure to build. Instead, default to FW_PIC=n if not supported. Note that an explicit FW_PIC=y is not overriden, to ensure the build fails rather than silently producing a position-dependent binary.

---
 Makefile            | 3 +++
 firmware/objects.mk | 2 +-
 2 files changed, 4 insertions(+), 1 deletion(-)
Makefile +3 -0
index 6b64205..ba06313 100644
@@ -106,6 +106,9 @@ ifndef PLATFORM_RISCV_XLEN
   endif
 endif
 
+# Check whether the linker supports creating PIEs
+OPENSBI_LD_PIE := $(shell $(CC) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n)
+
 # Setup list of objects.mk files
 ifdef PLATFORM
 platform-object-mks=$(shell if [ -d $(platform_src_dir)/ ]; then find $(platform_src_dir) -iname "objects.mk" | sort -r; fi)
firmware/objects.mk +1 -1
index 3bc83cd..8da7ca3 100644
@@ -14,7 +14,7 @@ firmware-asflags-y +=
 firmware-ldflags-y +=
 
 ifndef FW_PIC
-FW_PIC := y
+FW_PIC := $(OPENSBI_LD_PIE)
 endif
 
 ifeq ($(FW_PIC),y)
Signature · 1 line
2.31.0
#12 lore raw

PATCH v5 4/6 firmware: Only default FW_PIC to y if supported

? 2021-07-11???? 03:28 +0100?Jessica Clarke???

Quoted 22 lines

Bare-metal GNU ld does not support PIE, so if using it this will result
in a failure to build. Instead, default to FW_PIC=n if not supported. Note that an explicit FW_PIC=y is not overriden, to ensure the build fails rather than silently producing a position-dependent binary.

---
?Makefile??????????? | 3 +++
?firmware/objects.mk | 2 +-
?2 files changed, 4 insertions(+), 1 deletion(-)

Makefile +0 -0
index 6b64205..ba06313 100644
@@ -106,6 +106,9 @@ ifndef PLATFORM_RISCV_XLEN

?? endif
?endif
?

Makefile +2 -0
+# Check whether the linker supports creating PIEs
+OPENSBI_LD_PIE := $(shell $(CC) -fPIE -nostdlib -Wl,-pie -x c
Quoted 1 line

/dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n)

For llvm, --target and -fuse-ld need to be added here. Otherwise, this will detect the host instead of what you want

Quoted 22 lines

+
?# Setup list of objects.mk files
?ifdef PLATFORM
?platform-object-mks=$(shell if [ -d $(platform_src_dir)/ ]; then find $(platform_src_dir) -iname "objects.mk" | sort -r; fi)

firmware/objects.mk +0 -0
index 3bc83cd..8da7ca3 100644
@@ -14,7 +14,7 @@ firmware-asflags-y +=

?firmware-ldflags-y +=
?
?ifndef FW_PIC

firmware/objects.mk +1 -1
-FW_PIC := y
+FW_PIC := $(OPENSBI_LD_PIE)

?endif
?
?ifeq ($(FW_PIC),y)

Signature · 1 line
2.31.0
#13 lore raw

PATCH v5 4/6 firmware: Only default FW_PIC to y if supported

Quoted 22 lines · Xiang W

? 2021-07-11???? 03:28 +0100?Jessica Clarke???

Quoted 20 lines

Bare-metal GNU ld does not support PIE, so if using it this will result
in a failure to build. Instead, default to FW_PIC=n if not supported. Note that an explicit FW_PIC=y is not overriden, to ensure the build fails rather than silently producing a position-dependent binary.

---
 Makefile            | 3 +++
 firmware/objects.mk | 2 +-
 2 files changed, 4 insertions(+), 1 deletion(-)
Makefile +0 -0
index 6b64205..ba06313 100644
@@ -106,6 +106,9 @@ ifndef PLATFORM_RISCV_XLEN
   endif
 endif
 
Quoted 5 lines

Makefile ifndef PLATFORM_RISCV_XLEN · line 111

Quoted 3 lines
Makefile +2 -0
+# Check whether the linker supports creating PIEs
+OPENSBI_LD_PIE := $(shell $(CC) -fPIE -nostdlib -Wl,-pie -x c
 /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n)

For llvm, --target and -fuse-ld need to be added here. Otherwise, this will detect the host instead of what you want

This is patch 4/6, before LLVM support.

Patch 5/6 updates this when adding LLVM support.

Jess

Quoted 24 lines
Quoted 22 lines
Makefile +1 -0
+
 # Setup list of objects.mk files
 ifdef PLATFORM

platform-object-mks=$(shell if [ -d $(platform_src_dir)/ ]; then
find $(platform_src_dir) -iname "objects.mk" | sort -r; fi)

firmware/objects.mk +1 -1
index 3bc83cd..8da7ca3 100644
@@ -14,7 +14,7 @@ firmware-asflags-y +=
 firmware-ldflags-y +=
 
 ifndef FW_PIC
-FW_PIC := y
+FW_PIC := $(OPENSBI_LD_PIE)
 endif
 
 ifeq ($(FW_PIC),y)
Signature · 1 line
2.31.0
#14 lore raw

PATCH v5 4/6 firmware: Only default FW_PIC to y if supported

In reply to Jessica Clarke
Quoted 4 lines · Jessica Clarke

Bare-metal GNU ld does not support PIE, so if using it this will result in a failure to build. Instead, default to FW_PIC=n if not supported. Note that an explicit FW_PIC=y is not overriden, to ensure the build

typo: overridden

Quoted 8 lines

fails rather than silently producing a position-dependent binary.

---
 Makefile            | 3 +++
 firmware/objects.mk | 2 +-
 2 files changed, 4 insertions(+), 1 deletion(-)
Applied
#15 lore raw

PATCH v5 4/6 firmware: Only default FW_PIC to y if supported

Quoted 8 lines · Bin Meng
Quoted 4 lines · Jessica Clarke

Bare-metal GNU ld does not support PIE, so if using it this will result in a failure to build. Instead, default to FW_PIC=n if not supported. Note that an explicit FW_PIC=y is not overriden, to ensure the build

typo: overridden

I have fixed this typo at the time of merging this patch.

Quoted 10 lines
Quoted 9 lines

fails rather than silently producing a position-dependent binary.

---
 Makefile            | 3 +++
 firmware/objects.mk | 2 +-
 2 files changed, 4 insertions(+), 1 deletion(-)
Quoted 2 lines

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

Quoted 5 lines

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

#16 lore raw

PATCH v5 5/6 Makefile: Support building with Clang and LLVM binutils

In reply to Jessica Clarke

This is intended to mirror the Linux kernel. Building with CC=clang will use Clang as the compiler but default to using the existing binutils. Building with LLVM=1 will default to using Clang and LLVM binutils.

Whilst GCC will accept the -N linker option and forward it on to the linker, Clang will not, and so in order to support both compilers we must use -Wl, to forward it to the linker as is required for most other linker options.

Note that there is currently a bug when using Clang as the compiler and riscv64-linux-gnu-ld as the linker for FW_PIC=y. At first glance this appears to be a bug in GNU binutils, but this could also be Clang or OpenSBI at fault in some subtle way. Thus, for now, advise that this combination be avoided.

---
 Makefile  | 66 +++++++++++++++++++++++++++++++++++++++++++++++++------
 README.md | 48 ++++++++++++++++++++++++++++++++++++++--
 2 files changed, 105 insertions(+), 9 deletions(-)
Makefile +59 -7
index ba06313..5c188d5 100644
@@ -76,26 +76,54 @@ OPENSBI_VERSION_MINOR=`grep "define OPENSBI_VERSION_MINOR" $(include_dir)/sbi/sb
 OPENSBI_VERSION_GIT=$(shell if [ -d $(src_dir)/.git ]; then git describe 2> /dev/null; fi)
 
 # Setup compilation commands
+ifneq ($(LLVM),)
+CC		=	clang
+AR		=	llvm-ar
+LD		=	ld.lld
+OBJCOPY		=	llvm-objcopy
+else
 ifdef CROSS_COMPILE
 CC		=	$(CROSS_COMPILE)gcc
-CPP		=	$(CROSS_COMPILE)cpp
 AR		=	$(CROSS_COMPILE)ar
 LD		=	$(CROSS_COMPILE)ld
 OBJCOPY		=	$(CROSS_COMPILE)objcopy
 else
 CC		?=	gcc
-CPP		?=	cpp
 AR		?=	ar
 LD		?=	ld
 OBJCOPY		?=	objcopy
 endif
+endif
+CPP		=	$(CC) -E
 AS		=	$(CC)
 DTC		=	dtc
 
-# Guess the compillers xlen
-OPENSBI_CC_XLEN := $(shell TMP=`$(CC) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP})
+ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),)
+CC_IS_CLANG	=	y
+else
+CC_IS_CLANG	=	n
+endif
+
+ifneq ($(shell $(LD) --version 2>&1 | head -n 1 | grep LLD),)
+LD_IS_LLD	=	y
+else
+LD_IS_LLD	=	n
+endif
+
+ifeq ($(CC_IS_CLANG),y)
+ifneq ($(CROSS_COMPILE),)
+CLANG_TARGET	=	--target=$(notdir $(CROSS_COMPILE:%-=%))
+endif
+endif
+
+# Guess the compiler's XLEN
+OPENSBI_CC_XLEN := $(shell TMP=`$(CC) $(CLANG_TARGET) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP})
+
+# Guess the compiler's ABI and ISA
+ifneq ($(CC_IS_CLANG),y)
 OPENSBI_CC_ABI := $(shell TMP=`$(CC) -v 2>&1 | sed -n 's/.*\(with\-abi=\([a-zA-Z0-9]*\)\).*/\2/p'`; echo $${TMP})
 OPENSBI_CC_ISA := $(shell TMP=`$(CC) -v 2>&1 | sed -n 's/.*\(with\-arch=\([a-zA-Z0-9]*\)\).*/\2/p'`; echo $${TMP})
+endif
 
 # Setup platform XLEN
 ifndef PLATFORM_RISCV_XLEN
@@ -106,8 +134,21 @@ ifndef PLATFORM_RISCV_XLEN
   endif
 endif
 
+ifeq ($(CC_IS_CLANG),y)
+ifeq ($(CROSS_COMPILE),)
+CLANG_TARGET	=	--target=riscv$(PLATFORM_RISCV_XLEN)-unknown-elf
+endif
+endif
+
+ifeq ($(LD_IS_LLD),y)
+RELAX_FLAG	=	-mno-relax
+USE_LD_FLAG	=	-fuse-ld=lld
+else
+USE_LD_FLAG	=	-fuse-ld=bfd
+endif
+
 # Check whether the linker supports creating PIEs
-OPENSBI_LD_PIE := $(shell $(CC) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n)
+OPENSBI_LD_PIE := $(shell $(CC) $(CLANG_TARGET) $(RELAX_FLAG) $(USE_LD_FLAG) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n)
 
 # Setup list of objects.mk files
 ifdef PLATFORM
@@ -197,7 +238,11 @@ else
 endif
 
 # Setup compilation commands flags
-GENFLAGS	=	-I$(platform_src_dir)/include
+ifeq ($(CC_IS_CLANG),y)
+GENFLAGS	+=	$(CLANG_TARGET)
+GENFLAGS	+=	-Wno-unused-command-line-argument
+endif
+GENFLAGS	+=	-I$(platform_src_dir)/include
 GENFLAGS	+=	-I$(include_dir)
 ifneq ($(OPENSBI_VERSION_GIT),)
 GENFLAGS	+=	-DOPENSBI_VERSION_GIT="\"$(OPENSBI_VERSION_GIT)\""
@@ -211,6 +256,7 @@ CFLAGS		+=	-fno-omit-frame-pointer -fno-optimize-sibling-calls
 CFLAGS		+=	-mno-save-restore -mstrict-align
 CFLAGS		+=	-mabi=$(PLATFORM_RISCV_ABI) -march=$(PLATFORM_RISCV_ISA)
 CFLAGS		+=	-mcmodel=$(PLATFORM_RISCV_CODE_MODEL)
+CFLAGS		+=	$(RELAX_FLAG)
 CFLAGS		+=	$(GENFLAGS)
 CFLAGS		+=	$(platform-cflags-y)
 CFLAGS		+=	-fno-pie -no-pie
@@ -225,18 +271,24 @@ ASFLAGS		+=	-fno-omit-frame-pointer -fno-optimize-sibling-calls
 ASFLAGS		+=	-mno-save-restore -mstrict-align
 ASFLAGS		+=	-mabi=$(PLATFORM_RISCV_ABI) -march=$(PLATFORM_RISCV_ISA)
 ASFLAGS		+=	-mcmodel=$(PLATFORM_RISCV_CODE_MODEL)
+ASFLAGS		+=	$(RELAX_FLAG)
 ASFLAGS		+=	$(GENFLAGS)
 ASFLAGS		+=	$(platform-asflags-y)
 ASFLAGS		+=	$(firmware-asflags-y)
 
 ARFLAGS		=	rcs
 
-ELFFLAGS	+=	-Wl,--build-id=none -N -static-libgcc -lgcc
+ELFFLAGS	+=	$(USE_LD_FLAG)
+ELFFLAGS	+=	-Wl,--build-id=none -Wl,-N -static-libgcc -lgcc
 ELFFLAGS	+=	$(platform-ldflags-y)
 ELFFLAGS	+=	$(firmware-ldflags-y)
 
 MERGEFLAGS	+=	-r
+ifeq ($(LD_IS_LLD),y)
+MERGEFLAGS	+=	-b elf
+else
 MERGEFLAGS	+=	-b elf$(PLATFORM_RISCV_XLEN)-littleriscv
+endif
 MERGEFLAGS	+=	-m elf$(PLATFORM_RISCV_XLEN)lriscv
 
 DTSCPPFLAGS	=	$(CPPFLAGS) -nostdinc -nostdlib -fno-builtin -D__DTS__ -x assembler-with-cpp
README.md +46 -2

60 lines not on this page · raw message

Signature · 1 line
2.31.0
#17 lore raw

PATCH v5 5/6 Makefile: Support building with Clang and LLVM binutils

Quoted 95 lines · Jessica Clarke

This is intended to mirror the Linux kernel. Building with CC=clang will use Clang as the compiler but default to using the existing binutils. Building with LLVM=1 will default to using Clang and LLVM binutils.

Whilst GCC will accept the -N linker option and forward it on to the linker, Clang will not, and so in order to support both compilers we must use -Wl, to forward it to the linker as is required for most other linker options.

Note that there is currently a bug when using Clang as the compiler and riscv64-linux-gnu-ld as the linker for FW_PIC=y. At first glance this appears to be a bug in GNU binutils, but this could also be Clang or OpenSBI at fault in some subtle way. Thus, for now, advise that this combination be avoided.

---
 Makefile  | 66 +++++++++++++++++++++++++++++++++++++++++++++++++------
 README.md | 48 ++++++++++++++++++++++++++++++++++++++--
 2 files changed, 105 insertions(+), 9 deletions(-)
Makefile +38 -4
index ba06313..5c188d5 100644
@@ -76,26 +76,54 @@ OPENSBI_VERSION_MINOR=`grep "define OPENSBI_VERSION_MINOR" $(include_dir)/sbi/sb
 OPENSBI_VERSION_GIT=$(shell if [ -d $(src_dir)/.git ]; then git describe 2> /dev/null; fi)
 
 # Setup compilation commands
+ifneq ($(LLVM),)
+CC             =       clang
+AR             =       llvm-ar
+LD             =       ld.lld
+OBJCOPY                =       llvm-objcopy
+else
 ifdef CROSS_COMPILE
 CC             =       $(CROSS_COMPILE)gcc
-CPP            =       $(CROSS_COMPILE)cpp
 AR             =       $(CROSS_COMPILE)ar
 LD             =       $(CROSS_COMPILE)ld
 OBJCOPY                =       $(CROSS_COMPILE)objcopy
 else
 CC             ?=      gcc
-CPP            ?=      cpp
 AR             ?=      ar
 LD             ?=      ld
 OBJCOPY                ?=      objcopy
 endif
+endif
+CPP            =       $(CC) -E
 AS             =       $(CC)
 DTC            =       dtc
 
-# Guess the compillers xlen
-OPENSBI_CC_XLEN := $(shell TMP=`$(CC) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP})
+ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),)
+CC_IS_CLANG    =       y
+else
+CC_IS_CLANG    =       n
+endif
+
+ifneq ($(shell $(LD) --version 2>&1 | head -n 1 | grep LLD),)
+LD_IS_LLD      =       y
+else
+LD_IS_LLD      =       n
+endif
+
+ifeq ($(CC_IS_CLANG),y)
+ifneq ($(CROSS_COMPILE),)
+CLANG_TARGET   =       --target=$(notdir $(CROSS_COMPILE:%-=%))
+endif
+endif
+
+# Guess the compiler's XLEN
+OPENSBI_CC_XLEN := $(shell TMP=`$(CC) $(CLANG_TARGET) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP})
+
+# Guess the compiler's ABI and ISA
+ifneq ($(CC_IS_CLANG),y)
 OPENSBI_CC_ABI := $(shell TMP=`$(CC) -v 2>&1 | sed -n 's/.*\(with\-abi=\([a-zA-Z0-9]*\)\).*/\2/p'`; echo $${TMP})
 OPENSBI_CC_ISA := $(shell TMP=`$(CC) -v 2>&1 | sed -n 's/.*\(with\-arch=\([a-zA-Z0-9]*\)\).*/\2/p'`; echo $${TMP})
+endif
 
 # Setup platform XLEN
 ifndef PLATFORM_RISCV_XLEN
@@ -106,8 +134,21 @@ ifndef PLATFORM_RISCV_XLEN
   endif
 endif
 
+ifeq ($(CC_IS_CLANG),y)
+ifeq ($(CROSS_COMPILE),)
+CLANG_TARGET   =       --target=riscv$(PLATFORM_RISCV_XLEN)-unknown-elf
+endif
+endif
+

Makefile ifndef PLATFORM_RISCV_XLEN · line 151

Quoted 10 lines
Makefile +8 -1
+ifeq ($(LD_IS_LLD),y)
+RELAX_FLAG     =       -mno-relax
+USE_LD_FLAG    =       -fuse-ld=lld
+else
+USE_LD_FLAG    =       -fuse-ld=bfd
+endif
+
 # Check whether the linker supports creating PIEs
-OPENSBI_LD_PIE := $(shell $(CC) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n)
+OPENSBI_LD_PIE := $(shell $(CC) $(CLANG_TARGET) $(RELAX_FLAG) $(USE_LD_FLAG) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n)

It looks like this combination does not work? Both riscv64-unknown-elf-gcc and LLVM are in my $PATH, but ld.lld cannot be located.

$ riscv64-unknown-elf-gcc -fPIE -fuse-ld=lld -nostdlib -Wl,-pie -x c
/dev/null -o /dev/null 2>&1
collect2: fatal error: cannot find 'ld'
compilation terminated.

So this will result in a PDE image, even though LLD supports PIE.

Quoted 105 lines
Makefile +13 -2
 # Setup list of objects.mk files
 ifdef PLATFORM
@@ -197,7 +238,11 @@ else
 endif
 
 # Setup compilation commands flags
-GENFLAGS       =       -I$(platform_src_dir)/include
+ifeq ($(CC_IS_CLANG),y)
+GENFLAGS       +=      $(CLANG_TARGET)
+GENFLAGS       +=      -Wno-unused-command-line-argument
+endif
+GENFLAGS       +=      -I$(platform_src_dir)/include
 GENFLAGS       +=      -I$(include_dir)
 ifneq ($(OPENSBI_VERSION_GIT),)
 GENFLAGS       +=      -DOPENSBI_VERSION_GIT="\"$(OPENSBI_VERSION_GIT)\""
@@ -211,6 +256,7 @@ CFLAGS              +=      -fno-omit-frame-pointer -fno-optimize-sibling-calls
 CFLAGS         +=      -mno-save-restore -mstrict-align
 CFLAGS         +=      -mabi=$(PLATFORM_RISCV_ABI) -march=$(PLATFORM_RISCV_ISA)
 CFLAGS         +=      -mcmodel=$(PLATFORM_RISCV_CODE_MODEL)
+CFLAGS         +=      $(RELAX_FLAG)
 CFLAGS         +=      $(GENFLAGS)
 CFLAGS         +=      $(platform-cflags-y)
 CFLAGS         +=      -fno-pie -no-pie
@@ -225,18 +271,24 @@ ASFLAGS           +=      -fno-omit-frame-pointer -fno-optimize-sibling-calls
 ASFLAGS                +=      -mno-save-restore -mstrict-align
 ASFLAGS                +=      -mabi=$(PLATFORM_RISCV_ABI) -march=$(PLATFORM_RISCV_ISA)
 ASFLAGS                +=      -mcmodel=$(PLATFORM_RISCV_CODE_MODEL)
+ASFLAGS                +=      $(RELAX_FLAG)
 ASFLAGS                +=      $(GENFLAGS)
 ASFLAGS                +=      $(platform-asflags-y)
 ASFLAGS                +=      $(firmware-asflags-y)
 
 ARFLAGS                =       rcs
 
-ELFFLAGS       +=      -Wl,--build-id=none -N -static-libgcc -lgcc
+ELFFLAGS       +=      $(USE_LD_FLAG)
+ELFFLAGS       +=      -Wl,--build-id=none -Wl,-N -static-libgcc -lgcc
 ELFFLAGS       +=      $(platform-ldflags-y)
 ELFFLAGS       +=      $(firmware-ldflags-y)
 
 MERGEFLAGS     +=      -r
+ifeq ($(LD_IS_LLD),y)
+MERGEFLAGS     +=      -b elf
+else
 MERGEFLAGS     +=      -b elf$(PLATFORM_RISCV_XLEN)-littleriscv
+endif
 MERGEFLAGS     +=      -m elf$(PLATFORM_RISCV_XLEN)lriscv
 
 DTSCPPFLAGS    =       $(CPPFLAGS) -nostdinc -nostdlib -fno-builtin -D__DTS__ -x assembler-with-cpp
README.md +38 -2
index 03c02fb..b296f00 100644
@@ -96,8 +96,13 @@ Required Toolchain
 ------------------
 
 OpenSBI can be compiled natively or cross-compiled on a x86 host. For
-cross-compilation, you can build your own toolchain or just download
-a prebuilt one from the [Bootlin toolchain repository].
+cross-compilation, you can build your own toolchain, download a prebuilt one
+from the [Bootlin toolchain repository] or install a distribution-provided
+toolchain; if you opt to use LLVM/Clang, most distribution toolchains will
+support cross-compiling for RISC-V using the same toolchain as your native
+LLVM/Clang toolchain due to LLVM's ability to support multiple backends in the
+same binary, so is often an easy way to obtain a working cross-compilation
+toolchain.
 
 Please note that only a 64-bit version of the toolchain is available in
 the Bootlin toolchain repository for now.
@@ -202,6 +207,45 @@ export PLATFORM_RISCV_XLEN=32
 
 will generate 32-bit OpenSBI images. And vice vesa.
 
+Building with Clang/LLVM
+------------------------
+
+OpenSBI can also be built with Clang/LLVM. To build with just Clang but keep
+the default binutils (which will still use the *CROSS_COMPILE* prefix if
+defined), override the *CC* make variable with:
+```
+make CC=clang
+```
+
+To build with a full LLVM-based toolchain, not just Clang, enable the *LLVM*
+option with:
+```
+make LLVM=1
+```
+
+When using Clang, *CROSS_COMPILE* often does not need to be defined unless
+using GNU binutils with prefixed binary names. *PLATFORM_RISCV_XLEN* will be
+used to infer a default triple to pass to Clang, so if *PLATFORM_RISCV_XLEN*
+itself defaults to an undesired value then prefer setting that rather than the
+full triple via *CROSS_COMPILE*. If *CROSS_COMPILE* is nonetheless defined,
+rather than being used as a prefix for the executable name, it will instead be
+passed via the `--target` option with the trailing `-` removed, so must be a
+valid triple.
+
+These can also be mixed; for example using a GCC cross-compiler but LLVM
+binutils would be:
+```
+make CC=riscv64-unknown-elf-gcc LLVM=1
+```
+

README.md export PLATFORM_RISCV_XLEN=32 · line 250

Quoted 10 lines
README.md +8 -0
+These variables must be passed for all the make invocations described in this
+document.
+
+NOTE: Using Clang with a `riscv*-linux-gnu` GNU binutils linker has been seen
+to produce broken binaries with missing relocations; it is therefore currently
+recommended that this combination be avoided or *FW_PIC=n* be used to disable
+building OpenSBI as a position-independent binary.
+
 Contributing to OpenSBI
 -----------------------

Regards,
Bin

#18 lore raw

PATCH v5 5/6 Makefile: Support building with Clang and LLVM binutils

Quoted 112 lines · Bin Meng
Quoted 103 lines · Jessica Clarke

This is intended to mirror the Linux kernel. Building with CC=clang will use Clang as the compiler but default to using the existing binutils. Building with LLVM=1 will default to using Clang and LLVM binutils.

Whilst GCC will accept the -N linker option and forward it on to the linker, Clang will not, and so in order to support both compilers we must use -Wl, to forward it to the linker as is required for most other linker options.

Note that there is currently a bug when using Clang as the compiler and riscv64-linux-gnu-ld as the linker for FW_PIC=y. At first glance this appears to be a bug in GNU binutils, but this could also be Clang or OpenSBI at fault in some subtle way. Thus, for now, advise that this combination be avoided.

---
Makefile | 66 +++++++++++++++++++++++++++++++++++++++++++++++++------ README.md | 48 ++++++++++++++++++++++++++++++++++++++--

2 files changed, 105 insertions(+), 9 deletions(-)
Makefile +6 -0
index ba06313..5c188d5 100644
@@ -76,26 +76,54 @@ OPENSBI_VERSION_MINOR=`grep "define OPENSBI_VERSION_MINOR" $(include_dir)/sbi/sb
 OPENSBI_VERSION_GIT=$(shell if [ -d $(src_dir)/.git ]; then git describe 2> /dev/null; fi)
 
 # Setup compilation commands
+ifneq ($(LLVM),)
+CC             =       clang
+AR             =       llvm-ar
+LD             =       ld.lld
+OBJCOPY                =       llvm-objcopy
+else

ifdef CROSS_COMPILE
CC = $(CROSS_COMPILE)gcc
-CPP = $(CROSS_COMPILE)cpp

AR             =       $(CROSS_COMPILE)ar
LD             =       $(CROSS_COMPILE)ld
OBJCOPY                =       $(CROSS_COMPILE)objcopy
else
CC             ?=      gcc
-CPP            ?=      cpp
AR             ?=      ar
LD             ?=      ld
OBJCOPY                ?=      objcopy

endif

Makefile +2 -0
+endif
+CPP            =       $(CC) -E
AS             =       $(CC)
DTC            =       dtc
Makefile +23 -2
-# Guess the compillers xlen
-OPENSBI_CC_XLEN := $(shell TMP=`$(CC) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP})
+ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),)
+CC_IS_CLANG    =       y
+else
+CC_IS_CLANG    =       n
+endif
+
+ifneq ($(shell $(LD) --version 2>&1 | head -n 1 | grep LLD),)
+LD_IS_LLD      =       y
+else
+LD_IS_LLD      =       n
+endif
+
+ifeq ($(CC_IS_CLANG),y)
+ifneq ($(CROSS_COMPILE),)
+CLANG_TARGET   =       --target=$(notdir $(CROSS_COMPILE:%-=%))
+endif
+endif
+
+# Guess the compiler's XLEN
+OPENSBI_CC_XLEN := $(shell TMP=`$(CC) $(CLANG_TARGET) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP})
+
+# Guess the compiler's ABI and ISA
+ifneq ($(CC_IS_CLANG),y)

OPENSBI_CC_ABI := $(shell TMP=$(CC) -v 2>&1 | sed -n 's/.*\(with\-abi=\([a-zA-Z0-9]*\)\).*/\2/p'; echo $${TMP}) OPENSBI_CC_ISA := $(shell TMP=$(CC) -v 2>&1 | sed -n 's/.*\(with\-arch=\([a-zA-Z0-9]*\)\).*/\2/p'; echo $${TMP}) +endif

# Setup platform XLEN
ifndef PLATFORM_RISCV_XLEN

Makefile +0 -0
@@ -106,8 +134,21 @@ ifndef PLATFORM_RISCV_XLEN
  endif

endif

Makefile +13 -0
+ifeq ($(CC_IS_CLANG),y)
+ifeq ($(CROSS_COMPILE),)
+CLANG_TARGET   =       --target=riscv$(PLATFORM_RISCV_XLEN)-unknown-elf
+endif
+endif
+
+ifeq ($(LD_IS_LLD),y)
+RELAX_FLAG     =       -mno-relax
+USE_LD_FLAG    =       -fuse-ld=lld
+else
+USE_LD_FLAG    =       -fuse-ld=bfd
+endif
+

# Check whether the linker supports creating PIEs

Makefile ifndef PLATFORM_RISCV_XLEN · line 151

Quoted 2 lines
Makefile +1 -1
-OPENSBI_LD_PIE := $(shell $(CC) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n)
+OPENSBI_LD_PIE := $(shell $(CC) $(CLANG_TARGET) $(RELAX_FLAG) $(USE_LD_FLAG) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n)

It looks like this combination does not work? Both riscv64-unknown-elf-gcc and LLVM are in my $PATH, but ld.lld cannot be located.

Quoted 6 lines
$ riscv64-unknown-elf-gcc -fPIE -fuse-ld=lld -nostdlib -Wl,-pie -x c
/dev/null -o /dev/null 2>&1
collect2: fatal error: cannot find 'ld'
compilation terminated.

So this will result in a PDE image, even though LLD supports PIE.

Isn?t this the same issue as you reported elsewhere, and will ultimately result in the same error, this time not suppressed, at link time too? Using GCC as a driver for LLD has always been a painful combination, in part because historically there?s been very little interest in supporting it from the GNU side; it wasn?t until a few years ago (I think late 2018 or early 2019?) that -fuse-ld=lld was even accepted as an option, and that was just a minimal patch that reused all the logic for BFD and gold despite the fact that that?s unnecessarily restrictive for LLD and doesn?t support cases like this properly.

Jess

#19 lore raw

PATCH v5 5/6 Makefile: Support building with Clang and LLVM binutils

Quoted 119 lines · Jessica Clarke
Quoted 117 lines · Bin Meng
Quoted 103 lines · Jessica Clarke

This is intended to mirror the Linux kernel. Building with CC=clang will use Clang as the compiler but default to using the existing binutils. Building with LLVM=1 will default to using Clang and LLVM binutils.

Whilst GCC will accept the -N linker option and forward it on to the linker, Clang will not, and so in order to support both compilers we must use -Wl, to forward it to the linker as is required for most other linker options.

Note that there is currently a bug when using Clang as the compiler and riscv64-linux-gnu-ld as the linker for FW_PIC=y. At first glance this appears to be a bug in GNU binutils, but this could also be Clang or OpenSBI at fault in some subtle way. Thus, for now, advise that this combination be avoided.

---
Makefile | 66 +++++++++++++++++++++++++++++++++++++++++++++++++------ README.md | 48 ++++++++++++++++++++++++++++++++++++++--

2 files changed, 105 insertions(+), 9 deletions(-)
Makefile +6 -0
index ba06313..5c188d5 100644
@@ -76,26 +76,54 @@ OPENSBI_VERSION_MINOR=`grep "define OPENSBI_VERSION_MINOR" $(include_dir)/sbi/sb
 OPENSBI_VERSION_GIT=$(shell if [ -d $(src_dir)/.git ]; then git describe 2> /dev/null; fi)
 
 # Setup compilation commands
+ifneq ($(LLVM),)
+CC             =       clang
+AR             =       llvm-ar
+LD             =       ld.lld
+OBJCOPY                =       llvm-objcopy
+else

ifdef CROSS_COMPILE
CC = $(CROSS_COMPILE)gcc
-CPP = $(CROSS_COMPILE)cpp

AR             =       $(CROSS_COMPILE)ar
LD             =       $(CROSS_COMPILE)ld
OBJCOPY                =       $(CROSS_COMPILE)objcopy
else
CC             ?=      gcc
-CPP            ?=      cpp
AR             ?=      ar
LD             ?=      ld
OBJCOPY                ?=      objcopy

endif

Makefile +2 -0
+endif
+CPP            =       $(CC) -E
AS             =       $(CC)
DTC            =       dtc
Makefile +23 -2
-# Guess the compillers xlen
-OPENSBI_CC_XLEN := $(shell TMP=`$(CC) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP})
+ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),)
+CC_IS_CLANG    =       y
+else
+CC_IS_CLANG    =       n
+endif
+
+ifneq ($(shell $(LD) --version 2>&1 | head -n 1 | grep LLD),)
+LD_IS_LLD      =       y
+else
+LD_IS_LLD      =       n
+endif
+
+ifeq ($(CC_IS_CLANG),y)
+ifneq ($(CROSS_COMPILE),)
+CLANG_TARGET   =       --target=$(notdir $(CROSS_COMPILE:%-=%))
+endif
+endif
+
+# Guess the compiler's XLEN
+OPENSBI_CC_XLEN := $(shell TMP=`$(CC) $(CLANG_TARGET) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP})
+
+# Guess the compiler's ABI and ISA
+ifneq ($(CC_IS_CLANG),y)

OPENSBI_CC_ABI := $(shell TMP=$(CC) -v 2>&1 | sed -n 's/.*\(with\-abi=\([a-zA-Z0-9]*\)\).*/\2/p'; echo $${TMP}) OPENSBI_CC_ISA := $(shell TMP=$(CC) -v 2>&1 | sed -n 's/.*\(with\-arch=\([a-zA-Z0-9]*\)\).*/\2/p'; echo $${TMP}) +endif

# Setup platform XLEN
ifndef PLATFORM_RISCV_XLEN

Makefile +0 -0
@@ -106,8 +134,21 @@ ifndef PLATFORM_RISCV_XLEN
  endif

endif

Makefile +13 -0
+ifeq ($(CC_IS_CLANG),y)
+ifeq ($(CROSS_COMPILE),)
+CLANG_TARGET   =       --target=riscv$(PLATFORM_RISCV_XLEN)-unknown-elf
+endif
+endif
+
+ifeq ($(LD_IS_LLD),y)
+RELAX_FLAG     =       -mno-relax
+USE_LD_FLAG    =       -fuse-ld=lld
+else
+USE_LD_FLAG    =       -fuse-ld=bfd
+endif
+

# Check whether the linker supports creating PIEs

Makefile ifndef PLATFORM_RISCV_XLEN · line 151

Quoted 2 lines
Makefile +1 -1
-OPENSBI_LD_PIE := $(shell $(CC) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n)
+OPENSBI_LD_PIE := $(shell $(CC) $(CLANG_TARGET) $(RELAX_FLAG) $(USE_LD_FLAG) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n)

It looks like this combination does not work? Both riscv64-unknown-elf-gcc and LLVM are in my $PATH, but ld.lld cannot be located.

$ riscv64-unknown-elf-gcc -fPIE -fuse-ld=lld -nostdlib -Wl,-pie -x c
/dev/null -o /dev/null 2>&1
collect2: fatal error: cannot find 'ld'
compilation terminated.
Quoted 5 lines
Quoted 1 line

So this will result in a PDE image, even though LLD supports PIE.

Isn?t this the same issue as you reported elsewhere, and will ultimately result in the same error, this time not suppressed, at link time too?

Yeah, the same issue I reported in the cover letter. Sorry :)

So I followed your advice in that thread, that I created a symbolic link in my cross-compile GCC toolchain bin directory, riscv64-unknown-elf-ld.lld to point to the actual LLVM ld.lld. Then this 'collect2' error is gone. I suggest we can document this in the doc for the mixed usage (GCC to compile + LLD to link).

But still the build failed in the end:

$ make CC=riscv64-unknown-elf-gcc LLVM=1 PLATFORM=generic V=1

mkdir -p dirname /home/test/git/opensbi/build/platform/generic/firmware/payloads/test_head.o;
echo " AS platform/generic/firmware/payloads/test_head.o";
riscv64-unknown-elf-gcc -g -Wall -nostdlib -fno-omit-frame-pointer -fno-optimize-sibling-calls -mno-save-restore -mstrict-align
-mabi=lp64 -march=rv64imafdc -mcmodel=medany -mno-relax -I/home/test/git/opensbi/platform/generic/include -I/home/test/git/opensbi/include
-DOPENSBI_VERSION_GIT="\"v0.9-111-g401461d\""
-I/home/test/git/opensbi/lib/utils/libfdt/ -DFW_PIC
-DFW_TEXT_START=0x80000000 -DFW_JUMP_ADDR=0x80200000 -DFW_JUMP_FDT_ADDR=0x82200000 -DFW_PAYLOAD_PATH=\"/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.bin\"
-DFW_PAYLOAD_OFFSET=0x200000 -DFW_PAYLOAD_FDT_ADDR=0x82200000 -fpic
-I/home/test/git/opensbi/firmware/payloads -D__OBJNAME__=test_head -c /home/test/git/opensbi/firmware/payloads/test_head.S -o /home/test/git/opensbi/build/platform/generic/firmware/payloads/test_head.o
AS platform/generic/firmware/payloads/test_head.o
mkdir -p dirname /home/test/git/opensbi/build/platform/generic/firmware/payloads/test_main.o;
echo " CC platform/generic/firmware/payloads/test_main.o";
riscv64-unknown-elf-gcc -g -Wall -Werror -ffreestanding -nostdlib -fno-stack-protector -fno-strict-aliasing -O2 -fno-omit-frame-pointer -fno-optimize-sibling-calls -mno-save-restore -mstrict-align
-mabi=lp64 -march=rv64imafdc -mcmodel=medany -mno-relax -I/home/test/git/opensbi/platform/generic/include -I/home/test/git/opensbi/include
-DOPENSBI_VERSION_GIT="\"v0.9-111-g401461d\""
-I/home/test/git/opensbi/lib/utils/libfdt/ -DFW_PIC
-DFW_TEXT_START=0x80000000 -DFW_JUMP_ADDR=0x80200000 -DFW_JUMP_FDT_ADDR=0x82200000 -DFW_PAYLOAD_PATH=\"/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.bin\" -DFW_PAYLOAD_OFFSET=0x200000 -DFW_PAYLOAD_FDT_ADDR=0x82200000
-fno-pie -no-pie -fPIE -pie
-I/home/test/git/opensbi/firmware/payloads -D__OBJNAME__=test_main -c /home/test/git/opensbi/firmware/payloads/test_main.c -o /home/test/git/opensbi/build/platform/generic/firmware/payloads/test_main.o
CC platform/generic/firmware/payloads/test_main.o
mkdir -p dirname /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.o;
echo " MERGE platform/generic/firmware/payloads/test.o"; ld.lld -r
-b elf -m elf64lriscv /home/test/git/opensbi/build/platform/generic/firmware/payloads/test_head.o /home/test/git/opensbi/build/platform/generic/firmware/payloads/test_main.o
-o /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.o
MERGE platform/generic/firmware/payloads/test.o
mkdir -p dirname /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf;
echo " ELF platform/generic/firmware/payloads/test.elf";
riscv64-unknown-elf-gcc -g -Wall -Werror -ffreestanding -nostdlib -fno-stack-protector -fno-strict-aliasing -O2 -fno-omit-frame-pointer -fno-optimize-sibling-calls -mno-save-restore -mstrict-align
-mabi=lp64 -march=rv64imafdc -mcmodel=medany -mno-relax -I/home/test/git/opensbi/platform/generic/include -I/home/test/git/opensbi/include
-DOPENSBI_VERSION_GIT="\"v0.9-111-g401461d\""
-I/home/test/git/opensbi/lib/utils/libfdt/ -DFW_PIC
-DFW_TEXT_START=0x80000000 -DFW_JUMP_ADDR=0x80200000 -DFW_JUMP_FDT_ADDR=0x82200000 -DFW_PAYLOAD_PATH=\"/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.bin\" -DFW_PAYLOAD_OFFSET=0x200000 -DFW_PAYLOAD_FDT_ADDR=0x82200000
-fno-pie -no-pie -fPIE -pie /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.o /home/test/git/opensbi/build/platform/generic/lib/libplatsbi.a
-fuse-ld=lld -Wl,--build-id=none -Wl,-N -Wl,--no-dynamic-linker
-Wl,-pie -Wl,-T/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf.ld -o /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf
ELF platform/generic/firmware/payloads/test.elf
riscv64-unknown-elf-ld.lld: error: test_main.c:(.entry+0x0):
relocation R_RISCV_ALIGN requires unimplemented linker relaxation;
recompile with -mno-relax
riscv64-unknown-elf-ld.lld: error:
/home/test/git/opensbi/firmware/payloads/test_head.S:38:(.entry+0x84):
relocation R_RISCV_ALIGN requires unimplemented linker relaxation;
recompile with -mno-relax
riscv64-unknown-elf-ld.lld: error:
/home/test/git/opensbi/firmware/payloads/test_head.S:39:(.entry+0x90):
relocation R_RISCV_ALIGN requires unimplemented linker relaxation;
recompile with -mno-relax
collect2: error: ld returned 1 exit status

So far the only fully working one is the complete LLVM build. The other 2 mixed usages are broken. It seems the toolchains guys created some sort of flexibility but opened lots of possibilities to break :)

Quoted 8 lines

Using GCC as a driver for LLD has always been a painful combination, in part because historically there?s been very little interest in supporting it from the GNU side; it wasn?t until a few years ago (I think late 2018 or early 2019?) that -fuse-ld=lld was even accepted as an option, and that was just a minimal patch that reused all the logic for BFD and gold despite the fact that that?s unnecessarily restrictive for LLD and doesn?t support cases like this properly.

Regards,
Bin

#20 lore raw

PATCH v5 5/6 Makefile: Support building with Clang and LLVM binutils

Quoted 220 lines · Bin Meng
Quoted 119 lines · Jessica Clarke
Quoted 117 lines · Bin Meng
Quoted 103 lines · Jessica Clarke

This is intended to mirror the Linux kernel. Building with CC=clang will use Clang as the compiler but default to using the existing binutils. Building with LLVM=1 will default to using Clang and LLVM binutils.

Whilst GCC will accept the -N linker option and forward it on to the linker, Clang will not, and so in order to support both compilers we must use -Wl, to forward it to the linker as is required for most other linker options.

Note that there is currently a bug when using Clang as the compiler and riscv64-linux-gnu-ld as the linker for FW_PIC=y. At first glance this appears to be a bug in GNU binutils, but this could also be Clang or OpenSBI at fault in some subtle way. Thus, for now, advise that this combination be avoided.

---
Makefile | 66 +++++++++++++++++++++++++++++++++++++++++++++++++------ README.md | 48 ++++++++++++++++++++++++++++++++++++++--

2 files changed, 105 insertions(+), 9 deletions(-)
Makefile +6 -0
index ba06313..5c188d5 100644
@@ -76,26 +76,54 @@ OPENSBI_VERSION_MINOR=`grep "define OPENSBI_VERSION_MINOR" $(include_dir)/sbi/sb
 OPENSBI_VERSION_GIT=$(shell if [ -d $(src_dir)/.git ]; then git describe 2> /dev/null; fi)
 
 # Setup compilation commands
+ifneq ($(LLVM),)
+CC             =       clang
+AR             =       llvm-ar
+LD             =       ld.lld
+OBJCOPY                =       llvm-objcopy
+else

ifdef CROSS_COMPILE
CC = $(CROSS_COMPILE)gcc
-CPP = $(CROSS_COMPILE)cpp

AR             =       $(CROSS_COMPILE)ar
LD             =       $(CROSS_COMPILE)ld
OBJCOPY                =       $(CROSS_COMPILE)objcopy
else
CC             ?=      gcc
-CPP            ?=      cpp
AR             ?=      ar
LD             ?=      ld
OBJCOPY                ?=      objcopy

endif

Makefile +2 -0
+endif
+CPP            =       $(CC) -E
AS             =       $(CC)
DTC            =       dtc
Makefile +23 -2
-# Guess the compillers xlen
-OPENSBI_CC_XLEN := $(shell TMP=`$(CC) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP})
+ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),)
+CC_IS_CLANG    =       y
+else
+CC_IS_CLANG    =       n
+endif
+
+ifneq ($(shell $(LD) --version 2>&1 | head -n 1 | grep LLD),)
+LD_IS_LLD      =       y
+else
+LD_IS_LLD      =       n
+endif
+
+ifeq ($(CC_IS_CLANG),y)
+ifneq ($(CROSS_COMPILE),)
+CLANG_TARGET   =       --target=$(notdir $(CROSS_COMPILE:%-=%))
+endif
+endif
+
+# Guess the compiler's XLEN
+OPENSBI_CC_XLEN := $(shell TMP=`$(CC) $(CLANG_TARGET) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP})
+
+# Guess the compiler's ABI and ISA
+ifneq ($(CC_IS_CLANG),y)

OPENSBI_CC_ABI := $(shell TMP=$(CC) -v 2>&1 | sed -n 's/.*\(with\-abi=\([a-zA-Z0-9]*\)\).*/\2/p'; echo $${TMP}) OPENSBI_CC_ISA := $(shell TMP=$(CC) -v 2>&1 | sed -n 's/.*\(with\-arch=\([a-zA-Z0-9]*\)\).*/\2/p'; echo $${TMP}) +endif

# Setup platform XLEN
ifndef PLATFORM_RISCV_XLEN

Makefile +0 -0
@@ -106,8 +134,21 @@ ifndef PLATFORM_RISCV_XLEN
 endif

endif

Makefile +13 -0
+ifeq ($(CC_IS_CLANG),y)
+ifeq ($(CROSS_COMPILE),)
+CLANG_TARGET   =       --target=riscv$(PLATFORM_RISCV_XLEN)-unknown-elf
+endif
+endif
+
+ifeq ($(LD_IS_LLD),y)
+RELAX_FLAG     =       -mno-relax
+USE_LD_FLAG    =       -fuse-ld=lld
+else
+USE_LD_FLAG    =       -fuse-ld=bfd
+endif
+

# Check whether the linker supports creating PIEs

Makefile ifndef PLATFORM_RISCV_XLEN · line 151

Quoted 2 lines
Makefile +1 -1
-OPENSBI_LD_PIE := $(shell $(CC) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n)
+OPENSBI_LD_PIE := $(shell $(CC) $(CLANG_TARGET) $(RELAX_FLAG) $(USE_LD_FLAG) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n)

It looks like this combination does not work? Both riscv64-unknown-elf-gcc and LLVM are in my $PATH, but ld.lld cannot be located.

$ riscv64-unknown-elf-gcc -fPIE -fuse-ld=lld -nostdlib -Wl,-pie -x c
/dev/null -o /dev/null 2>&1
collect2: fatal error: cannot find 'ld'
compilation terminated.
Quoted 5 lines
Quoted 1 line

So this will result in a PDE image, even though LLD supports PIE.

Isn?t this the same issue as you reported elsewhere, and will ultimately result in the same error, this time not suppressed, at link time too?

Yeah, the same issue I reported in the cover letter. Sorry :)

So I followed your advice in that thread, that I created a symbolic link in my cross-compile GCC toolchain bin directory, riscv64-unknown-elf-ld.lld to point to the actual LLVM ld.lld. Then this 'collect2' error is gone. I suggest we can document this in the doc for the mixed usage (GCC to compile + LLD to link).

But still the build failed in the end:

$ make CC=riscv64-unknown-elf-gcc LLVM=1 PLATFORM=generic V=1

mkdir -p dirname /home/test/git/opensbi/build/platform/generic/firmware/payloads/test_head.o;
echo " AS platform/generic/firmware/payloads/test_head.o";
riscv64-unknown-elf-gcc -g -Wall -nostdlib -fno-omit-frame-pointer

(quoted diff) +0 -11
-fno-optimize-sibling-calls -mno-save-restore -mstrict-align
-mabi=lp64 -march=rv64imafdc -mcmodel=medany -mno-relax
-I/home/test/git/opensbi/platform/generic/include
-I/home/test/git/opensbi/include
-DOPENSBI_VERSION_GIT="\"v0.9-111-g401461d\""
-I/home/test/git/opensbi/lib/utils/libfdt/   -DFW_PIC
-DFW_TEXT_START=0x80000000 -DFW_JUMP_ADDR=0x80200000
-DFW_JUMP_FDT_ADDR=0x82200000
-DFW_PAYLOAD_PATH=\"/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.bin\"
-DFW_PAYLOAD_OFFSET=0x200000 -DFW_PAYLOAD_FDT_ADDR=0x82200000   -fpic
-I/home/test/git/opensbi/firmware/payloads -D__OBJNAME__=test_head -c

/home/test/git/opensbi/firmware/payloads/test_head.S -o /home/test/git/opensbi/build/platform/generic/firmware/payloads/test_head.o
AS platform/generic/firmware/payloads/test_head.o
mkdir -p dirname /home/test/git/opensbi/build/platform/generic/firmware/payloads/test_main.o;
echo " CC platform/generic/firmware/payloads/test_main.o";
riscv64-unknown-elf-gcc -g -Wall -Werror -ffreestanding -nostdlib

(quoted diff) +0 -13
-fno-stack-protector -fno-strict-aliasing -O2 -fno-omit-frame-pointer
-fno-optimize-sibling-calls -mno-save-restore -mstrict-align
-mabi=lp64 -march=rv64imafdc -mcmodel=medany -mno-relax
-I/home/test/git/opensbi/platform/generic/include
-I/home/test/git/opensbi/include
-DOPENSBI_VERSION_GIT="\"v0.9-111-g401461d\""
-I/home/test/git/opensbi/lib/utils/libfdt/   -DFW_PIC
-DFW_TEXT_START=0x80000000 -DFW_JUMP_ADDR=0x80200000
-DFW_JUMP_FDT_ADDR=0x82200000
-DFW_PAYLOAD_PATH=\"/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.bin\"
-DFW_PAYLOAD_OFFSET=0x200000 -DFW_PAYLOAD_FDT_ADDR=0x82200000
-fno-pie -no-pie  -fPIE -pie
-I/home/test/git/opensbi/firmware/payloads -D__OBJNAME__=test_main -c

/home/test/git/opensbi/firmware/payloads/test_main.c -o /home/test/git/opensbi/build/platform/generic/firmware/payloads/test_main.o
CC platform/generic/firmware/payloads/test_main.o
mkdir -p dirname /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.o;
echo " MERGE platform/generic/firmware/payloads/test.o"; ld.lld -r
-b elf -m elf64lriscv /home/test/git/opensbi/build/platform/generic/firmware/payloads/test_head.o /home/test/git/opensbi/build/platform/generic/firmware/payloads/test_main.o -o /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.o
MERGE platform/generic/firmware/payloads/test.o
mkdir -p dirname /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf;
echo " ELF platform/generic/firmware/payloads/test.elf";
riscv64-unknown-elf-gcc -g -Wall -Werror -ffreestanding -nostdlib

(quoted diff) +0 -12
-fno-stack-protector -fno-strict-aliasing -O2 -fno-omit-frame-pointer
-fno-optimize-sibling-calls -mno-save-restore -mstrict-align
-mabi=lp64 -march=rv64imafdc -mcmodel=medany -mno-relax
-I/home/test/git/opensbi/platform/generic/include
-I/home/test/git/opensbi/include
-DOPENSBI_VERSION_GIT="\"v0.9-111-g401461d\""
-I/home/test/git/opensbi/lib/utils/libfdt/   -DFW_PIC
-DFW_TEXT_START=0x80000000 -DFW_JUMP_ADDR=0x80200000
-DFW_JUMP_FDT_ADDR=0x82200000
-DFW_PAYLOAD_PATH=\"/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.bin\"
-DFW_PAYLOAD_OFFSET=0x200000 -DFW_PAYLOAD_FDT_ADDR=0x82200000
-fno-pie -no-pie  -fPIE -pie

/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.o /home/test/git/opensbi/build/platform/generic/lib/libplatsbi.a

(quoted diff) +0 -3
-fuse-ld=lld -Wl,--build-id=none -Wl,-N   -Wl,--no-dynamic-linker
-Wl,-pie -Wl,-T/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf.ld
-o /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf

ELF platform/generic/firmware/payloads/test.elf
riscv64-unknown-elf-ld.lld: error: test_main.c:(.entry+0x0):
relocation R_RISCV_ALIGN requires unimplemented linker relaxation; recompile with -mno-relax
riscv64-unknown-elf-ld.lld: error:
/home/test/git/opensbi/firmware/payloads/test_head.S:38:(.entry+0x84):
relocation R_RISCV_ALIGN requires unimplemented linker relaxation; recompile with -mno-relax
riscv64-unknown-elf-ld.lld: error:
/home/test/git/opensbi/firmware/payloads/test_head.S:39:(.entry+0x90):
relocation R_RISCV_ALIGN requires unimplemented linker relaxation; recompile with -mno-relax
collect2: error: ld returned 1 exit status

Quoted 3 lines

So far the only fully working one is the complete LLVM build. The other 2 mixed usages are broken. It seems the toolchains guys created some sort of flexibility but opened lots of possibilities to break :)

Ah, right, yes, that old GCC bug that only got fixed a couple of months ago[1]. When generating code with -mno-relax, GCC puts .option norelax in the generated assembly, and so doesn?t bother passing on -mno-relax to the assembler (I don?t think that flag even existed originally, only the assembly directive, but then people realised it was a good idea to be able to have one assembly file that can be used for both). This has the unfortunate effect that, when using GCC to assemble hand-written assembly, -mno-relax does nothing, and you have to pass -Wa,-mno-relax to manually forward it to the assembler. I?ll send a follow-up to add -Wa,-mno-relax to ASFLAGS for GCC+LLD.

Jess

[1] https://github.com/gcc-mirror/gcc/commit/3b0a7d624e64eeb81e4d5e8c62c46d86ef521857

Applied
#21 lore raw

PATCH v5 5/6 Makefile: Support building with Clang and LLVM binutils

In reply to Jessica Clarke
Quoted 10 lines · Jessica Clarke

This is intended to mirror the Linux kernel. Building with CC=clang will use Clang as the compiler but default to using the existing binutils. Building with LLVM=1 will default to using Clang and LLVM binutils.

Whilst GCC will accept the -N linker option and forward it on to the linker, Clang will not, and so in order to support both compilers we must use -Wl, to forward it to the linker as is required for most other linker options.

Quoted 7 lines

Note that there is currently a bug when using Clang as the compiler and riscv64-linux-gnu-ld as the linker for FW_PIC=y. At first glance this appears to be a bug in GNU binutils, but this could also be Clang or OpenSBI at fault in some subtle way. Thus, for now, advise that this combination be avoided.

I tried this patch using GCC linux toolchain and U-Boot SPL which worked fine for me. I don't have bare-metal GCC toolchain and LLVM but I assume LLVM is tested by Jessica.

If there are other compile issues then please send fix as a separate patch. I insist more people to play around with LLVM and GCC bare-metal so that we catch issues early.

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

Quoted 212 lines
---
 Makefile  | 66 +++++++++++++++++++++++++++++++++++++++++++++++++------
 README.md | 48 ++++++++++++++++++++++++++++++++++++++--
 2 files changed, 105 insertions(+), 9 deletions(-)
Makefile +59 -7
index ba06313..5c188d5 100644
@@ -76,26 +76,54 @@ OPENSBI_VERSION_MINOR=`grep "define OPENSBI_VERSION_MINOR" $(include_dir)/sbi/sb
 OPENSBI_VERSION_GIT=$(shell if [ -d $(src_dir)/.git ]; then git describe 2> /dev/null; fi)
 
 # Setup compilation commands
+ifneq ($(LLVM),)
+CC             =       clang
+AR             =       llvm-ar
+LD             =       ld.lld
+OBJCOPY                =       llvm-objcopy
+else
 ifdef CROSS_COMPILE
 CC             =       $(CROSS_COMPILE)gcc
-CPP            =       $(CROSS_COMPILE)cpp
 AR             =       $(CROSS_COMPILE)ar
 LD             =       $(CROSS_COMPILE)ld
 OBJCOPY                =       $(CROSS_COMPILE)objcopy
 else
 CC             ?=      gcc
-CPP            ?=      cpp
 AR             ?=      ar
 LD             ?=      ld
 OBJCOPY                ?=      objcopy
 endif
+endif
+CPP            =       $(CC) -E
 AS             =       $(CC)
 DTC            =       dtc
 
-# Guess the compillers xlen
-OPENSBI_CC_XLEN := $(shell TMP=`$(CC) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP})
+ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),)
+CC_IS_CLANG    =       y
+else
+CC_IS_CLANG    =       n
+endif
+
+ifneq ($(shell $(LD) --version 2>&1 | head -n 1 | grep LLD),)
+LD_IS_LLD      =       y
+else
+LD_IS_LLD      =       n
+endif
+
+ifeq ($(CC_IS_CLANG),y)
+ifneq ($(CROSS_COMPILE),)
+CLANG_TARGET   =       --target=$(notdir $(CROSS_COMPILE:%-=%))
+endif
+endif
+
+# Guess the compiler's XLEN
+OPENSBI_CC_XLEN := $(shell TMP=`$(CC) $(CLANG_TARGET) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP})
+
+# Guess the compiler's ABI and ISA
+ifneq ($(CC_IS_CLANG),y)
 OPENSBI_CC_ABI := $(shell TMP=`$(CC) -v 2>&1 | sed -n 's/.*\(with\-abi=\([a-zA-Z0-9]*\)\).*/\2/p'`; echo $${TMP})
 OPENSBI_CC_ISA := $(shell TMP=`$(CC) -v 2>&1 | sed -n 's/.*\(with\-arch=\([a-zA-Z0-9]*\)\).*/\2/p'`; echo $${TMP})
+endif
 
 # Setup platform XLEN
 ifndef PLATFORM_RISCV_XLEN
@@ -106,8 +134,21 @@ ifndef PLATFORM_RISCV_XLEN
   endif
 endif
 
+ifeq ($(CC_IS_CLANG),y)
+ifeq ($(CROSS_COMPILE),)
+CLANG_TARGET   =       --target=riscv$(PLATFORM_RISCV_XLEN)-unknown-elf
+endif
+endif
+
+ifeq ($(LD_IS_LLD),y)
+RELAX_FLAG     =       -mno-relax
+USE_LD_FLAG    =       -fuse-ld=lld
+else
+USE_LD_FLAG    =       -fuse-ld=bfd
+endif
+
 # Check whether the linker supports creating PIEs
-OPENSBI_LD_PIE := $(shell $(CC) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n)
+OPENSBI_LD_PIE := $(shell $(CC) $(CLANG_TARGET) $(RELAX_FLAG) $(USE_LD_FLAG) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n)
 
 # Setup list of objects.mk files
 ifdef PLATFORM
@@ -197,7 +238,11 @@ else
 endif
 
 # Setup compilation commands flags
-GENFLAGS       =       -I$(platform_src_dir)/include
+ifeq ($(CC_IS_CLANG),y)
+GENFLAGS       +=      $(CLANG_TARGET)
+GENFLAGS       +=      -Wno-unused-command-line-argument
+endif
+GENFLAGS       +=      -I$(platform_src_dir)/include
 GENFLAGS       +=      -I$(include_dir)
 ifneq ($(OPENSBI_VERSION_GIT),)
 GENFLAGS       +=      -DOPENSBI_VERSION_GIT="\"$(OPENSBI_VERSION_GIT)\""
@@ -211,6 +256,7 @@ CFLAGS              +=      -fno-omit-frame-pointer -fno-optimize-sibling-calls
 CFLAGS         +=      -mno-save-restore -mstrict-align
 CFLAGS         +=      -mabi=$(PLATFORM_RISCV_ABI) -march=$(PLATFORM_RISCV_ISA)
 CFLAGS         +=      -mcmodel=$(PLATFORM_RISCV_CODE_MODEL)
+CFLAGS         +=      $(RELAX_FLAG)
 CFLAGS         +=      $(GENFLAGS)
 CFLAGS         +=      $(platform-cflags-y)
 CFLAGS         +=      -fno-pie -no-pie
@@ -225,18 +271,24 @@ ASFLAGS           +=      -fno-omit-frame-pointer -fno-optimize-sibling-calls
 ASFLAGS                +=      -mno-save-restore -mstrict-align
 ASFLAGS                +=      -mabi=$(PLATFORM_RISCV_ABI) -march=$(PLATFORM_RISCV_ISA)
 ASFLAGS                +=      -mcmodel=$(PLATFORM_RISCV_CODE_MODEL)
+ASFLAGS                +=      $(RELAX_FLAG)
 ASFLAGS                +=      $(GENFLAGS)
 ASFLAGS                +=      $(platform-asflags-y)
 ASFLAGS                +=      $(firmware-asflags-y)
 
 ARFLAGS                =       rcs
 
-ELFFLAGS       +=      -Wl,--build-id=none -N -static-libgcc -lgcc
+ELFFLAGS       +=      $(USE_LD_FLAG)
+ELFFLAGS       +=      -Wl,--build-id=none -Wl,-N -static-libgcc -lgcc
 ELFFLAGS       +=      $(platform-ldflags-y)
 ELFFLAGS       +=      $(firmware-ldflags-y)
 
 MERGEFLAGS     +=      -r
+ifeq ($(LD_IS_LLD),y)
+MERGEFLAGS     +=      -b elf
+else
 MERGEFLAGS     +=      -b elf$(PLATFORM_RISCV_XLEN)-littleriscv
+endif
 MERGEFLAGS     +=      -m elf$(PLATFORM_RISCV_XLEN)lriscv
 
 DTSCPPFLAGS    =       $(CPPFLAGS) -nostdinc -nostdlib -fno-builtin -D__DTS__ -x assembler-with-cpp
README.md +46 -3
index 03c02fb..b296f00 100644
@@ -96,8 +96,13 @@ Required Toolchain
 ------------------
 
 OpenSBI can be compiled natively or cross-compiled on a x86 host. For
-cross-compilation, you can build your own toolchain or just download
-a prebuilt one from the [Bootlin toolchain repository].
+cross-compilation, you can build your own toolchain, download a prebuilt one
+from the [Bootlin toolchain repository] or install a distribution-provided
+toolchain; if you opt to use LLVM/Clang, most distribution toolchains will
+support cross-compiling for RISC-V using the same toolchain as your native
+LLVM/Clang toolchain due to LLVM's ability to support multiple backends in the
+same binary, so is often an easy way to obtain a working cross-compilation
+toolchain.
 
 Please note that only a 64-bit version of the toolchain is available in
 the Bootlin toolchain repository for now.
@@ -202,6 +207,45 @@ export PLATFORM_RISCV_XLEN=32
 
 will generate 32-bit OpenSBI images. And vice vesa.
 
+Building with Clang/LLVM
+------------------------
+
+OpenSBI can also be built with Clang/LLVM. To build with just Clang but keep
+the default binutils (which will still use the *CROSS_COMPILE* prefix if
+defined), override the *CC* make variable with:
+```
+make CC=clang
+```
+
+To build with a full LLVM-based toolchain, not just Clang, enable the *LLVM*
+option with:
+```
+make LLVM=1
+```
+
+When using Clang, *CROSS_COMPILE* often does not need to be defined unless
+using GNU binutils with prefixed binary names. *PLATFORM_RISCV_XLEN* will be
+used to infer a default triple to pass to Clang, so if *PLATFORM_RISCV_XLEN*
+itself defaults to an undesired value then prefer setting that rather than the
+full triple via *CROSS_COMPILE*. If *CROSS_COMPILE* is nonetheless defined,
+rather than being used as a prefix for the executable name, it will instead be
+passed via the `--target` option with the trailing `-` removed, so must be a
+valid triple.
+
+These can also be mixed; for example using a GCC cross-compiler but LLVM
+binutils would be:
+```
+make CC=riscv64-unknown-elf-gcc LLVM=1
+```
+
+These variables must be passed for all the make invocations described in this
+document.
+
+NOTE: Using Clang with a `riscv*-linux-gnu` GNU binutils linker has been seen
+to produce broken binaries with missing relocations; it is therefore currently
+recommended that this combination be avoided or *FW_PIC=n* be used to disable
+building OpenSBI as a position-independent binary.
+
 Contributing to OpenSBI
 -----------------------
 
--

2.31.0

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

#22 lore raw

PATCH v5 5/6 Makefile: Support building with Clang and LLVM binutils

Quoted 18 lines · Anup Patel
Quoted 16 lines · Jessica Clarke

This is intended to mirror the Linux kernel. Building with CC=clang will use Clang as the compiler but default to using the existing binutils. Building with LLVM=1 will default to using Clang and LLVM binutils.

Whilst GCC will accept the -N linker option and forward it on to the linker, Clang will not, and so in order to support both compilers we must use -Wl, to forward it to the linker as is required for most other linker options.

Note that there is currently a bug when using Clang as the compiler and riscv64-linux-gnu-ld as the linker for FW_PIC=y. At first glance this appears to be a bug in GNU binutils, but this could also be Clang or OpenSBI at fault in some subtle way. Thus, for now, advise that this combination be avoided.

Quoted 5 lines
Quoted 1 line

I tried this patch using GCC linux toolchain and U-Boot SPL which worked fine for me. I don't have bare-metal GCC toolchain and LLVM but I assume LLVM is tested by Jessica.

It's nothing related to bare-metal, but a mixed usage of GCC and LLVM (the 3rd combination in the README)

Quoted 9 lines

If there are other compile issues then please send fix as a separate patch. I insist more people to play around with LLVM and GCC bare-metal so that we catch issues early.

Applied this patch to the riscv/opensbi repo.

Oops, I was about to give my RB / TB tags, but that's not a big deal :)

Regards,
Bin

#23 lore raw

PATCH v5 6/6 Drop dependency on libgcc by importing part of FreeBSD's libquad

In reply to Jessica Clarke

We only need libgcc for 64-bit division on RV32. Whilst GCC toolchains bundle libgcc, Clang toolchains tend not to ship libclang_rt.builtins given every compiler is a cross-compiler for every target and so you would need a silly number of builds of it, with only the native library available; only vendor-provided Clang toolchains specifically for bare metal cross-compiling are likely to provide it.

Thus, import part of FreeBSD's implementation of the division support functions needed and stop linking against libgcc.

---
 Makefile                              |   2 +-
 lib/utils/libquad/divdi3.c            |  64 ++++++
 lib/utils/libquad/include/limits.h    |  12 ++
 lib/utils/libquad/include/sys/cdefs.h |  12 ++
 lib/utils/libquad/include/sys/types.h |  25 +++
 lib/utils/libquad/moddi3.c            |  66 ++++++
 lib/utils/libquad/objects.mk          |  14 ++
 lib/utils/libquad/qdivrem.c           | 278 ++++++++++++++++++++++++++
 lib/utils/libquad/quad.h              | 105 ++++++++++
 lib/utils/libquad/udivdi3.c           |  52 +++++
 lib/utils/libquad/umoddi3.c           |  54 +++++
 11 files changed, 683 insertions(+), 1 deletion(-)
Makefile +1 -1
index 5c188d5..129b8ed 100644
@@ -279,7 +279,7 @@ ASFLAGS		+=	$(firmware-asflags-y)
 ARFLAGS		=	rcs
 
 ELFFLAGS	+=	$(USE_LD_FLAG)
-ELFFLAGS	+=	-Wl,--build-id=none -Wl,-N -static-libgcc -lgcc
+ELFFLAGS	+=	-Wl,--build-id=none -Wl,-N
 ELFFLAGS	+=	$(platform-ldflags-y)
 ELFFLAGS	+=	$(firmware-ldflags-y)
 
lib/utils/libquad/divdi3.c new file +64 -0

64 lines not on this page · raw message

lib/utils/libquad/include/limits.h new file +12 -0
new file mode 100644
index 0000000..9c69d27
@@ -0,0 +1,12 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2021 Jessica Clarke <[email protected]>
+ */
+
+#ifndef __LIMITS_H__
+#define __LIMITS_H__
+
+#define CHAR_BIT 8
+
+#endif
lib/utils/libquad/include/sys/cdefs.h new file +12 -0
new file mode 100644
index 0000000..e8d6faa
@@ -0,0 +1,12 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2021 Jessica Clarke <[email protected]>
+ */
+
+#ifndef __SYS_CDEFS_H__
+#define __SYS_CDEFS_H__
+
+#define __FBSDID(s) struct __hack
+
+#endif
lib/utils/libquad/include/sys/types.h new file +25 -0
new file mode 100644
index 0000000..e372437
@@ -0,0 +1,25 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2021 Jessica Clarke <[email protected]>
+ */
+
+#ifndef __SYS_TYPES_H__
+#define __SYS_TYPES_H__
+
+#include <sbi/sbi_types.h>
+
+typedef unsigned long u_long;
+
+typedef int64_t quad_t;
+typedef uint64_t u_quad_t;
+
+#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
+#define _QUAD_LOWWORD 1
+#define _QUAD_HIGHWORD 0
+#else
+#define _QUAD_LOWWORD 0
+#define _QUAD_HIGHWORD 1
+#endif
+
+#endif
lib/utils/libquad/moddi3.c new file +66 -0

66 lines not on this page · raw message

lib/utils/libquad/objects.mk new file +14 -0
new file mode 100644
index 0000000..5e53cd0
@@ -0,0 +1,14 @@
+#
+# SPDX-License-Identifier: BSD-2-Clause
+#
+# Copyright (c) 2021 Jessica Clarke <[email protected]>
+#
+
+ifeq ($(PLATFORM_RISCV_XLEN),32)
+libsbiutils-objs-y += libquad/divdi3.o
+libsbiutils-objs-y += libquad/moddi3.o
+libsbiutils-objs-y += libquad/qdivrem.o
+libsbiutils-objs-y += libquad/udivdi3.o
+libsbiutils-objs-y += libquad/umoddi3.o
+libsbiutils-genflags-y += -I$(libsbiutils_dir)/libquad/include
+endif
lib/utils/libquad/qdivrem.c new file +278 -0

278 lines not on this page · raw message

lib/utils/libquad/quad.h new file +105 -0

105 lines not on this page · raw message

lib/utils/libquad/udivdi3.c new file +52 -0

52 lines not on this page · raw message

lib/utils/libquad/umoddi3.c new file +54 -0

54 lines not on this page · raw message

Signature · 1 line
2.31.0
Applied
#24 lore raw

PATCH v5 6/6 Drop dependency on libgcc by importing part of FreeBSD's libquad

Quoted 8 lines · Jessica Clarke

We only need libgcc for 64-bit division on RV32. Whilst GCC toolchains bundle libgcc, Clang toolchains tend not to ship libclang_rt.builtins given every compiler is a cross-compiler for every target and so you would need a silly number of builds of it, with only the native library available; only vendor-provided Clang toolchains specifically for bare metal cross-compiling are likely to provide it.

Quoted 4 lines

Thus, import part of FreeBSD's implementation of the division support functions needed and stop linking against libgcc.

I tried this patch with GCC linux toolchain and U-Boot SPL and it worked fine for me. Other please try LLVM and bare-metal toolchain at your end and if you see any issue then please send a fix patch.

Applied this patch to the riscv/opensbi repo.

@Bin Meng Can you also try RV32 U-Boot SPL ? I did not try that.

Regards,
Anup

Quoted 777 lines
---
 Makefile                              |   2 +-
 lib/utils/libquad/divdi3.c            |  64 ++++++
 lib/utils/libquad/include/limits.h    |  12 ++
 lib/utils/libquad/include/sys/cdefs.h |  12 ++
 lib/utils/libquad/include/sys/types.h |  25 +++
 lib/utils/libquad/moddi3.c            |  66 ++++++
 lib/utils/libquad/objects.mk          |  14 ++
 lib/utils/libquad/qdivrem.c           | 278 ++++++++++++++++++++++++++
 lib/utils/libquad/quad.h              | 105 ++++++++++
 lib/utils/libquad/udivdi3.c           |  52 +++++
 lib/utils/libquad/umoddi3.c           |  54 +++++
 11 files changed, 683 insertions(+), 1 deletion(-)
Makefile +1 -1
index 5c188d5..129b8ed 100644
@@ -279,7 +279,7 @@ ASFLAGS             +=      $(firmware-asflags-y)
 ARFLAGS                =       rcs
 
 ELFFLAGS       +=      $(USE_LD_FLAG)
-ELFFLAGS       +=      -Wl,--build-id=none -Wl,-N -static-libgcc -lgcc
+ELFFLAGS       +=      -Wl,--build-id=none -Wl,-N
 ELFFLAGS       +=      $(platform-ldflags-y)
 ELFFLAGS       +=      $(firmware-ldflags-y)
 
lib/utils/libquad/divdi3.c new file +64 -0
new file mode 100644
index 0000000..c5b4a37
@@ -0,0 +1,64 @@
+/*-
+ * SPDX-License-Identifier: BSD-3-Clause
+ *
+ * Copyright (c) 1992, 1993
+ *     The Regents of the University of California.  All rights reserved.
+ *
+ * This software was developed by the Computer Systems Engineering group
+ * at Lawrence Berkeley Laboratory under DARPA contract BG 91-66 and
+ * contributed to Berkeley.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ * 3. Neither the name of the University nor the names of its contributors
+ *    may be used to endorse or promote products derived from this software
+ *    without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#if defined(LIBC_SCCS) && !defined(lint)
+static char sccsid[] = "@(#)divdi3.c   8.1 (Berkeley) 6/4/93";
+#endif /* LIBC_SCCS and not lint */
+#include <sys/cdefs.h>
+__FBSDID("$FreeBSD$");
+
+#include "quad.h"
+
+/*
+ * Divide two signed quads.
+ * ??? if -1/2 should produce -1 on this machine, this code is wrong
+ */
+quad_t
+__divdi3(quad_t a, quad_t b)
+{
+       u_quad_t ua, ub, uq;
+       int neg;
+
+       if (a < 0)
+               ua = -(u_quad_t)a, neg = 1;
+       else
+               ua = a, neg = 0;
+       if (b < 0)
+               ub = -(u_quad_t)b, neg ^= 1;
+       else
+               ub = b;
+       uq = __qdivrem(ua, ub, (u_quad_t *)0);
+       return (neg ? -uq : uq);
+}
lib/utils/libquad/include/limits.h new file +12 -0
new file mode 100644
index 0000000..9c69d27
@@ -0,0 +1,12 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2021 Jessica Clarke <[email protected]>
+ */
+
+#ifndef __LIMITS_H__
+#define __LIMITS_H__
+
+#define CHAR_BIT 8
+
+#endif
lib/utils/libquad/include/sys/cdefs.h new file +12 -0
new file mode 100644
index 0000000..e8d6faa
@@ -0,0 +1,12 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2021 Jessica Clarke <[email protected]>
+ */
+
+#ifndef __SYS_CDEFS_H__
+#define __SYS_CDEFS_H__
+
+#define __FBSDID(s) struct __hack
+
+#endif
lib/utils/libquad/include/sys/types.h new file +25 -0
new file mode 100644
index 0000000..e372437
@@ -0,0 +1,25 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2021 Jessica Clarke <[email protected]>
+ */
+
+#ifndef __SYS_TYPES_H__
+#define __SYS_TYPES_H__
+
+#include <sbi/sbi_types.h>
+
+typedef unsigned long u_long;
+
+typedef int64_t quad_t;
+typedef uint64_t u_quad_t;
+
+#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
+#define _QUAD_LOWWORD 1
+#define _QUAD_HIGHWORD 0
+#else
+#define _QUAD_LOWWORD 0
+#define _QUAD_HIGHWORD 1
+#endif
+
+#endif
lib/utils/libquad/moddi3.c new file +66 -0
new file mode 100644
index 0000000..fc13b4d
@@ -0,0 +1,66 @@
+/*-
+ * SPDX-License-Identifier: BSD-3-Clause
+ *
+ * Copyright (c) 1992, 1993
+ *     The Regents of the University of California.  All rights reserved.
+ *
+ * This software was developed by the Computer Systems Engineering group
+ * at Lawrence Berkeley Laboratory under DARPA contract BG 91-66 and
+ * contributed to Berkeley.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ * 3. Neither the name of the University nor the names of its contributors
+ *    may be used to endorse or promote products derived from this software
+ *    without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#if defined(LIBC_SCCS) && !defined(lint)
+static char sccsid[] = "@(#)moddi3.c   8.1 (Berkeley) 6/4/93";
+#endif /* LIBC_SCCS and not lint */
+#include <sys/cdefs.h>
+__FBSDID("$FreeBSD$");
+
+#include "quad.h"
+
+/*
+ * Return remainder after dividing two signed quads.
+ *
+ * XXX
+ * If -1/2 should produce -1 on this machine, this code is wrong.
+ */
+quad_t
+__moddi3(quad_t a, quad_t b)
+{
+       u_quad_t ua, ub, ur;
+       int neg;
+
+       if (a < 0)
+               ua = -(u_quad_t)a, neg = 1;
+       else
+               ua = a, neg = 0;
+       if (b < 0)
+               ub = -(u_quad_t)b;
+       else
+               ub = b;
+       (void)__qdivrem(ua, ub, &ur);
+       return (neg ? -ur : ur);
+}
lib/utils/libquad/objects.mk new file +14 -0
new file mode 100644
index 0000000..5e53cd0
@@ -0,0 +1,14 @@
+#
+# SPDX-License-Identifier: BSD-2-Clause
+#
+# Copyright (c) 2021 Jessica Clarke <[email protected]>
+#
+
+ifeq ($(PLATFORM_RISCV_XLEN),32)
+libsbiutils-objs-y += libquad/divdi3.o
+libsbiutils-objs-y += libquad/moddi3.o
+libsbiutils-objs-y += libquad/qdivrem.o
+libsbiutils-objs-y += libquad/udivdi3.o
+libsbiutils-objs-y += libquad/umoddi3.o
+libsbiutils-genflags-y += -I$(libsbiutils_dir)/libquad/include
+endif
lib/utils/libquad/qdivrem.c new file +278 -0
new file mode 100644
index 0000000..ea09e7f
@@ -0,0 +1,278 @@
+/*-
+ * SPDX-License-Identifier: BSD-3-Clause
+ *
+ * Copyright (c) 1992, 1993
+ *     The Regents of the University of California.  All rights reserved.
+ *
+ * This software was developed by the Computer Systems Engineering group
+ * at Lawrence Berkeley Laboratory under DARPA contract BG 91-66 and
+ * contributed to Berkeley.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ * 3. Neither the name of the University nor the names of its contributors
+ *    may be used to endorse or promote products derived from this software
+ *    without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#if defined(LIBC_SCCS) && !defined(lint)
+static char sccsid[] = "@(#)qdivrem.c  8.1 (Berkeley) 6/4/93";
+#endif /* LIBC_SCCS and not lint */
+#include <sys/cdefs.h>
+__FBSDID("$FreeBSD$");
+
+/*
+ * Multiprecision divide.  This algorithm is from Knuth vol. 2 (2nd ed),
+ * section 4.3.1, pp. 257--259.
+ */
+
+#include "quad.h"
+
+#define        B       (1L << HALF_BITS)       /* digit base */
+
+/* Combine two `digits' to make a single two-digit number. */
+#define        COMBINE(a, b) (((u_long)(a) << HALF_BITS) | (b))
+
+/* select a type for digits in base B: use unsigned short if they fit */
+#if ULONG_MAX == 0xffffffff && USHRT_MAX >= 0xffff
+typedef unsigned short digit;
+#else
+typedef u_long digit;
+#endif
+
+/*
+ * Shift p[0]..p[len] left `sh' bits, ignoring any bits that
+ * `fall out' the left (there never will be any such anyway).
+ * We may assume len >= 0.  NOTE THAT THIS WRITES len+1 DIGITS.
+ */
+static void
+shl(digit *p, int len, int sh)
+{
+       int i;
+
+       for (i = 0; i < len; i++)
+               p[i] = LHALF(p[i] << sh) | (p[i + 1] >> (HALF_BITS - sh));
+       p[i] = LHALF(p[i] << sh);
+}
+
+/*
+ * __qdivrem(u, v, rem) returns u/v and, optionally, sets *rem to u%v.
+ *
+ * We do this in base 2-sup-HALF_BITS, so that all intermediate products
+ * fit within u_long.  As a consequence, the maximum length dividend and
+ * divisor are 4 `digits' in this base (they are shorter if they have
+ * leading zeros).
+ */
+u_quad_t
+__qdivrem(u_quad_t uq, u_quad_t vq, u_quad_t *arq)
+{
+       union uu tmp;
+       digit *u, *v, *q;
+       digit v1, v2;
+       u_long qhat, rhat, t;
+       int m, n, d, j, i;
+       digit uspace[5], vspace[5], qspace[5];
+
+       /*
+        * Take care of special cases: divide by zero, and u < v.
+        */
+       if (vq == 0) {
+               /* divide by zero. */
+               static volatile const unsigned int zero = 0;
+
+               tmp.ul[H] = tmp.ul[L] = 1 / zero;
+               if (arq)
+                       *arq = uq;
+               return (tmp.q);
+       }
+       if (uq < vq) {
+               if (arq)
+                       *arq = uq;
+               return (0);
+       }
+       u = &uspace[0];
+       v = &vspace[0];
+       q = &qspace[0];
+
+       /*
+        * Break dividend and divisor into digits in base B, then
+        * count leading zeros to determine m and n.  When done, we
+        * will have:
+        *      u = (u[1]u[2]...u[m+n]) sub B
+        *      v = (v[1]v[2]...v[n]) sub B
+        *      v[1] != 0
+        *      1 < n <= 4 (if n = 1, we use a different division algorithm)
+        *      m >= 0 (otherwise u < v, which we already checked)
+        *      m + n = 4
+        * and thus
+        *      m = 4 - n <= 2
+        */
+       tmp.uq = uq;
+       u[0] = 0;
+       u[1] = HHALF(tmp.ul[H]);
+       u[2] = LHALF(tmp.ul[H]);
+       u[3] = HHALF(tmp.ul[L]);
+       u[4] = LHALF(tmp.ul[L]);
+       tmp.uq = vq;
+       v[1] = HHALF(tmp.ul[H]);
+       v[2] = LHALF(tmp.ul[H]);
+       v[3] = HHALF(tmp.ul[L]);
+       v[4] = LHALF(tmp.ul[L]);
+       for (n = 4; v[1] == 0; v++) {
+               if (--n == 1) {
+                       u_long rbj;     /* r*B+u[j] (not root boy jim) */
+                       digit q1, q2, q3, q4;
+
+                       /*
+                        * Change of plan, per exercise 16.
+                        *      r = 0;
+                        *      for j = 1..4:
+                        *              q[j] = floor((r*B + u[j]) / v),
+                        *              r = (r*B + u[j]) % v;
+                        * We unroll this completely here.
+                        */
+                       t = v[2];       /* nonzero, by definition */
+                       q1 = u[1] / t;
+                       rbj = COMBINE(u[1] % t, u[2]);
+                       q2 = rbj / t;
+                       rbj = COMBINE(rbj % t, u[3]);
+                       q3 = rbj / t;
+                       rbj = COMBINE(rbj % t, u[4]);
+                       q4 = rbj / t;
+                       if (arq)
+                               *arq = rbj % t;
+                       tmp.ul[H] = COMBINE(q1, q2);
+                       tmp.ul[L] = COMBINE(q3, q4);
+                       return (tmp.q);
+               }
+       }
+
+       /*
+        * By adjusting q once we determine m, we can guarantee that
+        * there is a complete four-digit quotient at &qspace[1] when
+        * we finally stop.
+        */
+       for (m = 4 - n; u[1] == 0; u++)
+               m--;
+       for (i = 4 - m; --i >= 0;)
+               q[i] = 0;
+       q += 4 - m;
+
+       /*
+        * Here we run Program D, translated from MIX to C and acquiring
+        * a few minor changes.
+        *
+        * D1: choose multiplier 1 << d to ensure v[1] >= B/2.
+        */
+       d = 0;
+       for (t = v[1]; t < B / 2; t <<= 1)
+               d++;
+       if (d > 0) {
+               shl(&u[0], m + n, d);           /* u <<= d */
+               shl(&v[1], n - 1, d);           /* v <<= d */
+       }
+       /*
+        * D2: j = 0.
+        */
+       j = 0;
+       v1 = v[1];      /* for D3 -- note that v[1..n] are constant */
+       v2 = v[2];      /* for D3 */
+       do {
+               digit uj0, uj1, uj2;
+
+               /*
+                * D3: Calculate qhat (\^q, in TeX notation).
+                * Let qhat = min((u[j]*B + u[j+1])/v[1], B-1), and
+                * let rhat = (u[j]*B + u[j+1]) mod v[1].
+                * While rhat < B and v[2]*qhat > rhat*B+u[j+2],
+                * decrement qhat and increase rhat correspondingly.
+                * Note that if rhat >= B, v[2]*qhat < rhat*B.
+                */
+               uj0 = u[j + 0]; /* for D3 only -- note that u[j+...] change */
+               uj1 = u[j + 1]; /* for D3 only */
+               uj2 = u[j + 2]; /* for D3 only */
+               if (uj0 == v1) {
+                       qhat = B;
+                       rhat = uj1;
+                       goto qhat_too_big;
+               } else {
+                       u_long n = COMBINE(uj0, uj1);
+                       qhat = n / v1;
+                       rhat = n % v1;
+               }
+               while (v2 * qhat > COMBINE(rhat, uj2)) {
+       qhat_too_big:
+                       qhat--;
+                       if ((rhat += v1) >= B)
+                               break;
+               }
+               /*
+                * D4: Multiply and subtract.
+                * The variable `t' holds any borrows across the loop.
+                * We split this up so that we do not require v[0] = 0,
+                * and to eliminate a final special case.
+                */
+               for (t = 0, i = n; i > 0; i--) {
+                       t = u[i + j] - v[i] * qhat - t;
+                       u[i + j] = LHALF(t);
+                       t = (B - HHALF(t)) & (B - 1);
+               }
+               t = u[j] - t;
+               u[j] = LHALF(t);
+               /*
+                * D5: test remainder.
+                * There is a borrow if and only if HHALF(t) is nonzero;
+                * in that (rare) case, qhat was too large (by exactly 1).
+                * Fix it by adding v[1..n] to u[j..j+n].
+                */
+               if (HHALF(t)) {
+                       qhat--;
+                       for (t = 0, i = n; i > 0; i--) { /* D6: add back. */
+                               t += u[i + j] + v[i];
+                               u[i + j] = LHALF(t);
+                               t = HHALF(t);
+                       }
+                       u[j] = LHALF(u[j] + t);
+               }
+               q[j] = qhat;
+       } while (++j <= m);             /* D7: loop on j. */
+
+       /*
+        * If caller wants the remainder, we have to calculate it as
+        * u[m..m+n] >> d (this is at most n digits and thus fits in
+        * u[m+1..m+n], but we may need more source digits).
+        */
+       if (arq) {
+               if (d) {
+                       for (i = m + n; i > m; --i)
+                               u[i] = (u[i] >> d) |
+                                   LHALF(u[i - 1] << (HALF_BITS - d));
+                       u[i] = 0;
+               }
+               tmp.ul[H] = COMBINE(uspace[1], uspace[2]);
+               tmp.ul[L] = COMBINE(uspace[3], uspace[4]);
+               *arq = tmp.q;
+       }
+
+       tmp.ul[H] = COMBINE(qspace[1], qspace[2]);
+       tmp.ul[L] = COMBINE(qspace[3], qspace[4]);
+       return (tmp.q);
+}
lib/utils/libquad/quad.h new file +105 -0
new file mode 100644
index 0000000..a9f8447
@@ -0,0 +1,105 @@
+/*-
+ * SPDX-License-Identifier: BSD-3-Clause
+ *
+ * Copyright (c) 1992, 1993
+ *     The Regents of the University of California.  All rights reserved.
+ *
+ * This software was developed by the Computer Systems Engineering group
+ * at Lawrence Berkeley Laboratory under DARPA contract BG 91-66 and
+ * contributed to Berkeley.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ * 3. Neither the name of the University nor the names of its contributors
+ *    may be used to endorse or promote products derived from this software
+ *    without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ *     @(#)quad.h      8.1 (Berkeley) 6/4/93
+ * $FreeBSD$
+ */
+
+/*
+ * Quad arithmetic.
+ *
+ * This library makes the following assumptions:
+ *
+ *  - The type long long (aka quad_t) exists.
+ *
+ *  - A quad variable is exactly twice as long as `long'.
+ *
+ *  - The machine's arithmetic is two's complement.
+ *
+ * This library can provide 128-bit arithmetic on a machine with 128-bit
+ * quads and 64-bit longs, for instance, or 96-bit arithmetic on machines
+ * with 48-bit longs.
+ */
+
+#include <sys/types.h>
+#include <limits.h>
+
+/*
+ * Depending on the desired operation, we view a `long long' (aka quad_t) in
+ * one or more of the following formats.
+ */
+union uu {
+       quad_t  q;              /* as a (signed) quad */
+       quad_t  uq;             /* as an unsigned quad */
+       long    sl[2];          /* as two signed longs */
+       u_long  ul[2];          /* as two unsigned longs */
+};
+
+/*
+ * Define high and low longwords.
+ */
+#define        H               _QUAD_HIGHWORD
+#define        L               _QUAD_LOWWORD
+
+/*
+ * Total number of bits in a quad_t and in the pieces that make it up.
+ * These are used for shifting, and also below for halfword extraction
+ * and assembly.
+ */
+#define        QUAD_BITS       (sizeof(quad_t) * CHAR_BIT)
+#define        LONG_BITS       (sizeof(long) * CHAR_BIT)
+#define        HALF_BITS       (sizeof(long) * CHAR_BIT / 2)
+
+/*
+ * Extract high and low shortwords from longword, and move low shortword of
+ * longword to upper half of long, i.e., produce the upper longword of
+ * ((quad_t)(x) << (number_of_bits_in_long/2)).  (`x' must actually be u_long.)
+ *
+ * These are used in the multiply code, to split a longword into upper
+ * and lower halves, and to reassemble a product as a quad_t, shifted left
+ * (sizeof(long)*CHAR_BIT/2).
+ */
+#define        HHALF(x)        ((x) >> HALF_BITS)
+#define        LHALF(x)        ((x) & ((1L << HALF_BITS) - 1))
+#define        LHUP(x)         ((x) << HALF_BITS)
+
+int            __cmpdi2(quad_t a, quad_t b);
+quad_t         __divdi3(quad_t a, quad_t b);
+quad_t         __moddi3(quad_t a, quad_t b);
+u_quad_t       __qdivrem(u_quad_t u, u_quad_t v, u_quad_t *rem);
+int            __ucmpdi2(u_quad_t a, u_quad_t b);
+u_quad_t       __udivdi3(u_quad_t a, u_quad_t b);
+u_quad_t       __umoddi3(u_quad_t a, u_quad_t b);
+
+typedef unsigned int   qshift_t;
lib/utils/libquad/udivdi3.c new file +52 -0
new file mode 100644
index 0000000..0e6f27e
@@ -0,0 +1,52 @@
+/*-
+ * SPDX-License-Identifier: BSD-3-Clause
+ *
+ * Copyright (c) 1992, 1993
+ *     The Regents of the University of California.  All rights reserved.
+ *
+ * This software was developed by the Computer Systems Engineering group
+ * at Lawrence Berkeley Laboratory under DARPA contract BG 91-66 and
+ * contributed to Berkeley.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ * 3. Neither the name of the University nor the names of its contributors
+ *    may be used to endorse or promote products derived from this software
+ *    without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#if defined(LIBC_SCCS) && !defined(lint)
+static char sccsid[] = "@(#)udivdi3.c  8.1 (Berkeley) 6/4/93";
+#endif /* LIBC_SCCS and not lint */
+#include <sys/cdefs.h>
+__FBSDID("$FreeBSD$");
+
+#include "quad.h"
+
+/*
+ * Divide two unsigned quads.
+ */
+u_quad_t
+__udivdi3(u_quad_t a, u_quad_t b)
+{
+
+       return (__qdivrem(a, b, (u_quad_t *)0));
+}
lib/utils/libquad/umoddi3.c new file +54 -1
new file mode 100644
index 0000000..7f45134
@@ -0,0 +1,54 @@
+/*-
+ * SPDX-License-Identifier: BSD-3-Clause
+ *
+ * Copyright (c) 1992, 1993
+ *     The Regents of the University of California.  All rights reserved.
+ *
+ * This software was developed by the Computer Systems Engineering group
+ * at Lawrence Berkeley Laboratory under DARPA contract BG 91-66 and
+ * contributed to Berkeley.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ * 3. Neither the name of the University nor the names of its contributors
+ *    may be used to endorse or promote products derived from this software
+ *    without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#if defined(LIBC_SCCS) && !defined(lint)
+static char sccsid[] = "@(#)umoddi3.c  8.1 (Berkeley) 6/4/93";
+#endif /* LIBC_SCCS and not lint */
+#include <sys/cdefs.h>
+__FBSDID("$FreeBSD$");
+
+#include "quad.h"
+
+/*
+ * Return remainder after dividing two unsigned quads.
+ */
+u_quad_t
+__umoddi3(u_quad_t a, u_quad_t b)
+{
+       u_quad_t r;
+
+       (void)__qdivrem(a, b, &r);
+       return (r);
+}
--

2.31.0

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

#25 lore raw

PATCH v5 6/6 Drop dependency on libgcc by importing part of FreeBSD's libquad

Quoted 19 lines · Anup Patel
Quoted 8 lines · Jessica Clarke

We only need libgcc for 64-bit division on RV32. Whilst GCC toolchains bundle libgcc, Clang toolchains tend not to ship libclang_rt.builtins given every compiler is a cross-compiler for every target and so you would need a silly number of builds of it, with only the native library available; only vendor-provided Clang toolchains specifically for bare metal cross-compiling are likely to provide it.

Quoted 4 lines

Thus, import part of FreeBSD's implementation of the division support functions needed and stop linking against libgcc.

I tried this patch with GCC linux toolchain and U-Boot SPL and it worked fine for me. Other please try LLVM and bare-metal toolchain at your end and if you see any issue then please send a fix patch.

Quoted 6 lines

Applied this patch to the riscv/opensbi repo.

@Bin Meng Can you also try RV32 U-Boot SPL ? I did not try that.

It seems RV32 U-Boot SPL is broken, that it does not boot occasionally.

But this has nothing to do with the LLVM changes in this series. It was due to upstream libfdt changes. I have sent a patch on the U-Boot side. Booting S-mode U-Boot directly is not affected.

Regards,
Bin

#26 lore raw

PATCH v5 6/6 Drop dependency on libgcc by importing part of FreeBSD's libquad

Quoted 26 lines · Bin Meng
Quoted 24 lines · Anup Patel
Quoted 8 lines · Jessica Clarke

We only need libgcc for 64-bit division on RV32. Whilst GCC toolchains bundle libgcc, Clang toolchains tend not to ship libclang_rt.builtins given every compiler is a cross-compiler for every target and so you would need a silly number of builds of it, with only the native library available; only vendor-provided Clang toolchains specifically for bare metal cross-compiling are likely to provide it.

Quoted 4 lines

Thus, import part of FreeBSD's implementation of the division support functions needed and stop linking against libgcc.

I tried this patch with GCC linux toolchain and U-Boot SPL and it worked fine for me. Other please try LLVM and bare-metal toolchain at your end and if you see any issue then please send a fix patch.

Applied this patch to the riscv/opensbi repo.

Quoted 7 lines
Quoted 1 line

@Bin Meng Can you also try RV32 U-Boot SPL ? I did not try that.

It seems RV32 U-Boot SPL is broken, that it does not boot occasionally.

But this has nothing to do with the LLVM changes in this series. It was due to upstream libfdt changes. I have sent a patch on the U-Boot side. Booting S-mode U-Boot directly is not affected.

Yes, it's not related to this series. In fact, it is not related to FW_PIC at all.

I did little digging. It seems OpenSBI gets corrupted FDT from U-Boot SPL RV32. The fw_platform_init() get's stuck very early because it is not able to find nodeoffset of "/" path in FDT.

The OpenSBI RV32 firmwares work fine without U-Boot SPL which leads to the conclusion that FDT is corrupted for U-Boot SPL RV32 case.

I hope the above info helps ...

Regards,
Anup

Quoted 3 lines

Regards,
Bin

#27 lore raw

PATCH v5 6/6 Drop dependency on libgcc by importing part of FreeBSD's libquad

Hi Anup,

Quoted 30 lines · Anup Patel
Quoted 28 lines · Bin Meng
Quoted 25 lines · Anup Patel
Quoted 8 lines · Jessica Clarke

We only need libgcc for 64-bit division on RV32. Whilst GCC toolchains bundle libgcc, Clang toolchains tend not to ship libclang_rt.builtins given every compiler is a cross-compiler for every target and so you would need a silly number of builds of it, with only the native library available; only vendor-provided Clang toolchains specifically for bare metal cross-compiling are likely to provide it.

Quoted 4 lines

Thus, import part of FreeBSD's implementation of the division support functions needed and stop linking against libgcc.

I tried this patch with GCC linux toolchain and U-Boot SPL and it worked fine for me. Other please try LLVM and bare-metal toolchain at your end and if you see any issue then please send a fix patch.

Applied this patch to the riscv/opensbi repo.

@Bin Meng Can you also try RV32 U-Boot SPL ? I did not try that.

Quoted 8 lines
Quoted 5 lines

It seems RV32 U-Boot SPL is broken, that it does not boot occasionally.

But this has nothing to do with the LLVM changes in this series. It was due to upstream libfdt changes. I have sent a patch on the U-Boot side. Booting S-mode U-Boot directly is not affected.

Yes, it's not related to this series. In fact, it is not related to FW_PIC at all.

Correct.

Quoted 3 lines

I did little digging. It seems OpenSBI gets corrupted FDT from U-Boot SPL RV32. The fw_platform_init() get's stuck very early because it is not able to find nodeoffset of "/" path in FDT.

Yeah, that's because fdt_ro_probe_() returns -FDT_ERR_ALIGNMENT hence fw_platform_init() gets stuck.

Quoted 4 lines

The OpenSBI RV32 firmwares work fine without U-Boot SPL which leads to the conclusion that FDT is corrupted for U-Boot SPL RV32 case.

I hope the above info helps ...

I already sent a patch to U-Boot ML, see:
http://patchwork.ozlabs.org/project/uboot/patch/20210712035231.26475-1-bmeng.cn at gmail.com/

You may have a test on your side with this patch.

Regards,
Bin

#28 lore raw

PATCH v5 6/6 Drop dependency on libgcc by importing part of FreeBSD's libquad

Quoted 43 lines · Bin Meng

Hi Anup,

Quoted 30 lines · Anup Patel
Quoted 28 lines · Bin Meng
Quoted 25 lines · Anup Patel
Quoted 8 lines · Jessica Clarke

We only need libgcc for 64-bit division on RV32. Whilst GCC toolchains bundle libgcc, Clang toolchains tend not to ship libclang_rt.builtins given every compiler is a cross-compiler for every target and so you would need a silly number of builds of it, with only the native library available; only vendor-provided Clang toolchains specifically for bare metal cross-compiling are likely to provide it.

Quoted 4 lines

Thus, import part of FreeBSD's implementation of the division support functions needed and stop linking against libgcc.

I tried this patch with GCC linux toolchain and U-Boot SPL and it worked fine for me. Other please try LLVM and bare-metal toolchain at your end and if you see any issue then please send a fix patch.

Applied this patch to the riscv/opensbi repo.

@Bin Meng Can you also try RV32 U-Boot SPL ? I did not try that.

Quoted 8 lines
Quoted 5 lines

It seems RV32 U-Boot SPL is broken, that it does not boot occasionally.

But this has nothing to do with the LLVM changes in this series. It was due to upstream libfdt changes. I have sent a patch on the U-Boot side. Booting S-mode U-Boot directly is not affected.

Yes, it's not related to this series. In fact, it is not related to FW_PIC at all.

Quoted 8 lines

Correct.

Quoted 3 lines

I did little digging. It seems OpenSBI gets corrupted FDT from U-Boot SPL RV32. The fw_platform_init() get's stuck very early because it is not able to find nodeoffset of "/" path in FDT.

Yeah, that's because fdt_ro_probe_() returns -FDT_ERR_ALIGNMENT hence fw_platform_init() gets stuck.

Ahh, it seems your patch is not part of U-Boot-2021.07

Regards,
Anup

Quoted 13 lines
Quoted 5 lines

The OpenSBI RV32 firmwares work fine without U-Boot SPL which leads to the conclusion that FDT is corrupted for U-Boot SPL RV32 case.

I hope the above info helps ...

I already sent a patch to U-Boot ML, see:
http://patchwork.ozlabs.org/project/uboot/patch/20210712035231.26475-1-bmeng.cn at gmail.com/

You may have a test on your side with this patch.

Regards,
Bin

#29 lore raw

PATCH v5 6/6 Drop dependency on libgcc by importing part of FreeBSD's libquad

Quoted 47 lines · Anup Patel
Quoted 45 lines · Bin Meng

Hi Anup,

Quoted 30 lines · Anup Patel
Quoted 28 lines · Bin Meng
Quoted 25 lines · Anup Patel
Quoted 8 lines · Jessica Clarke

We only need libgcc for 64-bit division on RV32. Whilst GCC toolchains bundle libgcc, Clang toolchains tend not to ship libclang_rt.builtins given every compiler is a cross-compiler for every target and so you would need a silly number of builds of it, with only the native library available; only vendor-provided Clang toolchains specifically for bare metal cross-compiling are likely to provide it.

Quoted 4 lines

Thus, import part of FreeBSD's implementation of the division support functions needed and stop linking against libgcc.

I tried this patch with GCC linux toolchain and U-Boot SPL and it worked fine for me. Other please try LLVM and bare-metal toolchain at your end and if you see any issue then please send a fix patch.

Applied this patch to the riscv/opensbi repo.

@Bin Meng Can you also try RV32 U-Boot SPL ? I did not try that.

Quoted 8 lines
Quoted 5 lines

It seems RV32 U-Boot SPL is broken, that it does not boot occasionally.

But this has nothing to do with the LLVM changes in this series. It was due to upstream libfdt changes. I have sent a patch on the U-Boot side. Booting S-mode U-Boot directly is not affected.

Yes, it's not related to this series. In fact, it is not related to FW_PIC at all.

Correct.

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

I did little digging. It seems OpenSBI gets corrupted FDT from U-Boot SPL RV32. The fw_platform_init() get's stuck very early because it is not able to find nodeoffset of "/" path in FDT.

Yeah, that's because fdt_ro_probe_() returns -FDT_ERR_ALIGNMENT hence fw_platform_init() gets stuck.

Ahh, it seems your patch is not part of U-Boot-2021.07

Unfortunately we missed the 2021.07 train :(

Regards,
Bin

#30 lore raw

v5 cover Fully support standalone Clang/LLVM toolchains

In reply to Jessica Clarke
Quoted 53 lines · Jessica Clarke

This patch series is comprised of six parts.

The first patch fixes a warning seen when building with LLVM due to a bogus combination of assembly directives.

The second patch fixes errors seen when trying to build OpenSBI as a position-independent binary using LLD that may or may not be an LLD bug (the exact meaning of -N/--omagic isn't clear) but can easily be worked around without any issue either way.

The third patch bypasses the Clang driver's helpful nature of not honouring -pie for bare-metal binaries as it's normally not something you want (arguably that should be an error though, or passed on, since just giving a -Wunused-command-line-argument warning can get lost, and may be disabled).

The fourth patch fixes the FW_PIC support added recently to be disabled by default when unsupported, since bare-metal GNU ld does not support PIE and so currently building with a bare-metal GNU ld will fail unless FW_PIC is manually disabled. Disabling FW_PIC is suboptimal but restores the pre-FW_PIC behaviour so is no longer a regression.

The fifth patch adds build system support for using Clang and LLVM binutils, provided your Clang is able to locate a libgcc.a in its search path.

However, pure LLVM toolchains do not use libgcc, they use compiler-rt (libclang_rt.builtins.<arch>.a). We could change the Makefile to not hard-code -lgcc and instead use -print-libgcc-file-name, but that still requires a bare metal compiler-rt built for the right -march/-mabi to be present, which is often not the case. Moreover, we need very little from libgcc/compiler-rt; RV64 needs nothing, and RV32 only needs 64-bit division. Thus, the sixth patch vendors part of FreeBSD's libquad and stops linking against libgcc entirely, allowing OpenSBI to be built with just a cross-compiler. This means that building with any distro-provided LLVM just works, as does compiling with the system Clang compiler and LLD linker on FreeBSD without any external packages needed (beyond GNU make).

Note that using Clang with the current bleeding-edge riscv64 glibc toolchain from Bootlin's linker is currently broken; around 60 relocations, or about a third of the number expected, go missing, with a big gap in the relocations emitted, causing various global data structures to not be correctly initialised. In particular, root's possible_harts and regions fields lose their relocations during linking so are left as NULL, causing sanitize_domain to immediately fail with SBI_EINVAL. The relocations exist in the object files and look completely sane so this appears to be a GNU ld bug, but I have not narrowed it down enough to be sure it's not something weird that OpenSBI or Clang are doing. I do not think this should be a blocker, and have added a note about this currently-broken combination to the README.

Quoted 7 lines

Changes in v5:
* Disable FW_PIC by default when unsupported
* Pulled out -mno-relax and -fuse-ld flags into variables for reusing in the code for the above change, and to deduplicate the existing -mno-relax code
* Note in the README that riscv64-linux-gnu-ld + Clang is currently broken due to what appears to be a GNU ld bug

There is still one build error in the following environment:

If /opt/riscv/bin/riscv64-unknown-linux-gnu- is not in my $PATH, the following command fails:

$ make CC=clang
CROSS_COMPILE=/opt/riscv/bin/riscv64-unknown-linux-gnu-
PLATFORM=generic V=1 2>&1

mkdir -p dirname /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf;
echo " ELF platform/generic/firmware/payloads/test.elf"; clang
-g -Wall -Werror -ffreestanding -nostdlib -fno-stack-protector -fno-strict-aliasing -O2 -fno-omit-frame-pointer
-fno-optimize-sibling-calls -mno-save-restore -mstrict-align
-mabi=lp64 -march=rv64imafdc -mcmodel=medany
--target=riscv64-unknown-linux-gnu -Wno-unused-command-line-argument -I/home/test/git/opensbi/platform/generic/include -I/home/test/git/opensbi/include
-DOPENSBI_VERSION_GIT="\"v0.9-111-g401461d\"" -I/home/test/git/opensbi/lib/utils/libfdt/
-DFW_TEXT_START=0x80000000 -DFW_JUMP_ADDR=0x80200000 -DFW_JUMP_FDT_ADDR=0x82200000 -DFW_PAYLOAD_PATH=\"/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.bin\" -DFW_PAYLOAD_OFFSET=0x200000 -DFW_PAYLOAD_FDT_ADDR=0x82200000
-fno-pie -no-pie /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.o /home/test/git/opensbi/build/platform/generic/lib/libplatsbi.a
-fuse-ld=bfd -Wl,--build-id=none -Wl,-N -Wl,-T/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf.ld -o /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf
ELF platform/generic/firmware/payloads/test.elf
/usr/bin/ld.bfd: unrecognised emulation mode: elf64lriscv
Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu
elf_l1om elf_k1om i386pep i386pe
clang-12: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Makefile:406:
/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf]
Error 1

Regards,
Bin

#31 lore raw

v5 cover Fully support standalone Clang/LLVM toolchains

Quoted 95 lines · Bin Meng
Quoted 53 lines · Jessica Clarke

This patch series is comprised of six parts.

The first patch fixes a warning seen when building with LLVM due to a bogus combination of assembly directives.

The second patch fixes errors seen when trying to build OpenSBI as a position-independent binary using LLD that may or may not be an LLD bug (the exact meaning of -N/--omagic isn't clear) but can easily be worked around without any issue either way.

The third patch bypasses the Clang driver's helpful nature of not honouring -pie for bare-metal binaries as it's normally not something you want (arguably that should be an error though, or passed on, since just giving a -Wunused-command-line-argument warning can get lost, and may be disabled).

The fourth patch fixes the FW_PIC support added recently to be disabled by default when unsupported, since bare-metal GNU ld does not support PIE and so currently building with a bare-metal GNU ld will fail unless FW_PIC is manually disabled. Disabling FW_PIC is suboptimal but restores the pre-FW_PIC behaviour so is no longer a regression.

The fifth patch adds build system support for using Clang and LLVM binutils, provided your Clang is able to locate a libgcc.a in its search path.

However, pure LLVM toolchains do not use libgcc, they use compiler-rt (libclang_rt.builtins.<arch>.a). We could change the Makefile to not hard-code -lgcc and instead use -print-libgcc-file-name, but that still requires a bare metal compiler-rt built for the right -march/-mabi to be present, which is often not the case. Moreover, we need very little from libgcc/compiler-rt; RV64 needs nothing, and RV32 only needs 64-bit division. Thus, the sixth patch vendors part of FreeBSD's libquad and stops linking against libgcc entirely, allowing OpenSBI to be built with just a cross-compiler. This means that building with any distro-provided LLVM just works, as does compiling with the system Clang compiler and LLD linker on FreeBSD without any external packages needed (beyond GNU make).

Note that using Clang with the current bleeding-edge riscv64 glibc toolchain from Bootlin's linker is currently broken; around 60 relocations, or about a third of the number expected, go missing, with a big gap in the relocations emitted, causing various global data structures to not be correctly initialised. In particular, root's possible_harts and regions fields lose their relocations during linking so are left as NULL, causing sanitize_domain to immediately fail with SBI_EINVAL. The relocations exist in the object files and look completely sane so this appears to be a GNU ld bug, but I have not narrowed it down enough to be sure it's not something weird that OpenSBI or Clang are doing. I do not think this should be a blocker, and have added a note about this currently-broken combination to the README.

Quoted 7 lines

Changes in v5:
* Disable FW_PIC by default when unsupported
* Pulled out -mno-relax and -fuse-ld flags into variables for reusing in the code for the above change, and to deduplicate the existing -mno-relax code
* Note in the README that riscv64-linux-gnu-ld + Clang is currently broken due to what appears to be a GNU ld bug

There is still one build error in the following environment:

If /opt/riscv/bin/riscv64-unknown-linux-gnu- is not in my $PATH, the following command fails:

$ make CC=clang
CROSS_COMPILE=/opt/riscv/bin/riscv64-unknown-linux-gnu-
PLATFORM=generic V=1 2>&1

mkdir -p dirname /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf;
echo " ELF platform/generic/firmware/payloads/test.elf"; clang

(quoted diff) +0 -14
-g -Wall -Werror -ffreestanding -nostdlib -fno-stack-protector
-fno-strict-aliasing -O2 -fno-omit-frame-pointer
-fno-optimize-sibling-calls -mno-save-restore -mstrict-align
-mabi=lp64 -march=rv64imafdc -mcmodel=medany
--target=riscv64-unknown-linux-gnu -Wno-unused-command-line-argument
-I/home/test/git/opensbi/platform/generic/include
-I/home/test/git/opensbi/include
-DOPENSBI_VERSION_GIT="\"v0.9-111-g401461d\""
-I/home/test/git/opensbi/lib/utils/libfdt/
-DFW_TEXT_START=0x80000000 -DFW_JUMP_ADDR=0x80200000
-DFW_JUMP_FDT_ADDR=0x82200000
-DFW_PAYLOAD_PATH=\"/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.bin\"
-DFW_PAYLOAD_OFFSET=0x200000 -DFW_PAYLOAD_FDT_ADDR=0x82200000
-fno-pie -no-pie

/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.o /home/test/git/opensbi/build/platform/generic/lib/libplatsbi.a

(quoted diff) +0 -3
-fuse-ld=bfd -Wl,--build-id=none -Wl,-N
-Wl,-T/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf.ld
-o /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf

ELF platform/generic/firmware/payloads/test.elf

Quoted 8 lines

/usr/bin/ld.bfd: unrecognised emulation mode: elf64lriscv Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu elf_l1om elf_k1om i386pep i386pe
clang-12: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Makefile:406:
/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf] Error 1

The same problem exists if you do:

make CC=gcc CROSS_COMPILE=/opt/riscv/bin/riscv64-unknown-linux-gnu-

today (CC=gcc can be anything so long as it?s not ?next to? the linker). Do you also want:

make LD=/path/to/ld/not/in/path

to work? Because that doesn?t get used for the ELF rules with GCC today, and the same will be true of Clang too. For the former, -B$(abspath $(dir CROSS_COMPILE)) if it has a / in it is probably sufficient. For the latter, there is no way to support that with GCC (without hacking up a temporary directory with a symlink that you can use with -B). It can be supported with Clang, though it requires version parsing because Clang 12 changed how it does it (it deprecates reusing -fuse-ld for this and adds a new --ld-path specifically for specifying a path to the linker). Since these are not regressions, nor problems only faced by Clang (though it?s more likely you might hit them once you have two different toolchain families you?re mixing), I would prefer to see the current patch series land so that users and distributions can start building with LLVM today as it supports most of the common uses without (as far as we know) introducing any regressions for purely GNU toolchains, and then support for any additional more niche use cases can be added as follow-up patches, otherwise this patch series is just going to keep growing and growing with feature creep.

Jess

#32 lore raw

v5 cover Fully support standalone Clang/LLVM toolchains

Quoted 124 lines · Jessica Clarke
Quoted 95 lines · Bin Meng
Quoted 53 lines · Jessica Clarke

This patch series is comprised of six parts.

The first patch fixes a warning seen when building with LLVM due to a bogus combination of assembly directives.

The second patch fixes errors seen when trying to build OpenSBI as a position-independent binary using LLD that may or may not be an LLD bug (the exact meaning of -N/--omagic isn't clear) but can easily be worked around without any issue either way.

The third patch bypasses the Clang driver's helpful nature of not honouring -pie for bare-metal binaries as it's normally not something you want (arguably that should be an error though, or passed on, since just giving a -Wunused-command-line-argument warning can get lost, and may be disabled).

The fourth patch fixes the FW_PIC support added recently to be disabled by default when unsupported, since bare-metal GNU ld does not support PIE and so currently building with a bare-metal GNU ld will fail unless FW_PIC is manually disabled. Disabling FW_PIC is suboptimal but restores the pre-FW_PIC behaviour so is no longer a regression.

The fifth patch adds build system support for using Clang and LLVM binutils, provided your Clang is able to locate a libgcc.a in its search path.

However, pure LLVM toolchains do not use libgcc, they use compiler-rt (libclang_rt.builtins.<arch>.a). We could change the Makefile to not hard-code -lgcc and instead use -print-libgcc-file-name, but that still requires a bare metal compiler-rt built for the right -march/-mabi to be present, which is often not the case. Moreover, we need very little from libgcc/compiler-rt; RV64 needs nothing, and RV32 only needs 64-bit division. Thus, the sixth patch vendors part of FreeBSD's libquad and stops linking against libgcc entirely, allowing OpenSBI to be built with just a cross-compiler. This means that building with any distro-provided LLVM just works, as does compiling with the system Clang compiler and LLD linker on FreeBSD without any external packages needed (beyond GNU make).

Note that using Clang with the current bleeding-edge riscv64 glibc toolchain from Bootlin's linker is currently broken; around 60 relocations, or about a third of the number expected, go missing, with a big gap in the relocations emitted, causing various global data structures to not be correctly initialised. In particular, root's possible_harts and regions fields lose their relocations during linking so are left as NULL, causing sanitize_domain to immediately fail with SBI_EINVAL. The relocations exist in the object files and look completely sane so this appears to be a GNU ld bug, but I have not narrowed it down enough to be sure it's not something weird that OpenSBI or Clang are doing. I do not think this should be a blocker, and have added a note about this currently-broken combination to the README.

Quoted 7 lines

Changes in v5:
* Disable FW_PIC by default when unsupported
* Pulled out -mno-relax and -fuse-ld flags into variables for reusing in the code for the above change, and to deduplicate the existing -mno-relax code
* Note in the README that riscv64-linux-gnu-ld + Clang is currently broken due to what appears to be a GNU ld bug

There is still one build error in the following environment:

If /opt/riscv/bin/riscv64-unknown-linux-gnu- is not in my $PATH, the following command fails:

$ make CC=clang
CROSS_COMPILE=/opt/riscv/bin/riscv64-unknown-linux-gnu-
PLATFORM=generic V=1 2>&1

mkdir -p dirname /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf;
echo " ELF platform/generic/firmware/payloads/test.elf"; clang

(quoted diff) +0 -14
-g -Wall -Werror -ffreestanding -nostdlib -fno-stack-protector
-fno-strict-aliasing -O2 -fno-omit-frame-pointer
-fno-optimize-sibling-calls -mno-save-restore -mstrict-align
-mabi=lp64 -march=rv64imafdc -mcmodel=medany
--target=riscv64-unknown-linux-gnu -Wno-unused-command-line-argument
-I/home/test/git/opensbi/platform/generic/include
-I/home/test/git/opensbi/include
-DOPENSBI_VERSION_GIT="\"v0.9-111-g401461d\""
-I/home/test/git/opensbi/lib/utils/libfdt/
-DFW_TEXT_START=0x80000000 -DFW_JUMP_ADDR=0x80200000
-DFW_JUMP_FDT_ADDR=0x82200000
-DFW_PAYLOAD_PATH=\"/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.bin\"
-DFW_PAYLOAD_OFFSET=0x200000 -DFW_PAYLOAD_FDT_ADDR=0x82200000
-fno-pie -no-pie

/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.o /home/test/git/opensbi/build/platform/generic/lib/libplatsbi.a

(quoted diff) +0 -3
-fuse-ld=bfd -Wl,--build-id=none -Wl,-N
-Wl,-T/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf.ld
-o /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf

ELF platform/generic/firmware/payloads/test.elf

Quoted 8 lines

/usr/bin/ld.bfd: unrecognised emulation mode: elf64lriscv Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu elf_l1om elf_k1om i386pep i386pe
clang-12: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Makefile:406:
/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf] Error 1

The same problem exists if you do:

make CC=gcc CROSS_COMPILE=/opt/riscv/bin/riscv64-unknown-linux-gnu-

today (CC=gcc can be anything so long as it?s not ?next to? the linker). Do you also want:

make LD=/path/to/ld/not/in/path

to work? Because that doesn?t get used for the ELF rules with GCC today, and the same will be true of Clang too. For the former, -B$(abspath $(dir CROSS_COMPILE)) if it has a / in it is probably sufficient. For the latter, there is no way to support that with GCC (without hacking up a temporary directory with a symlink that you can use with -B). It can be supported with Clang, though it requires version parsing because Clang 12 changed how it does it (it deprecates reusing -fuse-ld for this and adds a new --ld-path specifically for specifying a path to the linker). Since these are not regressions, nor

Quoted 8 lines

problems only faced by Clang (though it?s more likely you might hit them once you have two different toolchain families you?re mixing), I would prefer to see the current patch series land so that users and distributions can start building with LLVM today as it supports most of the common uses without (as far as we know) introducing any regressions for purely GNU toolchains, and then support for any additional more niche use cases can be added as follow-up patches, otherwise this patch series is just going to keep growing and growing with feature creep.

I am just here for beta testing, so relax :) I did not meant to block the patch series. For this version, I felt it is in nice shape now.

Again, some additional use case as documented in patch 5. For this one both CC and LLVM are in my $PATH:

$ make CC=riscv64-unknown-elf-gcc LLVM=1 PLATFORM=generic V=1

mkdir -p dirname /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf;
echo " ELF platform/generic/firmware/payloads/test.elf";
riscv64-unknown-elf-gcc -g -Wall -Werror -ffreestanding -nostdlib -fno-stack-protector -fno-strict-aliasing -O2 -fno-omit-frame-pointer -fno-optimize-sibling-calls -mno-save-restore -mstrict-align
-mabi=lp64 -march=rv64imafdc -mcmodel=medany -mno-relax -I/home/test/git/opensbi/platform/generic/include -I/home/test/git/opensbi/include
-DOPENSBI_VERSION_GIT="\"v0.9-111-g401461d\"" -I/home/test/git/opensbi/lib/utils/libfdt/
-DFW_TEXT_START=0x80000000 -DFW_JUMP_ADDR=0x80200000 -DFW_JUMP_FDT_ADDR=0x82200000 -DFW_PAYLOAD_PATH=\"/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.bin\" -DFW_PAYLOAD_OFFSET=0x200000 -DFW_PAYLOAD_FDT_ADDR=0x82200000
-fno-pie -no-pie /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.o /home/test/git/opensbi/build/platform/generic/lib/libplatsbi.a
-fuse-ld=lld -Wl,--build-id=none -Wl,-N -Wl,-T/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf.ld -o /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf
ELF platform/generic/firmware/payloads/test.elf
collect2: fatal error: cannot find 'ld'
compilation terminated.
make: *** [Makefile:406:
/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf]
Error 1

Regards,
Bin

#33 lore raw

v5 cover Fully support standalone Clang/LLVM toolchains

Quoted 135 lines · Bin Meng
Quoted 124 lines · Jessica Clarke
Quoted 95 lines · Bin Meng
Quoted 53 lines · Jessica Clarke

This patch series is comprised of six parts.

The first patch fixes a warning seen when building with LLVM due to a bogus combination of assembly directives.

The second patch fixes errors seen when trying to build OpenSBI as a position-independent binary using LLD that may or may not be an LLD bug (the exact meaning of -N/--omagic isn't clear) but can easily be worked around without any issue either way.

The third patch bypasses the Clang driver's helpful nature of not honouring -pie for bare-metal binaries as it's normally not something you want (arguably that should be an error though, or passed on, since just giving a -Wunused-command-line-argument warning can get lost, and may be disabled).

The fourth patch fixes the FW_PIC support added recently to be disabled by default when unsupported, since bare-metal GNU ld does not support PIE and so currently building with a bare-metal GNU ld will fail unless FW_PIC is manually disabled. Disabling FW_PIC is suboptimal but restores the pre-FW_PIC behaviour so is no longer a regression.

The fifth patch adds build system support for using Clang and LLVM binutils, provided your Clang is able to locate a libgcc.a in its search path.

However, pure LLVM toolchains do not use libgcc, they use compiler-rt (libclang_rt.builtins.<arch>.a). We could change the Makefile to not hard-code -lgcc and instead use -print-libgcc-file-name, but that still requires a bare metal compiler-rt built for the right -march/-mabi to be present, which is often not the case. Moreover, we need very little from libgcc/compiler-rt; RV64 needs nothing, and RV32 only needs 64-bit division. Thus, the sixth patch vendors part of FreeBSD's libquad and stops linking against libgcc entirely, allowing OpenSBI to be built with just a cross-compiler. This means that building with any distro-provided LLVM just works, as does compiling with the system Clang compiler and LLD linker on FreeBSD without any external packages needed (beyond GNU make).

Note that using Clang with the current bleeding-edge riscv64 glibc toolchain from Bootlin's linker is currently broken; around 60 relocations, or about a third of the number expected, go missing, with a big gap in the relocations emitted, causing various global data structures to not be correctly initialised. In particular, root's possible_harts and regions fields lose their relocations during linking so are left as NULL, causing sanitize_domain to immediately fail with SBI_EINVAL. The relocations exist in the object files and look completely sane so this appears to be a GNU ld bug, but I have not narrowed it down enough to be sure it's not something weird that OpenSBI or Clang are doing. I do not think this should be a blocker, and have added a note about this currently-broken combination to the README.

Quoted 7 lines

Changes in v5:
* Disable FW_PIC by default when unsupported
* Pulled out -mno-relax and -fuse-ld flags into variables for reusing in the code for the above change, and to deduplicate the existing -mno-relax code
* Note in the README that riscv64-linux-gnu-ld + Clang is currently broken due to what appears to be a GNU ld bug

There is still one build error in the following environment:

If /opt/riscv/bin/riscv64-unknown-linux-gnu- is not in my $PATH, the following command fails:

$ make CC=clang
CROSS_COMPILE=/opt/riscv/bin/riscv64-unknown-linux-gnu-
PLATFORM=generic V=1 2>&1

mkdir -p dirname /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf;
echo " ELF platform/generic/firmware/payloads/test.elf"; clang

(quoted diff) +0 -14
-g -Wall -Werror -ffreestanding -nostdlib -fno-stack-protector
-fno-strict-aliasing -O2 -fno-omit-frame-pointer
-fno-optimize-sibling-calls -mno-save-restore -mstrict-align
-mabi=lp64 -march=rv64imafdc -mcmodel=medany
--target=riscv64-unknown-linux-gnu -Wno-unused-command-line-argument
-I/home/test/git/opensbi/platform/generic/include
-I/home/test/git/opensbi/include
-DOPENSBI_VERSION_GIT="\"v0.9-111-g401461d\""
-I/home/test/git/opensbi/lib/utils/libfdt/
-DFW_TEXT_START=0x80000000 -DFW_JUMP_ADDR=0x80200000
-DFW_JUMP_FDT_ADDR=0x82200000
-DFW_PAYLOAD_PATH=\"/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.bin\"
-DFW_PAYLOAD_OFFSET=0x200000 -DFW_PAYLOAD_FDT_ADDR=0x82200000
-fno-pie -no-pie

/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.o /home/test/git/opensbi/build/platform/generic/lib/libplatsbi.a

(quoted diff) +0 -3
-fuse-ld=bfd -Wl,--build-id=none -Wl,-N
-Wl,-T/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf.ld
-o /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf

ELF platform/generic/firmware/payloads/test.elf

Quoted 8 lines

/usr/bin/ld.bfd: unrecognised emulation mode: elf64lriscv Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu elf_l1om elf_k1om i386pep i386pe
clang-12: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Makefile:406:
/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf] Error 1

The same problem exists if you do:

make CC=gcc CROSS_COMPILE=/opt/riscv/bin/riscv64-unknown-linux-gnu-

today (CC=gcc can be anything so long as it?s not ?next to? the linker). Do you also want:

make LD=/path/to/ld/not/in/path

to work? Because that doesn?t get used for the ELF rules with GCC today, and the same will be true of Clang too. For the former, -B$(abspath $(dir CROSS_COMPILE)) if it has a / in it is probably sufficient. For the latter, there is no way to support that with GCC (without hacking up a temporary directory with a symlink that you can use with -B). It can be supported with Clang, though it requires version parsing because Clang 12 changed how it does it (it deprecates reusing -fuse-ld for this and adds a new --ld-path specifically for specifying a path to the linker). Since these are not regressions, nor

Quoted 8 lines

problems only faced by Clang (though it?s more likely you might hit them once you have two different toolchain families you?re mixing), I would prefer to see the current patch series land so that users and distributions can start building with LLVM today as it supports most of the common uses without (as far as we know) introducing any regressions for purely GNU toolchains, and then support for any additional more niche use cases can be added as follow-up patches, otherwise this patch series is just going to keep growing and growing with feature creep.

Quoted 2 lines

I am just here for beta testing, so relax :) I did not meant to block the patch series. For this version, I felt it is in nice shape now.

Ah, ok, it can be hard to tell blocking from non-blocking issue reports :)

Quoted 26 lines

Again, some additional use case as documented in patch 5. For this one both CC and LLVM are in my $PATH:

$ make CC=riscv64-unknown-elf-gcc LLVM=1 PLATFORM=generic V=1

mkdir -p dirname /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf;
echo " ELF platform/generic/firmware/payloads/test.elf";
riscv64-unknown-elf-gcc -g -Wall -Werror -ffreestanding -nostdlib

(quoted diff) +0 -12
-fno-stack-protector -fno-strict-aliasing -O2 -fno-omit-frame-pointer
-fno-optimize-sibling-calls -mno-save-restore -mstrict-align
-mabi=lp64 -march=rv64imafdc -mcmodel=medany -mno-relax
-I/home/test/git/opensbi/platform/generic/include
-I/home/test/git/opensbi/include
-DOPENSBI_VERSION_GIT="\"v0.9-111-g401461d\""
-I/home/test/git/opensbi/lib/utils/libfdt/
-DFW_TEXT_START=0x80000000 -DFW_JUMP_ADDR=0x80200000
-DFW_JUMP_FDT_ADDR=0x82200000
-DFW_PAYLOAD_PATH=\"/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.bin\"
-DFW_PAYLOAD_OFFSET=0x200000 -DFW_PAYLOAD_FDT_ADDR=0x82200000
-fno-pie -no-pie

/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.o /home/test/git/opensbi/build/platform/generic/lib/libplatsbi.a

(quoted diff) +0 -3
-fuse-ld=lld -Wl,--build-id=none -Wl,-N
-Wl,-T/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf.ld
-o /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf
Quoted 6 lines

ELF platform/generic/firmware/payloads/test.elf
collect2: fatal error: cannot find 'ld'
compilation terminated.
make: *** [Makefile:406:
/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf] Error 1

Yeah, this is GCC (specifically, collect2 itself) being extremely inflexible. I ran into something like this several years ago and the ?solution? I came up with was to ask an admin on the system to set up a symlink in the same directory as GCC to point to an LLD in my home directory. I don?t recall the details to know if it?s the exact same problem and whether it still exists today. You could try making sure you have a riscv64-unknown-elf-ld.lld on your path, I think collect2 only searches for triple-prefixed files when cross-compiling. Ultimately I don?t think there?s much else you can do beyond say ?yeah, GNU tools have an overly-opinionated view of the world that doesn?t always make sense with more modular toolchains like LLVM?, if you want to mix and match sometimes you have to introduce ugly workarounds to make an LLVM toolchain look enough like a GNU toolchain.

Jess