Skip to contentSkip to conversation

llvm for mesa3d

PATCH New Valentin Korenblit 3 participants 10 messages Feb 16, 2018 – Feb 19, 2018

lore
Share & tools
lore
#0 lore raw

cover llvm for mesa3d

Hello all,

This series provides LLVM support for Mesa3d, enabling llvmpipe software rasterizer.

It must be considered that it does not provide a full LLVM installation for the host, only the necessary components to cross-compile LLVM for the target. It will install all target components as libLLVM.so.

Regards,

Valent?n

Romain Naour (4):
package/llvm: new host package
package/llvm: enable target variant
package/llvm: add AMDGPU support
package/mesa3d: enable llvm support

 package/Config.in           |   1 +
 package/Config.in.host      |   1 +
 package/llvm/Config.in      |  38 +++++
 package/llvm/Config.in.host |   8 +
 package/llvm/llvm.hash      |   2 +
 package/llvm/llvm.mk        | 356 ++++++++++++++++++++++++++++++++++++++++++++
 package/mesa3d/Config.in    |   8 +
 package/mesa3d/mesa3d.mk    |  18 ++-
 8 files changed, 429 insertions(+), 3 deletions(-)
 create mode 100644 package/llvm/Config.in
 create mode 100644 package/llvm/Config.in.host
 create mode 100644 package/llvm/llvm.hash
 create mode 100644 package/llvm/llvm.mk
Signature · 1 line
2.14.3
#1 lore raw

PATCH 1/4 package/llvm: new host package

From: Romain Naour <[email protected]>

This patch installs only the necessary components of LLVM for the host that are needed to cross-compile LLVM for the target. The path to host's llvm-tblgen must be specified when cross-compiling using option -DLLVM_TABLEGEN.

Most build options are set to OFF in order to reduce the build time.

---
 package/Config.in.host      |   1 +
 package/llvm/Config.in.host |  29 +++++
 package/llvm/llvm.hash      |   2 +
 package/llvm/llvm.mk        | 284 ++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 316 insertions(+)
 create mode 100644 package/llvm/Config.in.host
 create mode 100644 package/llvm/llvm.hash
 create mode 100644 package/llvm/llvm.mk
package/Config.in.host +1 -0
index 199a8e9856..9a6227174b 100644
@@ -27,6 +27,7 @@ menu "Host utilities"
 	source "package/imx-usb-loader/Config.in.host"
 	source "package/jq/Config.in.host"
 	source "package/jsmin/Config.in.host"
+	source "package/llvm/Config.in.host"
 	source "package/lpc3250loader/Config.in.host"
 	source "package/lttng-babeltrace/Config.in.host"
 	source "package/mfgtools/Config.in.host"
package/llvm/Config.in.host new file +29 -0
new file mode 100644
index 0000000000..454cf10947
@@ -0,0 +1,29 @@
+config BR2_PACKAGE_LLVM_ARCH_SUPPORTS
+	bool
+	# http://llvm.org/docs/GettingStarted.html#host-c-toolchain-both-compiler-and-standard-library
+	# http://llvm.org/docs/GettingStarted.html#local-llvm-configuration
+	depends on BR2_HOST_GCC_AT_LEAST_4_8
+	default y if BR2_arm || BR2_armeb
+	default y if BR2_aarch64
+	default y if BR2_i386 || BR2_x86_64
+	default y if BR2_mips || BR2_mipsel || BR2_mips64 || BR2_mips64el
+	default y if BR2_powerpc || BR2_powerpc64 || BR2_powerpc64le
+	default y if BR2_sparc
+
+config BR2_PACKAGE_LLVM_TARGET_ARCH
+	string
+	default "ARM" if BR2_arm || BR2_armeb
+	default "AArch64" if BR2_aarch64
+	default "X86" if BR2_i386 || BR2_x86_64
+	default "Mips" if BR2_mips || BR2_mipsel || BR2_mips64 || BR2_mips64el
+	default "PowerPC" if BR2_powerpc || BR2_powerpc64 || BR2_powerpc64le
+	default "Sparc" if BR2_sparc
+
+config BR2_PACKAGE_HOST_LLVM
+	bool "host llvm"
+	depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
+	help
+	  The LLVM Project is a collection of modular and reusable
+	  compiler and toolchain technologies.
+
+	  http://llvm.org
package/llvm/llvm.hash new file +2 -0
new file mode 100644
index 0000000000..2d38445673
@@ -0,0 +1,2 @@
+# locally calculated
+sha256	5fa7489fc0225b11821cab0362f5813a05f2bcf2533e8a4ea9c9c860168807b0	llvm-5.0.1.src.tar.xz
package/llvm/llvm.mk new file +284 -0

284 lines not on this page · raw message

Signature · 1 line
2.14.3
#2 lore raw

PATCH 2/4 package/llvm: enable target variant

In reply to Valentin Korenblit

From: Romain Naour <[email protected]>

This patch installs LLVM libraries for the target (libLLVM*).

Option -DLLVM_BUILD_LLVM_DYLIB=ON allows having all LLVM components in a single shared library. This is used by mesa3d to enable llvmpipe software rasterizer.

Move BR2_PACKAGE_LLVM_ARCH_SUPPORTS and BR2_PACKAGE_LLVM_TARGET_ARCH to package/llvm/Config.in for consistency with other packages.

Define target triple with GNU_TARGET_NAME.

---
 package/Config.in           |  1 +
 package/llvm/Config.in      | 29 ++++++++++++++
 package/llvm/Config.in.host | 21 ----------
 package/llvm/llvm.mk        | 94 +++++++++++++++++++++++++++++++++++++++------
 4 files changed, 113 insertions(+), 32 deletions(-)
 create mode 100644 package/llvm/Config.in
package/Config.in +1 -0
index 9cd5d9c087..e570662985 100644
@@ -1520,6 +1520,7 @@ menu "Other"
 	source "package/libuv/Config.in"
 	source "package/lightning/Config.in"
 	source "package/linux-pam/Config.in"
+	source "package/llvm/Config.in"
 if BR2_PACKAGE_LINUX_PAM
 comment "linux-pam plugins"
 	source "package/libpam-radius-auth/Config.in"
package/llvm/Config.in new file +29 -0
new file mode 100644
index 0000000000..84dde8d5ff
@@ -0,0 +1,29 @@
+config BR2_PACKAGE_LLVM_ARCH_SUPPORTS
+	bool
+	# http://llvm.org/docs/GettingStarted.html#host-c-toolchain-both-compiler-and-standard-library
+	# http://llvm.org/docs/GettingStarted.html#local-llvm-configuration
+	depends on BR2_HOST_GCC_AT_LEAST_4_8
+	default y if BR2_arm || BR2_armeb
+	default y if BR2_aarch64
+	default y if BR2_i386 || BR2_x86_64
+	default y if BR2_mips || BR2_mipsel || BR2_mips64 || BR2_mips64el
+	default y if BR2_powerpc || BR2_powerpc64 || BR2_powerpc64le
+	default y if BR2_sparc
+
+config BR2_PACKAGE_LLVM_TARGET_ARCH
+	string
+	default "ARM" if BR2_arm || BR2_armeb
+	default "AArch64" if BR2_aarch64
+	default "X86" if BR2_i386 || BR2_x86_64
+	default "Mips" if BR2_mips || BR2_mipsel || BR2_mips64 || BR2_mips64el
+	default "PowerPC" if BR2_powerpc || BR2_powerpc64 || BR2_powerpc64le
+	default "Sparc" if BR2_sparc
+
+config BR2_PACKAGE_LLVM
+	bool "llvm"
+	depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
+	help
+	  The LLVM Project is a collection of modular and reusable
+	  compiler and toolchain technologies.
+
+	  http://llvm.org
package/llvm/Config.in.host +0 -21
index 454cf10947..fc46535fad 100644
@@ -1,24 +1,3 @@
-config BR2_PACKAGE_LLVM_ARCH_SUPPORTS
-	bool
-	# http://llvm.org/docs/GettingStarted.html#host-c-toolchain-both-compiler-and-standard-library
-	# http://llvm.org/docs/GettingStarted.html#local-llvm-configuration
-	depends on BR2_HOST_GCC_AT_LEAST_4_8
-	default y if BR2_arm || BR2_armeb
-	default y if BR2_aarch64
-	default y if BR2_i386 || BR2_x86_64
-	default y if BR2_mips || BR2_mipsel || BR2_mips64 || BR2_mips64el
-	default y if BR2_powerpc || BR2_powerpc64 || BR2_powerpc64le
-	default y if BR2_sparc
-
-config BR2_PACKAGE_LLVM_TARGET_ARCH
-	string
-	default "ARM" if BR2_arm || BR2_armeb
-	default "AArch64" if BR2_aarch64
-	default "X86" if BR2_i386 || BR2_x86_64
-	default "Mips" if BR2_mips || BR2_mipsel || BR2_mips64 || BR2_mips64el
-	default "PowerPC" if BR2_powerpc || BR2_powerpc64 || BR2_powerpc64le
-	default "Sparc" if BR2_sparc
-
 config BR2_PACKAGE_HOST_LLVM
 	bool "host llvm"
 	depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
package/llvm/llvm.mk +83 -11

288 lines not on this page · raw message

Signature · 1 line
2.14.3
#3 lore raw

PATCH 3/4 package/llvm: add AMDGPU support

In reply to Valentin Korenblit

From: Romain Naour <[email protected]>

Limit the number of supported target for mesa3d when openCL support is enabled. With -DLLVM_TARGETS_TO_BUILD="all" mesa3d fail to build on x86_64 target with AMDGPU target:

CXXLD libOpenCL.la
../../../../src/gallium/state_trackers/clover/.libs/libclover.a(libclllvm_la-invocation.o): In function `(anonymous namespace)::create_context(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)':
invocation.cpp:(.text+0x1a8): undefined reference to `LLVMInitializeAArch64Target'
invocation.cpp:(.text+0x1b2): undefined reference to `LLVMInitializeARMTarget'
invocation.cpp:(.text+0x1b7): undefined reference to `LLVMInitializeBPFTarget'
invocation.cpp:(.text+0x1bc): undefined reference to `LLVMInitializeHexagonTarget'
invocation.cpp:(.text+0x1c1): undefined reference to `LLVMInitializeLanaiTarget'
invocation.cpp:(.text+0x1c6): undefined reference to `LLVMInitializeMipsTarget'
invocation.cpp:(.text+0x1cb): undefined reference to `LLVMInitializeMSP430Target'
invocation.cpp:(.text+0x1d0): undefined reference to `LLVMInitializeNVPTXTarget'
invocation.cpp:(.text+0x1d5): undefined reference to `LLVMInitializePowerPCTarget'
invocation.cpp:(.text+0x1da): undefined reference to `LLVMInitializeRISCVTarget'
invocation.cpp:(.text+0x1df): undefined reference to `LLVMInitializeSparcTarget'
invocation.cpp:(.text+0x1e4): undefined reference to `LLVMInitializeSystemZTarget'
invocation.cpp:(.text+0x1ee): undefined reference to `LLVMInitializeXCoreTarget'
invocation.cpp:(.text+0x1f3): undefined reference to `LLVMInitializeAArch64TargetInfo'
invocation.cpp:(.text+0x1fd): undefined reference to `LLVMInitializeARMTargetInfo'
invocation.cpp:(.text+0x202): undefined reference to `LLVMInitializeBPFTargetInfo'
invocation.cpp:(.text+0x207): undefined reference to `LLVMInitializeHexagonTargetInfo'
invocation.cpp:(.text+0x20c): undefined reference to `LLVMInitializeLanaiTargetInfo'
invocation.cpp:(.text+0x211): undefined reference to `LLVMInitializeMipsTargetInfo'
invocation.cpp:(.text+0x216): undefined reference to `LLVMInitializeMSP430TargetInfo'
invocation.cpp:(.text+0x21b): undefined reference to `LLVMInitializeNVPTXTargetInfo'
invocation.cpp:(.text+0x220): undefined reference to `LLVMInitializePowerPCTargetInfo'
invocation.cpp:(.text+0x225): undefined reference to `LLVMInitializeRISCVTargetInfo'
invocation.cpp:(.text+0x22a): undefined reference to `LLVMInitializeSparcTargetInfo'
invocation.cpp:(.text+0x22f): undefined reference to `LLVMInitializeSystemZTargetInfo'
invocation.cpp:(.text+0x239): undefined reference to `LLVMInitializeXCoreTargetInfo'
invocation.cpp:(.text+0x23e): undefined reference to `LLVMInitializeAArch64TargetMC'
invocation.cpp:(.text+0x248): undefined reference to `LLVMInitializeARMTargetMC'
invocation.cpp:(.text+0x24d): undefined reference to `LLVMInitializeBPFTargetMC'
invocation.cpp:(.text+0x252): undefined reference to `LLVMInitializeHexagonTargetMC'
invocation.cpp:(.text+0x257): undefined reference to `LLVMInitializeLanaiTargetMC'
invocation.cpp:(.text+0x25c): undefined reference to `LLVMInitializeMipsTargetMC'
invocation.cpp:(.text+0x261): undefined reference to `LLVMInitializeMSP430TargetMC'
invocation.cpp:(.text+0x266): undefined reference to `LLVMInitializeNVPTXTargetMC'
invocation.cpp:(.text+0x26b): undefined reference to `LLVMInitializePowerPCTargetMC'
invocation.cpp:(.text+0x270): undefined reference to `LLVMInitializeRISCVTargetMC'
invocation.cpp:(.text+0x275): undefined reference to `LLVMInitializeSparcTargetMC'
invocation.cpp:(.text+0x27a): undefined reference to `LLVMInitializeSystemZTargetMC'
invocation.cpp:(.text+0x284): undefined reference to `LLVMInitializeXCoreTargetMC'
invocation.cpp:(.text+0x289): undefined reference to `LLVMInitializeAArch64AsmPrinter'
invocation.cpp:(.text+0x293): undefined reference to `LLVMInitializeARMAsmPrinter'
invocation.cpp:(.text+0x298): undefined reference to `LLVMInitializeBPFAsmPrinter'
invocation.cpp:(.text+0x29d): undefined reference to `LLVMInitializeHexagonAsmPrinter'
invocation.cpp:(.text+0x2a2): undefined reference to `LLVMInitializeLanaiAsmPrinter'
invocation.cpp:(.text+0x2a7): undefined reference to `LLVMInitializeMipsAsmPrinter'
invocation.cpp:(.text+0x2ac): undefined reference to `LLVMInitializeMSP430AsmPrinter'
invocation.cpp:(.text+0x2b1): undefined reference to `LLVMInitializeNVPTXAsmPrinter'
invocation.cpp:(.text+0x2b6): undefined reference to `LLVMInitializePowerPCAsmPrinter'
invocation.cpp:(.text+0x2bb): undefined reference to `LLVMInitializeSparcAsmPrinter'
invocation.cpp:(.text+0x2c0): undefined reference to `LLVMInitializeSystemZAsmPrinter'
invocation.cpp:(.text+0x2ca): undefined reference to `LLVMInitializeXCoreAsmPrinter'
collect2: error: ld returned 1 exit status

---
 package/llvm/Config.in | 9 +++++++++
 1 file changed, 9 insertions(+)
package/llvm/Config.in +9 -0
index 84dde8d5ff..417af6ebe2 100644
@@ -27,3 +27,12 @@ config BR2_PACKAGE_LLVM
 	  compiler and toolchain technologies.
 
 	  http://llvm.org
+
+if BR2_PACKAGE_LLVM
+
+config BR2_PACKAGE_LLVM_TARGET_ARCH_AMDGPU
+	bool "AMDGPU"
+	help
+	  TODO
+
+endif # BR2_PACKAGE_LLVM
Signature · 1 line
2.14.3
#4 lore raw

PATCH 4/4 package/mesa3d: enable llvm support

In reply to Valentin Korenblit

From: Romain Naour <[email protected]>

mesa: disable shared libs

---
 package/mesa3d/Config.in |  8 ++++++++
 package/mesa3d/mesa3d.mk | 18 +++++++++++++++---
 2 files changed, 23 insertions(+), 3 deletions(-)
package/mesa3d/Config.in +8 -0
index f141587c4d..cc8960b1b1 100644
@@ -25,6 +25,14 @@ menuconfig BR2_PACKAGE_MESA3D
 
 if BR2_PACKAGE_MESA3D
 
+config BR2_PACKAGE_MESA3D_LLVM
+	bool "mesa3d w/ llvm"
+	depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
+	select BR2_PACKAGE_LLVM
+
+comment "mesa3d w/ llvm needs a llvm compiler"
+	depends on !BR2_PACKAGE_LLVM_ARCH_SUPPORTS
+
 # inform the .mk file of gallium, dri or vulkan driver selection
 config BR2_PACKAGE_MESA3D_GALLIUM_DRIVER
 	bool
package/mesa3d/mesa3d.mk +15 -3
index c275579fb8..2822f82bc6 100644
@@ -32,6 +32,21 @@ ifeq ($(BR2_SHARED_STATIC_LIBS),y)
 MESA3D_CONF_OPTS += --disable-static
 endif
 
+ifeq ($(BR2_PACKAGE_MESA3D_LLVM),y)
+# Mesa want to build libgallium.a unconditionally, so it can't be linked dynamically with llvm shared libraries.
+MESA3D_DEPENDENCIES += host-llvm llvm
+MESA3D_CONF_OPTS += \
+	--with-llvm-prefix=$(STAGING_DIR)/usr \
+	--disable-llvm-shared-libs \
+	--enable-llvm
+else
+# Avoid automatic search of llvm-config
+MESA3D_CONF_OPTS += \
+	--with-llvm-prefix=$(STAGING_DIR)/usr \
+	--disable-llvm-shared-libs \
+	--disable-llvm
+endif
+
 # The Sourcery MIPS toolchain has a special (non-upstream) feature to
 # have "compact exception handling", which unfortunately breaks with
 # mesa3d, so we disable it here by passing -mno-compact-eh.
@@ -219,7 +234,4 @@ else
 MESA3D_CONF_OPTS += --disable-lmsensors
 endif
 
-# Avoid automatic search of llvm-config
-MESA3D_CONF_OPTS += --with-llvm-prefix=$(STAGING_DIR)/usr/bin
-
 $(eval $(autotools-package))
Signature · 1 line
2.14.3
#5 lore raw

cover llvm for mesa3d

In reply to Valentin Korenblit
Quoted 9 lines · Valentin Korenblit

Hello all,

This series provides LLVM support for Mesa3d, enabling llvmpipe software rasterizer.

It must be considered that it does not provide a full LLVM installation for the host, only the necessary components to cross-compile LLVM for the target. It will install all target components as libLLVM.so.

Hi Valentin,

As it happens I've just been updating my patches for llvm as I added it to get radeonsi gallium working in mesa3d. My patches are much messier than your and hence I've never tried to upstream anything.

A couple of overall questions:

Have you only added host-llvm because of llvm-tblgen and llvm-config? llvm will try to build native versions of those for you but I had problems with llvm-tblgen and so had to go to a host package like you. However I only built those two tools with:

HOST_LLVM_MAKE_OPTS = llvm-tblgen llvm-config

It makes the build smaller and faster and it means you don't need to care about almost all of the host conf options.

My other question is about AMDGPU support, you seem to have mixed this up into a number of your patches not just number 3. What are you trying to do here, add an option to build amdgpu as well as the target arch? In my case I wanted to be able to build for only amdgpu and not the target arch as this is what radeonsi needs.

Regards
Will

Signature · 5 lines
------------------------------------------------------------------------
Will Wagner                                     will_wagner at carallon.com
Engineering Manager                      Office Tel: +44 (0)20 7471 9424
Carallon Ltd, International House, 7 High Street, Ealing, London  W5 5DB
------------------------------------------------------------------------
#6 lore raw

cover llvm for mesa3d

Hi WIll,

Quoted 24 lines · Will Wagner
Quoted 9 lines · Valentin Korenblit

Hello all,

This series provides LLVM support for Mesa3d, enabling llvmpipe software rasterizer.

It must be considered that it does not provide a full LLVM installation for the host, only the necessary components to cross-compile LLVM for the target. It will install all target components as libLLVM.so.

Hi Valentin,

As it happens I've just been updating my patches for llvm as I added it to get radeonsi gallium working in mesa3d. My patches are much messier than your and hence I've never tried to upstream anything.

A couple of overall questions:

Have you only added host-llvm because of llvm-tblgen and llvm-config? llvm will try to build native versions of those for you but I had problems with llvm-tblgen and so had to go to a host package like you. However I only built those two tools with:

Quoted 4 lines

HOST_LLVM_MAKE_OPTS = llvm-tblgen llvm-config

It makes the build smaller and faster and it means you don't need to care about almost all of the host conf options.

Correct, host-llvm installs only llvm-tblgen and llvm-config and I agree with you that we could use that option for this case.

Quoted 6 lines

My other question is about AMDGPU support, you seem to have mixed this up into a number of your patches not just number 3. What are you trying to do here, add an option to build amdgpu as well as the target arch? In my case I wanted to be able to build for only amdgpu and not the target arch as this is what radeonsi needs.

You're right, I should move:

ifeq ($(BR2_PACKAGE_LLVM_TARGET_ARCH_AMDGPU),y) HOST_LLVM_TARGET_TO_BUILD = "$(HOST_LLVM_TARGET_ARCH);AMDGPU" endif

to patch 3. Yes, by using this option I limit the targets to build to 'target_arch + amdgpu' because by default it is set to 'all'.

Quoted 4 lines

Regards
Will

Thanks and regards,

Valent?n

#7 lore raw

cover llvm for mesa3d

Hi, thank you for your contribution. I've been manually building and using llvmpipe w/ mesa3d based on Romain's RFC series in patchwork (ignoring the clang, clc stuff), but it would be nice if we can get it officially added.

Out of curiosity, have you tested this with mesa3d 17.3.x? Those versions broke llvm for me w/ Romain's older patches ... mesa fails to find and link against the libs. I had to downgrade to the latest 17.2.x (17.2.8 at the time of this posting).

--
Sent from: http://buildroot-busybox.2317881.n4.nabble.com/

#8 lore raw

cover llvm for mesa3d

Quoted 11 lines · nimaim

Hi, thank you for your contribution. I've been manually building and using llvmpipe w/ mesa3d based on Romain's RFC series in patchwork (ignoring the clang, clc stuff), but it would be nice if we can get it officially added.

Out of curiosity, have you tested this with mesa3d 17.3.x? Those versions broke llvm for me w/ Romain's older patches ... mesa fails to find and link against the libs. I had to downgrade to the latest 17.2.x (17.2.8 at the time of this posting).

Quoted 6 lines

--
Sent from: http://buildroot-busybox.2317881.n4.nabble.com/
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

Hi,

I'm using mesa3d 17.3.4 and it works correctly. You should also consider that the RFC series uses llvm 4.0.1.

Regards,

Valent?n

#9 lore raw

cover llvm for mesa3d

Quoted 25 lines · Valentin KORENBLIT
Quoted 12 lines · nimaim

Hi, thank you for your contribution. I've been manually building and using llvmpipe w/ mesa3d based on Romain's RFC series in patchwork (ignoring the clang, clc stuff), but it would be nice if we can get it officially added.

Out of curiosity, have you tested this with mesa3d 17.3.x? Those versions broke llvm for me w/ Romain's older patches ... mesa fails to find and link
against the libs. I had to downgrade to the latest 17.2.x (17.2.8 at the time of this posting).

Quoted 7 lines

--
Sent from: http://buildroot-busybox.2317881.n4.nabble.com/
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

Hi,

Quoted 8 lines

I'm using mesa3d 17.3.4 and it works correctly. You should also consider that the RFC series uses llvm 4.0.1.

Regards,

Valent?n

Thanks for the confirmation. I actually bumped that series up to 5.0.1
before I tested it but it still broke mesa3d 17.3.x (17.2.x worked fine).
I'm guessing your other fixes to this patch did the trick. Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20180219/55806ac2/attachment.html>