Skip to contentSkip to conversation

Use llvmpile for mesa on x86/x86-64

PATCH Superseded Khem Raj 1 participant 7 messages Aug 18, 2018

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Khem Raj

author
#0 lore raw

cover Use llvmpile for mesa on x86/x86-64

This series fixes llvm build to support multilib configurations Adds dos2unix bbclass to core from meta-oe
Fixes ppc security flags world builds

The following changes since commit 176e50fb177dc40e439700d5f3f838dd7eaaa427:

runtime selftest: limit kernel hw bp arches (2018-08-16 22:40:06 +0100)

are available in the Git repository at:

git://git.openembedded.org/openembedded-core-contrib kraj/master
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=kraj/master

Khem Raj (6):
mesa: Enable gallium-llvm on x86 and x86_64
powerpc: Remove pie flags from compiler cmdline
llvm: Point llvm-config to one built with llvm-native dos2unix.bbclass: Move to oe-core
llvm: Use YOCTO_ALTERNATE_MULTILIB_NAME environment variable in
llvm-config
mesa: Define YOCTO_ALTERNATE_MULTILIB_NAME to consider multilib names

 meta/classes/dos2unix.bbclass                 | 14 ++++++
 meta/conf/distro/include/security_flags.inc   |  7 +--
 ...-llvm-allow-env-override-of-exe-path.patch | 47 ++++++++++++++++---
 meta/recipes-devtools/llvm/llvm_git.bb        |  4 +-
 meta/recipes-graphics/cairo/cairo.inc         |  3 +-
 meta/recipes-graphics/mesa/mesa.inc           |  4 ++
 6 files changed, 65 insertions(+), 14 deletions(-)
 create mode 100644 meta/classes/dos2unix.bbclass
Signature · 1 line
2.18.0
author
#1 lore raw

PATCH 1/6 mesa: Enable gallium-llvm on x86 and x86_64

---
 meta/recipes-graphics/cairo/cairo.inc | 3 ++-
 meta/recipes-graphics/mesa/mesa.inc   | 3 +++
 2 files changed, 5 insertions(+), 1 deletion(-)
meta/recipes-graphics/cairo/cairo.inc +2 -1
index 20e0d2c92a..7347f223ff 100644
@@ -22,7 +22,8 @@ X11DEPENDS = "virtual/libx11 libsm libxrender libxext"
 DEPENDS = "libpng fontconfig pixman glib-2.0 zlib"
 
 PACKAGECONFIG ??= "${@bb.utils.contains('DISTRO_FEATURES', 'x11', 'x11 xcb', '', d)} \
-           ${@bb.utils.filter('DISTRO_FEATURES', 'directfb', d)}"
+           ${@bb.utils.filter('DISTRO_FEATURES', 'directfb', d)} \
+           ${@bb.utils.contains('DISTRO_FEATURES', 'x11 opengl', 'opengl', '', d)}"
 
 PACKAGECONFIG[x11] = "--with-x=yes -enable-xlib,--with-x=no --disable-xlib,${X11DEPENDS}"
 PACKAGECONFIG[xcb] = "--enable-xcb,--disable-xcb,libxcb"
meta/recipes-graphics/mesa/mesa.inc +3 -0
index 6aedeee9dc..2671d4de4d 100644
@@ -47,6 +47,9 @@ PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', 'wayland vulkan', d)} \
                    ${@bb.utils.contains('DISTRO_FEATURES', 'x11 vulkan', 'dri3', '', d)} \
 		   "
 
+PACKAGECONFIG_append_x86 = " gallium-llvm gallium r600"
+PACKAGECONFIG_append_x86-64 = " gallium-llvm gallium r600"
+
 # "gbm" requires "dri", "opengl"
 PACKAGECONFIG[gbm] = "--enable-gbm,--disable-gbm"
 
Signature · 1 line
2.18.0
author
#2 lore raw

PATCH 2/6 powerpc: Remove pie flags from compiler cmdline

In reply to Khem Raj

Original approach to add -no-<pie> flags cause link time behavior changes where packages start to lose the -fPIC -DPIC in compiler cmdline and this list keeps growing as we build more and more packages,

Instead lets just remove the options we dont need from SECURITY_CFLAGS this makes it more robust and less intrusive

This also means we do not need to re-add pic options as we started to do for affected packages

---
 meta/conf/distro/include/security_flags.inc | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)
meta/conf/distro/include/security_flags.inc +2 -5
index 52e1e4ebc5..4e20a4d0f8 100644
@@ -28,13 +28,10 @@ SECURITY_LDFLAGS ?= "-fstack-protector-strong -Wl,-z,relro,-z,now"
 SECURITY_X_LDFLAGS ?= "-fstack-protector-strong -Wl,-z,relro"
 
 # powerpc does not get on with pie for reasons not looked into as yet
-SECURITY_CFLAGS_powerpc = "-fstack-protector-strong ${lcl_maybe_fortify} ${SECURITY_NOPIE_CFLAGS}"
-SECURITY_CFLAGS_pn-libgcc_powerpc = ""
-SECURITY_CFLAGS_append_powerpc_pn-libdrm = " -fPIC -DPIC"
-SECURITY_CFLAGS_append_powerpc_pn-libinput = " -fPIC -DPIC"
-SECURITY_CFLAGS_append_powerpc_pn-at-spi2-core = " -fPIC -DPIC"
 GCCPIE_powerpc = ""
 GLIBCPIE_powerpc = ""
+SECURITY_CFLAGS_remove_powerpc = "${SECURITY_PIE_CFLAGS}"
+SECURITY_CFLAGS_pn-libgcc_powerpc = ""
 
 SECURITY_CFLAGS_pn-glibc = ""
 SECURITY_CFLAGS_pn-glibc-initial = ""
Signature · 1 line
2.18.0
author
#3 lore raw

PATCH 3/6 llvm: Point llvm-config to one built with llvm-native

In reply to Khem Raj

If not defined, llvm build system tries to build one which then confuses the OE QA system since its building native tool and target packages in same package build moreover it is not required since we already have it via llvm-native

Fixes
ERROR: llvm-6.0-r0 do_package_qa: QA Issue: llvm: The compile log indicates that host include and/or library paths were used.

---
 meta/recipes-devtools/llvm/llvm_git.bb | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
meta/recipes-devtools/llvm/llvm_git.bb +3 -1
index bb83012231..d853d4151d 100644
@@ -64,17 +64,19 @@ EXTRA_OECMAKE += "-DLLVM_ENABLE_ASSERTIONS=OFF \
                   -DLLVM_ENABLE_FFI=ON \
                   -DFFI_INCLUDE_DIR=$(pkg-config --variable=includedir libffi) \
                   -DLLVM_OPTIMIZED_TABLEGEN=ON \
-                  -DLLVM_TARGETS_TO_BUILD="${LLVM_TARGETS}" \
+                  -DLLVM_TARGETS_TO_BUILD='${LLVM_TARGETS}' \
                   -G Ninja"
 
 EXTRA_OECMAKE_append_class-target = "\
                   -DCMAKE_CROSSCOMPILING:BOOL=ON \
                   -DLLVM_TABLEGEN=${STAGING_BINDIR_NATIVE}/llvm-tblgen${PV} \
+                  -DLLVM_CONFIG_PATH=${STAGING_BINDIR_NATIVE}/llvm-config${PV} \
                  "
 
 EXTRA_OECMAKE_append_class-nativesdk = "\
                   -DCMAKE_CROSSCOMPILING:BOOL=ON \
                   -DLLVM_TABLEGEN=${STAGING_BINDIR_NATIVE}/llvm-tblgen${PV} \
+                  -DLLVM_CONFIG_PATH=${STAGING_BINDIR_NATIVE}/llvm-config${PV} \
                  "
 
 do_configure_prepend() {
Signature · 1 line
2.18.0
author
#4 lore raw

PATCH 4/6 dos2unix.bbclass: Move to oe-core

In reply to Khem Raj

- Import from meta-oe layer

- This is useful for many packages where CR-LF
needs to be adjusted, many recipes depend on it
e.g. meta-multimedia libebml and so on.

---
 meta/classes/dos2unix.bbclass | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
 create mode 100644 meta/classes/dos2unix.bbclass
meta/classes/dos2unix.bbclass new file +14 -0
new file mode 100644
index 0000000000..3fc17e2196
@@ -0,0 +1,14 @@
+# Class for use to convert all CRLF line terminators to LF
+# provided that some projects are being developed/maintained
+# on Windows so they have different line terminators(CRLF) vs
+# on Linux(LF), which can cause annoying patching errors during
+# git push/checkout processes.
+
+do_convert_crlf_to_lf[depends] += "dos2unix-native:do_populate_sysroot"
+
+# Convert CRLF line terminators to LF
+do_convert_crlf_to_lf () {
+	find ${S} -type f -exec dos2unix {} \;
+}
+
+addtask convert_crlf_to_lf after do_unpack before do_patch
Signature · 1 line
2.18.0
author
#5 lore raw

PATCH 5/6 llvm: Use YOCTO_ALTERNATE_MULTILIB_NAME environment variable in llvm-config

In reply to Khem Raj

llvm-config is a tool on similar veins as pkg-config but provides a lot more information and packages which use llvm e.g. mesa use this tool to poke for llvm related informaiton e.g. version, libpath, includepaths to name a few, this has few challanges in cross build environments where llvm-config is supposed to be build for buildhost but provide information about target llvm which is addressed by building native llvm-config along with target llvm build, but this is frowned upon by OE build system since it detects that host paths are being used so we have to build it as part of llvm-native but then it means install paths for llvm and llvm-native are different and wrong paths get reported when llvm-config is used. This is solved by providing YOCTO_ALTERNATE_EXE_PATH variable to let llvm-config use that path instead of self-relative path to report back

Second problem is when building multi-lib packages base_libdir is different for target packages but native llvm-config does not know about it so it reports non-multilibbed paths as libdir and packages can not find llvm in sysroot. This is fixed by adding another environment variable YOCTO_ALTERNATE_MULTILIB_NAME which can be set from recipes to set proper multilib path

---
 ...-llvm-allow-env-override-of-exe-path.patch | 47 ++++++++++++++++---
 1 file changed, 40 insertions(+), 7 deletions(-)
meta/recipes-devtools/llvm/llvm/0002-llvm-allow-env-override-of-exe-path.patch +40 -7
index 21d2f81b58..6a9283323c 100644
@@ -1,4 +1,4 @@
-From 2f8ea767afdaa440c6368040630e1b3ea6a0977a Mon Sep 17 00:00:00 2001
+From aeccf16eaccdd80e4d5ecaa51673ce4b2bac1130 Mon Sep 17 00:00:00 2001
 From: Martin Kelly <[email protected]>
 Date: Fri, 19 May 2017 00:22:57 -0700
 Subject: [PATCH 2/2] llvm: allow env override of exe path
@@ -8,16 +8,19 @@ return the libraries, include directories, etc. from inside the sysroot rather
 than from the native sysroot. Thus provide an env override for calling
 llvm-config from a target sysroot.
 
+To let it work in multilib environment, we need to provide a knob to supply
+multilib dirname as well
+
+Upstream-Status: Inappropriate [OE-Specific]
+
 Signed-off-by: Martin Kelly <[email protected]>
 Signed-off-by: Khem Raj <[email protected]>
 ---
-Upstream-Status: Pending
-
- tools/llvm-config/llvm-config.cpp | 7 +++++++
- 1 file changed, 7 insertions(+)
+ tools/llvm-config/llvm-config.cpp | 17 ++++++++++++++++-
+ 1 file changed, 16 insertions(+), 1 deletion(-)
 
 diff --git a/tools/llvm-config/llvm-config.cpp b/tools/llvm-config/llvm-config.cpp
-index 08b096afb05..d8d7742744e 100644
+index 08b096afb05..360cc5abf4e 100644
 --- a/tools/llvm-config/llvm-config.cpp
 +++ b/tools/llvm-config/llvm-config.cpp
 @@ -225,6 +225,13 @@ Typical components:\n\
@@ -34,6 +37,36 @@ index 08b096afb05..d8d7742744e 100644
    // This just needs to be some symbol in the binary; C++ doesn't
    // allow taking the address of ::main however.
    void *P = (void *)(intptr_t)GetExecutablePath;
+@@ -306,12 +313,20 @@ int main(int argc, char **argv) {
+   std::string ActivePrefix, ActiveBinDir, ActiveIncludeDir, ActiveLibDir,
+               ActiveCMakeDir;
+   std::string ActiveIncludeOption;
++  // Hack for Yocto: we need to override the multilib path when we are using
++  // llvm-config from within a target sysroot.
++  std::string Multilibdir = std::getenv("YOCTO_ALTERNATE_MULTILIB_NAME");
++  if (Multilibdir.empty()) {
++    Multilibdir = "/lib" LLVM_LIBDIR_SUFFIX;
++  }
++
+   if (IsInDevelopmentTree) {
+     ActiveIncludeDir = std::string(LLVM_SRC_ROOT) + "/include";
+     ActivePrefix = CurrentExecPrefix;
+ 
+     // CMake organizes the products differently than a normal prefix style
+     // layout.
++
+     switch (DevelopmentTreeLayout) {
+     case CMakeStyle:
+       ActiveBinDir = ActiveObjRoot + "/bin";
+@@ -336,7 +351,7 @@ int main(int argc, char **argv) {
+     SmallString<256> path(StringRef(LLVM_TOOLS_INSTALL_DIR));
+     sys::fs::make_absolute(ActivePrefix, path);
+     ActiveBinDir = path.str();
+-    ActiveLibDir = ActivePrefix + "/lib" + LLVM_LIBDIR_SUFFIX;
++    ActiveLibDir = ActivePrefix + Multilibdir;
+     ActiveCMakeDir = ActiveLibDir + "/cmake/llvm";
+     ActiveIncludeOption = "-I" + ActiveIncludeDir;
+   }
 -- 
-2.16.1
+2.18.0
 
Signature · 1 line
2.18.0
author
#6 lore raw

PATCH 6/6 mesa: Define YOCTO_ALTERNATE_MULTILIB_NAME to consider multilib names

In reply to Khem Raj
---
 meta/recipes-graphics/mesa/mesa.inc | 1 +
 1 file changed, 1 insertion(+)
meta/recipes-graphics/mesa/mesa.inc +1 -0
index 2671d4de4d..dd626d9f00 100644
@@ -34,6 +34,7 @@ PLATFORMS ??= "${@bb.utils.filter('PACKAGECONFIG', 'x11 wayland', d)} \
 
 export LLVM_CONFIG = "${STAGING_BINDIR_NATIVE}/llvm-config${MESA_LLVM_RELEASE}"
 export YOCTO_ALTERNATE_EXE_PATH = "${STAGING_LIBDIR}/llvm${MESA_LLVM_RELEASE}/llvm-config"
+export YOCTO_ALTERNATE_MULTILIB_NAME = "${base_libdir}"
 EXTRA_OECONF = "--enable-shared-glapi \
                 --disable-opencl \
                 --enable-glx-read-only-text \
Signature · 1 line
2.18.0