qt5webkit: fix build issue using system leveldb
v2 cover qt5webkit: fix build issue using system leveldb
Hi all,
This patchset fixes the build issue reported by autobuilder [0].
/home/naourr/work/instance-2/output/build/qt5webkit-5.9.1/Source/WebCore//.obj/platform/leveldb/LevelDBDatabase.o: In function `WebCore::LevelDBDatabase::openInMemory(WebCore::LevelDBComparator const*)':LevelDBDatabase.cpp.text._ZN7WebCore15LevelDBDatabase12openInMemoryEPKNS_17LevelDBComparatorE+0x34): undefined reference to `leveldb::NewMemEnv(leveldb::Env*)'
collect2: error: ld returned 1 exit status
make[3]: *** [Makefile.api:97: ../lib/libQt5WebKit.so.5.9.1] Error 1
The issue happens when the package leveldb is enabled.
QtWebKit builds its own copy of leveldb [1] (as a third-party) if the system does not provided it (i.e. buildroot). It builds it differently and this is the origin of that issue. Instead of using the Makefile provided by leveldb [2], QtWebKit uses qmake to build that library [3].
The missing symbol issue happens because the symbol leveldb::NewMemEnv is bundled in the static library libmemenv.a (aside libleveldb.so). This static library consists of this single symbol which is like an extra that is built but *NOT* shipped by default at installation in the staging directory. Unfortunatly, that symbol is required later by WebCore [4].
The missing symbol issue happens because the symbol leveldb::NewMemEnv is bundled in the static library libmemenv.a (aside libleveldb.so). This static library consists of this single symbol which is like an extra that is built but *NOT* shipped by default at installation in the staging directory. Unfortunatly, that symbol is required later by WebCore [4].
The copy built by QtWebKit is an all-in-one library including both libleveldb and libmemenv; thus QtWebKit links against libleveldb only. Also, the linker finds the buildroot's copy first (not the third-party):
that explains why it is complaining about a missing symbol. That copy does not have the symbol leveldb::NewMemEnv.
Fortunatly, QtWebKit provides a facility to link against the system leveldb package. The qmake flag WEBKIT_CONFIG+=use_system_leveldb tells Qt5WebKit to link against libleveldb *AND* libmemenv [5].
To fix that issue, this patchset builds the leveldb package and tells QtWebKit to use it instead of building its own copy.
The first two patches concern the leveldb package.
The first one adds installation of headers and the missing static library in the staging directory.
The second one patches leveldb to generate position independant code for the static library libmemenv.a; in order let it linkable by the Qt5WebKit module. Without that patch, the build raises the error below:
/home/gportay/src/buildroot/output/host/opt/ext-toolchain/bin/../lib/gcc/x86_64-amd-linux-gnu/6.2.0/../../../../x86_64-amd-linux-gnu/bin/ld: /home/gportay/src/buildroot/output/host/x86_64-buildroot-linux-gnu/sysroot/usr/lib/libmemenv.a(memenv.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC /home/gportay/src/buildroot/output/host/x86_64-buildroot-linux-gnu/sysroot/usr/lib/libmemenv.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
It it not relevant to upstream that patch since the project has already moved to cmake [6] for the (upcoming?) version 1.21. Backporting the upstreamed patches was way too complicated.
The last patch selects the leveldb package in qt5webkit, installs memenv and sets the appropriate qmake flag to tell it to use the package provided by buildroot.
[0]: http://autobuild.buildroot.net/results/8afc8771741f62da3b2117d2124d1c6f2d941903/
[1]: https://github.com/qt/qtwebkit/tree/5.9/Source/ThirdParty/leveldb
[2]: https://github.com/qt/qtwebkit/blob/5.9/Source/ThirdParty/leveldb/Makefile#L167-L169
[3]: https://github.com/qt/qtwebkit/blob/5.9/Source/ThirdParty/leveldb/Target.pri#L80
[4]: https://github.com/qt/qtwebkit/blob/5.9/Source/WebCore/platform/leveldb/LevelDBDatabase.cpp#L185
[5]: https://github.com/qt/qtwebkit/blob/5.9/Source/WebCore/WebCore.pri#L254
[6]: https://github.com/google/leveldb/commit/739c25100e46576cdcdfff2d6f43f9f7008103c7
Regards,
Ga?l PORTAY (3):
leveldb: install memenv static library and header
leveldb: generate pic for static libraries
qt5webkit: select leveldb package and memenv
...sition-independant-code-for-static-librar.patch | 52 ++++++++++++++++++++++ package/leveldb/Config.in | 10 +++++ package/leveldb/leveldb.mk | 8 ++++ package/qt5/qt5webkit/Config.in | 4 ++ package/qt5/qt5webkit/qt5webkit.mk | 4 +- 5 files changed, 76 insertions(+), 2 deletions(-) create mode 100644 package/leveldb/0001-Generate-position-independant-code-for-static-librar.patch
Signature · 1 line
2.11.0
PATCH v2 1/3 leveldb: install memenv static library and header
The project builds a tiny static library that consists of a single symbol which creates an in-memory LevelDB database.
That library is not installed by default and may be used by other projects.
This commit creates a config option that installs in the staging directory the libmemenv.a static library and the memenv.h header file when the option is set.
--- package/leveldb/Config.in | 10 ++++++++++ package/leveldb/leveldb.mk | 8 ++++++++ 2 files changed, 18 insertions(+)
package/leveldb/Config.in +10 -0
@@ -10,5 +10,15 @@ config BR2_PACKAGE_LEVELDB https://github.com/google/leveldb +if BR2_PACKAGE_LEVELDB + +config BR2_PACKAGE_LEVELDB_MEMENV + bool "memenv" + help + Installs memenv static library and header to create in-memory + LevelDB database. + +endif + comment "leveldb needs a toolchain w/ C++, threads" depends on !BR2_INSTALL_LIBSTDCPP || !BR2_TOOLCHAIN_HAS_THREADS
package/leveldb/leveldb.mk +8 -0
@@ -21,10 +21,18 @@ define LEVELDB_BUILD_CMDS $(LEVELDB_MAKE_ARGS) -C $(@D) endef +ifdef BR2_PACKAGE_LEVELDB_MEMENV +define LEVELDB_INSTALL_MEMENV + $(INSTALL) -m 0644 $(@D)/out-static/libmemenv.a $(STAGING_DIR)/usr/lib + $(INSTALL) -D -m 0644 $(@D)/helpers/memenv/memenv.h $(STAGING_DIR)/usr/include/helpers/memenv/memenv.h +endef +endif + define LEVELDB_INSTALL_STAGING_CMDS $(TARGET_MAKE_ENV) $(MAKE) \ INSTALL_ROOT=$(STAGING_DIR) INSTALL_PREFIX=/usr \ $(LEVELDB_MAKE_ARGS) -C $(@D) install + $(LEVELDB_INSTALL_MEMENV) endef define LEVELDB_INSTALL_TARGET_CMDS
Signature · 1 line
2.11.0
PATCH v2 1/3 leveldb: install memenv static library and header
Hello,
Quoted 1 line · Ga?l PORTAY
+ifdef
BR2_PACKAGE_LEVELDB_MEMENV
Too much vacations here! :-) You have forgotten how Buildroot does this normally:
ifeq ($(BR2_PACKAGE_LEVELDB_MEMENV),y)
package/leveldb/leveldb.mk define LEVELDB_BUILD_CMDS · line 26
Quoted 2 lines
package/leveldb/leveldb.mk +2
-0+define LEVELDB_INSTALL_MEMENV + $(INSTALL) -m 0644 $(@D)/out-static/libmemenv.a $(STAGING_DIR)/usr/lib
-D option and full destination path please.
Thanks!
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH v2 1/3 leveldb: install memenv static library and header
Thomas,
Quoted 11 lines · Thomas Petazzoni
Hello,
Quoted 1 line · Ga?l PORTAY
+ifdef
BR2_PACKAGE_LEVELDB_MEMENVToo much vacations here! :-) You have forgotten how Buildroot does this normally:
ifeq ($(BR2_PACKAGE_LEVELDB_MEMENV),y)
... and I also forgot to add the Changes in the coverletter :(
Quoted 5 lines
package/leveldb/leveldb.mk define LEVELDB_BUILD_CMDS · line 26
Quoted 2 lines
package/leveldb/leveldb.mk +2
-0+define LEVELDB_INSTALL_MEMENV + $(INSTALL) -m 0644 $(@D)/out-static/libmemenv.a $(STAGING_DIR)/usr/lib-D option and full destination path please.
Okay, I will resend a v3 later that day.
Quoted 7 lines
Thanks!
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Regards,
Ga?l
PATCH v2 2/3 leveldb: generate pic for static libraries
In reply to Gaël PORTAYThe project's static libraries are not compiled with the -fPIC compiler flag that prevents any libraries to link against those libraries.
This commit adds a patch that sets the -fPIC compiler flag to the list of CFLAGS/CXXFLAGS.
The project now generates position independant code for all of its outputs (i.e. not limited anymore to its shared libraries).
Fixes:
/home/gportay/src/buildroot/output/host/opt/ext-toolchain/bin/../lib/gcc/x86_64-amd-linux-gnu/6.2.0/../../../../x86_64-amd-linux-gnu/bin/ld: /home/gportay/src/buildroot/output/host/x86_64-buildroot-linux-gnu/sysroot/usr/lib/libmemenv.a(memenv.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC /home/gportay/src/buildroot/output/host/x86_64-buildroot-linux-gnu/sysroot/usr/lib/libmemenv.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
--- ...sition-independant-code-for-static-librar.patch | 52 ++++++++++++++++++++++ 1 file changed, 52 insertions(+) create mode 100644 package/leveldb/0001-Generate-position-independant-code-for-static-librar.patch
package/leveldb/0001-Generate-position-independant-code-for-static-librar.patch new file +52 -0
@@ -0,0 +1,52 @@ +From 6ed1b57ef6bcee0d497c181730710b2b0fafbfb3 Mon Sep 17 00:00:00 2001 +From: =?utf-8?q?Ga=C3=ABl=20PORTAY?= <[email protected]> +Date: Fri, 31 Aug 2018 12:23:46 -0400 +Subject: [PATCH] Generate position independant code for static library +MIME-Version: 1.0 +Content-Type: text/plain; charset=utf-8 +Content-Transfer-Encoding: 8bit + +Currently, only shared libraries are using the PIC flag. + +Generalize this flag for static libraries in order to let them linkable +by dynamic libraries. + +Fixes: + + /home/gportay/src/buildroot/output/host/opt/ext-toolchain/bin/../lib/gcc/x86_64-amd-linux-gnu/6.2.0/../../../../x86_64-amd-linux-gnu/bin/ld: /home/gportay/src/buildroot/output/host/x86_64-buildroot-linux-gnu/sysroot/usr/lib/libmemenv.a(memenv.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC + /home/gportay/src/buildroot/output/host/x86_64-buildroot-linux-gnu/sysroot/usr/lib/libmemenv.a: error adding symbols: Bad value + collect2: error: ld returned 1 exit status + +Upstream-Status: Inappropriate [upstream has migrated to cmake] +Signed-off-by: Ga?l PORTAY <[email protected]> +--- + build_detect_platform | 6 +++--- + 1 file changed, 3 insertions(+), 3 deletions(-) + +diff --git a/build_detect_platform b/build_detect_platform +index d2a20ce..4839444 100755 +--- a/build_detect_platform ++++ b/build_detect_platform +@@ -55,8 +55,8 @@ fi + + COMMON_FLAGS= + CROSS_COMPILE= +-PLATFORM_CCFLAGS= +-PLATFORM_CXXFLAGS= ++PLATFORM_CCFLAGS="-fPIC" ++PLATFORM_CXXFLAGS="-fPIC" + PLATFORM_LDFLAGS= + PLATFORM_LIBS= + PLATFORM_SHARED_EXT="so" +@@ -197,7 +197,7 @@ else + EOF + if [ "$?" = 0 ]; then + COMMON_FLAGS="$COMMON_FLAGS -DLEVELDB_PLATFORM_POSIX -DLEVELDB_ATOMIC_PRESENT" +- PLATFORM_CXXFLAGS="-std=c++0x" ++ PLATFORM_CXXFLAGS="$PLATFORM_CXXFLAGS -std=c++0x" + else + COMMON_FLAGS="$COMMON_FLAGS -DLEVELDB_PLATFORM_POSIX" + fi +-- +2.18.0 +
Signature · 1 line
2.11.0
PATCH v2 3/3 qt5webkit: select leveldb package and memenv
In reply to Gaël PORTAYThis patch fixes the build issue reported by autobuilder [0].
/home/naourr/work/instance-2/output/build/qt5webkit-5.9.1/Source/WebCore//.obj/platform/leveldb/LevelDBDatabase.o: In function `WebCore::LevelDBDatabase::openInMemory(WebCore::LevelDBComparator const*)':
LevelDBDatabase.cpp.text._ZN7WebCore15LevelDBDatabase12openInMemoryEPKNS_17LevelDBComparatorE+0x34): undefined reference to `leveldb::NewMemEnv(leveldb::Env*)'
collect2: error: ld returned 1 exit status
make[3]: *** [Makefile.api:97: ../lib/libQt5WebKit.so.5.9.1]
Error 1
The issue happens when both packages leveldb and qt5webkit are enabled.
QtWebKit builds its own copy of leveldb [1] (as a third-party) if the system does not provided it (i.e. buildroot). It builds it differently and this is the origin of that issue. Instead of using the Makefile provided by leveldb [2], QtWebKit uses qmake to build that library [3].
The missing symbol issue happens because the symbol leveldb::NewMemEnv is bundled in the static library libmemenv.a (aside libleveldb.so). This static library consists of this single symbol which is like an extra that is built but *NOT* shipped by default at installation in the staging directory. Unfortunatly, that symbol is required later by WebCore [4].
The copy built by QtWebKit is an all-in-one library including both libleveldb and libmemenv; thus QtWebKit links against libleveldb only. Also, the linker finds the buildroot's copy first (not the third-party):
that explains why it is complaining about a missing symbol. That copy does not have the symbol leveldb::NewMemEnv.
Fortunatly, QtWebKit provides a facility to link against the system leveldb package. The qmake flag WEBKIT_CONFIG+=use_system_leveldb tells Qt5WebKit to link against libleveldb *AND* libmemenv [5].
To fix that issue, this commit selects the package leveldb and its option BR2_PACKAGE_LEVELDB_MEMENV to install libmemenv static library and its header. It ensures that QtWebKit has every thing it needs to be built. It also sets the appropriate qmake configure flags to tells QtWebKit to use the leveldb copy build by the system (instead of the one that can be build internaly as a third-party).
[0]: http://autobuild.buildroot.net/results/8afc8771741f62da3b2117d2124d1c6f2d941903/
[1]: https://github.com/qt/qtwebkit/tree/5.9/Source/ThirdParty/leveldb
[2]: https://github.com/qt/qtwebkit/blob/5.9/Source/ThirdParty/leveldb/Makefile#L167-L169
[3]: https://github.com/qt/qtwebkit/blob/5.9/Source/ThirdParty/leveldb/Target.pri#L80
[4]: https://github.com/qt/qtwebkit/blob/5.9/Source/WebCore/platform/leveldb/LevelDBDatabase.cpp#L185
[5]: https://github.com/qt/qtwebkit/blob/5.9/Source/WebCore/WebCore.pri#L254
[6]: https://github.com/google/leveldb/commit/739c25100e46576cdcdfff2d6f43f9f7008103c7
--- package/qt5/qt5webkit/Config.in | 4 ++++ package/qt5/qt5webkit/qt5webkit.mk | 4 ++-- 2 files changed, 6 insertions(+), 2 deletions(-)
package/qt5/qt5webkit/Config.in +4 -0
@@ -3,10 +3,14 @@ config BR2_PACKAGE_QT5WEBKIT depends on !BR2_STATIC_LIBS depends on BR2_PACKAGE_QT5_JSCORE_AVAILABLE depends on BR2_HOST_GCC_AT_LEAST_4_8 # icu + depends on BR2_INSTALL_LIBSTDCPP # leveldb depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 # icu + depends on BR2_TOOLCHAIN_HAS_THREADS # leveldb depends on !BR2_BINFMT_FLAT # icu # assumes a FPU is available on MIPS depends on !BR2_MIPS_SOFT_FLOAT + select BR2_PACKAGE_LEVELDB + select BR2_PACKAGE_LEVELDB_MEMENV select BR2_PACKAGE_QT5BASE select BR2_PACKAGE_QT5BASE_ICU select BR2_PACKAGE_QT5BASE_GUI
package/qt5/qt5webkit/qt5webkit.mk +2 -2
@@ -16,7 +16,7 @@ endif QT5WEBKIT_SOURCE = qtwebkit-opensource-src-$(QT5WEBKIT_VERSION).tar.xz QT5WEBKIT_DEPENDENCIES = \ host-bison host-flex host-gperf host-python host-ruby \ - qt5base sqlite + leveldb qt5base sqlite QT5WEBKIT_INSTALL_STAGING = YES QT5WEBKIT_LICENSE_FILES = Source/WebCore/LICENSE-LGPL-2 Source/WebCore/LICENSE-LGPL-2.1
@@ -45,7 +45,7 @@ endef QT5WEBKIT_PRE_CONFIGURE_HOOKS += QT5WEBKIT_PYTHON2_SYMLINK define QT5WEBKIT_CONFIGURE_CMDS - (cd $(@D); $(TARGET_MAKE_ENV) $(QT5WEBKIT_ENV) $(HOST_DIR)/bin/qmake) + (cd $(@D); $(TARGET_MAKE_ENV) $(QT5WEBKIT_ENV) $(HOST_DIR)/bin/qmake WEBKIT_CONFIG+=use_system_leveldb) endef define QT5WEBKIT_BUILD_CMDS
Signature · 1 line
2.11.0