boost: fix static detection of lock-free atomic ints
PATCH v2 1/1 boost: fix static detection of lock-free atomic ints
When build statically, boost is unable to detect lock-free atomics ints because it tries to link dynamically, see output/build/boost-1.67.0/bin.v2/config.log (with -d5 option):
Using shell: /bin/sh -c
argv[0] = '/bin/sh'
argv[1] = '-c'
argv[2] = '
"/home/fabrice/buildroot/output/host/bin/arm-linux-g++" -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -Wl,-elf2flt -static -Wl,-elf2flt -static -fPIC -pthread -O0 -fno-inline -Wall -pedantic -g -Wextra -Wno-long-long -Wno-unused-parameter -Wunused-function -pedantic -DBOOST_ALL_NO_LIB=1 -DBOOST_SYSTEM_DYN_LINK=1 -DBOOST_THREAD_BUILD_DLL=1 -I"." -c -o "bin.v2/libs/thread/build/gcc-6.4.0/debug/threading-multi/has_atomic_flag_lockfree_test.o" "libs/thread/src/../build/has_atomic_flag_lockfree_test.cpp"'
gcc.compile.c++ bin.v2/libs/thread/build/gcc-6.4.0/debug/threading-multi/has_atomic_flag_lockfree_test.o
"/home/fabrice/buildroot/output/host/bin/arm-linux-g++" -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -Wl,-elf2flt -static -Wl,-elf2flt -static -fPIC -pthread -O0 -fno-inline -Wall -pedantic -g -Wextra -Wno-long-long -Wno-unused-parameter -Wunused-function -pedantic -DBOOST_ALL_NO_LIB=1 -DBOOST_SYSTEM_DYN_LINK=1 -DBOOST_THREAD_BUILD_DLL=1 -I"." -c -o "bin.v2/libs/thread/build/gcc-6.4.0/debug/threading-multi/has_atomic_flag_lockfree_test.o" "libs/thread/src/../build/has_atomic_flag_lockfree_test.cpp"
0.033561 sec system; 0.126314 sec user; 288.682473 sec clock gcc.compile.c++ bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/error_code.o
"/home/fabrice/buildroot/output/host/bin/arm-linux-g++" -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -Wl,-elf2flt -static -Wl,-elf2flt -static -fPIC -pthread -O0 -fno-inline -Wall -pedantic -g -DBOOST_ALL_NO_LIB=1 -DBOOST_SYSTEM_DYN_LINK=1 -I"." -c -o "bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/error_code.o" "libs/system/src/error_code.cpp"
0.084060 sec system; 0.644133 sec user; 8.858824 sec clock
SEM: <s>gcc-link-semaphore now used by <pbin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi>libboost_system.so.1.67.0 Using shell: /bin/sh -c
argv[0] = '/bin/sh'
argv[1] = '-c'
argv[2] = '
"/home/fabrice/buildroot/output/host/bin/arm-linux-g++" -o "bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/libboost_system.so.1.67.0" -Wl,-h -Wl,libboost_system.so.1.67.0 -shared -Wl,--start-group "bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/error_code.o" -Wl,-Bstatic -Wl,-Bdynamic -lrt -Wl,--end-group -fPIC -pthread -g -Wl,-elf2flt -static'
gcc.link.dll bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/libboost_system.so.1.67.0
"/home/fabrice/buildroot/output/host/bin/arm-linux-g++" -o "bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/libboost_system.so.1.67.0" -Wl,-h -Wl,libboost_system.so.1.67.0 -shared -Wl,--start-group "bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/error_code.o" -Wl,-Bstatic -Wl,-Bdynamic -lrt -Wl,--end-group -fPIC -pthread -g -Wl,-elf2flt -static
ld (ld-elf2flt): -shared used without passing a shared library ID
collect2: error: ld a retourn? le statut de sortie 1
0.003123 sec system; 0.004732 sec user; 15.646509 sec clock
...failed gcc.link.dll bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/libboost_system.so.1.67.0...
Due to this wrong detection, boost_atomic is wrongly selected by boost_thread which cause a build failure on missing select
Add a hack to fix this behavior in libs/thread/build/Jamfile.v2 as I'm not sufficiently knowledgeable in bjam to know why the configuration is not correctly passed
Fixes:
- http://autobuild.buildroot.org/results/f46d38991385cbc2a4fa14eb31074e770cd79803
---
Changes v1 -> v2 (after review of Arnout Vandecappelle):
- Fix URL of build failure
package/boost/boost.mk | 10 ++++++++++ 1 file changed, 10 insertions(+)
package/boost/boost.mk +10 -0
@@ -106,6 +106,16 @@ endif # variants. Override that if we want static only or shared only. ifeq ($(BR2_STATIC_LIBS),y) BOOST_OPTS += link=static runtime-link=static + +# When detecting features such as lockfree atomic ints, boost does not take into +# account the configuration provided by the user. Indeed, boost does not specify +# any value and as a result the default value (shared) is used +define BOOST_FIX_STATIC_FEATURES_DETECTION +$(SED) 's/has_atomic_flag_lockfree_test.cpp/has_atomic_flag_lockfree_test.cpp : <link>static <runtime-link>static/g;' \ + $(@D)/libs/thread/build/Jamfile.v2 +endef + +BOOST_PRE_CONFIGURE_HOOKS += BOOST_FIX_STATIC_FEATURES_DETECTION else ifeq ($(BR2_SHARED_LIBS),y) BOOST_OPTS += link=shared runtime-link=shared endif
Signature · 1 line
2.17.1
PATCH v2 1/1 boost: fix static detection of lock-free atomic ints
Hello,
Thanks for investigating this issue, I know what it takes to dive into the Boost build system, however, I am a bit confused by what's happening.
Quoted 5 lines · Fabrice Fontaine
When build statically, boost is unable to detect lock-free atomics ints because it tries to link dynamically, see output/build/boost-1.67.0/bin.v2/config.log (with -d5 option):
Using shell:
/bin/sh-c
Quoted 8 lines
argv[0] = '/bin/sh' argv[1] = '-c' argv[2] = ' "/home/fabrice/buildroot/output/host/bin/arm-linux-g++" -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -Wl,-elf2flt -static -Wl,-elf2flt -static -fPIC -pthread -O0 -fno-inline -Wall -pedantic -g -Wextra -Wno-long-long -Wno-unused-parameter -Wunused-function -pedantic -DBOOST_ALL_NO_LIB=1 -DBOOST_SYSTEM_DYN_LINK=1 -DBOOST_THREAD_BUILD_DLL=1 -I"." -c -o "bin.v2/libs/thread/build/gcc-6.4.0/debug/threading-multi/has_atomic_flag_lockfree_test.o" "libs/thread/src/../build/has_atomic_flag_lockfree_test.cpp"'
gcc.compile.c++bin.v2/libs/thread/build/gcc-6.4.0/debug/threading-multi/has_atomic_flag_lockfree_test.o"
/home/fabrice/buildroot/output/host/bin/arm-linux-g++" -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -Wl,-elf2flt -static -Wl,-elf2flt -static -fPIC -pthread -O0 -fno-inline -Wall -pedantic -g -Wextra -Wno-long-long -Wno-unused-parameter -Wunused-function -pedantic -DBOOST_ALL_NO_LIB=1 -DBOOST_SYSTEM_DYN_LINK=1 -DBOOST_THREAD_BUILD_DLL=1 -I"." -c -o "bin.v2/libs/thread/build/gcc-6.4.0/debug/threading-multi/has_atomic_flag_lockfree_test.o" "libs/thread/src/../build/has_atomic_flag_lockfree_test.cpp"
So here, it is compiling (no linking) the has_atomic_flag_lockfree_test.cpp test file into its corresponding object file. It does not use -shared at this point.
Quoted 4 lines
0.033561 sec system; 0.126314 sec user; 288.682473 sec clock gcc.compile.c++
bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/error_code.o"
/home/fabrice/buildroot/output/host/bin/arm-linux-g++" -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -Wl,-elf2flt -static -Wl,-elf2flt -static -fPIC -pthread -O0 -fno-inline -Wall -pedantic -g -DBOOST_ALL_NO_LIB=1 -DBOOST_SYSTEM_DYN_LINK=1 -I"." -c -o "bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/error_code.o" "libs/system/src/error_code.cpp"
Here, it also compiles error_code.cpp into error_code.o, which I'm not sure how it is related to the atomic stuff.
Quoted 7 lines
0.084060 sec system; 0.644133 sec user; 8.858824 sec clock
SEM: <s>gcc-link-semaphore now used by <pbin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi>libboost_system.so.1.67.0 Using shell:/bin/sh-cargv[0] = '/bin/sh' argv[1] = '-c' argv[2] = ' "/home/fabrice/buildroot/output/host/bin/arm-linux-g++" -o "bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/libboost_system.so.1.67.0" -Wl,-h -Wl,libboost_system.so.1.67.0 -shared -Wl,--start-group "bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/error_code.o" -Wl,-Bstatic -Wl,-Bdynamic -lrt -Wl,--end-group -fPIC -pthread -g -Wl,-elf2flt -static
And here it tries to produce a shared library, libboost_system.so.1.67.0, which just the error_code.o object file in it ?
Questions:
- Why is Boost trying to _produce_ a shared library ?
- How is producing a shared library from the error_code.o object file related to testing the availability of lock-free atomics ?
package/boost/boost.mk endif · line 116
Quoted 7 lines
package/boost/boost.mk +7
-0+# When detecting features such as lockfree atomic ints, boost does not take into +# account the configuration provided by the user. Indeed, boost does not specify +# any value and as a result the default value (shared) is used +define BOOST_FIX_STATIC_FEATURES_DETECTION +$(SED) 's/has_atomic_flag_lockfree_test.cpp/has_atomic_flag_lockfree_test.cpp : <link>static <runtime-link>static/g;' \ + $(@D)/libs/thread/build/Jamfile.v2 +endef
Have you reported the issue upstream to see what they have to say ?
Thanks,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH v2 1/1 boost: fix static detection of lock-free atomic ints
Dear Thomas,
Le mer. 5 sept. 2018 ? 09:19, Thomas Petazzoni <[email protected]> a ?crit :
Quoted 70 lines
Hello,
Thanks for investigating this issue, I know what it takes to dive into the Boost build system, however, I am a bit confused by what's happening.
Quoted 9 lines · Fabrice Fontaine
When build statically, boost is unable to detect lock-free atomics ints because it tries to link dynamically, see output/build/boost-1.67.0/bin.v2/config.log (with -d5 option):
Using shell:
/bin/sh-cargv[0] = '/bin/sh' argv[1] = '-c' argv[2] = ' "/home/fabrice/buildroot/output/host/bin/arm-linux-g++"-D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -Wl,-elf2flt -static -Wl,-elf2flt -static -fPIC -pthread -O0 -fno-inline
(quoted diff) +0
-3-Wall -pedantic -g -Wextra -Wno-long-long -Wno-unused-parameter -Wunused-function -pedantic -DBOOST_ALL_NO_LIB=1 -DBOOST_SYSTEM_DYN_LINK=1 -DBOOST_THREAD_BUILD_DLL=1 -I"." -c -o"
bin.v2/libs/thread/build/gcc-6.4.0/debug/threading-multi/has_atomic_flag_lockfree_test.o" "libs/thread/src/../build/has_atomic_flag_lockfree_test.cpp"Quoted 2 lines
'
gcc.compile.c++
bin.v2/libs/thread/build/gcc-6.4.0/debug/threading-multi/has_atomic_flag_lockfree_test.oQuoted 2 lines
"
/home/fabrice/buildroot/output/host/bin/arm-linux-g++"-D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -Wl,-elf2flt -static -Wl,-elf2flt -static -fPIC -pthread -O0 -fno-inline
(quoted diff) +0
-3-Wall -pedantic -g -Wextra -Wno-long-long -Wno-unused-parameter -Wunused-function -pedantic -DBOOST_ALL_NO_LIB=1 -DBOOST_SYSTEM_DYN_LINK=1 -DBOOST_THREAD_BUILD_DLL=1 -I"." -c -o"
bin.v2/libs/thread/build/gcc-6.4.0/debug/threading-multi/has_atomic_flag_lockfree_test.o" "libs/thread/src/../build/has_atomic_flag_lockfree_test.cpp"So here, it is compiling (no linking) the has_atomic_flag_lockfree_test.cpp test file into its corresponding object file. It does not use -shared at this point.
Quoted 2 lines
0.033561 sec system; 0.126314 sec user; 288.682473 sec clock gcc.compile.c++
bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/error_code.oQuoted 2 lines
"
/home/fabrice/buildroot/output/host/bin/arm-linux-g++"-D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -Wl,-elf2flt -static -Wl,-elf2flt -static -fPIC -pthread -O0 -fno-inline
(quoted diff) +0
-2-Wall -pedantic -g -DBOOST_ALL_NO_LIB=1 -DBOOST_SYSTEM_DYN_LINK=1 -I"." -c -o"
bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/error_code.o" "libs/system/src/error_code.cpp"Here, it also compiles error_code.cpp into error_code.o, which I'm not sure how it is related to the atomic stuff.
Quoted 2 lines
0.084060 sec system; 0.644133 sec user; 8.858824 sec clock
SEM: <s>gcc-link-semaphore now used by<pbin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi>libboost_system.so.1.67.0
Quoted 5 lines
Using shell:
/bin/sh-cargv[0] = '/bin/sh' argv[1] = '-c' argv[2] = ' "/home/fabrice/buildroot/output/host/bin/arm-linux-g++" -o"bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/libboost_system.so.1.67.0" -Wl,-h -Wl,libboost_system.so.1.67.0 -shared -Wl,--start-group "
bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/error_code.o"(quoted diff) +0
-2-Wl,-Bstatic -Wl,-Bdynamic -lrt -Wl,--end-group -fPIC -pthread -g -Wl,-elf2flt -static
Quoted 9 lines
And here it tries to produce a shared library, libboost_system.so.1.67.0, which just the error_code.o object file in it ?
Questions:
- Why is Boost trying to _produce_ a shared library ?
From my understanding and thanks to your helpfull comments, I think I understand better the issue.
The exe directive is after the "project boost/thread" statement in libs/thread/build/Jamfile.v2, so bjam assumes that boost_system is needed and tries to build it even if I don't know why it uses the default shared linking mode (perhaps because the exe statement is executed before boost/thread retrieve the user configuration).
So I tried to move the exe directive in the beginning of the file as boost_system is not needed to check atomics and it seems to work, I will send a v3 and get the feedback from the upstream project.
Quoted 18 lines
- How is producing a shared library from the error_code.o object file related to testing the availability of lock-free atomics ?
Quoted 1 line
+# When detecting features such as lockfree atomic ints, boost does not
take into
Quoted 1 line
+# account the configuration provided by the user. Indeed, boost does
not specify
package/boost/boost.mk endif
Quoted 3 lines
package/boost/boost.mk +3
-0+# any value and as a result the default value (shared) is used +define BOOST_FIX_STATIC_FEATURES_DETECTION +$(SED)'s/has_atomic_flag_lockfree_test.cpp/has_atomic_flag_lockfree_test.cpp :
<link>static <runtime-link>static/g;' \package/boost/boost.mk endif · line 116
Quoted 2 lines
package/boost/boost.mk +2
-0+ $(@D)/libs/thread/build/Jamfile.v2 +endefHave you reported the issue upstream to see what they have to say ?
Quoted 7 lines
Thanks,
Thomas
--
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
Best Regards,
Fabrice
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20180905/214695ba/attachment.html>