libbsd: Fix conflict error when enable multilib.
PATCH v3 1/1 libbsd: Fix conflict error when enable multilib.
From: Lei Maohui <[email protected]>
- Fix the conflict error by oe_multilib_header. The conflict error is as the following:
Error: Transaction test error:
file /usr/include/bsd/sys/cdefs.h conflicts between attempted installs of libbsd-dev-0.12.1-r0.core2_64 and lib32-libbsd-dev-0.12.1-r0.core2_32
The difference of bsd/sys/cdefs.h between libbsd-dev and lib32-libbsd-dev is as following:
/* Define the ABI for the current system. */
-#define LIBBSD_SYS_TIME_BITS 64
+#define LIBBSD_SYS_TIME_BITS 32
#define LIBBSD_SYS_HAS_TIME64 1
- After oe_multilib_header on cdefs.h, the path of cdefs-64.h and cdefs-32.h in cdefs.h need to be corrected. Please reference to https://man.archlinux.org/man/libbsd.7 for details. @@ -12,19 +12,19 @@
#ifdef _MIPS_SIM
#if _MIPS_SIM == _ABIO32
-#include <bsd/sys/cdefs-32.h>
+#include <sys/cdefs-32.h>
#elif _MIPS_SIM == _ABIN32
-#include <bsd/sys/cdefs-n32.h>
+#include <sys/cdefs-n32.h>
#else
#error "Unknown _MIPS_SIM"
#endif
#else /* _MIPS_SIM is not defined */
-#include <bsd/sys/cdefs-32.h>
+#include <sys/cdefs-32.h>
#endif
#elif __WORDSIZE == 64
-#include <bsd/sys/cdefs-64.h>
+#include <sys/cdefs-64.h>
#else
#error "Unknown __WORDSIZE detected"
#endif /* matches #if __WORDSIZE == 32 */
--- meta/recipes-support/libbsd/libbsd_0.12.2.bb | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
meta/recipes-support/libbsd/libbsd_0.12.2.bb +7 -1
@@ -40,8 +40,14 @@ SRC_URI = "https://libbsd.freedesktop.org/releases/${BPN}-${PV}.tar.xz" SRC_URI[sha256sum] = "b88cc9163d0c652aaf39a99991d974ddba1c3a9711db8f1b5838af2a14731014" -inherit autotools pkgconfig +inherit autotools pkgconfig multilib_header DEPENDS += "libmd" +do_install:append () { + oe_multilib_header bsd/sys/cdefs.h + sed -i 's:#include <bsd/sys/cdefs:#include <sys/cdefs:g' ${D}${includedir}/bsd/sys/cdefs.h +} + + BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.34.1
Quoted 1 line · leimaohui via lists.openembedded.org
- After
oe_multilib_headeroncdefs.h, the path ofcdefs-64.handcdefs-32.hincdefs.hneed to be corrected. Please reference to https://man.archlinux.org/man/libbsd.7 for details.
I looked at this in a local build, and I actually don't think they need to be corrected. Without the sed command, there is #include <bsd/sys/cdefs-32.h>, which is fine, as it's relative to /usr/include, and unambigious.
After the sed command, there is #include <sys/cdefs-32.h> which requires special non-default arrangements in the build system, specifically, using libbsd in overlay mode with special compiler flags as described in the man page you refer to. But that's not the only way to use libbsd, and the other way won't work.
So what is the issue that happens if there is no sed, and only oe_multilib_header?
Alex
Hi, Alex
Quoted 4 lines
After the sed command, there is #include <
sys/cdefs-32.h> which requires special non-default arrangements in the build system, specifically, using libbsd in overlay mode with special compiler flags as described in the man page you refer to. But that's not the only way to use libbsd, and the other way won't work.
Reference to the man page, libbsd recommends overlay mode.
"The library can be used in an overlay mode, which is the preferred way, so that the code is portable and requires no modification to the original BSD code......."
From the build log and the source code of libbsd, I found that libbsd does provide header file in overlay mode by default as the man page says.
-----------------------------------------------------------------------
$ ls tmp/work/cortexa57-poky-linux/libbsd/0.12.2/image/usr/lib64/pkgconfig/ libbsd-ctor.pc libbsd-overlay.pc libbsd.pc $ grep include tmp/work/cortexa57-poky-linux/libbsd/0.12.2/temp/log.do_compile -r ...... -I.. -isystem ../../libbsd-0.12.2/include/bsd/...... ...... -I.. -isystem ../../libbsd-0.12.2/include/bsd/...... ...... $ vim tmp/work/cortexa57-poky-linux/libbsd/0.12.2/libbsd-0.12.2/include/bsd/stringlist.h #ifdef LIBBSD_OVERLAY #include <sys/cdefs.h> #else #include <bsd/sys/cdefs.h> #endif #include <sys/types.h> ----------------------------------------------------------------------
And I Investigated several OSS(e.g. shadow, mg, libxdmcp) that depend on libbsd. From the source code of theirs. I found that they does used libbsd in overlay mode by default. Such as shadow:
-----------------------------------------------------------------------
vim tmp/work/cortexa57-poky-linux/shadow/4.15.0/shadow-4.15.0/lib/readpassphrase.h
......
#ifdef LIBBSD_OVERLAY
#include <sys/cdefs.h>
#endif
......
-----------------------------------------------------------------------
The same as mg :
-----------------------------------------------------------------------
$ vim tmp/work/cortexa57-poky-linux/mg/20230501/git/CMakeLists.txt
if(CMAKE_SYSTEM_NAME MATCHES "Linux")
pkg_check_modules (BSD REQUIRED libbsd-overlay)
link_directories (${BSD_LIBRARY_DIRS})
string (REPLACE ";" " " LIBBSD_FLAGS "${BSD_CFLAGS}")
target_link_libraries (mg ${BSD_LIBRARIES})
set (CMAKE_C_FLAGS "-Wall -DREGEX -D_GNU_SOURCE ${LIBBSD_FLAGS} ${NCURSES_FLAGS} -L${NCURSES_LIBRARY_DIRS}")
else()
set (CMAKE_C_FLAGS "-Wall -DREGEX ${LIBBSD_FLAGS} ${NCURSES_FLAGS} -L${NCURSES_LIBRARY_DIRS}")
endif()
-----------------------------------------------------------------------So, I think the header file after oe_multilib_header should be in overlay mode as the other header files in libbsd.
Of course, If added "#ifdef LIBBSD_OVERLAY" into the header file after oe_multilib_header, libbsd will have a better compatibility. What do you think?
Best regards
Lei
Quoted 26 lines
-----Original Message-----
From: Alexander Kanavin <[email protected]>
Sent: Thursday, April 25, 2024 4:08 PM
To: Lei, Maohui <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core][PATCH v3] libbsd: Fix conflict error when enable multilib.Quoted 1 line · leimaohui via lists.openembedded.org
- After
oe_multilib_headeroncdefs.h, the path ofcdefs-64.handcdefs-32.hin
cdefs.hneed to be corrected. Please reference to
https://man.archlinux.org/man/libbsd.7 for details.I looked at this in a local build, and I actually don't think they need to be corrected. Without the sed command, there is #include <
bsd/sys/cdefs-32.h>, which is fine, as it's relative to/usr/include, and unambigious.After the sed command, there is #include <
sys/cdefs-32.h> which requires special non-default arrangements in the build system, specifically, using libbsd in overlay mode with special compiler flags as described in the man page you refer to. But that's not the only way to use libbsd, and the other way won't work.So what is the issue that happens if there is no sed, and only
oe_multilib_header?Alex
Quoted 16 lines · Maohui Lei (Fujitsu)
From the build log and the source code of libbsd, I found that libbsd does provide header file in overlay mode by default as the man page says.
-----------------------------------------------------------------------$ ls tmp/work/cortexa57-poky-linux/libbsd/0.12.2/image/usr/lib64/pkgconfig/ libbsd-ctor.pc libbsd-overlay.pc libbsd.pc $ grep include tmp/work/cortexa57-poky-linux/libbsd/0.12.2/temp/log.do_compile -r ...... -I.. -isystem ../../libbsd-0.12.2/include/bsd/...... ...... -I.. -isystem ../../libbsd-0.12.2/include/bsd/...... ...... $ vim tmp/work/cortexa57-poky-linux/libbsd/0.12.2/libbsd-0.12.2/include/bsd/stringlist.h #ifdef LIBBSD_OVERLAY #include <sys/cdefs.h> #else #include <bsd/sys/cdefs.h> #endif #include <sys/types.h> ----------------------------------------------------------------------
The man page does not say overlay mode is default, it only says that it's recommended. And indeed the above shows that you need to opt into it by defining LIBBSD_OVERLAY. Let's see how example projects handle it.
Quoted 10 lines
And I Investigated several OSS(e.g. shadow, mg, libxdmcp) that depend on libbsd. From the source code of theirs. I found that they does used libbsd in overlay mode by default. Such as shadow:
-----------------------------------------------------------------------
vimtmp/work/cortexa57-poky-linux/shadow/4.15.0/shadow-4.15.0/lib/readpassphrase.h
......
#ifdefLIBBSD_OVERLAY
#include <sys/cdefs.h>
#endif
......
-----------------------------------------------------------------------
shadow does not use overlay mode by default, it makes that decision based on other factors in configure.ac:
if test "$with_libbsd" != "no"; then
AC_SEARCH_LIBS([readpassphrase], [bsd], [], [
AC_MSG_ERROR([readpassphrase() is missing, either from
libc or libbsd])
])
AS_IF([test "$ac_cv_search_readpassphrase" = "-lbsd"], [
PKG_CHECK_MODULES([LIBBSD], [libbsd-overlay])
])
dnl Make sure either the libc or libbsd provide the header.
save_CFLAGS="$CFLAGS"
CFLAGS="$CFLAGS $LIBBSD_CFLAGS"
AC_CHECK_HEADERS([readpassphrase.h])
AS_IF([test "$ac_cv_header_readpassphrase_h" != "yes"], [
AC_MSG_ERROR([readpassphrase.h is missing])
])
CFLAGS="$save_CFLAGS"
AC_DEFINE(WITH_LIBBSD, 1, [Build shadow with libbsd support])else
AC_DEFINE(WITH_LIBBSD, 0, [Build shadow without libbsd support])
fiAM_CONDITIONAL(WITH_LIBBSD, test x$with_libbsd = xyes)
Quoted 13 lines
The same as mg :
-----------------------------------------------------------------------$ vim tmp/work/cortexa57-poky-linux/mg/20230501/git/CMakeLists.txt if(CMAKE_SYSTEM_NAME MATCHES "Linux") pkg_check_modules (BSD REQUIRED libbsd-overlay) link_directories (${BSD_LIBRARY_DIRS}) string (REPLACE ";" " " LIBBSD_FLAGS "${BSD_CFLAGS}") target_link_libraries (mg ${BSD_LIBRARIES}) set (CMAKE_C_FLAGS "-Wall -DREGEX -D_GNU_SOURCE ${LIBBSD_FLAGS} ${NCURSES_FLAGS} -L${NCURSES_LIBRARY_DIRS}") else() set (CMAKE_C_FLAGS "-Wall -DREGEX ${LIBBSD_FLAGS} ${NCURSES_FLAGS} -L${NCURSES_LIBRARY_DIRS}") endif() -----------------------------------------------------------------------
This one does use overlay mode by default, but that doesn't mean every other project does the same. We have to continue supporting both options until libbsd upstream provides only one option.
Quoted 2 lines
So, I think the header file after
oe_multilib_headershould be in overlay mode as the other header files in libbsd.
Of course, If added "#ifdefLIBBSD_OVERLAY" into the header file afteroe_multilib_header, libbsd will have a better compatibility. What do you think?
I don't think this is correct. Without sed, both modes will work (unless you can demonstrate how it will break; I asked that, and there's no answer in the above). With sed, only overlay mode will work, and now you're proposing additional fixing to undo that effect.
So I'd suggest you submit the patch without sed (just oe_multilib parts), and we can then consider if any further fixing is required.
Again: please show what is the problem that would occur without sed.
Alex
Hi, Alex
Quoted 3 lines
So I'd suggest you submit the patch without sed (just
oe_multilibparts), and we can then consider if any further fixing is required.
Again: please show what is the problem that would occur without sed.
I' sorry. But my first V1 version patch without sed (just oe_multilib parts) did not pass Khem Raj's compile test (bitbake mg https://errors.yoctoproject.org/Errors/Details/759473/). I'm not sure if it is the problem you want to know. So, I reproduce the build error and found that the cdefs.h that created by multilib_header has the path "#include <bsd/sys/cdefs-32.h>".
But mg find herder files without namespaced with bsd/. So, it failed. Shadow has the same build issue.
So, I submitted V2 patch with sed.
Quoted 2 lines
With sed, only overlay mode will work, and now you're proposing additional fixing to undo that effect.
Yes, I just tested my patch with sed on two recipes (mg and shadow) . The recipes that depend on libbsd are too few to have more test.
Best regards
Lei
Quoted 111 lines
-----Original Message-----
From: Alexander Kanavin <[email protected]>
Sent: Friday, April 26, 2024 4:40 PM
To: Lei, Maohui <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core][PATCH v3] libbsd: Fix conflict error when enable multilib.Quoted 1 line · Maohui Lei (Fujitsu)
From the build log and the source code of libbsd, I found that libbsd does
provide header file in overlay mode by default as the man page says.
Quoted 8 lines
----------------------------------------------------------------------
- $ ls tmp/work/cortexa57-poky-linux/libbsd/0.12.2/image/usr/lib64/pkgconfig/ libbsd-ctor.pc libbsd-overlay.pc libbsd.pc $ grep include tmp/work/cortexa57-poky-linux/libbsd/0.12.2/temp/log.do_compile -r ...... -I.. -isystem ../../libbsd-0.12.2/include/bsd/...... ...... -I.. -isystem ../../libbsd-0.12.2/include/bsd/......
......Quoted 10 lines
$ vim tmp/work/cortexa57-poky-linux/libbsd/0.12.2/libbsd-0.12.2/include/bsd/ stringlist.h #ifdef LIBBSD_OVERLAY #include <sys/cdefs.h> #else #include <bsd/sys/cdefs.h> #endif #include <sys/types.h> ----------------------------------------------------------------------The man page does not say overlay mode is default, it only says that it's recommended. And indeed the above shows that you need to opt into it by defining
LIBBSD_OVERLAY. Let's see how example projects handle it.Quoted 1 line
And I Investigated several OSS(e.g. shadow, mg, libxdmcp) that depend on
libbsd. From the source code of theirs. I found that they does used libbsd in overlay mode by default.
Quoted 4 lines
Such as shadow:
----------------------------------------------------------------------
- vim tmp/work/cortexa57-poky-linux/shadow/4.15.0/shadow-4.15.0/lib/readpassQuoted 8 lines
phrase.h
......
#ifdefLIBBSD_OVERLAY
#include <sys/cdefs.h>
#endif
......
----------------------------------------------------------------------
-shadow does not use overlay mode by default, it makes that decision based on other factors in configure.ac:
if test "$with_libbsd" != "no"; then
AC_SEARCH_LIBS([readpassphrase], [bsd], [], [
AC_MSG_ERROR([readpassphrase() is missing, either from
libc or libbsd])]) AS_IF([test "$ac_cv_search_readpassphrase" = "-lbsd"], [ PKG_CHECK_MODULES([LIBBSD], [libbsd-overlay]) ]) dnl Make sure either the libc or libbsd provide the header. save_CFLAGS="$CFLAGS" CFLAGS="$CFLAGS $LIBBSD_CFLAGS" AC_CHECK_HEADERS([readpassphrase.h]) AS_IF([test "$ac_cv_header_readpassphrase_h" != "yes"], [ AC_MSG_ERROR([readpassphrase.h is missing]) ]) CFLAGS="$save_CFLAGS" AC_DEFINE(WITH_LIBBSD, 1, [Build shadow with libbsd support]) else AC_DEFINE(WITH_LIBBSD, 0, [Build shadow without libbsd support]) fi
AM_CONDITIONAL(WITH_LIBBSD, test x$with_libbsd = xyes)Quoted 4 lines
The same as mg :
----------------------------------------------------------------------
- $ vimtmp/work/cortexa57-poky-linux/mg/20230501/git/CMakeLists.txtif(CMAKE_SYSTEM_NAME MATCHES "Linux")Quoted 8 lines
pkg_check_modules(BSD REQUIRED libbsd-overlay)link_directories(${BSD_LIBRARY_DIRS})
string (REPLACE ";" " "LIBBSD_FLAGS"${BSD_CFLAGS}")target_link_libraries(mg ${BSD_LIBRARIES})
set (CMAKE_C_FLAGS"-Wall -DREGEX -D_GNU_SOURCE ${LIBBSD_FLAGS}
${NCURSES_FLAGS} -L${NCURSES_LIBRARY_DIRS}")else()
set (CMAKE_C_FLAGS"-Wall -DREGEX ${LIBBSD_FLAGS}${NCURSES_FLAGS}
Quoted 4 lines
-L${NCURSES_LIBRARY_DIRS}")
endif()
----------------------------------------------------------------------
-This one does use overlay mode by default, but that doesn't mean every other project does the same. We have to continue supporting both options until libbsd upstream provides only one option.
Quoted 1 line
So, I think the header file after
oe_multilib_headershould be in overlay modeas the other header files in libbsd.
Quoted 1 line
Of course, If added "#ifdef
LIBBSD_OVERLAY" into the header file after
oe_multilib_header, libbsd will have a better compatibility. What do you think?I don't think this is correct. Without sed, both modes will work (unless you can demonstrate how it will break; I asked that, and there's no answer in the above). With sed, only overlay mode will work, and now you're proposing additional fixing to undo that effect.
So I'd suggest you submit the patch without sed (just
oe_multilibparts), and we can then consider if any further fixing is required.
Again: please show what is the problem that would occur without sed.Alex
Quoted 4 lines · Maohui Lei (Fujitsu)
I' sorry. But my first V1 version patch without sed (just
oe_multilibparts) did not pass Khem Raj's compile test (bitbake mg https://errors.yoctoproject.org/Errors/Details/759473/). I'm not sure if it is the problem you want to know. So, I reproduce the build error and found that thecdefs.hthat created bymultilib_headerhas the path "#include <bsd/sys/cdefs-32.h>".
But mg find herder files without namespaced with bsd/. So, it failed. Shadow has the same build issue.
So, I submitted V2 patch with sed.
Right, this is the error:
| In file included from /srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-mg/20230501/lib64-recipe-sysroot/usr/include/bsd/sys/cdefs.h:27,
| from/srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-mg/20230501/lib64-recipe-sysroot/usr/include/bsd/sys/queue.h:39,
| from region.c:12:
| /srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-mg/20230501/lib64-recipe-sysroot/usr/include/bsd/sys/cdefs-64.h:49:24:
error: no include path in which to search for sys/cdefs.h
| 49 | #if __has_include_next(<sys/cdefs.h>) | | ^~~~~~~~~~~~~
I'm utterly confused by that. Why can't the compiler find usr/include/sys/cdefs.h?
Why is it then happily finding it when bsd/ prefix is dropped from cdefs-64 include statement in usr/include/bsd/sys/cdefs.h ?
But the fix does seem to help, and makes multilib selection header match libbsd headers (in overlay mode).
So:
- the patch is ok; but please add a note (a comment in the recipe just above sed) that it hasn't been tested when libbsd is used in 'namespaced headers' mode according to
https://man.archlinux.org/man/libbsd.7
Alex
Hi, Alex
Quoted 4 lines
- the patch is ok; but please add a note (a comment in the recipe just above sed) that it hasn't been tested when libbsd is used in 'namespaced headers' mode according to
https://man.archlinux.org/man/libbsd.7
OK, I will add this note into recipe too and send V4 patch.
Best regards
Lei
Quoted 52 lines
-----Original Message-----
From: Alexander Kanavin <[email protected]>
Sent: Monday, April 29, 2024 6:46 PM
To: Lei, Maohui <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core][PATCH v3] libbsd: Fix conflict error when enable multilib.Quoted 1 line · Maohui Lei (Fujitsu)
I' sorry. But my first V1 version patch without sed (just
oe_multilibparts)did not pass Khem Raj's compile test (bitbake mg
https://errors.yoctoproject.org/Errors/Details/759473/). I'm not sure if it is the problem you want to know.Quoted 1 line
So, I reproduce the build error and found that the
cdefs.hthat created by
multilib_headerhas the path "#include <bsd/sys/cdefs-32.h>".Quoted 1 line
But mg find herder files without namespaced with bsd/. So, it failed.
Shadow has the same build issue.
Quoted 1 line
So, I submitted V2 patch with sed.
Right, this is the error:
| In file included from
/srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-mg/20230501/lib64-recipe-sysroot/usr/include/bsd/sys/cdefs.h:27,
| from/srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-mg/20230501/lib64-recipe-sysroot/usr/include/bsd/sys/queue.h:39,
| from region.c:12:
|/srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-mg/20230501/lib64-recipe-sysroot/usr/include/bsd/sys/cdefs-64.h:49:
24:
error: no include path in which to search forsys/cdefs.h| 49 | #if __has_include_next(<sys/cdefs.h>) | | ^~~~~~~~~~~~~I'm utterly confused by that. Why can't the compiler find
usr/include/sys/cdefs.h?
Why is it then happily finding it when bsd/ prefix is dropped from cdefs-64 include statement inusr/include/bsd/sys/cdefs.h?But the fix does seem to help, and makes multilib selection header match libbsd headers (in overlay mode).
So:
- the patch is ok; but please add a note (a comment in the recipe just above sed) that it hasn't been tested when libbsd is used in 'namespaced headers' mode according to
https://man.archlinux.org/man/libbsd.7Alex