multilib_header.bbclass: Do not install the stubs for non-multilib case
PATCH 1/1 multilib_header.bbclass: Do not install the stubs for non-multilib case
We mark certain headers which are not usable in multilib case through this function, this however is a unnessary indirection when building distros without multilib feature. Additionally it also addresses a case where we compile baremetal software e.g. bootloaders or BPF binaries e.g. which are a different target effectively but can be built using same toolchain with some flags. This does not work when we always install the stubs since the stubs require a fixed wordsize definition and that may not be true for the BPF cases e.g. it will work fine when multilib is enabled since that would have provided the headers it needed say for 32bit multilib. Currently they go missing for a non-multilib case and we end up with errors like
recipe-sysroot/usr/include/bits/long-double.h:23:10: fatal error: 'bits/long-double-32.h' file not found
| #include <bits/long-double-32.h>
Therefore lets install these stubs only when using multilib in distro
--- meta/classes/multilib_header.bbclass | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
meta/classes/multilib_header.bbclass +3 -1
@@ -6,7 +6,9 @@ inherit siteinfo # all of the ABI variants for that given architecture. # oe_multilib_header() { - + if [ -z "${MULTILIB_VARIANTS}" ]; then + return + fi case ${HOST_OS} in *-musl*) return
Signature · 1 line
2.29.0
Quoted 34 lines · Khem Raj
We mark certain headers which are not usable in multilib case through this function, this however is a unnessary indirection when building distros without multilib feature. Additionally it also addresses a case
where we compile baremetal software e.g. bootloaders or BPF binaries e.g. which are a different target effectively but can be built using same toolchain with some flags. This does not work when we always install the stubs since the stubs require a fixed wordsize definition and that may not be true for the BPF cases e.g. it will work fine when
multilib is enabled since that would have provided the headers it needed
say for 32bit multilib. Currently they go missing for a non-multilib case and we end up with errors like
recipe-sysroot/usr/include/bits/long-double.h:23:10: fatal error:
'bits/long-double-32.h' file
not foundQuoted 1 line
#include <
bits/long-double-32.h>Therefore lets install these stubs only when using multilib in distro
Signed-off-by Khem Raj <[email protected]>--- meta/classes/multilib_header.bbclass | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)meta/classes/multilib_header.bbclass +0
-0
meta/classes/multilib_header.bbclass inherit siteinfo · line 14
Quoted 10 lines
meta/classes/multilib_header.bbclass +3
-1@@ -6,7 +6,9 @@ inherit siteinfo # all of the ABI variants for that given architecture. # oe_multilib_header() { - + if [ -z "${MULTILIB_VARIANTS}" ]; then + return + fi case ${HOST_OS} in *-musl*) return
I have a feeling this was written specifically not to touch that variable so that a multilib build can reuse components from a non- multilib build and share the sstate if you configure it that way. This change would force *all* multilibs to be stores as their own sstate and require their own namespace.
We may therefore have to find a different control for this :/.
Cheers,
Richard
Quoted 43 lines · Richard Purdie
Quoted 42 lines · Khem Raj
We mark certain headers which are not usable in multilib case through this function, this however is a unnessary indirection when building distros without multilib feature. Additionally it also addresses a case
where we compile baremetal software e.g. bootloaders or BPF binaries e.g. which are a different target effectively but can be built using same toolchain with some flags. This does not work when we always install the stubs since the stubs require a fixed wordsize definition and that may not be true for the BPF cases e.g. it will work fine when
multilib is enabled since that would have provided the headers it needed
say for 32bit multilib. Currently they go missing for a non-multilib case and we end up with errors like
recipe-sysroot/usr/include/bits/long-double.h:23:10: fatal error:
'bits/long-double-32.h' file
not foundQuoted 1 line
#include <
bits/long-double-32.h>Therefore lets install these stubs only when using multilib in distro
Signed-off-by Khem Raj <[email protected]>--- meta/classes/multilib_header.bbclass | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)meta/classes/multilib_header.bbclass +3
-1@@ -6,7 +6,9 @@ inherit siteinfo # all of the ABI variants for that given architecture. # oe_multilib_header() { - + if [ -z "${MULTILIB_VARIANTS}" ]; then + return + fi case ${HOST_OS} in
Quoted 8 lines
meta/classes/multilib_header.bbclass inherit siteinfo · line 14
Quoted 2 lines
meta/classes/multilib_header.bbclass +0
-0*-musl*) returnI have a feeling this was written specifically not to touch that variable so that a multilib build can reuse components from a non- multilib build and share the sstate if you configure it that way. This change would force *all* multilibs to be stores as their own sstate and require their own namespace.
In practice either you use multilib or you don’t such a usecase is not that common besides perhaps testing scenarios and maybe some OS vendor test farms which considers all sort of combinations
And in some cases e.g x86-64 BASELIB gets defined differently if you enable multilib or not so I guess we already are not reusing sstate between non multilib and multilib builds for same arch
Quoted 11 lines
We may therefore have to find a different control for this :/.
Cheers,
Richard
Quoted 58 lines · Khem Raj
On Wed, Oct 21, 2020 at 2:47 PM Richard Purdie <[email protected] <mailto:[email protected]>> wrote:
On Wed, 2020-10-21 at 14:09 -0700, Khem Raj wrote:
> We mark certain headers which are not usable in multilib case through > this function, this however is a unnessary indirection when building > distros without multilib feature. Additionally it also addresses a > case
> where we compile baremetal software e.g. bootloaders or BPF binaries > e.g. which are a different target effectively but can be built using > same toolchain with some flags. This does not work when we always > install the stubs since the stubs require a fixed wordsize definition > and that may not be true for the BPF cases e.g. it will work fine > when
> multilib is enabled since that would have provided the headers it > needed
> say for 32bit multilib. Currently they go missing for a non-multilib > case and we end up with errors like
>
>recipe-sysroot/usr/include/bits/long-double.h:23:10: fatal error:
> 'bits/long-double-32.h' file
> not found
> > #include <bits/long-double-32.h>
>
> Therefore lets install these stubs only when using
> multilib in distro
>
> Signed-off-by: Khem Raj <[email protected]
<mailto:[email protected]>>
> ---
> meta/classes/multilib_header.bbclass | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/meta/classes/multilib_header.bbclass
> b/meta/classes/multilib_header.bbclass
> indexe03f5b13b2..9ccdf46be5100644
> --- a/meta/classes/multilib_header.bbclass
> +++ b/meta/classes/multilib_header.bbclass
> @@ -6,7 +6,9 @@ inherit siteinfo
> # all of the ABI variants for that given architecture.
> #
>oe_multilib_header(){
> -
> + if [ -z "${MULTILIB_VARIANTS}" ]; then
> + return
> + fi
> case ${HOST_OS} in
> *-musl*)
> returnI have a feeling this was written specifically not to touch that variable so that a multilib build can reuse components from a non- multilib build and share the sstate if you configure it that way. This change would force *all* multilibs to be stores as their own sstate and require their own namespace.
Quoted 5 lines
In practice either you use multilib or you don’t such a usecase is not that common besides perhaps testing scenarios and maybe some OS vendor test farms which considers all sort of combinations
And in some cases e.g x86-64 BASELIB gets defined differently if you enable multilib or not so I guess we already are not reusing sstate between non multilib and multilib builds for same arch
I ran a few combination builds without this patch applied.
build core-image-minimal/qemux86-64
then add just following to local.conf
require conf/multilib.conf
MULTILIBS = "multilib:lib32"
DEFAULTTUNE_virtclass-multilib-lib32 = "x86"
and rebuild core-image-minimal
here is buildhistory-diff output ( before and after adding multilib to local.conf)
It ends up rebuilding all target packages inclduing some cross packages e.g. libtool-cross, native sstate it still re-used and that remains so with above patch applied as well.
Therefore this patch does not make things any different than they are currently.
Let me know if I should be checking more things.
Quoted 9 lines
We may therefore have to find a different control for this :/.
Cheers,
Richard
Quoted 17 lines · Khem Raj
I ran a few combination builds without this patch applied.
build core-image-minimal/qemux86-64
then add just following to local.conf
require
conf/multilib.conf
MULTILIBS = "multilib:lib32"
DEFAULTTUNE_virtclass-multilib-lib32 = "x86"and rebuild core-image-minimal
here is buildhistory-diff output ( before and after adding multilib to local.conf)
Quoted 8 lines
It ends up rebuilding all target packages inclduing some cross packages e.g. libtool-cross, native sstate it still re-used and that remains so with above patch applied as well.
Therefore this patch does not make things any different than they are currently.
Let me know if I should be checking more things.
Imagine you configure 64 bit x86 to use /lib64 in all situations (which many do now, it probably should be the default).
Then rerun that test. I doubt core2-64 will rebuild in that case but it will after your patch.
Cheers,
Richard
Quoted 22 lines · Richard Purdie
Quoted 21 lines · Khem Raj
I ran a few combination builds without this patch applied.
build core-image-minimal/qemux86-64
then add just following to local.conf
require
conf/multilib.conf
MULTILIBS = "multilib:lib32"
DEFAULTTUNE_virtclass-multilib-lib32 = "x86"and rebuild core-image-minimal
here is buildhistory-diff output ( before and after adding multilib to local.conf)
It ends up rebuilding all target packages inclduing some cross packages e.g. libtool-cross, native sstate it still re-used and that remains so with above patch applied as well.
Quoted 7 lines
Quoted 4 lines
Therefore this patch does not make things any different than they are currently.
Let me know if I should be checking more things.
Imagine you configure 64 bit x86 to use /lib64 in all situations (which many do now, it probably should be the default).
interesting but its not default, and multilib is not default as well. Thirdly we are optimizing for a case where we want to switch distros between multilib and non-multilib case. are there practical usecases of these three conditions? I think we are optimizing for a relatively small usecase here if there are any.
Quoted 8 lines
Then rerun that test. I doubt core2-64 will rebuild in that case but it will after your patch.
Cheers,
Richard
Quoted 31 lines · Khem Raj
Quoted 29 lines · Richard Purdie
Quoted 28 lines · Khem Raj
I ran a few combination builds without this patch applied.
build core-image-minimal/qemux86-64
then add just following to local.conf
require
conf/multilib.conf
MULTILIBS = "multilib:lib32"
DEFAULTTUNE_virtclass-multilib-lib32 = "x86"and rebuild core-image-minimal
here is buildhistory-diff output ( before and after adding multilib to
local.conf)It ends up rebuilding all target packages inclduing some cross packages
e.g. libtool-cross, native sstate it still re-used and that remains so
with above patch applied as well.Therefore this patch does not make things any different than they are
currently.
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
Let me know if I should be checking more things.
Imagine you configure 64 bit x86 to use /lib64 in all situations (which
many do now, it probably should be the default).interesting but its not default, and multilib is not default as well.
I thought it was the default on other distros? We only have it as the default for nodistro and poky for historical reasons and I've not wanted the pain of switching.
Quoted 2 lines
Thirdly we are optimizing for a case where we want to switch distros between multilib and non-multilib case.
Ok, in 3.3, lets propose switching the default on 64 bit x86 and that in turn should let us optimise the autobuilder load to cut out a lot of these currently basically duplicated builds? If its not wanted for nodistro, I can propose we change poky.
Quoted 2 lines
are there practical usecases of these three conditions? I think we are optimizing for a relatively small usecase here if there are any.
See above, we can at the very least optimise the autobuilder. I'd actually thought it should be reusing now but it clearly isn't. We should fix that.
I do think its a sensible optimization to have working. I know it means we need to be a bit more careful with that patch.
Cheers,
Richard