package/pkg-meson: ensure the global cross-compilation.conf file is correct
PATCH 1/1 package/pkg-meson: ensure the global cross-compilation.conf file is correct
Currently, the cross-compilation.conf installed in $(HOST_DIR)/etc/meson/cross-compilation.conf for use by the SDK is generated in a post-install-staging hook of the toolchain package.
With per-package directory support enabled, this means that the generated cross-compilation.conf contains references to the per-package directory of the toolchain/ package, which is not want we want:
[binaries]
c = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-gcc' cpp = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-g++' ar = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-ar' strip = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-strip' pkgconfig = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/usr/bin/pkg-config'
So instead, we generate this file in TOOLCHAIN_TARGET_FINALIZE_HOOKS, so that the global paths are used:
[binaries]
c = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-gcc' cpp = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-g++' ar = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-ar' strip = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-strip' pkgconfig = '/home/thomas/projets/buildroot/output/host/usr/bin/pkg-config'
--- package/pkg-meson.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
package/pkg-meson.mk +1 -1
@@ -205,4 +205,4 @@ define PKG_MESON_INSTALL_CROSS_CONF > $(HOST_DIR)/etc/meson/cross-compilation.conf endef -TOOLCHAIN_POST_INSTALL_STAGING_HOOKS += PKG_MESON_INSTALL_CROSS_CONF +TOOLCHAIN_TARGET_FINALIZE_HOOKS += PKG_MESON_INSTALL_CROSS_CONF
Signature · 1 line
2.23.0
PATCH 1/1 package/pkg-meson: ensure the global cross-compilation.conf file is correct
Hello Thomas,
Quoted 8 lines · Thomas Petazzoni
Currently, the cross-compilation.conf installed in $(HOST_DIR)/etc/meson/cross-compilation.conf for use by the SDK is generated in a post-install-staging hook of the toolchain package.
With per-package directory support enabled, this means that the generated cross-compilation.conf contains references to the per-package directory of the toolchain/ package, which is not want we want:
Nitpick: s/not want we want/not what we want/
Regards,
Peter
Quoted 33 lines
[binaries]
c = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-gcc' cpp = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-g++' ar = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-ar' strip = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-strip' pkgconfig = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/usr/bin/pkg-config'So instead, we generate this file in
TOOLCHAIN_TARGET_FINALIZE_HOOKS, so that the global paths are used:[binaries]
c = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-gcc' cpp = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-g++' ar = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-ar' strip = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-strip' pkgconfig = '/home/thomas/projets/buildroot/output/host/usr/bin/pkg-config'Signed-off-by Thomas Petazzoni <[email protected]>--- package/pkg-meson.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)package/pkg-meson.mk +1
-1@@ -205,4 +205,4 @@ define PKG_MESON_INSTALL_CROSS_CONF > $(HOST_DIR)/etc/meson/cross-compilation.conf endef -TOOLCHAIN_POST_INSTALL_STAGING_HOOKS += PKG_MESON_INSTALL_CROSS_CONF +TOOLCHAIN_TARGET_FINALIZE_HOOKS += PKG_MESON_INSTALL_CROSS_CONF
PATCH 1/1 package/pkg-meson: ensure the global cross-compilation.conf file is correct
In reply to Thomas PetazzoniQuoted 36 lines · Thomas Petazzoni
Currently, the cross-compilation.conf installed in $(HOST_DIR)/etc/meson/cross-compilation.conf for use by the SDK is generated in a post-install-staging hook of the toolchain package.
With per-package directory support enabled, this means that the generated cross-compilation.conf contains references to the per-package directory of the toolchain/ package, which is not want we want:
[binaries]
c = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-gcc' cpp = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-g++' ar = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-ar' strip = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-strip' pkgconfig = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/usr/bin/pkg-config'So instead, we generate this file in
TOOLCHAIN_TARGET_FINALIZE_HOOKS, so that the global paths are used:[binaries]
c = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-gcc' cpp = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-g++' ar = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-ar' strip = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-strip' pkgconfig = '/home/thomas/projets/buildroot/output/host/usr/bin/pkg-config'Signed-off-by Thomas Petazzoni <[email protected]>--- package/pkg-meson.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)package/pkg-meson.mk +0
-0
package/pkg-meson.mk define PKG_MESON_INSTALL_CROSS_CONF · line 208
Quoted 5 lines
package/pkg-meson.mk +1
-1@@ -205,4 +205,4 @@ define PKG_MESON_INSTALL_CROSS_CONF > $(HOST_DIR)/etc/meson/cross-compilation.conf endef -TOOLCHAIN_POST_INSTALL_STAGING_HOOKS += PKG_MESON_INSTALL_CROSS_CONF +TOOLCHAIN_TARGET_FINALIZE_HOOKS += PKG_MESON_INSTALL_CROSS_CONF
Note that this essentially reverts [1].
The case for that patch was not terribly strong, so it should be OK. But then
again, the case for this patch is not that strong either.
Note that pkg-cmake has the same issue. And a number of other packages as well,
e.g. the Python sysconfig refers to the Python per-package dirs.
If we really want to do it fully correctly, we'd need to run something like therelocate-sdk.sh on every per-package copy...
Regards,
Arnout
[1]
https://git.buildroot.org/buildroot/commit/?id=ccc9e05990a44d8db5825c07ed61d405ebb40caa
PATCH 1/1 package/pkg-meson: ensure the global cross-compilation.conf file is correct
Hello,
Quoted 4 lines · On Thu, 5 Dec 2019 23:13:42 +0100 Arnout Vandecappelle <[email protected]> wrote:
package/pkg-meson.mk define PKG_MESON_INSTALL_CROSS_CONF · line 208
Quoted 2 lines
package/pkg-meson.mk +1
-1-TOOLCHAIN_POST_INSTALL_STAGING_HOOKS += PKG_MESON_INSTALL_CROSS_CONF +TOOLCHAIN_TARGET_FINALIZE_HOOKS += PKG_MESON_INSTALL_CROSS_CONFNote that this essentially reverts [1].
Yes, true.
Quoted 2 lines
The case for that patch was not terribly strong, so it should be OK. But then
again, the case for this patch is not that strong either.
Well, the case for my patch is that it allows to produce a meson cross-compilation.conf that has correct and usable values at the end of the build.
Quoted 2 lines
Note that pkg-cmake has the same issue. And a number of other packages as well,
e.g. the Python sysconfig refers to the Python per-package dirs.
For sysconfig, I have a pending patch that I will submit soon to fix the problem. It is in fact worse for sysconfig because if you don't tweak it during the build, the build simply fails. While for the global meson cross-compilation.conf, it is less dramatic as it is not used by Buildroot itself, just provided as a convenience.
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 1/1 package/pkg-meson: ensure the global cross-compilation.conf file is correct
Quoted 10 lines · Thomas Petazzoni
Hello,
Quoted 4 lines · On Thu, 5 Dec 2019 23:13:42 +0100 Arnout Vandecappelle <[email protected]> wrote:
package/pkg-meson.mk define PKG_MESON_INSTALL_CROSS_CONF · line 208
Quoted 2 lines
package/pkg-meson.mk +1
-1-TOOLCHAIN_POST_INSTALL_STAGING_HOOKS += PKG_MESON_INSTALL_CROSS_CONF +TOOLCHAIN_TARGET_FINALIZE_HOOKS += PKG_MESON_INSTALL_CROSS_CONFNote that this essentially reverts [1].
Quoted 10 lines
Yes, true.
Quoted 2 lines
The case for that patch was not terribly strong, so it should be OK. But then
again, the case for this patch is not that strong either.Well, the case for my patch is that it allows to produce a meson cross-compilation.conf that has correct and usable values at the end of the build.
Quoted 1 line
Note that pkg-cmake has the same issue.
Now that I'm awake again (<grin>) I see that it actually doesn't, because that
one uses relative paths. I don't suppose that meson's cross-compilation.conf can use a similar trick? (Adding meson "experts" in Cc...)
Quoted 8 lines
Quoted 2 lines
And a number of other packages as well,
e.g. the Python sysconfig refers to the Python per-package dirs.For sysconfig, I have a pending patch that I will submit soon to fix the problem. It is in fact worse for sysconfig because if you don't tweak it during the build, the build simply fails. While for the global meson cross-compilation.conf, it is less dramatic as it is not used by Buildroot itself, just provided as a convenience.
Just to be entirely clear: it actually works as long as you keep the build
directory around, it just doesn't work in the context of the SDK, right?
Regards,
Arnout
PATCH 1/1 package/pkg-meson: ensure the global cross-compilation.conf file is correct
Quoted 11 lines · On Fri, 6 Dec 2019 09:58:15 +0100 Arnout Vandecappelle <[email protected]> wrote:
Quoted 8 lines
Quoted 2 lines
And a number of other packages as well,
e.g. the Python sysconfig refers to the Python per-package dirs.For sysconfig, I have a pending patch that I will submit soon to fix the problem. It is in fact worse for sysconfig because if you don't tweak it during the build, the build simply fails. While for the global meson cross-compilation.conf, it is less dramatic as it is not used by Buildroot itself, just provided as a convenience.
Just to be entirely clear: it actually works as long as you keep the build
directory around, it just doesn't work in the context of the SDK, right?
A final pass in staging-finalize will be needed indeed to fixup the sysconfigdata for the SDK.
I think I tried to come up with a solution that involved using relative paths, but didn't get it to work. But it's been a while I looked into this, so I don't remember exactly.
Best regards,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 1/1 package/pkg-meson: ensure the global cross-compilation.conf file is correct
In reply to Arnout VandecappelleThanks Arnout for looping me in.
Needless to say, I am opposed to moving the meson file back from the target-finalize step :-)
I think most of the reasoning was already mentioned in that commit, but it seems I did not explain that we actually have HOST_DIR mounted read-only on subsequent 'make' commands after the initial make, exactly to verify that no-one is changing directories they shouldn't be changing. After all, a 'make' after the initial make will only normally do the target-finalize step.
Can this problem of incorrect paths not be handled in another way, e.g. by explicitly requesting the 'final' path variables outside the per-package staging, or by doing a fixup afterwards (but not in the scope of target-finalize), or by only creating the cross-compilation.conf file if it does not already exist?
Thanks,
Thomas
El jue., 5 dic. 2019 a las 23:13, Arnout Vandecappelle (<[email protected]>) escribi?:
Quoted 63 lines · Thomas Petazzoni
Quoted 36 lines
Currently, the cross-compilation.conf installed in $(HOST_DIR)/etc/meson/cross-compilation.conf for use by the SDK is generated in a post-install-staging hook of the toolchain package.
With per-package directory support enabled, this means that the generated cross-compilation.conf contains references to the per-package directory of the toolchain/ package, which is not want we want:
[binaries]
c = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-gcc' cpp = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-g++' ar = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-ar' strip = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-strip' pkgconfig = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/usr/bin/pkg-config'So instead, we generate this file in
TOOLCHAIN_TARGET_FINALIZE_HOOKS, so that the global paths are used:[binaries]
c = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-gcc' cpp = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-g++' ar = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-ar' strip = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-strip' pkgconfig = '/home/thomas/projets/buildroot/output/host/usr/bin/pkg-config'Signed-off-by Thomas Petazzoni <[email protected]>--- package/pkg-meson.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)package/pkg-meson.mk +0
-0package/pkg-meson.mk define PKG_MESON_INSTALL_CROSS_CONF · line 208
Quoted 5 lines
package/pkg-meson.mk +1
-1@@ -205,4 +205,4 @@ define PKG_MESON_INSTALL_CROSS_CONF > $(HOST_DIR)/etc/meson/cross-compilation.conf endef -TOOLCHAIN_POST_INSTALL_STAGING_HOOKS += PKG_MESON_INSTALL_CROSS_CONF +TOOLCHAIN_TARGET_FINALIZE_HOOKS += PKG_MESON_INSTALL_CROSS_CONFNote that this essentially reverts [1].
The case for that patch was not terribly strong, so it should be OK. But then
again, the case for this patch is not that strong either.Note that pkg-cmake has the same issue. And a number of other packages as well,
e.g. the Python sysconfig refers to the Python per-package dirs.If we really want to do it fully correctly, we'd need to run something like the
relocate-sdk.shon every per-package copy...Regards,
Arnout[1]
https://git.buildroot.org/buildroot/commit/?id=ccc9e05990a44d8db5825c07ed61d405ebb40caa
PATCH 1/1 package/pkg-meson: ensure the global cross-compilation.conf file is correct
[oops, sorry for the top-posting]
El vie., 6 dic. 2019 a las 10:54, Thomas De Schampheleire (<[email protected]>) escribi?:
Quoted 86 lines
Thanks Arnout for looping me in.
Needless to say, I am opposed to moving the meson file back from the target-finalize step :-)
I think most of the reasoning was already mentioned in that commit, but it seems I did not explain that we actually haveHOST_DIRmounted read-only on subsequent 'make' commands after the initial make, exactly to verify that no-one is changing directories they shouldn't be changing. After all, a 'make' after the initial make will only normally do the target-finalize step.Can this problem of incorrect paths not be handled in another way, e.g. by explicitly requesting the 'final' path variables outside the per-package staging, or by doing a fixup afterwards (but not in the scope of target-finalize), or by only creating the cross-compilation.conf file if it does not already exist?
Thanks,
ThomasEl jue., 5 dic. 2019 a las 23:13, Arnout Vandecappelle (<[email protected]>) escribi?:
Quoted 63 lines · Thomas Petazzoni
Quoted 36 lines
Currently, the cross-compilation.conf installed in $(HOST_DIR)/etc/meson/cross-compilation.conf for use by the SDK is generated in a post-install-staging hook of the toolchain package.
With per-package directory support enabled, this means that the generated cross-compilation.conf contains references to the per-package directory of the toolchain/ package, which is not want we want:
[binaries]
c = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-gcc' cpp = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-g++' ar = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-ar' strip = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-strip' pkgconfig = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/usr/bin/pkg-config'So instead, we generate this file in
TOOLCHAIN_TARGET_FINALIZE_HOOKS, so that the global paths are used:[binaries]
c = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-gcc' cpp = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-g++' ar = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-ar' strip = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-strip' pkgconfig = '/home/thomas/projets/buildroot/output/host/usr/bin/pkg-config'Signed-off-by Thomas Petazzoni <[email protected]>--- package/pkg-meson.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)package/pkg-meson.mk +0
-0package/pkg-meson.mk define PKG_MESON_INSTALL_CROSS_CONF · line 208
Quoted 5 lines
package/pkg-meson.mk +1
-1@@ -205,4 +205,4 @@ define PKG_MESON_INSTALL_CROSS_CONF > $(HOST_DIR)/etc/meson/cross-compilation.conf endef -TOOLCHAIN_POST_INSTALL_STAGING_HOOKS += PKG_MESON_INSTALL_CROSS_CONF +TOOLCHAIN_TARGET_FINALIZE_HOOKS += PKG_MESON_INSTALL_CROSS_CONFNote that this essentially reverts [1].
The case for that patch was not terribly strong, so it should be OK. But then
again, the case for this patch is not that strong either.Note that pkg-cmake has the same issue. And a number of other packages as well,
e.g. the Python sysconfig refers to the Python per-package dirs.If we really want to do it fully correctly, we'd need to run something like the
relocate-sdk.shon every per-package copy...Regards,
Arnout[1]
https://git.buildroot.org/buildroot/commit/?id=ccc9e05990a44d8db5825c07ed61d405ebb40caa
PATCH 1/1 package/pkg-meson: ensure the global cross-compilation.conf file is correct
In reply to Thomas De SchampheleireQuoted 8 lines · On Fri, 6 Dec 2019 10:54:13 +0100 Thomas De Schampheleire…
Needless to say, I am opposed to moving the meson file back from the target-finalize step :-)
I think most of the reasoning was already mentioned in that commit, but it seems I did not explain that we actually haveHOST_DIRmounted read-only on subsequent 'make' commands after the initial make, exactly to verify that no-one is changing directories they shouldn't be changing. After all, a 'make' after the initial make will only normally do the target-finalize step.
No, it will do staging-finalize as well. I don't think we provide the guarantee that HOST_DIR is unchanged/read-only between each "make" invocation, and I'm not sure why we would want to provide this guarantee.
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 1/1 package/pkg-meson: ensure the global cross-compilation.conf file is correct
El vie., 6 dic. 2019 a las 10:57, Thomas Petazzoni (<[email protected]>) escribi?:
Quoted 13 lines
Quoted 8 lines · On Fri, 6 Dec 2019 10:54:13 +0100 Thomas De Schampheleire…
Needless to say, I am opposed to moving the meson file back from the target-finalize step :-)
I think most of the reasoning was already mentioned in that commit, but it seems I did not explain that we actually haveHOST_DIRmounted read-only on subsequent 'make' commands after the initial make, exactly to verify that no-one is changing directories they shouldn't be changing. After all, a 'make' after the initial make will only normally do the target-finalize step.
Quoted 4 lines
No, it will do staging-finalize as well. I don't think we provide the guarantee that
HOST_DIRis unchanged/read-only between each "make" invocation, and I'm not sure why we would want to provide this guarantee.
The fact that staging-finalize is triggered from target-finalize is also a problem in my use case, and I have cleared it in our local copy (moving its action to another place). I think we talked about that earlier too.
I think Yann suggested me to look into the sdk in this context. I need to check it in more detail soon, to see whether it can really solve this friction we have between our current needs and changes upstream.
To recap our use case: we make an initial Buildroot build, package it, then in other machines but on the same virtual path (mounted to the same location inside a docker container) extract it. Then, applications are built outside of Buildroot, but stored to a copy of the output/target directory. Then a subsequent 'make' is run to let Buildroot package everything into an initramfs, setting TARGET_DIR to the copy. During all these subsequent builds, the host directory and some others are kept read-only to prevent corruption of that 'cache'.
Best regards,
Thomas
PATCH 1/1 package/pkg-meson: ensure the global cross-compilation.conf file is correct
Quoted 29 lines · Thomas De Schampheleire
El vie., 6 dic. 2019 a las 10:57, Thomas Petazzoni (<[email protected]>) escribi?:
Quoted 13 lines
Quoted 8 lines · On Fri, 6 Dec 2019 10:54:13 +0100 Thomas De Schampheleire…
Needless to say, I am opposed to moving the meson file back from the target-finalize step :-)
I think most of the reasoning was already mentioned in that commit, but it seems I did not explain that we actually haveHOST_DIRmounted read-only on subsequent 'make' commands after the initial make, exactly to verify that no-one is changing directories they shouldn't be changing. After all, a 'make' after the initial make will only normally do the target-finalize step.Quoted 4 lines
No, it will do staging-finalize as well. I don't think we provide the guarantee that
HOST_DIRis unchanged/read-only between each "make" invocation, and I'm not sure why we would want to provide this guarantee.The fact that staging-finalize is triggered from target-finalize is also a problem in my use case, and I have cleared it in our local copy (moving its action to another place). I think we talked about that earlier too.
I think Yann suggested me to look into the sdk in this context. I need to check it in more detail soon, to see whether it can really solve this friction we have between our current needs and changes upstream.
Quoted 7 lines
To recap our use case: we make an initial Buildroot build, package it, then in other machines but on the same virtual path (mounted to the same location inside a docker container) extract it. Then, applications are built outside of Buildroot, but stored to a copy of the output/target directory. Then a subsequent 'make' is run to let Buildroot package everything into an initramfs, setting
TARGET_DIRto the copy.
SDK is not going to work for that last bit.
The problem is that overall, Buildroot doesn't have a great story for
rebuilding things from a partial build. To be honest, I think you're better off not using Buildroot to package the initramfs, but just make a simple script that you run in fakeroot:
tar -x rootfs.tar
rsync custom packages
cpio ...
That is, assuming that the custom packages don't need to do things like adding
users etc. But if they're built outside of buildroot, that's not going to work anyway.
Regards,
Arnout
Quoted 6 lines
During all these subsequent builds, the host directory and some others are kept read-only to prevent corruption of that 'cache'.
Best regards,
Thomas
PATCH 1/1 package/pkg-meson: ensure the global cross-compilation.conf file is correct
El vie., 6 dic. 2019 a las 12:31, Arnout Vandecappelle (<[email protected]>) escribi?:
Quoted 48 lines · Thomas De Schampheleire
Quoted 29 lines
El vie., 6 dic. 2019 a las 10:57, Thomas Petazzoni (<[email protected]>) escribi?:
Quoted 13 lines
Quoted 8 lines · On Fri, 6 Dec 2019 10:54:13 +0100 Thomas De Schampheleire…
Needless to say, I am opposed to moving the meson file back from the target-finalize step :-)
I think most of the reasoning was already mentioned in that commit, but it seems I did not explain that we actually haveHOST_DIRmounted read-only on subsequent 'make' commands after the initial make, exactly to verify that no-one is changing directories they shouldn't be changing. After all, a 'make' after the initial make will only normally do the target-finalize step.Quoted 4 lines
No, it will do staging-finalize as well. I don't think we provide the guarantee that
HOST_DIRis unchanged/read-only between each "make" invocation, and I'm not sure why we would want to provide this guarantee.The fact that staging-finalize is triggered from target-finalize is also a problem in my use case, and I have cleared it in our local copy (moving its action to another place). I think we talked about that earlier too.
I think Yann suggested me to look into the sdk in this context. I need to check it in more detail soon, to see whether it can really solve this friction we have between our current needs and changes upstream.
Quoted 7 lines
To recap our use case: we make an initial Buildroot build, package it, then in other machines but on the same virtual path (mounted to the same location inside a docker container) extract it. Then, applications are built outside of Buildroot, but stored to a copy of the output/target directory. Then a subsequent 'make' is run to let Buildroot package everything into an initramfs, setting
TARGET_DIRto the copy.SDK is not going to work for that last bit.
The problem is that overall, Buildroot doesn't have a great story for
rebuilding things from a partial build. To be honest, I think you're better off not using Buildroot to package the initramfs, but just make a simple script that you run in fakeroot:
Quoted 8 lines
tar -x rootfs.tar
rsync custom packages
cpio ...That is, assuming that the custom packages don't need to do things like adding
users etc. But if they're built outside of buildroot, that's not going to work anyway.
Aside from the initramfs creation, we also rely on the other stuff happening in target-finalize, like stripping of objects, rootfs-overlays, post-build scripts, ... Duplicating this logic is not ideal.
But perhaps I could solve my use case by making the dependency of target-finalize on host-finalize and staging-finalize optional, steerable via a variable. Then I could use the default flow for the initial build, and pass the steer-variable to cut the dependency on the subsequent make. I'd need to try it to make sure it fits what I need.
This modification may or may not be acceptable for upstreaming, but probably is easier than trying to force you in keeping staging-finalize and host-finalize empty.
Best regards,
Thomas
PATCH 1/1 package/pkg-meson: ensure the global cross-compilation.conf file is correct
In reply to Thomas PetazzoniThomas?, All,
On 2019-12-04 16:02 +0100, Thomas Petazzoni spake thusly:
Quoted 19 lines
Currently, the cross-compilation.conf installed in $(HOST_DIR)/etc/meson/cross-compilation.conf for use by the SDK is generated in a post-install-staging hook of the toolchain package.
With per-package directory support enabled, this means that the generated cross-compilation.conf contains references to the per-package directory of the toolchain/ package, which is not want we want:
[binaries]
c = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-gcc' cpp = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-g++' ar = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-ar' strip = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-strip' pkgconfig = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/usr/bin/pkg-config'So instead, we generate this file in
TOOLCHAIN_TARGET_FINALIZE_HOOKS, so that the global paths are used:
Quoted 8 lines
[binaries]
c = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-gcc' cpp = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-g++' ar = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-ar' strip = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-strip' pkgconfig = '/home/thomas/projets/buildroot/output/host/usr/bin/pkg-config'Signed-off-by Thomas Petazzoni <[email protected]>
After giving it quite some thoughts, I think we do not want to provide any guarantee about the stableness of target/ vs. host/ on a partial rebuild, like Thomas P. said.
As such, I have applied this patch now.
Regards,
Yann E. MORIN.
Quoted 21 lines
--- package/pkg-meson.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)package/pkg-meson.mk +1
-1@@ -205,4 +205,4 @@ define PKG_MESON_INSTALL_CROSS_CONF > $(HOST_DIR)/etc/meson/cross-compilation.conf endef -TOOLCHAIN_POST_INSTALL_STAGING_HOOKS += PKG_MESON_INSTALL_CROSS_CONF +TOOLCHAIN_TARGET_FINALIZE_HOOKS += PKG_MESON_INSTALL_CROSS_CONFSignature · 6 lines
2.23.0 _______________________________________________ buildroot mailing list buildroot at busybox.net http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 1/1 package/pkg-meson: ensure the global cross-compilation.conf file is correct
Quoted 30 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Yann" == Yann E MORIN <[email protected]> writes:
Thomas?, All,
On 2019-12-04 16:02 +0100, Thomas Petazzoni spake thusly:Quoted 26 lines
Currently, the cross-compilation.conf installed in $(HOST_DIR)/etc/meson/cross-compilation.conf for use by the SDK is generated in a post-install-staging hook of the toolchain package.
With per-package directory support enabled, this means that the generated cross-compilation.conf contains references to the per-package directory of the toolchain/ package, which is not want we want:
[binaries]
c = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-gcc' cpp = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-g++' ar = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-ar' strip = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/bin/arm-linux-strip' pkgconfig = '/home/thomas/projets/buildroot/output/per-package/toolchain/host/usr/bin/pkg-config'So instead, we generate this file in
TOOLCHAIN_TARGET_FINALIZE_HOOKS, so that the global paths are used:[binaries]
c = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-gcc' cpp = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-g++' ar = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-ar' strip = '/home/thomas/projets/buildroot/output/host/bin/arm-linux-strip' pkgconfig = '/home/thomas/projets/buildroot/output/host/usr/bin/pkg-config'
Quoted 7 lines
Quoted 1 line
Signed-off-by Thomas Petazzoni <[email protected]>After giving it quite some thoughts, I think we do not want to provide any guarantee about the stableness of target/ vs. host/ on a partial rebuild, like Thomas P. said.
As such, I have applied this patch now.
Committed to 2020.02.x, 2020.05.x and 2020.08.x, thanks.
Signature · 1 line
Bye, Peter Korsgaard