fix kernel build failure for 5.1.0-rc3
cover fix kernel build failure for 5.1.0-rc3
This is my first time posting here, and I'm looking for feedback. Our team is attempting to build with a kernel revision above 5.1.0-rc3. I noticed in this build, the integration with buildroot is failing. I was able to pinpoint this down to a patch to the kernel which now uses pkg-config to find the libelf package to support non-default paths for libelf.
I was trying to quickly patch buildroot to work around this rather than changing the kernel itself. I noticed that pkgconf in buildroot creates a symbolic link in the target from pkgconf to pkg-config, but not in the host directory. I have a patch which adds this link to the host directory as well, but I quickly noticed this seems to open a can of worms with packages, at least the ones we have enabled in our configuration (graphics focused), down the toolchain now failing dependencies in configuration and build, after attempting to work through a few of the configuration dependency issues.
I wanted to reach out to the community for feedback on the approach I have here, and what might be the best path forward on this. I would prefer not to do a complete overhaul of the buildroot packages which depend on a host-based pkg-config, as I don't have a way to properly test these changes.
Thoughts?
Here's the commit in the kernel:
commit 056d28d135bca0b1d0908990338e00e9dadaf057
Author: Rolf Eike Beer <[email protected]> Date: Tue Mar 26 12:48:39 2019 -0500
objtool: Query pkg-config for libelf location
If it is not in the default location, compilation fails at several points.
Thanks in advance!
Stuart Summers (1):
package/pkgconf: add host link to pkg-config
linux/Config.in | 8 ++++++++ linux/linux.mk | 4 ++++ package/pkgconf/pkgconf.mk | 8 +++++++- 3 files changed, 19 insertions(+), 1 deletion(-)
Signature · 1 line
2.20.1
PATCH 1/1 package/pkgconf: add host link to pkg-config
A patch was added to the Linux kernel in 5.1.0-rc3 which adds a requirement that the host build environment include pkg-config. Since pkg-config was deprecated in buildroot, add a link in the host directory to /usr/bin/pkg-config.
Also add a new config option to enforce building host-pkgconf before building the Linux kernel.
--- linux/Config.in | 8 ++++++++ linux/linux.mk | 4 ++++ package/pkgconf/pkgconf.mk | 8 +++++++- 3 files changed, 19 insertions(+), 1 deletion(-)
linux/Config.in +8 -0
@@ -432,6 +432,14 @@ config BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF CONFIG_UNWINDER_ORC=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel". +config BR2_LINUX_KERNEL_NEEDS_HOST_PKG_CONFIG + bool "Needs host pkgconf" + help + Starting in 5.1.0-rc3, a change was added to the Linux kernel + which uses pkg-config to determine the location of the libelf + library. Without this option, the kernel will fail with build + errors. + # Linux extensions source "linux/Config.ext.in"
linux/linux.mk +4 -0
@@ -100,6 +100,10 @@ ifeq ($(BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF),y) LINUX_DEPENDENCIES += host-elfutils endif +ifeq ($(BR2_LINUX_KERNEL_NEEDS_HOST_PKG_CONFIG),y) +LINUX_DEPENDENCIES += host-pkgconf +endif + # If host-uboot-tools is selected by the user, assume it is needed to # create a custom image ifeq ($(BR2_PACKAGE_HOST_UBOOT_TOOLS),y)
package/pkgconf/pkgconf.mk +7 -1
@@ -16,6 +16,10 @@ define PKGCONF_LINK_PKGCONFIG ln -sf pkgconf $(TARGET_DIR)/usr/bin/pkg-config endef +define HOST_PKGCONF_LINK_PKGCONFIG + ln -sf pkgconf $(HOST_DIR)/usr/bin/pkg-config +endef + define HOST_PKGCONF_INSTALL_WRAPPER $(INSTALL) -m 0755 -D package/pkgconf/pkg-config.in \ $(HOST_DIR)/bin/pkg-config
@@ -32,7 +36,9 @@ define HOST_PKGCONF_SHARED endef PKGCONF_POST_INSTALL_TARGET_HOOKS += PKGCONF_LINK_PKGCONFIG -HOST_PKGCONF_POST_INSTALL_HOOKS += HOST_PKGCONF_INSTALL_WRAPPER +HOST_PKGCONF_POST_INSTALL_HOOKS += \ + HOST_PKGCONF_INSTALL_WRAPPER \ + HOST_PKGCONF_LINK_PKGCONFIG ifeq ($(BR2_STATIC_LIBS),y) HOST_PKGCONF_POST_INSTALL_HOOKS += HOST_PKGCONF_STATIC
Signature · 1 line
2.20.1
PATCH 1/1 package/pkgconf: add host link to pkg-config
Hello Stuart,
Thanks for the patch.
Quoted 4 lines · On Thu, 4 Apr 2019 12:24:13 -0700 Stuart Summers <[email protected]>…
A patch was added to the Linux kernel in 5.1.0-rc3 which adds a requirement that the host build environment include pkg-config. Since pkg-config was deprecated in buildroot, add a link in the host directory to
/usr/bin/pkg-config.
pkg-config is definitely not deprecated at all, and Buildroot's pkgconf package is already installing it as $(HOST_DIR)/usr/bin/pkg-config:
define HOST_PKGCONF_INSTALL_WRAPPER
$(INSTALL) -m 0755 -D package/pkgconf/pkg-config.in \
$(HOST_DIR)/bin/pkg-configHowever, the wrapper assumes by default we are building for the target, and therefore returns results valid for cross-compilation. To return results valid for native build, the following environment variables need to be passed:
PKG_CONFIG="$(PKG_CONFIG_HOST_BINARY)" \
PKG_CONFIG_SYSROOT_DIR="/" \
PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 \
PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 \
PKG_CONFIG_LIBDIR="$(HOST_DIR)/lib/pkgconfig:$(HOST_DIR)/share/pkgconfig"Quoted 2 lines
Also add a new config option to enforce building host-pkgconf before building the Linux kernel.
Do we need a new option ? If pkg-config is only used by Linux to detect libelf, what about just adding host-pkgconf to LINUX_DEPENDENCIES when BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF is enabled ?
package/pkgconf/pkgconf.mk define PKGCONF_LINK_PKGCONFIG · line 21
Quoted 3 lines
package/pkgconf/pkgconf.mk +3
-0+define HOST_PKGCONF_LINK_PKGCONFIG + ln -sf pkgconf $(HOST_DIR)/usr/bin/pkg-config +endef
This is definitely wrong and will break the wrapper we install.
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Summers
PATCH 1/1 package/pkgconf: add host link to pkg-config
Quoted 7 lines · Thomas Petazzoni
Hello Stuart,
Thanks for the patch.
On Thu, 4 Apr 2019 12:24:13 -0700
Stuart Summers <[email protected]> wrote:
Quoted 6 lines
Quoted 4 lines
A patch was added to the Linux kernel in 5.1.0-rc3 which adds a requirement that the host build environment include pkg-config. Since pkg-config was deprecated in buildroot, add a link in the host directory to
/usr/bin/pkg-config.pkg-config is definitely not deprecated at all, and Buildroot's
This was maybe a poorly worded commit message. I just meant that we are using pkgconf in place of pkg-config. I can remove this.
Quoted 13 lines
pkgconf
package is already installing it as $(HOST_DIR)/usr/bin/pkg-config:define
HOST_PKGCONF_INSTALL_WRAPPER$(INSTALL) -m 0755 -D package/pkgconf/pkg-config.in \ $(HOST_DIR)/bin/pkg-configHowever, the wrapper assumes by default we are building for the target,
and therefore returns results valid for cross-compilation. To return results valid for native build, the following environment variables need to be passed:
Quoted 6 lines
PKG_CONFIG="$(PKG_CONFIG_HOST_BINARY)" \ PKG_CONFIG_SYSROOT_DIR="/" \ PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 \ PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 \ PKG_CONFIG_LIBDIR="$(HOST_DIR)/lib/pkgconfig:$(HOST_DIR)/share/pkgconfig"
I apologize if I'm missing something obvious, but adding this to packages/pkgconf/pkgconf.mk did not seem to make any difference. I also tried adding this to linux/linux.mk. The latter I didn't do a full "make clean", but did on the former.
Quoted 5 lines
Quoted 3 lines
Also add a new config option to enforce building host-pkgconf before building the Linux kernel.
Do we need a new option ? If pkg-config is only used by Linux to
No need to add this, I agree.
Quoted 4 lines
detect
libelf, what about just adding host-pkgconf toLINUX_DEPENDENCIESwhenBR2_LINUX_KERNEL_NEEDS_HOST_LIBELFis enabled ?
At least without including the changes I had, I was seeing host-pkgconf show up in output/build even without adding this to LINUX_DEPENDENCIES, although that seems like the right thing to add anyway.
Quoted 6 lines
package/pkgconf/pkgconf.mk define PKGCONF_LINK_PKGCONFIG · line 21
Quoted 4 lines
package/pkgconf/pkgconf.mk +3
-0+define HOST_PKGCONF_LINK_PKGCONFIG + ln -sf pkgconf $(HOST_DIR)/usr/bin/pkg-config +endefThis is definitely wrong and will break the wrapper we install.
Thanks for the feedback and makes sense. Just looking at the right way to work around this.
Thanks,
Stuart
Quoted 2 lines
Thomas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3270 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20190404/894f8573/attachment.bin>
PATCH 1/1 package/pkgconf: add host link to pkg-config
Hello Stuart,
Quoted 13 lines · Stuart
Quoted 6 lines
However, the wrapper assumes by default we are building for the target,
and therefore returns results valid for cross-compilation. To return results valid for native build, the following environment variables need to be passed:Quoted 6 lines
PKG_CONFIG="$(PKG_CONFIG_HOST_BINARY)" \ PKG_CONFIG_SYSROOT_DIR="/" \ PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 \ PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 \ PKG_CONFIG_LIBDIR="$(HOST_DIR)/lib/pkgconfig:$(HOST_DIR)/share/pkgconfig"
Quoted 4 lines
I apologize if I'm missing something obvious, but adding this to
packages/pkgconf/pkgconf.mkdid not seem to make any difference. I also tried adding this tolinux/linux.mk. The latter I didn't do a full "make clean", but did on the former.
Adding those lines to pkgconf.mk will indeed make no difference: they should be added to the package calling pkg-config, in cases where pkg-config should return results valid for building native code (i.e not cross-compiled). Did you try to put this in LINUX_MAKE_ENV for example ?
Quoted 8 lines
Quoted 4 lines
detect
libelf, what about just adding host-pkgconf toLINUX_DEPENDENCIESwhenBR2_LINUX_KERNEL_NEEDS_HOST_LIBELFis enabled ?At least without including the changes I had, I was seeing host-pkgconf show up in output/build even without adding this to
LINUX_DEPENDENCIES, although that seems like the right thing to add anyway.
linux depends on host-kmod, and host-kmod depends on host-pkgconf, so indeed, host-pkgconf will always be pulled in. However, since Linux is using it directly, I think it makes sense to add the host-pkgconf dependency to LINUX_DEPENDENCIES.
Quoted 4 lines
Quoted 1 line
This is definitely wrong and will break the wrapper we install.
Thanks for the feedback and makes sense. Just looking at the right way to work around this.
The completely correct way would be:
- To have pkg-config return results for native build
- To have <tuple>-pkg-config return results for cross-compilation
But then, when cross-building, we need to convince the packages that they need to call <tuple>-pkg-config. The autotools PKG_CHECK_MODULES() macro does this by default, so a majority of packages would be OK. But there are also lots of packages that call "pkg-config" directly, and those would have to be fixed.
Best regards,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 1/1 linux: use host pkg-config when host libelf is set
In reply to Stuart SummersA patch was added to the Linux kernel in 5.1.0-rc3 which adds a requirement that the host build environment include pkg-config. Add the correct host-pkgconf dependency and environment variables to ensure Linux picks up the correct libraries.
--- linux/linux.mk | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)
linux/linux.mk +10 -4
@@ -96,10 +96,6 @@ ifeq ($(BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL),y) LINUX_DEPENDENCIES += host-openssl endif -ifeq ($(BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF),y) -LINUX_DEPENDENCIES += host-elfutils -endif - # If host-uboot-tools is selected by the user, assume it is needed to # create a custom image ifeq ($(BR2_PACKAGE_HOST_UBOOT_TOOLS),y)
@@ -125,6 +121,16 @@ LINUX_MAKE_ENV = \ $(TARGET_MAKE_ENV) \ BR_BINARIES_DIR=$(BINARIES_DIR) +ifeq ($(BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF),y) +LINUX_DEPENDENCIES += host-elfutils host-pkgconf +LINUX_MAKE_ENV += \ + PKG_CONFIG="$(PKG_CONFIG_HOST_BINARY)" \ + PKG_CONFIG_SYSROOT_DIR="/" \ + PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 \ + PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 \ + PKG_CONFIG_LIBDIR="$(HOST_DIR)/lib/pkgconfig:$(HOST_DIR)/share/pkgconfig" +endif + ifeq ($(BR2_REPRODUCIBLE),y) LINUX_MAKE_ENV += \ KBUILD_BUILD_VERSION=1 \
Signature · 1 line
2.20.1
Summers
PATCH 1/1 package/pkgconf: add host link to pkg-config
In reply to Thomas PetazzoniQuoted 27 lines · Thomas Petazzoni
Hello Stuart,
Quoted 16 lines · Stuart
Quoted 8 lines
However, the wrapper assumes by default we are building for the target,
and therefore returns results valid for cross-compilation. To return
results valid for native build, the following environment variables
need to be passed:Quoted 7 lines
PKG_CONFIG="$(PKG_CONFIG_HOST_BINARY)" \ PKG_CONFIG_SYSROOT_DIR="/" \ PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 \ PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 \ PKG_CONFIG_LIBDIR="$(HOST_DIR)/lib/pkgconfig:$(HOST_DIR)/shar
e/pkgconfig"Quoted 5 lines
I apologize if I'm missing something obvious, but adding this to
packages/pkgconf/pkgconf.mkdid not seem to make any difference. I also
tried adding this tolinux/linux.mk. The latter I didn't do a full "make clean", but did on the former.
Quoted 5 lines
Adding those lines to pkgconf.mk will indeed make no difference: they should be added to the package calling pkg-config, in cases where pkg-config should return results valid for building native code (i.e not cross-compiled). Did you try to put this in
LINUX_MAKE_ENVfor example ?
Pointing out my obvious misunderstanding here :). This does indeed fix the issue.
Quoted 13 lines
Quoted 7 lines
Quoted 6 lines
detect
libelf, what about just adding host-pkgconf toLINUX_DEPENDENCIES
whenBR2_LINUX_KERNEL_NEEDS_HOST_LIBELFis enabled ?Quoted 5 lines
At least without including the changes I had, I was seeing host- pkgconf
show up in output/build even without adding this toLINUX_DEPENDENCIES,
although that seems like the right thing to add anyway.
Quoted 5 lines
linux depends on host-kmod, and host-kmod depends on host-pkgconf, so indeed, host-pkgconf will always be pulled in. However, since Linux is
using it directly, I think it makes sense to add the host-pkgconf dependency toLINUX_DEPENDENCIES.
I agree.
Quoted 13 lines
Quoted 6 lines
Quoted 2 lines
This is definitely wrong and will break the wrapper we install.
Thanks for the feedback and makes sense. Just looking at the right way
to work around this.The completely correct way would be:
- To have pkg-config return results for native build
- To have <tuple>-pkg-config return results for cross-compilation
Quoted 7 lines
But then, when cross-building, we need to convince the packages that they need to call <tuple>-pkg-config. The autotools
PKG_CHECK_MODULES()
macro does this by default, so a majority of packages would be OK. But
there are also lots of packages that call "pkg-config" directly, and those would have to be fixed.
I did glance through the other packages. Enabling everything that touches host-pkgconf resulted in a build failure in the python module, which didn't appear to touch pkg-config. I decided to push something a bit more targeted instead. If you have another suggestion, though, let me know.
Thanks again for the feedback here!
-Stuart
Quoted 4 lines
Best regards,
Thomas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3270 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20190405/887008fb/attachment.bin>
PATCH 1/1 linux: use host pkg-config when host libelf is set
In reply to Stuart SummersStuart, All,
On 2019-04-05 11:47 -0700, Stuart Summers spake thusly:
Quoted 17 lines
A patch was added to the Linux kernel in 5.1.0-rc3 which adds a requirement that the host build environment include pkg-config. Add the correct host-pkgconf dependency and environment variables to ensure Linux picks up the correct libraries.
Suggested-by Thomas Petazzoni <[email protected]>Signed-off-by Stuart Summers <[email protected]>--- linux/linux.mk | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)linux/linux.mk +0
-0
linux/linux.mk ifeq ($(BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL),y) · line 101
Quoted 6 lines
linux/linux.mk +0
-3@@ -96,10 +96,6 @@ ifeq ($(BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL),y) LINUX_DEPENDENCIES += host-openssl endif -ifeq ($(BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF),y) -LINUX_DEPENDENCIES += host-elfutils -endif
Why did you move that block around?
Quoted 5 lines
linux/linux.mk +0
-0# If host-uboot-tools is selected by the user, assume it is needed to # create a custom image ifeq ($(BR2_PACKAGE_HOST_UBOOT_TOOLS),y)@@ -125,6 +121,16 @@ LINUX_MAKE_ENV = \ $(TARGET_MAKE_ENV) \
linux/linux.mk LINUX_MAKE_ENV = \ · line 131
Quoted 10 lines
linux/linux.mk +8
-0BR_BINARIES_DIR=$(BINARIES_DIR) +ifeq ($(BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF),y) +LINUX_DEPENDENCIES += host-elfutils host-pkgconf +LINUX_MAKE_ENV += \ + PKG_CONFIG="$(PKG_CONFIG_HOST_BINARY)" \ + PKG_CONFIG_SYSROOT_DIR="/" \ + PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 \ + PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 \ + PKG_CONFIG_LIBDIR="$(HOST_DIR)/lib/pkgconfig:$(HOST_DIR)/share/pkgconfig"
linux is a kconfig-package, so we should be able to run (for example):
make linux-menuconfig
However, this should use the real pkg-config from the host, not ours, so we explictly blacklist a set of PKG_CONFIG_* variables when calling the kconfig configurators:
https://git.buildroot.org/buildroot/tree/package/pkg-kconfig.mk#n174
If needed, please expand that blacklist with the two additional PKG_CONFIG_ALLOW_SYSTEM_{CFLAGS,LIBS} this patch introduces.
If that is not needed, please expand the commit log to explain it.
In the meantime, I've marked this patch as changes-requested in patchwork.
Thank! :-)
Regards,
Yann E. MORIN.
Quoted 12 lines
linux/linux.mk +2
-0+endif + ifeq ($(BR2_REPRODUCIBLE),y) LINUX_MAKE_ENV += \ KBUILD_BUILD_VERSION=1 \Signature · 6 lines
2.20.1 _______________________________________________ buildroot mailing list buildroot at busybox.net http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
Summers
PATCH 1/1 linux: use host pkg-config when host libelf is set
Quoted 23 lines · Yann E. MORIN
Stuart, All,
On 2019-04-05 11:47 -0700, Stuart Summers spake thusly:
Quoted 20 lines
A patch was added to the Linux kernel in 5.1.0-rc3 which adds a requirement that the host build environment include pkg-config. Add the correct host-pkgconf dependency and environment variables to ensure Linux picks up the correct libraries.
Suggested-by Thomas Petazzoni <[email protected]>Signed-off-by Stuart Summers <[email protected]>--- linux/linux.mk | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)linux/linux.mk +0
-0@@ -96,10 +96,6 @@ ifeq ($(BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL),y) LINUX_DEPENDENCIES += host-openssl endif
Quoted 5 lines
linux/linux.mk ifeq ($(BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL),y) · line 101
Quoted 3 lines
linux/linux.mk +0
-3-ifeq ($(BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF),y) -LINUX_DEPENDENCIES += host-elfutils -endifWhy did you move that block around?
Moved this for a single conditional after the initial LINUX_MAKE_ENV =
...
I could have alternatively separated into two LINUX_KERNEL_NEEDS_HOST_LIBELF conditions, but I opted to squash them together.
Quoted 29 lines
Quoted 17 lines
linux/linux.mk +0
-0# If host-uboot-tools is selected by the user, assume it is neededto
# create a custom image ifeq ($(BR2_PACKAGE_HOST_UBOOT_TOOLS),y)linux/linux.mk +8
-0@@ -125,6 +121,16 @@ LINUX_MAKE_ENV = \ $(TARGET_MAKE_ENV) \ BR_BINARIES_DIR=$(BINARIES_DIR) +ifeq ($(BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF),y) +LINUX_DEPENDENCIES += host-elfutils host-pkgconf +LINUX_MAKE_ENV += \ + PKG_CONFIG="$(PKG_CONFIG_HOST_BINARY)" \ + PKG_CONFIG_SYSROOT_DIR="/" \ + PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 \ + PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 \ + PKG_CONFIG_LIBDIR="$(HOST_DIR)/lib/pkgconfig:$(HOST_DIR)/share/Quoted 1 line
pkgconfig"
linux is a kconfig-package, so we should be able to run (for example):
make linux-menuconfigHowever, this should use the real pkg-config from the host, not ours, so
we explictly blacklist a set of PKG_CONFIG_* variables when calling the
kconfig configurators:
Quoted 4 lines
https://git.buildroot.org/buildroot/tree/package/pkg-kconfig.mk#n174
If needed, please expand that blacklist with the two additional PKG_CONFIG_ALLOW_SYSTEM_{CFLAGS,LIBS} this patch introduces.
I apologize for my lack of understanding in this area. I'm happy to add this to the blacklist you mentioned. At least local testing seems to indicate that "make linux-menuconfig" works both with and without the blacklist. I tried this with a default configuration (make distclean -> make menuconfig -> enable Linux and use an architecture default config file -> make linux-menuconfig), and this results in a full host structure being set up in output/host. So it seems like the blacklist of these two extra variables isn't strictly necessary. Or am I missing something? That said, looking at the documentation, maybe this takes away the ability for a user to have a custom pkg-config path - I'm using the default installation. So in that sense, blacklisting does seem like the right thing to do. I'll add this and post a quick update.
Quoted 7 lines
If that is not needed, please expand the commit log to explain it.
In the meantime, I've marked this patch as changes-requested in patchwork.
Thank! :-)
Thanks for the review!
-Stuart
Quoted 12 lines
Regards,
Yann E. MORIN.Quoted 8 lines
linux/linux.mk +2
-0+endif + ifeq ($(BR2_REPRODUCIBLE),y) LINUX_MAKE_ENV += \ KBUILD_BUILD_VERSION=1 \Signature · 1 line
2.20.1
Quoted 6 lines
Quoted 4 lines
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3270 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20190408/f912c38f/attachment.bin>
PATCH 1/1 linux: use host pkg-config when host libelf is set
Stuart, All,
On 2019-04-08 16:00 +0000, Summers, Stuart spake thusly:
Quoted 7 lines · Yann E. MORIN
Quoted 6 lines
On 2019-04-05 11:47 -0700, Stuart Summers spake thusly:
Quoted 5 lines
A patch was added to the Linux kernel in 5.1.0-rc3 which adds a requirement that the host build environment include pkg-config. Add the correct host-pkgconf dependency and environment variables to ensure Linux picks up the correct libraries.
[--SNIP--]
Quoted 8 lines
Quoted 5 lines
linux/linux.mk ifeq ($(BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL),y) · line 101
Quoted 3 lines
linux/linux.mk +0
-3-ifeq ($(BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF),y) -LINUX_DEPENDENCIES += host-elfutils -endifWhy did you move that block around?
Moved this for a single conditional after the initial
LINUX_MAKE_ENV=
...
Ah OK, makes sense.
However, this means that the various BR2_LINUX_KERNEL_NEEDS_HOST_* blocks are all scatered around in the file, now.
What about:
- move the existign LINUX_MAKE_ENV assignment earlier in the file,
- keep the block in-place and expand it, like:
ifeq ($(BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF),y)
LINUX_DEPENDENCIES += host-elfutils host-pkgconf
LINUX_MAKE_ENV += \
blablabla
endifQuoted 3 lines
I could have alternatively separated into two
LINUX_KERNEL_NEEDS_HOST_LIBELFconditions, but I opted to squash them together.
Which is correct, I just missed the existing LINUX_MAKE_ENV location.
[--SNIP--]
Quoted 15 lines
Quoted 2 lines
If needed, please expand that blacklist with the two additional PKG_CONFIG_ALLOW_SYSTEM_{CFLAGS,LIBS} this patch introduces.
I apologize for my lack of understanding in this area. I'm happy to add this to the blacklist you mentioned. At least local testing seems to indicate that "make linux-menuconfig" works both with and without the blacklist. I tried this with a default configuration (make distclean -> make menuconfig -> enable Linux and use an architecture default config file -> make linux-menuconfig), and this results in a full host structure being set up in output/host. So it seems like the blacklist of these two extra variables isn't strictly necessary. Or am I missing something? That said, looking at the documentation, maybe this takes away the ability for a user to have a custom pkg-config path - I'm using the default installation. So in that sense, blacklisting does seem like the right thing to do. I'll add this and post a quick update.
TBH, I think we should really blacklist them, indeed, so that we are sure the distro provided pkg-config works as expected.
Thanks for the feedback!
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 1/1 linux: use host pkg-config when host libelf is set
In reply to Stuart SummersA patch was added to the Linux kernel in 5.1.0-rc3 which adds a requirement that the host build environment include pkg-config. Add the correct host-pkgconf dependency and environment variables to ensure Linux picks up the correct libraries.
v2: Use LINUX_MAKE_ENV to pass PKG_CONFIG_* options (Thomas)
v3: Add PKG_CONFIG_ALLOW_SYSTEM_{CFLAGS,LIBS} to kconfig
blacklist and move LINUX_MAKE_ENV initial assign in linux.mk before DEP* assignments (Yann)
--- linux/linux.mk | 16 +++++++++++----- package/pkg-kconfig.mk | 4 +++- 2 files changed, 14 insertions(+), 6 deletions(-)
linux/linux.mk +11 -5
@@ -61,6 +61,10 @@ LINUX_PATCHES = $(call qstrip,$(BR2_LINUX_KERNEL_PATCH)) # be directories in the patch list (unlike for other packages). LINUX_PATCH = $(filter ftp://% http://% https://%,$(LINUX_PATCHES)) +LINUX_MAKE_ENV = \ + $(TARGET_MAKE_ENV) \ + BR_BINARIES_DIR=$(BINARIES_DIR) + LINUX_INSTALL_IMAGES = YES LINUX_DEPENDENCIES = host-kmod
@@ -97,7 +101,13 @@ LINUX_DEPENDENCIES += host-openssl endif ifeq ($(BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF),y) -LINUX_DEPENDENCIES += host-elfutils +LINUX_DEPENDENCIES += host-elfutils host-pkgconf +LINUX_MAKE_ENV += \ + PKG_CONFIG="$(PKG_CONFIG_HOST_BINARY)" \ + PKG_CONFIG_SYSROOT_DIR="/" \ + PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 \ + PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 \ + PKG_CONFIG_LIBDIR="$(HOST_DIR)/lib/pkgconfig:$(HOST_DIR)/share/pkgconfig" endif # If host-uboot-tools is selected by the user, assume it is needed to
@@ -121,10 +131,6 @@ LINUX_MAKE_FLAGS = \ CROSS_COMPILE="$(TARGET_CROSS)" \ DEPMOD=$(HOST_DIR)/sbin/depmod -LINUX_MAKE_ENV = \ - $(TARGET_MAKE_ENV) \ - BR_BINARIES_DIR=$(BINARIES_DIR) - ifeq ($(BR2_REPRODUCIBLE),y) LINUX_MAKE_ENV += \ KBUILD_BUILD_VERSION=1 \
package/pkg-kconfig.mk +3 -1
@@ -175,7 +175,9 @@ endif # nconfig, gconfig, xconfig). # So we simply remove our PATH and PKG_CONFIG_* variables. $(2)_CONFIGURATOR_MAKE_ENV = \ - $$(filter-out PATH=% PKG_CONFIG=% PKG_CONFIG_SYSROOT_DIR=% PKG_CONFIG_LIBDIR=%,$$($(2)_MAKE_ENV)) \ + $$(filter-out PATH=% PKG_CONFIG=% PKG_CONFIG_SYSROOT_DIR=% \ + PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=% PKG_CONFIG_ALLOW_SYSTEM_LIBS=% \ + PKG_CONFIG_LIBDIR=%,$$($(2)_MAKE_ENV)) \ PKG_CONFIG_PATH="$(HOST_PKG_CONFIG_PATH)" # Configuration editors (menuconfig, ...)
Signature · 1 line
2.20.1
PATCH 1/1 linux: use host pkg-config when host libelf is set
Stuart, All,
On 2019-04-08 11:42 -0700, Stuart Summers spake thusly:
Quoted 5 lines
A patch was added to the Linux kernel in 5.1.0-rc3 which adds a requirement that the host build environment include pkg-config. Add the correct host-pkgconf dependency and environment variables to ensure Linux picks up the correct libraries.
Additionally:
Move the existing LINUX_MAKE_ENV assignment earlier, to simplify the append-assignment in the libelf conditional block.
And since you posted your patch, someone opened a bug in our tracker, so:
Fixes: #11761
Quoted 4 lines
v2: Use
LINUX_MAKE_ENVto pass PKG_CONFIG_* options (Thomas)
v3: Add PKG_CONFIG_ALLOW_SYSTEM_{CFLAGS,LIBS} to kconfig
blacklist and moveLINUX_MAKE_ENVinitial assign in linux.mk before DEP* assignments (Yann)
This v2-v3 changelog should be after the --- line, below...
Quoted 5 lines
Cc Thomas Petazzoni <[email protected]>Cc Yann E. MORIN <[email protected]>Suggested-by Thomas Petazzoni <[email protected]>Signed-off-by Stuart Summers <[email protected]>---
... here.
Quoted 50 lines
linux/linux.mk | 16 +++++++++++----- package/pkg-kconfig.mk | 4 +++- 2 files changed, 14 insertions(+), 6 deletions(-)linux/linux.mk +11
-5@@ -61,6 +61,10 @@ LINUX_PATCHES = $(call qstrip,$(BR2_LINUX_KERNEL_PATCH)) # be directories in the patch list (unlike for other packages). LINUX_PATCH = $(filter ftp://% http://% https://%,$(LINUX_PATCHES)) +LINUX_MAKE_ENV = \ + $(TARGET_MAKE_ENV) \ + BR_BINARIES_DIR=$(BINARIES_DIR) + LINUX_INSTALL_IMAGES = YES LINUX_DEPENDENCIES = host-kmod@@ -97,7 +101,13 @@ LINUX_DEPENDENCIES += host-openssl endif ifeq ($(BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF),y) -LINUX_DEPENDENCIES += host-elfutils +LINUX_DEPENDENCIES += host-elfutils host-pkgconf +LINUX_MAKE_ENV += \ + PKG_CONFIG="$(PKG_CONFIG_HOST_BINARY)" \ + PKG_CONFIG_SYSROOT_DIR="/" \ + PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 \ + PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 \ + PKG_CONFIG_LIBDIR="$(HOST_DIR)/lib/pkgconfig:$(HOST_DIR)/share/pkgconfig" endif # If host-uboot-tools is selected by the user, assume it is needed to@@ -121,10 +131,6 @@ LINUX_MAKE_FLAGS = \ CROSS_COMPILE="$(TARGET_CROSS)" \ DEPMOD=$(HOST_DIR)/sbin/depmod -LINUX_MAKE_ENV = \ - $(TARGET_MAKE_ENV) \ - BR_BINARIES_DIR=$(BINARIES_DIR) - ifeq ($(BR2_REPRODUCIBLE),y) LINUX_MAKE_ENV += \ KBUILD_BUILD_VERSION=1 \package/pkg-kconfig.mk +0
-0
package/pkg-kconfig.mk endif · line 180
Quoted 7 lines
package/pkg-kconfig.mk +3
-1@@ -175,7 +175,9 @@ endif # nconfig, gconfig, xconfig). # So we simply remove our PATH and PKG_CONFIG_* variables. $(2)_CONFIGURATOR_MAKE_ENV = \ - $$(filter-out PATH=% PKG_CONFIG=% PKG_CONFIG_SYSROOT_DIR=% PKG_CONFIG_LIBDIR=%,$$($(2)_MAKE_ENV)) \ + $$(filter-out PATH=% PKG_CONFIG=% PKG_CONFIG_SYSROOT_DIR=% \ + PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=% PKG_CONFIG_ALLOW_SYSTEM_LIBS=% \ + PKG_CONFIG_LIBDIR=%,$$($(2)_MAKE_ENV)) \
I'm not too happy about the look the above has, but I have no better formatting suggestion to provide...
Regards,
Yann E. MORIN.
Quoted 6 lines
package/pkg-kconfig.mk +0
-0PKG_CONFIG_PATH="$(HOST_PKG_CONFIG_PATH)" # Configuration editors (menuconfig, ...)Signature · 1 line
2.20.1
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
Summers
PATCH 1/1 linux: use host pkg-config when host libelf is set
Quoted 9 lines · Yann E. MORIN
Stuart, All,
On 2019-04-08 11:42 -0700, Stuart Summers spake thusly:
Quoted 5 lines
A patch was added to the Linux kernel in 5.1.0-rc3 which adds a requirement that the host build environment include pkg-config. Add the correct host-pkgconf dependency and environment variables to ensure Linux picks up the correct libraries.
Quoted 4 lines
Additionally:
Move the existing
LINUX_MAKE_ENVassignment earlier, to simplify the append-assignment in the libelf conditional block.
Makes sense.
Quoted 5 lines
And since you posted your patch, someone opened a bug in our tracker, so:
Fixes: #11761
Ah, didn't see this bug until after I sent out the patch. Thanks for the update there!
Quoted 7 lines
Quoted 5 lines
v2: Use
LINUX_MAKE_ENVto pass PKG_CONFIG_* options (Thomas)
v3: Add PKG_CONFIG_ALLOW_SYSTEM_{CFLAGS,LIBS} to kconfig
blacklist and moveLINUX_MAKE_ENVinitial assign in linux.mk before DEP* assignments (Yann)This v2-v3 changelog should be after the --- line, below...
In the future, I'll format this way, thanks for the pointer.
Quoted 68 lines
Quoted 6 lines
Cc Thomas Petazzoni <[email protected]>Cc Yann E. MORIN <[email protected]>Suggested-by Thomas Petazzoni <[email protected]>Signed-off-by Stuart Summers <[email protected]>---
... here.
Quoted 59 lines
linux/linux.mk | 16 +++++++++++----- package/pkg-kconfig.mk | 4 +++- 2 files changed, 14 insertions(+), 6 deletions(-)linux/linux.mk +4
-0@@ -61,6 +61,10 @@ LINUX_PATCHES = $(call qstrip,$(BR2_LINUX_KERNEL_PATCH)) # be directories in the patch list (unlike for other packages). LINUX_PATCH = $(filter ftp://% http://% https://%,$(LINUX_PATCHES) ) +LINUX_MAKE_ENV = \ + $(TARGET_MAKE_ENV) \ + BR_BINARIES_DIR=$(BINARIES_DIR) + LINUX_INSTALL_IMAGES = YES
LINUX_DEPENDENCIES= host-kmodlinux/linux.mk +7
-1@@ -97,7 +101,13 @@ LINUX_DEPENDENCIES += host-openssl endif ifeq ($(BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF),y) -LINUX_DEPENDENCIES += host-elfutils +LINUX_DEPENDENCIES += host-elfutils host-pkgconf +LINUX_MAKE_ENV += \ + PKG_CONFIG="$(PKG_CONFIG_HOST_BINARY)" \ + PKG_CONFIG_SYSROOT_DIR="/" \ + PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 \ + PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 \ + PKG_CONFIG_LIBDIR="$(HOST_DIR)/lib/pkgconfig:$(HOST_DIR)/share/pkgconfig"
endif# If host-uboot-tools is selected by the user, assume it is needed
tolinux/linux.mk +0
-4@@ -121,10 +131,6 @@ LINUX_MAKE_FLAGS = \ CROSS_COMPILE="$(TARGET_CROSS)" \ DEPMOD=$(HOST_DIR)/sbin/depmod -LINUX_MAKE_ENV = \ - $(TARGET_MAKE_ENV) \ - BR_BINARIES_DIR=$(BINARIES_DIR) - ifeq ($(BR2_REPRODUCIBLE),y) LINUX_MAKE_ENV += \ KBUILD_BUILD_VERSION=1 \package/pkg-kconfig.mk +0
-1@@ -175,7 +175,9 @@ endif # nconfig, gconfig, xconfig). # So we simply remove our PATH and PKG_CONFIG_* variables. $(2)_CONFIGURATOR_MAKE_ENV = \ - $$(filter-out PATH=% PKG_CONFIG=% PKG_CONFIG_SYSROOT_DIR=% PKG_CONFIG_LIBDIR=%,$$($(2)_MAKE_ENV)) \
Quoted 8 lines
package/pkg-kconfig.mk endif · line 180
Quoted 4 lines
package/pkg-kconfig.mk +3
-0+ $$(filter-out PATH=% PKG_CONFIG=% PKG_CONFIG_SYSROOT_DIR=% \ + PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=% PKG_CONFIG_ALLOW_SYSTEM_LIBS=% \ + PKG_CONFIG_LIBDIR=%,$$($(2)_MAKE_ENV)) \I'm not too happy abfor the poout the look the above has, but I have no better
formatting suggestion to provide...
Yeah, I agree it looks ugly.. If you have suggestions, I'm happy to take a look.
Quoted 2 lines
Reviewed-by "Yann E. MORIN" <[email protected]>
Thanks for the review and the time here!
Given this is my first patch, I'm not sure I have push rights. Can you help with merging? Also, would you like a re-post with the commit message recommendations above?
Thanks,
Stuart
Quoted 12 lines
Regards,
Yann E. MORIN.Quoted 6 lines
package/pkg-kconfig.mk +0
-0PKG_CONFIG_PATH="$(HOST_PKG_CONFIG_PATH)"# Configuration editors (menuconfig, ...)
Signature · 1 line
2.20.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3270 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20190408/be12d5eb/attachment.bin>
PATCH 1/1 linux: use host pkg-config when host libelf is set
Stuart, All,
On 2019-04-08 19:41 +0000, Summers, Stuart spake thusly:
Quoted 1 line · Yann E. MORIN
[--SNIP--]
Quoted 11 lines
Quoted 11 lines
package/pkg-kconfig.mk endif · line 180
Quoted 7 lines
$(2)_CONFIGURATOR_MAKE_ENV = \
(quoted diff) +0
-1- $$(filter-out PATH=% PKG_CONFIG=% PKG_CONFIG_SYSROOT_DIR=%PKG_CONFIG_LIBDIR=%,$$($(2)_MAKE_ENV)) \
(quoted diff) +2
-0+ $$(filter-out PATH=% PKG_CONFIG=% PKG_CONFIG_SYSROOT_DIR=% \ + PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=%PKG_CONFIG_ALLOW_SYSTEM_LIBS=% \
package/pkg-kconfig.mk +1
-0+ PKG_CONFIG_LIBDIR=%,$$($(2)_MAKE_ENV)) \I'm not too happy abfor the poout the look the above has, but I have no better
formatting suggestion to provide...
Quoted 2 lines
Yeah, I agree it looks ugly.. If you have suggestions, I'm happy to take a look.
No, I ahve no better suggestion, really, hence I gave my:
Quoted 7 lines
Quoted 1 line
Reviewed-by "Yann E. MORIN" <[email protected]>Thanks for the review and the time here!
Given this is my first patch, I'm not sure I have push rights. Can you help with merging? Also, would you like a re-post with the commit message recommendations above?
When one of the three maintainer have time, they'll pick up that patch and apply; it's not going to be forgotten: all patches are tracked on patchwork:
https://patchwork.ozlabs.org/project/buildroot/list/
The committer can probably apply the tweaks I suggested when they apply, so no need to respin unless requested, now.
Thanks!
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
Arnout Vandecappelle
AppliedPATCH 1/1 linux: use host pkg-config when host libelf is set
In reply to Stuart SummersQuoted 11 lines · Stuart Summers
A patch was added to the Linux kernel in 5.1.0-rc3 which adds a requirement that the host build environment include pkg-config. Add the correct host-pkgconf dependency and environment variables to ensure Linux picks up the correct libraries.
v2: Use
LINUX_MAKE_ENVto pass PKG_CONFIG_* options (Thomas)
v3: Add PKG_CONFIG_ALLOW_SYSTEM_{CFLAGS,LIBS} to kconfig
blacklist and moveLINUX_MAKE_ENVinitial assign in linux.mk before DEP* assignments (Yann)
Quoted 4 lines
Cc Thomas Petazzoni <[email protected]>Cc Yann E. MORIN <[email protected]>Suggested-by Thomas Petazzoni <[email protected]>Signed-off-by Stuart Summers <[email protected]>
Applied to master with Yann's suggested changes to the commit message, thank.
Regards,
Arnout
Summers
PATCH 1/1 linux: use host pkg-config when host libelf is set
Quoted 13 lines · Arnout Vandecappelle
Quoted 11 lines · Stuart Summers
A patch was added to the Linux kernel in 5.1.0-rc3 which adds a requirement that the host build environment include pkg-config. Add the correct host-pkgconf dependency and environment variables to ensure Linux picks up the correct libraries.
v2: Use
LINUX_MAKE_ENVto pass PKG_CONFIG_* options (Thomas)
v3: Add PKG_CONFIG_ALLOW_SYSTEM_{CFLAGS,LIBS} to kconfig
blacklist and moveLINUX_MAKE_ENVinitial assign in linux.mk before DEP* assignments (Yann)
Quoted 7 lines
Quoted 4 lines
Cc Thomas Petazzoni <[email protected]>Cc Yann E. MORIN <[email protected]>Suggested-by Thomas Petazzoni <[email protected]>Signed-off-by Stuart Summers <[email protected]>Applied to master with Yann's suggested changes to the commit
message, thank.
Thanks!
-Stuart
Quoted 3 lines
Regards,
Arnout
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3270 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20190415/03e5ce56/attachment.bin>