Detect and prevent multiple providers (branch yem/fix-virt-pkg)
Subsystems (2)
cover Detect and prevent multiple providers (branch yem/fix-virt-pkg)
From: "Yann E. MORIN" <[email protected]>
Hello All!
Following a weird report about a cairo build failure, we came to the conclusion that the problem was due to multiple providers of the same virtual package were selected in the failing configuration. See the thread starting there:
http://lists.busybox.net/pipermail/buildroot/2014-May/096379.html
The matter was discussed on the list, and we came up to the conclusion we would not implement anything in the menuconfig to prevent this situation, but that we would just detect it prior to running the build. See the thread starting there:
http://lists.busybox.net/pipermail/buildroot/2014-May/096517.html
So, what this series does is just that: detect that more than one provider for a virtual package are selected.
The way it does that is fully described in patch 1/3, but basically:
- providers must now declare the virtual packages they are an
implementation of, in their .mk file;
- the generic package infrastructure checks that for each such declared virtual package, the current provider is indeed the selected implmenetation. If not, then it means two or more providers for that virtual pacakge are selected, and it bails out very early in the build.
Regards,
Yann E. MORIN.
The following changes since commit 655c0212f87ca4fc2621fb66a5dabd8bcc445eb7:
linux: bump to version 3.14.4 (2014-05-14 09:31:08 +0200)
are available in the git repository at:
git://gitorious.org/buildroot/buildroot.git yem/fix-virt-pkg
for you to fetch changes up to 40eb8c9da48a46856d0acb7576d35d621a9efc9c:
packages: make providers declare what they provide (2014-05-14 23:00:13 +0200)
----------------------------------------------------------------
Yann E. MORIN (3):
infra/pkg-virtual: validate only one provider provides an implementation docs/manual: document the new FOO_PROVIDES variable
packages: make providers declare what they provide
docs/manual/adding-packages-generic.txt | 3 +++ docs/manual/adding-packages-virtual.txt | 15 ++++++++++--- package/bellagio/bellagio.mk | 1 + package/eudev/eudev.mk | 1 + .../gpu-viv-bin-mx6q/gpu-viv-bin-mx6q.mk | 2 ++ package/lua/lua.mk | 2 ++ package/luajit/luajit.mk | 2 ++ package/mesa3d/mesa3d.mk | 5 +++++ package/pkg-generic.mk | 26 ++++++++++++++++++++++ package/rpi-userland/rpi-userland.mk | 2 ++ package/sunxi-mali/sunxi-mali.mk | 1 + package/systemd/systemd.mk | 2 ++ package/ti-gfx/ti-gfx.mk | 1 + 13 files changed, 60 insertions(+), 3 deletions(-)
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 1/3 infra/pkg-virtual: validate only one provider provides an implementation
From: "Yann E. MORIN" <[email protected]>
Currently, it is possible that more than one provider of a virtual package is selected in the menuconfig.
This leads to autobuild failures, and we do not protect the user from making a mistake in the configuration. The failure is then hard to troubleshoot in any case.
We can't use kconfig constructs to prevent this, since kconfig does not tell how many options did a select on another option.
This change introduces a new variable a provider *must* define to include all the virtual packages it is an implementation of. Then, when evaluating the package's rules, we check that the provider is indeed the declared one for each virtual package it claims to be an implementation of.
This works by taking advantage that when more than one provider is selected, only one of them will 'win' in setting the _PROVIDES_FOO option. Thus any provider just has to check it is indeed the declared provider. If not, it means that one or more other provider is selected.
This gives the opportunity to the user to change its configuration, and we can match the error message in the autobuilders to skip those failures (we can skip them instead of reporting them, since they are obviously configuration errors that should not happen in the first place.)
[Note: kudos to Arnout for suggesting this actual implementation. :-)]
Fixes:
http://autobuild.buildroot.org/results/285/2851069d6964aa46d26b4aabe7d84e8c0c6c72ce http://autobuild.buildroot.net/results/9b7/9b7870354d70e27e42d3d9c1f131ab54706bf20e [...]
--- package/pkg-generic.mk | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)
package/pkg-generic.mk +26 -0
@@ -241,6 +241,25 @@ $(BUILD_DIR)/%/.stamp_dircleaned: rm -Rf $(@D) ################################################################################ +# virt-provides-single -- check that provider-pkg is the declared provider for +# the virtual package virt-pkg +# +# argument 1 is the lower-case name of the virtual package +# argument 2 is the upper-case name of the virtual package +# argument 3 is the lower-case name of the provider +# +# example: +# $(call virt-provides-single,libegl,LIBEGL,rpi-userland) +################################################################################ +define virt-provides-single +ifneq ($$(call qstrip,$$(BR2_PACKAGE_PROVIDES_$(2))),$(3)) +$$(error Configuration error: both "$(3)" and $$(BR2_PACKAGE_PROVIDES_$(2))\ +are selected as providers for virtual package "$(1)". Only one provider can\ +be selected at a time. Please fix your configuration) +endif +endef + +################################################################################ # inner-generic-package -- generates the make targets needed to build a # generic package #
@@ -626,6 +645,13 @@ endif # ifneq ($(call qstrip,$$($(2)_SOURCE)),) # configuration ifeq ($$($$($(2)_KCONFIG_VAR)),y) +# Ensure the calling package is the declared provider for all the virtual +# packages it claims to be an implementation of. +ifneq ($$($(2)_PROVIDES),) +$$(foreach pkg,$$($(2)_PROVIDES),\ + $$(eval $$(call virt-provides-single,$$(pkg),$$(call UPPERCASE,$$(pkg)),$(1))$$(sep))) +endif + TARGETS += $(1) PACKAGES_PERMISSIONS_TABLE += $$($(2)_PERMISSIONS)$$(sep) PACKAGES_DEVICES_TABLE += $$($(2)_DEVICES)$$(sep)
Signature · 1 line
1.8.3.2
PATCH 2/3 docs/manual: document the new FOO_PROVIDES variable
In reply to Yann E. MORINFrom: "Yann E. MORIN" <[email protected]>
--- docs/manual/adding-packages-generic.txt | 3 +++ docs/manual/adding-packages-virtual.txt | 15 ++++++++++++--- 2 files changed, 15 insertions(+), 3 deletions(-)
docs/manual/adding-packages-generic.txt +3 -0
@@ -304,6 +304,9 @@ information is (assuming the package name is +libfoo+) : a similar way, +HOST_LIBFOO_DEPENDENCIES+ lists the dependencies for the current host package. +* +LIBFOO_PROVIDES+ lists all the virtual packages +libfoo+ is an + implementation of. See xref:virtual-package-tutorial[]. + * +LIBFOO_INSTALL_STAGING+ can be set to +YES+ or +NO+ (default). If set to +YES+, then the commands in the +LIBFOO_INSTALL_STAGING_CMDS+ variables are executed to install the package into the staging
docs/manual/adding-packages-virtual.txt +12 -3
@@ -39,7 +39,7 @@ In this file, we declare two options, +BR2_PACKAGE_HAS_SOMETHING_VIRTUAL+ and +BR2_PACKAGE_PROVIDES_SOMETHING_VIRTUAL+, whose values will be used by the providers. -==== Virtual package's +*.mk+ file +==== Virtual package's +.mk+ file The +.mk+ for the virtual package should just evaluate the +virtual-package+ macro:
@@ -59,8 +59,7 @@ The ability to have target and host packages is also available, with the ==== Provider's +Config.in+ file When adding a package as a provider, only the +Config.in+ file requires some -modifications. The +*.mk+ file should follow the Buildroot infrastructure with -no change at all. +modifications. The +Config.in+ file of the package 'some-provider', which provides the functionalities of 'something-virtual', should contain:
@@ -84,6 +83,16 @@ On line 3, we select +BR2_PACKAGE_HAS_SOMETHING_VIRTUAL+, and on line 11, we set the value of +BR2_PACKAGE_PROVIDES_SOMETHING_VIRTUAL+ to the name of the provider, but only if it is selected. +==== Provider's +.mk+ file + +The +.mk+ file should also declare an additional variable ++SOME_PROVIDER_PROVIDES+ to contain the names of all the virtual +packages it is an implementation of: + +--------------------------- +01: SOME_PROVIDER_PROVIDES = something-virtual +--------------------------- + Of course, do not forget to add the proper build and runtime dependencies for this package!
Signature · 1 line
1.8.3.2
PATCH 3/3 packages: make providers declare what they provide
In reply to Yann E. MORINFrom: "Yann E. MORIN" <[email protected]>
Fixes:
http://autobuild.buildroot.org/results/285/2851069d6964aa46d26b4aabe7d84e8c0c6c72ce http://autobuild.buildroot.net/results/9b7/9b7870354d70e27e42d3d9c1f131ab54706bf20e [...]
--- package/bellagio/bellagio.mk | 1 + package/eudev/eudev.mk | 1 + package/freescale-imx/gpu-viv-bin-mx6q/gpu-viv-bin-mx6q.mk | 2 ++ package/lua/lua.mk | 2 ++ package/luajit/luajit.mk | 2 ++ package/mesa3d/mesa3d.mk | 5 +++++ package/rpi-userland/rpi-userland.mk | 2 ++ package/sunxi-mali/sunxi-mali.mk | 1 + package/systemd/systemd.mk | 2 ++ package/ti-gfx/ti-gfx.mk | 1 + 10 files changed, 19 insertions(+)
package/bellagio/bellagio.mk +1 -0
@@ -11,5 +11,6 @@ BELLAGIO_AUTORECONF = YES BELLAGIO_INSTALL_STAGING = YES BELLAGIO_LICENSE = LGPLv2.1+ BELLAGIO_LICENSE_FILES = COPYING +BELLAGIO_PROVIDES = libopenmax $(eval $(autotools-package))
package/eudev/eudev.mk +1 -0
@@ -25,6 +25,7 @@ EUDEV_CONF_OPT = \ --enable-libkmod EUDEV_DEPENDENCIES = host-gperf host-pkgconf util-linux kmod +EUDEV_PROVIDES = udev ifeq ($(BR2_PACKAGE_EUDEV_RULES_GEN),y) EUDEV_CONF_OPT += --enable-rule_generator
package/freescale-imx/gpu-viv-bin-mx6q/gpu-viv-bin-mx6q.mk +2 -0
@@ -23,6 +23,8 @@ GPU_VIV_BIN_MX6Q_LICENSE = Freescale Semiconductor Software License Agreement # for which a separate license is needed... GPU_VIV_BIN_MX6Q_REDISTRIBUTE = NO +GPU_VIV_BIN_MX6Q_PROVIDES = libegl libgles + # DirectFB is not supported (wrong version) ifeq ($(BR2_PACKAGE_XORG7),y) GPU_VIV_BIN_MX6Q_DEPENDENCIES = xlib_libXdamage xlib_libXext
package/lua/lua.mk +2 -0
@@ -14,6 +14,8 @@ LUA_INSTALL_STAGING = YES LUA_LICENSE = MIT LUA_LICENSE_FILES = COPYRIGHT +LUA_PROVIDES = lua + LUA_CFLAGS = -Wall -fPIC -DLUA_USE_POSIX ifeq ($(BR2_PACKAGE_LUA_5_2),y)
package/luajit/luajit.mk +2 -0
@@ -12,6 +12,8 @@ LUAJIT_LICENSE_FILES = COPYRIGHT LUAJIT_INSTALL_STAGING = YES +LUAJIT_PROVIDES = lua + ifneq ($(BR2_LARGEFILE),y) LUAJIT_NO_LARGEFILE = TARGET_LFSFLAGS= endif
package/mesa3d/mesa3d.mk +5 -0
@@ -13,6 +13,8 @@ MESA3D_AUTORECONF = YES MESA3D_INSTALL_STAGING = YES +MESA3D_PROVIDES = + MESA3D_DEPENDENCIES = \ expat \ host-bison \
@@ -71,6 +73,7 @@ ifeq ($(MESA3D_DRI_DRIVERS-y),) MESA3D_CONF_OPT += \ --without-dri-drivers else +MESA3D_PROVIDES += libgl MESA3D_CONF_OPT += \ --enable-dri \ --enable-shared-glapi \
@@ -85,6 +88,7 @@ endif MESA3D_CONF_OPT += --enable-opengl ifeq ($(BR2_PACKAGE_MESA3D_OPENGL_EGL),y) +MESA3D_PROVIDES += libegl # egl depends on gbm, gbm depends on udev MESA3D_DEPENDENCIES += udev MESA3D_EGL_PLATFORMS = drm
@@ -105,6 +109,7 @@ MESA3D_CONF_OPT += \ endif ifeq ($(BR2_PACKAGE_MESA3D_OPENGL_ES),y) +MESA3D_PROVIDES += libgles MESA3D_CONF_OPT += --enable-gles1 --enable-gles2 else MESA3D_CONF_OPT += --disable-gles1 --disable-gles2
package/rpi-userland/rpi-userland.mk +2 -0
@@ -11,6 +11,8 @@ RPI_USERLAND_LICENSE_FILES = LICENCE RPI_USERLAND_INSTALL_STAGING = YES RPI_USERLAND_CONF_OPT = -DVMCS_INSTALL_PREFIX=/usr -DBUILD_SHARED_LIBS=OFF +RPI_USERLAND_PROVIDES = libegl libgles libopenmax libopenvg + define RPI_USERLAND_POST_TARGET_CLEANUP rm -Rf $(TARGET_DIR)/usr/src endef
package/sunxi-mali/sunxi-mali.mk +1 -0
@@ -9,6 +9,7 @@ SUNXI_MALI_SITE = $(call github,linux-sunxi,sunxi-mali,$(SUNXI_MALI_VERSION)) SUNXI_MALI_INSTALL_STAGING = YES SUNXI_MALI_DEPENDENCIES = sunxi-mali-prop +SUNXI_MALI_PROVIDES = libegl libgles # The options below must be provided in the environment. Providing these # through options overrides the value and prevents the makefiles from
package/systemd/systemd.mk +2 -0
@@ -17,6 +17,8 @@ SYSTEMD_DEPENDENCIES = \ kmod \ host-gperf +SYSTEMD_PROVIDES = udev + # Make sure that systemd will always be built after busybox so that we have # a consistent init setup between two builds ifeq ($(BR2_PACKAGE_BUSYBOX),y)
package/ti-gfx/ti-gfx.mk +1 -0
@@ -19,6 +19,7 @@ TI_GFX_LICENSE_FILES = TSPA.txt TI_GFX_INSTALL_STAGING = YES TI_GFX_DEPENDENCIES = linux +TI_GFX_PROVIDES = libegl libgles powervr ifeq ($(BR2_PACKAGE_TI_GFX_ES3),y) TI_GFX_OMAPES = 3.x
Signature · 1 line
1.8.3.2
PATCH 3/3 packages: make providers declare what they provide
Dear Yann E. MORIN,
package/lua/lua.mk LUA_INSTALL_STAGING = YES · line 17
Quoted 12 lines · Yann E. MORIN
# DirectFB is not supported (wrong version)
ifeq ($(BR2_PACKAGE_XORG7),y)GPU_VIV_BIN_MX6Q_DEPENDENCIES=xlib_libXdamagexlib_libXextpackage/lua/lua.mk +1
-0@@ -14,6 +14,8 @@ LUA_INSTALL_STAGING = YES LUA_LICENSE = MIT LUA_LICENSE_FILES = COPYRIGHT +LUA_PROVIDES = lua
This should be:
LUA_PROVIDES = luainterpreter
package/lua/lua.mk LUA_INSTALL_STAGING = YES · line 18
Quoted 1 line
package/lua/lua.mk +1
-0+LUAJIT_PROVIDES = lua
Same here.
I was surprised by "LUA_PROVIDES = lua". How can a real package provide itself as a provider for itself as a virtual package ? :-)
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
PATCH 3/3 packages: make providers declare what they provide
Thomas, All,
On 2014-05-15 09:40 +0200, Thomas Petazzoni spake thusly:
Quoted 12 lines · Yann E. MORIN
Quoted 11 lines
# DirectFB is not supported (wrong version)
ifeq ($(BR2_PACKAGE_XORG7),y)GPU_VIV_BIN_MX6Q_DEPENDENCIES=xlib_libXdamagexlib_libXextpackage/lua/lua.mk +0
-0@@ -14,6 +14,8 @@ LUA_INSTALL_STAGING = YES LUA_LICENSE = MIT LUA_LICENSE_FILES = COPYRIGHT
Quoted 5 lines
package/lua/lua.mk LUA_INSTALL_STAGING = YES · line 17
Quoted 1 line
package/lua/lua.mk +1
-0+LUA_PROVIDES = luaThis should be:
LUA_PROVIDES= luainterpreter
Obviously, indeed.
Quoted 6 lines
package/lua/lua.mk LUA_INSTALL_STAGING = YES · line 18
Quoted 1 line
package/lua/lua.mk +1
-0+LUAJIT_PROVIDES = luaSame here.
I was surprised by "
LUA_PROVIDES= lua". How can a real package provide itself as a provider for itself as a virtual package ? :-)
Hehe! :-)
I just did test the openGL virtual packages, and forgot to test lua et al. I'll fix and respin.
Thanks for noticing.
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'