Skip to contentSkip to conversation

Detect and prevent multiple providers (branch yem/fix-virt-pkg)

PATCH Superseded Yann E. MORIN 2 participants 6 messages May 14, 2014 – May 15, 2014

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover Detect and prevent multiple providers (branch yem/fix-virt-pkg)

From: "Yann E. MORIN" <[email protected]>

Hello All!

Following a weird report about a cairo build failure, we came to the conclusion that the problem was due to multiple providers of the same virtual package were selected in the failing configuration. See the thread starting there:

    http://lists.busybox.net/pipermail/buildroot/2014-May/096379.html

The matter was discussed on the list, and we came up to the conclusion we would not implement anything in the menuconfig to prevent this situation, but that we would just detect it prior to running the build. See the thread starting there:

    http://lists.busybox.net/pipermail/buildroot/2014-May/096517.html

So, what this series does is just that: detect that more than one provider for a virtual package are selected.

The way it does that is fully described in patch 1/3, but basically:

- providers must now declare the virtual packages they are an

    implementation of, in their .mk file;

- the generic package infrastructure checks that for each such declared virtual package, the current provider is indeed the selected implmenetation. If not, then it means two or more providers for that virtual pacakge are selected, and it bails out very early in the build.

Regards,
Yann E. MORIN.

The following changes since commit 655c0212f87ca4fc2621fb66a5dabd8bcc445eb7:

linux: bump to version 3.14.4 (2014-05-14 09:31:08 +0200)

are available in the git repository at:

git://gitorious.org/buildroot/buildroot.git yem/fix-virt-pkg

for you to fetch changes up to 40eb8c9da48a46856d0acb7576d35d621a9efc9c:

packages: make providers declare what they provide (2014-05-14 23:00:13 +0200)

----------------------------------------------------------------
Yann E. MORIN (3):
infra/pkg-virtual: validate only one provider provides an implementation docs/manual: document the new FOO_PROVIDES variable
packages: make providers declare what they provide

 docs/manual/adding-packages-generic.txt            |  3 +++
 docs/manual/adding-packages-virtual.txt            | 15 ++++++++++---
 package/bellagio/bellagio.mk                       |  1 +
 package/eudev/eudev.mk                             |  1 +
 .../gpu-viv-bin-mx6q/gpu-viv-bin-mx6q.mk           |  2 ++
 package/lua/lua.mk                                 |  2 ++
 package/luajit/luajit.mk                           |  2 ++
 package/mesa3d/mesa3d.mk                           |  5 +++++
 package/pkg-generic.mk                             | 26 ++++++++++++++++++++++
 package/rpi-userland/rpi-userland.mk               |  2 ++
 package/sunxi-mali/sunxi-mali.mk                   |  1 +
 package/systemd/systemd.mk                         |  2 ++
 package/ti-gfx/ti-gfx.mk                           |  1 +
 13 files changed, 60 insertions(+), 3 deletions(-)
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#1 lore raw

PATCH 1/3 infra/pkg-virtual: validate only one provider provides an implementation

From: "Yann E. MORIN" <[email protected]>

Currently, it is possible that more than one provider of a virtual package is selected in the menuconfig.

This leads to autobuild failures, and we do not protect the user from making a mistake in the configuration. The failure is then hard to troubleshoot in any case.

We can't use kconfig constructs to prevent this, since kconfig does not tell how many options did a select on another option.

This change introduces a new variable a provider *must* define to include all the virtual packages it is an implementation of. Then, when evaluating the package's rules, we check that the provider is indeed the declared one for each virtual package it claims to be an implementation of.

This works by taking advantage that when more than one provider is selected, only one of them will 'win' in setting the _PROVIDES_FOO option. Thus any provider just has to check it is indeed the declared provider. If not, it means that one or more other provider is selected.

This gives the opportunity to the user to change its configuration, and we can match the error message in the autobuilders to skip those failures (we can skip them instead of reporting them, since they are obviously configuration errors that should not happen in the first place.)

[Note: kudos to Arnout for suggesting this actual implementation. :-)]

Fixes:

    http://autobuild.buildroot.org/results/285/2851069d6964aa46d26b4aabe7d84e8c0c6c72ce
    http://autobuild.buildroot.net/results/9b7/9b7870354d70e27e42d3d9c1f131ab54706bf20e
    [...]
Cc Thomas Petazzoni <[email protected]>
Cc Peter Korsgaard <[email protected]>
Cc Thomas De Schampheleire <[email protected]>
Cc Arnout Vandecappelle <[email protected]>
Cc Samuel Martin <[email protected]>
---
 package/pkg-generic.mk | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
package/pkg-generic.mk +26 -0
index 6eca6d4..4e84e9a 100644
@@ -241,6 +241,25 @@ $(BUILD_DIR)/%/.stamp_dircleaned:
 	rm -Rf $(@D)
 
 ################################################################################
+# virt-provides-single -- check that provider-pkg is the declared provider for
+# the virtual package virt-pkg
+#
+# argument 1 is the lower-case name of the virtual package
+# argument 2 is the upper-case name of the virtual package
+# argument 3 is the lower-case name of the provider
+#
+# example:
+#   $(call virt-provides-single,libegl,LIBEGL,rpi-userland)
+################################################################################
+define virt-provides-single
+ifneq ($$(call qstrip,$$(BR2_PACKAGE_PROVIDES_$(2))),$(3))
+$$(error Configuration error: both "$(3)" and $$(BR2_PACKAGE_PROVIDES_$(2))\
+are selected as providers for virtual package "$(1)". Only one provider can\
+be selected at a time. Please fix your configuration)
+endif
+endef
+
+################################################################################
 # inner-generic-package -- generates the make targets needed to build a
 # generic package
 #
@@ -626,6 +645,13 @@ endif # ifneq ($(call qstrip,$$($(2)_SOURCE)),)
 # configuration
 ifeq ($$($$($(2)_KCONFIG_VAR)),y)
 
+# Ensure the calling package is the declared provider for all the virtual
+# packages it claims to be an implementation of.
+ifneq ($$($(2)_PROVIDES),)
+$$(foreach pkg,$$($(2)_PROVIDES),\
+	$$(eval $$(call virt-provides-single,$$(pkg),$$(call UPPERCASE,$$(pkg)),$(1))$$(sep)))
+endif
+
 TARGETS += $(1)
 PACKAGES_PERMISSIONS_TABLE += $$($(2)_PERMISSIONS)$$(sep)
 PACKAGES_DEVICES_TABLE += $$($(2)_DEVICES)$$(sep)
Signature · 1 line
1.8.3.2
#2 lore raw

PATCH 2/3 docs/manual: document the new FOO_PROVIDES variable

In reply to Yann E. MORIN

From: "Yann E. MORIN" <[email protected]>

Cc Thomas Petazzoni <[email protected]>
Cc Peter Korsgaard <[email protected]>
Cc Thomas De Schampheleire <[email protected]>
Cc Arnout Vandecappelle <[email protected]>
Cc Samuel Martin <[email protected]>
---
 docs/manual/adding-packages-generic.txt |  3 +++
 docs/manual/adding-packages-virtual.txt | 15 ++++++++++++---
 2 files changed, 15 insertions(+), 3 deletions(-)
docs/manual/adding-packages-generic.txt +3 -0
index f421651..d3fa45b 100644
@@ -304,6 +304,9 @@ information is (assuming the package name is +libfoo+) :
   a similar way, +HOST_LIBFOO_DEPENDENCIES+ lists the dependencies for
   the current host package.
 
+* +LIBFOO_PROVIDES+ lists all the virtual packages +libfoo+ is an
+  implementation of. See xref:virtual-package-tutorial[].
+
 * +LIBFOO_INSTALL_STAGING+ can be set to +YES+ or +NO+ (default). If
   set to +YES+, then the commands in the +LIBFOO_INSTALL_STAGING_CMDS+
   variables are executed to install the package into the staging
docs/manual/adding-packages-virtual.txt +12 -3
index 1c1116f..0cbe753 100644
@@ -39,7 +39,7 @@ In this file, we declare two options, +BR2_PACKAGE_HAS_SOMETHING_VIRTUAL+ and
 +BR2_PACKAGE_PROVIDES_SOMETHING_VIRTUAL+, whose values will be used by the
 providers.
 
-==== Virtual package's +*.mk+ file
+==== Virtual package's +.mk+ file
 
 The +.mk+ for the virtual package should just evaluate the +virtual-package+ macro:
 
@@ -59,8 +59,7 @@ The ability to have target and host packages is also available, with the
 ==== Provider's +Config.in+ file
 
 When adding a package as a provider, only the +Config.in+ file requires some
-modifications. The +*.mk+ file should follow the Buildroot infrastructure with
-no change at all.
+modifications.
 
 The +Config.in+ file of the package 'some-provider', which provides the
 functionalities of 'something-virtual', should contain:
@@ -84,6 +83,16 @@ On line 3, we select +BR2_PACKAGE_HAS_SOMETHING_VIRTUAL+, and on line 11, we
 set the value of +BR2_PACKAGE_PROVIDES_SOMETHING_VIRTUAL+ to the name of the
 provider, but only if it is selected.
 
+==== Provider's +.mk+ file
+
+The +.mk+ file should also declare an additional variable
++SOME_PROVIDER_PROVIDES+ to contain the names of all the virtual
+packages it is an implementation of:
+
+---------------------------
+01: SOME_PROVIDER_PROVIDES = something-virtual
+---------------------------
+
 Of course, do not forget to add the proper build and runtime dependencies for
 this package!
 
Signature · 1 line
1.8.3.2
#3 lore raw

PATCH 3/3 packages: make providers declare what they provide

In reply to Yann E. MORIN

From: "Yann E. MORIN" <[email protected]>

Fixes:

    http://autobuild.buildroot.org/results/285/2851069d6964aa46d26b4aabe7d84e8c0c6c72ce
    http://autobuild.buildroot.net/results/9b7/9b7870354d70e27e42d3d9c1f131ab54706bf20e
    [...]
Cc Thomas Petazzoni <[email protected]>
Cc Peter Korsgaard <[email protected]>
Cc Thomas De Schampheleire <[email protected]>
Cc Arnout Vandecappelle <[email protected]>
Cc Samuel Martin <[email protected]>
---
 package/bellagio/bellagio.mk                               | 1 +
 package/eudev/eudev.mk                                     | 1 +
 package/freescale-imx/gpu-viv-bin-mx6q/gpu-viv-bin-mx6q.mk | 2 ++
 package/lua/lua.mk                                         | 2 ++
 package/luajit/luajit.mk                                   | 2 ++
 package/mesa3d/mesa3d.mk                                   | 5 +++++
 package/rpi-userland/rpi-userland.mk                       | 2 ++
 package/sunxi-mali/sunxi-mali.mk                           | 1 +
 package/systemd/systemd.mk                                 | 2 ++
 package/ti-gfx/ti-gfx.mk                                   | 1 +
 10 files changed, 19 insertions(+)
package/bellagio/bellagio.mk +1 -0
index 993015c..6db753d 100644
@@ -11,5 +11,6 @@ BELLAGIO_AUTORECONF = YES
 BELLAGIO_INSTALL_STAGING = YES
 BELLAGIO_LICENSE = LGPLv2.1+
 BELLAGIO_LICENSE_FILES = COPYING
+BELLAGIO_PROVIDES = libopenmax
 
 $(eval $(autotools-package))
package/eudev/eudev.mk +1 -0
index 8e5e83d..720c2e9 100644
@@ -25,6 +25,7 @@ EUDEV_CONF_OPT =		\
 	--enable-libkmod
 
 EUDEV_DEPENDENCIES = host-gperf host-pkgconf util-linux kmod
+EUDEV_PROVIDES = udev
 
 ifeq ($(BR2_PACKAGE_EUDEV_RULES_GEN),y)
 EUDEV_CONF_OPT += --enable-rule_generator
package/freescale-imx/gpu-viv-bin-mx6q/gpu-viv-bin-mx6q.mk +2 -0
index e799fd2..49a8287 100644
@@ -23,6 +23,8 @@ GPU_VIV_BIN_MX6Q_LICENSE = Freescale Semiconductor Software License Agreement
 # for which a separate license is needed...
 GPU_VIV_BIN_MX6Q_REDISTRIBUTE = NO
 
+GPU_VIV_BIN_MX6Q_PROVIDES = libegl libgles
+
 # DirectFB is not supported (wrong version)
 ifeq ($(BR2_PACKAGE_XORG7),y)
 GPU_VIV_BIN_MX6Q_DEPENDENCIES = xlib_libXdamage xlib_libXext
package/lua/lua.mk +2 -0
index bc8799a..4f940ab 100644
@@ -14,6 +14,8 @@ LUA_INSTALL_STAGING = YES
 LUA_LICENSE = MIT
 LUA_LICENSE_FILES = COPYRIGHT
 
+LUA_PROVIDES = lua
+
 LUA_CFLAGS = -Wall -fPIC -DLUA_USE_POSIX
 
 ifeq ($(BR2_PACKAGE_LUA_5_2),y)
package/luajit/luajit.mk +2 -0
index c4b065d..61ea33b 100644
@@ -12,6 +12,8 @@ LUAJIT_LICENSE_FILES = COPYRIGHT
 
 LUAJIT_INSTALL_STAGING = YES
 
+LUAJIT_PROVIDES = lua
+
 ifneq ($(BR2_LARGEFILE),y)
 LUAJIT_NO_LARGEFILE = TARGET_LFSFLAGS=
 endif
package/mesa3d/mesa3d.mk +5 -0
index 63ba574..fa17177 100644
@@ -13,6 +13,8 @@ MESA3D_AUTORECONF = YES
 
 MESA3D_INSTALL_STAGING = YES
 
+MESA3D_PROVIDES =
+
 MESA3D_DEPENDENCIES = \
 	expat \
 	host-bison \
@@ -71,6 +73,7 @@ ifeq ($(MESA3D_DRI_DRIVERS-y),)
 MESA3D_CONF_OPT += \
 	--without-dri-drivers
 else
+MESA3D_PROVIDES += libgl
 MESA3D_CONF_OPT += \
 	--enable-dri \
 	--enable-shared-glapi \
@@ -85,6 +88,7 @@ endif
 MESA3D_CONF_OPT += --enable-opengl
 
 ifeq ($(BR2_PACKAGE_MESA3D_OPENGL_EGL),y)
+MESA3D_PROVIDES += libegl
 # egl depends on gbm, gbm depends on udev
 MESA3D_DEPENDENCIES += udev
 MESA3D_EGL_PLATFORMS = drm
@@ -105,6 +109,7 @@ MESA3D_CONF_OPT += \
 endif
 
 ifeq ($(BR2_PACKAGE_MESA3D_OPENGL_ES),y)
+MESA3D_PROVIDES += libgles
 MESA3D_CONF_OPT += --enable-gles1 --enable-gles2
 else
 MESA3D_CONF_OPT += --disable-gles1 --disable-gles2
package/rpi-userland/rpi-userland.mk +2 -0
index f6e4443..690fd1e 100644
@@ -11,6 +11,8 @@ RPI_USERLAND_LICENSE_FILES = LICENCE
 RPI_USERLAND_INSTALL_STAGING = YES
 RPI_USERLAND_CONF_OPT = -DVMCS_INSTALL_PREFIX=/usr -DBUILD_SHARED_LIBS=OFF
 
+RPI_USERLAND_PROVIDES = libegl libgles libopenmax libopenvg
+
 define RPI_USERLAND_POST_TARGET_CLEANUP
     rm -Rf $(TARGET_DIR)/usr/src
 endef
package/sunxi-mali/sunxi-mali.mk +1 -0
index c625c6e..c6900c8 100644
@@ -9,6 +9,7 @@ SUNXI_MALI_SITE = $(call github,linux-sunxi,sunxi-mali,$(SUNXI_MALI_VERSION))
 
 SUNXI_MALI_INSTALL_STAGING = YES
 SUNXI_MALI_DEPENDENCIES = sunxi-mali-prop
+SUNXI_MALI_PROVIDES = libegl libgles
 
 # The options below must be provided in the environment.  Providing these
 # through options overrides the value and prevents the makefiles from
package/systemd/systemd.mk +2 -0
index f7661ab..e1c984f 100644
@@ -17,6 +17,8 @@ SYSTEMD_DEPENDENCIES = \
 	kmod \
 	host-gperf
 
+SYSTEMD_PROVIDES = udev
+
 # Make sure that systemd will always be built after busybox so that we have
 # a consistent init setup between two builds
 ifeq ($(BR2_PACKAGE_BUSYBOX),y)
package/ti-gfx/ti-gfx.mk +1 -0
index e48715d..324e19c 100644
@@ -19,6 +19,7 @@ TI_GFX_LICENSE_FILES = TSPA.txt
 TI_GFX_INSTALL_STAGING = YES
 
 TI_GFX_DEPENDENCIES = linux
+TI_GFX_PROVIDES = libegl libgles powervr
 
 ifeq ($(BR2_PACKAGE_TI_GFX_ES3),y)
 TI_GFX_OMAPES = 3.x
Signature · 1 line
1.8.3.2
#4 lore raw

PATCH 3/3 packages: make providers declare what they provide

Dear Yann E. MORIN,

package/lua/lua.mk LUA_INSTALL_STAGING = YES · line 17

Quoted 12 lines · Yann E. MORIN

# DirectFB is not supported (wrong version)
ifeq ($(BR2_PACKAGE_XORG7),y) GPU_VIV_BIN_MX6Q_DEPENDENCIES = xlib_libXdamage xlib_libXext

package/lua/lua.mk +1 -0
index bc8799a..4f940ab 100644
@@ -14,6 +14,8 @@ LUA_INSTALL_STAGING = YES
 LUA_LICENSE = MIT
 LUA_LICENSE_FILES = COPYRIGHT
 
+LUA_PROVIDES = lua

This should be:

LUA_PROVIDES = luainterpreter

package/lua/lua.mk LUA_INSTALL_STAGING = YES · line 18

Quoted 1 line
package/lua/lua.mk +1 -0
+LUAJIT_PROVIDES = lua

Same here.

I was surprised by "LUA_PROVIDES = lua". How can a real package provide itself as a provider for itself as a virtual package ? :-)

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#5 lore raw

PATCH 3/3 packages: make providers declare what they provide

Thomas, All,

On 2014-05-15 09:40 +0200, Thomas Petazzoni spake thusly:

Quoted 12 lines · Yann E. MORIN
Quoted 11 lines

# DirectFB is not supported (wrong version)
ifeq ($(BR2_PACKAGE_XORG7),y) GPU_VIV_BIN_MX6Q_DEPENDENCIES = xlib_libXdamage xlib_libXext

package/lua/lua.mk +0 -0
index bc8799a..4f940ab 100644
@@ -14,6 +14,8 @@ LUA_INSTALL_STAGING = YES
 LUA_LICENSE = MIT
 LUA_LICENSE_FILES = COPYRIGHT
 
Quoted 5 lines

package/lua/lua.mk LUA_INSTALL_STAGING = YES · line 17

Quoted 1 line
package/lua/lua.mk +1 -0
+LUA_PROVIDES = lua

This should be:

LUA_PROVIDES = luainterpreter

Obviously, indeed.

Quoted 6 lines

package/lua/lua.mk LUA_INSTALL_STAGING = YES · line 18

Quoted 1 line
package/lua/lua.mk +1 -0
+LUAJIT_PROVIDES = lua

Same here.

I was surprised by "LUA_PROVIDES = lua". How can a real package provide itself as a provider for itself as a virtual package ? :-)

Hehe! :-)

I just did test the openGL virtual packages, and forgot to test lua et al. I'll fix and respin.

Thanks for noticing.

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'