Skip to contentSkip to conversation

Add tainting support to buildroot

PATCH Superseded Angelo Compagnucci 2 participants 8 messages Sep 5, 2018 Kbuild

v3 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v3 cover Add tainting support to buildroot

Packages that need to resolve dependencies internally and use a package manager would harm the reproducibility of a build, moreover they escape the legal infrastructure not giving enough informations on licensing.

This patch adds a tainting mechanism in the form of a variable FOO_TAINTS that can be used to signal that a package harms the reproducibility or licensing under certain conditions.

This opens the door to include per language dependency managers in buildroot.

Angelo Compagnucci (3):
Makefile: add tainting support
docs/manual: adding infos about tainting package/nodejs: taint the build on external modules

 Makefile                                | 10 ++++++++++
 docs/manual/adding-packages-generic.txt |  6 ++++++
 docs/manual/legal-notice.txt            | 12 ++++++++++++
 package/nodejs/nodejs.mk                |  1 +
 package/pkg-generic.mk                  |  8 ++++++++
 5 files changed, 37 insertions(+)
Signature · 1 line
2.7.4
#1 lore raw

PATCH v3 1/3 Makefile: add tainting support

From: Angelo Compagnucci <[email protected]>

Packages who harms the build reproducibility or licensing can declare FOO_TAINTS variable. If a package taints the build it will be added to a list of tainting packages. The build ends with a warning if the tainting packages list is not empty. Moreover, legal info will show a warning in presence of a tainting package.

Angelo Compagnucci <[email protected]>

---
 Makefile               | 10 ++++++++++
 package/pkg-generic.mk |  8 ++++++++
 2 files changed, 18 insertions(+)
Makefile +10 -0
index 9d66bba..ad61130 100644
@@ -758,12 +758,21 @@ endif
 
 	touch $(TARGET_DIR)/usr
 
+.PHONY: check-tainted
+check-tainted:
+ifneq ($(BR2_TAINTED_BY),)
+	$(error Your buildroot configuration is tainted by: $(BR2_TAINTED_BY))
+else
+	@echo "Your buildroot configuration is not tainted"
+endif
+
 .PHONY: target-post-image
 target-post-image: $(TARGETS_ROOTFS) target-finalize
 	@rm -f $(ROOTFS_COMMON_TAR)
 	@$(foreach s, $(call qstrip,$(BR2_ROOTFS_POST_IMAGE_SCRIPT)), \
 		$(call MESSAGE,"Executing post-image script $(s)"); \
 		$(EXTRA_ENV) $(s) $(BINARIES_DIR) $(call qstrip,$(BR2_ROOTFS_POST_SCRIPT_ARGS))$(sep))
+	$(if $(BR2_TAINTED_BY), at echo "WARNING: Your buildroot configuration is tainted by: $(BR2_TAINTED_BY).")
 
 .PHONY: source
 source: $(foreach p,$(PACKAGES),$(p)-all-source)
@@ -1070,6 +1079,7 @@ help:
 	@echo '  source                 - download all sources needed for offline-build'
 	@echo '  external-deps          - list external packages used'
 	@echo '  legal-info             - generate info about license compliance'
+	@echo '  check-tainted          - check if any selected package harms build reproducibility or licensing'
 	@echo '  printvars              - dump all the internal variables'
 	@echo
 	@echo '  make V=0|1             - 0 => quiet build (default), 1 => verbose build'
package/pkg-generic.mk +8 -0
index 91b61c6..bcb4acd 100644
@@ -551,6 +551,10 @@ ifndef $(2)_REDISTRIBUTE
  endif
 endif
 
+ifdef $(2)_TAINTS
+BR2_TAINTED_BY+=$$($(2)_RAWNAME)
+endif
+
 $(2)_REDISTRIBUTE		?= YES
 
 $(2)_REDIST_SOURCES_DIR = $$(REDIST_SOURCES_DIR_$$(call UPPERCASE,$(4)))/$$($(2)_BASENAME_RAW)
@@ -909,6 +913,10 @@ else
 	$(Q)$$(foreach F,$$($(2)_LICENSE_FILES),$$(call legal-license-file,$$($(2)_RAWNAME),$$($(2)_BASENAME_RAW),$$($(2)_PKGDIR),$$(F),$$($(2)_DIR)/$$(F),$$(call UPPERCASE,$(4)))$$(sep))
 endif # license files
 
+ifeq ($$(call qstrip,$$($(2)_TAINTS)),YES)
+	$(Q)$$(call legal-warning-pkg,$$($(2)_RAWNAME),unknown license for additional modules or dependencies)
+endif
+
 ifeq ($$($(2)_SITE_METHOD),local)
 # Packages without a tarball: don't save and warn
 	@$$(call legal-warning-nosource,$$($(2)_RAWNAME),local)
Signature · 1 line
2.7.4
#2 lore raw

PATCH v3 1/3 Makefile: add tainting support

Hello,

+Yann/Arnout in Cc.

package/pkg-generic.mk ifndef $(2)_REDISTRIBUTE · line 556

Quoted 11 lines · Angelo Compagnucci
package/pkg-generic.mk +3 -0
index 91b61c6..bcb4acd 100644
@@ -551,6 +551,10 @@ ifndef $(2)_REDISTRIBUTE
  endif
 endif
 
+ifdef $(2)_TAINTS
+BR2_TAINTED_BY+=$$($(2)_RAWNAME)
+endif

I was almost going to apply and push this, but then I found a fairly major problem. With your implementation, as soon as a package sets <pkg>_TAINTS = YES, it will be part of the BR2_TAINTED_BY list, regardless of whether the package is enabled or not.

It happens to work OK with nodejs, because the NODEJS_TAINTS = YES is inside a condition that NODEJS_MODULES_LIST != "". However, if you put NODEJS_TAINTS = YES outside of that condition in nodejs.mk, you will see that the build is always tainted by nodejs, regardless of whether nodejs is enabled or not. Not good.

This can easily be fixed by moving the BR2_TAINTED_BY += line inside the:

  ifeq ($$($$($(2)_KCONFIG_VAR)),y)

condition, which is true only when the package is really enabled.

However, this only works for target packages. What about host packages ? Can they taint the build ? If so, how do we handle that ?

When I was about to apply, I had added something like this in pkg-generic.mk:

ifndef $(2)_TAINTS
ifdef $(3)_TAINTS
$(2)_TAINTS = $$($(3)_TAINTS)
endif
endif

so that HOST_<pkg>_TAINTS is automatically defined to the same value as <pkg>_TAINTS. However, we have no way to know if a host package is enabled or not, since most host packages don't have any corresponding Config.in option.

So: do we care about host packages for "tainting" ? If we do care, how do we handle this ?

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#3 lore raw

PATCH v3 1/3 Makefile: add tainting support

Hi Thomas,

Quoted 23 lines · Thomas Petazzoni

Hello,

+Yann/Arnout in Cc.

package/pkg-generic.mk ifndef $(2)_REDISTRIBUTE · line 556

Quoted 11 lines · Angelo Compagnucci
package/pkg-generic.mk +3 -0
index 91b61c6..bcb4acd 100644
@@ -551,6 +551,10 @@ ifndef $(2)_REDISTRIBUTE
  endif
 endif
 
+ifdef $(2)_TAINTS
+BR2_TAINTED_BY+=$$($(2)_RAWNAME)
+endif

I was almost going to apply and push this, but then I found a fairly major problem. With your implementation, as soon as a package sets <pkg>_TAINTS = YES, it will be part of the BR2_TAINTED_BY list, regardless of whether the package is enabled or not.

Quoted 5 lines

It happens to work OK with nodejs, because the NODEJS_TAINTS = YES is inside a condition that NODEJS_MODULES_LIST != "". However, if you put NODEJS_TAINTS = YES outside of that condition in nodejs.mk, you will see that the build is always tainted by nodejs, regardless of whether nodejs is enabled or not. Not good.

Thank you so much for finding this.

Quoted 6 lines

This can easily be fixed by moving the BR2_TAINTED_BY += line inside the:

  ifeq ($$($$($(2)_KCONFIG_VAR)),y)

condition, which is true only when the package is really enabled.

Yes, it works like a charm!

Quoted 12 lines

However, this only works for target packages. What about host packages ? Can they taint the build ? If so, how do we handle that ?

When I was about to apply, I had added something like this in pkg-generic.mk:

ifndef $(2)_TAINTS
ifdef $(3)_TAINTS
$(2)_TAINTS = $$($(3)_TAINTS)
endif
endif

Quoted 7 lines

so that HOST_<pkg>_TAINTS is automatically defined to the same value as <pkg>_TAINTS. However, we have no way to know if a host package is enabled or not, since most host packages don't have any corresponding Config.in option.

So: do we care about host packages for "tainting" ? If we do care, how do we handle this ?

I think we can demand the decision to when we will have a host package that needs tainting support.

Moreover, I don't think the demand for host packages that needs tainting support will be high in the future.

I'll send an updated patch.

Quoted 8 lines

Best regards,

Thomas
--
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

#4 lore raw

PATCH v3 1/3 Makefile: add tainting support

Hello,

Quoted 8 lines · Angelo Compagnucci
Quoted 2 lines

So: do we care about host packages for "tainting" ? If we do care, how do we handle this ?

I think we can demand the decision to when we will have a host package that needs tainting support.

Moreover, I don't think the demand for host packages that needs tainting support will be high in the future.

Then we need to make that explicit:

ifeq ($$($(2)_TYPE),host)
ifneq ($$($(2)_TAINTS),)
$$(error "Host package $(1) has $(2)_TAINTS set: not supported) endif
endif

of course, please check that this really works as expected.

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#5 lore raw

PATCH v3 1/3 Makefile: add tainting support

Hi Thomas,

Quoted 15 lines · Thomas Petazzoni

Hello,

Quoted 8 lines · Angelo Compagnucci
Quoted 2 lines

So: do we care about host packages for "tainting" ? If we do care, how do we handle this ?

I think we can demand the decision to when we will have a host package that needs tainting support.

Moreover, I don't think the demand for host packages that needs tainting support will be high in the future.

Then we need to make that explicit:

Quoted 7 lines

ifeq ($$($(2)_TYPE),host)
ifneq ($$($(2)_TAINTS),)
$$(error "Host package $(1) has $(2)_TAINTS set: not supported) endif
endif

of course, please check that this really works as expected.

Done!

Quoted 8 lines

Best regards,

Thomas
--
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

#6 lore raw

PATCH v3 2/3 docs/manual: adding infos about tainting

In reply to Angelo Compagnucci

Adding documentation about the usage of LIBFOO_TAINTS and "make check-tainted".

---
 docs/manual/adding-packages-generic.txt |  6 ++++++
 docs/manual/legal-notice.txt            | 12 ++++++++++++
 2 files changed, 18 insertions(+)
docs/manual/adding-packages-generic.txt +6 -0
index 7be1754..8f159ad 100644
@@ -445,6 +445,12 @@ not and can not work as people would expect it should:
   to let you know, and +not saved+ will appear in the +license files+ field
   of the manifest file for this package.
 
+* +LIBFOO_TAINTS+ shoud be set to YES if a package taints a Buildroot
+  configuration. A Buildroot configuration is tainted when a packages uses
+  external dependencies for which Buildroot cannot clearly recover licensing
+  informations. If a configuration is tainted, it means that the licensing
+  information produced by +make legal-info+ could not be accurate.
+
 * +LIBFOO_ACTUAL_SOURCE_TARBALL+ only applies to packages whose
   +LIBFOO_SITE+ / +LIBTOO_SOURCE+ pair points to an archive that does
   not actually contain source code, but binary code. This a very
docs/manual/legal-notice.txt +12 -0
index 6975328..7fde09a 100644
@@ -73,6 +73,18 @@ distribution is required).
 When you run +make legal-info+, Buildroot produces warnings in the +README+
 file to inform you of relevant material that could not be saved.
 
+Furthermore, a Buildroot configuration could be tainted from a package that uses
+some custom external dependencies from the Buildroot tree. An example could be
+a package manager for a software stack that downloads the required dependencies
+during the building of a package. In such cases, Buildroot cannot check the
+licensing of the downloaded software and thus giving accurate licensing
+informations.
+To check if your configuration is tainted, run:
+
+--------------------
+make check-tainted
+--------------------
+
 Finally, keep in mind that the output of +make legal-info+ is based on
 declarative statements in each of the packages recipes. The Buildroot
 developers try to do their best to keep those declarative statements as
Signature · 1 line
2.7.4
#7 lore raw

PATCH v3 3/3 package/nodejs: taint the build on external modules

In reply to Angelo Compagnucci

From: Angelo Compagnucci <[email protected]>

This patch enables the tainting of the build when an external module is added.

---
 package/nodejs/nodejs.mk | 1 +
 1 file changed, 1 insertion(+)
package/nodejs/nodejs.mk +1 -0
index e2c94ba..322a1ec 100644
@@ -160,6 +160,7 @@ NPM = $(TARGET_CONFIGURE_OPTS) \
 # We can only call NPM if there's something to install.
 #
 ifneq ($(NODEJS_MODULES_LIST),)
+NODEJS_TAINTS = YES
 define NODEJS_INSTALL_MODULES
 	# If you're having trouble with module installation, adding -d to the
 	# npm install call below and setting npm_config_rollback=false can both
Signature · 1 line
2.7.4