Add tainting support to buildroot
v3 cover Add tainting support to buildroot
Packages that need to resolve dependencies internally and use a package manager would harm the reproducibility of a build, moreover they escape the legal infrastructure not giving enough informations on licensing.
This patch adds a tainting mechanism in the form of a variable FOO_TAINTS that can be used to signal that a package harms the reproducibility or licensing under certain conditions.
This opens the door to include per language dependency managers in buildroot.
Angelo Compagnucci (3):
Makefile: add tainting support
docs/manual: adding infos about tainting package/nodejs: taint the build on external modules
Makefile | 10 ++++++++++ docs/manual/adding-packages-generic.txt | 6 ++++++ docs/manual/legal-notice.txt | 12 ++++++++++++ package/nodejs/nodejs.mk | 1 + package/pkg-generic.mk | 8 ++++++++ 5 files changed, 37 insertions(+)
Signature · 1 line
2.7.4
PATCH v3 1/3 Makefile: add tainting support
From: Angelo Compagnucci <[email protected]>
Packages who harms the build reproducibility or licensing can declare FOO_TAINTS variable. If a package taints the build it will be added to a list of tainting packages. The build ends with a warning if the tainting packages list is not empty. Moreover, legal info will show a warning in presence of a tainting package.
Angelo Compagnucci <[email protected]>
--- Makefile | 10 ++++++++++ package/pkg-generic.mk | 8 ++++++++ 2 files changed, 18 insertions(+)
Makefile +10 -0
@@ -758,12 +758,21 @@ endif touch $(TARGET_DIR)/usr +.PHONY: check-tainted +check-tainted: +ifneq ($(BR2_TAINTED_BY),) + $(error Your buildroot configuration is tainted by: $(BR2_TAINTED_BY)) +else + @echo "Your buildroot configuration is not tainted" +endif + .PHONY: target-post-image target-post-image: $(TARGETS_ROOTFS) target-finalize @rm -f $(ROOTFS_COMMON_TAR) @$(foreach s, $(call qstrip,$(BR2_ROOTFS_POST_IMAGE_SCRIPT)), \ $(call MESSAGE,"Executing post-image script $(s)"); \ $(EXTRA_ENV) $(s) $(BINARIES_DIR) $(call qstrip,$(BR2_ROOTFS_POST_SCRIPT_ARGS))$(sep)) + $(if $(BR2_TAINTED_BY), at echo "WARNING: Your buildroot configuration is tainted by: $(BR2_TAINTED_BY).") .PHONY: source source: $(foreach p,$(PACKAGES),$(p)-all-source)
@@ -1070,6 +1079,7 @@ help: @echo ' source - download all sources needed for offline-build' @echo ' external-deps - list external packages used' @echo ' legal-info - generate info about license compliance' + @echo ' check-tainted - check if any selected package harms build reproducibility or licensing' @echo ' printvars - dump all the internal variables' @echo @echo ' make V=0|1 - 0 => quiet build (default), 1 => verbose build'
package/pkg-generic.mk +8 -0
@@ -551,6 +551,10 @@ ifndef $(2)_REDISTRIBUTE endif endif +ifdef $(2)_TAINTS +BR2_TAINTED_BY+=$$($(2)_RAWNAME) +endif + $(2)_REDISTRIBUTE ?= YES $(2)_REDIST_SOURCES_DIR = $$(REDIST_SOURCES_DIR_$$(call UPPERCASE,$(4)))/$$($(2)_BASENAME_RAW)
@@ -909,6 +913,10 @@ else $(Q)$$(foreach F,$$($(2)_LICENSE_FILES),$$(call legal-license-file,$$($(2)_RAWNAME),$$($(2)_BASENAME_RAW),$$($(2)_PKGDIR),$$(F),$$($(2)_DIR)/$$(F),$$(call UPPERCASE,$(4)))$$(sep)) endif # license files +ifeq ($$(call qstrip,$$($(2)_TAINTS)),YES) + $(Q)$$(call legal-warning-pkg,$$($(2)_RAWNAME),unknown license for additional modules or dependencies) +endif + ifeq ($$($(2)_SITE_METHOD),local) # Packages without a tarball: don't save and warn @$$(call legal-warning-nosource,$$($(2)_RAWNAME),local)
Signature · 1 line
2.7.4
PATCH v3 1/3 Makefile: add tainting support
Hello,
+Yann/Arnout in Cc.
package/pkg-generic.mk ifndef $(2)_REDISTRIBUTE · line 556
Quoted 11 lines · Angelo Compagnucci
package/pkg-generic.mk +3
-0@@ -551,6 +551,10 @@ ifndef $(2)_REDISTRIBUTE endif endif +ifdef $(2)_TAINTS +BR2_TAINTED_BY+=$$($(2)_RAWNAME) +endif
I was almost going to apply and push this, but then I found a fairly major problem. With your implementation, as soon as a package sets <pkg>_TAINTS = YES, it will be part of the BR2_TAINTED_BY list, regardless of whether the package is enabled or not.
It happens to work OK with nodejs, because the NODEJS_TAINTS = YES is inside a condition that NODEJS_MODULES_LIST != "". However, if you put NODEJS_TAINTS = YES outside of that condition in nodejs.mk, you will see that the build is always tainted by nodejs, regardless of whether nodejs is enabled or not. Not good.
This can easily be fixed by moving the BR2_TAINTED_BY += line inside the:
ifeq ($$($$($(2)_KCONFIG_VAR)),y)
condition, which is true only when the package is really enabled.
However, this only works for target packages. What about host packages ? Can they taint the build ? If so, how do we handle that ?
When I was about to apply, I had added something like this in pkg-generic.mk:
ifndef $(2)_TAINTS
ifdef $(3)_TAINTS
$(2)_TAINTS = $$($(3)_TAINTS)
endif
endif
so that HOST_<pkg>_TAINTS is automatically defined to the same value as <pkg>_TAINTS. However, we have no way to know if a host package is enabled or not, since most host packages don't have any corresponding Config.in option.
So: do we care about host packages for "tainting" ? If we do care, how do we handle this ?
Best regards,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH v3 1/3 Makefile: add tainting support
Hi Thomas,
Quoted 23 lines · Thomas Petazzoni
Hello,
+Yann/Arnout in Cc.
package/pkg-generic.mk ifndef $(2)_REDISTRIBUTE · line 556
Quoted 11 lines · Angelo Compagnucci
package/pkg-generic.mk +3
-0@@ -551,6 +551,10 @@ ifndef $(2)_REDISTRIBUTE endif endif +ifdef $(2)_TAINTS +BR2_TAINTED_BY+=$$($(2)_RAWNAME) +endifI was almost going to apply and push this, but then I found a fairly major problem. With your implementation, as soon as a package sets <pkg>_TAINTS = YES, it will be part of the
BR2_TAINTED_BYlist, regardless of whether the package is enabled or not.
Quoted 5 lines
It happens to work OK with nodejs, because the
NODEJS_TAINTS= YES is inside a condition thatNODEJS_MODULES_LIST!= "". However, if you putNODEJS_TAINTS= YES outside of that condition in nodejs.mk, you will see that the build is always tainted by nodejs, regardless of whether nodejs is enabled or not. Not good.
Thank you so much for finding this.
Quoted 6 lines
This can easily be fixed by moving the
BR2_TAINTED_BY+= line inside the:ifeq ($$($$($(2)_KCONFIG_VAR)),y)condition, which is true only when the package is really enabled.
Yes, it works like a charm!
Quoted 12 lines
However, this only works for target packages. What about host packages ? Can they taint the build ? If so, how do we handle that ?
When I was about to apply, I had added something like this in pkg-generic.mk:
ifndef $(2)_TAINTS
ifdef $(3)_TAINTS
$(2)_TAINTS = $$($(3)_TAINTS)
endif
endif
Quoted 7 lines
so that HOST_<pkg>_TAINTS is automatically defined to the same value as <pkg>_TAINTS. However, we have no way to know if a host package is enabled or not, since most host packages don't have any corresponding Config.in option.
So: do we care about host packages for "tainting" ? If we do care, how do we handle this ?
I think we can demand the decision to when we will have a host package that needs tainting support.
Moreover, I don't think the demand for host packages that needs tainting support will be high in the future.
I'll send an updated patch.
Quoted 8 lines
Best regards,
Thomas
--
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
PATCH v3 1/3 Makefile: add tainting support
Hello,
Quoted 8 lines · Angelo Compagnucci
Quoted 2 lines
So: do we care about host packages for "tainting" ? If we do care, how do we handle this ?
I think we can demand the decision to when we will have a host package that needs tainting support.
Moreover, I don't think the demand for host packages that needs tainting support will be high in the future.
Then we need to make that explicit:
ifeq ($$($(2)_TYPE),host)
ifneq ($$($(2)_TAINTS),)
$$(error "Host package $(1) has $(2)_TAINTS set: not supported) endif
endif
of course, please check that this really works as expected.
Best regards,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH v3 1/3 Makefile: add tainting support
Hi Thomas,
Quoted 15 lines · Thomas Petazzoni
Hello,
Quoted 8 lines · Angelo Compagnucci
Quoted 2 lines
So: do we care about host packages for "tainting" ? If we do care, how do we handle this ?
I think we can demand the decision to when we will have a host package that needs tainting support.
Moreover, I don't think the demand for host packages that needs tainting support will be high in the future.
Then we need to make that explicit:
Quoted 7 lines
ifeq ($$($(2)_TYPE),host)
ifneq ($$($(2)_TAINTS),)
$$(error "Host package $(1) has $(2)_TAINTS set: not supported) endif
endifof course, please check that this really works as expected.
Done!
Quoted 8 lines
Best regards,
Thomas
--
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
PATCH v3 2/3 docs/manual: adding infos about tainting
In reply to Angelo CompagnucciAdding documentation about the usage of LIBFOO_TAINTS and "make check-tainted".
--- docs/manual/adding-packages-generic.txt | 6 ++++++ docs/manual/legal-notice.txt | 12 ++++++++++++ 2 files changed, 18 insertions(+)
docs/manual/adding-packages-generic.txt +6 -0
@@ -445,6 +445,12 @@ not and can not work as people would expect it should: to let you know, and +not saved+ will appear in the +license files+ field of the manifest file for this package. +* +LIBFOO_TAINTS+ shoud be set to YES if a package taints a Buildroot + configuration. A Buildroot configuration is tainted when a packages uses + external dependencies for which Buildroot cannot clearly recover licensing + informations. If a configuration is tainted, it means that the licensing + information produced by +make legal-info+ could not be accurate. + * +LIBFOO_ACTUAL_SOURCE_TARBALL+ only applies to packages whose +LIBFOO_SITE+ / +LIBTOO_SOURCE+ pair points to an archive that does not actually contain source code, but binary code. This a very
docs/manual/legal-notice.txt +12 -0
@@ -73,6 +73,18 @@ distribution is required). When you run +make legal-info+, Buildroot produces warnings in the +README+ file to inform you of relevant material that could not be saved. +Furthermore, a Buildroot configuration could be tainted from a package that uses +some custom external dependencies from the Buildroot tree. An example could be +a package manager for a software stack that downloads the required dependencies +during the building of a package. In such cases, Buildroot cannot check the +licensing of the downloaded software and thus giving accurate licensing +informations. +To check if your configuration is tainted, run: + +-------------------- +make check-tainted +-------------------- + Finally, keep in mind that the output of +make legal-info+ is based on declarative statements in each of the packages recipes. The Buildroot developers try to do their best to keep those declarative statements as
Signature · 1 line
2.7.4
PATCH v3 3/3 package/nodejs: taint the build on external modules
In reply to Angelo CompagnucciFrom: Angelo Compagnucci <[email protected]>
This patch enables the tainting of the build when an external module is added.
--- package/nodejs/nodejs.mk | 1 + 1 file changed, 1 insertion(+)
package/nodejs/nodejs.mk +1 -0
@@ -160,6 +160,7 @@ NPM = $(TARGET_CONFIGURE_OPTS) \ # We can only call NPM if there's something to install. # ifneq ($(NODEJS_MODULES_LIST),) +NODEJS_TAINTS = YES define NODEJS_INSTALL_MODULES # If you're having trouble with module installation, adding -d to the # npm install call below and setting npm_config_rollback=false can both
Signature · 1 line
2.7.4