legal info: split host and package output
PATCH v3 1/1 legal info: split host and package output
--- Makefile | 25 +++++++++++++++---------- docs/manual/legal-notice.txt | 21 ++++++++++++--------- package/pkg-generic.mk | 27 ++++++++++++++++++--------- package/pkg-utils.mk | 39 ++++++++++++++++++++++++++------------- 4 files changed, 71 insertions(+), 41 deletions(-)
PATCH v3 1/1 legal info: fix saving of host package licenses
Due to some tricky make behavior, the license texts of host packages that did not provide an explicit HOST_FOO_LICENSE_FILES definition was not saved. The problem is that it is not straightforward to use a variable defined/updated inside an evaluated block as input to a foreach statement. If you try to use $(FOO) then only the original value of FOO is used for foreach, any update inside the block is ignored. However, if you use $$(FOO), the entire contents of FOO (typically a list of items) is passed as one item to foreach, thus causing just one iteration instead of several.
From Arnout Vandecapelle's explanation:
Any variable referenced with a single $ inside the inner-generic-package macro is expanded before the resulting contents are eval'ed. Therefore, it is not possible to refer to variables defined by the inner-generic-package macro from within a single-$ function call.
To fix the problem, one should defer the evaluation of the entire block using double dollar signs.
Additionally, a few empty lines have been added to the legal-info-foo block for clarity.
---
v3:
- simplify solution based on Arnout's insight
v2:
- implement a solution for packages that have multiple license files.
- fix accidental mixup of code from another patch
package/pkg-generic.mk | 17 +++++++++++++---- 1 files changed, 13 insertions(+), 4 deletions(-)
package/pkg-generic.mk +13 -4
@@ -502,26 +502,35 @@ endif $(1)-legal-info: # Packages without a source are assumed to be part of Buildroot, skip them. ifneq ($(call qstrip,$$($(2)_SOURCE)),) + ifeq ($$($(2)_SITE_METHOD),local) # Packages without a tarball: don't save and warn @$(call legal-warning-pkg-savednothing,$$($(2)_RAWNAME),local) + else ifneq ($$($(2)_OVERRIDE_SRCDIR),) @$(call legal-warning-pkg-savednothing,$$($(2)_RAWNAME),override) + else # Other packages + # Save license files if defined ifeq ($(call qstrip,$$($(2)_LICENSE_FILES)),) @$(call legal-license-nofiles,$$($(2)_RAWNAME)) @$(call legal-warning-pkg,$$($(2)_RAWNAME),cannot save license ($(2)_LICENSE_FILES not defined)) else - @$(foreach F,$($(2)_LICENSE_FILES),$(call legal-license-file,$$($(2)_RAWNAME),$(F),$$($(2)_DIR)/$(F))$$(sep)) -endif +# Double dollar signs are really needed here, to catch host packages +# without explicit HOST_FOO_LICENSE_FILES assignment, also in case they +# have multiple license files. + @$$(foreach F,$$($(2)_LICENSE_FILES),$$(call legal-license-file,$$($(2)_RAWNAME),$$(F),$$($(2)_DIR)/$$(F))$$(sep)) +endif # license files + ifeq ($$($(2)_REDISTRIBUTE),YES) # Copy the source tarball (just hardlink if possible) @cp -l $(DL_DIR)/$$($(2)_SOURCE) $(REDIST_SOURCES_DIR) 2>/dev/null || \ cp $(DL_DIR)/$$($(2)_SOURCE) $(REDIST_SOURCES_DIR) -endif -endif +endif # redistribute + +endif # other packages @$(call legal-manifest,$$($(2)_RAWNAME),$$($(2)_VERSION),$$($(2)_LICENSE),$$($(2)_MANIFEST_LICENSE_FILES),$$($(2)_MANIFEST_TARBALL)) endif # ifneq ($(call qstrip,$$($(2)_SOURCE)),) $(foreach hook,$($(2)_POST_LEGAL_INFO_HOOKS),$(call $(hook))$(sep))
PATCH v3 1/1 legal info: fix saving of host package licenses
Quoted 15 lines · Thomas De Schampheleire
Due to some tricky make behavior, the license texts of host packages that did not provide an explicit
HOST_FOO_LICENSE_FILESdefinition was not saved. The problem is that it is not straightforward to use a variable defined/updated inside an evaluated block as input to a foreach statement. If you try to use $(FOO) then only the original value of FOO is used for foreach, any update inside the block is ignored. However, if you use $$(FOO), the entire contents of FOO (typically a list of items) is passed as one item to foreach, thus causing just one iteration instead of several.From Arnout Vandecapelle's explanation:
Any variable referenced with a single $ inside the inner-generic-package macro is expanded before the resulting contents are eval'ed. Therefore, it is not possible to refer to variables defined by the inner-generic-package macro from within a single-$ function call.
Quoted 7 lines
To fix the problem, one should defer the evaluation of the entire block using double dollar signs.
Additionally, a few empty lines have been added to the legal-info-foo block for clarity.
Signed-off-by Thomas De Schampheleire <[email protected]>
Signature · 1 line
Luca
PATCH v3 1/1 legal info: fix saving of host package licenses
In reply to Thomas De SchampheleireQuoted 20 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Thomas" == Thomas De Schampheleire <[email protected]> writes:
Due to some tricky make behavior, the license texts of host packages that did not provide an explicit
HOST_FOO_LICENSE_FILESdefinition was not saved. The problem is that it is not straightforward to use a variable defined/updated inside an evaluated block as input to a foreach statement. If you try to use $(FOO) then only the original value of FOO is used for foreach, any update inside the block is ignored. However, if you use $$(FOO), the entire contents of FOO (typically a list of items) is passed as one item to foreach, thus causing just one iteration instead of several.Quoted 1 line
From Arnout Vandecapelle's explanation:
Any variable referenced with a single $ inside the inner-generic-package macro is expanded before the resulting contents are eval'ed. Therefore, it is not possible to refer to variables defined by the inner-generic-package macro from within a single-$ function call.
To fix the problem, one should defer the evaluation of the entire block using double dollar signs.
Quoted 8 lines
Additionally, a few empty lines have been added to the legal-info-foo block for clarity.
Signed-off-by Thomas De Schampheleire <[email protected]>---
v3:
- simplify solution based on Arnout's insight
Committed, thanks.
Signature · 1 line
Bye, Peter Korsgaard
PATCH v3 1/1 legal info: split manifest for host and target
In reply to Thomas De Schampheleire---
v3: no changes
v2:
- trivial rebase after changes in first patch
- move out documentation to a separate patch
Makefile | 8 +++++--- package/pkg-generic.mk | 2 +- package/pkg-utils.mk | 2 +- 3 files changed, 7 insertions(+), 5 deletions(-)
Makefile +5 -3
@@ -110,7 +110,8 @@ HOST_DIR:=$(BASE_DIR)/host LEGAL_INFO_DIR=$(BASE_DIR)/legal-info REDIST_SOURCES_DIR=$(LEGAL_INFO_DIR)/sources LICENSE_FILES_DIR=$(LEGAL_INFO_DIR)/licenses -LEGAL_MANIFEST_CSV=$(LEGAL_INFO_DIR)/manifest.csv +LEGAL_MANIFEST_CSV_TARGET=$(LEGAL_INFO_DIR)/manifest.csv +LEGAL_MANIFEST_CSV_HOST=$(LEGAL_INFO_DIR)/host-manifest.csv LEGAL_LICENSES_TXT=$(LEGAL_INFO_DIR)/licenses.txt LEGAL_WARNINGS=$(LEGAL_INFO_DIR)/.warnings LEGAL_REPORT=$(LEGAL_INFO_DIR)/README
@@ -575,8 +576,9 @@ legal-info-clean: legal-info-prepare: $(LEGAL_INFO_DIR) @$(call MESSAGE,"Collecting legal info") @$(call legal-license-file,buildroot,COPYING,COPYING) - @$(call legal-manifest,PACKAGE,VERSION,LICENSE,LICENSE FILES,SOURCE ARCHIVE) - @$(call legal-manifest,buildroot,$(BR2_VERSION_FULL),GPLv2+,COPYING,not saved) + @$(call legal-manifest,PACKAGE,VERSION,LICENSE,LICENSE FILES,SOURCE ARCHIVE,TARGET) + @$(call legal-manifest,PACKAGE,VERSION,LICENSE,LICENSE FILES,SOURCE ARCHIVE,HOST) + @$(call legal-manifest,buildroot,$(BR2_VERSION_FULL),GPLv2+,COPYING,not saved,HOST) @$(call legal-warning,the Buildroot source code has not been saved) @$(call legal-warning,the toolchain has not been saved) @cp $(BUILDROOT_CONFIG) $(LEGAL_INFO_DIR)/buildroot.config
package/pkg-generic.mk +1 -1
@@ -531,7 +531,7 @@ ifeq ($$($(2)_REDISTRIBUTE),YES) endif # redistribute endif # other packages - @$(call legal-manifest,$$($(2)_RAWNAME),$$($(2)_VERSION),$$($(2)_LICENSE),$$($(2)_MANIFEST_LICENSE_FILES),$$($(2)_MANIFEST_TARBALL)) + @$(call legal-manifest,$$($(2)_RAWNAME),$$($(2)_VERSION),$$($(2)_LICENSE),$$($(2)_MANIFEST_LICENSE_FILES),$$($(2)_MANIFEST_TARBALL),$(call UPPERCASE,$(5))) endif # ifneq ($(call qstrip,$$($(2)_SOURCE)),) $(foreach hook,$($(2)_POST_LEGAL_INFO_HOOKS),$(call $(hook))$(sep))
package/pkg-utils.mk +1 -1
@@ -104,7 +104,7 @@ legal-warning-pkg=echo "WARNING: $(1): $ define legal-warning-pkg-savednothing # pkg, {local|override} $(call legal-warning-pkg,$(1),sources and license files not saved ($(2) packages not handled)) endef -legal-manifest=echo '"$(1)","$(2)","$(3)","$(4)","$(5)"' >>$(LEGAL_MANIFEST_CSV) +legal-manifest=echo '"$(1)","$(2)","$(3)","$(4)","$(5)"' >>$(LEGAL_MANIFEST_CSV_$(6)) define legal-license-header echo -e "$(LEGAL_INFO_SEPARATOR)\n\t$(1):" \ "$(2)\n$(LEGAL_INFO_SEPARATOR)\n\n" >>$(LEGAL_LICENSES_TXT)
PATCH v3 1/1 legal info: split manifest for host and target
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Thomas" == Thomas De Schampheleire <[email protected]> writes:
Signed-off-by Thomas De Schampheleire <[email protected]>Acked-by Luca Ceresoli <[email protected]>Tested-by Luca Ceresoli <[email protected]>
Committed the rest of the series to next, thanks.
Signature · 1 line
Bye, Peter Korsgaard
PATCH v3 1/1 legal info: split license texts for host and target
In reply to Thomas De Schampheleire---
v3:
- rebase after changes in first patch
v2:
- rebase after changes in first patch
- fix accidental mixup of contents from first patch
Makefile | 8 +++++--- package/pkg-generic.mk | 4 ++-- package/pkg-utils.mk | 16 ++++++++-------- 3 files changed, 15 insertions(+), 13 deletions(-)
Makefile +5 -3
@@ -109,10 +109,12 @@ HOST_DIR:=$(BASE_DIR)/host LEGAL_INFO_DIR=$(BASE_DIR)/legal-info REDIST_SOURCES_DIR=$(LEGAL_INFO_DIR)/sources -LICENSE_FILES_DIR=$(LEGAL_INFO_DIR)/licenses +LICENSE_FILES_DIR_TARGET=$(LEGAL_INFO_DIR)/licenses +LICENSE_FILES_DIR_HOST=$(LEGAL_INFO_DIR)/host-licenses LEGAL_MANIFEST_CSV_TARGET=$(LEGAL_INFO_DIR)/manifest.csv LEGAL_MANIFEST_CSV_HOST=$(LEGAL_INFO_DIR)/host-manifest.csv -LEGAL_LICENSES_TXT=$(LEGAL_INFO_DIR)/licenses.txt +LEGAL_LICENSES_TXT_TARGET=$(LEGAL_INFO_DIR)/licenses.txt +LEGAL_LICENSES_TXT_HOST=$(LEGAL_INFO_DIR)/host-licenses.txt LEGAL_WARNINGS=$(LEGAL_INFO_DIR)/.warnings LEGAL_REPORT=$(LEGAL_INFO_DIR)/README
@@ -575,7 +577,7 @@ legal-info-clean: legal-info-prepare: $(LEGAL_INFO_DIR) @$(call MESSAGE,"Collecting legal info") - @$(call legal-license-file,buildroot,COPYING,COPYING) + @$(call legal-license-file,buildroot,COPYING,COPYING,HOST) @$(call legal-manifest,PACKAGE,VERSION,LICENSE,LICENSE FILES,SOURCE ARCHIVE,TARGET) @$(call legal-manifest,PACKAGE,VERSION,LICENSE,LICENSE FILES,SOURCE ARCHIVE,HOST) @$(call legal-manifest,buildroot,$(BR2_VERSION_FULL),GPLv2+,COPYING,not saved,HOST)
package/pkg-generic.mk +2 -2
@@ -515,13 +515,13 @@ else # Save license files if defined ifeq ($(call qstrip,$$($(2)_LICENSE_FILES)),) - @$(call legal-license-nofiles,$$($(2)_RAWNAME)) + @$(call legal-license-nofiles,$$($(2)_RAWNAME),$(call UPPERCASE,$(5))) @$(call legal-warning-pkg,$$($(2)_RAWNAME),cannot save license ($(2)_LICENSE_FILES not defined)) else # Double dollar signs are really needed here, to catch host packages # without explicit HOST_FOO_LICENSE_FILES assignment, also in case they # have multiple license files. - @$$(foreach F,$$($(2)_LICENSE_FILES),$$(call legal-license-file,$$($(2)_RAWNAME),$$(F),$$($(2)_DIR)/$$(F))$$(sep)) + @$$(foreach F,$$($(2)_LICENSE_FILES),$$(call legal-license-file,$$($(2)_RAWNAME),$$(F),$$($(2)_DIR)/$$(F),$(call UPPERCASE,$(5)))$$(sep)) endif # license files ifeq ($$($(2)_REDISTRIBUTE),YES)
package/pkg-utils.mk +8 -8
@@ -107,15 +107,15 @@ endef legal-manifest=echo '"$(1)","$(2)","$(3)","$(4)","$(5)"' >>$(LEGAL_MANIFEST_CSV_$(6)) define legal-license-header echo -e "$(LEGAL_INFO_SEPARATOR)\n\t$(1):" \ - "$(2)\n$(LEGAL_INFO_SEPARATOR)\n\n" >>$(LEGAL_LICENSES_TXT) + "$(2)\n$(LEGAL_INFO_SEPARATOR)\n\n" >>$(LEGAL_LICENSES_TXT_$(3)) endef define legal-license-nofiles - $(call legal-license-header,$(1),unknown license file(s)) + $(call legal-license-header,$(1),unknown license file(s),$(2)) endef -define legal-license-file # pkg, filename, file-fullpath - $(call legal-license-header,$(1),$(2) file) && \ - cat $(3) >>$(LEGAL_LICENSES_TXT) && \ - echo >>$(LEGAL_LICENSES_TXT) && \ - mkdir -p $(LICENSE_FILES_DIR)/$(1)/$(dir $(2)) && \ - cp $(3) $(LICENSE_FILES_DIR)/$(1)/$(2) +define legal-license-file # pkg, filename, file-fullpath, type + $(call legal-license-header,$(1),$(2) file,$(4)) && \ + cat $(3) >>$(LEGAL_LICENSES_TXT_$(4)) && \ + echo >>$(LEGAL_LICENSES_TXT_$(4)) && \ + mkdir -p $(LICENSE_FILES_DIR_$(4))/$(1)/$(dir $(2)) && \ + cp $(3) $(LICENSE_FILES_DIR_$(4))/$(1)/$(2) endef
PATCH v3 1/1 legal info: split sources for host and target
In reply to Thomas De Schampheleire---
This patch concludes the splitting of legal info for host and target.
v3: no changes
v2: trivial rebase after changes in first patch
Makefile | 9 +++++---- package/pkg-generic.mk | 6 +++--- 2 files changed, 8 insertions(+), 7 deletions(-)
Makefile +5 -4
@@ -108,7 +108,8 @@ TARGET_DIR:=$(BASE_DIR)/target HOST_DIR:=$(BASE_DIR)/host LEGAL_INFO_DIR=$(BASE_DIR)/legal-info -REDIST_SOURCES_DIR=$(LEGAL_INFO_DIR)/sources +REDIST_SOURCES_DIR_TARGET=$(LEGAL_INFO_DIR)/sources +REDIST_SOURCES_DIR_HOST=$(LEGAL_INFO_DIR)/host-sources LICENSE_FILES_DIR_TARGET=$(LEGAL_INFO_DIR)/licenses LICENSE_FILES_DIR_HOST=$(LEGAL_INFO_DIR)/host-licenses LEGAL_MANIFEST_CSV_TARGET=$(LEGAL_INFO_DIR)/manifest.csv
@@ -404,7 +405,7 @@ world: $(BASE_TARGETS) $(TARGETS_ALL) # dependencies anywhere else # ################################################################################ -$(BUILD_DIR) $(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR): +$(BUILD_DIR) $(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST): @mkdir -p $@ # We make a symlink lib32->lib or lib64->lib as appropriate
@@ -585,8 +586,8 @@ legal-info-prepare: $(LEGAL_INFO_DIR) @$(call legal-warning,the toolchain has not been saved) @cp $(BUILDROOT_CONFIG) $(LEGAL_INFO_DIR)/buildroot.config -legal-info: dirs legal-info-clean legal-info-prepare $(REDIST_SOURCES_DIR) \ - $(TARGETS_LEGAL_INFO) +legal-info: dirs legal-info-clean legal-info-prepare $(TARGETS_LEGAL_INFO) \ + $(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST) @cat support/legal-info/README.header >>$(LEGAL_REPORT) @if [ -r $(LEGAL_WARNINGS) ]; then \ cat support/legal-info/README.warnings-header \
package/pkg-generic.mk +3 -3
@@ -491,7 +491,7 @@ ifeq ($$($(2)_REDISTRIBUTE),YES) ifneq ($$($(2)_SITE_METHOD),local) ifneq ($$($(2)_SITE_METHOD),override) # Packages that have a tarball need it downloaded and extracted beforehand -$(1)-legal-info: $(1)-extract $(REDIST_SOURCES_DIR) +$(1)-legal-info: $(1)-extract $(REDIST_SOURCES_DIR_$(call UPPERCASE,$(5))) $(2)_MANIFEST_TARBALL = $$($(2)_SOURCE) endif endif
@@ -526,8 +526,8 @@ endif # license files ifeq ($$($(2)_REDISTRIBUTE),YES) # Copy the source tarball (just hardlink if possible) - @cp -l $(DL_DIR)/$$($(2)_SOURCE) $(REDIST_SOURCES_DIR) 2>/dev/null || \ - cp $(DL_DIR)/$$($(2)_SOURCE) $(REDIST_SOURCES_DIR) + @cp -l $(DL_DIR)/$$($(2)_SOURCE) $(REDIST_SOURCES_DIR_$(call UPPERCASE,$(5))) 2>/dev/null || \ + cp $(DL_DIR)/$$($(2)_SOURCE) $(REDIST_SOURCES_DIR_$(call UPPERCASE,$(5))) endif # redistribute endif # other packages
PATCH v3 1/1 legal info: cleanup utility functions
In reply to Thomas De SchampheleireThe legal-info utility functions where defined using two ways util-foo = command-foo
and
define util-bar # parameter description
command-bar
endef
This commit changes these functions to use the second form for clarity and additionally adds parameter descriptions on all functions.
---
v3: no changes
v2:
- add whitespace between functions
- replace 'type' with {HOST|TARGET} (comment Luca)
package/pkg-utils.mk | 25 +++++++++++++++++++------ 1 files changed, 19 insertions(+), 6 deletions(-)
package/pkg-utils.mk +19 -6
@@ -99,20 +99,33 @@ endef # legal-info helper functions # LEGAL_INFO_SEPARATOR="::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::" -legal-warning=echo "WARNING: $(1)" >>$(LEGAL_WARNINGS) -legal-warning-pkg=echo "WARNING: $(1): $(2)" >>$(LEGAL_WARNINGS) + +define legal-warning # text + echo "WARNING: $(1)" >>$(LEGAL_WARNINGS) +endef + +define legal-warning-pkg # pkg, text + echo "WARNING: $(1): $(2)" >>$(LEGAL_WARNINGS) +endef + define legal-warning-pkg-savednothing # pkg, {local|override} $(call legal-warning-pkg,$(1),sources and license files not saved ($(2) packages not handled)) endef -legal-manifest=echo '"$(1)","$(2)","$(3)","$(4)","$(5)"' >>$(LEGAL_MANIFEST_CSV_$(6)) -define legal-license-header + +define legal-manifest # pkg, version, license, license-files, source, {HOST|TARGET} + echo '"$(1)","$(2)","$(3)","$(4)","$(5)"' >>$(LEGAL_MANIFEST_CSV_$(6)) +endef + +define legal-license-header # pkg, license-file, {HOST|TARGET} echo -e "$(LEGAL_INFO_SEPARATOR)\n\t$(1):" \ "$(2)\n$(LEGAL_INFO_SEPARATOR)\n\n" >>$(LEGAL_LICENSES_TXT_$(3)) endef -define legal-license-nofiles + +define legal-license-nofiles # pkg, {HOST|TARGET} $(call legal-license-header,$(1),unknown license file(s),$(2)) endef -define legal-license-file # pkg, filename, file-fullpath, type + +define legal-license-file # pkg, filename, file-fullpath, {HOST|TARGET} $(call legal-license-header,$(1),$(2) file,$(4)) && \ cat $(3) >>$(LEGAL_LICENSES_TXT_$(4)) && \ echo >>$(LEGAL_LICENSES_TXT_$(4)) && \
PATCH v3 1/1 legal info: update documentation with split target/host output
In reply to Thomas De Schampheleire---
v3: no changes
v2: new patch
docs/manual/legal-notice.txt | 15 +++++++++------ 1 files changed, 9 insertions(+), 6 deletions(-)
docs/manual/legal-notice.txt +12 -9
@@ -39,16 +39,19 @@ There you will find: * +buildroot.config+: this is the Buildroot configuration file that is usually produced with +make menuconfig+, and which is necessary to reproduce the build. -* The source code for all packages; this is saved in the +sources/+ - subdirectory (except for proprietary packages, whose source code is not - saved); - patches applied to some packages by Buildroot are distributed with the - Buildroot sources and are not duplicated in the +sources/+ subdirectory. -* A manifest file listing the configured packages, their version, license and - related information. +* The source code for all packages; this is saved in the +sources/+ and + +host-sources/+ subdirectories for target and host packages respectively. + The source code for packages that set +<PKG>_REDISTRIBUTE = NO+ will not be + saved. + Patches applied to some packages by Buildroot are distributed with the + Buildroot sources and are not duplicated in the +sources/+ and +host-sources/+ + subdirectories. +* A manifest file (one for host and one for target packages) listing the + configured packages, their version, license and related information. Some of this information might not be defined in Buildroot; such items are marked as "unknown". -* A +licenses/+ subdirectory, which contains the license text of packages. +* The license texts of all packages, in the +licenses/+ and +host-licenses/+ + subdirectories for target and host packages respectively. If the license file(s) are not defined in Buildroot, the file is not produced and a warning in the +README+ indicates this.
@@ -72,7 +75,7 @@ License abbreviations --------------------- Here is a list of the licenses that are most widely used by packages in -Buildroot, with the name used in the manifest file: +Buildroot, with the name used in the manifest files: * `GPLv2`: http://www.gnu.org/licenses/old-licenses/gpl-2.0.html[