license.bbclass: Introduce AVAILABLE_LICENSES that lists all licenses
PATCH v3 1/8 license.bbclass: Introduce AVAILABLE_LICENSES that lists all licenses
Previously, there was SRC_DISTRIBUTE_LICENSES, an undocumented variable that contained a static list of licenses. It was used by expand_wildcard_licenses() to expand any wildcards used in, e.g., INCOMPATIBLE_LICENSE. However, since this static list of licenses has not been kept up-to-date, many licenses were missing, with the result that if one tried to use any of those licenses with a wildcard, no licenses would be found, effectively ignoring that they should be marked as incompatible.
This introduces a new (documented) variable, AVAILABLE_LICENSES, that is automatically updated to contain all licenses found in any directories specified by ${COMMON_LICENSE_DIR} and ${LICENSE_PATH}, and uses it instead of SRC_DISTRIBUTE_LICENSES when expanding wildcards.
---
PATCHv2: Correct spelling of 'available'.
meta/classes/license.bbclass | 27 ++++++++++++++++--- meta/conf/documentation.conf | 1 + .../oeqa/selftest/cases/incompatible_lic.py | 6 ++--- 3 files changed, 27 insertions(+), 7 deletions(-)
meta/classes/license.bbclass +23 -4
@@ -252,7 +252,7 @@ def canonical_license(d, license): """ Return the canonical (SPDX) form of the license if available (so GPLv3 becomes GPL-3.0), for the license named 'X+', return canonical form of - 'X' if availabel and the tailing '+' (so GPLv3+ becomes GPL-3.0+), + 'X' if available and the tailing '+' (so GPLv3+ becomes GPL-3.0+), or the passed license if there is no canonical form. """ lic = d.getVarFlag('SPDXLICENSEMAP', license) or ""
@@ -262,10 +262,29 @@ def canonical_license(d, license): lic += '+' return lic or license +def available_licenses(d): + """ + Return the available licenses by searching the directories specified by + COMMON_LICENSE_DIR and LICENSE_PATH. + """ + lic_dirs = ((d.getVar('COMMON_LICENSE_DIR') or '') + ' ' + + (d.getVar('LICENSE_PATH') or '')).split() + + licenses = [] + for lic_dir in lic_dirs: + licenses += os.listdir(lic_dir) + + licenses = sorted(licenses) + return licenses + +# Only determine the list of all available licenses once. This assumes that any +# additions to LICENSE_PATH have been done before this file is parsed. +AVAILABLE_LICENSES := "${@' '.join(available_licenses(d))}" + def expand_wildcard_licenses(d, wildcard_licenses): """ - Return actual spdx format license names if wildcard used. We expand - wildcards from SPDXLICENSEMAP flags and SRC_DISTRIBUTE_LICENSES values. + Return actual spdx format license names if wildcards are used. We expand + wildcards from SPDXLICENSEMAP flags and AVAILABLE_LICENSES. """ import fnmatch licenses = wildcard_licenses[:]
@@ -274,7 +293,7 @@ def expand_wildcard_licenses(d, wildcard_licenses): spdxflags = fnmatch.filter(spdxmapkeys, wld_lic) licenses += [d.getVarFlag('SPDXLICENSEMAP', flag) for flag in spdxflags] - spdx_lics = (d.getVar('SRC_DISTRIBUTE_LICENSES', False) or '').split() + spdx_lics = d.getVar('AVAILABLE_LICENSES').split() for wld_lic in wildcard_licenses: licenses += fnmatch.filter(spdx_lics, wld_lic)
meta/conf/documentation.conf +1 -0
@@ -69,6 +69,7 @@ ASSUME_SHLIBS[doc] = "List of shlib:package[_version] mappings. Useful for lib p AUTHOR[doc] = "Email address used to contact the original author(s) in order to send patches and forward bugs." AUTO_SYSLINUXMENU[doc] = "Enables creating an automatic menu for the syslinux bootloader." AUTOREV[doc] = "When SRCREV is set to the value of this variable, it specifies to use the latest source revision in the repository." +AVAILABLE_LICENSES[doc] = "List of licenses found in the directories specified by COMMON_LICENSE_DIR and LICENSE_PATH." #B
meta/lib/oeqa/selftest/cases/incompatible_lic.py +3 -3
@@ -12,12 +12,12 @@ class IncompatibleLicenseTests(OESelftestTestCase): if error_msg not in result.output: raise AssertionError(result.output) - # Verify that a package with an SPDX license (from SRC_DISTRIBUTE_LICENSES) + # Verify that a package with an SPDX license (from AVAILABLE_LICENSES) # cannot be built when INCOMPATIBLE_LICENSE contains this SPDX license def test_incompatible_spdx_license(self): self.lic_test('incompatible-license', 'GPL-3.0', 'GPL-3.0') - # Verify that a package with an SPDX license (from SRC_DISTRIBUTE_LICENSES) + # Verify that a package with an SPDX license (from AVAILABLE_LICENSES) # cannot be built when INCOMPATIBLE_LICENSE contains an alias (in # SPDXLICENSEMAP) of this SPDX license def test_incompatible_alias_spdx_license(self):
@@ -35,7 +35,7 @@ class IncompatibleLicenseTests(OESelftestTestCase): self.lic_test('incompatible-license-alias', 'GPLv3', 'GPLv3') # Verify that a package with a non-SPDX license (neither in - # SRC_DISTRIBUTE_LICENSES nor in SPDXLICENSEMAP) cannot be built when + # AVAILABLE_LICENSES nor in SPDXLICENSEMAP) cannot be built when # INCOMPATIBLE_LICENSE contains this license def test_incompatible_nonspdx_license(self): self.lic_test('incompatible-nonspdx-license', 'FooLicense', 'FooLicense')
Signature · 1 line
2.21.0
PATCH v3 2/8 licenses.conf: Remove the SRC_DISTRIBUTE_LICENSES variable
The SRC_DISTRIBUTE_LICENSES variable and its static list of licenses has been replaced by AVAILABLE_LICENSES, which automatically contains all available licenses.
--- meta/conf/licenses.conf | 43 ----------------------------------------- 1 file changed, 43 deletions(-)
meta/conf/licenses.conf +0 -43
@@ -1,44 +1,3 @@ -# These aren't actually used anywhere that I can tell -# They may be in the future (or are used by someone else -# For completion sake, I've updated them -SRC_DISTRIBUTE_LICENSES += "AAL Adobe AFL-1.2 AFL-2.0 AFL-2.1 AFL-3.0" -SRC_DISTRIBUTE_LICENSES += "AGPL-3.0 ANTLR-PD Apache-1.0 Apache-1.1 Apache-2.0" -SRC_DISTRIBUTE_LICENSES += "APL-1.0 APSL-1.0 APSL-1.1 APSL-1.2 APSL-2.0" -SRC_DISTRIBUTE_LICENSES += "Artistic-1.0 Artistic-2.0 BitstreamVera BSD" -SRC_DISTRIBUTE_LICENSES += "BSD-2-Clause BSD-3-Clause BSD-4-Clause BSL-1.0" -SRC_DISTRIBUTE_LICENSES += "CATOSL-1.1 CC0-1.0 CC-BY-1.0 CC-BY-2.0 CC-BY-2.5" -SRC_DISTRIBUTE_LICENSES += "CC-BY-3.0 CC-BY-NC-1.0 CC-BY-NC-2.0 CC-BY-NC-2.5" -SRC_DISTRIBUTE_LICENSES += "CC-BY-NC-3.0 CC-BY-NC-ND-1.0 CC-BY-NC-ND-2.0" -SRC_DISTRIBUTE_LICENSES += "CC-BY-NC-ND-2.5 CC-BY-NC-ND-3.0 CC-BY-NC-SA-1.0" -SRC_DISTRIBUTE_LICENSES += "CC-BY-NC-SA-2.0 CC-BY-NC-SA-2.5 CC-BY-NC-SA-3.0" -SRC_DISTRIBUTE_LICENSES += "CC-BY-ND-1.0 CC-BY-ND-2.0 CC-BY-ND-2.5 CC-BY-ND-3.0" -SRC_DISTRIBUTE_LICENSES += "CC-BY-SA-1.0 CC-BY-SA-2.0 CC-BY-SA-2.5 CC-BY-SA-3.0 CC-BY-SA-4.0" -SRC_DISTRIBUTE_LICENSES += "CDDL-1.0 CECILL-1.0 CECILL-2.0 CECILL-B CECILL-C" -SRC_DISTRIBUTE_LICENSES += "ClArtistic CPAL-1.0 CPL-1.0 CUA-OPL-1.0 DSSSL" -SRC_DISTRIBUTE_LICENSES += "ECL-1.0 ECL-2.0 eCos-2.0 EDL-1.0 EFL-1.0 EFL-2.0" -SRC_DISTRIBUTE_LICENSES += "Entessa EPL-1.0 EPL-2.0 ErlPL-1.1" -SRC_DISTRIBUTE_LICENSES += "EUDatagrid EUPL-1.0 EUPL-1.1 Fair Frameworx-1.0" -SRC_DISTRIBUTE_LICENSES += "FreeType GFDL-1.1 GFDL-1.2 GFDL-1.3 GPL-1.0" -SRC_DISTRIBUTE_LICENSES += "GPL-2.0 GPL-2.0-with-autoconf-exception" -SRC_DISTRIBUTE_LICENSES += "GPL-2.0-with-classpath-exception" -SRC_DISTRIBUTE_LICENSES += "GPL-2.0-with-font-exception" -SRC_DISTRIBUTE_LICENSES += "GPL-2.0-with-GCC-exception" -SRC_DISTRIBUTE_LICENSES += "GPL-2-with-bison-exception GPL-3.0" -SRC_DISTRIBUTE_LICENSES += "GPL-3.0-with-autoconf-exception" -SRC_DISTRIBUTE_LICENSES += "GPL-3.0-with-GCC-exception" -SRC_DISTRIBUTE_LICENSES += "gSOAP-1 gSOAP-1.3b HPND IPA IPL-1.0 ISC LGPL-2.0" -SRC_DISTRIBUTE_LICENSES += "LGPL-2.1 LGPL-3.0 Libpng LPL-1.02 LPPL-1.0 LPPL-1.1" -SRC_DISTRIBUTE_LICENSES += "LPPL-1.2 LPPL-1.3c MirOS MIT Motosoto MPL-1.0" -SRC_DISTRIBUTE_LICENSES += "MPL-1.1 MS-PL MS-RL Multics NASA-1.3 Nauman NCSA" -SRC_DISTRIBUTE_LICENSES += "NGPL Nokia NPOSL-3.0 NTP OASIS OCLC-2.0 ODbL-1.0" -SRC_DISTRIBUTE_LICENSES += "OFL-1.1 OGTSL OLDAP-2.8 OpenSSL OSL-1.0 OSL-2.0" -SRC_DISTRIBUTE_LICENSES += "OSL-3.0 PD PHP-3.0 PostgreSQL Proprietary" -SRC_DISTRIBUTE_LICENSES += "Python-2.0 QPL-1.0 RHeCos-1 RHeCos-1.1 RPL-1.5" -SRC_DISTRIBUTE_LICENSES += "RPSL-1.0 RSCPL Ruby SAX-PD SGI-1 Simple-2.0 Sleepycat" -SRC_DISTRIBUTE_LICENSES += "SPL-1.0 SugarCRM-1 SugarCRM-1.1.3 UCB VSL-1.0 W3C" -SRC_DISTRIBUTE_LICENSES += "Watcom-1.0 WXwindows XFree86-1.0 XFree86-1.1 Xnet XSL YPL-1.1" -SRC_DISTRIBUTE_LICENSES += "Zimbra-1.3 Zlib ZPL-1.1 ZPL-2.0 ZPL-2.1" - # Standards are great! Everyone has their own. In an effort to standardize licensing # names, common-licenses will use the SPDX standard license names. In order to not # break the non-standardized license names that we find in LICENSE, we'll set
@@ -188,5 +147,3 @@ FOSS_BASE_URL = "http://localhost/repo/?mod=spdx_license_once" FOSS_SERVER = "${FOSS_BASE_URL}&fullSPDXFlag=${FOSS_FULL_SPDX}&noCopyright=${FOSS_NO_COPYRIGHT}&recursiveUnpack=${FOSS_RECURSIVE_UNPACK}" FOSS_WGET_FLAGS = "-qO - --no-check-certificate --timeout=0" - -
Signature · 1 line
2.21.0
PATCH v3 3/8 license.bbclass: Make incompatible_pkg_license return incompatible lics
In reply to Peter KjellerstedtThis makes it possible to report the incompatible licenses.
--- meta/classes/license.bbclass | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-)
meta/classes/license.bbclass +9 -3
@@ -310,15 +310,21 @@ def incompatible_pkg_license(d, dont_want_licenses, license): # Handles an "or" or two license sets provided by # flattened_licenses(), pick one that works if possible. def choose_lic_set(a, b): - return a if all(oe.license.license_ok(canonical_license(d, lic), + return a if all(oe.license.license_ok(canonical_license(d, lic), dont_want_licenses) for lic in a) else b try: licenses = oe.license.flattened_licenses(license, choose_lic_set) except oe.license.LicenseError as exc: bb.fatal('%s: %s' % (d.getVar('P'), exc)) - return any(not oe.license.license_ok(canonical_license(d, l), \ - dont_want_licenses) for l in licenses) + + incompatible_lic = [] + for l in licenses: + license = canonical_license(d, l) + if not oe.license.license_ok(license, dont_want_licenses): + incompatible_lic.append(license) + + return sorted(incompatible_lic) def incompatible_license(d, dont_want_licenses, package=None): """
Signature · 1 line
2.21.0
PATCH v3 4/8 base.bbclass: Simplify the check for whitelisted licenses
In reply to Peter KjellerstedtAfter a number of rewrites, the code checking if a package has been whitelisted for an incompatible license was calculating the whitelisted packages twice (as 'whitelist' and as 'incompatwl').
--- meta/classes/base.bbclass | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-)
meta/classes/base.bbclass +4 -9
@@ -526,23 +526,21 @@ python () { bad_licenses = expand_wildcard_licenses(d, bad_licenses) whitelist = [] - incompatwl = [] for lic in bad_licenses: spdx_license = return_spdx(d, lic) whitelist.extend((d.getVar("WHITELIST_" + lic) or "").split()) if spdx_license: whitelist.extend((d.getVar("WHITELIST_" + spdx_license) or "").split()) + + if pn in whitelist: ''' We need to track what we are whitelisting and why. If pn is incompatible we need to be able to note that the image that is created may infact contain incompatible licenses despite INCOMPATIBLE_LICENSE being set. ''' - incompatwl.extend((d.getVar("WHITELIST_" + lic) or "").split()) - if spdx_license: - incompatwl.extend((d.getVar("WHITELIST_" + spdx_license) or "").split()) - - if not pn in whitelist: + bb.note("Including %s as buildable despite it having an incompatible license because it has been whitelisted" % pn) + else: pkgs = d.getVar('PACKAGES').split() skipped_pkgs = [] unskipped_pkgs = []
@@ -562,9 +560,6 @@ python () { elif all_skipped or incompatible_license(d, bad_licenses): bb.debug(1, "SKIPPING recipe %s because it's %s" % (pn, license)) raise bb.parse.SkipRecipe("it has an incompatible license: %s" % license) - elif pn in whitelist: - if pn in incompatwl: - bb.note("INCLUDING " + pn + " as buildable despite INCOMPATIBLE_LICENSE because it has been whitelisted") # Try to verify per-package (LICENSE_<pkg>) values. LICENSE should be a # superset of all per-package licenses. We do not do advanced (pattern)
Signature · 1 line
2.21.0
PATCH v3 5/8 base.bbclass: Report only the licenses that are incompatible for a package
In reply to Peter KjellerstedtInstead of reporting ${LICENSE} when a package is identified as using an incompatible license, report the license(s) that are actually incompatible.
--- meta/classes/base.bbclass | 26 ++++++++++++------- .../oeqa/selftest/cases/incompatible_lic.py | 6 ++--- 2 files changed, 19 insertions(+), 13 deletions(-)
meta/classes/base.bbclass +16 -10
@@ -542,24 +542,30 @@ python () { bb.note("Including %s as buildable despite it having an incompatible license because it has been whitelisted" % pn) else: pkgs = d.getVar('PACKAGES').split() - skipped_pkgs = [] + skipped_pkgs = {} unskipped_pkgs = [] for pkg in pkgs: - if incompatible_license(d, bad_licenses, pkg): - skipped_pkgs.append(pkg) + incompatible_lic = incompatible_license(d, bad_licenses, pkg) + if incompatible_lic: + skipped_pkgs[pkg] = incompatible_lic else: unskipped_pkgs.append(pkg) - all_skipped = skipped_pkgs and not unskipped_pkgs if unskipped_pkgs: for pkg in skipped_pkgs: - bb.debug(1, "SKIPPING the package " + pkg + " at do_rootfs because it's " + license) + bb.debug(1, "Skipping the package %s at do_rootfs because of incompatible license(s): %s" % (pkg, ' '.join(skipped_pkgs[pkg]))) mlprefix = d.getVar('MLPREFIX') - d.setVar('LICENSE_EXCLUSION-' + mlprefix + pkg, 1) + d.setVar('LICENSE_EXCLUSION-' + mlprefix + pkg, ' '.join(skipped_pkgs[pkg])) for pkg in unskipped_pkgs: - bb.debug(1, "INCLUDING the package " + pkg) - elif all_skipped or incompatible_license(d, bad_licenses): - bb.debug(1, "SKIPPING recipe %s because it's %s" % (pn, license)) - raise bb.parse.SkipRecipe("it has an incompatible license: %s" % license) + bb.debug(1, "Including the package %s" % pkg) + else: + incompatible_lic = incompatible_license(d, bad_licenses) + for pkg in skipped_pkgs: + incompatible_lic += skipped_pkgs[pkg] + incompatible_lic = sorted(list(set(incompatible_lic))) + + if incompatible_lic: + bb.debug(1, "Skipping recipe %s because of incompatible license(s): %s" % (pn, ' '.join(incompatible_lic))) + raise bb.parse.SkipRecipe("it has incompatible license(s): %s" % ' '.join(incompatible_lic)) # Try to verify per-package (LICENSE_<pkg>) values. LICENSE should be a # superset of all per-package licenses. We do not do advanced (pattern)
meta/lib/oeqa/selftest/cases/incompatible_lic.py +3 -3
@@ -4,7 +4,7 @@ from oeqa.utils.commands import bitbake class IncompatibleLicenseTests(OESelftestTestCase): def lic_test(self, pn, pn_lic, lic): - error_msg = 'ERROR: Nothing PROVIDES \'%s\'\n%s was skipped: it has an incompatible license: %s' % (pn, pn, pn_lic) + error_msg = 'ERROR: Nothing PROVIDES \'%s\'\n%s was skipped: it has incompatible license(s): %s' % (pn, pn, pn_lic) self.write_config("INCOMPATIBLE_LICENSE += \"%s\"" % (lic))
@@ -27,12 +27,12 @@ class IncompatibleLicenseTests(OESelftestTestCase): # license cannot be built when INCOMPATIBLE_LICENSE contains this SPDX # license def test_incompatible_spdx_license_alias(self): - self.lic_test('incompatible-license-alias', 'GPLv3', 'GPL-3.0') + self.lic_test('incompatible-license-alias', 'GPL-3.0', 'GPL-3.0') # Verify that a package with an alias (from SPDXLICENSEMAP) to an SPDX # license cannot be built when INCOMPATIBLE_LICENSE contains this alias def test_incompatible_alias_spdx_license_alias(self): - self.lic_test('incompatible-license-alias', 'GPLv3', 'GPLv3') + self.lic_test('incompatible-license-alias', 'GPL-3.0', 'GPLv3') # Verify that a package with a non-SPDX license (neither in # AVAILABLE_LICENSES nor in SPDXLICENSEMAP) cannot be built when
Signature · 1 line
2.21.0
PATCH v3 6/8 package.bbclass: Report only the licenses that are incompatible
In reply to Peter KjellerstedtWhen excluding a package from being packaged due to incompatible licenses, report the license(s) that are actually incompatible.
---
PATCHv2: Show the names of the incompatible licenses correctly.
meta/classes/package.bbclass | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
meta/classes/package.bbclass +3 -2
@@ -1263,8 +1263,9 @@ python populate_packages () { # Handle LICENSE_EXCLUSION package_list = [] for pkg in packages: - if d.getVar('LICENSE_EXCLUSION-' + pkg): - msg = "%s has an incompatible license. Excluding from packaging." % pkg + licenses = d.getVar('LICENSE_EXCLUSION-' + pkg) + if licenses: + msg = "Excluding %s from packaging as it has incompatible license(s): %s" % (pkg, licenses) package_qa_handle_error("incompatible-license", msg, d) else: package_list.append(pkg)
Signature · 1 line
2.21.0
PATCH v3 7/8 license_image.bbclass: Report only the licenses that are incompatible
In reply to Peter KjellerstedtInstead of reporting ${LICENSE} when a package cannot be installed into an image because it is using an incompatible license, report the license(s) that are actually incompatible.
--- meta/classes/license_image.bbclass | 5 +++-- meta/lib/oeqa/selftest/cases/incompatible_lic.py | 4 ++-- 2 files changed, 5 insertions(+), 4 deletions(-)
meta/classes/license_image.bbclass +3 -2
@@ -51,8 +51,9 @@ def write_license_files(d, license_manifest, pkg_dic, rootfs=True): for pkg in sorted(pkg_dic): if bad_licenses and pkg not in whitelist: try: - if incompatible_pkg_license(d, bad_licenses, pkg_dic[pkg]["LICENSE"]): - bb.fatal("Package %s has an incompatible license %s and cannot be installed into the image." %(pkg, pkg_dic[pkg]["LICENSE"])) + licenses = incompatible_pkg_license(d, bad_licenses, pkg_dic[pkg]["LICENSE"]) + if licenses: + bb.fatal("Package %s cannot be installed into the image because it has incompatible license(s): %s" %(pkg, ' '.join(licenses))) (pkg_dic[pkg]["LICENSE"], pkg_dic[pkg]["LICENSES"]) = \ oe.license.manifest_licenses(pkg_dic[pkg]["LICENSE"], bad_licenses, canonical_license, d)
meta/lib/oeqa/selftest/cases/incompatible_lic.py +2 -2
@@ -49,7 +49,7 @@ INCOMPATIBLE_LICENSE_pn-core-image-minimal = "GPL-3.0 LGPL-3.0" def test_bash_default(self): self.write_config(self.default_config()) - error_msg = "ERROR: core-image-minimal-1.0-r0 do_rootfs: Package bash has an incompatible license GPLv3+ and cannot be installed into the image." + error_msg = "ERROR: core-image-minimal-1.0-r0 do_rootfs: Package bash cannot be installed into the image because it has incompatible license(s): GPL-3.0+" result = bitbake('core-image-minimal', ignore_status=True) if error_msg not in result.output:
@@ -57,7 +57,7 @@ INCOMPATIBLE_LICENSE_pn-core-image-minimal = "GPL-3.0 LGPL-3.0" def test_bash_and_license(self): self.write_config(self.default_config() + '\nLICENSE_append_pn-bash = " & SomeLicense"') - error_msg = "ERROR: core-image-minimal-1.0-r0 do_rootfs: Package bash has an incompatible license GPLv3+ & SomeLicense and cannot be installed into the image." + error_msg = "ERROR: core-image-minimal-1.0-r0 do_rootfs: Package bash cannot be installed into the image because it has incompatible license(s): GPL-3.0+" result = bitbake('core-image-minimal', ignore_status=True) if error_msg not in result.output:
Signature · 1 line
2.21.0
PATCH v3 8/8 incompatible_lic.py: Add tests for incompatible licenses with wildcards
In reply to Peter Kjellerstedt---
PATCHv2: New in this patch set.
PATCHv3: Added the missing incompatible-licenses.bb recipe file.
.../license/incompatible-licenses.bb | 3 ++ .../oeqa/selftest/cases/incompatible_lic.py | 42 +++++++++++++++++++ 2 files changed, 45 insertions(+) create mode 100644 meta-selftest/recipes-test/license/incompatible-licenses.bb
meta-selftest/recipes-test/license/incompatible-licenses.bb new file +3 -0
@@ -0,0 +1,3 @@ +SUMMARY = "Recipe with multiple SPDX licenses" +DESCRIPTION = "Is licensed with multiple SPDX licenses to be used for testing" +LICENSE = "GPL-2.0 & GPL-3.0 & LGPL-3.0"
meta/lib/oeqa/selftest/cases/incompatible_lic.py +42 -0
54 lines not on this page · raw message
Signature · 1 line
2.21.0
This actually fails yocto-check-layer script for Yocto compliance checking, when a layer provides own licenses via LICENSE_PATH (in layer.conf - LICENSE_PATH += "${LAYERDIR}/licenses"):
Variable AVAILABLE_LICENSES value changed:
...
Is there a new way to provide additional licenses from a layer or should the yocto-check-layer script be modified to accomodate (i.e. ignore) this change?
Signature · 1 line
Denys
Quoted 127 lines · Peter Kjellerstedt
Previously, there was
SRC_DISTRIBUTE_LICENSES, an undocumented variable that contained a static list of licenses. It was used byexpand_wildcard_licenses()to expand any wildcards used in, e.g.,INCOMPATIBLE_LICENSE. However, since this static list of licenses has not been kept up-to-date, many licenses were missing, with the result that if one tried to use any of those licenses with a wildcard, no licenses would be found, effectively ignoring that they should be marked as incompatible.This introduces a new (documented) variable,
AVAILABLE_LICENSES, that is automatically updated to contain all licenses found in any directories specified by ${COMMON_LICENSE_DIR} and ${LICENSE_PATH}, and uses it instead ofSRC_DISTRIBUTE_LICENSESwhen expanding wildcards.Signed-off-by Peter Kjellerstedt <[email protected]>---
PATCHv2: Correct spelling of 'available'.
meta/classes/license.bbclass | 27 ++++++++++++++++--- meta/conf/documentation.conf | 1 + .../oeqa/selftest/cases/incompatible_lic.py | 6 ++--- 3 files changed, 27 insertions(+), 7 deletions(-)meta/classes/license.bbclass +23
-4@@ -252,7 +252,7 @@ def canonical_license(d, license): """ Return the canonical (SPDX) form of the license if available (so GPLv3 becomes GPL-3.0), for the license named 'X+', return canonical form of - 'X' if availabel and the tailing '+' (so GPLv3+ becomes GPL-3.0+), + 'X' if available and the tailing '+' (so GPLv3+ becomes GPL-3.0+), or the passed license if there is no canonical form. """ lic = d.getVarFlag('SPDXLICENSEMAP', license) or ""@@ -262,10 +262,29 @@ def canonical_license(d, license): lic += '+' return lic or license +def available_licenses(d): + """ + Return the available licenses by searching the directories specified by + COMMON_LICENSE_DIR and LICENSE_PATH. + """ + lic_dirs = ((d.getVar('COMMON_LICENSE_DIR') or '') + ' ' + + (d.getVar('LICENSE_PATH') or '')).split() + + licenses = [] + for lic_dir in lic_dirs: + licenses += os.listdir(lic_dir) + + licenses = sorted(licenses) + return licenses + +# Only determine the list of all available licenses once. This assumes that any +# additions to LICENSE_PATH have been done before this file is parsed. +AVAILABLE_LICENSES := "${@' '.join(available_licenses(d))}" + def expand_wildcard_licenses(d, wildcard_licenses): """ - Return actual spdx format license names if wildcard used. We expand - wildcards from SPDXLICENSEMAP flags and SRC_DISTRIBUTE_LICENSES values. + Return actual spdx format license names if wildcards are used. We expand + wildcards from SPDXLICENSEMAP flags and AVAILABLE_LICENSES. """ import fnmatch licenses = wildcard_licenses[:]@@ -274,7 +293,7 @@ def expand_wildcard_licenses(d, wildcard_licenses): spdxflags = fnmatch.filter(spdxmapkeys, wld_lic) licenses += [d.getVarFlag('SPDXLICENSEMAP', flag) for flag in spdxflags] - spdx_lics = (d.getVar('SRC_DISTRIBUTE_LICENSES', False) or '').split() + spdx_lics = d.getVar('AVAILABLE_LICENSES').split() for wld_lic in wildcard_licenses: licenses += fnmatch.filter(spdx_lics, wld_lic)meta/conf/documentation.conf +1
-0@@ -69,6 +69,7 @@ ASSUME_SHLIBS[doc] = "List of shlib:package[_version] mappings. Useful for lib p AUTHOR[doc] = "Email address used to contact the original author(s) in order to send patches and forward bugs." AUTO_SYSLINUXMENU[doc] = "Enables creating an automatic menu for the syslinux bootloader." AUTOREV[doc] = "When SRCREV is set to the value of this variable, it specifies to use the latest source revision in the repository." +AVAILABLE_LICENSES[doc] = "List of licenses found in the directories specified by COMMON_LICENSE_DIR and LICENSE_PATH." #Bmeta/lib/oeqa/selftest/cases/incompatible_lic.py +3
-3@@ -12,12 +12,12 @@ class IncompatibleLicenseTests(OESelftestTestCase): if error_msg not in result.output: raise AssertionError(result.output) - # Verify that a package with an SPDX license (from SRC_DISTRIBUTE_LICENSES) + # Verify that a package with an SPDX license (from AVAILABLE_LICENSES) # cannot be built when INCOMPATIBLE_LICENSE contains this SPDX license def test_incompatible_spdx_license(self): self.lic_test('incompatible-license', 'GPL-3.0', 'GPL-3.0') - # Verify that a package with an SPDX license (from SRC_DISTRIBUTE_LICENSES) + # Verify that a package with an SPDX license (from AVAILABLE_LICENSES) # cannot be built when INCOMPATIBLE_LICENSE contains an alias (in # SPDXLICENSEMAP) of this SPDX license def test_incompatible_alias_spdx_license(self):@@ -35,7 +35,7 @@ class IncompatibleLicenseTests(OESelftestTestCase): self.lic_test('incompatible-license-alias', 'GPLv3', 'GPLv3') # Verify that a package with a non-SPDX license (neither in - # SRC_DISTRIBUTE_LICENSES nor in SPDXLICENSEMAP) cannot be built when + # AVAILABLE_LICENSES nor in SPDXLICENSEMAP) cannot be built when # INCOMPATIBLE_LICENSE contains this license def test_incompatible_nonspdx_license(self): self.lic_test('incompatible-nonspdx-license', 'FooLicense', 'FooLicense')Signature · 7 lines
2.21.0 -- _______________________________________________ Openembedded-core mailing list [email protected] http://lists.openembedded.org/mailman/listinfo/openembedded-core
This actually also results in checksums changing for recipes that inherit license and call incompatible_license_contains() when adding/removing layers that add to the license path, such as lvm2.
Quoted 166 lines · Denys Dmytriyenko
This actually fails yocto-check-layer script for Yocto compliance checking, when a layer provides own licenses via
LICENSE_PATH(in layer.conf -LICENSE_PATH+= "${LAYERDIR}/licenses"):Variable
AVAILABLE_LICENSESvalue changed:
...Is there a new way to provide additional licenses from a layer or should the yocto-check-layer script be modified to accomodate (i.e. ignore) this change?
--
DenysQuoted 30 lines · Peter Kjellerstedt
Previously, there was
SRC_DISTRIBUTE_LICENSES, an undocumented variable that contained a static list of licenses. It was used byexpand_wildcard_licenses()to expand any wildcards used in, e.g.,INCOMPATIBLE_LICENSE. However, since this static list of licenses has not been kept up-to-date, many licenses were missing, with the result that if one tried to use any of those licenses with a wildcard, no licenses would be found, effectively ignoring that they should be marked as incompatible.This introduces a new (documented) variable,
AVAILABLE_LICENSES, that is automatically updated to contain all licenses found in any directories specified by ${COMMON_LICENSE_DIR} and ${LICENSE_PATH}, and uses it instead ofSRC_DISTRIBUTE_LICENSESwhen expanding wildcards.Signed-off-by Peter Kjellerstedt <[email protected]>---
PATCHv2: Correct spelling of 'available'.
meta/classes/license.bbclass | 27 ++++++++++++++++--- meta/conf/documentation.conf | 1 + .../oeqa/selftest/cases/incompatible_lic.py | 6 ++--- 3 files changed, 27 insertions(+), 7 deletions(-)meta/classes/license.bbclass +0
-0meta/classes/license.bbclass def canonical_license(d, license): · line 253
Quoted 2 lines
meta/classes/license.bbclass +0
-0@@ -252,7 +252,7 @@ def canonical_license(d, license): """ Return the canonical (SPDX) form of the license if available (soGPLv3
meta/classes/license.bbclass def canonical_license(d, license): · line 254
Quoted 1 line
meta/classes/license.bbclass +0
-0becomes GPL-3.0), for the license named 'X+', return canonical formof
meta/classes/license.bbclass def canonical_license(d, license): · line 267
Quoted 12 lines
meta/classes/license.bbclass +4
-1- 'X' if availabel and the tailing '+' (so GPLv3+ becomes GPL-3.0+), + 'X' if available and the tailing '+' (so GPLv3+ becomes GPL-3.0+), or the passed license if there is no canonical form. """ lic = d.getVarFlag('SPDXLICENSEMAP', license) or ""@@ -262,10 +262,29 @@ def canonical_license(d, license): lic += '+' return lic or license +def available_licenses(d): + """ + Return the available licenses by searching the directoriesspecified by
meta/classes/license.bbclass def canonical_license(d, license):
Quoted 13 lines
meta/classes/license.bbclass +13
-0+ COMMON_LICENSE_DIR and LICENSE_PATH. + """ + lic_dirs = ((d.getVar('COMMON_LICENSE_DIR') or '') + ' ' + + (d.getVar('LICENSE_PATH') or '')).split() + + licenses = [] + for lic_dir in lic_dirs: + licenses += os.listdir(lic_dir) + + licenses = sorted(licenses) + return licenses + +# Only determine the list of all available licenses once. This assumesthat any
meta/classes/license.bbclass def canonical_license(d, license):
Quoted 7 lines
meta/classes/license.bbclass +3
-2+# additions to LICENSE_PATH have been done before this file is parsed. +AVAILABLE_LICENSES := "${@' '.join(available_licenses(d))}" + def expand_wildcard_licenses(d, wildcard_licenses): """ - Return actual spdx format license names if wildcard used. We expand - wildcards from SPDXLICENSEMAP flags and SRC_DISTRIBUTE_LICENSESvalues.
meta/classes/license.bbclass def canonical_license(d, license): · line 286
Quoted 1 line
meta/classes/license.bbclass +1
-0+ Return actual spdx format license names if wildcards are used. Weexpand
meta/classes/license.bbclass def expand_wildcard_licenses(d, wildcard_licenses): · line 294
Quoted 7 lines
meta/classes/license.bbclass +1
-0+ wildcards from SPDXLICENSEMAP flags and AVAILABLE_LICENSES. """ import fnmatch licenses = wildcard_licenses[:]@@ -274,7 +293,7 @@ def expand_wildcard_licenses(d, wildcard_licenses): spdxflags = fnmatch.filter(spdxmapkeys, wld_lic) licenses += [d.getVarFlag('SPDXLICENSEMAP', flag) for flag inspdxflags]
meta/classes/license.bbclass def expand_wildcard_licenses(d, wildcard_licenses): · line 276
Quoted 2 lines
meta/classes/license.bbclass +0
-1- spdx_lics = (d.getVar('SRC_DISTRIBUTE_LICENSES', False) or'').split()
meta/conf/documentation.conf ASSUME_SHLIBS[doc] = "List of shlib:package[_version]
Quoted 9 lines
meta/classes/license.bbclass +1
-0+ spdx_lics = d.getVar('AVAILABLE_LICENSES').split() for wld_lic in wildcard_licenses: licenses += fnmatch.filter(spdx_lics, wld_lic)meta/conf/documentation.conf +0
-0@@ -69,6 +69,7 @@ ASSUME_SHLIBS[doc] = "List of shlib:package[_version]mappings. Useful for lib p
meta/conf/documentation.conf ASSUME_SHLIBS[doc] = "List of shlib:package[_version] · line 69
Quoted 1 line
meta/conf/documentation.conf +0
-0AUTHOR[doc] = "Email address used to contact the original author(s) inorder to send patches and forward bugs."
meta/conf/documentation.conf ASSUME_SHLIBS[doc] = "List of shlib:package[_version] · line 70
Quoted 1 line
meta/conf/documentation.conf +0
-0AUTO_SYSLINUXMENU[doc] = "Enables creating an automatic menu for thesyslinux bootloader."
meta/conf/documentation.conf ASSUME_SHLIBS[doc] = "List of shlib:package[_version] · line 71
Quoted 1 line
meta/conf/documentation.conf +0
-0AUTOREV[doc] = "When SRCREV is set to the value of this variable, itspecifies to use the latest source revision in the repository."
meta/conf/documentation.conf ASSUME_SHLIBS[doc] = "List of shlib:package[_version] · line 72
Quoted 1 line
meta/conf/documentation.conf +1
-0+AVAILABLE_LICENSES[doc] = "List of licenses found in the directoriesspecified by
COMMON_LICENSE_DIRandLICENSE_PATH."Review
Quoted 4 lines
meta/conf/documentation.conf +0
-0#B(quoted diff) +0
-0
b/meta/lib/oeqa/selftest/cases/incompatible_lic.pymeta/lib/oeqa/selftest/cases/incompatible_lic.py class IncompatibleLicenseTests(OESelftestTestCase): · line 15
Quoted 8 lines
index
904b5b4094..ad878571b5100644meta/lib/oeqa/selftest/cases/incompatible_lic.py +0
-1@@ -12,12 +12,12 @@ class IncompatibleLicenseTests(OESelftestTestCase): if error_msg not in result.output: raise AssertionError(result.output) - # Verify that a package with an SPDX license (from
SRC_DISTRIBUTE_LICENSES)meta/lib/oeqa/selftest/cases/incompatible_lic.py class IncompatibleLicenseTests(OESelftestTestCase): · line 15
Quoted 1 line
meta/lib/oeqa/selftest/cases/incompatible_lic.py +1
-0+ # Verify that a package with an SPDX license (from
AVAILABLE_LICENSES)meta/lib/oeqa/selftest/cases/incompatible_lic.py class IncompatibleLicenseTests(OESelftestTestCase): · line 16
Quoted 1 line
meta/lib/oeqa/selftest/cases/incompatible_lic.py +0
-0# cannot be built when INCOMPATIBLE_LICENSE contains this SPDXlicense
meta/lib/oeqa/selftest/cases/incompatible_lic.py class IncompatibleLicenseTests(OESelftestTestCase):
Quoted 4 lines
meta/lib/oeqa/selftest/cases/incompatible_lic.py +0
-1def test_incompatible_spdx_license(self): self.lic_test('incompatible-license', 'GPL-3.0', 'GPL-3.0') - # Verify that a package with an SPDX license (from
SRC_DISTRIBUTE_LICENSES)meta/lib/oeqa/selftest/cases/incompatible_lic.py class IncompatibleLicenseTests(OESelftestTestCase): · line 20
Quoted 1 line
meta/lib/oeqa/selftest/cases/incompatible_lic.py +1
-0+ # Verify that a package with an SPDX license (from
AVAILABLE_LICENSES)meta/lib/oeqa/selftest/cases/incompatible_lic.py class IncompatibleLicenseTests(OESelftestTestCase): · line 38
Quoted 8 lines
meta/lib/oeqa/selftest/cases/incompatible_lic.py +0
-1# cannot be built when INCOMPATIBLE_LICENSE contains an alias (in # SPDXLICENSEMAP) of this SPDX license def test_incompatible_alias_spdx_license(self):@@ -35,7 +35,7 @@ class IncompatibleLicenseTests(OESelftestTestCase): self.lic_test('incompatible-license-alias', 'GPLv3', 'GPLv3') # Verify that a package with a non-SPDX license (neither in - # SRC_DISTRIBUTE_LICENSES nor in SPDXLICENSEMAP) cannot be builtwhen
meta/lib/oeqa/selftest/cases/incompatible_lic.py class IncompatibleLicenseTests(OESelftestTestCase):
Quoted 4 lines
meta/lib/oeqa/selftest/cases/incompatible_lic.py +1
-0+ # AVAILABLE_LICENSES nor in SPDXLICENSEMAP) cannot be built when # INCOMPATIBLE_LICENSE contains this license def test_incompatible_nonspdx_license(self): self.lic_test('incompatible-nonspdx-license', 'FooLicense','FooLicense')
Quoted 8 lines
--
2.21.0--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Signature · 4 lines
Christopher Larson kergoth at gmail dot com Founder - BitBake, OpenEmbedded, OpenZaurus Senior Software Engineer, Mentor Graphics
Quoted 4 lines · Christopher Larson
This actually also results in checksums changing for recipes that inherit license and call
incompatible_license_contains()when adding/removing layers that add to the license path, such as lvm2.
Quoted 12 lines
Quoted 10 lines · Denys Dmytriyenko
This actually fails yocto-check-layer script for Yocto compliance checking, when a layer provides own licenses via
LICENSE_PATH(in layer.conf -LICENSE_PATH+= "${LAYERDIR}/licenses"):Variable
AVAILABLE_LICENSESvalue changed:
...Is there a new way to provide additional licenses from a layer or should the yocto-check-layer script be modified to accomodate (i.e. ignore) this change?
I think we either need to vardepsexclude this variable from its usage sites or we need to figure out a different way of handling these license issues.
I worry about generically whitelisting it as there are cases where adding a new license should cause rebuilds.
We'll therefore have to look at the usage sites and decide if we want a rebuild for each usage.
Cheers,
Richard