License compliance fixes

Hi,

If INCOMPATIBLE_LICENSE=GPLv3.0 but the recipe sets LICENSE=GPLv3, the current code won't trigger because they're different strings.

Fix this by attempting to canonicalise every license name to a SPDX name, so both names in this example become GPL-3.0.

The whitelists also have to be updated. This means using variable names that could be considered non-standard (or "crazy"), such as WHITELIST_GPL-3.0. This does look strange and if the community agrees we can take the compatibility hit I'll change the whitelists to use flags, i.e HOSTTOOLS_WHITELIST[GPL-3.0].

Ross

The following changes since commit 1dcdd877c7946be4c0b1203deb14e2f842f9d0c2:

bitbake: toasterui: fix build - project identification (2014-07-14 14:10:03 +0100)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib ross/license

for you to fetch changes up to 934289b8ba6a8b7a0f1eec8aabf723f823d515d8:

default-distrovars: update license whitelists to use canonical names (2014-07-16 21:02:45 +0100)

----------------------------------------------------------------
Ross Burton (2):
license.bbclass: canonicalise licenses when dealing with INCOMPATIBLE_LICENSE
default-distrovars: update license whitelists to use canonical names

 meta/classes/base.bbclass                       |    2 ++
 meta/classes/license.bbclass                    |   14 +++++++++++---
 meta/conf/distro/include/default-distrovars.inc |    6 +++---
 3 files changed, 16 insertions(+), 6 deletions(-)

Ross Burton (2):
license.bbclass: canonicalise licenses when dealing with
INCOMPATIBLE_LICENSE
default-distrovars: update license whitelists to use canonical names

 meta/classes/base.bbclass                       |    2 ++
 meta/classes/license.bbclass                    |   14 +++++++++++---
 meta/conf/distro/include/default-distrovars.inc |    6 +++---
 3 files changed, 16 insertions(+), 6 deletions(-)

1.7.10.4

If INCOMPATIBLE_LICENSE=GPLv3.0 but the recipe sets LICENSE=GPLv3, the current code won't trigger because they're different strings.

Fix this by attempting to canonicalise every license name to a SPDX name, so both names in this example become GPL-3.0.

[ YOCTO #5622 ]

---
 meta/classes/base.bbclass    |    2 ++
 meta/classes/license.bbclass |   14 +++++++++++---
 2 files changed, 13 insertions(+), 3 deletions(-)

index c0d2c8ec8..8114cf6 100644
@@ -540,6 +540,8 @@ python () {
                 check_license = False
 
         if check_license and bad_licenses:
+            bad_licenses = map(lambda l: canonical_license(d, l), bad_licenses)
+
             whitelist = []
             for lic in bad_licenses:
                 for w in ["HOSTTOOLS_WHITELIST_", "LGPLv2_WHITELIST_", "WHITELIST_"]:

index 08f0665..973a95b 100644
@@ -264,10 +264,18 @@ def return_spdx(d, license):
      """
     return d.getVarFlag('SPDXLICENSEMAP', license, True)
 
+def canonical_license(d, license):
+    """
+    Return the canonical (SPDX) form of the license if available (so GPLv3
+    becomes GPL-3.0), or the passed license if there is no canonical form.
+    """
+    return d.getVarFlag('SPDXLICENSEMAP', license, True) or license
+
 def incompatible_license(d, dont_want_licenses, package=None):
     """
-    This function checks if a recipe has only incompatible licenses. It also take into consideration 'or'
-    operand.
+    This function checks if a recipe has only incompatible licenses. It also
+    take into consideration 'or' operand.  dont_want_licenses should be passed
+    as canonical (SPDX) names.
     """
     import re
     import oe.license

@@ -298,7 +306,7 @@ def incompatible_license(d, dont_want_licenses, package=None):
         licenses = oe.license.flattened_licenses(license, choose_lic_set)
     except oe.license.LicenseError as exc:
         bb.fatal('%s: %s' % (d.getVar('P', True), exc))
-    return any(not license_ok(l) for l in licenses)
+    return any(not license_ok(canonical_license(d, l)) for l in licenses)
 
 def check_license_flags(d):
     """

1.7.10.4

Pedantic, but INCOMPATIBLE_LICENSE = "GPL-3.0" surely?

Cheers,
Paul

Paul Eggleton
Intel Open Source Technology Centre

Yes, sigh.

Ross

Now that all licenses are canonicalised to SPDX names when processing, we need to rename the whitelists to the match.

---
 meta/conf/distro/include/default-distrovars.inc |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

index 1bc8e46..8d24d66 100644
@@ -23,9 +23,9 @@ IMAGE_FEATURES ?= ""
 
 # This is a list of packages that are used by the build system to build the distribution, they are not
 # directly part of the distribution. 
-HOSTTOOLS_WHITELIST_GPLv3 ?= ""
-WHITELIST_GPLv3 ?= "less"
-LGPLv2_WHITELIST_GPLv3 ?= "libassuan gnutls libtasn1 libidn libgcc libgcc-initial gcc-runtime" 
+HOSTTOOLS_WHITELIST_GPL-3.0 ?= ""
+WHITELIST_GPL-3.0 ?= "less"
+LGPLv2_WHITELIST_GPL-3.0 ?= "libassuan gnutls libtasn1 libidn libgcc libgcc-initial gcc-runtime"
 
 COMMERCIAL_AUDIO_PLUGINS ?= ""
 # COMMERCIAL_AUDIO_PLUGINS ?= "gst-plugins-ugly-mad gst-plugins-ugly-mpegaudioparse"

1.7.10.4

File "/srv/ssd/poky/bitbake/lib/bb/event.py", line 158, in fire(event=<bb.event.RecipePreFinalise object at 0xaa61110>, d=<bb.data_smart.DataSmart object at 0x4eb9690>):

if worker_fire:
File "/srv/ssd/poky/bitbake/lib/bb/event.py", line 97, in fire_class_handlers(event=<bb.event.RecipePreFinalise object at 0xaa61110>, d=<bb.data_smart.DataSmart object at 0x4eb9690>):
if name in _catchall_handlers or name in evt_hmap:
> execute_handler(name, handler, event, d)

File "/srv/ssd/poky/bitbake/lib/bb/event.py", line 74, in execute_handler(name='multilib_virtclass_handler', handler=<function multilib_virtclass_handler at 0x2eff140>, event=<bb.event.RecipePreFinalise object at 0xaa61110>, d=<bb.data_smart.DataSmart object at 0x4eb9690>):
try:
> ret = handler(event)
except (bb.parse.SkipRecipe, bb.BBHandledException):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0xaa61110>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0x92ef390>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0xa0bdb10>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0xa704050>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0x251b110>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0xc2bf550>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0x9fa5c10>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0x251b110>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0xc2d00d0>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0xc2d9150>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0x92efc90>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0x2ede8d0>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0xc2d0ed0>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0x4eccd90>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0x4eb9c10>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0x2eecb10>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0x9205fd0>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0xc2eab10>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0x92ef650>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0x2ed4c90>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0x251b110>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0x92efad0>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0xc2e6510>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0x2ede890>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0x9b6f390>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0x920a850>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0x9d6e590>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0x4eb4f10>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0x2edea90>)
AttributeError: 'NoneType' object has no attribute 'split'

ERROR: Execution of event handler 'multilib_virtclass_handler' failed
Traceback (most recent call last):
File "multilib_virtclass_handler(e)", line 55, in multilib_virtclass_handler(e=<bb.event.RecipePreFinalise object at 0x2ee7d90>)
AttributeError: 'NoneType' object has no attribute 'split'

Summary: There were 33 ERROR messages shown, returning a non-zero exit code.

FWIW I've fixed this before I merged it, it needed a tweak to multilib.bbclass...

Cheers,

Richard