Skip to contentSkip to conversation

copyleft/license bits

PATCH Applied Christopher Larson 2 participants 6 messages Jan 10, 2012 – Jan 17, 2012

lore
Share & tools
lore
#0 lore raw

cover copyleft/license bits

From: Christopher Larson <[email protected]>

The following changes since commit fff18970706913e7fd7f4a119d798dddb44b388a:

base-files: filesystems: fix mount order (2012-01-06 14:41:19 +0000)

are available in the git repository at:
https://github.com/kergoth/oe-core license-bits

Christopher Larson (4):
copyleft_compliance: add control of recipe types to include
copyleft_compliance: add debug message with the reason for exclusion oe.license: add is_included convenience function oe.license: avoid the need to catch SyntaxError

 meta/classes/copyleft_compliance.bbclass |   56 ++++++++++++++++------------
 meta/lib/oe/license.py                   |   59 ++++++++++++++++++++++++++++--
 meta/lib/test.py                         |    3 ++
 3 files changed, 90 insertions(+), 28 deletions(-)
 create mode 100644 meta/lib/test.py

Christopher Larson (4):
copyleft_compliance: add control of recipe types to include
copyleft_compliance: add debug message with the reason for exclusion oe.license: add is_included convenience function oe.license: avoid the need to catch SyntaxError

 meta/classes/copyleft_compliance.bbclass |   56 ++++++++++++++++------------
 meta/lib/oe/license.py                   |   59 ++++++++++++++++++++++++++++--
 meta/lib/test.py                         |    3 ++
 3 files changed, 90 insertions(+), 28 deletions(-)
 create mode 100644 meta/lib/test.py
Signature · 1 line
1.7.8.rc4
#1 lore raw

PATCH 1/4 copyleft_compliance: add control of recipe types to include

From: Christopher Larson <[email protected]>

In this context, recipe "type" refers to whether it is 'target', 'native', 'cross', etc. COPYLEFT_RECIPE_TYPES is a space separated list of types to include. It defaults to 'target'.

---
 meta/classes/copyleft_compliance.bbclass |   21 ++++++++++++++++++---
 1 files changed, 18 insertions(+), 3 deletions(-)
meta/classes/copyleft_compliance.bbclass +18 -3
index 2fe524b..37ed09e 100644
@@ -16,6 +16,22 @@ COPYLEFT_LICENSE_EXCLUDE ?= 'CLOSED Proprietary'
 COPYLEFT_LICENSE_EXCLUDE[type] = 'list'
 COPYLEFT_LICENSE_INCLUDE[doc] = 'Space separated list of globs which exclude licenses'
 
+COPYLEFT_RECIPE_TYPE ?= '${@copyleft_recipe_type(d)}'
+COPYLEFT_RECIPE_TYPE[doc] = 'The "type" of the current recipe (e.g. target, native, cross)'
+
+COPYLEFT_RECIPE_TYPES ?= 'target'
+COPYLEFT_RECIPE_TYPES[type] = 'list'
+COPYLEFT_RECIPE_TYPES[doc] = 'Space separated list of recipe types to include'
+
+COPYLEFT_AVAILABLE_RECIPE_TYPES = 'target native nativesdk cross crosssdk cross-canadian'
+COPYLEFT_AVAILABLE_RECIPE_TYPES[type] = 'list'
+COPYLEFT_AVAILABLE_RECIPE_TYPES[doc] = 'Space separated list of available recipe types'
+
+def copyleft_recipe_type(d):
+    for recipe_type in oe.data.typed_value('COPYLEFT_AVAILABLE_RECIPE_TYPES', d):
+        if oe.utils.inherits(d, recipe_type):
+            return recipe_type
+    return 'target'
 
 def copyleft_should_include(d):
     """Determine if this recipe's sources should be deployed for compliance"""
@@ -23,9 +39,8 @@ def copyleft_should_include(d):
     import oe.license
     from fnmatch import fnmatchcase as fnmatch
 
-    if oe.utils.inherits(d, 'native', 'nativesdk', 'cross', 'crossdk'):
-        # not a target recipe
-        return
+    if d.getVar('COPYLEFT_RECIPE_TYPE', True) not in oe.data.typed_value('COPYLEFT_RECIPE_TYPES', d):
+        return False
 
     include = oe.data.typed_value('COPYLEFT_LICENSE_INCLUDE', d)
     exclude = oe.data.typed_value('COPYLEFT_LICENSE_EXCLUDE', d)
Signature · 1 line
1.7.8.rc4
#2 lore raw

PATCH 2/4 copyleft_compliance: add debug message with the reason for exclusion

In reply to Christopher Larson

From: Christopher Larson <[email protected]>

---
 meta/classes/copyleft_compliance.bbclass |   24 +++++++++++++++++-------
 1 files changed, 17 insertions(+), 7 deletions(-)
meta/classes/copyleft_compliance.bbclass +17 -7
index 37ed09e..fd04638 100644
@@ -39,8 +39,9 @@ def copyleft_should_include(d):
     import oe.license
     from fnmatch import fnmatchcase as fnmatch
 
-    if d.getVar('COPYLEFT_RECIPE_TYPE', True) not in oe.data.typed_value('COPYLEFT_RECIPE_TYPES', d):
-        return False
+    recipe_type = d.getVar('COPYLEFT_RECIPE_TYPE', True)
+    if recipe_type not in oe.data.typed_value('COPYLEFT_RECIPE_TYPES', d):
+        return False, 'recipe type "%s" is excluded' % recipe_type
 
     include = oe.data.typed_value('COPYLEFT_LICENSE_INCLUDE', d)
     exclude = oe.data.typed_value('COPYLEFT_LICENSE_EXCLUDE', d)
@@ -63,18 +64,27 @@ def copyleft_should_include(d):
         licenses = oe.license.flattened_licenses(d.getVar('LICENSE', True), choose_licenses)
     except oe.license.InvalidLicense as exc:
         bb.fatal('%s: %s' % (d.getVar('PF', True), exc))
-    except SyntaxError:
-        bb.warn("%s: Failed to parse it's LICENSE field." % (d.getVar('PF', True)))
-
-    return all(include_license(lic) for lic in licenses)
+    except SyntaxError as exc:
+        bb.warn('%s: error when parsing the LICENSE variable: %s' % (d.getVar('P', True), exc))
+    else:
+        excluded = filter(lambda lic: not include_license(lic), licenses)
+        if excluded:
+            return False, 'recipe has excluded licenses: %s' % ', '.join(excluded)
+        else:
+            return True, None
 
 python do_prepare_copyleft_sources () {
     """Populate a tree of the recipe sources and emit patch series files"""
     import os.path
     import shutil
 
-    if not copyleft_should_include(d):
+    p = d.getVar('P', True)
+    included, reason = copyleft_should_include(d)
+    if not included:
+        bb.debug(1, 'copyleft: %s is excluded: %s' % (p, reason))
         return
+    else:
+        bb.debug(1, 'copyleft: %s is included' % p)
 
     sources_dir = d.getVar('COPYLEFT_SOURCES_DIR', 1)
     src_uri = d.getVar('SRC_URI', 1).split()
Signature · 1 line
1.7.8.rc4
#3 lore raw

PATCH 3/4 oe.license: add is_included convenience function

In reply to Christopher Larson

From: Christopher Larson <[email protected]>

Given a license string and whitelist and blacklist, determine if the license string matches the whitelist and does not match the blacklist.

When encountering an OR, it prefers the side with the highest weight (more included licenses). It then checks the inclusion of the flattened list of licenses from there.

Returns a tuple holding the boolean state and a list of the applicable licenses which were excluded (or None, if the state is True)

Examples:

is_included, excluded = oe.license.is_included(licensestr, ['GPL*', 'LGPL*']) is_included, excluded = oe.license.is_included(licensestr, blacklist=['Proprietary', 'CLOSED'])

---
 meta/classes/copyleft_compliance.bbclass |   23 +++---------------
 meta/lib/oe/license.py                   |   36 ++++++++++++++++++++++++++++++
 meta/lib/test.py                         |    3 ++
 3 files changed, 43 insertions(+), 19 deletions(-)
 create mode 100644 meta/lib/test.py
meta/classes/copyleft_compliance.bbclass +4 -19
index fd04638..6f058e0 100644
@@ -46,32 +46,17 @@ def copyleft_should_include(d):
     include = oe.data.typed_value('COPYLEFT_LICENSE_INCLUDE', d)
     exclude = oe.data.typed_value('COPYLEFT_LICENSE_EXCLUDE', d)
 
-    def include_license(license):
-        if any(fnmatch(license, pattern) for pattern in exclude):
-            return False
-        if any(fnmatch(license, pattern) for pattern in include):
-            return True
-        return False
-
-    def choose_licenses(a, b):
-        """Select the left option in an OR if all its licenses are to be included"""
-        if all(include_license(lic) for lic in a):
-            return a
-        else:
-            return b
-
     try:
-        licenses = oe.license.flattened_licenses(d.getVar('LICENSE', True), choose_licenses)
+        is_included, excluded = oe.license.is_included(d.getVar('LICENSE', True), include, exclude)
     except oe.license.InvalidLicense as exc:
         bb.fatal('%s: %s' % (d.getVar('PF', True), exc))
     except SyntaxError as exc:
         bb.warn('%s: error when parsing the LICENSE variable: %s' % (d.getVar('P', True), exc))
     else:
-        excluded = filter(lambda lic: not include_license(lic), licenses)
-        if excluded:
-            return False, 'recipe has excluded licenses: %s' % ', '.join(excluded)
-        else:
+        if is_included:
             return True, None
+        else:
+            return False, 'recipe has excluded licenses: %s' % ', '.join(excluded)
 
 python do_prepare_copyleft_sources () {
     """Populate a tree of the recipe sources and emit patch series files"""
meta/lib/oe/license.py +36 -0
index 7ab66e7..3543cfe 100644
@@ -3,6 +3,7 @@
 
 import ast
 import re
+from fnmatch import fnmatchcase as fnmatch
 
 class InvalidLicense(StandardError):
     def __init__(self, license):
@@ -60,3 +61,38 @@ def flattened_licenses(licensestr, choose_licenses):
     flatten = FlattenVisitor(choose_licenses)
     flatten.visit_string(licensestr)
     return flatten.licenses
+
+def is_included(licensestr, whitelist=None, blacklist=None):
+    """Given a license string and whitelist and blacklist, determine if the
+    license string matches the whitelist and does not match the blacklist.
+
+    Returns a tuple holding the boolean state and a list of the applicable
+    licenses which were excluded (or None, if the state is True)
+    """
+
+    def include_license(license):
+        return (any(fnmatch(license, pattern) for pattern in whitelist) and not
+                any(fnmatch(license, pattern) for pattern in blacklist))
+
+    def choose_licenses(alpha, beta):
+        """Select the option in an OR which is the 'best' (has the most
+        included licenses)."""
+        alpha_weight = len(filter(include_license, alpha))
+        beta_weight = len(filter(include_license, beta))
+        if alpha_weight > beta_weight:
+            return alpha
+        else:
+            return beta
+
+    if not whitelist:
+        whitelist = ['*']
+
+    if not blacklist:
+        blacklist = []
+
+    licenses = flattened_licenses(licensestr, choose_licenses)
+    excluded = filter(lambda lic: not include_license(lic), licenses)
+    if excluded:
+        return False, excluded
+    else:
+        return True, None
meta/lib/test.py new file +3 -0
new file mode 100644
index 0000000..12f4d83
@@ -0,0 +1,3 @@
+import oe.license
+
+print(oe.license.is_included('LGPLv2.1 | GPLv3', ['*'], []))
Signature · 1 line
1.7.8.rc4
#4 lore raw

PATCH 4/4 oe.license: avoid the need to catch SyntaxError

In reply to Christopher Larson

From: Christopher Larson <[email protected]>

---
 meta/classes/copyleft_compliance.bbclass |    4 +---
 meta/lib/oe/license.py                   |   23 +++++++++++++++++++----
 2 files changed, 20 insertions(+), 7 deletions(-)
meta/classes/copyleft_compliance.bbclass +1 -3
index 6f058e0..2eb9ded 100644
@@ -48,10 +48,8 @@ def copyleft_should_include(d):
 
     try:
         is_included, excluded = oe.license.is_included(d.getVar('LICENSE', True), include, exclude)
-    except oe.license.InvalidLicense as exc:
+    except oe.license.LicenseError as exc:
         bb.fatal('%s: %s' % (d.getVar('PF', True), exc))
-    except SyntaxError as exc:
-        bb.warn('%s: error when parsing the LICENSE variable: %s' % (d.getVar('P', True), exc))
     else:
         if is_included:
             return True, None
meta/lib/oe/license.py +19 -4
index 3543cfe..5914506 100644
@@ -5,13 +5,25 @@ import ast
 import re
 from fnmatch import fnmatchcase as fnmatch
 
-class InvalidLicense(StandardError):
+class LicenseError(StandardError):
+    pass
+
+class LicenseSyntaxError(LicenseError):
+    def __init__(self, licensestr, exc):
+        self.licensestr = licensestr
+        self.exc = exc
+        LicenseError.__init__(self)
+
+    def __str__(self):
+        return "error in '%s': %s" % (self.licensestr, self.exc)
+
+class InvalidLicense(LicenseError):
     def __init__(self, license):
         self.license = license
-        StandardError.__init__(self)
+        LicenseError.__init__(self)
 
     def __str__(self):
-        return "invalid license '%s'" % self.license
+        return "invalid characters in license '%s'" % self.license
 
 license_operator = re.compile('([&|() ])')
 license_pattern = re.compile('[a-zA-Z0-9.+_\-]+$')
@@ -59,7 +71,10 @@ class FlattenVisitor(LicenseVisitor):
 def flattened_licenses(licensestr, choose_licenses):
     """Given a license string and choose_licenses function, return a flat list of licenses"""
     flatten = FlattenVisitor(choose_licenses)
-    flatten.visit_string(licensestr)
+    try:
+        flatten.visit_string(licensestr)
+    except SyntaxError as exc:
+        raise LicenseSyntaxError(licensestr, exc)
     return flatten.licenses
 
 def is_included(licensestr, whitelist=None, blacklist=None):
Signature · 1 line
1.7.8.rc4
Applied
#5 lore raw
In reply to Christopher Larson
Quoted 27 lines · Christopher Larson

From: Christopher Larson<[email protected]>

The following changes since commit fff18970706913e7fd7f4a119d798dddb44b388a:

   base-files: filesystems: fix mount order (2012-01-06 14:41:19 +0000)

are available in the git repository at:
https://github.com/kergoth/oe-core license-bits

Christopher Larson (4):
copyleft_compliance: add control of recipe types to include
copyleft_compliance: add debug message with the reason for exclusion oe.license: add is_included convenience function oe.license: avoid the need to catch SyntaxError

  meta/classes/copyleft_compliance.bbclass |   56 ++++++++++++++++------------
  meta/lib/oe/license.py                   |   59 ++++++++++++++++++++++++++++--
  meta/lib/test.py                         |    3 ++
  3 files changed, 90 insertions(+), 28 deletions(-)

create mode 100644 meta/lib/test.py

Christopher Larson (4):
copyleft_compliance: add control of recipe types to include
copyleft_compliance: add debug message with the reason for exclusion oe.license: add is_included convenience function oe.license: avoid the need to catch SyntaxError

Quoted 6 lines
  meta/classes/copyleft_compliance.bbclass |   56 ++++++++++++++++------------
  meta/lib/oe/license.py                   |   59 ++++++++++++++++++++++++++++--
  meta/lib/test.py                         |    3 ++
  3 files changed, 90 insertions(+), 28 deletions(-)

create mode 100644 meta/lib/test.py

Merged into OE-Core

Thanks
Sau!