Skip to contentSkip to conversation

license.bbclass: split incompatible license check into a helper function

RFC PATCH In review Alexander Kanavin 3 participants 5 messages Oct 9, 2019 – Oct 10, 2019

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/2 license.bbclass: split incompatible license check into a helper function

This would help with checking incompatible licenses at do_rootfs step in image creation, where it is beneficial to pass the license string directly to the function.

---
 meta/classes/license.bbclass | 25 ++++++++++++++-----------
 1 file changed, 14 insertions(+), 11 deletions(-)
meta/classes/license.bbclass +14 -11
index adca881c85b..648a4d78922 100644
@@ -287,17 +287,7 @@ def incompatible_license_contains(license, truevalue, falsevalue, d):
     bad_licenses = expand_wildcard_licenses(d, bad_licenses)
     return truevalue if license in bad_licenses else falsevalue
 
-def incompatible_license(d, dont_want_licenses, package=None):
-    """
-    This function checks if a recipe has only incompatible licenses. It also
-    take into consideration 'or' operand.  dont_want_licenses should be passed
-    as canonical (SPDX) names.
-    """
-    import oe.license
-    license = d.getVar("LICENSE_%s" % package) if package else None
-    if not license:
-        license = d.getVar('LICENSE')
-
+def incompatible_pkg_license(d, dont_want_licenses, license):
     # Handles an "or" or two license sets provided by
     # flattened_licenses(), pick one that works if possible.
     def choose_lic_set(a, b):
@@ -311,6 +301,19 @@ def incompatible_license(d, dont_want_licenses, package=None):
     return any(not oe.license.license_ok(canonical_license(d, l), \
                dont_want_licenses) for l in licenses)
 
+def incompatible_license(d, dont_want_licenses, package=None):
+    """
+    This function checks if a recipe has only incompatible licenses. It also
+    take into consideration 'or' operand.  dont_want_licenses should be passed
+    as canonical (SPDX) names.
+    """
+    import oe.license
+    license = d.getVar("LICENSE_%s" % package) if package else None
+    if not license:
+        license = d.getVar('LICENSE')
+
+    return incompatible_pkg_license(d, dont_want_licenses, license)
+
 def check_license_flags(d):
     """
     This function checks if a recipe has any LICENSE_FLAGS that
Signature · 1 line
2.17.1
#1 lore raw

RFC PATCH 2/2 license_image.bbclass: check and reject packages which have incompatible licenses

The use case is setting INCOMPATIBLE_LICENSE per image, rather than as an awkward, and too strict global setting.

This for example would allow building development images with gplv3 tools, but production images without them, and checking that nothing gpl3-licensed gets into the latter.

Examples are provided via the selftest: three scenarios are tested:

- bash is added to the image, with a default gpl3 license; this is rejected
- bash is added to the image, with a "gpl3 & other" license; this is also rejected
- bash is added to the image, with a "gpl3 | other" license; this is accepted, but only 'other' is added to the license manifest (this was already handled correctly previously).

Eventually, this would allow deprecating the meta-gplv2 layer, while still enforcing the no-gpl3 rule where possible and needed.

---
 meta/classes/license_image.bbclass            |  2 ++
 .../oeqa/selftest/cases/incompatible_lic.py   | 29 +++++++++++++++++++
 2 files changed, 31 insertions(+)
meta/classes/license_image.bbclass +2 -0
index 3f102d0fbc3..e5cb1b8c77d 100644
@@ -47,6 +47,8 @@ def write_license_files(d, license_manifest, pkg_dic, rootfs=True):
         for pkg in sorted(pkg_dic):
             if bad_licenses:
                 try:
+                    if incompatible_pkg_license(d, bad_licenses, pkg_dic[pkg]["LICENSE"]):
+                        bb.fatal("Package %s has an incompatible license %s and cannot be installed into the image." %(pkg, pkg_dic[pkg]["LICENSE"]))
                     (pkg_dic[pkg]["LICENSE"], pkg_dic[pkg]["LICENSES"]) = \
                         oe.license.manifest_licenses(pkg_dic[pkg]["LICENSE"],
                         bad_licenses, canonical_license, d)
meta/lib/oeqa/selftest/cases/incompatible_lic.py +29 -0
index 8fb93af8a89..e80c9783122 100644
@@ -39,3 +39,32 @@ class IncompatibleLicenseTests(OESelftestTestCase):
     # INCOMPATIBLE_LICENSE contains this license
     def test_incompatible_nonspdx_license(self):
         self.lic_test('incompatible-nonspdx-license', 'FooLicense', 'FooLicense')
+
+class IncompatibleLicensePerImageTests(OESelftestTestCase):
+    def default_config(self):
+        return """
+IMAGE_INSTALL_append = "bash"
+INCOMPATIBLE_LICENSE_pn-core-image-minimal = "GPL-3.0 LGPL-3.0"
+"""
+
+    def test_bash_default(self):
+        self.write_config(self.default_config())
+        error_msg = "ERROR: core-image-minimal-1.0-r0 do_rootfs: Package bash has an incompatible license GPLv3+ and cannot be installed into the image."
+
+        result = bitbake('core-image-minimal', ignore_status=True)
+        if error_msg not in result.output:
+            raise AssertionError(result.output)
+
+    def test_bash_and_license(self):
+        self.write_config(self.default_config() + '\nLICENSE_append_pn-bash = " & SomeLicense"')
+        error_msg = "ERROR: core-image-minimal-1.0-r0 do_rootfs: Package bash has an incompatible license GPLv3+ & SomeLicense and cannot be installed into the image."
+
+        result = bitbake('core-image-minimal', ignore_status=True)
+        if error_msg not in result.output:
+            raise AssertionError(result.output)
+
+    def test_bash_or_license(self):
+        self.write_config(self.default_config() + '\nLICENSE_append_pn-bash = " | SomeLicense"')
+
+        bitbake('core-image-minimal')
+
Signature · 1 line
2.17.1
#2 lore raw

Re: [RFC PATCH 2/2] license_image.bbclass: check and reject packages which have incompatible licenses

Does this obey the whitelist? Can I whitelist a gplv3 package to get it installed, ideally with a warning rather than silent or error? I don't have the code in front of me at the moment. I have a class in meta-mentor that does this sort of check, so wanted to see how the implementations compare. On a related note,
https://github.com/MentorEmbedded/meta-mentor/blob/master/meta-mentor-common/classes/incompatible-recipe-check.bbclass
+
https://github.com/MentorEmbedded/meta-mentor/blob/95d05dcc12651a7b246f91b240120f92d196b0de/meta-mel-support/recipes-core/packagegroups/packagegroup-tools-benchmark.bb#L10
might be of interest to you in your gplv2 work, perhaps? Specifically handling the whitelisting.

Quoted 100 lines · Alexander Kanavin

The use case is setting INCOMPATIBLE_LICENSE per image, rather than as an awkward, and too strict global setting.

This for example would allow building development images with gplv3 tools, but production images without them, and checking that nothing gpl3-licensed gets into the latter.

Examples are provided via the selftest: three scenarios are tested:

- bash is added to the image, with a default gpl3 license; this is rejected
- bash is added to the image, with a "gpl3 & other" license; this is also rejected
- bash is added to the image, with a "gpl3 | other" license; this is accepted, but
only 'other' is added to the license manifest (this was already handled correctly
previously).

Eventually, this would allow deprecating the meta-gplv2 layer, while still enforcing the no-gpl3 rule where possible and needed.

---
 meta/classes/license_image.bbclass            |  2 ++
 .../oeqa/selftest/cases/incompatible_lic.py   | 29 +++++++++++++++++++
 2 files changed, 31 insertions(+)
meta/classes/license_image.bbclass +0 -0
index 3f102d0fbc3..e5cb1b8c77d 100644
@@ -47,6 +47,8 @@ def write_license_files(d, license_manifest, pkg_dic,

rootfs=True):
for pkg in sorted(pkg_dic):
if bad_licenses:
try:
+ if incompatible_pkg_license(d, bad_licenses,
pkg_dic[pkg]["LICENSE"]):
+ bb.fatal("Package %s has an incompatible license
%s and cannot be installed into the image." %(pkg, pkg_dic[pkg]["LICENSE"]))

                     (pkg_dic[pkg]["LICENSE"], pkg_dic[pkg]["LICENSES"]) =

\

oe.license.manifest_licenses(pkg_dic[pkg]["LICENSE"],
bad_licenses, canonical_license, d)

meta/lib/oeqa/selftest/cases/incompatible_lic.py +0 -0
index 8fb93af8a89..e80c9783122 100644
@@ -39,3 +39,32 @@ class IncompatibleLicenseTests(OESelftestTestCase):
     # INCOMPATIBLE_LICENSE contains this license
     def test_incompatible_nonspdx_license(self):
         self.lic_test('incompatible-nonspdx-license', 'FooLicense',

'FooLicense')

meta/lib/oeqa/selftest/cases/incompatible_lic.py +11 -0
+
+class IncompatibleLicensePerImageTests(OESelftestTestCase):
+    def default_config(self):
+        return """
+IMAGE_INSTALL_append = "bash"
+INCOMPATIBLE_LICENSE_pn-core-image-minimal = "GPL-3.0 LGPL-3.0"
+"""
+
+    def test_bash_default(self):
+        self.write_config(self.default_config())
+        error_msg = "ERROR: core-image-minimal-1.0-r0 do_rootfs: Package

bash has an incompatible license GPLv3+ and cannot be installed into the image."

meta/lib/oeqa/selftest/cases/incompatible_lic.py +7 -0
+
+        result = bitbake('core-image-minimal', ignore_status=True)
+        if error_msg not in result.output:
+            raise AssertionError(result.output)
+
+    def test_bash_and_license(self):
+        self.write_config(self.default_config() +

'\nLICENSE_append_pn-bash = " & SomeLicense"')
+ error_msg = "ERROR: core-image-minimal-1.0-r0 do_rootfs: Package
bash has an incompatible license GPLv3+ & SomeLicense and cannot be installed into the image."

meta/lib/oeqa/selftest/cases/incompatible_lic.py +7 -0
+
+        result = bitbake('core-image-minimal', ignore_status=True)
+        if error_msg not in result.output:
+            raise AssertionError(result.output)
+
+    def test_bash_or_license(self):
+        self.write_config(self.default_config() +

'\nLICENSE_append_pn-bash = " | SomeLicense"')

meta/lib/oeqa/selftest/cases/incompatible_lic.py +3 -1
+
+        bitbake('core-image-minimal')
+
--

2.17.1

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Signature · 4 lines
Christopher Larson
kergoth at gmail dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Senior Software Engineer, Mentor Graphics
#3 lore raw

Re: [RFC PATCH 2/2] license_image.bbclass: check and reject packages which have incompatible licenses

It wouldn't be too hard to add a condition that checks the (image-specific) whitelist, I just wanted to gather a bit of feedback for the overall idea :)

Alex

Quoted 124 lines · Christopher Larson

Does this obey the whitelist? Can I whitelist a gplv3 package to get it installed, ideally with a warning rather than silent or error? I don't have the code in front of me at the moment. I have a class in meta-mentor that does this sort of check, so wanted to see how the implementations compare. On a related note,
https://github.com/MentorEmbedded/meta-mentor/blob/master/meta-mentor-common/classes/incompatible-recipe-check.bbclass
+
https://github.com/MentorEmbedded/meta-mentor/blob/95d05dcc12651a7b246f91b240120f92d196b0de/meta-mel-support/recipes-core/packagegroups/packagegroup-tools-benchmark.bb#L10
might be of interest to you in your gplv2 work, perhaps? Specifically handling the whitelisting.

Quoted 96 lines · Alexander Kanavin

The use case is setting INCOMPATIBLE_LICENSE per image, rather than as an awkward, and too strict global setting.

This for example would allow building development images with gplv3 tools, but production images without them, and checking that nothing gpl3-licensed
gets into the latter.

Examples are provided via the selftest: three scenarios are tested:

- bash is added to the image, with a default gpl3 license; this is rejected
- bash is added to the image, with a "gpl3 & other" license; this is also rejected
- bash is added to the image, with a "gpl3 | other" license; this is accepted, but
only 'other' is added to the license manifest (this was already handled correctly
previously).

Eventually, this would allow deprecating the meta-gplv2 layer, while still enforcing the no-gpl3 rule where possible and needed.

---
 meta/classes/license_image.bbclass            |  2 ++
 .../oeqa/selftest/cases/incompatible_lic.py   | 29 +++++++++++++++++++
 2 files changed, 31 insertions(+)
meta/classes/license_image.bbclass +0 -0
index 3f102d0fbc3..e5cb1b8c77d 100644
@@ -47,6 +47,8 @@ def write_license_files(d, license_manifest, pkg_dic,

rootfs=True):
for pkg in sorted(pkg_dic):
if bad_licenses:
try:
+ if incompatible_pkg_license(d, bad_licenses,
pkg_dic[pkg]["LICENSE"]):
+ bb.fatal("Package %s has an incompatible license
%s and cannot be installed into the image." %(pkg, pkg_dic[pkg]["LICENSE"]))

                     (pkg_dic[pkg]["LICENSE"], pkg_dic[pkg]["LICENSES"])

= \

oe.license.manifest_licenses(pkg_dic[pkg]["LICENSE"],
bad_licenses, canonical_license, d)

meta/lib/oeqa/selftest/cases/incompatible_lic.py +0 -0
index 8fb93af8a89..e80c9783122 100644
@@ -39,3 +39,32 @@ class IncompatibleLicenseTests(OESelftestTestCase):
     # INCOMPATIBLE_LICENSE contains this license
     def test_incompatible_nonspdx_license(self):
         self.lic_test('incompatible-nonspdx-license', 'FooLicense',

'FooLicense')

meta/lib/oeqa/selftest/cases/incompatible_lic.py +11 -0
+
+class IncompatibleLicensePerImageTests(OESelftestTestCase):
+    def default_config(self):
+        return """
+IMAGE_INSTALL_append = "bash"
+INCOMPATIBLE_LICENSE_pn-core-image-minimal = "GPL-3.0 LGPL-3.0"
+"""
+
+    def test_bash_default(self):
+        self.write_config(self.default_config())
+        error_msg = "ERROR: core-image-minimal-1.0-r0 do_rootfs: Package

bash has an incompatible license GPLv3+ and cannot be installed into the image."

meta/lib/oeqa/selftest/cases/incompatible_lic.py +7 -0
+
+        result = bitbake('core-image-minimal', ignore_status=True)
+        if error_msg not in result.output:
+            raise AssertionError(result.output)
+
+    def test_bash_and_license(self):
+        self.write_config(self.default_config() +

'\nLICENSE_append_pn-bash = " & SomeLicense"')
+ error_msg = "ERROR: core-image-minimal-1.0-r0 do_rootfs: Package
bash has an incompatible license GPLv3+ & SomeLicense and cannot be installed into the image."

meta/lib/oeqa/selftest/cases/incompatible_lic.py +7 -0
+
+        result = bitbake('core-image-minimal', ignore_status=True)
+        if error_msg not in result.output:
+            raise AssertionError(result.output)
+
+    def test_bash_or_license(self):
+        self.write_config(self.default_config() +

'\nLICENSE_append_pn-bash = " | SomeLicense"')

meta/lib/oeqa/selftest/cases/incompatible_lic.py +3 -1
+
+        bitbake('core-image-minimal')
+
--

2.17.1

Quoted 6 lines

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

--
Christopher Larson
kergoth at gmail dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Senior Software Engineer, Mentor Graphics

#4 lore raw

Re: [RFC PATCH 2/2] license_image.bbclass: check and reject packages which have incompatible licenses

Quoted 2 lines · Alexander Kanavin

It wouldn't be too hard to add a condition that checks the (image-specific) whitelist, I just wanted to gather a bit of feedback for the overall idea :)

I like the idea. I'm building images with e.g. GPLv3 forbidden but I enable building lots of GPLv3 components because they are needed in e.g. ptests. Resulting distro config snippet is large with lots of lines like:

WHITELIST_GPL-3.0 += "autoconf"
PACKAGE_EXCLUDE += "autoconf-dbg autoconf-staticdev autoconf-dev autoconf-doc autoconf-locale autoconf" WHITELIST_GPL-3.0 += "ccache"
PACKAGE_EXCLUDE += "ccache-sdktests-dbg ccache-sdktests ccache-dbg ccache-staticdev ccache-dev ccache-doc ccache-locale ccache"

In testing then I start with the pure product image without GPLv3 components and extend it with extra packages which are needed for the test execution. I have separate product feature and SW component test phases. Only in the latter case target SW is changed before test execution.

I want to avoid building separate test images with GPLv3, because in the past test vs. product images resulted in only the test images being actually tested.

But I can see that having separate product and test images from a single build could be useful.

Thanks for proposing this patch!

-Mikko