LICENSE_FLAGS, a replacement for COMMERCIAL_LICENSE

From: Tom Zanussi <[email protected]>

This patchset implements a replacement for COMMERCIAL_LICENSE called LICENSE_FLAGS. Please see '[PATCH 1/5] base.bbclass: add support for LICENSE_FLAGS' for background and details.

This patchset was posted earlier in the week and received no comments. The only changes since then have been to save the cover letter text describing the patchset as commentary for PATCH 1, and changing instead of removing the warning in documentation-audit.sh.

In addition to the "Commercial" license replacement for the existing recipes in COMMERCIAL_LICENSE, this version has also been tested with a recipe that uses a "License_${PN}_${PV}" string, and multiple flags.

The following changes since commit 468998cddbe1a803096c9b357e1b5daa3b7e8c2e:
Dongxiao Xu (1):
command.py: add parseConfigurationFiles API

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib.git tzanussi/license-flags.v1
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=tzanussi/license-flags.v1

Tom Zanussi (5):
base.bbclass: add support for LICENSE_FLAGS
Add LICENSE_FLAGS to packages mentioned in COMMERCIAL_LICENSE base.bbclass: remove COMMERCIAL_LICENSE code default-distrovars.inc: remove COMMERCIAL_LICENSE et al documentation-audit.sh: remove COMMERCIAL_LICENSE warning

 meta/classes/base.bbclass                          |   24 +++++++++++++++-----
 meta/conf/distro/include/default-distrovars.inc    |    5 ----
 .../gstreamer/gst-fluendo-mp3_0.10.16.bb           |    1 +
 .../gstreamer/gst-openmax_0.10.1.bb                |    1 +
 .../gstreamer/gst-plugins-ugly_0.10.18.bb          |    1 +
 meta/recipes-multimedia/lame/lame_3.99.3.bb        |    1 +
 meta/recipes-multimedia/libmad/libmad_0.15.1b.bb   |    1 +
 meta/recipes-multimedia/libomxil/libomxil_0.9.3.bb |    1 +
 meta/recipes-multimedia/mpeg2dec/mpeg2dec_0.4.1.bb |    1 +
 meta/recipes-qt/qt-apps/qmmp_0.5.2.bb              |    1 +
 scripts/contrib/documentation-audit.sh             |    3 +-
 11 files changed, 28 insertions(+), 12 deletions(-)

From: Tom Zanussi <[email protected]>

LICENSE_FLAGS are a per-recipe replacement for COMMERCIAL_FLAGS. Any flags listed in a recipe's LICENSE_FLAGS variable must have a match in the global LICENSE_FLAGS_WHITELIST variable.

For some background on these changes, the original proposal for the functionality covered by this replacement was drafted by Saul Wold - the relevant details of that proposal are copied below:

***

There has been some issues raised with the initial implementation of COMMERCIAL_LICENSE and we are looking for ways to address this. Currently COMMERCIAL_LICENSE (C_L) is defined in default-distrovars.conf to contain a list of packages that have additional license requirements when used commercially (such as royalty requirements, or acknowledging some type of commercial T&Cs). These packages are skipped during parsing.

It currently contains a number of Audio and Video packages that require additional licensing terms when used commercially. As we add additional layers, some of these layers want to add additional package to the C_L list, but how to easily enable them.

Since local.conf, where you would normally override things like this, is read in before base.bbclass, which contains tools like oe_filter_out() to modify lists, we can't use that mechanism.

That's the background, now for the proposal.

Do away with C_L and C_*_PLUGINS, move to a "Named Bit Flag" list in LICENSE_FLAGS, each recipe can then maintain their flags directly, instead of in a shared location like default-distrovars.conf.

LICENSE_FLAGS_WHITELIST would be set in local.conf with the values that are acceptable to include in this image, by default it would be blank.

Possible values for LICENSE_FLAGS could be:
- Binary - provides some kind of binary with no source
- Patent - provides a potential infringing item, that some may not want
- Commercial - include recipes that may have commercial T&C
- Commercial_${PN} - commercial licenses specific to ${PN}
- License_${PN} - include a recipe that has a specific license
- maybe similar or different than Commercial_${PN}

***

[T&C = Terms and Conditions]

So Saul's draft describes the LICENSE_FLAGS themselves; the only thing missing is a description of how the user specifies which LICENSE_FLAGS are OK to use in an image. For that, the LICENSE_FLAGS_WHITELIST variable is introduced, which simply lists the LICENSE_FLAGS that are OK.

This version converts all the existing packages listed in COMMERCIAL_LICENSE to the equivalent "Commercial" LICENSE_FLAGS.

So to get the same functionality you'd currently get by setting COMMERCIAL_LICENSE = "", you'd add this to your local.conf:

LICENSE_FLAGS_WHITELIST = "Commercial"

Similarly, as another example, if there was a package that included only binaries with no source and which additionally required a specific license, it could specify LICENSE_FLAGS as:

LICENSE_FLAGS = "License_${PN}_${PV} Binary"

Assuming there was a 1.4 version of a package called foo, in this case for the foo_1.4.bb package to be built and included in the image, the expanded LICENSE_FLAGS for that particular version of the foo package would be added to the whitelist:

LICENSE_FLAGS_WHITELIST = "Commercial License_foo_1.4 Binary"

Similarly, if a single license could be assumed to cover any foo package regardless of version, the recipe could simply specify license flags as:

LICENSE_FLAGS = "License_${PN}"

etc...

Note that there's no policy attached to any of the above license types
- this is simply string-matching that can be used for the purpose of screening packages - if the strings match, the recipe gets in, if not, it doesn't i.e. during parsing, we would inspect the recipe's data for LICENSE_FLAGS and if it has a value then try to match against the WHITELIST - if it matches it gets added to the parsed list, if there is no match then it gets Skip_Package()'ed.

---
 meta/classes/base.bbclass |   19 +++++++++++++++++++
 1 files changed, 19 insertions(+), 0 deletions(-)

index e65a722..4aeba1b 100644
@@ -349,6 +349,25 @@ python () {
     if license == "INVALID":
         bb.fatal('This recipe does not have the LICENSE field set (%s)' % pn)
 
+    def skip_package(pn, flag):
+        bb.debug(1, "Skipping %s because it has a restricted license (%s) not"
+             " whitelisted in LICENSE_FLAGS_WHITELIST" % (pn, flag))
+        raise bb.parse.SkipPackage("because it may require a special license"
+            " to ship in a product (listed in LICENSE_FLAGS)")
+
+    def all_license_flags_match(flags, whitelist):
+        for flag in flags.split():
+            if not flag in whitelist.split():
+                return False
+        return True
+
+    license_flags = d.getVar('LICENSE_FLAGS', True)
+    if license_flags:
+        license_flags_whitelist = d.getVar('LICENSE_FLAGS_WHITELIST', True)
+        if not license_flags_whitelist or not all_license_flags_match(
+                license_flags, license_flags_whitelist):
+            skip_package(pn, license_flags)
+
     commercial_license = " %s " % d.getVar('COMMERCIAL_LICENSE', 1)
     import re
     pnr = "[ \t]%s[ \t]" % pn.replace('+', "\+")

1.7.0.4

From: Tom Zanussi <[email protected]>

Per-recipe LICENSE_FLAGS replace the global COMMERCIAL_LICENSE list; add LICENSE_FLAGS varables to each the recipes mentioned in that list:

- lame
- gst-fluendo-mp3
- gst-openmax
- gst-plugins-ugly
- libmad
- libomxil
- mpeg2dec
- qmmp

---
 .../gstreamer/gst-fluendo-mp3_0.10.16.bb           |    1 +
 .../gstreamer/gst-openmax_0.10.1.bb                |    1 +
 .../gstreamer/gst-plugins-ugly_0.10.18.bb          |    1 +
 meta/recipes-multimedia/lame/lame_3.99.3.bb        |    1 +
 meta/recipes-multimedia/libmad/libmad_0.15.1b.bb   |    1 +
 meta/recipes-multimedia/libomxil/libomxil_0.9.3.bb |    1 +
 meta/recipes-multimedia/mpeg2dec/mpeg2dec_0.4.1.bb |    1 +
 meta/recipes-qt/qt-apps/qmmp_0.5.2.bb              |    1 +
 8 files changed, 8 insertions(+), 0 deletions(-)

index 5975513..0a90f9c 100644
@@ -2,6 +2,7 @@ require gst-fluendo.inc
 
 LICENSE = "MIT"
 LIC_FILES_CHKSUM = "file://COPYING;md5=98326cbb1723a5a97e9b1db62e9faa05"
+LICENSE_FLAGS = "Commercial"
 
 acpaths = "-I ${S}/common/m4 -I ${S}/m4"
 

index 776ed1c..aeb016d 100644
@@ -1,6 +1,7 @@
 DEPENDS = "gstreamer"
 RDEPENDS_${PN} = "libomxil"
 LICENSE = "LGPLv2.1"
+LICENSE_FLAGS = "Commercial"
 LIC_FILES_CHKSUM = "file://COPYING;md5=fbc093901857fcd118f065f900982c24 \
                     file://util/sem.h;beginline=1;endline=20;md5=accce5550d5583b839b441a0623f09fc"
 

index 2d7fa91..8adee3e 100644
@@ -1,6 +1,7 @@
 require gst-plugins.inc
 
 LICENSE = "GPLv2+ & LGPLv2.1+ & LGPLv2+"
+LICENSE_FLAGS = "Commercial"
 LIC_FILES_CHKSUM = "file://COPYING;md5=a6f89e2100d9b6cdffcea4f398e37343 \
                     file://gst/synaesthesia/synaescope.h;beginline=1;endline=20;md5=99f301df7b80490c6ff8305fcc712838 \
                     file://tests/check/elements/xingmux.c;beginline=1;endline=21;md5=4c771b8af188724855cb99cadd390068 \

index 3c42cc7..aecd5f9 100644
@@ -3,6 +3,7 @@ HOMEPAGE = "http://sourceforge.net/projects/lame/files/lame/"
 BUGTRACKER = "http://sourceforge.net/tracker/?group_id=290&atid=100290"
 SECTION = "console/utils"
 LICENSE = "LGPLv2+"
+LICENSE_FLAGS = "Commercial"
 LIC_FILES_CHKSUM = "file://COPYING;md5=c46bda00ffbb0ba1dac22f8d087f54d9 \
                     file://include/lame.h;beginline=1;endline=20;md5=a2258182c593c398d15a48262130a92b
 PR = "r0"

index aec929c..5ab0f0c 100644
@@ -2,6 +2,7 @@ DESCRIPTION = "MPEG Audio Decoder Library"
 HOMEPAGE = "http://sourceforge.net/projects/mad/"
 BUGTRACKER = "http://sourceforge.net/tracker/?group_id=12349&atid=112349"
 LICENSE = "GPLv2+"
+LICENSE_FLAGS = "Commercial"
 LIC_FILES_CHKSUM = "file://COPYING;md5=94d55d512a9ba36caa9b7df079bae19f \
 			file://COPYRIGHT;md5=8e55eb14894e782b84488d5a239bc23d \
 			file://version.h;beginline=1;endline=8;md5=aa07311dd39288d4349f28e1de516454"

index bb31c56..9173a59 100644
@@ -1,6 +1,7 @@
 DESCRIPTION = "Bellagio OpenMAX Integration Layer"
 HOMEPAGE = "http://omxil.sourceforge.net/"
 LICENSE = "LGPLv2.1+"
+LICENSE_FLAGS = "Commercial"
 LIC_FILES_CHKSUM = "file://COPYING;md5=ae6f0f4dbc7ac193b50f323a6ae191cb \
                     file://src/omxcore.h;beginline=1;endline=27;md5=806b1e5566c06486fe8e42b461e03a90"
 DEPENDS = "libvorbis libogg alsa-lib libmad"

index 351962f..b2cf5f8 100644
@@ -2,6 +2,7 @@ DESCRIPTION = "Library and test program for decoding mpeg-2 and mpeg-1 video str
 HOMEPAGE = "http://libmpeg2.sourceforge.net/"
 SECTION = "libs"
 LICENSE = "GPLv2+"
+LICENSE_FLAGS = "Commercial"
 LIC_FILES_CHKSUM = "file://COPYING;md5=94d55d512a9ba36caa9b7df079bae19f \
                     file://include/mpeg2.h;beginline=1;endline=22;md5=ead62602d4638329d3b5b86a55803154"
 

index b3ecc45..a8d7fbd 100644
@@ -1,6 +1,7 @@
 DESCRIPTION = "Qmmp (Qt-based Multimedia Player) is an audio-player, written with help of Qt library"
 HOMEPAGE = "http://qmmp.ylsoftware.com"
 LICENSE = "GPLv2"
+LICENSE_FLAGS = "Commercial"
 LIC_FILES_CHKSUM = "file://COPYING;md5=393a5ca445f6965873eca0259a17f833"
 SECTION = "multimedia"
 

1.7.0.4

Hmm, I don't think it's retrograde - it's true, this patchset simply replaces the existing functionality, where those particular packages previously were all essentially marked "COMMERCIAL" by virtue of all existing within the one-and-all COMMERCIAL_LICENSE variable, whereas now they're all marked as "Commercial" instead.

But the new mechanism is much more flexible, since it now allows you to make the distinctions you're mentioning, which was impossible with the previous scheme. So yeah, the idea is that you can use a string as specific as you want for any give recipe e.g. have gst-fluendo-mp3 define LICENSE_FLAGS = "Commercial-gst-fluendo-mp3" and have something similar for libomixl, or you might decide that libomxl is just fine falling under the more general "Commercial" along with anything else that might fall in the same general category.

I would expect whoever knows more about those particular packages to actually submit changes for those if applicable. For now, this just replaces it all with a more flexible mechanism that allows you to do that, but doesn't make judgments about the particular packages, which I actually don't know much about wrt licensing.

I think the new mechanism provides the ability do whatever's necessary, but as far as it goes, and as mentioned in the commentary for patch 1, these are just strings and matches between two sets of them, LICENSE_FLAGS and LICENSE_FLAGS_WHITELIST. I'm afraid I don't know enough about the distinctions between the different license categories that might be needed. To me defining those falls under the heading of 'policy', and is something I'll have to leave for others. I'm simply providing the mechanism with this patch.

Thanks,

Tom

Well, the sense in which it seems retrograde to me is that, previously, COMMERCIAL_LICENSE named a list of packages and I could add or remove things as I saw fit depending on my distro policy requirements. Now, they're all just marked "Commercial" in an undifferentiated way and there doesn't seem to be any easy mechanism for me to take some but not all of them.

p.

Yeah, the global COMMERCIAL_LICENSE is convenient in that there's one place you can go to see a list of affected packages, but it's kind of at odds with allowing per-recipe flexibility for layers to add their own license terms.

To accomplish the same thing as COMMERCIAL_LICENSE with LICENSE_FLAGS, we could simply have each of the packages now listed in COMMERCIAL_LICENSE define LICENSE_FLAGS = "Commercial_${PN}".

The downside is that to enable only the ones you want, you'd have to know which are the ones you want in order to name them, which I guess you should anyway (rather then have them all in convenient list to remind you). For convenience and to sort of address that problem for the packages in oe-core, we could add a commented-out LICENSE_FLAGS_WHITELIST containing all the currently-listed packages in COMMERCIAL_LICENSE to the default local.conf. Users of other layers would have to know which additional packages to put in the whitelist, but again, shouldn't they be conscious of that anyway?

Tom

From: Tom Zanussi <[email protected]>

The COMMERCIAL_LICENSE mechanism has been superseded by LICENSE_FLAGS so remove the code that implements COMMERCIAL_LICENSE.

---
 meta/classes/base.bbclass |    7 -------
 1 files changed, 0 insertions(+), 7 deletions(-)

index 4aeba1b..8a15a8b 100644
@@ -368,13 +368,6 @@ python () {
                 license_flags, license_flags_whitelist):
             skip_package(pn, license_flags)
 
-    commercial_license = " %s " % d.getVar('COMMERCIAL_LICENSE', 1)
-    import re
-    pnr = "[ \t]%s[ \t]" % pn.replace('+', "\+")
-    if commercial_license and re.search(pnr, commercial_license):
-        bb.debug(1, "Skipping %s because it's commercially licensed" % pn)
-        raise bb.parse.SkipPackage("because it may require a commercial license to ship in a product (listed in COMMERCIAL_LICENSE)")
-
     # If we're building a target package we need to use fakeroot (pseudo)
     # in order to capture permissions, owners, groups and special files
     if not bb.data.inherits_class('native', d) and not bb.data.inherits_class('cross', d):

1.7.0.4

From: Tom Zanussi <[email protected]>

The global COMMERCIAL_LICENSE mechanism has been obsoleted by per-recipe LICENSE_FLAGS, so remove the related variables.

---
 meta/conf/distro/include/default-distrovars.inc |    5 -----
 1 files changed, 0 insertions(+), 5 deletions(-)

index e1594f3..16b3108 100644
@@ -26,11 +26,6 @@ HOSTTOOLS_WHITELIST_GPLv3 ?= ""
 WHITELIST_GPLv3 ?= "less"
 LGPLv2_WHITELIST_GPLv3 ?= "libassuan gnutls libtasn1 libidn libgcc gcc-runtime" 
 
-# This is a list of packages that require a commercial license to ship
-# product. If shipped as part of an image these packages may have 
-# implications so they are disabled by default
-COMMERCIAL_LICENSE ?= "lame gst-fluendo-mp3 libmad mpeg2dec ffmpeg qmmp ${COMMERCIAL_LICENSE_DEPENDEES}"
-COMMERCIAL_LICENSE_DEPENDEES ?= "gst-plugins-ugly libomxil gst-openmax"
 COMMERCIAL_AUDIO_PLUGINS ?= ""
 # COMMERCIAL_AUDIO_PLUGINS ?= "gst-plugins-ugly-mad gst-plugins-ugly-mpegaudioparse"
 COMMERCIAL_VIDEO_PLUGINS ?= ""

1.7.0.4

From: Tom Zanussi <[email protected]>

COMMERCIAL_LICENSE no longer exists; the equivalent functionality is now has been replaced by LICENSE_FLAGS_WHITELIST, so replace the COMMERCIAL_LICENSE warning with a similarly equivalent warning.

---
 scripts/contrib/documentation-audit.sh |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

index 5070fee..5b66f03 100755
@@ -25,7 +25,8 @@ if [ -z "$BITBAKE" ]; then
 fi
 
 echo "REMINDER: you need to build for MACHINE=qemux86 or you won't get useful results"
-echo "REMINDER: you need to have COMMERCIAL_LICENSE = \"\" in local.conf or you'll get false positives"
+echo "REMINDER: you need to set LICENSE_FLAGS_WHITELIST appropriately in local.conf or "
+echo " you'll get false positives.  For example, LICENSE_FLAGS_WHITELIST = \"Commercial\""
 
 for pkg in `bitbake -s | awk '{ print \$1 }'`; do
 	if [[ "$pkg" == "Loading" || "$pkg" == "Loaded" ||

1.7.0.4