RFC: Package exclusion
Hi all,
As part of the 1.1 feature list I suggested a review of the INCOMPATIBLE_LICENSE and COMMERCIAL* package exclusion mechanisms we have within Poky. Below I've outlined my ideas and would appreciate any comments/additions/corrections.
==== Aims ====
* Make error messages clear when user/dependencies have asked for something
to be built that can't be due to restrictions
* Ensure that exclusion system is reliable
==== Proposed implementation ====
1) Ensure all documentation of LICENSE field value syntax is clear, concise
and up-to-date (wiki and manual)
2) Go through and audit all recipes LICENSE field values to ensure that they
all conform to the specifications. This includes making sure that | (package may be used under one of a selection of licences) and & (recipe has mixed licences that apply to the code base, so conditions of all must be observed) are used correctly.
3) bitbake/core changes:
* LICENSE field checking must fully parse the field and understand the
difference between | and &, and must not e.g. mark Qt as being GPLv3 only.
* Make the LICENSE validity checking more strict (given recipes have been
audited and rules are clear after above)
* Don't exclude any recipes at parse time - simply record all excluded
recipes and their runtime provides in a blacklist which also includes flags indicating the reason for blacklisting
* Ensure all excluded licences in INCOMPATIBLE_LICENSE are valid (e.g. catch
GPL3 as apposed to GPLv3) - if not, error out
* Check when calculating dependencies if anything is scheduled to be built
that is on the blacklist - if any are, gather all of them up and then stop and list them in an error message along with reason and depchain for each one
* Check when constructing the rootfs if anything in the runtime provides
blacklist is going to be included - if so, error out
Some further possible extensions:
* Possibly apply similar logic to COMPATIBLE_MACHINE?
* Replace COMMERCIAL* with some more generic exclusion mechanism that allows
the reason to be defined as part of the exclusion list?
* As a helper for non-en_US users, fail on parse if user defines any of the
*LICENSE* variables as *LICENCE*? (we definitely don't want the build to continue and just ignore this as the user might not realise what has happened)
Cheers,
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre
Quoted 40 lines · Paul Eggleton
Hi all,
As part of the 1.1 feature list I suggested a review of the
INCOMPATIBLE_LICENSEand COMMERCIAL* package exclusion mechanisms we have within Poky. Below I've outlined my ideas and would appreciate any comments/additions/corrections.==== Aims ====
* Make error messages clear when user/dependencies have asked for something
to be built that can't be due to restrictions* Ensure that exclusion system is reliable
==== Proposed implementation ====
1) Ensure all documentation of LICENSE field value syntax is clear, concise
and up-to-date (wiki and manual)2) Go through and audit all recipes LICENSE field values to ensure that they
all conform to the specifications. This includes making sure that | (package may be used under one of a selection of licences) and& (recipe has mixed licences that apply to the code base, so conditions of all must be observed) are used correctly.3) bitbake/core changes:
* LICENSE field checking must fully parse the field and understand the
difference between | and&, and must not e.g. mark Qt as being GPLv3 only.* Make the LICENSE validity checking more strict (given recipes have been
audited and rules are clear after above)* Don't exclude any recipes at parse time - simply record all excluded
recipes and their runtime provides in a blacklist which also includes flags indicating the reason for blacklisting* Ensure all excluded licences in
INCOMPATIBLE_LICENSEare valid (e.g. catch
GPL3 as apposed to GPLv3) - if not, error out
Quoted 7 lines
* Check when calculating dependencies if anything is scheduled to be built
that is on the blacklist - if any are, gather all of them up and then stop and list them in an error message along with reason and depchain for each one* Check when constructing the rootfs if anything in the runtime provides
blacklist is going to be included - if so, error out
Most of the issues surrounding the license field is currently on my plate for M1.D
License tracking Build a parser to do license tracking more gracefully and make sure all recipes are correct. (takes ~2 weeks) 1 Accept Beth Beth M1, Sprint D
There is currently no specification on the LICENSE field values and how they should be parsed. I'm going to be starting that in the next week or so. We should sync up with each other if this work is going to overlap. Schedule some time for this week?
-b
Quoted 3 lines
Cheers,
Paul
Signature · 4 lines
--------------- Elizabeth Flanagan Yocto Project Release Engineer
Hi Beth,
Quoted 5 lines · On Wednesday 04 May 2011 00:08:59 Elizabeth Flanagan wrote:
Most of the issues surrounding the license field is currently on my plate for M1.D
License tracking Build a parser to do license tracking more gracefully and
make sure all recipes are correct. (takes ~2 weeks) 1 Accept Beth Beth
M1,
Quoted 6 lines
Sprint D
There is currently no specification on the LICENSE field values and how they should be parsed. I'm going to be starting that in the next week or so. We should sync up with each other if this work is going to overlap. Schedule some time for this week?
Yes there's definitely quite an overlap here and we should coordinate efforts. Will catch you later on IRC to organise some time.
Cheers,
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre
Apologies for the tardy reply. I like where you're going, usually I have to disagree to reply swiftly. Sorry.
Quoted 7 lines · Paul Eggleton
Hi all,
As part of the 1.1 feature list I suggested a review of the
INCOMPATIBLE_LICENSEand COMMERCIAL* package exclusion mechanisms we have within Poky. Below I've outlined my ideas and would appreciate any comments/additions/corrections.
Quoted 6 lines
==== Aims ====
* Make error messages clear when user/dependencies have asked for something
to be built that can't be due to restrictions* Ensure that exclusion system is reliable
Solid.
Quoted 24 lines
==== Proposed implementation ====
1) Ensure all documentation of LICENSE field value syntax is clear, concise
and up-to-date (wiki and manual)2) Go through and audit all recipes LICENSE field values to ensure that they
all conform to the specifications. This includes making sure that | (package may be used under one of a selection of licences) and & (recipe has mixed licences that apply to the code base, so conditions of all must be observed) are used correctly.3) bitbake/core changes:
* LICENSE field checking must fully parse the field and understand the
difference between | and &, and must not e.g. mark Qt as being GPLv3 only.* Make the LICENSE validity checking more strict (given recipes have been
audited and rules are clear after above)* Don't exclude any recipes at parse time - simply record all excluded
recipes and their runtime provides in a blacklist which also includes flags indicating the reason for blacklisting
Quoted 6 lines
* Ensure all excluded licences in
INCOMPATIBLE_LICENSEare valid (e.g. catch
GPL3 as apposed to GPLv3) - if not, error out* Check when calculating dependencies if anything is scheduled to be built
that is on the blacklist - if any are, gather all of them up and then stop and list them in an error message along with reason and depchain for each one
This sounds like overlap with a task Scott Garman is set to work on (Error handling in bitbake), let's make sure you guys are collaborating here.
http://bugzilla.yoctoproject.org/show_bug.cgi?id=542
Quoted 11 lines
* Check when constructing the rootfs if anything in the runtime provides
blacklist is going to be included - if so, error outSome further possible extensions:
* Possibly apply similar logic to
COMPATIBLE_MACHINE?* Replace COMMERCIAL* with some more generic exclusion mechanism that allows
the reason to be defined as part of the exclusion list?
Sounds reasonable, care to elaborate? Something like:
COMMERCIAL_LICENSE = "naughty packages from copyright hell" PERL_PACKAGES = "packages I don't want to include because I dislike Perl"EXCLUDED_PATTERNS = "PERL_PACKAGES COMMERCIAL_LICENSE"
??
Quoted 4 lines
* As a helper for non-en_US users, fail on parse if user defines any of the
*LICENSE* variables as *LICENCE*? (we definitely don't want the build to continue and just ignore this as the user might not realise what has happened)
Darn tootin!
Looks like a good start, thanks Paul!
Cheers,
Joshua
Signature · 3 lines
Joshua Lock
Yocto Build System Monkey
Intel Open Source Technology CentreQuoted 9 lines · On Wednesday 11 May 2011 00:20:14 Joshua Lock wrote:
Quoted 4 lines
* Check when calculating dependencies if anything is scheduled to be built that is on the blacklist - if any are, gather all of them up and then stop and list them in an error message along with reason and depchain for each one
This sounds like overlap with a task Scott Garman is set to work on (Error handling in bitbake), let's make sure you guys are collaborating here.
http://bugzilla.yoctoproject.org/show_bug.cgi?id=542
I hadn't considered this - you may well be right. It depends on how Scott proposes to attack that particular problem, I will check with him.
Quoted 9 lines
Quoted 2 lines
* Replace COMMERCIAL* with some more generic exclusion mechanism that allows the reason to be defined as part of the exclusion list?
Sounds reasonable, care to elaborate? Something like:
COMMERCIAL_LICENSE= "naughty packages from copyright hell"PERL_PACKAGES= "packages I don't want to include because I dislike Perl"EXCLUDED_PATTERNS= "PERL_PACKAGESCOMMERCIAL_LICENSE"
More like:
EXCLUDED_PACKAGES += "broken1,broken2:These packages are broken by design and not supported;"EXCLUDED_PACKAGES += "licenserequired,clickthru,special1:These packages require a special license agreement;"
The example syntax is horrible but you get the idea.
Quoted 6 lines
Quoted 4 lines
* As a helper for non-en_US users, fail on parse if user defines any of the *LICENSE* variables as *LICENCE*? (we definitely don't want the build to continue and just ignore this as the user might not realise what has
happened)Darn tootin!
I think you meant to say "hear hear!" or maybe even "good show!"... ;)
Cheers,
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre
Quoted 6 lines · Paul Eggleton
Quoted 5 lines · On Wednesday 11 May 2011 00:20:14 Joshua Lock wrote:
Quoted 4 lines
* Check when calculating dependencies if anything is scheduled to be built that is on the blacklist - if any are, gather all of them up and then stop and list them in an error message along with reason and depchain for each one
Quoted 7 lines
Quoted 4 lines
This sounds like overlap with a task Scott Garman is set to work on (Error handling in bitbake), let's make sure you guys are collaborating here.
http://bugzilla.yoctoproject.org/show_bug.cgi?id=542I hadn't considered this - you may well be right. It depends on how Scott proposes to attack that particular problem, I will check with him.
Cool. If they don't overlap I'd be surprised.
Quoted 11 lines
Quoted 10 lines
Quoted 3 lines
* Replace COMMERCIAL* with some more generic exclusion mechanism that allows the reason to be defined as part of the exclusion list?
Sounds reasonable, care to elaborate? Something like:
COMMERCIAL_LICENSE= "naughty packages from copyright hell"PERL_PACKAGES= "packages I don't want to include because I dislike Perl"EXCLUDED_PATTERNS= "PERL_PACKAGESCOMMERCIAL_LICENSE"
Quoted 8 lines
More like:
EXCLUDED_PACKAGES+= "broken1,broken2:These packages are broken by design and not supported;"EXCLUDED_PACKAGES+= "licenserequired,clickthru,special1:These packages require a special license agreement;"The example syntax is horrible but you get the idea.
The idea is sound, yeah.
Quoted 10 lines
Quoted 7 lines
Quoted 5 lines
* As a helper for non-en_US users, fail on parse if user defines any of the *LICENSE* variables as *LICENCE*? (we definitely don't want the build to continue and just ignore this as the user might not realise what has
happened)Darn tootin!
I think you meant to say "hear hear!" or maybe even "good show!"... ;)
Bravo!
Joshua
Signature · 3 lines
Joshua Lock
Yocto Build System Monkey
Intel Open Source Technology Centre