Add recipe whitelisting class
v2 cover Add recipe whitelisting class
Changes since the RFC version:
* Check BPN in addition to PN for matching, to make it easier in the multilib case
* Use PNWHITELIST_LAYERS to specify which layers to apply whitelisting to, and consider an empty whitelist value to be no recipes whitelisted (so you can set up empty whitelists for layers not yet in your configuration if you wish to do so). This also makes setting PNWHITELIST without an override practical.
* Touch up the comments
The following changes since commit f07045fcae859c902434062d1725f1348f42d1dd:
oeqa/oetest.py: add better package search for hasPackage() (2015-08-26 08:26:37 +0100)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib paule/whitelist
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paule/whitelist
Paul Eggleton (1):
classes/whitelist: add class to allow whitelisting recipes from a
layer
meta/classes/whitelist.bbclass | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) create mode 100644 meta/classes/whitelist.bbclass
Signature · 1 line
2.1.0
PATCH v2 1/1 classes/whitelist: add class to allow whitelisting recipes from a layer
Allow restricting recipes brought from a layer to a specified list. This is similar in operation to blacklist.bbclass, but instead specifies a per-layer whitelist of recipes (matched by PN or BPN) that are able to be built from the layer - anything else is skipped. This is potentially useful where you want to bring in a select set of recipes from a larger layer e.g. meta-oe.
Implements [YOCTO #8150].
--- meta/classes/whitelist.bbclass | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) create mode 100644 meta/classes/whitelist.bbclass
meta/classes/whitelist.bbclass new file +34 -0
@@ -0,0 +1,34 @@ +# Class that allows you to restrict the recipes brought from a layer to +# a specified list. This is similar in operation to blacklist.bbclass +# but note the difference in how PNWHITELIST is set - we don't use varflags +# here, the recipe name goes in the value and we use an override for the +# layer name (although this is not strictly required - you can have one +# PNWHITELIST value shared by all of the layers specified in +# PNWHITELIST_LAYERS). The layer name used here is actually the name that +# gets added to BBFILE_COLLECTIONS in the layer's layer.conf, which may +# differ from how the layer is otherwise known - e.g. meta-oe uses +# "openembedded-layer". +# +# INHERIT += "whitelist" +# PNWHITELIST_LAYERS = "layername" +# PNWHITELIST_layername = "recipe1 recipe2" +# +# If you would prefer to set a reason message other than the default, you +# can do so: +# +# PNWHITELIST_REASON_layername = "not supported by ${DISTRO}" + +python() { + layer = bb.utils.get_file_layer(d.getVar('FILE', True), d) + if layer: + layers = (d.getVar('PNWHITELIST_LAYERS', True) or '').split() + if layer in layers: + localdata = bb.data.createCopy(d) + localdata.setVar('OVERRIDES', layer) + whitelist = (localdata.getVar('PNWHITELIST', True) or '').split() + if not (d.getVar('PN', True) in whitelist or d.getVar('BPN', True) in whitelist): + reason = localdata.getVar('PNWHITELIST_REASON', True) + if not reason: + reason = 'not in PNWHITELIST for layer %s' % layer + raise bb.parse.SkipRecipe(reason) +}
Signature · 1 line
2.1.0
Re: [PATCH v2 1/1] classes/whitelist: add class to allow whitelisting recipes from a layer
Quoted 9 lines · Paul Eggleton
Allow restricting recipes brought from a layer to a specified list. This is similar in operation to blacklist.bbclass, but instead specifies a per-layer whitelist of recipes (matched by PN or BPN) that are able to be built from the layer - anything else is skipped. This is potentially useful where you want to bring in a select set of recipes from a larger layer e.g. meta-oe.
For the record, I am not in favor of such feature in OE, Distros can still use BBMASK if they wish to
Quoted 57 lines
Implements [YOCTO #8150].
Signed-off-by Paul Eggleton <[email protected]>---
meta/classes/whitelist.bbclass | 34 ++++++++++++++++++++++++++++++++++1 file changed, 34 insertions(+)create mode 100644 meta/classes/whitelist.bbclass
meta/classes/whitelist.bbclass new file +34
-1@@ -0,0 +1,34 @@ +# Class that allows you to restrict the recipes brought from a layer to +# a specified list. This is similar in operation to blacklist.bbclass +# but note the difference in how PNWHITELIST is set - we don't use varflags +# here, the recipe name goes in the value and we use an override for the +# layer name (although this is not strictly required - you can have one +# PNWHITELIST value shared by all of the layers specified in +# PNWHITELIST_LAYERS). The layer name used here is actually the name that +# gets added to BBFILE_COLLECTIONS in the layer's layer.conf, which may +# differ from how the layer is otherwise known - e.g. meta-oe uses +# "openembedded-layer". +# +# INHERIT += "whitelist" +# PNWHITELIST_LAYERS = "layername" +# PNWHITELIST_layername = "recipe1 recipe2" +# +# If you would prefer to set a reason message other than the default, you +# can do so: +# +# PNWHITELIST_REASON_layername = "not supported by ${DISTRO}" + +python() { + layer = bb.utils.get_file_layer(d.getVar('FILE', True), d) + if layer: + layers = (d.getVar('PNWHITELIST_LAYERS', True) or '').split() + if layer in layers: + localdata = bb.data.createCopy(d) + localdata.setVar('OVERRIDES', layer) + whitelist = (localdata.getVar('PNWHITELIST', True) or '').split() + if not (d.getVar('PN', True) in whitelist or d.getVar('BPN', True) in whitelist): + reason = localdata.getVar('PNWHITELIST_REASON', True) + if not reason: + reason = 'not in PNWHITELIST for layer %s' % layer + raise bb.parse.SkipRecipe(reason) +} --2.1.0
--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [PATCH v2 1/1] classes/whitelist: add class to allow whitelisting recipes from a layer
Quoted 10 lines · On Thursday 27 August 2015 23:07:44 Khem Raj wrote:
Quoted 9 lines
On Aug 27, 2015, at 5:36 AM, Paul Eggleton <[email protected]> wrote:
Allow restricting recipes brought from a layer to a specified list. This is similar in operation to blacklist.bbclass, but instead specifies a per-layer whitelist of recipes (matched by PN or BPN) that are able to be built from the layer - anything else is skipped. This is potentially useful where you want to bring in a select set of recipes from a larger layer e.g. meta-oe.
Quoted 2 lines
For the record, I am not in favor of such feature in OE, Distros can still use BBMASK if they wish to
This is significantly easier to use than BBMASK though. (Perhaps that's why you don't like it?)
I think we need to be realistic here as I said in my most recent reply to the other thread - people will do this using whatever mechanism is available whether we like it or not. And what are they saying by doing so? They're effectively telling us that they can't consume the recipes in the way we are currently collecting them together. We know what the proper fix for the problem is (increased splitting of large layers such as meta-oe), it may not be easy but other than workarounds like this I don't see an alternative.
Cheers,
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre
Re: [PATCH v2 1/1] classes/whitelist: add class to allow whitelisting recipes from a layer
Quoted 12 lines · On Friday 28 August 2015 09:34:18 Paul Eggleton wrote:
Quoted 11 lines · On Thursday 27 August 2015 23:07:44 Khem Raj wrote:
Quoted 10 lines
On Aug 27, 2015, at 5:36 AM, Paul Eggleton
<[email protected]>
wrote:Allow restricting recipes brought from a layer to a specified list. This is similar in operation to blacklist.bbclass, but instead specifies a per-layer whitelist of recipes (matched by PN or BPN) that are able to be built from the layer - anything else is skipped. This is potentially useful where you want to bring in a select set of recipes from a larger layer e.g. meta-oe.
Quoted 5 lines
Quoted 2 lines
For the record, I am not in favor of such feature in OE, Distros can still use BBMASK if they wish to
This is significantly easier to use than BBMASK though. (Perhaps that's why you don't like it?)
I should expand upon this with a couple of important distinctions:
* BBMASK is a list of files to mask out, so it's a blacklist not a whitelist. (Sure, you could probably construct a regex that effectively gives you a whitelist; it probably won't be pretty though.)
* BBMASK tells bitbake not to even try reading the masked files, which means if you then request to build any of them (either directly or via a dependency) you'll be told that there's nothing providing that with no explanation as to why. With whitelist.bbclass (and blacklist.bbclass, which whitelist.bbclass is modelled after) we are using SkipPackage/SkipRecipe so you at least get a statement that the recipe was skipped as well as hopefully an explanatory reason as to why.
Cheers,
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre
Re: [PATCH v2 1/1] classes/whitelist: add class to allow whitelisting recipes from a layer
In reply to Paul EggletonQuoted 14 lines · Paul Eggleton
Quoted 10 lines · On Thursday 27 August 2015 23:07:44 Khem Raj wrote:
Quoted 9 lines
On Aug 27, 2015, at 5:36 AM, Paul Eggleton <[email protected]> wrote:
Allow restricting recipes brought from a layer to a specified list. This is similar in operation to blacklist.bbclass, but instead specifies a per-layer whitelist of recipes (matched by PN or BPN) that are able to be built from the layer - anything else is skipped. This is potentially useful where you want to bring in a select set of recipes from a larger layer e.g. meta-oe.
Quoted 5 lines
Quoted 2 lines
For the record, I am not in favor of such feature in OE, Distros can still use BBMASK if they wish to
This is significantly easier to use than BBMASK though. (Perhaps that's why you don't like it?)
I don’t think its about technical merits of the patch. I understand its pretty well writter. Its if we want to provide more tools
and ways which can be bad for layer health in general which are besides oe-core.
Quoted 16 lines
I think we need to be realistic here as I said in my most recent reply to the other thread - people will do this using whatever mechanism is available whether we like it or not. And what are they saying by doing so? They're effectively telling us that they can't consume the recipes in the way we are currently collecting them together. We know what the proper fix for the problem is (increased splitting of large layers such as meta-oe), it may not be easy but other than workarounds like this I don't see an alternative.
Cheers,
Paul--
Paul Eggleton
Intel Open Source Technology Centre
Re: [PATCH v2 1/1] classes/whitelist: add class to allow whitelisting recipes from a layer
In reply to Khem RajQuoted 11 lines · Khem Raj
Quoted 9 lines · Paul Eggleton
Allow restricting recipes brought from a layer to a specified list. This is similar in operation to blacklist.bbclass, but instead specifies a per-layer whitelist of recipes (matched by PN or BPN) that are able to be built from the layer - anything else is skipped. This is potentially useful where you want to bring in a select set of recipes from a larger layer e.g. meta-oe.
For the record, I am not in favor of such feature in OE, Distros can still use BBMASK if they wish to
I am also not in favor of the merge of this.
Signature · 3 lines
Otavio Salvador O.S. Systems http://www.ossystems.com.br http://code.ossystems.com.br Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750
Re: [PATCH v2 1/1] classes/whitelist: add class to allow whitelisting recipes from a layer
Quoted 11 lines · On Friday 28 August 2015 10:55:41 Otavio Salvador wrote:
Quoted 10 lines · Khem Raj
Quoted 9 lines
On Aug 27, 2015, at 5:36 AM, Paul Eggleton
<[email protected]> wrote:Allow restricting recipes brought from a layer to a specified list. This is similar in operation to blacklist.bbclass, but instead specifies a per-layer whitelist of recipes (matched by PN or BPN) that are able to be built from the layer - anything else is skipped. This is potentially useful where you want to bring in a select set of recipes from a larger layer e.g. meta-oe.
Quoted 4 lines
Quoted 2 lines
For the record, I am not in favor of such feature in OE, Distros can still use BBMASK if they wish to
I am also not in favor of the merge of this.
Do you have a practical alternative that has less downsides?
Cheers,
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre
Re: [PATCH v2 1/1] classes/whitelist: add class to allow whitelisting recipes from a layer
Quoted 12 lines · Paul Eggleton
Quoted 11 lines · On Friday 28 August 2015 10:55:41 Otavio Salvador wrote:
Quoted 10 lines · Khem Raj
Quoted 9 lines
On Aug 27, 2015, at 5:36 AM, Paul Eggleton
<[email protected]> wrote:Allow restricting recipes brought from a layer to a specified list. This is similar in operation to blacklist.bbclass, but instead specifies a per-layer whitelist of recipes (matched by PN or BPN) that are able to be built from the layer - anything else is skipped. This is potentially useful where you want to bring in a select set of recipes from a larger layer e.g. meta-oe.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
For the record, I am not in favor of such feature in OE, Distros can still use BBMASK if they wish to
I am also not in favor of the merge of this.
Do you have a practical alternative that has less downsides?
I do want people using full layers and offer tools to make fancy combinations does not enhance our quality in long term.
Signature · 3 lines
Otavio Salvador O.S. Systems http://www.ossystems.com.br http://code.ossystems.com.br Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750
Re: [PATCH v2 1/1] classes/whitelist: add class to allow whitelisting recipes from a layer
Quoted 22 lines · On Friday 28 August 2015 11:09:19 Otavio Salvador wrote:
On Fri, Aug 28, 2015 at 11:00 AM, Paul Eggleton
Quoted 19 lines · <[email protected]> wrote:
Quoted 18 lines · On Friday 28 August 2015 10:55:41 Otavio Salvador wrote:
Quoted 16 lines · Khem Raj
Quoted 12 lines
On Aug 27, 2015, at 5:36 AM, Paul Eggleton
<[email protected]> wrote:Allow restricting recipes brought from a layer to a specified list. This
is similar in operation to blacklist.bbclass, but instead specifies a per-layer whitelist of recipes (matched by PN or BPN) that are able to be built from the layer - anything else is skipped. This is potentially
useful where you want to bring in a select set of recipes from a larger
layer e.g. meta-oe.For the record, I am not in favor of such feature in OE, Distros can still
use BBMASK if they wish to
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
I am also not in favor of the merge of this.
Do you have a practical alternative that has less downsides?
I do want people using full layers and offer tools to make fancy combinations does not enhance our quality in long term.
OK, sure - but you do appreciate that we do have an underlying problem here irrespective of whether this class gets taken into OE-Core or not, right?
Look, if people are really opposed to this then fine, I withdraw it. I just hope that it's understood that this class or other equivalent methods *will* get used until we fix the problem properly - all it means is the class will get copied around instead of being in one place.
Cheers,
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre
Re: [PATCH v2 1/1] classes/whitelist: add class to allow whitelisting recipes from a layer
Quoted 30 lines · Paul Eggleton
Quoted 22 lines · On Friday 28 August 2015 11:09:19 Otavio Salvador wrote:
On Fri, Aug 28, 2015 at 11:00 AM, Paul Eggleton
Quoted 19 lines · <[email protected]> wrote:
Quoted 18 lines · On Friday 28 August 2015 10:55:41 Otavio Salvador wrote:
Quoted 16 lines · Khem Raj
Quoted 12 lines
On Aug 27, 2015, at 5:36 AM, Paul Eggleton
<[email protected]> wrote:Allow restricting recipes brought from a layer to a specified list. This
is similar in operation to blacklist.bbclass, but instead specifies a per-layer whitelist of recipes (matched by PN or BPN) that are able to be built from the layer - anything else is skipped. This is potentially
useful where you want to bring in a select set of recipes from a larger
layer e.g. meta-oe.For the record, I am not in favor of such feature in OE, Distros can still
use BBMASK if they wish toQuoted 6 lines
Quoted 3 lines
Quoted 1 line
I am also not in favor of the merge of this.
Do you have a practical alternative that has less downsides?
I do want people using full layers and offer tools to make fancy combinations does not enhance our quality in long term.
Quoted 7 lines
OK, sure - but you do appreciate that we do have an underlying problem here irrespective of whether this class gets taken into OE-Core or not, right?
Look, if people are really opposed to this then fine, I withdraw it. I just hope that it's understood that this class or other equivalent methods *will* get used until we fix the problem properly - all it means is the class will get copied around instead of being in one place.
I don't think we (as OE) should help people to do bad things; people are good in inventing ways of doing bad things and we should try to provide them guidance to avoid those, not promote tools to help them in doing bad development practices.
OE-Core shouldn't handle and maintain this infrastructure, in my view.
Signature · 3 lines
Otavio Salvador O.S. Systems http://www.ossystems.com.br http://code.ossystems.com.br Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750
Quoted 9 lines · Paul Eggleton
Changes since the RFC version:
* Check BPN in addition to PN for matching, to make it easier in the multilib case
* UsePNWHITELIST_LAYERSto specify which layers to apply whitelisting to, and consider an empty whitelist value to be no recipes whitelisted (so you can set up empty whitelists for layers not yet in your configuration if you wish to do so). This also makes setting PNWHITELIST without an override practical.
* Touch up the comments
I've tried various configuration variations and it works fine now, so +1 from me.
Signature · 6 lines
Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.