Skip to contentSkip to conversation

yocto beagleboard.conf -- should it not go away?

DISCUSSION Tomas Frydrych 10 participants 29 messages Sep 3, 2012 – Sep 8, 2012

lore
Share & tools
lore
#0 lore raw

Hi,

I am wondering why yocto includes a beagleboard.conf when it is unable to support much of the board's features, and there is a dedicated layer for TI stuff that defines the beagleboard machine properly.

I actually have a fairly practical gripe here: it is currently not possible to include both meta-yocto and meta-ti, and have it use the correct beagleboard.conf from meta-ti! The meta-yocto layer prepends itself to the BBPATH while meta-ti appends itself, so regarless the layer arrangement, the (not-much-useful) yocto beagleboard.conf is always used, and everything is broken. The only way to work around this that I can think of is to provide yet another beagleboard.conf from a custom layer that also preppends itself to the BBPATH, which feels like three sides of a square route.

So, is there a good reason not to get rid of yocto beagleboard.conf?

Tomas

#1 lore raw
Quoted 7 lines · Tomas Frydrych

Hi,

I am wondering why yocto includes a beagleboard.conf when it is unable to support much of the board's features, and there is a dedicated layer for TI stuff that defines the beagleboard machine properly.

Quoted 9 lines

I actually have a fairly practical gripe here: it is currently not possible to include both meta-yocto and meta-ti, and have it use the correct beagleboard.conf from meta-ti! The meta-yocto layer prepends itself to the BBPATH while meta-ti appends itself, so regarless the layer arrangement, the (not-much-useful) yocto beagleboard.conf is always used, and everything is broken. The only way to work around this that I can think of is to provide yet another beagleboard.conf from a custom layer that also preppends itself to the BBPATH, which feels like three sides of a square route.

So we fix the configuration of the layers .. and meta-ti was supposed to work properly with meta-yocto last I heard.

Quoted 2 lines

So, is there a good reason not to get rid of yocto beagleboard.conf?

The meta-yocto layer beagleboard configuration is a hardware ARM reference, that uses the linux-yocto kernel (policy and version) and is used for the project QA on
hardware.

The meta-ti layer, which no one argues has more hardware functionality and support, doesn't hit those other three elements.

The beagleboard in meta-yocto will be bumped shortly to match the 3.4 configs and policy of the rest of the reference boards.

If I make the statement that we need a ARM hardware reference, that uses only meta-yocto (linux-yocto + config + tools), the choices are limited. Any board that
I select, is followed by an series of comments about how other layers, other places,
other <something> has that board. So there's no right choice, including dropping it completely, since the requirements that I've listed above are equally not met by
any other layer configurations.

Cheers,

Bruce

Quoted 6 lines

Tomas
_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto

Signature · 2 lines
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"
#2 lore raw

Hi,

Quoted 2 lines · Bruce Ashfield

On Mon, Sep 3, 2012 at 4:06 PM, Tomas Frydrych
So we fix the configuration of the layers ..

I expect this is not actually that trivial. I think the only way to do this is properly is for the layer priorities to be respected by all files in the layer, not just the bb files. No idea how involved that change would be, or what the side effects would be.

Quoted 1 line

and meta-ti was supposed to work properly with meta-yocto last I heard.

If properly means 'out of the box', then no; the two layers can be made to work together, but it's fairly clear that nobody comprehensively tests that combination at the moment.

Quoted 3 lines

The meta-yocto layer beagleboard configuration is a hardware ARM reference, that uses the linux-yocto kernel (policy and version) and is used for the project QA on hardware.

I am all for QA, but in this instance your QA procedure breaks things for the end user. Could you not just call it something else, e.g., yocto-qa-arm-reference.conf?

Tomas

#3 lore raw
Quoted 10 lines · Tomas Frydrych

Hi,

Quoted 2 lines · Bruce Ashfield

On Mon, Sep 3, 2012 at 4:06 PM, Tomas Frydrych
So we fix the configuration of the layers ..

I expect this is not actually that trivial. I think the only way to do this is properly is for the layer priorities to be respected by all files in the layer, not just the bb files. No idea how involved that change would be, or what the side effects would be.

This is a very common layering scenario, so if the system as a whole can't handle it, that's a bigger problem. If two layers can't have a board with the same name, and there's no way for them to work together without tweaking multiple files in the layer, I'd say that's a problem and one that should be fixed. Otherwise, a OSV (or someone else) can never hope to extend anything from a yocto layer.

That being said, taking a step back, what are you trying to get out of meta-yocto in this scenario ? oe-core + meta-ti is probably what you should be using.

Quoted 7 lines
Quoted 3 lines

and meta-ti was supposed to work properly with meta-yocto last I heard.

If properly means 'out of the box', then no; the two layers can be made to work together, but it's fairly clear that nobody comprehensively tests that combination at the moment.

see above. I misspoke. I don't think there's an intent to make meta-yocto and meta-ti work together, but oe-core + meta-ti, that's the combo that makes sense.

Quoted 9 lines
Quoted 5 lines

The meta-yocto layer beagleboard configuration is a hardware ARM reference, that uses the linux-yocto kernel (policy and version) and is used for the project QA on hardware.

I am all for QA, but in this instance your QA procedure breaks things for the end user. Could you not just call it something else, e.g., yocto-qa-arm-reference.conf?

Hmmm. I'd say no to that, because the system should handle something like this, and dancing around the real problem with varying names of BSPs outside of the name of the board they actually support, isn't a solution.

Cheers,

Bruce

Quoted 6 lines

Tomas
_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto

Signature · 2 lines
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"
#4 lore raw
Quoted 10 lines · Bruce Ashfield
Quoted 8 lines · Tomas Frydrych

snip...

That being said, taking a step back, what are you trying to get out of meta-yocto in this scenario ? oe-core + meta-ti is probably what you should be using.

Sorry to jump in here, but this is actually a very interesting point. From coming to OpenEmbedded through Yocto and subsequently learning about the ecosystem as a whole, I have always considered meta-yocto to be providing a sane distro configuration. This implies that I believe the oe-core default distro configuration, even though it will build, the result won't be very satisfying - or maybe a bit disjointed.

You seem to be saying above that the meta-yocto layer is only any good for providing the kernel framework, and as he is not using the framework (i.e. using meta-ti) then there is no point in using the meta-yocto layer? So does the Poky distro config only really slap a name on the oe-core default distro policy - or does it have some added value?

I'm not trying to rile anyone up here, just trying to put the jigsaw together as the picture looks different every time I revisit it!!

Regards,

Signature · 5 lines
  Jack Mitchell ([email protected])
  Embedded Systems Engineer
  http://www.embed.me.uk

--
#5 lore raw
Quoted 15 lines · Jack Mitchell
Quoted 9 lines · Bruce Ashfield
Quoted 6 lines · Tomas Frydrych

snip...

Quoted 4 lines
Quoted 4 lines

That being said, taking a step back, what are you trying to get out of meta-yocto in this scenario ? oe-core + meta-ti is probably what you should be using.

Quoted 6 lines

Sorry to jump in here, but this is actually a very interesting point. From coming to OpenEmbedded through Yocto and subsequently learning about the ecosystem as a whole, I have always considered meta-yocto to be providing a sane distro configuration. This implies that I believe the oe-core default distro configuration, even though it will build, the result won't be very satisfying - or maybe a bit disjointed.

No worries. Take everything that I say as observations from a arms length distance to how things are maintained and the plans about compatibility, evolution, etc. So what I say, may not reflect reality :)

Quoted 6 lines

You seem to be saying above that the meta-yocto layer is only any good for providing the kernel framework, and as he is not using the framework (i.e. using meta-ti) then there is no point in using the meta-yocto layer? So does the Poky distro config only really slap a name on the oe-core default distro policy - or does it have some added value?

There's value, and Richard has covered it a few times on the mailing list. I probably wouldn't do it justice, so I'll wait to see if others with a better articulated description jump in.

The linux-yocto kernel itself can work without any build system at all, so my point of view is different :)

Cheers,

Bruce

Quoted 19 lines

I'm not trying to rile anyone up here, just trying to put the jigsaw together as the picture looks different every time I revisit it!!

Regards,

--

Jack Mitchell ([email protected])
Embedded Systems Engineer
http://www.embed.me.uk

--

_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto

Signature · 2 lines
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"
#6 lore raw
In reply to Jack Mitchell
Quoted 19 lines · Jack Mitchell
Quoted 10 lines · Bruce Ashfield
Quoted 8 lines · Tomas Frydrych

snip...

That being said, taking a step back, what are you trying to get out of meta-yocto in this scenario ? oe-core + meta-ti is probably what you should be using.

Sorry to jump in here, but this is actually a very interesting point.
From coming to OpenEmbedded through Yocto and subsequently learning
about the ecosystem as a whole, I have always considered meta-yocto to be providing a sane distro configuration. This implies that I believe the oe-core default distro configuration, even though it will build, the result won't be very satisfying - or maybe a bit disjointed.

Quoted 5 lines

You seem to be saying above that the meta-yocto layer is only any good for providing the kernel framework, and as he is not using the framework (i.e. using meta-ti) then there is no point in using the meta-yocto layer? So does the Poky distro config only really slap a name on the oe-core default distro policy - or does it have some added value?

In the old days, there was no default distro config. OE-Core changed that as an experiment which IMO has been relatively successful. The Poky distro config inherits as much as possible from the core defaults. The extras are (reading poky.conf):

* Example of Distro Branding
* Ensuring some particular kernel versions
* Particular SDK layout
* Minor tweaks to basic images/qemu images
* Mirror Setup (although OE-Core has these now too iirc)
* Sets connectivity check
* Sets list of compatible/tested distros
* Some signature stuff (now OE-Core default)
* Sets more stringent package sanity checks that OE-Core

So at least some of the above moved into the core as default although we still need to remove the remnants from the poky.conf file. Others will likely move in future too.

There isn't anything major above but there are several smaller usability tweaks/optimisations and I'd argue there is some added value.

There are also example LSB and "tiny" profiles showing how to take the system and customise it in different ways which I'd argue have value too.

Quoted 2 lines

I'm not trying to rile anyone up here, just trying to put the jigsaw together as the picture looks different every time I revisit it!!

Its an evolving picture, hopefully in a good way :)

Cheers,

Richard

#7 lore raw
In reply to Bruce Ashfield

Hi Bruce,

Quoted 2 lines · Bruce Ashfield

That being said, taking a step back, what are you trying to get out of meta-yocto in this scenario ?

a) I am targeting multiple chips, including TI Omap and Intel Atom. meta-yocto is a prerequisite for the various machines in meta-intel, so I have to include meta-yocto if I want to build images for an Intel chip. Nothing unusual here.

b) meta-yocto is the Poky distro layer; if you want to use Poky, then you need meta-yocto.

Quoted 3 lines

see above. I misspoke. I don't think there's an intent to make meta-yocto and meta-ti work together, but oe-core + meta-ti, that's the combo that makes sense.

See (b) above; you are not saying that Poky is only meant for Intel HW, are you?

The basic problem with meta-yocto is that it combines BSP stuff (meta-intel prerequisite, Atom & Beagle config) with distro stuff (Poky, Yocto branding). That's convenient for doing QA on a limited set of HW, but suboptimal for real use; BSP layers simply should not be dependent on distro layers, it largely defeats the purpose of having layers.

Splitting out the minimal beagle config into a layer of its own would improve things quite a bit.

Tomas

#8 lore raw
Quoted 14 lines · Tomas Frydrych

Hi Bruce,

Quoted 2 lines · Bruce Ashfield

That being said, taking a step back, what are you trying to get out of meta-yocto in this scenario ?

a) I am targeting multiple chips, including TI Omap and Intel Atom. meta-yocto is a prerequisite for the various machines in meta-intel, so I have to include meta-yocto if I want to build images for an Intel chip. Nothing unusual here.

b) meta-yocto is the Poky distro layer; if you want to use Poky, then you need meta-yocto.

Quoted 6 lines
Quoted 3 lines

see above. I misspoke. I don't think there's an intent to make meta-yocto and meta-ti work together, but oe-core + meta-ti, that's the combo that makes sense.

See (b) above; you are not saying that Poky is only meant for Intel HW, are you?

Definitely not. But I'm not familiar with anyone's (outside of the weekly meetings
that I attend) plans for meta-yocto, so I'm commenting on what I see. You'd need someone from TI to comment on their meta-yocto compatibility. And that person definitely is not me :)

Quoted 9 lines

The basic problem with meta-yocto is that it combines BSP stuff (meta-intel prerequisite, Atom & Beagle config) with distro stuff (Poky, Yocto branding). That's convenient for doing QA on a limited set of HW, but suboptimal for real use; BSP layers simply should not be dependent on distro layers, it largely defeats the purpose of having layers.

Splitting out the minimal beagle config into a layer of its own would improve things quite a bit.

All of the above are things that Richard would need to comment on, the yocto layering and maintenance is not something that I have control over :)

Cheers,

Bruce

Quoted 6 lines

Tomas
_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto

Signature · 2 lines
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"
#9 lore raw
Quoted 36 lines · Bruce Ashfield
Quoted 14 lines · Tomas Frydrych

Hi Bruce,

Quoted 2 lines · Bruce Ashfield

That being said, taking a step back, what are you trying to get out of meta-yocto in this scenario ?

a) I am targeting multiple chips, including TI Omap and Intel Atom. meta-yocto is a prerequisite for the various machines in meta-intel, so I have to include meta-yocto if I want to build images for an Intel chip. Nothing unusual here.

b) meta-yocto is the Poky distro layer; if you want to use Poky, then you need meta-yocto.

Quoted 6 lines
Quoted 3 lines

see above. I misspoke. I don't think there's an intent to make meta-yocto and meta-ti work together, but oe-core + meta-ti, that's the combo that makes sense.

See (b) above; you are not saying that Poky is only meant for Intel HW, are you?

Definitely not. But I'm not familiar with anyone's (outside of the weekly meetings
that I attend) plans for meta-yocto, so I'm commenting on what I see. You'd need someone from TI to comment on their meta-yocto compatibility. And that person definitely is not me :)

Quoted 7 lines

The basic problem with meta-yocto is that it combines BSP stuff (meta-intel prerequisite, Atom & Beagle config) with distro stuff (Poky, Yocto branding). That's convenient for doing QA on a limited set of HW, but suboptimal for real use; BSP layers simply should not be dependent on distro layers, it largely defeats the purpose of having layers.

Quoted 6 lines
Quoted 2 lines

Splitting out the minimal beagle config into a layer of its own would improve things quite a bit.

All of the above are things that Richard would need to comment on, the yocto layering and maintenance is not something that I have control over :)

As far as I understand it, meta-yocto and meta-ti should be able to work together. If they don't, it's a bug in one or the other or both and we should work to address it.

The machine namespace collision could be an example of something we need to fix.

--
Darren

Signature · 3 lines
Darren Hart
Intel Open Source Technology Center
Yocto Project - Technical Lead - Linux Kernel
#10 lore raw
Quoted 16 lines · Darren Hart
Quoted 13 lines · Bruce Ashfield
Quoted 11 lines · Tomas Frydrych

Hi Bruce,

Quoted 2 lines · Bruce Ashfield

That being said, taking a step back, what are you trying to get out of meta-yocto in this scenario ?

a) I am targeting multiple chips, including TI Omap and Intel Atom. meta-yocto is a prerequisite for the various machines in meta-intel, so I have to include meta-yocto if I want to build images for an Intel chip. Nothing unusual here.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

b) meta-yocto is the Poky distro layer; if you want to use Poky, then you need meta-yocto.

Quoted 3 lines

see above. I misspoke. I don't think there's an intent to make meta-yocto and meta-ti work together, but oe-core + meta-ti, that's the combo that makes sense.

oe-core + meta-ti should work or it needs to get fixed. poky + meta-ti should work or it needs to get fixed.

However I suspect the 2nd is not in the nightly builds yet.

Denys is out for the next few days. He can comment more when he gets back.

It has been our assumption that there is enough functionality in the layer mechanisms that any of the "light weight" BSPs in yocto layer could be completely overridden by a more complete layer (meta-ti in this example). In addition the end system integrator should be able to override definitions in any BSP layer.

I suspect the current issue is just growing pains for a case that has not been tested. Lets prove that false before taking more drastic action.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

The basic problem with meta-yocto is that it combines BSP stuff (meta-intel prerequisite, Atom& Beagle config) with distro stuff (Poky, Yocto branding). That's convenient for doing QA on a limited set of HW, but suboptimal for real use; BSP layers simply should not be dependent on distro layers, it largely defeats the purpose of having layers.

Darren: Is it true you can't get @ the Intel BSP's w/o also getting the poky distro defs? That does seem to mixing things a bit. (I am not claiming meta-ti is clean yet but I want to understand the Intel examples.)

#11 lore raw
Quoted 4 lines · William Mills

Darren: Is it true you can't get @ the Intel BSP's w/o also getting the poky distro defs? That does seem to mixing things a bit. (I am not claiming meta-ti is clean yet but I want to understand the Intel examples.)

It isn't something we test as part of the QA that we perform. I mostly expect people building meta-intel to be building with meta-yocto (although I wouldn't take a hard line on requiring it). That said, I removed meta-yocto from a meta-intel/meta-fri2 build and removed DISTRO=poky from my local.conf and successfully built and booted a core-image-minimal build on an FRI2 this afternoon without any changes.

This was with latest master of both poky and meta-intel. I have not tried this with oe-core and bitbake pulled down separately, I'll leave that as an exercise for the reader.

Signature · 3 lines
Darren Hart
Intel Open Source Technology Center
Yocto Project - Technical Lead - Linux Kernel
#12 lore raw
Quoted 9 lines · Darren Hart
Quoted 4 lines · William Mills

Darren: Is it true you can't get @ the Intel BSP's w/o also getting the poky distro defs? That does seem to mixing things a bit. (I am not claiming meta-ti is clean yet but I want to understand the Intel examples.)

Quoted 7 lines

It isn't something we test as part of the QA that we perform. I mostly expect people building meta-intel to be building with meta-yocto (although I wouldn't take a hard line on requiring it). That said, I removed meta-yocto from a meta-intel/meta-fri2 build and removed DISTRO=poky from my local.conf and successfully built and booted a core-image-minimal build on an FRI2 this afternoon without any changes.

Thanks! My confidence is restored.

As long as including meta-yocto does not interfere with other BSPs or distros etc then there should be no harm in your assumption.

I would be interested to know what Mentor Graphics and Wind River do on their products. Do they include meta-yocto? (YP is not all about comercial OS support but I know these orginatations have done the due diligence on layer compatibility for a non-poky distro.)

#13 lore raw
Quoted 20 lines · William Mills
Quoted 9 lines · Darren Hart
Quoted 4 lines · William Mills

Darren: Is it true you can't get @ the Intel BSP's w/o also getting the poky distro defs? That does seem to mixing things a bit. (I am not claiming meta-ti is clean yet but I want to understand the Intel examples.)

Quoted 7 lines

It isn't something we test as part of the QA that we perform. I mostly expect people building meta-intel to be building with meta-yocto (although I wouldn't take a hard line on requiring it). That said, I removed meta-yocto from a meta-intel/meta-fri2 build and removed DISTRO=poky from my local.conf and successfully built and booted a core-image-minimal build on an FRI2 this afternoon without any changes.

Thanks! My confidence is restored.

Quoted 7 lines

As long as including meta-yocto does not interfere with other BSPs or distros etc then there should be no harm in your assumption.

I would be interested to know what Mentor Graphics and Wind River do on their products. Do they include meta-yocto? (YP is not all about comercial OS support but I know these orginatations have done the due diligence on layer compatibility for a non-poky distro.)

I haven't heard one way or the other, but perhaps Bruce, Sean, and Matthew could comment on that.

Signature · 3 lines
Darren Hart
Intel Open Source Technology Center
Yocto Project - Technical Lead - Linux Kernel
#14 lore raw
In reply to William Mills
Quoted 21 lines · William Mills
Quoted 10 lines · Darren Hart
Quoted 5 lines · William Mills

Darren: Is it true you can't get @ the Intel BSP's w/o also getting the poky distro defs? That does seem to mixing things a bit. (I am not claiming meta-ti is clean yet but I want to understand the Intel examples.)

Quoted 7 lines

It isn't something we test as part of the QA that we perform. I mostly expect people building meta-intel to be building with meta-yocto (although I wouldn't take a hard line on requiring it). That said, I removed meta-yocto from a meta-intel/meta-fri2 build and removed DISTRO=poky from my local.conf and successfully built and booted a core-image-minimal build on an FRI2 this afternoon without any changes.

Thanks! My confidence is restored.

Quoted 7 lines

As long as including meta-yocto does not interfere with other BSPs or distros etc then there should be no harm in your assumption.

I would be interested to know what Mentor Graphics and Wind River do on their products. Do they include meta-yocto? (YP is not all about comercial OS support but I know these orginatations have done the due diligence on layer compatibility for a non-poky distro.)

Layering flexibility is important, so we work largely on oe-core vs meta-yocto. That being said, there are elements in meta-yocto that are of interest, so it can be included in used, but with layers that get the ordering and priority correct to override/preserve appropriately.

But sorting out exactly what we are talking about in this thread, would make using meta-yocto (or whatever the elements of it become), that much easier to do.

Cheers,

Bruce

Quoted 5 lines

_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto

#15 lore raw
In reply to William Mills
Quoted 20 lines · William Mills
Quoted 9 lines · Darren Hart
Quoted 4 lines · William Mills

Darren: Is it true you can't get @ the Intel BSP's w/o also getting the poky distro defs? That does seem to mixing things a bit. (I am not claiming meta-ti is clean yet but I want to understand the Intel examples.)

Quoted 7 lines

It isn't something we test as part of the QA that we perform. I mostly expect people building meta-intel to be building with meta-yocto (although I wouldn't take a hard line on requiring it). That said, I removed meta-yocto from a meta-intel/meta-fri2 build and removed DISTRO=poky from my local.conf and successfully built and booted a core-image-minimal build on an FRI2 this afternoon without any changes.

Thanks! My confidence is restored.

Quoted 7 lines

As long as including meta-yocto does not interfere with other BSPs or distros etc then there should be no harm in your assumption.

I would be interested to know what Mentor Graphics and Wind River do on their products. Do they include meta-yocto? (YP is not all about comercial OS support but I know these orginatations have done the due diligence on layer compatibility for a non-poky distro.)

Commercial OS support usually involves some of your own policy so meta-yocto is interesting as an example to them but they'd probably only use it as inspiration to write their own. That was always expected and meta-yocto is extremely thin deliberately.

Having said that, what meta-yocto was doing was wrong, it wasn't intentional, the implications not fully realised and is hopefully now fixed with the split :).

Cheers,

Richard

#16 lore raw
In reply to William Mills
Quoted 2 lines · William Mills

I suspect the current issue is just growing pains for a case that has not been tested.

The simplest fix would be for meta-ti to preppend itself to the path the same way meta-yocto does, i.e., in the layer.conf

  BBPATH := "${LAYERDIR}:${BBPATH}"

In fact, this is the *only* way that a layer can override any conf files in oe-core, which is probably what you always want in a BSP layer?

Just quickly scanning yocto git, there are a number of BSP layers that are configured the same way as meta-ti, so potentially have the same problem, e.g., meta-intel, meta-fsl-ppc. (There are other generic type layers configured this way too, but I think it's only a big issue for BSP layers and distro layers.)

As long as we are mixing layers that do both prepend and append, the layering will continue to be broken in subtle and hard to identify ways. I can't think of a practical use case where a layer other than oe-core might need to append itself to the BBPATH? If there were no genuine need for layers to append to the path, the BBPATH extension could be done automatically when including the layer, instead doing manually in each layer.conf, giving us some consistency.

Quoted 3 lines

Darren: Is it true you can't get @ the Intel BSP's w/o also getting the poky distro defs? That does seem to mixing things a bit. (I am not claiming meta-ti is clean yet but I want to understand the Intel examples.)

I apologize for getting this wrong, meta-intel is not dependent on meta-yocto (the meta-yocto bbappends are only apply to the machines meta-yocto itself contains configs for). But you do need meta-yocto for the atom-pc machine, because meta-yocto is the BSP layer for that (and should Intel decide to add an atom-pc to meta-intel, it will be broken exactly the same as the beagleboard machine is just now).

Tomas

#17 lore raw
Quoted 14 lines · On Wednesday 05 September 2012 09:49:11 Tomas Frydrych wrote:

The simplest fix would be for meta-ti to preppend itself to the path the same way meta-yocto does, i.e., in the layer.conf

  BBPATH := "${LAYERDIR}:${BBPATH}"

In fact, this is the *only* way that a layer can override any conf files in oe-core, which is probably what you always want in a BSP layer?

Just quickly scanning yocto git, there are a number of BSP layers that are configured the same way as meta-ti, so potentially have the same problem, e.g., meta-intel, meta-fsl-ppc. (There are other generic type layers configured this way too, but I think it's only a big issue for BSP layers and distro layers.)

Quoted 7 lines

As long as we are mixing layers that do both prepend and append, the layering will continue to be broken in subtle and hard to identify ways. I can't think of a practical use case where a layer other than oe-core might need to append itself to the BBPATH? If there were no genuine need for layers to append to the path, the BBPATH extension could be done automatically when including the layer, instead doing manually in each layer.conf, giving us some consistency.

It has been considered witin OE to be best practice to append to BBPATH and not prepend, the thinking being that then the search path matches the order of the layers listed in bblayers.conf rather than the reverse. I'm not sure I agree with it (I tend to prefer to list OE-Core first), but that's the convention adopted there.

Quite a few people have asked for the items which BBPATH controls (classes, conf files) to instead be found in layer priority order. If bitbake took over managing BBPATH, that would be a possibility, and as you say it would improve consistency at the expense of a little flexibility.

Quoted 4 lines

But you do need meta-yocto for the atom-pc machine, because meta-yocto is the BSP layer for that (and should Intel decide to add an atom-pc to meta-intel, it will be broken exactly the same as the beagleboard machine is just now).

Some time ago we discussed the possibility of moving the atom-pc BSP to meta- intel and then copying it back into meta-yocto, once the layer tooling allowed for that to be done dynamically. Since the layer tooling is now in place that is at least a practical possibility, but I'm not sure if it's still on the cards - it still makes sense to me at any rate.

Cheers,
Paul

Signature · 2 lines
Paul Eggleton
Intel Open Source Technology Centre
#18 lore raw
Quoted 4 lines · Paul Eggleton

On Wednesday 05 September 2012 09:49:11 Tomas Frydrych wrote:
It has been considered witin OE to be best practice to append to BBPATH and not prepend, the thinking being that then the search path matches the order of the layers listed in bblayers.conf rather than the reverse.

Then meta-yocto should follow that convention ... and it needs to be well documented, with the consequence of breaking that convention explained, and the terrible punishments to come described in great and sordid detail. Because this needs to be more than a convention, it needs to be an article of faith.

Tomas

#19 lore raw
Quoted 11 lines · Tomas Frydrych
Quoted 4 lines · Paul Eggleton

On Wednesday 05 September 2012 09:49:11 Tomas Frydrych wrote:
It has been considered witin OE to be best practice to append to BBPATH and not prepend, the thinking being that then the search path matches the order of the layers listed in bblayers.conf rather than the reverse.

Then meta-yocto should follow that convention ... and it needs to be well documented, with the consequence of breaking that convention explained, and the terrible punishments to come described in great and sordid detail. Because this needs to be more than a convention, it needs to be an article of faith.

I just want to clarify something here.

Its accepted that most layers will append to BBPATH. I do think its acceptable for a distro policy layer to prepend though and this is why meta-yocto does this. I don't remember the exact reason right now but the principle stands.

The root of the problem is that meta-yocto mixes up policy and hardware support which is bad. It also means its not compliant with the new Yocto Project compliance criteria and hence is not Yocto Project Compatible.

Now we've got the compliance criteria sorted out there are some adjustments that need to be made and I will shortly be cleaving meta-yocto into two pieces to resolve this. I hadn't looked at this until now mainly in case there were changes to the criteria.

FWIW I think this shows the strength of those criteria as by following them, we'd avoid a real world problem here.

Cheers,

Richard

#20 lore raw
Quoted 13 lines · Richard Purdie
Quoted 11 lines · Tomas Frydrych
Quoted 4 lines · Paul Eggleton

On Wednesday 05 September 2012 09:49:11 Tomas Frydrych wrote:
It has been considered witin OE to be best practice to append to BBPATH and not prepend, the thinking being that then the search path matches the order of the layers listed in bblayers.conf rather than the reverse.

Then meta-yocto should follow that convention ... and it needs to be well documented, with the consequence of breaking that convention explained, and the terrible punishments to come described in great and sordid detail. Because this needs to be more than a convention, it needs to be an article of faith.

Quoted 6 lines

I just want to clarify something here.

Its accepted that most layers will append to BBPATH. I do think its acceptable for a distro policy layer to prepend though and this is why meta-yocto does this. I don't remember the exact reason right now but the principle stands.

So how should we resolve the issue in meta-ti for the denzil/1.2 branch?

I think the expediency of prepending sounds right to me. We can shoot for a better fix in 1.3.

Quoted 5 lines

The root of the problem is that meta-yocto mixes up policy and hardware support which is bad. It also means its not compliant with the new Yocto Project compliance criteria and hence is not Yocto Project Compatible.

Quoted 7 lines

Now we've got the compliance criteria sorted out there are some adjustments that need to be made and I will shortly be cleaving meta-yocto into two pieces to resolve this. I hadn't looked at this until now mainly in case there were changes to the criteria.

FWIW I think this shows the strength of those criteria as by following them, we'd avoid a real world problem here.

I'm glad you are thinking of doing this. I think it sets a good example and is cleaner.

However we will still need priority or namespace to override the beagleboard definition in the meta-yocto-bsp (or whatever) layer if it contains both the beagleboard reference and the atom-pc.

#21 lore raw
Quoted 9 lines · William Mills
Quoted 4 lines

Its accepted that most layers will append to BBPATH. I do think its acceptable for a distro policy layer to prepend though and this is why meta-yocto does this. I don't remember the exact reason right now but the principle stands.

So how should we resolve the issue in meta-ti for the denzil/1.2 branch?

I think the expediency of prepending sounds right to me. We can shoot for a better fix in 1.3.

I think in the light of what Paul said, the meta-ti behaviour is what is expected and changing will potentially complicate things for other meta-ti consumers who follow the convention; the problem is in meta-yocto, not meta-ti.

The current problem can be worked around; it requires a custom layer that also prepends itself to the path in front of meta-yocto, and then provides a beagleboard.conf of it's own (which can simply 'include' or 'require' the beagleboard.conf from meta-ti). I think if Richard is working on a solution in meta-yocto, it might be enough to document this somewhere until it is ready.

Tomas

#22 lore raw
In reply to William Mills
Quoted 22 lines · William Mills
Quoted 13 lines · Richard Purdie
Quoted 11 lines · Tomas Frydrych
Quoted 4 lines · Paul Eggleton

On Wednesday 05 September 2012 09:49:11 Tomas Frydrych wrote:
It has been considered witin OE to be best practice to append to BBPATH and not prepend, the thinking being that then the search path matches the order of the layers listed in bblayers.conf rather than the reverse.

Then meta-yocto should follow that convention ... and it needs to be well documented, with the consequence of breaking that convention explained, and the terrible punishments to come described in great and sordid detail. Because this needs to be more than a convention, it needs to be an article of faith.

Quoted 6 lines

I just want to clarify something here.

Its accepted that most layers will append to BBPATH. I do think its acceptable for a distro policy layer to prepend though and this is why meta-yocto does this. I don't remember the exact reason right now but the principle stands.

Quoted 4 lines

So how should we resolve the issue in meta-ti for the denzil/1.2 branch?

I think the expediency of prepending sounds right to me. We can shoot for a better fix in 1.3.

I think the problem is in meta-yocto and is hitting a particular subset of users where people combine meta-yocto and meta-ti.

I'm not sure this can easily be fixed in meta-ti, or that we currently have a high volume of users mixing those two.

Quoted 12 lines
Quoted 11 lines

The root of the problem is that meta-yocto mixes up policy and hardware support which is bad. It also means its not compliant with the new Yocto Project compliance criteria and hence is not Yocto Project Compatible.

Now we've got the compliance criteria sorted out there are some adjustments that need to be made and I will shortly be cleaving meta-yocto into two pieces to resolve this. I hadn't looked at this until now mainly in case there were changes to the criteria.

FWIW I think this shows the strength of those criteria as by following them, we'd avoid a real world problem here.

Quoted 6 lines

I'm glad you are thinking of doing this. I think it sets a good example and is cleaner.

However we will still need priority or namespace to override the beagleboard definition in the meta-yocto-bsp (or whatever) layer if it contains both the beagleboard reference and the atom-pc.

With the solution, you can just order the meta-ti and meta-yocto-bsp layers such that the TI layer "wins" and no extra configuration is necessary. The trouble is that the meta-yocto doesn't allow this at the moment as things stand.

In many ways this is just part of the learning experience of how to use and how not to use layers...

Despite the fact we just passed the feature freeze point, I'm going to ensure we get this fixed for 1.3.

Cheers,

Richard

#23 lore raw
In reply to Richard Purdie
Quoted 13 lines · Richard Purdie
Quoted 11 lines · Tomas Frydrych
Quoted 4 lines · Paul Eggleton

On Wednesday 05 September 2012 09:49:11 Tomas Frydrych wrote:
It has been considered witin OE to be best practice to append to BBPATH and not prepend, the thinking being that then the search path matches the order of the layers listed in bblayers.conf rather than the reverse.

Then meta-yocto should follow that convention ... and it needs to be well documented, with the consequence of breaking that convention explained, and the terrible punishments to come described in great and sordid detail. Because this needs to be more than a convention, it needs to be an article of faith.

Quoted 6 lines

I just want to clarify something here.

Its accepted that most layers will append to BBPATH. I do think its acceptable for a distro policy layer to prepend though and this is why meta-yocto does this. I don't remember the exact reason right now but the principle stands.

Then the reason should be documented in layer's README file with warning that this layer wants to be first or last in BBPATH so whoever is setting BBLAYERS can decide where to put it.

Quoted 22 lines

The root of the problem is that meta-yocto mixes up policy and hardware support which is bad. It also means its not compliant with the new Yocto Project compliance criteria and hence is not Yocto Project Compatible.

Now we've got the compliance criteria sorted out there are some adjustments that need to be made and I will shortly be cleaving meta-yocto into two pieces to resolve this. I hadn't looked at this until now mainly in case there were changes to the criteria.

FWIW I think this shows the strength of those criteria as by following them, we'd avoid a real world problem here.

Cheers,

Richard

_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto

Signature · 1 line
Martin 'JaMa' Jansa     jabber: [email protected]
#24 lore raw
In reply to Richard Purdie
Quoted 8 lines · Richard Purdie

The root of the problem is that meta-yocto mixes up policy and hardware support which is bad. It also means its not compliant with the new Yocto Project compliance criteria and hence is not Yocto Project Compatible.

Now we've got the compliance criteria sorted out there are some adjustments that need to be made and I will shortly be cleaving meta-yocto into two pieces to resolve this. I hadn't looked at this until now mainly in case there were changes to the criteria.

This has now been fixed in:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=2000698b17011bbde1c3e5bb01a7d6763316db5a

#25 lore raw
In reply to Paul Eggleton
Quoted 39 lines · Paul Eggleton
Quoted 14 lines · On Wednesday 05 September 2012 09:49:11 Tomas Frydrych wrote:

The simplest fix would be for meta-ti to preppend itself to the path the same way meta-yocto does, i.e., in the layer.conf

  BBPATH := "${LAYERDIR}:${BBPATH}"

In fact, this is the *only* way that a layer can override any conf files in oe-core, which is probably what you always want in a BSP layer?

Just quickly scanning yocto git, there are a number of BSP layers that are configured the same way as meta-ti, so potentially have the same problem, e.g., meta-intel, meta-fsl-ppc. (There are other generic type layers configured this way too, but I think it's only a big issue for BSP layers and distro layers.)

Quoted 7 lines

As long as we are mixing layers that do both prepend and append, the layering will continue to be broken in subtle and hard to identify ways. I can't think of a practical use case where a layer other than oe-core might need to append itself to the BBPATH? If there were no genuine need for layers to append to the path, the BBPATH extension could be done automatically when including the layer, instead doing manually in each layer.conf, giving us some consistency.

It has been considered witin OE to be best practice to append to BBPATH and not prepend, the thinking being that then the search path matches the order of the layers listed in bblayers.conf rather than the reverse. I'm not sure I agree with it (I tend to prefer to list OE-Core first), but that's the convention adopted there.

Quite a few people have asked for the items which BBPATH controls (classes, conf files) to instead be found in layer priority order. If bitbake took over managing BBPATH, that would be a possibility, and as you say it would improve consistency at the expense of a little flexibility.

Quoted 4 lines

But you do need meta-yocto for the atom-pc machine, because meta-yocto is the BSP layer for that (and should Intel decide to add an atom-pc to meta-intel, it will be broken exactly the same as the beagleboard machine is just now).

Quoted 5 lines

Some time ago we discussed the possibility of moving the atom-pc BSP to meta- intel and then copying it back into meta-yocto, once the layer tooling allowed for that to be done dynamically. Since the layer tooling is now in place that is at least a practical possibility, but I'm not sure if it's still on the cards - it still makes sense to me at any rate.

A case can be made, but in my opinion, atom-pc is a generic BSP (not machine specific) and doesn't really fit with the other BSPs within meta-intel. I believe we should have an Intel BSP in meta-yocto, but I don't like the idea of duplicating a BSP in meta-yocto and meta-intel. atom-pc seems like a reasonable candidate to leave in meta-yocto to me.

I could probably be convinced otherwise, but that's my current thinking.

Signature · 3 lines
Darren Hart
Intel Open Source Technology Center
Yocto Project - Technical Lead - Linux Kernel
#26 lore raw
In reply to William Mills
Quoted 48 lines · William Mills
Quoted 16 lines · Darren Hart
Quoted 13 lines · Bruce Ashfield
Quoted 11 lines · Tomas Frydrych

Hi Bruce,

Quoted 2 lines · Bruce Ashfield

That being said, taking a step back, what are you trying to get out of meta-yocto in this scenario ?

a) I am targeting multiple chips, including TI Omap and Intel Atom. meta-yocto is a prerequisite for the various machines in meta-intel, so I have to include meta-yocto if I want to build images for an Intel chip. Nothing unusual here.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

b) meta-yocto is the Poky distro layer; if you want to use Poky, then you need meta-yocto.

Quoted 3 lines

see above. I misspoke. I don't think there's an intent to make meta-yocto and meta-ti work together, but oe-core + meta-ti, that's the combo that makes sense.

oe-core + meta-ti should work or it needs to get fixed. poky + meta-ti should work or it needs to get fixed.

However I suspect the 2nd is not in the nightly builds yet.

Denys is out for the next few days. He can comment more when he gets back.

It has been our assumption that there is enough functionality in the layer mechanisms that any of the "light weight" BSPs in yocto layer could be completely overridden by a more complete layer (meta-ti in this example). In addition the end system integrator should be able to override definitions in any BSP layer.

I suspect the current issue is just growing pains for a case that has not been tested. Lets prove that false before taking more drastic action.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

The basic problem with meta-yocto is that it combines BSP stuff (meta-intel prerequisite, Atom& Beagle config) with distro stuff (Poky, Yocto branding). That's convenient for doing QA on a limited set of HW, but suboptimal for real use; BSP layers simply should not be dependent on distro layers, it largely defeats the purpose of having layers.

Quoted 5 lines

Darren: Is it true you can't get @ the Intel BSP's w/o also getting the poky distro defs? That does seem to mixing things a bit. (I am not claiming meta-ti is clean yet but I want to understand the Intel examples.)

angstrom used meta-intel and it does not have meta-yocto that said I have not built for Intel machines using Angstrom so there might be issues but it parses fine for me so far.

Quoted 4 lines

_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto

Signature · 1 line
-Khem
#27 lore raw
In reply to Tomas Frydrych
Quoted 10 lines · Tomas Frydrych

Hi Bruce,

Quoted 2 lines · Bruce Ashfield

That being said, taking a step back, what are you trying to get out of meta-yocto in this scenario ?

a) I am targeting multiple chips, including TI Omap and Intel Atom. meta-yocto is a prerequisite for the various machines in meta-intel, so I have to include meta-yocto if I want to build images for an Intel chip. Nothing unusual here.

Is that really true? What in meta-intel depends on meta-yocto?

This certainly isn't intentional so I'd like to understand more.

Quoted 10 lines

b) meta-yocto is the Poky distro layer; if you want to use Poky, then you need meta-yocto.

Quoted 3 lines

see above. I misspoke. I don't think there's an intent to make meta-yocto and meta-ti work together, but oe-core + meta-ti, that's the combo that makes sense.

See (b) above; you are not saying that Poky is only meant for Intel HW, are you?

Quoted 8 lines

The basic problem with meta-yocto is that it combines BSP stuff (meta-intel prerequisite, Atom & Beagle config) with distro stuff (Poky, Yocto branding). That's convenient for doing QA on a limited set of HW, but suboptimal for real use; BSP layers simply should not be dependent on distro layers, it largely defeats the purpose of having layers.

Splitting out the minimal beagle config into a layer of its own would improve things quite a bit.

Effectively this is what we've now done and was always the intention (see the Yocto Project compatible criteria).

Cheers,

Richard

#28 lore raw
Quoted 7 lines · Richard Purdie
Quoted 6 lines · Tomas Frydrych

Hi Bruce,

Quoted 2 lines · Bruce Ashfield

That being said, taking a step back, what are you trying to get out of meta-yocto in this scenario ?

Quoted 6 lines
Quoted 4 lines

a) I am targeting multiple chips, including TI Omap and Intel Atom. meta-yocto is a prerequisite for the various machines in meta-intel, so I have to include meta-yocto if I want to build images for an Intel chip. Nothing unusual here.

Is that really true? What in meta-intel depends on meta-yocto?

No, it is not true; I corrected that erroneous statement earlier in this thread.

Quoted 2 lines

Effectively this is what we've now done and was always the intention (see the Yocto Project compatible criteria).

Yes, that looks like it solves the problem neatly. Now I should be able to include both meta-yocto and meta-yocto-bsb if I want to.

Tomas