OpenEmbedded Core - Ready for extended users
Hi,
The TSC has been tasked with working out how OE and Yocto could work together and people have no doubt seen the discussions about OE-Core.
We're now pleased to formally announce that we feel OE-Core is ready for exposure to more users and we'd like to encourage people to have a look at it and experiment and develop with it.
Just to recap on the background for this, splitting up OE into components has been a topic of discussion for a long time, certainly at every OEDEM. Until now this hasn't happened partly due to resource issues but also lack of tooling and the lack of a well thought out plan. We now feel these issues have been addressed and that we have a good plan in place. The promotion and use of layers is the only real way OE can scale as a project yet stay maintainable and testable.
Things that aren't part of oe-core fall to the meta-oe layer which is where we intend the current wider support contained in oe-devel to be maintained. The people working on meta-oe so far would love to see more contributions. Both repositories are currently working on a "pull" model and people using them so far have felt a benefit to this for code quality.
Discussion and development on oe-core is happening on the openembedded-core mailing list since otherwise it would be hard to filter out which patches and discussion were for which repository. We look forward to seeing more people getting involved there!
(http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core)
Cheers,
Richard
(Communicating on behalf of the TSC)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Quoted 6 lines · Richard Purdie
Things that aren't part of oe-core fall to the meta-oe layer which is where we intend the current wider support contained in oe-devel to be maintained. The people working on meta-oe so far would love to see more contributions. Both repositories are currently working on a "pull" model and people using them so far have felt a benefit to this for code quality.
The idea is that every layer has a top level README describing how to contribute.
For meta-oe:
http://cgit.openembedded.org/cgit.cgi/meta-openembedded/tree/meta-oe/README
And for meta-angstrom:
http://git.angstrom-distribution.org/cgi-bin/cgit.cgi/meta-angstrom/tree/README
If someone wants to come up with a standard for the layer README files, please do!
regards,
Koen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)
iD8DBQFN0RFEMkyGM64RGpERAps4AJ9WHjtvVSCIkp8AvJ9GtLij/11RGgCePlMc oMLNqa9vr5yd99HcV6aqrpY=
=Gb4s
-----END PGP SIGNATURE-----
2011/5/16 Richard Purdie <[email protected]>
Quoted 31 lines
Hi,
The TSC has been tasked with working out how OE and Yocto could work together and people have no doubt seen the discussions about OE-Core.
We're now pleased to formally announce that we feel OE-Core is ready for exposure to more users and we'd like to encourage people to have a look at it and experiment and develop with it.
Just to recap on the background for this, splitting up OE into components has been a topic of discussion for a long time, certainly at every OEDEM. Until now this hasn't happened partly due to resource issues but also lack of tooling and the lack of a well thought out plan. We now feel these issues have been addressed and that we have a good plan in place. The promotion and use of layers is the only real way OE can scale as a project yet stay maintainable and testable.
Things that aren't part of oe-core fall to the meta-oe layer which is where we intend the current wider support contained in oe-devel to be maintained. The people working on meta-oe so far would love to see more contributions. Both repositories are currently working on a "pull" model and people using them so far have felt a benefit to this for code quality.
Discussion and development on oe-core is happening on the openembedded-core mailing list since otherwise it would be hard to filter out which patches and discussion were for which repository. We look forward to seeing more people getting involved there!
(http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core)
Quoted 7 lines
Cheers,
Richard
(Communicating on behalf of the TSC)Richard, thanks for the announcement.
I think the community could benefit if you add info (a pointer) in this thread
- how to set up a build environment
- policies on what goes into oe-core and meta-oe and what not (and how to deal with things that do not go into it)
- a migration scenario from the current repo to the new structure (what are we doing with the existing recipes)
- how to add new distro's and bsp's
I know some (if not all) info has been sent around over time, but I feel it would help if this announcement would carry the info to get people started (and/or point to a web page that has the above info).
Best regards, Frans
Quoted 9 lines · Frans Meulenbroeks
I think the community could benefit if you add info (a pointer) in this thread
- how to set up a build environment
- policies on what goes into oe-core and meta-oe and what not (and how to deal with things that do not go into it)
- a migration scenario from the current repo to the new structure (what are we doing with the existing recipes)
- how to add new distro's and bsp's
Quoted 3 lines
I know some (if not all) info has been sent around over time, but I feel it would help if this announcement would carry the info to get people started (and/or point to a web page that has the above info).
Yes, agreed, I think that would be useful. The build environment setup is certainly a bit different to "classic" oe and it took me a while to figure it out. At the time there didn't seem to be much documentation on how that stuff was meant to work, though with hindsight I now suspect the Poky manual might have been the place to look.
p.
Quoted 6 lines · Phil Blundell
Quoted 5 lines · Frans Meulenbroeks
I think the community could benefit if you add info (a pointer) in this thread
- how to set up a build environment
- policies on what goes into oe-core and meta-oe and what not (and how to deal with things that do not go into it)
Yes, this is definitely needed. Which type of new applications/libraries can get into meta-oe, and which are supposed to be kept in higher layers...
In what cases can additional versions of the recipe in oe-core be added? E.g. dbus is built using 'EXTRA_OECONF = "--with-x"', which at least on head-less systems seems to be overkill. Still, it's preferable (from my point of view) to use as much recipes as possible from oe-core, instead of maintaining separate copy in a local layer.
Quoted 2 lines
Quoted 2 lines
- a migration scenario from the current repo to the new structure (what are we doing with the existing recipes)
This is part of what I'm trying to get some time to do now. I've at least got sofar as to get the build starting, however, before proceeding further on, I've got to double check our local/modified copies of a number of recipes (see e.g. the point on dbus above).
When I'm "finished" converting our distribution/repo; I might get some time to try to write up a short text on this (unless someone already has done this). However, this is probably a couple of weeks ahead, at least.
Quoted 3 lines
Quoted 3 lines
I know some (if not all) info has been sent around over time, but I feel it would help if this announcement would carry the info to get people started (and/or point to a web page that has the above info).
Yes, an agregated / summarized source for this info is definitely needed.
Quoted 5 lines
Yes, agreed, I think that would be useful. The build environment setup is certainly a bit different to "classic" oe and it took me a while to figure it out. At the time there didn't seem to be much documentation on how that stuff was meant to work, though with hindsight I now suspect the Poky manual might have been the place to look.
I guess that might be right. Currently the move that I'm doing will be quite similar to the old oe-style. I've got quite good help by studying the angstrom setup scripts. The Poky manual seems to use a style where other meta-layers are installed in the poky-tree, however I prefer a layout more similar to e.g. angstrom:
own-top level
|- conf/ |- bitbake/ |- openembedded-core/ |- meta-openembedded/
or the angstrom version:
own-top level
|- conf/
|- bitbake/ // not sure on which level bitbake is kept.
\ sources/
|- openembedded-core
|- meta-openembedded
Such a layout makes it a lot easier to keep the top-level in a private/public repo, just referencing the oe-core and meta-oe layers using git submodules.
Regards,
Anders Darander
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Quoted 8 lines · Anders Darander
Quoted 6 lines · Phil Blundell
Quoted 5 lines · Frans Meulenbroeks
I think the community could benefit if you add info (a pointer) in this thread
- how to set up a build environment
- policies on what goes into oe-core and meta-oe and what not (and how to deal with things that do not go into it)
Quoted 7 lines
Yes, this is definitely needed. Which type of new applications/libraries can get into meta-oe, and which are supposed to be kept in higher layers...
In what cases can additional versions of the recipe in oe-core be added? E.g. dbus is built using '
EXTRA_OECONF= "--with-x"', which at least on head-less systems seems to be overkill.
Dbus is a really good example where the above knee-jerk reaction is misplaced :) The --with-x only turns on building dbus-launch, which is x specific, but that's the only bit that is. All the other dbus subpackages lack X dependencies.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)
iD8DBQFN0hfyMkyGM64RGpERAmkdAJ96oZbdWIr0vN9+ys7FNBGoH2HtaACfVyTl /5bZEgjnUvyLTFB9apyiFQc=
=XEmW
-----END PGP SIGNATURE-----
Quoted 5 lines · Koen Kooi
Quoted 4 lines · Anders Darander
Yes, this is definitely needed. Which type of new applications/libraries can get into meta-oe, and which are supposed to be kept in higher layers...
Quoted 8 lines
Quoted 3 lines
In what cases can additional versions of the recipe in oe-core be added? E.g. dbus is built using '
EXTRA_OECONF= "--with-x"', which at least on head-less systems seems to be overkill.Dbus is a really good example where the above knee-jerk reaction is misplaced :) The --with-x only turns on building dbus-launch, which is x specific, but that's the only bit that is. All the other dbus subpackages lack X dependencies.
Yes, from that point, Dbus was not the optimal example, I agree...
Though, that was only one example of a package that has either runtime or only build-time dependencies on X in some form or another. Earlier, in the 2011.03-releases, we had systemd depending on gtk+, which has been removed in the lates updates. Thats just another example.
In some cases, removing direct or indirect dependencies like these, would be something that we'd like to see integrated upstreams. Of course, we are willing to supply patches for that, after a discussion on how such things should be handled. Is it desired for package Y or isn't it. This could sometimes be to reduce the image size, but in cases like dbus, it might only be to reduce build time and build-directory size.
In other cases, I assume that we're going to continue keeping our patchsets local, as I guess that we probably have quite uncommon desires to get a certain packet working without e.g. X or some other dependency.
Regards,
Anders
Anders Darander <[email protected]> [2011-05-17 09:04:35]:
Quoted 6 lines
Yes, from that point, Dbus was not the optimal example, I agree...
Though, that was only one example of a package that has either runtime or only build-time dependencies on X in some form or another. Earlier, in the 2011.03-releases, we had systemd depending on gtk+, which has been removed in the lates updates. Thats just another example.
Don't worry, you're not alone in that feeling, actually I think, that we're quite a large group fighting with this :-) One day the image has 1700 tasks, after one rebase within a week timespan it grows almost to 3000, don't mention bluetooth and similar stuff etc. Well, I'm also looking forward to see, how it will change with oe-core.
-- ynezz
Quoted 9 lines · Petr Štetiar
Anders Darander <[email protected]> [2011-05-17 09:04:35]:
Quoted 6 lines
Yes, from that point, Dbus was not the optimal example, I agree...
Though, that was only one example of a package that has either runtime or only build-time dependencies on X in some form or another. Earlier, in the 2011.03-releases, we had systemd depending on gtk+, which has been removed in the lates updates. Thats just another example.
Quoted 5 lines
Don't worry, you're not alone in that feeling, actually I think, that we're quite a large group fighting with this :-) One day the image has 1700 tasks, after one rebase within a week timespan it grows almost to 3000, don't mention bluetooth and similar stuff etc. Well, I'm also looking forward to see, how it will change with oe-core.
These are issues we need to address I think everyone agrees on that. What we do need is some overall architectural way to handle it though rather and doing it adhoc on a case by case basis. There has been some discussion on the OE-Core list about it but I don't think we've reached any conclusion.
Out of interesting are there any new proposals on how to handle this? I'm interested in proposals that are thought through to details like how distros handle differences and being able to identify which configurations are used when presented with packages...
Cheers,
Richard
Quoted 13 lines · Richard Purdie
Quoted 11 lines · Petr Štetiar
Anders Darander <[email protected]> [2011-05-17 09:04:35]:
Quoted 4 lines
Though, that was only one example of a package that has either runtime or only build-time dependencies on X in some form or another. Earlier, in the 2011.03-releases, we had systemd depending on gtk+, which has been removed in the lates updates. Thats just another example.
Don't worry, you're not alone in that feeling, actually I think, that we're quite a large group fighting with this :-) One day the image has 1700 tasks, after one rebase within a week timespan it grows almost to 3000, don't mention bluetooth and similar stuff etc. Well, I'm also looking forward to see, how it will change with oe-core.
Quoted 5 lines
These are issues we need to address I think everyone agrees on that. What we do need is some overall architectural way to handle it though rather and doing it adhoc on a case by case basis. There has been some discussion on the OE-Core list about it but I don't think we've reached any conclusion.
I had some trouble locating these discussions on the oe-core list. Do you have hint/link towards those discussions? I'd be interested to read them and see what ideas has been discussed.
Quoted 4 lines
Out of interesting are there any new proposals on how to handle this? I'm interested in proposals that are thought through to details like how distros handle differences and being able to identify which configurations are used when presented with packages...
Has there been any proposals earlier?
Sorry, no real ideas at the moment. The main alterniatives, I guess, are to conditionally add depends and configure options based on either:
* More variables. Although this could lead to _a lot more_ variables in the end. (I.e. see the recent discussion on the oe-core list on task-base etc.)
* On if certain other recipes are selected. I'm not sure that this is even possible? (I haven't looked that much into bitbake and recipe parsing) This is solution is obviously a lot easier when using e.g. Kconfig-based configurations, like Buildroot.
I'm not sure of which option is the best, or if there is another better option...
Regards,
Anders