The role of "distro" and "image"
I'm trying to wrap up my work on meta-tiny and integrate it into poky proper. I'm having some difficulty drawing a line between the responsibility of the distro definition versus that of the image definition.
For example, if I define a distro which uses tmpdevfs (no udev or mdev) and specifies KMACHINE=yocto/standard/tiny to build the tiny variant of the linux-yocto kernel, and customizes the busybox build to leave out various things - would it make sense for someone to then try to build core-image-sato with it? It doesn't to me, but nothing prevents the user from doing that (other than a likely build failure).
So what does the distro define and what does the image define?
Digging around in conf/distro for meta and meta-yocto, I see things like naming convention, additional dependencies, choice of toolchain and libc, source mirrors, default virtual providers, etc.
The image seems to basically define the package list for the image as well as the format of the rootfs and boot media.
If I understand this correctly, a new "tiny" distro definition would change the preferred linux-yocto provider to a linux-yocto-tiny recipe (which would in turn specify the default KMACHINE/KTYPE as well, the TCLIBC, DISTRO_FEATURES_LIBC, and some naming rules.
I currently define a new task-core-tiny.bb to reset some MACHINE_ESSENTIAL bits (qemu pulls in more than is necessary for tiny) and redefine the REDEPENDS for the image. I believe I could do this instead with assignments in the new distro definition and reuse task-core-boot.bb.
As for images, I might be able to reuse core-image-minimal - but it oddly contains "${POKY_EXTRA_INSTALL}" in the IMAGE_INSTALL. Since core-image-minimal.bb is defined in oe-core and POKY is a distro notion of meta-yocto, this contributes to my confusion over distro vs. image.
I'd appreciate some discussion to help me clarify where these lines should be.
Thanks,
Signature · 3 lines
Darren Hart Intel Open Source Technology Center Yocto Project - Linux Kernel
Quoted 10 lines · Darren Hart
I'm trying to wrap up my work on meta-tiny and integrate it into poky proper. I'm having some difficulty drawing a line between the responsibility of the distro definition versus that of the image definition.
For example, if I define a distro which uses tmpdevfs (no udev or mdev) and specifies KMACHINE=yocto/standard/tiny to build the tiny variant of the linux-yocto kernel, and customizes the busybox build to leave out various things - would it make sense for someone to then try to build core-image-sato with it? It doesn't to me, but nothing prevents the user from doing that (other than a likely build failure).
These all sounds like policy (and therefore distro) decisions to me. As with much of our system there's no hard and fast rule but this certainly sounds like a distro to me.
I think it's perfectly reasonable to assume that you can't build every image you find in the wild for every distro configuration you create.
Quoted 2 lines
So what does the distro define and what does the image define?
Distro defines the build configuration policy, image defines image contents.
You shouldn't set anything in the image definition which affects how things are built as the packages can then be used in another image.
Quoted 4 lines
Digging around in conf/distro for meta and meta-yocto, I see things like naming convention, additional dependencies, choice of toolchain and libc, source mirrors, default virtual providers, etc.
*nod*
Quoted 3 lines
The image seems to basically define the package list for the image as well as the format of the rootfs and boot media.
*nod* though arguably the latter 3 are likely poked elsewhere also.
Quoted 4 lines
If I understand this correctly, a new "tiny" distro definition would change the preferred linux-yocto provider to a linux-yocto-tiny recipe (which would in turn specify the default KMACHINE/KTYPE as well, the TCLIBC,
DISTRO_FEATURES_LIBC, and some naming rules.
Yes!
Quoted 5 lines
I currently define a new task-core-tiny.bb to reset some
MACHINE_ESSENTIALbits (qemu pulls in more than is necessary for tiny) and redefine the REDEPENDS for the image. I believe I could do this instead with assignments in the new distro definition and reuse task-core-boot.bb.
Seems like a good approach.
Quoted 7 lines
As for images, I might be able to reuse core-image-minimal - but it oddly contains "${POKY_EXTRA_INSTALL}" in the
IMAGE_INSTALL. Since core-image-minimal.bb is defined in oe-core and POKY is a distro notion of meta-yocto, this contributes to my confusion over distro vs. image.I'd appreciate some discussion to help me clarify where these lines should be.
That's just a stray variable that has yet to be properly renamed by the looks of it.
It sounds to me like your understanding is correct and this variable has just thrown you for a loop.
Cheers,
Joshua
Signature · 3 lines
Joshua Lock
Yocto Project "Johannes factotum"
Intel Open Source Technology CentreQuoted 11 lines · Joshua Lock
Quoted 8 lines
As for images, I might be able to reuse core-image-minimal - but it oddly contains "${POKY_EXTRA_INSTALL}" in the
IMAGE_INSTALL. Since core-image-minimal.bb is defined in oe-core and POKY is a distro notion of meta-yocto, this contributes to my confusion over distro vs. image.I'd appreciate some discussion to help me clarify where these lines should be.
That's just a stray variable that has yet to be properly renamed by the looks of it.
yes there was a patch to rename that to IMAGE_EXTRA_INSTALL or COREIMAGE_EXTRA_INSTALL
Quoted 4 lines · Darren Hart
I'm trying to wrap up my work on meta-tiny and integrate it into poky proper. I'm having some difficulty drawing a line between the responsibility of the distro definition versus that of the image definition.
Quoted 8 lines
For example, if I define a distro which uses tmpdevfs (no udev or mdev) and specifies KMACHINE=yocto/standard/tiny to build the tiny variant of the linux-yocto kernel, and customizes the busybox build to leave out various things - would it make sense for someone to then try to build core-image-sato with it? It doesn't to me, but nothing prevents the user from doing that (other than a likely build failure).
So what does the distro define and what does the image define?
I think Joshua's response is in line with what I'm thinking. Distro is where all the policy is specified. The image is just a list of pieces of software to include, its not meant to be much more than that.
Quoted 3 lines
Digging around in conf/distro for meta and meta-yocto, I see things like naming convention, additional dependencies, choice of toolchain and libc, source mirrors, default virtual providers, etc.
Right, all of this is "policy" at some level or another.
Quoted 2 lines
The image seems to basically define the package list for the image as well as the format of the rootfs and boot media.
The latter is often machine specific. There are some images which specify those things but they're a minority.
Quoted 4 lines
If I understand this correctly, a new "tiny" distro definition would change the preferred linux-yocto provider to a linux-yocto-tiny recipe (which would in turn specify the default KMACHINE/KTYPE as well, the TCLIBC,
DISTRO_FEATURES_LIBC, and some naming rules.
Yes, you'd inherit something for the base config and then customise it.
Quoted 5 lines
I currently define a new task-core-tiny.bb to reset some
MACHINE_ESSENTIALbits (qemu pulls in more than is necessary for tiny) and redefine the REDEPENDS for the image. I believe I could do this instead with assignments in the new distro definition and reuse task-core-boot.bb.
The latter reuse of task-core-boot would be ideal. If that can't work I'd be interested to see why and if we couldn't enhance it so it could work.
Quoted 4 lines
As for images, I might be able to reuse core-image-minimal - but it oddly contains "${POKY_EXTRA_INSTALL}" in the
IMAGE_INSTALL. Since core-image-minimal.bb is defined in oe-core and POKY is a distro notion of meta-yocto, this contributes to my confusion over distro vs. image.
This is a bug and there are some patches out there to turn it into COREIMAGE_EXTRA_INSTALL. Its a feature of the core-image.bbclass (formerly poky.bbclass, hence the name confusion).
Not all images can be built with a given distro, our base config is rather ride ranging though for obvious reasons. You could "enforce" this by adding some anonymous python to the distro which does something like:
if inherits("image") and PN != core-image-minimal:
raise SkipPackage("Image PN not compatible with DISTRO=XXX")
Cheers,
Richard
Quoted 6 lines · Richard Purdie
Not all images can be built with a given distro, our base config is rather ride ranging though for obvious reasons. You could "enforce" this by adding some anonymous python to the distro which does something like:
if
inherits("image")and PN != core-image-minimal:
raiseSkipPackage("Image PN not compatible with DISTRO=XXX")
This is a case where one is best 'controlling' this via policy, not mechanism, in my opinion. This sort of arbitrary technical limitation tends to be foolish and often bites someone somewhere down the line. I'm sure you just wanted to note how it could be done, not recommend that it should be done, but I thought it should be made clear. I wouldn't recommend that anyone do this unless there is an extremely good reason for it.
Signature · 1 line
Christopher Larson
Op 13 dec. 2011, om 22:45 heeft Chris Larson het volgende geschreven:
Quoted 9 lines
Quoted 6 lines · Richard Purdie
Not all images can be built with a given distro, our base config is rather ride ranging though for obvious reasons. You could "enforce" this by adding some anonymous python to the distro which does something like:
if
inherits("image")and PN != core-image-minimal:
raiseSkipPackage("Image PN not compatible with DISTRO=XXX")
Quoted 7 lines
This is a case where one is best 'controlling' this via policy, not mechanism, in my opinion. This sort of arbitrary technical limitation tends to be foolish and often bites someone somewhere down the line. I'm sure you just wanted to note how it could be done, not recommend that it should be done, but I thought it should be made clear. I wouldn't recommend that anyone do this unless there is an extremely good reason for it.
We had a similar problem at work, people were sprinkling COMPATIBLE_MACHINE left and right just to mark "I tested recipe X on machine Y". You can guess what happened when new machines needed to get added.
regards,
Koen
Quoted 19 lines · Koen Kooi
Op 13 dec. 2011, om 22:45 heeft Chris Larson het volgende geschreven:
Quoted 9 lines
Quoted 6 lines · Richard Purdie
Not all images can be built with a given distro, our base config is rather ride ranging though for obvious reasons. You could "enforce" this by adding some anonymous python to the distro which does something like:
if
inherits("image")and PN != core-image-minimal:
raiseSkipPackage("Image PN not compatible with DISTRO=XXX")Quoted 7 lines
This is a case where one is best 'controlling' this via policy, not mechanism, in my opinion. This sort of arbitrary technical limitation tends to be foolish and often bites someone somewhere down the line. I'm sure you just wanted to note how it could be done, not recommend that it should be done, but I thought it should be made clear. I wouldn't recommend that anyone do this unless there is an extremely good reason for it.
Quoted 1 line
We had a similar problem at work, people were sprinkling
COMPATIBLE_MACHINEleft and right just to mark "I tested recipe X on machine Y". You can guess what happened when new machines needed to get added.
Haha, ouch. It's also worth taking a moment to emphasize that the way distro/machine/image is structured is by design, not accidental. Having these pieces be orthogonal buys us a great deal of flexibility and capability, which is why we did it this way. Now, whether a given distro/machine/image combination is *useful* is a different question, and one I think is best addressed via policy / documentation.
Signature · 1 line
Christopher Larson
Quoted 25 lines · Chris Larson
Quoted 19 lines · Koen Kooi
Op 13 dec. 2011, om 22:45 heeft Chris Larson het volgende geschreven:
Quoted 9 lines
Quoted 6 lines · Richard Purdie
Not all images can be built with a given distro, our base config is rather ride ranging though for obvious reasons. You could "enforce" this by adding some anonymous python to the distro which does something like:
if
inherits("image")and PN != core-image-minimal:
raiseSkipPackage("Image PN not compatible with DISTRO=XXX")Quoted 7 lines
This is a case where one is best 'controlling' this via policy, not mechanism, in my opinion. This sort of arbitrary technical limitation tends to be foolish and often bites someone somewhere down the line. I'm sure you just wanted to note how it could be done, not recommend that it should be done, but I thought it should be made clear. I wouldn't recommend that anyone do this unless there is an extremely good reason for it.
Quoted 1 line
We had a similar problem at work, people were sprinkling
COMPATIBLE_MACHINEleft and right just to mark "I tested recipe X on machine Y". You can guess what happened when new machines needed to get added.
Quoted 6 lines
Haha, ouch. It's also worth taking a moment to emphasize that the way distro/machine/image is structured is by design, not accidental. Having these pieces be orthogonal buys us a great deal of flexibility and capability, which is why we did it this way. Now, whether a given distro/machine/image combination is *useful* is a different question, and one I think is best addressed via policy / documentation.
I understand the flexibility but I also think we need to consider usability a little. We have people using the system to whom it might not be immediately obvious why building a sato image with the tiny distro is a bad idea.
If you know enough to be able to make that work, the warning is easily bypassed and isn't going to bother you.
So I think there is a case for warning the user if the combination they're selecting is not one that is likely to work and we can easily predict it. There are many combinations we can't predict which is fine.
Like any mechanism, it can be abused of course and using COMPATIBLE_MACHINE as a sign of recipe testing clearly isn't a good idea or what it was designed for.
Cheers,
Richard
Quoted 38 lines · Richard Purdie
Quoted 25 lines · Chris Larson
Quoted 19 lines · Koen Kooi
Op 13 dec. 2011, om 22:45 heeft Chris Larson het volgende geschreven:
Quoted 9 lines
Quoted 6 lines · Richard Purdie
Not all images can be built with a given distro, our base config is rather ride ranging though for obvious reasons. You could "enforce" this by adding some anonymous python to the distro which does something like:
if
inherits("image")and PN != core-image-minimal:
raiseSkipPackage("Image PN not compatible with DISTRO=XXX")Quoted 7 lines
This is a case where one is best 'controlling' this via policy, not mechanism, in my opinion. This sort of arbitrary technical limitation tends to be foolish and often bites someone somewhere down the line. I'm sure you just wanted to note how it could be done, not recommend that it should be done, but I thought it should be made clear. I wouldn't recommend that anyone do this unless there is an extremely good reason for it.
Quoted 1 line
We had a similar problem at work, people were sprinkling
COMPATIBLE_MACHINEleft and right just to mark "I tested recipe X on machine Y". You can guess what happened when new machines needed to get added.Quoted 6 lines
Haha, ouch. It's also worth taking a moment to emphasize that the way distro/machine/image is structured is by design, not accidental. Having these pieces be orthogonal buys us a great deal of flexibility and capability, which is why we did it this way. Now, whether a given distro/machine/image combination is *useful* is a different question, and one I think is best addressed via policy / documentation.
I understand the flexibility but I also think we need to consider usability a little. We have people using the system to whom it might not be immediately obvious why building a sato image with the tiny distro is a bad idea.
Quoted 6 lines
If you know enough to be able to make that work, the warning is easily bypassed and isn't going to bother you.
So I think there is a case for warning the user if the combination they're selecting is not one that is likely to work and we can easily predict it. There are many combinations we can't predict which is fine.
There's a rather large difference between *warning* the user and raising SkipPackage, making it not possible to build without modifying the recipe.
Signature · 1 line
Christopher Larson
Quoted 43 lines · Chris Larson
Quoted 41 lines · Richard Purdie
Quoted 25 lines · Chris Larson
Quoted 19 lines · Koen Kooi
Op 13 dec. 2011, om 22:45 heeft Chris Larson het volgende geschreven:
Quoted 9 lines
Quoted 6 lines · Richard Purdie
Not all images can be built with a given distro, our base config is rather ride ranging though for obvious reasons. You could "enforce" this by adding some anonymous python to the distro which does something like:
if
inherits("image")and PN != core-image-minimal:
raiseSkipPackage("Image PN not compatible with DISTRO=XXX")Quoted 7 lines
This is a case where one is best 'controlling' this via policy, not mechanism, in my opinion. This sort of arbitrary technical limitation tends to be foolish and often bites someone somewhere down the line. I'm sure you just wanted to note how it could be done, not recommend that it should be done, but I thought it should be made clear. I wouldn't recommend that anyone do this unless there is an extremely good reason for it.
Quoted 1 line
We had a similar problem at work, people were sprinkling
COMPATIBLE_MACHINEleft and right just to mark "I tested recipe X on machine Y". You can guess what happened when new machines needed to get added.Quoted 6 lines
Haha, ouch. It's also worth taking a moment to emphasize that the way distro/machine/image is structured is by design, not accidental. Having these pieces be orthogonal buys us a great deal of flexibility and capability, which is why we did it this way. Now, whether a given distro/machine/image combination is *useful* is a different question, and one I think is best addressed via policy / documentation.
I understand the flexibility but I also think we need to consider usability a little. We have people using the system to whom it might not be immediately obvious why building a sato image with the tiny distro is a bad idea.
If you know enough to be able to make that work, the warning is easily bypassed and isn't going to bother you.
Quoted 8 lines
Quoted 3 lines
So I think there is a case for warning the user if the combination they're selecting is not one that is likely to work and we can easily predict it. There are many combinations we can't predict which is fine.
There's a rather large difference between *warning* the user and raising SkipPackage, making it not possible to build without modifying the recipe.
What's needed is something which results in a "stop the build and force the user to change something to continue" so I'm not sure I see the problem with that.
I know exactly how much attention people pay to the WARNING messages :( (those are getting cleaned up so hopefully those WARNINGS that are still present become more meaningful/serious)
Cheers,
Richard
Quoted 49 lines · Richard Purdie
Quoted 48 lines · Chris Larson
Quoted 44 lines · Richard Purdie
Quoted 25 lines · Chris Larson
Quoted 19 lines · Koen Kooi
Op 13 dec. 2011, om 22:45 heeft Chris Larson het volgende geschreven:
Quoted 9 lines
Quoted 6 lines · Richard Purdie
Not all images can be built with a given distro, our base config is rather ride ranging though for obvious reasons. You could "enforce" this by adding some anonymous python to the distro which does something like:
if
inherits("image")and PN != core-image-minimal:
raiseSkipPackage("Image PN not compatible with DISTRO=XXX")Quoted 7 lines
This is a case where one is best 'controlling' this via policy, not mechanism, in my opinion. This sort of arbitrary technical limitation tends to be foolish and often bites someone somewhere down the line. I'm sure you just wanted to note how it could be done, not recommend that it should be done, but I thought it should be made clear. I wouldn't recommend that anyone do this unless there is an extremely good reason for it.
Quoted 1 line
We had a similar problem at work, people were sprinkling
COMPATIBLE_MACHINEleft and right just to mark "I tested recipe X on machine Y". You can guess what happened when new machines needed to get added.Quoted 6 lines
Haha, ouch. It's also worth taking a moment to emphasize that the way distro/machine/image is structured is by design, not accidental. Having these pieces be orthogonal buys us a great deal of flexibility and capability, which is why we did it this way. Now, whether a given distro/machine/image combination is *useful* is a different question, and one I think is best addressed via policy / documentation.
I understand the flexibility but I also think we need to consider usability a little. We have people using the system to whom it might not be immediately obvious why building a sato image with the tiny distro is a bad idea.
If you know enough to be able to make that work, the warning is easily bypassed and isn't going to bother you.
So I think there is a case for warning the user if the combination they're selecting is not one that is likely to work and we can easily predict it. There are many combinations we can't predict which is fine.
Quoted 7 lines
Quoted 3 lines
There's a rather large difference between *warning* the user and raising SkipPackage, making it not possible to build without modifying the recipe.
What's needed is something which results in a "stop the build and force the user to change something to continue" so I'm not sure I see the problem with that.
I strongly disagree. That's exactly what we shouldn't have, at least unless there's an easy way to disable the behavior. Even our sanity checks can be opted out, and those are far more critical than building a rootfs that might not behave in a useful fashion on a particular piece of hardware.
Signature · 1 line
Christopher Larson