Skip to contentSkip to conversation

kernel-devsrc used to be full source - not the case since thud

DISCUSSION Andrei Gherzan 2 participants 11 messages May 23, 2019

lore
Share & tools
lore
#0 lore raw

Hello,

This might have been discussed before. I couldn't find a relevant thread, but if it is so, just link me to it.

Since thud, more specific since

commit 9af0f1a46bbb6ad9ee8b35957251f4aa826b023f
Author: Bruce Ashfield <[email protected]>
Date:   Sat Aug 18 22:50:44 2018 -0400
    kernel-devsrc: restructure for out of tree (and on target) module builds

... we switched from a recipe that was deploying the entire source code to one that provides mainly the kernel headers (but not only). This change broke people expectations of this recipe while the description is also confusing: "Development source linux kernel. When built, this recipe packages the \source of the preferred virtual/kernel provider and makes it available for full kernel \development or external module builds".

If size is not a problem (which can be the case when you compile on a builder for example and deploy only a OOT kernel module through other means), the full kernel source was a painless experience where things like

commit a9471601fedd1f5087304eaa5fd39b98ae220313
Author: Bruce Ashfield <[email protected]> Date:   Thu Aug 30 09:45:41 2018 -0400
    kernel-devsrc: fix arm/arm64 target module build

... would not appear. I understand the size impact on target and for those cases, continuously maintaining this recipe with new files/resources needed from the kernel, makes sense. So my proposal is to have two recipes, for example kernel-devsrc and kernel-fullsrc (kernel-src etc.) so people can choose what they need/want deploying/using. Or even have another devsrc provider. I'm open to any implementation detail. I'd just want to have an option for a full kernel source recipe.

Regards,

Signature · 2 lines
Andrei Gherzan
gpg: rsa4096/D4D94F67AD0E9640 | t: @agherzan
#1 lore raw
Quoted 29 lines · Andrei Gherzan

Hello,

This might have been discussed before. I couldn't find a relevant thread, but if it is so, just link me to it.

Since thud, more specific since

commit 9af0f1a46bbb6ad9ee8b35957251f4aa826b023f

Author: Bruce Ashfield <[email protected]>
Date:   Sat Aug 18 22:50:44 2018 -0400

kernel-devsrc: restructure for out of tree (and on target) module
builds

... we switched from a recipe that was deploying the entire source code to one that provides mainly the kernel headers (but not only). This change broke people expectations of this recipe while the description is also confusing: "Development source linux kernel. When built, this recipe packages the \source of the preferred virtual/kernel provider and makes it available for full kernel \development or external module builds".

If size is not a problem (which can be the case when you compile on a builder for example and deploy only a OOT kernel module through other means), the full kernel source was a painless experience where things like

commit a9471601fedd1f5087304eaa5fd39b98ae220313

Author: Bruce Ashfield <[email protected]>
Date:   Thu Aug 30 09:45:41 2018 -0400

kernel-devsrc: fix arm/arm64 target module build

Quoted 9 lines

... would not appear. I understand the size impact on target and for those cases, continuously maintaining this recipe with new files/resources needed from the kernel, makes sense. So my proposal is to have two recipes, for example kernel-devsrc and kernel-fullsrc (kernel-src etc.) so people can choose what they need/want deploying/using. Or even have another devsrc provider. I'm open to any implementation detail. I'd just want to have an option for a full kernel source recipe.

This is already planned, and hidden in bugzilla somewhere. I'll have some new kernel packaging
options available for the fall release.

Bruce

Quoted 13 lines

Regards,

--
Andrei Gherzan
gpg: rsa4096/D4D94F67AD0E9640 | t: @agherzan

--
_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II
#2 lore raw
Quoted 44 lines · Bruce Ashfield
Quoted 30 lines · Andrei Gherzan

Hello,

This might have been discussed before. I couldn't find a relevant thread, but if it is so, just link me to it.

Since thud, more specific since

commit 9af0f1a46bbb6ad9ee8b35957251f4aa826b023f

Author: Bruce Ashfield <[email protected]>
Date:   Sat Aug 18 22:50:44 2018 -0400

kernel-devsrc: restructure for out of tree (and on target) module
builds

... we switched from a recipe that was deploying the entire source code to one that provides mainly the kernel headers (but not only). This change broke people expectations of this recipe while the description is also confusing: "Development source linux kernel. When built, this recipe packages the \source of the preferred virtual/kernel provider and makes it available for full kernel \development or external module builds".

If size is not a problem (which can be the case when you compile on a builder for example and deploy only a OOT kernel module through other means), the full kernel source was a painless experience where things like

commit a9471601fedd1f5087304eaa5fd39b98ae220313

Author: Bruce Ashfield <[email protected]>
Date:   Thu Aug 30 09:45:41 2018 -0400

kernel-devsrc: fix arm/arm64 target module build

Quoted 9 lines

... would not appear. I understand the size impact on target and for those cases, continuously maintaining this recipe with new files/resources needed from the kernel, makes sense. So my proposal is to have two recipes, for example kernel-devsrc and kernel-fullsrc (kernel-src etc.) so people can choose what they need/want deploying/using. Or even have another devsrc provider. I'm open to any implementation detail. I'd just want to have an option for a full kernel source recipe.

Quoted 4 lines

This is already planned, and hidden in bugzilla somewhere. I'll have some new kernel packaging
options available for the fall release.

It looks like the bugs that I was using for development were finally moved to resolved (they were a bit old, and contained collected information on various kernel packaging options .. my searching of bugzilla isn't turning it up at the moment). So I just created a new bug to track the development for 2.8.

The issue with the multiple kernel source providers is really about test cycles. The smaller devsrc is for on-target module development and builds against the exported uapi headers, and that is what the nightly / automated tests will use. We had issues with both the amount of time it took to package the entire source, and the amount of space that it took up on the images. Hence the creation of devsrc.

With new kernel-source and kernel-headers packages (the working names), they are really provided as references to the running kernel, and will largely be an exercise left up to the developer to use them as they want.

Cheers,

Bruce

Quoted 26 lines

Bruce

Quoted 7 lines

Regards,

--
Andrei Gherzan
gpg: rsa4096/D4D94F67AD0E9640 | t: @agherzan

Quoted 6 lines

--
_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II
#3 lore raw
Quoted 81 lines · Bruce Ashfield

On Thu, May 23, 2019 at 10:32 AM Bruce Ashfield <[email protected] <mailto:[email protected]>> wrote:

On Thu, May 23, 2019 at 9:56 AM Andrei Gherzan <[email protected] <mailto:[email protected]>> wrote:

Hello,

This might have been discussed before. I couldn't find a relevant thread, but if it is so, just link me to it.

Since thud, more specific since

commit 9af0f1a46bbb6ad9ee8b35957251f4aa826b023f
Author: Bruce Ashfield <[email protected] <mailto:[email protected]>>
Date:   Sat Aug 18 22:50:44 2018 -0400
    kernel-devsrc: restructure for out of tree (and on target) module builds

... we switched from a recipe that was deploying the entire source code
to one that provides mainly the kernel headers (but not only). This
change broke people expectations of this recipe while the description is
also confusing: "Development source linux kernel. When built, this recipe packages the \source of the preferred virtual/kernel provider and
makes it available for full kernel \development or external module builds".

If size is not a problem (which can be the case when you compile on a
builder for example and deploy only a OOT kernel module through other
means), the full kernel source was a painless experience where things like

commit a9471601fedd1f5087304eaa5fd39b98ae220313
Author: Bruce Ashfield <[email protected] <mailto:[email protected]>> Date:   Thu Aug 30 09:45:41 2018 -0400
    kernel-devsrc: fix arm/arm64 target module build

... would not appear. I understand the size impact on target and for
those cases, continuously maintaining this recipe with new files/resources needed from the kernel, makes sense. So my proposal is
to have two recipes, for example kernel-devsrc and kernel-fullsrc (kernel-src etc.) so people can choose what they need/want deploying/using. Or even have another devsrc provider. I'm open to any
implementation detail. I'd just want to have an option for a full kernel
source recipe.

This is already planned, and hidden in bugzilla somewhere. I'll have some new kernel packaging
options available for the fall release.

It looks like the bugs that I was using for development were finally moved to resolved (they were a bit old, and contained collected information on various kernel packaging options .. my searching of bugzilla isn't turning it up at the moment). So I just created a new bug to track the development for 2.8.

The issue with the multiple kernel source providers is really about test cycles. The smaller devsrc is for on-target module development and builds against the exported uapi headers, and that is what the nightly / automated tests will use. We had issues with both the amount of time it took to package the entire source, and the amount of space that it took up on the images. Hence the creation of devsrc.

Quoted 4 lines

With new kernel-source and kernel-headers packages (the working names), they are really provided as references to the running kernel, and will largely be an exercise left up to the developer to use them as they want.

Right. So is there anything that holds us from creating two recipes - one for what we currently have and one for what we used to have pre-thud
- find some pretty names and start from that? I'm asking just in case I'm missing any internal architecture discussions or so.

And about test cycles, I'm not sure I understand why having a provider vs having two separate recipes impact automated tests. In both cases you will select whatever you want to test - by using a specific recipe or by setting a preferred provider (I don't have a strong opinion here but I was just curious about the time impact on automated tests).

Replying to the other email (this thread was forked). We have maintained a similar kernel headers approach for a couple of years now. And many times we found ourselves in the situation where something had to be added/removed/checked - that was either kernel fork specific or something that was added due to kernel updates. And this loop of fixes here and there was obviously completely resolved by just using the "old" kernel-devsrc which always had "everything" in. A good example is commit a9471601fedd1f5087304eaa5fd39b98ae220313 I'm mentioning above. This kind of stuff will always be needed - or at least in our experience was a pretty often loop (when you try to support one kernel headers recipe against tens of BSPs, you'll need cases, quirks and so on).

Regards,

Signature · 2 lines
Andrei Gherzan
gpg: rsa4096/D4D94F67AD0E9640 | t: @agherzan
#4 lore raw
Quoted 71 lines · Andrei Gherzan

On 23/05/2019 15.39, Bruce Ashfield wrote:

Quoted 45 lines · Bruce Ashfield
Quoted 31 lines · Andrei Gherzan

Hello,

This might have been discussed before. I couldn't find a relevant thread, but if it is so, just link me to it.

Since thud, more specific since

commit 9af0f1a46bbb6ad9ee8b35957251f4aa826b023f

Author: Bruce Ashfield <[email protected]>
Date:   Sat Aug 18 22:50:44 2018 -0400

kernel-devsrc: restructure for out of tree (and on target) module
builds

... we switched from a recipe that was deploying the entire source code to one that provides mainly the kernel headers (but not only). This change broke people expectations of this recipe while the description is also confusing: "Development source linux kernel. When built, this recipe packages the \source of the preferred virtual/kernel provider and makes it available for full kernel \development or external module builds".

If size is not a problem (which can be the case when you compile on a builder for example and deploy only a OOT kernel module through other means), the full kernel source was a painless experience where things like

commit a9471601fedd1f5087304eaa5fd39b98ae220313

Author: Bruce Ashfield <[email protected]>
Date:   Thu Aug 30 09:45:41 2018 -0400

kernel-devsrc: fix arm/arm64 target module build

Quoted 9 lines

... would not appear. I understand the size impact on target and for those cases, continuously maintaining this recipe with new files/resources needed from the kernel, makes sense. So my proposal is to have two recipes, for example kernel-devsrc and kernel-fullsrc (kernel-src etc.) so people can choose what they need/want deploying/using. Or even have another devsrc provider. I'm open to any implementation detail. I'd just want to have an option for a full kernel source recipe.

Quoted 4 lines

This is already planned, and hidden in bugzilla somewhere. I'll have some new kernel packaging
options available for the fall release.

It looks like the bugs that I was using for development were finally moved to resolved (they were a bit old, and contained collected information on various kernel packaging options .. my searching of bugzilla isn't turning it up at the moment). So I just created a new bug to track the development for 2.8.

The issue with the multiple kernel source providers is really about test cycles. The smaller devsrc is for on-target module development and builds against the exported uapi headers, and that is what the nightly / automated tests will use. We had issues with both the amount of time it took to package the entire source, and the amount of space that it took up on the images. Hence the creation of devsrc.

Quoted 9 lines

With new kernel-source and kernel-headers packages (the working names), they are really provided as references to the running kernel, and will largely be an exercise left up to the developer to use them as they want.

Right. So is there anything that holds us from creating two recipes - one for what we currently have and one for what we used to have pre-thud - find some pretty names and start from that? I'm asking just in case I'm missing any internal architecture discussions or so.

In some other non-core layer ? Sure :D The pre-thud copying of the kernel source was a disaster and needed to be killed with fire, as it was. It was all special cases and I commonly needed to fight with it. What you see in the current devsrc is not an invention from Yocto, it is based on the redhat spec files, and also (a bit) on some of the debian packaging of minimal source.

So no, I'd rather not restore the mess that was the pre-thud devsrc in any form.

Quoted 6 lines

And about test cycles, I'm not sure I understand why having a provider vs having two separate recipes impact automated tests. In both cases you will select whatever you want to test - by using a specific recipe or by setting a preferred provider (I don't have a strong opinion here but I was just curious about the time impact on automated tests).

If it goes into core, we would need to run a full set of tests against it to make sure that it supported development tasks. That would mean on target kernel module builds, as well as potential some new cases given that the full source is available (i.e. kernel rebuild). We already do that for kernel-devrsc, adding multiple providers of something similar means that we'd then have to test the multiple providers against all the architectures, across the actively supported reference kernels in any given release .. something always breaks, so then I'm on the hook for sorting through all the issues (devsrc was not bug free in its previous form). Given the amount of test cycles available, that's not necessarily a trivial thing to add. In the end, it would be up to Richard if thinks something like multiple provides are a good thing, and to commit the test resources/cycles to them.

Quoted 12 lines

Replying to the other email (this thread was forked). We have maintained a similar kernel headers approach for a couple of years now. And many times we found ourselves in the situation where something had to be added/removed/checked - that was either kernel fork specific or something that was added due to kernel updates. And this loop of fixes here and there was obviously completely resolved by just using the "old" kernel-devsrc which always had "everything" in. A good example is commit a9471601fedd1f5087304eaa5fd39b98ae220313 I'm mentioning above. This kind of stuff will always be needed - or at least in our experience was a pretty often loop (when you try to support one kernel headers recipe against tens of BSPs, you'll need cases, quirks and so on).

Those small tweaks really are trivial, I don't consider them to be a big load. The advantages of the smaller footprint package have outweighed any tuning we've done since things switched over.

Bruce

Quoted 7 lines

Regards,

--
Andrei Gherzan
gpg: rsa4096/D4D94F67AD0E9640 | t: @agherzan

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II
#5 lore raw
Quoted 104 lines · Bruce Ashfield

On Thu, May 23, 2019 at 11:00 AM Andrei Gherzan <[email protected] <mailto:[email protected]>> wrote:

    On 23/05/2019 15.39, Bruce Ashfield wrote:
Quoted 85 lines

On Thu, May 23, 2019 at 10:32 AM Bruce Ashfield <[email protected] <mailto:[email protected]>> wrote:

On Thu, May 23, 2019 at 9:56 AM Andrei Gherzan <[email protected] <mailto:[email protected]>> wrote:

Hello,

This might have been discussed before. I couldn't find a relevant
thread, but if it is so, just link me to it.

Since thud, more specific since

commit 9af0f1a46bbb6ad9ee8b35957251f4aa826b023f
Author: Bruce Ashfield <[email protected] <mailto:[email protected]>> Date:   Sat Aug 18 22:50:44 2018 -0400
    kernel-devsrc: restructure for out of tree (and on target) module builds

... we switched from a recipe that was deploying the entire source code
to one that provides mainly the kernel headers (but not only). This
change broke people expectations of this recipe while the description is
also confusing: "Development source linux kernel. When built, this
recipe packages the \source of the preferred virtual/kernel provider and
makes it available for full kernel \development or external module builds".

If size is not a problem (which can be the case when you compile on a
builder for example and deploy only a OOT kernel module through other
means), the full kernel source was a painless experience where things like

commit a9471601fedd1f5087304eaa5fd39b98ae220313
Author: Bruce Ashfield <[email protected] <mailto:[email protected]>> Date:   Thu Aug 30 09:45:41 2018 -0400
    kernel-devsrc: fix arm/arm64 target module build

... would not appear. I understand the size impact on target and for
those cases, continuously maintaining this recipe with new files/resources needed from the kernel, makes sense. So my proposal is
to have two recipes, for example kernel-devsrc and kernel-fullsrc
(kernel-src etc.) so people can choose what they need/want deploying/using. Or even have another devsrc provider. I'm open to any
implementation detail. I'd just want to have an option for a full kernel
source recipe.

This is already planned, and hidden in bugzilla somewhere. I'll have some new kernel packaging
options available for the fall release.

It looks like the bugs that I was using for development were finally moved to resolved (they were a bit old, and contained collected information on various kernel packaging options .. my searching of bugzilla isn't turning it up at the moment). So I just created a new bug to track the development for 2.8.

The issue with the multiple kernel source providers is really about test cycles. The smaller devsrc is for on-target module development and builds against the exported uapi headers, and that is what the nightly / automated tests will use. We had issues with both the amount of time it took to package the entire source, and the amount of space that it took up on the images. Hence the creation of devsrc.

Quoted 4 lines

With new kernel-source and kernel-headers packages (the working names), they are really provided as references to the running kernel, and will largely be an exercise left up to the developer to use them as they want.

Right. So is there anything that holds us from creating two recipes - one for what we currently have and one for what we used to have pre-thud - find some pretty names and start from that? I'm asking just in case I'm missing any internal architecture discussions or so.

Quoted 6 lines

In some other non-core layer ? Sure :D The pre-thud copying of the kernel source was a disaster and needed to be killed with fire, as it was. It was all special cases and I commonly needed to fight with it. What you see in the current devsrc is not an invention from Yocto, it is based on the redhat spec files, and also (a bit) on some of the debian packaging of minimal source.

I am aware of all the other distros having the similar approach (and they do this dance for disk space optimization) but I don't understand what were all the special cases you had to fight against? The recipe was barely touched in years. For example there is only one commit on it in 2017.

Quoted 13 lines

So no, I'd rather not restore the mess that was the pre-thud devsrc in any form.

And about test cycles, I'm not sure I understand why having a provider vs having two separate recipes impact automated tests. In both cases you will select whatever you want to test - by using a specific recipe or by setting a preferred provider (I don't have a strong opinion here but I was just curious about the time impact on automated tests).

Quoted 13 lines

If it goes into core, we would need to run a full set of tests against it to make sure that it supported development tasks. That would mean on target kernel module builds, as well as potential some new cases given that the full source is available (i.e. kernel rebuild). We already do that for kernel-devrsc, adding multiple providers of something similar means that we'd then have to test the multiple providers against all the architectures, across the actively supported reference kernels in any given release .. something always breaks, so then I'm on the hook for sorting through all the issues (devsrc was not bug free in its previous form).  Given the amount of test cycles available, that's not necessarily a trivial thing to add. In the end, it would be up to Richard if thinks something like multiple provides are a good thing, and to commit the test resources/cycles to them.

I see. You were comparing something else. That makes sense.

Quoted 17 lines

Replying to the other email (this thread was forked). We have maintained a similar kernel headers approach for a couple of years now. And many times we found ourselves in the situation where something had to be added/removed/checked - that was either kernel fork specific or something that was added due to kernel updates. And this loop of fixes here and there was obviously completely resolved by just using the "old" kernel-devsrc which always had "everything" in. A good example is commit a9471601fedd1f5087304eaa5fd39b98ae220313 I'm mentioning above. This kind of stuff will always be needed - or at least in our experience was a pretty often loop (when you try to support one kernel headers recipe against tens of BSPs, you'll need cases, quirks and so on).

Quoted 3 lines

Those small tweaks really are trivial, I don't consider them to be a big load. The advantages of the smaller footprint package have outweighed any tuning we've done since things switched over.

Again, I 100% understand the space advantage. I'm trying to see if there are any other advantages at all. In a world with infinite disk space :D

Signature · 2 lines
Andrei Gherzan
gpg: rsa4096/D4D94F67AD0E9640 | t: @agherzan
#6 lore raw
Quoted 95 lines · Andrei Gherzan

On 23/05/2019 16.10, Bruce Ashfield wrote:

Quoted 72 lines · Andrei Gherzan

On 23/05/2019 15.39, Bruce Ashfield wrote:

Quoted 46 lines · Bruce Ashfield
Quoted 31 lines · Andrei Gherzan

Hello,

This might have been discussed before. I couldn't find a relevant thread, but if it is so, just link me to it.

Since thud, more specific since

commit 9af0f1a46bbb6ad9ee8b35957251f4aa826b023f

Author: Bruce Ashfield <[email protected]>
Date:   Sat Aug 18 22:50:44 2018 -0400

kernel-devsrc: restructure for out of tree (and on target) module
builds

... we switched from a recipe that was deploying the entire source code to one that provides mainly the kernel headers (but not only). This change broke people expectations of this recipe while the description is also confusing: "Development source linux kernel. When built, this recipe packages the \source of the preferred virtual/kernel provider and makes it available for full kernel \development or external module builds".

If size is not a problem (which can be the case when you compile on a builder for example and deploy only a OOT kernel module through other means), the full kernel source was a painless experience where things like

commit a9471601fedd1f5087304eaa5fd39b98ae220313

Author: Bruce Ashfield <[email protected]>
Date:   Thu Aug 30 09:45:41 2018 -0400

kernel-devsrc: fix arm/arm64 target module build

Quoted 9 lines

... would not appear. I understand the size impact on target and for those cases, continuously maintaining this recipe with new files/resources needed from the kernel, makes sense. So my proposal is to have two recipes, for example kernel-devsrc and kernel-fullsrc (kernel-src etc.) so people can choose what they need/want deploying/using. Or even have another devsrc provider. I'm open to any implementation detail. I'd just want to have an option for a full kernel source recipe.

Quoted 4 lines

This is already planned, and hidden in bugzilla somewhere. I'll have some new kernel packaging
options available for the fall release.

It looks like the bugs that I was using for development were finally moved to resolved (they were a bit old, and contained collected information on various kernel packaging options .. my searching of bugzilla isn't turning it up at the moment). So I just created a new bug to track the development for 2.8.

The issue with the multiple kernel source providers is really about test cycles. The smaller devsrc is for on-target module development and builds against the exported uapi headers, and that is what the nightly / automated tests will use. We had issues with both the amount of time it took to package the entire source, and the amount of space that it took up on the images. Hence the creation of devsrc.

Quoted 9 lines

With new kernel-source and kernel-headers packages (the working names), they are really provided as references to the running kernel, and will largely be an exercise left up to the developer to use them as they want.

Right. So is there anything that holds us from creating two recipes - one for what we currently have and one for what we used to have pre-thud - find some pretty names and start from that? I'm asking just in case I'm missing any internal architecture discussions or so.

In some other non-core layer ? Sure :D The pre-thud copying of the kernel source was a disaster and needed to be killed with fire, as it was. It was all special cases and I commonly needed to fight with it. What you see in the current devsrc is not an invention from Yocto, it is based on the redhat spec files, and also (a bit) on some of the debian packaging of minimal source.

Quoted 5 lines

I am aware of all the other distros having the similar approach (and they do this dance for disk space optimization) but I don't understand what were all the special cases you had to fight against? The recipe was barely touched in years. For example there is only one commit on it in 2017.

There were corner cases popping up with architectures and the newer kernels. I had inflight changes that were tweaking the nasty find commands, etc, trying to get everything we needed, minimize the time spent copying and avoiding QA errors due to cross arch artifacts popping in. Let's just say, I'm not copying and pruning the same way in a plainer "kernel-source" package. Since it isn't installed by default, and is opt-in, it doesn't have to be quite so careful on those fronts.

Quoted 44 lines

So no, I'd rather not restore the mess that was the pre-thud devsrc in any form.

Quoted 6 lines

And about test cycles, I'm not sure I understand why having a provider vs having two separate recipes impact automated tests. In both cases you will select whatever you want to test - by using a specific recipe or by setting a preferred provider (I don't have a strong opinion here but I was just curious about the time impact on automated tests).

If it goes into core, we would need to run a full set of tests against it to make sure that it supported development tasks. That would mean on target kernel module builds, as well as potential some new cases given that the full source is available (i.e. kernel rebuild). We already do that for kernel-devrsc, adding multiple providers of something similar means that we'd then have to test the multiple providers against all the architectures, across the actively supported reference kernels in any given release .. something always breaks, so then I'm on the hook for sorting through all the issues (devsrc was not bug free in its previous form). Given the amount of test cycles available, that's not necessarily a trivial thing to add. In the end, it would be up to Richard if thinks something like multiple provides are a good thing, and to commit the test resources/cycles to them.

I see. You were comparing something else. That makes sense.

Quoted 12 lines

Replying to the other email (this thread was forked). We have maintained a similar kernel headers approach for a couple of years now. And many times we found ourselves in the situation where something had to be added/removed/checked - that was either kernel fork specific or something that was added due to kernel updates. And this loop of fixes here and there was obviously completely resolved by just using the "old" kernel-devsrc which always had "everything" in. A good example is commit a9471601fedd1f5087304eaa5fd39b98ae220313 I'm mentioning above. This kind of stuff will always be needed - or at least in our experience was a pretty often loop (when you try to support one kernel headers recipe against tens of BSPs, you'll need cases, quirks and so on).

Quoted 7 lines

Those small tweaks really are trivial, I don't consider them to be a big load. The advantages of the smaller footprint package have outweighed any tuning we've done since things switched over.

Again, I 100% understand the space advantage. I'm trying to see if there are any other advantages at all. In a world with infinite disk space :D

Nothing super dramatic :D The curated / whitelist approach does make sure that nothing new slips into the package until we've tested and ok'd it (with the downsides you are pointing out) .. so we have an easier route to patch / workaround issues as they pop up (I remember before uapi and hand sanitizing the kernel headers .. and wouldn't want to be that extreme ever again).

But I completely agree, having a simple, full copy of the source for situations where build/packaging time or diskspace are not a requirement is a good thing. I'll sort through the last symlink / provider issues and get something out ASAP.

Bruce

Quoted 5 lines

--
Andrei Gherzan
gpg: rsa4096/D4D94F67AD0E9640 | t: @agherzan

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II
#7 lore raw
Quoted 200 lines · Bruce Ashfield

On Thu, May 23, 2019 at 11:24 AM Andrei Gherzan <[email protected] <mailto:[email protected]>> wrote:

    On 23/05/2019 16.10, Bruce Ashfield wrote:
Quoted 108 lines

    On Thu, May 23, 2019 at 11:00 AM Andrei Gherzan
    <[email protected] <mailto:[email protected]>> wrote:


        On 23/05/2019 15.39, Bruce Ashfield wrote:
Quoted 89 lines

On Thu, May 23, 2019 at 10:32 AM Bruce Ashfield <[email protected] <mailto:[email protected]>> wrote:

On Thu, May 23, 2019 at 9:56 AM Andrei Gherzan <[email protected] <mailto:[email protected]>> wrote:

Hello,

This might have been discussed before. I couldn't find a relevant
thread, but if it is so, just link me to it.

Since thud, more specific since

commit 9af0f1a46bbb6ad9ee8b35957251f4aa826b023f
Author: Bruce Ashfield <[email protected] <mailto:[email protected]>> Date:   Sat Aug 18 22:50:44 2018 -0400
    kernel-devsrc: restructure for out of tree (and on target) module builds

... we switched from a recipe that was deploying the entire source code
to one that provides mainly the kernel headers (but not only). This
change broke people expectations of this recipe while the description is
also confusing: "Development source linux kernel. When built, this
recipe packages the \source of the preferred virtual/kernel provider and
makes it available for full kernel \development or external module builds".

If size is not a problem (which can be the case when you compile on a
builder for example and deploy only a OOT kernel module through other
means), the full kernel source was a painless experience where things like

commit a9471601fedd1f5087304eaa5fd39b98ae220313
Author: Bruce Ashfield <[email protected] <mailto:[email protected]>> Date:   Thu Aug 30 09:45:41 2018 -0400
    kernel-devsrc: fix arm/arm64 target module build

... would not appear. I understand the size impact on target and for
those cases, continuously maintaining this recipe with new
files/resources needed from the kernel, makes sense. So my proposal is
to have two recipes, for example kernel-devsrc and kernel-fullsrc
(kernel-src etc.) so people can choose what they need/want
deploying/using. Or even have another devsrc provider. I'm open to any
implementation detail. I'd just want to have an option for a full kernel
source recipe.

This is already planned, and hidden in bugzilla somewhere. I'll have some new kernel packaging options available for the fall release.

It looks like the bugs that I was using for development were finally moved to resolved (they were a bit old, and contained collected information on various kernel packaging options .. my searching of bugzilla isn't turning it up at the moment). So I just created a new bug to track the development for 2.8.

The issue with the multiple kernel source providers is really about test cycles. The smaller devsrc is for on-target module development and builds against the exported uapi headers, and that is what the nightly / automated tests will use. We had issues with both the amount of time it took to package the entire source, and the amount of space that it took up on the images. Hence the creation of devsrc.

Quoted 4 lines

With new kernel-source and kernel-headers packages (the working names), they are really provided as references to the running kernel, and will largely be an exercise left up to the developer to use them as they want.

Right. So is there anything that holds us from creating two recipes - one for what we currently have and one for what we used to have pre-thud - find some pretty names and start from that? I'm asking just in case I'm missing any internal architecture discussions or so.

Quoted 6 lines

In some other non-core layer ? Sure :D The pre-thud copying of the kernel source was a disaster and needed to be killed with fire, as it was. It was all special cases and I commonly needed to fight with it. What you see in the current devsrc is not an invention from Yocto, it is based on the redhat spec files, and also (a bit) on some of the debian packaging of minimal source.

I am aware of all the other distros having the similar approach (and they do this dance for disk space optimization) but I don't understand what were all the special cases you had to fight against? The recipe was barely touched in years. For example there is only one commit on it in 2017.

There were corner cases popping up with architectures and the newer kernels. I had inflight changes that were tweaking the nasty find commands, etc, trying to get everything we needed, minimize the time spent copying and avoiding QA errors due to cross arch artifacts popping in. Let's just say, I'm not copying and pruning the same way in a plainer "kernel-source" package. Since it isn't installed by default, and is opt-in, it doesn't have to be quite so careful on those fronts.

Quoted 13 lines

So no, I'd rather not restore the mess that was the pre-thud devsrc in any form.

And about test cycles, I'm not sure I understand why having a provider vs having two separate recipes impact automated tests. In both cases you will select whatever you want to test - by using a specific recipe or by setting a preferred provider (I don't have a strong opinion here but I was just curious about the time impact on automated tests).

Quoted 14 lines

If it goes into core, we would need to run a full set of tests against it to make sure that it supported development tasks. That would mean on target kernel module builds, as well as potential some new cases given that the full source is available (i.e. kernel rebuild). We already do that for kernel-devrsc, adding multiple providers of something similar means that we'd then have to test the multiple providers against all the architectures, across the actively supported reference kernels in any given release .. something always breaks, so then I'm on the hook for sorting through all the issues (devsrc was not bug free in its previous form).  Given the amount of test cycles available, that's not necessarily a trivial thing to add. In the end, it would be up to Richard if thinks something like multiple provides are a good thing, and to commit the test resources/cycles to them.

I see. You were comparing something else. That makes sense.

Quoted 18 lines

Replying to the other email (this thread was forked). We have maintained a similar kernel headers approach for a couple of years now. And many times we found ourselves in the situation where something had to be added/removed/checked - that was either kernel fork specific or something that was added due to kernel updates. And this loop of fixes here and there was obviously completely resolved by just using the "old" kernel-devsrc which always had "everything" in. A good example is commit a9471601fedd1f5087304eaa5fd39b98ae220313 I'm mentioning above. This kind of stuff will always be needed - or at least in our experience was a pretty often loop (when you try to support one kernel headers recipe against tens of BSPs, you'll need cases, quirks and so on).

Quoted 3 lines

Those small tweaks really are trivial, I don't consider them to be a big load. The advantages of the smaller footprint package have outweighed any tuning we've done since things switched over.

Again, I 100% understand the space advantage. I'm trying to see if there are any other advantages at all. In a world with infinite disk space :D

Nothing super dramatic :D The curated / whitelist approach does make sure that nothing new slips into the package until we've tested and ok'd it (with the downsides you are pointing out) .. so we have an easier route to patch / workaround issues as they pop up (I remember before uapi and hand sanitizing the kernel headers .. and wouldn't want to be that extreme ever again).

Quoted 4 lines

But I completely agree, having a simple, full copy of the source for situations where build/packaging time or diskspace are not a requirement is a good thing. I'll sort through the last symlink / provider issues and get something out ASAP.

Thanks Bruce. Would you please CC me when you push that? Just to make sure I don't miss it.

Signature · 2 lines
Andrei Gherzan
gpg: rsa4096/D4D94F67AD0E9640 | t: @agherzan
#8 lore raw
Quoted 181 lines · Andrei Gherzan

On 23/05/2019 17.11, Bruce Ashfield wrote:

Quoted 99 lines · Andrei Gherzan

On 23/05/2019 16.10, Bruce Ashfield wrote:

Quoted 75 lines · Andrei Gherzan

On 23/05/2019 15.39, Bruce Ashfield wrote:

Quoted 49 lines · Bruce Ashfield
Quoted 33 lines · Andrei Gherzan

Hello,

This might have been discussed before. I couldn't find a relevant thread, but if it is so, just link me to it.

Since thud, more specific since

commit 9af0f1a46bbb6ad9ee8b35957251f4aa826b023f

Author: Bruce Ashfield <[email protected]>
Date:   Sat Aug 18 22:50:44 2018 -0400

kernel-devsrc: restructure for out of tree (and on target) module
builds

... we switched from a recipe that was deploying the entire source code to one that provides mainly the kernel headers (but not only). This change broke people expectations of this recipe while the description is
also confusing: "Development source linux kernel. When built, this recipe packages the \source of the preferred virtual/kernel provider and
makes it available for full kernel \development or external module builds".

If size is not a problem (which can be the case when you compile on a builder for example and deploy only a OOT kernel module through other means), the full kernel source was a painless experience where things like

commit a9471601fedd1f5087304eaa5fd39b98ae220313

Author: Bruce Ashfield <[email protected]>
Date:   Thu Aug 30 09:45:41 2018 -0400

kernel-devsrc: fix arm/arm64 target module build

Quoted 10 lines

... would not appear. I understand the size impact on target and for those cases, continuously maintaining this recipe with new files/resources needed from the kernel, makes sense. So my proposal is to have two recipes, for example kernel-devsrc and kernel-fullsrc (kernel-src etc.) so people can choose what they need/want deploying/using. Or even have another devsrc provider. I'm open to any implementation detail. I'd just want to have an option for a full kernel
source recipe.

Quoted 4 lines

This is already planned, and hidden in bugzilla somewhere. I'll have some new kernel packaging
options available for the fall release.

It looks like the bugs that I was using for development were finally moved to resolved (they were a bit old, and contained collected information on various kernel packaging options .. my searching of bugzilla isn't turning it up at the moment). So I just created a new bug to track the development for 2.8.

The issue with the multiple kernel source providers is really about test cycles. The smaller devsrc is for on-target module development and builds against the exported uapi headers, and that is what the nightly / automated tests will use. We had issues with both the amount of time it took to package the entire source, and the amount of space that it took up on the images. Hence the creation of devsrc.

Quoted 9 lines

With new kernel-source and kernel-headers packages (the working names), they are really provided as references to the running kernel, and will largely be an exercise left up to the developer to use them as they want.

Right. So is there anything that holds us from creating two recipes - one for what we currently have and one for what we used to have pre-thud - find some pretty names and start from that? I'm asking just in case I'm missing any internal architecture discussions or so.

In some other non-core layer ? Sure :D The pre-thud copying of the kernel source was a disaster and needed to be killed with fire, as it was. It was all special cases and I commonly needed to fight with it. What you see in the current devsrc is not an invention from Yocto, it is based on the redhat spec files, and also (a bit) on some of the debian packaging of minimal source.

Quoted 5 lines

I am aware of all the other distros having the similar approach (and they do this dance for disk space optimization) but I don't understand what were all the special cases you had to fight against? The recipe was barely touched in years. For example there is only one commit on it in 2017.

There were corner cases popping up with architectures and the newer kernels. I had inflight changes that were tweaking the nasty find commands, etc, trying to get everything we needed, minimize the time spent copying and avoiding QA errors due to cross arch artifacts popping in. Let's just say, I'm not copying and pruning the same way in a plainer "kernel-source" package. Since it isn't installed by default, and is opt-in, it doesn't have to be quite so careful on those fronts.

Quoted 44 lines

So no, I'd rather not restore the mess that was the pre-thud devsrc in any form.

Quoted 6 lines

And about test cycles, I'm not sure I understand why having a provider vs having two separate recipes impact automated tests. In both cases you will select whatever you want to test - by using a specific recipe or by setting a preferred provider (I don't have a strong opinion here but I was just curious about the time impact on automated tests).

If it goes into core, we would need to run a full set of tests against it to make sure that it supported development tasks. That would mean on target kernel module builds, as well as potential some new cases given that the full source is available (i.e. kernel rebuild). We already do that for kernel-devrsc, adding multiple providers of something similar means that we'd then have to test the multiple providers against all the architectures, across the actively supported reference kernels in any given release .. something always breaks, so then I'm on the hook for sorting through all the issues (devsrc was not bug free in its previous form). Given the amount of test cycles available, that's not necessarily a trivial thing to add. In the end, it would be up to Richard if thinks something like multiple provides are a good thing, and to commit the test resources/cycles to them.

I see. You were comparing something else. That makes sense.

Quoted 12 lines

Replying to the other email (this thread was forked). We have maintained a similar kernel headers approach for a couple of years now. And many times we found ourselves in the situation where something had to be added/removed/checked - that was either kernel fork specific or something that was added due to kernel updates. And this loop of fixes here and there was obviously completely resolved by just using the "old" kernel-devsrc which always had "everything" in. A good example is commit a9471601fedd1f5087304eaa5fd39b98ae220313 I'm mentioning above. This kind of stuff will always be needed - or at least in our experience was a pretty often loop (when you try to support one kernel headers recipe against tens of BSPs, you'll need cases, quirks and so on).

Quoted 7 lines

Those small tweaks really are trivial, I don't consider them to be a big load. The advantages of the smaller footprint package have outweighed any tuning we've done since things switched over.

Again, I 100% understand the space advantage. I'm trying to see if there are any other advantages at all. In a world with infinite disk space :D

Nothing super dramatic :D The curated / whitelist approach does make sure that nothing new slips into the package until we've tested and ok'd it (with the downsides you are pointing out) .. so we have an easier route to patch / workaround issues as they pop up (I remember before uapi and hand sanitizing the kernel headers .. and wouldn't want to be that extreme ever again).

Quoted 8 lines

But I completely agree, having a simple, full copy of the source for situations where build/packaging time or diskspace are not a requirement is a good thing. I'll sort through the last symlink / provider issues and get something out ASAP.

Thanks Bruce. Would you please CC me when you push that? Just to make sure I don't miss it.

Absolutely. I'll make sure you are copied.

Bruce

Quoted 5 lines

--
Andrei Gherzan
gpg: rsa4096/D4D94F67AD0E9640 | t: @agherzan

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II
#9 lore raw
In reply to Bruce Ashfield
Quoted 52 lines · Bruce Ashfield

On Thu, May 23, 2019 at 9:56 AM Andrei Gherzan <[email protected] <mailto:[email protected]>> wrote:

Hello,

This might have been discussed before. I couldn't find a relevant thread, but if it is so, just link me to it.

Since thud, more specific since

commit 9af0f1a46bbb6ad9ee8b35957251f4aa826b023f
Author: Bruce Ashfield <[email protected] <mailto:[email protected]>>
Date:   Sat Aug 18 22:50:44 2018 -0400
    kernel-devsrc: restructure for out of tree (and on target) module builds

... we switched from a recipe that was deploying the entire source code
to one that provides mainly the kernel headers (but not only). This change broke people expectations of this recipe while the description is
also confusing: "Development source linux kernel. When built, this recipe packages the \source of the preferred virtual/kernel provider and
makes it available for full kernel \development or external module builds".

If size is not a problem (which can be the case when you compile on a builder for example and deploy only a OOT kernel module through other means), the full kernel source was a painless experience where things like

commit a9471601fedd1f5087304eaa5fd39b98ae220313
Author: Bruce Ashfield <[email protected] <mailto:[email protected]>> Date:   Thu Aug 30 09:45:41 2018 -0400
    kernel-devsrc: fix arm/arm64 target module build

... would not appear. I understand the size impact on target and for those cases, continuously maintaining this recipe with new files/resources needed from the kernel, makes sense. So my proposal is to have two recipes, for example kernel-devsrc and kernel-fullsrc (kernel-src etc.) so people can choose what they need/want deploying/using. Or even have another devsrc provider. I'm open to any implementation detail. I'd just want to have an option for a full kernel
source recipe.

Quoted 3 lines

This is already planned, and hidden in bugzilla somewhere. I'll have some new kernel packaging
options available for the fall release.

Nice to hear that. Can you share more details on that because we plan to implement it for BalenaOS and would be a waste to redo what you already have or do it in different ways and reset afterwards.

Signature · 2 lines
Andrei Gherzan
gpg: rsa4096/D4D94F67AD0E9640 | t: @agherzan
#10 lore raw
Quoted 46 lines · Andrei Gherzan

On 23/05/2019 15.32, Bruce Ashfield wrote:

Quoted 30 lines · Andrei Gherzan

Hello,

This might have been discussed before. I couldn't find a relevant thread, but if it is so, just link me to it.

Since thud, more specific since

commit 9af0f1a46bbb6ad9ee8b35957251f4aa826b023f

Author: Bruce Ashfield <[email protected]>
Date:   Sat Aug 18 22:50:44 2018 -0400

kernel-devsrc: restructure for out of tree (and on target) module
builds

... we switched from a recipe that was deploying the entire source code to one that provides mainly the kernel headers (but not only). This change broke people expectations of this recipe while the description is also confusing: "Development source linux kernel. When built, this recipe packages the \source of the preferred virtual/kernel provider and makes it available for full kernel \development or external module builds".

If size is not a problem (which can be the case when you compile on a builder for example and deploy only a OOT kernel module through other means), the full kernel source was a painless experience where things like

commit a9471601fedd1f5087304eaa5fd39b98ae220313

Author: Bruce Ashfield <[email protected]>
Date:   Thu Aug 30 09:45:41 2018 -0400

kernel-devsrc: fix arm/arm64 target module build

Quoted 9 lines

... would not appear. I understand the size impact on target and for those cases, continuously maintaining this recipe with new files/resources needed from the kernel, makes sense. So my proposal is to have two recipes, for example kernel-devsrc and kernel-fullsrc (kernel-src etc.) so people can choose what they need/want deploying/using. Or even have another devsrc provider. I'm open to any implementation detail. I'd just want to have an option for a full kernel source recipe.

Quoted 8 lines

This is already planned, and hidden in bugzilla somewhere. I'll have some new kernel packaging
options available for the fall release.

Nice to hear that. Can you share more details on that because we plan to implement it for BalenaOS and would be a waste to redo what you already have or do it in different ways and reset afterwards.

The best place for that is in the bugzilla, I (re)created bug 13360 and will start moving the details into that (they are scattered around).

Can you elaborate on the use case that is broken with devsrc ? I'd also like to make changes to that package to support more common workflows if we've missed them. Feel free to add them to the bug, that way I won't lose track of them again.

Bruce

Quoted 5 lines

--
Andrei Gherzan
gpg: rsa4096/D4D94F67AD0E9640 | t: @agherzan

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II