Skip to contentSkip to conversation

Last minute changes - Review Request

DISCUSSION Richard Purdie 6 participants 7 messages Oct 22, 2010

lore
Share & tools
lore
#0 lore raw

Coming up to release there are a few things that the extended testing has shown up which we have fixes for and which we should consider including in the release. I also finally got around to doing the final sstate stress testing and found several problematic issues. Given that sstate and checksums are a significant feature of this release, I'd really like them to work as well as we can make them. Prior to this I had stress tested the backend up not the use of the packages. These changes don't change any sstate packages themselves, just the use of them.

Since we already have the release images prepared and tested and these are not going to change, the criteria for potential changes:

a) We can unit test the changes and be confident they don't break/regress things.
b) They fix important bugs that the user can easily run into or that make the project look bad.
c) The changes are small, well documented and are obviously correct looking at the code/patch.
d) The don't change the generated images.

I'm proposing the following, for each I've provided a rationale:

http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=d5504a4275d94868e28b00c272411e82f4999d95

Printing "fatal:" to new users is worrying

[Reported by Dirk]

http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=2a69c58046a86d0f783acebd8a77e9419b43139a

Users can easily hit the sanity warning about missing 32 bit libs. We don't need this functionality at this time so we might as well turn it off by default

[Reported by Dirk]

http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=0068e55d8f64ae13a1049c37164e8b14dc33f53f

Doesn't change the build output but fixes a build issue people can easily run into in from scratch builds due to a missing dependency.

[Reported by Dexuan]

http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=6e277cb014a53aef66ae931b5142495f8a02404f

Removes the "WARNING: Function do_build doesn't exist" message which could worry users and looks bad

[Reported by Richard]

http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=fd4457199ef604dc4d5f8346c8b2a09dc3939129

Several complaints have been received about the inability to easily delete sstate and DL_DIR contents to recover from failures. This adds a cleanall task which does this. Downside is that this is undocumented.

[Reported by Darren and others, run into by Dirk]

http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=f806c499c031fe0c4da001d41bce635088a90c52

Fixes an sstate bug where the do_package sstate packages don't install correctly. A user could hit this if cleaning and rebuilding a package under certain circumstances.

[Reported by Richard]

http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=13f116b1ad6a955b07d4cbaba85879913c30e1ee

Fixes an issue with sstate where rebuilding a package using partially prebuilt state would break rpm generation and silently remove all the rpm packages.

[Reported by Richard]

http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=00a96a20995cefacc52e10559029de32941ecf6e

Fixes a typo spotted in the debian packaging backend. We don't use this by default but it should be fixed.

[Reported by Richard]

http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=36f1ae42fe13dae174b7fb5eb85dc49d7d7b516b

User testing keeps showing up issues with the pseudo directory creation failing to happen. This patch solves the problem in a brute force fashion, once and forall.

[Reported by Mark and others]

http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=3f599b3f6a47286277cdaa8503f8a8da024eadd4

Fixes an sstate issue seen on the Poky mailing list where file:// sstate mirrors wouldn't work.

[Reported by Gary Thomas (poky list)]

http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=535a77a9b681423e2f10744aca54858c25a03cb0

Just changes a log level to make the output from sstate slightly more readable.

[Reported by Richard]

I'm not happy about being in this position and I know Dave will be very nervous about these late changes. To mitigate this I'd like to propose that a selection of people (Josh, Mark, Saul?) review these changes and report back on whether they feel these are appropriate and also give the build some testing with these applied.

Cheers,

Richard

#1 lore raw
Quoted 117 lines

From: [email protected] [mailto:yocto- [email protected]] On Behalf Of Richard Purdie
Sent: Friday, October 22, 2010 7:24 AM

Coming up to release there are a few things that the extended testing has shown up which we have fixes for and which we should consider including in the release. I also finally got around to doing the final sstate stress testing and found several problematic issues. Given that sstate and checksums are a significant feature of this release, I'd really like them to work as well as we can make them. Prior to this I had stress tested the backend up not the use of the packages. These changes don't change any sstate packages themselves, just the use of them.

Since we already have the release images prepared and tested and these are not going to change, the criteria for potential changes:

a) We can unit test the changes and be confident they don't break/regress things.
b) They fix important bugs that the user can easily run into or that make the project look bad.
c) The changes are small, well documented and are obviously correct looking at the code/patch.
d) The don't change the generated images.

I'm proposing the following, for each I've provided a rationale:

http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=d5504a4275d94868e
28b00c272411e82f4999d95

Printing "fatal:" to new users is worrying

[Reported by Dirk]

http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=2a69c58046a86d0f7
83acebd8a77e9419b43139a

Users can easily hit the sanity warning about missing 32 bit libs. We don't need this functionality at this time so we might as well turn it off by default

[Reported by Dirk]

http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=0068e55d8f64ae13a
1049c37164e8b14dc33f53f

Doesn't change the build output but fixes a build issue people can easily run into in from scratch builds due to a missing dependency.

[Reported by Dexuan]

http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=6e277cb014a53aef6
6ae931b5142495f8a02404f

Removes the "WARNING: Function do_build doesn't exist" message which could worry users and looks bad

[Reported by Richard]

http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=fd4457199ef604dc4
d5f8346c8b2a09dc3939129

Several complaints have been received about the inability to easily delete sstate and DL_DIR contents to recover from failures. This adds a cleanall task which does this. Downside is that this is undocumented.

[Reported by Darren and others, run into by Dirk]

http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=f806c499c031fe0c4
da001d41bce635088a90c52

Fixes an sstate bug where the do_package sstate packages don't install correctly. A user could hit this if cleaning and rebuilding a package under certain circumstances.

[Reported by Richard]

http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=13f116b1ad6a955b0
7d4cbaba85879913c30e1ee

Fixes an issue with sstate where rebuilding a package using partially prebuilt state would break rpm generation and silently remove all the rpm packages.

[Reported by Richard]

http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=00a96a20995cefacc
52e10559029de32941ecf6e

Fixes a typo spotted in the debian packaging backend. We don't use this by default but it should be fixed.

[Reported by Richard]

http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=36f1ae42fe13dae17
4b7fb5eb85dc49d7d7b516b

User testing keeps showing up issues with the pseudo directory creation failing to happen. This patch solves the problem in a brute force fashion, once and forall.

[Reported by Mark and others]

http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=3f599b3f6a4728627
7cdaa8503f8a8da024eadd4

Fixes an sstate issue seen on the Poky mailing list where file:// sstate mirrors wouldn't work.

[Reported by Gary Thomas (poky list)]

http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=535a77a9b681423e2
f10744aca54858c25a03cb0

Just changes a log level to make the output from sstate slightly more readable.

Quoted 8 lines

[Reported by Richard]

I'm not happy about being in this position and I know Dave will be very nervous about these late changes. To mitigate this I'd like to propose that a selection of people (Josh, Mark, Saul?) review these changes and report back on whether they feel these are appropriate and also give the build some testing with these applied.

I'm so predictable... :-) Yes, I'm nervous. I looked at all of the patches and with the exception of one or two, they mostly seem like good ones. I will accept these if Josh/Mark/Saul give us a +1 on their review & testing.

Quoted 16 lines

Cheers,

Richard

_______________________________________________
yocto mailing list
[email protected]
https://lists.pokylinux.org/listinfo/yocto

#2 lore raw
Quoted 14 lines · David C
Quoted 14 lines

From: [email protected] [mailto:yocto- [email protected]] On Behalf Of Richard Purdie
Sent: Friday, October 22, 2010 7:24 AM

Coming up to release there are a few things that the extended testing has shown up which we have fixes for and which we should consider including in the release. I also finally got around to doing the final sstate stress testing and found several problematic issues. Given that sstate and checksums are a significant feature of this release, I'd really like them to work as well as we can make them. Prior to this I had stress tested the backend up not the use of the packages. These changes don't change any sstate packages themselves, just the use of them.

Quoted 5 lines
Quoted 5 lines

Since we already have the release images prepared and tested and these are not going to change, the criteria for potential changes:

a) We can unit test the changes and be confident they don't break/regress things.

For the Future: Besides doing a basic build, we need to have some real unit tests for bitbake and the poky infrastructure, I guess I need to turn this into a Testing feature request for 1.0 (look for it soon).

Quoted 2 lines
Quoted 2 lines

b) They fix important bugs that the user can easily run into or that make the project look bad.

After reviewing the changes I agree, don't get me wrong, I am still very nervous about these changes.

Quoted 2 lines
Quoted 2 lines

c) The changes are small, well documented and are obviously correct looking at the code/patch.

Some times we over look the obvious changes, been caught by that myself too many time.

Quoted 1 line
Quoted 1 line

d) The don't change the generated images.

<SNIP>

Quoted 8 lines
Quoted 5 lines

I'm not happy about being in this position and I know Dave will be very nervous about these late changes. To mitigate this I'd like to propose that a selection of people (Josh, Mark, Saul?) review these changes and report back on whether they feel these are appropriate and also give the build some testing with these applied.

I'm so predictable... :-) Yes, I'm nervous. I looked at all of the patches and with the exception of one or two, they mostly seem like good ones. I will accept these if Josh/Mark/Saul give us a +1 on their review& testing.

If there was 1 or 2 changes, I would be much happier, but there are almost a dozen changes, yes mostly individually they are OK, I am still reviewing them all, and have not started any testing with them yet.

I agree with Dave that there are a couple that I am more nervous about the pseudo/fakeroot as we have had so much trouble in the past, yes I know this will make things better, but what else will crop up?

Quoted 20 lines
Quoted 12 lines

Cheers,

Richard

Quoted 4 lines

_______________________________________________
yocto mailing list
[email protected]
https://lists.pokylinux.org/listinfo/yocto

_______________________________________________
yocto mailing list
[email protected]
https://lists.pokylinux.org/listinfo/yocto

#3 lore raw

Add a +1 to reviewed, worried, but accepting column. They each seem reasonable, low-enough risk..

--Mark

Quoted 81 lines · Saul G. Wold
Quoted 14 lines · David C
Quoted 14 lines

From: [email protected] [mailto:yocto- [email protected]] On Behalf Of Richard Purdie
Sent: Friday, October 22, 2010 7:24 AM

Coming up to release there are a few things that the extended testing has shown up which we have fixes for and which we should consider including in the release. I also finally got around to doing the final sstate stress testing and found several problematic issues. Given that sstate and checksums are a significant feature of this release, I'd really like them to work as well as we can make them. Prior to this I had stress tested the backend up not the use of the packages. These changes don't change any sstate packages themselves, just the use of them.

Quoted 5 lines
Quoted 5 lines

Since we already have the release images prepared and tested and these are not going to change, the criteria for potential changes:

a) We can unit test the changes and be confident they don't break/regress things.

For the Future: Besides doing a basic build, we need to have some real unit tests for bitbake and the poky infrastructure, I guess I need to turn this into a Testing feature request for 1.0 (look for it soon).

Quoted 2 lines
Quoted 2 lines

b) They fix important bugs that the user can easily run into or that make the project look bad.

After reviewing the changes I agree, don't get me wrong, I am still very nervous about these changes.

Quoted 2 lines
Quoted 2 lines

c) The changes are small, well documented and are obviously correct looking at the code/patch.

Some times we over look the obvious changes, been caught by that myself too many time.

Quoted 1 line
Quoted 1 line

d) The don't change the generated images.

<SNIP>

Quoted 8 lines
Quoted 5 lines

I'm not happy about being in this position and I know Dave will be very nervous about these late changes. To mitigate this I'd like to propose that a selection of people (Josh, Mark, Saul?) review these changes and report back on whether they feel these are appropriate and also give the build some testing with these applied.

I'm so predictable... :-) Yes, I'm nervous. I looked at all of the patches and with the exception of one or two, they mostly seem like good ones. I will accept these if Josh/Mark/Saul give us a +1 on their review& testing.

If there was 1 or 2 changes, I would be much happier, but there are almost a dozen changes, yes mostly individually they are OK, I am still reviewing them all, and have not started any testing with them yet.

I agree with Dave that there are a couple that I am more nervous about the pseudo/fakeroot as we have had so much trouble in the past, yes I know this will make things better, but what else will crop up?

Quoted 12 lines
Quoted 12 lines

Cheers,

Richard

Quoted 8 lines
Quoted 4 lines

_______________________________________________
yocto mailing list
[email protected]
https://lists.pokylinux.org/listinfo/yocto

_______________________________________________
yocto mailing list
[email protected]
https://lists.pokylinux.org/listinfo/yocto

_______________________________________________
yocto mailing list
[email protected]
https://lists.pokylinux.org/listinfo/yocto

#4 lore raw
Quoted 2 lines · Mark Hatle

Add a +1 to reviewed, worried, but accepting column. They each seem reasonable, low-enough risk..

same here

We won't have a bug free release. No one ever does. And any change increases the risk.

So the question is "are they worth the added risk?". I believe the proposed changes address issues that people WILL run into as they start playing with things, so I think the risk is worth the reward.

/D

Quoted 87 lines · Saul G. Wold
Quoted 73 lines · David C
Quoted 14 lines
Quoted 14 lines

From: [email protected] [mailto:yocto- [email protected]] On Behalf Of Richard Purdie
Sent: Friday, October 22, 2010 7:24 AM

Coming up to release there are a few things that the extended testing has shown up which we have fixes for and which we should consider including in the release. I also finally got around to doing the final sstate stress testing and found several problematic issues. Given that sstate and checksums are a significant feature of this release, I'd really like them to work as well as we can make them. Prior to this I had stress tested the backend up not the use of the packages. These changes don't change any sstate packages themselves, just the use of them.

Quoted 5 lines
Quoted 5 lines

Since we already have the release images prepared and tested and these are not going to change, the criteria for potential changes:

a) We can unit test the changes and be confident they don't break/regress things.

For the Future: Besides doing a basic build, we need to have some real unit tests for bitbake and the poky infrastructure, I guess I need to turn this into a Testing feature request for 1.0 (look for it soon).

Quoted 2 lines
Quoted 2 lines

b) They fix important bugs that the user can easily run into or that make the project look bad.

After reviewing the changes I agree, don't get me wrong, I am still very nervous about these changes.

Quoted 2 lines
Quoted 2 lines

c) The changes are small, well documented and are obviously correct looking at the code/patch.

Some times we over look the obvious changes, been caught by that myself too many time.

Quoted 1 line
Quoted 1 line

d) The don't change the generated images.

<SNIP>

Quoted 8 lines
Quoted 5 lines

I'm not happy about being in this position and I know Dave will be very nervous about these late changes. To mitigate this I'd like to propose that a selection of people (Josh, Mark, Saul?) review these changes and report back on whether they feel these are appropriate and also give the build some testing with these applied.

I'm so predictable... :-) Yes, I'm nervous. I looked at all of the patches and with the exception of one or two, they mostly seem like good ones. I will accept these if Josh/Mark/Saul give us a +1 on their review& testing.

If there was 1 or 2 changes, I would be much happier, but there are almost a dozen changes, yes mostly individually they are OK, I am still reviewing them all, and have not started any testing with them yet.

I agree with Dave that there are a couple that I am more nervous about the pseudo/fakeroot as we have had so much trouble in the past, yes I know this will make things better, but what else will crop up?

Quoted 16 lines
Quoted 16 lines

Cheers,

Richard

_______________________________________________
yocto mailing list
[email protected]
https://lists.pokylinux.org/listinfo/yocto

Quoted 8 lines
Quoted 4 lines

_______________________________________________
yocto mailing list
[email protected]
https://lists.pokylinux.org/listinfo/yocto

_______________________________________________
yocto mailing list
[email protected]
https://lists.pokylinux.org/listinfo/yocto

_______________________________________________
yocto mailing list
[email protected]
https://lists.pokylinux.org/listinfo/yocto

Signature · 2 lines
Dirk Hohndel
Intel Open Source Technology Center
#5 lore raw
Quoted 13 lines

From: [email protected] [mailto:yocto- [email protected]] On Behalf Of Dirk Hohndel
Sent: Friday, October 22, 2010 10:48 AM

Quoted 1 line · Mark Hatle

Add a +1 to reviewed, worried, but accepting column. They each seem

reasonable,

Quoted 1 line

low-enough risk..

same here

Quoted 8 lines

We won't have a bug free release. No one ever does. And any change increases the risk.

So the question is "are they worth the added risk?". I believe the proposed changes address issues that people WILL run into as they start playing with things, so I think the risk is worth the reward.

/D

Saul and Richard and I put a plan together for this, will create a new thread for it.

Quoted 119 lines
Quoted 8 lines · Saul G. Wold
Quoted 6 lines · David C
Quoted 5 lines
Quoted 5 lines

From: [email protected] [mailto:yocto- [email protected]] On Behalf Of Richard Purdie
Sent: Friday, October 22, 2010 7:24 AM

Coming up to release there are a few things that the extended

testing

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

has shown up which we have fixes for and which we should consider including in the release. I also finally got around to doing the

final

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

sstate stress testing and found several problematic issues. Given

that

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

sstate and checksums are a significant feature of this release,

I'd

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

really like them to work as well as we can make them. Prior to

this I

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

had stress tested the backend up not the use of the packages.

These

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

changes don't change any sstate packages themselves, just the use

of

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

them.

Since we already have the release images prepared and tested and

these

Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
Quoted 4 lines

are not going to change, the criteria for potential changes:

a) We can unit test the changes and be confident they don't break/regress things.

For the Future: Besides doing a basic build, we need to have some

real

Quoted 1 line
Quoted 1 line

unit tests for bitbake and the poky infrastructure, I guess I need

to

Quoted 5 lines
Quoted 5 lines

turn this into a Testing feature request for 1.0 (look for it soon).

Quoted 2 lines
Quoted 2 lines

b) They fix important bugs that the user can easily run into or that make the project look bad.

After reviewing the changes I agree, don't get me wrong, I am still

very

Quoted 3 lines
Quoted 3 lines

nervous about these changes.

Quoted 1 line
Quoted 1 line

c) The changes are small, well documented and are obviously

correct

Quoted 2 lines
Quoted 2 lines
Quoted 1 line
Quoted 1 line

looking at the code/patch.

Some times we over look the obvious changes, been caught by that

myself

Quoted 7 lines
Quoted 7 lines

too many time.

Quoted 1 line
Quoted 1 line

d) The don't change the generated images.

<SNIP>

Quoted 1 line
Quoted 1 line

I'm not happy about being in this position and I know Dave will be

very

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

nervous about these late changes. To mitigate this I'd like to

propose

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

that a selection of people (Josh, Mark, Saul?) review these

changes and

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

report back on whether they feel these are appropriate and also

give the

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

build some testing with these applied.

I'm so predictable... :-) Yes, I'm nervous. I looked at all of the

patches and with the exception of one or two, they mostly seem like good ones. I will accept these if Josh/Mark/Saul give us a +1 on their
review& testing.

Quoted 4 lines
Quoted 4 lines
Quoted 1 line

If there was 1 or 2 changes, I would be much happier, but there are almost a dozen changes, yes mostly individually they are OK, I am

still

Quoted 3 lines
Quoted 3 lines

reviewing them all, and have not started any testing with them yet.

I agree with Dave that there are a couple that I am more nervous

about

Quoted 1 line
Quoted 1 line

the pseudo/fakeroot as we have had so much trouble in the past, yes

I

Quoted 28 lines
Quoted 19 lines

know this will make things better, but what else will crop up?

Quoted 16 lines
Quoted 16 lines

Cheers,

Richard

_______________________________________________
yocto mailing list
[email protected]
https://lists.pokylinux.org/listinfo/yocto

Quoted 8 lines
Quoted 4 lines

_______________________________________________
yocto mailing list
[email protected]
https://lists.pokylinux.org/listinfo/yocto

_______________________________________________
yocto mailing list
[email protected]
https://lists.pokylinux.org/listinfo/yocto

Quoted 4 lines

_______________________________________________
yocto mailing list
[email protected]
https://lists.pokylinux.org/listinfo/yocto

--
Dirk Hohndel
Intel Open Source Technology Center
_______________________________________________
yocto mailing list
[email protected]
https://lists.pokylinux.org/listinfo/yocto

#6 lore raw
In reply to Richard Purdie
Quoted 6 lines · Richard Purdie

I'm not happy about being in this position and I know Dave will be very nervous about these late changes. To mitigate this I'd like to propose that a selection of people (Josh, Mark, Saul?) review these changes and report back on whether they feel these are appropriate and also give the build some testing with these applied.

I've looked over the changes and spent this afternoon testing them, I believe they are appropriate for the 0.9 release.

Cheers,
Joshua

Signature · 2 lines
Joshua Lock
        Intel Open Source Technology Centre