Skip to contentSkip to conversation

Image history (testlab)

DISCUSSION Paul Eggleton 3 participants 4 messages Nov 25, 2011

lore
Share & tools
lore
#0 lore raw

Hi Koen / Beth / all,

Following on from my work extending packagehistory (which is still under review), I'm looking at tracking history of the content of images so we can get warnings about regressions. Thanks to Koen we've had this capability via testlab.bbclass in OE for some time now, and from there into meta-oe, but what I'd like to do now is bring this into OE-core and extend it so that it supports rpm (or even better, all of the package backends we support.) This could involve abstracting the package backend functionality out into rootfs_*.bbclass.

It also looks like testlab.bbclass in meta-oe has some license dumping functionality. Beth, does that mesh with what you've been working on / preparing for Yocto 1.2?

Thoughts?

Cheers,
Paul

Signature · 2 lines
Paul Eggleton
Intel Open Source Technology Centre
#1 lore raw

Op 25 nov. 2011, om 12:44 heeft Paul Eggleton het volgende geschreven:

Quoted 11 lines

Hi Koen / Beth / all,

Following on from my work extending packagehistory (which is still under review), I'm looking at tracking history of the content of images so we can get warnings about regressions. Thanks to Koen we've had this capability via testlab.bbclass in OE for some time now, and from there into meta-oe, but what I'd like to do now is bring this into OE-core and extend it so that it supports rpm (or even better, all of the package backends we support.) This could involve abstracting the package backend functionality out into rootfs_*.bbclass.

Quoted 5 lines

It also looks like testlab.bbclass in meta-oe has some license dumping functionality. Beth, does that mesh with what you've been working on / preparing for Yocto 1.2?

Thoughts?

Don't forget its integration into remote git: http://git.angstrom-distribution.org/cgi-bin/cgit.cgi/testlab/?h=yocto
I must admit that it completely sucks when you have >1 builder, but that is fixable. There are a number of other things we need to store somewhere (e.g. package history, PR config, etc), so coming up with a plan to store all that in a single repo would be good.

The are a few issues with its current output:

* License info uses the filename instead of the package name (f'oo_1.2.3_arch.ipk' vs 'foo') which leads to too much changes with small version bumps
* Dependencies are not sorted intelligently, so sometimes you get diffs which don't contain any actual change, just ordering junk,
* It only reports oe-core rev, not layer revisions, config and shortlog. Ideally it would output something like:

meta: angstrom-staging:b3ffd40a4a1eacab90dac26df8a4a047486f3b96 pulseaudio: update to 1.1, delete 0.9.x
meta-oe: master:7019fbb50796fedcedef6ff92981a7f26659a251 mplayer: fix build with -Wl,-as-needed
meta-efl:
meta-gnome:
meta-xfce:
meta-angstrom master:bc72e71ccf7e79cdc8246e0ae0dbe62d11355263 angstrom-2010-preferred-versions.inc : Stop pinning python-cairo

Taking this a step further: it's output should be able to get passed into layer tooling (combo-layer to yocto people, setup-scripts for angstrom people) to reproduce builds.

regards,

Koen

#2 lore raw
Quoted 27 lines · Koen Kooi

Op 25 nov. 2011, om 12:44 heeft Paul Eggleton het volgende geschreven:

Quoted 11 lines

Hi Koen / Beth / all,

Following on from my work extending packagehistory (which is still under review), I'm looking at tracking history of the content of images so we can get warnings about regressions. Thanks to Koen we've had this capability via testlab.bbclass in OE for some time now, and from there into meta-oe, but what I'd like to do now is bring this into OE-core and extend it so that it supports rpm (or even better, all of the package backends we support.) This could involve abstracting the package backend functionality out into rootfs_*.bbclass.

Quoted 5 lines

It also looks like testlab.bbclass in meta-oe has some license dumping functionality. Beth, does that mesh with what you've been working on / preparing for Yocto 1.2?

Thoughts?

Don't forget its integration into remote git: http://git.angstrom-distribution.org/cgi-bin/cgit.cgi/testlab/?h=yocto
I must admit that it completely sucks when you have >1 builder, but that is fixable. There are a number of other things we need to store somewhere (e.g. package history, PR config, etc), so coming up with a plan to store all that in a single repo would be good.

The are a few issues with its current output:

* License info uses the filename instead of the package name (f'oo_1.2.3_arch.ipk' vs 'foo') which leads to too much changes with small version bumps
* Dependencies are not sorted intelligently, so sometimes you get diffs which don't contain any actual change, just ordering junk,
* It only reports oe-core rev, not layer revisions, config and shortlog. Ideally it would output something like:

Quoted 8 lines

meta: angstrom-staging:b3ffd40a4a1eacab90dac26df8a4a047486f3b96 pulseaudio: update to 1.1, delete 0.9.x
meta-oe: master:7019fbb50796fedcedef6ff92981a7f26659a251 mplayer: fix build with -Wl,-as-needed
meta-efl:
meta-gnome:
meta-xfce:
meta-angstrom master:bc72e71ccf7e79cdc8246e0ae0dbe62d11355263 angstrom-2010-preferred-versions.inc : Stop pinning python-cairo

Taking this a step further: it's output should be able to get passed into layer tooling (combo-layer to yocto people, setup-scripts for angstrom people) to reproduce builds.

So the real question is one of the approach. We have several options and its related to how compatible we want/need to be.

Do we import testlab from meta-oe as is, then add patches (multiple package backend support and so on)? How compatible do we need to be? I'm conscious its something being actively used.

Do we enhance packagehistory reusing code from testlab as appropriate?

Do we start something new as a hybrid of the two? (shouldn't be needed as I'm prepared to say packagehistory can radically change as needed, I don't care about compatibility there).

At the end of this I want to have one great "history" tracking mechanism for finding regressions covering testlab's functionality, package history and more.

Just to be clear I don't want to reinvent the wheel :).

Cheers,

Richard

#3 lore raw

Op 25 nov. 2011, om 14:52 heeft Richard Purdie het volgende geschreven:

Quoted 35 lines · Koen Kooi
Quoted 34 lines

Op 25 nov. 2011, om 12:44 heeft Paul Eggleton het volgende geschreven:

Quoted 11 lines

Hi Koen / Beth / all,

Following on from my work extending packagehistory (which is still under review), I'm looking at tracking history of the content of images so we can get warnings about regressions. Thanks to Koen we've had this capability via testlab.bbclass in OE for some time now, and from there into meta-oe, but what I'd like to do now is bring this into OE-core and extend it so that it supports rpm (or even better, all of the package backends we support.) This could involve abstracting the package backend functionality out into rootfs_*.bbclass.

Quoted 5 lines

It also looks like testlab.bbclass in meta-oe has some license dumping functionality. Beth, does that mesh with what you've been working on / preparing for Yocto 1.2?

Thoughts?

Don't forget its integration into remote git: http://git.angstrom-distribution.org/cgi-bin/cgit.cgi/testlab/?h=yocto
I must admit that it completely sucks when you have >1 builder, but that is fixable. There are a number of other things we need to store somewhere (e.g. package history, PR config, etc), so coming up with a plan to store all that in a single repo would be good.

The are a few issues with its current output:

* License info uses the filename instead of the package name (f'oo_1.2.3_arch.ipk' vs 'foo') which leads to too much changes with small version bumps
* Dependencies are not sorted intelligently, so sometimes you get diffs which don't contain any actual change, just ordering junk,
* It only reports oe-core rev, not layer revisions, config and shortlog. Ideally it would output something like:

meta: angstrom-staging:b3ffd40a4a1eacab90dac26df8a4a047486f3b96 pulseaudio: update to 1.1, delete 0.9.x
meta-oe: master:7019fbb50796fedcedef6ff92981a7f26659a251 mplayer: fix build with -Wl,-as-needed
meta-efl:
meta-gnome:
meta-xfce:
meta-angstrom master:bc72e71ccf7e79cdc8246e0ae0dbe62d11355263 angstrom-2010-preferred-versions.inc : Stop pinning python-cairo

Quoted 8 lines
Quoted 1 line

Taking this a step further: it's output should be able to get passed into layer tooling (combo-layer to yocto people, setup-scripts for angstrom people) to reproduce builds.

So the real question is one of the approach. We have several options and its related to how compatible we want/need to be.

Do we import testlab from meta-oe as is, then add patches (multiple package backend support and so on)? How compatible do we need to be? I'm conscious its something being actively used.

In this specific case I don't care about backwards compatibility.

Quoted 11 lines

Do we enhance packagehistory reusing code from testlab as appropriate?

Do we start something new as a hybrid of the two? (shouldn't be needed as I'm prepared to say packagehistory can radically change as needed, I don't care about compatibility there).

At the end of this I want to have one great "history" tracking mechanism for finding regressions covering testlab's functionality, package history and more.

Just to be clear I don't want to reinvent the wheel :).

For this history tracking both 'testlab' and 'packagehistory' are misnomers, so we can just start a new class that reuses ideas and code if needed.

Hopefully testlab can then fullfill its original goal: fully automated on-target testing. That should be doable with OE-core since that only has qemu machines :)

regards,

Koen