Skip to contentSkip to conversation

let PACKAGES_DYNAMIC be optional ?

RFC Robert Yang 3 participants 4 messages Nov 15, 2013 – Nov 18, 2013

lore
Share & tools
lore
#0 lore raw

Currently, the recipe which uses PACKAGES_DYNAMIC usually generates a lot of packages which costs a lot of time on building the recipe and do_rootfs, for example, the perl and kernel:

$ ls tmp/deploy/rpm/armv5te/perl-module-* | wc -l
621
$ ls tmp/deploy/rpm/qemux86/kernel-module-* | wc -l
268

Also, the eglibc-locale generates more than 300 packages.

Take perl as an example:

1) We generate 621 perl-module-* packages, but the package *perl-modules* requires all of them, so once *perl-modules* is installed, all the other perl-module-* will be installed and we can't remove any of them since perl-modules rdepends on it, if there is a way to package all of these perl-module-* into one package (they are about 10MB), it would save a lot of time on do_package* and do_rootfs.

2) The nativesdk.bbclass can't support PACKAGES_DYNAMIC, for example, it can't change the perl-module-app-cpan to nativesdk-perl-module-app-cpan since there is no perl-module-app-cpan in PACKAGES when nativesdk.bbclass changes the variable's name.

Can we add a way to let the PACKAGES_DYNAMIC be optional ? for example,

PACKAGES_DYNAMIC[perl] = "0"

will disable the perl's PACKAGES_DYNAMIC, and will pack the files as other recipes do, and of course we need to do some work on the recipe.

Signature · 3 lines
Thanks

Robert
#1 lore raw
Quoted 26 lines · Robert Yang

Currently, the recipe which uses PACKAGES_DYNAMIC usually generates a lot of packages which costs a lot of time on building the recipe and do_rootfs, for example, the perl and kernel:

$ ls tmp/deploy/rpm/armv5te/perl-module-* | wc -l
621
$ ls tmp/deploy/rpm/qemux86/kernel-module-* | wc -l
268

Also, the eglibc-locale generates more than 300 packages.

Take perl as an example:

1) We generate 621 perl-module-* packages, but the package *perl-modules* requires all of them, so once *perl-modules* is installed, all the other perl-module-* will be installed and we can't remove any of them since perl-modules rdepends on it, if there is a way to package all of these perl-module-* into one package (they are about 10MB), it would save a lot of time on do_package* and do_rootfs.

2) The nativesdk.bbclass can't support PACKAGES_DYNAMIC, for example, it can't change the perl-module-app-cpan to nativesdk-perl-module-app-cpan since there is no perl-module-app-cpan in PACKAGES when nativesdk.bbclass changes the variable's name.

Quoted 6 lines

Can we add a way to let the PACKAGES_DYNAMIC be optional ? for example,

PACKAGES_DYNAMIC[perl] = "0"

will disable the perl's PACKAGES_DYNAMIC, and will pack the files as other recipes do, and of course we need to do some work on the recipe.

Before we consider doing this, I'd actually like to see real numbers about how big this problem is.

Why? Speaking as someone who has looked specifically at perl and the kernel, I don't believe there is a huge amount of time spent dealing with the individual packages and that maintaining two build paths is actually worse than they minimal performance impact this has.

In particular, I'd note that the locale generation happens in parallel with other parts of the build and is not a significant factor in overall build times.

The time would be better spent reducing the size of the kernel source installed into the sysroot for example (Bruce is planning action on this).

Cheers,

Richard

#2 lore raw
Quoted 42 lines · Richard Purdie
Quoted 26 lines · Robert Yang

Currently, the recipe which uses PACKAGES_DYNAMIC usually generates a lot of packages which costs a lot of time on building the recipe and do_rootfs, for example, the perl and kernel:

$ ls tmp/deploy/rpm/armv5te/perl-module-* | wc -l
621
$ ls tmp/deploy/rpm/qemux86/kernel-module-* | wc -l
268

Also, the eglibc-locale generates more than 300 packages.

Take perl as an example:

1) We generate 621 perl-module-* packages, but the package *perl-modules* requires all of them, so once *perl-modules* is installed, all the other perl-module-* will be installed and we can't remove any of them since perl-modules rdepends on it, if there is a way to package all of these perl-module-* into one package (they are about 10MB), it would save a lot of time on do_package* and do_rootfs.

2) The nativesdk.bbclass can't support PACKAGES_DYNAMIC, for example, it can't change the perl-module-app-cpan to nativesdk-perl-module-app-cpan since there is no perl-module-app-cpan in PACKAGES when nativesdk.bbclass changes the variable's name.

Quoted 6 lines

Can we add a way to let the PACKAGES_DYNAMIC be optional ? for example,

PACKAGES_DYNAMIC[perl] = "0"

will disable the perl's PACKAGES_DYNAMIC, and will pack the files as other recipes do, and of course we need to do some work on the recipe.

Before we consider doing this, I'd actually like to see real numbers about how big this problem is.

Why? Speaking as someone who has looked specifically at perl and the kernel, I don't believe there is a huge amount of time spent dealing with the individual packages and that maintaining two build paths is actually worse than they minimal performance impact this has.

Quoted 7 lines

In particular, I'd note that the locale generation happens in parallel with other parts of the build and is not a significant factor in overall build times.

The time would be better spent reducing the size of the kernel source installed into the sysroot for example (Bruce is planning action on this).

I agree on Richard on this; keeping too many different build routes introduces more test burden and it's only worth if the gain is huge.

Signature · 3 lines
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750
#3 lore raw
Quoted 48 lines · Otavio Salvador
Quoted 46 lines · Richard Purdie
Quoted 26 lines · Robert Yang

Currently, the recipe which uses PACKAGES_DYNAMIC usually generates a lot of packages which costs a lot of time on building the recipe and do_rootfs, for example, the perl and kernel:

$ ls tmp/deploy/rpm/armv5te/perl-module-* | wc -l
621
$ ls tmp/deploy/rpm/qemux86/kernel-module-* | wc -l
268

Also, the eglibc-locale generates more than 300 packages.

Take perl as an example:

1) We generate 621 perl-module-* packages, but the package *perl-modules* requires all of them, so once *perl-modules* is installed, all the other perl-module-* will be installed and we can't remove any of them since perl-modules rdepends on it, if there is a way to package all of these perl-module-* into one package (they are about 10MB), it would save a lot of time on do_package* and do_rootfs.

2) The nativesdk.bbclass can't support PACKAGES_DYNAMIC, for example, it can't change the perl-module-app-cpan to nativesdk-perl-module-app-cpan since there is no perl-module-app-cpan in PACKAGES when nativesdk.bbclass changes the variable's name.

Quoted 6 lines

Can we add a way to let the PACKAGES_DYNAMIC be optional ? for example,

PACKAGES_DYNAMIC[perl] = "0"

will disable the perl's PACKAGES_DYNAMIC, and will pack the files as other recipes do, and of course we need to do some work on the recipe.

Before we consider doing this, I'd actually like to see real numbers about how big this problem is.

Why? Speaking as someone who has looked specifically at perl and the kernel, I don't believe there is a huge amount of time spent dealing with the individual packages and that maintaining two build paths is actually worse than they minimal performance impact this has.

In particular, I'd note that the locale generation happens in parallel with other parts of the build and is not a significant factor in overall build times.

Quoted 7 lines
Quoted 3 lines

The time would be better spent reducing the size of the kernel source installed into the sysroot for example (Bruce is planning action on this).

I agree on Richard on this; keeping too many different build routes introduces more test burden and it's only worth if the gain is huge.

Sounds reasonable, I will try sooner to see the gain.

// Robert