Skip to contentSkip to conversation

RPM vs IPK

DISCUSSION Gary Thomas 3 participants 5 messages May 19, 2011

lore
Share & tools
lore
#0 lore raw

Building Poky for various targets, I see some striking differences based on the packaging. I'm building for the beagleboard (RPM) and my own OMAP/3530 (IPK), so everything is the same for these packages (same compiler, architecture, etc), only the package method differs. This was built on an otherwise idle box
4-way (Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz), with
BB_NUMBER_THREADS ?= "4"
PARALLEL_MAKE ?= "-j 4"

Each of these tests are a complete build of the package, with all dependencies already built. For example, I use this sequence:
% bitbake perl
% bitbake perl -c clean
% rm sstate-cache/sstate-perl-arm*
% time bitbake perl

perl - RPM IPK

       real    12m15.520s          real    9m43.228s
       user    5m42.988s           user    4m40.692s
       sys     3m56.636s           sys     2m19.860s

eglibc RPM IPK

       real    32m19.984s          real    23m52.124s
       user    15m32.732s          user    20m48.214s
       sys     17m28.087s          sys     9m3.936s

Bottom line - it seems to take 20-30% longer to package via RPM.

I know there are reasons and tradeoffs for different packaging methods, but 30% extra?

Signature · 4 lines
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------
#1 lore raw
Quoted 22 lines · Gary Thomas

Building Poky for various targets, I see some striking differences based on the packaging. I'm building for the beagleboard (RPM) and my own OMAP/3530 (IPK), so everything is the same for these packages (same compiler, architecture, etc), only the package method differs. This was built on an otherwise idle box
4-way (Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz), with
BB_NUMBER_THREADS ?= "4"
PARALLEL_MAKE ?= "-j 4"

Each of these tests are a complete build of the package, with all dependencies already built. For example, I use this sequence:
% bitbake perl
% bitbake perl -c clean
% rm sstate-cache/sstate-perl-arm*
% time bitbake perl

perl - RPM IPK

        real    12m15.520s          real    9m43.228s
        user    5m42.988s           user    4m40.692s
        sys     3m56.636s           sys     2m19.860s

eglibc RPM IPK

Quoted 9 lines
        real    32m19.984s          real    23m52.124s
        user    15m32.732s          user    20m48.214s
        sys     17m28.087s          sys     9m3.936s

Bottom line - it seems to take 20-30% longer to package via RPM.

I know there are reasons and tradeoffs for different packaging methods, but 30% extra?

This matches my expectations for the most part. RPM creates and processes a lot more metadata then IPK. IPK is well optimized for small systems, and certainly I would advise someone generating a small system to use IPK.

For medium and (especially) large systems, RPM starts to give more abilities that IPK due to the additional meta data. This includes individual file type information, file checksum generation and evaluation on install, sparse file support, conflict detection and resolution [for multilib systems], ACID style upgrade, repackaging abilities for rollback, etc.. (Some of the items for RPM are not yet enabled in oe-core, but could be by individual developers.)

(I also wouldn't be surprised if we've got integration optimizations in the RPM backend in oe-core that could help with those numbers. The numbers seem to be much worse on packages with large numbers of files, vs the typical package with only a few to a hundred or so files.)

Downside of RPM for small systems is the Berkley DB and the amount of meta data.
If you will be doing on-device upgrades the space required to handle the
berkley DB and related items is quite large.. Also many of the advanced features available in RPM simple are not needed in small systems.

--Mark

#2 lore raw
Quoted 49 lines · Mark Hatle
Quoted 22 lines · Gary Thomas

Building Poky for various targets, I see some striking differences based on the packaging. I'm building for the beagleboard (RPM) and my own OMAP/3530 (IPK), so everything is the same for these packages (same compiler, architecture, etc), only the package method differs. This was built on an otherwise idle box
4-way (Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz), with

    BB_NUMBER_THREADS ?= "4"
    PARALLEL_MAKE ?= "-j 4"

Each of these tests are a complete build of the package, with all dependencies already built. For example, I use this sequence:
% bitbake perl
% bitbake perl -c clean
% rm sstate-cache/sstate-perl-arm*
% time bitbake perl

perl - RPM IPK

         real    12m15.520s          real    9m43.228s
         user    5m42.988s           user    4m40.692s
         sys     3m56.636s           sys     2m19.860s

eglibc RPM IPK

Quoted 9 lines
         real    32m19.984s          real    23m52.124s
         user    15m32.732s          user    20m48.214s
         sys     17m28.087s          sys     9m3.936s

Bottom line - it seems to take 20-30% longer to package via RPM.

I know there are reasons and tradeoffs for different packaging methods, but 30% extra?

This matches my expectations for the most part. RPM creates and processes a lot more metadata then IPK. IPK is well optimized for small systems, and certainly I would advise someone generating a small system to use IPK.

For medium and (especially) large systems, RPM starts to give more abilities that IPK due to the additional meta data. This includes individual file type information, file checksum generation and evaluation on install, sparse file support, conflict detection and resolution [for multilib systems], ACID style upgrade, repackaging abilities for rollback, etc.. (Some of the items for RPM are not yet enabled in oe-core, but could be by individual developers.)

(I also wouldn't be surprised if we've got integration optimizations in the RPM backend in oe-core that could help with those numbers. The numbers seem to be much worse on packages with large numbers of files, vs the typical package with only a few to a hundred or so files.)

Quoted 4 lines

Downside of RPM for small systems is the Berkley DB and the amount of meta data.
If you will be doing on-device upgrades the space required to handle the
berkley DB and related items is quite large.. Also many of the advanced features available in RPM simple are not needed in small systems.

Fair enough. Perhaps the packaging type should be chosen based on the target, e.g. the BeagleBoard probably would be best served by IPK.

Certainly, these data/tradeoffs should be made clear somewhere in the documentation.

Signature · 4 lines
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------
#3 lore raw
Quoted 62 lines

From: [email protected] [mailto:poky- [email protected]] On Behalf Of Gary Thomas
Sent: Thursday, May 19, 2011 7:26 AM

Quoted 29 lines · Mark Hatle
Quoted 28 lines · Gary Thomas

Building Poky for various targets, I see some striking differences based on the packaging. I'm building for the beagleboard (RPM) and my own OMAP/3530 (IPK), so everything is the same for these packages (same compiler, architecture, etc), only the package method differs. This was built on an otherwise idle box
4-way (Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz), with

    BB_NUMBER_THREADS ?= "4"
    PARALLEL_MAKE ?= "-j 4"

Each of these tests are a complete build of the package, with all dependencies already built. For example, I use this sequence:
% bitbake perl
% bitbake perl -c clean
% rm sstate-cache/sstate-perl-arm*
% time bitbake perl

perl - RPM IPK

         real    12m15.520s          real    9m43.228s
         user    5m42.988s           user    4m40.692s
         sys     3m56.636s           sys     2m19.860s

eglibc RPM IPK

         real    32m19.984s          real    23m52.124s
         user    15m32.732s          user    20m48.214s
         sys     17m28.087s          sys     9m3.936s

Bottom line - it seems to take 20-30% longer to package via RPM.

Quoted 7 lines
Quoted 3 lines

I know there are reasons and tradeoffs for different packaging methods, but 30% extra?

This matches my expectations for the most part. RPM creates and processes a lot more metadata then IPK. IPK is well optimized for small systems, and certainly I would advise someone generating a small

system to use IPK.

Quoted 9 lines

For medium and (especially) large systems, RPM starts to give more abilities that IPK due to the additional meta data. This includes individual file type information, file checksum generation and evaluation on install, sparse file support, conflict detection and resolution [for multilib systems], ACID style upgrade, repackaging abilities for rollback, etc.. (Some of the items for RPM are not yet enabled in oe-core, but could be by individual developers.)

Quoted 6 lines

(I also wouldn't be surprised if we've got integration optimizations in the RPM backend in oe-core that could help with those numbers. The numbers seem to be much worse on packages with large numbers of files, vs the typical package with only a few to a hundred or so files.)

Downside of RPM for small systems is the Berkley DB and the amount of

meta data.

Quoted 3 lines

If you will be doing on-device upgrades the space required to handle
the berkley DB and related items is quite large.. Also many of the advanced features available in RPM simple are not needed in small

Quoted 7 lines

systems.

Fair enough. Perhaps the packaging type should be chosen based on the target, e.g. the BeagleBoard probably would be best served by IPK.

Certainly, these data/tradeoffs should be made clear somewhere in the documentation.

I agree - hey Gary, can you please file a Bugzilla to add this to the Reference Manual? Thanks...

#4 lore raw
Quoted 64 lines · David C
Quoted 64 lines

From: [email protected] [mailto:poky- [email protected]] On Behalf Of Gary Thomas
Sent: Thursday, May 19, 2011 7:26 AM

Quoted 29 lines · Mark Hatle
Quoted 28 lines · Gary Thomas

Building Poky for various targets, I see some striking differences based on the packaging. I'm building for the beagleboard (RPM) and my own OMAP/3530 (IPK), so everything is the same for these packages (same compiler, architecture, etc), only the package method differs. This was built on an otherwise idle box
4-way (Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz), with

     BB_NUMBER_THREADS ?= "4"
     PARALLEL_MAKE ?= "-j 4"

Each of these tests are a complete build of the package, with all dependencies already built. For example, I use this sequence:
% bitbake perl
% bitbake perl -c clean
% rm sstate-cache/sstate-perl-arm*
% time bitbake perl

perl - RPM IPK

          real    12m15.520s          real    9m43.228s
          user    5m42.988s           user    4m40.692s
          sys     3m56.636s           sys     2m19.860s

eglibc RPM IPK

          real    32m19.984s          real    23m52.124s
          user    15m32.732s          user    20m48.214s
          sys     17m28.087s          sys     9m3.936s

Bottom line - it seems to take 20-30% longer to package via RPM.

Quoted 7 lines
Quoted 3 lines

I know there are reasons and tradeoffs for different packaging methods, but 30% extra?

This matches my expectations for the most part. RPM creates and processes a lot more metadata then IPK. IPK is well optimized for small systems, and certainly I would advise someone generating a small

system to use IPK.

Quoted 9 lines

For medium and (especially) large systems, RPM starts to give more abilities that IPK due to the additional meta data. This includes individual file type information, file checksum generation and evaluation on install, sparse file support, conflict detection and resolution [for multilib systems], ACID style upgrade, repackaging abilities for rollback, etc.. (Some of the items for RPM are not yet enabled in oe-core, but could be by individual developers.)

Quoted 6 lines

(I also wouldn't be surprised if we've got integration optimizations in the RPM backend in oe-core that could help with those numbers. The numbers seem to be much worse on packages with large numbers of files, vs the typical package with only a few to a hundred or so files.)

Downside of RPM for small systems is the Berkley DB and the amount of

meta data.

Quoted 3 lines

If you will be doing on-device upgrades the space required to handle
the berkley DB and related items is quite large.. Also many of the advanced features available in RPM simple are not needed in small

systems.

Quoted 7 lines
Quoted 5 lines

Fair enough. Perhaps the packaging type should be chosen based on the target, e.g. the BeagleBoard probably would be best served by IPK.

Certainly, these data/tradeoffs should be made clear somewhere in the documentation.

I agree - hey Gary, can you please file a Bugzilla to add this to the Reference Manual? Thanks...

Done - bug #1088

Signature · 4 lines
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------