gunzip slows booting

Hi All,

I'm building a 2.6.25 kernel and initramfs for an atmel at91sam9260ek using buildroot. I'm not sure if I'm experiencing a buildroot problem per se, but hopefully someone here might have some insight.

With the default uImage (uses compressed zImage) and default compressed initramfs I have a boot time from exiting u-boot to my user space application running (started from iniittab) on the order of 3.0 sec.

If I generate the uImage from Image (uncompressed kernel image), my
boot time drops to approx 2.0 sec. As far as I can see, the only
difference is that gunzip() is not called for the uncompressed kernel. It becomes a simple data copy. I don't have the image sizes at hand, but gzip typically gives a 2:1 compression ratio.

Similarly, if I generate an uncompressed initramfs ( I simply removed the gzip from gen_initramfs_list.sh) my boot time further drops to 1.2
sec. The filesystem is ~600k compressed and 1.3 MB uncompressed.
Again, the unzip is replaced by a data copy operation.

I guess the first question is whether there is a problem or not with
gzip appearing to be slow? I had expected the compressed kernel &
initramfs to have quicker boot times than uncompressed. I suspect the default kernel boot may be optimized for relatively slow disk copy and
very fast processors [typical desktop]. In such a configuration a
long unzip operation probably is faster than coping a bigger image from a slow disk.

Assuming there is a problem here, I have started to debug this.
basically gunzip() calls inflate(). After a few more calls we get to
inflate_codes() which ultimate ends in a call to memcpy(). What I have observed and find very strange is that memcpy() is being used to repeatedly copy a very small number of bytes (typically 1 to 4). Occasionally it is called to copy a larger # of bytes. There are literally tens (or hundreds) of thousands of memcpy to move a few MB
data. I suspect this is where the vast majority of the time is being
spent.

Is it possible that I am seeing an architecture specific problem?

Any comments/suggestions are welcome.

Mike

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Mike Sander skrev:

| Hi All,
|
| I'm building a 2.6.25 kernel and initramfs for an atmel at91sam9260ek
| using buildroot.  I'm not sure if I'm experiencing a buildroot problem
| per se, but hopefully someone here might have some insight.
|
| With the default uImage (uses compressed zImage) and default compressed
| initramfs I have a boot time from exiting u-boot to my user space
| application running (started from iniittab) on the order of 3.0 sec.
|
| If I generate the uImage from Image (uncompressed kernel image),  my
| boot time drops to approx 2.0 sec.    As far as I can see, the only
| difference is that gunzip() is not called for the uncompressed kernel.
| It becomes a simple data copy.  I don't have the image sizes at hand,
| but gzip typically gives a 2:1 compression ratio.
|
| Similarly, if I generate an uncompressed initramfs ( I simply removed
| the gzip from gen_initramfs_list.sh) my boot time further drops to 1.2
| sec.   The filesystem is ~600k compressed and 1.3 MB uncompressed.
| Again, the unzip is replaced by a data copy operation.
|
| I guess the first question is whether there is a problem or not with
| gzip appearing to be slow?   I had expected the compressed kernel &
| initramfs to have quicker boot times than uncompressed.  I suspect the
| default kernel boot may be optimized for relatively slow disk copy and
| very fast processors [typical desktop].    In such a configuration a
| long unzip operation probably is faster than coping a bigger image from
| a slow disk.
|
| Assuming there is a problem here, I have started to debug this.
| basically gunzip() calls inflate().   After a few more calls we get to
| inflate_codes() which ultimate ends in a call to memcpy().  What I have
| observed and find very strange is that memcpy() is being used to
| repeatedly copy a very small number of bytes (typically 1 to 4).
| Occasionally it is called to copy a larger # of bytes.  There are
| literally  tens (or hundreds) of thousands of memcpy to move a few MB
| data.   I suspect this is where the vast majority of the time is being
| spent.
|
| Is it possible that I am seeing an architecture specific problem?
|
|
| Any comments/suggestions are welcome.

I would be very surprised if you could get a compressed kernel to boot faster than an uncompressed kernel on an ARM if you store in parallel flash
You can read a 16 bit parallel flash at about ~20 MB/s and you can write a 32 bit SDRAM at 400 MB/s.

For 1,3 MB the flash read should be ~67 ms
and the SDRAM write should be ~ 3 ms.

With a compressed kernel you would read at ~33 ms. Then you have to do tab ~20 MB/s out ofle lookup to decompress, and this could take a lot of time.

zipping the kernel is mainly for reducing the cost by enabling the use of smaller flash.

Note that if you use NAND flash,then you have to do ECC calculations on all NAND reads and then compression becomes more favourable.

BR
Ulf Samuelsson

|
| Mike
| _______________________________________________
| buildroot mailing list
| buildroot at uclibc.org
| http://busybox.net/mailman/listinfo/buildroot

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.4-svn0 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org

iD8DBQFIduqRAyRRH5cXxqwRArogAJ9LHOFYpUbTkc8I2KaaCq874FYxZQCfQ1wK QxxqvTKmeXk7GQCVEUmU+V4=
=zE5v
-----END PGP SIGNATURE-----

Ulf, Thank You for the quick reply.

My times start immediately after u-boot has finished copying the image
[via tftp or from flash] to when user space code is running. I have
not included the time to copy the image from flash -> sdram. We intend to use nor flash which I do not have yet. Unless we can xip the image has to be initially copied to sdram. [please correct me if i'm wrong on this point].. so this seems to be a good starting point for timing.

Note that the flash read times you quote are on the order of 10s of
msec. Doing the gunzip() adds approx 1000 msec to the kernel boot
process . This overwhelms the difference in reading the compressed vs
uncompressed image. It is the 1000 msec gunzip() that I am questioning. Does this seem reasonable to you?

For completeness, this is my boot process. Perhaps I am using u-boot incorrectly?

When booting via tftp I do the following:

u-boot> tftp 21100000 uImage
u-boot> bootm

In booting from dataflash (and eventually nor flash) I do the following:

Initially use u-boot to get the image into flash
u-boot> tftp 21100000 uImage
u-boot> cp.b 21100000 <flash addr> <image size>

I then set up u-boot bootcmd as follows to boot from the flash image:

u-boot> setenv bootcmd 'cp.b <flash addr> 21100000 <image size>;bootm 21100000'

With this configuration bootcmd copies the entire image from flash to sdram, and then u-boot makes the jump to the start of the image in sdram (head.o i believe). The kernels first task is then to unzip (or copy) the payload to its final execution address.

Regards,

Mike