lib: enable lzma decompression support for SPL build
PATCH 11/16 lib: enable lzma decompression support for SPL build
This patch enables LZMA decompression support for SPL build
--- lib/Kconfig | 5 +++++ lib/Makefile | 1 + 2 files changed, 6 insertions(+)
lib/Kconfig +5 -0
@@ -418,6 +418,11 @@ config SPL_LZ4 fast compression and decompression speed. It belongs to the LZ77 family of byte-oriented compression schemes. +config SPL_LZMA + bool "Enable LZMA decompression support for SPL build" + help + This enables support for LZMA compression altorithm for SPL boot. + config SPL_LZO bool "Enable LZO decompression support in SPL" help
lib/Makefile +1 -0
@@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ obj-$(CONFIG_$(SPL_)ZSTD) += zstd/ obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o obj-$(CONFIG_$(SPL_)LZO) += lzo/ +obj-$(CONFIG_$(SPL_)LZMA) += lzma/ obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o obj-$(CONFIG_LIBAVB) += libavb/
Signature · 1 line
2.17.1
PATCH 11/16 lib: enable lzma decompression support for SPL build
Quoted 13 lines · Weijie Gao
This patch enables LZMA decompression support for SPL build
Signed-off-by Weijie Gao <[email protected]>--- lib/Kconfig | 5 +++++ lib/Makefile | 1 + 2 files changed, 6 insertions(+)lib/Kconfig +0
-0
lib/Kconfig config SPL_LZ4 · line 424
Quoted 7 lines
lib/Kconfig +4
-0@@ -418,6 +418,11 @@ config SPL_LZ4 fast compression and decompression speed. It belongs to the LZ77 family of byte-oriented compression schemes. +config SPL_LZMA + bool "Enable LZMA decompression support for SPL build" + help + This enables support for LZMA compression altorithm for SPL boot.
%s/altorithm/algorithm/g
Why do we need yet another algorithm for compression?
Doesn't Zstandard provide a much faster decompression? Cf. https://sites.google.com/site/powturbo/home/web-compression
Best regards
Heinrich
Quoted 17 lines
lib/Kconfig +1
-0+ config SPL_LZO bool "Enable LZO decompression support in SPL" helplib/Makefile +1
-0@@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ obj-$(CONFIG_$(SPL_)ZSTD) += zstd/ obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o obj-$(CONFIG_$(SPL_)LZO) += lzo/ +obj-$(CONFIG_$(SPL_)LZMA) += lzma/ obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o obj-$(CONFIG_LIBAVB) += libavb/
PATCH 11/16 lib: enable lzma decompression support for SPL build
Quoted 49 lines · Heinrich Schuchardt
Quoted 13 lines · Weijie Gao
This patch enables LZMA decompression support for SPL build
Signed-off-by Weijie Gao <[email protected]>--- lib/Kconfig | 5 +++++ lib/Makefile | 1 + 2 files changed, 6 insertions(+)lib/Kconfig +0
-0lib/Kconfig config SPL_LZ4 · line 424
Quoted 7 lines
lib/Kconfig +4
-0@@ -418,6 +418,11 @@ config SPL_LZ4 fast compression and decompression speed. It belongs to the LZ77 family of byte-oriented compression schemes. +config SPL_LZMA + bool "Enable LZMA decompression support for SPL build" + help + This enables support for LZMA compression altorithm for SPL boot.%s/altorithm/algorithm/g
Why do we need yet another algorithm for compression?
Doesn't Zstandard provide a much faster decompression? Cf. https://sites.google.com/site/powturbo/home/web-compression
Best regards
Heinrich
Quoted 15 lines
lib/Kconfig +1
-0+ config SPL_LZO bool "Enable LZO decompression support in SPL" helplib/Makefile +1
-0@@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ obj-$(CONFIG_$(SPL_)ZSTD) += zstd/ obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o obj-$(CONFIG_$(SPL_)LZO) += lzo/ +obj-$(CONFIG_$(SPL_)LZMA) += lzma/ obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o
Quoted 2 lines
lib/Makefile obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ · line 70
Quoted 2 lines
lib/Makefile +0
-0obj-$(CONFIG_LIBAVB) += libavb/
Hi Heinrich,
I need an algorithm with the maximum compression ratio to reduce the size of the u-boot payload as much as possible.
I believe currently lzma and xz are the best choices. Refer to:
https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO
Although they will take longer time for compression and decompression, considering that the u-boot payload is usually small (<= 1MiB), there will not be significant effect on decompression time.
Best Regards,
Weijie
PATCH 11/16 lib: enable lzma decompression support for SPL build
Quoted 58 lines · Weijie Gao
Quoted 49 lines · Heinrich Schuchardt
Quoted 13 lines · Weijie Gao
This patch enables LZMA decompression support for SPL build
Signed-off-by Weijie Gao <[email protected]>--- lib/Kconfig | 5 +++++ lib/Makefile | 1 + 2 files changed, 6 insertions(+)lib/Kconfig +0
-0lib/Kconfig config SPL_LZ4 · line 424
Quoted 7 lines
lib/Kconfig +4
-0@@ -418,6 +418,11 @@ config SPL_LZ4 fast compression and decompression speed. It belongs to the LZ77 family of byte-oriented compression schemes. +config SPL_LZMA + bool "Enable LZMA decompression support for SPL build" + help + This enables support for LZMA compression altorithm for SPL boot.%s/altorithm/algorithm/g
Why do we need yet another algorithm for compression?
Doesn't Zstandard provide a much faster decompression? Cf. https://sites.google.com/site/powturbo/home/web-compression
Best regards
Heinrich
Quoted 15 lines
lib/Kconfig +1
-0+ config SPL_LZO bool "Enable LZO decompression support in SPL" helplib/Makefile +1
-0@@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ obj-$(CONFIG_$(SPL_)ZSTD) += zstd/ obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o obj-$(CONFIG_$(SPL_)LZO) += lzo/ +obj-$(CONFIG_$(SPL_)LZMA) += lzma/ obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.oQuoted 2 lines
lib/Makefile obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ · line 70
Quoted 2 lines
lib/Makefile +0
-0obj-$(CONFIG_LIBAVB) += libavb/Hi Heinrich,
I need an algorithm with the maximum compression ratio to reduce the size of the u-boot payload as much as possible.
Quoted 7 lines
I believe currently lzma and xz are the best choices. Refer to:
https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZOAlthough they will take longer time for compression and decompression, considering that the u-boot payload is usually small (<= 1MiB), there will not be significant effect on decompression time.
Where does zstd fall in the list however? Can you please try that and report back? Thanks!
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200109/744b4b1c/attachment.sig>
PATCH 11/16 lib: enable lzma decompression support for SPL build
Quoted 63 lines · Tom Rini
Quoted 62 lines · Weijie Gao
Quoted 49 lines · Heinrich Schuchardt
Quoted 13 lines · Weijie Gao
This patch enables LZMA decompression support for SPL build
Signed-off-by Weijie Gao <[email protected]>--- lib/Kconfig | 5 +++++ lib/Makefile | 1 + 2 files changed, 6 insertions(+)lib/Kconfig +0
-0lib/Kconfig config SPL_LZ4 · line 424
Quoted 7 lines
lib/Kconfig +4
-0@@ -418,6 +418,11 @@ config SPL_LZ4 fast compression and decompression speed. It belongs to the LZ77 family of byte-oriented compression schemes. +config SPL_LZMA + bool "Enable LZMA decompression support for SPL build" + help + This enables support for LZMA compression altorithm for SPL boot.%s/altorithm/algorithm/g
Why do we need yet another algorithm for compression?
Doesn't Zstandard provide a much faster decompression? Cf. https://sites.google.com/site/powturbo/home/web-compression
Best regards
Heinrich
Quoted 15 lines
lib/Kconfig +1
-0+ config SPL_LZO bool "Enable LZO decompression support in SPL" helplib/Makefile +1
-0@@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ obj-$(CONFIG_$(SPL_)ZSTD) += zstd/ obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o obj-$(CONFIG_$(SPL_)LZO) += lzo/ +obj-$(CONFIG_$(SPL_)LZMA) += lzma/ obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.oQuoted 2 lines
lib/Makefile obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ · line 70
Quoted 2 lines
lib/Makefile +0
-0obj-$(CONFIG_LIBAVB) += libavb/Hi Heinrich,
I need an algorithm with the maximum compression ratio to reduce the size of the u-boot payload as much as possible.
I believe currently lzma and xz are the best choices. Refer to:
https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO
Quoted 7 lines
Quoted 3 lines
Although they will take longer time for compression and decompression, considering that the u-boot payload is usually small (<= 1MiB), there will not be significant effect on decompression time.
Where does zstd fall in the list however? Can you please try that and report back? Thanks!
My purpose is to reduce the size of u-boot binary, and my plan is to compress the payload and the SPL decompress it at runtime. I need smallest size here, not fastest decompression speed.
So there are two points must be taken into consideration:
the size of the decompressor (i.e. u-boot-spl.bin) and the size of the compressed payload (u-boot.bin.gz/bz2/...).
Below is my test results, including all compression algorithms supported by uImage, plus zstd:
Platform: mips - mt7628
Uncompressed size of u-boot.bin: 302736 (295KiB)
Commands for compression:
gzip -c -f -k -9
bzip2 -c -f -k -9
lzma -c -z -k -9
lzop -c -f -9
lz4 -f -9
zstd --ultra -f
Results:
algo. | spl.bin size | compressed size | % of uncomp | total size
------------------------------------------------------------------------
gzip | 26232 (25.6K) | 141207 (137KiB) | 46.64% | 167439 (163KiB)
------------------------------------------------------------------------
bzip2 | 30616 (29.8K) | 135403 (132KiB) | 44.72% | 166019 (162KiB)
------------------------------------------------------------------------
lzma | 18920 (18.4K) | 112660 (110KiB) | 37.21% | 131580 (128KiB)
------------------------------------------------------------------------
lzo | 13288 (12.9K) | 152409 (148KiB) | 50.34% | 165697 (161KiB)
------------------------------------------------------------------------
lz4 | 12408 (12.1K) | 167311 (163KiB) | 55.27% | 179719 (175KiB)
------------------------------------------------------------------------
zstd | 44824 (43.7K) | 149765 (146KiB) | 49.47% | 194589 (190KiB)
As you can see lzma has the best compression ratio. The is the reason I chose lzma here.
Best Regards
PATCH 11/16 lib: enable lzma decompression support for SPL build
Quoted 110 lines · Weijie Gao
Quoted 63 lines · Tom Rini
Quoted 62 lines · Weijie Gao
Quoted 49 lines · Heinrich Schuchardt
Quoted 13 lines · Weijie Gao
This patch enables LZMA decompression support for SPL build
Signed-off-by Weijie Gao <[email protected]>--- lib/Kconfig | 5 +++++ lib/Makefile | 1 + 2 files changed, 6 insertions(+)lib/Kconfig +0
-0lib/Kconfig config SPL_LZ4 · line 424
Quoted 7 lines
lib/Kconfig +4
-0@@ -418,6 +418,11 @@ config SPL_LZ4 fast compression and decompression speed. It belongs to the LZ77 family of byte-oriented compression schemes. +config SPL_LZMA + bool "Enable LZMA decompression support for SPL build" + help + This enables support for LZMA compression altorithm for SPL boot.%s/altorithm/algorithm/g
Why do we need yet another algorithm for compression?
Doesn't Zstandard provide a much faster decompression? Cf. https://sites.google.com/site/powturbo/home/web-compression
Best regards
Heinrich
Quoted 15 lines
lib/Kconfig +1
-0+ config SPL_LZO bool "Enable LZO decompression support in SPL" helplib/Makefile +1
-0@@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ obj-$(CONFIG_$(SPL_)ZSTD) += zstd/ obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o obj-$(CONFIG_$(SPL_)LZO) += lzo/ +obj-$(CONFIG_$(SPL_)LZMA) += lzma/ obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.oQuoted 2 lines
lib/Makefile obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ · line 70
Quoted 2 lines
lib/Makefile +0
-0obj-$(CONFIG_LIBAVB) += libavb/Hi Heinrich,
I need an algorithm with the maximum compression ratio to reduce the size of the u-boot payload as much as possible.
I believe currently lzma and xz are the best choices. Refer to:
https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZOQuoted 7 lines
Quoted 3 lines
Although they will take longer time for compression and decompression, considering that the u-boot payload is usually small (<= 1MiB), there will not be significant effect on decompression time.
Where does zstd fall in the list however? Can you please try that and report back? Thanks!
My purpose is to reduce the size of u-boot binary, and my plan is to compress the payload and the SPL decompress it at runtime. I need smallest size here, not fastest decompression speed.
So there are two points must be taken into consideration:
the size of the decompressor (i.e. u-boot-spl.bin) and the size of the compressed payload (u-boot.bin.gz/bz2/...).Below is my test results, including all compression algorithms supported by uImage, plus zstd:
Platform: mips - mt7628
Uncompressed size of u-boot.bin: 302736 (295KiB)
Commands for compression:
gzip -c -f -k -9
bzip2 -c -f -k -9
lzma -c -z -k -9
lzop -c -f -9
lz4 -f -9
zstd --ultra -fResults:
algo. | spl.bin size | compressed size | % of uncomp | total size
------------------------------------------------------------------------
gzip | 26232 (25.6K) | 141207 (137KiB) | 46.64% | 167439 (163KiB)
------------------------------------------------------------------------
bzip2 | 30616 (29.8K) | 135403 (132KiB) | 44.72% | 166019 (162KiB)
------------------------------------------------------------------------
lzma | 18920 (18.4K) | 112660 (110KiB) | 37.21% | 131580 (128KiB)
------------------------------------------------------------------------
lzo | 13288 (12.9K) | 152409 (148KiB) | 50.34% | 165697 (161KiB)
------------------------------------------------------------------------
lz4 | 12408 (12.1K) | 167311 (163KiB) | 55.27% | 179719 (175KiB)
------------------------------------------------------------------------
zstd | 44824 (43.7K) | 149765 (146KiB) | 49.47% | 194589 (190KiB)
Quoted 2 lines
As you can see lzma has the best compression ratio. The is the reason I chose lzma here.
Thanks, that's exactly the kind of data I was looking for.
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200110/e5894b33/attachment.sig>