Add compressed Image booting support
Subsystems (2)
v5 cover Add compressed Image booting support
This patch series extends booti to support compressed images as well. Following compressed images are supported for now.
lzma, lzo, bzip2, gz.
Other compression methods can easily be supported if required. The above compression methods are common ones that Linux kernel (ARM64/RISC-V) and U-Boot supports.
Changes from v4->v5
1. Moved back to explicit kernel_comp_size enviornemnt variable from filesize.
2. Rebased on top latest master.
Changes from v3->v4
1. Removed CONFIG_SYS_BOOTM_LEN usage.
Atish Patra (3):
lib: kconfig: Add option to set BZIP2 compression method
image: Add a common compression type detection function.
image: Add compressed Image parsing support in booti.
cmd/booti.c | 40 ++++++++++++++++++++++++++- common/image.c | 23 ++++++++++++++++ doc/README.distro | 12 +++++++++ doc/board/sifive/fu540.rst | 55 ++++++++++++++++++++++++++++++++++++++ include/image.h | 21 +++++++++++++++ lib/Kconfig | 5 ++++ 6 files changed, 155 insertions(+), 1 deletion(-)
--
2.24.0
PATCH v5 1/3 lib: kconfig: Add option to set BZIP2 compression method
There is no way to select BZIP2 compression method. Add it under library/compression config where all other compression related configs are present.
--- lib/Kconfig | 5 +++++ 1 file changed, 5 insertions(+)
lib/Kconfig +5 -0
@@ -409,6 +409,11 @@ config GZIP help This enables support for GZIP compression algorithm. +config BZIP2 + bool "Enable bzip2 decompression support" + help + This enables support for BZIP2 compression algorithm. + config ZLIB bool default y
Signature · 1 line
2.24.0
PATCH v5 1/3 lib: kconfig: Add option to set BZIP2 compression method
Quoted 6 lines · Atish Patra
There is no way to select BZIP2 compression method. Add it under library/compression config where all other compression related configs are present.
Signed-off-by Atish Patra <[email protected]>Reviewed-by Tom Rini <[email protected]>
Applied to u-boot/master, thanks!
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 659 bytes Desc: not available URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200421/b4080c89/attachment.sig>
PATCH v5 2/3 image: Add a common compression type detection function.
In reply to Atish PatraCurrently, there is no method that can detect compression types given a file. This is very useful where a compressed kernel image is loaded directly to the memory.
Inspect initial few bytes to figure out compression type of the image. It will be used in booti method for now but can be reused any other function in future as well.
--- common/image.c | 23 +++++++++++++++++++++++ include/image.h | 21 +++++++++++++++++++++ 2 files changed, 44 insertions(+)
common/image.c +23 -0
@@ -202,6 +202,14 @@ struct table_info {const table_entry_t *table; }; +static const struct comp_magic_map image_comp[] = { + {IH_COMP_BZIP2 ,"bzip2" , {0x42 ,0x5a },}, + {IH_COMP_GZIP ,"gzip" , {0x1f ,0x8b },}, + {IH_COMP_LZMA ,"lzma" , {0x5d ,0x00 },}, + {IH_COMP_LZO ,"lzo" , {0x89 ,0x4c },}, + {IH_COMP_NONE ,"none" , {}, }, +}; +static const struct table_info table_info[IH_COUNT ] = { {"architecture" ,IH_ARCH_COUNT , uimage_arch }, {"compression" ,IH_COMP_COUNT , uimage_comp },
@@ -407,6 +415,21 @@ static void print_decomp_msg(int comp_type, int type, bool is_xip)printf (" Uncompressing %s\n" , name); } +int image_decomp_type (const unsigned char *buf,ulong len) +{ +const struct comp_magic_map *cmagic = image_comp; + +if (len <2 ) +return -EINVAL ; + +for (; cmagic->comp_id >0 ; cmagic++) { +if (!memcmp (buf, cmagic->magic,2 )) +break ; + } + +return cmagic->comp_id; +} +int image_decomp (int comp,ulong load,ulong image_start,int type,void *load_buf,void *image_buf,ulong image_len,uint unc_len,ulong *load_end)
include/image.h +21 -0
@@ -452,6 +452,15 @@ typedef struct table_entry {char *lname;/* long (output) name to print for messages */ } table_entry_t; +/* +* Compression type and magic number mapping table. +*/ +struct comp_magic_map { +int comp_id; +const char *name; +unsigned char magic[2 ]; +}; +/* * get_table_entry_id() scans the translation table trying to find an * entry that matches the given short name. If a matching entry is
@@ -868,6 +877,18 @@ static inline int image_check_target_arch(const image_header_t *hdr) }#endif /* USE_HOSTCC */ +/** +* image_decomp_type() - Find out compression type of an image +* +* @buf: Address in U-Boot memory where image is loaded. +* @len: Length of the compressed image. +* @return compression type or IH_COMP_NONE if not compressed. +* +* Note: Only following compression types are supported now. +* lzo, lzma, gzip, bzip2 +*/ +int image_decomp_type (const unsigned char *buf,ulong len); +/** * image_decomp() - decompress an image *
Signature · 1 line
2.24.0
PATCH v5 2/3 image: Add a common compression type detection function.
Quoted 10 lines · Atish Patra
Currently, there is no method that can detect compression types given a file. This is very useful where a compressed kernel image is loaded directly to the memory.
Inspect initial few bytes to figure out compression type of the image. It will be used in booti method for now but can be reused any other function in future as well.
Signed-off-by Atish Patra <[email protected]>Reviewed-by Tom Rini <[email protected]>
Applied to u-boot/master, thanks!
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 659 bytes Desc: not available URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200421/6a48870f/attachment.sig>
PATCH v5 3/3 image: Add compressed Image parsing support in booti.
In reply to Atish PatraAdd compressed Image parsing support so that booti can parse both flat and compressed Image to boot Linux. Currently, it is difficult to calculate a safe address for every board where the compressed image can be decompressed. It is also not possible to figure out the size of the compressed file as well. Thus, user need to set two additional environment variables kernel_comp_addr_r and filesize to make this work.
Following compression methods are supported for now. lzma, lzo, bzip2, gzip.
lz4 support is not added as ARM64 kernel generates a lz4 compressed image with legacy header which U-Boot doesn't know how to parse and decompress.
Tested on HiFive Unleashed and Qemu for RISC-V.
Tested on Qemu for ARM64.
---
I could not test this patch on any ARM64 boards due to lack of access to any ARM64 board. If anybody can test it on ARM64, that would be great.
--- cmd/booti.c | 40 ++++++++++++++++++++++++++- doc/README.distro | 12 +++++++++ doc/board/sifive/fu540.rst | 55 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 106 insertions(+), 1 deletion(-)
cmd/booti.c +39 -1
@@ -14,6 +14,7 @@#include <linux/kernel.h> #include <linux/sizes.h> +DECLARE_GLOBAL_DATA_PTR ;/* * Image booting support */
@@ -24,6 +25,12 @@ static int booti_start(cmd_tbl_t *cmdtp, int flag, int argc,ulong ld;ulong relocated_addr;ulong image_size; +uint8_t *temp; +ulong dest; +ulong dest_end; +unsigned long comp_len; +unsigned long decomp_len; +int ctype; ret =do_bootm_states (cmdtp, flag, argc, argv,BOOTM_STATE_START , images,1 );
@@ -38,6 +45,33 @@ static int booti_start(cmd_tbl_t *cmdtp, int flag, int argc,debug ("* kernel: cmdline image address = 0x%08lx\n" , ld); } + temp =map_sysmem (ld,0 ); + ctype =image_decomp_type (temp,2 ); +if (ctype >0 ) { + dest =env_get_ulong ("kernel_comp_addr_r" ,16 ,0 ); + comp_len =env_get_ulong ("kernel_comp_size" ,16 ,0 ); +if (!dest || !comp_len) { +puts ("kernel_comp_addr_r or kernel_comp_size is not provided!\n" ); +return -EINVAL ; + } +if (dest < gd->ram_base || dest > gd->ram_top) { +puts ("kernel_comp_addr_r is outside of DRAM range!\n" ); +return -EINVAL ; + } + +debug ("kernel image compression type %d size = 0x%08lx address = 0x%08lx\n" , + ctype, comp_len, (ulong )dest); + decomp_len = comp_len *10 ; + ret =image_decomp (ctype,0 , ld,IH_TYPE_KERNEL , + (void *)dest, (void *)ld, comp_len, + decomp_len, &dest_end); +if (ret) +return ret; +/* dest_end contains the uncompressed Image size */ +memmove ((void *) ld, (void *)dest, dest_end); + } +unmap_sysmem ((void *)ld); + ret =booti_setup (ld, &relocated_addr, &image_size, false);if (ret !=0 )return 1 ;
@@ -100,10 +134,14 @@ int do_booti(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])#ifdef CONFIG_SYS_LONGHELP static char booti_help_text[] ="[addr [initrd[:size]] [fdt]]\n" -" - boot Linux 'Image' stored at 'addr'\n" +" - boot Linux flat or compressed 'Image' stored at 'addr'\n" "\tThe argument 'initrd' is optional and specifies the address\n" "\tof an initrd in memory. The optional parameter ':size' allows\n" "\tspecifying the size of a RAW initrd.\n" +"\tCurrently only booting from gz, bz2, lzma and lz4 compression\n" +"\ttypes are supported. In order to boot from any of these compressed\n" +"\timages, user have to set kernel_comp_addr_r and kernel_comp_size enviornment\n" + "\tvariables beforehand.\n"#if defined(CONFIG_OF_LIBFDT )"\tSince booting a Linux kernel requires a flat device-tree, a\n" "\tthird argument providing the address of the device-tree blob\n"
doc/README.distro +12 -0
@@ -246,6 +246,18 @@ kernel_addr_r: A size of 16MB for the kernel is likely adequate. +kernel_comp_addr_r: + Optional. This is only required if user wants to boot Linux from a compressed + Image(.gz, .bz2, .lzma, .lzo) using booti command. It represents the location + in RAM where the compressed Image will be decompressed temporarily. Once the + decompression is complete, decompressed data will be moved kernel_addr_r for + booting. + +kernel_comp_size: + Optional. This is only required if user wants to boot Linux from a compressed + Image using booti command. It represents the size of the compressed file. The + size has to at least the size of loaded image for decompression to succeed. + pxefile_addr_r: Mandatory. The location in RAM where extlinux.conf will be loaded to prior
doc/board/sifive/fu540.rst +55 -0
67 lines not on this page · raw message
Signature · 1 line
2.24.0
PATCH v5 3/3 image: Add compressed Image parsing support in booti.
Quoted 8 lines · Atish Patra
Add compressed Image parsing support so that booti can parse both flat and compressed Image to boot Linux. Currently, it is difficult to calculate a safe address for every board where the compressed image can be decompressed. It is also not possible to figure out the size of the compressed file as well. Thus, user need to set two additional environment variables
kernel_comp_addr_rand filesize to make this work.
Quoted 6 lines
Following compression methods are supported for now. lzma, lzo, bzip2, gzip.
lz4 support is not added as ARM64 kernel generates a lz4 compressed image with legacy header which U-Boot doesn't know how to parse and decompress.
Is that a "legacy lz4 header" or something else? Thanks!
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 659 bytes Desc: not available URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200306/76df7dd1/attachment.sig>
PATCH v5 3/3 image: Add compressed Image parsing support in booti.
Quoted 10 lines · Tom Rini
Quoted 8 lines · Atish Patra
Add compressed Image parsing support so that booti can parse both flat and compressed Image to boot Linux. Currently, it is difficult to calculate a safe address for every board where the compressed image can be decompressed. It is also not possible to figure out the size of the compressed file as well. Thus, user need to set two additional environment variables
kernel_comp_addr_rand filesize to make this work.
Quoted 9 lines
Quoted 6 lines
Following compression methods are supported for now. lzma, lzo, bzip2, gzip.
lz4 support is not added as ARM64 kernel generates a lz4 compressed image with legacy header which U-Boot doesn't know how to parse and decompress.
Is that a "legacy lz4 header" or something else? Thanks!
Yeah. I was talking about legacy lz4 header as per U-Boot documentation.
Quoted 2 lines
--
Tom
--
Regards,
Atish
PATCH v5 3/3 image: Add compressed Image parsing support in booti.
In reply to Atish PatraQuoted 11 lines · Atish Patra
Add compressed Image parsing support so that booti can parse both flat and compressed Image to boot Linux. Currently, it is difficult to calculate a safe address for every board where the compressed image can be decompressed. It is also not possible to figure out the size of the compressed file as well. Thus, user need to set two additional environment variables
kernel_comp_addr_rand filesize to make this work.Following compression methods are supported for now. lzma, lzo, bzip2, gzip.
Quoted 8 lines
lz4 support is not added as ARM64 kernel generates a lz4 compressed image with legacy header which U-Boot doesn't know how to parse and decompress.
Tested on HiFive Unleashed and Qemu for RISC-V.
Tested on Qemu for ARM64.Signed-off-by Atish Patra <[email protected]>
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 659 bytes Desc: not available URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200306/a33c0f4f/attachment.sig>
PATCH v5 3/3 image: Add compressed Image parsing support in booti.
In reply to Atish PatraQuoted 15 lines · Atish Patra
Add compressed Image parsing support so that booti can parse both flat and compressed Image to boot Linux. Currently, it is difficult to calculate a safe address for every board where the compressed image can be decompressed. It is also not possible to figure out the size of the compressed file as well. Thus, user need to set two additional environment variables
kernel_comp_addr_rand filesize to make this work.Following compression methods are supported for now. lzma, lzo, bzip2, gzip.
lz4 support is not added as ARM64 kernel generates a lz4 compressed image with legacy header which U-Boot doesn't know how to parse and decompress.
Quoted 5 lines
Tested on HiFive Unleashed and Qemu for RISC-V.
Tested on Qemu for ARM64.Signed-off-by Atish Patra <[email protected]>Reviewed-by Tom Rini <[email protected]>
Applied to u-boot/master, thanks!
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 659 bytes Desc: not available URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200421/e756cc6b/attachment.sig>
v5 cover Add compressed Image booting support
In reply to Atish PatraQuoted 31 lines · Atish Patra
This patch series extends booti to support compressed images as well. Following compressed images are supported for now.
lzma, lzo, bzip2, gz.
Other compression methods can easily be supported if required. The above compression methods are common ones that Linux kernel (ARM64/RISC-V) and U-Boot supports.
Changes from
v4->v5
1. Moved back to explicitkernel_comp_sizeenviornemnt variable from filesize.
2. Rebased on top latest master.Changes from
v3->v4
1. RemovedCONFIG_SYS_BOOTM_LENusage.Atish Patra (3):
lib: kconfig: Add option to set BZIP2 compression method
image: Add a common compression type detection function.
image: Add compressed Image parsing support in booti.cmd/booti.c | 40 ++++++++++++++++++++++++++- common/image.c | 23 ++++++++++++++++ doc/README.distro | 12 +++++++++ doc/board/sifive/fu540.rst | 55 ++++++++++++++++++++++++++++++++++++++ include/image.h | 21 +++++++++++++++ lib/Kconfig | 5 ++++ 6 files changed, 155 insertions(+), 1 deletion(-)
Quoted 3 lines
--
2.24.0
The series is not in master yet. I am not sure about the merge window cycle for U-Boot.
I am assuming nothing else is required for this series to go in. Please let me know otherwise.
The patchwork shows it is delegated to Tom. I guess it will end up in master in sometime before the next release.
https://patchwork.ozlabs.org/project/uboot/list/?submitter=74477
--
Regards,
Atish
v5 cover Add compressed Image booting support
Quoted 36 lines · Atish Patra
Quoted 31 lines · Atish Patra
This patch series extends booti to support compressed images as well. Following compressed images are supported for now.
lzma, lzo, bzip2, gz.
Other compression methods can easily be supported if required. The above compression methods are common ones that Linux kernel (ARM64/RISC-V) and U-Boot supports.
Changes from
v4->v5
1. Moved back to explicitkernel_comp_sizeenviornemnt variable from filesize.
2. Rebased on top latest master.Changes from
v3->v4
1. RemovedCONFIG_SYS_BOOTM_LENusage.Atish Patra (3):
lib: kconfig: Add option to set BZIP2 compression method
image: Add a common compression type detection function.
image: Add compressed Image parsing support in booti.cmd/booti.c | 40 ++++++++++++++++++++++++++- common/image.c | 23 ++++++++++++++++ doc/README.distro | 12 +++++++++ doc/board/sifive/fu540.rst | 55 ++++++++++++++++++++++++++++++++++++++ include/image.h | 21 +++++++++++++++ lib/Kconfig | 5 ++++ 6 files changed, 155 insertions(+), 1 deletion(-)Quoted 3 lines
--
2.24.0
Quoted 8 lines
The series is not in master yet. I am not sure about the merge window cycle for U-Boot.
I am assuming nothing else is required for this series to go in. Please let me know otherwise.The patchwork shows it is delegated to Tom. I guess it will end up in master in sometime before the next release.
https://patchwork.ozlabs.org/project/uboot/list/?submitter=74477
It won't be in v2020.04, but it should be in v2020.07, I'll likely pull it to -next soon. Thanks again!
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 659 bytes Desc: not available URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200327/3374a1f8/attachment.sig>