Skip to contentSkip to conversation

Add compressed Image booting support

PATCH New Atish Patra 3 participants 12 messages Mar 6, 2020 – Apr 21, 2020

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v5 cover Add compressed Image booting support

This patch series extends booti to support compressed images as well. Following compressed images are supported for now.

lzma, lzo, bzip2, gz.

Other compression methods can easily be supported if required. The above compression methods are common ones that Linux kernel (ARM64/RISC-V) and U-Boot supports.

Changes from v4->v5
1. Moved back to explicit kernel_comp_size enviornemnt variable from filesize.
2. Rebased on top latest master.

Changes from v3->v4
1. Removed CONFIG_SYS_BOOTM_LEN usage.

Atish Patra (3):
lib: kconfig: Add option to set BZIP2 compression method
image: Add a common compression type detection function.
image: Add compressed Image parsing support in booti.

cmd/booti.c                | 40 ++++++++++++++++++++++++++-
common/image.c             | 23 ++++++++++++++++
doc/README.distro          | 12 +++++++++
doc/board/sifive/fu540.rst | 55 ++++++++++++++++++++++++++++++++++++++
include/image.h            | 21 +++++++++++++++
lib/Kconfig                |  5 ++++
6 files changed, 155 insertions(+), 1 deletion(-)

--
2.24.0

#1 lore raw

PATCH v5 1/3 lib: kconfig: Add option to set BZIP2 compression method

There is no way to select BZIP2 compression method. Add it under library/compression config where all other compression related configs are present.

---
 lib/Kconfig | 5 +++++
 1 file changed, 5 insertions(+)
lib/Kconfig +5 -0
index ab6aff710dd6..ff8e955f899a 100644
@@ -409,6 +409,11 @@ config GZIP
 	help
 	  This enables support for GZIP compression algorithm.
 
+config BZIP2
+	bool "Enable bzip2 decompression support"
+	help
+	  This enables support for BZIP2 compression algorithm.
+
 config ZLIB
 	bool
 	default y
Signature · 1 line
2.24.0
#2 lore raw

PATCH v5 1/3 lib: kconfig: Add option to set BZIP2 compression method

Quoted 6 lines · Atish Patra

There is no way to select BZIP2 compression method. Add it under library/compression config where all other compression related configs are present.

Applied to u-boot/master, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200421/b4080c89/attachment.sig>
#3 lore raw

PATCH v5 2/3 image: Add a common compression type detection function.

In reply to Atish Patra

Currently, there is no method that can detect compression types given a file. This is very useful where a compressed kernel image is loaded directly to the memory.

Inspect initial few bytes to figure out compression type of the image. It will be used in booti method for now but can be reused any other function in future as well.

---
 common/image.c  | 23 +++++++++++++++++++++++
 include/image.h | 21 +++++++++++++++++++++
 2 files changed, 44 insertions(+)
common/image.c +23 -0
index 94873cb6ed50..d8d14e871c64 100644
@@ -202,6 +202,14 @@ struct table_info {
 	const table_entry_t *table;
 };
 
+static const struct comp_magic_map image_comp[] = {
+	{	IH_COMP_BZIP2,	"bzip2",	{0x42, 0x5a},},
+	{	IH_COMP_GZIP,	"gzip",		{0x1f, 0x8b},},
+	{	IH_COMP_LZMA,	"lzma",		{0x5d, 0x00},},
+	{	IH_COMP_LZO,	"lzo",		{0x89, 0x4c},},
+	{	IH_COMP_NONE,	"none",		{},	},
+};
+
 static const struct table_info table_info[IH_COUNT] = {
 	{ "architecture", IH_ARCH_COUNT, uimage_arch },
 	{ "compression", IH_COMP_COUNT, uimage_comp },
@@ -407,6 +415,21 @@ static void print_decomp_msg(int comp_type, int type, bool is_xip)
 		printf("   Uncompressing %s\n", name);
 }
 
+int image_decomp_type(const unsigned char *buf, ulong len)
+{
+	const struct comp_magic_map *cmagic = image_comp;
+
+	if (len < 2)
+		return -EINVAL;
+
+	for (; cmagic->comp_id > 0; cmagic++) {
+		if (!memcmp(buf, cmagic->magic, 2))
+			break;
+	}
+
+	return cmagic->comp_id;
+}
+
 int image_decomp(int comp, ulong load, ulong image_start, int type,
 		 void *load_buf, void *image_buf, ulong image_len,
 		 uint unc_len, ulong *load_end)
include/image.h +21 -0
index b316d167d8d7..9bbb1acfb3af 100644
@@ -452,6 +452,15 @@ typedef struct table_entry {
 	char	*lname;		/* long (output) name to print for messages */
 } table_entry_t;
 
+/*
+ * Compression type and magic number mapping table.
+ */
+struct comp_magic_map {
+	int		comp_id;
+	const char	*name;
+	unsigned char	magic[2];
+};
+
 /*
  * get_table_entry_id() scans the translation table trying to find an
  * entry that matches the given short name. If a matching entry is
@@ -868,6 +877,18 @@ static inline int image_check_target_arch(const image_header_t *hdr)
 }
 #endif /* USE_HOSTCC */
 
+/**
+ * image_decomp_type() - Find out compression type of an image
+ *
+ * @buf:	Address in U-Boot memory where image is loaded.
+ * @len:	Length of the compressed image.
+ * @return	compression type or IH_COMP_NONE if not compressed.
+ *
+ * Note: Only following compression types are supported now.
+ * lzo, lzma, gzip, bzip2
+ */
+int image_decomp_type(const unsigned char *buf, ulong len);
+
 /**
  * image_decomp() - decompress an image
  *
Signature · 1 line
2.24.0
#4 lore raw

PATCH v5 2/3 image: Add a common compression type detection function.

Quoted 10 lines · Atish Patra

Currently, there is no method that can detect compression types given a file. This is very useful where a compressed kernel image is loaded directly to the memory.

Inspect initial few bytes to figure out compression type of the image. It will be used in booti method for now but can be reused any other function in future as well.

Applied to u-boot/master, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200421/6a48870f/attachment.sig>
#5 lore raw

PATCH v5 3/3 image: Add compressed Image parsing support in booti.

In reply to Atish Patra

Add compressed Image parsing support so that booti can parse both flat and compressed Image to boot Linux. Currently, it is difficult to calculate a safe address for every board where the compressed image can be decompressed. It is also not possible to figure out the size of the compressed file as well. Thus, user need to set two additional environment variables kernel_comp_addr_r and filesize to make this work.

Following compression methods are supported for now. lzma, lzo, bzip2, gzip.

lz4 support is not added as ARM64 kernel generates a lz4 compressed image with legacy header which U-Boot doesn't know how to parse and decompress.

Tested on HiFive Unleashed and Qemu for RISC-V.
Tested on Qemu for ARM64.

---
I could not test this patch on any ARM64 boards due to lack of access to any ARM64 board. If anybody can test it on ARM64, that would be great.

---
 cmd/booti.c                | 40 ++++++++++++++++++++++++++-
 doc/README.distro          | 12 +++++++++
 doc/board/sifive/fu540.rst | 55 ++++++++++++++++++++++++++++++++++++++
 3 files changed, 106 insertions(+), 1 deletion(-)
cmd/booti.c +39 -1
index de5058236e0a..4fff8cfcf69f 100644
@@ -14,6 +14,7 @@
 #include <linux/kernel.h>
 #include <linux/sizes.h>
 
+DECLARE_GLOBAL_DATA_PTR;
 /*
  * Image booting support
  */
@@ -24,6 +25,12 @@ static int booti_start(cmd_tbl_t *cmdtp, int flag, int argc,
 	ulong ld;
 	ulong relocated_addr;
 	ulong image_size;
+	uint8_t *temp;
+	ulong dest;
+	ulong dest_end;
+	unsigned long comp_len;
+	unsigned long decomp_len;
+	int ctype;
 
 	ret = do_bootm_states(cmdtp, flag, argc, argv, BOOTM_STATE_START,
 			      images, 1);
@@ -38,6 +45,33 @@ static int booti_start(cmd_tbl_t *cmdtp, int flag, int argc,
 		debug("*  kernel: cmdline image address = 0x%08lx\n", ld);
 	}
 
+	temp = map_sysmem(ld, 0);
+	ctype = image_decomp_type(temp, 2);
+	if (ctype > 0) {
+		dest = env_get_ulong("kernel_comp_addr_r", 16, 0);
+		comp_len = env_get_ulong("kernel_comp_size", 16, 0);
+		if (!dest || !comp_len) {
+			puts("kernel_comp_addr_r or kernel_comp_size is not provided!\n");
+			return -EINVAL;
+		}
+		if (dest < gd->ram_base || dest > gd->ram_top) {
+			puts("kernel_comp_addr_r is outside of DRAM range!\n");
+			return -EINVAL;
+		}
+
+		debug("kernel image compression type %d size = 0x%08lx address = 0x%08lx\n",
+			ctype, comp_len, (ulong)dest);
+		decomp_len = comp_len * 10;
+		ret = image_decomp(ctype, 0, ld, IH_TYPE_KERNEL,
+				 (void *)dest, (void *)ld, comp_len,
+				 decomp_len, &dest_end);
+		if (ret)
+			return ret;
+		/* dest_end contains the uncompressed Image size */
+		memmove((void *) ld, (void *)dest, dest_end);
+	}
+	unmap_sysmem((void *)ld);
+
 	ret = booti_setup(ld, &relocated_addr, &image_size, false);
 	if (ret != 0)
 		return 1;
@@ -100,10 +134,14 @@ int do_booti(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])
 #ifdef CONFIG_SYS_LONGHELP
 static char booti_help_text[] =
 	"[addr [initrd[:size]] [fdt]]\n"
-	"    - boot Linux 'Image' stored at 'addr'\n"
+	"    - boot Linux flat or compressed 'Image' stored at 'addr'\n"
 	"\tThe argument 'initrd' is optional and specifies the address\n"
 	"\tof an initrd in memory. The optional parameter ':size' allows\n"
 	"\tspecifying the size of a RAW initrd.\n"
+	"\tCurrently only booting from gz, bz2, lzma and lz4 compression\n"
+	"\ttypes are supported. In order to boot from any of these compressed\n"
+	"\timages, user have to set kernel_comp_addr_r and kernel_comp_size enviornment\n"
+	"\tvariables beforehand.\n"
 #if defined(CONFIG_OF_LIBFDT)
 	"\tSince booting a Linux kernel requires a flat device-tree, a\n"
 	"\tthird argument providing the address of the device-tree blob\n"
doc/README.distro +12 -0
index ab6e6f4e74be..5076bebd1808 100644
@@ -246,6 +246,18 @@ kernel_addr_r:
 
   A size of 16MB for the kernel is likely adequate.
 
+kernel_comp_addr_r:
+  Optional. This is only required if user wants to boot Linux from a compressed
+  Image(.gz, .bz2, .lzma, .lzo) using booti command. It represents the location
+  in RAM where the compressed Image will be decompressed temporarily. Once the
+  decompression is complete, decompressed data will be moved kernel_addr_r for
+  booting.
+
+kernel_comp_size:
+  Optional. This is only required if user wants to boot Linux from a compressed
+  Image using booti command. It represents the size of the compressed file. The
+  size has to at least the size of loaded image for decompression to succeed.
+
 pxefile_addr_r:
 
   Mandatory. The location in RAM where extlinux.conf will be loaded to prior
doc/board/sifive/fu540.rst +55 -0

67 lines not on this page · raw message

Signature · 1 line
2.24.0
#6 lore raw

PATCH v5 3/3 image: Add compressed Image parsing support in booti.

Quoted 8 lines · Atish Patra

Add compressed Image parsing support so that booti can parse both flat and compressed Image to boot Linux. Currently, it is difficult to calculate a safe address for every board where the compressed image can be decompressed. It is also not possible to figure out the size of the compressed file as well. Thus, user need to set two additional environment variables kernel_comp_addr_r and filesize to make this work.

Quoted 6 lines

Following compression methods are supported for now. lzma, lzo, bzip2, gzip.

lz4 support is not added as ARM64 kernel generates a lz4 compressed image with legacy header which U-Boot doesn't know how to parse and decompress.

Is that a "legacy lz4 header" or something else? Thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200306/76df7dd1/attachment.sig>
#7 lore raw

PATCH v5 3/3 image: Add compressed Image parsing support in booti.

Quoted 10 lines · Tom Rini
Quoted 8 lines · Atish Patra

Add compressed Image parsing support so that booti can parse both flat and compressed Image to boot Linux. Currently, it is difficult to calculate a safe address for every board where the compressed image can be decompressed. It is also not possible to figure out the size of the compressed file as well. Thus, user need to set two additional environment variables kernel_comp_addr_r and filesize to make this work.

Quoted 9 lines
Quoted 6 lines

Following compression methods are supported for now. lzma, lzo, bzip2, gzip.

lz4 support is not added as ARM64 kernel generates a lz4 compressed image with legacy header which U-Boot doesn't know how to parse and decompress.

Is that a "legacy lz4 header" or something else? Thanks!

Yeah. I was talking about legacy lz4 header as per U-Boot documentation.

Quoted 2 lines

--
Tom

--
Regards,
Atish

#8 lore raw

PATCH v5 3/3 image: Add compressed Image parsing support in booti.

In reply to Atish Patra
Quoted 11 lines · Atish Patra

Add compressed Image parsing support so that booti can parse both flat and compressed Image to boot Linux. Currently, it is difficult to calculate a safe address for every board where the compressed image can be decompressed. It is also not possible to figure out the size of the compressed file as well. Thus, user need to set two additional environment variables kernel_comp_addr_r and filesize to make this work.

Following compression methods are supported for now. lzma, lzo, bzip2, gzip.

Quoted 8 lines

lz4 support is not added as ARM64 kernel generates a lz4 compressed image with legacy header which U-Boot doesn't know how to parse and decompress.

Tested on HiFive Unleashed and Qemu for RISC-V.
Tested on Qemu for ARM64.

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200306/a33c0f4f/attachment.sig>
#9 lore raw

PATCH v5 3/3 image: Add compressed Image parsing support in booti.

In reply to Atish Patra
Quoted 15 lines · Atish Patra

Add compressed Image parsing support so that booti can parse both flat and compressed Image to boot Linux. Currently, it is difficult to calculate a safe address for every board where the compressed image can be decompressed. It is also not possible to figure out the size of the compressed file as well. Thus, user need to set two additional environment variables kernel_comp_addr_r and filesize to make this work.

Following compression methods are supported for now. lzma, lzo, bzip2, gzip.

lz4 support is not added as ARM64 kernel generates a lz4 compressed image with legacy header which U-Boot doesn't know how to parse and decompress.

Quoted 5 lines

Tested on HiFive Unleashed and Qemu for RISC-V.
Tested on Qemu for ARM64.

Applied to u-boot/master, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200421/e756cc6b/attachment.sig>
#10 lore raw

v5 cover Add compressed Image booting support

In reply to Atish Patra
Quoted 31 lines · Atish Patra

This patch series extends booti to support compressed images as well. Following compressed images are supported for now.

lzma, lzo, bzip2, gz.

Other compression methods can easily be supported if required. The above compression methods are common ones that Linux kernel (ARM64/RISC-V) and U-Boot supports.

Changes from v4->v5
1. Moved back to explicit kernel_comp_size enviornemnt variable from filesize.
2. Rebased on top latest master.

Changes from v3->v4
1. Removed CONFIG_SYS_BOOTM_LEN usage.

Atish Patra (3):
lib: kconfig: Add option to set BZIP2 compression method
image: Add a common compression type detection function.
image: Add compressed Image parsing support in booti.

cmd/booti.c                | 40 ++++++++++++++++++++++++++-
common/image.c             | 23 ++++++++++++++++
doc/README.distro          | 12 +++++++++
doc/board/sifive/fu540.rst | 55 ++++++++++++++++++++++++++++++++++++++
include/image.h            | 21 +++++++++++++++
lib/Kconfig                |  5 ++++
6 files changed, 155 insertions(+), 1 deletion(-)
Quoted 3 lines

--
2.24.0

The series is not in master yet. I am not sure about the merge window cycle for U-Boot.
I am assuming nothing else is required for this series to go in. Please let me know otherwise.

The patchwork shows it is delegated to Tom. I guess it will end up in master in sometime before the next release.
https://patchwork.ozlabs.org/project/uboot/list/?submitter=74477

--
Regards,
Atish

#11 lore raw

v5 cover Add compressed Image booting support

Quoted 36 lines · Atish Patra
Quoted 31 lines · Atish Patra

This patch series extends booti to support compressed images as well. Following compressed images are supported for now.

lzma, lzo, bzip2, gz.

Other compression methods can easily be supported if required. The above compression methods are common ones that Linux kernel (ARM64/RISC-V) and U-Boot supports.

Changes from v4->v5
1. Moved back to explicit kernel_comp_size enviornemnt variable from filesize.
2. Rebased on top latest master.

Changes from v3->v4
1. Removed CONFIG_SYS_BOOTM_LEN usage.

Atish Patra (3):
lib: kconfig: Add option to set BZIP2 compression method
image: Add a common compression type detection function.
image: Add compressed Image parsing support in booti.

cmd/booti.c                | 40 ++++++++++++++++++++++++++-
common/image.c             | 23 ++++++++++++++++
doc/README.distro          | 12 +++++++++
doc/board/sifive/fu540.rst | 55 ++++++++++++++++++++++++++++++++++++++
include/image.h            | 21 +++++++++++++++
lib/Kconfig                |  5 ++++
6 files changed, 155 insertions(+), 1 deletion(-)
Quoted 3 lines

--
2.24.0

Quoted 8 lines

The series is not in master yet. I am not sure about the merge window cycle for U-Boot.
I am assuming nothing else is required for this series to go in. Please let me know otherwise.

The patchwork shows it is delegated to Tom. I guess it will end up in master in sometime before the next release.
https://patchwork.ozlabs.org/project/uboot/list/?submitter=74477

It won't be in v2020.04, but it should be in v2020.07, I'll likely pull it to -next soon. Thanks again!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200327/3374a1f8/attachment.sig>