Skip to contentSkip to conversation

linux: show compression options only on some architectures

PATCH New Thomas Petazzoni 3 participants 6 messages Sep 24, 2020 – Sep 25, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 linux: show compression options only on some architectures

Our Linux kernel compression options have the effect of enabling CONFIG_KERNEL_{GZIP,LZMA,LZ4,...}. However, those configuration options only exist on a subset of the CPU architectures, and it is confusing to have such a choice visible when it in fact has no effect. For example on ARM64, it has absolutely no effect, and the way to create a compressed image is different, and the image is not self-extractible.

So, we introduce a BR2_LINUX_KERNEL_ARCH_HAS_COMPRESSION to know which architectures have compression support. Of course, it's never going to be perfect as the set of architectures supporting compression might change over time. But at least, it will allow us to hide this option on architectures that do not at this point support compression.

---
 linux/Config.in | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
linux/Config.in +18 -0
index 49da2b81b2..9f2af6da5b 100644
@@ -299,6 +299,22 @@ endchoice
 # Kernel compression format
 #
 
+# Indicates whether this architecture has support for built-in kernel
+# compression, i.e architectures that select one of
+# HAVE_KERNEL_{GZIP,LZMA,LZ4,LZO,XZ,UNCOMPRESSED}
+config BR2_LINUX_KERNEL_ARCH_HAS_COMPRESSION
+	bool
+	default y if BR2_arcle || BR2_arceb
+	default y if BR2_arm || BR2_armeb
+	default y if BR2_csky
+	default y if BR2_i386
+	default y if BR2_mips || BR2_mipsel || BR2_mips64 || BR2_mips64el
+	default y if BR2_powerpc || BR2_powerpc64 || BR2_powerpc64le
+	default y if BR2_sh
+	default y if BR2_x86_64
+
+if BR2_LINUX_KERNEL_ARCH_HAS_COMPRESSION
+
 choice
 	prompt "Kernel compression format"
 	help
@@ -326,6 +342,8 @@ config BR2_LINUX_KERNEL_ZSTD
 
 endchoice
 
+endif
+
 config BR2_LINUX_KERNEL_IMAGE_TARGET_NAME
 	string "Kernel image target name"
 	depends on BR2_LINUX_KERNEL_IMAGE_TARGET_CUSTOM
Signature · 1 line
2.26.2
#1 lore raw

PATCH 2/2 linux: support Image.gz on arm64 and riscv

Both arm64 and riscv allow generating a compressed Image.gz file, so let's support that as a first class citizen.

---
 linux/Config.in | 4 ++++
 linux/linux.mk  | 2 ++
 2 files changed, 6 insertions(+)
linux/Config.in +4 -0
index 9f2af6da5b..73b9c30b26 100644
@@ -262,6 +262,10 @@ config BR2_LINUX_KERNEL_IMAGE
 	bool "Image"
 	depends on BR2_aarch64 || BR2_riscv
 
+config BR2_LINUX_KERNEL_IMAGE_GZ
+	bool "Image.gz"
+	depends on BR2_aarch64 || BR2_riscv
+
 config BR2_LINUX_KERNEL_LINUX_BIN
 	bool "linux.bin"
 	depends on BR2_microblaze
linux/linux.mk +2 -0
index e07e014d1e..133023f78c 100644
@@ -201,6 +201,8 @@ else ifeq ($(BR2_LINUX_KERNEL_SIMPLEIMAGE),y)
 LINUX_IMAGE_NAME = simpleImage.$(firstword $(LINUX_DTS_NAME))
 else ifeq ($(BR2_LINUX_KERNEL_IMAGE),y)
 LINUX_IMAGE_NAME = Image
+else ifeq ($(BR2_LINUX_KERNEL_IMAGE_GZ),y)
+LINUX_IMAGE_NAME = Image.gz
 else ifeq ($(BR2_LINUX_KERNEL_LINUX_BIN),y)
 LINUX_IMAGE_NAME = linux.bin
 else ifeq ($(BR2_LINUX_KERNEL_VMLINUX_BIN),y)
Signature · 1 line
2.26.2
#2 lore raw

PATCH 2/2 linux: support Image.gz on arm64 and riscv

Quoted 34 lines · Thomas Petazzoni

Both arm64 and riscv allow generating a compressed Image.gz file, so let's support that as a first class citizen.

---
?linux/Config.in | 4 ++++
?linux/linux.mk??| 2 ++
?2 files changed, 6 insertions(+)

linux/Config.in +0 -0
index 9f2af6da5b..73b9c30b26 100644
@@ -262,6 +262,10 @@ config BR2_LINUX_KERNEL_IMAGE

? bool "Image"
? depends on BR2_aarch64 || BR2_riscv
?

linux/Config.in +4 -0
+config BR2_LINUX_KERNEL_IMAGE_GZ
+	bool "Image.gz"
+	depends on BR2_aarch64 || BR2_riscv
+

?config BR2_LINUX_KERNEL_LINUX_BIN
? bool "linux.bin"
? depends on BR2_microblaze

linux/linux.mk +0 -0
index e07e014d1e..133023f78c 100644
@@ -201,6 +201,8 @@ else ifeq ($(BR2_LINUX_KERNEL_SIMPLEIMAGE),y)
 ?LINUX_IMAGE_NAME = simpleImage.$(firstword $(LINUX_DTS_NAME))
 ?else ifeq ($(BR2_LINUX_KERNEL_IMAGE),y)

?LINUX_IMAGE_NAME = Image

linux/linux.mk +2 -0
+else ifeq ($(BR2_LINUX_KERNEL_IMAGE_GZ),y)
+LINUX_IMAGE_NAME = Image.gz
Quoted 3 lines

?else ifeq ($(BR2_LINUX_KERNEL_LINUX_BIN),y)
?LINUX_IMAGE_NAME = linux.bin
?else ifeq ($(BR2_LINUX_KERNEL_VMLINUX_BIN),y)

#3 lore raw

PATCH 1/2 linux: show compression options only on some architectures

In reply to Thomas Petazzoni

Thomas, All,

On 2020-09-24 14:22 +0200, Thomas Petazzoni spake thusly:

Quoted 32 lines

Our Linux kernel compression options have the effect of enabling CONFIG_KERNEL_{GZIP,LZMA,LZ4,...}. However, those configuration options only exist on a subset of the CPU architectures, and it is confusing to have such a choice visible when it in fact has no effect. For example on ARM64, it has absolutely no effect, and the way to create a compressed image is different, and the image is not self-extractible.

So, we introduce a BR2_LINUX_KERNEL_ARCH_HAS_COMPRESSION to know which architectures have compression support. Of course, it's never going to be perfect as the set of architectures supporting compression might change over time. But at least, it will allow us to hide this option on architectures that do not at this point support compression.

---
 linux/Config.in | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
linux/Config.in +5 -0
index 49da2b81b2..9f2af6da5b 100644
@@ -299,6 +299,22 @@ endchoice
 # Kernel compression format
 #
 
+# Indicates whether this architecture has support for built-in kernel
+# compression, i.e architectures that select one of
+# HAVE_KERNEL_{GZIP,LZMA,LZ4,LZO,XZ,UNCOMPRESSED}
+config BR2_LINUX_KERNEL_ARCH_HAS_COMPRESSION
+	bool

linux/Config.in endchoice · line 316

Quoted 10 lines
linux/Config.in +10 -0
+	default y if BR2_arcle || BR2_arceb
+	default y if BR2_arm || BR2_armeb
+	default y if BR2_csky
+	default y if BR2_i386
+	default y if BR2_mips || BR2_mipsel || BR2_mips64 || BR2_mips64el
+	default y if BR2_powerpc || BR2_powerpc64 || BR2_powerpc64le
+	default y if BR2_sh
+	default y if BR2_x86_64
+
+if BR2_LINUX_KERNEL_ARCH_HAS_COMPRESSION

So, if the arch does not support kernel image compression, we will have none of the compression symbols defined, which means that, in linux.mk, we would have:

    104 LINUX_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_GZIP) += CONFIG_KERNEL_GZIP
    105 LINUX_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_LZ4) += CONFIG_KERNEL_LZ4
    106 LINUX_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_LZMA) += CONFIG_KERNEL_LZMA
    107 LINUX_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_LZO) += CONFIG_KERNEL_LZO
    108 LINUX_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_XZ) += CONFIG_KERNEL_XZ
    109 LINUX_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_ZSTD) += CONFIG_KERNEL_ZSTD
    [--SNIP--]
    319 define LINUX_KCONFIG_FIXUP_CMDS
    322     $(call KCONFIG_ENABLE_OPT,$(strip $(LINUX_COMPRESSION_OPT_y)))

But then, LINUX_COMPRESSION_OPT_y would be empty, so we'd call KCONFIG_ENABLE_OPT with an empty string, which would expand to:

    $(call KCONFIG_MUNGE_DOT_CONFIG, $(1), $(1)=y, $(2))

... and replacing the parameters would give (I'm pretty sure you'll see where we're going now):

    $(call KCONFIG_MUNGE_DOT_CONFIG, , =y, $(2))

When KCONFIG_MUNGE_DOT_CONFIG is expanded, we'd get:

    $(SED) "/\\<$(strip $(1))\\>/d" $(call KCONFIG_DOT_CONFIG,$(3))
    echo '$(strip $(2))' >> $(call KCONFIG_DOT_CONFIG,$(3))

... which would eventualy lead us to:

    $(SED) "/\\<)\\>/d" /some/.config-file
    echo '=y' >> /some/.config-file

And to be honest, I am not sure that last line would generate a fully legit kconfig setting... ;-)

Of course, running 'make oldconfig' on that modified .config would drop invalid lines, so you would not notivce the issue, but still this is not correct...

Regards,
Yann E. MORIN.

Quoted 19 lines
linux/Config.in +2 -0
 choice
 	prompt "Kernel compression format"
 	help
@@ -326,6 +342,8 @@ config BR2_LINUX_KERNEL_ZSTD
 
 endchoice
 
+endif
+
 config BR2_LINUX_KERNEL_IMAGE_TARGET_NAME
 	string "Kernel image target name"
 	depends on BR2_LINUX_KERNEL_IMAGE_TARGET_CUSTOM
Signature · 6 lines
2.26.2

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#4 lore raw

PATCH 1/2 linux: show compression options only on some architectures

Hello,

Quoted 34 lines · On Fri, 25 Sep 2020 09:37:58 +0200 "Yann E. MORIN" <[email protected]>…

So, if the arch does not support kernel image compression, we will have none of the compression symbols defined, which means that, in linux.mk, we would have:

    104 LINUX_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_GZIP) += CONFIG_KERNEL_GZIP
    105 LINUX_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_LZ4) += CONFIG_KERNEL_LZ4
    106 LINUX_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_LZMA) += CONFIG_KERNEL_LZMA
    107 LINUX_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_LZO) += CONFIG_KERNEL_LZO
    108 LINUX_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_XZ) += CONFIG_KERNEL_XZ
    109 LINUX_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_ZSTD) += CONFIG_KERNEL_ZSTD
    [--SNIP--]
    319 define LINUX_KCONFIG_FIXUP_CMDS
    322     $(call KCONFIG_ENABLE_OPT,$(strip $(LINUX_COMPRESSION_OPT_y)))

But then, LINUX_COMPRESSION_OPT_y would be empty, so we'd call KCONFIG_ENABLE_OPT with an empty string, which would expand to:

    $(call KCONFIG_MUNGE_DOT_CONFIG, $(1), $(1)=y, $(2))

... and replacing the parameters would give (I'm pretty sure you'll see where we're going now):

    $(call KCONFIG_MUNGE_DOT_CONFIG, , =y, $(2))

When KCONFIG_MUNGE_DOT_CONFIG is expanded, we'd get:

    $(SED) "/\\<$(strip $(1))\\>/d" $(call KCONFIG_DOT_CONFIG,$(3))
    echo '$(strip $(2))' >> $(call KCONFIG_DOT_CONFIG,$(3))

... which would eventualy lead us to:

    $(SED) "/\\<)\\>/d" /some/.config-file
    echo '=y' >> /some/.config-file
Quoted 6 lines

And to be honest, I am not sure that last line would generate a fully legit kconfig setting... ;-)

Of course, running 'make oldconfig' on that modified .config would drop invalid lines, so you would not notivce the issue, but still this is not correct...

Thanks for spotting this, I'll fix in v2. That being said, one can notice that you're pointing out that this bogus =y line will be dropped... just as equally as the bogus CONFIG_KERNEL_GZIP=y that we have today gets dropped on arm64 and other architectures for which those CONFIG_KERNEL_{GZIP,LZMA,LZ4,...} options don't exist :-)

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#5 lore raw

PATCH 1/2 linux: show compression options only on some architectures

Thomas, All,

On 2020-09-25 09:45 +0200, Thomas Petazzoni spake thusly:

Quoted 2 lines

On Fri, 25 Sep 2020 09:37:58 +0200
"Yann E. MORIN" <[email protected]> wrote:

[--SNIP--]

Quoted 7 lines
Quoted 7 lines
    $(SED) "/\\<)\\>/d" /some/.config-file
    echo '=y' >> /some/.config-file

And to be honest, I am not sure that last line would generate a fully legit kconfig setting... ;-)
Of course, running 'make oldconfig' on that modified .config would drop invalid lines, so you would not notivce the issue, but still this is not correct...

Quoted 5 lines

Thanks for spotting this, I'll fix in v2. That being said, one can notice that you're pointing out that this bogus =y line will be dropped... just as equally as the bogus CONFIG_KERNEL_GZIP=y that we have today gets dropped on arm64 and other architectures for which those CONFIG_KERNEL_{GZIP,LZMA,LZ4,...} options don't exist :-)

Yes, but an unknown symbol assignment is still valid syntax, and silently discarded by kconfig, because that is the goal of running
oldconfig: assigning new symbols and dropping old ones.

But the '=y' is not even syntactically valid. kconfig is still lenient about that, though (but it does warn, doesn't it?), but this is not a reason to generate an invalid file.

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'