Skip to contentSkip to conversation

Makefile.debug: support for -gz=zstd

PATCH Superseded Nick Desaulniers 6 participants 13 messages Oct 20, 2022 – Jan 3, 2023

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Makefile.debug: support for -gz=zstd

Make DEBUG_INFO_COMPRESSED a choice; DEBUG_INFO_UNCOMPRESSED is the default, DEBUG_INFO_COMPRESSED uses zlib, DEBUG_INFO_COMPRESSED_ZSTD uses zstd.

Some quick N=1 measurements with du, /usr/bin/time -v, and bloaty:

clang-16, x86_64 defconfig plus CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_UNCOMPRESSED=y:
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:55.43 488M vmlinux

27.6%   136Mi   0.0%       0    .debug_info
 6.1%  30.2Mi   0.0%       0    .debug_str_offsets
 3.5%  17.2Mi   0.0%       0    .debug_line
 3.3%  16.3Mi   0.0%       0    .debug_loclists
 0.9%  4.62Mi   0.0%       0    .debug_str

clang-16, x86_64 defconfig plus CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED=y (zlib):
Elapsed (wall clock) time (h:mm:ss or m:ss): 1:00.35 385M vmlinux

21.8%  85.4Mi   0.0%       0    .debug_info
 2.1%  8.26Mi   0.0%       0    .debug_str_offsets
 2.1%  8.24Mi   0.0%       0    .debug_loclists
 1.9%  7.48Mi   0.0%       0    .debug_line
 0.5%  1.94Mi   0.0%       0    .debug_str

clang-16, x86_64 defconfig plus
CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_ZSTD=y (zstd):
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:59.69 373M vmlinux

21.4%  81.4Mi   0.0%       0    .debug_info
 2.3%  8.85Mi   0.0%       0    .debug_loclists
 1.5%  5.71Mi   0.0%       0    .debug_line
 0.5%  1.95Mi   0.0%       0    .debug_str_offsets
 0.4%  1.62Mi   0.0%       0    .debug_str

That's only a 3.11% overall binary size savings over zlib, but at no performance regression.

---
 lib/Kconfig.debug      | 26 +++++++++++++++++++++++++-
 scripts/Makefile.debug |  4 ++++
 2 files changed, 29 insertions(+), 1 deletion(-)
lib/Kconfig.debug +25 -1
index 3fc7abffc7aa..4085ac77dc12 100644
@@ -312,8 +312,22 @@ config DEBUG_INFO_REDUCED
 	  DEBUG_INFO build and compile times are reduced too.
 	  Only works with newer gcc versions.
 
+choice
+	prompt "Compressed Debug information"
+	depends on DEBUG_KERNEL
+	help
+	  Compress the resulting debug info. Results in smaller debug info sections,
+	  but requires that consumers are able to decompress the results.
+
+	  If unsure, choose DEBUG_INFO_UNCOMPRESSED.
+
+config DEBUG_INFO_UNCOMPRESSED
+	bool "Don't compress debug information"
+	help
+	  Don't compress debug info sections.
+
 config DEBUG_INFO_COMPRESSED
-	bool "Compressed debugging information"
+	bool "Compress debugging information with zlib"
 	depends on $(cc-option,-gz=zlib)
 	depends on $(ld-option,--compress-debug-sections=zlib)
 	help
@@ -327,6 +341,16 @@ config DEBUG_INFO_COMPRESSED
 	  preferable to setting $KDEB_COMPRESS to "none" which would be even
 	  larger.
 
+config DEBUG_INFO_COMPRESSED_ZSTD
+	bool "Compress debugging information with zstd"
+	depends on $(cc-option,-gz=zstd)
+	depends on $(ld-option,--compress-debug-sections=zstd)
+	help
+	  Compress the debug information using zstd.  Requires GCC 13.0+ or Clang
+	  16.0+, binutils 2.40+, and zstd.
+
+endchoice # "Compressed Debug information"
+
 config DEBUG_INFO_SPLIT
 	bool "Produce split debuginfo in .dwo files"
 	depends on $(cc-option,-gsplit-dwarf)
scripts/Makefile.debug +4 -0
index 332c486f705f..8ac3379d2255 100644
@@ -31,6 +31,10 @@ ifdef CONFIG_DEBUG_INFO_COMPRESSED
 DEBUG_CFLAGS	+= -gz=zlib
 KBUILD_AFLAGS	+= -gz=zlib
 KBUILD_LDFLAGS	+= --compress-debug-sections=zlib
+else ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZSTD
+DEBUG_CFLAGS	+= -gz=zstd
+KBUILD_AFLAGS	+= -gz=zstd
+KBUILD_LDFLAGS	+= --compress-debug-sections=zstd
 endif
 
 KBUILD_CFLAGS	+= $(DEBUG_CFLAGS)
Signature · 1 line
2.38.0.135.g90850a2211-goog
maintainer Reviewed-by on v1 1/1
#1 lore raw
Quoted 36 lines · Nick Desaulniers

Make DEBUG_INFO_COMPRESSED a choice; DEBUG_INFO_UNCOMPRESSED is the default, DEBUG_INFO_COMPRESSED uses zlib, DEBUG_INFO_COMPRESSED_ZSTD uses zstd.

Some quick N=1 measurements with du, /usr/bin/time -v, and bloaty:

clang-16, x86_64 defconfig plus CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_UNCOMPRESSED=y:
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:55.43 488M vmlinux

27.6%   136Mi   0.0%       0    .debug_info
 6.1%  30.2Mi   0.0%       0    .debug_str_offsets
 3.5%  17.2Mi   0.0%       0    .debug_line
 3.3%  16.3Mi   0.0%       0    .debug_loclists
 0.9%  4.62Mi   0.0%       0    .debug_str

clang-16, x86_64 defconfig plus CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED=y (zlib):
Elapsed (wall clock) time (h:mm:ss or m:ss): 1:00.35 385M vmlinux

21.8%  85.4Mi   0.0%       0    .debug_info
 2.1%  8.26Mi   0.0%       0    .debug_str_offsets
 2.1%  8.24Mi   0.0%       0    .debug_loclists
 1.9%  7.48Mi   0.0%       0    .debug_line
 0.5%  1.94Mi   0.0%       0    .debug_str

clang-16, x86_64 defconfig plus
CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_ZSTD=y (zstd):
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:59.69 373M vmlinux

21.4%  81.4Mi   0.0%       0    .debug_info
 2.3%  8.85Mi   0.0%       0    .debug_loclists
 1.5%  5.71Mi   0.0%       0    .debug_line
 0.5%  1.95Mi   0.0%       0    .debug_str_offsets
 0.4%  1.62Mi   0.0%       0    .debug_str
Quoted 7 lines

That's only a 3.11% overall binary size savings over zlib, but at no performance regression.

One small comment below.

Quoted 10 lines
---
 lib/Kconfig.debug      | 26 +++++++++++++++++++++++++-
 scripts/Makefile.debug |  4 ++++
 2 files changed, 29 insertions(+), 1 deletion(-)
lib/Kconfig.debug +0 -0
index 3fc7abffc7aa..4085ac77dc12 100644

lib/Kconfig.debug config DEBUG_INFO_REDUCED · line 317

Quoted 6 lines
lib/Kconfig.debug +3 -0
@@ -312,8 +312,22 @@ config DEBUG_INFO_REDUCED
 	  DEBUG_INFO build and compile times are reduced too.
 	  Only works with newer gcc versions.
 
+choice
+	prompt "Compressed Debug information"
+	depends on DEBUG_KERNEL

I think you can drop this depends. The entire block is in an 'if DEBUG_INFO', which can only be true if CONFIG_DEBUG_KERNEL is set because of the dependencies of the "Debug information" prompt above this file, which is how CONFIG_DEBUG_INFO is selected.

Quoted 52 lines
lib/Kconfig.debug +22 -1
+	help
+	  Compress the resulting debug info. Results in smaller debug info sections,
+	  but requires that consumers are able to decompress the results.
+
+	  If unsure, choose DEBUG_INFO_UNCOMPRESSED.
+
+config DEBUG_INFO_UNCOMPRESSED
+	bool "Don't compress debug information"
+	help
+	  Don't compress debug info sections.
+
 config DEBUG_INFO_COMPRESSED
-	bool "Compressed debugging information"
+	bool "Compress debugging information with zlib"
 	depends on $(cc-option,-gz=zlib)
 	depends on $(ld-option,--compress-debug-sections=zlib)
 	help
@@ -327,6 +341,16 @@ config DEBUG_INFO_COMPRESSED
 	  preferable to setting $KDEB_COMPRESS to "none" which would be even
 	  larger.
 
+config DEBUG_INFO_COMPRESSED_ZSTD
+	bool "Compress debugging information with zstd"
+	depends on $(cc-option,-gz=zstd)
+	depends on $(ld-option,--compress-debug-sections=zstd)
+	help
+	  Compress the debug information using zstd.  Requires GCC 13.0+ or Clang
+	  16.0+, binutils 2.40+, and zstd.
+
+endchoice # "Compressed Debug information"
+
 config DEBUG_INFO_SPLIT
 	bool "Produce split debuginfo in .dwo files"
 	depends on $(cc-option,-gsplit-dwarf)
scripts/Makefile.debug +4 -0
index 332c486f705f..8ac3379d2255 100644
@@ -31,6 +31,10 @@ ifdef CONFIG_DEBUG_INFO_COMPRESSED
 DEBUG_CFLAGS	+= -gz=zlib
 KBUILD_AFLAGS	+= -gz=zlib
 KBUILD_LDFLAGS	+= --compress-debug-sections=zlib
+else ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZSTD
+DEBUG_CFLAGS	+= -gz=zstd
+KBUILD_AFLAGS	+= -gz=zstd
+KBUILD_LDFLAGS	+= --compress-debug-sections=zstd
 endif
 
 KBUILD_CFLAGS	+= $(DEBUG_CFLAGS)
Signature · 1 line
2.38.0.135.g90850a2211-goog
#2 lore raw
Quoted 63 lines · Nathan Chancellor
Quoted 36 lines · Nick Desaulniers

Make DEBUG_INFO_COMPRESSED a choice; DEBUG_INFO_UNCOMPRESSED is the default, DEBUG_INFO_COMPRESSED uses zlib, DEBUG_INFO_COMPRESSED_ZSTD uses zstd.

Some quick N=1 measurements with du, /usr/bin/time -v, and bloaty:

clang-16, x86_64 defconfig plus CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_UNCOMPRESSED=y:
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:55.43 488M vmlinux

27.6%   136Mi   0.0%       0    .debug_info
 6.1%  30.2Mi   0.0%       0    .debug_str_offsets
 3.5%  17.2Mi   0.0%       0    .debug_line
 3.3%  16.3Mi   0.0%       0    .debug_loclists
 0.9%  4.62Mi   0.0%       0    .debug_str

clang-16, x86_64 defconfig plus CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED=y (zlib):
Elapsed (wall clock) time (h:mm:ss or m:ss): 1:00.35 385M vmlinux

21.8%  85.4Mi   0.0%       0    .debug_info
 2.1%  8.26Mi   0.0%       0    .debug_str_offsets
 2.1%  8.24Mi   0.0%       0    .debug_loclists
 1.9%  7.48Mi   0.0%       0    .debug_line
 0.5%  1.94Mi   0.0%       0    .debug_str

clang-16, x86_64 defconfig plus
CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_ZSTD=y (zstd):
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:59.69 373M vmlinux

21.4%  81.4Mi   0.0%       0    .debug_info
 2.3%  8.85Mi   0.0%       0    .debug_loclists
 1.5%  5.71Mi   0.0%       0    .debug_line
 0.5%  1.95Mi   0.0%       0    .debug_str_offsets
 0.4%  1.62Mi   0.0%       0    .debug_str
Quoted 7 lines

That's only a 3.11% overall binary size savings over zlib, but at no performance regression.

One small comment below.

Quoted 13 lines
---
 lib/Kconfig.debug      | 26 +++++++++++++++++++++++++-
 scripts/Makefile.debug |  4 ++++
 2 files changed, 29 insertions(+), 1 deletion(-)
lib/Kconfig.debug +0 -0
index 3fc7abffc7aa..4085ac77dc12 100644
@@ -312,8 +312,22 @@ config DEBUG_INFO_REDUCED
        DEBUG_INFO build and compile times are reduced too.
        Only works with newer gcc versions.
 
Quoted 9 lines

lib/Kconfig.debug config DEBUG_INFO_REDUCED · line 317

Quoted 3 lines
lib/Kconfig.debug +3 -0
+choice
+     prompt "Compressed Debug information"
+     depends on DEBUG_KERNEL

I think you can drop this depends. The entire block is in an 'if DEBUG_INFO', which can only be true if CONFIG_DEBUG_KERNEL is set because of the dependencies of the "Debug information" prompt above this file, which is how CONFIG_DEBUG_INFO is selected.

Yes, dropping this is good.

Quoted 4 lines
Quoted 4 lines
lib/Kconfig.debug +4 -0
+     help
+       Compress the resulting debug info. Results in smaller debug info sections,
+       but requires that consumers are able to decompress the results.
+
Quoted 8 lines

lib/Kconfig.debug config DEBUG_INFO_REDUCED · line 329

Quoted 8 lines
lib/Kconfig.debug +7 -0
+       If unsure, choose DEBUG_INFO_UNCOMPRESSED.
+
+config DEBUG_INFO_UNCOMPRESSED
+     bool "Don't compress debug information"
+     help
+       Don't compress debug info sections.
+
 config DEBUG_INFO_COMPRESSED

Also, I prefer...

DEBUG_INFO_COMPRESSED_NONE
DEBUG_INFO_COMPRESSED_ZLIB
DEBUG_INFO_COMPRESSED_ZSTD

Following...

$ ld.lld-16 -v
Debian LLD 16.0.0 (compatible with GNU linkers)
$ ld.lld-16 --help | grep compress-debug-sections
 --compress-debug-sections=[none,zlib,zstd]
Quoted 9 lines

lib/Kconfig.debug config DEBUG_INFO_COMPRESSED · line 343

Quoted 9 lines
lib/Kconfig.debug +1 -1
-     bool "Compressed debugging information"
+     bool "Compress debugging information with zlib"
      depends on $(cc-option,-gz=zlib)
      depends on $(ld-option,--compress-debug-sections=zlib)
      help
@@ -327,6 +341,16 @@ config DEBUG_INFO_COMPRESSED
        preferable to setting $KDEB_COMPRESS to "none" which would be even
        larger.
 

Anyone, checked $KDEB_COMPRESS - does this need changes?

-Sedat-

Quoted 31 lines
Quoted 31 lines
lib/Kconfig.debug +10 -0
+config DEBUG_INFO_COMPRESSED_ZSTD
+     bool "Compress debugging information with zstd"
+     depends on $(cc-option,-gz=zstd)
+     depends on $(ld-option,--compress-debug-sections=zstd)
+     help
+       Compress the debug information using zstd.  Requires GCC 13.0+ or Clang
+       16.0+, binutils 2.40+, and zstd.
+
+endchoice # "Compressed Debug information"
+
 config DEBUG_INFO_SPLIT
      bool "Produce split debuginfo in .dwo files"
      depends on $(cc-option,-gsplit-dwarf)
scripts/Makefile.debug +4 -1
index 332c486f705f..8ac3379d2255 100644
@@ -31,6 +31,10 @@ ifdef CONFIG_DEBUG_INFO_COMPRESSED
 DEBUG_CFLAGS += -gz=zlib
 KBUILD_AFLAGS        += -gz=zlib
 KBUILD_LDFLAGS       += --compress-debug-sections=zlib
+else ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZSTD
+DEBUG_CFLAGS += -gz=zstd
+KBUILD_AFLAGS        += -gz=zstd
+KBUILD_LDFLAGS       += --compress-debug-sections=zstd
 endif
 
 KBUILD_CFLAGS        += $(DEBUG_CFLAGS)
--

2.38.0.135.g90850a2211-goog

#3 lore raw
Quoted 17 lines · Sedat Dilek
Quoted 8 lines · Nathan Chancellor

lib/Kconfig.debug config DEBUG_INFO_REDUCED · line 329

Quoted 6 lines · Nick Desaulniers
lib/Kconfig.debug +5 -0
+config DEBUG_INFO_UNCOMPRESSED
+     bool "Don't compress debug information"
+     help
+       Don't compress debug info sections.
+
 config DEBUG_INFO_COMPRESSED

Also, I prefer...

DEBUG_INFO_COMPRESSED_NONE
DEBUG_INFO_COMPRESSED_ZLIB
DEBUG_INFO_COMPRESSED_ZSTD

Quoted 7 lines

Following...

$ ld.lld-16 -v
Debian LLD 16.0.0 (compatible with GNU linkers)
$ ld.lld-16 --help | grep compress-debug-sections
 --compress-debug-sections=[none,zlib,zstd]

It's a good suggestion, though I deliberately did not modify the identifier used for the existing DEBUG_INFO_COMPRESSED. No defconfigs set DEBUG_INFO_COMPRESSED explicitly, but I think one issue with changing the identifier of existing kconfigs is that upon upgrading to newer kernel sources, if you run make oldmodconfig if you previously had DEBUG_INFO_COMPRESSED=y, kconfig doesn't know that that option has been renamed to DEBUG_INFO_COMPRESSED_ZLIB so it would be silently disabled, and image sizes would increase due to uncompressed debug info. That seems developer hostile.

Masahiro, what are your thoughts on how to proceed? Should I change the pre-existing Kconfig identifier, or "let sleeping dogs lie?"

Signature · 2 lines
Thanks,
~Nick Desaulniers
#4 lore raw

-----Ursprüngliche Nachricht-----
Von: Nick Desaulniers <[email protected]>
Gesendet: Montag, 24. Oktober 2022 19:44
An: [email protected]; Masahiro Yamada <[email protected]>
Cc: Nathan Chancellor <[email protected]>; Fangrui Song <[email protected]>; Sedat Dilek (DHL Supply Chain) <[email protected]>; Michal Marek <[email protected]>; Nick Terrell <[email protected]>; Tom Rix <[email protected]>; Andrew Morton <[email protected]>; Peter Zijlstra (Intel) <[email protected]>; David Gow <[email protected]>; Kees Cook <[email protected]>; Josh Poimboeuf <[email protected]>; Dan Williams <[email protected]>; Miguel Ojeda <[email protected]>; Isabella Basso <[email protected]>; Vlastimil Babka <[email protected]>; Rasmus Villemoes <[email protected]>; [email protected]; [email protected]; [email protected]
Betreff: Re: [PATCH] Makefile.debug: support for -gz=zstd

Quoted 17 lines · Sedat Dilek
Quoted 8 lines · Nathan Chancellor

lib/Kconfig.debug config DEBUG_INFO_REDUCED · line 329

Quoted 6 lines · Nick Desaulniers
lib/Kconfig.debug +5 -0
+config DEBUG_INFO_UNCOMPRESSED
+     bool "Don't compress debug information"
+     help
+       Don't compress debug info sections.
+
 config DEBUG_INFO_COMPRESSED

Also, I prefer...

DEBUG_INFO_COMPRESSED_NONE
DEBUG_INFO_COMPRESSED_ZLIB
DEBUG_INFO_COMPRESSED_ZSTD

Quoted 7 lines

Following...

$ ld.lld-16 -v
Debian LLD 16.0.0 (compatible with GNU linkers)
$ ld.lld-16 --help | grep compress-debug-sections  
--compress-debug-sections=[none,zlib,zstd]

It's a good suggestion, though I deliberately did not modify the identifier used for the existing DEBUG_INFO_COMPRESSED. No defconfigs set DEBUG_INFO_COMPRESSED explicitly, but I think one issue with changing the identifier of existing kconfigs is that upon upgrading to newer kernel sources, if you run make oldmodconfig if you previously had DEBUG_INFO_COMPRESSED=y, kconfig doesn't know that that option has been renamed to DEBUG_INFO_COMPRESSED_ZLIB so it would be silently disabled, and image sizes would increase due to uncompressed debug info. That seems developer hostile.

Masahiro, what are your thoughts on how to proceed? Should I change the pre-existing Kconfig identifier, or "let sleeping dogs lie?"

***

Cannot say what happens when CONFIG_XXX changes - I have not played with your patch.

Perhaps, we can use CONFIG_ DEBUG_INFO_COMPRESSED="none|zlib|zstd" with default: "none"?

-Sedat-

#5 lore raw
In reply to Nick Desaulniers
Quoted 37 lines · Nick Desaulniers
Quoted 17 lines · Sedat Dilek
Quoted 8 lines · Nathan Chancellor

lib/Kconfig.debug config DEBUG_INFO_REDUCED · line 329

Quoted 6 lines · Nick Desaulniers
lib/Kconfig.debug +5 -0
+config DEBUG_INFO_UNCOMPRESSED
+     bool "Don't compress debug information"
+     help
+       Don't compress debug info sections.
+
 config DEBUG_INFO_COMPRESSED

Also, I prefer...

DEBUG_INFO_COMPRESSED_NONE
DEBUG_INFO_COMPRESSED_ZLIB
DEBUG_INFO_COMPRESSED_ZSTD

Quoted 7 lines

Following...

$ ld.lld-16 -v
Debian LLD 16.0.0 (compatible with GNU linkers)
$ ld.lld-16 --help | grep compress-debug-sections
 --compress-debug-sections=[none,zlib,zstd]

It's a good suggestion, though I deliberately did not modify the identifier used for the existing DEBUG_INFO_COMPRESSED. No defconfigs set DEBUG_INFO_COMPRESSED explicitly, but I think one issue with changing the identifier of existing kconfigs is that upon upgrading to newer kernel sources, if you run make oldmodconfig if you previously had DEBUG_INFO_COMPRESSED=y, kconfig doesn't know that that option has been renamed to DEBUG_INFO_COMPRESSED_ZLIB so it would be silently disabled, and image sizes would increase due to uncompressed debug info. That seems developer hostile.

Quoted 2 lines

Masahiro, what are your thoughts on how to proceed? Should I change the pre-existing Kconfig identifier, or "let sleeping dogs lie?"

Hi Masahiro,
What are your thoughts on the above?

Quoted 3 lines

--
Thanks,
~Nick Desaulniers

Signature · 2 lines
Thanks,
~Nick Desaulniers
#6 lore raw
Quoted 40 lines · Nick Desaulniers
Quoted 37 lines · Nick Desaulniers
Quoted 17 lines · Sedat Dilek
Quoted 8 lines · Nathan Chancellor

lib/Kconfig.debug config DEBUG_INFO_REDUCED · line 329

Quoted 6 lines · Nick Desaulniers
lib/Kconfig.debug +5 -0
+config DEBUG_INFO_UNCOMPRESSED
+     bool "Don't compress debug information"
+     help
+       Don't compress debug info sections.
+
 config DEBUG_INFO_COMPRESSED

Also, I prefer...

DEBUG_INFO_COMPRESSED_NONE
DEBUG_INFO_COMPRESSED_ZLIB
DEBUG_INFO_COMPRESSED_ZSTD

Quoted 7 lines

Following...

$ ld.lld-16 -v
Debian LLD 16.0.0 (compatible with GNU linkers)
$ ld.lld-16 --help | grep compress-debug-sections
 --compress-debug-sections=[none,zlib,zstd]

It's a good suggestion, though I deliberately did not modify the identifier used for the existing DEBUG_INFO_COMPRESSED. No defconfigs set DEBUG_INFO_COMPRESSED explicitly, but I think one issue with changing the identifier of existing kconfigs is that upon upgrading to newer kernel sources, if you run make oldmodconfig if you previously had DEBUG_INFO_COMPRESSED=y, kconfig doesn't know that that option has been renamed to DEBUG_INFO_COMPRESSED_ZLIB so it would be silently disabled, and image sizes would increase due to uncompressed debug info. That seems developer hostile.

Quoted 5 lines
Quoted 2 lines

Masahiro, what are your thoughts on how to proceed? Should I change the pre-existing Kconfig identifier, or "let sleeping dogs lie?"

Hi Masahiro,
What are your thoughts on the above?

We renamed CONFIG options from time to time. We should strive to update in-tree defconfigs, but we did not care about external users very much. (we rely on "make oldconfig" showing new options and asking users for their choice.)

So, I like Sedat's DEBUG_INFO_COMPRESSED_{ZLIB,ZSTD,NONE}. (and it is consist with CONFIG_INITRAMFS_COMPRESSION_*)

Signature · 2 lines
Best Regards
Masahiro Yamada
#7 lore raw

PATCH v2 1/1 Makefile.debug: support for -gz=zstd

Make DEBUG_INFO_COMPRESSED a choice; DEBUG_INFO_UNCOMPRESSED is the default, DEBUG_INFO_COMPRESSED uses zlib, DEBUG_INFO_COMPRESSED_ZSTD uses zstd.

Some quick N=1 measurements with du, /usr/bin/time -v, and bloaty:

clang-16, x86_64 defconfig plus CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_UNCOMPRESSED=y:
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:55.43 488M vmlinux

27.6%   136Mi   0.0%       0    .debug_info
 6.1%  30.2Mi   0.0%       0    .debug_str_offsets
 3.5%  17.2Mi   0.0%       0    .debug_line
 3.3%  16.3Mi   0.0%       0    .debug_loclists
 0.9%  4.62Mi   0.0%       0    .debug_str

clang-16, x86_64 defconfig plus CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED=y (zlib):
Elapsed (wall clock) time (h:mm:ss or m:ss): 1:00.35 385M vmlinux

21.8%  85.4Mi   0.0%       0    .debug_info
 2.1%  8.26Mi   0.0%       0    .debug_str_offsets
 2.1%  8.24Mi   0.0%       0    .debug_loclists
 1.9%  7.48Mi   0.0%       0    .debug_line
 0.5%  1.94Mi   0.0%       0    .debug_str

clang-16, x86_64 defconfig plus
CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_ZSTD=y (zstd):
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:59.69 373M vmlinux

21.4%  81.4Mi   0.0%       0    .debug_info
 2.3%  8.85Mi   0.0%       0    .debug_loclists
 1.5%  5.71Mi   0.0%       0    .debug_line
 0.5%  1.95Mi   0.0%       0    .debug_str_offsets
 0.4%  1.62Mi   0.0%       0    .debug_str

That's only a 3.11% overall binary size savings over zlib, but at no performance regression.

This renames the existing KConfig options:
* DEBUG_INFO_UNCOMPRESSED -> DEBUG_INFO_COMPRESSED_NONE
* DEBUG_INFO_COMPRESSED -> DEBUG_INFO_COMPRESSED_ZLIB So users upgrading may need to reset the new Kconfigs.

---
Changes v1 -> v2:
* Remove depends on DEBUG_KERNEL as per Nathan.
* Rename Kconfigs as per Sedat and Masahiro.
* Add note about renamed Kconfigs to commit message.
* Add more help text to DEBUG_INFO_COMPRESSED_ZSTD.

 lib/Kconfig.debug      | 29 +++++++++++++++++++++++++++--
 scripts/Makefile.debug |  4 ++++
 2 files changed, 31 insertions(+), 2 deletions(-)
lib/Kconfig.debug +27 -2
index 29280072dc0e..7c28a8fba02e 100644
@@ -312,8 +312,21 @@ config DEBUG_INFO_REDUCED
 	  DEBUG_INFO build and compile times are reduced too.
 	  Only works with newer gcc versions.
 
-config DEBUG_INFO_COMPRESSED
-	bool "Compressed debugging information"
+choice
+	prompt "Compressed Debug information"
+	help
+	  Compress the resulting debug info. Results in smaller debug info sections,
+	  but requires that consumers are able to decompress the results.
+
+	  If unsure, choose DEBUG_INFO_COMPRESSED_NONE.
+
+config DEBUG_INFO_COMPRESSED_NONE
+	bool "Don't compress debug information"
+	help
+	  Don't compress debug info sections.
+
+config DEBUG_INFO_COMPRESSED_ZLIB
+	bool "Compress debugging information with zlib"
 	depends on $(cc-option,-gz=zlib)
 	depends on $(ld-option,--compress-debug-sections=zlib)
 	help
@@ -327,6 +340,18 @@ config DEBUG_INFO_COMPRESSED
 	  preferable to setting $KDEB_COMPRESS to "none" which would be even
 	  larger.
 
+config DEBUG_INFO_COMPRESSED_ZSTD
+	bool "Compress debugging information with zstd"
+	depends on $(cc-option,-gz=zstd)
+	depends on $(ld-option,--compress-debug-sections=zstd)
+	help
+	  Compress the debug information using zstd.  This may provide better
+	  compression than zlib, for about the same time costs, but requires newer
+	  toolchain support.  Requires GCC 13.0+ or Clang 16.0+, binutils 2.40+, and
+	  zstd.
+
+endchoice # "Compressed Debug information"
+
 config DEBUG_INFO_SPLIT
 	bool "Produce split debuginfo in .dwo files"
 	depends on $(cc-option,-gsplit-dwarf)
scripts/Makefile.debug +4 -0
index 332c486f705f..8ac3379d2255 100644
@@ -31,6 +31,10 @@ ifdef CONFIG_DEBUG_INFO_COMPRESSED
 DEBUG_CFLAGS	+= -gz=zlib
 KBUILD_AFLAGS	+= -gz=zlib
 KBUILD_LDFLAGS	+= --compress-debug-sections=zlib
+else ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZSTD
+DEBUG_CFLAGS	+= -gz=zstd
+KBUILD_AFLAGS	+= -gz=zstd
+KBUILD_LDFLAGS	+= --compress-debug-sections=zstd
 endif
 
 KBUILD_CFLAGS	+= $(DEBUG_CFLAGS)

base-commit: f0c4d9fc9cc9462659728d168387191387e903cc

Signature · 1 line
2.38.1.431.g37b22c650d-goog
#8 lore raw
Quoted 36 lines · Nick Desaulniers

Make DEBUG_INFO_COMPRESSED a choice; DEBUG_INFO_UNCOMPRESSED is the default, DEBUG_INFO_COMPRESSED uses zlib, DEBUG_INFO_COMPRESSED_ZSTD uses zstd.

Some quick N=1 measurements with du, /usr/bin/time -v, and bloaty:

clang-16, x86_64 defconfig plus CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_UNCOMPRESSED=y:
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:55.43 488M vmlinux

27.6%   136Mi   0.0%       0    .debug_info
 6.1%  30.2Mi   0.0%       0    .debug_str_offsets
 3.5%  17.2Mi   0.0%       0    .debug_line
 3.3%  16.3Mi   0.0%       0    .debug_loclists
 0.9%  4.62Mi   0.0%       0    .debug_str

clang-16, x86_64 defconfig plus CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED=y (zlib):
Elapsed (wall clock) time (h:mm:ss or m:ss): 1:00.35 385M vmlinux

21.8%  85.4Mi   0.0%       0    .debug_info
 2.1%  8.26Mi   0.0%       0    .debug_str_offsets
 2.1%  8.24Mi   0.0%       0    .debug_loclists
 1.9%  7.48Mi   0.0%       0    .debug_line
 0.5%  1.94Mi   0.0%       0    .debug_str

clang-16, x86_64 defconfig plus
CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_ZSTD=y (zstd):
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:59.69 373M vmlinux

21.4%  81.4Mi   0.0%       0    .debug_info
 2.3%  8.85Mi   0.0%       0    .debug_loclists
 1.5%  5.71Mi   0.0%       0    .debug_line
 0.5%  1.95Mi   0.0%       0    .debug_str_offsets
 0.4%  1.62Mi   0.0%       0    .debug_str
Quoted 6 lines

That's only a 3.11% overall binary size savings over zlib, but at no performance regression.

This renames the existing KConfig options:
* DEBUG_INFO_UNCOMPRESSED -> DEBUG_INFO_COMPRESSED_NONE
* DEBUG_INFO_COMPRESSED -> DEBUG_INFO_COMPRESSED_ZLIB

Might you want to update the DEBUG_INFO_(UN|)COMPRESSED above to the new names?

Kind regards,
Nicolas

Quoted 85 lines

So users upgrading may need to reset the new Kconfigs.

---
Changes v1 -> v2:
* Remove depends on DEBUG_KERNEL as per Nathan.
* Rename Kconfigs as per Sedat and Masahiro.
* Add note about renamed Kconfigs to commit message.
* Add more help text to DEBUG_INFO_COMPRESSED_ZSTD.

 lib/Kconfig.debug      | 29 +++++++++++++++++++++++++++--
 scripts/Makefile.debug |  4 ++++
 2 files changed, 31 insertions(+), 2 deletions(-)
lib/Kconfig.debug +27 -2
index 29280072dc0e..7c28a8fba02e 100644
@@ -312,8 +312,21 @@ config DEBUG_INFO_REDUCED
 	  DEBUG_INFO build and compile times are reduced too.
 	  Only works with newer gcc versions.
 
-config DEBUG_INFO_COMPRESSED
-	bool "Compressed debugging information"
+choice
+	prompt "Compressed Debug information"
+	help
+	  Compress the resulting debug info. Results in smaller debug info sections,
+	  but requires that consumers are able to decompress the results.
+
+	  If unsure, choose DEBUG_INFO_COMPRESSED_NONE.
+
+config DEBUG_INFO_COMPRESSED_NONE
+	bool "Don't compress debug information"
+	help
+	  Don't compress debug info sections.
+
+config DEBUG_INFO_COMPRESSED_ZLIB
+	bool "Compress debugging information with zlib"
 	depends on $(cc-option,-gz=zlib)
 	depends on $(ld-option,--compress-debug-sections=zlib)
 	help
@@ -327,6 +340,18 @@ config DEBUG_INFO_COMPRESSED
 	  preferable to setting $KDEB_COMPRESS to "none" which would be even
 	  larger.
 
+config DEBUG_INFO_COMPRESSED_ZSTD
+	bool "Compress debugging information with zstd"
+	depends on $(cc-option,-gz=zstd)
+	depends on $(ld-option,--compress-debug-sections=zstd)
+	help
+	  Compress the debug information using zstd.  This may provide better
+	  compression than zlib, for about the same time costs, but requires newer
+	  toolchain support.  Requires GCC 13.0+ or Clang 16.0+, binutils 2.40+, and
+	  zstd.
+
+endchoice # "Compressed Debug information"
+
 config DEBUG_INFO_SPLIT
 	bool "Produce split debuginfo in .dwo files"
 	depends on $(cc-option,-gsplit-dwarf)
scripts/Makefile.debug +4 -0
index 332c486f705f..8ac3379d2255 100644
@@ -31,6 +31,10 @@ ifdef CONFIG_DEBUG_INFO_COMPRESSED
 DEBUG_CFLAGS	+= -gz=zlib
 KBUILD_AFLAGS	+= -gz=zlib
 KBUILD_LDFLAGS	+= --compress-debug-sections=zlib
+else ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZSTD
+DEBUG_CFLAGS	+= -gz=zstd
+KBUILD_AFLAGS	+= -gz=zstd
+KBUILD_LDFLAGS	+= --compress-debug-sections=zstd
 endif
 
 KBUILD_CFLAGS	+= $(DEBUG_CFLAGS)

base-commit: f0c4d9fc9cc9462659728d168387191387e903cc

Signature · 1 line
2.38.1.431.g37b22c650d-goog
#9 lore raw
In reply to Nick Desaulniers

scripts/Makefile.debug ifdef CONFIG_DEBUG_INFO_COMPRESSED

Quoted 113 lines · Nick Desaulniers

Make DEBUG_INFO_COMPRESSED a choice; DEBUG_INFO_UNCOMPRESSED is the default, DEBUG_INFO_COMPRESSED uses zlib, DEBUG_INFO_COMPRESSED_ZSTD uses zstd.

Some quick N=1 measurements with du, /usr/bin/time -v, and bloaty:

clang-16, x86_64 defconfig plus CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_UNCOMPRESSED=y:
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:55.43 488M vmlinux

27.6%   136Mi   0.0%       0    .debug_info
 6.1%  30.2Mi   0.0%       0    .debug_str_offsets
 3.5%  17.2Mi   0.0%       0    .debug_line
 3.3%  16.3Mi   0.0%       0    .debug_loclists
 0.9%  4.62Mi   0.0%       0    .debug_str

clang-16, x86_64 defconfig plus CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED=y (zlib):
Elapsed (wall clock) time (h:mm:ss or m:ss): 1:00.35 385M vmlinux

21.8%  85.4Mi   0.0%       0    .debug_info
 2.1%  8.26Mi   0.0%       0    .debug_str_offsets
 2.1%  8.24Mi   0.0%       0    .debug_loclists
 1.9%  7.48Mi   0.0%       0    .debug_line
 0.5%  1.94Mi   0.0%       0    .debug_str

clang-16, x86_64 defconfig plus
CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_ZSTD=y (zstd):
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:59.69 373M vmlinux

21.4%  81.4Mi   0.0%       0    .debug_info
 2.3%  8.85Mi   0.0%       0    .debug_loclists
 1.5%  5.71Mi   0.0%       0    .debug_line
 0.5%  1.95Mi   0.0%       0    .debug_str_offsets
 0.4%  1.62Mi   0.0%       0    .debug_str

That's only a 3.11% overall binary size savings over zlib, but at no performance regression.

This renames the existing KConfig options:
* DEBUG_INFO_UNCOMPRESSED -> DEBUG_INFO_COMPRESSED_NONE
* DEBUG_INFO_COMPRESSED -> DEBUG_INFO_COMPRESSED_ZLIB So users upgrading may need to reset the new Kconfigs.

---
Changes v1 -> v2:
* Remove depends on DEBUG_KERNEL as per Nathan.
* Rename Kconfigs as per Sedat and Masahiro.
* Add note about renamed Kconfigs to commit message.
* Add more help text to DEBUG_INFO_COMPRESSED_ZSTD.

 lib/Kconfig.debug      | 29 +++++++++++++++++++++++++++--
 scripts/Makefile.debug |  4 ++++
 2 files changed, 31 insertions(+), 2 deletions(-)
lib/Kconfig.debug +27 -2
index 29280072dc0e..7c28a8fba02e 100644
@@ -312,8 +312,21 @@ config DEBUG_INFO_REDUCED
          DEBUG_INFO build and compile times are reduced too.
          Only works with newer gcc versions.
 
-config DEBUG_INFO_COMPRESSED
-       bool "Compressed debugging information"
+choice
+       prompt "Compressed Debug information"
+       help
+         Compress the resulting debug info. Results in smaller debug info sections,
+         but requires that consumers are able to decompress the results.
+
+         If unsure, choose DEBUG_INFO_COMPRESSED_NONE.
+
+config DEBUG_INFO_COMPRESSED_NONE
+       bool "Don't compress debug information"
+       help
+         Don't compress debug info sections.
+
+config DEBUG_INFO_COMPRESSED_ZLIB
+       bool "Compress debugging information with zlib"
        depends on $(cc-option,-gz=zlib)
        depends on $(ld-option,--compress-debug-sections=zlib)
        help
@@ -327,6 +340,18 @@ config DEBUG_INFO_COMPRESSED
          preferable to setting $KDEB_COMPRESS to "none" which would be even
          larger.
 
+config DEBUG_INFO_COMPRESSED_ZSTD
+       bool "Compress debugging information with zstd"
+       depends on $(cc-option,-gz=zstd)
+       depends on $(ld-option,--compress-debug-sections=zstd)
+       help
+         Compress the debug information using zstd.  This may provide better
+         compression than zlib, for about the same time costs, but requires newer
+         toolchain support.  Requires GCC 13.0+ or Clang 16.0+, binutils 2.40+, and
+         zstd.
+
+endchoice # "Compressed Debug information"
+
 config DEBUG_INFO_SPLIT
        bool "Produce split debuginfo in .dwo files"
        depends on $(cc-option,-gsplit-dwarf)
scripts/Makefile.debug +0 -0
index 332c486f705f..8ac3379d2255 100644
@@ -31,6 +31,10 @@ ifdef CONFIG_DEBUG_INFO_COMPRESSED

^^ ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZLIB

...from a quick (re)view.

-Sedat-

Quoted 15 lines
scripts/Makefile.debug +4 -0
 DEBUG_CFLAGS   += -gz=zlib
 KBUILD_AFLAGS  += -gz=zlib
 KBUILD_LDFLAGS += --compress-debug-sections=zlib
+else ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZSTD
+DEBUG_CFLAGS   += -gz=zstd
+KBUILD_AFLAGS  += -gz=zstd
+KBUILD_LDFLAGS += --compress-debug-sections=zstd
 endif
 
 KBUILD_CFLAGS  += $(DEBUG_CFLAGS)

base-commit: f0c4d9fc9cc9462659728d168387191387e903cc --
2.38.1.431.g37b22c650d-goog

#10 lore raw

PATCH v3 1/1 Makefile.debug: support for -gz=zstd

Make DEBUG_INFO_COMPRESSED a choice; DEBUG_INFO_COMPRESSED_NONE is the default, DEBUG_INFO_COMPRESSED_ZLIB uses zlib, DEBUG_INFO_COMPRESSED_ZSTD uses zstd.

This renames the existing KConfig option DEBUG_INFO_COMPRESSED to DEBUG_INFO_COMPRESSED_ZLIB so users upgrading may need to reset the new Kconfigs.

Some quick N=1 measurements with du, /usr/bin/time -v, and bloaty:

clang-16, x86_64 defconfig plus CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_NONE=y:
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:55.43 488M vmlinux

27.6%   136Mi   0.0%       0    .debug_info
 6.1%  30.2Mi   0.0%       0    .debug_str_offsets
 3.5%  17.2Mi   0.0%       0    .debug_line
 3.3%  16.3Mi   0.0%       0    .debug_loclists
 0.9%  4.62Mi   0.0%       0    .debug_str

clang-16, x86_64 defconfig plus CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_ZLIB=y:
Elapsed (wall clock) time (h:mm:ss or m:ss): 1:00.35 385M vmlinux

21.8%  85.4Mi   0.0%       0    .debug_info
 2.1%  8.26Mi   0.0%       0    .debug_str_offsets
 2.1%  8.24Mi   0.0%       0    .debug_loclists
 1.9%  7.48Mi   0.0%       0    .debug_line
 0.5%  1.94Mi   0.0%       0    .debug_str

clang-16, x86_64 defconfig plus CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_ZSTD=y:
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:59.69 373M vmlinux

21.4%  81.4Mi   0.0%       0    .debug_info
 2.3%  8.85Mi   0.0%       0    .debug_loclists
 1.5%  5.71Mi   0.0%       0    .debug_line
 0.5%  1.95Mi   0.0%       0    .debug_str_offsets
 0.4%  1.62Mi   0.0%       0    .debug_str

That's only a 3.11% overall binary size savings over zlib, but at no performance regression.

---
Changes v2 -> v3:
* Fix scripts/Makefile.debug as per Sedat.
* Update commit message as per Nicolas.

Changes v1 -> v2:
* Remove depends on DEBUG_KERNEL as per Nathan.
* Rename Kconfigs as per Sedat and Masahiro.
* Add note about renamed Kconfigs to commit message.
* Add more help text to DEBUG_INFO_COMPRESSED_ZSTD.

 lib/Kconfig.debug      | 29 +++++++++++++++++++++++++++--
 scripts/Makefile.debug |  6 +++++-
 2 files changed, 32 insertions(+), 3 deletions(-)
lib/Kconfig.debug +27 -2
index 29280072dc0e..7c28a8fba02e 100644
@@ -312,8 +312,21 @@ config DEBUG_INFO_REDUCED
 	  DEBUG_INFO build and compile times are reduced too.
 	  Only works with newer gcc versions.
 
-config DEBUG_INFO_COMPRESSED
-	bool "Compressed debugging information"
+choice
+	prompt "Compressed Debug information"
+	help
+	  Compress the resulting debug info. Results in smaller debug info sections,
+	  but requires that consumers are able to decompress the results.
+
+	  If unsure, choose DEBUG_INFO_COMPRESSED_NONE.
+
+config DEBUG_INFO_COMPRESSED_NONE
+	bool "Don't compress debug information"
+	help
+	  Don't compress debug info sections.
+
+config DEBUG_INFO_COMPRESSED_ZLIB
+	bool "Compress debugging information with zlib"
 	depends on $(cc-option,-gz=zlib)
 	depends on $(ld-option,--compress-debug-sections=zlib)
 	help
@@ -327,6 +340,18 @@ config DEBUG_INFO_COMPRESSED
 	  preferable to setting $KDEB_COMPRESS to "none" which would be even
 	  larger.
 
+config DEBUG_INFO_COMPRESSED_ZSTD
+	bool "Compress debugging information with zstd"
+	depends on $(cc-option,-gz=zstd)
+	depends on $(ld-option,--compress-debug-sections=zstd)
+	help
+	  Compress the debug information using zstd.  This may provide better
+	  compression than zlib, for about the same time costs, but requires newer
+	  toolchain support.  Requires GCC 13.0+ or Clang 16.0+, binutils 2.40+, and
+	  zstd.
+
+endchoice # "Compressed Debug information"
+
 config DEBUG_INFO_SPLIT
 	bool "Produce split debuginfo in .dwo files"
 	depends on $(cc-option,-gsplit-dwarf)
scripts/Makefile.debug +5 -1
index 332c486f705f..059ff38fe0cb 100644
@@ -27,10 +27,14 @@ else
 DEBUG_RUSTFLAGS	+= -Cdebuginfo=2
 endif
 
-ifdef CONFIG_DEBUG_INFO_COMPRESSED
+ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZLIB
 DEBUG_CFLAGS	+= -gz=zlib
 KBUILD_AFLAGS	+= -gz=zlib
 KBUILD_LDFLAGS	+= --compress-debug-sections=zlib
+else ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZSTD
+DEBUG_CFLAGS	+= -gz=zstd
+KBUILD_AFLAGS	+= -gz=zstd
+KBUILD_LDFLAGS	+= --compress-debug-sections=zstd
 endif
 
 KBUILD_CFLAGS	+= $(DEBUG_CFLAGS)

base-commit: 1767a722a708f1fa3b9af39eb091d79101f8c086

Signature · 1 line
2.38.1.431.g37b22c650d-goog
#11 lore raw
Quoted 44 lines · Nick Desaulniers

Make DEBUG_INFO_COMPRESSED a choice; DEBUG_INFO_COMPRESSED_NONE is the default, DEBUG_INFO_COMPRESSED_ZLIB uses zlib, DEBUG_INFO_COMPRESSED_ZSTD uses zstd.

This renames the existing KConfig option DEBUG_INFO_COMPRESSED to DEBUG_INFO_COMPRESSED_ZLIB so users upgrading may need to reset the new Kconfigs.

Some quick N=1 measurements with du, /usr/bin/time -v, and bloaty:

clang-16, x86_64 defconfig plus CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_NONE=y:
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:55.43 488M vmlinux

27.6%   136Mi   0.0%       0    .debug_info
 6.1%  30.2Mi   0.0%       0    .debug_str_offsets
 3.5%  17.2Mi   0.0%       0    .debug_line
 3.3%  16.3Mi   0.0%       0    .debug_loclists
 0.9%  4.62Mi   0.0%       0    .debug_str

clang-16, x86_64 defconfig plus CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_ZLIB=y:
Elapsed (wall clock) time (h:mm:ss or m:ss): 1:00.35 385M vmlinux

21.8%  85.4Mi   0.0%       0    .debug_info
 2.1%  8.26Mi   0.0%       0    .debug_str_offsets
 2.1%  8.24Mi   0.0%       0    .debug_loclists
 1.9%  7.48Mi   0.0%       0    .debug_line
 0.5%  1.94Mi   0.0%       0    .debug_str

clang-16, x86_64 defconfig plus CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_ZSTD=y:
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:59.69 373M vmlinux

21.4%  81.4Mi   0.0%       0    .debug_info
 2.3%  8.85Mi   0.0%       0    .debug_loclists
 1.5%  5.71Mi   0.0%       0    .debug_line
 0.5%  1.95Mi   0.0%       0    .debug_str_offsets
 0.4%  1.62Mi   0.0%       0    .debug_str

That's only a 3.11% overall binary size savings over zlib, but at no performance regression.

Quoted 6 lines

---

Applied to linux-kbuild.
Thanks.

Quoted 87 lines

Changes v2 -> v3:
* Fix scripts/Makefile.debug as per Sedat.
* Update commit message as per Nicolas.

Changes v1 -> v2:
* Remove depends on DEBUG_KERNEL as per Nathan.
* Rename Kconfigs as per Sedat and Masahiro.
* Add note about renamed Kconfigs to commit message.
* Add more help text to DEBUG_INFO_COMPRESSED_ZSTD.

 lib/Kconfig.debug      | 29 +++++++++++++++++++++++++++--
 scripts/Makefile.debug |  6 +++++-
 2 files changed, 32 insertions(+), 3 deletions(-)
lib/Kconfig.debug +27 -2
index 29280072dc0e..7c28a8fba02e 100644
@@ -312,8 +312,21 @@ config DEBUG_INFO_REDUCED
          DEBUG_INFO build and compile times are reduced too.
          Only works with newer gcc versions.
 
-config DEBUG_INFO_COMPRESSED
-       bool "Compressed debugging information"
+choice
+       prompt "Compressed Debug information"
+       help
+         Compress the resulting debug info. Results in smaller debug info sections,
+         but requires that consumers are able to decompress the results.
+
+         If unsure, choose DEBUG_INFO_COMPRESSED_NONE.
+
+config DEBUG_INFO_COMPRESSED_NONE
+       bool "Don't compress debug information"
+       help
+         Don't compress debug info sections.
+
+config DEBUG_INFO_COMPRESSED_ZLIB
+       bool "Compress debugging information with zlib"
        depends on $(cc-option,-gz=zlib)
        depends on $(ld-option,--compress-debug-sections=zlib)
        help
@@ -327,6 +340,18 @@ config DEBUG_INFO_COMPRESSED
          preferable to setting $KDEB_COMPRESS to "none" which would be even
          larger.
 
+config DEBUG_INFO_COMPRESSED_ZSTD
+       bool "Compress debugging information with zstd"
+       depends on $(cc-option,-gz=zstd)
+       depends on $(ld-option,--compress-debug-sections=zstd)
+       help
+         Compress the debug information using zstd.  This may provide better
+         compression than zlib, for about the same time costs, but requires newer
+         toolchain support.  Requires GCC 13.0+ or Clang 16.0+, binutils 2.40+, and
+         zstd.
+
+endchoice # "Compressed Debug information"
+
 config DEBUG_INFO_SPLIT
        bool "Produce split debuginfo in .dwo files"
        depends on $(cc-option,-gsplit-dwarf)
scripts/Makefile.debug +5 -1
index 332c486f705f..059ff38fe0cb 100644
@@ -27,10 +27,14 @@ else
 DEBUG_RUSTFLAGS        += -Cdebuginfo=2
 endif
 
-ifdef CONFIG_DEBUG_INFO_COMPRESSED
+ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZLIB
 DEBUG_CFLAGS   += -gz=zlib
 KBUILD_AFLAGS  += -gz=zlib
 KBUILD_LDFLAGS += --compress-debug-sections=zlib
+else ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZSTD
+DEBUG_CFLAGS   += -gz=zstd
+KBUILD_AFLAGS  += -gz=zstd
+KBUILD_LDFLAGS += --compress-debug-sections=zstd
 endif
 
 KBUILD_CFLAGS  += $(DEBUG_CFLAGS)

base-commit: 1767a722a708f1fa3b9af39eb091d79101f8c086 --
2.38.1.431.g37b22c650d-goog

Signature · 2 lines
Best Regards
Masahiro Yamada
#12 lore raw
Quoted 3 lines · Masahiro Yamada

On Fri, Nov 11, 2022 at 5:00 AM Nick Desaulniers
<[email protected]> wrote:

...

Quoted 3 lines

Applied to linux-kbuild.
Thanks.

...

Quoted 9 lines
Quoted 9 lines

Changes v2 -> v3:
* Fix scripts/Makefile.debug as per Sedat.
* Update commit message as per Nicolas.

Changes v1 -> v2:
* Remove depends on DEBUG_KERNEL as per Nathan.
* Rename Kconfigs as per Sedat and Masahiro.
* Add note about renamed Kconfigs to commit message.
* Add more help text to DEBUG_INFO_COMPRESSED_ZSTD.

...

Quoted 4 lines
Quoted 4 lines
lib/Kconfig.debug +0 -0
index 29280072dc0e..7c28a8fba02e 100644

...

Quoted 11 lines

lib/Kconfig.debug config DEBUG_INFO_COMPRESSED · line 353

Quoted 11 lines
lib/Kconfig.debug +11 -0
+config DEBUG_INFO_COMPRESSED_ZSTD
+       bool "Compress debugging information with zstd"
+       depends on $(cc-option,-gz=zstd)
+       depends on $(ld-option,--compress-debug-sections=zstd)
+       help
+         Compress the debug information using zstd.  This may provide better
+         compression than zlib, for about the same time costs, but requires newer
+         toolchain support.  Requires GCC 13.0+ or Clang 16.0+, binutils 2.40+, and
+         zstd.
+
+endchoice # "Compressed Debug information"

Just saw:

GNU Binutils 2.40 Branched With Zstd Support, Zen 4, New Intel Instructions

Debian GNU/Linux ships version 2.39.90 in unstable release.

Maybe someone can point me to the release schedule of LLVM-16?

Thanks.

Best regards,
-Sedat-

[1] https://www.phoronix.com/news/GNU-Binutils-2.40
[2] https://metadata.ftp-master.debian.org/changelogs//main/b/binutils/binutils_2.39.90.20221231-1_changelog