Makefile: support compressed debug info
Subsystems (3)
PATCH 1/1 Makefile: support compressed debug info
As debug information gets larger and larger, it helps significantly save the size of vmlinux images to compress the information in the debug information sections. Note: this debug info is typically split off from the final compressed kernel image, which is why vmlinux is what's used in conjunction with GDB. Minimizing the debug info size should have no impact on boot times, or final compressed kernel image size.
All of the debug sections will have a C flag set.
$ readelf -S <object file>
$ bloaty vmlinux.gcc75.compressed.dwarf4 -- \
vmlinux.gcc75.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +18 [ = ] 0 [Unmapped] -73.3% -114Ki [ = ] 0 .debug_aranges -76.2% -2.01Mi [ = ] 0 .debug_frame -73.6% -2.89Mi [ = ] 0 .debug_str -80.7% -4.66Mi [ = ] 0 .debug_abbrev -82.9% -4.88Mi [ = ] 0 .debug_ranges -70.5% -9.04Mi [ = ] 0 .debug_line -79.3% -10.9Mi [ = ] 0 .debug_loc -39.5% -88.6Mi [ = ] 0 .debug_info -18.2% -123Mi [ = ] 0 TOTAL
$ bloaty vmlinux.clang11.compressed.dwarf4 -- \
vmlinux.clang11.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +23 [ = ] 0 [Unmapped] -65.6% -871 [ = ] 0 .debug_aranges -77.4% -1.84Mi [ = ] 0 .debug_frame -82.9% -2.33Mi [ = ] 0 .debug_abbrev -73.1% -2.43Mi [ = ] 0 .debug_str -84.8% -3.07Mi [ = ] 0 .debug_ranges -65.9% -8.62Mi [ = ] 0 .debug_line -86.2% -40.0Mi [ = ] 0 .debug_loc -42.0% -64.1Mi [ = ] 0 .debug_info -22.1% -122Mi [ = ] 0 TOTAL
--- Makefile | 5 +++++ lib/Kconfig.debug | 9 +++++++++ 2 files changed, 14 insertions(+)
Makefile +5 -0
@@ -825,6 +825,11 @@ ifdef CONFIG_DEBUG_INFO_REDUCED DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \ $(call cc-option,-fno-var-tracking) endif + +ifdef CONFIG_DEBUG_INFO_COMPRESSED +DEBUG_CFLAGS += -gz=zlib +KBUILD_LDFLAGS += --compress-debug-sections=zlib +endif endif KBUILD_CFLAGS += $(DEBUG_CFLAGS)
lib/Kconfig.debug +9 -0
@@ -213,6 +213,15 @@ config DEBUG_INFO_REDUCED DEBUG_INFO build and compile times are reduced too. Only works with newer gcc versions. +config DEBUG_INFO_COMPRESSED + bool "Compressed debugging information" + depends on DEBUG_INFO + depends on $(cc-option,-gz=zlib) + depends on $(ld-option,--compress-debug-sections=zlib) + help + Compress the debug information using zlib. Requires GCC 5.0+ or Clang + 5.0+. + config DEBUG_INFO_SPLIT bool "Produce split debuginfo in .dwo files" depends on DEBUG_INFO
Signature · 1 line
2.17.1
Quoted 31 lines · Nick Desaulniers
As debug information gets larger and larger, it helps significantly save the size of vmlinux images to compress the information in the debug information sections. Note: this debug info is typically split off from the final compressed kernel image, which is why vmlinux is what's used in conjunction with GDB. Minimizing the debug info size should have no impact on boot times, or final compressed kernel image size.
All of the debug sections will have a
Cflag set.$ readelf -S <object file>$ bloaty vmlinux.gcc75.compressed.dwarf4 -- \ vmlinux.gcc75.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +18 [ = ] 0 [Unmapped] -73.3% -114Ki [ = ] 0 .debug_aranges -76.2% -2.01Mi [ = ] 0 .debug_frame -73.6% -2.89Mi [ = ] 0 .debug_str -80.7% -4.66Mi [ = ] 0 .debug_abbrev -82.9% -4.88Mi [ = ] 0 .debug_ranges -70.5% -9.04Mi [ = ] 0 .debug_line -79.3% -10.9Mi [ = ] 0 .debug_loc -39.5% -88.6Mi [ = ] 0 .debug_info -18.2% -123Mi [ = ] 0 TOTAL$ bloaty vmlinux.clang11.compressed.dwarf4 -- \ vmlinux.clang11.uncompressed.dwarf4FILE SIZE VM SIZE
Quoted 12 lines
-------------- -------------- +0.0% +23 [ = ] 0 [Unmapped] -65.6% -871 [ = ] 0 .debug_aranges -77.4% -1.84Mi [ = ] 0 .debug_frame -82.9% -2.33Mi [ = ] 0 .debug_abbrev -73.1% -2.43Mi [ = ] 0 .debug_str -84.8% -3.07Mi [ = ] 0 .debug_ranges -65.9% -8.62Mi [ = ] 0 .debug_line -86.2% -40.0Mi [ = ] 0 .debug_loc -42.0% -64.1Mi [ = ] 0 .debug_info -22.1% -122Mi [ = ] 0 TOTAL
Hi Nick,
thanks for the patch.
I have slightly modified it to adapt to Linux v5.7-rc4 (what was your base?).
Which linker did you use and has it an impact if you switch from ld.bfd to ld.lld?
I tried a first normal run and in a 2nd one with CONFIG_DEBUG_INFO_COMPRESSED=y both with clang-10 and ld.lld-10.
My numbers (sizes in MiB):
[ diffconfig ]
$ scripts/diffconfig /boot/config-5.7.0-rc4-1-amd64-clang /boot/config-5.7.0-rc4-2-amd64-clang BUILD_SALT "5.7.0-rc4-1-amd64-clang" -> "5.7.0-rc4-2-amd64-clang" +DEBUG_INFO_COMPRESSED y
[ compiler and linker ]
$ clang-10 -v ClangBuiltLinux clang version 10.0.1 (https://github.com/llvm/llvm-project 92d5c1be9ee93850c0a8903f05f36a23ee835dc2) Target: x86_64-unknown-linux-gnu Thread model: posix InstalledDir: /home/dileks/src/llvm-toolchain/install/bin Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9 Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10 Candidate multilib: .;@m64 Candidate multilib: 32;@m32 Candidate multilib: x32;@mx32 Selected multilib: .;@m64
$ ld.lld-10 -v LLD 10.0.1 (https://github.com/llvm/llvm-project 92d5c1be9ee93850c0a8903f05f36a23ee835dc2) (compatible with GNU linkers)
[ sizes vmlinux ]
$ du -m 5.7.0-rc4-*/vmlinux* 409 5.7.0-rc4-1-amd64-clang/vmlinux 7 5.7.0-rc4-1-amd64-clang/vmlinux.compressed 404 5.7.0-rc4-1-amd64-clang/vmlinux.o 324 5.7.0-rc4-2-amd64-clang/vmlinux 7 5.7.0-rc4-2-amd64-clang/vmlinux.compressed 299 5.7.0-rc4-2-amd64-clang/vmlinux.o
[ readelf (.debug_info as example) ]
$ readelf -S vmlinux.o
[33] .debug_info PROGBITS 0000000000000000 01d6a5e8
0000000006be1ee6 0000000000000000 0 0 1$ readelf -S vmlinux.o
[33] .debug_info PROGBITS 0000000000000000 01749f18
0000000002ef04d2 0000000000000000 C 0 0 1 <---
XXX: "C (compressed)" FlagKey to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info), L (link order), O (extra OS processing required), G (group), T (TLS), C (compressed), x (unknown), o (OS specific), E (exclude), l (large), p (processor specific)
[ sizes linux-image debian packages ]
$ du -m 5.7.0-rc4-*/linux-image*.deb 47 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang_5.7.0~rc4-1~bullseye+dileks1_amd64.deb 424 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb 47 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang_5.7.0~rc4-2~bullseye+dileks1_amd64.deb 771 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb
[ sizes linux-git dir (compilation finished ]
5.7.0-rc4-1-amd64-clang: 17963 /home/dileks/src/linux-kernel/linux
5.7.0-rc4-2-amd64-clang: 14328 /home/dileks/src/linux-kernel/linux
[ xz compressed linux-image-dbg packages ]
$ file linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb: Debian binary package (format 2.0), with control.tar.xz, data compression xz $ file linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb: Debian binary package (format 2.0), with control.tar.xz, data compression xz
[ file-lists ]
$ dpkg --contents linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb | wc -l 4395 $ dpkg --contents linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb | wc -l 4395
[ file-lists vmlinux ]
$ dpkg --contents linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb | grep vmlinux -rwxr-xr-x root/root 428588312 2020-05-04 06:15 ./usr/lib/debug/lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux lrwxrwxrwx root/root 0 2020-05-04 06:15 ./usr/lib/debug/boot/vmlinux-5.7.0-rc4-1-amd64-clang -> ../lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux lrwxrwxrwx root/root 0 2020-05-04 06:15 ./usr/lib/debug/vmlinux-5.7.0-rc4-1-amd64-clang -> lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux
$ dpkg --contents linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb | grep vmlinux -rwxr-xr-x root/root 339341456 2020-05-04 12:24 ./usr/lib/debug/lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux lrwxrwxrwx root/root 0 2020-05-04 12:24 ./usr/lib/debug/boot/vmlinux-5.7.0-rc4-2-amd64-clang -> ../lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux lrwxrwxrwx root/root 0 2020-05-04 12:24 ./usr/lib/debug/vmlinux-5.7.0-rc4-2-amd64-clang -> lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux
[ conclusion ]
As you can see there is a size-reduction in case of vmlinux/vmlinux.o (debug) files...
...and my linux-git directory in total is smaller: 17963M vs. 14328M.
But the resulting linux-image-dbg file is much fatter: 424M vs. 711M. XZ-compressing the gz/zlib-compressed vmlinux (debug) file results in a fatter linux-image-dbg package.
For a usage of vmlinux (debug) in a VM scenario this might be nice but seen from a debian repository perspective not.
For the sake of completeness: I have just installed and booted the "normal" linux-image debian package - not the debug packages.
Thanks.
Regards,
- Sedat -
Quoted 50 lines
Suggested-by David Blaikie <[email protected]>Signed-off-by Nick Desaulniers <[email protected]>--- Makefile | 5 +++++ lib/Kconfig.debug | 9 +++++++++ 2 files changed, 14 insertions(+)Makefile +5
-0@@ -825,6 +825,11 @@ ifdef CONFIG_DEBUG_INFO_REDUCED DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \ $(call cc-option,-fno-var-tracking) endif + +ifdef CONFIG_DEBUG_INFO_COMPRESSED +DEBUG_CFLAGS += -gz=zlib +KBUILD_LDFLAGS += --compress-debug-sections=zlib +endif endif KBUILD_CFLAGS += $(DEBUG_CFLAGS)lib/Kconfig.debug +9
-1@@ -213,6 +213,15 @@ config DEBUG_INFO_REDUCED DEBUG_INFO build and compile times are reduced too. Only works with newer gcc versions. +config DEBUG_INFO_COMPRESSED + bool "Compressed debugging information" + depends on DEBUG_INFO + depends on $(cc-option,-gz=zlib) + depends on $(ld-option,--compress-debug-sections=zlib) + help + Compress the debug information using zlib. Requires GCC 5.0+ or Clang + 5.0+. + config DEBUG_INFO_SPLIT bool "Produce split debuginfo in .dwo files" depends on DEBUG_INFO --2.17.1
--
You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/20200504031340.7103-1-nick.desaulniers%40gmail.com.
Quoted 46 lines · Sedat Dilek
Quoted 31 lines · Nick Desaulniers
As debug information gets larger and larger, it helps significantly save the size of vmlinux images to compress the information in the debug information sections. Note: this debug info is typically split off from the final compressed kernel image, which is why vmlinux is what's used in conjunction with GDB. Minimizing the debug info size should have no impact on boot times, or final compressed kernel image size.
All of the debug sections will have a
Cflag set.$ readelf -S <object file>$ bloaty vmlinux.gcc75.compressed.dwarf4 -- \ vmlinux.gcc75.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +18 [ = ] 0 [Unmapped] -73.3% -114Ki [ = ] 0 .debug_aranges -76.2% -2.01Mi [ = ] 0 .debug_frame -73.6% -2.89Mi [ = ] 0 .debug_str -80.7% -4.66Mi [ = ] 0 .debug_abbrev -82.9% -4.88Mi [ = ] 0 .debug_ranges -70.5% -9.04Mi [ = ] 0 .debug_line -79.3% -10.9Mi [ = ] 0 .debug_loc -39.5% -88.6Mi [ = ] 0 .debug_info -18.2% -123Mi [ = ] 0 TOTAL$ bloaty vmlinux.clang11.compressed.dwarf4 -- \ vmlinux.clang11.uncompressed.dwarf4FILE SIZE VM SIZE
Quoted 12 lines
-------------- -------------- +0.0% +23 [ = ] 0 [Unmapped] -65.6% -871 [ = ] 0 .debug_aranges -77.4% -1.84Mi [ = ] 0 .debug_frame -82.9% -2.33Mi [ = ] 0 .debug_abbrev -73.1% -2.43Mi [ = ] 0 .debug_str -84.8% -3.07Mi [ = ] 0 .debug_ranges -65.9% -8.62Mi [ = ] 0 .debug_line -86.2% -40.0Mi [ = ] 0 .debug_loc -42.0% -64.1Mi [ = ] 0 .debug_info -22.1% -122Mi [ = ] 0 TOTAL
Quoted 8 lines
Hi Nick,
thanks for the patch.
I have slightly modified it to adapt to Linux v5.7-rc4 (what was your base?).
Which linker did you use and has it an impact if you switch from ld.bfd to ld.lld?
lld has supported the linker option --compress-debug-sections=zlib since about 5.0.0 (https://reviews.llvm.org/D31941)
Quoted 175 lines
I tried a first normal run and in a 2nd one with
CONFIG_DEBUG_INFO_COMPRESSED=yboth with clang-10 and ld.lld-10.My numbers (sizes in MiB):
[ diffconfig ]
$ scripts/diffconfig /boot/config-5.7.0-rc4-1-amd64-clang /boot/config-5.7.0-rc4-2-amd64-clang BUILD_SALT "5.7.0-rc4-1-amd64-clang" -> "5.7.0-rc4-2-amd64-clang" +DEBUG_INFO_COMPRESSED y[ compiler and linker ]
$ clang-10 -v ClangBuiltLinux clang version 10.0.1 (https://github.com/llvm/llvm-project 92d5c1be9ee93850c0a8903f05f36a23ee835dc2) Target: x86_64-unknown-linux-gnu Thread model: posix InstalledDir: /home/dileks/src/llvm-toolchain/install/bin Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9 Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10 Candidate multilib: .;@m64 Candidate multilib: 32;@m32 Candidate multilib: x32;@mx32 Selected multilib: .;@m64$ ld.lld-10 -v LLD 10.0.1 (https://github.com/llvm/llvm-project 92d5c1be9ee93850c0a8903f05f36a23ee835dc2) (compatible with GNU linkers)[ sizes vmlinux ]
$ du -m 5.7.0-rc4-*/vmlinux* 409 5.7.0-rc4-1-amd64-clang/vmlinux 7 5.7.0-rc4-1-amd64-clang/vmlinux.compressed 404 5.7.0-rc4-1-amd64-clang/vmlinux.o 324 5.7.0-rc4-2-amd64-clang/vmlinux 7 5.7.0-rc4-2-amd64-clang/vmlinux.compressed 299 5.7.0-rc4-2-amd64-clang/vmlinux.o[ readelf (.debug_info as example) ]
$ readelf -S vmlinux.o [33] .debug_info PROGBITS 0000000000000000 01d6a5e8 0000000006be1ee6 0000000000000000 0 0 1$ readelf -S vmlinux.o [33] .debug_info PROGBITS 0000000000000000 01749f18 0000000002ef04d2 0000000000000000 C 0 0 1 <--- XXX: "C (compressed)" FlagKey to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info), L (link order), O (extra OS processing required), G (group), T (TLS), C (compressed), x (unknown), o (OS specific), E (exclude), l (large), p (processor specific)[ sizes linux-image debian packages ]
$ du -m 5.7.0-rc4-*/linux-image*.deb 47 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang_5.7.0~rc4-1~bullseye+dileks1_amd64.deb 424 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb 47 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang_5.7.0~rc4-2~bullseye+dileks1_amd64.deb 771 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb[ sizes linux-git dir (compilation finished ]
5.7.0-rc4-1-amd64-clang: 17963
/home/dileks/src/linux-kernel/linux
5.7.0-rc4-2-amd64-clang: 14328/home/dileks/src/linux-kernel/linux[ xz compressed linux-image-dbg packages ]
$ file linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb: Debian binary package (format 2.0), with control.tar.xz, data compression xz $ file linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb: Debian binary package (format 2.0), with control.tar.xz, data compression xz[ file-lists ]
$ dpkg --contents linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb | wc -l 4395 $ dpkg --contents linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb | wc -l 4395[ file-lists vmlinux ]
$ dpkg --contents linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb | grep vmlinux -rwxr-xr-x root/root 428588312 2020-05-04 06:15 ./usr/lib/debug/lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux lrwxrwxrwx root/root 0 2020-05-04 06:15 ./usr/lib/debug/boot/vmlinux-5.7.0-rc4-1-amd64-clang -> ../lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux lrwxrwxrwx root/root 0 2020-05-04 06:15 ./usr/lib/debug/vmlinux-5.7.0-rc4-1-amd64-clang -> lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux$ dpkg --contents linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb | grep vmlinux -rwxr-xr-x root/root 339341456 2020-05-04 12:24 ./usr/lib/debug/lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux lrwxrwxrwx root/root 0 2020-05-04 12:24 ./usr/lib/debug/boot/vmlinux-5.7.0-rc4-2-amd64-clang -> ../lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux lrwxrwxrwx root/root 0 2020-05-04 12:24 ./usr/lib/debug/vmlinux-5.7.0-rc4-2-amd64-clang -> lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux[ conclusion ]
As you can see there is a size-reduction in case of
vmlinux/vmlinux.o(debug) files...
...and my linux-git directory in total is smaller: 17963M vs. 14328M.But the resulting linux-image-dbg file is much fatter: 424M vs. 711M. XZ-compressing the gz/zlib-compressed vmlinux (debug) file results in a fatter linux-image-dbg package.
For a usage of vmlinux (debug) in a VM scenario this might be nice but seen from a debian repository perspective not.
For the sake of completeness: I have just installed and booted the "normal" linux-image debian package - not the debug packages.
Thanks.
Regards,
- Sedat -Quoted 31 lines
Suggested-by David Blaikie <[email protected]>Signed-off-by Nick Desaulniers <[email protected]>--- Makefile | 5 +++++ lib/Kconfig.debug | 9 +++++++++ 2 files changed, 14 insertions(+)Makefile +5
-0@@ -825,6 +825,11 @@ ifdef CONFIG_DEBUG_INFO_REDUCED DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \ $(call cc-option,-fno-var-tracking) endif + +ifdef CONFIG_DEBUG_INFO_COMPRESSED +DEBUG_CFLAGS += -gz=zlib +KBUILD_LDFLAGS += --compress-debug-sections=zlib +endif endif KBUILD_CFLAGS += $(DEBUG_CFLAGS)lib/Kconfig.debug +0
-0@@ -213,6 +213,15 @@ config DEBUG_INFO_REDUCED DEBUG_INFO build and compile times are reduced too. Only works with newer gcc versions.
Quoted 9 lines
lib/Kconfig.debug config DEBUG_INFO_REDUCED · line 224
Quoted 9 lines
lib/Kconfig.debug +9
-0+config DEBUG_INFO_COMPRESSED + bool "Compressed debugging information" + depends on DEBUG_INFO + depends on $(cc-option,-gz=zlib) + depends on $(ld-option,--compress-debug-sections=zlib) + help + Compress the debug information using zlib. Requires GCC 5.0+ or Clang + 5.0+. +
linker option --compress-debug-sections=zlib require binutils >= 2.26 (ld-option) Assembler option --compress-debug-sections=zlib also require binutils >= 2.26 (cc-option,-gz=zlib)
Quoted 15 lines
Quoted 10 lines
lib/Kconfig.debug +0
-1config DEBUG_INFO_SPLIT bool "Produce split debuginfo in .dwo files" depends on DEBUG_INFO --2.17.1
--
You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/20200504031340.7103-1-nick.desaulniers%40gmail.com.Signature · 3 lines
You received this message because you are subscribed to the Google Groups "Clang Built Linux" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CA%2BicZUUOaqeKeh6n4BJq2k6XQWAfNghUj57j42ZX5qyd3iOmLw%40mail.gmail.com.
Quoted 236 lines · Fangrui Song
Quoted 46 lines · Sedat Dilek
Quoted 31 lines · Nick Desaulniers
As debug information gets larger and larger, it helps significantly save the size of vmlinux images to compress the information in the debug information sections. Note: this debug info is typically split off from the final compressed kernel image, which is why vmlinux is what's used in conjunction with GDB. Minimizing the debug info size should have no impact on boot times, or final compressed kernel image size.
All of the debug sections will have a
Cflag set.$ readelf -S <object file>$ bloaty vmlinux.gcc75.compressed.dwarf4 -- \ vmlinux.gcc75.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +18 [ = ] 0 [Unmapped] -73.3% -114Ki [ = ] 0 .debug_aranges -76.2% -2.01Mi [ = ] 0 .debug_frame -73.6% -2.89Mi [ = ] 0 .debug_str -80.7% -4.66Mi [ = ] 0 .debug_abbrev -82.9% -4.88Mi [ = ] 0 .debug_ranges -70.5% -9.04Mi [ = ] 0 .debug_line -79.3% -10.9Mi [ = ] 0 .debug_loc -39.5% -88.6Mi [ = ] 0 .debug_info -18.2% -123Mi [ = ] 0 TOTAL$ bloaty vmlinux.clang11.compressed.dwarf4 -- \ vmlinux.clang11.uncompressed.dwarf4FILE SIZE VM SIZE
Quoted 12 lines
-------------- -------------- +0.0% +23 [ = ] 0 [Unmapped] -65.6% -871 [ = ] 0 .debug_aranges -77.4% -1.84Mi [ = ] 0 .debug_frame -82.9% -2.33Mi [ = ] 0 .debug_abbrev -73.1% -2.43Mi [ = ] 0 .debug_str -84.8% -3.07Mi [ = ] 0 .debug_ranges -65.9% -8.62Mi [ = ] 0 .debug_line -86.2% -40.0Mi [ = ] 0 .debug_loc -42.0% -64.1Mi [ = ] 0 .debug_info -22.1% -122Mi [ = ] 0 TOTALQuoted 8 lines
Hi Nick,
thanks for the patch.
I have slightly modified it to adapt to Linux v5.7-rc4 (what was your base?).
Which linker did you use and has it an impact if you switch from ld.bfd to ld.lld?
lld has supported the linker option --compress-debug-sections=zlib since about 5.0.0 (https://reviews.llvm.org/D31941)
Quoted 175 lines
I tried a first normal run and in a 2nd one with
CONFIG_DEBUG_INFO_COMPRESSED=yboth with clang-10 and ld.lld-10.My numbers (sizes in MiB):
[ diffconfig ]
$ scripts/diffconfig /boot/config-5.7.0-rc4-1-amd64-clang /boot/config-5.7.0-rc4-2-amd64-clang BUILD_SALT "5.7.0-rc4-1-amd64-clang" -> "5.7.0-rc4-2-amd64-clang" +DEBUG_INFO_COMPRESSED y[ compiler and linker ]
$ clang-10 -v ClangBuiltLinux clang version 10.0.1 (https://github.com/llvm/llvm-project 92d5c1be9ee93850c0a8903f05f36a23ee835dc2) Target: x86_64-unknown-linux-gnu Thread model: posix InstalledDir: /home/dileks/src/llvm-toolchain/install/bin Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9 Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10 Candidate multilib: .;@m64 Candidate multilib: 32;@m32 Candidate multilib: x32;@mx32 Selected multilib: .;@m64$ ld.lld-10 -v LLD 10.0.1 (https://github.com/llvm/llvm-project 92d5c1be9ee93850c0a8903f05f36a23ee835dc2) (compatible with GNU linkers)[ sizes vmlinux ]
$ du -m 5.7.0-rc4-*/vmlinux* 409 5.7.0-rc4-1-amd64-clang/vmlinux 7 5.7.0-rc4-1-amd64-clang/vmlinux.compressed 404 5.7.0-rc4-1-amd64-clang/vmlinux.o 324 5.7.0-rc4-2-amd64-clang/vmlinux 7 5.7.0-rc4-2-amd64-clang/vmlinux.compressed 299 5.7.0-rc4-2-amd64-clang/vmlinux.o[ readelf (.debug_info as example) ]
$ readelf -S vmlinux.o [33] .debug_info PROGBITS 0000000000000000 01d6a5e8 0000000006be1ee6 0000000000000000 0 0 1$ readelf -S vmlinux.o [33] .debug_info PROGBITS 0000000000000000 01749f18 0000000002ef04d2 0000000000000000 C 0 0 1 <--- XXX: "C (compressed)" FlagKey to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info), L (link order), O (extra OS processing required), G (group), T (TLS), C (compressed), x (unknown), o (OS specific), E (exclude), l (large), p (processor specific)[ sizes linux-image debian packages ]
$ du -m 5.7.0-rc4-*/linux-image*.deb 47 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang_5.7.0~rc4-1~bullseye+dileks1_amd64.deb 424 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb 47 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang_5.7.0~rc4-2~bullseye+dileks1_amd64.deb 771 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb[ sizes linux-git dir (compilation finished ]
5.7.0-rc4-1-amd64-clang: 17963
/home/dileks/src/linux-kernel/linux
5.7.0-rc4-2-amd64-clang: 14328/home/dileks/src/linux-kernel/linux[ xz compressed linux-image-dbg packages ]
$ file linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb: Debian binary package (format 2.0), with control.tar.xz, data compression xz $ file linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb: Debian binary package (format 2.0), with control.tar.xz, data compression xz[ file-lists ]
$ dpkg --contents linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb | wc -l 4395 $ dpkg --contents linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb | wc -l 4395[ file-lists vmlinux ]
$ dpkg --contents linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb | grep vmlinux -rwxr-xr-x root/root 428588312 2020-05-04 06:15 ./usr/lib/debug/lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux lrwxrwxrwx root/root 0 2020-05-04 06:15 ./usr/lib/debug/boot/vmlinux-5.7.0-rc4-1-amd64-clang -> ../lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux lrwxrwxrwx root/root 0 2020-05-04 06:15 ./usr/lib/debug/vmlinux-5.7.0-rc4-1-amd64-clang -> lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux$ dpkg --contents linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb | grep vmlinux -rwxr-xr-x root/root 339341456 2020-05-04 12:24 ./usr/lib/debug/lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux lrwxrwxrwx root/root 0 2020-05-04 12:24 ./usr/lib/debug/boot/vmlinux-5.7.0-rc4-2-amd64-clang -> ../lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux lrwxrwxrwx root/root 0 2020-05-04 12:24 ./usr/lib/debug/vmlinux-5.7.0-rc4-2-amd64-clang -> lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux[ conclusion ]
As you can see there is a size-reduction in case of
vmlinux/vmlinux.o(debug) files...
...and my linux-git directory in total is smaller: 17963M vs. 14328M.But the resulting linux-image-dbg file is much fatter: 424M vs. 711M. XZ-compressing the gz/zlib-compressed vmlinux (debug) file results in a fatter linux-image-dbg package.
For a usage of vmlinux (debug) in a VM scenario this might be nice but seen from a debian repository perspective not.
For the sake of completeness: I have just installed and booted the "normal" linux-image debian package - not the debug packages.
Thanks.
Regards,
- Sedat -Quoted 31 lines
Suggested-by David Blaikie <[email protected]>Signed-off-by Nick Desaulniers <[email protected]>--- Makefile | 5 +++++ lib/Kconfig.debug | 9 +++++++++ 2 files changed, 14 insertions(+)Makefile +5
-0@@ -825,6 +825,11 @@ ifdef CONFIG_DEBUG_INFO_REDUCED DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \ $(call cc-option,-fno-var-tracking) endif + +ifdef CONFIG_DEBUG_INFO_COMPRESSED +DEBUG_CFLAGS += -gz=zlib +KBUILD_LDFLAGS += --compress-debug-sections=zlib +endif endif KBUILD_CFLAGS += $(DEBUG_CFLAGS)lib/Kconfig.debug +0
-0@@ -213,6 +213,15 @@ config DEBUG_INFO_REDUCED DEBUG_INFO build and compile times are reduced too. Only works with newer gcc versions.
Quoted 12 lines
Quoted 9 lines
lib/Kconfig.debug config DEBUG_INFO_REDUCED · line 224
Quoted 9 lines
lib/Kconfig.debug +9
-0+config DEBUG_INFO_COMPRESSED + bool "Compressed debugging information" + depends on DEBUG_INFO + depends on $(cc-option,-gz=zlib) + depends on $(ld-option,--compress-debug-sections=zlib) + help + Compress the debug information using zlib. Requires GCC 5.0+ or Clang + 5.0+. +linker option --compress-debug-sections=zlib require binutils >= 2.26 (ld-option) Assembler option --compress-debug-sections=zlib also require binutils >= 2.26 (cc-option,-gz=zlib)
Nick,
I am OK with this patch.
Fangrui provided the minimal requirement for
--compress-debug-sections=zlib
Is it worth recording in the help text?
Do you want to send v2?
Signature · 2 lines
Best Regards Masahiro Yamada
Quoted 13 lines · Masahiro Yamada
Quoted 11 lines
Quoted 11 lines
Quoted 8 lines · Nick Desaulniers
As debug information gets larger and larger, it helps significantly save the size of vmlinux images to compress the information in the debug information sections. Note: this debug info is typically split off from the final compressed kernel image, which is why vmlinux is what's used in conjunction with GDB. Minimizing the debug info size should have no impact on boot times, or final compressed kernel image size.
Nick,
Quoted 8 lines
I am OK with this patch.
Fangrui provided the minimal requirement for
--compress-debug-sections=zlibIs it worth recording in the help text?
Do you want to send v2?
Yes I'd like to record that information. I can also record Sedat's Tested-by tag. Thank you for testing Sedat.
I don't know what "linux-image-dbg file" are, or why they would be bigger. The size of the debug info is the primary concern with this config. It sounds like however that file is created might be problematic.
Fangrui, I wasn't able to easily find what version of binutils first added support. Can you please teach me how to fish?
Another question I had for Fangrui is, if the linker can compress these sections, shouldn't we just have the linker do it, not the the compiler and assembler? IIUC the debug info can contain relocations, so the linker would have to decompress these, perform relocations, then recompress these? I guess having the compiler and assembler compress the debug info as well would minimize the size of the .o files on disk.
Otherwise I should add this flag to the assembler invocation, too, in v2. Thoughts?
I have a patch series that enables dwarf5 support in the kernel that I'm working up to. I wanted to send this first. Both roughly reduce the debug info size by 20% each, though I haven't measured them together, yet. Requires ToT binutils because there have been many fixes from reports of mine recently.
Signature · 2 lines
Thanks, ~Nick Desaulniers
Fangrui Song
Reviewed-byQuoted 26 lines · Nick Desaulniers
Quoted 13 lines · Masahiro Yamada
Quoted 11 lines
Quoted 11 lines
Quoted 8 lines · Nick Desaulniers
As debug information gets larger and larger, it helps significantly save the size of vmlinux images to compress the information in the debug information sections. Note: this debug info is typically split off from the final compressed kernel image, which is why vmlinux is what's used in conjunction with GDB. Minimizing the debug info size should have no impact on boot times, or final compressed kernel image size.
Nick,
Quoted 8 lines
I am OK with this patch.
Fangrui provided the minimal requirement for
--compress-debug-sections=zlibIs it worth recording in the help text?
Do you want to send v2?Yes I'd like to record that information. I can also record Sedat's Tested-by tag. Thank you for testing Sedat.
Quoted 7 lines
I don't know what "linux-image-dbg file" are, or why they would be bigger. The size of the debug info is the primary concern with this config. It sounds like however that file is created might be problematic.
Fangrui, I wasn't able to easily find what version of binutils first added support. Can you please teach me how to fish?
I actually downloaded https://ftp.gnu.org/gnu/binutils/ archives and located the sources... I think an easier way is:
% cd binutils-gdb
% git show binutils-2_26:./gas/as.c | grep compress-debug-sections --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi}]\n\
...
GNU as 2.25 only supports --compress-debug-sections which means "zlib-gnu" in newer versions.
Similarly, for GNU ld:
% git show binutils-2_26:./ld/lexsup.c | grep compress-debug-sections
--compress-debug-sections=[none|zlib|zlib-gnu|zlib-gabi]\n\
(I have spent a lot of time investigating GNU ld's behavior :)
Quoted 7 lines
Another question I had for Fangrui is, if the linker can compress these sections, shouldn't we just have the linker do it, not the the compiler and assembler? IIUC the debug info can contain relocations, so the linker would have to decompress these, perform relocations, then recompress these? I guess having the compiler and assembler compress the debug info as well would minimize the size of the .o files on disk.
The linker will decompress debug info unconditionally. Because input .debug_info sections need to be concatenated to form the output .debug_info . Whether the output .debug_info is compressed is controlled by the linker option --compress-debug-sections=zlib, which is not affected by the compression state of object files.
Both GNU as and GNU ld name the option --compress-debug-sections=zlib. In a compiler driver context, an unfamiliar user may find -Wa,--compress-debug-sections=zlib -Wl,--compress-debug-sections=zlib confusing:/
Quoted 2 lines
Otherwise I should add this flag to the assembler invocation, too, in v2. Thoughts?
Compressing object files along with the linked output should be fine. It can save disk space. (It'd be great if you paste the comparison with and w/o object files compressed)
Feel free to add:
Quoted 5 lines
I have a patch series that enables dwarf5 support in the kernel that I'm working up to. I wanted to send this first. Both roughly reduce the debug info size by 20% each, though I haven't measured them together, yet. Requires ToT binutils because there have been many fixes from reports of mine recently.
This will be awesome! I also heard that enabling DWARF v5 for our object files can easily make debug info size smaller by 20%. Glad that the kernel can benefit it as well:)
Quoted 2 lines · Fangrui Song
On 2020-05-12, Nick Desaulniers wrote:
...
Quoted 9 lines
Quoted 5 lines
I have a patch series that enables dwarf5 support in the kernel that I'm working up to. I wanted to send this first. Both roughly reduce the debug info size by 20% each, though I haven't measured them together, yet. Requires ToT binutils because there have been many fixes from reports of mine recently.
This will be awesome! I also heard that enabling DWARF v5 for our object files can easily make debug info size smaller by 20%. Glad that the kernel can benefit it as well:)
Please CC me on your pathcset - 20% smaller - I flip out.
- Sedat -
Quoted 9 lines · Fangrui Song
Quoted 3 lines
Fangrui, I wasn't able to easily find what version of binutils first added support. Can you please teach me how to fish?
I actually downloaded https://ftp.gnu.org/gnu/binutils/ archives and located the sources... I think an easier way is:
% cd binutils-gdb
% git show binutils-2_26:./gas/as.c | grep compress-debug-sections
This assumes you knew to look at the binutils-2_26 tag, which is putting the cart before the horse. ;)
I guess:
$ git log gas/as.c
/compress-debug-sections
commit 19a7fe52ae3d ("Make default compression gABI compliant")
looks related
$ git describe --contains "19a7fe52ae3d" | sed 's/~.*//'
users/hjl/linux/release/2.25.51.0.4
so it landed in 2.25.51.0.4.+ Nick, H.J.
I'm unfamiliar with the git tag conventions of binutils. Does a patch that landed in 2.25.51.0.4 mean it shipped in the official 2.25 release, or 2.26 release? Specifically, commit 19a7fe52ae3d.
Quoted 27 lines
--compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi}]\n\
...GNU as 2.25 only supports --compress-debug-sections which means "zlib-gnu" in newer versions.
Similarly, for GNU ld:
% git show binutils-2_26:./ld/lexsup.c | grep compress-debug-sections
--compress-debug-sections=[none|zlib|zlib-gnu|zlib-gabi]\n\(I have spent a lot of time investigating GNU ld's behavior :)
Quoted 7 lines
Another question I had for Fangrui is, if the linker can compress these sections, shouldn't we just have the linker do it, not the the compiler and assembler? IIUC the debug info can contain relocations, so the linker would have to decompress these, perform relocations, then recompress these? I guess having the compiler and assembler compress the debug info as well would minimize the size of the .o files on disk.
The linker will decompress debug info unconditionally. Because input .debug_info sections need to be concatenated to form the output .debug_info . Whether the output .debug_info is compressed is controlled by the linker option --compress-debug-sections=zlib, which is not affected by the compression state of object files.
Quoted 4 lines
Both GNU as and GNU ld name the option --compress-debug-sections=zlib. In a compiler driver context, an unfamiliar user may find -Wa,--compress-debug-sections=zlib -Wl,--compress-debug-sections=zlib confusing:/
The kernel uses the compiler as the driver for out of line assembly, as they are all preprocessed first. Most out of line assembly in the kernel uses the C preprocessor to #include headers that share #defines of common constants shared between C and asm. #ifdef __ASSEMBLY__ is used frequently in these headers. But for the linker, the linker itself is invoked as the driver, though there are a few inconsistencies we've cleaned up or still have to.
Quoted 11 lines
Quoted 3 lines
Otherwise I should add this flag to the assembler invocation, too, in v2. Thoughts?
Compressing object files along with the linked output should be fine. It can save disk space. (It'd be great if you paste the comparison with and w/o object files compressed)
Feel free to add:
Reviewed-by Fangrui Song <[email protected]>
Thanks, will add that to v2.
Quoted 10 lines
Quoted 6 lines
I have a patch series that enables dwarf5 support in the kernel that I'm working up to. I wanted to send this first. Both roughly reduce the debug info size by 20% each, though I haven't measured them together, yet. Requires ToT binutils because there have been many fixes from reports of mine recently.
This will be awesome! I also heard that enabling DWARF v5 for our object files can easily make debug info size smaller by 20%. Glad that the kernel can benefit it as well:)
Signature · 2 lines
Thanks, ~Nick Desaulniers
Hi Nick,
Quoted 4 lines
+ Nick, H.J.
I'm unfamiliar with the git tag conventions of binutils. Does a patch that landed in 2.25.51.0.4 mean it shipped in the official 2.25 release, or 2.26 release? Specifically, commit19a7fe52ae3d.
2.26.
The convention is that a released form of the binutils has a version number of X.XX or possible X.XX.N. The current mainline development sources have a version of X.XX.50 where X.XX is the latest release. (So the current mainline sources are version 2.34.50). When a release happens the XX value is incremented by one as part of the release process, and the .50 is dropped. (So the next binutils release will be 2.35).
So 2.25.51.0.4 is a development version which will then have been released as binutils 2.26.
Cheers
Nick
PATCH v2 1/1 Makefile: support compressed debug info
As debug information gets larger and larger, it helps significantly save the size of vmlinux images to compress the information in the debug information sections. Note: this debug info is typically split off from the final compressed kernel image, which is why vmlinux is what's used in conjunction with GDB. Minimizing the debug info size should have no impact on boot times, or final compressed kernel image size.
All of the debug sections will have a C flag set.
$ readelf -S <object file>
$ bloaty vmlinux.gcc75.compressed.dwarf4 -- \
vmlinux.gcc75.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +18 [ = ] 0 [Unmapped] -73.3% -114Ki [ = ] 0 .debug_aranges -76.2% -2.01Mi [ = ] 0 .debug_frame -73.6% -2.89Mi [ = ] 0 .debug_str -80.7% -4.66Mi [ = ] 0 .debug_abbrev -82.9% -4.88Mi [ = ] 0 .debug_ranges -70.5% -9.04Mi [ = ] 0 .debug_line -79.3% -10.9Mi [ = ] 0 .debug_loc -39.5% -88.6Mi [ = ] 0 .debug_info -18.2% -123Mi [ = ] 0 TOTAL
$ bloaty vmlinux.clang11.compressed.dwarf4 -- \
vmlinux.clang11.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +23 [ = ] 0 [Unmapped] -65.6% -871 [ = ] 0 .debug_aranges -77.4% -1.84Mi [ = ] 0 .debug_frame -82.9% -2.33Mi [ = ] 0 .debug_abbrev -73.1% -2.43Mi [ = ] 0 .debug_str -84.8% -3.07Mi [ = ] 0 .debug_ranges -65.9% -8.62Mi [ = ] 0 .debug_line -86.2% -40.0Mi [ = ] 0 .debug_loc -42.0% -64.1Mi [ = ] 0 .debug_info -22.1% -122Mi [ = ] 0 TOTAL
For x86_64 defconfig + LLVM=1 (before):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03 Maximum resident set size (kbytes): 43856
For x86_64 defconfig + LLVM=1 (after):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52 Maximum resident set size (kbytes): 1566776
---
Changes V1 -> V2:
* rebase on linux-next.
* Add assembler flags as per Fangrui.
* Add note about KDEB_COMPRESS+scripts/package/builddeb as per Sedat and Masahiro.
* Add note about bintutils version requirements as per Nick C.
* Add note about measured increased build time and max RSS.
Makefile | 6 ++++++ lib/Kconfig.debug | 15 +++++++++++++++ 2 files changed, 21 insertions(+)
Makefile +6 -0
@@ -822,6 +822,12 @@ DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \ $(call cc-option,-fno-var-tracking) endif +ifdef CONFIG_DEBUG_INFO_COMPRESSED +DEBUG_CFLAGS += -gz=zlib +KBUILD_AFLAGS += -Wa,--compress-debug-sections=zlib +KBUILD_LDFLAGS += --compress-debug-sections=zlib +endif + KBUILD_CFLAGS += $(DEBUG_CFLAGS) export DEBUG_CFLAGS
lib/Kconfig.debug +15 -0
@@ -225,6 +225,21 @@ config DEBUG_INFO_REDUCED DEBUG_INFO build and compile times are reduced too. Only works with newer gcc versions. +config DEBUG_INFO_COMPRESSED + bool "Compressed debugging information" + depends on DEBUG_INFO + depends on $(cc-option,-gz=zlib) + depends on $(as-option,-Wa,--compress-debug-sections=zlib) + depends on $(ld-option,--compress-debug-sections=zlib) + help + Compress the debug information using zlib. Requires GCC 5.0+ or Clang + 5.0+, binutils 2.26+, and zlib. + + Users of dpkg-deb via scripts/package/builddeb may + wish to set the $KDEB_COMPRESS env var to "none" to avoid recompressing + the debug info again with a different compression scheme, which can + result in larger binaries. + config DEBUG_INFO_SPLIT bool "Produce split debuginfo in .dwo files" depends on DEBUG_INFO
Signature · 1 line
2.26.2.761.g0e0b3e54be-goog
Quoted 47 lines · Nick Desaulniers
As debug information gets larger and larger, it helps significantly save the size of vmlinux images to compress the information in the debug information sections. Note: this debug info is typically split off from the final compressed kernel image, which is why vmlinux is what's used in conjunction with GDB. Minimizing the debug info size should have no impact on boot times, or final compressed kernel image size.
All of the debug sections will have a
Cflag set.$ readelf -S <object file>$ bloaty vmlinux.gcc75.compressed.dwarf4 -- \ vmlinux.gcc75.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +18 [ = ] 0 [Unmapped] -73.3% -114Ki [ = ] 0 .debug_aranges -76.2% -2.01Mi [ = ] 0 .debug_frame -73.6% -2.89Mi [ = ] 0 .debug_str -80.7% -4.66Mi [ = ] 0 .debug_abbrev -82.9% -4.88Mi [ = ] 0 .debug_ranges -70.5% -9.04Mi [ = ] 0 .debug_line -79.3% -10.9Mi [ = ] 0 .debug_loc -39.5% -88.6Mi [ = ] 0 .debug_info -18.2% -123Mi [ = ] 0 TOTAL$ bloaty vmlinux.clang11.compressed.dwarf4 -- \ vmlinux.clang11.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +23 [ = ] 0 [Unmapped] -65.6% -871 [ = ] 0 .debug_aranges -77.4% -1.84Mi [ = ] 0 .debug_frame -82.9% -2.33Mi [ = ] 0 .debug_abbrev -73.1% -2.43Mi [ = ] 0 .debug_str -84.8% -3.07Mi [ = ] 0 .debug_ranges -65.9% -8.62Mi [ = ] 0 .debug_line -86.2% -40.0Mi [ = ] 0 .debug_loc -42.0% -64.1Mi [ = ] 0 .debug_info -22.1% -122Mi [ = ] 0 TOTALFor
x86_64defconfig + LLVM=1 (before):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03 Maximum resident set size (kbytes): 43856
Quoted 5 lines
For
x86_64defconfig + LLVM=1 (after):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52 Maximum resident set size (kbytes): 1566776Suggested-by David Blaikie <[email protected]>
Sorry, I have mistyped David's email, it should be:
(I missed the first i in the last name)
Quoted 64 lines
Suggested-by Fangrui Song <[email protected]>Suggested-by Nick Clifton <[email protected]>Suggested-by Sedat Dilek <[email protected]>Tested-by Sedat Dilek <[email protected]>Signed-off-by Nick Desaulniers <[email protected]>---
Changes V1 -> V2:
* rebase on linux-next.
* Add assembler flags as per Fangrui.
* Add note about KDEB_COMPRESS+scripts/package/builddeb as per Sedat and Masahiro.
* Add note about bintutils version requirements as per Nick C.
* Add note about measured increased build time and max RSS.Makefile | 6 ++++++ lib/Kconfig.debug | 15 +++++++++++++++ 2 files changed, 21 insertions(+)Makefile +6
-0@@ -822,6 +822,12 @@ DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \ $(call cc-option,-fno-var-tracking) endif +ifdef CONFIG_DEBUG_INFO_COMPRESSED +DEBUG_CFLAGS += -gz=zlib +KBUILD_AFLAGS += -Wa,--compress-debug-sections=zlib +KBUILD_LDFLAGS += --compress-debug-sections=zlib +endif + KBUILD_CFLAGS += $(DEBUG_CFLAGS) export DEBUG_CFLAGSlib/Kconfig.debug +15
-1@@ -225,6 +225,21 @@ config DEBUG_INFO_REDUCED DEBUG_INFO build and compile times are reduced too. Only works with newer gcc versions. +config DEBUG_INFO_COMPRESSED + bool "Compressed debugging information" + depends on DEBUG_INFO + depends on $(cc-option,-gz=zlib) + depends on $(as-option,-Wa,--compress-debug-sections=zlib) + depends on $(ld-option,--compress-debug-sections=zlib) + help + Compress the debug information using zlib. Requires GCC 5.0+ or Clang + 5.0+, binutils 2.26+, and zlib. + + Users of dpkg-deb via scripts/package/builddeb may + wish to set the $KDEB_COMPRESS env var to "none" to avoid recompressing + the debug info again with a different compression scheme, which can + result in larger binaries. + config DEBUG_INFO_SPLIT bool "Produce split debuginfo in .dwo files" depends on DEBUG_INFO --2.26.2.761.g0e0b3e54be-goog
Signature · 2 lines
Thanks, ~Nick Desaulniers
Andrew Morton
maintainerQuoted 42 lines · On Wed, 20 May 2020 12:36:36 -0700 Nick Desaulniers <[email protected]>…
As debug information gets larger and larger, it helps significantly save the size of vmlinux images to compress the information in the debug information sections. Note: this debug info is typically split off from the final compressed kernel image, which is why vmlinux is what's used in conjunction with GDB. Minimizing the debug info size should have no impact on boot times, or final compressed kernel image size.
All of the debug sections will have a
Cflag set.$ readelf -S <object file>$ bloaty vmlinux.gcc75.compressed.dwarf4 -- \ vmlinux.gcc75.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +18 [ = ] 0 [Unmapped] -73.3% -114Ki [ = ] 0 .debug_aranges -76.2% -2.01Mi [ = ] 0 .debug_frame -73.6% -2.89Mi [ = ] 0 .debug_str -80.7% -4.66Mi [ = ] 0 .debug_abbrev -82.9% -4.88Mi [ = ] 0 .debug_ranges -70.5% -9.04Mi [ = ] 0 .debug_line -79.3% -10.9Mi [ = ] 0 .debug_loc -39.5% -88.6Mi [ = ] 0 .debug_info -18.2% -123Mi [ = ] 0 TOTAL$ bloaty vmlinux.clang11.compressed.dwarf4 -- \ vmlinux.clang11.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +23 [ = ] 0 [Unmapped] -65.6% -871 [ = ] 0 .debug_aranges -77.4% -1.84Mi [ = ] 0 .debug_frame -82.9% -2.33Mi [ = ] 0 .debug_abbrev -73.1% -2.43Mi [ = ] 0 .debug_str -84.8% -3.07Mi [ = ] 0 .debug_ranges -65.9% -8.62Mi [ = ] 0 .debug_line -86.2% -40.0Mi [ = ] 0 .debug_loc -42.0% -64.1Mi [ = ] 0 .debug_info -22.1% -122Mi [ = ] 0 TOTAL
Quoted 7 lines
For
x86_64defconfig + LLVM=1 (before):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03 Maximum resident set size (kbytes): 43856For
x86_64defconfig + LLVM=1 (after):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52 Maximum resident set size (kbytes): 1566776
I'm not sure who we're expecting to merge this, but I like shiny things ;)
Quoted 20 lines
Makefile +6
-0@@ -822,6 +822,12 @@ DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \ $(call cc-option,-fno-var-tracking) endif +ifdef CONFIG_DEBUG_INFO_COMPRESSED +DEBUG_CFLAGS += -gz=zlib +KBUILD_AFLAGS += -Wa,--compress-debug-sections=zlib +KBUILD_LDFLAGS += --compress-debug-sections=zlib +endif + KBUILD_CFLAGS += $(DEBUG_CFLAGS) export DEBUG_CFLAGSlib/Kconfig.debug +0
-0
lib/Kconfig.debug config DEBUG_INFO_REDUCED · line 233
Quoted 9 lines
lib/Kconfig.debug +6
-0@@ -225,6 +225,21 @@ config DEBUG_INFO_REDUCED DEBUG_INFO build and compile times are reduced too. Only works with newer gcc versions. +config DEBUG_INFO_COMPRESSED + bool "Compressed debugging information" + depends on DEBUG_INFO + depends on $(cc-option,-gz=zlib) + depends on $(as-option,-Wa,--compress-debug-sections=zlib) + depends on $(ld-option,--compress-debug-sections=zlib)
huh, I didn't know kbuild/kconfig could do this. Does it all work as expected when cross-compiling?
Quoted 9 lines
lib/Kconfig.debug +9
-0+ help + Compress the debug information using zlib. Requires GCC 5.0+ or Clang + 5.0+, binutils 2.26+, and zlib. + + Users of dpkg-deb via scripts/package/builddeb may + wish to set the $KDEB_COMPRESS env var to "none" to avoid recompressing + the debug info again with a different compression scheme, which can + result in larger binaries. +
Quoted 47 lines · Nick Desaulniers' via Clang Built Linux
As debug information gets larger and larger, it helps significantly save the size of vmlinux images to compress the information in the debug information sections. Note: this debug info is typically split off from the final compressed kernel image, which is why vmlinux is what's used in conjunction with GDB. Minimizing the debug info size should have no impact on boot times, or final compressed kernel image size.
All of the debug sections will have a
Cflag set.$ readelf -S <object file>$ bloaty vmlinux.gcc75.compressed.dwarf4 -- \ vmlinux.gcc75.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +18 [ = ] 0 [Unmapped] -73.3% -114Ki [ = ] 0 .debug_aranges -76.2% -2.01Mi [ = ] 0 .debug_frame -73.6% -2.89Mi [ = ] 0 .debug_str -80.7% -4.66Mi [ = ] 0 .debug_abbrev -82.9% -4.88Mi [ = ] 0 .debug_ranges -70.5% -9.04Mi [ = ] 0 .debug_line -79.3% -10.9Mi [ = ] 0 .debug_loc -39.5% -88.6Mi [ = ] 0 .debug_info -18.2% -123Mi [ = ] 0 TOTAL$ bloaty vmlinux.clang11.compressed.dwarf4 -- \ vmlinux.clang11.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +23 [ = ] 0 [Unmapped] -65.6% -871 [ = ] 0 .debug_aranges -77.4% -1.84Mi [ = ] 0 .debug_frame -82.9% -2.33Mi [ = ] 0 .debug_abbrev -73.1% -2.43Mi [ = ] 0 .debug_str -84.8% -3.07Mi [ = ] 0 .debug_ranges -65.9% -8.62Mi [ = ] 0 .debug_line -86.2% -40.0Mi [ = ] 0 .debug_loc -42.0% -64.1Mi [ = ] 0 .debug_info -22.1% -122Mi [ = ] 0 TOTALFor
x86_64defconfig + LLVM=1 (before):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03 Maximum resident set size (kbytes): 43856
Quoted 6 lines
For
x86_64defconfig + LLVM=1 (after):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52 Maximum resident set size (kbytes): 1566776Suggested-by David Blaikie <[email protected]>Suggested-by Fangrui Song <[email protected]>
Suggested-by -> Reviewed-by
https://patchwork.kernel.org/patch/11524939/#23349551
Quoted 1 line
Suggested-by Nick Clifton <[email protected]>
I do not know where this tag came from.
Nick Clifton taught us the version rule of binutils,but did not state anything about this patch itself.
https://patchwork.kernel.org/patch/11524939/#23355175
Quoted 1 line
Suggested-by Sedat Dilek <[email protected]>
I do not see the source of this tag, either...
Quoted 3 lines
Tested-by Sedat Dilek <[email protected]>Signed-off-by Nick Desaulniers <[email protected]>---
snip
lib/Kconfig.debug config DEBUG_INFO_REDUCED · line 232
Quoted 11 lines
lib/Kconfig.debug +5
-0@@ -225,6 +225,21 @@ config DEBUG_INFO_REDUCED DEBUG_INFO build and compile times are reduced too. Only works with newer gcc versions. +config DEBUG_INFO_COMPRESSED + bool "Compressed debugging information" + depends on DEBUG_INFO + depends on $(cc-option,-gz=zlib) + depends on $(as-option,-Wa,--compress-debug-sections=zlib)
This does not work. (always false)
You cannot enable this option.
The comma between -Wa and --compress-debug-sections=zlib is eaten by Kconfig parser because commas are delimiters of function parameters.
Please write like this.
depends on $(as-option,-Wa$(comma)--compress-debug-sections=zlib)
lib/Kconfig.debug config DEBUG_INFO_REDUCED · line 241
Quoted 9 lines
lib/Kconfig.debug +9
-0+ depends on $(ld-option,--compress-debug-sections=zlib) + help + Compress the debug information using zlib. Requires GCC 5.0+ or Clang + 5.0+, binutils 2.26+, and zlib. + + Users of dpkg-deb via scripts/package/builddeb may + wish to set the $KDEB_COMPRESS env var to "none" to avoid recompressing + the debug info again with a different compression scheme, which can + result in larger binaries.
No. This is not correct.
CONFIG_DEBUG_INFO_COMPRESSED compresses the only debug info part. The other parts still get by benefit from the default KDEB_COMPRESS=xz.
The numbers are here:
CONFIG_DEBUG_INFO_COMPRESSED=y
-rw-r--r-- 1 masahiro masahiro 209077584 May 21 11:19 linux-image-5.7.0-rc5+-dbg_5.7.0-rc5+-26_amd64.deb
CONFIG_DEBUG_INFO_COMPRESSED=y and KDEB_COMPRESS=none -rw-r--r-- 1 masahiro masahiro 643051712 May 21 11:22 linux-image-5.7.0-rc5+-dbg_5.7.0-rc5+-27_amd64.deb
CONFIG_DEBUG_INFO_COMPRESSED=n
-rw-r--r-- 1 masahiro masahiro 112200308 May 21 11:40 linux-image-5.7.0-rc5+-dbg_5.7.0-rc5+-30_amd64.deb
For the deb package size perspective,
it is better to keep KDEB_COMPRESS as default.
The main motivation for changing KDEB_COMPRESS
is the build speed. (see commit 1a7f0a34ea7d05)
CONFIG_DEBUG_INFO_COMPRESSED has a downside for the debug deb package, but we need to accept it.
Signature · 2 lines
Best Regards Masahiro Yamada
Quoted 7 lines · Masahiro Yamada
Quoted 2 lines
Suggested-by Fangrui Song <[email protected]>Suggested-by -> Reviewed-by
Yes, my mistake.
Quoted 8 lines
Quoted 1 line
Suggested-by Nick Clifton <[email protected]>I do not know where this tag came from.
Nick Clifton taught us the version rule of binutils,but did not state anything about this patch itself.
Quoted 6 lines
https://patchwork.kernel.org/patch/11524939/#23355175
Quoted 1 line
Suggested-by Sedat Dilek <[email protected]>I do not see the source of this tag, either...
Not all contributions to open source need to be in the form of patches. Both Sedat and Nick gave suggestions which ultimately informed the contents of this patch. They should be rewarded for their efforts, and incentivized to help improve the code base further. I think suggested by tags are a good way to do that; but if it's against a written convention or if you still disagree, it's not the end of the world to me, and you may drop those tags from the v3.
Quoted 6 lines
Quoted 6 lines
lib/Kconfig.debug +0
-0@@ -225,6 +225,21 @@ config DEBUG_INFO_REDUCED DEBUG_INFO build and compile times are reduced too. Only works with newer gcc versions.
Quoted 7 lines
lib/Kconfig.debug config DEBUG_INFO_REDUCED · line 232
Quoted 5 lines
lib/Kconfig.debug +5
-0+config DEBUG_INFO_COMPRESSED + bool "Compressed debugging information" + depends on DEBUG_INFO + depends on $(cc-option,-gz=zlib) + depends on $(as-option,-Wa,--compress-debug-sections=zlib)This does not work. (always false)
Technically, always true. -Wa disables all warnings from the assembler. Also, I did test this via make menuconfig.
Quoted 10 lines
You cannot enable this option.
The comma between -Wa and --compress-debug-sections=zlib is eaten by
Kconfigparser because commas are delimiters of function parameters.Please write like this.
depends on $(as-option,-Wa$(comma)--compress-debug-sections=zlib)
You're right, I knew this bug forgot. Will fix in v3.
Quoted 39 lines
lib/Kconfig.debug config DEBUG_INFO_REDUCED · line 241
Quoted 9 lines
lib/Kconfig.debug +9
-0+ depends on $(ld-option,--compress-debug-sections=zlib) + help + Compress the debug information using zlib. Requires GCC 5.0+ or Clang + 5.0+, binutils 2.26+, and zlib. + + Users of dpkg-deb via scripts/package/builddeb may + wish to set the $KDEB_COMPRESS env var to "none" to avoid recompressing + the debug info again with a different compression scheme, which can + result in larger binaries.No. This is not correct.
CONFIG_DEBUG_INFO_COMPRESSEDcompresses the only debug info part. The other parts still get by benefit from the default KDEB_COMPRESS=xz.The numbers are here:
CONFIG_DEBUG_INFO_COMPRESSED=y
-rw-r--r-- 1 masahiro masahiro 209077584 May 21 11:19 linux-image-5.7.0-rc5+-dbg_5.7.0-rc5+-26_amd64.deb
CONFIG_DEBUG_INFO_COMPRESSED=yand KDEB_COMPRESS=none -rw-r--r-- 1 masahiro masahiro 643051712 May 21 11:22 linux-image-5.7.0-rc5+-dbg_5.7.0-rc5+-27_amd64.deb
CONFIG_DEBUG_INFO_COMPRESSED=n
-rw-r--r-- 1 masahiro masahiro 112200308 May 21 11:40 linux-image-5.7.0-rc5+-dbg_5.7.0-rc5+-30_amd64.debFor the deb package size perspective,
it is better to keepKDEB_COMPRESSas default.
Quoted 8 lines
The main motivation for changing
KDEB_COMPRESS
is the build speed. (see commit1a7f0a34ea7d05)
CONFIG_DEBUG_INFO_COMPRESSEDhas a downside for the debug deb package, but we need to accept it.
Ah, I see. Thank you for those measurements. I'll send a v3 with fixed up help text, but ultimately, I don't really care what it says here. Please feel empowered to reword it should you choose to accept
+ apply it.
Signature · 2 lines
Thanks, ~Nick Desaulniers
PATCH v3 1/1 Makefile: support compressed debug info
As debug information gets larger and larger, it helps significantly save the size of vmlinux images to compress the information in the debug information sections. Note: this debug info is typically split off from the final compressed kernel image, which is why vmlinux is what's used in conjunction with GDB. Minimizing the debug info size should have no impact on boot times, or final compressed kernel image size.
All of the debug sections will have a C flag set.
$ readelf -S <object file>
$ bloaty vmlinux.gcc75.compressed.dwarf4 -- \
vmlinux.gcc75.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +18 [ = ] 0 [Unmapped] -73.3% -114Ki [ = ] 0 .debug_aranges -76.2% -2.01Mi [ = ] 0 .debug_frame -73.6% -2.89Mi [ = ] 0 .debug_str -80.7% -4.66Mi [ = ] 0 .debug_abbrev -82.9% -4.88Mi [ = ] 0 .debug_ranges -70.5% -9.04Mi [ = ] 0 .debug_line -79.3% -10.9Mi [ = ] 0 .debug_loc -39.5% -88.6Mi [ = ] 0 .debug_info -18.2% -123Mi [ = ] 0 TOTAL
$ bloaty vmlinux.clang11.compressed.dwarf4 -- \
vmlinux.clang11.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +23 [ = ] 0 [Unmapped] -65.6% -871 [ = ] 0 .debug_aranges -77.4% -1.84Mi [ = ] 0 .debug_frame -82.9% -2.33Mi [ = ] 0 .debug_abbrev -73.1% -2.43Mi [ = ] 0 .debug_str -84.8% -3.07Mi [ = ] 0 .debug_ranges -65.9% -8.62Mi [ = ] 0 .debug_line -86.2% -40.0Mi [ = ] 0 .debug_loc -42.0% -64.1Mi [ = ] 0 .debug_info -22.1% -122Mi [ = ] 0 TOTAL
For x86_64 defconfig + LLVM=1 (before):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03 Maximum resident set size (kbytes): 43856
For x86_64 defconfig + LLVM=1 (after):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52 Maximum resident set size (kbytes): 1566776
---
Changes V2 -> V3:
* Fix blaikie@'s email addr.
* Fix Fangrui's Reviewed-by tag as per Masahiro.
* Fix help text as per Masahiro.
* Fix -Wa$(comma)foo as per Masahiro.
Changes V1 -> V2:
* rebase on linux-next.
* Add assembler flags as per Fangrui.
* Add note about KDEB_COMPRESS+scripts/package/builddeb as per Sedat and Masahiro.
* Add note about bintutils version requirements as per Nick C.
* Add note about measured increased build time and max RSS.
Makefile | 6 ++++++ lib/Kconfig.debug | 17 +++++++++++++++++ 2 files changed, 23 insertions(+)
Makefile +6 -0
@@ -822,6 +822,12 @@ DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \ $(call cc-option,-fno-var-tracking) endif +ifdef CONFIG_DEBUG_INFO_COMPRESSED +DEBUG_CFLAGS += -gz=zlib +KBUILD_AFLAGS += -Wa,--compress-debug-sections=zlib +KBUILD_LDFLAGS += --compress-debug-sections=zlib +endif + KBUILD_CFLAGS += $(DEBUG_CFLAGS) export DEBUG_CFLAGS
lib/Kconfig.debug +17 -0
@@ -225,6 +225,23 @@ config DEBUG_INFO_REDUCED DEBUG_INFO build and compile times are reduced too. Only works with newer gcc versions. +config DEBUG_INFO_COMPRESSED + bool "Compressed debugging information" + depends on DEBUG_INFO + depends on $(cc-option,-gz=zlib) + depends on $(as-option,-Wa$(comma)--compress-debug-sections=zlib) + depends on $(ld-option,--compress-debug-sections=zlib) + help + Compress the debug information using zlib. Requires GCC 5.0+ or Clang + 5.0+, binutils 2.26+, and zlib. + + Users of dpkg-deb via scripts/package/builddeb may find an increase in + size of their debug .deb packages with this config set, due to the + debug info being compressed with zlib, then the object files being + recompressed with a different compression scheme. But this is still + preferable to setting $KDEB_COMPRESS to "none" which would be even + larger. + config DEBUG_INFO_SPLIT bool "Produce split debuginfo in .dwo files" depends on DEBUG_INFO
Signature · 1 line
2.27.0.rc0.183.gde8f92d652-goog
Quoted 51 lines · Nick Desaulniers
As debug information gets larger and larger, it helps significantly save the size of vmlinux images to compress the information in the debug information sections. Note: this debug info is typically split off from the final compressed kernel image, which is why vmlinux is what's used in conjunction with GDB. Minimizing the debug info size should have no impact on boot times, or final compressed kernel image size.
All of the debug sections will have a
Cflag set.$ readelf -S <object file>$ bloaty vmlinux.gcc75.compressed.dwarf4 -- \ vmlinux.gcc75.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +18 [ = ] 0 [Unmapped] -73.3% -114Ki [ = ] 0 .debug_aranges -76.2% -2.01Mi [ = ] 0 .debug_frame -73.6% -2.89Mi [ = ] 0 .debug_str -80.7% -4.66Mi [ = ] 0 .debug_abbrev -82.9% -4.88Mi [ = ] 0 .debug_ranges -70.5% -9.04Mi [ = ] 0 .debug_line -79.3% -10.9Mi [ = ] 0 .debug_loc -39.5% -88.6Mi [ = ] 0 .debug_info -18.2% -123Mi [ = ] 0 TOTAL$ bloaty vmlinux.clang11.compressed.dwarf4 -- \ vmlinux.clang11.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +23 [ = ] 0 [Unmapped] -65.6% -871 [ = ] 0 .debug_aranges -77.4% -1.84Mi [ = ] 0 .debug_frame -82.9% -2.33Mi [ = ] 0 .debug_abbrev -73.1% -2.43Mi [ = ] 0 .debug_str -84.8% -3.07Mi [ = ] 0 .debug_ranges -65.9% -8.62Mi [ = ] 0 .debug_line -86.2% -40.0Mi [ = ] 0 .debug_loc -42.0% -64.1Mi [ = ] 0 .debug_info -22.1% -122Mi [ = ] 0 TOTALFor
x86_64defconfig + LLVM=1 (before):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03 Maximum resident set size (kbytes): 43856For
x86_64defconfig + LLVM=1 (after):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52 Maximum resident set size (kbytes): 1566776
Quoted 5 lines
Suggested-by David Blaikie <[email protected]>Suggested-by Nick Clifton <[email protected]>Suggested-by Sedat Dilek <[email protected]>Reviewed-by Fangrui Song <[email protected]>Tested-by Sedat Dilek <[email protected]>
Re-tested V3 on top of Linux v5.7-rc6+ with Clang/LLD v10.0.1-rc1.
- Sedat -
Quoted 67 lines
Signed-off-by Nick Desaulniers <[email protected]>---
Changes V2 -> V3:
* Fix blaikie@'s email addr.
* Fix Fangrui's Reviewed-by tag as per Masahiro.
* Fix help text as per Masahiro.
* Fix -Wa$(comma)foo as per Masahiro.Changes V1 -> V2:
* rebase on linux-next.
* Add assembler flags as per Fangrui.
* Add note about KDEB_COMPRESS+scripts/package/builddeb as per Sedat and Masahiro.
* Add note about bintutils version requirements as per Nick C.
* Add note about measured increased build time and max RSS.Makefile | 6 ++++++ lib/Kconfig.debug | 17 +++++++++++++++++ 2 files changed, 23 insertions(+)Makefile +6
-0@@ -822,6 +822,12 @@ DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \ $(call cc-option,-fno-var-tracking) endif +ifdef CONFIG_DEBUG_INFO_COMPRESSED +DEBUG_CFLAGS += -gz=zlib +KBUILD_AFLAGS += -Wa,--compress-debug-sections=zlib +KBUILD_LDFLAGS += --compress-debug-sections=zlib +endif + KBUILD_CFLAGS += $(DEBUG_CFLAGS) export DEBUG_CFLAGSlib/Kconfig.debug +17
-1@@ -225,6 +225,23 @@ config DEBUG_INFO_REDUCED DEBUG_INFO build and compile times are reduced too. Only works with newer gcc versions. +config DEBUG_INFO_COMPRESSED + bool "Compressed debugging information" + depends on DEBUG_INFO + depends on $(cc-option,-gz=zlib) + depends on $(as-option,-Wa$(comma)--compress-debug-sections=zlib) + depends on $(ld-option,--compress-debug-sections=zlib) + help + Compress the debug information using zlib. Requires GCC 5.0+ or Clang + 5.0+, binutils 2.26+, and zlib. + + Users of dpkg-deb via scripts/package/builddeb may find an increase in + size of their debug .deb packages with this config set, due to the + debug info being compressed with zlib, then the object files being + recompressed with a different compression scheme. But this is still + preferable to setting $KDEB_COMPRESS to "none" which would be even + larger. + config DEBUG_INFO_SPLIT bool "Produce split debuginfo in .dwo files" depends on DEBUG_INFO --2.27.0.rc0.183.gde8f92d652-goog
Quoted 51 lines · Nick Desaulniers' via Clang Built Linux
As debug information gets larger and larger, it helps significantly save the size of vmlinux images to compress the information in the debug information sections. Note: this debug info is typically split off from the final compressed kernel image, which is why vmlinux is what's used in conjunction with GDB. Minimizing the debug info size should have no impact on boot times, or final compressed kernel image size.
All of the debug sections will have a
Cflag set.$ readelf -S <object file>$ bloaty vmlinux.gcc75.compressed.dwarf4 -- \ vmlinux.gcc75.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +18 [ = ] 0 [Unmapped] -73.3% -114Ki [ = ] 0 .debug_aranges -76.2% -2.01Mi [ = ] 0 .debug_frame -73.6% -2.89Mi [ = ] 0 .debug_str -80.7% -4.66Mi [ = ] 0 .debug_abbrev -82.9% -4.88Mi [ = ] 0 .debug_ranges -70.5% -9.04Mi [ = ] 0 .debug_line -79.3% -10.9Mi [ = ] 0 .debug_loc -39.5% -88.6Mi [ = ] 0 .debug_info -18.2% -123Mi [ = ] 0 TOTAL$ bloaty vmlinux.clang11.compressed.dwarf4 -- \ vmlinux.clang11.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +23 [ = ] 0 [Unmapped] -65.6% -871 [ = ] 0 .debug_aranges -77.4% -1.84Mi [ = ] 0 .debug_frame -82.9% -2.33Mi [ = ] 0 .debug_abbrev -73.1% -2.43Mi [ = ] 0 .debug_str -84.8% -3.07Mi [ = ] 0 .debug_ranges -65.9% -8.62Mi [ = ] 0 .debug_line -86.2% -40.0Mi [ = ] 0 .debug_loc -42.0% -64.1Mi [ = ] 0 .debug_info -22.1% -122Mi [ = ] 0 TOTALFor
x86_64defconfig + LLVM=1 (before):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03 Maximum resident set size (kbytes): 43856For
x86_64defconfig + LLVM=1 (after):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52 Maximum resident set size (kbytes): 1566776
Quoted 6 lines
Suggested-by David Blaikie <[email protected]>Suggested-by Nick Clifton <[email protected]>Suggested-by Sedat Dilek <[email protected]>Reviewed-by Fangrui Song <[email protected]>Tested-by Sedat Dilek <[email protected]>Signed-off-by Nick Desaulniers <[email protected]>
Suggested-by seems strange to me, but
I decided to not be worried too much.
Applied to linux-kbuild.
Thanks.
Signature · 2 lines
Best Regards Masahiro Yamada
Quoted 10 lines · Masahiro Yamada
Quoted 7 lines
Suggested-by David Blaikie <[email protected]>Suggested-by Nick Clifton <[email protected]>Suggested-by Sedat Dilek <[email protected]>Reviewed-by Fangrui Song <[email protected]>Tested-by Sedat Dilek <[email protected]>Signed-off-by Nick Desaulniers <[email protected]>
Quoted 2 lines
Suggested-by seems strange to me, but
I decided to not be worried too much.
Ah, sorry, Monday was a holiday. I unplugged for the long weekend.
https://en.wikipedia.org/wiki/Memorial_Day
I like the suggestion to simply say "thanks to ..." in the commit message and will use that next time. I was ok to send a v4 with it.
Quoted 2 lines
Applied to linux-kbuild.
Appreciated. I will have a dwarf-5 patch set, too. I'm not thrilled with how I wired up Kconfig; maybe posting it for feedback is better than me worrying about it too much. Hopefully will send this week, assuming there's not too many bugs that require my immediate attention (there never is...).
Signature · 2 lines
Thanks, ~Nick Desaulniers
Quoted 12 lines · Nick Desaulniers' via Clang Built Linux
Quoted 10 lines · Masahiro Yamada
Quoted 7 lines
Suggested-by David Blaikie <[email protected]>Suggested-by Nick Clifton <[email protected]>Suggested-by Sedat Dilek <[email protected]>Reviewed-by Fangrui Song <[email protected]>Tested-by Sedat Dilek <[email protected]>Signed-off-by Nick Desaulniers <[email protected]>
Quoted 8 lines
Quoted 2 lines
Suggested-by seems strange to me, but
I decided to not be worried too much.Ah, sorry, Monday was a holiday. I unplugged for the long weekend.
https://en.wikipedia.org/wiki/Memorial_DayI like the suggestion to simply say "thanks to ..." in the commit message and will use that next time. I was ok to send a v4 with it.
If you send v4, I will replace.
Thanks.
Signature · 2 lines
Best Regards Masahiro Yamada
PATCH v4 1/1 Makefile: support compressed debug info
As debug information gets larger and larger, it helps significantly save the size of vmlinux images to compress the information in the debug information sections. Note: this debug info is typically split off from the final compressed kernel image, which is why vmlinux is what's used in conjunction with GDB. Minimizing the debug info size should have no impact on boot times, or final compressed kernel image size.
All of the debug sections will have a C flag set.
$ readelf -S <object file>
$ bloaty vmlinux.gcc75.compressed.dwarf4 -- \
vmlinux.gcc75.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +18 [ = ] 0 [Unmapped] -73.3% -114Ki [ = ] 0 .debug_aranges -76.2% -2.01Mi [ = ] 0 .debug_frame -73.6% -2.89Mi [ = ] 0 .debug_str -80.7% -4.66Mi [ = ] 0 .debug_abbrev -82.9% -4.88Mi [ = ] 0 .debug_ranges -70.5% -9.04Mi [ = ] 0 .debug_line -79.3% -10.9Mi [ = ] 0 .debug_loc -39.5% -88.6Mi [ = ] 0 .debug_info -18.2% -123Mi [ = ] 0 TOTAL
$ bloaty vmlinux.clang11.compressed.dwarf4 -- \
vmlinux.clang11.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +23 [ = ] 0 [Unmapped] -65.6% -871 [ = ] 0 .debug_aranges -77.4% -1.84Mi [ = ] 0 .debug_frame -82.9% -2.33Mi [ = ] 0 .debug_abbrev -73.1% -2.43Mi [ = ] 0 .debug_str -84.8% -3.07Mi [ = ] 0 .debug_ranges -65.9% -8.62Mi [ = ] 0 .debug_line -86.2% -40.0Mi [ = ] 0 .debug_loc -42.0% -64.1Mi [ = ] 0 .debug_info -22.1% -122Mi [ = ] 0 TOTAL
For x86_64 defconfig + LLVM=1 (before):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03 Maximum resident set size (kbytes): 43856
For x86_64 defconfig + LLVM=1 (after):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52 Maximum resident set size (kbytes): 1566776
Thanks to:
Nick Clifton helped us to provide the minimal binutils version. Sedat Dilet found an increase in size of debug .deb package.
---
Changes V3 -> V4:
* Add thanks line to commit message as per Masahiro.
* Swap Sugguested-by to Cc for two lines in commit message, as per Masahiro.
Changes V2 -> V3:
* Fix blaikie@'s email addr.
* Fix Fangrui's Reviewed-by tag as per Masahiro.
* Fix help text as per Masahiro.
* Fix -Wa$(comma)foo as per Masahiro.
Changes V1 -> V2:
* rebase on linux-next.
* Add assembler flags as per Fangrui.
* Add note about KDEB_COMPRESS+scripts/package/builddeb as per Sedat and Masahiro.
* Add note about bintutils version requirements as per Nick C.
* Add note about measured increased build time and max RSS.
Makefile | 6 ++++++ arch/arm64/kernel/vdso32/Makefile | 2 +- lib/Kconfig.debug | 17 +++++++++++++++++ 3 files changed, 24 insertions(+), 1 deletion(-)
Makefile +6 -0
@@ -822,6 +822,12 @@ DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \ $(call cc-option,-fno-var-tracking) endif +ifdef CONFIG_DEBUG_INFO_COMPRESSED +DEBUG_CFLAGS += -gz=zlib +KBUILD_AFLAGS += -Wa,--compress-debug-sections=zlib +KBUILD_LDFLAGS += --compress-debug-sections=zlib +endif + KBUILD_CFLAGS += $(DEBUG_CFLAGS) export DEBUG_CFLAGS
arch/arm64/kernel/vdso32/Makefile +1 -1
@@ -135,7 +135,7 @@ c-obj-vdso-gettimeofday := vgettimeofday.o asm-obj-vdso := sigreturn.o ifneq ($(c-gettimeofday-y),) -VDSO_CFLAGS_gettimeofday_o += -include $(c-gettimeofday-y) +VDSO_CFLAGS_gettimeofday_o += -include $(c-gettimeofday-y) -marm endif VDSO_CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE) -Os
lib/Kconfig.debug +17 -0
@@ -225,6 +225,23 @@ config DEBUG_INFO_REDUCED DEBUG_INFO build and compile times are reduced too. Only works with newer gcc versions. +config DEBUG_INFO_COMPRESSED + bool "Compressed debugging information" + depends on DEBUG_INFO + depends on $(cc-option,-gz=zlib) + depends on $(as-option,-Wa$(comma)--compress-debug-sections=zlib) + depends on $(ld-option,--compress-debug-sections=zlib) + help + Compress the debug information using zlib. Requires GCC 5.0+ or Clang + 5.0+, binutils 2.26+, and zlib. + + Users of dpkg-deb via scripts/package/builddeb may find an increase in + size of their debug .deb packages with this config set, due to the + debug info being compressed with zlib, then the object files being + recompressed with a different compression scheme. But this is still + preferable to setting $KDEB_COMPRESS to "none" which would be even + larger. + config DEBUG_INFO_SPLIT bool "Produce split debuginfo in .dwo files" depends on DEBUG_INFO
Signature · 1 line
2.27.0.rc0.183.gde8f92d652-goog
Quoted 81 lines · Nick Desaulniers
As debug information gets larger and larger, it helps significantly save the size of vmlinux images to compress the information in the debug information sections. Note: this debug info is typically split off from the final compressed kernel image, which is why vmlinux is what's used in conjunction with GDB. Minimizing the debug info size should have no impact on boot times, or final compressed kernel image size.
All of the debug sections will have a
Cflag set.$ readelf -S <object file>$ bloaty vmlinux.gcc75.compressed.dwarf4 -- \ vmlinux.gcc75.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +18 [ = ] 0 [Unmapped] -73.3% -114Ki [ = ] 0 .debug_aranges -76.2% -2.01Mi [ = ] 0 .debug_frame -73.6% -2.89Mi [ = ] 0 .debug_str -80.7% -4.66Mi [ = ] 0 .debug_abbrev -82.9% -4.88Mi [ = ] 0 .debug_ranges -70.5% -9.04Mi [ = ] 0 .debug_line -79.3% -10.9Mi [ = ] 0 .debug_loc -39.5% -88.6Mi [ = ] 0 .debug_info -18.2% -123Mi [ = ] 0 TOTAL$ bloaty vmlinux.clang11.compressed.dwarf4 -- \ vmlinux.clang11.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +23 [ = ] 0 [Unmapped] -65.6% -871 [ = ] 0 .debug_aranges -77.4% -1.84Mi [ = ] 0 .debug_frame -82.9% -2.33Mi [ = ] 0 .debug_abbrev -73.1% -2.43Mi [ = ] 0 .debug_str -84.8% -3.07Mi [ = ] 0 .debug_ranges -65.9% -8.62Mi [ = ] 0 .debug_line -86.2% -40.0Mi [ = ] 0 .debug_loc -42.0% -64.1Mi [ = ] 0 .debug_info -22.1% -122Mi [ = ] 0 TOTALFor
x86_64defconfig + LLVM=1 (before):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03 Maximum resident set size (kbytes): 43856For
x86_64defconfig + LLVM=1 (after):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52 Maximum resident set size (kbytes): 1566776Thanks to:
Nick Clifton helped us to provide the minimal binutils version. Sedat Dilet found an increase in size of debug .deb package.Cc Nick Clifton <[email protected]>Cc Sedat Dilek <[email protected]>Suggested-by David Blaikie <[email protected]>Reviewed-by Fangrui Song <[email protected]>Tested-by Sedat Dilek <[email protected]>Signed-off-by Nick Desaulniers <[email protected]>---
Changes V3 -> V4:
* Add thanks line to commit message as per Masahiro.
* Swap Sugguested-by to Cc for two lines in commit message, as per Masahiro.Changes V2 -> V3:
* Fix blaikie@'s email addr.
* Fix Fangrui's Reviewed-by tag as per Masahiro.
* Fix help text as per Masahiro.
* Fix -Wa$(comma)foo as per Masahiro.Changes V1 -> V2:
* rebase on linux-next.
* Add assembler flags as per Fangrui.
* Add note about KDEB_COMPRESS+scripts/package/builddeb as per Sedat and Masahiro.
* Add note about bintutils version requirements as per Nick C.
* Add note about measured increased build time and max RSS.
Quoted 2 lines
Makefile | 6 ++++++ arch/arm64/kernel/vdso32/Makefile | 2 +-
Sorry, I was wondering why Will and TGLX got cc'ed. My tree was dirty when I amended ... was carrying another patch to send, please disregard v4, and sorry for the noise.
Quoted 64 lines
lib/Kconfig.debug | 17 +++++++++++++++++ 3 files changed, 24 insertions(+), 1 deletion(-)Makefile +6
-0@@ -822,6 +822,12 @@ DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \ $(call cc-option,-fno-var-tracking) endif +ifdef CONFIG_DEBUG_INFO_COMPRESSED +DEBUG_CFLAGS += -gz=zlib +KBUILD_AFLAGS += -Wa,--compress-debug-sections=zlib +KBUILD_LDFLAGS += --compress-debug-sections=zlib +endif + KBUILD_CFLAGS += $(DEBUG_CFLAGS) export DEBUG_CFLAGSarch/arm64/kernel/vdso32/Makefile +1
-1@@ -135,7 +135,7 @@ c-obj-vdso-gettimeofday := vgettimeofday.o asm-obj-vdso := sigreturn.o ifneq ($(c-gettimeofday-y),) -VDSO_CFLAGS_gettimeofday_o += -include $(c-gettimeofday-y) +VDSO_CFLAGS_gettimeofday_o += -include $(c-gettimeofday-y) -marm endif VDSO_CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE) -Oslib/Kconfig.debug +17
-1@@ -225,6 +225,23 @@ config DEBUG_INFO_REDUCED DEBUG_INFO build and compile times are reduced too. Only works with newer gcc versions. +config DEBUG_INFO_COMPRESSED + bool "Compressed debugging information" + depends on DEBUG_INFO + depends on $(cc-option,-gz=zlib) + depends on $(as-option,-Wa$(comma)--compress-debug-sections=zlib) + depends on $(ld-option,--compress-debug-sections=zlib) + help + Compress the debug information using zlib. Requires GCC 5.0+ or Clang + 5.0+, binutils 2.26+, and zlib. + + Users of dpkg-deb via scripts/package/builddeb may find an increase in + size of their debug .deb packages with this config set, due to the + debug info being compressed with zlib, then the object files being + recompressed with a different compression scheme. But this is still + preferable to setting $KDEB_COMPRESS to "none" which would be even + larger. + config DEBUG_INFO_SPLIT bool "Produce split debuginfo in .dwo files" depends on DEBUG_INFO --2.27.0.rc0.183.gde8f92d652-goog
Signature · 2 lines
Thanks, ~Nick Desaulniers
PATCH v5 1/1 Makefile: support compressed debug info
In reply to Masahiro YamadaAs debug information gets larger and larger, it helps significantly save the size of vmlinux images to compress the information in the debug information sections. Note: this debug info is typically split off from the final compressed kernel image, which is why vmlinux is what's used in conjunction with GDB. Minimizing the debug info size should have no impact on boot times, or final compressed kernel image size.
All of the debug sections will have a C flag set.
$ readelf -S <object file>
$ bloaty vmlinux.gcc75.compressed.dwarf4 -- \
vmlinux.gcc75.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +18 [ = ] 0 [Unmapped] -73.3% -114Ki [ = ] 0 .debug_aranges -76.2% -2.01Mi [ = ] 0 .debug_frame -73.6% -2.89Mi [ = ] 0 .debug_str -80.7% -4.66Mi [ = ] 0 .debug_abbrev -82.9% -4.88Mi [ = ] 0 .debug_ranges -70.5% -9.04Mi [ = ] 0 .debug_line -79.3% -10.9Mi [ = ] 0 .debug_loc -39.5% -88.6Mi [ = ] 0 .debug_info -18.2% -123Mi [ = ] 0 TOTAL
$ bloaty vmlinux.clang11.compressed.dwarf4 -- \
vmlinux.clang11.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +23 [ = ] 0 [Unmapped] -65.6% -871 [ = ] 0 .debug_aranges -77.4% -1.84Mi [ = ] 0 .debug_frame -82.9% -2.33Mi [ = ] 0 .debug_abbrev -73.1% -2.43Mi [ = ] 0 .debug_str -84.8% -3.07Mi [ = ] 0 .debug_ranges -65.9% -8.62Mi [ = ] 0 .debug_line -86.2% -40.0Mi [ = ] 0 .debug_loc -42.0% -64.1Mi [ = ] 0 .debug_info -22.1% -122Mi [ = ] 0 TOTAL
For x86_64 defconfig + LLVM=1 (before):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03 Maximum resident set size (kbytes): 43856
For x86_64 defconfig + LLVM=1 (after):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52 Maximum resident set size (kbytes): 1566776
Thanks to:
Nick Clifton helped us to provide the minimal binutils version. Sedat Dilet found an increase in size of debug .deb package.
---
Changes V4 -> V5:
* Drop unrelated hunk from a dirty tree.
Changes V3 -> V4:
* Add thanks line to commit message as per Masahiro.
* Swap Sugguested-by to Cc for two lines in commit message, as per Masahiro.
Changes V2 -> V3:
* Fix blaikie@'s email addr.
* Fix Fangrui's Reviewed-by tag as per Masahiro.
* Fix help text as per Masahiro.
* Fix -Wa$(comma)foo as per Masahiro.
Changes V1 -> V2:
* rebase on linux-next.
* Add assembler flags as per Fangrui.
* Add note about KDEB_COMPRESS+scripts/package/builddeb as per Sedat and Masahiro.
* Add note about bintutils version requirements as per Nick C.
* Add note about measured increased build time and max RSS.
Makefile | 6 ++++++ lib/Kconfig.debug | 17 +++++++++++++++++ 2 files changed, 23 insertions(+)
Makefile +6 -0
@@ -822,6 +822,12 @@ DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \ $(call cc-option,-fno-var-tracking) endif +ifdef CONFIG_DEBUG_INFO_COMPRESSED +DEBUG_CFLAGS += -gz=zlib +KBUILD_AFLAGS += -Wa,--compress-debug-sections=zlib +KBUILD_LDFLAGS += --compress-debug-sections=zlib +endif + KBUILD_CFLAGS += $(DEBUG_CFLAGS) export DEBUG_CFLAGS
lib/Kconfig.debug +17 -0
@@ -225,6 +225,23 @@ config DEBUG_INFO_REDUCED DEBUG_INFO build and compile times are reduced too. Only works with newer gcc versions. +config DEBUG_INFO_COMPRESSED + bool "Compressed debugging information" + depends on DEBUG_INFO + depends on $(cc-option,-gz=zlib) + depends on $(as-option,-Wa$(comma)--compress-debug-sections=zlib) + depends on $(ld-option,--compress-debug-sections=zlib) + help + Compress the debug information using zlib. Requires GCC 5.0+ or Clang + 5.0+, binutils 2.26+, and zlib. + + Users of dpkg-deb via scripts/package/builddeb may find an increase in + size of their debug .deb packages with this config set, due to the + debug info being compressed with zlib, then the object files being + recompressed with a different compression scheme. But this is still + preferable to setting $KDEB_COMPRESS to "none" which would be even + larger. + config DEBUG_INFO_SPLIT bool "Produce split debuginfo in .dwo files" depends on DEBUG_INFO
Signature · 1 line
2.27.0.rc0.183.gde8f92d652-goog
Quoted 27 lines · Nick Desaulniers' via Clang Built Linux
Quoted 7 lines · Masahiro Yamada
Quoted 2 lines
Suggested-by Fangrui Song <[email protected]>Suggested-by -> Reviewed-by
Yes, my mistake.
Quoted 8 lines
Quoted 1 line
Suggested-by Nick Clifton <[email protected]>I do not know where this tag came from.
Nick Clifton taught us the version rule of binutils,but did not state anything about this patch itself.
Quoted 6 lines
https://patchwork.kernel.org/patch/11524939/#23355175
Quoted 1 line
Suggested-by Sedat Dilek <[email protected]>I do not see the source of this tag, either...
Quoted 7 lines
Not all contributions to open source need to be in the form of patches. Both Sedat and Nick gave suggestions which ultimately informed the contents of this patch. They should be rewarded for their efforts, and incentivized to help improve the code base further. I think suggested by tags are a good way to do that; but if it's against a written convention or if you still disagree, it's not the end of the world to me, and you may drop those tags from the v3.
Documentation/process/submitting-patches.rst
gives the guideline.
"A Suggested-by: tag indicates that the patch idea is suggested by the person named and ensures credit to the person for the idea. Please note that this tag should not be added without the reporter's permission, especially if the idea was not posted in a public forum. That said, if we diligently credit our idea reporters, they will, hopefully, be inspired to help us again in the future."
I think this tag should be given to people who
gave the main idea to come up with
the main part of the patch.
Is that David Blaikie who suggested to compress the debug info ?
If so, definitely he deserves to have Suggested-by tag.
For the others, I do not think Suggested-by is a good fit.
I appreciate their contribution to improve this patch. So, maybe you can give credit in other form, for example, mention it in the commit log explicitly?
Nick Clifton helped us to provide the minimal binutils version. Sedat Dilet found an increase in size of debug .deb package.
Thanks.
Quoted 91 lines
Quoted 7 lines
Quoted 7 lines
lib/Kconfig.debug +0
-0@@ -225,6 +225,21 @@ config DEBUG_INFO_REDUCED DEBUG_INFO build and compile times are reduced too. Only works with newer gcc versions.Quoted 7 lines
lib/Kconfig.debug config DEBUG_INFO_REDUCED · line 232
Quoted 5 lines
lib/Kconfig.debug +5
-0+config DEBUG_INFO_COMPRESSED + bool "Compressed debugging information" + depends on DEBUG_INFO + depends on $(cc-option,-gz=zlib) + depends on $(as-option,-Wa,--compress-debug-sections=zlib)This does not work. (always false)
Technically, always true.
-Wadisables all warnings from the assembler. Also, I did test this viamake menuconfig.Quoted 10 lines
You cannot enable this option.
The comma between -Wa and --compress-debug-sections=zlib is eaten by
Kconfigparser because commas are delimiters of function parameters.Please write like this.
depends on $(as-option,-Wa$(comma)--compress-debug-sections=zlib)You're right, I knew this bug forgot. Will fix in v3.
Quoted 39 lines
lib/Kconfig.debug config DEBUG_INFO_REDUCED · line 241
Quoted 9 lines
lib/Kconfig.debug +9
-0+ depends on $(ld-option,--compress-debug-sections=zlib) + help + Compress the debug information using zlib. Requires GCC 5.0+ or Clang + 5.0+, binutils 2.26+, and zlib. + + Users of dpkg-deb via scripts/package/builddeb may + wish to set the $KDEB_COMPRESS env var to "none" to avoid recompressing + the debug info again with a different compression scheme, which can + result in larger binaries.No. This is not correct.
CONFIG_DEBUG_INFO_COMPRESSEDcompresses the only debug info part. The other parts still get by benefit from the default KDEB_COMPRESS=xz.The numbers are here:
CONFIG_DEBUG_INFO_COMPRESSED=y
-rw-r--r-- 1 masahiro masahiro 209077584 May 21 11:19 linux-image-5.7.0-rc5+-dbg_5.7.0-rc5+-26_amd64.deb
CONFIG_DEBUG_INFO_COMPRESSED=yand KDEB_COMPRESS=none -rw-r--r-- 1 masahiro masahiro 643051712 May 21 11:22 linux-image-5.7.0-rc5+-dbg_5.7.0-rc5+-27_amd64.deb
CONFIG_DEBUG_INFO_COMPRESSED=n
-rw-r--r-- 1 masahiro masahiro 112200308 May 21 11:40 linux-image-5.7.0-rc5+-dbg_5.7.0-rc5+-30_amd64.debFor the deb package size perspective,
it is better to keepKDEB_COMPRESSas default.Quoted 8 lines
The main motivation for changing
KDEB_COMPRESS
is the build speed. (see commit1a7f0a34ea7d05)
CONFIG_DEBUG_INFO_COMPRESSEDhas a downside for the debug deb package, but we need to accept it.Ah, I see. Thank you for those measurements. I'll send a v3 with fixed up help text, but ultimately, I don't really care what it says here. Please feel empowered to reword it should you choose to accept
+ apply it.
--
Thanks,
~Nick Desaulniers--
You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAKwvOd%3DjOr4ZaLx-dSNTqZnGRATY1PZktUfu4JGWKRwRH%3DUjnw%40mail.gmail.com.
Signature · 2 lines
Best Regards Masahiro Yamada
Quoted 66 lines · Masahiro Yamada
Quoted 27 lines · Nick Desaulniers' via Clang Built Linux
Quoted 7 lines · Masahiro Yamada
Quoted 2 lines
Suggested-by Fangrui Song <[email protected]>Suggested-by -> Reviewed-by
Yes, my mistake.
Quoted 8 lines
Quoted 1 line
Suggested-by Nick Clifton <[email protected]>I do not know where this tag came from.
Nick Clifton taught us the version rule of binutils,but did not state anything about this patch itself.
Quoted 6 lines
https://patchwork.kernel.org/patch/11524939/#23355175
Quoted 1 line
Suggested-by Sedat Dilek <[email protected]>I do not see the source of this tag, either...
Quoted 7 lines
Not all contributions to open source need to be in the form of patches. Both Sedat and Nick gave suggestions which ultimately informed the contents of this patch. They should be rewarded for their efforts, and incentivized to help improve the code base further. I think suggested by tags are a good way to do that; but if it's against a written convention or if you still disagree, it's not the end of the world to me, and you may drop those tags from the v3.
Documentation/process/submitting-patches.rst
gives the guideline."A Suggested-by: tag indicates that the patch idea is suggested by the person named and ensures credit to the person for the idea. Please note that this tag should not be added without the reporter's permission, especially if the idea was not posted in a public forum. That said, if we diligently credit our idea reporters, they will, hopefully, be inspired to help us again in the future."
I think this tag should be given to people who
gave the main idea to come up with
the main part of the patch.Is that David Blaikie who suggested to compress the debug info ?
If so, definitely he deserves to have Suggested-by tag.For the others, I do not think Suggested-by is a good fit.
I appreciate their contribution to improve this patch. So, maybe you can give credit in other form, for example, mention it in the commit log explicitly?
Quoted 5 lines
Nick Clifton helped us to provide the minimal binutils version. Sedat Dilet found an increase in size of debug .deb package.
Thanks.
Hi,
first my last name is Dilek - just for the sake of completeness. No, it is not my first name as Dilek is a female Turkish first name, so I do not want to change my gender.
So this discussions come up again and again.
Thus some own words on this - this is my personal opinion.
Like the author of Curl and DOH said at FOSDEM 2019 in Bruessel:
I am doing all this work - first - for myself - in my build and developing environment.
Very very egoistically!
"Share knowledge aggressively!"
...was Nick's words at First ClangBuiltLinux Meetup in Zurich 2020. In a 2nd round I share my knowledge and I like this - that's why I am doing Open Source.
For me it sounds like a "Suggested-by" tag or other credits like "Reviewed-by" have a higher value than a Tested-by tag.
*** The opposite is the case. ***
Here, I am on a Samsung SandyBridge CPU/GPU aka 2nd generation ultrabook series runing Debian/testing AMD64.
A slightly modified Debian-kernel linux-config takes me approx. 5 (in words five) hours of compiling and generating Debian packages.
Plus, testing.
Plus, testing.
Plus, testing.
In Linux-next times I run the whole Linux-Test-Project tests plus some FIO tests.
Finally, I decide depending from what is new and interesting to me to attend a full single Linux-kernel release cycle.
The last was Linux v5.3 which was the first release to be compile/link-able - with no modifications - with LLVM/Clang/LLD v9.0. For upcoming Linux v5.7 I have built each single RC Linux-kernel and used it in my daily work!
Since RC1 - for me running on bare metal counts - checking QEMU or other VM is nice - but showed me that says sometimes nothing.
Plus, I am building llvm-toolchains (LLVM/Clang/LLD) and testing with them (and report if needed).
"...if we diligently credit our idea reporters, they will, hopefully, be inspired to help us again in the future."
These are some motivating words...
My Tested-by is like a certificate - like a "Made in Germany" seal :-).
Virtual Greeting from North-West Germany,
- Sedat -
Quoted 25 lines · Sedat Dilek
*** The opposite is the case. ***
Here, I am on a Samsung SandyBridge CPU/GPU aka 2nd generation ultrabook series runing Debian/testing AMD64.
A slightly modified Debian-kernel linux-config takes me approx. 5 (in words five) hours of compiling and generating Debian packages.
Plus, testing.
Plus, testing.
Plus, testing.In Linux-next times I run the whole Linux-Test-Project tests plus some FIO tests.
Finally, I decide depending from what is new and interesting to me to attend a full single Linux-kernel release cycle.
The last was Linux v5.3 which was the first release to be compile/link-able - with no modifications - with LLVM/Clang/LLD v9.0. For upcoming Linux v5.7 I have built each single RC Linux-kernel and used it in my daily work!
Since RC1 - for me running on bare metal counts - checking QEMU or other VM is nice - but showed me that says sometimes nothing.
Quoted 2 lines
Plus, I am building llvm-toolchains (LLVM/Clang/LLD) and testing with them (and report if needed).
This is a lot of invaluable work. It means the world to me Sedat!
Quoted 7 lines
"...if we diligently credit our idea reporters, they will, hopefully, be inspired to help us again in the future."
These are some motivating words...
My Tested-by is like a certificate - like a "Made in Germany" seal :-).
I love this, it is. Maybe if folks on this thread are bored, they could help me with a personal project of mine?
https://github.com/nickdesaulniers/What-Open-Source-Means-To-Me
We could use more German, and Japanese.
Quoted 3 lines
Virtual Greeting from North-West Germany,
- Sedat -
Signature · 2 lines
Thanks, ~Nick Desaulniers
Quoted 24 lines · Nick Desaulniers
Quoted 13 lines · Masahiro Yamada
Quoted 11 lines
Quoted 11 lines
Quoted 8 lines · Nick Desaulniers
As debug information gets larger and larger, it helps significantly save the size of vmlinux images to compress the information in the debug information sections. Note: this debug info is typically split off from the final compressed kernel image, which is why vmlinux is what's used in conjunction with GDB. Minimizing the debug info size should have no impact on boot times, or final compressed kernel image size.
Nick,
Quoted 8 lines
I am OK with this patch.
Fangrui provided the minimal requirement for
--compress-debug-sections=zlibIs it worth recording in the help text?
Do you want to send v2?
Quoted 8 lines
Yes I'd like to record that information. I can also record Sedat's Tested-by tag. Thank you for testing Sedat.
I don't know what "linux-image-dbg file" are, or why they would be bigger. The size of the debug info is the primary concern with this config. It sounds like however that file is created might be problematic.
Hi Nick,
sorry, I try to explain the magic of "linux-image-dbg file".
In my workflow, I use the "scripts/package/{builddeb,mkdebian}" shipped with the Linux-kernel as a base to build my Debian packages.
With enabled debugging (CONFIG_DEBUG_INFO=y) a "linux-image-$KERNELRELEASE-dbg" Debian package is created.
As you can see below...
[ CONFIG_DEBUG_INFO_COMPRESSED=n ]
47 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang_5.7.0~rc4-1~bullseye+dileks1_amd64.deb
424 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb
[ CONFIG_DEBUG_INFO_COMPRESSED=y ]
47 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang_5.7.0~rc4-2~bullseye+dileks1_amd64.deb
771 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb
...there is minimal change in the size for the Debian package w/o debug-infos - approx. 47M.
As said 424M vs. 771M for the dbg packages.
There is another big benefit checking my recorded stats:
$ grep 'cache size' stats/5.7.0-rc4-*/ccache-s.txt
stats/5.7.0-rc4-1-amd64-clang/ccache-s.txt:cache size
4.7 GB
stats/5.7.0-rc4-1-amd64-clang/ccache-s.txt:max cache size
10.0 GB
stats/5.7.0-rc4-2-amd64-clang/ccache-s.txt:cache size
3.4 GB
stats/5.7.0-rc4-2-amd64-clang/ccache-s.txt:max cache size
10.0 GBSo the cache of ccache is reduced: 4.7 GB vs. 3.4 GB
If you have any questions, "Don't ask to ask - just ask." :-).
Thanks.
Regards,
- Sedat -
Nick,
Quoted 24 lines · Nick Desaulniers
Quoted 13 lines · Masahiro Yamada
Quoted 11 lines
Quoted 11 lines
Quoted 8 lines · Nick Desaulniers
As debug information gets larger and larger, it helps significantly save the size of vmlinux images to compress the information in the debug information sections. Note: this debug info is typically split off from the final compressed kernel image, which is why vmlinux is what's used in conjunction with GDB. Minimizing the debug info size should have no impact on boot times, or final compressed kernel image size.
Nick,
Quoted 8 lines
I am OK with this patch.
Fangrui provided the minimal requirement for
--compress-debug-sections=zlibIs it worth recording in the help text?
Do you want to send v2?
Quoted 7 lines
Yes I'd like to record that information. I can also record Sedat's Tested-by tag. Thank you for testing Sedat.
I don't know what "linux-image-dbg file" are, or why they would be bigger. The size of the debug info is the primary concern with this config. It sounds like however that file is created might be problematic.
As Sedat explained, deb package data
is compressed by xz, which is default.
You can use another compression method,
or disable compression if you desire.
"man dpkg-deb" says as follows:
-Zcompress-type
Specify which compression type to use when building a package. Allowed values are gzip, xz (since dpkg 1.15.6), and none (default is xz).
Kbuild supports KDEB_COMPRESS variable
to change the compression method.
See line 46 of scripts/package/builddeb.
If you are interested,
try "make bindeb-pkg" with/without CONFIG_DEBUG_INFO_COMPRESSED, and compare the size of the generated debug package.
As Sedat stated,
(plain data) -> compress by gzip -> compress by xz
is often less efficient than
(plain data) -> compress by xz
I hope this is clearer.
Quoted 22 lines
Fangrui, I wasn't able to easily find what version of binutils first added support. Can you please teach me how to fish?
Another question I had for Fangrui is, if the linker can compress these sections, shouldn't we just have the linker do it, not the the compiler and assembler? IIUC the debug info can contain relocations, so the linker would have to decompress these, perform relocations, then recompress these? I guess having the compiler and assembler compress the debug info as well would minimize the size of the .o files on disk.
Otherwise I should add this flag to the assembler invocation, too, in v2. Thoughts?
I have a patch series that enables dwarf5 support in the kernel that I'm working up to. I wanted to send this first. Both roughly reduce the debug info size by 20% each, though I haven't measured them together, yet. Requires ToT binutils because there have been many fixes from reports of mine recently.
--
Thanks,
~Nick Desaulniers
Signature · 2 lines
Best Regards Masahiro Yamada
Quoted 56 lines · Masahiro Yamada
Nick,
Quoted 24 lines · Nick Desaulniers
Quoted 13 lines · Masahiro Yamada
Quoted 11 lines
Quoted 11 lines
Quoted 8 lines · Nick Desaulniers
As debug information gets larger and larger, it helps significantly save the size of vmlinux images to compress the information in the debug information sections. Note: this debug info is typically split off from the final compressed kernel image, which is why vmlinux is what's used in conjunction with GDB. Minimizing the debug info size should have no impact on boot times, or final compressed kernel image size.
Nick,
Quoted 8 lines
I am OK with this patch.
Fangrui provided the minimal requirement for
--compress-debug-sections=zlibIs it worth recording in the help text?
Do you want to send v2?Quoted 7 lines
Yes I'd like to record that information. I can also record Sedat's Tested-by tag. Thank you for testing Sedat.
I don't know what "linux-image-dbg file" are, or why they would be bigger. The size of the debug info is the primary concern with this config. It sounds like however that file is created might be problematic.
As Sedat explained, deb package data
is compressed by xz, which is default.You can use another compression method,
or disable compression if you desire."man dpkg-deb" says as follows:
-Zcompress-type
Specify which compression type to use when building a package. Allowed values are gzip, xz (since dpkg 1.15.6), and none (default is xz).
Quoted 4 lines
KbuildsupportsKDEB_COMPRESSvariable
to change the compression method.
See line 46 ofscripts/package/builddeb.
Hi Masahiro,
thanks for the clarification and the informations.
With next RC I will add KDEB_COMPRESS=gzip to my build-script and report.
Regards,
- Sedat -
Quoted 55 lines
If you are interested,
try "make bindeb-pkg" with/withoutCONFIG_DEBUG_INFO_COMPRESSED, and compare the size of the generated debug package.As Sedat stated,
(plain data) -> compress by gzip -> compress by xz
is often less efficient than
(plain data) -> compress by xz
I hope this is clearer.
Quoted 14 lines
Fangrui, I wasn't able to easily find what version of binutils first added support. Can you please teach me how to fish?
Another question I had for Fangrui is, if the linker can compress these sections, shouldn't we just have the linker do it, not the the compiler and assembler? IIUC the debug info can contain relocations, so the linker would have to decompress these, perform relocations, then recompress these? I guess having the compiler and assembler compress the debug info as well would minimize the size of the .o files on disk.
Otherwise I should add this flag to the assembler invocation, too, in v2. Thoughts?
Quoted 8 lines
I have a patch series that enables dwarf5 support in the kernel that I'm working up to. I wanted to send this first. Both roughly reduce the debug info size by 20% each, though I haven't measured them together, yet. Requires ToT binutils because there have been many fixes from reports of mine recently.
--
Thanks,
~Nick Desaulniers--
Best Regards
Masahiro Yamada
Hi Sedat,
Quoted 181 lines · Sedat Dilek
Quoted 31 lines · Nick Desaulniers
As debug information gets larger and larger, it helps significantly save the size of vmlinux images to compress the information in the debug information sections. Note: this debug info is typically split off from the final compressed kernel image, which is why vmlinux is what's used in conjunction with GDB. Minimizing the debug info size should have no impact on boot times, or final compressed kernel image size.
All of the debug sections will have a
Cflag set.$ readelf -S <object file>$ bloaty vmlinux.gcc75.compressed.dwarf4 -- \ vmlinux.gcc75.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +18 [ = ] 0 [Unmapped] -73.3% -114Ki [ = ] 0 .debug_aranges -76.2% -2.01Mi [ = ] 0 .debug_frame -73.6% -2.89Mi [ = ] 0 .debug_str -80.7% -4.66Mi [ = ] 0 .debug_abbrev -82.9% -4.88Mi [ = ] 0 .debug_ranges -70.5% -9.04Mi [ = ] 0 .debug_line -79.3% -10.9Mi [ = ] 0 .debug_loc -39.5% -88.6Mi [ = ] 0 .debug_info -18.2% -123Mi [ = ] 0 TOTAL$ bloaty vmlinux.clang11.compressed.dwarf4 -- \ vmlinux.clang11.uncompressed.dwarf4FILE SIZE VM SIZE
Quoted 12 lines
-------------- -------------- +0.0% +23 [ = ] 0 [Unmapped] -65.6% -871 [ = ] 0 .debug_aranges -77.4% -1.84Mi [ = ] 0 .debug_frame -82.9% -2.33Mi [ = ] 0 .debug_abbrev -73.1% -2.43Mi [ = ] 0 .debug_str -84.8% -3.07Mi [ = ] 0 .debug_ranges -65.9% -8.62Mi [ = ] 0 .debug_line -86.2% -40.0Mi [ = ] 0 .debug_loc -42.0% -64.1Mi [ = ] 0 .debug_info -22.1% -122Mi [ = ] 0 TOTALHi Nick,
thanks for the patch.
I have slightly modified it to adapt to Linux v5.7-rc4 (what was your base?).
Which linker did you use and has it an impact if you switch from ld.bfd to ld.lld?
I tried a first normal run and in a 2nd one with
CONFIG_DEBUG_INFO_COMPRESSED=yboth with clang-10 and ld.lld-10.My numbers (sizes in MiB):
[ diffconfig ]
$ scripts/diffconfig /boot/config-5.7.0-rc4-1-amd64-clang /boot/config-5.7.0-rc4-2-amd64-clang BUILD_SALT "5.7.0-rc4-1-amd64-clang" -> "5.7.0-rc4-2-amd64-clang" +DEBUG_INFO_COMPRESSED y[ compiler and linker ]
$ clang-10 -v ClangBuiltLinux clang version 10.0.1 (https://github.com/llvm/llvm-project 92d5c1be9ee93850c0a8903f05f36a23ee835dc2) Target: x86_64-unknown-linux-gnu Thread model: posix InstalledDir: /home/dileks/src/llvm-toolchain/install/bin Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9 Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10 Candidate multilib: .;@m64 Candidate multilib: 32;@m32 Candidate multilib: x32;@mx32 Selected multilib: .;@m64$ ld.lld-10 -v LLD 10.0.1 (https://github.com/llvm/llvm-project 92d5c1be9ee93850c0a8903f05f36a23ee835dc2) (compatible with GNU linkers)[ sizes vmlinux ]
$ du -m 5.7.0-rc4-*/vmlinux* 409 5.7.0-rc4-1-amd64-clang/vmlinux 7 5.7.0-rc4-1-amd64-clang/vmlinux.compressed 404 5.7.0-rc4-1-amd64-clang/vmlinux.o 324 5.7.0-rc4-2-amd64-clang/vmlinux 7 5.7.0-rc4-2-amd64-clang/vmlinux.compressed 299 5.7.0-rc4-2-amd64-clang/vmlinux.o[ readelf (.debug_info as example) ]
$ readelf -S vmlinux.o [33] .debug_info PROGBITS 0000000000000000 01d6a5e8 0000000006be1ee6 0000000000000000 0 0 1$ readelf -S vmlinux.o [33] .debug_info PROGBITS 0000000000000000 01749f18 0000000002ef04d2 0000000000000000 C 0 0 1 <--- XXX: "C (compressed)" FlagKey to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info), L (link order), O (extra OS processing required), G (group), T (TLS), C (compressed), x (unknown), o (OS specific), E (exclude), l (large), p (processor specific)[ sizes linux-image debian packages ]
$ du -m 5.7.0-rc4-*/linux-image*.deb 47 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang_5.7.0~rc4-1~bullseye+dileks1_amd64.deb 424 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb 47 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang_5.7.0~rc4-2~bullseye+dileks1_amd64.deb 771 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb[ sizes linux-git dir (compilation finished ]
5.7.0-rc4-1-amd64-clang: 17963
/home/dileks/src/linux-kernel/linux
5.7.0-rc4-2-amd64-clang: 14328/home/dileks/src/linux-kernel/linux[ xz compressed linux-image-dbg packages ]
$ file linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb: Debian binary package (format 2.0), with control.tar.xz, data compression xz $ file linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb: Debian binary package (format 2.0), with control.tar.xz, data compression xz[ file-lists ]
$ dpkg --contents linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb | wc -l 4395 $ dpkg --contents linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb | wc -l 4395[ file-lists vmlinux ]
$ dpkg --contents linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb | grep vmlinux -rwxr-xr-x root/root 428588312 2020-05-04 06:15 ./usr/lib/debug/lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux lrwxrwxrwx root/root 0 2020-05-04 06:15 ./usr/lib/debug/boot/vmlinux-5.7.0-rc4-1-amd64-clang -> ../lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux lrwxrwxrwx root/root 0 2020-05-04 06:15 ./usr/lib/debug/vmlinux-5.7.0-rc4-1-amd64-clang -> lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux$ dpkg --contents linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb | grep vmlinux -rwxr-xr-x root/root 339341456 2020-05-04 12:24 ./usr/lib/debug/lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux lrwxrwxrwx root/root 0 2020-05-04 12:24 ./usr/lib/debug/boot/vmlinux-5.7.0-rc4-2-amd64-clang -> ../lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux lrwxrwxrwx root/root 0 2020-05-04 12:24 ./usr/lib/debug/vmlinux-5.7.0-rc4-2-amd64-clang -> lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux[ conclusion ]
Quoted 7 lines
As you can see there is a size-reduction in case of
vmlinux/vmlinux.o(debug) files...
...and my linux-git directory in total is smaller: 17963M vs. 14328M.But the resulting linux-image-dbg file is much fatter: 424M vs. 711M. XZ-compressing the gz/zlib-compressed vmlinux (debug) file results in a fatter linux-image-dbg package.
I also confirmed that, but this would not
be a blocker of this patch.
Users can disable CONFIG_DEBUG_INFO_COMPRESSED
if they care about the debug package size.
Signature · 2 lines
Best Regards Masahiro Yamada
Sedat Dilek
Tested-byQuoted 190 lines · Masahiro Yamada
Hi Sedat,
Quoted 185 lines · Sedat Dilek
Quoted 31 lines · Nick Desaulniers
As debug information gets larger and larger, it helps significantly save the size of vmlinux images to compress the information in the debug information sections. Note: this debug info is typically split off from the final compressed kernel image, which is why vmlinux is what's used in conjunction with GDB. Minimizing the debug info size should have no impact on boot times, or final compressed kernel image size.
All of the debug sections will have a
Cflag set.$ readelf -S <object file>$ bloaty vmlinux.gcc75.compressed.dwarf4 -- \ vmlinux.gcc75.uncompressed.dwarf4FILE SIZE VM SIZE
-------------- -------------- +0.0% +18 [ = ] 0 [Unmapped] -73.3% -114Ki [ = ] 0 .debug_aranges -76.2% -2.01Mi [ = ] 0 .debug_frame -73.6% -2.89Mi [ = ] 0 .debug_str -80.7% -4.66Mi [ = ] 0 .debug_abbrev -82.9% -4.88Mi [ = ] 0 .debug_ranges -70.5% -9.04Mi [ = ] 0 .debug_line -79.3% -10.9Mi [ = ] 0 .debug_loc -39.5% -88.6Mi [ = ] 0 .debug_info -18.2% -123Mi [ = ] 0 TOTAL$ bloaty vmlinux.clang11.compressed.dwarf4 -- \ vmlinux.clang11.uncompressed.dwarf4FILE SIZE VM SIZE
Quoted 12 lines
-------------- -------------- +0.0% +23 [ = ] 0 [Unmapped] -65.6% -871 [ = ] 0 .debug_aranges -77.4% -1.84Mi [ = ] 0 .debug_frame -82.9% -2.33Mi [ = ] 0 .debug_abbrev -73.1% -2.43Mi [ = ] 0 .debug_str -84.8% -3.07Mi [ = ] 0 .debug_ranges -65.9% -8.62Mi [ = ] 0 .debug_line -86.2% -40.0Mi [ = ] 0 .debug_loc -42.0% -64.1Mi [ = ] 0 .debug_info -22.1% -122Mi [ = ] 0 TOTALHi Nick,
thanks for the patch.
I have slightly modified it to adapt to Linux v5.7-rc4 (what was your base?).
Which linker did you use and has it an impact if you switch from ld.bfd to ld.lld?
I tried a first normal run and in a 2nd one with
CONFIG_DEBUG_INFO_COMPRESSED=yboth with clang-10 and ld.lld-10.My numbers (sizes in MiB):
[ diffconfig ]
$ scripts/diffconfig /boot/config-5.7.0-rc4-1-amd64-clang /boot/config-5.7.0-rc4-2-amd64-clang BUILD_SALT "5.7.0-rc4-1-amd64-clang" -> "5.7.0-rc4-2-amd64-clang" +DEBUG_INFO_COMPRESSED y[ compiler and linker ]
$ clang-10 -v ClangBuiltLinux clang version 10.0.1 (https://github.com/llvm/llvm-project 92d5c1be9ee93850c0a8903f05f36a23ee835dc2) Target: x86_64-unknown-linux-gnu Thread model: posix InstalledDir: /home/dileks/src/llvm-toolchain/install/bin Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9 Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10 Candidate multilib: .;@m64 Candidate multilib: 32;@m32 Candidate multilib: x32;@mx32 Selected multilib: .;@m64$ ld.lld-10 -v LLD 10.0.1 (https://github.com/llvm/llvm-project 92d5c1be9ee93850c0a8903f05f36a23ee835dc2) (compatible with GNU linkers)[ sizes vmlinux ]
$ du -m 5.7.0-rc4-*/vmlinux* 409 5.7.0-rc4-1-amd64-clang/vmlinux 7 5.7.0-rc4-1-amd64-clang/vmlinux.compressed 404 5.7.0-rc4-1-amd64-clang/vmlinux.o 324 5.7.0-rc4-2-amd64-clang/vmlinux 7 5.7.0-rc4-2-amd64-clang/vmlinux.compressed 299 5.7.0-rc4-2-amd64-clang/vmlinux.o[ readelf (.debug_info as example) ]
$ readelf -S vmlinux.o [33] .debug_info PROGBITS 0000000000000000 01d6a5e8 0000000006be1ee6 0000000000000000 0 0 1$ readelf -S vmlinux.o [33] .debug_info PROGBITS 0000000000000000 01749f18 0000000002ef04d2 0000000000000000 C 0 0 1 <--- XXX: "C (compressed)" FlagKey to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info), L (link order), O (extra OS processing required), G (group), T (TLS), C (compressed), x (unknown), o (OS specific), E (exclude), l (large), p (processor specific)[ sizes linux-image debian packages ]
$ du -m 5.7.0-rc4-*/linux-image*.deb 47 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang_5.7.0~rc4-1~bullseye+dileks1_amd64.deb 424 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb 47 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang_5.7.0~rc4-2~bullseye+dileks1_amd64.deb 771 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb[ sizes linux-git dir (compilation finished ]
5.7.0-rc4-1-amd64-clang: 17963
/home/dileks/src/linux-kernel/linux
5.7.0-rc4-2-amd64-clang: 14328/home/dileks/src/linux-kernel/linux[ xz compressed linux-image-dbg packages ]
$ file linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb: Debian binary package (format 2.0), with control.tar.xz, data compression xz $ file linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb: Debian binary package (format 2.0), with control.tar.xz, data compression xz[ file-lists ]
$ dpkg --contents linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb | wc -l 4395 $ dpkg --contents linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb | wc -l 4395[ file-lists vmlinux ]
$ dpkg --contents linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb | grep vmlinux -rwxr-xr-x root/root 428588312 2020-05-04 06:15 ./usr/lib/debug/lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux lrwxrwxrwx root/root 0 2020-05-04 06:15 ./usr/lib/debug/boot/vmlinux-5.7.0-rc4-1-amd64-clang -> ../lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux lrwxrwxrwx root/root 0 2020-05-04 06:15 ./usr/lib/debug/vmlinux-5.7.0-rc4-1-amd64-clang -> lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux$ dpkg --contents linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb | grep vmlinux -rwxr-xr-x root/root 339341456 2020-05-04 12:24 ./usr/lib/debug/lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux lrwxrwxrwx root/root 0 2020-05-04 12:24 ./usr/lib/debug/boot/vmlinux-5.7.0-rc4-2-amd64-clang -> ../lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux lrwxrwxrwx root/root 0 2020-05-04 12:24 ./usr/lib/debug/vmlinux-5.7.0-rc4-2-amd64-clang -> lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux[ conclusion ]
As you can see there is a size-reduction in case of
vmlinux/vmlinux.o(debug) files...
...and my linux-git directory in total is smaller: 17963M vs. 14328M.
Quoted 8 lines
Quoted 3 lines
But the resulting linux-image-dbg file is much fatter: 424M vs. 711M. XZ-compressing the gz/zlib-compressed vmlinux (debug) file results in a fatter linux-image-dbg package.
I also confirmed that, but this would not
be a blocker of this patch.
Hi Masahiro,
No, it is not a blocker.
I have this patch now in my Linux v5.7-rc5 series.
I see a lot of more benefits concerning disc-usage - in my linux-git and a reduced vmlinux file when I wanted to test with QEMU.
Feel free to add:
- Sedat -
Quoted 15 lines
Users can disable
CONFIG_DEBUG_INFO_COMPRESSED
if they care about the debug package size.--
Best Regards
Masahiro Yamada--
You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAK7LNAR%2Bpm-_nd5%3DB2OeLpimW42FXxm8TQUMru9DR_asT3qYnA%40mail.gmail.com.