Skip to contentSkip to conversation

kbuild: add an elfnote for whether vmlinux is built with lto

PATCH In review Yonghong Song 5 participants 16 messages Apr 1, 2021 – Apr 11, 2021

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v4 1/1 kbuild: add an elfnote for whether vmlinux is built with lto

Currently, clang LTO built vmlinux won't work with pahole. LTO introduced cross-cu dwarf tag references and broke current pahole model which handles one cu as a time. The solution is to merge all cu's as one pahole cu as in [1]. We would like to do this merging only if cross-cu dwarf references happens. The LTO build mode is a pretty good indication for that.

In earlier version of this patch ([2]), clang flag -grecord-gcc-switches is proposed to add to compilation flags so pahole could detect "-flto" and then merging cu's. This will increate the binary size of 1% without LTO though.

Arnaldo suggested to use a note to indicate the vmlinux is built with LTO. Such a cheap way to get whether the vmlinux is built with LTO or not helps pahole but is also useful for tracing as LTO may inline/delete/demote global functions, promote static functions, etc.

So this patch added an elfnote with a new type LINUX_ELFNOTE_LTO_INFO. The owner of the note is "Linux".

With gcc 8.4.1 and clang trunk, without LTO, I got

  $ readelf -n vmlinux
  Displaying notes found in: .notes
    Owner                Data size        Description
  ...
    Linux                0x00000004       func
     description data: 00 00 00 00
  ...
With "readelf -x ".notes" vmlinux", I can verify the above "func"
with type code 0x101.

With clang thin-LTO, I got the same as above except the following:

     description data: 01 00 00 00

which indicates the vmlinux is built with LTO.

[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/

---
 include/linux/elfnote-lto.h | 14 ++++++++++++++
 init/version.c              |  2 ++
 scripts/mod/modpost.c       |  2 ++
 3 files changed, 18 insertions(+)
 create mode 100644 include/linux/elfnote-lto.h

Changelogs:
v3 -> v4:
. put new lto note in its own header file similar to
build-salt.h. (Nick)
v2 -> v3:
. abandoned the approach of adding -grecord-gcc-switches,
instead create a note to indicate whether it is a lto build or not. The note definition is in compiler.h. (Arnaldo)
v1 -> v2:
. limited to add -grecord-gcc-switches for LTO_CLANG
instead of all clang build

include/linux/elfnote-lto.h new file +14 -0
new file mode 100644
index 000000000000..d4635a3ecc4f
@@ -0,0 +1,14 @@
+#ifndef __ELFNOTE_LTO_H
+#define __ELFNOTE_LTO_H
+
+#include <linux/elfnote.h>
+
+#define LINUX_ELFNOTE_LTO_INFO	0x101
+
+#ifdef CONFIG_LTO
+#define BUILD_LTO_INFO	ELFNOTE32("Linux", LINUX_ELFNOTE_LTO_INFO, 1)
+#else
+#define BUILD_LTO_INFO	ELFNOTE32("Linux", LINUX_ELFNOTE_LTO_INFO, 0)
+#endif
+
+#endif /* __ELFNOTE_LTO_H */
init/version.c +2 -0
index 92afc782b043..1a356f5493e8 100644
@@ -9,6 +9,7 @@
 
 #include <generated/compile.h>
 #include <linux/build-salt.h>
+#include <linux/elfnote-lto.h>
 #include <linux/export.h>
 #include <linux/uts.h>
 #include <linux/utsname.h>
@@ -45,3 +46,4 @@ const char linux_proc_banner[] =
 	" (" LINUX_COMPILER ") %s\n";
 
 BUILD_SALT;
+BUILD_LTO_INFO;
scripts/mod/modpost.c +2 -0
index 24725e50c7b4..98fb2bb024db 100644
@@ -2191,10 +2191,12 @@ static void add_header(struct buffer *b, struct module *mod)
 	 */
 	buf_printf(b, "#define INCLUDE_VERMAGIC\n");
 	buf_printf(b, "#include <linux/build-salt.h>\n");
+	buf_printf(b, "#include <linux/elfnote-lto.h>\n");
 	buf_printf(b, "#include <linux/vermagic.h>\n");
 	buf_printf(b, "#include <linux/compiler.h>\n");
 	buf_printf(b, "\n");
 	buf_printf(b, "BUILD_SALT;\n");
+	buf_printf(b, "BUILD_LTO_INFO;\n");
 	buf_printf(b, "\n");
 	buf_printf(b, "MODULE_INFO(vermagic, VERMAGIC_STRING);\n");
 	buf_printf(b, "MODULE_INFO(name, KBUILD_MODNAME);\n");
Signature · 1 line
2.30.2
Reviewed-by on v4 1/1
#1 lore raw
Quoted 35 lines · Yonghong Song

Currently, clang LTO built vmlinux won't work with pahole. LTO introduced cross-cu dwarf tag references and broke current pahole model which handles one cu as a time. The solution is to merge all cu's as one pahole cu as in [1]. We would like to do this merging only if cross-cu dwarf references happens. The LTO build mode is a pretty good indication for that.

In earlier version of this patch ([2]), clang flag -grecord-gcc-switches is proposed to add to compilation flags so pahole could detect "-flto" and then merging cu's. This will increate the binary size of 1% without LTO though.

Arnaldo suggested to use a note to indicate the vmlinux is built with LTO. Such a cheap way to get whether the vmlinux is built with LTO or not helps pahole but is also useful for tracing as LTO may inline/delete/demote global functions, promote static functions, etc.

So this patch added an elfnote with a new type LINUX_ELFNOTE_LTO_INFO. The owner of the note is "Linux".

With gcc 8.4.1 and clang trunk, without LTO, I got

  $ readelf -n vmlinux
  Displaying notes found in: .notes
    Owner                Data size        Description
  ...
    Linux                0x00000004       func
     description data: 00 00 00 00
  ...
With "readelf -x ".notes" vmlinux", I can verify the above "func"
with type code 0x101.

With clang thin-LTO, I got the same as above except the following:

Quoted 8 lines
     description data: 01 00 00 00

which indicates the vmlinux is built with LTO.

[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/

LGTM thanks Yonghong!

Quoted 76 lines
---
 include/linux/elfnote-lto.h | 14 ++++++++++++++
 init/version.c              |  2 ++
 scripts/mod/modpost.c       |  2 ++
 3 files changed, 18 insertions(+)
 create mode 100644 include/linux/elfnote-lto.h

Changelogs:
v3 -> v4:
. put new lto note in its own header file similar to
build-salt.h. (Nick)
v2 -> v3:
. abandoned the approach of adding -grecord-gcc-switches,
instead create a note to indicate whether it is a lto build or not. The note definition is in compiler.h. (Arnaldo)
v1 -> v2:
. limited to add -grecord-gcc-switches for LTO_CLANG
instead of all clang build

include/linux/elfnote-lto.h new file +14 -0
new file mode 100644
index 000000000000..d4635a3ecc4f
@@ -0,0 +1,14 @@
+#ifndef __ELFNOTE_LTO_H
+#define __ELFNOTE_LTO_H
+
+#include <linux/elfnote.h>
+
+#define LINUX_ELFNOTE_LTO_INFO 0x101
+
+#ifdef CONFIG_LTO
+#define BUILD_LTO_INFO ELFNOTE32("Linux", LINUX_ELFNOTE_LTO_INFO, 1)
+#else
+#define BUILD_LTO_INFO ELFNOTE32("Linux", LINUX_ELFNOTE_LTO_INFO, 0)
+#endif
+
+#endif /* __ELFNOTE_LTO_H */
init/version.c +2 -0
index 92afc782b043..1a356f5493e8 100644
@@ -9,6 +9,7 @@
 
 #include <generated/compile.h>
 #include <linux/build-salt.h>
+#include <linux/elfnote-lto.h>
 #include <linux/export.h>
 #include <linux/uts.h>
 #include <linux/utsname.h>
@@ -45,3 +46,4 @@ const char linux_proc_banner[] =
        " (" LINUX_COMPILER ") %s\n";
 
 BUILD_SALT;
+BUILD_LTO_INFO;
scripts/mod/modpost.c +2 -1
index 24725e50c7b4..98fb2bb024db 100644
@@ -2191,10 +2191,12 @@ static void add_header(struct buffer *b, struct module *mod)
         */
        buf_printf(b, "#define INCLUDE_VERMAGIC\n");
        buf_printf(b, "#include <linux/build-salt.h>\n");
+       buf_printf(b, "#include <linux/elfnote-lto.h>\n");
        buf_printf(b, "#include <linux/vermagic.h>\n");
        buf_printf(b, "#include <linux/compiler.h>\n");
        buf_printf(b, "\n");
        buf_printf(b, "BUILD_SALT;\n");
+       buf_printf(b, "BUILD_LTO_INFO;\n");
        buf_printf(b, "\n");
        buf_printf(b, "MODULE_INFO(vermagic, VERMAGIC_STRING);\n");
        buf_printf(b, "MODULE_INFO(name, KBUILD_MODNAME);\n");
--

2.30.2

Signature · 2 lines
Thanks,
~Nick Desaulniers
#2 lore raw
Quoted 40 lines · Nick Desaulniers' via Clang Built Linux
Quoted 38 lines · Yonghong Song

Currently, clang LTO built vmlinux won't work with pahole. LTO introduced cross-cu dwarf tag references and broke current pahole model which handles one cu as a time. The solution is to merge all cu's as one pahole cu as in [1]. We would like to do this merging only if cross-cu dwarf references happens. The LTO build mode is a pretty good indication for that.

In earlier version of this patch ([2]), clang flag -grecord-gcc-switches is proposed to add to compilation flags so pahole could detect "-flto" and then merging cu's. This will increate the binary size of 1% without LTO though.

Arnaldo suggested to use a note to indicate the vmlinux is built with LTO. Such a cheap way to get whether the vmlinux is built with LTO or not helps pahole but is also useful for tracing as LTO may inline/delete/demote global functions, promote static functions, etc.

So this patch added an elfnote with a new type LINUX_ELFNOTE_LTO_INFO. The owner of the note is "Linux".

With gcc 8.4.1 and clang trunk, without LTO, I got

  $ readelf -n vmlinux
  Displaying notes found in: .notes
    Owner                Data size        Description
  ...
    Linux                0x00000004       func
     description data: 00 00 00 00
  ...
With "readelf -x ".notes" vmlinux", I can verify the above "func"
with type code 0x101.

With clang thin-LTO, I got the same as above except the following:

     description data: 01 00 00 00

which indicates the vmlinux is built with LTO.

Quoted 20 lines
Quoted 5 lines

[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/

LGTM thanks Yonghong!

Quoted 11 lines
---
 include/linux/elfnote-lto.h | 14 ++++++++++++++
 init/version.c              |  2 ++
 scripts/mod/modpost.c       |  2 ++
 3 files changed, 18 insertions(+)
 create mode 100644 include/linux/elfnote-lto.h

Changelogs:
v3 -> v4:
. put new lto note in its own header file similar to
build-salt.h. (Nick)

That is a bit smarter (and smaller) than v3.
Queued up and building a new clang-lto kernel...
Will report later.

- Sedat -

Quoted 75 lines
Quoted 63 lines

v2 -> v3:
. abandoned the approach of adding -grecord-gcc-switches,
instead create a note to indicate whether it is a lto build or not. The note definition is in compiler.h. (Arnaldo)
v1 -> v2:
. limited to add -grecord-gcc-switches for LTO_CLANG
instead of all clang build

include/linux/elfnote-lto.h new file +14 -0
new file mode 100644
index 000000000000..d4635a3ecc4f
@@ -0,0 +1,14 @@
+#ifndef __ELFNOTE_LTO_H
+#define __ELFNOTE_LTO_H
+
+#include <linux/elfnote.h>
+
+#define LINUX_ELFNOTE_LTO_INFO 0x101
+
+#ifdef CONFIG_LTO
+#define BUILD_LTO_INFO ELFNOTE32("Linux", LINUX_ELFNOTE_LTO_INFO, 1)
+#else
+#define BUILD_LTO_INFO ELFNOTE32("Linux", LINUX_ELFNOTE_LTO_INFO, 0)
+#endif
+
+#endif /* __ELFNOTE_LTO_H */
init/version.c +2 -0
index 92afc782b043..1a356f5493e8 100644
@@ -9,6 +9,7 @@
 
 #include <generated/compile.h>
 #include <linux/build-salt.h>
+#include <linux/elfnote-lto.h>
 #include <linux/export.h>
 #include <linux/uts.h>
 #include <linux/utsname.h>
@@ -45,3 +46,4 @@ const char linux_proc_banner[] =
        " (" LINUX_COMPILER ") %s\n";
 
 BUILD_SALT;
+BUILD_LTO_INFO;
scripts/mod/modpost.c +2 -1
index 24725e50c7b4..98fb2bb024db 100644
@@ -2191,10 +2191,12 @@ static void add_header(struct buffer *b, struct module *mod)
         */
        buf_printf(b, "#define INCLUDE_VERMAGIC\n");
        buf_printf(b, "#include <linux/build-salt.h>\n");
+       buf_printf(b, "#include <linux/elfnote-lto.h>\n");
        buf_printf(b, "#include <linux/vermagic.h>\n");
        buf_printf(b, "#include <linux/compiler.h>\n");
        buf_printf(b, "\n");
        buf_printf(b, "BUILD_SALT;\n");
+       buf_printf(b, "BUILD_LTO_INFO;\n");
        buf_printf(b, "\n");
        buf_printf(b, "MODULE_INFO(vermagic, VERMAGIC_STRING);\n");
        buf_printf(b, "MODULE_INFO(name, KBUILD_MODNAME);\n");
--
Quoted 2 lines

2.30.2

--
Thanks,
~Nick Desaulniers

--
You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAKwvOdmX8d3XTzJFk5rN_PnOQYJ8bXMrh8DrhzqN%3DUBNdQiO3g%40mail.gmail.com.

#3 lore raw
Quoted 59 lines · Sedat Dilek
Quoted 56 lines · Nick Desaulniers' via Clang Built Linux
Quoted 35 lines · Yonghong Song

Currently, clang LTO built vmlinux won't work with pahole. LTO introduced cross-cu dwarf tag references and broke current pahole model which handles one cu as a time. The solution is to merge all cu's as one pahole cu as in [1]. We would like to do this merging only if cross-cu dwarf references happens. The LTO build mode is a pretty good indication for that.

In earlier version of this patch ([2]), clang flag -grecord-gcc-switches is proposed to add to compilation flags so pahole could detect "-flto" and then merging cu's. This will increate the binary size of 1% without LTO though.

Arnaldo suggested to use a note to indicate the vmlinux is built with LTO. Such a cheap way to get whether the vmlinux is built with LTO or not helps pahole but is also useful for tracing as LTO may inline/delete/demote global functions, promote static functions, etc.

So this patch added an elfnote with a new type LINUX_ELFNOTE_LTO_INFO. The owner of the note is "Linux".

With gcc 8.4.1 and clang trunk, without LTO, I got

  $ readelf -n vmlinux
  Displaying notes found in: .notes
    Owner                Data size        Description
  ...
    Linux                0x00000004       func
     description data: 00 00 00 00
  ...
With "readelf -x ".notes" vmlinux", I can verify the above "func"
with type code 0x101.

With clang thin-LTO, I got the same as above except the following:

Quoted 8 lines
     description data: 01 00 00 00

which indicates the vmlinux is built with LTO.

[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/

LGTM thanks Yonghong!

Quoted 7 lines
---
 include/linux/elfnote-lto.h | 14 ++++++++++++++
 init/version.c              |  2 ++
 scripts/mod/modpost.c       |  2 ++
 3 files changed, 18 insertions(+)
 create mode 100644 include/linux/elfnote-lto.h
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines

Changelogs:
v3 -> v4:
. put new lto note in its own header file similar to
build-salt.h. (Nick)

That is a bit smarter (and smaller) than v3.
Queued up and building a new clang-lto kernel...
Will report later.

link="https://lore.kernel.org/bpf/[email protected]/T/#t" b4 -d am $link

Needs this fix for the pahole side?

$ git diff
dwarf_loader.c +0 -0
index 026d13789ff9..244816042c88 100644
@@ -2501,8 +2501,8 @@ static int cus__load_debug_types(struct cus

*cus, struct conf_load *conf,

       return 0;
}

-/* Match the define in linux:include/linux/elfnote.h */
-#define LINUX_ELFNOTE_BUILD_LTO 0x101
+/* Match the define in linux:include/linux/elfnote-lto.h */
+#define LINUX_ELFNOTE_LTO_INFO 0x101

static bool cus__merging_cu(Dwarf *dw, Elf *elf)
{
@@ -2520,7 +2520,7 @@ static bool cus__merging_cu(Dwarf *dw, Elf *elf)

                       size_t name_off, desc_off, offset = 0;
                       GElf_Nhdr hdr;
                       while ((offset = gelf_getnote(data, offset,
&hdr, &name_off, &desc_off)) != 0) {
-                               if (hdr.n_type != LINUX_ELFNOTE_BUILD_LTO)
+                               if (hdr.n_type != LINUX_ELFNOTE_LTO_INFO)
                                       continue;

                               /* owner is Linux */

- Sedat -

Quoted 76 lines
Quoted 76 lines
Quoted 64 lines

v2 -> v3:
. abandoned the approach of adding -grecord-gcc-switches,
instead create a note to indicate whether it is a lto build or not. The note definition is in compiler.h. (Arnaldo)
v1 -> v2:
. limited to add -grecord-gcc-switches for LTO_CLANG
instead of all clang build

include/linux/elfnote-lto.h new file +14 -0
new file mode 100644
index 000000000000..d4635a3ecc4f
@@ -0,0 +1,14 @@
+#ifndef __ELFNOTE_LTO_H
+#define __ELFNOTE_LTO_H
+
+#include <linux/elfnote.h>
+
+#define LINUX_ELFNOTE_LTO_INFO 0x101
+
+#ifdef CONFIG_LTO
+#define BUILD_LTO_INFO ELFNOTE32("Linux", LINUX_ELFNOTE_LTO_INFO, 1)
+#else
+#define BUILD_LTO_INFO ELFNOTE32("Linux", LINUX_ELFNOTE_LTO_INFO, 0)
+#endif
+
+#endif /* __ELFNOTE_LTO_H */
init/version.c +2 -0
index 92afc782b043..1a356f5493e8 100644
@@ -9,6 +9,7 @@
 
 #include <generated/compile.h>
 #include <linux/build-salt.h>
+#include <linux/elfnote-lto.h>
 #include <linux/export.h>
 #include <linux/uts.h>
 #include <linux/utsname.h>
@@ -45,3 +46,4 @@ const char linux_proc_banner[] =
        " (" LINUX_COMPILER ") %s\n";
 
 BUILD_SALT;
+BUILD_LTO_INFO;
scripts/mod/modpost.c +2 -1
index 24725e50c7b4..98fb2bb024db 100644
@@ -2191,10 +2191,12 @@ static void add_header(struct buffer *b, struct module *mod)
         */
        buf_printf(b, "#define INCLUDE_VERMAGIC\n");
        buf_printf(b, "#include <linux/build-salt.h>\n");
+       buf_printf(b, "#include <linux/elfnote-lto.h>\n");
        buf_printf(b, "#include <linux/vermagic.h>\n");
        buf_printf(b, "#include <linux/compiler.h>\n");
        buf_printf(b, "\n");
        buf_printf(b, "BUILD_SALT;\n");
+       buf_printf(b, "BUILD_LTO_INFO;\n");
        buf_printf(b, "\n");
        buf_printf(b, "MODULE_INFO(vermagic, VERMAGIC_STRING);\n");
        buf_printf(b, "MODULE_INFO(name, KBUILD_MODNAME);\n");
--
Quoted 2 lines

2.30.2

--
Thanks,
~Nick Desaulniers

--
You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAKwvOdmX8d3XTzJFk5rN_PnOQYJ8bXMrh8DrhzqN%3DUBNdQiO3g%40mail.gmail.com.

#4 lore raw
Quoted 102 lines · Sedat Dilek
Quoted 59 lines · Sedat Dilek
Quoted 56 lines · Nick Desaulniers' via Clang Built Linux
Quoted 35 lines · Yonghong Song

Currently, clang LTO built vmlinux won't work with pahole. LTO introduced cross-cu dwarf tag references and broke current pahole model which handles one cu as a time. The solution is to merge all cu's as one pahole cu as in [1]. We would like to do this merging only if cross-cu dwarf references happens. The LTO build mode is a pretty good indication for that.

In earlier version of this patch ([2]), clang flag -grecord-gcc-switches is proposed to add to compilation flags so pahole could detect "-flto" and then merging cu's. This will increate the binary size of 1% without LTO though.

Arnaldo suggested to use a note to indicate the vmlinux is built with LTO. Such a cheap way to get whether the vmlinux is built with LTO or not helps pahole but is also useful for tracing as LTO may inline/delete/demote global functions, promote static functions, etc.

So this patch added an elfnote with a new type LINUX_ELFNOTE_LTO_INFO. The owner of the note is "Linux".

With gcc 8.4.1 and clang trunk, without LTO, I got

  $ readelf -n vmlinux
  Displaying notes found in: .notes
    Owner                Data size        Description
  ...
    Linux                0x00000004       func
     description data: 00 00 00 00
  ...
With "readelf -x ".notes" vmlinux", I can verify the above "func"
with type code 0x101.

With clang thin-LTO, I got the same as above except the following:

Quoted 8 lines
     description data: 01 00 00 00

which indicates the vmlinux is built with LTO.

[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/

LGTM thanks Yonghong!

Quoted 7 lines
---
 include/linux/elfnote-lto.h | 14 ++++++++++++++
 init/version.c              |  2 ++
 scripts/mod/modpost.c       |  2 ++
 3 files changed, 18 insertions(+)
 create mode 100644 include/linux/elfnote-lto.h
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines

Changelogs:
v3 -> v4:
. put new lto note in its own header file similar to
build-salt.h. (Nick)

That is a bit smarter (and smaller) than v3.
Queued up and building a new clang-lto kernel...
Will report later.

link="https://lore.kernel.org/bpf/[email protected]/T/#t" b4 -d am $link

Needs this fix for the pahole side?

$ git diff
dwarf_loader.c +2 -2
index 026d13789ff9..244816042c88 100644
@@ -2501,8 +2501,8 @@ static int cus__load_debug_types(struct cus
 *cus, struct conf_load *conf,
       return 0;
 }
 
-/* Match the define in linux:include/linux/elfnote.h */
-#define LINUX_ELFNOTE_BUILD_LTO                0x101
+/* Match the define in linux:include/linux/elfnote-lto.h */
+#define LINUX_ELFNOTE_LTO_INFO         0x101
 
 static bool cus__merging_cu(Dwarf *dw, Elf *elf)

{

dwarf_loader.c +1 -1
@@ -2520,7 +2520,7 @@ static bool cus__merging_cu(Dwarf *dw, Elf *elf)
                       size_t name_off, desc_off, offset = 0;
                       GElf_Nhdr hdr;
                       while ((offset = gelf_getnote(data, offset,
 &hdr, &name_off, &desc_off)) != 0) {
-                               if (hdr.n_type != LINUX_ELFNOTE_BUILD_LTO)
+                               if (hdr.n_type != LINUX_ELFNOTE_LTO_INFO)
                                       continue;
 
Quoted 2 lines
                               /* owner is Linux */

I applied above diff against [1] which includes v3:

dwarf_loader: Handle subprogram ret type with abstract_origin properly
dwarf_loader: Check .notes section for LTO build info
dwarf_loader: Check .debug_abbrev for cross-CU references

- Sedat -

[1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/log/?h=tmp.master

Quoted 77 lines
Quoted 77 lines
Quoted 77 lines
Quoted 65 lines

v2 -> v3:
. abandoned the approach of adding -grecord-gcc-switches,
instead create a note to indicate whether it is a lto build or not. The note definition is in compiler.h. (Arnaldo)
v1 -> v2:
. limited to add -grecord-gcc-switches for LTO_CLANG
instead of all clang build

include/linux/elfnote-lto.h new file +14 -0
new file mode 100644
index 000000000000..d4635a3ecc4f
@@ -0,0 +1,14 @@
+#ifndef __ELFNOTE_LTO_H
+#define __ELFNOTE_LTO_H
+
+#include <linux/elfnote.h>
+
+#define LINUX_ELFNOTE_LTO_INFO 0x101
+
+#ifdef CONFIG_LTO
+#define BUILD_LTO_INFO ELFNOTE32("Linux", LINUX_ELFNOTE_LTO_INFO, 1)
+#else
+#define BUILD_LTO_INFO ELFNOTE32("Linux", LINUX_ELFNOTE_LTO_INFO, 0)
+#endif
+
+#endif /* __ELFNOTE_LTO_H */
init/version.c +2 -0
index 92afc782b043..1a356f5493e8 100644
@@ -9,6 +9,7 @@
 
 #include <generated/compile.h>
 #include <linux/build-salt.h>
+#include <linux/elfnote-lto.h>
 #include <linux/export.h>
 #include <linux/uts.h>
 #include <linux/utsname.h>
@@ -45,3 +46,4 @@ const char linux_proc_banner[] =
        " (" LINUX_COMPILER ") %s\n";
 
 BUILD_SALT;
+BUILD_LTO_INFO;
scripts/mod/modpost.c +2 -1
index 24725e50c7b4..98fb2bb024db 100644
@@ -2191,10 +2191,12 @@ static void add_header(struct buffer *b, struct module *mod)
         */
        buf_printf(b, "#define INCLUDE_VERMAGIC\n");
        buf_printf(b, "#include <linux/build-salt.h>\n");
+       buf_printf(b, "#include <linux/elfnote-lto.h>\n");
        buf_printf(b, "#include <linux/vermagic.h>\n");
        buf_printf(b, "#include <linux/compiler.h>\n");
        buf_printf(b, "\n");
        buf_printf(b, "BUILD_SALT;\n");
+       buf_printf(b, "BUILD_LTO_INFO;\n");
        buf_printf(b, "\n");
        buf_printf(b, "MODULE_INFO(vermagic, VERMAGIC_STRING);\n");
        buf_printf(b, "MODULE_INFO(name, KBUILD_MODNAME);\n");
--
Quoted 2 lines

2.30.2

--
Thanks,
~Nick Desaulniers

--
You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAKwvOdmX8d3XTzJFk5rN_PnOQYJ8bXMrh8DrhzqN%3DUBNdQiO3g%40mail.gmail.com.

Tested-by
#5 lore raw
In reply to Nick Desaulniers
Quoted 40 lines · Nick Desaulniers' via Clang Built Linux
Quoted 38 lines · Yonghong Song

Currently, clang LTO built vmlinux won't work with pahole. LTO introduced cross-cu dwarf tag references and broke current pahole model which handles one cu as a time. The solution is to merge all cu's as one pahole cu as in [1]. We would like to do this merging only if cross-cu dwarf references happens. The LTO build mode is a pretty good indication for that.

In earlier version of this patch ([2]), clang flag -grecord-gcc-switches is proposed to add to compilation flags so pahole could detect "-flto" and then merging cu's. This will increate the binary size of 1% without LTO though.

Arnaldo suggested to use a note to indicate the vmlinux is built with LTO. Such a cheap way to get whether the vmlinux is built with LTO or not helps pahole but is also useful for tracing as LTO may inline/delete/demote global functions, promote static functions, etc.

So this patch added an elfnote with a new type LINUX_ELFNOTE_LTO_INFO. The owner of the note is "Linux".

With gcc 8.4.1 and clang trunk, without LTO, I got

  $ readelf -n vmlinux
  Displaying notes found in: .notes
    Owner                Data size        Description
  ...
    Linux                0x00000004       func
     description data: 00 00 00 00
  ...
With "readelf -x ".notes" vmlinux", I can verify the above "func"
with type code 0x101.

With clang thin-LTO, I got the same as above except the following:

     description data: 01 00 00 00

which indicates the vmlinux is built with LTO.

Quoted 9 lines
Quoted 5 lines

[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/

LGTM thanks Yonghong!

Thanks for the patch.

Feel free to add:

As a note for the pahole side:
Recent patches require an adaptation of the define and its comment.

1. LINUX_ELFNOTE_BUILD_LTO -> LINUX_ELFNOTE_LTO_INFO
2. include/linux/elfnote.h -> include/linux/elfnote-lto.h

- Sedat -

Quoted 86 lines
Quoted 74 lines
---
 include/linux/elfnote-lto.h | 14 ++++++++++++++
 init/version.c              |  2 ++
 scripts/mod/modpost.c       |  2 ++
 3 files changed, 18 insertions(+)
 create mode 100644 include/linux/elfnote-lto.h

Changelogs:
v3 -> v4:
. put new lto note in its own header file similar to
build-salt.h. (Nick)
v2 -> v3:
. abandoned the approach of adding -grecord-gcc-switches,
instead create a note to indicate whether it is a lto build or not. The note definition is in compiler.h. (Arnaldo)
v1 -> v2:
. limited to add -grecord-gcc-switches for LTO_CLANG
instead of all clang build

include/linux/elfnote-lto.h new file +14 -0
new file mode 100644
index 000000000000..d4635a3ecc4f
@@ -0,0 +1,14 @@
+#ifndef __ELFNOTE_LTO_H
+#define __ELFNOTE_LTO_H
+
+#include <linux/elfnote.h>
+
+#define LINUX_ELFNOTE_LTO_INFO 0x101
+
+#ifdef CONFIG_LTO
+#define BUILD_LTO_INFO ELFNOTE32("Linux", LINUX_ELFNOTE_LTO_INFO, 1)
+#else
+#define BUILD_LTO_INFO ELFNOTE32("Linux", LINUX_ELFNOTE_LTO_INFO, 0)
+#endif
+
+#endif /* __ELFNOTE_LTO_H */
init/version.c +2 -0
index 92afc782b043..1a356f5493e8 100644
@@ -9,6 +9,7 @@
 
 #include <generated/compile.h>
 #include <linux/build-salt.h>
+#include <linux/elfnote-lto.h>
 #include <linux/export.h>
 #include <linux/uts.h>
 #include <linux/utsname.h>
@@ -45,3 +46,4 @@ const char linux_proc_banner[] =
        " (" LINUX_COMPILER ") %s\n";
 
 BUILD_SALT;
+BUILD_LTO_INFO;
scripts/mod/modpost.c +2 -1
index 24725e50c7b4..98fb2bb024db 100644
@@ -2191,10 +2191,12 @@ static void add_header(struct buffer *b, struct module *mod)
         */
        buf_printf(b, "#define INCLUDE_VERMAGIC\n");
        buf_printf(b, "#include <linux/build-salt.h>\n");
+       buf_printf(b, "#include <linux/elfnote-lto.h>\n");
        buf_printf(b, "#include <linux/vermagic.h>\n");
        buf_printf(b, "#include <linux/compiler.h>\n");
        buf_printf(b, "\n");
        buf_printf(b, "BUILD_SALT;\n");
+       buf_printf(b, "BUILD_LTO_INFO;\n");
        buf_printf(b, "\n");
        buf_printf(b, "MODULE_INFO(vermagic, VERMAGIC_STRING);\n");
        buf_printf(b, "MODULE_INFO(name, KBUILD_MODNAME);\n");
--
Quoted 2 lines

2.30.2

--
Thanks,
~Nick Desaulniers

--
You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAKwvOdmX8d3XTzJFk5rN_PnOQYJ8bXMrh8DrhzqN%3DUBNdQiO3g%40mail.gmail.com.

#6 lore raw

Masahiro and Michal,

Friendly ping. Any comments on this patch?

The addition LTO .notes information emitted by kernel is used by pahole in the following patch:
https://lore.kernel.org/bpf/[email protected]/
(dwarf_loader: check .notes section for lto build info)

Thanks,

Yonghong

Quoted 152 lines · Sedat Dilek
Quoted 40 lines · Nick Desaulniers' via Clang Built Linux
Quoted 38 lines · Yonghong Song

Currently, clang LTO built vmlinux won't work with pahole. LTO introduced cross-cu dwarf tag references and broke current pahole model which handles one cu as a time. The solution is to merge all cu's as one pahole cu as in [1]. We would like to do this merging only if cross-cu dwarf references happens. The LTO build mode is a pretty good indication for that.

In earlier version of this patch ([2]), clang flag -grecord-gcc-switches is proposed to add to compilation flags so pahole could detect "-flto" and then merging cu's. This will increate the binary size of 1% without LTO though.

Arnaldo suggested to use a note to indicate the vmlinux is built with LTO. Such a cheap way to get whether the vmlinux is built with LTO or not helps pahole but is also useful for tracing as LTO may inline/delete/demote global functions, promote static functions, etc.

So this patch added an elfnote with a new type LINUX_ELFNOTE_LTO_INFO. The owner of the note is "Linux".

With gcc 8.4.1 and clang trunk, without LTO, I got

   $ readelf -n vmlinux
   Displaying notes found in: .notes
     Owner                Data size        Description
   ...
     Linux                0x00000004       func
      description data: 00 00 00 00
   ...
With "readelf -x ".notes" vmlinux", I can verify the above "func"
with type code 0x101.

With clang thin-LTO, I got the same as above except the following:

      description data: 01 00 00 00

which indicates the vmlinux is built with LTO.

Quoted 9 lines
Quoted 5 lines
   [1] https://lore.kernel.org/bpf/[email protected]/
   [2] https://lore.kernel.org/bpf/[email protected]/

LGTM thanks Yonghong!

Thanks for the patch.

Feel free to add:

As a note for the pahole side:
Recent patches require an adaptation of the define and its comment.

1. LINUX_ELFNOTE_BUILD_LTO -> LINUX_ELFNOTE_LTO_INFO
2. include/linux/elfnote.h -> include/linux/elfnote-lto.h

- Sedat -

Quoted 86 lines
Quoted 74 lines
---
  include/linux/elfnote-lto.h | 14 ++++++++++++++
  init/version.c              |  2 ++
  scripts/mod/modpost.c       |  2 ++
  3 files changed, 18 insertions(+)

create mode 100644 include/linux/elfnote-lto.h

Changelogs:
v3 -> v4:
. put new lto note in its own header file similar to
build-salt.h. (Nick)
v2 -> v3:
. abandoned the approach of adding -grecord-gcc-switches,
instead create a note to indicate whether it is a lto build or not. The note definition is in compiler.h. (Arnaldo)
v1 -> v2:
. limited to add -grecord-gcc-switches for LTO_CLANG
instead of all clang build

include/linux/elfnote-lto.h new file +14 -0
new file mode 100644
index 000000000000..d4635a3ecc4f
@@ -0,0 +1,14 @@
+#ifndef __ELFNOTE_LTO_H
+#define __ELFNOTE_LTO_H
+
+#include <linux/elfnote.h>
+
+#define LINUX_ELFNOTE_LTO_INFO 0x101
+
+#ifdef CONFIG_LTO
+#define BUILD_LTO_INFO ELFNOTE32("Linux", LINUX_ELFNOTE_LTO_INFO, 1)
+#else
+#define BUILD_LTO_INFO ELFNOTE32("Linux", LINUX_ELFNOTE_LTO_INFO, 0)
+#endif
+
+#endif /* __ELFNOTE_LTO_H */
init/version.c +2 -0
index 92afc782b043..1a356f5493e8 100644
@@ -9,6 +9,7 @@
 
  #include <generated/compile.h>
  #include <linux/build-salt.h>
+#include <linux/elfnote-lto.h>
  #include <linux/export.h>
  #include <linux/uts.h>
  #include <linux/utsname.h>
@@ -45,3 +46,4 @@ const char linux_proc_banner[] =
         " (" LINUX_COMPILER ") %s\n";
 
  BUILD_SALT;
+BUILD_LTO_INFO;
scripts/mod/modpost.c +2 -1
index 24725e50c7b4..98fb2bb024db 100644
@@ -2191,10 +2191,12 @@ static void add_header(struct buffer *b, struct module *mod)
          */
         buf_printf(b, "#define INCLUDE_VERMAGIC\n");
         buf_printf(b, "#include <linux/build-salt.h>\n");
+       buf_printf(b, "#include <linux/elfnote-lto.h>\n");
         buf_printf(b, "#include <linux/vermagic.h>\n");
         buf_printf(b, "#include <linux/compiler.h>\n");
         buf_printf(b, "\n");
         buf_printf(b, "BUILD_SALT;\n");
+       buf_printf(b, "BUILD_LTO_INFO;\n");
         buf_printf(b, "\n");
         buf_printf(b, "MODULE_INFO(vermagic, VERMAGIC_STRING);\n");
         buf_printf(b, "MODULE_INFO(name, KBUILD_MODNAME);\n");
--
Quoted 2 lines

2.30.2

--
Thanks,
~Nick Desaulniers

--
You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAKwvOdmX8d3XTzJFk5rN_PnOQYJ8bXMrh8DrhzqN=UBNdQiO3g@mail.gmail.com .

#7 lore raw
Quoted 11 lines · Yonghong Song

Masahiro and Michal,

Friendly ping. Any comments on this patch?

The addition LTO .notes information emitted by kernel is used by pahole in the following patch:
https://lore.kernel.org/bpf/[email protected]/
(dwarf_loader: check .notes section for lto build info)

Hi Yonghong,

the above pahole patch has this define and comment:

-static bool cus__merging_cu(Dwarf *dw)
+/* Match the define in linux:include/linux/elfnote.h */ +#define LINUX_ELFNOTE_BUILD_LTO 0x101

...and does not fit with the define and comment in this kernel patch:

+#include <linux/elfnote.h>
+
+#define LINUX_ELFNOTE_LTO_INFO 0x101

Thanks.

- Sedat -

Quoted 157 lines

Thanks,

Yonghong

Quoted 152 lines · Sedat Dilek
Quoted 40 lines · Nick Desaulniers' via Clang Built Linux
Quoted 38 lines · Yonghong Song

Currently, clang LTO built vmlinux won't work with pahole. LTO introduced cross-cu dwarf tag references and broke current pahole model which handles one cu as a time. The solution is to merge all cu's as one pahole cu as in [1]. We would like to do this merging only if cross-cu dwarf references happens. The LTO build mode is a pretty good indication for that.

In earlier version of this patch ([2]), clang flag -grecord-gcc-switches is proposed to add to compilation flags so pahole could detect "-flto" and then merging cu's. This will increate the binary size of 1% without LTO though.

Arnaldo suggested to use a note to indicate the vmlinux is built with LTO. Such a cheap way to get whether the vmlinux is built with LTO or not helps pahole but is also useful for tracing as LTO may inline/delete/demote global functions, promote static functions, etc.

So this patch added an elfnote with a new type LINUX_ELFNOTE_LTO_INFO. The owner of the note is "Linux".

With gcc 8.4.1 and clang trunk, without LTO, I got

   $ readelf -n vmlinux
   Displaying notes found in: .notes
     Owner                Data size        Description
   ...
     Linux                0x00000004       func
      description data: 00 00 00 00
   ...
With "readelf -x ".notes" vmlinux", I can verify the above "func"
with type code 0x101.

With clang thin-LTO, I got the same as above except the following:

      description data: 01 00 00 00

which indicates the vmlinux is built with LTO.

Quoted 9 lines
Quoted 5 lines
   [1] https://lore.kernel.org/bpf/[email protected]/
   [2] https://lore.kernel.org/bpf/[email protected]/

LGTM thanks Yonghong!

Thanks for the patch.

Feel free to add:

As a note for the pahole side:
Recent patches require an adaptation of the define and its comment.

1. LINUX_ELFNOTE_BUILD_LTO -> LINUX_ELFNOTE_LTO_INFO
2. include/linux/elfnote.h -> include/linux/elfnote-lto.h

- Sedat -

Quoted 86 lines
Quoted 74 lines
---
  include/linux/elfnote-lto.h | 14 ++++++++++++++
  init/version.c              |  2 ++
  scripts/mod/modpost.c       |  2 ++
  3 files changed, 18 insertions(+)

create mode 100644 include/linux/elfnote-lto.h

Changelogs:
v3 -> v4:
. put new lto note in its own header file similar to
build-salt.h. (Nick)
v2 -> v3:
. abandoned the approach of adding -grecord-gcc-switches,
instead create a note to indicate whether it is a lto build or not. The note definition is in compiler.h. (Arnaldo)
v1 -> v2:
. limited to add -grecord-gcc-switches for LTO_CLANG
instead of all clang build

include/linux/elfnote-lto.h new file +14 -0
new file mode 100644
index 000000000000..d4635a3ecc4f
@@ -0,0 +1,14 @@
+#ifndef __ELFNOTE_LTO_H
+#define __ELFNOTE_LTO_H
+
+#include <linux/elfnote.h>
+
+#define LINUX_ELFNOTE_LTO_INFO 0x101
+
+#ifdef CONFIG_LTO
+#define BUILD_LTO_INFO ELFNOTE32("Linux", LINUX_ELFNOTE_LTO_INFO, 1)
+#else
+#define BUILD_LTO_INFO ELFNOTE32("Linux", LINUX_ELFNOTE_LTO_INFO, 0)
+#endif
+
+#endif /* __ELFNOTE_LTO_H */
init/version.c +2 -0
index 92afc782b043..1a356f5493e8 100644
@@ -9,6 +9,7 @@
 
  #include <generated/compile.h>
  #include <linux/build-salt.h>
+#include <linux/elfnote-lto.h>
  #include <linux/export.h>
  #include <linux/uts.h>
  #include <linux/utsname.h>
@@ -45,3 +46,4 @@ const char linux_proc_banner[] =
         " (" LINUX_COMPILER ") %s\n";
 
  BUILD_SALT;
+BUILD_LTO_INFO;
scripts/mod/modpost.c +2 -1
index 24725e50c7b4..98fb2bb024db 100644
@@ -2191,10 +2191,12 @@ static void add_header(struct buffer *b, struct module *mod)
          */
         buf_printf(b, "#define INCLUDE_VERMAGIC\n");
         buf_printf(b, "#include <linux/build-salt.h>\n");
+       buf_printf(b, "#include <linux/elfnote-lto.h>\n");
         buf_printf(b, "#include <linux/vermagic.h>\n");
         buf_printf(b, "#include <linux/compiler.h>\n");
         buf_printf(b, "\n");
         buf_printf(b, "BUILD_SALT;\n");
+       buf_printf(b, "BUILD_LTO_INFO;\n");
         buf_printf(b, "\n");
         buf_printf(b, "MODULE_INFO(vermagic, VERMAGIC_STRING);\n");
         buf_printf(b, "MODULE_INFO(name, KBUILD_MODNAME);\n");
--
Quoted 2 lines

2.30.2

--
Thanks,
~Nick Desaulniers

--
You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAKwvOdmX8d3XTzJFk5rN_PnOQYJ8bXMrh8DrhzqN=UBNdQiO3g@mail.gmail.com .

#8 lore raw
Quoted 23 lines · Sedat Dilek
Quoted 11 lines · Yonghong Song

Masahiro and Michal,

Friendly ping. Any comments on this patch?

The addition LTO .notes information emitted by kernel is used by pahole in the following patch:
https://lore.kernel.org/bpf/[email protected]/
(dwarf_loader: check .notes section for lto build info)

Hi Yonghong,

the above pahole patch has this define and comment:

(quoted diff) +2 -1
-static bool cus__merging_cu(Dwarf *dw)
+/* Match the define in linux:include/linux/elfnote.h */
+#define LINUX_ELFNOTE_BUILD_LTO 0x101

...and does not fit with the define and comment in this kernel patch:

include/linux/elfnote-lto.h line 6

Quoted 3 lines
include/linux/elfnote-lto.h new file +3 -0
+#include <linux/elfnote.h>
+
+#define LINUX_ELFNOTE_LTO_INFO 0x101

Thanks, Sedat. I am aware of this. I think we can wait in pahole to make a change until the kernel patch is finalized and merged. The kernel patch may still change as we haven't get maintainer's comment. This will avoid unnecessary churn's in pahole side.

Quoted 77 lines

Thanks.

- Sedat -

Quoted 71 lines

Thanks,

Yonghong

Quoted 66 lines · Sedat Dilek
Quoted 40 lines · Nick Desaulniers' via Clang Built Linux
Quoted 38 lines · Yonghong Song

Currently, clang LTO built vmlinux won't work with pahole. LTO introduced cross-cu dwarf tag references and broke current pahole model which handles one cu as a time. The solution is to merge all cu's as one pahole cu as in [1]. We would like to do this merging only if cross-cu dwarf references happens. The LTO build mode is a pretty good indication for that.

In earlier version of this patch ([2]), clang flag -grecord-gcc-switches is proposed to add to compilation flags so pahole could detect "-flto" and then merging cu's. This will increate the binary size of 1% without LTO though.

Arnaldo suggested to use a note to indicate the vmlinux is built with LTO. Such a cheap way to get whether the vmlinux is built with LTO or not helps pahole but is also useful for tracing as LTO may inline/delete/demote global functions, promote static functions, etc.

So this patch added an elfnote with a new type LINUX_ELFNOTE_LTO_INFO. The owner of the note is "Linux".

With gcc 8.4.1 and clang trunk, without LTO, I got

    $ readelf -n vmlinux
    Displaying notes found in: .notes
      Owner                Data size        Description
    ...
      Linux                0x00000004       func
       description data: 00 00 00 00
    ...
With "readelf -x ".notes" vmlinux", I can verify the above "func"
with type code 0x101.

With clang thin-LTO, I got the same as above except the following:

       description data: 01 00 00 00

which indicates the vmlinux is built with LTO.

Quoted 9 lines
Quoted 5 lines
    [1] https://lore.kernel.org/bpf/[email protected]/
    [2] https://lore.kernel.org/bpf/[email protected]/

LGTM thanks Yonghong!

Thanks for the patch.

Feel free to add:

As a note for the pahole side:
Recent patches require an adaptation of the define and its comment.

1. LINUX_ELFNOTE_BUILD_LTO -> LINUX_ELFNOTE_LTO_INFO
2. include/linux/elfnote.h -> include/linux/elfnote-lto.h

- Sedat -

[...]

#9 lore raw
Quoted 27 lines · Yonghong Song
Quoted 23 lines · Sedat Dilek
Quoted 11 lines · Yonghong Song

Masahiro and Michal,

Friendly ping. Any comments on this patch?

The addition LTO .notes information emitted by kernel is used by pahole in the following patch:
https://lore.kernel.org/bpf/[email protected]/
(dwarf_loader: check .notes section for lto build info)

Hi Yonghong,

the above pahole patch has this define and comment:

(quoted diff) +2 -1
-static bool cus__merging_cu(Dwarf *dw)
+/* Match the define in linux:include/linux/elfnote.h */
+#define LINUX_ELFNOTE_BUILD_LTO 0x101

...and does not fit with the define and comment in this kernel patch:

Quoted 10 lines

include/linux/elfnote-lto.h line 6

Quoted 3 lines
include/linux/elfnote-lto.h new file +3 -0
+#include <linux/elfnote.h>
+
+#define LINUX_ELFNOTE_LTO_INFO 0x101

Thanks, Sedat. I am aware of this. I think we can wait in pahole to make a change until the kernel patch is finalized and merged. The kernel patch may still change as we haven't get maintainer's comment. This will avoid unnecessary churn's in pahole side.

I am OK with that.

- Sedat -

Quoted 78 lines
Quoted 77 lines

Thanks.

- Sedat -

Quoted 71 lines

Thanks,

Yonghong

Quoted 66 lines · Sedat Dilek
Quoted 40 lines · Nick Desaulniers' via Clang Built Linux
Quoted 38 lines · Yonghong Song

Currently, clang LTO built vmlinux won't work with pahole. LTO introduced cross-cu dwarf tag references and broke current pahole model which handles one cu as a time. The solution is to merge all cu's as one pahole cu as in [1]. We would like to do this merging only if cross-cu dwarf references happens. The LTO build mode is a pretty good indication for that.

In earlier version of this patch ([2]), clang flag -grecord-gcc-switches is proposed to add to compilation flags so pahole could detect "-flto" and then merging cu's. This will increate the binary size of 1% without LTO though.

Arnaldo suggested to use a note to indicate the vmlinux is built with LTO. Such a cheap way to get whether the vmlinux is built with LTO or not helps pahole but is also useful for tracing as LTO may inline/delete/demote global functions, promote static functions, etc.

So this patch added an elfnote with a new type LINUX_ELFNOTE_LTO_INFO. The owner of the note is "Linux".

With gcc 8.4.1 and clang trunk, without LTO, I got

    $ readelf -n vmlinux
    Displaying notes found in: .notes
      Owner                Data size        Description
    ...
      Linux                0x00000004       func
       description data: 00 00 00 00
    ...
With "readelf -x ".notes" vmlinux", I can verify the above "func"
with type code 0x101.

With clang thin-LTO, I got the same as above except the following:

       description data: 01 00 00 00

which indicates the vmlinux is built with LTO.

Quoted 9 lines
Quoted 5 lines
    [1] https://lore.kernel.org/bpf/[email protected]/
    [2] https://lore.kernel.org/bpf/[email protected]/

LGTM thanks Yonghong!

Thanks for the patch.

Feel free to add:

As a note for the pahole side:
Recent patches require an adaptation of the define and its comment.

1. LINUX_ELFNOTE_BUILD_LTO -> LINUX_ELFNOTE_LTO_INFO
2. include/linux/elfnote.h -> include/linux/elfnote-lto.h

- Sedat -

[...]

#10 lore raw
In reply to Yonghong Song

Em Tue, Apr 06, 2021 at 11:23:27PM -0700, Yonghong Song escreveu:

Quoted 19 lines · Sedat Dilek
Quoted 17 lines · Yonghong Song
Quoted 8 lines

Masahiro and Michal,

Friendly ping. Any comments on this patch?

The addition LTO .notes information emitted by kernel is used by pahole in the following patch:
https://lore.kernel.org/bpf/[email protected]/
(dwarf_loader: check .notes section for lto build info)

the above pahole patch has this define and comment:

(quoted diff) +2 -1
-static bool cus__merging_cu(Dwarf *dw)
+/* Match the define in linux:include/linux/elfnote.h */
+#define LINUX_ELFNOTE_BUILD_LTO 0x101

...and does not fit with the define and comment in this kernel patch:

Quoted 9 lines

include/linux/elfnote-lto.h line 6

Quoted 3 lines
include/linux/elfnote-lto.h new file +3 -0
+#include <linux/elfnote.h>
+
+#define LINUX_ELFNOTE_LTO_INFO 0x101

Thanks, Sedat. I am aware of this. I think we can wait in pahole to make a change until the kernel patch is finalized and merged. The kernel patch may still change as we haven't get maintainer's comment. This will avoid unnecessary churn's in pahole side.

So, I tested with clang 12 on fedora rawhide as well on fedora 33, and I'm satisfied with the current state to release v1.21, Masahiro, have you had the time to look at this?

Yonghong, as we have a fallback in case the ELF note isn't available, I think we're safe even if the notes patch merge gets delayed, right?

- Arnaldo

#11 lore raw
Quoted 30 lines · Arnaldo Carvalho de Melo

Em Tue, Apr 06, 2021 at 11:23:27PM -0700, Yonghong Song escreveu:

Quoted 19 lines · Sedat Dilek
Quoted 17 lines · Yonghong Song
Quoted 8 lines

Masahiro and Michal,

Friendly ping. Any comments on this patch?

The addition LTO .notes information emitted by kernel is used by pahole in the following patch:
https://lore.kernel.org/bpf/[email protected]/
(dwarf_loader: check .notes section for lto build info)

the above pahole patch has this define and comment:

(quoted diff) +2 -1
-static bool cus__merging_cu(Dwarf *dw)
+/* Match the define in linux:include/linux/elfnote.h */
+#define LINUX_ELFNOTE_BUILD_LTO 0x101

...and does not fit with the define and comment in this kernel patch:

Quoted 9 lines

Review

Quoted 3 lines
(quoted diff) +3 -0
+#include <linux/elfnote.h>
+
+#define LINUX_ELFNOTE_LTO_INFO 0x101

Thanks, Sedat. I am aware of this. I think we can wait in pahole to make a change until the kernel patch is finalized and merged. The kernel patch may still change as we haven't get maintainer's comment. This will avoid unnecessary churn's in pahole side.

Quoted 6 lines

So, I tested with clang 12 on fedora rawhide as well on fedora 33, and I'm satisfied with the current state to release v1.21, Masahiro, have you had the time to look at this?

Yonghong, as we have a fallback in case the ELF note isn't available, I think we're safe even if the notes patch merge gets delayed, right?

Right. That is why I separated the notes patch from other patches. We can revisit it once the kernel patch is settled.

Quoted 3 lines

- Arnaldo

#12 lore raw
Quoted 38 lines · Yonghong Song
Quoted 34 lines · Arnaldo Carvalho de Melo

Em Tue, Apr 06, 2021 at 11:23:27PM -0700, Yonghong Song escreveu:

Quoted 19 lines · Sedat Dilek
Quoted 17 lines · Yonghong Song
Quoted 8 lines

Masahiro and Michal,

Friendly ping. Any comments on this patch?

The addition LTO .notes information emitted by kernel is used by pahole in the following patch:
https://lore.kernel.org/bpf/[email protected]/
(dwarf_loader: check .notes section for lto build info)

the above pahole patch has this define and comment:

(quoted diff) +2 -1
-static bool cus__merging_cu(Dwarf *dw)
+/* Match the define in linux:include/linux/elfnote.h */
+#define LINUX_ELFNOTE_BUILD_LTO 0x101

...and does not fit with the define and comment in this kernel patch:

Quoted 9 lines

Review

Quoted 3 lines
(quoted diff) +3 -0
+#include <linux/elfnote.h>
+
+#define LINUX_ELFNOTE_LTO_INFO 0x101

Thanks, Sedat. I am aware of this. I think we can wait in pahole to make a change until the kernel patch is finalized and merged. The kernel patch may still change as we haven't get maintainer's comment. This will avoid unnecessary churn's in pahole side.

So, I tested with clang 12 on fedora rawhide as well on fedora 33, and I'm satisfied with the current state to release v1.21, Masahiro, have you had the time to look at this?

Quoted 9 lines
Quoted 2 lines

Yonghong, as we have a fallback in case the ELF note isn't available, I think we're safe even if the notes patch merge gets delayed, right?

Right. That is why I separated the notes patch from other patches. We can revisit it once the kernel patch is settled.

Quoted 3 lines

- Arnaldo

Applied to linux-kbuild. Thanks.

Signature · 2 lines
Best Regards
Masahiro Yamada
#13 lore raw
Quoted 46 lines · Masahiro Yamada
Quoted 45 lines · Yonghong Song
Quoted 30 lines · Arnaldo Carvalho de Melo

Em Tue, Apr 06, 2021 at 11:23:27PM -0700, Yonghong Song escreveu:

Quoted 19 lines · Sedat Dilek
Quoted 17 lines · Yonghong Song
Quoted 8 lines

Masahiro and Michal,

Friendly ping. Any comments on this patch?

The addition LTO .notes information emitted by kernel is used by pahole in the following patch:
https://lore.kernel.org/bpf/[email protected]/
(dwarf_loader: check .notes section for lto build info)

the above pahole patch has this define and comment:

(quoted diff) +2 -1
-static bool cus__merging_cu(Dwarf *dw)
+/* Match the define in linux:include/linux/elfnote.h */
+#define LINUX_ELFNOTE_BUILD_LTO 0x101

...and does not fit with the define and comment in this kernel patch:

Quoted 9 lines

Review

Quoted 3 lines
(quoted diff) +3 -0
+#include <linux/elfnote.h>
+
+#define LINUX_ELFNOTE_LTO_INFO 0x101

Thanks, Sedat. I am aware of this. I think we can wait in pahole to make a change until the kernel patch is finalized and merged. The kernel patch may still change as we haven't get maintainer's comment. This will avoid unnecessary churn's in pahole side.

Quoted 6 lines

So, I tested with clang 12 on fedora rawhide as well on fedora 33, and I'm satisfied with the current state to release v1.21, Masahiro, have you had the time to look at this?

Yonghong, as we have a fallback in case the ELF note isn't available, I think we're safe even if the notes patch merge gets delayed, right?

Right. That is why I separated the notes patch from other patches. We can revisit it once the kernel patch is settled.

Quoted 1 line
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines

- Arnaldo

Applied to linux-kbuild. Thanks.

Thanks!

Quoted 2 lines
#14 lore raw
Quoted 50 lines · Yonghong Song
Quoted 46 lines · Masahiro Yamada
Quoted 45 lines · Yonghong Song
Quoted 30 lines · Arnaldo Carvalho de Melo

Em Tue, Apr 06, 2021 at 11:23:27PM -0700, Yonghong Song escreveu:

Quoted 19 lines · Sedat Dilek
Quoted 17 lines · Yonghong Song
Quoted 8 lines

Masahiro and Michal,

Friendly ping. Any comments on this patch?

The addition LTO .notes information emitted by kernel is used by pahole in the following patch:
https://lore.kernel.org/bpf/[email protected]/
(dwarf_loader: check .notes section for lto build info)

the above pahole patch has this define and comment:

(quoted diff) +2 -1
-static bool cus__merging_cu(Dwarf *dw)
+/* Match the define in linux:include/linux/elfnote.h */
+#define LINUX_ELFNOTE_BUILD_LTO 0x101

...and does not fit with the define and comment in this kernel patch:

Quoted 9 lines

Review

Quoted 3 lines
(quoted diff) +3 -0
+#include <linux/elfnote.h>
+
+#define LINUX_ELFNOTE_LTO_INFO 0x101

Thanks, Sedat. I am aware of this. I think we can wait in pahole to make a change until the kernel patch is finalized and merged. The kernel patch may still change as we haven't get maintainer's comment. This will avoid unnecessary churn's in pahole side.

Quoted 6 lines

So, I tested with clang 12 on fedora rawhide as well on fedora 33, and I'm satisfied with the current state to release v1.21, Masahiro, have you had the time to look at this?

Yonghong, as we have a fallback in case the ELF note isn't available, I think we're safe even if the notes patch merge gets delayed, right?

Right. That is why I separated the notes patch from other patches. We can revisit it once the kernel patch is settled.

Quoted 1 line
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines

- Arnaldo

Applied to linux-kbuild. Thanks.

Thanks!

Great to see this applied.
Thanks.

- Sedat -

#15 lore raw
In reply to Nick Desaulniers

Em Fri, Apr 02, 2021 at 11:07:10AM -0700, Nick Desaulniers escreveu:

Quoted 38 lines · Yonghong Song
Quoted 37 lines

Currently, clang LTO built vmlinux won't work with pahole. LTO introduced cross-cu dwarf tag references and broke current pahole model which handles one cu as a time. The solution is to merge all cu's as one pahole cu as in [1]. We would like to do this merging only if cross-cu dwarf references happens. The LTO build mode is a pretty good indication for that.

In earlier version of this patch ([2]), clang flag -grecord-gcc-switches is proposed to add to compilation flags so pahole could detect "-flto" and then merging cu's. This will increate the binary size of 1% without LTO though.

Arnaldo suggested to use a note to indicate the vmlinux is built with LTO. Such a cheap way to get whether the vmlinux is built with LTO or not helps pahole but is also useful for tracing as LTO may inline/delete/demote global functions, promote static functions, etc.

So this patch added an elfnote with a new type LINUX_ELFNOTE_LTO_INFO. The owner of the note is "Linux".

With gcc 8.4.1 and clang trunk, without LTO, I got

  $ readelf -n vmlinux
  Displaying notes found in: .notes
    Owner                Data size        Description
  ...
    Linux                0x00000004       func
     description data: 00 00 00 00
  ...
With "readelf -x ".notes" vmlinux", I can verify the above "func"
with type code 0x101.

With clang thin-LTO, I got the same as above except the following:

     description data: 01 00 00 00

which indicates the vmlinux is built with LTO.

Quoted 8 lines
Quoted 5 lines

[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/

LGTM thanks Yonghong!

Thanks!

- Arnaldo