Skip to contentSkip to conversation

libbpf: ignore .eh_frame sections when parsing elf files

PATCH In review Toke Høiland-Jørgensen 4 participants 12 messages Aug 26, 2021 – Sep 7, 2021

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 libbpf: ignore .eh_frame sections when parsing elf files

When .eh_frame and .rel.eh_frame sections are present in BPF object files, libbpf produces errors like this when loading the file:

libbpf: elf: skipping unrecognized data section(32) .eh_frame
libbpf: elf: skipping relo section(33) .rel.eh_frame for section(32) .eh_frame

It is possible to get rid of the .eh_frame section by adding -fno-asynchronous-unwind-tables to the compilation, but we have seen multiple examples of these sections appearing in BPF files in the wild, most recently in samples/bpf, fixed by:
5a0ae9872d5c ("bpf, samples: Add -fno-asynchronous-unwind-tables to BPF Clang invocation")

While the errors are technically harmless, they look odd and confuse users. So let's make libbpf filter out those sections, by adding .eh_frame to the filter check in is_sec_name_dwarf().

v2:
- Expand explanation in the commit message

---
 tools/lib/bpf/libbpf.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
tools/lib/bpf/libbpf.c +2 -1
index 88d8825fc6f6..b1dc97b95965 100644
@@ -2909,7 +2909,8 @@ static Elf_Data *elf_sec_data(const struct bpf_object *obj, Elf_Scn *scn)
 static bool is_sec_name_dwarf(const char *name)
 {
 	/* approximation, but the actual list is too long */
-	return strncmp(name, ".debug_", sizeof(".debug_") - 1) == 0;
+	return (strncmp(name, ".debug_", sizeof(".debug_") - 1) == 0 ||
+		strncmp(name, ".eh_frame", sizeof(".eh_frame") - 1) == 0);
 }
 
 static bool ignore_elf_section(GElf_Shdr *hdr, const char *name)
Signature · 1 line
2.33.0
#1 lore raw
Quoted 7 lines · Toke Høiland-Jørgensen

When .eh_frame and .rel.eh_frame sections are present in BPF object files, libbpf produces errors like this when loading the file:

libbpf: elf: skipping unrecognized data section(32) .eh_frame
libbpf: elf: skipping relo section(33) .rel.eh_frame for section(32) .eh_frame

Quoted 7 lines

It is possible to get rid of the .eh_frame section by adding -fno-asynchronous-unwind-tables to the compilation, but we have seen multiple examples of these sections appearing in BPF files in the wild, most recently in samples/bpf, fixed by:
5a0ae9872d5c ("bpf, samples: Add -fno-asynchronous-unwind-tables to BPF Clang invocation")

While the errors are technically harmless, they look odd and confuse users.

These warnings point out invalid set of compiler flags used for compiling BPF object files, though. Which is a good thing and should incentivize anyone getting those warnings to check and fix how they do BPF compilation. Those .eh_frame sections shouldn't be present in BPF object files at all, and that's what libbpf is trying to say.

I don't know exactly in which situations that .eh_frame section is added, but looking at our selftests (and now samples/bpf as well), where we use -target bpf, we don't need -fno-asynchronous-unwind-tables at all.

So instead of hiding the problem, let's use this as an opportunity to fix those user's compilation flags instead.

Quoted 28 lines

So let's make libbpf filter out those sections, by adding .eh_frame to the filter check in is_sec_name_dwarf().

v2:
- Expand explanation in the commit message

---
 tools/lib/bpf/libbpf.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
tools/lib/bpf/libbpf.c +2 -2
index 88d8825fc6f6..b1dc97b95965 100644
@@ -2909,7 +2909,8 @@ static Elf_Data *elf_sec_data(const struct bpf_object *obj, Elf_Scn *scn)
 static bool is_sec_name_dwarf(const char *name)
 {
        /* approximation, but the actual list is too long */
-       return strncmp(name, ".debug_", sizeof(".debug_") - 1) == 0;
+       return (strncmp(name, ".debug_", sizeof(".debug_") - 1) == 0 ||
+               strncmp(name, ".eh_frame", sizeof(".eh_frame") - 1) == 0);
 }
 
 static bool ignore_elf_section(GElf_Shdr *hdr, const char *name)
--

2.33.0

#2 lore raw
Quoted 14 lines · Andrii Nakryiko
Quoted 13 lines · Toke Høiland-Jørgensen

When .eh_frame and .rel.eh_frame sections are present in BPF object files, libbpf produces errors like this when loading the file:

libbpf: elf: skipping unrecognized data section(32) .eh_frame
libbpf: elf: skipping relo section(33) .rel.eh_frame for section(32) .eh_frame

It is possible to get rid of the .eh_frame section by adding -fno-asynchronous-unwind-tables to the compilation, but we have seen multiple examples of these sections appearing in BPF files in the wild, most recently in samples/bpf, fixed by:
5a0ae9872d5c ("bpf, samples: Add -fno-asynchronous-unwind-tables to BPF Clang invocation")

Quoted 7 lines
Quoted 1 line

While the errors are technically harmless, they look odd and confuse users.

These warnings point out invalid set of compiler flags used for compiling BPF object files, though. Which is a good thing and should incentivize anyone getting those warnings to check and fix how they do BPF compilation. Those .eh_frame sections shouldn't be present in BPF object files at all, and that's what libbpf is trying to say.

Apart from triggering that warning, what effect does this have, though? The programs seem to work just fine (as evidenced by the fact that samples/bpf has been built this way for years, for instance)...

Also, how is a user supposed to go from that cryptic error message to figuring out that it has something to do with compiler flags?

Quoted 4 lines

I don't know exactly in which situations that .eh_frame section is added, but looking at our selftests (and now samples/bpf as well), where we use -target bpf, we don't need -fno-asynchronous-unwind-tables at all.

This seems to at least be compiler-dependent. We ran into this with bpftool as well (for the internal BPF programs it loads whenever it runs), which already had '-target bpf' in the Makefile. We're carrying an internal RHEL patch adding -fno-asynchronous-unwind-tables to the bpftool build to fix this...

Quoted 2 lines

So instead of hiding the problem, let's use this as an opportunity to fix those user's compilation flags instead.

This really doesn't seem like something that's helping anyone, it's just annoying and confusing users...

-Toke

#3 lore raw
Quoted 25 lines · Toke Høiland-Jørgensen
Quoted 14 lines · Andrii Nakryiko
Quoted 13 lines · Toke Høiland-Jørgensen

When .eh_frame and .rel.eh_frame sections are present in BPF object files, libbpf produces errors like this when loading the file:

libbpf: elf: skipping unrecognized data section(32) .eh_frame
libbpf: elf: skipping relo section(33) .rel.eh_frame for section(32) .eh_frame

It is possible to get rid of the .eh_frame section by adding -fno-asynchronous-unwind-tables to the compilation, but we have seen multiple examples of these sections appearing in BPF files in the wild, most recently in samples/bpf, fixed by:
5a0ae9872d5c ("bpf, samples: Add -fno-asynchronous-unwind-tables to BPF Clang invocation")

Quoted 7 lines
Quoted 1 line

While the errors are technically harmless, they look odd and confuse users.

These warnings point out invalid set of compiler flags used for compiling BPF object files, though. Which is a good thing and should incentivize anyone getting those warnings to check and fix how they do BPF compilation. Those .eh_frame sections shouldn't be present in BPF object files at all, and that's what libbpf is trying to say.

Quoted 6 lines

Apart from triggering that warning, what effect does this have, though? The programs seem to work just fine (as evidenced by the fact that samples/bpf has been built this way for years, for instance)...

Also, how is a user supposed to go from that cryptic error message to figuring out that it has something to do with compiler flags?

Google and find discussions like these?.. I don't think libbpf error messages have to include intro into DWARF and .eh_frame.

Just googling ".eh_frame" gives me [0] as a first link, which seems to describe what it is and how to get rid of it.

[0] https://stackoverflow.com/questions/26300819/why-gcc-compiled-c-program-needs-eh-frame-section

Quoted 11 lines
Quoted 5 lines

I don't know exactly in which situations that .eh_frame section is added, but looking at our selftests (and now samples/bpf as well), where we use -target bpf, we don't need -fno-asynchronous-unwind-tables at all.

This seems to at least be compiler-dependent. We ran into this with bpftool as well (for the internal BPF programs it loads whenever it runs), which already had '-target bpf' in the Makefile. We're carrying an internal RHEL patch adding -fno-asynchronous-unwind-tables to the bpftool build to fix this...

So instead of figuring out why your compilers cause .eh_frame generation (while they shouldn't), you are trying to hide the warning in libbpf? This hasn't been the problem in production apps at Facebook, nor with libbpf-tools or libbpf-bootstrap apps. Which just makes me keep this warning more. Once we support multiple .rodata/.data/.bss sections for libbpf, I think I'll turn all those unrecognized sections into actual errors. I'd rather not have unknown sections being just ignored by libbpf. Someday we might actually use .eh_frame with BPF objects, that's when this will become not an error or warning.

Quoted 6 lines
Quoted 3 lines

So instead of hiding the problem, let's use this as an opportunity to fix those user's compilation flags instead.

This really doesn't seem like something that's helping anyone, it's just annoying and confusing users...

Warnings like "libbpf: elf: skipping unrecognized data section(4) .rodata.str1.1" annoy me as well, and that's one of the reasons I'll add support for multiple .rodata sections. So annoying is fine, it raises awareness and incentivizes fixing the problem.

Quoted 3 lines

-Toke

#4 lore raw
In reply to Toke Høiland-Jørgensen
Quoted 36 lines · Toke Høiland-Jørgensen
Quoted 14 lines · Andrii Nakryiko
Quoted 13 lines · Toke Høiland-Jørgensen

When .eh_frame and .rel.eh_frame sections are present in BPF object files, libbpf produces errors like this when loading the file:

libbpf: elf: skipping unrecognized data section(32) .eh_frame
libbpf: elf: skipping relo section(33) .rel.eh_frame for section(32) .eh_frame

It is possible to get rid of the .eh_frame section by adding -fno-asynchronous-unwind-tables to the compilation, but we have seen multiple examples of these sections appearing in BPF files in the wild, most recently in samples/bpf, fixed by:
5a0ae9872d5c ("bpf, samples: Add -fno-asynchronous-unwind-tables to BPF Clang invocation")

Quoted 7 lines
Quoted 1 line

While the errors are technically harmless, they look odd and confuse users.

These warnings point out invalid set of compiler flags used for compiling BPF object files, though. Which is a good thing and should incentivize anyone getting those warnings to check and fix how they do BPF compilation. Those .eh_frame sections shouldn't be present in BPF object files at all, and that's what libbpf is trying to say.

Apart from triggering that warning, what effect does this have, though? The programs seem to work just fine (as evidenced by the fact that samples/bpf has been built this way for years, for instance)...

Also, how is a user supposed to go from that cryptic error message to figuring out that it has something to do with compiler flags?

Quoted 4 lines

I don't know exactly in which situations that .eh_frame section is added, but looking at our selftests (and now samples/bpf as well), where we use -target bpf, we don't need -fno-asynchronous-unwind-tables at all.

Quoted 5 lines

This seems to at least be compiler-dependent. We ran into this with bpftool as well (for the internal BPF programs it loads whenever it runs), which already had '-target bpf' in the Makefile. We're carrying an internal RHEL patch adding -fno-asynchronous-unwind-tables to the bpftool build to fix this...

I haven't seen an instance of .eh_frame as well with -target bpf. Do you have a reproducible test case? I would like to investigate what is the possible cause and whether we could do something in llvm to prevent its generatin. Thanks!

Quoted 9 lines
Quoted 3 lines

So instead of hiding the problem, let's use this as an opportunity to fix those user's compilation flags instead.

This really doesn't seem like something that's helping anyone, it's just annoying and confusing users...

-Toke

#5 lore raw
Quoted 43 lines · Yonghong Song
Quoted 36 lines · Toke Høiland-Jørgensen
Quoted 14 lines · Andrii Nakryiko
Quoted 13 lines · Toke Høiland-Jørgensen

When .eh_frame and .rel.eh_frame sections are present in BPF object files, libbpf produces errors like this when loading the file:

libbpf: elf: skipping unrecognized data section(32) .eh_frame
libbpf: elf: skipping relo section(33) .rel.eh_frame for section(32) .eh_frame

It is possible to get rid of the .eh_frame section by adding -fno-asynchronous-unwind-tables to the compilation, but we have seen multiple examples of these sections appearing in BPF files in the wild, most recently in samples/bpf, fixed by:
5a0ae9872d5c ("bpf, samples: Add -fno-asynchronous-unwind-tables to BPF Clang invocation")

Quoted 7 lines
Quoted 1 line

While the errors are technically harmless, they look odd and confuse users.

These warnings point out invalid set of compiler flags used for compiling BPF object files, though. Which is a good thing and should incentivize anyone getting those warnings to check and fix how they do BPF compilation. Those .eh_frame sections shouldn't be present in BPF object files at all, and that's what libbpf is trying to say.

Apart from triggering that warning, what effect does this have, though? The programs seem to work just fine (as evidenced by the fact that samples/bpf has been built this way for years, for instance)...

Also, how is a user supposed to go from that cryptic error message to figuring out that it has something to do with compiler flags?

Quoted 4 lines

I don't know exactly in which situations that .eh_frame section is added, but looking at our selftests (and now samples/bpf as well), where we use -target bpf, we don't need -fno-asynchronous-unwind-tables at all.

Quoted 5 lines

This seems to at least be compiler-dependent. We ran into this with bpftool as well (for the internal BPF programs it loads whenever it runs), which already had '-target bpf' in the Makefile. We're carrying an internal RHEL patch adding -fno-asynchronous-unwind-tables to the bpftool build to fix this...

Quoted 4 lines

I haven't seen an instance of .eh_frame as well with -target bpf. Do you have a reproducible test case? I would like to investigate what is the possible cause and whether we could do something in llvm to prevent its generatin. Thanks!

We found this in the RHEL builds of bpftool. I don't think we're doing anything special, other than maybe building with a clang version that's a few versions behind:

# clang --version
clang version 11.0.0 (Red Hat 11.0.0-1.module+el8.4.0+8598+a071fcd5)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

So I suppose it may resolve itself once we upgrade LLVM?

-Toke

#6 lore raw
Quoted 11 lines · Toke Høiland-Jørgensen
Quoted 8 lines · Yonghong Song
Quoted 7 lines · Toke Høiland-Jørgensen
Quoted 5 lines · Andrii Nakryiko
Quoted 4 lines · Toke Høiland-Jørgensen

When .eh_frame and .rel.eh_frame sections are present in BPF object files, libbpf produces errors like this when loading the file:

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

libbpf: elf: skipping unrecognized data section(32) .eh_frame
libbpf: elf: skipping relo section(33) .rel.eh_frame for section(32) .eh_frame

It is possible to get rid of the .eh_frame section by adding -fno-asynchronous-unwind-tables to the compilation, but we have seen multiple examples of these sections appearing in BPF files in the wild, most recently in samples/bpf, fixed by:
5a0ae9872d5c ("bpf, samples: Add -fno-

/to BPF Clang invocation")

Quoted 36 lines
Quoted 27 lines
Quoted 21 lines
Quoted 8 lines
Quoted 2 lines

While the errors are technically harmless, they look odd and confuse users.

These warnings point out invalid set of compiler flags used for compiling BPF object files, though. Which is a good thing and should incentivize anyone getting those warnings to check and fix how they do BPF compilation. Those .eh_frame sections shouldn't be present in BPF object files at all, and that's what libbpf is trying to say.

Apart from triggering that warning, what effect does this have, though? The programs seem to work just fine (as evidenced by the fact that samples/bpf has been built this way for years, for instance)...

Also, how is a user supposed to go from that cryptic error message to figuring out that it has something to do with compiler flags?

Quoted 4 lines

I don't know exactly in which situations that .eh_frame section is added, but looking at our selftests (and now samples/bpf as well), where we use -target bpf, we don't need -fno-asynchronous-unwind-tables at all.

Quoted 5 lines

This seems to at least be compiler-dependent. We ran into this with bpftool as well (for the internal BPF programs it loads whenever it runs), which already had '-target bpf' in the Makefile. We're carrying an internal RHEL patch adding -fno-asynchronous-unwind-tables to the bpftool build to fix this...

Quoted 4 lines

I haven't seen an instance of .eh_frame as well with -target bpf. Do you have a reproducible test case? I would like to investigate what is the possible cause and whether we could do something in llvm to prevent its generatin. Thanks!

We found this in the RHEL builds of bpftool. I don't think we're doing anything special, other than maybe building with a clang version that's a few versions behind:

Quoted 7 lines
# clang --version
clang version 11.0.0 (Red Hat 11.0.0-1.module+el8.4.0+8598+a071fcd5)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

So I suppose it may resolve itself once we upgrade LLVM?

That's odd. I don't think I've seen this issue even with clang 11 (but I built it myself).
If there is a fix indeed let's backport it to llvm 11. The user experience matters.
It could be llvm configuration too.
I'm guessing some build flags might influence default settings for unwind tables.

Yonghong, can we make bpf backend to ignore needsUnwindTableEntry ?

#7 lore raw
Quoted 63 lines · Alexei Starovoitov
Quoted 11 lines · Toke Høiland-Jørgensen
Quoted 8 lines · Yonghong Song
Quoted 7 lines · Toke Høiland-Jørgensen
Quoted 5 lines · Andrii Nakryiko
Quoted 4 lines · Toke Høiland-Jørgensen

When .eh_frame and .rel.eh_frame sections are present in BPF object files, libbpf produces errors like this when loading the file:

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

libbpf: elf: skipping unrecognized data section(32) .eh_frame
libbpf: elf: skipping relo section(33) .rel.eh_frame for section(32) .eh_frame

It is possible to get rid of the .eh_frame section by adding -fno-asynchronous-unwind-tables to the compilation, but we have seen multiple examples of these sections appearing in BPF files in the wild, most recently in samples/bpf, fixed by:
5a0ae9872d5c ("bpf, samples: Add -fno-

/to BPF Clang invocation")

Quoted 42 lines
Quoted 27 lines
Quoted 21 lines
Quoted 8 lines
Quoted 2 lines

While the errors are technically harmless, they look odd and confuse users.

These warnings point out invalid set of compiler flags used for compiling BPF object files, though. Which is a good thing and should incentivize anyone getting those warnings to check and fix how they do BPF compilation. Those .eh_frame sections shouldn't be present in BPF object files at all, and that's what libbpf is trying to say.

Apart from triggering that warning, what effect does this have, though? The programs seem to work just fine (as evidenced by the fact that samples/bpf has been built this way for years, for instance)...

Also, how is a user supposed to go from that cryptic error message to figuring out that it has something to do with compiler flags?

Quoted 4 lines

I don't know exactly in which situations that .eh_frame section is added, but looking at our selftests (and now samples/bpf as well), where we use -target bpf, we don't need -fno-asynchronous-unwind-tables at all.

Quoted 5 lines

This seems to at least be compiler-dependent. We ran into this with bpftool as well (for the internal BPF programs it loads whenever it runs), which already had '-target bpf' in the Makefile. We're carrying an internal RHEL patch adding -fno-asynchronous-unwind-tables to the bpftool build to fix this...

Quoted 4 lines

I haven't seen an instance of .eh_frame as well with -target bpf. Do you have a reproducible test case? I would like to investigate what is the possible cause and whether we could do something in llvm to prevent its generatin. Thanks!

We found this in the RHEL builds of bpftool. I don't think we're doing anything special, other than maybe building with a clang version that's a few versions behind:

# clang --version
clang version 11.0.0 (Red Hat 11.0.0-1.module+el8.4.0+8598+a071fcd5)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Quoted 4 lines
Quoted 1 line

So I suppose it may resolve itself once we upgrade LLVM?

That's odd. I don't think I've seen this issue even with clang 11 (but I built it myself).

I cannot reproduce it by self with self built llvm (11, 12, 13, 14). But I can reproduce it with an upstream built llvm12.

/bin/clang \
        -I. \
        -I/home/yhs/work/bpf-next/tools/include/uapi/ \
        -I/home/yhs/work/bpf-next/tools/lib/bpf/ \
        -I/home/yhs/work/bpf-next/tools/lib \
        -g -O2 -Wall -target bpf -c skeleton/pid_iter.bpf.c -o 

pid_iter.bpf.o && llvm-strip -g pid_iter.bpf.o
GEN pid_iter.skel.h
libbpf: elf: skipping unrecognized data section(11) .eh_frame
libbpf: elf: skipping relo section(12) .rel.eh_frame for section(11) .eh_frame

Quoted 7 lines

If there is a fix indeed let's backport it to llvm 11. The user experience matters.
It could be llvm configuration too.
I'm guessing some build flags might influence default settings for unwind tables.

Yonghong, can we make bpf backend to ignore needsUnwindTableEntry ?

Sure. I will try to get upstream build flags, reproduce and fix it in llvm.

#8 lore raw
Quoted 78 lines · Yonghong Song
Quoted 63 lines · Alexei Starovoitov
Quoted 11 lines · Toke Høiland-Jørgensen
Quoted 8 lines · Yonghong Song
Quoted 7 lines · Toke Høiland-Jørgensen
Quoted 5 lines · Andrii Nakryiko
Quoted 4 lines · Toke Høiland-Jørgensen

When .eh_frame and .rel.eh_frame sections are present in BPF object files, libbpf produces errors like this when loading the file:

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

libbpf: elf: skipping unrecognized data section(32) .eh_frame
libbpf: elf: skipping relo section(33) .rel.eh_frame for section(32) .eh_frame

It is possible to get rid of the .eh_frame section by adding -fno-asynchronous-unwind-tables to the compilation, but we have seen multiple examples of these sections appearing in BPF files in the wild, most recently in samples/bpf, fixed by:
5a0ae9872d5c ("bpf, samples: Add -fno-

/to BPF Clang invocation")

Quoted 42 lines
Quoted 27 lines
Quoted 21 lines
Quoted 8 lines
Quoted 2 lines

While the errors are technically harmless, they look odd and confuse users.

These warnings point out invalid set of compiler flags used for compiling BPF object files, though. Which is a good thing and should incentivize anyone getting those warnings to check and fix how they do BPF compilation. Those .eh_frame sections shouldn't be present in BPF object files at all, and that's what libbpf is trying to say.

Apart from triggering that warning, what effect does this have, though? The programs seem to work just fine (as evidenced by the fact that samples/bpf has been built this way for years, for instance)...

Also, how is a user supposed to go from that cryptic error message to figuring out that it has something to do with compiler flags?

Quoted 4 lines

I don't know exactly in which situations that .eh_frame section is added, but looking at our selftests (and now samples/bpf as well), where we use -target bpf, we don't need -fno-asynchronous-unwind-tables at all.

Quoted 5 lines

This seems to at least be compiler-dependent. We ran into this with bpftool as well (for the internal BPF programs it loads whenever it runs), which already had '-target bpf' in the Makefile. We're carrying an internal RHEL patch adding -fno-asynchronous-unwind-tables to the bpftool build to fix this...

Quoted 4 lines

I haven't seen an instance of .eh_frame as well with -target bpf. Do you have a reproducible test case? I would like to investigate what is the possible cause and whether we could do something in llvm to prevent its generatin. Thanks!

We found this in the RHEL builds of bpftool. I don't think we're doing anything special, other than maybe building with a clang version that's a few versions behind:

# clang --version
clang version 11.0.0 (Red Hat 11.0.0-1.module+el8.4.0+8598+a071fcd5)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Quoted 4 lines
Quoted 1 line

So I suppose it may resolve itself once we upgrade LLVM?

That's odd. I don't think I've seen this issue even with clang 11 (but I built it myself).

I cannot reproduce it by self with self built llvm (11, 12, 13, 14). But I can reproduce it with an upstream built llvm12.

/bin/clang \
         -I. \
         -I/home/yhs/work/bpf-next/tools/include/uapi/ \
         -I/home/yhs/work/bpf-next/tools/lib/bpf/ \
         -I/home/yhs/work/bpf-next/tools/lib \
         -g -O2 -Wall -target bpf -c skeleton/pid_iter.bpf.c -o 
Quoted 5 lines

pid_iter.bpf.o && llvm-strip -g pid_iter.bpf.o
GEN pid_iter.skel.h
libbpf: elf: skipping unrecognized data section(11) .eh_frame
libbpf: elf: skipping relo section(12) .rel.eh_frame for section(11) .eh_frame

Ah, that's interesting!

Quoted 10 lines
Quoted 7 lines

If there is a fix indeed let's backport it to llvm 11. The user experience matters.
It could be llvm configuration too.
I'm guessing some build flags might influence default settings for unwind tables.

Yonghong, can we make bpf backend to ignore needsUnwindTableEntry ?

Sure. I will try to get upstream build flags, reproduce and fix it in llvm.

Awesome, thanks for looking at this! :)

-Toke

#9 lore raw
Quoted 86 lines · Toke Høiland-Jørgensen
Quoted 78 lines · Yonghong Song
Quoted 63 lines · Alexei Starovoitov
Quoted 11 lines · Toke Høiland-Jørgensen
Quoted 8 lines · Yonghong Song
Quoted 7 lines · Toke Høiland-Jørgensen
Quoted 5 lines · Andrii Nakryiko
Quoted 4 lines · Toke Høiland-Jørgensen

When .eh_frame and .rel.eh_frame sections are present in BPF object files, libbpf produces errors like this when loading the file:

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

libbpf: elf: skipping unrecognized data section(32) .eh_frame
libbpf: elf: skipping relo section(33) .rel.eh_frame for section(32) .eh_frame

It is possible to get rid of the .eh_frame section by adding -fno-asynchronous-unwind-tables to the compilation, but we have seen multiple examples of these sections appearing in BPF files in the wild, most recently in samples/bpf, fixed by:
5a0ae9872d5c ("bpf, samples: Add -fno-

/to BPF Clang invocation")

Quoted 42 lines
Quoted 27 lines
Quoted 21 lines
Quoted 8 lines
Quoted 2 lines

While the errors are technically harmless, they look odd and confuse users.

These warnings point out invalid set of compiler flags used for compiling BPF object files, though. Which is a good thing and should incentivize anyone getting those warnings to check and fix how they do BPF compilation. Those .eh_frame sections shouldn't be present in BPF object files at all, and that's what libbpf is trying to say.

Apart from triggering that warning, what effect does this have, though? The programs seem to work just fine (as evidenced by the fact that samples/bpf has been built this way for years, for instance)...

Also, how is a user supposed to go from that cryptic error message to figuring out that it has something to do with compiler flags?

Quoted 4 lines

I don't know exactly in which situations that .eh_frame section is added, but looking at our selftests (and now samples/bpf as well), where we use -target bpf, we don't need -fno-asynchronous-unwind-tables at all.

Quoted 5 lines

This seems to at least be compiler-dependent. We ran into this with bpftool as well (for the internal BPF programs it loads whenever it runs), which already had '-target bpf' in the Makefile. We're carrying an internal RHEL patch adding -fno-asynchronous-unwind-tables to the bpftool build to fix this...

Quoted 4 lines

I haven't seen an instance of .eh_frame as well with -target bpf. Do you have a reproducible test case? I would like to investigate what is the possible cause and whether we could do something in llvm to prevent its generatin. Thanks!

We found this in the RHEL builds of bpftool. I don't think we're doing anything special, other than maybe building with a clang version that's a few versions behind:

# clang --version
clang version 11.0.0 (Red Hat 11.0.0-1.module+el8.4.0+8598+a071fcd5)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Quoted 4 lines
Quoted 1 line

So I suppose it may resolve itself once we upgrade LLVM?

That's odd. I don't think I've seen this issue even with clang 11 (but I built it myself).

I cannot reproduce it by self with self built llvm (11, 12, 13, 14). But I can reproduce it with an upstream built llvm12.

/bin/clang \
          -I. \
          -I/home/yhs/work/bpf-next/tools/include/uapi/ \
          -I/home/yhs/work/bpf-next/tools/lib/bpf/ \
          -I/home/yhs/work/bpf-next/tools/lib \
          -g -O2 -Wall -target bpf -c skeleton/pid_iter.bpf.c -o
Quoted 5 lines

pid_iter.bpf.o && llvm-strip -g pid_iter.bpf.o
GEN pid_iter.skel.h
libbpf: elf: skipping unrecognized data section(11) .eh_frame
libbpf: elf: skipping relo section(12) .rel.eh_frame for section(11) .eh_frame

Quoted 12 lines

Ah, that's interesting!

Quoted 10 lines
Quoted 7 lines

If there is a fix indeed let's backport it to llvm 11. The user experience matters.
It could be llvm configuration too.
I'm guessing some build flags might influence default settings for unwind tables.

Yonghong, can we make bpf backend to ignore needsUnwindTableEntry ?

Sure. I will try to get upstream build flags, reproduce and fix it in llvm.

I did some investigation and this is due to centos private patch:
https://git.centos.org/rpms/clang/blob/b99d8d4a38320329e10570f308c3e2d8cf295c78/f/SOURCES/0002-PATCH-clang-Make-funwind-tables-the-default-on-all-a.patch

In upstream, the original llvm-project source is patched with several private patches before building the rpm.
https://koji.mbox.centos.org/pkgs/packages/clang/12.0.1/1.module_el8.5.0+892+54d791e1/data/logs/x86_64/build.log

The above private patch enables unwind-table (.eh_frame section) by default for ALL architectures and bpf is a victim of this.

I filed a redhat bugzilla bug to fix their private patch.

https://bugzilla.redhat.com/show_bug.cgi?id=2002024

Hopefully future newer compiler build won't have this issue.

Quoted 5 lines

Awesome, thanks for looking at this! :)

-Toke

#10 lore raw
Quoted 103 lines · Yonghong Song
Quoted 86 lines · Toke Høiland-Jørgensen
Quoted 78 lines · Yonghong Song
Quoted 63 lines · Alexei Starovoitov
Quoted 11 lines · Toke Høiland-Jørgensen
Quoted 8 lines · Yonghong Song
Quoted 7 lines · Toke Høiland-Jørgensen
Quoted 5 lines · Andrii Nakryiko
Quoted 4 lines · Toke Høiland-Jørgensen

When .eh_frame and .rel.eh_frame sections are present in BPF object files, libbpf produces errors like this when loading the file:

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

libbpf: elf: skipping unrecognized data section(32) .eh_frame
libbpf: elf: skipping relo section(33) .rel.eh_frame for section(32) .eh_frame

It is possible to get rid of the .eh_frame section by adding -fno-asynchronous-unwind-tables to the compilation, but we have seen multiple examples of these sections appearing in BPF files in the wild, most recently in samples/bpf, fixed by:
5a0ae9872d5c ("bpf, samples: Add -fno-

/to BPF Clang invocation")

Quoted 42 lines
Quoted 27 lines
Quoted 21 lines
Quoted 8 lines
Quoted 2 lines

While the errors are technically harmless, they look odd and confuse users.

These warnings point out invalid set of compiler flags used for compiling BPF object files, though. Which is a good thing and should incentivize anyone getting those warnings to check and fix how they do BPF compilation. Those .eh_frame sections shouldn't be present in BPF object files at all, and that's what libbpf is trying to say.

Apart from triggering that warning, what effect does this have, though? The programs seem to work just fine (as evidenced by the fact that samples/bpf has been built this way for years, for instance)...

Also, how is a user supposed to go from that cryptic error message to figuring out that it has something to do with compiler flags?

Quoted 4 lines

I don't know exactly in which situations that .eh_frame section is added, but looking at our selftests (and now samples/bpf as well), where we use -target bpf, we don't need -fno-asynchronous-unwind-tables at all.

Quoted 5 lines

This seems to at least be compiler-dependent. We ran into this with bpftool as well (for the internal BPF programs it loads whenever it runs), which already had '-target bpf' in the Makefile. We're carrying an internal RHEL patch adding -fno-asynchronous-unwind-tables to the bpftool build to fix this...

Quoted 4 lines

I haven't seen an instance of .eh_frame as well with -target bpf. Do you have a reproducible test case? I would like to investigate what is the possible cause and whether we could do something in llvm to prevent its generatin. Thanks!

We found this in the RHEL builds of bpftool. I don't think we're doing anything special, other than maybe building with a clang version that's a few versions behind:

# clang --version
clang version 11.0.0 (Red Hat 11.0.0-1.module+el8.4.0+8598+a071fcd5)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Quoted 4 lines
Quoted 1 line

So I suppose it may resolve itself once we upgrade LLVM?

That's odd. I don't think I've seen this issue even with clang 11 (but I built it myself).

I cannot reproduce it by self with self built llvm (11, 12, 13, 14). But I can reproduce it with an upstream built llvm12.

/bin/clang \
          -I. \
          -I/home/yhs/work/bpf-next/tools/include/uapi/ \
          -I/home/yhs/work/bpf-next/tools/lib/bpf/ \
          -I/home/yhs/work/bpf-next/tools/lib \
          -g -O2 -Wall -target bpf -c skeleton/pid_iter.bpf.c -o
Quoted 5 lines

pid_iter.bpf.o && llvm-strip -g pid_iter.bpf.o
GEN pid_iter.skel.h
libbpf: elf: skipping unrecognized data section(11) .eh_frame
libbpf: elf: skipping relo section(12) .rel.eh_frame for section(11) .eh_frame

Quoted 12 lines

Ah, that's interesting!

Quoted 10 lines
Quoted 7 lines

If there is a fix indeed let's backport it to llvm 11. The user experience matters.
It could be llvm configuration too.
I'm guessing some build flags might influence default settings for unwind tables.

Yonghong, can we make bpf backend to ignore needsUnwindTableEntry ?

Sure. I will try to get upstream build flags, reproduce and fix it in llvm.

I did some investigation and this is due to centos private patch:
https://git.centos.org/rpms/clang/blob/b99d8d4a38320329e10570f308c3e2d8cf295c78/f/SOURCES/0002-PATCH-clang-Make-funwind-tables-the-default-on-all-a.patch

Quoted 6 lines

In upstream, the original llvm-project source is patched with several private patches before building the rpm.
https://koji.mbox.centos.org/pkgs/packages/clang/12.0.1/1.module_el8.5.0+892+54d791e1/data/logs/x86_64/build.log

The above private patch enables unwind-table (.eh_frame section) by default for ALL architectures and bpf is a victim of this.

Ah, doh! I had no idea we were doing this :/

Quoted 5 lines

I filed a redhat bugzilla bug to fix their private patch.

https://bugzilla.redhat.com/show_bug.cgi?id=2002024

Hopefully future newer compiler build won't have this issue.

Thank you for finding the root cause of this! I'll follow up internally and make sure we get this fixed...

-Toke

#11 lore raw
Quoted 114 lines · Toke Høiland-Jørgensen
Quoted 103 lines · Yonghong Song
Quoted 86 lines · Toke Høiland-Jørgensen
Quoted 78 lines · Yonghong Song
Quoted 63 lines · Alexei Starovoitov
Quoted 11 lines · Toke Høiland-Jørgensen
Quoted 8 lines · Yonghong Song
Quoted 7 lines · Toke Høiland-Jørgensen
Quoted 5 lines · Andrii Nakryiko
Quoted 4 lines · Toke Høiland-Jørgensen

When .eh_frame and .rel.eh_frame sections are present in BPF object files, libbpf produces errors like this when loading the file:

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

libbpf: elf: skipping unrecognized data section(32) .eh_frame
libbpf: elf: skipping relo section(33) .rel.eh_frame for section(32) .eh_frame

It is possible to get rid of the .eh_frame section by adding -fno-asynchronous-unwind-tables to the compilation, but we have seen multiple examples of these sections appearing in BPF files in the wild, most recently in samples/bpf, fixed by:
5a0ae9872d5c ("bpf, samples: Add -fno-

/to BPF Clang invocation")

Quoted 42 lines
Quoted 27 lines
Quoted 21 lines
Quoted 8 lines
Quoted 2 lines

While the errors are technically harmless, they look odd and confuse users.

These warnings point out invalid set of compiler flags used for compiling BPF object files, though. Which is a good thing and should incentivize anyone getting those warnings to check and fix how they do BPF compilation. Those .eh_frame sections shouldn't be present in BPF object files at all, and that's what libbpf is trying to say.

Apart from triggering that warning, what effect does this have, though? The programs seem to work just fine (as evidenced by the fact that samples/bpf has been built this way for years, for instance)...

Also, how is a user supposed to go from that cryptic error message to figuring out that it has something to do with compiler flags?

Quoted 4 lines

I don't know exactly in which situations that .eh_frame section is added, but looking at our selftests (and now samples/bpf as well), where we use -target bpf, we don't need -fno-asynchronous-unwind-tables at all.

Quoted 5 lines

This seems to at least be compiler-dependent. We ran into this with bpftool as well (for the internal BPF programs it loads whenever it runs), which already had '-target bpf' in the Makefile. We're carrying an internal RHEL patch adding -fno-asynchronous-unwind-tables to the bpftool build to fix this...

Quoted 4 lines

I haven't seen an instance of .eh_frame as well with -target bpf. Do you have a reproducible test case? I would like to investigate what is the possible cause and whether we could do something in llvm to prevent its generatin. Thanks!

We found this in the RHEL builds of bpftool. I don't think we're doing anything special, other than maybe building with a clang version that's a few versions behind:

# clang --version
clang version 11.0.0 (Red Hat 11.0.0-1.module+el8.4.0+8598+a071fcd5)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Quoted 4 lines
Quoted 1 line

So I suppose it may resolve itself once we upgrade LLVM?

That's odd. I don't think I've seen this issue even with clang 11 (but I built it myself).

I cannot reproduce it by self with self built llvm (11, 12, 13, 14). But I can reproduce it with an upstream built llvm12.

/bin/clang \
           -I. \
           -I/home/yhs/work/bpf-next/tools/include/uapi/ \
           -I/home/yhs/work/bpf-next/tools/lib/bpf/ \
           -I/home/yhs/work/bpf-next/tools/lib \
           -g -O2 -Wall -target bpf -c skeleton/pid_iter.bpf.c -o
Quoted 5 lines

pid_iter.bpf.o && llvm-strip -g pid_iter.bpf.o
GEN pid_iter.skel.h
libbpf: elf: skipping unrecognized data section(11) .eh_frame
libbpf: elf: skipping relo section(12) .rel.eh_frame for section(11) .eh_frame

Quoted 12 lines

Ah, that's interesting!

Quoted 10 lines
Quoted 7 lines

If there is a fix indeed let's backport it to llvm 11. The user experience matters.
It could be llvm configuration too.
I'm guessing some build flags might influence default settings for unwind tables.

Yonghong, can we make bpf backend to ignore needsUnwindTableEntry ?

Sure. I will try to get upstream build flags, reproduce and fix it in llvm.

I did some investigation and this is due to centos private patch:
https://git.centos.org/rpms/clang/blob/b99d8d4a38320329e10570f308c3e2d8cf295c78/f/SOURCES/0002-PATCH-clang-Make-funwind-tables-the-default-on-all-a.patch

Quoted 6 lines

In upstream, the original llvm-project source is patched with several private patches before building the rpm.
https://koji.mbox.centos.org/pkgs/packages/clang/12.0.1/1.module_el8.5.0+892+54d791e1/data/logs/x86_64/build.log

The above private patch enables unwind-table (.eh_frame section) by default for ALL architectures and bpf is a victim of this.

Ah, doh! I had no idea we were doing this :/

Quoted 8 lines
Quoted 5 lines

I filed a redhat bugzilla bug to fix their private patch.

https://bugzilla.redhat.com/show_bug.cgi?id=2002024

Hopefully future newer compiler build won't have this issue.

Thank you for finding the root cause of this! I'll follow up internally and make sure we get this fixed...

Thanks! Hopefully this can be resolved soon.

Quoted 3 lines

-Toke