bpf: net: emit anonymous enum with BPF_TCP_CLOSE value explicitly
PATCH v2 1/1 bpf: net: emit anonymous enum with BPF_TCP_CLOSE value explicitly
The selftest failed to compile with clang-built bpf-next.
Adding LLVM=1 to your vmlinux and selftest build will use clang.
The error message is:
progs/test_sk_storage_tracing.c:38:18: error: use of undeclared identifier 'BPF_TCP_CLOSE'
if (newstate == BPF_TCP_CLOSE)
^ 1 error generated.
make: *** [Makefile:423: /bpf-next/tools/testing/selftests/bpf/test_sk_storage_tracing.o] Error 1
The reason for the failure is that BPF_TCP_CLOSE, a value of an anonymous enum defined in uapi bpf.h, is not defined in vmlinux.h. gcc does not have this problem. Since vmlinux.h is derived from BTF which is derived from vmlinux DWARF, that means gcc-produced vmlinux DWARF has BPF_TCP_CLOSE while llvm-produced vmlinux DWARF does not have.
BPF_TCP_CLOSE is referenced in net/ipv4/tcp.c as
BUILD_BUG_ON((int)BPF_TCP_CLOSE != (int)TCP_CLOSE);
The following test mimics the above BUILD_BUG_ON, preprocessed with clang compiler, and shows gcc DWARF contains BPF_TCP_CLOSE while llvm DWARF does not.
$ cat t.c
enum {
BPF_TCP_ESTABLISHED = 1,
BPF_TCP_CLOSE = 7,
};
enum {
TCP_ESTABLISHED = 1,
TCP_CLOSE = 7,
}; int test() {
do {
extern void __compiletime_assert_767(void) ;
if ((int)BPF_TCP_CLOSE != (int)TCP_CLOSE) __compiletime_assert_767();
} while (0);
return 0;
}
$ clang t.c -O2 -c -g && llvm-dwarfdump t.o | grep BPF_TCP_CLOSE
$ gcc t.c -O2 -c -g && llvm-dwarfdump t.o | grep BPF_TCP_CLOSE
DW_AT_name ("BPF_TCP_CLOSE")Further checking clang code find clang actually tried to evaluate condition at compile time. If it is definitely true/false, it will perform optimization and the whole if condition will be removed before generating IR/debuginfo.
This patch explicited add an expression like
(void)BPF_TCP_ESTABLISHED
to enable generation of debuginfo for the anonymous enum which also includes BPF_TCP_CLOSE. I put
this explicit type generation in kernel/bpf/core.c to (1) avoid polute net/ipv4/tcp.c and more importantly
(2) provide a central place to add other types (e.g. in bpf/btf uapi header) if they are not referenced in the kernel or generated in vmlinux DWARF.
--- include/linux/btf.h | 1 + kernel/bpf/core.c | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+)
Changelog:
v1 -> v2:
use DWARF instead of dwarf for better consistency with other usages. (Andrii)
include/linux/btf.h +1 -0
@@ -9,6 +9,7 @@#include <uapi/linux/bpf.h> #define BTF_TYPE_EMIT (type) ((void)(type *)0) +#define BTF_TYPE_EMIT_ENUM (enum_val) ((void)enum_val)struct btf ;struct btf_member ;
kernel/bpf/core.c +19 -0
@@ -2378,3 +2378,22 @@ EXPORT_SYMBOL(bpf_stats_enabled_key);EXPORT_TRACEPOINT_SYMBOL_GPL (xdp_exception);EXPORT_TRACEPOINT_SYMBOL_GPL (xdp_bulk_tx); + +static int __init bpf_emit_btf_type (void ) +{ +/* bpf uapi header bpf.h defines an anonymous enum with values +* BPF_TCP_* used by bpf programs. Currently gcc built vmlinux +* is able to emit this enum in DWARF due to the following +* BUILD_BUG_ON test in net/ipv4/tcp.c: +* BUILD_BUG_ON((int)BPF_TCP_ESTABLISHED != (int)TCP_ESTABLISHED); +* clang built vmlinux does not have this enum in DWARF +* since clang removes the above code before generating IR/debuginfo. +* Let us explicitly emit the type debuginfo to ensure the +* above-mentioned anonymous enum in the vmlinux DWARF and hence BTF +* regardless of which compiler is used. +*/ +BTF_TYPE_EMIT_ENUM (BPF_TCP_ESTABLISHED ); + +return 0 ; +} +late_initcall (bpf_emit_btf_type);
Signature · 1 line
2.30.2
kernel/bpf/core.c EXPORT_SYMBOL(bpf_stats_enabled_key); · line 2399
Quoted 5 lines · Yonghong Song
kernel/bpf/core.c +5
-0+BTF_TYPE_EMIT_ENUM (BPF_TCP_ESTABLISHED ); + +return 0 ; +} +late_initcall (bpf_emit_btf_type);
I think if we burn a dummy function on this it would be a wrong pattern to follow.
This is just a nop C statement.
Typically we add BUILD_BUG_ON near the places that rely on that constraint. There is such a function already. It's tcp_set_state() as you pointed out. It's not using BTF of course, but I would move above BTF_TYPE_EMIT_ENUM there. I'm not sure why you're calling it "pollute net/ipv4/tcp.c". Consider that BTF_TYPE_EMIT macro is serving similar purpose
and it's scattered around net/core/filter.c
Quoted 9 lines · Alexei Starovoitov
kernel/bpf/core.c EXPORT_SYMBOL(bpf_stats_enabled_key); · line 2399
Quoted 5 lines · Yonghong Song
kernel/bpf/core.c +5
-0+BTF_TYPE_EMIT_ENUM (BPF_TCP_ESTABLISHED ); + +return 0 ; +} +late_initcall (bpf_emit_btf_type);I think if we burn a dummy function on this it would be a wrong pattern to follow.
Maybe we can pick another initcall to piggyback?
Quoted 5 lines
This is just a nop C statement.
Typically we addBUILD_BUG_ONnear the places that rely on that constraint. There is such a function already. It'stcp_set_state()as you pointed out. It's not using BTF of course, but I would move aboveBTF_TYPE_EMIT_ENUMthere. I'm not sure why you're calling it "pollutenet/ipv4/tcp.c".
This is the minor reason. I first coded in that place and feel awkward where we have macro referenced above and we still emit a BTF_TYPE_EMIT below although with some comments.
The major reason is I think we may have some uapi type/enum's (e.g., in uapi/linux/bpf.h) which will be used in bpf program but not in kernel itself. So we cannot generate types in vmlinux btf because of this. So I
used this case to find a place to generate these btf types.BPF_TCP_CLOSE is actually such an example, it happens we have a BUILD_BUG_ON in kernel to access it.
Maybe I am too forward looking?
Quoted 2 lines
Consider that
BTF_TYPE_EMITmacro is serving similar purpose and it's scattered aroundnet/core/filter.c
Quoted 26 lines · Yonghong Song
Quoted 9 lines · Alexei Starovoitov
kernel/bpf/core.c EXPORT_SYMBOL(bpf_stats_enabled_key); · line 2399
Quoted 5 lines · Yonghong Song
kernel/bpf/core.c +5
-0+BTF_TYPE_EMIT_ENUM (BPF_TCP_ESTABLISHED ); + +return 0 ; +} +late_initcall (bpf_emit_btf_type);I think if we burn a dummy function on this it would be a wrong pattern to follow.
Maybe we can pick another initcall to piggyback?
Quoted 5 lines
This is just a nop C statement.
Typically we addBUILD_BUG_ONnear the places that rely on that constraint. There is such a function already. It'stcp_set_state()as you pointed out. It's not using BTF of course, but I would move aboveBTF_TYPE_EMIT_ENUMthere. I'm not sure why you're calling it "pollutenet/ipv4/tcp.c".This is the minor reason. I first coded in that place and feel awkward where we have macro referenced above and we still emit a
BTF_TYPE_EMITbelow although with some comments.
Quoted 7 lines
The major reason is I think we may have some uapi type/enum's (e.g., in
uapi/linux/bpf.h) which will be used in bpf program but not in kernel itself. So we cannot generate types in vmlinux btf because of this. So I used this case to find a place to generate these btf types.BPF_TCP_CLOSEis actually such an example, it happens we have aBUILD_BUG_ONin kernel to access it.
Maybe I am too forward looking?
It's great to be forward looking :)
I'm just having a hard time justifying an empty function with single 'ret' insn that actually will be called at init time and it will stay empty like this for foreseeable future. Static analysis tools and whatnot will start sending patches to remove that empty function.
Daniel Borkmann
maintainerQuoted 32 lines · Alexei Starovoitov
Quoted 23 lines · Yonghong Song
Quoted 9 lines · Alexei Starovoitov
kernel/bpf/core.c EXPORT_SYMBOL(bpf_stats_enabled_key); · line 2399
Quoted 5 lines · Yonghong Song
kernel/bpf/core.c +5
-0+BTF_TYPE_EMIT_ENUM (BPF_TCP_ESTABLISHED ); + +return 0 ; +} +late_initcall (bpf_emit_btf_type);I think if we burn a dummy function on this it would be a wrong pattern to follow.
Maybe we can pick another initcall to piggyback?
Quoted 5 lines
This is just a nop C statement.
Typically we addBUILD_BUG_ONnear the places that rely on that constraint. There is such a function already. It'stcp_set_state()as you pointed out. It's not using BTF of course, but I would move aboveBTF_TYPE_EMIT_ENUMthere. I'm not sure why you're calling it "pollutenet/ipv4/tcp.c".This is the minor reason. I first coded in that place and feel awkward where we have macro referenced above and we still emit a
BTF_TYPE_EMITbelow although with some comments.Quoted 7 lines
The major reason is I think we may have some uapi type/enum's (e.g., in
uapi/linux/bpf.h) which will be used in bpf program but not in kernel itself. So we cannot generate types in vmlinux btf because of this. So I used this case to find a place to generate these btf types.BPF_TCP_CLOSEis actually such an example, it happens we have aBUILD_BUG_ONin kernel to access it.
Maybe I am too forward looking?
Quoted 5 lines
It's great to be forward looking :)
I'm just having a hard time justifying an empty function with single 'ret' insn that actually will be called at init time and it will stay empty like this for foreseeable future. Static analysis tools and whatnot will start sending patches to remove that empty function.
+1, even empty function exported as symbol for modules (to avoid compiler optimization) might be better than initcall, imho.
Quoted 35 lines · Alexei Starovoitov
Quoted 26 lines · Yonghong Song
Quoted 9 lines · Alexei Starovoitov
kernel/bpf/core.c EXPORT_SYMBOL(bpf_stats_enabled_key); · line 2399
Quoted 5 lines · Yonghong Song
kernel/bpf/core.c +5
-0+BTF_TYPE_EMIT_ENUM (BPF_TCP_ESTABLISHED ); + +return 0 ; +} +late_initcall (bpf_emit_btf_type);I think if we burn a dummy function on this it would be a wrong pattern to follow.
Maybe we can pick another initcall to piggyback?
Quoted 5 lines
This is just a nop C statement.
Typically we addBUILD_BUG_ONnear the places that rely on that constraint. There is such a function already. It'stcp_set_state()as you pointed out. It's not using BTF of course, but I would move aboveBTF_TYPE_EMIT_ENUMthere. I'm not sure why you're calling it "pollutenet/ipv4/tcp.c".This is the minor reason. I first coded in that place and feel awkward where we have macro referenced above and we still emit a
BTF_TYPE_EMITbelow although with some comments.Quoted 7 lines
The major reason is I think we may have some uapi type/enum's (e.g., in
uapi/linux/bpf.h) which will be used in bpf program but not in kernel itself. So we cannot generate types in vmlinux btf because of this. So I used this case to find a place to generate these btf types.BPF_TCP_CLOSEis actually such an example, it happens we have aBUILD_BUG_ONin kernel to access it.
Maybe I am too forward looking?
Quoted 5 lines
It's great to be forward looking :)
I'm just having a hard time justifying an empty function with single 'ret' insn that actually will be called at init time and it will stay empty like this for foreseeable future. Static analysis tools and whatnot will start sending patches to remove that empty function.
Okay, will go back to net/ipv4/tcp.c approach. We can address missing uapi type issue later if it ever comes up. Thanks!