bpf: Use named fields for certain bpf uapi structs
Yonghong Song
reviewerPATCH 1/1 bpf: Use named fields for certain bpf uapi structs
Martin and Vadim reported a verifier failure with bpf_dynptr usage. The issue is mentioned but Vadim workarounded the issue with source change ([1]). The below describes what is the issue and why there is a verification failure.
int BPF_PROG(skb_crypto_setup) {
struct bpf_dynptr algo, key;
... bpf_dynptr_from_mem(..., ..., 0, &algo);
...}
The bpf program is using vmlinux.h, so we have the following definition in vmlinux.h:
struct bpf_dynptr {
long: 64;
long: 64;
};
Note that in uapi header bpf.h, we have
struct bpf_dynptr {
long: 64;
long: 64;
} __attribute__((aligned(8)));So we lost alignment information for struct bpf_dynptr by using vmlinux.h. Let us take a look at a simple program below:
$ cat align.c
typedef unsigned long long __u64;
struct bpf_dynptr_no_align {
__u64 :64;
__u64 :64;
};
struct bpf_dynptr_yes_align {
__u64 :64;
__u64 :64;
} __attribute__((aligned(8))); void bar(void *, void *);
int foo() {
struct bpf_dynptr_no_align a;
struct bpf_dynptr_yes_align b;
bar(&a, &b);
return 0;
}
$ clang --target=bpf -O2 -S -emit-llvm align.cLook at the generated IR file align.ll:
...
%a = alloca %struct.bpf_dynptr_no_align, align 1 %b = alloca %struct.bpf_dynptr_yes_align, align 8
...
The compiler dictates the alignment for struct bpf_dynptr_no_align is 1 and the alignment for struct bpf_dynptr_yes_align is 8. So theoretically compiler could allocate variable %a with alignment 1 although in reallity the compiler may choose a different alignment by considering other local variables.
In [1], the verification failure happens because variable 'algo' is allocated on the stack with alignment 4 (fp-28). But the verifer wants its alignment to be 8.
To fix the issue, the RFC patch ([1]) tried to add '__attribute__((aligned(8)))' to struct bpf_dynptr plus other similar structs. Andrii suggested that we could directly modify uapi struct with named fields like struct 'bpf_iter_num':
struct bpf_iter_num {
/* opaque iterator state; having __u64 here allows to preserve correct
* alignment requirements in vmlinux.h, generated from BTF
*/
__u64 __opaque[1];} __attribute__((aligned(8)));
Indeed, adding named fields for those affected structs in this patch can preserve alignment when bpf program references them in vmlinux.h. With this patch, the verification failure in [1] can also be resolved.
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
--- include/uapi/linux/bpf.h | 23 +++++++---------------- tools/include/uapi/linux/bpf.h | 23 +++++++---------------- 2 files changed, 14 insertions(+), 32 deletions(-)
include/uapi/linux/bpf.h +7 -16
@@ -7151,40 +7151,31 @@ struct bpf_spin_lock { };struct bpf_timer { - __u64 :64; - __u64 :64; +__u64 __opaque[2 ]; } __attribute__((aligned (8 )));struct bpf_dynptr { - __u64 :64; - __u64 :64; +__u64 __opaque[2 ]; } __attribute__((aligned (8 )));struct bpf_list_head { - __u64 :64; - __u64 :64; +__u64 __opaque[2 ]; } __attribute__((aligned (8 )));struct bpf_list_node { - __u64 :64; - __u64 :64; - __u64 :64; +__u64 __opaque[3 ]; } __attribute__((aligned (8 )));struct bpf_rb_root { - __u64 :64; - __u64 :64; +__u64 __opaque[2 ]; } __attribute__((aligned (8 )));struct bpf_rb_node { - __u64 :64; - __u64 :64; - __u64 :64; - __u64 :64; +__u64 __opaque[4 ]; } __attribute__((aligned (8 )));struct bpf_refcount { - __u32 :32; +__u32 __opaque[1 ]; } __attribute__((aligned (4 )));struct bpf_sysctl {
tools/include/uapi/linux/bpf.h +7 -16
@@ -7151,40 +7151,31 @@ struct bpf_spin_lock { };struct bpf_timer { - __u64 :64; - __u64 :64; +__u64 __opaque[2 ]; } __attribute__((aligned (8 )));struct bpf_dynptr { - __u64 :64; - __u64 :64; +__u64 __opaque[2 ]; } __attribute__((aligned (8 )));struct bpf_list_head { - __u64 :64; - __u64 :64; +__u64 __opaque[2 ]; } __attribute__((aligned (8 )));struct bpf_list_node { - __u64 :64; - __u64 :64; - __u64 :64; +__u64 __opaque[3 ]; } __attribute__((aligned (8 )));struct bpf_rb_root { - __u64 :64; - __u64 :64; +__u64 __opaque[2 ]; } __attribute__((aligned (8 )));struct bpf_rb_node { - __u64 :64; - __u64 :64; - __u64 :64; - __u64 :64; +__u64 __opaque[4 ]; } __attribute__((aligned (8 )));struct bpf_refcount { - __u32 :32; +__u32 __opaque[1 ]; } __attribute__((aligned (4 )));struct bpf_sysctl {
Signature · 1 line
2.34.1
Quoted 80 lines · Yonghong Song
Martin and Vadim reported a verifier failure with
bpf_dynptrusage. The issue is mentioned but Vadim workarounded the issue with source change ([1]). The below describes what is the issue and why there is a verification failure.int BPF_PROG(skb_crypto_setup) { struct bpf_dynptr algo, key; ...bpf_dynptr_from_mem(..., ..., 0, &algo); ...}
The bpf program is using
vmlinux.h, so we have the following definition invmlinux.h:struct bpf_dynptr { long: 64; long: 64; }; Note that in uapi header bpf.h, we have struct bpf_dynptr { long: 64; long: 64; } __attribute__((aligned(8)));So we lost alignment information for struct
bpf_dynptrby usingvmlinux.h. Let us take a look at a simple program below:$ cat align.c typedef unsigned long long __u64; struct bpf_dynptr_no_align { __u64 :64; __u64 :64; }; struct bpf_dynptr_yes_align { __u64 :64; __u64 :64; } __attribute__((aligned(8)));void bar(void *, void *); int foo() { struct bpf_dynptr_no_align a; struct bpf_dynptr_yes_align b; bar(&a, &b); return 0; } $ clang --target=bpf -O2 -S -emit-llvm align.cLook at the generated IR file align.ll:
...
%a = alloca %struct.bpf_dynptr_no_align, align 1 %b = alloca %struct.bpf_dynptr_yes_align, align 8
...The compiler dictates the alignment for struct
bpf_dynptr_no_alignis 1 and the alignment for structbpf_dynptr_yes_alignis 8. So theoretically compiler could allocate variable %a with alignment 1 although in reallity the compiler may choose a different alignment by considering other local variables.In [1], the verification failure happens because variable 'algo' is allocated on the stack with alignment 4 (fp-28). But the verifer wants its alignment to be 8.
To fix the issue, the RFC patch ([1]) tried to add '__attribute__((aligned(8)))' to struct
bpf_dynptrplus other similar structs. Andrii suggested that we could directly modify uapi struct with named fields like struct 'bpf_iter_num':struct bpf_iter_num { /* opaque iterator state; having __u64 here allows to preserve correct * alignment requirements in vmlinux.h, generated from BTF */ __u64 __opaque[1];} __attribute__((aligned(8)));
Indeed, adding named fields for those affected structs in this patch can preserve alignment when bpf program references them in
vmlinux.h. With this patch, the verification failure in [1] can also be resolved.[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
Quoted 9 lines
Cc Vadim Fedorenko <[email protected]>Cc Martin KaFai Lau <[email protected]>Suggested-by Andrii Nakryiko <[email protected]>Signed-off-by Yonghong Song <[email protected]>--- include/uapi/linux/bpf.h | 23 +++++++---------------- tools/include/uapi/linux/bpf.h | 23 +++++++---------------- 2 files changed, 14 insertions(+), 32 deletions(-)
I think that's the best solution, thanks!
[...]
This patch was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This patch was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 10 lines · On Fri, 3 Nov 2023 19:49:00 -0700 you wrote:
Martin and Vadim reported a verifier failure with
bpf_dynptrusage. The issue is mentioned but Vadim workarounded the issue with source change ([1]). The below describes what is the issue and why there is a verification failure.int BPF_PROG(skb_crypto_setup) { struct bpf_dynptr algo, key; ...[...]
Here is the summary with links:
- [bpf-next] bpf: Use named fields for certain bpf uapi structs
https://git.kernel.org/bpf/bpf-next/c/e80742d91749
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html