bpf: fix a bpf_timer initialization issue
v4 cover bpf: fix a bpf_timer initialization issue
The patch [1] exposed a bpf_timer initialization bug in function check_and_init_map_value(). With bug fix here, the patch [1] can be applied with all selftests passed. Please see individual patches for fix details.
[1] https://lore.kernel.org/bpf/[email protected]/
Changelog:
v3 -> v4:
. move header file in patch #1 to avoid bpf-next merge conflict
v2 -> v3:
. switch patch #1 and patch #2 for better bisecting
v1 -> v2:
. add Fixes tag for patch #1
. rebase against bpf tree
Yonghong Song (2):
bpf: emit bpf_timer in vmlinux BTF
bpf: fix a bpf_timer initialization issue
include/linux/bpf.h | 6 ++---- kernel/bpf/helpers.c | 2 ++ 2 files changed, 4 insertions(+), 4 deletions(-)
Signature · 1 line
2.30.2
PATCH v4 1/2 bpf: emit bpf_timer in vmlinux BTF
Currently the following code in check_and_init_map_value()
*(struct bpf_timer *)(dst + map->timer_off) =
(struct bpf_timer){};can help generate bpf_timer definition in vmlinuxBTF. But the code above may not zero the whole structure due to anonymour members and that code will be replaced by memset in the subsequent patch andbpf_timer definition will disappear from vmlinuxBTF. Let us emit the type explicitly so bpf program can continue to use it from vmlinux.h.
--- kernel/bpf/helpers.c | 2 ++ 1 file changed, 2 insertions(+)
kernel/bpf/helpers.c +2 -0
@@ -2,6 +2,7 @@/* Copyright (c) 2011-2014 PLUMgrid, http://plumgrid.com */ #include <linux/bpf.h> +#include <linux/btf.h> #include <linux/bpf-cgroup.h> #include <linux/rcupdate.h> #include <linux/random.h>
@@ -1075,6 +1076,7 @@ static enum hrtimer_restart bpf_timer_cb(struct hrtimer *hrtimer)void *key;u32 idx; +BTF_TYPE_EMIT (struct bpf_timer ); callback_fn =rcu_dereference_check (t->callback_fn,rcu_read_lock_bh_held ());if (!callback_fn) goto out;
Signature · 1 line
2.30.2
PATCH v4 2/2 bpf: fix a bpf_timer initialization issue
In reply to Yonghong SongThe patch in [1] intends to fix a bpf_timer related issue, but the fix caused existing 'timer' selftest to fail with hang or some random errors. After some debug, I found an issue with check_and_init_map_value() in the hashtab.c. More specifically, in hashtab.c, we have code
l_new = bpf_map_kmalloc_node(&htab->map, ...) check_and_init_map_value(&htab->map, l_new...)
Note that bpf_map_kmalloc_node() does not do initialization so l_new contains random value.
The function check_and_init_map_value() intends to zero the bpf_spin_lock and bpf_timer if they exist in the map. But I found bpf_spin_lock is zero'ed but bpf_timer is not zero'ed. With [1], later copy_map_value() skips copying of bpf_spin_lock and bpf_timer. The non-zero bpf_timer caused random failures for 'timer' selftest.
Without [1], for both bpf_spin_lock and bpf_timer case, bpf_timer will be zero'ed, so 'timer' self test is okay.
For check_and_init_map_value(), why bpf_spin_lock is zero'ed properly while bpf_timer not. In bpf uapi header, we have
struct bpf_spin_lock {
__u32 val;
};
struct bpf_timer {
__u64 :64;
__u64 :64;} __attribute__((aligned(8)));
The initialization code:
*(struct bpf_spin_lock *)(dst + map->spin_lock_off) =
(struct bpf_spin_lock){};
*(struct bpf_timer *)(dst + map->timer_off) =
(struct bpf_timer){};
It appears the compiler has no obligation to initialize anonymous fields.
For example, let us use clang with bpf target as below:
$ cat t.c
struct bpf_timer {
unsigned long long :64;
};
struct bpf_timer2 {
unsigned long long a;
}; void test(struct bpf_timer *t) {
*t = (struct bpf_timer){};
}
void test2(struct bpf_timer2 *t) {
*t = (struct bpf_timer2){};
}
$ clang -target bpf -O2 -c -g t.c
$ llvm-objdump -d t.o
...
0000000000000000 <test>:
0: 95 00 00 00 00 00 00 00 exit
0000000000000008 <test2>:
1: b7 02 00 00 00 00 00 00 r2 = 0
2: 7b 21 00 00 00 00 00 00 *(u64 *)(r1 + 0) = r2
3: 95 00 00 00 00 00 00 00 exitgcc11.2 does not have the above issue. But from
INTERNATIONAL STANDARD ©ISO/IEC ISO/IEC 9899:201x Programming languages — C
http://www.open-std.org/Jtc1/sc22/wg14/www/docs/n1547.pdf
page 157:
Except where explicitly stated otherwise, for the purposes of this subclause unnamed members of objects of structure and union type do not participate in initialization. Unnamed members of structure objects have indeterminate value even after initialization.
To fix the problem, let use memset for bpf_timer case in check_and_init_map_value(). For consistency, memset is also used for bpf_spin_lock case.
[1] https://lore.kernel.org/bpf/[email protected]/
--- include/linux/bpf.h | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
include/linux/bpf.h +2 -4
@@ -209,11 +209,9 @@ static inline bool map_value_has_timer(const struct bpf_map *map)static inline void check_and_init_map_value (struct bpf_map *map,void *dst) {if (unlikely (map_value_has_spin_lock (map))) - *(struct bpf_spin_lock *)(dst + map->spin_lock_off) = - (struct bpf_spin_lock ){}; +memset (dst + map->spin_lock_off,0 ,sizeof (struct bpf_spin_lock ));if (unlikely (map_value_has_timer (map))) - *(struct bpf_timer *)(dst + map->timer_off) = - (struct bpf_timer ){}; +memset (dst + map->timer_off,0 ,sizeof (struct bpf_timer )); }/* copy everything but bpf_spin_lock and bpf_timer. There could be one of each. */
Signature · 1 line
2.30.2
This series was applied to bpf/bpf.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This series was applied to bpf/bpf.git (master)
by Alexei Starovoitov <[email protected]>:
Quoted 8 lines · On Fri, 11 Feb 2022 11:49:43 -0800 you wrote:
The patch [1] exposed a
bpf_timerinitialization bug in functioncheck_and_init_map_value(). With bug fix here, the patch [1] can be applied with all selftests passed. Please see individual patches for fix details.[1] https://lore.kernel.org/bpf/[email protected]/
[...]
Here is the summary with links:
- [bpf,v4,1/2] bpf: emit bpf_timer in vmlinux BTF
https://git.kernel.org/bpf/bpf/c/3bd916ee0ecb - [bpf,v4,2/2] bpf: fix a bpf_timer initialization issue https://git.kernel.org/bpf/bpf/c/5eaed6eedbe9
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html