bpf: fix a couple of issues with syscall program
Subsystems (5)
v2 cover bpf: fix a couple of issues with syscall program
syzbot ([1]) reported a rcu warning for syscall program when bpf_get_current_cgroup_id() is called in the program. The bpf_get_current_cgroup_id() helper expects cgroup related mutex or rcu_read_lock(). The sleepable program cannot be protected with rcu read_lock, hence the warning.
Previous version ([2]) tried to add rcu_read_lock() surrounding bpf_prog_run_pin_on_cpu() for syscall program. This doesn't work and will trigger other warning since syscall program is a sleepable program.
In this version, patch #1 disallows to call bpf_get_current_cgroup_id() and bpf_get_current_ancestor_cgroup_id() for sleepable programs to silence the warning. Further, syscall program should be protected with bpf_read_lock_trace() which is implemented in Patch #2.
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
Changelog:
v1 -> v2:
- use bpf_read_lock_trace() instead of bpf_read_lock() for bpf_prog_run_pin_on_cpu().
- disallow bpf_get_current_[ancestor_]cgroup_id() helper.
Yonghong Song (2):
bpf: don't call bpf_get_current_[ancestor_]cgroup_id() in sleepable
progs
bpf: add missing bpf_read_[un]lock_trace() for syscall program
kernel/trace/bpf_trace.c | 6 ++++-- net/bpf/test_run.c | 4 ++++ 2 files changed, 8 insertions(+), 2 deletions(-)
Signature · 1 line
2.30.2
PATCH v2 1/2 bpf: don't call bpf_get_current_[ancestor_]cgroup_id() in sleepable progs
Currently, if bpf_get_current_cgroup_id() or bpf_get_current_ancestor_cgroup_id() helper is called with sleepable programs e.g., sleepable fentry/fmod_ret/fexit/lsm programs, a rcu warning may appear. For example, if I added the following hack to test_progs/test_lsm sleepable fentry program test_sys_setdomainname:
--- a/tools/testing/selftests/bpf/progs/lsm.c
+++ b/tools/testing/selftests/bpf/progs/lsm.c
@@ -168,6 +168,10 @@ int BPF_PROG(test_sys_setdomainname, struct pt_regs *regs)
int buf = 0;
long ret; + __u64 cg_id = bpf_get_current_cgroup_id();
+ if (cg_id == 1000)
+ copy_test++;
+
ret = bpf_copy_from_user(&buf, sizeof(buf), ptr);
if (len == -2 && ret == 0 && buf == 1234)
copy_test++;I will hit the following rcu warning:
include/linux/cgroup.h:481 suspicious rcu_dereference_check() usage! other info that might help us debug this:
rcu_scheduler_active = 2, debug_locks = 1
1 lock held by test_progs/260:
#0: ffffffffa5173360 (rcu_read_lock_trace){....}-{0:0}, at: __bpf_prog_enter_sleepable+0x0/0xa0
stack backtrace:
CPU: 1 PID: 260 Comm: test_progs Tainted: G O 5.14.0-rc2+ #176
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
Call Trace:
dump_stack_lvl+0x56/0x7b
bpf_get_current_cgroup_id+0x9c/0xb1
bpf_prog_a29888d1c6706e09_test_sys_setdomainname+0x3e/0x89c
bpf_trampoline_6442469132_0+0x2d/0x1000
__x64_sys_setdomainname+0x5/0x110
do_syscall_64+0x3a/0x80
entry_SYSCALL_64_after_hwframe+0x44/0xaeI can get similar warning using bpf_get_current_ancestor_cgroup_id() helper. syzbot reported a similar issue in [1] for syscall program. Helper bpf_get_current_cgroup_id() or bpf_get_current_ancestor_cgroup_id() has the following callchain:
task_dfl_cgroup
task_css_set
task_css_set_check
and we have
#define task_css_set_check(task, __c) \
rcu_dereference_check((task)->cgroups, \
lockdep_is_held(&cgroup_mutex) || \
lockdep_is_held(&css_set_lock) || \
((task)->flags & PF_EXITING) || (__c))Since cgroup_mutex/css_set_lock is not held and the task is not existing and rcu read_lock is not held, a warning will be issued. Note that bpf sleepable program is protected by rcu_read_lock_trace().
To fix the issue, let us make these two helpers not available to sleepable program. I marked the patch fixing 95b861a7935b ("bpf: Allow bpf_get_current_ancestor_cgroup_id for tracing") which added bpf_get_current_ancestor_cgroup_id() to 5.14. I think backporting 5.14 is probably good enough as sleepable progrems are not widely used.
This patch should fix [1] as well since syscall program is a sleepable program and bpf_get_current_cgroup_id() is not available to syscall program any more.
[1] https://lore.kernel.org/bpf/[email protected]/
--- kernel/trace/bpf_trace.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
kernel/trace/bpf_trace.c +4 -2
@@ -1016,9 +1016,11 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)#endif #ifdef CONFIG_CGROUPS case BPF_FUNC_get_current_cgroup_id: -return &bpf_get_current_cgroup_id_proto; +return prog->aux->sleepable ? +NULL : &bpf_get_current_cgroup_id_proto;case BPF_FUNC_get_current_ancestor_cgroup_id: -return &bpf_get_current_ancestor_cgroup_id_proto; +return prog->aux->sleepable ? +NULL : &bpf_get_current_ancestor_cgroup_id_proto;#endif case BPF_FUNC_send_signal:return &bpf_send_signal_proto;
Signature · 1 line
2.30.2
Re: [PATCH bpf v2 1/2] bpf: don't call bpf_get_current_[ancestor_]cgroup_id() in sleepable progs
Quoted 85 lines · Yonghong Song
Currently, if
bpf_get_current_cgroup_id()orbpf_get_current_ancestor_cgroup_id()helper is called with sleepable programs e.g., sleepable fentry/fmod_ret/fexit/lsm programs, a rcu warning may appear. For example, if I added the following hack to test_progs/test_lsm sleepable fentry programtest_sys_setdomainname:--- a/tools/testing/selftests/bpf/progs/lsm.c +++ b/tools/testing/selftests/bpf/progs/lsm.c @@ -168,6 +168,10 @@ int BPF_PROG(test_sys_setdomainname, struct pt_regs *regs) int buf = 0; long ret;+
__u64cg_id=bpf_get_current_cgroup_id();
+ if (cg_id== 1000)
+ copy_test++;
+ret = bpf_copy_from_user(&buf, sizeof(buf), ptr); if (len == -2 && ret == 0 && buf == 1234) copy_test++;I will hit the following rcu warning:
include/linux/cgroup.h:481suspiciousrcu_dereference_check()usage! other info that might help us debug this:rcu_scheduler_active = 2, debug_locks = 1 1 lock held by test_progs/260: #0: ffffffffa5173360 (rcu_read_lock_trace){....}-{0:0}, at: __bpf_prog_enter_sleepable+0x0/0xa0 stack backtrace: CPU: 1 PID: 260 Comm: test_progs Tainted: G O 5.14.0-rc2+ #176 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 Call Trace: dump_stack_lvl+0x56/0x7b bpf_get_current_cgroup_id+0x9c/0xb1 bpf_prog_a29888d1c6706e09_test_sys_setdomainname+0x3e/0x89c bpf_trampoline_6442469132_0+0x2d/0x1000 __x64_sys_setdomainname+0x5/0x110 do_syscall_64+0x3a/0x80 entry_SYSCALL_64_after_hwframe+0x44/0xaeI can get similar warning using
bpf_get_current_ancestor_cgroup_id()helper. syzbot reported a similar issue in [1] for syscall program. Helperbpf_get_current_cgroup_id()orbpf_get_current_ancestor_cgroup_id()has the following callchain:
task_dfl_cgroup
task_css_set
task_css_set_check
and we have#define task_css_set_check(task, __c) \ rcu_dereference_check((task)->cgroups, \ lockdep_is_held(&cgroup_mutex) || \ lockdep_is_held(&css_set_lock) || \ ((task)->flags & PF_EXITING) || (__c))Since cgroup_mutex/css_set_lock is not held and the task is not existing and rcu
read_lockis not held, a warning will be issued. Note that bpf sleepable program is protected byrcu_read_lock_trace().To fix the issue, let us make these two helpers not available to sleepable program. I marked the patch fixing
95b861a7935b("bpf: Allowbpf_get_current_ancestor_cgroup_idfor tracing") which addedbpf_get_current_ancestor_cgroup_id()to 5.14. I think backporting 5.14 is probably good enough as sleepable progrems are not widely used.This patch should fix [1] as well since syscall program is a sleepable program and
bpf_get_current_cgroup_id()is not available to syscall program any more.[1] https://lore.kernel.org/bpf/[email protected]/
Reported-by [email protected]Fixes 95b861a7935b ("bpf: Allow bpf_get_current_ancestor_cgroup_id for tracing")Signed-off-by Yonghong Song <[email protected]>--- kernel/trace/bpf_trace.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)kernel/trace/bpf_trace.c +0
-0
kernel/trace/bpf_trace.c bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) · line 1023
Quoted 10 lines
kernel/trace/bpf_trace.c +4
-2@@ -1016,9 +1016,11 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)#endif #ifdef CONFIG_CGROUPS case BPF_FUNC_get_current_cgroup_id: -return &bpf_get_current_cgroup_id_proto; +return prog->aux->sleepable ? +NULL : &bpf_get_current_cgroup_id_proto;case BPF_FUNC_get_current_ancestor_cgroup_id: -return &bpf_get_current_ancestor_cgroup_id_proto; +return prog->aux->sleepable ? +NULL : &bpf_get_current_ancestor_cgroup_id_proto;
This feels too extreme. I bet these helpers are as useful in sleepable BPF progs as they are in non-sleepable ones.
Why don't we just implement a variant of get_current_cgroup_id (and the ancestor variant as well) which takes that cgroup_mutex lock, and just pick the appropriate implementation. Wouldn't that work?
Quoted 6 lines
kernel/trace/bpf_trace.c +0
-1#endif case BPF_FUNC_send_signal:return &bpf_send_signal_proto; --2.30.2
Re: [PATCH bpf v2 1/2] bpf: don't call bpf_get_current_[ancestor_]cgroup_id() in sleepable progs
Quoted 100 lines · Andrii Nakryiko
Quoted 85 lines · Yonghong Song
Currently, if
bpf_get_current_cgroup_id()orbpf_get_current_ancestor_cgroup_id()helper is called with sleepable programs e.g., sleepable fentry/fmod_ret/fexit/lsm programs, a rcu warning may appear. For example, if I added the following hack to test_progs/test_lsm sleepable fentry programtest_sys_setdomainname:--- a/tools/testing/selftests/bpf/progs/lsm.c +++ b/tools/testing/selftests/bpf/progs/lsm.c @@ -168,6 +168,10 @@ int BPF_PROG(test_sys_setdomainname, struct pt_regs *regs) int buf = 0; long ret;+ __u64 cg_id = bpf_get_current_cgroup_id(); + if (cg_id == 1000) + copy_test++; + ret = bpf_copy_from_user(&buf, sizeof(buf), ptr); if (len == -2 && ret == 0 && buf == 1234) copy_test++;I will hit the following rcu warning:
include/linux/cgroup.h:481 suspicious rcu_dereference_check() usage! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 1 1 lock held by test_progs/260: #0: ffffffffa5173360 (rcu_read_lock_trace){....}-{0:0}, at: __bpf_prog_enter_sleepable+0x0/0xa0 stack backtrace: CPU: 1 PID: 260 Comm: test_progs Tainted: G O 5.14.0-rc2+ #176 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 Call Trace: dump_stack_lvl+0x56/0x7b bpf_get_current_cgroup_id+0x9c/0xb1 bpf_prog_a29888d1c6706e09_test_sys_setdomainname+0x3e/0x89c bpf_trampoline_6442469132_0+0x2d/0x1000 __x64_sys_setdomainname+0x5/0x110 do_syscall_64+0x3a/0x80 entry_SYSCALL_64_after_hwframe+0x44/0xaeI can get similar warning using
bpf_get_current_ancestor_cgroup_id()helper. syzbot reported a similar issue in [1] for syscall program. Helperbpf_get_current_cgroup_id()orbpf_get_current_ancestor_cgroup_id()has the following callchain:
task_dfl_cgroup
task_css_set
task_css_set_check
and we have#define task_css_set_check(task, __c) \ rcu_dereference_check((task)->cgroups, \ lockdep_is_held(&cgroup_mutex) || \ lockdep_is_held(&css_set_lock) || \ ((task)->flags & PF_EXITING) || (__c))Since cgroup_mutex/css_set_lock is not held and the task is not existing and rcu
read_lockis not held, a warning will be issued. Note that bpf sleepable program is protected byrcu_read_lock_trace().To fix the issue, let us make these two helpers not available to sleepable program. I marked the patch fixing
95b861a7935b("bpf: Allowbpf_get_current_ancestor_cgroup_idfor tracing") which addedbpf_get_current_ancestor_cgroup_id()to 5.14. I think backporting 5.14 is probably good enough as sleepable progrems are not widely used.This patch should fix [1] as well since syscall program is a sleepable program and
bpf_get_current_cgroup_id()is not available to syscall program any more.[1] https://lore.kernel.org/bpf/[email protected]/
Reported-by [email protected]Fixes 95b861a7935b ("bpf: Allow bpf_get_current_ancestor_cgroup_id for tracing")Signed-off-by Yonghong Song <[email protected]>--- kernel/trace/bpf_trace.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)kernel/trace/bpf_trace.c +0
-0kernel/trace/bpf_trace.c bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) · line 1023
Quoted 10 lines
kernel/trace/bpf_trace.c +4
-2@@ -1016,9 +1016,11 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)#endif #ifdef CONFIG_CGROUPS case BPF_FUNC_get_current_cgroup_id: -return &bpf_get_current_cgroup_id_proto; +return prog->aux->sleepable ? +NULL : &bpf_get_current_cgroup_id_proto;case BPF_FUNC_get_current_ancestor_cgroup_id: -return &bpf_get_current_ancestor_cgroup_id_proto; +return prog->aux->sleepable ? +NULL : &bpf_get_current_ancestor_cgroup_id_proto;This feels too extreme. I bet these helpers are as useful in sleepable BPF progs as they are in non-sleepable ones.
Quoted 3 lines
Why don't we just implement a variant of
get_current_cgroup_id(and the ancestor variant as well) which takes thatcgroup_mutexlock, and just pick the appropriate implementation. Wouldn't that work?
This may not work. e.g., for sleepable fentry program, if the to-be-traced function is inside in cgroup_mutex, we will have a deadlock.
Currently, affected program types are tracing/fentry.s, tracing/fexit.s, tracing/fmod_ret.s, lsm.s and syscall. For fmod_ret.s, lsm.s, they all have
some kind of predefined attachment/context, we might be able to check all potential attachment points and allow these two helpers when attachment point is not surrounded by cgroup_mutex.
For syscall program, we should be okay as it is called with bpf_prog_test_run interface but I am not sure why user wants a cgroup_id for that.
Quoted 7 lines
Quoted 7 lines
kernel/trace/bpf_trace.c +0
-1#endif case BPF_FUNC_send_signal:return &bpf_send_signal_proto; --2.30.2
Re: [PATCH bpf v2 1/2] bpf: don't call bpf_get_current_[ancestor_]cgroup_id() in sleepable progs
Quoted 104 lines · Yonghong Song
Quoted 100 lines · Andrii Nakryiko
Quoted 85 lines · Yonghong Song
Currently, if
bpf_get_current_cgroup_id()orbpf_get_current_ancestor_cgroup_id()helper is called with sleepable programs e.g., sleepable fentry/fmod_ret/fexit/lsm programs, a rcu warning may appear. For example, if I added the following hack to test_progs/test_lsm sleepable fentry programtest_sys_setdomainname:--- a/tools/testing/selftests/bpf/progs/lsm.c +++ b/tools/testing/selftests/bpf/progs/lsm.c @@ -168,6 +168,10 @@ int BPF_PROG(test_sys_setdomainname, struct pt_regs *regs) int buf = 0; long ret;+ __u64 cg_id = bpf_get_current_cgroup_id(); + if (cg_id == 1000) + copy_test++; + ret = bpf_copy_from_user(&buf, sizeof(buf), ptr); if (len == -2 && ret == 0 && buf == 1234) copy_test++;I will hit the following rcu warning:
include/linux/cgroup.h:481 suspicious rcu_dereference_check() usage! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 1 1 lock held by test_progs/260: #0: ffffffffa5173360 (rcu_read_lock_trace){....}-{0:0}, at: __bpf_prog_enter_sleepable+0x0/0xa0 stack backtrace: CPU: 1 PID: 260 Comm: test_progs Tainted: G O 5.14.0-rc2+ #176 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 Call Trace: dump_stack_lvl+0x56/0x7b bpf_get_current_cgroup_id+0x9c/0xb1 bpf_prog_a29888d1c6706e09_test_sys_setdomainname+0x3e/0x89c bpf_trampoline_6442469132_0+0x2d/0x1000 __x64_sys_setdomainname+0x5/0x110 do_syscall_64+0x3a/0x80 entry_SYSCALL_64_after_hwframe+0x44/0xaeI can get similar warning using
bpf_get_current_ancestor_cgroup_id()helper. syzbot reported a similar issue in [1] for syscall program. Helperbpf_get_current_cgroup_id()orbpf_get_current_ancestor_cgroup_id()has the following callchain:
task_dfl_cgroup
task_css_set
task_css_set_check
and we have#define task_css_set_check(task, __c) \ rcu_dereference_check((task)->cgroups, \ lockdep_is_held(&cgroup_mutex) || \ lockdep_is_held(&css_set_lock) || \ ((task)->flags & PF_EXITING) || (__c))Since cgroup_mutex/css_set_lock is not held and the task is not existing and rcu
read_lockis not held, a warning will be issued. Note that bpf sleepable program is protected byrcu_read_lock_trace().To fix the issue, let us make these two helpers not available to sleepable program. I marked the patch fixing
95b861a7935b("bpf: Allowbpf_get_current_ancestor_cgroup_idfor tracing") which addedbpf_get_current_ancestor_cgroup_id()to 5.14. I think backporting 5.14 is probably good enough as sleepable progrems are not widely used.This patch should fix [1] as well since syscall program is a sleepable program and
bpf_get_current_cgroup_id()is not available to syscall program any more.[1] https://lore.kernel.org/bpf/[email protected]/
Reported-by [email protected]Fixes 95b861a7935b ("bpf: Allow bpf_get_current_ancestor_cgroup_id for tracing")Signed-off-by Yonghong Song <[email protected]>--- kernel/trace/bpf_trace.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)kernel/trace/bpf_trace.c +0
-0kernel/trace/bpf_trace.c bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) · line 1023
Quoted 10 lines
kernel/trace/bpf_trace.c +4
-2@@ -1016,9 +1016,11 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)#endif #ifdef CONFIG_CGROUPS case BPF_FUNC_get_current_cgroup_id: -return &bpf_get_current_cgroup_id_proto; +return prog->aux->sleepable ? +NULL : &bpf_get_current_cgroup_id_proto;case BPF_FUNC_get_current_ancestor_cgroup_id: -return &bpf_get_current_ancestor_cgroup_id_proto; +return prog->aux->sleepable ? +NULL : &bpf_get_current_ancestor_cgroup_id_proto;This feels too extreme. I bet these helpers are as useful in sleepable BPF progs as they are in non-sleepable ones.
Quoted 7 lines
Quoted 3 lines
Why don't we just implement a variant of
get_current_cgroup_id(and the ancestor variant as well) which takes thatcgroup_mutexlock, and just pick the appropriate implementation. Wouldn't that work?This may not work. e.g., for sleepable fentry program, if the to-be-traced function is inside in
cgroup_mutex, we will have a deadlock.
We can also do preempty_disable() + rcu_read_lock() inside the helper itself, no? I mean in the new "sleepable" variant.
Quoted 8 lines
Currently, affected program types are
tracing/fentry.s,tracing/fexit.s,tracing/fmod_ret.s, lsm.s and syscall. For fmod_ret.s, lsm.s, they all have
some kind of predefined attachment/context, we might be able to check all potential attachment points and allow these two helpers when attachment point is not surrounded bycgroup_mutex.
I don't think it's feasible to know if any given attached kernel function can be called with cgroup_mutex taken. Static analysis will be too complicated and too restrictive. Runtime checks might be too expensive and/or not generic enough. But see above, we can do rcu_read_lock() inside the helper while preventing preemption, and it will behave the same way as if it was called from non-sleepable BPF prog.
Quoted 11 lines
For syscall program, we should be okay as it is called with
bpf_prog_test_runinterface but I am not sure why user wants acgroup_idfor that.Quoted 7 lines
Quoted 7 lines
kernel/trace/bpf_trace.c +0
-1#endif case BPF_FUNC_send_signal:return &bpf_send_signal_proto; --2.30.2
Daniel Borkmann
maintainerRe: [PATCH bpf v2 1/2] bpf: don't call bpf_get_current_[ancestor_]cgroup_id() in sleepable progs
Quoted 106 lines · Andrii Nakryiko
Quoted 105 lines · Yonghong Song
Quoted 103 lines · Andrii Nakryiko
Quoted 85 lines · Yonghong Song
Currently, if
bpf_get_current_cgroup_id()orbpf_get_current_ancestor_cgroup_id()helper is called with sleepable programs e.g., sleepable fentry/fmod_ret/fexit/lsm programs, a rcu warning may appear. For example, if I added the following hack to test_progs/test_lsm sleepable fentry programtest_sys_setdomainname:--- a/tools/testing/selftests/bpf/progs/lsm.c +++ b/tools/testing/selftests/bpf/progs/lsm.c @@ -168,6 +168,10 @@ int BPF_PROG(test_sys_setdomainname, struct pt_regs *regs) int buf = 0; long ret; + __u64 cg_id = bpf_get_current_cgroup_id(); + if (cg_id == 1000) + copy_test++; + ret = bpf_copy_from_user(&buf, sizeof(buf), ptr); if (len == -2 && ret == 0 && buf == 1234) copy_test++;I will hit the following rcu warning:
include/linux/cgroup.h:481 suspicious rcu_dereference_check() usage! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 1 1 lock held by test_progs/260: #0: ffffffffa5173360 (rcu_read_lock_trace){....}-{0:0}, at: __bpf_prog_enter_sleepable+0x0/0xa0 stack backtrace: CPU: 1 PID: 260 Comm: test_progs Tainted: G O 5.14.0-rc2+ #176 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 Call Trace: dump_stack_lvl+0x56/0x7b bpf_get_current_cgroup_id+0x9c/0xb1 bpf_prog_a29888d1c6706e09_test_sys_setdomainname+0x3e/0x89c bpf_trampoline_6442469132_0+0x2d/0x1000 __x64_sys_setdomainname+0x5/0x110 do_syscall_64+0x3a/0x80 entry_SYSCALL_64_after_hwframe+0x44/0xaeI can get similar warning using
bpf_get_current_ancestor_cgroup_id()helper. syzbot reported a similar issue in [1] for syscall program. Helperbpf_get_current_cgroup_id()orbpf_get_current_ancestor_cgroup_id()has the following callchain:
task_dfl_cgroup
task_css_set
task_css_set_check
and we have#define task_css_set_check(task, __c) \ rcu_dereference_check((task)->cgroups, \ lockdep_is_held(&cgroup_mutex) || \ lockdep_is_held(&css_set_lock) || \ ((task)->flags & PF_EXITING) || (__c))Since cgroup_mutex/css_set_lock is not held and the task is not existing and rcu
read_lockis not held, a warning will be issued. Note that bpf sleepable program is protected byrcu_read_lock_trace().To fix the issue, let us make these two helpers not available to sleepable program. I marked the patch fixing
95b861a7935b("bpf: Allowbpf_get_current_ancestor_cgroup_idfor tracing") which addedbpf_get_current_ancestor_cgroup_id()to 5.14. I think backporting 5.14 is probably good enough as sleepable progrems are not widely used.This patch should fix [1] as well since syscall program is a sleepable program and
bpf_get_current_cgroup_id()is not available to syscall program any more.[1] https://lore.kernel.org/bpf/[email protected]/Reported-by [email protected]Fixes 95b861a7935b ("bpf: Allow bpf_get_current_ancestor_cgroup_id for tracing")Signed-off-by Yonghong Song <[email protected]>--- kernel/trace/bpf_trace.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)kernel/trace/bpf_trace.c +0
-0kernel/trace/bpf_trace.c bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) · line 1023
Quoted 10 lines
kernel/trace/bpf_trace.c +4
-2@@ -1016,9 +1016,11 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)#endif #ifdef CONFIG_CGROUPS case BPF_FUNC_get_current_cgroup_id: -return &bpf_get_current_cgroup_id_proto; +return prog->aux->sleepable ? +NULL : &bpf_get_current_cgroup_id_proto;case BPF_FUNC_get_current_ancestor_cgroup_id: -return &bpf_get_current_ancestor_cgroup_id_proto; +return prog->aux->sleepable ? +NULL : &bpf_get_current_ancestor_cgroup_id_proto;This feels too extreme. I bet these helpers are as useful in sleepable BPF progs as they are in non-sleepable ones.
Why don't we just implement a variant of
get_current_cgroup_id(and the ancestor variant as well) which takes thatcgroup_mutexlock, and just pick the appropriate implementation. Wouldn't that work?
Quoted 6 lines
Quoted 3 lines
This may not work. e.g., for sleepable fentry program, if the to-be-traced function is inside in
cgroup_mutex, we will have a deadlock.We can also do
preempty_disable()+rcu_read_lock()inside the helper itself, no? I mean in the new "sleepable" variant.
Yep, we do that for example in c5dbb89fc2ac ("bpf: Expose bpf_get_socket_cookie to tracing programs") as well (the sock_gen_cookie() disables preemption).
Thanks,
Daniel
Re: [PATCH bpf v2 1/2] bpf: don't call bpf_get_current_[ancestor_]cgroup_id() in sleepable progs
Quoted 114 lines · Daniel Borkmann
Quoted 113 lines · Andrii Nakryiko
Quoted 111 lines · Yonghong Song
Quoted 103 lines · Andrii Nakryiko
Quoted 90 lines · Yonghong Song
Currently, if
bpf_get_current_cgroup_id()orbpf_get_current_ancestor_cgroup_id()helper is called with sleepable programs e.g., sleepable fentry/fmod_ret/fexit/lsm programs, a rcu warning may appear. For example, if I added the following hack to test_progs/test_lsm sleepable fentry programtest_sys_setdomainname:---
a/tools/testing/selftests/bpf/progs/lsm.c
+++b/tools/testing/selftests/bpf/progs/lsm.c
@@ -168,6 +168,10 @@ intBPF_PROG(test_sys_setdomainname, struct pt_regs *regs)int buf = 0;
long ret;+
__u64cg_id=bpf_get_current_cgroup_id(); + if (cg_id== 1000)
+ copy_test++;
+
ret =bpf_copy_from_user(&buf, sizeof(buf), ptr); if (len == -2 && ret == 0 && buf == 1234) copy_test++;I will hit the following rcu warning:
include/linux/cgroup.h:481suspiciousrcu_dereference_check()usage!
other info that might help us debug this:
rcu_scheduler_active= 2,debug_locks= 1
1 lock held by test_progs/260:
#0:ffffffffa5173360(rcu_read_lock_trace){....}-{0:0}, at:__bpf_prog_enter_sleepable+0x0/0xa0stack backtrace: CPU: 1 PID: 260 Comm: test_progs Tainted: G O 5.14.0-rc2+ #176 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 Call Trace: dump_stack_lvl+0x56/0x7b bpf_get_current_cgroup_id+0x9c/0xb1 bpf_prog_a29888d1c6706e09_test_sys_setdomainname+0x3e/0x89c bpf_trampoline_6442469132_0+0x2d/0x1000 __x64_sys_setdomainname+0x5/0x110 do_syscall_64+0x3a/0x80 entry_SYSCALL_64_after_hwframe+0x44/0xaeI can get similar warning using
bpf_get_current_ancestor_cgroup_id()helper. syzbot reported a similar issue in [1] for syscall program. Helperbpf_get_current_cgroup_id()orbpf_get_current_ancestor_cgroup_id()has the following callchain:
task_dfl_cgroup
task_css_set
task_css_set_check
and we have
#definetask_css_set_check(task, __c)\rcu_dereference_check((task)->cgroups, \
lockdep_is_held(&cgroup_mutex) || \ lockdep_is_held(&css_set_lock) || \((task)->flags &
PF_EXITING) || (__c)) Since cgroup_mutex/css_set_lock is not held and the task is not existing and rcuread_lockis not held, a warning will be issued. Note that bpf sleepable program is protected byrcu_read_lock_trace().To fix the issue, let us make these two helpers not available to sleepable program. I marked the patch fixing
95b861a7935b("bpf: Allowbpf_get_current_ancestor_cgroup_idfor tracing") which addedbpf_get_current_ancestor_cgroup_id()to 5.14. I think backporting 5.14 is probably good enough as sleepable progrems are not widely used.This patch should fix [1] as well since syscall program is a sleepable program and
bpf_get_current_cgroup_id()is not available to syscall program any more.[1] https://lore.kernel.org/bpf/[email protected]/
Reported-by [email protected]Fixes 95b861a7935b ("bpf: Allow bpf_get_current_ancestor_cgroup_id for tracing")Signed-off-by Yonghong Song <[email protected]>---
kernel/trace/bpf_trace.c| 6 ++++--1 file changed, 4 insertions(+), 2 deletions(-)kernel/trace/bpf_trace.c +0
-0@@ -1016,9 +1016,11 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) #endif #ifdef CONFIG_CGROUPScase
BPF_FUNC_get_current_cgroup_id:Quoted 8 lines
- return
&bpf_get_current_cgroup_id_proto; + returnprog->aux->sleepable?
+ NULL :&bpf_get_current_cgroup_id_proto; caseBPF_FUNC_get_current_ancestor_cgroup_id:
- return&bpf_get_current_ancestor_cgroup_id_proto; + returnprog->aux->sleepable?
+ NULL :&bpf_get_current_ancestor_cgroup_id_proto;This feels too extreme. I bet these helpers are as useful in sleepable BPF progs as they are in non-sleepable ones.
Quoted 3 lines
Why don't we just implement a variant of
get_current_cgroup_id(and the ancestor variant as well) which takes thatcgroup_mutexlock, and just pick the appropriate implementation. Wouldn't that work?This may not work. e.g., for sleepable fentry program, if the to-be-traced function is inside in
cgroup_mutex, we will have a deadlock.
Quoted 5 lines
Quoted 2 lines
We can also do
preempty_disable()+rcu_read_lock()inside the helper itself, no? I mean in the new "sleepable" variant.Yep, we do that for example in
c5dbb89fc2ac("bpf: Exposebpf_get_socket_cookieto tracing programs") as well (thesock_gen_cookie()disables preemption).
I think rcu_read_lock() is enough. For all the cases I mentioned in the above, we already have migrate_disable(). Together with rcu_read_lock(), we should be okay. For non-sleepable programs (tracing,lsm), the helpers
are protected with migrate_disable() and rcu_read_lock().
Quoted 3 lines
Thanks,
Daniel
PATCH v2 2/2 bpf: add missing bpf_read_[un]lock_trace() for syscall program
In reply to Yonghong SongCommit 79a7f8bdb159d ("bpf: Introduce bpf_sys_bpf() helper and program type.") added support for syscall program, which is a sleepable program. But the program run missed bpf_read_lock_trace()/bpf_read_unlock_trace(), which is needed to ensure proper rcu callback invocations.
This patch added bpf_read_[un]lock_trace() properly.
--- net/bpf/test_run.c | 4 ++++ 1 file changed, 4 insertions(+)
net/bpf/test_run.c +4 -0
@@ -7,6 +7,7 @@#include <linux/vmalloc.h> #include <linux/etherdevice.h> #include <linux/filter.h> +#include <linux/rcupdate_trace.h> #include <linux/sched/signal.h> #include <net/bpf_sk_storage.h> #include <net/sock.h>
@@ -951,7 +952,10 @@ int bpf_prog_test_run_syscall(struct bpf_prog *prog, goto out; } } + +rcu_read_lock_trace (); retval =bpf_prog_run_pin_on_cpu (prog, ctx); +rcu_read_unlock_trace ();if (copy_to_user (&uattr->test.retval, &retval,sizeof (u32))) { err = -EFAULT ;
Signature · 1 line
2.30.2
Re: [PATCH bpf v2 2/2] bpf: add missing bpf_read_[un]lock_trace() for syscall program
Quoted 10 lines · Yonghong Song
Commit
79a7f8bdb159d("bpf: Introducebpf_sys_bpf()helper and program type.") added support for syscall program, which is a sleepable program. But the program run missed bpf_read_lock_trace()/bpf_read_unlock_trace(), which is needed to ensure proper rcu callback invocations.
This patch added bpf_read_[un]lock_trace() properly.Fixes 79a7f8bdb159d ("bpf: Introduce bpf_sys_bpf() helper and program type.")Signed-off-by Yonghong Song <[email protected]>---
LGTM.
Quoted 29 lines
net/bpf/test_run.c | 4 ++++ 1 file changed, 4 insertions(+)net/bpf/test_run.c +4
-1@@ -7,6 +7,7 @@#include <linux/vmalloc.h> #include <linux/etherdevice.h> #include <linux/filter.h> +#include <linux/rcupdate_trace.h> #include <linux/sched/signal.h> #include <net/bpf_sk_storage.h> #include <net/sock.h> @@ -951,7 +952,10 @@ int bpf_prog_test_run_syscall(struct bpf_prog *prog, goto out; } } + +rcu_read_lock_trace (); retval =bpf_prog_run_pin_on_cpu (prog, ctx); +rcu_read_unlock_trace ();if (copy_to_user (&uattr->test.retval, &retval,sizeof (u32))) { err = -EFAULT ; --2.30.2