Skip to contentSkip to conversation

bpf: fix a couple of issues with syscall program

PATCH Superseded Yonghong Song 3 participants 9 messages Aug 9, 2021

Subsystems (5)
Networking BPF Tracing BPF tracing +1 more subsystems

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v2 cover bpf: fix a couple of issues with syscall program

syzbot ([1]) reported a rcu warning for syscall program when bpf_get_current_cgroup_id() is called in the program. The bpf_get_current_cgroup_id() helper expects cgroup related mutex or rcu_read_lock(). The sleepable program cannot be protected with rcu read_lock, hence the warning.

Previous version ([2]) tried to add rcu_read_lock() surrounding bpf_prog_run_pin_on_cpu() for syscall program. This doesn't work and will trigger other warning since syscall program is a sleepable program.

In this version, patch #1 disallows to call bpf_get_current_cgroup_id() and bpf_get_current_ancestor_cgroup_id() for sleepable programs to silence the warning. Further, syscall program should be protected with bpf_read_lock_trace() which is implemented in Patch #2.

[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/

Changelog:
v1 -> v2:
- use bpf_read_lock_trace() instead of bpf_read_lock() for bpf_prog_run_pin_on_cpu().
- disallow bpf_get_current_[ancestor_]cgroup_id() helper.

Yonghong Song (2):
bpf: don't call bpf_get_current_[ancestor_]cgroup_id() in sleepable
progs
bpf: add missing bpf_read_[un]lock_trace() for syscall program

 kernel/trace/bpf_trace.c | 6 ++++--
 net/bpf/test_run.c       | 4 ++++
 2 files changed, 8 insertions(+), 2 deletions(-)
Signature · 1 line
2.30.2
#1 lore raw

PATCH v2 1/2 bpf: don't call bpf_get_current_[ancestor_]cgroup_id() in sleepable progs

Currently, if bpf_get_current_cgroup_id() or bpf_get_current_ancestor_cgroup_id() helper is called with sleepable programs e.g., sleepable fentry/fmod_ret/fexit/lsm programs, a rcu warning may appear. For example, if I added the following hack to test_progs/test_lsm sleepable fentry program test_sys_setdomainname:

  --- a/tools/testing/selftests/bpf/progs/lsm.c
  +++ b/tools/testing/selftests/bpf/progs/lsm.c
  @@ -168,6 +168,10 @@ int BPF_PROG(test_sys_setdomainname, struct pt_regs *regs)
          int buf = 0;
          long ret;

+ __u64 cg_id = bpf_get_current_cgroup_id();
+ if (cg_id == 1000)
+ copy_test++;
+

          ret = bpf_copy_from_user(&buf, sizeof(buf), ptr);
          if (len == -2 && ret == 0 && buf == 1234)
                  copy_test++;

I will hit the following rcu warning:

include/linux/cgroup.h:481 suspicious rcu_dereference_check() usage! other info that might help us debug this:

    rcu_scheduler_active = 2, debug_locks = 1
    1 lock held by test_progs/260:
      #0: ffffffffa5173360 (rcu_read_lock_trace){....}-{0:0}, at: __bpf_prog_enter_sleepable+0x0/0xa0
    stack backtrace:
    CPU: 1 PID: 260 Comm: test_progs Tainted: G           O      5.14.0-rc2+ #176
    Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
    Call Trace:
      dump_stack_lvl+0x56/0x7b
      bpf_get_current_cgroup_id+0x9c/0xb1
      bpf_prog_a29888d1c6706e09_test_sys_setdomainname+0x3e/0x89c
      bpf_trampoline_6442469132_0+0x2d/0x1000
      __x64_sys_setdomainname+0x5/0x110
      do_syscall_64+0x3a/0x80
      entry_SYSCALL_64_after_hwframe+0x44/0xae

I can get similar warning using bpf_get_current_ancestor_cgroup_id() helper. syzbot reported a similar issue in [1] for syscall program. Helper bpf_get_current_cgroup_id() or bpf_get_current_ancestor_cgroup_id() has the following callchain:
task_dfl_cgroup
task_css_set
task_css_set_check
and we have

   #define task_css_set_check(task, __c)                                   \
           rcu_dereference_check((task)->cgroups,                          \
                   lockdep_is_held(&cgroup_mutex) ||                       \
                   lockdep_is_held(&css_set_lock) ||                       \
                   ((task)->flags & PF_EXITING) || (__c))

Since cgroup_mutex/css_set_lock is not held and the task is not existing and rcu read_lock is not held, a warning will be issued. Note that bpf sleepable program is protected by rcu_read_lock_trace().

To fix the issue, let us make these two helpers not available to sleepable program. I marked the patch fixing 95b861a7935b ("bpf: Allow bpf_get_current_ancestor_cgroup_id for tracing") which added bpf_get_current_ancestor_cgroup_id() to 5.14. I think backporting 5.14 is probably good enough as sleepable progrems are not widely used.

This patch should fix [1] as well since syscall program is a sleepable program and bpf_get_current_cgroup_id() is not available to syscall program any more.

[1] https://lore.kernel.org/bpf/[email protected]/

Reported-by [email protected]
Fixes 95b861a7935b ("bpf: Allow bpf_get_current_ancestor_cgroup_id for tracing")
---
 kernel/trace/bpf_trace.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
kernel/trace/bpf_trace.c +4 -2
index b4916ef388ad..eaa8a8ffbe46 100644
@@ -1016,9 +1016,11 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 #endif
 #ifdef CONFIG_CGROUPS
 	case BPF_FUNC_get_current_cgroup_id:
-		return &bpf_get_current_cgroup_id_proto;
+		return prog->aux->sleepable ?
+		       NULL : &bpf_get_current_cgroup_id_proto;
 	case BPF_FUNC_get_current_ancestor_cgroup_id:
-		return &bpf_get_current_ancestor_cgroup_id_proto;
+		return prog->aux->sleepable ?
+		       NULL : &bpf_get_current_ancestor_cgroup_id_proto;
 #endif
 	case BPF_FUNC_send_signal:
 		return &bpf_send_signal_proto;
Signature · 1 line
2.30.2
#2 lore raw

Re: [PATCH bpf v2 1/2] bpf: don't call bpf_get_current_[ancestor_]cgroup_id() in sleepable progs

Quoted 85 lines · Yonghong Song

Currently, if bpf_get_current_cgroup_id() or bpf_get_current_ancestor_cgroup_id() helper is called with sleepable programs e.g., sleepable fentry/fmod_ret/fexit/lsm programs, a rcu warning may appear. For example, if I added the following hack to test_progs/test_lsm sleepable fentry program test_sys_setdomainname:

  --- a/tools/testing/selftests/bpf/progs/lsm.c
  +++ b/tools/testing/selftests/bpf/progs/lsm.c
  @@ -168,6 +168,10 @@ int BPF_PROG(test_sys_setdomainname, struct pt_regs *regs)
          int buf = 0;
          long ret;

+ __u64 cg_id = bpf_get_current_cgroup_id();
+ if (cg_id == 1000)
+ copy_test++;
+

          ret = bpf_copy_from_user(&buf, sizeof(buf), ptr);
          if (len == -2 && ret == 0 && buf == 1234)
                  copy_test++;

I will hit the following rcu warning:

include/linux/cgroup.h:481 suspicious rcu_dereference_check() usage! other info that might help us debug this:

    rcu_scheduler_active = 2, debug_locks = 1
    1 lock held by test_progs/260:
      #0: ffffffffa5173360 (rcu_read_lock_trace){....}-{0:0}, at: __bpf_prog_enter_sleepable+0x0/0xa0
    stack backtrace:
    CPU: 1 PID: 260 Comm: test_progs Tainted: G           O      5.14.0-rc2+ #176
    Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
    Call Trace:
      dump_stack_lvl+0x56/0x7b
      bpf_get_current_cgroup_id+0x9c/0xb1
      bpf_prog_a29888d1c6706e09_test_sys_setdomainname+0x3e/0x89c
      bpf_trampoline_6442469132_0+0x2d/0x1000
      __x64_sys_setdomainname+0x5/0x110
      do_syscall_64+0x3a/0x80
      entry_SYSCALL_64_after_hwframe+0x44/0xae

I can get similar warning using bpf_get_current_ancestor_cgroup_id() helper. syzbot reported a similar issue in [1] for syscall program. Helper bpf_get_current_cgroup_id() or bpf_get_current_ancestor_cgroup_id() has the following callchain:
task_dfl_cgroup
task_css_set
task_css_set_check
and we have

   #define task_css_set_check(task, __c)                                   \
           rcu_dereference_check((task)->cgroups,                          \
                   lockdep_is_held(&cgroup_mutex) ||                       \
                   lockdep_is_held(&css_set_lock) ||                       \
                   ((task)->flags & PF_EXITING) || (__c))

Since cgroup_mutex/css_set_lock is not held and the task is not existing and rcu read_lock is not held, a warning will be issued. Note that bpf sleepable program is protected by rcu_read_lock_trace().

To fix the issue, let us make these two helpers not available to sleepable program. I marked the patch fixing 95b861a7935b ("bpf: Allow bpf_get_current_ancestor_cgroup_id for tracing") which added bpf_get_current_ancestor_cgroup_id() to 5.14. I think backporting 5.14 is probably good enough as sleepable progrems are not widely used.

This patch should fix [1] as well since syscall program is a sleepable program and bpf_get_current_cgroup_id() is not available to syscall program any more.

[1] https://lore.kernel.org/bpf/[email protected]/

Reported-by [email protected]
Fixes 95b861a7935b ("bpf: Allow bpf_get_current_ancestor_cgroup_id for tracing")
---
 kernel/trace/bpf_trace.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
kernel/trace/bpf_trace.c +0 -0
index b4916ef388ad..eaa8a8ffbe46 100644

kernel/trace/bpf_trace.c bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) · line 1023

Quoted 10 lines
kernel/trace/bpf_trace.c +4 -2
@@ -1016,9 +1016,11 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 #endif
 #ifdef CONFIG_CGROUPS
        case BPF_FUNC_get_current_cgroup_id:
-               return &bpf_get_current_cgroup_id_proto;
+               return prog->aux->sleepable ?
+                      NULL : &bpf_get_current_cgroup_id_proto;
        case BPF_FUNC_get_current_ancestor_cgroup_id:
-               return &bpf_get_current_ancestor_cgroup_id_proto;
+               return prog->aux->sleepable ?
+                      NULL : &bpf_get_current_ancestor_cgroup_id_proto;

This feels too extreme. I bet these helpers are as useful in sleepable BPF progs as they are in non-sleepable ones.

Why don't we just implement a variant of get_current_cgroup_id (and the ancestor variant as well) which takes that cgroup_mutex lock, and just pick the appropriate implementation. Wouldn't that work?

Quoted 6 lines
kernel/trace/bpf_trace.c +0 -1
 #endif
        case BPF_FUNC_send_signal:
                return &bpf_send_signal_proto;
--

2.30.2

#3 lore raw

Re: [PATCH bpf v2 1/2] bpf: don't call bpf_get_current_[ancestor_]cgroup_id() in sleepable progs

Quoted 100 lines · Andrii Nakryiko
Quoted 85 lines · Yonghong Song

Currently, if bpf_get_current_cgroup_id() or bpf_get_current_ancestor_cgroup_id() helper is called with sleepable programs e.g., sleepable fentry/fmod_ret/fexit/lsm programs, a rcu warning may appear. For example, if I added the following hack to test_progs/test_lsm sleepable fentry program test_sys_setdomainname:

   --- a/tools/testing/selftests/bpf/progs/lsm.c
   +++ b/tools/testing/selftests/bpf/progs/lsm.c
   @@ -168,6 +168,10 @@ int BPF_PROG(test_sys_setdomainname, struct pt_regs *regs)
           int buf = 0;
           long ret;
   +       __u64 cg_id = bpf_get_current_cgroup_id();
   +       if (cg_id == 1000)
   +               copy_test++;
   +
           ret = bpf_copy_from_user(&buf, sizeof(buf), ptr);
           if (len == -2 && ret == 0 && buf == 1234)
                   copy_test++;

I will hit the following rcu warning:

   include/linux/cgroup.h:481 suspicious rcu_dereference_check() usage!
   other info that might help us debug this:
     rcu_scheduler_active = 2, debug_locks = 1
     1 lock held by test_progs/260:
       #0: ffffffffa5173360 (rcu_read_lock_trace){....}-{0:0}, at: __bpf_prog_enter_sleepable+0x0/0xa0
     stack backtrace:
     CPU: 1 PID: 260 Comm: test_progs Tainted: G           O      5.14.0-rc2+ #176
     Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
     Call Trace:
       dump_stack_lvl+0x56/0x7b
       bpf_get_current_cgroup_id+0x9c/0xb1
       bpf_prog_a29888d1c6706e09_test_sys_setdomainname+0x3e/0x89c
       bpf_trampoline_6442469132_0+0x2d/0x1000
       __x64_sys_setdomainname+0x5/0x110
       do_syscall_64+0x3a/0x80
       entry_SYSCALL_64_after_hwframe+0x44/0xae

I can get similar warning using bpf_get_current_ancestor_cgroup_id() helper. syzbot reported a similar issue in [1] for syscall program. Helper bpf_get_current_cgroup_id() or bpf_get_current_ancestor_cgroup_id() has the following callchain:
task_dfl_cgroup
task_css_set
task_css_set_check
and we have

    #define task_css_set_check(task, __c)                                   \
            rcu_dereference_check((task)->cgroups,                          \
                    lockdep_is_held(&cgroup_mutex) ||                       \
                    lockdep_is_held(&css_set_lock) ||                       \
                    ((task)->flags & PF_EXITING) || (__c))

Since cgroup_mutex/css_set_lock is not held and the task is not existing and rcu read_lock is not held, a warning will be issued. Note that bpf sleepable program is protected by rcu_read_lock_trace().

To fix the issue, let us make these two helpers not available to sleepable program. I marked the patch fixing 95b861a7935b ("bpf: Allow bpf_get_current_ancestor_cgroup_id for tracing") which added bpf_get_current_ancestor_cgroup_id() to 5.14. I think backporting 5.14 is probably good enough as sleepable progrems are not widely used.

This patch should fix [1] as well since syscall program is a sleepable program and bpf_get_current_cgroup_id() is not available to syscall program any more.

[1] https://lore.kernel.org/bpf/[email protected]/

Reported-by [email protected]
Fixes 95b861a7935b ("bpf: Allow bpf_get_current_ancestor_cgroup_id for tracing")
---
  kernel/trace/bpf_trace.c | 6 ++++--
  1 file changed, 4 insertions(+), 2 deletions(-)
kernel/trace/bpf_trace.c +0 -0
index b4916ef388ad..eaa8a8ffbe46 100644

kernel/trace/bpf_trace.c bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) · line 1023

Quoted 10 lines
kernel/trace/bpf_trace.c +4 -2
@@ -1016,9 +1016,11 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
  #endif
  #ifdef CONFIG_CGROUPS
         case BPF_FUNC_get_current_cgroup_id:
-               return &bpf_get_current_cgroup_id_proto;
+               return prog->aux->sleepable ?
+                      NULL : &bpf_get_current_cgroup_id_proto;
         case BPF_FUNC_get_current_ancestor_cgroup_id:
-               return &bpf_get_current_ancestor_cgroup_id_proto;
+               return prog->aux->sleepable ?
+                      NULL : &bpf_get_current_ancestor_cgroup_id_proto;

This feels too extreme. I bet these helpers are as useful in sleepable BPF progs as they are in non-sleepable ones.

Quoted 3 lines

Why don't we just implement a variant of get_current_cgroup_id (and the ancestor variant as well) which takes that cgroup_mutex lock, and just pick the appropriate implementation. Wouldn't that work?

This may not work. e.g., for sleepable fentry program, if the to-be-traced function is inside in cgroup_mutex, we will have a deadlock.

Currently, affected program types are tracing/fentry.s, tracing/fexit.s, tracing/fmod_ret.s, lsm.s and syscall. For fmod_ret.s, lsm.s, they all have
some kind of predefined attachment/context, we might be able to check all potential attachment points and allow these two helpers when attachment point is not surrounded by cgroup_mutex.
For syscall program, we should be okay as it is called with bpf_prog_test_run interface but I am not sure why user wants a cgroup_id for that.

Quoted 7 lines
Quoted 7 lines
kernel/trace/bpf_trace.c +0 -1
  #endif
         case BPF_FUNC_send_signal:
                 return &bpf_send_signal_proto;
--

2.30.2

#4 lore raw

Re: [PATCH bpf v2 1/2] bpf: don't call bpf_get_current_[ancestor_]cgroup_id() in sleepable progs

Quoted 104 lines · Yonghong Song
Quoted 100 lines · Andrii Nakryiko
Quoted 85 lines · Yonghong Song

Currently, if bpf_get_current_cgroup_id() or bpf_get_current_ancestor_cgroup_id() helper is called with sleepable programs e.g., sleepable fentry/fmod_ret/fexit/lsm programs, a rcu warning may appear. For example, if I added the following hack to test_progs/test_lsm sleepable fentry program test_sys_setdomainname:

   --- a/tools/testing/selftests/bpf/progs/lsm.c
   +++ b/tools/testing/selftests/bpf/progs/lsm.c
   @@ -168,6 +168,10 @@ int BPF_PROG(test_sys_setdomainname, struct pt_regs *regs)
           int buf = 0;
           long ret;
   +       __u64 cg_id = bpf_get_current_cgroup_id();
   +       if (cg_id == 1000)
   +               copy_test++;
   +
           ret = bpf_copy_from_user(&buf, sizeof(buf), ptr);
           if (len == -2 && ret == 0 && buf == 1234)
                   copy_test++;

I will hit the following rcu warning:

   include/linux/cgroup.h:481 suspicious rcu_dereference_check() usage!
   other info that might help us debug this:
     rcu_scheduler_active = 2, debug_locks = 1
     1 lock held by test_progs/260:
       #0: ffffffffa5173360 (rcu_read_lock_trace){....}-{0:0}, at: __bpf_prog_enter_sleepable+0x0/0xa0
     stack backtrace:
     CPU: 1 PID: 260 Comm: test_progs Tainted: G           O      5.14.0-rc2+ #176
     Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
     Call Trace:
       dump_stack_lvl+0x56/0x7b
       bpf_get_current_cgroup_id+0x9c/0xb1
       bpf_prog_a29888d1c6706e09_test_sys_setdomainname+0x3e/0x89c
       bpf_trampoline_6442469132_0+0x2d/0x1000
       __x64_sys_setdomainname+0x5/0x110
       do_syscall_64+0x3a/0x80
       entry_SYSCALL_64_after_hwframe+0x44/0xae

I can get similar warning using bpf_get_current_ancestor_cgroup_id() helper. syzbot reported a similar issue in [1] for syscall program. Helper bpf_get_current_cgroup_id() or bpf_get_current_ancestor_cgroup_id() has the following callchain:
task_dfl_cgroup
task_css_set
task_css_set_check
and we have

    #define task_css_set_check(task, __c)                                   \
            rcu_dereference_check((task)->cgroups,                          \
                    lockdep_is_held(&cgroup_mutex) ||                       \
                    lockdep_is_held(&css_set_lock) ||                       \
                    ((task)->flags & PF_EXITING) || (__c))

Since cgroup_mutex/css_set_lock is not held and the task is not existing and rcu read_lock is not held, a warning will be issued. Note that bpf sleepable program is protected by rcu_read_lock_trace().

To fix the issue, let us make these two helpers not available to sleepable program. I marked the patch fixing 95b861a7935b ("bpf: Allow bpf_get_current_ancestor_cgroup_id for tracing") which added bpf_get_current_ancestor_cgroup_id() to 5.14. I think backporting 5.14 is probably good enough as sleepable progrems are not widely used.

This patch should fix [1] as well since syscall program is a sleepable program and bpf_get_current_cgroup_id() is not available to syscall program any more.

[1] https://lore.kernel.org/bpf/[email protected]/

Reported-by [email protected]
Fixes 95b861a7935b ("bpf: Allow bpf_get_current_ancestor_cgroup_id for tracing")
---
  kernel/trace/bpf_trace.c | 6 ++++--
  1 file changed, 4 insertions(+), 2 deletions(-)
kernel/trace/bpf_trace.c +0 -0
index b4916ef388ad..eaa8a8ffbe46 100644

kernel/trace/bpf_trace.c bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) · line 1023

Quoted 10 lines
kernel/trace/bpf_trace.c +4 -2
@@ -1016,9 +1016,11 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
  #endif
  #ifdef CONFIG_CGROUPS
         case BPF_FUNC_get_current_cgroup_id:
-               return &bpf_get_current_cgroup_id_proto;
+               return prog->aux->sleepable ?
+                      NULL : &bpf_get_current_cgroup_id_proto;
         case BPF_FUNC_get_current_ancestor_cgroup_id:
-               return &bpf_get_current_ancestor_cgroup_id_proto;
+               return prog->aux->sleepable ?
+                      NULL : &bpf_get_current_ancestor_cgroup_id_proto;

This feels too extreme. I bet these helpers are as useful in sleepable BPF progs as they are in non-sleepable ones.

Quoted 7 lines
Quoted 3 lines

Why don't we just implement a variant of get_current_cgroup_id (and the ancestor variant as well) which takes that cgroup_mutex lock, and just pick the appropriate implementation. Wouldn't that work?

This may not work. e.g., for sleepable fentry program, if the to-be-traced function is inside in cgroup_mutex, we will have a deadlock.

We can also do preempty_disable() + rcu_read_lock() inside the helper itself, no? I mean in the new "sleepable" variant.

Quoted 8 lines

Currently, affected program types are tracing/fentry.s, tracing/fexit.s, tracing/fmod_ret.s, lsm.s and syscall. For fmod_ret.s, lsm.s, they all have
some kind of predefined attachment/context, we might be able to check all potential attachment points and allow these two helpers when attachment point is not surrounded by cgroup_mutex.

I don't think it's feasible to know if any given attached kernel function can be called with cgroup_mutex taken. Static analysis will be too complicated and too restrictive. Runtime checks might be too expensive and/or not generic enough. But see above, we can do rcu_read_lock() inside the helper while preventing preemption, and it will behave the same way as if it was called from non-sleepable BPF prog.

Quoted 11 lines

For syscall program, we should be okay as it is called with bpf_prog_test_run interface but I am not sure why user wants a cgroup_id for that.

Quoted 7 lines
Quoted 7 lines
kernel/trace/bpf_trace.c +0 -1
  #endif
         case BPF_FUNC_send_signal:
                 return &bpf_send_signal_proto;
--

2.30.2

maintainer
#5 lore raw

Re: [PATCH bpf v2 1/2] bpf: don't call bpf_get_current_[ancestor_]cgroup_id() in sleepable progs

Quoted 106 lines · Andrii Nakryiko
Quoted 105 lines · Yonghong Song
Quoted 103 lines · Andrii Nakryiko
Quoted 85 lines · Yonghong Song

Currently, if bpf_get_current_cgroup_id() or bpf_get_current_ancestor_cgroup_id() helper is called with sleepable programs e.g., sleepable fentry/fmod_ret/fexit/lsm programs, a rcu warning may appear. For example, if I added the following hack to test_progs/test_lsm sleepable fentry program test_sys_setdomainname:

    --- a/tools/testing/selftests/bpf/progs/lsm.c
    +++ b/tools/testing/selftests/bpf/progs/lsm.c
    @@ -168,6 +168,10 @@ int BPF_PROG(test_sys_setdomainname, struct pt_regs *regs)
            int buf = 0;
            long ret;

    +       __u64 cg_id = bpf_get_current_cgroup_id();
    +       if (cg_id == 1000)
    +               copy_test++;
    +
            ret = bpf_copy_from_user(&buf, sizeof(buf), ptr);
            if (len == -2 && ret == 0 && buf == 1234)
                    copy_test++;

I will hit the following rcu warning:

    include/linux/cgroup.h:481 suspicious rcu_dereference_check() usage!
    other info that might help us debug this:
      rcu_scheduler_active = 2, debug_locks = 1
      1 lock held by test_progs/260:
        #0: ffffffffa5173360 (rcu_read_lock_trace){....}-{0:0}, at: __bpf_prog_enter_sleepable+0x0/0xa0
      stack backtrace:
      CPU: 1 PID: 260 Comm: test_progs Tainted: G           O      5.14.0-rc2+ #176
      Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
      Call Trace:
        dump_stack_lvl+0x56/0x7b
        bpf_get_current_cgroup_id+0x9c/0xb1
        bpf_prog_a29888d1c6706e09_test_sys_setdomainname+0x3e/0x89c
        bpf_trampoline_6442469132_0+0x2d/0x1000
        __x64_sys_setdomainname+0x5/0x110
        do_syscall_64+0x3a/0x80
        entry_SYSCALL_64_after_hwframe+0x44/0xae

I can get similar warning using bpf_get_current_ancestor_cgroup_id() helper. syzbot reported a similar issue in [1] for syscall program. Helper bpf_get_current_cgroup_id() or bpf_get_current_ancestor_cgroup_id() has the following callchain:
task_dfl_cgroup
task_css_set
task_css_set_check
and we have

     #define task_css_set_check(task, __c)                                   \
             rcu_dereference_check((task)->cgroups,                          \
                     lockdep_is_held(&cgroup_mutex) ||                       \
                     lockdep_is_held(&css_set_lock) ||                       \
                     ((task)->flags & PF_EXITING) || (__c))

Since cgroup_mutex/css_set_lock is not held and the task is not existing and rcu read_lock is not held, a warning will be issued. Note that bpf sleepable program is protected by rcu_read_lock_trace().

To fix the issue, let us make these two helpers not available to sleepable program. I marked the patch fixing 95b861a7935b ("bpf: Allow bpf_get_current_ancestor_cgroup_id for tracing") which added bpf_get_current_ancestor_cgroup_id() to 5.14. I think backporting 5.14 is probably good enough as sleepable progrems are not widely used.

This patch should fix [1] as well since syscall program is a sleepable program and bpf_get_current_cgroup_id() is not available to syscall program any more.

   [1] https://lore.kernel.org/bpf/[email protected]/
Reported-by [email protected]
Fixes 95b861a7935b ("bpf: Allow bpf_get_current_ancestor_cgroup_id for tracing")
---
   kernel/trace/bpf_trace.c | 6 ++++--
   1 file changed, 4 insertions(+), 2 deletions(-)
kernel/trace/bpf_trace.c +0 -0
index b4916ef388ad..eaa8a8ffbe46 100644

kernel/trace/bpf_trace.c bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) · line 1023

Quoted 10 lines
kernel/trace/bpf_trace.c +4 -2
@@ -1016,9 +1016,11 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
   #endif
   #ifdef CONFIG_CGROUPS
          case BPF_FUNC_get_current_cgroup_id:
-               return &bpf_get_current_cgroup_id_proto;
+               return prog->aux->sleepable ?
+                      NULL : &bpf_get_current_cgroup_id_proto;
          case BPF_FUNC_get_current_ancestor_cgroup_id:
-               return &bpf_get_current_ancestor_cgroup_id_proto;
+               return prog->aux->sleepable ?
+                      NULL : &bpf_get_current_ancestor_cgroup_id_proto;

This feels too extreme. I bet these helpers are as useful in sleepable BPF progs as they are in non-sleepable ones.

Why don't we just implement a variant of get_current_cgroup_id (and the ancestor variant as well) which takes that cgroup_mutex lock, and just pick the appropriate implementation. Wouldn't that work?

Quoted 6 lines
Quoted 3 lines

This may not work. e.g., for sleepable fentry program, if the to-be-traced function is inside in cgroup_mutex, we will have a deadlock.

We can also do preempty_disable() + rcu_read_lock() inside the helper itself, no? I mean in the new "sleepable" variant.

Yep, we do that for example in c5dbb89fc2ac ("bpf: Expose bpf_get_socket_cookie to tracing programs") as well (the sock_gen_cookie() disables preemption).

Thanks,
Daniel

#6 lore raw

Re: [PATCH bpf v2 1/2] bpf: don't call bpf_get_current_[ancestor_]cgroup_id() in sleepable progs

Quoted 114 lines · Daniel Borkmann
Quoted 113 lines · Andrii Nakryiko
Quoted 111 lines · Yonghong Song
Quoted 103 lines · Andrii Nakryiko
Quoted 90 lines · Yonghong Song

Currently, if bpf_get_current_cgroup_id() or bpf_get_current_ancestor_cgroup_id() helper is called with sleepable programs e.g., sleepable fentry/fmod_ret/fexit/lsm programs, a rcu warning may appear. For example, if I added the following hack to test_progs/test_lsm sleepable fentry program test_sys_setdomainname:

    --- a/tools/testing/selftests/bpf/progs/lsm.c
    +++ b/tools/testing/selftests/bpf/progs/lsm.c
    @@ -168,6 +168,10 @@ int BPF_PROG(test_sys_setdomainname, struct pt_regs *regs) int buf = 0;
            long ret;

    +       __u64 cg_id = bpf_get_current_cgroup_id(); +       if (cg_id == 1000)
    +               copy_test++;
    +
            ret = bpf_copy_from_user(&buf, sizeof(buf), ptr); if (len == -2 && ret == 0 && buf == 1234) copy_test++;

I will hit the following rcu warning:

    include/linux/cgroup.h:481 suspicious rcu_dereference_check() usage!
    other info that might help us debug this:
      rcu_scheduler_active = 2, debug_locks = 1
      1 lock held by test_progs/260:
        #0: ffffffffa5173360 (rcu_read_lock_trace){....}-{0:0}, at: __bpf_prog_enter_sleepable+0x0/0xa0

      stack backtrace:
      CPU: 1 PID: 260 Comm: test_progs Tainted: G           O      5.14.0-rc2+ #176
      Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
      Call Trace:
        dump_stack_lvl+0x56/0x7b
        bpf_get_current_cgroup_id+0x9c/0xb1
        bpf_prog_a29888d1c6706e09_test_sys_setdomainname+0x3e/0x89c
        bpf_trampoline_6442469132_0+0x2d/0x1000
        __x64_sys_setdomainname+0x5/0x110
        do_syscall_64+0x3a/0x80
        entry_SYSCALL_64_after_hwframe+0x44/0xae

I can get similar warning using bpf_get_current_ancestor_cgroup_id() helper. syzbot reported a similar issue in [1] for syscall program. Helper bpf_get_current_cgroup_id() or bpf_get_current_ancestor_cgroup_id() has the following callchain:
     task_dfl_cgroup
       task_css_set
         task_css_set_check
and we have
     #define task_css_set_check(task, __c)                                   \

rcu_dereference_check((task)->cgroups,                          \

                     lockdep_is_held(&cgroup_mutex) ||                       \
                     lockdep_is_held(&css_set_lock) ||                       \

                     ((task)->flags & PF_EXITING) || (__c)) Since cgroup_mutex/css_set_lock is not held and the task is not existing and rcu read_lock is not held, a warning will be issued. Note that bpf sleepable program is protected by rcu_read_lock_trace().

To fix the issue, let us make these two helpers not available to sleepable program. I marked the patch fixing 95b861a7935b ("bpf: Allow bpf_get_current_ancestor_cgroup_id for tracing") which added bpf_get_current_ancestor_cgroup_id() to 5.14. I think backporting 5.14 is probably good enough as sleepable progrems are not widely used.

This patch should fix [1] as well since syscall program is a sleepable program and bpf_get_current_cgroup_id() is not available to syscall program any more.

   [1] https://lore.kernel.org/bpf/[email protected]/

Reported-by [email protected]
Fixes 95b861a7935b ("bpf: Allow bpf_get_current_ancestor_cgroup_id for tracing")

---
   kernel/trace/bpf_trace.c | 6 ++++--

   1 file changed, 4 insertions(+), 2 deletions(-)
kernel/trace/bpf_trace.c +0 -0
index b4916ef388ad..eaa8a8ffbe46 100644
@@ -1016,9 +1016,11 @@ bpf_tracing_func_proto(enum bpf_func_id 
 func_id, const struct bpf_prog *prog)
    #endif
    #ifdef CONFIG_CGROUPS

          case BPF_FUNC_get_current_cgroup_id:

Quoted 8 lines

-               return &bpf_get_current_cgroup_id_proto; +               return prog->aux->sleepable ?
+                      NULL : &bpf_get_current_cgroup_id_proto; case BPF_FUNC_get_current_ancestor_cgroup_id:
-               return &bpf_get_current_ancestor_cgroup_id_proto; +               return prog->aux->sleepable ?
+                      NULL :
&bpf_get_current_ancestor_cgroup_id_proto;

This feels too extreme. I bet these helpers are as useful in sleepable BPF progs as they are in non-sleepable ones.

Quoted 3 lines

Why don't we just implement a variant of get_current_cgroup_id (and the ancestor variant as well) which takes that cgroup_mutex lock, and just pick the appropriate implementation. Wouldn't that work?

This may not work. e.g., for sleepable fentry program, if the to-be-traced function is inside in cgroup_mutex, we will have a deadlock.

Quoted 5 lines
Quoted 2 lines

We can also do preempty_disable() + rcu_read_lock() inside the helper itself, no? I mean in the new "sleepable" variant.

Yep, we do that for example in c5dbb89fc2ac ("bpf: Expose bpf_get_socket_cookie to tracing programs") as well (the sock_gen_cookie() disables preemption).

I think rcu_read_lock() is enough. For all the cases I mentioned in the above, we already have migrate_disable(). Together with rcu_read_lock(), we should be okay. For non-sleepable programs (tracing,lsm), the helpers
are protected with migrate_disable() and rcu_read_lock().

Quoted 3 lines

Thanks,
Daniel

#7 lore raw

PATCH v2 2/2 bpf: add missing bpf_read_[un]lock_trace() for syscall program

In reply to Yonghong Song

Commit 79a7f8bdb159d ("bpf: Introduce bpf_sys_bpf() helper and program type.") added support for syscall program, which is a sleepable program. But the program run missed bpf_read_lock_trace()/bpf_read_unlock_trace(), which is needed to ensure proper rcu callback invocations.
This patch added bpf_read_[un]lock_trace() properly.

Fixes 79a7f8bdb159d ("bpf: Introduce bpf_sys_bpf() helper and program type.")
---
 net/bpf/test_run.c | 4 ++++
 1 file changed, 4 insertions(+)
net/bpf/test_run.c +4 -0
index 1cc75c811e24..caa16bf30fb5 100644
@@ -7,6 +7,7 @@
 #include <linux/vmalloc.h>
 #include <linux/etherdevice.h>
 #include <linux/filter.h>
+#include <linux/rcupdate_trace.h>
 #include <linux/sched/signal.h>
 #include <net/bpf_sk_storage.h>
 #include <net/sock.h>
@@ -951,7 +952,10 @@ int bpf_prog_test_run_syscall(struct bpf_prog *prog,
 			goto out;
 		}
 	}
+
+	rcu_read_lock_trace();
 	retval = bpf_prog_run_pin_on_cpu(prog, ctx);
+	rcu_read_unlock_trace();
 
 	if (copy_to_user(&uattr->test.retval, &retval, sizeof(u32))) {
 		err = -EFAULT;
Signature · 1 line
2.30.2
Acked-by on v2 2/2
#8 lore raw

Re: [PATCH bpf v2 2/2] bpf: add missing bpf_read_[un]lock_trace() for syscall program

Quoted 10 lines · Yonghong Song

Commit 79a7f8bdb159d ("bpf: Introduce bpf_sys_bpf() helper and program type.") added support for syscall program, which is a sleepable program. But the program run missed bpf_read_lock_trace()/bpf_read_unlock_trace(), which is needed to ensure proper rcu callback invocations.
This patch added bpf_read_[un]lock_trace() properly.

Fixes 79a7f8bdb159d ("bpf: Introduce bpf_sys_bpf() helper and program type.")

---

LGTM.

Quoted 29 lines
 net/bpf/test_run.c | 4 ++++
 1 file changed, 4 insertions(+)
net/bpf/test_run.c +4 -1
index 1cc75c811e24..caa16bf30fb5 100644
@@ -7,6 +7,7 @@
 #include <linux/vmalloc.h>
 #include <linux/etherdevice.h>
 #include <linux/filter.h>
+#include <linux/rcupdate_trace.h>
 #include <linux/sched/signal.h>
 #include <net/bpf_sk_storage.h>
 #include <net/sock.h>
@@ -951,7 +952,10 @@ int bpf_prog_test_run_syscall(struct bpf_prog *prog,
                        goto out;
                }
        }
+
+       rcu_read_lock_trace();
        retval = bpf_prog_run_pin_on_cpu(prog, ctx);
+       rcu_read_unlock_trace();
 
        if (copy_to_user(&uattr->test.retval, &retval, sizeof(u32))) {
                err = -EFAULT;
--

2.30.2