Skip to contentSkip to conversation

Always allow sleepable and fmod_ret programs on syscalls

PATCH Superseded Viktor Malik 4 participants 15 messages Mar 5 – Mar 6

Subsystems (4)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

cover Always allow sleepable and fmod_ret programs on syscalls

Both sleepable and fmod_ret programs are only allowed on selected functions. For convenience, the error injection list was originally used.

When error injection is disabled, that list is empty and sleepable tracing programs, as well as fmod_ret programs, are effectively unavailable.

This patch series addresses the issue by at least enabling sleepable and fmod_ret programs on syscalls, if error injection is disabled.

Viktor Malik (3):
bpf: Always allow sleepable programs on syscalls
bpf: Always allow fmod_ret programs on syscalls selftests/bpf: Move sleepable refcounted_kptr tests to syscalls

 kernel/bpf/verifier.c                         | 79 +++++++++++++++----
 .../selftests/bpf/progs/refcounted_kptr.c     |  4 +-
 .../bpf/progs/refcounted_kptr_fail.c          |  2 +-
 3 files changed, 68 insertions(+), 17 deletions(-)
Signature · 1 line
2.53.0
#1 lore raw

PATCH 1/3 bpf: Always allow sleepable programs on syscalls

Sleepable BPF programs can only be attached to selected functions. For convenience, the error injection list was originally used, which contains syscalls and several other functions.

When error injection is disabled (CONFIG_FUNCTION_ERROR_INJECTION=n), that list is empty and sleepable tracing programs are effectively unavailable. In such a case, at least enable sleepable programs on syscalls.

---
 kernel/bpf/verifier.c | 54 ++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 48 insertions(+), 6 deletions(-)
kernel/bpf/verifier.c +48 -6
index d92cf2821657..cb80a3ddb7a4 100644
@@ -24930,6 +24930,8 @@ static int check_attach_modify_return(unsigned long addr, const char *func_name)
 	return -EINVAL;
 }
 
+#ifdef CONFIG_FUNCTION_ERROR_INJECTION
+
 /* list of non-sleepable functions that are otherwise on
  * ALLOW_ERROR_INJECTION list
  */
@@ -24951,6 +24953,51 @@ static int check_non_sleepable_error_inject(u32 btf_id)
 	return btf_id_set_contains(&btf_non_sleepable_error_inject, btf_id);
 }
 
+static int check_attach_sleepable(u32 btf_id, unsigned long addr, const char *func_name)
+{
+	/* fentry/fexit/fmod_ret progs can be sleepable if they are
+	 * attached to ALLOW_ERROR_INJECTION and are not in denylist.
+	 */
+	if (!check_non_sleepable_error_inject(btf_id) &&
+	    within_error_injection_list(addr))
+		return 0;
+
+	return -EINVAL;
+}
+
+#else
+
+static bool has_arch_syscall_prefix(const char *func_name)
+{
+#if defined(__x86_64__)
+	return !strncmp(func_name, "__x64_", 6);
+#elif defined(__i386__)
+	return !strncmp(func_name, "__ia32_", 7);
+#elif defined(__s390x__)
+	return !strncmp(func_name, "__s390x_", 8);
+#elif defined(__aarch64__)
+	return !strncmp(func_name, "__arm64_", 8);
+#elif defined(__riscv)
+	return !strncmp(func_name, "__riscv_", 8);
+#elif defined(__powerpc__) || defined(__powerpc64__)
+	return !strncmp(func_name, "sys", 3);
+#else
+	return false;
+#endif
+}
+
+/* Without error injection, allow sleepable progs on syscalls. */
+
+static int check_attach_sleepable(u32 btf_id, unsigned long addr, const char *func_name)
+{
+	if (has_arch_syscall_prefix(func_name))
+		return 0;
+
+	return -EINVAL;
+}
+
+#endif /* CONFIG_FUNCTION_ERROR_INJECTION */
+
 int bpf_check_attach_target(struct bpf_verifier_log *log,
 			    const struct bpf_prog *prog,
 			    const struct bpf_prog *tgt_prog,
@@ -25230,12 +25277,7 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,
 			ret = -EINVAL;
 			switch (prog->type) {
 			case BPF_PROG_TYPE_TRACING:
-
-				/* fentry/fexit/fmod_ret progs can be sleepable if they are
-				 * attached to ALLOW_ERROR_INJECTION and are not in denylist.
-				 */
-				if (!check_non_sleepable_error_inject(btf_id) &&
-				    within_error_injection_list(addr))
+				if (!check_attach_sleepable(btf_id, addr, tname))
 					ret = 0;
 				/* fentry/fexit/fmod_ret progs can also be sleepable if they are
 				 * in the fmodret id set with the KF_SLEEPABLE flag.
Signature · 1 line
2.53.0
#9 lore raw

PATCH 2/3 bpf: Always allow fmod_ret programs on syscalls

In reply to Viktor Malik

fmod_ret BPF programs can only be attached to selected functions. For convenience, the error injection list was originally used (along with functions prefixed with "security_"), which contains syscalls and several other functions.

When error injection is disabled (CONFIG_FUNCTION_ERROR_INJECTION=n), that list is empty and fmod_ret programs are effectively unavailable for most of the functions. In such a case, at least enable fmod_ret programs on syscalls.

---
 kernel/bpf/verifier.c | 29 +++++++++++++++++++----------
 1 file changed, 19 insertions(+), 10 deletions(-)
kernel/bpf/verifier.c +19 -10
index cb80a3ddb7a4..6883b18f1355 100644
@@ -24921,15 +24921,6 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env)
 }
 #define SECURITY_PREFIX "security_"
 
-static int check_attach_modify_return(unsigned long addr, const char *func_name)
-{
-	if (within_error_injection_list(addr) ||
-	    !strncmp(SECURITY_PREFIX, func_name, sizeof(SECURITY_PREFIX) - 1))
-		return 0;
-
-	return -EINVAL;
-}
-
 #ifdef CONFIG_FUNCTION_ERROR_INJECTION
 
 /* list of non-sleepable functions that are otherwise on
@@ -24965,6 +24956,15 @@ static int check_attach_sleepable(u32 btf_id, unsigned long addr, const char *fu
 	return -EINVAL;
 }
 
+static int check_attach_modify_return(unsigned long addr, const char *func_name)
+{
+	if (within_error_injection_list(addr) ||
+	    !strncmp(SECURITY_PREFIX, func_name, sizeof(SECURITY_PREFIX) - 1))
+		return 0;
+
+	return -EINVAL;
+}
+
 #else
 
 static bool has_arch_syscall_prefix(const char *func_name)
@@ -24986,7 +24986,7 @@ static bool has_arch_syscall_prefix(const char *func_name)
 #endif
 }
 
-/* Without error injection, allow sleepable progs on syscalls. */
+/* Without error injection, allow sleepable and fmod_ret progs on syscalls. */
 
 static int check_attach_sleepable(u32 btf_id, unsigned long addr, const char *func_name)
 {
@@ -24996,6 +24996,15 @@ static int check_attach_sleepable(u32 btf_id, unsigned long addr, const char *fu
 	return -EINVAL;
 }
 
+static int check_attach_modify_return(unsigned long addr, const char *func_name)
+{
+	if (has_arch_syscall_prefix(func_name) ||
+	    !strncmp(SECURITY_PREFIX, func_name, sizeof(SECURITY_PREFIX) - 1))
+		return 0;
+
+	return -EINVAL;
+}
+
 #endif /* CONFIG_FUNCTION_ERROR_INJECTION */
 
 int bpf_check_attach_target(struct bpf_verifier_log *log,
Signature · 1 line
2.53.0
#10 lore raw

PATCH 3/3 selftests/bpf: Move sleepable refcounted_kptr tests to syscalls

In reply to Viktor Malik

Now that sleepable programs are always enabled on syscalls, let refcounted_kptr tests use syscalls rather than bpf_testmod_test_read, which is not sleepable with error injection disabled.

The tests just check that the verifier can handle usage of RCU locks in sleepable programs and never actually attach. So, the attachment target doesn't matter (as long as it is sleepable) and with syscalls, the tests pass on kernels with disabled error injection.

---
 tools/testing/selftests/bpf/progs/refcounted_kptr.c      | 4 ++--
 tools/testing/selftests/bpf/progs/refcounted_kptr_fail.c | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)
tools/testing/selftests/bpf/progs/refcounted_kptr.c +2 -2
index 1aca85d86aeb..c847398837cc 100644
@@ -500,7 +500,7 @@ long rbtree_wrong_owner_remove_fail_a2(void *ctx)
 	return 0;
 }
 
-SEC("?fentry.s/bpf_testmod_test_read")
+SEC("?fentry.s/" SYS_PREFIX "sys_getpgid")
 __success
 int BPF_PROG(rbtree_sleepable_rcu,
 	     struct file *file, struct kobject *kobj,
@@ -534,7 +534,7 @@ int BPF_PROG(rbtree_sleepable_rcu,
 	return 0;
 }
 
-SEC("?fentry.s/bpf_testmod_test_read")
+SEC("?fentry.s/" SYS_PREFIX "sys_getpgid")
 __success
 int BPF_PROG(rbtree_sleepable_rcu_no_explicit_rcu_lock,
 	     struct file *file, struct kobject *kobj,
tools/testing/selftests/bpf/progs/refcounted_kptr_fail.c +1 -1
index 836c8ab7b908..b2808bfcec29 100644
@@ -93,7 +93,7 @@ long rbtree_refcounted_node_ref_escapes_owning_input(void *ctx)
 	return 0;
 }
 
-SEC("?fentry.s/bpf_testmod_test_read")
+SEC("?fentry.s/" SYS_PREFIX "sys_getpgid")
 __failure __msg("function calls are not allowed while holding a lock")
 int BPF_PROG(rbtree_fail_sleepable_lock_across_rcu,
 	     struct file *file, struct kobject *kobj,
Signature · 1 line
2.53.0
#2 lore raw

Re: [PATCH bpf-next 1/3] bpf: Always allow sleepable programs on syscalls

In reply to Viktor Malik
The other architectures use prefixes that uniquely identify syscall wrappers (__x64_sys_*, __arm64_sys_*, etc.), but "sys" with length 3 would also match sysfs_*, sysctl_*, sysrq_*, and other…
Quoted 4 lines
kernel/bpf/verifier.c +0 -0
index d92cf2821657..cb80a3ddb7a4 100644

[ ... ]

Quoted 6 lines
kernel/bpf/verifier.c +6 -0
+#else
+
+static bool has_arch_syscall_prefix(const char *func_name)
+{
+#if defined(__x86_64__)
+	return !strncmp(func_name, "__x64_", 6);

kernel/bpf/verifier.c static int check_non_sleepable_error_inject(u32 btf_id) · line 24983

Quoted 10 lines
kernel/bpf/verifier.c +10 -0
+#elif defined(__i386__)
+	return !strncmp(func_name, "__ia32_", 7);
+#elif defined(__s390x__)
+	return !strncmp(func_name, "__s390x_", 8);
+#elif defined(__aarch64__)
+	return !strncmp(func_name, "__arm64_", 8);
+#elif defined(__riscv)
+	return !strncmp(func_name, "__riscv_", 8);
+#elif defined(__powerpc__) || defined(__powerpc64__)
+	return !strncmp(func_name, "sys", 3);
                                    ^^^^^

The other architectures use prefixes that uniquely identify syscall wrappers (__x64_sys_*, __arm64_sys_*, etc.), but "sys" with length 3 would also match sysfs_*, sysctl_*, sysrq_*, and other non-syscall kernel functions.

Looking at arch/powerpc/include/asm/syscall_wrapper.h, __SYSCALL_DEFINEx generates names like sys_read and SYSCALL_DEFINE0 generates sys_fork, so the actual common prefix for powerpc syscalls is "sys_" (4 characters with the underscore).

Should this be "sys_" with length 4 instead?

kernel/bpf/verifier.c static int check_non_sleepable_error_inject(u32 btf_id) · line 24987

Quoted 4 lines
kernel/bpf/verifier.c +4 -0
+#else
+	return false;
+#endif
+}

[ ... ]

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/22725431483

#3 lore raw

Re: [PATCH bpf-next 1/3] bpf: Always allow sleepable programs on syscalls

Quoted 31 lines · On Thu, 5 Mar 2026 at 17:08, <[email protected]> wrote:
Quoted 5 lines
kernel/bpf/verifier.c +0 -0
index d92cf2821657..cb80a3ddb7a4 100644

[ ... ]

Quoted 6 lines
kernel/bpf/verifier.c +6 -0
+#else
+
+static bool has_arch_syscall_prefix(const char *func_name)
+{
+#if defined(__x86_64__)
+     return !strncmp(func_name, "__x64_", 6);

kernel/bpf/verifier.c static int check_non_sleepable_error_inject(u32 btf_id) · line 24983

Quoted 10 lines
kernel/bpf/verifier.c +10 -0
+#elif defined(__i386__)
+     return !strncmp(func_name, "__ia32_", 7);
+#elif defined(__s390x__)
+     return !strncmp(func_name, "__s390x_", 8);
+#elif defined(__aarch64__)
+     return !strncmp(func_name, "__arm64_", 8);
+#elif defined(__riscv)
+     return !strncmp(func_name, "__riscv_", 8);
+#elif defined(__powerpc__) || defined(__powerpc64__)
+     return !strncmp(func_name, "sys", 3);
                                    ^^^^^

The other architectures use prefixes that uniquely identify syscall wrappers (__x64_sys_*, __arm64_sys_*, etc.), but "sys" with length 3 would also match sysfs_*, sysctl_*, sysrq_*, and other non-syscall kernel functions.

Quoted 6 lines

Looking at arch/powerpc/include/asm/syscall_wrapper.h, __SYSCALL_DEFINEx generates names like sys_read and SYSCALL_DEFINE0 generates sys_fork, so the actual common prefix for powerpc syscalls is "sys_" (4 characters with the underscore).

Should this be "sys_" with length 4 instead?

I think the bot is right, we should match sys_, that is less likely to include other non-syscall kernel functions.
It only affects powerpc with error injection disabled.

Other than that, the set looks good to me. I tested it by disabling error injection and things fail before your fix is applied.

Quoted 14 lines

kernel/bpf/verifier.c static int check_non_sleepable_error_inject(u32 btf_id) · line 24987

Quoted 5 lines
kernel/bpf/verifier.c +4 -0
+#else
+     return false;
+#endif
+}

[ ... ]

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/22725431483

#11 lore raw
In reply to Viktor Malik

Hi Viktor,

Quoted 11 lines · Viktor Malik

Both sleepable and fmod_ret programs are only allowed on selected functions. For convenience, the error injection list was originally used.

When error injection is disabled, that list is empty and sleepable tracing programs, as well as fmod_ret programs, are effectively unavailable.

This patch series addresses the issue by at least enabling sleepable and fmod_ret programs on syscalls, if error injection is disabled.

Could you please explain more about the purpose in cover letter?

I'm curious about enabling fmod_ret programs on syscalls, as they are able to modify the retval of syscalls.

Thanks,
Leon

Quoted 10 lines

Viktor Malik (3):
bpf: Always allow sleepable programs on syscalls
bpf: Always allow fmod_ret programs on syscalls selftests/bpf: Move sleepable refcounted_kptr tests to syscalls

 kernel/bpf/verifier.c                         | 79 +++++++++++++++----
 .../selftests/bpf/progs/refcounted_kptr.c     |  4 +-
 .../bpf/progs/refcounted_kptr_fail.c          |  2 +-
 3 files changed, 68 insertions(+), 17 deletions(-)
#5 lore raw

Re: [PATCH bpf-next 1/3] bpf: Always allow sleepable programs on syscalls

In reply to Viktor Malik
Quoted 53 lines · Viktor Malik

Sleepable BPF programs can only be attached to selected functions. For convenience, the error injection list was originally used, which contains syscalls and several other functions.

When error injection is disabled (CONFIG_FUNCTION_ERROR_INJECTION=n), that list is empty and sleepable tracing programs are effectively unavailable. In such a case, at least enable sleepable programs on syscalls.

---
 kernel/bpf/verifier.c | 54 ++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 48 insertions(+), 6 deletions(-)
kernel/bpf/verifier.c +24 -0
index d92cf2821657..cb80a3ddb7a4 100644
@@ -24930,6 +24930,8 @@ static int check_attach_modify_return(unsigned long addr, const char *func_name)
 	return -EINVAL;
 }
 
+#ifdef CONFIG_FUNCTION_ERROR_INJECTION
+
 /* list of non-sleepable functions that are otherwise on
  * ALLOW_ERROR_INJECTION list
  */
@@ -24951,6 +24953,51 @@ static int check_non_sleepable_error_inject(u32 btf_id)
 	return btf_id_set_contains(&btf_non_sleepable_error_inject, btf_id);
 }
 
+static int check_attach_sleepable(u32 btf_id, unsigned long addr, const char *func_name)
+{
+	/* fentry/fexit/fmod_ret progs can be sleepable if they are
+	 * attached to ALLOW_ERROR_INJECTION and are not in denylist.
+	 */
+	if (!check_non_sleepable_error_inject(btf_id) &&
+	    within_error_injection_list(addr))
+		return 0;
+
+	return -EINVAL;
+}
+
+#else
+
+static bool has_arch_syscall_prefix(const char *func_name)
+{
+#if defined(__x86_64__)
+	return !strncmp(func_name, "__x64_", 6);
+#elif defined(__i386__)
+	return !strncmp(func_name, "__ia32_", 7);
+#elif defined(__s390x__)
+	return !strncmp(func_name, "__s390x_", 8);

kernel/bpf/verifier.c static int check_non_sleepable_error_inject(u32 btf_id) · line 24987

Quoted 10 lines
kernel/bpf/verifier.c +10 -0
+#elif defined(__aarch64__)
+	return !strncmp(func_name, "__arm64_", 8);
+#elif defined(__riscv)
+	return !strncmp(func_name, "__riscv_", 8);
+#elif defined(__powerpc__) || defined(__powerpc64__)
+	return !strncmp(func_name, "sys", 3);
+#else
+	return false;
+#endif
+}

At first glance, it is not a good taste to hard code these prefixes here.

Furthermore, the arch_syscall_match_sym_name() of ftrace does the same thing. Consider it instead.

Thanks,
Leon

Quoted 30 lines
kernel/bpf/verifier.c +14 -6
+
+/* Without error injection, allow sleepable progs on syscalls. */
+
+static int check_attach_sleepable(u32 btf_id, unsigned long addr, const char *func_name)
+{
+	if (has_arch_syscall_prefix(func_name))
+		return 0;
+
+	return -EINVAL;
+}
+
+#endif /* CONFIG_FUNCTION_ERROR_INJECTION */
+
 int bpf_check_attach_target(struct bpf_verifier_log *log,
 			    const struct bpf_prog *prog,
 			    const struct bpf_prog *tgt_prog,
@@ -25230,12 +25277,7 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,
 			ret = -EINVAL;
 			switch (prog->type) {
 			case BPF_PROG_TYPE_TRACING:
-
-				/* fentry/fexit/fmod_ret progs can be sleepable if they are
-				 * attached to ALLOW_ERROR_INJECTION and are not in denylist.
-				 */
-				if (!check_non_sleepable_error_inject(btf_id) &&
-				    within_error_injection_list(addr))
+				if (!check_attach_sleepable(btf_id, addr, tname))
 					ret = 0;
 				/* fentry/fexit/fmod_ret progs can also be sleepable if they are
 				 * in the fmodret id set with the KF_SLEEPABLE flag.
#12 lore raw
In reply to Leon Hwang
Quoted 12 lines · Leon Hwang

Hi Viktor,

Quoted 8 lines · Viktor Malik

Both sleepable and fmod_ret programs are only allowed on selected functions. For convenience, the error injection list was originally used.

When error injection is disabled, that list is empty and sleepable tracing programs, as well as fmod_ret programs, are effectively unavailable.

Quoted 8 lines
Quoted 3 lines

This patch series addresses the issue by at least enabling sleepable and fmod_ret programs on syscalls, if error injection is disabled.

Could you please explain more about the purpose in cover letter?

I'm curious about enabling fmod_ret programs on syscalls, as they are able to modify the retval of syscalls.

It was already discussed at length here:
https://lore.kernel.org/bpf/CAADnVQJ4PU7bCb5XO9+zMZhLRSdbCKwxG6Rtojh2Rc3UZ8H43Q@mail.gmail.com/

I think the next version could include a link to the previous thread in the cover letter for posterity.

Quoted 16 lines

Thanks,
Leon

Quoted 10 lines

Viktor Malik (3):
bpf: Always allow sleepable programs on syscalls
bpf: Always allow fmod_ret programs on syscalls selftests/bpf: Move sleepable refcounted_kptr tests to syscalls

 kernel/bpf/verifier.c                         | 79 +++++++++++++++----
 .../selftests/bpf/progs/refcounted_kptr.c     |  4 +-
 .../bpf/progs/refcounted_kptr_fail.c          |  2 +-
 3 files changed, 68 insertions(+), 17 deletions(-)
#13 lore raw
Quoted 17 lines · Kumar Kartikeya Dwivedi
Quoted 16 lines · Leon Hwang

Hi Viktor,

Quoted 11 lines · Viktor Malik

Both sleepable and fmod_ret programs are only allowed on selected functions. For convenience, the error injection list was originally used.

When error injection is disabled, that list is empty and sleepable tracing programs, as well as fmod_ret programs, are effectively unavailable.

This patch series addresses the issue by at least enabling sleepable and fmod_ret programs on syscalls, if error injection is disabled.

Quoted 8 lines
Quoted 4 lines

Could you please explain more about the purpose in cover letter?

I'm curious about enabling fmod_ret programs on syscalls, as they are able to modify the retval of syscalls.

It was already discussed at length here:
https://lore.kernel.org/bpf/CAADnVQJ4PU7bCb5XO9+zMZhLRSdbCKwxG6Rtojh2Rc3UZ8H43Q@mail.gmail.com/

Thanks for pointing this out.

Thanks,
Leon

Quoted 19 lines

I think the next version could include a link to the previous thread in the cover letter for posterity.

Quoted 16 lines

Thanks,
Leon

Quoted 10 lines

Viktor Malik (3):
bpf: Always allow sleepable programs on syscalls
bpf: Always allow fmod_ret programs on syscalls selftests/bpf: Move sleepable refcounted_kptr tests to syscalls

 kernel/bpf/verifier.c                         | 79 +++++++++++++++----
 .../selftests/bpf/progs/refcounted_kptr.c     |  4 +-
 .../bpf/progs/refcounted_kptr_fail.c          |  2 +-
 3 files changed, 68 insertions(+), 17 deletions(-)
#4 lore raw

Re: [PATCH bpf-next 1/3] bpf: Always allow sleepable programs on syscalls

In reply to Kumar Kartikeya Dwivedi
Quoted 37 lines · Kumar Kartikeya Dwivedi
Quoted 36 lines · On Thu, 5 Mar 2026 at 17:08, <[email protected]> wrote:
Quoted 5 lines
kernel/bpf/verifier.c +0 -0
index d92cf2821657..cb80a3ddb7a4 100644

[ ... ]

Quoted 6 lines
kernel/bpf/verifier.c +6 -0
+#else
+
+static bool has_arch_syscall_prefix(const char *func_name)
+{
+#if defined(__x86_64__)
+     return !strncmp(func_name, "__x64_", 6);

kernel/bpf/verifier.c static int check_non_sleepable_error_inject(u32 btf_id) · line 24983

Quoted 10 lines
kernel/bpf/verifier.c +10 -0
+#elif defined(__i386__)
+     return !strncmp(func_name, "__ia32_", 7);
+#elif defined(__s390x__)
+     return !strncmp(func_name, "__s390x_", 8);
+#elif defined(__aarch64__)
+     return !strncmp(func_name, "__arm64_", 8);
+#elif defined(__riscv)
+     return !strncmp(func_name, "__riscv_", 8);
+#elif defined(__powerpc__) || defined(__powerpc64__)
+     return !strncmp(func_name, "sys", 3);
                                    ^^^^^

The other architectures use prefixes that uniquely identify syscall wrappers (__x64_sys_*, __arm64_sys_*, etc.), but "sys" with length 3 would also match sysfs_*, sysctl_*, sysrq_*, and other non-syscall kernel functions.

Looking at arch/powerpc/include/asm/syscall_wrapper.h, __SYSCALL_DEFINEx generates names like sys_read and SYSCALL_DEFINE0 generates sys_fork, so the actual common prefix for powerpc syscalls is "sys_" (4 characters with the underscore).

Quoted 5 lines
Quoted 1 line

Should this be "sys_" with length 4 instead?

I think the bot is right, we should match sys_, that is less likely to include other non-syscall kernel functions.
It only affects powerpc with error injection disabled.

Yeah, agreed, will update to sys_ in v2.

Quoted 19 lines

Other than that, the set looks good to me. I tested it by disabling error injection and things fail before your fix is applied.

Quoted 14 lines

kernel/bpf/verifier.c static int check_non_sleepable_error_inject(u32 btf_id) · line 24987

Quoted 5 lines
kernel/bpf/verifier.c +4 -0
+#else
+     return false;
+#endif
+}

[ ... ]

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/22725431483

#14 lore raw
In reply to Leon Hwang
Quoted 23 lines · Leon Hwang
Quoted 22 lines · Kumar Kartikeya Dwivedi
Quoted 20 lines · Leon Hwang

Hi Viktor,

Quoted 11 lines · Viktor Malik

Both sleepable and fmod_ret programs are only allowed on selected functions. For convenience, the error injection list was originally used.

When error injection is disabled, that list is empty and sleepable tracing programs, as well as fmod_ret programs, are effectively unavailable.

This patch series addresses the issue by at least enabling sleepable and fmod_ret programs on syscalls, if error injection is disabled.

Could you please explain more about the purpose in cover letter?

I'm curious about enabling fmod_ret programs on syscalls, as they are able to modify the retval of syscalls.

Quoted 11 lines
Quoted 3 lines

It was already discussed at length here:
https://lore.kernel.org/bpf/CAADnVQJ4PU7bCb5XO9+zMZhLRSdbCKwxG6Rtojh2Rc3UZ8H43Q@mail.gmail.com/

Thanks for pointing this out.

Thanks,
Leon

Quoted 2 lines

I think the next version could include a link to the previous thread in the cover letter for posterity.

Good point, I'll definitely add a link to the discussion.

Quoted 18 lines
Quoted 17 lines
Quoted 17 lines

Thanks,
Leon

Quoted 10 lines

Viktor Malik (3):
bpf: Always allow sleepable programs on syscalls
bpf: Always allow fmod_ret programs on syscalls selftests/bpf: Move sleepable refcounted_kptr tests to syscalls

 kernel/bpf/verifier.c                         | 79 +++++++++++++++----
 .../selftests/bpf/progs/refcounted_kptr.c     |  4 +-
 .../bpf/progs/refcounted_kptr_fail.c          |  2 +-
 3 files changed, 68 insertions(+), 17 deletions(-)
#6 lore raw

Re: [PATCH bpf-next 1/3] bpf: Always allow sleepable programs on syscalls

In reply to Leon Hwang
Quoted 67 lines · Leon Hwang
Quoted 53 lines · Viktor Malik

Sleepable BPF programs can only be attached to selected functions. For convenience, the error injection list was originally used, which contains syscalls and several other functions.

When error injection is disabled (CONFIG_FUNCTION_ERROR_INJECTION=n), that list is empty and sleepable tracing programs are effectively unavailable. In such a case, at least enable sleepable programs on syscalls.

---
 kernel/bpf/verifier.c | 54 ++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 48 insertions(+), 6 deletions(-)
kernel/bpf/verifier.c +24 -0
index d92cf2821657..cb80a3ddb7a4 100644
@@ -24930,6 +24930,8 @@ static int check_attach_modify_return(unsigned long addr, const char *func_name)
 	return -EINVAL;
 }
 
+#ifdef CONFIG_FUNCTION_ERROR_INJECTION
+
 /* list of non-sleepable functions that are otherwise on
  * ALLOW_ERROR_INJECTION list
  */
@@ -24951,6 +24953,51 @@ static int check_non_sleepable_error_inject(u32 btf_id)
 	return btf_id_set_contains(&btf_non_sleepable_error_inject, btf_id);
 }
 
+static int check_attach_sleepable(u32 btf_id, unsigned long addr, const char *func_name)
+{
+	/* fentry/fexit/fmod_ret progs can be sleepable if they are
+	 * attached to ALLOW_ERROR_INJECTION and are not in denylist.
+	 */
+	if (!check_non_sleepable_error_inject(btf_id) &&
+	    within_error_injection_list(addr))
+		return 0;
+
+	return -EINVAL;
+}
+
+#else
+
+static bool has_arch_syscall_prefix(const char *func_name)
+{
+#if defined(__x86_64__)
+	return !strncmp(func_name, "__x64_", 6);
+#elif defined(__i386__)
+	return !strncmp(func_name, "__ia32_", 7);
+#elif defined(__s390x__)
+	return !strncmp(func_name, "__s390x_", 8);

kernel/bpf/verifier.c static int check_non_sleepable_error_inject(u32 btf_id) · line 24987

Quoted 10 lines
kernel/bpf/verifier.c +10 -0
+#elif defined(__aarch64__)
+	return !strncmp(func_name, "__arm64_", 8);
+#elif defined(__riscv)
+	return !strncmp(func_name, "__riscv_", 8);
+#elif defined(__powerpc__) || defined(__powerpc64__)
+	return !strncmp(func_name, "sys", 3);
+#else
+	return false;
+#endif
+}

At first glance, it is not a good taste to hard code these prefixes here.

Quoted 2 lines

Furthermore, the arch_syscall_match_sym_name() of ftrace does the same thing. Consider it instead.

I did consider it but arch_syscall_match_sym_name() matches two strings (the arch-specific syscall function and the generic "sys_<name>") while we need to just check the prefix (we don't have the other string).

Another generic option is to implement has_arch_syscall_prefix() in arch code, next to arch_syscall_match_sym_name(). But that seems like too big of a change in ftrace for a small use-case such as this one.

That's why I ended up with the current form. In addition, arches hard-code the prefixes in their syscall code so we'll have to hard-code them anyways, no matter the solution we end up with.

Viktor

Quoted 35 lines

Thanks,
Leon

Quoted 30 lines
kernel/bpf/verifier.c +14 -6
+
+/* Without error injection, allow sleepable progs on syscalls. */
+
+static int check_attach_sleepable(u32 btf_id, unsigned long addr, const char *func_name)
+{
+	if (has_arch_syscall_prefix(func_name))
+		return 0;
+
+	return -EINVAL;
+}
+
+#endif /* CONFIG_FUNCTION_ERROR_INJECTION */
+
 int bpf_check_attach_target(struct bpf_verifier_log *log,
 			    const struct bpf_prog *prog,
 			    const struct bpf_prog *tgt_prog,
@@ -25230,12 +25277,7 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,
 			ret = -EINVAL;
 			switch (prog->type) {
 			case BPF_PROG_TYPE_TRACING:
-
-				/* fentry/fexit/fmod_ret progs can be sleepable if they are
-				 * attached to ALLOW_ERROR_INJECTION and are not in denylist.
-				 */
-				if (!check_non_sleepable_error_inject(btf_id) &&
-				    within_error_injection_list(addr))
+				if (!check_attach_sleepable(btf_id, addr, tname))
 					ret = 0;
 				/* fentry/fexit/fmod_ret progs can also be sleepable if they are
 				 * in the fmodret id set with the KF_SLEEPABLE flag.
#7 lore raw

Re: [PATCH bpf-next 1/3] bpf: Always allow sleepable programs on syscalls

Quoted 75 lines · Viktor Malik
Quoted 67 lines · Leon Hwang
Quoted 53 lines · Viktor Malik

Sleepable BPF programs can only be attached to selected functions. For convenience, the error injection list was originally used, which contains syscalls and several other functions.

When error injection is disabled (CONFIG_FUNCTION_ERROR_INJECTION=n), that list is empty and sleepable tracing programs are effectively unavailable. In such a case, at least enable sleepable programs on syscalls.

---
 kernel/bpf/verifier.c | 54 ++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 48 insertions(+), 6 deletions(-)
kernel/bpf/verifier.c +24 -0
index d92cf2821657..cb80a3ddb7a4 100644
@@ -24930,6 +24930,8 @@ static int check_attach_modify_return(unsigned long addr, const char *func_name)
 	return -EINVAL;
 }
 
+#ifdef CONFIG_FUNCTION_ERROR_INJECTION
+
 /* list of non-sleepable functions that are otherwise on
  * ALLOW_ERROR_INJECTION list
  */
@@ -24951,6 +24953,51 @@ static int check_non_sleepable_error_inject(u32 btf_id)
 	return btf_id_set_contains(&btf_non_sleepable_error_inject, btf_id);
 }
 
+static int check_attach_sleepable(u32 btf_id, unsigned long addr, const char *func_name)
+{
+	/* fentry/fexit/fmod_ret progs can be sleepable if they are
+	 * attached to ALLOW_ERROR_INJECTION and are not in denylist.
+	 */
+	if (!check_non_sleepable_error_inject(btf_id) &&
+	    within_error_injection_list(addr))
+		return 0;
+
+	return -EINVAL;
+}
+
+#else
+
+static bool has_arch_syscall_prefix(const char *func_name)
+{
+#if defined(__x86_64__)
+	return !strncmp(func_name, "__x64_", 6);
+#elif defined(__i386__)
+	return !strncmp(func_name, "__ia32_", 7);
+#elif defined(__s390x__)
+	return !strncmp(func_name, "__s390x_", 8);

kernel/bpf/verifier.c static int check_non_sleepable_error_inject(u32 btf_id) · line 24987

Quoted 10 lines
kernel/bpf/verifier.c +10 -0
+#elif defined(__aarch64__)
+	return !strncmp(func_name, "__arm64_", 8);
+#elif defined(__riscv)
+	return !strncmp(func_name, "__riscv_", 8);
+#elif defined(__powerpc__) || defined(__powerpc64__)
+	return !strncmp(func_name, "sys", 3);
+#else
+	return false;
+#endif
+}

At first glance, it is not a good taste to hard code these prefixes here.

Quoted 2 lines

Furthermore, the arch_syscall_match_sym_name() of ftrace does the same thing. Consider it instead.

I did consider it but arch_syscall_match_sym_name() matches two strings (the arch-specific syscall function and the generic "sys_<name>") while we need to just check the prefix (we don't have the other string).

Quoted 8 lines

Another generic option is to implement has_arch_syscall_prefix() in arch code, next to arch_syscall_match_sym_name(). But that seems like too big of a change in ftrace for a small use-case such as this one.

That's why I ended up with the current form. In addition, arches hard-code the prefixes in their syscall code so we'll have to hard-code them anyways, no matter the solution we end up with.

Hmm, I don't have a strong preference.

However, it is worth adding some explanation in the commit log and code comment.

Thanks,
Leon

[...]

#8 lore raw

Re: [PATCH bpf-next 1/3] bpf: Always allow sleepable programs on syscalls

Quoted 80 lines · Leon Hwang
Quoted 79 lines · Viktor Malik
Quoted 67 lines · Leon Hwang
Quoted 53 lines · Viktor Malik

Sleepable BPF programs can only be attached to selected functions. For convenience, the error injection list was originally used, which contains syscalls and several other functions.

When error injection is disabled (CONFIG_FUNCTION_ERROR_INJECTION=n), that list is empty and sleepable tracing programs are effectively unavailable. In such a case, at least enable sleepable programs on syscalls.

---
 kernel/bpf/verifier.c | 54 ++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 48 insertions(+), 6 deletions(-)
kernel/bpf/verifier.c +24 -0
index d92cf2821657..cb80a3ddb7a4 100644
@@ -24930,6 +24930,8 @@ static int check_attach_modify_return(unsigned long addr, const char *func_name)
 	return -EINVAL;
 }
 
+#ifdef CONFIG_FUNCTION_ERROR_INJECTION
+
 /* list of non-sleepable functions that are otherwise on
  * ALLOW_ERROR_INJECTION list
  */
@@ -24951,6 +24953,51 @@ static int check_non_sleepable_error_inject(u32 btf_id)
 	return btf_id_set_contains(&btf_non_sleepable_error_inject, btf_id);
 }
 
+static int check_attach_sleepable(u32 btf_id, unsigned long addr, const char *func_name)
+{
+	/* fentry/fexit/fmod_ret progs can be sleepable if they are
+	 * attached to ALLOW_ERROR_INJECTION and are not in denylist.
+	 */
+	if (!check_non_sleepable_error_inject(btf_id) &&
+	    within_error_injection_list(addr))
+		return 0;
+
+	return -EINVAL;
+}
+
+#else
+
+static bool has_arch_syscall_prefix(const char *func_name)
+{
+#if defined(__x86_64__)
+	return !strncmp(func_name, "__x64_", 6);
+#elif defined(__i386__)
+	return !strncmp(func_name, "__ia32_", 7);
+#elif defined(__s390x__)
+	return !strncmp(func_name, "__s390x_", 8);

kernel/bpf/verifier.c static int check_non_sleepable_error_inject(u32 btf_id) · line 24987

Quoted 10 lines
kernel/bpf/verifier.c +10 -0
+#elif defined(__aarch64__)
+	return !strncmp(func_name, "__arm64_", 8);
+#elif defined(__riscv)
+	return !strncmp(func_name, "__riscv_", 8);
+#elif defined(__powerpc__) || defined(__powerpc64__)
+	return !strncmp(func_name, "sys", 3);
+#else
+	return false;
+#endif
+}

At first glance, it is not a good taste to hard code these prefixes here.

Quoted 2 lines

Furthermore, the arch_syscall_match_sym_name() of ftrace does the same thing. Consider it instead.

I did consider it but arch_syscall_match_sym_name() matches two strings (the arch-specific syscall function and the generic "sys_<name>") while we need to just check the prefix (we don't have the other string).

Another generic option is to implement has_arch_syscall_prefix() in arch code, next to arch_syscall_match_sym_name(). But that seems like too big of a change in ftrace for a small use-case such as this one.

Quoted 8 lines
Quoted 4 lines

That's why I ended up with the current form. In addition, arches hard-code the prefixes in their syscall code so we'll have to hard-code them anyways, no matter the solution we end up with.

Hmm, I don't have a strong preference.

However, it is worth adding some explanation in the commit log and code comment.

Ok, I'll add an explanation.

Thanks,
Viktor

Quoted 6 lines

Thanks,
Leon

[...]