Skip to contentSkip to conversation

sched_ext: Deny SCX kfuncs to non-SCX struct_ops programs

PATCH Superseded Cheng-Yang Chou 2 participants 5 messages Apr 16 – Apr 18

Subsystems (3)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover sched_ext: Deny SCX kfuncs to non-SCX struct_ops programs

As discussed in [1], scx_kfunc_context_filter() currently allows non-SCX struct_ops programs (e.g. tcp_congestion_ops) to call SCX kfuncs that are only meaningful inside an SCX scheduler. This is wrong for two reasons.

First, it is semantically incorrect: a TCP congestion control program has no business calling SCX kfuncs such as scx_bpf_kick_cpu().

Second, with CONFIG_EXT_SUB_SCHED=y, kfuncs like scx_bpf_kick_cpu() call scx_prog_sched(aux), which retrieves the struct_ops kdata via bpf_prog_get_assoc_struct_ops() and casts it to struct sched_ext_ops * before reading ops->priv. For a non-SCX struct_ops program the kdata is far smaller than sched_ext_ops, turning the read into an out-of-bounds access (confirmed with KASAN).

Patch 1 extends scx_kfunc_context_filter() to also cover scx_kfunc_set_any and denies all context-sensitive kfuncs to any struct_ops program that is not the SCX struct_ops.

Patch 2 adds a selftest that loads a TCP congestion control program calling scx_bpf_kick_cpu() and verifies the BPF verifier rejects it.

[1] https://lore.kernel.org/r/f2ab3yg5niso6hxqe7sd4jmv5xzdizk3khcspm5bylfbn3mj44@tpyiezvs4cod/

Thanks,
Cheng-Yang

---
Cheng-Yang Chou (2):
sched_ext: Deny SCX kfuncs to non-SCX struct_ops programs selftests/sched_ext: Add non_scx_kfunc_deny test

 kernel/sched/ext.c                            | 25 +++++-----
 tools/testing/selftests/sched_ext/Makefile    |  1 +
 .../sched_ext/non_scx_kfunc_deny.bpf.c        | 44 +++++++++++++++++
 .../selftests/sched_ext/non_scx_kfunc_deny.c  | 47 +++++++++++++++++++
 4 files changed, 106 insertions(+), 11 deletions(-)
 create mode 100644 tools/testing/selftests/sched_ext/non_scx_kfunc_deny.bpf.c
 create mode 100644 tools/testing/selftests/sched_ext/non_scx_kfunc_deny.c
Signature · 1 line
2.48.1
#1 lore raw

PATCH 1/2 sched_ext: Deny SCX kfuncs to non-SCX struct_ops programs

scx_kfunc_context_filter() currently allows non-SCX struct_ops programs (e.g. tcp_congestion_ops) to call SCX unlocked kfuncs. This is wrong for two reasons:

- It is semantically incorrect: a TCP congestion control program has no business calling SCX kfuncs such as scx_bpf_kick_cpu().

- With CONFIG_EXT_SUB_SCHED=y, kfuncs like scx_bpf_kick_cpu() call scx_prog_sched(aux), which invokes bpf_prog_get_assoc_struct_ops(aux) and casts the result to struct sched_ext_ops * before reading ops->priv. For a non-SCX struct_ops program the returned pointer is the kdata of that struct_ops type, which is far smaller than sched_ext_ops, making the read an out-of-bounds access (confirmed with KASAN).

Extend the filter to cover scx_kfunc_set_any as well, and deny all context-sensitive kfuncs for any struct_ops program that is not the SCX struct_ops. This addresses both issues: the semantic contract is enforced at the verifier level, and the runtime out-of-bounds access becomes unreachable.

Fixes d1d3c1c6ae36 ("sched_ext: Add verifier-time kfunc context filter")
---
 kernel/sched/ext.c | 25 ++++++++++++++-----------
 1 file changed, 14 insertions(+), 11 deletions(-)
kernel/sched/ext.c +14 -11
index 012ca8bd70fb..768abc6cf7ad 100644
@@ -9479,6 +9479,7 @@ BTF_KFUNCS_END(scx_kfunc_ids_any)
 static const struct btf_kfunc_id_set scx_kfunc_set_any = {
 	.owner			= THIS_MODULE,
 	.set			= &scx_kfunc_ids_any,
+	.filter			= scx_kfunc_context_filter,
 };
 
 /*
@@ -9530,9 +9531,9 @@ static const u32 scx_kf_allow_flags[] = {
  * .filter field on each per-group btf_kfunc_id_set. The BPF core invokes this
  * for every kfunc call in the registered hook (BPF_PROG_TYPE_STRUCT_OPS or
  * BPF_PROG_TYPE_SYSCALL), regardless of which set originally introduced the
- * kfunc - so the filter must short-circuit on kfuncs it doesn't govern (e.g.
- * scx_kfunc_ids_any) by falling through to "allow" when none of the
- * context-sensitive sets contain the kfunc.
+ * kfunc - so the filter must short-circuit on kfuncs it doesn't govern by
+ * falling through to "allow" when none of the context-sensitive sets contain
+ * the kfunc.
  */
 int scx_kfunc_context_filter(const struct bpf_prog *prog, u32 kfunc_id)
 {
@@ -9541,18 +9542,19 @@ int scx_kfunc_context_filter(const struct bpf_prog *prog, u32 kfunc_id)
 	bool in_enqueue = btf_id_set8_contains(&scx_kfunc_ids_enqueue_dispatch, kfunc_id);
 	bool in_dispatch = btf_id_set8_contains(&scx_kfunc_ids_dispatch, kfunc_id);
 	bool in_cpu_release = btf_id_set8_contains(&scx_kfunc_ids_cpu_release, kfunc_id);
+	bool in_any = btf_id_set8_contains(&scx_kfunc_ids_any, kfunc_id);
 	u32 moff, flags;
 
-	/* Not a context-sensitive kfunc (e.g. from scx_kfunc_ids_any) - allow. */
-	if (!(in_unlocked || in_select_cpu || in_enqueue || in_dispatch || in_cpu_release))
+	/* Not a context-sensitive kfunc - allow. */
+	if (!(in_unlocked || in_select_cpu || in_enqueue || in_dispatch || in_cpu_release || in_any))
 		return 0;
 
 	/* SYSCALL progs (e.g. BPF test_run()) may call unlocked and select_cpu kfuncs. */
 	if (prog->type == BPF_PROG_TYPE_SYSCALL)
-		return (in_unlocked || in_select_cpu) ? 0 : -EACCES;
+		return (in_unlocked || in_select_cpu || in_any) ? 0 : -EACCES;
 
 	if (prog->type != BPF_PROG_TYPE_STRUCT_OPS)
-		return -EACCES;
+		return in_any ? 0 : -EACCES;
 
 	/*
 	 * add_subprog_and_kfunc() collects all kfunc calls, including dead code
@@ -9565,14 +9567,15 @@ int scx_kfunc_context_filter(const struct bpf_prog *prog, u32 kfunc_id)
 		return 0;
 
 	/*
-	 * Non-SCX struct_ops: only unlocked kfuncs are safe. The other
-	 * context-sensitive kfuncs assume the rq lock is held by the SCX
-	 * dispatch path, which doesn't apply to other struct_ops users.
+	 * Non-SCX struct_ops: context-sensitive kfuncs are not permitted.
 	 */
 	if (prog->aux->st_ops != &bpf_sched_ext_ops)
-		return in_unlocked ? 0 : -EACCES;
+		return -EACCES;
 
 	/* SCX struct_ops: check the per-op allow list. */
+	if (in_any)
+		return 0;
+
 	moff = prog->aux->attach_st_ops_member_off;
 	flags = scx_kf_allow_flags[SCX_MOFF_IDX(moff)];
 
Signature · 1 line
2.48.1
reviewer
#2 lore raw

Hello,

A few things:

scx_kfunc_set_idle (ext_idle.c) isn't covered by the patch. Its kfuncs (scx_bpf_cpu_node(), scx_bpf_get_idle_cpumask(), scx_bpf_pick_idle_cpu() etc.) also call scx_prog_sched(aux) and hit the same OOB pattern on non-SCX struct_ops programs. Please add an in_idle check in the filter and treat it the same as in_any.

Separately, please also set .filter on scx_kfunc_set_idle itself. In practice, the BPF core dedups filters per hook in btf_populate_kfunc_set(), so the filter is already invoked for idle kfuncs via the other sets' registrations on the same hook. But it's confusing to read the code without setting .filter on every set.

The following line is too long - please break and indent (it gets longer still once in_idle is added):

kernel/sched/ext.c int scx_kfunc_context_filter(const struct bpf_prog *prog, u32 kfunc_id) · line 9549

Quoted 1 line
kernel/sched/ext.c +1 -0
+	if (!(in_unlocked || in_select_cpu || in_enqueue || in_dispatch || in_cpu_release || in_any))

Once idle is covered, every SCX kfunc set ends up in this "context- sensitive" list. At that point "context-sensitive" no longer distinguishes anything - it just means "SCX kfuncs". Please update the function-level comment and the two inline comments ("Not a context- sensitive kfunc - allow." and "Non-SCX struct_ops: context-sensitive kfuncs are not permitted.") to drop the term and talk about SCX kfuncs directly.

Thanks.

--
tejun

#3 lore raw

Hi Tejun,

Quoted 7 lines · Tejun Heo

A few things:

scx_kfunc_set_idle (ext_idle.c) isn't covered by the patch. Its kfuncs (scx_bpf_cpu_node(), scx_bpf_get_idle_cpumask(), scx_bpf_pick_idle_cpu() etc.) also call scx_prog_sched(aux) and hit the same OOB pattern on non-SCX struct_ops programs. Please add an in_idle check in the filter and treat it the same as in_any.

Ack.

Quoted 5 lines

Separately, please also set .filter on scx_kfunc_set_idle itself. In practice, the BPF core dedups filters per hook in btf_populate_kfunc_set(), so the filter is already invoked for idle kfuncs via the other sets' registrations on the same hook. But it's confusing to read the code without setting .filter on every set.

Agree, will do.

Quoted 4 lines

The following line is too long - please break and indent (it gets longer still once in_idle is added):

kernel/sched/ext.c int scx_kfunc_context_filter(const struct bpf_prog *prog, u32 kfunc_id) · line 9549

Quoted 1 line
kernel/sched/ext.c +1 -0
+	if (!(in_unlocked || in_select_cpu || in_enqueue || in_dispatch || in_cpu_release || in_any))

Ack.

Quoted 7 lines

Once idle is covered, every SCX kfunc set ends up in this "context- sensitive" list. At that point "context-sensitive" no longer distinguishes anything - it just means "SCX kfuncs". Please update the function-level comment and the two inline comments ("Not a context- sensitive kfunc - allow." and "Non-SCX struct_ops: context-sensitive kfuncs are not permitted.") to drop the term and talk about SCX kfuncs directly.

Ahh, thanks for point out! I'll reword them and send v2 patch.

Signature · 2 lines
Thanks,
Cheng-Yang
#4 lore raw

PATCH 2/2 selftests/sched_ext: Add non_scx_kfunc_deny test

In reply to Cheng-Yang Chou

Verify that the BPF verifier rejects a non-SCX struct_ops program (tcp_congestion_ops) that attempts to call an SCX kfunc (scx_bpf_kick_cpu). The test expects the load to fail with -EACCES from scx_kfunc_context_filter.

---
 tools/testing/selftests/sched_ext/Makefile    |  1 +
 .../sched_ext/non_scx_kfunc_deny.bpf.c        | 44 +++++++++++++++++
 .../selftests/sched_ext/non_scx_kfunc_deny.c  | 47 +++++++++++++++++++
 3 files changed, 92 insertions(+)
 create mode 100644 tools/testing/selftests/sched_ext/non_scx_kfunc_deny.bpf.c
 create mode 100644 tools/testing/selftests/sched_ext/non_scx_kfunc_deny.c
tools/testing/selftests/sched_ext/Makefile +1 -0
index 789037be44c7..2880a122a214 100644
@@ -190,6 +190,7 @@ auto-test-targets :=			\
 	test_example			\
 	total_bw			\
 	cyclic_kick_wait		\
+	non_scx_kfunc_deny		\
 
 testcase-targets := $(addsuffix .o,$(addprefix $(SCXOBJ_DIR)/,$(auto-test-targets)))
 
tools/testing/selftests/sched_ext/non_scx_kfunc_deny.bpf.c new file +44 -0

44 lines not on this page · raw message

tools/testing/selftests/sched_ext/non_scx_kfunc_deny.c new file +47 -0

47 lines not on this page · raw message

Signature · 1 line
2.48.1