sched_ext: Deny SCX kfuncs to non-SCX struct_ops programs
Subsystems (3)
cover sched_ext: Deny SCX kfuncs to non-SCX struct_ops programs
As discussed in [1], scx_kfunc_context_filter() currently allows non-SCX struct_ops programs (e.g. tcp_congestion_ops) to call SCX kfuncs that are only meaningful inside an SCX scheduler. This is wrong for two reasons.
First, it is semantically incorrect: a TCP congestion control program has no business calling SCX kfuncs such as scx_bpf_kick_cpu().
Second, with CONFIG_EXT_SUB_SCHED=y, kfuncs like scx_bpf_kick_cpu() call scx_prog_sched(aux), which retrieves the struct_ops kdata via bpf_prog_get_assoc_struct_ops() and casts it to struct sched_ext_ops * before reading ops->priv. For a non-SCX struct_ops program the kdata is far smaller than sched_ext_ops, turning the read into an out-of-bounds access (confirmed with KASAN).
Patch 1 extends scx_kfunc_context_filter() to also cover scx_kfunc_set_any and denies all context-sensitive kfuncs to any struct_ops program that is not the SCX struct_ops.
Patch 2 adds a selftest that loads a TCP congestion control program calling scx_bpf_kick_cpu() and verifies the BPF verifier rejects it.
[1] https://lore.kernel.org/r/f2ab3yg5niso6hxqe7sd4jmv5xzdizk3khcspm5bylfbn3mj44@tpyiezvs4cod/
Thanks,
Cheng-Yang
---
Cheng-Yang Chou (2):
sched_ext: Deny SCX kfuncs to non-SCX struct_ops programs selftests/sched_ext: Add non_scx_kfunc_deny test
kernel/sched/ext.c | 25 +++++----- tools/testing/selftests/sched_ext/Makefile | 1 + .../sched_ext/non_scx_kfunc_deny.bpf.c | 44 +++++++++++++++++ .../selftests/sched_ext/non_scx_kfunc_deny.c | 47 +++++++++++++++++++ 4 files changed, 106 insertions(+), 11 deletions(-) create mode 100644 tools/testing/selftests/sched_ext/non_scx_kfunc_deny.bpf.c create mode 100644 tools/testing/selftests/sched_ext/non_scx_kfunc_deny.c
Signature · 1 line
2.48.1
PATCH 1/2 sched_ext: Deny SCX kfuncs to non-SCX struct_ops programs
scx_kfunc_context_filter() currently allows non-SCX struct_ops programs (e.g. tcp_congestion_ops) to call SCX unlocked kfuncs. This is wrong for two reasons:
- It is semantically incorrect: a TCP congestion control program has no business calling SCX kfuncs such as scx_bpf_kick_cpu().
- With CONFIG_EXT_SUB_SCHED=y, kfuncs like scx_bpf_kick_cpu() call scx_prog_sched(aux), which invokes bpf_prog_get_assoc_struct_ops(aux) and casts the result to struct sched_ext_ops * before reading ops->priv. For a non-SCX struct_ops program the returned pointer is the kdata of that struct_ops type, which is far smaller than sched_ext_ops, making the read an out-of-bounds access (confirmed with KASAN).
Extend the filter to cover scx_kfunc_set_any as well, and deny all context-sensitive kfuncs for any struct_ops program that is not the SCX struct_ops. This addresses both issues: the semantic contract is enforced at the verifier level, and the runtime out-of-bounds access becomes unreachable.
--- kernel/sched/ext.c | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-)
kernel/sched/ext.c +14 -11
@@ -9479,6 +9479,7 @@ BTF_KFUNCS_END(scx_kfunc_ids_any)static const struct btf_kfunc_id_set scx_kfunc_set_any = { .owner =THIS_MODULE , .set = &scx_kfunc_ids_any, + .filter = scx_kfunc_context_filter, };/*
@@ -9530,9 +9531,9 @@ static const u32 scx_kf_allow_flags[] = {* .filter field on each per-group btf_kfunc_id_set. The BPF core invokes this * for every kfunc call in the registered hook (BPF_PROG_TYPE_STRUCT_OPS or * BPF_PROG_TYPE_SYSCALL), regardless of which set originally introduced the -* kfunc - so the filter must short-circuit on kfuncs it doesn't govern (e.g. -* scx_kfunc_ids_any) by falling through to "allow" when none of the -* context-sensitive sets contain the kfunc. +* kfunc - so the filter must short-circuit on kfuncs it doesn't govern by +* falling through to "allow" when none of the context-sensitive sets contain +* the kfunc. */ int scx_kfunc_context_filter (const struct bpf_prog *prog,u32 kfunc_id) {
@@ -9541,18 +9542,19 @@ int scx_kfunc_context_filter(const struct bpf_prog *prog, u32 kfunc_id)bool in_enqueue =btf_id_set8_contains (&scx_kfunc_ids_enqueue_dispatch, kfunc_id);bool in_dispatch =btf_id_set8_contains (&scx_kfunc_ids_dispatch, kfunc_id);bool in_cpu_release =btf_id_set8_contains (&scx_kfunc_ids_cpu_release, kfunc_id); +bool in_any =btf_id_set8_contains (&scx_kfunc_ids_any, kfunc_id);u32 moff, flags; -/* Not a context-sensitive kfunc (e.g. from scx_kfunc_ids_any) - allow. */ -if (!(in_unlocked || in_select_cpu || in_enqueue || in_dispatch || in_cpu_release)) +/* Not a context-sensitive kfunc - allow. */ +if (!(in_unlocked || in_select_cpu || in_enqueue || in_dispatch || in_cpu_release || in_any))return 0 ;/* SYSCALL progs (e.g. BPF test_run()) may call unlocked and select_cpu kfuncs. */ if (prog->type ==BPF_PROG_TYPE_SYSCALL ) -return (in_unlocked || in_select_cpu) ?0 : -EACCES ; +return (in_unlocked || in_select_cpu || in_any) ?0 : -EACCES ;if (prog->type !=BPF_PROG_TYPE_STRUCT_OPS ) -return -EACCES ; +return in_any ?0 : -EACCES ;/* * add_subprog_and_kfunc() collects all kfunc calls, including dead code
@@ -9565,14 +9567,15 @@ int scx_kfunc_context_filter(const struct bpf_prog *prog, u32 kfunc_id)return 0 ;/* -* Non-SCX struct_ops: only unlocked kfuncs are safe. The other -* context-sensitive kfuncs assume the rq lock is held by the SCX -* dispatch path, which doesn't apply to other struct_ops users. +* Non-SCX struct_ops: context-sensitive kfuncs are not permitted. */ if (prog->aux->st_ops != &bpf_sched_ext_ops) -return in_unlocked ?0 : -EACCES ; +return -EACCES ;/* SCX struct_ops: check the per-op allow list. */ +if (in_any) +return 0 ; + moff = prog->aux->attach_st_ops_member_off; flags = scx_kf_allow_flags[SCX_MOFF_IDX (moff)];
Signature · 1 line
2.48.1
Tejun Heo
reviewerHello,
A few things:
scx_kfunc_set_idle (ext_idle.c) isn't covered by the patch. Its kfuncs (scx_bpf_cpu_node(), scx_bpf_get_idle_cpumask(), scx_bpf_pick_idle_cpu() etc.) also call scx_prog_sched(aux) and hit the same OOB pattern on non-SCX struct_ops programs. Please add an in_idle check in the filter and treat it the same as in_any.
Separately, please also set .filter on scx_kfunc_set_idle itself. In practice, the BPF core dedups filters per hook in btf_populate_kfunc_set(), so the filter is already invoked for idle kfuncs via the other sets' registrations on the same hook. But it's confusing to read the code without setting .filter on every set.
The following line is too long - please break and indent (it gets longer still once in_idle is added):
kernel/sched/ext.c int scx_kfunc_context_filter(const struct bpf_prog *prog, u32 kfunc_id) · line 9549
Quoted 1 line
kernel/sched/ext.c +1
-0+if (!(in_unlocked || in_select_cpu || in_enqueue || in_dispatch || in_cpu_release || in_any))
Once idle is covered, every SCX kfunc set ends up in this "context- sensitive" list. At that point "context-sensitive" no longer distinguishes anything - it just means "SCX kfuncs". Please update the function-level comment and the two inline comments ("Not a context- sensitive kfunc - allow." and "Non-SCX struct_ops: context-sensitive kfuncs are not permitted.") to drop the term and talk about SCX kfuncs directly.
Thanks.
--
tejun
Hi Tejun,
Quoted 7 lines · Tejun Heo
A few things:
scx_kfunc_set_idle(ext_idle.c) isn't covered by the patch. Its kfuncs (scx_bpf_cpu_node(),scx_bpf_get_idle_cpumask(),scx_bpf_pick_idle_cpu()etc.) also callscx_prog_sched(aux)and hit the same OOB pattern on non-SCXstruct_opsprograms. Please add anin_idlecheck in the filter and treat it the same asin_any.
Ack.
Quoted 5 lines
Separately, please also set .filter on
scx_kfunc_set_idleitself. In practice, the BPF core dedups filters per hook inbtf_populate_kfunc_set(), so the filter is already invoked for idle kfuncs via the other sets' registrations on the same hook. But it's confusing to read the code without setting .filter on every set.
Agree, will do.
Quoted 4 lines
The following line is too long - please break and indent (it gets longer still once
in_idleis added):kernel/sched/ext.c int scx_kfunc_context_filter(const struct bpf_prog *prog, u32 kfunc_id) · line 9549
Quoted 1 line
kernel/sched/ext.c +1
-0+if (!(in_unlocked || in_select_cpu || in_enqueue || in_dispatch || in_cpu_release || in_any))
Ack.
Quoted 7 lines
Once idle is covered, every SCX kfunc set ends up in this "context- sensitive" list. At that point "context-sensitive" no longer distinguishes anything - it just means "SCX kfuncs". Please update the function-level comment and the two inline comments ("Not a context- sensitive kfunc - allow." and "Non-SCX
struct_ops: context-sensitive kfuncs are not permitted.") to drop the term and talk about SCX kfuncs directly.
Ahh, thanks for point out! I'll reword them and send v2 patch.
Signature · 2 lines
Thanks, Cheng-Yang
PATCH 2/2 selftests/sched_ext: Add non_scx_kfunc_deny test
In reply to Cheng-Yang ChouVerify that the BPF verifier rejects a non-SCX struct_ops program (tcp_congestion_ops) that attempts to call an SCX kfunc (scx_bpf_kick_cpu). The test expects the load to fail with -EACCES from scx_kfunc_context_filter.
--- tools/testing/selftests/sched_ext/Makefile | 1 + .../sched_ext/non_scx_kfunc_deny.bpf.c | 44 +++++++++++++++++ .../selftests/sched_ext/non_scx_kfunc_deny.c | 47 +++++++++++++++++++ 3 files changed, 92 insertions(+) create mode 100644 tools/testing/selftests/sched_ext/non_scx_kfunc_deny.bpf.c create mode 100644 tools/testing/selftests/sched_ext/non_scx_kfunc_deny.c
tools/testing/selftests/sched_ext/Makefile +1 -0
@@ -190,6 +190,7 @@ auto-test-targets := \ test_example \ total_bw \ cyclic_kick_wait \ + non_scx_kfunc_deny \ testcase-targets := $(addsuffix .o,$(addprefix $(SCXOBJ_DIR)/,$(auto-test-targets)))
tools/testing/selftests/sched_ext/non_scx_kfunc_deny.bpf.c new file +44 -0
44 lines not on this page · raw message
tools/testing/selftests/sched_ext/non_scx_kfunc_deny.c new file +47 -0
47 lines not on this page · raw message
Signature · 1 line
2.48.1