sched_ext: Follow-up fixes and missing cap enforcement
Subsystems (2)
Tejun Heo
reviewerPATCH 1/1 sched_ext: Follow-up fixes and missing cap enforcement
Hello,
Follow-up fixes and enforcement holes plugged from the recent sub-scheduler work. The first patch fixes the tools' restart loop clobbering an exit that was requested while a restart was pending.
The rest tighten how cid-addressed operations from sub-schedulers are handled when the caller lacks the necessary caps. Reenq to another sched's local DSQs is now denied and counted, and a new event counts the already-denied kicks. These checks are lockless - transient false positives from cap rotation are harmless and false negatives can't happen by visibility. cpuperf control moves behind a new SCX_CAP_PERF which only the root holds by default, and scx_bpf_cidperf_set() reports the result synchronously through a new versioned interface as its check is authoritative under the target's rq lock.
0001-tools-sched_ext-Don-t-restart-over-a-pending-exit-re.patch 0002-sched_ext-Gate-local-DSQ-reenq-on-baseline-cid-acces.patch 0003-sched_ext-Count-kicks-denied-for-lacking-baseline-ci.patch 0004-sched_ext-Factor-out-scx_cpuperf_set.patch 0005-sched_ext-Gate-scx_bpf_cidperf_set-behind-a-new-SCX_.patch
Based on sched_ext/for-7.3 (c509d8107026).
The patches are also available in the following git branch:
git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext.git scx-sub-cap-gates
diffstat follows. Thanks.
kernel/sched/ext/ext.c | 152 ++++++++++++++++++++++--------- kernel/sched/ext/internal.h | 31 ++++++- tools/sched_ext/include/scx/common.bpf.h | 2 +- tools/sched_ext/include/scx/compat.bpf.h | 18 ++++ tools/sched_ext/scx_central.c | 2 +- tools/sched_ext/scx_cpu0.c | 2 +- tools/sched_ext/scx_flatcg.c | 2 +- tools/sched_ext/scx_pair.c | 2 +- tools/sched_ext/scx_qmap.bpf.c | 17 ++-- tools/sched_ext/scx_qmap.c | 2 +- tools/sched_ext/scx_sdt.c | 2 +- tools/sched_ext/scx_simple.c | 2 +- tools/sched_ext/scx_userland.c | 11 ++- 13 files changed, 181 insertions(+), 64 deletions(-)
--
tejun
Tejun Heo
reviewerPATCH 1/5 tools/sched_ext: Don't restart over a pending exit request
The tools restart when the kernel exits the scheduler with SCX_ECODE_ACT_RESTART. The restart decision doesn't consult exit_req, so an exit request arriving while the restart condition persists is ignored and the tool reloads in a tight loop. Test exit_req before restarting.
scx_userland needs more: its main loop never watches the kernel-side exit and exit_req doubles as the stats printer's stop signal, set by the teardown and reset on each restart. Add the missing UEI_EXITED() test and give the printer its own stop flag so that exit_req only means an exit request and stays latched like in the other tools.
--- tools/sched_ext/scx_central.c | 2 +- tools/sched_ext/scx_cpu0.c | 2 +- tools/sched_ext/scx_flatcg.c | 2 +- tools/sched_ext/scx_pair.c | 2 +- tools/sched_ext/scx_qmap.c | 2 +- tools/sched_ext/scx_sdt.c | 2 +- tools/sched_ext/scx_simple.c | 2 +- tools/sched_ext/scx_userland.c | 11 ++++++----- 8 files changed, 13 insertions(+), 12 deletions(-)
tools/sched_ext/scx_central.c +1 -1
@@ -120,7 +120,7 @@ int main(int argc, char **argv) ecode =UEI_REPORT (skel, uei);scx_central__destroy (skel); -if (UEI_ECODE_RESTART (ecode)) +if (!exit_req &&UEI_ECODE_RESTART (ecode)) goto restart;return 0 ; }
tools/sched_ext/scx_cpu0.c +1 -1
@@ -99,7 +99,7 @@ int main(int argc, char **argv) ecode =UEI_REPORT (skel, uei);scx_cpu0__destroy (skel); -if (UEI_ECODE_RESTART (ecode)) +if (!exit_req &&UEI_ECODE_RESTART (ecode)) goto restart;return 0 ; }
tools/sched_ext/scx_flatcg.c +1 -1
@@ -233,7 +233,7 @@ int main(int argc, char **argv) ecode =UEI_REPORT (skel, uei);scx_flatcg__destroy (skel); -if (UEI_ECODE_RESTART (ecode)) +if (!exit_req &&UEI_ECODE_RESTART (ecode)) goto restart;return 0 ; }
tools/sched_ext/scx_pair.c +1 -1
@@ -190,7 +190,7 @@ int main(int argc, char **argv) ecode =UEI_REPORT (skel, uei);scx_pair__destroy (skel); -if (UEI_ECODE_RESTART (ecode)) +if (!exit_req &&UEI_ECODE_RESTART (ecode)) goto restart;return 0 ; }
tools/sched_ext/scx_qmap.c +1 -1
@@ -465,7 +465,7 @@ int main(int argc, char **argv) ecode =UEI_REPORT (skel, uei);scx_qmap__destroy (skel); -if (UEI_ECODE_RESTART (ecode)) +if (!exit_req &&UEI_ECODE_RESTART (ecode)) goto restart;return 0 ; }
tools/sched_ext/scx_sdt.c +1 -1
@@ -96,7 +96,7 @@ int main(int argc, char **argv) ecode =UEI_REPORT (skel, uei);scx_sdt__destroy (skel); -if (UEI_ECODE_RESTART (ecode)) +if (!exit_req &&UEI_ECODE_RESTART (ecode)) goto restart;return 0 ; }
tools/sched_ext/scx_simple.c +1 -1
@@ -104,7 +104,7 @@ int main(int argc, char **argv) ecode =UEI_REPORT (skel, uei);scx_simple__destroy (skel); -if (UEI_ECODE_RESTART (ecode)) +if (!exit_req &&UEI_ECODE_RESTART (ecode)) goto restart;return 0 ; }
tools/sched_ext/scx_userland.c +6 -5
@@ -52,6 +52,7 @@ static __u32 batch_size = 8;static bool verbose;static volatile int exit_req; +static volatile int stats_stop;static int enqueued_fd, dispatched_fd;static pthread_t stats_printer;
@@ -286,7 +287,7 @@ static void dispatch_batch(void)static void *run_stats_printer (void *arg) { -while (!exit_req) { +while (!stats_stop) {__u64 nr_failed_enqueues, nr_kernel_enqueues, nr_user_enqueues, total; nr_failed_enqueues = skel->bss->nr_failed_enqueues;
@@ -374,7 +375,7 @@ static void pre_bootstrap(int argc, char **argv)static void bootstrap (char *comm) { - exit_req =0 ; + stats_stop =0 ; min_vruntime =0.0 ;__atomic_store_n (&nr_vruntime_enqueues,0 , __ATOMIC_RELAXED);__atomic_store_n (&nr_vruntime_dispatches,0 , __ATOMIC_RELAXED);
@@ -404,7 +405,7 @@ static void bootstrap(char *comm)static void sched_main_loop (void ) { -while (!exit_req) { +while (!exit_req && !UEI_EXITED (skel, uei)) {/* * Perform the following work in the main user space scheduler * loop:
@@ -434,13 +435,13 @@ int main(int argc, char **argv)bootstrap (argv[0 ]);sched_main_loop (); - exit_req =1 ; + stats_stop =1 ;bpf_link__destroy (ops_link);pthread_join (stats_printer,NULL ); ecode =UEI_REPORT (skel, uei);scx_userland__destroy (skel); -if (UEI_ECODE_RESTART (ecode)) +if (!exit_req &&UEI_ECODE_RESTART (ecode)) goto restart;return 0 ; }
Signature · 1 line
2.55.0
Tejun Heo
reviewerPATCH 2/5 sched_ext: Gate local DSQ reenq on baseline cid access
In reply to Tejun Heoscx_bpf_dsq_reenq() with an SCX_DSQ_LOCAL_ON target schedules deferred reenq work on the cid's cpu, raising an IPI when the target rq isn't the locked one. Nothing checks caps along the way, so a sub-sched holding no cap at all on a cid can force its cpu to take IPIs and rq lock cycles at will. The analogous scx_bpf_kick_cid() path gates delivery on SCX_CAP_BASE in kick_one_cpu() to prevent exactly this.
Apply the same rule at the reenq scheduling point: if the calling sched lacks SCX_CAP_BASE on the target cid, drop the reenq and count it in the new SCX_EV_SUB_REENQ_DENIED event. The check is lockless, which is fine: a reenq slipping through right after a revoke is harmless, and a wrong denial can't happen - if the caller has seen its ownership of the cpu, the check sees it too.
--- kernel/sched/ext/ext.c | 12 ++++++++++++ kernel/sched/ext/internal.h | 9 ++++++++- 2 files changed, 20 insertions(+), 1 deletion(-)
kernel/sched/ext/ext.c +12 -0
@@ -1071,6 +1071,18 @@ void schedule_dsq_reenq(struct scx_sched *sch, struct scx_dispatch_q *dsq,if (dsq->id ==SCX_DSQ_LOCAL ) { rq = container_of(dsq, struct rq, scx.local_dsq); +/* +* A sub-sched lacking baseline access on the target cid has no +* business triggering IPIs. The lockless test is fine: slipping +* through right after a revoke is harmless and a wrong denial +* can't happen - if the caller has seen its ownership, so does +* this test. +*/ +if (unlikely (scx_missing_caps (sch,cpu_of (rq),SCX_CAP_BASE ))) { +__scx_add_event (sch,SCX_EV_SUB_REENQ_DENIED ,1 ); +return ; + } +struct scx_sched_pcpu *sch_pcpu =per_cpu_ptr (sch->pcpu,cpu_of (rq));struct scx_deferred_reenq_local *drl = &sch_pcpu->deferred_reenq_local;
kernel/sched/ext/internal.h +8 -1
@@ -1193,6 +1193,12 @@ struct scx_event_stats {* kick degrades to a plain reschedule. */ s64 SCX_EV_SUB_PREEMPT_DENIED ; + +/* +* The number of times a local DSQ reenq was dropped because the +* sub-sched lacked baseline access on the target cid. +*/ +s64 SCX_EV_SUB_REENQ_DENIED ; };#define SCX_EVENTS_LIST (SCX_EVENT ) \
@@ -1212,7 +1218,8 @@ struct scx_event_stats {SCX_EVENT (SCX_EV_INSERT_NOT_OWNED ); \SCX_EVENT (SCX_EV_SUB_BYPASS_DISPATCH ); \SCX_EVENT (SCX_EV_SUB_FORCED_ADMIT ); \ -SCX_EVENT (SCX_EV_SUB_PREEMPT_DENIED) +SCX_EVENT (SCX_EV_SUB_PREEMPT_DENIED ); \ +SCX_EVENT (SCX_EV_SUB_REENQ_DENIED) struct scx_sched;
Signature · 1 line
2.55.0
Tejun Heo
reviewerPATCH 3/5 sched_ext: Count kicks denied for lacking baseline cid access
In reply to Tejun Heokick_one_cpu() silently skips a kick when the kicking sub-sched lacks SCX_CAP_BASE on the target cid, as does kick_one_cpu_if_idle() for idle kicks. The skips are sound with the same logic as the reenq gate but are invisible today, unlike the preempt degradation counted in SCX_EV_SUB_PREEMPT_DENIED. Count them in a new SCX_EV_SUB_KICK_DENIED event so every cap denial is observable.
--- kernel/sched/ext/ext.c | 20 ++++++++++++++------ kernel/sched/ext/internal.h | 8 ++++++++ 2 files changed, 22 insertions(+), 6 deletions(-)
kernel/sched/ext/ext.c +14 -6
@@ -8118,6 +8118,7 @@ static bool kick_one_cpu(s32 cpu, struct scx_sched_pcpu *pcpu, struct rq *this_rstruct scx_rq *this_scx = &this_rq->scx;const struct sched_class *cur_class;bool should_wait = false; +bool kickable;unsigned long flags;raw_spin_rq_lock_irqsave (rq, flags);
@@ -8131,9 +8132,10 @@ static bool kick_one_cpu(s32 cpu, struct scx_sched_pcpu *pcpu, struct rq *this_r * business forcing a reschedule there - skip. This is the authoritative * cap check: ecaps is read here under @rq's lock. */ - if ((cpu_online(cpu) || cpu == cpu_of(this_rq)) && - !sched_class_above(cur_class, &ext_sched_class) && - !scx_missing_caps(pcpu->sch, cpu, SCX_CAP_BASE)) { + kickable = (cpu_online(cpu) || cpu == cpu_of(this_rq)) && + !sched_class_above(cur_class, &ext_sched_class); + + if (kickable && !scx_missing_caps(pcpu->sch, cpu, SCX_CAP_BASE)) { if (cpumask_test_cpu(cpu, pcpu->cpus_to_preempt)) { if (cur_class == &ext_sched_class) { if (likely(!scx_missing_caps(pcpu->sch, cpu,
@@ -8157,6 +8159,9 @@ static bool kick_one_cpu(s32 cpu, struct scx_sched_pcpu *pcpu, struct rq *this_rresched_curr (rq); }else { +/* a kickable cpu was skipped solely for the missing caps */ +if (kickable) +__scx_add_event (pcpu->sch,SCX_EV_SUB_KICK_DENIED ,1 );cpumask_clear_cpu (cpu, pcpu->cpus_to_preempt);cpumask_clear_cpu (cpu, pcpu->cpus_to_wait); }
@@ -8176,9 +8181,12 @@ static void kick_one_cpu_if_idle(s32 cpu, struct scx_sched_pcpu *pcpu,/* idle kicks need baseline access too, see kick_one_cpu() */ if (!can_skip_idle_kick (rq) && - (cpu_online (cpu) || cpu ==cpu_of (this_rq)) && - !scx_missing_caps (pcpu->sch, cpu,SCX_CAP_BASE )) -resched_curr (rq); + (cpu_online (cpu) || cpu ==cpu_of (this_rq))) { +if (likely (!scx_missing_caps (pcpu->sch, cpu,SCX_CAP_BASE ))) +resched_curr (rq); +else +__scx_add_event (pcpu->sch,SCX_EV_SUB_KICK_DENIED ,1 ); + }raw_spin_rq_unlock_irqrestore (rq, flags); }
kernel/sched/ext/internal.h +8 -0
@@ -1194,6 +1194,13 @@ struct scx_event_stats {*/ s64 SCX_EV_SUB_PREEMPT_DENIED ; +/* +* The number of times a kick was skipped because the sub-sched lacked +* baseline access on the target cid. The preempt-part degradation of a +* delivered kick is counted in SCX_EV_SUB_PREEMPT_DENIED instead. +*/ +s64 SCX_EV_SUB_KICK_DENIED ; +/* * The number of times a local DSQ reenq was dropped because the * sub-sched lacked baseline access on the target cid.
@@ -1219,6 +1226,7 @@ struct scx_event_stats {SCX_EVENT (SCX_EV_SUB_BYPASS_DISPATCH ); \SCX_EVENT (SCX_EV_SUB_FORCED_ADMIT ); \SCX_EVENT (SCX_EV_SUB_PREEMPT_DENIED ); \ +SCX_EVENT (SCX_EV_SUB_KICK_DENIED ); \SCX_EVENT (SCX_EV_SUB_REENQ_DENIED) struct scx_sched;
Signature · 1 line
2.55.0
Tejun Heo
reviewerPATCH 4/5 sched_ext: Factor out scx_cpuperf_set()
In reply to Tejun HeoFactor the cpuperf target write out of scx_bpf_cpuperf_set() into scx_cpuperf_set() which takes the acting sched and returns 0 or -errno, and flatten the nested validation into early returns. No functional change. Prep for gating the write behind a cap and reporting the outcome from the cid-form kfunc.
--- kernel/sched/ext/ext.c | 77 ++++++++++++++++++++++++------------------ 1 file changed, 44 insertions(+), 33 deletions(-)
kernel/sched/ext/ext.c +44 -33
@@ -9796,49 +9796,60 @@ __bpf_kfunc u32 scx_bpf_cidperf_cur(s32 cid, const struct bpf_prog_aux *aux)* use. Consult hardware and cpufreq documentation for more information. The * current performance level can be monitored using scx_bpf_cpuperf_cur(). */ -__bpf_kfunc void scx_bpf_cpuperf_set(s32 cpu, u32 perf, const struct bpf_prog_aux *aux) +static s32 scx_cpuperf_set (struct scx_sched *sch,s32 cpu,u32 perf) { -struct scx_sched *sch; - -guard (rcu)(); - - sch =scx_prog_sched (aux); -if (unlikely (!sch)) -return ; +struct rq *rq, *locked_rq; +struct rq_flags rf;if (unlikely (perf >SCX_CPUPERF_ONE )) {scx_error (sch,"Invalid cpuperf target %u for CPU %d" , perf, cpu); -return ; +return -EINVAL ; } -if (scx_cpu_valid (sch, cpu,NULL )) { -struct rq *rq =cpu_rq (cpu), *locked_rq =scx_locked_rq (); -struct rq_flags rf; - -/* -* When called with an rq lock held, restrict the operation -* to the corresponding CPU to prevent ABBA deadlocks. -*/ -if (locked_rq && rq != locked_rq) { -scx_error (sch,"Invalid target CPU %d" , cpu); -return ; - } +if (!scx_cpu_valid (sch, cpu,NULL )) +return -EINVAL ; -/* -* If no rq lock is held, allow to operate on any CPU by -* acquiring the corresponding rq lock. -*/ -if (!locked_rq) { -rq_lock_irqsave (rq, &rf); -update_rq_clock (rq); - } + rq =cpu_rq (cpu); + locked_rq =scx_locked_rq (); - rq->scx.cpuperf_target = perf; -cpufreq_update_util (rq,0 ); +/* +* When called with an rq lock held, restrict the operation to the +* corresponding CPU to prevent ABBA deadlocks. +*/ +if (locked_rq && rq != locked_rq) { +scx_error (sch,"Invalid target CPU %d" , cpu); +return -EINVAL ; + } -if (!locked_rq) -rq_unlock_irqrestore (rq, &rf); +/* +* If no rq lock is held, allow to operate on any CPU by acquiring +* the corresponding rq lock. +*/ +if (!locked_rq) { +rq_lock_irqsave (rq, &rf); +update_rq_clock (rq); } + + rq->scx.cpuperf_target = perf; +cpufreq_update_util (rq,0 ); + +if (!locked_rq) +rq_unlock_irqrestore (rq, &rf); + +return 0 ; +} + +__bpf_kfunc void scx_bpf_cpuperf_set(s32 cpu, u32 perf, const struct bpf_prog_aux *aux) +{ +struct scx_sched *sch; + +guard (rcu)(); + + sch =scx_prog_sched (aux); +if (unlikely (!sch)) +return ; + +scx_cpuperf_set (sch, cpu, perf); }/**
Signature · 1 line
2.55.0
Tejun Heo
reviewerPATCH 5/5 sched_ext: Gate scx_bpf_cidperf_set() behind a new SCX_CAP_PERF
In reply to Tejun Heoscx_bpf_cidperf_set() reaches cpufreq with no cap check, so any cid-form sub-sched can steer the frequency of any cid in its view, including ones it holds nothing on.
Gate it behind a new SCX_CAP_PERF rather than SCX_CAP_BASE: hardware control is a separate axis from queue access - a parent may well delegate scheduling on a cid without handing over its frequency. PERF neither implies nor is implied by the other caps. The check runs under the target rq's lock, which ecaps updates are also folded under, so it is authoritative - a write can never land after a revoke has taken effect. Denials are counted in SCX_EV_SUB_CIDPERF_DENIED.
The operation is synchronous and the outcome is reported to the caller. scx_bpf_cidperf_set() shipped in v7.2 returning void, so add scx_bpf_cidperf_set___v2() returning 0 or -errno, -EACCES on denial, and keep the old kfunc as a gated void wrapper. The tools compat wrapper prefers the v2 and returns 0 on pre-gate kernels, which always apply the write.
scx_qmap grants PERF alongside its existing cid grants so the cpuperf demo keeps working in sub-scheds.
--- kernel/sched/ext/ext.c | 53 +++++++++++++++++++----- kernel/sched/ext/internal.h | 16 ++++++- tools/sched_ext/include/scx/common.bpf.h | 2 +- tools/sched_ext/include/scx/compat.bpf.h | 18 ++++++++ tools/sched_ext/scx_qmap.bpf.c | 17 +++++--- 5 files changed, 88 insertions(+), 18 deletions(-)
kernel/sched/ext/ext.c +43 -10
@@ -5144,6 +5144,7 @@ static const char *scx_cap_names[__SCX_NR_CAPS] = { [__SCX_CAP_ENQ_IMMED] = "enq_immed", [__SCX_CAP_ENQ] = "enq", [__SCX_CAP_PREEMPT] = "preempt", + [__SCX_CAP_PERF] = "perf", }; static ssize_t scx_attr_caps_show(struct kobject *kobj,
@@ -9800,6 +9801,7 @@ static s32 scx_cpuperf_set(struct scx_sched *sch, s32 cpu, u32 perf) {struct rq *rq, *locked_rq;struct rq_flags rf; +s32 ret;if (unlikely (perf >SCX_CPUPERF_ONE )) {scx_error (sch,"Invalid cpuperf target %u for CPU %d" , perf, cpu);
@@ -9830,13 +9832,24 @@ static s32 scx_cpuperf_set(struct scx_sched *sch, s32 cpu, u32 perf)update_rq_clock (rq); } - rq->scx.cpuperf_target = perf; -cpufreq_update_util (rq,0 ); +/* +* ecaps updates are folded under the rq lock, making this test +* authoritative: a write can never land after a revoke has taken +* effect on @cpu. +*/ +if (likely (!scx_missing_caps (sch, cpu,SCX_CAP_PERF ))) { + rq->scx.cpuperf_target = perf; +cpufreq_update_util (rq,0 ); + ret =0 ; + }else { +__scx_add_event (sch,SCX_EV_SUB_CIDPERF_DENIED ,1 ); + ret = -EACCES ; + }if (!locked_rq)rq_unlock_irqrestore (rq, &rf); -return 0 ; +return ret; } __bpf_kfunc void scx_bpf_cpuperf_set(s32 cpu, u32 perf, const struct bpf_prog_aux *aux)
@@ -9853,15 +9866,18 @@ __bpf_kfunc void scx_bpf_cpuperf_set(s32 cpu, u32 perf, const struct bpf_prog_au }/** -* scx_bpf_cidperf_set - Set the performance target of the CPU at @cid +* scx_bpf_cidperf_set___v2 - Set the performance target of the CPU at @cid * @cid: cid of the CPU to target * @perf: target performance level [0, %SCX_CPUPERF_ONE] * @aux: implicit BPF argument to access bpf_prog_aux hidden from BPF progs * -* cid-addressed equivalent of scx_bpf_cpuperf_set(). +* cid-addressed equivalent of scx_bpf_cpuperf_set(). A sub-sched needs +* SCX_CAP_PERF on @cid. Returns 0 if the target was applied, -%EACCES if +* the write was denied for missing caps, other -errnos if @cid didn't +* resolve. */ -__bpf_kfunc void scx_bpf_cidperf_set(s32 cid, u32 perf, -const struct bpf_prog_aux *aux) +__bpf_kfunc s32 scx_bpf_cidperf_set___v2(s32 cid, u32 perf, +const struct bpf_prog_aux *aux) {struct scx_sched *sch;s32 cpu;
@@ -9870,11 +9886,27 @@ __bpf_kfunc void scx_bpf_cidperf_set(s32 cid, u32 perf, sch =scx_prog_sched (aux);if (unlikely (!sch)) -return ; +return -ENODEV ; cpu =scx_cid_to_cpu (sch, cid);if (cpu <0 ) -return ; -scx_bpf_cpuperf_set (cpu, perf, aux); +return cpu; + +return scx_cpuperf_set (sch, cpu, perf); +} + +/** +* scx_bpf_cidperf_set - Set the performance target of the CPU at @cid +* @cid: cid of the CPU to target +* @perf: target performance level [0, %SCX_CPUPERF_ONE] +* @aux: implicit BPF argument to access bpf_prog_aux hidden from BPF progs +* +* Pre-v7.3 variant of scx_bpf_cidperf_set___v2() without the result. Kept +* for schedulers built against older kernels. +*/ +__bpf_kfunc void scx_bpf_cidperf_set(s32 cid, u32 perf, +const struct bpf_prog_aux *aux) +{ +scx_bpf_cidperf_set___v2 (cid, perf, aux); }/**
@@ -10306,6 +10338,7 @@ BTF_ID_FLAGS(func, scx_bpf_cpuperf_set, KF_IMPLICIT_ARGS) BTF_ID_FLAGS(func, scx_bpf_cidperf_cap, KF_IMPLICIT_ARGS) BTF_ID_FLAGS(func, scx_bpf_cidperf_cur, KF_IMPLICIT_ARGS) BTF_ID_FLAGS(func, scx_bpf_cidperf_set, KF_IMPLICIT_ARGS) +BTF_ID_FLAGS(func, scx_bpf_cidperf_set___v2, KF_IMPLICIT_ARGS) BTF_ID_FLAGS(func, scx_bpf_nr_node_ids) BTF_ID_FLAGS(func, scx_bpf_nr_cpu_ids)BTF_ID_FLAGS (func, scx_bpf_nr_cids)
kernel/sched/ext/internal.h +15 -1
@@ -1206,6 +1206,12 @@ struct scx_event_stats {* sub-sched lacked baseline access on the target cid. */ s64 SCX_EV_SUB_REENQ_DENIED ; + +/* +* The number of times scx_bpf_cidperf_set() was denied because the +* sub-sched lacked SCX_CAP_PERF on the target cid. +*/ +s64 SCX_EV_SUB_CIDPERF_DENIED ; };#define SCX_EVENTS_LIST (SCX_EVENT ) \
@@ -1227,7 +1233,8 @@ struct scx_event_stats {SCX_EVENT (SCX_EV_SUB_FORCED_ADMIT ); \SCX_EVENT (SCX_EV_SUB_PREEMPT_DENIED ); \SCX_EVENT (SCX_EV_SUB_KICK_DENIED ); \ -SCX_EVENT (SCX_EV_SUB_REENQ_DENIED) +SCX_EVENT (SCX_EV_SUB_REENQ_DENIED ); \ +SCX_EVENT (SCX_EV_SUB_CIDPERF_DENIED) struct scx_sched;
@@ -1353,6 +1360,11 @@ struct scx_sched_pnode {* - SCX_ENQ_PREEMPT inserts * - SCX_KICK_PREEMPT kicks * +* PERF control the cid's cpu power/perf management state, currently the +* cpufreq target set through scx_bpf_cidperf_set(). Hardware +* control is a separate axis from queue access: PERF neither +* implies nor is implied by the caps above. +* * Implied caps apply to the holder's own use of a cid, not to delegation. * scx_bpf_sub_grant() delegates literally-held caps, so a cap held only through * implication is usable but cannot be re-delegated to a child. When granting a
@@ -1363,6 +1375,7 @@ enum scx_cap_flags { __SCX_CAP_ENQ_IMMED =0 , __SCX_CAP_ENQ =1 , __SCX_CAP_PREEMPT =2 , + __SCX_CAP_PERF =3 , __SCX_NR_CAPS, __SCX_CAP_ALL = BIT_U64(__SCX_NR_CAPS) - 1,
@@ -1370,6 +1383,7 @@ enum scx_cap_flags {SCX_CAP_ENQ_IMMED =BIT_U64 (__SCX_CAP_ENQ_IMMED),SCX_CAP_ENQ =BIT_U64 (__SCX_CAP_ENQ),SCX_CAP_PREEMPT =BIT_U64 (__SCX_CAP_PREEMPT), +SCX_CAP_PERF =BIT_U64 (__SCX_CAP_PERF),/* alias for minimal cap to make any use of a cpu */ SCX_CAP_BASE = SCX_CAP_ENQ_IMMED,
tools/sched_ext/include/scx/common.bpf.h +1 -1
@@ -112,7 +112,7 @@ u32 scx_bpf_nr_cids(void) __ksym __weak; u32 scx_bpf_nr_online_cids(void) __ksym __weak; u32 scx_bpf_cidperf_cap(s32 cid) __ksym __weak; u32 scx_bpf_cidperf_cur(s32 cid) __ksym __weak; -void scx_bpf_cidperf_set(s32 cid, u32 perf) __ksym __weak; +/* scx_bpf_cidperf_set() is wrapped in compat.bpf.h */ /* sub-scheduler cap control, scx_bpf_sub_caps() cgroup_id 0 == self */ s32 scx_bpf_sub_grant(u64 cgroup_id, u64 caps, const struct scx_cmask *cmask,
tools/sched_ext/include/scx/compat.bpf.h +18 -0
@@ -420,6 +420,24 @@ static inline void scx_bpf_dsq_reenq(u64 dsq_id, u64 reenq_flags)scx_bpf_error ("kernel too old to reenqueue foreign local or user DSQs" ); } +/* +* v7.3: scx_bpf_cidperf_set() now returns 0 or -errno - -EACCES when a +* sub-sched lacks SCX_CAP_PERF on the cid and the write was denied. Pre-gate +* kernels always apply the write. Drop the wrapper and move the decl to +* common.bpf.h after v7.6. +*/ +s32 scx_bpf_cidperf_set___v2___compat(s32 cid, u32 perf) __ksym __weak; +void scx_bpf_cidperf_set___compat(s32 cid, u32 perf) __ksym __weak; + +static inline s32 scx_bpf_cidperf_set (s32 cid,u32 perf) +{ +if (bpf_ksym_exists (scx_bpf_cidperf_set___v2___compat)) +return scx_bpf_cidperf_set___v2___compat (cid, perf); + +scx_bpf_cidperf_set___compat (cid, perf); +return 0 ; +} +/* * Define sched_ext_ops. See compat.h::SCX_OPS_OPEN() for how backward * compatibility is handled (this macro can be expanded to emit multiple
tools/sched_ext/scx_qmap.bpf.c +11 -6
@@ -1585,11 +1585,13 @@ __noinline void apply_partition(void)cmask_copy (&qa.to_grant_cids.mask, &ssc->granted_cids.mask);cmask_andnot (&qa.to_grant_cids.mask, &ssc->prev_granted.mask); -scx_bpf_sub_revoke (cgid,SCX_CAP_ENQ_IMMED , +scx_bpf_sub_revoke (cgid,SCX_CAP_ENQ_IMMED |SCX_CAP_PERF , (void *)(long )&qa.prev_rr_cids.mask); -scx_bpf_sub_revoke (cgid,SCX_CAP_ENQ |SCX_CAP_PREEMPT |SCX_CAP_ENQ_IMMED , +scx_bpf_sub_revoke (cgid,SCX_CAP_ENQ |SCX_CAP_PREEMPT | +SCX_CAP_ENQ_IMMED |SCX_CAP_PERF , (void *)(long )&qa.to_revoke_cids.mask); -scx_bpf_sub_grant (cgid,SCX_CAP_ENQ |SCX_CAP_PREEMPT |SCX_CAP_ENQ_IMMED , +scx_bpf_sub_grant (cgid,SCX_CAP_ENQ |SCX_CAP_PREEMPT | +SCX_CAP_ENQ_IMMED |SCX_CAP_PERF , (void *)(long )&qa.to_grant_cids.mask,NULL ); }
@@ -1605,7 +1607,8 @@ __noinline void apply_partition(void) holder_cgid = qa.part.rr_slots[pos];/* 0 = self, nothing to grant */ if (holder_cgid) -scx_bpf_sub_grant (holder_cgid,SCX_CAP_ENQ_IMMED , +scx_bpf_sub_grant (holder_cgid, +SCX_CAP_ENQ_IMMED |SCX_CAP_PERF , (void *)(long )&qa.rr_cids.mask,NULL ); } }
@@ -1696,10 +1699,12 @@ static void rr_advance(void)* time-share. */ if (old_cgid) -scx_bpf_sub_revoke (old_cgid,SCX_CAP_ENQ_IMMED , +scx_bpf_sub_revoke (old_cgid, +SCX_CAP_ENQ_IMMED |SCX_CAP_PERF , (void *)(long )&qa.rr_cids.mask);if (new_cgid) -scx_bpf_sub_grant (new_cgid,SCX_CAP_ENQ_IMMED , +scx_bpf_sub_grant (new_cgid, +SCX_CAP_ENQ_IMMED |SCX_CAP_PERF , (void *)(long )&qa.rr_cids.mask,NULL ); }
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH 4/5] sched_ext: Factor out scx_cpuperf_set()
In reply to Tejun HeoThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] Kernel-doc comment disconnected from its intended function. --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Kernel-doc comment disconnected from its intended function.
--
--- Patch [4]: [PATCH 4/5] sched_ext: Factor out scx_cpuperf_set() --- commit b27126d56fa54ad2d995a8ef1399b04f6ce7c044
Author: Tejun Heo <[email protected]>
sched_ext: Factor out scx_cpuperf_set()
This commit factors out the CPU performance target write from scx_bpf_cpuperf_set() into a new internal scx_cpuperf_set() helper. It also flattens the nested validation into early returns to prepare for future changes.
kernel/sched/ext/ext.c __bpf_kfunc u32 scx_bpf_cidperf_cur(s32 cid, const struct bpf_prog_aux *aux) · line 9796
Quoted 10 lines
kernel/sched/ext/ext.c +1
-1@@ -9793,49 +9793,60 @@ __bpf_kfunc u32 scx_bpf_cidperf_cur(s32 cid, const struct bpf_prog_aux *aux)* use. Consult hardware and cpufreq documentation for more information. The * current performance level can be monitored using scx_bpf_cpuperf_cur(). */ -__bpf_kfunc void scx_bpf_cpuperf_set(s32 cpu, u32 perf, const struct bpf_prog_aux *aux) +static s32 scx_cpuperf_set (struct scx_sched *sch,s32 cpu,u32 perf)
[Severity: Low]
Does this disconnect the kernel-doc comment from its intended function?
By placing the new static helper scx_cpuperf_set() directly under the kernel-doc block for scx_bpf_cpuperf_set(), kernel-doc will parse this as the documentation for the helper instead of the kfunc.
This will trigger kernel-doc warnings during documentation builds about mismatched function names and arguments (such as missing @sch and @aux), and leaves the public kfunc scx_bpf_cpuperf_set() undocumented in the generated kernel documentation.
Quoted 8 lines
kernel/sched/ext/ext.c +0
-7{ -struct scx_sched *sch; - -guard (rcu)(); - - sch =scx_prog_sched (aux); -if (unlikely (!sch)) -return ;
[ ... ]
Quoted 9 lines
kernel/sched/ext/ext.c +9
-0+if (!locked_rq) +rq_unlock_irqrestore (rq, &rf); + +return 0 ; +} + +__bpf_kfunc void scx_bpf_cpuperf_set(s32 cpu, u32 perf, const struct bpf_prog_aux *aux) +{ +struct scx_sched *sch;
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
[email protected]
AI reviewRe: [PATCH 5/5] sched_ext: Gate scx_bpf_cidperf_set() behind a new SCX_CAP_PERF
In reply to Tejun HeoThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] The scx_qmap scheduler breaks backward compatibility on older kernels by unconditionally passing…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The scx_qmap scheduler breaks backward compatibility on older kernels by unconditionally passing SCX_CAP_PERF to scx_bpf_sub_grant() and scx_bpf_sub_revoke(). --
--- Patch [5]: [PATCH 5/5] sched_ext: Gate scx_bpf_cidperf_set() behind a new SCX_CAP_PERF --- commit 3bc27e508735d52ab01e9e24c0c210b6cb92366e
Author: Tejun Heo <[email protected]>
sched_ext: Gate scx_bpf_cidperf_set() behind a new SCX_CAP_PERF
scx_bpf_cidperf_set() reaches cpufreq with no cap check, so any cid-form sub-sched can steer the frequency of any cid in its view. This gates the operation behind a new SCX_CAP_PERF capability check.
Quoted 10 lines
tools/sched_ext/scx_qmap.bpf.c +1
-1@@ -1585,11 +1585,13 @@ __noinline void apply_partition(void)cmask_copy (&qa.to_grant_cids.mask, &ssc->granted_cids.mask);cmask_andnot (&qa.to_grant_cids.mask, &ssc->prev_granted.mask); - scx_bpf_sub_revoke(cgid, SCX_CAP_ENQ_IMMED, + scx_bpf_sub_revoke(cgid, SCX_CAP_ENQ_IMMED | SCX_CAP_PERF,
tools/sched_ext/scx_qmap.bpf.c __noinline void apply_partition(void) · line 1596
Quoted 10 lines
tools/sched_ext/scx_qmap.bpf.c +4
-2(void *)(long)&qa.prev_rr_cids.mask); -scx_bpf_sub_revoke (cgid,SCX_CAP_ENQ |SCX_CAP_PREEMPT |SCX_CAP_ENQ_IMMED , +scx_bpf_sub_revoke (cgid,SCX_CAP_ENQ |SCX_CAP_PREEMPT | +SCX_CAP_ENQ_IMMED |SCX_CAP_PERF , (void *)(long )&qa.to_revoke_cids.mask); -scx_bpf_sub_grant (cgid,SCX_CAP_ENQ |SCX_CAP_PREEMPT |SCX_CAP_ENQ_IMMED , +scx_bpf_sub_grant (cgid,SCX_CAP_ENQ |SCX_CAP_PREEMPT | +SCX_CAP_ENQ_IMMED |SCX_CAP_PERF , (void *)(long )&qa.to_grant_cids.mask,NULL ); }
[Severity: High]
Does unconditionally passing SCX_CAP_PERF break backward compatibility on older kernels?
If scx_qmap is loaded on a pre-gate kernel where SCX_CAP_PERF is not part of __SCX_CAP_ALL, won't the kernel's scx_bpf_sub_grant() and scx_bpf_sub_revoke() reject the unknown cap bit?
Since they call scx_error() on invalid caps, would this immediately abort the BPF scheduler and revert the system to CFS on older kernels?
[ ... ]
Quoted 4 lines
tools/sched_ext/scx_qmap.bpf.c +0
-0@@ -1696,10 +1699,12 @@ static void rr_advance(void)* time-share. */ if (old_cgid)
tools/sched_ext/scx_qmap.bpf.c static void rr_advance(void) · line 1709
Quoted 10 lines
tools/sched_ext/scx_qmap.bpf.c +4
-2-scx_bpf_sub_revoke (old_cgid,SCX_CAP_ENQ_IMMED , +scx_bpf_sub_revoke (old_cgid, +SCX_CAP_ENQ_IMMED |SCX_CAP_PERF , (void *)(long )&qa.rr_cids.mask);if (new_cgid) -scx_bpf_sub_grant (new_cgid,SCX_CAP_ENQ_IMMED , +scx_bpf_sub_grant (new_cgid, +SCX_CAP_ENQ_IMMED |SCX_CAP_PERF , (void *)(long )&qa.rr_cids.mask,NULL ); }
[Severity: High]
Does the same compatibility issue apply here during the round-robin advance in rr_advance()?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
Tejun Heo
reviewerRe: [PATCH 4/5] sched_ext: Factor out scx_cpuperf_set()
In reply to [email protected]Quoted 6 lines · On Fri, Jul 24, 2026 at 06:30:56PM +0000, [email protected] wrote:
[Severity: Low]
Does this disconnect the kernel-doc comment from its intended function?By placing the new static helper
scx_cpuperf_set()directly under the kernel-doc block forscx_bpf_cpuperf_set(), kernel-doc will parse this as the documentation for the helper instead of the kfunc.
Indeed, the factoring left the kernel-doc comment attached to the helper. Will move it to the kfunc when applying.
Thanks.
--
tejun
Tejun Heo
reviewerRe: [PATCH 5/5] sched_ext: Gate scx_bpf_cidperf_set() behind a new SCX_CAP_PERF
In reply to [email protected]Quoted 7 lines · On Fri, Jul 24, 2026 at 06:46:05PM +0000, [email protected] wrote:
[Severity: High]
Does unconditionally passingSCX_CAP_PERFbreak backward compatibility on older kernels?If
scx_qmapis loaded on a pre-gate kernel whereSCX_CAP_PERFis not part of__SCX_CAP_ALL, won't the kernel'sscx_bpf_sub_grant()andscx_bpf_sub_revoke()reject the unknown cap bit?
It would, but that's fine. The sub-scheduler caps interface is still going through initial development and there is no cross-version compatibility commitment for it yet. The in-tree schedulers track the in-tree kernel for these interfaces.
Thanks.
--
tejun