sched_ext: Follow-up fixes and missing cap enforcement
Subsystems (2)
Tejun Heo
reviewerPATCH v2 1/1 sched_ext: Follow-up fixes and missing cap enforcement
Hello,
Follow-up fixes and enforcement holes plugged from the recent sub-scheduler work. The first patch fixes the tools' restart loop clobbering an exit that was requested while a restart was pending.
The rest tighten how cid-addressed operations from sub-schedulers are handled when the caller lacks the necessary caps. Reenq to another sched's local DSQs is now denied and counted, and a new event counts the already-denied kicks. These checks are lockless - transient false positives from cap rotation are harmless and false negatives can't happen by visibility. cpuperf control moves behind a new SCX_CAP_PERF which only the root holds by default, and scx_bpf_cidperf_set() reports the result synchronously as its check is authoritative under the target's rq lock.
v2: - Kernel-doc comment moved to scx_bpf_cpuperf_set() instead of being
left attached to the factored-out helper (0004, reported by sashiko).
- The versioned scx_bpf_cidperf_set___v2() interface and the tools compat wrapper are dropped. The cid-form interface is still under initial development, so the signature is changed in place to return s32 (0005).
v1: https://lore.kernel.org/r/[email protected]
0001-tools-sched_ext-Don-t-restart-over-a-pending-exit-re.patch 0002-sched_ext-Gate-local-DSQ-reenq-on-baseline-cid-acces.patch 0003-sched_ext-Count-kicks-denied-for-lacking-baseline-ci.patch 0004-sched_ext-Factor-out-scx_cpuperf_set.patch 0005-sched_ext-Gate-scx_bpf_cidperf_set-behind-a-new-SCX_.patch
Based on sched_ext/for-7.3 (c509d8107026).
The patches are also available in the following git branch:
git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext.git scx-sub-cap-gates-v2
diffstat follows. Thanks.
kernel/sched/ext/ext.c | 139 +++++++++++++++++++++---------- kernel/sched/ext/internal.h | 31 ++++++- tools/sched_ext/include/scx/common.bpf.h | 2 +- tools/sched_ext/scx_central.c | 2 +- tools/sched_ext/scx_cpu0.c | 2 +- tools/sched_ext/scx_flatcg.c | 2 +- tools/sched_ext/scx_pair.c | 2 +- tools/sched_ext/scx_qmap.bpf.c | 17 ++-- tools/sched_ext/scx_qmap.c | 2 +- tools/sched_ext/scx_sdt.c | 2 +- tools/sched_ext/scx_simple.c | 2 +- tools/sched_ext/scx_userland.c | 11 +-- 12 files changed, 149 insertions(+), 65 deletions(-)
--
tejun
Tejun Heo
reviewerPATCH 1/5 tools/sched_ext: Don't restart over a pending exit request
The tools restart when the kernel exits the scheduler with SCX_ECODE_ACT_RESTART. The restart decision doesn't consult exit_req, so an exit request arriving while the restart condition persists is ignored and the tool reloads in a tight loop. Test exit_req before restarting.
scx_userland needs more: its main loop never watches the kernel-side exit and exit_req doubles as the stats printer's stop signal, set by the teardown and reset on each restart. Add the missing UEI_EXITED() test and give the printer its own stop flag so that exit_req only means an exit request and stays latched like in the other tools.
--- tools/sched_ext/scx_central.c | 2 +- tools/sched_ext/scx_cpu0.c | 2 +- tools/sched_ext/scx_flatcg.c | 2 +- tools/sched_ext/scx_pair.c | 2 +- tools/sched_ext/scx_qmap.c | 2 +- tools/sched_ext/scx_sdt.c | 2 +- tools/sched_ext/scx_simple.c | 2 +- tools/sched_ext/scx_userland.c | 11 ++++++----- 8 files changed, 13 insertions(+), 12 deletions(-)
tools/sched_ext/scx_central.c +1 -1
@@ -120,7 +120,7 @@ int main(int argc, char **argv) ecode =UEI_REPORT (skel, uei);scx_central__destroy (skel); -if (UEI_ECODE_RESTART (ecode)) +if (!exit_req &&UEI_ECODE_RESTART (ecode)) goto restart;return 0 ; }
tools/sched_ext/scx_cpu0.c +1 -1
@@ -99,7 +99,7 @@ int main(int argc, char **argv) ecode =UEI_REPORT (skel, uei);scx_cpu0__destroy (skel); -if (UEI_ECODE_RESTART (ecode)) +if (!exit_req &&UEI_ECODE_RESTART (ecode)) goto restart;return 0 ; }
tools/sched_ext/scx_flatcg.c +1 -1
@@ -233,7 +233,7 @@ int main(int argc, char **argv) ecode =UEI_REPORT (skel, uei);scx_flatcg__destroy (skel); -if (UEI_ECODE_RESTART (ecode)) +if (!exit_req &&UEI_ECODE_RESTART (ecode)) goto restart;return 0 ; }
tools/sched_ext/scx_pair.c +1 -1
@@ -190,7 +190,7 @@ int main(int argc, char **argv) ecode =UEI_REPORT (skel, uei);scx_pair__destroy (skel); -if (UEI_ECODE_RESTART (ecode)) +if (!exit_req &&UEI_ECODE_RESTART (ecode)) goto restart;return 0 ; }
tools/sched_ext/scx_qmap.c +1 -1
@@ -465,7 +465,7 @@ int main(int argc, char **argv) ecode =UEI_REPORT (skel, uei);scx_qmap__destroy (skel); -if (UEI_ECODE_RESTART (ecode)) +if (!exit_req &&UEI_ECODE_RESTART (ecode)) goto restart;return 0 ; }
tools/sched_ext/scx_sdt.c +1 -1
@@ -96,7 +96,7 @@ int main(int argc, char **argv) ecode =UEI_REPORT (skel, uei);scx_sdt__destroy (skel); -if (UEI_ECODE_RESTART (ecode)) +if (!exit_req &&UEI_ECODE_RESTART (ecode)) goto restart;return 0 ; }
tools/sched_ext/scx_simple.c +1 -1
@@ -104,7 +104,7 @@ int main(int argc, char **argv) ecode =UEI_REPORT (skel, uei);scx_simple__destroy (skel); -if (UEI_ECODE_RESTART (ecode)) +if (!exit_req &&UEI_ECODE_RESTART (ecode)) goto restart;return 0 ; }
tools/sched_ext/scx_userland.c +6 -5
@@ -52,6 +52,7 @@ static __u32 batch_size = 8;static bool verbose;static volatile int exit_req; +static volatile int stats_stop;static int enqueued_fd, dispatched_fd;static pthread_t stats_printer;
@@ -286,7 +287,7 @@ static void dispatch_batch(void)static void *run_stats_printer (void *arg) { -while (!exit_req) { +while (!stats_stop) {__u64 nr_failed_enqueues, nr_kernel_enqueues, nr_user_enqueues, total; nr_failed_enqueues = skel->bss->nr_failed_enqueues;
@@ -374,7 +375,7 @@ static void pre_bootstrap(int argc, char **argv)static void bootstrap (char *comm) { - exit_req =0 ; + stats_stop =0 ; min_vruntime =0.0 ;__atomic_store_n (&nr_vruntime_enqueues,0 , __ATOMIC_RELAXED);__atomic_store_n (&nr_vruntime_dispatches,0 , __ATOMIC_RELAXED);
@@ -404,7 +405,7 @@ static void bootstrap(char *comm)static void sched_main_loop (void ) { -while (!exit_req) { +while (!exit_req && !UEI_EXITED (skel, uei)) {/* * Perform the following work in the main user space scheduler * loop:
@@ -434,13 +435,13 @@ int main(int argc, char **argv)bootstrap (argv[0 ]);sched_main_loop (); - exit_req =1 ; + stats_stop =1 ;bpf_link__destroy (ops_link);pthread_join (stats_printer,NULL ); ecode =UEI_REPORT (skel, uei);scx_userland__destroy (skel); -if (UEI_ECODE_RESTART (ecode)) +if (!exit_req &&UEI_ECODE_RESTART (ecode)) goto restart;return 0 ; }
Signature · 1 line
2.55.0
Tejun Heo
reviewerPATCH 2/5 sched_ext: Gate local DSQ reenq on baseline cid access
In reply to Tejun Heoscx_bpf_dsq_reenq() with an SCX_DSQ_LOCAL_ON target schedules deferred reenq work on the cid's cpu, raising an IPI when the target rq isn't the locked one. Nothing checks caps along the way, so a sub-sched holding no cap at all on a cid can force its cpu to take IPIs and rq lock cycles at will. The analogous scx_bpf_kick_cid() path gates delivery on SCX_CAP_BASE in kick_one_cpu() to prevent exactly this.
Apply the same rule at the reenq scheduling point: if the calling sched lacks SCX_CAP_BASE on the target cid, drop the reenq and count it in the new SCX_EV_SUB_REENQ_DENIED event. The check is lockless, which is fine: a reenq slipping through right after a revoke is harmless, and a wrong denial can't happen - if the caller has seen its ownership of the cpu, the check sees it too.
--- kernel/sched/ext/ext.c | 12 ++++++++++++ kernel/sched/ext/internal.h | 9 ++++++++- 2 files changed, 20 insertions(+), 1 deletion(-)
kernel/sched/ext/ext.c +12 -0
@@ -1071,6 +1071,18 @@ void schedule_dsq_reenq(struct scx_sched *sch, struct scx_dispatch_q *dsq,if (dsq->id ==SCX_DSQ_LOCAL ) { rq = container_of(dsq, struct rq, scx.local_dsq); +/* +* A sub-sched lacking baseline access on the target cid has no +* business triggering IPIs. The lockless test is fine: slipping +* through right after a revoke is harmless and a wrong denial +* can't happen - if the caller has seen its ownership, so does +* this test. +*/ +if (unlikely (scx_missing_caps (sch,cpu_of (rq),SCX_CAP_BASE ))) { +__scx_add_event (sch,SCX_EV_SUB_REENQ_DENIED ,1 ); +return ; + } +struct scx_sched_pcpu *sch_pcpu =per_cpu_ptr (sch->pcpu,cpu_of (rq));struct scx_deferred_reenq_local *drl = &sch_pcpu->deferred_reenq_local;
kernel/sched/ext/internal.h +8 -1
@@ -1193,6 +1193,12 @@ struct scx_event_stats {* kick degrades to a plain reschedule. */ s64 SCX_EV_SUB_PREEMPT_DENIED ; + +/* +* The number of times a local DSQ reenq was dropped because the +* sub-sched lacked baseline access on the target cid. +*/ +s64 SCX_EV_SUB_REENQ_DENIED ; };#define SCX_EVENTS_LIST (SCX_EVENT ) \
@@ -1212,7 +1218,8 @@ struct scx_event_stats {SCX_EVENT (SCX_EV_INSERT_NOT_OWNED ); \SCX_EVENT (SCX_EV_SUB_BYPASS_DISPATCH ); \SCX_EVENT (SCX_EV_SUB_FORCED_ADMIT ); \ -SCX_EVENT (SCX_EV_SUB_PREEMPT_DENIED) +SCX_EVENT (SCX_EV_SUB_PREEMPT_DENIED ); \ +SCX_EVENT (SCX_EV_SUB_REENQ_DENIED) struct scx_sched;
Signature · 1 line
2.55.0
Tejun Heo
reviewerPATCH 3/5 sched_ext: Count kicks denied for lacking baseline cid access
In reply to Tejun Heokick_one_cpu() silently skips a kick when the kicking sub-sched lacks SCX_CAP_BASE on the target cid, as does kick_one_cpu_if_idle() for idle kicks. The skips are sound with the same logic as the reenq gate but are invisible today, unlike the preempt degradation counted in SCX_EV_SUB_PREEMPT_DENIED. Count them in a new SCX_EV_SUB_KICK_DENIED event so every cap denial is observable.
--- kernel/sched/ext/ext.c | 20 ++++++++++++++------ kernel/sched/ext/internal.h | 8 ++++++++ 2 files changed, 22 insertions(+), 6 deletions(-)
kernel/sched/ext/ext.c +14 -6
@@ -8118,6 +8118,7 @@ static bool kick_one_cpu(s32 cpu, struct scx_sched_pcpu *pcpu, struct rq *this_rstruct scx_rq *this_scx = &this_rq->scx;const struct sched_class *cur_class;bool should_wait = false; +bool kickable;unsigned long flags;raw_spin_rq_lock_irqsave (rq, flags);
@@ -8131,9 +8132,10 @@ static bool kick_one_cpu(s32 cpu, struct scx_sched_pcpu *pcpu, struct rq *this_r * business forcing a reschedule there - skip. This is the authoritative * cap check: ecaps is read here under @rq's lock. */ - if ((cpu_online(cpu) || cpu == cpu_of(this_rq)) && - !sched_class_above(cur_class, &ext_sched_class) && - !scx_missing_caps(pcpu->sch, cpu, SCX_CAP_BASE)) { + kickable = (cpu_online(cpu) || cpu == cpu_of(this_rq)) && + !sched_class_above(cur_class, &ext_sched_class); + + if (kickable && !scx_missing_caps(pcpu->sch, cpu, SCX_CAP_BASE)) { if (cpumask_test_cpu(cpu, pcpu->cpus_to_preempt)) { if (cur_class == &ext_sched_class) { if (likely(!scx_missing_caps(pcpu->sch, cpu,
@@ -8157,6 +8159,9 @@ static bool kick_one_cpu(s32 cpu, struct scx_sched_pcpu *pcpu, struct rq *this_rresched_curr (rq); }else { +/* a kickable cpu was skipped solely for the missing caps */ +if (kickable) +__scx_add_event (pcpu->sch,SCX_EV_SUB_KICK_DENIED ,1 );cpumask_clear_cpu (cpu, pcpu->cpus_to_preempt);cpumask_clear_cpu (cpu, pcpu->cpus_to_wait); }
@@ -8176,9 +8181,12 @@ static void kick_one_cpu_if_idle(s32 cpu, struct scx_sched_pcpu *pcpu,/* idle kicks need baseline access too, see kick_one_cpu() */ if (!can_skip_idle_kick (rq) && - (cpu_online (cpu) || cpu ==cpu_of (this_rq)) && - !scx_missing_caps (pcpu->sch, cpu,SCX_CAP_BASE )) -resched_curr (rq); + (cpu_online (cpu) || cpu ==cpu_of (this_rq))) { +if (likely (!scx_missing_caps (pcpu->sch, cpu,SCX_CAP_BASE ))) +resched_curr (rq); +else +__scx_add_event (pcpu->sch,SCX_EV_SUB_KICK_DENIED ,1 ); + }raw_spin_rq_unlock_irqrestore (rq, flags); }
kernel/sched/ext/internal.h +8 -0
@@ -1194,6 +1194,13 @@ struct scx_event_stats {*/ s64 SCX_EV_SUB_PREEMPT_DENIED ; +/* +* The number of times a kick was skipped because the sub-sched lacked +* baseline access on the target cid. The preempt-part degradation of a +* delivered kick is counted in SCX_EV_SUB_PREEMPT_DENIED instead. +*/ +s64 SCX_EV_SUB_KICK_DENIED ; +/* * The number of times a local DSQ reenq was dropped because the * sub-sched lacked baseline access on the target cid.
@@ -1219,6 +1226,7 @@ struct scx_event_stats {SCX_EVENT (SCX_EV_SUB_BYPASS_DISPATCH ); \SCX_EVENT (SCX_EV_SUB_FORCED_ADMIT ); \SCX_EVENT (SCX_EV_SUB_PREEMPT_DENIED ); \ +SCX_EVENT (SCX_EV_SUB_KICK_DENIED ); \SCX_EVENT (SCX_EV_SUB_REENQ_DENIED) struct scx_sched;
Signature · 1 line
2.55.0
Tejun Heo
reviewerPATCH 4/5 sched_ext: Factor out scx_cpuperf_set()
In reply to Tejun HeoFactor the cpuperf target write out of scx_bpf_cpuperf_set() into scx_cpuperf_set() which takes the acting sched and returns 0 or -errno, and flatten the nested validation into early returns. No functional change. Prep for gating the write behind a cap and reporting the outcome from the cid-form kfunc.
--- kernel/sched/ext/ext.c | 78 ++++++++++++++++++++++++------------------ 1 file changed, 45 insertions(+), 33 deletions(-)
kernel/sched/ext/ext.c +45 -33
@@ -9781,6 +9781,50 @@ __bpf_kfunc u32 scx_bpf_cidperf_cur(s32 cid, const struct bpf_prog_aux *aux)return arch_scale_freq_capacity (cpu); } +/* validate and apply a cpuperf target, see scx_bpf_cpuperf_set() */ +static s32 scx_cpuperf_set (struct scx_sched *sch,s32 cpu,u32 perf) +{ +struct rq *rq, *locked_rq; +struct rq_flags rf; + +if (unlikely (perf >SCX_CPUPERF_ONE )) { +scx_error (sch,"Invalid cpuperf target %u for CPU %d" , perf, cpu); +return -EINVAL ; + } + +if (!scx_cpu_valid (sch, cpu,NULL )) +return -EINVAL ; + + rq =cpu_rq (cpu); + locked_rq =scx_locked_rq (); + +/* +* When called with an rq lock held, restrict the operation to the +* corresponding CPU to prevent ABBA deadlocks. +*/ +if (locked_rq && rq != locked_rq) { +scx_error (sch,"Invalid target CPU %d" , cpu); +return -EINVAL ; + } + +/* +* If no rq lock is held, allow to operate on any CPU by acquiring +* the corresponding rq lock. +*/ +if (!locked_rq) { +rq_lock_irqsave (rq, &rf); +update_rq_clock (rq); + } + + rq->scx.cpuperf_target = perf; +cpufreq_update_util (rq,0 ); + +if (!locked_rq) +rq_unlock_irqrestore (rq, &rf); + +return 0 ; +} +/** * scx_bpf_cpuperf_set - Set the relative performance target of a CPU * @cpu: CPU of interest
@@ -9806,39 +9850,7 @@ __bpf_kfunc void scx_bpf_cpuperf_set(s32 cpu, u32 perf, const struct bpf_prog_auif (unlikely (!sch))return ; -if (unlikely (perf >SCX_CPUPERF_ONE )) { -scx_error (sch,"Invalid cpuperf target %u for CPU %d" , perf, cpu); -return ; - } - -if (scx_cpu_valid (sch, cpu,NULL )) { -struct rq *rq =cpu_rq (cpu), *locked_rq =scx_locked_rq (); -struct rq_flags rf; - -/* -* When called with an rq lock held, restrict the operation -* to the corresponding CPU to prevent ABBA deadlocks. -*/ -if (locked_rq && rq != locked_rq) { -scx_error (sch,"Invalid target CPU %d" , cpu); -return ; - } - -/* -* If no rq lock is held, allow to operate on any CPU by -* acquiring the corresponding rq lock. -*/ -if (!locked_rq) { -rq_lock_irqsave (rq, &rf); -update_rq_clock (rq); - } - - rq->scx.cpuperf_target = perf; -cpufreq_update_util (rq,0 ); - -if (!locked_rq) -rq_unlock_irqrestore (rq, &rf); - } +scx_cpuperf_set (sch, cpu, perf); }/**
Signature · 1 line
2.55.0
Tejun Heo
reviewerPATCH 5/5 sched_ext: Gate scx_bpf_cidperf_set() behind a new SCX_CAP_PERF
In reply to Tejun Heoscx_bpf_cidperf_set() reaches cpufreq with no cap check, so any cid-form sub-sched can steer the frequency of any cid in its view, including ones it holds nothing on.
Gate it behind a new SCX_CAP_PERF rather than SCX_CAP_BASE: hardware control is a separate axis from queue access - a parent may well delegate scheduling on a cid without handing over its frequency. PERF neither implies nor is implied by the other caps. The check runs under the target rq's lock, which ecaps updates are also folded under, so it is authoritative - a write can never land after a revoke has taken effect. Denials are counted in SCX_EV_SUB_CIDPERF_DENIED.
The operation is synchronous and the outcome is reported to the caller:scx_bpf_cidperf_set() now returns 0 or -errno, -EACCES on denial. The cid-form interface is still under initial development, so the signature is changed in place without versioning.
scx_qmap grants PERF alongside its existing cid grants so the cpuperf demo keeps working in sub-scheds.
--- kernel/sched/ext/ext.c | 35 ++++++++++++++++++------ kernel/sched/ext/internal.h | 16 ++++++++++- tools/sched_ext/include/scx/common.bpf.h | 2 +- tools/sched_ext/scx_qmap.bpf.c | 17 ++++++++---- 4 files changed, 53 insertions(+), 17 deletions(-)
kernel/sched/ext/ext.c +26 -9
@@ -5144,6 +5144,7 @@ static const char *scx_cap_names[__SCX_NR_CAPS] = { [__SCX_CAP_ENQ_IMMED] = "enq_immed", [__SCX_CAP_ENQ] = "enq", [__SCX_CAP_PREEMPT] = "preempt", + [__SCX_CAP_PERF] = "perf", }; static ssize_t scx_attr_caps_show(struct kobject *kobj,
@@ -9786,6 +9787,7 @@ static s32 scx_cpuperf_set(struct scx_sched *sch, s32 cpu, u32 perf) {struct rq *rq, *locked_rq;struct rq_flags rf; +s32 ret;if (unlikely (perf >SCX_CPUPERF_ONE )) {scx_error (sch,"Invalid cpuperf target %u for CPU %d" , perf, cpu);
@@ -9816,13 +9818,24 @@ static s32 scx_cpuperf_set(struct scx_sched *sch, s32 cpu, u32 perf)update_rq_clock (rq); } - rq->scx.cpuperf_target = perf; -cpufreq_update_util (rq,0 ); +/* +* ecaps updates are folded under the rq lock, making this test +* authoritative: a write can never land after a revoke has taken +* effect on @cpu. +*/ +if (likely (!scx_missing_caps (sch, cpu,SCX_CAP_PERF ))) { + rq->scx.cpuperf_target = perf; +cpufreq_update_util (rq,0 ); + ret =0 ; + }else { +__scx_add_event (sch,SCX_EV_SUB_CIDPERF_DENIED ,1 ); + ret = -EACCES ; + }if (!locked_rq)rq_unlock_irqrestore (rq, &rf); -return 0 ; +return ret; }/**
@@ -9859,10 +9872,13 @@ __bpf_kfunc void scx_bpf_cpuperf_set(s32 cpu, u32 perf, const struct bpf_prog_au* @perf: target performance level [0, %SCX_CPUPERF_ONE] * @aux: implicit BPF argument to access bpf_prog_aux hidden from BPF progs * -* cid-addressed equivalent of scx_bpf_cpuperf_set(). +* cid-addressed equivalent of scx_bpf_cpuperf_set(). A sub-sched needs +* SCX_CAP_PERF on @cid. Returns 0 if the target was applied, -%EACCES if +* the write was denied for missing caps, other -errnos if @cid didn't +* resolve. */ -__bpf_kfunc void scx_bpf_cidperf_set(s32 cid, u32 perf, -const struct bpf_prog_aux *aux) +__bpf_kfunc s32 scx_bpf_cidperf_set(s32 cid, u32 perf, +const struct bpf_prog_aux *aux) {struct scx_sched *sch;s32 cpu;
@@ -9871,11 +9887,12 @@ __bpf_kfunc void scx_bpf_cidperf_set(s32 cid, u32 perf, sch =scx_prog_sched (aux);if (unlikely (!sch)) -return ; +return -ENODEV ; cpu =scx_cid_to_cpu (sch, cid);if (cpu <0 ) -return ; -scx_bpf_cpuperf_set (cpu, perf, aux); +return cpu; + +return scx_cpuperf_set (sch, cpu, perf); }/**
kernel/sched/ext/internal.h +15 -1
@@ -1206,6 +1206,12 @@ struct scx_event_stats {* sub-sched lacked baseline access on the target cid. */ s64 SCX_EV_SUB_REENQ_DENIED ; + +/* +* The number of times scx_bpf_cidperf_set() was denied because the +* sub-sched lacked SCX_CAP_PERF on the target cid. +*/ +s64 SCX_EV_SUB_CIDPERF_DENIED ; };#define SCX_EVENTS_LIST (SCX_EVENT ) \
@@ -1227,7 +1233,8 @@ struct scx_event_stats {SCX_EVENT (SCX_EV_SUB_FORCED_ADMIT ); \SCX_EVENT (SCX_EV_SUB_PREEMPT_DENIED ); \SCX_EVENT (SCX_EV_SUB_KICK_DENIED ); \ -SCX_EVENT (SCX_EV_SUB_REENQ_DENIED) +SCX_EVENT (SCX_EV_SUB_REENQ_DENIED ); \ +SCX_EVENT (SCX_EV_SUB_CIDPERF_DENIED) struct scx_sched;
@@ -1353,6 +1360,11 @@ struct scx_sched_pnode {* - SCX_ENQ_PREEMPT inserts * - SCX_KICK_PREEMPT kicks * +* PERF control the cid's cpu power/perf management state, currently the +* cpufreq target set through scx_bpf_cidperf_set(). Hardware +* control is a separate axis from queue access: PERF neither +* implies nor is implied by the caps above. +* * Implied caps apply to the holder's own use of a cid, not to delegation. * scx_bpf_sub_grant() delegates literally-held caps, so a cap held only through * implication is usable but cannot be re-delegated to a child. When granting a
@@ -1363,6 +1375,7 @@ enum scx_cap_flags { __SCX_CAP_ENQ_IMMED =0 , __SCX_CAP_ENQ =1 , __SCX_CAP_PREEMPT =2 , + __SCX_CAP_PERF =3 , __SCX_NR_CAPS, __SCX_CAP_ALL = BIT_U64(__SCX_NR_CAPS) - 1,
@@ -1370,6 +1383,7 @@ enum scx_cap_flags {SCX_CAP_ENQ_IMMED =BIT_U64 (__SCX_CAP_ENQ_IMMED),SCX_CAP_ENQ =BIT_U64 (__SCX_CAP_ENQ),SCX_CAP_PREEMPT =BIT_U64 (__SCX_CAP_PREEMPT), +SCX_CAP_PERF =BIT_U64 (__SCX_CAP_PERF),/* alias for minimal cap to make any use of a cpu */ SCX_CAP_BASE = SCX_CAP_ENQ_IMMED,
tools/sched_ext/include/scx/common.bpf.h +1 -1
@@ -112,7 +112,7 @@ u32 scx_bpf_nr_cids(void) __ksym __weak; u32 scx_bpf_nr_online_cids(void) __ksym __weak; u32 scx_bpf_cidperf_cap(s32 cid) __ksym __weak; u32 scx_bpf_cidperf_cur(s32 cid) __ksym __weak; -void scx_bpf_cidperf_set(s32 cid, u32 perf) __ksym __weak; +s32 scx_bpf_cidperf_set(s32 cid, u32 perf) __ksym __weak;/* sub-scheduler cap control, scx_bpf_sub_caps() cgroup_id 0 == self */ s32 scx_bpf_sub_grant(u64 cgroup_id, u64 caps, const struct scx_cmask *cmask,
tools/sched_ext/scx_qmap.bpf.c +11 -6
@@ -1585,11 +1585,13 @@ __noinline void apply_partition(void)cmask_copy (&qa.to_grant_cids.mask, &ssc->granted_cids.mask);cmask_andnot (&qa.to_grant_cids.mask, &ssc->prev_granted.mask); -scx_bpf_sub_revoke (cgid,SCX_CAP_ENQ_IMMED , +scx_bpf_sub_revoke (cgid,SCX_CAP_ENQ_IMMED |SCX_CAP_PERF , (void *)(long )&qa.prev_rr_cids.mask); -scx_bpf_sub_revoke (cgid,SCX_CAP_ENQ |SCX_CAP_PREEMPT |SCX_CAP_ENQ_IMMED , +scx_bpf_sub_revoke (cgid,SCX_CAP_ENQ |SCX_CAP_PREEMPT | +SCX_CAP_ENQ_IMMED |SCX_CAP_PERF , (void *)(long )&qa.to_revoke_cids.mask); -scx_bpf_sub_grant (cgid,SCX_CAP_ENQ |SCX_CAP_PREEMPT |SCX_CAP_ENQ_IMMED , +scx_bpf_sub_grant (cgid,SCX_CAP_ENQ |SCX_CAP_PREEMPT | +SCX_CAP_ENQ_IMMED |SCX_CAP_PERF , (void *)(long )&qa.to_grant_cids.mask,NULL ); }
@@ -1605,7 +1607,8 @@ __noinline void apply_partition(void) holder_cgid = qa.part.rr_slots[pos];/* 0 = self, nothing to grant */ if (holder_cgid) -scx_bpf_sub_grant (holder_cgid,SCX_CAP_ENQ_IMMED , +scx_bpf_sub_grant (holder_cgid, +SCX_CAP_ENQ_IMMED |SCX_CAP_PERF , (void *)(long )&qa.rr_cids.mask,NULL ); } }
@@ -1696,10 +1699,12 @@ static void rr_advance(void)* time-share. */ if (old_cgid) -scx_bpf_sub_revoke (old_cgid,SCX_CAP_ENQ_IMMED , +scx_bpf_sub_revoke (old_cgid, +SCX_CAP_ENQ_IMMED |SCX_CAP_PERF , (void *)(long )&qa.rr_cids.mask);if (new_cgid) -scx_bpf_sub_grant (new_cgid,SCX_CAP_ENQ_IMMED , +scx_bpf_sub_grant (new_cgid, +SCX_CAP_ENQ_IMMED |SCX_CAP_PERF , (void *)(long )&qa.rr_cids.mask,NULL ); }
Signature · 1 line
2.55.0
Hi Tejun,
Quoted 6 lines · Tejun Heo
Hello,
Follow-up fixes and enforcement holes plugged from the recent sub-scheduler work. The first patch fixes the tools' restart loop clobbering an exit that was requested while a restart was pending.
Quoted 9 lines
The rest tighten how cid-addressed operations from sub-schedulers are handled when the caller lacks the necessary caps. Reenq to another sched's local DSQs is now denied and counted, and a new event counts the already-denied kicks. These checks are lockless - transient false positives from cap rotation are harmless and false negatives can't happen by visibility. cpuperf control moves behind a new
SCX_CAP_PERFwhich only the root holds by default, andscx_bpf_cidperf_set()reports the result synchronously as its check is authoritative under the target's rq lock.
I haven’t found any issues in this series, everything looks good to me.
Thanks,
-Andrea
Quoted 41 lines
v2: - Kernel-doc comment moved to
scx_bpf_cpuperf_set()instead of being
left attached to the factored-out helper (0004, reported by sashiko).
- The versionedscx_bpf_cidperf_set___v2()interface and the tools compat wrapper are dropped. The cid-form interface is still under initial development, so the signature is changed in place to return s32 (0005).v1: https://lore.kernel.org/r/[email protected]
0001-tools-sched_ext-Don-t-restart-over-a-pending-exit-re.patch 0002-sched_ext-Gate-local-DSQ-reenq-on-baseline-cid-acces.patch 0003-sched_ext-Count-kicks-denied-for-lacking-baseline-ci.patch 0004-sched_ext-Factor-out-scx_cpuperf_set.patch 0005-sched_ext-Gate-scx_bpf_cidperf_set-behind-a-new-SCX_.patch
Based on sched_ext/for-7.3 (
c509d8107026).The patches are also available in the following git branch:
git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext.gitscx-sub-cap-gates-v2diffstat follows. Thanks.
kernel/sched/ext/ext.c | 139 +++++++++++++++++++++---------- kernel/sched/ext/internal.h | 31 ++++++- tools/sched_ext/include/scx/common.bpf.h | 2 +- tools/sched_ext/scx_central.c | 2 +- tools/sched_ext/scx_cpu0.c | 2 +- tools/sched_ext/scx_flatcg.c | 2 +- tools/sched_ext/scx_pair.c | 2 +- tools/sched_ext/scx_qmap.bpf.c | 17 ++-- tools/sched_ext/scx_qmap.c | 2 +- tools/sched_ext/scx_sdt.c | 2 +- tools/sched_ext/scx_simple.c | 2 +- tools/sched_ext/scx_userland.c | 11 +-- 12 files changed, 149 insertions(+), 65 deletions(-)--
tejun
Tejun Heo
reviewerQuoted 5 lines · Tejun Heo
0001-tools-sched_ext-Don-t-restart-over-a-pending-exit-re.patch 0002-sched_ext-Gate-local-DSQ-reenq-on-baseline-cid-acces.patch 0003-sched_ext-Count-kicks-denied-for-lacking-baseline-ci.patch 0004-sched_ext-Factor-out-scx_cpuperf_set.patch 0005-sched_ext-Gate-scx_bpf_cidperf_set-behind-a-new-SCX_.patch
Applied 1-5 to sched_ext/for-7.3 with Andrea's Reviewed-by added.
Thanks.
--
tejun