sched_ext: Auto-manage ext/fair dl_server bandwidth
Subsystems (3)
Andrea Righi
reviewerPATCH v2 1/1 sched_ext: Auto-manage ext/fair dl_server bandwidth
Currently, a fixed bandwidth is reserved at boot for both the fair and ext deadline servers, and this reservation remains unchanged unless explicitly modified via debugfs. As a result, both servers permanently contribute to global bandwidth accounting, regardless of whether a BPF scheduler is active.
While unused bandwidth can still be reclaimed at runtime by other classes, this static reservation prevents RT from fully utilizing available headroom in situations where one of the sched_ext or fair class is guaranteed to be inactive (for example, when no BPF scheduler is loaded, or when sched_ext runs in full mode and replaces fair).
As discussed at the VIII OSPM summit in Cambridge [1], a better solution would be to dynamically register and unregister deadline server bandwidth based on the active sched_ext state. This allows the kernel to automatically enable bandwidth accounting only for the scheduling class that is currently active, while disabling it for inactive ones.
This patch series implements this automatic register/unregister logic. The sched_ext total_bw kselftest is also modified to validate the correct behavior across the different scheduling configurations and ensure that bandwidth accounting follows the expected state transitions.
[1] https://retis.santannapisa.it/ospm-summit/
Git tree: git://git.kernel.org/pub/scm/linux/kernel/git/arighi/linux.git dl-server-bw-v2
Changes in v2:
- Rework the sched_ext enable path as suggested by Peter: attach ext_server before committing the scheduler switch and fail the enable if admission control rejects the reservation; detach fair_server only after a successful full-mode switch.
- Added dl_server_swap_bw() for the disable/recovery path so ext_server detach and fair_server reattach happen under the same dl_b->lock, closing the window where concurrent SCHED_DEADLINE admission could steal the freed bandwidth (reported by Sashiko).
- Fixed the attach/detach accounting issue reported by Sashiko by updating rq->dl.this_bw together with root-domain total_bw, draining active or non-contending servers before detach and preventing detached servers from starting.
- Reuse dl_rq_change_utilization() to drain the server, so the detach path goes through the same machinery as dl_server_apply_params()
- Made root-domain accounting honor the same cpu_active() conditions used by root-domain rebuilds, while preserving runtime/period updates made while a server is detached.
- Fixed the total_bw selftest issues reported by Sashiko: check fclose() errors for debugfs writes, preserve per-CPU fair_server runtime values, and restore all CPUs on cleanup even if one write fails.
- Link to v1: https://lore.kernel.org/all/[email protected]/
Andrea Righi (2):
sched_ext: Auto-register/unregister dl_server reservations selftests/sched_ext: Validate dl_server attach/detach in total_bw test
include/linux/sched.h | 6 + kernel/sched/deadline.c | 207 +++++++++++++++++++++++++-- kernel/sched/ext.c | 71 +++++++++ kernel/sched/sched.h | 4 + tools/testing/selftests/sched_ext/total_bw.c | 201 +++++++++++++++++++++++++- 5 files changed, 480 insertions(+), 9 deletions(-)
Andrea Righi
reviewerPATCH 1/2 sched_ext: Auto-register/unregister dl_server reservations
Commit cd959a3562050d ("sched_ext: Add a DL server for sched_ext tasks") introduced an ext_server deadline server to protect sched_ext tasks from fair/RT starvation, mirroring the existing fair_server.
Currently, both servers reserve their 50ms/1000ms bandwidth at boot, regardless of whether a BPF scheduler is loaded. Unused bandwidth is still reclaimed at runtime by other classes, but the static reservation prevents the RT class from implicitly using that headroom when one of the two classes is guaranteed to be empty.
A sysadmin can work around this by writing /sys/kernel/debug/sched/{fair,ext}_server/cpu*/runtime, but that requires manual action and not all systems expose debugfs.
A better approach is to make server bandwidth reservations dynamic: only the scheduling policy that is currently active should register its reservation, while the inactive one should not artificially hold capacity (keeping both reservations only when the BPF scheduler is running in partial mode):
+---------------------------------------------+-------------+------------+ | BPF scheduler state | fair server | ext server | +---------------------------------------------+-------------+------------+ | not loaded (default boot) | reserved | none | | loaded full mode (!SCX_OPS_SWITCH_PARTIAL) | none | reserved | | loaded partial mode (SCX_OPS_SWITCH_PARTIAL)| reserved | reserved | +---------------------------------------------+-------------+------------+
To achieve this, introduce an "attached/detached" state for each deadline server, so the kernel can decide whether a server's bandwidth should be accounted in global bandwidth tracking.
At boot, the system starts with only the fair server contributing to bandwidth accounting. When a BPF scheduler is enabled, the ext server is attached and may replace or complement the fair server depending on whether full or partial mode is used. When sched_ext is disabled, the system restores the previous deadline bandwidth values and behavior.
The transition logic ensures that switching between scheduling modes is consistent and reversible, without losing runtime configuration or requiring manual intervention.
--- include/linux/sched.h | 6 ++ kernel/sched/deadline.c | 207 ++++++++++++++++++++++++++++++++++++++-- kernel/sched/ext.c | 71 ++++++++++++++ kernel/sched/sched.h | 4 + 4 files changed, 280 insertions(+), 8 deletions(-)
include/linux/sched.h +6 -0
@@ -702,6 +702,11 @@ struct sched_dl_entity {* running, skipping the defer phase. * * @dl_defer_idle tracks idle state +* +* @dl_bw_attached tells if this server's bandwidth currently +* contributes to the root domain's total_bw. Only meaningful for server +* entities (@dl_server == 1). Allows toggling the reservation on/off +* without losing the configured @dl_runtime/@dl_period. */ unsigned int dl_throttled : 1; unsigned int dl_yielded : 1;
@@ -713,6 +718,7 @@ struct sched_dl_entity { unsigned int dl_defer_armed : 1; unsigned int dl_defer_running : 1; unsigned int dl_defer_idle : 1; + unsigned int dl_bw_attached : 1;/* * Bandwidth enforcement timer. Each -deadline task has its
kernel/sched/deadline.c +199 -8
260 lines not on this page · raw message
kernel/sched/ext.c +71 -0
105 lines not on this page · raw message
kernel/sched/sched.h +4 -0
@@ -421,6 +421,10 @@ extern void ext_server_init(struct rq *rq);extern void __dl_server_attach_root (struct sched_dl_entity *dl_se,struct rq *rq);extern int dl_server_apply_params (struct sched_dl_entity *dl_se,u64 runtime,u64 period,bool init); +extern int dl_server_attach_bw (struct sched_dl_entity *dl_se); +extern void dl_server_detach_bw (struct sched_dl_entity *dl_se); +extern int dl_server_swap_bw (struct sched_dl_entity *detach_se, +struct sched_dl_entity *attach_se);static inline bool dl_server_active (struct sched_dl_entity *dl_se) {
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 1/2] sched_ext: Auto-register/unregister dl_server reservations
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
New issues:
- [High] Bypassing the overflow check for detached deadline servers allows configuring an arbitrarily large bandwidth for offline CPUs, breaking root domain admission control upon hotplug.
Pre-existing issues:
- [High] Modifying a deadline server's bandwidth via debugfs for an offline CPU causes division-by-zero crashes. --
commit e4e3b446a7561269980e46cfd4aec61572b0cae1
Author: Andrea Righi <[email protected]>
sched_ext: Auto-register/unregister dl_server reservations
This commit makes deadline server bandwidth reservations dynamic by introducing an attached/detached state. It ensures that only the currently active scheduling policy registers its bandwidth reservation, allowing unused bandwidth to be reclaimed without losing the configured runtime and period.
Quoted 4 lines
kernel/sched/deadline.c +0
-0
[ ... ]
kernel/sched/deadline.c int dl_server_apply_params(struct sched_dl_entity *dl_se, u64 runtime, u64 perio · line 1924
Quoted 10 lines
kernel/sched/deadline.c +4
-2@@ -1907,13 +1918,16 @@ int dl_server_apply_params(struct sched_dl_entity *dl_se, u64 runtime, u64 perio cpus =dl_bw_cpus (cpu); cap =dl_bw_capacity (cpu); -if (__dl_overflow (dl_b, cap, old_bw, new_bw)) -return -EBUSY ; +if (init || dl_se->dl_bw_attached) { +if (__dl_overflow (dl_b, cap, old_bw, new_bw)) +return -EBUSY ; + }
[Severity: High]
Does bypassing the overflow check for detached servers allow configuring an invalid bandwidth?
If userspace configures a massive bandwidth for an offline CPU while the server is detached, the overflow check is skipped. Later, when the BPF scheduler is loaded, scx_root_enable_workfn() calls dl_server_attach_bw() for all possible CPUs. For the offline CPU, __dl_server_attach_bw_locked() skips its own overflow check since the CPU is not active, but it still sets dl_bw_attached = 1.
When that CPU is later brought online, the hotplug rebuild path calls dl_server_add_bw(), which unconditionally adds the unvalidated bandwidth to the root domain's total_bw since dl_bw_attached is now true. Could this force the root domain into a permanently overcommitted state and break SCHED_DEADLINE admission control?
kernel/sched/deadline.c int dl_server_apply_params(struct sched_dl_entity *dl_se, u64 runtime, u64 perio · line 1928
Quoted 9 lines
kernel/sched/deadline.c +2
-1if (init) {__add_rq_bw (new_bw, &rq->dl);__dl_add (dl_b, new_bw, cpus); - }else { + dl_se->dl_bw_attached =1 ; + }else if (dl_se->dl_bw_attached) {__dl_sub (dl_b, dl_se->dl_bw, cpus);__dl_add (dl_b, new_bw, cpus);
[Severity: High]
This isn't a bug introduced by this patch, but can modifying an attached server's bandwidth on an offline CPU cause a division by zero?
When a CPU is offline, it belongs to a root domain with zero active CPUs, so dl_bw_cpus() returns 0. If a user modifies the runtime/period via debugfs for an offline CPU with an attached server, we reach this update path.
Both __dl_sub() and __dl_add() divide by cpus internally:
kernel/sched/deadline.c:__dl_sub() {
...
__dl_update(dl_b, (s32)tsk_bw / cpus);
}Since cpus is 0, will this trigger a kernel panic and allow a local denial of service?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Andrea Righi
reviewerPATCH 2/2 selftests/sched_ext: Validate dl_server attach/detach in total_bw test
In reply to Andrea RighiExtend the total_bw selftest to validate the fair/ext dl_server auto-attach/detach operations.
After the existing consistency checks, the test now doubles the fair_server's runtime on every CPU via debugfs and verifies that:
1. total_bw grew after the customization (proves fair_server was attached and apply_params() honored the dl_bw_attached flag),
2. with the minimal BPF scheduler loaded, total_bw drops back to the baseline value (proves fair_server was detached and ext_server was attached at its own default runtime),
3. after unload total_bw matches the doubled value from step 1 (proves fair_server was re-attached with the runtime customization preserved across the load/unload cycle).
--- tools/testing/selftests/sched_ext/total_bw.c | 201 ++++++++++++++++++- 1 file changed, 200 insertions(+), 1 deletion(-)
tools/testing/selftests/sched_ext/total_bw.c +200 -1
@@ -100,6 +100,98 @@ static int read_total_bw_values(long *bw_values, int max_cpus)return cpu_count; } +/* +* Read a per-CPU dl_server param (runtime or period) from debugfs. +* Returns the value in nanoseconds, or -1 on failure. +*/ +static long read_server_param (const char *server,const char *param,int cpu) +{ +char path[128 ]; +long value =-1 ; +FILE *fp; + +snprintf (path,sizeof (path), +"/sys/kernel/debug/sched/%s_server/cpu%d/%s" , + server, cpu, param); + fp =fopen (path,"r" ); +if (!fp) +return -1 ; +if (fscanf (fp,"%ld" , &value) !=1 ) + value =-1 ; +fclose (fp); + +return value; +} + +/* +* Write a per-CPU dl_server param to debugfs. Returns 0 on success. +*/ +static int write_server_param (const char *server,const char *param, +int cpu,long value) +{ +char path[128 ]; +FILE *fp; +int ret =0 ; + +snprintf (path,sizeof (path), +"/sys/kernel/debug/sched/%s_server/cpu%d/%s" , + server, cpu, param); + fp =fopen (path,"w" ); +if (!fp) +return -1 ; +if (fprintf (fp,"%ld" , value) <0 ) + ret =-1 ; +if (fclose (fp) !=0 ) + ret =-1 ; + +return ret; +} + +static int read_fair_runtime_all (int nr_cpus,long *runtimes) +{ +int i; + +for (i =0 ; i < nr_cpus; i++) { + runtimes[i] =read_server_param ("fair" ,"runtime" , i); +if (runtimes[i] <=0 ) +return -1 ; + } + +return 0 ; +} + +static int write_fair_runtime_all (int nr_cpus,long value) +{ +int i; + +for (i =0 ; i < nr_cpus; i++) { +if (write_server_param ("fair" ,"runtime" , i, value) <0 ) { +SCX_ERR ("Failed to write fair_server runtime on CPU %d" , i); +return -1 ; + } + } + +return 0 ; +} + +/* +* Restore per-CPU fair_server runtimes. +*/ +static int restore_fair_runtime_all (int nr_cpus,const long *runtimes) +{ +int ret =0 ; +int i; + +for (i =0 ; i < nr_cpus; i++) { +if (write_server_param ("fair" ,"runtime" , i, runtimes[i]) <0 ) { +SCX_ERR ("Failed to restore fair_server runtime on CPU %d" , i); + ret =-1 ; + } + } + +return ret; +} +static bool verify_total_bw_consistency (long *bw_values,int count) {int i;
@@ -217,6 +309,9 @@ static enum scx_test_status run(void *ctx)struct bpf_link *link;long loaded_bw[MAX_CPUS ];long unloaded_bw[MAX_CPUS ]; +long doubled_bw[MAX_CPUS ]; +long original_runtime[MAX_CPUS ], doubled_runtime; +enum scx_test_status ret;int i;/* Test scenario 2: BPF program loaded */
@@ -257,7 +352,111 @@ static enum scx_test_status run(void *ctx) }fprintf (stderr,"All total_bw values are consistent across all scenarios\n" ); -return SCX_TEST_PASS ; + +/* +* Validate auto-register/unregister of dl_server bandwidth reservations. +* +* Doubling fair_server's runtime doubles its bw contribution. With a +* full-mode BPF scheduler (minimal_ops), the kernel should detach +* fair_server and attach ext_server, dropping total_bw back to its +* pre-customization (default ext_server-only) value. On unload, the +* fair_server reservation should come back with its customized runtime +* preserved, so total_bw doubles again. +*/ +if (read_fair_runtime_all (test_ctx->nr_cpus, original_runtime) <0 ) { +fprintf (stderr,"Skipping attach/detach validation: debugfs not accessible\n" ); +return SCX_TEST_PASS ; + } + doubled_runtime = original_runtime[0 ] *2 ; + +fprintf (stderr, +"Setting fair_server runtime to %ld ns on all CPUs (orig %ld)\n" , + doubled_runtime, original_runtime[0 ]); + +if (write_fair_runtime_all (test_ctx->nr_cpus, doubled_runtime) <0 ) { + ret =SCX_TEST_FAIL ; + goto restore; + } + +if (fetch_verify_total_bw (doubled_bw, test_ctx->nr_cpus) <0 ) { +SCX_ERR ("Failed to get stable values after doubling fair runtime" ); + ret =SCX_TEST_FAIL ; + goto restore; + } + +/* +* After doubling the runtime, fair_server's bw contribution must grow. +* We don't assert exactly 2x, because the kernel's to_ratio() truncates +* the value, so 2 * to_ratio(period, runtime) and +* to_ratio(period, 2 * runtime) can differ. +*/ +for (i =0 ; i < test_ctx->nr_cpus; i++) { +if (doubled_bw[i] <= test_ctx->baseline_bw[i]) { +SCX_ERR ("CPU%d: fair did not increase total_bw (baseline=%ld, doubled=%ld)" , + i, test_ctx->baseline_bw[i], doubled_bw[i]); + ret =SCX_TEST_FAIL ; + goto restore; + } + } + + link =bpf_map__attach_struct_ops (test_ctx->skel->maps.minimal_ops); +if (!link) { +SCX_ERR ("Failed to attach scheduler for detach test" ); + ret =SCX_TEST_FAIL ; + goto restore; + } + +if (fetch_verify_total_bw (loaded_bw, test_ctx->nr_cpus) <0 ) { +SCX_ERR ("Failed to get stable values with BPF loaded (detach test)" ); +bpf_link__destroy (link); + ret =SCX_TEST_FAIL ; + goto restore; + } + +/* +* In full mode the customized fair_server is detached and ext_server is +* attached at its default runtime, total_bw must match baseline. +*/ +for (i =0 ; i < test_ctx->nr_cpus; i++) { +if (loaded_bw[i] != test_ctx->baseline_bw[i]) { +SCX_ERR ("CPU%d: expected bw %ld (fair detached, ext default), got %ld" , + i, test_ctx->baseline_bw[i], loaded_bw[i]); +bpf_link__destroy (link); + ret =SCX_TEST_FAIL ; + goto restore; + } + } + +bpf_link__destroy (link); + +if (fetch_verify_total_bw (unloaded_bw, test_ctx->nr_cpus) <0 ) { +SCX_ERR ("Failed to get stable values after BPF unload (detach test)" ); + ret =SCX_TEST_FAIL ; + goto restore; + } + +/* +* After unload, fair_server is re-attached with its preserved 2x +* runtime, so total_bw should return to the doubled value. +*/ +for (i =0 ; i < test_ctx->nr_cpus; i++) { +if (unloaded_bw[i] != doubled_bw[i]) { +SCX_ERR ("CPU%d: BPF unloaded: expected %ld (fair restored at 2x), got %ld" , + i, doubled_bw[i], unloaded_bw[i]); + ret =SCX_TEST_FAIL ; + goto restore; + } + } + +fprintf (stderr, +"dl_server attach/detach with customized fair runtime verified\n" ); + ret =SCX_TEST_PASS ; + +restore: +if (restore_fair_runtime_all (test_ctx->nr_cpus, original_runtime) <0 ) +SCX_ERR ("Failed to fully restore per-CPU fair_server runtimes" ); + +return ret; }static void cleanup (void *ctx)
Signature · 1 line
2.54.0