sched/deadline: Fix DL server initialization for initially offline CPUs
Subsystems (2)
PATCH 1/1 sched/deadline: Fix DL server initialization for initially offline CPUs
Commit 9f239df55546 ("sched/deadline: Initialize dl_servers after SMP") moved DL server initialization to sched_init_smp(), assuming that all CPUs are online after SMP initialization.
That assumption does not hold when CPUs are intentionally left offline at boot, for example with maxcpus=. sched_init_dl_servers() only walks online CPUs, so the fair and sched_ext DL servers of an initially offline CPU never get their default DL parameters configured. Bringing the CPU online later does not run this initialization again, leaving the server parameters at zero.
For example, boot a two-vCPU QEMU guest with maxcpus=1 and then bring CPU1 online:
# echo 1 > /sys/devices/system/cpu/cpu1/online # cat /sys/kernel/debug/sched/fair_server/cpu1/runtime 0 # cat /sys/kernel/debug/sched/fair_server/cpu1/period 0
With a zero runtime the fair server cannot be activated, so fair tasks on the late-online CPU can be starved when higher-priority tasks monopolize the CPU, potentially causing user-space workloads to stall.
Initialize DL servers for all possible CPUs instead. For CPUs that are already online, preserve the existing initialization behavior. For an initially offline CPU, initialize the server parameters and keep its bandwidth reservation local to the runqueue, but do not account it to the root domain until the CPU becomes active. A subsequent root-domain rebuild then publishes the reservation.
Avoid updating the runqueue clock and setting up the current CBS period for offline CPUs. The configured runtime and period are initialized at boot, while the dynamic runtime and absolute deadline are initialized through the normal CBS wakeup path when the server is first started.
After the fix, the same test reports:
# cat /sys/kernel/debug/sched/fair_server/cpu1/runtime 50000000 # cat /sys/kernel/debug/sched/fair_server/cpu1/period 1000000000
--- kernel/sched/deadline.c | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-)
kernel/sched/deadline.c +16 -8
@@ -1845,14 +1845,16 @@ void sched_init_dl_servers(void)struct rq *rq;struct sched_dl_entity *dl_se; -for_each_online_cpu (cpu) { +for_each_possible_cpu (cpu) {u64 runtime =50 *NSEC_PER_MSEC ;u64 period =1000 *NSEC_PER_MSEC ; +bool online =cpu_online (cpu); rq =cpu_rq (cpu);guard (rq_lock_irq)(rq); -update_rq_clock (rq); +if (online) +update_rq_clock (rq); dl_se = &rq->fair_server;
@@ -1862,7 +1864,8 @@ void sched_init_dl_servers(void) dl_se->dl_server =1 ; dl_se->dl_defer =1 ; -setup_new_dl_entity (dl_se); +if (online) +setup_new_dl_entity (dl_se);#ifdef CONFIG_SCHED_CLASS_EXT dl_se = &rq->ext_server;
@@ -1873,7 +1876,8 @@ void sched_init_dl_servers(void) dl_se->dl_server =1 ; dl_se->dl_defer =1 ; -setup_new_dl_entity (dl_se); +if (online) +setup_new_dl_entity (dl_se);/* * No BPF scheduler is loaded at boot, so the ext_server has no
@@ -1918,14 +1922,18 @@ int dl_server_apply_params(struct sched_dl_entity *dl_se, u64 runtime, u64 perioguard (raw_spinlock)(&dl_b->lock); cpus =dl_bw_cpus (cpu); - cap =dl_bw_capacity (cpu); -if (__dl_overflow (dl_b, cap, old_bw, new_bw)) -return -EBUSY ; +if (!init ||cpu_active (cpu)) { + cap =dl_bw_capacity (cpu); + +if (__dl_overflow (dl_b, cap, old_bw, new_bw)) +return -EBUSY ; + }if (init) {__add_rq_bw (new_bw, &rq->dl); -__dl_add (dl_b, new_bw, cpus); +if (cpu_active (cpu)) +__dl_add (dl_b, new_bw, cpus); dl_se->dl_bw_attached =1 ; }else if (dl_se->dl_bw_attached) {__dl_sub (dl_b, dl_se->dl_bw, cpus);
Signature · 1 line
2.54.0
Hi Hui Su,
Quoted 43 lines · Hui Su
Commit
9f239df55546("sched/deadline: Initializedl_serversafter SMP") moved DL server initialization tosched_init_smp(), assuming that all CPUs are online after SMP initialization.That assumption does not hold when CPUs are intentionally left offline at boot, for example with maxcpus=.
sched_init_dl_servers()only walks online CPUs, so the fair andsched_extDL servers of an initially offline CPU never get their default DL parameters configured. Bringing the CPU online later does not run this initialization again, leaving the server parameters at zero.For example, boot a two-vCPU QEMU guest with maxcpus=1 and then bring CPU1 online:
# echo 1 > /sys/devices/system/cpu/cpu1/online # cat /sys/kernel/debug/sched/fair_server/cpu1/runtime 0 # cat /sys/kernel/debug/sched/fair_server/cpu1/period 0With a zero runtime the fair server cannot be activated, so fair tasks on the late-online CPU can be starved when higher-priority tasks monopolize the CPU, potentially causing user-space workloads to stall.
Initialize DL servers for all possible CPUs instead. For CPUs that are already online, preserve the existing initialization behavior. For an initially offline CPU, initialize the server parameters and keep its bandwidth reservation local to the runqueue, but do not account it to the root domain until the CPU becomes active. A subsequent root-domain rebuild then publishes the reservation.
Avoid updating the runqueue clock and setting up the current CBS period for offline CPUs. The configured runtime and period are initialized at boot, while the dynamic runtime and absolute deadline are initialized through the normal CBS wakeup path when the server is first started.
After the fix, the same test reports:
# cat /sys/kernel/debug/sched/fair_server/cpu1/runtime 50000000 # cat /sys/kernel/debug/sched/fair_server/cpu1/period 1000000000
Quoted 4 lines
Fixes 9f239df55546 ("sched/deadline: Initialize dl_servers after SMP")Signed-off-by Hui Su <[email protected]>---
Thanks for the fix, makes sense to me.
BTW, while reviewing, sashiko noted a related pre-existing issue in the init=false path of dl_server_apply_params(): __dl_sub() and __dl_add() are called unconditionally with cpus that could be 0 during CPU hot-unplug, when the CPU is no longer active but still online. sched_server_write_common() only checks cpu_online(), so a debugfs write during that window could hit a divide-by-zero.
Could you take a look and consider posting an additional patch to address it?
Thanks,
Juri
Quoted 62 lines
Hi Hui Su,
Quoted 43 lines · Hui Su
Commit
9f239df55546("sched/deadline: Initializedl_serversafter SMP") moved DL server initialization tosched_init_smp(), assuming that all CPUs are online after SMP initialization.That assumption does not hold when CPUs are intentionally left offline at boot, for example with maxcpus=.
sched_init_dl_servers()only walks online CPUs, so the fair andsched_extDL servers of an initially offline CPU never get their default DL parameters configured. Bringing the CPU online later does not run this initialization again, leaving the server parameters at zero.For example, boot a two-vCPU QEMU guest with maxcpus=1 and then bring CPU1 online:
# echo 1 > /sys/devices/system/cpu/cpu1/online # cat /sys/kernel/debug/sched/fair_server/cpu1/runtime 0 # cat /sys/kernel/debug/sched/fair_server/cpu1/period 0With a zero runtime the fair server cannot be activated, so fair tasks on the late-online CPU can be starved when higher-priority tasks monopolize the CPU, potentially causing user-space workloads to stall.
Initialize DL servers for all possible CPUs instead. For CPUs that are already online, preserve the existing initialization behavior. For an initially offline CPU, initialize the server parameters and keep its bandwidth reservation local to the runqueue, but do not account it to the root domain until the CPU becomes active. A subsequent root-domain rebuild then publishes the reservation.
Avoid updating the runqueue clock and setting up the current CBS period for offline CPUs. The configured runtime and period are initialized at boot, while the dynamic runtime and absolute deadline are initialized through the normal CBS wakeup path when the server is first started.
After the fix, the same test reports:
# cat /sys/kernel/debug/sched/fair_server/cpu1/runtime 50000000 # cat /sys/kernel/debug/sched/fair_server/cpu1/period 1000000000Quoted 4 lines
Fixes 9f239df55546 ("sched/deadline: Initialize dl_servers after SMP")Signed-off-by Hui Su <[email protected]>---
Thanks for the fix, makes sense to me.
Acked-by Juri Lelli <[email protected]>BTW, while reviewing, sashiko noted a related pre-existing issue in the init=false path of
dl_server_apply_params():__dl_sub()and__dl_add()are called unconditionally with cpus that could be 0 during CPU hot-unplug, when the CPU is no longer active but still online.sched_server_write_common()only checkscpu_online(), so a debugfs write during that window could hit a divide-by-zero.
Quoted 2 lines
Could you take a look and consider posting an additional patch to address it?
---
Thanks for pointing this out. I will take a look at the issue and post a separate patch to fix it.
Thanks.
Hi Peter, Ingo,
Gentle ping on this fix. Juri has Acked it.
Quoted 3 lines
Thanks for the fix, makes sense to me.
Acked-by Juri Lelli <[email protected]>
Please let me know if any additional testing or changes are needed.
Thanks,
Hui