sched/deadline: Update total bandwidth when adding new task
Steven Rostedt
reviewerPATCH 1/1 sched/deadline: Update total bandwidth when adding new task
While debugging the crash with the bad nr_running accounting, I hit another bug where, after running my sched deadline test, I was getting failures to take a CPU offline. It was giving me a -EBUSY error.
Adding a bunch of trace_printk()s around, I found that the cpu notifier that called sched_cpu_inactive() was returning a failure. The overflow value was coming up negative?
Adding more trace_printk()s, I found that task_dead_dl() function was subtracting the exact amount that was keeping the CPU from going offline. I then realized that the task_dead_dl() was updating the total_bw for the task that was going away, but there was nothing that added to the total_bw when the task came alive. If total_bw is not zero for a CPU, it will keep that CPU from going offline.
---
kernel/sched/core.c +2 -0
@@ -3178,6 +3178,7 @@ static void __setparam_dl(struct task_struct *p, const struct sched_attr *attr) { struct sched_dl_entity *dl_se = &p->dl; +struct dl_bw *dl_b =dl_bw_of (task_cpu (p));init_dl_task_timer (dl_se); dl_se->dl_runtime = attr->sched_runtime;
@@ -3187,6 +3188,7 @@ __setparam_dl(struct task_struct *p, const struct sched_attr *attr) dl_se->dl_bw =to_ratio (dl_se->dl_period, dl_se->dl_runtime); dl_se->dl_throttled =0 ; dl_se->dl_new =1 ; +__dl_add (dl_b, dl_se->dl_bw); }/* Actually do priority change: must hold pi & rq lock. */
Quoted 8 lines · Steven Rostedt
While debugging the crash with the bad
nr_runningaccounting, I hit another bug where, after running my sched deadline test, I was getting failures to take a CPU offline. It was giving me a-EBUSYerror.Adding a bunch of trace_printk()s around, I found that the cpu notifier that called
sched_cpu_inactive()was returning a failure. The overflow value was coming up negative?
Quoted 5 lines
Adding more trace_printk()s, I found that
task_dead_dl()function was subtracting the exact amount that was keeping the CPU from going offline. I then realized that thetask_dead_dl()was updating thetotal_bwfor the task that was going away, but there was nothing that added to thetotal_bwwhen the task came alive.
We call __dl_add() from dl_overflow(), right before calling __setscheduler().
Quoted 18 lines
If
total_bwis not
zero for a CPU, it will keep that CPU from going offline.Signed-off-by Steven Rostedt <[email protected]>---
kernel/sched/core.c +1
-0@@ -3178,6 +3178,7 @@ static void __setparam_dl(struct task_struct *p, const struct sched_attr *attr) { struct sched_dl_entity *dl_se = &p->dl; +struct dl_bw *dl_b =dl_bw_of (task_cpu (p));init_dl_task_timer (dl_se); dl_se->dl_runtime = attr->sched_runtime;
kernel/sched/core.c __setparam_dl(struct task_struct *p, const struct sched_attr *attr) · line 3192
Quoted 5 lines
kernel/sched/core.c +1
-0@@ -3187,6 +3188,7 @@ __setparam_dl(struct task_struct *p, const struct sched_attr *attr) dl_se->dl_bw =to_ratio (dl_se->dl_period, dl_se->dl_runtime); dl_se->dl_throttled =0 ; dl_se->dl_new =1 ; +__dl_add (dl_b, dl_se->dl_bw); }
This is intended to set params for a task that is going to become -deadline. I don't think it is the right place to update dl_b. And it seems you add it twice (inside dl_overflow and inside __setscheduler).
I performed some tests, and it seems that dl_b is updated correctly, once when the task is started and once when task_dead_dl() is called.
That said, I'm still testing with your stress-cpu-hotplug.
Thanks,
- Juri
Quoted 3 lines
kernel/sched/core.c +0
-0/* Actually do priority change: must hold pi & rq lock. */
Steven Rostedt
reviewerPATCH 1/1 sched/deadline: Fix overflow to handle period==0 and deadline!=0
In reply to Steven RostedtWhile debugging the crash with the bad nr_running accounting, I hit another bug where, after running my sched deadline test, I was getting failures to take a CPU offline. It was giving me a -EBUSY error.
Adding a bunch of trace_printk()s around, I found that the cpu notifier that called sched_cpu_inactive() was returning a failure. The overflow value was coming up negative?
Talking this over with Juri, the problem is that the total_bw update was suppose to be made by dl_overflow() which, during my tests, seemed to not be called. Adding more trace_printk()s, it wasn't that it wasn't called, but it exited out right away with the check of new_bw being equal to p->dl.dl_bw. The new_bw calculates the ratio between period and runtime. The bug is that if you set a deadline, you do not need to set a period if you plan on the period being equal to the deadline. That is, if period is zero and deadline is not, then the system call should set the period to be equal to the deadline. This is done elsewhere in the code.
The fix is easy, check if period is set, and if it is not, then use the deadline.
---
kernel/sched/core.c +1 -1
@@ -1952,7 +1952,7 @@ static int dl_overflow(struct task_struct *p, int policy, {struct dl_bw *dl_b =dl_bw_of (task_cpu (p)); -u64 period = attr->sched_period; +u64 period = attr->sched_period ?: attr->sched_deadline;u64 runtime = attr->sched_runtime;u64 new_bw =dl_policy (policy) ?to_ratio (period, runtime) :0 ;int cpus, err =-1 ;
Re: [PATCH] sched/deadline: Fix overflow to handle period==0 and deadline!=0
Quoted 30 lines · Steven Rostedt
While debugging the crash with the bad
nr_runningaccounting, I hit another bug where, after running my sched deadline test, I was getting failures to take a CPU offline. It was giving me a-EBUSYerror.Adding a bunch of trace_printk()s around, I found that the cpu notifier that called
sched_cpu_inactive()was returning a failure. The overflow value was coming up negative?Talking this over with Juri, the problem is that the
total_bwupdate was suppose to be made bydl_overflow()which, during my tests, seemed to not be called. Adding more trace_printk()s, it wasn't that it wasn't called, but it exited out right away with the check ofnew_bwbeing equal top->dl.dl_bw. Thenew_bwcalculates the ratio between period and runtime. The bug is that if you set a deadline, you do not need to set a period if you plan on the period being equal to the deadline. That is, if period is zero and deadline is not, then the system call should set the period to be equal to the deadline. This is done elsewhere in the code.The fix is easy, check if period is set, and if it is not, then use the deadline.
Signed-off-by Steven Rostedt <[email protected]>---
kernel/sched/core.c +0
-0
kernel/sched/core.c static int dl_overflow(struct task_struct *p, int policy, · line 1958
Quoted 8 lines
kernel/sched/core.c +1
-1@@ -1952,7 +1952,7 @@ static int dl_overflow(struct task_struct *p, int policy, {struct dl_bw *dl_b =dl_bw_of (task_cpu (p)); -u64 period = attr->sched_period; +u64 period = attr->sched_period ?: attr->sched_deadline;u64 runtime = attr->sched_runtime;u64 new_bw =dl_policy (policy) ?to_ratio (period, runtime) :0 ;int cpus, err =-1 ;
Thanks!
- Juri
tip-bot for Steven Rostedt
merge bot[tip:sched/urgent] sched/deadline: Fix overflow to handle period= =0 and deadline!=0
In reply to Steven Rostedtsched/deadline: Fix overflow to handle period==0 and deadline!=0
Commit-ID: 4df1638cfaf9b2b7ad993979a41965acab9cd156 Gitweb: http://git.kernel.org/tip/4df1638cfaf9b2b7ad993979a41965acab9cd156 Author: Steven Rostedt <[email protected]> AuthorDate: Wed, 19 Feb 2014 13:53:35 -0500 Committer: Thomas Gleixner <[email protected]> CommitDate: Fri, 21 Feb 2014 21:27:09 +0100
sched/deadline: Fix overflow to handle period==0 and deadline!=0
While debugging the crash with the bad nr_running accounting, I hit another bug where, after running my sched deadline test, I was getting failures to take a CPU offline. It was giving me a -EBUSY error.
Adding a bunch of trace_printk()s around, I found that the cpu notifier that called sched_cpu_inactive() was returning a failure. The overflow value was coming up negative?
Talking this over with Juri, the problem is that the total_bw update was suppose to be made by dl_overflow() which, during my tests, seemed to not be called. Adding more trace_printk()s, it wasn't that it wasn't called, but it exited out right away with the check of new_bw being equal to p->dl.dl_bw. The new_bw calculates the ratio between period and runtime. The bug is that if you set a deadline, you do not need to set a period if you plan on the period being equal to the deadline. That is, if period is zero and deadline is not, then the system call should set the period to be equal to the deadline. This is done elsewhere in the code.
The fix is easy, check if period is set, and if it is not, then use the deadline.
--- kernel/sched/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched/core.c +1 -1
@@ -1952,7 +1952,7 @@ static int dl_overflow(struct task_struct *p, int policy, {struct dl_bw *dl_b =dl_bw_of (task_cpu (p)); -u64 period = attr->sched_period; +u64 period = attr->sched_period ?: attr->sched_deadline;u64 runtime = attr->sched_runtime;u64 new_bw =dl_policy (policy) ?to_ratio (period, runtime) :0 ;int cpus, err =-1 ;