sched_ext: Reverting @p->sched_class if @p->disallow is set
Subsystems (2)
PATCH 1/1 sched_ext: Reverting @p->sched_class if @p->disallow is set
From: Zhang Qiao <[email protected]>
when ops.init_task() sets @p->disallow, @p->policy was reverted to @SCHED_NORMAL, but @p->sched_class was not, so reverting p->sched_class to fair_sched_classs now.
--- kernel/sched/ext.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
kernel/sched/ext.c +3 -2
@@ -3220,13 +3220,14 @@ static int scx_ops_init_task(struct task_struct *p, struct task_group *tg, bool/* * We're either in fork or load path and @p->policy will be -* applied right after. Reverting @p->policy here and rejecting -* %SCHED_EXT transitions from scx_check_setscheduler() +* applied right after. Reverting @p->policy and @p->sched_class +* here and rejecting %SCHED_EXT transitions from scx_check_setscheduler() * guarantees that if ops.init_task() sets @p->disallow, @p can * never be in SCX. */ if (p->policy ==SCHED_EXT ) { p->policy =SCHED_NORMAL ; + p->sched_class = &fair_sched_class;atomic_long_inc (&scx_nr_rejected); }
Signature · 1 line
2.45.2.windows.1
Tejun Heo
reviewerHello,
On Thu, Jul 11, 2024 at 07:07:20PM +0800, Zhangqiao (2012 lab) wrote:
...
kernel/sched/ext.c static int scx_ops_init_task(struct task_struct *p, struct task_group *tg, bool · line 3231
Quoted 4 lines
kernel/sched/ext.c +1
-0if (p->policy ==SCHED_EXT ) { p->policy =SCHED_NORMAL ; + p->sched_class = &fair_sched_class;atomic_long_inc (&scx_nr_rejected);
Did you see any issues with the existing implementation? The policy is set before the task is attached, so it should work fine. Also, you can't change sched_class by just assigning to it.
Thanks.
Signature · 1 line
tejun
hi,
Quoted 11 lines · Tejun Heo
Hello,
On Thu, Jul 11, 2024 at 07:07:20PM +0800, Zhangqiao (2012 lab) wrote:
...kernel/sched/ext.c static int scx_ops_init_task(struct task_struct *p, struct task_group *tg, bool · line 3231
Quoted 4 lines
if (p->policy == SCHED_EXT) { p->policy = SCHED_NORMAL;kernel/sched/ext.c +1
-0+ p->sched_class = &fair_sched_class;atomic_long_inc (&scx_nr_rejected);Did you see any issues with the existing implementation? The policy is set before the task is attached, so it should work fine. Also, you can't change
sched_classby just assigning to it.
What does "attach" mean? I'm not sure. p->sched_class is assigned in __sched_fork() which is performed before scx_ops_init_task().
Quoted 3 lines
Thanks.
Tejun Heo
reviewerHello,
Quoted 11 lines · On Tue, Jul 16, 2024 at 07:32:09PM +0800, Zhangqiao (2012 lab) wrote:
Quoted 9 lines · Tejun Heo
On Thu, Jul 11, 2024 at 07:07:20PM +0800, Zhangqiao (2012 lab) wrote:
...kernel/sched/ext.c static int scx_ops_init_task(struct task_struct *p, struct task_group *tg, bool · line 3231
Quoted 4 lines
if (p->policy == SCHED_EXT) { p->policy = SCHED_NORMAL;kernel/sched/ext.c +1
-0+ p->sched_class = &fair_sched_class;atomic_long_inc (&scx_nr_rejected);Did you see any issues with the existing implementation? The policy is set before the task is attached, so it should work fine. Also, you can't change
sched_classby just assigning to it.
Quoted 2 lines
What does "attach" mean? I'm not sure.
p->sched_classis assigned in__sched_fork()which is performed beforescx_ops_init_task().
Ah, I see what you mean. I was referring to the classs switching operations in scx_ops_enable(). You're looking at the fork path. I don't think we can switch sched_class at that point and the .disallow mechanism is there to allow the scheduler to filter out tasks on scheduler start. I'll update the code so that .disallow is only allowed during the initial attach.
Thanks.
Signature · 1 line
tejun
Quoted 13 lines · Tejun Heo
Hello,
Quoted 10 lines · On Tue, Jul 16, 2024 at 07:32:09PM +0800, Zhangqiao (2012 lab) wrote:
Quoted 9 lines · Tejun Heo
On Thu, Jul 11, 2024 at 07:07:20PM +0800, Zhangqiao (2012 lab) wrote:
...kernel/sched/ext.c static int scx_ops_init_task(struct task_struct *p, struct task_group *tg, bool · line 3231
Quoted 4 lines
if (p->policy == SCHED_EXT) { p->policy = SCHED_NORMAL;kernel/sched/ext.c +1
-0+ p->sched_class = &fair_sched_class;atomic_long_inc (&scx_nr_rejected);Did you see any issues with the existing implementation? The policy is set before the task is attached, so it should work fine. Also, you can't change
sched_classby just assigning to it.
Quoted 4 lines
Quoted 2 lines
What does "attach" mean? I'm not sure.
p->sched_classis assigned in__sched_fork()which is performed beforescx_ops_init_task().Ah, I see what you mean. I was referring to the classs switching operations in
scx_ops_enable(). You're looking at the fork path. I don't think we can
Yes, i was referring to the fork path.
Quoted 6 lines
switch
sched_classat that point and the .disallow mechanism is there to allow the scheduler to filter out tasks on scheduler start. I'll update the code so that .disallow is only allowed during the initial attach.Thanks.
Tejun Heo
reviewerQuoted 8 lines · On Wed, Jul 17, 2024 at 10:01:13AM +0800, Zhangqiao (2012 lab) wrote:
Quoted 2 lines
Ah, I see what you mean. I was referring to the classs switching operations in
scx_ops_enable(). You're looking at the fork path. I don't think we canYes, i was referring to the fork path.
Quoted 3 lines
switch
sched_classat that point and the .disallow mechanism is there to allow the scheduler to filter out tasks on scheduler start. I'll update the code so that .disallow is only allowed during the initial attach.
So, something like this.
Thanks.
include/linux/sched/ext.h +6 -5
@@ -181,11 +181,12 @@ struct sched_ext_entity {* If set, reject future sched_setscheduler(2) calls updating the policy * to %SCHED_EXT with -%EACCES. * -* If set from ops.init_task() and the task's policy is already -* %SCHED_EXT, which can happen while the BPF scheduler is being loaded -* or by inhering the parent's policy during fork, the task's policy is -* rejected and forcefully reverted to %SCHED_NORMAL. The number of -* such events are reported through /sys/kernel/debug/sched_ext::nr_rejected. +* Can be set from ops.init_task() while the BPF scheduler is being +* loaded (!scx_init_task_args->fork). If set and the task's policy is +* already %SCHED_EXT, the task's policy is rejected and forcefully +* reverted to %SCHED_NORMAL. The number of such events are reported +* through /sys/kernel/debug/sched_ext::nr_rejected. Setting this flag +* during fork is not allowed. */ bool disallow;/* reject switching into SCX */
kernel/sched/ext.c +8 -6
@@ -3399,18 +3399,17 @@ static int scx_ops_init_task(struct task_struct *p, struct task_group *tg, boolscx_set_task_state (p,SCX_TASK_INIT ); -if (p->scx.disallow) { +if (!fork && p->scx.disallow) {struct rq *rq;struct rq_flags rf; rq =task_rq_lock (p, &rf);/* -* We're either in fork or load path and @p->policy will be -* applied right after. Reverting @p->policy here and rejecting -* %SCHED_EXT transitions from scx_check_setscheduler() -* guarantees that if ops.init_task() sets @p->disallow, @p can -* never be in SCX. +* We're in the load path and @p->policy will be applied right +* after. Reverting @p->policy here and rejecting %SCHED_EXT +* transitions from scx_check_setscheduler() guarantees that if +* ops.init_task() sets @p->disallow, @p can never be in SCX. */ if (p->policy ==SCHED_EXT ) { p->policy =SCHED_NORMAL ;
@@ -3418,6 +3417,9 @@ static int scx_ops_init_task(struct task_struct *p, struct task_group *tg, bool }task_rq_unlock (rq, p, &rf); + }else if (p->scx.disallow) { +scx_ops_error ("ops.init_task() set task->scx.disallow for %s[%d] during fork" , + p->comm, p->pid); } p->scx.flags |=SCX_TASK_RESET_RUNNABLE_AT ;
Signature · 1 line
tejun
Quoted 4 lines · Tejun Heo
Quoted 3 lines · On Wed, Jul 17, 2024 at 10:01:13AM +0800, Zhangqiao (2012 lab) wrote:
Quoted 2 lines
Ah, I see what you mean. I was referring to the classs switching operations in
scx_ops_enable(). You're looking at the fork path. I don't think we can
Quoted 8 lines
Quoted 5 lines
Yes, i was referring to the fork path.
Quoted 3 lines
switch
sched_classat that point and the .disallow mechanism is there to allow the scheduler to filter out tasks on scheduler start. I'll update the code so that .disallow is only allowed during the initial attach.So, something like this.
LGTM for this patch.
In addition, the @scx_nr_rejected is only updated while the BPF scheduler is being loaded and this update behavior is proected by scx_ops_enable_mutex, so is it appropriate to change the @scx_nr_rejcted's type from atomic to int ?
Quoted 63 lines
Thanks.
include/linux/sched/ext.h +6
-5@@ -181,11 +181,12 @@ struct sched_ext_entity {* If set, reject future sched_setscheduler(2) calls updating the policy * to %SCHED_EXT with -%EACCES. * -* If set from ops.init_task() and the task's policy is already -* %SCHED_EXT, which can happen while the BPF scheduler is being loaded -* or by inhering the parent's policy during fork, the task's policy is -* rejected and forcefully reverted to %SCHED_NORMAL. The number of -* such events are reported through /sys/kernel/debug/sched_ext::nr_rejected. +* Can be set from ops.init_task() while the BPF scheduler is being +* loaded (!scx_init_task_args->fork). If set and the task's policy is +* already %SCHED_EXT, the task's policy is rejected and forcefully +* reverted to %SCHED_NORMAL. The number of such events are reported +* through /sys/kernel/debug/sched_ext::nr_rejected. Setting this flag +* during fork is not allowed. */ bool disallow;/* reject switching into SCX */ kernel/sched/ext.c +8
-6@@ -3399,18 +3399,17 @@ static int scx_ops_init_task(struct task_struct *p, struct task_group *tg, boolscx_set_task_state (p,SCX_TASK_INIT ); -if (p->scx.disallow) { +if (!fork && p->scx.disallow) {struct rq *rq;struct rq_flags rf; rq =task_rq_lock (p, &rf);/* -* We're either in fork or load path and @p->policy will be -* applied right after. Reverting @p->policy here and rejecting -* %SCHED_EXT transitions from scx_check_setscheduler() -* guarantees that if ops.init_task() sets @p->disallow, @p can -* never be in SCX. +* We're in the load path and @p->policy will be applied right +* after. Reverting @p->policy here and rejecting %SCHED_EXT +* transitions from scx_check_setscheduler() guarantees that if +* ops.init_task() sets @p->disallow, @p can never be in SCX. */ if (p->policy ==SCHED_EXT ) { p->policy =SCHED_NORMAL ;@@ -3418,6 +3417,9 @@ static int scx_ops_init_task(struct task_struct *p, struct task_group *tg, bool }task_rq_unlock (rq, p, &rf); + }else if (p->scx.disallow) { +scx_ops_error ("ops.init_task() set task->scx.disallow for %s[%d] during fork" , + p->comm, p->pid); } p->scx.flags |=SCX_TASK_RESET_RUNNABLE_AT ;
Tejun Heo
reviewerPATCH 1/1 sched_ext: Allow p->scx.disallow only while loading
p->scx.disallow provides a way for the BPF scheduler to reject certain tasks from attaching. It's currently allowed for both the load and fork paths; however, the latter doesn't actually work as p->sched_class is already set by the time scx_ops_init_task() is called during fork.
This is a convenience feature which is mostly useful from the load path anyway. Allow it only from the load path.
--- include/linux/sched/ext.h | 11 ++++++----- kernel/sched/ext.c | 14 ++++++++------ 2 files changed, 14 insertions(+), 11 deletions(-)
include/linux/sched/ext.h +6 -5
@@ -181,11 +181,12 @@ struct sched_ext_entity {* If set, reject future sched_setscheduler(2) calls updating the policy * to %SCHED_EXT with -%EACCES. * -* If set from ops.init_task() and the task's policy is already -* %SCHED_EXT, which can happen while the BPF scheduler is being loaded -* or by inhering the parent's policy during fork, the task's policy is -* rejected and forcefully reverted to %SCHED_NORMAL. The number of -* such events are reported through /sys/kernel/debug/sched_ext::nr_rejected. +* Can be set from ops.init_task() while the BPF scheduler is being +* loaded (!scx_init_task_args->fork). If set and the task's policy is +* already %SCHED_EXT, the task's policy is rejected and forcefully +* reverted to %SCHED_NORMAL. The number of such events are reported +* through /sys/kernel/debug/sched_ext::nr_rejected. Setting this flag +* during fork is not allowed. */ bool disallow;/* reject switching into SCX */
kernel/sched/ext.c +8 -6
@@ -3399,18 +3399,17 @@ static int scx_ops_init_task(struct taskscx_set_task_state (p,SCX_TASK_INIT ); -if (p->scx.disallow) { +if (!fork && p->scx.disallow) {struct rq *rq;struct rq_flags rf; rq =task_rq_lock (p, &rf);/* -* We're either in fork or load path and @p->policy will be -* applied right after. Reverting @p->policy here and rejecting -* %SCHED_EXT transitions from scx_check_setscheduler() -* guarantees that if ops.init_task() sets @p->disallow, @p can -* never be in SCX. +* We're in the load path and @p->policy will be applied right +* after. Reverting @p->policy here and rejecting %SCHED_EXT +* transitions from scx_check_setscheduler() guarantees that if +* ops.init_task() sets @p->disallow, @p can never be in SCX. */ if (p->policy ==SCHED_EXT ) { p->policy =SCHED_NORMAL ;
@@ -3418,6 +3417,9 @@ static int scx_ops_init_task(struct task }task_rq_unlock (rq, p, &rf); + }else if (p->scx.disallow) { +scx_ops_error ("ops.init_task() set task->scx.disallow for %s[%d] during fork" , + p->comm, p->pid); } p->scx.flags |=SCX_TASK_RESET_RUNNABLE_AT ;
Re: [PATCH sched_ext/for-6.12] sched_ext: Allow p->scx.disallow only while loading
Quoted 64 lines · Tejun Heo
p->scx.disallowprovides a way for the BPF scheduler to reject certain tasks from attaching. It's currently allowed for both the load and fork paths; however, the latter doesn't actually work asp->sched_classis already set by the timescx_ops_init_task()is called during fork.This is a convenience feature which is mostly useful from the load path anyway. Allow it only from the load path.
Signed-off-by Tejun Heo <[email protected]>Reported-by "Zhangqiao (2012 lab)" <[email protected]>Fixes 7bb6f0810ecf ("sched_ext: Allow BPF schedulers to disallow specific tasks from joining SCHED_EXT")--- include/linux/sched/ext.h | 11 ++++++----- kernel/sched/ext.c | 14 ++++++++------ 2 files changed, 14 insertions(+), 11 deletions(-)include/linux/sched/ext.h +6
-5@@ -181,11 +181,12 @@ struct sched_ext_entity {* If set, reject future sched_setscheduler(2) calls updating the policy * to %SCHED_EXT with -%EACCES. * -* If set from ops.init_task() and the task's policy is already -* %SCHED_EXT, which can happen while the BPF scheduler is being loaded -* or by inhering the parent's policy during fork, the task's policy is -* rejected and forcefully reverted to %SCHED_NORMAL. The number of -* such events are reported through /sys/kernel/debug/sched_ext::nr_rejected. +* Can be set from ops.init_task() while the BPF scheduler is being +* loaded (!scx_init_task_args->fork). If set and the task's policy is +* already %SCHED_EXT, the task's policy is rejected and forcefully +* reverted to %SCHED_NORMAL. The number of such events are reported +* through /sys/kernel/debug/sched_ext::nr_rejected. Setting this flag +* during fork is not allowed. */ bool disallow;/* reject switching into SCX */ kernel/sched/ext.c +5
-6@@ -3399,18 +3399,17 @@ static int scx_ops_init_task(struct taskscx_set_task_state (p,SCX_TASK_INIT ); -if (p->scx.disallow) { +if (!fork && p->scx.disallow) {struct rq *rq;struct rq_flags rf; rq =task_rq_lock (p, &rf);/* -* We're either in fork or load path and @p->policy will be -* applied right after. Reverting @p->policy here and rejecting -* %SCHED_EXT transitions from scx_check_setscheduler() -* guarantees that if ops.init_task() sets @p->disallow, @p can -* never be in SCX. +* We're in the load path and @p->policy will be applied right +* after. Reverting @p->policy here and rejecting %SCHED_EXT +* transitions from scx_check_setscheduler() guarantees that if +* ops.init_task() sets @p->disallow, @p can never be in SCX. */ if (p->policy ==SCHED_EXT ) { p->policy =SCHED_NORMAL ;
kernel/sched/ext.c static int scx_ops_init_task(struct task · line 3420
Quoted 4 lines
kernel/sched/ext.c +1
-0@@ -3418,6 +3417,9 @@ static int scx_ops_init_task(struct task }task_rq_unlock (rq, p, &rf); + }else if (p->scx.disallow) {
Just to make it a bit easier on schedulers, should we do this:
} else if (p->scx.disallow && p->policy == SCHED_EXT)
That way if you have a task that isn't running with SCHED_EXT and forks, the scheduler can set p->scx.disallow without having to check that it's being set in a fork. Seems unnecessary to enforce that given that the end result is the same.
Otherwise this LG. Feel free to apply if you agree, and add my ack:
Thanks,
David
Tejun Heo
reviewerPATCH v2 1/1 sched_ext: Allow p->scx.disallow only while loading
From 1232da7eced620537a78f19c8cf3d4a3508e2419 Mon Sep 17 00:00:00 2001
From: Tejun Heo <[email protected]>
Date: Wed, 31 Jul 2024 09:14:52 -1000
p->scx.disallow provides a way for the BPF scheduler to reject certain tasks from attaching. It's currently allowed for both the load and fork paths; however, the latter doesn't actually work as p->sched_class is already set by the time scx_ops_init_task() is called during fork.
This is a convenience feature which is mostly useful from the load path anyway. Allow it only from the load path.
v2: Trigger scx_ops_error() iff @p->policy == SCHED_EXT to make it a bit
easier for the BPF scheduler (David).
---
David, how does this look?
Thanks.
include/linux/sched/ext.h | 11 ++++++----- kernel/sched/ext.c | 35 ++++++++++++++++++++--------------- 2 files changed, 26 insertions(+), 20 deletions(-)
include/linux/sched/ext.h +6 -5
@@ -179,11 +179,12 @@ struct sched_ext_entity {* If set, reject future sched_setscheduler(2) calls updating the policy * to %SCHED_EXT with -%EACCES. * -* If set from ops.init_task() and the task's policy is already -* %SCHED_EXT, which can happen while the BPF scheduler is being loaded -* or by inhering the parent's policy during fork, the task's policy is -* rejected and forcefully reverted to %SCHED_NORMAL. The number of -* such events are reported through /sys/kernel/debug/sched_ext::nr_rejected. +* Can be set from ops.init_task() while the BPF scheduler is being +* loaded (!scx_init_task_args->fork). If set and the task's policy is +* already %SCHED_EXT, the task's policy is rejected and forcefully +* reverted to %SCHED_NORMAL. The number of such events are reported +* through /sys/kernel/debug/sched_ext::nr_rejected. Setting this flag +* during fork is not allowed. */ bool disallow;/* reject switching into SCX */
kernel/sched/ext.c +20 -15
@@ -3396,24 +3396,29 @@ static int scx_ops_init_task(struct task_struct *p, struct task_group *tg, boolscx_set_task_state (p,SCX_TASK_INIT );if (p->scx.disallow) { -struct rq *rq; -struct rq_flags rf; +if (!fork) { +struct rq *rq; +struct rq_flags rf; - rq =task_rq_lock (p, &rf); + rq =task_rq_lock (p, &rf); -/* -* We're either in fork or load path and @p->policy will be -* applied right after. Reverting @p->policy here and rejecting -* %SCHED_EXT transitions from scx_check_setscheduler() -* guarantees that if ops.init_task() sets @p->disallow, @p can -* never be in SCX. -*/ -if (p->policy ==SCHED_EXT ) { - p->policy =SCHED_NORMAL ; -atomic_long_inc (&scx_nr_rejected); - } +/* +* We're in the load path and @p->policy will be applied +* right after. Reverting @p->policy here and rejecting +* %SCHED_EXT transitions from scx_check_setscheduler() +* guarantees that if ops.init_task() sets @p->disallow, +* @p can never be in SCX. +*/ +if (p->policy ==SCHED_EXT ) { + p->policy =SCHED_NORMAL ; +atomic_long_inc (&scx_nr_rejected); + } -task_rq_unlock (rq, p, &rf); +task_rq_unlock (rq, p, &rf); + }else if (p->policy ==SCHED_EXT ) { +scx_ops_error ("ops.init_task() set task->scx.disallow for %s[%d] during fork" , + p->comm, p->pid); + } } p->scx.flags |=SCX_TASK_RESET_RUNNABLE_AT ;
Signature · 1 line
2.45.2
Re: [PATCH v2 sched_ext/for-6.12] sched_ext: Allow p->scx.disallow only while loading
Quoted 15 lines · Tejun Heo
From
1232da7eced620537a78f19c8cf3d4a3508e2419Mon Sep 17 00:00:00 2001
From: Tejun Heo <[email protected]>
Date: Wed, 31 Jul 2024 09:14:52 -1000
p->scx.disallowprovides a way for the BPF scheduler to reject certain tasks from attaching. It's currently allowed for both the load and fork paths; however, the latter doesn't actually work asp->sched_classis already set by the timescx_ops_init_task()is called during fork.This is a convenience feature which is mostly useful from the load path anyway. Allow it only from the load path.
v2: Trigger
scx_ops_error()iff @p->policy ==SCHED_EXTto make it a bit
easier for the BPF scheduler (David).
Quoted 7 lines
Signed-off-by Tejun Heo <[email protected]>Reported-by "Zhangqiao (2012 lab)" <[email protected]>Fixes 7bb6f0810ecf ("sched_ext: Allow BPF schedulers to disallow specific tasks from joining SCHED_EXT")Signed-off-by Tejun Heo <[email protected]>---
David, how does this look?
Looks great, thanks.
Re: [PATCH v2 sched_ext/for-6.12] sched_ext: Allow p->scx.disallow only while loading
In reply to Tejun HeoQuoted 12 lines · Tejun Heo
From
1232da7eced620537a78f19c8cf3d4a3508e2419Mon Sep 17 00:00:00 2001
From: Tejun Heo <[email protected]>
Date: Wed, 31 Jul 2024 09:14:52 -1000
p->scx.disallowprovides a way for the BPF scheduler to reject certain tasks from attaching. It's currently allowed for both the load and fork paths; however, the latter doesn't actually work asp->sched_classis already set by the timescx_ops_init_task()is called during fork.This is a convenience feature which is mostly useful from the load path anyway. Allow it only from the load path.
Quoted 8 lines
v2: Trigger
scx_ops_error()iff @p->policy ==SCHED_EXTto make it a bit
easier for the BPF scheduler (David).Signed-off-by Tejun Heo <[email protected]>Reported-by "Zhangqiao (2012 lab)" <[email protected]>Fixes 7bb6f0810ecf ("sched_ext: Allow BPF schedulers to disallow specific tasks from joining SCHED_EXT")Signed-off-by Tejun Heo <[email protected]>
Applied to sched_ext/for-6.12.
Thanks.
Signature · 1 line
tejun