scheduler: introduce SCHED_RESET_ON_FORK scheduling policy flag, Second try
PATCH 1/1 scheduler: introduce SCHED_RESET_ON_FORK scheduling policy flag, Second try
(Nobody commented so far, but here's an updated version of the patch. Basically the only change is that the SCHED_RESET_ON_FORK flag is now reset on fork()).)
This patch introduces a new flag SCHED_RESET_ON_FORK which can be passed to the kernel via sched_setscheduler(), ORed in the policy parameter. If set this will make sure that when the process forks a) the scheduling priority is reset to DEFAULT_PRIO if it was higher and b) the scheduling policy is reset to SCHED_NORMAL if it was either SCHED_FIFO or SCHED_RR.
Why have this?
Currently, if a process is real-time scheduled this will 'leak' to all its child processes. For security reasons it is often (always?) a good idea to make sure that if a process acquires RT scheduling this is confined to this process and only this process. More specifically this makes the per-process resource limit RLIMIT_RTTIME useful for security purposes, because it makes it impossible to use a fork bomb to circumvent the per-process RLIMIT_RTTIME accounting.
This feature is also useful for tools like 'renice' which can then change the nice level of a process without having this spill to all its child processes.
Why expose this via sched_setscheduler() and not other syscalls such as prctl() or sched_setparam()?
prctl() does not take a pid parameter. Due to that it would be impossible to modify this flag for other processes than the current one.
The struct passed to sched_setparam() can unfortunately not be extended without breaking compatibility, since sched_setparam() lacks a size parameter.
How to use this from userspace? In your RT program simply replace this:
sched_setscheduler(pid, SCHED_FIFO, ¶m);
by this:
sched_setscheduler(pid, SCHED_FIFO|SCHED_RESET_ON_FORK, ¶m);
This current patch compiles but is not otherwise tested. For now, I am sending this primarily as a request for comments. So please, I'd love to hear your comments!
Lennart
--- include/linux/sched.h | 4 ++++ kernel/sched.c | 47 ++++++++++++++++++++++++++++++++++++++--------- 2 files changed, 42 insertions(+), 9 deletions(-)
include/linux/sched.h +4 -0
@@ -38,6 +38,8 @@#define SCHED_BATCH 3/* SCHED_ISO: reserved but not implemented yet */ #define SCHED_IDLE 5 +/* Can be ORed in to make sure the process is reverted back to SCHED_NORMAL on fork */ +#define SCHED_RESET_ON_FORK 0x40000000#ifdef __KERNEL__
@@ -1429,6 +1431,8 @@ struct task_struct {/* state flags for use by tracers */ unsigned long trace;#endif +/* Revert to default priority/policy when forking */ + unsigned sched_reset_on_fork:1; };/* Future-safe accessor for struct task_struct's cpus_allowed. */
kernel/sched.c +38 -9
93 lines not on this page · raw message
Signature · 10 lines
1.6.2.2 Lennart -- Lennart Poettering Red Hat, Inc. lennart [at] poettering [dot] net ICQ# 11060553 http://0pointer.net/lennart/ GnuPG 0x1A015CC4
Peter Zijlstra
maintainerQuoted 5 lines · Lennart Poettering
This patch introduces a new flag
SCHED_RESET_ON_FORKwhich can be passed to the kernel viasched_setscheduler(), ORed in the policy parameter. If set this will make sure that when the process forks a) the scheduling priority is reset toDEFAULT_PRIOif it was higher and b) the scheduling policy is reset toSCHED_NORMALif it was eitherSCHED_FIFOorSCHED_RR.
Maybe SCHED_OTHER_ON_FORK, or SCHED_DEFAULT_ON_FORK?
Quoted 14 lines
Why have this?
Currently, if a process is real-time scheduled this will 'leak' to all its child processes. For security reasons it is often (always?) a good idea to make sure that if a process acquires RT scheduling this is confined to this process and only this process. More specifically this makes the per-process resource limit
RLIMIT_RTTIMEuseful for security purposes, because it makes it impossible to use a fork bomb to circumvent the per-processRLIMIT_RTTIMEaccounting.This feature is also useful for tools like 'renice' which can then change the nice level of a process without having this spill to all its child processes.
Quoted 5 lines
Why expose this via
sched_setscheduler()and not other syscalls such asprctl()orsched_setparam()?
prctl()does not take a pid parameter. Due to that it would be impossible to modify this flag for other processes than the current one.
Yeah, same goes for rlimits, I think someone once proposed to allow changing them in /proc/$pid/limits but I'm not sure what happened to that proposal.
Quoted 3 lines
The struct passed to
sched_setparam()can unfortunately not be extended without breaking compatibility, sincesched_setparam()lacks a size parameter.
Yeah, we're going to have to cross that bridge at some point through, advanced scheduling policies really need more information. But yeah, using the upper bits of the policy might work for this.
Quoted 11 lines
How to use this from userspace? In your RT program simply replace this:
sched_setscheduler(pid, SCHED_FIFO, ¶m);by this:
sched_setscheduler(pid, SCHED_FIFO|SCHED_RESET_ON_FORK, ¶m);This current patch compiles but is not otherwise tested. For now, I am sending this primarily as a request for comments. So please, I'd love to hear your comments!
Seems reasonable, Ingo?
comments on the implementation below.
Quoted 20 lines
Lennart
--- include/linux/sched.h | 4 ++++ kernel/sched.c | 47 ++++++++++++++++++++++++++++++++++++++--------- 2 files changed, 42 insertions(+), 9 deletions(-)include/linux/sched.h +2
-0@@ -38,6 +38,8 @@#define SCHED_BATCH 3/* SCHED_ISO: reserved but not implemented yet */ #define SCHED_IDLE 5 +/* Can be ORed in to make sure the process is reverted back to SCHED_NORMAL on fork */ +#define SCHED_RESET_ON_FORK 0x40000000#ifdef __KERNEL__
include/linux/sched.h struct task_struct { · line 1436
Quoted 6 lines
include/linux/sched.h +2
-0@@ -1429,6 +1431,8 @@ struct task_struct {/* state flags for use by tracers */ unsigned long trace;#endif +/* Revert to default priority/policy when forking */ + unsigned sched_reset_on_fork:1; };
There seems to be a bit hole around in_execve, maybe add it there.
Quoted 13 lines
include/linux/sched.h +0
-0/* Future-safe accessor for struct task_struct's cpus_allowed. */ kernel/sched.c +2
-1@@ -2529,12 +2529,26 @@ void sched_fork(struct task_struct *p, int clone_flags)set_task_cpu (p, cpu);/* -* Make sure we do not leak PI boosting priority to the child: +* Revert to default priority/policy on fork if requested. Make sure we +* do not leak PI boosting priority to the child. */
kernel/sched.c void sched_fork(struct task_struct *p, int clone_flags) · line 2543
Quoted 10 lines
kernel/sched.c +9
-1- p->prio = current->normal_prio; +if (current->sched_reset_on_fork && + (p->policy ==SCHED_FIFO || p->policy ==SCHED_RR )) + p->policy =SCHED_NORMAL ; + +if (current->sched_reset_on_fork && + (current->normal_prio <DEFAULT_PRIO )) + p->prio =DEFAULT_PRIO ; +else + p->prio = current->normal_prio;
This bit hurt my brain, what's wrong with:
p->prio = current->normal_prio; /* don't leak boosted prio */
if (current->sched_reset_on_fork) {
p->policy = SCHED_NORMAL;
p->prio = p->normal_prio = p->static_prio = DEFAULT_PRIO;
}Quoted 10 lines
kernel/sched.c +4
-0if (!rt_prio (p->prio)) p->sched_class = &fair_sched_class; +/* We don't need the reset flag anymore after the fork. It has +* fulfilled its duty. */ + p->sched_reset_on_fork =0 ; +#if defined(CONFIG_SCHEDSTATS ) || defined(CONFIG_TASK_DELAY_ACCT )if (likely (sched_info_on ()))memset (&p->sched_info,0 ,sizeof (p->sched_info));
Quoted 7 lines · Peter Zijlstra
Quoted 5 lines
This patch introduces a new flag
SCHED_RESET_ON_FORKwhich can be passed to the kernel viasched_setscheduler(), ORed in the policy parameter. If set this will make sure that when the process forks a) the scheduling priority is reset toDEFAULT_PRIOif it was higher and b) the scheduling policy is reset toSCHED_NORMALif it was eitherSCHED_FIFOorSCHED_RR.Maybe
SCHED_OTHER_ON_FORK, orSCHED_DEFAULT_ON_FORK?
Hmm, I think this would be misleading, since it does more than just reset to SCHED_OTHER. It also resets the nice level.
Quoted 12 lines
kernel/sched.c void sched_fork(struct task_struct *p, int clone_flags) · line 2543
Quoted 9 lines
kernel/sched.c +9
-0+if (current->sched_reset_on_fork && + (p->policy ==SCHED_FIFO || p->policy ==SCHED_RR )) + p->policy =SCHED_NORMAL ; + +if (current->sched_reset_on_fork && + (current->normal_prio <DEFAULT_PRIO )) + p->prio =DEFAULT_PRIO ; +else + p->prio = current->normal_prio;This bit hurt my brain, what's wrong with:
Quoted 5 lines
p->prio = current->normal_prio; /* don't leak boosted prio */
if (current->sched_reset_on_fork) {
p->policy = SCHED_NORMAL;
p->prio = p->normal_prio = p->static_prio = DEFAULT_PRIO;
}This would do something very different. i.e. with this suggested code of yours SCHED_IDLE/SCHED_BATCH would be upgraded to SCHED_OTHER on fork and a process with a positive nice level would be upgraded to to nice level 0. My original patch would only downgrade, never upgrade.
It is of course a question what makes more sense here: a logic that makes sure that only "elevated" scheduling gets reset to normal scheduling, or one that would reset both 'higher' and 'lower' scheduling settings to normal.
Given that this is supposed to be a security feature it makes more sense to me to only 'decrease' scheduling settings, not 'increase' them. Dunno, what do you think?
Lennart
Signature · 3 lines
Lennart Poettering Red Hat, Inc. lennart [at] poettering [dot] net http://0pointer.net/lennart/ GnuPG 0x1A015CC4
Quoted 5 lines · Lennart Poettering
This patch introduces a new flag
SCHED_RESET_ON_FORKwhich can be passed to the kernel viasched_setscheduler(), ORed in the policy parameter. If set this will make sure that when the process forks a) the scheduling priority is reset toDEFAULT_PRIOif it was higher and b) the scheduling policy is reset toSCHED_NORMALif it was eitherSCHED_FIFOorSCHED_RR.
[...]
Quoted 3 lines
This current patch compiles but is not otherwise tested. For now, I am sending this primarily as a request for comments. So please, I'd love to hear your comments!
I have now done some testing to make sure it actually works as advertised. Everything seems to be fine. For those interested here's a little test program that verifies that RR scheduling is properly removed from the child process but is kept enabled in the parent process:
http://0pointer.de/public/reset-on-fork.c
Ingo, how can I bribe you into taking this patch, or at least commenting on it?
Thanks,
Lennart
Signature · 3 lines
Lennart Poettering Red Hat, Inc. lennart [at] poettering [dot] net http://0pointer.net/lennart/ GnuPG 0x1A015CC4
Quoted 10 lines · Lennart Poettering
Why expose this via
sched_setscheduler()and not other syscalls such asprctl()orsched_setparam()?
prctl()does not take a pid parameter. Due to that it would be impossible to modify this flag for other processes than the current one.The struct passed to
sched_setparam()can unfortunately not be extended without breaking compatibility, sincesched_setparam()lacks a size parameter.
Well, it could be extended, if we wanted to. Right now sched_priority has a valid value of 0 to 100. We could introduce a new value '-1' to mean: 'extended struct sched_param'.
A new getparam syscall could then be introduced - only used by new user-space.
Ingo
Quoted 15 lines · Ingo Molnar
Quoted 10 lines · Lennart Poettering
Why expose this via
sched_setscheduler()and not other syscalls such asprctl()orsched_setparam()?
prctl()does not take a pid parameter. Due to that it would be impossible to modify this flag for other processes than the current one.The struct passed to
sched_setparam()can unfortunately not be extended without breaking compatibility, sincesched_setparam()lacks a size parameter.
Quoted 6 lines
Well, it could be extended, if we wanted to. Right now
sched_priorityhas a valid value of 0 to 100. We could introduce a new value '-1' to mean: 'extended structsched_param'.A new getparam syscall could then be introduced - only used by new user-space.
The man page of sched_getparam() is pretty explicit in that the sched_priority field of the struct must lie between sched_get_priority_min() and sched_get_priority_max(). If you'd overload sched_priority like this you might end up breaking applications that rely on this, for example RT watchdogs that go through /proc and query the scheduling parameters of all threads.
Any further comments on the patch? Could this be merged? Any changes necessary?
Lennart
Signature · 3 lines
Lennart Poettering Red Hat, Inc. lennart [at] poettering [dot] net http://0pointer.net/lennart/ GnuPG 0x1A015CC4
Quoted 24 lines · Lennart Poettering
Quoted 15 lines · Ingo Molnar
Quoted 10 lines · Lennart Poettering
Why expose this via
sched_setscheduler()and not other syscalls such asprctl()orsched_setparam()?
prctl()does not take a pid parameter. Due to that it would be impossible to modify this flag for other processes than the current one.The struct passed to
sched_setparam()can unfortunately not be extended without breaking compatibility, sincesched_setparam()lacks a size parameter.Quoted 6 lines
Well, it could be extended, if we wanted to. Right now
sched_priorityhas a valid value of 0 to 100. We could introduce a new value '-1' to mean: 'extended structsched_param'.A new getparam syscall could then be introduced - only used by new user-space.
Quoted 6 lines
The man page of
sched_getparam()is pretty explicit in that thesched_priorityfield of the struct must lie betweensched_get_priority_min()andsched_get_priority_max(). If you'd overloadsched_prioritylike this you might end up breaking applications that rely on this, for example RT watchdogs that go through /proc and query the scheduling parameters of all threads.
Only the new sched_gtparam syscall would do this - and that syscall would have to be propagated to glibc and user-space with appropriate care of course. (probably resulting in a separate function not a change/breakage of existing functionality)
Probably not worth the pain though.
Ingo