kthread: run kthreadd with max priority SCHED_FIFO

kthreadd, the creator of other kernel threads, runs as a normal priority task. This is a potential for priority inversion when a task wants to spawn a high-priority kernel thread. A middle priority SCHED_FIFO task can block kthreadd's execution indefinitely and thus prevent the timely creation of the high-priority kernel thread.

This causes a practical problem. When a runaway real-time task is eating 100% CPU and we attempt to put the CPU offline, sometimes we block while waiting for the creation of the highest-priority "kstopmachine" thread.

The fix is to run kthreadd with the highest possible SCHED_FIFO priority. Its children must still run as slightly negatively reniced SCHED_NORMAL tasks.

Signed-off-by Michal Schmidt <[email protected]>

kernel/kthread.c +7 -0

index dcfe724..a7ce932 100644
@@ -94,10 +94,17 @@ static void create_kthread(struct kthread_create_info *create)
 	if (pid < 0) {
 		create->result = ERR_PTR(pid);
 	} else {
+		struct sched_param param = { .sched_priority = 0 };
 		wait_for_completion(&create->started);
 		read_lock(&tasklist_lock);
 		create->result = find_task_by_pid(pid);
 		read_unlock(&tasklist_lock);
+		/*
+		 * We (kthreadd) run with SCHED_FIFO, but we don't want
+		 * the kthreads we create to have it too by default.
+		 */
+		sched_setscheduler(create->result, SCHED_NORMAL, &param);
+		set_user_nice(create->result, -5);
 	}
 	complete(&create->done);
 }

@@ -217,11 +224,12 @@ EXPORT_SYMBOL(kthread_stop);
 int kthreadd(void *unused)
 {
 	struct task_struct *tsk = current;

kthreadd, the creator of other kernel threads, runs as a normal priority task. This is a potential for priority inversion when a task wants to spawn a high-priority kernel thread. A middle priority SCHED_FIFO task can block kthreadd's execution indefinitely and thus prevent the timely creation of the high-priority kernel thread.

This causes a practical problem. When a runaway real-time task is eating 100% CPU and we attempt to put the CPU offline, sometimes we block while waiting for the creation of the highest-priority "kstopmachine" thread.

The fix is to run kthreadd with the highest possible SCHED_FIFO priority. Its children must still run as slightly negatively reniced SCHED_NORMAL tasks.

Quoted 11 lines · On Mon, 17 Dec 2007 23:43:14 +0100 Michal Schmidt <[email protected]> wrote:

kthreadd, the creator of other kernel threads, runs as a normal priority task. This is a potential for priority inversion when a task wants to spawn a high-priority kernel thread. A middle priority SCHED_FIFO task can block kthreadd's execution indefinitely and thus prevent the timely creation of the high-priority kernel thread.

This causes a practical problem. When a runaway real-time task is eating 100% CPU and we attempt to put the CPU offline, sometimes we block while waiting for the creation of the highest-priority "kstopmachine" thread.

Quoted 3 lines

The fix is to run kthreadd with the highest possible SCHED_FIFO priority. Its children must still run as slightly negatively reniced SCHED_NORMAL tasks.

Did you hit this problem with the stock kernel, or have you been working on other stuff?

A locked-up SCHED_FIFO process will cause kernel threads all sorts of problems. You've hit one instance, but there will be others. (pdflush stops working, for one).

The general approach we've taken to this is "don't do that". Yes, we could boost lots of kernel threads in the way which this patch does but this actually takes control *away* from userspace. Userspace no longer has the ability to guarantee itself minimum possible latency without getting preempted by kernel threads.

And yes, giving userspace this minimum-latency capability does imply that userspace has a responsibility to not 100% starve kernel threads. It's a reasonable compromise, I think?

Quoted 9 lines

The general approach we've taken to this is "don't do that". Yes, we could boost lots of kernel threads in the way which this patch does but this actually takes control *away* from userspace. Userspace no longer has the ability to guarantee itself minimum possible latency without getting preempted by kernel threads.

And yes, giving userspace this minimum-latency capability does imply that userspace has a responsibility to not 100% starve kernel threads. It's a reasonable compromise, I think?

So, user tasks running with SCHED_FIFO should be able to lock a system?

I guess I see both sides of this argument - yes, it's userspace at fault, but in other cases when userspace is at fault, we take action (OOM, segfault, others). Isn't this situation just another case where the kernel needs to avoid the evils of userland going awry?

Should there be a watchdog which checks for a process which has run realtime for a certain period and which then takes some action? Such as descheduling it for a while, generating warnings, demoting its policy, killing it etc?

So, user tasks running with SCHED_FIFO should be able to lock a system? I guess I see both sides of this argument - yes, it's userspace at fault, but in other cases when userspace is at fault, we take action (OOM, segfault, others). Isn't this situation just another case where the kernel needs to avoid the evils of userland going awry?

Quoted 5 lines · Jon Masters

So, user tasks running with SCHED_FIFO should be able to lock a system? I guess I see both sides of this argument - yes, it's userspace at fault, but in other cases when userspace is at fault, we take action (OOM, segfault, others). Isn't this situation just another case where the kernel needs to avoid the evils of userland going awry?

FYI, Ingo queued the below.

http://lkml.org/lkml/2007/10/31/344

Quoted 9 lines · On Sat, 22 Dec 2007 11:39:30 +0100 Mike Galbraith <[email protected]> wrote:

Quoted 5 lines · Jon Masters

So, user tasks running with SCHED_FIFO should be able to lock a system? I guess I see both sides of this argument - yes, it's userspace at fault, but in other cases when userspace is at fault, we take action (OOM, segfault, others). Isn't this situation just another case where the kernel needs to avoid the evils of userland going awry?

Quoted 4 lines

FYI, Ingo queued the below.

http://lkml.org/lkml/2007/10/31/344

That's pretty different of course, but rlimit might be a suitable interface for implementing RLIMIT_MAX_CONTINUOUS_RT_MILLISECONDS.

Quoted 9 lines · On Sat, 22 Dec 2007 11:39:30 +0100 Mike Galbraith <[email protected]> wrote:

Quoted 5 lines · Jon Masters

So, user tasks running with SCHED_FIFO should be able to lock a system? I guess I see both sides of this argument - yes, it's userspace at fault, but in other cases when userspace is at fault, we take action (OOM, segfault, others). Isn't this situation just another case where the kernel needs to avoid the evils of userland going awry?

Quoted 4 lines

FYI, Ingo queued the below.

http://lkml.org/lkml/2007/10/31/344

That's pretty different of course, but rlimit might be a suitable interface for implementing RLIMIT_MAX_CONTINUOUS_RT_MILLISECONDS.

Quoted 11 lines · On Mon, 17 Dec 2007 23:43:14 +0100 Michal Schmidt <[email protected]> wrote:

kthreadd, the creator of other kernel threads, runs as a normal priority task. This is a potential for priority inversion when a task wants to spawn a high-priority kernel thread. A middle priority SCHED_FIFO task can block kthreadd's execution indefinitely and thus prevent the timely creation of the high-priority kernel thread.

This causes a practical problem. When a runaway real-time task is eating 100% CPU and we attempt to put the CPU offline, sometimes we block while waiting for the creation of the highest-priority "kstopmachine" thread.

Quoted 3 lines

The fix is to run kthreadd with the highest possible SCHED_FIFO priority. Its children must still run as slightly negatively reniced SCHED_NORMAL tasks.

Did you hit this problem with the stock kernel, or have you been working on other stuff?

A locked-up SCHED_FIFO process will cause kernel threads all sorts of problems. You've hit one instance, but there will be others. (pdflush stops working, for one).

The general approach we've taken to this is "don't do that". Yes, we could boost lots of kernel threads in the way which this patch does but this actually takes control *away* from userspace. Userspace no longer has the ability to guarantee itself minimum possible latency without getting preempted by kernel threads.

And yes, giving userspace this minimum-latency capability does imply that userspace has a responsibility to not 100% starve kernel threads. It's a reasonable compromise, I think?

Quoted 14 lines · On Sat, 22 Dec 2007 01:30:21 -0800 Andrew Morton <[email protected]>…

Quoted 11 lines · On Mon, 17 Dec 2007 23:43:14 +0100 Michal Schmidt <[email protected]> wrote:

kthreadd, the creator of other kernel threads, runs as a normal priority task. This is a potential for priority inversion when a task wants to spawn a high-priority kernel thread. A middle priority SCHED_FIFO task can block kthreadd's execution indefinitely and thus prevent the timely creation of the high-priority kernel thread.

This causes a practical problem. When a runaway real-time task is eating 100% CPU and we attempt to put the CPU offline, sometimes we block while waiting for the creation of the highest-priority "kstopmachine" thread.

Quoted 6 lines

Quoted 3 lines

The fix is to run kthreadd with the highest possible SCHED_FIFO priority. Its children must still run as slightly negatively reniced SCHED_NORMAL tasks.

Did you hit this problem with the stock kernel, or have you been working on other stuff?

This was with RHEL5 and with current Fedora kernels.

Quoted 10 lines

A locked-up SCHED_FIFO process will cause kernel threads all sorts of problems. You've hit one instance, but there will be others. (pdflush stops working, for one).

The general approach we've taken to this is "don't do that". Yes, we could boost lots of kernel threads in the way which this patch does but this actually takes control *away* from userspace. Userspace no longer has the ability to guarantee itself minimum possible latency without getting preempted by kernel threads.

Quoted 3 lines

And yes, giving userspace this minimum-latency capability does imply that userspace has a responsibility to not 100% starve kernel threads. It's a reasonable compromise, I think?

You're right. We should not run kthreadd with SCHED_FIFO by default. But the user should be able to change it using chrt if he wants to avoid this particular problem. So how about this instead?:

kthreadd, the creator of other kernel threads, runs as a normal priority task. This is a potential for priority inversion when a task wants to spawn a high-priority kernel thread. A middle priority SCHED_FIFO task can block kthreadd's execution indefinitely and thus prevent the timely creation of the high-priority kernel thread.

This causes a practical problem. When a runaway real-time task is eating 100% CPU and we attempt to put the CPU offline, sometimes we block while waiting for the creation of the highest-priority "kstopmachine" thread.

This could be solved by always running kthreadd with the highest possible SCHED_FIFO priority, but that would be undesirable policy decision in the kernel. kthreadd would cause unwanted latencies even for the realtime users who know what they're doing.

Let's not make the decision for the user. Just allow the administrator to change kthreadd's priority safely if he chooses to do it. Ensure that the kernel threads are created with the usual nice level even if kthreadd's priority is changed from the default.

Signed-off-by Michal Schmidt <[email protected]>

---
 kernel/kthread.c |   11 +++++++++++
 1 files changed, 11 insertions(+), 0 deletions(-)

kernel/kthread.c +4 -0

index dcfe724..e832a85 100644
@@ -94,10 +94,21 @@ static void create_kthread(struct kthread_create_info *create)
 	if (pid < 0) {
 		create->result = ERR_PTR(pid);
 	} else {
+		struct sched_param param = { .sched_priority = 0 };
 		wait_for_completion(&create->started);
 		read_lock(&tasklist_lock);
 		create->result = find_task_by_pid(pid);
 		read_unlock(&tasklist_lock);
+		/*
+		 * root may want to change our (kthreadd's) priority to
+		 * realtime to solve a corner case priority inversion problem

Quoted 5 lines

This causes a practical problem. When a runaway real-time task is eating 100% CPU and we attempt to put the CPU offline, sometimes we block while waiting for the creation of the highest-priority "kstopmachine" thread.

Let's not make the decision for the user. Just allow the administrator to change kthreadd's priority safely if he chooses to do it. Ensure that the kernel threads are created with the usual nice level even if kthreadd's priority is changed from the default.

Quoted 14 lines · On Sat, 22 Dec 2007 01:30:21 -0800 Andrew Morton <[email protected]>…

Quoted 11 lines · On Mon, 17 Dec 2007 23:43:14 +0100 Michal Schmidt <[email protected]> wrote:

kthreadd, the creator of other kernel threads, runs as a normal priority task. This is a potential for priority inversion when a task wants to spawn a high-priority kernel thread. A middle priority SCHED_FIFO task can block kthreadd's execution indefinitely and thus prevent the timely creation of the high-priority kernel thread.

This causes a practical problem. When a runaway real-time task is eating 100% CPU and we attempt to put the CPU offline, sometimes we block while waiting for the creation of the highest-priority "kstopmachine" thread.

Quoted 6 lines

Quoted 3 lines

The fix is to run kthreadd with the highest possible SCHED_FIFO priority. Its children must still run as slightly negatively reniced SCHED_NORMAL tasks.

Did you hit this problem with the stock kernel, or have you been working on other stuff?

This was with RHEL5 and with current Fedora kernels.

Quoted 10 lines

A locked-up SCHED_FIFO process will cause kernel threads all sorts of problems. You've hit one instance, but there will be others. (pdflush stops working, for one).

The general approach we've taken to this is "don't do that". Yes, we could boost lots of kernel threads in the way which this patch does but this actually takes control *away* from userspace. Userspace no longer has the ability to guarantee itself minimum possible latency without getting preempted by kernel threads.

Quoted 3 lines

And yes, giving userspace this minimum-latency capability does imply that userspace has a responsibility to not 100% starve kernel threads. It's a reasonable compromise, I think?

You're right. We should not run kthreadd with SCHED_FIFO by default. But the user should be able to change it using chrt if he wants to avoid this particular problem. So how about this instead?:

kthreadd, the creator of other kernel threads, runs as a normal priority task. This is a potential for priority inversion when a task wants to spawn a high-priority kernel thread. A middle priority SCHED_FIFO task can block kthreadd's execution indefinitely and thus prevent the timely creation of the high-priority kernel thread.

This causes a practical problem. When a runaway real-time task is eating 100% CPU and we attempt to put the CPU offline, sometimes we block while waiting for the creation of the highest-priority "kstopmachine" thread.

This could be solved by always running kthreadd with the highest possible SCHED_FIFO priority, but that would be undesirable policy decision in the kernel. kthreadd would cause unwanted latencies even for the realtime users who know what they're doing.

Let's not make the decision for the user. Just allow the administrator to change kthreadd's priority safely if he chooses to do it. Ensure that the kernel threads are created with the usual nice level even if kthreadd's priority is changed from the default.

Signed-off-by Michal Schmidt <[email protected]>

---
 kernel/kthread.c |   11 +++++++++++
 1 files changed, 11 insertions(+), 0 deletions(-)

kernel/kthread.c +11 -1

index dcfe724..e832a85 100644
@@ -94,10 +94,21 @@ static void create_kthread(struct kthread_create_info *create)
        if (pid < 0) {
                create->result = ERR_PTR(pid);
        } else {
+               struct sched_param param = { .sched_priority = 0 };
                wait_for_completion(&create->started);
                read_lock(&tasklist_lock);
                create->result = find_task_by_pid(pid);
                read_unlock(&tasklist_lock);
+               /*
+                * root may want to change our (kthreadd's) priority to
+                * realtime to solve a corner case priority inversion problem
+                * (a realtime task consuming 100% CPU blocking the creation of
+                * kernel threads). The kernel thread should not inherit the
+                * higher priority. Let's always create it with the usual nice
+                * level.
+                */
+               sched_setscheduler(create->result, SCHED_NORMAL, &param);
+               set_user_nice(create->result, -5);
        }
        complete(&create->done);
 }
--

1.5.3.3

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Hello Michal and Andrew,

Quoted 4 lines

Let's not make the decision for the user. Just allow the administrator to change kthreadd's priority safely if he chooses to do it. Ensure that the kernel threads are created with the usual nice level even if kthreadd's priority is changed from the default.

Last year, I posted a patchset (that was meant for Preempt-RT at that time) to be able to prioritise the interrupt-handler-threads (which are kthreads) and softirq-threads from the kernel commandline. See
http://lkml.org/lkml/2007/12/19/208

Maybe we can find a way to use a similar mechanism as I used in my patchset for the priorities of the remaining kthreads.
I do not like the way of forcing userland to change the priorities, because that would require a userland with the chrt tool installed, and that is not that practical for embedded systems (in which there could be cases that there is no userland at all, or the init-process is the whole embedded application). In that case an option to do it on the kernel commandline is more practical.

I propose this kernel cmd-line option:
kthread_pmap=somethread:50,otherthread:12,34

Quoted 102 lines · On Mon, 7 Jan 2008 11:06:03 +0100 Michal Schmidt <[email protected]> wrote:

Quoted 14 lines · On Sat, 22 Dec 2007 01:30:21 -0800 Andrew Morton <[email protected]>…

Quoted 11 lines · On Mon, 17 Dec 2007 23:43:14 +0100 Michal Schmidt <[email protected]> wrote:

kthreadd, the creator of other kernel threads, runs as a normal priority task. This is a potential for priority inversion when a task wants to spawn a high-priority kernel thread. A middle priority SCHED_FIFO task can block kthreadd's execution indefinitely and thus prevent the timely creation of the high-priority kernel thread.
This causes a practical problem. When a runaway real-time task is eating 100% CPU and we attempt to put the CPU offline, sometimes we block while waiting for the creation of the highest-priority "kstopmachine" thread.

Quoted 6 lines

Quoted 3 lines

The fix is to run kthreadd with the highest possible SCHED_FIFO priority. Its children must still run as slightly negatively reniced SCHED_NORMAL tasks.

Did you hit this problem with the stock kernel, or have you been working on other stuff?

This was with RHEL5 and with current Fedora kernels.

Quoted 11 lines

A locked-up SCHED_FIFO process will cause kernel threads all sorts of problems. You've hit one instance, but there will be others. (pdflush stops working, for one).

The general approach we've taken to this is "don't do that". Yes, we could boost lots of kernel threads in the way which this patch does but this actually takes control *away* from userspace. Userspace no longer has the ability to guarantee itself minimum possible latency without getting preempted by kernel threads.

Quoted 3 lines

And yes, giving userspace this minimum-latency capability does imply that userspace has a responsibility to not 100% starve kernel threads. It's a reasonable compromise, I think?

You're right. We should not run kthreadd with SCHED_FIFO by default. But the user should be able to change it using chrt if he wants to avoid this particular problem. So how about this instead?:

kthreadd, the creator of other kernel threads, runs as a normal priority task. This is a potential for priority inversion when a task wants to spawn a high-priority kernel thread. A middle priority SCHED_FIFO task can block kthreadd's execution indefinitely and thus prevent the timely creation of the high-priority kernel thread.

This causes a practical problem. When a runaway real-time task is eating 100% CPU and we attempt to put the CPU offline, sometimes we block while waiting for the creation of the highest-priority "kstopmachine" thread.

This could be solved by always running kthreadd with the highest possible SCHED_FIFO priority, but that would be undesirable policy decision in the kernel. kthreadd would cause unwanted latencies even for the realtime users who know what they're doing.

Let's not make the decision for the user. Just allow the administrator to change kthreadd's priority safely if he chooses to do it. Ensure that the kernel threads are created with the usual nice level even if kthreadd's priority is changed from the default.

Signed-off-by Michal Schmidt <[email protected]>

---
 kernel/kthread.c |   11 +++++++++++
 1 files changed, 11 insertions(+), 0 deletions(-)

kernel/kthread.c +3 -0

index dcfe724..e832a85 100644
@@ -94,10 +94,21 @@ static void create_kthread(struct
 kthread_create_info *create) if (pid < 0) {
 		create->result = ERR_PTR(pid);
 	} else {
+		struct sched_param param = { .sched_priority = 0 };
 		wait_for_completion(&create->started);
 		read_lock(&tasklist_lock);
 		create->result = find_task_by_pid(pid);
 		read_unlock(&tasklist_lock);
+		/*
+		 * root may want to change our (kthreadd's)

priority to

kernel/kthread.c +1 -0

+		 * realtime to solve a corner case priority

inversion problem

kernel/kthread.c +1 -0

+		 * (a realtime task consuming 100% CPU blocking

the creation of

kernel/kthread.c +1 -0

+		 * kernel threads). The kernel thread should not

inherit the

kernel/kthread.c +1 -0

+		 * higher priority. Let's always create it with

the usual nice

kernel/kthread.c +3 -0

+		 * level.
+		 */
+		sched_setscheduler(create->result, SCHED_NORMAL,

&param);

kernel/kthread.c

Quoted 4 lines

kernel/kthread.c +1 -0

+		set_user_nice(create->result, -5);
 	}
 	complete(&create->done);
 }

Seems reasonable.

As a followup thing, we now have two hard-coded magical -5's in kthread.c. It'd be nice to add a #define for this.

It'd be nicer to work out where on earth that -5 came from too ;)

Readers might wonder why kthreadd children disinherit kthreadd's policy and priority, but retain its cpus_allowed (and whatever other stuff root could have altered?)

Quoted 9 lines

Maybe we can find a way to use a similar mechanism as I used in my patchset for the priorities of the remaining kthreads.
I do not like the way of forcing userland to change the priorities, because that would require a userland with the chrt tool installed, and that is not that practical for embedded systems (in which there could be cases that there is no userland at all, or the init-process is the whole embedded application). In that case an option to do it on the kernel commandline is more practical.

Quoted 2 lines

I propose this kernel cmd-line option:
kthread_pmap=somethread:50,otherthread:12,34

I see. kthreadd would look up the priority for itself and kthread_create would consult the map for all other kernel threads. That should work.
Your sirq_pmap would not be needed anymore, as kthread_pmap could be used for softirq threads too, right?

Quoted 21 lines · On Mon, 7 Jan 2008 12:22:51 +0100 "Remy Bohmer" <[email protected]> wrote:

Hello Michal and Andrew,

Quoted 4 lines

Let's not make the decision for the user. Just allow the administrator to change kthreadd's priority safely if he chooses to do it. Ensure that the kernel threads are created with the usual nice level even if kthreadd's priority is changed from the default.

Last year, I posted a patchset (that was meant for Preempt-RT at that time) to be able to prioritise the interrupt-handler-threads (which are kthreads) and softirq-threads from the kernel commandline. See
http://lkml.org/lkml/2007/12/19/208

Maybe we can find a way to use a similar mechanism as I used in my patchset for the priorities of the remaining kthreads.
I do not like the way of forcing userland to change the priorities, because that would require a userland with the chrt tool installed, and that is not that practical for embedded systems (in which there could be cases that there is no userland at all, or the init-process is the whole embedded application). In that case an option to do it on the kernel commandline is more practical.

Quoted 2 lines

I propose this kernel cmd-line option:
kthread_pmap=somethread:50,otherthread:12,34

I see. kthreadd would look up the priority for itself and kthread_create would consult the map for all other kernel threads. That should work.
Your sirq_pmap would not be needed anymore, as kthread_pmap could be used for softirq threads too, right?

Michal

Quoted 5 lines

Quoted 5 lines

This causes a practical problem. When a runaway real-time task is eating 100% CPU and we attempt to put the CPU offline, sometimes we block while waiting for the creation of the highest-priority "kstopmachine" thread.

sched-devel.git has new mechanisms against runaway RT tasks. There's a new RLIMIT_RTTIME rlimit - if an RT task exceeds that rlimit then it is sent SIGXCPU.

there's also a new group scheduling extension that is driven via a sysctl:

/proc/sys/kernel/sched_rt_ratio

this way if a user has a runaway RT task, other users (and root) will still have some CPU time left. (in Peter's latest patchset that is replaced via rt_runtime_ns - but this is a detail)

so instead of the never-ending arms race of kernel thread priorities against RT task priorities, we are going towards making RT tasks safer on a policy level.

Ingo

Quoted 9 lines · On Mon, 7 Jan 2008 12:09:04 +0100 Ingo Molnar <[email protected]> wrote:

Quoted 5 lines

Quoted 5 lines

This causes a practical problem. When a runaway real-time task is eating 100% CPU and we attempt to put the CPU offline, sometimes we block while waiting for the creation of the highest-priority "kstopmachine" thread.

sched-devel.git has new mechanisms against runaway RT tasks. There's a new RLIMIT_RTTIME rlimit - if an RT task exceeds that rlimit then it is sent SIGXCPU.

Is that "total RT CPU time" or "elapsed time since last schedule()"?

If the former, it is not useful for this problem.

Quoted 3 lines

there's also a new group scheduling extension that is driven via a sysctl:

Quoted 5 lines

/proc/sys/kernel/sched_rt_ratio

this way if a user has a runaway RT task, other users (and root) will still have some CPU time left. (in Peter's latest patchset that is replaced via rt_runtime_ns - but this is a detail)

Doesn't this make the RT task non-RT? Would need to understand more details to tell.

Quoted 6 lines · On Mon, 7 Jan 2008 12:09:04 +0100 Ingo Molnar <[email protected]> wrote:

Quoted 5 lines

Quoted 5 lines

This causes a practical problem. When a runaway real-time task is eating 100% CPU and we attempt to put the CPU offline, sometimes we block while waiting for the creation of the highest-priority "kstopmachine" thread.

Quoted 3 lines

sched-devel.git has new mechanisms against runaway RT tasks. There's a new RLIMIT_RTTIME rlimit - if an RT task exceeds that rlimit then it is sent SIGXCPU.

Is that "total RT CPU time" or "elapsed time since last schedule()"?

If the former, it is not useful for this problem.

Allow the administrator to change kthreadd's priority and affinity. Ensure that the kernel threads are created with the usual nice level and affinity even if kthreadd's properties were changed from the default.

Signed-off-by Michal Schmidt <[email protected]>