Skip to contentSkip to conversation

MAX_USER_RT_PRIO and MAX_RT_PRIO are wrong!

PATCH In review Steven Rostedt 3 participants 9 messages Jun 7, 2005

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 MAX_USER_RT_PRIO and MAX_RT_PRIO are wrong!

According to the comments in include/linux/sched.h

/*
* Priority of a process goes from 0..MAX_PRIO-1, valid RT
* priority is 0..MAX_RT_PRIO-1, and SCHED_NORMAL tasks are
* in the range MAX_RT_PRIO..MAX_PRIO-1. Priority values
* are inverted: lower p->prio value means higher priority. *
* The MAX_USER_RT_PRIO value allows the actual maximum
* RT priority to be separate from the value exported to
* user-space. This allows kernel threads to set their
* priority to a value higher than any user task. Note:
* MAX_RT_PRIO must not be smaller than MAX_USER_RT_PRIO. */

This makes it look like the priority goes as follows:

prio: 0 .. MAX_RT_PRIO .. MAX_USER_RT_PRIO .. MAX_PRIO

where 0 is of highest priority

but in reality we have:

prio: 0 .. MAX_USER_RT_PRIO .. MAX_RT_PRIO .. MAX_PRIO

The comments say that MAX_RT_PRIO must not be smaller than MAX_USER_RT_PRIO, but if it is bigger (thinking bigger means greater than) then the system will crash on a SMP machine.

Here's how it works. The migration_thread sets the priority of its thread to MAX_RT_PRIO-1 via:

__setscheduler(p, SCHED_FIFO, MAX_RT_PRIO-1);

Now looking at __setscheduler

static void __setscheduler(struct task_struct *p, int policy, int prio)

{
        BUG_ON(p->array);
        p->policy = policy;
        p->rt_priority = prio;
        if (policy != SCHED_NORMAL)
                p->prio = MAX_USER_RT_PRIO-1 - p->rt_priority;
        else
                p->prio = p->static_prio;
}

If we have MAX_USER_RT_PRIO = 99 and MAX_RT_PRIO = 100 then we would get

  p->prio = 99-1 - 100-1 = -1;

This would be very bad when it comes time to schedule. Not to mention that kstop_machine uses MAX_RT_PRIO and then calls sys_sched_setscheduler, which would fail if MAX_RT_PRIO > MAX_USER_RT_PRIO. Below is a patch that makes MAX_RT_PRIO work if it is greater than MAX_USER_RT_PRIO on a SMP machine. The p->mm is to allow kstop_machine to work and any other kernel threads.

I tested the patch on an SMP machine where MAX_RT_PRIO = 100 and MAX_USER_RT_PRIO = 99. Without the patch, the system crashes with a reboot.

Funny, back in July 2002, this was noticed by an Anton Wilson and he was just lost in the noise!
http://seclists.org/lists/linux-kernel/2002/Jul/1695.html

-- Steve

kernel/sched.c +3 -2
diff -u linux-2.6.12-rc5.orig/kernel/sched.c linux-2.6.12-rc5/kernel/sched.c
@@ -3347,7 +3347,7 @@
 	p->policy = policy;
 	p->rt_priority = prio;
 	if (policy != SCHED_NORMAL)
-		p->prio = MAX_USER_RT_PRIO-1 - p->rt_priority;
+		p->prio = MAX_RT_PRIO-1 - p->rt_priority;
 	else
 		p->prio = p->static_prio;
 }
@@ -3379,7 +3379,8 @@
 	 * 1..MAX_USER_RT_PRIO-1, valid priority for SCHED_NORMAL is 0.
 	 */
 	if (param->sched_priority < 0 ||
-	    param->sched_priority > MAX_USER_RT_PRIO-1)
+	    (p->mm &&  param->sched_priority > MAX_USER_RT_PRIO-1) ||
+	    (!p->mm && param->sched_priority > MAX_RT_PRIO-1))
 		return -EINVAL;
 	if ((policy == SCHED_NORMAL) != (param->sched_priority == 0))
 		return -EINVAL;
#1 lore raw
Quoted 5 lines · Steven Rostedt

This makes it look like the priority goes as follows:

prio: 0 .. MAX_RT_PRIO .. MAX_USER_RT_PRIO .. MAX_PRIO

where 0 is of highest priority

I'm correcting my own post :-)

What we really want is:

prio: 0 .. [MAX_RT_PRIO - MAX_USER_RT_PRIO] .. MAX_RT_PRIO .. MAX_PRIO

                                                    |---- nice -------|
                         |------ user RT prio ------|
      |------------ kernel RT prio -----------------|

Remember, 0 is of highest priority.

-- Steve

#3 lore raw
Quoted 7 lines · Ingo Molnar
Quoted 3 lines · Steven Rostedt

I tested the patch on an SMP machine where MAX_RT_PRIO = 100 and MAX_USER_RT_PRIO = 99. Without the patch, the system crashes with a reboot.

If this patch does go in, then xpc_activating in arch/ia64/sn/kernel/xpc_main.c (from rc6) also needs to use MAX_RT_PRIO instead of MAX_USER_RT_PRIO. Unless it is OK that it runs lower in priority than other kernel threads.

-- Steve

#4 lore raw
Quoted 7 lines · Steven Rostedt
Quoted 6 lines · Ingo Molnar
Quoted 3 lines · Steven Rostedt

I tested the patch on an SMP machine where MAX_RT_PRIO = 100 and MAX_USER_RT_PRIO = 99. Without the patch, the system crashes with a reboot.

Quoted 6 lines
Quoted 1 line

If this patch does go in, then xpc_activating in arch/ia64/sn/kernel/xpc_main.c (from rc6) also needs to use MAX_RT_PRIO instead of MAX_USER_RT_PRIO. Unless it is OK that it runs lower in priority than other kernel threads.

You are correct xpc_activating() needs to be changed to use MAX_RT_PRIO. So please do add that change to your patch.

Thanks,
Dean

#5 lore raw
Quoted 2 lines · Dean Nelson

You are correct xpc_activating() needs to be changed to use MAX_RT_PRIO. So please do add that change to your patch.

I haven't tested this patch, I just used the previous patch (which I did test) and added your change.

-- Steve

kernel/sched.c renamed from kernel/sched.c.orig +3 -2
@@ -3347,7 +3347,7 @@
 	p->policy = policy;
 	p->rt_priority = prio;
 	if (policy != SCHED_NORMAL)
-		p->prio = MAX_USER_RT_PRIO-1 - p->rt_priority;
+		p->prio = MAX_RT_PRIO-1 - p->rt_priority;
 	else
 		p->prio = p->static_prio;
 }
@@ -3379,7 +3379,8 @@
 	 * 1..MAX_USER_RT_PRIO-1, valid priority for SCHED_NORMAL is 0.
 	 */
 	if (param->sched_priority < 0 ||
-	    param->sched_priority > MAX_USER_RT_PRIO-1)
+	    (p->mm &&  param->sched_priority > MAX_USER_RT_PRIO-1) ||
+	    (!p->mm && param->sched_priority > MAX_RT_PRIO-1))
 		return -EINVAL;
 	if ((policy = SCHED_NORMAL) != (param->sched_priority = 0))
 		return -EINVAL;
arch/ia64/sn/kernel/xpc_main.c renamed from arch/ia64/sn/kernel/xpc_main.c.orig +1 -1
@@ -420,7 +420,7 @@
 	partid_t partid = (u64) __partid;
 	struct xpc_partition *part = &xpc_partitions[partid];
 	unsigned long irq_flags;
-	struct sched_param param = { sched_priority: MAX_USER_RT_PRIO - 1 };
+	struct sched_param param = { sched_priority: MAX_RT_PRIO - 1 };
 	int ret;
 
 
#6 lore raw
Quoted 6 lines · Steven Rostedt
Quoted 2 lines · Dean Nelson

You are correct xpc_activating() needs to be changed to use MAX_RT_PRIO. So please do add that change to your patch.

I haven't tested this patch, I just used the previous patch (which I did test) and added your change.

I just built and tested a kernel and xp/xpc/xpnet modules with your patch applied. It ran fine. The priorities of the xpc kthreads were correct.

Looks good to me.

Thanks,
Dean

#7 lore raw
Quoted 4 lines · Dean Nelson

I just built and tested a kernel and xp/xpc/xpnet modules with your patch applied. It ran fine. The priorities of the xpc kthreads were correct.

Looks good to me.

Dean,

If you can do me a favor, the way you really want to test this is by changing MAX_USER_RT_PRIO to 99 and MAX_RT_PRIO to (MAX_USER_RT_PRIO+1). This will make sure that the patch is working. Your kernel thread should still run at priority 99.

Check it with: ps -eo pid,rtprio,comm

And grep for your thread name.

Thanks,

-- Steve

#8 lore raw
Quoted 9 lines · Steven Rostedt
Quoted 4 lines · Dean Nelson

I just built and tested a kernel and xp/xpc/xpnet modules with your patch applied. It ran fine. The priorities of the xpc kthreads were correct.

Looks good to me.

Dean,

Quoted 8 lines

If you can do me a favor, the way you really want to test this is by changing MAX_USER_RT_PRIO to 99 and MAX_RT_PRIO to (MAX_USER_RT_PRIO+1). This will make sure that the patch is working. Your kernel thread should still run at priority 99.

Check it with: ps -eo pid,rtprio,comm

And grep for your thread name.

Just did as you asked and things seem fine. Ran on an SGI altix. (The first process shown below shouldn't have a priority of 99, just the others.)

cranberry5:~ # ps -eo pid,rtprio,comm | grep xpc
13325 - xpc_hb

13327     99 xpc08
13467     99 xpc06
13469     99 xpc06c1
13501     99 xpc06c1
13502     99 xpc06c1

cranberry5:~ #