2.4.18-pre4 sys_sched_yield and goodness optimization

Ingo's sched cleanup code in 2.4.15 added move_last_runqueue() to sys_sched_yield(). This change should improve performance of user code that uses yield to implement user level locking. Should help fairness too.

This patch moves the move_last_runqueue() from sys_sched_yield() to schedule() and removes the SCHED_YIELD specific code from goodness(). Overall, it is a reduction of code path + the removal of the runqueue lock in sys_sched_yield().

There are about 40 or 50 places in the kernel where SCHED_YIELD is used. Those yields will also be moved to the end of the run queue with this patch.

The patch was tested on lmbench and VolanoMark.
VolanoMark does a lot of yields and showed a 34 % improvement on an 8-way. The test results and patch are below.

Bill Hartner
IBM Linux Technology Center

-----

We tested using the LMBench latency tests and VolanoMark on an 8-way 700 Mhz PIII, 1 GB mem.

LMBench results (best of 3 runs) :

2.4.18-pre4 (UP)

./lat_proc fork
Process fork+exit: 118.7021 microseconds ./lat_proc exec
Process fork+execve: 836.7143 microseconds ./lat_proc shell
Process fork+/bin/sh -c: 3481.5000 microseconds

2.4.18-pre4 + patch (UP)

./lat_proc fork
Process fork+exit: 118.5957 microseconds ./lat_proc exec
Process fork+execve: 838.0000 microseconds ./lat_proc shell
Process fork+/bin/sh -c: 3464.5000 microseconds

2.4.18-pre4 (2P)

./lat_proc fork
Process fork+exit: 480.0833 microseconds ./lat_proc exec
Process fork+execve: 1342.6000 microseconds ./lat_proc shell
Process fork+/bin/sh -c: 4778.5000 microseconds

2.4.18-pre4 + patch (2P)

./lat_proc fork
Process fork+exit: 474.9090 microseconds ./lat_proc exec
Process fork+execve: 1327.8000 microseconds ./lat_proc shell
Process fork+/bin/sh -c: 4680.5000 microseconds

-----

VolanoMark 10/100 loopback test results :

8-way 700 Mhz PIII, 1 GB mem
IBM JVM 1.3 (20010621), 64 MB initial heap size

			UP	4P	8P
2.4.18-pre4		11022	16022	11535 msg/sec
2.4.18-pre4 + patch	11252	15222	15550 msg/sec
---------------------------------------------
			+2.1%	-5.0%	+34.8%

The patch is on 2.4.18-pre4.

@@ -146,15 +146,6 @@
        int weight;
 
        /*
-        * select the current process after every other
-        * runnable process, but before the idle thread.
-        * Also, dont trigger a counter recalculation.
-        */
-       weight = -1;
-       if (p->policy & SCHED_YIELD)
-               goto out;
-
-       /*
         * Non-RT process - normal case first.
         */
        if (p->policy == SCHED_OTHER) {

@@ -603,8 +594,15 @@
        /*
         * Default process to select..
         */
-       next = idle_task(this_cpu);
-       c = -1000;
+
+       if (unlikely(prev->policy & SCHED_YIELD)) {
+               next = prev;
+               c = -1;
+               move_last_runqueue(prev);
+       } else {
+               next = idle_task(this_cpu);
+               c = -1000;
+       }
        list_for_each(tmp, &runqueue_head) {
                p = list_entry(tmp, struct task_struct, run_list);
                if (can_schedule(p, this_cpu)) {

@@ -1064,9 +1062,6 @@
                        current->policy |= SCHED_YIELD;
                current->need_resched = 1;
 
-               spin_lock_irq(&runqueue_lock);
-               move_last_runqueue(current);
-               spin_unlock_irq(&runqueue_lock);
        }
        return 0;
 }