Reduce TLB flushing during process migration

Hi Andrew,

Another optimization patch from Jack Steiner, intended to reduce TLB flushes during process migration.

CC'ed Rusty because this patch is applied on top of his cpuhotplug code.

This patch was generated against -rc4 with the rc3-mm1 patch merged on top of it.

mh

Martin Hicks                Wild Open Source Inc.
[email protected]     613-266-2296

# This is a BitKeeper generated patch for the following project:
# Project Name: Linux kernel tree
# This patch format is intended for GNU patch command version 2.5 or higher. # This patch includes the following deltas:
# ChangeSet 1.1680 -> 1.1681
# kernel/sched.c 1.240 -> 1.241
#
# The following is the BitKeeper ChangeSet Log
# --------------------------------------------
# 04/02/17 [email protected] 1.1681
# Process migration optimization.
# --------------------------------------------
#

diff -Nru a/kernel/sched.c b/kernel/sched.c
@@ -25,6 +25,7 @@
 #include <linux/highmem.h>
 #include <linux/smp_lock.h>
 #include <asm/mmu_context.h>
+#include <asm/tlbflush.h>
 #include <linux/interrupt.h>
 #include <linux/completion.h>
 #include <linux/kernel_stat.h>

@@ -1135,6 +1136,14 @@
 		task_rq_unlock(rq, &flags);
 		wake_up_process(rq->migration_thread);
 		wait_for_completion(&req.done);
+
+		/*
+		 * we want a new context here. This eliminates TLB
+		 * flushes on the cpus where the process executed prior to
+		 * the migration.
+		 */
+		flush_tlb_mm(current->mm);
+
 		return;
 	}
 out:

I like this change a lot.

This patch is only applicable to CONFIG_NUMA. Wouldn't SMP systems benefit from the same treatment?

And does this optimisation come with any benchmark results?

Thanks.

Here's some figures from Jack:

---

I dont have a benchmark for JUST the scheduler change, but the entire node history + sched change is:

nwchem on a 128p showed:

Before:
siosi7.sale.56-shm1cs: Time after atomic energies summed: 140.2
siosi7.sale.120-shm1cs: Time after atomic energies summed: 306.9

After:
56p = Time after atomic energies summed: 99.3
120p = Time after atomic energies summed: 110.4

Almost 3X improvement on 120p.

I dont recall how much was due to the sched fix, but I remember that it was significant.

Note that the amount of improvement is highly platform specific.

---

mh

Martin Hicks                Wild Open Source Inc.
[email protected]     613-266-2296

In message <20040217154926.GI12142@localhost> you write:

Actually, the #ifdef around that function is wrong:

#if defined(CONFIG_NUMA) || defined(CONFIG_HOTPLUG_CPU) /* If dest_cpu is allowed for this process, migrate us to it. */ void migrate_to_cpu(int dest_cpu)

We don't use this for HOTPLUG_CPU, but we call __migrate_task() directly.

ie. #ifdef CONFIG_NUMA is sufficient.

I do think this is an obviously good idea though. Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.