Skip to contentSkip to conversation

Reduce TLB flushing during process migration

PATCH Manfred Spraul 3 participants 5 messages Feb 17, 2004

lore
Share & tools
lore
#0 lore raw
Quoted 15 lines · Martin

diff -Nru a/kernel/sched.c b/kernel/sched.c

kernel/sched.c +1 -0
@@ -25,6 +25,7 @@
 #include <linux/highmem.h>
 #include <linux/smp_lock.h>
 #include <asm/mmu_context.h>
+#include <asm/tlbflush.h>
 #include <linux/interrupt.h>
 #include <linux/completion.h>
 #include <linux/kernel_stat.h>
@@ -1135,6 +1136,14 @@
 		task_rq_unlock(rq, &flags);
 		wake_up_process(rq->migration_thread);
 		wait_for_completion(&req.done);

kernel/sched.c line 1148

Quoted 10 lines
kernel/sched.c +8 -0
+
+		/*
+		 * we want a new context here. This eliminates TLB
+		 * flushes on the cpus where the process executed prior to
+		 * the migration.
+		 */
+		flush_tlb_mm(current->mm);
+
 
 

I think flush_tlb_mm() is the wrong function - e.g. for i386, it's a wasted flush, because i386 disconnects previous cpus from the tlb flush automatically. And it's always the wrong thing if you've migrated one thread of a task that runs on multiple cpus. I think you need a new hook.

--
Manfred

#1 lore raw
Quoted 5 lines · On Tue, 17 Feb 2004 17:22:14 +0100 Manfred Spraul <[email protected]>…
Quoted 5 lines
(quoted diff) +5 -0
+		 * we want a new context here. This eliminates TLB
+		 * flushes on the cpus where the process executed prior to
+		 * the migration.
+		 */
+		flush_tlb_mm(current->mm);

...

Quoted 5 lines

I think flush_tlb_mm() is the wrong function - e.g. for i386, it's a wasted flush, because i386 disconnects previous cpus from the tlb flush automatically.
And it's always the wrong thing if you've migrated one thread of a task that runs on multiple cpus. I think you need a new hook.

Yes, you're probably right. Just name it tlb_migrate_prepare(mm) or something like that.

I think most if not all non-x86 platforms will define this straight to flush_tlb_mm().

#2 lore raw
Quoted 15 lines · David S. Miller
Quoted 5 lines · On Tue, 17 Feb 2004 17:22:14 +0100 Manfred Spraul <[email protected]>…
Quoted 5 lines
(quoted diff) +5 -0
+		 * we want a new context here. This eliminates TLB
+		 * flushes on the cpus where the process executed prior to
+		 * the migration.
+		 */
+		flush_tlb_mm(current->mm);

...

Quoted 5 lines

I think flush_tlb_mm() is the wrong function - e.g. for i386, it's a wasted flush, because i386 disconnects previous cpus from the tlb flush automatically.
And it's always the wrong thing if you've migrated one thread of a task that runs on multiple cpus. I think you need a new hook.

Quoted 5 lines

Yes, you're probably right. Just name it tlb_migrate_prepare(mm) or something like that.

I think most if not all non-x86 platforms will define this straight to flush_tlb_mm().

Here's an updated patch that creates the new wrapper. I only changed ia64 to call flush_tlb_mm() because I'm a little wary about assuming that all non-x86 arches want this.

Boot tested on ia64, compile tested on x86.

mh

Signature · 2 lines
Martin Hicks                Wild Open Source Inc.
[email protected]     613-266-2296

# This is a BitKeeper generated patch for the following project:
# Project Name: Linux kernel tree
# This patch format is intended for GNU patch command version 2.5 or higher. # This patch includes the following deltas:
# ChangeSet 1.1680 -> 1.1681
# include/asm-generic/tlb.h 1.20 -> 1.21
# include/asm-ia64/tlb.h 1.18 -> 1.19
# kernel/sched.c 1.240 -> 1.241
#
# The following is the BitKeeper ChangeSet Log
# --------------------------------------------
# 04/02/17 [email protected] 1.1681
# Add tlb_migrate_prepare(). Most arches will define this to tlb_flush_mm().

# x86 is an exception.  For now make the default to do nothing.  Let each
# arch define tlb_migrate_prepare appropriately.
# --------------------------------------------
#
diff -Nru a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h
--- a/include/asm-generic/tlb.h	Tue Feb 17 11:31:56 2004
+++ b/include/asm-generic/tlb.h	Tue Feb 17 11:31:56 2004
@@ -146,4 +146,6 @@
 		__pmd_free_tlb(tlb, pmdp);			\
 	} while (0)

+#define tlb_migrate_prepare(mm)
+
#endif /* _ASM_GENERIC__TLB_H */

include/asm-ia64/tlb.h +2 -0
diff -Nru a/include/asm-ia64/tlb.h b/include/asm-ia64/tlb.h
@@ -210,6 +210,8 @@
 	tlb->end_addr = address + PAGE_SIZE;
 }
 
+#define tlb_migrate_prepare(mm) flush_tlb_mm(mm)
+
 #define tlb_start_vma(tlb, vma)			do { } while (0)
 #define tlb_end_vma(tlb, vma)			do { } while (0)
 
kernel/sched.c +9 -0
diff -Nru a/kernel/sched.c b/kernel/sched.c
@@ -25,6 +25,7 @@
 #include <linux/highmem.h>
 #include <linux/smp_lock.h>
 #include <asm/mmu_context.h>
+#include <asm/tlb.h>
 #include <linux/interrupt.h>
 #include <linux/completion.h>
 #include <linux/kernel_stat.h>
@@ -1135,6 +1136,14 @@
 		task_rq_unlock(rq, &flags);
 		wake_up_process(rq->migration_thread);
 		wait_for_completion(&req.done);
+
+		/*
+		 * we want a new context here. This eliminates TLB
+		 * flushes on the cpus where the process executed prior to
+		 * the migration.
+		 */
+		tlb_migrate_prepare(current->mm);
+
 		return;
 	}
 out:
#3 lore raw
Quoted 3 lines · On Tue, 17 Feb 2004 15:05:58 -0500 Martin Hicks <[email protected]> wrote:

Here's an updated patch that creates the new wrapper. I only changed ia64 to call flush_tlb_mm() because I'm a little wary about assuming that all non-x86 arches want this.

Please use "do { } while (0)" for the NOP define of the macro else we'll end up with empty-statement warnings from the compiler.

#4 lore raw
Quoted 9 lines · David S. Miller
Quoted 3 lines · On Tue, 17 Feb 2004 15:05:58 -0500 Martin Hicks <[email protected]> wrote:

Here's an updated patch that creates the new wrapper. I only changed ia64 to call flush_tlb_mm() because I'm a little wary about assuming that all non-x86 arches want this.

Please use "do { } while (0)" for the NOP define of the macro else we'll end up with empty-statement warnings from the compiler.

Certainly.

Andrew: Are you willing to pickup this change?

mh

Signature · 2 lines
Martin Hicks                Wild Open Source Inc.
[email protected]     613-266-2296

# This is a BitKeeper generated patch for the following project:
# Project Name: Linux kernel tree
# This patch format is intended for GNU patch command version 2.5 or higher. # This patch includes the following deltas:
# ChangeSet 1.1680 -> 1.1681
# include/asm-generic/tlb.h 1.20 -> 1.22
# include/asm-ia64/tlb.h 1.18 -> 1.19
# kernel/sched.c 1.240 -> 1.241
#
# The following is the BitKeeper ChangeSet Log
# --------------------------------------------
# 04/02/17 [email protected] 1.1681
# Add tlb_migrate_prepare(). Most arches will define this to tlb_flush_mm().

# x86 is an exception.  For now make the default to do nothing.  Let each
# arch define tlb_migrate_prepare appropriately.
# --------------------------------------------
#
diff -Nru a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h
--- a/include/asm-generic/tlb.h	Tue Feb 17 12:32:18 2004
+++ b/include/asm-generic/tlb.h	Tue Feb 17 12:32:18 2004
@@ -146,4 +146,6 @@
 		__pmd_free_tlb(tlb, pmdp);			\
 	} while (0)

+#define tlb_migrate_prepare(mm) do { } while(0)
+
#endif /* _ASM_GENERIC__TLB_H */

include/asm-ia64/tlb.h +2 -0
diff -Nru a/include/asm-ia64/tlb.h b/include/asm-ia64/tlb.h
@@ -210,6 +210,8 @@
 	tlb->end_addr = address + PAGE_SIZE;
 }
 
+#define tlb_migrate_prepare(mm) flush_tlb_mm(mm)
+
 #define tlb_start_vma(tlb, vma)			do { } while (0)
 #define tlb_end_vma(tlb, vma)			do { } while (0)
 
kernel/sched.c +9 -0
diff -Nru a/kernel/sched.c b/kernel/sched.c
@@ -25,6 +25,7 @@
 #include <linux/highmem.h>
 #include <linux/smp_lock.h>
 #include <asm/mmu_context.h>
+#include <asm/tlb.h>
 #include <linux/interrupt.h>
 #include <linux/completion.h>
 #include <linux/kernel_stat.h>
@@ -1135,6 +1136,14 @@
 		task_rq_unlock(rq, &flags);
 		wake_up_process(rq->migration_thread);
 		wait_for_completion(&req.done);
+
+		/*
+		 * we want a new context here. This eliminates TLB
+		 * flushes on the cpus where the process executed prior to
+		 * the migration.
+		 */
+		tlb_migrate_prepare(current->mm);
+
 		return;
 	}
 out: