Reduce TLB flushing during process migration
Quoted 15 lines · Martin
diff -Nru
a/kernel/sched.cb/kernel/sched.ckernel/sched.c +1
-0@@ -25,6 +25,7 @@#include <linux/highmem.h> #include <linux/smp_lock.h> #include <asm/mmu_context.h> +#include <asm/tlbflush.h> #include <linux/interrupt.h> #include <linux/completion.h> #include <linux/kernel_stat.h> @@ -1135,6 +1136,14 @@task_rq_unlock (rq, &flags);wake_up_process (rq->migration_thread);wait_for_completion (&req.done);
kernel/sched.c line 1148
Quoted 10 lines
kernel/sched.c +8
-0+ +/* +* we want a new context here. This eliminates TLB +* flushes on the cpus where the process executed prior to +* the migration. +*/ +flush_tlb_mm (current->mm); +
I think flush_tlb_mm() is the wrong function - e.g. for i386, it's a wasted flush, because i386 disconnects previous cpus from the tlb flush automatically. And it's always the wrong thing if you've migrated one thread of a task that runs on multiple cpus. I think you need a new hook.
--
Manfred
Quoted 5 lines · On Tue, 17 Feb 2004 17:22:14 +0100 Manfred Spraul <[email protected]>…
Quoted 5 lines
(quoted diff) +5
-0+ * we want a new context here. This eliminates TLB + * flushes on the cpus where the process executed prior to + * the migration. + */ + flush_tlb_mm(current->mm);
...
Quoted 5 lines
I think
flush_tlb_mm()is the wrong function - e.g. for i386, it's a wasted flush, because i386 disconnects previous cpus from the tlb flush automatically.
And it's always the wrong thing if you've migrated one thread of a task that runs on multiple cpus. I think you need a new hook.
Yes, you're probably right. Just name it tlb_migrate_prepare(mm) or something like that.
I think most if not all non-x86 platforms will define this straight to flush_tlb_mm().
Quoted 15 lines · David S. Miller
Quoted 5 lines · On Tue, 17 Feb 2004 17:22:14 +0100 Manfred Spraul <[email protected]>…
Quoted 5 lines
(quoted diff) +5
-0+ * we want a new context here. This eliminates TLB + * flushes on the cpus where the process executed prior to + * the migration. + */ + flush_tlb_mm(current->mm);...
Quoted 5 lines
I think
flush_tlb_mm()is the wrong function - e.g. for i386, it's a wasted flush, because i386 disconnects previous cpus from the tlb flush automatically.
And it's always the wrong thing if you've migrated one thread of a task that runs on multiple cpus. I think you need a new hook.
Quoted 5 lines
Yes, you're probably right. Just name it
tlb_migrate_prepare(mm)or something like that.I think most if not all non-x86 platforms will define this straight to
flush_tlb_mm().
Here's an updated patch that creates the new wrapper. I only changed ia64 to call flush_tlb_mm() because I'm a little wary about assuming that all non-x86 arches want this.
Boot tested on ia64, compile tested on x86.
mh
Signature · 2 lines
Martin Hicks Wild Open Source Inc. [email protected] 613-266-2296
# This is a BitKeeper generated patch for the following project:
# Project Name: Linux kernel tree
# This patch format is intended for GNU patch command version 2.5 or higher. # This patch includes the following deltas:
# ChangeSet 1.1680 -> 1.1681
# include/asm-generic/tlb.h 1.20 -> 1.21
# include/asm-ia64/tlb.h 1.18 -> 1.19
# kernel/sched.c 1.240 -> 1.241
#
# The following is the BitKeeper ChangeSet Log
# --------------------------------------------
# 04/02/17 [email protected] 1.1681
# Add tlb_migrate_prepare(). Most arches will define this to tlb_flush_mm().
# x86 is an exception. For now make the default to do nothing. Let each # arch define tlb_migrate_prepare appropriately. # -------------------------------------------- # diff -Nru a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h --- a/include/asm-generic/tlb.h Tue Feb 17 11:31:56 2004 +++ b/include/asm-generic/tlb.h Tue Feb 17 11:31:56 2004 @@ -146,4 +146,6 @@ __pmd_free_tlb(tlb, pmdp); \ } while (0)
+#define tlb_migrate_prepare(mm)
+
#endif /* _ASM_GENERIC__TLB_H */
include/asm-ia64/tlb.h +2 -0
@@ -210,6 +210,8 @@ tlb->end_addr = address +PAGE_SIZE ; } +#define tlb_migrate_prepare (mm) flush_tlb_mm(mm) +#define tlb_start_vma (tlb, vma) do { } while (0)#define tlb_end_vma (tlb, vma) do { } while (0)
kernel/sched.c +9 -0
@@ -25,6 +25,7 @@#include <linux/highmem.h> #include <linux/smp_lock.h> #include <asm/mmu_context.h> +#include <asm/tlb.h> #include <linux/interrupt.h> #include <linux/completion.h> #include <linux/kernel_stat.h>
@@ -1135,6 +1136,14 @@task_rq_unlock (rq, &flags);wake_up_process (rq->migration_thread);wait_for_completion (&req.done); + +/* +* we want a new context here. This eliminates TLB +* flushes on the cpus where the process executed prior to +* the migration. +*/ +tlb_migrate_prepare (current->mm); +return ; } out:
Quoted 3 lines · On Tue, 17 Feb 2004 15:05:58 -0500 Martin Hicks <[email protected]> wrote:
Here's an updated patch that creates the new wrapper. I only changed ia64 to call
flush_tlb_mm()because I'm a little wary about assuming that all non-x86 arches want this.
Please use "do { } while (0)" for the NOP define of the macro else we'll end up with empty-statement warnings from the compiler.
Quoted 9 lines · David S. Miller
Quoted 3 lines · On Tue, 17 Feb 2004 15:05:58 -0500 Martin Hicks <[email protected]> wrote:
Here's an updated patch that creates the new wrapper. I only changed ia64 to call
flush_tlb_mm()because I'm a little wary about assuming that all non-x86 arches want this.Please use "do { } while (0)" for the NOP define of the macro else we'll end up with empty-statement warnings from the compiler.
Certainly.
Andrew: Are you willing to pickup this change?
mh
Signature · 2 lines
Martin Hicks Wild Open Source Inc. [email protected] 613-266-2296
# This is a BitKeeper generated patch for the following project:
# Project Name: Linux kernel tree
# This patch format is intended for GNU patch command version 2.5 or higher. # This patch includes the following deltas:
# ChangeSet 1.1680 -> 1.1681
# include/asm-generic/tlb.h 1.20 -> 1.22
# include/asm-ia64/tlb.h 1.18 -> 1.19
# kernel/sched.c 1.240 -> 1.241
#
# The following is the BitKeeper ChangeSet Log
# --------------------------------------------
# 04/02/17 [email protected] 1.1681
# Add tlb_migrate_prepare(). Most arches will define this to tlb_flush_mm().
# x86 is an exception. For now make the default to do nothing. Let each # arch define tlb_migrate_prepare appropriately. # -------------------------------------------- # diff -Nru a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h --- a/include/asm-generic/tlb.h Tue Feb 17 12:32:18 2004 +++ b/include/asm-generic/tlb.h Tue Feb 17 12:32:18 2004 @@ -146,4 +146,6 @@ __pmd_free_tlb(tlb, pmdp); \ } while (0)
+#define tlb_migrate_prepare(mm) do { } while(0)
+
#endif /* _ASM_GENERIC__TLB_H */
include/asm-ia64/tlb.h +2 -0
@@ -210,6 +210,8 @@ tlb->end_addr = address +PAGE_SIZE ; } +#define tlb_migrate_prepare (mm) flush_tlb_mm(mm) +#define tlb_start_vma (tlb, vma) do { } while (0)#define tlb_end_vma (tlb, vma) do { } while (0)
kernel/sched.c +9 -0
@@ -25,6 +25,7 @@#include <linux/highmem.h> #include <linux/smp_lock.h> #include <asm/mmu_context.h> +#include <asm/tlb.h> #include <linux/interrupt.h> #include <linux/completion.h> #include <linux/kernel_stat.h>
@@ -1135,6 +1136,14 @@task_rq_unlock (rq, &flags);wake_up_process (rq->migration_thread);wait_for_completion (&req.done); + +/* +* we want a new context here. This eliminates TLB +* flushes on the cpus where the process executed prior to +* the migration. +*/ +tlb_migrate_prepare (current->mm); +return ; } out: