Skip to contentSkip to conversation

mm/mmu_notifier: get rid of srcu-based synchronization

RFC PATCH New Konstantin Khlebnikov 1 participant 1 message Jan 21, 2013

Subsystems (3)
Memory management MM core Memory mapping
lore
Share & tools
lore

Konstantin Khlebnikov

author
#0 lore raw

RFC PATCH 1/1 mm/mmu_notifier: get rid of srcu-based synchronization

This patch removes srcu-based protection from mmu-notitifier and collects all synchronization in mmu_notifier_unregister()/mmu_notifier_release().

All mmu notifier methods are called either under mmap_sem or under one of rmap
locks: root anon vma lock or inode mmaping lock. Because they always operates in particular vma and caller must protect it somehow. Thus we can use mmap_sem and rmap locks for waiting for all currently running mmu-notifier methods.

This patch adds new helper function: mm_synchronize_all_locks(). This function acquires and releases all rmap locks one by one, it much faster than sequence mm_take_all_locks() - mm_drop_all_locks(). mm_synchronize_all_locks() needs only mmap_sem read-lock, but caller must lock mmap_sem for write at least once to synchronize with the rest operations which are protected with mmap_sem read-lock.

Signed-off-by Konstantin Khlebnikov <[email protected]>
Cc Andrea Arcangeli <[email protected]>
Cc Andrew Morton <[email protected]>
Cc [email protected]
---
 include/linux/mm.h           |    1 
 include/linux/mmu_notifier.h |    1 
 mm/mmap.c                    |   26 ++++++++++++
 mm/mmu_notifier.c            |   91 ++++++++++++++----------------------------
 4 files changed, 57 insertions(+), 62 deletions(-)
include/linux/mm.h +1 -0
index 66e2f7c..86cf9f3 100644
@@ -1461,6 +1461,7 @@ extern void exit_mmap(struct mm_struct *);
 
 extern int mm_take_all_locks(struct mm_struct *mm);
 extern void mm_drop_all_locks(struct mm_struct *mm);
+extern void mm_synchronize_all_locks(struct mm_struct *mm);
 
 extern void set_mm_exe_file(struct mm_struct *mm, struct file *new_exe_file);
 extern struct file *get_mm_exe_file(struct mm_struct *mm);
include/linux/mmu_notifier.h +0 -1
index bc823c4..fa55d89 100644
@@ -4,7 +4,6 @@
 #include <linux/list.h>
 #include <linux/spinlock.h>
 #include <linux/mm_types.h>
-#include <linux/srcu.h>
 
 struct mmu_notifier;
 struct mmu_notifier_ops;
mm/mmap.c +26 -0
index 0fb6805..89e884e 100644
@@ -3043,6 +3043,32 @@ void mm_drop_all_locks(struct mm_struct *mm)
 }
 
 /*
+ * This function waits for all running pte operations which are protected with
+ * rmap locks like try_to_unmap(). To wait for rest operations like page-faults
+ * which runs under mmap_sem caller must lock mmap_sem for write at least once.
+ * This function itself requires only mmap_sem locked for read.
+ */
+void mm_synchronize_all_locks(struct mm_struct *mm)
+{
+	struct vm_area_struct *vma;
+
+	BUG_ON(down_write_trylock(&mm->mmap_sem));
+
+	for (vma = mm->mmap; vma; vma = vma->vm_next) {
+		if (vma->anon_vma) {
+			anon_vma_lock_write(vma->anon_vma);
+			anon_vma_unlock_write(vma->anon_vma);
+		}
+		if (vma->vm_file && vma->vm_file->f_mapping) {
+			struct address_space *mapping = vma->vm_file->f_mapping;
+
+			mutex_lock(&mapping->i_mmap_mutex);
+			mutex_unlock(&mapping->i_mmap_mutex);
+		}
+	}
+}
+
+/*
  * initialise the VMA slab
  */
 void __init mmap_init(void)
mm/mmu_notifier.c +30 -61

237 lines not on this page · raw message

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>