Skip to contentSkip to conversation

ksm: write protect pages from inside ksm

PATCH In review Izik Eidus 2 participants 12 messages Jun 12, 2009 – Jun 15, 2009

Subsystems (7)
lore
Share & tools
lore

Izik Eidus

author
#0 lore raw

cover ksm: write protect pages from inside ksm

Hugh, so untill here we are sync,

Question is what you want me to do now?,
(Beacuse we are skipping 2.6.31, It is ok to you to tell me something
like: "Shut up and let me see what i can get with this madvise" - that from one side.
From another side if you want me to do anything please say.

Thanks.

Izik Eidus (3):
ksm: remove ksm from being a module.
ksm: remove page_wrprotect() from rmap.c
withdraw ksm-add-page_wrprotect-write-protecting-page.patch

 include/linux/mm.h   |    2 +-
 include/linux/rmap.h |   12 ----
 mm/Kconfig           |    5 +-
 mm/ksm.c             |   92 +++++++++++++++++++++++----------
 mm/memory.c          |    2 +-
 mm/rmap.c            |  139 --------------------------------------------------
 6 files changed, 69 insertions(+), 183 deletions(-)
author
#1 lore raw

PATCH 1/3 ksm: remove ksm from being a module.

---
 include/linux/mm.h   |    2 +-
 include/linux/rmap.h |    4 ++--
 mm/Kconfig           |    5 ++---
 mm/memory.c          |    2 +-
 mm/rmap.c            |    2 +-
 5 files changed, 7 insertions(+), 8 deletions(-)
include/linux/mm.h +1 -1
index e617bab..c1259bf 100644
@@ -1258,7 +1258,7 @@ int vm_insert_pfn(struct vm_area_struct *vma, unsigned long addr,
 int vm_insert_mixed(struct vm_area_struct *vma, unsigned long addr,
 			unsigned long pfn);
 
-#if defined(CONFIG_KSM) || defined(CONFIG_KSM_MODULE)
+#ifdef CONFIG_KSM
 int replace_page(struct vm_area_struct *vma, struct page *oldpage,
 		 struct page *newpage, pte_t orig_pte, pgprot_t prot);
 #endif
include/linux/rmap.h +2 -2
index 469376d..8b98536 100644
@@ -118,7 +118,7 @@ static inline int try_to_munlock(struct page *page)
 }
 #endif
 
-#if defined(CONFIG_KSM) || defined(CONFIG_KSM_MODULE)
+#ifdef CONFIG_KSM
 int page_wrprotect(struct page *page, int *odirect_sync, int count_offset);
 #endif
 
@@ -136,7 +136,7 @@ static inline int page_mkclean(struct page *page)
 	return 0;
 }
 
-#if defined(CONFIG_KSM) || defined(CONFIG_KSM_MODULE)
+#ifdef CONFIG_KSM
 static inline int page_wrprotect(struct page *page, int *odirect_sync,
 				 int count_offset)
 {
mm/Kconfig +2 -3
index 5ebfd18..e7c118f 100644
@@ -227,10 +227,9 @@ config MMU_NOTIFIER
 	bool
 
 config KSM
-	tristate "Enable KSM for page sharing"
+	bool "Enable KSM for page sharing"
 	help
-	  Enable the KSM kernel module to allow page sharing of equal pages
-	  among different tasks.
+	  Enable KSM to allow page sharing of equal pages among different tasks.
 
 config NOMMU_INITIAL_TRIM_EXCESS
 	int "Turn on mmap() excess space trimming before booting"
mm/memory.c +1 -1
index 8b4e40e..203bbd0 100644
@@ -1617,7 +1617,7 @@ int vm_insert_mixed(struct vm_area_struct *vma, unsigned long addr,
 }
 EXPORT_SYMBOL(vm_insert_mixed);
 
-#if defined(CONFIG_KSM) || defined(CONFIG_KSM_MODULE)
+#ifdef CONFIG_KSM
 
 /**
  * replace_page - replace page in vma with new page
mm/rmap.c +1 -1
index f53074c..34a2029 100644
@@ -585,7 +585,7 @@ int page_mkclean(struct page *page)
 }
 EXPORT_SYMBOL_GPL(page_mkclean);
 
-#if defined(CONFIG_KSM) || defined(CONFIG_KSM_MODULE)
+#ifdef CONFIG_KSM
 
 static int page_wrprotect_one(struct page *page, struct vm_area_struct *vma,
 			      int *odirect_sync, int count_offset)
Signature · 1 line
1.5.6.5
author
#2 lore raw

PATCH 2/3 ksm: remove page_wrprotect() from rmap.c

Remove page_wrprotect() from rmap.c and instead embedded the needed code into ksm.c

Hugh pointed out that for the ksm usage case, we dont have to walk over the rmap and to write protected page after page beacuse when Anonymous page is mapped more than once, it have to be write protected already, and in a case that it mapped just once, no need to walk over the rmap, we can instead write protect it from inside ksm.c.

Thanks.

---
 mm/ksm.c |   92 +++++++++++++++++++++++++++++++++++++++++++------------------
 1 files changed, 65 insertions(+), 27 deletions(-)
mm/ksm.c +65 -27
index 74d921b..3aee221 100644
@@ -37,6 +37,7 @@
 #include <linux/swap.h>
 #include <linux/rbtree.h>
 #include <linux/anon_inodes.h>
+#include <linux/mmu_notifier.h>
 #include <linux/ksm.h>
 
 #include <asm/tlbflush.h>
@@ -643,6 +644,66 @@ static inline int pages_identical(struct page *page1, struct page *page2)
 }
 
 /*
+ * If this anonymous page is mapped only here, its pte may need
+ * to be write-protected, If it`s mapped elsewhere, all its
+ * ptes are necessarily already write-protected. In either
+ * case, we need to lock and check page_count is not raised.
+ */
+static inline int write_protect_page(struct page *page,
+				     struct vm_area_struct *vma,
+				     pte_t *orig_pte)
+{
+	struct mm_struct *mm = vma->vm_mm;
+	unsigned long addr;
+	pte_t *ptep;
+	spinlock_t *ptl;
+	int swapped;
+	int ret = 1;
+
+	addr = addr_in_vma(vma, page);
+	if (addr == -EFAULT)
+		goto out;
+
+	ptep = page_check_address(page, mm, addr, &ptl, 0);
+	if (!ptep)
+		goto out;
+
+	if (pte_write(*ptep)) {
+		pte_t entry;
+
+		swapped = PageSwapCache(page);
+		flush_cache_page(vma, addr, page_to_pfn(page));
+		/*
+		 * Ok this is tricky, when get_user_pages_fast() run it doesnt
+		 * take any lock, therefore the check that we are going to make
+		 * with the pagecount against the mapcount is racey and
+		 * O_DIRECT can happen right after the check.
+		 * So we clear the pte and flush the tlb before the check
+		 * this assure us that no O_DIRECT can happen after the check
+		 * or in the middle of the check.
+		 */
+		entry = ptep_clear_flush(vma, addr, ptep);
+		/*
+		 * Check that no O_DIRECT or similar I/O is in progress on the
+		 * page
+		 */
+		if ((page_mapcount(page) + 2 + swapped) != page_count(page)) {
+			set_pte_at_notify(mm, addr, ptep, entry);
+			goto out_unlock;
+		}
+		entry = pte_wrprotect(entry);
+		set_pte_at_notify(mm, addr, ptep, entry);
+	}
+	*orig_pte = *ptep;
+	ret = 0;
+
+out_unlock:
+	pte_unmap_unlock(ptep, ptl);
+out:
+	return ret;
+}
+
+/*
  * try_to_merge_one_page - take two pages and merge them into one
  * @mm: mm_struct that hold vma pointing into oldpage
  * @vma: the vma that hold the pte pointing into oldpage
@@ -661,9 +722,7 @@ static int try_to_merge_one_page(struct mm_struct *mm,
 				 pgprot_t newprot)
 {
 	int ret = 1;
-	int odirect_sync;
-	unsigned long page_addr_in_vma;
-	pte_t orig_pte, *orig_ptep;
+	pte_t orig_pte = __pte(0);
 
 	if (!PageAnon(oldpage))
 		goto out;
@@ -671,42 +730,21 @@ static int try_to_merge_one_page(struct mm_struct *mm,
 	get_page(newpage);
 	get_page(oldpage);
 
-	page_addr_in_vma = addr_in_vma(vma, oldpage);
-	if (page_addr_in_vma == -EFAULT)
-		goto out_putpage;
-
-	orig_ptep = get_pte(mm, page_addr_in_vma);
-	if (!orig_ptep)
-		goto out_putpage;
-	orig_pte = *orig_ptep;
-	pte_unmap(orig_ptep);
-	if (!pte_present(orig_pte))
-		goto out_putpage;
-	if (page_to_pfn(oldpage) != pte_pfn(orig_pte))
-		goto out_putpage;
 	/*
 	 * we need the page lock to read a stable PageSwapCache in
-	 * page_wrprotect().
+	 * write_protect_page().
 	 * we use trylock_page() instead of lock_page(), beacuse we dont want to
 	 * wait here, we prefer to continue scanning and merging diffrent pages
 	 * and to come back to this page when it is unlocked.
 	 */
 	if (!trylock_page(oldpage))
 		goto out_putpage;
-	/*
-	 * page_wrprotect check if the page is swapped or in swap cache,
-	 * in the future we might want to run here if_present_pte and then
-	 * swap_free
-	 */
-	if (!page_wrprotect(oldpage, &odirect_sync, 2)) {
+
+	if (write_protect_page(oldpage, vma, &orig_pte)) {
 		unlock_page(oldpage);
 		goto out_putpage;
 	}
 	unlock_page(oldpage);
-	if (!odirect_sync)
-		goto out_putpage;
-
-	orig_pte = pte_wrprotect(orig_pte);
 
 	if (pages_identical(oldpage, newpage))
 		ret = replace_page(vma, oldpage, newpage, orig_pte, newprot);
Signature · 1 line
1.5.6.5
author
#3 lore raw

PATCH 3/3 withdraw ksm-add-page_wrprotect-write-protecting-page.patch

Now that ksm have its private write_protect_page(), we dont need this helper function to do the write-protecting work for us.

---
 include/linux/rmap.h |   12 ----
 mm/rmap.c            |  139 --------------------------------------------------
 2 files changed, 0 insertions(+), 151 deletions(-)
include/linux/rmap.h +0 -12
index 8b98536..350e76d 100644
@@ -118,10 +118,6 @@ static inline int try_to_munlock(struct page *page)
 }
 #endif
 
-#ifdef CONFIG_KSM
-int page_wrprotect(struct page *page, int *odirect_sync, int count_offset);
-#endif
-
 #else	/* !CONFIG_MMU */
 
 #define anon_vma_init()		do {} while (0)
@@ -136,14 +132,6 @@ static inline int page_mkclean(struct page *page)
 	return 0;
 }
 
-#ifdef CONFIG_KSM
-static inline int page_wrprotect(struct page *page, int *odirect_sync,
-				 int count_offset)
-{
-	return 0;
-}
-#endif
-
 #endif	/* CONFIG_MMU */
 
 /*
mm/rmap.c +0 -139

145 lines not on this page · raw message

Signature · 1 line
1.5.6.5
#4 lore raw
In reply to Izik Eidus
Quoted 1 line · Izik Eidus

Hugh, so untill here we are sync,

Yes, that fits with what I have here, thanks (or where it didn't quite fit, e.g. ' versus `, I've adjusted to what you have!). And thanks for fixing my *orig_pte = *ptep bug, you did point that out before, but I misunderstood at first.

Quoted 6 lines

Question is what you want me to do now?,
(Beacuse we are skipping 2.6.31, It is ok to you to tell me something
like: "Shut up and let me see what i can get with this madvise" - that from one side.
From another side if you want me to do anything please say.

I had to get a bit further at my end before answering on that, but now the answer is clear: please do some testing of your RFC madvise() version (which is what I'm just tidying up a little), and let me know any bugfixes you find. Try with SLAB or SLUB or SLQB debug on e.g. CONFIG_SLUB=y, CONFIG_SLUB_DEBUG=y and boot option "slub_debug".

I'm finding, whether with your RFC or my tidyup, that kksmd soon oopses in get_next_mmlist (or perhaps find_vma): presumably accessing a vma or mm which already got freed (if you don't have slab debugging on, it's liable to hang instead).

(I've also not seen it actually merging yet: if you register or madvise a large anon area and memset it, the /dev/ksm version would merge all its pages, but I've not seen the madvise version do so yet - though maybe there's something stupidly wrong in my testing, really I'm more worried about the oopses at present.)

Note that mmotm includes a patch of Nick's which adds a function madvise_behavior_valid() - you'll need to add your MADVs into its list to get it to work at all there.

Here's a patch I added a month or so ago, when trying to experiment with KSM on all mms: shouldn't be necessary if your mm refcounting is right, but might help to avoid extra weirdness when things go
wrong: exit_mmap() leaves stale vma pointers around, reckoning that nobody can be interested by now; but maybe KSM might peep so better to tidy them up at least while debugging...

Thanks,
Hugh

mm/mmap.c +8 -0
@@ -2112,6 +2112,14 @@ void exit_mmap(struct mm_struct *mm)
 	tlb_finish_mmu(tlb, 0, end);
 
 	/*
+	 * Make sure get_user_pages() and find_vma() etc. will find nothing:
+	 * this may be necessary for KSM.
+	 */
+	mm->mmap = NULL;
+	mm->mmap_cache = NULL;
+	mm->mm_rb = RB_ROOT;
+
+	/*
 	 * Walk the list again, actually closing and freeing it,
 	 * with preemption enabled, without holding any MM locks.
 	 */
author
#5 lore raw
Quoted 18 lines · Hugh Dickins
Quoted 2 lines · Izik Eidus

Hugh, so untill here we are sync,

Yes, that fits with what I have here, thanks (or where it didn't quite fit, e.g. ' versus `, I've adjusted to what you have!). And thanks for fixing my *orig_pte = *ptep bug, you did point that out before, but I misunderstood at first.

Quoted 6 lines

Question is what you want me to do now?,
(Beacuse we are skipping 2.6.31, It is ok to you to tell me something
like: "Shut up and let me see what i can get with this madvise" - that from one side.
From another side if you want me to do anything please say.

Quoted 7 lines

I had to get a bit further at my end before answering on that, but now the answer is clear: please do some testing of your RFC madvise() version (which is what I'm just tidying up a little), and let me know any bugfixes you find. Try with SLAB or SLUB or SLQB debug on e.g. CONFIG_SLUB=y, CONFIG_SLUB_DEBUG=y and boot option "slub_debug".

Sure, let me check it.
(You do have Andrea patch that fix the "used after free slab entries" ?)

Quoted 43 lines

I'm finding, whether with your RFC or my tidyup, that kksmd soon oopses in get_next_mmlist (or perhaps find_vma): presumably accessing a vma or mm which already got freed (if you don't have slab debugging on, it's liable to hang instead).

(I've also not seen it actually merging yet: if you register or madvise a large anon area and memset it, the /dev/ksm version would merge all its pages, but I've not seen the madvise version do so yet - though maybe there's something stupidly wrong in my testing, really I'm more worried about the oopses at present.)

Note that mmotm includes a patch of Nick's which adds a function madvise_behavior_valid() - you'll need to add your MADVs into its list to get it to work at all there.

Here's a patch I added a month or so ago, when trying to experiment with KSM on all mms: shouldn't be necessary if your mm refcounting is right, but might help to avoid extra weirdness when things go
wrong: exit_mmap() leaves stale vma pointers around, reckoning that nobody can be interested by now; but maybe KSM might peep so better to tidy them up at least while debugging...

Thanks,
Hugh

mm/mmap.c +8 -0
@@ -2112,6 +2112,14 @@ void exit_mmap(struct mm_struct *mm)
 	tlb_finish_mmu(tlb, 0, end);
 
 	/*
+	 * Make sure get_user_pages() and find_vma() etc. will find nothing:
+	 * this may be necessary for KSM.
+	 */
+	mm->mmap = NULL;
+	mm->mmap_cache = NULL;
+	mm->mm_rb = RB_ROOT;
+
+	/*
 	 * Walk the list again, actually closing and freeing it,
 	 * with preemption enabled, without holding any MM locks.
 	 */
author
#6 lore raw
Quoted 27 lines · Izik Eidus
Quoted 18 lines · Hugh Dickins
Quoted 2 lines · Izik Eidus

Hugh, so untill here we are sync,

Yes, that fits with what I have here, thanks (or where it didn't quite fit, e.g. ' versus `, I've adjusted to what you have!). And thanks for fixing my *orig_pte = *ptep bug, you did point that out before, but I misunderstood at first.

Quoted 6 lines

Question is what you want me to do now?,
(Beacuse we are skipping 2.6.31, It is ok to you to tell me something
like: "Shut up and let me see what i can get with this madvise" - that from one side.
From another side if you want me to do anything please say.

Quoted 7 lines

I had to get a bit further at my end before answering on that, but now the answer is clear: please do some testing of your RFC madvise() version (which is what I'm just tidying up a little), and let me know any bugfixes you find. Try with SLAB or SLUB or SLQB debug on e.g. CONFIG_SLUB=y, CONFIG_SLUB_DEBUG=y and boot option "slub_debug".

Quoted 2 lines

Sure, let me check it.
(You do have Andrea patch that fix the "used after free slab entries" ?)

How fast is it crush opps to you?, I compiled it and ran it here on 2.6.30-rc4-mm1 with:
"Enable SLQB debugging support" and "SLQB debugging on by default, and it run and merge (i am using qemu processes to run virtual machines to merge the pages between them)

("SLQB debugging on by defaul" mean i dont have to add boot pareameter right?)

Maybe i should try update into newer version of the mm tree? (last commit here is Jul 22)

Quoted 47 lines
Quoted 45 lines

I'm finding, whether with your RFC or my tidyup, that kksmd soon oopses in get_next_mmlist (or perhaps find_vma): presumably accessing a vma or mm which already got freed (if you don't have slab debugging on, it's liable to hang instead).

(I've also not seen it actually merging yet: if you register or madvise a large anon area and memset it, the /dev/ksm version would merge all its pages, but I've not seen the madvise version do so yet - though maybe there's something stupidly wrong in my testing, really I'm more worried about the oopses at present.)

Note that mmotm includes a patch of Nick's which adds a function madvise_behavior_valid() - you'll need to add your MADVs into its list to get it to work at all there.

Here's a patch I added a month or so ago, when trying to experiment with KSM on all mms: shouldn't be necessary if your mm refcounting is right, but might help to avoid extra weirdness when things go
wrong: exit_mmap() leaves stale vma pointers around, reckoning that nobody can be interested by now; but maybe KSM might peep so better to tidy them up at least while debugging...

Thanks,
Hugh

mm/mmap.c 2009-05-03 11:34:47.000000000 +0100 renamed from mm/mmap.c 2009-05-01 13:47:45.000000000 +0100 +1 -0
@@ -2112,6 +2112,14 @@ void exit_mmap(struct mm_struct *mm)
     tlb_finish_mmu(tlb, 0, end);
 
     /*
+     * Make sure get_user_pages() and find_vma() etc. will find 

nothing:

mm/mmap.c 2009-05-03 11:34:47.000000000 +0100 renamed from mm/mmap.c 2009-05-01 13:47:45.000000000 +0100 +7 -0
+     * this may be necessary for KSM.
+     */
+    mm->mmap = NULL;
+    mm->mmap_cache = NULL;
+    mm->mm_rb = RB_ROOT;
+
+    /*
      * Walk the list again, actually closing and freeing it,
      * with preemption enabled, without holding any MM locks.
      */
author
#7 lore raw
Quoted 31 lines · Izik Eidus
Quoted 27 lines · Izik Eidus
Quoted 18 lines · Hugh Dickins
Quoted 2 lines · Izik Eidus

Hugh, so untill here we are sync,

Yes, that fits with what I have here, thanks (or where it didn't quite fit, e.g. ' versus `, I've adjusted to what you have!). And thanks for fixing my *orig_pte = *ptep bug, you did point that out before, but I misunderstood at first.

Quoted 6 lines

Question is what you want me to do now?,
(Beacuse we are skipping 2.6.31, It is ok to you to tell me something
like: "Shut up and let me see what i can get with this madvise" - that from one side.
From another side if you want me to do anything please say.

Quoted 7 lines

I had to get a bit further at my end before answering on that, but now the answer is clear: please do some testing of your RFC madvise() version (which is what I'm just tidying up a little), and let me know any bugfixes you find. Try with SLAB or SLUB or SLQB debug on e.g. CONFIG_SLUB=y, CONFIG_SLUB_DEBUG=y and boot option "slub_debug".

Quoted 2 lines

Sure, let me check it.
(You do have Andrea patch that fix the "used after free slab entries" ?)

Quoted 6 lines

How fast is it crush opps to you?, I compiled it and ran it here on 2.6.30-rc4-mm1 with:
"Enable SLQB debugging support" and "SLQB debugging on by default, and it run and merge (i am using qemu processes to run virtual machines to merge the pages between them)

("SLQB debugging on by defaul" mean i dont have to add boot pareameter right?)

Maybe i should try update into newer version of the mm tree? (last commit here is Jul 22)

OK, bug on my side, just got that oppss, will try to fix and send patch.

(Sorry for the noise)

Quoted 50 lines
Quoted 48 lines
Quoted 46 lines

I'm finding, whether with your RFC or my tidyup, that kksmd soon oopses in get_next_mmlist (or perhaps find_vma): presumably accessing a vma or mm which already got freed (if you don't have slab debugging on, it's liable to hang instead).

(I've also not seen it actually merging yet: if you register or madvise a large anon area and memset it, the /dev/ksm version would merge all its pages, but I've not seen the madvise version do so yet - though maybe there's something stupidly wrong in my testing, really I'm more worried about the oopses at present.)

Note that mmotm includes a patch of Nick's which adds a function madvise_behavior_valid() - you'll need to add your MADVs into its list to get it to work at all there.

Here's a patch I added a month or so ago, when trying to experiment with KSM on all mms: shouldn't be necessary if your mm refcounting is right, but might help to avoid extra weirdness when things go
wrong: exit_mmap() leaves stale vma pointers around, reckoning that nobody can be interested by now; but maybe KSM might peep so better to tidy them up at least while debugging...

Thanks,
Hugh

mm/mmap.c 2009-05-03 11:34:47.000000000 +0100 renamed from mm/mmap.c 2009-05-01 13:47:45.000000000 +0100 +1 -0
@@ -2112,6 +2112,14 @@ void exit_mmap(struct mm_struct *mm)
     tlb_finish_mmu(tlb, 0, end);
 
     /*
+     * Make sure get_user_pages() and find_vma() etc. will find 

nothing:

mm/mmap.c 2009-05-03 11:34:47.000000000 +0100 renamed from mm/mmap.c 2009-05-01 13:47:45.000000000 +0100 +7 -0
+     * this may be necessary for KSM.
+     */
+    mm->mmap = NULL;
+    mm->mmap_cache = NULL;
+    mm->mm_rb = RB_ROOT;
+
+    /*
      * Walk the list again, actually closing and freeing it,
      * with preemption enabled, without holding any MM locks.
      */
author
#8 lore raw
Quoted 42 lines · On Mon, 15 Jun 2009 03:05:14 +0300 Izik Eidus <[email protected]> wrote:
Quoted 41 lines · Izik Eidus
Quoted 27 lines · Izik Eidus
Quoted 18 lines · Hugh Dickins
Quoted 2 lines · Izik Eidus

Hugh, so untill here we are sync,

Yes, that fits with what I have here, thanks (or where it didn't quite fit, e.g. ' versus `, I've adjusted to what you have!). And thanks for fixing my *orig_pte = *ptep bug, you did point that out before, but I misunderstood at first.

Quoted 6 lines

Question is what you want me to do now?,
(Beacuse we are skipping 2.6.31, It is ok to you to tell me something like: "Shut up and let me see what i can get with this madvise" - that from one side.
From another side if you want me to do anything please say.

Quoted 7 lines

I had to get a bit further at my end before answering on that, but now the answer is clear: please do some testing of your RFC madvise() version (which is what I'm just tidying up a little), and let me know any bugfixes you find. Try with SLAB or SLUB or SLQB debug on e.g. CONFIG_SLUB=y, CONFIG_SLUB_DEBUG=y and boot option "slub_debug".

Quoted 3 lines

Sure, let me check it.
(You do have Andrea patch that fix the "used after free slab entries" ?)

How fast is it crush opps to you?, I compiled it and ran it here on 2.6.30-rc4-mm1 with:
"Enable SLQB debugging support" and "SLQB debugging on by default, and it run and merge (i am using qemu processes to run virtual machines to merge the pages between them)

("SLQB debugging on by defaul" mean i dont have to add boot pareameter right?)

Quoted 61 lines
Quoted 2 lines

Maybe i should try update into newer version of the mm tree? (last commit here is Jul 22)

OK, bug on my side, just got that oppss, will try to fix and send patch.

(Sorry for the noise)

Quoted 51 lines
Quoted 49 lines
Quoted 47 lines

I'm finding, whether with your RFC or my tidyup, that kksmd soon oopses in get_next_mmlist (or perhaps find_vma): presumably accessing a vma or mm which already got freed (if you don't have slab debugging on, it's liable to hang instead).

(I've also not seen it actually merging yet: if you register or madvise a large anon area and memset it, the /dev/ksm version would merge all its pages, but I've not seen the madvise version do so yet - though maybe there's something stupidly wrong in my testing, really I'm more worried about the oopses at present.)

Note that mmotm includes a patch of Nick's which adds a function madvise_behavior_valid() - you'll need to add your MADVs into its list to get it to work at all there.

Here's a patch I added a month or so ago, when trying to experiment with KSM on all mms: shouldn't be necessary if your mm refcounting is right, but might help to avoid extra weirdness when things go wrong: exit_mmap() leaves stale vma pointers around, reckoning that nobody can be interested by now; but maybe KSM might peep so better to tidy them up at least while debugging...

Thanks,
Hugh

mm/mmap.c 2009-05-03 11:34:47.000000000 +0100 renamed from mm/mmap.c 2009-05-01 13:47:45.000000000 +0100 +1 -0
@@ -2112,6 +2112,14 @@ void exit_mmap(struct mm_struct *mm)
     tlb_finish_mmu(tlb, 0, end);
 
     /*
+     * Make sure get_user_pages() and find_vma() etc. will find 

nothing:

mm/mmap.c 2009-05-03 11:34:47.000000000 +0100 renamed from mm/mmap.c 2009-05-01 13:47:45.000000000 +0100 +7 -0
+     * this may be necessary for KSM.
+     */
+    mm->mmap = NULL;
+    mm->mmap_cache = NULL;
+    mm->mm_rb = RB_ROOT;
+
+    /*
      * Walk the list again, actually closing and freeing it,
      * with preemption enabled, without holding any MM locks.
      */

Ok, below is ugly fix for the opss..

Quoted 1 line

From 3be1ad5a9f990113e8849fa1e74c4e74066af131 Mon Sep 17 00:00:00 2001

From: Izik Eidus <[email protected]>
Date: Mon, 15 Jun 2009 03:52:05 +0300
Subject: [PATCH] ksm: madvise-rfc: really ugly fix for the oppss bug.

This patch is just so it can run without to crush with the madvise rfc patch.

True fix for this i think is adding another list for ksm inside the mm struct. In the meanwhile i will try to think about other way how to fix this bug.

Hugh, i hope at least now you will be able to run it without it will crush to you.

---
 kernel/fork.c |   11 ++++++-----
 1 files changed, 6 insertions(+), 5 deletions(-)
kernel/fork.c +6 -5
index e5ef58c..771b89a 100644
@@ -484,17 +484,18 @@ void mmput(struct mm_struct *mm)
 {
 	might_sleep();
 
+	spin_lock(&mmlist_lock);
 	if (atomic_dec_and_test(&mm->mm_users)) {
+		if (!list_empty(&mm->mmlist))
+			list_del(&mm->mmlist);
+		spin_unlock(&mmlist_lock);
 		exit_aio(mm);
 		exit_mmap(mm);
 		set_mm_exe_file(mm, NULL);
-		if (!list_empty(&mm->mmlist)) {
-			spin_lock(&mmlist_lock);
-			list_del(&mm->mmlist);
-			spin_unlock(&mmlist_lock);
-		}
 		put_swap_token(mm);
 		mmdrop(mm);
+	} else {
+		spin_unlock(&mmlist_lock);
 	}
 }
 EXPORT_SYMBOL_GPL(mmput);
Signature · 1 line
1.5.6.5
#9 lore raw
Quoted 5 lines · Izik Eidus
Quoted 4 lines · On Mon, 15 Jun 2009 03:05:14 +0300 Izik Eidus <[email protected]> wrote:
Quoted 4 lines
Quoted 4 lines

Sure, let me check it.
(You do have Andrea patch that fix the "used after free slab entries" ?)

I do, yes, and the other fix he posted at the same time.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

How fast is it crush opps to you?, I compiled it and ran it here on 2.6.30-rc4-mm1 with:
"Enable SLQB debugging support" and "SLQB debugging on by default, and it run and merge (i am using qemu processes to run virtual machines to merge the pages between them)

I suspect too much of your testing has been on the sensible use of KSM, not enough on foolish abuse of it!

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

("SLQB debugging on by defaul" mean i dont have to add boot pareameter right?)

That's right. (I never switch it on by default in the .config, just love that we can switch it on by boot option when needed.)

Quoted 2 lines
Quoted 2 lines

OK, bug on my side, just got that oppss, will try to fix and send patch.

Thanks a lot for getting there so quickly.

Quoted 1 line

Ok, below is ugly fix for the opss..

Right, for several reasons* not a patch we'd like to include in the end, but enough to show that indeed fixes the problems I was seeing:
well caught.

[* We don't want to serialize everyone on a global lock there. You could make it a little better using atomic_dec_and_lock(), but still unpleasant. And that placing of the list_del() is bad for swapping:
I ended up using list_del_init() there, and restoring the check after calling set_mm_exe_file(). But never mind, I've a better fix.]

Quoted 10 lines

From 3be1ad5a9f990113e8849fa1e74c4e74066af131 Mon Sep 17 00:00:00 2001
From: Izik Eidus <[email protected]>
Date: Mon, 15 Jun 2009 03:52:05 +0300
Subject: [PATCH] ksm: madvise-rfc: really ugly fix for the oppss bug.

This patch is just so it can run without to crush with the madvise rfc patch.

True fix for this i think is adding another list for ksm inside the mm struct. In the meanwhile i will try to think about other way how to fix this bug.

I'm already working with a separate list for KSM inside the mm struct (prefer not to be mixing these two things up), but I don't see how that helps.

Quoted 3 lines

Hugh, i hope at least now you will be able to run it without it will crush to you.

Yes, thanks, and helped me to think of a better fix (I hope!) below...

Quoted 24 lines
---
 kernel/fork.c |   11 ++++++-----
 1 files changed, 6 insertions(+), 5 deletions(-)
kernel/fork.c +4 -2
index e5ef58c..771b89a 100644
@@ -484,17 +484,18 @@ void mmput(struct mm_struct *mm)
 {
 	might_sleep();
 
+	spin_lock(&mmlist_lock);
 	if (atomic_dec_and_test(&mm->mm_users)) {
+		if (!list_empty(&mm->mmlist))
+			list_del(&mm->mmlist);
+		spin_unlock(&mmlist_lock);
 		exit_aio(mm);
 		exit_mmap(mm);
 		set_mm_exe_file(mm, NULL);
-		if (!list_empty(&mm->mmlist)) {
-			spin_lock(&mmlist_lock);

kernel/fork.c void mmput(struct mm_struct *mm) · line 501

Quoted 10 lines
kernel/fork.c +2 -3
-			list_del(&mm->mmlist);
-			spin_unlock(&mmlist_lock);
-		}
 		put_swap_token(mm);
 		mmdrop(mm);
+	} else {
+		spin_unlock(&mmlist_lock);
 	}
 }
 EXPORT_SYMBOL_GPL(mmput);

I believe you can restore mmput() to how it was: patch below appears to fix it from the get_next_mmlist() end, using atomic_inc_not_zero() - swapoff's try_to_unuse() has to use that on mm_users too. But please check and test it carefully, I'd usually want to test a lot more myself before sending out.

And, hey, this very same patch appears to fix the other issue I mentioned, that I wasn't yet seeing any merging (within a single mm): I think your mm_users 1 test was badly placed before (whether or not we already have an mm_slot makes a big difference to the meaning of mm_users 1).

mm/ksm.c +12 -13
@@ -1137,7 +1137,6 @@ static void insert_to_mm_slots_hash(stru
 	bucket = &mm_slots_hash[((unsigned long)mm / sizeof (struct mm_struct))
 				% nmm_slots_hash];
 	mm_slot->mm = mm;
-	atomic_inc(&mm_slot->mm->mm_users);
 	INIT_LIST_HEAD(&mm_slot->rmap_list);
 	hlist_add_head(&mm_slot->link, bucket);
 }
@@ -1253,21 +1252,21 @@ static struct mm_slot *get_next_mmlist(s
 		if (test_bit(MMF_VM_MERGEABLE, &mm->flags) ||
 		    test_bit(MMF_VM_MERGEALL, &mm->flags)) {
 			mm_slot = get_mm_slot(mm);
-			if (unlikely(atomic_read(&mm->mm_users) == 1)) {
-				if (mm_slot)
+			if (mm_slot) {
+				if (atomic_read(&mm->mm_users) == 1) {
 					mm_slot->touched = 0;
-			} else {
-				if (!mm_slot) {
-					insert_to_mm_slots_hash(mm,
-							     pre_alloc_mm_slot);
-					*used_pre_alloc = 1;
-					mm_slot = pre_alloc_mm_slot;
+					goto next;
 				}
-				mm_slot->touched = 1;
-				return mm_slot;
-			}
+			} else if (atomic_inc_not_zero(&mm->mm_users)) {
+				insert_to_mm_slots_hash(mm, pre_alloc_mm_slot);
+				*used_pre_alloc = 1;
+				mm_slot = pre_alloc_mm_slot;
+			} else
+				goto next;
+			mm_slot->touched = 1;
+			return mm_slot;
 		}
-
+next:
 		cur = cur->next;
 	}
 	return NULL;
author
#10 lore raw
Quoted 3 lines · Hugh Dickins

I suspect too much of your testing has been on the sensible use of KSM, not enough on foolish abuse of it!

Must of the testing was on the ioctl version of ksm, this RFC i thought would end up in "list of requested changes" that will force me to rewrite it. (I did mention that i not sure if everything is safe, and it was mostly just to give the idea of possible desgien and its problems)

Quoted 5 lines

Right, for several reasons* not a patch we'd like to include in the end, but enough to show that indeed fixes the problems I was seeing:
well caught.

Yea, that why i said ugly :).

Quoted 6 lines

I'm already working with a separate list for KSM inside the mm struct (prefer not to be mixing these two things up), but I don't see how that helps.

So you dont have mmlist at all?, Good i think i found another problems with the usage the RFC made with the mmlist, Do you mind to send me the patch that move into diffrent mm list? or you still want to wait?

Quoted 91 lines
Quoted 4 lines

Hugh, i hope at least now you will be able to run it without it will crush to you.

Yes, thanks, and helped me to think of a better fix (I hope!) below...

Quoted 24 lines
---
 kernel/fork.c |   11 ++++++-----
 1 files changed, 6 insertions(+), 5 deletions(-)
kernel/fork.c +4 -2
index e5ef58c..771b89a 100644
@@ -484,17 +484,18 @@ void mmput(struct mm_struct *mm)
 {
 	might_sleep();
 
+	spin_lock(&mmlist_lock);
 	if (atomic_dec_and_test(&mm->mm_users)) {
+		if (!list_empty(&mm->mmlist))
+			list_del(&mm->mmlist);
+		spin_unlock(&mmlist_lock);
 		exit_aio(mm);
 		exit_mmap(mm);
 		set_mm_exe_file(mm, NULL);
-		if (!list_empty(&mm->mmlist)) {
-			spin_lock(&mmlist_lock);

kernel/fork.c void mmput(struct mm_struct *mm) · line 501

Quoted 10 lines
kernel/fork.c +2 -3
-			list_del(&mm->mmlist);
-			spin_unlock(&mmlist_lock);
-		}
 		put_swap_token(mm);
 		mmdrop(mm);
+	} else {
+		spin_unlock(&mmlist_lock);
 	}
 }
 EXPORT_SYMBOL_GPL(mmput);

I believe you can restore mmput() to how it was: patch below appears to fix it from the get_next_mmlist() end, using atomic_inc_not_zero() - swapoff's try_to_unuse() has to use that on mm_users too. But please check and test it carefully, I'd usually want to test a lot more myself before sending out.

And, hey, this very same patch appears to fix the other issue I mentioned, that I wasn't yet seeing any merging (within a single mm): I think your mm_users 1 test was badly placed before (whether or not we already have an mm_slot makes a big difference to the meaning of mm_users 1).

mm/ksm.c +7 -12
@@ -1137,7 +1137,6 @@ static void insert_to_mm_slots_hash(stru
 	bucket = &mm_slots_hash[((unsigned long)mm / sizeof (struct mm_struct))
 				% nmm_slots_hash];
 	mm_slot->mm = mm;
-	atomic_inc(&mm_slot->mm->mm_users);
 	INIT_LIST_HEAD(&mm_slot->rmap_list);
 	hlist_add_head(&mm_slot->link, bucket);
 }
@@ -1253,21 +1252,21 @@ static struct mm_slot *get_next_mmlist(s
 		if (test_bit(MMF_VM_MERGEABLE, &mm->flags) ||
 		    test_bit(MMF_VM_MERGEALL, &mm->flags)) {
 			mm_slot = get_mm_slot(mm);
-			if (unlikely(atomic_read(&mm->mm_users) == 1)) {
-				if (mm_slot)
+			if (mm_slot) {
+				if (atomic_read(&mm->mm_users) == 1) {
 					mm_slot->touched = 0;
-			} else {
-				if (!mm_slot) {
-					insert_to_mm_slots_hash(mm,
-							     pre_alloc_mm_slot);
-					*used_pre_alloc = 1;
-					mm_slot = pre_alloc_mm_slot;
+					goto next;
 				}
-				mm_slot->touched = 1;
-				return mm_slot;
-			}
+			} else if (atomic_inc_not_zero(&mm->mm_users)) {
+				insert_to_mm_slots_hash(mm, pre_alloc_mm_slot);
+				*used_pre_alloc = 1;
+				mm_slot = pre_alloc_mm_slot;

mm/ksm.c static struct mm_slot *get_next_mmlist(s · line 1272

Quoted 10 lines
mm/ksm.c +5 -1
+			} else
+				goto next;
+			mm_slot->touched = 1;
+			return mm_slot;
 		}
-
+next:
 		cur = cur->next;
 	}
 	return NULL;

Yea, that trick with the inc_not_zero is much better, This patch does make sense as a bug fix..
I really would like to see it running with your new list for ksm inside the mmstruct, beacuse I think the mmlist usage have another problems.

So if you can send me a patch I will start stress test, on the other side I dont mind to wait.

Anyway, thanks.

#11 lore raw
Quoted 5 lines · Izik Eidus

So you dont have mmlist at all?, Good i think i found another problems with the usage the RFC made with the mmlist,
Do you mind to send me the patch that move into diffrent mm list? or you still want to wait?

I'd really prefer to wait, there are too many questions I should resolve one way or another now I can test again e.g. do we need to put children on to the list when forking or not?

But don't let me hold you up: if you believe you've problems sharing the mmlist, go ahead and separate out - I called the KSM one ksm_mmlist (surprise!), hanging off init_mm.ksm_mmlist in the same way mmlist does. And then you can get rid of the MMF flags etc (I've set aside your 4/4 as not something to get into right now - probably the release after).

At present I'm still using mmlist_lock, but that's probably a mistake:
we don't really want to have to consider unrelated lock ordering constraints, a ksm_mmlist_lock would be better for now.

None of which precludes someone coming along later and combining ksm_mmlist with mmlist in order to save space: but for now I think we do better to keep them separate, and it sounds like you've seen a strong reason for that.

Hugh