use folio in ksm
cover use folio in ksm
From: "Alex Shi (tencent)" <[email protected]>
This is the 2nd part to use folio in ksm, it saves 82kbytes in ksm.o and passed all ksm testcases in kselftest.
KSM still split same folios to single page if they are good to be KSM, but will skip the rest subpage checking if a folio couldn't suit for ksm.
Alex Shi (tencent) (10):
mm/ksm: reduce the flush action for ksm merging page mm/ksm: skip subpages of compound pages mm/ksm: use folio in try_to_merge_one_page mm/ksm: add identical_folio func
mm/ksm: use folio in stable_tree_search mm/ksm: remove page_stable_node
mm/ksm: use folio in unstable_tree_search_insert mm/ksm: use folio in try_to_merge_xx serie funcs mm/ksm: calc_checksum for folio
m/ksm: use folio in ksm scan path
include/linux/mm.h | 17 ++++ mm/ksm.c | 230 +++++++++++++++++++++++---------------------- 2 files changed, 135 insertions(+), 112 deletions(-)
Signature · 1 line
2.43.0
PATCH 1/10 mm/ksm: reduce the flush action for ksm merging page
From: "Alex Shi (tencent)" <[email protected]>
We can put off the flush action util a merging is realy coming. That could reduce some unmerge page flushing.
BTW, flushing only do at arm, mips and few other archs.
--- mm/ksm.c | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-)
mm/ksm.c +8 -10
@@ -783,10 +783,7 @@ static struct page *get_mergeable_page(struct ksm_rmap_item *rmap_item) goto out;if (is_zone_device_page (page)) goto out_putpage; -if (PageAnon (page)) { -flush_anon_page (vma, page, addr); -flush_dcache_page (page); - }else { +if (!PageAnon (page)) { out_putpage:put_page (page); out:
@@ -1473,8 +1470,8 @@ static int replace_page(struct vm_area_struct *vma, struct page *page,* * This function returns 0 if the pages were merged, -EFAULT otherwise. */ -static int try_to_merge_one_page (struct vm_area_struct *vma, -struct page *page,struct page *kpage) +static int try_to_merge_one_page (struct vm_area_struct *vma,struct page *page, +struct ksm_rmap_item *rmap_item,struct page *kpage) {pte_t orig_pte =__pte (0 );int err = -EFAULT ;
@@ -1500,6 +1497,9 @@ static int try_to_merge_one_page(struct vm_area_struct *vma, goto out_unlock; } +flush_anon_page (vma, page, rmap_item->address); +flush_dcache_page (page); +/* * If this anonymous page is mapped only here, its pte may need * to be write-protected. If it's mapped elsewhere, all of its
@@ -1550,7 +1550,7 @@ static int try_to_merge_with_ksm_page(struct ksm_rmap_item *rmap_item,if (!vma) goto out; - err =try_to_merge_one_page (vma, page, kpage); + err =try_to_merge_one_page (vma, page, rmap_item, kpage);if (err) goto out;
@@ -2385,7 +2385,7 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_itemmap_read_lock (mm); vma =find_mergeable_vma (mm, rmap_item->address);if (vma) { - err =try_to_merge_one_page (vma, page, + err =try_to_merge_one_page (vma, page, rmap_item,ZERO_PAGE (rmap_item->address)); trace_ksm_merge_one_page( page_to_pfn(ZERO_PAGE(rmap_item->address)),
@@ -2663,8 +2663,6 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)if (is_zone_device_page (*page)) goto next_page;if (PageAnon (*page)) { -flush_anon_page (vma, *page, ksm_scan.address); -flush_dcache_page (*page); rmap_item =get_next_rmap_item (mm_slot, ksm_scan.rmap_list, ksm_scan.address);if (rmap_item) {
Signature · 1 line
2.43.0
Re: [PATCH 01/10] mm/ksm: reduce the flush action for ksm merging page
Quoted 6 lines · On 04.06.24 06:24, [email protected] wrote:
From: "Alex Shi (tencent)" <[email protected]>
We can put off the flush action util a merging is realy coming. That could reduce some unmerge page flushing.
BTW, flushing only do at arm, mips and few other archs.
I'm no expert on that flushing, but I thought we would have to do the flushing before accessing page content -- before calculating the checksum etc.
Now you would only do it before the pages_identical() check, but not when calculating the checksum.
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH 01/10] mm/ksm: reduce the flush action for ksm merging page
Quoted 12 lines · David Hildenbrand
Quoted 6 lines · On 04.06.24 06:24, [email protected] wrote:
From: "Alex Shi (tencent)" <[email protected]>
We can put off the flush action util a merging is realy coming. That could reduce some unmerge page flushing.
BTW, flushing only do at arm, mips and few other archs.I'm no expert on that flushing, but I thought we would have to do the flushing before accessing page content -- before calculating the checksum etc.
Now you would only do it before the
pages_identical()check, but not when calculating the checksum.
Hi David,
Thanks a lot for comments!
If calc_checksum() is wrong before pages_idential(), (that's just after page was write_protected, that's a real guarantee for page context secured) pages_identical could recheck and make thing right.
And as to 2 flush functions here, I didn't see the guarantee for other writer from any other place. So maybe we should remove these flush action?
Thanks
Alex
Re: [PATCH 01/10] mm/ksm: reduce the flush action for ksm merging page
Quoted 13 lines · Alex Shi
Quoted 10 lines · David Hildenbrand
Quoted 6 lines · On 04.06.24 06:24, [email protected] wrote:
From: "Alex Shi (tencent)" <[email protected]>
We can put off the flush action util a merging is realy coming. That could reduce some unmerge page flushing.
BTW, flushing only do at arm, mips and few other archs.I'm no expert on that flushing, but I thought we would have to do the flushing before accessing page content -- before calculating the checksum etc.
Quoted 9 lines
Quoted 2 lines
Now you would only do it before the
pages_identical()check, but not when calculating the checksum.Hi David,
Thanks a lot for comments!
If
calc_checksum()is wrong beforepages_idential(), (that's just after page waswrite_protected, that's a real guarantee for page context secured)pages_identicalcould recheck and make thing right.
Yes, but you would get more wrong checksums, resulting in more unnecessary pages_identical() checks.
That is missing from the description, and why we want to change that behavior.
What's the net win?
Quoted 1 line
And as to 2 flush functions here, I didn't see the guarantee for other writer from any other place. So maybe we should remove these flush action?
"I didn't see the guarantee for other writer from any other place" can you rephrase your comment?
If you mean "the process could modify that page concurrently", then you are right. But that's different than "the process modified the page in the past and we are reading stale content because we missed a flush".
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH 01/10] mm/ksm: reduce the flush action for ksm merging page
Quoted 28 lines · David Hildenbrand
Quoted 13 lines · Alex Shi
Quoted 10 lines · David Hildenbrand
Quoted 6 lines · On 04.06.24 06:24, [email protected] wrote:
From: "Alex Shi (tencent)" <[email protected]>
We can put off the flush action util a merging is realy coming. That could reduce some unmerge page flushing.
BTW, flushing only do at arm, mips and few other archs.I'm no expert on that flushing, but I thought we would have to do the flushing before accessing page content -- before calculating the checksum etc.
Quoted 9 lines
Quoted 2 lines
Now you would only do it before the
pages_identical()check, but not when calculating the checksum.Hi David,
Thanks a lot for comments!
If
calc_checksum()is wrong beforepages_idential(), (that's just after page waswrite_protected, that's a real guarantee for page context secured)pages_identicalcould recheck and make thing right.Yes, but you would get more wrong checksums, resulting in more unnecessary
pages_identical()checks.That is missing from the description, and why we want to change that behavior.
Quoted 7 lines
What's the net win?
Quoted 1 line
And as to 2 flush functions here, I didn't see the guarantee for other writer from any other place. So maybe we should remove these flush action?
"I didn't see the guarantee for other writer from any other place" can you rephrase your comment?
If you mean "the process could modify that page concurrently", then you are right. But that's different than "the process modified the page in the past and we are reading stale content because we missed a flush".
Maybe moving the flush before checksum could relief some worries. :)
But still no one knows what flush really help, since if page content only syncs to memory by the flush, the kernel or process can't be work with current code.
thanks
Alex
Re: [PATCH 01/10] mm/ksm: reduce the flush action for ksm merging page
Quoted 35 lines · Alex Shi
Quoted 32 lines · David Hildenbrand
Quoted 13 lines · Alex Shi
Quoted 10 lines · David Hildenbrand
Quoted 6 lines · On 04.06.24 06:24, [email protected] wrote:
From: "Alex Shi (tencent)" <[email protected]>
We can put off the flush action util a merging is realy coming. That could reduce some unmerge page flushing.
BTW, flushing only do at arm, mips and few other archs.I'm no expert on that flushing, but I thought we would have to do the flushing before accessing page content -- before calculating the checksum etc.
Quoted 9 lines
Quoted 2 lines
Now you would only do it before the
pages_identical()check, but not when calculating the checksum.Hi David,
Thanks a lot for comments!
If
calc_checksum()is wrong beforepages_idential(), (that's just after page waswrite_protected, that's a real guarantee for page context secured)pages_identicalcould recheck and make thing right.Yes, but you would get more wrong checksums, resulting in more unnecessary
pages_identical()checks.That is missing from the description, and why we want to change that behavior.
What's the net win?
Quoted 1 line
And as to 2 flush functions here, I didn't see the guarantee for other writer from any other place. So maybe we should remove these flush action?
Quoted 7 lines
Quoted 3 lines
"I didn't see the guarantee for other writer from any other place" can you rephrase your comment?
If you mean "the process could modify that page concurrently", then you are right. But that's different than "the process modified the page in the past and we are reading stale content because we missed a flush".
Maybe moving the flush before checksum could relief some worries. :)
But still no one knows what flush really help, since if page content only syncs to memory by the flush, the kernel or process can't be work with current code.
Please explain to me why we care about moving the flushs at all :)
If they are NOP on most architectures either way, why not simply leave them there and call it a day?
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH 01/10] mm/ksm: reduce the flush action for ksm merging page
Quoted 41 lines · David Hildenbrand
Quoted 40 lines · Alex Shi
Quoted 35 lines · David Hildenbrand
Quoted 13 lines · Alex Shi
Quoted 10 lines · David Hildenbrand
Quoted 6 lines · On 04.06.24 06:24, [email protected] wrote:
From: "Alex Shi (tencent)" <[email protected]>
We can put off the flush action util a merging is realy coming. That could reduce some unmerge page flushing.
BTW, flushing only do at arm, mips and few other archs.I'm no expert on that flushing, but I thought we would have to do the flushing before accessing page content -- before calculating the checksum etc.
Quoted 9 lines
Quoted 2 lines
Now you would only do it before the
pages_identical()check, but not when calculating the checksum.Hi David,
Thanks a lot for comments!
If
calc_checksum()is wrong beforepages_idential(), (that's just after page waswrite_protected, that's a real guarantee for page context secured)pages_identicalcould recheck and make thing right.Yes, but you would get more wrong checksums, resulting in more unnecessary
pages_identical()checks.That is missing from the description, and why we want to change that behavior.
What's the net win?
Quoted 1 line
And as to 2 flush functions here, I didn't see the guarantee for other writer from any other place. So maybe we should remove these flush action?
"I didn't see the guarantee for other writer from any other place" can you rephrase your comment?
If you mean "the process could modify that page concurrently", then you are right. But that's different than "the process modified the page in the past and we are reading stale content because we missed a flush".
Quoted 6 lines
Quoted 2 lines
Maybe moving the flush before checksum could relief some worries. :)
But still no one knows what flush really help, since if page content only syncs to memory by the flush, the kernel or process can't be work with current code.Please explain to me why we care about moving the flushs at all :)
If they are NOP on most architectures either way, why not simply leave them there and call it a day?
Uh, 2 reasons:
1, it uses page and can't convert to folio now.
2, as you pointed, flush action w/o page reading seems just waste time.
Thanks
Alex
Quoted 1 line
Re: [PATCH 01/10] mm/ksm: reduce the flush action for ksm merging page
Quoted 47 lines · Alex Shi
Quoted 44 lines · David Hildenbrand
Quoted 42 lines · Alex Shi
Quoted 28 lines · David Hildenbrand
Quoted 13 lines · Alex Shi
Quoted 10 lines · David Hildenbrand
Quoted 6 lines · On 04.06.24 06:24, [email protected] wrote:
From: "Alex Shi (tencent)" <[email protected]>
We can put off the flush action util a merging is realy coming. That could reduce some unmerge page flushing.
BTW, flushing only do at arm, mips and few other archs.I'm no expert on that flushing, but I thought we would have to do the flushing before accessing page content -- before calculating the checksum etc.
Quoted 9 lines
Quoted 2 lines
Now you would only do it before the
pages_identical()check, but not when calculating the checksum.Hi David,
Thanks a lot for comments!
If
calc_checksum()is wrong beforepages_idential(), (that's just after page waswrite_protected, that's a real guarantee for page context secured)pages_identicalcould recheck and make thing right.Yes, but you would get more wrong checksums, resulting in more unnecessary
pages_identical()checks.That is missing from the description, and why we want to change that behavior.
Quoted 7 lines
What's the net win?
Quoted 1 line
And as to 2 flush functions here, I didn't see the guarantee for other writer from any other place. So maybe we should remove these flush action?
"I didn't see the guarantee for other writer from any other place" can you rephrase your comment?
If you mean "the process could modify that page concurrently", then you are right. But that's different than "the process modified the page in the past and we are reading stale content because we missed a flush".
Maybe moving the flush before checksum could relief some worries. :)
But still no one knows what flush really help, since if page content only syncs to memory by the flush, the kernel or process can't be work with current code.
Quoted 6 lines
Quoted 3 lines
Please explain to me why we care about moving the flushs at all :)
If they are NOP on most architectures either way, why not simply leave them there and call it a day?
Uh, 2 reasons:
1, it uses page and can't convert to folio now.
2, as you pointed, flush action w/o page reading seems just waste time.
Alex, I don't think the approach you take for coming up with the current set of patches is a good idea.
Please reconsider what you can actually convert to folios and what must stay pages for now due to support for large folios in that code.
Then, please explain properly why changes are required and why they are safe.
For example, for in scan_get_next_rmap_item() we really *need* the page and not just the folio. So just leave the flushing there and be done with it.
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH 01/10] mm/ksm: reduce the flush action for ksm merging page
Quoted 55 lines · David Hildenbrand
Quoted 47 lines · Alex Shi
Quoted 44 lines · David Hildenbrand
Quoted 42 lines · Alex Shi
Quoted 28 lines · David Hildenbrand
Quoted 13 lines · Alex Shi
Quoted 10 lines · David Hildenbrand
Quoted 6 lines · On 04.06.24 06:24, [email protected] wrote:
From: "Alex Shi (tencent)" <[email protected]>
We can put off the flush action util a merging is realy coming. That could reduce some unmerge page flushing.
BTW, flushing only do at arm, mips and few other archs.I'm no expert on that flushing, but I thought we would have to do the flushing before accessing page content -- before calculating the checksum etc.
Quoted 9 lines
Quoted 2 lines
Now you would only do it before the
pages_identical()check, but not when calculating the checksum.Hi David,
Thanks a lot for comments!
If
calc_checksum()is wrong beforepages_idential(), (that's just after page waswrite_protected, that's a real guarantee for page context secured)pages_identicalcould recheck and make thing right.Yes, but you would get more wrong checksums, resulting in more unnecessary
pages_identical()checks.That is missing from the description, and why we want to change that behavior.
Quoted 7 lines
What's the net win?
Quoted 1 line
And as to 2 flush functions here, I didn't see the guarantee for other writer from any other place. So maybe we should remove these flush action?
"I didn't see the guarantee for other writer from any other place" can you rephrase your comment?
If you mean "the process could modify that page concurrently", then you are right. But that's different than "the process modified the page in the past and we are reading stale content because we missed a flush".
Maybe moving the flush before checksum could relief some worries. :)
But still no one knows what flush really help, since if page content only syncs to memory by the flush, the kernel or process can't be work with current code.Quoted 6 lines
Quoted 3 lines
Please explain to me why we care about moving the flushs at all :)
If they are NOP on most architectures either way, why not simply leave them there and call it a day?
Uh, 2 reasons:
1, it uses page and can't convert to folio now.
2, as you pointed, flush action w/o page reading seems just waste time.
Quoted 8 lines
Alex, I don't think the approach you take for coming up with the current set of patches is a good idea.
Please reconsider what you can actually convert to folios and what must stay pages for now due to support for large folios in that code.
Then, please explain properly why changes are required and why they are safe.
For example, for in
scan_get_next_rmap_item()we really *need* the page and not just the folio. So just leave the flushing there and be done with it.
Hi David,
Thanks a lot for your review.
Though all patches are passed in kernel selftest, but if we do care the saving more than quick processing, the main purpose of this patchset is gone. I'll drop this series.
Thanks
Alex
Re: [PATCH 01/10] mm/ksm: reduce the flush action for ksm merging page
Quoted 64 lines · Alex Shi
Quoted 61 lines · David Hildenbrand
Quoted 47 lines · Alex Shi
Quoted 44 lines · David Hildenbrand
Quoted 42 lines · Alex Shi
Quoted 28 lines · David Hildenbrand
Quoted 13 lines · Alex Shi
Quoted 10 lines · David Hildenbrand
Quoted 6 lines · On 04.06.24 06:24, [email protected] wrote:
From: "Alex Shi (tencent)" <[email protected]>
We can put off the flush action util a merging is realy coming. That could reduce some unmerge page flushing.
BTW, flushing only do at arm, mips and few other archs.I'm no expert on that flushing, but I thought we would have to do the flushing before accessing page content -- before calculating the checksum etc.
Quoted 9 lines
Quoted 2 lines
Now you would only do it before the
pages_identical()check, but not when calculating the checksum.Hi David,
Thanks a lot for comments!
If
calc_checksum()is wrong beforepages_idential(), (that's just after page waswrite_protected, that's a real guarantee for page context secured)pages_identicalcould recheck and make thing right.Yes, but you would get more wrong checksums, resulting in more unnecessary
pages_identical()checks.That is missing from the description, and why we want to change that behavior.
Quoted 7 lines
What's the net win?
Quoted 1 line
And as to 2 flush functions here, I didn't see the guarantee for other writer from any other place. So maybe we should remove these flush action?
"I didn't see the guarantee for other writer from any other place" can you rephrase your comment?
If you mean "the process could modify that page concurrently", then you are right. But that's different than "the process modified the page in the past and we are reading stale content because we missed a flush".
Maybe moving the flush before checksum could relief some worries. :)
But still no one knows what flush really help, since if page content only syncs to memory by the flush, the kernel or process can't be work with current code.Quoted 6 lines
Quoted 3 lines
Please explain to me why we care about moving the flushs at all :)
If they are NOP on most architectures either way, why not simply leave them there and call it a day?
Uh, 2 reasons:
1, it uses page and can't convert to folio now.
2, as you pointed, flush action w/o page reading seems just waste time.Alex, I don't think the approach you take for coming up with the current set of patches is a good idea.
Please reconsider what you can actually convert to folios and what must stay pages for now due to support for large folios in that code.
Then, please explain properly why changes are required and why they are safe.
Quoted 7 lines
Quoted 2 lines
For example, for in
scan_get_next_rmap_item()we really *need* the page and not just the folio. So just leave the flushing there and be done with it.Hi David,
Thanks a lot for your review.
Though all patches are passed in kernel selftest, but if we do care the saving more than quick processing, the main purpose of this patchset is gone. I'll drop this series.
I think there is value in more folio conversion, but we really have to be careful when we still want to/have to work on pages (folio+page pair).
Signature · 3 lines
Cheers, David / dhildenb
PATCH 2/10 mm/ksm: skip subpages of compound pages
In reply to [email protected]From: "Alex Shi (tencent)" <[email protected]>
When a folio isn't fit for KSM, the subpages are unlikely to be good, So let's skip the rest page checking to save some actions.
--- mm/ksm.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
mm/ksm.c +7 -2
@@ -2644,6 +2644,8 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) goto no_vmas;for_each_vma (vmi, vma) { +int nr =1 ; +if (!(vma->vm_flags &VM_MERGEABLE ))continue ;if (ksm_scan.address < vma->vm_start)
@@ -2660,6 +2662,9 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)cond_resched ();continue ; } + +VM_WARN_ON (PageTail (*page)); + nr =compound_nr (*page);if (is_zone_device_page (*page)) goto next_page;if (PageAnon (*page)) {
@@ -2672,7 +2677,7 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)if (should_skip_rmap_item (*page, rmap_item)) goto next_page; - ksm_scan.address +=PAGE_SIZE ; + ksm_scan.address += nr *PAGE_SIZE ; }else put_page (*page);mmap_read_unlock (mm);
@@ -2680,7 +2685,7 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) } next_page:put_page (*page); - ksm_scan.address +=PAGE_SIZE ; + ksm_scan.address += nr *PAGE_SIZE ;cond_resched (); } }
Signature · 1 line
2.43.0
Re: [PATCH 02/10] mm/ksm: skip subpages of compound pages
Quoted 43 lines · On 04.06.24 06:24, [email protected] wrote:
From: "Alex Shi (tencent)" <[email protected]>
When a folio isn't fit for KSM, the subpages are unlikely to be good, So let's skip the rest page checking to save some actions.
Signed-off-by Alex Shi (tencent) <[email protected]>--- mm/ksm.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)mm/ksm.c +6
-1@@ -2644,6 +2644,8 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) goto no_vmas;for_each_vma (vmi, vma) { +int nr =1 ; +if (!(vma->vm_flags &VM_MERGEABLE ))continue ;if (ksm_scan.address < vma->vm_start)@@ -2660,6 +2662,9 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)cond_resched ();continue ; } + +VM_WARN_ON (PageTail (*page)); + nr =compound_nr (*page);if (is_zone_device_page (*page)) goto next_page;if (PageAnon (*page)) {@@ -2672,7 +2677,7 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)if (should_skip_rmap_item (*page, rmap_item)) goto next_page; - ksm_scan.address +=PAGE_SIZE ; + ksm_scan.address += nr *PAGE_SIZE ; }else put_page (*page);mmap_read_unlock (mm);
mm/ksm.c static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) · line 2691
Quoted 8 lines
mm/ksm.c +1
-1@@ -2680,7 +2685,7 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) } next_page:put_page (*page); - ksm_scan.address +=PAGE_SIZE ; + ksm_scan.address += nr *PAGE_SIZE ;cond_resched (); } }
You might be jumping over pages that don't belong to that folio. What you would actually want to do is somehow use folio_pte_batch() to really know the PTEs point at the same folio, so you can skip them. But that's not that easy when using follow_page() ...
So I suggest dropping this change for now.
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH 02/10] mm/ksm: skip subpages of compound pages
Quoted 53 lines · David Hildenbrand
Quoted 42 lines · On 04.06.24 06:24, [email protected] wrote:
From: "Alex Shi (tencent)" <[email protected]>
When a folio isn't fit for KSM, the subpages are unlikely to be good, So let's skip the rest page checking to save some actions.
Signed-off-by Alex Shi (tencent) <[email protected]>---
mm/ksm.c| 9 +++++++--1 file changed, 7 insertions(+), 2 deletions(-)mm/ksm.c +6
-0@@ -2644,6 +2644,8 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) goto no_vmas; for_each_vma(vmi, vma) { + int nr = 1; + if (!(vma->vm_flags & VM_MERGEABLE)) continue; if (ksm_scan.address < vma->vm_start)@@ -2660,6 +2662,9 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) cond_resched(); continue; } + + VM_WARN_ON(PageTail(*page)); + nr = compound_nr(*page); if (is_zone_device_page(*page)) goto next_page; if (PageAnon(*page)) {@@ -2672,7 +2677,7 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) if (should_skip_rmap_item(*page, rmap_item)) goto next_page; - ksm_scan.address += PAGE_SIZE; + ksm_scan.address += nr * PAGE_SIZE; } else put_page(*page); mmap_read_unlock(mm);@@ -2680,7 +2685,7 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) }Quoted 7 lines
next_page:
put_page(*page);
- ksm_scan.address +=PAGE_SIZE;
+ ksm_scan.address += nr *PAGE_SIZE;
cond_resched();
}
}You might be jumping over pages that don't belong to that folio. What you would actually want to do is somehow use
folio_pte_batch()to really know the PTEs point at the same folio, so you can skip them. But that's not that easy when usingfollow_page()...
Quoted 2 lines
So I suggest dropping this change for now.
Hi David,
Forgive my stupidity, where I jump over normal page that not to belong to the folio?
Thanks
Alex
Re: [PATCH 02/10] mm/ksm: skip subpages of compound pages
Quoted 56 lines · Alex Shi
Quoted 53 lines · David Hildenbrand
Quoted 41 lines · On 04.06.24 06:24, [email protected] wrote:
From: "Alex Shi (tencent)" <[email protected]>
When a folio isn't fit for KSM, the subpages are unlikely to be good, So let's skip the rest page checking to save some actions.
Signed-off-by Alex Shi (tencent) <[email protected]>--- mm/ksm.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)mm/ksm.c +6
-0@@ -2644,6 +2644,8 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) goto no_vmas; for_each_vma(vmi, vma) { + int nr = 1; + if (!(vma->vm_flags & VM_MERGEABLE)) continue; if (ksm_scan.address < vma->vm_start)@@ -2660,6 +2662,9 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) cond_resched(); continue; } + + VM_WARN_ON(PageTail(*page)); + nr = compound_nr(*page); if (is_zone_device_page(*page)) goto next_page; if (PageAnon(*page)) {@@ -2672,7 +2677,7 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) if (should_skip_rmap_item(*page, rmap_item)) goto next_page; - ksm_scan.address += PAGE_SIZE; + ksm_scan.address += nr * PAGE_SIZE; } else put_page(*page); mmap_read_unlock(mm);mm/ksm.c static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) · line 2691
Quoted 8 lines
mm/ksm.c +1
-1@@ -2680,7 +2685,7 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) } next_page: put_page(*page); - ksm_scan.address += PAGE_SIZE; + ksm_scan.address += nr * PAGE_SIZE; cond_resched(); } }You might be jumping over pages that don't belong to that folio. What you would actually want to do is somehow use
folio_pte_batch()to really know the PTEs point at the same folio, so you can skip them. But that's not that easy when usingfollow_page()...
Quoted 6 lines
Quoted 2 lines
So I suggest dropping this change for now.
Hi David,
Forgive my stupidity, where I jump over normal page that not to belong to the folio?
IIUC, you assume that the folio is fully mapped by all PTEs that could span it, and that follow_page() would give you the head page, correct?
As a simple example, assume only a single page of a large folio is still mapped, which could be any tail page. You couldn't jump over any PTEs.
Or am I missing something?
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH 02/10] mm/ksm: skip subpages of compound pages
Quoted 18 lines · David Hildenbrand
Quoted 18 lines
Quoted 13 lines
Quoted 10 lines
(quoted diff) +0
-0@@ -2680,7 +2685,7 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) }
next_page:
put_page(*page);
- ksm_scan.address +=PAGE_SIZE;
+ ksm_scan.address += nr *PAGE_SIZE;
cond_resched();
}
}You might be jumping over pages that don't belong to that folio. What you would actually want to do is somehow use
folio_pte_batch()to really know the PTEs point at the same folio, so you can skip them. But that's not that easy when usingfollow_page()...Quoted 2 lines
So I suggest dropping this change for now.
Hi David,
Quoted 7 lines
Quoted 1 line
Forgive my stupidity, where I jump over normal page that not to belong to the folio?
IIUC, you assume that the folio is fully mapped by all PTEs that could span it, and that
follow_page()would give you the head page, correct?As a simple example, assume only a single page of a large folio is still mapped, which could be any tail page. You couldn't jump over any PTEs.
Or am I missing something?
Uh, thanks for explanations. for what's we concerned, the following code could take care of the FULL or ERR pages. And it still keep the step of single page.
page = follow_page(vma, ksm_scan.address, FOLL_GET);
if (IS_ERR_OR_NULL(page)) {
ksm_scan.address += PAGE_SIZE;
cond_resched();
continue;
}And after the above code, step folio_nr_pages on address should be safe, isn't it?
Thanks a lot
Alex
Re: [PATCH 02/10] mm/ksm: skip subpages of compound pages
Quoted 29 lines · Alex Shi
Quoted 18 lines · David Hildenbrand
Quoted 18 lines
Quoted 13 lines
mm/ksm.c static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) · line 2691
Quoted 10 lines
mm/ksm.c +1
-1@@ -2680,7 +2685,7 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) } next_page: put_page(*page); - ksm_scan.address += PAGE_SIZE; + ksm_scan.address += nr * PAGE_SIZE; cond_resched(); } }You might be jumping over pages that don't belong to that folio. What you would actually want to do is somehow use
folio_pte_batch()to really know the PTEs point at the same folio, so you can skip them. But that's not that easy when usingfollow_page()...Quoted 2 lines
So I suggest dropping this change for now.
Hi David,
Quoted 7 lines
Quoted 1 line
Forgive my stupidity, where I jump over normal page that not to belong to the folio?
IIUC, you assume that the folio is fully mapped by all PTEs that could span it, and that
follow_page()would give you the head page, correct?As a simple example, assume only a single page of a large folio is still mapped, which could be any tail page. You couldn't jump over any PTEs.
Or am I missing something?
Quoted 8 lines
Uh, thanks for explanations. for what's we concerned, the following code could take care of the FULL or ERR pages. And it still keep the step of single page.
page = follow_page(vma, ksm_scan.address, FOLL_GET); if (IS_ERR_OR_NULL(page)) { ksm_scan.address += PAGE_SIZE; cond_resched(); continue; }And after the above code, step
folio_nr_pageson address should be safe, isn't it?
Not sure if I follow. Let me try explaining once again:
Assume a PTE maps some tail page of the large anonymous folio. The other PTEs around it map some other anonymous folios, not pages of that large anonymous folio.
Without looking at the other PTEs you don't know how much you can skip.
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH 02/10] mm/ksm: skip subpages of compound pages
Quoted 39 lines · David Hildenbrand
Quoted 29 lines · Alex Shi
Quoted 18 lines · David Hildenbrand
Quoted 18 lines
Quoted 13 lines
Quoted 10 lines
(quoted diff) +0
-0@@ -2680,7 +2685,7 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) }
next_page:
put_page(*page);
- ksm_scan.address +=PAGE_SIZE;
+ ksm_scan.address += nr *PAGE_SIZE;
cond_resched();
}
}You might be jumping over pages that don't belong to that folio. What you would actually want to do is somehow use
folio_pte_batch()to really know the PTEs point at the same folio, so you can skip them. But that's not that easy when usingfollow_page()...Quoted 2 lines
So I suggest dropping this change for now.
Hi David,
Quoted 7 lines
Quoted 1 line
Forgive my stupidity, where I jump over normal page that not to belong to the folio?
IIUC, you assume that the folio is fully mapped by all PTEs that could span it, and that
follow_page()would give you the head page, correct?As a simple example, assume only a single page of a large folio is still mapped, which could be any tail page. You couldn't jump over any PTEs.
Or am I missing something?
Quoted 8 lines
Uh, thanks for explanations. for what's we concerned, the following code could take care of the FULL or ERR pages. And it still keep the step of single page. page =
follow_page(vma, ksm_scan.address, FOLL_GET);
if (IS_ERR_OR_NULL(page)) {
ksm_scan.address +=PAGE_SIZE;
cond_resched();
continue;
}
And after the above code, stepfolio_nr_pageson address should be safe, isn't it?
Quoted 3 lines
Not sure if I follow. Let me try explaining once again:
Assume a PTE maps some tail page of the large anonymous folio. The other PTEs around it map some other anonymous folios, not pages of that large anonymous folio.
Sorry, David,
Do you meaning there are 2 folio pages, in a same vma, in their address, 'ksm_scan.address', would be overlapped in a folio size space? If so, that do out of my expect. I do have no idea of this thing. Could you give me more hints of this problem or how things work on it in current kernel?
Thanks a lot!
Alex
Quoted 4 lines
Without looking at the other PTEs you don't know how much you can skip.
Re: [PATCH 02/10] mm/ksm: skip subpages of compound pages
Quoted 44 lines · Alex Shi
Quoted 41 lines · David Hildenbrand
Quoted 29 lines · Alex Shi
Quoted 18 lines · David Hildenbrand
Quoted 18 lines
Quoted 13 lines
Review static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) · line 2691
Quoted 10 lines
(quoted diff) +1
-1@@ -2680,7 +2685,7 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) } next_page: put_page(*page); - ksm_scan.address += PAGE_SIZE; + ksm_scan.address += nr * PAGE_SIZE; cond_resched(); } }You might be jumping over pages that don't belong to that folio. What you would actually want to do is somehow use
folio_pte_batch()to really know the PTEs point at the same folio, so you can skip them. But that's not that easy when usingfollow_page()...Quoted 2 lines
So I suggest dropping this change for now.
Hi David,
Quoted 7 lines
Quoted 1 line
Forgive my stupidity, where I jump over normal page that not to belong to the folio?
IIUC, you assume that the folio is fully mapped by all PTEs that could span it, and that
follow_page()would give you the head page, correct?As a simple example, assume only a single page of a large folio is still mapped, which could be any tail page. You couldn't jump over any PTEs.
Or am I missing something?
Quoted 8 lines
Uh, thanks for explanations. for what's we concerned, the following code could take care of the FULL or ERR pages. And it still keep the step of single page.
page =follow_page(vma, ksm_scan.address, FOLL_GET);
if (IS_ERR_OR_NULL(page)) {
ksm_scan.address +=PAGE_SIZE;
cond_resched();
continue;
}
And after the above code, stepfolio_nr_pageson address should be safe, isn't it?Not sure if I follow. Let me try explaining once again:
Quoted 7 lines
Quoted 1 line
Assume a PTE maps some tail page of the large anonymous folio. The other PTEs around it map some other anonymous folios, not pages of that large anonymous folio.
Sorry, David,
Do you meaning there are 2 folio pages, in a same vma, in their address, 'ksm_scan.address', would be overlapped in a folio size space? If so, that do out of my expect. I do have no idea of this thing. Could you give me more hints of this problem or how things work on it in current kernel?
We do fully support partially mapping of THPs/large folios. That means, you could map a single page of a large pagecache folio and the other PTEs could map anonymous folios (due to COW).
Simply because follow_page() returned a page of a large folio doesn't generally say that the PTEs around it map the same large folio.
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH 02/10] mm/ksm: skip subpages of compound pages
In reply to [email protected]Quoted 4 lines · On Tue, Jun 04, 2024 at 12:24:44PM +0800, [email protected] wrote:
From: "Alex Shi (tencent)" <[email protected]>
When a folio isn't fit for KSM, the subpages are unlikely to be good, So let's skip the rest page checking to save some actions.
Why would you say that is true? We have plenty of evidence that userspace allocators can allocate large folios, then use only the first few bytes, leaving many tail pages full of zeroes.
mm/ksm.c static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) · line 2667
Quoted 7 lines
mm/ksm.c +3
-0@@ -2660,6 +2662,9 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)cond_resched ();continue ; } + +VM_WARN_ON (PageTail (*page)); + nr =compound_nr (*page);
And this is simply wrong. *page can obviously be a tail page.
Re: [PATCH 02/10] mm/ksm: skip subpages of compound pages
Quoted 9 lines · Matthew Wilcox
Quoted 4 lines · On Tue, Jun 04, 2024 at 12:24:44PM +0800, [email protected] wrote:
From: "Alex Shi (tencent)" <[email protected]>
When a folio isn't fit for KSM, the subpages are unlikely to be good, So let's skip the rest page checking to save some actions.
Why would you say that is true? We have plenty of evidence that userspace allocators can allocate large folios, then use only the first few bytes, leaving many tail pages full of zeroes.
Um, that do need tail pages...
Is there some way to use more folio in ksm?
Quoted 11 lines
mm/ksm.c static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) · line 2667
Quoted 8 lines
mm/ksm.c +3
-0@@ -2660,6 +2662,9 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)cond_resched ();continue ; } + +VM_WARN_ON (PageTail (*page)); + nr =compound_nr (*page);And this is simply wrong. *page can obviously be a tail page.
Got it. Thanks a lot!
Alex
Re: [PATCH 02/10] mm/ksm: skip subpages of compound pages
Quoted 9 lines · Alex Shi
Quoted 6 lines · Matthew Wilcox
Quoted 4 lines · On Tue, Jun 04, 2024 at 12:24:44PM +0800, [email protected] wrote:
From: "Alex Shi (tencent)" <[email protected]>
When a folio isn't fit for KSM, the subpages are unlikely to be good, So let's skip the rest page checking to save some actions.
Quoted 6 lines
Quoted 3 lines
Why would you say that is true? We have plenty of evidence that userspace allocators can allocate large folios, then use only the first few bytes, leaving many tail pages full of zeroes.
Um, that do need tail pages...
Is there some way to use more folio in ksm?
My take, and Willy can correct me if I am wrong:
"struct page" is not going to away any time soon, but it might shrink at some point.
That is, we can use the "struct page" pointer to point at a page frame, and use "struct folio" to lookup/manage the metadata.
That is, use "struct page" when accessing the actual memory content (checksum, testing for identical content), but use the folio part when looking up metadata (folio_test_anon() etc). In the future we might want to replace the "struct page" pointer by an index into the folio, but that doesn't have to happen right now.
For KSM, that would mean that you have a folio+page (late folio+index) pair when possibly dealing with large folios, but you can use a folio without a page when dealing with KSM folios, that are always small.
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH 02/10] mm/ksm: skip subpages of compound pages
Quoted 17 lines · David Hildenbrand
Quoted 9 lines · Alex Shi
Quoted 6 lines · Matthew Wilcox
Quoted 4 lines · On Tue, Jun 04, 2024 at 12:24:44PM +0800, [email protected] wrote:
From: "Alex Shi (tencent)" <[email protected]>
When a folio isn't fit for KSM, the subpages are unlikely to be good, So let's skip the rest page checking to save some actions.
Quoted 6 lines
Quoted 3 lines
Why would you say that is true? We have plenty of evidence that userspace allocators can allocate large folios, then use only the first few bytes, leaving many tail pages full of zeroes.
Um, that do need tail pages...
Is there some way to use more folio in ksm?
Quoted 7 lines
My take, and Willy can correct me if I am wrong:
"struct page" is not going to away any time soon, but it might shrink at some point.
That is, we can use the "struct page" pointer to point at a page frame, and use "struct folio" to lookup/manage the metadata.
Right.
Quoted 5 lines
That is, use "struct page" when accessing the actual memory content (checksum, testing for identical content), but use the folio part when looking up metadata (
folio_test_anon()etc). In the future we might want to replace the "struct page" pointer by an index into the folio, but that doesn't have to happen right now.
My current thinking is that folio->pfn is how we know where the memory described by the folio is. Using an index would be memmap[folio->pfn + index] which isn't terribly expensive, but we may as well pass around the (folio, page) pair and save the reference to memmap.
Quoted 3 lines
For KSM, that would mean that you have a folio+page (late folio+index) pair when possibly dealing with large folios, but you can use a folio without a page when dealing with KSM folios, that are always small.
Yes, agreed.
Re: [PATCH 02/10] mm/ksm: skip subpages of compound pages
Quoted 34 lines · Matthew Wilcox
Quoted 17 lines · David Hildenbrand
Quoted 9 lines · Alex Shi
Quoted 6 lines · Matthew Wilcox
Quoted 4 lines · On Tue, Jun 04, 2024 at 12:24:44PM +0800, [email protected] wrote:
From: "Alex Shi (tencent)" <[email protected]>
When a folio isn't fit for KSM, the subpages are unlikely to be good, So let's skip the rest page checking to save some actions.
Quoted 6 lines
Quoted 3 lines
Why would you say that is true? We have plenty of evidence that userspace allocators can allocate large folios, then use only the first few bytes, leaving many tail pages full of zeroes.
Um, that do need tail pages...
Is there some way to use more folio in ksm?Quoted 7 lines
My take, and Willy can correct me if I am wrong:
"struct page" is not going to away any time soon, but it might shrink at some point.
That is, we can use the "struct page" pointer to point at a page frame, and use "struct folio" to lookup/manage the metadata.
Right.
Quoted 5 lines
That is, use "struct page" when accessing the actual memory content (checksum, testing for identical content), but use the folio part when looking up metadata (
folio_test_anon()etc). In the future we might want to replace the "struct page" pointer by an index into the folio, but that doesn't have to happen right now.
Quoted 4 lines
My current thinking is that
folio->pfnis how we know where the memory described by the folio is. Using an index would be memmap[folio->pfn + index] which isn't terribly expensive, but we may as well pass around the (folio, page) pair and save the reference to memmap.
Right, as soon as the folio does not overlay the head page it's going to be a bit different.
A (folio,page) pair, like we use in the RMAP code, is likely the best option for now and gives us sufficient flexibility for the future design.
Signature · 3 lines
Cheers, David / dhildenb
PATCH 3/10 mm/ksm: use folio in try_to_merge_one_page
In reply to [email protected]From: "Alex Shi (tencent)" <[email protected]>
scan_get_next_rmap_item() return folio actually now. So in the calling path to try_to_merge_one_page() parameter pages are actually folios. So let's use folio instead of of page in the function to save few compound checking in callee functions.
The 'page' left here since flush functions still not support folios yet.
--- mm/ksm.c | 61 ++++++++++++++++++++++++++++++++------------------------ 1 file changed, 35 insertions(+), 26 deletions(-)
mm/ksm.c +35 -26
@@ -1462,24 +1462,29 @@ static int replace_page(struct vm_area_struct *vma, struct page *page, }/* -* try_to_merge_one_page - take two pages and merge them into one -* @vma: the vma that holds the pte pointing to page -* @page: the PageAnon page that we want to replace with kpage -* @kpage: the PageKsm page that we want to map instead of page, -* or NULL the first time when we want to use page as kpage. +* try_to_merge_one_page - take two folios and merge them into one +* @vma: the vma that holds the pte pointing to folio +* @folio: the PageAnon page that we want to replace with kfolio +* @kfolio: the PageKsm page that we want to map instead of folio, +* or NULL the first time when we want to use folio as kfolio. * -* This function returns 0 if the pages were merged, -EFAULT otherwise. +* This function returns 0 if the folios were merged, -EFAULT otherwise. */ -static int try_to_merge_one_page (struct vm_area_struct *vma,struct page *page, -struct ksm_rmap_item *rmap_item,struct page *kpage) +static int try_to_merge_one_page (struct vm_area_struct *vma,struct folio *folio, +struct ksm_rmap_item *rmap_item,struct folio *kfolio) {pte_t orig_pte =__pte (0 );int err = -EFAULT ; +struct page *page =folio_page (folio,0 ); +struct page *kpage; -if (page == kpage)/* ksm page forked */ +if (kfolio) + kpage =folio_page (kfolio,0 ); + +if (folio == kfolio)/* ksm page forked */ return 0 ; -if (!PageAnon (page)) +if (!folio_test_anon (folio)) goto out;/*
@@ -1489,11 +1494,11 @@ static int try_to_merge_one_page(struct vm_area_struct *vma, struct page *page,* prefer to continue scanning and merging different pages, * then come back to this page when it is unlocked. */ -if (!trylock_page (page)) +if (!folio_trylock (folio)) goto out; -if (PageTransCompound (page)) { -if (split_huge_page (page)) +if (folio_test_large (folio)) { +if (split_folio (folio)) goto out_unlock; }
@@ -1506,35 +1511,36 @@ static int try_to_merge_one_page(struct vm_area_struct *vma, struct page *page,* ptes are necessarily already write-protected. But in either * case, we need to lock and check page_count is not raised. */ -if (write_protect_page (vma,page_folio (page), &orig_pte) ==0 ) { -if (!kpage) { +if (write_protect_page (vma, folio, &orig_pte) ==0 ) { +if (!kfolio) {/* * While we hold page lock, upgrade page from * PageAnon+anon_vma to PageKsm+NULL stable_node: * stable_tree_insert() will update stable_node. */ -folio_set_stable_node (page_folio (page),NULL ); -mark_page_accessed (page); +folio_set_stable_node (folio,NULL ); +folio_mark_accessed (folio);/* * Page reclaim just frees a clean page with no dirty * ptes: make sure that the ksm page would be swapped. */ -if (!PageDirty (page)) -SetPageDirty (page); +if (!folio_test_dirty (folio)) +folio_set_dirty (folio); err =0 ; }else if (pages_identical (page, kpage)) err =replace_page (vma, page, kpage, orig_pte); } out_unlock: -unlock_page (page); +folio_unlock (folio); out:return err; }/* * try_to_merge_with_ksm_page - like try_to_merge_two_pages, -* but no new kernel page is allocated: kpage must already be a ksm page. +* but no new kernel page is allocated, kpage is a ksm page or NULL +* if we use page as first ksm page. * * This function returns 0 if the pages were merged, -EFAULT otherwise. */
@@ -1544,13 +1550,17 @@ static int try_to_merge_with_ksm_page(struct ksm_rmap_item *rmap_item,struct mm_struct *mm = rmap_item->mm;struct vm_area_struct *vma;int err = -EFAULT ; +struct folio *kfolio;mmap_read_lock (mm); vma =find_mergeable_vma (mm, rmap_item->address);if (!vma) goto out; - err =try_to_merge_one_page (vma, page, rmap_item, kpage); +if (kpage) + kfolio =page_folio (kpage); + + err =try_to_merge_one_page (vma,page_folio (page), rmap_item, kfolio);if (err) goto out;
@@ -2385,8 +2395,8 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_itemmap_read_lock (mm); vma =find_mergeable_vma (mm, rmap_item->address);if (vma) { - err =try_to_merge_one_page (vma, page, rmap_item, -ZERO_PAGE (rmap_item->address)); + err =try_to_merge_one_page (vma,page_folio (page), rmap_item, +page_folio (ZERO_PAGE (rmap_item->address)));trace_ksm_merge_one_page (page_to_pfn (ZERO_PAGE (rmap_item->address)), rmap_item, mm, err);
@@ -2671,8 +2681,7 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) rmap_item =get_next_rmap_item (mm_slot, ksm_scan.rmap_list, ksm_scan.address);if (rmap_item) { - ksm_scan.rmap_list = - &rmap_item->rmap_list; + ksm_scan.rmap_list = &rmap_item->rmap_list;if (should_skip_rmap_item (*page, rmap_item)) goto next_page;
Signature · 1 line
2.43.0
Re: [PATCH 03/10] mm/ksm: use folio in try_to_merge_one_page
Quoted 62 lines · On 04.06.24 06:24, [email protected] wrote:
From: "Alex Shi (tencent)" <[email protected]>
scan_get_next_rmap_item()return folio actually now. So in the calling path totry_to_merge_one_page()parameter pages are actually folios. So let's use folio instead of of page in the function to save few compound checking in callee functions.The 'page' left here since flush functions still not support folios yet.
Signed-off-by Alex Shi (tencent) <[email protected]>--- mm/ksm.c | 61 ++++++++++++++++++++++++++++++++------------------------ 1 file changed, 35 insertions(+), 26 deletions(-)mm/ksm.c +15
-10@@ -1462,24 +1462,29 @@ static int replace_page(struct vm_area_struct *vma, struct page *page, }/* -* try_to_merge_one_page - take two pages and merge them into one -* @vma: the vma that holds the pte pointing to page -* @page: the PageAnon page that we want to replace with kpage -* @kpage: the PageKsm page that we want to map instead of page, -* or NULL the first time when we want to use page as kpage. +* try_to_merge_one_page - take two folios and merge them into one +* @vma: the vma that holds the pte pointing to folio +* @folio: the PageAnon page that we want to replace with kfolio +* @kfolio: the PageKsm page that we want to map instead of folio, +* or NULL the first time when we want to use folio as kfolio. * -* This function returns 0 if the pages were merged, -EFAULT otherwise. +* This function returns 0 if the folios were merged, -EFAULT otherwise. */ -static int try_to_merge_one_page (struct vm_area_struct *vma,struct page *page, -struct ksm_rmap_item *rmap_item,struct page *kpage) +static int try_to_merge_one_page (struct vm_area_struct *vma,struct folio *folio, +struct ksm_rmap_item *rmap_item,struct folio *kfolio) {pte_t orig_pte =__pte (0 );int err = -EFAULT ; +struct page *page =folio_page (folio,0 ); +struct page *kpage; -if (page == kpage)/* ksm page forked */ +if (kfolio) + kpage =folio_page (kfolio,0 ); + +if (folio == kfolio)/* ksm page forked */ return 0 ; -if (!PageAnon (page)) +if (!folio_test_anon (folio)) goto out;/* @@ -1489,11 +1494,11 @@ static int try_to_merge_one_page(struct vm_area_struct *vma, struct page *page,* prefer to continue scanning and merging different pages, * then come back to this page when it is unlocked. */
mm/ksm.c static int try_to_merge_one_page(struct vm_area_struct *vma, struct page *page, · line 1503
Quoted 10 lines
mm/ksm.c +3
-3-if (!trylock_page (page)) +if (!folio_trylock (folio)) goto out; -if (PageTransCompound (page)) { -if (split_huge_page (page)) +if (folio_test_large (folio)) { +if (split_folio (folio)) goto out_unlock; }
Careful: there is a subtle change:
In the old code, after split_huge_page() succeeded, you would have a ref on *page* that you have to drop.
Now, you would have a ref on *folio* -- IOW the head page that calling code has to drop.
Is that handled accordingly? IOW, is there no code that would assume it would drop the reference on the *page* instead of on the *folio* (that, after split succeeds, would be an order-0 folio)
If so, worth spelling out in the description (you say "So in the calling path to try_to_merge_one_page() parameter pages are actually folios", but I am not sure if that means that all page refcounting code was changed to folio refcounting code).
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH 03/10] mm/ksm: use folio in try_to_merge_one_page
Quoted 17 lines · David Hildenbrand
Quoted 4 lines
mm/ksm.c +0
-0@@ -1489,11 +1494,11 @@ static int try_to_merge_one_page(struct vm_area_struct *vma, struct page *page, * prefer to continue scanning and merging different pages, * then come back to this page when it is unlocked.mm/ksm.c static int try_to_merge_one_page(struct vm_area_struct *vma, struct page *page, · line 1503
Quoted 10 lines
mm/ksm.c +3
-2*/ - if (!trylock_page(page)) + if (!folio_trylock(folio)) goto out; - if (PageTransCompound(page)) { - if (split_huge_page(page)) + if (folio_test_large(folio)) { + if (split_folio(folio)) goto out_unlock; }Careful: there is a subtle change:
Quoted 7 lines
In the old code, after
split_huge_page()succeeded, you would have a ref on *page* that you have to drop.Now, you would have a ref on *folio* -- IOW the head page that calling code has to drop.
Is that handled accordingly? IOW, is there no code that would assume it would drop the reference on the *page* instead of on the *folio* (that, after split succeeds, would be an order-0 folio)
If so, worth spelling out in the description (you say "So in the calling path to
try_to_merge_one_page()parameter pages are actually folios", but I am not sure if that means that all page refcounting code was changed to folio refcounting code).
Hi David,
Thanks a lot for comments. Yes, the ref on folio was handled. I will mention it again in commit log.
The 2nd patch in this series would just pass folios into next step, tail page will be warned, so the ref on 'page' here is also the ref on folio. they are same.
Thanks
Alex
Re: [PATCH 03/10] mm/ksm: use folio in try_to_merge_one_page
Quoted 24 lines · Alex Shi
Quoted 21 lines · David Hildenbrand
Quoted 4 lines
mm/ksm.c +0
-0@@ -1489,11 +1494,11 @@ static int try_to_merge_one_page(struct vm_area_struct *vma, struct page *page, * prefer to continue scanning and merging different pages, * then come back to this page when it is unlocked.mm/ksm.c static int try_to_merge_one_page(struct vm_area_struct *vma, struct page *page, · line 1503
Quoted 10 lines
mm/ksm.c +3
-2*/ - if (!trylock_page(page)) + if (!folio_trylock(folio)) goto out; - if (PageTransCompound(page)) { - if (split_huge_page(page)) + if (folio_test_large(folio)) { + if (split_folio(folio)) goto out_unlock; }Careful: there is a subtle change:
In the old code, after
split_huge_page()succeeded, you would have a ref on *page* that you have to drop.Now, you would have a ref on *folio* -- IOW the head page that calling code has to drop.
Quoted 8 lines
Quoted 3 lines
Is that handled accordingly? IOW, is there no code that would assume it would drop the reference on the *page* instead of on the *folio* (that, after split succeeds, would be an order-0 folio)
If so, worth spelling out in the description (you say "So in the calling path to
try_to_merge_one_page()parameter pages are actually folios", but I am not sure if that means that all page refcounting code was changed to folio refcounting code).Hi David,
Thanks a lot for comments. Yes, the ref on folio was handled. I will mention it again in commit log.
The 2nd patch in this series would just pass folios into next step, tail page will be warned, so the ref on 'page' here is also the ref on folio. they are same.
Okay, but the 2nd patch is wrong and should be dropped. We can easily end up with tail pages here.
We have to be a bit careful here.
Signature · 3 lines
Cheers, David / dhildenb
PATCH 4/10 mm/ksm: add identical_folio func
In reply to [email protected]From: "Alex Shi (tencent)" <[email protected]>
To replace pages_identical() func and further reduce the 'page' using in try_to_merge_one_page function.
--- include/linux/mm.h | 17 +++++++++++++++++ mm/ksm.c | 2 +- 2 files changed, 18 insertions(+), 1 deletion(-)
include/linux/mm.h +17 -0
@@ -4164,6 +4164,23 @@ static inline void setup_nr_node_ids(void) {}extern int memcmp_pages (struct page *page1,struct page *page2); +static inline int identical_folio (struct folio *folio1,struct folio *folio2) +{ +int i, nr1, nr2, ret =0 ; + + nr1 =folio_nr_pages (folio1); + nr2 =folio_nr_pages (folio2); +if (nr1 != nr2) +return 0 ; + +for (i =0 ; i < nr1; i++) { + ret =memcmp_pages (folio_page (folio1, i),folio_page (folio2, i)); +if (ret) +break ; + } + +return !ret; +}static inline int pages_identical (struct page *page1,struct page *page2) {return !memcmp_pages (page1, page2);
mm/ksm.c +1 -1
@@ -1527,7 +1527,7 @@ static int try_to_merge_one_page(struct vm_area_struct *vma, struct folio *folioif (!folio_test_dirty (folio))folio_set_dirty (folio); err =0 ; - }else if (pages_identical (page, kpage)) + }else if (identical_folio (folio, kfolio)) err =replace_page (vma, page, kpage, orig_pte); }
Signature · 1 line
2.43.0
Re: [PATCH 04/10] mm/ksm: add identical_folio func
Quoted 26 lines · On 04.06.24 06:24, [email protected] wrote:
From: "Alex Shi (tencent)" <[email protected]>
To replace
pages_identical()func and further reduce the 'page' using intry_to_merge_one_pagefunction.Signed-off-by Alex Shi (tencent) <[email protected]>--- include/linux/mm.h | 17 +++++++++++++++++ mm/ksm.c | 2 +- 2 files changed, 18 insertions(+), 1 deletion(-)include/linux/mm.h +7
-0@@ -4164,6 +4164,23 @@ static inline void setup_nr_node_ids(void) {}extern int memcmp_pages (struct page *page1,struct page *page2); +static inline int identical_folio (struct folio *folio1,struct folio *folio2) +{ +int i, nr1, nr2, ret =0 ; + + nr1 =folio_nr_pages (folio1); + nr2 =folio_nr_pages (folio2); +if (nr1 != nr2)
include/linux/mm.h static inline void setup_nr_node_ids(void) {} · line 4183
Quoted 10 lines
include/linux/mm.h +10
-0+return 0 ; + +for (i =0 ; i < nr1; i++) { + ret =memcmp_pages (folio_page (folio1, i),folio_page (folio2, i)); +if (ret) +break ; + } + +return !ret; +}
We never get there with large folios. So handling large folios is completely unnecessary.
Keep it simple: either VM_WARN_ON_ONCE when a large folio is passed, or simply return "false". Then, check only a single page.
Signature · 3 lines
Cheers, David / dhildenb
PATCH 5/10 mm/ksm: use folio in stable_tree_search
In reply to [email protected]From: "Alex Shi (tencent)" <[email protected]>
The stable_tree_search() function is actually using folio internal, just pass the folio parameter from cmp_and_merge_page().
And in cmp_and_merge_page(), the parameter 'page' is actually a folio too, so use folio to save a few compound check in necessnary context.
--- mm/ksm.c | 36 ++++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-)
mm/ksm.c +18 -18
@@ -1825,7 +1825,7 @@ static __always_inline struct folio *chain(struct ksm_stable_node **s_n_d,* This function returns the stable tree node of identical content if found, * NULL otherwise. */ -static struct page *stable_tree_search (struct page *page) +static struct folio *stable_tree_search (struct folio *folio) {int nid;struct rb_root *root;
@@ -1833,14 +1833,12 @@ static struct page *stable_tree_search(struct page *page)struct rb_node *parent;struct ksm_stable_node *stable_node, *stable_node_dup, *stable_node_any;struct ksm_stable_node *page_node; -struct folio *folio; - folio =page_folio (page); page_node =folio_stable_node (folio);if (page_node && page_node->head != &migrate_nodes) {/* ksm page forked */ folio_get (folio); -return &folio->page; +return folio; } nid =get_kpfn_nid (folio_pfn (folio));
@@ -1907,7 +1905,7 @@ static struct page *stable_tree_search(struct page *page) goto again; } - ret =memcmp_pages (page, &tree_folio->page); + ret =memcmp_pages (folio_page (folio,0 ), &tree_folio->page);folio_put (tree_folio); parent = *new;
@@ -1973,7 +1971,7 @@ static struct page *stable_tree_search(struct page *page)folio_put (tree_folio); goto replace; } -return &tree_folio->page; +return tree_folio; } }
@@ -1987,7 +1985,7 @@ static struct page *stable_tree_search(struct page *page) out:if (is_page_sharing_candidate (page_node)) {folio_get (folio); -return &folio->page; +return folio; }else return NULL ;
@@ -2037,7 +2035,7 @@ static struct page *stable_tree_search(struct page *page) } stable_node_dup->head = &migrate_nodes;list_add (&stable_node_dup->list, stable_node_dup->head); -return &folio->page; +return folio; chain_append:/* stable_node_dup could be null if it reached the limit */
@@ -2324,6 +2322,7 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_iteunsigned int checksum;int err;bool max_page_sharing_bypass = false; +struct folio *folio, *kfolio; stable_node =page_stable_node (page);if (stable_node) {
@@ -2346,30 +2345,31 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite }/* We first start with searching the page inside the stable tree */ - kpage =stable_tree_search (page); -if (kpage == page && rmap_item->head == stable_node) { -put_page (kpage); + folio =page_folio (page); + kfolio =stable_tree_search (folio); +if (kfolio == folio && rmap_item->head == stable_node) { +folio_put (kfolio);return ; }remove_rmap_item_from_tree (rmap_item); -if (kpage) { -if (PTR_ERR (kpage) == -EBUSY ) +if (kfolio) { +if (kfolio ==ERR_PTR (-EBUSY ))return ; - err =try_to_merge_with_ksm_page (rmap_item, page, kpage); + err =try_to_merge_with_ksm_page (rmap_item, page,folio_page (kfolio,0 ));if (!err) {/* * The page was successfully merged: * add its rmap_item to the stable tree. */ -lock_page (kpage); -stable_tree_append (rmap_item,page_stable_node (kpage), +folio_lock (kfolio); +stable_tree_append (rmap_item,folio_stable_node (kfolio), max_page_sharing_bypass); -unlock_page (kpage); +folio_unlock (kfolio); } -put_page (kpage); +folio_put (kfolio);return ; }
Signature · 1 line
2.43.0
PATCH 6/10 mm/ksm: remove page_stable_node
In reply to [email protected]From: "Alex Shi (tencent)" <[email protected]>
Since folio_stable_node could replace page_stable_node now, the latter function could be removed and save a bit context.
--- mm/ksm.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-)
mm/ksm.c +2 -7
@@ -1086,11 +1086,6 @@ static inline struct ksm_stable_node *folio_stable_node(struct folio *folio)return folio_test_ksm (folio) ?folio_raw_mapping (folio) :NULL ; } -static inline struct ksm_stable_node *page_stable_node (struct page *page) -{ -return folio_stable_node (page_folio (page)); -} -static inline void folio_set_stable_node (struct folio *folio,struct ksm_stable_node *stable_node) {
@@ -2324,7 +2319,8 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_itebool max_page_sharing_bypass = false;struct folio *folio, *kfolio; - stable_node =page_stable_node (page); + folio =page_folio (page); + stable_node =folio_stable_node (folio); if (stable_node) { if (stable_node->head != &migrate_nodes && get_kpfn_nid(READ_ONCE(stable_node->kpfn)) !=
@@ -2345,7 +2341,6 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite }/* We first start with searching the page inside the stable tree */ - folio =page_folio (page); kfolio =stable_tree_search (folio);if (kfolio == folio && rmap_item->head == stable_node) {folio_put (kfolio);
Signature · 1 line
2.43.0
PATCH 7/10 mm/ksm: use folio in unstable_tree_search_insert
In reply to [email protected]From: "Alex Shi (tencent)" <[email protected]>
The calling path are using folio actually, so start to folio in unstable_tree_search_insert to save few compound checks.
--- mm/ksm.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
mm/ksm.c +7 -7
@@ -2194,15 +2194,14 @@ static struct ksm_stable_node *stable_tree_insert(struct folio *kfolio)*/ static struct ksm_rmap_item *unstable_tree_search_insert (struct ksm_rmap_item *rmap_item, -struct page *page, -struct page **tree_pagep) +struct folio *folio,struct folio **tree_foliop) {struct rb_node **new;struct rb_root *root;struct rb_node *parent =NULL ;int nid; - nid =get_kpfn_nid (page_to_pfn (page)); + nid =get_kpfn_nid (folio_pfn (folio)); root = root_unstable_tree + nid; new = &root->rb_node;
@@ -2220,12 +2219,12 @@ struct ksm_rmap_item *unstable_tree_search_insert(struct ksm_rmap_item *rmap_ite/* * Don't substitute a ksm page for a forked page. */ -if (page == tree_page) { +if (folio ==page_folio (tree_page)) {put_page (tree_page);return NULL ; } - ret =memcmp_pages (page, tree_page); + ret =memcmp_pages (folio_page (folio,0 ), tree_page); parent = *new;if (ret <0 ) {
@@ -2244,7 +2243,7 @@ struct ksm_rmap_item *unstable_tree_search_insert(struct ksm_rmap_item *rmap_iteput_page (tree_page);return NULL ; }else { - *tree_pagep = tree_page; + *tree_foliop =page_folio (tree_page);return tree_rmap_item; } }
@@ -2312,6 +2311,7 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_itestruct mm_struct *mm = rmap_item->mm;struct ksm_rmap_item *tree_rmap_item;struct page *tree_page =NULL ; +struct folio *tree_folio =NULL ;struct ksm_stable_node *stable_node;struct page *kpage;unsigned int checksum;
@@ -2411,7 +2411,7 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_itereturn ; } tree_rmap_item = -unstable_tree_search_insert (rmap_item, page, &tree_page); +unstable_tree_search_insert (rmap_item, folio, &tree_folio);if (tree_rmap_item) {bool split;
Signature · 1 line
2.43.0
PATCH 8/10 mm/ksm: use folio in try_to_merge_xx serie funcs
In reply to [email protected]From: "Alex Shi (tencent)" <[email protected]>
The try_to_merge_ serie funcs are using folios, so change the parameter and variable 'page' to folio to save compound checks.
--- mm/ksm.c | 54 ++++++++++++++++++++++++------------------------------ 1 file changed, 24 insertions(+), 30 deletions(-)
mm/ksm.c +24 -30
@@ -1540,22 +1540,18 @@ static int try_to_merge_one_page(struct vm_area_struct *vma, struct folio *folio* This function returns 0 if the pages were merged, -EFAULT otherwise. */ static int try_to_merge_with_ksm_page (struct ksm_rmap_item *rmap_item, -struct page *page,struct page *kpage) +struct folio *folio,struct folio *kfolio) {struct mm_struct *mm = rmap_item->mm;struct vm_area_struct *vma;int err = -EFAULT ; -struct folio *kfolio;mmap_read_lock (mm); vma =find_mergeable_vma (mm, rmap_item->address);if (!vma) goto out; -if (kpage) - kfolio =page_folio (kpage); - - err =try_to_merge_one_page (vma,page_folio (page), rmap_item, kfolio); + err =try_to_merge_one_page (vma, folio, rmap_item, kfolio);if (err) goto out;
@@ -1567,8 +1563,8 @@ static int try_to_merge_with_ksm_page(struct ksm_rmap_item *rmap_item,get_anon_vma (vma->anon_vma); out:mmap_read_unlock (mm); -trace_ksm_merge_with_ksm_page (kpage,page_to_pfn (kpage ? kpage : page), - rmap_item, mm, err); +trace_ksm_merge_with_ksm_page (kfolio,folio_pfn (kfolio ? kfolio : folio), + rmap_item, mm, err);return err; }
@@ -1582,17 +1578,17 @@ static int try_to_merge_with_ksm_page(struct ksm_rmap_item *rmap_item,* Note that this function upgrades page to ksm page: if one of the pages * is already a ksm page, try_to_merge_with_ksm_page should be used. */ -static struct page *try_to_merge_two_pages (struct ksm_rmap_item *rmap_item, -struct page *page, +static struct folio *try_to_merge_two_pages (struct ksm_rmap_item *rmap_item, +struct folio *folio,struct ksm_rmap_item *tree_rmap_item, -struct page *tree_page) +struct folio *tree_folio) {int err; - err =try_to_merge_with_ksm_page (rmap_item, page,NULL ); + err =try_to_merge_with_ksm_page (rmap_item, folio,NULL );if (!err) { err =try_to_merge_with_ksm_page (tree_rmap_item, - tree_page, page); + tree_folio, folio);/* * If that fails, we have a ksm page with only one pte * pointing to it: so break it.
@@ -1600,7 +1596,7 @@ static struct page *try_to_merge_two_pages(struct ksm_rmap_item *rmap_item,if (err)break_cow (rmap_item); } -return err ?NULL : page; +return err ?NULL : folio; } static __always_inline
@@ -2310,14 +2306,13 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite {struct mm_struct *mm = rmap_item->mm;struct ksm_rmap_item *tree_rmap_item; -struct page *tree_page =NULL ;struct folio *tree_folio =NULL ;struct ksm_stable_node *stable_node; -struct page *kpage; +struct folio *kfolio;unsigned int checksum;int err;bool max_page_sharing_bypass = false; -struct folio *folio, *kfolio; +struct folio *folio; folio =page_folio (page); stable_node =folio_stable_node (folio);
@@ -2353,7 +2348,7 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_iteif (kfolio ==ERR_PTR (-EBUSY ))return ; - err =try_to_merge_with_ksm_page (rmap_item, page,folio_page (kfolio,0 )); + err =try_to_merge_with_ksm_page (rmap_item, folio, kfolio);if (!err) {/* * The page was successfully merged:
@@ -2415,8 +2410,8 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_iteif (tree_rmap_item) {bool split; - kpage =try_to_merge_two_pages (rmap_item, page, - tree_rmap_item, tree_page); + kfolio =try_to_merge_two_pages (rmap_item, folio, + tree_rmap_item, tree_folio);/* * If both pages we tried to merge belong to the same compound * page, then we actually ended up increasing the reference
@@ -2427,23 +2422,22 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite* afterwards, the reference count will be correct and * split_huge_page should succeed. */ - split =PageTransCompound (page) - &&compound_head (page) ==compound_head (tree_page); -put_page (tree_page); -if (kpage) { + split =folio_test_large (folio) && folio == kfolio; +folio_put (tree_folio); +if (kfolio) {/* * The pages were successfully merged: insert new * node in the stable tree and add both rmap_items. */ -lock_page (kpage); - stable_node =stable_tree_insert (page_folio (kpage)); +folio_lock (kfolio); + stable_node =stable_tree_insert (kfolio);if (stable_node) {stable_tree_append (tree_rmap_item, stable_node, false);stable_tree_append (rmap_item, stable_node, false); } -unlock_page (kpage); +folio_unlock (kfolio);/* * If we fail to insert the page into the stable tree,
@@ -2465,10 +2459,10 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite* the page is locked, it is better to skip it and * perhaps try again later. */ -if (!trylock_page (page)) +if (!folio_trylock (folio))return ; -split_huge_page (page); -unlock_page (page); +split_folio (folio); +folio_unlock (folio); } } }
Signature · 1 line
2.43.0
PATCH 9/10 mm/ksm: calc_checksum for folio
In reply to [email protected]From: "Alex Shi (tencent)" <[email protected]>
Let's check the whole folio for contents change, don't count it if the folio changed.
--- mm/ksm.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-)
mm/ksm.c +8 -6
@@ -1258,11 +1258,13 @@ static int unmerge_and_remove_all_rmap_items(void) }#endif /* CONFIG_SYSFS */ -static u32 calc_checksum (struct page *page) +static u32 calc_checksum (struct folio *folio) {u32 checksum; -void *addr =kmap_local_page (page); - checksum =xxhash (addr,PAGE_SIZE ,0 ); +int nr =folio_nr_pages (folio); +void *addr =kmap_local_page (folio_page (folio,0 )); + + checksum =xxhash (addr, nr *PAGE_SIZE ,0 );kunmap_local (addr);return checksum; }
@@ -2369,7 +2371,7 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite* don't want to insert it in the unstable tree, and we don't want * to waste our time searching for something identical to it there. */ - checksum =calc_checksum (page); + checksum =calc_checksum (folio);if (rmap_item->oldchecksum != checksum) { rmap_item->oldchecksum = checksum;return ;
@@ -2385,7 +2387,7 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_itemmap_read_lock (mm); vma =find_mergeable_vma (mm, rmap_item->address);if (vma) { - err =try_to_merge_one_page (vma,page_folio (page), rmap_item, + err =try_to_merge_one_page (vma, folio, rmap_item,page_folio (ZERO_PAGE (rmap_item->address))); trace_ksm_merge_one_page( page_to_pfn(ZERO_PAGE(rmap_item->address)),
@@ -3916,7 +3918,7 @@ static int __init ksm_init(void)int err;/* The correct value depends on page size and endianness */ - zero_checksum =calc_checksum (ZERO_PAGE (0 )); + zero_checksum =calc_checksum (page_folio (ZERO_PAGE (0 )));/* Default to false for backwards compatibility */ ksm_use_zero_pages = false;
Signature · 1 line
2.43.0
Re: [PATCH 09/10] mm/ksm: calc_checksum for folio
Quoted 32 lines · On 04.06.24 06:24, [email protected] wrote:
From: "Alex Shi (tencent)" <[email protected]>
Let's check the whole folio for contents change, don't count it if the folio changed.
Signed-off-by Alex Shi (tencent) <[email protected]>--- mm/ksm.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-)mm/ksm.c +5
-3@@ -1258,11 +1258,13 @@ static int unmerge_and_remove_all_rmap_items(void) }#endif /* CONFIG_SYSFS */ -static u32 calc_checksum (struct page *page) +static u32 calc_checksum (struct folio *folio) {u32 checksum; -void *addr =kmap_local_page (page); - checksum =xxhash (addr,PAGE_SIZE ,0 ); +int nr =folio_nr_pages (folio); +void *addr =kmap_local_page (folio_page (folio,0 )); + + checksum =xxhash (addr, nr *PAGE_SIZE ,0 );kunmap_local (addr);return checksum; }
mm/ksm.c static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite · line 2374
Quoted 5 lines
mm/ksm.c +1
-1@@ -2369,7 +2371,7 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite* don't want to insert it in the unstable tree, and we don't want * to waste our time searching for something identical to it there. */ - checksum =calc_checksum (page); + checksum =calc_checksum (folio);
So for a large folio you suddenly checksum more than a single page? That's wrong.
Or am I missing something?
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH 09/10] mm/ksm: calc_checksum for folio
Quoted 10 lines · David Hildenbrand
mm/ksm.c static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite · line 2374
Quoted 6 lines
mm/ksm.c +1
-1@@ -2369,7 +2371,7 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite* don't want to insert it in the unstable tree, and we don't want * to waste our time searching for something identical to it there. */ - checksum = calc_checksum(page); + checksum = calc_checksum(folio);So for a large folio you suddenly checksum more than a single page? That's wrong.
Or am I missing something?
I am not sure if this change are good too, anyway, comparing the whole folio may have it advantages on efficiency, but more splitting do save more pages.
Anyway, this change could be dropped.
Thanks!
Re: [PATCH 09/10] mm/ksm: calc_checksum for folio
Quoted 12 lines · Alex Shi
Quoted 9 lines · David Hildenbrand
mm/ksm.c static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite · line 2374
Quoted 6 lines
mm/ksm.c +1
-1@@ -2369,7 +2371,7 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite* don't want to insert it in the unstable tree, and we don't want * to waste our time searching for something identical to it there. */ - checksum = calc_checksum(page); + checksum = calc_checksum(folio);So for a large folio you suddenly checksum more than a single page? That's wrong.
Quoted 3 lines
Quoted 1 line
Or am I missing something?
I am not sure if this change are good too, anyway, comparing the whole folio may have it advantages on efficiency, but more splitting do save more pages.
Calculating the checksum of something that could be a large folio when you might want to deduplicate subpages of the folio (and decide if you might want to split it) sound very wrong.
Please pay more attention to the details how the current code works and how it all works with tail pages.
Signature · 3 lines
Cheers, David / dhildenb
PATCH 10/10 m/ksm: use folio in ksm scan path
In reply to [email protected]From: "Alex Shi (tencent)" <[email protected]>
Now we could safely use folio in ksm_do_scan path to save a few compound checks. The 'page' still used since follow_page function still no folio version.
--- mm/ksm.c | 47 +++++++++++++++++++++++++---------------------- 1 file changed, 25 insertions(+), 22 deletions(-)
mm/ksm.c +25 -22
@@ -2304,7 +2304,7 @@ static void stable_tree_append(struct ksm_rmap_item *rmap_item,* @page: the page that we are searching identical page to. * @rmap_item: the reverse mapping into the virtual address of this page */ -static void cmp_and_merge_page (struct page *page,struct ksm_rmap_item *rmap_item) +static void cmp_and_merge_page (struct folio *folio,struct ksm_rmap_item *rmap_item) {struct mm_struct *mm = rmap_item->mm;struct ksm_rmap_item *tree_rmap_item;
@@ -2314,9 +2314,7 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_iteunsigned int checksum;int err;bool max_page_sharing_bypass = false; -struct folio *folio; - folio =page_folio (page); stable_node =folio_stable_node (folio);if (stable_node) { if (stable_node->head != &migrate_nodes &&
@@ -2518,12 +2516,12 @@ static unsigned int skip_age(rmap_age_t age) }/* -* Determines if a page should be skipped for the current scan. +* Determines if a folio should be skipped for the current scan. * -* @page: page to check -* @rmap_item: associated rmap_item of page +* @folio: folio to check +* @rmap_item: associated rmap_item of folio */ -static bool should_skip_rmap_item (struct page *page, +static bool should_skip_rmap_item (struct folio *folio,struct ksm_rmap_item *rmap_item) {rmap_age_t age;
@@ -2536,7 +2534,7 @@ static bool should_skip_rmap_item(struct page *page,* will essentially ignore them, but we still have to process them * properly. */ -if (PageKsm (page)) +if (folio_test_ksm (folio))return false; age = rmap_item->age;
@@ -2566,7 +2564,7 @@ static bool should_skip_rmap_item(struct page *page,return true; } -static struct ksm_rmap_item *scan_get_next_rmap_item (struct page **page) +static struct ksm_rmap_item *scan_get_next_rmap_item (struct folio **folio) {struct mm_struct *mm;struct ksm_mm_slot *mm_slot;
@@ -2655,36 +2653,41 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) ksm_scan.address = vma->vm_end;while (ksm_scan.address < vma->vm_end) { +struct page *page; +if (ksm_test_exit (mm))break ; - *page =follow_page (vma, ksm_scan.address,FOLL_GET ); -if (IS_ERR_OR_NULL (*page)) { + page =follow_page (vma, ksm_scan.address,FOLL_GET ); +if (IS_ERR_OR_NULL (page)) { ksm_scan.address +=PAGE_SIZE ;cond_resched ();continue ; } -VM_WARN_ON (PageTail (*page)); - nr =compound_nr (*page); -if (is_zone_device_page (*page)) + *folio =page_folio (page); +VM_WARN_ON (PageTail (page)); + nr =folio_nr_pages (*folio); + +if (folio_is_zone_device (*folio)) goto next_page; -if (PageAnon (*page)) { + +if (folio_test_anon (*folio)) { rmap_item =get_next_rmap_item (mm_slot, ksm_scan.rmap_list, ksm_scan.address);if (rmap_item) { ksm_scan.rmap_list = &rmap_item->rmap_list; -if (should_skip_rmap_item (*page, rmap_item)) +if (should_skip_rmap_item (*folio, rmap_item)) goto next_page; ksm_scan.address += nr *PAGE_SIZE ; }else -put_page (*page); +folio_put (*folio);mmap_read_unlock (mm);return rmap_item; } next_page: -put_page (*page); +folio_put (*folio); ksm_scan.address += nr *PAGE_SIZE ;cond_resched (); }
@@ -2755,16 +2758,16 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)static void ksm_do_scan (unsigned int scan_npages) {struct ksm_rmap_item *rmap_item; -struct page *page; +struct folio *folio;unsigned int npages = scan_npages;while (npages-- &&likely (!freezing (current))) {cond_resched (); - rmap_item =scan_get_next_rmap_item (&page); + rmap_item =scan_get_next_rmap_item (&folio);if (!rmap_item)return ; -cmp_and_merge_page (page, rmap_item); -put_page (page); +cmp_and_merge_page (folio, rmap_item); +folio_put (folio); } ksm_pages_scanned += scan_npages - npages;
Signature · 1 line
2.43.0
Quoted 9 lines · On 04.06.24 06:24, [email protected] wrote:
From: "Alex Shi (tencent)" <[email protected]>
This is the 2nd part to use folio in ksm, it saves 82kbytes in ksm.o and passed all ksm testcases in kselftest.
KSM still split same folios to single page if they are good to be KSM, but will skip the rest subpage checking if a folio couldn't suit for ksm.
Quoted 11 lines
Alex Shi (tencent) (10):
mm/ksm: reduce the flush action for ksm merging page mm/ksm: skip subpages of compound pages mm/ksm: use folio intry_to_merge_one_pagemm/ksm: addidentical_foliofunc
mm/ksm: use folio instable_tree_searchmm/ksm: removepage_stable_node
mm/ksm: use folio inunstable_tree_search_insertmm/ksm: use folio intry_to_merge_xxserie funcs mm/ksm:calc_checksumfor folio
m/ksm: use folio in ksm scan path
I skimmed over most patches, but I'm afraid you introduce quite some subtle problems.
Part of the rework should not be do convert things to folios that really operate on pages (of possibly large folios).
We really have to be careful about pages vs. folios before we have KSM folios. Only then, we know that we have small folios only.
Signature · 3 lines
Cheers, David / dhildenb
Quoted 22 lines · David Hildenbrand
Quoted 9 lines · On 04.06.24 06:24, [email protected] wrote:
From: "Alex Shi (tencent)" <[email protected]>
This is the 2nd part to use folio in ksm, it saves 82kbytes in ksm.o and passed all ksm testcases in kselftest.
KSM still split same folios to single page if they are good to be KSM, but will skip the rest subpage checking if a folio couldn't suit for ksm.
Quoted 11 lines
Alex Shi (tencent) (10):
mm/ksm: reduce the flush action for ksm merging page mm/ksm: skip subpages of compound pages mm/ksm: use folio intry_to_merge_one_pagemm/ksm: addidentical_foliofunc
mm/ksm: use folio instable_tree_searchmm/ksm: removepage_stable_node
mm/ksm: use folio inunstable_tree_search_insertmm/ksm: use folio intry_to_merge_xxserie funcs mm/ksm:calc_checksumfor folio
m/ksm: use folio in ksm scan path
Quoted 6 lines
I skimmed over most patches, but I'm afraid you introduce quite some subtle problems.
Part of the rework should not be do convert things to folios that really operate on pages (of possibly large folios).
We really have to be careful about pages vs. folios before we have KSM folios. Only then, we know that we have small folios only.
Thanks a lot for your reviewing and comments. Let's if we'd do it better with your comments! :)
Thanks
Alex